Anchen Chai

Mohammad-Mahdi Bazm

Sorina Camarasu-Pop

Tristan Glatard

Hugues Benoit-Cattin

Frédéric Suter

Modeling Distributed Platforms from Application Traces for Realistic File Transfer Simulation

Simulation is a fast, controlled, and reproducible way to evaluate new algorithms for distributed computing platforms in a variety of conditions. However, the realism of simulations is rarely assessed, which critically questions the applicability of a whole range of ndings.

In this paper, we present our eorts to build platform models from application traces, to allow for the accurate simulation of le transfers across a distributed infrastructure. File transfers are key to performance, as the variability of le transfer times has important consequences on the dataow of the application. We present a methodology to build realistic platform models from application traces and provide a quantitative evaluation of the accuracy of the derived simulations. Results show that the proposed models are able to correctly capture real-life variability and signicantly outperform the state-of-the-art model.

Introduction

Numerical simulation has become a critical tool to study distributed systems. Not only does it allow researchers to quickly evaluate new algorithms in a variety of conditions with literally no cost, but it also enables reproducible experiments whereas real systems are hampered by various uncontrolled factors such as background network trac, concurrent CPU usage, and hardware faults. Simulation also provides access to variables that could hardly be monitored in real systems, for instance network congestion.

Simulating application executions requires several components to complement the core simulation kernel. Simulators usually build on models of (i) the hardware platform, (ii) the software services deployed on the platform, e.g., schedulers, and (iii) the application itself, i.e., the workload characteristics. Each aspect needs to be accurately validated since it may have a serious impact on the simulated performance.

In practice, however, the realism of simulations is rarely assessed, which critically questions the veracity of a whole range of ndings derived from simulations. Even more importantly, studies such as [START_REF] Velho | On the Validity of Flow-Level TCP Network Models for Grid and Cloud Simulations[END_REF] have shown that widely-used simulation toolkits patently lack realism, in particular regarding the estimation of le transfer times across the network.

Yet, le transfers are critical to the performance of distributed applications. The variability of le transfer times increases with the number of storage and computing sites, which has important consequences on the dataow of the application. A subtle time dierence in a le transfer may delay a task, change the schedule of subsequent tasks in the application workow, impact the data storage policy, and eventually greatly aect the execution time. Therefore, le transfers have a substantial impact on the realism of simulations.

This paper presents our eorts to build platform models that accurately simulate le transfers across a distributed infrastructure of computing and storage resources. It proposes a methodology to build platform models from application traces and it provides a quantitative evaluation of the accuracy of the derived simulations. The methodology highlights and provides a detailed analysis of the phenomena involved in le transfer simulation on large distributed systems.

We focus on the platform as dened in the SimGrid toolkit [START_REF] Casanova | Versatile, Scalable, and Accurate Simulation of Distributed Applications and Platforms[END_REF], which includes the compute nodes and the network links and routes. The network topology is an important part of the platform that describes how resources are clustered into sites (called Autonomous Systems or ASes [START_REF] Bobelin | Scalable Multi-Purpose Network Representation for Large Scale Distributed System Simulation[END_REF]) and how sites are interconnected, possibly in a hierarchical way. The platform model also describes the resource properties, including their processing speed, network bandwidth and latency, or sharing policy. Other simulation toolkits may have dierent representations for platforms, but they usually include (i) a set of computation and communication resources and (ii) the network interconnection. OptorSim [START_REF] Bell | OptorSim -A Grid Simulator for Studying Dynamic Data Replication Strategies[END_REF] and SimGrid use external text les for the platform description while GridSim [START_REF] Buyya | Gridsim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing[END_REF] denes the platform directly in the code of the simulator.

We adopt a trace-based approach where platform models are built strictly from application logs, i.e., without requiring specic monitoring probes. Compared to invasive approaches such as network tomography [START_REF] Vardi | Network Tomography: Estimating Source-Destination Trac Intensities from Link Data[END_REF], the trace-based approach might be less accurate but also easier to use on infrastructures where probing is not possible. Ultimately, we aim at replaying application traces as accurately as possible in a simulated environment. In this paper, we rely on a corpus of traces extracted from the Virtual Imaging Platform (VIP) [START_REF] Glatard | A Virtual Imaging Platform for Multi-Modality Medical Image Simulation[END_REF], a web portal for medical image analysis and simulation.

The paper is organized as follows. In Section 2, we present the real system that we aim at simulating, the traces that will be used to dene the simulated platform, and the simulation framework we developed based on the SimGrid toolkit. Section 3 presents the platform construction methodology, i.e., techniques to estimate network bandwidths from application traces. Starting from a rough platform model that represents the state of the art used in works such as [START_REF] Camarasu-Pop | Combining Analytical Modeling, Realistic Simulation and Real Experimentation for the Optimization of Monte-Carlo Applications on the European Grid Infrastructure[END_REF], we present six successive improvements that we motivate through illustrative cases extracted from the trace corpus. Section 4 presents the nal simulation results that are evaluated against the state-of-the-art platform model. All the code and data used in this article are available online, to allow readers and reviewers to reproduce and further investigate our results [START_REF] Chai | Companion of the article on Modeling Distributed Platforms from Application Traces for Realistic File Transfer Simulation[END_REF]. Section 5 presents related work on simulation, platform models, trace-based approaches, and simulation evaluation. Finally, Section 6 summarizes our ndings and details our future work.

Technical Context and Traces

2.1

Real System

Our target system is the distributed execution of an application submitted through the VIP portal. VIP gives access to several applications while completely hiding the underlying storage and computing infrastructure. It leverages the resources of the Biomed Virtual Organization (VO) within the European Grid Infrastructure (EGI). This VO is supported by about 65 sites world-wide, oering a total of 130 computing clusters and 5 PB of storage. The high variability of the static and dynamic characteristics of these heterogeneous resources scattered across multiple organizations makes it particularly challenging to build realistic models for such a system. In this work, we focus on the GATE application [START_REF] Sébastien | GATE: a Simulation Toolkit for PET and SPECT[END_REF], whose workow is described in Figure 1. In the computing phase, a set of independent workers: (1) download the input les; (2) execute a single GATE job each; and (3) upload partial results on a storage element (SE) upon completion. Once all the GATE jobs are completed, the workow enters a merging phase, where all the partial results are downloaded from dierent SEs (4) and aggregated [START_REF] Buyya | Gridsim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing[END_REF]. The nal result is then copied back to a SE [START_REF] Calheiros | CloudSim: A Toolkit for Modeling and Simulation of Cloud Computing Environments and Evaluation of Resource Provisioning Algorithms[END_REF]. The data exchanges between the tasks of the workow are le-based and require transfers over the network and through SEs, since the worker nodes do not share a common le system.

Execution Traces

We collected execution traces of 60 workow executions submitted by 14 distinct VIP users over a period of one month. These workows are all dierent in terms of software release used, input les, number of jobs, and execution times. However, they do not all match the current capabilities of our simulator [START_REF] Suter | VIPSimulator: a Simulator of Gate Workow Execution[END_REF], which assumes that all the partial results les produced by the GATE jobs are downloaded by the Merge Figure 1: GATE workow including a computing phase, a merging phase, and associated le transfers.

jobs, and only them. In some workows, this assumption is not veried due to canceled or failed jobs. We thus discarded 31 out of the initial 60 workows where a majority of the logs les were empty (1/60), there were less (22/60) or more (1/60) downloads than uploads, or some downloaded les are not those uploaded by completed jobs (7/60). We also discarded 5 workows that have been severely impacted by the scheduled maintenance shutdown of a specic storage element. It left us with 24 workows submitted by 6 distinct users from Sep. 8, 2015 to Oct. 1, 2015. In these 24 workows, 1,796 jobs were executed by worker nodes across 32 dierent computing sites, which represents almost half of all the sites in the Biomed VO. These jobs performed 8,932 le transfers using 32 dierent storage elements.

Each job creates its own structured log le. It starts with a header that comprises the name of the workow the job belongs to and the job id. Then, information on the compute node that runs the job is logged, i.e., its name, number of cores, processing speed, and the bandwidth of the network interface.

The main steps of a job execution are listed as well. In the following, we present the steps for GATE job. First, the job tests whether it can reach its default SE by doing an upload test, i.e., a copy-andregister operation of a 12-byte le. Second, the job downloads the needed input les: (i) a wrapper script automatically created (73 kB); (ii) a tarball containing the GATE executable along with all the required libraries needed for execution (from 121.6 MB to 501.8 MB); (iii) a tarball containing the user-dened inputs, e.g., macro les describing the simulation, images, or ROOT les (from 4 kB to 130 kB). Third, the Monte-Carlo simulation is executed. Fourth, the job uploads its result le to its default SE. For all the occurring transfers, the source, destination, size, and duration are logged.

The second source of information is a database, which contains a timestamp for each milestone on the job execution timeline (i.e., creation, entering the queue, starting the download, computation, and upload phases, and termination).

We developed the log2sim framework [START_REF] Suter | The Log2sim Framework[END_REF] to parse these log les and the database and then produce three CSV les. One describes the worker nodes, the second the le transfers, and the third lists the milestones of the execution. We then use these CSV les to to generate and instantiate a model of the platform which will be an input of our simulator.

2.3

The VIP Simulator and the SimGrid Toolkit

We developed a simulator that implements several VIP services to simulate the execution of a GATE workow. It assumes that the tasks composing the workow have already been created and submitted and that the compute resources have been acquired. It also implements several services from EGI related to data transfers. They are designed to be as close as possible to the actual behavior of the production system. A central Logical File Catalog (LFC) stores information about le replicas. To download a le a worker node rst has to contact the LFC to know from which SE the requested le can be retrieved.

To ensure that simulated le transfers occur between the same hosts as in the real execution, we enforce the use of the le replica involved in the corresponding real transfers. Then the worker node contacts the targeted SE to download the le. Conversely, a worker uploads a le to a SE and then noties the LFC to register its location. For all le transfers, and based on an analysis of actual transfer durations, our simulator models these interactions between the dierent services by a 900ms delay that encompasses to the cost of processing transfer requests and storage access latency in addition to the network latency. This simulator is based on the SimGrid toolkit [START_REF] Casanova | Versatile, Scalable, and Accurate Simulation of Distributed Applications and Platforms[END_REF], that provides all the core functionalities for the simulation of distributed applications in heterogeneous distributed environments. SimGrid relies fast and tractable uid models, whose validity has been thoroughly assessed [START_REF] Velho | On the Validity of Flow-Level TCP Network Models for Grid and Cloud Simulations[END_REF], to simulate network trac and ensures the realism of the simulated communication times. SimGrid also provides a scalable engine that allows us to simulate large workows with many concurrent transfers on large infrastructures in a reasonable time.

SimGrid usually requires a model of the platform as input, in our case the EGI infrastructure accessed through VIP. This model describes the characteristics of the compute (i.e., hosts with a certain number of cores and processing speed) and network (i.e., links of a given bandwidth, latency, and contention model) resources. It also describes how hosts are grouped (e.g., in clusters, data centers, . . . ) and interconnected through a network topology. SimGrid formalism adopts a hierarchical organization based on Autonomous Systems (ASes) similar to that of the Internet [START_REF] Bobelin | Scalable Multi-Purpose Network Representation for Large Scale Distributed System Simulation[END_REF]. The correct instantiation of this platform model is key to simulation realism. To the best of our knowledge, there exists no full and faithful description of EGI that has been built for simulation purposes.

Modeling Platforms from Execution Traces

In this section, we detail incremental improvements to the platform model to increase the accuracy of le transfer simulation thanks to a thorough analysis of both trace contents and simulation results. For each improvement, we motivate and illustrate the impact of the proposed solution on a particular case extracted from the traces.

Baseline Model

The quality of le transfer simulation is mainly impacted by two main features of a platform model: the interconnection topology and the instantiation of network link bandwidth. To dene a baseline for our study, we consider a rst platform model that only minimally relies on the execution traces for these two important features. Nevertheless, the characteristics of the computing nodes (e.g., name or processing speed) are extracted from the execution traces, as well as their hierarchical organization in clusters and sites.

Without information on the interconnection topology coming from the traces, we have to assume a simple and uniform connectivity among the compute nodes and the storage that compose the distributed platform. A way to model such a platform is to connect each SE to all the computing entities through a single backbone (Figure 2 To instantiate the dierent links, we use the nominal bandwidth value of the network card for the compute nodes, which is typically of 1 Gb/s, and a latency of 500 microseconds. This information is extracted from the traces, but does not depend on a specic run. For the storage elements, we consider an interconnection bandwidth of 10 Gb/s and a network latency of 750 microseconds. The rationale is q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Wrapper Release 1 that SEs usually are large disk bays, hence with a better connectivity than simple compute nodes to ensure a good throughput. Such a model can be considered as state-of-the-art as it has been used for the simulation of similar workows running on the same distributed platform [START_REF] Camarasu-Pop | Simulating Application Workows and Services Deployed on the European Grid Infrastructure[END_REF][START_REF] Camarasu-Pop | Combining Analytical Modeling, Realistic Simulation and Real Experimentation for the Optimization of Monte-Carlo Applications on the European Grid Infrastructure[END_REF], but in a context where the focus was not on le transfers.

(A)). SITE2 SE1 SITE3 SITE1 SE2 (A) (B) (C) SE1 SITE1 SITE2 SITE3 SE2 SITE1 Cluster1 Cluster2 SE2 SE1
Figure 3 presents the le transfer durations (in seconds) of two of the input les downloaded by each of the hundred jobs composing one specic workow instance, as logged in the traces and simulated with the aforementioned model. The respective sizes of these two les, which are downloaded in the majority of the studied workows are 73 kB (wrapper le) and 121.57 MB (GATE release le). These transfers are representative of all the other le transfers for this workow, but also of all the other studied workows.

This gure illustrates two major drawbacks of relying on the "theoretical" bandwidth of 10 Gb/s to instantiate the platform model. First, the duration of the le transfers are globally and severely underestimated. For the release le, the simulated durations are in the [1.186s; 2.437s] range, while the measured transfer times are in the [2s; 223.5s] range. Similar dierences occur for the wrapper le. Second, and more importantly, this model fails to reproduce the variability because it does not properly capture the competition for resources between certain transfers due to inaccurate transfer time estimations.

Leveraging Trace Contents to Instantiate the Platform

To address the two issues raised by a uniform bandwidth instantiation of the link that connects a SE to the rest of the platform to a nominal value of 10 Gb/s, we exploit the contents of the execution traces.

Our aim is at using values that are more representative of the actual execution conditions. For each link, we consider two common-sense aggregation methods, which have their respective pros and cons.

First, we compute the average value over all the observed transfers to/from a given storage element.

Using average values is likely to be more realistic when it comes to reproduce the behavior of a single workow execution. It reects the network connectivity as experienced by the application. The sharing of network resources by concurrent transfers is thus directly captured in the model and not handled by the simulation kernel. The drawback of this approach is that the resulting instantiation of the model is limited to the workow used to produce it and it cannot be used to simulate the execution of another workow even though the same resources are involved. However, the accurate simulated replay of a given trace already allows for the investigation of several interesting what-if scenarios. Second, we determine the observed maximum value over all these transfers. This allows us to get an approximation of the nominal capacity of the network link. In that case, we let the simulation kernel determine how the network resources are shared among concurrent transfers. The main advantage of such an instantiation of the platform model is that it can be reused beyond the simulated replay of the execution that led to its generation. It can also be aggregated, both spatially and temporally, with information obtained from other traces. Figure 4 presents the same type of results as Figure 3, but here the simulation results obtained with a uniform 10 Gb/s instantiation have been replaced by those obtained with an average or maximum bandwidth value for each storage element. We use the same interconnection topology. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Wrapper Release Using trace contents to instantiate the bandwidth values of the network links partially addresses the capture of the variability of le transfer duration, especially for the large release le. However, both the aggregation methods fail to be accurate. Averaging the observations tends to overestimate the transfer times (by a factor of 2 in average with a median of 1.5, and a maximum of 14.5), while using the maximum leads to an underestimation of at least a factor of 2 for half of the transfers. This denotes biases in our model that we propose to investigate and solve in the subsequent sections.

Distinguishing Transfer Type and File Size

We rst looked at the distribution of the individual bandwidths derived from the le transfers that involve a given storage element in a given workow trace. We notice great dierence in some cases, sometimes of more than two orders of magnitude. We identied three root causes to this variability. First, some of the initial upload tests can suer a delay. This delay, of usually one second, is negligible with regard to the total transfer time, but as these transfers involve a 12-byte le, this leads to unrealistically low bandwidth values, i.e., of less than a kilobit per second. Second, the limited precision of the timings logged in the traces leads to almost instantaneous transfer times for les of a few kilobytes. Then the derived bandwidth is unrealistically high, i.e., greater than 10 Gb/s. Third, we observed a sometimes large dierence between the upload and download of the partial result les produced by the jobs. Each job uploads such a le to its local storage element which is then downloaded by the merge job from the same storage element. However, the concurrency conditions in which these transfers occur are dierent and impact the transfer times. The merge job downloads all the partial results in sequence while several worker nodes can upload their les to the same storage element simultaneously. Consequently, depending on the direction of the transfer, i.e., to or from a given SE, the derived bandwidth may greatly dier. All these observations are likely to negatively impact the computation of the average or maximum bandwidth of the link connecting a storage element to the rest of the platform.

To address these issues, we propose to group the transfers by type (i.e., upload test, each of the three input les, and both upload and download of partial result les) and direction (i.e., to or from a storage element) before computing the average and maximum bandwidth values. Bandwidth estimations are then performed based on specic types for which the measured le transfer durations are assumed more reliable. For transfers from a storage element, we favor that of the release le, whose larger size prevents to be hit by the timing precision. Similarly, for transfers to a storage element, upload of partial results is preferred over the more sensitive initial upload test.

Figure 5 shows the impact of the average bandwidth computation method on the simulation of the transfer of the release le by jobs in the fNUfzs workow instance. The transfer of this particular le is the one for which this improvement is the most noticeable, hence our focus on these results. The Average (all) data are the same as in Figure 4, while the Average (grouped) data refer to the distinction of transfer type and le size. In that case, the average bandwidth from a given storage element to any compute element is computed based only on the transfers of the release le found in the traces. We also distinguish the storage element from which the le has been downloaded in this gure. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q ccsrm02.in2p3.fr marsedpm.

Measured

Average (All) Average (Grouped) The proposed improvement reduces the overestimation of the transfer duration. Simulated transfers, whose durations were twice as long in average (and up to 14.5 times longer) than the actual transfers and are now 1.2 times shorter in average (with a maximum overestimation by a factor 7.5). However, we can clearly see in Figure 5 that, for a given storage element, the model still fails to capture the variability in transfer durations. To nd the origin of this modeling bias and further improve our platform model, we zoom in on a specic storage element in the next section.

Distinguishing Computing Sites

When we isolate the transfers for a given storage element and input le size, we clearly and obviously observe (as in Figure 5) that using an average bandwidth leads to a uniform transfer time to all the computing sites. However, the bottom part of Figure 6 shows an important variability in the average bandwidth when we distinguish the downloads of the release le from the marsedpm.in2p3.fr storage element in the fNUfzs workow instance by the computing site downloading this le.

The global average bandwidth for the SE, as computed in the previous section and depicted by a dashed line, is of 33.52 Mb/s. This value is a good approximation for the INFN-BARI and UKI-LT2-RHUL sites but an overestimation by a factor of around 5 for the INFN-PISA and UKI-LT2-IC-HEP sites and a clear underestimation for the UKI-LT2-QMUL site. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0 50 100 This deviation to the global average directly explains the results presented in the top part of Figure 6.

The "By SE " data corresponds to the middle panel of Figure 5 while the "By SE-Site" data relies on the distinct average value computed for each computing site. To improve the readability of the gure, we introduce a horizontal jitter to separate data points corresponding to similar durations. Now we make a similar analysis for the alternate approach based on the determination of the maximum observed bandwidth. As for the average, we see in Figure 7 (bottom) an important deviation from the global maximum value of 321 Mb/s for most of the computing sites. We also see in Figure 7 (top) that the use of a single value for all the sites also leads to similar simulated durations for all the transfers.

In this version of the model, it should be noted that distinguishing the links between a storage element and the computing sites may bias the way the simulation kernel handles the sharing of the network resources. Indeed, separate links mean independent trac and this can lead to a cumulative simulated bandwidth greater than what is observed from both the SE and computing site point of views.

To prevent such a bias, we introduce the notion of limiter links whose bandwidths are determined by taking the maximum value observed over all the transfers to/from a SE or a given site (Figure 2 (B)).

Unlike the improvement observed for the average-based model, we see here that determining a distinct maximum bandwidth for each site dramatically degrades the quality of the simulation. This suggests that our estimation of the maximum value is biased and returns underestimations of the nominal bandwidths of the links between the storage element and the dierent computing sites.

More precisely, the modication of the model improves the simulation accuracy when there are only a few transfers from the SE to a computing site (i.e., for the INFN-BARI and UKI-LT2-QMUL sites). q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0 1000 2000 Duration (in seconds)

q

Measured

By SE By SE-Site q q q q q 0 100 200 300

INFN-BARI INFN-PISA UKI-LT2-IC-HEP UKI-LT2-QMUL UKI-LT2-RHUL
Computing Sites Bandwidth (in Mb/s) When there are more transfers from the SE to a computing site, the durations become, sometimes largely, overestimated. We thus assume that our model fails to capture an important phenomenon, that we identify and take into account in the next section.

3.5

Correcting the Maximum Bandwidth We can see that almost all these transfers are simultaneous and of durations in the same range (from 154.4 to 193.4 seconds). As a consequence, the derived bandwidth of each individual transfer is impacted by the competition with the other transfers for a single shared network resource. In other words, the observed bandwidths correspond to the shares of the link capacity that are respectively allocated to each transfer, but they do not reect the total capacity of the link itself.

To get a better estimate of the nominal capacity of the link, we thus have to rst compute a correcting factor to the bandwidth derived from each transfer. The computation of this factor, during the generation of the platform model, somehow corresponds to the inverse of the computation made by the simulation kernel to assign a share of the network resource to a given transfer. We proceed as follows. First we sample the duration of the transfer in at most 50 uniform intervals. For instance, in Figure 8, we split the rst transfer (in blue) in 20 intervals. For each of them, we estimate the number of concurrent transfers, hence how the network resource is shared. The rst transfer will thus have only one ninth of the capacity of the link for the seven rst interval but one third of it during the last interval. Then we compute the correcting factor as:

f = n n i=1 1 ci , ( 1
)
where n is the number of intervals, and c i the number of concurrent transfers in the i th interval. In our example, we obtain a correcting factor of 20/(7/8+10/7+1/6+1/4+1/2) = 6.21 for the rst transfer.

For each transfer, we compute three dierent correcting factors by estimating the concurrency for each SE, each site and each (SE, Site) couple. This allows us to correct the instantiation of the link between a SE and a site and of the limiter links.

Figure 9 shows the impact of this correction of the maximum observed bandwidth on the results presented in Figure 7. We can see that the important overestimation of the transfer durations disappears and the simulation accuracy is globally improved. However, this model now leads to a moderate underestimation of the durations for two sites (INFN-PISA and UKI-LT2-RHUL), which is caused by dierent phenomena. q q q q q q q q q q q q q q q q q q qq q q q q q q q q q 0 1000 2000 For the INFN-PISA site, two transfers (second and fourth from top in Figure 8) start before a bulk of seven transfers. While these two transfers last for roughly the same time as the others in the real execution, they will experience less concurrency in simulation and will then be faster to complete (in around 20 seconds). More importantly they will end before the other transfers start, which in turn impacts the concurrency they experience and their simulated duration. We suspect the inuence of some external load during the rst two transfers that our model currently fails to capture.

INFN
For the UKI-LT2-RHUL site, we observed that the transfer durations dier depending on which cluster in the site is involved. We propose to illustrate and address this phenomenon in the next section using a more glaring example.

Distinguishing Clusters in Sites

In our analysis of the execution traces, we sometimes observed an important variability in transfer durations for a given (SE, Site) couple. This is particularly striking for downloads of the release le from the sbgse1.in2p3.fr SE to the INFN-PISA site in the LQz3XJ workow instance. In this particular case, the durations are in a [4.04s; 257.40s] interval. This variability is related to the cluster where the nodes downloading the le are located, as shown in Figure 10. Using a single bandwidth value for the site, be it the average or the maximum, thus fails to capture that cluster-related dierence in performance, and leads to inaccurate simulations. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Average Maximum 0 100 200 ne1wn-32cores s1wn-32cores so1wn-4cores ne1wn-32cores s1wn-32cores so1wn-4cores Clusters Duration (in seconds) q Measured By Site By Cluster By (Cluster-SE) To improve our models, we propose to dierentiate the links that connect the dierent clusters in a site to the rest of the platform (see Figure 2 (C)). Note that the clustering of worker nodes is based only on information available in the trace, i.e., name, number of cores, and processing speed, and do not leverage information by the sites themselves. As for the previous improvements, we modify the way we aggregate the individual bandwidth values to take this distinction into account. We also compute specic correction factors for the model based on the maximum bandwidth.

Unfortunately, this modication improves the accuracy of the simulations only partially. Figure 10 highlights two limitations with dierent causes but a common solution. For the average-based model, we observe an overestimation of the durations for the s1wn-32cores cluster. This is because the link between the SE and the site becomes a bottleneck. Indeed, the average bandwidth we compute for this link is hindered by the bad performance obtained for the so1wn-4cores cluster. Conversely, in the maximum-based model, we see an underestimation of the durations for the so1wn-4cores cluster. In this case it is the better bandwidth from another SE (not displayed in Figure 10) that denes the maximum observed bandwidth for this cluster. These two situations advocate for a common improvement that consists in dierentiating the links not between a site and a SE, but between each individual cluster and a SE.

The results obtained with this nal improvement are labeled as "By Cluster-SE " in Figure 10. We can see that, as expected, it completely solves the inaccuracy issue for the average-based model. For the maximum-based model, the proposed distinction improves the accuracy only partially. This can be explained, as in the previous section, by some abnormally long transfers in the real execution that have a shorter simulated duration which in turn modies the level of concurrency on the network resource. We can see three such transfers in Figure 10.

Experimental Evaluation

In this section we evaluate our improved platform models by confronting simulation results obtained using these models to other simulation results using the state-of-the-art model presented in Section 3.1 and to the reference given by the real transfer durations extracted from traces. Then we provide an analysis of the causes of the main simulation inacurracies.

Analysis of Simulated Transfer Durations

The evaluation uses transfer durations extracted from the execution traces described in Section 2.2, as well as simulated transfer durations obtained with the following platform models: 10G-SotA (as described in Section 3.1), Average and Maximum (as described in Section 3.6). For each model, we simulate 5,316 download transfers. Among these, 1,772 les (one third) correspond to the GATE release le and are larger than 121 MB. The other les correspond to the GATE inputs and wrappers and are smaller than 130 kB. Figure 11 shows a summary of the measured and simulated transfer durations for these two sets of les.

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Input and Wrapper Files (< 130 kB) Release Files (> 121 MB) For the GATE inputs and wrappers les, 80% of these 3,544 small le transfers last for less than 1.3 seconds in the real execution. The minimal incompressible delay imposed by control and network latency being of one second, there is no possible discrimination between the models and very few misestimations. We also observe 26 transfers (of the 73 kB wrapper le) whose durations are close to 64.5 seconds. The only common point to all these transfers is the storage element use to download this wrapper le. This highlights an issue with this specic SE rather than a modeling problem. However, it is interesting to note that both the Average and Maximum models are able to correctly reproduce one of this longer transfers. The proposed models are also able to partially capture the variability of 15% to 19% of the remaining transfers, while the 10G-SotA model cannot. In the remaining of this section, we focus our analysis on the transfers of the larger GATE release le. The longer durations allow us to get a better insight on the respective strengths and limitations of the dierent models. We complete Figure 11 with Table 1 that gives the corresponding statistics for the transfers of the release le. The poor quality of the 10G-SotA model is blatant: it largely underestimates all the transfer durations and cannot capture the variability that characterizes the real executions. The proposed Average and Maximum models, however, are able to reproduce that variability and correctly simulate the longest transfers. We also note a tendency of the Maximum platform model to underestimate the transfer durations, which can be explained by the fact that this models relies on estimations of the nominal bandwidths of the links that are less accurate than the average of the bandwidths as perceived by the application.

Analysis of Errors

Another way to assess the respective quality of the dierent platform models and to measure the benets of the proposed trace-based approach in terms of simulation accuracy is to compare errors with respect to the real transfer times. The relative dierence between simulation results and the reference of a real execution can be computed in many ways. For instance, the relative error is a standard error measure which gives a good indication on the precision of a simulation and whether it under-or overestimates the reality. However, this method has a certain number of disadvantages such as the fact that it is not symmetrical (the error lies in the interval ]-∞, 1]) or the fact that the relative dierence of bandwidths is dierent of the relative dierence of transfer times, while there is a priori no reason to favor one over the other. As explained in [START_REF] Velho | Accuracy Study and Improvement of Network Simulation in the SimGrid Framework[END_REF], an absolute logarithmic error solves these two issues and allows us to compare the dierent models more easily. We dene the absolute logarithmic error as follows:

LogErr = |log(R) -log(S)|, (2)
where R is the real time and S the simulated time.

The absolute logarithmic error also allow us to apply additive aggregation operators such as the maximum or the mean, hence easing the comparison of the dierent models.

Figure 12 shows the Cumulative Distribution Functions of the absolute logarithmic errors obtained for each model. The maximum errors respectively are 6.09 for the 10G-SotA model, 3.99 for the Maximum model, and 2.83 for the Average model. This gure conrms the poor accuracy of the 10G-SotA platform model that shows errors greater than two (i.e., relative error greater than 639%) for nearly half of the transfers. This corresponds to the systematic and large underestimations of the le transfer durations made by this model that was illustrated by Figure 11.

As expected, the Average platform model leads to best results as the sharing of network resources by concurrent transfers is directly captured in the model which reects the network connectivity as experienced by the application. This model is thus able to simulate 75% of the transfers with a logarithmic error smaller than 0.35 (relative error: 42%) and 92.9% of the transfers with an error smaller than 1.

Finally, the mean logarithmic error is 0.28 (relative error: 32%).

The Maximum platform model also clearly outperforms the 10G-SotA model but leads to greater errors than the Average model. The mean logarithmic error is 0.62 (relative error: 85%) and 75% of the transfers are simulated with a logarithmic error under 0.97 (relative error: 164%). This loss of accuracy is the cost of the higher potential for further studies of this model that oers a better reusability beyond the simulated replay of the execution that led to its generation. It also conrms the tendency of this model to be too optimistic in its determination of the bandwidth values, and thus to underestimates the transfer durations. A deeper analysis of the individual simulation results would be needed to determine whether the way to compute the bandwidth correction factor can be improved.

To summarize, our analysis of the distribution of the simulated transfer durations and of the logarithmic errors demonstrates that a correct modeling of the interconnection topology and a good instantiation of the characteristics of the network resources are key to the quality of simulations of le transfers on a large-scale distributed infrastructure. It also shows that leveraging the contents of execution traces is a sound approach that enables a pretty accurate simulated replay of a given execution with the Average model or even beyond with the Maximum model, with an aordable accuracy loss.

Analysis of the Root Causes of Main Simulation Errors

While the obtained results are promising and greatly outperform the state-of-the-art model, they also include some prohibitive errors. In this section, we analyze such errors to determine whether they correspond to a modeling aw or to phenomena that we cannot, or do not even want to, model. For instance, the cause behind the outlying durations for small le transfers does not have to be part of the platform model, but rather to be injected into the simulation as an additional parameter. Then, and before presenting two phenomena that we identied as sources of high inaccuracy, we recall that Table 1 shows that all the simulated durations lie in the same range of values as the actual durations. The rst identied source of high inaccuracy is illustrated by Figure 13 that shows a Gantt chart view of ve transfers of the release le from the same SE to worker nodes in the same cluster. We see that four les (1 to 4) are transferred much faster (from 14s to 45s) than a fth one (in 172s).

In the Maximum model, the bandwidth derives from transfer 4 and is, after correction, of 968 Mb/s. However, the actual bandwidth for transfer 5 is only of 23.5 Mb/s. This dierence leads to a dramatic underestimation of the simulated duration (only 5.15s) and one of the largest errors. With the Average model, this also impacts the simulation accuracy by lowering the computed average (at 163 Mb/s). Again the great deviation from the actual bandwidth leads to an important error.

From the point of view of our workow execution, transfer 5 is neither impacted by another transfer to other nodes, not even from another SE, nor by transfers done by other jobs executed on the same node. The problem thus does not come from a bad handling of network contention by our model. This particular transfer has to be impacted by some external transient load that we do not model.

The second main identied source of large errors pertains to transfers from the declared local SE. Such transfers can be very fast (only 2s to transfer 121 MB) but are also subject to an important variability that follows a certain pattern. We observed, for several workow instances, durations that increase exponentially. We suspect the inuence of some local conguration of the storage elements that limits the number of concurrent transfers and might trigger some timeout-retry mechanisms. Unfortunately, the traces do not make the distinction between the time spent waiting for the SE to be available from the time actually spent to transfer the le over the network. Our models are thus unable to capture this eect.

Related Work

The simulation of large-scale distributed computing infrastructures has received an enormous amount of attention in the literature. In the area of grid and cloud computing, many simulators have been developed and used by researchers over the last decade. Some were made available to the community but proved to be short-lived perhaps because they were too project-specic. For instance, ChicSim [START_REF] Ranganathan | Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applications[END_REF] and OptorSim [START_REF] Bell | OptorSim -A Grid Simulator for Studying Dynamic Data Replication Strategies[END_REF], designed to study data replication in grids, have been discontinued. SimGrid [START_REF] Casanova | Versatile, Scalable, and Accurate Simulation of Distributed Applications and Platforms[END_REF] and GridSim [START_REF] Buyya | Gridsim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing[END_REF] are two simulation toolkits widely used in grid computing research. More recently, simulators have been proposed for simulating cloud computing platforms and applications. GroudSim [START_REF] Ostermann | Dynamic Cloud Provisioning for Scientic Grid Workows[END_REF] is a framework that enables the simulation of both grid and cloud systems. CloudSim [START_REF] Calheiros | CloudSim: A Toolkit for Modeling and Simulation of Cloud Computing Environments and Evaluation of Resource Provisioning Algorithms[END_REF] builds on the same simulation internals as GridSim but exposes specic cloud interfaces. Platform models are mandatory to the simulation of distributed computing infrastructures. However, large production infrastructures such as EGI are dicult to model. Then, studies such as [START_REF] Camarasu-Pop | Simulating Application Workows and Services Deployed on the European Grid Infrastructure[END_REF][START_REF] Gupta | iFogSim: A Toolkit for Modeling and Simulation of Resource Management Techniques in Internet of Things, Edge and Fog Computing Environments[END_REF] come to either use existing descriptions of controlled environments, such as that of the Grid'5000 [START_REF] Bolze | Grid'5000: A Large Scale And Highly Recongurable Experimental Grid Testbed[END_REF] experimental testbed or more simplistic descriptions instantiated with homogeneous bandwidths (e.g., 10 Gb/s).

Trace analysis is a common approach to model large scale distributed platforms and predict the performance of individual operations, e.g., le transfers. In [START_REF] Barisits | A Hybrid Simulation Model for Data Grids[END_REF], authors propose a hybrid simulation model for data grids based on the statistical analysis of traces. In [START_REF] Ferreira Da Silva | Characterizing a High Throughput Computing Workload: The Compact Muon Solenoid (CMS) Experiment at LHC[END_REF], authors present a practical machine learning method to predict job characteristics by conducting an analysis of the workload traces.

Simulation results should ideally be close to results that would be obtained in a production system. The accuracy of a simulator can be evaluated by confronting simulation results to the ground truth of actual experiments. However, published evaluations are often weak and/or incomplete. Some evaluations are merely qualitative. When quantitative, evaluations often consist in exhibiting a few good cases in which simulation results lead to reasonable trends (few real executions are actually attempted). Recent work [START_REF] Camarasu-Pop | Combining Analytical Modeling, Realistic Simulation and Real Experimentation for the Optimization of Monte-Carlo Applications on the European Grid Infrastructure[END_REF] emphasizes the need for realistic simulators and the use of multiple approaches (mathematical models, discrete-event simulation, and real production experiments) for cross-validation, allowing for a good level of condence in the obtained results.

Conclusion and Future Work

Simulation is a powerful tool that allows for easier prototyping and testing of new methods, enabling at the same time their thorough evaluation through deterministic and reproducible experiments. However, simulators are rarely tuned to faithfully model large-scale distributed computing platforms and simulation results are seldom compared to the ground truth of actual production runs of real applications.

In this paper, we proposed a methodology for building realistic platform models that allow us to accurately simulate le transfers across a distributed infrastructure. The models were built and instantiated from execution traces obtained from a production environment. Last but not least, we evaluated the proposed models by confronting our simulation results to (i) the ground-truth of the real traces and (ii) simulation results using a more homogeneous state-of-the-art platform model.

Results show that the proposed platform models are able to correctly reproduce real-life variability, as opposed to the state-of-the-art platform model. Indeed, both the Maximum and Average models manage to correctly capture the distribution of the transfer durations. The analysis of the absolute logarithmic errors also shows that the proposed models clearly outperform the 10G-SotA model, which largely underestimates a vast majority of the transfer durations.

Based on the the obtained results, we conclude that realistic platform models are essential for the accuracy of simulations aiming at reproducing le transfers across a distributed infrastructure. They largely outperform the more simplistic, homogeneous existing models, thus fully accounting for the eorts needed to build such realistic models.

As future work, we plan to build on these very promising results and address the dierent limitations highlighted by our analysis of the root causes of the main simulation errors. It would also be interesting to conrm our ndings on a larger set of execution traces, and to conduct a deeper invalidation study. The spatial and temporal aggregations of dierent execution traces is a challenging task that would allow us move away from a specic trace and generate simulated platforms that mimic the characteristics of a production system.

We also aim at extending the simulation capacities of the VIPSimulator that underlies this study, not only to handle more complex scenarios that includes failed jobs, but also to simulate the other components of the workow execution.

Figure 2 :

 2 Figure 2: Representation of dierent platform models. Red boxes represent limiter links.

Figure 3 :

 3 Figure 3: Measured 1 and simulated (with a 10 Gb/s link per SE) transfer durations for two input les downloaded by each job in the fNUfzs workow instance.

Figure 4 :

 4 Figure 4: Measured 1 and simulated (with average or maximum bandwidth for each SE) transfer durations for two input les downloaded by each job in the fNUfzs workow instance.

Figure 5 :

 5 Figure 5: Measured and simulated (with global or grouped average bandwidth for each SE) durations for the download of the release le by each job in the fNUfzs workow instance.

Figure 6 :

 6 Figure 6: Measured and simulated durations for the downloads of the release le from the marsedpm.in2p3.fr SE in the fNUfzs workow instance (top). Simulated times are obtained with a single average bandwidth (SE) and distinct average bandwidths (SE-Site) for each site (bottom).

Figure 7 :

 7 Figure 7: Measured and simulated durations for the downloads of the release le from the marsedpm.in2p3.fr SE in the fNUfzs workow instance (top). Simulated times are obtained with a single maximum bandwidth (SE) and distinct maximum bandwidths (SE-Site) for each site (bottom).

Figure 8 Figure 8 :

 88 Figure 8 presents the part of the Gantt chart of the execution of the fNUfzs workow instance, when the nine downloads of the release le by worker nodes in the INFN-PISA site from the marsedpm.in2p3.fr storage elements take place.

Figure 9 :

 9 Figure 9: Measured and simulated durations for the downloads of the release le from the marsedpm.in2p3.fr SE in the fNUfzs workow instance with and without correction of the maximum observed bandwidth.

Figure 10 :

 10 Figure 10: Measured and simulated durations for the downloads of the release le from the sbgse1.in2p3.fr SE to the INFN-PISA site in the LQz3XJ workow instance with and without distinction of the clusters.

Figure 11 :

 11 Figure 11: Graphical summary of measured and simulated (with the 10G-SotA, Average, and Maximum models) le transfer durations (in seconds).

Figure 12 :

 12 Figure 12: Cumulative Distributed Functions of the absolute logarithmic error achieved by the three platform models over the whole set of transfers of the release les.

Figure 13 :

 13 Figure 13: Gantt chart view of the transfers of the release le from the ccsrm02.in2p3.fr SE to worker nodes in the AUVERGRID site in the 2RcwCY workow instance.

Table 1 :

 1 Statistics of measured and simulated (with the 10G-SotA, Average, and Maximum models) durations (in seconds) of the transfer of the GATE release les (> 121 MB).

		Min. 1st Qu. Median Mean 3rd Qu.	Max.
	Measured	2.00	5.01	17.42	49.91	77.31	888.80
	10G-SotA 1.19	2.02	2.03	2.55	2.15	11.52
	Average	2.01	5.21	11.44	45.17	73.62	873.80
	Maximum 1.98	3.21	7.44	31.72	35.09	873.80

For the sake of readibility, a data point corresponding to a transfer of the wrapper le that lasts for 65 seconds is not displayed.

Acknowledgments

This work is partially supported by the LABEX PRIMES (ANR-11-LABX-0063) of Université de Lyon, within the program "Investissements d'Avenir" (ANR-11-IDEX-0007) operated by the French National Research Agency (ANR) and the SONGS ANR project (11-ANR-INFR-13).


MD Josef Finsterer

PhD Sinda Zarrouk-Mahjoub

Why does Leigh syndrome respond to immunotherapy?

Mitochondrial DNA Leigh syndrome Immunosuppression Plasmapheresis Respiratory chain

Letter to the Editor

With interest we read the article by Chuquilin et al. about a 20yo female with Leigh syndrome due to the m.9176T N C mutation in the ATP6 gene who responded favourably to plasmapheresis and immunoglobulins [START_REF] Chuquilin | Response to immunotherapy in a patient with adult onset Leigh syndrome and T9176C mtDNA mutation[END_REF]. We have the following comments and concerns.

The main ambiguity of this report is the diagnosis. Except for the current case, Leigh syndrome has not been reported to respond to immunosuppressive treatment. Thus, the diagnosis of Leigh syndrome needs to be challenged. Which of the four variants in the ATP6, POLG1, DARS2, and LRPPRC respectively was the causative mutation? Was any of these mutations also detected in any of the parents or grandparents? Was there consanguinity between mother and father? Did the parents or other first degree relatives undergo neurologic work-up for mitochondrial disorder? Particularly the mother requires intensive work-up since she had a history of migraine, a frequent phenotypic manifestation of mitochondrial disorders [START_REF] Dermaut | Progressive myoclonic epilepsy as an adult-onset manifestation of Leigh syndrome due to m.14487T N C[END_REF]. Did the patient undergo muscle biopsy and a biochemical investigation? Were any deficiencies in the activity of the respiratory chain complexes detected? Did magnetic resonance spectroscopy (MRS) confirm elevated lactate in the cerebrum, particularly the basal ganglia [START_REF] Jurkiewicz | Spectroscopy in patients with Leigh syndrome[END_REF]? Was glycogen storage disease 3A genetically confirmed in one of the half-brothers? Did the patient or her relatives ever experience a stroke-like episode, seizures, migraine, or a psychotic episode?

Second, POLG1 mutations may not only cause CPEO, Alpers-Huttenlocher, and SANDO, but also mitochondrial depletion syndrome, encephalomyopathy, mitochondrial epilepsy, mitochondrial neuropathy, a MNGIE-like phenotype, MELAS/SANDO overlap syndrome, POLG-related levodopa-related parkinsonism, and a number of complex non-syndromic mitochondrial disorders [START_REF] Cohen | POLG-Related disorders[END_REF][START_REF] Finsterer | Leigh and Leigh-like syndrome in children and adults[END_REF].

Third, we should be informed if plasmapheresis and immunoglobulins also had a beneficial effect on the cerebral lesions? Was follow-up MRI and MRS normal?

Overall, this interesting case merits confirmation of the genetic diagnosis and a plausible explanation why immunosuppression had such a beneficial effect.

J. Finsterer, S. Zarrouk-Mahjoub / Molecular Genetics and Metabolism Reports 11 (2017) 90-91

Funding

No funding was received.

Conflict of interest

There are no conflicts of interest.