Filippo Rusconi
email: lippo.rusconi@u-psud.fr

†lcp

mineXpert: biological mass spectrometry data visualization and mining with full JavaScript-ability

Keywords: mass spectrometry, ion mobility, MS1 spectrum, data visualization, proteomics, isotopic cluster, metabolomics, software, C++, JavaScript, mzML, GPLv3+

Biological mass spectrometry mainly comprises three elds of endeavour, namely, proteomics, metabolomics and structural biology. In each of these specialties, the mass spectrometrist needs to access MS 1 mass spectral data, although not necessarily on the same basis. For example, the bottom-up proteomics scientist will occasionally access MS 1 data in order to perform data inspection, quality assessments and quantitation measurements, while top-down proteomics, structural biology or metabolomics scientists will actually spend most of their time mining prole-mode MS 1 data. Furthermore, the advent of ion mobilitymass spectrometry imposes new manners of mass spectral data visualization. An open-source MS 1 -only mass data visualization software for the desktop was developed to allow scientists to visualize conventional and drift time mass data. Various mass data integrations are possible, allowing a thorough mass spectral 1 data scrutiny. Isotopic cluster calculations are easily carried over from the chemical formula up to the display of the mass spectrum. Deconvolution of mass peaks can be achieved with a simple mouse drag. Flexible reporting of data inspection events and of mining discoveries is provided. Very large sparse datasets can be sliced into smaller chunks replicating the original data without data loss. Task automation is achieved in a JavaScript environment. This project allows users of mass spectrometry facilities to inspect and mine their MS 1 mass data outside of these facilities, without having to resort to the closed-source vendor software shipped with the instruments. mineXpert requires no proprietary software whatsoever once the mass spectrometry data have been converted to mzML. The reference implementation is version 5.8.2 or greater.

Introduction

Mass spectrometry is an essential technique in the eld of biology as it allows scientists to probe biological systems at dierent levels: the structure of isolated analytes (e.g. structural biology); the identication and quantitation of analytes in complex mixtures (e.g proteomics and metabolomics). In bottom-up proteomics and, to a lesser extent, in metabolomics, the sheer number of analytes in any given sample requires software to almost automatically process the data (OpenMS, [START_REF] Röst | OpenMS: A exible open-source software platform for mass spectrometry data analysis[END_REF] ProteoWizard, 2 X!TandemPipeline, 3 massChroQ 4 ). In topdown proteomics or structural biology mass spectrometry, the mass data are acquired on enriched or puried analytes with the aim of identifying them and characterizing their ne structure (e.g. chemical modication, covalent adduct, cross-link, non covalent polymerligand interactions). Whatever the eld of endeavour, the biological mass spectrometrist will invariably access the MS 1 mass data (i.e. survey scans), either to perform quality assessments, quantitation measurements or to actually mine the data. In the specic case of bottom-up proteomics, in the eventuality of dubious protein identications, resorting to the visualization of precursor ions in the original survey scans is common practice, as the shape or intensity of m/z peaks can enlighten the scientist on the validity of a peptidespectrum match. These tasks are highly time-consuming, making an ecient mass data visualization and mining tool highly desirable, even more so nowadays, as mass spectrometry data have ever greater depths: hyphenated mass spectrometry techniques produce massive amounts of time-resolved mass spectral datasets and the recent advent of ion mobilitymass spectrometry (IMMS) has added another extra layer of time-resolved spectral data. Finally, the technical validation of mass spectral data acquisitions is invariably performed by looking manually at the acquired data. In medium-sized datasets that are typical of the average mass spectrometry facility, this task becomes tedious and error prone. Therefore, the ability to automate the various steps involved in the technical validation, like data le loading, mass spectral integration, statistical calculations over the total ion current (referred to as TIC in this report), becomes an appealing feature.

Several biological mass spectrometry data visualization software projects exist. MZmine 2

and BatMass provide a Java-based software environment but do not support IMMS data.

5,6

The mMass software, a Python-based proteomics-oriented spectrum viewer, has no IMMS capabilities [START_REF] Strohalm | MMass 3: A cross-platform software environment for precise analysis of mass spectrometric data[END_REF] . Amphitrite provides a Python-based IMMS-centric software solution that lacks a exible mass spectrum viewer capable of loading an arbitrary number of datasets simultaneously [START_REF] Sivalingam | Amphitrite: A program for 15 processing travelling wave ion mobility mass spectrometry data[END_REF] . TOPPView 1 and SeeMS 2 both allow highly detailed mass spectral data scrutiny but have no combination/integration features. Skyline, also a free software program, is mainly aimed at simplifying the elaboration of mass data acquisition methods and the analysis of data from targeted proteomics experiments [START_REF] Maclean | An open source document editor for creating and analyzing targeted proteomics experiments[END_REF] . To the best of my knowledge, none of the software above can be automated using a scripting language.

mineXpert is a JavaScript-able desktop software program implementing uid navigation through the full depth of TIC and extracted ion current (XIC) chromatograms, MS 1 and drift time mass spectrometric data. Users of mass spectrometry facilities have to adapt to the various mass spectrometry software programs that are shipped with the instruments, making their data visualization and mining an unnecessarily dicult task. mineXpert implements all the commonest features found in a variety of proprietary software programs so that a single program will t all the needs, whatever the instrument whence the data originated. The scripting capabilities of mineXpert allow scientists to automate and reproduce data visualization and mining procedures. Data mining procedures include easy deconvolution of mass peaks, recording of useful features and reporting of mining discoveries in expressive ways.

mineXpert is bundled with the massXpert [START_REF] Rusconi | Desktop prediction/analysis of mass spectrometric data in proteomic projects by using massXpert[END_REF][START_REF] Rusconi | MassXpert 2: A cross-platform software environment for polymer chemistry modelling and simulation/analysis of mass spectrometric data[END_REF] polymer modelling and mass spectrometry simulation software to form the msXpertSuite software project (www.msxpertsuite.org).

The whole software suite, datasets and documentation les are licensed under the GNU General Public License version (GPL v3+).

Methods

The software is written in C++11 using the cross-platform Qt libraries (www.qt.io). The software is thus compatible with MS Windows (tested versions: The mass spectrometry data le loading process is handled by the libpwiz library (ProteoWizard) and the isotopic cluster calculations are performed using the libIsoSpec++ library. [START_REF] Lacki | IsoSpec: Hyperfast Fine Structure Calculator[END_REF] Both the libpwiz and libIsoSpec++ libraries are expressly built by the author for all the supported platforms and included in all the binary packages. The main development platform is Debian GNU/Linux (www.debian.org). As an ocial Debian Developer, the author produces software packages for Debian/Ubuntu and derivatives (typically installed using the apt-get install msxpertsuite command). Binary packages are available for MS Windows and MacOSX (www.msxpertsuite.org). mineXpert has been tested with ion mobiliy mass spectrometric data les from the Synapt G2Si-HDMS instrument and from the Agilent 6560 ESI-IMS-Q-TOF instrument. [START_REF] Gabelica | Optimizing Native Ion Mobility Q-TOF in Helium and Nitrogen for Very Fragile Noncovalent Structures[END_REF] Conventional mass spectrometry data les could be loaded from a wide variety of mass spectrometers.

mineXpert loads mass data les in the mzML format from the Proteomics Standards Initiative [START_REF] Deutsch | Proteomics Standards Initiative: Fifteen Years of Progress and Future Work[END_REF] (see [START_REF] Deutsch | Mass spectrometer output le format mzML[END_REF] for a thorough description), as obtained by conversion of raw data les using the msconvert [START_REF] Holman | Employing ProteoWizard to Convert Raw Mass Spectrometry Data[END_REF] tool in the ProteoWizard software package. When mass spectrum scrutiny is required for a unitary mass spectrum, mineXpert loads any data in the generic <m/z><separator><intensity> text format with <separator> being any non-numeric character sequence. A single mass spectrum can be loaded from le or directly from the clipboard.

The mzML format was selected because it is a standard supported on all the computing platforms. However, in future versions and on the MS Windows platform, the software might be able to load data directly from proprietary raw les using the libpwiz library.

For this report, the dataset that is used to describe mineXpert was from a size-exclusion chromatographyIMMS study of the bovine alcohol dehydrogenase in native conditions;

6 Gb dataset containing 318462 non-empty spectra. 17

Results and discussion

The capabilities of the mineXpert software will now be described in detail following a functional path from data le loading to the recording of mining discoveries.

Graphical user interface fundamentals mineXpert allows one to load any number of mass data les in the same session. The general approach of the mineXpert software architecture is to allow mass spectral scrutiny in dedicated windows (Figure S1) that get progressively populated with all the data plots created as a result of user activity: the TIC-chromatogram window displays the TIC chromatograms of all the loaded data les and any XIC chromatogram created by the user; the Mass-spectrum window displays all the mass spectra obtained as a result of mass data combinations. If IMMS data are loaded, the Color-map window displays the mass spectra vs drift times color map and the Drift-spectrum window displays all the drift spectra resulting from mass data integrations. Upon loading of a mass data le, a TIC chromatogram is automatically computed and, in case of IMMS data, a color map is computed also (Figures 1 andS3).

Rendering of mass datasets from dierent vendors

Successful combination of mass spectral data from dierent vendor data les is a challenge per se, because these data les may contain m/z data structured in distinct ways. For example, m/z data heterogeneity in a given dataset might be due to the mass spectra having 1) a variable number of data points, 2) varying origin m/z values, or 3) dierent intervals between neighbouring m/z values (that is, varying m/z steps). This mass data heterogeneity might only be problematic when performing mass data integrations to a mass spectrum (neither to a XIC chromatogram nor to a drift spectrum). Code was therefore developed to let the user tweak the conguration of the mass data combinations to a mass spectrum. The user is presented with a number of options involving (or not) explicit binning of the m/z data, and with preset values determined on the basis of a statistical analysis of the whole mass dataset (Figure S2). In the vast majority of all cases, the binning conguration automatically determined by statistical analysis of the whole mass spectral dataset yields perfectly ne mass spectra. However, tweaking the m/z integration paramaters may prove useful to rene the obtained mass spectrum in some corner cases or to accelerate the combination process when datasets are very large (by increasing the size of the bins to an acceptable level).

Chaining mass data integrations

A typical mass data visualization session starts within the TIC chromatogram that is computed upon loading of a mass data le (and the color map, in case of IMMS data). Data integrations allow probing the data in all their depth, as illustrated in compact form in Figure 1 and, in a form more closely representing the user interactions with the graphical interface, in Figures S3 to S6.

After an IMMS data le has been loaded (size-exclusion chromatography of bovine alcohol dehydrogenase (ADH) in native conditions), the initial plots are shown in the top panel of Figure 1 and Figure S3. The mass spectral data integration to a mass spectrum, starting either from the TIC chromatogram or from the color map, is performed by combination of all the mass spectra that have been acquired in the selected retention time region (or the drift time / mz region, in case of IMMS data). All combinations to a mass spectrum are congured according to Figure S2.

The mass spectrum that is obtained shows the various oligomerization states of the alcohol dehydrogenase and their varying ionization levels (upper middle panel of Figure 1 and Figure S4). In order to determine the drift time of the various oligomerization states, integrations to a drift spectrum are performed over each m/z region corresponding to an oligomerization state of interest. Each integration produces a new drift spectrum (lower middle panel of Figure 1 and Figure S5).

The three drift spectra show that, whatever the charge states of the ions, the monomer elutes rst and that the dimer elutes over a longer retention time range than the monomer, which is expected because this is a size-exclusion chromatography.

The various mass peaks belonging to the charge state envelopes of the monomer and oligomers might then be individually integrated to drift spectra. In this case (not illustrated), the resulting plots would show that the higher the charge of the tetramer (charge range 24 + 29 + ), the smaller the drift time, as would normally be expected. Back-integration of drift peaks C and D to a mass spectrum would show that drift peak C is not related to the protein but to another analyte. All the mass spectral integrations detailed in this example show that the capability to perform chained integrations in any direction allows one to very nely scrutinize mass spectral data in their full depth.

It is worthwhile noting that the original dataset loaded into the computer's memory did not change along the various integrations described above. The data consistency is preserved throughout successive integrations via a mechanism that records internally all the integrations that have led to a specic data plot (these internal integration details, called history, are described in detail in the user manual provided as Supplementary Information). When that data plot is used as a starting point for a new integration, that history is transmitted to the mass data integration engine so that it performs the new integration only on the subset of the original dataset that accounts for all the history steps that occurred prior to this last integration. The fact that the original dataset is always available for new integrations permits proper comparison of mass spectral features. This feature sets mineXpert apart from the Driftscope software that is shipped with the Waters Synapt instruments.

Mass peak deconvolutions

When mining data or when assessing the quality of a mass spectral acquisition, a procedure that is routinely performed is the determination of the molecular mass of analytes detected in a mass spectrum. All the mass spectrum-displaying widgets feature the possibility to deconvolute mass peaks according to the two commonly used methods: isotopic cluster-IsoSpec is not easily congurable and mineXpert provides a user-friendly interface to perform the various kinds of congurations and user interactions needed to produce a mass spectrum describing the isotopic cluster (the steps are described in great detail in the user manual provided as Supplementary Information).

Slicing of very large mass spectral datasets

When performing hyphenated techniques mass spectrometry, in particular with size exclusion chromatography, the mass spectral datasets are very large but the useful mass data might be much smaller, as in the example used above to illustrate the data integrations (see the TIC chromatogram in Figure 1, where useful data are located between min 9 and min 13 over a 30 min-long run). In order to reduce the size of huge datasets, code was developed to slice the original dataset into smaller chunks having an exact data subset copy of the original data (Figure S8).

Data processing automation using JavaScript

In mineXpert, data processing automation was designed in such a manner that JavaScript statements can be issued either in a scripting console window or from a script le that might be read at program launch or from within the scripting environment. Figure 2 displays the main scripting environment window.

The scripting environment allows one to control the behaviour of all the windows containing plot widgets and the plot widgets themselves. Data from the plot widgets can be extracted to JavaScript objects that can then be used in the scripting environment according to the standard JavaScript syntax. Any kind of data can be exported to text les and plots can be exported to graphics data les. A history feature allows one to easily recall past commands (Figure S9) and a complete JavaScript developer reference documentation is available (Figure S10).

With JavaScript, mineXpert allows the user to conceive data processing and quality . assessment workows that can run unattended and thus be totally automated. A typical use case is when calibrating an IMMS instrument's mobility cell. At that step, the mass analyzer has already undergone calibration, making mass measurements highly reliable, thus ensuring that the script code carries out awlessly all the steps involved in the mobility cell calibration.

Another interesting use case of scripting is when scientists who share the same datasets want to share data mining procedures to discuss the results. In this case, sharing a simple script allows anyone to replay in an identical manner the data mining steps (integrations, subtractions, zooming, panning, data export) that the script author performed in the rst place. Finally, in quality control procedures for proteomics or metabolomics experiments, where internal standards are used in a series of mass spectrometric acquisitions, automatic control of these standards can be achieved using script-based automation.

Data verication, mining and reporting

Very often, while mining data or performing quality assessments, recording of the ndings is either desired or required. A recording system was developed to allow a exible reporting of any nding, either to the clipboard, to a le, to the program console or to any combination of the three. The recording of the ndings works exactly the same way in all the data plot widgets, TIC or XIC chromatogram, mass or drift spectrum. The kind of data that are recorded are, however, dierent: in chromatograms, the recorded data are typically intensities versus retention times; in mass spectra, intensities versus m/z values; in drift spectra, intensities versus drift times. To cope with that diversity of data, a specic syntax was devised such that the user may exibly report any kind of data (Figure S11). While mining mass data, the user may record interesting ndings by hitting the space bar. minexpert-doc.pdf: user manual of the mineXpert software program.

  7 and 10), Linux and MacOSX (versions 10.11.6 and newer). The QCustomPlot library (www.qcustomplot.com) is used for plotting. Code parallelisation is based on OpenMP (www.open-mpi.org), a de facto standard that implements portable parallel code at compile time under the control of compiler directives in the code. Tests revealed that parallelized code shows execution speed gains almost proportional to the number of processing threads.

(

  A), dimer (B) and tetramer (C and D) have signicantly increasing drift times: dt (A) < dt (B) < dt (C, D): the higher the oligomerization state, the greater the drift time in the ion mobility cell. To determine at what retention time each of the various ions have eluted, integrations to a XIC chromatogram over the various regions of interest in the drift spectra are performed (bottom panel of Figure 1 and Figure S6). The results show that the tetramer

Figure 1 :

 1 Figure 1: Mass data integrations can be performed in sequence. Each time a new integration is performed, the generated data subset is displayed in a new plot widget that may serve as a new integration starting point. This process allows one to dig into the depths of mass spectral data in a reasoned sequential manner.

Figure 2 :

 2 Figure2: JavaScript-based automation of data processing tasks. The scripting environment lists, on the left hand side of the window, all the objects that have been produced by the software all along its use. Data plot widgets can be controlled as if by mouse using specic functions that are detailed in the JS reference tab. In this example, a MassSpectrum object is created from the data currently displayed in the massSpecPlotWidget0 widget and then output in a textual form. Combinations or subtractions can be performed using pure JavaScript objects.

Acknowledgement

The author thanks Dr Valérie Gabelica for providing IM-MS datasets that have served to test the software with Agilent instrument-produced les; Prof. Guillaume van der Rest and Dr Frédéric Halgand, for many enjoyable and interesting discussions and for the Waters Synapt2originating dataset used to illustrate mineXpert in this report; the PAPPSO team at the GQE-Moulon laboratory, for fruitful discussions on proteomics-oriented mass spectrometry data acquisition and analysis; Dr Olivier C. Martin for the critical reading of the manuscript.

based and charge state-based deconvolutions (Figure S7).

Comparisons of mass spectral feature intensities

Analyzing mass spectral data most often requires comparing the intensity of features from dierent mass datasets. The single TIC intensity value integration computes the TIC intensity of any data feature found in the dataset, be it an isotopic cluster, a mass or drift peak, a chromatogram peak, or a color map region. Whatever the feature, the TIC intensity is computed by going back to the original mass spectral dataset rst restricted to the data subset represented as the feature at hand (see above for the history concept).

For example, to compute the intensity of the signal encompassing the m/z region [5017 6234] of the mass spectrum in Figure 1 ([24 + 29 + ] charge levels of tetrameric ADH), the user would right-click drag the mouse cursor over that m/z region while pressing the `I' keyboard key. The single TIC intensity value of the data feature is shown in the status bar of the Mass-spectrum window in Figure S4. The interesting feature here, is that if the same process were applied to the whole drift time range of the third drift spectrum (tetramer, Figure S5) corresponding to that same m/z region, then the single TIC intensity value would be the same as that obtained above. This is because the drift spectrum plot is equivalent, from a mass spectral data standpoint, to the mass spectrum plot in region [50176234], as both are dierent representations of the same subset of the original dataset.

Isotopic cluster calculations

Isotopic cluster calculations are often required in order to verify that a given acquired ion signal actually corresponds to a given ion of interest. This kind of pattern-matching work is typically used in top-down proteomics or in studies involving heavy isotopes. The IsoSpec project [START_REF] Lacki | IsoSpec: Hyperfast Fine Structure Calculator[END_REF] implements a powerful algorithm in the form of a high performance library published under a Free Software license. To be used in mineXpert, the IsoSpec software was packaged for Debian and also built in the MS Windows and MacOSX ports of mineXpert.

Conclusions

The mineXpert software program described in this report will be of use to a wide mass spectrometry audience because it allows mass spectrometry facility operators and users to inspect, validate and mine mass spectrometry datasets without the need of the closed-source software shipped with the instruments. mineXpert features the ordinary functionalities found in proprietary software systems and implements original mass data viewing paradigms and reporting of data mining discoveries. The software capability to handle ion mobilitymass spectrometry data will be of interest to the ever greater community of mass spectrometrists who implement this new technique. The isotopic cluster calculation functionality will be of interest to the top-down proteomics scientists. The automation of all the data loading and integration processes is of great value to mass spectrometry facility operators and users willing to set up unattended processing workows.

Supporting Information Available

The following supporting information is available free of charge at ACS website http://pubs.acs.org: supplementary-information.pdf: detailed gures and explanations of the mineXpert program features.


Tristan Glatard

Adrien Marion

Hugues Benoit-Cattin

Sorina Camarasu-Pop

Patrick Clarysse

Rafael Ferreira Da Silva

Germain Forestier

Bernard Gibaud

Carole Lartizien

Hervé Liebgott

Kévin Moulin

Denis Friboulet

Multi-modality

Keywords: Medical image simulation, multi-modality, cardiac, MRI, echography

HAL is

INTRODUCTION

Medical image simulation is essential to improve the understanding of biological processes, pathology diagnosis and treatment. Wide-scale availability of simulated medical images would greatly help to design new image acquisition techniques, to develop realistic physiological models and to validate image analysis procedures. However, using simulators remains complex and requires (i) availability of proper object models, (ii) significant technical skills to parametrize and run the simulators and (iii) access to computing and storage resources to support heavy simulations.

The Virtual Imaging Platform1 (VIP) is an open web platform targeting the sharing of simulators and object models as well as their execution on distributed computing resources. It is designed to be extensible and currently includes simulators of ultrasound (US) imaging, magnetic resonance imaging (MRI), positron emission tomography (PET) and computed tomography (CT).

In this paper, we describe VIP features and show how it is used to simulate multi-modality medical images from generic models. Although realistic images can be simulated from in vivo images such as proposed in [START_REF] Kutter | Visualization and gpuaccelerated simulation of medical ultrasound from CT images[END_REF][START_REF] Reichl | Ultrasound goes GPU: real-time simulation using CUDA[END_REF], this paper considers a different approach where simulations are based on models of organ geometries and physical parameters. This approach enables model sharing for various image modalities in a common repository. It is also flexible since simulated images can be obtained directly from organ geometries and physical parameters, without any in vivo twin. For instance, model parameters can be varied to simulate individual populations or specific pathologies.

The integration of simulators and models in VIP is demonstrated on ultrasound and MRI simulations from the ADAM 4D cardiac model [START_REF] Haddad | A realistic anthropomorphic numerical model of the beating heart[END_REF]. The model repository, integration of simulators and supporting infrastructure are described in section 2; cardiac imaging simulation implementation and results are reported in section 3.

PLATFORM DESCRIPTION

Simulator integration

To facilitate their integration, applications in the VIP repository are described as workflows of elementary activities from which launching and monitoring interfaces are automatically generated. Workflow descriptions also enable parallelization since they are parallel by nature. Parallelization exploits data parallelism so that simulators only require minor modifications (mostly parameter adjustments) to be integrated.

Most simulators produce images from an object model with modality-specific physical parameters and a parametrized representation of a scanner. A specific workflow template was defined for image simulation [START_REF] Marion | Multi-modality medical image simulation of biological models with the virtual imaging platform (VIP)[END_REF]. It consists of object preparation (conversion to simulator-specific format), parameter file generation from parameter values, core simulation and post-processing (e.g. reconstruction). For now, this template is only used to structure simulator integration and to facilitate interaction with platform tools. In the near future, it will be used as input of a semi-automatic simulator workflow designer [START_REF] Cerezo | Scientific workflows reuse through conceptual workflows[END_REF].

Four image simulators are currently available: Field-II [START_REF] Jensen | Calculation of pressure fields from arbitrarily shaped, apodized, and excited ultrasound transducers[END_REF] for ultrasound, Simri [START_REF] Benoit-Cattin | The SIMRI project : a versatile and interactive MRI simulator[END_REF] for MRI, PET-Sorteo [START_REF] Reilhac | PET-SORTEO : Validation and development of database of simulated PET volumes[END_REF] for PET and Sindbad [START_REF] Tabary | SINDBAD: a realistic multi-purpose and scalable X-ray simulation tool for NDT applications[END_REF] for CT. A GATE plugin is also available for radiotherapy simulations [START_REF] Jan | GATE: a simulation toolkit for PET and SPECT[END_REF]. Other applications have also been integrated, including MR liver cartography, Mean-Shift image filtering and cardiac segmentation.

Model repository

Models in the VIP repository are annotated with the terms of an ontology of simulation object models that we proposed in [START_REF] Forestier | Sharing object models for multi-modality medical image simulation: a semantic approach[END_REF]. Model files are annotated as physical parameters or as objects belonging to a geometrical, anatomical, pathological, foreign body or external agent layer. Layers allow to easily add/remove structures such as pathological objects or contrast agent to a baseline anatomy. When a simulation is launched, layers are flattened by object preparation workflows to produce simulator inputs. Physical parameters can be maps or look-up tables of magnetic, chemical or echogenicity parameters. Model files are also annotated with time information. Two time scales are considered: instants define a fine scale (e.g. movement) while time points are used at a coarser scale (e.g. longitudinal follow-up).

These annotations are used to structure the repository and for searching. Inference rules can also be applied on the annotations of a model to check if it has enough physical parameters to be used in a simulation of a given modality. More elaborated rules may be envisaged too. Annotations are also used by the workflows, e.g., to split the simulation in independent time-points and instants. Soon, they will also be used to annotate simulated results.

Models can be visualized in a webGL2 -based 3D interface and scenes where image simulators can be positioned w.r.t the model can be defined. The model and selected simulators can be translated and rotated using spinners. This interface produces transformation matrices used by object preparation workflows. A screenshot is shown on Fig. 1.

Model annotations can describe files regardless of their type but interfaces and workflows only support limited file types, mostly based on VTK: mhd/raw files for labeled voxelic representation or physical parameter maps, vtp files for mesh representation, text files for lookup tables giving correspondence between labels and matters, xml files for lookup tables giving correspondence between matters and physical parameters, and vtu files containing a list of scatterers used in ultrasound simulation. Model files and annotations can be downloaded for local inspection or processing.

Infrastructure

Computations are performed on the biomed virtual organization of the European Grid Infrastructure (EGI-biomed) 3 , which is openly accessible for non-commercial computations. This infrastructure is shared among approximately 1,000 users, but VIP can usually run some 2,000 concurrent tasks. Personal clusters can also be declared in the platform.

Data files containing object models, intermediate results or simulated data are stored on distributed storage sites of EGI-biomed where 1.65 PB is currently available. Although storage is not an issue, simulations can be hampered by the transfer time of large files or file collections. To cope with downtimes of storage sites, model files and simulation results are replicated on three sites with good availability history.

CARDIAC IMAGE SIMULATION

The heart-beating and thorax-breathing ADAM model [START_REF] Haddad | A realistic anthropomorphic numerical model of the beating heart[END_REF] is used in these simulations. It consists of the pericardium, the left and right ventricles and atria, the aorta, the lungs, the spine, the spinal cord and the inner and outer thorax.

Ultrasound simulation

Field-II is a widely-used ultrasonic simulator that relies on an acoustical model to simulate propagation. The model is represented by a set of scatterers defined by 3D positions and scattering coefficients.

The position for each scatterer is defined by 3 values x, y and z. A scattering coefficient is assigned to each scatterer according to the intensity of the signal backscattered by the tissue it belongs to. Position and scattering values are generated from tissue-dependent statistical distributions (e.g. uniform for position, Gaussian for scattering coefficient). A density parameter is also used to define the number of scatterers per voxel. US simulation consists in generating a set of radio-frequency (RF) lines that are assembled to produce the final image.

If scatterers are already present in the model they are directly transformed to the geometry defined by the simulation scene. The simulation is then launched on distributed computing resources. Each computing job receives one RF line to simulate. RF lines are accumulated in an RF matrix as soon as they are produced. Once the matrix is complete, a B-mode image is obtained from envelope detection and cartesian reconstruction.

If scatterers have to be generated, the voxel representation file, the LUTs and the transformation are used to generate the scatterers according to Algorithm 1.

Once the scatterers are generated, they are transformed to the simulation scene and only those in a slice around the Algorithm 1 Generation of scatterers. // in: labeled volume (vol) -dimensions (D)-sampling rate (S)physical parameter LUT (LUT) -number of scatterers (Nd) // out: scatterer positions (pos) and amplitudes (amp) for i in LUT do tissueVox(i) = find(vol==tissue(i)) end for nb scat = createVector(1, imaging plane are extracted.

P N i=1 ni * di) // ni
Here, scatterer positions were generated from a uniform spatial distribution with a density of 10 scatterers per voxel. Scattering coefficients were generated from centered Gaussian distributions with amplitudes reported on Table 1.

MRI simulation

Simri simulates MR images from Bloch equations and is parallelized using MPI 4 . A medium is represented as a labeled volume associated to LUTs. LUTs link labels to Gaussian distributions from which physical parameter values are randomly sampled. Physical parameters used for ADAM classes are reported on Table 1. A spin-echo sequence with TE=20ms and TR=400ms was simulated with B 0 = 3T.

Results

Fig. 2 shows US simulation results for different views and Fig. 3 shows 3 MR slices simulated over the cardiac cycle. Each simulated US view represents a total of 750 CPU hours, computed in about 22 hours on VIP. Each MR slice represents 24 CPU minutes, computed in 4 minutes on VIP.

The realism of simulated images could be improved. Using more detailed object models or improving the distributions of physical parameters are the main axes for future work. For example, borders could receive more scatterers to mimic specular reflection in ultrasound.

CONCLUSION

VIP is an online open platform for medical image simulation. Simulators of four imaging modalities are available and object models can be shared in a semantic repository. Simulations can transparently benefit from resources of the European Grid Infrastructure and local clusters.

We demonstrated how VIP was used to simulate cardiac ultrasonic and MR images from a geometrical model. Images simulated from generic models are less realistic than imagebased simulations but allow more flexibility. VIP was designed as an extensible platform to support simulation-based research. Other simulators could also be integrated. Integrating more elaborated and diverse object models used in the same simulation workflow is also possible.

Fig. 1 .

 1 Fig. 1. Snapshot of the VIP interface. Top: model repository. Bottom: simulation scene interface showing a mesh representation of a cardiac model and an ultrasound probe.

  (a) Apical 4 chambers, at end systole. (b) Apical 4 chambers, at end diastole. (c) Apical 2 chambers, at end systole. (d) Apical 2 chambers, at end diastole. (e) Parasternal long axis, at end systole. (f) Parasternal long axis, at end diastole. (g) Parasternal short axis, at end systole. (h) Parasternal short axis, at end diastole.

Fig. 2 .

 2 Fig. 2. Simulated echocardiographic data.

Fig. 3 .

 3 Fig. 3. Simulated MRI short axis views.

Table 1 .

 1 is the number of voxels corresponding to tissue i, di is density of tissue i and N the number of tissues MR/US parameters for ADAM classes.

	for p in Nd do
	e = random(nb scat)
	ind = find((e-nb scat)≤0)
	ind vox = (e-nb scat(ind-1))/d ind-1
	voxCoord = tissueVox(i)(ind vox)
	pos(p) = voxCoord*N + N*random(0,1)
	amp(p) = LUT(ind-1).physParam
	end for

http://vip.creatis.insa-lyon.fr

http://www.khronos.org/webgl

http://www.egi.eu

http://www.mcs.anl.gov/research/projects/mpi

http://www.france-grilles.fr

ACKNOWLEDGMENT

This work is funded by the French National Agency for Research under grant ANR-09-COSI-03 "VIP". We thank the European Grid Initiative and "France-Grilles" 5 for providing the computing infrastructure and user support.