Giliandro Farias

David S Simeão

Thamires S Moreira

Paloma L Dos Santos

Ahmed Bentaleb

Edivandro Girotto

Andrew P Monkman

Juliana Eccher

Fabien Durola

Harald Bock
email: bock@crpp-bordeaux.cnrs.fr.

Paloma L Dos Santos

Bernardo De Souza
email: bernardo.souza@ufsc.br.

Ivan H Bechtold
email: ivan.bechtold@ufsc.br.

An unusual plank-shaped nematogen with a graphene nanoribbon core

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Nematics are the simplest and most abundant type of liquid crystals, and come in a great variety of molecular structures. To form a standard (uniaxial) nematic mesophase, i.e. a liquid with no long-range positional molecular order but with macroscopic anisotropic orientation along one preferred direction, one of two basic molecular shapes is generally required: Either the semi-flexible molecule is long in one dimension and short in the two others, i.e. a semi-flexible rod, or it is long in two dimensions and short in the third, i.e. a semi-flexible disk. Whilst rod-shaped nematogens are extremely abundant, disk-shaped ones are much rarer, because bidimensional disk-shaped molecules tend to aggregate more strongly than rods and thus prefer columnar mesophases with periodic positional order. The most common molecular structure of rod-shaped nematogens is a combination of two or more benzene units, linked either directly as in biphenyl, or via linker moieties such as -COO-or -N=CH-, plus one or two terminal flexible linear alkyl, alkoxy or alkylester substituents, that lengthen further the structure and add flexibility. Often also a polar head group, such as -CN, is present, increasing the polarizability of the conjugated pi-electron system along the rod. The flexible alkyl substituents are almost always linked to the head or tail of the aromatic rod-shaped unit, and not laterally, because lateral substitution reduces the anisotropic molecular shape and quickly leads to the loss of the nematic behavior. Even if the aromatic part is extremely elongated, only very small lateral alkyl extensions are tolerated, and terminal alkyl groups should preferably be present as well to compensate the lateral extension. A particularly illustrating example is the family of terminally and laterally alkyl-substituted 1,4-bis [2-[4-(2phenylethynyl)phenyl]ethynyl]-benzenes 1 described by Yatabe et al. 1 The alkyl substituents R1, R2 and R3 on its near-ideally rod-shaped, strongly nematogenic core have been varied systematically to study the destabilization of the nematic behavior by lateral alkyl substitution and its stabilization by terminal alkyl substitution. With R1=octyl, R2=hexyl and R3=H, i.e. with two long terminal and two long lateral chains, an enantiotropic nematic phase was observed, but if R3 is increased to ethyl, no mesophase was observed. The mesophase reappears if R2 is shortened to ethyl (i.e. with R1=octyl and R2=R3=ethyl), but even without R2, lengthening R3 just to propyl destroys the mesophase in spite of the presence of long terminal chains R1. With R1=R3=H and R2=octyl, i.e. with only two long lateral chains, no mesophase was observed either. Thus even in this family that has an extremely pronounced rod shape of the rigid part of the molecule, the nematic mesophase is very fragile towards the introduction of lateral alkyl chains.

Scheme 1. Near-ideally rod-shaped phenylethinyl tetramers, whose nematic mesophase is easily suppressed by introducing lateral alkyl chains R2 and R3. 1 Scheme 2. Plank-shaped phenacenes with four lateral alkylester chains. 5 Central unsubstituted phenacene fragments are marked in blue. The nematic liquid crystalline homologue 4 is highlighted in bold.

If the conjugated rigid part of the molecule is itself extended in one lateral direction, one would expect increased stacking, favoring 3D crystalline, 2D columnar or 1D smectic (lamellar) periodicity at the expense of nematic (i.e. orientation-only) order. This strong tendency to stack via intermolecular dipolar or pi-electron interactions also has thwarted efforts to obtain the elusive biaxial nematic mesophase with rigid plank-shaped mesogens; this goal has instead been reached by less rigid lateral extensions using flexible macrocycles 2 or bow-shaped mesogens. 3 Extended semi-flexible molecular cores bearing more than two terminal chains (the so-called tetra-and hexacatenar mesogens) have also failed to give a biaxial nematic mesophase, but they give in some cases a standard (uniaxial) nematic mesophase. 4 Some of us recently developed the synthesis of long phenacenes decorated with four solubilizing lateral alkylester moieties, 5 because phenacenes are the simplest fully condensed stable graphene ribbons (in contrast to the much less stable acenes) and thus are of fundamental interest as archetypal pi electron systems. All members of this family, i.e. tetraester derivatives 2-5 of [8]-, [10]-, [12]-and [14]phenacene, directly melt upon heating from the crystalline to the isotropic liquid state, without intermittent appearance of an enantiotropic liquid crystalline mesophase.

We have now found that the [12]phenacene derivative 4 surprisingly shows a monotropic nematic phase when, after melting at 247°C, 6 the isotropic liquid is cooled to 241°C. No nematic behavior could be observed with the shorter and longer homologues 2, 3 and 5. Given that nematic phases of rod-shaped molecules with only lateral flexible substituents have to our knowledge never been observed before, the occurrence of such a phase in elongated plank-shaped 4 is intriguing, especially in the absence of any such phase in the more elongated 5. Whilst the absence of a mesophase in the shortest homologue 2 may be attributed to its less anisotropic shape, one possible explanation for the absence of a mesophase in 5 and 3 is that contrary to 4, both possess a rather long unsubstituted chrysene fragment in the middle, which allows easier intermolecular non-parallel stacking interactions than the shorter central naphthalene unit in 4.

Upon slow cooling or when maintaining the sample at a temperature between 100°C and 220°C, crystallization is observed (various crystalline textures can be observed depending on the process). At temperatures above 170°C, this crystal growth nucleates slowly at the rim of the sample, i.e. at defects at its air interface, whilst below 170°C, crystallization starts from multiple nucleation sites in the bulk of the material. Upon fast cooling to temperatures around 100°C or below, the typical nematic Schlieren texture of 4 between glass plates is maintained down to ambient temperature (Figure 1). This suggests that a nematic glass is formed upon fast cooling below a critical temperature, allowing the conservation of nematic order at room temperature. To confirm this, we performed powder X-ray diffraction studies on a bulk sample in a glass capillary at different temperatures (Figure 2). Upon stepwise heating, with diffractograms recorded every 50 degrees, of a sample quickly cooled from 250 to 0°C, only a very broad peak at q = 0.35 Å -1 corresponding to an average intermolecular distance of c. 18 Å is observed at 0, 50 and 100°C, indicating the absence of any long-range periodicity, whereas crystalline order is manifest at 150 and 200°C. At 250°C, again only a diffuse peak at q = 0.35 Å -1 is observed. Upon stepwise cooling from high temperature to 150°C, with diffractograms recorded every 10 degrees, only the diffuse peak is observed down to 170°C, whilst a multitude of peaks characteristic of crystalline order appear at lower temperature. The lone broad peak corresponding to a distance of c. 18 Å does not shift significantly at the isotropicto-nematic transition and only a very slight contraction is observed on fast cooling to room temperature. Albeit no significant difference can thus be discerned between the diffractograms in the isotropic phase at 250°C and those either at lower temperature in the nematic mesophase before the onset of crystallization or at 0 to 100°C in the nematic glass after fast cooling, the absence of any sharp peaks together with the corresponding textures observed by POM confirm conclusively the formation of a nematic phase upon cooling the isotropic liquid and the conservation of the nematic order at low temperature without any "texture-neutral" crystallization that might pass unobserved at POM. The nematic-to isotropic phase transition manifests itself clearly by differential scanning calorimetry (DSC) as a low-enthalpy transition (0.2 J/g) that shows no significant temperature hysteresis between cooling and heating, whereas the transition to the crystalline state has a hundred times higher transition enthalpy (Figure 3). As the molecular shape of 4 is moderately biaxial, due both to the flat ribbon shape of the arene core and to the presence of four lateral chains, and as the occurrence of a nematic phase in a lateral-chains-only rod or plank is unprecedented, the question arises whether the nematic mesophase is biaxial. Contrary to Schlieren textures of uniaxial nematics, that display both four-and two-brush defects, four-brush defects are symmetry-impeded in biaxial phases. 7 Albeit the number of four-brush defects is lower in the nematic glass at room temperature than in the fluid mesophase at high temperatures, four-brush defects are clearly present in both (Figure 1); the mesophase is thus uniaxial at all temperatures.

The highly delocalized pi electron system suggests a high birefringence Δn, even in the absence of polarizing terminal groups such as cyano (-CN) or isothiocyanato (-NCS), commonly employed in small molecule nematic liquid crystals. Indeed, upon cooling a 4μm ± 0.5μm thick cell from 250°C down to the onset of crystallization, the birefringence color runs through three color cycles to reach the pale green-yellow beyond thirdorder red before crystallizing at 170°C, corresponding to a birefringence of Δn = 0.45 ± 0.1, 8 a value typically obtained with nematic isothiocyanate mixtures optimized for high birefringence applications. 9 The extended and rigid, yet apolar conjugated pi electron system of 4 thus gives rise to a birefringence that reaches the values of highly polar materials designed for maximum birefringence.

To explore whether the direction of polarization can be homogeneously and macroscopically orientated, we needed an aligning anisotropic surface that is stable and maintains its anisotropy and aligning effect at temperatures close to 250°C, in the vicinity of the phase transition temperatures of 4 where annealing is most efficient. Particularly temperature-resistant aligning surfaces are known to result from rubbing teflon onto heated glass substrates. 10,11 We confirmed that teflon-coated glass plates obtained by rubbing a teflon bar on glass substrates heated to 150°C maintain their microgroove surface structure upon annealing at 260°C (see supporting information). Compound 4 was sandwiched between two teflon-coated glass plates with parallel rubbing directions. After heating the sample to the isotropic liquid at 250°C and then cooling into the nematic phase at 240°C, a homogeneous planar texture was obtained. This uniform alignment could be preserved on fast cooling to room temperature, yielding uniformly oriented birefringent glassy nematic films, whereas slow cooling led to spherulitic crystallization (Figure 4-top). A perfect 90° twisted glassy nematic waveguiding film could be produced with similar ease when a sample between teflon-coated glass plates with perpendicular rubbing directions was cooled quickly from 250°C to room temperature (Figure 4-bottom). These films confirm that the homogeneous director orientation of highly birefringent 4 can be frozen in unidirectional or helical geometries in its nematic glass state. To assess the effect of homogeneous anisotropic orientation on the fluorescence emission, we compared the emission from solution, from a spin-coated solid film and from the homogeneously teflon-aligned nematic glass (Figure 5). Whereas the absorption maxima in solution (334 and 347 nm) and in the spin-coated film (337 nm) are very similar, the emission shifts from a dominant peak in solution at 451 nm to a much broader two-peaked emission spectrum in the solid (479 and 544 nm in the spin-coated film, 475 and 550 nm in the nematic glass). The absorption of the nematic glass is identical to the spin-coated film. The absorption and emission broadening in the solid and the nematic glass compared to the solution, together with the red shift of the emission, testifies to molecular aggregation. 12 The evolution from a unique shorterwavelength peak in solution via two near-equally intense peaks in the solid film to a pronounced dominance of the longerwavelength emission in the nematic glass strongly suggests that intermolecular aggregation with close chromophore contacts and excimer formation is most pronounced in the nematic glassy state with its supposed parallel alignment of chromophores, whilst being less pronounced in the spin-coated solid film and absent in chloroform solution. The absorption and emission in solution as a function of concentration were measured to investigate the origin of the dominant peak around 550 nm for the nematic glass (Figure S2 in supporting information). The appearance of a new absorption shoulder around 425 nm for concentrations higher than 510 -3 mol L -1 demonstrates that ground state dimers are formed. For concentrations close to 10 -2 mol.L -1 the peak around 550 nm is already visible, becoming dominant for concentrations close to 10 -1 mol.L -1 , and the shorter-wavelength peak at 475 nm is red-shifted compared to more dilute solutions. The appearance of a new fluorescence band without vibronic structure at much higher wavelength than the monomeric band observed in dilute solution and its increase with concentration agree with a dimeric origin. The lifetime  of the excited state of 4 was measured on the planar aligned nematic glass to probe the emitting species separately at 475 nm and 550 nm. The fluorescence decay curves could be satisfactory fitted only with bi-exponentials, with the main (>90% contribution) component having a lifetime 1 of 1.3 ns at 475 nm and of 3.6 ns at 550 nm, which indicates different emitting species (Table 1, Figure S3). Excimer formation is in most cases driven by the non-covalent interaction between excited π-π stacked molecules, and emission lifetimes of excimers are usually longer than those of monomeric states. 13,14 We thus conclude that the fluorescence of the nematic glass has a contribution of monomeric states at 475 nm and of aggregate states (dimer and excimer) centered at 550 nm. Surprisingly, the emission from the planar aligned nematic glass is much more strongly polarized at shorter than at longer wavelengths, with the dichroic ratio (between emission polarized parallel and perpendicular to the alignment direction) being 4.2:1 at 475 nm, and dropping to about at the longwavelength peak at 550 nm. The lateral aggregation manifested by the longer-wavelength peak leads to a less uniaxial character of the excited (emitting) state compared to the emission from strongly elongated and uniformly aligned individual chromophores at shorter wavelength. The theoretical modelling of the isolated molecules 2-5 and its dimers yields the same pattern for the observed photophysical properties. By using TD-DFT, we computed the anisotropy of the transition dipole moment from the ground to the first excited state, which is related to the emission spectra and the birefringence. The predicted vertical energy difference between S0 and S1 for the isolated molecules and for their dimeric aggregates are coherent with what was observed, with the aggregates having smaller energy differences (Table S1). The anisotropy of the transition dipole moment, here measured as the ratio of its z and y components (along the ribbon axis and perpendicular to the aromatic plane, respectively), accordingly is much higher for the monomers than for the dimers, and has a strong polarization on the z axis along the aromatic scaffold (Figure S4 has a full definition of the axes). This is consistent with the observed large birefringence and dichroic ratio. Figure 6 shows the difference in densities between the S0 and S1 states of 4, with the long chains substituted by a methyl, and its dimer (see Figures S5 andS6 for all molecules). During the electronic excitation (or deexcitation in fluorescence), there is a large electron density shift along the z axis for the monomer.

That is somewhat reduced in the dimer, because parts of the density go along the y direction, perpendicular to the aromatic rings and into the neighboring molecule. That diminishes the magnitude of the transition dipole vector and correlates directly with the lower emission rate and longer lifetime observed. [START_REF] Turro | Modern Molecular Photochemistry of Organic Molecules[END_REF][START_REF] Souza | [END_REF] It appears thus that from the properties of the isolated molecules alone, a reason for 4 being the only material presenting a mesophase cannot satisfactorily be inferred. Whilst the simulations of 2, with a more isotropic pattern of electron density shifts upon excitation (at least to the S1), may explain its disproportionately low polarizability compared to 3-5, the electric properties (dipole moment, quadrupole moment and polarizability) of these three longer homologs grow as expected with length and start to saturate at some point, without anything specific to 4 (Figures S7-S9 and Tables S2-S4). This suggests that the propensity to mesophase formation is attributable to a complex combination of short-and mediumrange intermolecular interactions in the bulk.

Conclusions

In summary, extended twelve-ring graphene nanoribbon segments that are appropriately decorated with solubilizing lateral chains self-assemble in parallel to form a strongly anisotropic nematic glass. This amplification of molecular anisotropy into an orientable bulk fluid anisotropic structure, whilst being a trivial feature of liquid crystals, is highly unexpected for graphene nanoribbon segments given their strong tendency to aggregate in positionally ordered π-stacked structures. Uniform macroscopic anisotropic alignment is conveniently obtained in twisted and straight configurations at ambient temperatures, showing high birefringence as well as polarized fluorescence as manifestations of strong anisotropic polarizability, a key prerequisite for graphene nanoribbon optoelectronics.

Notes and References

Theoretical Methods

The geometry optimization of 2-5 was carried out in vacuum, using the Orca 4.1 1 software package and DFT with the BP86 functional 2,3 and the DEF2-SVP basis set for all atoms. [4][5][6] Dispersion effects were included using Grimme's D3 correction with Becke-Johnson (BJ) damping. [7][8][9][10][11] The RIJ algorithm was employed to accelerate the evaluation of the functionals using the resolution of identity approximation for the Coulomb part (RIJ). The vibrational frequencies computed on the optimized geometry of 2-5 included only small negative frequencies (<30 cm -1 ). Their analogues with only methyl moieties instead of longer aliphatic carbon chains (2C-5C), or without any ester substituents at all (pure polycyclic arenes, 2A-5A) and their dimers were also optimized, using the same calculation protocol, but their vibrational frequencies were not verified. All the electric properties were obtained using the same calculation protocol under the expectation value formalism, using a more complete DEF2-TZVP basis for all atoms. [4][5][6] Time-dependent density functional theory (TD-DFT) under Tamn-Dancoff approximation 12 (TDA) was employed to obtain the first 10 singlet excited states. Images of the complex geometries were obtained using the Chemcraft program. Table S2. Calculated isotropic polarizability and its tensor components (in atomic units) for 2-5 and their methyl ester and unsubstituted analogues, at the BP86/def2-TVZP theory level. Compounds 6-7 are similar to 5, but extending the phenacene ribbon by one (6) or two (7) Table S3. Calculated isotropic quadrupole and its tensor components (in atomic units) for 2-5 at the BP86/def2-TVZP theory level.

Molecule

Differential scanning calorimetry of compound 3

Figure S10. DSC traces for homologue 3 (that melts at a similar temperature but contrary to 4 shows no nematic phase) on cooling from the isotropic liquid (blue) and subsequent reheating (red), with transition enthalpies and temperatures.

Figure 1 .

 1 Figure 1. Microscopic textures of 4 between untreated glass plates (POM with crossed polarizers): Top left: Melting needle-shaped crystals at 247°C. Top right: Nematic Schlieren texture at 240°C showing four-brush defects (highlighted by arrows) characteristic of a nematic phase of uniaxial symmetry. 7 Bottom left: Nematic texture with spherulitic crystals growing in the bulk upon waiting at 160°C. Bottom right: Nematic Schlieren texture after fast cooling to room temperature, showing four-brush defects (highlighted by arrows).

Figure 2 .

 2 Figure 2. Evolution of the powder X-ray diffraction pattern of a bulk sample of 4 after fast cooling from 250 to 0°C. The sample is first reheated stepwise to 250°C and then cooled stepwise to 150°C (in chronological order from top to bottom, data acquisition time at each temperature: 1 h).

Figure 3 .

 3 Figure 3. DSC traces for 4 on cooling from the isotropic liquid (blue) and subsequent reheating (red), with transition enthalpies and temperatures. The inset shows the nematic-isotropic transition with reheating after cooling only to just below the transition to avoid crystallization prior to reheating.

Figure 4 .

 4 Figure 4. Microscopic textures of 4 between teflon-coated surfaces (POM) after fast (left and center) or slow (right) cooling from 250°C to room temperature. The arrows indicate polarizer and analyzer orientations. Top: uniformly aligned cell; rubbing direction parallel to one polarizer (left) and rubbing direction oblique to both polarizers (center and right). Bottom: 90° twist cell with rubbing directions perpendicular to each other, between parallel (left) and crossed polarizers (center and right).

Figure 5 .

 5 Figure 5. Left: Normalized absorption and emission spectra of 4 in chloroform solution (dotted) and spin-coated solid film (continuous), and emission in planar Teflon-aligned nematic glass (dashed). Emission spectra upon excitation at 347 nm (solution) or 335 nm (solid film and nematic glass). Right: Polarized emission spectra (arbitrary units) of planar aligned glassy nematic sample parallel (long dashes) and perpendicular (short dashes) to the alignment direction, and their dichroic ratio (continuous).

Figure 6 .

 6 Figure 6. Difference in electronic density between the S0 and S1 states for a model of molecule 4 and its dimer, calculated using BP86/DEF2-TZVP. The blue represents a depletion while the red an increase in electronic density. Hydrogens are omitted for clarity.

Figure S1 . 1 Figure S3 .

 S11S3 Figure S1. AFM images of the Teflon treated surfaces. Top: as rubbed; Bottom: after annealing annealing at 260 °C for 30 s. The left images are cross sections indicated by the green solid line.

Figure S4 .

 S4 Figure S4. Definition of Cartesian axis used in all calculations: the z axis is along the phenacene ribbon and the y axis is perpendicular to the aromatic plane.

Figure S5 .

 S5 Figure S5. TD-DFT difference densities between ground and first excited states of 2-5 calculated using BP86/def2-TZVP. Hydrogens and aliphatic carbon chains were omitted for clarity. Blue indicates decreased and red indicates increased electronic density.

Figure S6 .Figure S7 .

 S6S7 Figure S6. TD-DFT difference densities between ground and first excited states of the dimers of 2C-5C calculated using BP86/def2-TZVP. Hydrogens were omitted for clarity. Blue indicates decreased and red indicates increased electronic density.

Figure S8 .

 S8 Figure S8. Calculated isotropic quadrupole and its tensor components (in atomic units) for 2-5 at the BP86/def2-TVZP theory level.

Figure S9 .

 S9 Figure S9. Calculated isotropic polarizability and tensor components for 2-5 at the BP86/def2-TVZP theory level.

Table 1 .

 1 Excited state lifetimes () obtained from bi-exponential fittings of the fluorescence decay curves, their relative amplitudes (A), and fitting factors ( 2 ) upon excitation at 475 and 550 nm.

	 / nm	1 / ns	A1 / %	2 / ns	A2 / %	 2
	475	1.3 ± 0.1	95	6.0 ± 0.1	5	0.998
	550	3.6 ± 0.2	92	11.3 ± 0.2	8	0.999

Table S1 .

 S1 Data for the TD-DFT vertical excitations using BP86/def2-TZVP on the optimized ground state geometries. The transition dipoles are in atomic units.

	Molecule

Theo. / eV a Exp. / eV b |<I||F>| 2 |<I|x|F>| 2 |<I|y|F>| 2 |<I|z|F>| 2 |z/y|

				Monomers
	2	2.61		2.9474	0.1712	-0.0639	1.7070	26.7136
	3	2.55		2.1576	0.0419	0.0463	1.4675	31.6954
	4	2.46	2.61	2.3715	-0.1350	0.0009	1.5340	1704.4444
	5	2.42		1.5417	0.1021	0.0539	1.2362	22.9350
					Dimers
	2C	2.23		0.8240	0.1611	-0.1545	-0.8798	5.6945
	3C	2.19		0.2240	-0.2405	-0.1611	0.3744	2.3240
	4C	2.06	2.25	0.0949	-0.1556	-0.0545	0.2602	4.7743
	5C	2.06		0.0125	0.0755	-0.0123	-0.0815	6.6260

a. All of the S1 states could be described simply from a HOMO-LUMO transition. b. From λmax in solid state.

  benzo (i.e. C4H2) unit(s) on each end.

	Molecule	xx	yy	zz	Isotropic α
			Complete molecules
	2	656.26	359.42	1189.98	735.22
	3	1029.29	576.67	1770.62	1125.52
	4	1102.71	633.90	2136.97	1291.19
	5	1212.18	639.93	2555.86	1469.32
			Methyl ester analogues
	2C	537.94	363.00	1144.87	681.94
	3C	668.46	387.72	1496.32	850.83
	4C	769.61	386.82	1915.73	1024.05
	5C	818.01	452.21	2331.74	1200.65
	6C	1051.17	569.73	2496.14	1372.35
	7C	1116.09	736.35	2751.34	1534.59
		Unsubstituted arene analogues
	2A	401.79	179.37	1028.37	536.51
	3A	480.18	231.78	1393.40	701.79
	4A	582.03	259.51	1779.69	873.75
	5A	658.21	340.74	2150.19	1049.71
	6A	667.69	297.79	2183.43	1049.63
	7A	749.78	470.17	2464.68	1228.21

Table S4 .

 S4 Calculated total dipole moment (in atomic units) and its magnitude for 2-5 at BP86/def2-TVZP theory level.

	Molecule	µx	µy	µz	Magnitude
	2	0.02718	0.07705	0.15424	0.17454
	3	0.44996	0.4736	0.18979	0.68028
	4	0.59137	0.21987	0.29075	0.69469
	5	0.66273	0.25474	0.3593	0.79574

CAPES, FAPESC, INCT-INEO, H2020-MSCA-RISE-2017 (OCTA, #778158) and CAPES-COFECUB (Ph-C 803-14) for financial support.

Supporting information

General

Spin-coated films of 125 nm thickness with a mean surface roughness of less than 2.0 nm were produced by spinning a solution of 10 mg/mL in chloroform at 1500 rpm for 30 s. The thickness and roughness of the spin-coated films were determined with a Nanosurf Flex AFM, operating in tapping mode under ambient conditions with a scanning rate of 1.0 Hz and 512×512 pixels. Excited state lifetimes were measured at room temperature in air with a DeltaFlex TCSPC system from Horiba using LED excitation at 357 nm and emission collection at 475 nm and 550 nm.

Atomic force microscopy images of the Teflon treated surfaces

The surface micro-grooves are not significantly affected by annealing the film at 260 °C for 30 s.


Nikos Parlavantzas
email: nikos.parlavantzas@irisa.fr

Linh Manh Pham
email: linhmp@vnu.edu.vn

Christine Morin

Sandie Arnoux

| Gaël Beaunée

| Luyuan Qi

Philippe Gontier

Pauline Ezanno

A Service-based Framework for Building and Executing Epidemic Simulation Applications in the Cloud

Keywords: Cloud computing, High Performance Computing, Simulation models, Epidemic simulation

The cloud has emerged as an attractive platform for resource-intensive scientific applications, such as epidemic simulators. However, building and executing such applications in the cloud presents multiple challenges, including exploiting elasticity, handling failures, and simplifying multi-cloud deployment. To address these challenges, this paper proposes a novel, service-based framework called DiFFuSE that enables simulation applications with a bag-of-tasks structure to fully exploit cloud platforms. The paper describes how the framework is applied to restructure two legacy applications, simulating the spread of bovine viral diarrhea virus and Mycobacterium avium subspecies paratuberculosis, into elastic, cloud-native applications. Experimental results show that the framework enhances application performance and allows exploring different cost-performance trade-offs while supporting automatic failure handling and elastic resource acquisition from multiple clouds.

INTRODUCTION

As global transport networks continue to expand, the risk of infectious disease transmission grows, making research in pathogen spread on humans and animals increasingly important. An essential tool for studying pathogen spread is computer simulation based on mechanistic modelling. However, running epidemic simulations at large scale is time consuming for multiple reasons. First, the simulation models can capture fine-grained dynamics, considering different population scales (individual, population, meta-population) and accounting for multiple layouts (e.g., combining population and infection dynamics with farm management decisions) . [START_REF] Eriksson | Dynamic Multicore Processing for Pandemic Influenza Simulation[END_REF] Second, some processes can be data-driven and exploit large amounts of real-life observed data, mostly concerning the population dynamics of hosts. Third, understanding the underlying biological system requires performing and comparing many simulation scenarios, e.g., through a model sensitivity analysis. Finally, producing reliable results demands large numbers of simulation runs when models are stochastic.

Speeding up the execution of large-scale simulations requires parallelism. Parallelism can be implemented at the single-computer level using multiple cores and processors as well as the distributed level using multiple computers interconnected through a network. There are currently three main options for obtaining the necessary computing resources for running simulations in parallel. The first option is using dedicated, high-end multicore systems or clusters. The limitation is that such infrastructures are typically shared among many local users, resulting in long waiting times.

At the same time, extending such infrastructures to increase user satisfaction is time consuming and costly. The second option is grid computing, which allows sharing computers from multiple clusters in multiple sites. The main limitation of grids is the heterogeneity of the software stacks in each cluster, which makes application deployment difficult. Moreover, grid-based applications are assigned a statically-fixed number of resources, leading to poor resource utilisation or reduced performance in the face of dynamic variations in resource demand.

The third option for obtaining computing resources is cloud computing, which enables users to rapidly obtain and release practically unlimited amounts of resources while being charged only for actual resource usage. Thanks to virtualisation, cloud computing enables applications to have their own software stacks and to run reliably in various environments. Cloud computing thus overcomes the limitations of the previous two options, bringing immediate economic and agility benefits, and lowering the barrier to running complex simulations. Despite the attractiveness of cloud computing, converting epidemic simulation applications to become cloud-native applications presents several challenges.

A main challenge is facilitating the deployment and operation of simulation applications. Indeed, using clouds requires considerable manual effort for obtaining cloud resources, configuring and launching multiple runs of the simulation on these resources, making input data available, and collecting and processing simulation results. This effort becomes more complex when the application is deployed on multiple clouds, which is useful for avoiding dependence on a single vendor, enhancing availability, and taking advantage of lower resource prices or specialised resource capabilities. Another challenge is exploiting cloud elasticity to optimise the performance and cost of simulation applications. Indeed, simulation applications may require different amounts of resources at different times. For instance, the initialisation phase may require fewer compute resources than the main processing phase, thus making it more efficient to scale up the resource allocation at run time. A third challenge is handling failures, which are unavoidable when executing long-running, distributed applications, such as epidemic simulators, on the cloud. Handling failures is essential for avoiding data loss and ensuring the correct and efficient completion of the simulation.

To address these challenges, this paper presents a framework, called DiFFuSE, that enables simulation applications to fully exploit cloud platforms. The framework assumes that the core of the application's computation takes the form of a bag of independent tasks, in which tasks are simulation runs and can be executed in parallel. Specifically, the paper makes two main contributions. First, it describes the DiFFuSE framework for building and executing pathogen spread simulators. The main novelty of DiFFuSE lies in its service-based architecture; simulation applications are decomposed into services that can be deployed across multiple clouds and independently scaled, while the framework transparently handles service failures. Second, the paper presents the application of DiFFuSE to building cloud-native simulators of the spread of bovine viral diarrhea virus (BVDV) and of Mycobacterium avium subsp. paratuberculosis (MAP) and uses these applications to evaluate the benefits of the DiFFuSE framework. The paper completes and extends our previous work on the DiFFuSE framework. [START_REF] Pham | a Distributed Framework for Cloud-Based Epidemic Simulations: A Case Study in Modelling the Spread of Bovine Viral Diarrhea Virus[END_REF][START_REF] Pham | a Distributed Framework for Cloud-based Epidemic Simulations: a Case Study in Modelling the Spread of Bovine Viral Diarrhea Virus[END_REF] Specifically, the paper provides further details on the operation of the framework, including the allocation of simulation runs and failure handling. Furthermore, the paper adds the case study of the MAP application and reports new experiments to provide a thorough evaluation of the framework using both case studies.

The rest of this paper is organised as follows. Section 2 discusses related work. Section 3 briefly presents the BVDV and MAP case studies, the simulations used throughout the paper as practical examples. Section 4 details the architecture of DiFFuSE and its application in the case studies.

Section 5 reports a set of experiments using the case studies, and Section 6 concludes the paper.

RELATED WORK

This section examines related research that focuses on exploiting large amounts of computing power to execute epidemic simulations.

Eriksson et al. [START_REF] Eriksson | Dynamic Multicore Processing for Pandemic Influenza Simulation[END_REF] use OpenMP to parallelise pandemic simulations, taking advantage of multi-core processors. The evaluation demonstrates that performance can be improved by dynamically switching between single-and multi-core modes as the computational load varies. This work does not consider parallelisation at the distributed level. Bhatele et al. [START_REF] Bhatele | Massively Parallel Simulations of Spread of Infectious Diseases over Realistic Social Networks[END_REF] introduce EpiSimdemics, a highly scalable, parallel code written in Charm++ using Zou et al. [START_REF] Zou | Epidemic simulation of a large-scale social contact network on GPU clusters[END_REF] propose using GPU clusters to implement large-scale, contact-network based epidemic simulations. The paper describes optimisation techniques for improving the efficiency of memory accesses and reducing the communication latency between compute nodes. The techniques are tested on a cluster whose compute nodes are equipped with GPUs, and experimental results show that the execution on GPUs can achieve 7.4 to 11.7 times speedup over the execution on CPUs. Holvenstot et al. [START_REF] Holvenstot | GPGPU parallelization of self-calibrating agent-based influenza outbreak simulation[END_REF] use general-purpose GPU devices to accelerate an agent-based simulation of influenza spread. Experimental results show that the GPU implementation achieves significantly greater speedups than a multi-threaded CPU implementation. Although the previous solutions demonstrate the performance benefits of using GPUs, they require purchasing expensive, highend hardware rather than using clouds, which currently provide GPU-based resources with low prices.

Eriksson et al. [START_REF] Eriksson | A Cloud-Based Simulation Architecture for Pandemic Influenza Simulation[END_REF] present a cloud-based approach for performing simulations of pandemic influenza. The approach relies on the HTCondor architecture in which a central master schedules and distributes jobs and a set of workers receive and execute the jobs. The approach allows adding workers by renting Amazon EC2 virtual machines (VMs). The tests show that the network capacity of the master node can become a bottleneck when the number of workers increases. The authors discuss possible solutions, such as compressing data or choosing nodes connected to high-speed networks. Unlike our work, this work does not exploit elasticity to dynamically handle increases in resource demand.

Sukcharoen et al. [START_REF] Sukcharoen | Design and analysis of a cloud-based epidemic simulation framework[END_REF] propose a cloud-based framework for simulating the spread of epidemic diseases. The framework applies loop decomposition to parallelise the simulation on multiple VMs of a private cloud, built using the Xen Cloud Platform. This framework is restricted to a specific epidemic model, namely, a modified version of the SEIR model (Susceptible, Exposed, Infectious, Recovered) and does not support elasticity. Price et al. [START_REF] Price | SaTScan on a Cloud: On-Demand Large Scale Spatial Analysis of Epidemics[END_REF] present an approach for executing a compute-intensive application for epidemic analysis on cloud resources. The approach uses the Nimbus cloud to obtain resources on demand. This work considers neither elasticity nor the cost of using cloud resources. Haris and Manzoor 14 present a cloudbased framework for simulating the spread of the dengue virus in Pakistan. This framework does not support parallelisation at single-computer level and does not exploit the elasticity of cloud resources.

Work focusing on executing bag-of-tasks applications on the cloud is also related to ours. Benedictis et al. [START_REF] Benedictis | A Framework for Cloud-aware Development of Bag-of-tasks Scientific Applications[END_REF] present a framework, called BOT, for developing bag-of-tasks scientific applications on the mOSAIC cloud platform. Similarly to our work, BOT can be used on the resources of any cloud providers, and it includes strategies for fault-tolerance. BOT was evaluated using an application that solves a classical combinatorial optimisation problem (binary knapsack). Agarwal and Prasad 16 present a similar framework for developing bag-of-tasks applications, called AzureBOT. However, this framework targets exclusively a single cloud, that is, the Azure cloud. AzureBOT was evaluated using two applications, an Internet data scraper and a master-slave simulator. Krol et al. [START_REF] Król | A Cloud-Based Data Farming Platform for Molecular Dynamics Simulations[END_REF] discuss a data farming platform, called Scalarm, which relies on the master-worker pattern, and presents its extension for running a molecular dynamics simulation on clusters and on clouds. The evaluations of BOT, AzureBOT, and Scalarm focus on application performance and cost without considering fault-tolerance and elasticity. In addition, unlike our work, frameworks for bag-of-tasks or master-worker applications lack specialised facilities for building simulation applications, making it complex and inefficient to use them for real-world applications. Lacking facilities include managing simulation experiments, coordinating simulation runs, replicating simulation data for resilience and performance, and elastically scaling the computation and data transfer capacities involved in running the simulation.

Much research has focused on algorithms for scheduling bag-of-tasks applications on cloud resources, aiming to optimise application performance or monetary cost. [START_REF] Thai | A survey and taxonomy of resource optimisation for executing bag-of-task applications on public clouds[END_REF] Similarly, such research does not provide a complete solution for building epidemic simulation applications.

Nevertheless, such research can form a basis of optimising task scheduling in our work, which currently only applies basic scheduling policies.

To summarise, some of the related work for executing epidemic simulations does not exploit clouds and their associated parallelisation, cost, and agility benefits. [START_REF] Eriksson | Dynamic Multicore Processing for Pandemic Influenza Simulation[END_REF][START_REF] Bhatele | Massively Parallel Simulations of Spread of Infectious Diseases over Realistic Social Networks[END_REF][START_REF] Martin | Towards Efficient Large Scale Epidemiological Simulations in EpiGraph[END_REF][START_REF] Perumalla | Reversible Parallel Discrete-Event Execution of Large-Scale Epidemic Outbreak Models[END_REF][START_REF] Rao | Parallel simulation of the global epidemiology of Avian Influenza[END_REF][START_REF] Bisset | Indemics: An Interactive High-performance Computing Framework for Data-intensive Epidemic Modeling[END_REF][START_REF] Zou | Epidemic simulation of a large-scale social contact network on GPU clusters[END_REF][START_REF] Holvenstot | GPGPU parallelization of self-calibrating agent-based influenza outbreak simulation[END_REF] Other related work exploits clouds but lacks support for elasticity, [START_REF] Eriksson | A Cloud-Based Simulation Architecture for Pandemic Influenza Simulation[END_REF][START_REF] Sukcharoen | Design and analysis of a cloud-based epidemic simulation framework[END_REF][START_REF] Price | SaTScan on a Cloud: On-Demand Large Scale Spatial Analysis of Epidemics[END_REF][START_REF] Haris | Spatiotemporal Study of Dengue Virus Infection via Cloud Based Framework[END_REF] multi-cloud deployment, [START_REF] Agarwal | AzureBOT: A Framework for Bag-of-Tasks Applications on the Azure Cloud Platform[END_REF] , fault tolerance, [START_REF] Król | A Cloud-Based Data Farming Platform for Molecular Dynamics Simulations[END_REF] or required facilities beyond bag-of-tasks execution. [START_REF] Benedictis | A Framework for Cloud-aware Development of Bag-of-tasks Scientific Applications[END_REF][START_REF] Thai | A survey and taxonomy of resource optimisation for executing bag-of-task applications on public clouds[END_REF] To the best of our knowledge, our work is the first to propose a complete, practical solution for building and executing simulation applications, structured as bags of tasks, across multiple clouds with support for elasticity and fault-tolerance.

CASE STUDIES: BVDV AND MAP SPREAD SIMULATORS

This section discusses the use of simulation for studying two diseases, bovine viral diarrhea and bovine paratuberculosis. These are endemic diseases in cattle, caused respectively by a virus (Bovine Viral Diarrhoea Virus, BVDV) and a bacteria (Mycobacterium avium subsp. paratuberculosis, MAP), that lead to economic losses owing to reproductive disorders, decrease in milk production as well as increases in morbidity and mortality among infected animals. [START_REF] Ezanno | Sensitivity analysis to identify key-parameters in modelling the spread of bovine viral diarrhoea virus in a dairy herd[END_REF][START_REF] Garcia | Invited review: The economic impact and control of paratuberculosis in cattle[END_REF] To identify the main drivers of the spread of the diseases in a region and to compare a large range of regional control strategies, two distinct original stochastic metapopulation models have been proposed. They are both discrete-time compartment models based on characteristics of dairy cattle herds in Brittany.

The model of BVDV spread accounts for the two main transmission pathways between herds: trade movements of animals carrying the virus and virus transmission during the grazing season due to over-the-fence contacts with animals from neighbouring herds. [START_REF] Qi | Modeling bovine viral diarrhea virus (BVDV) spread between dairy cattle farms at a regional scale: relative contribution of two between-herd transmission pathways[END_REF] In addition, the model accounts for the local within-herd virus spread, using as a building block a previous within-herd model. [START_REF] Ezanno | Sensitivity analysis to identify key-parameters in modelling the spread of bovine viral diarrhoea virus in a dairy herd[END_REF][START_REF] Ezanno | Influence of herd structure and type of virus introduction on the spread of bovine viral diarrhoea virus (BVDV) within a dairy herd[END_REF] Practically, the simulation uses a 2 GB database describing cattle life trajectories in Brittany from 2005 to 2013. This database provides information on the demography of dairy animals (e.g., birth, death, culling rates) and trade movements between farms (e.g., holding farms, dates and reasons of entry/exit). The metapopulation model represents the spread of BVDV within and between 12,750 dairy cattle herds located in Brittany and takes into account 1,056,565 animals movements over the 9-year period.

Resource Coordination Cloud Deployment and Execution

Data Services

Computing Services Experiment Services

Epidemic Simulation Services

FIGURE 1 Architecture of DiFFuSE

The model of MAP spread [START_REF] Beaunée | Modelling of paratuberculosis spread between dairy cattle farms at a regional scale[END_REF][START_REF] Beaunée | Controlling bovine paratuberculosis at a regional scale: Towards a decision modelling tool[END_REF] accounts only for between-herds spread through animal trade movements, which represents the main pathway.

The regional model consists in coupling numerous (one per farm) stochastic within-farm epidemiological models [START_REF] Marcé | Within-herd contact structure and transmission of Mycobacterium avium subspecies paratuberculosis in a persistently infected dairy cattle herd[END_REF] through cattle trade movements.

Practically, for both animal movements and farming management, pre-processed real observed data (extracted from the same database used for the BVDV model) are plugged into the model. The metapopulation model represents the spread of MAP within and between 12,857 dairy cattle herds located in Brittany, and takes into account 919,304 animals movements over the 9-year period.

These complex simulations pose the challenges of performance and operational costs, already discussed in Section 1 for generic epidemic simulations. Indeed, ensuring accurate predictions requires performing from 200 to 500 repetitions, which is time-consuming and highly susceptible to failures from the human or machine side. Moreover, researchers have to perform simulations for many possible spreading scenarios and carry out sensitivity analyses for many model parameters. As a result, performing such simulations remains beyond the budget of small and medium laboratories and research groups. The BVDV and MAP case studies will be used in demonstrating and evaluating the DiFFuSE framework.

DiFFuSE FRAMEWORK AND APPLICATIONS

Fig. 1 shows the architecture of the DiFFuSE framework, which is structured in three layers. Seen from the bottom up, the layers are increasingly specific to the domain of epidemic simulations. At the bottom is a cloud deployment and execution platform, capable of running any modular application across clouds. Next is the resource coordination layer that supports building services that exchange data using a specific set of provided mechanisms. At the top is the epidemic simulation services layer that supports building epidemic simulation applications as sets of cooperating services. The rest of this section provides details on each layer and on applying the framework in the case studies.

Cloud Deployment and Execution

The aim of this layer is to hide the intricacies of cloud technologies (e.g., diverse cloud providers, interfaces, services). Specifically, the layer facilitates deploying an application across multiple clouds and automatically adapting this deployment, when needed (e.g., elastically scaling components to handle workload variations). The layer assumes that applications are structured as interconnected, deployable components and relies on the PaaSage platform 26,[START_REF] Parlavantzas | Cost-Effective Reconfiguration for Multi-Cloud Applications[END_REF] , a holistic solution for automatic deployment and execution of cloud applications.

Briefly, the PaaSage platform operates as follows. First, PaaSage receives as input descriptions of cloud providers (e.g., VM types, prices) and applications (e.g., components and interconnections, objectives), all written in CAMEL, the PaaSage application modelling and execution language.

PaaSage then decides how to best deploy the application on available cloud resources and performs the deployment through obtaining VMs,

Resource Coordination

The aim of this layer is to facilitate the coordination of distributed services, communicating over the network. Specifically, the layer enables services to exchange data in a reliable and scalable way. Each service is running as a separate process and can be deployed using the cloud deployment and execution layer discussed previously. Next, we discuss how the resource coordination layer supports data exchange and failure handling.

Data Exchange

The layer relies on the concepts of resource, resource manager, resource user, resource provider, and resource monitor. A resource is any data that can be named and exchanged between services (e.g., experiment configuration, experiment data). A resource manager owns and provides access to local resources and is identified by a URL. Resource managers can be linked in a hierarchy in which a resource manager can provide access to the resources of its parent resource manager.

A service can play three roles with regards to a resource: (1) resource user, which sends requests to a resource, demanding the associated data, (2) resource provider, which receives and replies to such requests, providing access to the resource data, and (3) resource monitor, which tracks the state of a resource (e.g., providers are present, providers have failed). A resource supports one of four management styles determining how data are distributed among providers:

• Single provider management: the resource has only a single provider offering all the data.

• Replicated management: the resource can have multiple providers, each offering the same data. A request to the resource is delivered to any of the providers. This management style can be used to support fault-tolerance; if one provider fails, the requests are handled by the remaining providers. Moreover, it can be used to improve the request response time through balancing the load among providers. A service can obtain the data from an available provider and become itself a provider.

• Individual management: the resource can have multiple, independent providers, each offering different data. A resource user can send requests to an identified provider to obtain the data.

• Collective management: the resource has a fixed set of providers, each offering a different part of the data. A resource user can obtain the data if all providers are available. If a provider fails, a notification is sent that allows reacting to this failure.

Resource coordination in DiFFuSE is implemented using three C++ classes: the resourceManager class owns local resources and interacts with its parent resource manager, the resourceLocalClient supports interacting with local resources, and the resourceClient class supports interacting with remote resources.

Failure Handling

Services are distributed and interdependent, which introduces many opportunities for failure:

• Deployment-order failures: services may fail when they try to interact with services that are not yet deployed.

• Cleanup failures: services may fail to release allocated resources when the overall computation ends.

• Service failures: services may fail due to hardware errors or bugs.

DiFFuSE handles deployment-order failures through using the nanomsg message library. [START_REF] Nanomsg | [END_REF] With classical TCP sockets, a connection to a service fails if the service is not already running. With nanomsg, the library transparently uses retry and timeout mechanisms, removing any dependencies on deployment order. Cleanup failures are avoided by ensuring that when a specified service service stops, all other services stop and cleanly release their resources. DiFFuSE handles service failures through the connections between each resource manager (except the root manager) and its parent resource manager. The services on both sides of such connections check that they periodically receive messages from the other side and take appropriate actions when failures are detected. For instance, if the resource manager detects the failure of a remote service that was a resource provider, the manager updates the resource state and notifies all resource users and resource monitors.

Epidemic Simulation Services

The aim of this layer is to facilitate building epidemic simulation applications as sets of cooperating services. Specifically, the layer provides reusable design and code for commonly-required functionality in such applications, such as controlling experiments, executing simulation runs, and managing the flow of simulation data between services. This functionality is provided as a collection of services, built on the resource coordination layer, that can be easily customised and used by developers. Next, we discuss the types of provided services and how they collaborate to execute the simulation, handle associated failures, and transfer data.

Provided Services

The simulation services fall into three main categories.

• Data Services: retrieve data from databases and provide these data as well as related meta-data to other services.

• Computing Services: perform the main computations of the simulation, either sequentially or in parallel, using single-computer parallelisation (e.g., using OpenMP). Computing services include services that execute simulation runs (called worker services), a service that generates final results (called final results service) as well as services that process these results.

• Experiment Services: control the life-cycle of experiments. They track the progress of the computations, they handle service failures, and they provide configuration data about the experiments to other services.

DiFFuSE provides three generic C++ classes that serve as templates for developing services of the previously-mentioned categories. Each service defines a routine executed when requests are received and instantiates a resource manager class that provides access to local and remote resources.

Allocation of Simulation Runs

NbRR=r ExeR=0 NbTh=n

Calculate AlloR p1

Worker i a)

Experiment Service

Failure Detected

Execute AlloR p1

Probe RR=NbR ExpR=0 This section discusses how the experiment service allocates simulation runs to worker services. The allocation assumes that simulation runs are independent, and thus that the overall computation has the structure of a bag of tasks. Moreover, the allocation follows a simple policy that requires only information on the number of threads supported by the node hosting a worker. Adding more complex policies that exploit richer information on the underlying nodes (e.g., memory size, network bandwidth) and on simulation runs (e.g., expected execution time) could improve application performance. However, the current paper does not focus on optimising performance but on providing a complete solution for running simulations across clouds in a flexible, scalable, and fault-tolerant manner.

The simple policy is shown in Figure 2a. At the beginning (t 0 ), a worker i sends a request message to the experiment service to ask for the number of runs to execute. The request contains a suggested number of runs (NbRR), the number of threads (NbTh) supported by the node hosting the worker, and the number of runs executed successfully by the worker after the previous allocation (ExeR). Typically, the NbRR parameter is set to the NbTh value in order to enable parallel execution of the runs; if NbRR is larger than NbTh, some runs will be queued at the worker. The ExeR parameter is used to calculate the cumulative number of executed runs for this worker (ExeR i ) at the experiment-service side.

The experiment service maintains the list of executed runs of all participating workers as well as cumulative variables, such as remaining runs (RR)

and expected runs (ExpR). RR is the total number of remaining runs to be allocated to workers. ExpR is the number of runs that were allocated, but the experiment service is still waiting for their results. Initially, ExpR is set to zero while RR is set to the number of runs required by the simulation (NbR).

When the experiment service gets the request from a worker, it calculates the number of runs allocated to the worker (AlloR) using Algorithm 4.3.2. The algorithm takes into account the number of runs suggested by the worker (NbRR), the total number of workers (nbOfWorkers) and the remaining number of runs (RR). The algorithm allocates a larger number of runs at the beginning, which decreases as the remaining number of runs decreases. The goal of this strategy is to have all workers finish their computations at approximately the same time. The experiment service then sends the value of AlloR in a reply message to the worker and updates the cumulative variables including the AlloR i variable, storing the number of runs allocated to that specific worker.

When the worker gets the reply message from the experiment service, it executes the runs allocated to it, and then sends another request message to the execution service. This process repeats until all the simulation runs are allocated. The next subsection includes a discussion of cases when workers fail to execute the allocated runs.

Handling Worker Failures

This section discusses failures of workers to execute their allocated runs; generic failure handling mechanisms were discussed in Section 4.2.2.

The workers are resource providers that provide the partial results resource. The management style of this resource is collective (see 4.2.1), which means that workers offer different parts of results data and any failure to provide the data must be handled. There are two situations when the collective results cannot be provided. The first is when some of the simulation runs allocated to a worker fail to be successfully executed (e.g., due to insufficient memory). In this situation (see Figure 2b), the worker subtracts the amount of unsuccessful runs (UR) from the number of allocated runs and sends the result to the experiment service (t 1 ), which updates the corresponding variables. Specifically, the amount of the unsuccessful runs is added to RR, and the amount of successful ones is added to ExeR i . The unsuccessful runs will then be reallocated to other workers or again to worker i . The second situation is when the worker service fails completely and thus all its executed runs from the beginning are lost. When the execution service detects this failure (see Figure 2c), it updates the corresponding variables. Specifically, the total number of lost runs of worker i is added to RR, and ExeR i is set back to zero. The lost runs will then be reallocated to other workers. If worker i is later revived, all its variables are set to zero. The worker will still be recognised by the experiment service thanks to a serial identification number. The worker will send requests to obtain simulation runs, and the process continues as before. Similarly with the allocation policy of Section 4.3.2, more optimised failure-handling policies could be added (e.g., using replication), but this is beyond the scope of the current paper.

Data Management

In general, services exchange data through resources, which also serve as the synchronisation mechanism. For instance, in the case of collective management, a service acting as a resource user must wait for all providers to be available before obtaining the data. To transfer or process data, services do not need to use disk storage, increasing application performance. In addition, services do not need to update data, avoiding consistency issues.

The typical data flow in an epidemic simulation application is as follows. A data service provides access to input data (e.g., animal demography) contained in a local file or database. If further services are configured as providers of the same resource (i.e., following the replicated management style), they contact the initial service and obtain the complete data in a serialised form over the network. When the data are available at all providers, the experiment service is notified, and the workers start receiving simulation runs, following the previously-described allocation policy. Next, the workers obtain data from the data services, as needed, and execute the allocated simulation runs. A special case is when workers obtain data from other workers acting as resource providers; this configuration can reduce data transfer times in multi-cloud environments, as demonstrated in Section 5.3.2. In executing the simulation runs, the workers produce partial results. The workers store these results in memory, acting as providers of the partial results resource (following the collective management style). If there are lost runs, these runs are re-executed, as seen in Section 4.3.3, and the corresponding partial results are reproduced. When all runs are successfully executed, all providers of the partial results resource are available. This enables the final results service to gather the partial results and produce the final results. These data can then be obtained by other services for further analysis or visualisation.

The BVDV and MAP spread simulators

The legacy BVDV and MAP spread simulators were transformed to cloud-based applications based on the DiFFuSE framework. Specifically, the original BVDV code (around 4600 lines of C++ code) was sequential and was first modified to add single-computer parallelism using OpenMP. The original MAP code (around 14000 lines of C++ code) already supported single-computer parallelism using OpenMP. In both cases, the services to be used in the target version were then identified and implemented by extending the generic C++ classes and integrating the existing code. Most of the effort was spent on implementing the data services, which required decomposing the original data model into separate pieces of data (e.g., herds, movements) managed by independent services. Implementing computation services required minor modifications for adapting the original simulation computation code and inserting it into the generic class. The experiment services were largely based on the generic class.

In total, 6.3% of the original BVDV code and 1.78% of the original MAP code were modified in order to convert them to cloud-based applications.

In the final DiFFuSE-based BVDV application, the proportion of the code that is specific to the BVDV spread simulation (rather than reusable code) is around 17.7%. In the final MAP application, this proportion is around 5.93%. These measures provide evidence of the limited effort required for applying DiFFuSE and of the added value of the DiFFuSE reusable code.

The services used in the DiFFuSE-based applications are shown in Fig. 3. The data services (Herd, Move and Neighbour) provide descriptions of herds, movements, and herd neighbours applying replication to improve failure tolerance and responsiveness. Each Worker retrieves descriptions from the data services and configuration data from Experiment and interacts with Experiment to compute partial simulation results. Final Result gets the partial results from workers and computes the final results, while Plot Final Result generates a graphical view. Finally, Experiment controls the life-cycle of an experiment.

The services are deployed through the cloud deployment and execution layer based on PaaSage. Figure 4 shows a possible deployment of the BVDV application in a multi-cloud environment.

EVALUATION

We conducted a set of experiments to evaluate DiFFuSE in terms of performance, cost effectiveness, failure handling and elasticity in single or multi-cloud environments. The BVDV and MAP spread simulations are used throughout the experiments as the reference applications.

We ran experiments with the BVDV application in three cloud environments: Amazon EC2, GWDG [START_REF] Gwdg | [END_REF] , and an OpenStack deployment on Grid5000. 30 research unit equipped with 13 cores and 100 GB memory. This server has the same configuration with the used VMs, but note that the legacy code uses only 1 core. The figure also shows that the performance increases when the number of workers rises from 4 to 9, and the effect is greater with increasing number of runs. The cost slightly increases when the number of workers increases from 4 to 9.

As a public cloud, we use Amazon EC2. As each run of the simulation needs at least 35 GBs of memory, we consider two scenarios, each using a different type of memory-optimised EC2 VM: r4.xlarge (4 cores, 30.5 GB RAM + 16 GB swap) and r4.2xlarge (8 cores, 61 GB RAM). In a third scenario, we execute an OpenMP-based BVDV spread simulator, not integrated into DiFFuSE, on a single EC2 VM of r4.2xlarge type. The results are also compared with a fourth, base scenario, which executes the legacy, sequential BVDV code on the previously-mentioned BioEpAR server. simulator with 4 workers of the r4.2xlarge type shows the best performance. In particular, the simulator using r4.2xlarge outperforms the one using r4.xlarge. The reason is that r4.xlarge does not have enough RAM (< 35 GB), thus having to consume its swap space in both data loading and transferring phases. This leads to 60% slower time in these phases than in the case of r4.2xlarge, according to our measurements. The simulator using r4.xlarge, the lower configuration, performs better than the legacy code in the base scenario for more than 32 runs. The OpenMP-based simulator shows the worst performance owing to the high parallelisation overhead for the given number of runs. Regarding cost, the OpenMPbased, single-node scenario has the lowest cost for up to 40 runs. The r4.2xlarge scenario costs more than the r4.xlarge scenario, except with 10 runs, where their costs are almost equal. The base BioEpAR scenario has the highest cost.

This experiment validates that using DiFFuSE reduces execution time and cost compared to the legacy BVDV simulator solution, thus enabling scientists to perform simulations previously considered as impractical or prohibitively expensive. Moreover, the experiment demonstrates the flexibility in the deployment of DiFFuSE-based applications, which can use different number of workers on different types of VMs in private or public clouds.

Experiment 2

This experiment evaluates the execution time and cost of running the DiFFuSE-based MAP simulation in various private and public cloud environments. As a public cloud, we use Google Compute Engine. As private clouds, we use two OpenStack deployments on Grid5000, one on nodes of the Paranoia cluster in Rennes and one on the Econome cluster in Nantes. In the private clouds, each worker service is packed into one VM with 20 cores and 100 GB memory. We use the price of the same VM type in Google Compute Engine as the reference price increased by 30% to reflect the typically higher unit costs of private clouds. We also assume that the price of VMs in the Rennes cluster is 5% higher than that of VMs in Nantes. As a base scenario, we use the execution of the legacy MAP code, parallelised using OpenMP, on a BioEpAR server equipped with 20 cores and 64 GB memory. This experiment validates the performance and cost advantage of using DiFFuSE compared to the legacy MAP simulator solution. Moreover, it provides further evidence of the supported deployment flexibility, which allows users to make different cost-performance trade-offs by controlling the number and types of used cloud resources.

Experiment 3

This experiment evaluates the impact of increasing the number of workers on the cost and execution time. Fig. 8 shows the execution time and cost of running the BVDV simulator with an increasing number of workers (from 1 to 4) for 5 and 20 runs. As expected, the execution time decreases and then levels off. The cost keeps increasing as the number of workers increases. Fig. 9 and 10 show the execution time and cost of running the MAP simulator with an increasing number of workers in a private and a public cloud. Again, the execution time decreases and then levels off. The cost typically increases, except when passing from 4 to 5 workers in Google Compute Engine. In this case, the effect of the reduced execution time on total cost is greater than that of the added VM.

This experiment validates that deploying DiFFuSE-based simulators in diverse cloud environments provides expected scalability properties, and thus that DiFFuSE can effectively exploit cloud resources to perform growing amounts of computation.

Failure Handling

Experiment 1

This experiment evaluates the failure handling capabilities of DiFFuSE. We use the DiFFuSE-based application for BVDV spread simulation with three workers packed in VMs of our OpenStack private cloud (see Section 5.1.1). In one scenario, one of the workers fails early and the others get the burden of redoing the runs until there are zero remaining runs. In the second scenario, the failed worker is revived and takes on further work, if the number of remaining runs is larger than zero. Before the recovery of the failed worker, the other workers recalculate the lost blocks and get other blocks from the experiment service if they are free. We set the NbRR parameter to 2, which means that a worker can receive a maximum of two simulation runs. The simulation is repeated 21 times.

Experiment 2

This experiment evaluates the management of additional types of failures. Specifically, it considers also situations where workers fail to execute a certain number of simulation runs without crashing completely. These are situations described in Figure 2b. The previous experiment considered only situations described in Figure 2c. The setup of the experiment is similar to that of the previous experiment (Section 5.2.1) with 21 simulation runs distributed to 3 workers in VMs of our OpenStack private cloud. We observe how the runs are reallocated in four scenarios: when a worker fails and never revives (WF), when a worker loses one run (L1) or two runs (L2) and when a worker fails completely but is later revived (RL). Specifically, the failed worker is worker 3 and failure/loss and recovery follow the same timing as in the previous experiment. The results of this experiment are shown in Table 1. Specifically, the table presents the number of runs allocated to each worker for every scenario and for different values of NbRR (2, 4 and 8). The table also shows the maximum number of runs allocated to a worker (MAX) which determines the execution time of the scenario (the lower the MAX, the better).

We observe that the value of MAX is best in scenarios L1 and L2 that achieve better distributions of runs. The value of MAX is not affected by the NbRR parameter. When the worker fails without revival (scenario WF), larger values of NbRR lead to worse values of MAX. The reason is that a greater number of runs has to be redistributed to surviving workers. Inversely, when the worker fails but recovers (scenario RL), larger values of NbRR lead to better values of MAX since the recovered worker obtains more runs to execute. The experiment validates that DiFFuSE can handle a wide range of failure scenarios.

5.3

Multi-Clouds

Experiment 1

This experiment evaluates the support provided by DiFFuSE for multi-cloud deployment. We deploy the services of the BVDV spread simulator in

From the table, we see that the experiment service distributes more blocks of runs to VMs with better configuration (e.g., 10 for ec2_m4_4xlarge and only 2 for gwdg_m2_xlarge). Still, the time to finish 10 runs in ec2_m4_4xlarge is not much different than the time to finish 2 runs in gwdg_m2_xlarge. The cost per run of ec2_m4_4xlarge is lower than that of gwdg_m2_xlarge (0.065$ and 0.108$). While ec2_m4_4xlarge and gwdg_m2_xlarge exploited about 40% to 45% of the potential capacity of their cores, ec2_r4_2xlarge used its cores more efficiently (64.15%).

This experiment validates the ability of DiFFuSE to simultaneously exploit diverse VMs from multiple clouds and shows that the choice of VM types has a strong impact on the resulting cost. Moreover, it validates the ability of the simple allocation policy (see Section 4.3.2) to balance the computation load among heterogeneous VMs.

Experiment 2

This experiment evaluates the impact of network performance on application execution time in multi-cloud deployments. In such contexts, components of the simulation application are deployed at different data centers, located at different areas and even different continents. As a result, network characteristics can have a strong effect on application performance.

The setup of this experiment is similar to Figure 4 with data services in our OpenStack-Grid5000 private cloud and worker services in the EC2 public cloud. The workers are configured to be in the same EC2 availability zone in order to reduce network latency. The data services provide data on animals, herds, movements and neighbours. When a worker is initialised, the worker obtains these data causing a traffic of about 2GBs from the private cloud to the public cloud over a wide-area Internet connection. This is repeated for every worker that joins the simulator, having a negative effect on the performance of the BVDV simulator.

To reduce this negative effect, one can take advantage of the replication feature supported by DiFFuSE. Specifically, the data resources provided by data services are configured to support the replicated management style and the worker services are configured to play both user and provider roles for these resources (see Section 4.2.1). This enables the worker that first obtains the data to provide replicas of the data to other workers. In this way, data transfer traffic is isolated within the boundary of the public cloud, which provides high network performance.

We compare data transfer times for the BVDV simulator when data replication is enabled (E) and disabled (D) for different numbers of workers (4, 8, 12, 16 workers) processing a total of 40 simulation runs. Fig. 13 shows that the time is lower when the replication feature is used, regardless of the number of workers. The amount of data transferred inside (I) and outside (O) the public cloud is shown in Fig. 14. Regardless of the number of workers, the portion of the traffic inside the public cloud is greatly increased when replicated management is enabled. This experiment validates that the replication feature of DiFFuSE helps reducing data transfer times in multi-cloud deployments.

Elasticity

This experiment evaluates the elasticity support provided by DiFFuSE. We use a scenario in which several dozens of worker services are simultaneously accessing a data service to load initial data, which causes the data service to become a bottleneck. We compare the performance of the simulator with and without the elasticity feature, which enables scaling out the data service.

Fig. 15 shows the CPU utilisation (%) of the data service in response to 20 worker services. All services are deployed in EC2 r4.xlarge VMs. The green and orange lines represent the cases with and without elasticity. The scale-out action is triggered if the CPU utilisation of the data service goes over 80% (the point A), which results in adding one more VM with the data service (green line from point A to B). Without elasticity, the single data service is busy with processing requests from the workers (orange line from point A to C). Importantly, the duration of processing data requests with elasticity is about 30% shorter than without elasticity, showing the usefulness of elasticity support in DiFFuSE.

CONCLUSION

The main goal of this work is to facilitate building and executing epidemic simulation applications in the cloud. To this end, the paper proposes DiFFuSE, a framework for structuring such applications as cloud-native applications composed of a set of distributed, collaborating services, which are independently developed, deployed and scaled in multi-cloud environments. The framework provides (1) mechanisms that allow services to exchange data in a reliable and scalable way and to automatically handle failures, (2) reusable code for managing experiments, simulation data, and computations following the bag-of-tasks model, and (3) tools for configuring the deployment and auto-scaling of services across clouds in a declarative way.

The paper also describes the use of DiFFuSE to restructure two legacy applications (simulators of BVDV and MAP spread) into elastic, cloudnative applications. Based on these applications, a comprehensive set of experiments were performed, demonstrating the advantages of the framework. These advantages are outlined next. DiFFuSE can exploit variable numbers and types of cloud resources, allowing making different trade-offs between performance and cost. It can automatically handle a wide range of failure and recovery scenarios, reducing the time to complete the simulation. DiFFuSE allows using resources from multiple clouds; notably, it allows combining private resources with those of public clouds, enabling, for instance, small institutions, such as local centers for disease control and prevention, to find cost-optimal resource allocations.

Moreover, the replication support of DiFFuSE helps reducing data transfer times in multi-cloud deployments. Finally, DiFFuSE allows elastically modifying resource allocations to adapt to dynamic variations in resource demand; notably, it allows automatically and independently scaling the data service to avoid performance bottlenecks.

We intend to continue this work in three dimensions. First, we will apply DiFFuSE to further simulation applications, which will produce useful feedback for improving the framework. Second, we will refine the basic allocation and failure-handling policies in order to optimise the performance of an application deployment, even in the case of failures. Third, we will extend DiFFuSE with policies for intelligent deployment optimisation, integrated into the PaaSage platform. [START_REF] Parlavantzas | Cost-Effective Reconfiguration for Multi-Cloud Applications[END_REF] This will provide us with a fully automatic solution for identifying the best initial deployment for a simulation application and for dynamically modifying this deployment to react to environment changes.

  agent-based modelling to simulate disease outbreak over large, realistic, co-evolving interaction networks. The paper presents an implementation of EpiSimdemics simulating the spread of influenza on different supercomputers. The authors argue that EpiSimdemics obtains five times greater speedup than the second fastest parallel code in the field. Similarly to the work by Eriksson et al.[START_REF] Eriksson | Dynamic Multicore Processing for Pandemic Influenza Simulation[END_REF] , the authors do not consider parallelisation at the distributed level.Many solutions exploit parallelisation at the distributed level in order to enhance performance. Martin and Carretero 5 present an MPI-based epidemic simulator, EpiGraph, used to study the propagation of infectious diseases between regions due to people movements. EpiGraph can be executed efficiently both on clusters and shared-memory nodes. Perumalla and Seal 6 present an optimistic, parallel, discrete event execution of a reaction-diffusion simulation of epidemic outbreaks. The simulation scales to 65,536 cores of a large Cray XT5 system with speedup of over 10,000. Rao and Chernyakhovsky 7 present an eco-modeling, bio-simulation environment, called SEARUMS++, for studying the global epidemiology of avian influenza. The environment uses Time Warp synchronised parallel simulations executed on clusters. Bisset et al.[START_REF] Bisset | Indemics: An Interactive High-performance Computing Framework for Data-intensive Epidemic Modeling[END_REF] present a modelling environment, called Indemics, for supporting complex epidemic simulations, not limited to a specific disease model. Although all previous solutions are scalable, they do not exploit the advantages of the cloud in reducing the cost and in increasing the agility of obtaining resources.

  installing and configuring software. At runtime, PaaSage monitors and adapts the deployment to react to events, such as violations of performance objectives, or changes in cloud prices. It can notably trigger elasticity actions (e.g., scale out a component) based on ECA (Event Condition Action) rules defined in CAMEL. Within DiFFuSE, PaaSage receives as input CAMEL descriptions of the simulation services and of available cloud providers and generates and dynamically maintains an appropriate multi-cloud deployment.

Algorithm 4 . 3 :

 43 Pseudocode of Allocation Algorithm i n t g e t A l l o c a t i o n ( message r e q u e s t ) { i n t propose ; propose = ( ( RR + nbOfWorkers -1 ) / nbOfWorkers ) * r e q u e s t . NbTh ; i f ( propose > r e q u e s t . NbRR ) propose = r e q u e s t . NbRR ; i f ( propose >= RR ) propose = RR ; e l s e i f ( propose == 0 ) propose = RR ; r e t u r n propose ; } A l l o R = g e t A l l o c a t i o n ( r e q u e s t ) ;

FIGURE 3

 3 FIGURE 3 Services of the BVDV and MAP spread simulators

FIGURE 4

 4 FIGURE 4The BVDV application deployed in the cloud

Fig. 5

 5 Fig. 5 shows the time and cost to conduct 20, 40, 100, 200 and 500 runs in the private cloud. The execution time is almost linear in the number of runs. With 500 runs, the execution times are notably 268.36 and 127.76 minutes with 4 and 9 worker services. For comparison, according to experiments done at INRA, performing 500 runs with the sequential legacy code takes around 1980 minutes (33 hours) in a server of the BioEpAR 31

FIGURE 5 FIGURE 6

 56 FIGURE 5 BVDV in Private Cloud: Number of Runs vs. a) Execution Time, b) Cost

FIGURE 7

 7 FIGURE 7 MAP in various clouds: Number of Runs vs. a) Execution Time, b) Cost

FIGURE 8 FIGURE 9

 89 FIGURE 8 Execution Time and Cost to run the BVDV spread simulator: a) 5 runs b) 20 runs

FIGURE 10 FIGURE 11

 1011 FIGURE 10 Execution Time (a) and Cost (b) to run the MAP spread simulation with different number of workers in Google Compute cloud

FIGURE 12

 12 FIGURE 12DiFFuSE with failure handling -Scenario 2

FIGURE 13 FIGURE 14

 1314 FIGURE 13 Number of Workers Vs Data Transfer Time

FIGURE 15

 15 FIGURE15 CPU utilisation (%) of the BVDV data service in response to 20 workers (EC2.r4.xlarge)

RR+=AlloR i +ExeR i ExpR-=p2 AlloR i -=p2(==0) Calculate AlloR p3 RR-=p3 ExpR+=p3 AlloR i +=p3 AlloR=p3 Worker i Failed UR=ExeR i +p2 Worker i Revived c) RR: Remaining Runs NbR:

  Number of Runs required by the simulation NbRR: Number of Runs requested by a worker ExeR: Executed Runs by a worker NbTh: Number of Threads supported by a worker ExpR: Allocated Runs are expected to finish AlloR: Runs allocated by the simulation to a worker UR: Unsuccessful Runs of a worker due to failures AlloR i : Run allocated by the simulation to worker i ExeR i : Executed Runs by worker i

	ExeR i =0 t 0		ExeR i =0	Worker i ExeR i =0 t 0	NbRR=r ExeR=0 NbTh=n	Experiment Service RR=NbR ExpR=0 ExeR i =0	Worker i ExeR i +=p1 t 1	ExeR=p1	Experiment Service ExeR i +=p1
			AlloR i =0			AlloR i =0		ExpR-=p1
						Calculate AlloR p1		Calculate AlloR p2
		AlloR=p1	RR-=p1 ExpR+=p1		AlloR=p1	RR-= p1 ExpR+=p1		AlloR=p2	ExpR+=p2 RR-=p2
			AlloR i +=p1	Execute		AlloR i	+=p1		AlloR i +=p2
	ExeR i +=p1 t 1	ExeR=p1	Calculate AlloR p2 ExeR i +=p1 ExpR-=p1 AlloR i -=p1	AlloR p1 UR=u1<=p1 ExeR i +=p1-u1 t 1	ExeR=p1-u1	AlloR p2 ExeR i +=p1-u1 RR+=u1 ExpR-=p1 Calculate AlloR i -=p1		ExeR i =0
		AlloR=p2	RR-=p2 ExpR+=p2 AlloR i +=p2		AlloR=p2	RR-=p2 ExpR+=p2 AlloR i +=p2	ExeR i =0	ExeR=0	ExeR i =0 AlloR i =0
					b)

FIGURE 2 Protocol for Allocating Simulation Runs in Different Situations: a) Normal, b) Lost Runs and c) Failed Workers

TABLE 1

 1 DISTRIBUTION OF RUNS AMONG WORKERS . Total CPU time is the amount of time spent by all VM cores for simulation processing. R_CPU is the exploitation rate of the VM cores,

		NbRR		2				4				8
			WF	L1	L2	RL	WF	L1	L2	RL	WF	L1	L2	RL
		Worker 1	11	8	8	10	12	8	8	9	13	8	8	8
		Worker 2	10	6	7	9	9	6	7	8	8	8	8	8
		Worker 3	0	7	6	2	0	7	6	4	0	5	5	5
		MAX	11	8	8	10	12	8	8	9	13	8	8	8
	TABLE 2 THE BVDV SPREAD SIMULATOR IN A HETEROGENEOUS MULTI-CLOUD ENVIRONMENT
				BVDV spread simulator -Total runs 20
	VM Type	# Cores	# Runs	Execution Time	Total CPU Time	Total Cost	R_CPU	Cost per Run
	ec2_m4_4xlarge	16	10		32.62(m)		14591.30(s)		0.651$	46.60%	0.065$
	ec2_r4_2xlarge	8	8		36.10(m)		11115.20(s)		0.435$	64.15%	0.054$
	gwdg_m2_xlarge	4	2		27.93(m)		2706.91(s)		0.217$	40.38%	0.108$
	calculated as follows:
				R_CPU =	TotalCPUTime * 100 ExecutionTime * NumberofCores

experiment

ACKNOWLEDGMENTS

We thank Yvon Jégou for developing the original framework. This work was carried out with the financial support of the French Research Agency (ANR), Program Investments for the Future, project ANR-10-BINF-07 (MIHMES). We also thankfully acknowledge the support of Inria (Technological development action ADT-140), the PaaSage (FP7-317715) EU project, the European fund for the regional development of Pays de la Loire (FEDER), and the European fund for inventive researchers (Agreenskills plus). The Grid'5000 testbed is supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations (see https://www.grid5000.fr).