Flavien Sciortino

Gérald Casterou

Pierre-Antoine Eliat

Marie-Bérengère Troadec

Cédric Gaillard

Soizic Chevance

Myrtil L Kahn

Fabienne Gauffre

Simple Engineering of Polymer-Nanoparticle Hybrid Nanocapsules

A general method to generate hybrid hollow capsules is reported. The process is based on the stabilization of solvent droplets by nanoparticles, in macroscopically miscible mixtures of THF and water. After addition of a crosslinking polymer and removal of the solvent core, capsules of diameter ca 100 nm are obtained. This novel strategy does not require the use of block copolymers. In contrast, most methods reporting the formation of hybrid nanocapsules incorporate nanoparticles into block-copolymer polymersomes or use nanoparticles tethered with block-copolymers. The nanocapsules were characterized using a full set of techniques including nanoparticle tracking analysis, electron microscopy and liquid phase atomic force microscopy. Our results show that the vesicular shape of the nanocapsules is templated by the liquid droplets. Nanocapsules were prepared from Quantum Dots, gold nanoparticles, superparamagnetic iron oxide nanoparticles and mixtures of particles. The entrapment of a fluorescent dye was also demonstrated. Thus, nanocapsules with dual properties (e.g. magnetic and fluorescent) are easily obtained. Interestingly, the magnetic nanocapsules enable Magnetic Resonance Imaging contrast enhancement of tumors in vivo.

The interest in forming assemblies of inorganic nanoparticles and polymers is manifold. On the one hand, the nanoparticles bring optical or magnetic properties which can be used for sensing, imaging, or heating, whereas on the other hand the polymers provide functions such as response to external stimuli (pH, temperature), stealthiness and molecular recognition. [1] Although such assemblies held much promise as multifunctional platforms for nanomedicine, the control of their size and morphology remains a daunting challenge. A variety of strategies have been proposed to elaborate nanoparticle-polymer assemblies. [2] For instance, the in situ synthesis of nanoparticles within linear or hyperbranched polymers is generally a convenient one-pot procedure in which the polymer promotes the nucleation of the particle while preventing uncontrolled clustering. [3,4] Alternatively, nanoparticles can be encapsulated within block-copolymer micelles [5,6] or coassembled with polymers via nanoprecipitation or electrostatic interactions. [7,8] Recently, many efforts have been devoted to the elaboration of hybrid nanoparticle-polymer nanocapsules, which offer the possibility to encapsulate and potentially release drugs. Thus, polymersomes incorporating inorganic nanoparticles into their membranes were fabricated via the cooperative selfassembly of amphiphilic block-copolymers and nanoparticles. [9] Alternatively, preformed polymersomes were armored with a layer of nanoparticles via electrostatic interactions. [9] It was also demonstrated that inorganic nanoparticles tethered with blockcopolymers self-assemble into superstructures such as clusters and vesicles. [10][11][12][13][14] All these strategies have in common the use of block copolymers which direct the self-assembly. We report here for the first time a general process to generate hybrid hollow capsules without the need for block-copolymers. The formation of the vesicles is templated by solvent droplets in mixtures of water and THF. Indeed, although water and THF are macroscopically miscible, nanodroplets are formed in the water rich region, which we used as templates for the self-assembly of the nanoparticles. After crosslinking the nanoparticles with a suitable polymer the solvent core can be removed. The resulting vesicles possess a "composite" or "hybrid" nanoparticle-polymer shell. In the following we will refer to them as hybridosomes. Based on these findings, we elaborated hybridosomes from a variety of nanoparticles and evaluated the in vivo MRI efficiency of hybridosomes made from superparamagnetic iron oxide nanoparticles (SPION). We have all learned from our textbooks that water is miscible in all proportions with THF. However, dynamic light scattering (DLS) and small angle neutron scattering measurements clearly evidence fluctuations of concentrations for specific compositions of water/THF mixtures. Such fluctuations have also been reported for mixtures of water with other "miscible" solvents such as acetone or alcohols, [15][16][17] the nature of which is still in debate. [18] It has been suggested that they could be either solute droplets [15] or air bubbles. [19] Using Nanoparticle Tracking Analysis (NTA) and analytical centrifugation, Sedlak et al. have provided convincing evidence of the existence of droplets of about 100 nm in diameter in various mixtures of miscible liquids. [15] Inspired by these results, we hypothesized that such droplets could template the formation of nanoparticle shells, similarly to Pickering emulsions. Nanoparticles are not amphiphilic but strongly adsorb at interfaces between polar and non-polar liquid. [20] We first looked for solvent droplets in water/THF mixtures in a wide range of compositions (φw=0.05-0.9, water vol. fraction). Time fluctuations were clearly measured using DLS, with faster dynamics as more water was added (Figure S1). Particularly, a mode corresponding to the Brownian motion of small objects (length scale~100 nm) appeared for compositions close to φw=0.8. Note that DLS is sensitive to any fluctuation and that the dynamic modes that were measured do not necessarily correspond to droplets. To verify the presence of droplets, NTA was undertaken for compositions with the fastest correlation modes (φw=0.8-0.9). The results clearly evidenced finite structures with a hydrodynamic diameter Dh=110±35 nm (Figure S2). [Fe]=65 µg/ml; PAA450k.

We then investigated the water/THF/NP (NP = nanoparticles) mixtures using dodecanthiol-coated gold nanoparticles (AuNP) (Figure S3a). In the presence of small or moderate amounts of water (φw=0.05 to 0.4), a bad solvent for the NPs, they rapidly aggregate and sediment as expected. However, stable suspensions were found at higher water contents (0.5 <φw< 0.9), the most stable samples (φw=0.8) showing no sign of sedimentation over more than one year. A similar behavior was found for -Fe2O3 (maghemite) SPION (Figure S3b). TEM revealed relatively ill-defined structures for φw=0.42, whereas regular spherical aggregates with a diameter of ca 100 nm were obtained at φw=0.8. DLS measurements clearly evidenced the stabilization of the smaller droplets at the expenses of larger ones in the presence of the nanoparticles (Figure S4).

Based on these findings, we proposed a general strategy to elaborate stable nanoparticle-polymer capsules (Figure 1). Starting from THF solutions of hydrophobic nanoparticles the selfassembly is triggered by adding water to a final volume fraction φw=0.8. This causes the nanoparticles to gather at the surface of solvent droplets (step 1). On addition of a suitable polymer, a crosslinked network of nanoparticles and polymers forms around the droplets (step 2). Finally, the removal of the solvent yields the hybridosomes (step 3). As a proof of principle, we firstly elaborated magnetic hybridosomes from SPIONs. Assemblies of SPIONs are of particular interest in a wide range of applications including MRI [21] and protein separation, [22] among others. However, due to their small size, single SPIONs are difficult to collect under magnetic field and make poor MRI contrast agents. A large number of theoretical and experimental studies have demonstrated that clustering of SPIONs increase the transversal relaxivity r2, thereby enhancing the efficiency as T2 contrast agent. [23][24][25][26] Magnetic hybridosomes were prepared following the scheme depicted in Figure 1. Poly(acrylic acid) (PAA) was selected as the polymer for its ability to link to iron oxide, via the multiple carboxylic acid functions. [27] Alternatively, we used a poly(ethylenglycol)-co-PAA diblock copolymer to prepare PEGylated hybridosomes (Figure 4a). [28] The SEM analysis of the dried samples at step 1 and step 2 clearly evidenced deflated capsules (indicated by arrows, Figure 2) together with spherical ones. In their hydrated state, the hybridosomes adopt a swollen shape as shown by AFM analysis in water (Figure 3). To confirm their hollow structure, we imaged the hybridosomes using TEM tomography (Figure 5, Movie S1). Due to the small size of the pristine nanoparticles, it is difficult to identify them individually on the 3-dimensional reconstruction. However, the hollow structure is clearly confirmed both by the animated tilted projection (Movie S1) and the 3D-reconstruction. The templating effect of droplets clearly appears when a lower amount of nanoparticles is used as in Figure S5. In this case, the partial coverage of the surface by nanoparticles underlines the shape of the initial droplet. All together, these analyses strongly support the hypothesis that the hybridosomes are soft hollow capsules. The hybridosomes were easily separated from the supernatant by applying a magnet and then resuspended without any noticeable aggregation. Mixed hybridosomes of AuNP and SPIONs as well as hybridosomes from QD were also prepared (Figure 4b-c). TEM (Figure 4b) and EDX (Figure S6) analysis of the mixed AuNP/SPION hybridosomes both confirm the co-localization of gold and iron. Since the PAA does not link to the AuNP, a collapse of the AuNP within the core of the hybridosomes on removal of THF cannot be excluded. However, the blue resonance characteristic of coupled AuNp was not observed. SPIONs hybridosomes could also be loaded with a fluorescent BODIPY dye, simply by dissolving the dye in the THF solution at the initial stage of the process, yielding dual magnetic and fluorescent features (Figure S7). The transversal relaxivities r2 of the magnetic hybridosomes were found comparable to those of commercial contrast agents such as Feraspin™ in vitro (Figure S8). Therefore, we evaluated their imaging ability for in vivo Magnetic Resonance (MR) using a mouse model. The T2*-weighted images show a significant extinction of the liver signal after retro-orbital intravenous injection (Figure S9). Interestingly, AuNP/SPION hybridosomes also yielded significant contrast, which augurs interesting prospects for theranostic with MR-imaging and gold-based thermotherapy (Figure 6a-b). A further beneficial effect is the increased tumor detection (Figures 6c-d). We evaluated the in vitro cytotoxicity In summary, based on the observation that THF and water mixtures contain droplets of well-defined size, we propose a new strategy to form hybrid nanoparticle-polymer nanocapsules, using these droplets as templates. In contrast with previous methods, A c c e p t e d M a n u s c r i p t this novel strategy does not require the use of amphiphilic blockcopolymers and produces vesicles of smaller size. We demonstrated that such vesicles can be readily obtained from nanoparticles of various chemical composition provided that the polymer crosslinks the nanoparticles as it is the case for PAA with iron oxide or with CdSe@ZnS. Particularly, SPION containing hybridosomes provide efficient MRI contrast enhancement, enabling the imaging of tumor areas at low dose and show low if any toxicity. Not only does this study demonstrate a versatile method for the facile engineering of well-defined assemblies with combined plasmonic, magnetic and/or fluorescent properties, but it also brings into light the role of preformed solvent droplets in the formation of vesicular aggregates.

Hybridosomes. Although water and THF are known as miscible solvent, their mixtures contain droplets that template the self-assembly of nanoparticles and polymers. This strategy was successfully applied to elaborate hollow nanocapsules (~100nm) with a hybrid shell made of crosslinked polymers and nanoparticles.

Magnetic nanocapsules proved to be efficient as in vivo MRI contrast agent for tumor imaging.

Experimental section

Materials. Octylamine was purchased from Sigma-Aldrich and stored as received in an argon-filled glove box. {Fe[N(SiMe3)2]2} (purchased from Nanomeps) was stored as received in the fridge of the glove box. 5.4 nm dodecanthiol coated gold NP (AuNP) were purchased from Nanocomposix. Tetrahydrofuran (VWR) was used as received. Poly(acid acrylic) PAA (1.88k or 450k) was purchased from Aldrich. The BODIPY 1 was kindly provided by O. Mongin (ISCR) and synthesized following a reported procedure. [1] Octadecylamine-coated CdSe@ZnS core-shell quantum dots in powder form were purchased from Sigma-Aldrich. The MRI constrast agent Feraspin XXL™ was purchased from Miltenyi Biotec.

SPION synthesis.

The synthesis of the maghemite (-Fe2O3) SPIONs was achieved via hydrolysis of an organometallic Fe(II) precursor following a recently reported procedure. [1] The reaction was achieved in pure octylamine and in mild conditions of pressure and temperature. H2O was deoxygenated by bubbling argon 30 mins before the reaction and stored under argon. The precursor {Fe[N(SiMe3)2]2} and octylamine (2 eq. to the precursor) were introduced in vials in a glovebox under argon. After a dark homogeneous liquid was obtained the vials were placed in a closed reactor, removed from the glovebox, and set to 40°C. After 15 min, H2O (2.1 eq.) was introduced in the reactor. The reaction was stopped after 3 days by putting the reactor under vacuum. The mixture was then exposed to air and dispersed in THF. The solution was then centrifuged at 19 000 rcf for 10 min and the pellet discarded. The mean diameter measured from TEM analysis was 5.56 nm (0.60 nm standard deviation). After air oxidation, A c c e p t e d M a n u s c r i p t superparamagnetic magnetite nanocrystals with a mean diameter d= 5.5 nm and a narrow size distribution (standard deviation 1.0 nm) are obtained. [1] Elaboration of the hybridosomes. The hybridosomes were prepared by adding water to a THF solution containing the hydrophobic NP, in order to reach the composition φw=0.8, promptly followed by vortexing (total metal concentration C=26-66 µg/mL). After ca 15 h, PAA (2.1 mMol.L-1 in AA unit) or PEG-PAA (2.7 mMol.L-1, in AA unit) were added to the mixture. The THF was then totally removed by slow evaporation (15h at 40°C), after which the remaining volume was ca 500 µL. It was verified that after this treatment, no traces of THF could be detected by 1H NMR. Nanoparticle Tracking Analysis (NTA) and Dynamic Light scattering (DLS). NTA was carried out with a Nanosight LM10 device system equipped with a 40 mW laser working at  = 638 nm. Video sequences were recorded via a CCD camera operating at 30 frames per second and evaluated via the NANOSIGHT NTA 2.0 Analytical Software Suite. A blank NTA measurement with pure water was performed to exclude a possible contamination with scatterers. Calculations of the hydrodynamic diameters were achieved taking 1.22 cP as the viscosity of THF/water mixture at φw= 0.8, from Nayak et al. [2] DLS measurements were performed by using a NanoZS Instrument (Malvern) with Contin analysis to yield the size distribution by intensity.

Scanning Electronic Microscopy (SEM).

A few microliters of a solution of hybridosomes were dropped onto a polished silicium wafer. Observation of sample was done with a JEOL 7100f ttls SEM at 10kV equipped with a tungsten tip and a Everhart-Thornley detector.

Transmission Electronic Microscopy (TEM).

A 200mesh copper grid, covered with a carbon/formvar film was dipped into the solution of hybridosomes (at [Fe]=1mM) and allowed to dry in ambient conditions. The samples were examined using a JEOL 1400 TEM operating at 120kV and equipped with a GATAN Orius 1000 camera or alternatively a JEOL 2100 LaB6 equipped with a GATAN Orius 200D and a SDD detector for Energy-Dispersive X-ray Spectroscopy (EDS) analysis. For TEM tomography a GATAN dual orientation 927 tomography sample holder was used. Tilted projection images were recorded at 1° tilt intervals from -50° to +30° and we used Digital micrograph software to convert the TEM projections into the 3-dimensional reconstructed image.

Atomic Force Microscopy (AFM).

Samples were imaged in fluid over an area of 500 nm 2 to 20 mm 2 at 0.2 to 0.5 Hz operated in PeakForce Tapping mode on a BioScope Catalyst AFM (Bruker AXS, Santa Barbara, CA) using a small volume liquid cell and a set-point of around 1-2 nN. SCANASIST-Fluid AFM tips (Bruker AFM Probes, Camarillo, CA) with a nominal spring constant, resonant frequency and radius of 0.7 N/m, 150 kHz and 20 nm respectively, were used for each experiment. AFM data were analyzed using the Nanoscope Analysis 1.5 software (Bruker AXS, Santa Barbara, CA).

Relaxivity measurements. Relaxivity measurements of hybridosomes were performed by measuring T2 values at various iron concentrations in water (determined by ICP-OES). Acquisitions were performed at 4.7 T on a Bruker Biospec 47/40 horizontal MRI system equipped with a 72 mm inner diameter volume transmit/receive quadrature coil. The T2 values were measured by using a multi-echo spin echo sequence with 16 echoes, an echo time spacing of 8 ms, and a repetition time (TR) of 2000 ms. The T2 maps were generated with ImageJ using the MRI analysis Calculator V1 by fitting the signal decay to a monoexponential function. Other imaging parameters were an image matrix size of 128 x 128, a field of view (FOV) of 51.2 mm, a slice thickness of 3 mm, and one signal average per phaseencoding step. The measurements were performed at room temperature (20.0±0.5°C). For each sample, a circular region of interest (ROI) was drawn from which T2 maps and then the mean T2 and standard deviations were calculated.

Cytotoxicity. HEK cells were grown in Dulbecco's modified Eagle medium (DMEM) supplemented with 10% fetal calf serum and 1% antibiotics. The hybridosomes were dialyzed against cell culture medium for 4 h prior to cell exposure. The cells were then exposed to hybridosomes (100 µg of iron per In vivo MRI. Animal experiments were conducted in line with French Agriculture Ministry guidelines. The C57BL/6 healthy mice and tumor bearing mice received SPION hybridosomes (1mg iron/kg body weight, stabilized with PAA 450k) or SPION-AuNP hybridosomes (0.5 to 0.9mg iron/kg body weight, stabilized with PAA 450k or PAA 1.8k) by retro-orbital intravenous injection, after anesthesia with isoflurane. MR imaging sequences were recorded prior to the injection of the contrast agent, and 30 mins and 24 h after injection, using the 4.7 T (Brucker) small animal imager. Magnetic Resonance (MR) images were acquired using a Bruker Biospec 47/40 (Bruker Biospin, Wissembourg, France) with a 4.7 T magnet with a BG-06 gradient insert. Anesthetized animals were placed in a supine position in a 3.5-cm bird-cage resonator. Images were acquired using a gradient echo (FLASH) sequence with TR = 300 ms, = 5 ms, Flip Angle = 30°, 4 averages per phase-encoding step, a 212 x 212 matrix and FOV = 3.2 x 3.2. Fifteen 1 mm thick axial slices covering the mouse abdomen were acquired. Respiratory gating was used during the acquisition. A tube filled with water was used as reference. Respiratory triggering was performed using a highly sensitive pneumatic system placed under the mouse. Body temperature was maintained with a warming blanket.

2. Investigation of THF/water and THF/water/NP mixtures a) THF/water mixtures For dynamic light scattering measurements of THF/water mixture, great care was taken to avoid any impurities: water was filtered through a 0.2 µm cellulose filter before use and THF was never in contact with any plastic material. Only glassware was used for all syringes, flasks and measurements cell. Dynamic light scattering (DLS) measurements of mixtures of THF and water were achieved for various volume fractions of water (φw) in the range φw = 0.05-0.9 (Figure S1). The relaxation times extracted from the correlograms are interpreted in terms of Brownian motion of spherical particles. Figure S1 shows that fluctuations of concentrations are measured over the whole range of compositions. As the fraction of water increases, faster dynamics is observed, corresponding to smaller length scale. A mode corresponding to the Brownian motion of spherical objects of ca 250 nm can be observed at φw= 0.78 and φw= 0.89. Since DLS is sensitive to any fluctuation, the dynamic modes that were measured do not necessarily correspond to defined long-lived structures. Therefore, Nanoparticle Tracking Analysis (NTA) analysis was undertaken for compositions where the fastest correlation modes were observed (φw= 0.8-0.9). NTA is a microscopic analysis of individual trajectories of spotlights scattered by scattering objects in solution. It was shown to accurately analyze the size distribution of polydisperse samples, which is not the case for DLS. Individual objects were clearly observed (Figure S2) using NTA. The size distribution profile shows a narrow distribution with a main mode at Dh = 110±35 nm.

Fluo-magnetic hybridosomes

The magnetic SPION hybridosomes could also be loaded with a fluorescent BODIPY dye, simply by dissolving the dye into the THF solution at the initial stage of the fabrication process. The selected dye (BODIPY 1, Figure S7) is not soluble in water and fluoresces both in its dissolved and aggregated states. Two cycles of magnetic separation and washing with water were achieved to obtain bright fluorescent hybridosomes and a dye free supernatant as demonstrated by absorbance and emission measurements. Figure S7a shows the absorbance and emission spectra of the supernatant after the two cycles of washing, and Figure S7b shows the fluorescent hybridosomes after redispersion in water. No significant leakage of the dye was observed over months.

Transversal relaxivities

The transversal relaxivities of SPION and SPION/AuNP hybridosomes stabilized with PAA or PEG-PAA were measured in water using a 4.7 T magnetic resonance (MR) scanner (Figure S8). The measured values were r2 = 274±35 mM -1 .s -1 , 285±37 mM -1 .s -1 and 257±15 mM -1 .s -1 for the SPION/PAA, SPION/PEG-PAA, and SPION/AuNP PAA hybridosomes, respectively. All measured values compare Feraspin™, a commercial contrast agent for pre-clinical imaging, measured in the same conditions (r2 = 307±80 mM -1 .s -1 ). The choice of the polymer, PAA or PEG-PAA, has no significant

Figure 1 .

 1 Figure 1. Process of elaboration of the hybrid nanocapsules.

  , Dr. G. Casterou, Dr S. Chevance, Dr. F. Gauffre Institut des Sciences Chimiques de Rennes, UMR 6226 CNRS, Université de Rennes 1 263 av. Général Leclerc, 35042 Rennes Cedex, France E-mail: fabienne.gauffre@univ-rennes1.fr [b] Dr. G. Casterou, Dr. M. L. Kahn Laboratoire de Chimie de Coordination UPR8241 CNRS, 205 rte de Narbonne, Toulouse cedex 04, France. [c] Dr. P-A. Eliat PRISM-Biosit, Université de Rennes 1, CNRS-UMS 3480, INSERM UMS 018, Rennes, France [d] Dr. M-B. Troadec IGDR, UMR 6290 CNRS, Université de Rennes 1, 2 av. du Prof. Bernard, CS 34317, 35043 Rennes Cedex, France [e] Dr. C. Gaillard INRA, La Géraudière 44316 Nantes cedex 3, France

A c c e p t e d M a n u s c r i p tFigure 2 .

 2 Figure 2. SEM images of (a-b) assemblies of SPIONs in water/THF mixtures; and (c-d) magnetic hybridosomes. (a) [Fe]=65 µg/ml; (b) [Fe]=80 µg/ml (c-d) [Fe]=25 µg/ml; polymer: PAA450k. Bars: (a;c)=1 µm; (b;d)=100 nm. White arrows indicate deflated capsules.

Figure 3 .

 3 Figure 3. AFM of SPIONs hybridosomes in aqueous solution. (a-a') 3-D topography; (b-b') lateral views; (c) TEM image of the corresponding sample.

Figure 4 .

 4 Figure 4. TEM of (a) SPION hybridosomes ([Fe]=50 µg/mL; polymer: PEG-PAA); (b) mixed hybridosomes from AuNP and SPIONs, the AuNP appear with darker contrast ([Fe]=[Au]=26 µg/mL; polymer: PAA450k), (c) SEM of (CdSe@ZnS)QD-hybridosomes. ([QD]=26µg/mL, PAA450k) Bars: (a) 500 nm; (b) 50 nm; (c) 100 nm.

A c c e p t e d M a n u s c r i p t

Figure 5 .

 5 Figure 5. Top view of the 3-dimensional density map

  (Figure S10) to HEK cells for hybridosomes made from two types of particles (SPION and SPION/AuNp) and stabilized with the two polymers used in this study (PAA450k and PEG-PAA). After 48 h of exposure at an iron concentration of 100 µg per 106 cells, both SPION/PAA450k and SPION/PEG-PAA hybridosomes show no detectable toxicity. Mixed SPION/AuNP hybridosomes stabilized with PAA450k show only a slight toxicity, with 85% of viability relative to the control, despite the high metal concentration ([Fe]=[Au]=100 µg per 106 cells).

Figure 6 .

 6 Figure 6. T2*weighted MR images of healthy (a-b) and tumor bearing (c-d) mice acquired before (a;c) and at 30 min (b;d) after injection of the AuNP/SPION hybridosomes (0.5 to 0.9 mg/kg).

Flavien

  Sciortino, Gérald Casterou, Pierre-Antoine Eliat, Marie-Bérengère Troadec, Cédric Gaillard, Soizic Chevance, Myrtil L Kahn,* Fabienne Gauffre* Simple Engineering of Polymer-Nanoparticle Hybrid Nanocapsules A c c e p t e d M a n u s c r i p t Additional Author information for the electronic version of the article.

A c c e p t e d M a n u s c r i p t 106

 106 cells) for 48 h at 37°C in cell culture medium. Viability assays were carried out by measuring MTS (3-(4,5-dimethyl-2-yl)-5-(3-carboxymethoxyphenyl)-2-(4-sulfophenyl)-2H-tetrazolium) activity.

Figure S1 :

 S1 Figure S1: DLS measurements of THF/water mixtures. Volume fraction of water φw as indicated on graph.

Figure S2 :

 S2 Figure S2: NTA analysis of a THF/water mixture for φw =0.8. Left: size distribution profile. Right: optical microscopy image of the scattered spotlights.

  A c c e p t e d M a n u s c r i p t

Figure S4 :

 S4 Figure S4: Correlograms (a) and size distribution (b) (by scattered intensity) from DLS measurements of water/THF mixtures (φw=0.8) in absence of NP (blue) and in presence of SPION (green) and AuNP (red).

Figure S5 :

 S5 Figure S5: TEM picture of assemblies of SPION at low nanoparticle concentration, in order to evidence the templating effect of liquid droplets. [Fe]=5 µg/ml; polymer: PAA450k. Scale bar: 100 nm

  A c c e p t e d M a n u s c r i p t

Figure S6 :

 S6 Figure S6: Energy dispersive X-ray spectroscopy of a single AuNP/SPION hybridosome showing the co-localization of iron and gold.

Figure S7 :

 S7 Figure S7: Elaboration of magnetic and fluorescent hybridosomes: chemical structure of the BODIPY 1. (a) Absorbance and emission spectra of the supernatant after removal of the fluo-magnetic hybridosomes using magnetic separation. (b) Absorbance and emission spectra of the suspension after redispersion of the hybridosomes in water.

A c c e

  p t e d M a n u s c r i p t

  Experimental section ........................................................................................................................ 6 2. Investigation of THF/water and THF/water/NP mixtures ................................................................ 8 a) THF/water mixtures ......................................................................................................................... 8 b) Investigation of water/THF/NP mixtures ......................................................................................... 9

	F. Gauffre: 0000-0003-2835-3556
	Author:	ORCID identifier
	Author:	ORCID identifier
		Content
	1.

c) Effects of nanoparticles over THF/water mixtures (DLS) ............................................................. 10 3. Energy dispersive X-ray spectroscopy (EDX) ............................................................................... 11 4. Fluo-magnetic hybridosomes ......................................................................................................... 12 5. Transversal relaxivities .................................................................................................................. 12 6. In vivo MRI .................................................................................................................................... 13 7. In vitro toxicity ............................................................................................................................... 14 8. TEM tomography of hybridosomes ............................................................................................... 15

Acknowledgements

The authors thank O. Mongin (ISCR), M. Destarac (Univ. Toulouse) and O. Musso (INSERM) for providing BODIPY, PEG-PAA and the tumor bearing mice. We are grateful to BIOSIT-University of Rennes1 for financial support and to MRic (A. Burel), CMEBA (V. Dorcet) and THEMIS for analysis.

Keywords: nanoparticles • polymer • self-assembly • Magnetic Resonance Imaging • nanocapsules

effect on the relaxivity of SPION hybridosomes in water. Mixed SPION/AuNP hybridosomes show a sligthly lower relaxivity, most probably due to a lesser extent of magnetic coupling between SPION. 8. TEM tomography of hybridosomes Movie S1 shows animated slab of images collected at tilted intervals.

In vivo MRI

In vitro toxicity


C Labaki

A Karam

G Mjaess

Z Bakouny

A J Bizdikian

F Yared

J Otayek

W Skalli

I Ghanem

A Assi

The variation of lateral and posterior coverage of the femoral head by the acetabulum during walking influences stability during gait

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

The variation of lateral and posterior coverage of the femoral head by the acetabulum during walking influences stability during gait

Chris Labaki, Aya Karam, Georges Mjaess, Ziad Bakouny, Aren Joe Gait balance, assessed by the angle formed between the line joining the center-of-mass (COM) to the center-of-pressure (COP) and the vertical during gait, has been shown to be related to skeletal-postural and anthropometric parameters [START_REF] Bakouny | Gait and Posture[END_REF]. Although skeletal-postural parameters are measured on standing radiographs, they are known to vary during gait. There are currently no studies evaluating how the variations of skeletal-postural parameters during gait influence subject's balance during walking.

Research question

How does the variation of skeletal-postural parameters during gait influence subject's balance during walking?

Methods

72 asymptomatic subjects (age: 28.6 ± 11 years [18-59], 29F) underwent 3D gait analysis [START_REF] Davis | [END_REF] with additional markers on the thighs and shanks. The COM-COP angle with the vertical was calculated in both the frontal and sagittal planes during the gait cycle [3] (Fig. 1). Subjects then underwent low-dose full-body biplanar X-rays with the markers still in place. 3D reconstructions were obtained for the spine, pelvis and lower limbs. 3D bones were registered at each frame of the gait cycle [START_REF] Söderkvist | [END_REF]. A new technique developed for this study, utilizing finite element modelling, was used to reduce soft tissue artefacts. Skeletalpostural parameters were then computed during the gait cycle, using the 3D registered bones, at each time frame (Fig. 2); mean, minimum, maximum and ROM were calculated on the waveforms during the gait cycle. In order to determine which varying skeletal-postural parameter during gait determined the variation of the COM-COP angles, univariate analysis (Pearson's correlation) followed by a multivariate analysis (stepwise-multiple-linear-regression models) were computed; COM-COP parameters were the dependant variables and varying skeletalpostural parameters during gait were the independent variables.

Results

Minimum (-14.2 ± 3.4°) and average (3.1 ± 1.6°) of the sagittal COM-COP angle were found to be determined by the minimum of the posterior coverage (post_cov) of the femoral head by the acetabulum during gait (β = 0.40; R2 = 0.16; p = 0.003 and β = 0.32; R2 = 0.1; p = 0.001, respectively). ROM (33.9 ± 5.1°) and maximum (19.7 ± 2.8°) of the sagittal COM-COP angle were found to be related to the ROM (β = 0.29; R2 = 0.09; p = 0.03) and maximum (β = 0.34; R2 = 0.11; p = 0.006) of the acetabular abduction during gait, respectively. ROM of the frontal COM-COP angle (8.8 ± 2.51°) was found to be determined by the average of the post_cov (β = 0.51; R2 = 0.26; p = 0.004) during gait.

Discussion

This is the first study to evaluate how the variation of skeletalpostural parameters during walking influences the stability during gait (Fig. 3). A less pronounced posterior coverage of the acetabulum during gait predisposes to more instability by decreasing the minimum COM-COP angle; a more pronounced acetabular abduction (decreased lateral coverage) during gait predisposes to more instability by increasing the ROM and the maximum of the COM-COP angle. Therefore, gait instability in the sagittal plane seems to be influenced by the variation of the posterior and lateral coverage of the femoral head by the acetabulum during walking.

  Bizidikian, Fares Yared, Joeffroy Otayek, Wafa Skalli, Ismat Ghanem, Ayman Assi To cite this version: Chris Labaki, Aya Karam, Georges Mjaess, Ziad Bakouny, Aren Joe Bizidikian, et al.. The variation of lateral and posterior coverage of the femoral head by the acetabulum during walking influences stability during gait. Gait & Posture, 2018, 65 (S1), pp.139-141. 10.1016/j.gaitpost.2018.06.090. hal-02561805 1. Introduction

Fig. 2 .

 2 Fig. 2. 3D skeletal-postural parameters computed during the gait cycle.

Fig. 1 .

 1 Fig. 1. a) Frontal and sagittal COM-COP (Center Of Mass-Center Of Pressure) diagram computed during the gait cycle; b) Measurement methods of the frontal and sagittal COM-COP during the gait cycle.


Agniva Sengupta

Romain Lagneau

Alexandre Krupa

Eric Marchand

Maud Marchal

Maud Marchal Simultaneous

Simultaneous Tracking and Elasticity Parameter Estimation of Deformable Objects

come   L'archive ouverte pluridisciplinaire

Simultaneous Tracking and Elasticity Parameter Estimation of Deformable Objects

Agniva Sengupta * 1 , Romain Lagneau * 2 , Alexandre Krupa 1 , Eric Marchand 1 , Maud Marchal 3Abstract-In this paper, we propose a novel method to simultaneously track the deformation of soft objects and estimate their elasticity parameters. The tracking of the deformable object is performed by combining the visual information captured by a RGB-D sensor with interactive Finite Element Method simulations of the object. The visual information is more particularly used to distort the simulated object. In parallel, the elasticity parameter estimation minimizes the error between the tracked object and a simulated object deformed by the forces that are measured using a force sensor. Once the elasticity parameters are estimated, our tracking algorithm can be used to estimate the deformation forces applied to an object without the use of a force sensor. We validated our method on several soft objects with different shape complexities. Our evaluations show the ability of our method to estimate the elasticity parameters as well as its use to estimate the forces applied to a deformable object without any force sensor. These results open novel perspectives to better track and control deformable objects during robotic manipulations.

I. INTRODUCTION

Nowadays, robots are efficient in manipulating rigid objects. However, simultaneous manipulation and tracking of deformable objects remains a challenging problem and could improve different applications, such as humanoid robots interacting with soft objects, compliance testing in product manufacturing or detection of abnormal skin stiffness in preventive healthcare. The aim of our approach is to automatically estimate the elasticity parameters of a soft object that is being deformed by the end-effector of a robot. For that purpose, we propose a closed-loop method consisting of: a) a deformation tracking method and b) an elasticity estimation algorithm. This closed-loop method allows to achieve two objectives: STEPE (Simultaneous Tracking and Estimation of Parameters of Elasticity) and remote force estimation. The required inputs of STEPE are a coarse 3D geometric model (mesh) of the object to track and the external measurements of the forces deforming the object. The deformation tracking method uses a depth sensor to capture the deformation and tracks it using a physics-based simulation of the object deformations. The elasticity estimation algorithm uses the result of the deformation tracking method and measurements of the deformation forces, obtained from an external sensor, to estimate the elasticity parameters of the object. Once the elasticity estimation is achieved, the parameters can be used by the deformation tracking method, thereby closing the loop. The method will iteratively converge towards a correct estimation of the elasticity parameters of the object. The elasticity parameters obtained at the convergence of the method are thereafter used for remote force estimation. From this stage, the deformation tracking method can be used to estimate the deformation forces acting on the object without any external force sensor.

A. State of the Art 1) Deformation tracking: There are two possible approaches for tracking the deformation of non-rigid objects, using either a physics-based model, or a geometric-based model for computing the deformations. Both physics-based model such as [START_REF] Newcombe | Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time[END_REF], [START_REF] Innmann | Volumedeform: Real-time volumetric non-rigid reconstruction[END_REF] or geometric-based models such as [START_REF] Bronte | Model-based realtime non-rigid tracking[END_REF], [START_REF] Bronte | Real-time sequential model-based non-rigid sfm[END_REF] offer efficient mechanisms to track complex 3D shapes without relying on any kind of training data. However, it is not possible to use them to iteratively update the physical parameters of the object being tracked. Zhang et al. used a RGB-D camera to track external forces applied to an object, but with specialized soft robots for the purpose, instead of arbitrary deformable objects [START_REF] Zhang | Calibration and external force sensing for soft robots using an rgb-d camera[END_REF]. The approach of Petit et al. [START_REF] Petit | Capturing deformations of interacting non-rigid objects using rgb-d data[END_REF] is more suitable for the purpose of deformation tracking using a physics-based model. However, since they rely solely on depth data to determine the direction and magnitude of deformation, their approach is more susceptible to error induced from incorrect correspondences due to heavy occlusions. Among some learning-based approaches for tracking deformation, Varol et al. used a constrained latent variable model for learning the deformation behavior of an object from a latent state to an output state [START_REF] Varol | A constrained latent variable model[END_REF]. The results from this approach have however only been validated on planar objects. Among the available literature, the approach of Frank et al. [START_REF] Frank | Learning the elasticity parameters of deformable objects with a manipulation robot[END_REF] is the closest to the method proposed in this paper for simultaneously tracking deformation and estimating parameters. The approach utilizes a tightly coupled tracking and elasticity estimation module with a point-to-point ICP [START_REF] Besl | Method for registration of 3-d shapes[END_REF] for pointcloud registration along with a linear, tetrahedral FEM as underlying deformation model. The use of linear FEM makes this approach not suitable when large rotational deformations occur. Additionally, point-to-point ICP cannot handle occlusion adequately, which compels them to utilize a narrow wooden manipulator to avoid occlusion. To the best of our knowledge this approach has not been used as a force estimator.

2) Physical parameter estimation: Manipulation of deformable objects is facilitated when the elasticity parameters of the objects are known. Sedef et al. proposed a method using a haptic device and a force sensor to estimate the linear viscoelastic parameters of a homogeneous soft tissue [START_REF] Sedef | Real-time finite-element simulation of linear viscoelastic tissue behavior based on experimental data[END_REF]. Bickel et al. proposed a method handling heterogeneous elastic objects [START_REF] Bickel | Capture and modeling of non-linear heterogeneous soft tissue[END_REF]. A deformable object, observed by a marker-based stereovision system, is stimulated by a probe equipped with a force sensor to estimate the Lamé's coefficients of heterogeneous deformable objects. Data-driven methods have also been developed to determine elasticity parameters of heterogeneous elastic objects. For instance, Wang et al. proposed an approach that iteratively performs physics-based deformation tracking and physical parameter estimation using a two-step optimization method [START_REF] Wang | Deformation Capture and Modeling of Soft Objects[END_REF]. A method, based on the static Euler-Bernouilli deformation model along with a tracking method based on a multi-section parameterization, was proposed for simultaneous tracking and estimation of deformable object [START_REF] Fugl | Simultaneous estimation of material properties and pose for deformable objects from depth and color images[END_REF]. The method is based on two assumptions: on the one hand the object is homogeneous and linearly elastic, on the other hand the deformation is the bending of the object subject to gravity. However, this method has only been tested on planar object.

B. Contributions

We perform simultaneous tracking and elasticity estimation of arbitrary soft objects using the same physical model, which can be updated iteratively, as opposed to [START_REF] Frank | Learning the elasticity parameters of deformable objects with a manipulation robot[END_REF]. The deformation tracking does not require any external fiducial markers. A coarse model of the object is enough for both deformation tracking and elasticity parameter estimation. Contrary to [START_REF] Zhang | Calibration and external force sensing for soft robots using an rgb-d camera[END_REF], the remote force estimator does not require a prior knowledge of the elasticity parameter to estimate the deformation forces. The STEPE method can run on inexpensive hardware requiring only a RGB-D sensor and force measurements.

The remainder of the article is divided into three sections. The next section details the methodology for the proposed method. The experimental results are presented next, followed by discussions and perspectives about our method.

II. APPROACH FOR ELASTICITY ESTIMATION PROCESS

Our goal is to simultaneously track a deformable object and estimate its elasticity parameters. To achieve this goal, we propose a method composed of three different modules: one performing the deformation tracking, one estimating the elasticity parameters and one measuring the external deformation forces applied to the object. In STEPE, all the modules are involved. When doing remote force estimation, only the module performing the deformation tracking method is considered. The overall method is depicted in Fig. 1.

A. Modeling

We chose to model the objects using the Finite Element Method (FEM) [START_REF] Nealen | Physically based deformable models in computer graphics[END_REF]. Objects are represented using tetrahedral meshes, such as the cube depicted in Fig. 1. To avoid large rotational deformation artefacts, we use the corotational FEM variant of this method [START_REF] Muller | Interactive virtual materials[END_REF]. In this paper, we use the Hooke's law that relates the stress and the strain applied to an object using a linear equation that depends on the elasticity parameters of the object, that are the Young's modulus and the Poisson ratio. This law is valid only for small deformations of linear elastic objects.

Let n be the number of vertices of the mesh, M ∈ R 3n×3n the mass matrix and D ∈ R 3n×3n the damping matrix of the whole object, K ∈ R 3n×3n the stiffness matrix of the whole object that depends on the Young's modulus and Poisson ratio of the object, x(t) ∈ R 3n (resp x u ∈ R 3n ) the position of each vertex of the mesh at time t (respectively in the undeformed state), f ext ∈ R 3n the external forces applied to the object and f int ∈ R 3n the internal forces resulting from the deformation. The linear algebraic equation of motion of an object is given by:

M d 2 dt 2 x(t)-x u +D d dt x(t)-x u +K(x(t)-x u ) = f ext +f int (1)
This equation is used both to estimate the elasticity parameters from deformation observations and to estimate the external deformation forces from deformation observations when the elasticity parameters converged.

B. STEPE 1) External measurements module:

The external measurements module is in charge of measuring the deformation forces that are applied to the object. To ensure repeatability of experiments, we decided to use a robot equipped with a force sensor both to deform the object and to measure the resulting forces. The end-effector deforms the object while recording the forces that are exerted. The end-effector motion is stopped at one point in order to maintain a static deformed state. The module stores the external force measurements and timestamps at which they were acquired to transmit them to the estimation algorithm when a steady-state has been reached in the deformation tracking. Fig. 2. Every node that needs to be tracked is perturbed by a small force along 3-axis. Computation of (4) for each deformed configuration is used to compute the gradient of the error at that particular node.

2) Deformation tracking method: In this paper, we implemented a deformation tracking method similar to the one proposed in [START_REF] Sengupta | Tracking of non-rigid objects using rgb-d camera[END_REF]. The initial pose of the object(O) with regards to (w.r.t.) the camera (C) is obtained in the first image frame using pre-trained markers [START_REF] Marchand | Pose estimation for augmented reality: a hands-on survey[END_REF]. The pose of the camera w.r.t. the object, denoted by the homogeneous matrix C T O , is updated at the beginning of each frame, using an approximate rigid object tracking method with the assumption that the object is rigid. The cost function jointly minimized for approximate rigid tracking of the object is given by:

e D ( k-1 q k ) = k-1 R k k-1 P + k-1 t k • n j -d j (2) e K ( k-1 q k ) = x( k-1 q k ) -x * y( k-1 q k ) -y * (3)
where k-1 q k = ( k-1 t k , θu), such that θ and u are the angle and axis of the rotation k-1 R k from the current image frame 'k' to the previous image frame. k-1 P = X, Y, Z represents an arbitrary 3D point from the last pointcloud, which has been matched to the j-th plane of the object's 3D model, denoted by the normal vector n j = (n X j , n Y j , n Z j ) and distance to origin d j . (x( k-1 q k ), y( k-1 q k )) are the projection of the same arbitrary point in the (k-1)-th image while (x * , y * ) are the same image points matched in the k-th image using Harris corner features. Once an approximate estimate of k-1 q k has been obtained, the non-rigid tracking is done by minimizing the point-toplane error, given by:

e N (P) = n j • k-1 P -d j (4)
The Jacobian (J) is computed using the method shown in Fig. 2. Every node is subjected to a small force given by ∆F x , ∆F y and ∆F z , acting along the three axes. The displacement of all vertices of the mesh are computed by solving (1) using an Euler-Implicit mechanism of a conjugate gradient based linear solver [START_REF] Faure | SOFA: A Multi-Model Framework for Interactive Physical Simulation[END_REF]. The Jacobian J is computed using finite differences in the value of e N obtained from the three deformed configurations. The force update is given by:

∆F = -λ WJ + We N ( k-1 P) ( 5
)
where W is a weighting matrix from Tukey based Mestimator [START_REF] Meer | Robust regression methods for computer vision: A review[END_REF], and λ = λ X λ Y λ Z is a scaling factor. The deformation caused in the mesh due to the Jacobian computation is discarded and the update k-1 F+∆F is applied to the node, where k-1 F is the external force already existing on the node at the previous image frame. This Iteratively Re-weighted Least Squares scheme is allowed to repeat for a fixed number of iteration at every image frame.

3) Estimation algorithm: In this paper, we chose to use the Hooke's law evoked in II-A to represent the deformable objects. Consequently the elasticity parameters that can be estimated are the Young's modulus and the Poisson ratio. In order to estimate the elasticity parameters ξ at a contact point of a deformable object, the algorithm relies on the tracking results x t and the external measurements of the deformation forces f meas taken at the same point. Let V be the list of vertices that the camera can see denoted by v and used to compute a cost-function S , minimized using the Levenberg-Marquardt algorithm [START_REF] Moré | The levenberg-marquardt algorithm: implementation and theory[END_REF]. S is given by S = v∈V e E v (ξ) 2 . In this expression, e E is the error function defined by e E v (ξ) = x t vx s v (f meas , ξ) and is equal to the Euclidean distance between x t v and x s v , which are respectively the tracked and simulated 3D positions of v. x t v is taken at the final deformed state and x s v at the end of the simulation. The estimated parameters are the ones for which the costfunction S is minimized.

The simulation applies a force field, which replicates the forces that were recorded during the experiment, on the region surrounding the contact point location on the simulated object. The Jacobian matrix of the cost-function is needed to compute the update of the elasticity parameters between two steps k and k +1 of the minimization algorithm. This Jacobian matrix is numerically estimated by running several corotational FEM simulations using different values of elasticity parameters and computing the error between the experimental data and the simulation result. Let ∆ξ(k) be the update of the elasticity parameters at the k th step, J E be the Jacobian matrix at this step of the function e E w.r.t. the elasticity parameters ξ and µ be a damping factor that is adjusted at each Levenberg-Marquardt step. The update of the elasticity parameters can be computed by solving for ∆ξ(k) the linear equation:

∆ξ(k) = (J E ) J E + µdiag (J E ) J E -1
(J E ) e E ( 6)

ξ(k + 1) = ξ(k) -∆ξ(k) (7)
In this paper, we focus on estimating the Young's modulus E.

The Poisson ratio is assumed to be known a priori. Thus, the elasticity parameters vector is given by ξ = (E) ∈ R. At this stage, the method permits to estimate the Young's modulus Ê of a deformable object from deformation observations and external force measurements.

C. Remote force estimation

When doing remote force estimation, the deformation tracking module can be used without external force sensor. Let A be the set of vertices representing the active region of the object. The active region is an approximation of the surface of the model where the deformation is happening. This region is determined using some practical heuristics e.g., neither the vertices lying on the ground plane nor the vertices not visible from the camera are taken into account. Active vertices, denoted by a, are the vertices belonging to the active region. The method is able to estimate the external deformation forces, given by f ext = a f ext a , which are applied to the object in the active region, as shown in Fig. 3.

The estimated forces are projected onto the normal of the active surface n A = a n a a n a because the tracking method cannot have information about tangential forces, since ( 4) is a point-to-plane distance. This produces the following estimate of the magnitude of the deformation forces: Our method has been implemented on a computer that has an Intel Xeon CPU working at 3.70GHz with 16 logical cores. Our method has been tested with different soft objects: a foam block, a soft ball and a complex-shaped plush toy. The transformation between the robot frame and the object frame is computed once at the initialization of our method. Thereafter, it is used to express measured forces into the object coordinate frame. The mesh of the plush toy has been generated by photogrammetry using the Meshroom software [START_REF] Moulon | Adaptive structure from motion with a contrario model estimation[END_REF], [START_REF] Jancosek | Multi-view reconstruction preserving weakly-supported surfaces[END_REF]. The corotational FEM simulations are performed using SOFA framework [START_REF] Faure | SOFA: A Multi-Model Framework for Interactive Physical Simulation[END_REF].

f ext ≈ f ext • n A .

B. Results

The following section describes a series of experimental conditions and their corresponding results 1 . Each condition represents a particular position of a specific object, e.g., the first condition describes the rectangular foam being deformed at the center of its largest face, while being placed horizontally on a table. To evaluate the accuracy of the estimation process, the ground truth of Young's modulus E GT has been determined through indentation tests for each condition of experiment [START_REF] Mckee | Indentation versus tensile measurements of young's modulus for soft biological tissues[END_REF]. The indentation tests are repeated, slow and incremental vertical displacements applied to the objects using the robot while measuring the forces and the displacements. Forces are measured from the force sensor while displacements are obtained using the odometry of the robot. For each condition of experiment, the average of several indentation tests is taken as ground truth E GT .

Let Ê(i) be the estimated Young's modulus obtained from the i th deformation tracking results. The Young's modulus that is thereafter used for the external force estimation, denoted by E est , is the one for which the following convergence criterion is respected:

Ê(i) -Ê(i -1) Ê(i -1) < 0.05 (8)
Some examples of the output of the deformation tracking method are shown in Fig. 5. The mean of the norm of the error ( e N ) was found to be 1.53 mm, 0.51 mm and 0.21 mm for the foam block, soft ball and plush toy, while the standard deviations were 7.2 mm, 0.7 mm and 1.4 mm respectively, across all the experiments reported in this article. In the three example sequences shown in Fig. 5, the mean of the norm of error varied between a maximum of 0.381 mm (for the plush toy, deformed by its nose) to as low as 3 µm (for the undeformed sponge block), despite having outlying correspondence or noise in the range of -5.5 cm to +9.0 cm (which gets rejected by the M-estimator). In the experiments shown in Table I, the time required for tracking varies between 1.21 to 3.1 sec/frame. However, it was possible to run the same algorithm on the foam block at ∼ 800 ms/frame without using any GPU based computation, with negligible loss of accuracy (< 10%).

Our method can be evaluated on different criteria. The first set of experiments was conducted to evaluate the timeperformance of the tracking and the quality of the Young's modulus estimation w.r.t. the number of vertices of both the visual mesh and the mechanical mesh. The results of this set of experiments are grouped in Table I. In this table, #Visual designates the number of vertices of the visual mesh while #Mechanical designates the number of vertices of the mechanical mesh. t tr is the average deformation tracking time, std(t tr ) is its standard deviation, t st is the time that was required to deform the object and reach a steady-state and t est is the time to estimate the Young's modulus. All the times are expressed in seconds. E GT and E est correspond to the average ground truth of Young's modulus and estimated Young's modulus respectively, both expressed in kilo Pascals. Finally, Error designates the percentage of error of the estimation and is given by Error = 100 * abs(E GT -E est )/E GT where abs designates the absolute value.

The second set of experiments, whose results are grouped in Table II, was conducted to evaluate the consistency of the Young's modulus estimation w.r.t. the initial estimate E 0 . The last set of experiments was conducted to evaluate the estimation of the deformation forces in different conditions of experiments. For these experiments, the estimated Young's modulus values summarized in Table II were used by the deformation tracking method. The results of this set of experiments are grouped in Table III. f GT and f est designate the average norm of the ground truth forces measured by the force sensor and the norm of the forces estimated by the deformation tracking method respectively, expressed in Newtons. ∆ f = f estf GT represents the error between the estimated and ground truth forces. The estimated forces are projected onto the measured forces when compared with the ground truth. The force experiments have been conducted with different orientations of the objects (horizontal and tilted by approximately 25 degrees w.r.t. the z-axis of the object frame). Different depths of stimulation have also been tested and are indicated in the depth column of Table III.

Finally, Fig. 6 depicts a possible use case of our method to estimate deformation forces exerted on an object in an environment where no force measurements are available.

IV. DISCUSSION

The first set of experiments whose results are summarized in Table I shows that the accuracy of the Young's modulus estimation is improved as the resolution of the mechanical mesh increases. This results from the fact that the deformation can be represented with finer details and is thus closer to the reality. Table II shows that our method is able to estimate the Young's modulus of an arbitrary object even when the initial estimate is much different from the ground truth. A future work would consist in using our method to estimate both Poisson ratio and Young's modulus within an iterative process. Table III shows that the estimated Young's modulus could be used by our method to accurately estimate the deformation forces from visual information only. The use case scenario depicted in Fig. 6 shows one possible application of our method in environments where no force sensor could be used, for instance in human-performed manipulation tasks. This application opens novel perspectives for better estimating the forces applied to a deformable object by different operators, as well as on a cooperative task on such objects.

V. CONCLUSION

In this paper, we proposed a method for simultaneously tracking arbitrary linear elastic objects and estimating their elasticity parameter. For estimating the elasticity parameters, our method requires only a coarse 3D mesh of the object and external force measurements. Deformation tracking and elasticity parameter estimation are run alternatively. We showed experimentally that the method is able to correctly estimate the Young's modulus of an arbitrary soft object, even when starting from a highly coarse initial estimate. Finally, we demonstrated that our method could be used for remotely estimating the deformation forces applied to an arbitrary object using visual information only. These results pave the way for better control of deformable objects manipulated by a robot.

Fig. 1 .

 1 Fig. 1. Overview of the STEPE and Remote Force Estimation architecture.

Fig. 3 .

 3 Fig. 3. Force estimation in the active region.

Fig. 4 .

 4 Fig. 4. Experimental setup showing the robot (R r ) deforming a plush toy (R o ) while the data is captured from an Intel RealSense D435 RGB-Camera (R c .) The other objects that have been experimented upon are shown in the inset above.

Fig. 5 .

 5 Fig. 5. Tracking of the objects for estimating their elasticity parameters. The first and the fourth columns show the color image, the second and the fifth columns show the tracked object model and the third and the sixth columns show the value of e N , color-coded and augmented on the object

Fig. 6 .

 6 Fig. 6. Force estimation while the plush toy gets freely deformed by hand. deformation forces (13.8 N) are in the expected range.

TABLE I Evaluation

 I of tracking and estimation times and Young's modulus (in kPa) estimation accuracy w.r.t. different quality of visual and mechanical meshes.

	Objects #Visual	#Mechanical	t tr (s)	std(t tr ) (s)	t st (s)	t est (s)	E GT	E est	Error (%)
	foam	35	1049	1.94	0.14	62	310	454	431	5.08
	foam	415	1049	2.37	0.15	122	180	454	438	3.62
	foam	178	554	1.21	0.02	60	180	454	497	9.47
	ball	404	627	1.38	0.03	126	70	156	136	12.8
	ball	404	1060	1.61	0.02	118	96	156	149	5.0
	ball	404	1954	3.09	0.07	130	330	156	148	5.1

TABLE II Evaluation

 II of the estimation of the Young's modulus (in kPa).Experiments have been conducted on two locations of the foam and the plush toy, as indicated in the first column of TableII, to evaluate if the method can handle nonhomogeneous objects. Low initial estimate are of the order of 0.1×E GT and high initial estimate of the order of 10×E GT .

	Objects	E 0	E GT	E est	Error
		200	454	480	5.7%
	foam middle	1500	454	451	0.7%
		15000	454	488	7.5%
	foam corner	200	247	224	9.3%
	ball	1000	156	154	1.3%
	toy nose	200	51	53	3.7%
	toy leg	50	42	43.1	2.6%

TABLE III Comparison

 III of the average norm of the measured and estimated forces.

	Objects Orientation	Depth (cm)	f GT (N)	f est (N)	∆ f (N)
	foam	horizontal	2	18.8	20.6	1.8
	foam	tilted	2	21.36	23.11	1.75
	foam	horizontal	4	45.46	42.67	-2.79
	foam	tilted	4	38.94	35.59	-3.35
	ball	horizontal	2.5	9.49	8.57	-0.92
	toy	horizontal	2	5.32	5.81	0.49

Univ. Rennes, INSA, IRISA, Inria, CNRS, IUF, France e-mail: {agniva.sengupta, romain.lagneau, alexandre.krupa, eric.marchand, maud.marchal}@irisa.fr.

Additional material can be found at https://youtu.be/k1MPnmqmovQ

Acknowledgments

We gratefully acknowledge the support from The Research Council of Norway through participation in iProcess -255596 project. This research has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No 731761 for project "Imagine".


Edouard Chauvet

Catherine Tassin

Jean-Jacques Blandin

Rémy Dendievel

Guilhem Martin
email: guilhem.martin@simap.grenoble-inp.fr

Producing Ni-base Superalloys Single Crystal by Selective Electron Beam Melting

Keywords: Electron Beam Melting, Ni-base superalloys, Single Crystal, Grain selection, Crack-free

The possibility to produce Ni-base superalloy single crystals by selective electron beam melting (S-EBM) is demonstrated. The production of single crystals specimens was achieved by a tight control of the processing conditions without requiring a grain selector or a crystal seed. The melting parameters are controlled so as to promote columnar grains and intensify the competitive grain growth.

Additive manufacturing (AM) is a breakthrough for the production of complex components providing a high degree of freedom in the design of parts and offering the possibility to tailor the microstructure. Since the pioneering work of Hunt [1,2], the magnitude and direction of the thermal gradient as well as the solidification velocity are the keys parameters to modify the solidification structure. In the literature of S-EBM (Selective Electron Beam Melting), it is suggested that changes of the melting strategies enable to alter the thermal gradient and solidification velocity to achieve different grain structures and texture. Helmer et al. [3,[START_REF] Koerner | Eurosuperalloys 2014 -2nd European Symposium on Superalloys and Their Applications[END_REF] reported that grain size and morphology can be controlled. Dehoff et al. [START_REF] Dehoff | [END_REF]6] have demonstrated that spatially controlled texture within an Inconel 718 part can be achieved. Such microstructure engineering approach was further investigated by Helmer et al. [7] and Raghavan et al. [8,9]. They showed that different scanning strategies enable to control the thermal gradient magnitude and direction resulting in different grain morphologies and sizes. Processing non-weldable Ni-based superalloys by S-EBM appears much more challenging as severe hot cracks may develop [10][11][12]. Ramsperger et al. have reported the possibility to produce crack-free specimens of CMSX-4 [10]. A recent investigation focused on the mechanism at the origin of the development of hot cracks in a non-weldable Ni-based superalloy fabricated by S-EBM showed that only High Angle Grain Boundaries (HAGBs) are affected by cracks while the Low Angle Grain Boundaries (LAGBs) remain systematically uncracked [12]. This suggests that if one succeeds to tailor the microstructure so as to remove any HAGB, i.e. to generate a single crystal (SX), crack-free samples can be produced. Moreover, SXs are the most efficient way to increase the creep behaviour of Nibased superalloys in the high temperature regime. Achieving SX through AM is challenging. The literature dealing with the repair of SX components by AM processes is limited [13][14][15]. Selective Laser Epitaxy (SLE) or Direct Metal Deposition (DMD) were the first AM processes allowing the deposition of several SX layers on the CMSX-4 [14,15] or René N5 alloys [13]. If one considers the possibility to produce bulk SX using AM processing routes, the literature becomes even poorer. The possibility to generate bulk CMSX-4 single crystals by S-EBM was only reported in a conference paper by Ramsperger et al. [START_REF] Rampsberger | Electron Beam Melting of the single crystalline Ni-base superalloy CMSX-4: from columnar grains to single crystal[END_REF].

In the present contribution, thanks to a tight control of processing parameters and strategy, crack-free bulk single crystals of a non-weldable Ni-based superalloy are produced by S-EBM without requiring a crystal selector or a seed.

The studied alloy is a non weldable Ni-Co-Cr-Mo-Al-Ti-B Ni-based superalloy containing significant amounts of Cr, Co and Mo with Ti + Al wt.% = 8.6. The exact chemical composition cannot be indicated for industrial confidentiality reasons. The prealloyed powder was produced by gas-atomization and provided by ERASTEEL. The asreceived powder particles exhibit mostly a spherical morphology with some satellites (D50 = 77 μm and powder bed relative density = 53.6%, see [12] for more details). The prealloyed powders were used into an ARCAM A1 EBM machine operating at 60 kV accelerating voltage under a controlled pressure of 2.10 -3 mbar. The powder was deposited by 50 μm thick layers on a stainless steel plate, then slightly sintered with a defocused beam and finally selectively melted according to the input CAD geometry. The operating temperature was ~1020°C. Samples with dimensions of 23 × 23 × 30 mm melted in manual mode, i.e. with the automatic ARCAM functions disabled, were fabricated by varying the scan speed (v = 500-1500 mm/s) and beam power (P = 225-600 W). Such a processing window was chosen to ensure a high bulk density (> 99.5%, no lack-of-fusion defects) while avoiding significant overmelting leading to uneven surfaces [3,[START_REF] Chauvet | Additive Manufacturing of Inconel 625 Superalloy Producedby Electron Beam Melting[END_REF]. This was achieved by using linear energies (El = P/v) between 0.3 and 0.6 J/mm. The melting strategy and focus offset (20 mA) were held invariants. The melting strategy consists of scanning the area defined by the CAD file in a snake-line way with a line offset of 0.1 mm and a line order set to 1. The scanning direction was rotated by 90° after every layer. The building time for one layer was about 160 s for a build consisting of 9 samples. The microstructures were characterized by optical microscopy, SEM (ZEISS Gemini SEM500) and EBSD along the longitudinal cross sections (XZ plane). The microstructures were revealed using a fresh Glyceregia etchant [12]. EBSD measurements were performed using the OIM-TSL software.

A typical as-built sample is shown in Figure 1a. The observations were done in the XZ plane, see Figure 1a. Figure 1b shows an example of the region located just below the top surface. Figure 1b is analogous to the typical microstructure of a weld since the melted zone, the heat affected zone and the "base material" (built material unaffected thermally by the melted zone) can be distinguished. The terminology "base material" might be excessive here as it can be thermally affected but only during the preheating stages. The height of the last melted zone (depth of the melt pool) can be estimated based on the microstructural characterization performed in the top region, see example in Figure 1b. High power combined with slow scanning speed, corresponding to the highest linear energy (El = 0.6 J/mm), lead to the deepest melt pools. As expected, for a given power, the melt pool depth increases when the scanning speed decreases. Similarly, for a given scanning speed, the melt pool depth increases with power. Within the processing window selected for a 50 μm layer thickness, melting occurs from 3 up to 12 layers down corresponding to melt pool depths varying between 150 μm (El = 0.3 J/mm) and 600 μm (El = 0.6 J/mm).

The region just below the top surface is of particular interest because it is the only region where the dendritic microstructure inherited from the solidification can be observed. Below this region, the dendritic microstructure is partly or even completely erased by the fact that the underlying layers have been significantly thermally affected during the addition of new layers. It was thought that this rather complex thermal path undergone by the underneath layers could be seen as a sort of solutionizing heat treatment erasing the dendritic microstructure and changing drastically the precipitation state as shown in [12]. Primary dendrite trunks can be easily seen in this last melted zone as shown in Figure 1c. Primary dendrite arm spacing (PDAS), estimated by the intercept method, was found to vary between ~10 and 30 μm depending on the linear energy. The highest linear energy (El = 0.6 J/mm) within the identified processing window gives the highest PDAS (30 μm). Such PDAS values can be used to infer information regarding the cooling rate from literature data correlating PDAS with cooling rate for several Ni-based superalloys [START_REF] Whitesell | [END_REF]. A PDAS of 10 μm suggests a cooling rate of the order of ~500°C/s while a PDAS of 30 μm corresponds to a cooling rate of ~30°C/s (one order of magnitude smaller) [START_REF] Whitesell | [END_REF]. PDAS values with the same order of magnitude though a bit smaller (between 3 and 10 μm) were reported recently for Inconel 718 [9]. This is consistent with the localized melt-scan strategy used in [9] leading to higher cooling rates whereas a raster melt-scan strategy was used here. Microstructural observations in the entire XZ cross section reveal typical features of parts manufactured by AM. The edges of the samples were more or less affected by the local thermal conditions since in those regions, heat was not only extracted by the built material but also by the surrounding powder bed. In this contribution, the attention is focused on the central region of the samples. Melting parameters leading to deep melt pools were deliberately selected to promote epitaxial growth along the building direction. Columnar grains growing epitaxially along the build direction across several layers were found in the central region, see Figure 2a-c. Those columnar microstructures are typical of directionally solidified materials, see e.g. [19,20].

Over the first millimeters, all samples exhibit a similar microstructure consisting of relatively narrow columnar grains (region A in Figure 2a-c). With further addition of layers, texture develops and becomes strongly oriented with a [001]-fiber texture and larger columnar grains grow [12]. The main microstructural differences between the samples are (i) the columnar grain width and (ii) the selection of grain orientation which is more or less pronounced depending on the processing parameters. Figure 2a shows a sample exhibiting an homogeneous microstructure with relatively fine columnar grains (grain width ~100 μm). This sample was melted using a linear energy of 0.3 J/mm. The associated melt pool depth was found to be 150 ± 10 μm and the PDAS, estimated to 13 ± 1 μm. In Figure 2b is shown a sample and exhibiting a gradient of columnar grain width, their width being close to one millimeter in the upper part (Figure 2b). The melt pool depth was about 260 ± 10 μm and the PDAS about 16 ± 1 μm. Figure 2c, melted with a linear energy of 0.45 J/mm, shows another example where the gradient of columnar grain width was strengthened (columnar grain width of several millimeters in the upper part) using parameters leading to a melt pool depth of 366 ± 10 μm and a PDAS of 23 ± 1 μm. Those observations suggest that with appropriate changes of the processing parameters, the gradient of columnar grain width can be exacerbated by intensifying the grain growth competition until obtaining a single grain orientation after less than 10 mm of built material. This was achieved when using a linear energy 0.6 J/mm leading to a melt pool depth of 532 ± 10 μm and a PDAS close to 30 μm, see Figure 3a where no grain boundaries can be seen in the upper central part. To summarize, within the processing windows investigated here, the growth of a single crystal is promoted by using high linear energies (typically 0.6 J/mm) to (i) promote epitaxial growth through deep melt pools and (ii) achieve relatively slow cooling rates (highest PDAS) for the S-EBM process to instigate the grain growth selection mechanism while ensuring a high bulk density (El < 0.3 J/mm) and avoiding overmelting (El > 0.6 J/mm). To demonstrate that this region is indeed a single crystal, EBSD measurements were performed on different XZ cross sections. IPF-maps presented in Figure 3b-d clearly show the presence of a single crystal. The texture plots displayed in Figure 3c show that a single orientation is present. This single crystal sample was found to be crack-free confirming that removing HAGBs is a solution to suppress cracks [12]. To get insights into the mechanism of grain orientation selection leading to a single orientation, additional EBSD measurements were performed in the transition region between the polycrystalline region and the single crystal one, see Figure 4. Two regions can be distinguished in Figure 4a-b: the lower region consists of several columnar grains whereas the upper part consists of a single orientation. In the lower part, the columnar grains are strongly textured, their [001] direction being almost parallel to the building direction (Figure 4b). This is consistent with the fact that in FCC metals, the preferential grain growth directions are the <001> directions. Here, only the [001] direction develops because it corresponds to the direction of the main thermal gradient. However, there are significant misorientations in the XY plane as indicated by the IPF-X map in Figure 4a (two IPF maps are required to fully describe a crystallographic orientation). This is further illustrated by the IPF-Z map displayed in Figure 4c where all the grains having a misorientation smaller than 15° respect to the single crystal orientation are highlighted in blue. When the misorientation angle respect to the single orientation is set to 2°, fewer grains are highlighted in blue, see Figure 4d. The upper part is easier to describe since it consists of a single orientation with the building direction and X-direction aligned respectively with the [001] and [110] directions. The selection of a single grain orientation occurs without the requirement of a single crystal start plate acting as a seed or a crystal selector. From the first few layers, a competitive grain growth occurs and intensifies leading to the disappearance of the misoriented grains, i.e. those having their [001] direction not perfectly aligned with the direction of the main thermal gradient. As a result, a strong [001]-fiber texture develops because grains even slightly misoriented respect to the [001] direction tend to disappear in favor of the perfectly aligned ones. If a strong [001]-fiber texture develops, it is not enough to account for the selection of a single orientation. Indeed, grains can be oriented such as having their [001] direction aligned is not yet elucidated though it is suspected to be related to the 90° rotation of the scanning direction after every layer. In the conventional processes using a seed, a single orientation is grown epitaxially. In the single crystals produced here, in the first few layers, there are few grains possessing the [001]-direction perfectly aligned with the Z-direction and the [110] direction perfectly aligned with the X-direction. Those grains with a very similar orientation grow at the expense of the other, and coalesce leading to a 2 cm 3 single crystal (Figure 4c-d). To ensure the reproducibility of the single crystal growth, another build was run with similar geometry, preheating temperature and melting strategy. Single crystals were successfully grown showing the consistency of the identified melting parameters and strategy. Nevertheless, a change of the preheating temperature or geometry of the samples, would likely require changes in melting parameters or strategy to maintain favorable conditions for single crystal growth, meaning that depending on the geometry of the parts, it might be very challenging to preserve the conditions to grow single crystals.

The results show that within the processing windows investigated, we are clearly in a columnar growth regime where the competitive grain growth can be manipulated. Interpretation of the above-described grain structures can rely on the analogies between S-EBM and directional solidification such as Bridgman crystal growth. Indeed, the melted zone shown in Figure 1b can be assimilated to the melted zone of the ingot in the Bridgman furnace while the underlying layers in the central part of built material act as the cold zone in the Bridgman process, extracting heat in the opposite direction to the growth direction. Besides, at the end of each layer, the build plate goes down by a 50 μm-increment corresponding to the layer thickness. The layer time being constant throughout the build, the build plate down-velocity is constant similarly to the pulling direction in the Bridgman process.

Identifying the reasons why some processing conditions lead to a single crystal and others not is not an easy task and would require simulations including thermal considerations, melt pool dynamics and morphology as well as local solidification conditions. However, based on the present results, one can give some insights into the effect of some physical parameters on the grain selection mechanism. First, the effect of the melt pool geometry on the dendritic grain growth direction can be invoked. This has been investigated by Liu et al. [21,22]. The authors have established some links between the melt pool geometry and the number of selected <001> growth variants and to the relative size of each growth region. Here, the situation is more complicated because the melt pool geometry is not only affected by the melting parameters (beam power, velocity and linear energy) but also by the melting strategy. Moreover, high cooling rates are favorable towards fine grains unlike slower cooling rates that results in larger grains. This is consistent with our results since the samples with the largest grain width are those which were cooled with the slowest cooling rate, as shown by the PDAS measurements. To a certain extent, it indicates that tuning the melting parameters enables to control the solidification conditions (thermal gradient and growth rate) and consequently the location of the transition between a polycrystalline and monocrystalline region along the building direction.

A tight control of the melting parameters enables to produce bulk single crystals by S-EBM. This does not require the use of a crystal selector or a seed since melting parameters leading to the intensification of the competitive grain growth turns out to be enough.

Figure 1 .

 1 Figure 1. (a) Typical S-EBM as-built sample. Optical micrographs showing (b) an example of the region located just below the top surface and (c) the dendritic microstructure of the last melted layer.

Figure 2 .

 2 Figure 2. Optical micrographs showing examples of the grain structures obtained in manufactured samples. (a) Homogeneous microstructure with narrow columnar grains (~100 μm). (b) and (c) Graded microstructures with columnar grains of about 1 mm and several millimeters respectively in width in the upper part. Cracks are highlighted in black.

Figure 3 .

 3 Figure 3. (a) Sample exhibiting a monocrystalline central region. (b) IPF-Z map acquired in the central part. (c) (001) and (111) Pole Figures showing the orientation of the single crystal in the upper part. (d) IPF maps of the whole transverse section demonstrating the presence of a single crystal over a width of more than 15 mm.

  direction while being rotated around the [001] direction in the (XY) plane giving large misorientation, see typically the bottom region of Figure4a. With the addition of new layers, the competitive grain growth is exacerbated until a single orientation grows at the expense of the others. The orientation corresponding (i) to the alignment of the [001] direction along the building direction and (ii) to the alignment of the [110] direction along the Xdirection grows at the expense of the others, see Figure4a-b. The fact that the single crystal has its [110]-direction aligned with the X-direction (one of the scanning direction),

Figure 4 .

 4 Figure 4. Transition region between the polycrystalline region (bottom of the sample) and the single crystalline region (top of the sample). (a) IPF-X map. (b) IPF-Z map. (c), and (d) IPF-Z maps with highlighted in blue the orientation of the single crystal respectively with 15°and 2° of misorientation. Misorientation of HAGBs > 15° are highlighted in black.

Acknowledgements

This work was performed within the framework of the Center of Excellence of Multifunctional Architectured Materials ''CEMAM'' n°AN-10-LABX-44-01 funded by the ''Investments for the Future Program''. The authors are grateful to Poly-Shape for their contribution to funding this work.