Tuyen P T Dao

Fabio Bras Fernandes
email: fernandesf@tecnico.ulisboa.pt

Martin Fauquignon

Emmanuel Ibarboure

Manuel Prieto

Jean-François F Le Meins
email: lemeins@enscbp.fr

à la diffusion de documents scientifiques de niveau

Keywords:

recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers,

The combination of block copolymers and phospholipids to form Giant Hybrid Unilamellar Vesicles (GHUVs) does not systematically lead to "intermediate" membrane properties.

Introduction

Hybrid polymer/lipid vesicles are the subject of an increasing number of studies by different scientific communities (bioengineers, chemists, physical chemists, pharmacists, biochemists). The driving force behind this surging interest is the design of entities that could combine all the benefits of each component (biofunctionality, permeability of lipid phases, mechanical stability, chemical versatility of polymer phase…). These emerging systems have already been the subject of two reviews. 2, 3 A significant number of studies have focused on their use in different fields, such as drug delivery 4-7 , drug targeting, 8 nanoreactors, 7, 9, 10 biomolecular recognition and interaction with nanoparticles 11-14 , or their interaction with biological media. 15 So far, only a relatively moderate part of the work published on the subject has focused on the selfassembly problematic: how can we modulate the phase separation process during the self-assembly, preventing formation of separated liposomes and polymersomes, and obtain membranes with lipid domains of controlled size? The relation-ship between membrane composition (% polymer lipid, presence of domains) and properties like elasticity, permeability, tumor-targeting ability, is not clearly established so far, especially at the nanoscale in Large Hybrid Unilamellar Vesicles, LHUV, often used in the above mentioned studies. Indeed, proof of hybrid character and information about membrane structuration are not easy to obtain at nanoscale. DLS and DSC 4 have been used to evaluate the efficiency of polymer/lipid mixing but a real quantification is very hard to reach. Moreover, different structures can be obtained (e.g., wormlike hybrid micelles, or tubular vesicles), depending on the molar mass of the copolymer and lipid fractions. The presence of such heterogeneous structures, most of the time revealed by CryoTEM analysis 9,16-18, can limit the optimization of the benefits/advantages observed in different applications (e.g. drug delivery). Time resolved fluorescence spectroscopy techniques have been shown to be very useful to probe the efficiency of polymer/lipid mixing at the nanoscale. 17 Especially, dual-colour fluorescence cross-correlation spectroscopy (DC-FCCS) appears to be an efficient and elegant way to prove and quantify the hybrid character of vesicles at the nanoscale 19 . Fluorescence Lifetime Imaging technique (FLIM) is also a powerful approach to probe membrane structuration in GHUVs. 1 So far, most of the existing studies focusing on the understanding of parameters governing membrane structuration have been made on giant vesicles obtained by electroformation.

1, 20-24 Information has also been obtained through evaluation of nanoparticle-membrane interaction.

11 Globally, hydrophobic mismatch resulting from the difference of thickness between polymer and lipid hydrophobic cores, as well as lipid fluidity, have been identified as important parameters in membrane structuration. The cooling process after formation of giant vesicles at temperatures above the melting point of phospholipids, is also of paramount importance.

1, 20, 24

Scarce information is available regarding the mechanical properties of these hybrid vesicles. Area expansion modulus (or stretching modulus), lysis strain and stress have been estimated for two systems PBut-b-PEO/POPC 21 and PDMS-g-PEO/DPPC 24 , in the presence of a specific molar fraction of lipid (30%). In the case of PBut-b-PEO, values for the stretching modulus were found to be intermediate between pure lipid and pure polymer membranes (e.g. 17% higher for hybrid vesicles than that of pure polymersomes), whereas the effect was far less pronounced for PDMS-g-PEO/DPPC vesicles (7% higher stretching modulus for hybrid vesicles), which present patchy domains of DPPC for such fraction. Regarding membrane fluidity, the mobility of lipids has been evaluated by Fluorescence Recovery After Photobleaching (FRAP) measurements in PBut-b-PEO/POPC 21 and PIB-b-PEO/DPPC GHUVs 13 and has been shown to be modulated by the amount of polymer. However their diffusion within the membrane seems to be complex in PBut-b-PEO/POPC Membrane elastic properties of giant vesicles obtained from the self-assembly of triblock copolymers are unknown, and so far data are only available for diblock, [25][26][27][28][29] or grafted copolymers 30 . In addition, the membrane properties resulting from their association with fluid phospholipids are also unknown. It is interesting to note that differences in terms of membrane fluidity between diblock and triblock copolymers have recently been shown in a systematic FCS study performed on Giant Unilamellar Vesicle (GUV). 31 It was suggested that triblock copolymers adapt mixed conformations between an extended shape and a U-shape that may reduce their mobility.

In this work, we propose to analyse and discuss the elastic properties and membrane fluidity of Hybrid Giant Unilamellar Vesicles resulting from self-assembly of triblock copolymers and POPC. Micropipette aspiration technique (MPA), 32, 33 and FRAP measurements were carried out with that purpose. Two different molar mass of PDMS were considered, as this parameter is of importance regarding the structuration of hybrid membrane as mentioned previously. In addition, the influence of copolymer architecture on the polymersome's membrane properties will be shortly discussed.

Materials and methods

Materials

1-Palmitoyl-2-oleoyl-sn-glycero-3-phosphocholine (POPC) and 1,2-dioleoyl-sn-glycero- , and all the results were in agreement with data given by the manufacturer and data available in literature 35,36 This copolymer forms vesicles with a membrane thickness close to 5 nm. We used this copolymer to validate our MPA procedure, as GUVs of this copolymer have already been studied by MPA and their stretching elasticity is well established. Fluorescein-labelled graft copolymer (PDMS 26 -g-(PEO 12 ) 2 -F) was prepared as previously described. 23 Triblock copolymers have been synthesised following a procedure described in previous studies

Generation of GUVs

GUVs were generated by the electro-formation method, which is based on the process reported by Angelova. 37 Coverslips were cleaned with acetone and ethanol. The initial lipid or polymer solutions were prepared in a chloroform/methanol mixture (2/1 vol/vol) at a concentration of 1mM. For FRAP and MPA experiments, DOPE-Rhod was included at 0.2 mol % of lipid content, while Dow-F was included at 1.5 mol % of polymer content. For electro formation, about 3 µL of the solution was spread slowly on both sides of Pt electrodes. ITO plates (resistivity 15-35Ω .cm) were also used. After evaporation of the solvent during at least 5-6 hours, the electrodes were connected to AC voltage (sinusoidal tension 2V, 10Hz) and submerged in a 100 mM sucrose solution. Electro formation was performed at room temperature both for POPC or triblock copolymers. Samples were collected after 75 minutes and stored at room temperature, in the dark for samples containing fluorescent analogues, until measurements.

Confocal fluorescence microscopy

Confocal fluorescence microscopy was performed on a Leica TCS SP5 (Leica Microsystems CMS GmbH, Mannheim, Germany) inverted confocal microscope (DMI6000). A 63x apochromatic water immersion objective with a NA of 1.2 (Zeiss, Jena Germany) was used for all experiments. Dow-F and DOPE-Rhod excitation were achieved with the 488 nm and 514 nm argon laser lines respectively, while fluorescence emission was collected in the 500-530 nm range for Dow-F and in the 600-700 nm range for DOPE-Rhod.

Fluorescence recovery after photobleaching

Membrane fluidity (or viscosity) has an impact on both molecular rotational, and translational diffusion rates, described through the translational diffusion coefficient (D). In this section, the translational fluidity of the membrane is reported via Fluorescence recovery after photobleaching (FRAP) measurements. The phospholipid analogue DOPE-Rhod was used to study the mobility of phospholipids, whereas fluorescein modified grafted copolymer Dow-F was used to evaluate mobility of polymer chains. Investigations on both pure liposome and polymersome were performed in a first step in order to obtain reference values and validate all setting and experimental procedures. For the FRAP experiments, the same confocal microscope setup was used, using the above described conditions for excitation/emission. FRAP measurements of Dow-F were started with 10 image scans at low laser intensity (488 nm). Then Fluorescein was photobleached inside a region of interest (ROI) sized 5 µm, using a scan of 3 frames at high laser intensity (488 nm). Finally, the fluorescence recovery was monitored by the acquisition of a series of 150 images with the same low laser intensity as the pre-bleach. In all steps, the images were acquired using a frame size of 256 x 256 pixels and bidirectional scanning at a 1400 Hz line frequency scan speed, which gave a time-lapse of 113 ms per image. All measurements were made at T = 20°C ± 1°C. To fully immobilize the vesicles during acquisition, biotinylated lipid (10 -4 mol %) was added in the initial lipid mixture. Vesicles obtained were transferred to the wells of an eight-well µ-Slide from Ibidi (Munich, Germany) previously coated with avidin, 15 minutes before measurement. The avidin coating was carried out through incubation of 200 µL of an avidin solution at 0.1 mg/ml during at least 1h at 4°C and subsequently washed with MilliQ-water to remove all the excess avidin. The FRAP data was first normalized to take into account the variations in the absolute amount of fluorescent molecules between samples. In this way, the relative fluorescence changes after photobleaching become proportional to the initial values and independent of fluorophore concentration. The method used in this work was double normalization:

I Norm (t) = I Ref_pre I Ref (t) -I Bck (t) • I Frap (t) -I Bck (t) I Frap pre (Eq. 1)
where I Norm (t): normalized intensity, I Frap (t): measured average intensity inside the bleached spot, I Ref (t): measured average reference intensity and I Bck (t): measured average background intensity outside the ROI (Region Of Interest). Subscript _pre means the averaging of intensity in the corresponding ROI, immediately before bleaching and after subtraction of background intensity.

All experimental data was fitted with the circular spot model in 2D diffusion, described by:

I Norm (t) = a 0 + a 1 . e τ 2(t-t bleach ) . �I 0 � τ 2(t -t bleach ) � + I 1 � τ 2(t -t bleach ) �� (Eq. 2)
where a 0 and a 1 are constants and t bleach the characteristic time for diffusion. I 0 and I 1 are the modified Bessel function of order 0 and 1. Analysis of FRAP data was performed using the FRAP analyser software version 1.0.5 (University of Luxembourg, Cytoskeleton and Cell Plasticity Laboratory, http://actinsim.uni.lu/). The mobile fraction was also evaluated, to determine the presence of immobile fluorescent molecules that cannot participate in the exchange between bleached and unbleached regions, resulting in an incomplete recovery of the fluorescence signal. The mobile fraction is defined by:

M f = 1 -IM f (Eq. 3) With IM f = I i -I ∞ I i -I 0 (Eq. 4)
IM f is referred as the immobile fraction and I i , I 0 and I∞ are the fluorescence intensity at initial pre-bleached time, bleached time and at maximal recovery respectively.

Micropipette aspiration

Micropipette aspiration technique was carried out also on a confocal microscope. The micropipettes were obtained by stretching borosilicate capillaries (1mmOD, 0.58mmID) from WPI, with a pipette puller (Sutter Instrument P-97). The pulled pipets were then forged to the desired diameter (6-8 µm) using a micro-forge Narishige MF-900. Before using, micropipettes were coated with BSA to prevent vesicle adhesion, by suction via the pipette of a filtrated BSA solution at 0.1%. Thereafter, the pipettes were filled with appropriate solution (0.1M sucrose, water...) and connected via flexible tubing to a micromanipulator (Eppendorf, Patchman NP2). The vesicle tension was controlled by a home-made hydraulic watertight setup. The suction pressure exerted over the membrane is calculated from Equation 5:

DP = (h -h 0 )ρ w g (Eq. 5)
where ρ w is the density water (ρ = 1 g.cm -3 ), g is the gravitational acceleration (9.8 m.s -2

), h is the position of the water reservoir and h 0 is the initial position where the pressure is zero. The membrane tension was calculated classically from the Laplace equation:

σ = P s × R p 2(1- R p R v ) (Eq.6)
where R p and R v are the micropipette and vesicle radius (outside the micropipette), and P s is the suction pressure on the micropipette. A relative area change of the membrane (α) is defined as:

α = A -A 0 A 0 (Eq. 7)
A 0 being the membrane area of the vesicle at the lower suction pressure. α can be estimated from the increase in projection length ΔL of vesicle inside the capillary tip according to Eq.8:

α = 1 2 × R p R v 2 × ∆L × �1 - R p R v � (Eq. 8)
Under very low tension regime, the apparent expansion is dominated by smoothing of thermal bending undulations. Plotting ln(𝜎) vs. α at low-𝜎 values (typically 0.001-0.5 mN.m-1 (32) , gives a straight line whose slope is related to the bending modulus, κ (Eq.9). Under higher tensions, which were investigated in this work, (> 0.5 mN.m -1 ), membrane undulations are totally suppressed and membrane area increases as the result of increased spacing between molecules. In this regime, the rate of membrane area increase with tension can be used to calculate stretching or area expansion modulus Ka according to Eq.9. 38 In the investigated tension regime, bending contribution is neglected and only the second term is considered to estimate Ka.

α ≈ k B T 8πκ ln(σ) + σ K a (Eq.9)
Each parameter measured in this work is taken from an average value of at least 15 vesicles unless specified. The vesicles were analysed one day after electro formation.

RESULTS AND DISCUSSION

Pure Polymersomes and Liposomes

In order to validate our micropipette aspiration experimental procedure, measurement of stretching elasticity and lysis strain were performed on polymersomes obtained from selfassembly of grafted copolymer based on Poly(dimethylsiloxane) and poly(ethyleneoxide) Dow. The stretching elasticity of such membrane has already been studied previously, 24, 30 and this polymer has also been associated with phospholipids to generate GHUVs . Therefore this copolymer can be used as a reference.

The applied tension versus areal strain for the different copolymer used in this study is represented in Figure 1. From the slope the area expansion modulus is determined, and the values are indicated in and confirm the robustness of our protocol. Measurements were performed on pure polymersomes made from 3K triblock copolymers, and the recovered area expansion modulus (84 ± 13 mN/m) is only slightly higher than the one obtained for 1.5K (69±9 mN/m), which has a lower molar mass and whose membrane mechanical properties has been evaluated in a previous study. 40 The amplitude of the area expansion modulus is normally linked to the interfacial tension between blocks, and is not modulated by their molar masses. 28, 41 However, the lysis strain is more important for the 3K, and reflects an increase of the membrane toughness with the molar mass as already proven in a previous study. 41 It is interesting to note that although these experiments are known to be complex to carry out, and are known to generate values with a relatively high uncertainty, a slight difference between the grafted copolymer and triblock copolymer seems to be observed, and suggests an influence of the copolymer architecture on the stretching elasticity of the membrane. This has to be confirmed in further studies. The diffusion coefficient in polymersomes and liposomes membrane has been also measured by FRAP, following the procedure described previously. The results are summarized in Table 3. FRAP curves can be consulted in supporting information. (Fig S3)

Following Saffman Delbück's continuum hydrodynamic model 42 describing lateral and rotational diffusion of cylindrical objects moving in a two dimension fluid (e.g. a lipid membrane), the lateral diffusion coefficient (D) can be expressed as:

D = k b T 4πηh �ln � ηh η′R � -γ� (Eq. 10)
where k b is the Boltzmann constant, T is absolute temperature, h is the thickness of the bilayer, η is viscosity of the membrane, η' is viscosity of the outer liquid, R is the radius of the diffusing object and γ is Euler's constant. Using this equation, the membrane viscosities were calculated from our measurement of diffusion coefficient. The hydrophobic thicknesses d of the polymersomes measured by Cryo-TEM 17 were used as the thickness h of the bilayer in the calculation for the polymersome membrane. R was approximated to be close of the radius of gyration Rg = N α (b/6), where N is the number of siloxane units, b is the segments' length (Si-O-Si) (= 0.311 nm) and α is the scaling exponent (= 0.66) found for membrane thickness of triblock copolymers (ESI of 17

). Another approach consists in using the area/chain found in a vesicle to evaluate R. Table 2 summarizes the membrane characteristics obtained for the different copolymers. Diffusion coefficients are in good agreement with those reported previously on the PMOXA-b-PDMS-b-PMOXA triblock copolymer, for similar hydrophobic block molar mass. 31 Interestingly, the membrane viscosity for the grafted copolymer is slightly lower than the membrane viscosity of the triblock 1.5K, while the molar mass of hydrophobic block is larger. Again, this suggests that copolymer architecture play a role on viscoelastic properties of membrane.

Giant Hybrid Unilamellar Vesicles

Regarding stretching elasticity measurements, we will particularly focus on polymer/lipid compositions for which homogenous membrane at the microscale have been reported, but in which lipids nanodomains are present. In order to assess the influence of the presence of lipids in polymersomes on membrane properties, pipette aspiration experiments were performed on GHUVs, which present a relatively extended range of lipid fraction for which homogenous membrane structuration at the microscale has been reported. Such vesicles were obtained with the 3K triblock copolymer. 1 The results obtained are illustrated in Figure 2. Interestingly, whereas the few data available reported in literature mention a modulation of stretching moduli and lysis strain in between the value of pure lipid and polymer vesicles, here an increase of stretching moduli is observed when POPC is added, but a drastic decrease of lysis strain is observed even for 5% w/w lipid. The lysis strain obtained are even lower than the lysis strain of pure POPC vesicle (~5%), resulting in vesicles presenting a toughness even lower than pure liposomes. This result is quite unexpected, and suggests that if the association of lipids and thicker copolymer chains is effective, as attested by the regular increase of the area expansion modulus (Figure 3), instabilities are created probably at the polymer/lipid boundaries. It is worth to recall that the existence of lipid nanodomains has been proven previously by FLIM/FRET experiments. 1 These nanodomains are in a metastable state and the formation of micrometric lipid domain under tension has been observed for lipid fractions close to those where phase separation occurs at rest (>30% w/w). In our case, before rupture, no micrometric domains have been observed. This suggests that when lipid fractions are low enough, coalescence between lipid domains that allow the decrease of the boundary energy at the lipid/polymer interface is not possible, or is not rapid enough regarding the increase of the line tension at the interface that occurs under pipette aspiration. 45 This increase of line tension could thus be at the origin of creation of defects in the membrane leading to its premature disruption. Again this effect could be related to the triblock architecture of the copolymer. The behaviour under stretching of hybrid vesicles with a PDMS-g-PEO grafted copolymer, or a PBut-b-PEO diblock copolymer has been studied, although in a restricted range of lipid fraction, and any sign of appearance of embrittlement of the vesicle structure has never been detected, even in the case of a PDMS-g-PEO grafted copolymer, for which hybrid vesicles have been studied by MPA in a composition range where lipid patchy domain were present. The values obtained from different measurements of area expansion modulus are shown in Figure 3 .

It is interesting to note that the dispersity of Ka values is very high. At 20% of POPC, it seems that two types of vesicles are present in the sample, one with a low Ka, slightly higher than the pure copolymer, and another with significantly higher Ka (~ 155mN.m -1 ). This probably reflects the heterogeneity in vesicle composition due to the fission events commonly observed with 3K/POPC GHUVs. 1 These events lead to the formation of GHUVs with considerably different membrane compositions.

Lateral diffusion in GHUVs

Translational diffusion measurements of lipid molecules and copolymer chains were carried out for GHUVs presenting a homogenous and heterogeneous distribution of the component at the microscale. In short, for the 1.5K/POPC mixture, GHUVs with a POPC content of 10%, 30% and 50% were analysed. Based on our previous study 1 , GHUVs with 10% POPC are expected to present lipid nanodomains larger than 5 nm but non-visible under microscopy, whereas samples with 30% and 50% POPC show generally visible lipid microdomains. Hence, in FRAP measurements with only polymer fluorescent probe, GHUVs composed of 10% POPC were seen as homogenous as shown in Figure S 4-a, whereas dark domains were observed in GHUVs composed of 30% or 50% POPC (Figure S-4-b) since Dow-FITC does not incorporate in lipid domains at these high lipid fractions. FRAP measurements in these samples were always performed in the polymer-rich phases.

We first tried to evaluate the mobility of polymer chains. The typical FRAP curves for 1.5K/POPC GHUVs at each aforementioned POPC fraction are presented in Figure 4. The effect of POPC on the diffusion of the polymer chains can be qualitatively seen, particularly concerning the mobile fraction. The diffusion coefficients of copolymer chains values versus POPC concentration are represented in Figure 5, and all results are summarized in Tables 4 and5.

Diffusion coefficients of polymer molecules in hybrid vesicles seem to be unmodified at a low POPC fraction and then decrease when the POPC fraction increases. The mobile fraction however is slightly modified even at a low POPC content. This variation in diffusivity and mobile fraction is very likely linked to the phase separation occurring in 1.5K/POPC GHUVs. Polymer fluorescent marker Dow-FITC has been shown to be almost completely excluded from lipid phase as illustrated in previous studies.

POPC mass fraction (%) Therefore, the diffusion of copolymer chains is hindered due to the presence of these lipid domains. In this case, a small fraction of polymer fluorescent analogue would remain trapped by slower moving nanoscale lipid domains on the bleached ROI (FRAP was always carried out in fluorescent polymer-rich domains). With increasing POPC content, the diffusion coefficients decrease as a result of the presence of more lipid domain obstacles to diffusion.

Similar measurements were carried out for 3K/POPC GHUVs. Interestingly, when POPC content increases, only a slight decrease of diffusion coefficient is observed in this system, as displayed in Figure 5 and Table 5, for GHUVs that do not present phase coexistence under microscope.

In these lipid compositions for the previous block copolymer, variations were more pronounced. This may be due to the higher line tension observed in this system, which decreases the extent of formation of nanoscale lipid inclusions within polymer phases. In the 3K/POPC mixture, the higher line tension promotes fission of lipid domains at significantly higher rates than the ones observed for 1.5K/POPC GHUVs as discussed previously. Interestingly, for samples showing micro-domain coexistence under the confocal microscope (3K/POPC GHUVs composed of 50% POPC), there was no clear difference in both diffusion coefficient and mobile fraction values in comparison with pure polymersomes. This would suggest that there are no more lipid nanodomains in the polymer-rich phase (but there is still some lipid incorporated in the copolymer-rich phase, as Rhod DOPE signal is still detected in confocal images). The explanation for this observation is not obvious. This is probably due to a high line tension at the polymer/lipid boundaries, which drives, above a critical threshold in lipid composition, the existing nanoscale lipid domains to rapidly coalesce into micro-domains. When the lipid composition is close to this critical threshold, micro-domains can also be generated by applying a tension in the membrane as previously shown. 1 In the other system (1.5K/POPC), nanoscale lipid domains remain trapped within polymer-rich domains probably because of lower line tension.

Lateral diffusion of lipid molecules in hybrid membranes

The dynamic of lipid molecules in hybrid membranes was also studied through the diffusion coefficient of DOPE-Rhod. This probe partitions preferentially in the lipid phase but not exclusively as shown in previous studies. In phase-separated GHUVs, the lipid fluorescent signal is too low in polymer-rich phases to perform FRAP measurements, and the lipid-rich phases are rarely large enough to allow reliable measurements. Therefore, the study was only performed with GHUVs presenting homogeneous membrane structure at the micron scale. Based on the apparent phase diagram, 1 measurements were carried out with 1.5K/POPC GHUVs containing 10% and 15% of POPC. Results are shown in Figure 6. Considering the uncertainty of the measurement, there is no difference in diffusion coefficients of lipids in pure POPC and hybrid membranes composed of 10% and 15% of POPC. In this way, the diffusion of the lipid analogue is, unlike the copolymer analogue, not affected by nanoscale phase separation. This suggests that FRAP reveals the lateral diffusion of lipid molecules (and not lipid nanodomains which obviously should have a lower diffusion coefficient). Unlike the labelled polymer, the lipids dispersed in the polymer-rich phase can readily exchange with lipids in nanodomains and as such, these domains do not act as barriers for diffusion. This result also suggests that lipid diffusion in hybrid 1.5K/POPC membrane is insensitive to either a vesicle structuration or a composition of hybrid vesicle. This result is different from the observation reported by Vanderlick and col. 21 in which the lipid lateral-diffusion coefficient in PBd 46 -b-PEO 30 /POPC GHUVs decreased gradually in proportion to the amount of incorporated polymer. However, since the copolymer used possesses a very low fluidity (D = 0.22 ± 0.06 µm².s -1 -44 times lower than pure POPC), its influence on the lipid diffusion is probably more pronounced.

Conclusions

In this work, membrane mechanical properties of GHUVs made from the association of fluid phospholipid (POPC) and triblock copolymer were studied using micropipette aspiration technique. First, the elasticity under stretching of vesicles made from two triblock copolymers with different molar mass were compared to those of well-known grafted commercial copolymers based on the same hydrophobic (PDMS) and hydrophilic (PEO) blocks. The results obtained suggest that the architecture of the block copolymers play a role on membrane toughness, as critical lysis strain was observed to be twice less for triblock copolymers compared to grafted copolymers for the same membrane thickness (or hydrophobic molar mass).

Although further studies are needed, evidence obtained from the experiments demonstrated that chain conformation in the polymersome membrane plays an important role on membrane-mechanical properties. Differences in membrane viscosities have been reported in the literature for diblock and triblock copolymers 31 having the same molar mass, and this was attributed to the presence of a given amount of stretched chain conformation in the case of triblock copolymer. This difference in membrane viscosity has also been observed in this study.

Most importantly, micropipette experiments reveal that the association of triblock copolymer and phospholipids in the same membrane results in a huge decrease of membrane toughness, and leads to the formation of vesicles that are even more fragile than pure liposomes. This was not observed in the few studies available in literature on GHUVs obtained with diblock copolymers. Again, this suggests that the presence of a mixture of hairpin and stretched conformation of polymer chains in the membrane, which is the case for the triblock copolymer, is not really adapted to reach stable edifice in the presence of a low amount of phospholipids. Regarding the mobility of components in hybrid membranes, again the situation is more subtle than in a diffusion simply balanced by the amount of the two components. Polymer chain diffusion is limited by the presence of lipid nanodomains, whereas lipid diffusivity seems to be insensitive to either vesicle structuration or composition of a hybrid vesicle.

Globally, our study shows that the association of polymer and lipid in a single membrane does not lead systematically to "intermediate" properties that could be modulated by the membrane composition. Depending on the architecture of the copolymer used, a great care has to be paid to the characterisation of the membrane structure and the evaluation of hybrid membrane properties. Beyond the fundamental interest of such studies, this is of importance when such structures are considered in biomedical applications. For instance, in drug delivery toughness is important regarding the mechanical stresses that cargos can undergo during blood circulation through vessels. In this still restricted but emerging field of hybrid vesicles/membranes, so far diblock copolymers were most often used, but interest on triblock copolymer architecture is growing. 15 In this context, it is important that the exact role of copolymer architecture on membrane structuration and properties is deciphered, both at the micro and nanoscale. Further studies on GHUVs and LHUVs using diblock copolymers with same chemical nature (PDMS and PEO) are under way.

Figure 2 .

 2 Figure 2. Representative stress -strain curves of 3K/POPC GHUVs at different POPC content: (○): 0% POPC; (○): 5% POPC; (○): 10% POPC (○): 20% POPC.(% in w/w) (○): pure POPC (extracted from previous work of the group 40 )

Figure 3

 3 Figure 3 Variation of area expansion modulus of hybrid 3K/POPC GHUVs with POPC content: the open circles are experimental values obtained for individual GHUVs and the closed circles represent the corresponding averaged value with standard deviation (the symbol is slightly shifted to the right for clarity.

39

 39

Figure 4

 4 Figure 4 Representative FRAP data obtained from GHUVs with different 1.5K/POPC compositions: (○):0% POPC; (○):10% POPC; (○):30% POPC and (○): 50% POPC containing 1.5% molar Dow-FITC. The smooth lines correspond to the fit using the formalisms described in the experimental section.

Figure 5

 5 Figure 5 Diffusion coefficients of fluorescent polymer probe in 3K/POPC GHUVs (•) and in 1.5K/POPC GHUVs (•) as a function of POPC content.

Figure 6 .

 6 Figure 6. Diffusion coefficients of fluorescent lipid probe DOPE-Rhod in hybrid 1.5K/POPC GHUVs as a function of POPC content.Lipid molecules diffuse about 2.5 times faster than polymer chains in the same host hybrid membrane (D DOPE-Rhod = 8.4 ± 1.8 vs. D PDMS 26 -g-(PEO 12 ) 2 = 3.55 ± 0.61 in 1.5K/POPC GHUVs with 10% POPC). Considering the uncertainty of the measurement, there is no difference in diffusion coefficients of lipids in pure POPC and hybrid membranes composed of 10% and 15% of POPC. In this way, the diffusion of the lipid analogue is, unlike the copolymer analogue, not affected by

Table 2 .

 2 Histograms of Ka values for Dow and 3K are available in ESI. (Fig S1and S2). Considering the Dow GUVs, the area expansion modulus as well as lysis strain (α c ) and stress (σ c ) are very similar to those reported in literature (σ c~9 2±5mN/m, α c ~8%)

	30

Table 2 :

 2 Membrane mechanical characteristics of the different polymersomes used in this study.

		Triblock 3K
	Applied tension (mN.m -1 )	Triblock 1.5K Graft copolymer
		Areal strain
	Figure 1: Tension versus areal strain for pure polymersomes. Data for 1.5K copolymer were extracted from a previous study of our group. 40

Table 4

 4 Membrane morphology and corresponding average lateral diffusion coefficients (D ± SD) and mobile fractions (Mf ± SD) of fluorescent polymer analogue in different hybrid 1.5K/POPC membranes.

	POPC (%)	Membrane morphology from 1	D ± SD (µm².s -1 )	Mf ± SD
	0%	-	1.95 ± 0.57	1.00 ± 0.01
	10%	Nanodomains > 5 nm but invisible under microscope	1.59 ± 0.43	0.81 ± 0.08
	30%	Nanodomains > 5 nm but invisible under microscope	1.41 ± 0.20	0.82 ± 0.04
	50%	Micro-domains visible under microscope	2.05 ± 0.41	0.97 ± 0.07

Table 5 .

 5 Lateral diffusion coefficient (D ± SD) and mobile fraction (Mf ± SD) of fluorescent polymer probe in different hybrid 3K/POPC membranes.

| Soft Matter, 2018, 14, 6476-6484 Author manuscript https://hal.archives-ouvertes.fr/hal-01846474

| Soft Matter, 2018, 14, 6476-6484 Author manuscript https://hal.archives-ouvertes.fr/hal-01846474

| Soft Matter, 2018, 14, 6476-6484 Author manuscript https://hal.archives-ouvertes.fr/hal-01846474

| Soft Matter, 2018, 14, 6476-6484 Author manuscript https://hal.archives-ouvertes.fr/hal-01846474

Author manuscript https://hal.archives-ouvertes.fr/hal-01846474Soft Matter, 2018, 14, 6476-6484 | 9

| Soft Matter, 2018, 14, 6476-6484 Author manuscript https://hal.archives-ouvertes.fr/hal-01846474

Acknowledgements

International doctoral school on Functional Materials (IDS-FunMat), Erasmus Mundus (EU), FCT (RECI/CTM-POL/0342/2012, FAPESP/20107/2014, UID/NAN/50024/2013, F.F. is an FCT Researcher -IF/00386/2015)

Conflicts of interest

"There are no conflicts to declare".


Andrea Petreto

Arthur Hennequin

Thomas Koehler

Thomas Romera

Yohan Fargeix

Boris Gaillard
email: bgaillard@lheritier-alcen.com

Manuel Bouyer

Quentin L Meunier

Lionel Lacassagne

Energy and Execution Time Comparison of Optical Flow Algorithms on SIMD and GPU Architectures

published or not. The documents may come

I. INTRODUCTION

Embedded systems must satisfy the antagonist constraints of real-time processing and power consumption, especially for computer vision. Optical flow is a family of algorithms which are used to estimate the apparent velocity of every point in a pair of images. These algorithms are used in a wide range of applications, from movement compensation on cameras to autonomous vehicles.

To estimate optical flow in real time is a challenging task for embedded systems since it requires a lot of computational effort, while the power consumption must remain low. Therefore, the processor used must optimize the performance per Watt ratio, and this ratio is known to be better for small cores [START_REF] Borkar | Thousand core chips: a technology perspective[END_REF]. We believe that specific architectures like SIMD cores and GPU are suitable candidates for achieving the requested computation time while meeting the power constraints. This work considers several such architectures to perform movement detection and evaluate if they would fit the defined constraints.

This article aims to study and compare various implementations of two optical flow algorithms from a speed vs. energy consumption perspective, and highlights the impact of algorithmic and architectural optimizations for SIMD processors and GPUs.

Section II introduces the optical flow algorithms evaluated; section III presents all the optimizations implemented and algorithms variants; section IV presents the measurement protocol along with the target boards; section V presents and analyses the results obtained; and finally section VI concludes.

II. OPTICAL FLOW ITERATIVE ALGORITHMS

More than a hundred optical flow algorithms exist, and vary according to their computational speed, quality of detection, and management of complex situations such as occlusions. An extensive qualitative analysis [START_REF] Baker | A database and evaluation methodology for optical flow[END_REF] of existing algorithms is available on the Middlebury's website [START_REF] Middlebury | Optical flow database[END_REF], which also provides links to some of the codes. CMLA's IPol website [START_REF] Ipol | Image processing on line[END_REF] also provides various codes of recent algorithms. Optimized implementations of optical flow algorithms were the subject of numerous works on FPGA [START_REF] Gultekin | An FPGA-based high performance optical flow hardware design for computer vision[END_REF], [START_REF] Bako | Embedded implementation of a real-time motion estimation method in video sequences[END_REF], [START_REF] Kunz | An FPGA-optimized architecture of horn and schunck optical flow algorithm for real-time applications[END_REF], [START_REF] Chai | Using C to implement highefficient computation of dense optical flow on FPGA-accelerated heterogeneous platforms[END_REF] and on GPU [START_REF] Plyer | Massively parallel lucas kanade optical flow for real-time video processing applications[END_REF], [START_REF] Adarve | A filter formulation for computing real time optical flow[END_REF], [START_REF] Kroeger | Fast optical flow using dense inverse search[END_REF], [START_REF] Senst | Clustering motion for real-time optical flow based tracking[END_REF], but few on CPU [START_REF] Kroeger | Fast optical flow using dense inverse search[END_REF], [START_REF] Garcia-Dopico | Parallelization of the optical flow computation in sequences from moving cameras[END_REF]. It should also be noted that optical flow estimations based on machine learning are gaining in popularity in the scientific community [START_REF] Weinzaepfel | Deepflow: Large displacement optical flow with deep matching[END_REF], [START_REF] Ranjan | Optical flow estimation using a spatial pyramid network[END_REF].

This article focuses on two algorithms: the Horn-Schunck (HS) algorithm [START_REF] Horn | Determining optical flow[END_REF] and the TV-L1 algorithm [START_REF] Zach | A duality based approach for realtime tv-l 1 optical flow[END_REF]. While HS is not the most accurate, this well known algorithm has the advantage of simplicity. Its computation method is suitable for CPU and GPU implementations, and thus it can serve as a reference for more advanced algorithms which share its structure. Besides, the Horn-Schunck algorithm is used by the authors in a processing chain targeting the detection of meteors in real time from a nano-satellite [START_REF] Météorix | Nanosat Sorbonne Université[END_REF]. TV-L1, which more computationally intensive, is studied because it can handle more complex situations such as night vision denoising.

The Horn-Schunck algorithm works as follows: it starts by computing the space-time derivatives (I x , I y I t ) of the pair of input images, then iteratively computes an average over a local neighborhood (eq.1), along with the update in every point of the velocity vector field (u, v) (eq. 3). The term α 2 is a convergence parameter.

ūk = 1 12 1 2 1 2 0 2 1 2 1 * u k vk = 1 12 1 2 1 2 0 2 1 2 1 * v k (1)
u k+1 = ūk -Ix × Ix × ūk + Iy × vk + It α 2 + I 2 x + I 2 y ( 2
)
v k+1 = vk -Iy × Ix × ūk + Iy × vk + It α 2 + I 2 x + I 2 y
TV-L1, which is well described in [START_REF] Zach | A duality based approach for realtime tv-l 1 optical flow[END_REF] and [START_REF] Ipol | Image processing on line[END_REF], relies on the total variation method using the L 1 norm for the regularization terms (TV-L1). It allows to have better estimation of discontinuities in the flow and the method is also more robust to the noise. This makes it particularly suitable for applications such as video denoising where the optical flow method needs to be robust to noise [START_REF] Liu | A high-quality video denoising algorithm based on reliable motion estimation[END_REF] and where a lot of occlusions can occur. TV-L1 estimation has however a greater computational demand compared to HS, as it uses different intermediate vector fields v = (v 1 , v 2 ) and P = (P 1x , P 1y , P 2x , P 2y ) to compute the final field u = (u 1 , u 2 ). Therefore, there are 3 major steps to compute one iteration of TV-L1:

v k+1 = u k + T H(u k , u 0 ) (3)
u k+1 = v k+1 -θdiv(P) (4)
P k+1 = P k + τ /θ∇(v k+1 + θdiv(P k )) 1 + τ /θ ∇(v k+1 + θdiv(P k )) (5)
Where TH(u) is a conditional thresholding function. τ and θ are respectively the time step and tightness regularization terms.

Based on these equations, we can describe the algorithm with 3 major operators, each operator having a specific consumer/producer behavior, as presented in Figure 1. Using this representation highlights the difficulty to merge those operators since the computation of u i,j needs top and left points from P and the computation of P needs bottom and right points from u. This is due to the computation of divP and ∇u. These considerations are important to understand the pipeline optimization presented in section III.

In order to simplify the performance analysis, the monoresolution version of both algorithms are evaluated. Indeed, It gives good estimations in terms of computation time and power consumption as the hierarchical version empirically proves to be slightly less than twice slower. Besides, this allows to decouple the analysis from other concerns and choices, such as the interpolation method (bi-linear or bicubic), the number of scales, or the scaling method (Burt filter, Gaussian filter or a simple binomial filter). For the same reason, no warp [START_REF] Ipol | Image processing on line[END_REF] is considered for both algorithms. Large displacements require a multi-resolution version.

The qualitative aspects of these algorithms will not be studied in this article, as it puts the focus on the speed vs. power consumption trade-off for a real time detection with limited power. However, these algorithms have been chosen and are relevant to evaluate because the HS algorithm provides a reference baseline with good results for simple scenes, while the TV-L1 algorithm is able to handle more complex scenes with occlusions.

Lastly, a choice has to be made regarding the number of iterations: we decided to consider eight iterations per pixel for the HS algorithm, and three iterations for the TV-L1 algorithm, noticing experimentally that the corresponding Mean Squared Errors (MSE) were similar for these configurations. However, the results for both algorithms cannot be directly compared as their application domains are not identical.

III. SPEED AND POWER CONSUMPTION OPTIMISATIONS

Embedded system optimization for speed [START_REF] Datta | Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures[END_REF] and for power consumption [START_REF] Basmadjian | Evaluating and modeling power consumption of multi-core processors[END_REF], [START_REF] Hager | Exploring performance and power properties of modern multi-core chips via simple machine models[END_REF], [START_REF] Mittal | A survey of CPU-GPU heterogeneous computing techniques[END_REF] can be achieved by minimizing one of the two conditions under the hypothesis that the other follows a hard constraint. The proposed approach consists here in exploring the efficient frontier of the operating points in the (speed, consumption) space for codes with different levels of optimization, on different processors running for various frequencies. The points of compromise thus found can be used for example to define the maximum image size which can be processed by a configuration, and to help fuel the debate on power requirements.

The algorithmic transformations considered here for optimization are the ones described in [START_REF] Lacassagne | High level transforms for SIMD and low-level computer vision algorithms[END_REF], especially operators pipelining and operators fusion.

The HS and TV-L1 algorithms lend themselves well to this type of transformations: it is possible to merge the computation steps of the point-wise operators such as the V and U operators in Figure 1 for TV-L1. In order to reduce the number of memory accesses and to improve the arithmetic intensity, the result of V is no more stored in an array but directly used to compute the result of U . More importantly, the iterative part allows to pipeline iterations so that the accesses to a same memory cell are as close as possible in time. This way, data are more likely still in cache when they are accessed. Doing this requires to create intermediate arrays to store the results for each iteration. Figure 2 describes the pipeline optimization for 3 iterations of TV-L1 including a description of the prologue needed to deal with the images edges.

By analyzing the data dependencies within this transformation, we can first realize that all intermediate arrays can have modular memory allocation in order to reduce the number of lines per array to the number of iterations. With few adjustments of the prologue and the epilogue, we can even merge all the buffers so that we only have one buffer per vector field and the data modifications are made in-place.

These transformations maximize data persistence inside the processors (in the cache for CPUs, in the shared memory for GPUs) while minimizing transfers with the external memory. Similar transformations can be applied on the HS algorithm.

The computation format can also be optimized by switching from 32-bit floating point numbers (F 32 ) to 16-bit (F 16 ) on GPU, allowing more data parallelism and better bandwidth, while ensuring sufficient calculation accuracy [START_REF] Piskorski | Customizing CPU instructions for embedded vision systems[END_REF]. For CPU, this format will be available for computation with ARMV8.2.

IV. EXPERIMENTAL EVALUATION

The five target boards used for experiments are presented in table I, three of which own a GPU. For each architecture, different implementations have been studied. For the CPU architectures, all the versions are multi-threaded with as many threads as physical cores. The considered variants for HS algorithm are: the base algorithm (base), two SIMD nonpipelined versions with respectively one or two buffers (simd1 and simd2), and two pipelined SIMD versions with one or two buffers (pipe 1 et pipe 2). For GPU architectures, two versions are considered: the baseline version (base) and an optimized version (opt) comprising the following optimizations: shared memory, grouping of U and V , operator merging and iterations kept within block. Additionally, these two versions are declined in two flavors: F 32 et en F 16 . As for the TV-L1 algorithm, nine versions have been tested, but for the sake of clarity we present here only four of them. The first one is the standard version with scalar implementation (base), the second one is its SIMD equivalent named simd, the last two ones are respectively called pipe and pipe mono and both have SIMD, fusion and pipeline optimizations described in section III, the difference being that pipe mono also has merged buffers (i.e. no intermediate buffer).

In order to perform simple and reproducible power measurements, the electrical consumption of the entire embedded system is measured. A board was developed to this effect which is inserted between the power source and the target board. Voltage, through a voltage divider, and current, through a resistor in series and a voltage amplifier, are measured by a micro-controller. The latter also reads the state of 4 GPIOs from the target board, in order to match measurement points and events from the computation program. The microcontroller sends 5000 samples per second to a PC host. The power measurement board was also carefully calibrated. We simultaneously measured performance and power consumption for different frequencies and image sizes. The fre-quencies are taken among the available frequencies on each board. Whenever possible, the external memory frequency is set to its maximum. The cooling system of each target board is also set to the maximum and the energy savings processes in the OS are deactivated. When the target board does not contain an integrated cooling down system, we performed the measures in an isothermal oven. We used all the ARM cores on each board, but we deactivated the Denver 2 cores on the Jetson TX2 board. The images used for the experiments are square images, with a varying size from 64 and 1024 pixels with a 8 pixel increment for the CPU versions; and with a varying size from 208 to 2048 pixels with a 16 pixel increment for the GPU versions. The size of 1008×1008, often used, corresponds to the largest size which is not a cache line size multiple and which is compatible with the GPU splitting.

In the case of GPUs, performances are evaluated for each kernel [START_REF] Nvidia | programming model[END_REF] for blocks of 64 to 512 threads, using multiples of 32 (the size of a warp [START_REF] Nvidia | programming model[END_REF]). The blocks are from 8 to 256 threads large and 1 to 32 threads high. The best block size is then used within the comparisons.

V. RESULTS ANALYSIS

A. Operating points and efficient frontier

Figures 3 and4 show the efficient frontiers of the operating points in the (time per pixel, energy consumption per pixel) space for each of the studied configurations. Figure 3 presents the results for the HS algorithm on CPUs and GPUs, while Figure 4 shows those for TV-L1 on CPUs.

The best configuration is obtained with the TX2 board for both HS (for both the GPU and the CPU versions) and TV-L1 algorithms. Figure 3 highlights the significant performance improvements brought by the GPU versions, and the impact of the presented optimizations: focusing on the TX2, compared to the base CPU version, the optimizations roughly bring a factor of 2.3 in speed, and the optimized GPU version (F 32 ) a factor 6 in speed and 9 in power consumption. We also note that for each configuration, the efficient frontier is generally small, showing that only a few of the possible frequencies are interesting w.r.t. the optimization criteria -typically between one and 3 points among 10 tested frequencies. This last aspect is even more significant for TV-L1 as we can see on Figure 4 that the efficient frontier is often reduced to a single point (usually the highest frequency). This figure also shows that the regular implementation of TV-L1 is far less efficient than the HS one. However, the optimizations made on TV-L1 bring a better improvement with a speedup of ×4.5 and a consumption drop of more than ×6. Still, even if the execution time of TV-L1 tends to reach the one of HS for the best boards (TX2 and TX1), its consumption remains significantly higher.

B. Impact of CPU optimizations

Figure 5(a andc) shows the impact of the considered CPU optimizations on the processing speed for the Jetson TK1 board -its behavior is well representative of all the other tested configurations on CPUs. By varying the image size, we can observe a cache exit generally sooner for the non-optimized versions, particularly emphasized in the simd1 version of HS (Figure 5(a)). Furthermore, among the best optimizations, the impact of the cache exit remains less important than on the other versions. As for example, on Figure 5(c), we can observe than for the tvl1 simd version, the cache exit is penalized by being 2.5 times slower than before when on the tvl1 pipe version, the penalty is only of a factor 2, while starting from a 2 times faster speed. We figured out that some optimizations slightly increase the power consumption -like the use of the SIMD units for HS -but Figure 5(b andd) shows that the time saved for processing one image with these optimizations still requires a lot less energy to process the same image. Besides, the most efficient optimizations (pipe versions for each algorithm) not only speed up the computation, but also reduce the power consumption, even with SIMD use. This little drop in power consumption could be explained by the fact that the pipe versions reduce the number of memory accesses and improve data persistence in cache, what saves both energy and time.

C. Impact of GPU Optimizations

Figure 6 shows the evolution of the number of cycles per pixel (CPP) and of energy as functions of frequency for the TX1 target board, on baseline and optimized codes, both in F 16 and F 32 . Figure 6(a) shows that in F 32 , the opt version is about 3 times faster than the base version. The F 16 base version is 1.5 to 2 times faster than the F 32 base version. The opt F 16 version is faster than the F 32 only on high frequencies as the memory is no more fast enough compared to the processor. Because of the lack of ILP (Instruction Level Parallelism), F 16 becomes interesting when the memory becomes a bottleneck. This is the same phenomenon that can explain the raise of CPP at high frequencies: the required bandwidth to maintain the same speed reaches the maximum target board bandwidth; the programs then becomes memory bound.

Figure 6(b) shows that the trend in power consumption is the same as for the speed: the variation in power consumption is negligible before the power consumption duration. On figure 6(c), the idle power consumption is subtracted from the total power consumption. We can notice that the frequencies minimizing the power consumption correspond to the Compute Bound -Memory Bound transition on which the CPP starts to grow. However, it is better to go towards higher frequencies if we consider the idle power consumption

D. Comparison of studied target boards and cores

Figure 7(a) shows that GPUs are faster than CPUs at a same frequency, achieving a ×4 factor on the TX1 board at 1GHz: the HS algorithm thus manages to take advantage of the greater number of cores. The GPU remains even faster at 1GHz than the CPU at 2GHz. The CPU of the TX1 exhibits a speedup compared to the one of the TK1, but remains close. The GPU from the TX1 improves significantly the performances compared to the GPU from the TK1 (factor 2). Its maximum frequency is also higher.

Figure 7(b) shows that the instantaneous power consumption of the different CPUs varies almost linearly w.r.t. the frequency, but with different slopes. The power consumption of GPUs is higher than on CPUs at an identical frequency, but GPUs can reach lower frequencies in order to reduce their power consumption. We also measured the spent energy per pixel for each board in order to make a fairer comparison (figure 7(c)). This allows us to choose an optimal operating frequency for each of the processor and to see that the evaluated GPUs are energetically more effective than the CPU.

E. Comparison between Horn-Schunck and TV-L1

Figure 8 presents a comparison between the two algorithms on a 1008×1008 image on the 3 targeted boards TK1, TX1 and TX2 depending on the frequencies. The comparison is made in terms of execution time (a), and energy consumption (b). It is made for the best version for each algorithm: pipe1 for HS and simd piep mono for TV-L1. Even if the optimized version of TV-L1 takes advantage of a greater acceleration compared to its scalar version, it is still 1.4 time slower than HS (a). The power consumption of the two algorithms is quite the same, however we can notice that TV-L1 power consumption is always a little higher. This could be explained by the greater number of memory buffers used by TV-L1 which leads to more memory accesses.

F. Synthesis

The best configurations on CPU and GPU are obtained on the TX2 target board.

VI. CONCLUSION

This article presents a comparison between different implementations of two iterative optical flow algorithms: Horn-Schunck and TV-L1. It demonstrates the superiority of GPUs over CPUs both in terms of computational speed and energy consumption for this kind of algorithm.

For HS, real-time at 25 f.p.s. is achieved for up to 8Mpix frames for 0.35J per picture on GPU, against 1.8Mpix frames for 0.24J per frame on CPU. The results also show that the TV-L1 algorithm, which is far slower than HS without optimization, can almost match the performance of HS after optimization on CPU, and can achieve real-time performances with 0.25J for a 1.4Mpix frame. These results can help an application developer to choose his algorithm and size his system. We also showed that even if HS remains faster than TV-L1, the latter can benefit more of major optimizations, what significantly reduces the differences between the two. Future work will include the implementation of TV-L1 on GPU in order to have a more exhaustive comparison, and to evaluate the feasibility of implementing on CPU these algorithms in 16-bit fixed point in order to double the SIMD parallelism. A fine image qualitative analysis depending on the applications will also be made.

Fig. 1 .

 1 Fig. 1. Consumer/Producer representation of the main operators involved in TV-L1 optical flow estimation.

Fig. 2 .

 2 Fig. 2. Processing order for pipelined TV-L1 for 3 iterations (X #Y represents the data X at iteration Y ). Dashed steps represent the prologue.

Fig. 3 .Fig. 4 .

 34 Fig. 3. Efficient frontier of the operating points in the (time/pixel, energy/pixel) space, for the considered target boards and HS implementations. Each point represents a given frequency. The opt. version for the general processors is the fastest version (pipe 1). The Raspberry Pi 3 base configuration is located outside of the represented space (Energy = 1145 nJ, Time = 415 ns). Bottom and left are better.

Fig. 5 .

 5 Fig. 5. Impact of the optimizations on CPU for both HS and TV-L1 on target board TK1. CPP = Cycles Per Pixel, Size = side length of a square image. Lower is better.

Fig. 6 .

 6 Fig.6. CPP and power consumption as functions of frequency for the GPU TX1 versions. All image sizes are 1008×1008. Lower is better.

Fig. 7 .

 7 Fig. 7. Time per pixel (left), Power (middle) and Energy per pixel (right) as functions of Frequency on different target boards (optimized versions). The legend of the middle figure, omitted by lack of space, is identical. Lower is better.

Fig. 8 .

 8 Fig. 8. Performances and consumption comparison between 8 optimized iterations of HS and 3 optimized iterations of TV-L1. Lower is better.

TABLE I TECHNICAL

 I SPECIFICATIONS OF THE TARGET BOARDS.

	Board	Process CPU	Fmax (GHz)	GPU	Fmax (MHz)
	PCduino8 28 nm 8×A7 1.80	-	-
	RPi3	40 nm 4×A53 1.20	-	-
	TK1	28 nm 4×A15 2.32	192 C Kepler	852
	TX1	20 nm 4×A57 1.73 256 C Maxwell 998
	TX2	16 nm 4×A57 2.00	256 C Pascal	1300

  Table II sums up these configurations and gives the largest image size possibly processed at 25 fps.

TABLE II BEST

 II CONFIGURATIONS OBTAINED, TO GET REAL-TIME AT 25 FPS.

	Configuration	E (nJ/pix)	t (ns/pix)	max size (#pix)
	HS TX2 GPU opt F 16 energy min.	41	6.4	2501
	HS TX2 GPU opt F 16 time min.	43	5.0	2839
	HS TX2 CPU opt energy min.	129	36	1059
	HS TX2 CPU opt time min.	133	22	1355
	TVL1 TX2 CPU opt global.	180	29	1174

ACKNOWLEDGMENTS

This work has been partially funded by the french DGA. The authors wish to thank Tomoko Arai from the PERC project in Chiba University for video sequences and JM Morel from CMLA at ENS Paris Saclay.