Jiangang Chen

Josquin Foiret

Jean-Gabriel Minonzio
email: jean-gabriel.minonzio@upmc.fr

Maryline Talmant

Zhongqing Su

Li Cheng

Pascal Laugier

Measurement of guided mode wavenumbers in soft tissue-bone mimicking phantoms using ultrasonic axial transmission

Keywords: numbers 4320Ye Measurement methods and instrumentation, 4380Vj Acoustical medical instrumentation and measurement techniques 4320Mv Waveguides, wave propagation in tubes and ducts 4360Fg Acoustic array systems and processing, beam-forming Axial transmission, Bone, Guided wave, Quantitative ultrasound, Soft tissue, Singular value decomposition (SVD)

Human soft tissue is an important factor that influences the assessment of human long bones using quantitative ultrasound (QUS) techniques. To investigate such influence, a series of soft tissue-bone phantoms (a bone mimicking plate coated with a layer of water, glycerol or silicon rubber) were ultrasonically investigated using a probe with multi-emitter and multi-receiver arrays in an axial transmission configuration. A singular value decomposition (SVD) signal processing technique was applied to extract the frequency-dependent wavenumbers of several guided modes. The results indicate that the presence of a soft tissue-mimicking layer introduces additional guided modes predicted by a fluid waveguide model. The modes propagating in the bone mimicking plate covered by the soft-tissue phantom are only slightly modified compared to their counterparts in the free bone-mimicking plate, and they are still predicted by an elastic transverse isotropic two-dimensional waveguide. Altogether these observations suggest that the soft tissue-bone phantoms can be modeled as two independent waveguides.

Even in presence of the overlying soft tissue-mimicking layer, the modes propagating in the bonemimicking plate can still be extracted and identified. These results suggest that our approach can be applied for the purpose of the characterization of the material and structural properties of cortical bone.

Introduction

The axial transmission technique, employing a set of emitter(s) and receiver(s) to measure long bones, like tibia and radius, is considered to be one of the most promising modalities among quantitative ultrasound (QUS) techniques. The shaft of long bones has been evidenced to act as a natural waveguide [START_REF] Moilanen | Ultrasonic guided waves in bone[END_REF], Talmant et al 2010). A number of studies have demonstrated experimentally the potential of this technique to assess bone mechanical and structural properties (Moilanen et al 2003a[START_REF] Bossy | Bidirectional axial transmission can improve accuracy and precision of ultrasonic velocity measurement in cortical bone: A validation on test materials[END_REF][START_REF] Muller | Comparison of three ultrasonic axial transmission methods for bone assessment Ultrasound[END_REF]. Several in vivo studies evidenced that the velocity of the first-arrival signal (FAS) allows discriminating osteoporotic from healthy patients (Moilanen et al 2003a[START_REF] Knapp | Multisite quantitative ultrasound: Precision, age-and menopause-related changes, fracture discrimination, and T-score equivalence with dualenergy X-ray absorptiometry[END_REF][START_REF] Barkmann | A new method for quantitative ultrasound measurements at multiple skeletal sites -First Results of precision and fracture discrimination[END_REF], Hans et al 1999, Talmant et al 2009[START_REF] Kilappa | Low-frequency axial ultrasound velocity correlates with bone mineral density and cortical thickness in the radius and tibia in pre-and postmenopausal women[END_REF]. In other studies, the signal processing techniques were adapted to identify several guided modes propagating in long bones, focusing on their velocity dispersion or attenuation properties. For example, the energetic late arrival (ELA), associated with the low frequency fundamental flexural A 0 -like plate guided mode, has been measured in vitro in human radius specimens [START_REF] Moilanen | Measuring guided waves in long bones: Modeling and experiments in free and immersed plates Ultrasound[END_REF][START_REF] Sasso | Analysis of the Most Energetic Late Arrival in Axially Transmitted Signals in Cortical Bone[END_REF]. Higher order guided modes have also been observed in vitro in bovine and human bone specimens [START_REF] Lefebvre | Development of a new ultrasonic technique for bone and biomaterials in vitro characterization[END_REF][START_REF] Lee | Feasibility of bone assessment with leaky Lamb waves, in bone phantoms and a bovine tibia[END_REF][START_REF] Le | Probing long bones with ultrasonic body waves[END_REF][START_REF] Protopappas | Guided ultrasound wave propagation in intact and healing long bones Ultrasound[END_REF][START_REF] Song | Analysis of superimposed ultrasonic guided waves in long bones by the joint approximative diagonialization of eigen-matrices algorithm[END_REF], Tatarinov et al 2011).

In the above mentioned reports, the experiments were conducted in vitro on excised bones specimens, i.e., after removal of the overlying soft tissue.

While the feasibility of the identification and measurement of multiple guided modes has been demonstrated on long bones without overlying soft tissue on top of it, the influence of soft tissue on guided modes dispersion curves has not been extensively studied yet, apart from the studies reported by [START_REF] Moilanen | Modeling the impact of soft tissue on axial transmission measurements of ultrasonic guided waves in human radius[END_REF] dedicated to the fundamental flexural mode, i.e., A 0 in a plate or F 11 in a tube.

Previous in vivo studies by these authors suggested that the measurement of a signal contribution consistent with the fundamental flexural mode was possible in vivo on the tibia in some patients (Moilanen et al 2003a[START_REF] Moilanen | Ultrasonically determined thickness of long cortical bones: Two-dimensional simulations of in vitro experiments[END_REF], while in other patients the identification of such a mode was difficult, presumably due to the presence of soft tissues on top of the bone. In a subsequent modeling study [START_REF] Moilanen | Modeling the impact of soft tissue on axial transmission measurements of ultrasonic guided waves in human radius[END_REF], the authors reported an increasing mode density when the soft tissue thickness increases. From that latter study, the authors concluded that the identification of the flexural mode might be difficult when the soft tissue on top of the bone is thicker than a few millimetres. The presence of the soft tissue layer on top of long bones may be the cause for the ambiguity encountered in mode identification in vivo for some patients.

Our group has recently introduced a method for extracting guided mode phase velocities based on the singular value decomposition (SVD). This method consist in the analysis of the time responses for all possible pairs of emitter-receiver of a multi-emitter and multi-receiver array placed on top of the bone being investigated [START_REF] Minonzio | Guided wave phase velocity measurement using multiemitter and multi-receiver arrays in the axial transmission configuration[END_REF](Minonzio et al , 2011a)). High order guided modes are obtained from the maxima of the so-called Norm function in the temporal frequency -spatial frequency plane. The method has been shown to be efficient on bone mimicking plates made of a dissipative transverse isotropic elastic medium with characteristics mimicking those of actual bones (Minonzio et al 2011b).

The goal of the present work was to investigate the effect of overlying soft tissues on the propagation characteristics of guided modes in an axial transmission configuration by extending the method to the determination of the guided modes dispersion curves for a fluid-solid bilayer waveguide consisting of soft tissue-mimicking fluid or viscoelastic layers on top of a bone-mimicking plate. Our assumption is that the bilayer waveguide behaves as two uncoupled waveguides. Since the bonemimicking plate is a transverse isotropic solid and soft tissue-mimicking fluid is isotropic media, the problem in this study is described using a two dimensional model. The theory of guided mode propagation in plates is briefly summarized in section 2, where the dispersion equations for a fluid plate and for a transverse isotropic elastic plate are reminded. The experimental procedure using a multielement transmitting and receiving transducer array is described in section 3, where we outline some specifics of the SVD-based signal processing technique. In section 4, the results of axial transmission experiments on a variety of bilayer waveguides designed to mimic the overlying soft tissues on top of cortical bone are displayed. The measured guided mode wavenumbers are compared to the theoretical predictions for the two uncoupled fluid and solid waveguides model. We conclude by a discussion of our results and a summary of our findings.

Theoretical model

In an axial transmission measurement configuration, a linear array of emitters and receivers is aligned along the bone axis (Ox 1 in figure 1). In spite of the tubular shape of long bones, the propagation of guided waves in the cortical layer can be described in a first approximation by a plate model [START_REF] Moilanen | Ultrasonic guided waves in bone[END_REF], Talmant et al 2010). The plate model will thus be considered hereinafter. Consider a solid plate of thickness h covered by a fluid layer of thickness L (figure 1). The wavenumbers of the bilayer guided modes can be theoretically computed [START_REF] Dayal | Leaky Lamb Waves in an Anisotropic Plate .1. an Exact Solution and Experiments[END_REF], Yapura and Kinra 1995[START_REF] Simonetti | Lamb wave propagation in elastic plates coated with viscoelastic materials[END_REF]). They depend on the thicknesses, the mass densities, the plate elastic coefficients C ij and the fluid longitudinal velocity c f . However, for simplicity, following [START_REF] Simonetti | Lamb wave propagation in elastic plates coated with viscoelastic materials[END_REF] and Wu and Yang (2011), the bilayer waveguide is assumed in a first approximation to behave mainly like two decoupled waveguides. It implies that a measured guided mode wavenumber k exp corresponds to either a plate k p or a fluid wavenumber k f . This approximation will be discussed in section 4. First, consider a fluid layer of thickness L with clamped-free boundary conditions (figure 1). The boundary conditions implies that the wavenumber k 3 along the (Ox 3 ) direction is equal to (n+1/2)π/L, with n an integer. Moreover, the wavenumber k 1 along the (Ox 1 ) direction, satisfies ( )

2 2 2 1 3 f k k c ω + = , with
ω the angular frequency and c f the longitudinal velocity in fluid. Thus the fluid-guided wavenumber, denoted k f and equal to k 1 satisfies [START_REF] Royer | Elastic Waves in Solids I[END_REF]Dieulesaint 2000, Simonetti 2004) [ ]

1/2 2 2 1 2 f f n k = c L π ω + - , (1)
The different fluid guided modes are labeled according to their index n equal to 0, 1, 2, 3 … Secondly, consider a free transverse isotropic elastic plate of thickness h (figure 1). The direction (Ox 1 ) is the symmetry axis. Guided waves in the elastic plate are different from their counterparts in the fluid layer as the elastic plate supports the propagation of shear waves. Because the probe works in piston mode and does not excite horizontal shear displacement, only motion in the plane (Ox 1 x 3 ) is considered.

Thus a two dimensional approximation can be considered [START_REF] Dayal | Leaky Lamb Waves in an Anisotropic Plate .1. an Exact Solution and Experiments[END_REF]Kinra 1989, Rhee et al 2007).

Following previous conditions, the relation between k 3 and k 1 satisfies (Dayal and Kinra 1989, Rhee et al

2007) 2 2 2 2 3 1 4 2 M M N k k ± -± - = , (2)
where the sign ± indicates the positive and negative (Ox 3 ) directions (figure 1).

C C C C C C C k M C C ρω - - - + = , ( 3a
) 2 2 11 55 2 2 1 1 33 55 C C k k N C C ρω ρω - - = , ( 3b
)
where ρ is the plate density, ω is the angular frequency, and C 11 , C 33 , C 13 and C 55 are the stiffness coefficients of the transverse isotropic plate. Following equation ( 32) of [START_REF] Rhee | The group velocity variation of Lamb wave in fiber reinforced composite plate[END_REF], the dispersion equation of the symmetric Lamb modes S n can be written as

33 3 13 1 1 3 3 3 33 3 13 1 1 3 3 3 ( )( )sin( ) cos( ) ( )( ) sin( ) cos( ) 0 C R k C k R k k k h k h C R k C k R k k k h k h -- + + + - + + - - - + + + - + + = , (4a)
Following equation ( 34) of [START_REF] Rhee | The group velocity variation of Lamb wave in fiber reinforced composite plate[END_REF], the dispersion equation for the anti-symmetric modes A n is obtained by inverting the + and -subscripts inside the parentheses in (4a)

33 3 13 1 1 3 3 3 33 3 13 1 1 3 3 3 ( )( )sin( ) cos( ) ( )( ) sin( ) cos( ) 0 C R k C k R k k k h k h C R k C k R k k k h k h + + - - + - -- + + - + + + - + + = , (4b)
The terms R + and R -in (4) are given by 2 2 2 11 1 55 3

55 13 1 3 ( ) ( ) C k C k R C C k k ρω ± ± ± - - = + , (5)
In the following, the theoretical wavenumbers k f , associated with guided modes in the soft tissuemimicking layer, will be computed using (1). Likewise, the theoretical wavenumbers k p , associated with the bone-mimicking plate, will be computed using (2) to (5), with k p equal to k 1 . These theoretical wavenumbers will be compared with experimental data k exp in section 4 measured on a bone-mimicking plate covered by various soft tissue-mimicking layers. The experimental data are obtained using the experimental set up and the signal processing described in section 3.

Material and Method

Experimental set up

In order to acquire experimental data to compare with the model described in section 2, a custom made linear transducer array (Vermon, Tours, France) was used (figures 1 and 2). The characteristics of this probe have already been given in previous papers by our group [START_REF] Minonzio | Guided wave phase velocity measurement using multiemitter and multi-receiver arrays in the axial transmission configuration[END_REF](Minonzio et al , 2011a)). The Numerical values (GPa) 22.5 13.5 4.1 5.9

Three different soft tissue-mimicking layers have been used: water, 98% glycerol (VWR International, Fontenay-sous-Bois, France) and silicon rubber. The silicon rubber was artificially produced by mixing three substances, i.e., 4600A, 4600B and AK35 (all from Wacker Chemicals Ltd., Béziers, France) with the proportions of 1 : 0.1 : 1.6. The longitudinal velocity c f , density ρ and attenuation α of the three soft tissue phantoms are given in table 2. The longitudinal velocity in the silicon has been measured using two contact transducers. These materials, easy to handle, were chosen, owing to the similarity of their acoustical properties with those of soft tissues (table 2). Water is similar to soft tissues in term of velocity. Silicon rubber, 98% glycerol and soft tissue have similar attenuation values.

The longitudinal velocities in the silicon rubber and 98% glycerol are respectively lower and higher than that in soft tissues. These material have been used in order to test the model with a wide range of longitudinal velocity and attenuation values. Moreover, considering that soft tissues are not pure fluid but sustains shear stresses (Moilanen et al 2003b), silicon rubber is used as a tissue equivalent material [START_REF] Njeh | The use of quantitative ultrasound to monitor fracture healing: A feasibility study using phantoms[END_REF].

The bone phantom was placed into a tank, and supported by four tiny cylinders (height: 2 mm).

First, reference measurements were taken with the bone-mimicking phantom measured without any soft tissue-mimicking layer on top of it, with the probe being simply coupled to the bone plate with an acoustically transparent gel (Aquasonic ® , Parker Laboratories, Inc., NJ, USA). Subsequently, the fluid (water or 98% glycerol) was introduced into the tank. The position and the parallelism of the probe with respect to the plate could be adjusted by tuning the translational and rotational stages (figure 2). The position of the probe was adjusted above the plate to leave a 4 mm-thick and then 8 mm-thick layer of water, and subsequently a 4 mm-thick layer of 98% glycerol. Following the measurements with water and 98% glycerol, a 4 mm-thick layer of silicon rubber was placed on top of the bone-mimicking plate, such that any air bubbles in between could be carefully eliminated. The gel was used to improve the coupling between the probe and the silicon rubber.

Signal processing

The SVD-based signal processing technique has been described in detail elsewherere [START_REF] Minonzio | Guided wave phase velocity measurement using multiemitter and multi-receiver arrays in the axial transmission configuration[END_REF](Minonzio et al , 2011a)). Only the main steps are recalled here. The recorded radiofrequency signals r ij (t), with i and j the emission and reception indices ranging from 1 to N E and 1 to N R , respectively, are first Fouriertransformed. The singular value decomposition is subsequently applied to the N E ×N R sub-matrices, i.e., the matrices R ij (f) at each frequency. The signal subspace is determined by applying a first threshold t 1 on the singular values. The singular values below that threshold are considered associated with the noise subspace. The M singular values above this threshold are associated with the signal subspace and their associated reception singular vectors, denoted U n , are a basis of the experimental guided modes. The threshold t 1 is heuristically selected. Then, the so-called Norm function is defined by the following equation

( ) ( ) 2 1 , M pw n n Norm f k k = = U e , ( 6
)
where e pw (k) corresponds to a spatial plane wave at the spatial frequency k defined on the reception array.

Its norm is defined equal to 1.

The scalar product <U n | e pw (k)> corresponds to the normalized spatial Fourier transform of the singular vector U n . Thus, the SVD can be interpreted as a denoising step between the temporal and spatial Fourier transforms. If k corresponds to a guided mode wavenumber k exp , the Norm function is close to 1, and the guided mode is projected in the singular vectors basis. If k does not correspond to a guided mode wavenumber, the Norm function is small compared to 1. Subsequently, a second heuristically chosen threshold t 2 is applied to the Norm function, the values of which larger than this threshold correspond to the guided modes wavenumbers k exp .

As an example, the case of the 2.3 mm-thick bone-mimicking plate alone (i.e., without fluid layer on top) in discussed. First, the corresponding Norm function given by ( 6) is shown in figure 3

Experimental results and discussion

With the afore-addressed model and method, the coupling effect of soft tissue was interrogated by introducing four different soft tissue mimicking layers over the bone mimicking plate: water (layer thickness: 4 and 8 mm), 98 % glycerol (layer thickness: 4 mm) and silicon rubber (layer thickness: 4 mm). The results obtained with the four soft tissue-mimicking layers on top of the bone-mimicking bone plate are shown in figure 4. The experimental wavenumbers k exp are plotted against frequency together with the theoretical guided modes wavenumbers of the free bone-mimicking plate k p (already presented in figure 3) and of the fluid layers. The wavenumbers of the theoretical guided modes in the free fluid layers, denoted k f , are computed using (1) and the fluid velocity values given in table 2. The fluid modes are labeled with integers, denoted n, starting from 0.

A visual examination of the results shown in panels a and b of figure 4 reveals that the fluid-solid bilayer supports the propagation of additional modes compared to the bone mimicking plate alone (figure 3). There is an overall agreement between these new modes and the theoretical predictions from the fluid layer model, even in the case of viscoelastic soft tissue-mimicking media (98% glycerol or silicon). Note also that the modes of the silicon layer, which behaves more like a soft viscoelastic solid than a fluid, are still predicted by the fluid model. In addition, such an agreement is evaluated using

2 exp 2 exp th n n f n f k k error k - = , (7)
where k n th corresponds to n th theoretical fluid (k f ) or plate (k p ) mode. For the plate modes, the errors are equivalent with (figure 4) or without (figure 3) the fluid layer. The error ranges from 4 % (A 1 and S 1 ) to 9 % (S 0 , S 2 and S 4 ). For the fluid modes, the error varies from 2 % (n = 4 in the 4 mm water case) to 20 % (n = 7 in the 4 mm glycerol case). The average of the fluid mode error for the four cases is equal to 5 %. These additional modes are marked with circles on figure 4, whereas the experimental wavenumbers associated with the bone-mimicking plate modes are marked with crosses. The criteria for resolving the question of whether an observed trace is the result of a mode propagating in the soft tissuemimicking layer or in the bone-mimicking plate are the distances between the experimental trace and the simulated traces: an experimental trace is associated to the closest theoretical dispersion curve. In the present study, mode selection remains purely manual. For most of the cases, both the fluid and bonemimicking plate modes do not overlap so that their differentiation is trivial. When two modes overlap, the proper mode identification remains ambiguous, as is the case for mode S 0 which overlaps with the water layer mode n = 2 (panel a) and n = 4 (panel b) or for the mode A 3 which overlaps with the 98% glycerol layer mode n = 4 (panel c).

While most of the modes associated with the bone-mimicking plate with the soft tissuemimicking layer on top of it (figure 4, panels a-d) remain almost unchanged compared to their counterparts observed on the free bone-mimicking plate (figure 3), and are still predicted by an elastic transverse isotropic waveguide, we note slight differences like discontinuities in the trace of mode A 1 (panels a and c) or scalloping in the traces of mode S 1 and S 2 (panel b), presumably due to interferences with the fluid guided modes. However, these differences remain small, and should not prevent mode identification for the purpose of bone characterization.

We proposed to model the fluid-solid bilayer assemblies that were specifically designed to mimic the soft tissue layer on top of long bones as two uncoupled fluid and solid waveguides, each of which supporting its own guided modes. Results from theoretical computations based on this hypothesis appear to be consistent with measurements of both fluid and solid guided modes depending upon specific details of the soft-tissue mimicking layer considered.

These observations suggest that the modes propagating in the bone-mimicking plate can be extracted and identified in presence of the soft tissue-mimicking layer. Toward this goal, one needs to properly model the frequency-dependent wavenumbers of the soft tissue layer, which should be feasible once the thickness of the soft tissue has been estimated, from simple pulse echo measurements for example. Note that the speed of sound in the soft tissue is also required for the model. But because the speed of sound in soft biological tissues is fairly constant, its value derived from literature should be sufficient. It is then conceivable that meaningful bone properties can be recovered by carrying out an inversion scheme comparing the identified cortical bone modes to a theoretical plate model.

There are limitations to the preliminary work described in this brief report. The main limitation, of course, is that the measurements are performed in phantoms instead of actual bones. The phantoms only approximate the structure and acoustical properties of soft tissue and cortical bone. Additional complexities will arise from the inhomogeneity of real soft tissues and bones. It is not clear yet whether in vivo application of this approach will be feasible, until a range of experimental data has been analyzed.

An advantage of the present study is that a custom made probe designed for clinical examinations [START_REF] Talmant | In vivo performance evaluation of bi-directional ultrasonic axial transmission for cortical bone assessment Ultrasound[END_REF] has been used to conduct experiments on the phantoms. On the other hand, a plate model has been used in the study, while long bones are tubular-shaped. However, previous studies of axial transmission in cortical bone have previously demonstrated a high level of consistency of the plate model with experimental data [START_REF] Moilanen | Measuring guided waves in long bones: Modeling and experiments in free and immersed plates Ultrasound[END_REF].

Conclusion

In summary, this study of the coupling effect of soft tissue-mimicking layers on top of a bonemimicking plate on guided wave propagation suggests that the presence of a soft tissue-mimicking layer introduces additional guided modes, but that both layers can be considered as independent waveguides.

These results indicate that the modes propagating in the bone-mimicking plate can still be identified using an elastic transverse isotropic two-dimensional waveguide model and that they could be used for the purpose of the characterization of the material and structural properties of the bone waveguide. Future studies will be extended to in vivo measurements.

Figure 1 .

 1 Figure 1. Geometry of the problem: an elastic plate, covered by a fluid layer, is inspected with a multi-

  probe contains N E =5 emitters and N R =32 receivers (pitch of the array: 0.8 mm; minimum distance between both transmitting and receiving arrays: 11 mm ; central frequency 1 MHz; -6 dB spectrum spanning the frequency range of 0.5 to 1.5 MHz and the -20dB spectrum spanning the frequency range of 0.4 to 1.8 MHz). Such a configuration allowed 160 transmission-reception channels. During the measurement, each emitter was excited with wideband pulses. The radiofrequency signals recorded on the receivers were digitized (8 bits, 20 MHz, 1024 time samples) and finally stored on a computer for off-line analysis. Twenty radiofrequency signals were time averaged to increase signal-to-noise ratio. The driving electronics has been manufactured by Althaïs Technology (Tours, France).

Figure

  Figure 2. Experimental set-up

Figure 3 .

 3 Figure 3. Norm function from the 2.3 mm-thick bone-mimicking plate alone (a) and corresponding experimental guided mode wavenumbers k exp (black crosses), obtained from the Norm function maxima, compared with theoretical plate modes k p , shown with black continuous (anti-symmetric A n ) and dashed (symmetric S n ) lines (b).

Figure 4 .

 4 Figure 4. (color online) Experimental guided mode wavenumbers k exp (blue circles and black crosses) obtained from the bone-mimicking plate in the presence of a soft tissue-mimicking layer: 4 mmthick water layer (a); 8 mm-thick water layer (b); 4 mm-thick 98% glycerol layer (c); 4 mm-thick silicon rubber (d). The experimental wavenumbers are compared with the theoretical plate Lamb modes k p , shown with black continuous (anti-symmetric A n ) and dashed (symmetric S n ) lines and theoretical fluid layer modes k f , shown with blue dashed-dot lines (labeled with index n). The experimental wavenumbers are shown with symbols: black crosses are associated with plate modes, blue circles are associated with fluid layer modes.

Table 1 .

 1 2. Experimental set-upA 2.3 mm-thick bone-mimicking plate (Sawbones ® , Pacific Research Laboratories, Washington, USA) was used as a bone phantom. The thickness is typical of the cortical thickness of the human radius. The plate is made of oriented glass fibers embedded in epoxy. Its elastic properties, summarized in table 1, are close to those of actual bones. The mass density ρ of the bone mimicking plate is equal to 1.64 g.cm -3 .The stiffness coefficients C 11 , C 33 , C 44 and C 55 were deduced from bulk compression and shear velocities measured using contact transducers(Minonzio et al 2011b[START_REF] Granke | Change in porosity is the major determinant of the variation of cortical bone elasticity at the millimeter scale in aged women[END_REF]. The stiffness coefficient C 13 was deduced from C 33 and C 44 using C 13 = C 33 -2C 44 . Elastic coefficients of the bone mimicking plate

	Elastic coefficients	C 11	C 33	C 55	C 13

Table 2 .

 2 Longitudinal velocity c f , density ρ and attenuation α of the different soft tissue-mimicking materials considered in the experiments.

	material	c f (ms -1 )	ρ (gcm -1 )	α (dB.cm -1 ) at 1 MHz	References
	Silicon rubber	1230	0.8	1	(Itsumi et al 2009,
					Zheng et al 2011)
	Water	1480	1.0	10 -2	(Tang et al 1988,
					Culjat et al 2010)
	Soft tissues	1430 -1600	0.95 -1.05	0.5 -1	(Culjat et al 2010,
					Mast 2000)
	Glycerol (98%)	1910	1.26	0.2	(Bossy et al 2004,
					Fergusson et al 1954,
					Tang et al 1988,
					Figueiredo et al 2009)

  Chen et al. Physics in Medicine and Biology 22 Tatarinov A, Sarvazyan A, Beller G and Felsenberg D 2011 Comparative Examination of Human Proximal Tibiae In Vitro by Ultrasonic Guided Waves and pQCT Ultrasound Med. Biol. 2011 37 1791-1801 Yapura C L and Kinra V K 1995 Guided-Waves in a Fluid Solid Bilayer Wave Motion 21 35-46 Wu C H and Yang C H 2011 Guided waves propagating in a bi-layer system consisting of a piezoelectric plate and a dielectric fluid layer IEEE Trans. Ultrason. Ferroelectr. Freq. Control 58 1612-8 Zheng Y P, Chen J and Ling H Y 2011 Development of an Ultrasound Platform for the Evaluation of Plantar Soft Tissue Properties: A Feasibility Study on Silicone Phantom Feet Instrum. Sci. Technol. 39 248-60

Acknowledgements:

This work has been supported by ANR project "COSTUM" 09-TECS- 005-03 (2009-2012).


Binod Bhattarai
email: binod.bhattarai@unicaen.fr

Gaurav Sharma

Frédéric Jurie

Frederic Jurie
email: frederic.jurie@unicaen.fr

published or not. The documents may come

Deep fusion of visual signatures for client-server facial analysis

Deep fusion of visual signatures for client-server facial analysis

INTRODUCTION

We propose a novel method in a heterogeneous serverclient framework for the challenging and important task of analyzing images of faces. Facial analysis is a key ingredient for assistive computer vision and human-machine interaction methods, and systems and incorporating high-performing methods in daily life devices is a challenging task. The objective of the present paper is to develop state-of-the-art technologies for recognizing facial expressions and facial attributes on mobile and low cost devices. Depending on their computing resources, the clients (i.e. the devices on which the face image is taken) are capable of computing different types of face signatures, from the simplest ones (e.g. LPB)

. to the most complex ones (e.g. very deep CNN features), and should be able to eventually combine them into a single rich signature. Moreover, it is convenient if the face analyzer, which might require significant computing resources, is implemented on a server receiving face signatures and computing facial expressions and attributes from these signatures. Keeping the computation of the signatures on the client is safer in terms of privacy, as the original images are not transmitted, and keeping the analysis part on the server is also beneficial for easy model upgrades in the future. To limit the transmission costs, the signatures have to be made as compact as possible. In summary, the technology needed for this scenario has to be able to merge the different available features -the number of features available at test time is not known in advance but is dependent on the computing resources available on the client -producing a unique rich and compact signature of the face, which can be transmitted and analyzed by a server. Ideally, we would like the universal signature to have the following properties: when all the features are available, we would like the performance of the signature to be better than the one of a system specifically optimized for any single type of feature. In addition, we would like to have reasonable performance when only one type of feature is available at test time.

For developing such a system, we propose a hybrid deep neural network and give a method to carefully fine-tune the network parameters while learning with all or a subset of features available. Thus, the proposed network can process a number of wide ranges of feature types such as hand-crafted LBP and FV, or even CNN features which are learned endto-end.

While CNNs have been quite successful in computer vision [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF], representing images with CNN features is relatively time consuming, much more than some simple hand-crafted features such as LBP. Thus, the use of CNN in real-time applications is still not feasible. In addition, the use of robust hand-crafted features such as FV in hybrid architectures can give performance comparable to Deep CNN features [START_REF] Perronnin | Fisher vectors meet neural networks: A hybrid classification architecture[END_REF]. The main advantage of learning hybrid architectures is to avoid having large numbers of convolutional and pooling layers. Again from [START_REF] Perronnin | Fisher vectors meet neural networks: A hybrid classification architecture[END_REF], we can also observe that hybrid architectures improve the performance of hand-crafted features e.g. FVs. Therefore, hybrid architectures are useful for the cases where only hand-crafted features, and not the original images, are available during training and testing time. This scenario is useful when it is not possible to share training images due to copyright or privacy issues.

Hybrid networks are particularly adapted to our client-server setting. The client may send image descriptors either in the form of some hand-crafted features or CNN features or all of them, depending on the available computing power. The server has to make correct predictions with any number and combination of features from the client. The naive solution would be to train classification model for the type of features as well as for any of their combinations and place them in the server. This will increase the number of model parameters exponentially with the number of different feature types. The proposed hybrid network aligns the different feature before fusing them in a unique signature.

The main contribution of the paper is a novel multi-features fusion hybrid deep network, which can accept a number of wide ranges of feature types and fuse them in an optimal way. The proposed network first processes the different features with feature specific layers which are then followed by layers shared by all feature types. The former layer(s) generate(s) compact and discriminative signatures while the later ones process the signatures to make predictions for the faces. We learn both feature specific parameters and shared parameters to minimize the loss function using back propagation in such a way that all the component features are aligned in a shared discriminative subspace. During test time, even if all the features are not available, e.g. due to computation limitations, the network can make good predictions with graceful degradation depending on the number of features missing.

The thorough experimental validation provided, demonstrates that the proposed architecture gives state-of-the art result on attributes prediction on the CelabA dataset when all the features are available. The method also performs competitively when the number of features available is less i.e. in a resource-constrained situation.

The rest of the paper is organized as follows: Sec. 2 presents the related works, Sec. 3 gives the details of our approach while Sec. 4 presents the experimental validation.

RELATED WORKS

In this section we review some of the works which are, on one side, related to hybrid architectures or, on the other side, related to multimodal fusion and face attribute classification. Apart from face attributes classification, other critical applications on faces are: large scale face retrieval [START_REF] Bhattarai | Some faces are more equal than others: Hierarchical organization for accurate and efficient large-scale identity-based face retrieval[END_REF][START_REF] Bhattarai | CP-mtML: Coupled projection multi-task metric learning for large scale face retrieval[END_REF], face verification [START_REF] Sharma | Local higher-order statistics (LHS) describing images with statistics of local non-binarized pixel patterns[END_REF][START_REF] Simonyan | Fisher vector faces in the wild[END_REF][START_REF] Parkhi | Deep face recognition[END_REF][START_REF] Taigman | Deepface: Closing the gap to human-level performance in face verification[END_REF], age estimation [START_REF] Bhattarai | A joint learning approach for cross domain age estimation[END_REF][START_REF] Guo | A study on cross-population age estimation[END_REF], etc. For more details on the application of faces and comprehensive comparison of recent works, we suggest the readers refer [START_REF] Learned-Miller | Labeled faces in the wild: A survey[END_REF].

Hybrid Architectures. One of the closest works to our work is from Perronnin et al. [START_REF] Perronnin | Fisher vectors meet neural networks: A hybrid classification architecture[END_REF]. The main idea behind their work is to use Fisher Vectors as input to Neural Networks (NN) having few fully connected (supervised) layers (up to 3) and to learn the parameters of these layers to minimize the loss function. The parameters are optimized using back propagation. Unlike their architecture, our network takes a number of wide range of hand-crafted features including FVs, but not only. In addition, our architecture is also equipped with both feature specific parameters and common parameters. We have designed our network in such a way that the input features are aligned to each other in their sub-spaces. The advantage of such alignments is that our system can give good performance even when a single type of feature is present at test time. Moreover, such ability makes our system feature independent i.e. it can properly handle any types of features it encounters.

There are some works, such as [START_REF] Uricchio | Fisher encoded convolutional bag-of-windows for efficient image retrieval and social image tagging[END_REF], which, instead of taking hand-crafted features as input, takes CNN features and compute FVs in the context of efficient image retrieval and image tagging. This approach improves the performance of CNNs and attains state-of-art performance, showing that not only FVs but also CNNs benefit from hybrid architecture.

Face Attribute Classification. Some of the earliest and seminal work on facial attribute classification is the works from Kumar et al. [START_REF] Kumar | Facetracer: A search engine for large collections of images with faces[END_REF][START_REF] Kumar | Attribute and simile classifiers for face verification[END_REF]. Both of their papers use handcrafted low-level features to represent faces, sampled with AdaBoost in order to discover the most discriminative ones for a given attribute, and train binary SVM classifiers on this subset of features to perform attribute classification. The current state-of-art method of Liu et al. [START_REF] Liu | Deep learning face attributes in the wild[END_REF] uses two deep networks, one for face localization and another for identity based face classification. The penultimate layer of the identity classification network is taken as the face representation, and a binary SVM classifier is trained to perform an attribute classification. Some other recent state-of-theart methods such as PANDA [START_REF] Zhang | Panda: Pose aligned networks for deep attribute modeling[END_REF], Gated ConvNet [START_REF] Kang | Face attribute classification using attribute-aware correlation map and gated convolutional neural networks[END_REF], etc. also use deep learning to learn the image representation and do attribute classifications on it. From these works, we can observe that either hand-crafted features or CNN features are used for attribute classification. From our knowledge, the proposed method is the first to learn a hybrid structure combining multiple hand-crafted and CNN features for facial attribute classification. Moreover, most of the mentioned works here are performing binary attribute classification while we are predicting multiple attributes of faces.

Multi-modal fusion. Recently Neverova et al. [START_REF] Neverova | Moddrop: adaptive multi-modal gesture recognition[END_REF] proposed a method called Mod-Drop to fuse information from multiple sources. Their main idea is to take a batch of examples from one source at a time and feed into the network to learn the parameters, instead of taking examples from all the sources. The main drawbacks of their approach is, when a new source is encountered and is to be fused, it requires to re-trainthe whole network. Some other recent works such as [START_REF] Kahou | Combining modality specific deep neural networks for emotion recognition in video[END_REF][START_REF] Srivastava | Multimodal learning with deep boltzmann machines[END_REF][START_REF] Wu | Exploring inter-feature and inter-class relationships with deep neural networks for video classification[END_REF][START_REF] Ngiam | Multimodal deep learning[END_REF] fuse multiple sources of information to improve the performance of the final result. None of these works evaluated the performance of component sources or their possible combinations after fusion.

APPROACH

As mentioned before, a key challenge addressed in this paper is to learn an optimal way to fuse several image features into a common signature, through the use of a hybrid fully connected deep network. This section presents the proposed method in detail, explains how to learn the parameters and gives technical details regarding the architecture.

Network architecture

Fig. 2 shows a schematic diagram of the proposed network. A, B and C denote the different feature types to be aligned and fused, which are the input to the network. We recall that all or only a subset of the features can be available depending on the computing resources of the client. While we show a network with 3 features types, more can be used with similar layers for the new features. The key idea here is to train a single network which consists of feature specific layers (shown in blue), to be implanted on the clients, and common layers (shown in black), to be implanted on the server. The activation of the middle layer, obtained after merging the feature specific layers, gives the universal signature which will be transmitted from the client to the server. Each layer is fully connected with its parents in the network. In our application the output of the network is the facial expressions/attributes to be recognized, one neuron per expression/attribute, with the final values indicating the score for the presence of these attributes.

Learning the parameters of the network

Carefully setting up the learning of such hybrid network is the main issue for competitive performance. We propose to learn the parameters of this network with a multistage approach. We start by learning an initialization of the common parameters. To do this we work with the most discriminate feature type (e.g. A, B or C). For example, suppose we observed that A is the most discriminate for our application (as discussed in the experiment section, we will see that for our application FVs are the most discriminant features). Thus we start learning the parameters of the network corresponding to both (i) the feature specific parameters of network A (blue layers) and (ii) the part of the network common to all features (black layers). Then we fix the common parameters and learn the feature specific parameters of the feature B taking training examples encoded with B. In our case, the task is same but the features are different during each training round. By repeating the same procedure, we learn the feature specific parameters of the network for each of the remaining type of features. In the end, all the features are aligned into a common signature which can then be transmitted to the server for the computation.

The major advantage of this strategy is that although we are mapping all the features into same feature space, we do not require feature to feature correspondence e.g. we are not using a certain feature type to estimate or mimic any other feature type. Moreover, when we encounter a new feature type, we can easily branch out the existing network and learn its parameter without hindering the performance of other feature types. Thus the proposed learning strategy, while performing very well, also avoids the retraining of the whole network upon addition of a new features type. This is a major advantage of this our approach over existing Mod-drop [START_REF] Neverova | Moddrop: adaptive multi-modal gesture recognition[END_REF] algorithm. Finally, since there are fewer parameters to optimize than training one distinct network per feature, the computations required are less and the training is faster.

Another alternative, that we explored, is to learn the parameters of the whole network first with all the available feature types, and then fix the common parameters and finetune the feature specific parameters. The reason behind this approach is to make shared subspace more discriminative than with the one learned with the single most discriminative feature so that we can align all the component features in this subspace and improve the overall performance. We found the performance obtained with this approach is slightly better than the one we discussed before. However, this alternative requires feature to feature correspondence mapping. Moreover, training with all the features at a time requires more computing resource and also leads to slow convergence and longer training time. We compare the performances of these methods in more details in the experiment section.

Details of the architecture

The proposed network is composed of only fully connected (FC) layers. Once the features are fed into the network, they undergo feature specific linear projections followed by processing with Rectified Linear Units (ReLU). Eq. 1 gives the feature-specific transformations, where σ is the nonlinear transformation function i.e. ReLU, WA, WB, WC and bA, bB, bC are projection matrices and biases for the input features of the networks A, B, and C respectively. These representations further go into linear projections followed by ReLU depending upon the depth of the network.

h A = σ(xAWA + bA) h B = σ(xBWB + bB) h C = σ(xC WC + bC ) (1)
When the network takes more than one type of features at a time, it first transforms them with the FC and ReLU layers and then sums them and feeds into the common part of the network. We call this step as merging, as shown in the diagram. We further call the vector obtained at this point, after merging, as the signature of the face. In the common part of the network, intermediate hidden layers are projected into linear space followed by ReLU. The final layer of the network is a sigmoid layer. Since we are doing multilabel predictions, sigmoid will assign higher probabilities to the ground truth classes. We learn the parameters to minimize the sum of binary cross-entropy of all the predictions of the sigmoid layer. We minimize the loss function using Stochastic Gradient Descent (SGD) with standard back propagation method for network training.

In the heterogeneous client-server setting, the client is expected to compute the signature and send it to the server for processing. Since different clients can have very different computing capabilities they can compute their signature with different types and number of features -in the worst case with just one feature. The method allows for such diversity among clients and as the server side works with the provided signature while being agnostic about what and how many features were used to make it.

EXPERIMENTS

We now present the experimental validation of the proposed method on the task of facial attribute classification. All the quantitative evaluation is done on the CelebA dataset [START_REF] Liu | Deep learning face attributes in the wild[END_REF], the largest publicly available dataset annotated with facial attributes. There are more than 200,000 face images annotated with 40 facial attributes. This dataset is split into train, val, and test sets. We use train and val set for training and parameter selection respectively, and we report the results obtained on the test set.

In the rest of the section, we first give the implementation details and then discuss the results we obtained.

Implementation details

We have performed all our experiments with the publicly available aligned and cropped version of the CelebA1 [START_REF] Liu | Deep learning face attributes in the wild[END_REF] dataset (without any further pre-processing). We assume that up to 3 different types of features can be computed, namely, Local Binary Patterns, Fisher Vectors and Convolutional Neural Networks features, as described below.

Local Binary Patterns (LBP). We use the publicly available vlfeat [START_REF] Vedaldi | VLFeat: An open and portable library of computer vision algorithms[END_REF] library to compute the LBP descriptors. The images are cropped to 218 × 178 pixels. We set cell size equal to 20, which yields a descriptor of dimension 4640.

Fisher Vectors (FV). We compute Fisher Vectors following Simoyan et al [START_REF] Simonyan | Fisher vector faces in the wild[END_REF]. We compute dense SIFTs at multiple scales, and compress them to a dimension of 64 using Principal Component Analysis. We use a Gaussian mixture model with 256 Gaussian components. Thus, the dimension of the FV feature is of 32,768 (2×256×64). The performance of this descriptor is 77.6 ± 1.2% on LFW for the task of face verification, with unsupervised setting, which is comparable to the one reported [START_REF] Simonyan | Fisher vector faces in the wild[END_REF].

Convolutional Neural Networks (CNN). We use the publicly available state-of-art CNN mode trained on millions of faces presented in [START_REF] Parkhi | Deep face recognition[END_REF], to compute the CNN features. The dimension of CNN feature is of 4096. Our implementation of this feature gives 94.5±1.1% on LFW for verification in unsupervised setting. Here, these features are computed without flipping and/or multiples of cropping of faces.

Baseline methods.

We report two different types of baselines. In the first one, the network is trained with a given feature type (e.g. LBP) while the same type of feature is used at test time (e.g. LBP again). We call this type of network as Dedicated Networks. In the second setting, we allow the set of features at train time and the one used at test time to differ. Such networks are adapted to different sets of features. This is the particular situation we are interested in. All Feature Training Network (AllFeatNet). In this setting, all the available features are used to train the network. At test time, one or more than one type of features can be used, depending on its availability. For us, the available features are as described before FVs, CNNs, and LBPs.

Mod-Drop. This is currently the best method for learning cross-modal architectures, inspired by [START_REF] Neverova | Moddrop: adaptive multi-modal gesture recognition[END_REF]. It consists, at train time, in randomly sampling a batch of examples including only one type of features at a time, instead of taking all the available features, and learn the parameters in a stochastic manner. We refer the reader to the original work [START_REF] Neverova | Moddrop: adaptive multi-modal gesture recognition[END_REF] for more details.

The proposed method.

On the basis of which we fix the parameters of the common shared subspace, we categorize the proposed methods into two: FVNetInit. Tab. 2 shows the individual performance of different features we used for our experiments. From the table we can see that FVs are most discriminative for our application. Thus, we choose to take few top layer's parameters ( please refer Tab. 1 of for the number of layers in shared subspace ) of FVNet as common shared parameters of proposed network. Once we fix this, we learn the feature specific parameters for CNNs and LBPs to minimize the loss function. Fig. 4 shows the evolution of performances of FVs, LBPs, and CNNs with the amount of training epochs.

AllFeatNetInit. In this case, we use the common part of AllFeatNet as a starting point. Then we fix these parameters and learn the feature specific parameters of FVs, LBPs and CNNs to minimize the loss the function.

Quantitative results

We now present the results of the experiments we do to evaluate the proposed method. We measure the performance using average precision (AP) i.e. the area under the precision vs. recall curve. We do not consider attribute label imbalances for all the cases, unless explicitly stated.

Our experiments are mainly focused on validating two as- based networks. This shows that the proposed network and the multi-stage training strategy is capable of making better predictions in the presence of more information i.e. multiple types of features at a time and are optimal to every combination of features.

Fig. 3 shows the performance comparison between the proposed methods with AllFeatNet at different levels of feature combinations. From the bar-chart, we can observe that, when all the features are available at test time, AllFeat-Net performs better than ours. It is expected too, because this approach is optimized only for this combination. But this is the most unlikely scenario for the applications we are addressing, due to constraints such as computing resources and time, etc. Out of other 6 cases, our method performs substantially better and gives similar performance in one case. This shows that our method leverages all the features available and when more information is present, gives better performance. Unlike AllFeatNet, the proposed method is optimal in every combination of features too.

Qualitative results

Fig. 5 shows the qualitative performances comparison between the baselines and the proposed method. We randomly choose three different test images and used them for evaluation. Here, we consider LBPs (the simplest feature type) only for evaluation. Thus for both the single feature network (LBPNet) and multi-feature network ( AllFeatNet and ours), only LBPs are available at test time. In the figure we can see the top 7 attributes predicted by the compared methods. For each of the attributes, the corresponding score shows the probability of an attribute being present in the given image. On the basis of the number of correct predicted attributes, the performances of LBPNet and the pro-posed method is comparable in two cases (first two cases). While in the third case, our method (4 correct predictions ) is even better than LBPNet (3 correct predictions). This further validates that the proposed method retains the property of component features. The performance of AllFeatNet is comparatively poorer than LBPNet and ours for all test images. Moreover, it is important to note that the scores corresponding to the predicted attributes by AllFeatNet are small. This suggests that with this approach the predictive power of LBPs is masked by other strong features e.g. FV and CNNs.

CONCLUSIONS

We propose a novel hybrid deep neural network and a multistage training strategy, for facial attribute classification. We demonstrated, with extensive experiments, that the proposed method retains the performance of each of the component features while aligning and merging all the features in the same subspace. In addition to it, when more than one feature type is present, it improves the performance and attains state-of-art performance. The proposed method is also easily adaptable to new features simply learning the feature specific parameters. This avoids retraining the existing network. Since the majority part of the network is shared among all the feature types, the proposed method reduces the number of parameters.

Figure 1 :

 1 Figure 1: Randomly sampled images of CelebA and a subset of attributes. Green color attributes are relevant for the image whereas red color attributes are irrelevant (better viewed in color).

Figure 4 :

 4 Figure 4: Performance of FVs, CNNs, and LBPsmeasured on the validation set.

Figure 5 :

 5 Figure 5: Qualitative results comparison of the proposed method with other methods. Top 7 attributes predicted by these methods are shown. As before green color indicates relevant attributes whereas red color indicates irrelevant attributes for the image. (Better viewed in color)

Table 1 :

 1 Details of parameters of proposed network

						Feed-forward	Back-propagate
		A			B	C
						Feature
						specific
						parameters
						Merging
						Common
						parameters
						Sigmoid
				smiling ……..	male …….	bald	layer
			Figure 2: Illustration of proposed method.
	Parameters	Layer Type	A	B	C
	Type
		Input	xA	xB	xC
	Feature	FC(ReLU) 4096 4096 4096
	Specific	FC(ReLU) 1024 1024 1024
	Merge	Add		1024
		FC(ReLU)		1024
	Common	FC(ReLU)		1024
		Sigmoid		40

Table 4 :

 4 Comparing the proposed methods with other methods using dedicated networks. The table shows that the performance of the proposed methods is competitive to the one of dedicated networks, while the performance of other compared methods is significantly low, particularly in the case of LBPs.

			Method	mean Avg. Precision
			AllFeatNet	63.4 ± 9.5 %
			Mod-Drop	67.8 ± 3.7 %
			Ours(FVNetInit)	68.8 ± 3.0%
	Ours(AllFeatNetInit)	69.0 ± 3.4%
	Table 3: mean AP(mAP) of multi-feature baselines
		75 80	Performance comparison Ours(FVNetInit) Ours(AllFeatNetInit) AllFeatNet
		70
		65
	AP	60
		55
		50
		45
		Feature(s) at test time FCL FCx xCL FxL Fxx xCx xxL 40
	Figure 3: Performance comparison between differ-
	ent methods and different combinations of feature(s)
	at test time. FCL represents FVs, CNNs, and LBPs
	respectively. 'x' denotes the absence of the corre-
	sponding feature.
	pects of the proposed method. First, we demonstrate that
	the performance due to individual features are retained af-
	ter merging all the features in the same common subspace.
	Second, we demonstrate that the performance is improved
	in the presence of more information, i.e. presence of multiple
	types of features at a time.
	Performance comparison with Dedicated Networks.
	Tab. 2 and Tab. 4 give the performance of single features net-
	works and their comparison with that of the multi-feature
	trained network (when, at test time, only one type of fea-
	ture is present). From these tables, we can observe that,
	with both our approaches, the performance of the compo-
	nent features at test time is competitive to that of dedicated
	networks trained with those features only. Compared to ex-
	isting methods such as Mod-Drop and AllFeatNet, the range
	of performance drops in comparison to dedicated networks is
	the least in our case. More precisely, the widest drop range
	for us is up to -2.8% w.r.t. that of LBPNet in AllFeat-
	NetInit network. While for the same feature, it is up to
	-4.7% in Mod-Drop and up to -21.8% in AllFeatNet w.r.t.
	that of LBPNet. These results clearly demonstrate that our
	method is more robust in retaining the performances of indi-
	vidual features while projecting them in common subspace.
	Performance comparison with Multi-feature Networks.
	Table 3 compares the mean average precision (mAP) of dif-
	ferent multiple features based networks with the proposed
	method. For a network with 3 different types of input fea-
	tures, there are 7 different possible combinations of fea-
	ture(s) at test time. The performance shown in the ta-
	ble is the mean AP obtained with all these combinations.
	The proposed method outperforms the other multi-feature-

http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html

Acknowledgments This project is funded in part by the ANR (grant ANR-12-SECU-0005).


Michel Panfilov

Stéphane Zaleski

Christophe Josserand

Mechanisms and physico-chemical model of formation of natural hydrogen reservoirs in thermal aquifers

Keywords: serpentinization, hydrogen diffusion, two-phase flow, phase appearance, methanogen bacteria

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Underground hydrogen reservoirs, whose existence has been confirmed recently by geologists all over the world, represent a new source of renewable energy. Hydrogen is formed as the product of reaction of serpentinization between ferromagnesian minerals of the Mantle and water in the offshore zones of subduction or in continental zones at considerable depth beneath hydrothermal aquifers. Initially dissolved in water, hydrogen can create free gas bubbles when the local concentration of H2 dissolved in water exceeds the equilibrium value. The bubbles raise up and create the gas cap of free hydrogen. This process is significantly influenced by methanogen bacteria inhibiting in aquifers and consuming hydrogen for their metabolism. The reactions initiated by bacteria leads to the appearance of other volatile components as methane. The dimension, the form of the gas cap and the concentration of hydrogen and methane determine the efficiency of gas production and the energy potential of the reservoir.

In the present paper we develop the conceptual mathematical model of gas cap formation in hydrogen reservoir. The bacterial activity is described by the equation of population dynamics with specific kinetic functions obtained by the authors by treating experimental data.

The formation of free gas is modelled in terms of the formation of oversaturated nuclei, their growth and simultaneous motion of isolated bubbles. As far as the bubbles grow, they stick together and create continuous free gas. Thus, the hydrogen motion consists of two stages: (i) the motion of isolated bubbles, controlled by the equations of ganglion dynamics; and (ii) the motion of continuous gas through water, governed by Darcy's law. The bubble growth is ensured by the mass exchange with hydrogen dissolved in water, which involves a non-equilibrium in the average hydrogen concentrations.

Such a problem of hydrogen raising is solved analytically in simplified situation without bacteria. The problem can be reduced to a system of nonlinear hyperbolic equations, which has either continuous or discontinuous solutions (shock waves) depending on the degree of the non-equilibrium. Multiples deformations of the shocks under gas raising are described.

The impact of bacteria is studied numerically by using the open source Basilisk (developed by D'Alembert Institute). We have revealed non trivial scenarios of the appearance of piece-wise constant traveling waves, or auto-oscillations caused by the nonlinear population dynamics.

The result obtained enables us to give estimation to the characteristic time of gas cap formation and the evolution of its composition in time.


Michel Panfilov

Fikri Kuchuk

Kuchuk Fikri

Deep

M Panflov

Deep nanoscale desorption of gas in shale

Keywords: diffusion through solid, multiscale medium, homogenization, shale gas

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

In our paper we reveal three other non-trivial mechanisms of gas behaviour in shale: the deep desorption, the nonlocal phenomena generated by multiscale structure of shales, and the effect of super-memory.

The deep desorption was observed experimentally [for instance, Etminan etal, Int. J Coal Geology, 123 (2014)], and represents the gas release from deep zones of organic inclusions (kerogen) due to the diffusion of dissolved methane through the amorphous organic matter towards pore network. Although this diffusion is very slow, the small sizes of organic inclusions determine the finite time of gas release. In contrast to the superficial desorption of gas, which is an instantaneous process, the deep gas desorption introduces a significant delay, or memory, in the macroscopic behaviour of the system.

The appearance of the memory is the second new effect typical for shale, which is caused by its multiscale structure. The memory appears in all the types of mass transfers (not only in deep desorption) between the scales due to highly different flow rates characterizing various scales. Consequently, the mass transfer terms in conservation equations become integro-differential.

The mathematical model for such a process has been obtained by reiterated homogenization. For multiscale medium, the memory accumulation on different scales leads to the appearance of super-memory (the kernel of the integral operator is the solution of another integral equation on lower scale). For finite number of scales, the macroscopic model represents a recurrent system of integro-differential equations.

In the limit of infinite number of scales, this system can be split and reduced to a closed non-recurrent model of two integro-differential equations, in which the effect of super-memory leads to the appearance of a nonlinearity. By numerical simulations we have shown that this limit model describes well heterogeneous media with three-four and more scales.