Pablo Mesejo

Sandrine Saillet

Olivier David

Christian Bénar

Jan M Warnking

Florence Forbes

Estimating Biophysical Parameters from BOLD Signals through Evolutionary-based Optimization

Keywords: functional MRI, BOLD signal, Biophysical parameters, Evolutionary Computation, Differential Evolution, Expectation Maximization

Physiological and biophysical models have been proposed to link neural activity to the Blood Oxygen Level-Dependent (BOLD) signal in functional MRI (fMRI). They rely on a set of parameter values that cannot always be extracted from the literature. Their estimation is challenging because there are more than 10 potentially interesting parameters involved in non-linear equations and whose interactions may result in identifiability issues. However, the availability of statistical prior knowledge on these parameters can greatly simplify the estimation task. In this work we focus on the extended Balloon model and propose the estimation of 15 parameters using an Evolutionary Computation (EC) global search method. To combine both the ability to escape local optima and to incorporate prior knowledge, we derive the EC objective function from Bayesian modeling. This novel method provides promising results on a challenging real fMRI data set involving rats with epileptic activity and compares favorably with the conventional Expectation Maximization Gauss-Newton approach.

Introduction

In the past decade, physiological models have been proposed to describe the processes that link the neural and the hemodynamic activity in the brain. Different variations of the widely used "Balloon model" [START_REF] Buxton | Dynamics of blood flow and oxygenation changes during Brain activation: the balloon model[END_REF] have been introduced that provide a complete description of the physiological processes underlying hemodynamic activity, from neural activation to the Blood-Oxygen-Level-Dependent (BOLD) effect measurement [START_REF] Buxton | Modeling the hemodynamic response to brain activation[END_REF][START_REF] Friston | Nonlinear responses in fMRI: the balloon model, Volterra kernels, and other hemodynamics[END_REF]. These models all depend on physiological parameters for which different competing values have been proposed in the literature, e.g. [START_REF] Friston | Nonlinear responses in fMRI: the balloon model, Volterra kernels, and other hemodynamics[END_REF][START_REF] Khalidov | Activelets: Wavelets for sparse representation of hemodynamic responses[END_REF]. Most approaches that focus on such models currently use one of these empirical sets of values with no real justification, e.g. [START_REF] Frau-Pascual | Physiological models comparison for the analysis of ASL fMRI data[END_REF][START_REF] Stephan | Comparing hemodynamic models with DCM[END_REF], although it has been shown in [START_REF] Frau-Pascual | Physiological models comparison for the analysis of ASL fMRI data[END_REF], and to a lesser extent in [START_REF] Stephan | Comparing hemodynamic models with DCM[END_REF], that the selection of these parameters had a more critical impact than the choice of the Balloon model variant itself, via their influence on the system dynamics. Estimating these physiological parameters from observed data may therefore be of interest in a number of fMRI studies. A general method for estimating parameters involved in a dynamic system has been proposed [START_REF] Friston | Bayesian estimation of dynamical systems: an application to fMRI[END_REF] based on a Bayesian approach which allows the incorporation of prior knowledge. Such a priori knowledge is typically summarized by a Gaussian distribution for each physiological parameter and provides a generally accepted consensus avoiding the commitment to arbitrarily fixed values. The method in [START_REF] Friston | Bayesian estimation of dynamical systems: an application to fMRI[END_REF] has then been widely used as the method of reference to estimate the hemodynamic response in dynamical causal modelling (DCM). It is based on an Expectation-Maximization Gauss-Newton search (EM/GN) which requires the linearization of the original system and approximation such as the Laplace approximation for tractability. The EM algorithm is also more generally known to be sensitive to initialization and prone to get stuck in local optima. Alternative approaches include sampling, e.g. Monte Carlo Markov Chain (MCMC), or other stochastic techniques, e.g. Metaheuristics (MHs). Sampling techniques offer a number of attractive features such as robust and reliable performance and ability to escape local optima. MHs are in addition general purpose procedures that do not even require the availability of the objective function in analytic form. In the DCM context, the need for the Laplace approximation is relaxed by [START_REF] Chumbley | A Metropolis-Hastings algorithm for dynamic causal models[END_REF] which uses a MCMC implementation of the Bayesian inversion scheme of [START_REF] Friston | Bayesian estimation of dynamical systems: an application to fMRI[END_REF] and shows that the Laplace approximation actually yields sensible inferences under a large set of conditions. However, MCMC needs thousands of iterations to converge, constraints are not easy to introduce and it does not provide mechanisms to control the trade-off exploration-exploitation. Also, [START_REF] Chumbley | A Metropolis-Hastings algorithm for dynamic causal models[END_REF] focuses on DCM and neuronal parameter estimation while nothing is reported on the impact on the non-neuronal physiological parameters. In contrast, in [START_REF] Vakorin | Inferring neural activity from BOLD signals through nonlinear optimization[END_REF] the authors consider the Balloon model in a non-Bayesian setting using standard MHs with an objective, or so-called fitness function which does not include prior information. Without such valuable prior knowledge, it is quite challenging to put all parameters into the proposed optimization scheme due to potential identifiability issues. It results that the approach in [START_REF] Vakorin | Inferring neural activity from BOLD signals through nonlinear optimization[END_REF] is limited to the estimation of three of the physiological parameters out of the 15 considered in this paper.

In this work, our goal is to combine both the benefits from a Bayesian approach which allows incorporation of prior knowledge and from MHs which are generalpurpose global optimization techniques able to avoid local optima. Following the Bayesian inversion scheme of [START_REF] Friston | Bayesian estimation of dynamical systems: an application to fMRI[END_REF], we derive a fitness function that is directly comparable to EM/GN search DCM standard. It follows an estimation procedure able to estimate all physiological parameters of interest while being less likely to get trapped in local minima. This novel method is assessed on a challenging real EEG/fMRI data set involving rats with epileptic activity. A qualitative comparison with the EM/GN approach shows the ability of our method to provide more physiologically sensible parameter values.

The extended Balloon model

The Balloon model was first proposed in [START_REF] Buxton | Dynamics of blood flow and oxygenation changes during Brain activation: the balloon model[END_REF] to link neuronal and vascular processes by considering the venous vascular compartment as a balloon that inflates under the effect of blood flow variations. More specifically, the model describes how, after some stimulation, the local blood flow f in (t) increases and leads to the subsequent augmentation of the local deoxygenated blood volume v(t). The incoming blood is strongly oxygenated, and since the relative blood flow increase exceeds the increase in oxygen consumption, local deoxyhemoglobin concentration q(t) decreases and induces a BOLD signal increase. The Balloon model was subsequently extended [START_REF] Friston | Nonlinear responses in fMRI: the balloon model, Volterra kernels, and other hemodynamics[END_REF] to include the effect of the neuronal activity on the variation of some auto-regulated flow inducing signal s(t) so as to eventually link neuronal to hemodynamic activity. Variable n e (t) represents the activity of the excitatory neuron population and n i (t) the inhibitory neuron population [START_REF] Marreiros | Dynamic causal modelling for fMRI: A two-state model[END_REF]. The experimentally controlled input function (stimulus) is represented by u(t). In the following, the explicit time dependence '(t)' of the state variables will be omitted for compactness. The global physiological model corresponds then to a non-linear system with six state variables x = {n e , n i , s, f in , v, q} related to the excitatory and inhibitory neuronal activity, normalized flow inducing signal, local blood flow, local deoxygenated blood volume, and deoxyhemoglobin concentration. Their interactions over time are described by the following non-linear differential equations:

dn e dt = -En e -exp   A + Bu se + D T    n e s f in -1       n i + Cu se dn i dt = n e -2En i , ds dt = n e -sd s -ar (f in -1) , d ln(f in ) dt = s f in (1)
d ln(v) dt = 1 tt f in -v 1 α v , d ln(q) dt = 1 tt 1 -(1 -E 0 ) 1 f in E 0 f in q -v 1 α -1
From these state variables, the observed BOLD signal y is derived using an observation equation that includes intra-and extravascular BOLD signal components [START_REF] Buxton | Modeling the hemodynamic response to brain activation[END_REF]:

y = V 0 k 1 (1 -q) + k 2 1 - q v + k 3 (1 -v) (2)
where k 1 , k 2 , k 3 are physiology-and scanner-dependent constants

k 1 = 4.3 θ 0 E 0 T E, k 2 = ε r 0 E 0 T E and k 3 = 1 -ε.
The value θ 0 is the frequency offset at the outer surface of the magnetized vessel for fully deoxygenated blood, it is equal to 40.3 Hz • b 0 /1.5 T, where b 0 is the magnetic field strength. T E is the echo time and r 0 is the slope of the relation between the intravascular relaxation rate and oxygen saturation, which is set to 300 Hz [START_REF] Silvennoinen | Comparison of the dependence of blood R2 and R2 * on oxygen saturation at 1.5 and 4.7 Tesla[END_REF]. E 0 is the oxygen extraction fraction at rest and is considered as a free parameter as well as ε, the ratio of intra to extravascular signal.

The remaining parameters are the following: A, B, C, D and E are parameters as in non-linear Dynamic Causal Models [START_REF] Stephan | Nonlinear dynamic causal models for fMRI[END_REF], D = (D 1 , D 2 , D 3 ) T being the new component in the non-linear state equation above. Parameter se is the spike exponent introduced for the present dataset to control the scaling of the synaptic activity with respect to the spike amplitude derived from local field potentials (LFPs), sd is the vasodilatory signal decay, ar is the rate constant for autoregulatory feedback by blood flow, and tt represents the transit time of blood from the arteriolar to the venous compartment. The Grubb's vessel stiffness exponent corresponds to α, while V 0 is the resting venous cerebral blood volume fraction. The whole model depends on 15 different scalar parameters to optimize θ = {A, B, C, D, E, se, sd, ar, tt, α, E 0 , V 0 , ε}.

Bayesian estimation of dynamical systems

MHs require the definition of a fitness function to measure the goodness of the parameters found. We use the Bayesian inversion scheme of [START_REF] Friston | Bayesian estimation of dynamical systems: an application to fMRI[END_REF] to derive an appropriate fitness function. In the Balloon model, the first part describes the transitional dynamics of the state vector x = {n e , n i , s, f in , v, q}. The system is defined as dx dt = f (x, u, ψ), with ψ = {A, B, C, D, E, se, sd, ar, tt, α, E 0 }. The second part of the model is the observational equation for the BOLD signal y which is assumed to be observed with some additive Gaussian noise (in this context of BOLD data sampled at discrete time points, we represent both data and state variables as vectors of discrete samples), y = g(x, φ)+η , with φ = {V 0 , E 0 , ε} and η is a random error vector distributed according to the Gaussian distribution N (0, σ 2 η I) assuming unstructured noise. Under additional distributional assumptions about the model parameters θ = {ψ, φ} and noise variance σ 2 η , we can apply Bayesian inference. In [START_REF] Friston | Bayesian estimation of dynamical systems: an application to fMRI[END_REF], Gaussian priors are chosen for all parameters. As explained in [START_REF] Stephan | Comparing hemodynamic models with DCM[END_REF] for ε, it is more natural to use log-normal priors for parameters that are positive. A simple way to account for positivity while remaining in a Gaussian setting is to change the model parameterization. We consider equivalently θ = { Ã, B, C, D, Ẽ, se, sd, ãr, tt, α, Ẽ0 , Ṽ0 , ε}, where { Ã, B, C, D} = {A, B, C, D} remain unchanged while the other parameters take the form θ = log(θ/µ θ ) where the specific µ θ values may depend on the experiment (see section 5). An exception is E 0 for which we set E 0 = arctan( Ẽ0 +tan(π(µ E0 -0.5)))/π+0.5 in order to ensure E 0 ∈ [0, 1]. Gaussian priors can then be assumed for θ and the state and observational equations above lead to,

y = h( θ, u) + η , with η ∼ N (0, σ 2 η I), θ ∼ N ( θ, Σ θ ) and σ 2 η ∼ p(σ 2 η ) .
As another difference with [START_REF] Friston | Bayesian estimation of dynamical systems: an application to fMRI[END_REF][START_REF] Stephan | Comparing hemodynamic models with DCM[END_REF], we use a semi-conjugate prior for the unknown parameters ( θ, σ 2 η ) in which θ ∼ N ( θ, Σ θ ) independently of σ 2 η and a noninformative prior is used for σ 2 η , i.e. p(σ 2 η ) ∝ (σ 2 η ) -1 . Bayesian inference is then based on the posterior distribution p( θ, σ 2 η |y) ∝ p(y| θ, σ 2 η ) p( θ) p(σ 2 η ) whose mode provides the maximum a posteriori (MAP) estimate:

( θ, σ 2 η ) M AP = arg max θ,σ 2 η {log p(y| θ, σ 2 η ) + log p( θ) + log p(σ 2 η )} (3)
= arg min θ,σ 2 η {(N + 2) log σ 2 η + ||y -h( θ, u)|| 2 σ 2 η + ( θ -θ) T Σ -1 θ ( θ -θ)} .
where N is the y signal length. Setting to zero the gradient with respect to

σ 2 η yields (σ 2 η ) M AP = ||y-h( θMAP ,u)|| 2 N +2 . Plugin in (σ 2 η ) M AP into expression (3) leads to θMAP = arg min θ {(N + 2) log ||y -h( θ, u)|| 2 + ( θ -θ) T Σ -1 θ ( θ -θ)} . ( 4
)
The left-hand side above corresponds to our fitness function to be used in the EC framework described in the next section. In contrast to the conventional Laplace approximation and EM estimation algorithm, EC does not require the linearization or approximation of h( θ, u). It does not require an analytic form of the likelihood and h( θ, u) can typically be used as a numerical function. Another advantage of EC is its flexibility in particular as regards hard constraints often imposed for stability of the differential equations (1). The hyperparameters θ and Σ θ are specified in section 5.

Evolutionary Computation

Evolutionary Computation (EC) methods are population-based MH algorithms [START_REF] Eiben | Introduction to Evolutionary Computing[END_REF]. They include several computational models that reproduce natural evolution processes to reach a target which is generally represented as a fitness function to optimize. In practice, they implement an iterative process (artificial evolution) in which solutions improve over generations until they converge to an optimum, starting from an initial pool of randomly generated solutions. EC procedures are based on achieving a trade-off between intensification (exploitation of the best solutions, usually through selection operators and replacement strategies) and diversification (exploration of the search space thanks to crossover and mutation operators).

In this work, we choose to use Differential Evolution (DE) [START_REF] Das | Differential Evolution: A Survey of the State-of-the-Art[END_REF], which has recently been shown to be one of the most successful EC methods for global continuous optimization. DE perturbs individuals in the current generation by the scaled differences of other randomly selected and distinct individuals. In DE, each individual acts as a parent vector, and for each of them a new solution, called donor vector, is created. In the basic version of DE, the donor vector for the i th parent ( θi ) is generated by combining three random and distinct elements θr1 , θr2 and θr3 . The donor vector V i is computed as

V i = θr1 + F • ( θr2 -θr3 )
, where F (scale factor) is a parameter that strongly influences DE's performance and typically lies in the interval [0.4, 1]. The original method described above is called DE/rand/1, which means that the first element of the donor vector equation θr1 is randomly chosen and only one difference vector (in this case θr2 -θr3 ) is added. After mutation, every parent-donor pair generates a child (called trial vector) by means of a crossover operation. The crossover is applied with a certain probability, defined by a parameter Cr (crossover rate) that, like F , is one of the control parameters of DE. Then, the trial vector is evaluated and its fitness is compared to the parent's. The best, in terms of fitness, survives and will be part of the next generation.

Experimental results

The BOLD data used in the experiments was recorded with the goal of testing biophysical models in the context of epileptic activity in rats. An intracortical silica capillary was surgically implanted in the right primary somatosensory cortex of male Wistar rats (∼400 g) and subdural carbon EEG electrodes were placed close to the injection site and over the cerebellum. Epileptic activity was elicited using bicuculline methochloride (2.5 mM, 1 µl/5 min) injected intra-cortically during the MRI session. Simultaneous EEG and BOLD-fMRI data were acquired under <2% isoflurane anesthesia.

The EEG/fMRI data were acquired on a 4.7 T Advance III Bruker Biospec.In each scan, 300 volumes of five slices (0.25×0.25×0.8 mm 3 voxel size) were acquired using single-shot GE EPI with TE/TR of 20/600 ms. A total of 3-12 scans were performed for each of 12 rats (27 min of EEG/fMRI data per animal on average). The data from 3 animals were unexploitable and thus excluded from the analysis. Epileptic discharges (EDs) were automatically identified from the EEG data and ED amplitudes and onsets were recorded. For each rat, a single average fMRI signal concatenating all scans was extracted from the largest cluster of significantly active voxels identified in a linear analysis with a FIR hemodynamic response model. The fMRI signal size N ranged from 894 to 3576 with a median value of 2684. The EDs were entered in the biophysical model via the input function u as a series of short (8 ms) events.

To palliate the absence of ground truth (GT) in real data, a synthetic dataset is created to study the methods behavior under controlled conditions. Rat 5, whose physiological conditions are amongst the most stable ones, is selected as a reference to create this dataset, and the parameter estimates found by EM/GN are defined as the GT. BOLD signals are generated from either a full set of measured spikes or a subset (25%) to simulate more sparse events, adding AR(1) noise with three target SNRs (0.10, 0.46 and 2.15). The average distance to GT using 25% subsampling and the full set of spikes are 0.56 and 2.8 in EM/GN, and 0.28 ± 0.09 and 0.15 ± 0.01 in DE, respectively.

Then, for each rat, physiological parameters θ are estimated using DE with fitness function (4) and transforming back the resulting θ into θ. Each of the positive parameters in θ is specified using θ = log(θ/µ θ ) with µ θ defined respectively by {µ E , µ se , . . . , µ ε } = {1, 1, 0.64, 0.41, 0.98, 0.32, 0.55, 0.4, 1} [START_REF] David | Identifying neural drivers with functional MRI: an electrophysiological validation[END_REF]. The prior means are then set to θ = 0 and the prior covariance Σ θ is a diagonal matrix containing the prior variances set to {0.25, 0.25, 54.6, 0.05, 0.05, 0.05, 0.05, 0.14, 0.14, 0.05, 0.05, 0.007, 0.007, 0.05, 0.14} for each parameter in θ. The DE parameters used are among the most common ones in the state of the art [START_REF] Das | Differential Evolution: A Survey of the State-of-the-Art[END_REF]: F = 0.85, Cr = 1, with a DE/local-to-best/1 strategy that attempts a balance between robustness and fast convergence, and a population size of 150. The EM/GN algorithm [START_REF] Friston | Bayesian estimation of dynamical systems: an application to fMRI[END_REF] starts from physiologically reasonable parameter values (the prior means), which facilitates its convergence to a good solution. Since DE is a stochastic approach several runs need to be executed to evaluate its average performance. In this study, the number of runs per rat and the number of iterations per run are empirically set to 15 and 300, respectively. DE shows a very stable behavior with a standard deviation of the mean fitness values between 0.001% and 0.042%.

The θ parameter estimations from DE and EM/GN are shown in Table 1 . The experimental conditions for all animals were controlled as closely as possible. It is therefore expected that the physiologically meaningful parameters show limited variability across animals. However, the parameters related to the scaling of the stimulus and the neuronal signals (notably, A, B, C, D, E and se) may vary significantly, since the amplitudes of the elicited EEG responses varied between sessions. Estimates for 4 of the parameters that are expected to be among the most stable ones are shown in Figure 1. The ratio between intra-and extravascular signals, ε, depends on field strength, blood T 2 , and vascular geometry, but little on physiology. The values estimated by DE are markedly more stable between animals than the values obtained with EM/GN. Quantitatively, both methods yield plausible results. Blood transit times from arterioles to the deoxygenated vascular compartment estimated with DE are generally longer, and closer to the expected value, than those obtained with EM/GN, which are rather too short. For comparison, mean transit times across the entire vascular tree in a cortical voxel observed using DSC MRI in anesthesized rats are on the order of 1.6 s [START_REF] Coquery | Microvascular MRI and unsupervised clustering yields histology-resembling images in two rat models of glioma[END_REF]. Inversely, resting venous blood volumes estimated with DE are lower than those obtained with EM/GN. The prior estimate used here for this parameter, 4%, corresponds to the total cortical resting blood volume in isoflurane anesthesized male Wistar rats [START_REF] Coquery | Microvascular MRI and unsupervised clustering yields histology-resembling images in two rat models of glioma[END_REF]. In hindsight, the value that should actually be considered in the model is however only the venous (deoxygenated) fraction of that, such that a value of 2% actually seems much more realistic than the higher values of up to 6% estimated using EM/GN. Finally, both methods yield similar values for the resting oxygen extraction fraction. In summary, the estimates obtained from DE for these values seem both more realistic and more stable or at least as stable across sessions as the estimates from EM/GN.

Conclusion and future work

A novel method to infer physiological parameters from observed BOLD signals has been described, showing the robustness and flexibility of global search optimization methods while being able to incorporate prior information in a principled Bayesian way. This has not been proposed before and could have a strong impact on a number of fMRI studies. Traditionally, these parameters are manually set or, only few of them, are determined by using conventional but potentially suboptimal local search methods like EM. Preliminary results on synthetic and real data showed promising results providing sensible and more stable parameter estimates. Possible future research includes testing on multimodal fMRI data, since information from cerebral blood flow and volume dynamics may help to further improve the reliability of the parameter estimates.

  D2 D3 E se sd ar tt α V0 E0 ε RAT1 DE 0.56 0.02 0.10 0.01 0.02 0.05 0.84 1.44 0.89 0.53 0.83 0.32 0.38 0.55 0.90 EM/GN 0.14 -0.01 0.02 0.01 0.01 0.01 0.86 1.11 0.61 0.44 0.65 0.31 0.53 0.55 1.77 RAT2 DE 0.48 0.03 3.00 -0.14 -0.10 0.46 0.73 0.40 2.02 0.25 0.91 0.35 0.20 0.53 0.35 EM/GN 0.03 -0.00 0.02 0.01 0.01 0.01 0.96 0.60 0.73 0.36 0.44 0.30 0.58 0.54 2.13 RAT3 DE -0.66 -0.01 3.00 -0.14 -0.09 0.58 0.73 0.96 1.00 0.26 0.85 0.33 0.15 0.52 0.42 EM/GN 1.35 0.02 0.26 0.07 -0.01 0.18 0.72 0.83 1.12 0.59 0.65 0.32 0.34 0.54 0.94 RAT4 DE -0.14 0.01 2.18 0.31 -0.43 0.05 0.40 0.50 1.95 0.36 0.72 0.33 0.15 0.54 0.41 EM/GN -0.14 0.01 2.10 0.33 -0.42 0.07 0.41 0.50 1.86 0.35 0.71 0.33 0.16 0.54 0.41 RAT5 DE 1.11 0.03 1.60 -0.08 -0.11 0.17 0.57 0.82 2.05 0.46 0.67 0.35 0.23 0.54 0.36 EM/GN 1.07 0.03 1.21 -0.09 -0.12 0.19 0.56 0.78 2.07 0.46 0.65 0.34 0.26 0.55 0.39 RAT6 DE -0.85 0.01 3.00 0.23 0.24 0.18 0.44 0.33 1.15 0.25 0.97 0.33 0.22 0.52 0.49 EM/GN 0.21 -0.01 0.03 0.03 0.04 0.03 0.71 1.13 1.04 0.39 0.38 0.30 0.62 0.55 2.38 RAT7 DE 1.75 -0.03 0.09 -0.00 0.00 0.01 0.65 1.23 1.30 0.47 0.65 0.32 0.40 0.55 1.16 EM/GN 1.70 -0.03 0.07 -0.00 0.00 0.01 0.65 1.23 1.25 0.47 0.64 0.31 0.42 0.55 1.30 RAT8 DE 0.10 0.07 2.99 -0.43 -0.47 -0.02 0.39 0.19 1.92 0.14 0.60 0.31 0.22 0.53 0.49 EM/GN -0.67 0.08 0.30 -0.08 -0.10 0.11 0.50 0.14 1.40 0.57 0.51 0.31 0.32 0.54 1.14 RAT9 DE 0.74 0.03 0.55 0.02 -0.01 0.06 0.45 0.68 2.74 0.38 0.58 0.32 0.36 0.55 0.60 EM/GN 0.74 0.03 0.52 0.01 -0.01 0.07 0.45 0.68 2.71 0.38 0.57 0.32 0.37 0.55 0.62Table1. EM estimates and DE medians for each of the 9 rats and each parameter in θ.

Fig. 1 .

 1 Fig. 1. Boxplots of the DE runs and EM/GN results (green line) for ε, transit time (tt), V0 and E0. The parameter prior mean (resp. standard deviation) is indicated by the red (resp. blue) line.


Y Khidas

B Haffner

O Pitois

Critical size effect of particles reinforcing foamed composite materials

Keywords: Polymer foam, A. Particle reinforced composite, A. Polymer matrix composites, B. Mechanical properties, C. Elastic properties

We investigate the shear elastic modulus of soft polymer foams loaded with hard spherical particles and we show that, for constant bubble size and gas volume fraction, strengthening is strongly dependent on the size of those inclusions. Through an accurate control of the ratio λ that compares the particle size to the thickness of the struts in the foam structure, we evidence a transition in the mechanical behavior at ߣ ≈ 1. For ߣ < 1, every particle loading leads to a strengthening effect whose magnitude depends only on the particle volume fraction. On the contrary, for ߣ > 1, the strengthening effect weakens abruptly as a function of ߣ and a softening effect is even observed for ߣ ≳ 10. This transition in the mechanical behavior is reminiscent of the so-called "particle exclusion transition" that has been recently reported within the framework of drainage of foamy granular suspensions [Haffner B, Khidas Y, Pitois O. The drainage of foamy granular suspensions. J Colloid Interface Sci 2015;458:200-8]. It involves the evolution for the geometrical configuration of the particles with respect to the foam network, and it appears to control the mechanics of such foamy systems.

Introduction

The incorporation of particles into materials is a well-known strategy to improve their mechanical behavior. When dealing with foamy materials, one has to consider the typical size of the struts that form the solid skeleton compared to the filler size. Several studies have been devoted to the reinforcement of such foams. For example, polyurethane foams were filled with glass fibres [START_REF] Cotgreave | The mechanism of reinforcement of polyurethane foam by high-modulus chopped fibres[END_REF], metallic powders [START_REF] Goods | Mechanical properties of a particle-strengthened polyurethane foam[END_REF], organic particles [START_REF] König | Flexible Polyurethane Foam with the Flame-retardant Melamine[END_REF] or mineral particles [START_REF] Barma | Mechanical properties of particulate-filled polyurethane foams[END_REF][START_REF] Siegmann | Mechanical behavior of reinforced polyurethane foams[END_REF][START_REF] Vaidya | Flexural Properties of Mica Filled Polyurethane Foams[END_REF][START_REF] Saint-Michel | Mechanical properties of high density polyurethane foams: II Effect of the filler size[END_REF], … Aerated cementitious materials also belong to the class of foamed composite materials. In the current climate of sustainable development, those foam-based materials are destined to expand and in certain cases to replace advantageously conventional building materials. Numerous works were focused on those materials in order to improve their mechanics [START_REF] Tonyan | Structure and mechanics of cement foams[END_REF][START_REF] Mccarthy | Preliminary views on the potential of foamed concrete as a structural material[END_REF] by adding different types of filler, such as sand [START_REF] Nambiar | Influence of filler type on the properties of foam concrete[END_REF], fly ashes [START_REF] Nambiar | Influence of filler type on the properties of foam concrete[END_REF][START_REF] Kearsley | The effect of high volumes of ungraded fly ash on the properties of foamed concrete[END_REF], glass fibers [START_REF] Akthar | High porosity (> 90%) cementitious foams[END_REF], … One of the essential features of foamy composite materials is that the filler size has to be fine enough for strengthening to be observed [START_REF] Goods | Mechanical properties of a particle-strengthened polyurethane foam[END_REF][START_REF] König | Flexible Polyurethane Foam with the Flame-retardant Melamine[END_REF][START_REF] Barma | Mechanical properties of particulate-filled polyurethane foams[END_REF][START_REF] Siegmann | Mechanical behavior of reinforced polyurethane foams[END_REF][START_REF] Nambiar | Influence of filler type on the properties of foam concrete[END_REF], and it has been shown that reinforcement is not efficient if the added particles are bigger than the bubbles [START_REF] Saint-Michel | Mechanical properties of high density polyurethane foams: II Effect of the filler size[END_REF].

The effect of the filler size has been attributed to the fact that only the small particles can enter the struts that form the foam skeleton. Therefore, the distinction that can be made for the mechanics of foams involving "small" and "large" particles is necessary related to distinct geometrical configurations for the particles within the foam. Surprisingly, whereas this distinction is expected to be based on geometry only, it has never been studied from the purely geometrical point of view. Indeed, most of reported results correspond to the global effect of the filler, including filler-matrix

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT

3 interaction [START_REF] Barma | Mechanical properties of particulate-filled polyurethane foams[END_REF] and complex interaction in the chemical reaction from which bubbles are generated [START_REF] Barma | Mechanical properties of particulate-filled polyurethane foams[END_REF][START_REF] Vaidya | Flexural Properties of Mica Filled Polyurethane Foams[END_REF][START_REF] Nambiar | Air-void characterisation of foam concrete[END_REF]. Moreover, the bubble size and the gas volume fraction cannot be kept constant when studying the effect of the loading, which increases the difficulty to disentangle all types of effect. Consequently, our understanding of the geometrical effect for filler particles in solid foams is rather qualitative, whereas the engineering of such complex systems requires a quantitative estimate for this effect. Clearly, a suitable approach would consist in using perfectly inert particles and changing the particle loading only, all relevant parameters of the reference foam, i.e. the unloaded foam, being unchanged.

In this paper, we propose a generic approach to better understand the subtle interplay between the complex geometry of particulate foams and their mechanics. We investigate the shear elastic modulus of systems which consist of soft polymer foams loaded with hard spherical particles. The experimental study is conducted in such a way that monodisperse precursor foams are unaltered by the loading process, allowing us to assess the strengthening effect due solely to the presence of the particles.

Experimental

sample preparation

Our model system is a soft polymer foam loaded with hard spherical inclusions. The soft polymer is a porcine gelatin. The gelation is thermoreversible: above T gel ≈ 29°C gelatin is a liquid solution and below it becomes a soft elastic solid. This reversibility offers an easy-to-use material to generate reproducible foams thanks to foaming

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 4
processes developed for aqueous foam [START_REF] Haffner | Flow and jamming of granular suspensions in foams[END_REF]. The gelatin powder was kindly supplied by Rousselot® and used as received. The inclusions are polystyrene beads (dynoseeds® from microbeads SA) characterized by a weak polydispersity (standard deviation of particle size distributions around 5%). The large contrast in the elastic moduli of gelatin (few kPa) and polystyrene (few Gpa) will allow us to consider the beads as hard spheres.

In addition to gelatin, the foaming solution is composed of distilled water, glycerol to match the density of the beads (1.05) and trimethyl(tetradecyl)azanium bromide (TTAB) as a surfactant. Glycerol and TTAB were purchased from Aldrich and used as received. For the preparation, an aqueous solution of TTAB and glycerol is first prepared. Then, appropriated amounts of gelatin powder were dissolved under continuous stirring in the previous TTAB aqueous solution at 65 °C during 30 min to reach the final concentrations in gelatin 13.5 wt%, TTAB 0.3 wt% (well above the CMC of 1 g/L) and glycerol 12 wt%. After ultrasonic degassing during 1 min, the solution is kept at 50°C during 15 h. In the foaming solution, the polystyrene particles behave as fully hydrophilic particles and they do not adsorb at bubble interfaces. The Fig. 1 is an overview of the experimental procedure used to generate the particle laden foams. Inside a box maintained at 60°C, the liquid gelatin solution is mixed with nitrogen in a T-junction to produce a precursor monodisperse foam. The gas fraction of this foam is set by controlling the flow rates of the liquid and the gas, using a syringe pump and a mass flow controller respectively. In a second step, a suspension of particles in liquid gelatin is pushed and mixed to the precursor foam thanks to a Tjunction. The resulting particulate foam is pushed in an open cell composed of two transparent slabs attached to two spacers that impose the gap (Fig. 2). We rapidly quench the system in a refrigerator at 5°C during 5 min and then wait 55 min more at room temperature before the mechanical test. It is crucial that all the samples have the same temperature history. Indeed gelatin gels are known to exhibit logarithmic structural aging that leads to a natural increase of their elastic moduli [START_REF] Normand | Gelation kinetics of gelatin: a master curve and network modeling[END_REF]. In the same time, some water evaporation also contribute to make the system evolve, so that it is essential to follow the same protocol in order to have the required reproducibility on the mechanical results.

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 6

In the present study, the gas volume fraction of the particle-laden foam is kept constant: ϕ = 0.8. The particle size and the particle volume fraction are varied within the respective ranges: dp = 6 -500 µm and ϕp = 0 -6 %. ϕp corresponds to the particle volume over whole volume of foam, but it will be useful to define the particle volume fraction with respect to the foam skeleton: φp = ϕp/(1-ϕ) = 0 -30 %. In order to study the effect due solely to the presence of particles in the foam, it is of first importance that making dp and ϕp vary do not modify the bubble size. Thanks to a microscope and to image analysis, we have checked that the bubble monodispersity is not degraded by the loading, and the bubble diameter was measured to be constant and equal to Db = 400 ± 20 µm whatever the loading level. The use of fluorescent particles enables us to check that particles are uniformly distributed in the whole foam. Note that particles are always larger than the thickness of the foam films, so that they are excluded from the latter rapidly after the generation stage. Particles were observed to be well-distributed in nodes and plateau. The typical size of our foam sample is 70 x 25 x 7.7 mm 3 . Fig. 2. The foam is poured in a cell composed of two transparent slabs with a controlled gap thanks to two spacers. For sake of clarity, the fixing device between the slabs and the spacers is not represented. All the results presented in this paper are performed at the constant shear rate ߛሶ = 1.3•10 -3 s -1 . The high adhesive strength of gelatin avoids any wall sliding during the test [START_REF] Johnson | Surface Energy and the Contact of Elastic Solids[END_REF]. Thanks to the transparent slab and image analysis, we measure the surface area of each sample, allowing for the stress to be plotted as a function of the strain. The stress-strain curve on Fig. 3 illustrates that the strain is small enough to remain in the linear elastic regime and that the shear modulus ܩ൫߶ , ݀ ൯ can be simply deduced from the slope of the linear fit over a strain of 2.6 %. As an example, we can see on Fig. 3 how 5% of 20 µm particles strengthen the reference foam.

Results and discussions

The influence of the particle loading will be characterized by normalizing the shear modulus of particle-laden foams by the value for the corresponding particle-free foam, i.e. ܩ * = ܩ൫߶, ߮ ൯/,߶(ܩ 0). Fig. 4 shows the evolution of the normalized shear modulus as a function of the particle volume fraction for several particle sizes, where two behaviors can be distinguished. The shear moduli of foams loaded with particles of diameter below 40 µm follow the same increasing function of ϕp. Quantitatively, when particles are small enough compared to the bubble size, the shear modulus of particulate foam is roughly 3 times larger than the one of particle-free foam when only 6% of particles are added to the foam. In contrast, when the particle size increases, the

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT

9 stiffening can become negligible, as observed for the 250 µm particles. We can even notice a weak softening effect for the 500 µm particles. For that case, note that particle size becomes comparable to the bubble size. Each symbol corresponds to different particle size as indicated in the legend. The solid line corresponds to Eq. ( 3) and the dashed one to Eq. ( 5). The errors bars represent the fluctuations obtained on four measurements.

To understand this stiffening-softening transition, it is necessary to consider the foam at the scale of the solid skeleton, or foam network. The size of interest is ݀ , the diameter of the circle inscribed inside the minimal cross section of the struts, i.e. the

M A N U S C R I P T A C C E P T E D

ACCEPTED MANUSCRIPT 10 constrictions (Fig. 5). In order to compare the particle size with ݀ , we will use the socalled confinement parameter λ [START_REF] Louvet | Capture of particles in soft porous media[END_REF]:

ߣ = ݀ ݀ = 1 + 0.57(1 -߶) .ଶ 0.27ඥ1 -߶ + 3.17(1 -߶) ଶ.ହ ݀ ܦ (1)
For ߶ = 0.8, Eq. ( 1) gives ߣ ≃ 8.5 ݀ ܦ ⁄ . In [START_REF] Louvet | Capture of particles in soft porous media[END_REF] this geometrical parameter has been determined from both experiments involving the trapping/release of a single particle in aqueous foams and numerical simulations of foam structures. When ߣ < 1, particles can be included everywhere in the solid skeleton without deformation whereas when ߣ ≳ 1, particle deform the bubbles surface. One can also imagine that for ߣ ≫ 1 the particles cannot be included anymore in the space defined by the foam network but instead the network is restructured around them. Thus, for a given particle volume fraction, the resulting particle configuration in the foam network depends crucially on λ, which is expected to have a significant influence on the mechanical behavior.

M A N U S C R I P T A C C E P T E D

To quantify this influence, let's first consider the asymptotic case of small particles, i.e.

ߣ ≪ 1. The corresponding shear modulus will be called ܩ ௫ *

. In one hand, the elastic modulus of solid foams normalized by the modulus of the solid skeleton is a decreasing function of the gas volume fraction݂(߶) [START_REF] Gibson | Cellular Solids: Structure and Properties[END_REF]. On the other hand, the modulus of an elastic matrix with rigid inclusions normalized by the modulus of the matrix is an increasing function of the particle fraction ݃൫߮ ൯ [START_REF] Hashin | A variational approach to the theory of the elastic behaviour of multiphase materials[END_REF]. Thus, when the particles are small enough, the stiffness of the elastic skeleton is set by the particle volume fraction it contains. The reduced shear modulus of particulate foams can be written as a combination of those two functions:

ܩ ௫ * = ீ൫థ,ఝ ൯ ீ൫,ఝ ൯ ீ൫,ఝ ൯ ீ(,) ீ(,) ீ(థ,) = ݂(߶). ݃൫߮ ൯. ሾ݂(߶)ሿ ିଵ = ݃൫߮ ൯ (2).
In order to evaluate the function ݃൫߮ ൯ in the present foamy systems, we measured the reduced modulus of the gelatin matrix with several particle loadings (Fig. 6.a). The dashed-line in Fig. 6.a (݃൫߮ ൯ = ଶାଷఝ ଶ൫ଵିఝ ൯

) is the Hashin-Shtrikman lower bound for an isotropic elastic matrix with rigid inclusions [START_REF] Gibson | Cellular Solids: Structure and Properties[END_REF]. The discrepancy for the highest particle volume fraction reflects that this approach is not suitable to catch the divergence of elastic behavior when particles percolate around ϕ p = 0.6 [START_REF] Chateau | Homogenization approach to the behavior of suspensions of noncolloidal particles in yield stress fluids[END_REF]. Thus we make use of the Krieger-Dougherty equation [START_REF] Krieger | A Mechanism for Non-Newtonian Flow in Suspensions of Rigid Spheres[END_REF] to fit our data as suggested by previous studies on yield stress fluids loaded with particles [START_REF] Mahaut | Yield stress and elastic modulus of suspensions of noncolloidal particles in yield stress fluids[END_REF]:

ܩ ௫ * ൫߮ ൯ = ݃൫߮ ൯ ≃ ൫1 -߮ ߮ * ⁄ ൯ ିଶ.ହఝ * (3) M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 12
where ߮ * ≈ 0.6. This phenomenological function can be used to estimate ܩ ௫ * ൫߮ ൯, which is found to be in good agreement with experimental data for particles smaller than 40 µm, i.e. λ ≤ 0.84, as shown in Fig. 4. This agreement can be considered as partly surprising because the underlying assumption of Eq. ( 3) is ߣ ≪ 1. This indicates that a specific modeling should be developed to clarify the situations corresponding to , the reduced shear modulus of a particle-free gelatin foam for different gas volume fraction ϕ. The circles are experimental data and the solid line corresponds to Eq. (

ߣ ≲ 1.
Let's now consider the case of ܩ * , which corresponds to the case of very large particles compared to the size of constrictions, i.e. ߣ ≫ 1. That situation should be understood as particles large enough to be fully excluded from the space defined by the solid network, as illustrated in Fig. 7 for ߣ ≈ 10. In such a case, the system is

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT
13 composed of particle-free foam that embeds large inclusions. As the particles no more contribute to the volume of the network, the gas fraction of the embedding foam, ߶ ᇱ , is therefore increased with respect to the gas fraction in the whole system, ߶. The relation between ߶ ᇱ and ߶ is:

߶ ᇱ = ߶/(1 -߶ ).
In combining the effect of ߶ on ߶ ᇱ , with the effect of gas fraction on the shear modulus of foams, one can deduce that the shear modulus of the embedding foam decreases as ߶ increases. Consequently, two opposing effects interact for particle-laden foams characterized by ߣ ≫ 1: the particles stiffen the embedding foam whose shear modulus is decreased with respect to the corresponding unloaded foam. The global behavior of ܩ * depends on the magnitude of those two effects. Similarly to what has been done for ܩ ௫ *

, one can write:

ܩ * ൫߶ ൯ = ீ൫థ,థ ൯ ீ(థ,) = ீ൫థ,థ ൯ ீ(థᇱ,) ீ(థᇱ,) ீ(,) ீ(,) ீ(థ,) (4)
Therefore, the corresponding global shear modulus is: [START_REF] Siegmann | Mechanical behavior of reinforced polyurethane foams[END_REF]. ݂(߶) ≃ 0.26 -0.24߶ [START_REF] Vaidya | Flexural Properties of Mica Filled Polyurethane Foams[END_REF].

ܩ * ൫߶ ൯ = ݃൫߶ ൯ • ൬ ഝ భష ഝ ൰ (థ)
With functions ݂ and ݃, one can estimate ܩ * ൫߶ ൯. The corresponding values are plotted in Fig. 4 against experimental data, showing good agreement. In particular, Eq.

(5) predicts the global softening of the foamy material, in spite of the presence of solid inclusions.

Note that the two assumptions presented above have been considered earlier by [START_REF] Saint-Michel | Mechanical properties of high density polyurethane foams: II Effect of the filler size[END_REF],

who have predicted the corresponding mechanical behavior using a self-consistent homogenization scheme. Although their particles samples were significantly polydisperse, the results obtained for "big" carbonate fillers [START_REF] Saint-Michel | Mechanical properties of high density polyurethane foams: II Effect of the filler size[END_REF] were captured by the assumption corresponding to ܩ * in our paper, whereas the behavior induced by fillers of smaller size seem to be intermediate between ܩ * and ܩ ௫ *

. In contrast to those results, our data highlight a clear transition between the two asymptotic cases described above. This can be emphasized by setting ߮ and plotting ܩ * as a function of ߣ (inset of Fig. 8). The transition observed for ߣ ≳ 1 can be interpreted in terms of morphological evolution associated to the progressive exclusion of the particles from the foam skeleton towards totally excluded particles for ߣ ≫ 1. The modeling of the mechanical consequences of such a morphological evolution is challenging, and this aspect would certainly deserve a dedicated work. Here, we refer to a previous work [START_REF] Haffner | The drainage of foamy granular suspensions[END_REF] dealing with the drainage of foamy granular suspensions. At first sight, there is no direct comparison between foam drainage and the foam mechanics but, as we will

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 15
show in the following, those two issues involve the same morphological transition.

Indeed, in [START_REF] Haffner | The drainage of foamy granular suspensions[END_REF] the viscous drag (resistance) experienced by the draining liquid was measured to be maximal (ܴ ௫ ) when trapped particles were fully included in the foam network, i.e. ߣ ≈ 1, but minimal (ܴ ) when the particles were fully excluded from the network, i.e. ߣ ≫ 1 (see supplementary information). In introducing a function ߯ that measures the level of viscous drag between those two bounds, i.e.

߯ = (ܴ(ߣ) -ܴ ) (ܴ ௫ -ܴ ) ⁄
, the corresponding drainage data were reasonably described by the phenomenological functional form: ).

߯ = 2.65ߣ ି ସ ⁄ ( 7
In order to show that the transition observed for the shear modulus originates from the same morphological transition, we introduce a similar function:

ܩ ෨ = ሾܩ * (ߣ) -ܩ * ሿ ሾܩ ௫ * -ܩ * ሿ ⁄
, where ܩ ௫ * and ܩ * are respectively calculated with Eqs. (3) and [START_REF] Siegmann | Mechanical behavior of reinforced polyurethane foams[END_REF]. ܩ ෨ and ߯ are plotted in Fig. 8 and show that both the drainage of foamy granular suspensions and the shear modulus of particulate elastic foams undergo the same transition, which is controlled by λ and which accounts for the evolution of the particle configuration in the foam. This highlights the generic nature of the reported transition, which concerns more physical properties of particle-laden foams than mechanics. Moreover, from a practical point of view, drainage and mechanics are intimately linked insofar as before being solid, particle-laden foams are liquid and they undergo the effects of drainage.

M A N U S C R I P T A C C E P T E D

Fig. 8. The rescaled shear modulus of particulate gelatin foam for φp=26% is compared to Eq. 7 obtained from data on the hydrodynamic resistance an aqueous foam loaded with particles for λ > 1. For particulate gelatin foam, λ highlights a transition from a stiffening regime for small particles to a softening regime for large particles in the same λ-range than for the resistance of aqueous foam. Inset: Reduced shear modulus of particulate gelatin foam vs λ for φp=26%.

Conclusion

We have addressed the issue of the strengthening of foamy materials by hard inclusions. The shear elastic modulus of particle-laden polymer foams has been measured as a function of the particle loading, in such a way that the ratio ߣ of particle size to the foam network typical size was controlled. This geometric parameter was found to have a crucial influence on the mechanical behavior. For ߣ < 1, every particle loading leads to a strengthening effect whose magnitude depends only on the particle volume fraction. On the contrary, for ߣ > 1, the strengthening effect weakens abruptly

Fig. 1 .

 1 Fig. 1. Sketch of the experimental procedure to generate solid particulate foams with

2

 2 Plane shear testMechanical properties of our sample are characterized by the shear elastic modulus measured in a plane shear test. To shear the foam sample, one of the slabs of the cell is attached to a motorized translation stage (Newport ILS50 and its motion controller ESP301) while the other slab is maintained at a fixed position thanks to the belowbalance weighing of a precision balance (Sartorius BP211D). This balance measures the vertical force required to maintain the slab motionless. The experimental device enables attaching each slab either on the balance or on the translation stage and then removing the spacers while keeping unmodified the parallelism and the 7.7 mm gap.

Fig. 3 .

 3 Fig. 3. Stress-strain measurements illustrating the strengthening of a foam by adding

Fig. 4 .

 4 Fig. 4. The shear modulus of particle-laden foams normalized by the modulus of the

Fig. 5 .

 5 Fig. 5. Sketch of the foam network showing the nodes and the constrictions; dc is the diameter of passage through constrictions.

  Fig. 6. (a) ݃൫߮ ൯ = ீ൫,ఝ ൯ ீ(,), the reduced shear modulus of a gelatin matrix with

Fig. 7 .

 7 Fig. 7. Close-up on a gelatin foam loaded with 500 µm particles (λ ≈ 10).

  function ݂ has to be determined in order to evaluate ܩ * ൫߶ ൯, and we performed shear tests with particle-free gelatin foams, as shown in Fig.6.b. Within the limited range of gas fraction investigated in the present study, we conveniently approximate the function ݂(߶) by:

  of ߣ, and a softening effect was even observed for ߣ ≳ 10. This transition in the mechanical behavior of foamy composite materials has been interpreted in terms of the evolution for the particles configuration in the foam network. This morphological evolution, the so-called "particle exclusion transition", has been recently highlighted in the framework of the drainage of foamy suspensions and it seems to be also relevant for the mechanics of such foamy systems.M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT

Acknowledgements

We are grateful to G. Noiret and S. Audoly from Rousselot® who kindly supplied the gelatin powder. We thank D. Hautemayou and C. Mézière for technical support. We gratefully acknowledge financial support from Agence Nationale de la Recherche (Grant no. ANR-13-RMNP-0003-01) and French Space Agency (convention CNES/70980).

SUPPLEMENTARY INFORMATION: Particle exclusion transition on drainage kinetics of foamy granular suspension

The hydraulic resistance ܴ (∝ ܸ ିଵ ) of these foamy granular suspensions evolves from its maximal value ܴ ௫ (fully included particles) to its minimal one ܴ (fully excluded particles), we highlighted the so-called "particle exclusion transition" from a function ߯ that measures the level of viscous drag between those two bounds, i.e.

The symbols correspond to values obtained from experimental data. The dotted line corresponds to ߯ = 2.65ߣ ି ସ ⁄ (Eq. 7 used in the manuscript to fit mechanical data on foamed composite materials).


Michel Vacher
email: michel.vacher@imag.fr

Benjamin Lecouteux
email: benjamin.lecouteux@imag.fr

Javier Serrano Romero

Moez Ajili
email: moez.ajili@imag.frfrancois.portet

François Portet

Solange Rossato
email: solange.rossato@imag.fr

Javier Serrano-Romero

Michel Lecouteux

Moez Ajili Portet

François Rossato

Solange}

Speech and Speaker Recognition for Home Automation: Preliminary Results

Keywords: Home Automation, Voice controlled smart home, Vocal command, Speaker recognition { Speaker recognition, Vocal command, Home Automation, Voice controlled smart home}, }

In voice controlled multi-room smart homes ASR and speaker identification systems face distance speech conditions which have a significant impact on performance. Regarding voice command recognition, this paper presents an approach which selects dynamically the best channel and adapts models to the environmental conditions. The method has been tested on data recorded with 11 elderly and visually impaired participants in a real smart home. The voice command recognition error rate was 3.2% in off-line condition and of 13.2% in online condition. For speaker identification, the performances were below very speaker dependant. However, we show a high correlation between performance and training size. The main difficulty was the too short utterance duration in comparison to state of the art studies. Moreover, speaker identification performance depends on the size of the adapting corpus and then users must record enough data before using the system.

I. INTRODUCTION

With the ageing of the population, the issues and challenges created by homecare and the increase loss of autonomy is a major concern over the coming years in European Union (EU). Anticipating and responding to the needs of persons with loss of autonomy with Information and communications technology (ICT) is known as ambient assisted living (AAL). In this domain, the development of Smart Homes is seen as a promising way of achieving in-home daily assistance [START_REF] Chan | A review of smart homes-present state and future challenges[END_REF], [START_REF] De Silva | State of the art of smart homes[END_REF], [START_REF] Peetoom | Literature review on monitoring technologies and their outcomes in independently living elderly people[END_REF] because one of the first wishes of this population is to live in their own home as comfortably and safely even if their autonomy decreases. Efforts are needed for the building industry to put into practice smart home technologies that support independent living [START_REF] Labonnote | Smart home technologies that support independent living: challenges and opportunities for the building industry -a systematic mapping study[END_REF] because some sensors and Ambient Intelligence devices must be set up in the home. Some surveys listed the sensors which are involved in this framework [START_REF] Lê | Smart homes for older people: Positive aging in a digital world[END_REF], [START_REF] Rashidi | A survey on ambient-assisted living tools for older adults[END_REF] and the most recent of them include microphones and speech analysis [START_REF] Ni | The elderly's independant living in smart homes: A characterization of activities and sensing infrastructure survey to facilitate services development[END_REF] but they brought to light the challenges that must be taken up before putting into practice audio analysis in real home [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF]. These challenges include: 1) distant speech and noisy conditions [START_REF] Wölfel | Distant Speech Recognition[END_REF], [START_REF] Vincent | The second CHiME speech separation and recognition challenge: Datasets, tasks and baselines[END_REF], 2) hands-free interaction, 3) affordability by people with limited financial means, 4) real-time interaction, 5) respect of privacy, but it should be noted that the intrusiveness of an assistive technology can be accepted if the benefit is worth it. Indeed, speech technologies are well adapted to people with reduced mobility or some disabilities, and who are not familiar with technical devices and as well as some emergency situations because distant interaction is possible.

Nowadays, voice-user interfaces (VUIs) are frequently employed in close application domains (e.g., smart phone) and are able to provide interaction using natural language so that the user does not have to learn complex computing procedures [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF]. Some industrial products are available, such ase Amazon Echo [START_REF]Amazon Echo[END_REF], but they use a cloud-based processing system that could become a problem related to privacy.

A large number of projects were related to assistive tech-nologies such as CASAS [START_REF] Chen | Behavior-based home energy prediction[END_REF], AGING IN PLACE [START_REF] Skubic | A smart home application to eldercare: Current status and lessons learned[END_REF], DESD-HIS [START_REF] Fleury | SVM-based multi-modal classification of activities of daily living in health smart homes: Sensors, algorithms and first experimental results[END_REF], GER'HOME [START_REF] Zouba | A computer system to monitor older adults at home: Preliminary results[END_REF] and SOPRANO [START_REF] Wolf | Soprano -an extensible, open aal platform for elderly people based on semantical contracts[END_REF]. Moreover, there is a rising number of a smart home projects or studies that considers speech recognition in their design, COMPAN-IONABLE [START_REF] Badii | CompanionAble -integrated cognitive assistive & domotic companion robotic systems for ability & security[END_REF], PERS [START_REF] Hamill | Development of an automated speech recognition interface for personal emergency response systems[END_REF], Filho et al. [START_REF] Filho | From science fiction to science fact: a smarthouse interface using speech technology and a photorealistic avatar[END_REF], SWEET-HOME [START_REF] Lecouteux | Distant Speech Recognition in a Smart Home: Comparison of Several Multisource ASRs in Realistic Conditions[END_REF] and CIRDO [START_REF] Bouakaz | CIRDO: Smart companion for helping elderly to live at home for longer[END_REF]. Some of them is being focused on pathologic voices (i.e., Alzheimer) like ALADIN [START_REF] Gemmeke | Self-taught assistive vocal interfaces: an overview of the aladin project[END_REF], HOME-SERVICE [START_REF] Christensen | homeService: Voice-enabled assistive technology in the home using cloud-based automatic speech recognition[END_REF], and PIPIN [START_REF] König | Validation of an automatic video monitoring system for the detection of instrumental activities of daily living in dementia patients[END_REF]. The aim of DIRHA [START_REF] Cristoforetti | The DIRHA simulated corpus[END_REF] is the study of distant speech recognition at home. However, such technologies must be validated in real situations and SWEET-HOME [START_REF] Vacher | The SWEET-HOME Project: Audio Technology in Smart Homes to improve Well-being and Reliance[END_REF] is, to the best of our knowledge, the first vocal command system which was evaluated online in a real smart home with potential users [START_REF] Vacher | Evaluation of a context-aware voice interface for Ambient Assisted Living: qualitative user study vs. quantitative system evaluation[END_REF].

In this paper, we present an approach to provide voice commands in a multi-room smart home for seniors and people with visual impairments. Our aim is both to recognize, thanks to speech analysis, the home automation command and to identify the speaker. Indeed, before undertaking an action, it is necessary to have a good knowledge regarding the context [START_REF] Chahuara | Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach[END_REF]: the location of the person in the housing, her activity, her identity for permission and preferences. In our approach, we address the problem by using several mono-microphones set in the ceiling, selecting the "best" source and employing a pattern matching approach at the output of the speech recognition system. This approach has been chosen against a noise source separation approach which can be computationally expensive, (i) is sensitive to sample synchronization problem (which cannot be assumed with cheap non professional devices) and (ii) is still not solved in real uncontrolled conditions. Handsfree interaction is ensured by constant keyword detection.

Indeed, the user must be able to control the environment without having to wear a specific device for physical interaction (e.g., a remote control too far from the user when needed). Though microphones in a home are a real breach of privacy, by contrast to current smart-phones, we address the problem by using an in-home ASR engine rather than a cloud based one (private conversations do not go outside the home). Moreover, the limited vocabulary ensures that only speech relevant for the command of the home is correctly decoded. Finally, a strength of the approach is to have been evaluated with real users in realistic conditions.

The paper is organized as follow. Section II presents the method implemented for spoken command and speaker recognition in the home. Section III presents the experimentation and the results which are discussed in Section IV.

II. METHOD

The multi-source voice command recognition is to be performed in the context of a smart home which is equipped with microphones, one or two microphones are set into the ceiling of each room, as shown Figure 2. The audio processing task is both to recognize predefined sentences that correspond either to a home automation command or to a distress call, and to identify the speaker. The audio module should not process other private conversations. Once a command is recognized (e.g., "turn on the light"), it is sent to a intelligent controller which manages the home automation system (e.g., light up the lamp the closest to the person). For more information about the home automation management, the reader is refereed to [START_REF] Chahuara | Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach[END_REF]. The general architecture of the audio processing system is shown in Figure 1. For real-time audio analysis the PATSH framework was developed to manage the acquisition/processing flow of the sound events detected. In PATSH, several processors are plugged in and synchronised, such as the sound and speech analysis modules and the multichannel data acquisition card. The microphone data is continuously acquired and sound events are detected on the fly by using a wavelet decomposition and an adaptive thresholding strategy [START_REF] Vacher | Sound detection and classification through transient models using wavelet coefficient trees[END_REF]. Sound events are then classified as non-speech or speech.

Non-speech sounds are ignored while speech sounds are sent to an ASR system [START_REF] Sehili | Sound environment analysis in smart home[END_REF] and to the Speaker Identification system described in Section II-B. The answers of the ASR and of the Speaker Identification system are then sent to the intelligent controller, which, thanks to the context (the other information available through the home automation system), makes a context aware decision. In the remaining of this section, we describe the ASR system in Section II-A and the speaker recognition system in Section II-B.

We focus on the ASR system and present different strategies to improve the recognition rate of the voice commands in Section II-A. In the second instance, we focus on speaker identification as described in Section II-B.

A. Vocal command recognition

For the reason emphasized in the introduction, the methods do not concentrate on enhancement of the signal but on the decoding of data output from the channel with the best Signal to Noise Ratio ("best channel") and on the use of a priori information at the language level to generate the hypothesis which is consistent with the task. Therefore, the methods employed at the acoustic and decoding level are presented in this part in Sections II-A1, II-A2 and II-A5. The Spoken Keyword Spotting method is presented as baseline in Section II-A4.

1) The acoustic modeling: The Kaldi speech recognition tool-kit [START_REF] Povey | The subspace gaussian mixture model-a structured model for speech recognition[END_REF] was chosen as ASR system. Kaldi is an opensource state-of-the-art ASR system with a high number of tools and a strong support from the community. In the experiments, the models are context-dependent three-state left-right HMMs. Acoustic features are based on mel-frequency cepstral coefficients, 13 MFCC-features coefficients are first extracted and then expanded with delta and double delta features and energy (40 features). Acoustic models are composed of 11,000 context-dependent states and 150,000 Gaussians. The state tying is performed using a decision tree built from phonetical information. In addition, acoustic adaptation is performed thanks to off-line Feature space Maximum Likelihood Linear Regression (fMLLR) linear transformation.

The acoustic models were trained on 500 hours of transcribed French speech composed of the ESTER 1&2 (broadcast news and conversational speech recorder on the radio) and REPERE (TV news and talk-shows) challenges as well as from 7 hours of transcribed French speech from 60 speakers interacting in the Smart home [START_REF] Vacher | The Sweet-Home speech and multimodal corpus for home automation interaction[END_REF], called SH (SWEET-HOME) in the text. This corpus is made of the Multimodal subset (27 speakers, age: 22-63, 7 female&14 male), the Home Automation subset (23 speakers, age: 19-64, 9 female&14 male) and the Interaction subset (16 speakers, age: 19-62, 7 female&9 male).

2) Subspace GMM Acoustic Modelling: The GMM and Subspace GMM (SGMM) both model emission probability of each HMM state with a Gaussian mixture model, but in the SGMM approach, the Gaussian means and the mixture component weights are generated from the phonetic and speaker subspaces along with a set of weight projections. The SGMM model [START_REF] Povey | The subspace gaussian mixture model-a structured model for speech recognition[END_REF] is described in the following equations:

         p(x|j) = Mj m=1 c jm I i=1 w jmi N (x; µ jmi , Σ i ), µ jmi = M i v jm , w jmi = exp w T i vjm I i =1 exp w T i vjm .
where x denotes the feature vector, j ∈ {1..J} is the HMM state, i is the Gaussian index, m is the substate and c jm is the substate weight. Each state j is associated to a vector v jm ∈ R S (S is the phonetic subspace dimension) which derives the means, µ jmi and mixture weights, w jmi and it has a shared number of Gaussians, I. The phonetic subspace M i , weight projections w T i and covariance matrices Σ i i.e; the globally shared parameters Φ i = {M i , w T i , Σ i } are common across all states. These parameters can be shared and estimated over multiple recording conditions.

A generic mixture of I Gaussians, denoted as Universal Background Model (UBM), models all the speech training data for the initialization of the SGMM.

Our experiments involved obtaining SGMM shared parameters using both SWEET-HOME data (SH 7h) and clean data (500h). In the GMM system, the two training data set are just merged in a single one. We propose to train 2 UBMs:

• The first one is a classical SGMM system using all the data to train the UBM (1K gaussians). In the experiments, this SGMM model is named SGMM1.

• For the second one, two UBM are trained respectively on SWEET-HOME data and clean data. The two obtained UBMs contain 1K gaussians and are merged into a single one mixed down to 1K gaussian (closest Gaussians pairs are merged [START_REF] Zouari | Efficient gaussian mixture for speech recognition[END_REF]) , this SGMM is named SGMM2. The aim is to specifically bias the acoustic model with the smart home conditions.

3) The language models: For the decoding, a 2-gram language model (LM) with a 10K lexicon was used. It results from the interpolation of a generic LM and a specialized LM. The generic LM was estimated on about 1000M of words from the French newspapers Le Monde and Gigaword, and the broadcast news manual transcripts provided during the ESTER campaign. The specialized LM was estimated from the grammar and from the transcript of the 60 speakers, containing voice commands and casual speech.

4) Spoken Keyword Spotting: The problem of recognizing voice commands with a predefined grammar but not other conversation can be seen as a spoken keyword spotting problem. Given the uncertainty of the ASR system output, spoken keyword spotting has mainly been addressed by searching instances of particular keywords in the ASR set of hypotheses or lattice obtained after processing an utterance [START_REF] Szoke | Comparison of keyword spotting approaches for informal continuous speech[END_REF]. In this work, we use the method of Can and Saraçlar [START_REF] Can | Lattice indexing for spoken term detection[END_REF] for Spoken Term Detection (STD) from the ASR decoding lattice of an utterance. In this approach, the lattice is transformed into a deterministic weighted Finite State Transducer (FST) called Timed Factor Transducer (TFT) embedding informations for the detection (utterance ID, start and end time and posterior score). A search of a string X in a speech utterance is then a composition of the automaton representation of X and the TFT to give the resulting transducer R which contains all the possible successful detections and their posterior probability. The interest of such approach is the fact that search complexity is linear and that performing several searches in a same utterance can be done with the same TFT. Moreover, the FST formalism makes filtering with a predefined grammar easy by using the composition operator.

5) Detection of voice commands:

We propose to transcribe each voice command and ASR output into a phoneme graph in which each path corresponds to a variant of pronunciation. For each phonetized ASR output T , every voice commands H is aligned to T using Levenshtein distance. The deletion, insertion and substitution costs were computed empirically while the cumulative distance γ(i, j) between H j and T i is given by Equation 1.

γ(i, j) = d(T i , H j )+ min{γ(i -1, j -1), γ(i -1, j), γ(i, j -1)} (1)
The voice command with the aligned symbols score is then selected for decision according a detection threshold. This approach takes into account some recognition errors like word endings or light variations. Moreover, in a lot of cases, a missdecoded word is phonetically close to the true one (due to the close pronunciation). From this the DER (Domotic Error Rate i.e., home automation error rate) is defined as:

DER = Missed + False Alarms Voice Commands syntactically correct (2)
For the DER, the ground truth is the number of uttered voice commands respecting the grammar. I.e., the utterances where the person's intention was to utter a vocal command but was not following the voice command syntax were not considered as a true voice commands. The Missed correspond to the true voice commands not recognized and the False Alarms to sound events incorrectly classified as voice commands.

B. Speaker recognition

As emphasized in the introduction, speaker identification gives complementary information for context inference (location, activity, identity of speaker who is uttering a command). Figure 3 presents the speaker identification method; it is divided into two phases:

• Training phase in which a large data set is used to build an UBM that models the whole acoustic space. Then, speaker models are obtained by UBM adaptation. • Testing phase in which a speech occurrence will be assigned to one of the existing speakers. The used UBM is modeled by GMMs. 1) UBM: UBM is trained by Expectation-Maximisation (EM) algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF] which is divided in two steps:

• Expectation evaluation (E), where the likelihood expectation is calculated using the last observed values. • Maximisation step (M), where the parameters maximum likelihood are estimated by maximizing the likelihood calculated in the E step. The EM algorithm is iterative. Parameters estimated in the Maximisation (M) step are used as a check point to start a new Expectation (E) evaluation, by applying an iteration. Between two successive iterations, the algorithm guarantees the growth of a likelihood objective function parameters given the observed data "frames" x. We should notice that EM converges to a local maximum.

2) Adaptation to the speaker: We derive the speaker model by adapting the parameters of the UBM using the speaker's training speech and a form of Bayesian adaptation [START_REF] Reynolds | Speaker verification using adapted gaussian mixture models[END_REF]. Unlike the standard approach of maximum likelihood training of a model for the speaker independently of the UBM, the basic idea in the adaptation approach is to derive the speaker's model by updating the well-trained parameters in the UBM via Maximum A Posteriori adaptation (MAP). Like the EM algorithm, the adaption is a two step estimation process. The first step is identical to the expectation step of the EM algorithm, where estimates of the sufficient statistics of the speaker's training data are computed for each mixture in the UBM. Unlike the second step of the EM algorithm, for adaptation these new sufficient statistic estimates are then combined with the old sufficient statistics from the UBM mixture parameters using a data-dependent mixing coefficient. Typically, we only adapt the means of the Gaussian, keeping the variance unchanged.

3) Scoring: The system proposed by Jousse et al. [START_REF] Jousse | Automatic named identification of speakers using diarization and ASR systems[END_REF] is based on Semantic Classification Trees (SCT). For each utterance, a score is calculated indicating the degree of support to every speaker. Then, the speaker who has the maximum score is identified. A detailed description of this system is presented in [START_REF] Mauclair | Speaker diarization: About whom the speaker is talking ?[END_REF] and [START_REF] Estève | Extracting true speaker identities from transcriptions[END_REF].

III. EXPERIMENTATION AND RESULTS

A. Vocal Command Recognition: Live Experiment

An experiment involving elderly and visually impaired people was run in the DOMUS smart home which is part of the experimentation platform of the LIG laboratory. This experiment is fully described in [START_REF] Vacher | The Sweet-Home speech and multimodal corpus for home automation interaction[END_REF] and only the aspects essential for the comprehension of the reader are recalled in this article. This smart home is a four-room flat (see Figure 1) equipped with home automation system and with 7 microphones set in the ceiling for audio analysis. A communication device was also present to allow video-conferencing. The SWEET-HOME system consisted in the PATSH software presented in Section II which was continuously analysing the audio streams to detect voice commands [START_REF] Vacher | Experimental Evaluation of Speech Recognition Technologies for Voice-based Home Automation Control in a Smart Home[END_REF] and an intelligent controller [START_REF] Chahuara | Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach[END_REF] in charge of executing the correct action (e.g., lighting the lamp, or giving the temperature using TTS) based on these voice commands.

Each participant had to follow 4 successive scenarios with the following topics: 1) 'finishing breakfast and going out', 2) 'coming back from shopping and cleaning', 3) 'communicating with a relative', and 4) 'waiting for friends'. Each of these scenarios was designed to last between 5 to 10 minutes but there was no constraint on the execution time. Scenario 1 and 2 were designed to have the user performing daily activities while uttering voice commands. The participant was provided with a list of actions to perform and voice commands to utter. Each participant had to use vocal commands to turn the light on or off, open or close blinds, ask about temperature and ask to call his or her relative.

Six seniors and five people with visual impairments were recruited. The seniors (81.2 years old (Standard Deviation: SD=5.8)) were women living alone in an independent nonhospitalised accommodation. The focus of study was to target seniors who where on the edge of losing some autonomy and not seniors who have lost complete autonomy. In other words, we sought seniors who were still able to make a choice regarding how the technology could help them in case of any physical degradation. The visually impaired category (62.2 (SD=6.9) years old, 3 were women) was composed of adult people living either single or as couple and whose handicap was acquired after their childhood. No upper age limit was given. The participants were not completely blind but their visual acuity was very low.

1) Spoken commands: Possible vocal commands were defined using a very simple grammar which was built after a study revealing that targeted users prefer precise short sentences over more natural long sentences [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF], this was corroborated in [START_REF] Vacher | Evaluation of a context-aware voice interface for Ambient Assisted Living: qualitative user study vs. quantitative system evaluation[END_REF]. As shown below, each vocal command is either an emergency call or a command intended to control a device. Every command starts with an optional key-word (e.g. 'Nestor') to make clear whether the person is talking to the smart home or not. Some basic commands are then 'turn on the light' or 'what is the temperature': 2) Acquired data: At the beginning of the experiment, the participants were asked to read a short text (288 words in French), this text was used to adapt the acoustical models to the speaker before performing the scenarios.

During the experiment, audio data was recorded in two ways. Firstly, the 7-channel raw audio stream was stored for each participant for further analysis. Secondly, audio events were automatically extracted by the PATSH software on the fly. Some of the events were missed or discarded and some of the detected speech events were misclassified as everyday life sound, and some noise were misclassified as speech (bell ring, music, motor). In the later case, these non-speech events were sent to the ASR. A manual segmentation of the corpus was done thanks to the full records of the experiments using the Transcriber software [START_REF] Barras | Transcriber: development and use of a tool for assisting speech corpora production[END_REF]. Finally, these two speech data sets (manual vs. PATSH segmentation) were transcribed using Transcriber . For the PATSH data set, there are 617 uttered sentences. 291 were home automation commands (46%), 66 (10%) were actually generated by the speech synthesizer, 10 (2%) were noise occurrences wrongly classified as speech and 250 were other spontaneous speech (42%, mostly during the video-conferencing with a relative). Only 29 speech utterances were missed (4%), but 85 of the detected ones were rejected (14%) either because their SNR was below 0dB or because they were out of the acceptable duration range (2.2 seconds). Therefore, 18% of the utterances were not treated by the system. The recorded audio data are summarized in Table I.

B. Spoken command recognition: Off line experiments

The methods presented in Section II were run on the data set presented in Table I. Regarding S11, PATSH crashed in the middle of the experiment and due to time constraints S11 data was not considered in the study. Therefore 550 sentences (2559 words) including 250 commands, questions and distress calls (937 words) were used. Two acoustic models were used: AM (500h) and AM (500h+ SH, were SH = SWEET-HOME data), speaker adaptation was provided by fMLLR using the text read by each speaker. The two versions of the Subspace Gaussian Mixture Models (SGMM1 and 2 cf. Section II-A2) were also applied to all different combinations. All the methods were run on the transcribed data (manually annotated) and on the PATSH data (automatically segmented). We present the keyword spotting approach in order to show that a conventional approach is limited because of language variations introduced by the protagonists (i.e. home automation commands are rarely pronounced correctly).

Results on manually annotated data are given Table III. The most important performance measures are the Word Error Rate (WER) of the overall decoded speech and those of the specific voice commands as well as the Domotic Error Rate (DER: c.f. Equation 2).

It can be noticed that most of the improvement is brought by the use of fMLLR and the use of data adapted to the acoustic environment (the SH dataset). The WER obtained from the overall speech goes from 59.8 to 35.7. But most of this reduction is driven by the dramatic decrease of error in the voice command decoding. It starts from 32.9% for the baseline, and is reduced to 22.8% with the use of an acoustic model adapted to the smart home and to 14.0% thanks to speaker adaptation. Best results, WER=10.1%, DER=3.2%, are obtained by using SGMM applied to the utterance with the highest SNR (best channel).

Regarding the data set extracted by PATSH, the original ASR performance with a decoding online in real time during the experiment in the smart home and on only one channel [START_REF] Vacher | Experimental Evaluation of Speech Recognition Technologies for Voice-based Home Automation Control in a Smart Home[END_REF] was WER=43.2%, DER=41%. By using these same data, the automatically segmented corpus, and AM (500h+SH), SAT fMLLR, SGMM2 on the best channel or with a specific analysis system operating on 2 channels, we obtained the results presented in Table III. Results are somewhat lower than those obtained with the manually segmented data. The most important contribution to the DER is due to missed speech utterances at the detection or speech/sound discrimination level. Therefore this is a very significant improvement from the experimental condition.

C. Speaker Recognition

Acoustic features were of MFCC parameters, their derivatives, and 11 second order derivatives (the frequency window was restricted to 300-3400 Hz). A normalization file based process was applied, so that the distribution of each cepstral coefficient was 0-mean and 1-variance for a given utterance.

For modeling the GMM-UBM, ALIZE toolkit [START_REF] Larcher | Alize 3.0 -open source toolkit for state-of-the-art speaker recognition[END_REF] was used. UBM is trained using 7,803 sessions from 24 speakers (using the speech corpus "Home Automation subset" of the Sweet-Home corpus [START_REF] Vacher | The Sweet-Home speech and multimodal corpus for home automation interaction[END_REF] recorded by 24 speakers) trained by EM/ML algorithm (with a variance flooring ≈ 0 ) on about 2 millions of speech frames. It resulted in 64 Gaussian components.

For evaluation of the system, the "Interaction subset" of the Sweet-Home corpus [START_REF] Vacher | The Sweet-Home speech and multimodal corpus for home automation interaction[END_REF] was used. This corpus is made of two separated parts and was recorded by the 11 specific speakers (5 visually impaired and 6 elderly people). Before participating to the experiment, each speaker read 21 vocal commands (i.e., 99 words) and a newspaper article (205 words) in the Domus smart home. These data are dedicated to speaker adaptation and they compose the training part of the corpus. Next, the same persons play the scenarios interacting with the Vocal Home Automation system in order to record the second part of the corpus. This part of the corpus is the test data set.

For MAP adaptation to each user, we used the training set and MAP algorithm that is implemented in the LIUM toolkit [START_REF] Jousse | Automatic named identification of speakers using diarization and ASR systems[END_REF]. The duration of speech used to adapt speakers models was between 70 and 268s.

The test dataset is detailed detailed in Table IV. The speaker must be identified from a single sentence which represent a very short time, the average duration is 2.55s. Therefore, the speaker have to be identified among 11 possible speakers.

The performances are evaluated for each speaker in terms of Speaker Recognition Rate (SRR): SRR = Number of well identified utterances Total number of utterances

The recognition rate is given for each speaker Table V. The average recognition rate is 70% for all speaker, 75.6% for the visually impaired group and 62.6% for the elderly group.

Figure 4 gives a representation of the performance for elderly and visually impaired speakers as a function of the size of the adapting corpus for the corresponding speaker. A curve fit is plotted for these two kinds of people, the 2 curves show a great difference between these 2 groups. In most cases and for elderly speakers, the performance is lower with a same size of training data as for visually impaired people. Therefore for this kind of population and before the installation in the home of the individual, it will be necessary in a first phase to record speech in unconstrained conditions.

The confusion matrix in table VI shows a good discrimination in gender, only 3 sentences out of 229 are not assigned to the right gender.

IV. DISCUSSION

Efficient on-line recognition of voice commands is mandatory for the dissemination of in-home VUI. This task must address many challenges such as distant speech recognition and respect for privacy. Moreover, such technology must be evaluated in realistic conditions. In this paper, we showed that a careful selection of the best channel as well as good adaptation to the environmental and acoustic characteristics increase dramatically the voice command classification performance. In the manual segmentation, SGMM acoustic model learned from data previously acquired in the smart home as well as fMLLR diminish the DER from 20.8% to 3.2% surpassing more standard methods such as keyword spotting. In the recognition task based on the PATSH detection and discrimination, the best technique (2-channel SGMM, fMLLR) shows a rise of DER to 13.2%. This can be explained by the imperfect detection, segmentation and classification of the system. Indeed, some sentences were missing or split in two parts (e.g., 'Nestor light the lamp' → 'Nestor' then 'light the lamp'). Hesitations in real speech are natural but it is still unclear whether they are going to be frequent in real use or due to the experimental condition (people must learn the grammar).

Regarding the speaker identification, the presented results are not as good as those of the state of arts. Indeed, State of the Arts systems are configured for larger training data set

Speaker Genre

Type SRR and are evaluated on speech segments whose duration is about 30s while a home automation command lasts 2.55 seconds. For example, the STC Speaker Recognition System for the NIST i-Vector Challenge presented by Novosellov et al. [START_REF] Novoselov | STC Speaker Recognition System for the NIST i-Vector Challenge[END_REF] used segment durations with a mean of 39.58 seconds. Our system was configured in the same way. Nevertheless, our system shows good performances for male, evaluated to 93.3%. Our further research will be done to improve system precision by: (i) optimizing the training conditions, and (ii) using different process for male and female.

It can be noticed that Speakers 2, 4, 6 and 9 present the lowest SRR. Two explanations to this result could be proposed. Firstly, it is reasonable to think that speaker verification is very dependant to the enrollment and/or test conditions (utterance length, quality, channel mismatch, etc.). Secondly, this result could be linked directly to the "speaker factor". This issue has been discussed deeply in [START_REF] Kahn | Intra-speaker variability effects of Speaker Verification performance[END_REF] and [START_REF] Doddington | Sheep, goats, LAMBS and WOLVES: a statistical analysis of speaker performance in the NIST 1998 speaker recognition evaluation[END_REF].

V. CONCLUSION

Studies presented in this paper are a part of a more general research project aiming at developing a system for home automation vocal commands which could be implemented in the personal home of elderly or frail people living alone. There are still important challenges to overcome before implementing this solution, particularly speech analysis in distant speech conditions, automatic speech recognition of elderly people or of expressive speech and more generally context aware interaction. Moreover, this new technology must be validated in real conditions with potential users, therefore our evaluations were conducted using a corpus recorded in a smarthome by potential users [START_REF] Vacher | Evaluation of a context-aware voice interface for Ambient Assisted Living: qualitative user study vs. quantitative system evaluation[END_REF]. This paper presents and evaluates a system for vocal command recognition and a system for speaker identification aiming at delivering this information to an intelligent controller in charge of home automation driving.

Regarding spoken command recognition, performances are sufficient but there are still improvements to be performed before any use in real conditions. Indeed, in the experiment, people regularly deviated from the grammar (e.g., adding politeness terms or reformulation) and did not like the predefined chosen keyword. An interesting research direction would be to adapt the language model to the words a user 'naturally' utters in different situations, hence learning the grammar from the data rather than imposing a grammar. Another direction would be to exploit the smart-home capacity of sensing the environment to provide context-aware ASR. However, speaker recognition performances do not allow to identify the speaker who uttered a vocal command, especially if he is an elderly person, with a precision above 40%. This identification is an important information which can allow the home automation system to undertake the appropriate action and a future work will be related to determine the way of improvement of these first results. One possibility is to use information available from other sensors. For example, if it is established that there is only one person in the house and if a speaker was beforehand recognized, in that case the system could know that there is an error if another speaker is recognized. Thus the last data could be useful to serve as additional adaptation data.

Fig. 2 .

 2 Fig. 2. Position of the microphones in the smart home.

Fig. 1 .

 1 Fig. 1. General architecture of the audio processing system.

Fig. 3 .

 3 Fig. 3. General architecture of the speaker identification system.

Fig. 4 .

 4 Fig. 4. Speaker recognition rate as a function of the size of the adaptation corpus for the two speaker groups.

TABLE I SPEECH

 I AUDIO DATA RECORDED DURING THE SCENARIOS (FOR EACH SPEAKER, A READ TEXT WAS RECORDED FOR SPEAKER ADAPTION)

	ID	Category Age Sex Scenario	Speech Voice commands	SNR
					duration	utterances uttered missed	(dB)
	S01 Aged	91	F	24mn	59	37	30	16
	S02 Visually	66	F	17mn 49s	67	26	5	14
	S03 Visually	49	M	21mn 55s	53	26	9	20
	S04 Aged	82	F	29mn 46s	74	27	12	13
	S05 Visually	66	M	30mn 37s	47	25	10	19
	S06 Aged	83	F	22mn 41s	65	31	20	25
	S07 Aged	74	F	35mn 39s	55	25	13	14
	S08 Visually	64	F	18mn 20s	35	22	8	21
	S09 Aged	77	F	23mn 5s	46	23	17	17
	S10 Visually	64	M	24mn 48s	49	20	7	18
	S11 Aged	80	F	30mn 19s	79	26	18	23
	All	-	-	-	4h 39mn	629	291	149	-

TABLE IV AVERAGE

 IV DURATION OF A VOCAL COMMAND FOR THE HOME INTERACTION SUBSET.

ACKNOWLEDGEMENT

This study was funded by the National Agency for Research under the project SWEET-HOME (ANR-2009-VERS-011).