Mathieu Rouault

Pierrick Penven

Benjamin Pohl

On the recent warming of the Agulhas Current

The Agulhas Current is an energetic current driven by the wind field over the Indian Ocean. It has a profound effect on the climate and the coastal ecosystem of South Africa and plays a key role in the global ocean circulation. The current carries warm and salty water from the tropics polewards and controls the exchange of heat and salt between the Indian and Atlantic Oceans. Since the 1980s, the sea surface temperature of the Agulhas Current system has increased significantly. This is due to an increase of its transport in response to an augmentation in wind stress curl in the South Indian Ocean. This causes an intensification of the Agulhas Current system and leads to an increased flux of salt and heat into the Atlantic Ocean. There is also an augmentation in the transfer of energy from the Agulhas Current to the atmosphere due to increased evaporation. These observed changes could have far-reaching consequences over and above their potential regional impacts on ecosystems and climate.

Introduction

The Agulhas Current ( Changes in mean sea surface temperatures [8] have shown that this warming was due to an intensification of the Agulhas Current system in response to an increase in trade wind and a poleward shift in the westerly wind in the South Indian Ocean leading to an overall increase in wind stress curl at relevant latitude. A numerical model that reproduces the observed SST relatively well showed that the transport of the Agulhas Current system had increased since the 1980s leading to the observed warming. This also led to substantial increase in evaporation rate of up to 1 mm per day per decade in the Agulhas Current system and a 50% increase in the leakage of Agulhas water into the South Atlantic. A cooling of up to 0,5 °C per decade occurs at the west coast. Another cooling occurs in the dynamic upwelling cell of Port Alfred and Port Elizabeth where it seems to spread into the Agulhas Current itself and to the west. Cold water seems to have propagated offshore and eastward from the Port Alfred dynamic upwelling cell. Upwelling favorable wind could have contributed to the cooling but an intensification of the Agulhas Current and concurrent intensifying of the dynamic upwelling could be the principal reason of the cooling in that region. The cooling in the west of the country, from Cape Agulhas to the Namibian border, is due to an increased southerly wind. The west coast cooling occurs mostly in autumn and winter [8]. The greatest warming is evident in the Agulhas Current system and the Transkei at all months of the year. The origin of the cooling trends for the west and south coasts is found in an examination of the linear trend in European Center for Medium Range Weather Forecasts and National Centers for Environmental Prediction reanalysed surface wind speed in the region from 1982 to 2007 [8]. It shows that surface wind speed increased in the Southern Atlantic and Indian subtropics. This increased the wind stress curl in the South Indian Ocean and it is at the origin of the intensification of the Agulhas Current system [8]. The observed change in sea level pressure and wind speed is consistent with a poleward shift of the westerly wind in the Southern Hemisphere and an increase of the South Atlantic and South Indian Ocean high pressure systems due to intensification of the Hadley circulation and a trend towards a positive phase of the Antarctic Oscillation.

Conclusion

In conclusion, it seems that the most important changes to the climate and coastal ecosystem in the region is an intensification of the Agulhas Current system since the 1980s. This caused a warming of the Agulhas Current system and a cooling of the Port Alfred upwelling cell. The west and south coast presents a cooling pattern from April to August. All those changes seem to have been triggered by an intensification of the high pressure system in the South Atlantic and South Indian Ocean and a poleward shift of the westerly system.

  Figure 4.90) flows along the east coast of South Africa before moving offshore near latitude 34°S and subsequently retroflecting back into the mid-latitude South West Indian Ocean. It creates a coastal dynamic upwelling in the vicinity of Port Alfred and Port Elizabeth bringing nutrient rich water to the surface [1, 2]. High evaporation rates and associated turbulent latent and sensible heat fluxes occur above the Agulhas Current throughout the year due to an important sea surface temperature contrast between the Agulhas Current and its surroundings. Measurements in the Agulhas Current have shown substantial transfers of water vapour in the marine boundary layer, a deepening of the marine boundary layer due to intense mixing, and unstable atmospheric stability created by the advection of colder and drier air above the current [1, 2]. The intensity of mixing in the local boundary layer is such that cloud lines can often be observed above the current [3]. Rouault et al. [4] have provided evidence of the influence of the Agulhas Current on the evolution of a severe convective storm over southern South Africa. That particular storm, in December 1998, led to severe flooding and a tornado in Umtata that nearly killed President Nelson Mandela when the winds in the town caused a building to collapse. On the global scale, Agulhas water leakage around South Africa controls the exchange of heat and salt between the Indian and Atlantic Oceans and has a role in the Atlantic meridional overturning circulation [5].

Figure 4 .

 4 Figure 4.90 A composite satellite image of sea surface temperatures near South Africa for 14-19 December 1998. the information was gathered by the AVHRR on board the NOAA-14 satellite. Colours show the sea surface temperatures and clouds appear white. The core of the Agulhas current has temperature of 21 to 25°C. [amS]

Figure 4 .

 4 Figure 4.91, an image obtained using 4 × 4 km resolution Advanced Very High Resolution Radar Sea Surface Temperature imagery (AVHRR SST) [6], shows the 1985-2007 mean sea surface temperature around South Africa.The mean absolute geostrophic ocean current vector derived from merged altimetry[7] is superimposed on the image, which shows the major elements of the Agulhas Current system. The main loop is found south of the continent. The Retroflection is located in the domain delimited by 10°E to 20°E and 37°S to 42°S. Eddies shed from the Agulhas Current can be found as far as latitude 50°S but most of them are usually formed in the Retroflection and move northwestwards towards Brazil. The Agulhas Return Current flows eastwards and meanders from 37°S to 42°S. A coastal upwelling of cold water is evident from Cape Agulhas to Namibia and is the result of strong seasonal southerly wind. This is in contrast to the Port Alfred upwelling cell that is triggered by the Agulhas Current itself. Sporadic wind-driven upwelling also occurs to the east of Cape Agulhas.

Figure 4 .

 4 Figure 4.92 (also obtained using 4 × 4 km resolution AVHRR SST) shows the linear trend in sea surface temperature from 1985 to 2007. The most important change is found in the Agulhas Current system, which has warmed by up to 1,5 °C since the 1980s. Rouault et al.[8] have shown that this warming was due to an intensification of the Agulhas Current system in response to an increase in trade wind and a poleward shift in the westerly wind in the South Indian Ocean leading to an overall increase in wind stress curl at relevant latitude. A numerical model that reproduces the observed SST relatively well showed that the transport of the Agulhas Current system had increased since the 1980s leading to the observed warming. This also led to substantial increase in evaporation rate of up to 1 mm per day per decade in the Agulhas Current system and a 50% increase in the leakage of Agulhas water into the South Atlantic. A cooling of up to 0,5 °C per decade occurs at the west coast. Another cooling occurs in the dynamic upwelling cell of Port Alfred and Port Elizabeth where it seems to spread into the Agulhas Current itself and to the west. Cold water seems to have propagated offshore and eastward from the Port Alfred dynamic upwelling cell. Upwelling favorable wind could have contributed to the cooling but an intensification of the Agulhas Current and concurrent intensifying of the dynamic upwelling could be the principal reason of the cooling in that region. The cooling in the west of the country, from Cape Agulhas to the Namibian border, is due to an increased southerly wind. The west coast cooling occurs mostly in autumn and winter[8]. The greatest warming is evident in the Agulhas Current system and the Transkei at all months of the year.

Figure 4 .

 4 Figure 4.91 Mean 1985-2007 AVHHR SST. Mean 1993-2007 absolute geostrophic velocity vectors derived from combined altimeter is superimposed

Acknowledgement

  WRC, NRF, PICS, CNRS, IRD, AVISO, NOAA.

Figure 4 .

 4 Figure 4.92 Linear trend of AVHRR SST from 1985 to 2007. Mean 1993-2007 absolute geostrophic velocity vectors derived from combined altimeter is superimposed.


Franck Renaud
email: franck.renaud@supmeca.fr

Jean-Luc Dion
email: jean-luc.dion@supmeca.fr

Gaël Chevallier
email: gael.chevallier@supmeca.fr

Jean-Yves Choley
email: jean-yves.choley@supmeca.fr

Shear stiffness measurement against frequency of viscoelastic materials

In many mechanical systems polymers are used to damp vibrations. Shims are used in brake systems to damp squeal noises occuring above 1000Hz. Shims are viscoelastic materials composed of layers of steel, rubber and glue. Although the mechanical behavior of polymers are usually characterized by DMA (Dynamic Mechanical Analysis) thanks to the time-temperature superposition principle, this principle does not hold for shims. An analyser has been designed to characterize these materials up to 3kHz without using the time-temperature superposition principle. The characterization of samples is performed under a static preload.

Introduction

In many mechanical systems viscoelastic materials are used to damp vibrations. In disc brake systems, the squeal noise is due to an unstable vibration occuring between 1kHz and 10kHz. For instance, shims are viscoelastic multi-layer materials used to get rid of squeal and others noises. Shims are composed of layers of steel, rubber and glue. The stiffness of such material is strongly frequency dependent.

Squeal noise occurences of a brake system are usually predicted by a complex eigen value analysis (CEA) on a finite element (FE) model, see Chevallier [START_REF] Chevallier | Viscoelastic damping effect on brake squeal noise[END_REF], Fritz [START_REF] Fritz | Investigation of the relationship between damping and mode-coupling patterns in case of brake squeal[END_REF][START_REF] Fritz | Effects of damping on brake squeal coalescence patterns -application on a finite element model[END_REF]. Since viscoelasticity can lead to some counter intuitive destabilization effects, see Chevallier [START_REF] Chevallier | Viscoelastic damping effect on brake squeal noise[END_REF], Fritz [START_REF] Fritz | Investigation of the relationship between damping and mode-coupling patterns in case of brake squeal[END_REF][START_REF] Fritz | Effects of damping on brake squeal coalescence patterns -application on a finite element model[END_REF], disc brake FE models must account precisely for the viscoelatic behavior of materials such as shims. The Generalized Maxwell model describes well the viscoelastic stiffness of materials behaving between purely elastic and purely viscous, see Chevalier [START_REF] Chevalier | Mechanics of Viscoelastic Materials and Wave Dispersion[END_REF], ... . This viscoelastic model can be easily implemented in FE disc brake model, see Chevallier [START_REF] Chevallier | Viscoelastic damping effect on brake squeal noise[END_REF]. Some methods exist to identify the parameters of the Generalized Maxwell model based on the stiffness measurements, see Dion [START_REF] Jl Dion | Identification or rubber shock absorber mounts[END_REF], Renaud [START_REF] Renaud | A new identification method of viscoelastic behavior: Application to the generalized maxwell model[END_REF]. Thus, the knowledge of shims stiffness is required from 100Hz to 10000Hz to get a predictive disc brake model.

Since shims are composed of layers of different materials such as steel and glue, the time-temperature superposition principle does not hold. Thus a direct characterization is required. Several techniques already exist. A classical one is the Oberst's beam method, see [START_REF] Oberst | Damping of the bending vibrations of thin laminated metal beams connected through adherent layer[END_REF][START_REF] Barbosa | A finite element model for sandwich viscoelastic beams: Experimental and numerical assessment[END_REF][START_REF] Castello | Constitutive parameter estimation of a viscoelastic model with internal variables[END_REF]. It consists in comparing the eigen frequency of the normal mode of the same structure with and without the viscoelastic material. Unfortunately this kind of method allows characterization only at very few frequencies, the ones of normal modes. Another technique is to measure the relaxation function of a material, see Chen [START_REF] Chen | Determining a prony series for a viscoelastic material from time varying strain data[END_REF]. However, as it is difficult to assess a true strain unit step, this technique is only accurate at low frequencies below the frequency bandwidth of squeal noise. The last option is to perform a direct characterization with a DMA analyser without using time-temperature superposition. This requires to characterize materials below the frequency of the first normal mode of the analyser which interfer with the measurement by involving samples deformation.

A new analyser has been designed to mimic the stresses of shims during a squeal noise occurence, cf. Figure 1. This means a static normal preload representative of the braking pressure and a dynamic shear stress representative of the unstable vibration at the root cause of the squeal. The analyser allows characterizing materials between 100 and 3000Hz. This article presents the design of this shear DMA tester, the postprocessing of the data and the measured shear stiffnesses of some shims.

Design of the analyser

The operating principle of the shear DMA analyser is shown in Figure 2. It is almost symmetrical and requires four samples for one characterization. Thereby the measured complex shear stiffness is the mean value among four samples. The analyser (cf. Figure 1) is hung by rubber bands for any vibration of environment could not interfer with the measurement. Six piezoelectric accelerometers measure the tangential acceleration of the top face and the bottom face of each sample. Then, the shear deformation of each sample is measured by integrating the acceleration measurements.

Samples are rectangles of 20mm by 30mm with variable thickness depending on the shim. Each sample is pressed between one support and one jaw. This pressure is applied by a M6 screw and is measured with a static force sensor (cf. Figure 1). The maximum preload is 5000N and corresponds to a pressure on samples of 4MPa. Recall that eventually no DMA tester allow direct characterization above 1000Hz with a static preload ( ??????????? ). When characterizing samples, the dynamic excitation is obtained by moving the supports apart with piezoelectric actuators. In contrary to the Oberst beam test, this shear DMA tester allows pure shear deformation of samples without bending. Then, the stiffness is measured directly and no Finite Element model is required for comparison (?????????????? ). The forces generated by the two actuators are measured with two piezoelectric force sensors. The actuation chain alone is shown in Figure 5. The upper limit frequency of characterization is given by the frequency of the first normal mode of the analyser which interfer with the measurement. Actually, interferences occur when a normal mode involves samples deformation or sensors motion. The higher this eigen frequency occur, the larger is the frequency bandwidth of characterization. The first interfering eigen mode involves the deformation of the actuation chain. Figure 6 shows its mode shape. This eigen mode generally occurs about 3500Hz and limits the characterization to a frequency about 3000Hz.

To characterize shims, samples are first positioned manually between jaws and supports. When possible shims are stuck on jaws. Then, the preload is applied thanks to the screw. Second, a stepped sinus excitation is performed with a rate of 12 frequencies per octave. The excitation time of samples has to last as short as possible in order not to warm the material. For each single frequency, the level of excitation is controlled with the sample's strain. Finally the data acquired during the test are postprocessed in the Matlab environment.

Data postprocessing

To postprocess the data obtained during shear stiffness characterization of the samples, the effects of jaws deformations and the inertia efforts must be corrected. The shear DMA analyser is designed to work at frequencies lower than its first normal mode. Thus deformable parts are modelled with springs. Above a few hundred Hertz, even small displacements of lightweight parts can lead to inertia efforts comparable to the shear efforts. For example, a sinusoidally moving mass of 100g with a displacement amplitude of 1µm at 1000Hz leads to an inertia effort of |F | = |M Γ| = M (2πf ) 2 X = 10 -1 × (2π × 1000) 2 × 10 -6 4N . At 5000Hz, the same mass leads to an inertia effort around 100N. Thus attention must be paid to masses modelling, especially for the actuation chain shown in Figure 5. The actuation chain is composed of two supports, both with one accelerometer, two dynamic force sensors, two links and two piezoelectric actuators. The force sensors and the actuators are both composed of a piezoelectric part linking two masses. Thus, the force sensors measure a force between two masses and the actuators generate a force between two masses. Thereby, the actuation chain is modelled by three masses: M S1 , the left support, M R the link and M S2 the right support. These three masses are separated by the piezoelectrical part of the sensors and the actuators. The actuation chain modelling is shown in Figure 7. Let ω = 2πf denote the angular frequency, let Γ = -ω 2 X denote the Fourier transform of the acceleration and let F denote the Fourier transform of the force. Then, the fundamental principle of the dynamics give Equation (1):

     M S1 Γ 2 = F FS M R Γ 2 = -F FS + F piezo M S2 Γ 5 = F piezo (1)
The total weight of the actuation chain is:

M total = M S1 + M R + M S2 = 290g (2)
From Equation ( 1) and ( 2), one obtains the masses equations (3) against frequency. A test has been led between 1000 and 2000Hz on the actuation chain alone as shown in Figure 5. The values are

               M S1 (f ) = F FS Γ 2 120g M R (f ) = M total Γ 5 Γ 2 + Γ 5 -M S1 37g M S2 (f ) = M total Γ 2 Γ 2 + Γ 5 133g
(3)

Figure 9: Rheological modelling of the shear DMA analyser

The rheological model of the shear stiffness shear DMA tester is shown in Figure 9. Although the jaws are modelled by two identical masses, M P , linked by a spring K jaw , these parameters have no influence in the following postprocess. From the analyser model shown in Figure 9, the fundamental principle of the dynamics give Equation ( 4):

M S1 Γ 2 = -K S (2X 2 -X 1 -X 3 ) + f F S M S2 Γ 5 = -K S (2X 5 -X 4 -X 6 ) + f F S + M R Γ 2 (4)
Thus, the mean shear stiffness of the four samples K S is obtained according to Equation ( 5):

K S = ω 2 ((M S1 -M R )Γ 2 + M S2 Γ 5 -2F F S ) 2Γ 2 + 2Γ 5 -Γ 1 -Γ 3 -Γ 4 -Γ 6 ( 5
)
The mean deformation and the mean force undergone by the four samples are given by Equation ( 6):

         X mean = - 1 4ω 2 (2Γ 2 + 2Γ 5 -Γ 1 -Γ 3 -Γ 4 -Γ 6 ) F mean = - 1 4 (M S1 -M R )Γ 2 + M S2 Γ 5 -2F F S ) (6)
Figure 10: Mean deformation X mean and mean effort F mean undergone by the interface between the supports and the jaws (without any samples)

Confidence limit

The measured shear stiffness K S obtained from Equation ( 5) is composed of two stiffnesses in serie: the shear stiffness inside the sample K shim and the shear stiffness of the analyser itself K analyser . The analyser shear stiffness is due to the deformation of the supports, of the jaws and of the contact interfaces as well, thus:

K S = K shim K analyser K shim + K analyser (7)
A test has been led without any samples in order to measure the shear stiffness of the analyser, thus, in this test, K S = K analyser . The mean deformation and the mean effort undergone by the interfaces is shown in Figure 10 for three differents preloads. The modulus and the phase of the complex stiffness K analyser is shown in Figure 11. It depends on the static preload because of the surfaces misalignment and roughness. Thus, high static preload ensure that entire surface is in contact, not only asperities. Since in real life, shims are stuck on pads, they also present a stiffness due to surface roughness. Thus, the real K analyser intrinsic to the analyser is obtained for the higher preload at 3600N and K analyser 350 × 10 6 N/m. From Equation [START_REF] Fritz | Effects of damping on brake squeal coalescence patterns -application on a finite element model[END_REF], one obtains the shear stiffness inside the sample:

K shim = K analyser K S K analyser -K S ( 8
)
Equation ( 8) points out that for K S being almost equal to K shim , the analyser has to be several times stiffer than the samples to be characterized: K analyser K S ⇒ K shim K S . According to Equation [START_REF] Fritz | Investigation of the relationship between damping and mode-coupling patterns in case of brake squeal[END_REF], assuming K analyser = αK S leads to K S = (1 -1 α )K shim . This means that K S can be considered as K shim with a relative error of 1/α : 10% for K S = 35 × 10 6 N/m and 1% for K S = 3.5 × 10 6 N/m.

Results

A shim has been characterized thanks to the shear DMA analyser with and without its layer of adhesive (glue). Three kinds of static preload have been applied: 1200N, 2400N and 3600N. Two dynamic deformation amplitude have been applied: 0.2µm and 1µm. The deformation amplitude of samples has been controlled. Figures 12 and13 show the mean deformation and the mean effort undergone by the four samples with and without their adhesive layer. The actuators were driven with a linear voltage amplifier. When the required current was over the limit of the amplifier, the target of dynamic deformation amplitude was not reached (see Figures 12 and13). This happens especially at high frequency and high displacement amplitude. When an actuator breaks the amplitude also falls as in Figure 13.

Figures 14 and15 show the modulus and the phase of the complex shim stiffness. No filters have been applied on data for possible smoothing. A 0˚phase means purely elastic material without damping and a 90˚phase means purely viscous without elasticity. The shear stiffness modulus and phase at 1000Hz are given Table 1 for the differents preloads and dynamic deformations. The stiffness values are given in N/m. To get the equivalent coefficient G shim , one must use the following formulae: G shim = K shim × e/S with S the surface and e the thickness.

According to Table 1, increasing the static preload leads to an increase of the shear stiffness modulus but does not really affect the phase. Then, increasing the amplitude of deformation leads to a decrease of the shear stiffness modulus and to an increase of phase.

Conclusion

The need for improving disc brake models with the real constitutive law of shims leads us to design a shear DMA analyser. It does not rely on the time-temperature superposition principle. It is one rare analyser allowing both samples preload and excitation above 1000Hz. The upper limit of frequency characterisation is set by the first normal mode interfering with the measurement. This limit is about 3000Hz. Moreover the shearing of samples is pure whereas the classical Oberst's beam method induces a bending excitation of materials.

Figure 1 :

 1 Figure 1: DMA analyser for shear characterization

Figure 2 :

 2 Figure 2: Operating principle of the shear DMA analyser

Figure 3 :

 3 Figure 3: Whole deformation of the shear DMA analyser induced by the normal preload applied thanks to the M6 screw.

Figure 4 :

 4 Figure 4: Pressure undergone by the four samples due to the normal preload applied thanks to the M6 screw.

Figure 5 :

 5 Figure 5: Actuation chain

Figure 6 :

 6 Figure 6: Mode shape of the first eigen mode interfering with measurements. It occurs around 3500Hz depending on the shear stiffness of samples.

Figure 7 :

 7 Figure 7: Rheological modelling of the actuation chain

Figure 8 :

 8 Figure 8: Supports masses identification

Figure 11 :

 11 Figure 11: Mean shear stiffness K analyser of the interface between the supports and the jaws (without any samples)

Figure 12 :

 12 Figure 12: Mean deformation and mean effort undergone by MSC5048 samples without adhesive

Figure 13 :

 13 Figure 13: Mean deformation and mean effort undergone by MSC5048 samples with adhesive

Figure 14 :

 14 Figure 14: Mean shear stiffness of MSC5048 samples without adhesive

Table 1 :

 1 Shear stiffness modulus K and phase φ of each sample at 1000Hz. Some data is unavailable because an actuator has broken during the test.


Salima Mdhaffar

Jean-Franc ¸ois Bonastre

Marc Tommasi

Natalia Tomashenko

Yannick Estève

RETRIEVING SPEAKER INFORMATION FROM PERSONALIZED ACOUSTIC MODELS FOR SPEECH RECOGNITION

Keywords: Automatic speech recognition, acoustic model, personalized acoustic models, collaborative learning, speaker information

The widespread of powerful personal devices capable of collecting voice of their users has opened the opportunity to build speaker adapted speech recognition system (ASR) or to participate to collaborative learning of ASR. In both cases, personalized acoustic models (AM), i.e. fine-tuned AM with specific speaker data, can be built. A question that naturally arises is whether the dissemination of personalized acoustic models can leak personal information. In this paper, we show that it is possible to retrieve the gender of the speaker, but also his identity, by just exploiting the weight matrix changes of a neural acoustic model locally adapted to this speaker. Incidentally we observe phenomena that may be useful towards explainability of deep neural networks in the context of speech processing. Gender can be identified almost surely using only the first layers and speaker verification performs well when using middle-up layers. Our experimental study on the TED-LIUM 3 dataset with HMM/TDNN models shows an accuracy of 95% for gender detection, and an Equal Error Rate of 9.07% for a speaker verification task by only exploiting the weights from personalized models that could be exchanged instead of user data.

INTRODUCTION

Automatic speech recognition (ASR) is now at the heart of a large number of applications used on a daily basis by a large number of users. In order to improve the performance of their ASR solutions, it is common that companies collect and centralize data to train new acoustic models. New data regulations such as the General Data Protection Regulation in the European Union change the rules in order to protect the citizen privacy [START_REF] Nautsch | The GDPR & Speech Data: Reflections of legal and technology communities, first steps towards a common understanding[END_REF]. In order to improve the performance of ASR models by leveraging user experience without accessing their This work was supported by the French National Research Agency under project DEEP-PRIVACY (ANR-18-CE23-0018) and by the VoicePersonae project.

data, solutions such as federated learning are increasingly being proposed. They consist on exchanging personalized models, or their gradients, instead of data [START_REF] Leroy | Federated learning for keyword spotting[END_REF][START_REF] Hard | Training keyword spotting models on non-iid data with federated learning[END_REF][START_REF] Guliani | Training speech recognition models with federated learning: A quality/cost framework[END_REF][START_REF] Yu | Federated learning in ASR: Not as easy as you think[END_REF][START_REF] Cui | Federated acoustic modeling for automatic speech recognition[END_REF] to preserve the user privacy. In the framework of collaborative distributed learning, a personalized model is a model that has been locally adapted to a user [START_REF] Mansour | Three approaches for personalization with applications to federated learning[END_REF]. In a very recent work [START_REF] Mdhaffar | Study on acoustic model personalization in a context of collaborative learning constrained by privacy preservation[END_REF], we presented a such approach to personalize an hybrid HMM/TDNN acoustic model [START_REF] Peddinti | A time delay neural network architecture for efficient modeling of long temporal contexts[END_REF] in a context of collaborative learning.

In this paper we investigate the information contained in personalized acoustic models. Especially, we are interested in the information related to the speaker identity or the speaker gender that is retrievable from personalized acoustic models. Previous works have studied speech intermediate representations computed within neural end-to-end models for speech recognition. They illustrated the way such end-to-end models build phonetic and graphemic representations [START_REF] Belinkov | Analyzing hidden representations in end-to-end automatic speech recognition systems[END_REF][START_REF] Belinkov | Analyzing Phonetic and Graphemic Representations in Endto-End Automatic Speech Recognition[END_REF], or showed how speaker variability and noise are gradually removed as the layer goes deeper [START_REF] Li | What does a network layer hear? analyzing hidden representations of end-to-end asr through speech synthesis[END_REF]. To our knowledge, there is no study on the information provided by the changes in neural weight due to an acoustic model personalization. Our assumption is that the changes applied to the weights of a neural acoustic model when this model is adapted to a speaker brings information about this speaker. Thanks to our experimental protocol we expect to evaluate the level of speaker information that can be retrieved directly from these weight changes, and also highlight in which neural layers these changes are particularly discriminant for such information.

Section 2 describes the acoustic model personalization, section 3 presents the approach proposed to retrieve gender and speaker information from the personalized acoustic models, while section 4 describes the experimental set up and section 5 the experimental results.

ACOUSTIC MODEL PERSONALIZATION

In our scenario, a global acoustic model is available, trained on an initial public dataset. This global model is distributed to many devices -each device is linked to only one speaker -on which it is possible to fine-tune a local instance of the global model by locally exploiting the user data. Fine-tuning consists in continuing the training process of the generic acoustic model on a small dataset of the target speaker, by taking care on avoiding overfitting. The output of the fine-tuning process is considered as a personalized model for the local speaker. Figure 1 illustrates the model personalization explored in this work. Used in the context of a collaborative distributed learning, for instance federated learning, such personalized models, or their gradients, would be exchanged in order to aggregate and improve a global model without sharing user data in an iterative way. Even if local data are not transmitted, a possible leakage of personal information appears when personalized models are exchanged. Therefore, our study explores the amount of personal information that can be inferred from the model weights of the personalized models.

SPEAKER INFORMATION RETRIEVAL

During the model personalization, the weights of the neural generic model are updated. We assume that these weight updates are dependent to some speaker characteristics. We expect that it is possible to extract such speaker information by only studying these weight changes. We focus on gender and on speaker identity. In addition, we investigate in which hidden layers these changes are particularly informative to retrieve such speaker information.

Gender information

In order to retrieve the gender information from the personalized models, we propose an approach based on clustering into 2 classes. We assume the two clusters corresponds to a female/male classification. We evaluate this hypothesis by calculating the purity criterion, by using the gender labels as ground truth.

We perform as many clustering as the number of layers in the models. The inputs are the weights of the layers at the same depth. The clustering algorithm is an agglomerative clustering that merges the closest pair of clusters recursively, building a hierarchy of clusters in bottom-up fashion. The distance between layers is the Euclidean distance and the Ward linkage function is used to evaluate the distance between clusters. It is based on the minimum variance method and allows to minimize the total within cluster variance.

Speaker Identification

In the second part of this study, we want to evaluate the ability to identify speakers, again by only considering the changes applied to weight matrices during the personalization of an ASR acoustic model. However, such weight matrices, and even their hidden layers, are too large to characterize the speaker. Reduction dimensionality approaches like Principal Component Analysis (PCA) are a potential solution but the large reduction factor targeted, combined to a limited number of samples -one model by speaker -could result in this case in a large loss of discriminant information. In order to solve this dimension/discrimination problem, we propose to apply a method inspired from [START_REF] Snyder | X-vectors: Robust dnn embeddings for speaker recognition[END_REF], that consists in learning a speaker embedding extractor. This neural network-based extractor is trained on the weight matrices of a given hidden layer from personalized neural ASR models. Training objective is a speaker discrimination task. But we have to face two difficulties: the input matrices are very large, and the training dataset is very small.

We propose to modify the training objective by using classes of speakers as classification labels, in place of speakers. This allows us to increase the number of examples per class during the training phase, and so to reduce the risk of overfitting. The classes of speakers used for the speaker embedding extractor training are issued from hierarchical clustering of i-vector of speakers present in the training data.

In order to drastically reduce the memory footprint of the extractor and overcome this difficulty, we designed a specific structure for our extractor. Starting from a classical deep neural network (DNN) classifier, we apply a multi-stream input approach. The weight matrix is split into small blocks that are separately linked to a dedicated hidden layer. A small block of the input weight matrix is composed by all the weights related to a unit neural in the hidden layer targeted in the ASR acoustic model. For instance, if the targeted hidden layer H t of the ASR acoustic model architecture contains n units, the weight matrix used as input of our speaker embedding extractor will be split into n different blocks. Next, the outputs of the hidden layer dedicated to each block are concatenated to feed the upper hidden layer of the DNN-based extractor, composed of fully connected layers followed by the final softmax layer.

The structure of the resulting embedding extractor is illus-Fig. 2. Proposed DNN structure to train speaker's embedding from neural network weights of acoustic models trated in Figure 2. The embedding layer is the hidden layer just below the softmax one. The resulting DNN model is able to extract speaker embeddings from speech data, including for speakers that were not present in the training.

EXPERIMENTAL FRAMEWORK

ASR system

Our experiments target (chain) HMM/TDNN acoustic models for speech recognition [START_REF] Povey | Purely sequencetrained neural networks for asr based on lattice-free mmi[END_REF]. The ASR system is based on the Kaldi toolkit [START_REF] Povey | The Kaldi speech recognition toolkit[END_REF]. The chain-TDNN setup is based on 13 layers with 512 dimensions and is trained on cepstral mean and variance normalized 40-dimensional MFCC features. I-vectors are also incorporated as auxiliary input features. The resulting context of TDNN models is 28 left and 28 right neighbour frames. The acoustic model has about 14 million parameters. For the generic model, the initial and final scheduled learning rates are equal to 0.00025 and 0.000025 respectively. Training audio samples are randomly perturbed in speed and volume during the training process.

Dataset

All experiments to train generic and fine-tune acoustic models are conducted with the TED-LIUM 3 dataset [START_REF] Franc ¸ois Hernandez | TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation[END_REF], a large corpus of 452 hours of TED talks pronounced by 2,295 speakers. For the study presented in this paper, an original setup has to be defined. Similarly to [START_REF] Mdhaffar | Study on acoustic model personalization in a context of collaborative learning constrained by privacy preservation[END_REF], the dataset is splited into three parts and the sets of speakers in each part are pairwise disjoints. Characteristics of the three parts are reported in Table 1. The first part is called generic and has been used to train the initial acoustic model for ASR. The two other parts, called p1 and p2, are used for model personalization and evaluation.

In both subsets, the available audio material is split in order to get two sessions of 5 minutes per speaker. Each session is used to personalize a model. Table 1 presents the statistics of the three subsets. For p1 and p2, the table presents the exact number of speakers who have pronounced enough speech to have two sessions of five minutes (463 from 650 speakers for p1 and 581 from 765 speakers for p2, respectively).

The TED-LIUM 3 dataset is provided without information about the gender. Using the website of TED conference, the annotation of the corpus in gender was done manually for p2. 1

Personalized models

The initial generic model is trained on the generic part. Personalized models are obtained by fine-tuning the generic model on the speaker's data from p1 and p2: for each speaker, we personalize the generic model twice using separately his/her two five-minutes sessions. Thus, for most of the speakers (speakers with duration > 10 minutes), two different personalized models are obtained. When fine-tuning the generic model on target speaker data, we modify only the value of learning rate (the initial and final learning rates were equal to 0.000025 and 0.000015 respectively) and all hyperparameters (i.e. learning rate and local epochs number) are assumed to be homogeneous among all workers.

RESULTS AND ANALYSIS

Gender identification

There are several methods used to evaluate clustering performance. In our study, we use the purity. Purity focuses only on maximising the total number of true positive responses per cluster. Purity values range between 0 and 1 (perfect clustering). It is defined as

P urity = 1 N k i=1 max j |c i ∩ t j |
where N is the number of speakers, k is the number of clusters, c i is a cluster and t j is the classification count for cluster c i .

Figure 3 shows the results for the different hidden layers of neural network of ASR acoustic models from which we extract weights for data in p2. We observe that it is possible to get two gender-based clusters with a purity value of 0.96 for the layer 5. Results show that gender information can be identified for the five first layers.

Speaker verification

Speaker verification is evaluated in terms of false alarm (FA) and false reject (FR) error rates and reported using equal error rate (EER), with EER = FA = FR. Fig. 4. Speaker verification performance depending on the hidden layer of the acoustic model used to extract weights.

First, a speaker embedding extractor is trained using each layer of our acoustic models as input. To train the extractors, we use 926 personalized speaker models corresponding to 463+463 unique subsets from p1. The trained extractor models are then used on p2 data to extract the speaker's embedding (knowing that there is no overlap between p1 and p2 speakers). Respectively, a second experiment is conducted with p1 as a test set and p2 as a training set for the extractor. The number of target classes (issued from a hierarchical clustering of i-vectors of speakers present in the training data) used to train our extractor is fixed to 20 and the dimension of the output vectors, the speaker embeddings, is fixed to 100. We use a speaker verification task to evaluate the ability to recognize the speakers from a given layer weights. A simple cosine distance is used to compute the verification score for a trial (enrolment,test). The data of each speaker (see Section 4.2) is divided into two sessions, denoted s1 and s2. It gives one target trial, (x s1 i , x s2 i ), per speaker x i . Nontarget trials, (x s1 i , x s2 j ), are formed by crossing the first session of a given speaker with all the second sessions of the other speakers. It gives respectively 463 / 581 target trials and 213 906 / 336 980 non-target trials for p1 and p2. Figure 4 shows the comparative results in terms of EER.

The best performance is obtained using layer 9 (9.07% EER for p2 and 10% EER for p1), showing clearly that speaker specific information could be extracted from the weights of a personalised ASR acoustic model. For comparison purposes, we also computed the performance when the weight vectors are used directly to compute the cosine distance, without the embedding extractor. The EER is about 48% in this case for p2 (close to the random performance). This proves the effectiveness of the proposed approach to extract a speaker embeddings from the weights of personalized acoustic model.

CONCLUSION

In this study, we showed that it is possible to retrieve the gender and the identity of a speaker from the analysis of the changes applied to the weights of her/his personalized acoustic model. Experiments conducted on the TED-LIUM 3 dataset show that the gender information is mainly brought by the updates impacting the first five layers of a HMM/TDNN acoustic models composed of 13 hidden layers, when the speaker identity is mainly embedded in the middle-up hidden layers (5 to 9). To obtain the latter result, we also proposed an original way to build a speaker embedding extractor from personalized weight matrices. We obtained a gender purity of 0.96 on the five first layers and a speaker verification EER of 9% for layer 9. These results would be particularly interesting for future works focusing on distributed learning for privacy preservation. In this direction, we propose in a parallel study dedicated to attack approaches against federated learning for speech recognition, to use external speech data in order to analyze the behavior of personalized models on such data, see [START_REF] Tomashenko | Privacy attacks for automatic speech recognition acoustic models in a federated learning framework[END_REF].

Fig. 1 .

 1 Fig. 1. Model personalization: (a) A generic model is downloaded to each user device. (b) The generic model is locally fine-tuned on the user data stored on the device.

Fig.

  Fig. Clustering purity of hidden layer weights of the acoustic model using women and men labels as a reference.

Table 1

 1 presents also statistics about the gender.

	generic p1	p2

Table 1 .

 1 TED-LIUM 3 dataset

For the reproductibility of experimental results by research community, we will make available this annotation.