Leila Kerkeni
email: kerkeni.leila@gmail.com

Youssef Serrestou

Kosai Raoof

Mohamed Mbarki

Mohamed Ali Mahjoub

Catherine Cleder

Automatic Speech Emotion Recognition using an Optimal Combination of Features based on EMD-TKEO

Keywords: Speech Emotion Recognition, EMD, TKEO, feature extraction, spectral modulation, feature selection, machine learning, RNN, SVM, classification, human-computer interaction

In this paper, we propose a global approach for speech emotion recognition (SER) system using empirical mode decomposition (EMD). Its use is motivated by the fact that the EMD combined with the Teager-Kaiser Energy Operator (TKEO) gives an efficient time-frequency analysis of the non-stationary signals. In this method, each signal is decomposed using EMD into oscillating components called intrinsic mode functions (IMFs). TKEO is used for estimating the time-varying amplitude envelope and instantaneous frequency of a signal that is supposed to be Amplitude Modulation-Frequency Modulation (AM-FM) signal. A subset of the IMFs was selected and used to extract features from speech signal to recognize different emotions. The main contribution of our work is to extract novel features named modulation spectral (MS) features and modulation frequency features (MFF) based on AM-FM modulation model and combined them with cepstral features. It is believed that the combination of all features will improve the performance of the emotion recognition system. Furthermore, we examine the effect of feature selection on SER system performance. For classification task, Support Vecto Machine (SVM) and Recurrent Neural Networks (RNN) are used to distinguish seven basic emotions. Two databases-the Berlin corpus, and the Spanish corpusare used for the experiments. The results evaluated on the Spanish emotional database, using RNN classifier and a combination of all features extracted from the IMFs enhances the performance of the SER system and achieving 91.16 % recognition rate. For the Berlin database, the combination of all features using SVM classifier has 86.22% recognition rate.

Introduction

A speech emotion recognition (SER) is an important issue in the development of humancomputer interactions (HCI). It consists of predicting, from the speech signal, the emotion state of a speaker. There are many applications of detecting the emotion of the persons such as robot interface, audio surveillance, web-based E-learning, commercial applications, clinical studies, entertainment, banking, call centers, car board systems, computer games etc. For classroom orchestration or E-learning, information about the emotional state of students can provide a focus on enhancement of teaching quality. For example teacher can use SER to decide what subjects can be taught and must be able to develop strategies for managing emotions within the learning environment. That is why learners emotional state should be considered in the classroom. The performance of SER system depends on the quality of: features used to distinguish emotion, classifiers and data set used for training. For SER, several classification methods are used such as Support Vector Machines (SVM) [START_REF] Milton | Svm scheme for speech emotion recognition using mfcc feature[END_REF], [START_REF] Zhang | Deep learning and svm-based emotion recognition from chinese speech for smart affective services[END_REF], [START_REF] Chen | Emotion recognition using support vector machine and deep neural network[END_REF], Gaussian Mixture Model (GMM) [START_REF] Sree | Svm based speech emotion recognition compared with gmm-ubm and nn[END_REF], Hidden Markov Model (HMM), Neural Networks (NN) [START_REF] Prasomphan | Improvement of speech emotion recognition with neural network classifier by using speech spectrogram[END_REF], [START_REF] Sree | Svm based speech emotion recognition compared with gmm-ubm and nn[END_REF], [START_REF] Chen | Emotion recognition using support vector machine and deep neural network[END_REF], Recurrent Neural Networks (RNN) [START_REF] Lim | Speech emotion recognition using convolutional and recurrent neural networks[END_REF], [START_REF] Mirsamadi | Automatic speech emotion recognition using recurrent neural networks with local attention[END_REF], [START_REF] Kerkeni | A review on speech emotion recognition: Case of pedagogical interaction in classroom[END_REF], and Linear Regression (LR) [START_REF] Kerkeni | Speech emotion recognition: Recurrent neural networks compared to svm and linear regression[END_REF]. In this paper, we are particularly interested in the most important step on SER which is feature extraction. Many methods were used also at the stage of obtaining features from the signal. Among these approaches based on linearity and stationarity hypothesis is the Fast Fourier Transform (FFT) method which is the most useful method for frequency domain feature extraction. Nevertheless, this method loses some information [START_REF] Wang | Feature extraction techniques of non-stationary signals for fault diagnosis in machinery systems[END_REF], that may be useful for the classification task, in the time domain. To minimise this problem of stationarity, the Short Time Fourier Transform (STFT) is proposed to improve the traditional Fourier Transform [START_REF] Haque | Frequency analysis and feature extraction of impressive tools[END_REF], [START_REF] Wang | Feature extraction techniques of non-stationary signals for fault diagnosis in machinery systems[END_REF], [START_REF] Nayak | Advanced signal processing techniques for feature extraction in data mining[END_REF]. The STFT consists in repeating the multiplication of a signal by a shifted short time windows and performing a Fourier Transform on the produced signal. Nonetheless, the STFT is also limited by the fundamental uncertainty principle, according to which time and frequency cannot simultaneously be resolved with the same precision. The issue of non-linearity remains problematic [START_REF] Fonseca-Pinto | A new tool for nonstationary and nonlinear signals: The hilbert-huang transform in biomedical applications[END_REF] and the STFT is also not appropriate to extract features from nonlinear systems. However, the speech signal is naturally non-stationary. Thus, it is more convenient to apply a non-stationary and nonlinear signal processing methods on the speech signal. The difficulties associated with an accurate recognition of emotion from speech remains challenging due to variability, complexity and subtle changes of non-linear features of the speech emotion. In order to overcome this problem, we used the EMD method, introduced by Huang et al. [START_REF] Huang | The empirical mode decomposition and the hilbert spectrum for nonlinear and non-stationary time series analysis[END_REF], to decompose and analyze the input signal which may be non linear and/or non stationary without losing its original properties. This method has been successfully applied to the area of speech emotion recognition (SER) [START_REF] Li | Speech emotion recognition using novel hht-teo based features[END_REF], [START_REF] Li | Emd-teo based speech emotion recognition[END_REF], [START_REF] Shahnaz | Emotion recognition based on emd-wavelet analysis of speech signals[END_REF], [START_REF] Zhuang | Emotion recognition from eeg signals using multidimensional information in emd domain[END_REF]. A review of the advancements in the non-conventional analysis of speech signals was written by Rajib Sharma et al. [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF]. The EMD method decomposes the time series data into a number of components holding the highest local frequency, which is a collection of intrinsic mode functions (IMFs). However, the decomposition results using EMD method often suffer from mode-mixing problem [START_REF] Wu | Ensemble empirical mode decomposition: a noise-assisted data analysis method[END_REF], [START_REF] Goel | Speech emotion recognition using eemd[END_REF]. To solve this problem, Huang [START_REF] Wu | Ensemble empirical mode decomposition: a noise-assisted data analysis method[END_REF] proposes Ensemble empirical mode decomposition (EEMD). This method consists to add a Gaussian white noise to the original signal. In the last few years, the EMD method combined with an energy operator is also used as an alternative technique that can improve the time-frequency analysis of signals [START_REF] Antoniadou | Use of the teager-kaiser energy operator for condition monitoring of a wind turbine gearbox[END_REF]. Different energy operators were used for the changes in the sub-band signals. In [START_REF] Li | Speech emotion recognition using novel hht-teo based features[END_REF] and [START_REF] Li | Emd-teo based speech emotion recognition[END_REF], it is seen that effective results are obtained in the case of EMD and Teager-Kaiser Energy Operator (TKEO) being used together. TKEO can track the instantaneous amplitude and instantaneous frequency of the AM-FM component at any instant. The combination of EMD and Teager-Kaiser Energy Operator (TKEO) show an important role for time-frequency analysis and particularly for AM-FM signal demodulation [START_REF] Khaldi | Speech enhancement using empirical mode decomposition and the teager-kaiser energy operator[END_REF]. Features extracted using the methods described above might contain redundant and irrelevant information which may affect SER system accuracy. Feature selection can solve this problem. Several methods for feature selection were found in literature and they are being widely used. The work by Tang and al. [START_REF] Tang | Feature selection for classification: A review[END_REF] summaries these methods. Examples of feature extraction techniques include Linear Discriminant Analysis (LDA), Principle Component Analysis (PCA), Sequential Forward Selection (SFS), Recursive Feature elimination (RFE). All these methods aim to remove the redundant and irrelevant features, so as to reduce the number of features and to improve model accuracy.

In this paper, we propose a global approach for SER. We presented a novel features named modulation spectral (MS) features and frequency modulation features (MFF) based on AM-FM demodulation and formant tracking. We combined them with the cepstral features: Energy Cepstral Coefficients (ECC), Frequency weighted Energy Cepstral Coefficients (EFCC) and the Mel Frequency Cepstral Coefficients based on the Reconstructed Signal (SMFCC). A method to select speech emotion feature named Recursive Feature Elimination (RFE) is applied after. The performance of the new features extracted was improved by speaker normalization. The Recurrent Neural Networks (RNN) and Support Vector Machines (SVM) models are used here to classify seven emotion status. In previous work [START_REF] Kerkeni | Speech emotion recognition: Methods and cases study[END_REF], we compared several classifiers and the results of emotion classification by RNN, SVM and LR classifiers show that the RNN and SVM classifiers can improve classification accuracy significantly for both databases used in our work.

The next section of this paper gives an overview of the proposed emotion recognition system. The AM-FM modulation model using EMD and TKEO methods are explained in section 3. In section 4, we presented the features extracted in our work. Section 5 provides the feature selection method used in this paper. The RNN and SVM models used for classification are presented in section 6. The experimental results based on the features extracted in Section 4, are presented and discussed in Section 7. This section introduces also the databases employed. Finally, Section 8 concludes this work.

Overview of the emotion recognition system

Overall block diagram of our proposed system is shown in Figure 1. Among the most important step for developing a SER system is the extraction of speech features. For an efficient emotion recognition system, the best features are selected in this work to classify any speech signal into one of seven emotions. After that a machine learning algorithm is trained taking these features. Before features are being extracted from the input speech signal, the following pre-processing is done to increase information content: i) decompose speech signal into IMFs using EMD sifting process ii) applying the nonlinear TKEO operator to the extracted IMFs. The process of each block is explained in detail in the following sections.

AM-FM modulation model for speech analysis

In this paper, the AM-FM modulation model is used for speech analysis which represents the speech signal as the sum of formant resonance signals. Maragos and al. [26] describes this speech resonance r i (t) as a signal with a combined amplitude modulation (AM) and frequency modulation (FM) structure:

r i (t) = Re (a i (t) exp [ j φ i (t)]) (1)
According to this model, the speech signal can be expressed as follows:

x(t) = N i=1 r i (t) + res(t) (2)
where res(t) is the last few components, which are low-frequency trend-like waveforms and it is excluded in general from the speech signal [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF], Re represents the real part, φ i (t) represents the phase, a i (t) is the instantaneous amplitude and f i (t) the instantaneous frequency of the ith IMF:

φ i (t) = 2π f i (t)dt
In [START_REF] Potamianos | Speech analysis and synthesis using an am-fm modulation model1[END_REF], the speech resonance signal r(t) is extracted from the speech signal x(t) through bandpass filtering (Gabor filter). In our work, the EMD is used, which represents speech as the sum of IMFs (r i (t)). Next, the TKEO is used to demodulate the resonance signals r i (t) into the amplitude envelope (a i (t)) and instantaneous frequency (f i (t)) signals. The following sections provide a detailed description.

Empirical Mode Decomposition (EMD)

The EMD is a method for decomposing any non-stationary signal into a collection of intrinsic mode functions (IMFs) which are mono-component AM-FM signals. The extraction of IMFs is non-linear, but their recombination for exact reconstruction of the signal is linear. In fact, the addition of all IMFs allows linear reconstruction of the original signal without loss or distortion of the initial information.

Intrinsic Mode Function (IMF)

A function is called an intrinsic mode function when it satisfies the following properties:

• The number of extrema (maxima+minima) in the signal must be equal or differ at most by one to the number of zero crossing;

• The mean of the envelopes defined by local maxima and local minima must be zero at all times.

Algorithm of the EMD

The EMD decomposition procedure for extracting an IMF is called the sifting process and is described as follows (Algorithm 1): The process terminates when the residual res(t) is either a constant, a monotonic slope or a function with only one extrema. The result of the EMD process produces N IMFs (r 1 (t), r 2 (t),..., r N (t)) and residue signal (res N (t)). Consequently, the original data sequence x(t) may represent the sum of a group of IMF values plus a residual term as shown in the following formula:

x(t) = N i=1 r i (t) + res N (t) (3)
In this method, each input signal is decomposed into a finite number of IMFs using the previously described EMD algorithm. Each IMF can be then analyzed separately in order to obtain features for emotion classification. In [START_REF] Sethu | Empirical mode decomposition based weighted frequency feature for speech-based emotion classification[END_REF], authors are limited to the first five IMFs providing that these IMFs give a sufficient information about of energy and pitch in their cases. Several number of IMFs are used in our work of iterations to determine the optimal number of IMFS for reconstructing the signal without any loss of important information for each database. We have seen that after a certain number, the IMFs do not provide information.

Teager-Kaiser Energy Operator (TKEO)

The IMFs r(t) obtained from EMD as such do not convey any information nor provide a meaning. Applying the Teager-Kaiser Energy Operator (TKEO) in the IMFs allows an estimate of time-varying amplitude envelope and instantaneous frequency, which provides some physical meaning. The Teager-Kaiser energy operator is a nonlinear operator that calculates the energy of mono-component signals as the product of the square of the amplitude and the frequency of the signal. The instantaneous characteristics of these signals are then obtained by the application of the Discrete Energy Separation Algorithm (DESA-2) [START_REF] Maragos | Energy separation in signal modulations with application to speech analysis[END_REF]. It will be shown that TKEO improves the estimation of instantaneous characteristics of the vibration data compared to other commonly used techniques, e.g. the Hilbert Transform. A method based on EMD algorithm of Huang et al. [START_REF] Huang | The empirical mode decomposition and the hilbert spectrum for nonlinear and non-stationary time series analysis[END_REF] and TKEO is called Teager-Huang Transform (THT). TKEO is defined for an IMF r i (t) in a continuous time as [START_REF] Maragos | On amplitude and frequency demodulation using energy operators[END_REF]:

Ψ[r i (t)] = [ ṙi (t)] 2 -r i (t) ri (t) (4)
where ṙi (t) and ri (t) are respectively the first and the second time derivatives of r i (t). For a discrete time signal r i (n), the time derivatives of the equation 4 can be proposed as [START_REF] Maragos | Speech nonlinearities, modulations, and energy operators[END_REF]:

Ψ[r i (n)] = r 2 i (n) -r i (n + 1)r i (n -1) (5)
where n is the discrete time index.

The following equations describe exactly the instantaneous frequency f (n) and instantaneous amplitude a(n) at any time instant of the IMF r i (n) [START_REF] Maragos | Energy separation in signal modulations with application to speech analysis[END_REF]:

f (n) = 1 2 arccos(1 - Ψ[x(n + 1) -x(n -1)] 2Ψ[x(n)] (6)
| a(n) |= 2Ψ[x(n)] Ψ[x(n + 1) -x(n -1)] (7)
Researchers in [START_REF] Potamianos | A comparison of the energy operator and the hilbert transform approach to signal and speech demodulation[END_REF] found that the energy operator approximation incurs a high-frequency error component. So they proposed to eliminate the high-frequency error component by filtering the energy operator output through an appropriate low-pass filter as shown in Figure 2 . For this we used in our work a seven-point linear binomial smoothing filter with impulse response [START_REF] Milton | Svm scheme for speech emotion recognition using mfcc feature[END_REF][START_REF] Lim | Speech emotion recognition using convolutional and recurrent neural networks[END_REF][START_REF] Li | Speech emotion recognition using novel hht-teo based features[END_REF][START_REF] Wu | Ensemble empirical mode decomposition: a noise-assisted data analysis method[END_REF][START_REF] Li | Speech emotion recognition using novel hht-teo based features[END_REF][START_REF] Lim | Speech emotion recognition using convolutional and recurrent neural networks[END_REF][START_REF] Milton | Svm scheme for speech emotion recognition using mfcc feature[END_REF] [START_REF] Potamianos | A comparison of the energy operator and the hilbert transform approach to signal and speech demodulation[END_REF].

Feature Extraction

The crucial process in SER system consists of extracting the essential information from the signal. In this study, we have used EMD combined with the TKEO to extract features as illustrated in Figure 3, 4, 5 and 8. Each set of features is explained and detailed in the following sections.

Cepstral features

The cepstral features are important parameters in SER, which exploites the property of human auditory system and there are derived from the signal spectrum. In this section, we extracted these features to combine them with the new features presented in Section 4.2. Both types of features contain emotion information, it is believed that the combination of them will improve the performance of the emotion recognition system.

SMFCC features

MFCC is widely used for speech emotion recognition [START_REF] Zhang | Deep learning and svm-based emotion recognition from chinese speech for smart affective services[END_REF]. It has good performance in description of the human ear's auditory characteristics. Assuming that a speech signal is a short-term stationary process, MFCC convey the signal's short term spectral only and omit some important information. But really, the speech signal posses complex and random changes and the existence of signal trend would result in great error with respect to the power spectral analysis in frequency domain or correlation analysis in time domain, and even lead to the completely loss of the authenticity of the low-frequency spectrum. Hence the need to remove this signal trend. In this section, we calculate the features extracted after the removal of signal trend T (n). These features, called SMFCC and introduced in [START_REF] Li | Emd-teo based speech emotion recognition[END_REF], provide a more accurate description of the distribution of energy in the frequency domain. The EMD based signal reconstruction method is carried out to conduct the SMFCC feature extraction. The process of extracting SMFCC is represented in Figure 3. Firstly, the EMD method is conducted on the input speech signal. Secondly, the signal trend, according to [START_REF] Li | Speech emotion recognition using novel hht-teo based features[END_REF], is calculated by the formula below (equation 9) and removed after from the original signal using the zero-crossing rate (ZCR) detection method. T (n) is the sum of IMFs r(n) that respect the following constraint [START_REF] Li | Speech emotion recognition using novel hht-teo based features[END_REF]:

R r i R r 1 < 0.01 (i = 2, ...n) (8)
where R represents the zero-crossing rate.

T (n) = i r i (n) (9)
Subsequently, the final signal S(n) is obtained by the subtraction of the signal trend T (n) from the original speech signal x(n) [START_REF] Li | Speech emotion recognition using novel hht-teo based features[END_REF].

S(n) = x(n) -T (n) ( 10
)
The SMFCC is extracted from the obtained reconstituted signal S(n) through FFT algorithm and discrete Cosine Transform (DCT).

Energy Cepstral Coefficients (ECC) and Frequency weighted Energy Cepstral Coefficients (EFCC) features

In [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF], authors demonstrate that the distribution of spectral energy varies under different emotions. This means that emotion can affect energy distribution of speech among different frequency bands. Hence the importance of extracting the ECC feature that provides the distribution of energy in the Hilbert spectrum (in the time domain). In standard approach, the instantaneous energy of each IMF is considered to be proportional with the amplitude and has nothing to do with the instantaneous frequency. According to the physical model of instantaneous frequency, not only the instantaneous energy is included in the energy envelope of a(t), but also in the instantaneous frequency f (t). Since the internal energy is proportional with its circumferential velocity by physical sense [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF]. On the basis of the above considerations, the researchers developed the instantaneous frequency-weighted energy (EFCC) to improve the ECC features presented above. The standard implementation of computing the Energy Cepstral Coefficients (ECC) and the Frequency weighted Energy Cepstral Coefficients (EFCC) [START_REF] Li | Speech emotion recognition using novel hht-teo based features[END_REF] is shown in Figure 4. The first processing step is the decomposition of speech signal into IMFs using EMD. The imf's instantaneous amplitude (a(i, n)) and the IMF's instantaneous frequency (f (i, n)) are estimated using TKEO at the second step. The third processing step tries to register the instantaneous amplitude and frequency into short and overlapping frames (the duration of frame is 250 ms and the overlapping is equal to 64 ms) which gives a k (i, n) and f k (i, n) . The fourth processing step is the computation of the Marginal Hilbert Spectrum (MHS) using Hilbert spectrum. It offers a measure of the total amplitude from each frequency. Therefore, the spectrum is decomposed into different frequency bands (12 bands) and the power of each frequency band is computed. Afterwards, the natural logarithm of sub-band energy and complement the discrete cosine transform (DCT) are computed. The first 12 DCT coefficients provided the ECC and EFCC values used in the classification process. The MHS h j (f ) gives the probability that an ascillation of frequency could have occured locally at some time during the entire duration of the signal [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF] and it is obtained by:

h j (f ) = L f n=1 H(f, n)1 B j (f ) (11)
where L f represents the length of frame in samples and H(f, n) is the Hilbert spectrum. It is defined as the instantaneous energy envelope in a time-frequency space, which is the the squared magnitude of the amplitude envelope [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF]. H(f, n) is derived by:

H(f, n) = N i=1 a(i, n) 2 1 {f (i,n)} (f ) (12)
where i represents a particular IMF, and B j represents a particular sub-band. B j is defined as: B j = [f c (j -1), f c (j + 1)] where f c represents the frequency center.

The indicator function of a subset Ω of a set x is defined as:

1

Ω (x) = 1 if x ∈ Ω 0 if x / ∈ Ω 4.1.2.1 Energy Cepstral Coefficients (ECC)
ECC is given in a continuous time by [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF]:

ECC(B j , Π i ) = f ∈B j h j (f )df, t ∈ Π i , j = 1, ..., 12 (13)
where B j represents a particular sub-band, and Π i represents a particular speech frame.

In this work, these features are calculated, by using a discrete time of speech signal, as follows:

ECC k (j) = N i=1 1 L F L F n=1 a 2 k (i, n)1 B j (f k (i, n)), j = 1, ..., 12 (14) 4.1

.2.2 Frequency weighted Energy Cepstral Coefficients (EFCC)

EFCC is given in a continuous time by [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF]:

EF CC(B j , Π i ) = f ∈B j f (t)h j (f )df, t ∈ Π i , j = 1, ..., 12 (15)
Where B i represents a particular sub-band, and Π i represents a particular speech frame. For a discrete case, EFCC are calculated as follows:

EF CC k (j) = N i=1 1 L F L F n=1 f k (i, n)a 2 k (i, n)1 B j (f k (i, n)), j = 1, ..., 12 (16)
where B j represents a particular sub-band, i represents a particular IMF, and k represents a particular speech frame. In our work, we extract the first 12-order of the ECC and EFCC coefficients where the speech signals are sampled at 16 KHz. For each order coefficients, we compute the mean, variance, coefficient of variation, Kurtosis and Skewness. Each ECC and EFCC feature vector is composed of 60 points.

AM-FM modulation features

In this section, we presented our new features based on AM-FM modulation model. This model captures both long-term modulation features and modulation frequency features, thereby conveying information that is important for human speech perception.

Modulation Spectral (MS) features

The modulation spectral features (MSF) introduced in [START_REF] Wu | Automatic speech emotion recognition using modulation spectral features[END_REF] are extracted specifically to solve the problems of short-term spectral features (MFCC) and to better model the nature of human auditory perception. The method is based on emulating the Spectro-temporal (ST) processing performed in the human auditory system and considers regular acoustic frequency jointly with modulation frequency. These features are based on decomposition of speech signal by an auditory filter bank and the computation of the Hilbert envelope of each band. The Hilbert envelope is based on the Fourier transform (FT) and hence some of the limitations of FT are also associated with it. For this reason we propose in this work a new way of extracting these features based on speech analysis using AM-FM modulation model. The steps for extracting modulation spectral (MS) features depicted in Figure 5. After applying TKEO on IMFs decomposed by means of EMD. A modulation filter bank is further applied to the instantaneous amplitude to perform frequency analysis. The spectral contents of the modulation signals are referred to as modulation spectra, and the proposed features are named modulation spectral (MS) features. The energy, taken over all frames in every spectral band provides a feature (E(i, j)). Energy in every spectral band, is defined as:

E(i, j) = N f k=1 E k (i, j) (17)
Where E k (i, j) is the Energy over bands, N f is the number of frame for 1 ≤ j ≤ 8. For each frame k, E(i, j) is normalized to unit energy before further computation:

i,j E k (i, j) = 1
Three spectral measures Φ 1 , Φ 2 , Φ 3 are then calculated on a per-frame basis [START_REF] Wu | Automatic speech emotion recognition using modulation spectral features[END_REF]. The first spectral measure is the spectral energy Φ 1,k (j) which is defined as the mean of the energy samples belonging to the jth modulation band (1 ≤ j ≤ 8):

Φ 1,k (j) = N i=1 E k (i, j) N (18)
For frame k, Φ 2,k (j) is the spectral flatness which is defined as the ratio of the geometric mean of Φ 1 to the arithmetic mean. Φ 2 is thus defined as follows:

Φ 2,k (j) = N N i=1 E k (i, j) Φ 1,k (j) (19)
In our work, Φ 2 is expressed on a logarithmic scale as follows:

log Φ 2,k (j) = 1 N N i=1 log E k (i, j) -log Φ 1,k (j)
The last spectral measure employed Φ 3 is the spectral centroid which provides a measure of the "center of mass" of the spectrum in each modulation band. For jth modulation band, Φ 3 is defined as:

Φ 3,k (j) = N i=1 f (i)E k (i, j) N i=1 E k (i, j) (20)
Where f (i) = i . In [START_REF] Wu | Automatic speech emotion recognition using modulation spectral features[END_REF], the authors experiment two similar types of frequency measure f (i). It being in our case the index of the ith critical-band filter, i.e., f (i) = i. We choose it for its simplicity. We have noted that there is a considerable correlation between two adjacent modulation bands. In order to reduce the high correlation for the spectral flatness and the centroid parameters, Φ 2,k (j) and Φ 3,k (j) are only computed for j ∈ {1, 3, 5, 7, 8}.

Various statistics, mean, variance, coefficient of variation, kurtosis and skewness, are extracted from energy in each spectral band. Along with those statistics, the mean and variance of the spectral energy, spectral flatness and spectral centroid are evaluated, and used as features.

Figure 6 shows the mean of Energy E(i, j) for the seven emotions (anger, boredom, disgust, f ear, happiness, sadness and neutral) in the Berlin database. Every E(i, j) represents the mean over all frames from all speakers for an emotion. It is seen from this Figure that the energy distribution over the joint IMF-modulation frequency plane is similar for some emotions that could become confusion pairs, like anger and happiness, boredom and neutral. But it is very distinct for some others emotions, like anger and sadness, they could be well discriminated from each other.

Figure 6: Mean E(i, j) for seven emotion; For each emotion, E k (i, j) is averaged over all frames from all speakers of that emotion; "MC" denotes the modulation frequency channels.

Figure 7 shown that the emotions sadness and neutral have significantly more low acoustic frequency energy than anger emotion, which is approximately the same in the previous work [START_REF] Wu | Automatic speech emotion recognition using modulation spectral features[END_REF]. For the emotion of anger the dispersion is more greater. However, the less expressive emotions such as sadness exhibit more prominently low pass modulation spectral shapes, which suggest of lower speaking rates.

Modulation Frequency Features (MFF)

Studies on speech perception demonstrated that the most important perceptual information lies at low modulation frequencies [START_REF] Atlas | Joint acoustic and modulation frequency[END_REF]. In this section, we will illustrate how modulation frequency analysis can be formulated and applied to the problem of SER. The AM-FM modulation model is used for the analysis of signal and we are interested in the frequency distribution of the energy of the speech signal, in a frequency band. The steps to compute the frequencies based on AM-FM decomposition of the input speech are shown in Figure 5. The speech signal is decomposed into several intrinsic mode functions (IMFs) by EMD. The instantaneous amplitude envelope and frequency function of each IMF is estimated using the TKEO. Then, they are used to obtain short-time estimates of mean instantaneous frequency and bandwidth over a frame. The mean amplitude weighted instantaneous frequency (F w ) and the mean amplitude weighted instantaneous band-width (B w ) are described for a continuous time signal in [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF] by :

F w = t 0 +T t 0 f (t)a 2 (t)dt t 0 +T t 0 a 2 (t)dt (21) B 2 w = t 0 +T t 0 [{ ȧ(t)/2π} 2 + {f (t) -F w } 2 a 2 (t)]dt t 0 +T t 0 a 2 (t)dt (22)
where t 0 and T represent the beginning and duration of analyzed frame, f (t) and a(t) represent respectively the instantaneous frequency and amplitude envelope of each AM-FM signal and ȧ(n) is discretized and defined by:

ȧ(t) = a(t + 1) -a(t)
Using the amplitude envelope and instantaneous frequency signals short-time estimates are proposed in this work such as: the mean instantaneous frequency (F ), the mean instantaneous band width (B), the mean amplitude weighted instantaneous frequency (F w ) and the mean amplitude weighted instantaneous band-width (B w ), are given in the discrete case by:

F k (i) = 1 L F L F n=1 f i (k, n) (23)
B k (i) = 1 L F L F n=1 (f k (i, n) -F k (i)) (24)
F w k (i) = L F n=1 f k (i, n)a 2 k (i, n) L F n=1 a 2 k (i, n) (25)
B w k (i) = L F n=1 { ȧk (i, n)/2π} 2 + {f k (i, n) -F w k (i)} 2 a 2 k (i, n) L F n=1 a 2 k (i, n) (26)
where i represents a particular IMF, L F represents a number of samples per frame, f k and a k represent respectively the instantaneous frequency and amplitude frequency at a particular speech frame k. In this work the mean, maximum and minimum of these two values (B, B w ) and the mean of (F , F w ) were computed for each feature and it was used for classification.

We can see the usefulness of AM-FM analysis from the time-frequency representation of a speech signal shown in Figures 9 and10. For each IMF component, a short time frequency estimate F k (i) is obtained at every frame using Equation 23. The time duration of the analysed frame is 25 ms with a frame-shift of 10 ms. The figures below indicate that AM-FM analysis could provide useful information to discriminate emotion in speech signal. They identify regions with dense clusters for the purpose of tracking formant frequencies and their bandwidths [START_REF] Sharma | Empirical mode decomposition for adaptive am-fm analysis of speech: A review[END_REF]. It may be noted from Figures 9 and 10 that the dense clusters of curves in the time-frequency representation are different between the "anger" speech signal and "neutral" speech signal, the difference is due to emotion. The estimated density of probability, in Figure 12, confirm this assumption. In the case of 535 utterances here considered, we obtained that the fundamental frequency was embedded in the IMF10-IMF12. Figure 11 shown the mean values of the instantaneous frequency (F ), for indexes from 1 to 535, corresponding to each file on the Berlin database. In Figure 12, we find almost the same results as Figure 7. The anger emotion is shifting to the high frequencies (between 600 and 1300 Hz). However, the emotion of sadness towards the low frequencies.

Feature Selection (FS)

After extracting features from the speech signal, the most important thing is that how to select features that are capable of discriminating samples that belong to different classes. As reported by Aha and Bankert [START_REF] Aha | Feature selection for case-based classification of cloud types: An empirical comparison[END_REF], FS is a process to select a subset of relevant features used to characterize a dataset so as to improve a learning algorithm's performance on a given task. The extracted features contain unneeded, irrelevant and redundant data that do not contribute to the accuracy of a predictive model or may in fact decrease the accuracy of the model. The extraction of features is not enough for a better performance of the classifier. So it is necessary to select features. FS allows not only to lead to higher recognition accuracy but also to reduce drastically the running time of the learning algorithms. In this study, an effective and a simple feature selection method named Recursive Feature Elimination (RFE) is used.

Recursive Feature Elimination (RFE)

RFE uses a model (eg. linear Regression or SVM) to select either the best or worstperforming feature, and then excludes this feature. These estimators assign weights to features (e.g., the coefficients of a linear model), so the goal of recursive feature elimination (RFE) is to select features by recursively considering smaller and smaller sets of features. First, the estimator is trained on the initial set of features and the predictive power of each feature is measured [START_REF] Duan | Multiple svm-rfe for gene selection in cancer classification with expression data[END_REF]. Then, the least important features are removed from current set of features. This procedure is recursively repeated on the pruned set until the desired number of features to select is eventually reached. In this work, we implemented the Recursive Feature Elimination method of feature ranking via the use of basic Linear Regression (LR-RFE) [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]. SVM-RFE is an SVM-based feature selection algorithm created by [START_REF] Guyon | Gene selection for cancer classification using support vector machines[END_REF]. Using SVM-RFE, Guyon et al. selected an important feature sets. In addition to improving the classification accuracy rate, it can reduce classification computational time.

Classification models

In machine learning, classification is a supervised learning approach in which the computer program learns a mapping from the training samples in certain features space to some labels. This training samples may simply be bi-class (i.e identifying whether the person is male or female ) or it may be multi-class too. There are many types of classification algorithms in machine learning: Linear Regression, Decision Tree, SVM, Naive Bayes, KNN, Random Forest, Neural Networks, etc. These algorithms can be applied to almost any data problem. Here we have used SVM and RNN classifiers.

Support Vector Machine (SVM)

SVM is a simple and optimal classifier in machine learning. The advantage of SVM compared to other classifiers is, for a limited training data it shows better classification performance [START_REF] Sree | Svm based speech emotion recognition compared with gmm-ubm and nn[END_REF]. SVM has been used extensively in many studies that related to audio emotion recognition and has been shown to give best performance in many SER works [START_REF] Milton | Svm scheme for speech emotion recognition using mfcc feature[END_REF] and [START_REF] Pan | Speech emotion recognition using support vector machine[END_REF]. It is built by mapping the training patterns into a higher dimensional feature space where the points can be separated using a hyper plane. The idea behind the SVM is to transform the original input set to a high dimensional feature space by using kernel function. The kernel make the SVM method much more flexible and powerful. SVM kernels include: linear, polynomial, radial basis function (RBF) and sigmoid [START_REF] Wu | Recognition of human emotion in speech using modulation spectral features and support vector machines[END_REF]. SVM theoretical background can be found in [START_REF] Gunn | Support vector machines for classification and regression[END_REF]. A MATLAB toolbox implementing SVM is freely available in [START_REF]Svm and kernel methods matlab toolbox[END_REF]. Here in our work we have used polynomial kernel [START_REF] Wu | Recognition of human emotion in speech using modulation spectral features and support vector machines[END_REF], which is defined as:

K(x i , x j ) = ( x i , x j + c 0 ) d ( 27
)
where d is the degree of the polynomial. In general, nonzero c 0 is preferred [START_REF] Wu | Recognition of human emotion in speech using modulation spectral features and support vector machines[END_REF].

Recurrent Neural Networks (RNNs)

RNN is recommended for sequence classification where there are some sequence of inputs over space or time and the task is to predict a category for the sequence. In recent years, RNN have demonstrated ground-breaking performance for sequence analysis [START_REF] Lim | Speech emotion recognition using convolutional and recurrent neural networks[END_REF], [START_REF] Zhang | Deep learning for sentiment analysis: A survey[END_REF]. This work used the long short term memory (LSTM), which is a special kind of RNN capable of learning long-term dependencies, to implement the RNN layer. LSTM were developped by Hochreiter et al [START_REF] Hochreiter | Long short-term memory[END_REF]. RNN models suffer from the vanishing gradient problem which increases with the length of the training sequences. LSTM solving this problem in classical RNN by using memory cells that allow information to accumulate during operation and uses feedback to remember previous network call states [START_REF] Chen | Multi-modal Dimensional Emotion Recognition using Recurrent Neural Networks[END_REF]. Figure 13 shows a basic concept of RNN implementation. Unlike traditional neural network that uses different parameters at each layer, the RNN shares the same parameters (U,V and W in Figure 13) across all steps. The hidden state formulas and variables are as follows:

s t = f (U x t + W s t-1 ) (28)
with:

• x t , s t and o t are respectively the input, the hidden state and the output at time step t;

• U, V, W are parameters matrices.

Experimental results and analysis

Emotional databases

In the design of a SER system, one important thing upon which the performance of emotion recognition system depends is the database used to build its representation of humans emotions. Different types of databases have been used for the study of emotions in speech [46]. First type is the simulated databases, also called acted databases. Speakers are generally experienced professional actors. The actors are asked to express some sentences in different emotions. Another type is the induced databases, also called elicited databases. It is collected by provoking an emotional reaction using an artificial emotional situation. Emotional states are induced using different methods, such as videos, images, stories or computer games. Without the knowledge of the speaker that they have been recording, the databases are more natural and realistic than the simulated databases. The third type is the natural emotions. These emotions are obtained by recording speakers in natural situations such as call center conversations, TV talk shows, patient doctor conversations and so on. This type of material might obscure the exact nature of recorded emotions because of background noise, overlapping voices, artifacts, etc, which makes the quality of speech unsatisfactory. In this work we used acted databases because it is the only one that allows a complete control over the recorded text [47].

Berlin database

The Berlin database [START_REF] Burkhardt | A Database of German Emotional Speech[END_REF] is a German acted database. It is the most commonly used database in the context of speech emotion recognition, and also one of the common public databases which can be freely accessed by researches. It consists of recording 10 German sentences by 10 actors (5 male, 5 female). The final database consists of 535 utterances recorded in anger, boredom, disgust, f ear, happiness, sadness and neutral. The samples for each emotion are distributed as shown in Figure 14.

Spanish database

The Spanish emotional speech database is available for researches use [START_REF]Emotional speech synthesis database[END_REF], and it contains more data (6041 utterances) compared to other created databases. Six basic or primary emotions ( anger, boredom, disgust, f ear, happiness and sadness ) in addition to a neutral variations (normal, soft, loud, slow and fast) speaking style were recorded. The database contains two emotional speech recording sessions played by two actors (1 female, 1 male). The speakers had to portray the different emotions as shown in Figure 15.

Methodology, Results and analysis

For our experiments we used the two databases (Berlin and Spanish) described in section 7.1. All classification results are obtained under 10-fold cross-validation. Cross-validation is a common practice used in performance analysis that randomly partitions the data into N complementary subsets, with N -1 of them used for training in each validation and the remaining one used for testing. The system described in section 2 (Figure 1) was implemented with the different features presented in section 4. The impact of these features for the SER system accuracy is demonstrated in this section. Several number of IMFs are used in our work with regard to determine the optimal number of IMFs for reconstructing the signal without any loss of important information for each database. We have seen that after a certain number of iterations, the IMFs do not provide information. That is why the signal in our work is reconstructed from only the first nine modes (9 IMFs) for Spanish database and the first twelve modes (12 IMFs) for the Berlin database. The difference in the number of IMFs is due to the nature of the languages in the two databases. These IMFs was used after for feature extraction to recognize emotions. The neural network structure used is a simple LSTM. It consists of two consecutive LSTM layers with hyperbolic tangent activation followed by two classification dense layers. More detailed diagrams are shown in our previous work [START_REF] Kerkeni | Speech emotion recognition: Methods and cases study[END_REF]. For computing kernel and related parameters of the SVM classifier, we used an automatic technique of choice (cross validation) with different kernel, epsilon and coefficient c. Features from data are scaled to [-1, 1] before applying classifiers. Scaling features before recognition is important, because when a learning phase is fit on unscaled data, it is possible for large inputs to slow down the learning and convergence, in some cases prevents the used classifier from effectively learning for the classification problem. The effect of speaker normalization (SN) step prior to recognition is investigated, and there are three different SN schemes that are defined in [START_REF] Wu | Recognition of human emotion in speech using modulation spectral features and support vector machines[END_REF]. SN is useful to compensate for the variations due to speaker diversity rather than change of emotional state. We used in this section the SN scheme that has given the best results in [START_REF] Wu | Recognition of human emotion in speech using modulation spectral features and support vector machines[END_REF]. The features of each speaker are normalized with a zero mean and a standard deviation of 1. It is well known that feature selection has a huge influence on SER systems [START_REF] Liu | Speech emotion recognition based on feature selection and extreme learning machine decision tree[END_REF]. A good set of features can facilitate and improve model learning. Therefore, here, we assess the impact of feature selection on our proposed system. Table 1 gives a description of the number of the features extracted corresponding to each feature extraction label for both databases. The difference in the number of features (MS, MFF) for two databases is related to the number of IMFs (12 and 9 IMFs respectively for Berlin and Spanish database). The detailed numerical results were illustrated in Table 2 to Table 8. Tables 2 and3 present the recognition rates of different features for both databases. These experiments using feature set without feature selection. As shown in Table 2, SVM classifier yields better results above 76.28 % with SMFCC feature for Berlin database. We obtain the same results obtained in [START_REF] Li | Speech emotion recognition using novel hht-teo based features[END_REF]. These results improved in our work by speaker normalization and 81.13 % accuracy is achieved. From Table 1, it can be also concluded that applying SN improves recognition results for Berlin database. But this is not the case for the Spanish database, as demonstrated in Table 3. The same results are obtained with both classifiers. This can be explained by the number of speakers in each database, where the Berlin database contains 10 different speakers, compared to the Spanish database that contains only two speakers. The combination of different features extracted using EMD-TKEO method were performed in this group of comparative experiments. We present only the best combinations in Tables 4 and5. We can see from Table 4 that the combination of SMFCC, MFF and MS features has better performance 85.63 % on the Berlin database. Furthermore, the combination of SMFCC, MFF, MS and EFCC is found to provide the best discrimination 90.72 % for the Spanish database (Table 5). It should also be noted that the Berlin database has limited content compared to the Spanish database, which limits the generalization of the obtained results. To investigate whether a smaller feature space leads to better recognition performance, we repeated all evaluations on the development set by applying a Recursive Feature Elimination (LR-RFE) for each modality combination. The stability of RFE depends heavily on the type of model that is used for feature ranking at each iteration. In our case, we tested the RFE based on an SVM and a regression models. We found that linear regression provides more stable results. The numeric results of applying LR-RFE feature selection technique are summarized in Table 6 with RNN and SVM employed for classification on both databases. We observed that SMFCC reported a best recognition rate of 83.20 %, which is about 5 percentages higher than the Fourier Transform based feature MFCC [START_REF] Wu | Automatic speech emotion recognition using modulation spectral features[END_REF]. The combination of all features using LR-RFE feature selection gives the best results. The corresponding results of LR-RFE can be seen in Table 7. For the Spanish database, LR-RFE does not significantly improve the average accuracy. However, for recognition based on Berlin database, LR-RFE leads to a remarkable performance gain. This increase the average from 77.09 % to 85.94 % for RNN classifier. For the Spanish database, combination of all features after applying LR-RFE selection using RNN has the best recognition rate which is above 91.16%. The best accuracy achieved for Berlin database is 86.22 % using SVM classifier. In this paper, we presented a speech emotion recognition (SER) system using two machine learning algorithms to classify seven emotions. In fact, we study how features impact recognition accuracy of emotions in speech. This work presents a novel features named modulation spectral (MS) features and modulation frequency features (MFF) based on AM-FM modulation model. These features are also combined with others. This work uses a nonlinear and non-stationary analysis based on the EMD technique. The combination of EMD and Teager-Kaiser Energy Operator (TKEO) show an important role for time-frequency analysis and particularly for AM-FM signal demodulation. Experimental study using Berlin and Spanish databases are carried out to evaluate the effectiveness of the proposed system. After feature extraction, we used a RFE method to select the best couple of features from the two databases and to improve the recognition rate. The conducted experiments clearly highlight the impact of the selected features in a classification approach. SER reported the best recognition rate of 91.16 % using RNN classifier on the Spanish database, and 86.22 % using SVM classifier on the Berlin database. From this result we can see that RNN often perform better with more data and it suffers from the problem of very long training times. Therefore, we concluded that the SVM model has a good potential for practical usage in comparison with RNN. In the future work, a speech emotion recognition system that combines multi-classifier will be investigated. We aim also to use other feature selection methods because the quality of the feature selection affects the emotion recognition rate: a good emotion feature selection method can select features reflecting emotion state quickly. We project also to improve the performance of our system by analyzing the correlation between set of features and emotions, because we have observed that for a specific emotion some features are more efficient than others. The overall aim of our work is to develop a system that will be used in a pedagogical interaction in classrooms, in order to help the teacher to orchestrate his class. For achieving this goal, we aim to test the system proposed in this work.

Figure 1 :

 1 Figure 1: Block diagram of the proposed system.

Figure 2 :

 2 Figure 2: The block diagram of the smoothed energy operator.

Figure 3 :

 3 Figure 3: Schema of SMFCC extraction.

Figure 4 :

 4 Figure 4: Schema of ECC and EFCC extraction.

Figure 5 :

 5 Figure 5: Schema of MS features extraction.

Figure 7 :

 7 Figure 7: Estimated Probability Density Function of Φ 3 for three basic emotions (sadness, neutral and anger).

Figure 8 :

 8 Figure 8: Schema of MFF extraction.

Figure 9 :

 9 Figure 9: (9a) Signal of a "anger" speech file; (9b) The time-frequency representation of (9a) using 12 IMFs. Framesize of 25 ms with framshift of 10 ms is used.

Figure 10 :

 10 Figure 10: (10a) Signal of a "neutral" speech file; (10b) The time-frequency representation of (10a) using 12 IMFs. Framesize of 25 ms with framshift of 10 ms is used.

Figure 11 :

 11 Figure 11: Estimated Mean Frequency from all speakers

Figure 12 :

 12 Figure 12: Estimated Probability Density Function of Mean Frequency for three basic emotions (sadness, neutral and anger).

Figure 13 :

 13 Figure 13: A basic concept of RNN and unfolding in time of the computation involved in its forward computation [6].

Figure 14 :

 14 Figure 14: Emotions percentage distribution for the Berlin database.

Figure 15 :

 15 Figure 15: Emotions percentage distribution for the Spanish database.

Table 1 :

 1 Description of the number of features extracted.

	Label feature extraction	Number of features
		Berlin DB Spanish DB
	MS	132	72
	MFF	96	72
	ECC	60	60
	EFCC	60	60
	SMFCC	60	60

Table 2 :

 2 Recognition rates for different features on Berlin database; "AVG." denotes average recognition rate; σ denotes standard deviation of the 10-cross-validation accuracies.

		Berlin (a:fear, e:disgust, f:happiness, l:boredom, n:neutral, t:sadness, w:anger)
						Recognition Rate (%)
	Test Feature Method SN A	E	F	L	N	T	W	AVG. (σ)
	MS		62.50 30.00 32.60 66.80 57.20 77.10 77.70 61.04 (7.40)
	MFF		42.60 36.50 48.50 32.00 51.00 66.90 58.80 53.68 (9.44)
	#1 ECC	RNN	No 52.50 43.00 42.60 32.50 48.40 54.60 66.30 51.73 (4.51)
	EFCC		44.00 51.50 39.80 34.70 46.60 61.20 74.90 51.60 (6.83)
	SMFCC		73.50 62.00 52.40 51.90 59.50 83.70 82.80 68.12(6.15)
	MS		61.95 71.22 43.66 86.32 70.38 83.28 86.08 69.38 (6.18)
	MFF		67.46 64.64 55.28 60.08 57.24 86.88 75.88 66.23 (5.85)
	#2 ECC	SVM	No 68.90 28.45 65.30 67.60 56.43 71.23 82.34 64.18 (6.31)
	EFCC		75.36 64.72 56.46 59.63 58.15 67.35 85.84 63.78 (6.00)
	SMFCC		76.60 67.91 69.71 77.24 75.30 89.84 90.02 76.28(3.53)
	MS		53.30 39.00 40.90 73.00 72.10 85.30 67.20 64.04 (6.40)
	MFF		45.30 65.00 60.40 56.20 52.4 85.40 78.70 65.29 (8.57)
	#3 ECC	RNN	Yes 59.50 43.50 55.30 43.10 50.00 70.60 74.60 58.37 (7.65)
	EFCC		53.80 52.50 46.10 43.30 51.60 69.50 72.40 57.11 (5.73)
	SMFCC		73.70 69.00 69.10 74.20 68.60 91.60 85.10 76.81(4.95)
	MS		67.30 55.71 48.06 81.53 81.88 86.88 88.49 72.41 (4.17)
	MFF		67.65 62.85 64.50 64.54 68.81 90.41 87.51 71.75 (3.91)
	#4 ECC	SVM	Yes 66.43 55.55 67.01 61.93 60.68 68.68 78.55 62.48 (5.48)
	EFCC		68.07 52.36 51.82 68.53 61.12 77.58 79.05 61.91 (5.79)
	SMFCC		86.05 75.54 70.33 80.38 83.79 98.75 88.39 81.13(3.07)

Table 3 :

 3 Recognition rates for different features on Spanish database;Spanish (a:anger, d:disgust, f:fear, j:joy, n:neutral, s:surprise, t: sadness)

	Recognition Rate (%)

Table 4 :

 4 Recognition results using different combinations of features evaluated on the Berlin database

	Method	Feature	SN Average (avg) Standard deviation (σ)
		SMFCC+MFF+MS		73.81	7.28
		SMFCC+MFF+MS+EFCC No	76.50	7.79
	RNN	all features		77.09	7.19
		SMFCC+MFF+MS		83.14	4.62
		SMFCC+MFF+MS+EFCC Yes	82.34	4.91
		all features		82.97	4.19
		SMFCC+MFF+MS		77.75	3.41
		SMFCC+MFF+MS+EFCC No	80.21	4.54
	SVM	all features		80.41	4.84
		SMFCC+MFF+MS		85.63	4.30
		SMFCC+MFF+MS+EFCC Yes	84.69	4.22
		all features		83.38	3.86

Table 5 :

 5 Recognition results using different combinations of features evaluated on the Spanish database

	Method	Feature	SN Average (avg) Standard deviation (σ)
		SMFCC+MFF+MS		90.52	2.39
		SMFCC+MFF+MS+EFCC No	90.72	1.79
	RNN	all features		90.65	1.85
		SMFCC+MFF+MS		88.18	0.47
		SMFCC+MFF+MS+EFCC Yes	87.25	1.20
		all features		87.25	1.11
		SMFCC+MFF+MS		89.98	1.76
		SMFCC+MFF+MS+EFCC No	90.69	0.86
	SVM	all features		90.69	1.30
		SMFCC+MFF+MS		85.61	1.05
		SMFCC+MFF+MS+EFCC Yes	85.67	1.69
		all features		85.77	1.02

Table 6 :

 6 Recognition rates for different features after applying LR-RFE feature selection method (Berlin and Spanish databases

	AVG. (σ)

Table 7 :

 7 Recognition results with combination of all features using RNN and SVM before and after applying LR-RFE feature selection method (Berlin and Spanish databases)

	SN	Classifier LR-RFE	Berlin	Spanish
		RNN	No	77.09 (7.19) 90.65 (1.85)
	No		Yes	85.94 (4.21) 91.16 (1.79)
		SVM	No	80.41 (4.84) 90.69 (1.30)
			Yes	84.52 (5.61) 90.36 (1.05)
		RNN	No	82.97 (4.19) 87.25 (1.11)
	Yes		Yes	85.94 (4.21) 88.39 (0.83)
		SVM	No	83.38 (3.86) 85.46 (1.02)
			Yes	86.22 (4.27) 85.77 (1.13)

The performance comparison for two Machine Learning paradigms (RNN, SVM) without speaker normalization (SN), for the Berlin database, is shown in Figure 16a. For the Spanish database, the performance comparison using speaker normalization (SN) is shown in Figure 16b. The confusion matrix for the best recognition performance achieved by combination of all features on Spanish database, is shown in Table 8. The left-most column being the true emotions. The column indicates for a class, the number of correct predictions for this class and the number of samples confused with others class. The correct values are organized in a diagonal line from top-left to bottom-right of the matrix. For example, the total number of samples for emotion "Fear" in the data set is the sum of the values on the "Fear" column. Table 8 show that anger is most often confused with joy and the neutral emotion is most often confused with the anger emotional state.


Keoma Brun-Laguna
email: keoma.brun@inria.fr

Pascale Minet
email: pascale.minet@inria.fr

Yasuyuki Tanaka
email: 3yasuyuki.tanaka@inria.fr

Optimized Scheduling for Time-Critical Industrial IoT

Keywords: IIoT, Industrial IoT, TSCH, Latency, Reliability, Scheduling, Network Lifetime, IEEE802.15.4e, 6TiSCH

come   L'archive ouverte pluridisciplinaire

I. INTRODUCTION

The Internet of Things (IoT) is a technology concept that is easy to deploy and brings sustainability and economic growth. It is a major component of the Industry 4.0 [START_REF] Hermann | Design principles for industrie 4.0 scenarios[END_REF], where machines are capable of sensing their physical environment, taking decentralized decisions and collaborating with other devices or users. Periodic monitoring applications are the most common, where data are periodically collected from sensors. But current solutions do not provide guarantees on latency and are thus not suitable to address applications that require time-critical data delivery. We need to know how far a solution is from the optimal. Does-it worth trying to improve the network performance? What are the trade-offs? Those are the main issues addressed in this paper.

This paper is organized as follows. In Section II, we give a brief state of the art about transmission scheduling in a Time Slotted Channel Hopping (TSCH) network specified in the latest IEEE 802.15.4 standard [START_REF]IEEE Standard for Low-Rate Wireless Networks[END_REF]. In Section III, we define the two scheduling problems addressed and describe the approach proposed for comparative performance evaluation. Section IV provides theoretical bounds for latency and network lifetime, whereas results of the performance evaluation are reported in Section V both in a canonical case and in a more realistic case with unreliable links. Finally, Section VI concludes the paper.

II. RELATED WORK

In this paper, the chosen MAC layer for the multi-hop wireless mesh network supporting time-critical applications is the IEEE802.15.4-2015 standard [START_REF]IEEE Standard for Low-Rate Wireless Networks[END_REF] with the TSCH mode. Using the TSCH mode allows both low duty cycle (<1%) and high reliability (>99.999%). In TSCH, the scheduling consists in allocating cells to nodes, where a cell is defined by a pair (slot offset, channel offset). The more cells a node has, the more opportunities it has to transmit or receive. Although scheduling has a strong impact on latency, reliability and energy consumption, it is not defined in IEEE802.15.4-2015.

A. Scheduling algorithms

In their survey [START_REF] Hermeto | Scheduling for IEEE802.15.4-TSCH and slow channel hopping MAC in low power industrial wireless networks: A survey[END_REF], Teles Hermeto et al. classify the scheduling algorithms according to their paradigms (i.e. centralized or distributed) and optimization goal (e.g. low latency or reliability). They point out that centralized schedules are ideal for static topologies with periodic traffic, whereas distributed scheduling is more suited for mobile topologies and traffic that is not determined in advance.

The IETF 6TiSCH working group defined: 1) Neighbor-to-Neighbor Scheduling: each node exchanges messages with its neighbors to allocate/deallocate cells between each other; 2) Per Flow Scheduling: nodes allocate cells along a path from the flow source to its destination; 3) Id-based Scheduling: nodes decide which cells to allocate according to an identifier. This identifier is either this from the sender or from the receiver. When two nodes want to communicate, they choose one of the two identifiers and translate it into cell coordinates (i.e. slot offset and channel offset). How to translate a unique identifier into cell coordinates is not defined. The only constraint is to make sure the mapping to a cell coordinate is unique. This approach presented in Orchestra [START_REF] Duquennoy | Orchestra: Robust Mesh Networks Through Autonomously Scheduled TSCH[END_REF] is now being merged in the 6TiSCH Minimal Scheduling Function (MSF). It does not require any wireless communication and negotiation, but may lead to scheduling collisions.

One pioneer centralized scheduling algorithm over TSCH is Traffic Aware Scheduling Algorithm (TASA) [START_REF] Palattella | Traffic Aware Scheduling Algorithm for Reliable Low-Power Multi-Hop IEEE[END_REF]. TASA is built for periodic data gathering. TASA starts to allocate cells to the most loaded node and uses matching and coloring heuristics. TASA guarantees a schedule of minimal length but does not provide any guarantees in terms of reliability, mainly because it assumes perfect links and no retransmissions.

Gaillard et al. extended TASA by adding extra cells for retransmissions. They also proposed Kausa, a KPI-aware scheduling function [START_REF] Gaillard | Kausa: KPI-aware Scheduling Algorithm for Multi-flow in Multi-hop IoT Networks[END_REF] that guarantees per flow QoS, even if several applications share the same network.

Khoufi et al. [START_REF] Khoufi | Scheduling transmissions with latency constraints in an IEEE 802.15.4e TSCH network[END_REF] propose a debt-based scheduler where the node with the highest amount of traffic to transmit is scheduled first. We reuse this concept in Section III-A, taking retransmissions into account.

B. The Scheduling Function in the 6TiSCH stack

The IETF 6TiSCH working group is standardizing Minimal Scheduling Function (MSF) [START_REF] Chang | 6TiSCH Minimal Scheduling Function (MSF)[END_REF], which operates with the 6TiSCH minimal configuration defined by RFC 8180 [START_REF] Vilajosana | Minimal IPv6 over the TSCH Mode of IEEE 802.15.4e (6TiSCH) Configuration[END_REF]. MSF uses a combination of Neighbor-to-neighbor Scheduling and Id-base scheduling that may lead to scheduling collisions. MSF defines how a node joining the network behaves and how the the distributed schedule adapts to application and routing changes as well as scheduling collisions. More generally, any Scheduling Function (SF) in 6TiSCH decides which cells to allocate/deallocate locally, and then triggers a 6top Protocol (6P) Transaction, which is a series of messages a node and its neighbor exchange to negotiate schedule modification. If both nodes agree, they modify their schedule.

C. Evaluation of end-to-end latency and network lifetime

The energy consumption is directly linked to the number of cells used, because the main source of consumption is the radio. The latency depends on how the schedule is built. As the schedule is typically done as a function of the amount of data to transfer and link quality (how many retransmissions needed), if those parameters are fixed and known in advance, we can estimate latency (as done in Khoufi et al. [START_REF] Khoufi | Scheduling transmissions with latency constraints in an IEEE 802.15.4e TSCH network[END_REF]) and energy consumption (as done in Vilajosana et al. [START_REF] Vilajosana | A Realistic Energy Consumption Model for TSCH Networks[END_REF]).

Several studies report latency results for multi-hop TSCH networks. However, there is still a lack of understanding about how TSCH behaves in different environments and what are the trade-offs between reliability, latency and network lifetime.

III. SCHEDULING PROBLEM AND APPROACH PROPOSED

We first define the two addressed scheduling problems, and then show how to compare scheduling algorithms.

A. Scheduling problem statement 1) IIoT Application:

We focus on data gathering applications where data generated at a sampling rate s i by sensor nodes, denoted as N i with i = 1 • • • n, are transmitted, through several hops if needed, to a sink denoted as N 0 . The set of sensor nodes is denoted as N odes. There is no data aggregation on sensor nodes. There is no fragmentation of data samples: each data sample is transmitted in a single frame. Applications express their requirements with regard to 1) the end-to-end latency required, denoted as L, which is the maximum acceptable time elapsed between data generation by any sensor node and its delivery to the sink; 2) the end-to-end reliability required, denoted as R, which is the minimum delivery probability to the sink of data generated by any sensor node; 3) the required network lifetime, denoted as E, which is the time up to the first failure of a sensor node due to battery exhaustion. The goal is to minimize the energy consumption of each sensor node, while providing the services requested by the application.

2) Radio environment: Sensor nodes are deployed in a given environment characterized by the deployment area, sensor node density (i.e. average number of neighbor nodes per sensor node) and radio propagation features (multipath fading, external interference, link quality and their dynamics) [START_REF] Brun-Laguna | Moving Beyond Testbeds? Lessons (We) Learned about Connectivity[END_REF]. Each wireless link l, has an intrinsic reliability evaluated by the probability P l of a successful transmission over this link. Each sensor node has a single radio interface.

The routing protocol, RPL, is in charge of maintaining a routing tree rooted at node N 0 (i.e. the sink) and covering each sensor node N i , with

1 = 1 • • • n.
3) Optimal scheduling problems: In the following, we only consider schedules that are periodic, traffic-aware, valid and conflict-free, do not provide spatial reuse and schedule the maximum number of transmissions per message over a given link and for a given flow. We define a flow as a set of successive messages having the same source address, destination address and QoS (Quality of Service) parameters (e.g. end-to-end reliability and latency).

A Traffic-aware schedule assigns a number of cells to each sensor node which is sufficient to transmit successfully all its data to its parent in the routing tree. A Valid schedule ensures that no node is involved as transmitter or receiver in more than one transmission in the same slotOffset. A Conflict-free schedule ensures that no node involved as transmitter or receiver in a transmission T 1 , is also one-hop away from a transmitting or receiving node in a transmission T 2 = T 1 scheduled in the same slotOffset and channelOffset. There is No spatial reuse: in any cell there is at most one transmission scheduled. Such schedules are more robust with regard to dynamic radio environment. In addition, the schedule systematically schedules the Maximum number of transmissions per message, per link and per flow in order to reach the targeted end-to-end reliability.

Property 1. All the scheduling algorithms considered in this paper avoid conflict between simultaneous transmissions in the same slotOffset by assigning them different channelOffsets. If the number of simultaneous transmissions in the same slotOffset exceeds the number of available channels, the scheduler tries the next slotOffset.

We define two optimal scheduling problems, which are the two extremes of the trade-off between end-to-end latency and network lifetime.

-The fastest schedule: aims at minimizing the worst case end-to-end latency L, while meeting the requested end-to-end reliability R.

-The longest lifetime schedule: aims at minimizing the energy consumed, while meeting the requested end-to-end latency L and the requested end-to-end reliability R.

The choice between these two scheduling problems is application-dependent. Some applications prefer a short latency even at the cost of a shorter lifetime, whereas others prefer a longer lifetime due to the battery-replacement cost.

B. The approach proposed 1) A per-flow approach: Since the requirements expressed by the IIoT application may be specified per type of flows (e.g. event-triggered or periodic), even per flow, we adopt a per-flow approach: first when we compute the maximum number of retransmissions per link and per flow, and then when we schedule all the transmissions needed to deliver a message to the sink. Without loss of generality, we assume that each message is labeled with its flow tag.

2) Local selection of the message to transmit: Each sensor node selects for transmission the message with the highest priority. We distinguish two levels of prioritization: -inter-flow prioritization: the message belonging to the flow having the highest priority is selected first.

-intra-flow prioritization: within a same flow, the oldest message is selected first. All messages are timestamped with the generation time of the oldest data they contain.

3) Canonical case study versus real deployment: Most scheduling algorithms are evaluated in a perfect radio environment where the reliability of each link is equal to 100% and the routing tree never changes. We call such a case the canonical case. This necessary step is not sufficient to assess the correctness of the tested algorithms and their performance in terms of latency, reliability and energy consumption. That is why, in this paper we first study the canonical case and then we consider unreliable links which may generate message losses. In both cases, we use connectivity traces collected from real network deployments as explained in Section V-A.

4) Estimating Retransmissions: A simple way to determine for any given flow, the maximum number of retransmissions per link taking into account the unreliability of each link, consists in requiring the same targeted reliability for each link, that is R 1/h , if the flow considered has to travel h hops to reach the sink. Let M j i denote the maximum number of transmissions for any message of the flow originated at N i , on the link from node N j to its parent:

1 -R 1/h ≥ (1 -P j ) M j i (1) M j i = Log(1 -R 1/h ) Log(1 -P j ) (2)
Let us consider the routing tree depicted in Figure 1, where the number written under any link i → parent(i) gives the success probability P i of a transmission on this link. The numbers written above links give the maximum number of transmissions per flow for each visited link, needed to reach the targeted end-to-end reliability R = 0.999. These numbers are written in the color of the flow origin.

5) Scheduling algorithms evaluated:

To schedule a flow originated at any sensor node N i , any cascading scheduler assigns the number of cells needed per message successively on each node visited, from N i to the sink. Cascading schedulers differ by the order in which they schedule flows. This order is given by the W eight of nodes generating these flows. The node's W eight can be its load, its depth, the number of transmissions to reach the sink, or its debt.

A Load-based scheduler schedules first the flows of the node with the highest load. The load of a node is computed as the sum of the number of cells needed by the node to send and receive. Figure 2 illustrates the schedule of the flows depicted in Figure 1. The flow is depicted with the color of its origin node. Since Load(B) = 21, Load(C) = 22 Load(D) = 12, Load(E) = 22 and Load(F ) = 9, the flows are scheduled in the order C, E, B, D, F . Since Load-based uses 26 slots, which is the minimum number of slots according to Property 2, it is optimal for this configuration.

A generalized Depth-based scheduler schedules first the flows of the node requiring the highest number of transmissions to transmit a single frame originated at this node up to the sink. It can be considered as a generalization of the depth-based algorithm to an unreliable environment. As depicted in Figure 3, the generalized Depth-based schedules flows in the order D, F , C, E and B, leading to 30 slots, which is 4 slots more than Load-based. Hence, this scheduler is not optimal, this contradicts an intuitive idea. By scheduling first the flow requiring the highest total number of slots, the other flows tend to be assigned later slots on the loaded nodes, leading to a larger schedule size A total Transmission-based scheduler schedules first the flows of the node with the highest total number of transmissions up to the sink, taking into account both local and forwarded traffic. This is a generalization of the previous scheduling. In the example considered, we get T otalT rans = 11, 18, 21, 13 and 16, for B, C, D, E and F , respectively, leading to the same schedule as the generalized Depth-based scheduler. Hence, the total Transmission-based scheduler is not optimal.

A Debt-based scheduler schedules first the flows of the node with the highest debt. The debt of a node is equal to the maximum between the total number of transmissions needed to reach the sink and the load of the node. Intuitively, this schedule schedules first the node with the strongest constraint imposed either by the node load or by the total number of transmissions the network has to perform for the frames visiting this node. Require: the W eight of each sensor node M i node the max. number of transmissions from i to its parent, for each msg originated from node Ensure: Allocate cascading cells to flows, starting with the flow originated at the largest W eight node N odes ← sensor nodes set ordered by decreasing W eight for node ∈ N odes do % Schedule each flow originated at this node for each f low originated at node do % Consider the flows by decreasing priority order % N odeOf f set = last slot used by flow on its origin N odeOf f set ← 0 for each message of f low do % Assign cascading cells for message slotOf f set ← N odeOf f set child ← node while child! = sink do M child node transmissions are required per message for each requiredTransmission do parent ← parent(child) AssignFirstAvailCell(child,parent) ≥ slotOffset slotOf f set ← assignedslotOf f set end for if child is the origin of the flow then %Next message assigned from this slotOffset N odeOf f set ← slotOf f set end if child ← parent end while end for end for end for All these schedulers proceed as described in Algorithm 1. They first compute the W eight of each sensor node. Then, they sort the nodes by decreasing W eight order. Starting with the highest W eight node, for each message generated by this node, these scheduling algorithms allocate the cells the node requires to transfer this message to the sink, taking into account the number of transmissions required for each message. They start with the first available cell on both the considered node and its parent, they proceed in a time (i.e. slot offset) increasing manner. They repeat this for each message generated by the node considered. As a consequence, cells are allocated in a cascading way for each message originated from the node considered, as illustrated in Figure 2 and Figure 3. This processing is repeated for each sensor node, until all sensor nodes are scheduled.

Notice that if several nodes have the same weight, the farthest node from the sink is scheduled first. These flows are scheduled by decreasing priority order. For each flow considered, all the cascade of any message generated by this flow is scheduled before scheduling the next message of this flow.

Cascading schedulers have a complexity in O(nf mh) where n denotes the number of nodes, f the number of flows generated per node, m the number of messages generated per flow and h is the maximum path length. Such schedulers present many advantages. 1) They are able to provide a QoS customized per flow. 2) They are traffic-aware, since each sensor node is assigned a number of cells depending on its traffic (i.e. traffic generated + traffic forwarded). 3) They minimize the buffer size needed on any sensor node, because a message received from a child is promptly transmitted to its parent. 4) They minimize the end-to-end latency, since they minimize the number of time slots separating the time slot of the first message transmission on the origin node of the flow, from the time slot of message delivery to the sink. 5) They maximize the end-to-end reliability, because they avoid conflicts by applying Property 1.

6) Energy consumption: We use the results from the radio chip (LTC5800-IPM) datasheet to determine the charge consumed by the mote in each timeslot. The values are listed in Table I, where 1 C = 1 As. We assume that each node has a 2821.5 mAh capacity (a pair of Energizer L-91 AA batteries). We first introduce some additional scheduling notations: M j i is the number of transmissions made by any sensor node N j to transmit a message originated at node N i . Gen(N i ) denotes the number of messages generated by sensor node N i . Desc(N i ) is the set of descendants of N i in the routing tree, whereas Desc + (N i ) = Desc(N i ) {N i } is the set of descendants of N i including itself.

The Load of any sensor node N is equal to the number of cells needed by N to transmit and receive messages.

Load(N ) = D∈Desc + (N ) M N D × Gen(D) + C=child(N ) D∈Desc + (C) M C D × Gen(D) (3)
Load(sink) = C=child(sink) D∈Desc + (C) M C D × Gen(D) (4)
The total number of transmissions needed to deliver the data generated by the sensor node N to the sink is:

T trans(N ) = Gen(N ) × X∈P ath(N ) M X N (5)
We also define T trans as the total number of transmissions needed to deliver the data from all sensor nodes.

T trans = N ∈N odes T trans(N ) (6)
The schedule of minimum length (i.e. minimum number of slots used) should meet the following constraints: -Constraint C1: it should allow the sink to receive all messages sent to it. The number of slots should be higher than or equal to Load(sink).

-Constraint C2: it should schedule T trans, all the transmissions needed to collect data generated on each sensor node. According to Property 1 and our assumptions, the number of transmissions that can be scheduled is equal to the number of cells available in the schedule. The number of cells is equal to the number of slots multiplied by N umChannels, the number of channels used by the TSCH network. Hence, the number of slots should be higher than or equal to T trans N umChannels .

-Constraint C3: it should allow the last message transmitted by any sensor node to reach the sink. Since the last slot used by any sensor node N is a Tx slot. This slot is higher than or equal to Load(N ). Since, we do not know a priori to which flow this message belongs, it may be any flow visiting N , we have to take into account the weakest condition generated by these flows. Hence, to allow this last message transmitted by N to reach the sink, the number of slots in the schedule should be higher than: M X D [START_REF] Khoufi | Scheduling transmissions with latency constraints in an IEEE 802.15.4e TSCH network[END_REF] Notice that in a real deployed TSCH network, unlike the perfect environment with no retransmission, the most loaded node is not necessarily the sink or one of the first-hop nodes. It can be a node with many retransmissions, and not necessarily with many descendants.

Property 2 tells that no schedule can obtain a lower number of slots than (8), it does not mean that such a schedule exists.

B. Canonical Case Study

To capture the topology of deployed networks, collected traces are injected in the 6TiSCH simulator. In this subsection only, the PDR of radio links is set to 100%. Figure 4 depicts the performance results of the 4 schedulings for 99.9% of the data, and the number in the legend is the maximum value got by each scheduling. For this topology, all scheduling algorithms give a schedule of 49 slots, the minimum size (Property 2). As per Eq. 9, the smallest end-to-end latency is 0.70325s, which is achieved by the Load-based scheduling. Depth-based needs one additional slotframe, because it does not use timestamps for message prioritization (Section III-B2). Links are now unreliable. The 6TiSCH simulator uses the PDR of eack link, computed from traces collected from real deployed networks. Only links with a P DR ≥ 0.5 are used. Figure 5 shows that the unreliability of links degrades network

Fig. 1 :

 1 Fig. 1: Example of a routing tree with the PDR of each link.

Fig. 2 :

 2 Fig. 2: Load-based schedule.

Fig. 3 :

 3 Fig. 3: Generalized Depth-based schedule.

Algorithm 1

 1 In the example considered, we get Debt(B) = 21, Debt(C) = 22, Debt(D) = 21, Debt(E) = 22 and Debt(F ) = 16, leading to the same schedule as the Load-based scheduler. Cascading Scheduler.

N

  Load(N ) = Load(N ) + min D∈Desc + (N )X∈P ath(parent(N ))

Property 2 .

 2 A bound on the minimum slotframe size is:M inLength ≥ M ax{Load(sink), T trans N umChannels , M ax N ∈N odes N Load(N )}

Fig. 4 :

 4 Fig. 4: End-to-end latency in the canonical case.

TABLE I :

 I Charge consumed by a mote per timeslot.

	IV. THEORETICAL BOUNDS
	A. Lower Bound on the number of slots

performances. The schedule size ranges from 288 to 371 slots, which is up to 7.57 times higher than in the canonical case. The Load-based scheduling gives the largest probability to have an end-to-end latency ≤ 5 s. If timestamps are not used to prioritize messages, one additional slotframe is needed wrt Eq. 9, leading to a maximum latency of 6.8585 s, representing an increase of 50%.

VI. CONCLUSION

Time-critical industrial applications have to meet three Key Performance Indicators (KPIs): a maximum acceptable E2E latency, a minimum E2E reliability and a minimum network lifetime. In this paper, we showed how a TSCH network meets such requirements, by means of simulations based on connectivity traces collected from real deployed TSCH networks. We highlighted the best trade-off between E2E latency and network lifetime. We proved that the Load-based scheduling is optimal for many configurations. Simulation results showed that it always gives latencies very close to the optimal. They also highlighted the strong influence of message prioritization on end-to-end latencies at high loads. Link unreliability has a large impact on KPIs: for instance, a schedule size up to 6.57 times higher and end-to-end latencies up to 9.66 higher. We will study how to minimize the total number of transmissions per message to reach the sink with the targeted E2E reliability.

Property 3. With our assumptions, the Load-based scheduler is optimal for any configuration meeting Load(N ) < Load(parent(N )), ∀N at least 2-hop away from the sink, and the most loaded node is 1-hop away from the sink.

Proofs are omitted for space reasons.

B. Latency Upper Bound

With our assumptions given in Section III) and assuming that all flows have the same priority and there is no parent change in the routing tree, the worst data acquisition time is right after the last slot assigned to this flow on its origin node starts. The first opportunity to transmit these data arrives Slotf rameSize -1 slots later. Then, in the worst case, these data need the maximum number of retransmissions per hop to reach the destination. Knowing M inLength the minimum number of slots to schedule the application flows, the smallest maximum end-to-end latency provided to the IIoT application is:

C. Lifetime Bounds

We can evaluate the energy consumption of a node per slotframe by means of energy consumption parameters per slot type given in Table I. Network lifetime can be expressed as:

We can increase the lifetime of a mote by adding Idle slots at the end of the slotframe, provided that the number of messages generated during the new slotframe duration remains equal to that obtained with the current slotframe. This reduces the average energy consumption but also increases latency, which highlights the trade-off latency versus network lifetime. The maximum average consumption of a node is given by:

Where ActiveAvgCharge is the average charge spent during the minimal slotframe of size M inLength, and IdleCharge is the charge spent during one Idle slotframe.

V. PERFORMANCE RESULTS

A. Simulation using traces of deployed networks 1) Trace-based simulation:

Simulation enables rapid performance evaluation of networking protocols. Too simplified radio propagation models often lead to results that are not representative of the real world, whereas complex models require large processing times. Trace-based simulation offers a good trade-off between complexity and real world representativeness. It uses connectivity traces collected from real deployments operating in environments close to this of the application considered.

Connectivity traces consist of the Packet Delivery Ratios (PDRs) measured in an IEEE802.15.4-based low-power wireless network. The PDR of each link is evaluated as the ratio of acknowledged frames over the number of frames sent. PDR is preferred to RSSI, because it is hardware-independent.

Topology changes observed in the real deployment are replayed in the simulator. When the simulation reaches the time of a change in the trace file, the simulation topology is updated (including the neighbor tables and preferred parents). The scheduling task is run every time there is a change in the trace file. The network schedule built is then pushed to the nodes.

2) Simulation parameters: We selected the 6TiSCH simulator [START_REF] Municio | Simulating 6TiSCH networks[END_REF], which is a discrete-event simulator written in Python designed for fast prototyping of 6TiSCH. The simulator uses the parameters given in Table II


of p m targD OwN of p ac targD OwN of p m targD OwN of p ac targD Frequency (kHz) Time (ms)

Who controls the attack transients of recorder? zugustin

  Ernoult. Venoît Fabre Lutherie"zcoustique"Musique. Institut Jean le Rond dTzlembert. Université Pierre et Marie %urie. Paris 8. %NRS. Sorbonne Universités

												zcquisition of musical attacks
	♫ : equiped recorders (Vass. zlto. Soprano. Sopranino)						Rising time 10% 90%
	♫ 3 playersk									300
		♪ Expert 1 (: recordersk 4&ww notes) ♪ Expert 2 (zlto S Sopranok 9ww notes) ♪ Novice (zlto S Sopranok 8ww notes)	No blow	100 200					target pressure	Steady Pressure
	♫ Tasksk Scales S Music			p m	10 0	20	30	40	50	60	70	80	90 100
	PRondeauP
	JD5D Vraun
	Supply Pressure zcD Pressure	w w :ww &ww -2ww w 2ww p m (Pa) p ac (Pa)	wD4	wD:	wD8	wD&	O Time (s)	OD4	OD:	OD8	OD&	4	No Jet No sound	100 0 -100		p ac			target amplitude	Oscillating Jet Steady Soundk frequency. amplitude
		w	wD4	wD:	wD8	wD&	O Time (s)	OD4	OD:	OD8	OD&	4			10	20 Onset	30 10%	40	50 Growth 60	70	80 90%	90 100
													zir supply
						Onset of Oscillation				Growth of Oscillation