Liam Schoneveld

Alice Othmani
email: alice.othmani@u-pec.fr

Hazem Abdelkawy

Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition

Emotional expressions are the behaviors that communicate our emotional state or attitude to others. They are expressed through verbal and non-verbal communication. Complex human behavior can be understood by studying physical features from multiple modalities; mainly facial, vocal and physical gestures. Recently, spontaneous multi-modal emotion recognition has been extensively studied for human behavior analysis. In this paper, we propose a new deep learning-based approach for audio-visual emotion recognition. Our approach leverages recent advances in deep learning like knowledge distillation and high-performing deep architectures. The deep feature representations of the audio and visual modalities are fused based on a model-level fusion strategy. A recurrent neural network is then used to capture the temporal dynamics. Our proposed approach substantially outperforms state-of-the-art approaches in predicting valence on the RECOLA dataset. Moreover, our proposed visual facial expression feature extraction network outperforms state-of-the-art results on the AffectNet and Google Facial Expression Comparison datasets.

Introduction

Darwin concluded through his observations and descriptions of human emotional expressions that emotions adapt to evolution, are biologically innate, and universal across all human and even non-human primates [START_REF] Matsumoto | The handbook of culture and psychology[END_REF]). Formal, systematic research studies have since been realized on the universality of emotions. This work demonstrated: (i) the universality of six basic emotions (anger, disgust, fear, happiness, sadness and surprise) and (ii) the cultural differences in spontaneous emotional expressions [START_REF] Ekman | Universals and cultural differences in the judgments of facial expressions of emotion[END_REF]).

A human's emotion resulting from an interaction with stimuli is referred to as an affect. In psychology, an affect refers to the mental counterparts of internal bodily representations associated with emotions. In fact, humans express affect through facial, vocal or gestural behaviors. The notion of affect is subjective, and in the literature it is represented by two alternative views: the categorical view where affects are represented as discrete states with a wide variety of affective displays and the dimensional view, where we suppose that affects might not be culturally universal and alternatively, should be represented in a continuous arousal-valence space.

Recently, a trend in the scientific community has emerged towards developing new technologies for processing, interpreting or simulating human emotions through Affective Computing or through Artificial Emotional Intelligence. Consequently, a broad range of applications have been developed in Human-Computer Interaction, health informatics and assistive technologies. Initial research on affect recognition focused mainly on unimodal approaches, with speech emotion recognition (SER) and facial expression recognition (FER) [START_REF] Rouast | Deep learning for human affect recognition: Insights and new developments[END_REF]) treated as separate problems. More recently however, work in affective computing has paid more attention to multimodal emotion recognition by developing approaches to multimodal data fusion.

Research on affect recognition has seen considerable progress as the focus has shifted from the study of laboratorycontrolled databases to databases covering real-world scenarios. In traditional emotion recognition databases, subjects posed a particular basic emotion in laboratory-controlled conditions. In more recent databases, videos are obtained from reallife scenarios with in-the-wild environmental conditions and less constrained settings, which exhibit characteristics like illumination variation, noise, occlusion, non-frontal head poses, and so on. Today, automatic emotion recognition of the six basic emotions in acted visual and/or audio expressions can be performed with high accuracy. However, in-the-wild emotion recognition is a more challenging problem due to the fact that spontaneously occurring behavior varies more widely in its audio profile, visual aspects, and timing.

In the present era, deep learning-based approaches are revolutionizing many areas of technology. Automatic emotion recognition likewise can benefit from the effectiveness of deep learning. In this paper, we propose a new approach for audiovisual emotion recognition (AVER). This approach is based on pre-training separate audio and visual deep convolutional neural network (CNN) recognition modules. A fusion module is then trained on the specific audio-visual dataset of interest. The fusion module is trained with the combination of generic emotion recognition features extracted by our pre-trained audio and visual components. The remainder of the paper is organized as follows. Section 2 reviews the literature on AVER and presents the contributions of our paper. Section 3 describes the proposed approach. Section 4 presents the experiments then reports and discusses the results. Finally, Section 5 concludes the paper and suggests future work.

Related Works and paper contributions

Related Work

Multimodal fusion for emotion recognition concerns the family of machine learning approaches that integrate information from multiple modalities in order to predict an outcome measure. Such is usually either a class with a discrete value (e.g., happy vs. sad), or a continuous value (e.g., the level of arousal/valence). Several literature review papers survey existing approaches for multimodal emotion recognition [START_REF] Rouast | Deep learning for human affect recognition: Insights and new developments[END_REF]; BaltruÅ ąaitis et al. [2018]; [START_REF] Zeng | A survey of affect recognition methods: Audio, visual, and spontaneous expressions[END_REF]; [START_REF] Poria | A review of affective computing: From unimodal analysis to multimodal fusion[END_REF]). There are three key aspects to any multimodal fusion approach: (i) which features to extract, (ii) how to fuse the features, and (iii) how to capture the temporal dynamics. Extracted features: several handcrafted features have been designed for AVER. These low-level descriptors concern mainly geometric features like facial landmarks. Meanwhile, commonly-used audio signal features include spectral, cepstral, prosodic, and voice quality features. Recently, deep neural network-based features have become more popular for AVER. These deep learning-based approaches fall into two main categories. In the first, several handcrafted features are extracted from the video and audio signals and then fed to the deep neural network [START_REF] Ringeval | Prediction of asynchronous dimensional emotion ratings from audiovisual and physiological data[END_REF]; [START_REF] He | Multimodal affective dimension prediction using deep bidirectional long short-term memory recurrent neural networks[END_REF]; [START_REF] Othmani | Towards robust deep neural networks for affect and depression recognition from speech[END_REF]; [START_REF] Rejaibi | Mfccbased recurrent neural network for automatic clinical depression recognition and assessment from speech[END_REF]; [START_REF] Muzammel | Audvowelconsnet: A phoneme-level based deep cnn architecture for clinical depression diagnosis[END_REF]). In the second category, raw visual and audio signals are fed to the deep network [START_REF] Tzarakis | Endto-end multimodal emotion recognition using deep neural networks[END_REF]; [START_REF] Tzirakis | End-to-end speech emotion recognition using deep neural networks[END_REF]; [START_REF] Basnet | Estimation of affective dimensions using cnn-based features of audiovisual data[END_REF]). Deep convolutional neural networks (CNNs) have been observed as outperforming other AVER methods [START_REF] Rouast | Deep learning for human affect recognition: Insights and new developments[END_REF]). Multimodal features fusion: An important consideration in multimodal emotion recognition concerns the way in which the audio and visual features are fused together. Four types of strategy are reported in the literature: feature-level fusion, decisionlevel fusion, hybrid fusion and model-level fusion [START_REF] Zhang | Learning affective features with a hybrid deep model for audioâ Ȃ Şvisual emotion recognition[END_REF]; [START_REF] Poria | A review of affective computing: From unimodal analysis to multimodal fusion[END_REF]). Feature-level fusion also called early-fusion concerns approaches where features are immediately integrated after extraction via simple concatenation into a single highdimensional feature vector. Such is the most common strategy for multimodal emotion recognition. Decision-level fusion or late fusion concerns approaches that perform fusion after an independent prediction is made by a separate model for each modality. In the audio-visual case, this typically means taking the predictions from an audio-only model, and the prediction from a visual-only model, and applying an algebraic combination rule of the multiple predicted class labels such as 'min', 'sum', and so on. Score-level fusion is a subfamily of the decision-level family that employs an equally weighted summation of the individual unimodal predictors. Hybrid fusion combines outputs from early fusion and from individual classification scores of each modality. Model-level fusion aims to learn a joint representation of the multiple input modalities by first concatenating the input feature representations, and then passing these through a model that computes a learned, internal representation prior to making its prediction. In this family of approaches, multiple kernel learning [START_REF] Chen | Emotion recognition in the wild with feature fusion and multiple kernel learning[END_REF]), and graphical models [START_REF] Baltruå Ąaitis | Multimodal machine learning: A survey and taxonomy[END_REF][START_REF] Baltruå Ąaitis | Dimensional affect recognition using continuous conditional random fields[END_REF]) have been studied, in addition to neural network-based approaches. Modelling temporal dynamics: audio-visual data represents a dynamic set of signals across both spatial and temporal dimensions. [START_REF] Rouast | Deep learning for human affect recognition: Insights and new developments[END_REF] identify three distinct methods by which deep learning is typically used to model these signals: Spatial feature representations: concerns learning features from individual images or very short image sequences, or from short periods of audio. Temporal feature representations: where sequences of audio or image inputs serve as the model's input. It has been demonstrated that deep neural networks and especially recurrent neural networks are capable of capturing the temporal dynamics of such sequences [START_REF] Kim | Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition[END_REF]). Joint feature representations: in these approaches, the features from unimodal approaches are combined. Once features are extracted from multiple modalities at multiple time points, they are fused using one of strategies of modality fusion [START_REF] Ringeval | Prediction of asynchronous dimensional emotion ratings from audiovisual and physiological data[END_REF]).

Contributions of this work

In this work, a higher-performing deep neural network-based approach for AVER is presented. The proposed model is a fusion of two deep neural networks: (i) a deep CNN model, trained with knowledge distillation, for FER and (ii) a modified and fine-tuned VGGish model for SER. A model-level fusion based approach is employed to fuse the audio and the visual feature representations. To model the temporal dynamics, the spatial and temporal representations are processed using recurrent neural networks. The contributions of this work can be summarized as follows:

• A new high-performing deep neural network-based approach for AudioVisual Emotion Recognition (AVER)

• Learning two independent feature extractors -one for audio and one for face images -that are specialised for emotion recognition, and that could be employed for any downstream audiovisual emotion recognition task or dataset

• Applying knowledge distillation (specifically, selfdistillation), alongside additional unlabeled data for FER

• Learning the spatio-temporal dynamics via a recurrent neural network for AVER.

Proposed multimodal deep CNN architecture

Our proposed multimodal deep CNN architecture is made up of three components:

Visual facial expression embedding network

The first component of our multimodal architecture is a deep convolutional neural network (CNN) for facial expression recognition. The input to this network is a single RGB face image, detected and cropped using Multi-Task Cascaded Convolutional Networks (MTCNN) [START_REF] Zhang | Joint face detection and alignment using multitask cascaded convolutional networks[END_REF]). The output of this network is a compact vector of dimension D face .

We refer to this network as our 'facial expression embedding network', and we train it using knowledge distillation [START_REF] Hinton | Distilling the knowledge in a neural network[END_REF]). Knowledge distillation is a two step process whereby a teacher network is trained on the task of interest, and then a (typically smaller) student network is trained on predictions made by the teacher. Specifically in this work, we leverage the benefits of self-distillation, whereby the student network has the same size as (or at least, is not smaller than) the teacher network. Self-distillation was recently leveraged to achieve state-of-the-art results on the well-known Imagenet classification dataset [START_REF] Xie | Self-training with noisy student improves imagenet classification[END_REF]). It has also been shown theoretically that self-distillation can improve a model's performance via a regularization effect [START_REF] Mobahi | Self-distillation amplifies regularization in hilbert space[END_REF]). We use self-distillation to improve the performance of our facial expression embedding network. The training procedure for this network thus consists of two phases:

1. Training a teacher model: our teacher model is a finetuned FaceNet [START_REF] Schroff | Facenet: A unified embedding for face recognition and clustering[END_REF]), trained simultaneously on two different visual facial expression recognition datasets (Section 3.1.1). 2. Training a student model: a second CNN is additionally trained to mimic the outputs of this fine-tuned FaceNet (Section 3.1.2).

The teacher network

The starting point for our teacher model is a pre-trained FaceNet [START_REF] Schroff | Facenet: A unified embedding for face recognition and clustering[END_REF]). This model is then trained to learn specialised features for facial emotion recognition using two datasets:

• AffectNet [START_REF] Mollahosseini | Affectnet: A database for facial expression, valence, and arousal computing in the wild[END_REF]), which consists of around 440,000 in-the-wild face crop images, each of which is human-annotated into one of eight facial expression categories (Neutral, Happy, Sad, Surprise, Fear, Disgust, Anger and Contempt).

• Google Facial Expression Comparison (FEC) [START_REF] Agarwala | A compact embedding for facial expression similarity[END_REF]), which consists of around 700,000 triplets of unique face crop images. Annotations denoting the most similar pair of face expressions in each triplet are provided. The goal is to train a model that places the similar pair closer together in a learned embedding space.

Our teacher model's architecture (Fig. 1) is almost identical to the model proposed in [START_REF] Agarwala | A compact embedding for facial expression similarity[END_REF], the only difference is that we add an additional output head for the AffectNet loss. A pre-trained FaceNet1 is taken up until the Inception 4e block. This is followed by a 1x1 convolution and a series of five untrained DenseNet [START_REF] Huang | Densely connected convolutional networks[END_REF]) blocks. After this, another 1x1 convolution followed by global average pooling reduces this representation to a single D face dimensional vector. After pooling, two independent linear transformations serve as output heads. These heads take the D facedimensional facial expression representation vector as input and make separate predictions for the AffectNet and FEC tasks. A 32-dimensional embedding is used for the FEC triplets task, while an 8-dimensional output head produces class logits for AffectNet (which has 8 classes). The teacher network training procedure is detailed in Algorithm 1, while implementation details are provided in the supplementary materials.

To improve the regularization effects of self-distillation through model ensembling, we in fact train two teacher networks, and concatenate their outputs to serve as distillation targets (see Section 3.1.2 for details). The only difference between the two teachers are the random seeds used for initialization, and penultimate layer dimensionalities: we use D face = 128 for one teacher network, and D face = 256 for the other.

Student network

Our student network is a DenseNet201 pre-trained on Imagenet. 2 The student network training procedure is essentially the same as described in Algorithm 1, except that we additionally sample batches of unlabeled data from an internal dataset, which we refer to as PowderFaces. The Pow-derFaces dataset was created by downloading approximately 20,000 short, publicly-available videos from various online sources such as YouTube. To increase the frequency of faces in the dataset, specific search terms and topics were used when searching for videos, such as 'podcast', 'interview', or 'monologue'. MTCNN face detection was then applied to the extracted frames from those videos, producing approximately 1 million individual face crops. The sampled batches of face crops from the Google FEC, AffectNet, and Powder-Faces datasets are passed through our two teacher networks. Each of the two teacher networks produces predictions for the Google FEC task (32-dimensional) and AffectNet class logits The audio and the visual embedding vectors are each fed to a small, independent convolutional network. This results in one tensor of size (9, 64) for each modality. We concatenate these two to give a tensor of size (9, 128), which is fed to a two-layer LSTM network with dimensionality of 256. Taking the final time step's output of this LSTM gives a single vector of size 256, which is pass through a single fully-connected layer with two outputs, which after a tanh activation gives our predictions between -1 and 1 for arousal and valance.

(8-dimensional). These four vectors (i.e., two vectors from two teacher networks) are individually L2-normalized The four normalized vectors are then concatenated, producing one long vector of dimension 80. A knowledge distillation loss (we specifically use 'Relational Knowledge Distillation' [START_REF] Park | Relational knowledge distillation[END_REF]) for our loss function) is then calculated by comparing the output of a third output head in the student network to this 80-dimensional target vector. This knowledge distillation loss is then added to the standard AffectNet and Google FEC losses, which are calculated as per the teacher network training procedure. Implementation details for our student network are provided in the supplementary materials.

Audio embedding network for emotion recognition

This section details our proposed deep learning-based approach for recognizing emotions from audio segments. The approach is based on fine-tuning the VGGish model [START_REF] Hershey | Cnn architectures for large-scale audio classification[END_REF]) on the RECOLA dataset [START_REF] Ringeval | Introducing the recola multimodal corpus of remote collaborative and affective interactions[END_REF]).

Audio pre-processing

For each input audio file, a set of Mel-spectrogram representations (R) are created. The input audio files are down-sampled at a 16KHz sampling rate. Then, the short-time Fourier transform (STFT) is performed to create windows with length (l) of 40 milliseconds and a hop length of 40 milliseconds. To create the latter windows, a set of 128 Mel filters (M f ) are applied with a Mel frequency range of 125-7500 Hz. Finally, for each audio file, a tensor of shape [R, l, M f ] is generated to create a compatible pre-processed data input for the VGGish backbone network.

Algorithm [START_REF] Hershey | Cnn architectures for large-scale audio classification[END_REF]). Our starting point is the original VGGish model, pre-trained on the Audio Set dataset [START_REF] Gemmeke | Audio set: An ontology and humanlabeled dataset for audio events[END_REF]). The VGGish backbone consists of 6 convolutional layers that output 64, 128, 256, and 512 feature maps ( f ) respectively. For each convolution layer, a kernel (k) with size 3x3, and stride (s) of 1x1 is used. A max pooling layer with a kernel (k) of size 2x2, and stride (s) 2x2 is then applied. We take this VGGish backbone, but replace its last convolution and max pooling layers with a global average pooling layer. The resulting model produces an output vector of dimension 256. After this, we add three randomly-initialized, fully-connected layers, with output dimensionalities of 4096, 4096, and 128. These layer sizes were chosen to mimic the fully-connected penultimate layers of the original VGGish architecture. The aim of these layers is to extract a standard embedding vector with size of 128 that reflects the emotional characteristics of the input audio segment.

We take this expanded VGGish backbone architecture and fine-tune it end-to-end on the RECOLA dataset. Two separate VGGish networks are fine-tuned: one to predict arousal and the other to predict valence. We pass inputs of size [480,128] to the VGGish model, which are the mel-spectrogram representations of 30 seconds of audio from one of the videos in the RECOLA dataset. The target used for fine tuning is then the average ground truth arousal or valence for the target values corresponding to the input 30 seconds of audio. We predict this target by passing the 128-dimensional audio representation through a fully-connected layer f φ with a tanh activation. The training procedure for fine tuning our audio feature extraction model is detailed in Algorithm 2.

Algorithm 2: VGGish fine-tuning algorithm for predicting arousal. Given the VGGish feature extractor network f Θ , arousal prediction head f φ , number of training steps N.

for iteration in range(N) do (X, y) ← batch of RECOLA spectrograms and targets e ← f Θ (X)

Calculate VGGish embeddings for batch p ← f θ (e) Predict arousal for all elements in batch Loss = -concordance_correlation_coeff(p, y) Obtain all gradients ∆ all = ( ∂Loss ∂Θ , ∂Loss ∂θ ) (Θ, θ) ← Adam(∆ all ) Update VGGish model, output head end

Audio-visual fusion model

A model-level fusion based approach is considered as shown in Fig. 3. Visual features are extracted by taking face crops from the video sequence of interest using MTCNN and passing them to our student network (Section 3.1.2). Audio features are extracted using our fine-tuned VGGish backbone (the temporal granularity of these features is increased by removing the global average pooling over the temporal dimension from our fine-tuned VGGish model). To have the same reduced size, audio and visual features are passed through small, independent pre-transform networks consisting of 1D convolutions, where the convolution filters slide over the temporal dimension. The pre-transform networks are designed to give an output tensor of shape [9,64]. These are concatenated into a single audio-visual features tensor, which is passed to a two-layer LSTM network with hidden and output dimensionality of 256. The final output of the LSTM network is then taken, and passed to a simple linear transform with two outputs. These outputs are passed through a tanh activation to produce the final arousal and valence predictions. Our loss function is the negative CCC. Further implementation details for our fusion network are provided in the supplementary materials.

Experimental results

Datasets

The performances of the proposed approach have been evaluated using the REmote COLlaborative and affective interactions (RECOLA) corpus [START_REF] Ringeval | Introducing the recola multimodal corpus of remote collaborative and affective interactions[END_REF]). In RECOLA, participants' spontaneous interactions were collected while being engaged in a remote discussion that aimed to manipulate their moods. Then, six annotators measured the emotional state present in all sequences continuously on the valence and arousal dimensions. 27 audio-visual recordings of 5 minutes of interaction, which includes 9 videos for training and 9 for validation, are publicly available. In order to perform a fair comparison, the test set annotations (for the last 9 videos) of the AVEC challenge are not given. We use the training videos to train our models, validate on the validation videos, and submitted our results on the test set to the RECOLA dataset managers for evaluation. We also evaluate our visual feature extraction network on held-out evaluation data from the AffectNet and Google FEC datasets, which were introduced in Section 3.1.1.

Evaluation metric

The Concordance Coefficient Correlation (CCC) [START_REF] Lawrence | A concordance correlation coefficient to evaluate reproducibility[END_REF]) is used to evaluate the performance of the proposed approach on RECOLA, as it is standard metric for emotion recognition on the RECOLA dataset. The CCC (Equation 1) measures the agreement between a vector of predicted (Pred) and true (T rue) values for a continuous variable:

CCC(T rue, Pred) = 2 * Corr(T rue, Pred) * σ T rue * σ Pred σ 2 T rue + σ 2 Pred + (µ T rue -µ Pred ) 2
(1) Where µ x is the mean of x, σ x is the standard deviation of x, and Corr(x, y) returns Pearson's correlation coefficient between x and y.

Visual facial expression embedding network performance

We evaluate our visual facial expression embedding network on the standard evaluation subsets of the two datasets it was trained on: 1). 2. Google FEC: following Agarwala and Vemulapalli

[2019], we evaluate using triplet accuracy on the Google FEC test set. Using this metric, we find our approach substantially improves on state-of-the-art on the FEC test set, with an accuracy of 86.5% (Table 2).

To experimentally verify the importance of the different components of our approach, we perform an ablation study. Training the student model architecture without the distillation loss component reveals the importance of distillation. Without distillation, our model's performance dropped substantially: to 58.8% on AffectNet and 85% on Google FEC.

Similarly, to determine the importance of the unlabeled Pow-derFaces dataset, we again train the student model with distillation, but without the additional distillation targets provided by using this unlabeled data. The results suggest that the additional unlabeled data may not be so important to our results: accuracy on AffectNet dropped only slightly to 61.1%, and performance on Google FEC reduced by only 0.1% to 86.4%. We discuss these ablation results further in Section 4.6.

Performance on RECOLA in the visual-only and audioonly context

To test performance on RECOLA of our visual and audio feature extractors separately, we retrain the same fusion architecture, but disable either the audio or visual feature inputs. Visual-only: feeding the embeddings from our visual feature extractor into the visual-only version of our fusion model performs well on the RECOLA dataset (Table 3). Such reaches a CCC of 0.55 for predicting valence and 0.57 for predicting arousal on the validation set, while on the test set our CCC reaches 0.66 for valence and 0.57 for arousal. This result illustrates the robustness of our visual feature extractor: when predicting valence, our method achieves state-of-the-art performance when compared to other multimodal approaches, even though we use only visual features as input. Audio-only: our results (Table 3) show that our modified VG-Gish backbone feature extractor for audio segments performs well, CCCs of 0.52 and 0.70 for valence and arousal, respectively on the RECOLA test set. The achieved results for arousal prediction match the existing state-of-the-art methods when only audio features are used (Table 4). This shows that our approach to transfer learning from the acoustic events domain, and the VGGish architecture, provide a robust means to extracting audio-based features for emotion recognition.

Fusion model performance and comparison with state-ofthe-art approaches

Our multimodal fusion model achieves a CCC of 0.740 in valence prediction, and 0.719 in arousal prediction in RECOLA test set (Table 5). These results reflect the robustness of our learned visual and audio features extraction techniques, and the efficacy of the approach to fusing these modalities and accounting for temporal dynamics. Our proposed method, with a CCC of 0.740, substantially outperforms all existing methods in predicting valence, with the previous best performing approach of [START_REF] Tzarakis | Endto-end multimodal emotion recognition using deep neural networks[END_REF] achieving a CCC of 0.612. Simultaneously, our approach achieves strong results in predicting arousal with a CCC of 0.719, compared to the state-of-the-art performance of [START_REF] Ringeval | Prediction of asynchronous dimensional emotion ratings from audiovisual and physiological data[END_REF], who obtained a CCC of 0.796.

Discussion

The results in Section 4.3 suggest that the unlabeled Powder-Faces dataset only provides a marginal benefit in performance, if any at all. This ran contrary to our expectations: we expected unlabeled data to help in this context, given a similar approach achieved state-of-the-art results on Imagenet [START_REF] Xie | Self-training with noisy student improves imagenet classification[END_REF]). In that work however, the unlabeled dataset consisted of 300 million images -300 times more than the Imagenet dataset itself. In our case, our unlabeled dataset of one million images is only about twice the size of the number of faces in AffectNet and Google FEC combined. Thus, to truly conclude whether the benefits of unlabeled data as shown on Imagenet transfer well to facial expressions, many more unlabeled images are needed. We plan to address this point in future work.

On the other hand, it appears that self-distillation is indeed beneficial in the facial expressions domain. This is illustrated by the marked improvement in our accuracy on the Google FEC dataset when distillation is applied, compared to training without distillation. For AffectNet, the benefits of distillation are less pronounced, and it seems that the combination of using a pre-trained FaceNet model, plus training on Google FEC at the same time as AffectNet are all necessary components to achieving our results.

Finally, our results for valence on the RECOLA test set show a dramatic improvement over existing state-of-the-art approaches (Table 5). As mentioned, our visual-only model for RECOLA also achieved state-of-the-art for valence (Table 3). This provides further evidence for the effectiveness of our visual feature extraction approach.

Conclusion and Future work

This paper introduces a high-performing deep neural network-based approach for AVER that fuses a distilled visual feature extractor network with a modified VGGish backbone and a model-level fusion architecture. The proposed visual facial expression embedding network shows that end-toend training on both AffectNet and FEC in tandem is a highly effective method for learning robust facial expression representations. We have also demonstrated that knowledge distillation can provide further improvements for facial expression recognition. The performance of our modified VGGish backbone feature extractor presents a promising new direction for predicting emotion from audio. Moreover, our deep neural network approach to multimodal fusion has been shown to be effective in AVER, outperforming the state-of-art methods in predicting valence on the RECOLA dataset. For future work, we plan to investigate the best strategy for continuous emotion encoding: classification with coarse categories, regression, label distribution learning or even ranking.

Fig. 1 .

 1 Fig. 1. Our facial expression recognition neural network architecture, before distillation (i.e. the teacher network). Faces are detected and cropped using MTCNN. The resulting 140x140 RGB images are then fed an Inception Resnet V1 until the Inception 4e block. This is followed by a 1x1 convolution layer (1x1 Conv), batch normalization (BN) and a ReLU activation. Then, five DenseNet blocks are applied. Finally, another set of 1x1 Conv, BN and ReLU is applied. The output is then averaged over the spatial dimensions, resulting in a vector of size D face (in the figure, D face = 128). Two separate linear layers then give us the final model outputs -a vector for the Google FEC triplets task, and class logits for AffectNet. The model is trained to minimize both the AffectNet and Google FEC losses simultaneously. The numbers over each block represent the tensor's output shape after applying that block.

Fig. 2 .Fig. 3 .

 23 Fig. 2. Modified VGGish backbone feature extractor for Speech Emotion Recognition. The Mel-Spectogram is computed from the audio signal and then fed to the modified VGGish backbone network consisting of 6 convolutional layers followed by 3 fully connected layers of size (4096, 4096 and 128) to output an embedding vector of size 128. The size of the feature maps (f ) of each convolutional and fully-connected layer are shown above each block of operations. The kernel size (k) and stride (s) are specified below the convolution blocks.

  emotional state or attitude to others. They are expressed through verbal and non-verbal communication. Complex human behavior can be understood by studying physical features from multiple modalities; mainly facial, vocal and physical gestures. Recently, spontaneous multi-modal emotion recognition has been extensively studied for human behavior analysis. In this paper, we propose a new deep learning-based approach for audio-visual emotion recognition. Our approach leverages recent advances in deep learning like knowledge distillation and high-performing deep architectures. The deep feature representations of the audio and visual modalities are fused based on a model-level fusion strategy. A recurrent neural network is then used to capture the temporal dynamics. Our proposed approach substantially outperforms state-of-the-art approaches in predicting valence on the RECOLA dataset. Moreover, our proposed visual facial expression feature extraction network outperforms state-of-the-art results on the Af-fectNet and Google Facial Expression Comparison datasets.

  1: Visual model: training the teacher network. Given feature extractor network f Θ , Google FEC output head g φ , AffectNet output head h θ , number of training steps N, AffectNet loss weight α.

		3.2.2. VGGish backbone network
		Our deep model for audio-based emotion recognition is
		based on a modified version of the VGGish model
	for iteration in range(N) do
	(X FEC , y FEC ) ← batch of Google FEC triplets and
	labels
	(X Aff , y Aff ) ← batch of AffectNet images and class
	labels
	e FEC ← f Θ (X FEC )	Face embeddings for FEC
	images
	e Aff ← f Θ (X Aff )	Face embeddings for AffectNet
	images
	v FEC ← g φ (e FEC )	Predict vectors for triplet loss
	p Aff ← h θ (e Aff )	Predict class probabilities for
	AffectNet
	L FEC = triplet_loss(v FEC , y FEC )
	L Aff = cross_entropy_loss(p Aff , y Aff )
	Total loss for training step Obtain all gradients ∆ all = ( ∂L L = L FEC + α * L Aff ∂Θ , ∂L ∂φ , ∂L ∂θ ) (Θ, φ, θ) ← SGD(∆ all ) Update feature extractor and
	output heads' parameters simultaneously
	end

Table 1 .

 1 Performances of the proposed visual facial expression embedding network on the AffectNet validation set comparing to existing state-of-theart methods AffectNet: for AffectNet, which requires classifying faces into eight discrete facial expression classes, we train a logistic regression model on the features extracted by our student network for the entire AffectNet training set. 3 This method achieves state-of-the-art results on the AffectNet validation set, with an accuracy of 61.6% (Table

	Methods	Accuracy
	Georgescu et al. [2019]	59.6%
	Siqueira et al. [2020]	59.3%
	Ours (Teacher model)	61.3%
	Ours (Student, no distillation)	58.8%
	Ours (Distilled student, no PowderFaces) 61.1%
	Ours (Distilled student)	61.6%
	Table 2. Triplet prediction performances of the proposed visual facial ex-
	pression embedding network on the Google FEC test set comparing to ex-
	isting state-of-the-art methods
	Methods	Accuracy
	Agarwala and Vemulapalli [2019]	81.8%
	Ours (Teacher model)	84.5%
	Ours (Student, no distillation)	85.0%
	Ours (Distilled student, no PowderFaces) 86.4%
	Ours (Distilled student)	86.5%
	1.

Table 3 .

 3 RECOLA dataset results (in terms of CCC) for predicting arousal and valence on train, development and test sets.

		Valence			Arousal
	CCC	Train	Dev Test Train	Dev Test
	Visual only	.6	.55	.66	.49	.57	.57
	Audio only	.55	.46	.52	.78	.80	.70
	Audio-visual .69	.63	.74	.78	.81	.72

Table 4 .

 4 Performances of the proposed audio embedding network on the RECOLA dataset comparing to existing state-of-the-art methods. In parenthesis are the performances obtained in the development set. --: no results reported in the original papers.

	Methods	Arousal	Valence
	Tzarakis et al. [2017] .70 (.75)	.31 (.41)
	Han et al. [2017]	.67 (.76)	.36 (.48)
	He et al. [2015]	--(.80) --(.40)
	Ours	.70 (.80)	.52 (.46)

Table 5 .

 5 RECOLA dataset results (in terms of CCC) for predicting arousal and valence. S.M.: Strength modeling of SVR + BLSM

	Methods	Audio features	Visual features Modality fusion	Arousal Valence
	Ringeval et al. [2015] LLDs + BLSTM	LLDs	Feature-level	.761	.492
	Ringeval et al. [2015] LLDs + BLSTM	LLDs	Decision-level	.796	.501
	He et al. [2015]	LLDs	LLDs	model-based (BLSTM)	.747	.609
	Han et al. [2017]	LDDs+S.M.	Geom. + S.M.	modality and model-based	.685	.554
	Tzarakis et al. [2017] 1D CNN	ResNet50	Model-level (2 LSTM)	.714	.612
	Proposed	Fine-tuned VGGish Distilled CNN	Model-based (LSTM)	.719	.740

Emotional expressions are the behaviors that communicate our

In this, we used a FaceNet pre-trained on the VGGFace2 dataset, as we found performance to be slightly improved. The pre-trained FaceNet model architecture and weights were obtained from https://github.com/timesler/facenet-

pytorch.2 We use the implementation and pre-trained Imagenet weights provided in the torchvision Python package.

When training this logistic regression, we re-weight the classes in the Af-fectNet training set to have equal representation, as per the validation set.

Acknowledgements

This work was supported by Powder, a deep tech startup. Powder is a video editing and sharing platform for gamers. https://powder.gg/


Emilie Salvia
email: emilie.salvia@parisdescartes

Ania Aïte

Julie Vidal

Grégoire Borst

Hot and cool response inhibition abilities develop linearly from late childhood to young adulthood

Keywords: Response inhibition, Stop signal task, Cool, Hot, Development

Inhibitory control involves different types of inhibition processes, such as interference control and response inhibition. In an affectively neutral context, both inhibition processes develop linearly with age. In an affectively charged context, however, the development of these inhibition processes is still debated, and no study has investigated the development of the ability to inhibit an initiated motor response, such as in a stop signal task (SST), from late childhood to young adulthood. We asked children, adolescents and young adults to perform a cool or a hot version of the SST using neutral or emotional nonverbal vocalizations as stop signals. We showed that cool and hot response inhibition abilities develop linearly from late childhood to young adulthood. The development of this ability in an affectively charged context appears to be more protracted than in a neutral one. Exerting response inhibition in an affectively charged context is more challenging than in a neutral context especially during adolescence.

2

HIGHLIGHTS

• Hot and cool response inhibition develop linearly from late childhood to young adulthood.

• Hot response inhibition development appears to be more protracted than cool one.

• Exerting response inhibition in an affectively charged context is more challenging.

• Hot response inhibition is less efficient than cool inhibition in adolescents.

INTRODUCTION

As humans living in complex societies, we rely on a number of automatisms to efficiently navigate our physical and social environments. In some contexts, we need to resist such automatisms to avoid negative outcomes. For instance, during a pandemic crisis, such as the COVID-19 crisis, people need to resist automatic social behaviors (e.g., hugging, shaking hands, or kissing) and adopt new behaviors (e.g., physical distancing) to control the spread of the virus. Adopting such behaviors typically requires withholding a prepotent response in an affectively charged context. Resisting such automatisms as well as temptations, distractions or interferences and adapting to conflicting situations relies on inhibitory control (IC, [START_REF] Diamond | Executive Functions[END_REF][START_REF] Diamond | Conclusions about interventions, programs, and approaches for improving executive functions that appear justified and those that, despite much hype, do not[END_REF], which develops differently in affectively neutral and charged contexts from childhood to adulthood (e.g., [START_REF] References Aïte | Adolescents' inhibitory control: Keep it cool or lose control[END_REF][START_REF] Tottenham | Behavioral Assessment of Emotion Discrimination, Emotion Regulation, and Cognitive Control in Childhood, Adolescence, and Adulthood[END_REF]. In the present study, we investigated for the first time the development of the ability to inhibit an initiated motor response, such as when we refrain from hugging someone after we have started moving toward him/her, in an affectively neutral or charged context from late childhood to young adulthood.

IC is not a unitary construct but rather involves different types of inhibition processes, such as interference control and response inhibition, depending on the activity that needs to be inhibited [START_REF] Eagle | The neuropsychopharmacology of action inhibition: Cross-species translation of the stop-signal and go/no-go tasks[END_REF][START_REF] Khng | The Relationship between Stroop and Stop-Signal Measures of Inhibition in Adolescents: Influences from Variations in Context and Measure Estimation[END_REF][START_REF] Rubia | Mapping Motor Inhibition: Conjunctive Brain Activations across Different Versions of Go/No-Go and Stop Tasks[END_REF][START_REF] Scheres | Executive functioning in boys with ADHD: Primarily an inhibition deficit[END_REF].

Interference control is a set of processes that allows the selective attendance to task-relevant information by inhibiting task-irrelevant information [START_REF] Diamond | Executive Functions[END_REF]. For instance, in the Color-Word Stroop task (Stroop, 1935), in which participants must identify the ink colors of printed color words (e.g., BLUE written in red), individuals typically need to inhibit the color denoted by the word (i.e., blue, the task-irrelevant information) to identify the ink color of the printed word (i.e., red, the task-relevant information). Response inhibition is a set of processes that allows withholding a prepotent motor response in response to a stop signal. In the Go/NoGo task, participants press a button in response to a given stimulus and have to refrain from doing so when a stop signal is presented simultaneously with (or instead of) the go signal (e.g., [START_REF] Donders | On the speed of mental processes[END_REF]. The Go/NoGo task typically assesses the ability to inhibit a reinforced motor response. In the stop signal task (SST), participants make simple decisions in response to the presentation of a stimulus but must withhold their motor response when a stop signal is presented with various delays after a go signal [START_REF] Logan | On the ability to inhibit thought and action: A users' guide to the stop signal paradigm[END_REF][START_REF] Logan | On the ability to inhibit thought and action: a theory of an act of control[END_REF]. The SST assesses the ability to inhibit an initiated motor response. Importantly, these two types of response inhibition appear to be relatively independent of one another [START_REF] Eagle | The neuropsychopharmacology of action inhibition: Cross-species translation of the stop-signal and go/no-go tasks[END_REF][START_REF] Rubia | Mapping Motor Inhibition: Conjunctive Brain Activations across Different Versions of Go/No-Go and Stop Tasks[END_REF][START_REF] Scheres | Executive functioning in boys with ADHD: Primarily an inhibition deficit[END_REF].

In an affectively neutral context, there is a general consensus that interference control [START_REF] References Aïte | Adolescents' inhibitory control: Keep it cool or lose control[END_REF] and response inhibition (Go/NoGo task: [START_REF] Hare | Biological Substrates of Emotional Reactivity and Regulation in Adolescence During an Emotional Go-Nogo Task[END_REF][START_REF] Schel | Development of response inhibition in the context of relevant versus irrelevant emotions[END_REF][START_REF] Somerville | Frontostriatal Maturation Predicts Cognitive Control Failure to Appetitive Cues in Adolescents[END_REF][START_REF] Tottenham | Behavioral Assessment of Emotion Discrimination, Emotion Regulation, and Cognitive Control in Childhood, Adolescence, and Adulthood[END_REF]SST: Bedard et al., 2002;[START_REF] Carver | Age Related Changes in Inhibitory Control as Measured by Stop Signal Task Performance[END_REF][START_REF] Tillman | Motor Response Inhibition and Execution in the Stop-Signal Task: Development and Relation to ADHD Behaviors[END_REF][START_REF] Urben | Emotional Modulation of the Ability to Inhibit a Prepotent Response During Childhood[END_REF][START_REF] Williams | Development of inhibitory control across the life span[END_REF]) abilities develop linearly with age. In an affectively charged context, the development of interference control and response inhibition is still debated (see [START_REF] Schweizer | The role of affective control in emotion regulation during adolescence[END_REF].

Neurodevelopmental imbalance models of adolescence hypothesize an adolescent-specific developmental pattern of IC abilities in an affectively charged context due to an imbalance between a hyperactive emotional subcortical network and an immature prefrontal network during this period of life [START_REF] Casey | Beyond simple models of self-control to circuit-based accounts of adolescent behavior[END_REF][START_REF] Casey | The teenage brain: self-control[END_REF]. In agreement with this model, studies have reported quadratic developmental patterns for both interference control and response inhibition abilities. For instance, Aïte et al. ( 2018) revealed a quadratic effect of age on performance in an affectively charged version of the Stroop task, with adolescents displaying worse interference control abilities than children and adults. Similarly, some studies have reported a quadratic developmental pattern for response inhibition abilities in an affectively charged Go/NoGo task associated with stronger ventral striatum activity in adolescents than in children and adults and a linear decrease in inferior frontal gyrus activity with age [START_REF] Hare | Biological Substrates of Emotional Reactivity and Regulation in Adolescence During an Emotional Go-Nogo Task[END_REF][START_REF] Somerville | Frontostriatal Maturation Predicts Cognitive Control Failure to Appetitive Cues in Adolescents[END_REF]. However, other studies have reported a linear developmental pattern for this ability using the same task with a protracted development of response inhibition in an affectively charged context [START_REF] Schel | Development of response inhibition in the context of relevant versus irrelevant emotions[END_REF][START_REF] Tottenham | Behavioral Assessment of Emotion Discrimination, Emotion Regulation, and Cognitive Control in Childhood, Adolescence, and Adulthood[END_REF]. This finding is in line with another model of the development of executive functions in affectively neutral or charged contexts that assumes a later development of executive functions and of IC in particular in an affectively charged than in an affectively neutral context [START_REF] Zelazo | Hot and Cool Executive Function in Childhood and Adolescence: Development and Plasticity[END_REF]. Regarding the ability to inhibit an initiated motor response in an affectively charged context, several studies on adults using the SST provided evidence that response inhibition was impaired [START_REF] Herbert | Response inhibition and memory retrieval of emotional target words: evidence from an emotional stop-signal task[END_REF][START_REF] Kalanthroff | Stop feeling: Inhibition of emotional interference following stop-signal trials[END_REF][START_REF] Krypotos | Individual differences in heart rate variability predict the degree of slowing during response inhibition and initiation in the presence of emotional stimuli[END_REF][START_REF] Rebetez | Do emotional stimuli interfere with two distinct components of inhibition?[END_REF][START_REF] Verbruggen | Do emotional stimuli interfere with response inhibition? Evidence from the stop signal paradigm[END_REF], improved [START_REF] Pessoa | Interactions between cognition and emotion during response inhibition[END_REF] or was unaffected (Sagaspe, Schwartz, & Vuillemier, 2011) in an affectively charged context. To date, only one study using the SST investigated the development of the ability to inhibit an initiated motor response in an affectively neutral or charged context [START_REF] Urben | Emotional Modulation of the Ability to Inhibit a Prepotent Response During Childhood[END_REF]. [START_REF] Urben | Emotional Modulation of the Ability to Inhibit a Prepotent Response During Childhood[END_REF] reported a linear development of both cool and hot response inhibition abilities from childhood (i.e., 6 years of age) to early adolescence (i.e., 13 years of age).

However, no study to date has investigated the development of the ability to inhibit an initiated motor response in an affectively charged context from late childhood to adolescence and young adulthood. To do so, we asked children (9 -11 years old), adolescents (13 -16 years old) and young adults (18 -30 years old) to perform an affectively neutral (hereafter referred to as "cool") and charged (hereafter referred to as "hot") version of the SST. In the Go trials of the SST, participants were required to determine the left-right direction of an arrow displayed on a screen by pressing one of two response buttons. In the Stop trials, an auditory stop signal was presented with variable delay after the presentation of the Go stimulus. Participants were instructed to withhold their response whenever they heard the stop signal. The stop signal consisted of an affectively neutral or charged nonverbal vocalization [START_REF] Belin | The Montreal Affective Voices: A validated set of nonverbal affect bursts for research on auditory affective processing[END_REF]. If the imbalance adolescence model [START_REF] Casey | Beyond simple models of self-control to circuit-based accounts of adolescent behavior[END_REF][START_REF] Casey | The teenage brain: self-control[END_REF] is correct, then performance (i) should increase with age in the cool SST and (ii) be lower in the hot than the cool SST in adolescents but not in children and in adults. In addition, if the Zelazo's model [START_REF] Zelazo | Hot and Cool Executive Function in Childhood and Adolescence: Development and Plasticity[END_REF] is correct then (i) performance should increase with age in both the cool and hot SST and (ii) the performance should improve earlier in the cool than the hot SST.

METHODS

Participants

Eighty-two young adults (18 -30 years old), 70 adolescents (13 -16 years old) and 95 children (9 -11 years old) were recruited from public universities and middle and elementary schools supporting diverse populations with a wide range of socioeconomic statuses. One adult participant, 6 adolescents and 3 children were discarded from the analyses due to technical issues during data collection (i.e., data were not fully recorded for those participants). An additional 10 participants (4 children, 1 adolescent and 5 adults) were discarded because their accuracy on the Stop trials (below 30% or above 70% of correct Stop trials) and 12 (3 children, 4 adolescents and 5 adults) because their SSRTs differ by more than 2 SD of the mean SSRTs of their age by type of SST groups. The proportion of participants discarded did not differ between the age by type of SST groups, χ2(2) = .78, p = .67 Therefore, the data analyses were restricted to 71 adults (M = 20.83 ± 1.75 years old; 25 males), 59 adolescents (M = 14.05 ± 0.68 years old; 25 males) and 85 children (M = 9.85 ± 0.61 years old; 43 males). The proportions of males to females did not differ among the young adults, the adolescents and the children, χ2(2) = 3.75, p = .15. Participants were randomly assigned to the cool (38 adults, 26 adolescents and 47 children) or hot SST (33 adults,33 adolescents and 38 children). Twenty-five participants were left-handed, 13 of which were assigned to the cool condition (8 children, 2 adolescents and 3 adults) and 12 to the hot condition (6 children, 4 adolescents and 2 adults). The proportion of left-handed participants did not differ between the age by type of SST groups, χ2(2) = 1.11, p = .57. Sample size was determined by running an a priori power analysis using G*Power 3.1.9.2 [START_REF] Faul | Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses[END_REF], revealing that a minimum of 211 participants would be needed to detect a medium effect size of 0.25 (according to Cohen's effect size conventions) within a 2 (SST: cool vs. hot) x 3 (Age: children vs adolescent vs. young adult) between-subject analysis of variance (ANOVA) with power (1β) set at .80 and α set at .05. All participants had normal or corrected-to-normal vision and were typically developing. All of the participants, or parents (legal guardians) for minors, gave written consent. All participants were also tested in accordance with the national and international norms that govern the use of human research participants.

Stimuli

Each participant performed either the cool or the hot SST. Both versions of the task were implemented with E-prime 2.0.10 (Psychology Software Tools, Pittsburgh, PA) and performed individually on a computer equipped with size-adjustable headphones through which an auditory signal could be presented. In both the cool and hot SST, participants were presented with arrows pointing to the right or to the left (i.e., Go stimulus) (see Figure 1).

In the present study, nonverbal vocalizations, adapted from the Montreal Affective Voices (MAV) database (see [START_REF] Belin | The Montreal Affective Voices: A validated set of nonverbal affect bursts for research on auditory affective processing[END_REF], and consisting of interjections, using the vowel /a/, expressing neutral, fear (i.e., screams) and anger (i.e., expression of rage) expressions, were used as stop signals. Note that previous studies also used auditory cues (e.g., bip) as stop signal in both cool (e.g., [START_REF] Bedard | The Development of Selective Inhibitory Control Across the Life Span[END_REF][START_REF] Carver | Age Related Changes in Inhibitory Control as Measured by Stop Signal Task Performance[END_REF][START_REF] Logan | On the ability to inhibit thought and action: a theory of an act of control[END_REF][START_REF] Logan | Impulsivity and Inhibitory Control[END_REF][START_REF] Williams | Development of inhibitory control across the life span[END_REF] and hot [START_REF] Herbert | Response inhibition and memory retrieval of emotional target words: evidence from an emotional stop-signal task[END_REF][START_REF] Kalanthroff | Stop feeling: Inhibition of emotional interference following stop-signal trials[END_REF][START_REF] Krypotos | Individual differences in heart rate variability predict the degree of slowing during response inhibition and initiation in the presence of emotional stimuli[END_REF][START_REF] Rebetez | Do emotional stimuli interfere with two distinct components of inhibition?[END_REF] versions of the SST. Four neutral stop signals (i.e., two female and two male neutral nonverbal vocalizations) and four emotional stop signals (i.e., two female and two male nonverbal vocalizations conveying fear and anger) were thus used in the cool and hot version of the SST, respectively.

To meet the SST demands, all the non-verbal vocalizations from the MAV database (i.e., four neutral, two conveying fear and two conveying anger) were shortened to last 300 ms. A pre-test was thus conducted ahead on 21 young adults to enable to objectively shift our choice towards the neutral and emotional vocalizations reported above. Indeed, using a 1 (i.e., negative emotion) to 7 (i.e., positive emotion) Likert scale, participants rated the selected nonverbal vocalizations as followed: 3.95 (± 0.38) and 4.19 (± 0.40) for neutral female vocalizations; 3.95 (± 0.50) and 4.09 (± 0.62) for neutral male vocalizations; 1.38 (± 1.32) and 2.65 (± 1.31) for fear from female and male voices, respectively; and 1.90 (± 0.89) and 1.76 (± 0.99) for anger from female and male voices, respectively. Using one-sample t-tests, we confirmed that all the vocalizations conveying negative emotions scored significantly below 4 (p < .001 for all the four emotional vocalizations used in this SST paradigm) and neutral vocalizations did not scored differently from 4.

Procedure

In each Go trial, an arrow oriented either to the right or to the left was displayed on a screen. Participants were instructed to press, as quickly and accurately as possible, either the right or the left button of a computer mouse, respectively, using their right hand. In half of the trials, the arrow was pointing to the right, and in the other half, to the left. Each Go trial began with the presentation of (i) a fixation stimulus (i.e., a white square) for 500 ms, followed by (ii) an arrow for 1000 ms, and (iii) a black screen for 1500 ms (see Figure 1). Participants could provide an answer up to 2000 ms after the presentation of the arrow. Stop trials were similar to Go trials, except that a brief sound of 300 ms (i.e., stop signal) was presented with a variable stop signal delay (SSD) after the presentation of the arrow (Figure 1). Participants were instructed to refrain from pressing either of the two response buttons when they heard the stop signal. The initial delay between the presentation of the arrow and the stop signal (i.e., the SSD) was set to 150 ms. This SSD was then dynamically adjusted throughout the experiment, i.e., if participants successfully inhibited their motor response on the previous Stop trial, the SSD was increased by 50 ms on the subsequent Stop trial (up to a maximum delay of 1000 ms), while if they failed to inhibit their response, the SSD was decreased by 50 ms on the subsequent Stop trial (down to a delay of 0 ms) [START_REF] Logan | Impulsivity and Inhibitory Control[END_REF]. Using such a staircase procedure allows to maximize the probability of having participants successfully inhibiting their prepotent motor response on 50% of the Stop trials for each condition. Note that participants who deliberately engage in a strategy consisting in delaying their response could reach accuracy rate well above 50% on the Stop trials. Similarly, participants rushing to press the response buttons in all trials will reach accuracy rate well below 50%.

-Insert Figure 1 about here -

Statistical analyses

For the cool and hot SSTs, we computed a stop signal reaction time (SSRT) by subtracting the average SSD from the median reaction time on correctly performed Go trials [START_REF] Aron | Cortical and Subcortical Contributions to Stop Signal Response Inhibition: Role of the Subthalamic Nucleus[END_REF][START_REF] Logan | On the ability to inhibit thought and action: a theory of an act of control[END_REF]. Longer SSRTs revealed lower IC efficiency. To investigate the developmental pattern of the ability to withhold an initiated motor response in an affectively neutral or charged context, we ran a 3 (Age: children, adolescents and young adults) x 2 (i.e., SST: cool vs. hot) between-subjects ANOVA on the SSRTs. For this analysis, we reported the effect size in terms of the amount of variance accounted for by the independent variables (partial eta-squared, η 2 p). Assuming that the mean SSD of the second block may be more sensitive to developmental differences in initial SSD, we ran a 3 (Age: children, adolescents and young adults) x 2 (SST: cool vs. hot) x 2 (Block: first vs. second) mixed-design ANOVA on the SSRTs.

RESULTS

The 3 (Age: children vs. adolescents vs. young adults) x 2 (SST: cool vs. hot) ANOVA on the SSRTs revealed a main effect of condition, F(1, 209) = 6.02, p = .015, η 2 p = .03, with longer SSRTs for hot trials relative to cool trials, a main effect of age, F(2, 209) = 54.08, p < .001, η 2 p = .34, with longer SSRTs in children than in adolescents (p < .001) and longer SSRTs in adolescents than in young adults (p < .001), but no two-way interaction between age and condition, F < 1 (see Figure 2). Although the two-way interaction between age and condition did not reach the significance, we conducted post hoc analyses, using Holm-Bonferroni correction, which revealed longer SSRTs in children than in young adults and in adolescents than in young adults in both the cool and hot SSTs (all ps < .001) and longer SSRTs in children than in adolescents in the cool SST (p = .046) but not in the hot SST (p = .19).

-Insert Figure 2 Given that all participants started with the same stop signal delay, young adults might need more stop trials to reach their optimal SSD than children or adolescents. If so, developmental effects could be underestimated when analyzing jointly the data of the two blocks of trials. Thus, we ran a 3 (Age: children vs. adolescents vs. young adults) x 2 (SST: cool vs. hot) x 2 (blocks of trials: first vs. second) mixed-design ANOVA on the SSRTs. Importantly, we found a marginal three-way interaction between block, condition and age, F(2, 209) = 2.69, p = .07, η 2 p = .02. To characterize the three-way interaction, we ran separate ANOVAs for each block of trials.

For the first block, the 3 (Age: children vs. adolescents vs. young adults) x 2 (SST: cool condition, F < 1. For the second block, a similar ANOVA on the SSRTs revealed a main effect of condition, F(1, 209) = 16.57, p < .001, η 2 p = .073, with longer SSRTs for hot than cool trials, a main effect of age, F(2, 209) = 20.43, p < .001, η 2 p = .16, with longer SSRTs in children than in adolescents (p = .011) and longer SSRTs in adolescents than in young adults (p = .002), but no two-way interaction between age and condition, F < 1. Although the twoway interaction between age and condition did not reach the significance, we conducted post hoc analyses, using Holm-Bonferroni correction which revealed longer SSRTs in children than in adults in both the cool and hot SSTs (all ps < .001), longer SSRTs in adolescents than in young adults in the hot SSTs (p = .006) but not in the cool SST (p = .75) and marginally longer SSRTs in children than in adolescents in the cool SST (p = .065) but not in the hot SST (p = .92). Finally, hot SSRTs were longer than cool SSRTs in adolescents (p = .009).

DISCUSSION

In the present study, we investigated the development of the ability to stop an initiated motor response from late childhood to young adulthood in an affectively neutral or charged context using a cool and hot version of the SST, respectively. As expected and consistent with previous studies on both interference control (e.g., [START_REF] References Aïte | Adolescents' inhibitory control: Keep it cool or lose control[END_REF] and response inhibition abilities (e.g., [START_REF] Schel | Development of response inhibition in the context of relevant versus irrelevant emotions[END_REF][START_REF] Tillman | Motor Response Inhibition and Execution in the Stop-Signal Task: Development and Relation to ADHD Behaviors[END_REF], we found that the ability to inhibit an initiated motor response increases linearly with age from late childhood to young adulthood in an affectively neutral context. The development of this ability in an affectively charged context appears to be more protracted, with children and adolescents being less efficient at stopping an initiated motor response than young adults but with no developmental difference between children and adolescents. The developmental patterns in the cool and hot SSTs reported in the present study are consistent with Zelzao's developmental model of cool and hot executive functions [START_REF] Zelazo | Hot and Cool Executive Function in Childhood and Adolescence: Development and Plasticity[END_REF], which assumes cool executive functions develop earlier than hot executive functions. The results of the post-hoc analyses conducted on the second block of trials tentatively support the imbalance model of adolescence [START_REF] Casey | Beyond simple models of self-control to circuit-based accounts of adolescent behavior[END_REF][START_REF] Casey | The teenage brain: self-control[END_REF], which suggests that adolescents have specific difficulties resisting prepotent responses in an affectively charged context. Our results provide preliminary evidence that a negatively affectively charged context could affect selectively the ability to inhibit an initiated motor response in adolescents. However, this result needs to be interpreted with caution given that the two-way interaction between age and condition did not reach significance. Future studies need to replicate such adolescent specificity effect and investigate the extent to which this ability is also affected by a positive affectively charged context.. Indeed, adolescents might be less efficient to inhibit an initiated motor response in a positive affectively charged context because they are more sensitive to appetitive cues (e.g., rewards assignment) at the root of their risk seeking behaviors (e.g., alcohol and substance abuse, dangerous driving, practice of dangerous sports; [START_REF] Casey | The adolescent brain[END_REF][START_REF] Reyna | Risk and rationality in adolescent decision making: Implications for theory, practice, and public policy[END_REF]. Consistent with this view, [START_REF] Somerville | Frontostriatal Maturation Predicts Cognitive Control Failure to Appetitive Cues in Adolescents[END_REF] reported less efficient response inhibition for happy faces than calm faces in a Go/NoGo task associated with an increased activity within the ventral striatum. Importantly, consistent with previous studies in children, young adolescents [START_REF] Urben | Emotional Modulation of the Ability to Inhibit a Prepotent Response During Childhood[END_REF]) and adults (e.g., [START_REF] Kalanthroff | Stop feeling: Inhibition of emotional interference following stop-signal trials[END_REF], we found that an affectively charged context alters response inhibition abilities in children, adolescents and young adults, suggesting that the stop signals used in the hot SST produced an affectively charged context. The detrimental effect of an affectively charged context on response inhibition abilities could provide evidence that emotional stimuli may capture more attentional resources than neutral stimuli, leaving fewer processing resources available for the successful inhibition of motor responses (e.g., [START_REF] Kalanthroff | Stop feeling: Inhibition of emotional interference following stop-signal trials[END_REF].

A limitation of the present study is the cross-sectional design to assess the developmental trajectory of response inhibition abilities from late childhood to young adulthood in affectively neutral and charged contexts. The cross-sectional design typically does not provide information on the developmental rate of these abilities at the individual level and is prone to cohort effect which may be confounded with age-related effects [START_REF] Setia | Methodology series module 3: Cross-sectional studies[END_REF]. Using a between-subject design to study the development of inhibitory control in neutral and affectively charged contexts constitutes another limitation of the present study. A withinsubject design would have allowed to control for potential difference in the inhibitory control efficiency of the participants assigned to the cool and the hot versions of the SST. Finally, we couldn't determine whether the effect of the affectively charged context on the SSRTs varied in function of the type of emotions elicited by the stop signal (i.e., fear vs anger) because the SSD in the hot SST was not adjusted throughout the blocks separately for anger and fear stop signal trials.

In conclusion, we provide the first evidence that response inhibition and, in particular, the ability to stop an initiated motor response develops linearly from late childhood to young adulthood in both neutral and affectively charged contexts, in line with a previous study conducted on children and young adolescents up to 13 years of age [START_REF] Urben | Emotional Modulation of the Ability to Inhibit a Prepotent Response During Childhood[END_REF]. The development of this ability in an affectively charged context appears to be more protracted than in an affectively neutral context. Also, analyses from the second block provide tentative evidence that adolescents have specific difficulties resisting prepotent responses in an affectively charged context. Finally, exerting response inhibition in affectively charged contexts is more challenging than in neutral contexts at all ages and might explain why it might be difficult to change people's behaviors during crises. This research did not receive any specific grant from funding agencies in the public,

Figure 1 :

 1 Figure 1: Go and Stop trials in the cool and hot SSTs.

Figure 2 :

 2 Figure 2: SSRTs for both cool (gray line) and hot (black line) SSTs for each age group. The error bars represent the standard error of the mean. * p < .05, ** p < .01, *** p < .001 (Holm-Bonferroni post hoc analyses).

  vs. hot) ANOVA on the SSRTs revealed a main effect of age, F(2, 209) = 47.26, p < .001, η 2 p