Giuseppe Garofalo

Davy Preuveneers

Wouter Joosen

Data privatizer for biometric applications and online identity management

Biometric data embeds information about the user which enables transparent and frictionless authentication. Despite being a more reliable alternative to traditional knowledge-based mechanisms, sharing the biometric template with third-parties raises privacy concerns for the user. Recent research has shown how biometric traces can be used to infer sensitive attributes like medical conditions or soft biometrics, e.g. age and gender. In this work, we investigate a novel methodology for private feature extraction in online biometric authentication. We aim to suppress soft biometrics, i.e. age and gender, while boosting the identification potential of the input trace. To this extent, we devise a min-max loss function which combines a siamese network for authentication and a predictor for private attribute inference. The multi-objective loss function harnesses the output of the predictor through adversarial optimization and gradient flipping to maximize the final gain. We empirically evaluate our model on gait data extracted from accelerometer and gyroscope sensors: our experiments show a drop from 73% to 52% accuracy for gender classification while loosing around 6% in the identity verification task. Our work demonstrates that a better trade-off between privacy and utility in biometric authentication is not only desirable but feasible.

Introduction

Biometrics have become a prevalent form of authentication. A broad spectrum of services, with their own unique security requirements, uses some form of biometric authentication, e.g. messaging applications or banking services. Biometrics are preferred over traditional knowledge-based systems, such as PINs and passwords, due to their ease of use, robustness and uniqueness. Moreover, the wide availability of mobile sensors allows for the deployment of near frictionless multi-modal systems.

Sensor based gait recognition is regarded as a promising approach towards unobtrusive user authentication [START_REF] Gafurov | A survey of biometric gait recognition: Approaches, security and challenges[END_REF][START_REF] Marsico | A survey on gait recognition via wearable sensors[END_REF][START_REF] Van Hamme | Improving resilience of behaviometric based continuous authentication with multiple accelerometers[END_REF][START_REF] Wan | A survey on gait recognition[END_REF]. Despite being less robust than well-established biometrics, motion data takes advantage of body worn sensors that are widely implemented in modern devices and require little to no effort by the user. By enabling continuous user authentication, gait authentication is a natural candidate for multi-modal settings, e.g. by combining face recognition to walking data [START_REF] Marsico | A survey on gait recognition via wearable sensors[END_REF]. In this way, we not only improve accuracy, but also strengthen Fig. 1: Biometric authentication pipeline for online identity management. The user interacts with the identity provider and the service provider through his/her device.

our system against forging and spoofing attacks [START_REF] Gafurov | A survey of biometric gait recognition: Approaches, security and challenges[END_REF]. Moreover, gender recognition is a relevant topic to be addressed in gait recognition for future applications in healthcare [START_REF] Marsico | A survey on gait recognition via wearable sensors[END_REF][START_REF] Wan | A survey on gait recognition[END_REF]. The ever-improving resilience of continuous authentication systems based on accelerometer and gyroscope measurements clashes with the lack of a comprehensive assessment in terms of sensitive data leakage, demanding for techniques to protect a user's privacy against sensitive inferences. In this work, we explore gait authentication and soft biometric recognition, i.e. gender and age, as a use case for our adversarial framework for privacy. As shown in Figure 1, biometric authentication systems typically involve three entities [START_REF] Rui | A survey on biometric authentication: Toward secure and privacypreserving identification[END_REF]: a device equipped with sensors, a service provider that authorizes access, and an identity provider that verifies the identity. The authentication pipeline is composed of three steps. During step one, the user device collects the biometric signal. The latter is then cleaned and prepared for feature extraction, which is the second step in the pipeline. Consequently, the pre-processed signal is transformed into a set of relevant features that can be matched with a stored user template. For example, a face image may be turned into a vector of numeric features, while a gait trace could become a 2D image. Herein, the feature extraction and matching scheme are implemented by the identity provider. The final step consists of sending the output of template matching to the service provider, which grants access to its services based on proper access control policies. These three blocks can be incorporated as parts of the user's device or exist in isolation. In alternative, hybrid implementation are possible, e.g feature are extracted in the user's device while the templates are matched remotely. The latter scenario, i.e. online authentication, requires the user to send sensitive data over an unreliable network, exposing him/her to potential privacy leaks. Handling biometric data, including storing and processing templates, calls for additional security and privacy guarantees.

Misuse of biometric templates leads to severe privacy leakages for the enduser. Recent work has shown the presence of sensitive data in biometric traces, including medical conditions and soft biometrics [START_REF]American Academy of Ophthalmology: Evidence mounts that an eye scan may detect early alzheimer's disease[END_REF][START_REF] Matovu | Your substance abuse disorder is an open secret! gleaning sensitive personal information from templates in an eeg-based authentication system[END_REF][START_REF] Mordini | The Transparent Body: Medical Information, Physical Privacy and Respect for Body Integrity[END_REF]. If the user consented for his/her biometric template to be stored on a third-party server, he/she has to be aware of the potential disclosure of such sensitive data. For example, a curious service provider might want to learn more about its customers to advertise them with tailored products and increase its sales. Even in the unrealistic hypothesis that the user can blindly trust his/her recipient, adversaries might steal the user's template by impersonating the trusted party or attacking remote servers [START_REF] Zeitz | Security issues of internet-based biometric authentication systems: risks of man-in-the-middle and biophishing on the example of biowebauth[END_REF]. However, in many cases the third-party will need access to relevant information to keep its services alive. This calls for techniques which enable the sharing of the least amount of sensitive identifying information, e.g. private biometric features.

Recent history testifies that algorithms are prone to discriminate based on racial or sex attributes [START_REF] Buolamwini | Gender shades: Intersectional accuracy disparities in commercial gender classification[END_REF][START_REF] Cohn | Googles algorithms discriminate against women and people of coloue[END_REF], leading to discussions on how to mitigate bias in AI [START_REF] Hao | This is how AI bias really happensand why its so hard to fix[END_REF]. Typical sources of bias are the training dataset, directly reflecting unfair patterns in the external world, and a flawed machine learning (ML) process, not accounting for discrimination or, even worse, deliberately fueling inequalities. By suppressing highly-sensitive information like gender or age, we propose a novel representation of the biometric trace which discourages differences based on group membership.

Our work is also motivated by the General Data Protection Regulation (GDPR) [START_REF]of the European Parliament and of the Council, on the protection of natural persons with regard to the processing of personal data and on the free movement of such data[END_REF], which tackles private data collection and processing problems. Article 25 puts the emphasis on the scope limitation by defining the data minimization principle which explicitly requires third-parties to limit their data collection to what is useful for their purposes, upon consent. However, storing data such as biometrics clashes with this principle because of its intrinsic re-purposable nature, which has been theorized and validated thoroughly in recent years. Hence, there is a need to design data reduction processes for sharing sensitive information in order to protect the users against unprompted attribute inferences. By sharing only what is needed for a predefined task, we can also address the discrimination problem, bringing fairness and transparency in the ML pipeline.

Figure 2 shows our pipeline for private biometric sharing. The user's device is in charge of collecting motion data and extracting features to be shared with an external authentication provider. However, the user typically lacks the necessary resources, both hardware and data, to train a feature extractor him/her-self. Thus, we embrace a data-driven approach to derive a private feature extractor on the server side. This approach exploits a publicly available dataset to train the model, allowing to derive only one feature extractor for all the users willing to authenticate. In particular, our adversarial framework is composed of three entities closely working together: a (private) feature extractor, an (adversarial) siamese identity verifier and a predictor for the private task. During training, the feature extractor will iteratively adapt to changes in the classifier, which in turn will challenge the extractor. This procedure models the mutual information between the identity and the private attributes, guaranteeing protection against sensitive inferences. Eventually, the feature extractor is published by the identity/service provider and becomes available for local usage on the user's device, as depicted in Figure 2. By harnessing our feature extractor, neither third parties nor channel eavesdroppers can accurately infer the target classification attribute from the shared embedding.

We apply our proposed framework in a gait verification scenario using fixed inertial sensors. In our evaluation, we compare different privatized traces to assess the identifiability of the users while the new extracted features cannot be used to infer the user's gender or age, which are our private attributes. We emulate our adversary's ability to infer the sensitive attributes by means of transfer learning, i.e. training an unseen classifier for the private task on the private features. Our main contributions include:

1. Devising an adversarial framework exploiting a novel loss function to train a private feature extractor starting from variable-length gait traces. 2. Evaluating the privacy-utility trade-off w.r.t soft biometrics privacy in the gait authentication domain. 3. Using the biggest known inertial sensors dataset, which includes almost 500 users and 5 different activities.

The rest of the paper is organized as follows. We identify the gap and differences with related work in Section 2. We present our framework for privacypreserving feature extraction in Section 3. The experimental protocol and results are presented in Section 4. Section 5 concludes our work.

Related Work

Mordini and Ashton [START_REF] Mordini | The Transparent Body: Medical Information, Physical Privacy and Respect for Body Integrity[END_REF] have performed an extensive study of medical pattern retainment in biometric templates: psychiatric conditions can be inferred from gait traces, chromosomal diseases can be accurately guessed from face images or fingerprints, while neurological pathologies have been associated to a broad range of behavioural biometrics. The same leakage potential holds true for electrocardiogram (ECG) signals [START_REF] Matovu | Your substance abuse disorder is an open secret! gleaning sensitive personal information from templates in an eeg-based authentication system[END_REF], iris recognition [START_REF]American Academy of Ophthalmology: Evidence mounts that an eye scan may detect early alzheimer's disease[END_REF] and other bio or behavio-metrics [START_REF] Dantcheva | What else does your biometric data reveal? a survey on soft biometrics[END_REF]. Similarly, soft biometrics like age, gender or race are linked to physiological or behavioural traits of the user. In a recent work, we proved the feasibility of age and gender estimation from gait traces in the frame of the OU-ISIR Wearable Sensor-based Gait Challenge: Age and Gender (GAG 2019) competition at the 12th IAPR International Conference on Biometrics1 [START_REF] Van Hamme | A systematic comparison of age and gender prediction on imu sensor-based gait traces[END_REF]. The goal of this competition was to improve the state-of-the-art in soft biometric prediction from accelerometer and gyroscope traces. Even without crucial information on sensors position, we were able to achieve ∼76% accuracy for gender classification and a mean absolute error of ∼6 years for age estimation, eventually obtaining the best result among all contestants. Our model is inspired by [START_REF] Van Hamme | A systematic comparison of age and gender prediction on imu sensor-based gait traces[END_REF] as follows: we harness temporal convolutional networks (TCNs) for feature extraction and few dense layers for soft biometric prediction. On top of the extracted features, we have built a siamese network for user verification and we have plugged a gradient reversal layer for attribute privatization.

Several works tackled the problem of discrimination in the ML pipeline [START_REF] Alvi | Turning a blind eye: Explicit removal of biases and variation from deep neural network embeddings[END_REF][START_REF] Das | Mitigating bias in gender, age and ethnicity classification: a multi-task convolution neural network approach[END_REF]. Typically, they focus on the output of the decision function and how to make it independent from a particular group membership. In contrast to previous work, we address this problem indirectly, aiming to achieve soft biometric privacy. By suppressing information deemed to be private, we discourage discriminative attributes to influence the learning process, thereby representing a source of bias. For example, by minimizing the information about the gender in motion data, we encourage the building of a gender-agnostic gait verification system. In the soft biometric privacy landscape, our work is the first one focusing on time sequences and, specifically, gait authentication.

The approaches to protect user's privacy divide into context-free and contextaware techniques. Context-free techniques, like differential privacy (DP), model worst-case adversaries regardless of his/her real capabilities and discarding relevant contextual information, i.e. about the problem to be solved. DP provides strong privacy guarantees, delivering a shrinking in data usefulness. Contextaware strategies, on the other hand, incorporate the retainment of task-specific utility by selectively adding noise where it matters. This advantage comes at the expenses of a formal characterization of the relationship between public variables, i.e. what we aim to share, and private variables, i.e. what we aim to protect, which is rarely available in practice.

Data-driven optimization has been recently proposed as a mean to achieve context-aware privacy. By exploiting recent advances in adversarial optimization, it is possible to model the joint distribution between shared and private variables. Generative adversarial networks (GANs) have been recently proposed as an effective tool to achieve this goal [START_REF] Goodfellow | Generative adversarial nets[END_REF]. They model a min-max game between a generator and a discriminator, where the former tries to fool the latter in an iterative learning process. This concept has been first adapted to the privacy domain by Huang et al. who define the generative adversarial privacy (GAP) framework [START_REF] Huang | Context-aware generative adversarial privacy[END_REF]. Inspired by their work, we harness adversarial training to obtain a private feature extractor, representing our feature generator. This generator is used to obtain a compressed template representation for a specific, measurable and limited purpose, while also minimizing sensitive disclosure.

Morales et al. [START_REF] Morales | Sensitivenets: Learning agnostic representations with application to face recognition[END_REF] recently proposed a method to reduce gender and race information in latent representations of face images. Their method is based on a modification of the triplet loss function, which is commonly employed in face verification scenarios [START_REF] Schroff | Facenet: A unified embedding for face recognition and clustering[END_REF]. Our work differs from theirs for two reasons: first, we exploit adversarial optimization to maximize the privacy-utility trade-off; second, our feature extractor is designed to model temporal dependencies in the input data, making it more suitable for gait samples than face images. Similarly, Mirjalili et al. proposed a framework to impart gender privacy to face images [START_REF] Mirjalili | Gender privacy: An ensemble of semi adversarial networks for confounding arbitrary gender classifiers[END_REF][START_REF] Mirjalili | Flowsan: Privacy-enhancing semi-adversarial networks to confound arbitrary face-based gender classifiers[END_REF]. Drawing from their work, we empirically evaluate the privacy and generalizability of our approach by training several models, which simulate the ability of a malicious entity. As before, they focus on face recognition systems rather than temporal data.

Malekzadeh et al. [START_REF] Malekzadeh | Mobile sensor data anonymization[END_REF] have considered motion data and gait authentication in a different min-max optimization scenario: perturbing identity while preserving task-specific utility. Their classification task is activity recognition, which has been extensively studied in the gait literature in addition to being arguably a private variable. Moreover, we shift the focus towards building a private extractor to be used by end-users instead of generating a privatized trace in the input domain. By compacting the trace in a latent space representation, we reduce the interpretability of the shared sample while minimizing the risks of sensitive inference. Similarly, Osia et al. [START_REF] Ossia | A hybrid deep learning architecture for privacy-preserving mobile analytics[END_REF] investigates the use of siamese networks for privatizing the user's identity while preserving gender classification accuracy. Besides the different learning goal, they focus on fine-tuning existing, pre-trained networks. In our framework, the minimization of sensitive attribute is embedded in the learning process itself. By simply applying fine-tuning to the last layers of the feature extractor, we would discourage the achievement of a better suboptimal solution for the min-max optimization problem.

Private Feature Extraction Framework

We propose a novel framework for protecting sensitive variables when sharing biometric data in an online authentication scenario. In this section, we tackle 4 key aspects which characterize our framework: (i) the main steps, stakeholders, and threat model, (ii) the nominal privacy-preserving loss function, (iii) the designed architecture, i.e. the neural networks to approximate the nominal loss, (iv) and the architecture min-max optimization strategy.

High-level framework and threat model

Our framework faces the problem of sharing biometric data without exposing user's private information. This process requires the interaction between two entities: the user and the service provider. The user is willing to share what is needed to accomplish the main task but he is worried that certain information might leak along the way. Let us assume user A wants to be authenticated towards service B, then sharing the raw data will reveal attributes which A might consider private, e.g. A's gender. As discussed in Section 2, traditional techniques can be employed to solve this problem, however they come with several limitations such as expensive computations at the edge or having to trust external entities. Instead, we propose the use of a contained set of private features extracted from biometric data on the user side. These features are carefully optimized during the training phase, with a twofold purpose: (1) to preserve the information which identifies the user and (2) to suppress a specific private variable, e.g. the gender of the user.

Our proposed framework requires three actions by the involved actors: (i) the identity/service provider trains a private feature extractor for authentication purposes; (ii) the feature extractor is published, which allows the user to extract authentication features and assess its privacy guarantees; (iii) by following the authentication protocol, the authentication features are shared with the service provider which grants access to the system based on given access control policies. This three-step procedure protects the user against unprompted sensitive inferences: the service provider will not be able to improve its knowledge about the user w.r.t. to the selected private variables, thus we protect against function creep [START_REF] Winner | Autonomous Technology: Technics-Out-of-Control as a Theme in Political Thought[END_REF]. This is inherent to the local feature extraction step (ii), which suppresses private variables within the user's device. Moreover, sharing the private extractor enables any external entities, like the user, to assess and analyze the privacy of the model, which is only trained on public data. It is worth noticing that the authentication-party might want to train a ML model to authenticate the user based on the received features, i.e. step (iii), and this part of the model has to be kept private in order to guarantee users' privacy. In addition, the feature extractor could be trained by different parties than the third-party, but we rely on the realistic hypothesis that the features are especially crafted for the main task. Therefore, the service provider is better suited to design the feature extraction step. If we assume that the features are intended for different uses, then we can assign step (i) to another external, mediating entity without affecting the presented framework.

While our solution overcomes traditional techniques limitations, several challenges regarding privacy estimation arise. By delegating the training phase to a cloud-based service, we cut down the computational power which is requested to the user. Thus, only feature extraction of a pre-trained model is performed on the user side. In addition to energy consumption, by extracting the biometric representation locally, we avoid the need for an external mediator. Therefore, we free the user from trusting an external entity. However, unlike traditional techniques like DP, we can only provide empirical privacy guarantees of the shared representation. This is due to a different sharing scenario, which involves single temporal traces as opposed to large databases of many users.

We evaluate the privacy of the extracted representation by looking at the performance of a newly trained ML classifier. Having fixed the discernment of a discrete sensitive variable as the learning goal, and provided the extracted features as the model input, we derive an empirical definition of privacy: the better the classifier performs, the higher the sensitive leakage. In practice, the classifier mimics the capabilities of a curious service provider willing to obtain valuable information about its users. The provider has access to the public dataset used to train the feature extractor (step (i)), and it also knows the training details as well as the trained model weights. Thus, the third-party is able to use public data to train a classifier discerning the private variable from the extracted features. We test the classifier against a group of test users, which simulates the sharing of features during regular usage. The service provider acts as the most powerful adversary for knowledge and resources, so we indirectly test the privacy of the extractor against any, less or equally powerful, external adversaries trying to infer sensitive information from the shared representation. In conclusion, if the service provider is not able to train a classifier which discerns the private variable from the shared representation, we consider the features to be safe against inferences. However, it has to be stressed that a more powerful estimator which is able to extract private information from the shared template might exist, which is a realistic assumption drawn from recent work [START_REF] Pittaluga | Learning Privacy Preserving Encodings Through Adversarial Training[END_REF].

Privacy-preserving training objective

We present here our nominal loss function for soft biometric privacy, i.e. the ideal training objective to be approximated via neural networks optimization. Given an input X, we search for the optimal feature extractor F E(•), which outputs the private embedding Z = F E(X). We define D(•, •) as a measure of the dependency between two variables, such that D(Z, T ) measures the dependency between the private embedding and the classification task T we aim to suppress, while D(Z, I) describes the usefulness of the latent representation for authentication purposes, being I the identity of the user. The nominal loss to be minimized becomes

N L = α * D(Z, I) -β * D(Z, T ) (1)
where α and β regulate the importance of each term, which purpose is to fine-tune the privacy-utility trade-off. As mentioned before, we make use of deep neural networks to approximate D(•, •). Therefore, an estimation of this measure is embedded in the weights of the models after training. We estimate it by extracting features and analyzing what newly created ML models are capable of learning from these features.

Neural networks architecture

Our framework is composed of three competing blocks: a private extractor (Figure 4), an identity verifier (Figure 5b) and a task-dependent predictor (Figure 5a). As mentioned earlier, the classification pipeline is inspired by [START_REF] Van Hamme | A systematic comparison of age and gender prediction on imu sensor-based gait traces[END_REF] with the addition of a siamese block for gait verification. An overview of the interacting components is presented in Figure 3.

The input of the model is 3-dimensional raw accelerometer and 3-dimensional raw gyroscope measurements recorded from inertial sensors. In our implementation, we account for variable-length input by simply stacking sensors measurements (6-dimensional measurements) without pruning the obtained sequences.

Input data is fed to a siamese feature extractor FE, which details are presented in Figure 4. Every layer is presented along with its input and output shapes, where None represents either a variable length trace or a variable number of samples, i.e. a batch. As first proposed by Bai et al. [START_REF] Bai | An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[END_REF], we perform dynamic feature extraction by harnessing temporal convolutional networks (TCNs). This family of networks have been demonstrated to achieve state-of-the-art performance when dealing with temporal data, behaving equal to or better than recurrent neural networks. TCNs potential is mainly due to dilated convolutions, which address both complexity of the network and low-level spatial accuracy. Convolutional layers are intertwined with dropout layers to prevent over-fitting the training set, thus acting as regularizers by zeroing-out random filters which are re-activated when testing the model. Following best practices, L2 regularization is also introduced to penalize complex models through the loss function. Finally, a global average layer flattens the output of the extractor to obtain 128 features, which are the objective of the privatization of our optimization framework. They are then conveyed into two branches: a soft biometric predictor and a siamese identity verifier. The first branch is the soft biometric predictor Pr, which details are depicted in Figure 5a. This model is composed of several fully connected layers with ReLU activations. As before, dropout and regularization techniques are employed to improve generalizability. The model is designed once but tuned and optimized separately for gender and age. During optimization, we minimize binary crossentropy to maximize our sensitive variable prediction accuracy.

The second branch is a siamese neural network for identity verification Id, which details are shown in Figure 5b. This model is composed of two stacked fully connected layers which are duplicated into two parallel branches sharing their weights (in Figure 5b, only one branch is shown). These branches converge into a distance-based function, the contrastive loss, which is typically used in a verification scenario to increase the similarity, i.e. decrease the distance in the latent space, of samples belonging to the same class while driving away dissimilar pairs. In order to obtain two feature vectors, the feature extractor is turned into a siamese model as follows: we duplicate the FE, obtaining two branches which share their weights and allowing to feed a pair of gait traces into the input layer.

By combining our three blocks, we end up with the following multi-objective loss function

G(θ, φ) = min θ,φ (α * Id(φ) + β * P r(θ)) (2)
where θ and φ are the weights of PR and Id, respectively, and α and β weight each term. By minimizing G, however, the predictor accuracy is maximized, thus we need to reverse this trend to protect the latent representation against inferences of the sensitive variable. First introduced in domain adaptation by Ganin et al. [START_REF] Ganin | Domain-adversarial training of neural networks[END_REF], we plug a gradient reversal layer between FE and Pr which minimizes the Pr accuracy when training FE. This leads to a min-max loss function which is the approximated realization of Equation 1.

Optimization of the networks

As common in adversarial training, the three blocks are optimized separately, via strict alternation. Figure 3 shows the interaction among the elements, which single iteration works as follows. Id and Pr are fed with features extracted from FE and trained for one epoch to minimize the contrastive loss and the cross-entropy, respectively. FE is later trained for a single epoch to optimize Equation 2 by freezing the weights of the other networks. By reversing the gradient of Pr, FE will adapt to its changes becoming more and more resilient to sensitive inferences. This challenge approximates the dependency measure D (cfr. Equation 1) which is key to achieve a satisfactory sub-optimal trade-off between privacy and utility.

By alternating the gradient updates among the three networks, we enforce dynamic adaptation to future updates. Suppose Pr is trained w.r.t. the true labels until a acceptable sub-optimal solution is found. FE can be trained to beat Pr, hiding the sensitive variable from the learnt representation. However, this holds true only for one sub-optimal solution of Pr and does not prevent inferences from future re-training. Instead, by letting Pr adapt to changes in FE, and vice-versa, both models converge to a more satisfactory solution in terms of privacy vs. utility. Hyper-parameters play an important role in network convergence, impeding Pr to win over FE ; yet, we can only empirically estimate the best hyper-parameters for our task.

Our feature extraction strategy differs from traditional dimensionality reduction techniques because we actively trigger the reaction of a discriminator, exploiting its input to refine our final output. However, approaches like Principal Component Analysis (PCA) or noise addition at the bottleneck of the network could still be applied as a complementary, simple aid to achieve a better suboptimal solution.

Evaluation

In this section, we evaluate the ability of our framework to hinder classification of gender from motion features while preserving the identity of the user. We present the experimental setup, followed by the evidence we found.

Experimental setup

We define our networks using Keras 2.2.4 with a Tensorflow backend, which runs on a machine with a 3.4GHz i5-7500 CPU, 16GB RAM, and an NVIDIA Titan V GPU. As in [START_REF] Van Hamme | A systematic comparison of age and gender prediction on imu sensor-based gait traces[END_REF], we choose the OU-ISIR labelled gait action dataset [START_REF] Ngo | Similar gait action recognition using an inertial sensor[END_REF] to train our model. Angular velocity and acceleration in the 3 dimensions are collected from sensors fixed on a belt, at a sampling rate of 100 Hz. Every trace is associated to age, gender and current activity of the user.

In order to generate training pairs, we undertake several steps. We first divide users into training and test sets following a 80%-20% proportion, after which we use the same ratio to split the training set into training and validation sets. Then we generate fixed length sequences through windowing with overlapping: a user is selected and his/her trace is divided into several traces of length 2.56s with 20% overlap, i.e. 0.5s at the end and 0.5s at the beginning of the trace. Finally, we create pairs to feed our model with through an iterative procedure in two steps: [START_REF] Alvi | Turning a blind eye: Explicit removal of biases and variation from deep neural network embeddings[END_REF] for each user, the current trace is coupled with the subsequent one and a similar pair label is assigned to the pair; (2) the second trace of the previous pair is coupled with a random trace from a different user and a dissimilar pair label is assigned to this couple. Since the first branch of the siamese FE (i.e., branch A in Figure 3) is responsible for feeding Pr, a label with the gender of the first user is associated to the pair.

We empirically select the best hyper-parameter configuration for our networks. We select the Nadam optimizer to train the models, following the procedure explained in Section 3.4. Every model is trained on mini-batches within the set [START_REF] Cohn | Googles algorithms discriminate against women and people of coloue[END_REF][START_REF] Pittaluga | Learning Privacy Preserving Encodings Through Adversarial Training[END_REF]50], while the number of epochs varies between 15 and 50. We fix α and β to 1. Intuitively, one can expect α and β to have a predictable impact on the privacy-utility trade-off when training the features extractor. Due to the adversarial nature of the training procedure, however, tuning these variables proved to be highly sensitive w.r.t. the given setting and selected hyper-parameters. We argue that this effect can be associated to the training of the predictors, which are independent from α and β in our implementation. Nevertheless, as underlined before, trivial noise addition and dimensionality reduction could be employed as a better, more stable alternative for privacy-utility trade-off tuning.

After training, the sensitive variable predictor and the identity verifier are re-trained on the privatized features. As suggested by previous work [START_REF] Ossia | A hybrid deep learning architecture for privacy-preserving mobile analytics[END_REF], the privacy of a sensitive variable can be evaluated via transfer learning, i.e. freezing our pre-trained feature extractor and training a soft biometric predictor from scratch. In order for a thorough evaluation of the generalizability of our approach, unseen classifiers have to be taken into consideration. Hence, we define two models: (1) a DNN-based predictor resembling the one we used in our adversarial framework, which ensures protection against our target model; (2) a Support Vector Machine (SVM) which is typically used downstream of DNN for feature extraction. The SVM is optimized by applying grid search, which exhaustively searches for the hyper-parameter combination with the best score. L2-normalization is also performed to maximize our accuracy metric, i.e. f1score. We repeat each experiment 10 times, reporting the f1-score for the gender and the average verification accuracy for the verifier. All the results, besides the f1-score which results from a deterministic search, are presented with their standard deviation.

Experimental results

Table 1 shows the results for our baseline model: a siamese feature extractor for identity verification. After training, the extracted features are used to infer the gender of the user, resulting in a f1-score of 72.58% for the SVM in the best configuration. This underlines the retainment of soft biometric information in the authentication features, especially if we compare this figure to the state-ofthe-art gender prediction accuracy presented in [START_REF] Van Hamme | A systematic comparison of age and gender prediction on imu sensor-based gait traces[END_REF], i.e. 75.77%.

We compare our baseline with our proposed approach for feature privatization, which is summarized in Table 2. A high variability in the results can be observed, which is mainly due to the instability of the adversarial learning procedure. The f1-score shows how our privatization mechanism protects the features against possible re-training of our target predictor in each setting. However, we have to take into account generalization and we must be able to protect against unseen classifiers. SVM f1-score proves that we are able to achieve a nearly optimal result (50%) by carefully tuning our hyper-parameters, i.e. number of epochs and batch size. This comes at the expense of a nearly 6% loss in verification accuracy. For a batch size of 50, and increase in the number of epochs corresponds with a slight decline in verification accuracy. This drop indicates how letting the network train for a larger number of epochs improves the empirical privacy (see smaller f1-scores in Table 2) while decreasing the utility, even by just a tiny fraction.

We identify two main limitations for this work which are linked to our privacy evaluation and chosen dataset. First, we empirically evaluate the privacy of our framework by re-training different models on the extracted features. Future work could derive a formal evaluation of the privacy guarantees from an informationtheory point of view. Second, the OU-ISIR dataset provides us with sufficient data for our scopes but its data is collected in a constrained environment by sensors fixed on a belt. In a real world scenario, we deal with different orientations of mobile devices, and its sensors, carried by the users. A more realistic dataset is needed to properly evaluate and compare solutions in the gait domain.

As a future direction, we aim to tackle the linkability of templates across services while hiding different private variables for one user. Since we are not delivering a full-fledged biometric template protection (BTP) scheme, we do not directly address linkability of traces, assessing instead the retainment of private information for a specific use-case, i.e. gender classification. Hence, our framework alone does not fulfill the two requirements of the standard on biometric data protection ISO/IEC 24745 (2011) [START_REF]Information technology -Security techniques -Biometric information protection[END_REF], i.e. irreversibility and unlinkability. However, by tackling the data minimization problem we aim to address problems which are complementary to BTP schemes: (1) we exclude unnecessary data from transmission and processing, possibly improving privacy and performance of crypto schemes, and (2) we help preventing or fighting back algorithmic bias by feeding algorithms with more neutral and task-specific data. We envision a hybrid system where BTP schemes and adversarial training maximize the utility for a specific task without compromising users privacy. Future directions include exploring age or race prediction to evaluate cross-task linkability, and analyzing the advantage of applying a biometric crypto scheme on top of our minimization framework for compliance with existing requirements for private and secure biometric data processing and management. To this extent, Barrero et al. [START_REF] Gomez-Barrero | General framework to evaluate unlinkability in biometric template protection systems[END_REF] have proposed a metric to evaluate the local and global linkability of biometric templates.

Conclusion

In this work, we demonstrated the effectiveness of an adversarial learning technique towards privatization of biometric features from sequential data. We evaluated our approach on the gender estimation use-case, inspired by a recent work. Our evaluation supported our approach, showing a dip in the f1-score from 73% to 52.99% in the best case, which is very close to random guess (50%).

Further evaluation is needed to assess the effectiveness of our approach against different use-cases, but our results show that a solution to the long standing problem of data-minimization for biometrics is possible. Data-driven techniques have the potential to achieve the optimal trade-off between privacy and utility, something traditional techniques usually struggle with. We advocate for new tools for the user to manage his own identity and the amount of sensitive information which is shared with third-parties.

Fig. 3 :

 3 Fig.3: Our framework with its three challenging entities: the siamese feature extractor (FE), the soft biometric predictor, and the siamese identity verifier (Id).

Fig. 4 :

 4 Fig. 4: Implementation of Siamese FE. The sub-model siamese branch is shared among the two input layers and contains a quintessential element, namely Conv1D.

Fig. 5 :

 5 Fig. 5: Implementations of Predictor (a) and Siamese Id (b). In (b), the submodel siamese branch is shared among the two input layers.

  Pipeline for private feature extraction in a biometric sharing scenario. The central element is the extractor, which is shared with the user.

	Sensors	Identity Age, Gender...	Private feature extractor	Identity provider Service provider
	User	User's device	Extracted features
	Fig. 2:

Table 1 :

 1 Scores for baseline siamese verifier trained without predictor feedback.

	Epochs	Verification accuracy f1-score (SVM)
	200 (early stopping)	90.93%±0.15	72.58%

Table 2 :

 2 Re-train f1-score for the predictors and accuracy for the siamese identity verifier after adversarial privatization.

	Epochs	Batch size Verification accuracy	f1-score	f1-score (SVM)
	15	25	82.14%±0.89%	51.15%±0.70%	60.97%
	15	50	87.15%±0.38%	50.68%±0.57%	65.26%
	25	25	84.47%±0.53%	50.20%±0.15%	63.28%
	25	50	85.28%±0.48%	50.10%±0.00%	52.99%

http://www.am.sanken.osaka-u.ac.jp/GAG2019/

Acknowledgement. This research is partially funded by the Research Fund KU Leuven. Work for this paper was supported by the European Commission

through the H2020 project CyberSec4Europe (https://www.cybersec4europe.eu/) under grant No. 830929. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research.


Francis Bach
email: francis.bach@inria.fr

Lénaïc Chizat
email: lenaic.chizat@epfl.ch

Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization

Many supervised machine learning methods are naturally cast as optimization problems. For prediction models which are linear in their parameters, this often leads to convex problems for which many mathematical guarantees exist. Models which are non-linear in their parameters such as neural networks lead to non-convex optimization problems for which guarantees are harder to obtain. In this review paper, we consider two-layer neural networks with homogeneous activation functions where the number of hidden neurons tends to infinity, and show how qualitative convergence guarantees may be derived.

Introduction

In the past twenty years, data in all their forms have played an increasing role: in personal lives, with various forms of multimedia and social networks, in the economic sector where most industries monitor all of their processes and aim at making data-driven decisions, and in sciences, where databased research is having more and more impact, both in fields which are traditionally data-driven such as medecine and biology, but also in humanities.

This proliferation of data leads to a need for automatic processing, with striking recent progress in some perception tasks where humans excel, such as image recognition, or natural language processing. These advances in artificial intelligence were fueled by the combination of three factors:

(1) massive data to learn from, such as millions of labeled images, (2) increased computing resources to treat this data, and (3) continued scientific progress in algorithms.

Machine learning is one of the scientific disciplines that have made this progress possible, by blending statistics and optimization to design algorithms with theoretical generalization guarantees. The goal of this review paper that will be published in the Proceedings of the 2022 International Congress of Mathematicians is to highlight our recent progress from already published works [START_REF] Chizat | On the global convergence of gradient descent for over-parameterized models using optimal transport[END_REF][START_REF] Chizat | Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss[END_REF], and to present a few open mathematical problems.

Supervised learning

In this paper, we will focus on the supervised machine learning problem, where we are being given n pairs of observations (x i , y i ) ∈ X × Y, i = 1, . . . , n, for example images (X is then the set of all possible images), with a set of labels (Y is then a finite set, which we will assume to be a subset of R for simplicity). The goal is to be able to predict a new output y ∈ Y, given a previously unobserved input x ∈ X.

Following the traditional statistical M-estimation framework [START_REF] Van Der | Asymptotic statistics[END_REF], this can be performed by considering prediction functions x → h(x, θ) ∈ R, parameterized by θ ∈ R d . The vector θ is then estimated through regularized empirical risk minimization, that is by solving min

θ∈R d 1 n n i=1 y i , h(x i , θ) + λΩ(θ), (2.1)
where : Y × R → R is a loss function, and Ω : R d → R is a regularization term that avoids overfitting (that is learning a carbon copy of the observed data that does not generalize well to unseen data). Typical loss functions are the square loss y i , h(x i , θ) = 1 2 y i -h(x i , θ) 2 for regression problems, and the logistic loss y i , h(x i , θ) = log 1 + exp(-y i h(x i , θ)) for binary classification where Y = {-1, 1}. In this paper, we will always assume that the loss function is continuously twice differentiable and convex with respect to the second variable. This applies to a wide variety of output spaces beyond regression and binary classification (see [START_REF] Nowak-Vila | A general theory for structured prediction with smooth convex surrogates[END_REF] and references therein).

When the predictor depends linearly in the parameters, typical regularizers are the squared Euclidean norm Ω(θ) = 1 2 θ 2 2 or the 1 -norm Ω(θ) = θ 1 , that both lead to improved generalization performance, with the 1 -norm providing additional variable selection benefits [START_REF] Giraud | Introduction to high-dimensional statistics[END_REF].

Statistics and optimization

The optimization problem in Eq. (2.1) leads naturally to two sets of questions, which are often treated separately. Given that some minimizer θ is obtained (no matter how), how does the corresponding prediction function generalize to unseen data? This is a statistical question that requires assumptions on the link between the observed data (usually called the "training data"), and the unseen data (usually called the "testing data"). It is typical to assume that the training and testing data are sampled independently and identically from the same fixed distribution. Then a series of theoretical guarantees applies, based on various probabilistic concentration inequalities (see, e.g., [START_REF] Mohri | Foundations of machine learning[END_REF]).

The second question is how to obtain an approximate minimizer θ, which is an optimization problem, regardless on the relevance of θ on unseen data (see, e.g., [START_REF] Bubeck | Convex optimization: Algorithms and complexity[END_REF]). For high-dimensional problems where d is large (up to millions or billions), classical gradient-based algorithms are preferred because of their simplicity, efficiency, robustness and favorable convergence properties. The most classical one is gradient descent, which is an iterative algorithm with iteration:

θ k = θ k-1 -γ∇R(θ k-1 ), where R(θ) = 1 n n i=1 y i , h(x i , θ) + λΩ(θ)
is the objective function in Eq. (2.1), and γ > 0 the step-size.

In this paper, where we aim at tackling high-dimensional problems, we will often consider the two problems of optimization and statistical estimation jointly.

Linear predictors and convex optimization

In many applications, a prediction function which is linear in the parameter θ is sufficient for good predictive performance, that is, we can write

h(x, θ) = θ Φ(x)
for some function Φ : X → R d , which is often called a "feature function". For simplicity we have assumed finite-dimensional features, but infinite-dimensional features can also be considered, with a specific computational argument to allow finite-dimensional computations through reproducing kernel Hilbert spaces (see, e.g., [START_REF] Schölkopf | Learning with kernels[END_REF] and references therein).

Given a convex loss function, the optimization problem is convex and gradient descent on the objective function, together with its stochastic extensions, has led to a number of efficient algorithms with strong generalization guarantees of convergence towards the global optimum of the objective function [START_REF] Bubeck | Convex optimization: Algorithms and complexity[END_REF]. For example, for the square loss or the logistic loss, if the feature function is bounded in 2 -norm by R for all observations, and for the squared Euclidean norm Ω(θ) = 1 2 θ 2 2 , bounds on the number of iterations to reach a certain precision ε (difference between the candidate function value R(θ) and the minimal value) can be obtained:

• For gradient descent, R 2
λ log 1 ε iterations are needed, but each iteration has a running time complexity of O(nd), because the d-dimensional gradients of the n functions θ → y i , h(x i , θ) , i = 1, . . . , n, are needed.

• For stochastic gradient descent, with iteration θ k = θ k-1 -γ∇ y i(k) , h(x i(k) , θ k-1 ) , with i(k) ∈ {1, . . . , n} taken uniformly at random, the number of iterations is at most R 2 λ 1 ε . We lose the logarithmic dependence, but each iteration has complexity O(d), which can be a substantial gain when n is large.

• More recent algorithms based on variance reduction can achieve an overall complexity proportional to n + R 2 λ log 1 ε , thus with an exponential convergence rate at low iteration cost (see [START_REF] Gower | Variance-reduced methods for machine learning[END_REF] and references therein).

In summary, for linear models, algorithms come with strong performance guarantees that reasonably match their empirical behavior. As shown below, non-linear models exhibit more difficulties.

Neural networks and non-convex optimization

In many other application areas, in particular in multimedia processing, linear predictors have been superseded by non-linear predictors, with neural networks being the most classical example (see [START_REF] Goodfellow | Deep learning[END_REF]). A vanilla neural network is a prediction function of the form

h(x, θ) = θ s σ(θ s-1 σ(• • • θ 2 σ(θ 1 x)),
where the function σ : R → R is taken component-wise, with the classical examples being the sigmoid function σ(t) = (1 + exp(-t)) -1 and the "rectified linear unit" (ReLU), σ(t) = t + = Figure 1: Neural network with a single hidden layer, with an input weight matrix θ 1 ∈ R d×m and a output weight vector θ 2 ∈ R m . max{t, 0}. The matrices θ 1 , . . . , θ s are called weight matrices. The simplest non-linear predictor is for s = 2, and will be the main subject of study in this paper. See Figure 1 for an illustration.

The main difficulty is that now the optimization problem in Eq. (2.1) is not convex anymore, and gradient descent can converge to stationary points that are not global minima. Theoretical guarantees can be obtained regarding the decay of the norm of the gradient of the objective function, or convergence to a local minimizer may be ensured [START_REF] Lee | Gradient descent only converges to minimizers[END_REF][START_REF] Jin | How to escape saddle points efficiently[END_REF], but this does not exclude bad local minima, and global quantitative convergence guarantees can only be obtained with exponential dependence in dimension for the class of (potentially non-convex) functions of a given regularity [START_REF] Nesterov | [END_REF].

An extra difficulty is related to the number of hidden neurons, also referred to as the width of the network (equal to the size of θ 2 when s = 2), which is often very large in practice, which poses both statistical and optimization issues. We will see that this is precisely this overparameterization that allows to obtain qualitative global convergence guarantees.

Mean field limit of overparameterized one-hidden layer neural networks

We now tackle the study of neural networks with one infinitely wide hidden layer. They are also referred to as (wide) two-layer neural networks, because they have two layers of weights. We first rescale the prediction function by 1/m (which can be obtained by rescaling θ 2 by 1/m), and express it explicitly as en empirical average, as

h(x, θ) = 1 m θ 2 σ(x θ 1 ) = 1 m m j=1 θ 2 (j) • σ x θ 1 (•, j) ,
where θ 2 (j) ∈ R is the output weight associated to neuron j, and θ 1 (•, j) ∈ R d the corresponding vector of input weights. The key observation is that the prediction function x → h(x, θ) is the average of m prediction functions x → θ 2 (j) • σ θ 1 (•, j) x , for j = 1, . . . , m, with no sharing of the parameters (which is not true if extra layers of hidden neurons are added).

In order to highlight this parameter separability, we define

w j = θ 2 (j), θ 1 (•, j) ∈ R d+1
the set of weights associated to the hidden neuron j, and we define Ψ(w) : x → w(1) • σ x w(2, . . . , d + 1) , so that the prediction function x → h(•, w 1 , . . . , w m ), parameterized by w 1 , . . . , w m , is now

h(•, w 1 , . . . , w m ) = 1 m m j=1 Ψ(w j ). (3.1)
The empirical risk is of the form

R(h) = E (y, h(x)) ,
which is convex in h for convex loss functions (even for neural networks), but typically non convex in w. Note that the resulting problem of minimizing a convex function R(h) for h = 1 m m j=1 Ψ(w j ) applies beyond neural networks, for example, for sparse deconvolution [START_REF] Chizat | Sparse optimization on measures with over-parameterized gradient descent[END_REF].

Reformulation with probability measures

We now define by P(W) the set of probability measures on W = R d+1 . We can rewrite Eq. (3.1) as

h = W Ψ(w)dµ(w),
with µ = 1 m m j=1 δ w j an average of Dirac measures at each w 1 , . . . , w m . Following a physics analogy, we will refer to each w j as a particle. When the number m of particles grow, then the empirical measure 1 m m j=1 δ w j may converge in distribution to a probability measure with a density, often referred to as a mean field limit. Our main reformulation will thus be to consider an optimization problem over probability measures.

The optimization problem we are faced with is equivalent to

inf µ∈P(W) R W Ψ(w)dµ(w) , (3.2)
with the constraint that µ is an average of m Dirac measures. In this paper, following a long line of work in statistics and signal processing [START_REF] Barron | Universal approximation bounds for superpositions of a sigmoidal function[END_REF][START_REF] Kurkova | Bounds on rates of variable-basis and neural-network approximation[END_REF], we consider the optimization problem without this constraint, and relate optimization algorithms for finite but large m (thus acting on W = (w 1 , . . . , w m ) in W m ) to a well-defined algorithm in P(W).

Note that we now have a convex optimization problem, with a convex objective in µ over a convex set (all probability measures). However, it is still an infinite-dimensional space that requires dedicated finite-dimensional algorithms. In this paper we focus on gradient descent on w, which corresponds to standard practice in neural networks (e.g., back-propagation). For algorithms based on classical convex optimization algorithms such as the Frank-Wolfe algorithm, see [START_REF] Bach | Breaking the curse of dimensionality with convex neural networks[END_REF].

From gradient descent to gradient flow

Our general goal is to study the gradient descent recursion on W = (w 1 , . . . , w m ) ∈ W m , defined as

W k = W k-1 -γm∇G(W k-1 ), (3.3)
with

G(W ) = R h(•, w 1 , . . . , w m ) = R 1 m m j=1 Ψ(w j ) .
In the context of neural networks, this is exactly the back-propagation algorithm. We include the factor m in the step-size to obtain a well-defined limit when m tends to infinity (see Section 3.3).

For convenience in the analysis, we look at the limit when the step-size γ goes to zero. If we consider a function V : R → W m , with values V (kγ) = W k at t = kγ, and we interpolate linearly between these points, then, we obtain exactly the standard Euler discretization of the ordinary differential equation (ODE) [START_REF] Suli | An introduction to numerical analysis[END_REF]:

V = -m∇G(V ). (3.4)
This gradient flow will be our main focus in this paper. As highlighted above, and with extra regularity assumptions, it is the limit of the gradient recursion in Eq. ( 3.3) for vanishing step-sizes γ. Moreover, under appropriate conditions, stochastic gradient descent, where we only observe an unbiased noisy version of the gradient, also leads in the limit γ → 0 to the same ODE [START_REF] Kushner | Stochastic approximation and recursive algorithms and applications[END_REF]. This allows to apply our results to probability distributions of the data (x, y) which are not the observed empirical distribution, but the unseen test distribution, where the stochastic gradients come from the gradient of the loss from a single observation. Three questions now emerge:

1. What is the limit (if any) of the gradient flow in Eq. (3.4) when the number of particles m gets large?

2. Where can the gradient flow converge to?

3. Can we ensure a good generalization performance when the number of parameters grows unbounded?

In this paper, we will focus primarily in the next sections on the first two questions, and tackle the third question in Section 5.

Wasserstein gradient flow

Above, we have described a general framework where we want to minimize a function F defined on probability measures:

F (µ) = R W Ψ(w)dµ(w) , (3.5)
with an algorithm minimizing

G(w 1 , . . . , w m ) = R 1 m m j=1 Ψ(w j ) through the gradient flow Ẇ = -m∇G(W ), with W = (w 1 , . . . , w m ).
As shown in a series of works concerned with the infinite width limit of two-layer neural networks [START_REF] Nitanda | Stochastic particle gradient descent for infinite ensembles[END_REF][START_REF] Chizat | On the global convergence of gradient descent for over-parameterized models using optimal transport[END_REF][START_REF] Mei | A mean field view of the landscape of two-layer neural networks[END_REF][START_REF] Sirignano | Mean field analysis of neural networks: A law of large numbers[END_REF][START_REF] Rotskoff | Parameters as interacting particles: long time convergence and asymptotic error scaling of neural networks[END_REF], this converges to a well-defined mathematical object called a Wasserstein gradient flow [START_REF] Ambrosio | Gradient flows: in metric spaces and in the space of probability measures[END_REF]. This is a gradient flow derived from the Wasserstein metric on the set of probability measures, which is defined as [START_REF] Santambrogio | Optimal transport for applied mathematicians[END_REF],

W 2 (µ, ν) 2 = inf γ∈Π(µ,ν) v -w 2 2 dγ(v, w),
where Π(µ, ν) is the set of probability measures on W × W with marginals µ and ν. In a nutshell, the gradient flow is defined as the limit when γ tends to zero of the extension of the following discrete time dynamics:

µ(t + γ) = inf ν∈P(W) F (ν) + 1 2γ W 2 (µ(t), ν) 2 .
When applying such a definition in a Euclidean space with the Euclidean metric, we recover the usual gradient flow μ = -∇F (µ), but here with the Wasserstein metric, this defines a specific flow on the set of measures. When the initial measure is a weighted sum of Diracs, this is exactly asymptotically (when γ → 0) equivalent to backpropagation. When initialized with an arbitrary probability measure, we obtain a partial differential equation (PDE), satisfied in the sense of distributions. Moreover, when the sum of Diracs converges in distribution to some measure, the flow converges to the solution of the PDE. More precisely, assuming Ψ : R d+1 → F a Hilbert space, and ∇R(h) ∈ F the gradient of R, we consider the mean potential

J(w|µ) = Ψ(w), ∇R W Ψ(v)dµ(v) . (3.6)
The PDE is then the classical continuity equation:

∂ t µ t (w) = div(µ t (w)∇J(w|µ t )), (3.7)
which is understood in the sense of distributions. The following result formalizes this behavior (see [START_REF] Chizat | On the global convergence of gradient descent for over-parameterized models using optimal transport[END_REF] for details and a more general statement).

Theorem 1 Assume that R : F → [0, +∞[ and Ψ : W = R d+1 → F are (Fréchet) differentiable with Lipschitz differentials, and that R is Lipschitz on its sublevel sets. Consider a sequence of initial weights (w j (0)) j≥1 contained in a compact subset of W and let µ t,m := 1 m m j=1 w j (t) where (w 1 (t), . . . , w m (t)) solves the ODE (3.4). If µ 0,m weakly converges to some µ 0 ∈ P(W) then µ t,m weakly converges to µ t where (µ t ) t≥0 is the unique weakly continuous solution to (3.7) initialized with µ 0 .

In the following section, we will study the solution of this PDE (i.e., the Wasserstein gradient flow), interpreting it as the limit of the gradient flow in Eq. (3.4), when the number of particles m tend to infinity.

Global convergence

We consider the Wasserstein gradient flow defined above, which leads to the PDE in Eq. (3.7). Our goal is to understand when we can expect that when t → ∞, µ t converges to a global minimum of F defined in Eq. (3.5). Obtaining a global convergence result is not out of the question because F is a convex functional defined on the convex set of probability measures. However, it is non trivial because with our choice of the Wasserstein geometry on measures, which allows an approximation through particles, the flow has some stationary points which are not the global optimum (see examples in Section 4.4).

We start with an informal general result without technical assumptions before stating a formal simplified result.

Informal result

In order to avoid too many technicalities, we first consider an informal theorem in this paper and refer to [START_REF] Chizat | On the global convergence of gradient descent for over-parameterized models using optimal transport[END_REF] for a detailed set of technical assumptions (in particular smoothness assumptions). This leads to the informal theorem:

Theorem 2 (Informal) If the support of the initial distribution includes all directions in R d+1 , and if the function Ψ is positively 2-homogeneous then if the Wasserstein gradient flow weakly converges to a distribution, it can only be to a global optimum of F .

In [START_REF] Chizat | On the global convergence of gradient descent for over-parameterized models using optimal transport[END_REF] another version of this result that allows for partial homogeneity (e.g., with respect to a subset of variables) of degree 1 is proven, at the cost of a more technical assumption on the initialization. For neural networks, we have Ψ(w j )(x) = mθ 2 (j) • σ θ 1 (•, j) x , and this more general version applies. For the classical ReLU activation function u → max{0, u}, we get a positively 2-homogeneous function, as required in the previous statement. A simple way to spread all directions is to initialize neural network weights from Gaussian distributions, which is standard in applications [START_REF] Goodfellow | Deep learning[END_REF].

From qualitative to quantitative results? Our result states that for infinitely many particles, we can only converge to a global optimum (note that we cannot show that the flow always converges). However, it is only a qualitative result in comparison with what is known for convex optimization problems in Section 2.2:

• This is only for m = +∞, and we cannot provide an estimation of the number of particles needed to approximate the mean field regime that is not exponential in t (see such results e.g. in [START_REF] Mei | Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit[END_REF]).

• We cannot provide an estimation of the performance as the function of time, that would provide an upper bound on the running time complexity.

Moreover, our result does not apply beyond a single hidden layer, and understanding the nonlinear infinite width limits for deeper networks is an important research area [START_REF] Nguyen | A rigorous framework for the mean field limit of multilayer neural networks[END_REF][START_REF] Araújo | A mean-field limit for certain deep neural networks[END_REF][START_REF] Fang | Modeling from features: a mean-field framework for over-parameterized deep neural networks[END_REF][START_REF] Hanin | Finite depth and width corrections to the neural tangent kernel[END_REF][START_REF] Sirignano | Mean field analysis of deep neural networks[END_REF][START_REF] Wojtowytsch | On the Banach spaces associated with multi-layer relu networks: Function representation, approximation theory and gradient descent dynamics[END_REF][START_REF] Yang | Feature learning in infinite-width neural networks[END_REF].

From informal to formal results. Beyond the lack of quantitative guarantees, obtaining a formal result requires regularity and compactness assumptions which are not satisfied for the classical ReLU activation function u → max{0, u}, which is not differentiable at zero (a similar result can be obtained in this case but under stronger assumptions on the data distribution and the initialization [START_REF] Wojtowytsch | On the convergence of gradient descent training for two-layer ReLU-networks in the mean field regime[END_REF][START_REF] Chizat | Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss[END_REF]). In the next section, we will consider a simplified formal result, with a detailed proof.

Simplified formal result

In order to state a precise result, we will cast the flow on probability measures on W = R d+1 to a flow on measures on the unit sphere

S d = {w ∈ R d+1 , w 2 = 1}.
This is possible when the function Ψ is positively 2-homogeneous on W = R d+1 , that is, such that Ψ(λw) = λ 2 Ψ(w) for λ > 0. We can use homogeneity by reparameterizing each particle w j in polar coordinates as w j = r j η j , with r j ∈ R and η j ∈ S d .

Using homogenetity, we have a prediction function:

h = 1 m m j=1 Ψ(w j ) = 1 m m j=1 r 2 j Ψ(η j ).
Moreover the function J defined in Eq. (3.6) is also 2-homogeneous, and its gradient then 1homogeneous. The flow from Eq. (3.4), can be written

ẇj = -∇J(w j |µ) with µ = 1 m m i=1 δ w j .
A short calculation shows that the flow ṙj = -2r j J(η

j |ν) ηj = -(I -η j η j )∇J(η j |ν) with ν = 1 m m i=1 r 2 j δ η j , (4.1)
leads to exactly the same dynamics. Indeed, by homogeneity of Ψ, the two definitions of µ and ν (through the w j 's, or the η j 's and r j 's) lead to the same functions J(•|µ) and J(•|ν), and we get ẇj = ṙj η j + r j ηj = -2r j J(η j |ν)η j -r j (I -η j η j )∇J(η j |ν) = -r j ∇J(η j |ν) -r j 2J(η j |ν) -η j ∇J(η j |ν) η j = -∇J(w j |µ), because w → ∇J(w|ν) is 1-homogeneous and by the Euler identity for the 2-homogeneous function w → J(w|ν) = J(w|µ). Moreover, the flow defined in Eq. (4.1) is such that η j remains on the sphere S d . We will study this flow under the assumption that the function Ψ is sufficient regular, which excludes ReLU neural networks, but makes the proof easier (see more details in [START_REF] Chizat | Sparse optimization on measures with over-parameterized gradient descent[END_REF]).

We first derive a PDE analogous to Eq. (3.7). We consider a smooth test function f : S d → R, and the quantity

a = S d f (η)dν(η) = 1 m m j=1 r 2 j f (η j ).
We have

ȧ = 1 m m j=1 2r j ṙj f (η j ) + 1 m m j=1 r 2 j ∇f (η j ) ηj = - 1 m m j=1 4r 2 j J(η j |ν)f (η j ) - 1 m m j=1 r 2 j ∇f (η j ) (I -η j η j )∇J(η j |ν) = -4 S d f (η)J(η|ν)dν(η) - S d ∇f (η|ν) (I -ηη )∇J(η|ν)dν(η). (4.2)
This exactly shows that we have the PDE for the density ν t at time t

∂ t ν t (η) = -4J(η|ν t ) + div(ν t (η)∇J(η|ν t )) (4.3)
satisfied in the sense of distributions. We can now state our main result.

Theorem 3 Assume the function Ψ : S d → F is d-times continuously differentiable. Assume ν 0 is a nonnegative measure on the sphere S d with finite mass and full support. Then the flow defined in Eq. (4.3) is well defined for all t 0. Moreover, if ν t converges weakly to some limit ν

∞ , then ν ∞ is a global minimum of the function ν → F (ν) = R S d
Ψ(η)dν(η) over the set of nonnegative measures.

Proof of Theorem 3

The global optimality conditions for minimizing the convex functional F is that on the support of ν ∞ then J(η|ν ∞ ) = 0, while on the entire sphere J(η|ν ∞ ) 0. The proof, adapted from [START_REF] Chizat | Sparse optimization on measures with over-parameterized gradient descent[END_REF], then goes a follows:

• The existence and uniqueness of the flow (ν t ) t≥0 can be proved by using the equivalence with a Wasserstein gradient flow (µ t ) t≥0 in P(R d+1 ) and the theory of Wasserstein gradient flows [START_REF] Ambrosio | Gradient flows: in metric spaces and in the space of probability measures[END_REF]. As a matter of fact, (ν t ) t≥0 is itself a gradient flow for a certain metric between nonnegative measures that is, in a certain sense, the inf-convolution between the Wasserstein and the Hellinger metric, see the discussion in [START_REF] Chizat | Sparse optimization on measures with over-parameterized gradient descent[END_REF].

• The flow ν t has a full support at all time t. This can be deduced from the representation of the solutions to Eq. (4.3) as

ν t = X(t, •) # ν 0 exp -4 t 0 J(X(s, •)|ν s )ds ,
where X : [0, +∞[×S d → S d is the flow associated to the time-dependent vector field -∇J(• | ν t ), i.e., it satisfies X(0, η) = η and d dt X(t, η) = -∇J(X(t, η)|ν t ) for all η ∈ S d , see, e.g., [START_REF] Maniglia | Probabilistic representation and uniqueness results for measure-valued solutions of transport equations[END_REF]. Under our regularity assumptions, standard stability results for ODEs guarantee that at all time t, X(t, •) is a diffeomorphism of the sphere. Thus ν t is the image measure (this is what the "sharp" notation stands for) by a diffeomorphism of a measure of the form ν 0 exp(. . . ) which has full support and thus ν t has full support.

• We assume that the flow converges to some measure ν ∞ (which could be singular). From Eq. (4.1), this imposes by stationarity of ν ∞ that J(η|ν ∞ ) = 0 on the support of ν ∞ , but nothing is imposed beyond the support of ν ∞ (and we need non-negativity of J(η|ν ∞ ) for all η ∈ S d ).

In order to show that min η∈S d J(η|ν ∞ ) 0, we assume that it is strictly negative and will obtain a contradiction. We first need a v < 0 such that v > min η∈S d J(η|ν ∞ ), and the gradient ∇J(η|ν ∞ ) does not vanish on the v-level-set {η ∈ S d , J(η|ν ∞ ) = v} of J(•|ν ∞ ). Such a v exists because of Morse-Sard lemma which applies because under our assumptions, J(•|ν) is d-times continuously differentiable for any finite nonnegative measure ν.

We then consider the set K = η ∈ S d , J(η|ν ∞ ) v , which has some boundary ∂K, such that the gradient ∇J(η|ν ∞ ) has strictly positive dot-product with an outward normal vector to the level set at η ∈ ∂K.

Since ν t converges weakly to ν ∞ , there exists t 0 > 0 such that for all t t 0 , sup η∈K J(η|ν t ) < v/2, while on the boundary ∇J(η|ν ∞ ) has non-negative dot-product with an outward normal vector. This means that for all t > t 0 , applying Eq. (4.2) to the indicator function of K, if a t = ν t (K), a (t) -4 sup η∈K J(η|ν t )a(t).

By the previous point, a(t 0 ) > 0 and thus, by Grönwall's lemma, a(t) diverges, which is a contradiction with the convergence of ν t to ν ∞ .

Experiments

In order to illustrate1 the global convergence result from earlier sections, we consider a supervised learning problem on R2 , with Gaussian input data x, and output data given by a "teacher" neural network

y = m 0 j=1 θ 2 (j) max{θ 1 (:, j) x, 0}
for some finite m 0 and weights θ 1 and θ 2 . We consider R(h) the expected square loss and stochastic gradient descent with fresh new samples (x i , y i ) and a small step-size. We consider several number m of hidden neurons, to assess when the original neurons can be recovered. In Figure 2, for large m (e.g., m = 100 or m = 1000), all learned neurons converge to the neurons that generated the function which is in accordance with our main global convergence result (note that in general, recovering the neurons of the teacher is not a necessary condition for optimality, but it is always sufficient), while for m = 5 > m 0 , where the global optimum will lead to perfect estimation, we may not recover the global optimum with a gradient flow. An interesting open question is to characterize mathematically the case m = 20, where we obtain the global optimum with moderate m.

In Figure 3, we consider several random initializations and random "teacher" networks and compute the generalization performance of the neural network after optimization. We see that for large m, good performance is achieved, while when m is too small, local minima remain problematic. This experiment suggests that the probability of global convergence quickly tends to 1 as m increases beyond m 0 in this setting, even in moderately high dimension.

Generalization guarantees and implicit bias for overparameterized models

A shown above, overparameterization -which takes the form of large number of hidden neurons in our context -is a blessing for optimization, as it allows to ensure convergence to a global minimizer. When stochastic gradient descent with fresh observations at each iteration is used, then the predictor will converge to the optimal predictor (that is, it will minimize the performance on unseen data), but will do so potentially at a slow speed, and with the need for many observations. In this context, overparameterization does not lead to overfitting, but may rather underfit.

In practice, several passes over a finite amount of data (n observations) are used, and then overparameterization can in principle lead to overfitting. Indeed, among all predictors that will perfectly predict the training data, some will generalize, some will not. In this section, we show that the predictor obtained after convergence of the gradient flow can in certain cases be characterized precisely.

To obtain the simplest result, following [START_REF] Gunasekar | Implicit regularization in matrix factorization[END_REF][START_REF] Soudry | The implicit bias of gradient descent on separable data[END_REF][START_REF] Gunasekar | Characterizing implicit bias in terms of optimization geometry[END_REF] this will be done for binary classification problems with the logistic loss. We will first review the implicit bias for linear models before considering neural networks.

Implicit bias for linear logistic regression

In this section, we consider a linear model h(x, θ) = θ Φ(x) and we consider the minimization of the unregularized empirical risk with the logistic loss, that is,

R(θ) = 1 n n i=1 log 1 + exp(-y i θ Φ(x i ) . (5.1)
We consider a separable problem where there exists a linear function in Φ(x), θ Φ(x) such that y i θ Φ(x i ) > 0 for all i ∈ {1, . . . , n}. By rescaling, we may equivalently assume that there exists

θ ∈ R d such that ∀i ∈ {1, . . . , n}, y i θ Φ(x i ) 1.
This means that the objective function in Eq. (5.1) has an infimal value of zero, which is not attained for any θ, since it is strictly positive. However, taking any θ that separates the data as above, it holds that R(tθ) converges towards 0 as t tends to infinity. There are thus in general an infinite number of directions towards which θ can tend to reach zero risk. It turns out that gradient descent selects a particular one: the iterate of gradient descent will diverge, but its direction (that is the element of the sphere it is proportional to) will converge [START_REF] Soudry | The implicit bias of gradient descent on separable data[END_REF] to the direction of a maximum margin classifier defined as [START_REF] Vapnik | On a perceptron class[END_REF] a solution to min

θ∈R d θ 2
2 subject to ∀i ∈ {1, . . . , n}, y i θ Φ(x i ) 1.

(

The optimization problem above has a nice geometric interpretation (see Figure 4). These classifiers with a large margin has been shown to have favorable generalization guarantees in a wide range of contexts [START_REF] Koltchinskii | Empirical margin distributions and bounding the generalization error of combined classifiers[END_REF].

Extension to two-layer neural networks

We will now extend this convergence of gradient descent to a minimum norm classifier beyond linear models. We consider the minimization of the logistic loss

1 n n i=1 log 1 + exp(-y i h(x i ) ,
where h(x) = 1 m m j=1 θ 2 (j) max{θ 1 (:, j) x, 0} is a two-layer neural network. We will consider two regimes: [START_REF] Akiyama | On learnability via gradient method for two-layer relu neural networks in teacher-student setting[END_REF] where only the output weights θ 2 (j), j = 1, . . . , m are optimized, and (2) where all Figure 4: Geometric interpretation of Eq. ( 5.2) with a linearly separable binary classification problem in two dimensions (with each observation represented by one of the two labels × or •): among all separating hyperplanes going through zero, the one with the largest minimal distance from observations to the hyperplane will be selected. weights are optimized. In these two situations, we will let the width m go to infinity and consider the infinite-dimensional resulting flows. As shown in the previous section, when they converge, these flows converge to the global optimum of the objective function. But in the separable classification setting, the functions h should diverge. We essentially characterize towards which directions they diverge, by identifying the norms that are implicitly minimized [START_REF] Chizat | Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss[END_REF].

Kernel regime

In this section, we consider random input weights θ 1 (:, j), sampled from the uniform distribution on the sphere, and kept fixed throughout the optimization procedure. In other words, we only run the gradient flow with respect to the output weights θ 2 ∈ R m .

Since the model is a linear model with feature vectors in m dimensions with components Φ(x) j = 1 √ m max{θ 1 (:, j) x, 0}, we can apply directly the result above from [START_REF] Soudry | The implicit bias of gradient descent on separable data[END_REF], and the resulting classifier will minimize implicitly θ 2 2 2 , that is the direction of θ 2 will tend to a maximum margin direction.

In order to study the situation when the number of features m tends to infinity, it is classical within statistics and machine learning to consider the kernel function k :

R d × R d → R defined as k(x, x ) = Φ(x) Φ(x ) = 1 m m j=1
max{θ 1 (:, j) x, 0} max{θ 1 (:, j) x , 0}.

When m tends to infinity, the law of large number implies that k(x, x ) tends to k(x, x ) = E max{η x, 0} max{η x , 0} , for η uniformly distributed on the sphere.

Thus, we should expect that in the overparameterized regime, the predictor behaves like predictors associated with the limiting kernel function [START_REF] Neal | Bayesian learning for neural networks[END_REF][START_REF] Rahimi | Random features for large-scale kernel machines[END_REF]. It turns out that the kernel k can be computed in closed form [START_REF] Cho | Kernel methods for deep learning[END_REF], and that the reproducing kernel Hilbert space (RKHS) functional norm • associated to the kernel k is well understood (see below for a formula that defines it). In particular, this norm is infinite unless the function is at least d/2-times differentiable [START_REF] Bach | Breaking the curse of dimensionality with convex neural networks[END_REF], and thus very smooth in high dimension (this is to be contrasted with the fact that each individual neuron leads to a non-smooth function). We thus expect smooth decision boundaries at convergence (see experiments below). This leads to the following result (see details in [START_REF] Chizat | Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss[END_REF]):

Theorem 4 (informal) When m, t → +∞ (limits can be interchanged), the predictor associated to the gradient flow converges (up to normalization) to the function in the RKHS that separates the data with minimum RKHS norm • , that is the solution to

min f f 2 subject to ∀i ∈ {1, . . . , n}, y i f (x i ) 1.
Note that the minimum RKHS norm function can also be found by using the finite-dimensional representation f (x) = n i=1 α i k(x, x i ) and minimizing n i,j=1 α i α j k(x i , x j ) under the margin constraint, which is a finite-dimensional convex optimization problem.

A striking phenomenon is the absence of catastrophic overfitting, where the observed data are perfectly classified but with a very irregular function that would potentially not generalize well. Despite the strong overparameterization, the classifier selected by gradient descent can be shown to generalize through classical results from maximum margin estimation. See [START_REF] Mei | The generalization error of random features regression: Precise asymptotics and the double descent curve[END_REF] for a related result where the performance as a function of m, and not only for infinite m, is considered in special settings. We will see a similar behavior when optimizing the two layers, but with a different functional norm.

Feature learning regime

We now consider the minimization with respect to both input and output weights. This will correspond to another functional norm that will not anymore be an RKHS norm, and will allow for more adaptivity, where the learned function can exhibit finer behaviors.

We first provide an alternative formulation of the RKHS norm as [4]

f 2 = inf a(•) S d-1 |a(η)| 2 dτ (η) such that f (x) = S d-1 (η x) + a(η)dτ (η),
where the infimum is taken over all square-integrable functions on the sphere S d-1 , and τ is the uniform probability measure on the sphere. This formulation highlights that functions in the RKHS combine infinitely many neurons. We can then define the alternative variation norm [START_REF] Kurkova | Bounds on rates of variable-basis and neural-network approximation[END_REF] as

Ω(f ) = inf a(•) S d-1 |a(η)|dτ (η) such that f (x) = S d-1 (η x) + a(η)dτ (η),
where the infimum is now taken over all integrable functions on S d-1 . Going from squared L 2norms to L 1 -norms enlarges the space by adding non-smooth functions. For example, a single neuron corresponds to a(•)dτ (•) tending to a Dirac measure at a certain point, and thus has a finite variation norm. This leads to the following result (see details and full set of assumptions in [START_REF] Chizat | Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss[END_REF]).

Theorem 5 (informal) When m, t → +∞, if the predictor associated to the gradient flow converges (up to normalization), then the limit is the function that separates the data with minimum variation norm Ω(f ), that is the solution to min f Ω(f ) subject to ∀i ∈ {1, . . . , n}, y i f (x i ) 1.

Compared to the RKHS norm result, there is no known finite-dimensional convex optimization algorithms to efficiently obtain the minimum variation norm algorithm. Moreover, the choice of an L 1 -norm has a sparsity-inducing effect, where the optimal a(•)dτ (•) will often corresponds to singular measure supported by a finite number of elements of the sphere. These elements can be seen as features learned by the algorithm: neural networks are considered as methods that learn representations of the data, and we provide here a justification with a single hidden layer. Such feature learning can be shown to lead to improved prediction performance in a series of classical situations, such as when the optimal function only depends on a few of the d original variables [START_REF] Bach | Breaking the curse of dimensionality with convex neural networks[END_REF][START_REF] Chizat | Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss[END_REF].

Experiments

In this section, we consider a large ReLU network with m = 1000 hidden units, and compare the implicit bias and statistical performances of training both layers -which leads to a max margin classifier with the variation norm -versus the output layer -which leads to max margin classifier in the RKHS norm. These experiments are reproduced from [START_REF] Chizat | Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss[END_REF].

Setting. Our data distribution is supported on [-1/2, 1/2] d and is generated as follows. In dimension d = 2, the distribution of input variables is a mixture of k 2 uniform distributions on disks of radius 1/(3k -1) on a uniform 2-dimensional grid with step 3/(3k -1), see Figure 6(a) for an illustration with k = 3. In dimension larger than 2, all other coordinates follow a uniform distribution on [-1/2, 1/2]. Each cluster is then randomly assigned a class in {-1, +1}.

Low dimensional illustrations. Figure 5 illustrates the differences in the implicit biases when d = 2. It represents a sampled training set and the resulting decision boundary between the two classes for 4 examples. The variation norm max-margin classifier is non-smooth and piecewise affine, which comes from the fact that the L 1 -norm favors sparse solutions. In contrast, the maxmargin classifier for the RKHS norm has a smooth decision boundary, which is typical of learning in a RKHS.

Performance. In higher dimensions, we observe the superiority of training both layers by plotting the test error versus m or d on Figure 6(b) and 6(c). We ran 20 independent experiments with k = 3 and show with a thick line the average of the test error P(yf (x) < 0) after training. For each m, we ran 30 experiments using fresh random samples from the same data distribution.

Discussion

In this paper, we have presented qualitative convergence guarantees for infinitely-wide two layer neural networks. These were obtained with a precise scaling -in the number of neurons -of the both layers output layer prediction function, the initialization and the step-size used in the gradient flow. With those scalings, the mean-field limit exhibits feature learning capabilities, as illustrated in binary classification where precise functional spaces could be used to analyse where optimization converges to. However, this limit currently does not lead to quantitative guarantees regarding the number of neurons or the convergence time, and obtaining such guarantees remains an open problem. This is an active area of research with, in particular, recent results concerning the local convergence [START_REF] Zhou | A local convergence theory for mildly over-parameterized two-layer neural network[END_REF][START_REF] Akiyama | On learnability via gradient method for two-layer relu neural networks in teacher-student setting[END_REF][START_REF] Chizat | Sparse optimization on measures with over-parameterized gradient descent[END_REF] or global convergence under strong assumption on the data [START_REF] Li | Learning over-parametrized two-layer neural networks beyond NTK[END_REF]. Moreover, extending this analysis to more than a single hidden layer or convolutional networks remains difficult. Different scalings lead to different behaviors [START_REF] Chizat | On lazy training in differentiable programming[END_REF]. In particular, there is a scaling for which the limit behaves as a kernel method (even though all layers are trained, and not just the output layer) leading to another RKHS norm with a larger space than the one from Section 5.3, see [START_REF] Jacot | Neural tangent kernel: Convergence and generalization in neural networks[END_REF]. While not leading to representation learning, extensions to deeper networks are possible with this scaling and provide one of few optimization and statistical guarantees for these models. Some recent progress has been made in the categorization of the various possible scalings for deep networks [START_REF] Yang | Feature learning in infinite-width neural networks[END_REF], and this emerging general picture calls for a large theoretical effort to understand the asymptotic behaviors of wide neural networks.

Figure 2 :

 2 Figure 2: Gradient flow on a two-layer ReLU neural network with respectively m = 5, m = 20, m = 100 and m = 1000. The position of the particles is given by |θ 2 (j)| • θ 1 (•, j) and the color depends on the sign of θ 2 (j). The dashed directions represent the neurons of the network that generates the data distribution (with m 0 = 4). The unit circle, where the particles are initialized, is plotted in black and the radial axis is scaled by tanh to improve lisibility.

Figure 3 :

 3 Figure 3: SGD on the square loss in the "teacher-student" setting (10 4 iterations, batch size 100, learning rate 0.005, d = 100, the teacher has m 0 = 10 neurons). (left) Risk (expected square loss) after training as a function of m over 30 random repetitions; (right) Success rate as a function of m over 30 repetitions (success means that the risk after training is below 10 -3 ).

Figure 5 :

 5 Figure 5: Comparison of the implicit bias of training (top) both layers versus (bottom) only the output layer for wide two-layer ReLU networks with d = 2 and for 4 different random training sets.

  Test error vs. d

Figure 6 :

 6 Figure 6: (a) Projection of the data distribution on the two first dimensions, (b) test error as a function of n with d = 15, (c) test error as a function of d with = 256.

The code to reproduce Figures

and

is available on this webpage https://github.com/lchizat/2021-exp-ICM.

Acknowledgements. This work was funded in part by the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avenir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute). We also acknowledge support the European Research Council (grant SEQUOIA 724063).