Keywords: stochastic approximation, avoidance of traps, nonsmooth optimization, semialgebraic, o-minimality, stratifications, stochastic subgradient descent, ADAM, Nesterov's accelerated gradient descent, adaptive algorithms with momentum

Je tenais tout d'abord à remercier mes directeurs de thèse, Walid et Pascal, de m'avoir pris en thèse et soutenu pendant ces trois ans. Vous m'avez beaucoup apporté par vos conseils et soutiens, que ce soit sur le plan professionnel ou sur le plan humain. Je vous remercie tout particulièrement pour votre confiance dans ma manière de travailler (je sais que je n'étais pas l'élève le plus assidu) ainsi que dans mes choix des sujets de recherche. Cette confiance m'a vraiment permis de me sentir libre pendant ces trois années, ça a énormément compté pour moi; j'espère que ça a donné quelque chose de bien.

Je remercie vivement Jérôme Bolte et Aris Daniilidis pour avoir accepté de rapporter cette thèse. Il m'est dur de sous-estimer l'influence qu'ont eue vos travaux concernant l'optimisation semi-algébrique/o-minimale sur ma recherche et c'est un véritable honneur de pouvoir vous présenter mes (quelques) résultats.

Je voudrais remercier Michel Benaïm, Éric Moulines et Gersende Fort d'avoir accepté de participer à ma soutenance. Vos travaux ont eu une influence considérable sur la partie probabiliste de ma thèse. En particulier la méthode de l'ODE/inclusion différentielle développée par Michel Benaïm est à la base de tous(!) mes papiers publiés durant ces trois années. Éric et Gersende, on commence déjà à travailler ensemble et je suis impatient de collaborer avec vous tout prochainement.

Je remercie les gens de Télécom et de Marnes que j'ai croisés tout au long de ma thèse. J'ai toujours trouvé dans ces labos éloignés un accueil chaleureux. Merci Anas mon co-thésard/co-bureau/(co-)docteur pour ces trois (co-)années et tes qualités humaines. Je n'ai aucun doute que nos chemins se recroiseront encore.

Je ne listerai pas ici tous mes amis car un ordre ne leur ferait pas justice. Néanmoins, vous savez tous que votre présence, dans des moments heureux et moins, est ce qui me permet d'avancer. Les amitiés que j'ai forgées font partie des choses qui m'ont le mieux réussi dans cette vie. Cette formule, maladroite mais sincère (ou l'inverse), est ce que j'ai trouvé de mieux pour exprimer mes sentiments.

Je remercie mes professeurs de prépa Jean-Christophe Feauveau et Éric Desmeules. Vous étiez les premiers à me montrer comment se construit un raisonnement scientifique, l'importance de l'intuition géométrique et, probablement, étiez les meilleurs professeurs que je n'ai jamais eus. Si aujourd'hui j'entame une carrière de chercheur c'est sans doute grâce à vous.

Je remercie Klo$ iqik qui est toute tchik tchik, qui m'accepte comme je suis et que j'aime beaucoup beaucoup.

Last but not least, je remercie ma famille pour leur amour, leur présence et leur soutien inconditionnel à tout moment.

Quelques problèmes en optimisation non convexe et stochastique Résumé: Le sujet de cette thèse est l'analyse de divers algorithmes stochastiques visant à résoudre un problème d'optimisation non convexe. Nous commençons par un problème d'optimisation lisse en analysant une famille d'algorithmes adaptatifs avec moments qui comprend entre autres ADAM et la descente de gradient accélérée de Nesterov. La convergence et la fluctuation des itérés sont établies. Un résultat général d'évitement des pièges pour les algorithmes stochastiques sous-tendus par une équation différentielle non autonome est présenté. Il est appliqué pour établir la non-convergence des itérés aux points-selles. La suite du manuscrit est consacrée au cas où la fonction que l'on cherche à minimiser est non lisse. La plupart de nos résultats dans cette partie s'appliquent aux fonctions définissables dans une structure o-minimale. Tout d'abord, nous analysons la version à pas constants de la descente de sous-gradient stochastique (SGD) et montrons que ses itérés convergent en grande probabilité vers l'ensemble des points critiques. Deuxièmement, nous montrons que chaque point critique d'une fonction Lipschitz, définissable, générique se trouve sur une variété active, satisfaisant une condition de Verdier et d'angle et est soit un minimum local, un point selle actif ou un point critique fortement répulsif. Nous montrons, sous des conditions légères sur les perturbations, que le SGD évite les deux derniers types de points. Une amélioration de la formule de projection pour les fonctions définissables, donnant une condition de type Lipschitz sur ses sous-gradients de Clarke, est présentée. Enfin, nous établissons un phénomène d'oscillation des itérés du SGD et de ses extensions proximales.

Mots clés: optimisation stochastique, évitement des pièges, optimisation non lisse, semi-algébrique, o-minimalité, stratifications, descente de sous-gradient stochastique, ADAM, algorithmes adaptatifs avec moments Some Problems in Nonconvex Stochastic Optimization Abstract: The subject of this thesis is the analysis of several stochastic algorithms in a nonconvex setting. The aim is to prove and characterize their convergence. First, we study a smooth optimization problem, analyzing a family of adaptive algorithms with momentum which includes the widely used ADAM and Nesterov's accelerated gradient descent. Convergence and fluctuation of the iterates are established. A general avoidance of traps result for stochastic algorithms underlined by a nonautonomous differential equation is presented and applied to establish the nonconvergence of the iterates to saddle points. The rest of the manuscript is devoted to the case where the function that we seek to minimize is nonsmooth. Most of our results in this part apply to functions definable in an o-minimal structure. Firstly, we analyze the constant step version of the stochastic subgradient descent (SGD) and show that the iterates converge with high probability to the set of critical points. Secondly, we show that every critical point of a generic, definable, locally Lipschitz continuous function is lying on an active manifold, satisfying a Verdier and an angle condition and is either a local minimum, an active strict saddle or a sharply repulsive critical point. We show that, under mild conditions on the perturbation sequence, the SGD escapes active strict saddles and sharply repulsive critical points. An improvement of the projection formula for definable functions, giving a Lipschitz-like condition on its Clarke subgradients is presented and is of independent interest. Finally, we establish an oscillation phenomena of the iterates of the SGD and its proximal extensions. v Résumé substantiel en français L'objet de cette thèse est l'étude de divers algorithmes stochastiques visant à résoudre des problèmes d'optimisation non convexe. L'objectif dans chacun des cas est de démontrer et de caractériser la convergence de l'algorithme vers l'ensemble des points critiques de la fonction à minimiser.

Le chapitre 3, le seul à aborder un problème d'optimisation lisse, analyse une famille d'algorithmes adaptatifs et à moment qui comprend entre autres ADAM et la descente de gradient accélérée de Nesterov. En appliquant la méthode de l'ODE, qui consiste à voir ces algorithmes comme une discrétisation d'Euler d'une équation différentielle (ED), nous montrons la convergence des itérés envers les points critiques de la fonction à minimiser. La difficulté principale de l'analyse est que l'équation différentielle mentionnée est non autonome. Nous établissons dans certains cas un phénomène de fluctuation des itérés sous forme d'un théorème central limite. Enfin nous abordons la question d'évitement des pièges. Cette question est importante car l'ensemble des points critiques d'une fonction est, dans le cas non-convexe, généralement strictement plus large que l'ensemble des minimiseurs (locals) de la fonction. Cette question avait été abordée auparavant pour des algorithmes sous-tendus par une ED autonome, l'approche étant basée sur l'application du théorème de la variété invariante de Poincaré. En utilisant la version non autonome de ce théorème nous établissons un résultat d'évitement de piège général pour tout algorithme discrétisation d'une ED non autonome. Enfin, nous appliquons ce résultat aux algorithmes étudiés pour montrer la non-convergence presque sûre des itérés envers les points selles.

Le reste du manuscrit se concentre sur le cas où la fonction à minimiser est non différentiable. La plupart de nos résultats dans ce cas s'appliquent aux fonctions semi-algébrique ou, plus généralement, aux fonctions définissables dans une structure o-minimale. Cette classe de fonction, popularisée en optimisation par les travaux de Bolte, Lewis, Daniilidis et Shiota, comprend la grande majorité des fonctions étudiées en optimisation, statistiques et traitement de signal.

Le chapitre 4 analyse la version à pas constant de l'algorithme de la descente de sous-gradient stochastique (SGD). A pas décroissants cet algorithme a déjà été étudié dans la littérature en supposant l'existence en chaque point d'un estimateur "oracle" tel que son espérance est égal au sous-gradient de Clarke de la fonction à minimiser. L'existence d'un tel oracle dans des cas pratiques étant rarement vérifiée, nous montrons sous des conditions légères, que pour presque tout point d'initialisation l'existence d'un tel oracle n'est pas nécessaire. Dans un second temps, nous montrons que quand le pas tend vers zéro, l'interpolation affine des itérés converge vers l'ensemble des solutions du flot de sous-gradients (au sens de la convergence uniforme sur les compacts). Enfin, en analysant le SGD à pas fixé comme une chaîne de Markov, nous montrons que quand le pas tend vers zéro, la mesure invariante de cette chaîne de Markov tend faiblement vers l'ensemble des mesures invariantes du flot de sous-gradient. Ce résultat nous permet de montrer que, quand le pas est petit, les itérés du SGD au pas constant convergent vers les vi points critiques de la fonction en grande probabilité.

Dans le chapitre 5 nous établissons un résultat d'évitement par le SGD des points selles actifs. Ces points critiques sont d'une grande importance car, comme montré par les travaux de Davis et Drusvyatskiy, les seuls points critiques que possède une fonction définissable, faiblement convexe, générique, sont des minima locaux ou des points selles actifs. Par définition ces points selles actifs se trouvent sur une variété active telle que la fonction est différentiable sur cette variété et "change rapidement" en dehors de cette variété. Afin d'étudier le SGD au voisinage de ces points nous introduisons deux conditions supplémentaires sur la variété active: la condition de Verdier et la condition de l'angle. La première permet d'avoir une condition de type Lipschitz entre le "gradient riemannien" de la fonction sur la variété et ses sous-gradients de Clarke alors que la condition de l'angle permet de montrer que le SGD converge rapidement vers la variété active. A l'aide de ces deux conditions, sous des conditions d'isotropie sur les perturbations similaires à celles qui sont nécessaires dans le cas lisse, nous montrons que le SGD évite un point selle actif avec probabilité un. De manière indépendante nous établissons une version renforcée de la formule de projection de Bolte et al. en donnant une condition de type Lipschitz sur les sous-gradients d'une fonction définissable et Lipschitz. Nous pensons que ce type de résultat peut être important pour l'étude des problèmes d'optimisation non lisse et définissable dans une structure o-minimale. En particulier, ce résultat nous permet de démontrer la généricité de nos deux conditions: les points selles actifs d'une fonction définissable, faiblement convexe, générique se trouvent sur une variété active vérifiant les conditions de Verdier et de l'angle. Ainsi, une interprétation possible des résultats de ce chapitre est que le SGD sur une fonction générique, définissable et faiblement convexe converge vers un minimum local.

Naturellement, au vu des résultats énoncés ci-dessus on voudrait savoir quels sont les points critiques d'une fonction définissable, générique sans l'hypothèse de faible convexité. Nous montrons dans le chapitre 6 l'émergence dans la classe des fonctions définissables et localement Lipschitz d'un troisième type de point: un point critique fortement répulsif. Un tel point se trouve sur une variété active telle qu'au voisinage de cette variété il existe une région répulsive ou les sous-gradients de la fonction sont dirigés vers la variété. Le premier résultat du chapitre 6 est que tous les points critiques d'une fonction définissable, localement Lipschitz, générique se trouvent sur une variété active et sont soit des minima locaux, des points selles actifs ou des points critiques fortement répulsifs. De plus, les variétés actives correspondantes vérifient toujours les conditions de Verdier et de l'angle introduites précédemment. La question d'évitement d'un point selle actif étant abordée dans le chapitre précédent, nous montrons que, sous une condition de densité sur la loi des perturbations, les points critiques fortement répulsifs sont évités par le SGD avec probabilité un. Ainsi, une interprétation possible des résultats de ce chapitre est que le SGD sur une fonction générique, définissable et localement Lipschitz converge vers un minimum local.

Le chapitre 7 donne une caractérisation de la convergence du SGD et de ses versions proximales vers l'ensemble des points critiques. Alors que ces algorithmes vii peuvent avoir plusieurs points d'accumulation, nous montrons que le temps mis par les itérés de passer d'un voisinage d'un de ces points vers un autre tend vers l'infini. De plus, un phénomène d'oscillations des itérés est établi. Ce type de résultat pour la descente de gradient déterministe avait été établi auparavant par Bolte, Pauwels et Ríos-Zertuche en utilisant la théorie des mesures fermées. Dans le chapitre 7 nous établissons nos résultats sur la base de la théorie sur l'approximation stochastique et les inclusions différentielles de Benaïm, Hofbauer et Sorin ce qui permet de traiter les cas déterministe, stochastique et proximal avec une approche unifiée.

Chapter 1

Introduction

Various problems that arise in machine learning, signal processing and high dimensional statistics can be formulated as an optimization problem consisting into finding a minimum of a real valued function F : R d Ñ R.

In practical settings, this function might be unknown or the computation of its different characteristics (such as the gradient or the Hessian) might be expensive. In this case, stochastic approximation algorithms are particularly interesting since at each iteration they require only an estimator of F . When the function of interest is convex such algorithms can be studied through the tools of convex analysis (see e.g. [Bottou et al. 2018]). However, in many applications the convexity assumption fails. In such a case the standard way to analyze the convergence of a stochastic approximation algorithm is to view it as an Euler-like discretization of its continuous counterpart: an ordinary differential equation (ODE).

If the corresponding ODE is well behaved then one will usually be able to prove the convergence of the iterates to the set Z " tx P R d : ∇F pxq " 0u of critical points of F (see [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF], Kushner & Yin 2003[START_REF] Borkar | [END_REF]). This, however, gives only a partial answer to the question of the convergence to the set of (local) minimizers of F . Indeed, without the convexity assumption, Z is usually strictly larger than the latter and contains all kinds of spurious points such as local maxima or saddle points. The nonconvergence of stochastic approximation algorithms to such points was analyzed in the literature under the name of "avoidance of traps". (see e.g. [Brandière & Duflo 1996, Pemantle 1990[START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF]).

The aim of this thesis is to analyze the convergence of stochastic approximation algorithms to the set of critical points, when the continuous counterpart is no longer a simple ODE. We will give various characterizations of this convergence and establish, among other things, several avoidance of traps results. With the exception of Chapter 3, which analyzes a family of algorithms driven by a nonautonomous differential equation, we will focus on the case where the function that we seek to minimize is nonsmooth. The latter being especially important for various applications, the most notable one being the training of a neural networks with ReLU activation functions.

To better understand our approach we start by an illustrative example -the stochastic gradient descent (SGD). The SGD, an archetype of stochastic approximations algorithms, dates back to Robbins and Monro [START_REF] Robbins | [END_REF] and is written as follows:

x n`1 " x n ´γn ∇F px n q `γn η n`1 ,

(1.1)

where pγ n q is a sequence of positive real numbers decreasing to zero and pη n`1 q is a sequence of random perturbations (usually with zero mean) that modelizes our partial knowledge of F . One can view the SGD as an Euler-like discretization of the gradient flow: 9 xptq " ´∇F pxptqq .

(1.

2)

The so-called ODE method [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF][START_REF] Borkar | [END_REF], Kushner & Yin 2003] allows to rigorously compare the path taken by the iterates of Equation (1.1) to the solutions to the differential equation (1.2) and to establish that the iterates converge to the set Z " tx P R d : 0 P ∇F pxqu of critical points of F .

In the case of the SGD, the question of the avoidance of saddle points was first addressed by [Brandière & Duflo 1996] and [Pemantle 1990]. Their technique of proof is build upon the Poincaré invariant manifold theorem, which states that the set of points from which ODE (1.2) converges to a saddle point is a manifold of a dimension strictly smaller than d. The idea of [Brandière & Duflo 1996, Pemantle 1990] is then to show that, under an isotropic condition on the sequence pη n q, the iterates of the SGD will be driven away of this invariant manifold.

Throughout this thesis we will study different generalizations of the ODE method and the ideas of [Brandière & Duflo 1996, Pemantle 1990]. In particular, with the exception of Chapter 3, which analyzes a family of algorithms driven by a nonautonomous differential equation, we will focus on the case where the function that we seek to minimize is nonsmooth.

If F is merely locally Lipschitz continuous, then a natural generalization of the SGD is the stochastic subgradient descent, which reads as follows:

x n`1 P x n ´γn BF px n q `γn η n`1 ,

(1.3)

where BF px n q is the set of Clarke subgradients of F at x n , a notion that generalizes the one of the gradient. Algorithm (1.3) being a generalization of (1.1) we will still refer to it as the SGD. The set of critical points for the Clarke subgradient is now Z " tx P R d : 0 P BF pxqu, which still contains the (local) minima of F . Following the work of [Benaïm et al. 2005], the continuous counterpart of this algorithm is no longer an ODE but a differential inclusion (DI):

9 xptq P ´BF pxptqq . (1.4)
To obtain (and characterize) the convergence of Equation (1.3) to Z we first need to restrict the class of functions that we analyze. Indeed, in full generality the Clarke subgradient might not even be the right operator to consider. For instance, Rockafellar in [Rockafellar 1981] constructs a Lipschitz function F : R d Ñ R such that for every x P R d , we have BF pxq " r´1, 1s d . Results of [START_REF] Borwein | Lipschitz Functions with Maximal Clarke Subdifferentials Are Generic[END_REF] show that this example is actually typical, e.g. almost every continuous function F : r0, 1s Ñ R has BF pxq " r0, 1s for all x P r0, 1s 1 . This implies that, generally, BF gives us no information about the behavior of F . In fact, [START_REF] Daniilidis | Aris Daniilidis and Dmitriy Drusvyatskiy[END_REF] shows 1.1. Stochastic algorithms with momentum 3 that pathological dynamics can be exhibited such that px n q is not even converging to Z.

While these counterexamples may seem, at first sight, discouraging, in practical settings such pathological behavior is rare. Indeed, the vast majority of functions encountered in optimization are functions definable in an o-minimal structure, a notion popularized in the optimization literature by the work of [START_REF] Bolte | [END_REF]]. The family of definable functions is broad: every semialgebraic function is definable, the exponential and the logarithm are definable. Moreover, the notion of definableness is stable by many of the elementary operations such as composition, sum, multiplication and taking the inverse. While definable functions may be nonsmooth, the nonsmoothness here appear in a very structured manner. Example given, in [START_REF] Bolte | [END_REF]] the authors have established the so-called projection formula, which gives a description of the Clarke subgradients of a definable function. More precisely, the authors of [START_REF] Bolte | [END_REF]] have shown that, given a definable function F and p an integer, there exists pX i q a finite partition of the domain of F into C p manifolds such that F is C p smooth on X i and, moreover, if y P X i , then we have:

P TyX i pBF pyqq " t∇ X i F pyqu , (1.5)
where ∇ X i F pyq is the Riemannian gradient of F restricted to X i at y and P TyX i denotes the orthogonal projection onto the the tangent space at y of X i . Equation (1.5) is the starting point in the proof of various properties of definable functions such as the nonsmooth Kurdyka-Łojasiewicz inequality or the path differentiability [Bolte & Pauwels 2019].

In recent years, this implicit smooth structure has allowed a thorough analysis of algorithms operating on definable functions [Attouch et al. 2011, Bolte et al. 2009, Davis & Drusvyatskiy 2021]. In particular, the work of [START_REF] Davis | [END_REF] shows that under mild conditions on the sequence pη n q, the iterates of Equation (1.3) converge to Z. In this thesis we will give different characterizations of this convergence, with a particular focus in Chapters 5 and 6 on the question of the avoidance of traps in a nonsmooth setting.

With Chapter 2 being dedicated to mathematical tools that will be used throughout this thesis, we finish the present chapter by a detailed descriptions of the main obtained results.

Stochastic algorithms with momentum

In Chapter 3, which is based on the publication {5}, we study a class of stochastic algorithms which admits as a continuous counterpart the following ODE, introduced in [Belotto da Silva & Gazeau 2018]. Let F : R d Ñ R be a differentiable function to minimize and let S : R d Ñ R d `be a continuous function. Let h, r, p, q : p0, `8q Ñ R `be continuous functions and let ε ą 0. Starting from a point pvp0q, mp0q, xp0qq P where for two vectors x, y P R d , x{y denotes the vector px 1 {y 1 , . . . , x d {y d q P R d . The main challenge in the analysis of the algorithms that are underlined by this ODE is the fact that ODE (1.6) is nonautonomous. ODE (1.6) generalizes various differential equations encountered in stochastic approximation. For instance, its particular case is 9 mptq " ∇F pxptqq ´rptqmptq 9 xptq " ´mptq which can be rewritten as : xptq `rptq9 xptq `∇F pxptqq " 0.

If we choose rptq " α ą 0, then we obtain the well-known Heavy-Ball with friction algorithm [Attouch et al. 2000, Gadat et al. 2018]. Choosing rptq " α{t, with α ą 0, gives us the Nesterov's accelerated gradient algorithm which was studied from this ODE in [Su et al. 2016a].

Going back to ODE (1.6) and choosing this time hptq " rptq " apt, λ, α 1 q, pptq " qptq " apt, λ, α 2 q for apt, λ, αq " λ ´1p1 ´expp´λαqq{p1 ´expp´αtqq, λ, α 1 , α 2 ą 0 and S " ∇F d2 , we recover the widely used ADAM algorithm [START_REF] Kingma | [END_REF] (see also [Belotto da Silva & Gazeau 2020, Sections 2.4-4.2] and [Barakat & Bianchi 2021] for the stochastic version of this algorithm).

In this chapter we establish the convergence of the stochastic algorithms driven by ODE (1.6) to the set of critical points of F . In this level of generality, the presented results are new. Convergence rates in the form of a central limit theorem are given. Last but not least, an avoidance of traps result is established. This result extends previous works of [START_REF] Gadat | [END_REF] obtained in the context of SHB. This result not only allows to study a broader class of algorithms but also significantly weakens the assumptions. In particular, [START_REF] Gadat | [END_REF]] uses a sub-Gaussian assumption on the noise and a rather stringent assumption on the stepsizes. The main difficulty in the approach of [START_REF] Gadat | [END_REF]] lies in the use of the classical autonomous version of Poincaré's invariant manifold theorem. The key ingredient of our proof is a general avoidance of traps result, adapted to nonautonomous settings, which we believe to be of independent interest. It extends usual avoidance of traps results to a nonautonomous setting, by making use of a nonautonomous version of Poincaré's theorem [Dalec 1 kiȋ & Kreȋn 1974, [START_REF] Kloeden | [END_REF].

Contributions.

1.2. Convergence of the stochastic subgradient descent with a constant stepsize 5

• First, we analyze ODE (1.6) by showing the existence and the uniqueness of its solutions. Convergence of these solutions to the set of critical points of F is established. In particular, no convexity assumption is made and, to the best of our knowledge, the convergence statement for the ODE that underlines the Nesterov's accelerated gradient descent is new.

• Second, we analyze a class of stochastic approximation algorithms that are Euler-like discretizations of this ODE. Examples of these are ADAM, Ada-Grad, Heavy-Ball and Nesterov's accelerated gradient descent. Boundedness and convergence to the set of critical points of F is established. Under additional assumptions, convergence rates in the form of a central limit theorem are given. These results extend the works of [START_REF] Gadat | [END_REF], Barakat & Bianchi 2021] to a more general setting. In particular, we highlight the almost sure convergence result for the (stochastic) Nesterov's accelerated gradient descent in a nonconvex setting, which is, to the best of our knowledge, new.

• Finally, a general avoidance of traps result is established for algorithms underlined by a nonautonomous ODE. An application of this result to the algorithms that we analyze is given by establishing that, under assumptions on the perturbation sequence similar to [Brandière & Duflo 1996], the iterates avoid saddle points with probability one.

Convergence of the stochastic subgradient descent with a constant stepsize

Chapter 4, based on the publication {4}, analyzes the constant step version of the SGD (1.3). While from a theoretical point of view, the vanishing step size is convenient to show the convergence of the algorithm to Z, in practical applications such as the training of a neural net, a vanishing step size is rarely used because of slow convergence issues. In most computational frameworks, a possibly small but nevertheless constant step size is used by default. The price to pay is that the iterates are no longer expected to converge almost surely to the set Z but to fluctuate in the vicinity of Z as n is large. Therefore, in this chapter we aim to establish a result of the type @ε ą 0, lim sup nÑ8

Ppdistpx n , Zq ą εq Ý Ý Ñ γÓ0 0 .

(1.7)

Although this result is weaker than in the vanishing step case, constant step stochastic algorithms can reach a neighborhood of Z faster than their decreasing step analogues, which is an important advantage in the applications where the accuracy of the estimates is not essential. Moreover, in practice they are able to cope with non stationary or slowly changing environments which are frequently encountered in signal processing, and possibly track a changing set of solutions [Benveniste et al. 1990, Kushner & Yin 2003].

Before proving this convergence result we address the question of the pertinence of the algorithm (1.3) from a practical standpoint. Indeed, in stochastic approximation the designer has usually no access to the function F but rather to a sequence of i.i.d random variables pξ n q, following a law µ, and to a function f px, ξq such that F pxq " E ξ"µ rf px, ξqs. In this case, the natural algorithm that comes to mind is:

x n`1 " x n ´γv n`1 ,

(1.8)

where v n`1 is a selection of the Clarke subgradient B x f px, ξq (taken relatively to the first variable) at the point px n , ξ n`1 q. Denoting F n the sigma algebra generated by px 0 , . . . , x n q, we can rewrite this algorithm as:

x n`1 " x n ´γErv n`1 |F n s `γη n`1 , (1.9) where η n`1 " ´vn`1 `Erv n`1 |F n s is a martingale increment. The issue that arise in this case is that the continuous counterpart of this equation is now 9 xptq P ´Eξ"µ rB x f pxptq, ξqs .

(1.10) This differential inclusion is not necessarily an instance of the DI (1.4), because we do not generally have that Erv n`1 |F n s P BF px n q. Indeed, the interchange E Ø B holds for convex or smooth functions but fails in general. In [START_REF] Majewski | [END_REF] the authors restrict their analysis to Clarke regular functions [Clarke et al. 1998, §2.4], for which the interchange of the expectation and the subdifferentiation applies. However, this assumption can be restrictive, since a function as simple as ´|x| is not regular at the critical point zero.

In Chapter 4 we consider a slightly more general version of algorithm (1.8), which includes the case where v n`1 is a selection of a so-called conservative field. This notion, introduced in [Bolte & Pauwels 2019], modelizes the output of the celebrated backpropagation algorithm used in numerical libraries such as PyTorch or Tensorflow [START_REF] Paszke | [END_REF]. A similar issue arise in this case since the interchange between the expectation and a conservative field might not hold.

Our first result is that, under some mild conditions on the functions F and f p¨, ξq (for instance if they are definable), for almost every initialization point and for every n P N, x n almost never hits a nondifferentiable point of f p¨, ξ n`1 q. As a consequence, algorithm (1.3) can be rewritten as:

x n`1 " x n ´γ∇F px n q `γη n`1 , and its continuous counterpart is indeed the DI (1.4). Using this result, we show that the continuous process obtained by a piecewise affine interpolation of px n q is a weak asymptotic pseudotrajectory of the DI (1.4). In other words, the interpolated process converges in probability to the set of solutions to the DI, as γ Ñ 0, for the metric of uniform convergence on compact intervals.

The proof technique to establish the convergence (1.7) is then rather standard and consists to view (1.9) as a Markov process. For each γ ą 0, under a drift assumption on its kernel, this Markov process admits an invariant distribution µ γ . Every accumulation point of pµ γ q, in the sense of the weak convergence and when γ Ñ 0, is then shown to be supported on Z, which in turn implies (1.7).

Contributions.

• We analyze the SGD algorithm with a constant step size in the non-smooth, non-convex setting. Under mild conditions, we prove that when the initialization x 0 is chosen randomly x n almost never hits a non-differentiable point of f p¨, ξ n`1 q and x n`1 ´xn γ " ´∇F px n q `ηn`1 , where pη n q is a martingale difference sequence, and ∇F px n q is the true gradient of F at x n . This argument allows to bypass the oracle assumption of [START_REF] Majewski | [END_REF], Davis et al. 2020].

• We establish that the continuous process obtained by a piecewise affine interpolation of px n q is a weak asymptotic pseudotrajectory of the DI (1.4). In other words, the interpolated process converges in probability to the set of solutions to the DI, as γ Ñ 0, for the metric of uniform convergence on compact intervals.

• We establish the long run convergence of the iterates x n to the set Z of Clarke critical points of F , in the sense of Equation (1.7). This result holds under two main assumptions. First, it is assumed that F admits a chain rule, which is satisfied for instance if F is a definable function. Second, we assume a standard drift condition on the Markov chain (1.9). Finally, we provide verifiable conditions of the functions f p¨, sq under which the drift condition holds.

• In many practical situations, the drift condition alluded to above is not satisfied. To circumvent this issue, we analyze a projected version of the SGD algorithm, which is similar in its principle to the well-known projected gradient algorithm in the classical stochastic approximation theory.

SGD escapes active strict saddles

In Chapter 5, which is based on the publication {2}, we address the question of the avoidance of traps in a nonsmooth setting. The traps that are considered here are the active strict saddles, a notion that was recently introduced in [Davis & Drusvyatskiy 2021]. Formally, a Clarke critical point is an active strict saddle if it lies on an active manifold M such that the Riemannian Hessian of F on M has at least one negative eigenvalue. An active manifold in this setting is a manifold M such that i) F varies sharply outside of M , ii) F M , the restriction of F to M , is smooth. For instance, the function py, zq Þ Ñ |z| ´y2 admits the point p0, 0q as an active strict saddle (see Figure 1.1).

Figure 1.1: The point p0, 0q is an active strict saddle of F py, zq " |z| ´y2 . The corresponding active manifold is M " R ˆtp0, 0qu.

The importance of this notion comes from the fact, proved in [Drusvyatskiy et al. 2016, Davis & Drusvyatskiy 2021], that, given a weakly convex, definable function F : R d Ñ R, for almost every u P R d , each of the Clarke critical points of the linearly perturbated function F u pxq ∆ " F pxq ´xu, xy is either a local minimum or an active strict saddle. In that sense, active strict saddles are generic in the class of weakly convex, definable functions.

In [Davis & Drusvyatskiy 2021] the authors have proven that, upon a random initialization, proximal methods escape active strict saddles with probability one. Such a result is possible due to the fact that proximal methods implicitly run a gradient descent on a smoothened version of F -the Moreau envelope. On the contrary, in Chapter 5 we analyze algorithm (1.3) which is inherently nonsmoooth. The aim is to establish that the SGD (with decreasing stepsizes) escapes active strict saddles with probability one.

The intuition behind our approach could be grasped by looking at the stochastic subgradient descent on the function from Figure 1.1. In this case, it is natural to write down the iterates px n q as py n , z n q and to notice that py n q follows an SGD dynamic on a smoooth function y Þ Ñ ´y2 . Thus, applying the results of [Brandière & Duflo 1996, Pemantle 1990] to the sequence py n q, we obtain that, under similar assumptions on the perturbation sequence, Ppy n Ñ 0q " 0. This implies that the stochastic subgradient descent avoids p0, 0q with probability one. On an independent note, observe that in this example the sequence pz n q converges to zero in a very fast manner.

To formalize this type of behavior in a more general setting we have introduced two additional conditions on the active manifold. The first one, the Verdier condition, states that for x close to M : (1.11) where P M pxq is the projection of x onto M , ∇ M F is the "Riemannian gradient" of F M and v M is the projection of v along the tangent space of M (see Section 5.3.2 for a precise statement). A consequence of this condition is that, writing down py n q " pP M px n qq, a simple application of Taylor's formula gives us:

@v P BF pxq, v M « ∇ M F pP M pxqq `Opdistpx, M qq ,
y n`1 « y n ´γn ∇ M F py n q `γn η M n`1 `γn Opdistpx n , M qq `Opγ 2 n q , (1.12)
where η M n`1 is the projection of η n`1 on the tangent space of M at y n . That is to say, up to a residual error term, py n q follows an SGD dynamic on the (smooth) function

F M .
The purpose of the angle condition is to control this residual term. First, a following observation is made. Let x ˚be a Clarke critical point of F lying on an active manifold M . Then, on the event rx n Ñ x ˚s, for n large enough, we have:

F px n q ´F pP M px n qq Á x n ´PM px n q .
(1.13)

The angle condition then states that close to M we have:

F pxq´F pP M pxqq Á x ´PM pxq ùñ xv, x´P M pxqy Á x ´PM pxq , @v P BF pxq .

(1.14) Combining (1.13) with (1.14), we obtain that, for n large enough, the angle between the set BF px n q and the normal direction to M is lower bounded. This is used to show that distpx n , M q converges to zero in a very fast manner and thus allows to control the residual term in Equation (1.12).

The angle and the Verdier conditions provide a general way to analyze the stochastic subgradient descent in a neighborhood of an active manifold by decomposing the iterates px n q into a sum of two sequences: py n q " pP M px n qq and pz n q " px n ´yn q. The angle condition ensures the fact that z n " distpx n , M q Ñ 0 (and hence x n Ñ M ) fast enough. Combining this fact with the Verdier condition, this implies that py n q, up to a residual term, follows an SGD dynamic on the smooth function F M . In Chapter 5 we illustrate this proof technique by showing that, under conditions on pη n`1 q similar to the ones obtained in a smooth setting by [Brandière & Duflo 1996], the stochastic subgradient descent avoids active strict saddles with probability one.

An important contribution of this chapter is an improvement of the projection formula (1.5). One of the consequence of Equation (1.5) is that if we have a sequence px n , v n q, with x n Ñ y P X i and v n P BF px n q, then we always have P TyX i pv n q Ñ ∇ X i F pyq. In Theorem 5.2.1 of Chapter 5 we reinforce this result by showing that if F is locally Lipschitz continuous, then there exists (a perhaps finer) finite partition pX i q such that for any y P X i , there is C ą 0 such that for any y 1 P X i and x P R d that are close enough to y, we have:

@v P BF pxq , P T y 1 X i pvq ´∇X i F py 1 q ď C x ´y1 .
(1.15)

The Verdier condition (1.11) thus merely states that M is one of the element of this partition. The projection formula was initially proved in [START_REF] Bolte | [END_REF] using the fact that the graph of a definable function admits a so-called Whitney-(a) stratification. Our proof of the reinforced projection formula is based on the fact, well known literature on o-minimal theory (see [Loi 1998]), that the graph of a definable function admits a so-called Verdier stratification. Formula (1.15) gives us a Lipschitz-like condition on the subgradient operator BF . We believe that such a result is of independent interest and might be important in the analysis of nonsmooth algorithms operating in a definable setting.

One of the consequences of the reinforced projection formula that we prove is that the Verdier and the angle conditions are generic in the class of weakly convex functions. That is to say, given F : R d Ñ R a weakly convex and definable function, for almost every u P R d , every critical point of F u is either a local minimum or an active strict saddle, with the corresponding active manifold satisfying the Verdier and the angle conditions. Therefore, a possible interpretation of the results of this chapter is that the stochastic subgradient descent on a generic, weakly convex function converges to a local minimum. Contributions.

• Firstly, we bring to the fore the fact that definable functions admit stratifications of the Verdier type. These are more refined than the Whitney stratifications which were popularized in the optimization literature by [START_REF] Bolte | [END_REF]].

While such stratifications are well-known in the literature on o-minimal structures [Loi 1998], up to our knowledge, they have not been used yet in the field of non smooth optimization. To illustrate their interest in this field, we study the properties of the Verdier stratifiable functions as regards their Clarke subdifferentials. Specifically, we refine the so-called projection formula to the case of definable, locally Lipschitz continuous functions by establishing a Lipschitz-like condition on the (Riemannian) gradients of two adjacent stratas.

• Secondly, we introduce two additional assumptions on an active manifold: the Verdier and the angle conditions. We prove that a generic active strict saddle of a definable and weakly convex function is lying on an active manifold satisfying both of these conditions.

• Finally, with the help of the Verdier and the angle conditions, we show that the SGD avoids the active strict saddles if the noise η n is omnidirectional enough. We emphasize here that, while our genericity result holds under a weak convexity assumption, no weak convexity is assumed for our avoidance of traps result.

SGD on a generic definable function converges to a minimizer

In Chapter 5 we have established that, given

tx P R d : 0 P BF u pxqu , (Z u )
for a non Lebesgue-null set of u P R d . First, we must notice that such a simple classification in a weakly convex setting comes from the fact that, from a minimization perspective, the local behavior of F on an active manifold M dictates its shape outside of M . In particular, if F is weakly convex and x ˚is a local minimum of F M , then x ˚is a local minimum of F (see [Drusvyatskiy & Lewis 2014]). Looking at an example as simple as py, zq Þ Ñ y 2 ´|z| (see Figure 1.2), we see that this is no longer true when the weak convexity assumption fails.

Figure 1.2: The point p0, 0q is a sharply repulsive critical point of F py, zq " ´|z|`y 2 . The corresponding active manifold is M " R ˆtp0, 0qu.

This motivates the introduction of a third type of points: a sharply repulsive critical point. We say that a Clarke critical point x ˚P M is a sharply repulsive critical point if i) M is an active manifold (for F and x ˚), ii) x ˚is a local minimum of F M , iii) 0 P B L F px ˚qzBF px ˚q, where B L F pxq denotes the limiting subgradient of F at x ˚. Intuitively speaking, if x ˚P M is a sharply repulsive critical point, then there is a large region neighboring M on which the subgradients of F are pointing towards M . A typical example of this situation is illustrated on Figure 1.2.

In Chapter 6 we show that such an example is generic. More precisely, in Theorem 6.2.5 we establish that for almost every u P R d , every point in (1.4) is lying on an active manifold and is either a local minimum, an active strict saddle or a sharply repulsive critical point (for the function F u ). Moreover, the corresponding active manifolds satisfy the Verdier and the angle conditions. In this sense, a generic trap of a definable, locally Lipschitz continuous function is either an active strict saddle or a sharply repulsive critical point. We must notice here that almost all of the points of Theorem 6.2.5 readily follow from the work of [Drusvyatskiy et al. 2016]. However, the question of the genericity of the angle condition seems to be delicate and our proof of this point is based on some deep results of o-minimal theory.

Since the question of the avoidance of active strict saddles was treated in Chapter 5, the rest of Chapter 6 is devoted to the question of the avoidance of sharply repulsive critical points. Our first result shows that if x ˚is such a point, then, on the event rx n Ñ x ˚s, for n large enough, we have:

F px n q ě F px ˚q .
While the proof of this inequality readily follows from the observations of Chapter 5, this result reveals to be interesting. Indeed, it implies that while the iterates px n q may in theory converge to x ˚this happens only if the SGD fails to explore the repulsive region near x ˚. In some sense, the algorithm perceive the function F as if x ˚was indeed its local minimum.

In a second time, we show that a density-like assumption on pη n q forces the SGD to visit the repulsive region near M and will imply the nonconvergence of the SGD to a sharply repulsive critical point. We must notice here the difference with the proof of Chapter 5 on the avoidance of active strict saddles. Indeed, if x ˚is a sharply repulsive critical point, then asymptotically the sequence py n q " pP M px n qq still follows an SGD dynamic on a smooth function F M . However, since in this case x ˚is a local minimum of F M this is not sufficient to prove the nonconvergence of px n q to M . Therefore, in the setting of Chapter 6 the avoidance of traps result is established by using a density-like condition on pη n q.

The final Section 6.3.3 shows that, while such a density-like assumption on pη n q might not hold, a way to ensure it in a standard stochastic approximation model is to add a small perturbation (e.g. a nondegenerate Gaussian) at each iteration of (6.1). This fact, combined with the results of Chapter 5 on the avoidance of active strict saddles, provides a practical way to avoid generic traps of definable functions, and, therefore, ensure the convergence of the SGD to a local minimum.

We must mention here that shortly after the publication {2} and just before the submission of {1} a concurrent work [Davis et al. 2021] has appeared. The latter, sharing a lot of similarities with Chapter 5, analyzes the SGD (and its proximal versions) in a neighborhood of an active manifold. An avoidance of active strict saddles result was obtained as well as (local) rates of convergence and asymptotic normality of the iterates were established. These results support our claim on the importance of the Verdier and the angle conditions. A major difference with our work is that their proximal aiming condition assume (close to the active manifold) the left hand side of formula (1.14). Such an assumption rules out functions with downward cusps such as py, zq Þ Ñ ˘y2 ´|z|, which are treated in Chapters 5 and 6. As a consequence, the question of genericity in [Davis et al. 2021] is addressed only for the class of Clarke regular functions in which sharply repulsive critical points do not exist. In particular, we believe that convergence rates of a similar kind could 1.5. Oscillations of the SGD and its proximal extensions be obtained upon replacing the proximal aiming condition of [Davis et al. 2021] by ours angle condition. Contributions.

• We introduce the concept of a sharply repulsive critical point. Given F : R d Ñ R a locally Lipschitz continuous function that is definable in an o-minimal structure, we show that for a full-measure set of u P R d , each of the critical points of the linearly perturbated function F u is lying on an active manifold satisfying a Verdier and an angle condition and is either a local minimum, an active strict saddle or a sharply repulsive critical point.

• We show that if x ˚is a sharply repulsive critical point, then on the event rx n Ñ x ˚s, for n large enough, we have F px n q ě F px ˚q. Furthermore, if the corresponding active manifold satisfies an angle condition and under a densitylike assumption on the perturbation sequence pη n q we show that the iterates of the SGD will avoid a sharply repulsive critical point with probability one. Finally, in a standard stochastic approximation model, we show that such an assumption can be ensured by adding at each iteration a small perturbation with a density. The latter, combined with the results of Chapter 5, gives a practical way to ensure the avoidance of the generic traps by the SGD.

Oscillations of the SGD and its proximal extensions

The purpose of Chapter 7, which is based on the publication {3}, is to give some characterizations of the convergence of the algorithm (1.3) and its proximal extensions. Given F, g : R d Ñ R two locally Lipschitz continuous functions and X a closed convex set, we are seeking to minimize F `g over X . A popular choice of algorithm in this case is the stochastic proximal subgradient descent (SPGD), which reads as follows:

x n`1 P prox γn g,X px n ´γn v n `γn η n`1 q , (1.16)

where prox γn g,X is the proximal operator for the function g on X and v n is in the set BF px n q.

From the work of [START_REF] Davis | [END_REF] it is known that in this case the iterates px n q will converge to the set of composite critical points Z :" tx : 0 P BF pxq Bgpxq `NX pxqu, where N X pxq is the normal cone of X at x. However, the iterates px n q might not converge to a unique point. Indeed, in [Rios-Zertuche 2020, Section 2] Ríos-Zertuche considers the deterministic subgradient descent (that is to say g " 0, η n " 0 and X " R d ) and constructs F , which verifies main assumptions of nonsmooth optimization (such as Whitney stratifiability of its graph or Kurdyka-Łojasiewicz inequality) but the limit set of px n q is equal to Z " tx : x " 1u. This encourages a more precise study of Equation (1.16).

In Chapter 7 we establish two additional results on the convergence of Equation (1.16). First, we show that if x, y P Z are two distinct accumulation points, then the time that the iterates spend to get from a neighborhood of x to a neighborhood of y goes to infinity. Secondly, we rewrite Equation (1.16) as:

x n`1 " x n ´γn pv n `vg n `vX n q `γn η n`1 , (1.17)
where v g n P Bgpx n`1 q and v X n P N X px n`1 q and establish an oscillation-type phenomena. In a first approximation our results imply that, given δ ą 0 and any accumulation point x, we have:

ř n i"1 γ i pv i `vg i `vX i `ηi`1 q1 Bpx,δq px i q ř n i"1 γ i 1 Bpx,δq px i q ÝÝÝÝÑ nÑ`8 0 .
(1.18)

Intuitively speaking, this type of behavior shows that even if x n ´x0 "

ř n i"1 γ i pv i vg i `vX i `ηi`1 q might not converge, the drift coming from the sugbradients v n , v g n , v X n
and the perturbation sequence η n`1 , on average, compensate itself. This suggests that the subgradient descent and its stochastic and proximal versions oscillates around its accumulation set, with the center of these oscillations moving in a vanishing speed. This type of results was obtained by [Bolte et al. 2020b] for the deterministic gradient descent using the theory of closed measures. A nice feature of this chapter is that all of our results are proved using the theory of [Benaïm et al. 2005]. We feel that this approach gives a simpler proof of the convergence and the oscillation phenomena of the subgradient descent and its stochastic/proximal extensions.

Contributions.

• We show that the time spent by the SPGD to move from one accumulation point to another goes to infinity and establish an oscillation-type behavior of the drift. These two results extend [Bolte et al. 2020b, Theorem 7.] to a stochastic and a proximal setting. Our technique of proof doesn't rely on the theory of closed measures used in [Bolte et al. 2020b] but is build upon the classical work of Benaïm, Hofbauer and Sorin [Benaïm et al. 2005]. We feel that this approach gives a simpler proof and allows us to treat the deterministic, the stochastic and the proximal cases in a unified manner.

Publications

Subgradients

Most of the results of this section can be found in classical monographs on variational analysis such as [START_REF] Rockafellar | [END_REF], Clarke et al. 1998].

When the function of interest is nonsmooth various notions of subgradients generalize the one of the gradient.

Definition 2.1.1 (Frechet subgradient). Consider f : R d Ñ R a locally Lipschitz continuous function and x P R d . The set B F f pxq Ă R d of Frechet subgradients of f at x is the set of v P R d for which: lim inf x 1 Ñx f px 1 q ´f pxq ´xv, x 1 ´xy x 1 ´x ě 0 .
The set B F f pxq can be empty (e.g. for the function f pxq " ´ x at 0). This motivates the following definitions.

Definition 2.1.2. Consider f : R d Ñ R a locally Lipschitz continuous function and x P R d . The set B L f pxq of limiting subgradients of f at x is the set of v P R d for which there is a sequence px n , v n q Ñ px, vq, with v n P B F f px n q for all n P N.

If f is C 1 around x, then Bf pxq " B L f pxq " B F f pxq " t∇f pxqu.
Definition 2.1.4 (Clarke critical points). We say that x P R d is a Clarke critical point of f if 0 P Bf pxq.

Similarly to the smooth setting, the set of Clarke critical points of f contains local maxima and minima of f . Since f is locally Lipschitz continuous, by Rademacher's theorem, f is differentiable almost everywhere. The following proposition describes an alternative characterization of Bf .

Chapter 2. Mathematical preliminaries Proposition 2.1.1 ( [Clarke et al. 1998, Theorem 8.1]). Assume that f : R d Ñ R is locally Lipschitz continuous. Denote D Ă R d the set of points at which f is differentiable and let A Ă R d be any Lebesgue-null set. Then Bf pxq " convtv : there is a sequence

x n Ñ x s.t. x n P D X A c , ∇f px n q Ñ vu .
The key notion in the analysis of the stochastic subgradient descent is the one of path-differentiability.

Definition 2.1.5 (Path-differentiability [Bolte & Pauwels 2019]). A locally Lipschitz continuous function f : R d Ñ R is said to be path-differentiable if for every absolutely continuous curve c : p0, 1q Ñ R d , one has for almost every t P p0, 1q, pf ˝cq 1 ptq " xv, 9 cptqy, @v P Bf pcptqq .

Examples of path-differentiable functions include convex, concave, semialgebraic and more generally definable (see Section 2.4) functions [Bolte & Pauwels 2019]. In nonsmooth optimization, the path-differentiability condition is often a crucial hypothesis in order to obtain relevant results e.g., on the convergence of iterates [START_REF] Bolte | [END_REF], Davis et al. 2020, Bolte & Pauwels 2019]. In particular, as we will see in Section 2.2.2 path-differentiability of a function will ensure that it is a Lyapounov function for the subgradient flow.

Asymptotic pseudotrajectories and differential inclusions

ODE method

The ODE method analyzes the convergence properties of a stochastic approximation algorithm by studying its continuous counterpart: an ordinary differential equation.

Setting up the stage, let G : R d Ñ R d be a continuous function and consider the following ODE:

9 xptq " Gpxptqq .
(2.1) For x 0 P R d , we denote Φpx 0 q : R `Ñ R d the solution to this ODE starting at x 0 .

The key notion of the ODE method is the one of the asymptotic pseudotrajectory (APT).

Definition 2.2.1 (Asymptotic pseudotrajectory (APT) [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF]). We say that a continuous function X : R `Ñ R d is an asymptotic pseudotrajectory for the ODE (2.1) if for all T ą 0, we have:

sup hPr0,T s Xpt `hq ´Φh pXptqq Ý ÝÝÝ Ñ tÑ`8 0 .
We endow CpR `, R d q, the space of continuous functions from R `to R d , with the metric of uniform convergence on compact intervals of R `:

d C px, yq " ÿ nPN 2 ´n ˜1 ^sup tPr0,ns }xptq ´yptq} ¸. (2.2)
Equivalently, X is an APT for the ODE (2.1) if

d C pXpt `¨q, ΦpXptqqq Ý ÝÝÝ Ñ tÑ`8
0 .

As the following example shows, the notion of an APT naturally arise in the study of stochastic approximation algorithms.

Example 2.2.1. Consider an R d -valued sequence px n q satisfying the following equation:

x n`1 " x n `γn Gpx n q `γn η n`1 , (2.3)
where pγ n q is a positive sequence and pη n`1 q are R d -valued. One can view Equation (2.3) as an Euler-like discretization of (2.1). Assume the following.

• pγ n q is such that ř n i"0 γ i Ñ `8.

• The sequence px n q is bounded.

• For every T ą 0, we have:

N pT,nq ÿ i"n γ i η i`1 ÝÝÝÝÑ nÑ`8 0 ,
where for n P N and T ą 0, N pT, nq " suptk ě n : ř k i"n γ i ď T u. Then the linearly interpolated process X : R `Ñ R d defined as:

Xptq " x n `t ´τn γ n`1 px n`1 ´xn q, if t P rτ n , τ n`1 q ,
where τ n " ř n i"0 γ i , is an APT of the ODE (2.1) (see [Benaïm 1999, Proposition 4.1]).

Remark 1. A typical situation when the assumption on pη n q in Example 2.2.1 is verified is when ř n i"0 γ i η i`1 converges. For instance, this is the case when pη n q is a sequence of martingale increments relatively to some filtration, lim nÑ`8 ř n i"0 γ 2 i ă `8 and sup nPN Er η n 2 s ă `8.

To further characterize the convergence of a stochastic algorithm we need the notion of a Lyapounov function.

Definition 2.2.2 (Lyapounov function). Let A be a set in R d . A continuous function V : R d Ñ R is a Lyapounov function for A and the ODE (2.1) if for all x P R d , the function t Þ Ñ V pΦ t pxqq is decreasing and is strictly decreasing as soon as x R A.

Chapter 2. Mathematical preliminaries Example 2.2.2. Let f : R d Ñ R be C 1 and consider the ODE (2.1), with G " ´∇f . Then f is a Lyapounov function for its set of critical points. Indeed,

f pΦ t pxqq " f pxq ´ż t 0 ∇f pΦ u pxqq 2 du .
When a process is an APT of an ODE with a Lyapounov function more can be said about its convergence properties.

Proposition 2.2.1 ( [Benaïm 1999, Proposition 5.7 and 6.4]). Let X be a bounded APT of the ODE (2.1), let A Ă R d and let V be a Lyapounov function for A.

Denote

L X " č tPR `Xprt, `8qq the limit set of X. If V pAq is of empty interior, then the following holds.

• We have that L X Ă A.

• The function V is constant on L X .
Notice that in the context of Example 2.2.1 the set L X is equal to the set of accumulation points of px n q.

Example 2.2.3. Let f : R d Ñ R be C 1 and consider the setting of Example 2.2.1, with G " ´∇f . Applying Proposition 2.2.1, we obtain that f px n q converges and every accumulation point of px n q is a critical point of f .

Differential inclusions

In nonsmooth analysis the notion of an ODE is replaced by the one of a differential inclusion (DI).

We say that H : R d Ñ R d is a set valued map if for each x P R d , we have that Hpxq is a subset of R d . Consider the DI:

9 xptq P Hpxptqq .
(2.4)

We say that an absolutely continuous curve (a.c.) x : R `Ñ R d is a solution to (2.4) starting at x P R d , if xp0q " x and Equation (2.4) holds for almost every t ě 0. We denote S H pxq the set of these solutions. Various notions of continuity exist for set valued maps. The one that will be important for us is the notion of upper semicontinuouty.

Definition 2.2.3. We say that a set valued map H : R d Ñ R d is upper semi continuous at x P R d if for every U , a neighborhood of Hpxq, there is δ ą 0 such that y ´x ď δ ùñ Hpyq Ă U .

We say that H is upper semi continuous (usc) if it is upper semicontinuous at every point.

For a locally Lipschitz continuous function f : R d Ñ R, the set valued map Bf : R d Ñ R d is upper semi continuous. Moreover, for x P R d , the set Bf pxq is nonempty, convex and compact. For this type of maps, we have the following existence result.

Proposition 2.2.2 ( [Aubin & Cellina 1984]). Assume that for each x in R d , Hpxq is nonempty, convex and compact and that there is a constant C ě 0 s.t. supt v : v P Hpxqu ď Cp1 ` x q. Assume that H is usc, then for every x P R d , the set S H pxq is nonempty.

The notion of an APT generalizes to the case of differential inclusions. Definition 2.2.4 ( [Benaïm et al. 2005]). We say that a continuous curve

X : R `Ñ R d is an APT of the DI (2.4) if for all T ą 0, sup hPr0,T s inf xPS H pxptqq Xpt `hq ´xphq Ý ÝÝÝ Ñ tÑ`8 0 .
(2.5)

Example 2.2.4. Consider an R d -valued sequence px n q satisfying the following inclusion:

x n`1 P x n `γn Hpx n q `γn η n`1 , (2.6)

with pγ n q, pη n q satisfying the assumptions of Example 2.2.1 and H satisfying the assumptions of Proposition 2.2.2. Assume that px n q is bounded, then the linearly interpolated process constructed from px n q is an APT for the DI (2.4) (see [Benaïm et al. 2005, Theorem 4.1]).

There is a notion of a Lyapounov function in the context of differential inclusions.

Definition 2.2.5 (Lyapounov function (DI)). Let A be a set in R d . A continuous function V : R d Ñ R is a Lyapounov function for A and the DI (2.4) if for all x P R d , t ą 0 and x P S H pxq, we have:

V pxptqq ď V pxq ,
with strict inequality as soon as x R A.

Example 2.2.5. Let f : R d Ñ R be a locally Lipschitz continuous, path-differentiable, function. Consider the DI (2.4), with H " ´Bf . For x P R d , consider x P S H pxq.

By path-differentiability of f we have:

f pxptqq " f pxp0qq ´ż t 0 9 xpuq 2 du .
In particular, f is a Lyapounov function for this DI and the set tx P R d : 0 P Bf pxqu of Clarke critical points of F .

Similarly to Proposition 2.2.1, a Lyapounov function allows to characterize the convergence properties of an APT related to a DI. Proposition 2.2.3 ( [Benaïm et al. 2005, Theorem 3.6 and Proposition 3.27]). Assume that H verifies the assumptions of Proposition 2.2.2, X is a bounded APT of the DI (2.4) and V is a Lyapounov function for a set A. Assume that V pAq is of empty interior and denote L X the limit set of X. The following holds.

• We have that L X Ă A.

• The function V is constant on L X .

Example 2.2.6. Let f : R d Ñ R be a path-differentiable function. Consider the setting of Example 2.2.4, with H " ´Bf . We have that f px n q converges and the accumulation points of x n are in the set tx P R d : 0 P Bf pxqu of Clarke critical points of f . Remark 2. In the context of the preceding example it is not necessary to have the existence of a C ě 0 such that:

supt v : v P Bf pxqu ď Cp1 ` x q .
Indeed, since the sequence px n q is bounded, it is not hard to construct f that agrees with f on a compact set containing px n q and such that B f satisfies all of the assumptions of Proposition 2.2.2. Such a construction is presented in Chapter 7, Section 7.4.1.

Remark 3. Example 2.2.6 along with the preceding remark provides a simple proof of the main result of [Davis & Drusvyatskiy 2021] on the convergence of the stochastic subgradient descent towards the set of (Clarke) critical points.

Submanifolds

In this section we present some standard results of differential geometry. An interested reader can find more on this subject in [Lafontaine 2015[START_REF] Boumal | An introduction to optimization on smooth manifolds[END_REF].

We say that a smooth function is an immersion if its Jacobian is injective at each point.

Definition 2.3.1 (Submanifold). Consider p ě 1. We say that M Ă R d is a C p submanifold of dimension k if for each y P M , there is U a neighborhood of y, V Ă R k a neighborhood of 0 and ϕ : V Ñ U a C p immersion such that ϕp0q " y, ϕpV q " U X M and ϕ is an homeomorphism on its image.

The function ϕ from the preceding proposition is called a local parametrization of M around y.

Definition 2.3.2 (Tangent and normal spaces). Consider p ě 1 and let M be a C p submanifold of dimension k. Consider y P M , ϕ as in Definition 2.3.1 and denote J ϕ pyq the Jacobian of ϕ at y. The tangent space of M at y, denoted T y M , is a vector space of dimension k defined as:

T y M " Im J ϕ pyq .
The normal space of M at y is N y M " pT y M q K .

Submanifolds 23

Remark 4. Another characterization of the tangent space that we will use is:

T y M " tv P R d : there is a C p curve γ :s ´ε, εrÑ M, γp0q " y, 9
γp0q " vu .

Definition 2.3.3. Consider p ě 1. We say that a function f :

M Ñ R k is C p if
M is a C p submanifold and for every y P M , there is a neighborhood U of y and a C p function F : U Ñ R k that agrees with f on M X U . We call F a (local) smooth extension of f around y.

Lemma 2.3.1. Consider p ě 1 and M Ă R d a C p manifold of dimension k. Let ϕ, U, V be as in Definition 2.3.1. Then the map ϕ ´1 : M X U Ñ V is C p .
Proof. By [Lafontaine 2015, Theorem 1.21] there is Ṽ Ă R d a neighborhood of zero, U 1 Ă U a neighborhood of y and φ : V Ñ U 1 a C p diffeomorphism such that φpx 1 , . . . , x k , 0, . . . , 0q " ϕpx 1 , . . . , x k q. As a consequence, φ´1 is smooth and P R k ˝φ ´1 is a local smooth extension on ϕ ´1 around y, where P R k is the projection onto the first k coordinates.

If a function on a manifold is C 1 , then as in the euclidian case we can define its gradient.

Definition 2.3.4 ( [Boumal 2020, Proposition 3.53]). Let f : M Ñ R be C 1 , y P M and F be a local smooth extension of f around y. We define ∇ M f pyq, the gradient of f at y as:

∇ M f pyq " P TyM ∇F pyq , where P TyM is the orthogonal projection onto T y M . This definition is independent from our choice of F .

Definition 2.3.5 (critical points). Let f : M Ñ R be C 1 . We say that

x ˚P M is a critical point of f if ∇ M f px ˚q " 0.
Every local extremum of a function defined on a submanifold is a critical point. In the euclidian setting the type of a critical point can be determined by the Hessian. A similar information is available for functions defined on a submanifold. Definition 2.3.6. Consider M a C 2 submanifold of R d of dimension greater than 0. Let f : M Ñ R be C 2 and let x ˚P M be a critical point of f . Consider ϕ a local parametrization around x ˚and denote H the Hessian of f ˝ϕ at ϕ ´1px ˚q.

i) We say that x ˚is a nondegenerate critical point if H is invertible.

ii) We say that x ˚is a saddle point if H has at least one negative eigenvalue.

The consistency of the preceding definition comes from the fact that that if x is a critical point of f , then for ϕ 1 , ϕ 2 , any two local parametrizations of M around x ˚, we have:

H 1 " ´Jϕ ´1 2 ˝ϕ1 pϕ ´1px ˚qq ¯T H 2 J ϕ ´1 2 ˝ϕ1 pϕ ´1px ˚qq ,
where for i P t1, 2u, H i is the Hessian of f ˝ϕi at ϕ ´1 i px ˚q and J ϕ ´1 2 ˝ϕ1 is the Jacobian of ϕ ´1 2 ˝ϕ1 . 1 The proof of this result can be found in [Victor 1974, Page 42-43].

1 By Lemma 2.3.1 the composition ϕ ´1 2 ˝ϕ1 is indeed C 2 . Remark 5. An equivalent point of view on saddle points is given by the notion of the Riemannian Hessian. Let f : M Ñ R be C 2 and x ˚be a critical point of f , we define the (Riemannian) Hessian of f at a x ˚as the quadratic form H f px ˚q, defined on R d ˆRd by:

H f,M px ˚q : v Þ Ñ v T P T x ˚M J G px ˚qP T x ˚M v ,
where G is a C 1 function, defined on a neighborhood of x ˚, which agrees with ∇ M f on M . This definition is independent of the choice of G (see [Boumal 2020, Section 5.5]) and a saddle point in this context is a critical point x ˚such that H f,M px ˚q has at least one negative eigenvalue.

The following lemma gathers useful properties of P M , the projection onto M .

Lemma 2.3.2 ( [Lewis & Malick 2008, Lemma 4]). Consider p ě 1 and let M Ă R d be a C p submanifold and y be in M . There is r ą 0 such that P M : Bpy, rq Ñ M is well defined, is C p´1 and the following properties hold. i) For y 1 P M X Bpy, rq, the Jacobian of P M at y 1 is the projection onto T y 1 M .

ii) For x P Bpy, rq, we have x ´PM pxq P N P M pxq M .

We finish this section by a lemma that gives us a Taylor-like expansion of f around a point on a manifold.

Lemma 2.3.3. Let f : M Ñ R be C 2 and consider y P M . For y 1 P M , we have: f py 1 q " f pyq `x∇ M f pyq, y 1 ´yy `Op y 1 ´y 2 q .

Proof. Consider F a local smooth extension of f around y and ϕ a local parametrization of M around y. We have:

f py 1 q " f pyq `x∇F pyq, y 1 ´yy `Op y 1 ´y 2 q .

Moreover, in the neighborhood of y: y 1 ´y " ϕpϕ ´1py 1 qq ´ϕpϕ ´1pyqq

" J ϕ pϕ ´1pyqqpϕ ´1py 1 q ´ϕ´1 pyqq `Op ϕ ´1py 1 q ´ϕ´1 pyq 2 q " J ϕ pϕ ´1pyqqpϕ ´1py 1 q ´ϕ´1 pyqq `Op y 1 ´y 2 q , where the last equality comes from the fact that ϕ ´1 is Lipschitz around y (since it is C 2 ). Moreover, Im J ϕ pϕ ´1pyqq " T y M . Therefore, py 1 ´yq ´PTyM py 1 ´yq " Op y 1 ´y 2 q. This implies:

f py 1 q " f pyq `x∇ M f pyq, y 1 ´yy `Op y 1 ´y 2 q .

o-minimality

An o-minimal structure can be viewed as an axiomatization of diverse properties of semialgebraic sets. In an o-minimal structure, pathological sets such as Peano curves or the graph of the function sin 1

x do not exist. To our knowledge the first work to link ideas between optimization and o-minimal structures was [START_REF] Bolte | [END_REF], where the authors analyzed the structure of the Clarke subdifferential of definable functions and extended the Kurdyka-Łojasiewicz inequality [Kurdyka 1998] to the nonsmooth setting. Nowadays, a rich body of literature enforces this link, see e.g. [START_REF] Davis | [END_REF], Drusvyatskiy & Lewis 2010a, Bolte et al. 2009, Attouch et al. 2011, Bolte & Pauwels 2019]. A nice exposure about usefulness of o-minimal theory in optimization is [Ioffe 2009]. Results on the Verdier and Whitney stratification of definable sets can be found in [Coste 2002[START_REF] Van Den | Geometric categories and o-minimal structures[END_REF], Loi 1998].

Definition and basic properties

Most of the results of this section can be found in [Coste 2002[START_REF] Van Den | Geometric categories and o-minimal structures[END_REF]].

An o-minimal structure is a family O " pO n q nPN˚, where O n is a set of subsets of R n , verifying the following axioms. Sets contained in O are called definable. We call a map f : R k Ñ R m definable if its graph is definable. Definable sets and maps have remarkable stability properties, for instance, if f and A are definable, then f pAq and f ´1pAq are definable, any composition of two functions definable in the same o-minimal structure is definable, and many others. Let us look at some examples of o-minimal structures. Semialgebraic. Semialgebraic sets form an o-minimal structure. A set A Ă R n is semialgebraic if it is a finite union of intersections of sets of the form tQpxq ď 0u, where Q : R n Ñ R is some polynomial. A function is semialgebraic if its graph is a semialgebraic set. Example of such functions include any piecewise polynomial functions but also functions such as x Þ Ñ x q , where q is any rational number. It can be shown that any o-minimal structure contains every semialgebraic set. Globally subanalytic. There is an o-minimal structure that contains, for every n P N, sets of the form tpx, tq : t " f pxqu, where f : r´1, 1s n Ñ R is an analytic function that can be analytically extended in the neighborhood of the hypercube. This comes from the fact that subanalytic sets are stable by taking a projection, which was shown by Gabrielov [Gabrielov 1968, Gabrielov 1996]. The sets belonging to this structure are called globally subanalytic (see [Bolte et al. 2009[START_REF] Bierstone | Edward Bierstone and Pierre Milman. Semianalytic and subanalytic sets[END_REF] for more details). Log-exp. There is an o-minimal structure that contains globally sub-analytic sets as well as the graph of the exponential and the logarithm (see [Wilkie 2009]). As a consequence of this result it can be shown that the loss of a neural network is a definable function [START_REF] Davis | [END_REF]].

1. If Q : R n Ñ R is a polynomial, then tQpxq " 0u P O n .
In the following we fix some o-minimal structure O. Definable will always means definable in O.

An attractive property of definable sets is that they can be constructed by means of first order formulas. A first order formula is constructed according to the following rules.

i) If Q : R n Ñ R is a polynomial, then Qpxq " 0 and Qpxq ą 0 are first order formulas.

ii) If A Ă R n is definable, then x P A is a first order formula.

iii) If Φpxq and Ψpxq are first order formulas, "Ψpxq and Φpxq", "Ψpxq or Φpxq", "not Φpxq" and "Ψpxq ùñ Φpxq" are first order formulas.

iv) If Φpx, yq is a first order formula, where px, yq P R n ˆRl , and A Ă R n is definable, then "Dx P A Ψpx, yq" and "@x P A Ψpx, yq" are first order formulas.

Proposition 2.4.1 ([Coste 2002, Theorem 1.13]). If Φpxq is a first order formula, then the set of x that satisfies Φpxq is a definable set.

The following lemmas show that one dimensional, definable functions behave particularly well.

Lemma 2.4.2 (Monotonicity lemma [van den Dries & Miller 1996, Theorem 4.1 ]). Let f : pa, bq Ñ R, with ´8 ď a ă b ď `8, be a definable function and p ě 0. There is a finite subdivision a " a 0 ă ¨¨¨ă a k " b such that on each interval pa i , a i`1 q f is C p and either constant or strictly monotone.

Lemma 2.4.3 (de l'Hôpital inverse rule [Bolte et al. 2009, Lemma 1]). Let φ, ψ : r0, εq Ñ R be two definable functions that are C 1 on p0, εq and continuous at 0, with φp0q " ψp0q " 0. Assume that @t P p0, εq we have ψ 1 ptq ą 0 and there is l P R s.t. lim tÑ0 φptq ψptq " l. Then lim tÑ0 φ 1 ptq ψ 1 ptq " l.

Lemma 2.4.4 (Definable choice). Let A Ă R n ˆRl be a definable set. Let π n denote the projection on the first n coordinates. Then there is a definable function ρ : π n pAq Ñ R l s.t. for any x P π n pAq, px, ρpxqq P A.

Lemma 2.4.5 (Curve selection lemma [van den Dries & Miller 1996, Theorem 4.6], [Bolte et al. 2009]). Let A Ă R n be a definable set and a P A. For any p ą 0, there is ε ą 0 and a definable curve γ : p´ε, 1q Ñ R n such that γ is C p , γp0q " a and γpp0, 1qq Ă A.

Every definable set can be partitioned into simpler sets called cells. The definition is by induction on n.

Definition 2.4.1. A C p cell of R is either a point tau or an open interval pa, bq, with ´8 ď a ă b ď `8. Assume that we have constructed the C p cells of R n , then there are two types of

C p cells in R n`1 . Graphs. D 1 " tpx, ζ 1 pxqq : x P Du, where D is a C p cell of R n and ζ 1 : D Ñ R is a C p definable function. Bands. D 1 " tpx, yq : ζ 1 pxq ă y ă ζ 2 pxqu, where D is a C p cell of R d and ζ 1 , ζ 2 : D Ñ R are C p definable functions. Definition 2.4.2 (Cylindrical Definable Cell Decomposition (cdcd)). A C p cdcd of R n is a finite partition of R n into C p cells.
We say that a cdcd of R n is compatible with a family A 1 , . . . , A k , where A i Ă R n if every set of the family is a finite union of cells of cdcd.

Proposition 2.4.6 (Cell decomposition [van den Dries & Miller 1996, 4.2]). Given a finite family of definable sets A 1 , . . . , A k Ă R n , there is a C p cdcd of R n compatible with A 1 , . . . , A k .

Proposition 2.4.7 (Piecewise smoothness). Let A Ă R n be a definable set and f : A Ñ R be a definable function. For any p ě 0, there is a C p cdcd of R n compatible with A such that f is continuous on any of its cell.

To each cell we can inductively associate a dimension.

Definition 2.4.3 (Dimension of a cell). Dimension of a point is 0, dimpa, bq " 1.

If a cell D 1 " tpx, ζ 1 pxqq : x P Du is a graph, then dim D 1 " dim D. If a cell D 1 " tpx, yq : ζ 1 pxq ă y ă ζ 2 pxqu is a band, then dim D 1 " dim D `1.
With this definition in hand, Proposition 2.4.6 allows us to define the dimension of any definable set. Definition 2.4.4 (Dimension of a definable set). Given A Ă R n , choose a C p cdcd of R n compatible with A. We define dim A as the maximum dimension of a cell of this cdcd contained in A, dim A is then independent of the chosen cdcd.

Dimension of definable sets verifies many intuitive properties. 1. Let A, B be two definable sets. Then dimpA Y Bq " maxpdim A, dim Bq.

2. Let A, B be definable. Then dimpA ˆBq " dim A `dim B.

If A and f

: A Ñ R n are definable, then dimpf pAqq ď dim A.
4. Let A Ă R nˆl be definable. For x P R n , denote A x " ty P R l : px, yq P Au. Then for d P N, the set A d " tx P R n : dim A x " du is definable and dimpA X A d ˆRl q " dim A d `d.

Chapter 2. Mathematical preliminaries Remark 6. It can be established by induction that every C p cell of dimension k is a k-dimensional submanifold (see [Coste 2002, Chapter 6]). Since the Hausdorff dimension of a k-dimensional C p submanifold is k, this implies that the Hausdorff dimension of a definable set is equal to its "definable dimension" in the sense of Definition 2.4.4.

We finish this section by a result that can be viewed as parametrized version of the curve selection lemma. Its proof is an adaptation of [Loi 1998, Lemma 1.7].

Lemma 2.4.9 (Wing lemma). Let V, S be definable sets such that V Ă SzS. Assume that dim V " k, with k ą 0, let p be an integer and denote P V the projection onto V . There is a definable set U Ă V , open in V , a constant c ą 0 and a definable C p map ρ : U ˆp0, cq Ñ S such that P V pρpy, tqq " y and P V pρpy, tqq ´y " t.

Proof. First, notice that by Proposition 2.4.6 and Remark 6, without loss of generality, we can assume that V is a k-dimensional manifold and, therefore, the projection on V is well defined on its neighborhood.

Let A Ă V ˆR ˆS be the following definable set:

A " tpy, t, xq : y P V, x P S, t ą 0, P V pxq " y, P V pxq ´y " tu .

Let ε : V Ñ R Y t`8u be defined as εpyq :" inftt ą 0 : Dx P S, py, t, xq P Au. Claim: dimpty : εpyq ą 0uq ă k. By contradiction suppose that the dimension is k.

Then by Proposition 2.4.7 there is a set B Ă V , open in V , such that ε is continuous on B. Shrinking B, we can assume that there is a constant c ą 0 such that @y P B, εpyq ą c. This implies that B Ć SzS, a contradiction. Therefore, there is U open in V such that for each y P U , δ ą 0 there is t ă δ and x P S such that py, t, xq P A. Fix y P U , the set tt : Dx P S, py, t, xq P Au Ă R is definable and therefore it is a finite union of points and intervals. Therefore, for each y P U , there is δ ą 0 s.t. for every t ă δ, there is x P S and py, t, xq P A. Let δ : U Ñ R be a function defined as δpyq " suptt 1 : @t P p0, t 1 q, Dx P S, py, t, xq P Au.

We know that for all y P U , δpyq ą 0. Moreover, upon replacing U by a smaller open set, we can assume that δ : U Ñ R is continuous. Upon shrinking U one more time, we have the existence of c ą 0 such that c ă δpyq.

By the curve selection lemma there is ρ : U ˆp0, cq Ñ S s.t. py, t, ρpy, tqq P A. Applying Proposition 2.4.7 to ρ, we obtain that, upon shrinking U and reducing c, ρ is C p on U ˆp0, cq, which finishes the proof.

Stratifications

Various types of cdcd decompositions exist depending on how the neighboring cells fit together. The notion that will be important for us is the notion of stratification. Let A be a set in R d , a C p stratification of A is a finite partition of A into a family of stratas pS i q such that each of the S i is a C p submanifold verifying S i X S j ‰ H ùñ S i Ă S j zS j .

Given a family tA 1 , . . . , A k u of subsets of of A, we say that a stratification pS i q is compatible with tA 1 , . . . , A k u, if each of the A i is a finite union of stratas. We say that a stratification pS i q is definable, if every strata S i is definable.

Different types of stratifications exist depending on how tangent spaces of neighboring stratas fit together. Let us first define the asymmetric distance between two vector spaces E 1 , E 2 :

d a pE 1 , E 2 q " sup uPE 1 , u "1 distpu, E 2 q.
(2.7)

Note that due to the lack of symmetry d a is not a distance. Nevertheless, we have that d a pE 1 , E 2 q " 0 ùñ E 1 Ă E 2 . A distance d between E 1 and E 2 is then classically defined as

dpE 1 , E 2 q " maxtd a pE 1 , E 2 q, d a pE 2 , E 1 qu . (2.8)
This distance is equal to zero if and only if E 1 " E 2 . For a sequence of vector spaces pE n q nPN , we will denote E n Ñ E if dpE n , Eq Ñ 0.

Definition 2.4.5. We say that a C p stratification pS i q satisfies a Whitney-(a) property, if for every couple of distinct stratas S i , S j , for each y P S i X S j and for each sequence px n q nPN P pS j q N such that x n Ñ y, we have:

w-(a) dpT xn S j , τ q Ñ 0 ùñ T y S i Ă τ .

(2.9)

We will refer to pS i q as a Whitney C p stratification.

It is known (see [Coste 2002[START_REF] Van Den | Geometric categories and o-minimal structures[END_REF]) that every definable function f admits a Whitney C p (for any p) stratification pX i q of its domain such that f is C p on each strata. The following "projection formula" relates the Clarke subdifferential Bf pyq of f at y, to ∇ X i f pyq.

Lemma 2.4.10 (Projection formula, [Bolte et al. 2007, Lemma 8]). Let f : R d Ñ R be a locally Lipschitz, definable function and p a positive integer. There is pS i q, a definable Whitney C p stratification of Graphpf q, such that if one denotes by X i the projection of S i onto its first d coordinates, the restriction f : X i Ñ R is C p and the family pX i q is a Whitney C p stratification of R d . Moreover, for any y P X i and v P Bf pyq, we have P TyX i pvq " ∇ X i f pyq.

Lemma 2.4.10 has important consequences. One of them (see [Davis et al. 2020, Section 5] is that every locally Lipschitz continuous and definable function is pathdifferentiable.

Lemma 2.4.11 ([Davis et al. 2020, Theorem 5.8]). Let f : R d Ñ R be a locally Lipschitz continuous function. If Graphpf q admits a Whitney C 1 stratification, then f is path-differentiable.

Chapter 2. Mathematical preliminaries

A Verdier stratification is a special case of Whitney stratification, which posit a stronger condition on the (asymmetric) distance between adjacent stratas. Whereas the Whitney stratification can now be considered as well known in optimization community, the Verdier stratification is comparatively less popular. We illustrate its advantage by establishing in Theorem 5.2.1 a Lipschitz-like condition in the "projection formula" (Lemma 2.4.10). We believe that this strengthened result is of independent interest. Definition 2.4.6. Let pS i q be a C p stratification of some set A Ă R d . We say that pS i q satisfies a Verdier property (v), if for every couple of distinct stratas S i , S j and for each y P S i X S j ‰ H, there are two positive constants δ, C such that: pvq y 1 P Bpy, δq X S i x P Bpy, δq X S j ùñ d a pT y 1 S i , T x S j q ď C y 1 ´x .

(2.10)

We refer to pS i q as a Verdier C p stratification of A.

It is clear from the definitions that a Verdier C p stratification is always a Whitney C p stratification. A fundamental result is that every definable set admits a Verdier stratification.

Proposition 2.4.12 ( [Loi 1998, Theorem 1.3]). Let tA 1 , . . . , A k u be a family of definable sets of R d . For any p ě 1, there is a Verdier C p stratification of R d compatible with tA 1 , . . . , A k u.

In Chapter 5 this proposition will be used to prove a reinforced version of Lemma 2.4.10.

Chapter 3

Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance

Introduction

Given a probability space Ξ, an integer d ą 0, and a function f : R d ˆΞ Ñ R, consider the problem of finding a local minimum of the function F pxq ∆ " E ξ rf px, ξqs w.r.t. x P R d , where E ξ represents the expectation w.r.t. the random variable ξ on Ξ. This chapter focuses on the case where F is possibly non-convex. It is assumed that the function F is unknown to the observer, either because the distribution of ξ is unknown, or because the expectaction cannot be evaluated. Instead, a sequence pξ n : n ě 1q of i.i.d. copies of the random variable ξ is revealed online.

While the Stochastic Gradient Descent is the most classical algorithm that is used to solve such a problem, recently, several other algorithms became very popular. These include the Stochastic Heavy Ball (SHB), the stochastic version of Nesterov's Accelerated Gradient method (S-NAG) and the large class of the so-called adaptive gradient algorithms, among which Adam [START_REF] Kingma | [END_REF] is perhaps the most used in practice. As opposed to the vanilla Stochastic Gradient Descent, the study of such algorithms is more elaborate, for three reasons. First, the update of the iterates involves a so-called momentum term, or inertia, which has the effect of "smoothing" the increment between two consecutive iterates. Second, the update equation at the time index n is likely to depend on n, making these systems inherently nonautonomous. Third, as far as adaptive algorithms are concerned, the update also depends on some additional variable (a.k.a. the learning rate) computed online as a function of the history of the computed gradients.

In this chapter, we study in a unified way the asymptotic behavior of these algorithms in the situation where F is a differentiable function which is not necessarily convex, and where the stepsize of the algorithm is decreasing.

Our starting point is a generic non-autonomous Ordinary Differential Equation (ODE) introduced by Belotto da Silva and Gazeau [Belotto da Silva & Gazeau 2020] (see also [Barakat & Bianchi 2021] for Adam), depicting the continuous-time versions of the aforementioned florilegium of algorithms. The solutions to the ODE are shown to converge to the set of critical points of F . This suggests that a general provably convergent algorithm can be obtained by means of an Euler discretization of the ODE, including possible stochastic perturbations. Special cases of our general algorithm include SHB, Adam and S-NAG. We establish the almost sure boundedness and the convergence to critical points. Under additional assumptions, we obtain convergence rates, under the form of a central limit theorem. These results are new. They extend the works of [START_REF] Gadat | [END_REF], Barakat & Bianchi 2021] to a general setting. In particular, we highlight the almost sure convergence result of S-NAG in a non-convex setting, which is new to the best of our knowledge.

Next, we address the question of the avoidance of "traps". In a non-convex setting, the set of critical points of a function F is generally larger than the set of local minimizers. A "trap" stands for a critical point at which the Hessian matrix of F has negative eigenvalues, namely, it is a local maximum or saddle point. We establish that the iterates cannot converge to such a point, if the noise is exciting in some directions. The result extends previous works of [START_REF] Gadat | [END_REF] obtained in the context of SHB. This result not only allows to study a broader class of algorithms but also significantly weakens the assumptions. In particular, [START_REF] Gadat | [END_REF]] uses a sub-Gaussian assumption on the noise and a rather stringent assumption on the stepsizes. The main difficulty in the approach of [START_REF] Gadat | [END_REF] lies in the use of the classical autonomous version of Poincaré's invariant manifold theorem. The key ingredient of our proof is a general avoidance of traps result, adapted to non-autonomous settings, which we believe to be of independent interest. It extends usual avoidance of traps results to a non-autonomous setting, by making use of a non-autonomous version of Poincaré's theorem [Dalec 1 kiȋ & Kreȋn 1974, [START_REF] Kloeden | [END_REF].

Chapter organization. In Section 3.2, we introduce and study the ODE's governing our general stochastic algorithm. We establish the existence and uniqueness of the solutions, as well as the convergence to the set of critical points. In Section 3.3, we introduce the main algorithm. We provide sufficient conditions under which the iterates are bounded and converge to the set of critical points. A central limit theorem is stated. Section 3.4 introduces a general avoidance of traps result for non-autonomous settings. Next, this result is applied to the proposed algorithm. Sections 3.5, 3.6 and 3.7 are devoted to the proofs of the results of Sections 3.2, 3.3 and 3.4, respectively.

Notations. Given an integer d ě 1, two vectors x, y P R d , and a real α, we denote by x d y, x dα , x{y, |x|, and a |x| the vectors in R d whose i-th coordinates are respectively given by x i y i ,

x α i , x i {y i , |x i |, a |x i |.
Inequalities of the form x ď y are to be read componentwise. The standard Euclidean norm is denoted } ¨}. Notation M T represents the transpose of a matrix M . For x P R d and ρ ą 0, the notation Bpx, ρq stands for the open ball of R d with center x and radius ρ. We also write R `" r0, 8q. If z P R d and A Ă R d , we write distpz, Aq ∆ " inft}z ´z1 } : z 1 P Au. By 1 A pxq, we refer to the function that is equal to one if x P A and to zero elsewhere.

The set of zeros of a function h : R d Ñ R d 1 is zer h " tx : hpxq " 0u. Let D be a domain in R d . Given an integer k ě 0, the class C k pD, Rq is the class of D Ñ R maps such that all their partial derivatives up to the order k exist and are continuous. For a function h P C k pD, Rq and for every i P t1, . . . , du, we denote as B k i hpx 1 , . . . , x d q the k th partial derivative of the function h with respect to x i . When k " 1, we just write B i hpx 1 , . . . , x d q. The gradient of a function F : R d Ñ R at a point x P R d is denoted as ∇F pxq, and its Hessian matrix at x is ∇ 2 F pxq as usual. For a function S : R d Ñ R d , the notation ∇Spxq stands for the jacobian matrix of S at point x.

Ordinary Differential Equations

A general ODE

Our starting point will be a non-autonomous ODE which is almost identical to the one introduced in [Belotto da Silva & Gazeau 2020] and close to the one in [Barakat & Bianchi 2021]. Let F be a function in C 1 pR d , Rq, let S be a continuous R d Ñ R d `function, let h, r, p, q : p0, 8q Ñ R `be four continuous functions, and let ε ą 0. Let v 0 P R d `and

x 0 , m 0 P R d . Starting at vp0q " v 0 , mp0q " m 0 , and xp0q " x 0 , our ODE on

R ẁith trajectories in Z `∆ " R d `ˆR d ˆRd reads $ ' ' & ' ' % 9
vptq " pptqSpxptqq ´qptqvptq 9 mptq " hptq∇F pxptqq ´rptqmptq

9 xptq " ´mptq{ a vptq `ε (ODE-1)
This ODE can be rewritten compactly in the following form. Write z 0 " pv 0 , m 0 , x 0 q, and let zptq " pvptq, mptq, xptqq P Z `for t P R `. Let Z ∆ " R d ˆRd ˆRd , and define the map g : Z `ˆp0, 8q Ñ Z as gpz, tq "

» - pptqSpxq ´qptqv hptq∇F pxq ´rptqm ´m{ ? v `ε fi fl (3.1)
for z " pv, m, xq P Z `. With these notations, we can rewrite (ODE-1) as zp0q " z 0 , 9 zptq " gpzptq, tq for t ą 0.

By setting Spxq " ∇F pxq d2 when necessary and by properly choosing the functions h, r, p, and q, a large number of iterative algorithms used in Machine Learning can be obtained by an Euler's discretization of this ODE. For instance, choosing hptq " rptq " apt, λ, α 1 q and pptq " qptq " apt, λ, α 2 q with apt, λ, αq " λ ´1p1 éxpp´λαqq{p1 ´expp´αtqq and λ, α 1 , α 2 ą 0, one obtains a version of the Adam algorithm [START_REF] Kingma | [END_REF] (see [Belotto da Silva & Gazeau 2020, Sections 2.4-4.2] for details). To give another less specific example, if we set p " q " 0, then the resulting ODE covers a family of algorithms to which the well-known Heavy Ball with friction algorithm [Attouch et al. 2000] belongs. For a comprehensive and more precise view of the deterministic algorithms that can be deduced from (ODE-1) by an Euler's discretization, the reader is referred to [Belotto da Silva & Gazeau 2020, Table 1].

In this chapter, since we are rather interested in stochastic versions of these algorithms, Equation (ODE-1) will be the basic building block of the classical "ODE method" which is widely used in the field of stochastic approximation [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF]]. In order to analyze the behavior of this equation in preparation of the stochastic analysis, we need the following assumptions. i) h P C 1 pp0, `8q, R `q, 9 hptq ď 0 on p0, `8q and the limit h 8 ∆ " lim tÑ8 hptq is positive.

ii) r and q are non-increasing and r 8 ∆ " lim tÑ8 rptq , q 8 ∆ " lim tÑ8 qptq are positive.

iii) p converges towards p 8 as t Ñ 8. iv) For all t P p0, `8q, rptq ě qptq{4 and r 8 ą q 8 {4.

These assumptions are sufficient to prove the existence and the uniqueness of the solution to (ODE-1) starting at a time t 0 ą 0. The following additional assumption extends the solution to t 0 " 0. Assumption 3.2.5. Either h, r, p, q P C 1 pr0, `8q, R `q, or the following holds:

i) For every x P R d , we have Spxq ě ∇F pxq d2 . ii) The functions h p , h q´2r , t Þ Ñ thptq, t Þ Ñ trptq, t Þ Ñ tpptq, t Þ Ñ tqptq are bounded near zero.
iii) There exists t 0 ą 0 such that for all t ă t 0 , 2rptq ´qptq ą 0 . iv) There exists δ ą 0 such that h r , p q P C 1 pr0, δq, R `q .

v) The initial condition z 0 " pv 0 , m 0 , x 0 q P Z `satisfies m 0 " ∇F px 0 q lim tÓ0 hptq rptq and v 0 " Spx 0 q lim tÓ0 pptq qptq .

Remark 7. The functions h, r, p, q corresponding to Adam satisfy these conditions. We leave the straightforward verifications to the reader. We just observe here that the function S that will correspond to our stochastic algorithm in Section 3.3 below will satisfy Assumption 3.2.5-i) by an immediate application of Jensen's inequality.

The following theorem slightly generalizes the results of [Belotto da Silva & Gazeau 2020, Theorem 3 and Theorem 5].

Theorem 3.2.1. Let Assumptions 3.2.1 to 3.2.4 hold true. Consider z 0 P Z `and t 0 ą 0. Then, there exists a unique global solution z : rt 0 , `8q Ñ Z `to (ODE-1) with initial condition zpt 0 q " z 0 . Moreover, zprt 0 , `8qq is a bounded subset of Z `. As t Ñ `8, zptq converges towards the set Υ ∆ " tz ‹ " pp 8 Spx ‹ q{q 8 , 0, x ‹ q : x ‹ P zer ∇F u .

(3.2)

If, additionally, Assumption 3.2.5 holds, then we can take t 0 " 0.

Remark 8. Theorem 3.2.1 only shows the convergence of the trajectory zptq towards a set. Convergence of the trajectory towards a single point is not guaranteed when the set Υ is not countable.

Remark 9. A simpler version of (ODE-1) is obtained when omitting the momentum term. It reads: # 9 vptq " pptqSpxptqq ´qptqvptq

9 xptq " ´∇F pxptqq{ a vptq `ε . (ODE-1 1 )
This ODE encompasses the algorithms of the family of RMSProp [Tieleman & Hinton 2012], as shown in [Barakat & Bianchi 2021, Belotto da Silva & Gazeau 2020]. The approach for proving the previous theorem can be adapted to (ODE-1 1 ) with only minor modifications. In the proofs below, we will point out the particularities of (ODE-1 1 ) when necessary.

The following paragraph is devoted to a particular case of (ODE-1), which does not satisfy Assumption 3.2.4, and which requires a more involved treatment than (ODE-1 1 ).

The Nesterov case

The authors of [START_REF] Cabot | [END_REF]], [Su et al. 2016b] and others studied the ODE : xptq `α t 9 xptq `∇F pxptqq " 0, α ą 0, F P C 1 pR d , Rq, which Euler's discretization generates the well-known Nesterov's accelerated gradient algorithm, see also [Attouch et al. 2018, Aujol et al. 2019]. This ODE can be rewritten as # 9 mptq " ∇F pxptqq ´α t mptq

9 xptq " ´mptq, (ODE-N)
which is formally the particular case of (ODE-1) that is taken for pptq " qptq " 0, hptq " 1, and rptq " α{t. Obviously, this case is not covered by Assumption 3.2.4. Moreover, it turns out that, contrary to the situation described in Remark 9 above, this case cannot be dealt with by a straightforward adaptation of the proof of Theorem 3.2.1. The reason for this is as follows. Heuristically, the proof of Theorem 3.2.1 is built around the fact that the solution of (ODE-1) "shadows" for large can have solutions that do not converge to the critical points of F . As an example of such solutions, take d " 1 and F pxq " x 2 {2. Then, t Þ Ñ pcosptq, sinptqq is an oscillating solution of the latter ODE. Yet, we have the following result. Up to our knowledge, the proof of the convergence below as t Ñ `8 is new.

Theorem 3.2.2. Let Assumptions 3.2.1 and 3.2.2 hold true. Then, for each x 0 P R d , there exists a unique bounded global solution pm, xq : R `Ñ R d ˆRd to (ODE-N) with the initial condition pmp0q, xp0qq " p0, x 0 q. As t Ñ `8, pmptq, xptqq converges towards the set Ῡ ∆ " tp0, x ‹ q : x ‹ P zer ∇F u.

(3.3)

Related works

The continuous-time dynamical system (ODE-1) we consider was first introduced in [Belotto da Silva & Gazeau 2020, Equation (2.1)] with S " ∇F d2 . Theorem 3.2.1 above is roughly the same as [Belotto da Silva & Gazeau 2020, Ths. 3 and 5], with some slight differences regarding the assumptions on the function F , or Assumption 3.2.4-iv). We point out that the main focus of [Belotto da Silva & Gazeau 2020] is to study the properties of the deterministic continous-time dynamical system (ODE-1). In the present chapter, we highlight that the purpose of Theorem 3.2.1 is to pave the way to our analysis of the corresponding stochastic algorithms in Section 3.3. Concerning Theorem 3.2.2, the existence and the uniqueness of a global solution to (ODE-N) has been previously shown in the literature, for instance in [Cabot et al. 2009, Proposition 2.1] or in [Su et al. 2016b, Theorem 1]. The convergence statement in Theorem 3.2.2 is new to the best of our knowledge. In particular, we stress that we do not make any convexity assumption on F . The closest result we are aware of is the one of Cabot-Engler-Gadat [START_REF] Cabot | [END_REF]. In [Cabot et al. 2009, Proposition 2.5], it is shown that if xptq converges towards some point x, then necessarily x is a critical point of F . Our result in Theorem 3.2.2 strengthens this statement, by establishing that xptq actually converges to the set of critical points.

Stochastic Algorithms

In this section, we discuss the asymptotic behavior of stochastic algorithms that consist in noisy Euler's discretizations of (ODE-1) and (ODE-N) studied in the previous section.

We first set the stage. Let pΞ, T , µq be a probability space. Denoting as BpR d q the Borel σ-algebra on R d , consider a BpR d qbT -measurable function f : R d ˆΞ Ñ R that satisfies the following assumption. Assumption 3.3.1. The following conditions hold:

i) For every x P R d , f px, ¨q is µ-integrable.
ii) For every s P Ξ, the map f p¨, sq is differentiable. Denoting as ∇f px, sq its gradient w.r.t. x, the function ∇f px, ¨q is integrable.

iii) There exists a measurable map κ : R d ˆΞ Ñ R `s.t. for every x P R d :

a) The map κpx, ¨q is µ-integrable, b) There exists ε ą 0 s.t. for every s P Ξ, @ u, v P Bpx, εq, }∇f pu, sq ´∇f pv, sq} ď κpx, sq}u ´v} .

Under Assumption 3.3.1, we can define the mapping F : R d Ñ R as

F pxq " E ξ rf px, ξqs (3.4)
for all x P R d , where we write E ξ ϕpξq " ş ϕpξqµpdξq. It is easy to see that the mapping F is differentiable, ∇F pxq " E ξ r∇f px, ξqs for all x P R d , and ∇F is locally Lipschitz.

Let pγ n q ně1 be a sequence of positive real numbers satisfying Assumption 3.3.2. γ n`1 {γ n Ñ 1 and ř n γ n " `8. Define for every integer n ě 1

τ n " n ÿ k"1 γ k .
Let pΩ, F , Pq be a probability space, and let pξ n : n ě 1q be a sequence of iid random variables defined from pΩ, F , Pq into pΞ, T , µq with the distribution µ.

General algorithm

Our first algorithm is a discrete and noisy version of (ODE-1). Let z 0 " pv 0 , m 0 , x 0 q P Z `and h 0 , r 0 , p 0 , q 0 P p0, 8q. Define for every n ě 1 h n " hpτ n q, r n " rpτ n q, p n " ppτ n q, and q n " qpτ n q.

(3.5)

The algorithm is written as follows.

Algorithm 1 (general algorithm)

Initialization: z 0 P Z `. for n " 1 to n iter do v n`1 " p1 ´γn`1 q n qv n `γn`1 p n ∇f px n , ξ n`1 q d2 m n`1 " p1 ´γn`1 r n qm n `γn`1 h n ∇f px n , ξ n`1 q x n`1 " x n ´γn`1 m n`1 { ? v n`1 `ε .
We suppose throughout this chapter that 1 ´γn`1 q n ě 0 for all n P N. This will guarantee that the quantity ? v n `ε is always well-defined (see Algorithm 1). This mild assumption is satisfied as soon as q 0 ď 1 γ 1 since the sequence pq n q is non-increasing and the sequence of stepsizes pγ n q can also be supposed to be nonincreasing.

Since this algorithm makes use of the function ∇f px, ξq d2 , a strengthening of Assumption 3.3.1 is required: Assumption 3.3.3. In Assumption 3.3.1, Conditions ii) and iii) are respectively replaced with the stronger conditions ii') For each x P R d , the function ∇f px, ¨qd2 is µ -integrable.

iii') There exists a measurable map κ : R d ˆΞ Ñ R `s.t. for every x P R d :

a) The map κpx, ¨q is µ-integrable. b) There exists ε ą 0 s.t. @ u, v P Bpx, εq, }∇f pu, sq ´∇f pv, sq} _ }∇f pu, sq d2 ´∇f pv, sq d2 } ď κpx, sq}u ´v}.

Under Assumption 3.3.3, we can also define the mapping S : R d Ñ R d as:

Spxq " E ξ r∇f px, ξq d2 s for all x P R d . Notice that Assumptions 3.2.1 and 3.2.3 are satisfied for F and S. Assumption 3.3.4. Assume either of the following conditions. i) There exists q ě 2 s.t. for every compact set

K Ă R d , sup xPK E ξ }∇f px, ξq} 2q ă 8 and ÿ n γ 1`q{2 n ă 8 . 3.3. Stochastic Algorithms 39 ii) For every compact set K Ă R d , there exists a real σ K ‰ 0 s.t. E ξ expxu, ∇f px, ξq ´∇F pxqy1 xPK ď exp `σ2 K }u} 2 {2 ˘and E ξ expxu, ∇f px, ξq d2 ´Spxqy1 xPK ď exp `σ2 K }u} 2 {2 ˘,
for every x, u P R d . Moreover, for every α ą 0, ř n expp´α{γ n q ă 8 .

Remark 10. We make the following comments regarding Assumption 3.3.4.

• Assumption 3.3.4-i) allows to use larger stepsizes in comparison to the classical condition ř n γ 2 n ă 8 which corresponds to the particular case q " 2.

• Recall that a random vector X is said to be subgaussian if there exists a real σ ‰ 0 s.t. Ee xu,Xy ď e σ 2 }u} 2 {2 for every constant vector u P R d . In Assumption 3.3.4-ii), the subgaussian noise offers the possibility to use a sequence of stepsizes with an even slower decay rate than in Assumption 3.3.4-i).

Assumption 3.3.5. The set F ptx : ∇F pxq " 0uq has an empty interior.

Remark 11. Assumption 3.3.5 excludes a pathological behavior of the objective function F at critical points. It is satisfied when

F P C k pR d , Rq for k ě d.
Indeed, in this case, Sard's theorem stipulates that the Lebesgue measure of F ptx : ∇F pxq " 0uq is zero in R.

Theorem 3.3.1. Let Assumptions 3.2.2, 3.2.4, and 3.3.2-3.3.5 hold true. Assume that the random sequence pz n " pv n , m n , x n q : n P Nq given by Algorithm 1 is bounded with probability one. Then, w.p.1, the sequence pz n q converges towards the set Υ defined in Equation (3.2). If, in addition, the set of critical points of the objective function F is finite or countable, then w.p.1, the sequence pz n q converges to a single point of Υ.

We now deal with the boundedness problem of the sequence pz n q. We introduce an additional assumption for this purpose.

Assumption 3.3.6. The following conditions hold. 2, 3.2.4, 3.3.2, 3.3.3, 3.3.4-i) (with q " 2) and 3.3.6 hold. Then, the sequence pv n , m n , x n q given by Algorithm 1 is bounded with probability one.

i) ∇F is (globally) Lipschitz continuous. ii) There exists C ą 0 s.t. for all x P R d , E ξ r}∇f px, ξq} 2 s ď Cp1 `F pxqq , iii) ř n γ 2 n ă 8 . Theorem 3.3.2. Let Assumptions 3.2.
Remark 12. The above stability result requires square summable step sizes. Showing the same boundedness result under the Assumption 3.3.4 that allows for larger step sizes is a challenging problem in the general case. In these situations, the boundedness of the iterates can be sometimes ensured by ad hoc means.

Remark 13. We can also consider the noisy discretization of (ODE-1 1 ) introduced in Remark 9 above. This algorithm reads

# v n`1 " p1 ´γn`1 q n qv n `γn`1 p n ∇f px n , ξ n`1 q d2 (3.6a) x n`1 " x n ´γn`1 ∇f px n , ξ n`1 q{ ? v n`1 `ε (3.6b)
for pv 0 , x 0 q P R d `ˆR d . With only minor adaptations, Theorem 3.3.1 and Theorem 3.3.2 can be shown to hold as well for this algorithm. We refer to the concomitant paper [Gadat & Gavra 2020, Sec. 2.2] for the link between this algorithm and the seminal algorithms AdaGrad [START_REF] Duchi | [END_REF] and RMSProp [Tieleman & Hinton 2012].

Stochastic Nesterov's Accelerated Gradient (S-NAG)

S-NAG is the noisy Euler's discretization of (ODE-N). Given α ą 0, it generates the sequence pm n , x n q on R d ˆRd given by Algorithm 2.

Algorithm 2 (S-NAG with decreasing steps)

Initialization: m 0 " 0, x 0 P R d . for n " 1 to n iter do m n`1 " p1 ´αγ n`1 {τ n qm n `γn`1 ∇f px n , ξ n`1 q x n`1 " x n ´γn`1 m n`1 .
Assumption 3.3.7. Assume either of the following conditions.

i) There exists q ě 2 s.t. for every compact set

K Ă R d , sup xPK E ξ }∇f px, ξq} q ă 8 and ÿ n γ 1`q{2 n ă 8 .
ii) For every compact set K Ă R d , there exists a real σ K ‰ 0 s.t.

E ξ expxu, ∇f px, ξq ´∇F pxqy1 xPK ď exp `σ2 K }u} 2 {2 ˘,
for every x, u P R d . Moreover, for every α ą 0, ř n expp´α{γ n q ă 8 . Theorem 3.3.3. Let Assumptions 3.2.2, 3.3.1, 3.3.2, 3.3.5 and 3.3.7 hold true. Assume that the random sequence py n " pm n , x n q : n P Nq given by Algorithm 2 is bounded with probability one. Then, w.p.1, the sequence py n q converges towards the set Ῡ defined in Equation (3.3). If, in addition, the set of critical points of the objective function F is finite or countable, then w.p.1, the sequence py n q converges to a single point of Ῡ.

The almost sure boundedness of the sequence py n q is handled in what follows.

Theorem 3.3.4. Let Assumptions 3.2.2, 3.3.1, 3.3.2 and 3.3.6 hold. Then, the sequence py n " pm n , x n q : n P Nq given by Algorithm 2 is bounded with probability one.

Remark 14. Assumption 3.3.4-i) in Theorem 3.3.2 is not needed for Theorem 3.3.4.

Central Limit Theorem

In this section, we establish a conditional central limit theorem for Algorithm 1.

Assumption 3.3.8. Let x ‹ P zer ∇F . The following holds. i) F is twice continuously differentiable on a neighborhood of x ‹ and the Hessian ∇ 2 F px ‹ q is positive definite.

ii) S is continuously differentiable on a neighborhood of x ‹ .

iii) There exists M ą 0 and b M ą 4 s.t.

sup xPBpx‹,M q E ξ r}∇f px, ξq} b M s ă 8 . (3.7)
Under Assumptions 3.2.4-i) to iii), it follows from Equation (3.5) that the sequences ph n q, pr n q, pp n q and pq n q of nonnegative reals converge respectively to h 8 , r 8 , p 8 and q 8 where h 8 , r 8 and q 8 are supposed positive. Define v ‹ ∆ " q ´1 8 p 8 Spx ‹ q. Consider the matrix

V ∆ " diag ´pε `v‹ q d´1 2 ¯.
(3.8)

Let P be an orthogonal matrix s.t. the following spectral decomposition holds:

V 1 2 ∇ 2 F px ‹ qV 1 2 " P diagpπ 1 , ¨¨¨, π d qP ´1 ,
where

π 1 ď ¨¨¨ď π d are the (positive) eigenvalues of V 1 2 ∇ 2 F px ‹ qV 1 2 . Define H ∆ " " ´r8 I d h 8 ∇ 2 F px ‹ q ´V 0 
where I d is the d ˆd identity matrix. Then the matrix H is Hurwitz. Indeed, it can be shown that the largest real part of the eigenvalues of H coincides with ´L, where

L ∆ " r 8 2 ˜1 ´dˆ1 ´4h 8 π 1 r 2 8 ˙_ 0 ¸ą 0 . (3.9)
Assumption 3.3.9. The sequence pγ n q is given by γ n " γ 0 n α for some α P p0, 1s, γ 0 ą 0. Moreover, if α " 1, we assume that γ 0 ą 1 2pL^q8q .

Theorem 3.3.5. Let Assumptions 3.2.4-i) to iii), 3.3.3, 3.3.8 and 3.3.9 hold. Consider the iterates z n " pv n , m n , x n q given by Algorithm 1. Set θ ∆ " 0 if α ă 1 and θ ∆ " 1{p2γ 0 q if α " 1. Assume that the event tz n Ñ z ‹ u, where z ‹ " pv ‹ , 0, x ‹ q, has a positive probability. Then, given that event,

1 ? γ n " m n x n ´x‹  ñ N p0, Γq ,
where ñ stands for the convergence in distribution and N p0, Γq is a centered Gaussian distribution on R 2d with a covariance matrix Γ given by the unique solution to the Lyapunov equation pH `θI 2d qΓ `ΓpH `θI 2d q T "

´"Covph 8 ∇f px ‹ , ξqq 0 0 0  .

In particular, given tz n Ñ z ‹ u, the vector ? γ n ´1px n ´x‹ q converges in distribution to a centered Gaussian distribution with a covariance matrix given by:

Γ 2 " V 1 2 P » - C k, r8´2θ h8 pπ k `π `2θpθ´r8q h8 q `pπ k ´π q 2 2pr8´2θq fi fl k, "1...d P ´1V 1 2 (3.10)
where

C ∆ " P ´1V 1 2 E ξ " ∇f px ‹ , ξq∇f px ‹ , ξq T ‰ V 1 2 P .
A few remarks are in order.

• The matrix Γ 2 coincides with the limiting covariance matrix associated to the iterates

# m n`1 " m n `γn`1 ph 8 V ∇f px n , ξ n`1 q ´r8 m n q x n`1 " x n ´γn`1 m n`1 .
This procedure can be seen as a preconditioned version of the stochastic heavy ball algorithm [START_REF] Gadat | [END_REF]] although the iterates are not implementable because of the unknown matrix V . Notice also that the limiting covariance Γ 2 depends on v ‹ but does not depend on the fluctuations of the sequence pv n q.

• When h 8 " r 8 (which is the case for Adam), we recover the expression of the asymptotic covariance matrix previously provided in [Barakat & Bianchi 2021, Section 5.3] and the remarks formulated therein.

• The assumption r 8 ą 0 is crucial to establish Theorem 3.3.5. For this reason, Theorem 3.3.5 does not generalize immediately to Algorithm 2. The study of the fluctuations of Algorithm 2 is left for future works.

Related works

In [START_REF] Gadat | [END_REF]], Gadat, Panloup and Saadane study the SHB algorithm, which is a noisy Euler's discretization of (ODE-1) in the situation where h " r and p " q " 0 (i.e., there is no v variable). In this framework, if we set h " r " r ą 0 in Algorithm 1 above, then Theorem 3.3.1 above recovers the analogous case in [Gadat et al. 2018, Theorem 2.1], which is termed as the exponential memory case. The other important case treated in [START_REF] Gadat | [END_REF] is the case where hptq " rptq " r{t for some r ą 0, referred to as the polynomially memory case. Actually, it is known that the ODE obtained for hptq " rptq " r{t and p " q " 0 boils down to (ODE-N) after a time variable change (see, e.g., Lemma 3.5.3 below).

Nevertheless, we highlight that the stochastic algorithm that stems from this ODE and that is studied in [START_REF] Gadat | [END_REF] is different from the S-NAG algorithm introduced above which stems from a different ODE (ODE-N). Hence, the convergence result of Theorem 3.3.3 for the S-NAG algorithm we consider is not covered by the analysis of [START_REF] Gadat | [END_REF]].

The specific case of the Adam algorithm is analyzed in [Barakat & Bianchi 2021] in both the constant and vanishing stepsize settings (see [Barakat & Bianchi 2021, Ths. 5.2-5.4] which are the analogues of our Ths. 3.3.1-3.3.2). Note that we deal with a more general algorithm in the present chapter. Indeed, Algorithm 1 offers some freedom in the choice of the functions h, r, p, q satisfying Assumption 2.4 beyond the specific case of the Adam algorithm studied in [Barakat & Bianchi 2021]. Apart from this generalization, we also emphasize some small improvements. Regarding Theorem 3.1, we provide noise conditions allowing to choose larger stepsizes (see Assumption 3.4 compared to [Barakat & Bianchi 2021, Assumption 4.2]). Concerning the stability result (Theorem3.3.2), we relax [Barakat & Bianchi 2021, Assumption 5.3-(iii)] which is no more needed in the present chapter (see Assumption 3.3.6) thanks to a modification of the discretized Lyapunov function used in the proof (see Section 6.4 compared to [Barakat & Bianchi 2021, Section 9.2]).

In most generality, the almost sure convergence result of the iterates of Algorithm 1 using vanishing stepsizes (Ths. 3.3.1-3.3.2) is new to the best of our knowledge. Moreover, while some recent results exist for S-NAG in the constant stepsize and for convex objective functions (see for e.g. [Assran & Rabbat 2020]), Ths. 3.3.3 and 3.3.4 which tackle the possibly non-convex setting are also new to the best of our knowledge.

In the work [Gadat & Gavra 2020] that was posted on the arXiv repository a few days after our submission, Gadat and Gavra study the specific case of the algorithm described in Equation (3.6) encompassing both Adagrad and RMSProp, with the possibility to use mini-batches. For this specific algorithm, the authors establish a similar almost sure convergence result to ours [Gadat & Gavra 2020, Theorem 1] for decreasing stepsizes and derive some quantitative results bounding in expectation the gradient of the objective function along the iterations for constant stepsizes [Gadat & Gavra 2020, Theorem 2]. We highlight though that they do not consider the presence of momentum in the algorithm. Therefore, their analysis does not cover neither Algorithm 1 nor Algorithm 2.

In contrast to our analysis, some works in the literature explore the constant stepsize regime for some stochastic momentum methods either for smooth [START_REF][END_REF] or weakly convex objective functions [Mai & Johansson 2020]. Furthermore, concerning Adam-like algorithms, several recent works control the minimum of the norms of the gradients of the objective function evaluated at the iterates of the algorithm over N iterations in expectation or with high probability [START_REF] De | [END_REF], Zhou et al. 2018, Chen et al. 2018, Zou et al. 2019, Chen et al. 2019, Zaheer et al. 2018, Alacaoglu et al. 2020a, Défossez et al. 2020, Alacaoglu et al. 2020b] and establish regret bounds in the convex setting [Alacaoglu et al. 2020b].

Similar central limit theorems to Theorem 3.3.5 are established in the cases of the stochastic heavy ball algorithm with exponential memory [Gadat et al. 2018, Theorem 2.4] and Adam [Barakat & Bianchi 2021, Theorem 5.7]. In comparison to [START_REF] Gadat | [END_REF], we precise that our theorem recovers their result and provides a closed formula for the asymptotic covariance matrix Γ 2 . Our proof of Theorem 3.3.5 differs from the strategies adopted in [START_REF] Gadat | [END_REF]] and [Barakat & Bianchi 2021].

Avoidance of Traps

In Theorem 3.3.1 and Theorem 3.3.3 above, we established the almost sure convergence of the iterates x n towards the set of critical points of the objective function F for both Algorithms 1 and 2. However, the landscape of F can contain what is known as "traps" for the algorithm, namely, critical points where the Hessian matrix of F has negative eigenvalues, making these critical points local maxima or saddle points. In this section, we show that the convergence of the iterates to these traps does not take place if the noise is exciting in some directions.

Starting with the contributions of Pemantle [Pemantle 1990] and Brandière and Duflo [Brandière & Duflo 1996], the numerous so-called avoidance of traps results that can be found in the literature deal with the case where the ODE that underlies the stochastic algorithm is an autonomous ODE. Obviously, this is neither the case of (ODE-1), nor of (ODE-N). To deal with this issue, we first state a general avoidance of traps result that extends [Pemantle 1990, Brandière & Duflo 1996] to a non-autonomous setting, and that has an interest of its own. We then apply this result to Algorithms 1 and 2.

A general avoidance-of-traps result in a non-autonomous setting

The notations in this subsection and in Sections 3.7.1-3.7.2 are independent from the rest of the chapter. We recall that for a function h : R d Ñ R d 1 , we denote by B k i hpx 1 , . . . , x d q the k th partial derivative of the function h with respect to x i . The setting of our problem is as follows. Given an integer d ą 0 and a continuous function b : R d ˆR`Ñ R d , we consider a stochastic algorithm built around the non-autonomous ODE 9

zptq " bpzptq, tq. Let z ‹ P R d , and assume that on V ˆRẁ here V is a certain neighborhood of z ‹ , the function b can be developed as bpz, tq " Dpz ´z‹ q `epz, tq, (3.11)

where epz ‹ , ¨q " 0, and where the matrix D P R dˆd is assumed to admit the following spectral factorization: Given 0 ď d ´ă d and 0 ă d `ď d with d ´`d `" d, we can write

D " QΛQ ´1, Λ " " Λ ´Λ` , (3.12)
where the Jordan blocks that constitute

Λ ´P R d ´ˆd ´(respectively Λ `P R d `ˆd `)
are those that contain the eigenvalues λ i of D for which λ i ď 0 (respectively λ i ą 0). Since d `ą 0, the point z ‹ is an unstable equilibrium point of the ODE 9 zptq " bpzptq, tq, in the sense that the ODE solution will only be able to converge to z ‹ along a specific so-called invariant manifold which precise characterization will be given in Section 3.7.1 below.

We now consider a stochastic algorithm that is built around this ODE. The condition d `ą 0 makes that z ‹ is a trap that the algorithm should desirably avoid. The following theorem states that this will be the case if the noise term of the algorithm is omnidirectional enough. The idea is to show that the case being, the algorithm trajectories will move away from the invariant manifold mentioned above.

Theorem 3.4.1. Given a sequence pγ n q of nonnegative deterministic stepsizes such that ř n γ n " `8, ř n γ 2 n ă `8, and a filtration pF n q, consider the stochastic approximation algorithm in

R d z n`1 " z n `γn`1 bpz n , τ n q `γn`1 η n`1 `γn`1 ρ n`1 where τ n " ř n k"1 γ k .
Assume that the sequences pη n q and pρ n q are adapted to F n , and that z 0 is F 0 -measurable. Assume that there exists z ‹ P R d such that Equation (3.11) holds true on V ˆR`, where V is a neighborhood of z ‹ . Consider the spectral factorization (3.12), and assume that d `ą 0. Assume moreover that the function e at the right hand side of Equation (3.11) satisfies the conditions: i) epz ‹ , ¨q " 0.

ii) On V ˆR`, the functions B n 2 B k 1 epz, tq exist and are continuous for 0 ď n ă 2 and 0 ď k `n ď 2.

iii) The following convergence holds : lim pz,tqÑpz‹,8q }B 1 epz, tq} " 0 .

(3.13) iv) There exist t 0 ą 0 and a neighborhood

W Ă R d of z ‹ s.t. sup zPW,tět 0 B 2 epz, tq ă `8 and sup zPW,tět 0 B 2 1 epz, tq ă `8 .
Moreover, suppose that :

v) ř n }ρ n`1 } 2 1 znPW ă 8 almost surely. vi) lim sup Er}η n`1 } 4 | F n s1 znPW ă 8, and Erη n`1 | F n s1 znPW " 0.
vii) Writing ηn " Q ´1η n " pη ń , ηǹ q with ηn P R d ˘, for some c 2 ą 0, it holds that

lim inf Er}η ǹ`1 } 2 | F n s1 znPW ě c 2 1 znPW .
Then, Pprz n Ñ z ‹ sq " 0.

Remark 15. Assumptions i) to iv) of Theorem 3.4.1 are related to the function e defined in Equation (3.11), which can be seen as a non-autonomous perturbation of the autonomous linear ODE 9 zptq " Dpzptq ´z‹ q. These assumptions guarantee the existence of a local (around the unstable equilibrium z ‹ ) non-autonomous invariant manifold of the non-autonomous ODE 9

zptq " bpzptq, tq with enough regularity properties, as provided by Proposition 3.7.1 and Proposition 3.7.3 below.

Application to the stochastic algorithms

Trap avoidance of the general algorithm 1

In Theorem 3.3.1 above, we showed that the sequence pz n q generated by Algorithm 1 converges almost surely towards the set Υ defined in Equation (3.2). Our purpose now is to show that the traps in Υ (to be characterized below) are avoided by the stochastic algorithm 1 under a proper omnidirectionality assumption on the noise.

Our first task is to write Algorithm 1 in a manner compatible with the statement of Theorem 3.4.1. The following decomposition holds for the sequence pz n " pv n , m n , x n q, n P Nq generated by this algorithm:

z n`1 " z n `γn`1 gpz n , τ n q `γn`1 η n`1 `γn`1 ρn`1 , where ρn`1 " ´0 , 0 , mn ? vn`ε ´mn`1 ? v n`1
`ε ¯, and where η n`1 is the martingale increment with respect to the filtration pF n q which is defined by Equation (3.28).

Observe from Equation (3.2) that each z ‹ P Υ is written as z ‹ " pv ‹ , 0, x ‹ q where x ‹ P zer ∇F , and v ‹ " q ´1 8 p 8 Spx ‹ q (in particular, x ‹ and z ‹ are in a oneto-one correspondence). We need to linearize the function gp¨, tq around z ‹ . The following assumptions will be required. Assumption 3.4.2. The functions h, r, p, q belong to C 1 pp0, 8q, R `q and have bounded derivatives on rt 0 , `8q for some t 0 ą 0. Lemma 3.4.2. Let Assumptions 3.2.4-i) to iii), 3.4.1 and 3.4.2 hold. Let z ‹ " pv ‹ , 0, x ‹ q P Υ. Then, for every z P Z `and every t ą 0, the following decomposition holds true: gpz, tq " Dpz ´z‹ q `epz, tq `cptq, where D "

» - ´q8 I d 0 p 8 ∇Spx ‹ q 0 ´r8 I d h 8 ∇ 2 F px ‹ q 0 ´V 0 fi fl , cptq " » - pptqSpx ‹ q ´qptqv ‹ 0 0 fi fl ,
and the function epz, tq (defined in Section 3.7.3.1 below for conciseness) has the same properties as its analogue in the statement of Theorem 3.4.1.

Using this lemma, the algorithm iterate z n`1 can be rewritten as an instance of the algorithm in the statement of Theorem 3.4.1, namely, (3.14) where in our present setting, bpz, tq " gpz, tq ´cptq " Dpz ´z‹ q `epz, tq and ρ n " cpτ n´1 q `ρ n . In the following assumption, we use the well-known fact that a symmetric matrix H has the same inertia as AHA T for an arbitrary invertible matrix A.

z n`1 " z n `γn`1 bpz n , τ n q `γn`1 η n`1 `γn`1 ρ n`1 ,
Assumption 3.4.3. Let x ‹ P zer ∇F , let v ‹ " q ´1 8 p 8 Spx ‹ q, and define the diagonal matrix V " diagppv ‹ `εq d´1 2 q as in (3.8). Assume the following conditions: i) ř n pq 8 p n ´p8 q n q 2 ă 8 ,

ii) The Hessian matrix ∇ 2 F px ‹ q has a negative eigenvalue.

iii) There exists δ ą 0 such that sup xPBpx‹,δq E ξ r}∇f px, ξq} 8 s ă 8 .

iv) Defining Π u as the orthogonal projector on the eigenspace of

V 1 2 ∇ 2 F px ‹ qV 1 2
that is associated with the negative eigenvalues of this matrix, it holds that 4, 3.3.3, and 3.4.1, 3.4.2 hold true. Let z ‹ P Υ be such that Assumption 3.4.3 holds true for this z ‹ . Then, the eigenspace associated with the eigenvalues of D with positive real parts has the same dimension as the eigenspace of ∇ 2 F px ‹ q associated with the negative eigenvalues of this matrix. Let pz n " pv n , m n , x n q : n P Nq be the random sequence generated by Algorithm 1 with stepsizes satisfying ř n γ n " `8 and

Π u V 1 2 E ξ p∇f px ‹ , ξq ´∇F px ‹ qqp∇f px ‹ , ξq ´∇F px ‹ qq T V 1 2 Π u ‰ 0. Theorem 3.4.3. Let Assumptions 3.2.
ř n γ 2 n ă `8. Then, Pprz n Ñ z ‹ sq " 0.
The assumptions and the result call for some comments.

Remark 16. The definition of a trap as regards the general algorithm in the statement of Theorem 3.4.1 is that the matrix D in Equation (3.11) has eigenvalues with positive real parts. Theorem 3.4.3 states that this condition is equivalent to ∇ 2 F px ‹ q having negative eigenvalues. What's more, the dimension of the invariant subspace of D corresponding to the eigenvalues with positive real parts is equal to the dimension of the negative eigenvalue subspace of ∇ 2 F px ‹ q. Thus, Assumption 3.4.3-iv) provides the "largest" subspace where the noise energy must be non zero for the purpose of avoiding the trap.

Remark 17. Assumptions 3.4.2 and 3.4.3-i) are satisfied by many widely studied algorithms, among which RMSProp and Adam.

Remark 18. The results of Theorem 3.4.3 can be straightforwardly adapted to the case of (ODE-1 1 ). Assumption 3.4.3-iv) on the noise is unchanged.

In the case of the S-NAG algorithm, the assumptions become particularly simple. We state the afferent result separately.

Trap avoidance for S-NAG

Assumption 3.4.4. Let x ‹ P zer ∇F and let the following conditions hold.

i) The Hessian matrix ∇ 2 F px ‹ q has a negative eigenvalue.

ii) There exists δ ą 0 such that sup xPBpx‹,δq E ξ r}∇f px, ξq} 4 s ă 8 .

iii) Πu E ξ p∇f px ‹ , ξq ´∇F px ‹ qqp∇f px ‹ , ξq ´∇F px ‹ qq T Πu ‰ 0,
where Πu is the orthogonal projector on the eigenspace of ∇ 2 F px ‹ q associated with its negative eigenvalues.

Theorem 3.4.4. Let Assumptions 3.2.4, 3.3.1, 3.4.1 and 3.4.4 hold. Define y ‹ " p0, x ‹ q. Let py n " pm n , x n q : n P Nq be the random sequence given by Algorithm 2 with stepsizes satisfying ř n γ n " `8 and ř n γ 2 n ă `8. Then, Ppry n Ñ y ‹ sq " 0 .

Related works

Up to our knowledge, all the avoidance of traps results that can be found in the literature, starting from [Pemantle 1990, Brandière & Duflo 1996], refer to stochastic algorithms that are discretizations of autonomous ODE's (see for e.g., [Benaïm 1999, Sec. 9] for general Robbins Monro algorithms and [Mertikopoulos et al. 2020a, Sec. 4.3] for SGD). In this line of research, a powerful class of techniques relies on Poincaré's invariant manifold theorem for an autonomous ODE in a neighborhood of some unstable equilibrium point. In our work, we extend the avoidance of traps results to a non-autonomous setting, by borrowing a non-autonomous version of Poincaré's theorem from the rich literature that exists on the subject [Dalec 1 kiȋ & Kreȋn 1974, [START_REF] Kloeden | [END_REF].

In [START_REF] Gadat | [END_REF], the authors succeeded in establishing an avoidance of traps result for their non-autonomous stochastic algorithm which is close to our S-NAG algorithm (see the discussion at the end of Section 3.3.4 above), at the expense of a sub-Gaussian assumption on the noise and a rather stringent assumption on the stepsizes. The main difficulty in the approach of [START_REF] Gadat | [END_REF] lies in the use of the classical autonomous version of Poincaré's theorem (see [Gadat et al. 2018, Remark 2.1]). This kind of difficulty is avoided by our approach, which allows to obtain avoidance of traps results with close to minimal assumptions. More recently, in the contribution of [Gadat & Gavra 2020] discussed in Sec. 3.3.4, the authors establish an avoidance of traps result ([Gadat & Gavra 2020, Theorem 3]) for the algorithm described in Equation (3.6) using techniques inspired from [Pemantle 1990[START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF]]. As previously mentioned, this recent work does not handle momentum and hence neither Algorithm 1 nor Algorithm 2. Moreover, as indicated in our discussion of [START_REF] Gadat | [END_REF], our strategy of proof is different.

Taking another point of view as concerns the trap avoidance, some recent works [START_REF] Lee | [END_REF], Du et al. 2017, Jin et al. 2017[START_REF] Panageas | [END_REF], Panageas et al. 2019] address the problem of escaping saddle points when the algorithm is deterministic but when the initialization point is random. In contrast to this line of research, our work considers a stochastic algorithm for which randomness enters into play at each iteration of the algorithm via noisy gradients.

3.5 Proofs for Section 3.2

Proof of Theorem 3.2.1

The arguments of the proof of this theorem that we provide here follow the approach of [Belotto da Silva & Gazeau 2020] with some small differences. Close arguments can be found in [Barakat & Bianchi 2021]. We provide the proof here for completeness and in preparation of the proofs that will be related with the stochastic algorithms.

Existence and uniqueness

The following lemma guarantees that the term a vptq `ε in (ODE-1) is well-defined.

Lemma 3.5.1. Let t 0 P R `and T P pt 0 , 8s. Assume that there exists a solution zptq " pvptq, mptq, xptqq to (ODE-1) on rt 0 , T q for which vpt 0 q ě 0. Then, for all t P rt 0 , T q, vptq ě 0.

Proof. Assume that ν ∆ " inftt P rt 0 , T q, vptq ă 0u satisfies ν ă T . If vpt 0 q ą 0, Gronwall's lemma implies that vptq ě vpt 0 q expp´ş t t 0 qptqq on rt 0 , νs which is in contradiction with the fact that vpνq " 0. If vpt 0 q " 0, since ν ă T , there exists t 1 P pt 0 , νq s.t. 9 vpt 1 q ă 0. Hence, using the first equation from (ODE-1), we obtain vpt 1 q ą 0. This brings us back to the first case, replacing t 0 by t 1 .

Recall that F ‹ " inf F is finite by Assumption 3.2.2. Of prime importance in the proof will be the energy (Lyapunov) function E : R `ˆZ `Ñ R, defined as

Eph, zq " hpF pxq ´F‹ q `1 2 › › › › › m pv `εq d 1 4 › › › › › 2 , (3.15)
for every h ě 0 and every z " pv, m, xq P Z `. This function is slightly different from its analogues that were used in [Alvarez 2000, Barakat & Bianchi 2021, Belotto da Silva & Gazeau 2020].

Consider pt, zq P p0, `8q ˆZ`a nd set z " pv, m, xq. Then, using Assump-tion 3.2.1, we can write B t Ephptq, zq `x∇ z Ephptq, zq, gpz, tqy

" 9 hptqpF pxq ´F‹ q ´1 4 x m d2 pv `εq d 3 2 , pptqSpxq ´qptqvy `x m pv `εq d 1 2 , hptq∇F pxq ´rptqmy ´x m pv `εq d 1 2 , hptq∇F pxqy ď ´ˆrptq ´qptq 4 ˙› › › › › m pv `εq d 1 4 › › › › › 2 `9 hptqpF pxq ´F‹ q ´pptq 4 xSpxq, m d2 pv `εq d 3 2 y.
(3.16)

With the help of this function, we can now establish the existence, the uniqueness and the boundedness of the solution of (ODE-1) on rt 0 , 8q for an arbitrary t 0 ą 0. Lemma 3.5.2. For each t 0 ą 0 and z 0 P Z `, (ODE-1) has a unique solution on rt 0 , 8q starting at zpt 0 q " z 0 . Moreover, the orbit tzptq : t ě t 0 u is bounded.

Proof. Let t 0 ą 0, and fix z 0 P Z `. On each set of the type rt 0 , t 0 `As ˆBpz 0 , Rq where A, R ą 0 and Bpz 0 , Rq Ă p´ε, 8q d ˆRd ˆRd , we easily obtain from our assumptions that the function g defined in (3.1) is continuous, and that gp¨, tq is uniformly Lipschitz on t P rt 0 , t 0 `As. In these conditions, Picard's theorem asserts that (ODE-1) starting from zpt 0 q " z 0 has a unique solution on a certain maximal interval rt 0 , T q. Lemma 3.5.1 shows that vptq ě 0 on this interval.

Let us show that T " 8. Applying Inequality (3.16) with pv, m, xq " pvptq, mptq, xptqq and using Assumption 3.2.4, we obtain that the function t Þ Ñ Ephptq, zptqq is decreasing on rt 0 , T q. By the coercivity of F (Assumption 3.2.2) and Assumption 3.2.4i), we get that the trajectory txptqu is bounded. Recall the equation 9 mptq " hptq∇F pxptqq ´rptqmptq. Using the continuity of the functions ∇F , h and r along with Gronwall's lemma, we get that tmptqu is bounded if T ă 8. We can show a similar result for tvptqu. Thus, tzptqu is bounded on rt 0 , T q if T ă 8 which is a contradiction, see, e.g., [Hartman 2002, Cor.3.2].

It remains to show that the trajectory tzptqu is bounded. To that end, let us apply the variation of constants method to the equation 9 mptq " hptq∇F pxptqq ŕptqmptq. Writing Rptq " ş t t 0 rpuq du, we get that d dt ´eRptq mptq ¯" e Rptq hptq∇F pxptqq.

Therefore, for every t ě t 0 ,

mptq " e ´Rptq mpt 0 q `ż t t 0 e Rpuq´Rptq hpuq∇F pxpuqqdu .

Using the continuity of ∇F together with the boundedness of x, Assumption 3.2.4 and the triangle inequality, we obtain the existence of a constant C ą 0 independent of t s.t.

mptq ´mpt 0 q ´ mpt 0 q ď Chpt 0 q ż t t 0 e ´şt u rpsq ds du ď Chpt 0 q ż t t 0 e ´r8pt´uq du ď Chpt 0 q r 8 .
The same reasoning applies to vptq using the continuity of S and Assumption 3.2.4. This completes the proof.

We can now extend this solution to t 0 " 0 along the approach of [Belotto da Silva & Gazeau 2020], where the detailed derivations can be found. The idea is to replace hptq with hpmaxpη, tqq for some η ą 0 and to do the same for p, q, and r. It is then easy to see that the ODE that is obtained by doing these replacements has a unique global solution on R `. By making η Ñ 0 and by using the Arzelà-Ascoli theorem along with Assumption 3.2.5, we obtain that (ODE-1) has a unique solution on R `.

Convergence

The first step in this part consists in transforming (ODE-1) into an autonomous ODE by including the time variable into the state vector. More specifically, we start with the following ODE:

" 9 zptq 9 uptq  " " gpzptq, uptqq 1  with " zp0q up0q  " " z 0 t 0  ,
then, we perform the following change of variable in time

" z u  Þ Ñ " z s " 1{u
 allowing the solution to lie in a compact set.

We initialize the above ODE at a time instant t 0 ą 0. Define the functions H, R, P, Q : R `Ñ R `by setting Hpsq " hp1{sq, Rpsq " rp1{sq, Ppsq " pp1{sq; Qpsq " qp1{sq for s ą 0; Hp0q " h 8 , Rp0q " r 8 , Pp0q " p 8 and Qp0q " q 8 . Our autonomous dynamical system can then be described by the following system of equations: (3.17)

$ ' ' ' ' ' & ' ' ' ' ' % 9
Since the solution of the ODE 9 sptq " ´sptq 2 for which spt 0 q " 1{t 0 is sptq " 1{t, the trajectory tsptqu is bounded. The three remaining equations are a reformulation of (ODE-1) for which the trajectories have already been shown to exist and to be bounded in Lemma 3.5.2. In the sequel, we denote by Φ : Z `ˆR `Ñ Z `ˆR `the semiflow induced by the autonomous ODE (3.17), i.e., for every u " pz, sq P Z `R `, Φpu, ¨q is the unique global solution to the autonomous ODE (3.17) initialized at u. Observe that the orbits of this semiflow are precompact. Moreover, the function Φppz, 0q, ¨q is perfectly defined for each z P Z `since the associated solution satisfies the ODE (3.19) defined below, which three first equations satisfy the hypotheses of Lemma 3.5.2.

Consider now a continuous function V : Z `ˆR `Ñ R defined by:

V puq " E pHpsq, zq , u " pz, sq P Z `ˆp0, 8q.

As for Inequality (3.16) above, we have here that

d dt V pΦpu, tqq ď ´ˆrptq ´qptq 4 ˙› › › › › mptq pvptq `εq d 1 4 › › › › › 2 `9 hptqpF pxptqq ´F‹ q ´pptq 4 xSpxptqq, mptq d2
pvptq `εq d 3 2 y if s ą 0, and the same inequality with p 9 hptq, pptq, rptq, qptqq being replaced with p0, p 8 , r 8 , q 8 q otherwise. Since V ˝Φpu, ¨q is non-increasing and nonnegative, we can define

V 8 ∆ " lim tÑ8 V pΦpu, tqq. Let ωpuq ∆ " Ş są0 Ť
těs Φpu, tq be the ω-limit set of the semiflow Φ issued from u. Recall that ωpuq is an invariant set for the flow Φpu, ¨q, and that distpΦpu, tq, ωpuqq Ý ÝÝ Ñ tÑ8 0, see, e.g., [Haraux 1991, Theorem 1.1.8]). In order to finish the proof of Theorem 3.2.1, we need to make explicit the structure of ωpuq.

We know from La Salle's invariance principle that ωpuq Ă V ´1pV 8 q. In particular, @y P ωpuq, @t ě 0, V pΦpy, tqq " V pyq " V 8

(3.18) by the invariance of ωpuq.

From ODE (3.17), we have that any y P ωpuq is of the form y " pz, 0q since sptq Ñ 0. As a consequence, Φpy, ¨q is a solution to the autonomous ODE ¯.

(3.21) Fixing x, let S x be the section of ωpuq defined by: S x ωpuq " ! y P ωpuq : y " pṽ, 0, x, 0q , ṽ P R d `) .

As ωpuq is invariant, we have S x ωpuq " S x Φpωpuq, tq for all t ě 0. Since the set tṽ P R d `s.t. pṽ, 0, x, 0q P S x ωpuqu lies in a compact, we deduce from Equation (3.21) that this set is reduced to the singleton tp 8 Spxq{q 8 u and in particular v " p 8 Spxq{q 8 . Therefore, the union of ω-limit sets of the semiflow Φ induced by ODE (3.17) coincides with the set of equilibrium points of this semiflow. The latter set itself corresponds to the set of points pz, 0q s.t. z P zer g 8 . It remains to notice that Υ " zer g 8 to finish the proof.

Remark 19. Commenting on Remark 9, the same proof works for (ODE-1 1 ) by using the function F ´F‹ as a Lyapunov function. The corresponding limit set (as t Ñ `8) is then of the form tz 8 " pṽ 8 , x8 q P R d `ˆR d : ∇F px 8 q " 0 , ṽ8 " p 8 Spx 8 q{q 8 u.

Similarly, if we set p " q " 0 in (ODE-1) and we keep what remains in Assumption 3.2.4, the function hptqpF pxq ´F‹ q `1 2 }m} 2 works as a Lyapunov function, and the limit set has the form tp0, xq : ∇F pxq " 0u.

Proof of Theorem 3.2.2

The existence and the uniqueness of the solution to (ODE-N) have been shown in the literature. We refer to [START_REF] Cabot | [END_REF], Proposition 2.1-2.2.c)] for an identical statement of this result and [Su et al. 2016b, Theorem 1, Appendix A] for a complete proof. The boundedness of the solution follows immediately from the coercivity of F together with the fact that the function t Þ Ñ F pxptqq `1 2 }mptq} 2 is nonincreasing.

Concerning the convergence statement, our proof is based on comparing the solutions of (ODE-N) to the solutions of the ODE in [Gadat et al. 2018, Equation (2.3)]. We first note that under a change of variable, a solution to (ODE-N) gives a solution to [START_REF] Gadat | [END_REF] Consider a solution pm, xq of (ODE-N) starting at pm 0 , x 0 q P R d ˆRd . As in Section 3.5.1.2, for every t 0 ą 0, on rt 0 , `8q, we have that pm, x, sq is a solution to the autonomous ODE starting at pm 0 , x 0 , 1{t 0 q. Denote by Φ N " pΦ m N , Φ x N , Φ s N q the semiflow induced by ODE (3.23) and ω N ppm 0 , x 0 , 1{t 0 qq its limit set.

Define py, uq as in Lemma 3.5.3. Starting at pypt 0 q, upt 0 q, 1{t 0 q, we also have that py, u, sq is a solution on rt 0 , `8q to the "autonomized" Heavy-Ball ODE Denote by Φ H " pΦ y H , Φ u H , Φ s H q the semiflow induced by ODE (3.24) and ω H ppypt 0 q, upt 0 q, 1{t 0 qq its limit set. Lemma 3.5.4. For any compact set K Ă R 2d`1 and any T ą 0, the family of functions Φpz, ¨q : r0, T s Ñ R 2d`1 ( zPK , where Φ is either Φ H or Φ N , is relatively compact in pC 0 pr0, T s, R 2d`1 q, ¨ 8 q.

Proof. The map Φ : R 2d`1 ˆR`Ñ R 2d`1 is continuous, hence uniformly continuous on K ˆr0, T s. The result follows from the application of the Arzelà-Ascoli theorem to the family Φpz, ¨q : r0, T s Ñ R 2d`1 ( zPK .

Let pm, x, 0q P ω N ppm 0 , x 0 , 1{t 0 qq. There exists a sequence pt k q of nonnegative reals such that pm, x, 0q " lim kÑ8 pmpt k q, xpt k q, 1{t k q. For any T ą 0 , using Lemma 3.5.4, up to an extraction, we can say that the sequence of functions tΦ N ppmpt k q, xpt k q, 1{t k q, ¨qu k converges towards p m, x, 0q in C 0 pr0, T s, R d q, where p m, xq is a solution to # 9 mptq " ∇F pxptqq 9 xptq " ´mptq , (3.25) with p mp0q, xp0qq " pm, xq. Moreover, by Lemma 3.5.3, we also have that:

sup hPr0,T 2 {κ 2 s xpκ ? hq ´Φx N ppmpt k q, xpt k q, 1{t k q, κ ? hq " sup hPr0,T 2 {κ 2 s xpκ ? hq ´Φu H ppmpt k q, xpt k q, 1{t k q, hq ÝÝÝÝÑ kÑ`8 0 . (3.26)
Using Lemma 3.5.4, up to an additional extraction, we get on C 0 pr0, T 2 {κ 2 s, R 2d`1 q that tΦ H ppxpt k q, mpt k q, 1{t k q, ¨qu k converges to pu, y, 0q, where pu, yq is a solution to # 9 yptq " 0 9 uptq " ´yptq .

(3.27) Therefore, uptq " A `Bt for some A and B in R d . Imagine that B ‰ 0. We previously proved that x (and therefore u) is bounded by some constant C ą 0. Let T 1 ą C`}A} }B} . Up to an extraction, we obtain that tΦ H ppxpt k q, mpt k q, 1{t k q, ¨qu k converges to u 1 on C 0 pr0, T 1 s, R 2d`1 q, with u 1 ptq " A 1 `B1 t for some A 1 and B 1 in R d . We then have by uniqueness of the limit that A 1 " A and B 1 " B. As a consequence, u 1 pT 1 q " A `BT 1 ą C and we obtain a contradiction. Hence B " 0.

This implies that u is constant. Then, if we go back to Eqs. (3.26) and (3.25), we get that x is constant, hence m " 0 and then ∇F pxq " 0. In particular, this means that m " mp0q " 0 and ∇F pxq " ∇F pxp0qq " 0.

3.6 Proofs for Section 3.3

Preliminaries

We first recall some useful definitions and results. Let Ψ represent any semiflow on an arbitrary metric space pE, dq. As in the previous section, a point z P E is called an equilibrium point of the semiflow Ψ if Ψpz, tq " z for all t ě 0. We denote by Λ Ψ the set of equilibrium points of Ψ. A continuous function V : E Ñ R is called a Lyapunov function for the semiflow Ψ if VpΨpz, tqq ď Vpzq for all z P E and all t ě 0. It is called a strict Lyapunov function if, moreover, tz P E : @t ě 0, VpΨpz, tqq " Vpzqu " Λ Ψ .

If V is a strict Lyapunov function for Ψ and if z P E is a point s.t. tΨpz, tq : t ě 0u is relatively compact, then it holds that Λ Ψ ‰ ∅ and dpΨpz, tq, Λ Ψ q Ñ 0, see [Haraux 1991, Theorem 2.1.7]. A continuous function z : r0, `8q Ñ E is said to be an asymptotic pseudotrajectory (APT, [Benaïm & Hirsch 1996]) for the semiflow Ψ if lim tÑ`8 sup sPr0,T s dpzpt `sq, Ψpzptq, sqq " 0 for every T P p0, `8q .

Proof of Theorem 3.3.1

Recall that Φ is the semiflow induced by the autonomous ODE (3.17) which is an "autonomized" version of our initial (ODE-1). In the remainder of this section, the proof will be divided into two main steps : (a) we show that a certain continuoustime linearly interpolated process constructed from the iterates of our algorithm 1 is an APT of Φ; (b) we exhibit a strict Lyapunov function for a restriction to a carefully chosen compact set of a well chosen semiflow related to Φ. Then, we characterize the limit set of the APT using [Benaïm 1999, Theorem 5.7] and [Benaïm 1996, Proposition 3.2]. The sequence pz n q converges almost surely to this same limit set.

(a) APT. For every n ě 1, define zn " pv n , m n , x n´1 q (note the shift in the index of the variable x). We have the decomposition

zn`1 " zn `γn`1 gpz n , τ n q `γn`1 η n`1 `γn`1 ς n`1 ,
where g is defined in Equation (3.1), η n`1 " `pn p∇f px n , ξ n`1 q d2 ´Spx n qq, h n p∇f px n , ξ n`1 q ´∇F px n qq, 0 ˘, (3.28) is a martingale increment and where we set ς n`1 " pς v n`1 , ς m n`1 , ς x n`1 q with the components defined by:

$ ' ' & ' ' % ς v n`1 " p n pSpx n q ´Spx n´1 qq ς m n`1 " h n p∇F px n q ´∇F px n´1 qq ς x n`1 " p γn γ n`1 ´1q mn ? vn`ε .
We first prove that ς n Ñ 0 a.s. by considering the components separately. The components ς m n`1 and ς v n`1 converge a.s. to zero by using Assumptions 3.2.1, 3.2.3, together with the boundedness of the sequences pp n q and ph n q (which are both convergent). Indeed, since ∇F is locally Lipschitz continuous and the sequence pz n q is supposed to be almost surely bounded, there exists a constant C s.t. }∇F px n q ∇F px n´1 q} ď C}x n ´xn´1 } ď C ε γ n }m n }. The same inequality holds when replacing ∇F by S which is also locally Lipschitz continuous. The component ς x n`1 also converges a.s. to zero by observing that }ς x n`1 } ď |1 ´γn γ n`1 |.}m n }{ ? ε and using Assumption 3.3.2 together with the a.s. boundedness of pz n q. Now consider the martingale increment sequence pη n q, adapted to F n . Take δ ą 0. Since pz n q is a.s bounded, there is a constant C 1 ą 0 such that Ppsup x n ą C 1 q ď δ. Denoting 3.6. Proofs for Section 3.3 57 ηn ∆ " η n 1 xn ďC 1 and combining Assumptions 3.2.4 with 3.3.4-i) we can show using convexity inequalities that sup n E}η n`1 } q ă 8.

Then, we deduce from this result together with the corresponding stepsize assumption from 3.3.4-i) and [Benaïm 1999, Proposition 4.2] (see also [Métivier & Priouret 1987, Proposition 8]) the key property:

@T ą 0 , max !› › › L´1 ÿ k"n γ k`1 ηk`1 › › › : L " n `1, . . . , Jpτ n `T q ) a.s. ÝÝÝÑ nÑ8 0 (3.29)
where Jptq " maxtn ě 0 : τ n ď tu. Hence, for all T ą 0, with probability at least

1 ´δ : max !› › › L´1 ÿ k"n γ k`1 η k`1 › › › : L " n `1, . . . , Jpτ n `T q ) ÝÝÝÑ nÑ8 0 . (3.30)
Since δ can be chosen arbitrary small, Equation (3.30) remains true with probability 1. This result also holds under Assumption 3.3.4-ii) (instead of 3.3.4-i)) by applying [Benaïm 1999, Proposition 4.4]. Let z : r0, `8q Ñ Z `be the continous-time linearly interpolated process given by zptq " zn `pt ´τn q zn`1 ´z n γ n`1 p@n P N , @t P rτ n , τ n`1 qq (where τ n " ř n k"1 γ k ). Let t 0 ą 0. Define u : rt 0 , 8q Ñ Z ˆp0, 1{t 0 s by uptq "

" zptq 1{t
 , for t ě t 0 ą 0.

Using Equation (3.30) and the almost sure boundedness of the sequence pz n q along with the fact that ς n converges a.s. to zero, it follows from [Benaïm 1999, Proposition 4.1, Remark 4.5] that uptq is an APT of the already defined semiflow Φ induced by (3.17). Remark that it also holds that zptq is an APT of the semiflow Φ 8 induced by (3.20). As the trajectory of uptq is precompact, the limit set Our objective now is to prove that

Lpuq " č tět 0 uprt, 8qq
S Ă Λ Φ 8 . (3.32)
In order to establish this inclusion, we study the behavior of the restriction Φ|L of the semiflow Φ to the set L (which is well-defined since L is Φ-invariant). Remark that

Φ|L " " Φ 8 |S 0  ,
where Φ 8 is the semiflow associated to (3.20). In the second part of the proof, we establish Equation (3.32) combining item (a) we just proved with [Benaïm 1999, Theorem 5.7] and [Benaïm 1999, Proposition 6.4]. In order to use the latter proposition, we prove a useful proposition in item (b).

(b) Strict Lyapunov function and convergence. For every δ ą 0 and every z " pv, m, xq P Z `, define:

W δ pv, m, xq ∆ " E 8 pzq ´δx∇F pxq, my `δ}q 8 v ´p8 Spxq} 2 , (3.33)
where, under Assumption 3.2.4-i), the function E 8 is defined by

E 8 pzq ∆ " lim tÑ`8
Ept, zq " h 8 pF pxq ´F‹ q `1 2

› › › › › m pv `εq d 1 4 › › › › › 2 .
(3.34) Proposition 3.6.1. Let t 0 ą 0 and let Assumptions 3.2.1 to 3.2.4 and 3.3.5 hold true. Let S be the limit set defined in Equation (3.31). Let Φ 8 : S ˆrt 0 , `8q Ñ S be the restriction of the semiflow Φ 8 to S i.e., Φ 8 pz, tq " Φ 8 pz, tq for all z P S, t ě t 0 .Then, i) S is compact.

ii) Φ

8 is a well-defined semiflow on S.

iii) The set of equilibrium points of Φ 8 is equal to Λ Φ 8 X S.

iv) There exists δ ą 0 s.

ď ¨L∇F C 1 2 1 ε 1 4 `r8 C 1 2u 2 1 '› › › › › mptq pvptq `εq d 1 4 › › › › › 2 ´ˆh 8 ´r8 u 2 1 2 ˙}∇F pxptqq} 2 (3.36)
where we used the classical inequality |xa, by| ď }a} 2 {p2u 2 q `u2 }b} 2 {2 for any nonzero real u to derive the last above inequality. We now study L H . For every t ě t 0 , The second term in the righthand side coincides with ´2q 8 xp 8 Spxptqq´q 8 vptq, 9 vptqy " ´2q 8 }p 8 Spxptqq´q 8 vptq} 2 . Denote by L S the Lipschitz constant of S on the set tx : pv, m, xq P Su. Note that s ´1p}Spxpt `sqq ´Spxptqq} 2 q ď L 2 S s}s ´1pxpt `sq ´xptqq} 2 which converges to zero as s Ñ 0. Thus,

L
L H ptq " ´2q 8 }p 8 Spxptqq ´q8 vptq} 2 `lim sup sÑ0 2p 8 s ´1xSpxptqq ´Spxpt `sqq, q 8 vpt `sq ´p8 Spxptqqy ď ´2q 8 }p 8 Spxptqq ´q8 vptq} 2 `2p 8 }9 xptq}L S }q 8 vptq ´p8 Spxptqq} ď p 8 ε 1 2 u 2 2 › › › › › mptq pvptq `εq d 1 4 › › › › › 2 ´p2q 8 ´p8 u 2 2 L 2 S q}p 8 Spxptqq ´q8 vptq} 2 . (3.37)
Recalling that L W δ " L E8 `δL G `δL H and combining Eqs. (3.35), (3.36) and (3.37), we obtain for every t ě t 0 ,

L W δ ptq ď ´M pδq › › › › › mptq pvptq `εq d 1 4 › › › › › 2 ´δ ˆh8 ´r8 u 2 1 2 ˙}∇F pxptqq} 2 ´δ `2q 8 ´p8 u 2 2 L 2 S ˘}p 8 Spxptqq ´q8 vptq} 2 . (3.38) where M pδq ∆ " r 8 ´q8 4 ´δ ˜r8C1 2u 2 1 `L∇F C 1 2 1 ε 1 4 `p8 ε 1 2 u 2 2 ¸. Now select u 1 , u 2 small enough s.t. h 8 ´r8 u 2 1 {2 ą 0 and 2q 8 ´p8 u 2 2 L 2 S ą 0.
Then, choose δ in such a way that M pδq ą 0. Thus, there exists a constant c depending on δ s.t.

@t ě t 0 , L W δ ptq ď ´c ¨› › › › › mptq pvptq `εq d 1 4 › › › › › 2 `}∇F
´c ¨› › › › › mptq pvptq `εq d 1 4 › › › › › 2 `}∇F pxptqq} 2 `}p 8 Spxptqq ´q8 vptq} 2
' is equal to zero for every t ě t 0 a.e. (hence, for every t ě t 0 , by continuity of Φ 8 ). In particular for t " t 0 , m " ∇F pxq " 0 and p 8 Spxq ´q8 v " 0. Hence, z P zer g 8 X S. This concludes the proof since Λ Φ 8 " zer g 8 .

End of the Proof of Theorem 3.3.1. Finally, Assumption 3.3.5 implies that W δ pΛ Φ 8 X Sq is of empty interior. Recall that Assumptions 3.2.1 and 3.2.3 both follow from Assumption 3.3.3 made in Theorem 3.3.1. Given Proposition 3.6.1, the proof is concluded by applying [Benaïm 1999, Proposition 6.4] to the restricted semiflow Φ8 (with pM, Λq " pS, ΛΦ8q). Note that a Lyapunov function for ΛΦ8 is what is called a strict Lyapunov function. Such a function is provided by Proposition 3.6.1. We obtain as a conclusion of [Benaïm 1999, Proposition 6.4] that S Ă ΛΦ8. This gives the desired result (Equation (3.32)) given Proposition 3.6.1-iii).

The last assertion of Theorem 3.3.1 is a consequence of [Benaïm 1999, Cor. 6.6].

Proof of Theorem 3.3.3

We can rewrite the iterates from Algorithm 2 as follows:

# m n`1 " m n `γn`1 p∇F px n q ´α τn m n q `γn`1 p∇f px n , ξ n`1 q ´∇F px n qq

x n`1 " x n ´γn`1 m n`1 .

(3.40) We prove that the sequence py n " pm n , x n q : n P Nq of iterates of this algorithm converges almost surely towards the set Ῡ defined in Equation (3.3) if it is supposed to be bounded with probability one. The proof follows a similar path to the proof in Section 3.5.2. Indeed, denote by X and M the linearly interpolated processes constructed respectively from the sequences px n q and pm n q and let sptq " 1{t. Recall that Φ N " pΦ m N , Φ x N , Φ s N q is the semiflow induced by (3.23). As in Section 3.6.2, we have that Z ∆ " pM, X, sq is an APT of (3.23). In particular, this means that @T ą 0 , sup Let pm, xq be a limit point of the sequence py n q and let T ą 0. Using Lemma 3.5.4, we can proceed in the same manner as in Section 3.5.2 and get a sequence pt k q such that pMpt k `¨q, Xpt k `¨qq Ñ pm, xq and pΦ y H pZpt k q, ¨q, Φ u H pZpt k q, ¨qq Ñ py, uq , where pmp0q, xp0qq " pm, xq , and pm, xq and px, uq are respectively solutions to (3.25) and (3.27). As in the end of Section 3.5.2, we obtain that u and x are constant, therefore m " 0 and ∇F pxq " 0 , which finishes the proof.

Proof of Theorem 3.3.2

The idea of the proof is to apply Robbins-Siegmund's theorem [Robbins & Siegmund 1971] to

V n " h n´1 F px n q `1 2 xm d2 n , 1 ? v n `ε y
(note the similarity of V n with the energy function (3.15)). Since inf F ą ´8, we assume without loss of generality that F ě 0. In this subsection, we use the notation ∇f n`1 as a shorthand notation for ∇f px n , ξ n`1 q and C denotes some positive constant which may change from line to line. We write E n " Er¨| F n s for the conditional expectation w.r.t the σ-algebra

F n . Define P n ∆ " 1 2 xD n , m d2 n y, with D n ∆ " 1 ? vn`ε .
We have the decomposition:

P n`1 ´Pn " 1 2 xD n`1 ´Dn , m d2 n`1 y `1 2 xD n , m d2 n`1 ´md2 n y. (3.43)
We estimate the vector

D n`1 ´Dn " ? v n `ε ´?v n`1 `ε ? v n`1 `ε d ? v n `ε .
Remarking that v n`1 ě p1 ´γn`1 q n qv n and using the update rule of v n , we obtain for a sufficiently large n that

? v n `ε ´?v n`1 `ε " γ n`1 q n v n ´pn ∇f d2 n`1 ? v n `ε `?v n`1 `ε ď γ n`1 q n v n p1 `?1 ´γn`1 q n q ? v n `ε " γ n`1 q n 1 `?1 ´γn`1 q n ? v n d ? v n ? v n `ε ď c n`1 ? v n`1 where c n`1 ∆ " γ n`1 q n ?
1 ´γn`1 q n p1 `?1 ´γn`1 q n q .

(3.44)

It is easy to see that c n`1 {γ n Ñ q 8 {2. Thus, for any δ ą 0, c n`1 ď pq 8 `2δqγ n {2 for all n large enough. Using also that

? v n`1 { ? v n`1 `ε ď 1, we obtain D n`1 ´Dn ď q 8 `2δ 2 γ n D n . (3.45)
Substituting the above inequality in Equation (3.43), we obtain

P n`1 ´Pn ď ˆq8 `2δ 2 ˙γn 2 xD n , m d2 n`1 y `1 2 xD n , m d2 n`1 ´md2 n y ď q 8 `2δ 2 γ n P n `ˆ1 `q8 `2δ 2 γ n ˙1 2 xD n , m d2 n`1 ´md2 n y .
Using m d2 n`1 ´md2 n " 2m n d pm n`1 ´mn q `pm n`1 ´mn q d2 , and noting that E n pm n`1 ´mn q " γ n`1 h n ∇F px n q ´γn`1 r n m n ,

E n 1 2 xD n , m d2 n`1 ´md2 n y " γ n`1 h n x∇F px n q, m n ? v n `ε y ´2γ n`1 r n P n `1 2 xD n , E n rpm n`1 ´mn q d2 sy .
There exists δ ą 0 such that r 8 ´q8 4 ´δ 2 ą 0 by Assumption 3.2.4-iv). As γ n`1 γn r n q8

4 Ñ r 8 ´q8 4 , for all n large enough, γ n`1 γn r n ´q8 4 ą r 8 ´q8 4 ´δ 2 ą 0. Hence, for all n large enough,

E n P n`1 ´Pn ď ´2 ˆr8 ´q8 4 ´δ 2 ˙γn P n `γn`1 h n x∇F px n q, m n ? v n `ε y `Cγ 2 n x∇F px n q, m n ? v n `ε y `CxD n , E n rpm n`1 ´mn q d2 sy . (3.46)
Using the inequality xu, vy ď p}u} 2 `}v} 2 q{2 and Assumption 3.3.6-ii), it is easy to show the inequality x∇F px n q, mn ? vn`ε y ď Cp1 `F px n q `Pn q. Moreover, using the componentwise inequality ph n ∇f n`1 ´rn m n q d2 ď 2h 2 n ∇f d2 n`1 `2r 2 n m d2 n along with Assumption 3.3.6-ii) and the boundedness of the sequences ph n q, pr n q and pγ n`1 {γ n q, we obtain xD n , E n rpm n`1 ´mn q d2 sy ď Cγ 2 n p1 `F px n q `Pn q .

(3.47)

Combining Equation (3.46) and Equation (3.47), we get

E n pP n`1 ´Pn q ď γ n`1 h n x∇F px n q, m n d D n y `Cγ 2 n p1 `F px n q `Pn q . (3.48)
Denoting by M the Lipschitz coefficient of ∇F , we also have

F px n`1 q ď F px n q ´γn`1 x∇F px n q, m n`1 d D n`1 y `γ2 n`1 M 2 }m n`1 d D n`1 } 2 .
(3.49)

Using (3.45) and the update rule of m n , we have

m n`1 d D n`1 ´mn d D n 2 ď C pm n`1 ´mn q d D n 2 `C m n`1 d pD n`1 ´Dn q 2 ď Cγ 2 n`1 p ∇f n`1 2 ` m n d D n 2 q `Cγ 2 n`1 m n`1 d D n 2 ď Cγ 2 n`1 p m n d D n 2 ` ∇f n`1 2 q .
(3.50)

Finally, recalling that V n " h n´1 F px n q `Pn , ph n q is decreasing, combining Equation (3.48),(3.49),(3.50), and using Assumption 3.3.6, we have

E n rV n`1 s ď V n `γn`1 h n x∇F px n q, E n rm n d D n ´mn`1 d D n`1 sy `Cγ 2 n`1 ´1 `F px n q `Pn ` m n d D n 2 Cγ 2 n`1 E n r m n d D n ´mn`1 d D n`1 2 s ď V n `Cγ 2 n ´1 `F px n q `Pn ` m n d D n 2 `En " ∇f n`1 2 ıď V n `Cγ 2 n p1 `F px n q `Pn q ď p1 `Cγ 2 n qV n `Cγ 2 n ,
where we used Cauchy-Schwarz's inequality and the fact that

m n d D n 2 ď CP n .
By the Robbins-Siegmund's theorem [Robbins & Siegmund 1971], the sequence pV n q converges almost surely to a finite random variable V 8 P R `. Then, the coercivity of F implies that px n q is almost surely bounded. We now establish the almost sure boundedness of pm n q. Assume in the sequel that n is large enough to have p1 ´γn`1 r n q ě 0. Consider the martingale difference sequence ∆ n`1 ∆ " ∇f n`1 ´∇F px n q. We decompose m n " mn `m n where mn`1 " p1 ´γn`1 r n q mn `γn`1 h n ∇F px n q and mn`1 " p1 ´γn`1 r n q mn `γn`1 h n ∆ n`1 , setting m0 " 0 and m0 " m 0 . We prove that both terms mn and mn are bounded. Consider the first term: } mn`1 } ď p1´γ n`1 r n q} mn }`γ n`1 sup k }h k ∇F px k q} , where the supremum in the above inequality is almost surely finite by continuity of ∇F . We immediately get that if mn ě sup k }h k ∇F px k q} r8 , then mn`1 ď } mn }. Thus

mn`1 ď sup k }h k ∇F px k q} r 8 `sup k γ k`1 }h k ∇F px k q} ,
which implies that mn is bounded.

Chapter 3. Stochastic optimization with momentum

Consider now the term mn :

E n r} mn`1 } 2 s " p1 ´γn`1 r n q 2 } mn } 2 `γ2 n`1 h 2 n E n r}∆ n`1 } 2 s ď } mn } 2 `γ2 n`1 h 2 n E n r}∆ n`1 } 2 s .
Then, the inequality E n r}∆ n`1 } 2 s ď E n r}∇f n`1 } 2 s combined with Assumption 3.3.4i) and the a.s. boundedness of the sequence px n q imply that there exists a finite random variable

C K (independent of n) s.t. E n r}∇f n`1 } 2 s ď C K . As a consequence, since ř n γ 2
n`1 ă 8 and the sequence ph n q is bounded, we obtain that a.s.:

ÿ ně0 γ 2 n`1 h 2 n E n r}∆ n`1 } 2 s ď CC K ÿ ně0 γ 2 n`1 ă `8 .
Hence, we can apply the Robbins-Siegmund theorem to obtain that sup n } mn } 2 ă 8 w.p.1. Finally, it can be shown that pv n q is almost surely bounded using the same arguments, decomposing v n into vn `ṽ n as above. Indeed, first, we have:

E n r}ṽ n`1 } 2 s ď }ṽ n } 2 `γ2 n`1 p 2 n E n r}∇f d2 n`1 ´Spx n q} 2 s .
Second, it also holds that:

E n r}∇f d2 n`1 ´Spx n q} 2 s ď E n r}∇f d2 n`1 } 2 s ď E n r}∇f n`1 } 4 s .
Then, using Assumption 3.3.4-i) and the a.s. boundedness of the sequence px n q, there exists a finite random variable

C 1 K (independent of n) s.t. E n r}∇f n`1 } 4 s ď C 1 K .
Moreover, the sequence pp n q is bounded and ř n γ 2 n`1 ă 8. As a consequence, it holds that a.s:

ÿ ně0 γ 2 n`1 p 2 n E n r}∇f d2 n`1 ´Spx n q} 2 s ď CC 1 K ÿ ně0 γ 2 n`1 ă `8 .
It follows that the Robbins-Siegmund theorem can be applied to the sequence }ṽ n } 2 as for the sequence } mn } 2 to obtain that sup n }ṽ n } 2 ă 8 w.p.1.

Proof of Theorem 3.3.4

The proof of Theorem 3.3.2 easily adapts to Algorithm 2 by replacing V n by

Ṽn ∆ " F px n q `1 2 m n 2 .
The boundedness of pm n q is an immediate consequence of the convergence of Ṽn .

3.6.6 Proof of Theorem 3.3.5

We shall use the following result.

Theorem 3.6.2 (adapted from [Pelletier 1998], Theorem 7). Let k ě 1. On some probability space equipped with a filtration F " pF n q nPN , consider a sequence of r.v. on R k given by

Z n`1 " pI `γn`1 HqZ n `γn`1 b n`1 `?γ n`1 η n`1
and Er}Z 0 } 2 s ă 8, where H is a k ˆk Hurwitz matrix, pb n q and pη n q are random sequences, and γ n " γ 0 n ´α for some γ 0 ą 0 and α P p0, 1s. Let Ω 0 P F 8 have a positive probability. Assume that the following holds almost surely on Ω 0 :

i) Erη n`1 |F n s " 0.
ii) There exists a constant b ą 2 s.t. sup ně0 Er}η n`1 } b|F n s ă 8.

iii) Erη n`1 η T n`1 |F n s " Σ `∆n where Er}∆ n }1 Ω 0 s Ñ 0 and Σ is a positive semidefinite matrix.

iv) The sequence pb n q is the sum of two sequences pb n,1 q and pb n,2 q, adapted to F , s.t.

sup ně0 Er}b n,1 } 2 s ă 8, Er}b n,1 }1 Ω 0 s Ñ 0 and b n,2 Ñ 0 a.s. on Ω 0 .
Then, given Ω 0 , pZ n q converges in distribution to the unique stationary distribution µ ‹ of the generalized Ornstein-Uhlenbeck process

dX t " HX t dt `?ΣdB t
where pB t q is the standard Brownian motion and ? Σ is the unique positive semidefinite square root of Σ. The distribution µ ‹ is the zero mean Gaussian distribution with covariance matrix Γ given as the solution to p H `1α"1

2γ 0 I k qΓ`Γp H `1α"1 2γ 0 I k q T " ´Σ. Proof.
The proof is identical to the proof of [Pelletier 1998, Theorem 7], only substituting the inverse of the square root of Σ by the Moore-Penrose inverse. Finally, the uniqueness of the stationary distribution µ ‹ and its expression follow from [Karatzas & Shreve 1991, Theorem 6.7, p. 357] We define v n " vn `δn where δ 0 " 0, v0 " v 0 and δ n`1 " p1 ´γn`1 q n qδ n `γn`1 pp n ´qn q ´1 8 p 8 qSpx n q vn`1 " p1 ´γn`1 q n qv n `γn`1 q n q ´1 8 p 8 Spx n q `γn`1 p n p∇f px n , ξ n`1 q d2 ´Spx n qq .

For every z " pv, m, xq P Z `and δ ě 0, we define

r n pz, δq ∆ " » - - q n q ´1 8 p 8 pSpx ´γn m ? v`δ`ε q ´Spxqq h n p∇F px ´γn m ? v`δ`ε q ´∇F pxqq γn γ n`1 p 1 ? v`ε ´1 ? v`δ`ε q d m fi ffi fl .
Moreover, for every z " pv, m, xq P Z `and every n P N, we set

g n pzq " » - - q n q ´1 8 p 8 Spxq ´qn v h n ∇F pxq ´rn m ´γn γ n`1 m ? v`ε fi ffi fl .
Defining ζ n " pv n , m n , x n´1 q and recalling the definition of pη n q from Equation (3.28), we have the decomposition

ζ n`1 " ζ n `γn`1 g n pζ n q `γn`1 η n`1 `γn`1 r n pζ n , δ n q . Define z ‹ ∆ " px ‹ , 0, v ‹ q.
Note that g n pz ‹ q " 0. Evaluating the Jacobian matrix G n of g n at z ‹ , we obtain that there exist constants C ą 0, M ą 0 and n 0 P N s.t. for all n ě n 0 , }g n pzq ´Gn pz ´z‹ q} ď C}z ´z‹ } 2 p@z P Bpz ‹ , M qq , (3.51)

where G n is given by

G n ∆ " » - - ´qn I d 0 q n q ´1 8 p 8 ∇Spx ‹ q 0 ´rn I d h n ∇ 2 F px ‹ q 0 ´γn γ n`1 V 0 fi ffi fl ,
where ∇S is the Jacobian of S and the matrix V is defined in Equation (3.8). We define

G 8 ∆ " lim n G n " » - ´q8 I d 0 p 8 ∇Spx ‹ q 0 ´r8 I d h 8 ∇ 2 F px ‹ q 0 ´V 0 fi fl .
One can verify that G 8 is Hurwitz, and that the largest real part of its eigenvalues is ´L1 , where L 1 ∆ " L ^q8 and L is defined in Equation (3.9). We define Ω p0q ∆ " tz n Ñ z ‹ u. We assume PpΩ p0q q ą 0. Using for instance [START_REF] Delyon | [END_REF], Lemma 4 and Lemma 5], it holds that δ n pωq Ñ 0 for every ω P Ω p0q , and since x n pωq ´xn´1 pωq Ñ 0 on that set, we obtain that Ω p0q " tζ n Ñ z ‹ u. Let M P p0, M q be a constant, whose value will be specified later on. For every

N 0 P N, define Ω p0q N 0 ∆
" tζ n Ñ z ‹ and sup něN 0 }ζ n ´z‹ } ď M u. We seek to show that ? γ n ´1pζ n ´z‹ q ñ ν given Ω p0q , for some Gaussian measure ν, using Theorem 3.6.2.

As Ω p0q N 0 Ò Ω p0q , it is sufficient to show that the latter convergence holds given Ω p0q N 0 , for every N 0 large enough. From now on, we consider that N 0 is fixed. We define the sequence p ζn q něN 0 as ζN 0 " ζ N 0 and for every n ě N 0 , ζn`1 " ζn `γn`1 gn p ζn q `γn`1 pη n`1 `rn p ζn , δ n qq1 An where A n is the event defined by

A n ∆ " n č k"N 0 t}x k ´x‹ } ď M u X t} ζn ´z‹ } ď M u and gn pzq ∆ " g n pzq1 }z´z‹}ďM ´Kpz ´z‹ q1 }z´z‹}ąM ,
where K ą 0 is a large constant which will be specified later on. The sequences p ζn q něN 0 and pζ n q něN 0 coincide on Ω p0q N 0 . Thus, it is sufficient to study the weak convergence of p ζn q něN 0 .

An estimate of }r n p ζn , δ n q}1 An . We start by studying the sequence p}δ n }1 An q. Unfolding the update rule defining δ n and using the fact that pq n q is a sequence of positive reals converging to q 8 ą 0, we obtain that

}δ n }1 An ď n ÿ k"1 » - n ź j"k`1 |1 ´γj q j´1 | fi fl γ k |p k´1 ´qk´1 q ´1 8 p 8 |}Spx k´1 q}1 An ď C n ÿ k"1 exp ¨´β n ÿ j"k`1 γ j 'γ k |p k´1 ´qk´1 q ´1 8 p 8 | ∆ " w n ,
for some β ą 0. The sequence pw n q is deterministic and converges to zero by [START_REF] Delyon | [END_REF] Tightness of ? γ n ´1p ζn ´z‹ q. We decompose ζn`1 ´z‹ " pI 3d `γn`1 G n qp ζn ´z‹ q `γn`1 ´gn p ζn q ´Gn p ζn ´z‹ q ¯1} ζn´z‹}ďM ´γn`1 pK `Gn qp ζn ´z‹ q1 } ζn´z‹}ąM `γn`1 pη n`1 `rn p ζn , δ n qq1 An . (3.53)

For a given t ą 0, we write G 8 " B ´1 t G t B t the Jordan-like decomposition of G 8 , where the ones of the second diagonal of the usual Jordan decomposition are replaced by t, and where B t is some invertible matrix. We define S n ∆ " B t p ζn ´z‹ q. Setting G

ptq n ∆ " B t G n B ´1
t , we obtain S n`1 " pI 3d `γn`1 G ptq n qS n `γn`1 B t ´gn p ζn q ´Gn p ζn ´z‹ q ¯1} ζn´z‹}ďM ´γn`1 pK `Gptq n qS n 1 } ζn´z‹}ąM `γn`1 B t pη n`1 `rn p ζn , δ n qq1 An .

Choose A P p0, 2L 1 q and A 1 P pA, 2L 1 q. There exists γ and t ą 0 s.t. for every γ ă γ, }I `γG t } 2 ď 1 ´γpA 1 `2L 1 q{2, where } ¨}2 is the spectral norm. As G ptq n Ñ G t , there exists n 2 ě n 1 , such that for all n ě n 2 , }I `γG ptq n } 2 ď 1 ´γA 1 . Recall the notation E n " Er¨| F n s. We expand }S n`1 } 2 and use the inequality › › ›g n p ζn q ´Gn p ζn ´z‹ q

› › › 2 1 } ζn´z‹}ďM ď C}S n } 2 to obtain after straightforward alge- bra E n }S n`1 } 2 ď p1 ´γn`1 A 1 q}S n } 2 `Cγ 2 n`1 }S n } 2 `Cγ 2 n`1 pE n }η n`1 } 2 `
E n }S n`1 } 2 ď p1 ´γn`1 pA 1 ´cqq}S n } 2 `Cγ 2 n`1 }S n } 2 `Cγ 2 n`1 pE n }η n`1 } 2 `}r n p ζn , δ n q} 2 q1 An `2γ n`1 }B t }}S n }}r n p ζn , δ n q}1 An .
Using Equation (3.52),

E n }S n`1 } 2 ď p1 ´γn`1 pA 1 ´c ´wn qq}S n } 2 `Cγ 2 n`1 p1 `w2 n q}S n } 2 `Cγ 2 n`1 E n }η n`1 } 2 1 An .
Therefore, there exists n 3 ě n 2 s.t. for all n ě n 3 ,

E}S n`1 } 2 ď p1 ´γn`1 AqE}S n } 2 `Cγ 2 n`1 Ep}η n`1 } 2 1 }xn´x‹}ďM q .
The second expectation in the righthand side is bounded uniformly in n by the condition (3.7). Using [START_REF] Delyon | [END_REF], Lemma 4 and Lemma 5], we conclude that sup n γ ´1 n E}S n } 2 ă 8. Therefore, sup n γ ´1 n E} ζn ´z‹ } 2 ă 8, which in turn implies sup n γ ´1 n Ep}ζ n ´z‹ } 2 1 Ω p0q N 0 q ă 8.

Strongly perturbed iterations. We define ỹn " ? γ n ´1p ζn ´z‹ q. Define

Ḡn ∆ " γ ´1 n`1 ˆc γ n γ n`1 ´1˙I 3d `c γ n γ n`1 G n .
The sequence Ḡn converges to Ḡ8

∆ " G 8 `1α"1 2γ 0 I 3d . Recalling Equation (3.53), we can write ỹn`1 " pI 3d `γn`1 Ḡ8 qỹ n `γn`1 rn `?γ n`1 ηn`1

where ηn`1 " η n`1 1 An and rn " rn,1 `r n,2 `r n,3 , where rn,1

∆ " ? γ n`1 ´1r n p ζn , δ n q1 An `p Ḡn ´Ḡ 8 qỹ n rn,2 ∆ " ? γ n`1
´1 ´gn p ζn q ´Gn p ζn ´z‹ q ¯1} ζn´z‹}ďM rn,3

∆ " ´?γ n`1 ´1pK `Gn qp ζn ´z‹ q1 } ζn´z‹}ąM .

We now check that the assumptions of Theorem 3.6.2 are fulfilled. On the event Ω

p0q N 0 , we recall that ζn " ζ n , hence rn,3 is identically zero. Moreover, using Equation (3.52), it holds that for all n large enough,

}r n,1 } ď C ˆc γ n γ n`1 pγ n`1 `wn q `} Ḡn ´Ḡ 8 } ˙}ỹ n }
and therefore, Er}r n,1 } 2 s Ñ 0. Now consider the term rn,2 . By Equation (3.51),

}r n,2 } ď C ? γ n`1 ´1} ζn ´z‹ } 2 1 } ζn´z‹}ďM .
Thus, }r n,2 } 2 ď C}ỹ n } 2 which implies that sup něN 0 Er}r n,2 } 2 s ă 8. Moreover,

Er}r n,2 }s ď C ? γ n`1 E}ỹ n } 2 tends to zero. Finally, consider ηn`1 . Using condition (3.7), there exist M ą 0 and b M ą 4 s.t.

E n r}η n`1 } b M {2 s ď E n r}η n`1 } b M {2 s1 }xn´x‹}ďM ď CE n r}∇f px n , ξ n`1 q} b M s1 }xn´x‹}ďM ď C .
Moreover, E n rη n`1 s " 0 and finally, almost surely on

Ω p0q N , E n rη n`1 ηT n`1 s converges to Σ ∆ " » - - E ξ « " p 8 p∇f px ‹ , ξq d2 ´Spx ‹ qq h 8 ∇f px ‹ , ξq  " p 8 p∇f px ‹ , ξq d2 ´Spx ‹ qq h 8 ∇f px ‹ , ξq  T ff 0 0 0 0 0 fi ffi fl .
(3.54) Therefore, the assumptions of Theorem 3.6.2 are fulfilled for the sequence ỹn . We obtain the desired result for the sequence pm n , x n´1 q. We now show that the same result also holds for the sequence pm n , x n q. For this purpose, observe that

1 ? γ n " m n x n ´x‹  " 1 ? γ n " m n x n´1 ´x‹  `« 0 1 ? γn px n ´xn´1 q ff .
Then, notice that } xn´x n´1 ? γn } " ? γ n } mn ? vn`ε } ď b γn ε }m n } Ñ 0 as n Ñ 8 since it is assumed that z n Ñ z ‹ (which implies in particular that m n Ñ 0). Hence, it holds that ? γ n ´1px n ´xn´1 q converges a.s. to 0. We conclude by invoking Slutsky's lemma.

Proof of Equation (3.10). We have the subsystem: ´1 " ı (inspecting the characteristic polynomial of H, these are the eigenvalues of H). Set M ˘∆ " diag pν 1 , ¨¨¨, ν d q and

HΓ `Γ HT " " ´h2 8 Q 0 0 0  where H ∆ " " pθ ´r8 qI d h 8 ∇ 2 F px ‹ q ´V θI d  ( 3
R ˘∆ " V ´1 2 P M ˘P T V ´1 2 . Using the identities M ``M ´" ´r8 I d and M `M ´" h 8 diag pπ 1 , ¨¨¨, π d q, it can be checked that R H " " R ´V `θI d 0 ´V V R ``θI d  R, where R ∆ " " I d R 0 I d  .
Set Γ ∆ " RΓR T . Denote by p Γi,j q i,j"1,2 the blocks of Γ. Note that Γ2,2 " Γ 2,2 . By left/right multiplication of Equation (3.55) respectively by R and R T , we obtain

pR ´V `θI d q Γ1,1 `Γ 1,1 pV R ´`θI d q " ´h2 8 Q (3.56) pR ´V `θI d q Γ1,2 `Γ 1,2 pR `V `θI d q " Γ1,1 V (3.57) pV R ``θI d q Γ2,2 `Γ 2,2 pR `V `θI d q " V Γ1,2 `Γ T 1,2 V . (3.58) Set Γ1,1 " P ´1V 1 2 Γ1,1 V 1 2 P . Define C ∆ " P ´1V 1 2 QV 1 2 P . Equation (3.56) yields pM ´`θI d q Γ1,1 `Γ 1,1 pM ´`θI d q " ´h2 8 C . Set Γ1,2 " P ´1V 1 2 Γ1,2 V ´1 2 P . Equation (3.57) is rewritten pM ´`θI d q Γ1,2 Γ1,2 pM ``θI d q " Γ1,1 . The component pk, q is given by Γk, 1,2 " pν ḱ `ν` `2θq ´1 Γk, 1,1 " ´h2 8 C k, pν ḱ `ν` `2θqpν ḱ `ν´ `2θq . Set finally Γ2,2 " P ´1V ´1 2 Γ 2,2 V ´1 2 P . Equation (3.58) becomes pM ``θI d q Γ2,2 `Γ 2,2 pM ``θI d q " Γ1,2 `Γ T 1,2 .
Thus,

Γk, 2,2 " Γk, 1,2 `Γ ,k 1,2 ν k `ν` `2θ " ´h2 8 C k, pν k `ν` `2θqpν ḱ `ν´ `2θq ˆ1 ν ḱ `ν` `2θ `1 ν k `ν´ `2θ ˙.
After tedious but straightforward computations, we obtain Γk, 2,2 "

h 2 8 C k, pr 8 ´2θqph 8 pπ k `π q `2θpθ ´r8 qq `h2 8 pπ k ´π q 2 2pr8´2θq
, and the result is proved.

3.7 Proofs for Section 3.4

Preliminaries

Most of the avoidance of traps results in the stochastic approximation literature deal with the case where the ODE that underlies the stochastic algorithm under study is an autonomous ODE 9 z " hpzq. In this setting, a point z ‹ P zer h is called a trap if hpzq admits an expansion around z ‹ of the type hpzq " Dpz ź‹ q `op}z ´z‹ }q, where the matrix D has at least one eigenvalue which real part is (strictly) positive. Initiated by Pemantle [Pemantle 1990] and by Brandière and Duflo [Brandière & Duflo 1996], the most powerful class of techniques for establishing avoidance of traps results makes use of Poincaré's invariant manifold theorem for the ODE 9

z " hpzq in a neighborhood of some point z ‹ P zer h. The idea is to show that with probability 1, the stochastic algorithm strays away from the maximal invariant manifold of the ODE where the convergence to z ‹ of the ODE flow can take place. As previously mentioned, since we are dealing with algorithms derived from non-autonomous ODEs, we extend the results of [Pemantle 1990, Brandière & Duflo 1996]

z Þ Ñ y " " y ý ` " Q ´1z,
where y ˘P R d ˘. In this new basis, the ODE (3.59) is written as

" 9 y 9 y ` " " Λ ´Λ` " y ý ` , (3.60)
which solution is y ˘ptq " expptΛ ˘qy ˘p0q. One can readily check that for each couple of real numbers α `and α ´that satisfy

0 ă α ´ă α `ă mint λ i : λ i ą 0u, (3.61)
there exists a so-called exponential dichotomy of the ODE solutions, which amounts in our case to the existence of two constants K ´, K `ě 1 such that } expptΛ ´q} ď K ´eα ´t for t ě 0, } expptΛ `q} ď K `eα `t for t ď 0, see, e.g., [Horn & Johnson 1994].

We now consider a non-autonomous perturbation of this ODE, which is represented in the basis of the columns of Q as 9 yptq " hpyptq, tq with hpy, tq " " Λ ´Λ` y `εpy, tq, (3.62) and ε : R d ˆR Ñ R d is a continuous function. In the sequel, we shall be interested in the asymptotic behavior of this equation for the large values of t, and therefore, restrict our study to the interval I " rt 0 , 8q for some given t 0 ě 0 that we shall fix later. We assume that εp0, ¨q " 0 on I. We denote as φ : I ˆI ˆRd Ñ R d the so-called general solution of (3.62), which is defined by the fact that φp¨, t, xq is the unique noncontinuable solution of (3.62) such that φpt, t, xq " x for t P I and x P R d , assuming this solution exists and is unique for each px, tq P R d ˆI.

In the linear autonomous case provided by the ODE (3.60), the subspace

G " "ˆt , " y 0 ˙P R ˆRd : y ´P R d
´*

is trivially invariant in the sense that if pt, yq P G, then, ps, φps, t, yqq P G for each s P R. This concept can be generalized to the non-linear and non-autonomous case. We say that the C 1 function w : R

y ´P R d ´* ,
which obviously satisfies pt, yq P G ñ ps, φps, t, yqq P G for each s P I. These invariant manifolds are described by the following proposition, which is a straightforward application of [Pötzsche & Rasmussen 2006, Theorem A.1] (see also [Kloeden & Rasmussen 2011, Theorem 6.3 p. 106, Rem. 6.6 p. 111]). It is useful to note that under the conditions provided in the statement of this proposition, the existence of the general solution φ of the ODE (3.62) is ensured by Picard's theorem.

Proposition 3.7.1. Let I " rt 0 , 8q for some t 0 ě 0. Assume that the function εpy, tq is such that εp0, ¨q " 0 on I, the function εp¨, tq is continuously differentiable for each t P I, and furthermore, the Jacobian matrix B 1 εpy, tq satisfies

|ε| 1 ∆ " sup py,tqPR d ˆI }B 1 εpy, tq} ă α `´α 4K (3.63)
with K " K ´`K ``K ´K`p K ´_ K `q and α ´, α `chosen as in Equation (3.61).

Then, for each δ P p2K|ε| 1 , pα `´α ´q{2q and each γ P pα ´`δ, α `´δq, the set

" Λ ´y´`ε´p y, tq Λ `y``ε`p y, tq  , (3.67)
where

h ˘: R d ˆI Ñ R d ˘, y ˘P R d ˘and ε ˘: R d ˆI Ñ R d ˘.
With these notations, the previous proposition leads to the following lemma.

Lemma 3.7.2. In the setting of Proposition 3.7.1, for each t in the interior of I and each vector y " py ´, y `q such that y ˘P R d ˘and y `" wpy ´, tq, it holds that h `py, tq " B 1 wpy ´, tqh ´py, tq `B2 wpy ´, tq .

(3.68)

Assume that α ´is small enough so that Inequality (3.65) and Equation (3.64) hold true with m " 2. Assume in addition that B n 2 B k 1 ε exists and is continuous for 0 ď n ă 2 and 0 ď k `n ď 2, and furthermore, that there exists a bounded neighborhood V Ă R d of zero such that sup py,tqPVˆI B 2 εpy, tq ă `8.

(3.69)

Then, there exists a neighborhood V ´Ă R Proposition 3.7.1 deals with the case where the function ε is globally Lipschitz continuous. In practical cases, such a strong assumption is not necessarily verified. In particular, for the ODEs we consider for our application, it is not satisfied (see the function e defined in Subsec. 3.7.3.1 below). Nonetheless, recall that we only need the existence of a local non-autonomous invariant manifold, i.e. defined in the vicinity of an arbitrary solution such as the trivial zero solution (since we suppose here εp0, ¨q " 0) whereas the aforementioned strong assumption provides a global non-autonomous invariant manifold. Indeed, as for the avoidance of traps result we intend to show, we will only need to look at the behavior of our ODE in the neighborhood of a trap z ‹ . Therefore, in prevision of the proof of Theorem 3.4.1, we localize the ODE (3.62) in the neighborhood of zero. This is the purpose of the next proposition.

Proposition 3.7.3. Let I " rt 0 , `8q for some t 0 ě 0 and let h : R d ˆI Ñ R d be defined as in Equation (3.62). Assume that εp0, ¨q " 0 on I, that the function εp¨, tq is continuously differentiable for every t P I and that lim py,tqÑp0,`8q B 1 εpy, tq " 0 .

(3.78)

Then, there exist σ ą 0, t 1 ą 0, a function ε : R d ˆI1 Ñ R d where I 1 ∆ " rt 1 , `8q and a function h : R d ˆI1 Ñ R d defined for every y P R d , t P I 1 by hpy, tq " Λy `εpy, tq s.t. h and ε verify the assumptions of Proposition 3.7.1 and for every py, tq P Bp0, σqˆI 1 , we have that hpy, tq " hpy, tq and εpy, tq " εpy, tq. Moreover, for any δ ą 0, we can choose σ, t 1 respectively small and large enough s. , where we used the mean value inequality along with εp0, tq " 0 to obtain the second inequality. Thus, the function hpy, tq " Λy `εpy, tq satisfies all the assumptions of Proposition 3.7.1. In addition, the function ε coincides with the function ε on Bp0, σ 1 q ˆI1 , and so it is for the functions h and h. Finally, it follows from [Kloeden & Rasmussen 2011, Theorem 6.3] that

|w| 1 ď 2K 2 α `´α ´´4K|ε| 1 |ε| 1 (note that L in [Kloeden &

Proof of Theorem 3.4.1

We shall rely on the following result of Brandière and Duflo. Recall that pΩ, F , Pq is a probability space equipped with a filtration pF n q nPN . Proposition 3.7.4. ( [Brandière & Duflo 1996, Proposition 4]) Given a sequence pγ n q of deterministic nonnegative stepsizes such that ř k γ k " `8 and ř k γ 2 k ă `8, consider the R d -valued stochastic process pz n q nPN given by z n`1 " pI `γn`1 H n qz n `γn`1 η n`1 `γn`1 ρ n`1 .

Assume that z 0 is F 0 -measurable and that the sequences pη n q, pρ n q together with the sequence of random matrices pH n q are pF n q-adapted. Moreover, on a given event A P F , assume the following facts:

i) ř n }ρ n } 2 ă 8. ii) lim sup Er}η n`1 } 2`a | F n s ă 8 for some a ą 0, and Erη n`1 | F n s " 0. iii) lim inf Er}η n`1 } 2 | F n s ą 0.
Let H P R dˆd be a deterministic matrix such that the real parts of its eigenvalues are all positive. Then, P pA X rz n Ñ 0s X rH n Ñ Hsq " 0.

We now enter the proof of Theorem 3.4.1. Recall the development (3.11) of bpz, tq near z ‹ and the spectral factorization (3.12) of the matrix D. To begin with, it will be convenient to make the variable change y " Q ´1pz ´z‹ q, and set hpy, tq " Q ´1bpQy `z‹ , tq " Λy `ẽpy, tq, with ẽpy, tq " Q ´1epQy `z‹ , tq, in such a way that our stochastic algorithm is rewritten as y n`1 " y n `γn`1 hpy n , τ n q `γn`1 ηn`1 `γn`1 ρn`1 where ηn is as in the statement of the theorem and ρn " Q ´1ρ n . Observe that the assumptions on the function e in the statement of the theorem remain true for ẽ with z ‹ replaced by zero.

If the matrix Λ has only eigenvalues with (strictly) positive real parts, i.e., d ´" 0, then we can apply Proposition 3.7.4 to the sequence pz n q. Henceforth, we deal with the more complicated case where d ´ą 0.

Apply Proposition 3.7.3 to h to obtain h and σ, t 1 respectively small and large enough and w : R d ´ˆI 1 Ñ R d `where I 1 :" rt 1 , `8q. By Assumption iv) of Theorem 3.4.1 and Proposition 3.7.3 we can choose σ 1 ď σ such that Equation (3.80) and Equation (3.81) hold. Now, given p P N, let us define the event E p " r@n ě p, }y n } ă σ 1 , τ n P I 1 s .

On E p , it holds that hpy n , τ n q " hpy n , τ n q and @n ě p, y n`1 " y n `γn`1 hpy n , τ n q `γn`1 ηn`1 `γn`1 ρn`1

" " y ń y ǹ  `γn`1 " h ´py n , τ n q h `py n , τ n q  `γn`1 " ηń `1 ηǹ `1 `γn`1 " ρń `1 ρǹ `1 (3.82)
where h is partitioned as in (3.67), and where ηn , ρn P R d ˘. Note that, by Proposition 3.7.3 and Assumptions vi) and vii) on the sequence pη n q, we can choose σ, t 1 respectively small and large enough such that

lim inf Er ηǹ `1 2 |F n s1 Ep py n q´2 lim sup Er B 1 wpy ń , τ n qη ń`1 2 |F n s1 Ep py n q ą c 2 2 . (3.83)
This inequality will be important in the end of our proof. Let t be in the interior of I 1 , and let y " py ´, y `q be in a neighborhood of 0. Make the variable change py ´, y `q Þ Ñ pu ´, u `q with u `" y `´wpy ´, tq, u ´" y

´, where w is the function defined in the statement of Proposition 3.7.3, and let W pu ´, u `, tq " h `py, tq ´B1 wpy ´, tqh ´py, tq ´B2 wpy ´, tq " h `ppu ´, u ``wpu ´, tqq, tq ´B1 wpu ´, tqh ´ppu ´, u ``wpu ´, tqq, tq ´B2 wpu ´, tq.

By Proposition 3.7.3 and Lemma 3.7.2, it holds that W pu ´, 0, tq " 0. Moreover, W pu ´, ¨, tq P C 1 by the assumptions on h. Therefore, writing yprq " pu ´, ru `` wpu ´, tqq for r P r0, 1s, and using the decomposition (3.67), we get that

W pu ´, u `, tq " ż 1 0 B 2 W pu ´, ru `, tqu `dr " Λ `uż 1 0 ˆB1 ε `pyprq, tq " 0 I d ` ´B1 wpu ´, tqB 1 ε ´pyprq, tq " 0 I d `˙u `dr.
We can also write yprq " py ´, ry ``p1 ´rqwpy ´, tqq. Recalling that wp0, tq " 0 and that }B 1 wpy ´, tq} is bounded on R d ´ˆI, we get by the mean value inequality that wpy ´, tq ď C y ´ where C ą 0 is a constant. Thus, yprq ď p1 `Cq y . Moreover, εpy, tq " Q ´1epQy, tq for }y} ă σ. Thus, we get by (3.13) that B 1 εpyprq, tq Ñ 0 as py, tq Ñ p0, 8q uniformly in r P r0, 1s. Using again the boundedness of }B 1 wp¨, ¨q}, we eventually obtain that W pu ´, u `, tq " `Λ``∆ py, tq ˘u`, with lim py,tqÑp0,8q ∆py, tq " 0.

On the event E p , assume that n ě p, and write u ǹ " y ǹ ´wpy ń , τ n q, u ń " y ń , (see Equation (3.82)). Choosing α ´ą 0 small enough so that the gap condition (3.65) is satisfied with m " 2, we have by Taylor's expansion wpy ń`1 , τ n`1 q ´wpy ń , τ n q " wpy ń`1 , τ n`1 q ´wpy ń , τ n`1 q `wpy ń , τ n`1 q ´wpy ń , τ n q " B 1 wpy ń , τ n`1 qpy ń`1 ´yń q `γn`1 B 2 wpy ń , τ n q ` n`1 ` γ n`1 , with n`1 ď sup

y ´Pry ń ,y ń`1 s B 2 1 wpy ´, τ n`1 q y ń`1 ´yń 2 , and γ n`1 ď sup τ Prτn,τ n`1 s B 2 2 wpy ń , τ q γ 2 n`1 .
Using this equation, we obtain

u ǹ`1 ´uǹ " γ n`1 W pu ń , u ǹ , τ n q `γn`1 `η ǹ`1 ´B1 wpy ń , τ n`1 qη ń`1 γn`1 `ρ ǹ`1 ´B1 wpy ń , τ n`1 qρ ń`1 ˘´ n`1 ´ γ n`1
`γn`1 `B1 wpy ń , τ n q ´B1 wpy ń , τ n`1 q ˘h´p y n , τ n q , which leads to

u ǹ`1 " u ǹ `γn`1 `Λ``∆ py n , τ n q ˘uǹ `γn`1 ηn`1 `γn`1 ρn`1 , (3.84)
with ηn`1 " ηǹ `1 ´B1 wpy ń , τ n qη ń`1 and

ρn`1 " ρǹ `1 ´B1 wpy ń , τ n qρ ń`1 ´1γ n`1 ą0 n`1 ` γ n`1 γ n`1
``B 1 wpy ń , τ n q ´B1 wpy ń , τ n`1 q ˘h´p y n , τ n q . (3.85)

Proofs for Section 3.4 79

To finish the proof, it remains to check that the noise sequence satisfies the assumptions of Proposition 3.7.4 on the event A p " E p Xry n Ñ 0s. In the remainder, C 1 will indicate some positive constant which can change from an inequality to another one.

First, we verify that ř n }ρ n } 2 ă 8 on A p by controlling each one of the terms of ρn . Combining the boundedness of B 1 wp¨, ¨q with the summability assumption ř n }ρ n`1 } 2 1 znPW ă `8 a.s., we immediately obtain on A p that

ř n }ρ ǹ`1 B1 wpy ń , τ n qρ ń`1 } 2 ă `8 given our choice of σ. Moreover, it holds that `} γ n`1 }{γ n`1 ˘2 ď C 1 γ 2
n`1 by invoking Proposition 3.7.3. In addition, using the boundedness of B 2 1 wp¨, ¨q, we can write

1 γ n`1 ą0 › › › › n`1 γ n`1 › › › › 2 ď 1 γ n`1 ą0 C 1 γ 2 n`1 y n`1 ´yn 4 ď C 1 γ 2 n`1 p hpy n , τ n q 4 ` ηn`1 4 ` ρn`1 4 q .
A coupling argument (see [Brandière & Duflo 1996, p. 401]) shows that we can simplify the condition

lim sup Er}η n`1 } 4 | F n s1 znPW ă 8 to Er}η n`1 } 4 | F n s1 znPW ă C 1 . The latter condi- tion implies that Er1 Ap ř n γ 2 n`1 η n`1 4 s ď ř n C 1 γ 2 n`1
, and therefore

ř n γ 2 n`1 η n`1 4 1 Ap ă
`8 a.s. As a consequence, noticing also the boundedness of phpy n , τ n qq and pρ n q on A p , we deduce that ř

n 1 γ n`1 ą0 › › › n`1 γ n`1 › › › 2
ă `8 on A p . We now briefly control the last term of ρn . By the mean value inequality, we obtain that › › `B1 wpy ń , τ n q ´B1 wpy ń , τ n`1 q ˘h´p y n , τ n q

› › ď γ n`1 sup py ´,tq B 2 B 1 wpy ´, tq }h ´py n , τ n q} ď C 1 γ n`1 ,
where the last inequality stems from Proposition 3.7.3-Equation (3.80) together with the boundedness of the sequence phpy n , τ n qq. In view of Equation (3.85) and the above estimates, we deduce that ř

n }ρ n`1 } 2 1 Ap ă `8 a.s. on A p .
We verify the remaining conditions on the noise sequence pη n q. We can easily remark that Erη n`1 |F n s " 0 and ηn`1 ď C 1 η n`1 on A p . Hence, lim sup Er}η n`1 } 4 | F n s1 znPW ă 8. The last condition, meaning that the noise is exciting enough, stems from noting that

2 lim inf Er ηn`1 2 |F n s1 Ap ě lim inf Er ηǹ `1 2 |F n s1 Ap ´2 lim sup Er B 1 wpy ń , τ n qη ń`1 2 |F n s1 Ap ą c 2 2 ,
where we used our choice of σ, t 1 and Equation (3.83).

Noticing that ry n Ñ 0s Ă r∆py n , τ n q Ñ 0s, we can now apply Proposition 3.7.4 to the sequence pu ǹ q (see Equation (3.84)) with A " A p to obtain P `Ap X ru ǹ Ñ 0s ˘" P `Ap X ru ǹ Ñ 0s X r∆py n , τ n q Ñ 0s ˘" 0 .

We now show that ry n Ñ 0s Ă ru ǹ Ñ 0s , which amounts to prove that wpy ń , τ n q Ñ 0 given y n Ñ 0. To that end, upon noting that wp0, ¨q " 0 and that B 1 wp¨, ¨q is bounded, it suffices to apply the mean value inequality, writing : }wpy ń , τ n q} " }wpy ń , τ n q ´wp0, τ n q} ď sup py ´,tq }B 1 wpy ´, tq} }y ń } ď K}y ń } .

We have shown so far that PpA p q " 0. Since y n " Q ´1z n and ry n Ñ 0s Ă Ť pPN E p , we finally obtain that Prz n Ñ 0s " Pry n Ñ 0s " P ˜ď pPN pry n Ñ 0s X E p q ¸" P ˜ď pPN A p ¸" 0.

Theorem 3.4.1 is proven. The matrix D coincides with ∇g 8 pz ‹ q, where the function g 8 is defined in (3.20).

As such, its expression is immediate. Recalling that p 8 Spx ‹ q ´q8 v ‹ " 0, we get gpz, tq ´Dpz ´z‹ q " » --pptqSpxq ´qptqv ´p8 ∇Spx ‹ qpx ´x‹ q `q8 pv ´v‹ q hptq∇F pxq ´rptqm ´h8

∇ 2 F px ‹ qpx ´x‹ q `r8 m ´m ´pv `εq ´1 2 ´pv ‹ `εq ´1 2 ¯fi ffi fl " » - - ´qptq `q8 0 ppptq ´p8 q∇Spx ‹ q 0 r 8 ´rptq phptq ´h8 q∇ 2 F px ‹ q m 2pv‹`εq 3 2 0 0 fi ffi fl » - v ´v‹ m x ´x‹ fi fl `» - - - pptqpSpxq ´Spx ‹ q ´∇Spx ‹ qpx ´x‹ qq hptqp∇F pxq ´∇2 F px ‹ qpx ´x‹ qq ´m d ˆ1 ? v`ε ´1 ? v‹`ε `v´v‹ 2pv‹`εq 3 2 ˙fi ffi ffi fl `» - pptqSpx ‹ q ´qptqv ‹ 0 0 fi fl ∆ " epz, tq `cptq.
Under the assumptions made, it is easy to see that the function epz, tq has the properties required in the statement of Theorem 3.4.1.

Proof of Theorem 3.4.3

Consider the matrix D defined in the statement of Lemma 3.4.2. A spectral analysis of this matrix as regards its eigenvalues with positive real parts is done in the following lemma.

Lemma 3.7.5. Let D be the matrix provided in the statement of Lemma 3.4.2. Each eigenvalue ζ of the matrix D such that ζ ą 0 is real, and its algebraic and geometric multiplicities are equal. Moreover, there is a one-to-one correspondence ϕ between these eigenvalues and the negative eigenvalues of

V 1 2 ∇ 2 F px ‹ qV 1 2 . Let d `be the dimension of the eigenspace of V 1 2 ∇ 2 F px ‹ qV 1 2 that is associated with its negative eigenvalues, let W " » - - w 1 . . . w d `fi ffi fl P R d `ˆd be a matrix which rows are independent eigenvectors of V 1 2 ∇ 2 F px ‹ qV 1 2
that generate this eigenspace, and denote as β k ă 0 the eigenvalue associated with w k . Then, the rows of the rank d `-matrix

A `" " 0 d `ˆd , W V 1 2 , ´diagpr 8 `ϕ´1 pβ k qqW V ´1 2 ı P R d `ˆ3d
generate the left eigenspace of D, the row k being an eigenvector for the eigenvalue ϕ ´1pβ k q.

Proof. It is obvious that the block lower-triangular matrix D has d eigenvalues equal to ´q8 and 2d eigenvalues which are those of the sub-matrix r D "

" ´r8 I d h 8 ∇ 2 F px ‹ q ´V 0  .
Given λ P C, we obtain by standard manipulations involving determinants that

detp r D ´λq " detpλpr 8 `λq `h8 V ∇ 2 F px ‹ qq " detpλpr 8 `λq `h8 V 1 2 ∇ 2 F px ‹ qV 1 2 q.
Denoting as

tβ k u d k"1 the eigenvalues of h 8 V 1 2 ∇ 2 F px ‹ qV 1 2
counting the multiplicities, we obtain from the last equation that the eigenvalues of r D are the solutions of the second order equations λ 2 `r8 λ `βk " 0, k " 1, . . . , d.

The product of the roots of such an equation is β k , and their sum is ´r8 ď 0. Thus, denoting as ζ k,1 and ζ k,2 these roots, it is easy to see that if

β k ě 0, then ζ k,1 , ζ k,2 ď 0, while if β k ă 0, then both ζ k,i are
real, and only one of them is positive. Thus, we have so far shown that the eigenvalues of D which real parts are positive are themselves real, and there is a one-to-one map ϕ from the set of positive eigenvalues of D to the set of negative eigenvalues of V 1 2 ∇ 2 F px ‹ qV 1 2 . Moreover, the algebraic multiplicity of the eigenvalue ζ ą 0 of D is equal to the multiplicity of ϕpζq.

Let us now turn to the left (row) eigenvectors of D that correspond to these eigenvalues. To that end, we shall solve the equation uD " ζu with u " r0, u 1 , u 2 s, u 1,2 P R 1ˆd , (3.86) for a given eigenvalue ζ ą 0 of D. Developing this equation, we get

´r8 u 1 ´u2 V " ζu 1 , h 8 u 1 ∇ 2 F px ‹ q " ζu 2 .
If we now write ũ1 " u 1 V ´1 2 and ũ2 " u 2 V 1 2 , this system becomes

´r8 ũ1 ´ũ 2 " ζ ũ1 , h 8 ũ1 V 1 2 ∇ 2 F px ‹ qV 1 2 " ζ ũ2 , or, equivalently, ũ2 " ´pr 8 `ζqũ 1 , ũ1 ´ζ2 `r8 ζ `h8 V 1 2 ∇ 2 F px ‹ qV 1 2 ¯" 0, which shows that ũ1 is a left eigenvector of V 1 2 ∇ 2 F px ‹ qV
1 2 associated with the eigenvalue ϕpζq. What's more, assume that r is the multiplicity of ϕpζq, and, without generality loss, that the submatrix W r,¨m ade of the first r rows of W generates the left eigenspace of ϕpζq. Then, the matrix

" 0 rˆd W r¨V 1 2 ´pr 8 `ζqW r¨V ´1 2
ı is a r-rank matrix which rows are independent left eigenvectors that generate the left eigenspace of D for the eigenvalue ζ. In particular, the algebraic and geometric multiplicities of this eigenvalue are equal. The same argument can be applied to the other positive eigenvalues of D.

We now have all the elements to prove Theorem 3.4.3. Recall Equation (3.14):

z n`1 " z n `γn`1 bpz n , τ n q `γn`1 η n`1 `γn`1 ρ n`1 ,
where bpz, tq " gpz, tq ´cptq " Dpz ´z‹ q `epz, tq and ρ n " cpτ n´1 q `ρ n . With these same notations, we check that Assumptions i)-vi) in the statement of Theorem 3.4.1 are satisfied. The function epz, tq satisfies Assumptions i)-iv) by Lemma 3.4.2. We now verify that the sequence pρ n q fulfills Assumption v). First, observe that ř n }cpτ n q} 2 ă 8 under Assumption 3.4.3-i). Then, we control the second term pρ n q. After straightforward derivations, one can show the existence of a positive constant C (depending only on ε and a neighborhood W of z ‹ ) such that }ρ n`1 } 2 1 znPW ď Cp}m n ´mn`1 } 2 `}v n`1 ´vn } 2 q1 znPW .

(3.87) Using the boundedness of the sequences ph n q and pr n q together with the update rule of m n and Assumption 3.4.3-iii), there exists a positive constant C 1 independent of n (which may change from an inequality to another) such that Lemma 3.7.5 shows that the row spaces of B `and A `are the same, which implies that Assumption vii) can be rewritten equivalently as Er A `ηn`1 2 | F n s1 znPW ě c 2 1 znPW . By inspecting the form of η n provided by Equation (3.28) (written as a column vector), one can readily check that Assumption 3.4.3-iv) implies Assumption vii) of Theorem 3.4.1 for a small enough neighborhood W, using the continuity of the covariance matrix

E " }m n ´mn`1 } 2 1 znPW ‰ ď γ 2 n`1 C 1 E " p1 `Eξ " }∇f px n , ξq} 2 ‰ q1 znPW ‰ ď C 1 γ 2 n`1 . ( 3
V 1 2 E ξ p∇f px, ξq ´∇F pxqqp∇f px, ξq ´∇F pxqq T V 1 2 when x is near x ‹ .

Proof of Theorem 3.4.4

As mentioned in Section 3.4.2.2, the proof of Theorem 3.4.4 is almost identical to the one of Theorem 3.4.3. We point out the main differences here. In Lemma 3.4.2, replace D by D " " 0 h 8 ∇ 2 F px ‹ q ´Id 0  and set cptq " 0. Then, in Lemma 3.7.5, replace the matrix V 1{2 ∇ 2 F px ‹ qV 1{2 by the Hessian ∇ 2 F px ‹ q.

Chapter 4

Constant step stochastic approximation involving the Clarke subdifferentials of non smooth functions

Introduction

In this chapter, we study the asymptotic behavior of the constant step Stochastic Gradient Descent (SGD) when the objective function is neither differentiable nor convex. Given an integer d ě 1 and a probability space pΞ, T , µq, let f : R d ˆΞ Ñ R, px, sq Þ Ñ f px, sq be a function which is assumed to be locally Lipschitz, generally non-differentiable and non-convex in the variable x, and µ-integrable in the variable s. The goal is to find a local minimum, or at least a critical point of the function F pxq " ş f px, sq µpdsq " Ef px, ¨q, i.e., a point x ‹ such that 0 P BF px ‹ q, where BF is the so-called Clarke subdifferential of F . It is assumed that the function f is available to the observer along with a sequence of independent Ξ-valued random variables pξ k q kPN on some probability space with the same probability law µ. The function F itself is assumed unknown due to, e.g., the difficulty of computing the integral Ef px, ¨q. Such non-smooth and non-convex problems are frequently encountered in the field of statistical learning. For instance this type of problem arises in the study of neural networks when the activation function is non-smooth, which is the case of the commonly used ReLU function.

We say that a sequence of random variables px n q nPN on R d is a SGD sequence with step size γ ą 0 if, with probability one,

x n`1 " x n ´γ∇f px n , ξ n`1 q (4.1)
for every n such that the function f p¨, ξ n`1 q is differentiable at point x n , where ∇f px n , ξ n`1 q represents the gradient w.r.t. the variable x n . When f p¨, ξ n`1 q is non-differentiable at x n , the update equation x n Ñ x n`1 is left undefined. The practioner is free to choose the value of x n`1 according to a predetermined selection policy. Typically, a reasonable choice is to select x n`1 in the set x n ´γBf px n , ξ n`1 q, where Bf px, sq represents the Clarke subdifferential of the function f p¨, sq at the point x. When such a policy is used, the resulting sequence will be referred to as a Clarke-SGD sequence. A second option used by practioners is to compute the derivative using the automatic differentiation provided in popular API's such as Tensorflow, PyTorch, etc. i.e., for all n,

x n`1 " x n ´γa f p¨,ξ n`1 q px n q (4.2)

where a h stands for the output of the automatic differentiation applied to a function h. We refer to such a sequence as an autograd sequence. This approach is useful when f p¨, sq is a composition of matrix multiplications and non-linear activation functions, of the form

f px, sq " pσ L pW L σ L´1 pW L´1 ¨¨¨σ 1 pW 1 X s qqq, Y s q , (4.3)
where x " pW 1 , ¨¨¨, W L q are the weights of the network represented by a finite sequence of L matrices, σ 1 , ¨¨¨, σ L are vector-valued functions, X s is a feature vector, Y s is a label and p¨, ¨q is some loss function. In such a case, the automatic differentiation is computed using the chain rule of function differentiation, by means of the celebrated backpropagation algorithm. When the mappings σ 1 , ¨¨¨, σ L , p¨, Y s q are differentiable, the chain rule indeed applies and the output coincides with the gradient. However, the chain rule fails in case of non-differentiable functions. The properties of the map a h are studied in the recent work [Bolte & Pauwels 2019].

In general, a h pxq may not be an element of the Clarke-subdifferential Bhpxq. It can even happen that a h pxq ‰ ∇hpxq at some points x where h is differentiable. However, the set of such peculiar points is proved to be Lebesgue negligible. As a consequence, if the initial point x 0 is chosen random according to some density w.r.t. the Lebesgue measure, an autograd sequence can be shown to be a SGD sequence in the sense of Equation (4.1) under some conditions. The aim of this chapter is to analyze the asymptotic behavior of SGD sequences in the case where the step γ is constant. About the literature. In two recent papers [START_REF] Majewski | [END_REF]] and [START_REF] Davis | [END_REF]], a closely related algorithm is analyzed under the assumption that the step size is vanishing, i.e., γ is replaced with a sequence pγ n q that tends to zero as n Ñ 8. From a theoretical point of view, the vanishing step size is convenient because, under various assumptions, it allows to demonstrate the almost sure convergence of the iterates x n to the set

Z ∆ " tx P R d : 0 P BF pxqu (4.4)
of critical points of F . However, in practical applications such as neural nets, a vanishing step size is rarely used because of slow convergence issues. In most computational frameworks, a possibly small but nevertheless constant step size is used by default. The price to pay is that the iterates are no longer expected to converge almost surely to the set Z but to fluctuate in the vicinity of Z as n is large. In this chapter, we aim at establishing a result of the type

@ε ą 0, lim sup nÑ8 Ppdpx n , Zq ą εq Ý Ý Ñ γÓ0 0, (4.5)
where d is the Euclidean distance between x n and the set Z. Although this result is weaker than in the vanishing step case, constant step stochastic algorithms can reach a neighborhood of Z faster than their decreasing step analogues, which is an important advantage in the applications where the accuracy of the estimates is not essential. Moreover, in practice they are able to cope with non stationary or slowly changing environments which are frequently encountered in signal processing, and possibly track a changing set of solutions [Benveniste et al. 1990, Kushner & Yin 2003].

The second difference between the present chapter and the papers [START_REF] Majewski | [END_REF]] and [START_REF] Davis | [END_REF] lies in the algorithm under study. In [START_REF] Majewski | [END_REF], Davis et al. 2020], the iterates are supposed to satisfy the inclusion

x n`1 ´xn γ n`1 P ´BF px n q `ηn`1 (4.6) for all n, where pη n q is a martingale increment noise w.r.t. the filtration pσpx 0 , ξ 1 , . . . , ξ n qq ně1 . Under the assumption that γ n Ñ 0 as n Ñ 8, the authors of [START_REF] Majewski | [END_REF], Davis et al. 2020] prove that almost surely, the continuous time linearly interpolated process constructed from a sequence px n q satisfying (4.6) is a so-called asymptotic pseudotrajectory [Benaïm et al. 2005] of the Differential Inclusion (DI)

9 xptq P ´BF pxptqq , (4.7)
that will be defined on R `" r0, 8q. Heuristically, this means that a sequence px n q satisfying (4.6) shadows a solution to (4.7) as n tends to infinity. This result is one of the key ingredients to establish the almost sure convergence of x n to the set Z. Unfortunately, a SGD sequence does not satisfy the condition (4.6) in general (setting apart the fact that γ is constant). To be more precise, consider a Clarke-SGD sequence as defined above. For all n, x n`1 " x n ´γBf px n , ξ n`1 q, which in turn implies

x n`1 ´xn γ P ´EBf px n , . q `ηn`1 , where pη n q is a martingale increment noise sequence, and where EBf px, . q represents the set-valued expectation ş Bf px, sqdµpsq. The above inclusion is analogous to (4.6) in the case where BF pxq " EBf px, ¨q for all x i.e., if one can interchange the expectation E and the Clarke subdifferential operator B. Although the interchange holds if e.g., the functions f p¨, sq are convex (in which case Bf px, sq would coincide with the classical convex subdifferential), one has in general BEf px, ¨q Ă EBf px, ¨q and the inclusion can be strict [Clarke et al. 1998, Proposition 2.2.2]. As a consequence, a Clarke-SGD sequence does not admit the oracle form (4.6) in general. For such a sequence, the corresponding DI reads 9 xptq P ´EBf pxptq, . q , (4.8) but unfortunately, the flow of this DI may contain spurious equilibria (an example is provided in this chapter). In [START_REF] Majewski | [END_REF]] the authors restrict their analysis to regular functions [Clarke et al. 1998, §2.4], for which the interchange of the expectation and the subdifferentiation applies. However, this assumption can be restrictive, since a function as simple as ´|x| is not regular at the critical point zero.

A second example where the oracle form Equation (4.6) does not hold is given by autograd sequences. Such an example is studied in [Bolte & Pauwels 2019], assuming that the step size is vanishing and that ξ takes its values over a finite set. It is proved that, the autograd sequence is an almost sure asymptotic pseudotrajectory of the DI 9

xptq P ´Dpxptqq, for some set-valued map D which is shown to be a conservative field with F as a potential. Properties of conservative fields are studied in [Bolte & Pauwels 2019]. In particular, it is proved that D " t∇f u Lebesgue almost everywhere. Despite this property, the DI 9 xptq P ´Dpxptqq substantially differs from (4.7). In particular, the set of equilibria may be strictly larger than the set Z of critical points of F .

Contributions

• We analyze the SGD algorithm (4.1) in the non-smooth, non-convex setting, under realistic assumptions: the step size is assumed to be constant along the iterations, and we neither assume the regularity of the functions involved, nor the knowledge of an oracle of BF as in (4.6). Our assumptions encompass Clarke SGD sequences and autograd sequences as special cases.

• Under mild conditions, we prove that when the initialization x 0 is randomly chosen with a density, all SGD sequences coincide almost surely, irrespective to the particular selection policy used at the points of non-differentiability. In this case, x n almost never hits a non-differentiable point of f p¨, ξ n`1 q and Equation (4.1) actually holds for all n. Moreover, we prove that

x n`1 ´xn γ " ´∇F px n q `ηn`1 , where pη n q is a martingale difference sequence, and ∇F px n q is the true gradient of F at x n . This argument allows to bypass the oracle assumption of [START_REF] Majewski | [END_REF], Davis et al. 2020].

• We establish that the continuous process obtained by piecewise affine interpolation of px n q is a weak asymptotic pseudotrajectory of the DI (4.7). In other words, the interpolated process converges in probability to the set of solutions to the DI, as γ Ñ 0, for the metric of uniform convergence on compact intervals.

• We establish the long run convergence of the iterates x n to the set Z of Clarke critical points of F , in the sense of Equation (4.5). This result holds under two main assumptions. First, it assumed that F admits a chain rule, which is satisfied for instance if F is a so-called tame function. Second, we assume a standard drift condition on the Markov chain (4.1). Finally, we provide verifiable conditions of the functions f p¨, sq under which the drift condition holds.

• In many practical situations, the drift conditions alluded to above are not satisfied. To circumvent this issue, we analyze a projected version of the SGD algorithm, which is similar in its principle to the well-known projected gradient algorithm in the classical stochastic approximation theory.

Chapter organization

Section 4.2 recalls some known facts about Clarke subdifferentials, conservative fields and differential inclusions. In Section 4.3, we study the elementary properties of almost-everywhere gradient functions, defined as the functions ϕpx, sq which coincide with ∇f px, sq almost everywhere. Practical examples are provided. In Section 4.4, we study the elementary properties of SGD sequences. Section 4.5 establishes the convergence in probability of the interpolated process to the set of solutions to the DI. In Section 4.6, we establish the long run convergence of the iterates to the set of Clarke critical points. Section 4.7 is devoted to the projected subgradient algorithm. The proofs are found in Section 4.8.

Preliminaries

Notations

If ν, ν 1 are two measures on some measurable space pΩ, Fq, ν ! ν 1 means that ν is absolutely continuous w.r.t. ν. The ν-completion of F is defined as the sigmaalgebra consisting of the sets S Ă Ω such that there exist A, B P F with A Ă S Ă B and νpBzAq " 0. For these sets, νpSq " νpAq.

If E is a metric space, we denote by BpEq the Borel sigma field on E. Let d be an integer. We denote by M pR d q the set of probability measures on BpR d q and by M 1 pR d q ∆ " tν P M pR d q : ş }x}νpdxq ă 8u. We denote as λ d the Lebesgue measure on R d . When the dimension is clear from the context, we denote as λ this Lebesgue measure. For a subset K Ă R d , we denote by M abs pKq ∆ " tν P M pR d q : ν ! λ and supppνq Ă Ku , where supppνq represents the support of ν.

If P is a Markov kernel on R d and g : R d Ñ R is a measurable function, P g represents the function on R d Ñ R given by P gpxq " ş P px, dyqgpyq, whenever the integral is well-defined. For every measure π P MpR d q, we denote by πP the measure given by πP " ş πpdxqP px, ¨q. We use the notation πpgq " ş g dπ whenever the integral is well-defined.

For every x P R d , r ą 0, Bpx, rq is the open Euclidean ball with center x and radius r. The notation 1 A stands for the indicator function of a set A, equal to one on that set and to zero otherwise. The notation A c represents the complementary set of a set A and clpAq its closure.

Clarke Subdifferential and Conservative Fields

In this section we briefly review some recent results of [Bolte & Pauwels 2019]

Almost-Everywhere Gradient Functions

Definition

Let pΞ, T , µq be a probability space, where the σ-field T is µ-complete. Let d ą 0 be an integer. Consider a function f : R d ˆΞ Ñ R. We denote by ∆ f ∆ " tpx, sq P R d ˆΞ : x P D f p¨,sq u the set of points px, sq s.t. f p¨, sq is differentiable at x. We denote by ∇f px, sq the gradient of f p¨, sq at x, whenever it exists.

The following technical lemma, which proof is provided in Section 4.8.1, is essential.

Almost-Everywhere Gradient Functions 91

Lemma 4.3.1. Assume that f is BpR d q b T -measurable and that f p¨, sq is continuous for every s P Ξ. Then ∆ f P BpR d q b T , and the function ϕ 0 : R d ˆΞ Ñ R d defined as

ϕ 0 px, sq " " ∇f px, sq if px, sq P ∆ f 0 otherwise, (4.10)
is BpR d q b T -measurable. Moreover, if f p¨, sq is locally Lipschitz continuous for every s P Ξ, then pλ b µqp∆ c f q " 0. Thanks to this lemma, the following definition makes sense. By Lemma 4.3.1, we observe that a.e.-gradients exist, since pλ b µqp∆ c f q " 0. Note that in Definition 4.3.1, we do not assume that ϕ is BpR d q b T {BpR d qmeasurable. The reason is that this property is not always easy to check on practical examples. However, if one denotes by BpR d q b T the λbµ completion of the σ-field BpR d q b T , an immediate consequence of Lemma 4.3.1 is that any a.e.-gradient of f is a BpR d q b T {BpR d q-measurable function.

Examples

Lazy gradient function. The function ϕ 0 given by Equation (4.10) is an a.e. gradient function.

ż 1 A pzqλ b µpdzq " ż ż 1 A ppx, sqqλpdxqµpdsq " 0 ,
where the last equality comes from the fact that for every s, D s " t∇f p¨, squ λ-a.e.

[Bolte & Pauwels 2019, Theorem 1].

We provide below an application of Proposition 4.3.2.

Autograd function. Consider Equation (4.3), which represents a loss of a neural network. Although f is just a composition of some simple functions, a direct calculation of the gradient (if it exists) may be tedious. Automatic differentiation deals with such functions by recursively applying the chain rule to the components of f . More formally consider a function f : R d Ñ R that can be written as a closed formula of simple functions, mathematically speaking this means that we can represent f by a directed graph. This graph (with q ą d vertices) is defined through a set-valued function parentspiq Ă t1, . . . , i ´1u, a directed edge in this setting will be j Ñ i with j P parentspiq. Associate to each vertex a simple function g i : R |parentspiq| Ñ R, given an input x " px 1 , . . . , x d q P R d we recursively define x i " g i ppx j q jPparentspiq q for i ą d and finally f pxq " x q . For instance, if f is a cross entropy loss of a neural network, with activation functions being ReLu or sigmoid functions, then g i are some compositions of simple functions log, exp, 1 1`x 2 , norms and piecewise polynomial functions, all being path differentiable [Bolte & Pauwels 2019, section 6], [Davis et al. 2020, Section 5.2]. Automatic differentiation libraries calculate the gradient of f by successively applying the chain rule (in the sense pg 1 ˝g2 q 1 " pg 1 1 ˝g2 qg 1 2 ) to the simple functions g i . While the chain rule is no longer valid in a nonsmooth setting (see e.g. [START_REF] Kakade | [END_REF]), it is shown in [Bolte & Pauwels 2019, Section 5] that when the simple functions are path-differentiable, the output of automatic differentiation (e.g. autograd in Py-Torch ( [START_REF] Paszke | [END_REF])) is a selection of some conservative field D for f . We refer to [Bolte & Pauwels 2019] for a more detailed account. We denote by a f pxq the output of automatic differentiation of a function f at some point x.

Assume that Ξ " N and for each s P Ξ, f p¨, sq is defined through a recursive graph of path differentiable functions (in the machine learning paradigm f p¨, sq will represent the loss related to one data point, while F p¨q is the average loss). By Proposition 4.3.2, the map px, sq Þ Ñ a f p¨,sq pxq is an a.e. gradient function for f .

SGD Sequences

Definition

Given a probability measure ν on BpR d q, define the probability space pΩ, F , P ν q as Ω " R d ˆΞN , F " BpR d q b T bN , and P ν " ν b µ bN . We denote by px 0 , pξ n q nPN ˚q the canonical process on Ω Ñ R d i.e., writing an elementary event in the space Ω as ω " pω n q nPN , we set x 0 pωq " ω 0 and ξ n pωq " ω n for each n ě 1. Under P ν , x 0 4.4. SGD Sequences 93 is a R d -valued random variable with the probability distribution ν, and the process pξ n q nPN ˚is an independent and identically distributed (i.i.d.) process such that the distribution of ξ 1 is µ, and x 0 and pξ n q are independent. We denote by F the λ b µ bN -completion of F .

Let f : R d ˆΞ Ñ R be a BpR d q b T {BpRq-measurable function.

Definition 4.4.1. Assume that f p¨, sq is locally Lipschitz continuous for every s P Ξ.

A sequence px n q nPN ˚of functions on Ω Ñ R d is called an SGD sequence for f with the step γ ą 0 if there exists an a.e.-gradient ϕ of f such that

x n`1 " x n ´γϕpx n , ξ n`1 q p@n ě 0q .

All SGD Sequences Are Almost Surely Equal

Consider the SGD sequence

x n`1 " x n ´γϕ 0 px n , ξ n`1 q, (4.12) generated by the lazy a.e. gradient ϕ 0 . Denote by P γ : R d ˆBpR d q Ñ r0, 1s the kernel of the homogeneous Markov process defined by this equation, which exists thanks to the BpR d q b T -measurability of ϕ 0 . This kernel is defined by the fact that its action on a measurable function g : R d Ñ R `, denoted as P γ gp¨q, is P γ gpxq " ż gpx ´γϕ 0 px, sqq µpdsq. (4.13) Define Γ as the set of all steps γ ą 0 such that P γ maps M abs pR d q into itself:

Γ ∆ " tγ P p0, `8q : @ρ P M abs pR d q, ρP γ ! λu .

Proposition 4.4.1. Consider γ P Γ and ν P M abs pR d q. Then, each SGD sequence px n q with the step γ is F {BpR d q bN -measurable. Moreover, for any two SGD sequences px n q and px 1 n q with the step γ, it holds that P ν rpx n q ‰ px 1 n qs " 0. Finally, the probability distribution of x n under P ν is Lebesgue-absolutely continuous for each n P N.

Note that P ν ! λ b µ bN since ν ! λ. Thus, the probability P ν rpx n q ‰ px 1 n qs is well-defined as an integral w.r.t. λ b µ bN .

Proof. Let px n q be the lazy SGD sequence given by (4.12). Given an a.e. gradient ϕ, define the SGD sequence pz n q as z 0 " x 0 , z n`1 " z n ´γϕpz n , ξ n`1 q for n ě 0. The sequence px n q is F {BpR d q bN -measurable thanks to Lemma 4.3.1. Moreover, applying recursively the property that ρP γ ! λ when ρ ! λ, we obtain that the distribution of x n is absolutely continuous for each n P N.

To establish the proposition, it suffices to show that z n is F {BpR d q-measurable for each n P N, and that P ν rz n ‰ x n s " 0, which results in particular in the absolute continuity of the distribution of z n . We shall prove these two properties by induction on n. They are trivial for n " 0. Assume they are true for n. Recall that z n`1 " z n ´γ∇f pz n , ξ n`1 q if pz n , ξ n`1 q P A, where A P BpR d q b T is such that pλ b µqpA c q " 0, and x n`1 " x n ´γ∇f px n , ξ n`1 q1 pxn,ξ n`1 qP∆ f . The set B " tω P Ω : z n`1 ‰ x n`1 u satisfies B Ă B 1 Y B 2 , where B 1 " tω P Ω : z n ‰ x n u and B 2 " tω P Ω : pz n , ξ n`1 q R Au.

By induction, B 1 P F and P ν pB 1 q " 0. By the aforementioned properties of A, the F -measurability of z n , and the absolute continuity of its distribution, we also obtain that B 2 P F and P ν pB 2 q " 0. Thus, B P F and P ν pBq " 0, and since x n`1 is F -measurable, z n`1 is F -measurable. Proposition 4.4.1 means that the SGD sequence does not depend on the specific a.e. gradient used by the practioner, provided that the law of x 0 has a density and γ P Γ. Let us make this last assumption clearer. Consider for instance d " 1 and suppose that f px, sq " 0.5x 2 for all s. If γ " 1, the SGD sequence x n`1 " x n ´γx n satisfies x 1 " 0 for any initial point and thus, does not admit a density, whereas for any other value of γ, x n has a density for all n, provided that x 0 has a density. Otherwise stated, Γ " R `zt1u in this example.

It is desirable to ensure that Γ contains almost all the points of R `. The next proposition shows that this will be the case under mild conditions. The proof is given in 4.8.2. Proposition 4.4.2. Assume that for µ-almost every s P Ξ, the function f p¨, sq satisfies the property that at λ-almost every point of R d , there is a neighborhood of this point on which it is C 2 . Then, Γ c is Lebesgue negligible.

This assumption holds true as soon as for µ-almost all s, f p¨, sq is tame, since in this case R d can be partitioned in manifolds on each of which f p¨, sq is C 2 ( [START_REF] Bolte | [END_REF]), and therefore f p¨, sq is C 2 (in the classical sense) on the union of manifolds of full dimension, and therefore almost everywhere.

SGD as a Robbins-Monro Algorithm

We make the following assumption on the function f : R d ˆΞ Ñ R.

Assumption 4.4.1.

i) There exists a measurable function κ : R d ˆΞ Ñ R s.t.

for each x P R d , ş κpx, sq µpdsq ă 8 and there exists ε ą 0 for which @y, z P Bpx, εq, @s P Ξ, |f py, sq ´f pz, sq| ď κpx, sq}y ´z}.

ii) There exists x P R d such that f px, ¨q is µ-integrable.

By this assumption, f px, ¨q is µ-integrable for each x P R d , and the function

F : R d Ñ R, x Þ Ñ ż f px, sq µpdsq (4.14)
is locally Lipschitz on R d . We denote by Z the set of (Clarke) critical points of F , as defined in Equation (4.4). Let pF n q ně0 be the filtration F n " σpx 0 , ξ 1 , . . . , ξ n q. We denote by E n "

Er¨|F n s the conditional expectation w.r.t. F n , where F n , stands for the λ b µ Ncompletion of F n .

Theorem 4.4.3. Let Assumption 4.4.1 holds true. Consider γ P Γ and ν P M abs pR d q X M 1 pR d q. Let px n q nPN ˚be a SGD sequence for f with the step γ. Then, for every n P N, it holds P ν -a.e. that i) F , f p¨, ξ n`1 q and f p¨, sq (for µ-almost every s) are differentiable at x n .

ii) x n`1 " x n ´γ∇f px n , ξ n`1 q.

iii) E n rx n`1 s " x n ´γ∇F px n q.

Theorem 4.4.3 is important because it shows that P ν -a.e., the SGD sequence px n q verifies

x n`1 " x n ´γ∇F px n q `γη n`1

for some random sequence pη n q which is a martingale difference sequence adapted to pF n q.

Dynamical Behavior

Assumptions and Result

In this section we prove that the SGD sequence px n q nPN ˚(which is by Theorem 4.4.3, under the stated assumptions, unique) closely follows a trajectory of a solution to the DI (4.7) as the step size γ tends to zero. To state the main result of this section, we need to strengthen Assumption 4.4.1.

Assumption 4.5.1. The function κ of Assumption 4.4.1 satisfies:

i) There exists a constant K ě 0 s.t. ş κpx, sq µpdsq ď Kp1 `}x}q for all x.

ii) For each compact set K Ă R d , sup xPK ş κpx, sq 2 µpdsq ă 8.

The first point guarantees the existence of global solutions to (4.7) starting from any initial point (see Section 2.2.2). Assumption 4.5.2. The closure of Γ contains 0.

By Proposition 4.4.2, Assumption 4.5.2 is mild. It holds for instance if every f p¨, sq is a tame function.

We recall that S ´BF pAq is the set of solutions to (4.7) that start from any point in the set A Ă R d . Theorem 4.5.1. Let Assumptions 4.4.1-4.5.2 hold true. Let tpx γ n q nPN ˚: γ P p0, γ 0 su be a collection of SGD sequences of steps γ P p0, γ 0 s. Denote by x γ the piecewise affine interpolated process x γ ptq " x γ n `pt{γ ´nqpx γ n`1 ´xγ n q p@t P rnγ, pn `1qγqq .

Then, for every compact set K Ă R d , @ε ą 0, lim γÑ0 γPΓ ˜sup νPM abs pKq

P ν pd C px γ , S ´BF pKqq ą εq ¸" 0 ,
where the distance d C is defined in (2.2). Moreover, the family of distributions tP ν px γ q ´1 : ν P M abs pKq, 0 ă γ ă γ 0 , γ P Γu is tight.

The proof is given in Section 4.8.4. Theorem 4.5.1 implies that the interpolated process x γ converges in probability as γ Ñ 0 to the set of solutions to (4.7). Moreover, the convergence is uniform w.r.t. to the choice of the initial distribution ν in the set of absolutely continuous measures supported by a given compact set.

Importance of the Randomization of x 0

In this paragraph, we discuss the case where x 0 is no longer random, but set to an arbitrary point in R d . In this case, there is no longer any guarantee that the iterates x n only hit the points where a gradient exist. We focus on the case where px n q is a Clarke-SGD sequence of the form (4.11), where the function ϕ is assumed BpR d qbT {BpR d q measurable for simplicity. By Assumption 4.4.1, it is not difficult to see that ϕpx, ¨q is µ-integrable for all x P R d and, denoting by Epϕpx, ¨qq the corresponding integral w.r.t. µ, we can rewrite the iterates under the form:

x n`1 " x n ´γEϕpx n , ¨q `γη n`1 , where η n`1 " Erϕpx n , ¨qs´ϕpx n , ξ n`1 q is a martingale difference sequence for the filtration pF n q. Obviously, Eϕpx, ¨q P EBf px, ¨q. As said in the introduction, we need Eϕpx, ¨q to belong to BF pxq in order to make sure that the algorithm trajectory shadows the DI 9

xptq P ´BF pxptqq. Unfortunately, the inclusion BF pxq Ă EBf px, ¨q can be strict, which can result in the fact that the DI 9 xptq P ´EBf pxptq, ¨q generates spurious trajectories that converge to spurious zeroes. The following example, which can be easily adapted to an arbitrary dimension, shows a case where this phenomenon happens.

Example 4.5.1. Take a finite probability space Ξ " t1, 2u and µpt1uq " µpt2uq " 1{2. Let f px, 1q " 2x1 xď0 and f px, 2q " 2x1 xě0 . We have F pxq " x, and therefore BF p0q " t1u, whereas Bf p0, 1q " Bf p0, 2q " r0, 2s and therefore ş Bf p0, sqµpdsq " r0, 1s. We see that 0 P EBf p0, ¨q while 0 R BF p0q. Furthermore, the trajectory defined on R `as xptq " "

1 ´t for t P r0, 1s 0 for t ą 1 , xp0q " 1, is a solution to the DI 9 xptq P ´EBf pxptq, ¨q, but not to the DI 9 xptq P ´BF pxptqq.

Example 4.5.2. Consider the same setting as in the previous example. Consider a stochastic gradient algorithm of the form (4.1), initialized at x 0 " 0 with ϕ such that ϕp0, 1q " ϕp0, 2q " 0. Then, the iterates x γ n are identically zero. This shows that the stochastic gradient descent may converge to a non critical point of F . Theorem 4.5.1 may fail unless a random initial point is chosen.

Long Run Convergence

Assumptions and Result

As discussed in the introduction, the SGD sequence px n q is not expected to converge in probability to Z when the step is constant. Instead, we shall establish the convergence (4.5). The "long run" convergence referred to here is understood in this sense.

In all this section, we shall focus on the lazy SGD sequences described by Equation (4.12). This incurs no loss of generality, since any two SGD sequences are equal P ν -a.e. by Proposition 4.4.1 as long as ν ! λ. Our starting point is to see the process px n q and as a Markov process which kernel P γ is defined by Equation (4.13). Our first task is to establish the ergodicity of this Markov process under the convenient assumptions. Namely, we show that P γ has a unique invariant probability measure π γ , i.e., π γ P γ " π γ , and that }P n γ px, ¨q ´πγ } TV Ñ 0 as n Ñ 8 for each x P R d , where } ¨}TV is the total variation norm. Further, we need to show that the family of invariant distributions tπ γ u γPp0,γ 0 s for a certain γ 0 ą 0 is tight. The long run behavior referred to above is then intimately connected with the properties of the accumulation points of this family as γ Ñ 0. To study these properties, we get back to the DI 9

x P ´BF pxq (we recall that a concise account of the notions relative to this dynamical system and needed in this chapter is provided in Section 2.2.2). The crucial point here is to show, with the help of Theorem 4.5.1, that the accumulation points of tπ γ u as γ Ñ 0 are invariant measures for the set-valued flow induced by the DI. In its original form, this idea dates back to the work of [START_REF] Has'minskiȋ | The averaging principle for parabolic and elliptic differential equations and Markov processes with small diffusion[END_REF]. We observe here that while the notion of invariant measure for a single-valued semiflow induced by, say, an ordinary differential equation, is classical, it is probably less known in the case of a set-valued differential inclusion. We borrow it from the work of Roth and Sandholm [START_REF] Roth | [END_REF].

Having shown that the accumulation points of tπ γ u are invariant for the DI 9

x P ´BF pxq, the final step of the proof is to make use of Poincaré's recurrence theorem, that asserts that the invariant measures of a semiflow are supported by the so-called Birkhoff center of this semiflow (again, a set-valued version of Poincaré's recurrence theorem is provided in [Aubin et al. 1991, Faure & Roth 2013]). To establish the convergence (4.5), it remains to show that the Birkhoff center of the DI 9

x P ´BF pxq coincides with zer BF . The natural assumption that ensures the identity of these two sets will be that F admits a chain rule [Clarke et al. 1998, Bolte et al. 2007, Davis et al. 2020].

Our assumption regarding the behavior of the Markov kernel P γ reads as follows.

Assumption 4.6.1. There exist measurable functions V : R d Ñ r0, `8q, p : R d Ñ r0, `8q, α : p0, `8q Ñ p0, `8q and a constant C ě 0 s.t. the following holds for every γ P Γ X p0, γ 0 s. i) There exists R ą 0 and a positive Borel measure ρ on R d (R, ρ possibly depending on γ) such that @x P clpBp0, Rqq, @A P BpR d q, P γ px, Aq ě ρpAq.

ii) sup clpBp0,Rqq V ă 8 and inf Bp0,Rq c p ą 0. Moreover, for every x P R d ,

P γ V pxq ď V pxq ´αpγqppxq `Cαpγq1 }x}ďR . (4.15)
iii) The function ppxq converges to infinity as }x} Ñ 8.

Assumptions of this type are frequently encountered in the field of Markov chains. Assumption 4.6.1-(i)) states that clpBp0, Rqq is a so-called small set for the kernel P γ , and Assumption 4.6.1-(ii)) is a standard drift assumption. Taken together, they ensure that the kernel P γ is a so-called Harris-recurrent kernel, that it admits a unique invariant probability distribution π γ , and finally, that this kernel is ergodic in the sense that }P γ px, ¨q´π γ } TV Ñ 0 as n Ñ 8 (see [Meyn & Tweedie 2009]). The introduction of the factors αpγq and Cαpγq in Equation (4.15) guarantees moreover the tightness of the family tπ γ u γPp0,γ 0 s .

In Section 4.6.2, we provide sufficient and verifiable conditions ensuring the validity of Assumption 4.6.1 for P γ .

As announced above, we also need:

Assumption 4.6.2. The function F defined by (4.14) admits a chain rule, namely, for any absolutely continuous curve z : R `Ñ R d , for almost all t ą 0, @v P BF pzptqq, xv, 9 zptqy " pF ˝zq 1 ptq .

Assumption 4.6.2 is satisfied as soon as F is path-differentiable, for instance when F is either convex, regular, Whitney stratifiable or tame (see [Bolte & Pauwels 2019, Proposition 1]and [START_REF] Bolte | [END_REF], Davis et al. 2020]).

Theorem 4.6.1. Let Assumptions 4.4.1-4.5.2 and 4.6.1-4.6.2 hold true. Let tpx γ n q nPN ˚:

γ P p0, γ 0 su be a collection of SGD sequences of step-size γ. Then, the set Z " tx : 0 P BF pxqu is nonempty and for all ν P M abs pR d q and all ε ą 0, lim sup In this paragraph, we provide sufficient conditions under which Assumption 4.6.1 hold true. A simple way to ensure the truth of Assumption 4.6.1-(i)) is to add a small random perturbation to the function ϕ 0 px, sq. Formally, we modify algorithms described by Equation (4.12) and (4.18), and write

x n`1 " x n ´γϕ 0 px n , ξ n`1 q `γ n`1
where p n q is a sequence of centered i.i.d. random variables of law µ d , independent from tx 0 , pξ n qu, and such that the distribution of 1 " µ d has a continuous and positive density on R d . The Gaussian case 1 " N p0, aI d q where a ą 0 is some small variance is of course a typical example of such a perturbation. Consider now a fixed γ and denote by r P the Markov kernel induced by the modified equation.

Proposition 4.6.2. Let Assumption 4.5.1 hold true. Then, for each R ą 0, there exists ε ą 0 such that @x P clpBp0, Rqq, @A P BpR d q, r P px, Aq ě ε λpA X clpBp0, 1qqq, Thus, Assumption 4.6.1-(i)) is satisfied for r P .

We now turn to the assumptions 4.6.1-(ii)) and 4.6.1-(iii)).

Proposition 4.6.3. Assume that there exists R ě 0, C ą 0, and a measurable function β : Ξ Ñ R `such that the following conditions hold: i) For every s P Ξ, the function f p¨, sq is differentiable outside the ball clpBp0, Rqq. Moreover, for each x, x 1 R clpBp0, Rqq, }∇f px, sq ´∇f px 1 , sq} ď βpsq}x ´x1 } and ş β 2 dµ ă 8.

ii) For all x R clpBp0, Rqq, ş }∇f px, sq} 2 µpdsq ď Cp1 `}∇F pxq} 2 q.

iii) lim }x}Ñ8 }∇F pxq} " `8.

iv) Function F is lower bounded i.e., inf F ą ´8.

Then, it holds that P γ F pxq ď F pxq ´γp1 ´γKq1 }x}ą2R }∇F pxq} 2 `γ2 K1 }x}ą2R `γK1 }x}ď2R (4.17)

for some constant K ą 0. In particular, Assumptions 4.6.1-(ii)) and 4.6.1-(iii)) hold true.

We finally observe that this proposition can be easily adapted to the case where the kernel P γ is replaced with the kernel r P of Proposition 4.6.2.

The Projected Subgradient Algorithm

In many practical settings, the conditions of Proposition 4.6.3 that ensure the truth of Assumptions 4.6.1-(ii)) and 4.6.1-(iii)) are not satisfied. This is for instance the case when the function f is described by Equation ( 4.3) with the mappings σ at the right hand side of this equation being all equal to the ReLU function. In such situations, it is often pertinent to replace the SGD sequence with a projected version of the algorithm. Given an a.e.-gradient ϕ of the function f and a non empty compact and convex set K Ă R d , a projected SGD sequence px γ,K n q is given by the recursion

x γ,K 0 " x 0 , x γ,K n`1 " Π K px γ,K n ´γϕpx γ,K n , ξ n`1 qq , (4.18)
where Π K stands for a Euclidean projection onto K. Our purpose is to generalize Theorem 4.5.1 to this situation. This generalization is not immediate for several reasons. First, the projection step is likely to introduce spurious local minima. As far as the iterates (4.18) are concerned, the role of differential inclusion (4.7) is now played by the differential inclusion:

9 xptq P ´BF pxptqq ´NK pxptqq , (4.19)
where N K pxq stands the normal cone of K at point x. The set of equilibria of the above differential inclusion coincides with the set Z K :" tx P R d : 0 P ´BF pxq ´NK pxqu , which we shall refer to as the set of Karush-Kuhn-Tucker points. A second theoretical difficulty is related to the fact that Proposition 4.4.1 does no longer hold. Indeed, it can happen x 0 has a density, but the next iterates x γ,K n don't. The reason is that x γ,K n generally has a non zero probability to be in the (Lebesgue negligible) border of K, that is, clpKqz intpKq, where clpKq and intpKq respectively stand for the closure and the interior of K.

We shall focus here on the case where K " clpBp0, rqq with r ą 0. We shall use Π r , x γ,r n , N r as shorthand notations for Π K , x γ,K n , and N K respectively. In this case N r pxq " t0u if x ă r, N r pxq " tλx : λ ě 0u if x " r and N r pxq " H otherwise.

We make the following assumption.

Assumption 4.7.1. For every x P R d , the law of ϕ 0 px, ξq, where ξ " µ, is absolutely continuous relatively to Lebesgue.

Assumption 4.7.1 is much stronger than Assumption 4.5.2. Indeed, it implies that the distribution of x γ,r n ´γϕpx γ,r n , ξ n`1 q is always Lebesgue-absolutely continuous. It is useful to note though that Assumption 4.7.1 holds upon adding at each step a small random perturbation to ϕ 0 as in Section 4.6.2 above.

In order to state our first result in this framework, we need to introduce some new notations. We let Sprq :" tx : x " r, x P R d u be the sphere of radius r. By [Folland 2013, Theorem 2.49], there is a unique measure2 1 on Sp1q such that for any positive function f : R d Ñ R, we have:

ż f pxqλ d pdxq " ż 8 0 ż Sp1q f prθqr d´1 1 pdθqλ 1 pdrq . (4.20)
We define the measure r on Sprq as r pAq " 1 pA{rq for each Borel set A Ă Sprq.

We denote as M r the set of measures ν " ν 1 `ν2 , where ν 1 P M abs and ν 2 ! r .

For a set C Ă R d we define M r pCq as the measures in M r that are supported on C.

Notice that M abs pCq Ă M r pCq.

The next proposition, which is proven in the same way as Proposition 4.4.1, shows that for almost every r ą 0, all projected SGD sequences are almost surely equal.

Proposition 4.7.1. Let Assumption 4.7.1 hold true. Then, for almost every r ą 0, @ν P M r , each projected SGD sequence px γ,r n q is F {BpR d q bN -measurable. Moreover, for any two projected SGD sequences px γ,r n q and py γ,r n q, it holds that P ν rpx γ,r n q ‰ py γ,r n qs " 0. Finally, under P ν , for every n P N, the probability distribution of x γ,r n is in M r .

By Proposition 4.7.1 we can focus on the lazy projected SGD sequence:

x γ,r n`1 " Π r px γ,r n ´γϕ 0 px γ,r n , ξ n`1 qq . (4.21)
We define its associated kernel P r γ gpxq " ż gpΠ r px ´γϕ 0 px, sqqqµpdsq .

(4.22)

The next two theorems are analogous to Theorems 4.4.3 and 4.5.1.

Theorem 4.7.2. Let Assumptions 4.4.1 and 4.7.1 hold. Then for almost every r ą 0 , @ν P M r , for every n P N it holds P ν -a.e. i) F , f p¨, ξ n`1 q and f p¨, sq (for µ-a.e. s) are differentiable at x γ,r n .

ii) x γ,r n`1 P x γ,r n ´γ∇f px γ,r n , ξ n`1 q ´γN r pΠ r px γ,r n ´γ∇f px γ,r n , ξ n`1 qqq.

Theorem 4.7.3. Let Assumptions 4.4.1-4.5.1 and 4.7.1 hold true. Denote x γ,r the piecewise affine interpolated process:

x γ,r ptq " x γ,r n `pt{γ ´nqpx γ,r n`1 ´xγ,r n q p@t P rnγ, pn `1qγqq .

Then, for almost every r ą 0, for every compact set K Ă clpBp0, rqq, @ε ą 0, lim γÑ0 ˜sup νPM r pKq P ν pd C px γ,r , S ´BF ´Nr pKqq ą εq ¸" 0 .

Moreover, for any γ 0 ą 0, the family of distributions tP ν px γ,r q ´1 : ν P M r pKq, 0 ă γ ă γ 0 u is tight.

We compare Theorems 4.4.3 and 4.5.1. First, because of the projection step (and with the help of Assumption 4.7.1), the law of the n-th iterate is no longer in M abs , but in M r . Second, the continuous counterpart of Equation (4.18) is now the differential inclusion (4.19) Note that, if the solutions of the DI (4.7) that start from K all lie in clpBp0, rqq, then the set of these solutions coincides with the set of solutions of the DI (4.19) that start from K.

The analysis of the convergence of the iterates in the "long run" is greatly simplified by the introduction of the projection step. Compared to Assumption 4.6.1, we only assume the existence of a small set for P r γ , the drift condition of the form 4.6.1-(ii))-(iii)) is then automatically satisfied, thanks to the projection step (see Section 4.8.5).

Assumption 4.7.2. There is R ą 0 and γ 0 ą 0 such that for every γ P p0, γ 0 s there is ρ γ such that Assumption 4.6.1-(i)) hold for pR, ρ γ q (note that R is independent of γ here).

As shown in Section 4.6.2, Assumption 4.7.2 holds upon adding to ϕ 0 a small random perturbation.

Theorem 4.7.4. Let Assumptions 4.4.1-4.5.1 and 4.6.2-4.7.2 hold. Let tpx γ,r n q nPN ˚:

γ P p0, γ 0 su be a collection of projected SGD sequences of step-size γ. Then, for almost every 0 ă r ď R, the set Z r " tx : 0 P BF pxq `Nr pxqu is nonempty and for all ν P M r and all ε ą 0, lim sup nÑ8 P ν pdpx γ,r n , Z r q ą εq Ý ÝÝ Ñ γÑ0 0.

(4.23) Theorem 4.7.4 is analogous to Theorem 4.6.1. Notice that, since M abs Ă M r , x 0 can still be initialized under a Lebesgue-absolutely continuous measure. On the other hand, as explained in the beginning of this section, due to the projection step, the iterates, instead of converging to Z, are now converging to the set of Karush-Kuhn-Tucker points related to the DI (4.19). In addition, using that f p¨, sq is continuous, the above set is unchanged if the inner intersection over 0 ă h ď δ is replaced by an intersection over the h s.t. 0 ă h ď δ and having rational coordinates i.e., h P Q d . Define:

Proofs

∆ 1 f :" č ε 1 PQ ď dPQ d č εPQ ď δPQ č 0ă h ďδ hPQ d
" px, sq : ˇˇˇf px `h, sq ´f px, sq ´xd, hy h ˇˇˇă ε `ε1 * (4.25) By construction, ∆ 1 f is a measurable set. We prove that ∆ 1 f " ∆ f . Consider px, sq P ∆ f and let d x be the gradient of f p¨, sq at x. By (4.24) for all ε P Q, there is a δ P Q such that: Moreover, one may choose δ k pεq ď δ 0 pεq. Inspecting first the inequality (4.26) for k " 0, we easily obtain that the quantity f px`h,sq´f px,sq h is bounded uniformly in h s.t. 0 ă }h} ď δ 0 pεq. Using this observation and again Equation (4.26), this in turn implies that pd k q is a bounded sequence. There exists d P R d and s.t. d k Ñ d along some extracted subsequence. Now consider ε ą 0 and choose k such that d k ´d ă ε 2 and ε 1 k ă ε 2 . For all h ď δ k pε{2q, ˇˇˇf px `h, sq ´f px, sq ´xd, hy h

ˇˇˇď ˇˇˇf px `h, sq ´f px, sq ´xd k , hy h ˇˇˇ` d ´dk ă ε
This means that d is the gradient of f p¨, sq at x, hence ∆ 1 f Ă ∆ f . Hence, the first point of the Lemma 4.3.1 is proved.

Denoting as e i the i th canonical vector of R d , the i th -component rϕ 0 s i in R d of the function ϕ 0 is given as

rϕ 0 px, sqs i " lim tÑ0 f px `te i , sq ´f px, sq t 1 ∆ f px, sq,
and the measurability of ϕ 0 follows from the measurability of f and the measurability of 1 ∆ f .

Finally, assume that f p¨, sq is locally Lipschitz continuous for every s P Ξ. From Rademacher's theorem [Clarke et al. 1998, Ch. 3], f p¨, sq is almost everywhere differentiable, which reads ş p1 ´1∆ f px, sqqλpdxq " 0. Using Fubini's theorem, ş R d ˆΞp1 ´1∆ f px, sqq λpdxq b µpdsq " 0, and the last point is proved.

Proof of Proposition 4.4.2

The idea of the proof is to show that for almost every γ and s we have that g s,γ pxq :" px ´γ∇f px, sqq1 ∆ f px, sq is almost everywhere a local diffeomorphism.

In order to prove that we define for each px, sq P R d ˆΞ the pseudo-hessian Hpx, sq P R dˆd as Hpx, sq i,j " lim sup tÑ0 x∇f px `te j , sq1 ∆ f px `te j , sq ´∇f px, sq, e i y t

1 ∆ f px, sq .
Since it is a limit of measurable functions, H is BpR d q b T measurable, and if f p¨, sq is two times differentiable at x then Hpx, sq is just the ordinary hessian. Now we define lpx, s, γq " detpγHpx, sq ´Idq if every entry in Hpx, sq is finite, and lpx, s, γq " 1 otherwise, it is a BpR d q b T b BpR `q measurable function (as a sum of two measurable functions). By the inverse function theorem we have that if f p¨, sq is C 2 at x and if detpγHpx, sq ´Idq ‰ 0, then g s,γ p¨q is a local diffeomorphism at x. Therefore lpx, s, γq ‰ 0 implies either the latter or f p¨, sq is not C 2 at x (or both). Let λ d , λ 1 denote Lebesgue measures respectively on R d and R `, we have by Fubini's theorem:

ż 1 lpx,s,γq"0 λ d pdxq b µpdsq b λ 1 pdγq " ż λ d b µptpx, sq : lpx, s, γq " 0uqλ 1 pdγq " ż ż ż 1 lpx,s,γq"0 λ 1 pdγqλ d pdxqµpdsq " 0 ,
where the last equality comes from the fact that for px, sq fixed lpx, s, γq " 0 only if 1{γ is in the spectrum of Hpx, sq which is finite. Therefore we have a Γ a set of full measure in R `such that for γ P Γ we have λ d b µptpx, sq : lpx, s, γq " 0uq " 0.

Once again applying Fubini's theorem we get that for almost every s P Ξ we have tx : g s,γ p¨q is a local diffeomorphism at xuq is of λ d -full measure (since for each s, f p¨, xq is almost everywhere C 2 ). Finally, for A Ă R d , γ P Γ and ν P M abs pR d q, we have Since an image of a null set by a diffeomorphism is a null set we have

νP γ pAq " ν b µptpx
λ d ptx : g s,γ pxq P Au X V i q " 0 .
Hence, νP γ pAq " 0, which proves our claim.

Proof of Theorem 4.4.3

Take ν ! λ and a SGD sequence px n q nPN , let S 1 Ă R d be the set of x for which ∇f px, sq exists for µ-almost every s, i.e.,

S 1 ∆ " " x P R d : ż Ξ p1 ´1∆ f px, sqq µpdsq " 0 * .
When Assumption 4.4.1 holds, Rademacher's theorem, lemma 4.3.1 and Fubini's theorem imply that S 1 P BpR d q and λpR d zS 1 q " 0. Hence, for µ-a.e. s we have f p¨, sq differentiable at x 0 , and since ξ 1 " µ, f p¨, ξ 1 q is differentiable at x 0 . Now by Rademacher's theorem again, the set S 2 Ă R d where F is differentiable satisfies λpR d zS 2 q " 0, therefore F is differentiable at x 0 . Moreover, with probability one x 0 is in S 1 X S 2 . Define Apxq where the interchange between the limit and the integral follows from Assumption 4.4.1 and the dominated convergence theorem. Hence, ∇F pxq " ş ∇f px, sq1 ∆ f px, sq µpdsq for all x P S 1 X S 2 . Now denote by ν n the law of x n . Since we assumed that ν 0 ! λ, it holds that P ν px 0 P S 1 X S 2 q " 1. Therefore, with probability one,

x 1 " x 1 1 S 1 XS 2 px 0 q " px 0 ´γ∇f px 0 , ξ 1 qq1 S 1 XS 2 px 0 q " x 0 ´γ∇f px 0 , ξ 1 q .

Thus, x 1 is integrable whenever x 0 is integrable, and E 0 px 1 q " x 0 ´γ∇F px 0 q. Since by Assumption ν 1 ! λ we can iterate our argument for x 2 and then for all x n and the conclusions of Theorem 4.4.3 follow.

Proof of Theorem 4.5.1

We want to apply [Bianchi et al. 2019, Theorem 5.1.], and therefore verify its assumptions [Bianchi et al. 2019, Assumption RM]. In order to fall in its setting we first need to rewrite our kernel in a more appropriate way. As BF takes nonempty compact values, it admits a measurable selection ϕpxq P BF pxq [Aliprantis & Border 2006, Lemma 18.2 and Corollary 18.15]. Take γ P Γ, a SGD sequence px γ n q and notice that by Theorem 4.4.3 it is P ν almost surely always in D F X S 1 , where S 1 is the set of x where ∇f px, sq exists for µ-a.e. s. Therefore its Markov kernel can be equivalently defined as:

P 1 γ px, gq ∆ " 1 D F XS 1 pxqP γ px, gq `1pD F XS 1 q c pxqgpx ´γϕpxqq .
Now we can apply [Bianchi et al. 2019, Theorem 5.1.] with h γ ps, xq " ´p1 D F XS 1 pxq∇F pxq1

pD F XS 1 q c pxqϕpxqq (note that it is independent of s) and we have hpx, sq P Hpx, sq " Hpxq ∆ " ´BF pxq. As we show next, [Bianchi et al. 2019, Assumption RM] now easily follows. First, it is immediate from the general properties of the Clarke subdifferential that the set-valued map ´BF is proper and uppersemicontinuous with convex and compact values, hence the assumption (iii) of [Bianchi et al. 2019, Assumption RM]. Assumption (ii) is immediate by the uppersemicontinuity of ´BF . Moreover, we obtain from Assumption 4.5.1 that there exists a constant K ě 0 such that }BF pxq} ď Kp1 `}x}q.

Thus, S ´BF is defined on the whole R d , and S ´BF is closed in pCpR `, R d q, dq (see [Aubin & Cellina 1984]), hence assumption (v). Finally, assumption (vi) comes from Assumption 4.5.1.

We remark that although, [Bianchi et al. 2019, Theorem 5.1] deals with a family of measures pP a q aPK , the proofs remain unchanged when we consider pP ν q νPM abs pKq .

4.8.5 Proof of Theorems 4.6.1 and 4.7.4

Both theorems are proved in the same way. In the following Q γ will denote either P γ and in this case H will denote ´BF , or Q γ " P r γ and H " ´BF ´Nr . The proof will be done in three steps:

• Lemma 4.8.2: Q γ has a unique invariant probability distribution π γ , with π γ P M abs if Q γ " P γ and π γ P M r otherwise, moreover Q γ is ergodic in the sense of the Total Variation norm.

• Lemma 4.8.3: The family tπ γ u γPp0,γ 0 s is tight.

• Proposition 4.8.4: The accumulation points of tπ γ u γPp0,γ 0 s as γ Ñ 0 are invariant for the DI 9 x P Hpxq.

Proofs 107

Before stating Lemma 4.8.2, we recall a general result on Markov processes. Let Q : R d ˆBpR d q Ñ r0, 1s be a Markov kernel on R d . A set B Ă R d is said to be a small-set for the kernel Q if there exists a positive measure ρ on R d such that Qpx, Aq ě ρpAq for each A P BpR d q, x P B.

Proposition 4.8.1. Assume that B is a small set for Q. Furthermore, assume that there exists a measurable function W : R d Ñ r0, 8q that is defined on R d and bounded on B, and a real number b ě 0, such that

QW ď W ´1 `b1 B . (4.27)
Then, Q admits a unique invariant probability distribution π, and moreover, the ergodicity result @x P R d , }Q n px, ¨q ´π} TV ÝÝÝÑ nÑ8 0 (4.28) holds true.

Indeed, by [Meyn & Tweedie 2009, Theorem 11.3.4], the kernel Q is a so-called positive Harris recurrent, meaning among others that it has a unique invariant probability distribution. Moreover, Q is aperiodic, hence the convergence (4.28), as shown by, e.g., [Meyn & Tweedie 2009, Theorem 13.0.1].

Lemma 4.8.2. Assume that either Assumptions 4.6.1-(i)) 4.6.1-(ii)) hold if Q γ " P γ or Assumption 4.7.2 holds and r ď R if Q γ " P r γ , then for every γ P p0, γ 0 s, the kernel Q γ admits a unique invariant measure π γ . Moreover,

@x P R d , › › Q n γ px, ¨q ´πγ › › TV ÝÝÝÑ nÑ8 0. (4.29)
Finally, if Q γ " P γ , assumptions of Theorem 4.4.3 hold true and γ P Γ then π γ is absolutely continuous w.r.t. the Lebesgue measure. If Q γ " P r γ and assumptions of Theorem 4.7.2 hold true, then π γ P M r .

Proof. By the inequality (4.15), the kernel P γ satisfies an inequality of the type (4.27), namely, P γ V ď V ´αpγqθ `Cαpγq1 }x}ďR , for some θ, C ą 0. Similarly, under Assumption 4.7.2 and r ď R, we have that for every x P clpBp0, rqq:

P r γ px, Aq " P γ px, Π ´1 r pAqq ě ρ γ pΠ ´1 r pAqq ,
that is to say clpBp0, rqq is a small set for P r γ . Inequality of the type Assumption 4.6.1-(ii))-(iii)) then hold for e.g. C " r, αpγq " 1, V " x `r1 x ąr and p " x .

Consider the case where Q γ " P γ , to prove that π γ is absolutely continuous w.r.t. the Lebesgue measure, consider a λ-null set A. By the convergence (4.29), we obtain that for any x P R d , P n γ px, Aq Ñ π γ pAq. Now take ν ! λ. By Proposition 4.4.1, we have that νP n γ ! λ. Hence, by the dominated convergence theorem, 0 " νP n γ pAq "

ż P n γ px, Aqνpdxq Ñ ż π γ pAqνpdxq " π γ pAq .
If Q γ " P r γ we obtain the same result with the help of Proposition 4.7.1.

Lemma 4.8.3. Let either Assumptions 4.6.1-(i)) -4.6.1-(iii)) hold if Q γ " P γ or Assumption 4.7.2 hold and r ď R if Q γ " P r γ . Let π γ be the invariant distribution of Q γ . Then, the family tπ γ : γ P p0, γ 0 su is tight.

Proof. If Q γ " P r γ then the family π γ is supported by clpBp0, rqq and is, therefore, tight. Otherwise we iterate (4.15), to obtain:

n ÿ k"0 Q k`1 γ V ď n ÿ k"0 Q k γ V ´αpγq n ÿ k"0 Q k γ p `Cpn `1qαpγq .
Therefore, since 0 ď Q k γ V ă `8 we have:

αpγq n ÿ k"0 Q k γ p ď V `Cpn `1qαpγq .
For a fixed M ą 0 we will bound now π γ pp^M q. Since π γ is an invariant distribution for Q γ , we have π γ P k γ " π γ . Hence, we have:

π γ pp ^M q " 1 n `1 n ÿ k"0 π γ Q k γ pp ^M q ď 1 n `1 n ÿ k"0 π γ pQ k γ p ^M q ď π γ ˆ" V pn `1qαpγq `C ^M ˙.
Letting n Ñ `8, by the dominated convergence theorem we obtain π γ pp ^M q ď π γ pC ^M q. And therefore by monotone convergence theorem π γ ppq ď C. Fix now ε ą 0, there is a K ą 0 such that C K ď ε, and by coercivity of p there is r ą 0 such that:

π γ p x ą rq ď π γ pp ą Kq ď C K
where the last bound comes from Markov's inequality. This concludes the proof.

The next proposition will show that any accumulation point of π γ is an invariant measure for the set-valued flow induced by the DI 9

xptq P Hpxptqq, first we introduce some definitions. Define the shift operator Θ t : CpR `, R d q Ñ CpR `, R d q by Θ t pxq " xpt `¨q, and the projection operator p 0 : CpR `, R d q Ñ R d by p 0 pxq " xp0q. Then, we have the following definition (see [START_REF] Roth | [END_REF] for details): Definition 4.8.1. We say that π P M pR d q is an invariant distribution for the flow induced by the DI 9 xptq P Hpxptqq, if there is ν P M pCpR `, R d qq, such that:

i) supp ν P S H pR d q, ii) νΘ ´1 t " ν,
iii) νp ´1 0 " π.

Proofs 109

Proposition 4.8.4. Let Assumptions 4.4.1-4.5.2 and 4.6.1 hold true. Denote by π γ the unique invariant distribution of P γ . Let pγ n q be a sequence on p0, γ 0 s X Γ s.t. γ n Ñ 0 and π γn converges narrowly to some probability measure π. Then, π is an invariant distribution for the flow induced by 9 xptq P ´BF pxptqq. Similarly, under Assumptions 4.4.1-4.5.1 and 4.7.1-4.7.2, for r ď R, denoting π γ the unique invariant distribution of P r γ , if π γn Ñ π, then π is an invariant distribution for the flow induced by 9

xptq P ´BF pxptqq ´Nr pxptqq.

Proof. Consider the case where Q γ " P γ . The proof essentially follows [Bianchi et al. 2019, section 7.]. Fix an ε ą 0 and write π n instead of π γn for simplicity. By Lemma 4.8.3 we have a compact K such that π n pKq ą 1 ´ε, we thus can define the conditional measures π K n pAq :" πnpAXKq πnpKq . Moreover, we have π K n P M abs pKq, therefore we can apply Theorem 4.5.1 and get that there is a compact set C of CpR `, R d q such that P π K γn ,γn X ´1 γn pCq ě 1 ´ε. Now we have

P πn,γn p¨q " ż R d P a,γn p¨qπ n pdaq ě ż K P a,γn p¨qπ n pdaq ě π n pKqP π K n ,γn p¨q , hence P πγ n ,γn X ´1 γn pCq ě π n pKqP π K γn ,γn X ´1 γn pCq ě p1 ´εq 2 .
Since ε is arbitrary this proves the tightness of v n :" P πγ n ,γn X ´1 γn . Take π n Ñ π and v n Ñ v P M pCpR `, R d qq. We now prove that v is an invariant distribution for the flow induced by the DI associated to ´BF (see Definition 4.8.1.)

We have π n " v n p ´1 0 , by continuity of p 0 . Thus, π " vp ´1 0 . Therefore, we have (iii) of Definition 4.8.1. Let η ą 0. By weak convergence of v n , vptx P CpR `, R d q : dpx, S ´BF pR d qq ď ηuq ě lim sup n v n ptx P CpR `, R d q : dpx, S ´BF pR d qq ď ηuq and v n ptx P CpR `, R d q : dpx, S ´BF pR d qq ď ηuq ě v n ptx P CpR `, R d q : dpx, S ´BF pKqq ă ηuq ě π n pKqP π K γn ,γn pdpX γn , S ´BF pKqq ă ηq ě p1 ´εqP π K γn ,γn pdpX γn , S ´BF pKqq ă ηq .

The last term converges to 1´ε, by Theorem 4.5.1, and by weak convergence we have vptx P CpR `, R d q : dpx, S ´BF pR d qq ě ηuq ě p1 ´εq, now letting η Ñ 0, by monotone convergence we have vpS ´BF pR d qqq ě 1 ´ε which proves (i) of Definition 4.8.1. Finally, the second point of Definition 4.8.1 is shown just like in [Bianchi et al. 2019, section 7.].

The proof of the case Q γ " P r γ is substantially the same under straightforward adaptations.

After some definitions we recall an important result about the support of a flow-invariant measure. The limit set L f of a function ) .

f P CpR `, R d q is L f " č tě0 f prt,
In [Faure & Roth 2013] (see also [Aubin et al. 1991]), a version of Poincaré's recurrence theorem, well-suited for our set-valued evolution systems, was provided:

Proposition 4.8.5. Each invariant measure for S H is supported by BC S H .

With the help of Proposition 4.8.5 we can finally prove Theorem 4.6.1.

Proof. Take γ P Γ, ε ą 0 and px γ n q an associated SGD sequence. We have by (4.28):

lim sup nÑ8 P ν rdistpx γ n , Zq ą εs " π γ ptx P R d : dpx, Zq ą εuq .
Now take any sequence γ i Ñ 0 with γ i P Γ, and π γ i the associated invariant distribution, we know from Lemmas 4.8.3-4.8.4 that we can extract a subsequence such that π γ i Ñ π, with π an invariant measure for the evolution system S ´BF . Therefore by weak convergence we have:

lim iÑ`8 π γ i ptx P R d : dpx, Zq ą 2εuq ď lim iÑ`8 π γ i ptx P R d : dpx, Zq ě εuq ď πptx P R d : dpx, Zq ě εuq,
where the last line comes from the Portmanteau theorem. We show that supp π Ă S, and therefore the last term is equal to zero, which concludes the proof. To that end, we make use of Proposition 4.8.5, that shows that each invariant measure of S ´BF is supported by BC S ´BF . Thus, it remains to show that BC S ´BF " Z (which at the same time will ensure us that Z is nonempty). It is obvious that Z Ă BC S ´BF . To show the reverse inclusion, take a P L S ´BF paq . Then, there exists a solution x to the differential inclusion such that xp0q " a and a P L x . But under Assumption 4.6.2 it holds ( [Davis et al. 2020, lemma 5.2]) that }9 xptq} " }B 0 F pxptqq} almost everywhere, and, moreover,

@t ě 0, F pxptqq ´F pxp0qq " ´ż t 0 }B 0 F pxpuqq} 2 du.
Therefore xptq " a for each t ě 0, thus, a P S. Observing that Z is a closed set (since BF is graph-closed, see [Clarke et al. 1998, Proposition 2.1.5]), we obtain that BC S ´BF " Z.

Similarly, take γ i Ñ 0 and and px γ i ,r n q the associated projected SGD sequences. After an extraction we get that π γ i Ñ π, with π an invariant measure for the flow S ´BF ´Nr and:

4.8. Proofs 111 lim γ i Ñ0 lim sup nÑ8 P ν rdistpx γ i ,r
n , Z r q ą 2εs ď πptx P R d : dpx, Z r q ą εuq .

Taking a P L S ´BF ´Nr paq , and x a solution to the associated differential inclusion with xp0q " a, we get under Assumption 4.6.2 [Davis et al. 2020, Lemma 6.3.] that 9 xptq " mint v : v P BF pxptqq `Nr pxptqqu, and moreover,

@t ě 0, F pxptqq ´F pxp0qq " ´ż t 0 9 xpuq 2 du .
That is to say xptq " a and a P Z r , which finishes the proof.

4.8.6 Proof of Proposition 4.6.2

Denote as ρ the probability distribution of the random variable γ 1 . By assumption, ρ has a continuous density that is positive at each point of R d . We denote as f this density. Let θ x be the probability distribution of the random variable Z "

x ´γϕ 0 px, ξ 1 q, which is the image of µ by the function x ´γϕ 0 px, ¨q. Our purpose is to show that Dε ą 0, @x P clpBp0, Rqq, @A P BpR d q, pθ x bρq rZ `γη 1 P As ě ε λpAXclpBp0, 1qqq.

Given L ą 0, we have by Assumption 4.5.1 and Markov's inequality that there exists a constant K ą 0 such that

θ x rZ R clpBp0, Lqqs ď K L p1 `}x}q.
Thus, taking L large enough, we obtain that @x P clpBp0, Rqq, θ x rZ R clpBp0, Lqqs ă 1{2. Moreover, we can always choose ε ą 0 is such a way that f puq ě 2ε for u P clpBp0, L `1qq, by the continuity and the positivity of f on the compact clpBp0, L 1qq. Thus,

pθ x b ρq rZ `γη 1 P As " ż A du ż R d θ x pdvq f pu ´vq ě ż AXclpBp0,1qq du ż clpBp0,Lqq θ x pdvq f pu ´vq ě 2ε ż AXclpBp0,1qq du ż clpBp0,Lqq θ x pdvq ě ε λpA X clpBp0, 1qqq.
4.8.7 Proof of Proposition 4.6.3 By Lebourg's mean value theorem [Clarke et al. 1998, Theorem 2.4], for each n P N, there exists α n P r0, 1s and ζ n P BF pu n q with u n "

x n ´αn γ∇f px n , ξ n`1 q1 ∆ f px n , ξ n`1 q, such that F px n`1 q " F px n q ´γxζ n , ∇f px n , ξ n`1 qy1 ∆ f px n , ξ n`1 q,
and the proof of this theorem (see [Clarke et al. 1998, Theorem 2.4] again) shows that u n can be chosen measurably as a function of px n , ξ n`1 q.

In the following, for the ease of readability, we make use of shorthand (and abusive) notations of the type 1 }x}ą2R x∇F pxq, ¨¨¨y to refer to x∇F pxq, ¨¨¨y if }x} ą 2R and to zero if not. We also denote ∇f px n , ξ n`1 q as ∇f n`1 to shorten the equations. We write

F px n`1 q " F px n q ´γ1 }xn}ď2R xζ n , ∇f n`1 y1 ∆ f px n , ξ n`1 q ´γ1 }xn}ą2R xζ n ´∇F px n q, ∇f n`1 y ´γ1 }xn}ą2R x∇F px n q, ∇f n`1 y.
We shall prove that

E n F px n`1 q ď F px n q ´γ1 }xn}ą2R }∇F px n q} 2 `γK1 }xn}ď2R `γ2 K1 }xn}ą2R ˆp1 `}∇F px n q}q ´ż }∇f px n , sq} 2 µpdsq ¯1{2 `ż }∇f px n , sq} 2 µpdsq (4.30)
where the constant K ą 0 is an absolute finite constant that can change from line to line in the derivations below. To that end, we write

F px n`1 q " F px n q ´γ1 }xn}ď2R 1 }un}ďR xζ n , ∇f n`1 y1 ∆ f px n , ξ n`1 q ´γ1 }xn}ď2R 1 }un}ąR xζ n , ∇f n`1 y1 ∆ f px n , ξ n`1 q ´γ1 }xn}ą2R 1 }un}ďR xζ n ´∇F px n q, ∇f n`1 y ´γ1 }xn}ą2R 1 }un}ąR x∇F pu n q ´∇F px n q, ∇f n`1 y ´γ1 }xn}ą2R x∇F px n q, ∇f n`1 y (4.31)
We start with the second term at the right hand side of this inequality. Noting from Assumption 4.5.1 that

1 }un}ďR }ζ n } ď sup }x}ďR }BF pxq} ď sup }x}ďR ż }Bf px, sq} µpdsq ď sup }x}ďR ż κpx, sq µpdsq ď K,
we have

γ1 }xn}ď2R 1 }un}ďR |xζ n , ∇f px n , ξ n`1 qy| ď γK1 }xn}ď2R }∇f n`1 },
and by integrating with respect to ξ n`1 and using Assumption 4.5.1 again, we get that

γ1 }xn}ď2R E n r1 }un}ďR |xζ n , ∇f n`1 y1 ∆ f px n , ξ n`1 q|s ď γK1 }xn}ď2R . (4.32)
Using Assumption 4.5.1, the next term at the right hand side of (4.31) can be bounded as

γ1 }xn}ď2R 1 }un}ąR |xζ n , ∇f n`1 y1 ∆ f px n , ξ n`1 q| ď γ1 }xn}ď2R 1 }un}ąR }∇F pu n q} }∇f n`1 } ď γ1 }xn}ď2R K p1 `}x n } `γ}∇f n`1 }q }∇f n`1 } ď γK1 }xn}ď2R `1 `}∇f n`1 } `γ}∇f n`1 } 2 ˘,
γ1 }xn}ď2R E n r1 }un}ąR |xζ n , ∇f n`1 y1 ∆ f px n , ξ n`1 q|s ď γK1 }xn}ď2R (4.33)
by using Assumption 4.5.1. We tackle the next term at the right hand side of (4.31). Fix a x ‹ R clpBp0, Rqq. By our assumptions it holds that each x R clpBp0, Rqq, }∇f px, sq} ď }∇f px ‹ , sq} `βpsq}x ´x‹ } ď β 1 psqp1 `}x}q, where β 1 p¨q is square integrable thanks to Assumption 4.5.1. Since

ż β 1 psq 2 µpdsq " ż 8 0 µrβ 1 p¨q ě ? t s dt ă 8,
it holds that µrβ 1 p¨q ě 1{t s " o tÑ0 pt 2 q. Using triangle inequality, we get that

1 }xn}ą2R 1 }un}ďR " 1 }xn}ą2R 1 }xn´αnγ∇f n`1 }ďR ď 1 }xn}ą2R 1 }∇f n`1 }ěp}xn}´Rq{γ ď 1 }xn}ą2R 1 β 1 pξ n`1 qě }xn}´R γp1`}xn}q ď 1 }xn}ą2R 1 β 1 pξ n`1 qě R γp1`2Rq
.

Using this result, we write

γ1 }xn}ą2R 1 }un}ďR |xζ n , ∇f n`1 y| ď Kγ1 }xn}ą2R 1 }un}ďR }∇f n`1 } ď Kγ1 }xn}ą2R }∇f n`1 }1 β 1 pξ n`1 qě R γp1`2Rq
Consequently,

γ1 }xn}ą2R E n r1 }un}ďR |xζ n , ∇f n`1 y|s ď γK1 }xn}ą2R ´ż }∇f px n , sq} 2 µpdsq ¯1{2 µrβ 1 p¨q ě K{γs 1{2 ď γ 2 K1 }xn}ą2R ´ż }∇f px n , sq} 2 µpdsq ¯1{2 .
(4.34)

Similarly,

γ1 }xn}ą2R 1 }un}ďR |x∇F px n q, ∇f n`1 y| ď γK1 }xn}ą2R }∇F px n q} }∇f n`1 }1 β 1 pξ n`1 qě R γp1`2Rq
, thus,

γ1 }xn}ą2R E n " 1 }un}ďR |x∇F px n q, ∇f n`1 y| ‰ ď γ 2 K1 }xn}ą2R }∇F px n q} ´ż }∇f px n , sq} 2 µpdsq ¯1{2 .
(4.35) We have that ∇F is Lipschitz outside clpBp0, Rqq. Thus, the next to last term at the right hand side of (4.31) satisfies γ1 }xn}ą2R 1 }un}ąR |x∇F pu n q ´∇F px n q, ∇f n`1 y| ď γ 2 K1 }xn}ą2R }∇f n`1 } 2 , and we get that γ1 }xn}ą2R 1 }un}ąR E n r|x∇F pu n q ´∇F px n q, ∇f n`1 y|s ď γ 2 K1 }xn}ą2R ż }∇f px n , sq} 2 µpdsq.

(4.36) Finally, we have ´γ1 }xn}ą2R E n rx∇F px n q, ∇f n`1 ys " ´γ1 }xn}ą2R }∇F px n q} 2 .

(4.37)

Inequalities (4.32)-(4.37) lead to (4.30).

Using Assumption (iii) of Proposition 4.6.3, Inequality (4.30) leads to Inequality (4.17). The validity of Assumptions 4.6.1-(ii)) and 4.6.1-(iii)) can then be checked easily.

Proof of Proposition 4.7.1

The next Lemma is the key ingredient in the proofs of Section 4.7.

Lemma 4.8.6. Assume that f p¨, sq is locally Lipschitz continuous for every s P Ξ. Then for λ 1 b λ d b µ-almost all pr, x, sq with r ą 0, it holds that pΠ r pxq, sq P ∆ f . For λ 1 b λ d -almost all pr, xq with r ą 0, it holds that Π r pxq P D F . Proof. Our first aim is to show that ż

1 ∆ c f pΠ r pxq, sq λ 1 pdrq b λ d pdxq b µpdsq " 0 . (4.38)
First, note by Fubini's theorem that

0 " ż 1 ∆ c f px, sq λ d pdxq b µpdsq " ż ΞˆR `żSp1q 1 ∆ c
f prθ, sqr d´1 1 pdθq µ b λ 1 pds ˆdrq , (4.39) that is to say, ptθ : prθ, sq P ∆ f uq " 0 for µ b λ 1 almost every ps, rq with r ą 0. Decompose Equation (4.38) as

ż 1 ∆ c f pΠ r pxq, sq λ 1 pdrq b λ d pdxq b µpdsq " ż 1 x ěr 1 ∆ c f pΠ r pxq, sq λ 1 pdrqbλ d pdxqbµpdsq`ż 1 x ăr 1 ∆ c
f px, sq λ 1 pdrqbλ d pdxqbµpdsq. Since for each s, f p¨, sq is differentiable almost everywhere, we have by Fubini's theorem: ż

1 x ăr 1 ∆ c f px, sq λ 1 pdrq b λ d pdxq b µpdsq " 0. Similarly, ż 1 x ěr 1 ∆ c f pΠ r pxq, sq λ 1 pdrq b λ d pdxq b µpdsq " ż 1 x ěr 1 ∆ c f ´rx x , s ¯λ1 pdrq b λ d pdxq b µpdsq " ż R `żΞˆR `żSp1q 1 r 1 ěr 1 ∆ c
f pr 1 θ, sqpr 1 q d´1 pdθq µ b λ 1 pds ˆdrq λ 1 pdr 1 q " 0 , 4.8. Proofs 115 with the last equality coming from Equation (4.39). Hence (4.38). The second statement can be proven along similar lines.

Consider r ą 0 such that the conclusion of Lemma 4.8.6 hold. Then the almost sure equality of all projected SGD sequence is proven in the same way as in Proposition 4.4.1. We can therefore consider the lazy projected SGD sequence x γ,r n`1 " Π r px γ,r n ´γϕ 0 px γ,r n , ξ n`1 qq. By Assumption 4.7.1 the law of x γ,r n`1{2 ∆ " x γ,r n ´γϕ 0 px γ,r n , ξ n`1 q is Lebesgue-absolutely continuous. Take A a borel set of R d such that λpAq " r pAq " 0. Then Ppx γ,r n`1 P Aq ď Ppx γ,r n`1{2 P Aq `P ¨r x γ,r n`1{2

x γ,r n`1{2

P A ' .
The first term is equal to zero by Lebesgue-absolutely continuity of the law of x γ,r n`1{2 . For the second term we write:

P ¨r x γ,r n`1{2
x γ,r n`1{2 P A '" ż pr 1 q d´1 1 A prθq pdθqλ 1 pdr 1 q " ż pr 1 q d´1 r pAqλ 1 pdr 1 q " 0 , which finishes the proof.

4.8.9 Proof of Theorems 4.7.2 and 4.7.3

Noting that the law of x γ,r n ´γϕ 0 px γ,r n , ξ n`1 q is Lebesgue-absolutely continuous by Assumption 4.7.1, the first point of Theorem 4.7.2 comes from Lemma 4.8.6. The second point comes upon noticing that Π r pxq ´x P ´Nr pΠ r pxqq.

Theorem 4.7.3 is proved in the same way as Theorem 4.5.1, by applying [Bianchi et al. 2019, Theorem 5.1.] with hps, xq " ´∇F pxq ´1{γpx ´γ∇f px, sq ´Πr px ´γ∇f px, sqqq P ´∇F pxq ´Nr px ´γ∇f px, sqq and Hpxq " Hps, xq " ´BF pxq ´Nr pxq.

Chapter 5

Stochastic subgradient descent escapes active strict saddles

Introduction

Stochastic approximation algorithms that operate on non-convex and non-smooth functions have recently attracted a great deal of attention, owing to their numerous applications in machine learning and in high-dimensional statistics. The archetype of such algorithms is the so-called Stochastic Subgradient Descent (SGD), which reads as follows. Given a locally Lipschitz function f : R d Ñ R which is not necessarily smooth nor convex, the R d -valued sequence px n q of iterates generated by such an algorithm satisfy the inclusion

x n`1 P x n ´γn Bf px n q `γn η n`1 , (5.1)
where the set-valued function Bf is the so-called Clarke subdifferential of f , the sequence pγ n q is a sequence of positive step sizes converging to zero, and η n`1 is a zero-mean random vector on R d which presence is typically due to the partial knowledge of Bf by the designer. It is desired that px n q converges to the set of local minimizers of the function f . Before delving into the subject of convergence towards minimizers, let us first consider the set Z :" tx P R d : 0 P Bf pxqu of Clarke critical points of f , which is generally larger than the set of minimizers, in the non-convex case. In order to ensure the convergence of px n q to Z, the sole local Lipschitz property of f is not enough (see [START_REF] Daniilidis | Aris Daniilidis and Dmitriy Drusvyatskiy[END_REF]] for a counterexample), and some form of structure for the function f is required. Since the work of Bolte et.al. [START_REF] Bolte | [END_REF] in optimization theory, it is well known that the so-called definable on an o-minimal structure (henceforth definable) functions, which belong to the family of Whitney stratifiable functions (cf. Section 2.4), is relevant for the convergence analysis of px n q and beyond. This class of functions is general enough so as to contain all the functions that are practically used in machine learning, statistics, or applied optimization. In this framework, the almost sure convergence of px n q to Z was established by Davis et.al. in [Davis et al. 2020]. Another work in the same line is [START_REF] Majewski | [END_REF]]. Bolte and Pauwels [Bolte & Pauwels 2019] generalize the algorithm (5.1) by replacing Bf with an arbitrary so-called conservative field. The constant step size regime γ n " γ is considered in [Bianchi et al. 2021a].

Thanks to these contributions, the convergence of px n q to the set Z is now well understood. However, as said above, Z is in general strictly larger than the set of minimizers, and can contain "spurious" points such as local maximizers or saddle points. The issue of the non-convergence of the sequence given by (5.1) towards spurious critical points is therefore crucial. The present chapter investigates this issue.

Before getting into the core of our subject, it is useful to make a quick overview of the results devoted to the avoidance of spurious critical points by the iterative algorithms. The rich literature on this subject has been almost entirely devoted to the smooth setting. In this framework, the research has followed two main axes:

• The noisy case, where the analogue of the sequence pη n q in the smooth version of Algorithm (5.1) is non zero. Here, the seminal works of Pemantle [Pemantle 1990] and Brandière and Duflo [Brandière & Duflo 1996] allow to establish the non-convergence of the Stochastic Gradient Descent (and, more generally, of Robbins-Monro algorithms) to a certain type of spurious critical points, sometimes referred to as traps or strict saddle. A critical point of a smooth function f is called a trap if the Hessian matrix of f at this point admits at least one negative eigenvalue. With probability one, the sequence px n q cannot converge to a trap, provided that the projection of the random perturbation η n onto the eigenspace of corresponding to the negative eigenvalues of the Hessian matrix (henceforth, eigenspace of negative curvature) has a non vanishing variance.

• The noiseless case where η n " 0, studied for smooth functions by [Lee et al. 2016].

Here the authors show that for Lebesgue almost all initialization points, the algorithm with constant step will avoid the traps.

While both of these approaches rely on the center-stable invariant manifold theorem which finds its roots in the work of Poincaré, they are different in spirit. Indeed, in [Lee et al. 2016] the trap avoidance is due to the random initialization of the algorithm, whereas in [Brandière & Duflo 1996, Pemantle 1990], it is due to the inherent stochasticity brought by the sequence pη n q.

We now get back to the non-smooth case. Here, the only paper that tackles the problem of the spurious points avoidance is, up to our knowledge, the recent contribution [Davis & Drusvyatskiy 2021] of Davis and Drusvyatskiy. The spurious points that were considered in this reference are the so-called active strict saddles. Informally, a critical point is an active strict saddle if it lies on a manifold M such that i) f varies sharply outside of M , ii) the restriction of f to M is smooth, and iii) the Riemannian Hessian of f on M has at least one negative eigenvalue. For instance, the function f : R 2 Ñ R, py, zq Þ Ñ |z| ´y2 admits the point p0, 0q as an active strict saddle with M " R ˆt0u, and the restriction of f to M is the function f M py, 0q " ´y2 , which obviously has a second-order negative curvature. In this setting, and assuming that f is weakly convex, the article [Davis & Drusvyatskiy 2021] focuses on the noiseless case, and study variants of the (implicit) proximal point algorithm rather than the (explicit) subgradient descent. Similarly to [Lee et al. 2016], they show that for Lebesgue almost every initialization point, different versions of the proximal algorithm avoid active strict saddles with probability one. Such a result is possible due to the fact that proximal methods implicitly run a gradient descent on a smoothened version of f -the Moreau envelope.

Contrary to [Davis & Drusvyatskiy 2021], the algorithm (5.1) studied in this chapter is explicit, meaning that it does not require the computation of a proximal operator associated with the non-smooth function. In this situation, the sole randomization of the initial point is not sufficient to expect an avoidance of active strict saddles. Here, in the same line as [Pemantle 1990, Brandière & Duflo 1996], our analysis strongly relies on the presence of the additive random perturbation η n .

In the framework of definable functions, we investigate the problem of the avoidance of the active strict saddle points. Our approach goes as follows. First, we need to show that the iterates px n q converge sufficiently fast to M , thanks to the sharpness of f outside this manifold. To that end, we first rely on the fact that when f is definable, its graph always admits a so-called Verdier stratification, which is perhaps less known than the Whitney stratification, and is a refinement of the latter [Loi 1998]. The key advantage of the Verdier over the Whitney stratification lies in a Lipschitz-like condition on the (Riemannian) gradients of f on two adjacent stratas, which is established in the chapter. Our second tool is an assumption that we term as the angle condition. Roughly, this assumption provides a lower bound on the inner product between the subgradients of f at x and the normal direction from M to x when the point x is near M . The angle condition allow to control the distance between the iterate x n of Algorithm (5.1) and the manifold M . As the restriction f M of f to M is smooth, the projected iterates, using the Verdier stratification property, are shown to follow a dynamics which is similar to a (smooth) Stochastic Gradient Descent, up to a residual term induced by the projection step. In that sense, the avoidance of active strict saddles in the non-smooth setting follows from the avoidance of traps in the smooth setting, as established in [Brandière & Duflo 1996]. We show that the strict saddle is avoided under the assumption that the (conditional) noise covariance matrix has a non zero projection on the subspace with negative curvature associated with f M near the active strict saddle.

Before pursuing, it is important to discuss the matter of the genericity of the assumptions that we just outlined. First, since our avoidance results are restricted to the active strict saddles, the question of the presence of critical points that are neither local minima nor active strict saddles is immediately raised. Actually, this question was considered in [Drusvyatskiy et al. 2016, Davis & Drusvyatskiy 2021]. It is established there that if f is definable and weakly convex, then for Lebesgue almost all vectors u P R d , the function f u pxq ∆ " f pxq ´xu, xy admits a finite number of Clarke critical points, and that each of these points is either an active strict saddle or a local minimizer. In that sense, in the class of definable weakly convex functions, spurious critical points generically coincide with active strict saddles. We also need to inspect the generality of the Verdier and the angle conditions. In Theorem 5.3.2 below, we show that these assumptions are automatically satisfied when f is weakly convex. From these considerations, we conclude that generically in the sense of [Drusvyatskiy et al. 2016, Davis & Drusvyatskiy 2021], the SGD algorithm (5.1) converges to a local minimum when f is a weakly convex function, assuming that the noise is omnidirectional enough at the strict saddles. We emphasize the fact that, while the genericity of the active strict saddles is established in the above sense for weakly convex functions, no assumption on weak convexity is made for our avoidance of traps result.

Let us summarize the contributions of this chapter:

• Firstly, we bring to the fore the fact that definable functions admit stratifications of the Verdier type. These are more refined than the Whitney stratifications which were popularized in the optimization literature by [START_REF] Bolte | [END_REF]].

While such stratifications are well-known in the literature on o-minimal structures [Loi 1998], up to our knowledge, they have not been used yet in the field of non smooth optimization. To illustrate their interest in this field, we study the properties of the Verdier stratifiable functions as regards their Clarke subdifferentials. Specifically, we refine the so-called projection formula (see [Bolte et al. 2007, Proposition 4] and Lemma 2.4.10 below) to the case of definable, locally Lipschitz continuous functions by establishing a Lipschitzlike condition on the (Riemannian) gradients of two adjacent stratas.

• With the help of the Verdier and the angle conditions, we show that the SGD avoids the active strict saddles if the noise η n is omnidirectional enough.

The chapter is organized as follows. In Section 5.2 we fix the notations and prove the reinforced projection formula stated in Theorem 5.2.1. In Section 5.3, we discuss the notion of an active strict saddle. After recalling some results of [Davis & Drusvyatskiy 2021], we introduce the Verdier and angle conditions. We also discuss the genericity of the these conditions, in the class of weakly convex functions. In Section 5.4, we state the main result of this chapter, namely, the avoidance of active strict saddles. Section 5.5 is devoted to the proofs.

Preliminaries

Notations. Let d ě 1 be an integer. Given a set S Ă R d , S denotes the closure of S, and convpSq and convpSq respectively denote the convex hull and the closed convex hull of S. The distance to S is denoted as distpx, Sq :" inft}y ´x} : y P Su. If E Ă R d is a vector space, we denote by P E the d ˆd orthogonal projection matrix onto E. We say that a function f : R d Ñ R is weakly convex if there is ρ ą 0 such that the function gpxq :" f pxq `ρ x 2 is convex. For two sequences pa n q, pb n q, we write a n Á b n if lim inf an bn ą 0. With this notation a n " b n means a n Á b n and b n Á a n . For r ą 0, Bp0, rq denotes the open ball of radius r.

Throughout this chapter, C and C 1 will refer to positive constants that can change from line to line and from one statement to another.

Reinforced projection formula

The following theorem, which we believe to be of independent interest, is the first main result of this chapter. It is an improvement of the projection formula of [Bolte & Pauwels 2019] (see Lemma 2.4.10) when the definable function is locally Lipschitz continuous.

Theorem 5.2.1 (Reinforced projection formula). Let f : R d Ñ R be a definable, locally Lipschitz continuous function. Let p be a positive integer. There is pX i q, a definable Verdier C p stratification of R d , such that for each y P X i and each X j such that X i X X j ‰ H, there is C, δ ą 0, such that for any two points y 1 P Bpy, δq X X i , x P Bpy, δq X X j ,

P T y 1 X i p∇ X j f pxqq ´∇X i f py 1 q ď C x ´y1 , (5.2)
and, moreover, for any x P Bpy, δq X X c i and any v P Bf pxq,

P T y 1 X i pvq ´∇X i f py 1 q ď C x ´y1 . (5.3)
Proof. In this proof C 1 ą 0 will denote some constant that can change from line to line. Consider pS i q and pX i q as in Lemma 2.4.10. We claim that for any index j and x P X j , we have T x,f pxq S j " tph, x∇ X i f pxq, hyq : h P T x X j u. Indeed, consider ph x , h f q P T x,f pxq S j and a C p curve c : p´ε, εq s.t. 9 cp0q " ph x , h f q. Consider a C p function F that agrees with f on X j , then pc x ptq, c f ptqq " pc x ptq, F pc x ptqqq and we have 9 c x p0q " h x and 9 c f p0q " x∇F pxq, h x y " x∇ X j f pxq, h x y. Consider pS 1 i q a Verdier stratification of Graphpf q compatible with pS i q. Then the projection of S 1 i onto its first d coordinates, that we denote X 1 i , is still a submanifold s.t. f is C p on X 1 i . Consider py, f pyqq P S 1 i , S 1 j a neighboring strata and C, δ as in Equation (2.10). Denote by L the Lipschitz constant of f on Bpy, δq and δ 1 " δ L`1 . Then, for every x P Bpy, δ 1 q, we have:

py, f pyqq ´px, f pxqq ď p1 `Lq y ´x ď δ ,
that is to say px, f pxqq P Bppy, f pyqq, δq.

Consider y 1 P X 1 i XBpy, δ 1 q, x P X 1 j XBpy, δ 1 q and h y 1 P T y 1 X 1 i with h y 1 " 1. We have that ph y 1 , x∇ X 1 i f py 1 q, h y 1 yq P T py 1 ,f py 1 qq S 1 i and by the Verdier's condition there is h

x P T x X 1 j s.t. 1 c h ´hy 1 , x∇ X 1 i f py 1 q, h y 1 y ¯´ph x , x∇f X 1 j pxq, h x yq ď CpL `1q x ´y1 , where c h " ph y 1 , x∇ X 1 i f py 1 q, h y 1 yq ď C 1 . Therefore, h y 1 ´ch h x ď C 1 x ´y1 , and
x∇ X 1 j f pxq ´∇X 1 i f py 1 q, h y 1 y ď x∇ X 1 j f pxq, h y 1 ´ch h x y ` c h x∇ X 1 j f pxq, h x y ´x∇ X 1 i f py 1 q, h y 1 y ď C 1 x ´y1 , 122
Chapter 5. SGD escapes active strict saddles which proves the first statement. Now, one can choose C, δ such that Inequality (5.2) holds uniformly on all of the stratas X1 j that are neighboring X 1 i . Consider a sequence x n Ñ x such that px n q lies in the stratas of full dimension (which implies that f is differentiable at x n ) and ∇f px n q Ñ v, for n large enough we will have that x n P Bpy, δq and, therefore,

P T y 1 X i p∇f px n qq ´∇X i f py 1 q ď C x n
´y1 . Hence, passing to the limit, we have that P T y 1 X i pvq ´∇X i f py 1 q ď C y 1 ´x . Since any element of Bf pxq is a convex combination of such v, the second statement is proved.

Active strict saddles

In this section, f : R d Ñ R is supposed to be a locally Lipschitz continuous function.

We recall the definition Z :" tx P R d : 0 P Bf pxqu.

Definition and Existing Results

Let p ě 2 be an integer.

Definition 5.3.1 (Active manifold, [Lewis 2002]). Consider x ˚P Z. A set M Ă R d is called a C p active manifold around x ˚, if there is a neighborhood U of x ˚such that the following holds. i) Smoothness condition: M X U is a C p submanifold and f is C p on M X U .
ii) Sharpness condition:

inft v : v P Bf pxq, x P U X M c u ą 0 .
Definition 5.3.2 (Active strict saddle). We say 1 that a point x ˚P Z is an active strict saddle (of order p) if there exists a C p active manifold M around x ˚, and a vector w P T x ˚M , such that ∇ M f px ˚q " 0 and H f,M px ˚qpwq ă 0.

We say that f satisfies the active strict saddle property (of order p), if it has a finite number of Clarke critical points, and each of these points is either an active strict saddle of order p or a local minimizer .

In the special case of a smooth function f , the space M " R d is trivially an active manifold around any critical point x ˚of f . If x ˚is moreover a trap in the sense provided in the introduction (i.e., the Hessian matrix of f at x ˚admits a negative eigenvalue), then x ˚is trivially an active strict saddle. Hence, the smooth setting can be handled as a special case.

The archetype of an active strict saddle is given by the following example.

Active strict saddles 123

Example 5.3.1. The point p0, 0q is an active strict saddle of the function f : R 2 Ñ R given by f py, zq " ´y2 `|z|. Indeed, Bf ppy, zqq "

$ ' ' & ' ' % tp´2y, 1qu if z ą 0 , tp´2y, ´1qu if z ă 0 , t´2yu ˆr´1, 1s otherwise ,
and the set M " Rˆt0u is a C 2 active manifold. Moreover, ∇ M f ppy, 0qq " p´2y, 0q and H f,M p0qpp1, 0qq " ´2, which proves the statement.

While the definition of an active strict saddle might seem peculiar at first glance, the following proposition of Davis and Drusvyatskiy shows that a generic definable and weakly convex function satisfies a strict saddle property. The proof is grounded in the work of [Drusvyatskiy et al. 2016].

Proposition 5.3.1 ([Davis & Drusvyatskiy 2021, Theorem 2.9]). Assume that f is definable and weakly convex. Define f u pxq :" f pxq ´xu, xy, for every u P R d . Then, for every p ě 2 and for Lebesgue-almost every u P R d , f u has the active strict saddle property of order p.

It is worth noting that the result of [Davis & Drusvyatskiy 2021, Theorem 2.9] is in fact a bit stronger than Proposition 5.3.1, because it states moreover that for almost all u, the cardinality of the set of Clarke critical points of f u is upper bounded by a finite constant which depends only on f .

One can wonder if Proposition 5.3.1 may still hold if f is definable and locally Lipschitz, but not weakly convex. The answer is negative, as shown by the following example.

Example 5.3.2. Let f : R 2 Ñ R be defined as f py, zq " ´|y| `|z|. Then for any u P Bp0, 1q, p0, 0q is a critical point for f u , but is neither a local minimum nor an active strict saddle.

Verdier and Angle Conditions

On the top of the items i-ii) of Definition 5.3.1, we introduce the following useful conditions.

Definition 5.3.3. Let M be a C 1 active manifold around some x ˚P Z. We say that M satisfies the Verdier condition and the angle condition, if the following conditions hold respectively.

iii) Verdier condition. There is a neighborhood U of x ˚and C ě 0, such that for every y P M X U and every x P U , P TyM pvq ´∇M f pyq ď C x ´y , @v P Bf pxq .

iv) Angle condition. For every α ą 0, there is β ą 0 and a neighborhood U of x ˚, such that for every x P U , f pxq´f pP M pxqq ě α x ´PM pxq ùñ xv, x´P M pxqy ě β x ´PM pxq , @v P Bf pxq .

Definition 5.3.4. An active strict saddle x ˚is said to satisfy the Verdier and angle conditions, if the active manifold M in Definition 5.3.2 satisfies the Verdier and angle conditions. The function f is said to satisfy the active strict saddle property of order p with the Verdier and angle conditions, if it satisfies the active strict saddle property of order p and if every active strict saddle satisfies the Verdier and angle conditions.

The Verdier condition merely states that M is one of the stratas of the Verdier stratification of Theorem 5.2.1 The purpose of the angle condition is to relate, close to M , the linear growth of the function f and the lower boundedness of the inner product between the subgradients of f at x and the normal direction to M . The latter will allow us to prove that the iterates of SGD converge to M fast enough.

Remark 20. Let M be an active manifold around x ˚. As it will be clear from the proof of Theorem 5.3.2, when f is weakly convex, M always satisfies the angle condition. Otherwise stated, the angle condition is simply true in case of weakly convex functions. However, as the following example shows, one is able to find many natural examples of functions which are not weakly convex, and yet satisfy this condition.

Example 5.3.3. The function f : R2 Ñ R given by f py, zq " ´y2 ´|z| is not weakly convex. Its unique Clarke critical point p0, 0q is an active strict saddle, satisfying the Verdier and the angle conditions.

Example 5.3.3 shows that the Verdier and angle conditions can be satisfied with no need for f to be weakly convex. Nevertheless, more can be said when this assumption holds. The following theorem strengthen the genericity result of Proposition 5.3.1 by establishing that the active strict saddle property with the Verdier and angle conditions is satisfied by a generic definable and weakly convex function. We recall the notation f u pxq " f pxq ´xu, xy.

Theorem 5.3.2. Assume that f : R d Ñ R is a definable, weakly convex function. For every p ě 2, and for Lebesgue-almost every u P R d , f u satisfies the active strict saddle property of order p with the Verdier and angle conditions.

Proof. Let tX 1 , . . . , X k u be the C p Verdier stratification from Theorem 5.2.1. Upon noticing that in the proof of [Drusvyatskiy et al. 2016, Corollary 4.8 and Theorem 4.16] the active manifold 2 can be chosen adapted to tX 1 , . . . , X k u, the existence of an active manifold with a Verdier condition follows from [Davis & Drusvyatskiy 2021, 5.4. Avoidance of Active Strict Saddles 125 Theorem 2.9, Appendix A]. For the angle condition note that by weak convexity of f there is ρ ě 0 such that: f pP M pxqq ´f pxq ě xv, P M pxq ´xy ´ρ x ´PM pxq 2 @v P Bf pxq .

Therefore, if f pxq ě f pP M pxqq `α P M pxq ´x , then:

@v P Bf pxq, xv, x ´PM pxqy ě α x ´PM pxq ´ρ x ´PM pxq 2 .
Taking U a neighborhood of x ˚close enough to zero, we see that the angle condition is satisfied.

Avoidance of Active Strict Saddles

Let f : R d Ñ R be a locally Lipschitz continuous function. On a probability space pΩ, A, Pq, consider a random variable x 0 and random sequences pv n q, pη n q on R d . Define the iterates:

x n`1 " x n ´γn v n `γn η n`1 , (5.4)

where pγ n q is a deterministic sequence of positive numbers. Let pF n q be a filtration on pΩ, A, Pq.

Assumption 5.4.1.

i) The function f is path differentiable.

ii) For every n, v n P Bf px n q.

iii) The sequences pv n q, pη n q are adapted to pF n q, and x 0 is F 0 -measurable.

iv) There are constants c 1 , c 2 ą 0 and α P p1{2, 1s s.t. for all n P N:

c 1 n α ď γ n ď c 2 n α .
Consider a point x ˚P Z.

Assumption 5.4.2. The point x ˚is an active strict saddle of order 4 satisfying the Verdier and angle conditions.

Since H f,M px ˚q is a quadratic form we can write down R d " E ´'E `, where E

(respectively E `) is the vector space spanned by the eigenvectors of the associated symmetric bilinear form that have negative (respectively nonnegative eigenvalues). Note that by results of Section 2.3 we have that E ´Ă T x ˚M and by Assumption 5.4.2 we have that dim E ´ě 1.

Assumption 5.4.3. The following holds almost surely on the event rx n Ñ x ˚s.

i) Erη n`1 |F n s " 0, for all n.

ii) lim sup Er η n`1 4 |F n s ă `8.

iii) Denote η ń`1 the projection of η n`1 onto E ´. We have:

lim inf Er η ń`1 |F n s ą 0
The following theorem is the main result of this chapter.

Theorem 5.4.1. Let Assumptions 5.4.1-5.4.3 hold. Then Ppx n Ñ x ˚q " 0.

Combining Theorem 5.4.1 with the results of Section 5.3.2 we obtain that, under appropriate assumptions, the SGD on a generic definable, weakly convex function converges to a local minimizer. We state this result in the following corollary.

Corollary 5.4.2. Let Assumptions 5.4.1 and 5.4.2 hold. Assume that f has the active strict saddle property of order 4 with the Verdier and angle conditions. Moreover, assume that almost surely the following holds. i) Erη n`1 |F n s " 0, for all n.

ii) For every C ą 0, lim sup Er η n`1 4 |F n s1 xn ďC ă `8 .

iii) For all w P R d zt0u, lim inf Er|xw, η n`1 y| |F n s ą 0 .

Then, almost surely, the sequence px n q is either unbounded, or converges to a local minimizer of f .

Proof of Theorem 5.4.1

From now on, we assume without restriction that x ˚" 0. Thus, ∇ M f p0q " 0, and there exists a vector w P T 0 M such that H f,M p0qpwq ă 0.

The general idea of the proof of Theorem 5.4.1 is that on the event rx n Ñ 0s, the function P M is defined for all large n, enabling us to write x n " y n `zn for these n, where y n " P M px n q. The iterates py n q can then be written under the form of a standard smooth Robbins-Monro algorithm for which the trap avoidance can be established by the technique of Brandière and Duflo [Brandière & Duflo 1996]. In this setting, the remainders z n will be shown to be small enough so as not to alter fundamentally the approach of [Brandière & Duflo 1996].

Let us provide more details on our proof. We first show that on rx n Ñ 0s, there is an integer n 0 such that for all all n ě n 0 , the norms x n are small, and moreover, @v P Bf px n q, xv, z n y Á z n .

(5.5)

ÿ i"n γ i E z i " Opχ n q
where χ n :"

`8 ÿ i"n γ 2 i .
This is where Inequality (5.5) comes into play to establish this bound. So far, we have assumed abusively that the moment n 0 after which x n is small and (5.5) is satisfied is deterministic. To deal with this issue, in Section 5.5.2, on an arbitrary large event A, we construct a sequence py n q that is (for n large enough) equal to pP M px n qq on AXrx n Ñ 0s and satisfies an equation of the form (5.6) almost surely. Proposition 5.5.1 will allow us to prove that Pprx n Ñ 0s X Aq ď Ppry n Ñ 0sq " 0 and since the event A is arbitrary large, this will prove Theorem 5.4.1.

Preliminary: Avoidance of Traps in the Smooth Case

The following proposition is nearly a quote of Brandière and Duflo's theorem [Brandière & Duflo 1996, Theorem 1]. As discussed below, we alleviate some hypotheses of [Brandière & Duflo 1996].

To state this proposition recall that, by a standard result from linear algebra, for a matrix H P R dˆd , there is a decomposition R d " Λ `' Λ ´such that Λ `, Λ áre stable by H and the eigenvalues of H |Λ ´(respectively H |Λ `) have eigenvalues with negative (respectively nonpositive) real parts. Recall that for a smooth map D : R d Ñ R d , we denote J D its jacobian and that χ n :" ř 8 i"n γ 2 i .

Proposition 5.5.1. Let pΩ, A, Pq be a probability space, pF n q a filtration and pγ n q a sequence of deterministic nonnegative step sizes such that ř k γ k " `8 and ř k γ 2 k ă `8. Let d be an integer and D : R d Ñ R d be such that Dp0q " 0 and there is a neighborhood of 0 such that on it D is continuously differentiable, with Lipschitz continuous Jacobian. Consider the R d -valued stochastic process py n q given by y n`1 " y n ´γn Dpy n q `γn ηn`1 `γn n`1 `γn ˜ n`1 ,

(5.7)

where y 0 is F 0 -measurable and the sequences pη n q, p n q and p˜ n q are pF n q-adapted.

Assume that Λ ´, the vector space associated to the eigenvectors of J D p0q that have negative real parts, is of positive dimension. Denote ηń `1 the projection of ηn`1 on Λ ´and assume that on the event ry n Ñ 0s the following almost surely holds.

i) For all n, Erη n`1 |F n s " 0.

ii) lim sup E " ηn`1 4 ˇˇF n s ă `8. iii) lim inf E " ηń `1 |F n ‰ ą 0 . iv) ř `8 k"0 k`1 2 ă `8 . v)
We have that:

E « 1 rynÑ0s `8 ÿ i"n γ i ˜ i`1 ff " Opχ n q.
Then Ppry n Ñ 0sq " 0.

Proposition 5.5.1 is similar to [Brandière & Duflo 1996, Theorem 1], except for the presence of the sequence p˜ n q. As the proof is mainly an adaptation of the proof of [Brandière & Duflo 1996, Theorem 1], we provide a sketch of proof in the appendix.

Application to Algorithm (5.4)

To apply the results of the preceding section we need, first, to find a candidate for D, this is the purpose of the next lemma. Its proof readily follows from results of Section 2.3. Lemma 5.5.2. Let Assumption 5.4.2 hold and let r ą 0 be such that P M : Bp0, rq Ñ M is well defined and is C 3 and that there is a C 4 function F : Bp0, rq Ñ R that agrees with f on M X Bp0, rq. Then, the function F ˝PM is C 3 on Bp0, rq and for y P M X Bp0, rq, we have:

∇pF ˝PM qpyq " ∇ M f pyq .
Moreover, for w P R d :

H f,M p0qpwq " w T ∇ 2 pF ˝PM qw .
By Tietze's extension theorem the function ∇pF ˝PM q : Bp0, rq Ñ R d can be extended to a bounded continuous function D : R d Ñ R d that we shall use in the remainder of the chapter.

For r ą 0 such that P M is well defined on Bp0, rq, and for C ą 0, denote V r pCq " tx P Bp0, rq : @v P Bf pxq, xv, x ´PM pxqy ě C x ´PM pxq u .

The next proposition is a key element in our proof. To not interrupt our exposition its proof is provided in Section 5.5.3.

Proposition 5.5.3. Let Assumptions 5.4.1-5.4.3 hold. There is β, r 1 ą 0, such that for every r ă r 1 , almost surely on the event rx n Ñ 0s, x n P V r pβq for all n large enough.

In the remainder, we fix β, r 1 ą 0 as those provided by the previous proposition. We let U be the neighborhood around zero that verify conditions of Definition 5.3.3. In the following, we choose r ď r 1 such that P M is C 3 on Bp0, rq, and Bp0, rq Ă U . The value of r, while always satisfying these requirements, will be adjusted in the course of the proof.

Firstly, to reduce technical issues, we notice that as in [Brandière & Duflo 1996, Section I.2] to prove Theorem 5.4.1 we can actually replace Assumption 5.4.3 by the following, more easy to handle, assumption. The notation E n r¨s stands for Er¨|F n s.

Assumption 5.5.1. Almost surely, the sequence pη n q is such that E n rη n`1 s " 0 and there is A, B ą 0 such that for all n P N, we have:

E n r η n`1 4 s ď B and E n r η ń`1 s ě A .
Given an integer N ě 0, we define the probability event A N " r@n ě N, x n P V r pβqs .

Note that the sequence of events pA N q is increasing for the inclusion. Furthermore, Proposition 5.5.3 shows that

rx n Ñ 0s Ă 8 ď N "0 A N " lim N Ñ8 A N .
Using Lemma 2.3.2 in conjunction with the Verdier condition (iii)) of Definition 5.3.3, we also have

J P M py N n qv n 1 τ N ąn`1 " P T y N n M pv n q1 τ N ąn`1 " ∇ M f py N n q1 τ N ąn`1 `ζ N n`1 " Dpy N n q1 τ N ąn`1 `ζ N n`1 ,
where ζN n`1 is F n`1 -measurable, and satisfies

ζN n`1 ď C x n ´yN n 1 τ N ąn`1 ď C z N n .
Gathering these expressions, we get

y N n`1 " y N n ´γn Dpy N n q `γn ηN n`1 `γn n`1 `γn ˜ n`1 ,
where ηN n`1 " `1τ N ąn J P M py N n q `1τ N ďn ˘ηn`1 , and (5.10)

˜ N n`1 " ζ N n`1 `ζ N n .
The assertions i)) and ii)) of the statement are obtained from what precedes. The noise ηN n is obviously F n -measurable. Moreover, E n ηN n`1 " 0 since 1 τ N ąn J P M py N n q1 τ N ďn is F n -measurable. The last bound in iii)) follows from Assumption 5.5.1.

Assertion iv)) follows from Lemma 5.5.2. To establish v)), we write

pη N n`1 q ´ " P E ´JP M py N n qη n`1 1 τ N ąn ` P E ´ηn`1 1 τ N ďn ě P E ´ηn`1 ´ P E ´JP M py N n qη n`1 ´PE ´ηn`1 1 τ N ąn .
On the event ry N n Ñ n 0s, it holds that J P M py N n q Ñ n J 0 . By Lemma 2.3.2, J 0 is the orthogonal projection on T 0 M , thus, lim y N n Ñn0 P E ´JP M py N n q " P E ´. Consequently, we obtain on the event ry N n Ñ n 0s:

lim inf n E n pη N n`1 q ´ ě lim inf n E n η ń`1 ´lim sup n ` P E ´JP M py N n q ´PE ´ E n η n`1 ě lim inf n E n η ń`1 ą 0 ,
and by Assumption 5.5.1. Proposition 5.5.4 is proven.

Proposition 5.5.6. Let Assumptions 5.4.1-5.4.2 and 5.5.1 hold true. Then, there is C ą 0 such that

E n z N n`1 2 ď z N n 2 ´γn ˆ2β r ´C˙ z N n 2 `Cγ 2 n , and
E n z N n`1 2 ď z N n 2 ´γn p2β ´Crq z N n `Cγ 2 n .
Proof. We shall use the notation

p N n " x n ´yN n ,
which enables us to write z N n " p N n 1 năτ N .

We start with the development

z N n`1 2 " p N n`1 2 1 n`1ăτ N ď p N n`1 2 1 năτ N " p N n`1 ´pN n `pN n 2 1 năτ N " z N n 2 `2xx n`1 ´xn , z N n y ´2xy N n`1 ´yN n , z N n y ` p N n`1 ´pN n 2 1 năτ N .
(5.11)

We now deal separately with each of the three rightmost terms in the last expression. We first show that

E n |xy N n`1 ´yN n , z N n y| ď Cγ n z N n 2 `Cγ 2 n .
(5.12) By Proposition 5.5.4,

xy N n`1 ´yN n , z N n y " γ n x´Dpy N n q `η N n`1 ` N n`1 `˜ N n`1 , z N n y.
We have xDpy N n q, z N n y " x∇ M f py N n q, z N n y " 0 since ∇ M f py N n q P T y N n M . Furthermore, we get from Equation (5.10) that

1 năτ N ηN n`1 " 1 năτ N J P M py N n qη n`1 " 1 năτ N P T y N n M pη n`1 q
by Lemma 2.3.2, thus, xη N n`1 , z N n y " 0. As a consequence,

|xy N n`1 ´yN n , z N n y| ď γ n p z N n 2 ` N n`1 `˜ N n`1 2 q ď γ n z N n 2 `2γ n p N n`1 2 ` ˜ N n`1 2 q.
From Proposition 5.5.4 again, we have

E n N n`1 2 ď Cγ n E n p1 ` η n`1 2 q1 τ N ąn`1 ď Cγ n E n p1 ` η n`1 2 q ď Cγ n ,
and

E n ˜ N n`1 2 ď C z N n 2 p1 `En η n`1 2 q ď C z N n 2 .
Inequality (5.12) is obtained by combining these inequalities.

We next show succinctly that

E n p N n`1 ´pN n 2 1 năτ N ď Cγ 2 n .
(5.13) Indeed,

p N n`1 ´pN n 2 1 năτ N " x n`1 ´xn ´py N n`1 ´yN n q 2 1 năτ N ď Cγ 2 n ´ v n 2 ` η n`1 2 ` Dpy N n q 2 ` ηN n`1 2 ` N n`1 2 ` ˜ N n`1 2 ¯1năτ N ,
and the result follows by standard calculations making use of the results of Proposition 5.5.4. We finally deal with the term xx n`1 ´xn , z N n y. Since E n η n`1 " 0, we have E n xx n`1 ´xn , z N n y " ´γn xv n , z N n y. Observing that x n P V r pβq when z N n ‰ 0, we obtain from the very definition of the set V r pβq that E n xx n`1 ´xn , z N n y ď ´γn β z N n . Getting back to Inequality (5.11), and using this result in conjunction with the inequalities (5.12) and (5.13), we obtain that

E n z N n`1 2 ď z N n 2 `Cγ n z N n 2 ´2γ n β z N
n `Cγ 2 n . Since x n P Bp0, rq on the event rn ă τ N s, it holds that z N n ď r and thus, z N n 2 ď r z N n . This leads at once to the inequalities in the statement of the proposition.

Corollary 5.5.7. Under the assumptions of the previous proposition, there is C ą 0 such that

8 ÿ i"n γ i E z N i ď Cχ n for n ě N .
The proof of this corollary makes use of a technical result which is attributed to [Chung 1954]. Its proof can be found in, e.g., [Bravo et al. 2018]: Lemma 5.5.8 (Lemma D.2 in [Bravo et al. 2018]). Let pa n q be a nonnegative sequence such that for all n large enough,

a n`1 ď a n ˆ1 ´P n p ˙`Q n p`q ,
where p P p0, 1s, q ą 0, and P, Q ą 0. It is further assumed that P ą q if p " 1.

Then, there exists C ą 0 such that a n ď C n q . Proof of Corollary 5.5.7. Let C ą 0 be the constant provided in the statement of Proposition 5.5.6. Choose r ą 0 small enough so that 2βr ´1 ´C ą 0. Replacing γ n in this statement with the bounds on this step size provided by Assumption 5.4.1-(iv)), we get from the first inequality in Proposition 5.5.6

E z N n`1 2 ď ˆ1 ´c1 n α ˆ2β r ´C˙˙E z N n 2 `c2 C n 2α .
We apply the previous lemma with a n " E z N n 2 , after adjusting r ą 0 when needed in order that all the conditions in the statement of this lemma are satisfied. We get that there exists a constant C 1 ą 0 such that Let k ą 0 be an integer. Telescoping the second inequality stated by Proposition 5.5.6 from n `k back to n, we get

E z N n 2 ď C 1 n α .
E z N n`k 2 ď E z N n 2 ´p2β ´Crq n`k´1 ÿ i"n γ i E z N i `C n`k´1 ÿ i"n γ 2 i ,
which implies that

p2β ´Crq n`k´1 ÿ i"n γ i E z N i ď E z N n 2 `C n`k´1 ÿ i"n γ 2 i ď C 1 n α `C n`k´1 ÿ i"n γ 2 i .
Making k Ñ 8, we obtain that p2β ´Crq

8 ÿ i"n γ i E z N i ď C 1 n α `Cχ n .
To complete the proof, it remains to notice that since γ n " n ´α with α P p1{2, 1s, it holds that χ n " n 1´2α Á n ´α.

Theorem 5.4.1: end of the proof. We now have all the elements to establish the identity (5.9), proving Theorem 5.4.1. For this, notice that,for every N ě 0, by Proposition 5.5.4, y N n satisfies an equation of the form Equation (5.7). The assumption of Proposition 5.5.1 on the sequence pη n q are satisfied by Proposition 5.5.4 and the assumptions on the sequences p n q, p˜ n q follow from Assumption 5.5.1 and Corollary 5.5.7.

Hence, applying Proposition 5.5.1, we obtain that Ppry N n Ñ 0sq " 0, for all N ě 0. As previously explained, the latter implies that Pprx n Ñ 0sq " 0.

To complete the proof of Theorem 5.4.1 it remains to prove Proposition 5.5.3, which is the purpose of the next section.

Proof of Proposition 5.5.3

The standard way to analyze the convergence of the SGD to the set of Clarke critical points is by studying its continuous counterpart -the subgradient flow:

9 xptq P ´Bf pxptqq .
(5.14)

We say that an absolutely continuous curve x : R `Ñ R is a solution of the differential inclusion (DI) (5.14) starting at x P R d if xp0q " x and if for almost every t P R `, the inclusion (5.14) is verified. We denote S ´Bf pxq the set of these solutions.

The idea of the proof of Proposition 5.5.3 goes as follows. For each initial point x P Bp0, r 0 q with r 0 ą 0 small enough, either all the trajectories of (5.14) issued from x leave Bp0, r 0 q in a fixed time horizon, or f pxq ´f pP M pxqq ě α x ´PM pxq . This will be the content of the next lemma. Next, we use the well-known fact that the interpolated process constructed from our iterates px n q is a so-called Asymptotic Pseudo Trajectory (APT) of the DI (5.14), as formalized in [Benaïm et al. 2005] (see also, e.g., [START_REF] Duchi | [END_REF], Schechtman 2021a]). The consequence is that on the event rx n Ñ 0s, necessarily f px n q ´f pP M px n qq ě α x n ´PM px n q after a certain finite moment. To complete the proof, it remains to make use of the angle condition (iv)) of Definition 5.3.3. Lemma 5.5.9. Let f : R d Ñ R a locally Lipschitz continuous, path differentiable function. Let M be a C 2 active manifold for f such that 0 P M , f p0q " 0, and ∇ M f p0q " 0. Then, there is α, T ą 0 and r 0 ą 0 s.t. for every x P S ´BF pxq, with x P Bp0, r 0 q, either xpr0, T sq Ć Bp0, r 0 q or f pxq ´f pP M pxqq ě α x ´PM pxq .

Proof. Let r ą 0 be such that Bp0, rq Ă U , where U is the neighborhood from Definition 5.3.1. Since f is C 2 on M XBp0, rq and ∇ M f p0q " 0, there is some constant C s.t. we have sup xPBp0,rq ∇ M f pP M pxqq ď C P M pxq . Denote L the Lipschitz constant of f on Bp0, rq and let c m be such that inft v : v P Bf pxq, x P Bp0, rqXM c u ě c m . Fix r 0 ď minp c 2 m 2LC , rq and consider x P Bp0, r 0 q and x P S ´BF pxq. Denote t 1 " inftt : xptq P M or xptq R Bp0, r 0 qu. Since f is path differentiable, we have:

inf x 1 PBp0,r 0 q f px 1 q ď f pxptqq " f pxq´ż t 0 9 xpuq du ď f pxq´c 2 m t 1 ď sup x 1 PBp0,r 0 q f px 1 q´c 2 m t 1 .
Hence, if we choose T s.t. c 2 m T ą 2 sup x 1 PBp0,r 0 q |f px 1 q|, we have t 1 ď T and either xpt 1 q R Bp0, rq or xpt 1 q P M . Assume that xpt 1 q P M and denote yptq " P M pxptqq and zptq " xptq ´yptq. Notice that for almost every t ě 0, we have 9 yptq " P T yptq 9 xptq ď L. Moreover, by path-differentiability of f we have:

|f pypt 1 qq ´f pyp0qq| ď

ż t 1 0 |x∇ M f pypuqq, 9 ypuqy| du ď ż t 1 0 ∇ M f pypuqq 9 ypuq du ď C ż t 1 0 ypuq 9 ypuq du ď LCr 0 t 1 ď 1 2 c 2 m t 1 .
Where the first inequality comes from the fact that f is path differentiable and that for all u P r0, T s, 9 ypuq P T ypuq M . Denote α " c 2 m 4L and assume by contradiction that f pxq ´f pP M pxqq ď α x ´PM pxq . We have:

0 " f pxpt 1 qq ´f pypt 1 qq ď f pxq ´c2 m t 1 ´f pypt 1 qq ď f pxq ´f pyp0qq `c2 m 2 t 1 ´c2 m t 1 ď α x ´PM pxq ´c2 m 2 t 1 .
Which implies that x ´PM pxq ě c 2 m 2α t 1 ě 2Lt 1 . On the other hand, we have that zptq " distpxptq, M q. Since the distance function is 1-Lipschitz, we have for almost every t ě 0: ˇˇˇd dt zptq ˇˇˇď 9 xptq ď L .

Therefore, 0 " zpt 1 q ě zp0q ´Lt 1 " x ´PM pxq ´Lt 1 , which implies that x ´PM pxq ď Lt 1 , a contradiction.

Let X : R `Ñ R d be the linearly interpolated process defined as:

Xptq " x n `t ´řn i"0 γ i γ n`1 px n`1 ´xn q, if t P rτ n , τ n`1 q , where τ n " ř n i"0 γ i . It is well known that under our assumptions, on the event rx n Ñ 0s, X is an APT for the DI (5.14), as shown in [Benaïm et al. 2005, Duchi & Consider α, T and r 0 from Lemma 5.5.9. On the event rx n Ñ 0s let x n P S ´BF px n q be such that sup hPr0,T s

Xpτ n `hq ´xn phq ÝÝÝÝÑ nÑ`8 0 .

Consider r 1 ď r 0 such that Bp0, r 1 q Ă U , where U is the neighborhood associated to α by the angle condition. If for n large enough, x n pr0, T sq remains in Bp0, r 1 q, then by Lemma 5.5.9 we have:

f px n q ě α x n ´PM px n q `f pP M px n qq , which, by the angle condition, implies that there is β ą 0 xv n , x n ´PM px n qy ě β x n ´PM px n q .

(5.15)

Otherwise, on the event rx n Ñ 0s, there is h n P r0, T s such that after an extraction Xpτ n `hn q Ñ x, with x R Bp0, r 1 q. Since the limit points of X are the accumulation points of the sequence px n q, this contradicts the fact that x n Ñ 0.

5.6 Sketch of proof of Proposition 5.5.1

We recall that E n r¨s denotes Er¨|F n s. Denote d ´the dimension of Λ ´. Using the center-stable manifold theorem, the authors of [Brandière & Duflo 1996, Page 407-409] construct a sequence pw n q 3 in R d ´such that w n " w n `γn H n w n `γn pr n`1 `r1 n`1 `en`1 q , where the sequences pw n q, pr n q, pr 1 n q, pe n q are adapted to pF n q and we have the inclusion ry n Ñ 0s Ă rw n Ñ 0s. Moreover, on the event ry n Ñ 0s, the following almost surely holds. i) There is H an invertible matrix such that all of the real parts of its eigenvalues are positive and H n Ñ H .

ii) The sequence pe n q is such that E n re n`1 s " 0 and 0 ă lim inf E n r e n`1 2 s ď lim sup E n r e n`1 2 s ă `8 .

iii) The sequence pr n q is such that ř `8 i"0 r i`1 2 ă `8.

iv) The sequence pr 1 n q is such that Er1 Γ ř `8 i"n γ i r 1 i`1 s " Opχ n q.

The only difference with [Brandière & Duflo 1996] is in the presence of pr 1 n`1 q and the point (iv)).

Using this representation, the avoidance of traps result follows from the following proposition. The only difference with [Brandière & Duflo 1996, Proposition 4] is, once again, in the presence of the sequence pr 1 n q.

Proposition 5.6.1 ( [Brandière & Duflo 1996, Proposition 4]). Let d be an integer, pΩ, A, Pq be a probability space, pF n q a filtration on it and pw n q be a sequence in R d verifying:

w n`1 " w n `γn H n `γn pr n`1 `r1 n`1 `en`1 q , (5.16)

where the sequences pw n q, pH n q, pr n q, pr 1 n qpe n q are adapted to pF n q and pγ n q is a sequence of positive stepsizes s.t.

ř `8 i"0 γ i " `8 and ř `8 i"0 γ 2 i ă `8. Assume that on an event Γ P A we have the following.

i) The sequence pγ n q is such that ř `8 i"0 γ i " `8 and ř `8 i"0 γ 2 i ă `8.

ii) The sequence pe n q is such that E n re n`1 s " 0 and 0 ă lim inf E n r e n`1 s ď lim sup E n r e n`1 2 s 1{2 ă `8 .

iii) The sequence pr n q is such that ř `8 i"0 r i`1 2 ă `8.

iv) The sequence pr 1 n q is such that

E " 1 Γ ř `8 i"n γ i r 1 i`1 ‰ " Opχ n q.
Let H P R dˆd be a matrix such that all of the real parts of its eigenvalues are positive. Then, denoting Υ " Γ X rw n Ñ 0s X rH n Ñ Hs, we have P pΥq " 0.

Proof. In this proof C will denote some absolute constant that can change from line to line. The proof closely follows the one of [Brandière & Duflo 1996, Proposition 4]. As in [Brandière & Duflo 1996] it is sufficient to prove the proposition in the case where there A, B, K ą 0 such that almost surely E n re n`1 s " 0, A ď E n r e n`1 s ď E n r e n`1 2 s 1{2 ď B and ř `8 i"0 r i`1 2 ď K.

We can rewrite Equation (5.16) as:

w n`1 " w n `γn Hw n `γn ∆ n w n `γn pe n`1 `rn`1 `r1 n`1 q , where ∆ n " H n ´H. Let Q be a positive definite symmetric matrix such that QH `HT Q " 2I, where I P R dˆd is the identity matrix. Denote U n " pw T n Qw n q 1{2 . Following the same calculations as in [Brandière & Duflo 1996], we obtain that:

pU n`1 ´Un q ě 1 U n w T n`1 Qw n ě γ n U n ´ w n 2 `wT n Q∆ n w n `wT n Qpe n`1 `rn`1 `r1 n`1 q ě γ n w n ˜1 λ 1{2 max ´ Q∆ n λ 1{2 min ¸`γ n w T n Qpe n`1 `rn`1 `r1 n`1 q U n ,
where λ max , λ min are respectively the maximal and the minimal eigenvalue of Q.

The event Υ is included in a union of events Υ p defined as:

Υ p " ΥX « @n ě p, 1 λ 1{2 max ´ Q∆ n λ 1{2 min ě 1 2λ 1{2 max ff X " sup něp w n ď 1  X « `8 ÿ i"p γ i r 1 i`1 ă 1 ff .
Therefore, on Υ p , there is C ą 0 such that for M ě n ě p, we have:

M ÿ i"n γ i w i ď CU M `1 `C M ÿ i"n γ i w T i Qpe i`1 `ri`1 `r1 i`1 q U i .
Hence,

ˇˇˇˇM ÿ

i"n

γ i w i ˇˇˇˇ2 ď C U M `1 2 `C M ÿ i"n γ i w T i Qe i`1 U i 2 `C ˜`8 ÿ i"n γ 2 i ¸˜`8 ÿ i"n r i`1 2 ¸`C ˇˇˇˇ`8 ÿ i"n γ i r 1 i`1 ˇˇˇˇ2 ď C U M `1 2 `C sup M ěp M ÿ i"n γ i w T i Qe i`1 U i 2 `Cχ n `C ˇˇˇˇ`8 ÿ i"n γ i r 1 i`1 ˇˇˇˇ2 ,
where we used the fact that

w T n Q Un is bounded. On Υ p we have that Er U M `1 2 s Ñ 0. The sequence p ř M i"n γ i w T i Qe i`1 U i
q M ěn is a square summable martingale difference sequence. Therefore, by Doob's maximal inequality:

E » -1 Γ sup M PN ˇˇˇˇM ÿ i"n γ i w T i Qe i`1 U i ˇˇˇˇ2 fi fl ď CE « `8 ÿ i"n γ 2 i e i`1 2 ff ď Cχ n .
Finally, on Υ p we have ř `8 i"n γ i r 1 i`1 ă 1. Therefore, by assumptions:

E » -1 Υp ˇˇˇˇ`8 ÿ i"n γ i r 1 i`1 ˇˇˇˇ2 fi fl ď E « 1 Υ `8 ÿ i"n γ i r 1 i`1 ff ď Cχ n
Hence, there is C ą 0 such that:

E » -1 Υp ˇˇˇˇ`8 ÿ i"n γ i w i ˇˇˇˇ2 fi fl ď Cχ n .
(5.17)

On the other hand, following the calculations of [Brandière & Duflo 1996], on Υ p we have: (5.18) where we denote R n " ∆ n w n and for n ě p:

´wp " `8 ÿ i"p `R1 i `γi pe i`1 `ri`1 `r1 i`1 q ˘,
R 1 n " γ n R n ´pB ´1 n´1 ´B´1 n qS n , S n " `8 ÿ i"n γ i pR i `ei`1 `ri`1 `r1 i`1 q , B n " n ź i"p
p1 `γi Hq .

The idea of the remaining part of the proof is to apply [Brandière & Duflo 1996, Theorem A] to obtain that the left hand side of Equation 5.18 can be F p -measurable only with probability 0. The latter will imply PpΥ p q " 0 and since Υ " Ť pPN Υ p , the proof will be finished. As in the proof [Brandière & Duflo 1996], one of the assumptions of [Brandière & Duflo 1996, Theorem A], to obtain the remaining part it suffices to have:

E « 1 Υp `8 ÿ i"n R 1 i `γi r 1 i`1 ff " op ? χ n q , (5.19)
where the difference with the proof of [Brandière & Duflo 1996, Proposition 4] is in the presence of the term r 1 i`1 . To prove Equation (5.19) we write down:

E « 1 Υp `8 ÿ i"n R 1 i `γi r 1 i`1 ff ď CE « 1 Υp sup iěn ∆ i `8 ÿ i"n γ i w i ff `CE « 1 Υp `8 ÿ i"n B ´1 i´1 ´B´1 i S i ff `E « 1 Υp `8 ÿ i"n γ i r 1 i`1
ff By Inequality (5.17) we have:

E « 1 Υp sup iěn ∆ i `8 ÿ i"n γ i w i ff ď CEr1 Υp sup iěn ∆ i 2 s 1{2 E » -ˇˇˇˇ`8 ÿ i"n γ i w i ˇˇˇˇ2 fi fl 1{2 ď CEr1 Υp sup iěn ∆ i 2 s 1{2 ? χ n ď opχ n q .
(5.20)

As noticed in [Brandière & Duflo 1996] we have (5.21) and by assumptions

ř `8 i"1 B ´1 i´1 ´B´1 i ă `8. There- fore, E " 1 Υp S i ‰ `8 ÿ i"n B ´1 i´1 ´B´1 i ď C ? χ n `8 ÿ i"n B ´1 i´1 ´B´1 i " op ? χ n q ,
E « 1 Υp `8 ÿ i"n γ i r 1 i`1
ff ď Cχ n " op ? χ n q .

(5.22)

Combining (5.20), (5.21) and (5.22) we obtain Equation (5.19). Hence, we can apply [Brandière & Duflo 1996, Theorem A] to obtain that PpΥ p q " 0. Since Υ " Ť pPN Υ p , the proof is finished.

Chapter 6

Stochastic subgradient descent on a generic definable function converges to a minimizer

Introduction

Design and analysis of optimization algorithms are usually relying on some kind of optimality conditions. Canonical examples of such conditions are the second order sufficiency in nonlinear programming [START_REF] Nocedal | [END_REF]] and strict complementarity in semidefinite programming [START_REF] Alizadeh | [END_REF]. While a specific optimization problem might not verify such conditions, a standard way to justify their ubiquity is that they are in some mathematical sense generic. Formally, given a class of optimization problems pQ u q that is parametrized by a set of vectors u P R d , we say that a condition is generic within this class if it is satisfied for the problem Q u , for almost every u P R d . Analysis of such a kind dates back at least to the works of [START_REF] Simon | Generic properties of the complementarity problem[END_REF] and Spingarn and Rockafellar [Spingarn & Rockafellar 1979]. In the latter pQ u q are the linear perturbations of some specific nonlinear programming problem Q and it is showed that for almost every u P R d , the second order sufficiency conditions are indeed necessary in Q u .

In the present chapter, in the spirit of [Spingarn & Rockafellar 1979], given a locally Lipschitz continuous function f : R d Ñ R that is not necessarily smooth nor convex, we analyze the following class of problems:

min xPR d f u pxq , (Q u )
where for u, x P R d , we denote f u pxq :" f pxq ´xu, xy. In this case, the first order necessary condition for x to be a solution to (Q u ) is that 0 P Bf u pxq, where Bf u pxq is the set of Clarke subgradients of f u at x. Hence, we are interested in the generic properties of the following class of sets:

tx P R d : 0 P Bf u pxqu , (Z u )
where for each u P R d , Z u (respectively Z) denotes the set of Clarke critical points of f u (respectively f ).

We are specifically interested in the question of genericity from the perspective of the simplest first order algorithm -the stochastic subgradient descent (SGD):

x n`1 P x n ´γn Bf px n q `γn η n`1 , (6.1)

where pγ n q is a sequence of positive stepsizes and pη n q is some perturbation sequence which presence is typically due to a partial knowledge of Bf by the designer. It is known (see [Davis & Drusvyatskiy 2021, Majewski et al. 2018]) that, under mild conditions on the sequence pη n q and the function f , the iterates of the SGD converge to Z. While the set Z contains local minima it also contains all kinds of spurious points (e.g. local maxima and saddle points) convergence to which might be highly undesirable. We are thus interested in describing a generic set of conditions that ensures the convergence of the SGD to a local minimum.

The first important remark that we should make here is that, in the pursuit of this path, we must distinguish between the conditions that characterize a generic Clarke critical point, which are inherent to the class of problems that we analyze, and the conditions on the perturbation sequence pη n q, which depend on the practical situation and, to some degree, can be imposed by the designer.

This observation is consistent with the existing analysis of Equation (6.1) in the smooth setting. In this case, for almost every u P R d (henceforth generic vector u P R d ), every critical point of f u is either a local minimum or a saddle point (i.e. the Hessian of f u at this point has at least one negative eigenvalue). The nonconvergence of the SGD to a saddle point (and hence its convergence to a local minimum on a generic smooth function) was established in [Pemantle 1990, Brandière & Duflo 1996] under an assumption that, more or less, requires the lower boundedness of the (conditional) covariance of pη n q. When this type of assumption is not satisfied, as it happens for e.g. the deterministic gradient descent (η n " 0), it can indeed be guaranteed by the designer by adding a small perturbation, with lower bounded covariance, at every step.

Following this discussion, the present chapter consists of two, largely independent, parts.

• The first part is devoted to the analysis of the generic properties of Clarke critical points. Our main result, Theorem 6.2.5, proposes a classification of the types of points that might appear in (Z u ) for a non Lebesgue-null set of vectors u P R d . An emphasis is put on the conditions that allow the analysis of the SGD in a neighborhood of a generic critical point.

• The second part of this chapter is devoted to the analysis of the SGD in a neighborhood of a generic trap, i.e. a Clarke critical point that might appear in (Z u ) for a non Lebesgue-null set of vectors u P R d without being a local minimum. Specifically, we will present a set of conditions on the sequence pη n q that ensure that the iterates of the SGD will avoid a generic trap.

Generic critical points

In our analysis of genericity we restrict ourselves to the case where f , the function of interest, is definable in an o-minimal structure (henceforth definable). Formally defined in Section 2.4, the class of such functions encompasses the vast majority of functions encountered in optimization. It includes every semialgebraic function, the exponential, the logarithm as well as any of their compositions. While a definable function might be nonsmooth the nonsmoothness here appears in a very structured manner. For instance, a domain of a definable function can be partitioned into a set of manifolds called stratas such that on each of these stratas the function is differentiable. Starting from the seminal work of [START_REF] Bolte | [END_REF], this implicit smooth structure has allowed a thorough analysis of optimization algorithms in a definable setting (see e.g. [Attouch et al. 2011, Bolte et al. 2009, Davis et al. 2020, Bolte et al. 2020a]). Analysis of the generic properties of (Z u ) when f is definable goes back to the work of [Bolte et al. 2011, Drusvyatskiy et al. 2016] and more recently to [Davis & Drusvyatskiy 2021, Bianchi et al. 2021b]. The central notion in all of these works is the notion of an active manifold. Informally, M is an active manifold for a Clarke critical point x ˚P M if f varies smoothly on M and sharply outside of it. The importance of this notion lies in the fact, proved in [Drusvyatskiy et al. 2016] 1 , that if f is definable, then for a generic vector u P R d , the number of Clarke critical points of f u is finite and every one of them lies on an active manifold. Recently, following the ideas of [Drusvyatskiy et al. 2016, Drusvyatskiy & Lewis 2014], Davis and Drusvyatskiy [Davis & Drusvyatskiy 2021] have introduced the notion of an active strict saddle: a Clarke critical point x ˚of a function f , lying on an active manifold M , such that f M , the restriction of f to M , admits a second order negative curvature at x ˚. They have shown that if f is weakly convex, then for a generic vector u P R d , every point in (Z u ) is either a local minimum or an active strict saddle. Hence, in the weakly convex case the following two examples are typical. Local minimum. In order to be a local minimum a critical point lying on an active manifold M must be a local minimum of f M . As an example consider f 1 : R 2 Ñ R be defined as f 1 py, zq " y 2 `|z|. Then x ˚" p0, 0q is a Clarke critical point of f 1 and M 1 " R ˆt0u is the corresponding active manifold. Active strict saddle. Consider f 2 : R 2 Ñ R defined as f 2 py, zq " ´y2 `|z|. Then x ˚" p0, 0q is a Clarke critical point of f 2 and M 2 " R ˆt0u is the corresponding active manifold. Observe that in this case x ˚is not a local minimum of f 2 due to the fact that it is not a local minimum of f 2|M .

The reason behind such a simple classification lies in the fact that, from a minimization perspective, the behavior of a weakly convex function in a neighborhood of an active manifold M is dictated by its behavior on M . Examples presented in Section 6.2 show that such a result does not hold true as soon as the weak convexity assumption fails. Indeed, in full generality, it is clear that if a critical point x ˚of a function f lies on an active manifold M , then the local shape of f (and hence the type of x ˚) depends both on the behavior of f M and on the directions of the subgradients of f outside of M . Therefore, to obtain a proper classification in this general case, both of these informations must be taken into account.

This discussion motivates the introduction of a third type of a generic Clarke critical point: a sharply repulsive critical point. Its formal definition is given in Section 6.2 but informally it is a Clarke critical point x ˚of a function f , lying on an active manifold M , such that x ˚is a local minimum of f M , but there is a region close to M such that the subgradients of f point towards M . The following example is typical. Sharply repulsive critical point. Consider f 3 : R 2 Ñ R defined as f 3 py, zq " y 2 ´|z|. Then x ˚" p0, 0q is a Clarke critical point of f 3 and M 3 " R ˆt0u is the corresponding active manifold. In this example x ˚is indeed a local minimum of f 3|M 3 but every subgradient outside of M 3 is directed towards the active manifold.

Our first result, Theorem 6.2.5, shows that for a definable, locally Lipschitz continuous function f and for a generic vector u P R d , every critical point of f u is either a local minimum, an active strict saddle or a sharply repulsive critical point. Furthermore, we establish that the corresponding active manifolds are satisfying the Verdier and the angle conditions, introduced in [Bianchi et al. 2021b]. Importance of these conditions in the analysis of the SGD are discussed in the next section.

The role of the Verdier and the angle conditions

Analyzing the iterates of the SGD in a neighborhood of an active manifold M , it might be helpful to decompose Bf into components that are respectively tangent and normal to M . This technique of proof, developed in [Bianchi et al. 2021b], is natural when we think about the SGD applied to the previously presented functions f 1 , f 2 , f 3 . In this case we can decompose the iterates px n q into a sum of two sequence py n q, pz n q and notice that the sequence py n q (respectively pz n q) represents the iterates of the SGD applied to the function y Þ Ñ ˘y2 (respectively z Þ Ñ ˘|z|), where the respective signs should be obvious from the considered examples. Observe that in all of these cases py n q are the SGD iterates of a smooth function, while pz n q are either converging or diverging from 0 in a very fast manner.

To formalize this type of behavior authors of [Bianchi et al. 2021b] have introduced two additional assumptions on the active manifold M . The first one, the Verdier condition, states that for x close to M :

@v P Bf pxq, v M « ∇ M f pP M pxqq `Opdistpx, M qq ,
where P M pxq is the projection of x onto M , ∇ M f is the "Riemannian gradient" of f M and v M is the projection of v along the tangent space of M (see Section 6.2 for a precise statement). A consequence of this condition is that, writing down py n q " pP M px n qq, we obtain:

y n`1 « y n ´γn ∇ M f py n q `γn η M n`1 `γn Opdistpx n , M qq `Opγ 2 n q , (6.2)
where η M n`1 is the projection of η n`1 on the tangent space of M at y n . That is to say, up to a residual error term, py n q follows an SGD dynamic on the (smooth) function f M .

To motivate the angle condition a following observation was made in [Bianchi et al. 2021b]. Let x ˚be a Clarke critical point of f lying on an active manifold M . Then, on the event rx n Ñ x ˚s, for n large enough we have: f px n q ´f pP M px n qq Á x n ´PM px n q .

(6.

3)

The angle condition then states that close to M we have:

f pxq´f pP M pxqq Á x ´PM pxq ùñ xv, x´P M pxqy Á x ´PM pxq , @v P Bf pxq . (6.4) Combining (6.3) with (6.4), we obtain that for n large enough the angle between the set Bf px n q and the normal direction to M is lower bounded. The latter allows to control the residual term in Equation (6.2).

Both of these conditions provide a way to analyze the SGD in a neighborhood of an active manifold by decomposing the iterates px n q into a sum of two sequences: py n q " pP M px n qq and pz n q " px n ´yn q. The angle condition ensures the fact that distpx n , M q " z n Ñ 0 (and hence x n Ñ M ) fast enough. Combining this fact with the Verdier condition, this implies that py n q, up to a residual term, follows an SGD dynamic on the smooth function f M .

In Chapter 5 this technique of proof was used to show that, under assumptions on pη n q similar to [Brandière & Duflo 1996], the SGD avoid active strict saddles with probability one. In this chapter we illustrate the interest of the angle condition in the analysis of the SGD in a neighborhood of a sharply repulsive critical point.

Avoidance of generic traps

The final part of this chapter is devoted to the analysis of the SGD in a neighborhood of a generic trap. Since the question of the nonconvergence to an active strict saddle was tackled in [Bianchi et al. 2021b] we focus in this part on the question of nonconvergence of the SGD to x ˚P M a sharply repulsive critical point.

Our first result, which requires only very mild, moment assumptions on the sequence pη n q, is that on the event rx n Ñ x ˚s, where x ˚is a sharply repulsive critical point, we have that, for n large enough, f px n q ě f px ˚q , While the proof of this statement readily follows from Chapter 5 such a result is interesting. Indeed, it implies that while the iterates px n q may in theory converge to x ˚this happens only if the SGD fails to explore the repulsive region near x ˚.

In some sense, the algorithm perceive the function f as if x ˚was indeed its local minimum.

In a second time, we show that a density-like assumption on pη n q forces the SGD to visit the repulsive region near M and will imply the nonconvergence of the SGD to a sharply repulsive critical point.

The final Section 6.3.3 shows that while such a density-like assumption on pη n q might not hold, in a standard stochastic approximation model, a way to ensure it is to add a small perturbation (e.g. a nondegenerate Gaussian) at each iteration of (6.1). This fact, combined with the results of [Bianchi et al. 2021b] on the avoidance of active strict saddles, provides a practical way to avoid generic traps of a definable function, and, therefore, ensure the convergence of the SGD to a local minimum.

Previous avoidance of traps results and contributions

We finish the introduction by a discussion on diverse avoidance of traps results previously stated in the literature. In the smooth setting avoidance of saddle points by the SGD (and more generally by a Robbins-Monroe algorithm) was first addressed by Brandière and Duflo [Brandière & Duflo 1996] and Pemantle [Pemantle 1990]. Under a condition that, more or less, requires a lower boundedness of the covariance of pη n q they have established that the iterates of the SGD avoid saddle points with probability one. Later on, these results were extended in many ways, we mention here the nonconvergence to periodic hyperbolic sets by [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF]], the nonautonomous setting [Barakat et al. 2021] and many others [Mertikopoulos et al. 2020b, Gadat & Gavra 2020]. From another perspective, the authors of [Lee et al. 2016] have established the nonconvergence to a saddle point of the deterministic gradient descent under a random initialization. Davis and Drusvyatskiy, in [Davis & Drusvyatskiy 2021], have presented a first nonconvergence result in the nonsmooth setting. They have introduced the concept of an active strict saddle and similarly to [Lee et al. 2016] have established that proximal methods avoid active strict saddle under a random initialization. As mentioned earlier, in our previous work [Bianchi et al. 2021b], under the same conditions on the perturbation sequence as in [Brandière & Duflo 1996], we have established the nonconvergence of the SGD to an active strict saddle lying on an active manifold that satisfies a Verdier and an angle conditions.

Finally, shortly after the publication [Bianchi et al. 2021b] and just before the submission of [Schechtman 2021b], on which is based the current chapter, a concurrent work [Davis et al. 2021] has appeared. The latter, sharing a lot of similarities with [Bianchi et al. 2021b], analyzes the SGD (and its proximal versions) in a neighborhood of an active manifold. An avoidance of active strict saddles result was obtained as well as (local) rates of convergence and asymptotic normality of the iterates was established. These results support our claim of the importance of the Verdier and the angle conditions. A major difference with this chapter is that their proximal aiming condition assume (close to the active manifold) the left hand side of formula (6.4). Such an assumption rules out functions with downward cusps such as py, zq Þ Ñ ˘y2 ´|z|, which are treated in [Bianchi et al. 2021b] and in the present chapter. As a consequence, the question of genericity in [Davis et al. 2021] is addressed only for the class of Clarke regular functions in which sharply repulsive critical points do not exist. In particular, we believe that convergence rates of a similar kind could be obtained upon replacing the proximal aiming condition of [Davis et al. 2021] by ours angle condition.

Paper organization. Section 6.2 deals with the generic properties of Clarke critical points. In Section 6.3 we state an avoidance of traps result and discuss the convergence of the SGD to minimizers. Section 6.4 is devoted to proofs.

Notations. For x P R d and r ą 0, we denote Bpx, rq the open ball centered of radius r centered at x. Given a set S Ă R d , S will denote its closure and S c its complementary. The distance to S will be denoted as distp¨, Sq. We say that

V Ă S is open in S if there is an open set U Ă R d such that U X S " V . We say that H : R d Ñ R d is a set-valued mapping if for each x P R d , Hpxq Ă R d ,

we denote

GraphpHq " tpx, yq : x P dompHq, y P Hpxqu its graph. We say that a property holds locally around x if this property holds on U an open neighborhood of x. We say that a function f : R d Ñ R is weakly convex if there is ρ ą 0 such that f p¨q `ρ ¨ 2 is convex.

Given n random variables X 1 , . . . , X n on some probability space, we denote σpX 1 , . . . , X n q the sigma algebra generated by them. The set of borelians of R d will de denoted as BpR d q. Given some probability space on which we have pF n q a filtration and X a random variable, we will denote E n rXs " ErX|F n s. Given a matrix B P R mˆn , we will denote B T its transpose.

Generic critical points

Theorem 6.2.5 of this section classifies generic critical points of a locally Lipschitz continuous, definable function. A reader who is more interested in our avoidance of traps result can take Definitions 6.2.1-6.2.4 as granted and jump to Section 6.3. We recall that for a function f : R d Ñ R and u, x P R d , we denote f u pxq " f pxq ´xu, xy and that, given a manifold M , we denote f M the restriction of f to M .

To motivate our presentation consider first the case where f : R d Ñ R is C 2 . Applying Sard's theorem to the function x Þ Ñ ∇f pxq, we obtain that the set tu P R d : Dx P R d , ∇f pxq " u and the Hessian of f at x is degenerate u is Lebesgue-null. Hence, for almost every u P R d , the critical points of f u pxq " f pxq ´xu, xy are nondegenerate. This result can be extended to functions defined on a submanifold. Proposition 6.2.1 ( [Victor 1974, Chapter 7, §7]). Consider M Ă R d a C 2 submanifold of dimension greater than 0. Let f : M Ñ R be C 2 . Then for almost every u P R d , the critical points of f u|M are nondegenerate.

Remark 21. A function f : M Ñ R d such that every of its critical points is nondegenerate is called a Morse function. Proposition 6.2.1 shows that Morse functions always exist. This result can be strengthened to the fact that the set of Morse functions is open and dense in the Whitney C 2 topology (cf. e.g. [Audin et al. 2014]). In that sense almost every smooth function on M is Morse.

Active manifolds

The central notion of this work is the notion of an active manifold. It was introduced by Lewis in [Lewis 2002] and was thoroughly studied in [Drusvyatskiy & Lewis 2014, Drusvyatskiy & Lewis 2012]. Definition 6.2.1 (Active manifold). Consider f : R d Ñ R a locally Lipschitz continuous function and x ˚such that 0 P Bf px ˚q. For p ě 1, we say that M is a C p active manifold around x ˚if there is a neighborhood U of x ˚such that the following holds.

i) Smoothness. M X U is a C p submanifold and f is C p on M X U . ii) Sharpness. inft v : v P Bf pxq, x P U X M c u ą 0 .
Note that in the preceding definition M can be the whole space R d . As a consequence any C p function f : R d Ñ R admits R d as an active manifold on any of its critical point.

The following conditions on an active manifold were introduced in [Bianchi et al. 2021b]. We recall that P M denotes the projection onto M and that by Lemma 2.3.2 it is well defined in the neighborhood of M . Definition 6.2.2 (Verdier and angle conditions). Let f : R d Ñ R be a locally Lipschitz continuous function. Let M be a C 2 active manifold around a critical point x ˚. We say that M satisfies a Verdier and an angle conditions if the following holds.

• Verdier condition. There is U a neighborhood of x ˚and C ě 0 such that for y P M X U and x P U , we have: @v P Bf pxq, P TyM pvq ´∇M f pyq ď C x ´y , where P TyM is the orthogonal projection onto T y M .

• Angle condition. For every α ą 0, there is β ą 0 and U α a neighborhood of x ˚such that for all x P U α , we have:

f pxq´f pP M pxqq ě α x ´PM pxq ùñ xv, x´P M pxqy ě β x ´PM pxq , @v P Bf pxq . (6.5)

In practice an active manifold is an element of the stratification presented in Theorem 5.2.1. Hence, the Verdier condition is just a transcription of Inequality (5.3) in this setting. The importance of the angle condition can be grasped from the following observation made in [Bianchi et al. 2021b]. Proposition 6.2.2 ( [Bianchi et al. 2021b, Lemma 7]). Let f : R d Ñ R be pathdifferentiable and assume that there is x ˚P R d a Clarke critical point lying on a C 2 active manifold M . There is r, T, α ą 0 such that for any x P Bpx ˚, rq and x P S ´Bf pxq, either xpr0, T sq Ć Bpx ˚, rq or f pxq ě f pP M pxqq `α x ´PM pxq .

The preceding lemma shows that the set tx P Bp0, rq : f pxq ă f pP M pxqq `α x ´PM pxq u , (6.6)

where α, r are the one of Proposition 6.2.2, can be viewed as a repulsive region for the subgradient flow. The angle condition ensures the fact that as soon as we are not in this repulsive region the negative subgradients of f are directed towards M . This information will help us to show that the iterates of the SGD converge to M fast enough. The reason behind the chosen denomination of Definition 6.2.4 comes from the following proposition. It shows that the active manifold of a sharply repulsive critical point is always neighbored by a large repulsive region of the form (6.6). Proposition 6.2.3. Let f : R d Ñ R be a continuous function and x ˚P R d such that 0 P Bf px ˚qzB L f px ˚q. There is C ą 0 such that for all ε ą 0, there is x P Bpx ˚, εq such that:

Generic traps

f pxq ď f px ˚q ´C x ˚´x .

If, moreover, x ˚is a sharply repulsive critical point, lying on a C 2 active manifold M , then there is ε ą 0 such that for all y P Bpx ˚, εq X M and for all ε y ą 0, there is x P Bpy, ε y q such that we have:

f pxq ă f pP M pxqq .
If f is weakly convex, then B L f " Bf . Hence, such a function does not have sharply repulsive critical points. As the following proposition shows, in this case, the notion of an active strict saddle is generic. In its initial version this proposition follows from the work [Drusvyatskiy et al. 2016] and was proved in [Davis & Drusvyatskiy 2021]. Statements concerning the Verdier and the angle conditions were proved in [Bianchi et al. 2021b]. Proposition 6.2.4 ([Davis & Drusvyatskiy 2021, Theorem 2.9] and [Bianchi et al. 2021b, Theorem 2]). Let f : R d Ñ R be definable and weakly convex. There is N P N such that for almost every u P R d , the set (Z u ) is of cardinality less than N . Moreover, every such a point lies on an active manifold satisfying the Verdier and the angle conditions and is either a local minimum or an active strict saddle.

The reason behind such a simple classification in Proposition 6.2.4 comes from the fact that under weak convexity a local minimum of f M is also a local minimum of the unrestricted function f . As the following example shows this is no longer true without the weak convexity assumption.

Example 6.2.1. Consider f 4 : R 2 Ñ R defined as f 4 py, zq " ´|y| `|z|. For u P Bp0, 1q, the point p0, 0q is a sharply repulsive critical point lying on the active manifold M 4 " tp0, 0qu.

We are now ready to state the main result of this section which is a generalization of Proposition 6.2.4 to the non weakly convex case. Theorem 6.2.5. Let f : R d Ñ R be a locally Lipschitz continuous, definable function and p ě 2. There is N P N such that for almost every u P R d the set (Z u ) is of cardinality less than N and every point x ů P Z u lies on a C p active manifold M such that the following holds.

i) The manifold M satisfies the Verdier and the angle conditions.

ii) If the dimension of M is greater than 0, then x ů is a nondegenerate critical point for the function f u : M Ñ R.

iii) The point x ů is either a local minimum, an active strict saddle or a sharply repulsive critical point of f u .

Avoidance of generic traps

Escaping a sharply repulsive critical point

Let f : R d Ñ R be a locally Lipschitz continuous function. Let G : R d Ñ R d be a measurable function such that for all x P R d , Gpxq P Bf pxq, such a "measurable selection" always exists (cf. [START_REF] Rockafellar | [END_REF]). On a probability space pΩ, A, Pq, consider a random variable x 0 P R d and a random sequence pη n q P pR d q N . Define the iterates:

x n`1 " x n ´γn Gpx n q `γn η n`1 " x n ´γn v n `γn η n`1 , (6.7)

where v n :" Gpx n q and pγ n q is a deterministic sequence of positive numbers. Let pF n q be a filtration on pΩ, A, Pq.

Assumption 6.3.1.

i) The function f is path-differentiable.

ii) The sequence pη n q is adapted to pF n q and x 0 is F 0 -measurable.

iii) The sequence pγ n q is such that ř `8 i"0 γ i " `8, ř `8 i"0 γ 2 i ă `8 and there is c 1 , c 2 ą 0 such that:

c 1 ď γ 2 n γ 2 n`1 ď 1 `c2 γ n .
6.3. Avoidance of generic traps 153 Assumption 6.3.1 is a standard assumption in the field of stochastic approximation. We notice that the point (iii)) is satisfied by the sequences of the form γ n " 1 n for P p1{2, 1s.

Recall that E n r¨s denotes Er¨|F n s.

Assumption 6.3.2. The sequence pη n q is such that E n rη n`1 s " 0 and for every C ą 0, there is KpCq ą 0 such that we have:

sup nPN E n r η n`1 2 s1 xn ďC ď KpCq .
Assumption 6.3.3. The point x ˚is a sharply repulsive critical point of f such that the corresponding active manifold M is C 2 .

Our first result concerning the behavior of the SGD in the neighborhood of a sharply repulsive critical point is the following proposition. Its proof is provided in Section 6.4.2. Proposition 6.3.1. Let Assumptions 6.3.1-6.3.2 hold. Assume that a point x ˚P Z is lying on a C 2 active manifold. There is α ą 0 such that, almost surely on the event rx n Ñ x ˚s, there is n 0 P N such that for n ě n 0 , we have:

f px n q ě α x n ´PM px n q `f pP M px n qq .
As a consequence, under Assumptions 6.3.1-6.3.3, for n large enough, f px n q ě f px ˚q .

A consequence of the preceding theorem is the fact that while the iterates of the SGD may in theory converge to a sharply repulsive critical point, this happens only if the sequence px n q fails to explore the repulsive region of the form (6.6) neighboring the active manifold. Without additional assumptions on the perturbation sequence pη n q, the following example shows that such behavior is easy to construct. Recall that f 4 : R 2 Ñ R is defined as f 4 py, zq " ´|y| `|z|.

Example 6.3.1. Consider z P R and let x 0 " py 0 , z 0 q " p0, zq. For n P N, define η n " 0 and v n " p0, zn |zn| q1 zn ą0 . Then the sequence px n q defined by Equation (6.7) represents the iterates of the SGD applied to f 4 and x n Ñ p0, 0q.

The next two assumptions will force the SGD to explore the repulsive region around x ˚. Assumption 6.3.4. For every C ą 0, there is a continuous, positive function h C : R d Ñ R such that for every n P N and any measurable function ψ : R d Ñ R d , if x n ď C, then: @δ ą 0, Ppη n`1 P Bpψpx n q, δq|F n q ě ż 1 Bpψpxnq,δq pyqh C pyq dy . Assumption 6.3.5. The active manifold from Assumption 6.3.3 satisfies an angle condition. Assumption 6.3.4 describes a density-like behavior of the conditional law of pη n`1 q. Indeed, it is satisfied if the conditional laws of pη n`1 q are identically distributed according to some law that has a density relatively to Lebesgue which is positive at every point. As we show in Section 6.3.3 to enforce this assumption it is sufficient to add a "nondegenerate" perturbation at each step.

On the other hand, as the following proposition shows, Assumption 6.3.5 allows to control the speed of convergence of px n q towards a sharply repulsive critical point. Proposition 6.3.2. Let Assumptions 6.3.1-6.3.3 and 6.3.5 hold. There is κ ą 0 such that on rx n Ñ x ˚s the event rdistpx n , M q ď κγ n s occurs infinitely often.

Finally, with this result in hand we have that a sharply repulsive critical point is avoided by the SGD with probability one. Theorem 6.3.3. Let Assumptions 6.3.1-6.3.5 hold. Then, Pprx n Ñ x ˚sq " 0.

The proof of Theorem 6.3.3 is slightly technical but conceptually it can be described as follows. By Proposition 6.3.2 the iterates are infinitely often located at a distance less than κγ n from the active manifold. Since x ˚is a sharply repulsive critical point, M is neighbored by a repulsive region (6.6). Assumption 6.3.4 then forces the algorithm to recur in this repulsive region, which in turn contradicts Proposition 6.3.1.

Convergence to minimizers

From the results of Section 6.2 we have that every Clarke critical point of a generic definable function that is not a minimizer is either a sharply repulsive critical point or an active strict saddle. Hence, to obtain the convergence of the SGD to minimizers we need to investigate the question of the avoidance of active strict saddles. As previously mentioned, this question was tackled in [Bianchi et al. 2021b]. Proposition 6.3.4 ( [Bianchi et al. 2021b, Theorem 3]). Let f : R d Ñ R be locally Lipschitz continuous. Consider the iterates (6.7) under Assumptions 6.3.1(i))-(ii)). Assume that x ˚is an active strict saddle lying on a C 4 active manifold that satisfies the Verdier and the angles conditions. Furthermore, assume that the following holds. i) There is c 3 , c 4 ą 0 and P p1{2, 1s such that for all n P N,

c 3 n ď γ n ď c 4 n .
ii) The sequence pη n`1 q is such that E n rη n`1 s " 0.

iii) For all w P R d zt0u, we have almost surely:

lim inf nPN E n r|xη n`1 , wy|s ą 0 ,
and on the event rx n Ñ x ˚s:

sup nPN E n r η n`1 4 s ă `8 , Then Pprx n Ñ x ˚sq " 0.
Notice that the assumption on pγ n q of the preceding proposition implies Assumption 6.3.1-(iii)). Therefore, combining Proposition 6.3.4 with Theorem 6.3.3, we obtain that on a generic definable, locally Lipschitz continuous function the SGD converges to a local minimum. We state this result in the following corollary. Corollary 6.3.5. Let f : R d Ñ R be locally Lipschitz continuous. Assume that every of its Clarke critical points is isolated and is either a local minimum, an active strict saddle or a sharply repulsive critical point with the corresponding active manifolds being C 4 -smooth and satisfying the Verdier and the angle conditions. Consider the iterates (6.7) under Assumptions 6.3.1(i))-(ii)) and 6.3.4. Moreover, assume that the following almost surely holds. i) For all n P N, E n rη n`1 s " 0.

ii) For every C ą 0, there is KpCq ą 0 such that:

sup nPN E n r η n`1 4 s1 xn ďC ă KpCq .
iii) There is c 3 , c 4 ą 0 and P p1{2, 1s such that for all n P N,

c 3 n ď γ n ď c 4 n .
Then, almost surely, the sequence px n q is either unbounded or converges to a local minimum of f .

Proof. Let x ˚be one of the Clarke critical points of f . The only thing that we have to show is that under Assumption 6. We finish this section by a discussion on differences in the proofs of Proposition 6.3.4 and Theorem 6.3.3.

The idea of the proof of Proposition 6.3.4 can be described as follows.

• Using the angle condition, show that the iterates of the SGD converge to M fast enough.

• Using the Verdier condition, show that the sequence pP M px n qq of the projected SGD iterates follows a gradient descent on a smooth function f M .

• Since x ˚is an active strict saddle, it is a saddle point of the function f M and, with some minor adaptations, the nonconvergence follows from the works of [Brandière & Duflo 1996, Pemantle 1990] on avoidance of saddle points when the objective is smooth.

In [Bianchi et al. 2021b] the technique used to prove the first point is similar to the one used for the proof of Proposition 6.3.2. However, afterwards, the reasons for the nonconvergence to an active strict saddle are different. Indeed, in Theorem 6.3.3 the SGD avoids a sharply repulsive critical point due to the fact that the iterates px n q visits infinitely often a repulsive region of the form (6.6). Such repulsive region does not necessarily exist in the case of an active strict saddle (think of the function f 2 from the introduction). Hence, the proof of Proposition 6.3.4 heavily relies on the Verdier condition which is not necessary in our case.

Nevertheless, we notice that if one wants to describe the speed of convergence to x ˚lying on an active manifold M such that x ˚is a local minimum of f M (e.g. a sharply repulsive critical point or a local minimum of f ), then both of the Verdier and the angle conditions are useful since, as in [Bianchi et al. 2021b], it can be established that the iterates will converge promptly to M and the Verdier condition allows to show that, up to a manageable error term, the sequence pP M px n qq is simply an SGD sequence applied to a smooth function f M . It should be possible in that case to obtain rates of convergence in the spirit of [Mertikopoulos et al. 2020b]. We defer such considerations to future work.

Validity of Assumption 6.3.4

In this section we present a model that satisfies Assumptions 6.3.1-6.3.2 and show how to alter it to obtain Assumption 6.3.4. This will provide us with a practical way to ensure the convergence of the SGD to a minimizer.

We start by a motivational example.

Example 6.3.2. In machine learning we are usually interested to optimize f : R d Ñ R written as an average of N functions:

f pxq " 1 N N ÿ i"1 f i pxq .
Here, N is the number of data points and f i is the loss function related to the ith data point. In this case a version of the SGD is obtained by choosing, at each iteration n P N, an index i P t1, . . . , N u in an uniform manner and updating:

x n`1 " x n ´γn g i px n q , (6.8)

where g i : R d Ñ R are such that 1 N ř N i"1 g i pxq P Bf pxq. If for all i P t1, . . . , N u are smooth, concave or weakly convex2 , then we can choose g i pxq P Bf i pxq. In this case Equation (6.8) can be viewed as Equation (6.7) by putting v n " 1 n ř N i"1 g i px n q and η n`1 " g i px n q ´vn .

In the stochastic approximation litterature (see e.g. [START_REF] Borkar | [END_REF], Kushner & Yin 2003]) this and more general settings are modelized by a probability space pΞ, T , µq and a measurable function g : R d ˆΞ Ñ R d such that for each x P R d , the function gpx, ¨q is µ-integrable and we have the following:

Gpxq :" ż Ξ gpx, sqµpdsq P Bf pxq .

Starting from x 0 P R d , at each iteration n P N the practitioner samples ξ n " µ in an independent way and update the iterates according to the following rule:

x n`1 " x n ´γn gpx n , ξ n`1 q .

We obtain Equation (6.7) by putting v n " Gpx n q, η n`1 " Gpx n q ´gpx n , ξ n`1 q and F n " σpx 0 , ξ 1 , . . . , ξ n q.

In the context of this model consider a sequence pη 1 n q of i.i.d R d -valued random variables, with η 1 " ν s.t. the following holds.

1. For each n P N, η 1 n is independent from F n . 2. The law ν is zero-mean with finite variance.

3. The law ν has a continuous density relatively to the Lebesgue measure on R d .

Moreover, denoting this density h 1 : R d Ñ R, we have that for each point x P R d , h 1 pxq ą 0.

An example of a law that verifies the last two points is e.g. a nondegenerate gaussian.

Starting from a F 0 -measurable point x0 P R d , consider the following algorithm:

xn`1 " xn ´γn gpx n , ξ n`1 q `γn η 1 n`1 " xn ´γn ṽn `γn ηn`1 , (6.9)

where ṽn " Gpx n q P Bf px n q and ηn`1 " ṽn ´gpx n , ξ n`1 q `η1 n`1 . Proposition 6.3.6. Assume that for every C ą 0, there is KpCq ą 0 such that:

sup x ďC ż
Gpx, sq ´gpx, sq 2 µpdsq ď KpCq .

Then the sequence pη n`1 q satisfies Assumptions 6.3.2 and 6.3.4 relatively to the filtration Ă F n " F n b σpη 1 1 , . . . , η 1 n q.

The preceding proposition provides a practical way to ensure the convergence of the SGD to a minimizer. As the following corollary states, if the objective function is generic, then it suffices to add a small random perturbation at every iteration of the algorithm. Corollary 6.3.7. Let f : R d Ñ R be satisfying the assumptions of Corollary 6.3.5. Consider the iterates of Equation (6.9) and assume that the following holds. i) For every C ą 0, there is KpCq ą 0 such that:

sup x ďC ż
Gpxq ´gpx, sq 4 µpdsq ă KpCq .

ii) The law ν has a finite fourth order moment.

iii) There is c 3 , c 4 ą 0 and P p1{2, 1s such that for all n P N:

c 3 n ď γ n ď c 4 n .
Then the sequence px n q is either unbounded or converges to a local minimum of f .

Proofs

We recall that most of the results on o-minimality that are used in this proof are gathered in Section 2.4. In particular, we will use the notion of first-order formula and Proposition 2.4.1 without further mentioning. The proof of Theorem 6.2.5 is based on the following result of Drusvyatskiy, Ioffe and Lewis [Drusvyatskiy et al. 2016]. Proposition 6.4.1 ([Drusvyatskiy et al. 2016, Theorem 4.7 andCorollary 4.8]). Consider p ě 2 and f : R d Ñ R a locally Lipschitz continuous, definable function. There is N ą 0 such that for almost every u, the function f u has at most N Clarke critical points. Moreover, denoting x ů such a point, the following holds. i) There is M a C p active manifold around x ů (for the function f u ).

ii) There is W, V neighborhoods of respectively u and x ů such that the mapping w Þ Ñ V X pBf w q ´1p0q is single valued, C p smooth on W and maps W onto M .

iii) If 0 P B L f u px ůq and x ů is a local minimum of the function f u|M : M Ñ R, then x ů is a local minimum of the unrestricted function f u .

In [Drusvyatskiy et al. 2016] the preceding proposition was stated for the limiting subgradient B L f u (i.e. in Definition 6.2.1 the Clarke subgradient Bf was replaced by B L f and the critical point x ů was defined as 0 P B L f u px ˚q). However, the only property that was used for the proof of these points was the fact that dim GraphpB L f pxqq " d (here dimension is understood in the sense of Definition 2.4.4). Since by [Drusvyatskiy & Lewis 2010b, Theorem 3.5] dim GraphpBf pxqq " d, Proposition 6.4.1 remains true with our definition.

Let tX 1 , . . . , X k u be the C p stratification from Theorem 5.2.1. The existence of an active manifold with a Verdier condition follows from Proposition 6.4.1 upon noticing that in the proof of [Drusvyatskiy et al. 2016, Corollary 4.8] the active manifold can be chosen compatible with tX 1 , . . . , X k u. To deal with the angle condition and the nondegeneracy of critical points we consider separately the case where dim M " 0 and dim M ą 0. First case: dim M " 0. In this case the angle condition follows from the following result of [Bolte et al. 2009 , denote f 1 px; dq " lim tÑ0 f px`tdq´f pxq t (notice that this limit always exists by Lemma 2.4.2 and the local Lipschitz continuity of f ). We have:

|f px `dq ´f pxq ´f 1 px `d; dq| " o x p d q , (6.10)

where o x p d q means that oxp d q d Ñ dÑ0 0.

Without loss of generality, assume that the critical point x ů is equal to zero and f u p0q " 0. We have that M " t0u and by Equation (6.10):

|f u px ů `xq ´fu px ůq ´f 1 px ˚`x; xq| " |f u pxq ´f 1 u px; xq| " op x q .

For α ą 0, we have:

f u pxq ě α x ùñ f 1 u px; xq ě α x `op x q .

Therefore, for x close enough to zero:

f 1 u px; xq ě α 2 x .
Notice that if f u is differentiable at x, then f 1 u px; xq " x∇f u pxq, xy. Hence, the angle condition is proved on a point of differentiability.

For the general case, consider v P Bf u pxq. By Proposition 2.1.1 there is k P N, a sequence px n 1 , . . . , x n k q Ñ px, . . . , xq and a sequence pλ 1 , . . . , λ k q such that ř k i"1 λ i " 1, for each pi, nq P r1, . . . , ks ˆN, f u is differentiable at x n i and v " lim nÑ`8 ř k i"1 λ i ∇f u px n i q. Since at x n i we have f 1 u px n i ; x n i q " x∇f px n i q, x n i y, we ob-

λ i x∇f u px n i q, x ´xn i y ě α 2 x ,
where the last inequality is obtain by the triangular inequality, the fact that x n i Ñ x and that ∇f px n i q is bounded. Second case: dim M " k ą 0. Let u, W, x ů be as in Proposition 6.4.1. In the following, without loss of generality, we will assume that u " 0 and W is bounded. We start by outlining the proof.

• Using Proposition 6.2.1 and the Verdier condition, we show that for almost every w P W the critical point of f w|M are nondegenerate.

• In Lemmas 6.4.3 and 6.4.4 we show that that the dimension of y P M such that the angle condition is verified in the neighborhood of y is equal to k.

• The preceding point along with Lemma 6.4.5 and the second point of Proposition 6.4.1 shows that for almost every perturbation w P W , the angle condition is verified in a neighborhood of x ẘ.

Since R d is covered by a countable union of such neighborhoods W these three points will prove Theorem 6.2.5.

To prove the first point of the outline notice that by the Verdier condition the map from Proposition 6.4.1 is actually equal to:

w Þ Ñ x ẘ P M s.t. ∇ M f w px ẘq " 0 .
Therefore, by Proposition 6.2.1 we have that for almost every w P W the critical point x ẘ is nondegenerate for the function f w|M .

To prove the second point denote P α,β pxq the first-order formula: @w P W, f w pxq ě α x ´PM pxq `fw pP M pxqq ùñ xv w , x ´PM pxqy ě β x ´PM pxq , @v w P Bf w pxq , where, implicitly, in this formula we consider only such x for which P M pxq is well defined. The first-order formula "not P α,β pxq" can be written as:

Dw P W, Dv w P Bf w pxq such that: f w pxq ě α x ´PM pxq `fw pP M pxqq and xv w , x ´PM pxqy ă β x ´PM pxq . Fix α ą 0 and let A α be the following definable set:

A α " ty P M : @ε ą 0, Dx P Bpy, εq, "not P 2α,α pxq"u . (6.11)

The following lemma shows that for a fixed α, for almost every y P M , there is a neighborhood around y such that the formula P 2α,α holds.

Lemma 6.4.3. We have that dim A α ă dim M " k.

Proof. Assume the contrary, by construction A α lies in the boundary of the following definable set. Q α :" tx R M : " not P 2α,α pxq"u .

Applying Lemma 2.4.9, we obtain a k-dimensional definable set A 1 , δ ą 0 and a definable C 1 map ρ : A 1 ˆp0, δq Ñ Q α such that P M pρpy, tqq " y and ρpy, tq ´y " t. By the definition of Q α this means that for all py, tq P A 1 ˆp0, δq there is v P Bf pρpy, tqq and w P W such that xv ´w, ρpy, tq ´yy ă αt ă 2αt ď f pρpy, tqq ´f pyq ´xw, ρpy, tq ´yy .

Fix y P A 1 and denote ρ y p¨q " ρpy, ¨q. There are two definable selections vptq P Bf pρ y ptqq and wptq P W such that xvptq ´wptq, ρ y ptq ´yy ă αt ă 2αt ď f pρ y ptqq ´f pyq ´xwptq, ρ y ptq ´yy . (6.12)

Since f is path-differentiable, we have:

f pρpy, tqq ´f pyq " ż t 0 xvpuq, 9 ρ y puqy du .

For t 1 small enough, we have that the expression under the integral is continuous on p0, t 1 q. Therefore, by the mean value theorem, for every t P p0, t 1 q, there is u t P p0, tq such that f pρ y ptqq ´f pyq " txvpu t q, 9 ρ y pu t qy .

Denote v y " lim tÑ0 vptq and w y " lim tÑ0 wptq (these limits exists by the monotonicity lemma and the fact that f is locally Lipschitz continuous). By Lemma 2.4.3 applied to each coordinate of ρ y ptq ´y we have the existence of R y P R d such that R y " lim tÑ0 ρyptq´y t

" lim tÑ0 9 ρ y ptq. Hence, from Inequality (6.12) we obtain:

xv y ´wy , R y y ď α ă 2α ď xv y ´wy , R y y , which is a contradiction.

Proof of Proposition 6.2.3

The first statement immediately follows from the definition of the limiting subgradient.

To prove the second statement, without loss of generality, assume that x ˚" 0. By contradiction assume that for all ε ą 0 there is y ε P Bp0, εq X M and ε yε ą 0 such that for all x P Bpy ε , ε yε q we have:

f pxq ě f pP M pxqq .
By Lemma 2.3.3 we have: f pxq ě f py ε q `f pP M pxqq ´f py ε q ě f py ε q `x∇ M f py ε q, P M pxq ´yε y `Op P M pxq ´yε 2 q ě f py ε q `x∇ M f py ε q, x ´yε y `x∇ M f py ε q, P M pxq ´xy `Op P M pxq ´yε 2 q

Since f is C 2 on M , we have:

x∇ M f py ε q, P M pxq ´xy ě x∇ M f pP M pxqq, P M pxq ´xy ´ ∇ M f pP M pxqq ´∇M f py ε q P M pxq ´x ě x∇ M f pP M pxqq, P M pxq ´xy `Op x ´yε q P M pxq ´x .

Notice that ∇ M f pP M pxqq P T P M pxq and that by Lemma 2.3.2 for x close enough to M , we have P M pxq ´x P N P M pxq M . Therefore, x∇ M f pP M pxqq, P M pxq ´xy " 0 and we obtain:

f pxq ě f py ε q `x∇ M f py ε q, x ´yε y ` x ´PM pxq Op y ε ´PM pxq q `Op P M pxq ´yε 2 q .

Since P M is Lipschitz continuous, we have Op P M pxq ´yε q " Op x ´yε q. Hence, ∇ M f py ε q P B L f py ε q. Since ∇ M f py ε q Ñ yεÑ0 0, this implies that 0 P B L f p0q, a contradiction.

6.4.2 Proof of Proposition 6.3.1

Consider the linearly interpolated process X : R `Ñ R d defined as:

Xptq " x n `t ´řn i"0 γ n γ n`1 px n`1 q ´xn , if t P

« n ÿ i"0 γ i , n`1 ÿ i"0 γ i ¸.
By Assumption 6.3.2 and [Schechtman 2021a, Lemma 1] the sequence p ř n i"0 γ i η i`1 q converges on the event rx n Ñ x ˚s. Hence, from the work of Benaïm, Hofbauer and Sorin [Benaïm et al. 2005, Proposition 1.3], on the event rx n Ñ x ˚s, X is a so-called asymptotic pseudo trajectory of the subgradient flow (5.14). That is to say, for every T ě 0, by [Benaïm et al. 2005, Proposition 4.1] we have that: The proof will be done in three steps.

1. Lemma 6.4.9 shows that there is a constant C such that if distpx n , M q is of order of γ n , then there is x 1 n such that distpx 1 n , M q is also of order γ n and every point Bpx 1 n , Cγ n q is in the repulsive region of the form (6.6).

2. Using Assumption 6.3.4, Lemma 6.4.10 then shows that in such a case the probability of x n`1 visiting a repulsive region is lower bounded.

3. The preceding point along with Proposition 6.3.2 and Lemma 6.4.11 then shows that the iterates px n q visit a repulsive region infinitely often. The latter is impossible by Proposition 6.3.1.

Without loss of generality, assume that x ˚" 0. In this section U will be a bounded neighborhood of zero such that the following holds on U .

i) There is c m ą 0 s.t. inft Bf pxq : x P U X M c u ě c m .

ii) The function P M is C 1 on U .

iii) The functions f, P M , f ˝PM are Lipschitz on U , with Lipschitz constants L, L π , L M . Lemma 6.4.9. Let Assumption 6.3.3 hold. There is r 1 ą 0 s.t. Bp0, r 1 q Ă U , and for all y P Bp0, r 1 q X M , for all δ s.t. y `δ ă r 1 , there is x such that the following holds.

i) x ´y " δ.

ii) f pxq ď f pP M pxqq ´c2 m δ 4L .

Moreover, for every such x, denoting δ 1 " c 2 m δ 8LpL`L M q , for every x 1 P Bpx, δ 1 q, we have:

f px 1 q ă f pP M px 1 qq .

Proof. Consider r from Proposition 6.2.2 and let r 1 ď r 2 be such that Bp0, r 1 q Ă U and L π L sup y ďr 1 ∇ M f pyq ă c 2 m 2 . Consider y, δ as in the lemma. By Proposition 6.2.3 there is x 0 P Bpy, δ 2 q s.t. f px 0 q ă f pP M px 0 qq. Let x : R `Ñ R d be in S ´Bf px 0 q and define t r 1 :" inftt ě 0 : xptq ě r 1 u. By Proposition 6.2.2 we have that t r 1 ă `8. For t ď t r 1 , we have:

f pxptqq ď f pxp0qq ´c2 m t .
Denoting yptq " P M pxptqq, for t ď t r 1 , by path-differentiability of f we have: (6.16) This implies that ypt r 1 q " P M pxpt r 1 qq ‰ xpt r 1 q and, therefore, xpt r 1 q " r 1 . Hence,

f
x 0 ´y ď δ 2 ă δ ă r 1 ´ y " xpt r q ´ y ď xpt r q ´y . Since x is continuous, this implies that there is t δ P p0, t r q s.t. xpt δ q ´y " δ. Denote x " xpt δ q and notice that xpt δ q ´x0 ď Lt δ . Hence, δ " xpt δ q ´y ď x 0 ´y ` xpt δ q ´x0 ď δ 2 `Lt δ .

Therefore, t δ ě δ 2L and by Inequality (6.16) we have:

f pxq ď f pP M pxqq ´tδ c 2 m 2 ď f pP M pxqq ´c2 m δ 4L ,
which proves the first statement. Finally, for x 1 P Bpx, δ 1 q, we have:

f px 1 q ď f pxq `L x ´x1 ď f pP M pxqq ´c2 m δ 4L `Lδ 1 ď f pP M px 1 qq `pL `LM qδ 1 ´c2 m δ 4L .
With our choice of δ 1 the last inequality implies that f px 1 q ă f pP M px 1 qq.

Lemma 6.4.10. Let Assumptions 6.3.1-6.3.4 hold. Consider κ from Proposition 6.3.2 and r 1 from Lemma 6.4.9. For n large enough, there is κ,r 1 ą 0 such that:

P prf px n`1 q ă f pP M px n`1 qqs|F n q 1 distpxn,M qďκγn 1 P M pxn´γnvnq ď 1 2 r 1 1 xn ďr 1 ě κ,r 1 .

Proof. The set-valued mapping Ψ n : R d Ñ R d , defined as:

Ψ n pxq " "

x : x ´PM px n ´γn v n q " γ n , f pxq ď f pP M pxqq ´c2 is closed valued and, by Lemma 6.4.9, for n large enough, it is nonempty on r P M px n ´γn v n q ď 1 2 r 1 s. Hence, by [Rockafellar & Wets 1998, Corollary 14.6], we can choose x 1 n in a measurable way such that x 1 n P Ψpx n q. Define ψ n : R d Ñ R as ψ n pxq " x 1 n ´x γn `vn and δ 1 " c 2 m 4pL`L M qL . By Lemma 6.4.9 we have that for all x 1 P Bpx 1 n , γ n δ 1 q: f px 1 q ă f pP M px 1 qq .

Denote S u the set tx : f pxq ă f pP M pxqqu. On the event rdistpx n , M q ď γ n κs X r P M px n ´γn v n q ď r 1 2 s X r x n ď r 1 s, using Assumption 6.3.4, we have:

Ppx n`1 P S u |F n q ě Ppx n ´γn v n `γn η n`1 P Bpx 1 n , γ n δ 1 q|F n q ě Ppγ n η n`1 P Bpx 1 n ´xn `γn v n , γ n δ 1 q|F n q ě Ppη n`1 P Bpψ n px n q, δ 1 q|F n q ě ż uPR d 1 Bpψnpxnq,δ 1 q puqh r 1 puq du (6.17) Denote L 1 " L `1 `Lπ L `κ. We have:

ψ n px n q ď v n `
x 1 n ´PM px n ´γn v n q ` P M px n ´γn v n q ´PM px n q ` x n ´PM px n q γ n ď L `1 `Lπ L `κ " L 1 .

Therefore, by Assumption 6.3.4, on the event rdistpx n , M q ď γ n κsXr P M px n ´γn v n q ď r 1 2 s X r x n ď r 1 s we have:

Ppx n`1 P S u |F n q ě inf

x ďL 1 `δ1 h r 1 pxq ż R d 1 Bp0,δ 1 q puq du .

Since h r 1 is positive and continuous, the infimum in the last inequality is positive, which finishes the proof.

To finish the proof of Theorem 6.3.3 we will use the following lemma.

Lemma 6.4.11 ([Borkar 2008, Chapter 4, Lemma 14]). Consider pΩ, A, Pq a probability space and pF n q a filtration. Let pF n q, pH n q be two sequences of events adapted to pF n q and assume that there is a constant C ą 0 such that:

PpF n`1 |F n q1 Hn ě C .

Then

PprF n occurs infinitely oftens c X rH n occurs infinitely often sq " 0 .

By Proposition 6.3.2 we know that on the event rx n Ñ 0s the event rdistpx n , M q ď γ n κs X r P M px n ´γn v n q ď 1 2 r 1 s will happen infinitely often. Therefore, by Lemmas 6.4.11 and 6.4.10 the event rf px n q ă f pP M px n qqs happens infinitely often. By Proposition 6.3.1 this can happen only with probability zero. Hence, Pprx n Ñ 0sq " 0.

6.4.5 Proof of Proposition 6.3.6

The validity of Assumption 6.3.2 is immediate. To prove Assumption 6.3.4 denote Q : R d ˆBpR d q Ñ R `the Markov kernel of η n`1 . For every px, Aq P R d ˆBpR d q, it is defined as: Qpx, Aq " Ppη n`1 P A|x n " xq " ż 1 A pGpxq ´gpx, sqqµpdsq .

The Markov kernel of ηn`1 , denoted Q, is then defined for every px, Aq P R d ˆBpR d q, as:

Qpx, Aq " Ppη n`1 P A|x n " xq " Qpx n , dyq " Pp η n`1 ď a| Ă F n q ě 1 ´KpCq a 2 ě 1 2 .

Therefore, for a measurable function ψ : R d Ñ R d and C, δ ą 0, if x n ď C, then:

Ppη n`1 P Bpψpx n q, δq| Ă F n q ě 1 2 ż R d

1 Bpψpxnq,δq puq inf y ď ? 2C

h 1 pu ´yq du .

A simple exercise shows that the function u Þ Ñ inf y ď ?

2C h 1 pu ´yq is continuous and positive. Hence, Assumption 6.3.4 is verified. Stochastic proximal subgradient descent oscillates in the vicinity of its accumulation set

Introduction

Let d be a positive integer, let X be a nonempty, closed and convex set and let f, g : R d Ñ R be two locally Lipschitz functions. In this chapter, we study the behavior of the stochastic proximal subgradient descent (SPGD):

x n`1 P prox γn g,X px n ´γn v n `γn η n`1 q , (7.1)

where prox γn g,X is the proximal operator for the function g on X (see Equation (7.7) for a definition), pγ n q is a sequence of stepsizes, pη n q is a noise sequence and for each n P N, v n is in the set Bf px n q of Clarke's subgradients of f at x n .

Let N X pxq be the normal cone of X at x. It is known (see [START_REF] Davis | [END_REF], Majewski et al. 2018]) that, under mild conditions on f , g and pη n q, every limit point of px n q is included in the set Z :" tx : 0 P Bf pxq `Bgpxq `NX pxqu. The proof leans on the seminal paper of Benaïm, Hofbauer and Sorin [Benaïm et al. 2005] (see also [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF]), which analyzes Equation (7.1) as an Euler-like discretization of the differential inclusion (DI): 9 xptq P ´Bf pxptqq ´Bgpxptqq ´NX pxptqq .

(7.2)

While the sequence px n q is known to converge to Z, recent work [Rios-Zertuche 2020] shows that in principle, it might not converge to a unique point. In [Rios-Zertuche 2020, Section 2] Ríos-Zertuche considers the deterministic subgradient descent (that is to say g " 0, X " R d , η n " 0) and constructs f , which verifies main assumptions of nonsmooth optimization (such as Whitney stratifiability or Kurdyka-Łojasiewicz inequality) but the limit set of px n q is equal to Z " tx : x " 1u. This encourages a more precise study of Equation (7.1).

In [Bolte et al. 2020b] the authors, using the theory of closed measures, show that in the case of the deterministic subgradient descent the convergence to Z arises in a structured manner. First, they prove that if x, y are two distinct accumulation points of px n q, then the time that the iterates spend to get from a neighborhood of x to a neighborhood of y goes to infinity. Second, in a first approximation their results imply that if x is an accumulation point of px n q, then ř n i"1 γ i v i 1 x i PBpx,δq ř n i"1 γ i 1 x i PBpx,δq ÝÝÝÝÑ nÑ`8 0 , (see [Bolte et al. 2020b, Theorem 7] or Section 7.3 for a precise statement). Intuitively speaking, this means that even if x n ´x0 " ř n i"0 γ i v i does not converge, on average, the drift coming from the subgradients compensate itself and vanishes at infinity. This behavior captures an oscillation phenomenon of the iterates around the critical set. Results of this type show a strong stability property of the deterministic subgradient descent.

In practical settings, when the function f is either unknown or computation of its gradient is expensive, the deterministic gradient descent is often replaced by its stochastic version, in many cases, this may lead to a faster convergence (see e.g. [Bottou et al. 2018]). Proximal methods, on the other hand, along with the regularizer function g, are widely used to regularize the initial problem of minimizing f . Depending on the choice of g, we can, for instance, preserve the boundedness of the iterates [START_REF] Duchi | [END_REF] or promote the sparsity of solutions [Tibshirani 1996]. It is therefore interesting to establish stability results of the type [Bolte et al. 2020b] for the SPGD.

In this chapter we investigate further the questions of oscillations of the SPGD. Our contributions are threefold. First, we show that the time spent by the SPGD to move from one accumulation point to another goes to infinity. Second, we establish an oscillation-type behavior of the drift. These two results extend [Bolte et al. 2020b, Theorem 7.] to a stochastic and a proximal setting. Finally, our technique of proof doesn't rely on the theory of closed measures used in [Bolte et al. 2020b] but is build upon the classical work of Benaïm, Hofbauer and Sorin [Benaïm et al. 2005]. We feel that this approach gives a simpler proof and allows us to treat the deterministic, the stochastic and the proximal cases in a unified manner.

Chapter organization. In Section 7.2, we recall some known facts about the DI (7.2) and its Lyapounov function. Our main results are given in Section 7.3. Section 7.4 is devoted to proofs.

Preliminaries

Notations

For S Ă R d , we denote cl S its closure and conv S its closed convex hull. For a function F : R d Ñ R, we denote ∇F its gradient. Constants will usually be denoted as C, C 1 , C 2 . . . , they can change from line to line. For a sequence px n q, we denote acctx n u its set of accumulation points. The space of continuous functions from R to R d will be denoted as CpR `, R d q, we endow this set with d C the metric of uniform convergence on compact intervals (see Section 2.2.1). Given a convex set X Ă R d , the normal cone of X is a set valued map N X : R d Ñ R d , defined as: N X pxq " tv : xv, y ´xy ď 0, @y P X u .

(7.3)

For each x P X , N X pxq is a closed convex subset of R d .

A Lyapounov function for the differential inclusion

We (7.6) This implies that pf `gqpxptqq ă pf `gqpxp0qq if xp0q R Z. In other words, f `g is a strict Lyapounov function for the DI (7.2).

Main results

Consider pΩ, Ξ, Pq a probability space and pη n q a sequence of random variables with values in R d . Define prox γ g,X : R d Ñ R d , the proximal operator for g on X with a step γ: prox γ g,X pxq " arg min yPX tgpyq `1 2γ y ´x 2 u . (7.7)

We study Equation (7.1) under the following assumptions.

Assumption 7.3.1.

i) The set X is a closed convex subset of R d .

ii) The functions f, g : R d Ñ R are locally Lipschitz continuous.

iii) There is a filtration pF n q nPN , such that pη n q is a martingale difference sequence adapted to it, and x n is F n measurable for every n P N.

iv) The sequence of stepsizes pγ n q is nonnegative and such that ř `8 i"0 γ i " `8.

Proposition 7.3.1. Let Assumptions 7.3.1 and 7.3.2 hold, then the family pXpt qq tě0 is relatively compact. Moreover, if a sequence t n Ñ `8 and x P CpR `, R d q is such that d C pXpt n `¨q, xq Ñ 0, then x is a solution to the DI (7.2).

Proposition 7.3.2. Under Assumptions 7.3.1-7.3.4, the set acctx n u is included in Z and f `g is constant on acctx n u.

The next theorem tells us that even if acctx n u is not a single point, the time that it takes to px n q to go from one accumulation point to another goes to infinity. This is an extension of [Bolte et al. 2020b, Theorem 6.i), Theorem 7.i)], to the best of our knowledge this result is new in a stochastic and proximal setting.

Theorem 7.3.3. Let Assumptions 7.3.1-7.3.4 hold. Let x, y be two distinct points in acctx n u. Consider two sequences n i , n j , with n i ď n j , such that x n i Ñ x and x n j Ñ y. Then τ n j ´τn i Ñ `8. Under Assumptions 7.3.1-7.3.3, the same result is true if pf `gqpxq ď pf `gqpyq.

As it is shown in [Rios-Zertuche 2020], it is possible that acctx n u is not reduced to a unique point. Nevertheless, Theorem 7.3.3 implies that the "nonconvergence" happens in a very slow manner. Asymptotically, the time spent by the algorithm to move from one accumulation point to another goes to infinity.

We now investigate the question of oscillations. Given U, V two open sets, such that cl U Ă V , we will call I " rn 1 , n 2 s a maximal interval related to U, V if the set X n 2 n 1 :" tx n 1 , x n 1 `1, . . . , x n 2 u is such that X n 2 n 1 Ă V , X n 2 n 1 X U ‰ H and either x n 1

´1

or x n 2 `1 is not in V . The next two results are an extension of [Bolte et al. 2020b, Theorem 7] to a stochastic setting.

Theorem 7.3.4 (Long intervals). Let Assumptions 7.3.1-7.3.4 hold. Consider x P acctx n u and U, V two neighborhoods of x such that cl U Ă V . For i P N, denote I i " rn 1i , n 2i s a sequence of distinct maximal intervals related to U, V . Then, either one of I i is unbounded or τ n 2i ´τn 1i Ñ `8.

Theorem 7.3.5 (Oscillation compensation). Let Assumptions 7.3.1-7.3.4 hold, and fix U , V and I i as in Theorem 7.3.4. Denote A " Ť I i , then

ř n i"1 γ i pv i `vg i `vX i q1 A px i q ř n i"1 γ i 1 A px i q ÝÝÝÝÑ nÑ`8 0 .
(7.11) Theorem 7.3.5 gives an intuitive explanation of why Theorem 7.3.3 holds. Indeed, while the drift coming from one iteration v i `vg i `vX i might not go to zero (as it happens for such a simple example as f pxq " x , g " 0 and X " R d ), on average, it compensates itself. Theorem 7.3.3 and 7.3.5 suggest that the algorithm oscillates around its accumulation set, while the center of these oscillations moves in acctx n u with a vanishing speed.

Let us finish with a remark on the Equation (7.11). At first sight, maximal intervals in Theorem 7.3.5 and Theorem 7.3.4 may seem artificial. A more satisfactory result would be ř n i"1 γ i pv i `vg i `vX i q1 U px i q ř n i"1 γ i 1 U px i q ÝÝÝÝÑ nÑ`8 0 , (7.12)

Lipschitz functions g, f : R d Ñ R and a bounded set-valued map r N X : R d Ñ R d s.t. in Equation (7.8) we have v n pwq P B f px n pwqq, v g n pwq P Bgpx n`1 pwqq and v X n pwq P r N X px n`1 pwqq. Moreover, if x is a solution to the DI: 9 xptq P ´B f pxptqq ´Bgpxptqq ´r N X pxptqq , (7.15)

and that x remains in Bp0, Cq X X , then x is a solution to the DI (7.2). Finally, denoting r Z " tx : 0 P B f pxq `Bgpxq `r N X pxqu, we have the equality r Z X Bp0, Cq " Z X Bp0, Cq.

Proof. Let Π C`1 : R d Ñ R d be the projection on Bp0, C `1q. Define f pxq " f pΠ C`1 pxqq, gpxq " gpΠ C`1 pxqq. By construction, we have that v n P B f px n q and v g n P Bgpx n`1 q and that v n , v g n are bounded by L f , L g the Lipschitz constants of f and g. Hence, since x n`1{2 is bounded, there is C 2 s.t. supt v X n : n P Nu ă C 2 . Defining r N X pxq " tv : v ď maxpC 2 , L f , L g q, v P Π X pxqu, where Π X is a projection on X , proves the first claim. The two other statements immediately follow from our construction.

We say that an a.c. curve x : R `Ñ R d is a perturbed solution to the DI (7.15) if there is ρ : R `Ñ R `and a locally integrable function b : R `Ñ R d s.t. for almost every t ě 0, we have: 9 xptq ´ρptq P ´B f bptq pxptqq ´Bg bptq pxptqq ´r N bptq X pxptqq , where H δ pxq " tv P Hpyq : y ´x ď δu, lim tÑ`8 bptq " 0 and for every T ą 0, we have: If x is a bounded perturbed solution to (7.15), then by [Benaïm et al. 2005, Theorem 4.2] it is also an APT of (7.15). Thus, to prove Proposition 7.3.1 it remains to show that X is a perturbed solution to the DI (7.15).

For t P rτ pnq, τ pn `1qq, we define ρptq " η n`1 and bptq " x n`1 ´xn . The condition on ρ immediately follows from Lemma 7.4.1. The condition on b follows from the following lemma. Proof. By Lemma 7.4.1, we have that x n`1{2 ´xn ÝÝÝÝÑ nÑ`8 0. Moreover, we have:

gpx n`1 q `1 2γ n x n`1 ´xn`1{2 2 ď gpx n q `1 2γ n x n ´xn`1{2 2 .
Therefore, 1 2γ n

x n`1 ´xn 2 ď gpx n`1 q ´gpx n q ´1 γ n xx n`1 ´xn , x n ´xn`1{2 y ď x n`1 ´xn ˜Lg `

x n ´xn`1{2 γ n ¸, and

x n`1 ´xn ď γ n L g ` x n ´xn`1{2 , which finishes the proof.

To finish the proof of Proposition 7.3.1 consider t n Ñ `8 and x s.t. d C pXpt n q, xq Ñ 0. Then, by [Benaïm et al. 2005, Theorem 4.2], x is a solution to the DI (7.15), moreover, it remains in Bp0, Cq X X , therefore, it is also a solution to the DI (7.2).

For the proof of Proposition 7.3.2, notice that f `g is path differentiable (as a composition of path differentiable functions). Then, in the same way as in Section 7.2.2, we have that f `g is a strict Lyapounov function for the DI (7.15) and for the set Z. Since acctx n u " L X Ă cl Bp0, Cq, by Proposition 2.2.3 we have that L X Ă r Z X cl Bp0, Cq Ă Z, and that f `g is constant on acctx n u.

Remark 22. Strictly speaking, following [Benaïm et al. 2005], a perturbed solution to the DI is of the form 9 xptq ´ρptq P H bptq pxptqq, where H " ´B f ´Bg ´Ñ X . Nevertheless, the proof of [Benaïm et al. 2005, Theorem 4.2] goes through with our definition.

7.4.2 Proof of Theorem 7.3.3 Lemma 7.4.4. Let Assumptions 7.3.1-7.3.3 hold, let τ n be a positive sequence, with τ n Ñ `8, and x s.t. Xpτ n `¨q Ñ x, then pf `gqpxphqq ď pf `gqpxp0qq, @h P R `.

(7.16)

Moreover, if for some h ě 0, pf `gqpxphqq " pf `gqpxp0qq, then xph 1 q " xp0q for every h 1 P r0, hs. If additionally Assumption 7.3.4 holds, then:

xphq " xp0q, @h P R `.

(7.17) (7.18) This implies that ş h 0 9 xpuq 2 du " 0. Hence, 9 xphq " 0 for almost every h P r0, T s and we obtain Equation (7.17).

Proof
Suppose that there is T ą 0 such that τ n j ´τn i ď T . The sequence Xpτ pn j q`¨q is relatively compact, and after extraction it converges to x a solution to (7.2). Extract once again to have τ n j ´τn i Ñ h. Then Xpτ pn j qq ´Xpτ pn i qq Ñ xphq ´xp0q " y ´x , and we obtain a contradiction with Lemma 7.4.4. Proof. Suppose that we have ε ą 0 and two sequences n k and n k ď j k ď N pT, n k q, such that for n k large enough:

j k ÿ i"n k γ i pv i `vg i `vX i q ą ε .
This implies:

x j k ´xn k `jk ÿ i"n k γ i η i`1 ą ε .
Extract a sequence such that Xpτ n k `¨q converges to x and τ j k ´τn k Ñ h, with h ď T . Then x j k Ñ xpT 1 q and x n k Ñ xp0q, but xpT 1 q ´xp0q ě ε which is impossible by Lemma (7.4.4).

Suppose that no I i is unbounded, then we can choose n i P I i " rn 1i , n 2i s such that x n i P U . Since x n 2i`1 is in V c , after extraction x n i Ñ y 1 and x n 2i`1 Ñ y 2 , with y 2 ‰ y 1 , moreover:

τ n 2i`1 ´τn i ´γn 2i`1 ď τ n 2i ´τn 1i .

(7.19) By Theorem 7.3.3, the first term of this inequality tends to infinity.

7.4.4 Proof of Theorem 7.3.5

Take I i as in Theorem 7.3.4, and A N " Ť iďN I i . Define

u N " a N b N " ř `8 i"0 γ i pv i `vg i `vX i q1 A N px i q ř `8 i"0 γ i 1 A N px i q .
Then,

u N `1 " a n `ř`8 i"0 γ i pv i `vg i `vX i q1 I N `1 px i q b N `ř`8
i"0 γ i 1 I N `1 px i q .

(7.20)

Fix ε ą 0, by Lemma 7.4.5, there is n 0 such that, for n k ě n 0 and j k ď N pT, n k q, ř j k i"n k γ i pv i `vg i `vX i q ď ε. Decompose I i " rn 1i , n 2i s " Ť 1ďkďK i ra ki , a ki`1 s, with a 1i " n 1i and a ki`1 " mintN pT, a ki q, n 2i u. We obtain:

u N `1 " a N `řkďK N ř a kN `1 i"a kN γ i pv i `vg i `vX i q b N `řkďK N ř a kN `1 i"a kN γ i ď a N `pK N qε b N `pK N ´1qT
.

  R d `ˆR d ˆRd , the differential equation is written as follows:

  Definition 2.1.3. Consider f : R d Ñ R a locally Lipschitz continuous function and x P R d . The set Bf pxq of Clarke subgradients of f at x is defined as follows Bf pxq " convtB L f pxqu , where conv denotes the convex hull.

  2. IfA and B are in O n , then the same is true for A X B, A Y B and A c . 3. If A P O n and B P O m , then A ˆB P O n`m . 4. If A P O n , then the projection of A on its first (n ´1) coordinates is in O n´1 . 5. Every element of O 1 is exactly a finite union of intervals and points.

  Assumption 3.4.1. The functions F and S belong respectively to C 3 pR d , Rq and C 2 pR d , R d `q.

"8

  Cov p∇f px ‹ , ξqq. The next step is to triangularize the matrix H in order to decouple the blocks of Γ. For every k " 1, . . . , d, set ν {4 ´h8 π k with the convention that ?

  to this setting. The proof of Theorem 3.4.1 relies on a nonautonomous version of Poincaré's theorem. We borrow this result from the rich literature that exists on the subject [Dalec 1 kiȋ & Kreȋn 1974, Kloeden & Rasmussen 2011]. Let us start by setting the context for the non-autonomous version that we shall need for the invariant manifold theorem. Given an integer d ą 0 and a matrix D P R dˆd , consider the linear autonomous differential equation 9 zptq " Dzptq, (3.59) which solution is obviously zptq " e Dt zp0q for t P R. Let us factorize D as in (3.12), and write D " QΛQ ´1 with Λ " " Λ ´Λ` where we recall that the Jordan blocks that constitute Λ ´P R d ´ˆd ´(respectively Λ `P R d `ˆd `) are those that contain the eigenvalues λ i of D such that λ i ď 0 (respectively λ i ą 0). Let us assume here that both d ´and d `are positive. It will be convenient to work in the basis of the columns of Q by making the variable change

  3.7.3 Proofs for Section 3.4.2.1 3.7.3.1 Proof of Lemma 3.4.2

  Definition 4.3.1. Assume that f p¨, sq is locally Lipschitz continuous for every s P Ξ. A function ϕ : R d ˆΞ Ñ R d is called an almost everywhere (a.e.)-gradient of f if ϕ " ∇f λ b µ-almost everywhere.

nÑ8P

  ν pdpx γ n , Zq ą εq Ý ÝÝ Ñ

  Ruan 2018, Schechtman 2021a]. Namely, for every T ą 0, sup hPr0,T s inf xPS ´Bf pXptqq Xpt `hq ´xphq Ý ÝÝÝ Ñ tÑ`8 0 .

  ]. Lemma 6.4.2 ([Bolte et al. 2009, Theorem 1 and Proposition 1]). Consider f : R d Ñ R a locally Lipschitz continuous and definable function. For px, dq P R d Rd

  Dw P W, Dv P Bf pxq such that:f pxq ´f pP M pxqq ě α x ´PM pxq `xw, x ´PM pxqy xv ´w, x ´PM pxqy ă β x ´PM pxq .

  ´yqQpx, dyq du .Fix C ą 0, notice that if xn ď C, then by Markov's inequality we have for a

  Lemma 7.4.3. Under Assumptions 7.3.1 and 7.3.2, almost surely, we have that x n`1 ´xn ÝÝÝÝÑ nÑ`8 0.

  The next lemma is the key ingredient for the proofs of Theorem 7.3.4 and Theorem 7

  SGD on a generic definable function converges to a minimizer 11 angle conditions. Naturally, one might want to know what happens when the weak convexity assumption fails. The first part of Chapter 6, which is based on the publication {1}, addresses this question by classifying the generic Clarke critical points of locally Lipschitz continuous, definable functions. Specifically, given such a function F : R d Ñ R it analyzes the type of points that might appear in:

F : R d Ñ R a weakly convex, definable function, for almost every u P R d , the critical points of F u are either local minima or active strict saddles lying on active manifolds satisfying the Verdier and the 1.4.

  Assumption 3.2.1. The function F belongs to C 1 pR d , Rq and ∇F is locally Lipschitz continuous.Note that this assumption implies that the infimum F ‹ of F is finite, and the set zer ∇F of zeros of ∇F is nonempty.

	Assumption 3.2.2. F is coercive, i.e., F pxq Ñ `8 as }x} Ñ `8.
	Assumption 3.2.3. The map S : R d Ñ R d `is locally Lipschitz continuous.
	Assumption 3.2.4. The continuous functions h, r, p, q : p0, `8q Ñ R `satisfy:

  , Equation (2.3)]. Lemma 3.5.3. Let pm, xq be a solution to (ODE-N). Define yptq "

	x	`κ? t ˘, with κ "	?	2α `2 and β " κ 2 4 . Then, py, uq verifies	κmpκ ? tq 2 ? t	, uptq "
								#	9 yptq " β t p∇F puptqqq ´yptqq 9 uptq " ´yptq .	(3.22)
	Proof. By simple differentiation, we get:
	9 yptq "	β t	"	∇F ´xpκ	? tq ¯´α κ ? t	m	´κ?	t ¯	´κ 4t 3 2	m	´κ? t	¯" β t	p∇F puptqq ´yptqq ,
								9 uptq "	´κ 2 ? t	m	´κ?	t ¯" ´yptq .

  Notice that this quantity is bounded as a function of the variable t. For any map W : Z `Ñ R, define for all t ě t 0 , L W ptq Spxq} 2 for every z " pv, m, xq P Z `. Consider δ ą 0 (to be specified later on). We study L W δ " L E8 `δL G `δL H . Note that Φ 8 pz, tq P S X Z `for all t ě t 0 by an analogous result to Lemma 3.5.1 for Φ 8 . Thus, t Þ Ñ E 8 pΦWe now study L G . For every t ě t 0 , ∇F be the Lipschitz constant of ∇F on the bounded set tx : pv, m, xq P Su.

	3.6. Proofs for Section 3.3			59
	L G ptq " lim sup	s ´1p´x∇F pxpt `sqq, mpt `sqy `x∇F pxptqq, mptqyq
		sÑ0
	ď lim sup	s ´1}∇F pxptqq ´∇F pxpt `sqq}}mpt `sq} ´x∇F pxptqq, 9 mptqy .
		sÑ0
	Let L Define C 1	∆ " sup t } a	vptq `ε}. Then,
	L G ptq ď L ∇F lim sup	s ´1}xptq ´xpt `sq}}mpt `sq} ´x∇F pxptqq, 9 mptqy
			sÑ0
		ď L ∇F }9 xptq}}mptq} ´x∇F pxptqq, 9 mptqy
								∆ " lim sup sÑ0 s ´1pWpΦ 8	pz, t `sqq	ẂpΦ
								8	pz, tqq is differentiable
	at any point t ě t 0 and L E8 ptq " d dt E 8 pΦ	8	pz, tqq. Using similar derivations to
	Inequality (3.16), we obtain that
		L E8 ptq ď ´´r 8	´q8 4	¯› › › › ›	mptq pvptq `εq d 1 4	› › 2 › › ›	.	(3.35)

t. W δ is a strict Lyapunov function for the semiflow Φ 8 . Proof. The first point is a consequence of the definition of S and the boundedness of z. The second point stems from the definition of Φ 8 . Observing that Φ 8 is valued in S, the third point is immediate from the definition of Λ Φ 8 . We now prove the last point. Consider z P S and write Φ 8 pz, tq under the form Φ 8 pz, tq " pvptq, mptq, xptqq. 8 pz, tqqq . Introduce Gpzq ∆ " ´x∇F pxq, my and Hpzq ∆ " }q 8 v ´p8 ď L ∇F }9 xptq}}mptq} ´h8 }∇F pxptqq} 2 `r8 x∇F pxptqq, mptqy

  pxptqq} 2 `}p 8 Spxptqq ´q8 vptq} 2 ' .(3.39) It can easily be seen that for every z P S, t Þ Ñ W δ pΦ 8 pz, tqq is Lipschitz continuous, hence absolutely continuous. Its derivative almost everywhere coincides with L W δ , which is nonpositive. Thus, W δ is a Lyapunov function for Φ W δ pzq for all t ě t 0 . The derivative almost everywhere of t Þ Ñ W δ pΦ

	8	pz, tqq is
	identically zero, and by Equation (3.39), this implies that

8

. We prove that the Lyapunov function is strict. Consider z " pv, m, xq P S s.t. W δ pΦ 8 pz, tqq "

  , Lemma 4]. There exists n 1 ě n 0 s.t. w n ď M . As v Þ Ñ 1 is Lipschitz and ∇F and S are locally Lipschitz, for every z " pv, m, xq and δ s.t. }z ´z‹ } ď M and }δ} ď M , we have }r n pz, δq} ď Cγ n`1 }pv `δ `εq d´1 2 }}m} `C}pv `δ `εq d´1 2 ´pv `εq d´1 2 }}m}

	? v`ε

ď Cγ n`1 }z ´z‹ } `C}δ}}z ´z‹ } .

This implies that for every n ě n 1 , }r n p ζn , δ n q}1 An ď Cpγ n`1 `wn q} ζn ´z‹ } .

(3.52)

  }r n p ζn , δ n q} 2 q1 An If M is chosen small enough, }g n p ζn q ´Gn p ζn ´z‹ q}1 } ζn´z‹}ďM ď c 2 }B t } ´1}B ´1 t }} ζn ´z‹ } .

	Choose c
	Moreover, choosing K ą sup n }G	ptq n } 2 , it holds that Re ´Sn pK	`Gptq n qS n ¯ě 0.
	Then,

`2γ n`1 Re ´Sn B t ´gn p ζn q ´Gn p ζn ´z‹ q ¯¯1 } ζn´z‹}ďM ´2γ n`1 Re ´Sn pK `Gptq n qS n ¯1} ζn´z‹}ąM `2γ n`1 Re ´Sn B t r n p ζn , δ n q ¯1An . ∆ " pA 1 ´Aq{2.

  R d ´ˆI Ñ R d `.In addition, if the partial derivatives B k 1 ε : R d ˆI exist and are continuous for k P t1, . . . , mu with globally bounded partial derivatives Finally, if B n 2 B k 1 ε exist and are continuous for 0 ď n ă m and 0 ď k `n ď m, then w is m-times continuously differentiable.

		|ε| k	∆ " sup py,tqPR d ˆI }B k 1 εpy, tq} ă 8 ,	(3.64)
	under the gap condition
					mα ´ă α `, m P N ˚,	(3.65)
	the partial derivatives B k 1 w : R d ´ˆI exist and are continuous with
		sup py ´,tqPR d ´ˆI	}B k 1 wpy ´, tq} ă 8 for all k P t1, . . . , mu.	(3.66)
	Let us partition the function hpy, tq as
		hpy, tq "	"	h ´py, tq h `py, tq		"
		"
	G "	pt, yq P I ˆRd : sup	}φps, t, yq} exppγpt ´sqq ă 8
					sět

*

is nonempty, and does not depend on γ. Moreover, this set is a global invariant manifold for the ODE (3.62) that is defined by a continuously differentiable mapping w :

  d ´of zero such that R d´ˆI Ñ R d , py ´, tq Þ Ñ py ´, wpy ´, tqq, Equation (3.68) can be rewritten as B 2 wpy ´, tq " h `pgpy ´, tq, tq ´B1 wpy ´, tqh ´pgpy ´, tq, tq.(3.72) By Proposition 3.7.1, the function w is twice differentiable, and we can writeB 2 2 wpy ´, tq " B 1 h `B2 g `B2 h `´pB 1 B 2 wqh ´´pB 1 wqpB 1 h ´B2 g `B2 h ´q, (3.73) where, e.g., h `is a shorthand notation for h `pgpy ´, tq, tq. It holds from Equation (3.67) and the assumptions of Proposition 3.7.1 that for each py, tq P R d ˆI, }B 1 hpy, tq} ď }Λ} `}B 1 εpy, tq} ď C, (3.74)where the constant C ą 0 is independent of py, tq and can change from an inequality to another in the remainder of the proof. By the mean value inequality and Proposition 3.7.1, we also get that }wpy ´, tq} " }wpy ´, tq ´wp0, tq} ď sup ´Ă R d ´be a small enough neighborhood of zero so that gpy ´, tq P V for each y ´P V ´, which is possible by the inequality }gpy ´, tq} ď C}y ´}. By the assumption on }B 2 εpy, tq} in the statement of Lemma 3.7.2, we have @y ´P V ´, B 2 hpgpy ´, tq, tq " B 2 εpgpy ´, tq, tq ď C.

			(3.77)
	sup py ´,tqPV ´ˆI The bound (3.70) is an immediate consequence of Equation (3.76). Getting back to B 1 B 2 wpy ´, tq ă `8 , (3.70) Equation (3.73), the bound (3.71) follows from the inequalities (3.74)-(3.77).
	sup py ´,tqPV ´ˆI	B 2 2 wpy ´, tq ă `8 .	(3.71)
	Proof. By Proposition 3.7.1, the general solution φps, t, yq of the ODE (3.62) can be

written as φps, t, yq " pφ ´ps, t, yq, φ `ps, t, yqq with φ `ps, t, yq " wpφ ´ps, t, yq, sq for each s P I. Equating the derivatives with respect to s of the two members of this equation and taking s " t, we get the first equation. Writing g : pu,sq }B 1 wpu, sq} }y ´} ď C}y ´}, thus, }gpy ´, tq} ď C}y ´}. By the mean value inequality again, hpgpy ´, tq, tq " hpgpy ´, tq, tq ´hp0, tq ď sup pu,tq B 1 hpu, tq gpy ´, tq ď C gpy ´, tq ď C}y ´}. By Equation (3.72) and Proposition 3.7.1, this implies that B 2 gpy ´, tq " B 2 wpy ´, tq " h `´pB 1 wqh ´ ď C y ´ , and (3.75) B 1 B 2 wpy ´, tq " B 1 h `B1 g ´pB 2 1 wqh ´´pB 1 wqpB 1 h ´B1 gq ď Cp y ´ `1q. (3.76) Let V

  t. the mapping w : R d ´ˆI 1 Ñ R d `obtained from Proposition 3.7.1 (applied to h and ε) satisfies Let C " max y }∇ψpyq} where ∇ψ is the Jacobian matrix of ψ. Thanks to the convergence (3.78), we can choose t 1 ą 0 large enough and σ ą 0 small enough so that and we set I 1 " rt 1 , 8q. Writing εpy, tq " ψpy{σqεpy, tq, it holds that for each pt, yq P I 1 ˆRd , }B 1 εpy, tq} ď σ ´1C1 }y}ď2σ }εpy, tq} `1}y}ď2σ }B 1 εpy, tq}

	|w| 1 "	sup
					(3.80)
	sup py ´,tqPBp0,σ 1 qˆI 1	B 2 2 wpy ´, tq ă `8 .	(3.81)
	sup pt,yqPrt 1 ,8qˆBp0,2σq	}B 1 εpy, tq} ă	4Kp1 `2Cq α `´α ,
	ď ˆmax }y}ď2σ	}B 1 εpy, tq} ˙`σ ´1C}y}	`1˘1	}y}ď2σ
	ď	α `´α	4K

py,tqPR d ´ˆI 1 }B 1 wpy, tq} ă δ .

(3.79) Furthermore, Equation (3.68) holds for h and w for all py, tq P Bp0, σq ˆI1 . If, additionally, Equation (3.69) holds for ε, then there exists σ 1 ď σ such that

sup py ´,tqPBp0,σ 1 qˆI 1 B 1 B 2 wpy ´, tq ă `8 ,

Proof. The idea of the proof is to localize the function hpy, tq to a neighborhood of zero in the variable y for the purpose of applying Proposition 3.7.1. This cut-off technique is known in the non-autonomous ODE literature, see, e.g.,

[Kloeden & Rasmussen 2011, Theorem 6.10

]. Let ψ : R d Ñ r0, 1s be a smooth function such that ψpyq " 1 if }y} ď 1, and ψpyq " 0 if }y} ě 2.

  Rasmussen 2011, Theorem 6.3] corresponds to |ε| 1 with our notations). Using Equation (3.78), we can make |ε| 1 as small as needed by choosing σ, t 1 respectively small and large enough, which gives us Equation (3.79). The proof of the last two equations follows from the application of Lemma 3.7.2 to h and w. The result is immediate after noticing that for py, tq P R d ˆI1 , we have B 2 εpy, tq ď B 2 εpy, tq .

  . A set-valued map D : R d Ñ R d is called a conservative field, if for each x P R d , Dpxq is a nonempty and compact subset of R d , D has a closed graph, and for each absolutely continuous a : r0, 1s Ñ R d , with ap0q " ap1q, it holds that: If F is path differentiable, then the Clarke subdifferential BF is a conservative field for the potential F [Bolte & Pauwels 2019, Corollary 2]. Another useful example of a conservative field for composite functions is the automatic differentiation field [Bolte & Pauwels 2019, Section 5]. A broad class of functions used in optimization are path differentiable, e.g. any convex, concave, regular or tame.

	ż 1			ż 1
	0	min vPDpaptqq	x 9 aptq, vy dt "	0	max vPDpaptqq	x 9 aptq, vy dt " 0 .
		F pxq " F p0q	`ż 1 0	min vPDpaptqq	x 9 aptq, vy dt .	(4.9)

We say that a function F : R d Ñ R is a potential for the conservative field D if for every x P R d and every absolutely continuous a : r0, 1s Ñ R d , with ap0q " 0 and ap1q " x, In this case, such a function F is locally Lipschitz continuous, and for every absolutely continuous curve a : r0, 1s Ñ R d , the function t Þ Ñ F paptqq satisfies for almost every t P r0, 1s, d dt F paptqq " xv, 9 aptqy p@v P Dpaptqqq , that is to say, F admits a "chain rule" [Bolte & Pauwels 2019, Lemma 2]. Moreover, by [Bolte & Pauwels 2019, Theorem 1], it holds that D " t∇F u Lebesgue almost everywhere.

We say that a function F is path differentiable if there exists a conservative field D such that F is a potential for D.

  Clarke gradient function. We shall refer to as a Clarke gradient function as any function ϕpx, sq such that Assume that for every s P Ξ, f p¨, sq is locally Lipschitz, path differentiable, and is a potential of some conservative field D s : R d Ñ R d . Consider a function ϕ : R d ˆΞ Ñ R d which is BpR d q b T {BpR d q measurable and satisfies ϕpx, sq P D s pxq for all px, sq P R d ˆΞ. Then, ϕ is an a.e. gradient function for f .

	Selections of conservative fields. Proposition 4.3.2. Chapter 4. Constant step SGD Proof. Define A

# ϕpx, sq " ∇f px, sq if px, sq P ∆ f , ϕpx, sq P Bf px, sq otherwise. (4.11) Note that the inclusion ϕpx, sq P Bf px, sq obviously holds for all px, sq P R d ˆΞ, because ∇f px, sq is an element of Bf px, sq when the former exists. However, conversely, a function ψpx, sq P Bf px, sq does not necessarily satisfy ψpx, sq " ∇f px, sq if px, sq P ∆ f (see the footnote 1 ). By construction, a Clarke gradient function is an a.e. gradient function. ∆ " tpx, sq s.t. ϕpx, sq ‰ ∇f px, squ. Applying Fubini's theorem we have:

  By definition, px, sq P ∆ f means that there exists d x P R d (the gradient) s.t. f px h, sq " f px, sq `xd x , hy `op h q. That is to say px, sq belongs to the set:

	4.8.1 Proof of Lemma 4.3.1
	č εPQ	ď δPQ	č 0ă h ďδ	" py, sq : ˇˇˇf py `h, sq ´f py, sq ´xd x , hy h	ˇˇˇă ε	*	.	(4.24)

  For any ε 1 ą 0, choose d 1 P Q d such that d 1 ´dx ď ε 1 . Using the previous inclusion, for all ε, there exists therefore δ P Q s.t. a positive sequence of rationals converging to zero. By definition, for every k, there exists d k P Q d s.t. for all ε, there exists δ k pεq, s.t. for all (rational) h ď δ k pεq,

	px, sq P	č hďδ,hPQ d	"ˇˇˇˇf	px `h, sq ´f px, sq ´xd x , hy h	ˇˇˇă ε	*
	px, sq P	č hďδ,hPQ d	"ˇˇˇˇf	px `h, sq ´f px, sq ´xd q , hy h	ˇˇˇă ε `ε1	*
	which means ∆ ˇˇˇf px `h, sq ´f px, sq ´xd k , hy h	ˇˇˇă ε `ε1 k .	(4.26)

f Ă ∆ 1 f . To show the converse, consider px, sq P ∆ 1 f . Let pε 1 k q be

  8qq, 110 Chapter 4. Constant step SGD and the limit set L S H paq of a point a P R d for S H is A point a P R d is said S H -recurrent if a P L S H paq . The Birkhoff center BC S H of S H is the closure of the set of its recurrent points:

		ď
	L S H paq "	L x .
		xPS H paq
		!
	BC S H "	a P R

d : a P L S H paq

  This work focuses on the two following types of Clarke critical points. Definition 6.2.3 (Active strict saddle[Davis & Drusvyatskiy 2021]). Let f : R d Ñ R be a locally Lipschitz continuous function. We say that a point x ˚P R d is an active strict saddle if there is M a C 2 active manifold around x ˚, of dimension greater than 0, and x ˚is a saddle point for the function f M .

Definition 6.2.4 (Sharply repulsive critical point). Let f : R d Ñ R be a locally Lipschitz continuous function. We say that a Clarke critical point x ˚is sharply repulsive if it lies on an active manifold M such that x ˚is a local minimum of f M and 0 P Bf px ˚qzB L f px ˚q.

  3.4 we have that for all w P R d zt0u, almost surely, lim inf Consider w P R d and define δ " w 2 . Notice that for x P Bpw, δq we have |xw, xy| ě w 2 ´|xx ´w, wy| ě w 2 2 . Therefore, E n r|xw, η n`1 y|s ě E n r|xw, η n`1 y|1 η n`1 PBpw,δq s ě

		w 2 2	P pη n`1 P B pw, δq |F n q
	ě	w 2 2	ż xPBpw,δq

nPN

E n r|xw, η n`1 y|s ą 0 . h w,δ pxq dx , and the right hand side of this inequality is positive by Assumption 6.3.4.

  Since 9 yptq ď L π L, for t ď t r 1 , we have:f pyp0qq ď f pyptqq `LL π t sup

	y ďr 1	∇ M f pyq ď f pyptqq	`c2 m 2	t .
	Therefore, for t ď t r 1 ,
	f pxptqq ď f pxp0qq ´c2 m t ď f pyp0qq ´c2 m t ď f pyptqq	´c2 m 2	t .

pyptqq " f pyp0qq ´ż t 0 x∇ M f pyptqq, 9

yptqy dt .

  recall that a locally Lipschitz function f : R d Ñ R is path differentiable if for any a.c. curve x : r0, 1s Ñ R d , for almost every t P r0, 1s: pf ˝xq 1 ptq " xv, 9 xptqy @v P Bf pxptqq. (7.4) By [Bolte & Pauwels 2019, Proposition 2], every convex, concave, semialgebraic or definable function is path differentiable. Moreover, if another function g : R d Ñ R is path differentiable, then f `g is also path differentiable [Bolte & Pauwels 2019, Corollary 4]. From a similar point of view, if X is a convex set, then for any a.c. curve x : r0, 1s Ñ R d , for almost every t P r0, 1s:xv, 9 xptqy " 0 @v P N X pxptqq . (7.5)Consider now f, g : R d Ñ R path differentiable, X Ă R d a convex set and x a solution to the DI (7.2). Using Equation (7.4) and (7.5) and the fact that Bpf `gq Ă Bf `Bg,

	we obtain	pf `gqpxptqq ´pf `gqpxp0qq "	´ż t	9 xpuq 2 du .
			0

  . By Proposition 7.3.1, x is a solution to the DI (7.2), and the first result follows by Equation (7.6). Under Assumption 7.3.4, we have that xpR `q Ă acctx n u Ă Z, hence, by Proposition 7.3.2, we have that pf `gq ˝x is constant. Using Assumption 7.3.3, we have for all h P R `,

	0 " pf `gqpxphqq ´pf `gqpxp0qq "	´ż h	9 xpuq 2 du .
		0

If a locally Lipschitz function g is differentiable at a point x, we have t∇gpxqu Ă Bgpxq but the inclusion could be strict (the two sets are equal if g is regular at x): for example, gpxq " x

sinp1{xq is s.t. ∇gp0q " 0 and Bgp0q " r´1, 1s. There even exist functions for which the set of x s.t. t∇gpxqu Ĺ Bgpxq is a set of full measure (see[Lebourg 1979, Proposition 1.9]).

As it is clear from Equation (4.20) we can see pλ 1 , 1q as a polar coordinates representation of the Lebesgue measure λ d .

The definition of active strict saddles provided in[Davis & Drusvyatskiy

2021] involves the notion of parabolic subderivatives. In this paper, we found convenient to use the equivalent Definition 5.3.2, which is closer in spirit to notions of differential geometry.

The name active manifold follows the work of[Davis & Drusvyatskiy 2021], while in[Drusvyatskiy et al. 2016] they are called identifiable manifolds.

U ǹ in their notations.

Although this result is explicitly stated only for the limiting subgradient.

Or more generally Clarke regular. For an existence of such an oracle we invite the reader to consult[Bianchi et al. 2021a].

Remerciements

This will be the object of Proposition 5.5.3 below. The idea is to show that for these n, it holds that f px n q ´f py n q Á z n , and then, to use the angle condition (iv)) of Definition 5.3.3.

Let us temporarily assume that n 0 is deterministic, and work on n ě n 0 . Keeping Inequality (5.5) aside for further use, the next step is to make a Taylor development of y n`1 " P M px n`1 q around x n . This leads to P M px n`1 q " P M px n q `JP M px n qpx n`1 ´xn q `Op x n`1 ´xn 2 q " P M px n q `JP M py n qpx n`1 ´xn q `Op x n`1 ´xn 2 q `Op z n x n`1 ´xn q,

where we used the Lipschitz continuity of the Jacobian matrix function J P M p¨q.

Using Equation (5.4), we rewrite the last display as y n`1 " y n ´γn J P M py n qv n `γn J P M py n qη n`1 `γ2 n Op1` η n`1 2 q`γ n Op z n p1` η n`1 qq. Now, Lemma 2.3.2 shows that J P M py n q coincides with the linear operator P Ty n M . Furthermore, the Verdier condition (iii)) of Definition 5.3.3 asserts that P Ty n M pv n q " ∇ M f py n q `Op z n q. Altogether, we obtain the Robbins-Monro iteration y n`1 " y n ´γn ∇ M f py n q`γ n P Ty n M η n`1 `γ2 n Op1` η n`1 2 q`γ n Op z n p1` η n`1 qq.

(5.6) Had we not have the last term γ n Op z n p1 ` η n`1 qq at the right hand side, the approach of Brandière and Duflo would have been enough to obtain the nonconvergence of y n to zero under our assumptions on the noise. The presence of this term requires us to weaken a bit their conditions. This will be done in Proposition 5.5.1. In the case of Equation (5.6), this proposition asserts that the trap avoidance remains true if Thus,

Consequently, given an arbitrary δ ą 0, there is an integer N pδq ě 0 such that

(5.8)

For an integer N ě 0, define the stopping time

with inf H " 8, and recall from the definition of r that for N ď n ă τ N , the projection P M px n q is well-defined. Define recursively the process py N n q něN ´1 as follows: y N N ´1 " 0,

and let z N n " px n ´yN n q1 năτ N for n ě N. Observe that y N n and z N n are both F n -measurable for all n ě N . To establish Theorem 5.4.1, we shall show that for each N ě 0,

(5.9) Indeed, on the event A N pδq , it holds that y

Consequently, with the convergence (5.9) at hand, we get from Inequality (5.8) that

Prx n Ñ 0s ď δ. Since δ is arbitrary, we obtain that Prx n Ñ 0s " 0.

In the remainder of this section, N ě 0 is a fixed integer.

Proposition 5.5.4. Let Assumptions 5.4.1-5.4.2 and 5.5.1 hold. Then, the sequence py N n q něN satisfies the recursion:

where the random sequences pη N n q něN , p N n q něN , and p˜ N n q něN are adapted to pF n q. Moreover, there is C ą 0 such that for all n ě N , i)

5.5. Proof of Theorem 5.4.1 131

iii) E n ηN n`1 " 0, and E n ηN n`1

We furthermore have:

iv) The subspace E ´defined before Assumption 5.4.3 coincides with the eigenspace of the matrix J D p0q corresponding to its negative eigenvalues.

v) On the event ry N n Ñ n 0s, it holds that lim inf n E n P E ´η N n`1 ą 0. To prove this proposition, the following result will be needed.

Lemma 5.5.5. For r small enough, there is C ą 0 such that for x, x 1 P Bp0, rq, we have:

where y 1 , y " P M px 1 q, P M pxq, and where R 1 px, x 1 , yq ď C x 1 ´x x ´y , and R 2 px, x 1 q ď C x 1 ´x 2 .

Proof. Since P M is C 2 near zero, there is ε ą 0 such that t Þ Ñ P M px `tpx 1 ´xqq is C 2 on p´ε, 1 `εq. Hence, by Taylor's theorem, we have

where C is a bound on the second derivatives of P M . Similarly, since P M is C 2 , x Þ Ñ J P M pxq is Lipschitz continuous. Therefore, for some C ą 0, J P M pxq ´JP M pyq ď C x ´y , which finishes the proof.

Proof of Proposition 5.5.4. Letting n ě N , we write

accepting the small notational abuse in the expression P M px n`1 q1 τ N ąn`1 , since the projection might not be defined when the indicator is zero. Similar abuses will also be made in the derivations below. Using Lemma 5.5.5 and Equation (5.4), we obtain

where N n`1 and ζ N n`1 are F n`1 -measurable, and satisfy with the notations of Lemma 5.5.5

(in the last inequality, we used that v n is bounded on rτ N ą ns), and

Chapter 6. SGD escapes generic traps

Denote L Ă M the set:

ty P M : Dε ą 0 s.t. @x P Bpy, εq, P α,β pxqu . (6.13)

The second point of the outline comes from the following lemma.

Lemma 6.4.4. The following holds.

i) The set L is definable.

ii) We have that č αPQ,αą0

Proof. The first statement comes from the fact that the set L can be written as a first-order formula. The second statement is immediate from definitions. Finally, the last statement comes from the fact that we have:

By Remark 6 the "definable dimension" of a set coincides with its Hausdorff dimension and by Lemma 6.4.3 we have dim A α ă k. Hence, L c XM is included in a countable union of sets of Hausdorff dimension less than k. Therefore, dim L c X M ă k.

Define S :" tpy, wq : y P L c X M, w P Bf pyq X W u and S w :" tw P W : Dy P L c X M, py, wq P Su Lemma 6.4.5. We have that dim S w ă d.

Proof. For y P L c X M define the set S y :" tw : w P Bf pyqu. By Theorem 5.2.1 we have that S y Ă ∇ M f pyq `Ny M and therefore dim S y ď dim N y M " d ´k. By Lemma 6.4.4 we have that dim L c X M ă k. Therefore, applying Proposition 2.4.8, we obtain that

Since S w is the image of S by the projection on the last d coordinates, applying Proposition 2.4.8, we obtain dim S w ď dim S ă d.

Therefore, for almost every w P W , the set ty P M : w P Bf pyqu " tx ẘ P M : ∇ M f w px ẘq " 0u lies in L. By the definition of L for each y P L and every α ą 0, there is β, ε ą 0 such that for all x P Bpy, εq and w P W , f w pxq´f w pP M pxqq ě α x ´PM pxq ùñ xv w , x´P M pxqy ě β x ´PM pxq , @v w P Bf w pxq , which finishes the proof.

Proofs

Assume that the statement of the theorem is not true. Choose T, r, α as in Proposition 6.2.2 and a sequence px n k q that converges to x ˚but f px n k q ă f pP M px n k qq ὰ

x n k ´PM px n k q . Denote t n k " ř n k i"0 γ i and x n k a solution in S ´Bf px n k q such that inf hPr0,T s

x n k phq ´Xpt n k `hq Ñ 0 .

By Proposition 6.2.2 for n k large enough, we have that x n k pr0, T sq Ć Bpx ˚, rq.

Therefore, we can extract a sequence t n 1 k ě t n k such that Xpt n 1 k q ´x˚ ě r 2 . Since on the event rx n Ñ x ˚s the limit set of X is equal to x ˚, this is a contradiction.

Proof of Proposition 6.3.2

In this proof C will denote some absolute constant that can change from line to line and from one statement to another. Without loss of generality assume that x ˚" 0 and f px ˚q " 0.

To prove this proposition the following result will be needed.

Lemma 6.4.6 ( [Bianchi et al. 2021b, Lemma 5]). Let Assumption 6.3.3 hold. There is C, r ą 0 such that the conclusions of Lemma 2.3.2, with p " 2, are verified on Bp0, r 1 q and, moreover, for any x 1 , x 2 P Bp0, rq, we have:

where y 1 , y 2 " P M px 1 q, P M px 2 q.

Let α ą 0 be the one of Proposition 6.3.1 and let U α , β be as in Definition 6.2.2. Consider r 1 as in the preceding lemma and let r ą 0 be such that Bp0, rq Ă U α and r ď r 1 . The value of r, while always satisfying this requirement, will be adjusted in the course of the proof. Denote z n " px n ´PM px n qq1 xn ďr .

Notice that if x n ď r, then z n " distpx n , M q and z n P N P M pxnq M .

Consider κ ą 0 and for k P N, denote

By a slight abuse of notations we will denote τ k " τ k pκ, r, αq and z τ k n " z n^τ k . The aim of this proof is to show that for any k P N, Ppτ k " `8q " 0. Since on the event rx n Ñ 0s, for n large enough, we have x n ď r and f px n q ă f pP M px n qq ὰ

x n ´PM px n q this will implies that distpx n , M q ď κγ n happens infinitely often.

To establish this result we study the difference between z τ k n`1

Using the angle condition, we show that for τ k ą n, it will decrease at least at a rate γ n z τ k n . Since for r small enough, z τ k n is much larger than z τ k n 2 this will help us to conclude.

Proofs 165

We have:

(6.15)

The following lemma bound the last two quantities.

Lemma 6.4.7. There is d 1 , d 2 ą 0 s.t. if r was chosen small enough, then:

and

Proof. To prove the first inequality apply Lemma 6.4.6. On the event rτ k ą ns, noticing that z n is orthogonal to T P M pxnq M , we obtain:

Hence,

The second inequality is obtained by writing down:

Taking the conditional expectation, we obtain the desired result using Lemma 6.4.6, Equation (4.1) and Assumption 6.3.2.

Using the preceding lemma and the angle condition, taking the conditional expectation on Inequality (6.15), we obtain:

Denote θ n " zn γn and θ τ k n " θ n^τ k .

Lemma 6.4.8. There is d 3 , d 4 ą 0 s.t. if r ą 0 was chosen small enough, we have:

166 Chapter 6. SGD escapes generic traps

Proof.

Using Assumption 6.3.1, we obtain:

Hence,

If r was chosen such that pc 2 `d1 qr `c2 d 1 γ n r ´2c 1 β 1 ă 0, then we have:

which is the desired result.

Consider d 3 , d 4 from Lemma 6.4.8. If κ was chosen greater than 2d 3 d 4 , then from the preceding lemma we obtain:

Hence, for all n ě k:

Hence, for any k P N, τ k is almost surely finite. Noticing that by Proposition 6.3.1, on the event rx n Ñ 0s, the events r x n ą rs and rf px n q ă f pP M px n qq ὰ

x n ´PM px n q s happen only a finite number of times, this implies the statement of Proposition 6.3.2.

Proofs

Chapter 7. Oscillations of the SPGD Note that if g is nonconvex, prox γ g,X pxq is a set in R d . Assumption 7.3.1-(iii)) then implicitly states that x n`1 is chosen in a measurable manner, such a choice is always possible (see e.g. [START_REF] Davis | [END_REF]). By [Rockafellar & Wets 1998, 10.2 and 10.10], we can rewrite Equation (7.1) as:

where v g n P Bgpx n`1 q and v X n P N X px n`1 q.

Assumption 7.3.2.

i) Almost surely, sup n x n ă `8.

ii) There is q ě 2 such that

and, for any compact set

Assumptions of this type are standard in the field of stochastic approximation. Assumption 7.3.2-(i)) prevent the algorithm to diverge. Note that it is superfluous if X is compact. Otherwise it can be obtained by a proper choice of the regularizer g (see [START_REF] Duchi | [END_REF]).

Let τ n " ř n i"1 γ i be the discrete time of the algorithm. Define the linearly interpolated process X P CpR `, R d q by: Xptq " x n `t ´τn γ n`1

x n`1 for τ n ď t ă τ n`1 .

Following [Benaïm et al. 2005] we will show that X is an APT of the DI (7.2). The next two assumptions ensure us that f `g will be a Lyapounov function for the DI (7.2).

Assumption 7.3.3. The functions f and g are path differentiable.

Assumption 7.3.4. The set of Clarke critical values tf pxq `gpxq : x P Zu has an empty interior.

Assumption 7.3.4 is a classical Sard-type condition. It ensures the fact that if x is a solution to the DI (7.2), with xp0q P Z, then x is constant. As established in [START_REF] Bolte | [END_REF], it is satisfied as soon as f, g and X are definable.

The next two propositions are not new and can be found in one way or another in e.g. [START_REF] Davis | [END_REF], Bolte & Pauwels 2019, Majewski et al. 2018, Bolte et al. 2020b]. Nevertheless, since our set of assumptions is slightly different and their proof is a simple application of Section 2.2.2, for completeness, we include it in Section 7.4.1.

where U is an open neighborhood of an accumulation point x. Looking at the proof of Theorem 7.3.5, to obtain Equation (7.12), we could think of defining maximal intervals as I i " rn 1i , n 2i s such that tx n 1i , . . . , x n 2i u Ă U and x n 1i ´1, x n 2i `1 R U . Unfortunately, for this type of intervals we dont have an equivalent of Theorem 7.3.4, i.e. it may very well be that the quantity τ n 2i ´τn 1i is bounded. Actually, it is not very hard to show, that for the function from [Rios-Zertuche 2020, Section 2], there are x, U such that Equation (7.12) is false.

Nevertheless, as explained in [Bolte et al. 2020b], Equation (7.11) is a good approximation of Equation (7.12). Indeed, apply Theorem 7.3.5 with U and V " U δ , where U δ " ty P R d : Dz P U, z ´y ă δu, then, as an approximation, we have

Proofs

In the following we will denote x n`1{2 " x n ´γn v n `γn η n`1 and N pT, nq " inftj ě n s.t. τ j ´τn ě T u . To put ourselves in the context of Section 2.2.2 we need to alter the map ´Bf ´Bg ŃX in a way that it verifies assumptions of Propositions 2.2.2 and 2.2.3. While this section is slightly technical, conceptually, we just find a set-valued map G verifying assumptions of Proposition 2.2.3 and s.t. x n`1 P Gpx n q. This is done using the Lipschitz continuity of f, g and the boundedness of px n q. A convinced reader may want to skip to Section 7.4.2. We start with two technical lemmas. As a consequence, the sequence p x n`1{2 q is almost surely bounded.

Proof. Indeed, since almost surely sup x n ă `8, for each δ ą 0, there is C ą 0 s.t. if we denote A " t@n P N x n ď Cu, then PpAq ą 1 ´δ. Define ηn`1 "

η n`1 1 xn ďC , then Erη n`1 |F n s " 0 and sup nPN Er ηn`1 q s ă `8. Hence, by [Benaïm 1999, Proposition 4.2], we have sup nďjďN pT,nq ř j i"n γ i ηi`1 ÝÝÝÝÑ nÑ`8 0.

Since δ is arbitrary, Equation (7.14) follows.

Lemma 7.4.2. Let Assumptions 7.3.1 and 7.3.2 hold. Let A P Ξ be a probability one set on which px n q and px n`1{2 q are bounded, and let C be a random variable s.t.

x n ă C and C is finite valued on A. Then for each ω P A, there are two globally 7.4. Proofs By Theorem 7.3.4, we have that K N Ñ `8 and, therefore, for N large enough:

Hence, by induction:

.

Therefore, lim u N ď 2ε T . Since ε is arbitrary, this finishes the proof.