Ya-Ping Hsieh
email: yaping.hsieh@inf.ethz.ch

Mohammad Reza Karimi
email: mkarimi@inf.ethz.ch

Andreas Krause
email: krausea@ethz.ch

Panayotis Mertikopoulos
email: panayotis.mertikopoulos@imag.fr.

RIEMANNIAN STOCHASTIC OPTIMIZATION METHODS AVOID STRICT SADDLE POINTS

Keywords: 2020 Mathematics Subject Classification. Primary 62L20, 37N40, secondary 90C15, 90C48 Optimization on manifolds, stochastic approximation, saddle-point avoidance, Riemannian Robbins-Monro algorithms

Many modern machine learning applications -from online principal component analysis to covariance matrix identification and dictionary learningcan be formulated as minimization problems on Riemannian manifolds, and are typically solved with a Riemannian stochastic gradient method (or some variant thereof). However, in many cases of interest, the resulting minimization problem is not geodesically convex, so the convergence of the chosen solver to a desirable solution -i.e., a local minimizer -is by no means guaranteed. In this paper, we study precisely this question, that is, whether stochastic Riemannian optimization algorithms are guaranteed to avoid saddle points with probability 1. For generality, we study a family of retraction-based methods which, in addition to having a potentially much lower per-iteration cost relative to Riemannian gradient descent, include other widely used algorithms, such as natural policy gradient methods and mirror descent in ordinary convex spaces. In this general setting, we show that, under mild assumptions for the ambient manifold and the oracle providing gradient information, the policies under study avoid strict saddle points / submanifolds with probability 1, from any initial condition. This result provides an important sanity check for the use of gradient methods on manifolds as it shows that, almost always, the limit state of a stochastic Riemannian algorithm can only be a local minimizer.

Introduction

Modern machine learning systems have achieved remarkable success in the efficient optimization of highly non-convex functions using straightforward Euclidean techniques like stochastic gradient descent. A widely accepted hypothesis to explain this phenomenon is that, when the learning system under study -e.g., a neural network -is sufficiently expressive, local minimizers are essentially as good as global ones [START_REF] Choromanska | The loss surfaces of multilayer networks[END_REF][START_REF] Kawaguchi | Deep learning without poor local minima[END_REF]; by this token, a training algorithm can attain satisfactory performance by simply evading saddle points of the model's loss surface.

This observation has sparked a far-reaching research thread examining the behavior of various algorithms around saddle points in non-convex functions. Informally, these studies aim to tackle two fundamental questions: (Q1) When does a given scheme, like stochastic gradient descent, avoid saddle points? (Q2) Can we augment a given scheme so that it efficiently escapes saddle points?

Of the above questions, (Q1) focuses on explaining the empirical success of commonly used schemes, while the resolution of (Q2) usually revolves around proposing new schemes with desirable escape guarantees. These complementary perspectives have been extensively studied over the past decade, leading to a fairly complete understanding of how and when a Euclidean (stochastic) algorithm escapes saddle points, cf. [START_REF] Antonakopoulos | AdaGrad avoids saddle points[END_REF][START_REF] Hou | Analysis of asymptotic escape of strict saddle sets in manifold optimization[END_REF][START_REF] Hsieh | The limits of min-max optimization algorithms: Convergence to spurious non-critical sets[END_REF][START_REF] Lee | Gradient descent only converges to minimizers[END_REF][START_REF] Lee | First-order methods almost always avoid strict saddle points[END_REF][START_REF] Mertikopoulos | On the almost sure convergence of stochastic gradient descent in non-convex problems[END_REF][START_REF] Mertikopoulos | A unified stochastic approximation framework for learning in games[END_REF][START_REF] Panageas | First-order methods almost always avoid saddle points: The case of vanishing step-sizes[END_REF][START_REF] Pemantle | Nonconvergence to unstable points in urn models and stochastic aproximations[END_REF] and references therein.

In parallel to the above, the recent surge of interest in Riemannian optimization has prompted a closer examination of Riemannian methods, thereby motivating an extension of (Q1) and (Q2) to a manifold setting -itself due to a wide range of breakthrough applications to machine learning and data science, from natural language processing, and signal processing to dictionary learning and robotics [START_REF] Meyer | Linear regression under fixed-rank constraints: a riemannian approach[END_REF][START_REF] Oja | Principal components, minor components, and linear neural networks[END_REF][START_REF] Sra | Conic geometric optimization on the manifold of positive definite matrices[END_REF][START_REF] Sun | Complete dictionary recovery over the sphere ii: Recovery by riemannian trust-region method[END_REF]. As a result, there is an increasing demand for a comprehensive exploration of various spaces, such as the d-dimensional torus, Grassmannian or Stiefel manifolds, hyperbolic spaces, and many others.

Unfortunately, in a proper Riemannian setting, only (Q2) has received sufficient scrutiny thus far. Recent works by Criscitiello & Boumal [START_REF] Criscitiello | Efficiently escaping saddle points on manifolds[END_REF] and Sun et al. [START_REF] Sun | Escaping from saddle points on riemannian manifolds[END_REF] have shown that standard Riemannian deterministic algorithms can be augmented via the injection of an infinitesimal amount of noise (proportional to the method's desired accuracy), to achieve comparable escape guarantees in terms of oracle complexity as the corresponding Euclidean methods [START_REF] Jin | How to escape saddle points efficiently[END_REF]. To the best of our knowledge, all existing results for (Q1) concern deterministic methods [START_REF] Hou | Analysis of asymptotic escape of strict saddle sets in manifold optimization[END_REF][START_REF] Lee | Gradient descent only converges to minimizers[END_REF][START_REF] Lee | First-order methods almost always avoid strict saddle points[END_REF][START_REF] Panageas | First-order methods almost always avoid saddle points: The case of vanishing step-sizes[END_REF] which are significantly limited in scope in large-scale machine learning applications because of their prohibitively high per-iteration cost.

Our results and techniques. In view of the above, our paper aims to provide a general answer to (Q1) for a broad class of Riemannian stochastic optimization methodsincluding Riemannian stochastic gradient descent, its retraction-based and/or optimistic variants, etc. Concretely, we focus throughout on a flexible template of Riemannian Robbins-Monro (RRM) schemes [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF][START_REF] Robbins | A stochastic approximation method[END_REF] which, in addition to the specific algorithms of interest mentioned above, also includes a range of Euclidean methods that can be analyzed efficiently from a Riemannian viewpoint.

Informally, our main result may be stated as follows:

Under any stochastic Riemannian Robbins-Monro method, the probability of converging to a strict saddle point (or a submanifold thereof ) is zero. This statement provides firm grounds for accepting the output of a stochastic Riemannian optimization method as valid, as it shows that saddle points are avoided with probability 1. 1 In the context of stochastic methods, our result builds on a series of foundational results by Pemantle [START_REF] Pemantle | Nonconvergence to unstable points in urn models and stochastic aproximations[END_REF] and Brandière & Duflo [START_REF] Brandière | Les algorithmes stochastiques contournent-ils les pièges ?[END_REF] who focused on hyperbolic traps (isolated saddle points with invertible Hessian). These results were subsequently extended by Benaïm & Hirsch [START_REF] Benaïm | Asymptotic pseudotrajectories and chain recurrent flows, with applications[END_REF] to a more general class of unstable sets, but this analysis remained grounded in a flat, Euclidean setting. The connecting tissue of our analysis with these works is the notion of an asymptotic pseudotrajectory (APT), which allows us to couple the long-run behavior of discrete-time RRM methods to that of an associated Riemannian gradient flow. [This discrete-to-continuous comparison is crucial for our analysis in order 1 We recall here that a strict saddle manifold is a set of critical points each of which has at least one negative Hessian eigenvalue. Such manifolds include ridge hypersurfaces and other connected sets of non-isolated saddle points that are common in the loss landscapes of high-dimensional machine learning models, so this result has significant cutting power in this regard.

to apply center stable manifold techniques [START_REF] Shub | Global Stability of Dynamical Systems[END_REF] to the RRM framework.] However, this comes at a significant cost, as establishing the APT property in a Riemannian setting is quite challenging. To achieve this, we employ a set of techniques recently developed by [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF] which allow us to make this comparison precise and establish the desired avoidance result.

Background on Riemannian Manifolds

We begin with a brief overview of some basic definitions from Riemmanian geometry and optimization, solely intended to set notation and terminology; our presentation roughly follows the masterful account of Lee [START_REF] Lee | Riemannian Manifolds: an Introduction to Curvature[END_REF][START_REF] Lee | Introduction to Smooth Manifolds[END_REF], to which we refer the reader for a comprehensive introduction to the topic.

Le M be a d-dimensional, geodesically complete Riemannian manifold. Throughout the sequel, the tangent space to M at a point x ∈ M will be denoted by T x M, and we will write γ(t) ∈ T γ(t) M for the velocity vector to a smooth curve γ : R → M at time t ∈ R. We will also write ⟨•, •⟩ x for the metric at x ∈ M, ∥•∥ x for the associated norm, and dist(•, •) for the induced distance function on M, the latter being defined via the minimization of the length functional

L[γ] = ∥ γ(t)∥ γ(t) dt.
Given a point x ∈ M and a tangent vector z ∈ T x M, the (necessarily unique) geodesic emanating from x along z will be denoted by γ z , and we define the exponential map at x as exp x (z) = γ z (1) for all z ∈ T x M (recall here that M is assumed complete, so this map is well-defined for all x ∈ M and all z ∈ T x M). Whenever well-defined, the inverse of exp x will be written as log x : M → T x M, with the understanding that the domain of log x is actually the largest neighborhood of x ∈ M on which the restriction of exp x is a (global) diffeomorphism; by definition, we have log x (exp x (z)) = z for all z for which the relevant quantities are well-defined. Finally, given a pair of points x, x ′ ∈ M and a tangent vector z ∈ T x M, we will write Γ x→x ′ (z) for the vector obtained by parallel transporting z along any minimizing geodesic connecting x and x ′ .

In this general context, we will be interested in solving the Riemannian optimization problem

minimize x∈M f (x) (Opt)
for some smooth objective function f : M → R. We will also respectively write

v(x) := -gradf (x) and H(x) := Hess(f (x)) (1)
for the negative (Riemannian) gradient and the (Riemannian) Hessian of f at x. Finally, in terms of solutions of (Opt), we will focus on the avoidance of strict saddle points of f , i.e., points x ∈ M for which

v(x) = 0 and λ min (H(x)) < 0 (2)
where λ min denotes the minimum eigenvalue of the tensor in question. We will also say that a smooth compact component (in the sense of manifolds) of critical points of f is a strict saddle manifold if there exist constants c ± > 0 such that all negative eigenvalues of H(x), x ∈ S, are bounded from above by -c -< 0, and any positive eigenvalues (if they exist) are bounded from below by c + > 0.

To differentiate the above from the Euclidean setting, when M is a real space equipped with the Euclidean metric, we will instead write ∇f and ∇ 2 f for the (ordinary) gradient and Hessian matrix of f . In this case, as is customary, we will not distinguish between primal and dual vectors.

Core algorithmic framework

For generality, our avoidance analysis will be carried out in an abstract stochastic approximation framework which includes several popular Riemannian optimization algorithms -from ordinary Riemannian (stochastic) gradient descent, to its retraction-based variants, optimistic methods, etc. For concreteness, we start with the general template below, and we present a (nonexhaustive!) series of representative examples right after.

3.1. The Riemannian Robbins-Monro template. The Riemannian Robbins-Monro (RRM) framework that we will consider for solving (Opt) is an iterative family of methods which directly extends the seminal stochastic approximation scheme of Robbins & Monro [START_REF] Robbins | A stochastic approximation method[END_REF] to a manifold setting by replacing vector addition with the Riemannian exponential. Roughly following [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF], we will focus on the abstract update rule

X n+1 = exp Xn (γ n vn ) (RRM)
where (1) X n ∈ M denotes the state of the algorithm at each iteration n = 1, 2, . . .

(2) vn ∈ T Xn M is a surrogate for the (negative) gradient v(X n ) of f at X n (defined in detail below). (3) γ n > 0 is the method's step-size (discussed in detail in Section 4). In the above, the defining element of (RRM) is the sequence of "surrogate gradients" vn , n = 1, 2, . . . , so this will be our first object of interest. Formally, letting F n denote the history of X n up to stage n (inclusive), we will write

vn := v(X n ) + U n + b n (3)
where we have defined

U n := vn -E[v n | F n ] and b n := E[v n | F n ] -v(X n ), (4)
as the random error and the offset of vn relative to v(X n ) respectively. It will also be convenient to introduce the total error W n = vn -v(X n ) = U n + b n , which captures both random and systematic fluctuations in vn , and which measures the total deviation of vn from v(X n ). Two points are worth noting here: First, vn is not adapted to F n , so U n is random relative to F n ; on the other hand, b n is F n -measurable, so it is deterministic relative to F n . This brings us to the second important point regarding vn : given the systematic offset term b n in vn , the latter should not be seen as the output of a gradient oracle for v(X n ). In particular, b n is intended to capture possible corrective terms, deviations from the exponential mapping, different algorithmic update structures (such as optimism), etc. We make this distinction precise below.

Specific algorithms and examples.

In the series of examples that follow, we will assume that the optimizer can access f via a stochastic first-order oracle (SFO) returning noisy gradients of f at the evaluation point. Formally, following Nesterov [START_REF] Nesterov | Introductory Lectures on Convex Optimization: A Basic Course[END_REF], an SFO is a black-box mechanism which, when queried at x ∈ M, returns a (negative) stochastic gradient of the form V (x; θ) = v(x) + err(x; θ) (SFO) where the seed θ ∈ Θ is a random variable taking values in some complete probability space Θ, and err(x; θ) is an umbrella error term capturing all sources of uncertainty in the model.

The archetypal example of an SFO occurs when f is itself a stochastic expectation of the form f (x) = E[F (x; θ)] for some random function F : M×Θ → R -the so-called stochastic optimization framework. In this case, V is typically given by V (x; θ) = -grad x F (x; θ), so, under standard assumptions for exchanging differentiation and expectation, we have E[V (x; θ)] = v(x). Extrapolating from this basic framework, our only assumption for the moment will be that E[err(x; θ)] = 0; for a detailed discussion of the required assumptions for (SFO), see Section 4.

In practice, (SFO) will be queried repeatedly at a sequence of states X n , n = 1, 2, . . . , with a different random seed θ n drawn i.i.d. from Θ. In this manner, we obtain the following specific algorithms as special cases of (RRM):

Algorithm 1 (Riemannian stochastic gradient descent). Following Bonnabel [START_REF] Bonnabel | Stochastic gradient descent on Riemannian manifolds[END_REF], the Riemannian stochastic gradient descent (RSGD) algorithm queries (SFO) at X n and proceeds as

X n+1 = exp Xn (γ n V (X n ; θ n )) . (RSGD)
As such, (RSGD) can be seen as an RRM scheme with vn = V (X n ; θ n ) or, equivalently, U n = err(X n ; θ n ) and b n = 0.

♢

A key factor limiting the applicability of (RSGD) is that the exponential map exp Xn (•) could be prohibitively expensive to compute in practice, even for relatively low-dimensional manifolds. On that account, a popular alternative to (RSGD) is to employ a retraction map [START_REF] Absil | Optimization Algorithms on Matrix Manifolds[END_REF][START_REF] Boumal | An introduction to optimization on smooth manifolds[END_REF], that is, a smooth mapping R : T M → M that agrees with the exponential map up to first order, namely

R x (0) = x and d dt t=0 R x (tz) = z for all (x, z) ∈ T M. (Rtr)
With this machinery in hand, we obtain the following retraction-based variant of (RSGD):

Algorithm 2 (Retraction-based stochastic gradient descent). By replacing the exponential map in (RSGD) with a retraction, we obtain the retraction-based stochastic gradient descent scheme

X n+1 = R Xn (γ n V (X n ; θ n )). (Rtr-SGD)
This algorithm does not seem immediately related to the RRM template -and, indeed, the whole point of introducing a retraction was to get rid of the exponential map in (RRM). The expressive power of (RRM) can be seen in the fact that, despite this apparent disconnect, (Rtr-SGD) can still be expressed as a special case of (RRM) in a fairly straightforward fashion.

To do so, define the "forward-backward" gradient mapping

vn := 1 γ n log Xn (R Xn (γ n V (X n ; θ n ))) (5)
with the proviso that the Riemannian logarithm in (5) is well-defined (we discuss the conditions under which this holds later in the paper). Under this definition, (Rtr-SGD) can be recast as a special case of (RRM) by running the latter with the surrogate gradient sequence vn of Eq. (5). To streamline our presentation, we defer the discussion about the inherent error W n = vn -v(X n ) to Appendix A.

♢

As we mentioned before, retraction-based algorithms typically exhibit significantly lower per-iteration complexity compared to geodesic methods, resulting in their remarkable success in practical machine learning applications [START_REF] Absil | Optimization Algorithms on Matrix Manifolds[END_REF][START_REF] Boumal | An introduction to optimization on smooth manifolds[END_REF]. In addition, as we show below, the use of a retraction mapping allows us to provide a unified perspective for several classical algorithms which, at first sight, might seem completely unrelated. An important example is provided by the (stochastic) mirror descent (MD) family of algorithms [START_REF] Nemirovski | Problem Complexity and Method Efficiency in Optimization[END_REF]: [START_REF] Rockafellar | Convex Analysis[END_REF]Chap. 26]. Then, the stochastic mirror descent (SMD) algorithm unfolds as

Algorithm 3 (Stochastic mirror descent). Let M be an open convex subset of R M and let h : M → R be a C 2 -smooth, strongly convex Legendre function on M, that is, ∥∇h(x)∥ → ∞ whenever x → bd(M) [cf.
X n+1 = P Xn (γ n V (X n ; θ n )) (SMD)
where V (X n ; θ n ) is the output of an SFO query for ∇f (X n ) at X n , and P : M×R M → M is the so-called prox-mapping associated to h [7-9, 31], viz.

P x (y) = arg max x ′ ∈M {⟨∇h(x) + y, x ′ ⟩ -h(x ′ )} for all x ∈ M, y ∈ R M . ( 6
)
where ⟨•, •⟩ stands for the ordinary Euclidean inner product in R M . Now, even though the notation in (SMD) is reminiscent of (Rtr-SGD), the definition (6) of P does not bear any resemblance to a geodesic exponential or a retraction -and, indeed, its origins are starkly different. However, as we show below, P can indeed be seen as a retraction relative to a specific Riemannian structure on M, the Hessian Riemannian (HR) metric associated to h [START_REF] Alvarez | Hessian Riemannian gradient flows in convex programming[END_REF][START_REF] Bomze | Hessian barrier algorithms for linearly constrained optimization problems[END_REF][START_REF] Duistermaat | On Hessian Riemannian structures[END_REF][START_REF] Mertikopoulos | Riemannian game dynamics[END_REF].

To make this precise, the first step is to note that the basic recursive structure x + = P x (y) of (SMD) can be rewritten as

x + = P x (y) = ∇h * (∇h(x) + y) (7)
where h * (y) = max x∈M {⟨y, x⟩ -h(x)} denotes the convex conjugate of h, and we have used Danskin's theorem [START_REF] Shapiro | Lectures on Stochastic Programming[END_REF] to write arg max x∈M {⟨y, x⟩ -h(x)} = ∇h * (y). Then, if we endow M with the Hessian Riemannian metric g(x) = ∇ 2 h(x), the Riemannian gradient of f relative to g becomes gradf (x) = [∇ 2 h(x)] -1 ∇f (x); more generally, given a cotangent (dual) vector y to M at x, the corresponding tangent (primal) vector will be z = g(x) -1 y = [∇ 2 h(x)] -1 y. In view of this, by inverting the relation z = g(x) -1 y, the abstract mirror descent recursion (7) can be rewritten as

x + = R x (z) := P x (g(x)z). (8)
Now, to proceed, consider the curve

γ(t) = R x (tz) = P x (tg(x)z) = ∇h * (∇h(x) + tg(x)z), (9)
so, by definition, γ(0) = x. In addition, by a direct differentiation, we readily obtain

γ(0) = ∇ 2 h * (∇h(x))g(x)z = z ( 10
)
where we used the standard identity [START_REF] Beck | Mirror descent and nonlinear projected subgradient methods for convex optimization[END_REF][START_REF] Rockafellar | of A Series of Comprehensive Studies in Mathematics[END_REF]. This shows that the map R x (z) = P x (g(x)z) is, in fact, a retraction, so (SMD) is a special case of (Rtr-SGD) -and hence, of the general stochastic approximation template (RRM).

∇ 2 h * (∇h(x)) = [∇ 2 h(x)] -1 [

♢

Remark. Even though elements of the above ideas are implicit in previous works on mirror descent and Hessian Riemannian metrics [START_REF] Alvarez | Hessian Riemannian gradient flows in convex programming[END_REF][START_REF] Antonakopoulos | Online and stochastic optimization beyond Lipschitz continuity: A Riemannian approach[END_REF][START_REF] Bomze | Hessian barrier algorithms for linearly constrained optimization problems[END_REF][START_REF] Raskutti | The information geometry of mirror descent[END_REF][START_REF] Zhou | On the convergence of mirror descent beyond stochastic convex programming[END_REF], to the best of our knowledge, this is the first time that (SMD) is formalized as a retraction-based (Hessian) Riemannian scheme.

♢ Algorithm 4 (Riemannian optimistic gradient). Moving forward, an important algorithm for solving online optimization problems and games is the so-called optimistic gradient method -originally pioneered by Popov [START_REF] Popov | A modification of the Arrow-Hurwicz method for search of saddle points[END_REF] and subsequently popularized by Rakhlin & Sridharan [START_REF] Rakhlin | Online learning with predictable sequences[END_REF]. In the Euclidean case, this method introduces an interim, "optimistic" correction to gradient dynamics and updates as

X + n = X n + γ n V (X + n-1 ; θ n-1 ) X n+1 = X n + γ n V (X + n ; θ n ) (OG)
where, as usual, V is an SFO for the (negative) gradient ∇f of f . This idea can then be directly transported to a manifold setting [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF], leading to the Riemannian optimistic gradient method

X + n = exp Xn (γ n V (X + n-1 ; θ n-1 )), X n+1 = exp Xn (Γ X + n →Xn (γ n V (X + n ; θ n ))). (ROG)
Importantly, the recursion (ROG) may be seen as a special case of (RRM) by setting

vn = (1/γ n ) • Γ X + n →Xn (γ n V (X + n ; θ n )) or, equivalently U n = Γ X + n →Xn (err(X + n ; θ n )) and b n = Γ X + n →Xn (v(X + n )) -v(X n ).
We defer the calculation details to Appendix A.

♢

Algorithm 5 (Natural gradient descent). Our last example concerns the influential natural gradient descent (NGD) method of Amari [START_REF] Amari | Natural gradient works efficiently in learning[END_REF], a stochastic optimization scheme for Euclidean spaces, but adapted to the local geometry defined by a strictly convex function h. Specifically, NGD queries an SFO and proceeds as

X n+1 = X n -γ n (gradf (X n ) + err(X n ; θ n )) (NGD)
where gradf (x) := [∇ 2 h(x)] -1 ∇f (x) denotes the Riemannian gradient of f relative to Hessian Riemannian metric g(x) = ∇h 2 (x) on R M . It is well known that (NGD) can be seen as a retraction-based Riemannian scheme [START_REF] Bonnabel | Stochastic gradient descent on Riemannian manifolds[END_REF], and may thus be integrated directly within the framework of (RRM); we defer the details to Appendix A. Importantly, (NGD) also includes the celebrated natural policy gradient [START_REF] Kakade | A natural policy gradient[END_REF] which plays an important role in reinforcement learning.

♢

The above examples have been chosen to illustrate a range of different update mechanisms that can be integrated within the general algorithmic template provided by (RRM). Of course, it is not possible to be exhaustive but, for illustration purposes, we provide some more examples in Appendix A.

Analysis and results

We are now in a position to state and discuss our main result concerning the avoidance of saddle points under (RRM). For concreteness, we begin by discussing the technical assumptions that we will need in Section 4.1; subsequently, we proceed with the formal statement of our result and some direct applications thereof in Section 4.2.

4.1. Technical assumptions. Our technical assumptions concern the four main ingredients of (RRM), namely (i) the regularity of f ; (ii) the method's step-size sequence γ n ; (iii) the statistics of the surrogate gradients vn entering (RRM); and (iv) the ambient manifold M. Specifically, we will require the following: Assumption 1 (Regularity of f ). The function f is C 2 and v = -gradf is (geodesically) L-Lipschitz, i.e., for all x, x ′ ∈ M, we have

∥Γ x→x ′ (v(x)) -v(x ′ )∥ x ′ = ∥v(x) -Γ x ′ →x (v(x ′ ))∥ x ≤ L dist(x, x ′ ). ( 11
)
Assumption 2 (Step-size schedule). The step-size sequence γ n of (RRM) satisfies

∞ n=1 γ n = ∞ and ∞ n=1 λ 1/γn < ∞ for all λ ∈ (0, 1). ( 12
)
Assumption 3 (Surrogate gradients). The offset and random error components of vn satisfy

∥b n ∥ Xn ≤ Cγ n , ∥U n ∥ Xn ≤ σ, E[[⟨U n , z⟩ Xn ] + | F n ] ≥ ζ (13
) for suitable constants C, σ, ζ > 0 and for all z ∈ T Xn M, ∥z∥ Xn = 1 (in the above, all conditions are to be interpreted in the almost sure sense and [t] + = max{0, z} denotes the positive part of t).

Assumption 4 (Injectivity radius). The injectivity radius of M is bounded from below by ϱ > 0.

Before proceeding, we discuss the implications and range of validity of each of the above assumptions. Since Assumption 1 is standard, we focus on the remaining three below:

On Assumption 2. The step-size conditions typically encountered in the analysis of Robbins-Monro schemes is the L 2 -L 1 ("square-summabe-but-not-summable") condition [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF][START_REF] Benveniste | Adaptive Algorithms and Stochastic Approximations[END_REF][START_REF] Bertsekas | Gradient convergence in gradient methods with errors[END_REF][START_REF] Bonnabel | Stochastic gradient descent on Riemannian manifolds[END_REF][START_REF] Kushner | Stochastic Approximation Methods for Constrained and Unconstrained Systems[END_REF][START_REF] Robbins | A stochastic approximation method[END_REF]] and references therein. This puts a hard threshold on the range of allowed step-size schedules at Ω(1/n 1/2 ): any step-size that decays at least as slow as 1/n 1/2 cannot be used under the L 2 -L 1 assumption. By contrast, the step-size condition [START_REF] Benaïm | Asymptotic pseudotrajectories and chain recurrent flows, with applications[END_REF] is considerably more lax and can tolerate nearconstant step-sizes of the form γ n ∝ 1/(log n) 1+ε for some ε > 0. This is enough to cover all dereasing step-size policies used in practice. [We also recall here that, in stochastic non-convex settings, trajectory convergence cannot be guaranteed in general without a vanishing step-size, cf. [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF][START_REF] Borkar | Stochastic Approximation: A Dynamical Systems Viewpoint[END_REF][START_REF] Kushner | Stochastic approximation algorithms and applications[END_REF] and references therein.] On Assumption 3. Three remarks are in order for the noise and offset requirements [START_REF] Benveniste | Adaptive Algorithms and Stochastic Approximations[END_REF]. First, we should note that the condition b n = O(γ n ) is, a priori, implicit, because it depends on the statistics of the feedback sequence vn , and these may be difficult to estimate in general. However, in most practical applications, this quantity is under the explicit control of the optimizer: in particular, as we show later in this section, this requirement is satisfied by all the specific algorithms of Section 3.2.

n γ n = ∞, n γ 2 n < ∞, cf.
Likewise, the bounded noise requirement is satisfied in many practical cases of interest. For example, when the problem's objective function admits a finite-sum decomposition of the form f (x) = N i=1 f i (x) for an ensemble of empirical instances f i , i = 1, . . . , N (the standard framework for applications to data science and machine learning), U n is typically generated by sampling a minibatch of f , which in turn results in an error term of the form U n = q(X n ) where q(x) : M → T x M is bounded on all compact subsets of M. Therefore, ∥U n ∥ Xn ≤ ∥q(X n )∥ Xn < σ for some constant σ for any convergent algorithm {X n } n .

Finally, the "uniform excitability" condition E[[⟨U n , z⟩ Xn ] + | F n ] ≥ ζ is also standard in the avoidance literature [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF][START_REF] Pemantle | Nonconvergence to unstable points in urn models and stochastic aproximations[END_REF], and it is substantially weaker than the isotropic condition, which, roughly speaking, requires the noise to have the same L 2 magnitude along all directions in space [START_REF] Ge | Escaping from saddle points -Online stochastic gradient for tensor decomposition[END_REF][START_REF] Jin | How to escape saddle points efficiently[END_REF][START_REF] Pemantle | Nonconvergence to unstable points in urn models and stochastic aproximations[END_REF]. Instead, [START_REF] Benveniste | Adaptive Algorithms and Stochastic Approximations[END_REF] only posits that the noise U n has a non-zero component along each direction, and imposes no other restrictions on the statistical profile of the noise.

On Assumption 4. For our last assumption, recall first that the injectivity radius of M at a point x ∈ M is the largest radius for which exp x is a diffeomorphism onto its image; the injectivity radius of M is then taken to be the infimum over all such radii [START_REF] Lee | Riemannian Manifolds: an Introduction to Curvature[END_REF]. In this regard, Assumption 4 simply serves to ensure that the exponential map is invertible at consecutive iterates of (RRM) so no local topological complications can arise. This assumption is automatically satisfied in closed manifolds (independent of curvature), as well as in non-positively curved manifolds -such as Cartan-Hadamard spaces and the like [START_REF] Lee | Riemannian Manifolds: an Introduction to Curvature[END_REF][START_REF] Lee | Introduction to Smooth Manifolds[END_REF]. This assumption (and its variants) is also standard in the literature, cf. [START_REF] Bonnabel | Stochastic gradient descent on Riemannian manifolds[END_REF][START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF][START_REF] Sun | Escaping from saddle points on riemannian manifolds[END_REF] and references therein.

4.2.

Avoidance of saddle points. We are now in a position to state our main avoidance result:

Theorem 1. Let X n , n = 1, 2, . . . , be the sequence of states generated by (RRM), and let S be a strict saddle manifold of f . Then, under Assumptions 1-4, we have

P(dist(S, X n ) → 0 as n → ∞) = 0 (14)
where dist(S,

X n ) = inf x∈S dist(x, X n ) denotes the (Riemannian) distance of X n from S.
Before discussing the proof of Theorem 1, it is worthwhile to compare our work with its closest antecedents. First, in regard to the general avoidance theory in Euclidean spaces [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF][START_REF] Benaïm | Asymptotic pseudotrajectories and chain recurrent flows, with applications[END_REF], the statement is similar in scope (avoidance of unstable manifolds with probability 1), but the techniques and challenges involved are very different. The reason for this is simple: the additive, vector space structure of R m is ingrained at every step of the way in the Euclidean analysis of [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF], and adapting the various constructions to a manifold setting can be a complicated affair. For an illustration of the technical difficulties involved, see the recent stochastic approximation analysis of [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF].

By contrast, the recent results of [START_REF] Criscitiello | Efficiently escaping saddle points on manifolds[END_REF][START_REF] Sun | Escaping from saddle points on riemannian manifolds[END_REF] paint a complementary picture: they concern Riemannian problems but, at their core, they are deterministic results. More precisely, the noise in [START_REF] Criscitiello | Efficiently escaping saddle points on manifolds[END_REF][START_REF] Sun | Escaping from saddle points on riemannian manifolds[END_REF] is actually injected in an otherwise deterministic gradient scheme to facilitate the escape from flat regions in the vicinity of a saddle point; other than that, the magnitude of the noise must be proportional to the solver's desired accuracy, and hence is typically extremely small. As a result, the analysis of [START_REF] Criscitiello | Efficiently escaping saddle points on manifolds[END_REF][START_REF] Sun | Escaping from saddle points on riemannian manifolds[END_REF] cannot be extended to bona fide stochastic schemes -like (RSGD) -which also explains why these results involve a constant step-size (as opposed to a decreasing step-size schedule, which is required to guarantee trajectory convergence in settings with persistent noise). In this regard, Theorem 1 simultaneously complements the stochastic analysis of [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF][START_REF] Benaïm | Asymptotic pseudotrajectories and chain recurrent flows, with applications[END_REF] to Riemannian problems, and the Riemannian analysis of [START_REF] Criscitiello | Efficiently escaping saddle points on manifolds[END_REF][START_REF] Sun | Escaping from saddle points on riemannian manifolds[END_REF] to a stochastic setting.

Proof outline. To facilitate the reading of our proof, we provide below a detailed outline of the main steps and techniques involved therein, deferring the full proof to Appendix B. We begin with a high-level description of our proof strategy and then encode the main arguments in a series of steps right after.

For the purposes of illustration, suppose that M is a subset of R m . Then, given a tangent vector z ∈ T x M, we define the geodesic offset (see Fig. 1) from x along z as

∆(x; z) = exp x (z) -x -z (15)
i.e., as the difference between the geodesic emanating from x along z and its first-order approximation relative to x in the ambient space R m (with all differences expressed in the ordinary vector space structure of R m ). The offset ∆(x; z) is readily checked to be second-order in z so, while the curve x + tz does not in general induce a retraction on the target manifold M (in particular, the point x + tz may not even belong to M), the converse is true: the map exp x (z) is always a retraction on the ambient, Euclidean space R m . In this way, the basic iteration (RRM) can be expressed as

X n+1 = exp Xn (γ n vn ) = X n + γ n vn + ∆(X n ; γ n vn ) (16)
leading to the fundamental question below: What is the maximum offset ϵ n := ∆(X n ; γ n vn ) that can be tolerated by a Euclidean stochastic approximation algorithm to avoid saddle points? A key technical step in our work is to develop the means to control the offset term ϵ n under (RRM) under a sufficiently broad class of assumptions that includes Algorithms 1-5. Crucially, this step is made possible thanks to the very recent -and technical -stochastic approximation work of [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF]. To help the reader navigate our proof strategy, we outline the main steps below, focusing for simplicity on the case of a single saddle point.

Step 1: From discrete to continuous time (and back). Let x be a strict saddle point of f . By the stable manifold theorem [START_REF] Shub | Global Stability of Dynamical Systems[END_REF], the set of all initializations such that the Riemannian gradient flow ẋ(t) = -gradf (x(t)) (RGF) converges to x is of measure 0. Then, assuming for the moment that the geodesic offset error ϵ n = ∆(X n ; γ n vn ) in ( 16) is sufficiently small, the iterates of (RRM) can be seen as a noisy, approximate Euler discretization of (RGF); as such, it is reasonable to expect that the induced trajectories of (RRM) will never converge to x.

To make this intuition precise, our first step will be to show that the iterates of (RRM) comprise an asymptotic pseudotrajectory of (RGF) in the sense of Benaïm [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF], i.e., they asymptotically track the orbits of (RGF) with arbitrary precision over windows of arbitrary length. To formalize this, define the "effective time" variable τ n = n-1 k=1 γ k and the associated geodesic interpolation X(t) of X n as

X(t) = exp Xn ((t -τ n )v n ) for all t ∈ [τ n , τ n+1 ), n ≥ 1 (GI)
so, by construction, (a) X(τ n ) = X n for all n; and (b) each segment of X(t) is a geodesic. Then, letting Φ : R + × M → M denote the flow of (RGF) -i.e., Φ h (x) is simply the position at time h ≥ 0 of the solution orbit of (RGF) that starts at x ∈ M -we will say that X(t) is an APT of (RGF) if, for all T > 0, we have

lim t→∞ sup 0≤h≤T dist(X(t + h), Φ h (X(t))) = 0. (APT)
This requirement is non-trivial, and our first technical result is to guarantee precisely this: Theorem 2. Suppose that Assumptions 2-4 hold. Then, with probability 1, the geodesic interpolation X(t) of the sequence of iterates X n , n = 1, 2, . . . , generated by (RRM) is an APT of (RGF). A version of Theorem 2 was very recently derived by [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF] under a different set of assumptions: On the one hand, [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF] imposes a much more restrictive step-size schedule for γ n (square summability) but, on the other hand, it only posits that the noise increments U n are bounded in L 2 (as opposed to L ∞ in our case). Our proof relies on the same construction of the Picard iteration map as [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF], but otherwise diverges significantly in the probabilistic analysis required to establish (APT).

Step 2: From Riemannian to Euclidean schemes (and back). Albeit crucial, the APT property is decidedly not enough to guarantee avoidance: after all, the constant orbit X(t) = x for all t ≥ 0 is trivially an APT of (RGF) but, of course, it does not avoid x.

To proceed, we will need to exploit the precise update structure of (RRM) in conjunction with the stable manifold theorem applied to (RGF).

In the Euclidean case, this is achieved by means of an intricate Lyapunov function argument, originally due to [START_REF] Benaïm | Dynamics of Morse-Smale urn processes[END_REF]. Our second step is to devise a new geometric argument to reduce the analysis from an arbitrary intrinsic manifold to an isometrically embedded submanifold of R m . This step is carried out by a combination of the celebrated Nash embedding theorem and a (smooth) Tietze extension argument to rewrite (RRM) as a "corrected" Robbins-Monro scheme on R m that actually evolves on M. This construction also requires a "perturbation analysis" to ensure that certain subtle topological issues do not arise when we invoke the stable manifold theorem; we present the details in Appendix B.

Step 3: Controlling the geodesic offset. As we briefly described in the beginning of the proof overview, this Euclidean reframing of (RRM) introduces an intrinsic offset error ϵ n = ∆(X n ; γ n vn ), which is difficult to analyze in detail (the offset incurred by a retraction on M is of similar order, so the exponential-retraction distinction is not important at this stage). Our crucial observation here is that, under our blanket assumptions, ϵ n is small relative to γ n and, in particular, ϵ n = O(γ 2 n ). Thanks to this bound, we are able to leverage a series of stochastic bounds -originally developed by Pemantle [START_REF] Pemantle | Nonconvergence to unstable points in urn models and stochastic aproximations[END_REF] -to show that the probability that these terms will have an adverse effect on exiting the center manifold of x is zero (this is also where Assumption 3 comes in). We formalize this in Appendix B; Theorem 1 then follows by putting everything together. Remark. A concept similar to our geodesic offset ∆(x; z) has been explored in the reverse direction by the very recent works [START_REF] Bianchi | Stochastic subgradient descent escapes active strict saddles on weakly convex functions[END_REF][START_REF] Davis | Subgradient methods near active manifolds: saddle point avoidance, local convergence, and asymptotic normality[END_REF] whose goal was to study of avoidance of Euclidean subgradient methods as an inexact Riemannian gradient scheme. They further show that this inexact Riemannian gradient descent can avoid saddle points if uniform noise is injected. While their core idea bears some similarity to ours, it remains unclear how to apply the analysis in [START_REF] Bianchi | Stochastic subgradient descent escapes active strict saddles on weakly convex functions[END_REF][START_REF] Davis | Subgradient methods near active manifolds: saddle point avoidance, local convergence, and asymptotic normality[END_REF] to handle general RRM schemes, such as retraction-based and natural policy gradient methods.

Applications.

As an illustration of the generality of Theorem 1, we now instantiate it to the range of specific algorithms discussed in Section 3.2. Since all these algorithms are run with gradient input generated by (SFO), applying Theorem 1 would require mapping the requirements of Assumption 3 to the primitives of (SFO). A convenient way to achieve this is by means of the proposition below:

Proposition 1. Suppose that Algorithms 1-5 are run with a gradient oracle V (x; θ) = v(x) + err(x; θ) such that

∥err(x; θ)∥ x ≤ σ(x) and E[[⟨err(x; θ), z⟩ x ] + ] ≥ ζ(x) (17)
for all z ∈ T x M, ∥z∥ x = 1, and for suitable functions ζ, σ : M → R + with σ bounded on bounded subsets of M and inf x ζ(x) > 0. Then, under Assumptions 2 and 4, the conclusion of Theorem 1 holds, that is, Algorithms 1-5 avoid strict saddle manifolds of f .

The proof of Proposition 1 is deferred to Appendix B; we only note here that its proof mainly hinges on verifying the bias requirement ∥b n ∥ Xn = O(γ n ) of ( 13) by means of (i ) the boundedness of the error function σ(x) on bounded subsets of M; and (ii) controlling the maximal deviation between a retraction and the exponential map for input vectors bounded by ϱ.

Numerical Illustrations

In this section, we aim to demonstrate the practical applicability of the theoretical framework proposed in our paper by providing numerical illustrations. To do so, we utilize a 2-dimensional torus as the optimization landscape, where the complexity and multi-modal nature of the objective function can be easily visualized in Fig. 2.

Our objective function features three saddle points (in black) and one global minimizer (in red). We subject two RRM schemes, i.e., (RSGD) and (RSEG), to initialization in proximity to these saddle points. This strategic choice rigorously tests their ability to navigate and converge to the global optimum. In line with our theoretical predictions, both RRM methods avoid the saddle points and eventually reach the desirable global minimum. This empirical confirmation reinforces the central message of our paper: stochastic Riemannian schemes only converge to local minimizers.

Conclusions and future work

In this paper, we addressed the question of when Riemannian stochastic algorithms can effectively evade saddle points, focusing on the broad category of Riemannian Robbins-Monro schemes. We introduced a novel framework for analyzing the avoidance of Riemannian saddle points within the RRM framework, which encompasses many commonly used Riemannian stochastic algorithms, including retraction-based algorithms. Our framework builds upon the notion of strict saddle points and provides a set of easily verifiable conditions that guarantee the avoidance of such traps.

Our work paves the way for several promising research directions in learning with Riemannian methods. One intriguing avenue for exploration is the investigation of whether Riemannian zeroth-order methods, such as the Riemannian extension of the work by Kiefer & Wolfowitz [START_REF] Kiefer | Stochastic estimation of the maximum of a regression function[END_REF], can effectively evade strict saddle points. We believe that combining the insights from the asymptotic pseudotrajectory theory with Euclidean analysis can shed light on this question and provide valuable insights into the behavior of these methods in the Riemannian setting.

Furthermore, an interesting direction for future research is the extension of the avoidance of unstable limit cycles in Euclidean min-max optimization, as studied by Hsieh et al. [START_REF] Hsieh | The limits of min-max optimization algorithms: Convergence to spurious non-critical sets[END_REF], to the realm of Riemannian games. Investigating the avoidance of unstable limit cycles in this context has the potential to uncover novel phenomena specific to the manifold settings, leading to a deeper understanding on the intricate dynamics and strategies involved in Riemannian games.

The RRM representation of (RPPM) is then obtained by taking b n = Γ Xn+1→Xn (v(X n+1 ))v(X n ) and U n = 0 in the decomposition (4) of the error term W n of (RRM). If, in additional, the true gradient v(X n+1 ) is replaced by an oracle V (X n+1 ; θ n+1 ), then (RPPM) becomes the stochastic version of RPPM by setting

b n = E[Γ Xn+1→Xn (V (X n+1 ; θ n+1 )) -v(X n ) | F n ] and U n = Γ Xn+1→Xn (V (X n+1 ; θ n+1 )) -v(X n ) -b n .
For a detailed discussion, see [START_REF] Ferreira | Proximal point algorithm on Riemannnian manifolds[END_REF] and references therein.

♢

Algorithm 7 (Riemannian stochastic extra-gradient). Inspired by the original work of Korpelevich [START_REF] Korpelevich | The extragradient method for finding saddle points and other problems[END_REF], the Riemannian stochastic extra-gradient (RSEG) method [START_REF] Neto | An extragradient method for equilibrium problems on Hadamard manifolds[END_REF][START_REF] Tang | Korpelevich's method for variational inequality problems on Hadamard manifolds[END_REF] proceeds as

X + n = exp Xn (γ n V (X n ; θ n )), X n+1 = exp Xn (Γ X + n →Xn (γ n V (X + n ; θ + n ))) (RSEG)
where θ n and θ + n are independent seeds for (SFO). Thus, to cast (RSEG) in the RRM framework, it suffices to take

U n = Γ X + n →Xn (err(X + n ; θ + n )) and b n = Γ X + n →Xn (v(X + n )) - v(X n ).
♢ Under the assumptions in Proposition 1, one can show that Algorithms 6-7 also avoid strict saddle points of f ; we provide the relevant details in Appendix B.3.

Appendix B. Missing proofs from Section 4

B.1. Proof of Theorem 2. We begin by proving Theorem 2, which will play a crucial role in our proof of Theorem 1. For the reader's convenience, we restate the result below: Theorem 2. Suppose that Assumptions 2-4 hold. Then, with probability 1, the geodesic interpolation X(t) of the sequence of iterates X n , n = 1, 2, . . . , generated by (RRM) is an APT of (RGF).

Proof. To begin, let {e i (n)} d i=1 be an arbitrary sequence of orthonormal bases for T Xn M, and let U n be the (Euclidean) noise vector composed of components of the noise U n in the basis {e

i (n)} i=1...d , viz. U i,n := ⟨U n , e i (n)⟩ Xn . (B.1)
It is then easy to see that E[U n |F n ] = 0, and, moreover

U n = ∥U n ∥ Xn ≤ σ (B.2)
by Assumption 3. Then, following Benaïm [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF], consider the "continuous-to-discrete" counter

M (t) = sup{n ≥ 1 : t ≥ τ n } (B.3)
which measures the number of iterations required for the effective time τ n = n-1 k=1 γ k to reach a given timestamp t ≥ 0. We further denote the piecewise-constant interpolation of the noise sequence as Moving forward, since X n → S by assumption, we will also have

Ū (t) = U n for all t ∈ [τ n , τ n+1 ), n ≥ 1 (B.
sup n dist(X n , x) =: R < ∞ for all x ∈ S (B.6)
for some (possibly random) R ≥ 0. Moreover, since f is assumed to be C 2 , (B.6) implies that

sup n ∥v(X n )∥ Xn =: G < ∞ (B.7)
for some (possibly random) non-negative constant G ≥ 0. Moreover, since the manifold M is assumed to be smooth, the sectional curvatures at each X n , n = 1, 2, . . . , must be likewise bounded by some constant K max . Then, by the analysis of [START_REF] Karimi | The dynamics of Riemannian Robbins-Monro algorithms[END_REF]Eq. 53], there exists a constant C ≡ C L,G,Kmax,R depending only on L, G, K max and R such that

sup 0≤h≤T dist(X(t+h), Φ h (X(t))) ≤ C L,G,Kmax,R • sup n≥M (t) (∥b n ∥ Xn + γ n ) + ∆(t -1; T + 1)
(B.8) By Assumption 3, we have ∥b n ∥ Xn = O(γ n ). Since γ n → 0, it suffices to show that ∆(t; T ) → 0 with probability 1 under Assumptions 2 and 3. This is equivalent to showing that, for any ε > 0, we have

lim t→∞ ∆(t; T ) ≤ ε with probability 1. (B.9)
To this end, let n = M (t) and recall that, by (B.2), we have

E exp ⟨w, Ū n ⟩ F n ≤ exp σ 2 2 ∥w∥ 2 (B.10)
for all w ∈ R d where d is the dimension of M. Therefore, for each w ∈ R d , the sequence Y n (w) defined by

Y n (w) := exp n k=1 ⟨w, γ k Ū k ⟩ - σ 2 ∥w∥ 2 2 n k=1 γ 2 k (B.11)
is a supermartingale. Since Y n is a supermartingale, we have

P sup n<k≤M (τn+T ) k-1 i=n ⟨w, γ i Ū i ⟩ ≥ δ ≤ P   sup n<k≤M (τn+T ) Y k (w) ≥ Y n (w) exp   δ - σ 2 ∥w∥ 2 2 M (τn+T )-1 i=n γ 2 i     ≤ exp   σ 2 ∥w∥ 2 2 M (τn+T )-1 i=n γ 2 i -δ   (B.12)
for any δ > 0. Now, let e i be the i-th basis vector of R d . Then, by (B.12), we have

P sup n<k≤M (τn+T ) k-1 i=n ⟨±de i , γ k Ū k ⟩ ≥ ε = P sup n<k≤M (τn+T ) k-1 i=n ⟨±ε -1 δde i , γ k Ū k ⟩ ≥ δ ≤ exp   σ 2 δ 2 d 2 2ε 2 M (τn+T )-1 i=n γ 2 i -δ   . (B.13)
Optimizing (B.13) over δ, we get

P sup n<k≤M (τn+T ) k-1 i=n ⟨±de i , γ k Ū k ⟩ ≥ ε ≤ exp - ε 2 2σ 2 d 2 M (τn+T )-1 i=n γ 2 i . (B.14)
Since γ n → 0, with loss of generality we may assume that γ n ≤ 1, and hence

P sup n<k≤M (τn+T ) k-1 i=n ⟨±de i , γ k Ū k ⟩ ≥ ε ≤ exp - ε 2 2σ 2 d 2 M (τn+T )-1 i=n γ i ≤ exp - ε 2 2σ 2 d 2 t+T t γ(t) (B.15)
where, analogously to (B.4), we have defined the piece-wise constant interpolated step-size sequence

γ(t) = γ n for all t ∈ [τ n , τ n+1 ), n ≥ 1. (B.16) Since k-1 i=n γ i Ū i ≥ ε ⇒ ∃ i such that k-1 i=n ⟨±de i , γ i Ū i ⟩ ≥ ε, (B.17)
by the union bound, we have

P(∆(t, T ) ≥ ε) ≤ 2d • exp - ε 2 2σ 2 d 2 t+T t γ(t) ≤ 2d • exp - ε 2 2σ 2 d 2 T γ(s) (B.18)
for some t ≤ s ≤ t + T . Therefore, by setting λ := exp

-ε 2 2σ 2 d 2 T < 1, we have k P(∆(kT, T ) ≥ ε) ≤ 2d k λ 1/γ k < ∞ (B.19)
by Assumption 2. The Borel-Cantelli Lemma then implies that the following event happens almost surely:

lim k→∞ ∆(kT ; T ) ≤ ε. (B. 20
)
The proof is finished by noting that, for kT ≤ t < (k + 1)T ,

∆(t; T ) ≤ 2∆(kT ; T ) + ∆(kT + T ; T ) (B.21) by triangle inequality. ■ B.2. Proof of Theorem 1.
We are now in a position to present our proof of Theorem 1, which we restate below for convenience:

Theorem 1. Let X n , n = 1, 2, . . . , be the sequence of states generated by (RRM), and let S be a strict saddle manifold of f . Then, under Assumptions 1-4, we have

P(dist(S, X n ) → 0 as n → ∞) = 0 (14)
where dist(S, X n ) = inf x∈S dist(x, X n ) denotes the (Riemannian) distance of X n from S.

Proof. Assume that X n → S. We will show that this event has zero probability in a series of steps which we outline below.

Step 1: Isometrically embedded Robbins-Monro iterates. Since M is assumed to be smooth, the second Nash embedding theorem [START_REF] Kobayashi | Foundations of Differential Geometry[END_REF] implies there exists a smooth and isometric embedding ι : M → R M such that, for all x ∈ M and all z, w ∈ T x M, we have

⟨z, w⟩ x = ⟨D ι x (z), D ι x (w)⟩. (B. 22
)
Since ι is an embedding, it is surjective. Since it is isometric, it preserves distance and hence must be one-to-one. Therefore, ι is an diffeomorphism since it is also smooth. We can therefore define the pushforward of the vector field v on M to a vector field on the image M E ⊂ R M in the usual way as

v E 0 (x E ) := D ι x v(x) for all x E = ι(x) ∈ R M . (B.23)
We also set S E := ι(S).

By the Tietze extension theorem and the smooth manifold extension lemma [START_REF] Lee | Introduction to Smooth Manifolds[END_REF], v E 0 (x E ) can be extended to a Lipschitz continuous vector field on all of R M , which we still denote by v E 0 (x E ). To avoid trivialities, we will also need to ensure that v E 0 (x E ) is not 0 in a neighborhood of S E : If this is the case, then we set our target field v E (x E ) := v E 0 (x E ); otherwise, let 1 denote the vector of 1's in all coordinates, and define a new vector field

v E on R M as v E (x E ) := v E 0 (x E ) + dist E (x E , S E ) 2 • 1 (B.24)
where dist E (x E , S E ) := inf y E ∈S E ∥x E -y E ∥. Obviously, this new vector field agrees with v E 0 (x E ) on M E and therefore is still the pushforward of v under ι. Moreover, it is not uniformly 0 in a neighborhood of S E . [It is worth noting that the so-defined vector field v E is in general not the (Euclidean) gradient of any function, a fact which presents significant difficulty to our analysis.]

Step 2: S E is an unstable invariant set. Our next goal is to show that there exists an unstable neighborhood U E around S E in the following sense: First, for each x ∈ S, consider its image xE = ι(x) ∈ S E . Since v E agrees with the pushforward of v ≡ gradf under ι, and since ι is an isometry, we have the following relation for all tangent vector z ∈ T xM:

⟨D v E x E D ι x(z), D ι x(z)⟩ = ⟨Hess f (x)z, z⟩ x. (B.25)
Since x is a strict saddle point, (B.25) implies that λ min D v E xE < -c -< 0 for all xE ∈ S E . By an established series of arguments [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF][START_REF] Benaïm | Asymptotic pseudotrajectories and chain recurrent flows, with applications[END_REF][START_REF] Mertikopoulos | On the almost sure convergence of stochastic gradient descent in non-convex problems[END_REF], using the stable manifold theorem for a strict saddle [START_REF] Shub | Global Stability of Dynamical Systems[END_REF] and the transversality of the strict saddle manifold [START_REF] Abraham | Foundations of mechanics[END_REF], there exists a (M -m)-dimensional embedded submanifold Q E in R M that contains S E . [Here, 1 ≤ m ≤ M , and M -m represents the dimension of the unstable manifold of v E .] Moreover, writing Φ E for the flow generated by v E , it follows that Q E is locally invariant under Φ E . Hence, there exists a neighborhood N E of S E in R M and a positive time t 0 such that for all |t| ≤ t 0 , the following inclusion holds:

Φ E t (N E ∩ Q E ) ⊂ Q E . (B.26)
To proceed, note that R M can be decomposed further as the direct sum of the tangent space to Q E at x E , denoted by T x E Q E , and an additional complementary subspace denoted by E u x E :

R M = T x E Q E ⊕ E u x E . (B.27)
The mapping x E → E u x E is continuous, where x E varies over S E and E u x E belongs to the Grassmanian manifold G(m, M ). It is important to note that E u

x E contains at least one direction in T x E M E due to (B.25). Then, for all t ∈ R and x ∈ S E , the Jacobian of

Φ E t evaluated at x E maps E u x E to E u Φ E t (x E ) , i.e., D Φ E t (x E )E u x E = E u Φ E t (x E ) . (B.28)
Finally, and most importantly, we have the following characterization that formalizes the idea that all directions in the unstable manifold should diverge under Φ E t : There exist positive constants c and C such that for all x E ∈ S E , w E ∈ E u x E , and t ≥ 0, the following inequality holds:

∥D

Φ E t (x E )w E ∥ ≥ Ce ct ∥w E ∥. (B.29)
The above verifies all the conditions for a unstable invariant set for S E in the sense of Benaïm [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF]. A deep result by Benaïm & Hirsch [START_REF] Benaïm | Dynamics of Morse-Smale urn processes[END_REF] then asserts the existence of a local Lyapunov function near a neighborhood of S E , whose construction we outline below.

Step 3: Local Lyapunov function η E . For a right-differentiable function η E : R M → R we define its right derivative D η E applied to a vector h ∈ R M by

D η E (x E )h E = lim t→0 + η E (x E + th E ) -η E (x E ) t . (B.30) If η E is differentiable, then (B.30) is simply ⟨∇ η E (x E ), h E ⟩.
In view of all this, Benaïm [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF] provides the following crucial result:

Proposition B.1 (Benaïm, 1999, Prop. 9.5). There exists a compact neighborhood U E (S E ) of S E , positive numbers l, β > 0, and a map η

E : U E (S E ) → R + such that η E (x E ) = 0 if and only if x E ∈ Q E ,
and the following holds:

(i) η E is C 2 on U E (S E ) \ Q E . (ii) For all x E ∈ U E (S E ) ∩ Q E , η E admits a right derivative D η E (x E ) : R M → R M
which is Lipschitz, convex and positively homogeneous. (iii) There exists k > 0 and a neighborhood W E ⊂ R M of 0 such that for all

x E ∈ U E (S E ) and z E ∈ W E , η E (x E + z E ) ≥ η E (x E ) + D η E (x E )z E -k∥z E ∥ 2 . (B.31) (iv) There exists c 1 > 0 such that for all x E ∈ U E (S E ) \ Q E ∥Π M E (D η E (x E ))∥ ≥ c 1 (B.32)
where Π M E is the projection on M E . 2 In addition, for all

x E ∈ U E (S E ) ∩ Q E and z E ∈ R M ⟨D η E (x E ), z E ⟩ ≥ c 1 ∥z E -D Π(x E )z E ∥ (B. 33
)
where Π is the projection of a neighborhood of

S E onto Q E . (v) For all x E ∈ U E (S E ) ∩ Q E , w E ∈ T x E Q E and z E ∈ R M , D η E (x E )(w E + z E ) = D η E (x E )z E . (B.34) (vi) For all x E ∈ U E (S E ) we have D η E (x)v E (x E ) ≥ βη E (x E ). (B.35)
The function η E will serve as a local "energy function" that plays an instrumental role in our analysis; the well-posedness of Π is guaranteed by [START_REF] Hirsch | Differential Topology[END_REF]Chap 4]. 2 The original statement in [START_REF] Benaïm | Dynamics of stochastic approximation algorithms[END_REF] is ∥D η E (x E )∥ ≥ c 1 . The inequality in (B.32) is obtained via the same proof and noting that E u

x E contains at least one direction in T x E M E .

Step 4: Geodesic offset. Consider the image of an RRM scheme X

E n := ι(X n ). If X E n / ∈ U E (S E
) for all n, then there is nothing to prove. Otherwise, without loss of generality we may assume that X E 1 ∈ U E (S E ). Accordingly, define the first exit time T from U E (S E ) as

T := inf{k ≥ 1 : X E n / ∈ U E (S E )}.
(B.36) Evidently, T is a stopping time adaptive to F n , so it suffices to show that3 P(T = ∞) = 0.

(B.37)

To this end, a notion that plays a central role in our analysis is the geodesic offet, defined as follows. Define the pushforward of the respective noise and bias vectors in the RRM scheme X n by

U E n := D ι Xn U n , b E n := D ι Xn b n . (B.38)
It is important to remember that U E n ∈ T X E n M E , a fact that we will use freely in the sequel.

We now formally define the geodesic offset ∆(x; z) ∈ R M as, for any x ∈ M and z ∈ T x M,

∆(x; z) := ι(exp x (z)) -ι(x) -D ι x (z). (B.39)
By Assumption 4, there exists ϱ > 0 such that, for all ∥z∥ x < ϱ, the exponential mapping is the unique minimizing geodesic. Furthermore, for all such z's, define the curve γ

E (t) := ι(x) + t D ι x (z), then γ E (0) = ι(x), γE (0) = D ι x (z) (B.40)
so that γ E (t) agrees with the image of the geodesics ι(exp x (tz)). As a result, for any ∥z∥ x < ϱ, we have ∆(x; z) = O(∥z∥ 2 x ). Now, setting x ← X n and z ← γ n (v(X n )+U n +b n ), we have

X E n+1 = X E n + γ n (v E (X E n ) + U E n + b E n ) + ϵ E n (B.41)
where ϵ E n := ∆(X n ; γ n (v(X n )+U n +b n )). By Assumption 3, we know that U E n +b E n = O(1) almost surely. Moreover, since v is smooth, on the event T = ∞, X E n ∈ U E (S E ) for all n and therefore sup n≥1 ∥v E (X E n )∥ < ∞. Since γ n → 0, for any n large enough, we get

ϵ E n = O(γ 2 n ). (B.42)
Now, define two sequences of random variables {Y n } n≥1 and {S n } n≥1 as

Y n+1 = η E (X E n+1 ) -η E (X E n ) 1 {n≤T } +γ n 1 {n>T } , (B.43a) S 0 = η E (X E 0 ), S n = S 0 + n k=1 Y k . (B.43b)
The importance of these sequences is that the event {T = ∞} is contained in the event {S n → 0}. To see this, assume T = ∞. Then we have

Y n+1 = η E (X E n+1 ) -η E (X E n ) and S n = η E (X n
({X E n }) of {X E n } is a nonempty compact invariant subset of U E (S E ), so that for all y E ∈ L({X E n }) and t ∈ R, Φ E t (y E ) ∈ U E (S E ). But then Proposition B.1(vi ) implies that η E (Φ E t (y E )) ≥ e βt η E (y E ) for all t > 0, forcing η E (y E ) to be zero. Since η E (x E ) = 0 if and only if x E ∈ Q E , we have L({X E n }) ⊂ Q E , which implies S n = η E (X E n ) → 0.
Therefore, the rest of the proof is devoted to showing that P(lim n→∞ S n = 0) = 0.

Step 5: Probabilistic estimates. To this end, we will need two technical lemmas, originally due to Pemantle [START_REF] Pemantle | Vertex-reinforced random walk[END_REF], and extended to their current form by Benaïm & Hirsch [START_REF] Benaïm | Asymptotic pseudotrajectories and chain recurrent flows, with applications[END_REF]. Our proof will be concluded by verifying all the premises of Lemmas B.1-B.2. To that end, note first that the sequence S n defined in (B.43b) is nonnegative by construction. We will then separate the analysis into two cases: k therein. The verification process then proceeds as follows:

• Verifying Lemma B.1(i ) and (iv): By the Lipschitz continuity of η E , we know that

∥η E X E n -η E X E n+1 ∥ ≤ L ′ ∥X E n -X E n+1 ∥ = γ n ∥v E (X E n ) + U E n + b E n ∥ (B.47)
where L ′ is the Lipschitz constant of η E . We have seen in the analysis of (B.42) that ∥v E (X

E n ) + U E n + b E n ∥ = O(
η E (X E n+1 ) -η E (X E n ) ≥ γ n D η E (X E n )(v E (X E n ) + U E n + b E n ) + D η E (X E n )ϵ E n -kγ 2 n (∥v E ∥ + ∥U E n ∥ + ∥b E n ∥)
Remark. In additional to the claimed Algorithms 1-5, we will further prove the same conclusion for the two algorithms considered in Appendix A.

Proof. By Theorem 1, it suffices to verify Assumption 3 under (17) and the event dist(X n , S) → 0. We proceed method by method.

Algorithm 1. Since b n = 0 in Algorithm 1, Assumption 3 holds trivially by [START_REF] Bonnabel | Stochastic gradient descent on Riemannian manifolds[END_REF].

Algorithms 2, 3 and 5. By definition, R x (z) is a smooth map and hence satisfies lim z→0 R x (z) = x. On the event dist(X n , S) → 0, we have v(X n ) + U n + b n = O(1), and therefore X n+1 lies in the injectivity radius of X n with probability 1 for n large enough. As a result, the mapping log Xn (X n+1 ) is well-define for all n large enough. We first consider Algorithms 2 and 5 whose proofs are identical since they are both are the form:

X n+1 = R Xn (γ n V (X n ; θ n )). (B.65)
Let ṽn ∈ T Xn M be the vector such that exp Xn (γ n ṽn ) = X n+1 , i.e.,

γ n ṽn = log Xn R Xn (γ n V (X n ; θ n )) . (B.66)

Then (B.65) is an RRM scheme with W n = ṽn -v(X n ) where ṽn is defined in (B.66). Now, consider the curve c(t) := R Xn (tV (X n ; θ n )). By [START_REF] Bonnabel | Stochastic gradient descent on Riemannian manifolds[END_REF], on the event dist(X n , S) → 0, the curve c(t) lies in the injectivity radius of X n almost surely for all t ∈ [0, γ n ] and all n large enough. Let ĉ(t) be the smooth curve of c(t) in the normal coordinate with base X n and an arbitrary orthonormal frame, and let Xn+1 be the normal coordinate of X n+1 . Also, let ṽN n be the (Euclidean) vector of ṽn expanded in the chosen orthonormal basis, and define V N (X n ; θ n ) and err N (X n ; θ n ) similarly. By definition, Xn+1 is nothing but γ n ṽN n . In addition, by [START_REF] Bonnabel | Stochastic gradient descent on Riemannian manifolds[END_REF], we have ∥err N (X n ; θ n )∥ = ∥err(X n ; θ n )∥ Xn ≤ σ (B.67) for some σ < ∞. Since X n = c(0) and X n+1 = c(γ n ), by properties of a retraction map we must have

γ n ṽN n = ĉ(γ n ) = ĉ(0) + γ n ċ(0) + O γ 2 n ∥ ċ(0)∥ 2 2 = γ n V N (X n ; θ n ) + O γ 2 n ∥V (X n ; θ n )∥ 2

Figure 1 :

 1 Figure 1: The geodesic offset ∆(x; z).

  (a) Riemannian stochastic gradient descent (b) Riemannian stochastic extra-gradient

Figure 2 :

 2 Figure 2: Avoidance of saddle points by stochastic RRM schemes on the torus.

4 )

 4 and we let ∆(t; T ) := sup 0≤h≤T t+h t Ū (s) ds . (B.5)

  ) by Eqs. (B.43a) and (B.43b). In addition, since {X E n } remains in U E (S E ) by definition of the stopping time T , Theorem 2 combined with [10, Theorem 5.7] asserts that the limit set L

Lemma B. 1 .

 1 Let S n be a nonnegative stochastic process,S n = S 0 + n k=1 Y k where Y n is F n -measurable. Let α n := ∞ k=n γ 2 k . Assume there exist a sequence 0 ≤ ε n = o( √ α n ),constants a 1 , a 2 > 0 and an integer N 0 such that for all n ≥ N 0 , (i)|Y n | = o( √ α n ).(ii)1 {Sn>εn} E[Y n+1 |F n ] ≥ 0. (iii) E[S 2 n+1 -S 2 n |F n ] ≥ a 1 γ 2 n . (iv) E[Y 2 n+1 |F n ] ≤ a 2 γ 2 n . Then P(lim n→∞ S n = 0) = 0. Lemma B.2. Let S n be a nonnegative stochastic process, S n = S 0 + n k=1 Y k where Y n is F n -measurableand |Y n | ≤ C almost surely for some constant C. Assume that n γ 2 n = ∞, and there exists c > 0, N ′ ∈ N such that for all n ≥ N ′ , E[S 2 n+1 -S 2 n | F n ] ≥ cγ 2 n .

Case 1 :

 1 Square-summable step-sizes ( n γ 2 n < ∞). In this case, we havelim γ n = o ∞ k=n γ 2k . This fact will be used in the proof when we invoke Lemma B.1 below with ε n = O(γ n ) and α n = ∞ k=n γ 2

1 )

 1 almost surely by Assumption 3 on the event T = ∞. Therefore, |Y n+1 | = O(γ n ) = o( √ α n ) which implies both Lemma B.1(i ) and (iv). • Verifying Lemma B.1(ii): Let k ′ = k∥v E ∥ + σ where k is given by Proposition B.1(iii) and ∥v E ∥ := sup{v E (x E ) : x E ∈ U E (S E )} and σ is the uniform bound of U n . If n ≤ T , using Proposition B.1(ii ), (iii ), (v) and (vi ) we have

Xn=: 2 Xn=

 2 γ n V N (X n ; θ n ) + γ n bn (B.[START_REF] Zhou | On the convergence of mirror descent beyond stochastic convex programming[END_REF])where bn = O γ n ∥V (X n ; θ n )∥ O(γ n ). Therefore, ∥b n ∥ Xn = ∥E[W n | F n ]∥ Xn = E bn F n = O(γ n ) (B.69)which proves the condition for b n in Assumption 3. On the other hand, (B.68) shows that∥U n ∥ Xn ≤ ∥V N (X n ; θ n ) + bn ∥ + ∥E V N (X n ; θ n ) + bn ∥ = O(1) since ∥V N (X n ; θ n )∥ = O(1)by (17) and bn = O(γ n ). Finally, for any unit vector z ∈ T Xn M, (B.68) implies E[[⟨z, U n ⟩ Xn ] + ] ≥ E[[⟨z, err(X n ; θ n )⟩ Xn ] + ] -∥ bn ∥

Theorem 1 follows from (B.37) because the event {dist(Xn, S) → 0} is contained in {T = ∞}.

Acknowledgments

This work has been partially supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program grant agreement No 815943. YPH acknowledges funding through an ETH Foundations of Data Science (ETH-FDS) postdoctoral fellowship. PM is grateful for financial support by the French National Research Agency (ANR) in the framework of the "Investissements d'avenir" program (ANR-15-IDEX-02), the LabEx PERSYVAL (ANR-11-LABX-0025-01), MIAI@Grenoble Alpes (ANR-19-P3IA-0003), and project MIS 5154714 of the National Recovery and Resilience Plan Greece 2.0 funded by the European Union under the NextGenerationEU Program.

Appendix A. Further examples of RRM schemes

In this section, we provide two additional algorithmic examples supplementing the range of Algorithms 1-5 to illustrate the applicability of our RRM template.

Algorithm 6 (Riemannian proximal point methods). The (deterministic) Riemannian proximal point method (RPPM) [START_REF] Ferreira | Proximal point algorithm on Riemannnian manifolds[END_REF] is an implicit ("backward") update rule of the form log Xn+1 (X n ) = -γ n v(X n+1 ).

(RPPM)

By Assumption 3, there exists a constant c ′ > 0 such that -∥b E n ∥ ≥ -c ′ γ n (a.s.). Combining this with the Lipschitz continuity of η E and (B.42), we can merge the last four terms in (B. [START_REF] Meyer | Linear regression under fixed-rank constraints: a riemannian approach[END_REF] as

for some constant k ′′ > 0. We thus get

since we have assumed the noise to be zero mean. Combining (B.50) and (B.51), we then get

In other words, (B.54) can be rewritten as From (B.49), we deduce

We now claim that

where c 1 is given by Proposition B.1(iv) and ζ is defined in Assumption 3. To see this, recall that ∥U n ∥ Xn < σ by Assumption 3. Moreover, we have

and Proposition B.1(iv) further shows that

where we have used the fact that U E n ∈ T Xn M E and Assumption 3.

By Proposition B.1(iv), Cauchy-Schwartz, and Assumption 3, we get

where v n is the pullback of v E n under ι. Since ι is an isometry, the pullback preserves the inner product, and therefore

On the other hand, we always have

for some b 1 > 0 and large enough n as desired. We have now verified conditions (i)-(iv ) in Lemma B.1. Thus, Lemma B.1 concludes that

which finishes the proof for the case of n γ 2 n < ∞.

Case 2: Non-square-summable step-sizes

for all z ∈ T x M, ∥z∥ x = 1, and for suitable functions ζ, σ : M → R + with σ bounded on bounded subsets of M and inf x ζ(x) > 0. Then, under Assumptions 2 and 4, the conclusion of Theorem 1 holds, that is, Algorithms 1-5 avoid strict saddle manifolds of f .

Since γ n → 0, this finishes the proof of Algorithms 2 and 5. For Algorithm 3, an Euclidean oracle of the form [START_REF] Bonnabel | Stochastic gradient descent on Riemannian manifolds[END_REF] translates to a Riemannian oracle with err ′ (x; θ) := ∇ 2 h(x) -1 err(x; θ). It then suffices to note that, on the event dist(X n , S) → 0, ∇ 2 h(X n ) is both upper and lower bounded.

Algorithms 4, 6 and 7. For (RSEG),

by [START_REF] Bonnabel | Stochastic gradient descent on Riemannian manifolds[END_REF] and the fact that the parallel transport map is a linear isometry. For the bias term, the definition of (RSEG) yields

(B.72) by the same argument as for Algorithms 2, 3 and 5.

For (ROG), we have U n = Γ X + n →Xn (err(X n ; θ + n )) and b n = Γ X + n →Xn (v(X + n )) -v(X n ), so Assumption 3 can be checked exactly as in the case of Algorithm 7 above. The analysis for Algorithm 6 is similar so we omit the details. ■


M Espi

X Charmetant

C Mathieu

L Koppe

C Pelletier

E Kalbacher

S Mezaache

J P Fauvel

D Fouque

O Thaunat

Persistance de la réponse vaccinale anti SARS-CoV-2 chez le patient hémodialysé chronique : intérêt d'un rappel à 6 mois

Introduction

Les patients hémodialysés chroniques présentent un risque accru d'infection par le SARSCoV2 et de décès en cas de COVID-19. Pour atteindre un titre d'anticorps protecteur permet-tant une seroneutralisation virale, les hémodialysés naïfs pour le virus nécessitent un schéma vaccinal intensifié (3 doses de vaccin à ARNm). La persistance dans le temps de cette protection et notamment l'impact du statut d'hémodialysé, d'une infection préalable par le virus et l'efficacité d'un rappel vaccinal ne sont pas connus. Description L'évolution de la réponse humorale (titre d'IgG anti-RBD et capacité de seroneutralisation virale in vitro) et cellulaire (interferon gamma releasing assay, IGRA) a été évaluée chez 62 patients hémodialysés naïfs pour le virus (HD naïf) entre le pic de la réponse vaccinal et 6 mois après la dernière dose. La réponse à un rappel a été évaluée chez les 33 patients ayant rec¸ u une nouvelle dose de vaccin. Méthodes Ces données ont été comparées à celles de 21 volontaires sains (VS) et de 9 hémodialysés ayant développé une immunité hybride (infection puis vaccin ; HD hybride).

Résultats

Six mois après la dernière dose vaccinale, seuls 13/62 (21 %) des HD naïf ont conservé une capacité de neutralisation virale. Ce résultat est similaire à celui des VS, 6/21 (29 %), mais diffère nettement des HD hybride 9/9 (100 %). Par ailleurs, 29/39 (74 %) des HD naïf, qui présentaient une réponse cellulaire détectable au pic la conservait à M6, contre 18/21 (86 %) des VS, et 9/9 (100 %) des HD hybride. Le rappel a permis de restaurer une capa-cité de séroneutralisation virale chez 100 % (33/33) des patients HD naïfs l'ayant rec¸ u.

Conclusion

  Contrairement à la réponse cellulaire, le titre d'anticorps vaccinal décroît rapidement mais de manière similaire chez les hémodialysés et les VS. En dehors d'un contact avec le virus, la majorité des individus (hémodialysés ou pas) n'est plus correctement protégé après 6 mois et doit recevoir une dose de rappel. Déclaration de liens d'intérêts Les auteurs déclarent ne pas avoir de liens d'intérêts. https://doi.org/10.1016/j.nephro.2022.07.144