Hedy Attouch
email: hedy.attouch@umontpellier.fr

• Radu

Ioan Bot

Dang-Khoa Nguyen
email: dang-khoa.nguyen@univie.ac.at

Radu Ioan Bot
email: radu.bot@univie.ac.at

Fast convex optimization via closed-loop time scaling of gradient dynamics

Keywords: fast convex optimization, damped inertial dynamic, time scaling, averaging, closed-loop control, Nesterov and Ravine algorithms, Hessian driven damping, proximal algorithms AMS subject classification 37N40, 46N10, 49M30, 65B99, 65K05, 65K10, 90B50, 90C25

In a Hilbert setting, for convex differentiable optimization, we develop a general framework for adaptive accelerated gradient methods. They are based on damped inertial dynamics where the coefficients are designed in a closed-loop way. Specifically, the damping is a feedback control of the velocity, or of the gradient of the objective function. For this, we develop a closed-loop version of the time scaling and averaging technique introduced by the authors. We thus obtain autonomous inertial dynamics which involve vanishing viscous damping and implicit Hessian driven damping. By simply using the convergence rates for the continuous steepest descent and Jensen's inequality, without the need for further Lyapunov analysis, we show that the trajectories have several remarkable properties at once: they ensure fast convergence of values, fast convergence of the gradients towards zero, and they converge to optimal solutions. Our approach leads to parallel algorithmic results, that we study in the case of proximal algorithms. These are among the very first general results of this type obtained using autonomous dynamics.

Introduction

In a real Hilbert space H, we develop a dynamic approach to the rapid resolution of convex optimization problems which relies on inertial dynamics whose damping is designed as a closed-loop control. We consider the minimization problem min {f (x) : x ∈ H} ,

where, throughout the paper, we make the following assumptions on the function f to be minimized

(A) f : H → R is a convex function of class C 1 ; S = argmin H f = ∅;
∇f is Lipschitz continuous on the bounded sets of H.

(

) 2
Our study is part of the close links between dissipative dynamical systems and optimization algorithms, the latter being obtained by temporal discretization of the continuous dynamics. Our study comes as a natural extension of the authors' previous work [START_REF] Attouch | Fast convex optimization via time scale and averaging of the steepest descent[END_REF] where the technique of time scaling and averaging was used in an open-loop way, giving rise to non-autonomous damped inertial dynamics with fast convergence properties. In the present paper, we take advantage of the simplicity and flexibility of this technique to develop it in a closed-loop way. This will give rise to autonomous damped inertial dynamics with fast convergence properties. Recall that the low-resolution ODE obtained by Su, Boyd, and Candès [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] of the accelerated gradient method of Nesterov, together with the corresponding high-resolution ODE [START_REF] Attouch | First-order algorithms via inertial systems with Hessian driven damping[END_REF], [START_REF] Shi | Understanding the acceleration phenomenon via high-resolution differential equations[END_REF] (which involves an additional Hessian driven damping term) are non-autonomous dynamics, the coefficient of viscous friction being of the form α/t. Our study therefore opens a new path in the field of first-order adaptive optimization methods.

Time scale and averaging: the open-loop approach

Let us first briefly explain the time scaling and averaging method in the open-loop case on a model example (see [START_REF] Attouch | Fast convex optimization via time scale and averaging of the steepest descent[END_REF] for more details). Then we will look at how to develop a corresponding closed-loop approach. As the basic starting dynamic, we consider the continuous steepest descent (SD) ż(s) + ∇f (z(s)) = 0,

for which we have the classical convergence result

f (z (s)) -inf H f = o 1 s as s → +∞.
Then, we make the change of time variable s = τ (t) in (SD), where τ (•) is an increasing function from R + to R + , continuously differentiable, and satisfying lim t→+∞ τ (t) = +∞. Setting y(t) := z(τ (t)), we get ẏ(t) + τ (t)∇f (y(t)) = 0.

The convergence rate becomes

f (y(t)) -inf H f = o 1 τ (t) as t → +∞. (5)
Taking τ (•) which grows faster than the identity, makes the solution trajectories unchanged but travelled faster. The price to pay is that (4) is a non-autonomous dynamic in which the coefficient in front of the gradient term tends to infinity as t → +∞. This prevents from using gradient methods to discretize it.

Recall that for gradient methods the step size has to be less than or equal to twice the inverse of the Lipschitz constant of the gradient. To overcome this difficulty we come with the second step of our method which is averaging. Let us attach to y(•) the new function x : [t 0 , +∞[→ H defined by ẋ(t) + 1 τ (t) (x(t) -y(t)) = 0, [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF] with x(t 0 ) = x 0 given in H. We shall explain further the averaging interpretation. Equivalently y(t) = x(t) + τ (t) ẋ(t). [START_REF] Attouch | Convergence of a relaxed inertial proximal algorithm for maximally monotone operators[END_REF] By temporal derivation of [START_REF] Attouch | Convergence of a relaxed inertial proximal algorithm for maximally monotone operators[END_REF] we get ẏ(t) = ẋ(t) + τ (t) ẋ(t) + τ (t)ẍ(t).

Replacing y(t) and ẏ(t) as given by ( 7) and ( 8) in (4), we get ẍ(t) + 1 + τ (t) τ (t) ẋ(t) + ∇f x(t) + τ (t) ẋ(t) = 0.

In doing so, we passed from the first-order differential equation ( 4) to the second-order differential equation [START_REF] Attouch | Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven damping[END_REF], with the advantage that now the coefficient in front of the gradient is fixed. Let us now particularize the time scale τ (•). Taking

τ (t) = t 2 2(α -1) , (10)
gives 1+τ (t) τ (t) = α t , and the corresponding dynamic with implicit Hessian driven damping ẍ(t) + α t ẋ(t) + ∇f x(t) + t α -1 ẋ(t) = 0. [START_REF] Attouch | Fast convex minimization via inertial dynamics with Hessian driven damping[END_REF] In this dynamic, the Hessian driven damping appears in an implicit form. This type of dynamic was initiated in [START_REF] Alecsa | An extension of the second order dynamical system that models Nesterov's convex gradient method[END_REF], see also [START_REF] Muehlebach | A Dynamical Systems Perspective on Nesterov Acceleration[END_REF] for a related autonomous system in the case of a strongly convex function f . The rationale justifying the use of the term "implicit" comes from the observation that by a Taylor expansion (as t → +∞ we have t ẋ(t) → 0 which justifies the use of Taylor expansion), we have

∇f x(t) + t α -1 ẋ(t) ≈ ∇f (x(t)) + t α -1 ∇ 2 f (x(t)) ẋ(t),
thus making the Hessian damping appear indirectly in [START_REF] Attouch | Fast convex minimization via inertial dynamics with Hessian driven damping[END_REF]. Because of its important role in attenuating the oscillations, several recent studies have been devoted to inertial dynamics combining the asymptotic vanishing damping with the geometric Hessian-driven damping (coined sometimes Newton-type inertial dynamics); see e.g., [START_REF] Attouch | Fast convex optimization via inertial dynamics combining viscous and Hessian-driven damping with time rescaling[END_REF][START_REF] Attouch | Fast convex minimization via inertial dynamics with Hessian driven damping[END_REF][START_REF] Attouch | First-order algorithms via inertial systems with Hessian driven damping[END_REF][START_REF] Attouch | Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven damping[END_REF][START_REF] Attouch | On the effect of perturbations, errors in first-order optimization methods with inertia and Hessian driven damping[END_REF][START_REF] Bot | Tikhonov regularization of a second order dynamical system with Hessian damping[END_REF][START_REF] Castera | An Inertial Newton Algorithm for Deep Learning[END_REF][START_REF] Maingé | First-Order Frameworks for Continuous Newton-like Dynamics Governed by Maximally Monotone Operators[END_REF][START_REF] Shi | Understanding the acceleration phenomenon via high-resolution differential equations[END_REF]. In turn, the corresponding algorithms, among which IGAHD enjoys several favorable properties, introduce a correction term in the Nesterov accelerated gradient method (see [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate O(1/k2)[END_REF][START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF]) which reduces the oscillatory aspects. Note that in [START_REF] Attouch | Fast convex minimization via inertial dynamics with Hessian driven damping[END_REF] the coefficient of the Hessian damping is proportional to the inverse of the viscosity damping. Thus asymptotically when the viscous damping tends towards zero, and therefore can cause many small oscillations to appear, the coefficient of the Hessian driven damping tends towards infinity, and therefore has an effective effect on the attenuation of the oscillations. This is the situation considered by Attouch-Bot ¸-Nguyen [START_REF] Attouch | Fast convex optimization via time scale and averaging of the steepest descent[END_REF], who obtained convergence rates comparable to those associated with the Nesterov accelerated gradient method. A major advantage of this approach is that there is no need to do a Lyapunov analysis, we only use the classical convergence rate for the continuous steepest descent. Moreover, the convergence of the trajectories is a direct consequence of the known results for the steepest descent.

Closed-loop control

The idea is to exploit the time scaling and averaging method and the fact that (SD) provides several quantities which are increasing and converge to +∞ as t → +∞, so which are eligible for time scaling. This will enable us to perform time scaling and averaging in a closed-loop way. Indeed, in (SD), the velocity and the norm of the gradient are monotonically decreasing to zero. So, the idea is to use their inverse for defining the time scaling. Specifically, in a first result we are going to define the derivative of the time scaling τ (•) as a function of the inverse of the speed. This means acceleration of the time scaling when the speed decreases. Following this approach, we will obtain in Theorem 5 the following model result.

Theorem 1 Suppose that f : H → R satisfies (A). Let us choose the positive parameters according to q > 0, p ≥ 1, and γ > 1. Let x : [t 0 , +∞[ → H be a solution trajectory of the following system

                   ẍ(t) + (1 + γ) τ (t) 2 -τ (t)τ (t) τ (t) τ (t) ẋ(t) + γ τ (t) 2 τ (t) ∇f x(t) + 1 γ τ (t) τ (t) ẋ(t) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p | τ (t)| p-1 ∇f x (t) + 1 γ τ (t) τ (t) ẋ (t) p-1 = 1. ( 12
)
Then we have the fast convergence of values:

as t → +∞ f (x(t)) -inf H f = o 1 t 1+q-1 p . ( 13
)
Moreover, the solution trajectory x(t) converges weakly as t → +∞, and its limit belongs to S = argmin f .

As a special case, take p = 1, q = 2. Then, the last equation of [START_REF] Attouch | Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces[END_REF] gives λ(t) ≡ 1. According to this, the second equation of [START_REF] Attouch | Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces[END_REF] gives τ (t) = t 2 4 , and we find a case with time scaling in an open-loop form. After elementary calculation, the first equation of ( 12) is written as

ẍ(t) + α t ẋ(t) + α -1 2 ∇f x(t) + t α -1 ẋ(t) = 0,
with α = 2γ + 1 > 3, and the convergence rate of the values becomes

f (x(t)) -inf H f = o 1 t 2 . ( 14
)
We therefore recover the results obtained by the authors in the case of the open loop, giving the optimal convergence rates for general convex differentiable optimization. This inertial formulation may seem at first glance complicated. Indeed it is equivalent to the first-order system in time and space

                   ẏ (t) + τ (t) ∇f (y (t)) = 0 ẋ(t) + γ τ (t) τ (t) x(t) -γ τ (t) τ (t) y(t) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p ẏ (t) p-1 = 1, (15)
whose temporal discretization provides corresponding optimization algorithms, see Theorem 11.

Link with the existing literature

Contrary to the rich literature that has been devoted to non-autonomous damped inertial methods and their links with the fast first-order optimization algorithms for general convex optimization (in particular the Nesterov accelerated gradient method), only a small number of papers have been devoted to these questions, based on autonomous methods. Indeed the heavy ball method of Polyak only provides the asymptotic convergence rate 1/t for general convex functions. The idea is therefore to see if we can mimic the fast convergence properties of the Su, Boyd, and Candès dynamic model (see [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF]) of the Nesterov accelerated gradient method, using autonomous dynamics. A natural idea is to design the damping term, on which is based the optimization properties of the system, in a closed-loop way. In this direction, we can mention the following contributions.

a) Our study has a natural link with works devoted to regularized Newton methods for solving monotone inclusions (and (1) in particular). Given a general maximally monotone operator A : H ⇒ H, to overcome the ill-posed character of the continuous Newton method, in line with [START_REF] Attouch | A continuous dynamical Newton-Like approach to solving monotone inclusions[END_REF], Attouch, Redont and Svaiter have studied in [START_REF] Attouch | Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces[END_REF] the following closed-loop dynamic version of the Levenberg-Marquardt method

v(t) ∈ A(x(t)) v(t) γ ẋ(t) + β v(t) + v(t) = 0.
When γ > 1, they showed the well-posedness of the above system, and analyzed its convergence properties. When A = ∇f this system writes

∇f (x(t)) γ ẋ(t) + β∇ 2 f (x(t)) ẋ(t) + ∇f (x(t)) = 0. Thus, its inertial version ẍ(t) + ∇f (x(t)) γ ẋ(t) + β∇ 2 f (x(t)) ẋ(t) + ∇f (x(t)) = 0
falls within the framework of our study with the damping equal to a closed-loop control of the norm of the gradient. The techniques developed in [START_REF] Attouch | Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces[END_REF] are particularly useful for studying the well-posedness of dynamics with implicit features. b) Although significantly different, our approach has several points in common with the article by Lin and Jordan [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF]. In this article, the authors study the closed-loop dynamical system

               ẏ (t) + τ (t) ∇f (x (t)) = 0 ẋ (t) + τ (t) τ (t) (x (t) -y (t)) + [ τ (t)] 2 τ (t) ∇f (x (t)) = 0 τ (t) - 1 4 t 0 λ (r)dr + c 2 = 0 [λ (t)] p ∇f (x (t)) p-1 = θ, (16)
where c > 0 and 0 < θ < 1. The corresponding second-order in time damped inertial system writes as follows

ẍ(t) + 2 [ τ (t)] 2 -τ (t)τ (t) τ (t) τ (t) ẋ(t) + [ τ (t)] 2 τ (t) ∇ 2 f (x(t)) ẋ(t) + τ (t)( τ (t) + τ (t)) τ (t) ∇f (x(t)) = 0. ( 17
)
In the above system, the Hessian driven damping comes in an explicit way because of the structure of the first equation which differs from the structure of the continuous steepest descent. In contrast, in our approach, the first equation is the rescaled continuous steepest descent, and the Hessian driven damping comes implicitly. Let us highlight some advantages of our approach.

• Our system is introduced in a natural way by using the time scaling and averaging method. This makes unnecessary to perform a Lyapunov analysis for the inertial system. It has already been done for the continuous steepest descent. This results in a significantly simplified mathematical analysis.

• Our dynamic model contains an additional parameter q which, when q = 2, gives the setting of Lin and Jordan, and which, when judiciously tuned, gives better convergence rates.

• Our approach provides the weak convergence of the trajectories to optimal solutions. We shall return later to the precise comparison between the two systems. c) In [START_REF] Attouch | Fast optimization via inertial dynamics with closed-loop damping[END_REF], Attouch, Bot ¸and Csetnek study the convergence properties of the Autonomous Damped Inertial Gradient Equation

(ADIGE) ẍ(t) + G ẋ(t), ∇f (x(t)), ∇ 2 f (x(t)) + ∇f (x(t)) = 0,
where the damping term G ẋ(t), ∇f (x(t)), ∇ 2 f (x(t)) acts as a closed-loop control. They pay particular attention to the role played by the parameter r > 1 in the asymptotic convergence analysis of the dynamic

ẍ(t) + ẋ(t) r-2 ẋ(t) + ∇f (x(t)) = 0.
They show that the case r = 2 separates the weak damping (r > 2) from the strong damping (r < 2), hence the importance of this case. These questions have also been considered by Haraux and Jendoubi in [START_REF] Haraux | Jendoubi The convergence problem for dissipative autonomous systems[END_REF]. d) In [START_REF] Song | Unified Acceleration of High-Order Algorithms under General Hölder Continuity[END_REF], Song, Jiang, and Ma develop an interesting technique for accelerating high-order algorithms under general Hölder continuity assumption. Their continuous-time framework reduces to an inertial system without Hessian-driven damping in the first-order setting, which has been proven to be an inaccurate surrogate. Although underlying their approach, the acceleration via time scaling, the averaging technique, and the closed-loop tuning of the coefficients are not clearly identified.

Organization of the paper

After a general presentation of the article in the introduction, we provide in Section 2 a general estimate of the time scaling for the continuous steepest descent when it is defined in a closed-loop way. This is crucial for the rest of the paper. Then we specialize these results to situations of particular interest, and examine in details the case of closed-loop systems induced respectively by velocities, and then by gradients. In Section 3, which is the main part of the paper, we develop the next important step in our approach, which is the averaging operation. This provides accelerated damped inertial dynamics that are autonomous and with fast convergence properties. Finally, in Section 4 we analyze the fast convergence properties of proximal algorithms which come naturally from the temporal discretization of the continuous dynamics.

2 Closed-loop time scaling of the steepest descent 2.1 Formulation of the closed-loop time scaling Given t 0 ≥ 0, q > 0, and p ≥ 1, the time scale function τ : [t 0 , +∞[ → R ++ is defined by

           ẏ (t) + τ (t) ∇f (y (t)) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p [G (y (t))] p-1 = 1, (18)
where G(•) is a given positive, continuous function that depends on the information of the trajectory y(•). This general formalism allows us to unify the various situations coming from different choices of the time scaling as a feedback control of the state of the system. For example G may be a function of y, ẏ, f (y) , ∇f (y) and/or any mixture combination of them. Then the function λ(•) is continuous and it links the coefficient of ∇f , namely τ (•), with the solution trajectory y(•).

As a useful result, note that for every t ≥ t 0 , it holds τ (t) = 1

q q-1 t t0 [λ (r)] 1 q dr + t 0 q-1 [λ (t)] 1 q = [τ (t)] q-1 q [λ (t)] 1 q > 0. ( 19
) Moreover, the relations [START_REF] Iutzeler | A generic online acceleration scheme for optimization algorithms via relaxation and inertia[END_REF] allow us to cover the open-loop case. In particular, when p = 1 it holds λ (t) = 1 for every t ≥ t 0 . This yields for every q > 0

τ (t) = t q q .
Taking further q := 1, then τ (t) becomes the regular time in variable t, namely τ (t) = t for every t ≥ t 0 . Let us specify the interpretation of (18) as a steepest descent dynamic which is rescaled in time in a closed-loop way.

Proposition 1 Suppose that f : H → R satisfies (A). Let t 0 ≥ 0, q > 0, p ≥ 1 and y : [t 0 , +∞[ → H be a solution trajectory of the system [START_REF] Iutzeler | A generic online acceleration scheme for optimization algorithms via relaxation and inertia[END_REF]. Suppose that lim s→+∞ τ (s) = +∞.

Then y(•) is a solution trajectory of a time rescaled continuous steepest descent (SD), as described below: Let s 0 = τ (t 0 ) and z : [s 0 , +∞[ → H be a solution trajectory of the following system ż (s)

+ ∇f (z (s)) = 0. ( 20
)
Then we have

y (t) = z (τ (t)) ∀t ≥ t 0 ,
and there exists a continuously differentiable function

σ : [s 0 , +∞[ → R ++ such that z (s) = y (σ (s)) ∀s ≥ s 0 .
Proof We already interpreted how to go from a solution trajectory z(•) of (SD) to the closed-loop system above via the time scaling function τ (•). Let us now show the reverse direction. Let y : [t 0 , +∞[ → H be a solution trajectory of [START_REF] Iutzeler | A generic online acceleration scheme for optimization algorithms via relaxation and inertia[END_REF]. We have that λ is continuous and positive on [t 0 , +∞[, therefore τ is a monotonically increasing function, hence injective. On the other hand, we have t 0 = τ (t 0 ) = t0 q q . Since by assumption lim t→+∞ τ (t) = +∞, this means τ is a continuous function whose image contains [s 0 , +∞[, hence surjective. Combining these premises, we have shown that τ is a bijection, which means it is invertible. Set σ ≡ τ -1 and make the change of time variable t := σ (s) in [START_REF] Iutzeler | A generic online acceleration scheme for optimization algorithms via relaxation and inertia[END_REF]. Let us define

z (s) = y (σ (s)) = y τ -1 (s) .
Then by the chain rule, we have

ż (s) = ẏ τ -1 (s) 1 τ (τ -1 (s)) = ẏ (σ (s)) 1 τ (σ (s)) . This leads to ż (s) + ∇f (z (s)) = 0.
In other words, z : [s 0 , +∞[ → H is a solution trajectory of (SD).

The above assertion allows us to transfer the convergence results of (SD) to some closed-loop systems. In particular, given a time scaling function τ (•) as described above, by making the change of time variable s := τ (t), we obtain the following results from Theorem 12 in the appendix applied to the unperturbed continuous steepest descent system.

+∞ t0 τ (t) τ (t) ẏ (t) 2 dt < +∞, ( 21
) +∞ t0 τ (t) τ (t) ∇f (y (t)) 2 dt < +∞, (22)
f (y(t)) -inf H f = o 1 τ (t) as t → +∞, ( 23
) ∇f (y (t)) = o 1 τ (t) as t → +∞. ( 24
)
2.2 Lower bound estimate of the time scaling τ (t)

As key ingredient of our approach, the next step is to establish a lower bound for τ (t) in terms of t. This will reflect the acceleration of our dynamic via time scaling and allow us to achieve fast convergence rates.

For this, we will need the following technical lemma, which can be seen as a nonlinear Gronwall result.

Lemma 1 Suppose that there exists

C 0 > 0 and b > a ≥ 0 such that t t0 [τ (r)] a [λ (r)] -b dr ≤ C 0 < +∞ ∀t ≥ t 0 . ( 25
)
Then there exists

C 1 > 0 such that τ (t) ≥ C 1 (t -t 0 ) qb+1 b-a ∀t ≥ t 0 . ( 26
)
Proof Let t ≥ t 0 be fixed. By applying the Hölder inequality, we get

t t0 [τ (r)] a qb+1 dr ≤ t t0 [τ (r)] a [λ (r)] -b dr 1 qb+1 t t0 [λ (r)] 1 q dr qb qb+1 ≤ C 1 qb+1 0 t 0 + t t0 [λ (r)] 1 q dr qb qb+1 = C 0 q qb 1 qb+1 [τ (t)] b qb+1 . ( 27
)
If a = 0 then (26) follows immediately. From now on suppose that a > 0, so that the inequality ( 27) can be rewritten as

t t0 [τ (r)] a qb+1 dr ≤ C 0 q qb 1 qb+1 [τ (t)] a qb+1
b a [START_REF] Shi | Understanding the acceleration phenomenon via high-resolution differential equations[END_REF] The arguments are now adapted from [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF], which is inspired by the proof of Bihari-LaSalle inequality. Let

C q,b := C 0 q qb 1 qb+1 > 0 and A (t) := t t0 [τ (r)] a qb+1 dr ∀t ≥ t 0 ,
so that (28) becomes

A (t) ≤ C q,b Ȧ (t) b a ∀t ≥ t 0 or, equivalently, C -a b q,b ≤ [A (t)] -a b Ȧ (t) ∀t ≥ t 0 .
Integrating from t 0 to t, we obtain

C -a b q,b (t -t 0 ) ≤ 1 - a b [A (t)] 1-a b -[A (t 0 )] 1-1 b ≤ [A (t)] 1-a b ≤ C q,b [τ (t)] b qb+1 1-a b = C b-a b q,b [τ (t)] b-a qb+1 ,
where the last inequality comes from [START_REF] Scieur | Regularized nonlinear acceleration[END_REF]. Since b > a, the conclusion follows.

Let us now particularize our results to some model situations.

Closed-loop control of (SD) via the velocity

Theorem 2 Suppose that f : H → R satisfies (A). Let q > 0, p ≥ 1 and y : [t 0 , +∞[ → H be a solution trajectory of the following system

           ẏ (t) + τ (t) ∇f (y (t)) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p ẏ (t) p-1 = 1. ( 29
)
Then the following statements are satisfied:

(i) (convergence of values) f (y (t)) -inf H f = o t -(1+q-1 p ) as t → +∞. (ii) (convergence of gradients towards zero) ∇f (y (t)) = o t -(1+q-1 p ) as t → +∞.
(iii) (integral estimate of the velocities)

+∞ t0 t (1+ 1 q -1 pq ) ẏ (t) 2+ p-1 pq dt < +∞.
(iv) The solution trajectory y(t) converges weakly as t → +∞, and its limit belongs to S = argmin f .

Proof When p = 1, we recover the open loop case with the time scaling function τ (t) = t q q

. The result is a direct consequence of Theorem 12. Therefore, from now on we only consider the case p > 1. Recall that from [START_REF] Maingé | First-Order Frameworks for Continuous Newton-like Dynamics Governed by Maximally Monotone Operators[END_REF] we have

+∞ t0 τ (t) τ (t) ẏ (t) 2 dt < +∞. ( 30
)
By using successively the definition of λ, and relation [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF], we obtain

τ (t) τ (t) ẏ (t) 2 = τ (t) τ (t) [λ (t)] -2p p-1 = [τ (t)] 1 q [λ (t)] -1 q -2p p-1 ∀t ≥ t 0 .
According to the two above results we get

+∞ t0 [τ (r)] 1 q [λ (r)] -1 q -2p
p-1 dr < +∞.

We are now in position to apply Lemma 1 with p > 1, a := 1 q and b := 1 q + 2p p-1 . We have

qb + 1 b -a = 2 + 2pq p-1 2p p-1 = p -1 + pq p = 1 + q - 1 p ,
and therefore there exists some constant C 1 > 0 such that

τ (t) ≥ C 1 (t -t 0 ) 1+q-1 p ∀t ≥ t 0 . (31)
This leads to lim t→+∞ τ (t) = +∞. Therefore, according to Proposition 1, we can extract the results from Theorem 12 and the corresponding formulas ( 21), ( 22), [START_REF] Monteiro | Svaiter An Accelerated Hybrid Proximal Extragradient Method for Convex Optimization and Its Implications to Second-order Methods[END_REF]. Specifically, we obtain (i) for the values

f (y (t)) -inf H f = o 1 τ (t) = o 1 t 1+q-1 p ,
(ii) for the gradients

∇f (y (t)) = o 1 τ (t) = o 1 t 1+q-1 p .
(iii) for the velocities: we start from (30), i.e. +∞ t0 τ (t) τ (t) ẏ (t) 2 dt < +∞, that we evaluate as follows:

τ (t) τ (t) ẏ (t) 2 = τ (t) τ (t) q-1 q λ(t) 1 q ẏ (t) 2 = τ (t) 1 q λ(t) 1 q ẏ (t) 2 = τ (t) 1 q ẏ (t) 2+ p-1 pq ∀t ≥ t 0 .
According to (31) we deduce that

+∞ t0 t (1+ 1 q -1 pq ) ẏ (t) 2+ p-1 pq dt < +∞.
(iv) Let us finally examine the convergence of the solution trajectories. We know that the solution trajectory of the continuous steepest descent converges weakly when t → +∞, and its limit belong to S = argmin H f = ∅; see Theorem 12 in appendix. With our notation we therefore have that z(s) converges weakly when s → +∞. Since τ (t) → +∞ as t → +∞, we immediately deduce that y(t) = z(τ (t) converges weakly as t → +∞, and its limit belong to S = argmin H f = ∅. This completes the proof.

Closed-loop control of (SD) via the norm of gradient

We develop an analysis parallel to that of the previous section, replacing speed control with gradient control.

Theorem 3 Suppose that f : H → R satisfies (A). Let q ≥ 1 2 , p ≥ 1 and y : [t 0 , +∞[ → H be a solution trajectory of the following system

           ẏ (t) + τ (t) ∇f (y (t)) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p ∇f (y (t)) p-1 = 1. ( 32
)
Then the following statements are satisfied:

(i) (convergence of values) f (y (t)) -inf H f = o t -pq as t → +∞.
(ii) (convergence of gradients towards zero) ∇f (y (t)) = o t -pq as t → +∞.

(iii) (integral estimate of the gradients)

+∞ t0 t pq(2-1 q ) ∇f (y (t)) 2+ p-1 pq dt < +∞.
(iv) The solution trajectory y(t) converges weakly as t → +∞, and its limit belongs to S = argmin f .

Proof Again, we only consider the case p > 1. We know from [START_REF] Muehlebach | A Dynamical Systems Perspective on Nesterov Acceleration[END_REF] that

+∞ t0 τ (t) τ (t) ∇f (y (t)) 2 dt < +∞.
By using successively the definition of λ, and the relation [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF], we obtain

τ (t) τ (t) ∇f (y (t)) 2 = τ (t) τ (t) [λ (t)] -2p p-1 = [τ (t)] 2-1 q [λ (t)] 1 q -2p p-1 ∀t ≥ t 0 . Therefore +∞ t0 [τ (t)] 2-1 q [λ (t)] 1 q -2p p-1 dt < +∞.
Let us apply Lemma 1 with a := 2 -1 q and b = 2p p-1 -1 q . We have b > a, a ≥ 0 for q ≥ 1 2 , and

qb + 1 b -a = 2pq p-1 2p p-1 -2 = pq. Therefore τ (t) ≥ C 1 (t -t 0 ) pq ∀t ≥ t 0 . (33)
This gives lim t→+∞ τ (t) = +∞. According to Proposition 1, we can extract the results from Theorem 12 and the corresponding formulas ( 21), ( 22), [START_REF] Monteiro | Svaiter An Accelerated Hybrid Proximal Extragradient Method for Convex Optimization and Its Implications to Second-order Methods[END_REF]. Specifically, we obtain (i) for the values

f (y (t)) -inf H f = o 1 τ (t) = o 1 t pq ; (ii) for the gradients ∇f (y (t)) = o 1 τ (t) = o t -pq ;
(iii) for the integral estimate of the gradients: we start from ( 30)

+∞ t0 τ (t) τ (t) ∇f (y (t)) 2 dt < +∞,
that we evaluate as follows:

τ (t) τ (t) ∇f (y (t)) 2 = τ (t) [τ (t)] q-1 q [λ (t)] 1 q ∇f (y (t)) 2 = τ (t) 2-1 q λ(t) 1 q ∇f (y (t)) 2 = τ (t) 2-1 q ∇f (y(t)) 2-p-1 pq ∀t ≥ t 0 .
According to (33) we deduce that

+∞ t0 t pq(2-1 q ) ∇f (y (t)) 2+ p-1 pq dt < +∞.
(iv) The convergence of the solution trajectory follows from an argument similar to that of the previous section. This completes the proof.

Remark 1 a) We thus achieved our first goal which was to accelerate the convergence properties of the continuous steepest descent using closed-loop time scaling. For example, concerning the convergence rate of the values, we passed from the convergence rate 1/t for the steepest descent to 1/t (1+q-1 p ) when the closed-loop control acts on the velocity, and 1/t pq in the case of the gradient. Clearly, by playing with the parameters p and q we can get arbitrary fast convergence results. The same observation holds for the convergence of the gradients towards zero. b) By introducing a time scale function τ (•) which grows faster than the identity (i.e. τ (t) ≥ t) either in open-loop or closed-loop, we have thus accelerated the continuous steepest descent dynamic. The price to pay is that we no longer have an autonomous dynamic in (4), with as major drawback the fact that the coefficient in front of the gradient term tends towards infinity as t → +∞. This prevents from using gradient methods to discretize it. Recall that for gradient methods, the step size has to be less than or equal to twice the inverse of the Lipschitz constant of the gradient. To overcome this, we come with the second step of our method which is averaging.

3 Accelerated gradient systems with closed-loop control of the damping

General results concerning time scale and averaging

We will prove the following general result which puts forward a damped inertial dynamics which comes by time scale and averaging of the continuous steepest descent. Then we will specialize it and consider time scale obtained in a closed-loop way, and thus cover the two model situations.

Theorem 4 Suppose that f : H → R satisfies (A). Let γ > 1, and let τ : [t 0 , +∞[→ R ++ be an increasing function, continuously differentiable, such that lim t→+∞ τ (t) = +∞. Let x : [t 0 , +∞[ → H be a solution trajectory of the following second-order differential equation

ẍ(t) + (1 + γ) τ (t) 2 -τ (t)τ (t) τ (t) τ (t) ẋ(t) + γ τ (t) 2 τ (t) ∇f x(t) + 1 γ τ (t) τ (t) ẋ(t) = 0. ( 34
)
Then we have the convergence rate of the values: as t → +∞

f (x(t)) -inf H f = o 1 τ (t) , ( 35
)
and x(t) converges weakly as t → +∞, and its limit belongs to S = argmin f . Proof a) We first interpret x as coming from the time scale and averaging of the continuous steepest descent. We start from y(•) solution of ẏ (t) + τ (t) ∇f (y (t)) = 0.

(36)

According to the time scale analysis developed in (5) we have

f (y (t)) -inf H f = o 1 τ (t) as t → +∞.
This means there exists a positive function ε which satisfies lim t→+∞ ε (t) = 0 and

f (y (t)) -inf H f = ε (t) τ (t) ∀t ≥ t 0 . ( 37
)
Let us define the time averaging process as the transformation from y to x according to the formula

ẋ(t) + γ τ (t) τ (t) x(t) = γ τ (t) τ (t) y(t), (38)
where γ > 1. Equivalently

y(t) = x(t) + 1 γ τ (t) τ (t) ẋ(t). ( 39
)
By derivating y(•) we get

ẏ (t) = ẋ (t) + 1 γ τ (t) τ (t) ẍ(t) + 1 γ τ (t) 2 -τ (t)τ (t) τ (t) 2 ẋ(t). ( 40
)
Replacing ẏ (t) by this expression in the constitutive rescaled steepest descent equation (36), we get

ẋ (t) + 1 γ τ (t) τ (t) ẍ(t) + 1 γ τ (t) 2 -τ (t)τ (t) τ (t) 2 ẋ(t) + τ (t) ∇f x(t) + 1 γ τ (t) τ (t) ẋ(t) = 0. Equivalently 1 γ τ (t) τ (t) ẍ(t) + (1 + γ) τ (t) 2 -τ (t)τ (t) γ τ (t) 2 ẋ(t) + τ (t) ∇f x(t) + 1 γ τ (t) τ (t) ẋ(t) = 0.
After multiplication by γ τ (t) τ (t) we get

ẍ(t) + (1 + γ) τ (t) 2 -τ (t)τ (t) τ (t) τ (t) ẋ(t) + γ τ (t) 2 τ (t) ∇f x(t) + 1 γ τ (t) τ (t) ẋ(t) = 0. (41)
b) Let us now come to the corresponding estimate of the convergence rates with x(t) instead of y(t). The idea is to express x as an average of y, and then conclude thanks to Jensen's inequality. Set

b(t) = τ (t) τ (t) ≥ 0 (42) B(t) = t t0 b(u)du = t t0 τ (u) τ (u) du = ln τ (t) τ (t 0 ) . ( 43
)
Therefore

e B(t) = τ (t) τ (t 0 ) . ( 44
)
In order to express x in terms of y, we need to integrate the first-order linear differential equation (38) which is written equivalently as follows

ẋ(t) + γb(t)x(t) = γb(t)y(t).
After multiplying by e γB(t) , we get equivalently

e γB(t) ẋ(t) + γb(t)e γB(t) x(t) = γb(t)e γB(t) y(t),
that is, d dt e γB(t) x(t) = γb(t)e γB(t) y(t).

After integration we get e γB(t) x(t) = e γB(t0) x(t 0 ) + γ t t0 b(u)e γB(u) y(u)du.

According to e γB(t0) = e 0 = 1 we get

x(t) = e -γB(t) x(t 0 ) + γe -γB(t) t t0 b(u)e γB(u) y(u))du = e -γB(t) y(t 0 ) + γe -γB(t) t t0 b(u)e γB(u) y(u)du, (45)
where the last equality follows from the choice of the Cauchy data y(t 0 ) = x(t 0 ). Then, observe that x(t) can be simply written as follows

x(t) = t t0 y(u) dµ t (u), ( 46
)
where µ t is the positive Radon measure on [t 0 , t] defined by

µ t = e -γB(t) δ t0 + γb(u)e γ(B(u)-B(t)) du. (47)
Precisely, in (47), δ t0 is the Dirac measure at t 0 , and b(u)e B(u)-B(t) du is the measure with density b(u)e B(u)-B(t) with respect to the Lebesgue measure on [t 0 , t]. According to

γe -γB(t) t t0 b(u)e γB(u) du = 1 -e -γB(t) ,
we have that µ t is a positive Radon measure on [t 0 , t] whose total mass is equal to 1. It is therefore a probability measure, and x(t) is obtained by averaging the trajectory y(•) on [t 0 , t] with respect to µ t .

From there, let us show how to deduce fast convergence properties for the so defined trajectory x(•).

According to the convexity of f , and Jensen's inequality, we deduce that

f t t0 y(u) dµ t (u) -inf H f = (f -inf H f ) t t0 y(u)dµ t (u) ≤ t t0 f (y(u)) -inf H f dµ t (u) = t t0 ε (u) τ (u) dµ t (u),
where the last inequality above comes from (37). According to the definition of µ t (see (47)) and the formulation of x(t) (see ( 46)), we deduce that

f (x(t)) -inf H f ≤ ε (t 0 ) τ (t 0 ) e -γB(t) + γe -γB(t) t t0 ε (u) τ (u) b(u)e γB(u) du.
Equivalently,

τ (t) f (x(t)) -inf H f ≤ ε (t 0 ) τ (t) τ (t 0 ) 1-γ + γτ (t) e -γB(t) t t0 ε (u) τ (u) b(u)e γB(u) du. ( 48
)
Since γ > 1 and lim t→+∞ τ (t) = +∞, it holds

lim sup t→+∞ τ (t) f (x(t)) -inf H f ≤ γ lim sup t→+∞ τ (t) e -γB(t) t t0 ε (u) τ (u) b(u)e γB(u) du.
It is therefore enough to show that lim sup t→+∞ γτ (t) e -γB(t)

t t0 ε (u) τ (u) b(u)e γB(u) du ≤ 0.
In order to prepare for integration by parts, note that

γb(u)e γB(u) = d du e γB(u) and τ (u) [τ (u)] 2-γ = d du 1 γ -1 1 [τ (t)] 1-γ .
Given an arbitrary η > 0 we consider Tη > t 0 such that ε (u) ≤ η for every u ≥ Tη. Therefore, for every t ≥ Tη, by integration by parts and by taking into consideration the relations (42)-(44), we get γτ (t) e -γB(t)

t t0 ε (u) τ (u) b(u)e γB(u) du = γτ (t) e -γB(t) Tη t0 ε (u) τ (u) b(u)e γB(u) du + t Tη ε (u) τ (u) b(u)e γB(u) du ≤ τ (t) e -γB(t) γ Tη t0 ε (u) τ (u) b(u)e γB(u) du + ηγ t Tη 1 τ (u) b(u)e γB(u) du = τ (t) e -γB(t) γ Tη t0 ε (u) τ (u) b(u)e γB(u) du + η τ (t) e γB(t) - η τ (Tη) e γB(Tη) + η t Tη τ (u) [τ (u)] 2 e γB(u) du = τ (t) e -γB(t) γ Tη t0 ε (u) τ (u) b(u)e γB(u) du + η τ (t) e γB(t) - η τ (Tη) e γB(Tη) + η [τ (t 0 )] γ t Tη τ (u) [τ (u)] 2-γ du = τ (t) e -γB(t) γ Tη t0 ε (u) τ (u) b(u)e γB(u) du + η τ (t) e γB(t) - η τ (Tη) e γB(Tη) + η [τ (t 0 )] -γ γ -1 1 [τ (t)] 1-γ - 1 [τ (Tη)] 1-γ ≤ γ Tη t0 ε (u) τ (u) b(u)e γB(u) du τ (t) e -γB(t) + η + η γ -1 ≤ C [τ (t)] 1-γ + ηγ γ -1 .
Since lim t→+∞ τ (t) = +∞, and γ > 1, we obtain lim sup t→+∞ γτ (t) e -γB(t)

t t0 ε (u) τ (u) b(u)e γB(u) du ≤ ηγ γ -1 . ( 49
)
This being true for every η > 0, we infer

f (x(t)) -inf H f = o 1 τ (t) .
(50) c) For trajectories convergence, we take advantage of the fact that the solution trajectory z (•) of the continuous steepest descent converges weakly towards a solution x * ∈ S. Since lim t→+∞ τ (t) = +∞, this immediately implies that y(t) = z(τ (t)) converges weakly to x * as s → +∞. In other words, for each v ∈ H y (t) , v → x * , v as t → +∞.

To pass from the convergence of y to that of x, we use the interpretation of x as an average of y. The convergence then results from the general property which says that convergence entails ergodic convergence. Let us make this precise. Using again that lim t→+∞ τ (t) = +∞, we have

x(t) ∼ γe -γB(t) t t0 b (u) e γB(u) y (u) du = γ [τ (t)] γ t t0 τ (u) [τ (u)] γ-1 y(u)du.
After elementary calculus, we just need to prove that if a(•) is a positive real-valued function which verifies lim u→+∞ a(u) = 0, then lim t→+∞ A(t) = 0, where

A(t) = γ [τ (t)] γ t t0 τ (u) [τ (u)] γ-1 a(u)du.
Given an arbitrary η > 0, let us take Tη such that t 0 < Tη and a(u) ≤ η for u ≥ Tη. For t > Tη, we have

A(t) = γ [τ (t)] γ Tη t0 τ (u) [τ (u)] γ-1 a(u)du + γ [τ (t)] γ t Tη τ (u) [τ (u)] γ-1 a(u)du ≤ γ [τ (t)] γ Tη t0 τ (u) [τ (u)] γ-1 a(u)du + ητ (t 0 ).
Letting t converge to +∞ we get lim sup t→+∞ A(t) ≤ ητ (t 0 ).

This being true for any η > 0, we infer that lim t→+∞ A(t) = 0, which completes the proof.

Remark 2 By taking γ := α-1 2 and τ (t) := t 2 2(α-1) , equation ( 41) becomes (see [START_REF] Attouch | Fast convex optimization via time scale and averaging of the steepest descent[END_REF])

ẍ(t) + α t ẋ(t) + ∇f x(t) + t α -1 ẋ(t) = 0.
We have γ > 1 ⇐⇒ α > 3, which is in accordance with the convergence results attached to Nesterov method.

Damped inertial system via closed-loop control of the velocity

Let us now examine the model situation where the time scaling is defined in a closed-loop way as a feedback control of the velocity. Completing this construction with the averaging process, as described as above, we get that (x, y) : [t 0 , +∞[ → H × H is a solution trajectory of the following algebraic-differential system

                       ẏ (t) + τ (t) ∇f (y (t)) = 0 ẋ(t) + γ τ (t) τ (t) x(t) -γ τ (t) τ (t) y(t) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p ẏ (t) p-1 = 1. (51)
By specializing Theorem 4 to this situation we get the following result.

Theorem 5 Suppose that f : H → R satisfies (A). Let q > 0, p ≥ 1, γ > 1 and x : [t 0 , +∞[ → H be a solution trajectory of the following system

                   ẍ(t) + (1 + γ) τ (t) 2 -τ (t)τ (t) τ (t) τ (t) ẋ(t) + γ τ (t) 2 τ (t) ∇f x(t) + 1 γ τ (t) τ (t) ẋ(t) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p | τ (t)| p-1 ∇f x (t) + 1 γ τ (t) τ (t) ẋ (t) p-1 = 1. ( 52
)
Then we have the fast convergence of values: as t → +∞

f (x(t)) -inf H f = o 1 t 1+q-1 p . (53)
Moreover, the solution trajectory x(t) converges weakly as t → +∞, and its limit belongs to S = argmin f . Proof We showed in the proof of Theorem 4 how to pass from (51) to (52). Conversely, let x(•) be a solution trajectory of the damped inertial dynamic (52). Let us show that by setting

y (t) = 1 γ τ (t) τ (t) ẋ (t) + x (t) , then (x, y) : [t 0 , +∞[ → H × H is a solution trajectory of                        ẏ (t) + τ (t) ∇f (y (t)) = 0 ẋ(t) + γ τ (t) τ (t) x(t) -γ τ (t) τ (t) y(t) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p ẏ (t) p-1 = 1.
(54) Indeed, by taking the time derivative of y(•), as given by the second equation of (54), we get

ẏ (t) = 1 γ τ (t) τ (t) ẍ (t) + 1 γ 1 + γ - τ (t) τ (t) [ τ (t)] 2 ẋ (t) = 1 γ τ (t) τ (t) ẍ (t) + (1 + γ) [ τ (t)] 2 -τ (t)τ (t) τ (t) τ (t) ẋ(t) = -τ (t) ∇f x (t) + 1 γ τ (t) τ (t) ẋ (t) = -τ (t) ∇f (y (t)) .
This gives the first equation in (54) and

[λ (t)] p ẏ (t) p-1 = [λ (t)] p | τ (t)| p-1 ∇f x (t) + 1 γ τ (t) τ (t) ẋ (t) p-1 = 1.
This shows the equivalence of the two systems. According to Theorem 2, and formula (31), there exists a constant C 1 > 0 such that

τ (t) ≥ C 1 (t -t 0 ) 1+q-1 p . ( 55
)
Therefore lim t→+∞ τ (t) = +∞. According to Theorem 4 we deduce

f (x(t)) -inf H f = o 1 t 1+q-1 p , (56)
and the convergence of the trajectory.

Damped inertial system via closed-loop control of the gradient

We proceed in parallel to the previous section to obtain the following result.

Theorem 6 Suppose that f : H → R satisfies (A). Let q > 0, p ≥ 1, γ > 1, and x : [t 0 , +∞[ → H be a solution trajectory of the following system

                   ẍ(t) + (1 + γ) τ (t) 2 -τ (t)τ (t) τ (t) τ (t) ẋ(t) + γ τ (t) 2 τ (t) ∇f x(t) + 1 γ τ (t) τ (t) ẋ(t) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p ∇f x (t) + 1 γ τ (t) τ (t) ẋ (t) p-1 = 1. ( 57
)
Then we have the fast convergence of values: as t → +∞

f (x(t)) -inf H f = o 1 t pq .
(58)

Moreover, the solution trajectory x(t) converges weakly as t → +∞, and its limit belongs to S = argmin f .

Proof Let x(•) be a solution trajectory of the damped inertial dynamic (57). Let us show show that by setting

y (t) = 1 γ τ (t) τ (t) ẋ (t) + x (t) , then (x, y) : [t 0 , +∞[ → H × H is a solution trajectory of                        ẏ (t) + τ (t) ∇f (y (t)) = 0 ẋ(t) + γ τ (t) τ (t) x(t) -γ τ (t) τ (t) y(t) = 0 τ (t) - 1 q q t 0 + t t0 [λ (r)] 1 q dr q = 0 [λ (t)] p ∇f (y (t)) p-1 = 1. ( 59
)
Indeed, by the same argument as for the velocity case, we get ẏ (t) = -τ (t) ∇f (y (t)) . This gives the first equation in ( 59) and

[λ (t)] p ∇f (y (t)) p-1 = [λ (t)] p ∇f x (t) + 1 γ τ (t) τ (t) ẋ (t) p-1 = 1.
This shows the equivalence of the two systems. According to Theorem 3, and formula (33), there exists a constant C 1 > 0 such that

τ (t) ≥ C 1 (t -t 0 ) pq . (60)
Therefore from Theorem 4 we deduce, as t → +∞

f (x(t)) -inf H f = o 1 t pq , (61)
and the convergence of the trajectory.

Comparison with the Lin-Jordan approach

In [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF], the authors study the second-order closed-loop dynamical system

               ẍ (t) + 2 τ (t) τ (t) - τ (t) τ (t) ẋ (t) + [ τ (t)] 2 τ (t) ∇ 2 f (x (t)) ẋ (t) + τ (t) [ τ (t) + τ (t)] τ (t) ∇f (x (t)) = 0 τ (t) - 1 4 t 0 λ (t)dr + c 2 = 0 [λ (t)] p ∇f (x (t)) p-1 = θ, (62)
whose first-order reformulation reads

               ẏ (t) + τ (t) ∇f (x (t)) = 0 ẋ (t) + τ (t) τ (t) (x (t) -y (t)) + [ τ (t)] 2 τ (t) ∇f (x (t)) = 0 τ (t) - 1 4 t 0 λ (t)dr + c 2 = 0 [λ (t)] p ∇f (x (t)) p-1 = θ, (63)
where c > 0 and 0 < θ < 1 are given parameters. See also [START_REF] Lin | Monotone Inclusions, Acceleration and Closed-Loop Control[END_REF] for some extensions to monotone inclusions. a) In [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF], the authors obtained the following convergence rate of function values

f (x (t)) -inf H f = O 1 t 3p+1 2
as t → +∞.

Note that the last two equations in (63) are nothing else than those in (32) with q := 2. For comparison, in our approach the convergence rate of the values obtained in Theorem 6 when q = 2 is

f (x (t)) -inf H f = o 1 t 2p
which is better for every p > 1. b) Let us now compare the convergence estimates of the gradients. In [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF], the authors obtain the integral estimate

+∞ t0 t 3p+1 2 ∇f (x (t)) p+1 p dt < +∞, which leads to inf t0≤σ≤t ∇f (x (σ)) = O t -3p 2 as t → +∞.
In our approach, the right variable to consider is y(t), instead of x(t). According to [START_REF] Muehlebach | A Dynamical Systems Perspective on Nesterov Acceleration[END_REF] we have

+∞ t0 τ (t) τ (t) ∇f (y (t)) 2 dt < +∞.
Since q = 2, according to [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF] we have

τ (t) = [τ (t)] 1 2 [λ (t)] 1 2 .
Therefore

τ (t) τ (t) ∇f (y (t)) 2 = τ (t) 3 2 [λ (t)] 1 2 ∇f (y (t)) 2 = τ (t) 3 2
∇f (y (t)) Again, our approach gives a better convergence rate than [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF]. Let us also specify that our analysis provides the convergence of the trajectories, which is an open question for [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF]. Moreover, since our approach is consistent with the steepest continuous descent, it can naturally be extended to the non-smooth case, and to the case of cocoercive operators, as it was done in the open-loop case in [START_REF] Attouch | Fast convex optimization via time scale and averaging of the steepest descent[END_REF].

The limiting case γ = 1

Our previous results are valid under the assumption γ > 1. It is a natural question to examine the limiting case γ = 1. Close examination of the proof of the theorem reveals a slight change in the integration procedure and a logarithm factor appears. The corresponding result obtained is written as follows.

Theorem 7 Suppose that f : H → R satisfies (A). Let x : [t 0 , +∞[ → H be a solution trajectory of the following second-order differential equation

ẍ(t) + 2 [ τ (t)] 2 -τ (t)τ (t) τ (t) τ (t) ẋ(t) + [ τ (t)] 2 τ (t) ∇f x(t) + τ (t) τ (t) ẋ(t) = 0 ( 64
)
where τ : [t 0 , +∞[ → R ++ is an increasing function, continuously differentiable, and satisfying lim t→+∞ τ (t) = +∞. Then we have the convergence rate of the values: as t → +∞

f (x(t)) -inf H f = o ln (τ (t)) τ (t) , (65)
and the solution trajectory x(t) converges weakly as t → +∞, and its limits belongs to S = argmin f . Suppose moreover that there exists some θ > 0 and C 1 > 0 such that for t sufficiently large

(A)asymp τ (t) ≥ C 1 (t -t 0 ) θ . ( 66
)
Then we have the fast convergence of values: as t → +∞

f (x(t)) -inf H f = o ln (t) t θ . ( 67
)
When specialized to the closed-loop control of the velocity, we obtain

f (x(t)) -inf H f = o ln (t) t 1+q-1 p , (68)
and in the case of the closed-loop control of the gradient

f (x(t)) -inf H f = o ln (t) t pq . (69)
So, the convergence rates are a little less good because of the logarithm term.

4 Associated proximal algorithms

A proximal-explicit discretization

In the following, we present a numerical approach based on a proximal-explicit temporal discretization of the closed-loop systems investigated in this paper. By proximal-explicit we mean that the function f is evaluated using a proximal step while the step size sequence (λ k ) k≥0 and the time scaling sequence (τ k ) k≥0 are computed explicitly. This makes our numerical scheme much easier implementable than the numerical algorithm proposed in [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF] as well as the large-step A HPE approach by Monteiro and Svaiter [START_REF] Monteiro | Svaiter An Accelerated Hybrid Proximal Extragradient Method for Convex Optimization and Its Implications to Second-order Methods[END_REF] which are in fact approximations of a proximal-implicit discrete time method. We restrict ourselves to the case q = 1, which gives τ (t) = λ (t). In this case, the continuous time closed-loop dynamical system is written as follows

   ẏ (t) + λ (t) ∇f (y (t)) = 0 [λ (t)] p [G (y (t))] p-1 = 1. (70)
Let us describe the general structure of the algorithm which is obtained by a proximal-explicit discretization of the continuous system (70). Given y k , y k-1 in H, we first define λ k by [λ k ] p [G (y k , y k-1 )] p-1 = 1. and consider then an implicit finite difference scheme for the first equation of (70)

y k+1 -y k + λ k ∇f (y k+1 ) = 0.
(71) This gives the following algorithm, called PEAS for Proximal Explicit Algorithm with Adaptive Step Size.

Algorithm 1: Proximal-explicit algorithm with adaptive step size (PEAS)

Input: y 0 = y -1 ∈ H 1 for k = 0, 1, • • • do 2 λ k := [G (y k , y k-1 )] -p-1 p 3 y k+1 := prox λ k f (y k ) 4 end
Note that (λ k ) k≥0 is computed explicitly in terms of (y k ) k≥0 . In other words, the definition of the sequence (λ k ) k≥0 is decoupled from the computation of (y k ) k≥0 . This is different from the method in [START_REF] Lin | A control-theoretic perspective on optimal high-order optimization[END_REF], which ultimately leads to the large-step A HPE approach by Monteiro and Svaiter in [START_REF] Monteiro | Svaiter An Accelerated Hybrid Proximal Extragradient Method for Convex Optimization and Its Implications to Second-order Methods[END_REF].

Let us now specify the link between λ k and τ k . We start from the relation (recall that we take q = 1)

τ (t) = λ (t) . (72)
Then, for every k ≥ 0 we discretize (72) as follows

τ k+1 -τ k = λ k ⇐⇒ τ k+1 = λ k + τ k (73)
with the convention λ 0 := t 0 and τ 0 := 0, which then yields

τ k = k-1 i=0 λ i .
Drawing inspiration from continuous analysis, we will first show that the function value

f (y k ) -inf H f attains the o 1 τ k+1
rate of convergence, and the sequence (y k ) k≥0 converges weakly to a solution. Then, as a crucial result, we will derive a lower bound of τ k+1 in terms of k.

The following result emphasizes that the rate of convergence and summability results holds for (y k ) k≥0 for arbitrary step sizes λ k that satisfy k≥0 λ k = +∞. The proof is an adaptation of [START_REF] Attouch | Fast convex optimization via time scale and averaging of the steepest descent[END_REF]Theorem 4.1].

Theorem 8 Let y 0 ∈ H, (λ k ) k≥0 be a given positive sequence satisfying k≥0 λ k = +∞, τ 0 = 0 and τ k = k-1 i=0 λ i for every k ≥ 1. Then for any sequence (y k ) k≥0 generated by the proximal algorithm

y k+1 := prox λ k f (y k ) ∀k ≥ 0, (74)
the following properties are satisfied:

(i) (summability of function values) k≥0 λ k f (y k+1 ) -inf H f < +∞; (ii) (summability of gradients) k≥0 τ k λ k ∇f (y k+1 ) 2 < +∞; (iii) (summability of velocities) k≥0 τ k λ k y k+1 -y k 2 < +∞; (iv) (convergence of function values) f (y k+1 ) -inf H f = o 1 τ k+1 as k → +∞; (v) (convergence of gradient) ∇f (y k+1 ) = o   1 k l=0 τ l λ l   as k → +∞;
(vi) the sequence of iterates (y k ) k≥0 converges weakly as k → +∞, and its limit belongs to S = argmin H f .

Proof Let k ≥ 0 be fixed. Take z * ∈ S = argmin f . According to (71) and the convexity of f , we deduce that 1 2

y k+1 -z * 2 = 1 2 y k -z * 2 + y k+1 -z * , y k+1 -y k - 1 2 y k+1 -y k 2 = 1 2 y k -z * 2 -λ k y k+1 -z * , ∇f (y k+1 ) - 1 2 λ 2 k ∇f (y k+1 ) 2 ≤ 1 2 y k -z * 2 -λ k f (y k+1 ) -inf H f - 1 2 λ 2 k ∇f (y k+1 ) 2 . (75)
Statement (i) follows from [START_REF] Bauschke | Convex Analysis and Monotone Operator Theory in Hilbert Spaces[END_REF]Lemma 5.31]. In addition, the limit lim k→+∞ y k -z * ∈ R exists, which means that the first condition of the discrete Opial's lemma is fulfilled.

On the other hand, the sequence (f (y k ) -inf H f ) k≥0 is nonincreasing. Precisely, we have for every k ≥ 0

f (y k ) -inf H f -f (y k+1 ) -inf H f ≥ ∇f (y k+1 ) , y k -y k+1 = λ k ∇f (y k+1 ) 2 ≥ 0. ( 76
)
According to [6, Lemma 22] we get

f (y k+1 ) -inf H f = o 1 k i=0 λ i ,
which proves (iv). Let k ≥ 1. Multiplying both sides of (76) by τ k = k-1 i=0 λ i > 0, then adding the result into (75), we get

τ k+1 f (y k+1 ) -inf H f + 1 2 y k+1 -z * 2 ≤ τ k f (y k ) -inf H f + 1 2 y k -z * 2 - 1 2 λ 2 k ∇f (y k+1 ) 2 -τ k λ k ∇f (y k+1 ) 2 .
This implies k≥0 λ 2 k ∇f (y k+1 ) 2 < +∞ and k≥1 τ k λ k ∇f (y k+1 ) 2 < +∞, which yields (ii). From (71), we infer (iii). To deduce (v), it suffices to show that the sequence ( ∇f (y k ) ) k≥0 is nonincreasing. Indeed, it follows from (74) and the cocoercivity of ∇f that 1 2 ∇f (y k+1

) 2 = 1 2 ∇f (y k ) 2 + ∇f (y k+1 ) , ∇f (y k+1 ) -∇f (y k ) - 1 2 ∇f (y k+1 ) -∇f (y k ) 2 = 1 2 ∇f (y k ) 2 - 1 λ k y k+1 -y k , ∇f (y k+1 ) -∇f (y k ) - 1 2 ∇f (y k+1 ) -∇f (y k ) 2 ≤ 1 2 ∇f (y k ) 2 .
Taking into account also (ii), we obtain (v). Finally, according to the assumption k≥0 λ k = +∞, and (iv), we have that lim k→+∞ f (y k ) = inf H f . Since f is convex and lower semicontinuous, the second condition of Opial's lemma is also fulfilled. This gives the weak convergence of the sequence (y k ) k≥0 to an element in S = argmin f .

Then, we give a statement which can be seen as a discrete counterpart of Lemma 1. The result is more complex not only because we are in the discrete setting, but also because it allows an explicit choice of the stepsize, as we will see later.

Lemma 2 Let (λ k ) k≥0 be a positive sequence and (τ k ) k≥0 such that τ 0 = 0 and τ k = k-1 i=0 λ i for all k ≥ 1. Suppose that there exist C 2 > 0 and a, b, c ≥ 0 such that b + c > a and

k≥0 τ a k λ -b k λ -c k+1 ≤ C 2 < +∞
Then there exists C 3 > 0 such that for every k ≥ 1 it holds

τ k+1 ≥ C 3 k b+c+1 b+c-a . ( 77
)
Proof By applying the Hölder inequality twice we get for all k ≥ 0

k i=0 τ a b+c+1 i ≤ k i=0 τ a i λ -b i λ -c i+1 1 b+c+1 k i=0 λ i b b+c+1 k i=0 λ i+1 c b+c+1 ≤ C 1 b+c+1 2 k+1 i=0 λ i b+c b+c+1 = C 1 b+c+1 2 τ b+c b+c+1 k+2 .
(78) If a = 0 then (77) follows immediately. From now on we suppose that a > 0. Inequality (78) becomes

k i=0 τ a b+c+1 i ≤ C 1 b+c+1 2 τ a b+c+1 k+2 b+c a ∀k ≥ 0. ( 79
)
Following the continuous counterpart, let us define

C b+c := C 1 b+c+1 2 > 0 and A k := k i=0 τ a b+c+1 i ∀k ≥ 0 so that (79) becomes A k ≤ C b+c (A k+2 -A k+1 ) b+c a ∀k ≥ 0. From here, C -a b+c b+c ≤ A -a b+c k (A k+2 -A k+1 ) ∀k ≥ 1. ( 80
)
For convenience, we define the following function ψ : R ++ → R ++ as ψ (r) := r -a b+c . It is clear that

d dr b + c b + c -a r 1-a b+c = ψ (r) and ψ (r) = - a b + c r -a b+c -1 < 0. Since (A k ) k≥0 is increasing, this means ψ (A k+2 ) ≤ ψ (r) ≤ ψ(A k ) for every A k ≤ r ≤ A k+2 .
Let k ≥ 1 fixed. We consider two separate cases.

Case 1: ψ (A k ) ≤ 2ψ (A k+2 ). Then (80) leads to

C -a b+c b+c ≤ A -a b+c k (A k+2 -A k ) = ψ (A k ) (A k+2 -A k ) ≤ 2ψ (A k+2 ) (A k+2 -A k ) = 2ψ (A k+2 ) A k+2 A k 1dr ≤ 2 A k+2 A k ψ (r) dr = 2 b + c b + c -a A 1-a b+c k+2 -A 1-a b+c k . Case 2: ψ (A k ) > 2ψ (A k+2 ). This is equivalent to A k+2 > 2 b+c a A k . Since b + c > a, we can deduce further A 1-a b+c k+2 > 2 b+c a -1 A 1-a b+c k . Consequently, A 1-a b+c k+2 -A 1-a b+c k > 2 b+c a -1 -1 A 1-a b+c k ≥ 2 b+c a -1 -1 A 1-a b+c 1 ,
recall that the last inequality follows from the increasing property of (A k ) k≥1 .

In conclusion, for every k ≥ 0 we have

A 1-a b+c k+2 -A 1-a b+c k ≥ C 4 := min 1 2 1 - a b + c C -a b+c b+c , 2 b+c a -1 -1 A 1-a b+c 1 , A 1-a b+c 2 > 0.
Telescoping sum arguments combined with (79) imply for every k ≥ 1

C 4 k ≤ A 1-a b+c 2k -A 1-a b+c 0 ≤ A 1-a b+c 2k ≤ C b+c-a (b+c)(b+c+1) 2 τ b+c-a b+c+1 2k+2 .
This gives for every k ≥ 1

τ 2k+3 ≥ τ 2k+2 ≥ C 3 k b+c+1 b+c-a ,
where C 3 > 0. We therefore deduce that there exists C 3 > 0 such that

τ k+1 ≥ C 3 k b+c+1 b+c-a ∀k ≥ 1,
which gives (77).

Following a plan identical to the continuous case, we successively consider the case where the control by feedback is formulated in terms of speed, then of gradient.

Adaptive stepsize rules resulting from the discretization of the velocity based system

In this subsection we specialize the algorithm PEAS to the case where G(y k , y k-1 ) = y k -y k-1 .

Algorithm 2: Proximal algorithm with adaptive step size defined via velocity

Input: y 0 = y -1 ∈ H 1 for k = 0, 1, • • • do 2 if ∇f (y k ) = 0 then 3 stop 4 else 5 λ k := y k -y k-1 -p-1 p 6 y k+1 := prox λ k f (y k ) 7 end 8 end
Theorem 9 Let (y k ) k≥0 be the sequence generated by Algorithm 2. Then it holds

f (y k ) -inf H f = o 1 k 2-1 p as k → +∞,
and the sequence of iterates (y k ) k≥0 converges weakly as k → +∞, and its limit belongs to S = argmin H f .

Proof By the choice of the step size, we have from Theorem 8 (iii) that

k≥0 τ k λ k y k+1 -y k 2 = k≥0 τ k λ -1 k λ -2p p-1 k+1 < +∞,
where τ 0 = 0 and τ k := k-1 i=0 λ i for every k ≥ 1. We are in position to apply Lemma 2 with (a, b, c) := 1, 1, 2p p-1 . We get

τ k+1 ≥ C 3 k 2-1 p ∀k ≥ 1. (81)
Therefore k≥0 λ k = lim k→+∞ τ k = +∞, and we can apply Theorem 8 to obtain the conclusion.

4.3 Adaptive stepsize resulting from the discretization of the gradient based system Now let us specialize the algorithm PEAS to the case where G(y k , y k-1 ) = ∇f (y k ) .

Algorithm 3: Proximal algorithm with adaptive step size defined via gradient

Input: y 0 ∈ H 1 for k = 0, 1, • • • do 2 if ∇f (y k ) = 0 then 3 stop 4 else 5 λ k := ∇f (y k ) -p-1 p 6 y k+1 := prox λ k f (y k ) 7 end 8 end
Theorem 10 Let (y k ) k≥0 be the sequence generated by Algorithm 3. Then it holds

f (y k+1 ) -inf H f = o 1 k 2-1 p as k → +∞
and the sequence of iterates (y k ) k≥0 converges weakly as k → +∞, and its limit belongs to S = argmin H f .

Proof In this case we have from Theorem 8 (ii)

k≥0 τ k λ k ∇f (y k+1 ) 2 = k≥0 τ k λ k λ -2p p-1 k+1 < +∞, (82)
where τ 0 = 0 and τ k := k-1 i=0 λ i for every k ≥ 1. Let us establish an inequality of the type

∇f (y k ) ≤ C k ∇f (y k+1 ) ,
for some sequence C k > 0 which is to be precised. We have for all

k ≥ 0 ∇f (y k ) 2 = ∇f (y k+1 ) 2 -2 ∇f (y k+1 ) , ∇f (y k+1 ) -∇f (y k ) + ∇f (y k+1 ) -∇f (y k ) 2 = ∇f (y k+1 ) 2 + 2 λ k y k+1 -y k , ∇f (y k+1 ) -∇f (y k ) + ∇f (y k+1 ) -∇f (y k ) 2 ≤ ∇f (y k+1 ) 2 + 2L λ k + L 2 y k+1 -y k 2 = (1 + Lλ k ) 2 ∇f (y k+1 ) 2 ≤ (1 + Lλ k+1 ) 2 ∇f (y k+1 ) 2 (83)
where L > 0 denotes the Lipschitz constant of ∇f on a bounded set containing the sequence (y k ) k≥0 . Combining (82) and (83), we get

k≥0 τ k λ k 1 (1 + Lλ k+1 ) 2 ∇f (y k ) 2 = k≥0 τ k λ k 1 (1 + Lλ k+1 ) 2 λ -2p p-1 k < +∞. ( 84
)
Let us now show that lim k→+∞ λ k = +∞. According to the decreasing property of the sequence (f (y k ) -inf H f ) k≥0 , by summing inequalities (76) we get k≥0 λ k ∇f (y k+1 ) 2 < +∞.

(85)

From the closed-loop rule we deduce that

k≥0 λ k λ -2p p-1 k+1 < +∞. ( 86
) Therefore lim k→+∞ λ k λ -2p p-1 k+1 = 0.
Since (λ k ) k≥0 is increasing, let us denote by l > 0 its limit. If l is finite then, by passing to the limit on the above inequality we get l 1-2p p-1 = 0, a clear contradiction with l > 0. Therefore lim k→+∞ λ k = +∞.

In this case

1 (1+Lλ k+1 ) 2 ∼ (Lλ k+1 ) -2 , which gives k≥0 τ k λ 1-2p p-1 k λ k+1 -2 < +∞. ( 87
)
We are in position to apply Lemma 2 with (a, b, c) := 1, 2p p-1 -1, 2 . We get

τ k+1 ≥ C 3 k 2-1 p ∀k ≥ 1.
We have k≥0 λ k = lim k→+∞ τ k = +∞, and we can apply Theorem 8 to obtain, as k → +∞

f (y k+1 ) -inf H f = o 1 k 2-1 p .
This completes the proof.

Remark 3 Note that the closed-loop control of the velocity and the closed-loop control of the gradient give the same convergence rate of the values. Clearly, we have obtained a faster convergence result.

Inertial proximal algorithms obtained by closed-loop damping

Let us now consider the convergence properties of the sequences (x k ) k≥0 which are obtained by applying the averaging process to the sequences generated by Algorithm 2. Indeed, we limit our investigation to the closed loop control of the velocity, the case of the closed loop control of the gradient is very similar. Let us discretize the continuous averaging relation

ẋ(t) + τ (t) τ (t) (x(t) -y(t)) = 0
as follows (recall that, because of the choice q = 1, we have τ (t) = λ(t))

x k+1 -x k + λ k τ k+1 (x k -y k+1 ) = 0. Equivalently x k+1 = 1 - λ k τ k+1 x k + λ k τ k+1 y k+1 .
This gives the following proximal inertial algorithm:

Theorem 11 Let (x k ) k≥0 be the sequence generated by Algorithm 4. Then it holds

f (x k ) -inf H f = O 1 k 2-1 p as k → +∞
and the sequence of iterates (x k ) k≥0 converges weakly as k → +∞, and its limit belongs to S = argmin H f . Algorithm 4: Proximal inertial algorithm with adaptive step size defined via velocity Input: τ 0 := 0 and x 0 , y 0 which gives (recall that

= y -1 ∈ H 1 for k = 0, 1, • • • do 2 if ∇f (y k ) = 0 then 3 stop 4 else 5 λ k := y k -y k-1 -p-1 p 6 y k+1 := prox λ k f (y k ) 7 τ k+1 := τ k + λ k 8 x k+1 := 1 - λ k τ k+1 x k + λ k τ k+1 y k+1 .
τ k+1 = k i=0 λ i ) τ k+1 x k+1 -τ k x k = (τ k+1 -λ k )x k + λ k y k+1 -τ k x k = (τ k+1 -τ k -λ k )x k + λ k y k+1 = λ k y k+1 .
Therefore, by telescoping arguments we obtain

x k+1 = k i=0 λ i y i+1 τ k+1 ∀k ≥ 0. ( 88
)
By convexity of f we infer

f (x k+1 ) -inf H f = f -inf H f (x k+1 ) = f -inf H f k i=0 λ i y i+1 τ k+1 ≤ 1 τ k+1 k i=0 λ i f -inf H f (y i+1 ) = 1 τ k+1 k i=0 λ i f (y i+1 ) -inf H f ,
By Theorem 8 (i), we have k≥0 λ k (f (y k+1 ) -inf H f ) < +∞, and by (81) we have τ k+1 ≥ k 2-1 p , which gives the claim. The weak convergence of (x k ) k≥0 to an element in S = argmin H f follows from the weak convergence of (y k ) k≥0 and the Stolz-Cesáro Theorem.

Geometric interpretation of PEAS

First note that (PEAS) can be equivalently written as follows

x k+1 = 1 - λ k τ k+1 x k + λ k τ k+1 prox λ k f x k-1 + τ k λ k-1 (x k -x k-1 . (89)
Since τ k λ k-1 > 1, the algorithm first involves an extrapolation step (this is the inertial aspect), then a proximal step, and finally a relaxation step which balances the inertia effect and dampens the oscillations. This is shown in the figure below. We set θ k = λ k τ k+1 ∈]0, 1[. Despite some analogies, (PEAS) is different from the relaxed inertial proximal algorithm (RIPA) considered by Attouch and Cabot in [START_REF] Attouch | Convergence of a relaxed inertial proximal algorithm for maximally monotone operators[END_REF], and which writes

y k = x k + α k (x k -x k-1 ) x k+1 = (1 -ρ k )y k + ρ k prox λ k f (y k ) (90)
As main difference, in (PEAS) relaxation is taken between x k and prox λ k f (y k ), while in (RIPA) it is taken between y k and prox λ k f (y k ). Consequently (PEAS) involves a Hessian damping effect which is not present in (RIPA). Note in (PEAS) the balance between the extrapolation (inertial, acceleration) effect and the relaxation effect. Moreover, our construction provides coefficients which are generated automatically in closed loop way, whereas in (RIPA) they require subtle adjustment. The importance of the relaxation technique when combined with inertia has been put to the fore in [START_REF] Iutzeler | A generic online acceleration scheme for optimization algorithms via relaxation and inertia[END_REF]. According to (88), x k+1 can be

y k = x k-1 + 1 θ k-1 (x k -x k-1 ) • x k • x k-1 • • x k+1 = (1 -θ k ) x k + θ k prox λ k f (y k ) prox λ k f (y k ) S & & & a £ £ £ £ 7 7 7 7
Fig. 1 A geometrical illustration of algorithm PEAS interpreted as an average of the {yn : 0 ≤ n ≤ k + 1}, which makes our approach somewhat analogous to the nonlinear averaging technique developed in [START_REF] Scieur | Regularized nonlinear acceleration[END_REF], where it is assumed that there is a unique minimizer. Averaging techniques have also been used in [START_REF] Poveda | Robust Hybrid Zero-Order Optimization Algorithms with Acceleration via Averaging in Time[END_REF] in the context of hybrid systems. Indeed, adjusting the damping in a closed-loop ad hoc manner bears some analogy to restarting methods.

Conclusion, perspective

Our study proposes new fast adaptive optimization methods for convex optimization. We have shown that the time scaling and averaging technique, previously developed by the authors in the context of non-autonomous systems, can be developed by taking closed-loop time parameterization, giving rise to autonomous dynamics. The method turns out to be flexible, because it is based on elementary mathematical tools, namely the dynamics of the steepest descent, and the operations of temporal parameterization and averaging. It is therefore not necessary to redo a Lyapunov analysis, one relies on the classic results for the steepest descent. The results obtained for the continuous dynamics pass quite naturally to the corresponding proximal algorithms, where the iterates are expressed in a direct way according to the proximal terms. This study is one of the very first to develop an algorithmic framework based on autonomous dynamics and which, when specialized, provides the convergence rates of the dynamical surrogate of the Nesterov acceleration gradient method. Another important aspect of our analysis is that it exhibits Hessian-driven damping, which plays a key role in damping oscillations. Our work opens up many perspectives, our method naturally extending to gradient algorithms, proximal-gradient algorithms for composite optimization, cocercive monotone operators, and the study of the stochastic version, to name only a few. Under the standing assumption (A) on f , we know that, for any z 0 ∈ H there exists a unique classical global solution z ∈ C 1 ([t 0 , +∞[: H) of (SD) satisfying z(t 0 ) = z 0 , see [START_REF] Attouch | Variational analysis in Sobolev and BV spaces. Applications to PDE's and optimization[END_REF]Theorem 17.1.1]. We fix t 0 as the origin of time. Recall classical facts concerning the continuous steepest descent.

) 93
Then the following statements are satisfied:

9 end 10 end

 910 Proof Let k ≥ 0. By definition of x k+1 we haveτ k+1 x k+1 = (τ k+1 -λ k )x k + λ k y k+1

7 Appendix 7 . 1

 771 Classical facts concerning the continuous steepest descent Consider the classical continuous steepest descent (SD) ż(t) + ∇f (z(t)) = 0.(91)

  Theorem 12 Suppose that f :H → R satisfies (A). Let z : [t 0 , +∞[ → H be a solution trajectory of ż(t) + ∇f (z(t)) = g(t)(92)where g : [t 0 , +∞[ → H is such that

+∞ t0 g (t) dt < +∞ and +∞ t0 t g (t) 2 dt < +∞. (

The research of RIB and DKN has been supported by FWF (Austrian Science Fund), projects W 1260 and P 34922-N, respectively.

(i) (convergence of gradients towards zero) ∇f (z (t)) = o 1 √ t as t → +∞.

(ii) (integral estimate of the velocities) +∞ t0 t ż (t) 2 dt < +∞.

(iii) (integral estimate of the gradients)

(vi) The solution trajectory z(t) converges weakly as t → +∞, and its limit belongs to S = argmin f .

If g(t) ≡ 0 we have that t → ∇f (z (t)) is nonincreasing since in this case

Therefore, from the integral estimate of the gradients we deduce that ∇f (z (t)) = o 1 t .

Auxiliary result

Opial's Lemma is a basic ingredient of the convergence analysis.

Lemma 3 (Opial) Let S be a nonempty subset of H and let (x k ) k≥0 be a sequence in H. Assume that (i) for every z ∈ S, lim k→+∞ x k -z exists;

(ii) every weak sequential limit point of (x k ) k≥0 , as k → +∞, belongs to S.

Then (x k ) k≥0 converges weakly as k → +∞, and its limit belongs to S.