Avant tout, je souhaiterais exprimer ma reconnaissance et toute mon amitié à mes directeurs de thèse, Hédy et Juan. Je n'aurais pu espérer trouver de personnes plus talentueuses et sympathiques pour m'encadrer. Comme un long discours ne saurait exprimer tout ce que je leur dois, je me contenterai donc ici de les remercier chaleureusement pour leur soutien, leur patience(!) et plus généralement pour tout ce qu'ils m'ont transmis et appris ces dernières années.

Mes remerciements vont aussi à Aris Daniilidis et Adrian Lewis, qui m'ont fait l'honneur de rapporter ce manuscrit, ainsi qu'à Lionel Thibault, Jalal Fadili et Pedro Gajardo qui ont accepté de faire partie de mon jury de thèse, ces deux derniers ayant fait un long (voire très long) chemin pour venir.

Durant ces trois années passées à l'I3M, j'ai pu profiter de la présence de moult personnes qui ont fait de cette thèse une successions de pauses cafés fort agréables. Merci Micka d'avoir fait quelques arrêts dans notre bureau pendant tes parties de cache-cache, et de m'avoir laissé te faire perdre ton temps quand j'en avais besoin. Un jour la valeur des services que tu as rendus à la communauté seront reconnus à leur juste valeur, et une statue à ton effigie trônera fièrement quelque part. Mes salutations vont aussi aux visiteurs réguliers du bureau 115: Christophe (qui nous a quittés bien trop tôt), Benjamin (merci pour ces discussions totalement vaines), Myriam (puisse ta bonne humeur bénéficier aux autres encore longtemps), mais aussi Gauthier, Joub, Julien, Etienne, Theo, Elsa, Antoine, et tous ceux dont les noms n'apparaissent pas ici mais qui font (ou ont fait) de ce labo un endroit très sympathique.

Ese tesis fue también la ocasión para mi de viajar a Chile y descubrir allá gente formidable. Primero entre todos, Cecio, hermano institucional, pero mas que todo, de corazón. Grazie para tu buena onda comunicativa, perpetua e indefectible, y todas ese noches tontas que tuvimos. Un agradezco también a todos los Sansanitos, gracia a quien me sentí como en casa en Valpo: Pato (le plus génial de tous les canards), Andrei, Felipe, Laurent, Mathieu. Gracias a David y su hospitalidad, que me hizo descubrir un poco su capital. Doy también a un grande abrazo a Willy, Meike, Miriam y Martina, compañeros de Casa Viejas, asi que a la gente de Casa Chancho y calle Perez.

Évidemment, je ne saurais oublier les copains, qui sont toujours là quand le besoin de passer un bon moment se fait sentir 1 . En premier lieu Laurent, qui m'a amicalement recueilli sur son canapé, et avec qui nous avons vécu des aventures toujours plus chocolatées. Puis bien entendu Brice, Marion, Julie, Kévin, Jérémy, Laurène, Renaud, et tous les autres Vinassanais de coeur. J'ai aussi une pensée chaleureuse pour ma famille, qui me supporte (dans tous les sens du terme) depuis toujours! Enfin, mes dernières pensées (et non les moindres) vont à toi, Julie, qui est ma première supportrice (et relectrice!), d'un soutien sans réserve, et sans qui je n'aurais peut-être pas trouvé le courage de commencer/terminer cette thèse. Merci pour tout! 1 Rectification: vous êtes TOUJOURS fourrés à la maison. De vrais parasites! VII VIII

Abstract.

In a first part, we focus on gradient dynamical systems governed by nonsmooth but also nonconvex functions, satisfying the so-called Kurdyka-Lojasiewicz inequality. After obtaining preliminary results for a continuous steepest descent dynamic, we study a general descent algorithm. We prove, under a compactness assumption, that any sequence generated by this general scheme converges to a critical point of the function to be minimized. We also obtain new convergence rates both for the values and the iterates. The analysis covers alternating versions of the forward-backward method, with variable metric and relative errors. As an example, a nonsmooth and nonconvex version of the Levenberg-Marquardt algorithm is detailed. Applications to nonconvex feasibility problems, and to sparse inverse problems are discussed.

In a second part, the thesis explores descent dynamics associated to constrained vector optimization problems. For this, we adapt the classic steepest descent dynamic to functions with values in a vector space ordered by a closed convex cone with nonempty interior. It can be seen as the continuous analogue of various descent algorithms developed in the last years. We have a particular interest for multi-objective decision problems, for which the dynamic make decrease all the objective functions along time. We prove the existence of trajectories for this continuous dynamic, and show their convergence to weak efficient points. Then, we explore an inertial dynamic for multi-objective problems, with the aim to provide fast methods converging to Pareto points.

Résumé.

Dans une première partie, nous nous intéressons aux systèmes dynamiques gradients gouvernés par des fonctions non lisses, mais aussi non convexes, satisfaisant l'inégalité de Kurdyka-Lojasiewicz. Après avoir obtenu quelques résultats préliminaires pour la dynamique de la plus grande pente continue, nous étudions un algorithme de descente général. Nous prouvons, sous une hypothèse de compacité, que tout suite générée par ce schéma général converge vers un point critique de la fonction à minimiser. Nous obtenons aussi de nouveaux résultats sur la vitesse de convergence, tant pour les valeurs que pour les itérés. Ce schéma général couvre en particulier des versions parallélisées de la méthode forward-backward, autorisant une métrique variable et des erreurs relatives. Cela nous permet par exemple de proposer une version non convexe non lisse de l'algorithme Levenberg-Marquardt. Enfin, nous proposons quelques applications de ces algorithmes aux problèmes de faisabilité, et aux problèmes inverses parcimonieux.

Dans une seconde partie, cette thèse développe une dynamique de descente associée à des problèmes d'optimisation vectoriels sous contrainte. Pour cela, nous adaptons la dynamique de la plus grande pente usuelle aux fonctions à valeurs dans un espace ordonné par un cône convexe fermé d'intérieur non vide. Cette dynamique peut être vue comme l'analogue continu de nombreux algorithmes développés ces dernières années. Nous avons un intérêt particulier pour les problèmes de décision multi-objectifs, pour lesquels cette dynamique de descente fait décroitre toutes les fonctions objectif au cours du temps. Nous prouvons l'existence de trajectoires pour cette dynamique continue, ainsi que leur convergence vers des points faiblement efficients. Finalement, nous explorons une nouvelle dynamique inertielle pour les problèmes multi-objectif, avec l'ambition de développer des méthodes rapides convergeant vers des équilibres de Pareto.

Resumen.

En una primera parte, nos interesamos en sistemas dinámicos de tipo gradiente gobernados por funciones no diferenciables, también no convexas, que satisfacen la desigualdad de Kurdyka-Lojasiewicz. Después de obtener resultados preliminares para la dinámica continua del máximo descenso, estudiamos un algoritmo de descenso general. Demostramos, bajo una hipótesis de compacidad, que cualquier sucesión generada según este esquema general converge a un punto crítico de la función. Además, obtenemos nuevas estimaciones sobre la velocidad de convergencia para la secuencia y sus valores. Este análisis cubre versiones alternadas del método forward-backward, con métrica variable y errores aditivos. Nos permite por ejemplo de proponer una versión no diferenciable y no convexa del algoritmo de Levenberg-Marquardt. Por fin, proponemos algunas aplicaciones de esos algoritmos para resolver problemas de viabilidad non convexos, y problemas inversos.

En una segunda parte, la tesis explora una dinámica de descenso asociada con problemas de optimización vectoriales con restricción. Para esto, adaptamos la dinámica del máximo descenso clásica para los funciones con valores en un espacio vectorial ordenado por un cono sólido cerrado convexo. Puede ser visto como el análogo continuo de varios algoritmos de descenso desarrollados en los años pasados. Tenemos un interés particular por los problemas de decisión multiobjetivo, por lo cuales cada función objetivo decrece con el tiempo. Demostramos la existencia de trayectorias para esta dinámica continua, y mostramos su convergencia a puntos débilmente eficientes. Después, investigamos una nueva dinámica inercial para problemas multiobjetivo, con el propósito de proponer métodos rápidos que convergen a equilibrios de Pareto.

Chapter 1

Introduction

The idea of using a descent method to solve an optimization problem goes back, at least, to Cauchy and its gradient method [START_REF] Cauchy | Méthode générale pour la résolution des systèmes d'équations simultanées[END_REF]. This algorithm has been the object of an intense research along the 20th century, considering for instance a line-search method for the choice of the stepsize [START_REF] Curry | The method of steepest descent for nonlinear minimization problems[END_REF][START_REF] Goldstein | Cauchy's method of minimization[END_REF][START_REF] Armijo | Minimization of functions having Lipschitz continuous first partial derivatives[END_REF][START_REF] Zangwill | Convergence conditions for nonlinear programming algorithms[END_REF]. The development of convex analysis in the 60-70s opened the door to a nonsmooth version of the implicit gradient method, namely, the proximal algorithm, introduced by Martinet [START_REF] Martinet | Régularisation d'inéquations variationnelles par approximations successives[END_REF], and further developed in [START_REF] Rockafellar | Monotone operators and the proximal point algorithm[END_REF][START_REF] Brézis | Produits infinis de résolvantes[END_REF]. It appeared clear at that time that the proximal algorithm is strongly connected to its continuous-time counterpart, the steepest descent dynamic [START_REF] Rosenbloom | The method of steepest descent[END_REF][START_REF] Gavurin | Nonlinear functional equations and continuous analogues of iteration methods[END_REF][START_REF] Brézis | Monotonicity methods in Hilbert spaces and some applications to nonlinear partial differential equations[END_REF][START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF][START_REF] Crandall | Generation of semi-groups of nonlinear transformations on general Banach spaces[END_REF]:

(1.1) u(t) + ∂f (u(t)) ∋ 0,
where f : H -→ R∪{+∞} is a proper lower semi-continuous convex function defined on a Hilbert space H. Clearly, the explicit and implicit versions of the gradient method can both be derived from (1.1) after a time discretization. But the inter-connexions between the continuous and discrete dynamics go beyond this simple observation. For instance, the trajectories of (1.1) can be built from sequences generated by the proximal algorithm [START_REF] Crandall | Generation of semi-groups of nonlinear transformations on general Banach spaces[END_REF]. Furthermore, the equivalence between the asymptotic behaviour of the proximal algorithm and the trajectories of (1.1) has been recently established [START_REF] Alvarez | Asymptotic almost-equivalence of Lipschitz evolution systems in Banach spaces[END_REF][START_REF] Alvarez | A unified approach to the asymptotic almost-equivalence of evolution systems without Lipschitz conditions[END_REF][START_REF] Peypouquet | Evolution equations for maximal monotone operators: asymptotic analysis in continuous and discrete time[END_REF]. In practical, the connections between continuous and discrete dynamics are fruitful for the optimizer. For example, finding a Lyapunov function for a continuous dynamic is in general easier than for the discrete case. On the reverse, the discrete dynamics benefit from the fact that their analysis do not require any derivability assumption on the trajectory.

In this thesis, we develop descent dynamics, being discrete or continuous, for two optimization problems which gained in interest these last years. Firstly, we deal with a tame optimization problem, which consists in the minimization of a nonsmooth nonconvex function being well-behaved, in some way. The growing interest for tame problems comes from the fact that various problems in image and signal processing are tame, by nature. Secondly, we consider a multi-objective (also multi-criteria) optimization problem. Such problem involves a finite family of cost functions, which have to be minimized simultaneously, as far as possible. This kind of problem arises naturally in engineering (shape optimization), or in decision sciences. For these two problems, we adopt the same approach: building a descent dynamic, and proving its convergence to a solution. As much as we can, we shall underline the connexions between the continuous and discrete dynamics studied. We introduce now in detail the two parts of this manuscript, corresponding to the two aforementioned problems.

Part I: Dynamics for tame optimization

In classic analysis, it is known for a long time that there exists generically wild functions, being for instance continuous but nowhere differentiable [START_REF] Banach | Über die Baire'sche Kategorie gewisser Funktionenmengen[END_REF][START_REF] Mazurkiewicz | Sur les fonctions non dérivables[END_REF]. In the more recent nonsmooth analysis setting, such pathologic functions are also known to be pre-eminent. Consider for instance, among the locally Lipschitz functions, the ones having a Clarke subdifferential constant to the unit ball [START_REF] Borwein | Generalized subdifferentials: a Baire categorical approach[END_REF]. On this subject, Grothendieck [START_REF] Grothendieck | Esquisse d'un programme[END_REF] condemns "the wildness carried [by the general topology] as an inevitable fate" and calls for a new "tame topology", more adapted to "the context in which we live, breath, work". Indeed, most of the functions/sets encountered in optimization problems (at least in finite dimensions) are not so much monstrous: least squares, ℓ p norms, functions being polynomial by parts, etc. In the literature, various notions can be found that ensure the good behaviour of a function, in a setting broader than the smooth or convex ones. Let us mention for instance the classes of semi-smooth [START_REF] Mifflin | Semismooth and Semiconvex Functions in Constrained Optimization[END_REF], lower-C 2 [START_REF] Rockafellar | Favorable classes of Lipschitz continuous functions in subgradient optimization[END_REF][START_REF] Hiriart-Urruty | Generalized Differentiability / Duality and Optimization for Problems Dealing with Differences of Convex Functions[END_REF][START_REF] Penot | Favorable classes of mappings and multimappings in nonlinear analysis and optimization[END_REF], primal-lower-nice [START_REF] Poliquin | Integration of subdifferentials of nonconvex functions[END_REF], prox-regular [START_REF] Poliquin | Prox-regular functions in variational analysis[END_REF], or partly smooth functions [START_REF] Lewis | Active sets, nonsmoothness, and sensitivity[END_REF], whose interest in optimization is not to be proved [START_REF] Qi | A nonsmooth version of Newton's method[END_REF][START_REF] Levy | Partial extensions of Attouch's theorem with applications to proto-derivatives of subgradient mappings[END_REF][START_REF] Thibault | Integration of subdifferentials of lower semicontinuous functions on Banach spaces[END_REF][START_REF] Noll | Convergence of non-smooth descent methods using the Kurdyka-Lojasiewicz inequality[END_REF][START_REF] Liang | Local linear convergence of Forward-Backward under partial smoothness[END_REF].

Tame optimisation, as mentioned in the title of this thesis, is a general term, referring to optimization of functions being pathologic-free. We intentionally do not give a precise meaning to the word 'tame', and prefer for now letting the reader keep in mind the picture of well-behaved objects 1 .

According to van den Dries [START_REF] Van Den Dries | Tame Topology and O-minimal Structures[END_REF], the o-minimal structures provide the first known framework in which Grothendieck's tame topology could be developed. An o-minimal structure is a collection of sets, stable under usual operations, and a function is said to be definable in this structure, whenever its graph -or its epigraph-is a set lying in the structure. These structures have been modelled to generalize the properties of two known collections of sets: the semialgebraic [START_REF] Whitney | Elementary structure of real algebraic varieties[END_REF][START_REF] Bochnak | Géométrie algébrique réelle[END_REF][START_REF] Van Den Dries | A generalization of the Tarski-Seidenberg theorem, and some nondefinability results[END_REF] and globally subanalytic sets [START_REF] Lojasiewicz | Une propriété topologique des sous-ensembles analytiques réels[END_REF][START_REF] Gabriélov | Projections of semi-analytic sets[END_REF][START_REF] Hironaka | Introduction to real-analytic sets and real-analytic maps[END_REF]. While this theory was originally designed as a model theory in the 80s [START_REF] Pillay | Definable sets in ordered structures. I[END_REF], it appeared soon that these o-minimal structures offer a good setting in which doing analysis. Indeed, the objects definable in these structures enjoy a lot of finitude properties. For instance, the sets definable in these structures admit a finite number of connected components. This implies for instance that the level sets of definable functions have a finite number of connected components, which excludes pathological functions like 1 x sin(x). The definable sets also admit a Whitney stratification, which means that we can always decompose them into a finite number of smooth manifolds fitting together in a regular way. These examples are far to be exhaustive, and can be found for instance in [START_REF] Van Den Dries | Geometric categories and o-minimal structures[END_REF].

Among all the properties satisfied by the functions definable in an o-minimal structure, there is one which will keep our attention: the so-called Kurdyka-Lojasiewicz property. It is a property describing the behaviour of function in the neighbourhood of its critical points. Its main feature is that it ensures the finite length for the trajectories of descent dynamics (being continuous or discrete). Thus, this Kurdyka-Lojasiewicz property is an excellent tool in optimization, in order to validate the convergence of numerical methods. Let us give some details on this property.

In 1984, Lojasiewicz [START_REF] Lojasiewicz | Sur les trajectoires du gradient d'une fonction analytique[END_REF] is interested by the asymptotic behaviour of the trajectories being solution of the steepest descent dynamic (1.2) u(t) + ∇f (u(t)) = 0, where f : R n -→ R is an analytic function. Analyticity is not a too strong hypothesis, since it is known at this time that the gradient flow associated to a C ∞ function can not converge [START_REF] Palis | Geometric theory of dynamical systems: An introduction[END_REF]. Lojesiewicz proves that any bounded trajectory of (1.2) has finite length, and converges to a critical point of f . The main tool in its proof is the following: for analytic functions, we have a particular estimate, previously proved by Lojasiewicz itself [START_REF] Lojasiewicz | Une propriété topologique des sous-ensembles analytiques réels[END_REF], linking the variations of the values f (u(t)) and of the norm of ∇f (u(t)). More exactly, for any critical point x of f , there exists C > 0 and θ ∈]0, 1 2 ], such that

(1.3) ∀x ∼ x, x = x, |f (x) -f (x)| 1-θ ď C ∇f (x) .
The equation above is nowadays known as the Lojasiewicz inequality. If we assume that f (x) > f (x) = 0, then the Lojasiewicz inequality can be rewritten as

(1.4) ∀x ∼ x, x = x, 1 ď ∇(ϕ • f )(x) , with ϕ(t) = C θ t θ .
In other words, up to a polynomial reparametrization, f is sharp around x.

In parallel, Simon [START_REF] Simon | Asymptotics for a class of nonlinear evolution equations, with applications to geometric problems[END_REF] studies the asymptotic behaviour of general nonlinear evolution equations. For example, they can take the form u(t, x) -∆u(t, x) = φ(u(t, x)) on R + × Ω,

u(t, x) = 0 on R + × ∂Ω,
where φ : R -→ R is a source term. Assuming that the source term φ is analytic, and using the Lojasiewicz inequality (1.3), Simon proves under some conditions that the trajectories of its dynamic converge strongly to an equilibrium. This approach will be reused later for the asymptotic analysis of some PDE's involving an analytic feature. Let us mention, among others, some studies of the semilinear wave equation [START_REF] Haraux | Convergence of bounded weak solutions of the wave equation with dissipation and analytic nonlinearity[END_REF], degenerate parabolic equations [START_REF] Feireisl | Convergence for semilinear degenerate parabolic equations in several space dimensions[END_REF], the Cahn-Hilliard equation [START_REF] Hoffmann | Convergence of solutions to Cahn-Hilliard equation[END_REF], equations for phase transition [START_REF] Feireisl | A non-smooth version of the Lojasiewicz-Simon theorem with applications to non-local phase-field systems[END_REF] or quasi-linear parabloic equations [START_REF] Chill | Convergence and decay rate to equilibrium of bounded solutions of quasilinear parabolic equations[END_REF]. See also [START_REF] Baudoin | Constructive solution of a bilinear optimal control problem for a Schrödinger equation[END_REF] for an optimal control problem governed by a Schrödinger equation, and [START_REF] Alvarez | A second-order gradient-like dissipative dynamical system with Hessian-driven damping: application to optimization and mechanics[END_REF] for an inertial gradient-like system involving a Hessian-driven damping.

In 1998, Kurdyka [START_REF] Kurdyka | On gradients of functions definable in o-minimal structures[END_REF] considers the steepest descent dynamic (1.2), and adapts Lojasiewicz's convergence result to any C 1 function definable in an o-minimal structure. For this, he needs an appropriate extension of Lojasiewicz's inequality. Indeed, the function x → e -1

x 2 , which lies in some o-minimal structure, cannot satisfy the Lojasiewicz inequality, because it is exponentially flat around its minimum. That is why Kurdyka proves in a first time that, for any differentiable function definable in an o-minimal structure, the inequality (1.4) holds around critical points, for a general definable function ϕ (not necessarily a polynomial). This inequality, which can be rewritten as ϕ ′ (f (x)f (x)) ∇f (x) ě 1, is now called the Kurdyka-Lojasiewicz inequality. More recently, Bolte, Daniilidis and Lewis [START_REF] Bolte | The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF] adapt the Lojasiewicz inequality to nonsmooth subanalytic functions, and derive a finite length property for the bounded trajectories of

(1.5) u(t) + ∂ F f (u(t)) ∋ 0.
Finally, in 2007, a strong result of Bolte, Daniilidis, Lewis and Shiota [START_REF] Bolte | Clarke subgradients of stratifiable functions[END_REF] asserts that any lower semi-continuous function definable in an o-minimal structure satisfies a nonsmooth version of the Kurdyka-Lojasiewicz inequality.

The convergence analysis of descent algorithms for functions satisfying the Kurdyka-Lojasiewicz inequality is more recent. See [START_REF] Absil | Convergence of the iterates of descent methods for analytic cost functions[END_REF] for gradient-related methods, and [START_REF] Noll | Convergence of non-smooth descent methods using the Kurdyka-Lojasiewicz inequality[END_REF] for a nonsmooth subgradient-oriented descent method. The proximal method is investigated in [START_REF] Attouch | On the convergence of the proximal algorithm for nonsmooth functions involving analytic features[END_REF][START_REF] Bolte | Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity[END_REF], as well as in [START_REF] Merlet | Convergence to equilibrium for the backward Euler scheme and applications[END_REF][START_REF] Grasselli | Convergence to equilibrium of solutions of the backward Euler scheme for asymptotically autonomous second-order gradient-like systems[END_REF][START_REF] Alaa | Convergence to equilibrium for discretized gradient-like systems with analytic features[END_REF] with a specific attention to applications for PDE's discretization. The celebrated Forward-Backward algorithm, a splitting method exploiting the nonsmooth/smooth structure of the objective function, has been studied in [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF], and extended in [START_REF] Chouzenoux | Variable metric forward-backward algorithm for minimizing the sum of a differentiable function and a convex function[END_REF] to take in account a variable metric. Another splitting approach comes from Gauss-Seidel-like methods, which apply to functions with separated variables, and consist in doing a descent method relatively to each (block of) variable alternatively. See [START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF][START_REF] Xu | A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion[END_REF] for a proximal alternating method, and [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF] for a variable-metric version. Recent papers [START_REF] Xu | A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion[END_REF][START_REF] Bolte | Proximal alternating linearized minimization for nonconvex and nonsmooth problems[END_REF][START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF] propose to combine these two splitting approaches in order to exploit both the smooth/nonsmooth character and the separated structure of the function.

Let us now precise our contributions to this domain, which are presented in Part I. We consider a proper lower semi-continuous function f : H -→ R ∪ {+∞}, where H is a Hilbert space. Working in a Hilbert space setting allows to encompass the analytic functions, the functions definable in an o-minimal structure, or even some energies underlying some nonlinear PDE. Along all this part, we assume that f is a function satisfying the Kurdyka-Lojasiewicz inequality around each point x ∈ H. Such function will be called a K L function.

Chapter 3 contains the main theoretical results of Part I. We start in Section 3.1 by giving some keys for the understanding of the Kurdyka-Lojasiewicz inequality, and its implications in descent dynamical systems. After giving a proper definition of the Kurdyka-Lojasiewicz inequality in the nonsmooth case, we revisit the result of [START_REF] Bolte | The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF] on the asymptotic behaviour of (1.5). Under general hypotheses on f , and a compactness assumption on u, we recover the finite length for the trajectories of this dynamic. The proof of this result for the continuous dynamic is instructive, since it provides a sketch for the proof in the discrete case. We also prove in Theorem 3.1.12 general convergence rates for both the trajectory and its image, in the line of [START_REF] Chill | Convergence and decay rate to equilibrium of bounded solutions of quasilinear parabolic equations[END_REF].

In Section 3.2, we are interested in descent algorithms to solve

min x∈H f (x).
Most of the algorithms studied in the papers mentionned above share the same asymptotic behaviour: under a compactness assumption, the generated sequences converge strongly to critical points, and the affine interpolations have finite length. This is not surprising since the algorithms described in [START_REF] Attouch | On the convergence of the proximal algorithm for nonsmooth functions involving analytic features[END_REF][START_REF] Merlet | Convergence to equilibrium for the backward Euler scheme and applications[END_REF][START_REF] Bolte | Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity[END_REF][START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF][START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF][START_REF] Bolte | Proximal alternating linearized minimization for nonconvex and nonsmooth problems[END_REF], together with the ones of [START_REF] Chouzenoux | Variable metric forward-backward algorithm for minimizing the sum of a differentiable function and a convex function[END_REF][START_REF] Xu | A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion[END_REF] (without extrapolation step), fall into the general convergence result for abstract descent methods of Attouch, Bolte and Svaiter [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF]. Besides, these methods essentially share the same hypotheses on the parameters with the abstract method of [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF]: the step sizes (respectively the eigenvalues of the matrices underlying the metric) are required to remain in a compact subinterval of the positive numbers. Moreover, they have little flexibility regarding to the presence of computational errors. To our knowledge, vanishing step sizes (resp. unbounded eigenvalues) or sufficiently general errors have never been treated in the K L context. For these reasons, we introduce an abstract descent scheme, generating sequences (x k ) k∈N ⊂ H verifying for all k ∈ N:

a k x k+1 -x k 2 ď f (x k ) -f (x k+1 ), a k > 0, b k+1 ∂ F f (x k+1 ) -ď x k+1 -x k + ε k+1 , b k+1 , ε k+1 > 0.
This approach follows the ideas of [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF], that we adapt to allow more flexibility for the parameters, and to introduce an error term ε k+1 . Theorem 3.2.2 proves, under a compactness assumption, that any sequence generated by this scheme has finite length and converges strongly to a critical point of f . Then, provided that the sequence (x k ) k∈N is initialized close enough to a minimum of f , Theorem 3.2.3 ensures that the sequence converges to a minimum of the function. Such a capture result is of particular importance, since we work with functions being non necessarily convex. In section 3.3, we focus on the convergence rates of this abstract method. An interesting fact is that, in the literature, the convergence rates of several methods are essentially the same, and depend on the K L inequality rather than the nature of the algorithm. We give a theoretical basis for this statement by proving general convergence rates in Theorem 3.3.2, under the assumption that f satisfies the Kurdyka-Lojasiewicz inequality with a polynomial function ϕ. Furthermore, in Theorem 3.3.4, we prove new convergence rates when a more general function ϕ is involved. This result can be seen as the discrete counterpart of the convergence rates obtained for the continuous steepest descent in Theorem 3.1.12.

Chapter 4 contains the most practical aspects of Part I. We introduce in Section 4.1 a particular instance of the model defined in Chapter 3, which provides further insight into a large class of known methods, and present some innovative variants. More exactly, we revisit the Alternating Forward-Backward algorithm, already considered in [START_REF] Bolte | Proximal alternating linearized minimization for nonconvex and nonsmooth problems[END_REF][START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF][START_REF] Li | A proximal alternating linearization method for nonconvex optimization problems[END_REF], but allowing inexact computation of the iterates, and a dynamic choice of metric. After a description of the method in Section 4.1.1, we prove its convergence to critical points in Section 4.1.2. As a by-product of the results of Chapter 4, we obtain in Theorems 4.1.6 and 4.1.8 a global and local convergence result, respectively. In Section 4.1.3, we exploit the flexibility given by the Alternating Forward-Backward method to propose a new projected-Newton-like algorithm, for solving constrained optimization problems.

Section 4.2 is devoted to two particular applications of the Alternating Forward-Backward method. In Section 4.2.1 a nonconvex feasibility problem, involving both 'soft' constraints (like convex, smooth, or more generally prox-regular) and 'hard' constraints (no particular assumption is made). In Section 4.2.2, we discuss some problems arising in signal processing, such that the compressed sensing and the low-rank and sparse matrix decomposition problem. In each case, we use the flexibility of the Alternating Forward-Backward method to design algorithm for solving these problems. Some of the proposed methods are new, like the hard shrinkage projection algorithm that we numerically illustrate in Section 4.2.2.3.

Part II: Dynamics for vector optimization

The problems for which a decider have to manage simultaneously different costs are common in optimization: we want to maximize the correlation with respect to different data sets, together with the minimization of some energy, and being close to a desired state, etc. This kind of problem, called multi-objective (or multi-criteria) problem, arises a lot in shape optimization, where we have to design a shape submitted to different constraints [START_REF] Kreisselmeier | Application of vector performance optimization to a robust control loop design for a fighter aircraft[END_REF][START_REF] Lipej | Optimization method for the design of axial hydraulic turbines[END_REF][START_REF] Hasenjäger | Three dimensional evolutionary aerodynamic design optimization using single and multi-objective approaches[END_REF]. But it also appears in various other domains, such that engineering [START_REF] Petterson | Neural networks analysis of steel plate processing augmented by multi-objective genetic algorithms[END_REF], optimal control [START_REF] Liu | Multiobjective optimization and control[END_REF][START_REF] Gambier | Multi-objective optimal control: An overview[END_REF], game theory [START_REF] Maschler | Stable sets and stable points of set-valued dynamic systems with applications to game theory[END_REF], medical treatment [START_REF] Kiran | Multi-objective optimization of cancer immuno-chemotherapy[END_REF], finance [START_REF] White | Epsilon-dominating solutions in mean-variance portfolio analysis[END_REF][START_REF] Steuer | Portfolio selection in the presence of multiple criteria[END_REF], management, and more generally in decision sciences and operational research [START_REF] Bagchi | Simultaneous minimization of mean and variation of flow time and waiting time in single machine systems[END_REF][START_REF] Janssen | Multiobjective decision support for land-use planning[END_REF]. Historically, it is in economy that multicriteria equilibrium has been studied for the first time, in the context of the utility theory and welfare economics, introduced among others by Walras [START_REF] Walras | Éléments D'économie politique pure, L. Corbaz and Companie[END_REF] and Edgeworth [START_REF] Edgeworth | Mathematical psychics[END_REF]. Pareto [START_REF] Pareto | Manuate di economia politica[END_REF], and then Barone [START_REF] Barone | Il Ministero della Produzione hello Stato Colletivista[END_REF], gave the first definition of what we call nowadays a Pareto equilibrium, a state for which: "It must be impossible by any allocation of resources to enhance the welfare of one household without reducing that of another." E. Barone [START_REF] Barone | Il Ministero della Produzione hello Stato Colletivista[END_REF] The reader looking for more details on the historical development of these notions in mathematics and economy can consult the complete survey of Stadler [START_REF] Stadler | A survey of multicriteria optimization or the vector maximum problem, part I: 1776-1960[END_REF].

The mathematical formulation of this notion under its actual form is due to Debreu [START_REF] Debreu | Theory of value[END_REF][START_REF] Debreu | Valuation equilibrium and Pareto optimum[END_REF]. Basically, we consider a finite family {f 1 , ..., f m } of functions from R n to R, called objective functions, and we look for points x ∈ R n for which no objective function can be improved without penalizing an other. More precisely, points x for which there is no x ∈ R n such that f i (x) ď f i (x) for all i ∈ {1, ..., m} and (f 1 (x), ..., f m (x)) = (f 1 (x), ..., f m (x)).

Multi-objective optimization problems ((MOP) for short) are generally written under the form (MOP) MIN x∈R n (f 1 (x), ..., f m (x)).

These multi-objective optimization problems are part of a more general setting, the one of vector optimization problems:

(VOP) MIN x∈X F (x),
where F is a function from some vector space X to an ordered vector space (Y, ĺ Y ). In these problems, we look for the efficient points of (VOP), that is, the points x ∈ X for which there is no x ∈ X such that F (x) ĺ Y F (x) and F (x) = F (x).

The multi-objective problem can then be recovered by taking F := (f 1 , ..., f m ) : R n -→ R m , R m being equipped with the usual partial order ĺ R m induced by the positive orthant R m + . These vector equilibrium and optimization problems were introduced by Debreu [START_REF] Debreu | Theory of value[END_REF] to model the preferences of a decider, and have been the object of various studies (see the survey [START_REF] Dauer | A survey of vector optimization in infinite-dimensional spaces: part II[END_REF] and its references, or more recently [START_REF] Degiovanni | Critical point theory for vector valued functions[END_REF][START_REF] Mordukhovich | Variational analysis and generalized differentiation II: Applications[END_REF][START_REF] Bao | Sufficient conditions for global weak Pareto solutions in multiobjective optimization[END_REF]). From the applications point of view, there is few vector optimization problems which are not simply multi-objective, except in economy. Let us mention for instance some recent developments on equilibrium in financial markets [START_REF] Aliprantis | Equilibrium analysis in financial markets with countably many securities[END_REF][START_REF] Aliprantis | General equilibrium analysis in ordered topological vector spaces[END_REF].

When we face a multi-objective problem, one of the purposes is to detect the set of Pareto points. A well-known approach is the weighting method [START_REF] Zadeh | Optimality and non-scalar-valued performance criteria[END_REF][START_REF] Geoffrion | Proper efficiency and the theory of vector maximization[END_REF], which minimizes some convex combination of the objective functions. Its popularity comes from its easy implementation, since it reduces to a classical optimization problem. It works well in the convex setting, since it is known in that case that the minimizers of all the convex combinations cover the Pareto points [START_REF] The | Theory of vector optimization[END_REF]. Nevertheless, in the general nonconvex case, this method does not recover well the nonconvex parts of the Pareto front [START_REF] Das | A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set generation in multi-criteria optimization problems[END_REF][START_REF] Messac | Ability of objective functions to generate points on nonconvex Pareto frontiers[END_REF]. Moreover, we cannot use the weighting method if we look for a cooperative approach: for instance, in shape optimization, if we are given an initial shape, we might want to reach a Pareto equilibrium by improving all the involved criteria along time. But the objective functions corresponding to the weakest weightings will suffer from consequent variations. Furthermore, this method requires the choice of an a priori convex combination, which is not always easy when the objectives are not correlated.

Of course, other approaches exist, like the hierarchical minimization [START_REF] Fishburn | Exceptional paper-lexicographic orders, utilities and decision rules: A survey[END_REF] which first minimizes one objective function, then an other under the constraint of remaining in the minimizers of the first, and so on. This hierarchy asks to make a choice among the objectives, and it is easy to see on simple examples that the Pareto set cannot be entirely recovered like this. The ε-constraint method [START_REF] Haimes | On a bicriterion formulation of the problems of integrated system identification and system optimization[END_REF] relies more or less on the same idea: we minimize one objective, under a sublevel constraint concerning the other objective functions. Let us finally mention other empiric methods, called evolutive or genetic methods [START_REF] Fonseca | An overview of evolutionary algorithms in multiobjective optimization[END_REF][START_REF] Deb | Multi-objective genetic algorithms: problem difficulties and construction of test problems[END_REF][START_REF] Deb | A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II[END_REF], for which there is no theoretical guarantee of convergence .

We detail now some works published in the 70s, which influenced our own work and were precursor for multi-objective descent methods. These works have in common that they have been written for economical purposes, in particular for modeling allocation of resources in planned procedures.

Smale 2 is the first to be interested in a continuous dynamic promoting the simultaneous decrease of a finite family of objective functions {f 1 , ..., f m }, without introducing a subjective choice or combination [START_REF] Smale | Global analysis and economics I: Pareto optimum and a generalization of Morse theory[END_REF]. Its approach consists in defining a trajectory whose derivative is, at each point, a common descent direction for all objective functions f i . Given a convex constraint C ⊂ X = R n , he defines at each x ∈ C the cone3 of common descent directions D(x), as the intersection of the cones of admissible descent directions for each objective function f i , that is: D(x) := {d ∈ T C (x) | ∇f i (x), d < 0, ∀i ∈ {1, ..., n}}.

He defines then a Pareto critical point as a point x ∈ C ⊂ R n such that D(x) is empty. Roughly speaking, it is a point such that, at the first order, we cannot improve strictly all the objective functions simultaneously, and remain in the constraint. It is a weaker notion than the one of Pareto optimality (strictly weaker in general), and is well-known at this time [START_REF] Pareto | Manuate di economia politica[END_REF][START_REF] Samuelson | Foundations of economic analysis[END_REF][START_REF] Wan | On local Pareto optima[END_REF]. Note that this first-order optimality condition is exactly the usual notion of critical point that we are used to work with, when m = 1. Thus, Smale is interested in trajectories u : C -→ R n satisfying the following differential inclusion:

u(t) ∈ D(u(t)) if u(t) is not Pareto critical, {0} else.
As it can easily be verified, such trajectories make each objective function decrease along time. Moreover, this dynamic is defined in such a way that each stationary point is a Pareto critical point (and so, candidate to be a Pareto point). These results are effectively proved by Cornet [START_REF] Cornet | Monotone planning procedures and accessibility of Pareto optima[END_REF] some years later. This last author proves also, under some (restrictive) conditions, the existence of a smooth selection, that is, a function ψ : R n -→ R n verifying ψ(x) ∈ D(x) for all x ∈ R n , and such that u(t) = ψ(u(t)) admits a unique solution of class C 1 .

In its thesis, Cornet [START_REF] Cornet | Contributions to the mathematical theory of dynamical mechanisms of resource allocation[END_REF] defines explicitly a cooperative dynamic, which enters in the setting of Smale's, and generalizes to the multi-objective setting the classic steepest descent dynamic. This dynamic writes as follows:

(SD) u(t) + ( N C (u(t)) + co∇f i (u(t)) ) 0 = 0, where co∇f i (u(t)) denotes the convex hull of the family of gradients {∇f i (u(t))} i∈{1,...,m} , and we use the notation A 0 to indicate the element of minimal norm of a closed convex set A. Cornet observes that the vector field governing its dynamic can be interpreted as an analogue of the usual steepest descent direction. Indeed, if we note s(u) = -( N C (u(t)) + co∇f i (u(t)) ) 0 , then

s(u) s(u) = argmin d∈T C (u) d =1 max i∈{1,...,m} ∇f i (u), d ,
which generalizes a well-known property of the gradient in classic optimization. The author gives also a proof of the existence of global trajectories (based on the Kakutani-Fan fixed point theorem), together with their convergence to Pareto critical points when t → +∞, in the convex case. A very interesting aspect of this dynamic is that, at each instant t, there exists a convex combination (θ i (t)) i∈{1,...,m} such that

u(t) + N C (u(t)) + ∇ m i=1 θ i (t)f i (u(t)) = 0, m i=1 θ i (t) = 1, θ i (t) ě 0.
In other words, the dynamic behaves like the steepest descent dynamic associated to a convex combination of the objective functions, except that this combination evolves dynamically. The weighting is chosen endogenously by the dynamic, to promote the common decrease of the objective functions. This idea of extending the steepest descent dynamic to vector optimization problems appears also in the works of Malivert [START_REF] Malivert | A descent method for Pareto optimization[END_REF] and Pascoletti-Serafini [START_REF] Pascoletti | An iterative procedure for vector optimization[END_REF]. Nevertheless, there has been almost no works in this direction until the end of the century.

In the 2000s, in a paper of Fliege and Svaiter [START_REF] Fliege | Steepest descent methods for multicriteria optimization[END_REF], we observe a new interest for the steepest descent method for multi-criteria optimization problems. These authors propose an algorithm to solve an unconstrained smooth multi-objective optimization problem. For this, they define on X = R n the following vector field: This direction s(x) being a common descent 4 direction at x for each objective function f 1 , ..., f m , we can define a descent algorithm

x n+1 = x n + t n s(x n ),
where t n is a stepsize chosen by an Armijo-like rule. As we will see later in Chapter 5, s(x) is nothing but the steepest descent direction at x, in the sense of Cornet, that is s(x) = -( co∇f i (u(t)) ) 0 . Thus, we can see the algorithm of Svaiter and Fliege as a discretization in time of the steepest descent dynamic, and we recover (up to some differences) the same asymptotic behaviour.

In the next years, a series of papers will extend this steepest descent method to multiobjective optimization problems with constraint, taking into account relative errors, and/or adapting these methods to more general vector optimization problems [START_REF] Drummond | A projected gradient method for vector optimization problems[END_REF][START_REF] Drummond | A steepest descent method for vector optimization[END_REF][START_REF] Miglierina | Box-constrained multi-objective optimization, a gradient-like method without a priori scalarization[END_REF][START_REF] Bello Cruz | Convergence of the projected gradient method for quasiconvex multiobjective optimization[END_REF][START_REF] Bento | Unconstrained steepest descent method for multicriteria optimization on riemannian manifolds[END_REF][START_REF] Fukuda | Inexact projected gradient method for vector optimization[END_REF][START_REF] Bento | The self regulation problem as an inexact steepest descent method for multicriteria optimization[END_REF]. Since this steepest descent method extends the usual gradient descent method to the multi-objective case, some authors adapt other well-known optimization methods. Let us mention for instance the Newton method [START_REF] Fliege | Newton's method for multiobjective optimization[END_REF][START_REF] Graña Drummond | A quadratically convergent Newton method for vector optimization[END_REF], or quasi-Newton [START_REF] Qu | Quasi-Newton methods for solving multiobjective optimization[END_REF][START_REF] Povalej | Quasi-Newton's method for multiobjective optimization[END_REF], the interior point method [START_REF] Villacorta | An interior proximal method in vector optimization[END_REF][START_REF] Villacorta | A trust-region method for unconstrained multiobjective problems with applications in satisficing processes[END_REF], the proximal algorithm [START_REF] Bonnel | Proximal methods in vector optimization[END_REF][START_REF] Ceng | Approximate proximal methods in vector optimization[END_REF][START_REF] Ceng | Hybrid approximate proximal method with auxiliary variational inequality for vector optimization[END_REF][START_REF] Bento | A proximal point-type method for multicriteria optimization[END_REF], and subgradient methods [START_REF] Cruz Neto | A subgradient method for multiobjective optimization[END_REF][START_REF] Bento | A Subgradient Method for Multiobjective Optimization on Riemannian Manifolds[END_REF].

In parallel, other authors focus again on the continuous steepest descent dynamic. Schäffler-Schultz-Weinzierl [START_REF] Schäffler | Stochastic method for the solution of unconstrained vector optimization problems[END_REF] consider 5 a stochastic version of the steepest descent dynamic (SD). Through the notion of vector pseudo-gradient, Miglierina [START_REF] Miglierina | Slow solutions of a differential inclusion and vector optimization[END_REF] recovers the (SD) dynamic in the unconstrained case. At the same time, Recchionni [START_REF] Recchioni | A path following method for box-constrained multiobjective optimization with applications to goal programming problems[END_REF] introduces a dynamic close to (SD), considering an interior penalization for a box constraint. This author is the first to verify numerically that this steepest descent approach is efficient for recovering nonconvex Pareto fronts. Miglierina-Molho-Recchionni [START_REF] Miglierina | Box-constrained multi-objective optimization, a gradient-like method without a priori scalarization[END_REF] study a similar dynamic, and show that their approach is more efficient than the weighting method. More recently, Attouch-Goudou [START_REF] Attouch | A continuous gradient-like dynamical approach to Paretooptimization in Hilbert spaces[END_REF] revisit also (SD) in the convex case under a general convex constraint. They propose a new constructive proof of the existence of the trajectories, based on Peano's Theorem and a Moreau-Yosida regularization of the nonsmooth operator N C .

Let us now describe our main contributions and the contents of Part II.

In a first place, we studied the (SD) dynamic by considering the multi-objective optimization problem associated to Lipschitz continuous functions {f 1 , ..., f m }. The idea was to replace in (SD) the gradients of the objective functions by their Clarke subdifferentials:

(SD) u(t) + ( N C (u(t)) + co ∂ C f i (u(t)) ) 0 = 0. We studied this dynamic, but also the properties of the operator governing it, namely x -→ co ∂ C f i (x). This guided us to think about the links between this operator x -→ co ∂ C f i (x), and the family of objective functions {f 1 , ..., f m }. Indeed, why are involved the gradients (or subgradients) of all convex combinations (1.7)

f θ := m i=1 θ i f i | m i=1 θ i = 1, θ i (t) ě 0 ,
when we only look for the minimization of a finite family of functions? Why do we have to call this larger family of functions? By taking the more general point of view of vector optimization, we understood that the minimization of F = (f 1 , ..., f m ) : X -→ (R m , ĺ R m ) expresses naturally as the simultaneous minimization of the convex combinations in (1.7). Even better, we understood that the central role given to the objective functions {f 1 , ..., f m } is essentially due to the extremely favorable geometrical properties of the ordering cone R m + . This brought us to the study of descent dynamics associated to general vector optimization problems, which seems to be the good setting to work in.

Our aim, in the second part of this thesis, is to give a new look on vector optimization problems. For this, we place at the center of our analysis the notion of steepest descent direction, and the dynamic(s) that it induces. The setting in which we will work is the following: we consider a function F : X -→ Y between two Banach spaces, the space Y being equipped with a partial order noted ĺ. We will always assume that the order on Y is induced by a closed convex cone with nonempty interior. Given a nonempty closed convex set C ⊂ X which models the constraints, we study the optimization problem

(VOP) MIN x∈C F (x).
Taking X as a Banach space allows to cover situations appearing in decision science, engineering or economy, where the decision space can be taken as a space of integrable functions [START_REF] Radner | Efficiency prices for infinite horizon production programmes[END_REF][START_REF] Peleg | Efficiency prices in an infinite-dimensional space[END_REF]. Taking Y as an ordered Banach space allows of course to cover the muli-objective case, but also to consider functions with values in a space of continuous functions, or L ∞ ([0, 1], R). This might lead to new modelisations in economy, or applications in optimal control.

In Chapter 5, we study the notions of descent direction for F , and of solution(s) for (VOP). For this, we introduce in Section 5.1 the ordered (Clarke) subdifferential, ∂ C F : X ⇒ X * , which will give us some first-order local information on the behaviour of F with respect to the order on Y. When we take place in the multi-objective setting with F = (f 1 , ..., f m ), ∂ C F (x) reduces to co ∂ C f i (x), and we recover the operator introduced by Cornet. After giving some calculus rules and basic properties for this ordered subdifferential, we show in Section 5.2 that ∂ C F (x) has interesting properties for the analysis of (VOP):

• It verifies a Fermat's rule, giving a first-order necesssary condition for the solutions of (VOP), see Theorem 5.2.10.

• Like in the scalar case (Y = R), under some regularity conditions, ∂ C F (x) spans the normal cone of the sublevel set of F at x, see Theorem 5.2.12.

• When X is a Hilbert space, the ordered subdifferential gives us descent directions. It suffices to consider at each point x ∈ X the element of minimal norm of -∂ C F (x), see Propositions 5.2.18 and 5.2.8. In fact, this direction is the steepest, in the sense given by Theorem 5.2.20.

For this two last properties, the key point is that, for x ∈ X, the ordered subdifferential ∂ C F (x) is a subset of X * , related to the decision space X. This is in clear opposition with the usual differentiability notions, or their generalizations to the nonsmooth setting, which give a subset of L(X, Y) (see Remark 5.3.2 for more discussion on this subject).

In Chapter 6, we assume that X is a Hilbert space, and we use ∂ C F to define the steepest descent dynamic (SD) u(t) + (N C (u(t)) + ∂ C F (x)) 0 = 0, t ě 0.

In the smooth multi-objective case, we recover the dynamic introduced by Cornet. Furthermore, a simple discretization in time of (SD) gives the descent algorithms developed in [START_REF] Fliege | Steepest descent methods for multicriteria optimization[END_REF][START_REF] Drummond | A projected gradient method for vector optimization problems[END_REF][START_REF] Drummond | A steepest descent method for vector optimization[END_REF][START_REF] Miglierina | Box-constrained multi-objective optimization, a gradient-like method without a priori scalarization[END_REF][START_REF] Bello Cruz | Convergence of the projected gradient method for quasiconvex multiobjective optimization[END_REF][START_REF] Bento | Unconstrained steepest descent method for multicriteria optimization on riemannian manifolds[END_REF][START_REF] Fukuda | Inexact projected gradient method for vector optimization[END_REF][START_REF] Bento | The self regulation problem as an inexact steepest descent method for multicriteria optimization[END_REF], or even the proximal methods of [START_REF] Bonnel | Proximal methods in vector optimization[END_REF][START_REF] Ceng | Approximate proximal methods in vector optimization[END_REF][START_REF] Ceng | Hybrid approximate proximal method with auxiliary variational inequality for vector optimization[END_REF][START_REF] Bento | A proximal point-type method for multicriteria optimization[END_REF]. Being able to compare the continuous and discrete dynamics is always fruitful, since they often share the same asymptotic behaviour (see for instance the works of Peypouquet-Sorin [START_REF] Peypouquet | Evolution equations for maximal monotone operators: asymptotic analysis in continuous and discrete time[END_REF] and Alvarez-Peypouquet [START_REF] Alvarez | A unified approach to the asymptotic almost-equivalence of evolution systems without Lipschitz conditions[END_REF]).

We discuss into more details this comparison in Section 6.3.1. Given a global solution of (SD) (in a sense to precise), we prove in Proposition 6.1.7 that it has a descent property:

∀s, t ∈ [0, +∞[, s ď t ⇒ F (u(t)) ĺ F (u(s)).
Then, we study the asymptotic behaviour of a trajectory u(•) in Theorem 6.2.6. We prove in particular in the convex case that, when t goes to +∞, the trajectory u(t) weakly converges to a weak efficient point of (VOP). Weaker results are obtained in the more general quasi-convex setting. Section 6.3 is devoted to some numerical simulations of this dynamic in the multiobjective setting. In particular, we agree with [START_REF] Miglierina | Box-constrained multi-objective optimization, a gradient-like method without a priori scalarization[END_REF] which observe that the steepest descent dynamic is efficient to recover nonconvex Pareto fronts. The fact that a convex combination of the objective functions is chosen dynamically in (SD) seems to be an advantage, contrary to the weighting method. Chapter 7 is entirely devoted to the question of the existence of trajectories for (SD), in the nonsmooth convex case. Our proof relies essentially on two ingredients, like in [START_REF] Attouch | A continuous gradient-like dynamical approach to Paretooptimization in Hilbert spaces[END_REF]: a Moreau-Yosida regularization for ∂ C F , and Peano's theorem. In particular, because of Peano's existence theorem, we have to assume that X has finite dimension. We cannot hope to use the Cauchy-Lipschitz theorem, because the vector field governing (SD) is not locally Lipschitz continuous (see Remark 6.1.16). As a consequence, for now, the uniqueness of the trajectories remain an open problem. In Section 7.1, we give a first result of existence in the smooth case, using an abstract result from [START_REF] Attouch | A continuous gradient-like dynamical approach to Paretooptimization in Hilbert spaces[END_REF]. In Section 7.2, we prove the existence of trajectories of (SD) in the nonsmooth convex multi-objective case. In this favorable setting, we only deal with a finite number of objective functions, on which we can apply a Moreau-Yosida regularization. In Section 7.3, we focus on the general vector case, and the main difficulty lies in the definition of an appropriate regularization for ∂ C F . This technical question (which has its own interest) is addressed in Sections 7.3.1 and 7.3.2. This being done, we are able to prove Theorem 7.0.1 which establishes the existence of global trajectories for (SD).

Finally, in Chapter 8, we open a completely new road: an inertial approach to multiobjective optimization problems. Indeed, generating a set of Pareto points asks the simulation of a consequent number of trajectories (using the steepest descent dynamic (SD), or a weighting method). The problem is that these methods are quite slow to converge, and we look for fast methods, which arise in general by considering second-order methods. It already exists methods involving second-order informations in space, using the curvature (second derivatives) of the objective functions. These are the Newton (or quasi-Newton) methods for multi-criteria optimization [START_REF] Fliege | Newton's method for multiobjective optimization[END_REF][START_REF] Graña Drummond | A quadratically convergent Newton method for vector optimization[END_REF][START_REF] Povalej | Quasi-Newton's method for multiobjective optimization[END_REF]. Nevertheless, they can be expensive, due to the necessity to solve a quadratic problem at each iteration.

In scalar-valued optimization, inertial methods, which involve second-order information in time, are very popular because they are quick to converge, and easy to implement. In the same spirit than [START_REF] Polyack | Some methods of speeding up the convergence of iterative methods[END_REF][START_REF] Attouch | The heavy ball with friction method I, the continuous dynamical system[END_REF], we look for an inertial version of the steepest descent dynamic, called the Inertial Steepest Descent with Friction:

(ISDF) ü(t) + γ u(t) + ∂ C F (u(t)) 0 = 0,
where γ > 0 can be interpreted as a friction parameter. We restrict ourselves in Chapter 8 to the multi-objective case, and assume that F = (f 1 , ..., f m ) is convex and of class C 2 . We prove the existence of global trajectories for (ISDF) in Theorem 8.1.5, and study the asymptotic behaviour of its solutions. In particular, we show in Theorem 8.2.8 that the trajectories of (ISDF) weakly converge to weak Pareto points. We think that this first approach opens the road to future studies of Nesterov-like algorithms for multi-objective optimization problems.

We end this introduction with some practical comments on the structure of the thesis. Since it is separated in two independent parts, we chose not to write a general conclusion. Instead, we placed at the end of each chapter a small concluding section, in which we discuss in detail particular points of the chapter, or present some directions of research for the future.

This thesis resulted in two published papers [START_REF] Frankel | Splitting methods with variable metric for Kurdyka-Lojasiewicz functions and general convergence rates[END_REF][START_REF] Attouch | A dynamic gradient approach to Pareto optimization with nonsmooth convex objective functions[END_REF], and a preprint which should be submitted soon [START_REF] Attouch | Multiobjective optimization: an inertial dynamical approach to Pareto optima[END_REF].

The article [START_REF] Frankel | Splitting methods with variable metric for Kurdyka-Lojasiewicz functions and general convergence rates[END_REF] comes from a joined work with P. Frankel and J. Peypouquet. It has been split in two parts, covering the Sections 3.2 and 3.3 of Chapter 3, and Section 4.1 of Chapter 4. The rest of Chapters 3 and 4 are unpublished, and have been mostly written during the redaction of [START_REF] Frankel | Splitting methods with variable metric for Kurdyka-Lojasiewicz functions and general convergence rates[END_REF]. Most of the contents of Chapters 5 to 7 for the multi-objective optimization problems, results from a collaboration with H. Attouch and X. Goudou, published in [START_REF] Attouch | A dynamic gradient approach to Pareto optimization with nonsmooth convex objective functions[END_REF]. Chapter 8 comes from a joint work with H. Attouch, whose preprint should be submitted soon [START_REF] Attouch | Multiobjective optimization: an inertial dynamical approach to Pareto optima[END_REF]. The notions and results of Chapter 5 to 7, concerning the general vector optimization problem are new, in particular Section 7.3. Some developments for the multi-objective setting are also new, such as Sections 5.2.2 or 6.3.

Chapter 2

Variational analysis tools in Banach spaces

In this introducing chapter, we present the basic notions and tools which will be needed along this thesis. It is also the occasion to fix some notations. As much as we could, we avoided the proofs in this chapter, referring properly to some textbooks. If a result is given without reference, this means that its proof is putted into Appendix A. The idea is that the reader can quickly browse through this introductory chapter, and focus only on what he could not know.

The content of this chapter is the following: In Section 2.1 we recall classic properties of the convex sets and cones, which can be found in the textbooks of Aliprantis and Border [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF] and Fabian and al. [START_REF] Fabian | Banach space theory: The basis for linear and nonlinear analysis[END_REF]. In Section 2.2, we present some of the most standard tools used in nonsmooth optimization, such as the Fréchet, limiting or Clarke subdifferentials, and the corresponding tangent and normal cones. See the books of Clarke [START_REF] Clarke | Optimization and nonsmooth analysis[END_REF], Mordukhovich [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF] and Penot [START_REF] Penot | Calculus without derivatives[END_REF] for a wide account on this subject. Section 2.3 is devoted to the notions involved in vector optimization problems, in particular the important notion of base for a cone. The majority of the results there comes from the books of Dinh [START_REF] The | Theory of vector optimization[END_REF] and Jahn [START_REF] Jahn | Vector optimization: Theory, applications, and extensions[END_REF].

In this thesis, we will consider functions F : X -→ Y, where X will often be a Hilbert space. While in Part I we will consider extended-real-valued functions, in Part II we will deal with functions taking values in a general Banach spaces. This is why most of this introductory chapter considers objects in a Banach space. Moreover, we will need to state some results which applies for both the norm and weak topologies in X, or the weak * topology in X * . This is why, in this chapter, we will sometimes work in the general setting of a topological vector space (X, τ ). In the interest of the sanity of the reader, we will not recall here all the basic necessary notions about topological vector spaces, and refer for this material to the excellent book [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF].

Let us fix some notations which will be used throughout this thesis. Given a real Banach space (X, • X ), we note X * its topological dual space, which is also a Banach space once equipped with its operator norm • X * . If there is no ambiguity on the space we work with, we will just note x (resp. x * ) instead of x X (resp. x * X * ). We note B X and B X * the unit balls of X and X * , respectively.

The primal space X can be equipped with the weak topology w(X, X * ), which is the weakest (or coarser) topology that makes all the linear forms x * ∈ X * continuous. We will also equip the dual X * with the weak-star topology w * (X * , X), which is the restriction to X * of the pointwise convergence topology of R X . In all this document, we will write w * instead of w * (X * , X), and w instead of w(X, X * ).

If we have to mention some topological property (such as closedness, compactness, convergence, continuity, ...), we shall precise for what topology it refers to, except for the case of the norm topology. For instance, a set will be said weakly closed if it is closed for the weak topology. If we just mention that it is closed, it means that we implicitly refer to the norm topology of the ambient space.

Given a topological vector space (X, τ ), we will write cl τ A for the closure of A ⊂ X with respect to the topology τ . Whenever a sequence (x n ) n∈N is τ -convergent to some x, we note x n τ -→ n→+∞ x. For such topological space, we also note X * its topological dual, and note w * the corresponding weak * topology on X * . We shall use the fact that, in that case, the topological dual of (X * , w * ) is X.

Since we deal with weak topologies in Banach spaces, which are not sequential, we are going to use nets to characterize continuity or closedness. Generically, we will use nets (x α ) α∈A indexed by a directed set A. Sometimes, using for instance an argument involving weak * compactness in X * , we will make use of subnets. A subnet of (x α ) α∈A is (x f (β) ) β∈B , for some directed set B and a final monotone function f : B → A. To lighten the notations, we will abusively write (x β ) β∈B instead of (x f (β) ) β∈B when referring to such subnet.

Basic topology and differential calculus 2.1.1 Convex sets

Let (X, τ ) be a Banach space. We recall that C ⊂ X is said to be convex if

for all x ∈ C, y ∈ C, λ ∈ [0, 1], λx + (1 -λ)y ∈ C.
According to Hahn-Banach separation theorem [START_REF] Zalinescu | Convex analysis in general vector spaces[END_REF]Theorem 1.1.5], closed convex sets are exactly the weakly closed convex sets. In particular, for any convex set C ⊂ X, the norm closure and weak closure coincide: cl C = cl w C. An other consequence of the Hahn-Banach theorem is that the convex sets in X can be written as the intersection of all closed half-spaces which contain it [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Corollary 5.83]. When a set C is defined as a finite intersection of closed half-spaces, we say that C is a polyhedron.

Given A ⊂ X, its convex hull is defined by

(2.1) coA = m i=1 θ i a i | m ∈ N, (θ i ) ∈ ∆ m , (x i ) ⊂ A ,
where ∆ m denotes the unit simplex in R m

∆ m := θ = (θ i ) ∈ R m | m i=1 θ i = 1 and θ i ě 0 ∀i ∈ {1, ..., m} . If A = m i=1
A i , we just note its convex hull coA i instead of co A i admits the more simple representation (see [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Lemma 5.29]):

(2.2) coA i = m i=1 λ i a i | (λ i ) ∈ ∆ m , a i ∈ A i .
The convex hull of a finite set is called a polytope. Polytopes are always compact [5, Corollary 5.30], and coincide with polyhedrons whether X has finite dimension1 [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Theorem 7.79].

Are the topological properties of a set conserved after taking its convex hull? The answer is in general negative: Example 2.1.1. The convex hull of a closed set is not necessary closed. Take for instance

A = {(0, 0)} ∪ R × {1} in R 2 .
Example 2.1.2. The convex hull of a compact set is not necessary compact. Take for instance A = {0} ∪ {u 1 , u 2 , ...} in ℓ 2 (N), where u n = 1 n e n . Here, (e n ) n∈N is the usual canonical basis. Then it can be shown that A is compact but its convex hull is not even closed, for the norm topology (see [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Example 5.34]).

Nevertheless, the answer is yes in some cases. For instance the convex hull of a finite set (hence compact) is compact. More generally, if A is structured as a finite union of convex compact sets (see also [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Theorems 5.35 and 6.35]): Lemma 5.29]). Let (X, τ ) be a topological vector space. Suppose that

Proposition 2.1.3 ([5,
A = m i=1
A i with the A i being convex and compact. Then coA is compact.

We note coA the closed convex hull of A ⊂ X, defined as the closure of coA for the norm topology (or equivalently, the weak topology). It can also be defined as the intersection of all closed convex sets containing A. Equivalently, we note co * A := cl w * coA the weakly * closed convex hull of a set A ⊂ X * .

Given a nonempty A ⊂ X * , define the support function of A as

σ A : X -→ R ∪ {+∞} x -→ sup x * ∈A x * , x ,
where •, • denotes the dual pairing between X * and X. It is lower semi-continuous and sublinear, and even continuous if A is weakly * compact (see Proposition A.3.4). Clearly, σ A = σ co * A , but we have a stronger and very useful characterization:

Theorem 2.1.4. For all A, B ⊂ X * nonempty, σ A = σ B ⇔ co * A = co * B.
This theorem is a direct consequence of the bijection between weakly * closed convex sets and lower semi-continuous sublinear functions (see [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Theorem 7.51], or [START_REF] Clarke | Optimization and nonsmooth analysis[END_REF]Proposition 2.1.4]).

A last interesting notion about convex sets are the extreme points. Given a convex set C, we say that x ∈ C is an extreme point of C if C \ {x} is still a convex set. The Krein-Milman theorem [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Theorem 7.68] ensures that in a locally convex separated topological vector space, any convex compact set can be written as the closed convex hull of its extreme points. The polytopes are exactly the sets having a finite number of extreme points [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Lemma 7.76].

Cones and duality

A set K ⊂ X is said to be a cone if R + K ⊂ K, where R + := [0, +∞[. A set S ⊂ X is said to be a strict cone if R ++ S ⊂ S and 0 /
∈ S, where R ++ :=]0, +∞[. Once given an arbitrary set A, one can consider its conical hull as R + A. As before, a natural question is to know whether if a set remains closed by taking its cone hull (it is obviously false for the compact property since a nontrivial cone is always unbounded). It is a question of interest, see for instance a direct application in Theorem 5.2.12.

Example 2.1.5. Take X = R 2 and consider

A = {(x, y) ∈ R 2 + | xy = 1}. It is a closed set, but R + A = R 2 ++ ∪ {(0, 0)} is not.
Example 2.1.6. Take X = R 2 and take A as the closed disc of center (1, 0) and radius 1 in R 2 . It is a closed (and even compact) set, but

R + A = (R ++ × R) ∪ {(0, 0)} is not closed.
We see here that the conical hull of a closed set can fail to be closed. In Example 2.1.5, it comes from a lack of boundedness of A. In Example 2.1.6, it comes from the infinite number of rays arising from 0 ∈ bd A. In fact a positive result for compact sets can be obtained under a finiteness assumption (see [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Corollary 5.25]), or if 0 / ∈ A (the proof is left in Appendix):

Proposition 2.1.7. In a Hausdorff locally convex topological vector space (X, τ ), let A ⊂ X be a compact set not containing the origin. Then R + A is closed.

The end of this section is devoted to dual objects. Given a Banach space (X, • ) and a nonempty A ⊂ X, we define the polar set of A in X * by (2.3)

A * := {x * ∈ X * | x * , x ď 1 ∀x ∈ A}.
One sees immediately that A * is a weakly * closed convex set containing the origin.

Example 2.1.8. Let A be the unit ball B X of a normed space X, then A * is the unit ball

B X * of X * . Indeed, x * ∈ (B X ) * ⇔ sup x ď1 x * , x ď 1 ⇔ x * ď 1 ⇔ x * ∈ B X * .
If K is a cone, its polar is also a cone, called the polar cone, which can be equivalently defined by

(2.4)

K * := {x * ∈ X * | x * , x ď 0 ∀x ∈ K}.
It can be also useful to consider its dual cone K + , which is just the opposite of the polar cone

K + := -K * = {x * ∈ X | x * , x ě 0 ∀x ∈ K}.
When X is a Hilbert space, we say that a cone is self-dual whenever K = K + .

Example 2.1.9.

• If A is a closed linear subspace of a Hilbert space, then A * is its orthogonal.

• If A = R + × {0} in R 2 then A * = R -× R. • If A is the orthant cone R m + := {x ∈ R m | x i ě 0} then A * = -A. It is an example of self-dual cone.
• The cone of symmetric positive real matrices S m + (R) in M m (R), or the cone of almosteverywhere positive functions

L 2 + ([0, 1], R) in L 2 ([0, 1], R) are also examples of self-dual cones.
Given a set B ⊂ X * , we can naturally2 define its polar in X by (2.5)

B * := {x ∈ X | x * , x ď 1 ∀x * ∈ B}.
It is a natural question to compare a set A and its bipolar (A * ) * . For instance, a cone K ⊂ X is closed and convex if and only if K = (K * ) * . Equivalently, a cone K ⊂ X * is weakly * closed and convex if and only if K = (K * ) * (see [START_REF] Fabian | Banach space theory: The basis for linear and nonlinear analysis[END_REF]Theorems 3.38,3.45]).

Let us give a word on the particular case of (X, •, • ) being a Hilbert space. In that setting, a cone and its dual can be considered to live in the same space, and we can derive a direct sum decomposition result, due to Moreau [START_REF] Moreau | Décomposition orthogonale d'un espace hilbertien selon deux cones mutuellement polaires[END_REF] (see also [START_REF] Bauschke | Convex analysis and monotone operator theory[END_REF]Theorem 6.29]): Theorem 2.1.10. (Moreau) Let X be a Hilbert space and K ⊂ X a closed convex cone. Then, for all x ∈ X, there exists a unique couple

(x K , x K * ) ∈ K × K * such that (2.6) x = x K + x K * and x K , x K * = 0.
In that case, x K = proj K (x) and x K * = proj K * (x). In particular,

x 2 = proj K (x) 2 + proj K * (x) 2 .
In this Theorem, we used the projection operator proj : X ⇒ X, which is defined as follows: for a nonempty set A in a Banach space X and x ∈ X, the projection of x onto A is the set noted proj A (x) defined as proj A (x) := argmin a∈A xa .

If X is a Hilbert space and A ⊂ X is a nonempty closed convex set, then proj A (x) reduces to a single point, which is the situation occurring in Moreau's decomposition theorem.

We end with polyhedral cones in a topological space (X, τ ). A cone K is said to be polyhedral if there exists a finite family {x 1 , ..., x m } ⊂ X such that

K = R + co{a 1 , ..., a m }.
In other words, a polyhedral cone is the cone hull of a polytope. According to a well-known result (see [START_REF] Farkas | Theorie der einfachen Ungleichungen[END_REF][START_REF] Weyl | Elementare Theorie der konvexen Polyeder[END_REF][START_REF] Uzawa | A Theorem on convex polyhedral cones[END_REF][START_REF] Charnes | The strong Minkowski-Farkas-Weyl theorem for vector spaces over ordered fields[END_REF] or more recently [START_REF] Borwein | Convex analysis and nonlinear optimization: Theory and examples[END_REF]Theorem 5.1.7]), when X has finite dimension, a cone K is polyhedral if and only if it is a polyhedron. Equivalently, if there is a finite number of linear forms {x * 1 , ..., x * m } ⊂ X * such that

K = {x ∈ X | x * i , x ď 0, ∀i ∈ {1, ..., m}}.
Exploiting both definitions, it is easy to see that in the finite dimensional setting, a cone K is polyhedral if and only if its polar cone K * is polyhedral. Note that the finite dimension of X is guaranteed as soon as K is a polyhedral cone with a nonempty interior.

Differential and directional derivative

We set here some differentiability notions for a function F : X -→ Y, where X and Y are two Banach spaces.

We say that F is directionally derivable at x ∈ X in the direction d ∈ X if the following limit exists in (Y, • )

lim t↓0 F (x + td) -F (x) t .
In such a case, this limit is noted DF (x; d), and called the directional derivative of F at x in the direction d. We shall say that F is directionally derivable at x if DF (x; d) exists for all d ∈ X. We directly see from the definition that, if F : X -→ Y is directionally derivable at x, and A ∈ L(Y, Z) for some Banach space Z, then A • F is also directionally derivable at x, with

∀d ∈ X, D(A • F )(x; d) = A(DF (x; d)).
The function F is said to be Gateaux differentiable at x ∈ X if it is directionally derivable at x, and if the application DF (x) :

X -→ Y d -→ DF (x; d)
is linear and continuous (for the norm topologies of X and Y). In that case, we say that DF (x) is the differential (or Gateaux derivative) of F at x. We will also note

D * F (x) ∈ L(Y * , X * ) the adjoint of DF (x). If F is Gateaux differentiable at each point of an open set U ⊂ X, we can consider the differential of F DF : U -→ L(X, Y).
We say that F is strictly Gateaux differentiable at x if there exists A ∈ L(X, Y) which satisfies the stronger property lim x→x t↓0

F (x + td) -F (x) t = A(d).
In that case, A is unique and equals DF (x). Since we ask the point x to 'move' around x, we can interpret this stronger notion as a form of continuity of DF (•, d) around x. More precisely, we have:

Proposition 2.1.11. Let F : X -→ Y be Gateaux differentiable on an open set U ⊂ X.
Then, for all x ∈ U , the following is equivalent:

i) F is strictly Gateaux differentiable at x, ii) DF : U -→ L(X, Y)
is pointwise continuous at x, i.e. continuous with respect to the norm topology of X and the pointwise3 topology of L(X, Y). In other words, for all converging net (x α ) α∈A ⊂ U converging to x, we have DF (x α ; d)

• Y --→ α∈A DF (x; d) for all d ∈ X.
We will also use the differentiability in the sense of Fréchet. F is said to be Fréchet differentiable at x if there exist a continuous operator

A ∈ L(X, Y) such that lim x→x F (x) -F (x) -A(x -x) x -x = 0. Similarly, F is strictly Fréchet differentiable at x if there exists A ∈ L(X, Y) such that (2.7) lim x→x x ′ →x F (x) -F (x ′ ) -A(x -x ′ ) x -x ′ = 0.
In both cases, F is Gateaux differentiable and the operator A is equal to the differential DF (x).

Here also the strict Fréchet differentiability is somehow equivalent to the continuity of DF :

Proposition 2.1.12. Let F : X -→ Y be Fréchet differentiable on an open set U ⊂ X. Then, for all x ∈ U , the following is equivalent:

i) F is strictly Fréchet differentiable at x, ii) DF : U -→ L(X, Y)
is strongly continuous at x, i.e. continuous with respect to the norm topology of X and the usual operator norm topology of L(X, Y).

We say that a function

F : U ⊂ X -→ Y is of class C 1 if it is Gateaux differentiable on U and DF : U -→ L(X, Y
) is strongly continuous. According to the Proposition above, functions of class C 1 on U are exactly the strictly Fréchet differentiable functions on U . We say moreover that a function is of class C 1,1 on U if its derivative DF : U -→ L(X, Y) is Lipschitz continuous. Functions of class C 1,1 enjoy the following property, very useful when studying descent methods (see Chapter 4). Its proof can be found for instance in [START_REF] Peypouquet | Convex optimization in normed spaces. Theory, methods and examples[END_REF]Lemma 1.30].

Lemma 2.1.13. [Descent lemma] Let U be an open subset of X. Let f : X -→ R be a function of class C 1 , its derivative being L-Lipschitz continuous on U . Then, for all x 1 , x 2 ∈ X,

f (x 2 ) -f (x 1 ) -Df (x 1 ), x 2 -x 1 ď L 2 x 2 -x 1 2 .

Nonsmooth analysis for extended-real-valued functions

All along this section, X is a Banach space and we consider a function f :

X -→ R ∪ {+∞}. The domain of f is dom f := {x ∈ X | f (x) ∈ R}.
We say that f is proper

whenever dom f = ∅. The epigraph of f is epi f := {(x, λ) ∈ X × R | f (x) ď λ}.
A convex function is a function for which the epigraph is convex. Equivalently, a function f

satisfying ∀x, x ′ ∈ X, ∀t ∈ [0, 1], f (tx + (1 -t)x ′ ) ď tf (x) + (1 -t)f (x ′ ).
For A ⊂ X, we define the Lipschitz modulus of f on A by Lip(f, A) := sup

x =x ′ ∈A |f (x ′ ) -f (x)| x ′ -x ∈ [0, +∞].
We say that f is Lipschitz continuous on A if Lip(f, A) < +∞. If A = X, F is said to be globally Lipschitz continuous. We say that f is locally Lipschitz continuous at x ∈ X if there exists some neighborhood U of x on which f is Lipschitz continuous. In that case, we will note Lip(f, x) instead of Lip(f, U ).

Fenchel analysis

Given a nonempty set Ω ⊂ X and x ∈ Ω, we define respectively the radial tangent cone and the admissible tangent cone by

T r Ω (x) := R + (Ω -x) = {d ∈ X | ∃t > 0, x + td ∈ Ω}, T ad Ω (x) := {d ∈ X | ∃ǫ > 0, ∀t ∈]0, ǫ[, x + td ∈ Ω}.
An equivalent way to define the radial cone T r Ω (x) is to see it as the cone hull of Ωx:

T r Ω (x) = R + (Ω -x).
According to Proposition 2.1.7 there is no particular reason for T r Ω (x) to be closed, since 0 ∈ Ωx.

Example 2.2.1.

• Let Ω = R 2 + ∪ R 2 -⊂ R 2 , then T r Ω ((0, 0)) = T ad Ω ((0, 0)) = Ω.
In particular these cones can fail to be convex.

• Let Ω = {(x, y) ∈ R 2 | (x -1) 2 + (y) 2 ď 1} be a disc in the plane. Then T r Ω ((0, 0)) = T ad Ω ((0, 0)) = {(x, y) ∈ R 2 | x > 0}.
We see here that these cones are not closed in general, even if Ω is closed.

• Let Ω = {(x, y) ∈ R 2 | (x -1) 2 + (y) 2 = 1} be a circle in the plane. Then T r Ω ((0, 0)) = {(x, y) ∈ R 2 | x > 0}, while T ad Ω ((0, 0)) = ∅.
Hence, these cones doesn't coincide in general.

It is clear that T ad Ω (x) ⊂ T r Ω (x) always holds, and the equality between these cones will characterize a geometrical property of Ω, as shown below (see Proposition A.2.1 in Appendix A.2). We say that Ω is radial (or absorbing) at x if for all y ∈ Ω, there exists a neighborhood

U of x such that [x, y] ∩ U ⊂ Ω holds. Proposition 2.2.2. Let x ∈ Ω ⊂ X. Then T ad Ω (x) = T r Ω (x) if and only if Ω is radial at x. In particular T ad C (x) = T r C (x) for all x ∈ C when C is convex.
Given the radial cone, we define the Fenchel tangent cone T Ω (x) as the closure of the radial cone T r Ω (x), with respect to the norm topology of X. In other words,

T Ω (x) := cl R + (Ω -x).
We also define the Fenchel normal cone N Ω (x) as the polar cone of T Ω (x), which is convex and weakly * closed. It is quite immediate from the definitions to see that this normal cone admits the equivalent formulation, involving an obtuse angle property:

(2.8)

N Ω (x) = {x * ∈ X * | ∀y ∈ X, x * , y -x ď 0}.
Consider now a function f : X -→ R ∪ {+∞}. Given x ∈ dom f and d ∈ X, we define the lower Dini directional derivative of f at x in the direction d by

d D -f (x; d) := lim inf t↓0 f (x + td) -f (x) t ,
which can possibly take infinite values.

We are now in position to define the Fenchel subdifferential of f at x ∈ dom f . For this, consider the following sets:

i) {x * ∈ X * | ∀y ∈ X, f (y) -f (x) -x * , y -x ě 0}, ii) {x * ∈ X * | x is a global minimum of y → f (y) -x * , y }, iii) {x * ∈ X * | (x * , -1) ∈ N epi f (x, f (x))}, iv) {x * ∈ X * | ∀d ∈ X, x * , d ď d D -f (x; d)}.
Hence the first three sets always coincide, and we call it the convex subdifferential of f at x, noted ∂f (x). Moreover, the fourth set always contains ∂f (x), with equality when f is convex (see Appendix A.2). We pose ∂f (x) = ∅ when x / ∈ dom f , and define the domain of ∂f as

dom ∂f := {x ∈ X | ∂f (x) = ∅}.
Remark 2.2.3. These different presentations of ∂f (x) reunify almost all of the different classical approaches to build a subdifferential in a nonconvex setting. In general, we will replace a global property (characteristic of convex functions) by a local one.

The first set describes the elements of ∂f (x) as the slope of some global exact affine minorant. In the nonconvex setting, we will drop the global assumption and ask for some local/asymptotically affine minorant (Fréchet [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF], Lipschitz-Smooth [START_REF] Aussel | Mean value property and subdifferential criteria for lower semicontinuous functions[END_REF] subdifferentials). Instead of asking for exact affine minorant, we can also consider affine minorant, which are exact up to a small constant ǫ, leading to the so-called approximate subdifferentials (approximate Fréchet [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF], Hadamard subdifferentials [START_REF] Ioffe | Proximal analysis and approximate subdifferentials[END_REF]).

In the second set, we can see the viscosity approach: we take x * as a subgradient at x if and only if there exists some 'smooth' function g defined in the neighbourhod of x satisfying ∇g(x) = x * and x is a local minimum of fg. For the convex subdifferential, we restrict ourselves to linear functions, but with this general approach we could consider a broader class of functions. The most important point is the following: what do we mean by 'smooth' function? We can take for definition the Fréchet differentiability, or the Gateaux differentiability, but more generally we can consider differentiability relatively to a given bornolgy. This construction leads to the family of the so-called viscosity subdifferentials, sometimes called bornology subdifferentials or β-subdifferentials (in particular the Fréchet or Gateaux subdifferentials, see [START_REF] Ioffe | Metric regularity and subdifferential calculus[END_REF]).

The third set corresponds to what we call a geometric subdifferential. We built a geometric tool which gives local informations of a set, namely a normal cone, and we derive local property of f by looking at the local property of its epigraph in X × R. Here the convex normal cone is built as the polar cone of a tangent cone, but one can consider other ways to define it, in particular if one wants a normal cone not necessarily convex. It can be defined through a local obtuse angle property like in (2.8) (see the Fréchet or Lipschitz-Smooth normals), or using the subdifferential of the distance function (see the Clarke-Rockafellar normals [START_REF] Clarke | Optimization and nonsmooth analysis[END_REF], and also the proximal normals in Hilbert spaces [START_REF] Ioffe | Metric regularity and subdifferential calculus[END_REF]).

The fourth set corresponds to what could be called an analytic subdifferential. We compute some directional derivative and take ∂f (x) as the set of its linear minorants. See the family of Dini derivative [START_REF] Aussel | Mean value property and subdifferential criteria for lower semicontinuous functions[END_REF], Hadamard/Bouligand derivative [START_REF] Ioffe | Metric regularity and subdifferential calculus[END_REF], or the Clarke-Rockafellar derivative [START_REF] Clarke | Optimization and nonsmooth analysis[END_REF][START_REF] Aussel | Normal cones to sublevel sets: an axiomatic approach[END_REF].

From its definition, the Fenchel subdifferential is clearly convex and weakly * closed, and satisfies the following Fermat's rule:

x is a global minimum of f ⇔ 0 ∈ ∂f (x).
Moreover, the Fenchel subdifferential is consistent with the Fenchel normal cone. Indeed, for any set Ω ⊂ X and x ∈ X, the Fenchel normal cone N Ω (x) is exactly ∂δ Ω (x), the Fenchel subdifferential of δ Ω the indicator function of Ω. It is also an easy exercise to show the following closure properties of the graph of ∂f :

X ⇒ X * . Proposition 2.2.4. Let f : X -→ R ∪ {+∞} be lower semi-continuous at x ∈ X. Let (x α , x * α ) α∈A be a bounded net in X × X * such that x α • --→ α∈A x and x * α w * --→ α∈A x * with x * α ∈ ∂f (x α ).
Then x * ∈ ∂f (x). Suppose now that f :

X -→ R ∪ {+∞} is weakly lower semi-continuous at x ∈ X. If (x α , x * α ) α∈A is a bounded net in X × X * such that x α w --→ α∈A x and x * α • --→ α∈A x * with x * α ∈ ∂f (x α ), then x * ∈ ∂f (x).
Remark 2.2.5. In the nonconvex setting, a common practical consists in enlarging a given subdifferential by closing its graph, sequentially or topologically (see for instance the limiting Fréchet/Mordukhovitch subdifferential [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF], or the limiting Dini/Approximate subdifferential [START_REF] Ioffe | Proximal analysis and approximate subdifferentials[END_REF]). These limiting subdifferentials are in general more robust and have better calculus rules, see [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF][START_REF] Penot | Calculus without derivatives[END_REF].

The Fenchel subdifferential is essentially useful if f is convex 4 , in which case it satisfies the most interesting properties. For instance if f is convex, then the Gateaux differentiability at

x ∈ int dom f implies ∂f (x) = {DF (x)}.
The reverse is also true as soon as we assume f being continuous at x [325, Theorem 2.4.

x ∈ X, ∂f (x) + ∂g(x) ⊂ ∂(f + g)(x).
If moreover f is continuous at some x ∈ dom g, then

∂f (x) + ∂g(x) = ∂(f + g)(x).
In the convex case, we also have a nice relationship between the Fenchel subdifferential and the sublevel sets (see [START_REF] Penot | Calculus without derivatives[END_REF]Proposition 5.48] together with Proposition 2.1.7):

Proposition 2.2.7. Let f : X -→ R be a continuous convex function. Let x ∈ X be such that 0 / ∈ ∂f (x), and note [f ď f (x)] := {x ∈ X | f (x) ď f (x)} be the sublevel set of f at x. Then, N [f ďf (x)] (x) = R + ∂f (x).
Remark 2.2.8. The fact that the Fenchel subdifferential is normal to the sublevel sets is important in Hilbert spaces, since it helps to find descent directions. For instance, we can show that the element of minimal norm of -∂f (x) is a descent direction for f at x (see Chapter 5).

In the nonconvex setting, there has been some attempts to define a subdifferential satisfying this relationship with the sublevel sets. This is the approach in [START_REF] Borde | Continuity properties of the normal cone to the level sets of a quasiconvex function[END_REF][START_REF] Aussel | Normal characterization of the main classes of quasiconvex functions[END_REF][START_REF] Aussel | Normal cones to sublevel sets: an axiomatic approach[END_REF], and it is well adapted to the study of quasiconvex functions (whose sublevel sets are convex, see Section 2.2.5).

We will introduce in Chapter 5 a subdifferential for functions with values in ordered vector spaces, which satisfies this same nice property.

Bouligand analysis

Given a nonempty set Ω ⊂ X, we define the Bouligand tangent cone (also called contingent tangent cone) by: (2.9)

T B Ω (x) := {d ∈ X | ∃d n → d, ∃t n ↓ 0, x + t n d n ∈ Ω} if x ∈ Ω, T B Ω (x) = ∅ else.
It is a closed cone (see Proposition A.2.4), but not necessarily convex 5 . We compare the Bouligand tangent cone to the previously seen radial, admissible and Fenchel tangent cones:

Proposition 2.2.9. Let x ∈ Ω ⊂ X. Then: i) cl T ad Ω (x) ⊂ T B Ω (x) ⊂ cl T r Ω (x) = T Ω (x),
ii) If Ω is radial at x, then the above inclusions become equalities. In particular

T B Ω (x) = T Ω (x).
Given the Bouligand tangent cone, we can introduce the corresponding Bouligand normal cone by taking its polar cone:

N B Ω (x) := {x * ∈ X * | x * , d ď 0 ∀d ∈ T B Ω (x)}. (2.10)
It is clearly a weakly * closed convex cone in X * . Observe that, since T B Ω (x) is not convex, the polar cone of N B Ω (x) is generally strictly bigger than T B Ω (x). Now, through a classic 'geometric' procedure using the normal cone to the epigraph, we define the Bouligand subdifferential of any function f : X → R ∪ {+∞} at x ∈ X as:

(2.11) ∂ B f (x) := {x * ∈ X | (x * , -1) ∈ N B epi f (x, f (x))} if x ∈ dom f, ∂ B f (x) = ∅ else. 5 Take Ω = R 2 + ∪ R 2 -⊂ R 2 , such that T B Ω ((0, 0)) = Ω.
It is a weakly * closed convex set, and its definition is consistent with the previously defined Bouligand normal cone, i.e. N B Ω (x) = ∂ B δ Ω (x) (see [START_REF] Penot | Calculus without derivatives[END_REF]Proposition 4.13]). Moreover, according to Proposition 2.2.9, the Bouligand subdifferential coincides with the Fenchel one whenever f is a convex function.

Define now the Bouligand directional derivative (also called contingent, lower Hadamard or lower Dini-Hadamard directional derivative) of f at x ∈ dom f in the direction d ∈ X:

(2.12) d B f (x; d) := liminf d ′ →d, t↓0 f (x + td ′ ) -f (x) t .
From its definition, d B f (x; •) is lower semi-continuous, but not convex in general 6 . As we can expect from the notations, this partial derivative is directly related to the previous Bouligand subdifferential (see [START_REF] Penot | Calculus without derivatives[END_REF]Corollary 4.15]):

(2.13)

∂ B f (x) = {x * ∈ X | ∀d ∈ X, x * , d ď d B f (x; d)}. Remark 2.2.10. The property ∂ B f (x) = {x * ∈ X | ∀d ∈ X, x * , d ď d B f (x; d)}
x * ∈∂ B f (x) x * , d ď d B f (x; d).
Equality instead of inequality in (2.14) would be called analytic consistency, but it does not hold in general. Take for instance f :

x ∈ R → -|x| ∈ R, then d B (0, d) = -|d| while ∂ B f (0) = ∅, so sup x * ∈∂ B f (0)
x * , d = -∞. In the light of [ The Bouligand subdifferential enjoys the following chain rule, which will be useful in our future study of dynamical systems (see Proposition A.2.5): Proposition 2.2.11. Let u : I ⊂ R -→ X and f • u : I -→ R ∪ {+∞}, where I is an open interval of R. Suppose that both u and f • u are derivable at t ∈ I, and that u(t)

∈ dom ∂ B f . Then, (f • u) ′ (t) = x * , u(t) , ∀x * ∈ ∂ B f (u(t)).

Fréchet analysis

Given a function f :

X -→ R ∪ {+∞}, its Fréchet subdifferential at x ∈ dom f is defined as the set ∂ F f (x) of elements x * ∈ X * such that : (2.15) lim inf x→x x =x f (x) -f (x) -x * , x -x x -x ě 0.
As usual its domain dom ∂ F f is the (possibly strict or empty) subset of dom f of points x such that ∂ F f (x) is nonempty. In the particular case of the indicator function of a closed set Ω ⊂ H, we denote by N F Ω := ∂ F δ Ω the Fréchet normal cone of F , which reduces to

N F Ω (x) =      x * ∈ X * | Limsup x Ω --→ x =x x x * , x -x x -x ď 0      .
For all x ∈ dom f , ∂ F f (x) is convex and closed for the norm topology, maybe empty 7 .

It is easy to see from the definitions that the Fenchel subdifferential ∂f is included in the Fréchet one ∂ F f , and equality holds between them if f is convex [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF]Theorem 1.93]. Now, by taking x = x + td ′ in (2.15), we easily obtain the following dual relation between the Fréchet subdifferential and Bouligand derivative: (2.16) for all d ∈ X, sup

x * ∈∂ F f (x) x * , d ď d B f (x; d).
We deduce directly from this inequality and Proposition 2.13 that, for all x ∈ X,

∂ F f (x) ⊂ ∂ B f (x).
In fact, Fréchet and Bouligand subdifferential coincide if X has finite dimension8 [290, Proposition 6.5].

Let us give some calculus rules for this Fréchet subdifferential. We first observe that this subdifferential is related to the Fréchet differentiability. Indeed, for f :

X -→ R ∪ {+∞} Fréchet differentiable at x we have ([212, Proposition 1.1]) ∂ F f (x) = {Df (x)}.
We now look at a sum rule. Given two functions f, g : X -→ R ∪ {+∞}, we have for all x ∈ X

∂ F (f + g)(x) ⊃ ∂ F f (x) + ∂ F g(x).
This comes immediately from the definition of ∂ F (see also [START_REF] Penot | Calculus without derivatives[END_REF]Proposition 4.34]). If we assume moreover that f is Fréchet differentiable at x ∈ dom g, then we have equality [START_REF] Penot | Calculus without derivatives[END_REF]Corollary 4.35])

∂ F (f + g)(x) = {Df (x)} + ∂ F g(x).
We introduce now the limiting Fréchet subdifferential (or just limiting subdifferential ) by taking the sequential closure of the graph of ∂ F f . More precisely, given f : X -→ R ∪ {+∞}, it consists in the set ∂ L f (x) of elements x * ∈ X * for which there exists:

• a sequence (x n ) n∈N in X * such that x n f --→ n→+∞ x with x n ∈ dom ∂ F f , • a sequence (x * n ) n∈N in X * such that x * n w * --→ n→+∞ x * with x * n ∈ ∂ F f (x n ).
We used in this definition the notation

x n f --→ n→+∞
x, which denotes the f -attentive convergence of x n to x, and means that

x n • --→ n→+∞ x, together with f (x n ) R --→ n→+∞ f (x).
As previously, we define the domain dom ∂ L , and the related limiting normal cone N L Ω := ∂ L δ Ω . The limiting subdifferential ∂ L f (x) is not necessarily weakly * closed9 neither convex 10 . Clearly, the Fréchet subdifferential is included in the limiting one. Using Proposition 2.2.4, we also see that, if f is a lower semi-continuous convex function, then its limiting subdifferential equals to the Fenchel subdifferential.

Remark 2.2.12. It is important to notice that the Limiting subdifferential introduced here is not the same than the one used for instance in the book of Mordukhovich [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF]. In that case, it involves approximate Fréchet subgradients in its definition, but both constructions coincide in reflexive spaces [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF]Theorem 2.34], which is far enough for our needs in Part I.

If f is Fréchet differentiable at x, then DF (x) ∈ ∂ L f (x), but the limiting subdifferential can be strictly larger (see [290, p. 304]). This is essentially because ∂ L f (x) "collects" some information on f around x and not at x. The strict Fréchet differentiability of f at x guarantees that ∂ L f (x) = {DF (x)} (see [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF]Corollary 1.82]). In the same spirit, we have the following sum rule: Proposition 2.2.13 ([244, Proposition 1.107]). Let f, g : X -→ R ∪ {+∞}, with X being reflexive. Assume that f is strictly Fréchet differentiable at x ∈ dom g. Then,

∂ L (f + g)(x) = {Df (x)} + ∂ L g(x).

Clarke analysis

Given a locally Lipschitz continuous function f : X -→ R, its Clarke directional derivative at x ∈ X in the direction d ∈ X, is given by

(2.17) d C f (x; d) = limsup x→x, t↓0 f (x + td) -f (x) t .
Since f is locally Lipschitz continuous, for each x ∈ X the function d

C f (x; •) : X → R
∂ C f (x) = {x * ∈ X * | x * , d ď d C f (x; d) ∀d ∈ X}. From (2.
d C f (x; d) = max { x * , d | x * ∈ ∂ C f (x)}.
This max formula can be easily deduced from Theorem 2.1.4, see also Remark 2.2.10.

Let us give some calculus rules for the Clarke subdifferential. First of all, using basic properties of the lim sup, we see that for two locally Lipschitz functions f, g : X -→ R, it always holds (2.20) for all x ∈ X and all d ∈ X,

d C (f + g)(x; d) ď d C f (x; d) + d C g(x; d).
By duality in (2.18), we also have

(2.21) for all x ∈ X, ∂ C (f + g)(x) ⊂ ∂ C f (x) + ∂ C g(x).
Similarly to the limiting subdifferential, the Clarke subdifferential is sensitive to the strict Gateaux differentiability [111, Proposition 2.2.4]:

∂ C f (x) = {x * } ⇔ f is strictly Gateaux differentiable at x and Df (x) = x * .
The Clarke subdifferential enjoys also an exact sum rule under a strict differentiability assumption. If f, g : X -→ R are two locally Lipschitz functions such that f is strictly Gateaux differentiable at x ∈ X, then [111, Proposition 2.3.3]:

(2.22)

∂ C (f + g)(x) = {Df (x)} + ∂ C g(x).

A last few things

Let us make some links between the subdifferentials we presented below. First note that, if f : X -→ R is locally lipschitz continuous, we have for all x ∈ X, and all d ∈ X, lim inf

d ′ →d t↓0 f (x + td ′ ) -f (x) t = lim inf t↓0 f (x + td) -f (x) t ,
in other words,

(2.23) d B f (x; d) = d D -f (x; d).
Since we can easily compare lower Dini derivatives and Clarke derivatives, we deduce using (2.18) and (2.13) that in the locally Lipschitz case,

(2.24) d B f (x; •) ď d C f (x; •) and ∂ F f (x) ⊂ ∂ B f (x) ⊂ ∂ C f (x).
In the Lipschitz case, we can also bound these subdifferentials [111, Proposition 2.1.1]:

Proposition 2.2.14. Let f : X -→ R be locally Lipschitz continuous at x. Then

∂ F f (x) ⊂ ∂ B f (x) ⊂ ∂ C f (x) ⊂ Lip(f, x)B.
We say that a locally Lipschitz continuous function f is Clarke regular at x if f is directionally derivable at x, and for all d ∈ X,

Df (x; d) = d C f (x; d).
Using (2.23) and (2.24), we see that f is Clarke regular at x if and only if

d B f (x; •) = d C f (x; •), which is also equivalent to ∂ B f (x) = ∂ C f (x).
We easily see in this Lipschitz case that strictly Gateaux differentiable functions are Clarke regular. But it also true for convex functions, and any positive linear combination of Clarke regular functions [START_REF] Clarke | Optimization and nonsmooth analysis[END_REF]Proposition 2.3.6]. Now, let f : X -→ R ∪ {+∞} be an arbitrary function. In [307, Theorem 6.1], Treiman showed11 that for all x ∈ X,

(2.25) co * ∂ L f (x) ⊂ ∂ C f (x),
the inclusion being an equality if X is reflexive. This implies in particular that

∂ F f (x) ⊂ ∂ L f (x) ⊂ ∂ C f (x).
An other concept of regularity is the lower regularity. We say that a function f :

X -→ R is lower regular at x if ∂ F f (x) = ∂ L f (x).
If X is reflexive and f locally Lipschitz, with the representation formula (2.25) we can deduce that the lower regularity at x is equivalent to ask

∂ F f (x) = ∂ C f (x).
In that case, lower regularity implies Clarke regularity. Moreover, if X has finite dimension, lower regularity and Clarke regularity coincide, because Bouligand and Fréchet subdifferentials both coincide. Equivalently, a closed set Ω ⊂ H is said lower regular at x if N F Ω (x) = N L Ω (x). For instance (still with X being reflexive), C 2 manifolds, prox-regular sets, strictly Fréchet differentiable functions, primal-lower-nice functions [START_REF] Correa | Subdifferential representation formula and subdifferential criteria for the behavior of nonsmooth functions[END_REF]Corollary 3.1] are lower regular.

It follows directly from its definition that the Fréchet subdifferential enjoys this Fermat's rule:

if x is a local minimum of f, then 0 ∈ ∂ F f (x).
Of course, since ∂ F is included in all other subdifferentials, the limiting and Clarke subdifferentials enjoy also the same Fermat's rule.

We end this section with a few words about quasi-convex functions. A function f :

X -→ R ∪ {+∞} is said to be quasi-convex if for all x ∈ dom f , the sublevel set [f ď f (x)] is convex. Equivalently, ∀x, x ′ ∈ dom f, ∀t ∈ [0, 1], f (tx + (1 -t)x ′ ) ď max{f (x), f (x ′ )}
These functions enjoy the following nice property, which generalizes a common fact for convex functions (see Appendix A.2):

Proposition 2.2.15. Let f : X -→ R ∪ {+∞} be a quasi-convex function such that f (x 2 ) ď f (x 1 ). Then x * , x 2 -x 1 ď 0 for all x * ∈ ∂ B f (x 1 ) ⊃ ∂ F f (x 1 ).
A similar property exists for the limiting and Clarke subdifferentials, under a slightly stricter assumption:

Proposition 2.2.16. Let f : X -→ R ∪ {+∞} be a quasi-convex function such that f (x 2 ) < f (x 1 ). Then x * , x 2 -x 1 ď 0 for all x * ∈ ∂ L f (x 1 ). If f is locally Lipschitz then ∂ L can be replaced by ∂ C .

Optimization for vector-valued functions 2.3.1 Ordered space

We say that a relation R on X is

• reflexive if {x ∈ X | xRx} = X, • irreflexive if {x ∈ X | xRx} = ∅,
• transitive if ∀x, y, z ∈ X, xRy and yRz implies xRz,

• antisymmetric if ∀x, y ∈ X, xRy and yRx implies x = y.

We call order a reflexive and transitive relation 12 , strict order an irreflexive and transitive relation. Observe that each order ĺ induces a strict order ň defined by x ň y iff x ĺ y and x = y. In a similar way, each strict order ă induces an order defined by x y iff x ă y and x = y.

If X is a vector space, we say that an order ĺ respects the vectorial structure of X if :

• ∀x 1 , x 2 , y 1 , y 2 ∈ X, x 1 ĺ y 1 and x 2 ĺ y 2 implies x 1 + x 2 ĺ y 1 + y 2 ,

• ∀x, y ∈ X, λ ∈ R + , x ĺ y implies λx ĺ λy.

In the same way, we say that a strict order ă respects the vectorial structure of X whether it holds for the associated order . In the following, we will always assume that the considered orders (resp. strict orders) respect the vectorial structure of X, without mentioning it. In vector spaces, there is a natural correspondence between orders and convex cones. Consider the following :

Given a relation R, define K R := {x ∈ X | 0Rx}, Given a set K, define xR K y ⇔ y -x ∈ K.
Then, it can be easily verified that the above constructions define a one-to-one correspondence between the orders (resp. strict orders) on X and the convex cones (resp. convex strict cones) of X. This is why we will say in the following that X is ordered by a given convex cone K', referring implicitly to the corresponding order ĺ K , and will note this ordered vector space by the couple (X, K).

Example 2.3.1.

• Let X = R, equipped with the usual order ď. It is characterized by the convex cone of positive reals R + . The corresponding strict order < is characterized by the convex strict cone of strictly positive reals R ++ . Note that there is no other orders in R than the ones corresponding to the cones R + , R -and {0}.

•

Let X = R m , m ∈ N * . It is usually ordered by a ĺ b ⇔ a i ď b i in R, for all i ∈ {1, ..., m}.
The corresponding convex cone is the positive orthant R m

+ := (R + ) m . A common strict order on R m is a ă b iff a i < b i in R, for all i ∈ {1, ..., m}. The corresponding convex strict cone is R m ++ := (R ++ ) m = int R m + .
R m can be equipped with other orders, like the lexicographical order, which is characterized when m = 2 by the convex cone R 2 ++ ∪ {0} × R + . Let us also mention the order induced by the second order cone in R m+1 , which is just the epigraph of the euclidean norm of R m .

• Let X = S m (R), the vector space of real symmetric matrices of size m ∈ N * . In general, it is ordered by the convex cone S m + (R) of positive symmetric matrices, which consists in symmetric positive matrices.

• Consider some set Ω and X = F(Ω, R) the vector space of functions on Ω with real values.

It is ordered by the convex cone of positive functions

F + (Ω, R) := {f ∈ F(Ω, R) | f (x) ě 0 ∀x ∈ Ω}.
It induces a similar order on any subspace Y of F(Ω, R) by taking the intersection Y ∩ F + (Ω, R).

• Suppose now that (Ω, Σ, µ) is a σ-finite measured space and consider L 0 (Ω, Σ) the space of measurable functions on Ω with real values. We can quotient this space by the equivalence relation f ∼ g ⇔ f (x) = g(x) for µ-a.e. x ∈ Ω, to obtain the vector space L 0 (Ω, Σ, µ).

It is ordered by the convex cone of µ-almost positive functions L 0 + (Ω, R, µ) := {f ∈ L 0 (Ω, Σ, µ) | f (x) ě 0 for µ-a.e. x ∈ Ω}. Here again, this cone induces a similar order on any subspace, including all L p (Ω, Σ, µ) spaces, for p ∈ [1, +∞].

Of course, this list is non exhaustive, and we could also have define positive convex cones in spaces of measures for instance. But it contains the ordered spaces which will be the most interesting for us in the sequel. R m + will be our first object of study, because it is particularly easy to handle and benefits from good "finite" properties : it is a finite-dimensional object, and it is polyhedral. (S m (R), S m + (R)) is also finite-dimensional, but has a more complicated geometrical nature: it is for instance not polyhedral. Finally, (L ∞ (Ω, Σ, µ), L ∞ + (Ω, Σ, µ)) is an example of a "good" infinite-dimensional ordered space since, as we will see later, L ∞ + (Ω, Σ, µ) has a nonempty interior.

Base of a cone

We focus now on the concept of base for a cone. Its interest is that the knowledge of the order structure on (X, K) is equivalent to know a base of the dual cone K + (see Theorem 2.3.7 below). Definition 2.3.2. Let S ⊂ (X, τ ) be a nonempty strict cone. We say that Θ ⊂ S is a base of S if for all d ∈ S, there exists a unique (λ, θ) ∈ R ++ × Θ such that d = λθ. We say that Θ is a base of a cone K, if Θ is a base of K \ {0}.

Observe that, by definition, a base do not contain the origin. Note also that any cone admits a base, as a direct consequence of the axiom of choice. We give now some examples.

Example 2.3.3.

• Let (X, • ) be a normed space, and note S X its unit sphere. Then K ∩ S X is a base for any cone K ⊂ X.

• R m + admits as a base the unit simplex

∆ m := {θ ∈ R m + | m i=1 θ i = 1}.
• S m + (R) admits as a base the set {M ∈ S m + (R) | tr(M ) = 1}.

• Let (Ω, Σ, µ) be a finite measured space and consider

L p + (Ω, Σ, µ) for p ∈ [1, +∞]. It admits as a base {f ∈ L p + (Ω, Σ, µ) | Ω f dµ = 1}.
Note that the bases presented here share a common structure : it is the intersection of a cone K with the level set of some function. The first example deals with the norm while in the three others we intersect with an hyperplane. It can be also noticed that the last base can equivalently be written as the intersection of L p + (Ω, Σ, µ) with the unit sphere of L 1 (Ω, Σ, µ).

In Part II, we will need for our analysis to work with a weakly * compact and convex base of K + . We follow then with some necessary and sufficient conditions on K for the existence of such a base (see Appendices A.3 for a proof). Theorem 2.3.4. Let K be a closed convex cone in a Banach space X. Then K has a nonempty interior if and only if K + admits a convex weakly * compact base Θ. In that case, there exists e ∈ int K such that

Θ = {x * ∈ K + | x * , e = 1}.
Example 2.3.5. Let us briefly convince ourselves that this result is optimal. Take in X = R 2 the closed convex cone K = R + × {0}, which has an empty interior. Then K + = R + × R, and it does not admit any convex base since such a base would contain the origin.

Example 2.3.6. We have already seen in Example 2.3.3 bases of convex cones built on the model Θ = {x * ∈ K + | x * , e = 1} (that we will note Θ e for short).

• The positive orthant R m + ⊂ R m is self-dual for the euclidean scalar product, and int R m + = R m ++ . By taking e = (1, ..., 1) ∈ R m ++ , we recognize Θ e as the unit simplex ∆ m . It is convex and compact.

• The cone of positive symmetric matrices is self-dual for the scalar product X, Y = tr t XY.

By taking e = Id ∈ int S m + (R) = S m ++ (R), we recognize Θ e as the set of positive symmetric matrices of trace one. It is convex and compact.

• Let (Ω, Σ, µ) be a finite measured space and consider the Banach space L p (Ω, Σ, µ) for p ∈ [1, +∞[. Let q ∈]1, +∞] be the conjugate 13 of p. The dual cone of L p + (Ω, Σ, µ) is

L q + (Ω, Σ, µ).
We consider e = 1 Ω , i.e., the function being constant to 1 on all Ω. It is clear that e ∈ L p + (Ω, Σ, µ), since Ω is finite and e, f L q ×L p = Ω f dµ for all f ∈ L p (Ω, Σ, µ). Then we recover the base Θ e presented in Example 2.3.3. Nevertheless, note that it is not bounded, because the L 1 ball is not bounded in such L p . This is not surprising in the light of Theorem 2.3.4, since it is known that the interior of L p + (Ω, Σ, µ) is empty for p ∈ [1, +∞[.

• Let (Ω, Σ, µ) be a finite measured space and consider the Banach space L ∞ (Ω, Σ, µ).

Its dual is ba(Ω, Σ, µ), the Banach space of bounded additive measures which are absolutely continuous with µ (see [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Chapter 10.10]). The dual cone of L ∞ + (Ω, Σ, µ) is ba + (Ω, Σ, µ), the convex cone of such positive measures. By taking again e = 1 Ω , it is easy to see, for the same reasons as above, that e ∈ int L ∞ + (Ω, Σ, µ). It gives us Θ e = {π ∈ ba + (Ω, Σ, µ) | π(Ω) = 1}. This is nothing but the set of additive probability measures on Ω absolutely continuous with respect to µ, which is weakly * compact and convex.

As announced, a base Θ of the dual K + controls the monotonic properties in (X, K). In fact, the extreme points of Θ are enough to characterize the monotonicity.

Theorem 2.3.7. Let K be a closed convex cone with nonempty interior in a Banach space X. Let Θ be a weakly * compact convex base of a K + , and let Ξ be the set of its extreme points. Note ĺ the order on X induced by K. Then for all x, x ′ ∈ X:

i) x ĺ x ′ if and only if θ, x ď θ, x ′ for all θ ∈ Θ, ii) x ĺ x ′ if and only if ξ, x ď ξ, x ′ for all ξ ∈ Ξ, iii) If x ĺ x ′ then σ Θ (x) ď σ Θ (x ′ ). (monotonicity)
Note ă the strict order on X induced by int K. Then for all x, x ′ ∈ X: iv) x ă x ′ if and only if θ, x < θ, x ′ for all θ ∈ Θ, v) x ă x ′ if and only if ξ, x < ξ, x ′ for all ξ ∈ Ξ,

vi) If x ă x ′ then σ Θ (x) < σ Θ (x ′ ). (strict monotonicity)
The proof of this result is left in the Appendice A.3. Let us give en exact formulation of σ Θ for some important examples :

Example 2.3.8. • If X = R m and Θ = ∆ m the unit simplex, then it is easy to see that σ Θ is the max function, that is σ Θ (x) = max i∈{1,...,m} x i . • If X = S m (R) and Θ = {M ∈ S m + (R) | tr(M ) = 1}
, then we can demonstrate that σ Θ (A) is the greatest eigenvalue of A ∈ S m + , that we note λ max (A). Indeed, by compactness of Θ, there exists some M ∈ Θ such that σ Θ (A) = tr(M A). Using Ky Fan's inequality, we easily see that this is bounded from above by λ max (A). On the other hand, λ max (A) = Av, v for some eigenvector v ∈ R m such that v = 1. Since Av, v = tr(v t vA), and v t v ∈ Θ, the conclusion follows.

• Take X = L ∞ (Ω, Σ, µ) as introduced in Example 2.3.6. Let Θ = {λ ∈ ba + (Ω, Σ, µ) | λ(Ω) = 1}. Then we can show, analogously to the previous examples, that σ Θ corresponds to the essential supremum on L ∞ (Ω, Σ, µ). For a given φ ∈ L ∞ (Ω, Σ, µ), use the weak * compactness of Θ to obtain a probability π ∈ Θ such that σ Θ (φ) = π, φ ba,L ∞ := Ω φ dπ.

Since φ is bounded from above by supess φ for µ-a.e. x ∈ Ω, and π is absolutely continuous with respect to µ, then φ(x) ď supess φ for π-a.e. x ∈ Ω. Moreover π(Ω) = 1, then it follows that Ω φ dπ ď supess φ. On the other hand, first suppose that φ reach its essential supremum on non µ-negligeable A ⊂ Ω. Then we can see that π := 1 A µ(A) lies in Θ and verify π, φ ba,L ∞ = supess φ. In the general case, write φ as the uniform limit of a sequence of step functions φ n . Each step function attains its essential supremum on a non negligeable set and we can define π n as previously such that π n , φ n ba,L ∞ = supess φ n .

Passing to the limit, using the uniform convergence of φ n in L ∞ (Ω, Σ, µ), and the strong continuity of supess , gives supess φ ď lim n σ Θ (φ n ). Using the strong continuity of σ Θ gives the conclusion.

We end with a final result concerning the bases of polyhedral cones (see Appendix A.3) Theorem 2.3.9. Let K be a closed convex cone with nonempty interior in a finite-dimensional Banach space X. Let Θ be a weakly * compact convex base of a K + . Then the following statements are equivalent:

i) K is polyhedral, ii) Θ is a polytope,
iii) Θ has a finite number of extremal points.

Vector optimization problem

Let X and Y be two Banach spaces, and F : X → Y. Suppose that Y is equipped with an order induced by a closed convex cone K, with nonempty interior. In the following, ĺ will denote the order induced by K, and ň the associated strict order induced by K \ {0 Y }. Moreover, we will note ă the strict order induced by the strict cone int K. We recall from Section 2.3.1 that this is equivalent to say that

• y 1 ĺ y 2 if and only if y 2 -y 1 ∈ K, • y 1 ň y 2 if and only if y 2 -y 1 ∈ K \ {0}, • y 1 ă y 2 if and only if y 2 -y 1 ∈ int K.
We consider a nonempty closed convex set C ⊂ X, which will model the constraints. The associated vector optimization problem ((VOP) for short) consists in solving

(VOP) MIN x∈C F (x),
where a precise meaning shall be given to MIN F . Clearly, it would not be very useful to define a solution of (VOP) as a point x ∈ C satisfying F (x) ĺ F (x) for all x ∈ C. Indeed, the order ĺ is not total, and there is few chances that all the elements of C can be compared. Instead, we say that x is an efficient solution of (VOP) (or that x is an efficient point of F on C) if there is no x ∈ C such that F (x) ň F (x). We note ARGMIN x∈C F the set of such efficient points. An other way to define the efficient points is to say that it is the points x ∈ C such that

[F ň F (x)] ∩ C = ∅,
where [F ň F (x)] denotes a sublevel set of F at x, defined by

[F ň F (x)] := {x ∈ X | F (x) ň F (x)}.
If, instead of considering ň, we consider the strict order ă induced by int K, we obtain a weaker notion of solution for (VOP). Indeed, we say that x is a weak efficient solution of (VOP) (or that x is a weak efficient point of F on C) if there is no x ∈ C such that F (x) ă F (x). We note ARGMIN w x∈C F the set of such efficient points. Equivalently, x ∈ C is weakly efficient if and

only if [F ă F (x)] ∩ C = ∅.
Since int K ⊂ K \ {0}, all efficient points are in particular weakly efficient, the converse being false in general. When facing a vector optimization problem, we usually look for its efficient points, but usual methods generally provide only weak efficient points (see Section 6.3 of Chapter 6 for a discussion on this question).

Observe that, in the scalar-valued case (i.e. (Y, K) = (R, R + )), both notions of efficiency and weak efficiency coincide with the usual notion of minimum. In other words, (VOP) reduces to a classic optimization problem. An other essential case is the multi-objective optimization problem ((MOP) for short), or Pareto optimization problem, where (Y, K) = (R m , R m + ). In that case, there exists a finite family (f 1 , ..., f m ) of real-valued functions on X such that for all x ∈ X, F (x) = (f 1 (x), ..., f m (x)).

With this notation, the multi-objective optimization problem writes as

(MOP) MIN x∈C (f 1 (x), ..., f m (x)),
and we can revisit the notions of efficiency (reps. weak efficiency). We say that a point x ∈ C is:

• Pareto efficient if there is no x ∈ C such that f i (x) ď f i (x) for all i ∈ {1, ..., m}, and f I (x) < f I (x) for some I ∈ {1, ..., m},

• weakly Pareto efficient if there is no x ∈ C such that f i (x) < f i (x) for all i ∈ {1, ..., m}.

The guideline of this thesis is to use descent methods to solve optimization problems. For solving (VOP), the idea essentially reduces to the following: given a current state x ∈ C, can we improve the value F (x)? That is, can we find some x ′ ∈ C such that F (x ′ ) ň F (x) (resp. F (x ′ ) ă F (x))? If so, x ′ becomes our new current state, otherwise x is by definition an efficient (resp. weakly efficient) solution of (VOP).

Then it is clear that being able to compare two points x, x ′ is essential, and that is why Proposition 2.3.7 is really important. Suppose that we choose some e ∈ int K, and take

Θ = {y * ∈ K + | y * , e = 1}.
According to Theorem 2.3.4 and Proposition 2.3.7, we have for all x, x ′ ∈ X (2.26)

F (x ′ ) ĺ F (x) ⇔ ∀θ ∈ Θ, (θ • F )(x ′ ) ď (θ • F )(x),
where θ • F : X -→ R is the composition between the linear form θ ∈ Θ ⊂ Y * and F . We will generally use the notation f θ := θ • F , and call these functions the cost functions associated 14 14 Of course these functions do not depend only on F , but also on K and the choice of a base for K + (that is, according to Theorem 2.3.4, the choice of an element e ∈ int K). But note that 1) we will always work with a fixed cone K and a fixed base Θ, 2) the choice of a base for K + has no influence on the main concepts which will be used later (see Chapter 5).

to F . What (2.26) means, is that controlling the descent of F is equivalent to control the simultaneous descent of its cost functions {f θ } θ∈Θ . What is interesting in this approach is that we went back from vector values to scalar values, so we will be able to use the tools we are familiar with for scalar optimization.

Proposition 2.3.7 says also that it is not necessary to control all the cost functions {f θ } θ∈Θ , but only the ones corresponding to Ξ, the set extreme points of Θ:

(2.27) F (x ′ ) ĺ F (x) ⇔ ∀ξ ∈ Ξ, (ξ • F )(x ′ ) ď (ξ • F )(x).
We will call these functions {f ξ := ξ • F } ξ∈Ξ the extreme cost functions associated to F . They are of particular interest when K is polyhedral, since it is exactly the case for which Θ has a finite number of extreme points (see Theorem 2.3.9). This means a finite number of extreme cost functions to manage. Take for instance the multi-objective case F = (f 1 , ..., f m ) : X -→ R m , R m being ordered by the usual posititve orthant R m + . We choose e = (1, ..., 1) ∈ int R m + = R m ++ , whose corresponding basis is ∆ m the unit simplex (see Example 2.3.6), and the extreme points of ∆ m corresponds to the canonical basis Ξ = {e 1 , ..., e m }. The corresponding extreme cost functions are exactly the family {f 1 , ..., f m }. In that case, (2.26) says that the decrease of F is governed by the decrease of all convex combinations (the cost functions)

f θ = m i=1 θ i f i , θ = (θ i ) ∈ ∆ m ,
while (2.27) tells us that it is enough to have the decrease of the finite family {f 1 , ..., f m } (the extreme cost functions).

We now discuss the convexity of a vector-valued function F : X -→ (Y, K) with respect to K. Keep the notations that we used above. We say that F is convex (with respect to

K) if ∀x, x ′ ∈ X, ∀t ∈ [0, 1], F (tx + (1 -t)x ′ ) ĺ tF (x) + (1 -t)F (x ′ ).
Applying the monotonicity Theorem 2.3.7, we see that F is convex if and only if each cost function {f θ } θ∈Θ is convex. It is in fact sufficient for the extreme cost functions {f ξ } ξ∈Ξ to be convex, which is a quite useful characterization in the polyhedral case. We also say that F is strictly convex (with respect to K)

if ∀x = x ′ ∈ X, ∀t ∈]0, 1[, F (tx + (1 -t)x ′ ) ă tF (x) + (1 -t)F (x ′ ).
Convex functions enjoy a useful characterization of their weakly efficient points, which is the base of all scalarization techniques mentionned in the introduction.

Theorem 2.3.10. Let F : X -→ (Y, K) be a convex function between two Banach spaces. Suppose that C ⊂ X is convex and K is a closed convex cone with nonempty interior. Then,

ARGMIN w x∈C F (x) = θ∈Θ argmin x∈C f θ (x).
This result can be found in the book of Dihn [139, Proposition IV.2.10], or in [73, Theorem 2.1]. We will see in Chapter 5 how this result can be simply interpreted as a Fermat's rule for functions with values in an ordered vector space.

We say that F :

X -→ (Y, K) is quasiconvex (with respect to K) if ∀x, x ′ , x ∈ X, ∀t ∈ [0, 1], F (x) ĺ F (x) and F (x ′ ) ĺ F (x) ⇒ F (tx + (1 -t)x ′ ) ĺ F (x). Equivalently, F is quasiconvex if and only if ∀x ∈ X, [F ĺ F (x)] is convex, where [F ĺ F (x)] = {x ∈ X | F (x) ĺ F (x)}
denotes as before a sublevel set of F at F (x). We say moreover that F is scalarly quasiconvex whenever all cost functions {f θ } θ∈Θ are quasiconvex. Scalar quasiconvexity entails quasiconvexity since we can write (use Theorem 2.3.7)

∀x ∈ X, [F ĺ F (x)] = θ∈Θ [f θ ď f θ (x)].
But contrary to the what happens in the convex setting, the reverse is not true in general.

We end with a basic notion for functions with values in an ordered space. We say that F : E -→ Y is bounded from below if there exists y ∈ Y such that y ĺ F (x) for all x ∈ E. In our setting, where K has a nonempty interior, the boundedness of F is equivalent to the uniform boundedness of the cost functions {f θ } θ∈Θ : Proposition 2.3.11. Let F : X -→ Y, let e ∈ int K and θ the corresponding base of K + , noting Ξ its set of extremal points. The following are equivalent :

i) F is bounded from below, ii) ∃m ∈ R such that me ĺ F (x) for all x ∈ E, iii) ∃m ∈ R such that m ď f ξ (x) for all x ∈ E and ξ ∈ Ξ, iv) ∃m ∈ R such that m ď f θ (x) for all x ∈ E and θ ∈ Θ.
Proof. Suppose that i) holds. Then there exists some y ∈ Y such that, for all x ∈ E and θ ∈ Θ, θ, y ď f θ (x). Let M > 0 be a bound for Θ in Y * , then by using Cauchy-Schwarz's inequality, we obtain -M y ď f θ (x). So item iv) holds, with m = -M y . The implication iv) ⇒ iii) ⇒ ii) follows Theorem 2.3.7 and the fact that θ, e = 1 for all θ ∈ Θ. Implication ii) ⇒ i) is obvious.

Part I

Dynamics for Tame optimization

Most analysts spend half their time hunting through the literature for inequalities they want to use, but cannot prove.

Harald Bohr

Chapter 3

The Kurdyka-Lojasiewicz inequality and general descent methods

Along this chapter, we consider a Hilbert space (H, •, • ), and a proper lower semi-continuous function f : H -→ R ∪ {+∞}. We want to solve the associated optimization problem

min x∈H f (x).
For this, we look for descent dynamical systems corresponding to f , being discrete or continuous.

Our main tool in this chapter is the assumption that f satisfies the so-called Kurdyka-Lojasiewicz inequality. This notion is introduced and illustrated in Section 3.1: after an informal introduction in Section 3.1.1, we properly define it in Section 3.1.2, and give some examples of functions satisfying this inequality. In Section 3.1.3, we illustrate one of the main consequences of the Kurdyka-Lojasiewicz inequality in Theorem 3.1.8. Considering the subgradient differential inclusion

(3.1) u(t) + ∂ F f (u(t)) ∋ 0,
any of its trajectories has finite length and converges towards a critical point of f , provided that u satisfies some compactness assumption, and that f has the Kurdyka-Lojasiewicz property. Then, we detail in Theorem 3.1.12 some rates of convergence for the trajectories. In Section 3.2, we adapt this convergence result to a general abstract descent algorithm. It is inspired by the work of [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF], but we extend its setting in order to account for additive computational errors, and more versatility in the choice of the parameters. The strong convergence of the iterates towards a critical point of f , with a finite-length condition is obtained in Theorem 3.2.2. A local convergence result to global minimum is also provided in Theorem 4.1.8, under certain hypotheses. Then, in Section 3.3, we prove new and interesting general convergence rates. They are similar to the ones obtained in the literature for various numerical methods, see e.g. [START_REF] Attouch | On the convergence of the proximal algorithm for nonsmooth functions involving analytic features[END_REF][START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF][START_REF] Xu | A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion[END_REF][START_REF] Bolte | Proximal alternating linearized minimization for nonconvex and nonsmooth problems[END_REF][START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF]. Surprisingly, an explicit form of the algorithm terminates in a finite number of iterations in several cases, and shares the same convergence rates with the continuous-time dynamical system.

The Kurdyka-Lojasiewicz inequality

Introduction

In a first time, we propose an informal and naive approach to the Kurdyka-Lojasiewicz inequality 1 .

Suppose that f : H -→ R is a continuously differentiable function, and consider the corresponding steepest descent dynamic:

(SD) ẋ(t) + ∇f (x(t)) = 0.
Given a classic global solution x : [0, +∞[-→ H of (SD), we focus on its asymptotic behaviour, when t goes to +∞. What we aim to prove here is that the trajectory has finite length, i.e. +∞ 0 ẋ(t) dt < +∞. This is a strong property, which implies in particular that the trajectory strongly converges when t → +∞.

First observe that the function f is Lyapunov for the (SD) dynamic. Indeed, it suffices to take the scalar product of (SD) with ẋ(t), and to use a chain rule, to obtain

(3.2) d dt (f • x)(t) = -ẋ(t) 2 ď 0.
If we assume that ẋ(t) is nonzero on [0, +∞[, (3.2) means that f • x is strictly decreasing on [0, +∞[. If we assume moreover that f is bounded from below, we obtain that f • x realizes a diffeomorphism between [0, +∞[ and ]s ∞ , s 0 ], where s 0 := f (x(0)) and s ∞ := inf We can use this reparametrized trajectory to rewrite the length of x(•):

(3.3) +∞ 0 ẋ(t) dt = s 0 s∞ 1 ∇f (y(s))
ds.

Observe that the quantity in (3.3) is finite, as soon as ∇f (y(s)) ě C > 0 for all s ∈ [s ∞ , s 0 ]. In other words, the question of the finite length of x(•) is essentially dependent on how ∇f behaves around its zeroes, namely, the critical points of f . Suppose now that there exists some smooth increasing function ϕ : [s ∞ , s 0 ] -→ R + , such that ϕ(s ∞ ) = 0 and

(3.4) 1 ∇f (y(s)) ď ϕ ′ (s) for all s ∈]s ∞ , s 0 ].
Then, it follows from (3.3) that x(•) has finite length:

+∞ 0 ẋ(t) dt ď s 0 s∞ ϕ ′ (s) ds = ϕ(s 0 ) < +∞.
Clearly, the key point here is the inequality (3.4), which can be rewritten, after using the change of variables s = f (x(t)), as

ϕ ′ (f (x(t))) ∇f (x(t)) ě 1, ∀t ě 0.
As we said before, the question of the finite length of the trajectory only matters around the critical points (points x such that ∇f (x) = 0). Roughly speaking, the Kurdyka-Lojasiewicz inequality asks that, for any critical point x ∈ H, there exists a smooth and increasing function ϕ : [0, +∞[-→ [0, +∞[ such that ϕ(0) = 0 and

(3.5) ϕ ′ (f (x) -f (x)) ∇f (x) ě 1, for all x ∼ x such that f (x) < f (x).
Assuming that f (x) = 0 and using a chain rule, we see that the Kurdyka-Lojasiewicz inequality becomes

(3.6) ∇(ϕ • f )(x) ě 1, ∀x ∼ x.
In other words, we ask for the possibility to locally reparametrize f , in order to make it become sharp around its critical points. That is why, in general, ϕ is called the desingularizing function for f at x. Indeed, the more f is flat around its critical point, the more ϕ has to be steep around 0.

The Kurdyka-Lojasiewicz inequality

In the previous discussion, we took for convenience a smooth function f : H -→ R. Consider now a real-extended valued function f : H -→ R∪{+∞}. In that case, the Kurdyka-Lojasiewicz inequality will set a relation between the variations of f and the norm of its subgradients. More exactly, we will consider the limiting subgradients of f , noted ∂ L f , introduced in Section 2.2.3. We say that x ∈ H is a limiting critical point of f if 0 ∈ ∂ L f (x). We also consider the lazy slope of f at x:

∂ L f (x) -:= inf x * ∈∂ L f (x)
x * .

Let η > 0. We say that ϕ

: [0, η[-→ [0, +∞[ is a desingularizing function if i) ϕ(0) = 0, ii) ϕ is continuous on [0, η[ and of class C 1 on ]0, η[, iii) ϕ ′ (t) > 0 for all t ∈]0, η[.
(KL) ϕ ′ (f (x) -f (x)) ∂ L f (x) -ě 1
holds, for all x in the strict local upper level set

Γ η (x, δ) := {x ∈ B(x, δ) | f (x) < f (x) < f (x) + η}.
A proper lower-semicontinuous function having the Kurdyka-Lojasiewicz property (K L property for short) at any x ∈ H is said to be a K L function.

Remark 3.1.2. In the following, we will give a special attention to the functions satisfying the K L inequality with a desingularizing function of the form ϕ(t) = C θ t θ , θ ∈]0, 1]. In that case, f will be said to be K L with the Lojasiewicz exponent θ.

Remark 3.1.3. As mentioned before, for functions of class C 1 , the K L inequality holds automatically at any regular point (i.e. being not limiting critical points), with a desingularising function ϕ(t) = Ct. This property is also verified by all proper lower semi-continuous functions, when H has finite dimension. Indeed, assume by contradiction that for some regular x ∈ H there exists

(x k ) k∈N in H such that ∀k ∈ N, x k -x ď 1 k , f (x) < f (x k ) < f (x) + 1 k and ∂ L f (x k ) -< 1 n .
This sequence would satisfy

x k f --→ k→+∞ x and ∂ L f (x k ) ---→ k→+∞ 0.
But in finite dimension, the limiting subdifferential ∂ L f has a f -closed graph (see [START_REF] Rockafellar | Variational analysis[END_REF]Proposition 8.7]), so it would follow that 0 ∈ ∂ L f (x), which contradicts the noncritical assumption on x. In more general Hilbert spaces, the limiting subdifferential is no more guaranteed to have a f -closed graph (see [START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF]Example 1.7]). In that case, we can assume that f is lower-regular, in which case the argument above still works.

We present now some known classes of K L functions.

Example 3.1.4 (Analytic functions). Lojasiewicz [START_REF] Lojasiewicz | Une propriété topologique des sous-ensembles analytiques réels[END_REF] proved that, for any critical point x of an analytic function f : R n -→ R, there exists an open ball B(x, δ) such that

|f (x) -f (x)| 1-θ ď C ∇f (x) ,
for some C > 0 and θ ∈]0, 1 2 ]. This proves that analytic functions are K L functions, with a Lojasiewicz exponent θ ∈]0, 1 2 ]. Let us mention that Lojasiewicz's result applies only in finite dimension. Indeed, in infinite-dimensional spaces, analytic functions need not have the K L property, see [START_REF] Haraux | The Lojasiewicz gradient inequality in the infinite dimensional Hilbert space framework[END_REF] for a counter-example. In a similar fashion to Lojasiewicz, Bolte-Daniilidis-Lewis show that continuous subanalytic functions on R n , with closed domain, are K L with a Lojasiewicz exponent θ ∈]0, 1], see [START_REF] Bolte | The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF]Theorem 3.1].

Example 3.1.5 (Functions definable in o-minimal structures). In the finite-dimensional setting, the o-minimal structures provide a huge class of functions satisfying the K L property. Ominimal structures were developed around the 90's to generalize the good stability and regularity properties of the class of semi-algebraic sets [START_REF] Pillay | Definable sets in ordered structures. I[END_REF][START_REF] Van Den Dries | Geometric categories and o-minimal structures[END_REF]. These o-minimal structures includes the semi-algebraic structure, in which are definable the polynomials, the linear maps, the ℓ p norms with p ∈ Q * , and the counting function, among others. It is known that the lower semicontinuous functions lying in this semi-algebraic structure are K L, with a Lojasiewicz exponent θ ∈]0, 1] [70, Corollary 16]. There also exists bigger o-minimal structures, containing for instance the exponential function [START_REF] Van Den Dries | The elementary theory of restricted analytic fields with exponentiation[END_REF][START_REF] Wilkie | Model completeness results for expansions of the ordered field of real numbers by restricted Pfaffian functions and the exponential function[END_REF], the ℓ p norms with p ∈ R * [START_REF] Miller | Expansions of the real field with power functions[END_REF] and the primitives of all the aforementioned functions [START_REF] Van Den Dries | The real field with convergent generalized power series[END_REF][START_REF] Van Den Dries | The field of reals with multisummable series and the exponential function[END_REF][START_REF] Speissegger | The Pfaffian closure of an o-minimal structure[END_REF]. Note that besides being huge, these class of functions are stable by usual operations which are very useful in optimization: addition, composition, supremum, restriction to an o-minimal set. Furthermore, the distance function to an o-minimal set and the indicator function of an o-minimal set are also o-minimal functions (this list is far from being exhaustive). The fact that any lower semi-continuous function definable in an o-minimal structure is a K L function is a strong result, due to Bolte, Daniilidis, Lewis and Shiota [70, Corollary 15]. It extends a previous result of Kurdyka [START_REF] Kurdyka | On gradients of functions definable in o-minimal structures[END_REF] on functions of class C 1 definable on an o-minimal structures. Observe that, in the general case, a definable function may not be K L for any Lojasiewicz exponent θ ∈]0, 1]. Indeed, a function such that x → e -1 x 2 needs a desingularizing function function ϕ with an exponential growth around the origin. For more information about o-minimal structures, its definition and the properties they enjoy, the reader shall consult [START_REF] Van Den Dries | Tame topology and o-minimal structures[END_REF][START_REF] Van Den Dries | Tame Topology and O-minimal Structures[END_REF][START_REF] Ioffe | An Invitation to Tame Optimization[END_REF].

Example 3.1.6 (K L inequality versus convexity). Let us mention the relationships between convexity and the K L property. It is easy to check that strongly convex functions are K L with a Lojasiewicz exponent θ = 1 2 (see [START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF]Section 4.2]). But in general, convex functions are not K L: see [START_REF] Bolte | Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity[END_REF]Section 4.3], where the authors coined a counter-example of a convex function not being K L. Nevertheless, it requires such a twisted construction that we can reasonably say that 'common' convex functions are K L. For instance, a convex function is K L as soon as it verifies some growth condition around its critical points [START_REF] Bolte | Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity[END_REF]Theorem 30].

Let us finally mention that the K L property can be derived from appropriate assumptions on the Hessian D 2 f (x) ( [START_REF] Chill | The Lojasiewicz-Simon gradient inequality in Hilbert spaces[END_REF][START_REF] Haraux | The Lojasiewicz gradient inequality in the infinite dimensional Hilbert space framework[END_REF]). It is also worth noticing that the K L inequality has strong connexions with metric regularity, see [START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF][START_REF] Bolte | Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity[END_REF].

Asymptotic behaviour of a subgradient differential inclusion

Given a proper lower-semicontinuous function f : H -→ R ∪ {+∞}, consider the following subgradient differential inclusion:

(3.7) u(t) + ∂ F f (u(t)) ∋ 0.
When f is smooth, this dynamical system specialises in the usual steepest descent dynamic:

(3.8) u(t) + ∇f (u(t)) = 0.
We saw in the previous section -at least formally-that in this smooth case, the trajectories converge to a critical point of f , provided that f is a K L function. We shall verify below that the subgradient dynamic (3.7) shares the same properties that the steepest descent one (3.8): it is a descent dynamic, and its trajectories converge to critical points whenever f is a K L function.

First of all, let us precise the notion of solution we will use for (3.7). Indeed, this dynamic is governed by the set-valued mapping x ⇒ ∂ F f (x) which is not continuous, so we cannot use a classical notion of solution. We recall here the definition of absolutely continuous functions (see the monograph of Brezis [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]Appendix] for more details). Definition 3.1.7. Given T ∈ R + , a function u : [0, T ] -→ H is said to be absolutely continuous if one of the following equivalent properties holds: i) there exists an integrable function g : [0, T ] → H such that

u (t) = u (0) + t 0 g (s) ds ∀t ∈ [0, T ] ;
ii) u is continuous and its distributional derivative belongs to the Lebesgue space L 1 ([0, T ] ; H); iii) for every ǫ > 0, there exists η > 0 such that for any finite family of intervals

I k =]a k , b k [, I k ∩ I j = ∅ for k = j and k |b k -a k | ď η =⇒ k u (b k ) -u (a k ) ď ǫ.
We can now make precise the notion of solution for the subgradient differential inclusion. We say that u : [0, +∞[-→ H is a strong global solution of (3.7) if it is absolutely continuous on each bounded interval [0, T ], T < +∞, and if it satisfies the inclusion (3.7) for a.e. t ∈ [0, +∞[. We also introduce the notion of f -precompactness (or f -attentive precompactness) for a trajectory (u(t)) tě0 , meaning that there exists a sequence t n → +∞ and some u ∞ ∈ H such that

u(t n ) f --→ n→+∞ u ∞ , meaning that u(t n ) --→ n→+∞ u ∞ and f (u(t n )) --→ n→+∞ f (u ∞ ).
We are now ready to prove the announced result, which is a simple extension of the one obtained by Bolte-Daniilidis-Lewis [START_REF] Bolte | The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF] for continuous subanalytic functions. A very similar result has been obtained by Ioffe [START_REF] Ioffe | An Invitation to Tame Optimization[END_REF] in o-minimal structures, with a different proof.

(f • u)(t) = -u(t) 2 . ii) (Laziness) For a.e. t ∈ [0, +∞[, u(t) + ∂ F f (u(t)) 0 = 0, where ∂ F f (u(t)) 0 denotes the element of minimal norm of ∂ F f (u(t)).
Assume moreover that f is a K L function, and that (u(t)) tě0 is f -precompact, then:

iii) (Strong convergence) The trajectory has finite length and converges, when t → +∞, to some u ∞ . This limit point is ∂ L -critical, in the sense that 0 ∈ ∂ L f (u ∞ ).

Remark 3.1.9. Let us briefly discuss the hypotheses involved in Theorem 3.1.8. The absolute continuity of f • u can be directly derived from the absolute continuity of u, whenever f is locally Lipschitz continuous on its domain. The f -precompactness assumption can be reduced to simple precompactness, provided that f is continuous on its domain. It is worth noticing that precompactness comes from boundedness if H is finite-dimensional. More generally, according to the decrease property i), precompactness is guaranteed as soon as f has compact level sets. In finite dimensions, the latter is equivalent for f to be coercive.

Remark 3.1.10. We do not address here the question of the existence of such solutions. It is well-known for C 1,1 functions (it is the Cauchy-Lipschitz theorem), or for lower-semi continuous convex functions [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]. For an existence result involving a wider class of functions, the reader can consult [START_REF] Degiovanni | Evolution equations with lack of convexity[END_REF][START_REF] Marcellin | Evolution problems associated with primal lower nice functions[END_REF].

Remark 3.1.11. To prove the convergence of the trajectories, we will follow Lojasiewicz's argument [START_REF] Lojasiewicz | Sur les trajectoires du gradient d'une fonction analytique[END_REF]. Roughly speaking, the proof is divided in three steps: in a first time, use a compactness argument to prove that the trajectory will enter into Γ η (u ∞ , δ) of u ∞ , at some time t ě 0. In a second time, we prove that the length of the trajectory, when it lies in Γ η (u ∞ , δ), is independent of the time. Finally, prove that the set Γ η (u ∞ , δ) captures the trajectory: once the trajectory enters Γ η (u ∞ , δ), it cannot escape from it. Once combined, these three facts entails the finite length of the trajectory.

Proof. Items i) and ii) are quite immediate from the definition of the dynamic. For item ii), use a chain rule (Proposition 2.2.11) together with the absolute continuity of f • u and u to write for a.e. t ∈ [0, +∞[:

(3.9) d dt (f • u)(t) = x * , u(t) , ∀x * ∈ ∂ F f (u(t)).
Since by definition we have -

u(t) ∈ ∂ F f (u(t)), the decrease property d dt (f • u)(t) = -u(t) 2 follows immediately.
For the lazy property, let us start by recalling that in a Hilbert space, the Fréchet subdifferential is closed and convex, so its element of minimal norm is well-defined, provided it is nonempty. This being said, it suffices to show, for a.e.

t ∈ [0, +∞[, that u(t) = ∂ F f (u(t)) -. The inequality u(t) ě ∂ F f (u(t)) -comes directly from -u(t) ∈ ∂ F f (u(t)).
For the reverse inequality, use (3.9) together with the Cauchy-Schwarz inequality to obtain

∀x * ∈ ∂ F f (u(t)), u(t) 2 = x * , u(t) ď x * u(t) .
Thus, u(t) ď x * for all x * ∈ ∂ F f (u(t)), and item i) follows.

We turn now on item iii). First of all, recall that the f -precompactness gives a sequence

t n → +∞ such that u(t n ) --→ n→+∞ u ∞ and f (u(t n )) --→ n→+∞ f (u ∞ ).
Moreover, we supposed that f is bounded from below, so we deduce from item i) that f (u(t)) ↓ f (u ∞ ). We can assume, without loss of generality, that f (u(t)) > f (u ∞ ) for all t ě 0. Otherwise, we would have from item i) that u(t) ≡ u ∞ for all t ě T , for some T ě 0. We invoke now the Kurdyka-Lojasiewicz inequality at u ∞ , which holds on the local strict upper level set Γ η (u ∞ , δ):

∀x ∈ Γ η (u ∞ , δ), ϕ ′ (f (x) -f (u ∞ )) ∂ L f (x) -ě 1.
The key element to obtain the convergence of the trajectory is the introduction of the Lyapunov function h(t)

:= ϕ(f (u(t))-u ∞ )
. By definition of the desingularizing function, it is an absolutely continuous function on bounded intervals, positive, nonincreasing, such that h(t) ↓ 0 when t → +∞.

For an arbitrary pair 0 ď t 1 < t 2 ď +∞, we claim that

(3.10) u(]t 1 , t 2 [) ⊂ Γ η (u ∞ , δ) ⇒ t 2 t 1 u(t) dt ď h(t 1 ).
To see this, use item ii) together with the K L inequality to write for a.e. t ∈]t 1 , t 2 [:

u(t) = ∂ F f (u(t)) -ď ϕ ′ (f (u(t)) -f (u ∞ )) ∂ F f (u(t)) 2 -. Since ∂ F f (u(t)) 2 -= u(t) 2 = -d dt (f • u)(t), we deduce that u(t) ď -ϕ ′ (f (u(t)) -f (u ∞ )) d dt (f • u)(t) = - d dt h(t).
Recalling that h is positive, (3.10) is proved after integrating the above equation on ]t 1 , t 2 [. Now, we deduce from

u(t n ) --→ n→+∞ u ∞ , f (u(t)) ↓ f (u ∞
) and h(t) ↓ 0, that there exists some

t 1 ě 0 such that (3.11) u(t 1 ) -u ∞ ď δ 3 and ∀t ě t 1 , h(t) ď δ 3 , f (u(t)) < f (u ∞ ) + η.
We also define

t 2 := inf{t ě t 1 | u(t) / ∈ Γ η (u ∞ , δ)} ∈]t 1 , +∞].
By definition of these two numbers, and according to the continuity of u together with the monotonicity of (f (u(t))) tě0 , we have u

([t 1 , t 2 [) ⊂ Γ η (u ∞ , δ) and u(t 2 ) / ∈ Γ η (u ∞ , δ).
The claim (3.10), together with (3.11), imply (3.12)

t 2 t 1 u(t) dt ď δ 3 .
If we assume that t 2 < +∞, we would obtain from (3.11) and (3.12):

u(t 2 ) -u ∞ ď u(t 2 ) -u(t 1 ) + u(t 1 ) -u ∞ ď t 2 t 1 u(t) dt + u(t 1 ) -u ∞ ď 2δ 3 , which contradicts t 2 / ∈ Γ η (u ∞ , δ) and f (u(t 2 )) ď f (u(t 1
)) < η. Thus, we deduce that t 2 = +∞, and obtain from (3.12) that u has finite length. In particular, this means that the trajectory converges strongly to u ∞ . Furthermore, item ii) combined with (3.12) gives

∞ 0 ∂ F f (u(t)) -dt < +∞.
The latter implies the existence of some sequence t m --→ m→+∞ +∞ and a corresponding sequence

of Fréchet subgradients x * m ∈ ∂ F f (u(t m )) such that x * m --→ m→+∞ 0.
This means, by definition of the limiting subdifferential, that 0

∈ ∂ L f (u ∞ ).
We can also derive for the solutions of (3.7) an estimate of the rate of convergence, depending only on the behaviour of the desingularizing function around the origin. Next result generalises [START_REF] Chill | Convergence and decay rate to equilibrium of bounded solutions of quasilinear parabolic equations[END_REF]Theorem 2.7] to the nonsmooth case, and we recover the rates of [START_REF] Bolte | The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF] in the subanalytic case.

Theorem 3.1.12. Let f : H -→ R ∪ {+∞} be a proper lower semi-continuous function. Let u(•) be a strong global solution of (3.7), which f -converges when t → +∞ to some u ∞ ∈ H. We assume that f has the K L property at u ∞ , with a desingularizing function ϕ

: [0, η[-→ R + . Let Φ :]0, η[-→ R be any primitive of -ϕ ′2 . i) If lim t→0 Φ(t) ∈ R, then the algorithm converges in a finite number of steps. ii) If lim t→0 Φ(t) = +∞, then there exists some t 1 ∈ R such that: ii.a) f (u(t)) -f (u ∞ ) = O Φ -1 (t -t 1 ) , and ii.b) u(t) -u ∞ = O ϕ • Φ -1 (t -t 1 ) .
Proof. Since we suppose that u(t) f -converges to u ∞ , there exists some t 0 ∈ R + such that, for all t ě t 0 , u(t) lies in the local strict upper level set Γ η (u ∞ , δ) where the K L inequality holds. Assume that, for all t ě t 0 , f (u(t)) > f (u ∞ ). Otherwise, as we seen in the previous proof, the trajectory would stop in finite time. Define r(t) := f (u(t))f (u ∞ ) and as before h(t) := ϕ(r(t)). Observe that we can take t 0 big enough, so that r(t) < η for all t ě 0, making h well-defined on [t 0 , +∞[. We saw in the previous proof that, for all t ě t 0 , u(t) ď -h ′ (t). By integrating this inequality, we obtain for all T ě t ě t 0 :

u(t) -u(T ) ď T t u(τ ) dτ ď - T t h(τ ) dτ = h(t) -h(T ) ď h(t).
After taking the limit when T → +∞, we deduce that

(3.13) u(t) -u ∞ ď h(t) = ϕ(f (u(t)) -f (u ∞ )).
Let now Φ be a primitive of -ϕ ′2 . Then, for all t ě t 0 , we can use Theorem 3.1.8 to write

(Φ • r) ′ (t) = -ϕ ′2 (r(t))(f • u) ′ (t) = ϕ ′2 (f (u(t)) -f (u ∞ )) ∂ F f (u(t)) 2 -ě 1.
Integrate the inequality above to obtain for all t ě t 0 :

(3.14) Φ(r(t)) -Φ(r(t 0 )) = t t 0 (Φ • r) ′ (τ )dτ ě t -t 0 .
Observe that, by definition, the function Φ is strictly decreasing on ]0, η[, hence invertible on this interval. We consider now two cases. If lim r→0 Φ(r) = Φ 0 ∈ R, then we would obtain from (3.14) that Φ 0 ě +∞ by taking the limit when t → +∞. This is a clear contradiction, meaning that our assumption made in the beginning is false. As a consequence, the trajectory converge in finite time. On the other hand, if lim r→0 Φ(r) = +∞, we can assume that t 0 is big enough so that tt 0 + Φ(r(t 0 )) lies in the domain of Φ -1 . Thus, we can deduce from (3.14) that

(3.15) f (u(t)) -f (u ∞ ) ď Φ -1 (t -t 1 ), where t 1 := t 0 -Φ(r(t 0 )).
The conclusion follows from the combination of (3.15) and (3.13).

Convergence of an abstract inexact descent method

Throughout this section, f : H -→ R∪{+∞} is a proper function, lower semi-continuous for the strong topology. We consider a sequence (x k ) k∈N , computed by means of an abstract algorithm satisfying the following hypotheses:

H 1 (Sufficient decrease): For each k ∈ N, for some a k > 0, f (x k+1 ) + a k x k+1 -x k 2 ď f (x k ). H 2 (Relative error ): For each k ∈ N, for some b k+1 > 0 and ε k+1 ě 0, b k+1 ∂ F f (x k+1 ) -ď x k+1 -x k + ε k+1 . H 3 (Parameters): The sequences (a k ) k∈N , (b k ) k∈N and (ε k ) k∈N satisfy: (i) a k ě a > 0 for all k ě 0. (ii) (b k ) k∈N / ∈ ℓ 1 ; (iii) sup k∈N * 1 a k b k < +∞; (iv) (ǫ k ) k∈N ∈ ℓ 1 .
In Section 4.1, we complement this axiomatic description of descent methods by providing a large class of implementable algorithms that produce sequences verifying hypotheses H 1 , H 2 and H 3 . A simple example is: [65, p.36]) is an algorithm whose each iteration has the form

Example 3.2.1. If f is differentiable, a gradient-related method (see
x k+1 = x k + λ k d k ,
where λ k > 0 is a stepsize, and d k ∈ H is a direction agreeing with the steepest descent direction -∇f (x k ), in the sense that

d k , ∇f (x k ) + C 1 ∇f (x k ) 2 ď 0 and d k ď C 2 ∇f (x k ) , with C 1 , C 2 > 0. If ∇f is L-Lipschitz continuous on H,
a k = C 1 C 2 λ k -C 2 L 2 , and b k+1 = (L + 1 C 1 λ k ) -1 .
We easily deduce that H 3 is satisfied if we assume for instance that

λ k ∈ [λ, λ], with λ > 0 and λ < C 1 C 2 2 2 L .
Sequences generated by the procedure described above converge strongly to critical points of f , and the piecewise linear curve obtained by interpolation has finite length. By finite length for a sequence (x k ) k∈N , we mean that

+∞ k=0 x k+1 -x k < +∞.
This is stated in the main convergence result of this chapter: Theorem 3.2.2. Let f : H -→ R ∪ {+∞} be a K L function and let H 1 , H 2 and H 3 hold. If the sequence (x k ) k∈N is f -precompact, then it has finite length, and f -converges to a ∂ L -critical point of f . It is possible in Theorem 3.2.2 to drop the f -precompactness assumption and obtain a capture result, near a global minimum of f . To simplify the notation, for x * ∈ H, η ∈]0, +∞] and δ > 0, define the local upper level set by

(3.16) Γ η (x, δ) := { x ∈ H : x -x < δ and f (x) ď f (x) < f (x) + η }.
We have then:

Theorem 3.2.3. Let f : H -→ R ∪ {+∞} having the K L property at a global minimum x of f . Let (x k ) k∈N be a sequence satisfying H 1 , H 2 and H 3 , with ǫ k ≡ 0.
Then, there exist δ > 0 and η > 0 such that, if x 0 ∈ Γ η (x, δ), then the sequence (x k ) k∈N has finite length, and f -converges to a global minimum of f .

As mentioned in [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF], Theorem 3.2.3 admits a more general formulation. For instance, if x * is a local minimum of f , where a growth property is locally satisfied (see [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF]Remark 2.11]).

The proofs of Theorems 3.2.2 and 3.2.3 rely on the Kurdyka-Lojasiewicz inequality, and we adapt for this the Lojasiewicz's argument used in Section 3.1.3 for the continuous case. In a first time, assuming that the sequence remains in a local strict upper level set Γ η (x, δ), where the K L inequality holds, we show that it has finite length. In a second time, we prove the existence of such a set Γ η (x, δ) capturing the sequence. The adaptation of this argument to the discrete case goes back to Absil, Mahony and Andrews [START_REF] Absil | Convergence of the iterates of descent methods for analytic cost functions[END_REF], who studied algorithms satisfying

a x k+1 -x k ∇f (x k ) ď f (x k ) -f (x k+1 ).
Since then, this proof has been refined and adapted to different methods and settings [START_REF] Lageman | Pointwise convergence of gradient-like systems[END_REF][START_REF] Attouch | On the convergence of the proximal algorithm for nonsmooth functions involving analytic features[END_REF][START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF][START_REF] Bolte | Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity[END_REF][START_REF] Merlet | Convergence to equilibrium for the backward Euler scheme and applications[END_REF][START_REF] Alaa | Convergence to equilibrium for discretized gradient-like systems with analytic features[END_REF][START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF][START_REF] Bolte | Proximal alternating linearized minimization for nonconvex and nonsmooth problems[END_REF][START_REF] Xu | A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion[END_REF][START_REF] Chouzenoux | Variable metric forward-backward algorithm for minimizing the sum of a differentiable function and a convex function[END_REF][START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF]. From a technical point of view, our proof follows closely the arguments in [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF], adapted to the presence of errors and the variability of the parameters.

First, we prove below Proposition 3.2.4, which summarizes the first step mentioned above. In view to express precisely the capture/stability property we need in Γ η (x, δ), let us introduce an auxiliary property. Given a point x ∈ H, and two radius 0 < r < R < +∞, we say that S(x, r, R) holds if:

i) for each k ∈ N, {x 0 , ..., x k } ⊂ Γ η (x, r) implies x k+1 ∈ Γ η (x, R),
ii) the initial point x 0 belongs to Γ η (x, r) and (3.17)

x

-x 0 + 2 f (x 0 ) -f (x) a 0 + M ϕ(f (x 0 ) -f (x)) + +∞ i=1 ǫ i < r.
Basically, assuming S(x, r, R) means that the initialization x 0 is close enough to x, and that we can control how can x k+1 escape from Γ η (x, r).

Proposition 3.2.4. Let f : H -→ R ∪ {+∞} be a lower semi-continuous function, satisfying the K L inequality in Γ η (x, R). Let (x k ) k∈N be a sequence such that H 0 , H 1 , H 2 and S(x, r, R) hold. Then: i) ∞ k=0 x k+1 -x k < ∞.
ii) The sequence (x k ) k∈N remains in the local upper level set Γ η (x, r), and converges strongly to some x ∞ ∈ cl B(x, r).

iii) lim inf

k→+∞ ∂ F f (x k ) -= 0. iv) f (x ∞ ) ď f (x) = lim k→+∞ f (x k ).
Proof of Proposition 3.2.4. The key point is obtaining the finite length property of item i). For this, we will prove estimations on the sequence by using the involved hypotheses. We start with the following claim: if x k and x k+1 belong to Γ η (x, r), then

(3.18) 2 x k+1 -x k ď x k -x k-1 + 1 a k b k ϕ(f (x k ) -f (x)) -ϕ(f (x k+1 ) -f (x)) + ǫ k .
If x k+1 = x k this inequality holds trivially, so we can assume that x k+1 = x k . Using H 1 and

x k+1 ∈ Γ η (x, r) gives us f (x k ) > f (x k+1 ) ě f (x). With x k ∈ Γ η (x, r) ⊂ Γ η (x, R),
we are allowed to use the K L inequality, together with H 2 , to obtain:

(3.19) ϕ ′ (f (x k ) -f (x)) ě 1 ∂ F f (x k ) - ě b k x k -x k-1 + ǫ k > 0.
As a desingularizing function, ϕ is concave and ϕ ′ is positive, which implies with H 1 and (3.19):

ϕ(f (x k ) -f (x)) -ϕ(f (x k+1 ) -f (x)) ě ϕ ′ (f (x k ) -f (x))(f (x k ) -f (x k+1 )) ě ϕ ′ (f (x k ) -f (x))a k x k+1 -x k 2 ě a k b k x k+1 -x k 2 x k -x k-1 + ǫ k . Whence x k+1 -x k 2 ≤ [ x k -x k-1 + ǫ k ] 1 a k b k ϕ(f (x k ) -f (x)) -ϕ(f (x k+1 ) -f (x)) .
Taking the square root on both sides, and using the fact that 2 √ αβ ď α + β, we obtain (3.18) as desired.

We are now ready to estimate the length of the sequence (x k ) k∈N . For this, we claim that for all K ∈ N * , x K belongs to Γ η (x, r), and (3.20)

K k=1 x k+1 -x k + x K+1 -x K ď x 1 -x 0 + M ϕ(f (x 1 ) -f (x)) -ϕ(f (x K+1 ) -f (x)) + K k=1 ǫ k , where M := sup k∈N * 1 a k b k .
We proceed for this by induction on K.

Initial step: Since it is assumed in S(x, r, R)(ii) that x 0 ∈ Γ η (x, r), we prove with S(x, r, R)(i) that x 1 ∈ Γ η (x, R). Using in particular f (x 1 ) ě f (x), coupled with H 1 , we can write (3.21) x 1 -x 0 ď f (x 0 ) -f (x 1 ) a 0 ď f (x 0 ) -f (x) a 0 .
Using S(x, r, R)(ii) together with the triangle inequality and (3.21), we deduce that

x -x 1 ď x -x 0 + x 0 -x 1 ď x -x 0 + f (x 0 ) -f (x) a 0 < r,
and so x 1 ∈ B(x, r). Now we have x 0 , x 1 ∈ Γ η (x, r), so we can use (3.18) with k = 1, and

H 3 (iii), to obtain 2 x 2 -x 1 ď x 1 -x 0 + M ϕ(f (x 1 ) -f (x)) -ϕ(f (x 2 ) -f (x)) + ǫ 1 .
Induction step: Suppose now that x 1 , . . . , x K ∈ Γ η (x, R), and assume that (3.20) holds. It is immediate from S(x, r, R)(i) that x K+1 ∈ Γ η (x, R). Using successively the triangular inequality, the equations (3.17) and (3.21), and S(x, r, R)(ii), we obtain:

x -x K+1 (3.22) ď x -x 0 + x 0 -x 1 + K k=1 x k+1 -x k ď x -x 0 + 2 x 0 -x 1 + M ϕ(f (x 1 ) -f (x)) -ϕ(f (x K+1 ) -f (x)) + K k=1 ǫ k ď x -x 0 + 2 f (x 0 ) -f (x) a 0 + M ϕ(f (x 1 ) -f (x)) -ϕ(f (x K+1 ) -f (x)) + K k=1 ǫ k < r.
This last inequality (3.22) gives in particular x K+1 ∈ B(x, r), and so x K+1 ∈ Γ η (x, r). Furthermore, if we write (3.18)

with k = K + 1 2 x K+2 -x K+1 ď x K+1 -x K + M ϕ(f (x K+1 ) -f (x)) -ϕ(f (x K+2 ) -f (x)) + ǫ K+1 ,
and add it to (3.22), we obtain the desired result.

We just proved that the sequence (x k ) k∈N remains in the local level set Γ η (x, r). Furthermore, we deduce from (3.20) and the positivity of ϕ that, for all K ∈ N * :

K k=1 x k+1 -x k ď x 1 -x 0 + M ϕ(f (x 1 ) -f (x)) + K k=1 ǫ k .
Because of the assumption H 3 (iv), we can deduce by taking the limit when K → +∞ that

+∞ k=1 x k+1 -x k ď x 1 -x 0 + M ϕ(f (x 1 ) -f (x)) + +∞ k=1 ǫ k ,
which proves item i). In particular, this finite length property entails the strong convergence of the sequence to some x ∞ ∈ H. Since we proved that the sequence remains in Γ η (x, r), item ii) follows. To prove item iii), use

H 2 to write ∞ k=1 b k+1 ∂ F f (x k+1 ) -ď ∞ k=1 x k+1 -x k + ∞ k=1 ε k+1 < +∞,
and conclude with the assumption (b k ) / ∈ ℓ 1 in H 3 . To prove item iv), observe in H 1 that f (x k ) is decreasing, and use the lower semi-continuity of f , to deduce that

f (x ∞ ) ď lim k→+∞ f (x k ). Defining ℓ := lim k→+∞ f (x k ), we need to show that ℓ = f (x). First observe that (x k ) k∈N ⊂ Γ η (x, r) entails directly f (x) ď ℓ.
Moreover, if we assume by contradiction that f (x) < ℓ, the K L inequality together with the fact that ϕ ′ is decreasing would give

ϕ ′ (ℓ -f (x)) ∂ F f (x k ) -ě ϕ ′ (f (x k ) -f (x)) ∂ F f (x k ) -ě 1
for all k ∈ N. This would contradict item iii), so ℓ = f (x).

We are now in position to prove Theorems 3.2.2 and 3.2.3. For this, all we need is verifying the stability assumption S(x, r, R), and apply Proposition 3.2.4. In both cases, the fact that the abstract method described in H 1 and H 2 is a descent method for f is essential.

Proof of Theorem 3.2.2. Recall that a sequence is f -precompact if it admits a f -convergent subsequence. Let x n k --→ k→+∞ x with f (x n k ) --→ k→+∞ f (x). Since f (x k
) is nonincreasing and admits a limit point, we deduce that f (x k ) ↓ f (x). In particular, we have f (x) ď f (x k ) for all k ∈ N. The function f satisfies the K L inequality on Γ η (x, R) for some η ∈]0, +∞] and R > 0, with desingularizing function ϕ. Let K 0 ∈ N be sufficiently large so that f

(x K )-f (x) < min{η, aR 2 }, and pick r ∈]0, R[ such that f (x K ) -f (x) < a(R -r) 2 . Hence, for all k ě K, f (x) ď f (x k+1 ) < f (x) + η and x k+1 -x k ď f (x k ) -f (x k+1 ) a k ď f (x K ) -f (x) a < R -r. Now take K ě K 0 such that x -x K + 2 f (x K ) -f (x) a n K + M ϕ(f (x K ) -f (x)) + +∞ k=K+1 ǫ k < r.
Then, the sequence (y k ) k∈N defined by y k = x K+k satisfies S(x, r, R), and so, the hypotheses of Proposition 3.2.4. In particular, the sequence (x k ) k∈N converges to some x ∞ , and Proposition 3.2.4 gives us lim inf

k→+∞ ∂ F f (x k ) -= 0.
We conclude from the definition of the limiting

subdifferential that 0 ∈ ∂ L f (x ∞ ).
Remark 3.2.5. In the previous proof, it suffices for f to have the K L property at any x such that

x n k --→ k→+∞ x and f (x n k ) --→ k→+∞ f (x).
Proof of Theorem 3.2.3 . Since f has the K L property in x, there is a strict local upper level set Γ η (x, R), where the K L inequality holds with ϕ as a desingularising function. Take r = 2 3 R and, if necessary, shrink η so that

(3.23) 2 η a + M ϕ(η) < r 2
This is possible since ϕ is continuous in 0, with ϕ(0) = 0. Choose now to take x 0 ∈ Γ η (x, r 2 ). It suffices to verify that S(x, r, R) is fulfilled and to use Proposition 3.2.4. It is easy to see, under our assumptions on x 0 and η, that S(x, r, R)(ii) is verified. Indeed, use the monotonicity of ϕ and x 0 ∈ Γ η (x, r

2 ), together with (3.23), to write

x 0 -x + 2 f (x 0 ) -f (x) a 0 + M ϕ(f (x 0 ) -f (x)) < r 2 + 2 η a + M ϕ(η) < r.
For item i), let us assume that x 0 , . . . , x k lie in Γ η (x, r) and prove that x k+1 ∈ Γ η (x, R). Since x is a global minimum, from H 1 and the fact that (f (x k )) k∈N is decreasing, we obtain

f (x) + a x k+1 -x k 2 ≤ f (x k+1 ) + a x k+1 -x k 2 ≤ f (x k ) ď f (x 0 ) < f (x) + η.
In other words,

x k+1 -x k ď η a < r 2 .
It follows, using the inequality above and the triangular inequality, that

x k+1 -x ď x k+1 -x k + x k -x < r 2 + r = R,
and so x k+1 ∈ Γ η (x, R).

Rates of convergence for an abstract descent method

Along this section, it is assumed that (x k ) k∈N is a sequence f -converging to a point x ∞ , where f verifies the K L property with a desingularizing function ϕ. We assume that H 1 , H 2 and H 3 hold, and for simplicity and precision, we restrict ourselves to the case where ε k ≡ 0. We study three types of convergence rate results, depending on the nature of the desingularising function ϕ:

i) Theorem 3.3.1 establishes the relationship between the distance to the limit x kx ∞ and the gap f (x k )f (x ∞ ), for a generic desingularising function. It is similar to the result in [START_REF] Bolte | Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity[END_REF]Theorem 24] for the proximal method in the convex case.

ii) Theorem 3.3.2 gives explicit convergence rates in terms of the parametersboth for the distance and the gapwhen the desingularising function is of the form ϕ(t) = C θ t θ with C > 0 and θ ∈]0, 1]. Several results obtained in the literature for various methods are recovered. In particular, according to [START_REF] Merlet | Convergence to equilibrium for the backward Euler scheme and applications[END_REF], these convergence rates are optimal for θ ∈]0, 1 2 ]. iii) Finally, Theorem 3.3.4 provides convergence rates when H 2 is replaced by a slightly different hypothesis that holds for certain explicit schemes, namely gradient-related methods. This result is valid for a generic desingularising function ϕ. However, when ϕ is of the form ϕ(t) = C θ t θ (C > 0, θ ∈]0, 1]) the prediction is considerably better than the one provided by Theorem 3.3.2.

Distance to the limit in terms of the gap

Theorem 3.3.1. Set φ(t) := max{ϕ(t), √ t}. Then, x ∞ -x k = O ( φ(f (x k-1 ) -f (x ∞ ))) . Proof. By assumption, x k f --→ k→+∞
x ∞ and f satisfies the K L inequality on some Γ η (x ∞ , δ). Let

r k := f (x k ) -f (x ∞ ) ě 0.
We may suppose that r k > 0 for all k ∈ N, because otherwise the algorithm terminates in a finite number of steps. For K large enough, we have

x k ∈ Γ η (x ∞ , δ)
for all k ě K. Then, we can use the estimation (3.18) to write (recall that ǫ k ≡ 0):

2 x k+1 -x k ď x k -x k-1 + M [ϕ(r k ) -ϕ(r k+1 )].
Summing this inequality for k = K, . . . , N , we obtain

N k=K x k+1 -x k ď x K -x K-1 + M ϕ(r K ).
Using the triangle inequality and passing to the limit, we get

x ∞ -x K ď ∞ k=K x k+1 -x k ď x K -x K-1 + M ϕ(r K ) ď f (x K-1 ) -f (x K ) √ a K + M ϕ(r K )
by H 1 . Then, using H 0 , along with the fact that f

(x K ) ě f (x ∞ ) and that (r k ) is decreasing, we deduce that x ∞ -x K ď 1 √ a √ r K-1 + M ϕ(r K-1
), which gives the desired result.

Explicit rates when

ϕ(t) ∝ t θ with θ ∈]0, 1]
We assume now that the desingularizing function of f at x ∞ has the form ϕ(t) = C θ t θ for some C > 0, θ ∈]0, 1]. This holds for instance when f is semi-algebraic, or strongly convex (see the end of Section 3.1.2). Theorem 3.3.2 below is qualitatively analogous to the results in [START_REF] Attouch | On the convergence of the proximal algorithm for nonsmooth functions involving analytic features[END_REF][START_REF] Merlet | Convergence to equilibrium for the backward Euler scheme and applications[END_REF][START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF][START_REF] Xu | A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion[END_REF][START_REF] Bolte | Proximal alternating linearized minimization for nonconvex and nonsmooth problems[END_REF][START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF]: we prove convergence in a finite number of steps if θ = 1, exponential convergence if θ ∈ [ 1 2 , 1[ and polynomial convergence if θ ∈]0, 1 2 [. In the general convex case, finite-time termination of the proximal point algorithm was already proved in [START_REF] Rockafellar | Monotone operators and the proximal point algorithm[END_REF] and [START_REF] Ferris | Finite termination of the proximal point algorithm[END_REF] (see also [START_REF] Peypouquet | Asymptotic convergence to the optimal value of diagonal proximal iterations in convex minimization[END_REF]).

Theorem 3.3.2. Assume ϕ(t) = C θ t θ for some C > 0, θ ∈]0, 1]. i) If θ = 1 and inf k∈N a k b 2 k+1 > 0, then x k converges in finite time. ii) If θ ∈ [ 1 2 , 1[, sup k∈N b k < +∞ and inf k∈N a k b k+1 > 0, there exist c > 0 and k 0 ∈ N such that: ii.a) f (x k ) -f (x ∞ ) = O exp -c k-1 n=k 0 b n+1 , and ii.b) x ∞ -x k = O exp - c 2 k-2 n=k 0 b n+1 . iii) If θ ∈]0, 1 2 [, sup k∈N b k < +∞ and inf k∈N a k b k+1 > 0, there is k 0 ∈ N such that: iii.a) f (x k ) -f (x ∞ ) = O k-1 n=k 0 b n+1 -1 1-2θ
, and

iii.b) x ∞ -x k = O k-2 n=k 0 b n+1 -θ 1-2θ . Remark 3.3.3. Note that a simple sufficient -yet not necessary -condition for inf k∈N a k b 2 k+1 > 0 and inf k∈N a k b k+1 > 0 is that inf k∈N b k > 0.
Proof. As before, we use the notation r k := f (x k )f (x ∞ ) ě 0. We can assume that r k > 0 for all k ∈ N, because otherwise the algorithm would terminate in a finite number of steps. Since x k converges to x ∞ , there exists k 0 ∈ N such that, for all k ě k 0 , x k remains in Γ η (x ∞ , δ), where the K L inequality holds. Using successively H 1 , H 2 and the K L inequality, we obtain

∀k ě k 0 , ϕ ′2 (r k+1 )(r k -r k+1 ) ě ϕ ′2 (r k+1 )a k b 2 k+1 ∂ F f (x k+1 ) 2 -ě a k b 2 k+1 .
Equivalently, using the fact that

ϕ(t) = C θ t θ , ∀k ě k 0 , Cr 2θ-2 k+1 (r k -r k+1 ) ě a k b 2 k+1 . (3.24)
Let us now consider different cases for θ: Case θ = 1: Recall that we assume that r k > 0 for all k ∈ N, and deduced from that (3.24), rewritten here with θ = 1:

∀k ě k 0 , C 2 (r k -r k+1 ) ě a k b 2 k+1 ě inf k∈N a k b 2 k+1 > 0.
Since r k converges, we must have inf k∈N a k b 2 k+1 = 0, which is a contradiction. Therefore, it must exist some k ∈ N such that r k = 0, which means that the algorithm terminates in a finite number of steps. Case θ ∈]0, 1[: This case covers both items ii) and iii). We only have to prove the convergence rates on the values, and the rates for the iterates will follow Theorem 3.

3.1. Write b := sup k∈N b k , m := inf k∈N a k b k+1 and c = m C 2 (1+ b) and, for each k ∈ N, β k := b k m C 2 . For each k ≥ k 0 , (3.24)
can be rewritten, after dividing by C 2 r 2θ-2 k+1 :

(3.25) (r k -r k+1 ) ě a k b 2 k+1 r 2-2θ k+1 C 2 ě β k+1 r 2-2θ k+1 .
Subcase θ ∈ [ 1 2 , 1[: Since r k tends to zero and 0 < 2 -2θ ≤ 1, we may assume, by enlarging k 0 if necessary, that r 2-2θ k+1 ě r k+1 for all k ě k 0 . Inequality (3.25) implies (r kr k+1 ) ě β k+1 r k+1 or, equivalently, r k+1 ď r k 1 1 + β k+1 for all k ě k 0 . By induction, we obtain

r k+1 ď r k 0   k n=k 0 1 1 + β n+1   = r k 0 exp   k n=k 0 ln 1 1 + β n+1   for all k ě k 0 . Moreover, using a classic estimation gives ln 1 1 + β n+1 ď -β n+1 1 + β n+1 ď -1 1 + b β n+1 .
We can deduce the desired convergence rate by using the definitions of β k and c:

r k+1 ď r k 0 exp    k n=k 0 -1 1 + b β n+1    = r k 0 exp   -c k n=k 0 b n+1   . Subcase θ ∈]0, 1 2 [: Recall from inequality (3.25) that r 2θ-2 k+1 (r k -r k+1 ) ě β k+1 . Set φ(t) := C 1-2θ t 2θ-1 . Then φ ′ (t) = -Ct 2θ-2
, and

φ(r k+1 ) -φ(r k ) = r k+1 r k φ ′ (t) dt = C r k r k+1 t 2θ-2 dt ě C(r k -r k+1 )r 2θ-2 k .
We distinguish now two cases. On one hand, if we suppose that r 2θ-2 k+1 ď 2r 2θ-2 k , then

(3.26) φ(r k+1 ) -φ(r k ) ě C 2 (r k -r k+1 )r 2θ-2 k+1 ě C 2 β k+1 .
On the other hand, assume that

r 2θ-2 k+1 > 2r 2θ-2 k . Since 2θ -2 < 2θ -1 < 0, we have 2θ-1 2θ-2 > 0. Thus r 2θ-1 k+1 > qr 2θ-1 k , where q := 2 2θ-1 2θ-2 > 1. Therefore, φ(r k+1 ) -φ(r k ) = C 1 -2θ (r 2θ-1 k+1 -r 2θ-1 k ) > C 1 -2θ (q -1)r 2θ-1 k ě C ′ , with C ′ := C 1-2θ (q -1)r 2θ-1 k 0 > 0. Since β k+1 ď bm C 2 , we can write (3.27) φ(r k+1 ) -φ(r k ) ě C ′ C 2 bm β k+1 . Setting c := min{ C 2 , C ′ C 2
bm } > 0 we can write by using (3.26) and (3.27)

∀k ě k 0 , φ(r k+1 ) -φ(r k ) ě cβ k+1 .
This implies, after using φ(r k 0 ) ě 0,

φ(r k+1 ) ě φ(r k+1 ) -φ(r k 0 ) = k n=k 0 φ(r n+1 ) -φ(r n ) ě c k n=k 0 β n+1 , which is precisely r k+1 ď D k n=k 0 b n+1 -1 1-2θ with D = cm(1-2θ) C 3 -1 1-2θ .

Sharper results for gradient-related methods

Convergence rates for the continuous subgradient differential inclusion

(3.28) u(t) + ∂ F f (u(t)) ∋ 0,
are given in Theorem 3.1.12. For any desingularizing function ϕ on ]0, η[, this result states that for some

t 1 ∈ R, i) f (u(t)) -f (u ∞ ) = O Φ -1 (t -t 1 ) , and ii) u ∞ -u(t) = O ϕ • Φ -1 (t -t 1 ) , where Φ is any primitive of -(ϕ ′ ) 2 on ]0, η[. If the desingularising function ϕ is of the form ϕ(t) = Ct θ θ , we recover (see Remark 3.3.5 later) convergence in finite time if θ ∈] 1 2 , 1], exponen- tial convergence if θ = 1
2 , and polynomial convergence if θ ∈]0, 1 2 [. The same conclusion was established in [START_REF] Bolte | The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF]Theorem 4.7] for a nonsmooth version of (3.28) when f is any subanalytic function on H = R n . This prediction is better than the one given by Theorem 3.3.2 above, as well as the results in [START_REF] Attouch | On the convergence of the proximal algorithm for nonsmooth functions involving analytic features[END_REF][START_REF] Merlet | Convergence to equilibrium for the backward Euler scheme and applications[END_REF][START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF][START_REF] Xu | A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion[END_REF][START_REF] Bolte | Proximal alternating linearized minimization for nonconvex and nonsmooth problems[END_REF][START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF], since it guarantees convergence in finite time for θ > 1 2 . We shall prove that, for certain algorithms including gradient-related methods, this better estimation remains true. To this end, consider the following variant of hypothesis H 2 :

H ′ 2 (Relative error ): For each k ∈ N, b k+1 ∂ F f (x k ) -ď x k+1 -x k . Theorem 3.3.4. Let condition H ′ 2 be satisfied instead of H 2 , and assume m := inf k∈N a k b k+1 > 0. Let Φ :]0, η[→ R be any primitive of -(ϕ ′ ) 2 . i) If lim t→0 Φ(t) ∈ R, then the algorithm converges in a finite number of steps. ii) If lim t→0 Φ(t) = +∞, then there exists k 0 ∈ N such that: ii.a) f (x k ) -f (x ∞ ) = O Φ -1 m k-1 n=k 0 b n+1 , and ii.b) x ∞ -x k = O φ • Φ -1 m k-1 n=k 0 b n+1 , with φ(t) := max{ϕ(t), √ t}.
Proof. The following proof is inspired by the one of [START_REF] Chill | Convergence and decay rate to equilibrium of bounded solutions of quasilinear parabolic equations[END_REF] in the continuous case. First, we claim that, if

r k := f (x k ) -f (x ∞ ) > 0 for all k ∈ N, then there is k 0 ∈ N such that (3.29) ∀k ě k 0 , Φ(r k+1 ) ě Φ(r k 0 ) + m k n=k 0 b n+1 .
To see this, let k 0 be large enough to have

x k ∈ Γ η (x ∞ , δ)
, where the K L inequality holds, for all k ě k 0 . We apply successively H 1 , H ′ 2 , the K L inequality and

H 3 to obtain ϕ ′ (r k ) 2 (r k -r k+1 ) ě ϕ ′ (r k ) 2 a k b 2 k+1 ∂ F f (x k ) 2 -ě a k b 2 k+1 ě b k+1 m. Let Φ be a primitive of -(ϕ ′ ) 2 on ]0, η[.
Then, because ϕ is increasing and concave, and using the assumption r k > 0, we can write For ii), we may suppose that r k > 0 for all k ∈ N (otherwise the algorithm stops in a finite number of steps), and so (3.29) holds for all k ∈ N. Since lim k→+∞ Φ(r k ) = +∞, we can take

Φ(r k+1 ) -Φ(r k ) = r k r k+1 ϕ ′ (t) 2 dt ě (r k -r k+1 )ϕ ′ (r k ) 2 ě b k+1 m. Therefore, Φ(r k+1 ) -Φ(r k 0 ) = k n=k 0 Φ(r n+1 ) -Φ(r n ) ě m
k 0 large enough to have Φ(r k 0 ) > 0, whence Φ(r k+1 ) ě m k n=k 0 b n+1 . Since (b n ) / ∈ ℓ 1 , for all sufficiently large k, m k n=k 0 b n+1 is in the domain of Φ -1
and we obtain the first estimation, namely:

(3.30) r k+1 ď Φ -1   m k n=k 0 b n+1   .
For the second one, since ϕ is concave and differentiable, we have by

H 1 ϕ(r k ) -ϕ(r k+1 ) ě ϕ ′ (r k )(r k -r k+1 ) ě ϕ ′ (r k )a n x k+1 -x k 2 .
The K L property and

H ′ 2 then give ϕ(r k ) -ϕ(r k+1 ) ě m x k+1 -x k , which in turn yields x ∞ -x k ď 1 m ∞ n=k [ϕ(r n ) -ϕ(r n+1 )] ď 1 m ϕ(r k ).
We conclude by using (3.30).

Remark 3.3.5. Let us analyse the results of Theorem 3.3.4 when ϕ(t) has the form C θ t θ . We assume that C = 1, for simplicity. In that case, we have ϕ

′2 (t) = -1 t 2-2θ . • If θ ∈] 1 2 , 1[, then we can write ϕ ′2 (t) = -t α-1 with α := 2θ -1 ∈]0, 1[. Hence, we can take Φ(t) = 1 -1 α t α , such that Φ(t) --→ t↓0
1. This entails the convergence of the sequence in a finite number of steps, which clearly improves Theorem 3.3.2. This convergence in finite time is natural, if ones considers that the convergence rate is governed by Φ -1 , which, in that case, goes to zero in finite time.

• If θ = 1 2 , then we have ϕ ′2 (t) = -1 t . In that case, take Φ(t) =ln(t) --→ t↓0 +∞. Thus, the convergence rates of the values and the iterates are respectively governed by

Φ -1 (t) = exp(-t) and (ϕ • Φ)(t) = exp(- 1 2 t). • If θ ∈]0, 1 2 [, we write ϕ ′2 (t) = -1 t 1+α with α := 1 -2θ ∈]0, 1[. So, consider Φ(t) = 1 α 1
t α , which tends to +∞ when t ↓ 0. We can deduce that, in that case, the convergence rates of the values and the iterates are respectively governed by

Φ -1 (t) = α -1 α t -1 α = O t -1 1-2θ and (ϕ • Φ)(t) = O t -θ 1-2θ .
We clearly recover in the last two points the estimates of Theorem 3.3.2.

x k+1 = x k -λ k ∇f (x k ),
or more generally by gradient-related methods (see Example 3.2.1). At first look, H ′ 2 seems to refer to explicit methods, in opposition with implicit schemes (like the proximal method).

Obtaining convergence in finite time for an explicit scheme is quite surprising, since it is a property that we usually know for implicit methods, under some conditions. Take for instance f = δ C , the indicator function of a closed convex set C, for which the proximal algorithm converges in one single step. More generally, it is known that the proximal method converges in a finite number of steps whenever f is a lower semi-continuous function, which is sharp around its minimum points (see [START_REF] Rockafellar | Monotone operators and the proximal point algorithm[END_REF][START_REF] Ferris | Finite termination of the proximal point algorithm[END_REF][START_REF] Peypouquet | Asymptotic convergence to the optimal value of diagonal proximal iterations in convex minimization[END_REF]):

∃σ > 0, ∀x ∈ argmin f, ∀x ∈ H, f (x) -f (x) ě σ x -x .
In fact, it must be remembered that we also need the H 1 assumption. For explicit methods like in (3.31), we usually derive H 1 from the descent lemma, which asks the function f to be of class C 1,1 . So the question is the following: is it compatible for a function to be at the same time of class C 1,1 , and satisfying the K L inequality with a Lojasiewicz coefficient θ > 1 2 ? We exhibit a simple example illustrating that these two properties are hardly compatible.

Let α ě 1 and f α : R -→ R be defined by f α (x) := α|x| α . Its derivative is given by

f ′ α (x) = |x| α-1 . Clearly, • if α ě 2, f α is of class C ∞ , • if 1 ď α < 2, f α is of class C 1 but f ′ α is not Lipschitz continuous around zero.
Thus, satisfying H 1 would require α ě 2. On an other hand, let us discuss the K L property around zero. We look for a constant C > 0 and a Lojasiewicz exponent

θ ∈]0, 1] such that ∀x ∼ 0, f α (x) 1-θ ď C|f ′ α (x)|, or equivalently, ∀x ∼ 0, α|x| α(1-θ) ď C|x| α-1 .
This can be rewritten as

∀x ∼ 0, |x| 1-αθ ď C α
This boundedness assumption on |x| 1-αθ around zero is satisfied if and only if αθ ď 1. Hence, if we want the K L inequality to hold with θ > 1 2 , we would have in particular that α < 2. From this example, and the fact that explicit gradient methods are not used to converge in a finite number of steps, one can conjecture that the functions satisfying the K L inequality with a Lojasiewicz exponent θ > 1 2 cannot have a Lipschitz continuous gradient2 . Remark 3.4.2 (On the asymptotic equivalence between continuous and discrete dynamics). Doing the comparison between a continuous dynamic and its discrete counterpart is always fruitful, since each of them enjoys its own advantages/difficulties. In the convex setting, the correspondence of the asymptotic behaviour between the proximal algorithm and the steepest descent dynamic has been recently established [START_REF] Peypouquet | Analyse asymptotique de systèmes d'évolution et applications en optimisation[END_REF][START_REF] Alvarez | Asymptotic almost-equivalence of Lipschitz evolution systems in Banach spaces[END_REF][START_REF] Alvarez | A unified approach to the asymptotic almost-equivalence of evolution systems without Lipschitz conditions[END_REF][START_REF] Peypouquet | Evolution equations for maximal monotone operators: asymptotic analysis in continuous and discrete time[END_REF]. In our K L setting, we observed in Section 3.3 that the convergence rates between the continuous and discrete dynamics are the same. Furthermore, it is clear that the proofs of Theorems 3.1.8 and 3.2.2 work on the same arguments. It is thus natural to wonder if some asymptotic equivalence can be established in the K L context, like in the convex case.

In fact, such a correspondence does exist. In [START_REF] Bolte | Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity[END_REF]Theorem 39], the authors consider a K L convex function of class C 1,1 f : H -→ R, which has compact level sets. Thus, they coined a link between the finite length of the trajectories of u(t) + ∇f (u(t)) = 0 and the sequences satisfying the abstract descent scheme

a x k+1 -x k ∇f (x k ) ď f (x k ) -f (x k+1 ).
In [START_REF] Bolte | Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity[END_REF]Remark 40], the authors observe that the convexity assumption can be dropped, provided that the desingularizing function ϕ of f is concave. Recall that this concavity is required in our Definition 3.1.1, so this convex hypothesis on f is not necessary in our setting. It is an interesting problem for the future to investigate whether this asymptotic equivalence can be extended to nonsmooth functions.

Chapter 4

Splitting methods for KL functions

As stressed in [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF], the abstract scheme developed in Chapter 3 covers, among others, the gradient-related methods (a wide variety of schemes based on the gradient method sketched in [START_REF] Cauchy | Méthode générale pour la résolution des systèmes d'équations simultanées[END_REF], see also Example 3.2.1), the proximal algorithm [START_REF] Martinet | Régularisation d'inéquations variationnelles par approximations successives[END_REF][START_REF] Brézis | Produits infinis de résolvantes[END_REF][START_REF] Rockafellar | Monotone operators and the proximal point algorithm[END_REF], and the forward-backward algorithm (a combination of the preceding, see [START_REF] Lions | Splitting algorithms for the sum of two nonlinear operators[END_REF][START_REF] Passty | Ergodic convergence to a zero of the sum of monotone operators in Hilbert space[END_REF]). This last one is a splitting method, used to solve structured optimization problems with the form

(4.1) minimize x∈H f (x) = g(x) + h(x),
where g is a nonsmooth, proper and lower semi-continuous function, and h is differentiable with a L-Lipschitz gradient. Basically, the forward-backward method consists in performing a gradient descent step with respect to h, followed by a proximal step with respect to g. It has been studied in the nonsmooth and nonconvex setting in [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF], and the algorithm was stated as follows: starting with

x 0 ∈ H, consider (λ k ) k∈N ⊂ [λ, λ] with 0 < λ ď λ < 1 L , and ∀k ∈ N (4.2) x k+1 ∈ prox λ k g (x k -λ k ∇h(x k )) .
It satisfies H 1 , H 2 and H 3 (see [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF]Theorem 5.1]) and falls into the setting of Theorem 3.2.2. In section 4.1, we shall extend this class of algorithms in different directions.

We allow the consideration of an alternative choice of metric for the ambient space, which may vary at each step (see [START_REF] Alvarez | Hessian Riemannian gradient flows in convex programming[END_REF][START_REF] Alvarez | Interior proximal algorithm with variable metric for second-order cone programming: applications to structural optimization and support vector machines[END_REF] and the references therein). Let S ++ (H) denote the space of bounded, uniformly elliptic and self-adjoint operators on H. Each A ∈ S ++ (H) induces a metric on H by the inner product x, y A := Ax, y , and the norm x A :=

x, x A . Thus, the proximal operator of f in the metric induced by A is the set-valued mapping prox A f : H ⇒ H, defined as

(4.3) prox A f (x) := argmin y∈H f (y) + 1 2 y -x 2 A .
Observe that prox A f (x) = ∅ if f is weakly lower semi-continuous and bounded from below [21, Theorem 3.2.5], which holds in many relevant applications. If f is the indicator function of a set, then prox A f (x) is the projection mapping relatively to the metric induced by A.

If A = 1 λ id H , we just note prox λf instead of prox 1 λ id H f
. Considering metrics induced by a sequence (A k ) k∈N in S ++ (H), the forward-backward method becomes (4.4)

x k+1 ∈ prox A k g x k -A -1 k ∇h(x k ) .
Observe that (4.4) can be rewritten as

(4.5) x k+1 ∈ argmin y∈H g(y) + h(x k ) + y -x k , ∇h(x k ) + 1 2 y -x k , A k (y -x k ) .
At each step, an approximation of f , replacing its smooth part h by a quadratic model, is minimized. See [START_REF] Chouzenoux | Variable metric forward-backward algorithm for minimizing the sum of a differentiable function and a convex function[END_REF] for a similar algorithm called Variable Metric Forward-Backward, and [START_REF] Noll | Convergence of non-smooth descent methods using the Kurdyka-Lojasiewicz inequality[END_REF] for an approach considering more general models. Note that, when A k = 1 λ k id H , one recovers (4.2). Allowing variable metric can improve convergence rates, help to implicitly deal with certain constraints, or compensate the effect of ill-conditioning. Rather than simply giving a convergence result for a general choice of A k , we handle, in Subsection 4.1.3, a detailed method to select these operators, using a second-order information on h. It can be seen as a generalized Levenberg-Marquardt algorithm, a Newton-like method adapted for nonconvex and nonsmooth functions.

We also want to effectively solve structured problems as (4.6) minimize

x 1 ∈H 1 ,x 2 ∈H 2 f (x 1 , x 2 ) = g 1 (x 1 ) + g 2 (x 2 ) + h(x 1 , x 2 ),
where g 1 , g 2 are nonsmooth, proper and lower semi-continuous functions, and h is differentiable with Lipschitz gradient. One approach is the regularized Gauss-Seidel method, which exploits the fact that the variables are separated in the nonsmooth part of f [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF][START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF][START_REF] Xu | A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion[END_REF]. It consists in minimizing alternatively a regularized version of f with respect to each variable. In other words, it is an alternating proximal algorithm, of the form:

x 1,k+1 ∈ prox f (•,x 2,k ) (x 1,k ) x 2,k+1 ∈ prox f (x 1,k+1 ,•) (x 2,k ) .
However this algorithm does not exploit the smooth nature of h. An alternative is to use an alternating minimization method which can deal with the nonsmooth character, while it benefits from the smooth features.

We present such a method, considering variable metrics, in Section 4.1. More exactly, we revisit the Alternating Forward-Backward methods, already considered in [START_REF] Bolte | Proximal alternating linearized minimization for nonconvex and nonsmooth problems[END_REF][START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF][START_REF] Li | A proximal alternating linearization method for nonconvex optimization problems[END_REF], but we point out that our setting differs from these two works on the following points:

• We allow more flexibility in the choice of parameters, accounting, in particular, for vanishing step sizes or unbounded eigenvalues for the metrics.

• We allow additive errors. Indeed, the computation of xk := x k -A -1 k ∇h(x k ) and x k+1 ∈ prox A k g (x k ) often require solving some subroutines, which may produce xk and x k+1 inexactly. To take these errors into account we introduce two sequences (r k ) k∈N and (s k ) k∈N , and consider (4.7)

x k+1 -s k+1 ∈ prox A k g x k -A -1 k ∇h(x k ) + r k .
Convergence of the Alternating Forward-Backward method with errors is given in Theorem 4.1.6. This result is the consequence of the convergence for the abstract scheme studied in Section 3.2, as it is stated in Proposition 4.1.3.

The Alternating Forward-Backward method provides a framework suitable for the numerical resolution of a wide variety of structured problems. Section 4.2 is devoted to a discussion on some applications of this method. In Section 4.2.1, we adapt the AFB method to Feasability problems involving eventually nonconvex constraints, considering semi-algebraic constraints, or doing a regularity hypothesis on the intersection (see also [START_REF] Lewis | Local linear convergence for alternating and averaged nonconvex projections[END_REF][START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF]). Consider for instance the problems arising in image processing and data compression, which are generally semi-algebraic by nature [START_REF] Donoho | For most large underdetermined systems of linear equations the minimal ℓ 1 -norm solution is also the sparsest solution[END_REF][START_REF] Donoho | Counting the faces of randomly-projected hypercubes and orthants, with applications[END_REF][START_REF] Daubechies | An iterative thresholding algorithm for linear inverse problems with a sparsity constraint[END_REF]. Indeed, they generally involve the semi-algebraic counting function x 0 := ♯{i | x i = 0}, whose proximal operator is easily implementable (see Section 4.2.2.1). Such problems can also involve the rank function, like the sparse and low-rank matrix decomposition problem (see Section 4.2.2.2). They are also well suited for our analysis since the proximal operator of the rank function admits an explicit formulation [START_REF] Hiriart-Urruty | From Eckart and Young approximation to Moreau envelopes and vice versa[END_REF]. We end with some numerical results in Section 4.2.2.3, solving the data compression problem with a seemingly new Hard Shrinkage Projection algorithm. Consider the problem (4.8) minimize

x i ∈H i f (x 1 , . . . , x p ) := h(x 1 , . . . , x p ) + p i=1 g i (x i ),
where h : H -→ R is continuously differentiable and each g i : H i -→ R ∪ {+∞} is a lower semicontinuous function. Moreover, we suppose that there is L ě 0 such that, for each (x 1 , ..., x p ) ∈ H and i ∈ {1, ..., p}, the application (4.9)

x ∈ H i → h(x 1 , ..., x i-1 , x, x i+1 , ..., x p ) has a L-Lipschitz continuous gradient on bounded sets. We shall present an algorithm that generates sequences converging to critical points of f . The sequences will be updated cyclically, meaning that, given (x 1,k , ..., x p,k ), we start by updating the first variable x 1,k into x 1,k+1 , and then we use (x 1,k+1 , x 2,k , ..., x p,k ) to update the second variable, and so on. In order to have concise and clear notations, throughout this section we shall denote: (4.10)

X k := (x 1,k , ..., x p,k ) and X i,k := (x 1,k+1 , ..., x i-1,k+1 , x i,k , ..., x p,k ).

Observe that X 1,k = X k and that we can write X p+1,k = X k+1 .

Let us now present the Alternate Forward-Backward (AFB) algorithm. As said before, it consists in doing a forward-backward step relatively to each variable, taking in account a possibly different metric. Then, for all i ∈ {1, ..., p}, consider a sequence (A i,k ) k∈N ⊂ S ++ (H i ) which will model the metrics. Given a starting point X 0 ∈ H, the AFB algorithm generates a sequence (X k ) k∈N by taking, for all k ∈ N and i ∈ {1, ..., p},

(AFB) x i,k+1 ∈ prox A i,k g i x i,k -A -1 i,k ∇ i h(X i,k ) .
Here ∇ i h denotes the i-th component of ∇h in H = p i=1 H i . We shall consider some hypotheses on the operators A i,k . For A ∈ S ++ (H i ), we set α(A) as the infimum of the spectral values of A, being the best constant x 2 A ě α(A) x 2 for all x ∈ H. We define then α k := min

i=1..p α(A i,k )
and

β k := max i=1..p A i,k
, which give bounds on the spectral values of (A i,k ) i=1..p . We make the following hypotheses on the parameters α k and β k :

(HP) 1. There exists α > 0 such that

α k ě α > L 2. 1 β k / ∈ ℓ 1 3. sup k∈N β k α k+1 < +∞. Remark 4.1.1.
Here HP 1 is a bound on the spectral values by the Lipschitz constant of the gradient of h, in order to enforce the descent property of the sequence. For operators of the form 1 λ i,k id H i , we recover the classical bound Lλ i,k ď L λ < 1. In [START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF], the authors prove that, with an additional convexity assumption on the g i 's, and boundedness of the parameters, one can consider Lλ i,k ď L λ < 2. Item HP 2 states that the spectral values may diverge, but not too fast. Finally, HP 3 can be seen as an hypothesis on the variations of the extreme spectral values of the chosen operators. It clearly holds, for instance, if β k is bounded. It is also sufficient to assume that the condition numbers

κ i,k := A i,k α(A i,k ) are bounded, with also min α k α k+1 , β k β k+1
remaining bounded.

Remark 4.1.2. Even if ∇h is globally Lipschitz continuous, L is not the Lipschitz constant of ∇h, but a common Lipschitz constant for the functions defined in (4.9). As a consequence, the partial gradients ∇ i h are √ pL-Lipschitz continuous while ∇h is pL-Lipschitz.

This allows us to have a better bound in HP 1 which is of particular importance in the applications (see Section 4.2). In [START_REF] Bolte | Proximal alternating linearized minimization for nonconvex and nonsmooth problems[END_REF], the authors give a more precise analysis: at each substep X i,k of the algorithm, they consider L i,k as the Lipschitz constant of the gradient of x ∈ H i → h(x 1,k+1 , ..., x i-1,k+1 , x, x i+1,k , ..., x p,k . Then they take step sizes equal to

λ i,k = ǫ i L i,k
where ǫ i < 1 is a fixed and nonnegative constant. This approach can be related to the one in [START_REF] Chouzenoux | Variable metric forward-backward algorithm for minimizing the sum of a differentiable function and a convex function[END_REF][START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF]. However, they suppose a priori that the values L i,k remain bounded. It would be interesting to know if it is possible to combine the two approaches (a variable Lipschitz constant and vanishing step sizes).

The AFB Method with errors

In order to allow for approximate computation of the descent direction or the proximal mapping, we go further by considering an inexact AFB method. We introduce the sequences (r i,k ) and (s i,k ) for i ∈ {1, ..., p}, which correspond respectively to errors arising at the explicit and implicit steps, relatively to the variable x i . The AFB method with Errors is computed from an initial (x 1,0 , ..., x p,0 ) ∈ H by

(AFBE) y i,k+1 ∈ prox A i,k g i x i,k -A -1 i,k ∇ i h(X i,k ) + r i,k , x i,k+1 = y i,k+1 + s i,k+1 .
We do specific hypothesis on the errors, in view to guarantee the convergence of the method. Observe in particular that we do not assume -a priori-that the errors converge to zero:

(HE) There exists σ ∈ [0, +∞[, ρ ∈]0, 1] with σ+1 ρ < αL -1 such that 1. S i,k ď σ 2 y i,k+1 -y i,k , with S i,k defined from (s i,k ) as in (4.10), 2. r i,k ď σ 2 y i,k+1 -y i,k + µ k , where µ k ě 0 with µ k ∈ ℓ 1 , 3. r i,k + s i,k , y i,k+1 -y i,k A i,k ď 1-ρ 2 y i,k+1 -y i,k 2 A i,k .
This AFB algorithm (with errors) is related to the abstract descent method studied in Section 3.2, as stated in the next proposition. Proof. Since X i,k = Y i,k + S i,k , we can rewrite the algorithm as (4.11)

y i,k+1 ∈ prox A i,k g i (y i,k -A -1 i,k ∇ i h(Y i,k + S i,k ) + r i,k + s i,k
). We start by showing that H 1 is satisfied. Let i ∈ {1, ..., p} be fixed; using the definition of the proximal operator prox A i,k g i in (4.11) and developing the squared norms gives

g i (y i,k )-g i (y i,k+1 ) ě 1 2 y i,k+1 -y i,k 2
A i,k + y i,k+1 -y i,k , ∇ i h(Y i,k +S i,k ) -y i,k+1 -y i,k , r i,k +s i,k A i,k .
Using HE 3 in the inequality above, together with the definition of α k , it results in (4.12)

g i (y i,k ) -g i (y i,k+1 ) ě ρα k 2 y i,k+1 -y i,k 2 + y i,k+1 -y i,k , ∇ i h(Y i,k + S i,k ) .
For fixed k ∈ N and i ∈ {1, ..., p}, introduce the function hi,k :

y i ∈ H i → (y 1,k+1 , .., y i-1,k+1 , y i , y i+1,k , .., y p,k ) ∈ R, which satisfies hi,k (y i,k ) = h(Y i,k ), hi,k (y i,k+1 ) = h(Y i+1,k ) and ∇ hi,k (y i,k ) = ∇ i h(Y i,k
). Applying the descent lemma 2.1.13 to hi,k , we obtain

(4.13) h(Y i+1,k ) -h(Y i,k ) -y i,k+1 -y i,k , ∇ i h(Y i,k ) ď L 2 y i,k+1 -y i,k 2 .
Then, combining (4.12) and (4.13) we get

(4.14) ρα k -L 2 y i,k+1 -y i,k 2 + y i,k+1 -y i,k , ∇ i h(Y i,k + S i,k ) -∇ i h(Y i,k ) ď g i (y i,k ) -g i (y i,k+1 ) + h(Y i,k ) -h(Y i+1,k ),
Using successively the Cauchy-Schwarz inequality, the Lipschitz property of ∇ i h (see Remark 4.1.2) and HE 1 , one gets

- σL 2 √ p y i,k+1 -y i,k 2 ď y i,k+1 -y i,k , ∇ i h(Y i,k + S i,k ) -∇ i h(Y i,k ) .
By inserting this estimation in (4.14), we deduce that

(4.15) ρα k -L( σ √ p + 1) 2 y i,k+1 -y i,k 2 ď g i (y i,k ) -g i (y i,k+1 ) + h(Y i,k ) -h(Y i+1,k ).
By summing (4.15) for i ∈ {1, ..., p}, we obtain

ρα k -L( σ √ p + 1) 2 p i=1 y i,k+1 -y i,k 2 ď p i=1 g i (y i,k ) -g i (y i,k+1 ) + h(Y i,k ) -h(Y i+1,k ), which is exactly H 1 with a k = ρα k -L( σ √ p +1) 2 : ρα k -L( σ √ p + 1) 2 Y k+1 -Y k 2 ď f (Y k ) -f (Y k+1 ).
To prove H 2 , fix i ∈ {1, ..., p} and use Fermat's rule for the Fréchet subdifferential in (4.11) to get: (4.16). The triangle inequality gives (4.17)

(4.16) 0 ∈ ∂ F g i (y i,k+1 ) + {A i,k (y i,k+1 -y i,k ) -A i,k (r i,k + s i,k ) + ∇ i h(Y i,k + S i,k )} . Define w i,k+1 := ∇ i h(Y k ) -∇ i h(Y i,k + S i,k ) -A i,k (y i,k+1 -y i,k ) + A i,k (r i,k + s i,k ), which lies in ∂ F g i (y i,k+1 ) + ∇ i h(Y k+1 ), by
w i,k+1 ď β k ( y i,k+1 -y i,k + r i,k + s i,k ) + ∇ i h(Y i,k + S i,k ) -∇ i h(Y k+1 ) .
Using the error estimations from HE, and the √ pL-Lipschitz continuity of ∇ i h in (4.17), we obtain:

(4.18) w i,k+1 ď (β k (1 + σ) + √ pLσ) y i,k+1 -y i,k + √ pL Y k+1 -Y k + β k µ k .
Define now W k+1 := (w 1,k+1 , ..., w p,k+1 ) ∈ ∂ F f (Y k+1 ) (recall the definition of w i,k+1 ). Then, through the sum over i ∈ {1, ..., p} of inequality (4.18), we deduce (using

√ p ď p ď p 2 ) W k+1 ď p i=1 w i,k+1 ď pβ k µ k + p 2 (β k + L)(1 + σ) Y k+1 -Y k . Hence H 2 is verified with b k+1 = 1 p 2 (1+σ)(β k +L) and ǫ k+1 = β k µ k p(1+σ)(β k +L)
. Now we just need to check that the hypotheses H 3 are satisfied with our hypotheses on α k , β k and µ k . Clearly H 3 (i) holds since we've supposed that α k ě α > (σ

√ p p + 1) L ρ . Then H 3 (ii) asks that b k / ∈ ℓ 1 , which is equivalent to 1 β k +L / ∈ ℓ 1 in our context. This holds since we've supposed that 1 β k / ∈ ℓ 1 . Hypothesis H 3 (iii) is satisfied because β k α k+1 is supposed to be bounded. Finally, H 3 (iv) asks the summability of β k µ k β k +L which is bounded by µ k ∈ ℓ 1 .
Given this result, one could directly apply Theorem 3.2.2 to obtain convergence of the sequence (Y k ) k∈N to a critical point of f . However this result would suffer from some drawbacks. First, we are expecting that (X k ) k∈N converges to a critical point, not (Y k ) k∈N . So we should make the assumption that the errors S k := X k -Y k tend to zero. Moreover we would suppose that (Y k ) is f -precompact, while we may only have an access to (X k ). To handle this, we make the link between the asymptotic behaviour of (Y k ) and (X k ): Proposition 4.1.4. For any bounded sequence generated by the AFB method with errors satisfying HP and HE:

i) If (Y k ) k∈N has finite length, then so does (X k ) k∈N . ii) If (f (Y k )) k∈N is bounded from below, then for all i ∈ {1, ..., p}, s i,k and r i,k lie in ℓ 2 .
In particular, (Y k ) k∈N and (X k ) k∈N share the same limit points.

iii) (Y k ) k∈N is precompact if and only if (f (Y k )) k∈N is bounded from below and (X k ) k∈N is precompact.

Proof. Item i) comes directly from HE 1 . To prove item ii), we use Proposition 4.1.3: from H 1 and H 3 (i), we have

(4.19) a Y k+1 -Y k 2 ď f (Y k ) -f (Y k+1 ), whence (f (Y k ))
a k∈N Y k+1 -Y k 2 ď f (Y 0 ) -inf k∈N f (Y k ) < +∞.
Since we have r i,k ď σ 2 y i,k+1 -y i,k +µ k where µ k ∈ ℓ 1 , and

y i,k+1 -y i,k ď Y k+1 -Y k which is in ℓ 2 ,
we deduce that r i,k ∈ ℓ 2 , and the same holds for s i,k . So the errors converge to zero, and (X k ) and (Y k ) have the same limit points. Item iii) follows from item ii) and the following argument: suppose that we have a subsequence (Y n k ) converging to some

Y ∞ = (y 1,∞ , ..., y p,∞ ) in H. Since f is lower semi-continuous and (f (Y k )) is decreasing, we have that inf k∈N f (Y k ) is bounded from below by f (Y ∞ ).
An other disadvantage of the direct application of Theorem 3.2.2 to Proposition 4.1.3 is that it asks the f -precompactness of (Y k ) k∈N . In some cases, precompactness of a sequence can be deduced using compact embeddings between Hilbert spaces. Sequences remaining in a sublevel set of an inf-compact function f are also precompact. However, f -precompactness is harder to obtain without further continuity assumption on f . Actually, both limit and f -limit points coincide whenever the parameters are bounded: Proposition 4.1.5. If either β k ď β, or f is continuous on its domain, then (Y k ) k∈N is f -precompact if and only if it is precompact.

Proof. Suppose that we have Y kn converging to Y ∞ , and show that f (Y kn ) converges also to f (Y ∞ ). Since f (Y k ) is decreasing and f is lower semi-continuous, we know that Y ∞ must lie in the domain of f . If f is continuous on its domain, the conclusion is immediate. On the other hand, suppose that β k ď β. Since h is continuous, we only need to verify that lim n→+∞ g i (y i,kn ) = g i (y i,∞ ) for each i ∈ {1, ..., p}. By lower semi-continuity of g i , we just have to prove that lim sup n→∞ g i (y i,kn ) ď g i (y i,∞ ), following the ideas of [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF].

Let n ∈ N * and k = k n -1; by using the definition of the proximal operator, we have

g i (y k+1 i ) + 1 2 y i,k+1 -y i,k + A -1 i,k ∇ i h(Y i,k + S i,k ) -r i,k -s i,k 2 A i,k ď g i (y ∞ i ) + 1 2 y ∞ i -y i,k + A -1 i,k ∇ i h(Y i,k + S i,k ) -r i,k -s i,k 2
A i,k ,
and the latter implies (using Cauchy-Schwarz and A i,k ď β): (4.21)

g i (y i,k+1 ) ď g i (y i,∞ ) + β 2 y i,∞ -y i,k 2 + y i,∞ -y i,k+1 ∇ i h(Y i,k + S i,k ) + β r i,k + s i,k .
Now recall that y i,k+1 = y i,kn tends to y i,∞ , while r i,k + s i,k goes to zero (see Proposition 4.1.4).

Observe also that (4.20)). Passing to the upper limit in (4.21) leads finally to lim sup

∇ i h(Y i,k + S i,k ) is bounded since Y i,k+1 + S i,k converges to Y ∞ . Moreover, y i,∞ -y i,k goes to zero since y i,∞ -y i,k ď y i,∞ -y i,kn + y i,k+1 -y i,k , with y i,kn → y i,∞ and y i,k+1 -y i,k ∈ ℓ 2 (see
n→+∞ g i (y i,kn ) ď g i (y i,∞ ).
As a direct consequence of Propositions 4.1.3, 4.1.4, 4.1.5, together with Theorem 3.2.2, we finally obtain our convergence result for the AFB algorithm with errors. It extends the results of [START_REF] Chouzenoux | A block coordinate variable metric forwardbackward algorithm[END_REF] (when taking a cyclic permutation on the variables) in two directions: the functions g i need not be continuous on their domain, or the step sizes can tend to 0. Theorem 4.1.6. Let f be a K L function. Let (Y k ) k∈N be a bounded precompact sequence generated by the AFB algorithm with errors, with HP and HE satisfied. Suppose that either β k remains bounded, or that f is continuous on its domain. Hence, the sequence (X k ) k∈N has finite length, and converges toward a ∂ L -critical point of f . Remark 4.1.7. In the particular case where S k ≡ 0, we know furthermore that the sequence (X k ) is convergent with respect to f . This is no longer true in general if f is not continuous and S k = 0. As a simple counterexample, take f :

x ∈ R → |x| 0 ∈ R, where |x| 0 = 0 if x = 0, |x| 0 = 1 else. By taking as parameters A k ≡ 2id, r k ≡ 0, s k = 1
k and x 0 = 0, it is easy to see, after applying the AFB algorithm, that f (y

k ) ≡ 0 but f (x k ) ≡ 1.
An analog of the capture result in Theorem 3.2.3 can also be deduced: Theorem 4.1.8. Suppose that the K L property holds at a global minimum X of f . Let (X k ) k∈N be a bounded sequence generated by the AFB algorithm with errors, satisfying HP and HE with µ k ≡ 0. Hence, there exist δ > 0 and η > 0 such that, if X 0 ∈ Γ η ( X, δ), then (X k ) k∈N has finite length and converges to a global minimum of f .

To prove this theorem, it suffices to use Y 0 = X 0 , and to see at the end of the proof of Proposition 4.1.3 that µ k = 0 iff ǫ k = 0, where ǫ k is the parameter involved in H 3 . Then, apply Theorem 3.2.3 together with Propositions 4.1.3 and 4.1.4.

Variable metric: Towards generalized newton methods

We focus here on the problem of minimizing a C 1,1 function h : R N -→ R over a closed and nonempty set C ⊂ R N . The AFB algorithm reduces in this case to a projected-gradient method, and allow us to compute in the explicit step a descent direction governed by a chosen metric A k . As an example, take h(x) = 1 2 Ax, xb, x with A ∈ S ++ (R N ) being a symmetric definite positive matrix. In the unconstrained case, the Newton method (that is, taking A k ≡ A) is known to solve in a single step the problem. If we add a constraint C, it is easy to see that the Newton-projected method (4.22)

x k+1 ∈ proj A k C x k -A -1 k ∇h(x k )
gives the minimum of h over C in one single step. For a general function h, (4.22) reduces to the minimisation over C of a quadratic model of h, as stressed in (4.5). One can see on this example that computing the proximal operator relatively to the metric A n , used in the explicit step, (and not the ambient metric !) is of crucial importance in this method.

The spirit here is to use second-order information from h in order to improve the convergence of the method. In the unconstrained case, a popular choice of metric is given by Newton-like methods, where the metric at step k is induced by (an approximation of) the Hessian ∇ 2 h(x k ). Since it is often impossible to know in advance whether or not the Hessian is uniformly elliptic at each x k , a positive definite approximation has to be chosen. We detail here a natural way to chose this positive definite A k ∼ ∇ 2 h(x k ) in closed loop, and show that this method remains in the setting of Theorem 4.1.6. Since it generalizes the Levenberg-Marquardt method used in the convex case (see [START_REF] Attouch | A continuous dynamical Newton-like approach to solving monotone inclusions[END_REF]), we will refer to the Generalized Levenberg-Marquardt method for this way of designing A k . One of the interesting aspect of the method is that such a matrix can be defined even if h is only C 1,1 and not C 2 , since the differentiability of ∇h is not necessary in Theorem 4.1.6. Another interesting aspect is that the splitting approach led us to solve constrained minimization problems with a Newton-projected approach.

We set S + (R N ) as the closed convex cone of nonnegative N × N matrices. Consider the generalized Hessian of h, by taking the generalized Jacobian of ∇h in the sense of Clarke:

∂ 2 h(x) := co{ lim n→+∞ ∇ 2 h(x n ),
where ∇h is differentiable at x n and x n → x}.

This set contains symmetric matrices bearing second-order information on h. Hence, the Generalized Levenberg-Marquardt method to compute

A k ∈ S ++ (R N ) from a given x k ∈ R N is the following: for ε > 0, Take H k ∈ ∂ 2 h(x k ), Project P k := proj S + (R N ) (H k ), Regularize A k := P k + εI N .
A globalized version of the method can be considered by taking step sizes ensuring descent. Then, the following convergence result holds:

Proposition 4.1.9. Let f (x) := h(x) + δ C (x) be a K L function, where C ⊂ R N is closed and nonempty, and h is differentiable with a L-Lipschitz gradient. Let x 0 ∈ H and suppose that (x k ) is a bounded sequence generated by

x k+1 ∈ proj A k C x k -λ k A -1 k ∇h(x k )
, where A k is selected with the Generalized Levenberg-Marquardt process detailed above, and the stepsizes λ k satisfy:

0 < λ k ď λ < ε L , λ k / ∈ ℓ 1 and sup k∈N λ k+1 λ k < +∞.
Then, the sequence (x k ) k∈N has finite length and is converging to a critical point of f .

Proof. Start by observing that proj

A k C = proj λ -1 k A k C
, so the algorithm falls in the setting of the AFB algorithm. According with the previous notations, ∇h being L-Lipschitz continuous implies that the sequence ( H k ) k∈N is bounded by L, and so ( P k ) k∈N remains bounded by 2L. To conclude through Theorem 4.1.6, we just need to check the hypotheses HP on the parameters

1 λ k A k . We have here α k = α( 1 λ k A k ) ě ελ k -1 ě ε λ-1 > L and β k = 1 λ k A k ď (2L + ǫ)λ -1 k .
Thus HP 1 is satisfied, while items HP 2 and HP 3 follow directly from the hypotheses made on (λ k ) k∈N . Since the indicator function δ C is continuous on its domain, the hypotheses of Theorem 4.1.6 are satisfied. This extends, in a way, results from the convex setting to the nonconvex one, enforcing moreover the strong convergence (see [START_REF] Attouch | A continuous dynamical Newton-like approach to solving monotone inclusions[END_REF]Theorem 7.1]).

A drawback of this method is that the Hessian increases the complexity of implementation since a matrix must be inverted in the explicit step. An alternative is the Broyden-Fletcher-Goldfarb-Shanno (BFGS) update scheme (see [START_REF] Bertsekas | Nonlinear programming[END_REF], [START_REF] Sra | Optimization for machine learning[END_REF]), using only first-order information to compute the inverse of the Hessian. On the other hand, the implicit step gains also in complexity since one must project onto a constraint relatively to a given metric, which is nontrivial even for simple constraints. For linear constraints, a particular second-order model of the Hessian can be taken in order to reduce the implicit step in a trivial orthogonal projection step (see [START_REF] Sra | Optimization for machine learning[END_REF][START_REF] Gafni | Two-metric projection methods for constrained optimization[END_REF][START_REF] Bertsekas | Projected Newton methods for optimization problems with simple constraints[END_REF]).

Newton-like methods are expected to have good convergence rates in exchange for a more expensive implementation. An interesting question is whether one can obtain convergence rates beyond the results in Subsection 3.3, by exploiting, not only the K L nature of the function, but also the specific properties of the matrices selected by the Generalized Levenberg-Marquardt process.

Some problems arising in the KL context 4.2.1 Nonconvex Feasibility

We present here a method to solve

Find x ∈ s i=1 C i ,
where each C i is a closed nonempty subset of a Hilbert space H. This problem is called a feasibility problem. We suppose that each constraint C i is well-known: we can easily access its elements, and the projection operator proj C i take nonempty values and can be computed. We emphasize the fact that the sets might be non-convex, which implies that proj C i can be set-valued. A natural approach to solve this problem consists in designing an iterative scheme involving projections onto each constraint C i .

For instance, we can chose to successively project the current iterate onto one constraint, chosen in a cyclic way. This is the alternate projection method, designed by von Neumann for closed subspaces [START_REF] Neumann | Functional operators: The geometry of orthogonal spaces[END_REF]. There is an abundant literature on this algorithm, see for instance [START_REF] Bauschke | On the convergence of von Neumann's alternating projection algorithm for two sets[END_REF][START_REF] Bauschke | On projection algorithms for solving convex feasibility problems[END_REF][START_REF] Deutsch | Best approximation in inner product spaces[END_REF] in the convex setting, and more recently [START_REF] Lewis | Local linear convergence for alternating and averaged nonconvex projections[END_REF][START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF] for tame constraints. An other approach is the averaged projection algorithm, which works in two steps: first, project in parallel the current iterate onto each constraint, and then, define the new iterate as a convex combination of each projected point. The first convergence proof for this method is due to Auslender [START_REF] Auslender | Méthodes numériques pour la résolution des problèmes d'optimisation avec contraintes[END_REF], see also [START_REF] Auslender | Optimisation: Méthodes numériques[END_REF][START_REF] De Pierro | A parallel projection method for finding a common point of a family of convex sets[END_REF][START_REF] Crombez | Image recovery by convex combinations of projections[END_REF][START_REF] Bauschke | On projection algorithms for solving convex feasibility problems[END_REF][START_REF] Lewis | Local linear convergence for alternating and averaged nonconvex projections[END_REF]. There exists various other methods to solve the feasibility problem, let us mention for instance Dykstra's method [START_REF] Deutsch | Best approximation in inner product spaces[END_REF]Chapter 9], or Haugazeau's method [START_REF] Haugazeau | Sur les inéquations variationnelles et la minimisation de fonctionnelles convexes[END_REF][START_REF] Bauschke | A weak-to-strong convergence principle for Fejérmonotone methods in Hilbert spaces[END_REF][START_REF] Combettes | A block-iterative surrogate constraint splitting method for quadratic signal recovery[END_REF]. In this section, we will focus on relaxations and combinations of the averaged and alternate projection methods, invoking special instances of the Alternate Forward-Backward algorithm. In Theorem 4.2.8, we obtain a local convergence result.

We chose to approach this feasibility problem by introducing a penalization function. Consider for instance the following weighted sum of the distance functions to the constraints:

(4.23) minimize x∈H 1 2 s i=1 w i dist 2 (x, C i ) where w i ∈]0, 1[, m i=1 w i = 1.
If the functions x → 1 2 dist 2 (x, C i ) are of class C 1,1 on bounded sets (for instance if the sets are convex), we fall into the setting of the AFB method. Their gradient are x → xproj C i (x), which are 1-Lipschitz continuous, so the algorithm is written as follows:

(4.24) It might happen that the functions dist 2 (•, C i ) are not regular enough, in which case we say that the constraints are hard (in the previous case, we can talk about soft constraints). In that case, there is an other way to turn the original feasibility problem into a minimization problem solvable by the AFB method. It suffices to separate the variables, and solve (4.25) minimize

x k+1 = (1 -λ k )x k + λ k s i=1 w i proj C i (x k ) , with (λ k ) k∈N in [λ, λ] ⊂]0, 1[. It
(x 1 ,...,xs)∈H s s i=1 δ C i (x i ) + 1 4(s -1) 1ďi,i ′ ďs x i -x i ′ 2 .
The AFB method is written in that case as:

(4.26) x i,k+1 ∈ proj C i (1 -λ i,k )x i,k + λ i,k x 1,k+1 + .. + x i-1,k+1 + x i+1,k + .. + x s,k s -1 , with (λ i,k ) k∈N in [λ, λ] ⊂]0, 1[.
Here, we obtain an algorithm combining averaging and parallel projections.

In both cases, if the function to minimize satisfies the K L property, Theorem 3.2.3 guarantees the convergence to a solution, if the starting point is close enough to the solution set. The behaviour of the algorithms (4.24) and (4.26) is described in the following simple example.

x k+1 = (1 -λ k ) + λ k proj C 1 (x k ) + proj C 2 (x k ) 2 .
If we take λ k close to one, this scheme is very close to an averaged projection algorithm. It is illustrated in Figure 4.1, where we take λ k ≡ 0.9. The algorithm (4.26) looks more to an alternating projection method. It generates two sequences (x k ) k∈N and (y k ) k∈N , defined by

x k+1 = proj C 1 ((1 -λ 1,k )x k + λ 1,k y k ), y k+1 = proj C 2 ((1 -λ 2,k )y k + λ 2,k x k+1 ).
It is illustrated in Figure 4.1, where we also take λ 1,k = λ 2,k ≡ 0.9. Remark 4.2.2. At first glance, it seems that the alternating projection method is slightly faster than the averaged projection method. This fact is observed in [START_REF] Lewis | Local linear convergence for alternating and averaged nonconvex projections[END_REF]Remark 7.5], under some regularity assumptions on the intersection. Remark 4.2.3. In Example 4.2.1, we take m = 2, and observe that the algorithm (4.26) is very close to the classic alternating projection algorithm. Nevertheless, this is no more the case when m ě 3, even if λ i,k = 1. We do not know if it is possible to find a functional f for which the AFB method would lead to a 'pure' alternating projection algorithm. According to a recent result of Baillon-Combettes-Cominetti [START_REF] Baillon | There is no variational characterization of the cycles in the method of periodic projections[END_REF], we are tempted to conjecture that such a function does not exist. Remark 4.2.4. When the distance functions dist 2 (x, C i ) are not of class C 1,1 , we said that we cannot use the averaged projection method (4.24). In fact, there exists an other penalization leading to an averaged projection-like method, and which is tractable even if the functions dist 2 (x, C i ) are not regular. It suffices to consider

(4.27) minimize X,Y ∈H s δ C (X) + δ ∆ (Y ) + 1 2 X -Y 2 ,
where

C := s i=1 C i and ∆ := {(x, . . . , x) ∈ H s | x ∈ H}. Using the fact that proj C (X) = s i=1 proj C i (x i ) and proj ∆ (X) = 1 s ( s i=1 x i , . . . , s i=1 x i ),
the AFB method applied to (4.27) writes as

x i,k+1 = proj C i ((1 -λ k )x i,k + λ k y k ) for all i ∈ {1, ..., s}, y k+1 = (1 -µ k )y k + µ k s i=1 1 s x i,k+1 .
This algorithm can also be seen as a relaxation of the averaged projection algorithm.

We propose in the following a method to solve mixed feasibility problems, involving both soft and hard constraints. This will result in an algorithm combining the averaged projection algorithm and the alternate projection method.

Mixed feasibility problem

Let s ě 1, p ě 0 and C 1 , ..., C s , D 1 , ..., D p be a family of weakly closed nonempty subsets of H, having nonempty intersection. Consider the problem:

(4.28) Find x ∈ S := C 1 ∩ .. ∩ C s ∩ D 1 ∩ .. ∩ D p .
Here the C i 's plays the role of the hard constraints (no regularity will be imposed), and we will always suppose that s ě 1, taking if necessary C 1 = H. The D j 's are assumed to be prox-regular: a prox-regular set D ⊂ H is a closed set admitting for each x ∈ D a neighborhood U such that the projection proj D : U → D is well defined (single-valued) and strong-to-weak continuous. For instance convex sets, as well as manifolds of class C 2 , are prox-regular. The prox-regular sets are interesting in our setting since the corresponding distance functions enjoy a good regularity property: Proposition 4.2.5. [START_REF] Poliquin | Local differentiability of distance functions[END_REF] Let D ⊂ H be a nonempty closed prox-regular se. Then, for all x ∈ D, there exists δ > 0 such that: i) the projection proj D (x) reduces to one element for all x ∈ B(x, δ),

ii) the function h(x) := 1 2 dist 2 (x, D) is of class C 1,1 on B(x, δ
), with a 1-Lipschitz-continuous gradient given by ∇h(x) = xproj D (x).

To solve (4.28), we introduce a cost function, penalizing the prox-regular sets with their distance functions, and the hard constraints with their indicator functions:

(4.29) f (x 1 , ..., x s ) = s i=1   δ C i (x i ) + 1 2 p j=1 w j dist 2 (x i , D j ) + 1 4 s i ′ =1 w x i -x i ′ 2   .
Here w, w

) := s i=1   1 2 p j=1 w j dist 2 (x i , D j ) + 1 4 s i ′ =1 w x i -x i ′ 2   .
Note nevertheless that, under the prox-regularity assumption, this function h is not differentiable on the whole space H s , but only in a neighborhood of the solution set:

∈ {1, ..., s}, ∀(x 1 , ..., x s ) ∈ H s , ∇ i h(x 1 , ..., x s ) = x i - p j=1 w j proj D j (x i ) - i ′ =i wx i ′ ,
and satisfy the following Lipschitz property:

∀(x 1 , ..., x s ), (y 1 , ..., y s ) ∈ H s , ∇ i h(x 1 , ..., x s ) -∇ i h(y 1 , ..., y s ) ď s i=1
x iy i .

Proof. For each j ∈ {1, . . . , p}, let U j be the neighborhood of x given by (4.2.6) for the function

x ∈ H → 1 2 dist 2 (x, D j ). Thus, we can take U = p j=1 U j on which h is continuously differentiable.

The computation of ∇ i h is direct, coming directly from (4.2.6) and (s -1)w + p j=1 w j = 1. Now, let x := (x 1 , . . . , x s ) and y := (y 1 , . . . , y s ) be in U s . Using the definition of ∇ i h together with the triangular inequality and Proposition 4.2.6, one obtains

∇ i h(x) -∇ i h(y) = w(s -1)(x i -y i ) + p j=1 w j x i -proj D j (x i ) -y i + proj D j (y i ) - i ′ =i w(y i ′ -x i ′ ) ď w(s -1) x i -y i + p j=1 w j x i -proj D j (x i ) -y i + proj D j (y i ) + i ′ =i w y i ′ -x i ′ ď w(s -1) x i -y i + p j=1 w j x i -y i + i ′ =i w y i ′ -x i ′ .
Using again (s -1)w + p j=1 w j = 1 and w ď 1 we obtain:

∇ i h(x) -∇ i h(y) ď x i -y i + w i ′ =i y i ′ -x i ′ ď s i=1 x i -y i .
We can now define an alternating average projection algorithm, by applying the AFB method to (4.29). Each iteration of this algorithm consists in two steps: first, we compute a convex combination of the points lying in the hard constraints with some projections on the proxregular sets. Next, we project onto a hard constraint, cyclically. More precisely, start with (x 1,0 , ..., x s,0 ) ∈ s i=1 C i and for all k ∈ N, i ∈ {1, ..., s}, take λ i,k ∈ [λ, λ] ⊂]0, 1[ and apply (4.31)

x i,k+1 = proj C i   (1 -λ i,k )x i,k + λ i,k   i ′ <i wx i ′ ,k+1 + p j=1 w j proj D j (x i,k ) + i ′ >i wx i ′ ,k    
The idea to obtain local convergence for this algorithm is to use Proposition 4.1.4 together with Theorem 3.2.3, under the hypothesis that f is K L and that the starting points are sufficiently close to an element of S. Nevertheless, the hypotheses of Proposition 4.1.4 are not directly satisfied here since in this case the gradient of h is not Lipschitz continuous on bounded sets but only in a neighborhood of any point of S. Then we shall verify that the sequence generated by this algorithm stays in an appropriate neighborhood of S. This is the purpose of the following technical lemma: Lemma 4.2.7. Let x ∈ S and U its neighborhood given by the previous proposition. We note z = (x, ..., x) ∈ S s . Then, for any R > 0 satisfying B(x, R) ⊂ U , there exists r < R such that the stability hypothesis S(z, r, R)(i) holds for the algorithm given by (4.31), provided that (x 1,0 , . . . , x s,0 ) is taken close enough to z.

Proof. Let us start by defining a 'good' radius r < R. Given arbitrary r > 0 and R 1 > 0, we define (R 2 , ..., R s ) by: R i+1 = 4s(s + p)R i + (2 + 4s(s + p))r, for i ∈ {2, . . . , s}.

This progression being arithmetic-geometric, we can choose r and R 1 small enough to have

r < R 1 and R s < R √ s .
By definition of these numbers, we easily see that R 1 < ... < R s . Moreover, we have this estimation which will be useful in the following:

(4.32) for all 1 ď i < s, (R i+1 -2r) 2 8s(s + p) ě (R i+1 -2r) 2 16s(s + p) = (r + R i ) 2 .
Set X := (x, ..., x) and, for all k ∈ N, let X k = (x 1,k , . . . , x s,k ). We assume that the initial point (x 1,0 , . . . , x s,0 ) is close enough to X, in the following sense (recall that f is continuous):

f (X 0 ) < (R 1 -2r) 2 32(s + p)
and ∀i ∈ {1, ..., s}, x 0,ix < r.

In view to prove S( X, R, r)(i), we assume that X 0 , ..., X k lies in B( X, r), and show that X k+1 ∈ B( X, R).

We start by giving a first estimation on y i,k+1x i,k , where y i,k+1 is defined by:

y i,k+1 = (1 -λ i,k )x i,k + λ i,k   p j=1 w j proj D j (x i,k ) + i ′ <i wx i ′ ,k+1 + i ′ >i wx i ′ ,k   .
Using the triangle inequality and the fact that λ i,k , w and w j are strictly smaller than 1 for each i ∈ {1, . . . , s} and j ∈ {1..p}, one obtains:

y i,k+1 -x i,k 2 ď p j=1 w j (x i,k -proj D j (x i,k )) + i ′ <i w(x i,k -x i ′ ,k+1 ) + i ′ >i w(x i,k -x i ′ ,k ) 2 ď (s + p) p j=1 w j x i,k -proj D j (x i,k ) 2 + i ′ <i x i,k -x i ′ ,k+1 2 + i ′ >i w x i,k -x i ′ ,k 2 Since x i,k -proj D j (x i,k ) 2 = dist 2 (x i,k , D j ), we can write that (4.33) y i,k+1 -x i,k 2 ď 4(s + p)f (z k ) + (s + p) i ′ <i x i,k -x i ′ ,k+1 2 .
We assumed that X 0 , ..., X k ∈ B( X, r), where f is of class C 1,1 . We can use the descent lemma 4.1.4, as in the proof of Proposition 4.1.3, to obtain that f

(X k ) ď f (X k ), where f (X 0 ) < (R 1 -2r) 2 32(s + p)
. Recalling (4.33), we deduce (4.34)

y i,k+1 -x i,k 2 ď (R 1 -2r) 2 8 + (s + p) i ′ <i x i,k -x i ′ ,k+1 2 .
We shall prove now that x i,k+1x ď R i , by using a recurrence on i ∈ {1, . . . , s}.

If i = 1 then, by (4.34), we have

y 1,k+1 -x 1,k 2 ď (R 1 -2r) 2 8 ď (R 1 -2r) 2 4 , so (4.35) y 1,k+1 -x 1,k ď (R 1 -2r) 2 .
On the other hand, x 1,k+1 being the projection of y 1,k+1 onto C 1 ∋ x, we have x k+1 1 x ď 2 y 1,k+1x . This last inequality, combined with (4.35), gives:

x 1,n+1 -x ď 2 y 1,k+1 -x 1,k + 2 x 1,k -x ď (R 1 -2r) + 2r = R 1 .
Suppose now that x i ′ ,k+1x ď R i ′ for any i ′ < i. In inequality (4.34) we have, using the recurrence hypothesis, that for all i ′ < i,

x i,k -x i ′ ,k+1 ď x i,k -x + x -x i ′ ,k+1 ď r + R i ′ .
Thus (recalling the monotonicity of the R i ′ 's):

i ′ <i x i,k -x i ′ ,k+1 2 ď i ′ <i (r + R i ′ ) 2 ď i ′ <i (r + R i-1 ) 2 ď s(r + R i-1 ) 2
where s(r

+ R i-1 ) 2 ď (R i -2r) 2 8(s + p) by definition of R i .
Combining this last inequality with (4.34)

gives

y i,k+1 -x i,k 2 ď (R 1 -2r) 2 8 + (R i -2r) 2 8 , where (R 1 -2r) 2 8 ď (R i -2r) 2 8 since r < R 1 < R i . Thus y 1,k+1 -x 1,k ď (R i -2r) 2
, and using exactly the same arguments as for the case i = 1 we can deduce that x i,k+1x ď R i . Hence, we obtain

X k+1 -X 2 = s i=1 x i,k+1 -x 2 ď s i=1 R 2 i ď sR 2 s < R 2 ,
which completes the proof.

We

m i=1 x * i 2 ď α m i=1 x * i holds, for all x i ∈ F i ∩ B(x, δ) and x * i ∈ N L F i (x i ), i ∈ {1, ..., m}.
If the intersection S is strongly regular at any of its points, we just say that it is strongly regular. Remark 4.2.10. Note that this notion is local, which means that the strong regularity at x is affected by the behaviour of the limiting normal cones N L F i at x ∼ x. Observe also that, in finite dimension, we can equivalently replace the limiting normals by the Fréchet ones in Definition 4.2.9. Remark 4.2.11. In its survey on regularity of collections of sets, Kruger [START_REF] Kruger | About regularity of collections of sets[END_REF] defines the strong regularity of

F 1 , ..., F m at x ∈ F 1 ∩ • • • ∩ F m as follows: there exists R > 0 and α > 0 such that ∀x i ∈ F i ∩ B(x, R), ∀r ∈]0, R], ∀ε i ∈ B(0, αr), m i=1 (F i -x i -ε i ) ∩ B(0, r) = ∅.
Actually, these two notions of strong regularity are the same in finite dimensions spaces (see [214, Proposition 10, Corollary 2] and Remark 4.2.10).

In finite dimensions, the strong regularity also coincides with a well-known notion: Proposition 4.2.12. Let F 1 , . . . , F m be a family of nonempty closed subsets of H, and assume that H has finite dimension. Let x ∈ F 1 ∩• • •∩F m . Then, the strong regularity of the intersection at x is equivalent to the following linear regularity:

(4.37) ∀x * i ∈ N L F i (x), m i=1 x * i = 0 ⇒ x * 1 = • • • = x * m = 0.
Proof. Assume that the intersection is strongly regular at x, i.e. there exists α > 0 and δ > 0 so that (4.36) holds.

Let (x * 1 , . . . , x * m ) ∈ m i=1 N L F i (x) such that m i=1
x * i = 0. By definition of the limiting normal cone, there exists sequences

(x i,k ) k∈N ⊂ F i and (x * i,k ) k∈N ⊂ H such that x * i,k ∈ N F F i (x i,k ), with x i,k F i --→ k→+∞ x and x * i,k • --→ k→+∞ x * i .
Because of the convergence of x i,k to x in F i , we can assume, taking eventually a subsequence, that the sequence (x i,k ) k∈N remains in F i ∩ B(x, δ). Hence, we can invoke the strong regularity at x to write, for all k ∈ N:

m i=1 x * i,k 2 ď α m i=1
x * i,k .

Passing to the limit when k goes to +∞, together with the fact that m i=1 x * i = 0, we deduce that m i=1

x * i 2 = 0, which proves the linear regularity.

On the reverse way, suppose that the linear regularity holds at x, and assume, by contradiction, that there exists two converging sequences

(x i,k ) k∈N ⊂ F i and (x * i,k ) k∈N ⊂ H such that x * i,k ∈ N L F i (x i,k ) and x i,k F i --→ k→+∞ x and m i=1 x * i,k 2 > k m i=1 x * i,k .
Adapting the proof in [222, Proposition 8.5], we define

y * i,k :=   m i=1 x * i,k 2   -1 x * i,k ∈ N L F i (x i,k ) which satisfies (4.38) m i=1 y * i,k 2 = 1 and m i=1 y * i,k ď 1 k .
Since the sequences (y * i,k ) k∈N are bounded and that H is finite-dimensional, we can suppose that it converge to some y * i when k goes to +∞. Moreover, the limiting normal cones have a closed graph in finite dimension, so y * i ∈ N L F i (x) by construction. By taking the limit in (4.38), we obtain moreover

m i=1 y * i 2 = 1 and m i=1 y * i ď 1 k .
This clearly in contradiction with the linear regularity assumption.

Example 4.2.13. In finite dimensions, we can derive from (4.37) the following simple characterization of the strong regularity between two sets:

N L F 1 (x) ∩ -N L F 2 (x) = {0}.
In [START_REF] Lewis | Local linear convergence for alternating and averaged nonconvex projections[END_REF]Proposition 8.5], the authors consider a collection of closed prox-regular sets D 1 , . . . , D m in R n having a strong regular intersection, and derive the K L property of f . The same approach is used in [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF]Theorem 5.8], where the authors take a family of closed convex sets D 1 , . . . , D m in R n (one of them being compact), together with a general closed set C. We extend their approach to family of closed prox-regular sets in a Hilbert space, together with a general closed set. Remark ). On the other hand, the Kurdyka-Lojasiewicz property of a function is also equivalent (under some assumptions) to the metric regularity on an (other) set-valued mapping. Thus, we can see -at least informally-that the result of Proposition 4.2.14 is not surprising.

Proof. We will find δ > 0 such that the K L inequality holds in Γ +∞ (x, δ) = {x ∈ B(x, δ) : 0 < f (x) < +∞}. The strong regularity of the intersection implies the existence of parameters α > 0 and δ > 0 such that (4.39)

x * 2 + m i=1 x * i 2 ď α x * + p i=1 x * i for each x * i ∈ N L D i (x i ) with x i ∈ D i ∩ B(x, δ
), i = 1, . . . , p, and each x * ∈ N L C (x) with x ∈ C ∩ B(x, δ). If necessary, shrink δ to be in the context of Proposition 4.2.6 so that each projection proj D i for i = 1, . . . , p, is single-valued on B(x, δ). Observe that

∂ L f (x) = N L C (x) + p i=1 w i x -proj D i (x) . Take x ∈ C ∩ B(x, δ) and set x i := proj D i (x) ∈ D i ∩ B(x, δ). Note that x * i := w i (x -x i ) lies in N L D i (x i ).
We have

∂ L f (x) 2 -= inf x * ∈N L C (x) x * + p i=1 x * i 2 ě 1 α 2 inf x * ∈N L C (x) x * 2 + p i=1 x * i 2
by (4.39). Whence

∂ L f (x) 2 -≥ 1 α 2 p i=1 x * i 2 = 1 α 2 p i=1 w 2 i x -x i 2 .
If we note w := min i=1..p w i > 0, then we obtain as desired

∂ L f (x) 2 -ě w 2 α 2 f (x) = w 2 α 2 [f (x) -f (x)].

Inverse problems and sparsity

With the increase of data's size in signal or image processing, the research for sparse solutions gained in interest. The sparsity of a solution is generally expressed by means of the counting function

1 : ∀x ∈ R N , x 0 := ♯{i ∈ {1, . . . , N } | x i = 0}.
When dealing with matrices, a natural analogue to the counting function is the rank function.

These functions are lower semi-continuous and semi-algebraic, so they enter in the setting of the Alternate Forward-Backward algorithm. We present in this section some inverse problems involving sparsity, and provide different algorithms for solving them.

Compressed sensing

The search for sparse solutions of under-determinated linear systems is an important problem in compressive sensing and appears naturally when studying signal denoising, deblurring or compression processes. Consider the following problem:

(P 0 ) min x∈R N x 0 such that Ax = b,
where b ∈ R M with M < N and A ∈ R M ×N . The interest of (P 0 ) in data compression is simple: suppose we have a measurement b := Ax of a signal x, which we know to be sparse (meaning that x 0 is small). If x 0 < M 2 , and the columns of the matrix are supposed to be in general position, then x is the only solution of problem (P 0 ) (see [START_REF] Donoho | For most large underdetermined systems of linear equations the minimal ℓ 1 -norm solution is also the sparsest solution[END_REF][START_REF] Donoho | Counting the faces of randomly-projected hypercubes and orthants, with applications[END_REF]). This property holds in R N ×M with probability 1 and can be obtained by considering a matrix with random entries.

Solving (P 0 ) by means of combinatorial techniques is very difficult, because it is an NP-hard problem [START_REF] Natarajan | Sparse approximate solutions to linear systems[END_REF]. This is why this problem is generally handled with an iterative method. In order to solve (P 0 ), which can be rewritten as

min x∈R N x 0 + δ {Ay=b} (x),
we could apply the proximal algorithm to the lower-semicontinuous function x → α x 0 + δ {Ay=b} (x). But the proximal operator associated to this sum is not easy to compute while the proximal operator associated to each summand is well known. Indeed, prox α • 0 is the hard shrinkage operator, that we will note H α (x), and is defined by:

for all i ∈ {1, . . . , N }, (H α (x)) i =      0 if |x i | < √ 2α, {0, x i } if |x i | = √ 2α, x i if |x i | > √ 2α.
On the other hand, prox δ {Ax=b} (x) is the projection of x onto the linear set {Ax = b}. In other words, prox

δ {Ax=b} (x) = proj {Ax=b} (x) = (I N -A † A)x + A † b,
where A † is the pseudo-inverse of A. Thus, it only remains to apply the splitting method seen in Section 4.1. Let us give some relaxations of (P 0 ), which leads to different algorithms:

• Penalize quadratically the constraint, and consider for some α > 0:

(P

0 ) minimize

x∈R N α x 0 + 1 2 Ax -b 2 .
1 Sometimes called the ℓ 0 norm, which is not really an appropriate name.

The AFB algorithm becomes the Hard-Shrinked Gradient algorithm, defined as follows:

x k+1 ∈ H αλ k (x k -λ k A * (Ax k -b)),
where (λ k ) k∈N are step sizes lying in [λ, λ] ⊂]0, A * A -1 [. This method appears in [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF], and can be seen as an adaptation of the Thresholded Landweber algorithm [START_REF] Daubechies | An iterative thresholding algorithm for linear inverse problems with a sparsity constraint[END_REF], where the ℓ 1 norm is replaced by the counting function.

• Separate the variables by introducing the problem, for some α > 0:

(P

0 ) minimize

x∈R N α x 0 + δ {Ay=b} (y) + 1 2 x -y 2 .
The AFB algorithm becomes the Hard-Shrinkage Projection method: )) is semi-algebraic, we obtain that any bounded sequence generated by this algorithm converges. Moreover, Theorem 4.1.8 guarantees that the limit point is a solution of (P

y k+1 = proj {Ax=b} (µ k x k + (1 -µ k )y k ), x k+1 ∈ H αλ k (λ k y k + (1 -λ k )x k+1 ), with parameters (λ k ) k∈N ,(µ k ) k∈N lying in [τ , τ ] ⊂]0, 1[.
(1) 0 ) (resp. (P (2)
0 )), provided that the initial point is close enough to the solution.

Sparse and Low-rank Matrix Decomposition

The problem of recovering the sparse and low-rank components of a matrix arises naturally in various areas, such as model selection in statistics or system identification in engineering (see [START_REF] Chandrasekaran | Rank-sparsity incoherence for matrix decomposition[END_REF] and references therein). Denote by X 0 the number of nonzero components of a matrix X ∈ M m,n (R). Given A ∈ M m,n (R), the low-rank sparse matrix decomposition problem consists in finding X, Y in M m,n (R) such that A = X + Y , with X having a low rank, and Y being sparse:

(P rk,0 ) minimize

X+Y =A rank (x) + Y 0 .
An approach to solve this problem consists in doing a convex relaxation of the objective function: the counting norm and the rank functions are replaced by ℓ 1 and nuclear norms, respectively (see [START_REF] Recht | Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization[END_REF][START_REF] Ganesh | Fast algorithms for recovering a corrupted low-rank matrix[END_REF][START_REF] Yuan | Sparse and low-rank matrix decomposition via alternating direction method[END_REF]). Nevertheless, the K L framework is well adapted to this nonconvex (but semialgebraic!) problem, and offers convergent numerical methods. Moreover, the AFB method is well suited for its structure in separated variables.

If we know some estimate on the desired rank (resp. sparsity) for X (resp. Y ), a tractable relaxation is: minimize

X,Y ∈Mm,n(R) δ {rank (•)ďr} (X) + δ { • 0 ďs} (Y ) + 1 2 A -X -Y 2 F ,
where • F denotes the Frobenius norm in M m,n (R), and r, s ∈ N. Applying the AFB method to this problem leads to an Alternating Averaged Projected Method :

X k+1 ∈ proj {rank (•)ďr} (λ k (A -Y k ) + (1 -λ k )X k ), Y k+1 ∈ proj { • 0 ďs} (µ k (A -X k+1 ) + (1 -µ k )Y k ),
where (λ k ) k∈N , (µ k ) k∈N are two sequences of parameters in [τ , τ ] ⊂]0, 1[. This algorithm involves two projections at each step. To project onto { • 0 ď s}, one simply sets all the coefficients to zero, except for the s largest ones (in absolute value). The projection of M ∈ M m,n (R) onto {rank • ď r} is given by Eckart-Young's Theorem [START_REF] Eckart | The approximation of one matrix by another of lower rank[END_REF]: writes the Singular Value Decomposition M = U ΣV , and put all the coefficients of the diagonal matrix Σ to zero, except for the s largest ones (in absolute value).

An other way to relax the problem (P rk,0 ) is the following: take some α, β > 0 and minimize

X,Y ∈Mm,n(R) α rank (x) + β Y 0 + 1 2 A -X -Y 2 F .
The AFB algorithm applied to this problem leads to a similar algorithm:

X k+1 ∈ prox αrank (λ k (A -Y k ) + (1 -λ k )X k ), Y k+1 ∈ prox β • 0 (µ k (A -X k+1 ) + (1 -µ k )Y k ), with (λ k ) k∈N , (µ k ) k∈N in [τ , τ ] ⊂]0, 1[.
Here, two proximal operators are involved. The proximal operator of the counting norm for the matrices is exactly the same than for the vectors, the hard-shrinkage operator. Recently, Hiriart-Urruty and Le [START_REF] Hiriart-Urruty | From Eckart and Young approximation to Moreau envelopes and vice versa[END_REF] gave an explicit description of the elements of prox αrank (M ). It suffices, for instance, to write the singular value decomposition M = U ΣV , and to apply the hard-shrinkage operator to Σ.

Numerical results on data compression

Let x ∈ R N be an original signal with sparsity s := x 0 , and b = Ax the compressed signal through A ∈ R M ×N generated with i.i.d. Gaussian entries2 . The purpose is to recover x from b, by solving (P

0 ) minimize

x∈R N α x 0 + δ {Ay=b} (y) + 1 2 x -y 2
with the Hard Shrinkage Projection method. Rather than competing with the vast literature on compressive sensing, our principal aim here is to illustrate a new method from tame optimization to tackle problems involving sparsity. However, this clearly opens the door to new ideas from nonconvex optimization and maybe future works could focus on the enhancement of these algorithms, and their comparison to specific methods such as LASSO or Matching Pursuit.

Let us precise how we run the Hard Shrinkage Projection method. We generate randomly x 0 = y 0 and run 10 steps of the algorithm with parameters µ k ≡ 10 -3 , λ k ≡ 10 -2 and α = 10 5 . Then, we do a restart at the current point for 10 more steps, with parameters µ k ≡ 10 -3 , λ k ≡ 1 3 and α = 130. Finally, we let the algorithm run with µ k ≡ 10 -3 , λ k ≡ 1 3 and α = 3. The choice of this initialization can be understood as follows: in a first time we use the hard shrinkage operator H αλ with a high value for αλ, which enforces x k to keep a large number of zeroes. Then, we diminish αλ progressively, which allows x k to capture new nonzeros coefficients. All our numerical experimentations tend to show that a decreasing α improves the convergence of the method.

Let us start by evaluating the performance of this algorithm. For this, we generate random initial signals x ∈ [0, 255] 100 , that we try to recover for different values of s and M . We define δ := M 100 and ρ := s M , being respectively the levels of compression and sparsity. We let δ and ρ taking values between 0 and 1, and take for each couple (δ, ρ) the values M = ⌊100δ⌋ and s = ⌊100ρδ⌋. For each couple of M and s, we run 10 instances of the Hard Shrinkage Projection algorithm with 100 steps, and take e(δ, ρ) the average of the relative errors x-x 100 We obtain a phase transition diagram: below a certain limit, the performance of the algorithm decreases dramatically. This behaviour is well-known for methods based on the ℓ 1 norm, see the works of Donoho [START_REF] Donoho | High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension[END_REF] and Donoho-Stodden [START_REF] Donoho | Breakdown point of model selection when the number of variables exceeds the number of observations[END_REF]. Observe nevertheless that our phase transition lies in a higher part of the diagram than the ones of [START_REF] Donoho | High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension[END_REF][START_REF] Donoho | Breakdown point of model selection when the number of variables exceeds the number of observations[END_REF]. This tends to show that our method based on the direct minimization of the counting norm • 0 is more demanding in terms of the couple (δ, ρ).

We illustrate now the compression/decompression process by applying it to some picture X ∈ R 512×512 . In general, images of every-day life are not sparse, so we cannot directly work with them. Instead, we will work in a wavelet space, applying to X a wavelet transform W . Thus, W X is almost sparse, in the sense that W X contains a lot of nearly zero coefficients. By taking X = H ε (W X) for some ε ∼ 0, we obtain a sparse matrix, such that W -1 X is close to the original image (see Figure 4.4 below). Bottom line: the matrix X obtained after putting to zero 90% of the coefficients of W X, and the corresponding picture W -1 X.

Once we have a sparse matrix X to work with, we can compress it. For computation purposes, we do not compress directly X through a huge matrix A of size ∼ 512 4 , but decompose X into several vectors of size 200, that we can compress/decompress in parallel: X = (x 1 , . . . , x1311 ). The decompression of these fragments xi is done by using exactly the same method as described above. Recall that the exact recovery is not guaranteed for each run of the Hard-Shrinkage Projection method. Hence, we keep in memory the sparsity of the original fragment s i := xi , and restart eventually the algorithm until it converges to a vector having the same sparsity s i .

Comments and perspectives

Remark 4.3.1 (On the additive/relative errors). In Section 4.1, we presented a general splitting method, admitting additive errors. The convergence of the method is guaranteed under the set of assumptions HE, which is difficult to satisfy in practical. It would be of interest to design an algorithm admitting relative errors.

Consider for instance the gradient-like method

x k+1 = x k -λ k A -1 k ∇f (x k ), λ k > 0, A k ∈ S ++ (H),
which covers the gradient or Newton-like methods. At each iteration, the direction d k := -A -1 k ∇f (x k ) must be computed. It can be obtained by solving, for instance, a quadratic program:

(4.40) d k = argmin d∈H 1 2 A k d, d + ∇f (x k ), d .
The approach adopted for instance by Fliege and Svaiter [START_REF] Fliege | Steepest descent methods for multicriteria optimization[END_REF] consists in solving (4.40) inexactly.

Define for this the optimal value

v k := inf d∈H 1 2 A k d, d + ∇f (x k ), d = -1 2 ∇f (x k ) 2 A -1 k ď 0.
Given a tolerance σ ∈]0, 1], take

x k+1 = x k + λ k d k , where d k is a descent direction such that (4.41) 1 2 A k d k , d k + ∇f (x k ), d k ď σv k .
The interest of this approach is that it allows an inexact computation of -A -1 k ∇f (x k ), with an error estimation depending only on the parameter σ. It can be derived from (4.41) that such direction

d k satisfies ∇f (x k ), d k + σ 2 A k ∇f (x k ) 2 ď 0 and d k ď 2 α(A) ∇f (x k ) ,
which means that this method is gradient related, as defined in Example 3.2.1. As a consequence, this inexact method satisfies the hypotheses H 1 and H 2 , and enters into the setting of the convergence Theorem 3.2.2.

It would be of interest to take into account such relative errors for the proximal (or Forward-Backward) method. In that case, the analysis should rely on a perturbed proximal operator, for nonconvex functions. See the work of Rockafellar [START_REF] Rockafellar | Monotone operators and the proximal point algorithm[END_REF], or more recently the study of Salzo-Villa [START_REF] Salzo | Inexact and accelerated proximal point algorithms[END_REF] (and the references therein), for examples of perturbed proximal methods.

Remark 4.3.2 (Compressed sensing in optimal control). From the applications point of view, the counting norm • 0 evoked in Section 4.2.2 has a natural extension to an infinite-dimensional functional setting, namely, the measure of the support of a function u defined on some Ω ⊂ R N . An interesting -but challenging-issue is to apply our algorithm to this extension in order to solve the problem of sparse-optimal control of partial differential equations. From the implementation point of view, it suffices to apply the one-dimensional hard shrinkage operator at each point. Nevertheless, the verification of the K L inequality for this function has not been established and will probably rely on sophisticated arguments concerning the geometry of Hilbert spaces. Then, there is the natural question whether this approach is more efficient than those using the L 1 norm (see, for instance, [START_REF] Casas | Approximation of sparse controls in semilinear equations by piecewise linear functions[END_REF]).

Remark 4.3.3 (On numerical methods for solving semilinear PDE's). Let Ω be an open bounded subset of R N of class C 2 . We aim to study the following semilinear heat equation with Dirichlet boundary conditions:

u(x, t) = ∆u(x, t) + g(x, u(x, t)) for (x, t) ∈ Ω × R + u(x, t) = 0 for (x, t) ∈ ∂Ω × R + ,
where g : Ω × R -→ R is a source term. As usual, we can use a variational approach for this parabolic PDE. Thus, consider trajectories

u(t) = u(•, t) satisfying (4.42) for all t ∈ R + , u(t) = ∆u(t) + g(•, u(t)),
this equation being understood in H -1 (Ω). A natural problem when facing this equation evolution is the following: how to numerically implement the evolution of this dynamic, given an initial state u 0 ∈ H 1 0 (Ω). A popular approach consists in discretizing in time the equation (4.42). This leads to the following Euler scheme:

(4.43) ∀k ∈ N, u k+1 -u k λ k = ∆u n k + g(•, u n k ) in H -1 (Ω),
where λ k is some nonnegative stepsize, and n k ∈ {k, k + 1}. Here too, this equation shall be understood in a variational sense. When n k = k (respectively n k = k + 1), we talk about an explicit (resp. implicit) Euler scheme. The explicit Euler scheme is easy to implement, since u k+1 is computed through an explicit formula, but this method is not satisfactory since it suffers from a lack of stability, see for instance [START_REF] Hundsdorfer | Numerical solution of time-dependent advection-diffusionreaction equations[END_REF]Example 2.3]. On the contrary, the implicit Euler method enjoys a good stability property, see for instance the seminal work of Crandall-Liggett [START_REF] Crandall | Generation of semi-groups of nonlinear transformations on general Banach spaces[END_REF], or also Bénilian-Crandall-Pazy [START_REF] Bénilian | Bonnes solutions" d'un problème d'évolution semi-linéaire[END_REF]. This is the approach adopted by Merlet and Pierre [START_REF] Merlet | Convergence to equilibrium for the backward Euler scheme and applications[END_REF], which interpret the implicit Euler scheme as a 'proximal' method, applied to the underlying energy of the system:

E : H 1 0 (Ω) -→ R u -→ 1 2 Ω |∇u(x)| 2 dx - Ω u(x) 0 g(x, t) dtdx.
Under some growth conditions on g, we are ensured that E is of class C 2 (see [209, Proposition I.17.6] and [97, Corollary 5.5.7]). Moreover, assuming g(x, t) to be analytic in t, uniformly with respect to x ∈ Ω, a result of Haraux and Jendoubi [START_REF] Haraux | Convergence of bounded weak solutions of the wave equation with dissipation and analytic nonlinearity[END_REF] states3 that E inherits the K L property. Thus, the convergence of the implicit Euler method follows (see [START_REF] Merlet | Convergence to equilibrium for the backward Euler scheme and applications[END_REF]Theorem 5.4]).

In the light of the work done in this chapter on splitting methods, it seems natural to adopt an explicit-implicit 4 Euler scheme (see [START_REF] Temam | Sur la stabilité et la convergence de la méthode des pas fractionnaires[END_REF][START_REF] Crouzeix | Une méthode multipas implicite-explicite pour l'approximation des équations d'évolution paraboliques[END_REF][START_REF] Ostermann | Stability of W-methods with applications to operator splitting and to geometric theory[END_REF][START_REF] Hansen | Convergence of the implicit-explicit Euler scheme applied to perturbed dissipative evolution equations[END_REF]):

(4.44) ∀k ∈ N, u k+1 -u k λ k = ∆u k+1 + g(•, u k ) in H -1 (Ω).
This method would be interesting, since it reduces the cost of computing u k+1 , (we only have a linear equation to solve), and we can hope that this method keeps some stability. Thus, it looks like it suffices to apply the Forward-Backward method to E, to obtain the convergence of this explicit-implicit Euler scheme (4.44).

In fact, things are more complicated here. This is essentially due to the fact that the heat equation can be seen as a gradient flow for E with respect to the L 2 (Ω) scalar product. In particular, the implicit Euler method corresponds to a proximal algorithm with respect to the L 2 (Ω) norm:

u k+1 ∈ H 1 0 (Ω) and u k+1 -u k λ k = ∆u k+1 + g(•, u k+1 ) in H -1 (Ω) ⇔ u k+1 = argmin u∈H 1 0 (Ω) E(u) + 1 2λ k u -u k 2 L 2 .
Nevertheless, there is still some information to exploit there, like the decrease property:

1 2λ k u -u k 2 L 2 ď E(u k ) -E(u k+1 ).
When applying the Forward-Backward algorithm to E, even a basic property like the descent property becomes complicated to obtain. To see this, define

J : H 1 0 (Ω) -→ R and G : H 1 0 (Ω) -→ R u -→ 1 2 Ω |∇u(x)| 2 dx u -→ Ω u(x) 0 g(x, t) dtdx.
The explicit-implicit Euler scheme (4.44) can then be rewritten as

u k+1 = argmin u∈H 1 0 (Ω) J (u) + 1 2λ k u -u k + g(•, u k ) 2 H -1 .
As usual, we derive from the optimality condition above this first inequality (use the compact embedding L 2 (Ω) ֒→ H -1 (Ω)):

1 2λ k u k+1 -u k 2 L 2 + g(•, u k ), u k+1 -u k H -1 ×H 1 0 ď J (u k ) -J (u k+1 ).
Assuming that the derivative DG : [209, Proposition I.17.6]), we can use the descent lemma 2.1.13 to obtain:

H 1 0 (Ω) -→ H -1 (Ω) is L-Lipschitz continuous (use for in- stance
1 2λ k u k+1 -u k 2 L 2 - L 2 u k+1 -u k 2 H 1 0 ď E(u k ) -E(u k+1 ).
We see that we cannot derive the descent property by means of usual methods, which is a clear obstacle. Thus, the study of this splitting method in this setting provides a good challenge for the future.

Remark 4.3.4 (Going further). Although the set of hypotheses considered in Section 3.2 account for a wide variety of numerical methods described in this Chapter, there exist some algorithms that do not fit into this framework. An interesting task would be to extend the present convergence analysis to such methods:

• Acceleration schemes, like the ones studied in [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate O(1/k 2 )[END_REF][START_REF] Beck | A fast iterative shrinkage-thresholding algorithm for linear inverse problems[END_REF][START_REF] Beck | Gradient-based algorithms with applications in signal recovery problems[END_REF], have been proved to have a remarkable performance at least in terms of the values of the objective function. We point out a recent work [START_REF] Ochs | iPiano: inertial proximal algorithm for nonconvex optimization[END_REF] containing a first attempt in this direction.

• Dealing with on-line rules for updating certain parameterssuch as (limited) minimization, Armijo, Goldstein, or Wolfe line search for the step sizesmay be quite useful. See for instance [START_REF] Burachik | Full convergence of the steepest descent method with inexact line searches[END_REF] for a gradient method with an inexact line-search satisfying H 1 and H 2 .

• Bregman distances provide a relevant alternative to Newton methods, as variable-metric schemes. A first attempt in this direction are the works of Quiroz-Oliveira [START_REF] Quiroz | Proximal point methods for functions involving Lojasiewicz, quasiconvex and convex properties on Hadamard manifolds[END_REF] and Lageman [START_REF] Lageman | Pointwise convergence of gradient-like systems[END_REF], which study descent methods on manifolds.

• Gradient-type methods usually require Lipschitz-continuity of the gradient, at least in sublevel sets. The possibility to consider an asymptotic degeneration of this condition will broaden the scope of these methods.

• Finally, a challenging open problem is to obtain convergence results general primal-dual methods following a Lagrangian approach. Recent preprints give a first advance in this direction, providing convergence results for the ADMM method [START_REF] Li | Global convergence of splitting methods for nonconvex composite optimization[END_REF][START_REF] Hong | Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems[END_REF][START_REF] Wang | Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems[END_REF].

Part II

Dynamics for Vector optimization

Chapter 5

An ordered subdifferential for vector-valued functions

We study a general vector optimization problem associated to a locally Lipschitz function F : X -→ Y, where X and Y are two Banach spaces. We consider a nonempty closed convex set C ⊂ X, which will model the constraints, and we equip Y with an order induced by a pointed closed convex cone K, with nonempty interior. In the following, ĺ will denote the order induced by K, and ň the associated strict order induced by K \ {0 Y }. Moreover, we will note ă the strict order induced by the strict cone int K. We recall from Section 2.3.1 that this is equivalent to:

• y 1 ĺ y 2 if and only if y 2 -y 1 ∈ K, • y 1 ň y 2 if and only if y 2 -y 1 ∈ K \ {0},
• y 1 ă y 2 if and only if y 2y 1 ∈ int K.

Here we try to solve (VOP) MIN x∈C F (x), and look for the set of its efficient points

ARGMIN x∈C F = {x ∈ X | [F ň F (x)] = ∅},
or weak efficient points

ARGMIN w x∈C F = {x ∈ X | [F ă F (x)] = ∅}.
From now, and until the end of this chapter (and in fact, until the end of Part II), we fix an element e ∈ int K, which gives us a base Θ of K + by taking:

(5.1)

Θ := {θ ∈ K + | θ, e = 1}.
As seen in Section 2.3, Θ is weakly * compact and convex, and the family of real-valued cost functions

{f θ := θ • F | θ ∈ Θ}
controls the monotonicity of F (see Theorem 2.3.7). It is also matters to consider Ξ the set of extreme points of Θ in Y * , and the corresponding subfamily of extreme cost functions

{f ξ := ξ • F | ξ ∈ Ξ}.
The reader can keep in mind along its reading the essential case of multi-objective optimization. When (Y, K) = (R m , R m + ) and F = (f 1 , ..., f m ), we recover from the definitions of efficiency the usual notions of Pareto and weak Pareto optimality (see Section 2.3). Moreover, if we take e = (1, ..., 1), the family of extreme cost functions is exactly {f 1 , ..., f m }, while the cost functions {f θ } θ∈∆m corresponds to all the convex combinations

f θ = m i=1 θ i f i | m i= θ i = 1, θ i ě 0 .
In Section 5.1 we introduce the ordered Clarke subdifferential of F as a set-valued mapping ∂ C F : X ⇒ X * , defined trough the Clarke subgradients of the cost functions:

∂ C F (x) = co * θ∈Θ ∂ C f θ (x).
In particular, in the multi-objective setting F = (f 1 , ..., f m ), we show that the ordered Clarke subdifferential reduces to the convex hull of the subgradients of the f i 's:

∂ C F (x) = co{∂ C f 1 (x), ..., ∂ C f m (x)}.
We study in Section 5.1.2 some basic properties, such as its computation in the convex or smooth cases, in Proposition 5.1.8. In Propositions 5.1.11 and 5.1.13 we show some closure properties of the graph of ∂ C F : X ⇒ X * . This will be useful in the next chapters, when considering the asymptotic behaviour of a dynamic governed by ∂ C F . In Section 5.2 we study the descent directions provided by this ordered subdifferential, defined through the notion of ordered Clarke derivative

d C F (x, d) := sup θ∈Θ d C f θ (x, d) < 0.
In Section 5.2.1, after some definitions, we derive an Armijo's rule in Proposition 5.2.9. We also provide a Fermat's rule in Theorem 5.2.10, which relates the critical points satisfying

0 ∈ N C (x) + ∂ C F (x)
to the weak Pareto points. In Section 5.2.2 we show that the ordered Clarke subdifferential ∂ C F (x) spans the normal cone to the sublevel set [F ĺ F (x)]. Dually, the set of descent directions defined before is exactly the interior of the tangent cone to [F ĺ F (x)]. In Section 5.2.3, we take X = H as a Hilbert space, and design one particular choice of descent direction, by considering the element of minimal norm in

-∂ C F (x) (or -N C (x) -∂ C F (x), if there is a constraint).
In a sense specified in Theorem 5.2.20, this direction is the steepest among the descent directions at x. That is why we call it the steepest descent direction at x, and note1 it s(x). This steepest descent vector field s : H -→ H generalises the steepest descent vector field ∇f : H -→ H usually involved in the minimization of a smooth real-valued function f . This will naturally led us to study, in the next chapters, dynamics governed by s. The Figure 5.1 below sums quickly up the situation, for a bi-objective smooth function

F = (f 1 , f 2 ) : R 2 -→ (R 2 , R 2 + ).
5.1 An ordered Clarke subdifferential for vector-valued functions

Definition

We introduce what will be the central tools for our study of vector optimization problems.

Definition 5.1.1. For any x ∈ X, and d ∈ X, we define the ordered (Clarke) directional derivative of F at x in the direction d by

d C Θ F (x, d) := sup θ∈Θ d C f θ (x, d).
In general, if there is no ambiguity, we will omit the mention of the base Θ and just write d C F (x, d).

We easily see that d Like in the classic construction of the Clarke subdifferential, we derive from the ordered Clarke directional derivative above a dual object: Definition 5.1.3. For any x ∈ X, we define the ordered (Clarke) subdifferential of F at x by (5.2)

∂ C Θ F (x) := {x * ∈ X * | ∀d ∈ X, x * , d d C F (x, d)}.
Again, if there is no ambiguity, we will omit the mention of the base Θ and just write ∂ C F (x).

It is a nonempty weakly * compact convex set. So, by applying [5, Theorem 7.51] we see that the following max formula is satisfied for all x, d ∈ X :

(5.3) d C F (x, d) = max x * ∈∂ C F (x)
x * , d .

From this max formula we can deduce an explicit expression for ∂ C F (x), involving the subgradients of the cost functions {f θ } θ∈Θ :

Proposition 5.1.4. For all x ∈ X,

∂ C F (x) = co * θ∈Θ ∂ C f θ (x).
Proof. From the sum rule for the Clarke subdifferential of scalar-valued functions, we know that for all θ ∈ Θ,

d C f θ (x, d) = max x * ∈∂ C f (x)
x * , d .

It follows that d C F (x, d) is, by definition, max x * ∈ θ∈Θ ∂ C f θ (x)
x * , d .

From the max formula (5.3) and Proposition 2.1.4 on support functions, the conclusion follows.

Next proposition shows that ∂ C F (x) is simpler in the polyhedral case: it only involves the subgradients of the extreme cost functions, corresponding to the extreme points of Θ.

Proposition 5.1.5. (Polyhedral case) Suppose that K is polyhedral. Write Θ = co{ξ 1 , ..., ξ m } and let f i := ξ i • F for i ∈ {1, ..., m}. Then

∂ C F (x) = co ∂ C f i (x) and d C F (x, d) = max i∈{1,...,m} d C f i (x, d).
Example 5.1.6. This Proposition applies in particular to the multicriteria optimization con-

text: if (Y, K) = (R m , R m + ) and F (x) = (f 1 (x), ..., f m (x)), then d C F (x; d) = max i∈{1,...,m} d C f i (x; d),
and

∂ C F (x) is the convex hull of the subdifferentials ∂ C f i (x)
. We see that we recover here the set studied by Cornet for smooth functions in [START_REF] Cornet | Contributions to the mathematical theory of dynamical mechanisms of resource allocation[END_REF], and revisited in [START_REF] Attouch | A continuous gradient-like dynamical approach to Paretooptimization in Hilbert spaces[END_REF]. In particular, ∂ C F (x) reduces to the classical Clarke subdifferential when m = 1.

Figure 5.2: The ordered Clarke subdifferential of F = (f 1 , f 2 ) at x with respect to R 2 + is the convex hull of ∇f 1 (x), ∇f 2 (x).

Proof of Proposition 5.1.5. First, recall that, since it is assumed that int K is nonempty, K being polyhedral implies that Y is finite dimensional. It follows from Section 2.1.2 that K + is also polyhedral, so we can indeed write Θ = co{ξ 1 , ..., ξ m }.

Let us start by proving that d

C F (x, d) = max{d C f i (x, d) | i = 1, ..., m}. Fix x and d in X, then it is clear from the definition of the f i 's that (5.4) max i∈{1,...,m} d C f i (x, d) ď sup θ∈Θ d C f θ (x, d) = d C F (x, d). Let (θ n ) n∈N be a sequence in Θ such that d C F (x, d) = lim n→+∞ d C f θn (x, d).
For each n ∈ N, there exists (λ n 1 , ..., λ n m ) in the simplex unit ∆ m such that θ n = m i=1 λ n i ξ i . Hence, using the sum rule for the Clarke directional derivative :

(5.5) d C f θn (x, d) = d C (θ n • F )(x, d) = d C m i=1 λ n i (ξ i • F ) (x, d) ď m i=1 λ n i d C f i (x, d).
Using the fact that m i=1 λ n i = 1, we see that the right member in (5.5) is bounded from above by max{d C f i (x, d) | i = 1, ..., m} for all n ∈ N. By taking the limit when n → +∞, we conclude that

(5.6) d C F (x, d) = max i∈{1,...,m} d C f i (x, d)
Now, we use the max formula (5.3) for the ordered subdifferential to see that (

∂ C F (x) = co * m i=1 ∂ C f i (x). Since ∂ C f i (x)
∂ C f i (x) is compact, hence closed.
In other words,

∂ C F (x) = co m i=1 ∂ C f i (x).
In Section 5.2 we will study in details the descent directions induced by the directional derivative d C F (x; •), exploiting the duality between d C F (x; •) and ∂ C F (x). But first, we will focus on topological and geometrical properties of ∂ C F which will be needed later.

Properties of the ordered Clarke subdifferential

Let us start by showing that ∂ C F : X ⇒ X * is locally bounded, as a consequence of the local Lipschitz continuity of F . Proposition 5.1.7 (Local boundedness). For all x ∈ X, there exists a neighbourhood U of x, and

M ě 0 such that ∀x ∈ U, ∀x * ∈ ∂ C F (x), x * ď M.
Proof. Let U be a neighborhood of x on which F is Lipschitz continuous. Since Θ is bounded, the family

{f θ = θ • F | θ ∈ Θ} is equi-Lipschitz continuous on U , for some Lipschitz constant M ě 0. From Proposition 2.2.14 we have ∀x ∈ U, ∀θ ∈ Θ, ∀x * ∈ ∂ C f θ (x), x * ď M.
The conclusion follows the formula We study now some hypotheses under which the ordered Clarke subdifferential has a simpler form. More exactly, we are interested on the cases for which ∂ C F (x) is exactly the reunion of the subgradients of the cost functions {f θ } θ∈Θ . In other words, we look for conditions insuring that we can remove the closed convex hull in the expression

∂ C F (x) = co * θ∈Θ ∂ C f θ (x)
∂ C F (x) = co * θ∈Θ ∂ C f θ (x).
We will say that F : X -→ Y is positively Clarke regular if, for all θ ∈ Θ,

f θ := θ • F : X -→ R is Clarke regular, i.e. ∂ B f θ = ∂ C f θ .
It is equivalent to ask the directional derivatives d B f θ (x, •) and d C f θ (x, •) to be equal. Recall also from Section 2.2.5 that, if X is reflexive, it is also equivalent to ask ∂ F f θ = ∂ C f θ for all θ ∈ Θ. It happens for instance if F is the sum of a convex and a strictly Gateaux differentiable functions. Proposition 5.1.8 (Ordered Clarke subdifferential in regular cases). i) If F is the sum of a convex function and a strictly Gateaux differentiable function at

x ∈ X, then

∂ C F (x) = θ∈Θ ∂ C f θ (x) and d C F (x; d) = max θ∈Θ d C f θ (x; d) for all d ∈ X.
ii) If F is positively Clarke regular, with X being a reflexive Banach space and Y finite dimensional, then the same conclusion than item i) holds.

iii) When F is strictly Gateaux differentiable at x ∈ X, then

d C F (x, d) = σ Θ (DF (x; d)) and ∂ C F (x) = Θ • DF (x) = D * F (x; Θ).
Remark 5.1.9. When F is smooth, according to item iii), taking an element in ∂ C F (x) is equivalent to take some θ ∈ Θ and compute θ • DF (x). For instance, in the multi-objective setting, it corresponds to the choice of some convex combination of the derivatives Df i (x). We will see later in Theorem 7.3.3 that it is also equivalent to chose a Radon probability µ over Θ, and compute Θ Df θ dµ(θ).

Proof. We start with item iii). Considering that F is strictly Gateaux differentiable, we have for all θ ∈ Θ and d

∈ X that d C f θ (x; d) = D(θ • F )(x; d) = θ • DF (x; d). It follows that d C F (x; d) = sup θ∈Θ θ • DF (x; d) = σ Θ (DF (x; d)).
In We turn now on items i) and ii). The whole point is to prove that θ∈Θ ∂ C f θ (x) is convex and weakly * closed. Note that, in both items i) and ii), the cost functions

f θ verify ∂ F f θ = ∂ C f θ .
Hence, it is enough to prove that θ∈Θ ∂ F f θ (x) is convex and weakly * closed.

Let us prove first that it is convex. Consider x * 1 and x * 2 be in θ∈Θ ∂ F f θ (x), and let x * :=

λx * 1 + (1 -λ)x * 2 for some λ ∈ [0, 1]. By definition, there exists θ 1 , θ 2 ∈ Θ such that x * 1 ∈ ∂ F f θ 1 (x) and x * 2 ∈ ∂ F f θ 2 (x). Let θ := λθ 1 + (1 -λ)θ 2 , from convexity of Θ we know that θ ∈ Θ. We still have to verify that x * ∈ ∂ F f θ (x). Using f θ = λf θ 1 + (1 -λ)f θ 2 , we obtain lim inf x ′ →x f θ (x ′ ) -f θ (x) -x * , x ′ -x x ′ -x = lim inf x ′ →x λ f θ 1 (x ′ ) -f θ 1 (x) -x * 1 , x ′ -x x ′ -x + (1 -λ) f θ 2 (x ′ ) -f θ 2 (x) -x * 2 , x ′ -x x ′ -x λ lim inf x ′ →x f θ 1 (x ′ ) -f θ 1 (x) -x * 1 , x ′ -x x ′ -x + (1 -λ) lim inf x ′ →x f θ 2 (x ′ ) -f θ 2 (x) -x * 2 , x ′ -x x ′ -x 0.
Now that the convexity of θ∈Θ ∂ C f θ (x) is proved, we study its weak * closedness, distinguishing items i) and ii).

Suppose first that the hypothesis of item i) is satisfied, i.e. that F = G + H, where G is convex and H is strictly Gateaux differentiable at x. Let (x * α ) α∈A be a net in

θ∈Θ ∂ C f θ (x),
weakly * converging to some x * ∈ X * . For all α ∈ A, there exists some θ α ∈ Θ such that

x * α ∈ ∂ C f θα (x) = ∂g θα (x) + θ α • DH(x),
where g θα := θ α • G is a convex function. We can assume, without loss of generality, that θ α weakly * converges to some θ ∈ Θ. Define z *

α := x * α -θ α • DH(x) ∈ ∂g θα (x), which is a net weakly * convergent to z * := x * -θ • DH(x). Let us show that z * ∈ ∂g θ (x). Using z * α ∈ ∂g θα (x), we have for all x ′ ∈ X : (θ • G)(x ′ ) -(θ • G)(x) -z * , x ′ -x = lim α∈A (θ α • G)(x ′ ) -(θ α • G)(x) -z * α , x ′ -x ě 0.
So z * ∈ ∂g θ (x), and it follows that

x * ∈ ∂ C f θ (x) ⊂ θ∈Θ ∂ C f θ (x).
Suppose now that the hypotheses of item ii) are satisfied. Since X is supposed to be reflexive, and given that θ∈Θ ∂ C f θ (x) is convex, it is enough to verify that it is closed for the norm topology in X * . Take then a sequence (x * n ) n∈N in θ∈Θ ∂ C f θ (x) such that x * n tends to some x * ∈ X * for the norm topology. There exists a corresponding sequence θ n ∈ Θ such that x * n ∈ ∂ C f θn (x). Since Θ is compact in Y (which is supposed to be finite-dimensional), we can assume, taking eventually a subsequence, that θ n converges to some θ ∈ Θ. We just need to show now that x * ∈ ∂ C f θ (x), which is equal to ∂ F f θ (x), according to the positive Clarke regularity of F and the reflexivity of X. Start by writing, for all n ∈ N:

lim inf x ′ →x f θ (x ′ ) -f θ (x) -x * , x ′ -x x ′ -x = lim inf x ′ →x (θ n • F )(x ′ ) -(θ n • F )(x) -x * n , x ′ -x x ′ -x + θ -θ n , F (x ′ ) -F (x) x ′ -x + x * n -x * , x ′ -x x ′ -x lim inf x ′ →x θ -θ n , F (x ′ ) -F (x) x ′ -x + lim inf x ′ →x x * n -x * , x ′ -x x ′ -x ,
where we used x * n ∈ ∂ F f θn (x) in the last inequality. Using the Lipschitz property of F around x, together with the Cauchy-Schwarz inequality, we deduce that (5.7) lim inf

x ′ →x f θ (x ′ ) -f θ (x) -x * , x ′ -x x ′ -x -θ -θ n Y Lip(F, x) -x * n -x * .
Passing to the limit when n → +∞ in (5.7), we finally obtain that

x * ∈ ∂ F f θ (x) = ∂ C f θ (x).
Example 5.1.10. We give a simple example for which Proposition 5.1.8 fails. Take F = (f 1 , f 0 ) : R 2 -→ (R 2 , R 2 + ), defined by

f 1 (x) := x 1 + (2, 0), x , f 0 (x) := -x 1 + (0, 2), x ,
where

• 1 denotes the usual ℓ 1 -norm ∀x = (x 1 , x 2 ) ∈ R 2 , x 1 := |x 1 | + |x 2 |.
We have in particular (we also note 0 for the origin in R 2 ):

∂ C f 1 (0) = [-1, 1] + (2, 0) and ∂ C f 0 (0) = [-1, 1] + (0, 2).
We consider now the convex combinations of f 1 and f 0 . For all λ ∈ [0, 1], we note

f λ := λf 1 + (1 -λ)f 0 , that is f λ (x) = |2λ -1| x 1 + a λ , x , with a λ := 2(λ, (1 -λ)).
The Clarke subdifferential of these functions is given by

∂ C f λ (0) = |2λ -1|[-1, 1] + a λ .
Thus, it is easy to see that

∀λ ∈ [0, 1 2 ], ∂ C f λ (0) ⊂ ∂ C f 0 (0) and ∀λ ∈ [ 1 2 , 1], ∂ C f λ (0) ⊂ ∂ C f 1 (0).
In other words (see Figure 5.3 below),

λ∈[0,1] ∂ C f λ (0) ∂ C F (0) = co{∂ C f 1 (0), ∂ C f 0 (0)}.
This is essentially due to the lack of Clarke regularity of f 0 . We end now by studying the closed-graph properties of ∂ C F : X ⇒ X * . More generally, and because we shall want to take into account some constraints, we study the sum of ∂ C F and N C , where N C (x) denotes the normal cone to C at x. Proposition 5.1.11 (Bounded weak-strong outer semi-continuity, convex case). Suppose that F is convex, and let (x α , x * α ) n∈N be a bounded net in X × X * such that (5.8)

x α w --→ α∈A x, x * α • --→ α∈A x * , and x * α ∈ N C (x α ) + ∂ C F (x α ).
Then x * ∈ N C (x) + ∂ C F (x) holds, provided that one of the two following hypotheses is satisfied :

i) dim Y < +∞ and F is Lipschitz continuous on bounded sets;

ii) for all α ∈ A, F (x) ĺ F (x α ).

Remark 5.1.12. The case covered by item ii) will be of particularly interest when considering descent dynamics associated to N C + ∂ C F , in Section 6.2.

Proof. Take nets as in (5.8), and use the convexity of F together with Proposition 5.1.8, to write

∂ C F (x α ) = θ∈Θ ∂f θ (x α ).
Then, for all α ∈ A, there exists θ α ∈ Θ such that

x * α ∈ N C (x α ) + ∂ C f θ (x α ).
Since f θ is a scalar-valued convex continuous function, we can use the sum rule (Proposition 2.2.6) and equivalently say that

x * α ∈ ∂(δ C + f θ )(x α )
, where δ C is the indicator 2 function of C. From the compactness of Θ, we can assume that (θ α ) α∈A weakly * converges to some θ ∈ Θ. We write now, for all α ∈ A, that x * α ∈ ∂(δ C + f θα )(x α ) is equivalent to :

(5.9)

∀x ′ ∈ C, f θα (x α ) ď f θα (x ′ ) -x * α , x ′ -x α .
Look at the right member in (5.9). The first term

f θα (x ′ ) = θ α , F (x ′ ) converges to θ, F (x ′ ) = f θ (x ′ ) because θ α w *
-→ θ α . The second term x * α , x ′x α converges to x * , x ′x , since x * α strongly converges to x * and x α is a bounded net weakly convergent to x. So, we have, passing to the limit in (5.9), that

∀x ′ ∈ C, lim sup α∈A f θα (x α ) ď f θ (x ′ ) -x * , x ′ -x .
So, all we need to prove now is (5.10)

f θ (x) ď lim sup α∈A f θα (x α ), since we would have ∀x ′ ∈ C, f θ (x) ď f θ (x ′ ) -x * , x ′ -x ,
and x * ∈ ∂(δ C + f θ )(x) = N C (x) + ∂f θ (x) would follow.
We prove now (5.10) using the hypotheses. Suppose first that dim Y < +∞ and F is Lipschitz continuous on bounded sets. Using the convexity of f θ , and so, its weak lower-semicontinuity, we can write for all α ∈ A : In the last term, we have (θθ α ) which converges to zero for the norm topology (recall dim Y < +∞). Moreover, (x α ) α∈A is bounded in X and F is Lipschitz continuous on bounded sets, so F (x α ) is also bounded. It follows that (θθ α )(F (x α )) goes to zero, and (5.10) is proved.

f θ (x) ď lim inf α∈A f θ (x α ) ď lim sup α∈A (θ α • F )(x α ) + lim sup α∈A θ -θ α , F (x α ) .
Suppose now that F (x) ĺ F (x α ) is satisfied. In particular (see Proposition 2.3.7), for all α ∈ A,

(θ α • F )(x) ď (θ α • F )(x α ).
Taking the limsup over α ∈ A on the inequality above, leads to

f θ (x) ď lim inf α∈A f θα (x α )
and proves again (5.10).

We now study the strong-weak outer semi-continuity of ∂ C F . In the following, we will use of this set of hypotheses : One of the three following properties is satisfied :

H 1) K is polyhedral,
2) Y has finite dimension, X is reflexive and F is positively Clarke regular, 3) F is the sum of a convex function and a strictly Gateaux differentiable function.

Observe that there is in these three hypotheses some trade-off between the regularity of the function F and the properties of X, Y, K).

Proposition 5.1.13 (Strong-weak sequential outer semi-continuity, without constraint). Suppose that H holds, and let (x α , x * α ) n∈N be a net in X × X * such that (5.11)

x α • --→ α∈A x, x * α w * --→ α∈A x * , and x * α ∈ ∂ C F (x α ). Then x * ∈ ∂ C F (x).
Proof. Because of the hypothesis, we will prove this statement in two times: first we will consider that F is the sum of a convex and a strictly Gateaux differentiable function, and in a second time that either H1) or H2) holds. Suppose then that F = G + H, with G being convex and H strictly Gateaux differentiable. Thanks to Proposition 5.1.8, and applying the sum rule for the convex subdifferential, we have for all α ∈ A that there exists θ α satisfying x * α ∈ ∂g θα (x α ) + θ α • DH(x α ). Without loss of generality, we can assume that θ α weakly * converges to some θ ∈ Θ.

We know from Proposition 2.1.11 that the strict Gateaux differentiability of H in a neighbourhood of x implies that DH : U → L(X, Y) is continuous, L(X, Y) being endowed with the topology of the pointwise convergence. In other words, (5.12) for all d ∈ X, DH(x α ; d)

• Y --→ α∈A DH(x; d).
Hence, for all d ∈ X, we can use the boundedness and weak * convergence of (θ α ) α∈A to obtain :

(5.13) lim

α∈A θ α • DH(x α )(d) = lim α∈A θ α , DH(x α ; d) Y * ×Y = θ, DH(x; d) Y * ×Y = θ • DH(x)(d).
The latter being true for any d ∈ X, it means that θ α • DH(x α ) converges weakly * to θ • DH(x).

Write now xα

:= x * α -θ α • DH(x α ) ∈ ∂g θα (x α ).
From what we just have shown, xα weakly * converges to x := x *θ • DH(x). Use the convexity of g θα to write, for all α ∈ A, (5.14) ∀y ∈ X, g θα (y)g θα (x α )xα , yx α ě 0.

The first term g θα (y) = θ α , G(y) tends to g θ (y), using the weak * convergence of (θ α ) α∈A . The second term g θα (x α ) converges to g θα (x) because of the weak * convergence of the bounded net (θ α ) α∈A , and the strong convergence of F (x α ) to F (x). In the third term, we use strong-weak * convergence of (x α , x * α ) to (x, x * ), together with the fact that (x * α ) α∈A is bounded, this being a consequence of the local boundedness of ∂ C F (see Proposition 5.1.7). We finally obtain (5.15) ∀y ∈ X, g θ (y)g θ (x)x, yx ě 0, and it follows that x ∈ ∂g θ (x). Hence,

x * = x + θ • DH(x) ∈ ∂g θ (x) + θ • DH(x) ⊂ ∂ C F (x).
Now we suppose that either H1) or H2) holds. In both cases, the outer semi-continuity of ∂ C F will be a direct consequence of the upper semi-continuity of d C F (•, d), that we prove first.

Let then d be in X, consider a converging net x α

• X --→ α∈A
x, and show that (5.16) limsup

α∈A d C F (x α , d) ď d C F (x, d).
We claim that there exists for all α ∈ A some

θ α ∈ Θ such that d C F (x α ; d) = d C f θα (x α ; d).
To see this when H1) holds, use Proposition 5.1.5, and if H2) holds, use Proposition 5.1.8. We assume that θ α converges to some θ ∈ Θ, taking eventually a subnet. Consider a subnet (α β ) β∈B for which the limsup is attained in (5.16), i.e.

(5.17) lim

β∈B d C f θα β (x α β ; d) = lim sup α∈A d C f θα (x α ; d) (and so = lim sup α∈A d C F (x α ; d)).
What we aim to show now is that (5.18) lim sup

β∈B d C f θ (x α β ; d) -d C f θα β (x α β ; d) ě 0.
Note c the limsup appearing in (5.18). By definition of Clarke's directional derivative,

(5.19) c = lim sup β∈B lim sup x(β) →xα β t↓0 f θ (x (β) + td) -f θ (x (β) ) t -lim sup x(β) →xα β t↓0 f θα β (x (β) + td) -f θα β (x (β) ) t .
For all β ∈ B, let x(β) γ --→ γ∈Γ x α β and t γ --→ γ∈Γ 0 be two nets such that lim sup

x(β) →xα β t↓0 f θα β (x (β) + td) -f θα β (x (β) ) t = lim γ∈Γ f θα β (x (β) γ + t γ d) -f θα β (x (β) γ ) t γ .
Injecting the latter in (5. [START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF], we obtain

c = lim sup β∈B lim sup x(β) →xα β t↓0 f θ (x (β) + td) -f θ (x (β) ) t -lim γ∈Γ f θα β (x (β) γ + t γ d) -f θα β (x (β) γ ) t γ ě lim sup β∈B lim sup γ∈Γ f θ (x (β) γ + td) -f θ (x (β) γ ) t γ - f θα β (x (β) γ + t γ d) -f θα β (x (β) γ ) t γ = lim sup β∈B lim sup γ∈Γ θ -θ α β , F (x (β) γ + t γ d) -F (x (β) γ ) t γ Y * ×Y . Now, use the fact that F is Lipschitz continuous around x to write (5.20) c ě lim sup β∈B lim sup γ∈Γ -θ -θ α β Y Lip(F, x) d X .
Since Y has finite dimension, the right member in (5.20) tends to zero, so c ě 0 and (5.18) is proved. Now, use (5.17), (5.18), together with the upper semi-continuity of d C f θ (•, d) (see Section 2.2.4) to deduce lim sup

α∈A d C F (x α ; d) = lim sup α∈A d C f θα (x α ; d) = lim β∈B d C f θα β (x α β ; d) ď lim sup β∈B d C f θ (x α β ; d) ď d C f θ (x; d) ď d C F (x; d),
which proves the upper semi-continuity of d C F (•; d). As an immediate consequence, simply using its definition, we obtain the outer semi-continuity of ∂ C F . Corollary 5.1.14 (Bounded strong-weak sequential outer semi-continuity). Suppose H holds, and let (x α , x * α ) n∈N be a bounded net in X × X * such that (5.21)

x α • -→ x, x * α w * -→ x * , and x * α ∈ N C (x α ) + ∂ C F (x α ). Then x * ∈ N C (x) + ∂ C F (x).
Proof. For all α ∈ A, we can write

x * α = η * α + u * α , where η * α ∈ N C (x α ) and u * α ∈ ∂ C F (x α
). Because of the strong convergence of x α to x, and using Proposition 5.1.7, we know that (u * α ) α∈A is bounded. Hence, taking eventually a subnet, it weakly * converges to some u * , and we know from Proposition 5.1.13 that u * ∈ ∂ C F (x). Define now η * := x *u * , which is the weak * limit of η * α . From η * α = x * αu * α , we know that (η * α ) α∈A is bounded. So, we deduce from the upper semi-continuity of N C that η * ∈ N C (x) (see Proposition 2.2.4).

Descent directions

Descent direction and Fermat's rule

Recall that C ⊂ X is a nonempty closed convex set, which models the constraints. Definition 5.2.1 (Vector descent direction). We say that d ∈ X is a K-descent direction for F at x ∈ C if d C F (x, d) < 0. For convenience, we will just say that d is a descent direction. We say that it is an admissible descent direction if, moreover, d lies in the tangent cone T C (x).

Because of the positive homogeneity of d C F (x, •), we see that the set of descent directions at x form a strict cone. It is important to note that the definition of decent direction is independent of the choice of Θ: Proposition 5.2.2. Let Θ and Θ ′ be two w * -compact bases of K + . Then for all x, d ∈ X,

d C Θ F (x, d) < 0 if and only if d C Θ ′ F (x, d) < 0.
Proof. Suppose by contradiction that there exists some d). Since Θ ′ ⊂ K and Θ is a base of K, there exists λ n > 0 and θ n ∈ Θ for all n ∈ N such that θ ′ n = λ n θ n . In particular, f θ ′ n = λ n f θn . Let us verify that the sequence (λ n ) n∈N is bounded from above. Indeed, suppose that λ n tends to +∞ (taking eventually a subsequence). Since λ n θ n lies in Θ ′ , which is bounded, the sequence (λ n θ n ) n∈N is bounded from above by some M > 0. Hence, θ n ď M λn tends to zero. But Θ is w * -closed, thus • -closed, so 0 ∈ Θ would follow, which is impossible for a base. Now we are assured that λ n ď λ for all n ∈ N, for λ > 0. So

d ∈ X such that d C Θ F (x, d) < 0 and d C Θ ′ F (x, d) ě 0. Let θ ′ n be a sequence in Θ ′ such that 0 d C Θ ′ F (x, d) = lim n→+∞ d C f θ ′ n (x,
0 d C Θ ′ F (x, d) = lim n→+∞ d C f θ ′ n (x, d) = lim n→+∞ λ n d C f θn (x, d) λ lim n→+∞ d C f θn (x, d) λd C Θ F (x, d) < 0,
which is a contradiction. In other words, it is the intersection of the interiors of the tangents cones to [f 1 ď f 1 (x)] and [f 2 ď f 2 (x)]. We will see in Theorem 5.2.12 that under some conditions, it is exactly the interior of the tangent cone to

[F ĺ F (x)] = [f 1 ď f 1 (x)] ∩ [f 2 ď f 2 (x)].
Figure 5.4: In dotted lines: the strict cone of descent directions for F = (f 1 , f 2 ) at x.

We say that x ∈ C is a critical point of F if 0 ∈ N C (x) + ∂ C F (x).
According to the following result, this notion coincide with the one introduced by Smale [START_REF] Smale | Global analysis and economics I: Pareto optimum and a generalization of Morse theory[END_REF]. In other words, critical points are exactly the points on which there is no admissible descent direction available. Theorem 5.2.4 (Pareto alternative). Let x ∈ C. Then one and only one of the following statements is true :

• x is a critical point,
• there exists an admissible descent direction at x. Remark 5.2.5. As a consequence of this Theorem and Proposition 5.2.2, the fact that x is critical does not depend on the choice of the base Θ.

Proof. Suppose first that 0 ∈ N C (x) + ∂ C F (x). Then there exists some x * ∈ ∂ C F (x) such that -x * ∈ N C (x). Since N C (x) is defined as the polar cone of T C (x), for all d ∈ T C (x) we have -x * , d ď 0. From this we can deduce that

d C F (x; d) = sup x * ∈∂ C F (x)
x * , d ě 0 for all d ∈ T C (x).

As a consequence, there is no admissible descent direction at x.

Suppose now that 0 / ∈ N C (x) + ∂ C F (x). Since ∂ C F (x) is convex and w * -compact and N C (x) is weakly * closed convex, we have that N C (x) + ∂ C F (x) is closed and convex in (X * , w * ). Using Hahn-Banach's separation theorem [325, Theorem 1.1.5], we obtain some d ∈ X and α ∈ R such that

(5.22) ∀x * ∈ N C (x) + ∂ C F (x), x * , d ď α < 0. Since 0 ∈ N C (x), it follows directly that d C F (x; d) = σ ∂ C F (x) (d) ď α < 0.
Moreover, using the weak * compactness of ∂ C F (x) and the weak * continuity of d as a linear functional over X * , we can deduce that σ

N C (x)+∂ C F (x) (d) = σ ∂ C F (x) (d) + σ N C (x) (d).
As a direct consequence of (5.22),

σ N C (x) (d) ď α -σ ∂ C F (x) (d). Since ∂ C F (x) is compact, σ N C (x) (d) < +∞ follows.
We can see, using (N C (x)) * = T C (x), that this is equivalent to d ∈ T C (x). , for a bi-objective optimization problem. We take two points x 1 and x 2 , and we plotted the two corresponding strict cones of descent directions at x 1 and x 2 respectively. The thick dashed line represents the set of critical points. One can see that the closer x is to the set of Pareto points, the smaller is the strict cone of descent directions.

Remark 5.2.6. Note that in the smooth multi-objective case, this alternative theorem is exactly Gordan's alternative theorem [START_REF] Gordan | Über die Auflösung linearer Gleichungen mit reellen Coefficienten[END_REF],[77, Theorem 2.2.1].

We define now a stronger notion of descent direction, useful for designing algorithms. Recall that in all this Chapter, e denotes the element of int K, generating the base Θ of K + . Definition 5.2.7 (Armijo descent direction). Let d ∈ X be a descent direction at x ∈ C. We say that d is an Armijo descent direction at x if (5.23)

∀β ∈]0, 1[, ∃ε > 0 such that ∀t ∈]0, ε[, F (x + td) ĺ F (x) + βtd C F (x, d)e.
We say that d is an admissible Armijo descent direction, if moreover x + td ∈ C for all t ∈]0, ε[.

Observe that admissible Armijo directions are exactly Armijo directions which lie in T ad C (x), the admissible tangent cone to C at x (see Section 2.2.1). Note also that e being in int K, an Armijo descent direction d satisfies in particular

F (x + td) ă F (x) for all t ∈]0, ε[.
In a way, Armijo descent directions for F can be seen as vectors satisfying a kind of uniform Armijo condition for the family of cost functions {f θ } θ∈Θ : Proposition 5.2.8. Let x ∈ C and d ∈ X be a descent direction for F at x. Consider the following statements :

i) ∀β ∈]0, 1[, ∃ε > 0 s.t. ∀θ ∈ Θ, ∀t ∈]0, ε[, f θ (x + td) f θ (x) + βtd C f θ (x, d). ii) ∀β ∈]0, 1[, ∃ε > 0 s.t. ∀θ ∈ Θ, ∀t ∈]0, ε[, f θ (x + td) f θ (x) + βtd C F (x, d).
iii) d is an Armijo descent direction for F at x.

iv) ∃ β ∈]0, 1[, ∀β ∈]0, β[, ∃ε > 0 s.t. ∀θ ∈ Θ, ∀t ∈]0, ε[, f θ (x + td) f θ (x) + βtd C f θ (x, d). Then, i) ⇒ ii) ⇔ iii) ⇒ iv).
Proof. From the definition of d C F (x, d) = sup θ∈Θ d C f θ (x, d), we see that i) implies ii). In order to prove the equivalence between ii) and iii), use Proposition 2.3.7. Indeed, let β ∈]0, 1[ and t > 0, then

F (x + td) ĺ F (x) + βtd C F (x, d)e ⇔ ∀θ ∈ Θ, f θ (x + td) f θ (x) + tβd C F (x, d) (because θ(e) = 1).
Assume now item ii), and define

α := inf θ∈Θ d C f θ (x, d) d C F (x, d)
which is bounded by Lipschitz property of F and boundedness of Θ. By definition, α 1. Let then β ∈]0, 1 α [, and show that item iv) holds for this β. Since αβ ∈]0, 1[, we can apply item ii) to have some ε > 0 such that for all t ∈]0, ε[ and θ ∈ Θ,

f θ (x + td) f θ (x) + αβtd C F (x, d).
But, for all θ ∈ Θ, we have αd C F (x, d) ď d C f θ (x, d), so the desired result follows with β = 1 α .

We show now that, under assumptions, the Armijo descent directions coincide with the descent direction, and that they characterize the critical points of F (recall Theorem 5.2.4). Proposition 5.2.9. Let x ∈ C, and suppose that either dim Y < +∞, or F is the sum of a convex function and a directionally derivable function. i) If d ∈ X is a descent direction at x, then it is an Armijo descent direction at x.

ii) There exists an admissible descent direction at x if and only if there exists an admissible Armijo descent direction at x.

Proof. i) Let d ∈ X be a descent direction at x, that is, d C F (x, d) < 0. We argue by contradiction, using Proposition 5.2.8: suppose that there exists β ∈]0, 1[ such that ∀ε > 0, there exists t ∈]0, ε[ and θ ∈ Θ satisfying (5.24)

f θ (x + td) > f θ (x) + βtd C F (x, d).
In other words, there exists a net (t α ) α∈A in R ++ converging to zero, and a net (θ α ) α∈A in Θ weakly * converging to some θ ∈ Θ (use the compactness of Θ), such that for all α ∈ A (5.25)

βd C F (x, d) < f θα (x + t α d) -f θα (x) t α .
Suppose in a first time that dim Y < +∞, and write (5.26)

f θα (x + t α d) -f θα (x) t α = f θ (x + t α d) -f θ (x) t α + θ α -θ, F (x + t α d) -F (x) t α .
Using the Lipschitz property of F , we see that the second term of the right member of (5.26) is bounded from above by θ αθ Lip(F, x) d when t α is close enough to zero. So, using the norm-convergence of θ α to θ, we deduce that lim sup

α∈A f θα (x + t α d) -f θα (x) t α ď lim sup α∈A f θ (x + t α d) -f θ (x) t α d C f θ (x, d) d C F (x, d).
It follows that βd C F (x, d) d C F (x, d) which is in contradiction with the facts that d C F (x, d) 0 and β ∈]0, 1[. Suppose now that F = G + H, with G being convex and H being directionally derivable at x. Then, (5.25) rewrites as (we note g θ := θ • G as usual)

(5.27) βd C F (x, d) < g θα (x + t α d) -g θα (x) t α + θ α , H(x + t α d) -H(x) t α
, for all α ∈ A.

Consider any fixed t > 0. Using the convergence of t α to zero, we obtain some ᾱ ∈ A such that for all α ᾱ, t > t α . Write (5.28)

x + t α d = t α t (x + td) + (1 - t α t )x, with t α t ∈]0, 1[ for all α ᾱ.
The convexity of G in (5.27) together with (5.28) leads to:

(5.29)

βd C F (x, d) < g θα (x + td) -g θα (x) t + θ α , H(x + t α d) -H(x) t α
, for all α ᾱ. Since (5.29) holds for all α ᾱ, we can take the liminf over α ∈ A :

(5.30)

βd C F (x, d) ď lim inf α∈A g θα (x + td) -g θα (x) t + θ α , H(x + t α d) -H(x) t α .
Using the bounded weak * convergence of (θ α ) α∈A to θ α , together with the directional derivability of H at x, we deduce from (5.30) that (5.31)

βd C F (x, d) ď g θ (x + td) -g θ (x) t + θ, DH(x, d) .
This being true for any fixed t > 0, we obtain by taking the limit when t ↓ 0, together with (2.24):

(5.32)

βd C F (x, d) ď d C (θ • G)(x, d) + θ, DH(x, d) d C F (x, d).
As before, we see here a contradiction, and item i) is proved. ii) From Definition 5.2.7 and T ad C (x) ⊂ T C (x), it is obvious that the admissible Armijo descent directions are admissible descent directions. We prove here the reverse statement. Let d ∈ X be an admissible descent direction at x, that is d ∈ T C (x) and d C F (x, d). Since C is convex, we know from Proposition 2.2.2 that T C (x) = cl T ad C (x). Hence, there exists a sequence

(d n ) n∈N in T ad C (x) converging in norm to d. Moreover, d C F (x,
•) is continuous, s there must exist some N ∈ N such that d C F (x, d N ) < 0, which proves the claim.

As a consequence, we derive a natural Fermat's rule for vector optimization problems, giving a necessary condition for weak efficiency. In the convex setting, it becomes a necessary and sufficient condition, and we recover a known description of weak efficient points [139, Theorem 2.10].

Theorem 5.2.10 (Fermat's rule). Suppose that either dim Y < +∞, or F is the sum of a convex function and a directionally derivable function. Then any weak efficient point x is critical, i.e.

0 ∈ N C (x) + ∂ C F (x).
If F is convex, then critical points coincide with weak efficient points, and we have If F is strictly convex, then critical points are efficient points.

Remark 5.2.11. According to (5.33), a good approach for generating the set of Pareto points of a convex function F , is to take a sample among the cost functions {f θ } θ∈Θ , and minimize them. In the multi-objective setting F = (f 1 , ..., f m ), this is exactly the weighting method, which consists in the minimization of arbitrary (or chosen) convex combinations of the f i 's. We discuss some drawbacks of this method later in Section 6.3.

Proof. Let x ∈ C be a weak efficient point. We argue by contradiction and suppose that x is not critical. By Theorem 5.2.4, there would exist an admissible descent direction at x. Because of the hypotheses, we can apply Proposition 5.2.9, so there exists d ∈ X an admissible Armijo direction. In particular, there exists ε > 0 such that for all t ∈]0, ε[, (5.34)

x + td ∈ C and F (x + td) ă F (x).

Take y := x + td for some t ∈]0, ε[ and sees that it contradicts the weak efficiency of x. Suppose now that F is convex, and show that critical points are weakly efficient. We know from Proposition 5.1.4 that in the convex case, (5.35)

∂ C F (x) = θ∈Θ ∂f θ (x).
If x is critical, there must exist θ ∈ Θ such that 0 ∈ N C (x) + ∂f θ (x). Using the classic Fermat's rule for convex scalar functions, we see that it is equivalent for x to minimize f θ over C. If we suppose now that x is not weakly efficient, it would exist x ′ ∈ C such that F (x ′ ) ă F (x).

In particular (see Proposition 2.3.7) we would have f θ (x ′ ) < f θ (x) which would contradict x ∈ argmin y∈C f θ . As a consequence, critical points are weak minimizers.

Note that the equality (5.33) is proved by the same argument : x is weakly efficient if and only if it is a critical point, this being equivalent to say (see above) that there exists θ ∈ Θ such that x ∈ argmin y∈C f θ .

For the strictly convex case, argue exactly as for the convex case, just using the fact that a scalar-valued strictly convex function admits at most one minimizer.

Normals to level sets

There is an other geometric consequence of Proposition 5.2.9 : under appropriate assumptions, the subdifferential generates the Bouligand normal cone to the sublevel sets.

Theorem 5.2.12 (Normals to sublevel sets). Let x ∈ X be such that 0 / ∈ ∂ C F (x). Suppose that either F is weakly Clarke regular with dim Y < +∞, or F is the sum of a convex function and a strictly Gateaux differentiable function at x.

Then i) T B [F ĺF (x)] (x) = {d ∈ X | d C F (x, d) ď 0}, ii) int T B [F ĺF (x)] (x) = {d ∈ X | d C F (x, d) < 0} (i.e. the descent directions at x), iii) N B [F ĺF (x)] (x) = R + ∂ C F (x). Example 5.2.13. Let F = (f 1 , f 2 ) : R 2 -→ (R 2 , R 2
+ ) be a smooth convex function and x ∈ R 2 . We plot in Figure 5.6 the sublevel set [F ĺ F (x)] in thick continuous curves, and dotted lines represent respectively the normal and tangent cone to [F ĺ F (x)] at x. Proof of Theorem 5.2.12. We start by showing the inclusion (5.36)

T B [F ĺF (x)] (x) ⊂ {d ∈ X | d C F (x, d) ď 0}. So let us consider d such that there exists t n ↓ 0 and d n → d such that x + t n d n ∈ [F ĺ F (x)] for all n ∈ N. That is, f θ (x + t n d n ) ď f θ (x)
for all θ ∈ Θ. After division by t n > 0 and taking the liminf on n ∈ N, we obtain that d B f θ (x; d) ď 0 for all θ ∈ Θ. In other words, because of the positive Clarke regularity of F , d C F (x, d) ď 0.

We show now that

(5.37) {d ∈ X | d C F (x, d) < 0} ⊂ T B [F ĺF (x)] (x). Consider d ∈ X such that d C F (x, d) < 0.
It is a descent direction at x, so it is an Armijo descent direction, according to Proposition 5.2.9 and the hypotheses made on F . In particular, there exists some ε > 0 such that x + td ∈ [F ĺ F (x)] for all t ∈]0, ε[. By definition of the Bouligand tangent cone, it follows that d ∈ T B [F ĺF (x)] (x) and (5.37) is proved. In order to obtain item i), we will take the closure in (5.37). The left member is the strict sublevel set of the convex continuous function d

C F (x, •) = d B F (x, •). It is nonempty since it is assumed that 0 / ∈ ∂ C F (x)
, so we can use Therorem 5.2.4. Hence, we can apply Proposition A.1.4 which says that:

cl {d ∈ X | d C F (x, d) < 0} = {d ∈ X | d B F (x, d) ď 0}, (5.38) {d ∈ X | d C F (x, d) < 0} = int {d ∈ X | d B F (x, d) ď 0}. (5.39)
From (5.37), (5.38) and the closure of T B [F ĺF (x)] (x), we deduce item i). Item ii) follows item i) and (5.39).

To prove item iii), start from item i) and write

T B [F ĺF (x)] (x) = {d ∈ X | d C F (x, d) ď 0} (5.40) = {d ∈ X | x * , d ď 0 ∀x * ∈ ∂ C F (x)} = {d ∈ X | x * , d ď 0 ∀x * ∈ R + ∂ C F (x)} = (R + ∂ C F (x)) * .
Since ∂ C F (x) is w * -compact and does not contain the origin, Proposition 2.1.7 ensures that R + ∂ C F (x) is a closed convex cone. By taking the polar on (5.40) we obtain the desired inequality.

Remark 5.2.14. It must be underlined that, while we are minimizing a function F from X to Y, all the objects involved here (the ordered subdifferential, the sublevel sets) live in the decision space X or its dual. Of course, the order on Y determines these objects, but it is somehow implicit. This will be determinant in Chapter 7, where we construct an auxiliary function F : X -→ Z, for some well chosen ordered Banach space Z, which has the same sublevel set than F , and so, the same ordered subdifferential (see Theorem 7.3.3).

Remark 5.2.15. In a recent work [START_REF] Cabot | Sequential formulae for the normal cone to sublevel sets[END_REF], Cabot and Thibault give a general description of N [F ĺF (x)] (x) in the multi-objective convex case, without the condition 0 / ∈ ∂ C F (x). When this condition is satisfied, they recover

N [F ĺF (x)] (x) = R + ∂ C F (x).
It would be of interest to overcome the hypothesis 0 / ∈ ∂ C F (x) in Theorem 5.2.12 by adapting the ideas in [91, Theorem 5.1] to the general vector case. Remark 5.2.16. It is a known fact in scalar optimization (see [START_REF] Clarke | Optimization and nonsmooth analysis[END_REF]Theorem 2.4.7] or [START_REF] Penot | Calculus without derivatives[END_REF]Proposition 5.48]) that the sublevel sets of Clarke regular functions are themselves Clarke regular, which means that their Clarke and Bouligand tangent cones coincide. This result extends to our vectorial case when K is polyhedral: it is easy to exploit the finite number of objective functions to adapt the argument of [START_REF] Penot | Calculus without derivatives[END_REF]Proposition 5.48]. It is not clear whether it is true or not for nonpolyhedral cones. An other approach consists in writing [F ĺ F (x)] as the infinite intersection of sublevel sets [f θ ď f θ (x)]. Since the inclusions

T C [F ĺF (x)] (x) ⊂ T B [F ĺF (x)] (x) ⊂ θ∈Θ T B [f θ ďf θ (x)] (x)
always hold, it is enough to prove, after using the Clarke regularity of Γ θ , that

θ∈Θ T C [f θ ďf θ (x)] (x) ⊂ T C [F ĺF (x)] (x).
This kind of inclusion is known to be valid when a finite number of sets is involved, under appropriate conditions, see [START_REF] Rockafellar | Clarke's tangent cones and the boundaries of closed sets in R n[END_REF]Theorem 5]. But is is a nontrivial question to know whether it can be extended to an infinite number of sets.

The steepest descent direction in Hilbert spaces

Suppose now that X = H is a Hilbert space, endowed with its scalar product •, • , and identify H with its dual H * . We still suppose that Y is an arbitrary Banach space. We aim to introduce an analog of the usual steepest descent direction used in scalar optimization. Recall that we note C 0 the element of minimal norm of a closed convex set3 .

Definition 5.2.17. For any x ∈ C, the unique element of minimal norm of the closed convex set -N C (x) -∂ C F (x) is called the steepest descent direction of F at x. It is denoted by

(5.41) s(x) := -N C (x) -∂ C F (x) 0 .
Note that, for any x ∈ C, the set -N C (x) -∂ C F (x) is a closed convex set, as being equal to the vectorial sum of two closed convex sets, one of them being weakly compact. Hence, it has a unique element of minimal norm, and s(x) is well defined. This vector field clearly satisfies that x is a critical point if and only if s(x) = 0. Furthermore, s(x) is an admissible descent direction for F at any non critical x : Proposition 5.2.18 (Descent property). For all x ∈ C we have s(x) ∈ T C (x) and

(5.42) d C F (x; s(x)) ď -s(x) 2 .
In particular, s(x) is an admissible descent direction for F at any x, as soon as x is not critical.

Proof. By definition, -s(x) is the projection of the origin onto the closed convex set N C (x) + ∂ C F (x). Hence, using the variational definition of the projection in a Hilbert space and 0 ∈ N C (x), we obtain for any x * ∈ ∂ C F (x) :

(5.43) 0 -(-s(x)), x * -(-s(x)) ď 0.

In other words, (5.44)

s(x) 2 + s(x), x * ď 0 for all x * ∈ ∂ C F (x),
and the desired inequality follows, by taking the sup over x * ∈ ∂ C F (x). Suppose now that x is not critical. As noticed before, it implies that s(x) = 0, so it makes s(x) a descent direction. Verify now that s(x) is an admissible descent direction. By definition of s(x), we can write s(x) =x * -N C (x) 0 for some x * ∈ ∂ C F (x). Since T C (x) is the polar cone of N C (x), we can use Moreau's decomposition Theorem 2.1.10 and obtain,

(z -N C (x)) 0 = z -proj N C (x) z = proj T C (x) z
which shows that s(x) ∈ T C (x), and concludes the proof.

The result above has a simple geometrical interpretation. Take simply the multi-objective unconstrained problem, i.e., C = H, (Y, K) = (R 2 , R 2 + ) with F = (f 1 , f 2 ) being smooth. Then -s(x) is the orthogonal projection of the origin on the segment [∇f 1 (x), ∇f 2 (x)]. By the classical result on the sum of the angles of a triangle, this forces the angles between -s(x) and ∇f i (x), i = 1, 2, to be acute and so s(x), ∇f i (x) ď 0. Remark 5.2.19. In the multi-objective case F : H -→ (R m , R m + ), F (x) = (f 1 (x), ..., f m (x)), computing the steepest descent direction at x requires to project the origin on the convex hull of the subdifferentials ∂ C f i (x):

-s(x) = proj co∂ C f i (x) (0).

Suppose that we are in a non regular point x, such that the subdifferentials ∂ C f i (x) are not reduced to one point. A naive approach to compute s(x) could be the following: take p i (x) := ∂ C f i (x) 0 for each i ∈ {1, ..., m}, and then compute -s(x) as the element of minimal norm of co {p i }. But this approach doesn't work, because the operations of taking the convex hull and doing the projection do not commute. Example 5.1.10 provides a simple counter-example to this situation. In this example, such a direction would not be a descent direction because x is critical (recall the alternative Theorem 5.2.4).

Now that we have established that s(x) is a descent direction, one may wonder why it is called the steepest descent direction. Observe first that in the case of a smooth real-valued function f : H -→ R, the direction s(x) at x ∈ C is given by (use Moreau's theorem 2.1.10)

s(x) = (-N C (x) -∇f (x))) 0 , = proj T C (x) (-∇f (x)).
It is known in that setting that the normalized vector s(x) s(x) is the solution, if ∇f (x) = 0, of the minimization problem

min { ∇f (x), d | d ∈ T C (x), d = 1} ,
whence the name of steepest descent direction for proj T C (x) (-∇f (x)). As shown below, this steepest descent property can be extended to vector-valued functions. Moreover, still in the case of a smooth real-valued function, it can be easily verified that (5.45) s(x) = argmin

d∈T C (u) 1 2 d 2 + ∇f (x), d ,
and this further characterization will also be generalized to the multiobjective case.

Theorem 5.2.20 (Steepest descent property). Let x ∈ C be such that s(x) = 0. Then s(x) can be formulated in the following equivalent forms:

1. s(x) = -N C (x) -∂ C F (x) 0 , 2. s(x) s(x) p-2 p-1 = argmin d∈T C (u) 1 p d p + d C F (x, d), ∀p ∈ [2, +∞[, 3. s(x) s(x) = argmin d∈T C (x) d =1 d C F (x, d).
Remark 5.2.21. The equivalence between formulations 1. and 3. of the steepest descent direction has been first obtained, in the multicriteria and smooth case, by Cornet in [116, Proposition 3.1]. In the same context, formulation 2. seems to be introduced for the first time by Fliege and Svaiter [START_REF] Fliege | Steepest descent methods for multicriteria optimization[END_REF], for p = 2

s(u) = argmin d∈T C (u) 1 2 d 2 + d C F (x, d) ,
but the equivalence between formulations 1. and 2. is seemingly new. The second formulation for p = 2 is new, although it was stressed in [159] that 1 2 d 2 could be replaced by any positive proper l.s.c strictly convex function which is dominated by the norm around the origin. The interest of considering p arbitrary large is that we can see -at least formally-the third formulation as the limit of the second when p → +∞: p-2 p-1 tends to 1, while the function , where s = s(x). We use a duality argument which relies on the equivalent formulation of (5.46) as the convex-concave saddle value problem (5.47) min

d∈T C (x) max a∈∂ C F (x) 1 p d p + a, d .
It is associated with the convex-concave Lagrangian function , as expected.

L(d, a) = 1 p d p + a, d defined on T C (x) × ∂ C F (x).
L(d, ā) = L( d, ā) = sup a∈∂ C F (x) L( d, a).
Let us complete the proof by proving the third characterisation. As we said before, it relies on a limit argument. Define, for any p ě 2, the functions

F p : d ∈ H → 1 p d p +d C F (x; d)+δ T C (x) (d).
It can be easily verified that the sequence (F p ) pě2 epiconverges when p → +∞ to

F : d ∈ H → δ { • ď1} (d) + d C F (x; d) + δ T C (x) (d).
From item 1 and [21, Theorem 12.1.1] we can deduce that (5.57)

s s = argmin v∈T C (x) v ď1 d C F (x; d),
where the inequality constraint v ď 1 can be replaced by v = 1, since s s is a normalized vector.

Example 5.2.22. We work in the same setting that Example 2.3.8, and we assume to simplify that F : X -→ Y is strictly Gateaux differentiable and C = H.

• If (Y, K) = (R m , R m + ), s(x)
is the element of minimal norm in the convex hull of the gradients ∇f i (x), and is characterized by s(x) s(x) = argmin

d =1 max i∈{1,...m} ∇f i (x), d . • If (Y, K) = (S m (R), S m + (R)), then s(x) s(x) = argmin d =1
λ max (DF (x; d)), where λ max (M ) is the greatest eigenvalue of M .

• If (Y, K) = (L ∞ (Ω, Σ, µ), L ∞ + (Ω, Σ, µ)), then s(x) s(x) = argmin d =1
supess (DF (x; d)).

Comments

We end here with a couple of remarks about this chapter.

Remark 5.3.1 (The choice of the Clarke subdifferential). We chose to build the ordered subdifferential on the Clarke subdifferential for several reasons. The first one, and the most obvious, is that it enjoys very good duality properties with its Clarke directional derivatives, like the max formula (5.3). This allows us to characterize easily the ordered subdifferential ∂ C F (x) with its support function, namely, the ordered directional derivative d C F (x; •). An other reason, directly related to the first one, is that the Clarke directional derivative provides a good Armijo rule at descent directions (see Proposition 5.2.9). If we use for instance the Bouligand directional derivative for a function f : X -→ R, we would have a poorer Armijo rule (see for instance [START_REF] Lewis | Nonsmooth optimization via quasi-Newton methods[END_REF]Section 4]):

d B f (x; d) < 0 ⇒ ∀β ∈]0, 1[, ∀T > 0, there exists t 0 ∈]0, T [ and ε > 0 such that f (x + td) < f (x) + βtd B f (x; d), ∀t ∈ [t 0 -ε, t 0 + ε].
While the Armijo rule has a clear importance for the study of descent algorithms with linesearch, it has also a theoretical interest for us, since it is the key to the proof of Fermat's Theorem 5.2.10.

The last reason is that the convexity of the Clarke subdifferential is essential for us, since our main results rely on the separation of convex sets, see the proof of the alternative Theorem 5.2.4. Of course, one can try to go beyond convexity, using for instance the limiting Fréchet subdifferential. This is the approach adopted by Morduckhovich [START_REF] Mordukhovich | Variational analysis and generalized differentiation II: Applications[END_REF] and Bao, Mordukhovich [START_REF] Bao | Sufficient conditions for global weak Pareto solutions in multiobjective optimization[END_REF], see also [START_REF] Bento | A proximal point-type method for multicriteria optimization[END_REF]. Such an approach would, of course, benefits from the fact that the limiting subdifferential is smaller. Thus, it would provide more accurate necessary conditions for weak optimality (see [START_REF] Ioffe | Critical points of simple functions[END_REF] for a discussion about criticalities). But it would ask for a completely different approach, less centered on the primal space and its directional derivatives.

This remark is also the occasion to say that, in most of the results of this chapter, we could have considered nonconvex constraints. If the constraint is a nonempty closed set Ω ⊂ X, it suffices to consider the Clarke tangent and normal cones to Ω. These cones are closed and convex, and enjoys a full duality by taking the polar. The only point for which more regularity is needed is Proposition 5.2.9, for which we would need the fact that the admissible tangent cone is dense in the Clarke tangent cone. Such a property is satisfied if Ω is for instance prox-regular.

Remark 5.3.2 (Relationship with generalized Jacobians, and other works). From the beginning of nonsmooth analysis, there have been attempts to provide good generalizations of the differential for nonsmooth vector-valued functions. For a locally Lipschitz continuous function F : R n -→ R m , Clarke [START_REF] Clarke | Optimization and nonsmooth analysis[END_REF] defined the Clarke generalized Jacobian of

F at x ∈ R n by (5.58) J F (x) := co{ lim n→+∞ DF (x n ) | x n --→ n→+∞ x and F is differentiable at x n } ⊂ L(R n , R m ).
This definition is based on Rademacher's theorem, which asserts that a locally Lipschitz function is Gateaux differentiable almost everywhere. When m = 1, it is equal to the usual Clarke subdifferential introduced in Section 2.2. This Jacobian satisfies the following chain rule (see [START_REF] Clarke | Optimization and nonsmooth analysis[END_REF]Proposition 2.6.4]):

∀y * ∈ Y * , ∂ C (y * • F )(x) = y * • J F (x) := {y * • A | A ∈ J F (x)}.
Thus, if Y is ordered by a closed convex cone with nonempty interior, we can see that the Clarke generalized Jacobian is related to the ordered Clarke subdifferential by:

∂ C F (x) = Θ • J F (x) := {θ • A | θ ∈ Θ, A ∈ J F (x)}.
The latter generalizes Proposition 5.1.8 iii), in finite dimensions. See the work of Thibault [START_REF] Thibault | On generalized differentials and subdifferentials of Lipschitz vector-valued functions[END_REF] for an extension to functions F : X -→ Y between a separable Banach space X and a reflexive separable Banach space Y.

Considering a function F : X -→ Y with values in an ordered Banach space, there has been in the 70's some attempts to build a generalized Jacobian which would respect the order structure of Y. In its seminal work, Valadier [START_REF] Valadier | Sous-différentiabilité de fonctions convexes à valeurs dans un espace vectoriel ordonné[END_REF] considered convex continuous functions with values in an order complete Banach lattice 4 and introduced what we call5 an ordered generalized Jacobian of F at x:

J ĺ F (x) := {A ∈ L(X, Y) | ∀x ′ ∈ X, 0 ĺ F (x ′ ) -F (x) -A(x ′ -x)}.
It is clear that J ĺ F (x) enjoys the following Fermat's rule

0 ∈ J ĺ F (x) ⇔ F (x) ĺ F (x ′ ) ∀x ′ ∈ X.
But, as we mentioned it in Section 2.3.3, this is in general pointless, since in a non totally ordered space, such global minimum does not always exist. It is easy to see that, in the context of this chapter,

Θ • J ĺ F (x) ⊂ ∂ C F (x),
but this inclusion is strict in general since J ĺ F (x) might be empty. When X is a Banach space and Y a reflexive separable Banach space, Thibault [START_REF] Thibault | On generalized differentials and subdifferentials of Lipschitz vector-valued functions[END_REF] extended this notion to locally Lipschitz continuous functions, by taking 6

J ĺ F (x) := cl W OT coo { lim n→+∞ DF (x n ) | x n --→ n→+∞ x and F is differentiable at x n }.
It is an exercise to verify that, in the context of this chapter, we also have

Θ • J ĺ F (x) ⊂ ∂ C F (x).
The generalized Jacobian has been used in the analysis of vector-valued functions, in particular in the Lagrangian theory [START_REF] Clarke | A new approach to Lagrange multipliers[END_REF][START_REF] Hiriart-Urruty | Refinements of necessary optimality conditions in nondifferentiable programming I[END_REF][START_REF] Clarke | Optimization and nonsmooth analysis[END_REF]. But it seems to us that these ordered Jacobians have not been a successful tool for the analysis of vector optimization problems. The fact that they characterize global minimum instead of Pareto/efficient points might be its major drawback.

Remark 5.3.3 (Links with qualification conditions). We say that two sets C, D ⊂ X have a (Bouligand) linearly regular intersection at x ∈ C ∩D, if (see e.g. Example 4.2.13 and [START_REF] Kruger | About regularity of collections of sets[END_REF][START_REF] Lewis | Local linear convergence for alternating and averaged nonconvex projections[END_REF])

N B C (x) ∩ N B D (x) = {0}.
Recall from the alternative Theorem 5.2.4 that there exists an admissible descent direction at

x if and only if 0 / ∈ N C (x) + ∂ C F (x). It can be verified that this noncriticality of x is equivalent to 0 / ∈ ∂ C F (x) and N C (x) ∩ -R + ∂ C F (x) = {0}.
In the context of Theorem 5.2.12, we see that this is equivalent to

0 / ∈ ∂ C F (x) and N C (x) ∩ -N B [F ĺF (x)] (x) = {0},
where this second property expresses that the intersection between the constraint C and the sublevel set [F ĺ F (x)] must be linearly regular at x . We illustrate this situation in Figure 5.8. We take

F = (f 1 , f 2 ) : R 2 -→ (R 2 , R 2 + )
, and consider a linear constraint C (plotted in a continuous line) for which there exists no admissible descent directions at x ∈ C. This is because the intersection between C and [F ĺ F (x)] is not regular : see how This kind of regularity condition arises naturally when considering optimization problems with equality and inequality constraints. Consider for instance a finite family of functions f, g 1 , ..., g m , h 1 , ..., h p : R n -→ R, 6 Here cl W OT denotes the closure with respect with the weak operator norm, which is the topology of the pointwise convergence to respect with the weak topology of Y. In other words, Aα where f and G := (g 1 , ..., g m ) are locally Lipschitz continuous , and H := (h 1 , ..., h p ) is smooth.

N C (x)∩-N [F ĺF (x)] (x) = {0}.
A classical approach to study (P) min x∈R n f (x) such that g 1 (x) ď 0, ..., g m (x) ď 0 and H(x) = 0 consists in writing the corresponding Lagrangian

L(µ, θ, λ, x) := µf (x) + m i=1 θ i g i (x) + p j=1 λ j h j (x), for θ = (θ i ) ∈ R m and λ = (λ j ) ∈ R p ,
and study its critical points. For instance, if x ∈ R n is a local solution for (P), there exists a triplet (µ, θ, λ [START_REF] Hiriart-Urruty | Refinements of necessary optimality conditions in nondifferentiable programming I[END_REF]Theorem 2.1]). The whole point is to find qualification conditions on x which ensures that (µ, θ, λ) are Karush-Kuhn-Tucker coefficients, i.e. µ = 0. Such a standard assumption is the Mangasarian-Fromovitz condition, which demands that (assume that all the constraints are qualified7 at x): i) DH(x) is surjective, ii) there exists d ∈ Ker DH(x) such that d C g i (x; d) < 0, ∀i ∈ {1, ..., m}.

) ∈ R × R m + × R p , called Fritz-John coefficients, such that 0 ∈ ∂ C L(µ, θ, λ, •)(x) (see
According to Liusternik's theorem [START_REF] Borwein | Convex analysis and nonlinear optimization: Theory and examples[END_REF]Theorem 7.1.6], the Mangasaria-Fromovitz condition can be equivalently rewritten as:

i) DH(x) is surjective, ii) [H = 0] and [G ĺ 0] have a strongly regular intersection at x, where [H = 0] := {x ∈ R n | H(x) = 0} and [G ĺ 0] := {x ∈ R n | g i (x) ď 0, i ∈ {1, ..., m}}.
The discussion at the beginning of this remark says in particular that item ii) could be interpreted as the noncriticality of x with respect to the multi-objective optimisation problem (MOP) MIN

H(x)=0

G(x).

Of course, this would require the constraint [H = 0] to be convex, to fall into the setting of Theorems 5.2.4 and 5.2.12, which is false in general8 . We will also see in Chapter 6, Proposition 6.1.17, that the condition i) will provide a good regularity property for the steepest descent vector field s.

Chapter 6

The continuous steepest descent dynamic for vector optimization Along all this chapter, we are considering a locally Lipschitz function F : H -→ Y , where H is a Hilbert space identified with its dual, and Y is a Banach space ordered by a nonempty closed convex cone K having nonempty interior. As in the previous chapter, we equip the dual cone K + with a fixed convex weakly * compact base Θ ⊂ Y * . We refer to Section 2.3 for more details on ordered Banach spaces and bases of cones.

We recall from the introduction that Smale [START_REF] Smale | Global analysis and economics I: Pareto optimum and a generalization of Morse theory[END_REF] studied the multi-objective optimisation problem

(MOP) MIN x∈C F (x), with F (x) = (f 1 (x), ..., f m (x)),
and defined the notion of gradient process for this problem. It is a differential equation

u(t) = ψ(u(t))
where ψ : H -→ H is a mapping which satisfies the following properties: ψ(u) is an admissible descent direction whenever u is not a Pareto critical point, ψ(u) = 0 else.

The interest of such a gradient process is twofold. First, the stationary points of the dynamic, those satisfying u(t) = 0, are exactly the critical Pareto points for (MOP). This is a simple consequence of the Pareto alternative Theorem 5.2.4, which says that the critical Pareto points are exactly those for which no admissible descent direction can be found. Secondly, as long as u(t) is not a critical Pareto point, all the objective functions {f 1 , ..., f m } are guaranteed to decrease.

The steepest descent vector field

s : H -→ H x -→ -(N C (x) + co ∂ C f i (x)) 0 ,
introduced in the previous chapter, induces such a gradient process for multi-objective optimization problems, according to Proposition 5.2.18. The idea of considering a continuous dynamic governed by this vector field goes back to Cornet [START_REF] Cornet | Contributions to the mathematical theory of dynamical mechanisms of resource allocation[END_REF], and has been revisited in the last years [START_REF] Schäffler | Stochastic method for the solution of unconstrained vector optimization problems[END_REF][START_REF] Recchioni | A path following method for box-constrained multiobjective optimization with applications to goal programming problems[END_REF][START_REF] Miglierina | Box-constrained multi-objective optimization, a gradient-like method without a priori scalarization[END_REF][START_REF] Attouch | A continuous gradient-like dynamical approach to Paretooptimization in Hilbert spaces[END_REF].

This Chapter 6 is devoted to the study of the trajectories solutions of the steepest descent dynamical system

(SD) u(t) + N C (u(t)) + ∂ C F (u(t)) 0 = 0,
where F : H -→ Y is a locally Lipschitz function with values in an ordered Banach space (Y, K). It can be written u(t) = s(u(t)), where s denotes the steepest descent vector field introduced in Definition 5.2.17. This dynamic is a gradient process, in the sense of Smale, for the vector optimization problem (VOP) MIN x∈C F (x).

Indeed, if u : [0, +∞[-→ H is a solution of (SD), its derivative u(t) is, by definition, a descent direction for F at u(t). Then, we expect that F (u(t)) decreases along time, with respect to the order in Y . Moreover, the stationary points of u(•) are critical points for (VOP), so we can also expect the trajectories solutions of (SD) to converge, when t → +∞, to a critical point of (VOP).

The Chapter is structured as follows. Section 6.1 is devoted to a first basic study of the steepest descent dynamic (SD). We start in Section 6.1.1 by defining properly what we mean by a solution of (SD). Indeed, the dynamic is governed by two nonsmooth operators, namely the normal cone to the constraint N C and the ordered subdifferential ∂ C F . While it is clear that N C (•) is discontinuous, it also appears that ∂ C F : H ⇒ H suffers from a lack of smoothness, even if F is regular (see Example 6.1.11). This is why we consider absolutely continuous solutions. In Section 6.1.2 we prove the announced decrease property of F along the trajectories, in Proposition 6.1.7. As it was already underlined in Chapter 5 and Section 2.3, this decrease property of F is essentially due to the simultaneous decrease of the family of cost functions

{f θ = θ • F : H -→ R | θ ∈ Θ ⊂ Y * }.
In sections 6.1.3 and 6.1.4 we illustrate the dynamic through some examples, and we compare (SD) to other dynamics, in particular the ones arising from scalarization methods. Section 6.1.5 consists in a discussion on the uniqueness of the trajectories. Indeed, since the steepest descent vector field is neither smooth or monotone, we cannot use classical results or techniques which ensure the uniqueness. We give in particular some geometrical sufficient conditions ensuring the uniqueness. The question of the uniqueness in general remains open.

Then comes Section 6.2, which is entirely devoted to the asymptotic behaviour of the solutions of (SD). Our main result is Theorem 6.2.6, which gives the convergence of the trajectories to solutions of (VOP) under some conditions. When F is convex, any bounded trajectory converges weakly to a weak Pareto point. If F is not convex but scalarly quasiconvex, we can still guarantee the convergence under additional hypotheses. The proof relies on the Féjer monotonicity of the trajectories, and Opial's Lemma. We also guarantee in Theorem 6.2.9 that the convergence is strong in the convex symmetric case.

Note finally that, in this Chapter, we assume the existence of solutions for (SD). This question will be treated in Chapter 7.

The steepest descent dynamic 6.1.1 Definitions

The dynamical system which is governed by u -→ s(u), will be called the steepest descent dynamical system, (SD) for short. Its solution trajectories t -→ u(t) verify (6.1)

(SD) u(t) + N C (u(t)) + ∂ C F (u(t)) 0 = 0.
Remark 6.1.1. The vector fields that appear in Theorem 5.2.20 generate the same integral curves, with a different time scale. We could have chosen any of them to generate our dynamic.

In [START_REF] Smale | Global analysis and economics I: Pareto optimum and a generalization of Morse theory[END_REF], it is assumed that the vector field ψ governing the gradient process is continuous, in a finite dimensional setting. In our context, the corresponding notions have been extended in order to cover dynamical systems governed by a discontinuous vector field on a general Hilbert space, to model a general preference relation on Y . In particular, instead of classical (continuously differentiable) solutions, we will consider strong solutions (absolutely continuous on bounded time intervals), the equality (6.1) being satisfied almost everywhere. We recall here the definition of absolutely continuous functions (see the monograph of Brezis [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]Appendix] for more details). Definition 6.1.2. Given T ∈ R + , a function u : [0, T ] -→ H is said to be absolutely continuous if one of the following equivalent properties holds: i) there exists an integrable function g : [0, T ] → H such that

u (t) = u (0) + t 0 g (s) ds ∀t ∈ [0, T ] ;
ii) u is continuous and its distributional derivative belongs to the Lebesgue space L 1 ([0, T ] ; H); iii) for every ǫ > 0, there exists η > 0 such that for any finite family of intervals

I k =]a k , b k [, I k ∩ I j = ∅ for k = j and k |b k -a k | ď η =⇒ k u (b k ) -u (a k ) ď ǫ.
We can now make precise the notion of solution for the steepest descent dynamic (SD).

Qualitative properties of the trajectories

We establish now the first qualitative properties of strong solutions of (SD). We take here for granted the existence of such strong global solutions. We will examine in details their existence in Chapter 7.

The main point of the following proposition is that the family of cost functions {f θ } θ∈Θ is equicontinuous along the trajectories. This will be of great interest when studying the existence of such trajectories in Chapter 7.

(6.3) u(t) 2 = η(t), -u(t) + v(t), -u(t) .
Since u(t) ∈ C, we know from Proposition 2.2.11 that

(6.4) η, u(t) = d dt δ C • u(t) = 0 for all η ∈ N C (u(t))
, so (6.3) becomes u(t) 2 = v(t),u(t) . Using the Cauchy-Schwarz inequality, we obtain (6.5) u(t) ď v(t) .

From the compactness of [0, T ] and the continuity of u, we know that u([0, T ]) is compact in H. Using this compactness and the local boundedness of ∂ C F (recall Proposition 5.1.7), we deduce that ∂ C F is uniformly bounded on u([0, T ]). From v(t) ∈ ∂ C F (u(t)) and (6.5), we deduce that u is uniformly bounded on [0, T ], so u is Lipschitz continuous therein. Item ii) follows exactly the same line, using the boundedness assumptions.

iii) For all θ ∈ Θ, and a.e. t ∈ [0, T ], we have

∂ C f θ (u(t)) ⊂ ∂ C F (x)
. Since the ordered subdifferential ∂ C F (x) has been proved to be uniformly bounded on u([0, T ]), there exists L ě 0 such that ∂ C f θ (u(t)) ⊂ LB H , for all θ ∈ Θ and a.e. t ∈ [0, T ]. Therefore, each cost function f θ is L-Lipschitz continuous on a neighbourhood of u([0, T ]). Added to the fact that u is Lipschitz continuous on the interval [0, T ], item iii.a) follows. To prove item iii.b), we want to use Ascoli's Theorem C.0.1. For this, consider for any t ∈ [0, T ] the set (6.6)

I t := {(f θ • u)(t) | θ ∈ Θ}.
Using the fact that the base Θ is bounded, and the definition f θ = θ • F , it is clear that I t is bounded in R, hence relatively compact. Then Ascoli's Theorem applies, and item iii.b) follows.

Now we show that the cost functions are Lyapunov for the (SD) dynamic: they decrease along the trajectories. Even more, we see that the dynamic is a descent method, relatively to the order endowing Y . Proposition 6.1.7 (Descent property). Let u : [0, +∞[-→ C be a strong global solution of (SD). Then, for each θ ∈ Θ, and for almost all t > 0 :

(6.7) d dt (f θ • u)(t) ď -u(t) 2 .
In particular, f θ is decreasing along the trajectories. It is also the case for F : (6.8)

t 1 ď t 2 ⇒ F (u(t 2 )) ĺ F (u(t 1 )).
Proof. Let θ ∈ Θ. From Proposition 6.1.5, we know that f θ • u is locally Lipschitz on R + , thus differentiable almost everywhere. So, for a.e. t > 0, f θ • u is differentiable and u(t) = s(u(t)) holds. For such t > 0, we have, using the chain rule for the Fréchet subdifferential (see Proposition 2.2.11):

(6.9) d dt (f θ • u)(t) = x * , u(t) , ∀x * ∈ ∂ F f θ (u(t)). Since ∂ F f θ (u(t)) ⊂ ∂ C f θ (u(t)) ⊂ ∂ C F (x), (6.9) implies (6.10) d dt (f θ • u)(t) ď d C F (u(t), u(t)).
Using u(t) = s(u(t)) and its descent property (see Proposition 5.2.18), we obtain the desired inequality (6.7). The monotonic property of F • u (6.8) is a consequence of the monotony of f θ •u, and the characterization of the order in Y by the elements of the base Θ (recall Proposition 2.3.7).

Remark 6.1.8. If we assume that, for a.e. t > 0, u(t) = 0, then we can easily improve the proof of (6.8) to obtain a strict monotonicity of F • u:

(6.11) t 1 < t 2 ⇒ F (u(t 2 )) ă F (u(t 1 )).
When we have a result of uniqueness for the trajectories, there is only two alternatives concerning the stationary points. Either the trajectory reaches a stationary point in finite time, and remains there, or the trajectory is never stationary. This question of uniqueness of trajectories is discussed in Section 6.1.5. In particular, Proposition 6.1.21 shows that, in the convex case, (6.11) holds until the trajectory reaches a weak Pareto point, in which case the trajectory stops there.

Some basic examples

We illustrate the (SD) dynamic through some examples. They suggest that its study is not a mere extension of some classic situation, because (SD) is governed by a vector field which is neither monotone, nor locally Lipschitz continuous. Without any further assumption, we cannot expect more than the Hölder continuity of this vector field (see Example 6.1.11 below, and Proposition 7.1.2 in Chapter 7).

For simplicity, we consider a strictly Gateaux differentiable bi-objective unconstrained case, i.e. (6.12)

F = (f 1 , f 2 ) : H -→ R 2 , with C = H and K = R 2 + .
The following elementary result provides an explicit description of the vector field in that case.

Proposition 6.1.9. Suppose we are in the setting of (6.12). Then, for any x ∈ H be such that ∇f 1 (x) = ∇f 2 (x), (6.13)

s(x) = -λ(x)∇f 1 (x) -(1 -λ(x))∇f 2 (x), λ(x) = proj [0,1] ∇f 2 (x)-∇f 1 (x),∇f 2 (x) ∇f 2 (x)-∇f 1 (x) 2 .
Proof. By definition of s(x), there exists λ(x) ∈ [0, 1] verifying

   s(x) = -λ(x)∇f 1 (x) -(1 -λ(x))∇f 2 (x), λ(x) = argmin λ∈[0,1] λ∇f 1 (x) + (1 -λ)∇f 2 (x) 2 .
Writing down the first-order optimality condition for λ(x), we obtain, when ∇f 1 (x) = ∇f 2 (x),

λ(x) + N [0,1] (λ(x)) ∋ ∇f 2 (x) -∇f 1 (x), ∇f 2 (x) ∇f 2 (x) -∇f 1 (x) 2 .
We conclude by using that the resolvent of the normal cone mapping is equal to the projection.

Example 6.1.10. Given a, b ∈ H, consider the quadratic functions Example 6.1.11. Given a ∈ H, consider the quadratic form f 1 (v) = 1 2 v 2 and the linear form f 2 (v) = a, v . The sets of Pareto and weak Pareto points both coincide with the half-line -R + a. The steepest descent vector field is deduced from Proposition 6.1.9:

f 1 (v) = 1 2 v -a 2 and f 2 (v) = 1 2 v -b 2
(6.14) s(x) =      -a if a, x ě a 2 , -x if x -a 2 2 ď a 2 2 , -a + a,a-x a-x 2 (a -x) else.
The domain of this vector field appears to be split in three parts, where it behaves differently.

Those three areas are a half-space supported by a, a ball centered in a 2 , and the rest of the space. The fact that this domains splits in three can also be seen in Example 6.1.10. It is simply due to the fact that s(x) is the projection of the origin onto the segment [∇f 1 (x), ∇f 2 (x)]. When doing this projection, three cases occurs. Either we project onto the extreme points of the segment, namely ∇f 1 (x) and ∇f 2 (x), or we project onto the interior of [∇f 1 (x), ∇f 2 (x)].

Let us consider the particular case H = R 2 , with a = (1, 0). Then f 1 (x, y) = 1 2 (x 2 + y 2 ) and f 2 (x, y) = x. The corresponding Pareto set is ] -∞, 0] × {0}, and

(6.15) s(x, y) =      -(1, 0) if x ě 1, -(x, y) if (x -1 2 ) 2 + y 2 ď 1 4 , - 1
(x-1) 2 +y 2 (y 2 , y(1x)) else. This example is very interesting, because it has the property that the steepest descent vector field s(x) = -(co{∇f 1 (x), ∇f 2 (x)}) 0 governing the dynamic may fail to be locally Lipschitz. This can be quite surprising at first sight, since the functions involved here are smooth. The lack of Lipschitz continuity occurs at the point (1, 0), which is the point where the vector field "splits" into three parts, see (6.15). Figure 6.5 provides a simple example of parametrized vectors u α , v α that both converge to (1, 0), when the angle parameter α goes to zero, but such that s(u α )s(v α ) = sin(α) and u αv α = sin(α) tan(α).

As a consequence, s(uα)-s(vα) uα-vα ≃ α α 2 is unbounded when α → 0, and this breaks the Lipschitz continuity in the neighbourhood of (1, 0). The point (1, 0) where fails the Lipschitz continuity of s is in fact the only one in the plane. To see this, go to Proposition 6.1.18, where we give a necessary condition for the failure of the Lipschitz continuity of s. Example 6.1.12. Take H = R 2 , and consider the two quadratic degenerate forms f 1 (x, y) = 1 2 x 2 and f 2 (x, y) = 1 2 y 2 . Here the only Pareto point is the origin, and the set of weak Pareto points is and R × {0} ∪ {0} × R. The multiobjective steepest descent vector field is, once computed: Observe in Figure 6.6 that the trajectories tend to move away from each other. This reflects the fact that (x, y) → -s(x, y) is not a monotone operator. Indeed, for x > 0, y > 0, x = y -s(x, y) + s(y, x), (x, y) -(y, x) = -2 xy(xy) 2

(0, 0) α (1, 0) α v α u α
s(x, y) = - xy 2 x 2 + y 2 , yx 2 x 2 + y 2 if (x, y) = (0, 0), s(0, 0) = (0, 0).
x 2 + y 2 < 0. Nevertheless, in this example, it can be shown that (x, y) → -s(x, y) is hypomonotone on (R * ) 2 . This means that, locally, there exists some α ě 0 such that -s + αI is monotone. To see this, compute the Jacobian of -s at (x, y)

D(-s)(x, y) = 1 (x 2 + y 2 ) 2 y 4 -x 2 y 2 2xy 3 2x 3 y x 4 -x 2 y 2 ,
and observe that D(-s)(x, y) + I is positive whenever (x, y) = (0, 0). It would be interesting to find some conditions on F which would guarantee that -s is hypomonotone, since in that case the study of the dynamic would fall into a well-understood class of dynamics. This example shows also that the operator s : H -→ H does not derive from a potential. Indeed, we can see above that its differential Ds(x, y) is not symmetric.

Related dynamics

In classic optimization problems, when F = f : H -→ R, the (SD) dynamic reduces to

u(t) + N C (u(t)) + ∂ C f (u(t)) 0 = 0.
If moreover the function f is convex, this system is equivalent to

u(t) + N C (u(t)) + ∂ C f (u(t)) ∋ 0,
because, in this case, the lazy solution property is automatically satisfied by the trajectories of the semigroup of contractions generated by the maximal monotone operator N C + ∂f , see [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]Theorem 3.1]. In particular, our existence and asymptotic analysis for (SD) in Sections 2 and 3 extends the well-known results for the nonsmooth gradient flow, see [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]. This leads us to a natural question, which is the study of the relationship (or differences) between (SD) and the Vector Differential Inclusion ((VDI) for short)

(6.16) (VDI) u(t) + N C (u(t)) + ∂ C F (x) ∋ 0.
For the sake of simplicity, we assume here that F is convex. It appears that (VDI) enjoys a weaker form of Proposition 6.1.7: we can only guarantee that one of the cost functions is decreasing at each instant.

Proposition 6.1.13. Let u : [0, +∞[-→ C be a strong global solution of (VDI) in the sense of Definition 6.1.3 (except the lazy property). Suppose that F is convex. Then, for almost all t ě 0, if u(t) is not critical, there exists some θ ∈ Θ (which depends on t) such that

d dt (f θ • u)(t) < 0.
Proof. Since u is a strong solution of (VDI), there exists η, p : [0, +∞[-→ H, which satisfy for almost all t ě 0

η(t) ∈ N C (u(t)), p(t) ∈ ∂ C F (u(t)), u(t) + η(t) + p(t) = 0.
Taking the scalar product of the above equation with u(t), we obtain (6.17)

u(t) 2 + η(t), u(t) + p(t), u(t) = 0.
By a similar argument than the one used for (6.4) in the proof of Proposition 6.1.5, we have

(6.18) 0 = d dt δ C (u(t)) = η(t), u(t) .
Hence, if we assume that u(t) is not critical, we have u(t) = 0, and it follows that p(t), u(t

) < 0. Since p(t) ∈ co * θ∈Θ ∂ C f θ (u(t))
, arguing by contradiction we easily see that there must exist some θ ∈ Θ (depending on t) and some p θ ∈ ∂ C f θ (u(t)) such that p θ , u(t) < 0. Using the chain rule of Proposition 2.2.11, together with the convexity of F , gives

(6.19) d dt f θ (u(t)) = p θ , u(t) < 0.
Remark 6.1.14. Proposition 6.1.13 tells us that, for any trajectory of (VDI), for almost all t > 0, at least one of the objective functions decreases. We will illustrate this on a few examples, and highlight the fact that, by contrast, for trajectories of (SD), they are all decreasing.

i) The scalarization approach consists in taking a constant merit function, say f θ . Clearly, ∂f θ (u) ⊂ ∂ C F (x), and the corresponding trajectories are solutions of (VDI). According to Bruck's theorem [START_REF] Bruck | Asymptotic convergence of nonlinear contraction semigroups in Hilbert spaces[END_REF], any orbit of the generalized gradient flow generated by ∂f θ converges to a minimizer of f θ , which, by Theorem 5.2.10, is a weak efficient point. This is the strategy underlying the weighting method, but it suffers from two drawbacks. First, it doesn't recover well the Pareto front in nonconvex cases (see Section 6.3). Secondly, there is no particular reason for this dynamic to improve all the cost functions. Take for instance in Example 6.1.10, θ = (λ, (1λ)) ∈ ∆ 2 for any λ ∈]0, 1]. When starting from (1, 0), the trajectory goes straight to (1 -2λ, 0) by decreasing f 1 but increasing f 2 .

ii) Consider the multiobjective setting F = (f 1 , ..., f m ), and consider the steepest descent dynamic associated to the function f = max i f i . This dynamic has some similarities with (SD), but it is different. As a supremum of a finite number of convex continuous functions, f is still convex continuous. The classical subdifferential rule for the supremum of convex functions (see for example [START_REF] Bauschke | Convex analysis and monotone operator theory[END_REF]Theorem 18.5]) gives in our setting (6.20) ∂f

(u) = co {∂f i (u) : i ∈ I(u)}
where

I(u) = {i ∈ I : f i (u) = f (u)} is the set of the active indices at u. Clearly, ∂f (u) ⊂ co ∂f i (u) = ∂ C F (u).
As a consequence, the trajectories of the steepest descent for f = max f i are also solutions of (VDI). But, in general, they fail to make decrease all the objective functions. Take for instance Example 6.1.10: when starting from some (x 0 , y 0 ) with x 0 > y 0 > 1, f 2 is first decreasing along the trajectory, until the current point reaches the projection of (1, 0) on the line segment joining (x 0 , y 0 ) to (-1, 0), then it is increasing.

In the next proposition, we show that, among all the possible dynamics satisfying (VDI), the unique one making all the merit functions decrease with the estimation of Proposition 6.1.7, is the lazy one, i.e., the steepest descent dynamic. Proposition 6.1.15. Let u : [0, T [-→ H be a strong solution of (VDI), and suppose that F is convex. Then the two following statements are equivalent : i) u is a solution of (SD), ii) u is decreasing for all cost functions, more exactly, for a.e. t ∈ [0, T ] and for all θ ∈ Θ,

d dt (f θ • u)(t) ď -u(t) 2 .
Proof. The implication i) ⇒ ii) has already been proved in Proposition 6.1.7. To prove the reverse implication, start by using the chain rule in Proposition 2.2.11, together with the convexity of F , to obtain for almost every t ∈ [0, T [:

∀θ ∈ Θ, ∀x * ∈ ∂ C f θ (u(t)), x * , u(t) ď -u(t) 2 .
By definition of ∂ C F (u(t)), we obtain immediately (6.21)

∀x * ∈ ∂ C F (u(t)), x * , u(t) ď -u(t)
∀x * ∈ N C (u(t)) + ∂ C F (u(t)), x * , u(t) ď -u(t) 2 ,
which is equivalent to say thatu(t) is the projection of the origin onto N C (u(t)) + ∂ C F (u(t)).

In other words,u(t) = (N C (u(t)) + ∂ C F (u(t))) 0 .

6.1.5 About the uniqueness of the trajectories Remark 6.1.16. In the unconstrained multicriteria case, and for convex differentiable objective functions, illustrative examples of the (SD) dynamic were given in Section 6.1.3. In these elementary situations, we have been able to explicitely compute the vector field x → s(x).

We observed that it can be Lipschitz continuous (Examples 6.1.10 and 6.1.12) or only Hölder continuous (Example 6. 1.11). This naturally raises the following question: in the unconstrained case, and for differentiable objective functions, what are the assumptions ensuring that the vector field x → s(x) is Lipschitz continuous? This is clearly a key property for the uniqueness of (SD), and we bring a first tribute to this question below.

The next result gives a sufficient condition for the local Lipschitz property of the vector field x → s(x), and by extension to the uniqueness question: Proposition 6.1.17. Suppose that H and Y are two Euclidean spaces, and C = H. Suppose that F is Gateaux differentiable with DF being Lipschitz continuous in a neighborhood of u ∈ H. If DF (u) is surjective, then the steepest descent vector field s

: v → -(∂ C F (v)) 0 is Lipschitz continuous in a neighborhood of u.
Proof. We have, for any v in the neighborhood of u, (6.24)

s(v) = -θ(v) • DF (v) where θ(v) := argmin θ∈Θ 1 2 θ • DF (v) 2 .
Since θ • DF (v) can also be written D * F (v)(θ), we obtain after writing down the first-order optimality condition in (6.24)

(6.25) 0 ∈ N B (θ(v)) + A(v)(θ(v)),
where A(v) := DF (v) • D * F (v) is a positive symmetric linear operator of Y . Note that A is Lipschitz continuous from H to L(Y ) in a neighborhood of u, by assumption. Since we assume that DF (u) is surjective, then A(u) is in particular definite positive. Hence, by continuity of eigenvalues on matrix entries, we can suppose that there exists some α > 0 such that A(v)x, x ě α x 2 for all v in a neighborhood of u, and all x ∈ H. The end of the proof is based on a standard argument for the study of stability for the solution of a strongly monotone variational inequality. Take two elements v 1 , v 2 , make the difference of the corresponding equations (6.25), and take the scalar product with θ(v 2 )θ(v 1 ). By monotonicity of the normal cone mapping θ → N B (θ), we obtain (6.26)

A(v 2 )θ(v 2 ) -A(v 1 )θ(v 1 ), θ(v 2 ) -θ(v 1 ) ď 0.
Let us rewrite (6.26) as (6.27)

A(v 2 )(θ(v 2 ) -θ(v 1 )), θ(v 2 ) -θ(v 1 ) ď (A(v 1 ) -A(v 2 ))θ(v 1 ), θ(v 2 ) -θ(v 1 ).
By the positive definite property of A(v), and Cauchy-Schwarz inequality, we obtain (6.28)

α θ(v 2 ) -θ(v 1 ) 2 ď (A(v 2 ) -A(v 1 ))θ(v 1 ) θ(v 2 ) -θ(v 1 ) .
After simplification, and by using the Lipschitz continuous dependence of A(v) with respect to v, it follows that v → θ(v) is locally Lipschitz continuous. Then combine this result with the local Lipschitz continuity of DF to conclude.

If we apply directly this result to the multiobjective case F = (f 1 , ..., f m ), we obtain the Lipschitz continuity of s around u ∈ H under the condition that the gradients ∇f i (u) are linearly independent. In fact, in this context, one can modify slightly the proof above to obtain the same result under a slightly weaker assumption, which is the affine independence of the gradients. We say that a family {x 1 , ..., x m } in H is affinely independent whether (6.29)

∀Λ = (λ 1 , ..., λ m ) ∈ R m , m i=1 λ i = 1 and m i=1 λ i x i = 0 implies Λ = 0.
Equivalently, this means that the family {x 1x m , ..., x m-1x m } is linearly independent.

Proposition 6.1.18. Suppose that H is an Euclidean space, with C = H, and that (Y, K) = (R m , R m + ). Suppose that for all i ∈ {1, ..., m}, f i : H -→ R is Gateaux differentiable with ∇f i being Lipschitz continuous in a neighborhood of u ∈ H. If {∇f i (u)} i∈{1,...,m} is affinely independent, then the steepest descent vector field s : v → -(∂ C F (v)) 0 is Lipschitz continuous on a neighborhood of u. Remark 6.1.19. This result is quite optimal, since in the case m = 2 it ensures the Lipschitz continuity of s at u under the condition that ∇f 1 (u) = ∇f 2 (u). Observe that in Example 6.1.11, this fails only at u = (-1, 0), where the Lipschitz continuity is missing.

Proof. This proof follows the same lines than the previous one, but we will deal with additional affine terms. Write, for any v in the neighborhood of u, (6.30)

s(v) = - m i=1 θ i (v)∇f i (v) where θ(v) := argmin θ∈∆m 1 2 m i=1 θ i (v)∇f i (v) 2 .
Equivalently, since θ m (v) = 1 -m-1 i=1 θ i (v), we can write (6.31)

s(v) = -∇f m (v) - m-1 i=1 θ i (v)(∇f i (v) -∇f m (v))
where (6.32) θ(v) := argmin

θ∈T m-1 1 2 ∇f m (v) + m-1 i=1 θ i (v)(∇f i (v) -∇f m (v)) 2 with the notation T m-1 := {θ ∈ R m-1 + | m-1 i=1 θ i ď 1}. Define G := (g i ) i∈{1,...,m-1} : H -→ R m-1
, where g i := f if m . Then, writing down the first-order optimality condition in (6.31) leads to (6.33) 0

∈ N T m-1 (θ(v)) + A(v)(θ(v)) + B(v),
where

A(v) := DG(v) • D * G(v) and B(v) := DG(v)(∇f N (v)
). Note that A(u) is the Gram matrix corresponding to the linearly independent family {∇g i (u)} i∈{1,...,m-1} = {∇f i (u) -∇f m (u)} i∈{1,...,m-1} , so A(u) is a definite positive symmetric linear operator on R m-1 . Since A(v) and B(v) are both Lipschitz continuous with respect to v around u, we can use the same arguments than in the previous proof to conclude. As already observed by Smale [START_REF] Smale | Global analysis and economics I: Pareto optimum and a generalization of Morse theory[END_REF], this property is generically satisfied at the critical points of F . Thus, if we try to find a counterexample to the uniqueness of trajectories, we should look for a function F : R n -→ R m with n ∼ m, or even n ≪ m.

Since the uniqueness of trajectories are not guaranteed, one could fear some wild behaviour, such that a trajectory reaching a solution/stationary point and then escaping from it. Fortunately, it happens that weak Pareto points are stable stationary points. Proposition 6.1.21. Let u be a strong global solution of (SD). If u(T ) is a weak efficient point for some T > 0, then u(t) = u(T ) for all t ě T .

Proof. Let t ∈ [T, +∞[ be fixed. Since u(T ) is a weak efficient point, there must exist a θ ∈ Θ such that (6.34) f θ (u(t)) ě f θ (u(T )).

But, we know from Proposition 6.1.7 that f θ is decreasing along the trajectory, so this, combined with (6.34), means that

(6.35) ∀s ∈ [T, t], f θ (u(s)) = f θ (u(T )).
As a consequence, we have d ds (f θ • u)(s) = 0 for a.e. s ∈ [T, t]. Using the energy estimation in Proposition REF, we deduce that u(s) = 0 for a.e. s ∈ [T, t]. Hence, u ≡ u(T ) on [T, t], this being true for all t ě T .

Here, we just tackled the problem of uniqueness from the point of view of the regularity of s. Another interesting aspect concerns the monotonicity-like property of s. Indeed, in the Examples 6.1.11 and 6.1.12 of Section 6.1.3, there exists locally some positive constant α such that x → -s(x) + αx is monotone, a property which classically implies the uniqueness (see [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]Theorem 3.17]). Thus, it is an open question, that we do not address here, to know whether this property is satisfied by the (SD) system, at least under some general asumptions.

Asymptotic properties of the steepest descent dynamic

We start with a very simple result concerning the convergence of the values. Proposition 6.2.1. Let u : [0, +∞[-→ C be a strong global solution of (SD). Then :

i) For all θ ∈ Θ, f θ (u(t)) ↓ inf tě0 f θ (u(t)) when t goes to +∞.
ii) The sublevel sets ([F ĺ F (u(t))]) tě0 converge, in the Painlevé -Kuratowski sense, to tě0 [F ĺ F (u(t))], when t goes to +∞.

Proof. Item i) is a direct consequence of the monotonicity of t → f θ (u(t)), see Proposition 6.1.7. Now let us prove the convergence of [F ĺ F (u(t))] in the Painlevé -Kuratowski sense, see [21, Remark 12.1.2] for a definition. For short, we will note Γ F (u(t)) := [F ĺ F (u(t))]. We will prove successively the following inclusions:

(6.36) Limsup t→+∞ Γ F (u(t)) ⊂ Liminf t→+∞ Γ F (u(t)) ⊂ tě0 Γ F (u(t)) ⊂ Limsup t→+∞ Γ F (u(t)).
Start by taking x ∈ Limsup t→+∞ Γ F (u(t)), then there exists some

x n → x, t n → +∞, such that x n ∈ Γ F (u(t n ))
. Given an arbitrary sequence tk → +∞, let us show that there exits a corresponding sequence xk converging to x such that xk ∈ Γ F (u( tk )). For any k ∈ N, there exists some n k ∈ N such that tk ď t n k , hence by Proposition 6.1.7 we deduce that

x n k ∈ Γ F (u(t n k )) ⊂ Γ F (u( tk )). So it suffices to take xk = x n k to deduce that x ∈ Liminf t→+∞ Γ F (u(t)).
Consider now x ∈ Liminf t→+∞ Γ F (u(t)), then there exists a sequence x n → x such that x n ∈ Γ F (u(n)) for all n ∈ N. Given some t ě 0, and using again Proposition 6.1.7, we see that

x n ∈ Γ F (u(n)) ⊂ Γ F (u(t)
) for all n ě t. Hence, by passing to the limit when n → +∞, along with the fact that Γ F (u(t)) is a closed set (due to the lower semi-continuity of the functions), we deduce that x ∈ Γ F (u(t)). This being true for an arbitrary t ě 0, we conclude that x ∈ tě0 Γ F (u(t)).

Take now x ∈ tě0 Γ F (u(t)). By defining t n := n and x n := x, we trivially have that

x n ∈ Γ F (u(t n )) for all n ∈ N, so x ∈ Limsup t→+∞ Γ F (u(t)) follows.
Remark 6.2.2. Note that this property of convergence for the sublevel sets holds more generally for any decreasing sequence of sets, in the sense of the inclusion.

We follow now with a general asymptotic property for the iterates which guarantees that the strong limit points of the trajectory are critical Pareto points. It is a direct consequence of the dissipative property of the dynamic, and the weak-strong outer semi-continuity of the operator x ⇒ N C (x) + ∂ C F (x), studied in Section 5.1 (see Propositions 5.1.11 and 5.1.13). Recall that we introduced in Section 5.1 the following set of hypotheses (here H is a Hilbert space, thus reflexive):

One of the three following properties is satisfied:

H 1) K is polyhedral,
2) Y has finite dimension, and F is positively Clarke regular, 3) F is the sum of a convex function and a strictly Gateaux differentiable function.

Proposition 6.2.3. Suppose that one of the cost functions f θ remains bounded from below along the trajectory.

i) The trajectory has a finite energy:

(6.37) +∞ 0 u(t) 2 dt < +∞.
ii) If H holds, then any strong limit point of the trajectory is Pareto critical.

iii) If the functions are convex, any weak limit point of the trajectory is a weak Pareto.

Proof. Let f θ be the cost function such that inf tě0 f θ (x(t)) > -∞. From Proposition 6.1.7 and by integrating (6.7), we obtain (6.38)

+∞ 0 u(t) 2 dt ď f θ (x(0)) -inf tě0 f θ (x(t)).
This proves item i).

Let u ∞ be a strong limit point of the trajectory, i.e. u(t n ) → u ∞ strongly in H for some t n → +∞. The finite energy property (6.38) implies that (6.39) liminfess t→+∞ u(t) = 0.

Since relations (6.38) and (6.39) are satisfied for almost all t > 0, we can suppose, for some t n → +∞, that:

u(t n ) → x ∞ strongly in H, (6.40) u(t n ) → 0 strongly in H, -u(t n ) ∈ N C (u(t n )) + ∂ C F (u(t n )) for each n ∈ N.
From Corollary 5.1.14 we know that u ⇒ N C (u)+∂ C F (u) is sequentially upper semi-continuous, so point ii) follows. Point iii) is proved by the same argument, using the weak-strong closure instead of the strong-weak closure (cf. Proposition 5.1.11), and Fermat's rule (Theorem 5.2.10) for convex fonctions.

We are now going to prove a global convergence result for the iterates, under the assumption that the merit functions are quasi-convex. (Our argument is in the line of the proof of convergence of the steepest descent by Goudou and Munier [START_REF] Goudou | The gradient and heavy ball with friction dynamical systems: the quasiconvex case[END_REF] in the case of a single (quasi-convex) objective function.) Recall from Chapter 2, Section 2.3, that F is said to be scalarly quasiconvex whenever f θ is quasiconvex, for all θ ∈ Θ. It implies in particular that F is quasiconvex. In the following, we will make use of the following limit sets: (6.41) the set of weak limit points

Ω[u(t)] := {u ∈ H | ∃t n → +∞ s.t. u(t n ) w → u}, (6.42) the common sublevel limit set Γ F [u(t)] := tě0 [F ĺ F (u(t))]. Observe that Γ F [u(t)] is nothing but {v ∈ H | f θ (v) ď f θ (u(t)
) ∀t ě 0, ∀θ ∈ Θ}, and is closed convex in the quasiconvex setting.

Proposition 6.2.4. Let F be scalarly quasiconvex, then Ω[x(t)] ⊂ C ∩ Γ F [x(t)].
Proof. Suppose that u(t n ) w → u ∞ for some t n → +∞. Since the trajectory remains in C which is closed convex, then u ∞ ∈ C. Moreover, recall that the quasiconvex functions are weakly lower semi-continuous, so for all θ ∈ Θ :

f θ (u ∞ ) ď lim inf n→+∞ f θ (u(t n )) = inf tě0 f θ (u(t)). Whence u ∞ ∈ Γ F [u(t)].

Proposition 6.2.5. (Fejer property)

Let F be scalarly quasiconvex, then for all z ∈ C ∩ Γ F [u(t)], t → u(t)z is decreasing.

Proof. Let z ∈ C ∩ Γ F [u(t)],
and define h(t) := 1 2 u(t)z 2 . Since the trajectory u(•) is absolutely continuous, so is h, and we can derive it for almost every t > 0:

h ′ (t) = u(t), u(t) -z = -s(u(t)), z -u(t) = η(u(t)), z -u(t) + p(u(t)), z -u(t) , where η(u(t)) ∈ N C (u(t)), p(u(t)) ∈ ∂ C F (u(t)
). The fact that z ∈ C, implies immediately from the definition of N C that η(u(t)), zu(t) ď 0. Using the definition of the Clarke directional derivative, one obtains:

h ′ (t) ď d C F (u(t), z -u(t)). Since z ∈ Γ F [u(t)], we know that f θ (z) ď f θ (u(t)
) for all θ ∈ Θ and all t ě 0. Let t > 0 be fixed, and suppose on one hand that there is some θ ∈ Θ such that f θ (u(t)) = f θ (z). Then the trajectory stops there. Indeed, by decrease property (Proposition 6.1.7) one has f θ (u(s)) = f θ (u(t)) for all s ě t. Using (6.7) we see that u(s) = 0 for a.e. s ě t. Since u(•) is absolutely continuous, we deduce that u(s) = u(t) for all s ě t. In that case h is constant on [t; +∞[. On the other hand, if f θ (z) < f θ (u(t)) for all θ ∈ Θ, then the quasiconvexity and Proposition 2.2.16 tell us that d C f θ (u(t), zu(t)) ď 0. In other words, d C F (u(t), zu(t)) ď 0 and so h ′ (t) ď 0. Both cases lead to the fact that h is decreasing along time.

We can now state our main convergence result. Theorem 6.2.6. Let F be scalarly quasiconvex. Then the trajectory is bounded if and only if

C ∩ Γ F [u(t)] = ∅. In that case, i) the trajectory weakly converges to a point in C ∩ Γ F [u(t)],
ii) the trajectory has finite energy :

+∞ 0 u(t) 2 dt < +∞.
Under additional hypotheses, we have the stronger results :

(iii) if F is convex, then the weak limit point is a weak efficient point.

(iv) if the trajectory is precompact and H holds, then the trajectory converges strongly to a critical point.

The theorem is a direct corollary of Propositions 6.2.

(ii) Ω[x(t)] ⊂ S.
Then x(t) weakly converges to some x ∞ ∈ S, when t → +∞.

Proof of Theorem 6.2.6. If the trajectory is bounded, then it admits a weakly convergent subsequence. In other words, Ω[u(t)] = ∅, and with Proposition 6.2.4 it follows that

C ∩ Γ F [u(t)] = ∅. If C ∩ Γ F [u (t)
] = ∅, then using Opial's Lemma together with Propositions 6.2.4 and 6.2.5, gives the weak convergence of the trajectory. In that case one sees that the trajectory is in particular bounded. Items ii-iv) follow Proposition 6.2.3, using the fact that if the trajectory converges to u ∞ ∈ Γ F [u(t)], then each cost function f θ is minimized along the trajectory by f θ (u ∞ ).

Remark 6.2.8. a) Since each function t → f θ (u(t)) is nonincreasing (see Proposition 6.1.7), a natural condition insuring that the trajectory remains bounded, is that one of the functions has bounded sublevel sets. b) Similarly, if one of the merit functions has relatively compact sublevel sets (inf-compactness property), then the trajectory is relatively compact, which is needed for the strong convergence in H.

As in the convex monocriteria case, strong convergence can be obtained by doing a symmetry hypothesis on the involved functions. We say that F is even if F (x) = F (-x) for all x ∈ H., and that C is symmetric whether C = -C. Theorem 6.2.9. Suppose that F is convex and even, and that C is symmetric. Hence the trajectory strongly converges to a weak efficient point. Proof. Let s ě 0 and define, for t ∈ [0, s] :

γ(t) := u(t) 2 -u(s) 2 - 1 2 u(t) -u(s) 2 .
We will show that γ is decreasing and then derive a Cauchy property for the trajectory. Since u, γ is differentiable almost everywhere on [0, s], then for a.e t ∈ [0, s] :

γ(t) = 2 u(t), u(t) -u(t), u(t) -u(s) = -u(t), -u(s) -u(t) (6.43) = η(u(t)), -u(s) -u(t) + p(u(t)), -u(s) -u(t) ď η(u(t)), -u(s) -u(t) + d C F (u(t), -u(s) -u(t)) where η(u(t)) ∈ N C (u(t)) and p(u(t)) ∈ ∂ C F (u(t)).
Using the symmetry of C, we see that u(s) ∈ C ⇒ -u(s) ∈ C, so combined with η(u(t)) ∈ N C (u(t)) we obtain (6.44) η(u(t)), -u(s)u(t) ď 0. Now, thanks to the decrease property (Proposition 6.1.7) and the symmetry, we have

(6.45) for all θ ∈ Θ, f θ (-u(s)) = f θ (u(s)) ď f θ (u(t)).
Since the merit functions are supposed to be convex, we can use Proposition 2.2.15 to deduce from (6.45) that d C F (u(t), -u(s)u(t)) ď 0. By combining (6.43) with (6.44), we obtain γ(t) ď 0 for a.e.

t ∈ [0, s], so γ(•) is decreasing on [0, s].
The decrease property of γ indicates in particular that γ(t) ě γ(s) = 0 for all t ∈ [0, s]. In other words : (6.46) for all 0 ď t ď s,

1 2 u(t) -u(s) 2 ď u(t) 2 -u(s) 2 .
A first observation on (6.46) is that 0 ď u(s) 2 ď u(t) 2 . Being decreasing and positive, we see that u(•) converges in R, and so it is Cauchy. As a consequence, by going back on (6.46), we see that the trajectory u(•) is Cauchy, whence strongly convergent in H. We know that the limit point is weakly efficient using Theorem 6.2.6.

6. [START_REF] Adly | A nonsmooth algorithm for cone-constrained eigenvalue problems[END_REF] The discrete steepest descent method, and numerical results 6.3.1 Review of the existing algorithms, and links with (SD)

We discuss the discretization(s) in time of the continuous dynamic

(SD) u(t) + N C (u(t)) + ∂ C F (u(t)) 0 = 0,
and relate it to the existing literature.

We start by assuming that F is of class C 1 , and that there is no constraint (C = H). In that case, a naive discretization of (SD) gives (6.47)

x n+1 = x n + λ n s(x n ),
where λ n is a real stepsize, and s(x n ) is the element of minimal norm of ∂ C F (x n ). It can be useful to recall that the steepest descent direction has also the equivalent form (see Theorem 5.2.20 and Proposition 5.1.8):

(6.48) s(x n ) = argmin d∈X 1 2 d 2 + σ θ (DF (x; d)).
Since the steepest descent direction enjoys an Armijo rule (see Propositions 5.2.18 and 5.2.9), it is natural to chose λ n as:

λ n := max 1 2 k | k ∈ N, F (x n + 1 2 k d) ĺ F (x n ) + β 1 2 k d C F (x n ; d) .
As usual, if DF is L-Lipschitz continuous at x n , then it is sufficient to take λ n < 1-β L .

This algorithm (6.47) has been first introduced by Fliege and Svaiter [START_REF] Fliege | Steepest descent methods for multicriteria optimization[END_REF], for the multiobjective case, and extended by Graña and Svaiter [START_REF] Drummond | A steepest descent method for vector optimization[END_REF] to the general vector case (see also [START_REF] Cruz Neto | A subgradient method for multiobjective optimization[END_REF][START_REF] Bento | The self regulation problem as an inexact steepest descent method for multicriteria optimization[END_REF]). According to these works, this discrete dynamic shares exactly the same behaviour with our (SD) dynamic. That is, like in Theorem 6.2.6, this algorithm weakly converges in the quasiconvex case, and the limit point is a weak efficient point in the convex case. Still like in Theorem 6.2.6, the convergence of the iterates (x n ) n∈N is conditioned by the boundedness

hypothesis {x ∈ H | F (x) ĺ F (x n ), ∀n ∈ N} = ∅.
If we consider the presence of a convex constraint C, the discretization of (SD) becomes more delicate. Of course, we could directly write

x n+1 = x n + λ n s(x n ) = x n + λ n N C (x n ) + ∂ C F (x n ) 0 ,
but this would face a serious drawback. Indeed, we know in the constrained case that the steepest descent direction s(x n ) is an admissible descent direction, i.e. s(x n ) ∈ T C (x n ). But while we perform an Armijo step, what we really need is s(

x n ) ∈ T ad C (x n ), that is, s(x n ) ∈ C -x n µ , ∀µ ∼ 0.
The problem is that T C (x n ) can be strictly bigger than T ad C (x n ), and that following such a direction could make us getting out from the constraint. Of course, there is some cases for which the tangent cone of admissible descent directions is closed (e.g. the locally polyhedral sets, see [START_REF] Meng | On local coincidence of a convex set and its tangent cone[END_REF]), but an other method must be considered in the general case.

This problem already appears in the scalar case (f : H -→ R), for which we generally perform a gradient-projection step:

x n+1 = proj C (x n -λ n ∇f (x n )).
Thus, a natural approach would be, in the vector case, to perform first a steepest descent step to decrease F , and then project onto the constraint C:

(6.49) x n+1 = proj C (x n + λ n d n ), where d n = (∂ C F (x n )) 0 .
But, unfortunately, this approach does not work anymore in the vector case, because we loose the descent property of F . The least we can say is that, at each step, one of the cost functions decreases (do the parallel with Section 6.1.4). But this is not true in general for all the cost functions, and hence for F (see a simple counter-example below).

Proposition 6.3.1. Let (x n ) n∈N be a sequence generated by the algorithm (6.49). Suppose that F is continuously differentiable, with a Lipschitz continuous derivative. Let L := sup θ∈Θ Lip(∇f θ ; x n ), and suppose that λ n ∈]0, 1 L [. Then, for all n ∈ N, there exists θ n ∈ Θ such that (6.50)

1 2λ n - L 2 x n+1 -x n 2 ď f θn (x n ) -f θn (x n+1 ).
Proof. By definition of the projection, and since x n ∈ C, we have

(6.51) 1 2 x n+1 -x n -λ n d n 2 ď 1 2 λ n d n 2 ,
which can be rewritten as

(6.52) 1 2t n x n+1 -x n -λ n d n 2 ď x n+1 -x n , d n .
Because of Proposition 5.1.8, d n writes as θ n • DF (x n ) = ∇f θn (x n ) for some θ n ∈ Θ. The Lipschitz continuity of ∇f θn gives (6.53)

x n+1 -x n , -∇f θn (x n ) ď f θn (x n ) -f θn (x n+1 ) + L 2 x n+1 -x n 2 .
We obtain from (6.52) and (6.53) that (6.54)

1 2λ n - L 2 x n+1 -x n 2 ď f θn (x n ) -f θn (x n+1 ).
The following example shows that we cannot expect an individual decrease property for all the functions.

Example 6.3.2. We build our counter-example in the bi-objective case. Consider two quadratic functions, distances to two points:

f 1 (x) = 1 2
xa 2 and 1 2 xb 2 for two distinct points a = b. They are smooth functions, with 1-Lipschitz gradient. For now we do not precise what is C. Suppose that at our current iterate, we have a point

x n such that x n -a = x n -b . At this point, s(x n ) is exactly a+b 2 -x n .
Then, whatever can be C, the algorithm reads, for some

λ n ∈]0, 1[, x n+1 = proj C (1 -λ n )x n + λ n a + b 2 .
Introduce now the constraint C: consider at x n the level set of the function f 1 , which is exactly the ball B(a, x na ). Consider the hyperplane tangent to B(a, x na ) at x n , at take C as the closed half space delimited by this hyperplane, which does not contain B(a, x na ):

(6.55) C = {x ∈ H | x -x n , a -x n ď 0}.
One can easily see that for any choice of λ n ∈]0, 1[, the point x n+1 will fall outside B(a, x n -a ). Figure 6.8: Failure of the steepest descent-projection method for bi-objective optimization problems.

The problem here is that, from the point of view of f 1 , we are performing an inexact projected-gradient method, where the gradient of f 1 is replaced by a a more general descent direction, and this cannot work.

An other approach to handle the constraint is to use an approximation of s(x n ) based on its variational form s(x n ) = argmin

d∈T C (xn) 1 2 d 2 + σ θ (DF (x; d)).
Approximating T C (x n ) by C-xn µn for some µ n ∈, we define

d n = argmin d∈ C-xn µn 1 2 d 2 + σ θ (DF (x; d)).
It is not a difficult exercise to verify that such d n is an admissible Armijo direction, so it makes sense to use it to design an algorithm. This approach has been developed by Graña and Iusem [START_REF] Drummond | A projected gradient method for vector optimization problems[END_REF], and followed by Fukuda and Graña [START_REF] Fukuda | On the convergence of the projected gradient method for vector optimization[END_REF][START_REF] Fukuda | Inexact projected gradient method for vector optimization[END_REF], Bello, Lucambio and Melo [START_REF] Bello Cruz | Convergence of the projected gradient method for quasiconvex multiobjective optimization[END_REF]. As for the unconstrained case, this algorithm shares the same asymptotic behaviour than (SD).

In the last fifteen years, there has been other algorithms proposed to manage the constraint. Fliege and Svaiter [START_REF] Fliege | Steepest descent methods for multicriteria optimization[END_REF] propose a method to deal with inequality constraints. Recchioni [START_REF] Recchioni | A path following method for box-constrained multiobjective optimization with applications to goal programming problems[END_REF], together with Miglierina, Molho and Recchioni [START_REF] Miglierina | Box-constrained multi-objective optimization, a gradient-like method without a priori scalarization[END_REF], consider a box constraint that they penalize with a barrier method. Villacorta, Oliveira and Soubeyran [START_REF] Villacorta | A trust-region method for unconstrained multiobjective problems with applications in satisficing processes[END_REF] study also an interior point method, by adapting the trust-region methods to the multi-objective case. Bento, Ferreira, Oliveira [START_REF] Bento | Unconstrained steepest descent method for multicriteria optimization on riemannian manifolds[END_REF] and then Bento and Cruz [START_REF] Bento | A Subgradient Method for Multiobjective Optimization on Riemannian Manifolds[END_REF] adopted an approach based on Riemannian manifolds.

Suppose now that F is convex but nonsmooth, without constraint. A common way to discretize (SD) is to do it implicitly with respect to ∂ C F :

x n+1 + λ n ∂ C F (x n+1 ) 0 = x n .
But this nonlinear implicit equation in x n+1 seems to be too difficult to be solved. We can relax this equation by taking general elements in ∂ C F (x n+1 ), instead of taking specifically its element of minimal norm (recall (VDI) in Section 6.1.4): (6.56)

x n+1 + λ n ∂ C F (x n+1 ) ∋ x n .
Introducing a formal resolvent for ∂ C F , we see that the above equation can be equivalently rewritten as

x n+1 ∈ PROX λnF (x n ), where PROX λF := (Id + λ∂ C F ) -1 .
Since F is assumed to be convex, we have (Proposition 5.1.8) that

∂ C F (x) = θ∈Θ ∂ C f θ (x), so we can deduce that PROX λnF (x n ) = θ∈Θ prox λnf θ (x n ),
where (using that θ, e = 1)

prox λnf θ (x n ) = argmin x∈H θ • F (x) + 1 2λ n x -x n 2 = argmin x∈H θ, F (x) + 1 2λ n x -x n 2 e .
Using the classical representation of weak efficient points for convex functions (Theorem 5.2.10), we see that the algorithm in (6.56) is equivalent to (6.57)

x n+1 ∈ ARGMIN w x∈H F (x) + 1 2λ n x -x n 2 e.
Since the algorithm in (6.57) is based on the vector differential inclusion

(VDI) u(t) + ∂ C F (u(t)) ∋ 0,
instead of the steepest descent one, there is no guarantee for F to decrease at each iteration.

In their seminal paper, Bonnel, Iusem and Svaiter [START_REF] Bonnel | Proximal methods in vector optimization[END_REF] propose a modified version of (6.57), enforcing the decrease property by adding a sublevel constraint:

x n+1 ∈ ARGMIN w x∈[F ĺF (xn)] F (x) + 1 2λ n x -x n 2 e.
This proximal method for vector optimization problems has also been studied by Ceng, Yao [START_REF] Ceng | Approximate proximal methods in vector optimization[END_REF] and Ceng, Mordukhovich, Yao [START_REF] Ceng | Hybrid approximate proximal method with auxiliary variational inequality for vector optimization[END_REF]. Villacorta and Oliveira [START_REF] Villacorta | An interior proximal method in vector optimization[END_REF] consider a modified version of this algorithm, involving Bregman distances to handle a convex constraint C. Bento, Cruz and Soubeyran [START_REF] Bento | A proximal point-type method for multicriteria optimization[END_REF] propose a particular instance of this proximal algorithm, defined by

x n+1 ∈ argmin x∈H σ Θ F (x) + 1 2λ n x -x n 2 e + δ [F ĺF (xn)] (x)e .
But a quick computation, involving Proposition A.3.6, shows that it is in fact just a proximal method associated to the scalar-valued function

σ Θ • F , with a moving constraint [F ĺ F (•)].
None of the aforementioned proximal methods can be seen as a direct discretization of the (SD) dynamic, because of the constraint [F ĺ F (•)]. It is, to our opinion, more related to the sweeping process u

(t) + N [F ĺF (u(t))] (u(t)) + ∂ C F (u(t)) ∋ 0.

The effective computation of the steepest descent direction

Given a point x ∈ H, Theorem 5.2.20 provides a formula for the computation of the steepest descent at x:

min d∈H 1 2 d 2 + d C F (x; d).
Following [START_REF] Fliege | Steepest descent methods for multicriteria optimization[END_REF], this problem is equivalent to the following quadratic problem with convex constraint: min

(d,α)∈H×R 1 2 d 2 + α such that d C F (x; d) ď α.
If F is strictly Gateaux differentiable, it is also equivalent to the quadratic program with cone constraints min

(d,α)∈H×R 1 2 d 2 + α such that DF (x; d) ĺ αe.
For instance, if (Y, K) = (S m , S m + ), the space of symmetric matrices ordered by the cone of positive symmetric matrices, it reduces to a SDP program: min

(d,α)∈H×R 1 2 d 2 + α such that DF (x; d) ĺ αId.
In the multi-objective case, it reduces to a quadratic program with linear constraints min

(d,α)∈H×R 1 2 d 2 + α such that ∇f i (x), d ď α for all i ∈ {1, ..., m}.
Theorem 5.2.20 provides also a dual version of this problem, involving the linear forms θ ∈ Θ:

max y * ∈Y * -D * F (x; y * ) 2 such that y * ∈ Θ, which is equivalent to min θ∈Θ θ • DF (x) 2 .
In its general form, it is a quadratic program, with linear and cone constraints since

θ ∈ Θ ⇔ θ ∈ K + and θ, e = 1.
In the multi-objective case F = (f 1 , ..., f m ), this dual problem reduces to the projection onto the polyhedron co{∇f 1 (x), ..., ∇f m (x)}. It is particularly easy to implement when m is low, for instance m = 2 or 3.

In the nonsmooth case, the computation of s(x) can be challenging. We propose a naive approach to compute it, in the case of a bi-criteria optimization problem F = (f, g) : R n -→ (R 2 , R 2 + ). We assume that f is a smooth function, but that g is nonnecessarily differentiable. In that setting, we need to be able to project the origin onto co{∇f (x), ∂ C g(x)}. We aim to show that this problem can be solved quickly, provided that the projection onto ∂ C g(x) is easy. For instance, the subdifferential of the ℓ 1 norm falls into this setting.

The problem we need to solve here is

min λ∈R inf x * ∈∂ C g(x) (1 -λ)∇f (x) + λx * , such that λ ∈ [0, 1].
This is equivalent to solve min

λ∈[0,1] proj C λ (0) ,
where

C λ := (1 -λ)∇f (x) + λ∂ C g(x)
. In other words, we need to minimize real-valued function

φ : [0, 1] -→ R λ -→ proj (1-λ)∇f (x)+λ∂ C g(x) (0) .
The main trick is that this function is convex: Lemma 6.3.3. The function φ defined above is convex.

Proof. Let λ 1 , λ 2 ∈ [0, 1] and α ∈ [0, 1]. Define λ := αλ 1 + (1 -α)λ 2 , and show that φ(λ) ď αφ(λ 1 ) + (1 -α)φ(λ 2 ).
Using basic algebra on sets, we can write (6.58)

αC λ 1 + (1 -α)C λ 2 = C λ . Note p 1 := proj C λ 1 (0), p 2 := proj C λ 2 (0)
, and p := αp 1 + (1α)p 2 . From (6.58), we have p ∈ C λ . Thus, using the triangle inequality:

φ(λ) ď p ď α p 1 + (1 -α) p 2 = αφ(λ 1 ) + (1 -α)φ(λ 2 ).
So, we can apply a golden section method to φ. It finds (one of) its minimum λ by dichotomy. This method only asks to compute the values of φ, which is computationally equivalent to compute a projection on

∂ C g(x), since proj (1-λ)∇f (x)+λ∂ C g(x) (0) = proj λ∂ C g(x) (-(1 -λ)∇f (x)) + (1 -λ)∇f (x).
Once λ is found, it suffices to take s(x) = proj (1-λ)∇f (x)+ λ∂ C g(x) (0).

Generation of Pareto fronts

Example 6.3.4. Let F = (f 1 , f 2 ) : R 2 -→ R 2 be defined for all x = (x 1 , x 2 ) by f 1 (x) = x 1 and f 2 (x) = 1 x 1 + x 2 2 + 3e -100(x 1 -0.3) 2 + 3e -100(x 1 -0.6) 2 ,
where • 1 and • 2 denote respectively the usual ℓ 1 and ℓ 2 -norms on R 2 . This example is taken from [START_REF] Mohammadi | Improving the identification of general Pareto fronts by global optimization[END_REF], see also [START_REF] Miglierina | Box-constrained multi-objective optimization, a gradient-like method without a priori scalarization[END_REF]Test 1]. We take a constraint C = [0.1, 1] 2 , on which F is well-defined and locally Lipschitz continuous.

Our purpose is to generate the Pareto front, that is, the minimal elements of F (C) in (R 2 , ĺ).

Next figure shows an approximation of F (C), and the Pareto front we try to recover. Due to the oscillations of f 2 around the values x 1 = 0.3, 0.6, the image F (C) is not convex, and we see that the Pareto front is not connected. To compute the Pareto front, we compare two methods: the weighting method, and the steepest descent method. Let us give in detail our protocol. The weighting method consists in the choice of some θ ∈ ∆ 2 = {(θ 1 , 1-θ 1 ) | θ 1 ∈ [0, 1]}, and the minimization of the corresponding convex combination f θ = θ 1 f 1 + (1θ 1 )f 2 . Its minimization will be performed by a gradient projected algorithm

x k+1 = proj C (x k -λ∇f θ (x k ))
, where λ = 0.05.

The stopping criterion is

x k+1 -x k λ < 0.05. In Figure 6.10, we show some samples of the weighting method. The starting point x 0 and the combination θ ∈ ∆ 2 are either chosen randomly, or chosen on a uniformly distributed grid. We plot the value in Y = R 2 of the last iterate of each trajectory. We observe that the points generated by the weighting method tends to be attracted in the zones where f 2 has low values. We now pass to the steepest descent method. We use a basic discretized version of (SD), namely

x k+1 = x k + λs(x k )
, where λ = 0.05, and the stopping criterion is again

x k+1 -x k λ
< 0.05. The results concerning the steepest descent method are given in Figure 6.11. We observe that the steepest descent approach tends to cover the Pareto front more uniformly.

Comments and perspectives

Remark 6.4.1 (On the selection of an efficient point among others). Because the set of (weak) efficient points is rarely reduced to one point, we can aim to select one among them satisfying a desired property. For instance, we could look for the efficient point being the nearest from a desired state u d . This problem of optimizing over the set of efficient points is sometimes called post-Pareto analysis, see [START_REF] Philip | Algorithms for the vector maximization problem[END_REF][START_REF] Benson | Optimization over the efficient set[END_REF][START_REF] Bolintineanu | Optimality conditions for minimization over the (weakly or properly) efficient set[END_REF][START_REF] Bonnel | Optimization over the efficient set of multi-objective convex optimal control problems[END_REF]. By analogy with the scalar optimization, we propose an approach based on a Tikhonov-like regularization. That is, we could add to the (SD) dynamic a vanishing term ε(t)(u(t)u d ), where ε(t) --→ t→+∞ 0:

(6.59) u(t) + ∂ C F (u(t)) 0 + ε(t)(u(t) -u d ) = 0.
More generally, if we want our solution to minimize some smooth potential g :

H -→ R, we could consider u(t) + ∂ C F (u(t)) 0 + ε(t)∇g(u(t)) = 0, which covers (6.59) by taking g = 1 2 • -u d 2 .
This hierarchical dynamic is well-known in scalar optimization and is based on Tikhonov regularization ideas (see [START_REF] Attouch | A dynamical approach to convex minimization coupling approximation with the steepest descent method[END_REF][START_REF] Cabot | The steepest descent dynamical system with control. Applications to constrained minimization[END_REF][START_REF] Czarnecki | Asymptotic control and stabilization of nonlinear oscillators with non isolated equilibria, a note: from L 1 to non L 1[END_REF][START_REF] Cominetti | Strong asymptotic convergence of evolution equations governed by maximal monotone operators with Tikhonov regularization[END_REF][START_REF] Attouch | Asymptotic behavior of coupled dynamical systems with multiscale aspects[END_REF] for the continuous dynamic and [START_REF] Attouch | Coupling forward-backward with penalty schemes and parallel splitting for constrained variational inequalities[END_REF][START_REF] Peypouquet | Coupling the gradient method with a general exterior penalization scheme for convex minimization[END_REF][START_REF] Czarnecki | Splitting forward-backward penalty scheme for constrained variational problems[END_REF] for the corresponding algorithms). When F = f : H -→ R, it is proved in [START_REF] Reich | Nonlinear evolution equations and nonlinear ergodic theorems[END_REF][START_REF] Cominetti | Strong asymptotic convergence of evolution equations governed by maximal monotone operators with Tikhonov regularization[END_REF] that the dynamic (6.59) converges to the closest element to u d among argmin x∈H f , provided that ǫ(t) tends to zero, but not too fast (ε(•) / ∈ L 1 ([0, +∞[, R)). We can hope to obtain similar results in our vector optimization problem, that is, finding the closest element to u d among the weak efficient points. Remark 6.4.2 (On the convergence towards efficient points instead of weak efficient points). Theorem 6.2.6 only guarantees the convergence of the trajectories of (SD) towards weak efficient points. Of course, being a weak efficient point is a necessary condition to be an efficient point, but one might ask for a method ensuring the convergence to efficient points. A well-known method to turn weak efficient points into efficient points is to equip Y with a bigger cone than K. More exactly, consider a closed convex cone K such that K \ {0} ⊂ int K. It is easy, according to the definitions, to see that weak efficient points with respect to K are efficient points with respect to the original cone K. Thus, it suffices to consider a dynamic governed by the order induced by K instead of K.

As an example, consider the bi-objective smooth case

F = (f 1 , f 2 ) : H -→ (R 2 , R 2 +
). Fix a parameter ε > 0, and consider the cone

Kε := {x = (x 1 , x 2 ) ∈ R 2 | (1 -ε)x 1 + εx 2 ě 0 and εx 1 + (1 -ε)x 2 ě 0}.
Its dual cone is

K+ ε = {x = (x 1 , x 2 ) ∈ R 2 | εx 2 ď (1 -ε)x 1 and εx 1 ď (1 -ε)x 2 },
whose base can be taken as Clearly, Kε is an enlargement of R 2 + , such that R 2 + \ {0} ⊂ int Kε . From a dual point of view, the basis ∆ 2,ε is a retract of the unit simplex ∆ 2 . Since this new cone Kε is still polyhedral, the dynamic induced by this order is the same, from the complexity point of view. Indeed, according to Proposition 5.1.5, the ordered subdifferential of F with respect to the basis ∆ 2,ε is

∆ 2,ε := {(λ, 1 -λ) ∈ R 2 | λ ∈ [ε, 1 -ε]}.
∂ C F (x) = co{ε∇f 1 (x) + (1 -ε)∇f 2 (x), ε∇f 2 (x) + (1 -ε)∇f 1 (x)}.
This approach can be generalized to the general multi-objective setting F :

H -→ (R m , R * + ), by considering the reduced base ∆ m,ε := {θ = (θ i ) ∈ ∆ m | θ i ∈ [ε, 1 -(m -1)ε]}.
Note that this approach suffers some drawbacks. For instance, it guarantees the convergence to an efficient point, but not that any efficient point can be recovered. See also [START_REF] Fukuda | On the convergence of the projected gradient method for vector optimization[END_REF] for a short discussion on this topic. Remark 6.4.3 (The general differential inclusion and the parameter selection problem). Consider here that we are in the multi-objective setting F = (f 1 , ..., f m ) : H -→ R m . A very interesting feature of (SD) is that it selects itself, at (almost) each time, a convex combination θ ∈ ∆ m . Thus, we can see locally this dynamic as a steepest descent associated to the convex combination m i=1 θ i f i . It is a clear improvement with respect to the usual weighting method, which asks to choose a fixed convex combination.

A parallel with this situation can be done with what we call the parameter selection problem. In optimization, and in particular in inverse problems, it happens that we often need to solve the problem (6.60) min

f (x)=0 R(x),
where R : H -→ R is a regularization term, and f :

H -→ R is a data fidelity term. A common example is H = R n , R(x) = x 1 and f (x) = 1 2 Ax -b , for A ∈ M n,p (R) and b ∈ R p .
In general R is not smooth, and managing both a nonsmooth function and a constraint can be difficult in practical. To overcome this difficulty, we can chose to relax the original problem into the following one (6.61)

min x∈H R(x) + αf (x),
where α > 0 is a parameter to choose. The whole problem is: how to find an α so that the relaxed problem (6.61) will give us a satisfactory solution with respect to the original problem (6.60)? In general, finding this parameter is an expensive task which asks to try different values for α until we are satisfied. We feel that the situation here is similar to the multi-objective one, when one tries to find the good convex combination of the objective functions {f 1 , ..., f m }. Our opinion is that a good approach could be to consider a dynamic which "naturally" let the parameter α evolve. Note that (6.61) can be equivalently rewritten as (6.62) min

x∈H λR(x) + (1 -λ)f (x),
where λ = 1 1+α ∈]0, 1[. Thus, the dynamic we are looking for can be searched in the Vector Differential Inclusion discussed in Section 6.1.4:

(VDI) u(t) + co{∂ C R(x), ∂ C f (x)} ∋ 0.
One can try to consider the (SD) dynamic for F = (R, f ), but this might not be a good idea. Indeed, the (SD) dynamic minimizes simultaneously too many scalar functions, namely all the convex combinations of R and f . Thus, the trajectory will be quickly stucked in a Pareto critical point, which has nothing to do with the scalar optimization problem (6.62). Worst, the corresponding coefficient λ can tend to 0 or 1, which is too extreme in (6.62). In general, we look for a λ close to zero, but not too much.

A different approach could be to perform a steepest descent-like dynamic, but enforcing the parameter λ to remain in a desired zone, namely λ ∈ [a, b] ⊂]0, 1[. This is equivalent to consider the vector optimization problem for the function F = (R, f ) : H -→ R 2 , R 2 being endowed with a bigger cone K than R 2 + . As in the previous remark, consider the cone

K := {x = (x 1 , x 2 ) ∈ R 2 | ax 1 + (1 -a)x 2 ě 0 and bx 1 + (1 -b)x 2 ě 0}, then its dual cone K + admits the desired base ∆ a,b := {(λ, (1 -λ)) ∈ ∆ 2 | λ ∈ [a, b]}.
It would be interesting to look at how λ evolves with time, if it converges to a particular value, or if it tends to remain close to a particular value. Of course this is a simple suggestion, and we could imagine other dynamics for this problem, which would take more into account the structure of the original problem.

Remark 6.4.4 (Links with cone-constrained problems). It would be interesting to investigate applications of the tools developed in the two previous chapters, to the class of optimization problems under cone constraint. They wrote as follows:

min x∈H h(x) under the constraint G(x) ∈ -K,
where h : H -→ R, G : H -→ Y and K ⊂ Y is a closed convex cone with nonempty interior (we assume these functions to be smooth). This general problem covers for instance the Nonlinear Complementary Problems, or the Semi Definite Programming. It arises for instance in mechanics [START_REF] Pinto Da Costa | The directional instability problem in systems with frictional contacts[END_REF], and is the subject of an active research, see [START_REF] Vandenberghe | Semidefinite programming[END_REF][START_REF] Seeger | Eigenvalue analysis of equilibrium processes defined by linear complementarity conditions[END_REF][START_REF] Adly | A nonsmooth algorithm for cone-constrained eigenvalue problems[END_REF][START_REF] Adly | A new method for solving Pareto eigenvalue complementarity problems[END_REF][START_REF] Haddou | Smoothing methods for nonlinear complementarity problems[END_REF] and the references therein. In the spirit of [START_REF] Fliege | Steepest descent methods for multicriteria optimization[END_REF], we can propose an interior-point method for solving this cone-constrained problem, based on the steepest descent direction. Choose an error parameter ε > 0, and note Θ a weakly * compact convex base of K + . The condition σ Θ (y) ď -ε is a barrier for the constraint -K, since Proposition A.3.5 asserts that

-K = {y ∈ Y | σ Θ (y) ď 0}.
Thus, it seems natural to design an interior-point method as follows: Given a current iterate x n ∈ H, test whether the condition σ Θ (G(x n )) ď -ε holds or no. If so, this means that G(x n ) lies in the interior of of -K. In that case, we use a step of any classic descent method with respect to h, to obtain

x n+1 . If σ Θ (G(x n )) > -ε, it means that G(x n ) is too close to the boundary of -K.
In that case, we make decrease both h and G, by computing a steepest descent step with respect to

F : H -→ R × Y x -→ F (x) := (h(x), G(x)),
where R × Y is ordered by the cone R + × K. Its dual cone is R + × K + , whose base can be taken as {1} × Θ. In other words, we propose to perform

x n+1 = x n + λ n s(x n ) where (6.63) s(x n ) = argmin d∈H 1 2 d 2 + ∇h(x n ), d + σ Θ (DG(x n ; d)).
The main difficulty in this method would be the resolution of (6.63), which asks to solve a convex quadratic programming. Using the definition of the support function σ Θ , it can be rewritten as a saddle-point problem:

s(x n ) = argmin d∈H sup θ∈Θ 1 2 d 2 + ∇h(x n ) + D * G(x n ; θ), d .
Together with the explicit description of σ Θ that we know for classical cones (see Example 2.3.8), we also have from Proposition A.3.6 this useful characterization:

∀y ∈ Y, σ Θ (y) = inf{t ∈ R | y -te ∈ -K},
where e ∈ int K defines the base Θ by {y * ∈ K + | y * , e = 1}. Remark 6.4.5 (Vector optimization problems and the Kurdyka-Lojasiewicz property). In this chapter, we proved the convergence for the trajectories of (SD), assuming that F : X -→ Y is convex. In the light of the results of Part I, it would be interesting to know whether a similar result can be obtained for vector-valued functions satisfying some analogue of the Kurdyka-Lojasiewicz inequality. We present briefly some ideas going in this direction. For simplicity, we assume that we are in the multiobjective setting, and that

F (x) = (f 1 (x), . . . , f m (x)) is of class C 1 .
A natural hypothesis, when considering the finite family of functions {f 1 , . . . , f m }, is assuming that each of them is a K L function. From this, we aim to derive the finite length property for the trajectories of (SD). Consider then a strong global solution u : [0, +∞[-→ H of (SD), and assume that it has some strong limit point u

∞ . Consider h(t) = m i=1 ϕ i (f i (u(t)) -f i (u ∞ )),
where each ϕ i is the desingularizing function corresponding to f i around u ∞ . Its derivative is:

h ′ (t) = m i=1 ϕ ′ i (f i (u(t)) -f i (u ∞ )) ∇f i (u(t)), s(u(t)) .
Assume that the following angle condition holds: (6.64) ∃ε > 0 such that for a.e. t ě 0,

m i=1 -∇f i (u(t)), s(u(t)) ∇f i (u(t)) s(u(t)) ě ε.
Then, by using successively the Kurdyka-Lojasiewicz inequality and this angle condition, we can deduce that

-h ′ (t) ě ε u(t) ,
which is the key estimation for the proof of convergence in Theorems 3.1.8 and 3.2.2. Clearly, the angle hypothesis (6.64) is automatically satisfied when m = 1, since in that case s(u(t)) = -∇f 1 (u(t)). More generally, (6.64) means that, at almost every t ě 0, there exists some i(t) such that the cosine between ∇f i(t) (u(t)) and s(u(t)) is bounded from above by ε. This angle hypothesis is quite difficult to verify in practice, so we propose an other approach by making a K L-like assumption directly on F . It is based on a generalized Kurdyka-Lojasiewicz inequality used in [START_REF] Bárta | Every ordinary differential equation with a strict Lyapunov function is a gradient system[END_REF] to study general gradient-like systems. Suppose that we are given some locally Lipschitz Lyapunov function E : H -→ R, such that (E • u) ′ (t) ď 0 for a.e. t ě 0. Then, similarly to what has been done in Section 3.1.1, the finite length of u(•) can be derived from the following generalized Kurdyka-Lojasiewicz inequality:

(6.65) ∀x ∼ x, ∀x * ∈ ∂ C E(x), ϕ ′ (E(x) -E(x)) x * , -s(x) ě s(x) .
We know from Proposition 6.1.7 that each function f i is Lyapunov for the (SD) dynamic, as any convex combination of them. The max function max i∈{1,...,m} f i is also Lyapunov for the dynamic.

It is an open question to know whether (6.65) is satisfied or no for one of the aforementioned Lyapunov functions.

result used in [START_REF] Attouch | A continuous gradient-like dynamical approach to Paretooptimization in Hilbert spaces[END_REF], which relies itself on Peano's Theorem. After that, we will need to define a 'good' approximation F λ of F , and this will rely on the Moreau-Yosida approximation for convex real-valued functions, which is a widely used method in nonsmooth convex analysis (see [START_REF] Attouch | Variational convergence for functions and operators[END_REF][START_REF] Aubin | Applied nonlinear analysis[END_REF][START_REF] Bauschke | Convex analysis and monotone operator theory[END_REF][START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF][START_REF] Zeidler | Nonlinear functional analysis and its applications, Part II: Monotone operators[END_REF][START_REF] Peypouquet | Convex optimization in normed spaces. Theory, methods and examples[END_REF] for a detailed presentation). Section 7.2 will be uniquely devoted to the convex (nonsmooth) multi-objective case. Indeed, in that setting, the function F is explicitly defined by a finite family of real-valued functions (f 1 , ..., f m ). Then it suffices to define F λ := (f 1,λ , ..., f m,λ ), where f i,λ is the Moreau-Yosida regularisation of f i with index λ > 0 (see Proposition 7.2.2 in Section 7.2 for more details). In that case (SD) λ writes (SD) λ uλ (t) + (N C (u λ (t)) + co∇f i,λ (u λ (t))) 0 = 0, and we prove the existence of a solution of (SD) by passing to the limit in (SD) λ when λ → 0.

In section 7.3, we consider the general vector case, and we start by defining properly some function F λ satisfying the desired properties. As one can expect, this construction will rely on Moreau-Yosida approximations of the cost functions {f θ } θ∈Θ . This being done, the analysis is almost the same than for the multi-criteria case.

Since the general existence result in Section 7.3 covers the multi-objective one in Section 7.2, one might question the pertinence of presenting in a separate way the results in section 7.2. We made this choice essentially because the definition of F λ in the multicriteria case is direct and intuitive, which is not the case for the general vector case. Moreover, the proof in Section 7.3 faces some delicate questions about measurabiliy which does not occur in the multicriteria case. With this presentation, the reader which is only interested in the multicriteria case can directly access an easier proof.

In all this chapter, it is assumed, if not specified, that E is an Euclidean space, that Y is a Banach space, and that F : E -→ Y is locally Lipschitz continuous.

Existence in the smooth vector case

We start with a regularity result for the steepest descent vector field, when F is smooth. But before, we recall the notion of Hausdorff continuity for a set-valued mapping. We say that a set-valued mapping S : E ⇒ E is Hausdorff continuous at x ∈ E whenever d H (S(x n ), S(x)) --→ n→+∞ 0 for all sequence (x n ) n∈N converging to x. In an obvious way, we say that S is Lipschitz Hausdorff continuous on U ⊂ E if there exists some L ě 0 such that

∀x, x ′ ∈ U, d H (S(x), S(x ′ )) ď L x -x ′ .
Of course, Lipschitz Hausdorff continuity on U entails Hausdorff continuity on U . Proposition 7.1.2. Let F : E -→ Y be a Fréchet differentiable function, with its derivative being Lipschitz continuous

U ⊂ E. Then, i) x -→ ∂ C F (x) is Lipschitz Hausdorff continuous on U , ii) x -→ ∂ C F (x) 0 is Holder continuous on U , iii) there exists c > 0 such that ∀x ∈ U , ∀x ′ ∈ ∂ C F (x), x ′ ď c(1 + x ).
Proof. During all the proof, we note M > 0 a bound for Θ in Y * . We start with item i). Take an arbitrary pair x 1 , x 2 ∈ U . Consider any p 1 ∈ ∂ C F (x 1 ), then there exists θ 1 ∈ Θ such that

p 1 = θ * 1 • DF (x 1 ). Define p 2 := θ * 1 • DF (x 2 ) ∈ ∂ C F (x 2 ). It follows that d(p 1 , ∂ C F (x 2 )) = inf p∈∂ C F (x 2 ) p 1 , p ď p 1 -p 2 = θ * 1 • (DF (x 1 ) -DF (x 2 )) ď θ * 1 Y * DF (x 1 ) -DF (x 2 ) L(E,Y) ď M Lip(DF ; U ) x 1 -x 2 .
Since the latter is true for any p 1 ∈ ∂ C F (x 1 ), it follows that sup

p 1 ∈∂ C F (x 1 ) d(p 1 , ∂ C F (x 2 )) ď M Lip(DF ; U ) x 1 -x 2 .
By symmetry, we obtain

(7.2) d H (∂ C F (x 1 ), ∂ C F (x 2 )) ď M Lip(DF ; U ) x 1 -x 2 .
We follow with item ii). Take two arbitrary x 1 , x 2 ∈ U . For the sake of simplicity, we write s 1 := ∂ C F (x 1 ) 0 and s 2 := ∂ C F (x 2 ) 0 . Then :

(7.3) s 1 -s 2 = proj ∂ C F (x 1 ) (0) -proj ∂ C F (x 2 ) (0) .
According to [32, Proposition 5.1], we have the following estimation:

(7.4) proj ∂ C F (x 1 ) (0) -proj ∂ C F (x 2 ) (0) 2 ď ( s 1 + s 2 ) d H (∂ C F (x 1 ), ∂ C F (x 2 )).
Then, combine (7.3) and (7.4) together with (7.2) to obtain

(7.5) s 1 -s 2 2 ď ( s 1 + s 2 ) M Lip(DF ; U ) x 1 -x 2 .
Since DF is Lipschitz continuous on U , it follows from the mean value theorem that F is also Lipschitz continuous on U . So, according to Proposition 5.1.7, we finally obtain

s 1 -s 2 2 ď 2M 2 Lip(F ; U )Lip(DF ; U ) x 1 -x 2 .
We end the proof with item iii). Fix now some x U ∈ U , and take any x ∈ U and x ′ ∈ ∂ C F (x). Then, using the triangular inequality and the Lipschitz continuity of DF on U :

x ′ ď M DF (x) ď M DF (x) -DF (x U ) + M DF (x U ) ď M Lip(DF ; U ) x ′ -x U + M DF (x U ) ď c(1 + x ),
where c := max{M Lip(DF ; U ); M Lip(DF ; U ) x U + M DF (x U ) }.

We can now state the main result of this section. It involves a space of bounded weak * measurable functions, for which we refer to Appendix B.3 for its definitions and properties. If we suppose moreover that Y is separable, then θ ∈ L ∞ w * ([0, +∞[, Y * ). The proof is based on the following abstract existence result, which can be found in the work of Attouch and Goudou [START_REF] Attouch | A continuous gradient-like dynamical approach to Paretooptimization in Hilbert spaces[END_REF]Theorem 3.5] (see also the paper of Henry [START_REF] Henry | An existence theorem for a class of differential equations withmultivalued righthand side[END_REF]). Theorem 7.1.4. Let S : E ⇒ E be a Hausdorff continuous set-valued mapping, taking convex compact values, and satisfying the following growth condition :

∃c > 0 such that ∀x ∈ E, ∀x ∈ S(x), x ď c(1 + x ).
Suppose that E is Euclidean. Then, for all u 0 ∈ C, there exists a strong global solution u : [0, +∞[-→ E of Define the application

(7.6) u(t) + (N C (u(t)) + S(u(t))) 0 = 0 such that u(0) = u 0 .
j : R + × Θ -→ R (7.7) (t, θ) -→ θ • DF (u(t)) + η(t) E .
We verify first that j(t, •) is w * -continuous on Θ, for any fixed t ě 0. Consider for this any ). But we supposed E to be finite dimensional, so this convergence holds in (E, • ). Because of the strong continuity of the norm in E, it follows that j(t, •) is w * -continuous on Θ. Now we verify that j(•, θ) is measurable for all θ ∈ Θ. We know that u(•) is continuous, and so is DF by assumption, so it follows that t -→ θ • DF (u(t)) is continuous between R + and E, hence measurable. Moreover η is also measurable, because of item iii). Since the sum and the composition of measurable functions is measurable (see [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Lemma 4.22]), we deduce that j(•, θ) is measurable.

We proved that j is a Caratheodory function, and because of the separability assumption on Y, (Θ, w * ) is a separable metrizable compact space (see Proposition C.0.2). So we can apply Castaing's measurable selection Theorem B.1.2, which asserts that the function θ : R + -→ (Θ, w * ) defined by (7.8) θ(t) := argmin

θ∈Θ θ • DF (u(t)) + η(t) E
is Borel measurable. Observe that, by definition of θ, we have θ(t) = θ(t) for a.e. t ě 0. It remains only to verify that θ is weakly- * measurable. For this, consider any y ∈ Y and show that θ • ev y : R + -→ R is measurable, where ev y : y * ∈ Y * -→ y * , y Y * ×Y . We shown that θ is Borel measurable between R + and (Θ, w * ), and we know that ev y is continuous from (Θ, w * ) to R, so their composition is Borel measurable ([5, Lem 4.22]).

Existence in the nonsmooth convex multiobjective case

Here we aim to prove a weaker version of Theorem 7.0.1, in the multi-objective case: Theorem 7.2.1. Let (f i ) i∈{1,...,m} : E -→ R be a finite family of convex continuous functions being bounded from below, and C a nonempty closed convex subset of E. Suppose that E is an Euclidean space. Then, for all u 0 ∈ E, there exists a strong global solution u : [0, +∞[-→ E of (SD) u(t) + (N C (u(t)) + co ∂f i (u(t))) 0 = 0 satisfying the Cauchy condition u(0) = u 0 .

As we explained at the beginning of this chapter, the proof will heavily rely on a Moreau-Yosida regularization of the objective functions (f i ) i∈{1,...,m} . We recall briefly the definition of the Moreau-Yosida approximation for a proper lower semi-continuous convex function f : E -→ R ∪ {+∞}, and summarize its main properties in the following statement, see in particular [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]Proposition 2.11], or more recently [START_REF] Peypouquet | Convex optimization in normed spaces. Theory, methods and examples[END_REF]Section 3.5.4]. Recall that we suppose in this Chapter 7 that E is a finite-dimensional Hilbert space. Proposition 7.2.2. Let f : E → R ∪ {+∞} be a lower semi-continuous convex proper function. The Moreau-Yosida approximation of index λ > 0 of f is the function f λ : E → R which is defined for all x ∈ E by (7.9)

f λ (x) = inf d∈E f (d) + 1 2λ d -x 2 .
i) The infimum in (7.9) is attained at a unique point prox λf (x) ∈ E, which satisfies

f λ (x) = f (prox λf (x)) + 1 2λ
xprox λf (x) 2 ; (7.10) prox λf (x) + λ∂f (prox λf (x)) ∋ x. (7.11) prox λf : E -→ E is a nonexpansive operator. It is the resolvent of index λ of ∂f : E ⇒ E.

ii) f λ is convex, and Gateaux differentiable. Its gradient at x ∈ E is equal to (7.12) ∇f λ (x) = 1 λ (xprox λf (x)).

The operator ∇f λ : E -→ E is the Yosida approximation of index λ of the maximal monotone operator ∂f . It is Lipschitz continuous with Lipschitz constant

1 λ . iii) For any x ∈ dom∂f , ∇f λ (x) ď ∂ 0 f (x) , (∂ 0 f (x) is the element of minimal norm of ∂f (x)). iv) For any x ∈ E, f λ (x) ď f (x), inf E f λ = inf E f and f λ (x) ↑ f (x) as λ ↓ 0.
For all i ∈ {1, ..., m} and λ > 0, we will note f i,λ := (f i ) λ the Moreau-Yosida regularization of f i , with index λ > 0. Define F λ : E -→ (R m , R m + ), for λ > 0, by F λ (x) := (f 1,λ (x), ..., f m,λ (x)). Proposition 7.2.2 asserts that F λ is convex and continuously differentiable, with a 1 λ -Lipschitz gradient. In that case, we can apply Theorem 7.1.3 to obtain, for all λ > 0, the existence of a trajectory u λ : [0, +∞[-→ E such that:

• u λ is absolutely continuous on [0, T ] for all T > 0,

• uλ (t) + (N C (u λ (t)) + co ∇f i,λ (u λ (t))) 0 = 0 for a.e. t ě 0, and u λ (0) = u 0 ,

• there exists η λ : [0, +∞[-→ E and θ λ = (θ i,λ ) i∈{1,...,m} : [0, +∞[-→ R m such that -uλ (t) + η λ (t) + m i=1 θ i,λ (t)∇f i,λ (u λ (t)) = 0 for a.e. t ě 0, -η λ (t) ∈ N C (u λ (t)) and θ λ (t) ∈ ∆ m for a.e. t ě 0,

-η λ ∈ L 2 ([0, T ], E) and m i=1 θ i,λ (•)∇f i,λ (u λ (•)) ∈ L 2 ([0, T ], E) for all T > 0, -θ λ ∈ L ∞ ([0, +∞[, R m ).
Now we turn on the proof of Theorem 7.2.1, by passing to the limit when λ tends to zero. We chose to divide the proof into three Lemmas. Basically, in the first lemma we pass to the limit when λ ↓ 0, and in the two last lemmas we prove that the limiting trajectory is a solution of (SD). We will not recall that we assume the hypotheses of Theorem 7.2.1 to hold. Lemma 7.2.3. When λ ↓ 0, we have (taking eventually a subnet) for all T > 0:

i) u λ • --→ u in C([0, T ], E), ii) ( uλ ) λ>0 is bounded and uλ w --→ u in L 2 ([0, +∞[, E), iii) (η λ ) λ>0 is bounded and η λ w --→ η in L 2 ([0, T ], E), with η(t) ∈ N C (u(t)) for a.e. t ě 0, iv) ∀i ∈ {1, ..., m}, θ i,λ w * --→ θ i in L ∞ ([0, T ], R), with θ i (t) ě 0 and m i=1 θ i (t) = 1 for a.e. t ě 0, v) ∀i ∈ {1, ..., m}, f i,λ • u λ • --→ f i • u in C([0, T ], R).
Proof. Let us establish bounds for the net (u λ ) λ>0 , which are independent of λ. By a similar argument to that used in the asymptotic study (see Theorem 6.2.6), we obtain

+∞ 0 uλ (t) 2 dt ď f i,λ (u 0 ) -inf E f i,λ .
Then notice that f i,λ (u 0 ) ď f i (u 0 ), and inf

E f i,λ = inf E f i . Hence +∞ 0 uλ (t) 2 dt ď f i (u 0 ) -inf E f i ,
∇f i,λ (v) ď (∂f i ) 0 (v) .
Combining (7.15) with (7.17), and using that f i is a convex continuous function whose subdifferential ∂f i is bounded on bounded sets, we obtain, for any T > 0 (7.18) sup

λ ∇f i,λ (u λ ) L ∞ ([0,T ];E) < +∞.
We deduce that the net (η λ ) λ>0 remains bounded in L 2 ([0, T ]; E) for any T > 0, and admits then a weak limit point η. We now turn on item iv). For all i ∈ {1, ..., m}, (θ i,λ ) λ>0 is bounded in L ∞ ([0, +∞[, R), so we can assume, taking eventually a subnet, that these nets weakly * converge to some

θ i ∈ L ∞ ([0, +∞[, R). Since L ∞ + ([0, +∞[, R
) is a weakly * closed cone (as the dual of L 1 + ([0, +∞[, R)), we know that θ i (t) ě 0 for a.e. t ě 0. We note θ := (θ i ) i∈{1,...,m} , which is a weak * limit of θ λ when λ ↓ 0. Endowing L ∞ ([0, +∞[, R) with the weakly * lower semi-continuous norm φ ∞ := supess For item v), use Proposition 7.2.2 to obtain the pointwise convergence of (f i,λ ) to f i . Combining (7.17) with the local Lipschitz property of the objective functions (f i ) i∈{1,...,m} , we obtain that the net (f i,λ ) λ>0 is equi-Lipschitz continuous on bounded sets (recall that E has finite dimension). Hence, by Ascoli's Theorem, (f i,λ ) uniformly converges to f i on bounded sets. Combining this result with the uniform convergence of (u λ ) gives the result. Lemma 7.2.4. For a.e. t ě 0, u(t) + N C (u(t)) + co ∂f i (u(t)) ∋ 0. Proof. Our difficult task consists in passing to the limit in (7.19) uλ (t) + η λ (t) + m i=1 θ i,λ (t)∇f i,λ (u λ (t)) = 0, which contains a product of two weakly converging sequences (θ i,λ ) λ>0 and (∇f i,λ (u λ )) λ>0 .

0

uλ (t)η λ (t), ξ(t)u λ (t) dt.

Since f i,λ (ξ(t)) ď f i (ξ(t)), and θ i,λ (t) ě 0, we obtain (7.20)

T 0 q i=1 θ i,λ (t)f i (ξ(t))dt ě T 0 q i=1 θ i,λ (t)f i,λ (u λ (t))dt + T 0 -uλ (t) -η λ (t), ξ(t) -u λ (t) dt.
For any ξ ∈ L ∞ ([0, T ]; E) and all i ∈ {1, ..., m}, since f i is Lipschitz continuous on bounded sets, we have Let us now pass to the limit on the right-hand-side of (7.20). For the first term, we use Lemma 7.2.3. For the second term, we notice that uλ + η λ converges weakly in L 2 ([0, T ], E) to u + η, ξu λ converges uniformly, and hence strongly in L 2 ([0, T ], E) to ξu. We obtain (7.21)

f i • ξ ∈ L ∞ ([0, T ], R). Moreover θ i,λ w * --→ λ→0 θ i in L ∞ ([0, T ], R).
T 0 m i=1 θ i (t)f i (ξ(t)) dt ě T 0 m i=1 θ i (t)f i (u(t)) dt + T 0 -u(t) -η(t), ξ(t) -u(t) dt.
Let us interpret this inequality in the duality pairing bewteen the functional spaces

L ∞ ([0, T ], E) and L 1 ([0, T ], E) ⊂ (L ∞ ([0, T, E)) * .
For this, introduce Φ, the integral functional on L ∞ ([0, T ], E)

(7.22) Φ(ξ) = T 0 m i=1 θ i (t)f i (ξ(t)) dt. Φ : L ∞ ([0, T ], E) → R is convex and continuous on L ∞ ([0, T ], E).
Hence, (7.21) can be rewritten as

(7.23) -u -η ∈ ∂Φ(u).
According to a duality theorem of Rockafellar for convex functional integrals that we extended in Appendix (see Corollary B.2.11), the inclusion (7.23) implies that for almost all t > 0,

-u(t) -η(t) = m i=1 θ i (t)v i (t)
with measurable functions v i ∈ L ∞ ([0, T ], E) such that (7.24) v i (t) ∈ ∂f i (u(t)) for almost all t > 0.

In other words, we proved that u(t) + N C (u(t)) + co ∂f i (u(t)) ∋ 0.

Lemma 7.2.5. For a.e. t ě 0, u(t) + (N C (u(t)) + co ∂f i (u(t))) 0 = 0.

Proof. Since we already proved in 7.2.4 that u is a solution of the vector differential inclusion u(t) + N C (u(t)) + co ∂f i (u(t)) ∋ 0, we will use the criterion of Proposition 6.1.15 to verify that it is in fact a lazy solution. In other words, we need to prove that

(7.25) ∀θ = (θ i ) ∈ ∆ m , u(t) 2 + m i=1 θ i d dt (f i • u)(t) ď 0.
We will use of course the fact that each u λ is a lazy solution of (SD) λ , and satisfies (using again Proposition 6.1.15)

(7.26) ∀θ = (θ i ) ∈ ∆ m , uλ (t) 2 + m i=1 θ i d dt (f i,λ • u λ )(t) ď 0.
So the whole point is to pass to the limit in (7.26), when λ → 0, to obtain (7.25). Take any ψ ∈ D(]0, T [, R), the space of infinitely differentiable functions with support uncluded in ]0, T [, which is dense in L 2 ([0, T ], R) (see [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]Appendices]). Consider then the corresponding linear integrand

Ψ : L 2 ([0, T ], E) -→ R w -→ T 0 ψ(t) w(t) 2 dt.
It is easy to see that Ψ is strongly continuous, hence weakly lower semi-continuous. As a direct consequence of uλ w --→ λ→0 u in L 2 ([0, T ], E), we deduce that (7.27)

T 0 ψ(t) u(t) 2 dt ď lim inf λ→0 T 0 ψ(t) uλ (t) 2 dt.
Now, use an integration by parts together with the fact that ψ(0) = ψ(T ) = 0 to write for all θ ∈ ∆ m (7.28)

T 0 ψ(t) m i=1 θ i f i • u ′ (t) dt = - T 0 ψ ′ (t) m i=1 θ i f i (u(t)) dt.
Because of Proposition 7.3.20, f i,λ • u λ converges uniformly to f i • u in C([0, T ], R), hence from (7.28) we deduce

T 0 ψ(t) m i=1 θ i f i • u ′ (t) dt = -lim λ→0 T 0 ψ ′ (t) m i=1 θ i f i,λ (u λ (t)) dt.
Doing again an integration by parts, we obtain (7.29)

T 0 ψ(t) m i=1 θ i f i • u ′ (t) dt = lim λ→0 T 0 ψ(t) m i=1 θ i f i,λ • u λ ′ (t) dt.
Combines now (7.27) and (7.29) to find

T 0 ψ(t) u(t) 2 + m i=1 θ i d dt (f i • u)(t) dt ď lim inf λ→0 T 0 ψ(t) uλ (t) 2 + m i=1 θ i d dt (f i,λ • u λ )(t) dt.
If we suppose that ψ ∈ D + (]0, T [, R), i.e. that ψ takes positive values, we can deduce from (7.26) that

T 0 ψ(t) u(t) 2 + m i=1 θ i d dt (f i • u)(t) dt ď 0.
By using the density of

D + (]0, T [, R) in L 2 + (]0, T [, R), which is a closed convex self-dual cone, we obtain that u(•) 2 + m i=1 θ i d dt (f i • u) ∈ -L 2 + (]0, T [, R
). This proves (7.25), and ends the proof with Proposition 6.1.15.

Existence in the nonsmooth convex vector case

When observing what we did in the multicriteria case, one might be tempted in the general case F : E -→ Y to regularize the cost functions {f θ } θ∈Θ . Indeed we saw in the previous chapters that these functions were the key to understand the behaviour of F with respect to the order in Y. Hence it would be natural, adapting the method of the previous section, to consider the following dynamic:

(SD) λ uλ (t) + N C (u λ (t)) + co * θ∈Θ ∇f θ,λ (u λ (t)) 0 = 0,
where f θ,λ denotes the Moreau-Yosida regularisation of f θ with index λ > 0. Indeed it seems, at least at first sight, that co * θ∈Θ ∇f θ,λ is a good approximation for

∂ C F = co * θ∈Θ ∂ C f θ . But
two basic observations must be made at this point.

First, if we reduce to the multi-objective case, this (SD) λ dynamic is in principle different from the one used in the previous section, which involved the convex hull of the gradients ∇f i,λ (u λ (t)). This is because the Moreau-Yosida regularization f θ,λ is not linear with respect to θ. In other words, for general θ ∈ ∆ m , it happens that

∇f θ,λ (x) = ∇ m i=1 θ i f i λ (x) = m i=1 θ i ∇f i,λ (x).
The second concern with this formulation is that there is no explicit vector-valued function F λ : E -→ Y whose ordered subdifferential is given by co * θ∈Θ ∇f θ,λ . And this is essential for who wants to pass to the limit when λ goes to zero.

The aim of the following subsection 7.3.1 is to give the good setting in which we will be able to exploit the cost functions f θ . In subsection 7.3.2 we define properly a Moreau-Yosida-like regularization of F , for which the ordered subdifferential will be as desired co * θ∈Θ ∇f θ,λ . It will rely on the appropriate choice of a function F λ : E -→ Z for some well-chosen Z = Y. The essential point in this trick is that the ordered subdifferential essentially depends on the initial space. Subsection 7.3.3 contains the proof of our main existence theorem. It follows the lines of the proof in the multi-objective case.

An equivalent description of ∂F

Recall that Θ is a w * -compact convex subset of Y * . We note C(Θ) the vector space of (bounded) continuous functions from (Θ, w * ) to (R, | • |). It is a Banach space, once equipped with the uniform norm :

(7.30) ∀ψ ∈ C(Θ), ψ C(Θ) := sup θ∈Θ ψ(θ).
This space is naturally ordered by the closed convex cone of positive continuous functions on Θ:

C + (Θ) := {ψ ∈ C(Θ) | ψ(θ) ě 0 for all θ ∈ Θ}.
We will note ĺ C(Θ) the order induced by C + (Θ), and ă C(Θ) the strict order induced by its interior cone, namely :

C ++ (Θ) := int C + (Θ) = {ψ ∈ C(Θ) | ψ(θ) > 0 for all θ ∈ Θ} = {ψ ∈ C(Θ) | inf θ∈Θ ψ(θ) > 0}.
The topological dual space of C(Θ) is noted M R (Θ), and can be identified with the Banach space of Radon measures on (Θ, w * ), equipped with the total variation norm2 • M(Θ) . This space of Radon measures inherits an order induced by the dual cone of C + (Θ), that we note M R + (Θ), and which can be identified with the set of positive Radon measures on Θ. Since C + (Θ) has a nonempty interior, we can find a weakly * compact convex base for M R + (Θ). In the following, we will always consider that the base of M R + (Θ) is

(7.31) P R (Θ) := {µ ∈ M R + (Θ) | µ, 1 = 1} where 1 ∈ C ++ (Θ) = int C + (Θ) is the constant application 1 : θ ∈ Θ → 1 ∈ R.
In that case, P R (Θ) can be identified with the set of Radon probabilities on Θ. It is an easy exercise to verify that this space of Radon probabilities can be equivalently described by (7.32)

P R (Θ) = {µ ∈ M R (Θ) | µ, 1 = 1 and µ M(Θ) ď 1}.
We will often use the weak * topology in M R (Θ), relatively to the dual pair C(Θ), M R (Θ) . Hence two weak * topologies will appear in the following : the one in Y * and the one in M R (Θ). For the sake of simplicity we will both note them w * , instead of w * Y * and w * C(Θ) . In general it will be obvious to know which topology we are dealing with, because of the context, or the notations (we try as possible to use y * , θ for denoting the elements in Y * ⊃ Θ, and µ for the elements of M R (Θ) ⊃ P R (Θ)).

Our interest for C(Θ) is that there is a natural linear embedding i : Y ֒→ C(Θ). It is the composition of the canonical embeddedding Y ֒→ (Y * , w * ) * , together with the embedding by restriction (Y * , w * ) * ֒→ (Θ, w * ) * and the inclusion (Θ, w * ) * ⊂ C(Θ), that is :

i : Y ֒-→ C(Θ) y -→ i(y) : Θ -→ R θ -→ θ, y Y * ×Y
This lead to the following definition : Definition 7.3.1. We define F := i • F : E -→ C(Θ), i.e. for all x ∈ E, F(x) ∈ C(Θ) is defined by

F(x) : Θ -→ R (7.33) θ -→ f θ (x).
As we can guess from its simple definition, F shares a lot of properties with F . Proposition 7.3.2. We have the following :

i) for all x 1 , x 2 ∈ E, F (x 1 ) ĺ F (x 2 ) ⇔ F(x 1 ) ĺ C(Θ) F(x 2 ). ii) for all x 1 , x 2 ∈ E, F (x 1 ) ă F (x 2 ) ⇔ F(x 1 ) ă C(Θ) F(x 2 ). iii) If F is locally Lipschitz at x, then so is F. iv) If F is convex, then so is F. v) If F is Gateaux differentiable at x, then so is F, and for all d ∈ E, θ ∈ Θ, DF(x; d)(θ) = D * F (x; θ)(d).
vi) If F is bounded from below by me for some m ∈ R, then F is bounded from below by m1.

Proof. Items i) and ii) essentially relies on Proposition A.3.5. Indeed,

F (x 1 ) ĺ F (x 2 ) ⇔ σ Θ (F (x 1 ) -F (x 2 )) ď 0 ⇔ ∀θ ∈ Θ, f θ (x 1 ) ď f θ (x 2 ) ⇔ F(x 1 ) ĺ C(Θ) F(x 2 ),
and

F (x 1 ) ă F (x 2 ) ⇔ σ Θ (F (x 1 ) -F (x 2 )) < 0 ⇔ inf θ∈Θ θ, F (x 2 ) -F (x 1 ) Y * ×Y ⇔ inf θ∈Θ (F(x 2 ) -F(x 1 ))(θ) > 0 ⇔ F(x 1 ) ĺ C(Θ) F(x 2 ).
Items ii) to v) are a direct consequence of the definition of F as the composition of F together with the continuous linear application i. Item vi) relies on the same argument than the one of item i).

An important consequence of Proposition 7.3.2 is that F and F share the same sublevel sets, i.e. [F ĺ F (x)] = [F ĺ C(Θ) F(x)] for all x ∈ E. Hence, minimizing F or F is the same : Moreover we saw in Proposition 5.2.12 that the ordered subdifferential of F is strongly related to the geometry of its sublevel sets in E. As we can expect from this observation, we will prove below the following important result:

Theorem 7.3.3. If Y is separable, for all x ∈ E, ∂ C F (x) = ∂ C F(x).
Recall that from its definition, the ordered subdifferential ∂ C F(x) depends on the choice of a base for M R + (Θ). Here, and in all what follows, we fix the base of M R + (Θ) to be P R (Θ). In other words, (7.35)

∂ C F(x) = co * µ∈P R (Θ) ∂ C f µ (x),
where

f µ := µ • F : E -→ R x -→ µ, F(x) = Θ f θ (x) dµ(θ).
For a given θ ∈ Θ, we note δ θ the Dirac probability at θ defined by (7.36) for all ψ ∈ C(Θ), δ θ , ψ M R (Θ)×C(Θ) = ψ(θ).

Then, we can see that δ θ • F is exactly f θ . This means that the family of functions {f θ } θ∈Θ is included in the family {f µ } µ∈P R (Θ) , which gives directly that ∂ C F (x) ⊂ ∂ C F(x). In fact, we have:

A Moreau-Yosida approximation for convex vector-valued functions

Our purpose here is to propose an analogous Moreau-Yosida approximation for F, so that we can approach equation (SD) by a sequence of approximate equations involving a smooth approximation of F. Indeed, this case has been treated in Theorem 7.1.3, and we hope to pass to the limit on the corresponding solutions to obtain a solution of (SD).

Before defining what will be F λ , we need two technical results:

Lemma 7.3.5. If F is bounded from below, then for all x ∈ E, T > 0, the family

{prox λf θ (x) | θ ∈ Θ, λ ∈]0, T [} is bounded in E.
Proof. By definition, for all θ ∈ Θ, λ ∈]0, T [, x ∈ E, we have

(7.37) 1 2λ prox λf θ (x) -x 2 ď f θ (x) -f θ (prox λf θ (x)).
Use the fact that Θ is bounded in Y * , and that F is bounded from below together with Proposition 2.3.11 to conclude.

Recall that, given θ ∈ Θ and λ > 0, we note f θ,λ := (θ • F ) λ , i.e. the Moreau-Yosida transform of f θ = θ • F with index λ. Proposition 7.3.6. Let F : E -→ Y be convex and bounded from below, where E is Euclidean.

Let (θ α ) α∈A ⊂ Θ such that θ α w * --→ α∈A θ is Θ, and (λ α ) ⊂]0, +∞[ such that λ α R --→ α∈A λ in [0, +∞[. Then, for all x ∈ E, i) lim α∈A prox λαf θα (x) = prox λf θ (x) if λ = 0, x if λ = 0. ii) lim α∈A f θα,λα (x) = f θ,λ (x) if λ = 0, f θ (x) if λ = 0.
Proof. For the sake of notation, we write here p α := prox λαf θα (x), and p := prox λf θ (x) if λ = 0,

x if λ = 0. Hence item i) resumes to the proof of p α E --→ α∈A p. We know from Lemma 7.3.5 that (p α ) α∈A is bounded in E, which is finite dimensional. Then, it suffices to prove that any limit point of (p α ) α∈A is p. Take then any limit point p of (p α ) α∈A , i.e. suppose that there exists a

subnet satisfying p β E --→ β∈A p.
Now, use the definition of p β = prox λ β (θ β •F ) (x) to write, for all x ′ ∈ E :

(7.38) f θ β (p β ) + 1 2λ β p β -x 2 ď f θ β (x ′ ) + 1 2λ β x ′ -x 2 .
We want to pass to the limit in (7.38) when β ∈ A. Observe that p β is supposed to strongly converge to p ∈ E, that F is continuous from E to Y, and that θ β is bounded and w * -converging to θ in Y * . It follows that for all x ′ ∈ E,

f θ β (p β ) R --→ β∈A f θ (p), (7.39) f θ β ,λα (x ′ ) R --→ β∈A f θ (x ′ ).
To pass to the limit in the other terms of (7.38), we need to distinguish the cases λ = 0 and λ = 0.

We first start with λ = 0. In that case, for all x ′ ∈ E,

1 2λ p β -x R --→ β∈A 1 2λ p -x , (7.40) 1 2λ x ′ -x R --→ β∈A 1 2λ
x ′x .

Passing to the limit in (7.38) by using (7.39) and (7.40), we obtain

(7.41) for all x ′ ∈ E, f θ (p) + 1 2λ p -x 2 ď f θ (x ′ ) + 1 2λ x ′ -x 2 .
The above inequality states that p = prox λf θ (x). Now consider that λ = 0. In (7.38), take x ′ = x and pass to the limit using (7.39) to obtain lim sup

β∈A 1 2λ β p β -x 2 ď f θ (x) -f θ (p).
We have lim sup β∈A 1 2λ β p βx 2 < +∞, where λ β → 0 and p β → p, so necessarily, p = x. Hence item i) is proved.

In view to prove item ii), write

(7.42) f θα,λα (x) = f θα (prox λαf θα (x)) + 1 2λ α prox λαf θα (x) -x 2 .
If λ = 0, it is clear that the right member of (7.42) converges toward f θ (p) + 1 2λα px 2 , where p = prox λf θ (x), according to item i). In that case, we obtain lim α∈A f θα,λα (x) = f θ,λ (x). If λ = 0, then the first term of the right member of (7.42) tends to f θ (x) according to item i). For the second term, note that by definition of prox λαf θα (x) we have

(7.43) 1 2λ α prox λαf θα (x) -x 2 ď f θα (p α ) -f θα (x).
The right member of (7.43) goes to zero, since p α = prox λαf θα (x) tends to x, according to item i). It follows that 1 2λα prox λαf θα (x)x 2 goes also to zero in (7.42), which means that lim

α∈A f θα,λα (x) = f θ (x).
For all x ∈ E and λ > 0, consider the application

F λ (x) : (Θ, w * ) -→ R θ -→ f θ,λ (x),
where f λ,θ is the Moreau-Yosida approximation of f θ = θ • F with index λ. Using Proposition 7.3.6, we see that F λ (x) ∈ C(Θ). Hence, we can define the Moreau-Yosida approximation of F, with index λ > 0. It is the application

F λ : E -→ C(Θ) (7.44) x -→ F λ (x),
where F λ (x) has been defined above. Its properties are gathered in the following theorem : Theorem 7.3.7. Let F : E -→ Y be convex and bounded from below, with E finite-dimensional. Then :

i) for all λ > 0, F λ : E -→ C(Θ) is convex. ii) for all λ > 0, F λ is Gateaux differentiable. Its derivative DF λ : E -→ L(E, C(Θ)) satisfies the following properties: ii.a) DF λ is 1 λ -Lipschitz continuous, ii.b) for all x ∈ E, d ∈ E, θ ∈ Θ, DF λ (x; d)(θ) = ∇f θ,λ (x), d , ii.c) for all x ∈ E, µ ∈ M R (Θ), D * F λ (x; µ) = Θ ∇f θ,λ (x) dµ(θ) ∈ E, ii.d) for all x ∈ E, ∂ C F λ (x) = co * θ∈Θ ∇f θ,λ (x).
iii) for all λ > 0, x ∈ E, DF λ (x) L(E,C(Θ)) ď sup θ∈Θ ∂ 0 f θ (x) , where ∂ 0 f θ (x) denotes the element of minimal norm of ∂f θ (x).

Proof. Let (θ α ) α∈A be a net in Θ, weakly- * convergent to some θ ∈ Θ. Let us show that ∇f θα,λ (x) converges to ∇f θ,λ (x) in E.

Observe that, according to Proposition 7.2.2, ∇f θα,λ (x) ď ∂ 0 f θα (x) . Given that ∂ 0 f θα (x) ∈ ∂F (x) for all α ∈ A, and using that ∂F (x) is bounded in E, it follows that the net (∇f θα,λ (x)) α∈A is bounded in E. Since we assume that E is finite-dimensional, it is then sufficient to show that the only limit point of (∇f θα,λ (x)) α∈A is ∇f θ,λ (x).

Suppose that, for some subnet, ∇f θ β ,λ (x) converges when β ∈ A to some x ∈ E. Using the convexity of f θ β ,λ , we have for all β ∈ A and x ′ ∈ E, that (7.46)

f θ β ,λ (x ′ ) -f θ β ,λ (x) -∇f θ β ,λ (x), x ′ -x ě 0.
This can be rewritten, using the definition of F λ , (7.47)

F λ (x ′ )(θ β ) -F λ (x)(θ β ) -∇(θ β • F ) λ (x), x ′ -x ě 0.
We pass to the limit in this last expression, using our hypothesis ∇f θ β ,λ (x)

E --→ β∈A
x, and the continuity of F λ (x) on Θ (recall Proposition 7.3.6) :

(7.48) for all x ′ ∈ E, f θ,λ (x ′ ) -f θ,λ (x) -x, x ′ -x .
Since f θ,λ is convex, the latter means that x lies in the (Fenchel) subdifferential of f θ,λ . But this function being Gateaux differentiable (Proposition 7.2.2), this means that x = ∇f θ,λ (x). Proof. We define, for all x ∈ E, the application A x : E -→ C(Θ), where A x (d) ∈ C(Θ) is defined for all d ∈ E by A x (d) : θ -→ ∇f θ,λ (x), d . We shown in Lemma 7.3.10 that this last application is continuous on (Θ, w * ). Hence A x is well-defined, and it is immediate to see that it is linear and continuous, i.e. A x ∈ L(E, C(Θ)). Let us verify now that A x is the Gateaux derivative of F λ at x. We know from Proposition 7.2.2 that, for all λ > 0 and all θ ∈ Θ, that f θ,λ is convex and Gateaux differentiable at x, so (7.50) for all

x ′ ∈ E, f θ,λ (x ′ ) -f θ,λ (x) -∇f θ,λ (x), x ′ -x ě 0.
Moreover ∇f θ,λ is 1 λ -Lipschitz continuous on E, so we can deduce from the classic descent lemma (see [START_REF] Ortega | Iterative solutions of nonlinear equations in several variables[END_REF]Proposition 3.2.12] or [START_REF] Peypouquet | Convex optimization in normed spaces. Theory, methods and examples[END_REF]Lemma 1.30]) that (7.51) for all x ′ ∈ E,

1 λ x ′ -x 2 ě f θ,λ (x ′ ) -f θ,λ (x) -∇f θ,λ (x), x ′ -x .
By combining (7.50) and (7.51), we obtain for all x ′ ∈ E, x ′ = x :

(7.52) f θ,λ (x ′ ) -f θ,λ (x) -∇f θ,λ (x), x ′ -x x ′ -x ď 1 λ x ′ -x .
Take now an arbitrary nonzero d ∈ E, t ∈ R ++ . By replacing x ′ by x + td in (7.52), we obtain

(7.53) f θ,λ (x + td) -f θ,λ (x) |t| -∇f θ,λ (x), d ď 1 λ t d 2 ,
which can be rewritten as (7.54)

F λ (x + td)(θ) -F λ (x)(θ) |t| -A x (d)(θ) ď 1 λ t d 2 .
Taking the supremum over θ ∈ Θ in the above expression leads to (7.55)

F λ (x + td) -F λ (x) |t| -A x (d) C(Θ) ď 1 λ t d 2 .
If we take the limit when t goes to zero, we see that A x (d) is the Gateaux derivative of F λ at x in the direction d. Now we just need to check that DF λ is 1 λ -Lipschitz continuous. Take two arbitrary x, x ′ ∈ E, then :

DF λ (x ′ ) -DF λ (x) L(E,C(Θ)) = sup d E =1 DF λ (x ′ ; d) -DF λ (x; d) C(Θ) (7.56) = sup d E =1 sup θ∈Θ |DF λ (x ′ ; d)(θ) -DF λ (x; d)(θ)| = sup d E =1 sup θ∈Θ | ∇f θ,λ (x ′ ) -∇f θ,λ (x), d |.
Using the Cauchy-Schwarz inequality in E and the 1 λ -Lipschitz continuity of all ∇f θ,λ (x), for all θ ∈ Θ, we finally obtain

DF λ (x ′ ) -DF λ (x) L(E,C(Θ)) ď sup d E =1 sup θ∈Θ ∇f θ,λ (x ′ ) -∇f θ,λ (x) E d E ď sup d E =1 sup θ∈Θ 1 λ x ′ -x E = 1 λ x ′ -x E .
Corollary 7.3.12. Let F : E -→ Y be a convex function bounded from below, with E finitedimensional. Then, for all x ∈ E:

i) for all µ ∈ M R (Θ), D * F λ (x; µ) = Θ ∇f θ,λ (x) dµ(θ) ∈ E, ii) ∂ C F λ (x) = co * θ∈Θ ∇f θ,λ (x)
∇f θ,λ (x), d = D * F λ (x; δ θ ), d = δ θ , DF λ (x; d) = δ θ • DF λ (x), d = ∇(δ θ • F λ )(x), d .
We deduce that ∇f θ,λ (x) = ∇(δ θ • F λ )(x), so from the definition of the ordered subdifferential we obtain ∇f θ,λ (x) ∈ ∂ C F λ (x). Since this is true for any θ ∈ Θ, and because the ordered subdifferential ∂ C F λ (x) is convex and weakly * closed, we can deduce

∂ C F λ (x) ⊃ co * θ∈Θ ∇f θ,λ (x).
For the reverse inclusion, recall from Proposition 7.3.12 that F λ is smooth, so we have

∂ C F λ (x) = {D * F λ (x; µ) | µ ∈ P R (Θ)}.
Then, take any µ ∈ P R (Θ), and use item i) together with [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Theorem 11.54] to obtain D * F λ (x; µ) ∈ co * θ∈Θ ∇f θ,λ (x). It follows that

∂ C F λ (x) ⊂ co * θ∈Θ ∇f θ,λ (x),
and the claim is proved.

We follow with item iii) of Theorem 7.3.7, which states a uniform boundedness of the derivatives, with respect to λ. Proposition 7.3.13. Let F : E -→ Y be a convex function bounded from below, with E finite-dimensional. Then, for all λ > 0, x ∈ E,

DF λ (x) L(E,C(Θ)) ď sup θ∈Θ ∂ 0 f θ (x) ,
where ∂ 0 f θ (x) denotes the element of minimal norm of ∂f θ (x).

Proof. The proof follows similar arguments as in the end of the proof of Proposition 7.3.11. Let x ∈ E, then

DF λ (x) L(E,C(Θ)) = sup d E =1 DF λ (x; d) C(Θ) (7.57) = sup d E =1 sup θ∈Θ |DF λ (x; d)(θ)| = sup d E =1 sup θ∈Θ | ∇f θ,λ (x), d |.
Using the Cauchy-Schwarz inequality in E, and Proposition 7.2.2, the result follows.

We end now by studying the monotone properties of F λ , with respect to λ, to prove items iv), v) and vi) of Theorem 7.3.7.

Proposition 7.3.14. Let F : E -→ Y be a convex function bounded from below, with E finite-dimensional. Then, for all λ > 0, x ∈ E,

m1 ĺ C(Θ) F λ (x) ĺ C(Θ) F(x).
Proof. Start with the first inequality. By hypothesis, we have for all x ∈ E and all θ ∈ Θ that f θ (x) ě m (see Proposition 2.3.11). Equivalently, for all θ ∈ Θ, inf E f θ ě m. From Proposition 7.2.2 we deduce that for all x ∈ E and all θ ∈ Θ that f θ,λ (x) ě m. The latter means that m1 ĺ C(Θ) F λ (x) for all x ∈ E. Hence, the first inequality is proved.

The second inequality comes from the fact that f θ,λ (x) ď f θ (x), for all x ∈ E, θ ∈ Θ, see Proposition 7.2.2.

To prove the convergence of F λ (x) to F(x) in C(Θ), we will use the following technical lemmas: Lemma 7.3.15. Let (K, τ ) be a compact topological vector space. Let ψ ∈ C(K) and (ψ α ) α∈A ⊂ C(K). Suppose that (7.58) for all (k α

) α∈A ⊂ K, if k α τ --→ α∈A k then ψ α (k α ) R --→ α∈A ψ(k).
Hence ψ α converges uniformly to ψ on K, i.e. ψ α

C(K) --→ α∈A ψ.
Proof. Suppose by contradiction that ψ α does not converge uniformly to ψ on K. Then there exists ε > 0 and a subnet (ψ β ) β∈A such that for all β ∈ A, ψ βψ ě ε. Since ψ βψ is continuous by hypothesis, and K compact, by the extreme value theorem we obtain for all

β ∈ A some k β ∈ K such that |ψ β (k β ) -ψ(k β )| ě ε.
But, (k β ) β∈A lies in the compact space K, so there exists a subnet (k γ ) γ∈A converging to some k ∈ K. By hypothesis, it satisfies

ψ γ (k γ ) R --→ γ∈A ψ(k) and ψ(k γ ) R --→ γ∈A ψ(k), in other words |ψ γ (k γ ) -ψ(k γ )| --→
γ∈A 0 which is a contradiction.

Lemma 7.3.16. Let F : E -→ Y be a convex function bounded from below, with E finitedimensional. For all x ∈ E, the following application is continuous :

Ψ : [0, 1] -→ C(Θ) λ -→ F λ (x) if λ = 0 F(x) if λ = 0. Proof. Let (λ i ) i∈I be any net in [0, 1] converging to λ ∈ [0, 1]. If λ i > 0 for all i ∈ I,
i ) i∈I ⊂ Θ, if θ i w * --→ i∈I θ then Ψ(λ i )(θ i ) R --→ i∈I ψ(λ)(θ).
In the case λ = 0, it is clear that we can assume that λ i = 0, taking eventually a subnet. If λ = 0, then two cases must be considered. Either there exists some i ∈ I such that for all j i, λ j = 0, in which case we have

Ψ(λ j )(θ j ) = Ψ(0)(θ j ) = θ j , F (x) Y * ×Y R --→ j i θ, F (x) Y * ×Y = ψ(λ)(θ).
Or it holds that for all i ∈ I, there exists some j i such that λ j > 0. In that case J := {j ∈ I | λ j > 0} induces two subnets (λ j ) j∈J and (θ j ) j∈J for which we can apply Proposition 7.

Proof of the existence

We are now ready to prove Theorem 7.0. • u λ is absolutely continuous on [0, T ] for all T > 0,

• uλ (t) + (N C (u λ (t)) + ∂ C F λ (u λ (t))) 0 = 0 for a.e. t ě 0, and u λ (0) = u 0 ,

• there exists η

λ : [0, +∞[-→ E and µ λ : [0, +∞[-→ M R (Θ) such that -uλ (t) + η λ (t) + µ λ (t) • DF λ (u λ (t)) = 0 for a.e. t ě 0, -η λ (t) ∈ N C (u λ (t)) and µ λ (t) ∈ P R (Θ) for a.e. t ě 0, -η λ ∈ L 2 ([0, T ], E) and µ λ (•) • DF λ (u λ (•)) ∈ L 2 ([0, T ], E) for all T > 0, -µ λ ∈ L ∞ w * ([0, +∞[, M R (Θ)).
Our intention is to pass to the limit when λ goes to zero, to obtain a trajectory solution of (SD).

During all this section, we use the following notations for the cost functions: for all λ > 0 and each µ ∈ P R (Θ), note

f µ,λ := µ • F λ : E -→ R x -→ µ • F λ (x) = Θ F λ (x) dµ = Θ f θ,λ (x) dµ(θ). By Theorem 7.3.7, f µ,λ is a convex Fréchet differentiable function, whose gradient ∇f µ,λ (x) = µ • DF λ (x) is 1 λ -Lipschitz continuous. Similarly, define f µ := µ • F : E -→ R x -→ µ • F(x) = Θ F(x) dµ = Θ f θ (x) dµ(θ).
It is also a convex function, such that (f µ,λ ) λ>0 is pointwise converging to f µ from below.

Proposition 7.3.19. Let F : E -→ Y be convex and bounded from below, with E finitedimensional and Y separable. Then, when λ ↓ 0, we have (taking eventually a subnet) for all T > 0:

i) u λ • --→ u in C([0, T ], E), ii) ( uλ ) λ>0 is bounded and uλ w --→ u in L 2 ([0, +∞[, E), iii) (η λ ) λ>0 is bounded and η λ w --→ η in L 2 ([0, T ], E), with η(t) ∈ N C (u(t)) for a.e. t ě 0, iv) (µ λ ) λ>0 is bounded and µ λ w * --→ µ in L ∞ w * ([0, T ], M R (Θ))
, with µ(t) ∈ P R (Θ) for a.e. t ě 0.

Proof. We start with energies estimations for the trajectory. For all λ > 0 and µ ∈ P R (Θ), f µ,λ is differentiable so for a.e. t ě 0,

(7.60) d dt (f µ,λ • u λ )(t) = ∇f µ,λ (u λ (t)), uλ (t) = d C f µ,λ (u λ (t); uλ (t)). Using uλ (t) = -(N C (u λ (t)) + ∂ C F λ (u λ (t))) 0 ,
T 0 uλ (t) 2 dt ď f µ,λ (u 0 ) -f µ,λ (u(T )).
Use now the fact that f µ,λ ď f µ on E, and that F is bounded from below on E together with Theorem 7.3.7 v) to obtain:

(7.63) T 0 uλ (t) 2 dt ď f µ (u 0 ) -m.
Since the estimation in (7.63) is independent of λ and T , it follows that uλ ∈ L 2 ([0, +∞[, E) with

(7.64) sup λ>0 uλ L 2 ([0,+∞[,E) < +∞.
Consider, for all λ > 0 and all T > 0, the trajectory u λ ∈ C([0, T ], E). To prove the existence of a converging subnet of this family, we will invoke Ascoli's Theorem. First recall that u λ is absolutely continuous. So, for all t ě 0, (7.65) u λ (t) = u λ (0) + t 0 uλ (s) ds, which implies, after using the Cauchy-Schwarz inequality in L 2 ([0, t], R),

(7.66) u λ (t) ď u λ (0) + √ t t 0 uλ (s) 2 ds.
As a direct consequence of (7.66) and (7.64), we have

(7.67) sup λ>0 u λ (t) ď u λ (0) + √ T sup λ>0 uλ L 2 ([0,+∞[,E) < +∞.
So {u λ } λ>0 is pointwise bounded, hence pointwise relatively compact in C([0, T ], E) (recall that E has finite dimension). Now we verify that {u λ } λ>0 is equicontinuous in C([0, T ], E). For this, fix λ > 0 and consider 0 ď s < t ď T , and use the absolute continuity of u λ to write

u λ (t) -u λ (s) = t s uλ (τ ) dτ.
Using the Cauchy-Schwarz inequality as before, together with (7.64), we obtain (7.68)

u λ (t) -u λ (s) ď t s uλ (τ ) dτ ď √ t -s t s uλ (τ ) 2 dτ ď √ t -s sup λ>0 uλ L 2 ([0,+∞[,E) .
The latter means that {u λ } λ>0 is uniformly Hölder continuous, hence equicontinuous. So we can apply Ascoli's Theorem, and obtain that {u λ } λ>0 is relatively compact in C([0, T ], E), so by taking eventually a subnet, we obtain that there exists some u : [0, +∞[-→ E such that u λ converges uniformly to u on [0, T ] for each T > 0.

We prove now the weak convergence of uλ to u in L 2 ([0, T ], E). We already now, from (7.64), that { uλ } λ>0 is relatively compact in L 2 ([0, T ], E). Let v ∈ L 2 ([0, T ], E) be any limit point of { uλ } λ>0 when λ ↓ 0, then use [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]Proposition A.6] to verify that v(t) = u(t) for a.e. t ∈]0, T [.

We focus now on item iii). We have, for all λ > 0 and a.e. We end now the proof with item iv). We know that µ λ (t) M R (Θ) = 1 for all λ > 0 and a.e. t ě 0, so µ λ L ∞ w * ([0,T ],M R (Θ)) = 1 for all λ > 0. This means that (µ λ ) λ>0 is bounded in L ∞ w * ([0, T ], M R (Θ)), hence relatively weakly * compact. So, taking eventually a subnet, µ λ weakly * converges to some µ ∈ L ∞ w * ([0, T ], M R (Θ)). It remains to verify that µ(t) ∈ P R (Θ) for a.e. t ě 0.

Consider any measurable I ⊂ [0, T ], and define 1 I : [0, T ] -→ C(Θ) as follows: if t ∈ I, then 1 I (t) = 1 ∈ C(Θ), else 1 I (t) = 0 i.e. the null function. This function is well-defined and measurable, and we easily see that it is Bochner integrable, i.e.

1 I ∈ L 1 ([0, T ], C(Θ)). From µ λ w * --→ λ→0 µ, it follows 4 : (7.73) µ λ , 1 I L ∞ w * ×L 1 R --→ λ→0 µ, 1 I L ∞ w * ×L 1
On one hand, we know that for all λ > 0,

µ λ , 1 I L ∞ w * ×L 1 = T 0 µ λ (t), 1 I (t) M R (Θ)×C(Θ) dt = I µ λ (t), 1 M R (Θ)×C(Θ) dt = I 1 dt = |I|.
Using (7.73), we deduce that µ, 1 I L ∞ w * ×L 1 = |I|. On the other hand, Recall now this alternative characterization of P R (Θ) (see (7.32)):

(7.74) µ, 1 I L ∞ w * ×L 1 = I µ(t), 1 M R (Θ)×C(Θ) dt, 4 •, • L ∞ w * ×L 1 denotes the duality product between L ∞ w * ([0, T ], M R (Θ)) and L 1 ([0, T ], C(Θ)).
P R (Θ) = {µ ∈ M R (Θ) | µ, 1 M R (Θ)×C(Θ) = 1 and µ M R (Θ) ď 1}.
Since we already proved (7.75), it remains to verify that µ(t) M R (Θ) ď 1 for a.e. t ∈ [0, T ].

From the weak * convergence of µ λ to µ, and the weakly * lower semi-continuity

(7.76) µ L ∞ w * ď lim inf λ→0 µ λ L ∞ w * .
Moreover we know that µ λ (t) ∈ P R (Θ) for all λ > 0 and a.e. t ě 0, which implies that µ λ L ∞ w * = 1 for all λ > 0. In the light of (7.76), we deduce that µ L ∞ w * ď 1, which means that µ(t) M R (Θ) ď 1 for a.e. t ∈ [0, T ], and ends the proof.

Proposition 7.3.20. Let F : E -→ Y be convex and bounded from below, with E finitedimensional and Y separable. Then, when λ ↓ 0, we have (taking eventually a subnet)

F λ •u λ -→ F • u in C([0, T ], C(Θ)).
Proof. We start by proving that {F λ • u} λ∈]0,1[ is relatively compact in C([0, T ], C(Θ)). For this we aim to use Ascoli's Theorem, for which we need to verify the hypotheses. Given a fixed t ∈ [0, T ], and using Lemma 7.3.16, we obtain the relative compactness of {F λ (u(t))} λ∈]0,1[ in C(Θ). Now we prove the equicontinuity of {F λ •u} λ∈]0,1[ in C([0, T ], C(Θ)). Take 0 ď s < t ď T , and any λ > 0. Using the fact that u is absolutely continuous, and F λ is differentiable :

(7.77) F λ (u(t)) -F λ (u(s)) ď t s d dt F λ • u(τ ) dτ ď t s DF λ (u λ (τ )) uλ (τ ) dτ.
From the estimation we obtained in (7.71), and the Cauchy-Schwarz inequality in L 2 ([0, T ], R), we obtain

(7.78) F λ (u(t)) -F λ (u(s)) ď M √ t -s u L 2 ([0,T ],E) .
We deduce from the latter that {F λ • u} λ∈]0,1[ is uniformly Hölder continuous, hence equicontinuous. So we can apply Ascoli's Theorem, and obtain that

{F λ • u} λ∈]0,1[ is relatively compact in C([0, T ], C(Θ)). Now we pass to the uniform convergence of F λ • u λ to F • u in C([0, T ], C(Θ)).
For this, take any t ∈ [0, T ], λ > 0, and use the triangular inequality to write (7.79)

F λ • u λ (t) -F • u(t) ď F λ • u λ (t) -F λ • u(t) + F λ • u(t) -F • u(t) .
On one hand, the first term of the right member in (7.79) can be bounded from above by using the mean value theorem:

(7.80) F λ • u λ (t) -F λ • u(t) ď sup c∈[u λ (t),u(t)] DF λ (c) u λ (t) -u(t) .
Using the same argument as in (7.71), we obtain some M > 0 such that for all λ ∼ 0 and any t ∈ [0, T ], (7.81)

F λ • u λ (t) -F λ • u(t) ď M u λ (t) -u(t) .
Since u λ converges uniformly to u, we deduce from the latter that F λ • u λ -F λ • u converges uniformly to zero. On the other hand, we know from Theorem 7.3.7 that F λ • u is pointwise convergent to F • u. Moreover we shown in the first part of this proof that {F λ • u} λ∈]0,1[ is relatively compact in C([0, T ], C(Θ)). These two properties entails the uniform convergence of F λ • u to F • u, which means that the second term of the right member of (7.79) converges uniformly to zero when λ → 0. So we proved the uniform convergence of

F λ • u λ to F • u in C([0, T ], C(Θ)). Proposition
µ λ (t) • F λ (w(t)) -µ λ (t) • F λ (u λ (t)) -∇(µ λ (t) • F λ )(u λ (t)), w(t) -u λ (t) ě 0.
Observe that by definition of µ λ , we have

(7.83) ∇(µ λ (t) • F λ )(u λ (t)) = µ λ (t) • DF λ (u λ (t)) = -uλ (t) -η λ (t).
Using (7.83) together with the fact that F λ ĺ F on E (recall Theorem 7.3.7), we obtain from (7.82) that

(7.84) µ λ (t) • F(w(t)) -µ λ (t) • F λ (u λ (t)) --uλ (t) -η λ (t), w(t) -u λ (t) ě 0.
This expression is integrable on [0, T ], and we obtain after integration:

T 0 µ λ (t), F(w(t)) M R (Θ)×C(Θ) dt - T 0 µ λ (t), F λ (u λ (t)) M R (Θ)×C(Θ) dt - T 0 -uλ (t) -η λ (t), w(t) -u λ (t) E dt ě 0
In the latter, we recognize in the two first terms a duality product between L 1 ([0, T ], C(Θ)) and E), which can be seen as a subspace of L ∞ ([0, T ], E) * . Hence this last inequality can be rewritten as:

L ∞ w * ([0, T ], M R (Θ)). The last term involves u λ ∈ C([0, T ], E) ⊂ L ∞ ([0, T ], E) and uλ + η λ ∈ L 2 ([0, T ], E) ⊂ L 1 ([0, T ],
(7.85) µ λ , F • w L ∞ w * ×L 1 -µ λ , F λ • u λ L ∞ w * ×L 1 --uλ -η λ , w -u λ L 1 ×L ∞ ě 0.
We can now pass to the limit in (7.85) when λ goes to zero, using Propositions 7.3.19 and 7.3.20 to obtain for all w ∈ L ∞ ([0, T ], E):

(7.86) µ, F • w L ∞ w * ×L 1 -µ, F • u L ∞ w * ×L 1 --u -η, w -u L 1 ×L ∞ ě 0.
We want to interpret the equation above as a subdifferential inequality. Consider for this the integrand

Φ : L ∞ ([0, T ], E) -→ R w -→ T 0 φ(t, w(t)) dt,
where φ is defined as follows

φ : [0, T ] × E -→ R (t, x) -→ φ t (x) := µ(t), F(x) M R (Θ)×C(Θ) .
In other words, Φ(w) = µ, F • w L ∞ w * ×L 1 . The convexity of F implies the one of Φ, and (7.86) expresses the fact thatuη ∈ ∂Φ(u). Using a celebrated result of Rockafellar ([286, Theorem 4]), we obtain for a.e. t ∈ [0, T ] that (7.87) u(t)η(t) ∈ ∂φ t (u(t)).

But for each t ∈ [0, T ], φ t = µ(t) • F, so for all x ∈ E,

(7.88) ∂φ t (x) = ∂(µ(t) • F)(x) ⊂ ∂ C F(x). Since ∂ C F = ∂ C F
(u(t)) + ∂ C F (u(t))) 0 = 0.
Proof. To prove the lazyness of the trajectory u, we will use the one of each u λ and pass to the limit when λ ↓ 0. Indeed, we know for all λ > 0 that for a.e. t ě 0,

(7.89) uλ (t) + (N C (u λ (t)) + ∂ C F (u λ (t))) 0 = 0. Using the variational characterization of (N C (u λ (t)) + ∂ C F (u λ (t))) 0 as the projection of the origin onto N C (u λ (t)) + ∂ C F (u λ (t)),
∈ P R (Θ), uλ (t) 2 + d dt (µ • F λ • u λ )(t) ď 0.
Our aim is to pass to the limit in (7.92) to prove

(7.93) ∀µ ∈ P R (Θ), u(t) 2 + d dt (µ • F • u)(t) ď 0.
Indeed, this is a sufficient condition for the trajectory u to be lazy, recall Proposition 6.1.15. Take any ψ ∈ D(]0, T [, R), the space of infinitely differentiable functions with support uncluded in ]0, T [, which is dense in L 2 ([0, T ], R) (see [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]Appendices]). Consider then the corresponding linear integrand

Ψ : L 2 ([0, T ], E) -→ R w -→ T 0 ψ(t) w(t) 2 dt.
It is easy to see that Ψ is strongly continuous, hence weakly lower semi-continuous. As direct consequence of uλ Now, use an integration by parts together with the fact that ψ(0) = ψ(T ) = 0 to write for all µ ∈ P R (Θ) (7.95)

T 0 ψ(t)(µ • F • u) ′ (t) dt = - T 0 ψ ′ (t)(µ • F • u)(t) dt. Because of Proposition 7.3.20, µ • F λ • u λ converges uniformly to µ • F • u in C([0, T ], R), hence from (7.95) we deduce (7.96) T 0 ψ(t)(µ • F • u) ′ (t) dt = -lim λ→0 T 0 ψ ′ (t)(µ • F λ • u λ )(t) dt.
Doing again an integration by parts, we obtain (7.97)

T 0 ψ(t)(µ • F • u) ′ (t) dt = lim λ→0 T 0 ψ(t)(µ • F λ • u λ ) ′ (t) dt.
Combines now (7.94) and (7.97) to find

T 0 ψ(t) u(t) 2 + d dt (µ • F • u)(t) dt ď lim inf λ→0 T 0 ψ(t) uλ (t) 2 + d dt (µ • F λ • u λ )(t) dt.
If we suppose that ψ ∈ D + (]0, T [, R), i.e. that ψ takes positive values, we can deduce from (7.92) that (7.98)

T 0 ψ(t) u(t) 2 + d dt (µ • F • u)(t) dt ď 0.
By using the density of D

+ (]0, T [, R) in L 2 + (]0, T [, R), which is a closed convex self-dual cone, we obtain that u(•) 2 + d dt (µ • F • u) ∈ -L 2 + (]0, T [, R
). This proves (7.93), and ends the proof with Proposition 6.1.15.

Comments

Remark 7.4.1 (On the necessity to work in general Banach spaces). When considering Chapters 5 and 6, it might seem superfluous to work with a function F taking its values in a general ordered Banach space Y. Indeed, we make the assumption that the cone K inducing the order in Y must have a nonempty interior, which reduces the "useful" spaces to work in to Y = L ∞ ([0, 1], R p ) or Y = C([0, 1]). When one sees the technicalities induced by this infinite-dimensional Banach setting, we could reasonably wonder if it is worth it.

In fact, even if Y = R m , as soon as K is not polyhedral, we cannot apply the techniques of Section 7.2 to prove the existence of trajectories. We need in the Section 7.3 to introduce an auxiliary Banach space of continuous functions on the base Θ, C(Θ), which is in general a nonreflexive Banach space.

(x) = (Id + λ∂ C F ) -1 (x) = θ∈Θ prox λf θ (x).
It is a known fact (see [START_REF] Bauschke | Convex analysis and monotone operator theory[END_REF]Chapter 23]) that prox λf θ (x) = x -λ∇f θ,λ (x). Then, we can write from Proposition 7.3.12 that

∂ C F λ (x) = co * θ∈Θ ∇f θ,λ (x) = 1 λ x -co * θ∈Θ prox λf θ (x) .
In other words,

∂ C F λ = 1 λ (Id -co * PROX λF ) ,
that we can interpret as the fact that ∂ C F λ is a formal Yosida transform of ∂ C F .

Chapter 8

A continuous inertial approach to multi-objective optimization

Let H be a Hilbert space, and consider the unconstrained multi-objective problem

(MOP) MIN x∈H F (x)
where

F = (f 1 , ..., f m ) : H -→ (R m , R m +
) is a continuously differentiable function. In Chapter 5, we introduced the steepest descent vector field for F :

s : H -→ H (8.1) x -→ s(x) := -(co∇f i (x)) 0
where co∇f i (u) 0 denotes the element of minimal norm of the convex compact set co∇f i (u). The vector s(u) is called the steepest descent direction at x, and simply reduces to -∇f (u) if m = 1. It enjoys the following nice properties, which extends known facts about -∇f (u) in the mono-criteria case (recall Section 5.2): i) s(u) = 0 if and only if u is Pareto critical.

ii) s(u) is a common descent direction at u for all the objective functions. More exactly, (8.2) ∀i = {1, ..., m}, ∇f i (u), s(u) ďs(u) 2 .

iii) It is the steepest common descent direction, in the sense that

(8.3) s(u) s(u) = argmin d =1 max i∈{1,...,m} ∇f i (u), d whenever s(u) = 0.
In Chapter 6, we introduced and studied the steepest descent dynamic (SD) to solve problem (MOP). It is a dynamic governed by the steepest descent vector field s, which reduces in this multi-objective case to (SD) u(t) + (co ∇f i (u(t))) 0 = 0.

The main feature of (SD) is that, when the objective functions f i are convex, all its bounded solutions weakly converge to weak Pareto point of (MOP). When m = 1, this dynamic corresponds exactly to the classic steepest descent dynamic u(t) + ∇f (u(t)) = 0.

It is known that its trajectories have poor convergence rates, when t → +∞. Thus, the (SD) dynamic for multi-objective optimization problems shares the same slow behaviour, which is characteristic of first-order methods. This is a clear issue in the multi-objective setting. Indeed, while in scalar optimization we often look for one minimum, in multi-objective problems we aim to have a description of the set of Pareto efficient points. Hence, we need to perform in parallel a high number of trajectories, and it would be better that the trajectories converge quickly.

To circumvent this problem, we propose to introduce an inertial term in our dynamic. Inertial continuous dynamics for scalar optimization have been studied since Polyack [START_REF] Polyack | Some methods of speeding up the convergence of iterative methods[END_REF], and Attouch-Goudou-Redont [START_REF] Attouch | The heavy ball with friction method I, the continuous dynamical system[END_REF] introduced the so-called Heavy Ball with Friction dynamic:

(HBF) ü(t) + γ u(t) + ∇f (u(t)) = 0.

This system has a clear mechanical interpretation. The term ü(t) is the acceleration of the physical point u(t), on which acts the sum of two forces: the friction -γ u(t) and the vector field ∇f (u(t)). Just like a heavy ball sliding down the graph of f , due to the viscous friction effect, each trajectory tends to stabilize at a local minimum of f . This (HBF) dynamic has been extended by Cabot-Engler-Gaddat [START_REF] Cabot | On the long time behavior of second order differential equations with asymptotically small dissipation[END_REF], which introduced a variable damping coefficient γ(•). This variable viscosity is of importance, in the light of the works of Su-Boyd-Candès [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: Theory and Insights[END_REF] and Attouch-Chbani-Peypouquet-Redont [START_REF] Attouch | Fast convergence of an inertial dynamical system with asymptotic vanishing damping[END_REF]. Indeed they show that, by taking γ(t) = α t , we recover after discretization in time the inertial algorithms of Nesterov [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate O(1/k 2 )[END_REF] and Chambolle-Dossal [START_REF] Chambolle | On the convergence of the iterates of Fista[END_REF].

Our aim in this chapter is to provide a first study of an inertial gradient-based dynamical system for multi-objective optimization. To our knowledge, the combination of both inertial and multi-objective aspects has not been considered before. It is a second-order in time differential equation, which generalizes the (HBF) dynamic to the multi-objective setting, replacing the gradient of f by the ordered subdifferential of F . It is called the Inertial Steepest Descent with Friction dynamic, and is defined as follows:

(ISDF) ü(t) + γ u(t) + co∇f i (u(t)) 0 = 0, where the viscous damping coefficient γ is a fixed positive parameter. Clearly, when there is only one objective function, we recover the (HBF) dynamic.

Thus our program consists in studying the Inertial Steepest Descent with Friction dynamic, by combining the technics of the previous chapters and the ones in [START_REF] Attouch | The heavy ball with friction method I, the continuous dynamical system[END_REF]. After introducing the Inertial Steepest Descent with Friction dynamic, we investigate in Section 8.1 the existence of solution trajectories for (ISDF) in finite dimensions. In Section 8.2, we study the properties of the trajectories generated by (ISDF). Under a convexity assumption on the objective functions, we show that the bounded trajectories converge to weak Pareto points of the problem. We recover in that case the convergence result concerning (HBF) in [START_REF] Attouch | The heavy ball with friction method I, the continuous dynamical system[END_REF]. Of course, due to the effects of inertia, (ISDF) is not a descent dynamic, i.e. the values of the cost functions may not decrease over time. But we show that, with an appropriate choice of the initial velocity, the cost values are improved along the trajectory relative to the starting point (see Proposition 8.2.5). This is essential for one which wants to implement a numerical method with restarting, see [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: Theory and Insights[END_REF] for instance. In Section 8.3, we illustrate a discretized version of this dynamic, and compare its convergence rate with the (SD) dynamic. The proper analysis of the corresponding algorithm is left for a future work, see Section 8.4.

Existence of trajectories

In this section, we question the existence of solutions for the Cauchy problem associated to (ISDF). Let t 0 ∈ R, T ∈]t 0 , +∞], and (u 0 , u0 ) ∈ H 2 . We say that u : [t 0 , T [-→ H is a solution of (ISDF) if u is continuous on [t 0 , T [, of class C 2 on ]t 0 , T [, and satisfies ü(t) = -γ u(t) + s(u(t)) for all t ∈]t 0 , T [, (IMOG) u(t 0 ) = u 0 , u(t 0 ) = u0 .

Here, contrary to Chapter 7, we consider solutions in the classic sense. This is because we restrict our analysis to the smooth case.

In view to apply an existence theorem for the dynamical system (IMOG), the key point is the regularity of the steepest descent vector field s. As a particular case of Proposition 7.1.2, we have: Proposition 8.1.1. Suppose that the gradients ∇f i : H → H are Lipschitz continuous on bounded sets. Then s is 1 2 -Holder continuous on bounded sets.

We will face for the (ISDF) dynamic the same regularity problem that with (SD). Thus, here also our existence proof will rely on Peano's existence result:

(8.7) u(t) = v(t), v(t) = -γv(t) + s(u(t)) for all t ∈ [t 0 , t 0 + T [, with u(t 0 ) = u 0 , v(t 0 ) = u0 .
Since x is of class C 1 , we deduce that it is also the case for u and v. But from u(t) = v(t), we can see that u is of class C 2 and satisfies (8.5).

Remark 8.1.4. For this result we use Peano's theorem, which asks the space to be finite dimensional. In fact, Peano's theorem can be stated in the Banach space setting, if one asks the vector field involved to be compact. We recall that an operator φ : H -→ H is said to be compact whenever it is continuous and maps bounded sets to relatively compact sets. Observe that if the gradients ∇f i are all compact operators, then s is also compact. Hence, one might want to apply Peano's result in this context. Nevertheless, by reducing (ISDF) to a first-order ODE, we do not deal directly with s but with (u, v) → (v, -γv + s(u)). And it can be easily proved that if s is compact, then v → v is also compact, which would mean that H has finite dimension.

We can now state our main existence result. From the growth condition (8.9) we deduce an upper bound for φ(x(t)) . Indeed, for all x = (u, v) ∈ H 2 , φ(x) ď ( v + s(u)γv ) using the equivalence between ℓ 1 and ℓ 2 norms ď (1 + γ) v + c(1 + u ) using the triangle inequality with (8.9)

ď c 2 (1 + x ) with c 2 := √ 2 max{c; 1 + γ}.
Using the triangle inequality with c 3 := c 2 (1 + x(t 0 ) ), it follows for all t ∈ [t 0 , T [ that (8.12) φ(x(t)) ď c 3 (1 + h(t)).

Combining (8.11) and (8.12), we obtain

(8.13) d dt 1 2 h 2 (t) ď c 3 h(t)(1 + h(t)) for all t ∈ [t 0 , T [.
We will now conclude by using a Gronwall-type argument. Consider an arbitrary ε ∈ ]0, Tt 0 [. After integration of (8.13) on [t 0 , Tε], and using h(t 0 ) = 0, we obtain

(8.14) 1 2 h 2 (t) ď t t 0 c 3 (1 + h(s))h(s) ds for all t ∈ [t 0 , T -ε]. Since h is continuous on [0, T -ε], the function g : t ∈ [0, T -ε] → c 3 (1+h(t)) is in L 1 ([0, T -ε], R).
Hence we can apply Lemma C.0.

(φ • u)(t) = d dt (∇φ • u)(t), u(t) + ∇φ • u(t), ü(t) .
Moreover, we have in the second member above (using the Cauchy-Schwarz inequality and the Lipschitz property of ∇φ) :

d dt (∇φ • u)(t), u(t) = lim h→0 1 h ∇φ • u(t + h) -∇φ • u(t), u(t) ď lim h→0 1 |h| ∇φ • u(t + h) -∇φ • u(t) u(t) ď lim h→0 L 1 |h| u(t + h) -u(t) u(t) = L u(t) 2
where L := Lip(∇φ; u(I)).

Let us prove now the dissipativity of our dynamic : Proposition 8.2.2 (Dissipative property). Let u : [t 0 , T [-→ H be a solution of (ISDF). For all i ∈ {1, ..., m}, define for all t ∈ [t 0 , T [ :

(8.20) E i (t) := (f i • u)(t) + 1 γ (f i • u) ′ (t) + u(t) 2 .
Then, for a.e.

t ∈ [t 0 , T [, if L i := Lip(∇f i ; u([t 0 , T [)) < +∞, we have (8.21) E ′ i (t) ď - 1 γ ü(t) 2 - 1 γ γ 2 -L i u(t) 2
Proof. The dissipative property is a direct consequence of the variational characterisation of the projection of 0 over co ∇f i (u(t)) in (ISDF). Indeed, for a.e.

i ; u([t 0 , T [)) < +∞ to obtain 1 γ d 2 dt 2 (f i • u)(t) -L i u(t) 2 + d dt (f i • u) + u 2 (t) ď - 1 γ ü(t) 2 -γ u(t) 2 ,
which ends the proof.

Proposition 8.2.2 suggests that we need an hypothesis on the parameters to ensure the dissipative property :

HP i γ 2 > L i where L i := Lip(∇f i ; u([t 0 , T [)).
If HP i holds for all i ∈ {1, ..., q} then we just write HP. This hypothesis asks the friction parameter γ to be large enough, in order to limit the inertial effects, which induce oscillations (see Example 8.3.1). The hypothesis asks also implicitly the gradients ∇f i to be Lipschitz over the trajectory (since γ ∈ R). Note that this last property holds whenever u(•) is bounded, since the gradients are assumed to be Lipschitz continuous on bounded sets (see Corollary 8.2.4). See Section 8.4 for a discussion on this hypothesis.

As a direct consequence of the dissipative nature of the system, we obtain that the values (f i (u(t)) are bounded from above by max{f i (u 0 ); E i (t 0 )} : Corollary 8.2.3. (Upper bound for the values) Let u : [t 0 , T [-→ H be a solution of (ISDF), such that HP i holds. Then, for all i ∈ {1, ..., m} and t ě t 0 , we have the following upper bounds :

(8.24) f i (u(t)) ď E i (t 0 ) + (f i (u 0 ) -E i (t 0 ))e -γ(t-t 0 ) .
Proof. It is a trivial consequence of the monotonicity property of E i obtained in Proposition 8.2.2. Indeed, we obtain for all t ∈ [t 0 , +∞[ :

(8.25) 1 γ (f i • u) ′ (t) ď E i (t 0 ) -(f i • u)(t).
The conclusion follows Gronwall's Lemma (see Lemma C.0.3), applied to t

→ (f i • u)(t) - E i (t 0 ).
This upper bound for the values has two interesting consequences. The first one is immediate, and gives a useful sufficient condition for the trajectory u(•) to be bounded: Corollary 8.2.4. Suppose that there exists i ∈ {1, ..., m} such that f i is coercive, and globally L i -Lipschitz continuous, with γ 2 > L i . Then any trajectory of (ISDF) is bounded.

The second consequence is that it tells us how to enforce the interesting property f i (u(•)) ď f i (u 0 ). Indeed, we know that this dynamic is not a descent method for the functions because of the inertial effects which can create damped oscillations. But at least, one can choose appropriately the initial velocity so that each point on the trajectory is better than the initial one.

Corollary 8.2.5. Suppose that HP i holds for some i ∈ {1, ..., q}. For all u 0 ∈ H, if u0 ∈ H is chosen to satisfy (8.26) ∇f i (u 0 ), u0 ď -γ u0 2 , then f i (u(t)) ď f i (u 0 ) for all t ě t 0 . In particular, for all λ ∈ [0, s(u 0 ) 2 + ∇f i (u 0 ), s(u 0 ) ď 0.

So we deduce that (8.26) holds for λs(u 0 ) .

Energy estimations

We will now use the dissipative property of the system to deduce energy estimations for a global solution of (ISDF).

(t) ↓ E ∞ i ∈ R whenever t → +∞. (ii) u ∈ L ∞ ([t 0 , +∞[, H) ∩ L 2 ([t 0 , +∞[, H) and lim t→+∞ u(t) = 0. (iii) ü ∈ L ∞ ([t 0 , +∞[, H) ∩ L 2 ([t 0 , +∞[, H) and liminfess t→+∞ ü(t) = 0. (iv) For all i ∈ {1, ..., m}, (f i • u) ′ ∈ L ∞ ([t 0 , +∞[, R) and lim t→+∞ (f i • u) ′ (t) = 0. (v) For all i ∈ {1, ..., m}, (f i • u) ∈ L ∞ ([t 0 , +∞[, R) and lim t→+∞ (f i • u)(t) = E ∞ i .
(vi) For all i ∈ {1, ..., m}, there exists

θ i ∈ L ∞ ([t 0 , +∞[, R) such that for all t ∈ [t 0 , T [, mü(t) + γ u(t) + m i=1 θ i (t)∇f i (u(t)) = 0 with θ(t) ∈ ∆ m .
In particular, it follows that m i=1 θ H). Proof. We start by proving that u ∈ L ∞ ([t 0 , +∞[, H), from which the other results will follow easily. Take any i ∈ {1, ..., m}, and define c := inf

i (•)(f i • u) ′ ∈ L 1 ([t 0 , +∞[,
tět 0 f i (u(t))-E I (t 0 ) and M := max i∈{1,...,m} sup tět 0 ∇f i (u(t)) .
Given that the gradients ∇f i are Lipschitz continuous on bounded sets, we deduce (using the mean value theorem) that the functions f i are bounded on bounded sets. Since the trajectory is bounded, it follows that M and c are finite. In particular, it implies that ü + γ u ∈ L ∞ ([t 0 , +∞[, H), since, according to (ISDF), we have for a.e. t ě t 0 that -ü(t)γ u(t) ∈ co ∇f i (u(t)) which is bounded by M .

Using the monotonicity property of E i (see Proposition 8.2.2), we have for all t ě t 0 :

(8.28) 0 ě E i (t) -E i (t 0 ) ě u(t) 2 + 1 γ (f i • u) ′ (t) + c.
Using Cauchy-Schwarz inequality and the definition of M , one has

(8.29) (f i • u) ′ (t) = ∇f i (u(t)), u(t) ě -∇f i (u(t)) u(t) ě -M u(t) . If we note b = 1 γ M , we obtain (8.30) 0 ě u(t) 2 -b u(t) + c.
If we consider now the real polynomial X 2 -bX + c, we can see that it takes negative values on a compact interval, independent of t. Since u(t) lies therein, we conclude that u ∈ L ∞ ([t 0 , +∞[, H).

We can now derive the other properties, and we start with (i). The decreasing property of the energies E i (see Proposition 8.2.2) ensures the existence of a limit E ∞ i , taking eventually the value -∞. But now we can prove that for all i ∈ {1, ..., m}, E ∞ i ∈ R. Indeed, using the same inequality as in (8.29), (8.31)

E ∞ i = lim t→+∞ E i (t) ě inf tět 0 f i (u(t)) - 1 γ M u L ∞ ([t 0 ,+∞[,H) > -∞.
We now prove (iii). Since ü + γ u and u lie in L ∞ ([t 0 , +∞[, H), we directly obtain that ü ∈ L ∞ ([t 0 , +∞[, H). For the L 2 estimation, use Proposition 8.2.2 to obtain:

(8.32) 1 γ +∞ t 0 ü(t) 2 dt ď +∞ t 0 - d dt E i (t) dt = E i (t 0 ) -E ∞ i .
It follows that ü ∈ L 2 ([t 0 , +∞[, H), and then, liminfess t→+∞ ü(t) = 0.

Let us now to prove (ii). Using exactly the same argument as for ü, one obtains u ∈ L 2 ([t 0 , +∞[, H). Moreover, we know that u is Lipschitz continuous on [t 0 , +∞[ (since ü ∈ L ∞ ([t 0 , +∞[, H)), so it follows that lim t→+∞ u(t) = 0.

We continue with items (iv) and (v). From Cauchy-Schwarz inequality, |(f i • u) ′ (t)| ď M u(t) for all t ě t 0 . As a direct consequence of (ii), we deduce (

f i • u) ′ ∈ L ∞ ([t 0 , +∞[, H) and lim t→+∞ (f i • u) ′ (t) = 0. Then it follows directly from (i) that lim t→+∞ (f i • u)(t) = E ∞ i , and (f i • u) ∈ L ∞ ([t 0 , +∞[, H).
We end the proof with item (vi). It is clear from the definition of (ISDF) that for all t ě t 0 , there exists θ

(t) = (θ i (t)) ∈ ∆ m such that ü(t) + γ u(t) + m i=1 θ i (t)∇f i (u(t)) = 0. To get θ i ∈ L ∞ ([t 0 , +∞[, R)
, the whole point is to verify that it can be taken measurable. For this, we write θ(t) as a solution of the following optimality problem

θ(t) ∈ argmin θ∈S q j(t, θ), where j(t, θ) := m i=1 θ i ∇f i (u(t)) .
Since j is a Caratheodory function, we are guaranteed of the existence of a measurable selection θ : t → θ(t) ∈ argmin

i=1 θ i (t)(f i • u) ′ (t) = m i=1 θ i (t) ∇f i (u(t)), u(t) = -ü(t) -γ u(t), u(t)
where u, ü ∈ L 2 ([t 0 , +∞[, H). So, using the Cauchy-Schwarz inequality and the measurability of θ i , we get directly that m i=1 θ

i (•)(f i • u) ′ ∈ L 1 ([t 0 , +∞[, H).

Convergence of the trajectories

We present here the main result of this section. Under a convexity assumption, we show that the bounded trajectories of (ISDF) weakly converge to a solution. Recall that the HP hypothesis asks in particular the gradients ∇f i to be Lipschitz continuous in a neighborhood of the trajectory.

Theorem 8.2.8. Suppose that the objective functions f i are convex. Then any bounded trajectory of (ISDF) u : [t 0 , +∞[-→ H satisfying HP converges weakly to a weak Pareto optimum.

We sketch here the main points of the proof.

Examples and numerical results

Example 8.3.1. Consider two quadratic functions from R 2 to R, defined by f 1 (x, y) = 1 2 (x + 1) 2 + 1 2 y 2 and f 2 (x, y) = 1 2 (x -1) 2 + 1 2 y 2 . The corresponding Pareto set is [-1, +1] × {0} and the steepest descent vector field is given by : .1 shows some trajectories of the (ISDF) dynamic, with the steepest descent vector field plotted in background. We took the Cauchy data (u 0 , u0 ) randomly. Here the trajectories are computed exactly, since in this simple example (ISDF) can be solved explicitly. We can observe the following: all the trajectories converge to a Pareto point, the dynamic is clearly not a descent method, and can be highly oscillating whenever the friction parameter is too close to zero. where γ(t) should be able to tend to zero. In the mono-criteria case, this dynamic has been studied only very recently [START_REF] Cabot | On the long time behavior of second order differential equations with asymptotically small dissipation[END_REF][START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: Theory and Insights[END_REF][START_REF] Attouch | Fast convergence of an inertial dynamical system with asymptotic vanishing damping[END_REF]. The analysis of this multi-objective generalization might be very delicate, because of the assumption we need on the damping parameter γ 2 > L.

The idea is to develop, in a second time, fast converging Nesterov-like algorithms for multiobjective problems. They would write as follows:

y k = x k + λs(x k ), x k+1 = y k + β n (y k -y k-1 )
where λ > 0 is a stepsize, and β n ∈ [0, 1[ is an inertial parameter converging to 1. To guarantee the fast convergence of the method, β n shall be finely tuned, look for instance in the works of Nesterov [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate O(1/k 2 )[END_REF] and Beck-Teboulle [START_REF] Beck | A fast iterative shrinkage-thresholding algorithm for linear inverse problems[END_REF].

Example 8.3.4. Let F = (f 1 , f 2 ) : R n -→ R 2 be the function defined for all x = (x 1 , ..., x n ) by:

f 1 (x) = n i=1
x 2 i -10 cos(2πx i ) + 10 .

It is the function considered in Example 6.3.5, in which we removed the square roots, so that F can be smooth. We compare here three numerical methods:

• The first order steepest descent, used in Examples 6.3.4 and 6.3.5.

• The Euler discretization of the (ISDF) dynamic, used in Example 8.3.2.

• A FISTA-like version of the steepest descent, defined as follows:

y k = x k-1 + τ s(x k )
x k = y k + .

We take n = 10, τ = 0.5, γ = 1, and run in parallel these methods for 1000 iterations. The result is presented in Figure 8.3, in which we measure on k ∈ {1, . . . , 1000} the relative decay max f i (x k )-f i (x 1000 ) f i (x 0 )-f i (x 1000 ) , i ∈ {1, 2} .

Figure 8.3: Convergence rates for the steepest descent method (dotted curve), the discretized (ISDF) (continuous curve) and its improvement using FISTA-like decreasing friction (thick cirve).

Comments and perspectives

Remark 8.4.1 (On the hypothesis on the friction parameter). To ensure the convergence of the trajectories of (ISDF), we assume in Section 8.2 the relation γ2 > L, where γ is the damping parameter and L is a Lipschitz constant for the gradients of the objective functions {f 1 , ..., f m }.

When we reduce to the mono-objective case (m = 1), we recover the (HBF) dynamic (HBF) ü(t) + γ u(t) + ∇f (u(t)) = 0 for which such assumption is not needed to guarantee the convergence of the trajectories, see [START_REF] Attouch | The heavy ball with friction method I, the continuous dynamical system[END_REF]. Thus, one might think that this hypothesis HP is not necessary, and is just a consequence of a 'bad' choice of Lyapunov function in Proposition 8.2.2. In addition, the analysis of (HBF) relies on the Lyapunov function

E(t) = f (u(t)) + u(t) 2 ,
while our energy functions involve in addition the derivatives of the f i 's:

E i (t) = f i (u(t)) + 1 γ ∇f i (u(t)), u(t) + u(t) 2 .
Thus, the analysis of (ISDF) and (HBF) are of different nature.

Recently, Attouch and Maingé [START_REF] Attouch | Asymptotic behavior of second-order dissipative evolution equations combining potential with non-potential effects[END_REF] studied the following extension of (HBF): ü(t) + γ u(t) + ∇f (u(t)) + Au(t) = 0, where f : H -→ R is a convex differentiable function, and A : H -→ H is a β-cocoercive1 maximal monotone operator. The solutions of this dynamic are proved to weakly converge, when t → +∞, under the assumption γ 2 > 1 β . As underlined by the authors, this assumption only involves the cocoercivity of the nonpotential part of the operator ∇f + A which governs the dynamic. Thus, the parallel can be done with our dynamic (ISDF), which is governed by an operator s : H -→ H which does not derive from a potential (see Example 6.1.12). Of course, -s is not monotone, and it is not clear whether it is 1 L -cocoercive, with L := max i∈{1,...,m}

Lip(∇f i , H).

But the similarities between those dynamics is striking.

Remark 8.4.2 (Some things left behind). In this chapter, we did not take into account a constraint C for (MOP). This is because the analysis of a nonsmooth operator N C is really difficult once we introduce inertia into the dynamic. From a mechanical point of view, in this second-order dynamic (ISDF), when a trajectory reaches the boundary of the constraint, there is a shock, which is hard to manage. Nevertheless, we still can penalize the constraint with a smooth barrier function φ, and solve the relaxed problem (MOP) MIN x∈H (f 1 (x) + φ(x), ..., f m (x) + φ(x)).

The corresponding (ISDF) dynamic would be (ISDF) ü(t) + γ u(t) + (∇φ(u(t)) + co ∇f i (u(t))) 0 .

For more details on this approach, see [START_REF] Alvarez | A second-order gradient-like dissipative dynamical system with Hessian-driven damping: application to optimization and mechanics[END_REF].

An other missing point in this chapter, is the quantitative analysis of the trajectories around their limit point. We think that it will be worth it to study the convergence rate of this dynamic whenever we will be able to let the friction parameter to tend to zero. Proposition A.1.4. Let (X, τ ) be a topological vector space, and f : X -→ R be a convex continuous function. If there exists x ∈ X such that f (x) < 0, then

cl τ [f < 0] = [f ď 0] and int τ [f ď 0] = [f < 0].
Proof. This proof is inspired by the one of Hiriart-Urrity and Lemaréchal in [START_REF] Hiriart-Urruty | Convex analysis and minimization algorithms I: Fundamentals[END_REF]Proposition VI.1.33], where this result is stated with X = R n .

Let x ∈ cl τ [f < 0], i.e. there exists a net (x α ) α∈A in [f < 0] such that x α τ --→ α∈A

x. Since f (x α ) < 0 for all α ∈ A, and using the lower semi-continuity of f , we obtain f (x) ď lim inf α∈A f (x α ) ď 0, that is, x ∈ [f ď 0]. Take now any x ∈ [f ď 0], and define for all n ∈ N:

x n := 1 n x + 1 - 1 n x.
We can rewrite the latter as

x n -x = 1 n (x -x),
where the right member goes to zero when n → +∞. Hence, x n converges to x. Moreover, using the definition of x n together with the convexity of f and the hypothesis f (x) < 0 gives Proposition A.1.5. If F : X -→ Y is Gateaux differentiable at x ∈ X, then for all d ∈ X, the application φ : t ∈ R -→ F (x + td) ∈ Y is continuous and derivable at zero. In particular, φ ′ (0) = DF (x; d).

f (x n ) ď 1 n f (x) + 1 - 1 n f (x) ď 1 n f (x) < 0.
Proof. Let us show that φ is right derivable at zero, using the Gateaux differentiability of F at x: Since -DF (x; -d) = DF (x; d) by linearity of DF (x), we have that φ is derivable at zero, and its derivative is φ ′ (0) = DF (x; d). In particular, φ is continuous at zero.

We immediately derive the following corollary, just by observing that the derivability at t ∈ [0, 1] of t → F (x + td) is exactly the same than the derivability at zero of t → F (x + td + td). Proof. Suppose first that F is strictly Gateaux differentiable at x and fix some d ∈ X. Using the strict Gateaux differentiability at x, there exists some δ > 0 such that (A.1) ∀x ∈ x + δB, ∀t ∈]0, δ[, F (x + td) -F (x) t -DF (x; d) ď ε 2 .

Given this δ, for any x ∈ x+δB we can assume (taking eventually a smaller δ) that F is Gateaux differentiable at x. Then, for all x ∈ x + δB there exists some t x ∈]0, δ[ such that (A.2) F (x + t x d) -F (x) t x -DF (x; d) ď ε 2 .

Combining (A.1) and (A.2) together with the triangle inequality, we proved that ∃δ > 0, ∀x ∈ x + δB, DF (x; d) -DF (x; d) ď ε.

Item ii) being proved, suppose now the reverse, that is item i) holds. To prove that F is strictly Gateaux differentiable at x, fix some d ∈ X, and ε > 0. Using the continuity of DF at x in the sense of item ii), we obtain the existence of δ > 0 such that B(x, δ) ⊂ U , and which satisfies In a similar fashion, we prove Proposition 2.1.12:

Proposition A.1.9. Let F : X -→ Y be Frechet differentiable on an open set U ⊂ X. Then, for all x ∈ U , the following is equivalent: i) F is strictly Frechet differentiable at x, ii) DF : U -→ L(X, Y) is strongly continuous at x, i.e. continuous with respect to the norm topology of X and the usual operator norm topology of L(X, Y).

We will use for this proof the fact that the Frechet differentiability (and strict Frechet differentiability) of F at x is equivalent to Given this δ, for any x ∈ B(x, δ) we can assume (taking eventually a smaller δ) that F is Frechet differentiable at x. Then, for all x ∈ B(x, δ) we use (A.5) to obtain some t x ∈]0, δ[ such that Hence the first three sets always coincide, and we call it the convex subdifferential of f at x, noted ∂f (x). Moreover, the fourth set always contains ∂f (x), with equality when f is convex.

Proof. Equivalence between the sets 1 and 2 is immediate. Now prove equivalence between sets 1 and 3. Suppose that (x * , -1) ∈ N epi f (x, f (x)), then by definition, ∀(d, λ) ∈ T epi f (x, f (x)), x * , d ď λ. Take now any y ∈ X, and pose d := yx, λ := f (y)f (x). One easily sees that (x, f (x)) + (d, λ) ∈ epi f , that is (d, λ) ∈ T r epi f (x, f (x)) ⊂ T epi f (x, f (x)). So we deduce x * , yx ď f (y)f (x) which is what we wanted. Now suppose that for all y ∈ X we have f (y)f (x)x * , yx ě 0. Take now any (d, λ) ∈ T epi f (x, f (x)), then by definition there exists sequences d n → d, λ n → λ and t n > 0 such that f (x + t n d n ) ď f (x) + t n λ n . Then (A.12)

x * , d = lim Suppose now that f is convex, and take any linear minorant x * of d D -f (x; •). Then for any y ∈ X, (A.14)

x * , yx ď lim inf ii) If Ω is radial at x, then the above inclusions become equalities.

Proof. We start with item i), and prove that cl T ad Ω (x) ⊂ T B Ω (x). Let d = lim n→+∞ d n with d n ∈ T ad Ω (x), that is there exists some δ n > 0 such that for all t ∈ [0, δ n [, one has x + td n ∈ Ω. We can select a sequence satisfying t n ↓ 0 and t n < δ n , so that x + t n d n ∈ Ω for all n ∈ N. Whence d ∈ T B Ω (x). Now we show that T B Ω (x) ⊂ cl T r Ω (x). Let d ∈ T B Ω (x), then there exists t n ↓ 0 and d n → d such that x + t n d n ∈ Ω for all n ∈ N. Clearly, d n ∈ T r Ω (x), which proves the claim. The inequalities concerning the weak Bouligand tangent cone follows exactly the same proof, replacing strong by weak topology. Item ii) is a direct consequences of Proposition A. f (u(t) + h u(t+ǫ)-u(t) ǫ )f (u(t)) h

Taking ǫ = h we obtain x * , u(t) ď (f • u) ′ (t). Now we write x * ,u(t) ď d B f (u(t),u(t)), and using the same method with ǫ = -h one obtains x * ,u(t) ď -(f • u) ′ (t) which ends the proof.

Proposition A.2.6. Let f : X -→ R ∪ {+∞} be a quasi-convex function such that f (x 2 ) ď f (x 1 ). Then x * , x 2x 1 ď 0 for all x * ∈ ∂ B f (x 1 ) ⊃ ∂ F f (x 1 ).

Proof. Introducing the convex sublevel set Ω = [f ď f (x 1 )], we see that x 1 and x 2 lie therein. Hence tx 2 + (1t)x 1 lies in this set for all t ∈ [0, 1], and: (A.17) f (x 1 + t(x 2x 1 ))f (x 1 ) ď 0 for any t ∈ [0, 1].

After dividing by t and passing to the limit when t ↓ 0, we obtain d D -f (x 1 ; x 2x 1 ) ď 0. Conclusion follows d B f (x 1 ; •) ď d D -f (x 1 ; •).

Proposition A.2.7. Let f : X -→ R ∪ {+∞} be a quasi-convex function such that f (x 2 ) < f (x 1 ). Then x * , x 2x 1 ď 0 for all x * ∈ ∂ L f (x 1 ). If f is locally Lipschitz then ∂ L can be replaced by ∂ C .

Proof. We start by proving our statement for the limiting subdifferential. Let x * ∈ ∂ L f (x 1 ), then it is the weak limit of a sequence (x * n ) n∈N such that

x * n ∈ ∂ F f (x n ) with x n • --→ n→+∞ x 1 and f (x n ) R --→ n→+∞ f (x 1
). Since we have the strict inequality f (x 2 ) < f (x 1 ), we can assume that f (x 2 ) < f (x n ) for all n ∈ N. In particular, Proposition 2.2.15 applies, and x * n , x 2x n ď 0. The conclusion follows directly by passing to the limit on n.

It is easy to extend this property to the Clarke subdifferential when f is locally Lipschitz continuous and X reflexive, by using ∂ C f (x 1 ) = co ∂ L f (x 1 ). Otherwise, we have to prove that d C f (x 1 ; x 2x 1 ) ď 0.

If x ′ ∈ X is taken arbitrary close to x 1 , and because of the continuity of f together with f (x 2 ) < f (x 1 ), we can assume that f (x 2 ) ď f (x ′ ). Using that f is quasi-convex together with x ′ , x 2 ∈ [f ď f (x 1 )], we obtain that ∀t ∈ [0, 1], f (x ′ + t(x 2x ′ ))f (x ′ ) ď 0.

Hence, for all t ∈ [0, 1] and x ′ close to x 1 , use the inequality above and the Lipschitz continuity at x 1 to obtain:

f (x ′ + t(x 2 -x 1 )) -f (x ′ ) = f (x ′ + t(x 2 -x 1 )) -f (x ′ + t(x 2 -x ′ )) + f (x ′ + t(x 2 -x ′ )) -f (x ′ ) ď tLip(f, x 1 ) x ′ -x .
As a consequence, d C f (x 1 ; x 2x 1 ) = lim sup

x ′ →x 1 t↓0

f (x ′ + t(x 2x 1 ))f (x ′ ) t ď lim sup

x ′ →x 1 t↓0

Lip(f, x 1 ) x ′x = 0.

A.3 Optimization for vector-valued functions

Here are some results concerning the bases of a cone K in a locally convex topological vector space (X, τ ). Let us fix some definitions in that setting. We say that a set A ⊂ K is τ -bounded if for any neighbourhood of zero V , there exists some λ > 0 such that A ⊂ λV . This simply corresponds to the usual norm-boundedness in the following cases: X is a normed space equipped with its norm or weak topology, or X is the topological dual of a Banach space, equipped with the norm, weak or weak star topology. It is a consequence of the uniform boundedness principle [5, Theorems 6.14-6.15], see also Mackey's theorem [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Theorem 6.20], [START_REF] Köthe | Topological vector spaces I[END_REF]XX.11.7]. We will also consider the strong topology associated to (X, τ ), that we note β(X, τ ). It is the topology induced by the w * -bounded sets of (X, τ ) * , see [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Chapter 5.19]. Here again, as expected, the strong topology reduces to the usual norm topology when X is a normed space equipped with its norm or weak topology, or X is the topological dual of a Banach space, equipped with the norm, weak or weak star topology.

We present some necessary and sufficient conditions on a cone K for the existence of such a base. We say that a cone K is pointed if K ∩ -K = {0}.

Proposition A.3.1. i) If K admits a τ -closed and τ -bounded base, then K is τ -closed.

ii) If K admits a convex base, then K is pointed and convex.

As a consequence, if K admits a τ -closed convex and τ -bounded base, then K is τ -closed, convex, pointed, with a nonempty strong interior.

Proof. Item ii) can be found in [START_REF] Jahn | Vector optimization: Theory, applications, and extensions[END_REF]Lemma 1.14]. Both items i) and ii) can be found in [START_REF] The | Theory of vector optimization[END_REF]Proposition I.1.7], where they are presented as a result of Jameson [START_REF] Jameson | Ordered linear spaces[END_REF].

Given a strict cone S ⊂ X, we also introduce its strict dual cone S ++ ⊂ (X, τ ) * as (A.18)

S ++ := {x * ∈ X * | x * , x > 0 for all x ∈ S}.

Analogously, we define K ++ the strict dual cone of a cone K, as the strict dual cone of K \ {0}. Do not mistake K ++ with the bidual (K + ) + . The strict dual cone of K is sometimes called the quasi-interior of K + , since K ++ is exactly the quasi-interior of K + with respect to the weak * toology of X * when K is closed convex (see [START_REF] Bot | Duality in vector optimization[END_REF]Proposition 2.1.1]). It is easy to see that if K ++ = ∅ then K is pointed, and in fact the reverse statement is true in the normed separable spaces:

Given a base Θ of K ⊂ (X, τ ), we study now its support function Proof. Because of Mackey's theorem, the τ -boundedness of Θ implies its boundedness for the weak topology on X. As a consequence, for all x * ∈ X * , x * (Θ) is a bounded set in R. It is why σ Θ take finite values on X * . Let us turn now on continuity. From its definition, we know that σ Θ is lower semi-continuous for the weak * topology, which is coarser than the strong topology β(X * , w * ). It follows that σ Θ is β-lower semi-continuous. Let us show now that it is also strongly upper semi-continuous. Take for this an arbitrary net (x * α ) α∈A converging to some x * ∈ X * for the strong topology. In other words, we assume for all τ -bounded sets A ⊂ X that sup Recall that the polar cone K * and the dual cone K + are just the negative of each other:

K * = -K + .
Proposition A.3.5. Let K ⊂ (X, τ ) be a cone. Proof. Item i) is immediate from K = R + Θ:

x * ∈ K * ⇔ ∀x ∈ K, x * , x ď 0 ⇔ ∀x ∈ Θ, x * , x ď 0 ⇔ σ Θ (x * ) ď 0.

For item ii), take the strong interiors in item i), i.e.

β int K * = β int [σ Θ ď 0].
We will now apply apply Proposition A.1.4 to conclude. We already know that σ Θ is convex, and is strongly continuous by Proposition A.3.4. We just need to verify that [σ Θ < 0] is nonempty. For this, consider the hypothesis 0 / ∈ τ cl coΘ, and apply Hanh-Banach separation theorem to obtain some x * ∈ X * and α ∈ R such that σ Θ (x * ) ď α < 0.

We can see that Theorem 2.3.7 is a direct consequence of Proposition A.3.5, applied to K + ⊂ (X * , w * ).

Proposition A.3.6. Let K be a convex cone having a convex base Θ. Let e ′ ∈ K

′ ++ be such that Θ = {x ∈ K | e ′ , x = 1}. Then : i) for all x * ∈ X * , σ Θ (x * ) = inf t∈R {x * ĺ X * te ′ } where ĺ X * denotes the dual order on X * induced by K + .

ii) for all x * ∈ X * and λ ∈ R, σ Θ (x * + λe ′ ) = σ Θ (x * ) + λ.

Proof. Let t ∈ R, then using the definition of e ′ and Proposition A.3.5, the item i) follows (A.21)

x * ĺ X * te ′ ⇔ ∀θ ∈ Θ, x * , θ ď te ′ , θ ⇔ ∀θ ∈ Θ, x * , θ ď t ⇔ σ Θ (x * ) ď t.

Item ii) is a direct consequence of item i): and pick some t n k ∈ B n k . Define then f n := Kn k=0 f (t n k )χ I n k , which is clearly a simple function. Thus, given any ε > 0, we can take N ∈ N satisfying 1 N < ε, so that f (t)f n (t) ď ε for all n ě N and t ∈ T .

σ Θ (x * + λe ′ ) =
According to [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Lemma 11.39] or [137, Theorem II.2.2], for any strongly measurable function f : T -→ Z, the function f : T -→ R defined by f (t) = f (t) is also strongly measurable, so we can define f L 1 (T ,Z) := T f (t) dµ(t) ∈ R ∪ {+∞}.

We define then L 1 (T , Z) the space of Bochner integrable functions as In an obvious way, we define the corresponding spaces of (classes of) strongly measurable functions L p (T , Z) and L ∞ (T , Z).

Next Proposition shows that the Bochner integral commutes with the bounded operators. From now, we focus on the case T = [0, T ], T ∈]0, +∞[, and give some properties of the spaces L p ([0, T ], Z). It is easy to see, as in the real case, that because of the finite measure of [0, T ], we have an ordered embedding of spaces : L ∞ ([0, T ], Z) ⊂ L q ([0, T ], Z) ⊂ L p ([0, T ], Z) ⊂ L 1 ([0, T ], Z) for all 1 ď p ď q ď +∞.

We also have the useful: Proposition B.2.6. C([0, T ], Z) ⊂ L ∞ ([0, T ], Z) is an embedding.

Proof. Note that, by definition, the set of t ∈ [0, T ] such that f L ∞ ([0,T ],Z) < f (t) has null measure, and is open for a continuous function f . Both properties imply that this set is empty, and that f C([0,T ],Z) = f L ∞ ([0,T ],Z) , so the result follows.

An important point is to identify the topological dual of such L p spaces. A first partial result can be found in [137, Theorem IV.1.1 (and the discussion above)], see also [START_REF] Tulcea | Topics in the theory of lifting[END_REF]Chapter VI] or [START_REF] Fonseca | Modern methods in the calculus of variations: L p spaces[END_REF]Theorem 2.112]: Theorem B.2.7. Let p ∈ [1, +∞[, and q ∈]1, +∞] its conjugate exponent, i.e. such that 1 p + 1 q = 1. Then L q ([0, T ], Z * ) can be isometrically viewed as a subset of L p ([0, T ], Z) * . Moreover, the equality L q ([0, T ], Z * ) = L p ([0, T ], Z) * holds if Z is a reflexive4 space. In both cases, the duality pairing is defined as follows:

∀g ∈ L q ([0, T ], Z * ), ∀f ∈ L p ([0, T ], Z), g, f := T 0 g(t), f (t) Z * ×Z dt. Remark B.2.8. In Section 7.3, we need to work with the whole dual of L 1 ([0, T ], Z), for Z being not reflexive. In that case, L 1 ([0, T ], Z) * is strictly bigger than L ∞ ([0, T ], Z * ), and its complete description pass through the consideration of weak * measurable functions, see the next section.

We end this section with a classic result due to Rockafellar on the subdifferential of convex integrands [START_REF] Rockafellar | Integrals which are convex functionals[END_REF]Theorem 4].

Theorem B.2.9. Let X be an Euclidean space. Let f : [0, T ] × X -→ R be such that • for all x ∈ X, f x := f (•, x) : [0, T ] -→ R is Borel measurable and has bounded values,

• for all t ∈ [0, T ], f t := f (t, •) : X -→ R is convex.

Consider the associated convex integrand

I f : L ∞ ([0, T ], X) -→ R ω -→ T 0 f (t, ω(t)) dt.
Then I f is well-defined, convex and (strongly) continuous. Moreover its subdifferential enjoys the following characterization: ∀v ∈ L 1 ([0, T ], X), ∀ω ∈ L ∞ ([0, T ], X), v ∈ ∂I f (ω) ⇔ v(t) ∈ ∂f t (ω(t)) for a.e. t ∈ [0, T ].

Remark B.2.10. Note that in the above result, the subdifferential lives in the dual of L ∞ ([0, T ], X), in which L 1 ([0, T ], X) can be identified as a subspace, and where the subdifferential characterization holds.

Adapting the proof of the above theorem, we derive a useful corollary for multi-objective problems:

Corollary B.2.11. Let X be an Euclidean space, and let f 1 , ..., f m : X -→ R be a finite family of convex continuous functions. Let θ = (θ i ) ∈ L ∞ ([0, T ], R m ) be such that θ(t) ∈ ∆ m for a.e. t ∈ [0, T ]. Consider the associated convex integrand

I : L ∞ ([0, T ], X) -→ R ω -→ T 0 m i=1 θ i (t)f i (ω(t)) dt.
Then I is well-defined, convex and (strongly) continuous. Moreover, its subdifferential enjoys the following characterization: ∀v ∈ L 1 ([0, T ], X), ∀ω ∈ L ∞ ([0, T ], X), Applying [START_REF] Rockafellar | Integrals which are convex functionals[END_REF]Theorem 2] to the function f (t, x) := m i=1 θ i (t)f i (x), we obtain that the integrand I is well-defined, convex, continuous, and its Fenchel transform (see [START_REF] Bauschke | Convex analysis and monotone operator theory[END_REF][START_REF] Peypouquet | Convex optimization in normed spaces. Theory, methods and examples[END_REF]) is given by

v ∈ ∂I(ω) ⇔ v(t) =
I * (v) = T 0 m i=1 θ i (t)f i * (v(t))dt.
Let us analyze this last expression. Since the f i are convex continuous functions, their conjugate are coercive functions, and (θ i (t)f i ) (ω(t)) + (θ i (t)f i ) * (z i (t))ω(t), z i (t) dt = 0.

Since each of the elements of this last sum expression is nonnegative, we deduce that, for each i ∈ {1, ..., m} and for almost all t > 0, θ i (t)f i (ω(t)) + (θ i (t)f i ) * (z i (t))ω(t), z i (t) = 0.

Equivalently, z i (t) ∈ ∂ (θ i (t)f i ) (ω(t)). Since we know from (B.3) that v(t) = m i=1 z i (t), we need to verify that there exists v i ∈ L ∞ ([0, T ], X) such that z i (t) = θ i (t)v i (t).

Take some zi ∈ L ∞ ([0, T ], X) such that zi (t) ∈ ∂f i (ω(t)) for almost all t > 0, (such an element exists, take for example zi (t) = (∂f i ) 0 (ω(t))). We have z i (t) = θ i (t)v i (t) for almost all t > 0, where

v i (t) = v(t) θ i (t)
if θ i (t) > 0, zi (t) if θ i (t) = 0. This choice of v i is measurable, and v i (t) ∈ ∂f i (ω(t)) for almost all t > 0. By continuity of f i , we conclude that v i ∈ L ∞ ([0, T ], X).

B.3 Weak

Résumé.

Dans une première partie, nous nous intéressons aux systèmes dynamiques gradients gouvernés par des fonctions non lisses, mais aussi non convexes, satisfaisant l'inégalité de Kurdyka-Lojasiewicz. Après avoir obtenu quelques résultats préliminaires pour la dynamique de la plus grande pente continue, nous étudions un algorithme de descente général. Nous prouvons, sous une hypothèse de compacité, que tout suite générée par ce schéma général converge vers un point critique de la fonction à minimiser. Nous obtenons aussi de nouveaux résultats sur la vitesse de convergence, tant pour les valeurs que pour les itérés. Ce schéma général couvre en particulier des versions parallélisées de la méthode forward-backward, autorisant une métrique variable et des erreurs relatives. Cela nous permet par exemple de proposer une version non convexe non lisse de l'algorithme Levenberg-Marquardt. Enfin, nous proposons quelques applications de ces algorithmes aux problèmes de faisabilité, et aux problèmes inverses parcimonieux.

Dans une seconde partie, cette thèse développe une dynamique de descente associée à des problèmes d'optimisation vectoriels sous contrainte. Pour cela, nous adaptons la dynamique de la plus grande pente usuelle aux fonctions à valeurs dans un espace ordonné par un cône convexe fermé d'intérieur non vide. Cette dynamique peut être vue comme l'analogue continu de nombreux algorithmes développés ces dernières années. Nous avons un intérêt particulier pour les problèmes de décision multi-objectifs, pour lesquels cette dynamique de descente fait décroitre toutes les fonctions objectif au cours du temps. Nous prouvons l'existence de trajectoires pour cette dynamique continue, ainsi que leur convergence vers des points faiblement efficients. Finalement, nous explorons une nouvelle dynamique inertielle pour les problèmes multi-objectif, avec l'ambition de développer des méthodes rapides convergeant vers des équilibres de Pareto.

Abstract.

In a first part, we focus on gradient dynamical systems governed by non-smooth but also non-convex functions, satisfying the so-called Kurdyka-Lojasiewicz inequality. After obtaining preliminary results for a continuous steepest descent dynamic, we study a general descent algorithm. We prove, under a compactness assumption, that any sequence generated by this general scheme converges to a critical point of the function to be minimized. We also obtain new convergence rates both for the values and the iterates. The analysis covers alternating versions of the forward-backward method, with variable metric and relative errors. As an example, a non-smooth and non-convex version of the Levenberg-Marquardt algorithm is detailed. Applications to non-convex feasibility problems, and to sparse inverse problems are discussed.

In a second part, the thesis explores descent dynamics associated to constrained vector optimization problems. For this, we adapt the classic steepest descent dynamic to functions with values in a vector space ordered by a closed convex cone with nonempty interior. It can be seen as the continuous analogue of various descent algorithms developed in the last years. We have a particular interest for multi-objective decision problems, for which the dynamic make decrease all the objective functions along time. We prove the existence of trajectories for this continuous dynamic, and show their convergence to weak efficient points. Then, we explore an inertial dynamic for multi-objective problems, with the aim to provide fast methods converging to Pareto points.

  ∇f i (x), d .

tě0f

  (x(t)). Given the diffeomorphism f • x : [0, +∞[-→]s ∞ , s 0 ], consider the change of variables s = (f • x)(t), and the associated trajectory ∀s ∈]s ∞ , s 0 ], y(s) := x (f • x) -1 (s) .

  This trajectory verifies ẏ(s) -∇f (y(s)) ∇f (y(s)) 2 = 0, in other words, y(•) describes the same curve than x(•), but at reverse and with a different speed.

  claimed. Let us now distinguish the two cases: For item i), we argue by contradiction. If r k > 0 for all k ∈ N, then (3.29), together with the hypothesis (b k ) k∈N / ∈ ℓ 1 in H 3 , imply lim k→+∞ Φ(r k+1 ) = +∞. This contradicts the fact that lim t→0 Φ(t) ∈ R. Hence, r k = 0 for some k ∈ N and H 1 ensures that the sequence remains stationary there.

Figure 3 . 1 :

 31 Figure 3.1: Asymptotic behaviour of Φ -1 , for different values of θ

  endow it with the inner product •, • := p i=1 •, • H i and the associated norm • := •, • .

Proposition 4 . 1 . 3 .

 413 Any bounded sequence Y k = (y 1,k , ..., y p,k ) generated by the AFB algorithm with errors, which verify HP and HE, satisfies H 1 , H 2 and H 3 .

Example 4 . 2 . 1 .

 421 Consider two closed half-planes C 1 , C 2 in R 2 , with nonempty intersection. The algorithm (4.24) writes in that case

Figure 4 . 1 :

 41 Figure 4.1: An relaxed averaged projection algorithm.

Figure 4 . 2 :

 42 Figure 4.2: An relaxed alternate projection algorithm.

Proposition 4 . 2 . 14 .

 4214 Let D 1 , . . . , D p be a collection of closed prox-regular subsets of H, and let C ⊂ H be closed and nonempty. Assume that the intersection S = C ∩ D 1 ∩ • • • ∩ D p = ∅ is strongly regular at x * ∈ S. Then, the function f (x) := δ C (x) + 1 2 p j=1 w j dist 2 (x, D j ), has the Kurdyka-Lojasievicz property at x with a Lojasiewicz exponent θ = 1 2 .

  the values e(δ, ρ) for (δ, ρ) ∈ [0, 1] 2 , in a gray scale: an exact recovery e(δ, ρ) = 1 corresponds to a black pixel, while e(δ, ρ) = 0 corresponds to a white pixel.

Figure 4 . 3 :

 43 Figure 4.3: Values of e(δ, ρ) for (δ, ρ) ∈ [0, 1] 2 .

Figure 4 . 4 :

 44 Figure 4.4: Top line: the original picture X and a representation of its wavelet transform W X.Bottom line: the matrix X obtained after putting to zero 90% of the coefficients of W X, and the corresponding picture W -1 X.

Figure 4 . 5 :

 45 Figure 4.5: Left: original image. Right: Image obtained after compression to 57% of the original size, and then decompression using 100 iterations of the Hard-Shrinkage Projection algorithm.

Figure 4 . 6 :

 46 Figure 4.6: Left: original image. Right: Image obtained after compression to 60% of the original size, and then decompression using 100 iterations of the Hard-Shrinkage Projection algorithm.

Figure 4 . 7 :

 47 Figure 4.7: Reconstruction of the Shepp-Logan image, after compression to 60% of the original size. From left to right, top to bottom, reconstructed pictures after 0, 1, 5, 10, 25 and 32 iterations of the Hard-Shrinkage Projection algorithm.

  5.6) means that the sets ∂ C F (x) and m i=1 ∂ C f i (x) have the same support functions. Applying Theorem 2.1.4, we deduce that

  proved in Proposition 5.1.4, and the weak * lower semi-continuity of the norm in X * .

  other words, d C F (x; d) = σ Θ•DF (x) (d), where Θ • DF (x) := {θ • DF (x) | θ ∈ Θ}, which is also equal to D * F (x; Θ). From the definition of ∂ C F (x), and the characterization of support functions in Theorem 2.1.4, we deduce that ∂ C F (x) = co * Θ • DF (x). Since Θ is convex and weakly * closed, item iii) follows.

Figure 5 . 3 :

 53 Figure 5.3: A counterexample to Proposition 5.1.8.

2

  The indicator function of C is a function taking as values δC (x) = 0 if x ∈ C, +∞ otherwise.

Example 5 . 2 . 3 .

 523 If F = (f 1 , f 2 ) : R 2 -→ R 2 issmooth, the strict cone of descent directions at x ∈ R 2 consists in the vectors d satisfying ∇f 1 (x), d < 0 and ∇f 2 (x), d < 0.

Figure 5 . 5 :

 55 Figure 5.5: Illustration of the Pareto alternative in Theorem 5.2.4, for a bi-objective optimization problem. We take two points x 1 and x 2 , and we plotted the two corresponding strict cones of descent directions at x 1 and x 2 respectively. The thick dashed line represents the set of critical points. One can see that the closer x is to the set of Pareto points, the smaller is the strict cone of descent directions.

Figure 5 . 6 :

 56 Figure 5.6: First-order analysis at the intersection of two sublevel sets.

Figure 5 . 7 :

 57 Figure 5.7: The steepest descent direction associated to a smooth biobjective optimization problem.

For 2 ,

 2 any a ∈ ∂ C F (x) let us define (5.49) d(a) := argmin d∈T C (x) 1 p d p + a, d . Of course, by definition we have d = d(ā). Using Fermat's rule for the above primal problem (5.49) together with a sum rule (recall 2.2.6) gives (5.50) 0 ∈ N T C (x) (d(a)) + d(a) d(a) p-2 + a, where we used that the derivative of 1 p • p at any x for p ě 2 exists and is x x p-2 . Using the variational characterization of the projection, it follows (5.51) d(a) = proj T C (x) -a d(a) p-2 , which can be rewritten, by Moreau's theorem, as (5.52) d(a) = 1 d(a) p-2 (-a -N C (x)) 0 . Since d = d(p), we just need to prove that d(ā) = a∈∂ C F (x) min d∈T C (x) 1 p d p + a, d , which, by (5.49) and (5.51), can be rewritten as (5.54) ā = argmax a∈∂ C F (x) 1 p d(a) pd(a) p-2 -a d(a) p-2 , proj T C (x) -a d(a) p-2 . ) pd(a) p-2 proj T C (x) -a d(a) p-2 which, by (5.51) and p ∈ [2, +∞], is equivalent to (5.56) ā = argmin a∈∂ C F (x) d(a) p-1 . From (5.52), we know that d(a) p-1 = (-a -N C (x)) 0 . Therefore, s = (-ā -N C (x)) 0 with d(ā) p-1 = s . Using again (5.52), we obtain d(ā) =

Figure 5 . 8 :

 58 Figure 5.8: First-order analysis at the intersection of two sublevel sets and a constraint.

A

  if and only if y * , Aα(x) R --→ α∈A y * , A(x) for all y * ∈ Y * and all x ∈ X. The "coo" is the operator convex hull, defined for a family F ⊂ L(X, Y) as the set of finite sums N i=1 ui • Ai, where the Ai's are elements of F , and the ui's are positive endomorphisms of Y (i.e. 0 ĺ y ⇒ 0 ĺ ui(y)) such that N i=1 ui = id Y .

Proposition 6 . 1 . 5 (

 615 Equicontinuity of the cost functions). Let u : [0, +∞[-→ C be a strong global solution of (SD). i) Lipschitz continuity: The trajectory is Lipschitz continuous on any finite time interval [0, T ]. ii) Global Lipschitz continuity: If we suppose that the trajectory is bounded, and that F is Lipschitz continuous on bounded sets, then the trajectory is Lipschitz continuous on the entire interval [0, +∞[. iii) Equicontinuity: The family {f θ • u : [0, T ] -→ R | θ ∈ Θ} is: iii.a) uniformly Lipschitz continuous (i.e. Lipschitz continuous with the same Lipschitz constant), therefore equicontinuous, iii.b) relatively compact in (C[0, T ], R), • ∞ ), the space of continuous real-valued functions on [0, T ], equipped with the uniform convergence topology.

.

  The set of Pareto points coincide with the set of weak Pareto points, and is equal to the segment [a, b]. The steepest descent vector field is given by s(x) = -(xproj [a,b] x). Trajectories are straight lines connecting the starting point and its projection on the Pareto set.

Figure 6 .Figure 6 . 1 :

 661 Figure 6.1: Trajectories associated to two distance functions in H = R 2 .

Figure 6 . 2 :

 62 Figure 6.2: Pareto front in the value space Y = R 2 . F (X) is in grey, the Pareto front is the thick black curve.

Figure 6 .Figure 6 . 3 :

 663 Figure 6.3 below shows some trajectories of the (SD) dynamic. One can see that both objective function decrease along the trajectories. Figure 6.4 shows the Pareto front in the value space.

Figure 6 . 4 :

 64 Figure 6.4: Pareto front in the value space Y = R 2 . F (X) is in grey, the Pareto front is the thick black curve.

Figure 6 . 5 :

 65 Figure 6.5: Lack of Lipschitz continuity for u → s(u).

Figure 6 . 6 :

 66 Figure 6.6: Trajectories in H = R 2 associated to two degenerated quadratic forms.

Figure 6 . 7 :

 67 Figure 6.7: Pareto front in the value space Y = R 2 . F (X) is in grey, the Pareto front is the thick black curve.

Remark 6 . 1 . 20 .

 6120 In the multi-objective case, if H = R n with n ě m, then the necessary condition of Proposition 6.1.18 is equivalent to rank DF (u) ď m -1.

Figure 6 . 9 :

 69 Figure 6.9: Left: approximation of F (C). Right: the Pareto points are plotted in thick continuous red lines, and the local Pareto points are in thin dotted blue lines.

Figure 6 . 10 :

 610 Figure 6.10: Top Left: 20 points with initial data chosen randomly. Top Right: 100 points with initial data chosen randomly. Bottom: 4000 points with initial data chosen on a uniform grid.

Figure 6 .

 6 Figure 6.11: Left: 20 points with initial data chosen randomly. Right: 100 points with initial data chosen randomly.

Figure 6 . 13 :

 613 Figure 6.13: From left to right, and top to bottom: 1000 samples of the steepest descent method, with n taking respectively the values 1, 2, 3, 50.

Figure 6 . 14 :

 614 Figure 6.14: An enlargement for R 2 + .

Definition 7 . 1 . 1 (

 711 Hausdorff distance). Let A, B be two nonempty subsets of E. The Hausdorff distance between A and B is defined by d H (A, B) = max{sup a∈A d(a, B); sup b∈B d(b, A)}, where d(a, B) := inf b∈B ab .

Theorem 7 . 1 . 3 .

 713 Let F : E -→ Y be a Fréchet differentiable function, with a Lipschitz continuous gradient. Suppose that E is Euclidean. Then, for all u 0 ∈ C, there exists a strong global solutionu : [0, +∞[-→ E of (SD) u(t) + (N C (u(t)) + ∂ C F (u(t))) 0 = 0 such that u(0) = u 0 . More precisely, there exists 1 η : [0, +∞[-→ E and θ : [0, +∞[-→ Y * such that i) u(t) + η(t) + θ(t) • DF (u(t)) = 0 for a.e. t ∈ [0, +∞[, ii) η(t) ∈ N C (u(t)) and θ(t) ∈ Θ for a.e. t ∈ [0, +∞[, iii) η ∈ L 2 ([0, T ], E) and θ(•) • DF (u(•)) ∈ L 2 ([0, T ], E) for each T > 0.

  weakly- * convergent net θ α w * --→ α∈A θ in Θ. For all d ∈ E, DF (x; d) ∈ Y and θ α , DF (u(t); d) Y * ×Y R --→ α∈A θ, DF (u(t); d) Y * ×Y . In other words, θ α • DF (u(t)) tends to θ • DF (u(t)) in (E * , w *

η

  and η λ (t) ∈ N C (u λ (t)), and from the demi-closedness property of the extension to L 2 (0, T ; E) of the maximal monotone normal cone mapping N C , we obtain η(t) ∈ N C (u(t)) for a.e. t ě 0.

  i (t)|, we deduce that, for a.e. t ě 0, m i=1 θ i (t) ď 1. To prove that m i=1 θ i (t) = 1, consider any measurable I ⊂ [0, T ], and use the weak * convergence of θ λ to θ to show I m i=1 θ i (t) dt = |I|, and the conclusion follows.

  Therefore, by passing to the limit on the left member of (7.20), we obtain lim t)f i (ξ(t)) dt.

Lemma 7 . 3 . 4 .

 734 Let (Θ, w * ) be metrizable. Let µ ∈ P R (Θ), and take the meanθµ := Θ θ dµ(θ).Then θµ ∈ Θ and f µ = fθ µ .Proof. Considerφ : Θ -→ Y * and A : Y * -→ R θ -→ θ y * -→ y * , F (x) ,where φ and A are continuous.

Proposition 7 . 3 . 11 .

 7311 Let F : E -→ Y be a convex function bounded from below, with E finitedimensional. Then, for all λ > 0, F λ is Gateaux differentiable. Its derivative DF λ : E -→ L(E, C(Θ)) is 1 λ -Lipschitz continuous, and is defined by : (7.49) for all x ∈ E, d ∈ E, θ ∈ Θ, DF λ (x; d)(θ) = Df θ,λ (x; d) = ∇f θ,λ (x), d .

.

  Proof. Let x ∈ E. Using Proposition 7.3.11, we know that for all d ∈ E and µ ∈ M R (Θ), we haveD * F λ (x; µ), d = µ, DF λ (x; d) = Θ ∇f θ,λ (x), d dµ(θ).Moreover, because of Lemma 7.3.10 and Theorem B.2.2, θ → ∇f θ,λ (x) is Bochner integrable. It follows then that D * F λ (x; µ) = Θ ∇f θ,λ (x) dµ(θ). Now we turn on the proof of item ii). Start by taking any θ ∈ Θ, and observe that ∇f θ,λ (x) = Θ ∇f θ,λ (x) dδ θ ( θ), where δ θ ∈ P R (Θ) is the Dirac measure at θ. Write now for all d ∈ E:

1 M

 1 R (Θ)×C(Θ) dt = |I| for any measurable I ⊂ [0, T ]. The latter implies that (7.75) µ(t), 1 M R (Θ)×C(Θ) = 1 for a.e. t ∈ [0, T ].

  w --→ λ→0 u in L 2 ([0, T ], E), we deduce that (7.94) T 0 ψ(t) u(t) 2 dt ď lim inf

  ) uλ (t) 2 dt.

Remark 7 . 4 . 2 (

 742 Seeing ∂ C F λ as a Yosida approximation of ∂ C F ). In Section 6.3.1, we formally defined the resolvent of ∂ C F for a convex function F by ∀x ∈ H, PROX λF

  θ∈∆m j(t, θ) (see B.1.2). Now we can write

  m

  -1, y) if x > 1, -(0, y) if -1 ď x ď 1, -(x + 1, y) if x < -1.

Figure 8

 8 Figure 8.1 shows some trajectories of the (ISDF) dynamic, with the steepest descent vector field plotted in background. We took the Cauchy data (u 0 , u0 ) randomly. Here the trajectories are computed exactly, since in this simple example (ISDF) can be solved explicitly. We can observe the following: all the trajectories converge to a Pareto point, the dynamic is clearly not a descent method, and can be highly oscillating whenever the friction parameter is too close to zero.

Figure 8 . 1 :

 81 Figure 8.1: Friction parameter γ = 1 (left) and γ = 0.1 (right). For each trajectory, the initial point is indicated by the symbol ×, and the limit point by ⊕.

(x i - 1 . 5 ) 2 -

 152 10 cos(2π(x i -1.5)) + 10 1 4

t k- 1 - 1 t

 11 k (y ky k-1 ), where t k =

  So, x n ∈ [f < 0] and it follows that x ∈ cl τ [f < 0]. This proves item i)For item ii), we apply Proposition A.1.3 to C = [f < 0], which is convex since f is convex.For this we need to verify that [f < 0] has a nonempty τ -interior. But f being upper semicontinuous implies that [f ě 0] is τ -closed, and its complement is exactly [f < 0]. Hence, [f < 0] is τ -open and nonempty since we assume that x ∈ [f < 0]. So Proposition A.1.3 applies, and we obtain int τ cl τ [f < 0] = int τ [f < 0]. Since [f < 0] is open and satisfies item i), we deduce that int τ [f ď 0] = [f < 0]. Now we prove the results concerning the differentiability of a function F : X -→ Y, where X and Y are two Banach spaces. First, we will need some technical results.

F

  (x + hd) -F (x) h = DF (x; d).Similarly, it is left derivable at zero:lim h→0 - φ(h)φ(0) h = lim h→0 - F (x + hd) -F (x) h = lim h→0 + F (xhd) -F (x) -h = -DF (x; -d).

Corollary A. 1 . 6 . 1 0

 161 If F : X -→ Y is Gateaux differentiable on a segment [x 1 , x2 ] ⊂ X, then for all d ∈ X, the application φ : t ∈ R -→ F ((1t)x 2 + tx 1 ) ∈ Y is continuous and derivable on [0, 1]. In particular, φ ′ (t) = DF ((1t)x 2 + tx 1 ; x2 -x1 ) for all t ∈ [0, 1].Applying the fundamental theorem of calculus to φ : [0, 1] -→ Y, ones obtainCorollary A.1.7. If F : X -→ Y is Gateaux differentiable on a segment [x 1 , x2 ] ⊂ X, then F (x 2 ) -F (x 1 ) = DF ((1t)x 2 + tx 1 ; x2 -x1 ) dt.Now we can prove Proposition 2.1.11, that we recall here:Proposition A.1.8. Let F : X -→ Y be Gateaux differentiable on an open set U ⊂ X.Then, for all x ∈ U , the following is equivalent: i) F is strictly Gateaux differentiable at x, ii) DF : U -→ L(X, Y) is pointwise continuous at x, i.e. continuous with respect to the norm topology of X and the pointwise 1 topology of L(X, Y). In other words, for all converging net (x α ) α∈A ⊂ U converging to x, we have DF (x α ; d) d) for all d ∈ X.

(A. 3 ) 1 0 1 0

 311 for all x ∈ B(x, δ), DF (x; d) -DF (x; d) ď ε.Take arbitrary x ∈ B(x, δ 2 ) and t ∈]0, δ d 2 [. These bounds ensure that x + td remains in B(x, δ), where F is Gateaux differentiable. Since [x; x + td] ⊂ B(x, δ), we can use Corollary A.1.7 which gives usF (x + td) -F (x) = DF (x + τ td, td) dτ, or equivalently (A.4) F (x + td) -F (x) -tDF (x; d) = t(DF (x + τ td, d) -DF (x; d)) dτ,But for all τ ∈ [0, 1], x + τ td lies in [x, x + td] ⊂ B(x, d) where (A.3) applies. We deduce immediately from (A.4) thatF (x + td) -F (x) -tDF (x; d) ď εt.Hence, we proved that for all x ∈ B(x, δ 2 ) and t ∈]0, δ d 2 [,F (x + td) -F (x) t -DF (x; d) ď ε,which proves the strict Gateaux differentiability of F at x.

lim t↓0 sup d ď1 FFF

 ď1 (x + td) -F (x) -tDF (x; (x + td) -F (x) -tDF (x; d) t = 0. (A.6)Proof. Suppose first that F is strictly Frechet differentiable at x. Using the strict Frechet differentiability at x and (A.6), there exists some δ > 0 such that(A.7) ∀x ∈ B(x, δ), ∀t ∈]0, δ[, sup d ď1 (x + td) -F (x) t -DF (x; d) ď ε 2 .

F 2 .

 2 (x + t x d) -F (x) t x -DF (x; d) ď ε Combining (A.7) and (A.8) together with the triangle inequality, we proved that ∃δ > 0, ∀x ∈ B(x, δ), sup d ď1 DF (x; d) -DF (x; d) ď ε.In other words, using the definition of the operator norm in L(X, Y),∃δ > 0, ∀x ∈ B(x, δ), DF (x) -DF (x) ď ε. i) {x * ∈ X * | ∀y ∈ X, f (y)f (x)x * , yx ě 0}, ii) {x * ∈ X * | x is a global minimum of y → f (y)x * , y }, iii) {x * ∈ X * | (x * , -1) ∈ N epi f (x, f (x))}, iv) {x * ∈ X * | ∀d ∈ X, x * , d ď d D -f (x; d)}.

  n→+∞ x * , x + t n d nx t n ď lim n→+∞ f (x + t n d n )f (x) t n ď lim n→+∞ λ n = λ.Now we check that in the general case, any x * ∈ ∂f (x) is a linear minorant for d D -f (x; •). Indeed for any d ∈ X,(A.13) x * , d = lim inf t↓0 x * , x + tdx t n ď lim inf t↓0 f (x + td)f (x) t = d D -f (x; d).

t↓0f

  (x + t(yx))f (x) t ď lim inf t↓0 tf (y) + (1t)f (x)f (x) t = f (y)f (x). Proposition A.2.3. Let x ∈ Ω ⊂ X. Then: i) cl T ad Ω (x) ⊂ T B Ω (x) ⊂ cl T r Ω (x) = T Ω (x) .

σ

  Θ : X * -→ R ∪ {+∞} x * -→ sup θ∈Θ x * , θ . Proposition A.3.4.Let Θ be a τ -bounded set of K. Then σ Θ : X * → R ∪ {+∞} takes finite values, and is continuous for the strong topology on X * .

  x∈A | x *x * α , x | tends to zero when α ∈ A. In particular it holds for Θ and we deduce that(A.19) |σ Θ (x *x * α )| ď sup x∈Θ | x *x * α , x | R sublinearity of σ Θ , we have (A.20) σ Θ (x * α ) ď σ Θ (x *x * α ) + σ Θ (x * ) for all α ∈ A.So take the limsup over α ∈ A in (A.20), together with the limit obtained in (A.[START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality[END_REF], to deduce lim sup α∈A σ Θ (x * α ) ď σ Θ (x * ) and conclude.

  i) If Θ is a base of K, then K * = [σ Θ ď 0]. ii) If moreover Θ is τ -bounded and 0 / ∈ τ cl coΘ, then β int K * = [σ Θ < 0] is nonempty.

L 1 (

 1 T , Z) := {f : T -→ Z | f is strongly measurable and f L 1 (T ,Z) < +∞}.We define L 1 (T , Z) as the quotient of L 1 (T , Z) by this equivalence relation :f ∼ g ⇔ f (t) = g(t) for µ-a.e. t ∈ T .Similarly, we define for all p ∈]1, +∞[f L p (T ,Z) := T f (t) p dµ(t) 1 p ∈ R ∪ {+∞},and f L ∞ (T ,Z) := supess t∈T f (t) .

Proposition B. 2 . 4 (

 24 [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF],Lemma 11.45). Let φ : T -→ X be a Bochner integrable function, and A ∈ L(X, Y) a continuous linear operator. Then A • φ is also Bochner integrable, andT A(φ(t)) dµ(t) = A T φ(t) dµ(t) .

Theorem B. 2 . 5 .

 25 For all p ∈ [1, +∞], • L p ([0,T ],Z) is a norm on L p ([0, T ], Z), which makes it a Banach space. This result is exposed in [162, Theorem 2.100], [137, p.50, p.97] or [308, Theorem VI.1.1, Section VI.3]. One can also verify that L 2 ([0, T ], Z) is Hilbert space if Z is a Hilbert space, if it is endowed with the inner product f, g L 2 ([0,T ],Z) := T 0 f (t), g(t) Z dt.

θ

  i (t)v i (t) with v i (t) ∈ ∂f i (ω(t)) for a.e. t ∈ [0, T ].Proof. By the Fenchel extremality relation,(B.2) v ∈ ∂I(ω) ⇔ I(ω) + I * (v)ω, v (L ∞ (0,T ;H),L 1 (0,T ;H)) = 0.

  (t)f i ) * (z i ) | m i=1 z i = v(t) .The same measurable selection argument as the one used in Theorem 7.1.3 gives the existence of measurable functions z i (•) such that(B.3)

  (t)f i ) * (z i (t)), with

  * measurability and L ∞ w * space Let f : [0, T ] -→ Z * , where Z is a Banach space. We say that f is weak * measurable if, for all z ∈ Z, the functionz • f : t ∈ [0, T ] -→ f (t), z ∈ Ris strongly measurable. Note that we abusively note z • f , as if z was a linear functional on Z * . In fact this notation makes sense if one sees z as an element of Z * * through the canonical embedding Z ֒→ Z * * . For example, Borel measurable functions from [0, T ] to Z * are in particular weakly * measurable Define L ∞ w * ([0, T ], Z * ) as the vector space of bounded weak * measurable functions. Consider the following equivalence relation on L ∞ w * ([0, T ], Z * ): f ∼ g ⇔ ∀z ∈ Z, for a.e. t ∈ [0, T ], f (t), z = g(t), z . Define then L ∞ w * ([0, T ], Z * ) as the quotient of L ∞ w * ([0, T ], Z * ) by this equivalence relation. It is a Banach space, once equipped with the essential supremum norm (see [308, Corollary VI.4 p.78 and Remark VII.2 p.89]). As announced (see [308, Corollary VII.4 p.95]): Theorem B.3.1. L ∞ w * ([0, T ], Z * ) is isometrically equal to the dual space L 1 ([0, T ], Z) * . The duality pairing is defined as follows: ∀g ∈ L ∞ w * ([0, T ], Z * ), ∀f ∈ L 1 ([0, T ], Z), g, f := T 0 g(t), f (t) Z * ×Z dt.

  Let f : H -→ R ∪ {+∞}. We say that f satisfies the Kurdyka-Lojasiewicz property at x ∈ H if there exists a neighbourhood B(x, δ) of x, and a concave desingularizing function ϕ : [0, η[-→ [0, +∞[, such that the Kurdyka-Lojasiewicz inequality

	Typical examples of desingularizing functions are the functions ϕ(t) = C θ t θ , for C > 0 and θ ∈]0, 1].
	Definition 3.1.1.

  we can use a classic descent lemma 2.1.13 to obtain, after basic calculus, that H 1 and H 2 are satisfied with

  4.1 Splitting methods with errors and variable metric 4.1.1 The Alternating Forward-Backward (AFB) Method Let H 1 , . . . , H p be Hilbert spaces, each H i provided with its own inner product •, • H i and norm • H i . If there is no ambiguity, we will just note x i instead of x i H i . Set H :=

  Note that (x 1 , ..., x s ) is a global minimum for f if and only if x 1 = ... = x s ∈ S, and that we recover (4.23) (respectively (4.25)) if there is only soft (respectively hard) constraints. This penalization function fits well with the setting of the previous chapter, since it is the sum of nonsmooth functions with separated variables (namely, δ C i (x i )), and of a smooth function(4.30) h(x 1 , ..., x s

	p
	w j = 1.
	j=1

1 , ..., w p are nonnegative weights satisfying (s -1)w +

  Proposition 4.2.6. Let D 1 , ..., D p be nonempty, weakly closed and prox-regular subsets of H.

Let s ě 1 and consider the function h defined in (4.30) with (s -1)w + p j=1 w j = 1. Then, for all x ∈ p j=1 D k , there exists a neighborhood U of x in H such that h is continuously differentiable on U s . Its partial gradients are ∀i

  can now state our capture result: Theorem 4.2.8. Let C 1 , ..., C s , D 1 , ..., D p be a collection of nonempty weakly closed subsets of H with D 1 , ..., D p supposed to be prox-regular. Suppose that f , defined by (4.29), satisfies the K L property at some x ∈ S. Then if each x i,0 is close enough to x, the sequence generated by (4.31) has finite length and converges to some (x ∞ , ..., x ∞ ) ∈ S s .Proof. Let X = (x, ..., x) ∈ H s . By Proposition 4.2.5 there is δ > 0 such that h is of class C 1,1 , and f has the Kurdyka-Lojasievicz property at x both in a set Γ η (x, δ) = {x ∈ B(x, δ) : 0 < f (x) < η}. Therefore, Lemma 4.2.7 tells us that if each starting point is close enough to x, then S( X, δ, r)(i) holds. We can even be sufficiently close to have S( X, δ, r)(ii). So the sequence remains in a ball where ∇h is Lipschitz continuous, and it can be shown that H 0 , H 1 and H 2 are satisfied, following the lines of the proof of Proposition 4.1.3. Thus Proposition 3.2.4 allows us to conclude.

4.2.1.2 The KL property for the penalization function

The convergence result of Theorem 4.2.8 holds under the assumption that the cost function f , defined by (4.29), has the K L property. This property holds, for instance, if all the involved sets are defined in an o-minimal structure. We focus in this section on geometric sufficient conditions on the sets ensuring that f is K L. Definition 4.2.9. Let F 1 , . . . , F m be a family of nonempty closed subsets of H. Its intersection S is said to be strongly regular at x ∈ S if there exists α > 0 and δ > 0 such that

(4.36)

  4.2.15. Note that in this setting, we know precisely from Theorem 4.2.8 that the convergence rate of the algorithm (4.31) is exponential. Remark 4.2.16. Note that the strong regularity of an intersection of sets is equivalent to the metric regularity of an appropriate set-valued mapping (see the works of Kruger [213, Section 8], [214, Section 3.3] or [222, Section 3]

  To our knowledge, this algorithm for solving (P 0 ) is new. See Section 4.2.2.3 for a practical implementation of this algorithm.As a consequence of Theorem 4.1.6, together with the fact that the problem (P

	(P	(2) 0	(1) 0 ) (resp.

  C F (x, •) is sublinear, as the supremum of a family of sublinear functions. Moreover, since Θ is bounded and F is locally Lipschitz, we also see that d C F (x, •) takes finite values and is Lipschitz continuous (see Proposition A.3.4).

Remark 5.1.2. Pay attention to the fact that d C F (x, d) is a scalar, and must not be taken for the directional derivative DF (x, d) ∈ Y. When F is strictly Gateaux differentiable, we show in Proposition 5.1.8 that d C F (x, d) and DF (x, d) are related through the support function of Θ.

  is convex and compact for all i ∈ {1, ..., m}, it follows from Proposition 2.1.3 that

	m
	co
	i=1

  1 p • p is pointwise converging to the indicator function of the unit ball δ B (•). of Proposition 5.2.20. Let us start by proving the formulation 2. Because of the powered norm term, and the fact that H is a Hilbert space, d → 1 p d p + d C F (x; d) is a coercive strictly convex function. Therefore, there exists a unique solution d to the minimization problem (see for instance [21, Proposition 2.3.3])

	Proof (5.46)	min d∈T C (x)	1 p	d p + d C F (x; d).
	Let us show that d =	s p-2
	s	p-1

  Remark 6.1.[START_REF] Aliprantis | Principles of real analysis[END_REF]. If H has finite dimension, then the hypothesis of Lipschitz continuity on bounded sets which appears in item ii) can be removed, since it is equivalent to local Lipschitz continuity.

	Proof. i) By Definition 6.1.3 of a strong global solution, we can write for a.e. t ∈]0, +∞[,
	(6.2)	u(t) + η(t) + v(t) = 0,

where η(t) ∈ N C (u(t)) and v(t) ∈ ∂ C F (u(t)). Taking the scalar product with u(t) in (6.2) gives

  More precisely, there exists η, p: [0, +∞[-→ E such that i) u(t) + η(t) + p(t) = 0 for a.e. t ∈ [0, +∞[, ii) η(t) ∈ N C (u(t)) and p(t) ∈ S(u(t)) for a.e. t ∈ [0, +∞[, iii) η, p ∈ L 2 ([0, T ], E) for each T > 0.Proof of Theorem 7.1.3. Under our assumptions, and thanks to Proposition 7.1.2, we can apply Theorem 7.1.4 to obtain a strong global solution of (SD) satisfying items i-iii). The point now is to prove that θ ∈ L ∞ w * ([0, +∞], Y * ) when Y is separable. The fact that θ is bounded on [0, +∞[ is clear since it takes it values in the bounded base Θ. The whole point is to prove that it is weakly * measurable.

  By(7.13),(7.15), the generalized sequence (u λ ) is uniformly bounded and equi-continuous on [0, T ]. Since E is finite dimensional, we deduce from Ascoli's theorem that, for any 0 < T < +∞, the generalized sequence (u λ ) is relatively compact for the uniform convergence topology on [0, T ]. Thus, we can assume that (u λ ) λ>0 converges uniformly on bounded intervals to some u, and item i) is proved. For item ii), use the weak convergence in L 2 ([0, T ], E) of (u λ ) λ>0 and ( uλ ) λ>0 , together with[START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF] Proposition A.6], to see that uλ λ>0 is bounded in L 2 ([0, +∞[, E) and θ i,λ is bounded in L ∞ ([0, +∞[, R m ). Moreover, by Proposition 7.2.2, for any v ∈ E, λ > 0, and i = 1, 2, ..., m

	w λ→0 --→ Item iii) comes from the following observation: by definition of u λ , u.
				m
	(7.16)	η λ (t) = -uλ (t) -	i=1	θ i,λ (t)∇f i,λ (u λ (t)),
	where ( uλ ) (7.17)
	and
	(7.13)	sup	+∞	uλ (t) 2 dt < +∞.
		λ>0	0
	Since ( uλ ) λ>0 is bounded in L 2 ([0, +∞[, E), we can assume that it converges.
	From					t
		u λ (t) = u 0 +	uλ (τ )dτ,
						0
	and Cauchy-Schwarz inequality, we obtain
	(7.14)	u λ (t) ď u 0 +	√	t	t	uλ (τ ) 2 dτ	1 2	.
						0
	Combining (7.13) with (7.14) we deduce that, for any T > 0
	(7.15)	sup

λ u λ L ∞ ([0,T ];E) < +∞.

  The metrizability assumption ensures, via Proposition B.2.3, that θµ is well-defined. Use[START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF] Theorem 11.54] to verify that θµ ∈ Θ. It suffices to apply[START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF] Lemma 11.45] (see Proposition B.2.4) to obtain the result.In other words, this Lemma with the above discussion say that the families {f θ } θ∈Θ and {f µ } µ∈P R (Θ) are equal. Since the metrizability of Θ is ensured when Y is separable (see Proposition C.0.2) this proves Theorem 7.3.3.

  Just use the continuity at zero in the previous Lemma.Remark 7.3.18. Clearly, using Lemma 7.3.16 to prove Proposition 7.3.17 is kind of using a bazooka to kill a fly. Nevertheless we chose this way because we will need, later, to obtain some compactness for the familiy {F λ (x)} λ in C(Θ) (see LemmaLemma 7.3.16).

		3.6.
	Hence, we case assume in generality that (7.59) holds. Then, we can use Lemma 7.3.15 to
	obtain that Ψ(λ α )	C(Θ) --→

α∈A Ψ(λ), which ends the proof. Proposition 7.3.17. Let F : E -→ Y be a convex function bounded from below, with E finite-dimensional. Then, for all x ∈ E, lim λ→0

F λ (x) = F(x) in C(Θ).

Proof. If one wants to prove Proposition 7.3.17 directly, note that the net (F λ (x)) λ is monotone in C(Θ) when λ ↓ 0, so we can apply Dini's monotone convergence theorem(see

[START_REF] Aliprantis | Principles of real analysis[END_REF] Theorem 9

.04, p.70] 3 ).

  1. We consider satisfied its hypotheses, i.e. we suppose that F : E -→ Y is a convex locally Lipschitz function, with E Euclidean and Y a separable Banach space. Let u 0 ∈ E be fixed. For all λ > 0, F λ : E -→ C(Θ) is a locally Lipschitz function, whose derivative DF λ is globally 1 λ -Lipschitz continuous. Hence, we can use Theorem 7.1.3 to obtain the existence of a trajectory u λ : [0, +∞[-→ E such that :

  T ] is compact and u continuous, u([0, T ]) is compact in E. But F is locally Lipschitz continuous, so there exists some ε > 0 such that F is Lipschitz continuous on u([0, T ]) + εB E .

		t ě 0,
	(7.69)	η λ (t) = uλ (t) + µ λ (t) • DF λ (u λ (t)) .
	Since [0, Hence, using Proposition 5.1.7,
	(7.70)	∃M > 0 such that x

* ď M ∀x * ∈ ∂ C F (x), ∀x ∈ u([0, T ]) + εB.

Since u λ converges uniformly to u on [0, T ], we can say that u λ ([0, T ]) ⊂ u([0, T ]) + εB for λ close enough to zero. From Theorem 7.3.7 iii) and (7.70), it follows that

(7.71) ∀λ ∼ 0, ∀t ∈ [0, T ], DF λ (u λ (t)) ď M,

and since µ λ (t) M R (Θ) = 1, we deduce (7.72) ∀λ ∼ 0, ∀t ∈ [0, T ], µ λ (t) • DF λ (u λ (t)) ď M.

Combining (7.72), (7.64) and (7.69), we obtain that the net (η λ ) λ is bounded in L 2 ([0, T ], E), hence weakly relatively compact. So, by taking eventually a subnet, we can say that η λ weakly converges to some η ∈ L 2 ([0, T ], E). Since we know that u λ strongly converges to u in L 2 ([0, T ], E), by using a classic argument involving the maximal monotonicity of N C (see

[START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF] Proposition 2.16]

), it follows that η(t) ∈ N C (u(t)) for a.e. t ě 0.

  7.3.21. For a.e. t ě 0, u(t) + N C (u(t)) + ∂ C F (u(t)) ∋ 0. Proof. Consider any w ∈ L ∞ ([0, T ], E) and λ > 0. Since F λ is convex and differentiable, we have for a.e. t ∈ [0, T ]

	(7.82)

  To get a global solution on [0, +∞[, we do a stronger hypothesis on the gradients. Theorem 8.1.5. Suppose that H has finite dimension, and that the gradients ∇f i are globally Lipschitz continuous. Then, for all t 0 ∈ R, (u 0 , u0 ) ∈ H × H, there exists u :[t 0 , +∞[-→ H of class C 2 , such that for all t ∈ [t 0 , +∞[, with u(t 0 ) = u 0 , u(t 0 ) = u0 .Proof of Theorem 8.1.5. Proposition 8.1.3 provides us a local solution and, using Zorn's lemma, we can suppose that it is a maximal solution u : [t 0 , T [-→ H, with T ∈ [t 0 , +∞]. The whole point is to prove that T = +∞. For this, we argue by contradiction by supposing that T < +∞. We will show that the solution does not blow up in finite time, and extend it at T to obtain a contradiction.Using the fact that the gradients are globally Lipschitz continuous, we obtain from Proposition 7.1.2 the following global growth property for s :

	(8.8) ü(t) = -γ u(t) + s(u(t)) (8.11) d dt 1 2 h

(8.9)

∃c > 0 s.t. ∀u ∈ H, s(u) ď c(1 + u ).

From this growth condition, we will obtain some energy estimates on the trajectory. Let us show that u and ü lie in L ∞ ([t 0 , T ], H). For this, we consider as before

(8.10) φ : H 2 -→ H 2 , (u, v) → φ(u, v) = (v, -γv + s(u)).

By defining x(t) := (u(t), u(t)) for all t ∈ [t 0 , T [, we see that ẋ(t) = φ(x(t)) on [t 0 , T [. Define h(t) := x(t)-x(t 0 ) on [t 0 , T [, which is continuous on [t 0 , T [. Equip H 2 with the scalar product inherited from H, and note that h 2 is derivable on [t 0 , T [, so we can write for all t ∈ [t 0 , T [ : 2 (t) = ẋ(t), x(t)x(t 0 ) = φ(x(t)), x(t)x(t 0 ) ď φ(x(t)) h(t).

  Proof. By hypothesis, u and ∇φ • u are locally Lipschitz continuous, hence differentiable almost everywhere. So, from d dt (φ • u)(t) = ∇φ(u(t)), u(t) , we have for a.e. t ∈ I

	(8.19)	d 2 dt 2

4 (we left it in the Appendix) to obtain (8.15) h(t) ď t t 0 c 3 (1 + h(s)) ds for all t ∈ [t 0 , Tε].

  1 γ ], u0 = λs(u 0 ) satisfies(8.26).

	Remark 8.2.6. Observe that the set of vectors satisfying (8.26) recalls the notion of pseudo-
	gradient introduced by Miglierina [247].
	Proof of Corollary 8.2.5. We saw in Corollary 8.2.3 that the conclusion holds whenever f i (u 0 )-
	E i (t 0 ) ě 0. This condition, once rewritten, is exactly (8.26). Now, consider u0 = λs(u 0 ) for some λ ∈ [0, 1 γ ]. We recall that this steepest descent direction satisfies for all i ∈ {1, ..., m}, see (8.2),
	(8.27)

  Proposition 8.2.7. (Energy estimations) Let u : [t 0 , +∞[-→ H be a bounded global solution of (ISDF) satisfying HP. Then, (i) For all i ∈ {1, ..., m}, E i

  Like in Chapter 6, it is applied to the set S := {x ∈ H | f i (x) ď lim t→+∞ f i (u(t)) for all i ∈ {1, ..., m} }, for which (i) is easy to obtain. The key point to prove the Féjer property (ii) is that h(t) := Lemma 8.2.10. Let h ∈ C 1 ([t 0 , +∞[, R) be a positive function satisfying ḧ + γ ḣ ď g where γ > 0 and g ∈ L 1 ([t 0 , +∞[, R). Then lim Once obtained the weak convergence of the trajectory, the characterisation of its limit point as a weak Pareto point is a direct consequence of the upper semi-continuity of u ⇒ co ∇f i (u) (see Proposition 5.1.11) :Lemma 8.2.11. Assume that the objective function {f 1 , ..., f m } are convex. If u n n ∈ co ∇f i (u n ), then 0 ∈ co ∇f i (u ∞ ).Proof of Theorem 8.2.8. Since u(•) is bounded, there exists some t n → +∞ such that u(t n ) converges weakly to some u ∞ . For all i ∈ {1, ..., m}, since f i is convex continuous, it is in particular weakly semi-continuous. Hence, using Proposition 8.2.7 we get(8.33) f i (u ∞ ) ď lim inf To obtain convergence of the trajectory through Opial's Lemma, it remains to prove the Fejer property (ii). That is, given some z ∈ S, prove that lim z 2 for all t ě 0. Since u is absolutely continuous, then h is twice differentiable for a.e. t ∈ [0, +∞[, and For any i ∈ {1, ..., m}, we use the monotone property of E i and z ∈ S (recall that E ∞ (u(t))) together with the convexity of f i , to obtain for all t ∈ [0, +∞[ :∞ i ě f i (z) ě f i (u(t)) + ∇f i (u(t)), zu(t) . ′ (t),where the right member of(8.39) lies in L 1 ([t 0 , +∞[, H) (see Proposition 8.2.7). Thus, hypothesis of Lemma 8.2.10 is satisfied, and lim t→+∞ h(t) exists. It follows from Opial's Lemma that u(t) weakly converges to some u ∞ ∈ S. It remains to prove that u ∞ is a weak Pareto. In (ISDF), we have -ü(t)γ u(t) ∈ co ∇f i (u(t)), where (see Proposition 8.2.7) Thus, we can apply Lemma 8.2.11 to obtain 0 ∈ co ∇f i (u ∞ ). Following Theorem 5.2.10, this is equivalent for u ∞ to be a weak Pareto point. Remark 8.2.12. If the objective functions are not convex, we still can say something on the limits points: each weak limit point of a bounded trajectory of (ISDF) is a critical Pareto point (see Proposition 8.2.7 and 8.2.11).

	The convergence essentially relies on Opial's Lemma that we recall below (note Ω[u(t)] the set of weak sequential cluster points of the trajectory) : Lemma 8.2.9 (Opial). Let S be a non empty subset of H, and u : [t 0 , +∞[→ H. Assume that 1 2 u(t) -z 2 satisfies a differential inequality. We will use for this the following result from [29, Lemma 4.2] or [30, Lemma 2.3] : t→+∞ h(t) exists. w --→ n→+∞ u ∞ and u * n --→ ḣ(t) = u(t), u(t) -z , (8.34) ḧ(t) = ü(t), u(t) -z + u(t) 2 . (8.35) A linear combination of (8.34) and (8.35) gives (8.36) i = lim E i (t) = f i (u(t)) + 1 γ (f i • u) ′ (t) + u(t) 2 (8.38) ě E u n w --→
	(i) (ii)	Ω[u(t)] ⊂ S; for every z ∈ S, lim

t→+∞ u(t)z exists.

Then u(t) weakly converges to some element u ∞ ∈ S.

n→+∞ 0 with u * n→+∞ f i (u(t n )) = lim t→+∞ f i (u(t)). This proves that Ω[u(t)] ⊂ S = ∅. t→+∞ u(t)z exists. Define h(t) := 1 2 u(t) ḧ(t) + γ ḣ(t) = u(t) 2 + -ü(t)γ u(t), zu(t) . Let θ i (t) ∈ ∆ m be such that -ü(t)γ u(t) = m i=1 θ i (t)∇f i (u(t)), then we can rewrite (8.37) ḧ(t) + γ ḣ(t) = u(t) 2 + m i=1 θ i (t) ∇f i (u(t)), zu(t

) . t→+∞ f i Thus, it follows from (8.37) and (8.38) that (8.39) ḧ(t) + γ ḣ(t) ď 2 u(t) 2 + 1 γ m i=1 θ i (t)(f i • u) n→+∞ u ∞ and liminfess t→+∞ ü(t) + γ u(t) = 0.

  2.1.Proposition A.2.4. Let x ∈ Ω ⊂ X, then T B Ω (x) is closed.Proof. Let d n be a sequence in T B Ω (x) strongly converging to some d ∈ X. Then there exists for all n ∈ N a sequence d k n -→ k→+∞ d n and t k n -→ k→+∞ 0 such that for all k ∈ N we have x + t k n d k n ∈ Ω. So for all n ∈ N, there exists some k n ∈ N such that for all k ě k n we have one can see that dn → d, tn → 0 with x + tn dn ∈ Ω. This achieve the proof.Proposition A.2.5. Let u : I ⊂ R -→ X and f • u : I -→ R ∪ {+∞}, where I is an open interval of R. Suppose that both u and f • u are derivable at t ∈ I, and that u(t) ∈ dom ∂ B f . Then,(A.16) (f • u) ′ (t) = x * , u(t) , ∀x * ∈ ∂ B f (u(t)).Proof. Let x * be in ∂ B f (u(t)), then x * , u(t) ď d B f (u(t), u(t)) where

	(A.15)	d k n -d n ď	1 n	and t k n ď	1 n	.
	By setting dn := d kn n and t kn n d B f (u(t), u(t)) =	lim inf h↓0, d ′ → u(t)
		ď lim inf h↓0, ǫ→0

f (u(t) + hd ′ )f (u(t)) h

  inf t∈R {x * + λe ′ ĺ X * te ′ } = inf t∈R {x * ĺ X * (tλ)e ′ } ĺ X * (t)e ′ }λ = σ Θ (x * )λ.207 Definition B.2.1. We say that f : T -→ Z is strongly measurable (or Bochner measurable) if there exists a sequence (f n ) n∈N of simple functions such that for µ-a.e. t ∈ T , f n (t) Strong measurability is related to Borel measurability by the following Pettis's result 2 . Theorem B.2.2 (Pettis's measurability Theorem,[START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF] Lemma 11.37]). A function f : T -→ Z is strongly measurable if and only if it is Borel measurable and is essentially separably valued3 .In particular, we see that strong and Borel measurability coincide if Z is a separable Banach space. An easy example of strongly measurable function are the continuous functions: Proposition B.2.3. Let f : T -→ Z be continuous, and assume that (T , τ ) is metrizable. Then f is strongly measurable.Proof. It suffices to exploit the fact that f is continuous on a compact metric space, hence uniformly continuous, to build a sequence of simple functions pointwise converging to f . Let d be the metric inducing the topology of T . Using the uniform continuity of f , we can say that for all n ∈ N, there exists α n > 0 such that(B.1) ∀t, s ∈ T , d(t, s) ď α n ⇒ f (t)f (s) ď 1 n .Since a compact metric space is totally bounded [5, Theorem 3.28], we can cover T with a finite number of open balls of radius αn 2 . Using a finite number of boolean operations on these balls, we obtain a disjoint covering of T T ⊂ where the B n k are Borel sets, with diameter lower than αn 2 . For all k ∈ {1, ..., K n }, define

	= inf
	Kn
	B n k ,
	k=1
	χ B n k : T -→ R t -→ 1 if t ∈ B n k , 0 if t / ∈ B n k ,

t∈R {x * Z --→ n→+∞ f (t).

We are aware of the fact that a notion of tame function is developed in[203,

[START_REF] Ioffe | An Invitation to Tame Optimization[END_REF]69], but we will not use it here.

Who was a co-worker of Debreu at the University of California, Berkeley.

In fact a strict cone, as defined later in Chapter 2, since it does not contain the origin.

Just take d = 0 in (1.6).

Pay attention to the fact that one of their main lemmas is false. Indeed, the authors prove that the steepest descent vector field s : R n -→ R n defined in (1.6) is locally Lipschitz continuous. We will see in Chapter

6, Section 6.1.3, that this is false, through a counter-example.

In fact, the fact that all polytopes are polyhedrons characterizes the finite dimension, since polyhedrons cannot be compact in an infinite dimensional space.

In fact, given a general topological vector space (X, τ ), we can define the polar of A ⊂ X as a set in (X, τ ) * . Hence, when considering a set A in the dual of a Banach space X * , it is enough to endow it with the weak * topology in order to obtain A * in the primal X, since (X * , w * ) * = X.

The pointwise topology is the locally convex topology τpw on L(X, Y) defined by the family of seminorms {px} x∈X , where px : A ∈ L(X, Y) → Ax Y . This topology is also called the weak operator topology.

In fact, we can even show that the Fenchel subdifferential of f coincides on dom ∂f with the Fenchel subdifferential of its closed convex envelope, see[START_REF] Zalinescu | Convex analysis in general vector spaces[END_REF] Theorem 2.4.1].

Take for instance f :x ∈ R → -|x| ∈ R, such that d B (0, d) = -|d|.

Consider for instance f (x) = -|x|, for which ∂ F f (0) = ∅.

In fact, in reflexive spaces, the Fréchet subdifferential can be seen as a weakly Bouligand subdifferential, since it is exactly the set of linear minorants of a weak Bouligand derivative d Bw f (x; d), see[START_REF] Kruger | On Fréchet subdifferentials[END_REF] Proposition 1.17] or[START_REF] Mordukhovich | Variational analysis and generalized differentiation I: Basic theory[END_REF] Theorem 1.10].

See [244, Example 1.7].

Take f (x) = -|x|, whose limiting subdifferential at zero is ∂ L f (x) = {-1; +1}.

Note that Treiman defines the limiting subdifferential trough ε-Fréchet subgradients, which gives a bigger set than our, so the inclusion remains valid. For the equality in reflexive spaces, recall Remark 2.2.12.

In some books, reflexive transitive relations are called preorders, and antisymmetry is asked to be an order.

That is, satisfying 1 p + 1 q = 1 with the convention 1 +∞ = 0.

Most of the contents presented here are strongly inspired by two oral presentations given by A. Daniilidis, given at the SOMACHI

and ISMP

I became aware only very recently of a work of Bégout, Bolte and Jendoubi, in which the authors verify partially this conjecture [55, Proposition 2.8].

A = (ai,j) with ai,j ∼ N (0, 1 M ).

In fact,[START_REF] Haraux | Convergence of bounded weak solutions of the wave equation with dissipation and analytic nonlinearity[END_REF] Theorem 2.1] only states that E verifies the K L inequality at the critical points having the regularity H 2 (Ω). But an elliptic regularity argument ensures that all critical points of E does have this regularity (see[START_REF] Cazenave | An introduction to semilinear elliptic equations[END_REF] Theorem

4.4.3] and[START_REF] Gilbarg | Elliptic partial differential equations of second order[END_REF] Theorem 8.12]).[START_REF] Alaa | Convergence to equilibrium for discretized gradient-like systems with analytic features[END_REF] Sometimes written IMEX for short, see[START_REF] Hundsdorfer | Numerical solution of time-dependent advection-diffusionreaction equations[END_REF][START_REF] Hansen | Convergence of the implicit-explicit Euler scheme applied to perturbed dissipative evolution equations[END_REF].

Letter s stands for steepest.

In other words, C 0 is the projection of zero on C.

A Banach lattice is an ordered Banach space such that, given two elements x, y, we can define their minimum and maximum. Order complete is an assumption which essentially ensures the existence of a supremum for bounded sets. See[START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF] for more details.

We choose this denomination to underline the fact that it is an object living in L(X, Y), in opposition with our ordered subdifferential living in X * . In the mentionned papers, theses objects are also called subdifferentials.

In other words, that gi(x) = 0 for all i ∈ {1, ..., m}.

But most of the results of this chapter can be extended to more general constraints C, see Remark 5.3.1.

The reader might pay attention to the following fact: the theta θ used here is a function with values in Θ, whose generic elements are denoted by θ, which is slightly more italic than θ.

See [5, Section 10.10].

Note that the Theorem there only deals with a sequence of functions, but the proof can easily be adapted to take into account our net (F λ (x)) λ∈]0,+∞[ .

An operator A : H -→ H is said to be β-cocoercive if, for all x, y ∈ H, Ax -Ay, xy ě β Ax -Ay

(see[START_REF] Bauschke | Convex analysis and monotone operator theory[END_REF] Definition 4.4]). For instance, for a convex Fréchet differentiable function f :H -→ R, its gradient ∇f : H -→ H is β-cocoercive ifand only if it is 1 β -Lipschitz continuous. It is the Baillon-Haddad theorem, see [51, Corollary 18.16].

The pointwise topology is the locally convex topology τpw on L(X, Y) defined by the family of seminorms {px} x∈X , where px : A ∈ L(X, Y) → Ax Y . This topology is also called the weak operator topology

For Jameson, 'cone' means pointed convex cone in our setting.

Which is strange since it would prove the equivalence between a purely algebraic property with a topological one.

See [5, Definition 4.21 and Corollary 4.24]

In fact the original Pettis's result is stronger, see[START_REF] Diestel | Vector measures[END_REF] Theorem II.1.2], but the one presented here is sufficient for us.

This means that there exists a closed separated subspace Z0 of Z such that f (t) ∈ Z0 for µ-a.e. t ∈ T .

In fact, as it is stated in[START_REF] Diestel | Vector measures[END_REF] Theorem IV.1.1], the equality L q ([0, T ], Z * ) = L p ([0, T ], Z) * holds if and only if Z * has the Radon-Nykodym property, which is also equivalent for Z to be an Asplund space, due to Stegall's Theorem[START_REF] Stegall | The duality between Asplund spaces and spaces with the Radon-Nikodym property[END_REF]. This class includes the reflexive spaces, and the Banach spaces whose dual is separable.

Acknowledgements

Example 6.3.5. Let F = (f 1 , f 2 ) : R n -→ R 2 be defined for all x = (x 1 , ..., x n ) by

x 2 i -10 cos(2πx i ) + 10 , and consider the constraint C = [-0.5, 2] n . This example is adapted from [248, Test 5], see also [START_REF] Beausoleil | MOSS' multiobjective scatter search applied to non-linear multiple criteria optimization[END_REF]Section 6.3]. Here again, due to the oscillatory behaviour of f 1 and f 2 , the set of Pareto points is disconnected in R n , and the Pareto front is concave in R 2 .

We compare again the classic scalarization method with the steepest descent method. They are implemented exactly as we described it in the previous example, and we choose the initial data randomly. The results are presented in Figures 6.12 and 6.13. We observe that, with the increase of the dimension n, both methods present difficulties to recover the Pareto front. The trajectories tend to be attracted by some particular components of the Pareto front. An explanation could be the fact that the density of the Pareto set is not uniform (see [START_REF] Beausoleil | MOSS' multiobjective scatter search applied to non-linear multiple criteria optimization[END_REF]). Figure 6.12: From left to right, and top to bottom: 1000 samples of the scalarization method, with n taking respectively the values 1, 2, 3, 50.

Chapter 7

The continuous steepest descent: existence of trajectories This chapter is devoted to the proof of the existence of trajectories for the dynamic (SD), introduced in Chapter 6. Its main result is: Theorem 7.0.1. Suppose that E is an Euclidean space, and let C ⊂ E be a nonempty closed convex set. Suppose that Y is a separable Banach space, ordered by a closed convex cone with nonempty interior. Let F : E -→ Y be a locally Lipschitz continuous function, being convex and bounded from below. Then, for all u 0 ∈ E, there exists a strong global solution u : [0, +∞[-→ E of (SD) u(t) + (N C (u(t)) + ∂ C F (u(t))) 0 = 0 satisfying the Cauchy condition u(0) = u 0 .

In Theorem 7.0.1, we make two structural hypotheses on the spaces E and Y. First, we suppose that E has finite-dimension. This is essentially because our existence result is based on the following Peano's Theorem (see [START_REF] Toledano | Measures of noncompactness in metric fixed point theory[END_REF]Theorem 2.8]), and not Cauchy-Lipschitz:

Theorem 7.0.2 (Peano). Let φ : R n -→ R n be continuous. Then, for all x 0 ∈ H and t 0 ∈ R, there exists some T > 0 and x : [t 0 , t 0 + T [-→ H of class C 1 , such that

The assumption of separability on Y relies on the necessity to use measurability results, like Castaing's measurable selection theorem, see Section B.1 in Appendix. Now we discuss our strategy to prove this existence result. Our idea is to provide a constructive proof of existence, involving a regularization of the operator ∂ C F governing the (SD) dynamic. We aim to adapt to our situation the classical proof of the existence of strong solutions for evolution equations governed by subdifferentials of convex lower semicontinuous functions, see [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]. The idea is to approach the dynamic (SD) by a regularized one, parametrized by a real λ > 0:

where F λ must be defined as a smooth function such that

The problem (SD) λ being easier because of the smoothness of F λ (but not trivial, because of the noncontinuous operator N C ), we should be able to prove the existence of approximated trajectories u λ : [0, +∞[-→ E. Then, it 'suffices' to let λ tend to zero and prove that the net (u λ ) λ>0 tends to a trajectory satisfying (SD). So we will proceed as follows. In Section 7.1, we focus on the smooth case and prove the existence of trajectories under this assumption. Note that the smooth multi-objective case has been addressed by Attouch and Goudou [START_REF] Attouch | A continuous gradient-like dynamical approach to Paretooptimization in Hilbert spaces[END_REF]. Our result derives from an abstract existence iv) for all λ > 0, x ∈ E, F λ (x) ĺ C(Θ) F(x). v) If F is bounded from below by me ∈ Y for some m ∈ R, then for all λ > 0, F λ is bounded from below by m1 ∈ C(Θ).

vi) for all x ∈ E, F λ (x)

Observe that in the monocriteria case, i.e. when (Y, K) = (R, R + ), we recover the results of Proposition 7.2.2.

The rest of this Section 7.3.2 will be devoted to the proof of Theorem 7.3.7. For the sake of presentation, we will divide the result in individual Propositions. We start with item i) of Theorem 7.3.7, which is quite immediate. Proposition 7.3.8. Let F : E -→ Y be a convex function bounded from below, with E finitedimensional. Then, for all λ > 0, F λ : E -→ C(Θ) is convex.

Proof. Let x 1 , x 2 ∈ E, and t ∈ [0, 1]. Using the definition of ĺ C(Θ) and F λ , we have

for all θ ∈ Θ.

Since the latter inequality always holds (see Proposition 7.2.2 ii)), the convexity of F λ is proved.

Now we focus on one of the most important properties of the Moreau-Yosida approximation F λ , which is its regularity. Lemma 7.3.9. Let F : E -→ Y be a convex function bounded from below, with E finitedimensional. Then, for all λ > 0, the family of functions

Indeed, for all x ∈ E, use the triangle inequality and the 1 λ -Lipschitz continuity of f θ,λ to obtain ∇f θ,λ (x) ď ∇f θ,λ (x) -∇f θ,λ (0) + ∇f θ,λ (0) (7.45)

We saw in Proposition 7.2.2 that ∇f θ,λ (0) = -1 λ prox λf θ (0), that we know to be uniformly bounded (with respect to θ) thanks to Lemma 7.3.5. Hence, it follows from (7.45) that ∇f θ,λ sends bounded sets onto bounded sets, uniformly with respect to θ ∈ Θ. Using the mean value theorem, we conclude that f θ,λ is Lipschitz continuous on bounded sets, uniformly with respect to θ ∈ Θ. Lemma 7.3.10. Let F : E -→ Y be a convex function bounded from below, with E finitedimensional. Then, for all λ > 0 and x ∈ E, the application

We easily obtain from (8.15) and T < +∞ that (8.16)

so we can use the Gronwall-Bellman's Lemma (see Lemma C.0.3 in the Appendix), and obtain :

Since the upper bound in (8.17) is independent of ε and t, we deduce that h ∈ L ∞ ([0, T ], R).

From the definition of h, this means that u and u lie in L ∞ ([0, T ], H). Moreover, using the growth condition (8.9), it implies that s

and we can do the same for u. Hence, we can apply Proposition 8.1.3 at t 0 = T with (u 0 , u0 ) = (u(T ), u(T )) to extend the solution u(•), which contradicts its maximality.

As already discussed in Section 6.1.5, the steepest descent vector field governing the dynamic is not Lipschitz continous, neither monotone (even in the convex setting). So we cannot use methods from monotone operator theory, and the question of uniqueness of the trajectories remains open in the general context. Nevertheless, under some assumptions, we still can ensure the uniqueness. Corollary 8.1.6. Let u be a trajectory solution of the Cauchy problem (8.8). Suppose that for all t ∈ [t 0 , +∞[, the family {∇f 1 (u(t)), ..., ∇f m (u(t))} is affinely independent. Then, u is the unique solution to (8.8).

Proof. It is proved in Proposition 6.1.17 that under these hypotheses, for all t ∈ [t 0 , +∞[, the steepest descent vector field is locally Lipschitz in the neighbourhood of u(t). Hence, it suffices to apply the Cauchy-Lipschitz theorem to derive the uniqueness of u.

Qualitative study of the dynamic

Recall that, for a given function φ : H -→ H and a nonempty subset U ⊂ H, we note Lip(φ; U ) the best Lipschitz constant of φ over U , that is Lip(φ; U ) := sup

x-y

. We say that φ is Lipschitz over U whenever Lip(φ; U ) < +∞.

A dissipative system

We start our study of the dynamic by showing that it is a dissipative system. But before, we need the following chain rule: Lemma 8. 8.2. Once computed, we see that the steepest descent vector field is defined according to three areas of the plane (these areas are delimited by red lines in Figure 8.2) :

In this case we plot the trajectories using an explicit discretization in time of (IMOG):

We take γ = 1, τ = 0.05, and consider a randomly chosen initial data. More exactly, generate randomly (u 0 , u0 ), and take u 1 = u 0 + τ u 1 .

Variational analysis tools in Banach spaces

In this chapter, we give in detail the proofs of some results exposed in Chapter 2. This chapter is structured exactly as the introducing Chapter, with three sections corresponding to the ones of Chapter 2.

A.1 Basic topology and differential calculus

Proposition A.1.1. In a Hausdorff locally convex topological vector space (X, τ ), let A ⊂ X be a compact set not containing the origin. Then R + A is closed.

Proof. Let (t α a α ) α∈A ⊂ R + A be a net, converging to x ∈ X. Let us show that x ∈ R + A. By compactness of A, and considering eventually a subnet, one can assume that the net (a α ) α∈A converges to some a ∈ A. Let (p i ) i∈I be a family of semi-norms generating the topology of τ . Since it is assumed that 0 / ∈ A, and that (X, τ ) is Hausdorff, there exists some i ∈ I such that p i (a) = 0.

We have t α a α → x in X and p i (a α ) → p i (a) = 0 in R, so tαaα p i (aα) converges to x p i (a) in X. Now we can write t α = p i tαaα p i (aα) , which tends to p i x p i (a) . Define t := p i x p i (a) ě 0, and use again the Hausdorff property to conclude that x = ta. Proposition A.1.2. Let (X, τ ) be a locally convex topological vector space, and K ⊂ X a polyhedral cone with a nonempty interior. Then X has finite dimension.

Proof. Since K has finite interior, there exists a nonempty open set U ⊂ K. For any x ∈ U , U -x is a neighborhood of the origin which implies that

By definition of a polyhedral cone, there exists m ∈ N and some {a 1 , ..., a m } ⊂ X such that K = R + co{a 1 , ..., a m }. It follows that X = span K = span {a 1 , ..., a m }, and proves the claim. For a function f : X -→ R, we use the following notation to denote its sublevel sets:

Item ii) being proved, suppose now the reverse, that is item i) holds. To prove that F is strictly Frechet differentiable at x, fix some ε > 0. Using the continuity of DF at x in the sense of item ii), we obtain the existence of δ > 0 such that B(x, δ) ⊂ U , and which satisfies (A.9) for all x ∈ B(x, δ), DF (x) -DF (x) ď ε, or, equivalently, (A.10) for all x ∈ B(x, δ), sup

Hence, we proved that for all x ∈ B(x, δ 2 ) and t ∈]0, δ d 2 [, sup

which proves the strict Gateaux differentiability of F at x.

Remark A.1.10. The similarity between the proofs of Propositions A.1.8 and A.1.9 is simply due to the fact that and Frechet differentiabiliy are just two particular insances of bornological differentiabilities, for which this equivalence extends.

A.2 Nonsmooth analysis for extended-real-valued functions

Proposition A.2.1. Let x ∈ Ω ⊂ X. Then T ad Ω (x) = T r Ω (x) if and only if Ω is radial at x. In particular T ad Ω (x) = T r Ω (x) for all x ∈ Ω when Ω is convex.

Proof. Suppose that T ad Ω (x) = T r Ω (x) and take y ∈ Ω. We have that y = x + (yx) ∈ Ω, so yx ∈ T r Ω (x) = T ad Ω (x). So there exists some δ > 0 such that for all t ∈ [0, δ[, x + td ∈ Ω. In other words, [x, y] ∩ B(x, δ) ⊂ Ω. Conversely, suppose that Ω is radial at x, and consider d ∈ T r Ω (x). So we have x + λd ∈ Ω for some λ > 0. By hypothesis, there exists a δ > 0 such that [x, x + λd] ∩ B(x, δ) ⊂ Ω. In other words, for all t ∈ [0, λδ[, we have x + td ∈ Ω, whence d ∈ T ad Ω (x). Now suppose that Ω is convex. In particular it is radial, and the equality

Proposition A.2.2. Let f : X -→ R ∪ {+∞} and x ∈ dom f . Consider the following sets:

Theorem A.3.2 (Krein-Rutman, [START_REF] Jahn | Vector optimization: Theory, applications, and extensions[END_REF]Theorem 3.38]). Let K be a convex closed cone. If K ++ = ∅, then K is pointed. The reverse is true if X is a separable normed vector space. Now we will see that the existence of a base having good properties is directly connected with the properties of K + . We mention in this result the algebraic strict dual cone K ′ ++ which is defined exactly as the strict dual cone K ++ , but with elements in the algebraic dual space X ′ instead of the topological one X * . Despite the fact that it won't be useful in the sequel, it helps to have a better comprehension of what happen here.

Theorem A.3.3. Let K be a convex cone in a locally convex Hausdorff topological vector space (X, τ ), and Θ ⊂ X.

We see that deeper in the 'interiors' of K + we can take e * ( β int

is the base we construct for K. Moreover the Theorem gives us an easy way to construct such bases, it suffices to pick an appropriate e * ∈ K ++ .

Proof. Proof of item i) is based on Zorn's Lemma (see [START_REF] Jahn | Vector optimization: Theory, applications, and extensions[END_REF]Lemma 3.3]), but can also be proved without the axiom of choice if K spans X (see [START_REF] Jahn | Vector optimization: Theory, applications, and extensions[END_REF]Lemma 1.28]). Item iii) is proved in [START_REF] Jameson | Ordered linear spaces[END_REF]Theorem 3.8.4], and makes use of the duality provided by the Alaoglu-Bourbaki theorem. Note that Jameson assume the cone K to be pointed 2 but this is not necessary since both members of the equivalence imply pointedness. The direct implication of item ii) is proved in [START_REF] Jahn | Vector optimization: Theory, applications, and extensions[END_REF]Corollary 3.19], but without mentioning 0 / ∈ cl τ Θ in the hypotheses 3 . It seems that the author made a mistake there. Given a convex base he proves that 0 do not lie in the algebraic closure of Θ, and says that this algebraic closure equals the topological one cl τ B invoking [START_REF] Jahn | Vector optimization: Theory, applications, and extensions[END_REF]Lemma 1.32]. But this result asks the interior of Θ to be nonempty, which is not guaranteed here. So adding 0 / ∈ cl τ Θ in the left member saves the day. The reverse implication is immediate, since it is clear that such Θ is convex (see also item i)), and 0 / ∈ cl τ Θ follows the continuity of e * .

As a corollary, we can prove Theorem 2.3.4:

Proof of Theorem 2.3.4. Suppose that K is a closed convex cone in a Banach space X, and apply item iii) of Theorem A.3.3 to the weakly * closed convex cone K + in (X * , w * ). Suppose in a first time that we have some weakly * compact convex base Θ of K + . We have in particular that Θ is w * -bounded. Moreover a base satisfies by definition 0 / ∈ Θ, and this Θ is weakly * closed, so 0 / ∈ cl w * Θ. Then Theorem A.3.3 applies, and using the fact that (X * , w * ) * = X, we obtain that β int (K + ) + ⊂ X is nonempty. Since K is assumed to be closed and convex, we have (K + ) + = K, and as said at the beginning of this section, the strong topology coincides here with the norm topology. In other words, int K = ∅, and Θ = {x * ∈ K + | x * , e = 1} for some e ∈ int K.

Suppose now that int K = ∅. Take some e ∈ int K and consider Θ = {x * ∈ K + | x * , e = 1}. Applying Theorem A.3.3, we obtain that this is a w * -bounded convex base of K + . Moreover, K + is weakly * closed, so by definition Θ is also weakly * closed. Since bounded weakly * closed sets are exactly the weakly * compact sets in (X * , w * ), the desired property is proved.

Appendix B

Measurability and L p spaces

We present here some definitions and results which are necessary in Chapter 7. For the basic definitions in measure theory, we refer to [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF].

B.1 Measurability

Let (X, τ X ) and (Y, τ Y ) be two topological spaces. Let Σ X be the Borel σ-algebra of (X, τ X ), i.e. the σ-algebra generated by the open sets of (X, τ X ). We say that f :

for all open sets V of (Y, τ Y ). For instance, continuous functions are Borel measurable. Definition B.1.1. Let T > 0 and (X, τ ) a topological space. We say that f :

If S : X ⇒ Y is a set-valued mapping, we say that it admits a Borel measurable selection whenever it exists a Borel measurable function s : X -→ Y such that s(x) ∈ S(x) for all x ∈ X. Theorem B.1.2 (Castaing's measurable selection). Let (X, τ ) be a metrizable separable compact topological space, and f : [0, T ]×X -→ R a Caratheodory function. Then the set-valued mapping

has nonempty compact values, and admits a Borel measurable selection.

The proof of this result can be found for instance in [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Theorem 18.19], or in the book of Castaing and Valadier [START_REF] Castaing | Convex analysis and measurable multifunctions[END_REF] (Lemma III.39 p.36 and the application below).

B.2 Strong measurability and L p spaces

Let (T , τ ) be a compact topological space, and (Z, • ) be a Banach space. We equip T with a measure µ on its Borel sets, so that (T , Σ τ , µ) is a finite measure space. We say that f : T -→ Z is a (µ-)simple function whenever f (T ) is finite, say f (T ) = {z 1 , ..., z N }, and f -1 (z i ) is Borel measurable for each i ∈ {1, ..., N }.

Appendix C

Others

Theorem C.0.1 (Arzela-Ascoli's Theorem). Let X be a compact topological vector space, and E a metric space. Let C(X, E) be the space of continuous functions from X to E, equipped with the uniform metric. Then a set A ⊂ C(X, E) is relatively compact if and only if the family A is equicontinuous, and A(x Proposition C.0.2. Let Y be a separable Banach space, and Θ a weakly * compact subset of Y * . Then, i) (Θ, w * ) is a metrizable and separable topological vector space, ii) (C(Θ), • C(Θ) is a separable Banach space, iii) (P R (Θ), w * ) is a metrizable separable topological space.

Proof. Since (Y, • Y ) is separable, it follows from [START_REF] Fabian | Banach space theory: The basis for linear and nonlinear analysis[END_REF]Corollary 3.104] that (B Y * , w * ) is separable, where B Y * denotes the unit ball in Y * . From [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Theorem 6.30] it follows also that (B Y * , w * ) is metrizable. Since Θ is compact, we have Θ ⊂ nB Y * for some n ∈ N. But (nB Y * , w * ) is metrizable and separable, so any of its topological subspace is also metrizable and separable (see for instance [138, 3.10.9 p.43]). This proves that (Θ, w * ) is metrizable and separable. Now use [START_REF] Fabian | Banach space theory: The basis for linear and nonlinear analysis[END_REF]Lemma 3.102] to obtain that (C(Θ), • C(Θ) ) is separable. Using now the same arguments for (P R (Θ), w * ) than the ones for (Θ, w * ), we finally obtain that (P R (Θ), w * ) is metrizable and separable (see also [START_REF] Aliprantis | Infinite dimensional analysis, a Hitchhiker's Guide[END_REF]Theorem 15.11]).

We give here the two integral forms of Gronwall's Lemma that we used in the proof of Theorem 8.1.5. They can be found in Brezis's book [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]Lemma A.4 & Lemma A.5,.