Keywords: Optimisation convexe, Algorithmes inertiels, Systèmes dynamiques, Analyse de Lyapunov, Condition de croissance Inertial Gradient-Descent algorithms for convex minimization Convex optimization, Inertial algorithms, Dynamical systems, Lyapunov analysis, Growth condition

Titre : Algorithmes de descente de gradient inertiels pour la minimisation convexe.

Résumé Cette thèse porte sur l'étude des méthodes inertielles pour résoudre les problèmes de minimisation convexe structurés. Depuis les premiers travaux de Polyak et Nesterov, ces méthodes sont devenues très populaires, grâce à leurs effets d'accélération. Dans ce travail, on étudie une famille d'algorithmes de gradient proximal inertiel de type Nesterov avec un choix spécifique de suites de sur-relaxation. Les différentes propriétés de convergence de cette famille d'algorithmes sont présentées d'une manière unifiée, en fonction du paramètre de sur-relaxation. En outre, on étudie ces propriétés, dans le cas des fonctions lisses vérifiant des hypothèses géométriques supplémentaires, comme la condition de croissance (ou condition de Łojasiewicz). On montre qu'en combinant cette condition de croissance avec une condition de planéité (flatness) sur la géométrie de la fonction minimisante, on obtient de nouveaux taux de convergence. La stratégie adoptée ici, utilise des analogies du continu vers le discret, en passant des systèmes dynamiques continus en temps à des schémas discrets. En particulier, la famille d'algorithmes inertiels qui nous intéresse, peut être identifiée comme un schéma aux différences finies d'une équation/inclusion différentielle. Cette approche donne les grandes lignes d'une façon de transposer les différents résultats et leurs démonstrations du continu au discret. Cela ouvre la voie à de nouveaux schémas inertiels possibles, issus du même système dynamique.

Remerciements/Ευχαριστήρια

Dans la (probablement seule) partie que les gens liront pendant ma soutenance, je tiens à remercier en premier lieu mes deux directeurs, Charles et Jean-François. Merci de m'avoir encadré tout au long de cette thèse et de m'avoir fait confiance depuis le début. Vos rôles ont été complémentaires et décisifs pour une partie du succès (?) de cette thèse.

Je tiens également à remercier les rapporteurs Jérôme Bolte et Samir Adly d'avoir accepté de lire et d'évaluer ce travail. Grazie a Silvia Villa per aver accetato di valutare questo lavoro e per essere venuta da così lontano per la difesa della mia tesi. Sono molto contento di poter lavorare con te l'anno prossimo. Merci à Hedy Attouch, Guillaume Garrigos et Aude Rondepierre -avec qui j'ai eu la chance de collaborer-pour votre participation au jury, aujourd'hui. C'est avec un grand plaisir que je soutiens ma thèse auprès de gens dont la lecture des travaux m'a procuré de la joie.

Je voudrais aussi remercier l'ensemble du personnel administratif de l'IMB et en particulier Karine et Cathy pour leur aide et leur bonne humeur pendant les pauses. Catherine, Ida et Muriel (Big Mamma) pour leur capacité à répondre de manière efficace à tous les petits besoins des doctorants. Merci aux personnes de la Cellule info d'avoir aussi ajouté une brique d'aide sur le mur de la vie doctorante. À Andreas Hartmann pour toutes les questions-réponses "d'une minute" et surtout Sylvaine pour celles bien plus chronophages, concernant l'administration de l'EDMI. Je veux également remercier tous les membres de l'équipe IOP et plus particulièrement, Nicolas, Edo et Arthur pour tous vos mots et sourires d'encouragement.

Avant de revenir sur ces trois dernières années de thèse à Bordeaux, je dois tout d'abord faire un petit détour par mes deux années de Master (pas toujours très agréables et faciles) à Lyon. Je voudrais tout d'abord remercier mes professeurs, Louis Dupaigne pour être un enseignant de mathématiques exemplaire et pour m'avoir transmis la passion pour les preuves astucieuses en analyse et Simon Masnou pour son excellent cours d'analyse et pour m'avoir dirigé vers Bordeaux pour mon stage de M2, me donnant ainsi la possibilité v vi de continuer avec cette thèse. Mes pensées vont à Dizi mon ami de loin (Aschach), pour tous les bons moments partagés et les discussions que l'on a eues ensemble. À Enzo pour les moments tranquilles de M1 et à Simon pour être mon ami et m'avoir montré un visage différent de l'ambiance normalienne à Lyon. Merci aussi pour tes supers soirées-repas. Un grand merci aux "Grecs" de Lyon : Kanellos, Ntina, Aimilia, Lida et les autres qui m'ont aussi aidé à me détendre dans des moments durs. Un merci spécial à Nicolas (ο αλήτης) pour qui je n'ai pas pu décider jusqu'au dernier moment dans quelle section, française ou grecque, le mentionner. Merci de m'avoir donné du courage ainsi que pour toutes les discussions et expériences "franco-grecques" que l'on a eues ensemble. Finalement, je souhaite dire un grand merci à Francesca pour tous les moments que l'on a pu passer ensemble à Lyon (ou ailleurs) et pour m'avoir fait découvrir cet amour pour l'Italie (et surtout pour la nourriture italienne hein...). Pendant cette période, tu m'as donné le courage qu'il me fallait pour continuer dans les mathématiques, malgré les circonstances difficiles. Aujourd'hui encore, je sens qu'une partie de ma force pour faire cette thèse vient de toi.

Venons-en à Bordeaux. Je souhaite remercier les doctorantes/doctorants (et autres) que j'ai eu la chance de connaître ici. Tout d'abord Elsa, la première-meilleure rencontre et co-bureau à l'IMB. Merci pour ton accueil et ta bienveillance, sans toi mon intégration à l'IMB n'aurait pas été la même. Merci à Jean d'avoir partagé le même bureau et le même directeur que moi pendant quatre mois de stage. Sami -le frère-le plus beau de l'IMB (le jour où les luttes ouvrières vont converger, tes estimateurs s'effondreront). À Philippe pour tes rigolades et tes têtes bougonnes. Merci à Fabien mon premier-deuxième co-bureau et premier frère de thèse pour ta patience et tes réponses à toutes mes questions de 1ère année de thèse, ainsi qu'à Marc l'autre premier frère de thèse pour ta jovialité. Roberto pour toutes les discussions philosophiques autour de la vie et l'enseignement des maths (ah j'ai oublié aussi le prix pour le meilleur pot de thèse). À tous les autres doctorants (vieux ou pas) Bianca, Alice, Jonathan et Nicolas et Nikola pour les premiers moments conviviaux et les soirées "chez Niko" (ou ailleurs). Aux gens du Labri comme Antonin et mon frère de thèse Pierre. À Mohammed, Louis-Marie et Luis pour les repas partagés au Haut Carré, ainsi que pour les matchs de ping-pong au Labri. Revenant à l'IMB, j'aimerais dire un grand merci à mon voisin de bureau Baptiste pour sa bonne humeur quotidienne, pour les petites -longues-discussions chaque matin, sa patience pour venir répondre à toutes mes questions linguistiques (et pas seulement), ainsi que pour toutes les corrections "indécises" de mes mails (tu es vraiment αναποφάσιστος). Merci aussi à l'autre voisin Yann pour ses visites-siestes au labo ainsi que pour son chaleureux sourire à chaque fois que je le revéill... vii euh pardon j'entrais dans son bureau. Une pensée spéciale à mes deuxlong terme-co-bureaux, Thomas et Lara, de m'avoir supporté dans le même espace, même quand mon humeur n'était pas exceptionnelle. À Yiye pour avoir apporté une tonalité exotique pendant les repas au Haut Carré. À Co(co)rentin et (r)Alexandre pour m'avoir rappelé qu' "on mange à midi" chaque jour. Merci aux stagiaires Robin (grand troll), Anaelle et Paul, le tout nouveau doctorant, (et pour répondre à tes questions : tout va bien, et oui, la thèse est une bonne expérience). Petite pause pour remercier Luca. Merci de m'avoir soutenu pendant mes petits séjours à Paris et pour tous tes conseils académiques. Ci vendiamo a Genova o a Nizza. Retour à l'IMB pour remercier Gaston et Abhi pour votre co-présence au labo les samedis et à Antoine pour ta compréhension et ton soutien actif sur le sujet "piedsnus dans le bureau" (ainsi que pour m'avoir rappelé constamment qu' "à Paris 5 ce sont les meilleurs"). Montons au troisième étage pour remercier Davide et Jared (merci d'être passé par mon bureau vers 20h30 pour me souhaiter une "bonne soirée" ces dernières semaines et merci d'avoir donné la réponse-chanson "I will survive" qui mérite un Oscar, sur la question "Quels étaient vos premières impressions en venant à l'étranger"). Merci aux gens de l'équipe "cancer" Cristina, Giuliano, Cynthia, Cedric, Floriane et la toute jeune chercheuse Christèle pour les conversations post-repas divertissantes. Merci à Olivier pour les rencontres-manifs et également pour les discussions intéressantes. À Costanza pour ton humeur et ton tempérament italien caché derrière ton visage impassible ainsi qu'à Robin. Merci aussi aux gens du groupe "international" que j'ai rencontrés à Bordeaux, je cite Eva, Evi, Margarita, Nadia et Giovanni, Luigi, Dimitris et les autres. À Amélie aussi pour ta générosité et ta douceur (ok aussi pour ton humour). Un grand merci à Sergio mon frère méditerranéen pour toutes les pauses cafés, les vacances (estivales ou hivernales) et autres moments divertissants passés ensemble et pour ceux qui viendront dans le futur (proche) bien évidemment (je t'attends toujours en Grèce). À Marco pour toute ton aide mathématique (j'admire ton intuition mathématique et ta manière d'expliquer les choses), je suis sûr que tu feras une excellente carrière académique. Merci aussi de m'avoir accompagné jusqu'à deux heures du matin au labo dans ma journée la plus longue (et probablement la plus stressante) à l'IMB (rentrer à deux sur un V 3 à 2h du matin ce n'est pas facile hein). Merci aussi à Edo pour toutes les soirées post-boulot, les repas partagés et les débats interminables sur des sujets divers (et oui, LeBron n'est plus le joueur qu'il était et les Lakers ne vont pas prendre le trophée cette année...). Merci aussi pour avoir partagé les taches du bureau Lambda quand les autres n'étaient pas disponibles.

Finalement je souhaite remercier spécialement Marc'Anto pour tout ton soutien ces 2 dernières années. Pour être le premier à m'avoir dit "je n'ai viii aucun problème à me considérer en tant qu'anarchiste" dans un laboratoire de mathématiques, pour tous ces moments entre les pauses-café, les repas, les soirées et les manifs où tu m'as fait sentir "proche" de quelqu'un et parce que tu comprends (un peu mieux que les autres) ce que c'est être un "étranger" en France. Pour tous les moments conviviales de détente avec Johanna que je remercie aussi. Pour ta passion du débat et ton esprit libertaire tout court. Tu vas me manquer gros bavard corse. Ti ringraziu per tuttu amicu.

Je souhaite remercier également les gens du cercle Jean Barrué (Ph. et S., E. et P., F. et T., A. et A.) de m'avoir donné l'opportunité de mieux connaître le milieu anarchiste bordelais et de faire ma première émission radiophonique. Salutations libertaires camarades! Περνώντας στην ελληνική στήλη των ευχαριστηρίων, θα ήθελα να ξεκινήσω από μπρος προς τα πίσω (Forward-Backward1 ). Ξεκινώντας λοιπόν με τους "Αμί μπορντελαί" (ναι έτσι λέγονται οι του Μπορντώ), θα ήθελα να ευχαριστήσω το Σπύρο και την Ειρήνη, για όλες τις (ελληνικές) στιγμές στο Μπορντώ, τα παγωτά, τις βόλτες, τα τραπέζια που κάναμε και τα ασταμάτητα πειράγματα του Σπύρου (είσαι αδιόρθωτος). Τη Δήμητρα και τη Μάρα, τη γνωριμία "του τελευταίου καλοκαιριού" για όλες τις κουλ στιγμές και τα κουτσομπολιά που ανταλλάξαμε (σας περιμένω Γένοβα). Τέλος, τη Δάφνη που τόσο πολύ ταιριάξαμε από την πρώτη στιγμή, για όλες τις ενδιαφέρουσες συζητήσεις που είχαμε (στο Μπορντώ, στην Αθήνα, στο Λιτόχωρο, στην (εξωτική) Ραψάνη και ελπίζω που θα έχουμε στο μέλλον στην Αστυπάλαια ή αλλού).

Πηγαίνοντας πίσω στο χρόνο, θα ήθελα να ευχαριστήσω τους "γνήσιους" (και παλιούς) κατερινιώτες συντοπίτες. Τον Αντώ για την παρουσία σου στην Κατερίνη και τις βραδίες στην πλατεία και στο "μαγαζί". Τον Μπούμπου για όλες εκείνες τις στιγμές που το μοναδικό στήριγμα που είχαμε στη ζωή μας, ήταν ο ένας τον άλλο (κι αν κάποτε διαβάσεις αυτές τις γραμμές, να ξέρεις ότι ακόμη σε σκέφτομαι και περιμένω το μεγάλο ταξίδι που μου υποσχέθηκες..). Θα ήθελα να ευχαριστήσω τον (επίσης συντοπίτη) Τάσο για όλες τις (αλήτικες) στιγμές που ζήσαμε μαζί στην Κατερίνη και στη Θεσσαλονίκη, για το "σπρώξιμο σου" να ασχοληθώ με τα κοινά και για τον αθεράπευτα (αυτοκαταστροφικό) ποιητικό χαρακτήρα σου. Από τους μαθηματικούς συμφοιτητές στη Θεσσαλονίκη θα ήθελα να ευχαριστήσω ιδιαίτερα τη Μαίρη για τον ειλικρινή και ευθύ χαρακτήρα σου, τον Νικόλα για τις βραδινές (για μένα)-πρωινες (για σένα) κλήσεις και τις συζητήσεις γύρω από τα μαθηματικά και τον κόσμο της έρευνας. Το Μιχάλη για όλες τις καλοκαιρινο-χειμωνιάτικες συναντήσεις και το άφθονο γέλιο που μου χάρισες. Την Κική για τις σούπερ βραδιές ξεφαντώματος και για τις γαλλικές συναντήσεις στο Παρίσι. Θα ήθελα επίσης να ευχαριστήσω τη φίλη μου τη Γεωργία που παρέμεινε φίλη παρ'όλο τον καιρό που πέρασε και ix που ελπίζω να συνεχίσω να βλέπω κάθε φορά που επιστρέφω στην Ελλάδα. Τη Μαίρη και το Χαρηστέλιο για τις ωραίες βραδιές που περάσαμε στη Θεσσαλονίκη και τα γενέθλια στον Τρίλοφο. Την Αναστασία που με το απαράμιλλο στυλ σου δίνεις μια εναλλακτική νότα σε κάθε επιστροφή μου στην Ελλάδα, την Κωνσταντίνα για όλες τις συζητήσεις που είχαμε μαζί και όλες εκείνες τις στιγμές που μου έδειξες την κατανόηση που δεν είχα από άλλους. Τον Αντώνη, για την ευφυία σου και τη δημιουργικότητα σου (αλήθεια το πιστεύω θα ήσουν ένας λαμπρός επιστήμονας φυσικής) και τη Δήμητρα (Μητσάκι) που τόσο πολύ ταιριάζετε (περιμένω διακοπές μαζί σας..). Θα ήθελα επίσης να ευχαριστήσω τους συντρόφους από τη Libertatia παλιούς και καινούριους (τη Μυρτώ, το Θανάση, το Στέλιο, τον Αλεξ(ψηλό), το Γιώργο (Βετζέτα), το Δημήτρη, το Χάρη και τους υπόλοιπους) για όλες τις ευχάριστες και όχι τόσο ευχάριστες στιγμές που ζήσαμε από κοντά ή μακριά όπως το κάψιμο της Libertatia. Η φωτιά δε μας καίει σύντροφοι, η φωτιά καίει μέσα μας. Ένα τεράστιο ευχαριστώ στο φίλο μου και σύντροφο Στάθη (που τα κατάφερες και ήρθες τελικά στο Μπορντώ για να δεις την παρουσίαση) για τη συμπαρασταση σου όλα αυτά τα χρόνια, για όλες τις βαθιές συζητήσεις από κοντά κι από μακριά και για την εμπιστοσύνη σου σε μένα. Κι όπως κάποτε μου μετέφερες τα λόγια του Τσε, αξίζει να υπάρχεις φίλε μου για ένα όνειρο κι ας είναι η φωτιά του που θα σε κάψει. Φτάνοντας στο τέλος θα ήθελα να ευχαριστήσω ιδιαίτερα την Αννα, πραγματική συντρόφισσα στη ζωή που πολλές φορές με καταλαβαίνεις ίσως καλύτερα από τον ίδιο μου τον εαυτό. Σ'ευχαριστώ για όλη τη δύναμη, το κουράγιο και την αγάπη που μου έδωσες όλα αυτά τα χρόνια και συνεχίζεις να μου δίνεις. Για όλες αυτές τις φορές που γέμισες κάθε φορά μ'αυτό που χρειάζόταν τα μισο-άδεια μου ποτήρια ώσπου αυτά να γίνουν μισο-γεμάτα. Σ'ευχαριστώ.

Θα ήθελα επίσης να ευχαριστήσω ξεχωριστά την Αθηνά (που ίσως και να ανησυχεί που δεν είδε ακόμη το όνομα της ) που με έζησε σχεδόν σε όλη τη διάρκεια του διδακτορικού από κοντά και μάλλον ξέρει καλύτερα από οποιονδήποτε άλλο τις δυσκολίες που μου παρουσιάστηκαν κατά τη διάρκεια αυτής της προσπάθειας. Σε ευχαριστώ λοιπόν για όλες τις στιγμές που ζήσαμε

Introduction (Français)

L'objectif principal de cette thèse, consiste à étudier et améliorer certaines méthodes pour la résolution des problèmes d'optimisation mathématique.

Une formulation abstraite pour ces problèmes, peut s'exprimer de la manière suivante :

F * = min x∈H F (x) (1)
où H est un espace de Hilbert et F : H -→ R est la fonction modélisant le problème.

En général, même si une telle solution existe, le problème (1) peut être difficile à résoudre, si l'on ne fait pas d'hypothèses supplémentaires sur la fonction minimisante F .

La convexité et la régularité de la fonction minimisante F sont quelques conditions, qui peuvent rendre la résolution d'un tel problème réalisable. La propriété de régularité, exclut des fonctions ayant des comportements "sauvages" (comme x sin 1 Introduction 2 globalement lipschitzien. Les problèmes inverses sont des exemples classiques de problèmes qui peuvent être résolus en utilisant une telle formulation, avec un terme d'attache aux données (partie lisse) et un terme de régularisation (partie non-lisse) lié à la nature précise du problème.

Les méthodes du premier ordre sont des schémas (algorithmes), qui ne font intervenir que le (sous)gradient de la fonction F , plutôt que de l'information d'ordre supérieur, comme la hessienne de F . Ces méthodes se révèlent très efficaces, dans le contexte des problèmes d'optimisation de grande échelle (large-scale optimization), où la dimension de l'espace est grande et les calculs des dérivées d'ordre supérieur peuvent être très couteux à réaliser.

Notre étude suit un chemin "du continu au discret", en passant des systèmes dynamiques continus en (temps), à des schémas discrets (algorithmes), pour la résolution numérique du problème de minimisation (1).

Cette thèse se divise en deux grandes parties. Dans la partie I, on étudie l'évolution des trajectoires, générées par des systèmes dynamiques, qui minimisent la fonction F . Outre l'intérêt de traiter des questions théoriques d'une manière autonome, cette partie joue également le rôle d'un prélude pour la partie II et l'étude des schémas discrets pour l'approximation numérique d'une solution au problème de minimisation de F . En particulier, ces algorithmes peuvent être identifiés comme des schémas numériques d'un système continu associé. Par conséquent, étudier les systèmes dynamiques avant de passer aux algorithmes s'avère offrir plusieurs avantages, car les calculs peuvent être plus faciles et plus simples à traiter que ceux du cas discret (algorithmes). De plus, ils offrent une meilleur intuition (comme par exemple pour la recherche de "bons candidats" des énergies de Lyapunov). Néanmoins, comme on le verra au chapitre 2, l'étude des systèmes continus dépend naturellement des hypothèses de régularité sur F et devient plus compliquée lorsque l'on traite des fonctions non lisses. Dans ce cas, des outils plus avancés de calcul différentiel sont nécessaires afin d'établir les différentes propriétés de convergence. D'un autre coté, les hypothèses sur la régularité de la trajectoire continue, jouent un rôle moins essentiel pour l'analyse des trajectoires discrètes "sans dérivée" générées par un algorithme.

Dans le chapitre 1, on présente certains systèmes dynamiques qui ont été étudiés récemment et qui génèrent des trajectoires pour minimiser la fonction F . Plus précisément, on présente l'analyse pour un système dynamique du deuxième ordre avec un choix particulier de terme de viscosité évanescente :

ẍ(t) + b t ẋ(t) + ∇F (x(t)) = 0 (2)
avec b > 0 le paramètre de friction. Des systèmes comme (2), sont fortement liés au problème de minimisation de la fonction F . En outre, comme il a été Introduction observé dans [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF], le système modélise un choix d'algorithme de descente de gradient inertiel de type Nesterov. Ceci est d'une importance particulière, puisque l'on s'intéresse aux propriétés de convergence de cet algorithme en partie II . Dans le chapitre 1, on présente différents résultats concernant l'équation différentielle (2). Ces résultats donneront une ligne directrice, pour l'analyse des schémas discrets dans la partie II, ainsi que pour l'étude de l'inclusion différentielle associée à (2):

ẍ(t) + b t ẋ(t) + ∂F (x(t)) 0 (3)
où la fonction F est convexe, mais pas nécessairement différentiable. L'étude des inclusions différentielles telle que (3), recouvre une large gamme de problèmes issus du domaine de l'optimisation convexe non-lisse. Les problèmes sous contraintes, les problèmes de type LASSO faisant intervenir la norme 1 , les problèmes de débruitage mettant en jeu la variation totale, ou d'autres problèmes inverse faisant intervenir des termes de régularisation non-différentiables, sont des exemples typiques que l'on rencontre souvent dans ce contexte.

Dans le même esprit que pour l'équation différentielle (2), le système (3) modélise un algorithme de descente de gradient proximal de type Nesterov (voir algorithme 7 dans le chapitre 3), utilisé dans le cadre de la minimisation convexe non lisse structurée.

Dans le chapitre 2, on s'intéresse à l'étude de l'inclusion différentielle (3) et on étend les résultats trouvés dans le cas différentiable pour une solution du système (2), concernant ses propriétés pour la minimisation de F . En particulier, on déduit que la solution de (3) (voir définition 2.1 dans le chapitre 2), satisfait les mêmes propriétés de convergence, en fonction du paramètre de friction b > 0. De plus, dans le cas b < 3, on montre que l'ordre de convergence pour la fonction objectif F (x(t)) -F * est optimal.

Dans la partie II, on étudie certains types d'algorithmes inertiels, dans le cadre de l'optimisation convexe structurée. Ceci est le cas du problème de minimisation d'une fonction F = f +g, qui se décompose en une fonction convexe et lisse f et une fonction convexe (non nécessairement différentiable) g, que l'on suppose proximable (i.e. dont on peut calculer l'opérateur proximal (voir définition 7)). Depuis les travaux d'Euler, Lagrange et Cauchy [START_REF] Cauchy | Méthode générale pour la résolution des systemes d'équations simultanées[END_REF] et l'algorithme de descente de gradient, de nombreuses avancées ont été faites, ces dernières décennies, pour le développement de méthodes efficaces dans le cadre d'optimisation convexe (non-lisse) structurée. L'algorithme du point proximal ( [START_REF] Martinet | Brève communication. régularisation d'inéquations variationnelles par approximations successives[END_REF], [START_REF] Tyrrell | Monotone operators and the proximal point algorithm[END_REF]) ou plus généralement l'algorithme du gradient proximal ( [START_REF] Patrick | Signal recovery by proximal forward-backward splitting[END_REF]), sont parmi les méthodes les plus célèbres pour la résolution des problèmes de minimisation dans ce cadre. Ces schémas, sont relativement simples à implémenter et jouissent de certaines "bonnes" propriétés qualitatives. Pourtant, sans faire d'hypothèses supplémentaires sur la fonction minimisante, la vitesse de convergence que l'on peut obtenir (en termes de la fonction objective par exemple) avec ces méthodes est relativement lente, à savoir d'ordre O(n -1 ) (où n est le nombre d'itérations). Ceci montre que ces méthodes sont sous-optimales parmi celles du premier-ordre, pour lesquelles la complexité optimale est d'ordre O(n -2 ), comme il a été montré dans [START_REF] Semenovich | Problem complexity and method efficiency in optimization[END_REF] (voir aussi [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF]). Cet effet d'accélération et le passage de l'ordre O(n -1 ) à l'ordre optimal O(n -2 ), s'obtient via les méthodes inertielles.

Les méthodes inertielles consistent à ajouter un terme d'inertie supplémentaire à l'étape de descente de gradient (ou l'étape de descente de gradient proximal dans le cadre de minimisation non-lisse), à chaque itération. Les premières méthodes inertielles ont été considérées dans les travaux de Polyak [START_REF] Boris | Some methods of speeding up the convergence of iteration methods[END_REF] (voir aussi [START_REF] Stanley | Convergence rates of iterative treatments of partial differential equations[END_REF]), avec l'algorithme de boule pesante (Heavy-Ball) pour la minimisation des fonctions lisses et fortement convexes. Toutefois ce n'est que dans les travaux de Nesterov [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF], qu'un choix de terme de frottement particulier a été introduit permettant d'accélérer la vitesse de convergence de la fonction objective à un O(n -2 ), pour la minimisation des fonctions convexes et lisses. Le schéma inertiel initial qui a été proposé dans [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF] est le suivant:

t n+1 = 1 + 4t 2 n + 1 2 y n = x n + t n -1 t n+1 (x n -x n-1 )
x n+1 = y n -γ∇F (y n ) (4) avec x 0 = x 1 et t 1 = 1 et γ un pas adéquat. En effet le phénomène d'accélération reste toujours valable pour une plus large gamme de choix de suite t n , vérifiant le règle général suivant :

t 2 n + t n+1 -t 2 n+1 ≥ 0 (5)
Introduction l'algorithme du gradient proximal respectivement, dans le cadre de minimisation non lisse. Plus précisément, dans la partie II de cette thèse, on s'intéresse à un schéma inertiel particulier de type Nesterov, introduit récemment dans [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF] (voir aussi [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] et [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF]), dans le cadre de l'optimisation convexe nonlisse, structurée :

y n = x n + n n + b (x n -x n-1 ) x n+1 = Prox γg (y n -γ∇f (y n )) (6)
où Prox désigne l'opérateur proximal (voir définition 7) et b > 0 est le paramètre de sur-relaxation, mesurant l'ampleur d'inertie appliquée à chaque étape de l'algorithme. On peut remarquer que quand b est petit, le terme inertiel n n+b (x n -x n-1 ) est plus prépondérant, alors que lorsque b est grand, l'inertie devient plus faible. Comme il a été montré dans les travaux récents de [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF], [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF], [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF], l'algorithme (6) a de nombreux avantages, parrapport au schéma initial (4) de Nesterov. En particulier en prenant b ≥ 3 on peut assurer l'effet d'accélération O(n -2 ), tandis qu'en choisissant b > 3, on peut établir la convergence faible des itérés vers un minimiseur de F ( [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF]), ainsi qu'améliorer asymptotiquement l'ordre de convergence qui est o(n -2 ) (voir [START_REF] Attouch | Rate of convergence of inertial gradient dynamics with timedependent viscous damping coefficient[END_REF]). Un deuxième avantage de l'algorithme (6) se base sur le fait que l'on peut l'identifier comme un schéma de différence finies de l'inclusion différentielle (3) (ou (2) dans le cas où F est différentiable). Cette dernière observation éclaire le rôle de l'étude des systèmes dynamiques continus comme (3) (ou (2)), puisqu'ils fournissent une meilleur interprétation du schéma (6), ils apportent une compréhension des résultats et de leurs preuves plus profonde et ils ouvrent la voie pour des extensions possibles dans le futur.

Tandis que le cas b ≥ 3, correspondant à la règle de Nesterov (5) à été beaucoup étudié, celui de b < 3 correspondant au cas de l'inertie "forte" reste inexploré. Dans le chapitre 3, on traite également ce cas. On étend l'étude du schéma (6) dans le cas où le paramètre de sur-relaxation vérifie b < 3, et on obtient des ordres de convergence pour la fonction objective et la variation locale de la suite générée. Pour cela on suit les lignes directrices du cadre continu dans la partie I, en transposant les techniques de Lyapunov au cadre discret.

En outre, on complète ce chapitre, en présentant quelques résultats sur une version inexacte du schéma (6), avec la présence d'erreurs de perturbation (à la fois sur le calcul du gradient de la partie différentiable f et à l'opérateur proximal de la partie non-différentiable g). En particulier on étudie ses différentes propriétés de convergence en fonction des erreurs de perturbations et le paramètre de sur-relaxation b > 0.

De ce qui précède, il est clair que le choix du paramètre de sur-relaxation pour les schémas inertiels et en particulier le réglage du paramètre b dans le schéma (6), joue un rôle important pour les propriétés de convergence de l'algorithme. Dans le chapitre 4 de la partie II, on s'intéresse à ce sujet, en liaison avec quelques hypothèses supplémentaires sur la géométrie de la fonction minimisante F .

En effet, la complexité optimale de O(n -2 ) pour les méthodes du premier ordre (voir [START_REF] Semenovich | Problem complexity and method efficiency in optimization[END_REF] et [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF]) peut être améliorée, si la fonction minimisante F satisfait des conditions géométriques supplémentaires. Un exemple typique d'une telle condition est la propriété de forte convexité (voir définition 5). Dans ce cadre, en considérant par exemple des méthodes de descente (i.e. des méthodes qui génèrent des suites de la fonction objective décroissantes (voir aussi définition 4.9 dans le chapitre 4)), on peut obtenir des suites convergeant linéairement vers l'unique minimiseur de la fonction minimisante F . Des exemples classiques de telles méthodes sont l'algorithme de descente de gradient, ou l'algorithme de descente de gradient proximal dans le cas général, qui s'avèrent être très efficaces dans ce contexte.

En effet, la convergence linéaire reste toujours valable sous des hypothèses plus générales, comme la 2-croissance (ou 2-conditionnement), qui indique que la fonction F , vérifie localement une condition de type Kd(x, X * ) 2 ≤ F (x) -F * où X * = arg min F , d(x, X * ) désigne la distance de x ∈ H à X * et K > 0 est une constante appropriée. Plus généralement, on peut considérer la condition de p-croissance pour un paramètre p ≥ 1 général :

K p d(x, X * ) p ≤ F (x) -F * (7)
pour une constant K p > 0. La condition de p-croissance est aussi associée avec la condition θ-Łojasiewicz [Łoj63, Łoj93] ou inégalité du gradient :

c θ F (x) -F * θ ≤ ∇F (x) (8) où θ = 1 -1 p ∈ [0, 1] et c θ > 0.
Les conditions de croissance comme (7) ou (8), se révèlent être des outils très puissants pour les méthodes de descente du premier ordre, à la fois dans le cadre de minimisation convexe, mais aussi pour le cas non-convexe. En particulier, les propriétés de convergence de ces méthodes sont gouvernées par le paramètre géométrique p ≥ 1, plutôt que par la nature précise de l'algorithme (voir par exemple [BDLM10, FGP15, Gar15, BNPS17] et [START_REF] Garrigos | Convergence of the forward-backward algorithm: Beyond the worst case with the help of geometry[END_REF]). En général, pour ces méthodes, plus la fonction est affutée (sharp) (p proche de 1), plus la vitesse de convergence vers un minimiseur est rapide.

Tandis que les méthodes de descente sont bien adaptées à des conditions géométriques comme (7) ou (8), le rôle de ces conditions sur les propriétés Introduction de convergence des méthodes inertiels comme (6) est moins connu, pour des valeurs générales de p ≥ 1.

Dans le chapitre 4, on s'intéresse à cette question pour l'algorithme (6), dans le contexte de la minimisation convexe et lisse (i.e. quand g = 0 dans (6)). Plus précisément, on étudie les propriétés de convergence du schéma (6), en fonction du paramètre de sur-relaxation b > 0 et les différentes propriétés géométriques sur F , comme l'hypothèse de p-croissance (7). A cette fin on considère une hypothèse géométrique supplémentaire sur la fonction minimisante F , qui exprime le niveau de sa planéité (flatness) autour de ses minimiseurs:

β F (x) -F * ≤ ∇F (x), x -x * ∀x * ∈ arg min F (9)
avec β ≥ 1. La condition (9), a été introduite dans [START_REF] Cabot | On the long time behavior of second order differential equations with asymptotically small dissipation[END_REF] et a été explorée à nouveau dans [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] et [START_REF] Aujol | Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b > 0[END_REF][START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF], pour l'étude de l'équation différentielle (2). Comme on verra dans le chapitre 4, la condition (9), joue un rôle important dans cette analyse et avec la condition de croissance (7), caractérise les fonctions convexes se comportant comme x -x * r pour tout p ≤ r ≤ β au voisinage de leur minimiseurs. Cette étude ouvre la voie pour de nouvelles avancées dans un cadre plus général, comme celui de l'algorithme du gradient proximal inertiel (6) pour la minimisation convexe non-lisse, ou ses versions inexactes.

Introduction (English)

The basic purpose of the current Thesis is to study and improve some methods in order to solve mathematical optimization problems.

An abstract formulation of such a problem can be mathematically expressed as follows :

F * = min x∈H F (x) (1)
where H denotes a Hilbert space and F : H -→ R is the modeling minimizing function. In general, even if a solution of (1) exists, such a problem can be very hard -if not impossible-to solve, without any specific assumptions on the minimizing function F . A possible solution of problem F , is denoted by x * ∈ arg min F . Some of the basic assumptions that can render the resolution of problem (1) feasible, is the convexity and the smoothness property of the minimizing function F . Roughly speaking, smoothness excepts functions with "wild behaviors" (think of functions like x sin 1

x near 0) and permit to upper bound the minimizing function by a quadratic approximation. Convexity is a powerful tool, allowing to develop plenty of methods to estimate efficiently the solution of (1). Convex functions enjoys local-to global property, meaning that every local minima is global (i.e. F * is unique). This transposes problem (1), into the equivalent problem of finding roots of the (sub)gradient of function F .

In the context of the current Thesis we are interested in the case of Firstorder methods for solving composite convex minimization problems. There is a large variety of problems arising in modern times, in domains such as statistics, machine learning, image and signal processing or more generally inverse problems, which can be formulated by such a composite structure and be efficiently solved by such methods.

Composite convex minimization refers to problems as (1), where the minimizing function F is convex (see Definition 4) and has some additional splitting structure information. In the framework of the current thesis this ad-ditional structure information is translated to a particular decomposition of the minimizing function F = f + g into a smooth convex part f and a nonsmooth convex part g. Typical cases with such formulation are the inverse problems, where the minimizing function F is composed by a smooth datafit term, often chosen to be a squared distance from the initial data and a suitable (convex) regularizer term corresponding to the particular nature of the problem (see Chapter 3, for a brief description of these problems).

First order methods refer to schemes (algorithms) that exploit only firstorder information from the minimizing function F , such as gradients or subgradients, rather than higher-order information such as the Hessian of F . These methods turn out to be more suitable, when one has to deal with large-scale optimization problems, where the dimension of the space is large and the computations of second -or higher-order derivatives can be very hard and costful.

Our study follows a continuous-to-discrete path passing from continuous -on time-dynamical systems, to discrete schemes (algorithms), aiming to solve numerically the minimization problem of F .

In particular this Thesis is structured into two major parts. In Part I we are addressing the question of the minimization problem of F in a continuous setting. More precisely we are taking look on the evolution of trajectories generated by a dynamical system (differential equation with dependence on time) that minimizes the function F . Besides treating questions of a theoretical interest in an autonomous way, this part also plays the role of a prelude for the second part where we study some particular algorithms, in order to approximate numerically a solution of the minimization problem of the function F . In fact as we shall see these algorithms can be identified as numerical schemes of the associated dynamical system. Therefore, studying dynamical systems before passing to algorithms turns out to offer multiple advantages, as calculations can be easier and simpler to deal with, than the ones in the discrete case (algorithms). Moreover, they -often-offer a better insight (such as finding "good candidates" for Lyapunov energies for example). Nevertheless, as we shall see in Chapter 2, the study of continuous -on timesystems naturally depends also on the regularity assumptions on F and it becomes more complicated when one has to deal with non-smooth functions. In this case more developed tools for differential calculus are needed in order to establish the different convergence properties. On the other hand, the (additional) regularity assumptions (such as smoothness) of the continuous trajectory becomes a less essential element for the "derivative-free" discrete trajectory generated by an algorithm.

In particular in Chapter 1, we present some special dynamical systems which have been studied recently, which promote motions-trajectories that Introduction minimize F , in a smooth and convex setting. More precisely we take a look to a second-order damped dynamical system studied in [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] (see also [START_REF] Cabot | On the long time behavior of second order differential equations with asymptotically small dissipation[END_REF] and [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF]), with a particular choice of vanishing viscosity term :

ẍ(t) + b t ẋ(t) + ∇F (x(t)) = 0 (2)
where b > 0 denotes the friction parameter. Systems like (2) are linked with the minimization problem of the function F depending on the damping parameter (in our case b t ). In addition, as it was first remarked in [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] (see also [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF]), one of the motivations of studying system (2), is that it corresponds to a particular Nesterov-type inertial gradient descent algorithm for minimizing F . This is of a great interest in our case, since in the second Part we are investigating the convergence properties of this algorithm. In Chapter 1 we revisit some of the results in the existing literature concerning the differential equation (2). These results and their proofs will offer a useful guideline for the forthcoming analysis, both for the algorithms studied in Part II, as also for the corresponding differential inclusion :

ẍ(t) + b t ẋ(t) + ∂F (x(t)) 0 (3)
where the minimizing function F is convex but not necessarily smooth.

Studying differential inclusions such as (2), covers of course a wider class of interesting problems, coming as a subset of the non-smooth optimization domain. Typical examples which are often met in optimization can be constrained minimization problems, LASSO-type problems involving the 1 norm, denoising problems including the total variation norm, or other inverse problems with general non-smooth regularization terms.

In the same way as for the differential equation (2), the importance of studying system (3) comes from the different connections between it and a fast Nesterov-type algorithm in order to solve non-smooth composite convex minimization problems. In particular, it can be derived by (or model inversely) the proximal version of the inertial gradient descent algorithm (see (6) or the i-FB algorithm 7 later on) with a proper implicit or explicit-implicit finite difference scheme respectively.

In Chapter 2, we study the differential inclusion (3) and we extend the convergence results found in the smooth case for a solution of (2), concerning its minimization properties over F . In particular we find that the same fast orders hold true depending on the friction parameter b > 0, for a solution of (3) in some proper sense ( see Definition 2.1 later on). In addition we show an optimality result for the convergence rates of the objective function F (x(t))-F * , in the case where the friction parameter satisfies b ∈ (0, 3). This is done by giving an example of a function which is not everywhere differentiable, hence only treatable in the context of (3). This study offers a better insight and it paves the way for the forthcoming analysis of the algorithms associated to it, for solving non-smooth composite convex minimization problems.

In Part II, we study some inertial algorithms for composite, non-smooth, convex optimization, that is the case of Problem (1) with a function F = f +g, structured by a smooth part f and a (possibly) non-smooth g, which we assume simple or proximable (meaning that we can calculate its proximal operator (see Definition 7)). Starting from the differential setting (i.e. when F is smooth) and the simple case of the classical Gradient-Descent algorithm, which dates back to the works of Euler and Lagrange and Cauchy [START_REF] Cauchy | Méthode générale pour la résolution des systemes d'équations simultanées[END_REF], a lot of progress has been made in the last decades, for developing efficient first-order methods for solving general structured convex minimization problems. Some of the most famous are the proximal-point algorithm ([Mar70], [START_REF] Tyrrell | Monotone operators and the proximal point algorithm[END_REF]) or the more general Forward-Backward splitting algorithm ([LM79], [START_REF] Patrick | Signal recovery by proximal forward-backward splitting[END_REF]). Whereas these schemes are simple to implement and have some good qualitative properties such as generating a relaxation sequence (i.e. the sequence of objective function that converges decreasingly to the minimum), they lack of efficiency in terms of speed of convergence of the values of the objective function (i.e. how fast is the convergence of the objective function to the minimum). Without any further assumptions on the minimizing function F , the -worst case-order of these descent methods is O(n -1 ) (where n is the number of iterations executed). Nevertheless, as it was shown in [START_REF] Semenovich | Problem complexity and method efficiency in optimization[END_REF] (see also [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF]), the optimal -worst case-convergence rate for a general first-order method for smooth and convex minimization is of order O(n -2 ), which consists of an improvement of O(n -1 ). Of course this does not directly imply that schemes such as Gradient-Descent or Forward-Backward are suboptimal, but as it was shown in the seminal work of Nesterov [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF], the optimal order of O(n -2 ) can be achieved. The key-element for this acceleration effect and the achievement of this optimal rate is made by considering inertial (or momentum) type methods. Inertial methods consist in adding an extra inertial term on the gradient step (or equivalently at the proximal-gradient step in the non-smooth setting) at every iteration. Initially the first inertial methods appeared in [START_REF] Boris | Some methods of speeding up the convergence of iteration methods[END_REF] (see also [START_REF] Stanley | Convergence rates of iterative treatments of partial differential equations[END_REF]) with the Heavy-Ball algorithm for strongly convex functions, which can also be seen as multi-step methods. However, it was in the seminal work of Nesterov [Nes83] that the author proposed a particular type of the momentum term, which actually accelerates the convergence of the objective function to the optimal O(n -2 ) for any smooth and convex function. The initial choice of inertial scheme as considered in [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF], starting from x 0 = x 1 Introduction and t 1 = 1 is the following:

t n+1 = 1 + 4t 2 n + 1 2 y n = x n + t n -1 t n+1 (x n -x n-1 ) x n+1 = y n -γ∇F (y n ) (4)
for a suitable step-size γ > 0. In fact the acceleration effect holds true for a more general choice of positive sequences t n , which satisfy the Nesterov's rule, i.e. :

t 2 n + t n+1 -t 2 n+1 ≥ 0 (5)
and as one can remark the choice t n in (4), corresponds to the limiting case of equality in (5). Since these works, a lot of attention has been given to inertial or momentum methods, over the last decades, notably in [START_REF] Güler | New proximal point algorithms for convex minimization[END_REF] and [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF], where the authors extend Nesterov's scheme and the acceleration effect in a non-smooth case for the inertial versions of the proximal point and the Forward-Backward algorithms respectively.

More precisely in this second Part, we are interested in a particular splitting inertial proximal algorithm for non-smooth composite convex minimization, also known as Nesterov-type, introduced recently in [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF] (see also [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] and [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF]), which reads :

y n = x n + n n + b (x n -x n-1 ) x n+1 = Prox γg (y n -γ∇f (y n )) (6)
where Prox stands for the proximal operator (see Definition 7) and b > 0 is the over-relaxation parameter measuring the magnitude of the applied inertia at each step. Notice that, when b is small, then the inertia n n+b (x n -x n-1 ) is stronger, while when b is large, the inertia becomes weaker. As it was shown in the recent works of [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF], [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] and [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF] the algorithm (6) has several additional benefits in comparison to Nesterov's initial choice. In particular, while preserving the optimal rate O(n -2 ) in the case of b ≥ 3, by taking b > 3 one can prove the weak convergence of the sequence generated to a minimizer (see [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF]), as also a slight amelioration of the order of convergence asymptotically which actually is o(n -2 ) (see [START_REF] Attouch | Rate of convergence of inertial gradient dynamics with timedependent viscous damping coefficient[END_REF]). A second major advantage of algorithm (6), as it was shown in [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] and discussed previously, is that it can be identified as a proper numerical scheme of the differential inclusion (3). As already mentioned, this last observation enlightens the role of the study of continuous systems such as (3), since it offers a better interpretation of scheme (6), and gives a better insight on the different convergence results as also on the lines of their proofs and opens the way for future studies, with other possible choices of numerical approximations.

While the case of low-region momentum b ≥ 3, corresponding in Nesterov's rule (5) for acceleration, is given a lot of attention and is widely studied during the last years, the case of high-region momentum b < 3 still remains undiscovered. In Chapter 3 we extend the study of schema (6) in the case of a high-region momentum (i.e when the parameter of over relaxation b in (6) satisfies 0 < b < 3) and we show that some relatively fast convergence rates also hold for both of the objective function and the local variation of the sequence generated. For that, we follow the lines of the Lyapunov analysis made in the continuous setting in Part I for the differential inclusion (3).

In addition we complete this Chapter, with Section 3.4 by giving some new results of an inexact version of the scheme (6), concerning the tradeoff between the magnitude of the perturbation errors (over the computation of the gradient of the smooth part f and of the proximal operator of the non-smooth g) and the inertial parameter b > 0.

From all the above discussion, it is apparent that the choice of the sequence of over-relaxation for inertial schemes and in particular the tuning of parameter b > 0 in the scheme (6), plays a very important role in the convergence properties of the algorithm both in terms of qualitative and quantitative results. In the last Chapter 4 of this Part II, we are addressing this question in link with some additional properties over the local geometry of the minimizing function F .

While the optimal -up to constants-rate for first-order methods is O(n -2 ), where n is the number of iterations (see [START_REF] Semenovich | Problem complexity and method efficiency in optimization[END_REF] or [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF]) for the class of convex and differentiable functions, this complexity can be further improved by making additional hypotheses on the geometry of the minimizing function F . A first classical type of such an hypothesis is the strong convexity property (see Definition 5), which generates linear convergence for descent methods (i.e. methods that generate decreasing sequences in terms of values of F (see also Definition (4.9)) later on Chapter 4 ), such as Gradient-Descent, or Forward-Backward splitting in the general setting (see for example [START_REF] Goldstein | Cauchy's method of minimization[END_REF]). In the case of strongly convex functions, the linear convergence holds also true for some special inertial-type algorithms, such as the Heavy-Ball algorithm or the Nesterov's scheme with a constant over-relaxation term (see algorithms 9 and 10 respectively in Chapter 4 ). In addition, these particular inertial versions are proven to give linear rates with a better (smaller) linear factor than the one of Gradient-Descent or Forward-Backward, which is optimal for Introduction the class of smooth, strongly convex functions ( [START_REF] Semenovich | Problem complexity and method efficiency in optimization[END_REF]). For general descent methods the linear rates can be achieved, even for weaker relaxations of strong convexity, such as the 2-conditioning or 2-growth condition, which is when the function F satisfies locally a condition of the type : Kd(x, X * ) 2 ≤ F (x) -F * , where X * = arg min F , d(x, X * ) designs the distance from x ∈ H to X * and K > 0 is a suitable constant. Extending to more general geometries one can consider the p-growth condition for a general p ≥ 1 :

K p d(x, X * ) p ≤ F (x) -F * (7)
for a suitable constant K p > 0, which is also closely related to the famous θ-Łojasiewicz condition [Łoj63, Łoj93] :

c θ F (x) -F * θ ≤ ∇F (x) (8)
where θ = 1 -1 p ∈ [0, 1] and c θ > 0. Growth-type conditions such as (7) or (8) turn out to provide powerful tools for general descent methods, both in the context of convex and nonconvex optimization. More precisely, general descent methods, enjoy several fast convergence properties depending to the "geometric" parameter p ≥ 1 and not on the precise nature of the algorithm (see for example [BDLM10, FGP15, Gar15, BNPS17] and [START_REF] Garrigos | Convergence of the forward-backward algorithm: Beyond the worst case with the help of geometry[END_REF]). Roughly speaking, for first order descent methods, the sharper the minimizing function is (p close to 1), the fastest the convergence to a minimum is.

Whereas general descent schemes are proven to be well adapted to geometrical properties like (7) or equivalently (8), this is not the case for inertial schemes such as (6), whose convergence properties are largely unknown for functions satisfying p-growth condition (7) for general values of p ≥ 1.

In Chapter 4 we are addressing this question in the context of smooth convex minimization and we extend the study of inertial gradient descent scheme (6) with g = 0. More precisely, we investigate its convergence properties depending on the local geometrical assumptions over F such as the p-growth condition (7) and the over-relaxation parameter b > 0. For that purpose, we consider an additional geometric property on function F , which expresses its flatness-level in a neighborhood of its minimizers :

β F (x) -F * ≤ ∇F (x), x -x * ∀x * ∈ arg min F (9)
with β ≥ 1. Condition (9) was firstly introduced in [CEG09a] and it was afresh explored in [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] and [START_REF] Aujol | Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b > 0[END_REF][START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF], for the study of the differential equation (2). As we shall see condition (9), with β ≥ 1 plays a key-role in our analysis, and together with growth condition (7) with p ≥ 1, characterizes convex functions which behave like x -x * r for all p ≤ r ≤ β, near the set of their minimizers. Our approach again follows the line of the study made in the continuous setting for the dynamical system (2) in the recent work of [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF]. This study opens the road for further developments in more general frameworks such as for the inertial Forward-Backward type algorithm (6) for composite non-smooth minimization problems, or their inexact versions, which corresponds in a larger variety of interesting applications.

A Thesis outline and Contributions

Part I

Chapter 1 In this Chapter we study the second-order differential equation (2). We give some of the results of the existing literature, concerning the convergence properties of a solution of the dynamical system (2). We present the Lyapunov analysis associated to this ODE and discuss about some particular choices of its discretization. The analysis and presentation of these results will provide a useful guide for the forthcoming analysis, both for the continuous, but non-differential setting in Chapter 2 , as also for the discrete case in Chapters 3 and 4.

Chapter 2 In this Chapter we turn our attention to the study of the differential inclusion (3), modeling the inertial proximal-gradient algorithm (6) for convex minimization problems, studied later on Part II. Firstly, we give some notions of a possible solution for such an inclusion. We extend the asymptotic convergence results of the smooth case, as presented in Chapter 1, into the non-smooth one. For this we employ an approximation scheme of differential equations, whose solutions converge to some proper sense to a solution of the inclusion (3). The contributions of this Chapter rely on the journal article [START_REF] Vassilis Apidopoulos | The differential inclusion modeling FISTA algorithm and optimality of convergence rate in the case b ≤ 3[END_REF].

Introduction

give a unified presentation together with the related results. Our analysis follows the one made in the continuous case in Chapters 1 and 2, by adapting the Lyapunov techniques for some properly defined energy sequences. In addition we complete the analysis of an inexact version of this algorithm in the case of presence of perturbation errors on both the gradient of the smooth part and the proximal step of the non-smooth part. We show that the convergence properties shown for the unperturbed version, are relatively robust to these errors, under some control conditions depending on the overrelaxation parameter b > 0. The contributions of this Chapter rely on the journal article [START_REF] Vassilis Apidopoulos | Convergence rate of inertial forward-backward algorithm beyond nesterov's rule[END_REF].

Chapter 4 We are further analyzing the Nesterov-type inertial gradient descent algorithm (6), in the context of smooth, convex minimization problem. We are considering additional local geometrical assumptions on the minimizing function F , such as the growth ( or Łojasiewicz) condition (7), as also the flatness-type condition (9). Combining both of these conditions, permits to derive new convergence rates for this algorithm, depending both on the geometry of the minimizing function, as also on the over-relaxation parameter of the algorithm b > 0. The analysis is still based on Lyapunov arguments, for a proper energy-sequence and it follows the line of the analysis made in the continuous case, in Chapters 1 and 2. The contributions of this Chapter rely on the (submitted) paper [START_REF] Vassilis Apidopoulos | Convergence rates of an inertial gradient descent algorithm under growth and flatness conditions[END_REF].

B Mathematical background

Before passing to the main core of this Thesis, we set up some basic notations and give some necessary elements that we will frequently use in this manuscript. For all these classical tools and results, one can consult some standard books in Convex analysis and Optimization, such as [START_REF] Heinz | Convex analysis and monotone operator theory in Hilbert spaces[END_REF], [START_REF] Clarke | Functional analysis, calculus of variations and optimal control[END_REF], [START_REF] Cd Aliprantis | Infinite dimensional analysis[END_REF] and [START_REF] Boyd | Convex optimization[END_REF]. convex and lower semi-continuous functions from H to (-∞, +∞]. For a closed set C ⊂ H, for every x ∈ H, we denote the distance from x to C with d(x, C) = min{ x -y : y ∈ C}.

For an interval I ⊂ R + , the classical notations L p (I, H) and W s,p (I, H) stand for the classical p-Lebesgue and p-Sobolev space of order k respectively, with values to H (for the definitions and the different properties of these spaces, we address the reader to [START_REF] Brezis | Functional analysis, Sobolev spaces and partial differential equations[END_REF] and [START_REF] Evans | Measure theory and fine properties of functions[END_REF]). In a similar way BV (I; H) and AC(I; H) denote the spaces of the functions of bounded variation and the absolute continuous functions, respectively (see [START_REF] Evans | Measure theory and fine properties of functions[END_REF] for more details).

Finally, for a sequence {f n } n∈N defined in H, we will use the classical notation of weak convergence to f (which also coincides with the weak-star convergence, since H is a Hilbert space ) with f n f .

B.2 Convex and smooth optimization

Next we give some basic definitions and facts concerning the notions of convexity and smoothness of a function, which allow to tackle optimization problems such as (1).

Definition 1. Let F : H -→ R. F is Fréchet differentiable at x ∈ H if there exists a linear operator ∇F (x) ∈ H, such that :

lim h→0 h =0 F (x + h) -F (x) -∇F (x), h h = 0 (10)
We call ∇F as the gradient of F .

Definition 2. Let F : H -→ R be a (Fréchet) differentiable function. We say that ∇F is L-Lipschitz if there exists L > 0, such that for all (x, y) ∈ H 2 it holds:

∇F (x) -∇F (y) ≤ L x -y (11)
Equivalently, ∇F is L-Lipschitz if and only if for all (x, y) ∈ H 2 , it holds:

F (x) ≤ F (y) + ∇F (y), x -y + L 2 x -y 2 (12)
or even (Baillon-Haddad Theorem) if and only if for all (x, y) ∈ H 2 :

∇F (x) -∇F (y), x -y ≥ 1 L ∇F (x) -∇F (y) 2 (13)
As mentioned earlier, another important feature, which allows to construct methods for estimating a solution of the minimization Problem 1, is the notion of convexity.

F (θx + (1 -θ)y) ≤ θF (x) + (1 -θ)F (y) ( 15
)
If F is also Fréchet differentiable, then the notion of convexity is equivalent to the fact that for all x, y ∈ domF :

F (x) -F (y) ≤ ∇F (x), x -y (16)
If in addition the previous inequalities (15) and (16) hold strictly (≤ is replaced by <), then we say that the function F is strictly convex.

The notion of convexity of a function F , is an essential element for the existence of a minimizer of F . In fact for a convex function F which is proper and lower semi-continuous, the existence of a minimizer of F is guaranteed in every compact set K ⊂ H, such that K ∩ domF = ∅. If in addition the function is strictly convex, then the minimizer is unique.

One of the first and classical condition over the function F , which reinforces the notion of convexity, is the strong convexity property. Definition 5. A function F : H -→ R is strongly convex if its domain is convex and if there exists µ > 0 such that the function F (x) -µ 2 x 2 is convex.

If F is also differentiable, the notion of strong convexity is equivalent to the fact that for all x, y ∈ domF :

F (x) ≤ F (y) + ∇F (x), x -y - µ 2 x -y 2 (17)
The set of all sub-gradients of F at a point x ∈ domF , is denoted by ∂F (x), i.e. :

∂F (x) = v ∈ domF : F (x) ≤ F (y) + v, x -y , ∀y ∈ domF (19)
and the set-valued operator ∂F : H ⇒ 2 H , which maps x ∈ domF to ∂F (x), is called the sub-differential of F .

A simple observation from the definition of the subdifferential of a convex function F , gives the following characterization, which forms the generalized first-order optimality condition for minimizing a function F :

x * ∈ arg min F ⇐⇒ x * ∈ domF s.t. : 0 ∈ ∂F (x * ) (20)
According to the previous characterization (20) the problem of minimization of a convex function F , is equivalent to the research of a zero of its subdifferential (or its gradient if F is smooth).

B.3 The proximal operator

An essential tool for the analysis of convex functions, is the notion of proximal operator of a convex function, introduced and studied by Moreau in [START_REF] Jacques | Proximité et dualité dans un espace hilbertien[END_REF], [START_REF] Martinet | Brève communication. régularisation d'inéquations variationnelles par approximations successives[END_REF] and Rockafellar [START_REF] Tyrrell | Monotone operators and the proximal point algorithm[END_REF]. Definition 7. Let F : H -→ R, be a proper, lower semi-continuous and convex function. The proximal operator Prox F : H -→ H of F is defined by:

Prox F (x) = arg min y∈H F (y) + y -x 2 2 (21)
Notice that since the function G x (y) = F (y) + y-x 2

2

, proper, lower semicontinuous and strongly convex, it admits a unque minimizer (for every x), hence the proximal operator of a convex function F is well-defined. In addition by the first-order optimality condition for G x , for all x ∈ H, we have the following important equivalent characterization:

p = Prox F (x) ⇔ 0 ∈ ∂G x (p) ⇔ x ∈ {p} + ∂F (p) (22)
In fact, relation (22), is an equivalent implicit definition of the proximal operator. Remark that by (22), we can identify the proximal operator of a convex function F , as the resolvent of its subdifferential ∂F (see also Appendix A), i.e. for all x ∈ H :

Prox F (x) := J ∂F (x) = (Id + ∂F ) -1 (x) (23)
Chapter 1

Gradient dynamics for convex minimization

In the present chapter, we denote with H a Hilbert space and we are interested on the minimization problem of a function F :

min x∈H F (x) (1.1)
where F : H -→ R = R ∪ {+∞}, with the following conditions:

C.1 F a convex function in C 1,1 L (H) (continuously differentiable, with L-Lipschitz gradient).

C.2

The set of minimizers X * = arg min{F } is non-empty.

We also denote

F * = min x∈H F (x).
In particular we are taking look on the evolution of trajectories generated by a dynamical system (differential equation with dependence on time) that minimize the function F . More precisely we study the properties of a solution of the following second-order differential equation :

ẍ(t) + b t ẋ(t) + ∇F (x(t)) = 0 (1.2)
where b is a positive parameter. The differential equation (1.2) was introduced recently in [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] (see also [START_REF] Cabot | On the long time behavior of second order differential equations with asymptotically small dissipation[END_REF]) and studied in numerous following works (see [CEG09a, SBC16, ACPR18, AD17] and their related references), and it is linked with the minimization problem of F . The analysis presented in the current Chapter follows the aforementioned works and it constitutes an introductory guideline both for the differential inclusion associated to (1.2) and studied in Chapter 2, as also for the second part of this Thesis, where we study different discrete schemes (algorithms) in order to minimize numerically the function F . As already mentioned, studying the evolution of solutions-trajectories of continuous time systems, offers a lot of advantages in an optimization point of view, since one can exploit the rich ODEs theory in order to deduce interesting convergence results. This offers a deeper insight and a useful guideline for the analysis of their correspondent discrete schemes.

Roughly speaking, algorithms can be interpreted as finite difference approximation schemes of continuous dynamical systems. Hence one can expect that -under some suitable discretization-the convergence properties of the (continuous) dynamical system, are close enough to the ones of the corresponding numerical scheme.

Dynamical systems and Lyapunov analysis

In its most abstract form, a (continuous) dynamical system is a system describing the evolution of a unit x(t) with respect to time t ∈ R + , which is guided by a vector field G : R + × H → H and a differential equation :

ẋ(t) = G(t, x(t)) (1.3)
starting at an instant t 0 ≥ 0 from a point x(t 0 ) = x 0 ∈ H. From (1.3), by integrating, we have equivalently:

x(t) = x 0 + t 0 G(s, x(s))ds (1.4)
which indicates the moving rule of the unit x(t) at every instant t ∈ R + .

In particular, in the context of problem (1.1), we are interested by the study of dynamical systems, which promote trajectories (flows) which minimize F . In this setting, the vector field G(t, x(t)) is usually expressed via the potential ∇F (x(t)). We recall here that the first-order optimality condition for the smooth and convex function F , leads us to the research of zeros of the gradient of the function F , i.e. x * ∈ H, such that :

∇F (x * ) = 0 (1.5)

The Gradient Flow

Therefore this leads to the research of stationary (or equilibrium) points of (1.3), which also coincide with the critical points of F . A classical powerful tool for this task is the Lyapunov method ([Lya92]). Standard Lyapunov analysis for finding critical (or stationary) points of a general evolution system in a time interval

I ⊂ R + ẋ(t) = G(t, x(t)) (1.6)
consists of constructing an energy-functional Φ : H -→ R, which dissipates along the solutions-trajectories of the system (1.3) (see for example [START_REF] Mikhailovich | The general problem of the stability of motion[END_REF]). This translates to the fact that the function Φ • x : I -→ R is non-increasing for a possible solution x(t) of (1.3). Classically, depending on the regularity of a solution of (1.3), this is done by showing that

Φ(x(t)) = ∇Φ(x(t)), G(t, x(t)) ≤ 0. (1.7)
Of course multiple choices for a Lyapunov function Φ are possible, depending on the structure problem (1.3) (i.e. the function G) and the expected results.

In our framework we will use Lyapunov-type arguments in order to prove the minimizing convergence properties of trajectories generated by evolutionsystems like (1.3), as also the associated rates. In particular this is done by constructing suitable Lyapunov energies that interfere quantities, such as the objective function F (x(t)) -F * , the velocity ẋ(t) or the distance to a minimizer x * , x(t) -x * , up to a suitable function-order of time. Such a function can have the following form :

Φ(t) = a(t) F (x(t)) -F * + b(t) ẋ(t) 2 + c(t) x(t) -x *

The Gradient Flow

One of the first and simplest dynamical system studied for minimizing function F , or finding critical points is the Gradient Flow (or steepest descent flow) system.

  ẋ(t) + ∇F (x(t)) = 0 x(0) = x 0 (GF)
System (GF) generates motions (flows) which follows the locally-steepest descent direction over the graph of F . This ensures that the solution x(t), will always decrease (locally) the function F , until it minimizes it.

Formally, from Hypotheses C.1 and C.2, the Cauchy-Lipschitz Theorem guarantees the existence and uniqueness of a global solution x ∈ C 1 ([0, ∞); H). In fact by convexity of F , the existence and uniqueness of a generalized notion of strong solution for (GF) can also be assured, without the hypothesis on the differentiability of F (see Chapter 2 or [START_REF] Brezis | Monotonicity Methods in Hilbert Spaces and Some Applications to Nonlinear Partial Differential Equations[END_REF][START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF][START_REF] Bruck | Asymptotic convergence of nonlinear contraction semigroups in hilbert space[END_REF]).

In addition a simple derivation can show that if x(t) is the solution of (GF), the objective function

w(t) = F (x(t)) -F * is a Lyapunov function, since: ẇ(t) = ∇F (x(t)), ẋ(t) = -ẋ(t) 2 (1.9)
which shows the minimizing property of x(t). Here we can even notice here that the convexity of F is not needed. This shows that the gradient flow is still locally minimizing functions that are not necessarily convex.

A more refined choice of Lyapunov energy for system (GF), is the following:

U (t) = t F (x(t)) -F * + x(t) -x * 2 2 (1.10)
where x * ∈ arg min F . Then by differentiating and using (GF) it is direct that :

U (t) = F (x(t)) -F * -∇F (x(t)), x(t) -x * -t ẋ(t) 2 (1.11)
and by using the convexity of F , we find :

U (t) ≤ -t ẋ(t) 2 (1.12)
In particular the properties of the Lyapunov energy U and Opial's Lemma (see Lemma A.0.1 in Appendix A) allow to deduce the following Theorem (see for example [START_REF] Brezis | Monotonicity Methods in Hilbert Spaces and Some Applications to Nonlinear Partial Differential Equations[END_REF][START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF], [START_REF] Bruck | On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in hilbert space[END_REF], [START_REF] Jb Baillon | Un exemple concernant le comportement asymptotique de la solution du problème du dt + ∂ϕ(u(t)) 0[END_REF], [START_REF] Peypouquet | Evolution equations for maximal monotone operators: Asymptotic analysis in continuous and discrete time[END_REF] and [START_REF] Güler | Convergence rate estimates for the gradient differential inclusion[END_REF]).

Theorem 1.2.1. Let x be a solution of (GF) then the following hold true • F (x(t)) converges non-increasingly to F * . In addition :

F (x(t)) -F * = O 1 t (1.13)
1.3. Second order damped dynamics

• {x(t)} t≥0 weakly converges to x * ∈ arg min F asymptotically. If in addition this convergence is strong, then :

F (x(t)) -F * = o 1 t asymptotically (1.14)
While the Gradient flow enjoys some good local properties for minimizing F (such as the decreasing property of F (x(t))), it suffers from a slow global convergence rate for the objective function, without any further assumptions on F . As we shall see in the next section, by using more precise dynamical systems of higher order information, such as the acceleration of the unit x(t), one can obtain better results. More precisely this can be done by considering some particular second-order dynamical systems.

Second order damped dynamics

One of the first and simplest second-order system considered for minimizing F is the Heavy-Ball friction system, which was studied in [START_REF] Alvarez | On the minimizing property of a second order dissipative system in hilbert spaces[END_REF] and [START_REF] Attouch | The heavy ball with friction method, i. the continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system[END_REF] (see also [START_REF] Boris | Some methods of speeding up the convergence of iteration methods[END_REF], [START_REF] Sergeevich | Minimization of convex functions on convex sets by means of differential equations[END_REF], [START_REF] Haraux | Convergence of solutions of second-order gradient-like systems with analytic nonlinearities[END_REF] and [START_REF] Bégout | On damped second-order gradient systems[END_REF] in the case of strongly convex function F ).

The Heavy-Ball with friction system reads the following :

ẍ(t) + α ẋ(t) + ∇F (x(t)) = 0 (HBS)
where α is a positive parameter. In a mechanical interpretation system describes the motion of the unit x(t) moving along the graph of F with a constant friction term -α ẋ(t). Historically it was studied by Alvarez in [START_REF] Alvarez | On the minimizing property of a second order dissipative system in hilbert spaces[END_REF] (see also [START_REF] Alvarez | A second-order gradient-like dissipative dynamical system with hessian-driven damping.: Application to optimization and mechanics[END_REF]) in a convex setting and in Attouch et al. in [START_REF] Attouch | The heavy ball with friction method, i. the continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system[END_REF], in the non-convex setting providing a first way for exploring local minima of F . In particular by tuning properly the initial velocity of the system ẋ(0), a solution of (HBS) can "escape" a neighborhood of a local minimum, which is not possible for the trajectory generated by the gradient flow (GF) (see [START_REF] Attouch | The heavy ball with friction method, i. the continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system[END_REF] for more details). Notice in general that second-order in time systems, such as (HBS), allow to model suitably the acceleration of the unit x(t), on the contrary to first-order systems such as the gradient flow, where the acceleration is constrained by the velocity (and hence the initial position x 0 ). Nevertheless, for general convex functions, the -worst case-convergence rate of the (HBS) is not better than the one obtained by the Gradient Flow (GF), as the following Theorem asserts.

Theorem 1.3.1 ([Alv00]). Let x be a solution of (HBS), with some initial conditions x(0) = x 0 ∈ H and ẋ(0) = v 0 ∈ H. Then {x(t)} t≥0 weakly converges to x * ∈ arg min F asymptotically and

F (x(t)) -F * = O 1 t (1.15)
In a more general way in [START_REF] Cabot | On the long time behavior of second order differential equations with asymptotically small dissipation[END_REF] (see also [CEG + 09b]), the authors studied systems like (HBS) with t 0 ≥ 0 and a general damping parameter α(t), where α

: [t 0 , +∞) -→ R + ẍ(t) + α(t) ẋ(t) + ∇F (x(t)) = 0 (1.16)
As it was shown in [START_REF] Cabot | On the long time behavior of second order differential equations with asymptotically small dissipation[END_REF], systems like (1.16), are linked with the problem of minimizing F (or finding critical points of F in the non-convex case). It turns out that the dissipation term α(t) in (1.16), plays a crucial role for the minimization of F . In particular it rules the asymptotic behavior and the convergence properties of the trajectory associated to (1.16). As it was shown in [START_REF] Cabot | On the long time behavior of second order differential equations with asymptotically small dissipation[END_REF] (see also [CEG + 09b]) the global (potential and kinetic) energy of the system (1.16), U (t) = F (x(t)) -F * + 1 2 ẋ(t) 2 is a Lyapunov function. If in addition the damping parameter α(t) vanishes to zero relatively slow ( ∞ 0 α(t)dt = +∞), then U (t) is dissipating until it reaches zero. This corresponds to the minimizing property of the solution x(t), i.e. F (x(t)) -→ F * and ẋ(t) -→ 0. These studies were further extended recently in [START_REF] Attouch | Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity[END_REF] and [START_REF] Attouch | Rate of convergence of inertial gradient dynamics with timedependent viscous damping coefficient[END_REF].

In our case, we are interested on the -limiting order-particular choice of the friction term α(t), which vanishes asymptotically exactly as α(t) 1 t .

Second order damped dynamics with vanishing viscosity

As it was firstly pointed out in [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] (see also [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF]), an interesting choice for the viscosity parameter α(t) in (1.16) is the vanishing damping parameter α(t) = b t , where b > 0 is measuring the friction. In this case, second-order damped system (1.16) takes the following form:

ẍ(t) + b t ẋ(t) + ∇F (x(t)) = 0 (1.17)
Since the seminal work of Su et al. in [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF], the gradient system (1.17), gained special attention and it was studied intensively for the last years. The first reason is that, under some suitable assumptions, system (1.17) turns 1.3. Second order damped dynamics out to accelerate the minimizing property of the trajectory (i.e. how fast F (x(t)) -F * -→ t→∞ 0), in comparison to the one which is ruled by the gradient flow (GF) or the Heavy-Ball system HBS. More precisely, as we shall see in a bit, the parameter b > 0 governs the convergence properties of system (1.17). The second reason (which is related with the first one), is that system (1.17) is closely linked with Nesterov's type accelerated scheme and its variants (see algorithm (7) in Chapter 3 in the smooth setting), which are of a great interest in the optimization community and more precisely in the current Thesis. Hence under reasonable assumptions and suitable discretization, it is expected that the discrete schemes derived by systems like (1.17) maintain the same convergence properties as the ones of (1.17). This observation is of a high interest since the analysis of the continuous-time system (1.17) will give additional insight and it will pave the way for the one of the discrete scheme (7), which turns out to be more technical. For a discussion about the discretization of (1.17), we address the reader to Section 1.4 at the end of the current Chapter, or to Section 2.4 of Chapter 2, in the more general setting of the differential inclusion associated to (1.17).

Our presentation and proofs follow the same lines of the analysis of a series of recent works [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF], [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF], [START_REF] Aujol | Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b > 0[END_REF], and [START_REF] May | Asymptotic for a second-order evolution equation with convex potential and vanishing damping term[END_REF] for the system (1.17).

More precisely the Cauchy-problem associated to (1.17), with some initial conditions x 0 ∈ H is:

     ẍ(t) + b t ẋ(t) + ∇F (x(t)) = 0 x(t 0 ) = x 0 , ẋ(t 0 ) = 0 (1.18)
Since we are mostly interested in the asymptotic properties, we consider the system (1.18), with some initial condition on t 0 > 0, in order to guarantee the existence and uniqueness of a strong solution (in the classical sense) x ∈ C 2 ([t 0 , +∞); H) for the system (1.18). In fact, under the L-Lipschitz condition on ∇F , the existence of a slight weaker notion of solution is still valid in the case of H = R d and t 0 = 0 (see Theorem 1 in [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF]). In this case one can prove the existence and uniqueness of a solu-

tion x ∈ C 2 ((0, ∞); R d ) ∩ C 1 ([0, ∞); R d ) of (1.18).
In general a sufficient condition, in order to guarantee the existence and the uniqueness of a strong solution of (1.18), is the Lipschitz character of ∇F on bounded sets.

In fact, by setting

H : [t 0 , ∞) × H 2 -→ H 2 , with H(t, (v 1 , v 2 ) T ) = v 2 , -b t v 2 + ∇F (v 1 )
T , and X(t) = (x(t), ẋ(t)) T , we can reformulate system (1.18), in a first-order system, as follows : At this point we should also stress out that apart the uniqueness of the solution trajectory of (1.18), the Lipschitz character of the gradient of F also assures the existence of a solution into a general -infinite dimensional-Hilbert space H. Remark that without this regularity assumption, the general Peano's Theorem (see for example [START_REF] Peano | Démonstration de l'intégrabilité des équations différentielles ordinaires[END_REF]) for existence of a solution fails to apply in infinite dimensional spaces ( for a counter-example consult the works [START_REF] Dieudonné | Deux exemples singuliers d'équations différentielles[END_REF] or [START_REF] Godunov | Peano's theorem in an infinite-dimensional hilbert space is false even in a weakened formulation[END_REF] and [START_REF] Godunov | Peano's theorem in banach spaces[END_REF]). In order to guarantee the existence of a solution for a general function F , this assumption can be relaxed simply to a continuously differentiable function F , only in the case where H = R d , d ≥ 1. Despite the fact that Lipschitz continuity of the gradient of F , is a necessary condition to obtain both existence and uniqueness of a solution, this condition is not necessary and it will not be exploited in order to deduce the different convergence properties of such a solution. Hence in what follows we consider the existence of solution as granted without mentioning the Lipschitz continuity of ∇F .

Ẋ(t) = H(t, X(t)) X(t 0 ) = (x 0 , 0) T (1.
Here we give the basic results concerning the convergence properties for a solution of (1.18) for minimizing function F , depending on the friction parameter b > 0. As already mentioned, these results have been established in the works of [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF], [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF]), [START_REF] Aujol | Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b > 0[END_REF], [START_REF] Attouch | Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3[END_REF] and [START_REF] May | Asymptotic for a second order evolution equation with convex potential and vanishing damping term[END_REF] and they are summarized in the following Theorem.

Theorem 1.3.2. Let x ∈ C 2 ([t 0 , +∞); H) be a solution of (1.18) and x * ∈ arg min F . The following estimates hold true : 1. ( [START_REF] Aujol | Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b > 0[END_REF], [START_REF] Attouch | Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3[END_REF]) If b ∈ (0, 3) then there exist some positive constants C 1 , C 2 such that for all t ≥ t 0 , it holds :

F (x(t)) -F * ≤ C 1 t 2b 3 and ẋ(t) ≤ C 2 t b 3
(1.20)

([SBC16], [ACPR18]

) If b ≥ 3 then there exist some positive constants C 1 , C 2 such that for all t ≥ t 0 , it holds : and the trajectory weakly converges to a minimizer x * .

F (x(t)) -F * ≤ C 1 t
( [START_REF] May | Asymptotic for a second order evolution equation with convex potential and vanishing damping term[END_REF]) In fact asymptotically, it holds :

F (x(t)) -F * = o t -2 and ẋ(t) = o t -1 (1.23)
Remark 1. As it was pointed out in [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF] (see also [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF]), the rates in the estimations (1.21) are -worst case-optimal for the class of smooth convex functions. To see this one can consider one-dimensional functions of type F (x) = |x| p , for any p > 2. Then the system (1.18), admits a solution of the form

x(t) = ct -2 p-2 , for a suitable constant c. This implies that F (x(t)) -F * = C 1 t -2p p-2 and ẋ(t) = C 2 t -p p-2
for all p > 2, and some suitable constants C 1 , C 2 , which shows that the rates in (1.21) are optimal, since for p large enough, the orders -2p p-2 andp p-2 are sufficiently close to -2 and -1 respectively.

Remark 2. The previous example of functions F (x) behaving as |x| p , suggests that by exploiting more the geometry of the minimizing function, one can obtain more precise rates, than the ones in (1.21). As we shall see later in this Chapter, this is possible under some additional suitable assumptions on the geometry of F .

Convergence analysis

In this paragraph we give the main elements for the convergence analysis of a solution of system (1.18), and in particular the proof of Theorem 1.3.2. As mentioned earlier, the basic approach in order to prove the convergence results as announced in Theorem 1.3.2, is based on finding suitable Lyapunov energies. For that, for x * ∈ arg min F and x(t) a solution of (1.18), we will use the following notations :

w(t) = F (x(t)) -F * and h(t) = x(t) -x * 2
(1.24)

A first natural candidate for the Lyapunov analysis for system (1.18) is the global ( potential and kinetic ) energy of system (1.18), i.e.:

U (t) = F (x(t)) -F * + 1 2 ẋ(t) 2 = w(t) + 1 2 ẋ(t) 2 (1.25)
where x(t) is a solution of (1.18). By differentiating and using the fact that x(t) is a solution of (1.18), we find:

U (t) = t ∇F (x(t)), ẋ(t) + ẍ(t), ẋ(t) = - b t ẋ(t) 2 (1.26)
which shows that U is a Lyapunov function for the system (1.18). This energy-functional can be used in order to deduce the minimizing property of x(t), i.e. F (x(t)) -→ t→∞ F * (see for example Theorem 2.3 in [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF]), unconditionally of the choice of the friction parameter b > 0. Nevertheless it is not clear how to obtain the estimations and the orders of convergence rates as they are presented in Theorem 1.3.2, from this energy. For that we have to consider a "finer" Lyapunov function, which balances better the quantities of interest, such as the objective function F (x(t)) -F * and ẋ(t) , with a "proper" power of t.

Let x ∈ C 2 ([t 0 , ∞); H) a solution of (1.18). For all λ ≥ 0 and ξ ≥ 0 define:

v(t) = λ(x(t) -x * ) + t ẋ(t) 2 (1.27)
and

E(t) = t 2 F (x(t)) -F (x * ) + 1 2 λ(x(t) -x * ) + t ẋ(t) 2 + ξ 2 x(t) -x * 2 = t 2 w(t) + 1 2 v(t) + ξ 2 h(t)
(1.28) Notice that by developing the square in v(t) and using the fact that:

ḣ(t) = 2 ẋ(t), x(t) -x * ,
(1.29)

we have the following equivalent expression for the function E

E(t) = t 2 w(t) + t 2 2 ẋ(t) 2 + (λ 2 + ξ) 2 h(t) + λt 2 ḣ(t) (1.30)
This function can be seen as the negative entropy up to the balanced distance ξ 2 x(t) -x * 2 . This functional was considered in [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] and in [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF], as also in [START_REF] Aujol | Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b > 0[END_REF] and [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF], in order to deduce some fast convergence asymptotic behavior for w(t) and ẋ(t) as well as the weak convergence of the trajectory to a minimizer x * . In fact from the definition of E(t), for suitable values of parameters λ and ξ, one can observe that the order of the upper-bounds -with respect to time t-for this energy, will also furnish the same order for the estimates of t 2 w(t), v(t), t 2 ẋ(t) 2 and h(t).

In particular, by differentiating E and using the fact that the solution x verifies (1.18), one can find the following result : 1.3. Second order damped dynamics Lemma 1.3.1. Let x be a solution of (1.18), x * ∈ arg min F . Then for all λ ≥ 0, ξ ∈ R, we have:

Ė(t) = 2tw(t) -λt ∇F (x(t)), x(t) -x * + (λ + 1 -b)t ẋ(t) 2 + ξ + λ(λ + 1 -b) ẋ(t), x(t) -x * (1.31)
or equivalently :

Ė(t) = 2(b -λ)tw(t) -λt ∇F (x(t)), x(t) -x * + 2(λ + 1 -b) E(t) t + (λ 2 + ξ)(λ + 1 -b) h(t) t + ξ -λ(λ + 1 -b) ẋ(t), x(t) -x *
(1.32) In fact, in view of convexity of F , it holds :

Ė(t) ≤ (2 -λ)tw(t) + (λ + 1 -b)t ẋ(t) 2 + ξ + λ(λ + 1 -b) ẋ(t), x(t) -x * (1.33)
or equivalently :

Ė(t) = (2b -3λ)tw(t) + 2(λ + 1 -b) E(t) t -(λ 2 + ξ)(λ + 1 -b) h(t) t + ξ -λ(λ + 1 -b) ẋ(t), x(t) -x * (1.34)
Corollary 1.3.1. Let x be a solution of (1.18) and x * ∈ arg min F . Then we have the following cases:

1. For b ∈ (0, 3), λ = 2b 3 and ξ = λ(λ + 1 -b) > 0, the function H(t) = t 2b 3 -2 E(t) is non-increasing in (t 0 , +∞). 2. For b ≥ 3, λ ∈ [2, b -1] and ξ = λ(b -λ -1) ≥ 0, the function E(t) is non-increasing in (t 0 , +∞).
In particular, for all t ≥ t 0 , we have :

Ė(t) ≤ (2 -λ)tw(t) + (λ + 1 -b)t ẋ(t) 2 (1.35)
Proof. The proof of point 1. is direct from equation (1.34) of Lemma 1.3.1, by replacing λ and ξ and observing that when b ∈ (0, 3), then Ė(t)

≤ 2(3-b) 3t E(t) (which is equivalent to the fact that the function H(t) = t 2b 3 -2 E(t) is non- increasing) The proof of 2. is direct from (1.33) of Lemma 1.3.1 by replacing ξ = λ(λ + 1 -b), with λ ∈ [2, b -1].
Lemma 1.3.1 and Corollary 1.3.1 permit to prove the Theorem 1.3.2, by tuning properly the friction parameter b > 0 together with the Lyapunovparameters λ and ξ in the definition of the energy-functional E(t).

Proof of Theorem 1.3.2. For the first case b < 3, for λ = 2b 3 and ξ = λ(λ + 1 -b) > 0 from point 1. of Corollary 1.3.1, and the definition of E(t) and H(t) = t 2b 3 -2 , for all t ≥ t 0 , we have :

t 2b 3 w(t) ≤ t 2b 3 -2 E(t) = H(t) ≤ H(t 0 ) = t 2b 3 -2 0 E(t 0 ) (1.36)
which gives the first bound of (1.20).

In the same way, since ξ = λ(λ + 1 -b) > 0, we have :

t 2b 3 -2 h(t) ≤ t 2b 3 -2 E(t) = H(t) ≤ H(t 0 ) = t 2b 3 -2 0 E(t 0 ) (1.37)
Hence, we deduce the existence of a constant C > 0, such that :

h(t) ≤ Ct 2-2b 3 (1.38)
By using the basic convex inequality and the fact that H(t) is non-increasing, we find :

t 2b 3 -2 t 2 4 ẋ(t) 2 + (ξ -λ 2 ) 2 h(t) ≤ t 2b 3 -2 E(t) = H(t) ≤ H(t 0 ) (1.39)
Hence by multiplying by t 2-2b 3 and using estimation (1.38), we conclude proof of the first point (1.20) of Theorem 1.3.2.

For the second point if b ≥ 3, from (1.35) of Corollary 1.3.1, with λ = 2, for all t ≥ t 0 , we have: Ė(t) ≤ 0, hence the function E is non-increasing and thus bounded. From definition (1.28), since ξ = λ(b -λ -1) ≥ 0, we have directly the first estimation of (1.21) with C 1 = E(t 0 ).

In addition by definition (1.30), since λ 2 + ξ ≥ λ (recall that λ = 2 and ξ = λ(λ + 1 -b) ≥ 0) we have that :

h(t) + t ḣ(t) ≤ E(t 0 ) (1.40)
By integrating the previous relation, we deduce the existence of a constant C > 0, such that for all t ≥ t 0 , it holds :

h(t) ≤ C (1.41)
By using the basic convex inequality u 2 ≤ 2 u+z 2 +2 z 2 with u = t ẋ(t) and z = λ(x(t) -x * ) and the fact that E(t) is non-increasing, from (1.28), we find :

t 2 4 ẋ(t) 2 + (ξ -λ 2 ) 2 h(t) ≤ E(t 0 ) (1.42)
1.3. Second order damped dynamics which by using estimation (1.41), gives :

t 2 ẋ(t) 2 ≤ 4E(t 0 ) + 2|λ 2 -ξ|h(t) ≤ 4E(t 0 ) + 2|λ 2 -ξ|C ≤ C (1.43)
which allows to conclude the second estimation of (1.21) for a suitable positive constant C 2 .

For the third point, since b > 3, by evaluating λ = b -1 and λ = 2 in (1.35) we obtain respectively, for all t ≥ t 0 :

t t 0 s(F (x(s)) -F * )ds ≤ E(t 0 ) b -3 and t t 0 s ẋ(s) ds ≤ E(t 0 ) b -3 (1.44)
In order to prove (1.23), we consider the global energy function of system (1.18) :

U (t) = w(t) + 1 2 ẋ(t) 2 (1.45)
By differentiating and using the fact that x(t) satisfies (1.18), we find :

U (t) = ∇F (x(t)) + ẍ(t), ẋ(t) = - b t ẋ(t) 2 (1.46)
Hence the function U is non-increasing, and bounded from below therefore convergent. In addition, from (1.22), for all ε > 0, there exist some T > t 0 , such that for all t ≥ T :

t t 2 sU (s)ds = t t 2 s(w(s) + 1 2 ẋ(s) 2 ds < ε (1.47)
By using the fact that U (t) is non-increasing, for all t ≥ T , we find

U (t) t t 2 sds ≤ t t 2 sU (s)ds < ε (1.48) hence, t 2 U (t) = o(1)
, which by positivity of w(t) and ẋ(t) 2 , gives the expected result.

For the weak convergence, the proof necessitates the use of Opial's Lemma (see Lemma A.0.1). Since the proof of the weak convergence is given for the more general setting of the differential inclusion studied in Chapter 2, we are addressing the reader to the proof of Corollary 2.2.1 presented later on.

Perturbed system with vanishing viscosity term

Further studies (see [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF], [START_REF] Aujol | Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b > 0[END_REF]) include the analysis of system (1.18), with the presence of an additional integrable term p ∈ L 1 ([t 0 , +∞); H) :

ẍ(t) + b t ẋ(t) + ∇F (x(t)) + p(t) = 0 (1.49)
or more generally (see [START_REF] May | Asymptotic for a second-order evolution equation with convex potential and vanishing damping term[END_REF], [START_REF] Ali | Asymptotics for a second-order differential equation with nonautonomous damping Bibliography and an integrable source term[END_REF], [START_REF] Balti | Asymptotic for the perturbed heavy ball system with vanishing damping term[END_REF]) :

ẍ(t) + α(t) ẋ(t) + ∇F (x(t)) + p(t) = 0 (1.50)
The term p(t) can be seen as a perturbation error over the gradient ∇F . This corresponds to the case when ∇F is p-approximately computed at every instant t ≥ t 0 . As we shall see in Part II and in particular in section 3.4, apart of questions of theoretical interest, the study of systems like (1.49) is of a high importance in connection with the analysis of numerical algorithms with inexact computations of the gradient of the minimizing function (see section 3.4, later on).

In order to guarantee the existence and uniqueness of a solution of the Cauchy problem associated to (1.49), apart from the Lipschitz character of ∇F , it suffices to impose that the perturbation function p is integrable, i.e. p ∈ L 1 ((t 0 , ∞); H).

In particular, all the convergence rates found in Theorem 3.3.1 are relatively robust, with respect to the perturbation term p(t), which means that they remain true, under some control conditions on the function p. More precisely we have the following Theorem, whose proof follows the same Lyapunov arguments of the unperturbed case, is let to the reader.

Theorem 1.3.3 ([AD17],[ACR19]). Let x ∈ C 2 (t 0 , +∞) be a solution of (1.49) and x * ∈ arg min F . If p = min{1, b
3 } and ∞ t 0 t p p(t) dt < +∞ , then the following estimates hold true :

F (x(t)) -F * = O t -2p and ẋ(t) = O t -p (1.51)
The proof of Theorem 1.3.3, is similar to the one of 3.3.1, for the unperturbed one and is based in a slightly modified Lyapunov energy of the one in (1.28) :

E p (t) = t 2-2p E(t) + t t-0 t 2p-1 λ(x(s) -x * ) + s ẋ(s), p(s) ds (1.52)
with p = min{1, b 3 } and the use of Grönwall-Bellman Lemma (see for example Lemma 6.2 in [ACR19]).

Second order damped dynamics

The differential equation (1.18) under additional geometrical assumptions

In various situations, apart convexity, the minimizing function F enjoys additional geometrical features, such as strong convexity, or more generally the p-growth condition for a general p ≥ 1 :

Definition 1.1. Let F : H → R be a convex differentiable function with X * = arg min F = ∅. Let p ≥ 1.
The function F satisfies the p-growth condition L(p) if, for any minimizer x * ∈ X * , there exists a constant K p > 0 and ε > 0 such that:

∀x ∈ B(x * , ε), K p x -x * p ≤ F (x) -F (x * ). (1.53)
Roughly speaking, the growth condition L(p), indicates the level of sharpness of the function F near the set of its minimizers (i.e. how fast the function F increases around X * ). For convex functions the growth condition is related with the Łojasiewicz (or more generally Kurdyka-Łojasiewicz) property (see Definition 4.2 in Chapter 4), studied and used intensively in various domains the last decades (see for example the works of [Łoj63, Łoj93] as also [BDL07, BDL06, BDLM10, ABRS10a] and their possible references).

On the other hand, another important geometric property for the minimizing function, which was introduced in [START_REF] Cabot | On the long time behavior of second order differential equations with asymptotically small dissipation[END_REF], and revisited in the recent works of [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] and [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF], is a flatness-type condition.

Definition 1.2. Let β ≥ 1. The function F satisfies the condition H(β) if, for any critical point x * ∈ X * , there exists η > 0 such that:

∀x ∈ B(x * , η), F (x) -F (x * ) ≤ 1 β ∇F (x), x -x * .
Opposingly to the p-growth condition L(p), condition H(β) expresses the level of flatness of the function F , near the set of its minimizers. In fact H(β) can be seen as a generalization of the notion of convexity of F near its minimizers. It can be shown (see Lemma 4.4.1 in Chapter 4) that a convex function that verifies H(β), for β ≥ 1, also verifies

F (x) -F (x * ) ≤ M d(x, x * ) β .
(1.54) in a suitable neighborhood of X * = arg min F and a suitable constant M > 0. This shows that condition H(β) forms an upper threshold for the function F , measured by the parameter β, in contrary with condition L(p), which forms a lower threshold measured by p.

For a more detailed presentation and more comments about Hypotheses H(β) and L(p) and their interplay, we are redirecting the reader in Sections 4.1 and 4.4.1 of Chapter 4.

In this paragraph we briefly present some of the recent results concerning the convergence properties of a solution to (1.18), under the additional geometrical assumptions H(β) and L(p) on the minimizing function F . Under these conditions one can expect finer convergence rates, than the ones presented in Theorem 1.3.2, under the sole assumption of convexity of F . For a detailed presentation and full proofs of these results, we are addressing the interested reader to the recent work of [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF]. These results will provide some useful landmarks for the analysis made in the discrete case in Chapter 4, for the associated inertial-gradient algorithm (see algorithm (11) in Chapter 4). In particular we have the following two Theorems, expressing the convergence properties of a solution of (1.18), with respect to the friction parameter b > 0 and the geometry of the function F (i.e. the parameters β ≥ 1 and p ≥ 1).

Theorem 1.3.4 ([ADR18]

). Let F be a convex differentiable function satisfying H(β), with β ≥ 1 and x * ∈ arg min F . If x is a solution of (1.18), then the following hold true:

1. If b ∈ (0, 1 + 2 β ) then : F (x(t)) -F * ≤ C 1 t 2βb β+2 and ẋ(t) ≤ C 2 t βb β+2
(1.55)

2. If b ≥ 1 + 2 β then : F (x(t)) -F * ≤ C 1 t 2 and ẋ(t) ≤ C 2 t (1.56) 3. If b > 1 + 2 β then : +∞ t 0 t F (x(t)) -F * dt < +∞ and +∞ t 0 t ẋ(t) 2 dt < +∞ (1.57)
In this case the trajectory x(t), weakly converges to a minimizer as also:

F (x(t)) -F * = o t -2 and ẋ(t) = o t -1 (1.58) 4. If b > 1 + 2 β and F satisfies L(2)
and has a unique minimizer x * , then

F (x(t)) -F * ≤ C 1 t 2βb β+2 and ẋ(t) ≤ C 2 t βb β+2
(1.59)

Second order damped dynamics

In addition the trajectory x(t) strongly converges to the minimizer x * ∈ arg min F .

Remark 3. As it shown in [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF] and [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF], by considering functions of type F (x) = |x| r , for r > 1, one can show that all the above convergence rates of Points 1., 2. and 4. are attained. This shows that these orders are -worst case-optimal (see Proposition 4.2 in [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF]).

As for functions F with flat-enough geometry near the set of their minimizers, we have the following Theorem.

Theorem 1.3.5 ([ADR18]

). Let F be a convex function satisfying H(β), with β ≥ 2 and L(p), with p ≥ 2. If x is a solution of (1.18), then there exist some positive constants C 1 , C 2 , such that for all t ≥ t 0 it holds:

F (x(t)) -F * ≤ C 1 t 2p p-2 and ẋ(t) ≤ C 2 t p p-2 (1.60)
In addition the trajectory x(t) strongly converges to a minimizer x * ∈ arg min F . Remark 4. As for Theorem 1.3.4, by considering functions of type F (x) = |x| r , for r > 2, one can show that the rates in (1.60) are attained, hence are -worst case-optimal for the class of the continuously differentiable convex functions (see Example 2.13 in [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF]).

Notice that Points 1., 2. and 3. of Theorem 1.3.4 consist of a generalization of the results found in Theorem 1.3.2. Remark that when β = 1 (F is convex), one can exactly recover Theorem 1.3.2.

In fact one can easily verify from Lemma 1.3.1 that by using hypothesis H(β) with β ≥ 1, instead of the convexity of F , by choosing properly λ and ξ, we can recover a similar version of Corollary 1.3.1 (here the number 3 appearing to Corollary 1.3.1 is to be replaced by 1 + 2 β ). More generally, for a real parameter r, by extending the definition of the functional E, one can consider the r-scaled energy E r as follows:

E r (t) = t r E(t) = t 2+r w(t) + t r 2 v(t) + ξt p 2 h(t) (1.61)
By differentiating and using equation (1.18), for λ ≥ 0 and ξ = λ(λ+1-b) we find (see Lemma 5.1 in [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF]) :

Ėr (t) ≤ t r (2 -βλ + r)tw(t) + (2λ + 2 -2b + r) v(t) 2t + λ(λ + 1 -b)(r -2λ) h(t) 2t (1.62)
Then the associated results of Theorems 1.3.4 and 1.3.5 can be obtained, by choosing properly the Lyapunov-parameters λ and r, in order to recover the Lyapunov property for E r and using Hypotheses H(β) and L(p), in each case when needed. For the details of this procedure, we address the reader to the recent works [START_REF] Aujol | Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b > 0[END_REF] and [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF].

Discretization of the ODE (1.18)

In this Section we discuss about some possible discretizations of the dynamical system (1.18). As mentioned earlier by discretizing properly the differential equation (1.18), one can expect that the convergence properties of its solution-trajectory, are inherited in the corresponding discrete scheme.

Here we give a possible discretization of (1.18), which corresponds to a particular case of an inertial algorithm, which is of our interest in Part II.

Firstly note that by introducing the auxiliary function v(t) = ẋ(t), the second-order ODE (1.18) is equivalent to the following coupled system :

   v(t) = ẋ(t) v(t) = -b t v(t) -∇F (x(t))
(1.63) with x(0) = x 0 ∈ H and v 0 = 0 We will discretize system (1.63) implicitly with respect to the first equation and explicitly with respect to the second. If we fix a time-step h, t 0 = bh > 0 and set the grid

t n = t 0 + nh = (n + b)h and x n ≈ x(t n ) and v n ≈ v(t n ), from (1.63) we obtain :    x n+1 = x n + hv n+1 v n+1 = 1 -b n+b v n -h∇F (y n ) (1.64)
where y n depends on x n and v n and is to be defined (the reason of the choice for y n will be clear in a while). System (1.64) is then equivalent to x 0 = x 1 ∈ H and ∀n ≥ 1 :

x n+1 = x n + α n (x n -x n-1 ) -h 2 ∇F (y n ) (1.65)
with and

α n = 1 -b n+b = n n+b . Hence if we choose y n = x n + α n (x n -x n-1
), we obtain y 1 = x 1 and for all n ≥ 1 :

y n = x n + α n (x n -x n-1 ) x n+1 = y n -h 2 ∇F (y n ) (1.66) 1.4. Discretization of the ODE (1.18)
System (1.66) corresponds exactly to the generalized Nesterov's type acceleration scheme for smooth convex functions, with a step γ = h 2 . This is the algorithm of our interest in Part II in the more general setting of composite convex optimization problems. This correspondence is of a high importance, since the results and the proofs for a solution of the continuous system (1.18), will trace the strategy of proofs for the associated discrete scheme (1.66).

Of course scheme (1.66) is one among the numerous possible discretizations that one can do for the system (1.63). Notice also that the choice of discretization can play an important role.

Remark for example that by full implicit discretization of (1.63), one can recover a particular version of the accelerated proximal point algorithm as considered in [START_REF] Güler | New proximal point algorithms for convex minimization[END_REF] :

y n = x n + n n + b (x n -x n-1 ) x n+1 = Id + h 2 ∂g -1 y n -h 2 ∇f (y n ) = Prox h 2 F y n -h 2 ∇f (y n ) (1.67)
In this case, one can also notice that the scheme (1.67) is also valid when the function F is not necessarily smooth, since the proximal operator is well defined for general convex functions. In fact as we shall see in the following Chapter, the fast convergence properties of the ODE (1.18) can be extended in the case of a possibly non-smooth convex function F . Hence, by discretizing similarly the differential inclusion associated to (1.18), we can recover the inertial proximal-gradient algorithm (see algorithm 7 in Chapter 3) and expect the same convergence properties.

More generally, different combinations of explicit-implicit or explicit-explicit discretizations of the coupled system (1.63), can lead to other versions of interesting discrete inertial schemes. Clearly for such variants, one should check the consistency and stability with the associated ODE (1.18), in order to expect similar convergence properties. In view of the Lipschitz character of ∇F , classical discretization methods seem to be consistent with (1.18), under some mild assumptions on the magnitude of the step size h and a "proper choice" for the sequence α n . In particular, it seems that the choice of over-relaxation sequence α n is "closely connected" to the damping coefficient b t . Nevertheless the possible options for α n are endless and they constitute a challenging future area for study and it goes beyond the scope of the current Thesis.

Concluding remarks and perspectives

In this Chapter we presented some results concerning the convergence analysis of the differential equation (1.18), from an optimization point of view. As already mentioned the key-tool of the analysis is based on Lyapunov techniques. These results and their proofs will provide us a useful guideline for the discrete scheme found in Section 1.4. In a parallel way, one can consider systems such as (1.16), with a general damping parameter α(t):

ẍ(t) + α(t) ẋ(t) + ∇F (x(t)) = 0 (1.68)
or even more general systems :

ẍ(t) + α(t) ẋ(t) + β(t)∇F (x(t)) = 0 (1.69)
with some positive function β(t). Systems as (1.69) are still connected with the minimization problem of F . It would be interesting to extend the study made for the system (1.18), for the more general one (1.69) and explore the different conditions over the functions α(t) and β(t), in order to obtain some "good" convergence properties for the solution of (1.69). This question can be also linked with the particular structure of the minimizing function F , as done in Section 1.3.4. Naturally, these studies can also open the way for the exploration of generalized inertial discrete schemes, which preserve the convergence properties of solutions of (1.69), as also pointed out in Section 1.4 (for some results on this direction, one can consult the works of [AC17, AC18d, ACR18]).

In fact, systems such as (1.69), are also connected with the one studied in this Chapter (1.18), via time rescaling, as it was shown in the very recent work [START_REF] Attouch | Fast proximal methods via time scaling of damped inertial dynamics[END_REF] (see also [START_REF] Wibisono | On accelerated methods in optimization[END_REF][START_REF] Wibisono | A variational perspective on accelerated methods in optimization[END_REF] and [START_REF] Ashia C Wilson | A lyapunov analysis of momentum methods in optimization[END_REF] on this question). In particular by rescaling in time, system (1.18) by a positive increasing function s(t), one can obtain :

ÿ(t) + b ṡ(t) s(t) - s(t) ṡ(t) ẏ(t) + ṡ(t) 2 ∇F (x(t)) = 0 (1.70)
where x is a solution of (1.18) and y(t) = x(s(t)).

It is then clear that the system (1.70), is of the form of (1.69). Notice then, that by tuning properly α(t) and β(t), (with respect to s(t)), it is possible that the orders of convergence rates found in Theorem 1.3.2, still hold true for the rescaled solution of (1.18). This can lead to further acceleration of convergence rates, via time reparametrization. It would be interesting to extend the study for general systems such as (1.69), linked with the time rescaling of (1.18) (for some first results on this question, one can consult the work of [START_REF] Attouch | Fast proximal methods via time scaling of damped inertial dynamics[END_REF]). Again this particular question is of a high importance in link with the different possible discretizations of (1.18), or (1.69), for obtaining even more competitive schemes.

Chapter 2

The differential inclusion

In this Chapter we are further analyzing the second-order damped system (1.18) studied in Chapter 1, in a non-differential setting.

In particular in a finite-dimensional Hilbert space (i.e. H = R d , d ∈ N) we are interested in the minimization problem of a proper, lower semicontinuous and convex function F : R d -→ R (not necessarily differentiable). In this setting, instead of the differential equation (1.18), we are led to the study of a differential inclusion which takes the following form :

ẍ(t) + b t ẋ(t) + ∂F (x(t)) 0 (DI)
More precisely we show that the same convergence properties, as in Theorem 1.3.2, hold true (up to minor constraints ) for a general convex (possible non-smooth) function F . Due to a compactness argument used in our analysis, our study is valid in a finite dimensional Hilbert space, hence from now on, all along the current Chapter, we consider, H = R d , where d ∈ N.

In particular, we give an existence result of a shock solution x of (DI) (see Definition 2.1 later on this Chapter), via an approximation scheme. For such a solution we show that "almost" the same fast asymptotic properties as the ones obtained in the differential setting in Theorem 1.3.2 hold true. Due to the lack of regularity of such shock solutions, the convergence bound estimates (1.20), (1.21) and (1.22) found before, hold true almost everywhere.

In the case when the domain of F is the whole space R d , we show that the regularity of a solution x of (DI) is sufficient to obtain exactly the same results concerning the asymptotical behavior of this solution, to the ones obtained for the solution of the ODE (1.18) in the differential setting in Theorem 1.3.2.

In addition, we show that the convergence rate O t -2b convex, lower semi-continuous and proper functions. In particular we show that when F is the absolute value function and b < 3, the rate O t -2b 3 for F (x(t)) to the minimum is exactly achieved, showing therefore the optimality of this bound. The "limit-example" concerning the absolute value function also stresses out the motivation of studying the differential inclusion (DI), instead of the differential equation (1.18).

Finally, as in the previous Chapter, we present a suitable finite difference scheme for the differential inclusion (DI), which corresponds to the inertial algorithm that we study in Chapter 3, for solving composite non-smooth convex minimization problems.

Naturally, the study of a differential inclusion such as (DI), covers a wider class of interesting cases of convex optimization problems, than the study of differential equations like (1.18), considered in the previous Chapter. Typical examples such as constrained minimization problems, LASSO variants, or other general versions involving the 1 norm, or total variation regularizers, are some of the possibilities, arising in the non-smooth, convex optimization domain, where the minimizing function F is convex but not necessarily smooth. Notice also that the case of composite convex minimization problems (i.e. F = f +g where f and g are convex with f smooth) are also covered in the study of systems such as (DI), by simply taking ∂F = ∇f + ∂g.

At the same time, the study of differential inclusions is in general harder and more demanding in terms of technical analysis. For example, existence and regularity of possible solutions of a differential inclusion such as (DI), are also some non-trivial issues that have to be taken into consideration, before proceeding to the convergence analysis.

Nevertheless, as we shall see in Part II, the numerical schemes that one can obtain by discretizing the differential inclusion (DI), correspond to some interesting cases of inertial algorithms for solving non-smooth, convex optimization problems (such as the inertial proximal point algorithm or a particular inertial Forward-Backward type algorithms when F is composed by a smooth and a non-smooth part).

Moving towards the numerical schemes of systems like (DI), some other alternative approaches of the differential equation (1.18) can be taken into consideration, by replacing ∂F in (DI), by different regularized maximal monotone operator-terms. For further details on this topic one can also consult the recent works

[AP17] [AC18b] [BDES18], [BCL18b] [BCL18a]).
The study of time-evolution general systems in involving maximal monotone operators, goes back to [START_REF] Brezis | Monotonicity Methods in Hilbert Spaces and Some Applications to Nonlinear Partial Differential Equations[END_REF] and [START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF] (see also [START_REF] Michael | Differential equations on convex sets[END_REF] and [START_REF] Michael | Generation of semigroups of nonlinear transformations on general banach spaces[END_REF]), where systems like the following ẋ(t) + A(x(t)) 0

(2.1)

Existence and regularity of solutions

with A : H ⇒ H, a maximal monotone operator (for the definition of a maximal monotone operator, see Definition 20.1 in [START_REF] Heinz | Convex analysis and monotone operator theory in Hilbert spaces[END_REF]), are taken into consideration.

Classical theory for maximal monotone inclusions, introduced in [Bre73], permits to prove existence and uniqueness of a solution x(•) of (2.1), such that x ∈ W 1,∞ ((0, +∞); H), for a given initial condition x(0) = x 0 ∈ H.

In the context of the minimization problem of a convex function F , by replacing A with ∂F (recall that from convexity an lower semi-continuity of F , ∂F is maximal monotone ), by (2.1), we can recover the subgradient flow:

ẋ(t) + ∂F (x(t)) 0 (SF)
which generalizes the Gradient Flow (GF) and whose unique solution-trajectory minimizes the convex function F ( see for example [START_REF] Brezis | Monotonicity Methods in Hilbert Spaces and Some Applications to Nonlinear Partial Differential Equations[END_REF], [START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF], [START_REF] Jb Baillon | Un exemple concernant le comportement asymptotique de la solution du problème du dt + ∂ϕ(u(t)) 0[END_REF], [START_REF] Cabot | Inclusion of subdifferentials, linear well-conditioning, and steepest descent equation[END_REF], [START_REF] Güler | Convergence rate estimates for the gradient differential inclusion[END_REF], [START_REF] Jérôme Bolte | The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF], [START_REF] Marcellin | Evolution problems associated with primal lower nice functions[END_REF], [GAS + 05] and their possible references). Following the second-order extensions on the differential setting, further analysis for second-order differential inclusions have been made in the works [START_REF] Attouch | The dynamics of elastic shocks via epigraphical regularization of a differential inclusion. barrier and penalty approximations[END_REF] and [START_REF] Cabot | Asymptotics for some vibro-impact problems with a linear dissipation term[END_REF] for the Heavy-Ball with constant-friction system :

ẍ(t) + α ẋ(t) + ∂F (x(t)) 0 (2.2)
where α > 0. Some other more general similar schemes, where the viscosity term α ẋ(t) is replaced by a general Lipschitz continuous function h(t, x(t), ẋ(t)) or other dry friction terms ∂F ( ẋ(t)), can be found in [START_REF] Schatzman | A class of nonlinear differential equations of second order in time[END_REF], [START_REF] Schatzman | Uniqueness and continuous dependence on data for one-dimensional impact problems[END_REF], [START_REF] Paoli | Analyse numérique de vibrations avec contraintes unilatérales[END_REF], [START_REF] Paoli | An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints[END_REF] and [START_REF] Paoli | A numerical scheme for impact problems i: The one-dimensional case[END_REF]) and [START_REF] Adly | Finite time stabilization of nonlinear oscillators subject to dry friction[END_REF] respectively. Here we mainly follow the strategy of works [START_REF] Paoli | An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints[END_REF] and [START_REF] Attouch | The dynamics of elastic shocks via epigraphical regularization of a differential inclusion. barrier and penalty approximations[END_REF] in order to establish the existence of a proper solution to (DI).

Existence and regularity of solutions

Firstly we give some preliminary notions on the definition of a solution of the differential inclusion (DI). As already mentioned, basic results for existence and regularity of solutions of first-order systems like (2.1), with some initial condition, were first given in [START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]. Classical approaches to establish existence of solutions for general differential inclusions, invoke Moreau-Yosida approximations (see for example [START_REF] Brezis | Monotonicity Methods in Hilbert Spaces and Some Applications to Nonlinear Partial Differential Equations[END_REF], [START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF], [START_REF] Cabot | Asymptotics for some vibro-impact problems with a linear dissipation term[END_REF] [Bai78], [START_REF] Schatzman | Uniqueness and continuous dependence on data for one-dimensional impact problems[END_REF]) or constructions of other approximations by constructing interpolation functions, converging to a solution (see for example [GAS + 05], [START_REF] Paoli | Analyse numérique de vibrations avec contraintes unilatérales[END_REF], [START_REF] Paoli | A numerical scheme for impact problems i: The one-dimensional case[END_REF]).

Following the same strategy, we construct an approximating scheme that converge to a solution of (DI), in some proper way (see the approximation scheme in Theorem 2.1.2), in order to obtain the fast asymptotic properties of a solution of (2.4).

Remark 5. The inclusion (DI) can be written equivalently as

Ẋ(t) + H(t, X(t)) + A(X(t)) 0 (2.3)
where

X(t) = (x(t), ẋ(t)) T , H t, (a 1 , a 2 ) = (-a 2 , b t a 1 ) T for all t ≥ t 0 and all a = (a 1 , a 2 ) ∈ R d × R d and A (a 1 , a 2 ) = (0, ∂F (a 1 )) T for all a = (a 1 , a 2 ) ∈ R d × R d .
Nevertheless, under this reformulation, the operator A is not necessarily maximal monotone, hence the classical theory for monotone inclusions for existence and uniqueness of a solution of (2.3), can not be applied directly ( for more information in this topic, we address the reader to Proposition 3.13 in Chapter 3 in [START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF] ).

Shock solutions

In this section we will present the results concerning the existence and the regularity of a solution of (DI). As mentioned before, most of these results are established in a more general setting (see for example [START_REF] Attouch | The dynamics of elastic shocks via epigraphical regularization of a differential inclusion. barrier and penalty approximations[END_REF] and [START_REF] Paoli | An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints[END_REF]).

Let us first recall the system (DI), starting from an instant t 0 > 0, with some initial conditions x(t 0 ) = x 0 ∈ domF ⊂ R d and a zero starting velocity, ẋ(t 0 ) = 0 :

   ẍ(t) + b t ẋ(t) + ∂F (x(t)) 0 x(t 0 ) = x 0 and ẋ(t 0 ) = 0 (2.4)
As already mentioned the system (2.4) falls into the general one studied in [START_REF] Paoli | An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints[END_REF], for a general continuous function h from R + × R d × R d to R d and Lipschitz in its last two arguments with respect to the first one.

   ẍ + ∂F (x) h(t, x, ẋ) x(t 0 ) = x 0 ∈ domF and ẋ(t 0 ) = v 0 ∈ T domF (x 0 ) (2.5)
where T K denotes the tangent cone of a closed convex set K , i.e. for all x ∈ K,

T K (x) = u -x s : s > 0, u ∈ K
As one can expect, the solutions of (DI) can not enjoy the same regularity properties (such as the C 2 (R d ) character), as the ones of the ODE (1.18) in the differential setting. To see that, we can, for example, simply consider the case of a constrained minimization problem, which can be formulated as minimizing the function F = f + ι C , where f is a smooth and convex function and ι C is the indicator function of a closed convex set C. In this case the possible solutions of (DI) can eventually "bounce" on the domain of F , creating some shock-behavioral trajectories, which mathematically can be translated to the presence of possible discontinuities of the derivative (velocity) of the trajectory.

Here we recall the basic results concerning the definition and the existence of a solution for (DI). For a detailed presentation and proofs of these results for the general differential inclusion (2.5), we address the reader in the work of [START_REF] Paoli | An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints[END_REF].

Definition 2.1. Let I = [t 0 , +∞). A function x : I -→ R d is an energyconserving shock solution of (DI) if the following conditions hold :

1. x ∈ C 0,1 ([t 0 , T ]; R d ), for all T > t 0 i.e. x is a Lipschitz continuous function, 2. ẋ ∈ BV ([t 0 , T ]; R d ), for all T > t 0 , 3. x(t) ∈ domF , for all t ∈ I,
4. For all φ ∈ C 1 c (I, R + ) and v ∈ C(I, domF ), it holds :

T t 0 F (x(t)) -F (v(t) φ(t)dt ≤ ẍ + b t ẋ, (v -x)φ M×C , (2.6)
In fact in this case, we have that (2.4) holds almost everywhere in I.

5.

x satisfies the following energy-conserving equation :

F (x(t)) -F (x 0 ) + 1 2 ẋ(t) 2 - 1 2 v 0 2 + t t 0 b s ẋ(s) 2 ds = 0 (2.7)
almost everywhere in I.

We then have the following existence result (see in particular Theorem 3.1 in [START_REF] Paoli | An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints[END_REF]).

Theorem 2.1.1. Let F be a lower semi-continuous convex function. The system (2.4) admits a shock solution x in the sense of Definition 2.1. In fact we have that (DI) holds a.e. in I.

Following the work [START_REF] Paoli | An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints[END_REF], in order to establish the existence of a shock solution in the sense of Definition 2.1, we consider the Moreau-Yosida approximation of F (see Appendix A), which we denote as F γ and for all γ > 0 we consider the following family of approximating ODEs :

   ẍγ (t) + b t ẋγ (t) + ∇F γ (x γ (t)) = 0 x γ (t 0 ) = x 0 ∈ domF and ẋγ (t 0 ) = 0 (ADE)
where ∇F γ = ∂F γ is the Yosida approximation of ∂F (we recall that F γ is a continuously differentiable function which gradient coincides with the Yosida approximation ∂F γ of ∂F , see Lemma A.1.1 in Appendix A.)

We give a sketch of the proof of Theorem 2.1.1, since we use some of its elements in the following section, for the study of the asymptotical properties of a solution of (2.4). For a detailed proof of Theorem 2.1.1, we address the reader to [START_REF] Paoli | An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints[END_REF].

The schema of the proof is classic. Find some a-priori estimates for the family of solutions {x γ } γ>0 of (ADE) and its derivatives { ẋγ } γ>0 , {ẍ γ } γ>0 and then conclude by extracting a subsequence which converge to a solution of (2.4) in some suitable space. Here we must stress out that the finite dimensional setting is due to the compactness theorem that we use for the existence of a solution. To our knowledge an existence result of such a solution in an infinite dimensional Hilbert space is not yet known.

Of course -depending on the particular choice of F -other type of approximations of ∂F are also possible and can potentially lead to a different shock solution (than the one obtained by the Moreau-Yosida approximation), or even approximations by construction of interpolating functions (see for example [GAS + 05] or [START_REF] Aubert | A variational approach to removing multiplicative noise[END_REF] in the case of the subgradient flow SF). This could be an interesting topic in relation with the exploration of the convergence properties of different possible numerical approximation schemes, that lead to the differential inclusion (2.4).

In particular, in our case for the Moreau-Yosida approximation, we have the following Theorem ( see proof of Theorem 3.1 in [START_REF] Paoli | An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints[END_REF]) : Theorem 2.1.2. Let {F γ } γ>0 be a family of functions such that F γ is the Moreau -Yosida approximation of F for all γ > 0. Then there exists a subsequence {x γ } γ>0 of solutions of (ADE), that converge to a shock solution x of (2.4) in the following sense : +∞), for all T > t 0 51 2.1. Existence and regularity of solutions

A.1 x γ -→ γ→0 x uniformly on [t 0 , T ] for all T > t 0 A.2 ẋγ -→ γ→0 ẋ in L p ([t 0 , T ]; R d ) , for all p ∈ [1,
A.3 F γ (x γ ) -→ γ→0 F (x) in L p ([t 0 , T ]; R d ) , for all p ∈ [1, +∞), for all T > t 0
In order to prove Theorems 2.1.1 and 2.1.2 we make use of the following a-priori estimates for the approximations {x γ } γ>0 . In particular we have the following :

Lemma 2.1.1. Let {x γ } γ>0 be a family of solutions of (ADE) for any γ > 0. Then :

sup γ>0 { x γ ∞ , ẋγ ∞ } < +∞ (2.8)
Lemma 2.1.2. Let {x γ } γ>0 be a family of solutions of (ADE) for any γ > 0.

Then :

sup γ>0 { ∇F γ (x γ ) 1 , ẍγ 1 } < +∞ (2.9)
From Lemmas 2.1.1 and 2.1.2, one can extract a subsequence denoted as {x γ } γ>0 which converges according to a suitable approximate scheme (see scheme (AS) later on) to a solution of (2.4) in the sense of Definition 2.1.

Let us now turn our attention in the case where domF = R d .

The case of domF = R d

In the case when domF = R d , one can expect more regularity over the solution x of (2.4). In particular we have the following corollary.

Corollary 2.1.1. Let F : R d -→ R be a convex function such that domF = R d . Then the differential inclusion (2.4) admits a solution x in the sense of Definition 2.1, such that :

x ∈ W 2,∞ ((t 0 , T ); R d ) ∩ C 1 ([t 0 , +∞); R d ) , for all T > t 0 i.e.
x is defined everywhere in [t 0 , +∞) and is differentiable with locally Lipschitz gradient.

Remark 6. Notice that when domF = R d , the function is continuous ( as it is convex in R d ), hence the lower semi-continuity property of F is automatically satisfied.

In order to achieve this supplementary regularity for the solution x, we use the following Lemma : Lemma 2.1.3. Let {x γ } γ>0 be a family of solutions of (ADE) for γ > 0. Then :

sup γ>0 { ẍγ ∞ } < +∞ (2.
} γ>0 is bounded in W 2,∞ (t 0 , T ); R d ).
In addition, by using the fact that W 2,∞ ((t 0 , T );

R d ) ⊂ C 1,1 ([t 0 , T ]; R d ) C 1 ([t 0 , T ]; R d ) ( see Theorem 4.5 in [EG15]
and Theorem 1.34 in [START_REF] Robert | Sobolev spaces[END_REF]), we deduce the existence of a subsequence (still denoted as)

{x γ } γ>0 that converges to a function x in C 1 ([t 0 , T ]; R d ).
Furthermore, as ẍγ is bounded in L ∞ ((t 0 , T ); R d ) and L ∞ ((t 0 , T ); R d ) can be identified with the dual space of L 1 ((t 0 , T ); R d ), we also have ( that is the Banach-Alaoglu Theorem, see for example Theorem 3.14 in [START_REF] Clarke | Functional analysis, calculus of variations and optimal control[END_REF] ) that up to a subsequence ( here we extract from the subsequence considered before ) still denoted by {ẍ γ } γ>0 , :

ẍγ * u in L ∞ ((t 0 , T ); R d ) (2.11)
where by uniqueness of the limit (in the distributional sense) we have that ẍ ≡ u ∈ L ∞ ((t 0 , T ); R d ). Hence we have that

x ∈ C 1 ([t 0 , T ]; R d )∩W 2,∞ ((t 0 , T ); R d ).
In fact for all i ∈ N * , one can construct the sequences (of sequences) of functions {{x i h(γ) } γ>0 } i∈N as follows :

x1 h(γ) -→ γ→0 x1 ∈ W 2,∞ ([t 0 , t 0 + 1]) x2 h(γ) -→ γ→0 x2 ∈ W 2,∞ ([t 0 , t 0 + 2]) . . . xi h(γ) -→ γ→0 xi ∈ W 2,∞ ([t 0 , t 0 + i])
(2.12) in a way that every time we extract a subsequence {x i+1 h(γ) } γ>0 from the subsequence considered before {x i h(γ) } γ>0 , for every i ∈ N * . By diagonal extraction we consider the sequence of functions {x i h(1/i) } i∈N . We then define the sequence of functions

{w i } i∈N in [t 0 , +∞), as the W 2,∞ ([t 0 + i, +∞)) extensions of x i h(1/i)
, for all i ∈ N. By this construction there exists a function x : [t 0 , +∞) -→ R d such that the sequence of functions {w i } i∈N converges to, with respect to the W 2,∞ loc ([t 0 , +∞)) norm. This shows that

x ∈ W 2,∞ ((t 0 , T ); R d ) ∩ C 1 ([t 0 , +∞); R d ), for all T > t 0 .
Once the notion of a shock solution of (2.4) is established, we are ready to pass to the asymptotic properties of such a solution. As we shall see the key-element for the analysis is the approximation scheme established in Theorem 2.1.2 and the Lyapunov analysis made in the differential case.

Asymptotic behavior

In this section we are interested in the asymptotic properties of a solution of (2.4). As the regularity of such a solution depends on the domain of F , we will split the presentation into two parts, one which treats the case of a shock solution and the other one concerning the case when domF = R d . In what follows in this section, we denote as x * a minimizer of F and w(t) = F (x(t)) -F (x * ).

Energy estimates for shock solutions

In the same way than the differential setting, we use the same Lyapunov energy in order to deduce the bound-estimates for a shock solution of (DI). The difficulty in the Lyapunov analysis comes from the fact that the solution is not everywhere differentiable, hence we can not differentiate directly E. Nevertheless via an approximation scheme we show that for w(t) and ẋ(t) , the same bound estimates as the ones obtained in in the differential setting for a solution of (1.18), hold true for almost every t ≥ t 0 .

Thus, for λ ≥ 0 and ξ ≥ 0 we define the following energy-function associated to a shock solution of (2.4) :

E(t) = t 2 w(t) + 1 2 λ(x(t) -x * ) + t ẋ(t) 2 + ξ 2 x(t) -x * 2 (2.13)
For the asymptotic properties of a shock solution we will systematically make use of its approximation scheme in the spirit of the study made in [START_REF] Attouch | The dynamics of elastic shocks via epigraphical regularization of a differential inclusion. barrier and penalty approximations[END_REF].

Let {x γ } γ>0 a suitable subsequence of solutions of (ADE) such that the approximation scheme (AS) holds true, i.e. :

A.1 x γ -→ γ→0 x uniformly on [0,T] for all T > 0, A.2 ẋγ -→ γ→0 ẋ in L p ([t 0 , T ]; R d ), for all p ∈ [1, +∞), for all T > t 0 , A.3 F γ (x γ ) -→ γ→0 F (x) in L p ([t 0 , T ]; R d ), for all p ∈ [1, +∞), for all T > t 0 .
Chapter 2. The differential inclusion 54 We will also use the following notations w γ

(t) = F γ (x γ (t)) -F γ (x * ) and h γ (t) = x γ (t) -x * 2 , for all γ > 0. E γ (t) = t 2 w γ (t) + 1 2 λ(x γ (t) -x * ) + t ẋγ (t) 2 + ξ 2 x γ (t) -x * 2 (2.14)
Here we point out that since x γ is solution to (ADE), Lemma 1.3.1 and Corollary 1.3.1, as presented in Chapter 1, hold true for E γ .

Lemma 2.2.1. Let x be a shock solution of (DI) obtained as a limit of the approximation scheme (AS) and x * a minimizer of F . Then the following estimates hold true :

1. For b ∈ (0, 3), λ = 2b 3 and ξ = λ(λ + 1 -b) > 0, the function H(t) = t 2b 3 -2 E(t) is essentially non-increasing in (t 0 , +∞), i.e. for a.e. (s, t) ∈ [t 0 , ∞) 2 with s ≤ t, we have H(t) ≤ H(s). 2. For b ≥ 3, λ ∈ [2, 1 -b] and ξ = λ(b -λ -1) ≥ 0, the function E(t) is essentially non-increasing in (t 0 , +∞).
Proof. Let {x γ } γ>0 a suitable subsequence of solutions of (ADE) such that the approximation scheme (AS) holds. If we name

H γ = t -c E γ , for λ = 2b 3 , ξ = λ(λ + 1 -b) ≥ 0,
≥ 3, λ ∈ [2, 1 -b] and ξ = λ(b -λ -1) ≥ 0, by the second point of Corollary 1.3.1 E γ (t) ≤ E γ (s) for all t 0 ≤ s ≤ t.
(2.17)

Hence by extracting a suitable subsequence when γ → 0, thanks to the approximation scheme (AS) we conclude the second point of Lemma 2.2.1

Asymptotic behavior

We are now ready to give the main Theorem, showing that for a shock solution of (2.4) obtained as a limit of the approximation scheme (AS), the same estimates of Theorem 1.3.2 hold true almost everywhere in (t 0 , +∞), for the different values of the parameter b > 0.

Theorem 2.2.1. Let x be a shock solution of (2.4) obtained as a limit of the approximation scheme (AS) and x * a minimizer of F . Then the following estimates hold true :

1. If b < 3 then there exist some positive constants C 1 , C 2 , such that :

F (x(t)) -F * ≤ C 1 t 2b 3 and ẋ(t) ≤ C 2 t b 3 for a.e. t ≥ t 0 (2.18)
2. If b ≥ 3 then the trajectory {x(t)} t≥t 0 is bounded and there exist some positive constants C 1 , C 2 , such that :

F (x(t)) -F * ≤ C 1 t 2 and ẋ(t) ≤ C 2 t for a.e. t ≥ t 0 (2.19) 3. If b > 3, we have : +∞ t 0 t F (x(t)) -F * dt < +∞ and +∞ t 0 t ẋ(t) 2 dt < +∞ (2.20)
Proof of Theorem 2.2.1. For the first case b < 3, for λ = 2b 3 and ξ = λ(λ+1b) > 0 from point 1. of Lemma 2.2.1, since H(t), is essentially non-increasing, we have :

t 2b 3 w(t) ≤ t 2b 3 -2 E(t) = H(t) ≤ H(t 0 ) = t 2b 3 -2 0 E(t 0 ) for a.e. t ≥ t 0 (2.21)
which gives the first bound of (1.20).

In addition, since ξ = λ(λ + 1 -b) > 0 and H γ is non-increasing, we have:

t 2b 3 -2 x γ (t) -x * 2 ≤ 2H γ (t) ξ ≤ 2H γ (t 0 ) ξ (2.22)
hence, by passing to the limit (up to a suitable subsequence), when γ → 0, we find :

t 2b 3 -2 x(t) -x * 2 ≤ 2H(t) ξ ≤ 2H(t 0 ) ξ < +∞ for a.e. t ≥ t 0 (2.23)
Chapter 2. The differential inclusion 56 Hence, by using the basic convex inequality α + β 2 ≤ 2 α 2 + 2 β 2 , we deduce the existence of a constant C > 0, such that :

t 2b 3 ẋ(t) 2 ≤ 2t 2b 3 -2 E(t) + t 2b 3 -2 x(t) -x * 2 = 2H(t) + t 2b 3 -2 x(t) -x * 2 ≤ 2H(t 0 ) + sup t≥t 0 t 2b 3 -2 x(t) -x * 2 ≤ C for a.e. t ≥ t 0
(2.24) which allows to conclude the first point of Theorem 2.2.1.

For the second point of Theorem

2.2.1, if b ≥ 3, λ ∈ [2, 1 -b] and ξ = λ(b -λ -1) ≥ 0,
from (2.17) for s = t 0 , for all γ > 0, we have :

E γ (t) ≤ E γ (t 0 ) for all t ≥ t 0 (2.25)
By neglecting the non-negative terms we find :

λ(x γ -x * ) + t ẋγ (t) 2 ≤ 2E γ (t 0 ) for all t ≥ t 0 (2.26)
By developing the square term in the left, and re-neglecting the nonnegative terms, we find :

λ 2 x γ -x * 2 + 2λt x γ -x * , ẋγ (t) ≤ 2E γ (t 0 ) for all t ≥ t 0 (2.27)
and since λ ≥ 2, we obtain :

λ x γ -x * 2 + 2λt x γ -x * , ẋγ (t) ≤ 2E γ (t 0 ) for all t ≥ t 0 (2.28)
which is equivalent to :

d dt λt x γ (t) -x * 2 ≤ 2E γ (t 0 ) for all t ≥ t 0 (2.29)
By integrating (2.29) over (t 0 , t) and dividing by λt ≥ t 0 , we find :

x γ (t) -x * 2 ≤ t 0 x γ (t 0 ) -x * 2 t + 2(t -t 0 ) λt E γ (t 0 ) ≤ t 0 x γ (t 0 ) -x * 2 t + 2 λ E γ (t 0 )
(2.30)

By passing to the limit (up to a suitable subsequence) when γ → 0 and using the approximation scheme (AS), we find :

x(t) -x * 2 ≤ t 0 x(t 0 ) -x * 2 t + 2 λ E(t 0 ) for a.e. t ≥ t 0 (2.31)
which shows that sup

t≥t 0 { x(t) -x * } < +∞ 57 2.

Asymptotic behavior

In addition since E is essentially non-increasing, for 2 ≤ λ ≤ b -1 we have :

t 2 w(t) ≤ E(t) ≤ E(t 0 ) < +∞ for a.e. t ≥ t 0 and t ẋ(t) ≤ E(t) + sup t≥t 0 { x(t) -x * } ≤ E(t 0 ) + sup t≥t 0 { x(t) -x * } < +∞ (2.32)
which concludes the second point of the Theorem 2.2.1, with C 1 = E(t 0 ) and

C 2 = E(t 0 ) + sup t≥t 0 { x(t) -x * } For the third point, for b > 3, by choosing λ = b -1 in (1.35) in Lemma 1.3.1, we obtain : Ėγ (t) ≤ (3 -b)tw γ (t) (2.33)
By integrating in [t 0 , T ], we have :

T t 0 tw γ (t)dt ≤ E γ (t 0 ) -E γ (T ) b -3 ≤ E γ (t 0 ) b -3 < +∞ (2.34)
By passing to the limit (up to a subsequence) when γ → 0 thanks to the approximation scheme (AS) we deduce that : T t 0 tw(t)dt ≤ E(t 0 ) b-3 < +∞ Since the last inequality hold for all T > t 0 , we obtain ∞ t 0 tw(t)dt < +∞ . In the same way as before, since b > 3, for λ = 2 in (1.35) of Lemma 1.3.1, we find :

Ėγ (t) ≤ (3 -b)t ẋγ (t) 2 .
(2.35)

By integrating and passing to the limit ( up to a suitable subsequence ) when γ → 0 and using the approximation scheme (AS), we find +∞ t 0 t ẋ(t) 2 dt < +∞ which concludes the proof of the third point of Theorem 2.2.1.

In fact, as in the differential case for b > 3, one has slightly better results, which are summarized in the following Corollary : Corollary 2.2.1. Let x : be a shock solution of (DI) obtained as a limit of the approximation scheme (AS) and x * a minimizer of F . If b > 3 then :

ess lim t→∞ t 2 F (x(t)) -F * = 0 and ess lim t→∞ t ẋ(t) 2 = 0 (2.36)
and the trajectory converges to a minimizer x * .

Proof of Corollary 2.2.1. The proof follows the ones made in [START_REF] May | Asymptotic for a second order evolution equation with convex potential and vanishing damping term[END_REF] and [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF], by passing from the approximation scheme (AS).

First of all, we consider the following energy function :

U (t) = t 2 w(t) + t 2 2 ẋ(t) 2 ≥ 0 (2.37)
and its approximation,i.e. for all γ > 0 and x γ solution to (ADE) :

U γ (t) = t 2 W γ (t) + t 2 2 ẋγ (t) 2 ≥ 0 , ∀t ∈ [t 0 , +∞) (2.38)
By differentiating, we have :

d dt U γ (t) = t 2 z, ẋγ (t) + t 2 ẍγ , ẋ(t) + 2tW γ (t) + t ẋ(t) 2 (2.39)
By using (ADE) and b > 3, we find

d dt U γ (t) = 2tW γ (t) + (1 -b)t ẋγ (t) 2 ≤ 2tW γ (t) (2.40)
We now define the function Θ γ (t) = U γ (t) -t t 0 2sw γ (s)ds. By definition, Θ has non-positive derivative, hence it is non-increasing, i.e.

Θ γ (t) ≤ Θ γ (s) ∀t 0 ≤ s ≤ t
(2.41)

By passing to the limit in (2.41) up to a subsequence when γ → 0, thanks to the convergence scheme (AS), we obtain that the function Θ(t) = U (t)t t 0 2sw(s)ds is essentially non-increasing. In addition from Theorem 2.2.1 for b > 3 we have that tw(t) is integrable, therefore the function Θ is essentially bounded from below. Since it is also essentially non-increasing, it is essentially convergent i.e. : ess lim t→∞ Θ(t) = l ∈ R As a consequence we have that U (t) is also essentially convergent when t → +∞ with :

ess lim t→+∞ U (t) = ess lim t→+∞ Θ(t) + +∞ t 0 2tw(t) dt ∈ R (2.42)
Finally since b > 3, by Theorem 2.2.1 on a : For this we use the continuous version of Opial's Lemma (see Lemma A.0.1 in Appendix A). In fact we will invoke the previous Opial's Lemma A.0.1 with S = arg min F and since in our framework as H = R d is finitedimensional, we also deduce strong convergence of x(t) to a point of S = arg min F .

+∞ t 0 1 t U (t) dt = +∞ t 0 tw(t) dt + 1 2 +∞ t 0 t ẋ(t) 2 dt < +∞ (2.
By Theorem 2.2.1 for b > 3 and suitable λ and ξ the energy function E is essentially non-increasing and bounded from below (at least by zero ), so it is essentially convergent. By developing the term λ(x(t) -x * ) + t ẋ(t) 2 in the definition of E, we have :

E(t) = t 2 w(t) + t 2 ẋ(t) 2 + λt x(t) -x * , ẋ(t) + λ 2 + ξ 2 x(t) -x * 2 (2.45)
Since for b > 3, we shown that ess lim t→∞ t 2 w(t) = 0 and ess lim t→∞ t ẋ(t) = 0, from (2.45), we deduce that x(t) -x * essentially converges with :

ess lim t→∞ x(t) -x * = ess lim t→∞ 2E(t) λ 2 + ξ (2.46)
Since x is Lipschitz continuous function we deduce that lim t→∞ x(t) -x * ∈ R. This shows that the first condition of Opial's Lemma is satisfied.

For the second condition, let x be a weak-cluster point of the trajectory x(t), when t → +∞. By lower semi-continuity of F , we have that :

F (x) ≤ lim inf t→∞ F (x(t))
(2.47) By Theorem 2.2.1 we have that ess lim t→∞ F (x(t)) = F (x * ), where x * is a minimizer, so that x ∈ arg min F , which shows that the second condition of Opial's Lemma is satisfied, therefore we can conclude the proof by applying Opial's Lemma.

The case of domF = R d

In this section we present the results concerning the asymptotic analysis in the case when domF = R d . In that case the regularity of a solution of (DI) allow to have finer results than in the previous paragraph. In fact given the regularity W 2,∞ ((t 0 , T ); R d ) ∩ C 1 ([t 0 , +∞); R d ) of a solution x of (DI), most of the results presented here can be obtained as direct corollaries from Theorem 2.2.1 and Corollary 4.5.2 ( remark that when domF = R d , w(t) and ẋ(t) are defined for all t ≥ t 0 ). Nevertheless we only give some brief notions to stress out the importance of this supplementary regularity of the solution in the case where domF is the whole space R d .

In particular, the key-property for the analysis made in this subsection is the absolute continuity of ẋ. This together with a generalized chain rule for differentiation (see Lemma A.2.1 in appendix) make the proofs well adapted to the ones made in the differential setting in chapter 1 and there is no need pass through the different approximation schemes as done in subsection 2.2.1.

In fact, for any T > t 0 , if x is a solution of (2.4) in W 2,∞ ((t 0 , T ); R d ), we have in particular that x ∈ W 1,2 ((t 0 , T )) and the function

h(t) = -ẍ(t) - b t ẋ(t) is in L 2 ((t 0 , T ); R d ).
Therefore, in view of Lemma A.2.1, the function w(t) is absolutely continuous in [t 0 , T ] with :

ẇ(t) = z, ẋ(t) ∀z ∈ ∂F (x(t)) a.e. in (t 0 , T ) In addition as ẋ ∈ W 1,∞ ((t 0 , T ); R d ), it is in particular Lipschitz continu- ous in (t 0 , T ) ( see characterization of W 1,∞ space, Theorem 4.1 in [Hei05] or Theorem 4.5 in [EG15] ), therefore it is also absolutely continuous in [t 0 , T ].
As a consequence we have the following proposition. Proposition 2.2.2. Let x ∈ W 2,∞ ((t 0 , T ); R d ) ∩ C 1 ([t 0 , +∞); R d ) , ∀T > t 0 , be a solution of (DI) obtained as a limit of the approximation scheme (AS) and x * a minimizer of F . Then E(t) is absolutely continuous with :

Ė(t) ≤ (2 -λ)tw(t) + (λ + 1 -b)t ẋ(t) 2 + ξ + λ(λ + 1 -b) ẋ(t), x(t) -x * a.e. in (t 0 , T ) (2.48)
or equivalently :

Ė(t) ≤ (2b -3λ)tw(t) + 2(λ + 1 -b) E(t) t -(λ 2 + ξ)(λ + 1 -b) h(t) t + ξ -λ(λ + 1 -b) ẋ(t), x(t) -x * (2.49)
From Proposition 2.2.2, we deduce the following Theorem whose proof follows the same lines of the ones of Theorem 2.2.1 and Corollary 4.5.2 and is let to the reader.

Theorem 2.2.2. Let x ∈ W 2,∞ ((t 0 , T ); R d ) ∩ C 1 ([t 0 , +∞); R d ) , ∀T > t 0 ,
be a solution of (DI) obtained as a limit of the approximation scheme (AS) and x * a minimizer of F . Then the following estimates hold true for all t ∈ [t 0 , T ]: 2.3. Optimality in the sub-critical region 1. If b < 3 then there exist some positive constants C 1 , C 2 , such that :

F (x(t)) -F * ≤ C 1 t 2b 3 and ẋ(t) ≤ C 2 t b 3
(2.50)

2. If b ≥ 3 then the trajectory {x(t)} t≥t 0 is bounded and there exist some positive constants C 1 , C 2 , such that : In addition, the trajectory {x(t)} t≥t 0 converges asymptotically to x * .

F (x(t)) -F * ≤ C 1 t 2 and ẋ(t) ≤ C 2 t (2.51) 3. If b > 3,
We are now ready to give the last result of this Chapter, concerning the optimality of convergence rate of F (x(t)) -F * in the case b ∈ (0, 3).

Optimality in the sub-critical region

As it was already mentioned in Remark 1 in Chapter 1, in [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF] (see example 2.13), the authors give an example that shows that the rate O(t -2 ), for w(t) = F (x(t)) -F * , is optimal, for the class of smooth and convex functions ( where x(t) is a solution of (1.18) ).

In this section we are interested in establishing the optimality of convergence rate for w(t) = F (x(t))-F * , in the sub-critical region for the parameter b > 0 (i.e. b ∈ (0, 3) ), for the class of proper, lower semi-continuous and convex functions.

In this case, Theorem 2.1.1 asserts that (DI) admits a solution

x such that x ∈ W 2,∞ ((t 0 , T ); R d ) ∩ C 1 ([t 0 , +∞); R d ),
for all T > t 0 . In addition, the first point of Theorem 2.2.1 asserts that when 0 < b < 3, the convergence rate of w(t) = |x(t)| to x * = 0 is of order of O t -2b 3 . We show that this order is optimal. we will study the differential inclusion (DI), for 0 < b < 3 when F (x) = |x|. This function enters in the framework studied before and in particular domF = R.

More precisely we have the following Theorem.

Theorem 2.3.1. Let x be a solution of (DI) with F (x) = |x| and 0 < b < 3 such that x(t 0 ) = 0. Then there exists a constant K 1 > 0, such that for any T > t 0 , there exists t > T such that :

|x(t)| ≥ K 1 t 2b 3
.

(2.54)

Before proceeding to the proof, let us stress out some facts concerning the particular example of the absolute value function F (x) = |x|, which we use in our analysis.

Since the minimizer of F is clearly zero (i.e. x * = 0) and F is a convex and positively 1-homogeneous function, we have :

w(t) = F (x(t)) -F (x * ) = |x(t)| = z, x(t) with z ∈ ∂F (x(t)) (2.55)
In addition for any λ, ξ ≥ 0 :

E(t) = t 2 |x(t)| + 1 2 |λx(t) + t ẋ(t)| 2 + ξ 2 |x(t)| 2 (2.56) and for λ = 2b 3 , ξ = 2b(3-b) 9 > 0 and c = 2 -2b 3
H(t) = t -c E(t) = t 2-c |x(t)| + t -c 2 |λx(t) + t ẋ(t)| 2 + ξt -c 2 |x(t)| 2 (2.57)
In order to prove Theorem 2.3.1, we will make use of the following Lemma : Proof. Let T > t 0 . From (2.48) and (2.55) we have :

Ė(t) = (2 -λ)tw(t) + ξ + λ(λ + 1 -b) ẋ(t) • x(t)+(λ + 1 -b)t| ẋ(t)| 2 a.e in (t 0 , T ) (2.58) Since λ = 2b 3 and ξ = 2b(3-b) 9
> 0, by substituting the term t| ẋ(t)| 2 as exactly done in the previous paragraph, we find :

Ė(t) = c t E(t) - 2b(9 -b 2 ) 27t |x(t)| 2 a.e. in (t 0 , T ) (2.59)
where c = 2 -2b 3 By rewriting the previous equation in terms of the functional H(t) = t c E(t), we have :

Ḣ(t) = - 2b(9 -b 2 ) 27 t -c-1 |x(t)| 2 a.e. in (t 0 , T ) (2.60)

Optimality in the sub-critical region

By definition of H (2.57) and its non-increasing property, for all t ≥ t 0 , we have :

t 2-c |x(t)| ≤ H(t) ≤ H(t 0 ) (2.61)
By injecting the last inequality into (2.60), we find :

Ḣ(t) = - 2b(9 -b 2 ) 27 t c-5 t 2-c |x(t)|t 2-c |x(t)| ≥ - 2b(9 -b 2 ) 27 t c-5 H(t 0 )H(t) a.e. in (t 0 , T ) (2.62)
Hence if we set the functions ψ(t) = 2b(9-b 2 )H(t 0 ) 27(c-4) t c-4 and Ψ(t) = H(t)e ψ(t) for all t ≥ t 0 , we have that ψ and Ψ are absolutely continuous with :

Ψ(t) = e ψ(t) Ḣ(t) + ξH(t 0 )H(t) ≥ 0 a.e. in (t 0 , T ) (2.63)
where we used the relation (2.62) for the last inequality. From (2.63), and the absolute continuity of Ψ, we deduce that it is nondecreasing on every interval (t 0 , T ) and since it is continuous we have that Ψ is non-decreasing function for all t ≥ t 0 .

Hence for all t ≥ t 0 , we obtain :

H(t) ≥ H(t 0 )e ψ(t 0 )-ψ(t) ≥ H(t 0 )e ψ(t 0 ) > 0 (2.64)
Since H is non increasing function and bounded from below, with inf t≥t 0

{H(t)} ≥

H(t 0 )e ψ(t 0 ) > 0, we have that lim t→∞ H(t) = l ≥ H(t 0 )e ψ(t 0 ) > 0.

We are now ready to give the proof of Theorem 2.3.1.

Proof. From relation 2.64, we have that :

E(t) = H(t)t c ≥ K 1 t c (2.65)
where K 1 = H(t 0 )e ψ(t 0 ) . Let T > t 0 . We distinguish four cases :

1. There exists some t 1 > T , such that :

1 2 |λx(t 1 ) + t ẋ(t 1 )| 2 + ξ 2 |x(t 1 )| 2 ≤ K 1 2 t c 0 (2.66)
Then from definition of E(t) and (2.65), we deduce that :

t 2 1 |x(t 1 )| ≥ K 1 t c 1 - K 1 2 t c 0 ≥ K 1 2 t c 1 (2.67)
which concludes the proof.

Chapter 2. The differential inclusion 64 2. There exists some t ≥ T such that ẋ(t) = 0. By using the fact that E(t) = t 2 x(t) + λ 2 +ξ 2 |x(t)| 2 and (2.65), we have :

t 2 |x(t)| ≥ K 1 t c - λ 2 + ξ 2 |x(t)| 2 (2.68)
Since lim t→∞ |x(t)| 2 = 0, there exists some t ≥ T , such that |x(t)| 2 ≤ K 1 2 t c 0 , hence we can conclude as in the first point.

3. There exists some t 3 > T such that x(t 3 ) = 0. Since lim t→∞ |x(t)| = 0, there exists t > t 3 such that ẋ(t) = 0 thus we can use the previous point to conclude.

4. Finally we suppose that x(T ) > 0 and that the sign of ẋ is constant for all t ≥ T . Since lim t→∞ |x(t)| = 0 we deduce that sign( ẋ(t)) < 0, for all t ≥ T . In addition for all t ≥ T , we have :

x(t) -x(T ) = t T ẋ(s)ds (2.69)
Since x(t) converges to 0, we deduce that for any η > 0, there exists t η ≥ T such that |t η ẋ(t)| < η. Hence for any , there exists t ≥ T , such that :

1 2 |λx(t ) + t ẋ(t )| 2 + ξ 2 |x(t )| 2 < (2.70)
thus we can conclude as in the first point.

This concludes the proof of Theorem 2.3.1.

Discretization of the differential inclusion (DI)

Finally, as in the differential setting we derive a particular finite difference scheme for the system (2.4), which corresponds to the inertial Forward-Backward algorithm that interests us in Chapter 3 in the second Part of this Thesis.

In particular, in the case of non-smooth, composite convex optimization problems, where the minimizing function can be decomposed as F = f + g, with f a convex differentiable function with L-Lipschitz gradient and g a 65 2.4. Discretization of the differential inclusion (DI) proper, lower semi-continuous convex function, the inclusion (DI) takes the following form:

ẍ(t) + b t ẋ(t) + ∇f (x(t)) + ∂g(x(t)) 0 (2.71)
Because of the regularity of functions f and g, we will discretize (2.71) implicitly with respect to the non-smooth part g and explicitly with respect to the smooth one f . By proceeding as in the differential setting in Section 1.4, if we fix a time-step h, t 0 = hb and the grid t n = (n+b)h and x n ≈ x(t n ), by (2.71) we have :

x n+1 -2x n + x n-1 h 2 + b (n + b)h x n -x n-1 h + ∇f (y n ) + ∂g(x n+1 ) 0 (2.72)
where

y n = x n + n n+b (x n -x n-1
) (the reason of this choice follows the one in the differential case in Chapter 1). By multiplying by h 2 , the last inclusion is equivalent to

x n+1 -2x n + b n + b (x n -x n-1 ) + h 2 ∇f (y n ) + h 2 ∂g(x n+1 ) 0 (2.73)
Hence by re-arranging the terms we obtain :

x n+1 + h 2 ∂g(x n+1 ) x n + (1 - b n + b )(x n -x n-1 ) -h 2 ∇f (y n ) (2.74)
which by using the definition of y n and of the proximal operator, gives :

y n = x n + 1 - b n + b (x n -x n-1 ) x n+1 = Id + h 2 ∂g -1 y n -h 2 ∇f (y n ) = Prox h 2 F y n -h 2 ∇f (y n ) (2.75)
By the definition of the proximal operator, schema (2.75) is well defined and corresponds exactly to the inertial Forward-Backward algorithm (see algorithm 7) with a step γ = h 2 that we study in Chapter 3, for composite (non-smooth) convex minimization problems.

Here we shall stress out that even if the differential inclusion (possibly) admits more than one shock solutions, the particular discretization (2.72), leads to a uniquely defined discrete scheme. As we shall see in Part II, the scheme (2.75), preserves the same convergence properties of the differential inclusion (DI) as stated in Theorem 2.2.1 and Corollary 2.2.1.

Notice that unlike the differential setting, here the generalized Lipschitz continuity for ∂F is not valid in general (for a definition of Lipschitz continuity for set-valued mappings see Definition 9.26 in [START_REF] Tyrrell | Variational analysis[END_REF]). Hence studying the convergence (consistence and stability) of every possible finite difference scheme, can be a much more hard and challenging issue (see for example [START_REF] Taubert | Converging multistep methods for initial value problems involving multivalued maps[END_REF] and [START_REF] Dontchev | Difference methods for differential inclusions: A survey[END_REF] for some specific cases of differential inclusions). In addition, in view of the non-uniqueness of a shock solution to 2.4, even if a possible discretization scheme converges to such a solution, it is possible that this particular solution does not satisfy the convergence properties of Theorem 2.2.1 and Corollary 2.2.1.

Concluding remarks and perspectives

In this Chapter we showed that the fast asymptotic convergence properties of the second order-damped system, introduced in Chapter 1, extend to the non-differential case, for the differential inclusion (DI). Our analysis followed the construction of an appropriate approximation scheme, which permits to deduce the existence of a shock solution to (DI), as also to take advantage of the different Lyapunov techniques used in the differential setting.

We let for future study the analysis of the differential inclusion (DI), under some additional assumptions on the geometry of F , as exactly done in Section 1.3.4 of Chapter 1, in the differential setting. In fact, since this analysis follows the same Lyapunov arguments, we believe that the corresponding results of Section 1.3.4, are still valid in the more general setting of a nondifferentiable convex function.

Similarly to the differential setting, it would be also interesting to extend the study for second-order systems with a general damping parameter, in the non-differentiable setting, i.e. :

ẍ(t) + α(t) ẋ(t) + ∂F (x(t)) 0 (2.76)
where α(t) is a positive function or even more generally:

ẍ(t) + α(t) ẋ(t) + β(t)∂F (x(t)) 0.
(2.77)

The study of inclusions (2.76) (or (2.77)) also opens the way for different choices of inertial algorithms, identified as finite difference scheme of (2.76) (or (2.77)), for solving convex non-smooth minimization problems.

for γ > 0, one can define its Yosida approximation ∂Φ γ as :

∂Φ γ = 1 γ (Id -J γ∂Φ ) where J γ∂Φ = (Id + γ∂Φ) -1 (A.2)
In particular, for any γ > 0, one can define the Moreau-Yosida approximation of Φ, Φ γ as follows :

Φ γ (x) = min y∈R d {Φ(y) + x -y 2 2γ } (A.3)
For any γ > 0, Φ γ is a convex, continuously differentiable function with Lipschitz gradient.

Lemma A.1.1. (Proposition 2.11 in [START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF])

Let Φ : R d -→ R be a proper, lower semi-continuous and convex function. For all γ > 0, the gradient of the Moreau-Yosida approximation of Φ coincides with the Yosida approximation of ∂Φ (i.e. ∇Φ γ = ∂Φ γ ). In addition the following convergence property holds :

Φ γ (x) γ→0 Φ(x) , ∀x ∈ R d (A.4)
Here we recall a basic definition of absolutely continuous functions (see for instance Definition 1.1.1 in [GAS + 05] )

Definition A.1. Let [a, b] be an interval in [t 0 , +∞). A function G : [a, b] -→ R
is said to be absolutely continuous if for every ε > 0, there exists δ > 0 such that for every finite collection

{[a i , b i ]} i∈J of disjoint subin- tervals of [a, b], we have i∈J b i -a i < δ =⇒ i∈J |G(b i ) -G(a i )| < ε (A.5) Equivalently a function G : [a, b] -→ R is absolutely continuous if there exists a function v ∈ L 1 (a, b), such that G(t) = G(s) + t s v(τ )dτ ∀a ≤ s ≤ t ≤ b (A.6)
and in that case we have that G is differentiable a.e. in (a,b) with Ġ(t) = v(t) a.e. in (a, b).

A.2. Subdifferential calculus

A.2 Subdifferential calculus

The following Lemma consists of a generalization of the chain rule for differentiation for composition of W 1,2 -regular functions with convex functions ( see Lemme 3.3 in [START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF] ).

Lemma A.2.1. Let T > t 0 and F be a convex, lower semi-continuous, proper function and

x ∈ W 1,2 ((t 0 , T ); R d ). Let also h ∈ L 2 ((t 0 , T ); R d ), such that h ∈ ∂F (x(t)) a.e. in (t 0 , T ). Then the function F • x : [t 0 , T ] -→ R is absolutely continuous in [t 0 , T ] with : d dt F (x(t)) = z, ẋ(t) ∀z ∈ ∂F (x(t)) a.e. in (t 0 , T ) (A.7)
The following Lemma shows that the subdifferential of a convex function defined in R d , preserves the boundedness of sets. Proof. By contradiction we assume that there exists a subsequence in A noted as {z n } n∈N such that z n ∈ ∂g(x n ) for all n ∈ N and z n → +∞, where {x n } n∈N is bounded (x n ∈ K for all n ∈ N).

From boundedness of {x n } n∈N we deduce that up to a subsequence still noted as {x n } n∈N we have that x n -→ x ∈ K. For all n ∈ N we define the sequence {e n } n∈N as

e n =    zn zn if z n = 0 1 otherwise
It is clear that e n ≤ 1, hence there exists a subsequence noted again as

{e n } n∈N such that e n -→ e ∈ R.
From the definition of subdifferential, as z n ∈ ∂g(x n ), we have that :

g(x n + e n ) -g(x n ) ≥ z n , e n = z n ∀n ∈ N (A.8)
By taking the limit to n → +∞ from continuity of g ( since it is convex on an open set in a finite dimensional space ) we obtain that the Left-Hand-Side of the previous inequality converges to g(x + e) -g(x) which is finite. On the other side by hypothesis we have that z n diverges to infinity, which leads to a contradiction.

Chapter 3

Inertial proximal-gradient algorithms

In this Chapter we are interested in studying some efficient methods for solving numerically non-smooth, convex composite minimization problems with a specific splitting structure. The basic formulation for such problems is the following :

min x∈H F (x) (M)
where H is a Hilbert space and the minimizing function F can be decomposed as F = f + g : H -→ R = R ∪ {+∞}, with :

H.1 f a convex function in C 1,1 L (H) with L-Lipschitz gradient, i.e. ∃L > 0, such that : ∇f (x) -∇f (y) ≤ L x -y (3.1)
H.2 g a convex, lower semi-continuous and proper function (possibly nonsmooth).

H.3

The set of minimizers X * = arg min{F } is non-empty.

A classical hypothesis that assures condition H.3 is the coercivity of the function F which is usually satisfied in practice. In any case, in what follows, we consider these conditions as guaranteed and we also denote

F * = inf x∈H F (x) and x * ∈ H a minimizer of F .
The case of smooth convex optimization, is naturally included in hypotheses H.1, H.2 and H.3, by simply considering g = 0. Therefore all of the convergence results of presented in the following sections are of course also valid in the case of smooth convex minimization.

The structure of (M), includes various interesting optimization problems arising from several domains which became popular the last decades, such as Machine Learning or Image and Signal processing. In a general setting, typical families of such problems can be formulated as the well known linear inverse problems i.e.:

Find x ∈ H , such that : y = Ax (3.2)
where A is a linear bounded operator.

Unfortunately, in the majority of interesting cases the operator A is not invertible, or the calculation of its inverse is very costful in resources (time, memory), hence problem (3.2) is in generally not well posed (for example the solution is not unique or does not depend continuously on the data y ) and it necessitates more information-structure in order to be approached numerically. In order to get around with this inconvenience, instead of searching a solution of problem (3.2), we search for particular solutions of (3.2) that enjoy additional properties, depending on the nature of the problem. Formally this can be done by choosing a function D(x, y) often called data-fit term (with respect to data y) and R(x) often called regularizer term. Then, the problem (3.2) can be then transformed to a generalized minimization problem, as follows : where the real term λ, plays the role of a trade-off parameter between the data-fit term D (fidelity on the initial data of the problem) and the regularizer R (additional properties of the solution of the problem). Some standard smooth choices for the data-fit term D(x, y), is the squared 2 -norm, i.e. D(x, y) = x-y 2 2 or more generally D(x, y) = Ax-y 2 2 , with A a linear mask-operator or the Huber-loss function (see [START_REF] Garrigos | Iterative regularization via dual diagonal descent[END_REF]) D(x, y) = H δ (x -y), with:

H (x) =    1 2 x 2 if |x| ≤ δ δ|x| -1 2 δ 2 otherwise
Some of the classical choices for the regularizer term, include the following:

• the indicator function of a closed convex set C:

R(x) = ι C (x) =    0 if x ∈ C +∞ otherwise
for solving convex constrained optimization problems.

• the squared associated norm:

R(x) = x 2 or R(x) = Bx 2
with a linear bounded operator B, which stands for Tikhonov regularization [START_REF] Tikhonov | Solutions of illposed problems[END_REF].

• the 1 -norm:

R(x) = x 1 , or R(x) = T x 1
with T a linear operator (a dictionary, or wavelet transform for example), which promotes sparsity for the solution (up to the transform T ) and is used for image or signal reconstruction, such as deblurring, inpainting and others, (see for example [START_REF] Donoho | Uncertainty principles and signal recovery[END_REF][START_REF] Donoho | Signal recovery and the large sieve[END_REF][START_REF] Donoho | Compressed sensing[END_REF] or [START_REF] Emmanuel | Enhancing sparsity by reweighted 1 minimization[END_REF]).

• The total-variation norm: R(x) = ∇x 1 which has the ability to preserve the edges (the contour of an image for example), used frequently for image and signal denoising (see for example [START_REF] Rudin | Nonlinear total variation based noise removal algorithms[END_REF] and [START_REF] Chambolle | Image recovery via total variation minimization and related problems[END_REF])

More generally instead of looking to problem (M), one can also consider the general maximal monotone inclusion problem :

Find x ∈ H such that : 0 ∈ A + B (x) (MI)
where A and B are maximal monotone operators, with B cocoercive and such that the set of zeros of A + B is non-empty (i.e. zer(A + B) = ∅). In this context, the minimization problem (M) is a special case of (MI) with A = ∂g and B = ∇f . Since the basic part of this thesis lies on studying the convergence rates for the objective function (i.e. F -F * ) for the minimization problem (M), in what follows we treat the case (M) and not the general one (MI). Nevertheless for similar results concerning algorithms for solving maximal monotone inclusion problems such as (MI), we address the interested reader to some of the possible references [START_REF] Bruck | On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in hilbert space[END_REF], [START_REF] Brézis | Produits infinis de résolvantes[END_REF] [LM79], [START_REF] Gregory B Passty | Ergodic convergence to a zero of the sum of monotone operators in hilbert space[END_REF], [START_REF] George | Convergence rates in forward-backward splitting[END_REF], as also the more recent ones [START_REF] Lorenz | An inertial forward-backward algorithm for monotone inclusions[END_REF], [START_REF] Moudafi | Convergence of a splitting inertial proximal method for monotone operators[END_REF], [START_REF] Liang | Activity identification and local linear convergence of forward-backward-type methods[END_REF], [START_REF] Attouch | Convergence of a relaxed inertial forward-backward algorithm for structured monotone inclusions[END_REF] and [START_REF] Attouch | Convergence of inertial dynamics and proximal algorithms governed by maximally monotone operators[END_REF].

In this chapter we are interested in the convergence properties of an inertial Forward-Backward type algorithm with a particular choice of an overrelaxation term (see Algorithm 7 later on this section), as the one considered in [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF] (see also [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] and [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF]). Firstly we introduce the exact framework of non-smooth convex composite problems and briefly present some of the first methods developed for solving such problems. We then introduce a particular type of Forward-Backward algorithm of Nesterov-type and we present its convergence analysis. As we shall see, the efficiency of this algorithm depends on the inertial parameter. We complete some recent results on the existing literature on Algorithm 7 in the sub-critical case b ≤ 3 (see point 1 of Theorem 3.3.1) and we give a unified presentation (see Theorem 3.3.1) of the different known results of this algorithm (see the works [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF], [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF], [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF], [START_REF] Attouch | The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/kˆ2[END_REF] and [START_REF] Attouch | Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3[END_REF]).

In addition, in Section 3.4, we complete this study by considering an inexact version of the inertial Forward-Backward algorithm (see algorithm 3.72). In particular we study the performance of the i-FB algorithm, with the presence of perturbation error terms, both on the expression of the gradient of f and the proximal operator of g on every iteration. Our analysis follows the same spirit of works [START_REF] Schmidt | Convergence rates of inexact proximal-gradient methods for convex optimization[END_REF], [START_REF] Villa | Accelerated and inexact forward-backward algorithms[END_REF] and [START_REF] Aujol | Stability of overrelaxations for the forward-backward algorithm, application to FISTA[END_REF]. In Theorem 3.4.1, we give some new results for this inexact version, in the case when b ≤ 3 and a unified presentation together with the ones of the existing literature for a general b > 0 (see [START_REF] Schmidt | Convergence rates of inexact proximal-gradient methods for convex optimization[END_REF], [START_REF] Villa | Accelerated and inexact forward-backward algorithms[END_REF], [START_REF] Aujol | Stability of overrelaxations for the forward-backward algorithm, application to FISTA[END_REF], [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF]).

Proximal algorithms

To study problem (M) various method have been developed over the last decades depending on the conditions over the minimizing function F To mention but a few, some of the more popular Gradient descent methods, Proximal point algorithm, splitting methods such as Forward-Backward ([CR97], Here we are interested in the case of solving the minimization problem (M), where the function g is simple, meaning that the expression of its proximal operator has a closed form.

Proximal algorithms

Firstly we recall the definition of the proximal operator (see [START_REF] Jacques | Proximité et dualité dans un espace hilbertien[END_REF]) of a proper, lower semi-continuous and convex function F , which will be necessary for the proposed algorithms.

Prox F = arg min y∈H F (y) + x -y 2 2 (3.5)
Several algorithms have been proposed in order to solve the problem (M), based on the use of the proximal operator, due to the non differentiable part g.

One of the basics and simplest algorithms to tackle minimization problems such as (M) is the Proximal-Gradient splitting algorithm or Forward-Backward algorithm (FB). Forward-Backward scheme consists of a generalization of the classical Gradient-Descent algorithm for solving non-smooth minimization problems. It exploits both smooth and non-smooth structure of the objective function F , by considering a proximal (backward) step with respect to the non-differentiable function g and a gradient (forward) step with respect to the differentiable part f . The FB algorithm consists in applying iteratively at every point with a size-step γ > 0 the non-expansive operator T γ : H -→ H, defined as:

T γ (x) = Prox γg (x -γ∇f (x)) ∀x ∈ H (3.6)
where Prox γg designs the proximal operator of g. The definition of the operator T γ comes from the following observation: By the first-optimality condition for any minimizer x * ∈ arg min F , for all γ > 0, we have:

0 ∈ ∂γF (x * ) = γ∇f (x * ) + γ∂g(x * ) ∈ -x * + γ∇f (x * ) + x * + γ∂g(x * ) (3.7) Hence, x * -γ∇f (x * ) ∈ Id + γ∂g x * (3.8)
or equivalently

x * = Prox γg x * -γ∇f (x * ) = T γ (x * ) (3.9)
Therefore, the problem of minimizing F is equivalent to the problem of finding a fixed point x * of the operator T γ . The F B algorithm with fixed step-size, takes then the following form :

Algorithm 1 FB Let 0 < γ < 2
L and x 0 ∈ H. Repeat :

x n+1 = T γ (x n ) := Prox γg x n -γ∇f (x n ) (3.10)
In the case of convex, smooth minimization problems (i.e. g = 0 in (M)), the F-B algorithm 1 is simply the well-known (explicit) Gradient-Descent algorithm with constant step-size ([Cau47]):

Algorithm 2 G-D Let 0 < γ < 2
L and x 0 ∈ H. Repeat :

x n+1 = x n -γ∇f (x n ) (3.11)
Similarly, if f = 0, the F-B algorithm 1, gives the Proximal point algorithm with constant step-size ([Mar70] [Roc76]) :

Algorithm 3 PP Let 0 < γ < 2
L and x 0 ∈ H. Repeat :

x n+1 = Prox γg x n (3.12)
Forward-Backward algorithm (or similarly Gradient-Descent 2 in the smooth setting or Proximal-Point algorithm 3 if f = 0), has been widely studied (see for example [START_REF] Patrick | Signal recovery by proximal forward-backward splitting[END_REF] or [START_REF] Davis | Convergence rate analysis of several splitting schemes[END_REF] for a unified presentation of this scheme) . In fact, it turns out that the quantitative and qualitative convergence properties of FB algorithm (such as convergence of the iterates and convergence rates) are the same as of the GD (i.e. as if the non-differentiable part g was not present).

In the following Theorem we sum up all the known results of the existing literature, concerning the FB algorithm for solving problem (M).

Theorem 3.1.1. Let F be as in (M) and {x n } n≥1 a sequence generated by the FB Algorithm 1. Then the following hold true : 4. There exist some positive constants C 1 and C 2 such that for all n ≥ 1, it holds:

1. F (x n ) -F *
F (x n ) -F * ≤ C 1 γn and x n -x n-1 ≤ C 2 √ n . (3.13)
In fact asymptotically, we have that:

F (x n ) -F * = o n -1 and x n -x n-1 = o n -1 2 . (3.14)
Optimal rates for first-order methods In plenty of cases Forward-Backward algorithm is very practical and simple to implement and it enjoys good qualitative properties (such as monotonicity of the objective function, Fejer monotonicity of the iterates and weak convergence to a minimizer of F ). Nevertheless, without any other additional properties on F , the quantitative properties such as the convergence rates of F (x n ) -F * or x n -x n-1 , are often slow. In particular as shown in [START_REF] Semenovich | Problem complexity and method efficiency in optimization[END_REF] ( see also [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF]), the optimal rate for first order methods is of order of O(n -2 ), which is not achieved by Forward-Backward algorithm 1. As proven in [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF], the key-element in order to achieve this optimal order and accelerate Forward-Backward algorithm 1, is done by adding an appropriate inertial term on every iteration.

Inertial proximal-gradient algorithms

Inertial (or multi-step) variants of the classical gradient-descent algorithm go back to the works of Polyak [START_REF] Boris | Some methods of speeding up the convergence of iteration methods[END_REF], Faddeev [START_REF] Faddeev | Computational methods of linear algebra[END_REF], Frankel [START_REF] Stanley | Convergence rates of iterative treatments of partial differential equations[END_REF] and Nesterov [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF] (see also [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF]). The idea initially comes from considering finite difference schemes of second-order differential equations such as (1.16) seen in Part I. Thus, in contrary with the FB algorithm, in inertial methods, every new iterate does not only depend on the previous one, but in its extrapolation by a suitable term ( or momentum term ).

In a general way in the setting of the minimization problem (M), an inertial proximal-gradient splitting algorithm takes the following form ([JM17], [START_REF] Liang | Activity identification and local linear convergence of forward-backward-type methods[END_REF]): Algorithm 4 GDIFB Let 0 < γ < 1 L , x 0 = x 1 ∈ H and {α n } n∈N and {β n } n∈N two positive sequences. Repeat :

y n = x n + α n (x n -x n-1 ) (3.15) z n = x n + β n (x n -x n-1 )
(3.16)

x n+1 = Prox γgn y n -γ∇f (z n ) (3.17)
or more simply if we take α n = β n , then the Algorithm 4 takes the following simplified form:

Algorithm 5 GIFB Let 0 < γ ≤ 1
L , x 0 = x 1 ∈ H and {α n } n∈N a positive sequence. Repeat :

y n = x n + α n (x n -x n-1 )
(3.18)

x n+1 = T γ (y n ) = Prox γg y n -γ∇f (y n ) (3.19)
In what follows, we will consider the simplified version of Algorithm 5, unless if otherwise stated.

In particular, in the seminal work of Nesterov in [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF] (see also [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF]) in the smooth convex setting (i.e. g = 0 in (M)), it was shown that considering a particular choice for the inertial parameter α n can lead to some significant fast convergence properties for the trajectories generated. These ideas were further developed in the semi-differential case (where g is not necessarily zero) in [START_REF] Güler | On the convergence of the proximal point algorithm for convex minimization[END_REF], [START_REF] Güler | New proximal point algorithms for convex minimization[END_REF] for the proximal point algorithm (i.e. f = 0) and notably in [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF]. The basic scheme of this inertial Forward-Backward algorithm (i-FB) which is introduced in [BT09] (see also [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF] for the case g = 0) under the name FISTA (Fast Iterative Soft-Thresholding Algorithm) is the following :

Inertial proximal-gradient algorithms

Algorithm 6 FISTA [BT09] [Nes83] Let 0 < γ ≤ 1
L , t 0 = 1 and x 0 = x 1 ∈ H. Repeat :

t n+1 = 1 + 4t 2 n + 1 2
(3.20)

y n+1 = x n + α n (x n -x n-1
) where :

α n = t n -1 t n+1 (3.21) x n+1 = T γ (y n ) = Prox γg y n -γ∇f (y n ) (3.22)
With this particular choice of the inertial parameter α n , one can obtain faster global rates for both the objective function F (x n ) -F * and the local variation x n -x n-1 . In particular we have the following result: Proposition 3.2.1 (Nesterov [Nes83],Beck and Teboulle [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF]). Let F be as in (M) and {x n } n ≥ 1 a sequence generated by FISTA algorithm 6. Then there exist some positive constants C 1 and C 2 , such that for all n ≥ 1:

F (x n ) -F * ≤ C 1 γn 2 and x n -x n-1 ≤ C 2 n . (3.23)
There is a vast literature concerning the study of this type of inertial FB algorithms sometimes called as FISTA (to name but a few, we address the reader to the works [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF], [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF], [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF], [START_REF] Kim | Optimized first-order methods for smooth convex minimization[END_REF], and their possible references). As mentioned before, the choice of the over-relaxation parameter α n plays important role for the convergence of the algorithm. Following the work of Nesterov [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF] and Beck et Teboulle [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF], the basic idea behind the acceleration effect seems to be an "algebraic trick" for the sequence {α n } n∈N , based on the construction of estimate sequences for F (x n ) (see for example [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF], [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF] and [START_REF] Villa | Accelerated and inexact forward-backward algorithms[END_REF]). In particular, if the sequence α n can be written as α n = tn-1 t n+1 , where {t n } n∈N , is a sequence that verifies Nesterov's rule, i.e. :

t 2 n+1 -t n+1 -t 2 n ≤ 0, (3.24)
then the optimal order of O(n -2 ) can be achieved. Of course, other choices for the over-relaxation sequence {α n } n∈N , leading to acceleration are possible. Such choices can depend on additional features of the problem (M), such as the (possible) additional assumptions on the geometry of the minimizing function F . This issue is also treated later on, in Chapter 4. All the same, the question of the choice of the over-relaxation parameter α n , depending on the structure of the minimizing function F is a challenging question and the study of the general inertial schemes such as algorithms 5 and 4, is an active research area the last years. For this issue we address the interested reader to the recent works [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF], [START_REF] Attouch | Convergence rate of a relaxed inertial proximal algorithm for convex minimization[END_REF], as also to [START_REF] Liang | Activity identification and local linear convergence of forward-backward-type methods[END_REF], [START_REF] Patrick | Local and global convergence of a general inertial proximal splitting scheme for minimizing composite functions[END_REF] and [START_REF] Liang | Faster fista[END_REF] where the authors study different conditions over the sequence {α n } n∈N and {β n } n∈N , in order to obtain several convergence properties for the GIFB Algorithms 4 and 5.

Inertial Forward-Backward algorithm

Another choice of a particular interest for the sequence of inertial parameters {α n } n∈N which raised up the last years, is when α n = n n+b for all n ≥ 1 and a parameter b > 0. This corresponds to choose α n = tn-b+1 t n+1 , with t n = n + b -1, where b > 0. This version was considered in numerous recent works (to mention but a few, we consult the works [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF]

, [CD15], [ACPR18] [AP16]

) and as we shall see, leads to significant advantages in comparison to the initial version (i.e. the FISTA algorithm 6 considered in [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF] and [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF]). Formally, the algorithm is the following:

Algorithm 7 i-FB Let 0 < γ < 1 L and x 0 = x 1 ∈ H. Repeat : y n = x n + α n (x n -x n-1
) where :

α n = n n + b (3.25) x n+1 = T γ (y n ) = Prox γg y n -γ∇f (y n ) (3.26)
In the forthcoming analysis we shall refer to the i-FB algorithm for algorithm 7 and we are interested in its convergence properties.

The tuning parameter b

In this point we should mention that with the choice (3.25) for the parameter α n , Nesterov's rule (3.24) is equivalent to considering b ≥ 3. Notice also that with the choices (3.20) and (3.21) for t n and α n (respectively), we have

α n = tn-1 t n+1 ∼ n-1 n+2 ∼ n n+3
, hence the initial choice of the parameter α n in FISTA algorithm 6 is asymptotically equivalent to the choice of α n in the i-FB algorithm 7 with b = 3.

In [START_REF] Nesterov | A method of solving a convex programming problem with convergence rate o(1/k 2 )[END_REF] (see also [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF]) the choice (3.20) for t n that satisfy Nesterov's rule with equality (3.24), seem to be justified by the fact that it minimizes the constant C 1 in (3.23) and it gives the optimal order O(n -2 ) (in fact there is yet a slight amelioration of the hidden constant in the "big Oh" by a factor of 2, as it was shown in the recent work [START_REF] Kim | Optimized first-order methods for smooth convex minimization[END_REF]).

Convergence results for i-FB algorithm

Nevertheless, as we shall see, this choice of the inertial parameter α n , offers a lot of advantages in comparison to the one in (3.21) in FISTA algorithm 6. As it was shown in [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF] (see also [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF]) the authors prove that by assuming that b > 3, one can additionally expect the weak convergence of the iterates {x n } n∈N generated by the i-FB algorithm (7), to a minimizer x * of F . In addition in [START_REF] Attouch | The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/kˆ2[END_REF] the authors show that by taking b > 3 can asymptotically improve the rate of convergence of F (x n ) -F (x * ) which is actually o(n -2 ).

Last but not least, as we already saw in Chapter 1 and 2 of Part I, Algorithm 7 can be identified as a particular finite difference scheme of the second-order dynamical system (DI) (or (1.18) in the differential setting). This remark is of a high importance, since the study of these systems will trace the trail for the different results and their proofs for the discrete scheme 7. In particular, by following the lines of the Lyapunov analysis made in the continuous case, we are able to deduce the corresponding convergence properties for the sequence generated by the i-FB algorithm 7.

Convergence results for i-FB algorithm

In what follows in this Section, we turn our attention into the convergence analysis of the i-FB algorithm 7. We give the main result which treats the case of low momentum (i.e. b ≥ 3) for the i-FB algorithm 7 which was already established in the works [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF], [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF], [START_REF] Attouch | The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/kˆ2[END_REF] and [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF] and a new result concerning the convergence properties in the case of high momentum, (i.e. 0 < b < 3) (see [START_REF] Vassilis Apidopoulos | Convergence rate of inertial forward-backward algorithm beyond nesterov's rule[END_REF] and [START_REF] Attouch | Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3[END_REF]). Theorem 3.3.1. Let b > 0, x n be the sequence generated by i-FB algorithm (7) and x * ∈ arg min F and t n = n + b -1 , for all n ∈ N. Then :

1. If 0 < b < 3 then there exist some positive constants C 1 , C 2 such that for all n ≥ 1, it holds :

F (x n ) -F * ≤ C 1 t 2b 3 n = O n -2b 3 and x n -x n-1 ≤ C 2 t b 3 n = O n -b 3 .
(3.27)

([SBC16], [ACPR18]) If b ≥ 3 then there exist some positive constants

Chapter 3. Inertial proximal-gradient algorithms 84 C 1 , C 2 such that for all n ≥ 1, it holds :

F (x n ) -F * ≤ C 1 t 2 n = O n -2 and x n -x n-1 ≤ C 2 t n = O n -1 .
(3.28)

([CD15], [ACPR18]

) If b > 3 then :

+∞ n=1 t n F (x n ) -F * < +∞ and +∞ n=1 t n x n -x n-1 2 < +∞. (3.29)
In fact as it is pointed out in the continuous setting, one can have better convergence results if b > 3. As it was shown in [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF] and [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF], one can prove the weak convergence of the trajectory to a minimizer, as also that the order of convergence rates found in (3.28) is actually a small "o" asymptotically. In particular we have the following Corollary Corollary 3.3.1. [[CD15], [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF]] Let b > 3, x n be the sequence generated by i-FB algorithm 7. Then it holds :

lim n→∞ F (x n ) -F * = o n -2 and lim n→∞ x n -x n-1 = o n -1 (3.30)
In addition, the sequence {x n } n≥1 weakly converges to a minimizer x * of F .

Lyapunov analysis

As discussed previously, in order to prove the different convergence rates as presented in Theorem 3.3.1, our basic approach is to construct a suitable "energy" function and find some appropriate estimates by using some Lyapunov-type analysis. To that purpose the analysis made in the continuous case in Chapter 1 of Part I and the works of [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF], [START_REF] Attouch | Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity[END_REF] and [START_REF] Aujol | Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b > 0[END_REF], shall provide us a useful guideline.

For that we recall that if {x n } n∈N and x * ∈ arg min F we use the following notations :

w n = F (x n ) -F (x * ), δ n = x n -x n-1 2 and h n = x n -x * 2 . (3.31)
In addition we define the following sequences :

v n = λ(x n-1 -x * ) + t n (x n -x n-1 ) 2 , n ≥ 1, (3.32) with t n = n + b -1 and α n = t n -b + 1 t n+1 = n n + b (3.33)
and the basic Lyapunov-sequence :

E n = t 2 n F (x n -F (x * )) + 1 2γ λ(x n-1 -x * ) + t n (x n -x n-1 ) 2 + ξ 2γ x n-1 -x * 2 = t 2 n w n + 1 2γ v n + ξ 2γ h n-1 (3.34)
Let us remark that by developing the square in v n in the definition (3.34) and using the basic identity

2 u -v, v -z = u -z 2 -v -z 2 -u -v 2 ∀u, v, z ∈ H (3.35) with u = x n , v = x n
-1 and z = x * , we also have the following equivalent definition for E n :

E n = t 2 n w n + 1 2γ v n + ξ 2γ h n-1 = t 2 n w n + (t 2 n -λt n ) 2γ δ n + λt n 2γ (h n -h n-1 ) + (λ 2 + ξ) 2γ h n-1 (3.36)
In particular by the definition of the energy-sequence {E n } n≥1 , we can see that the upper estimates of E n rule the ones for the sequences, t 2 n w n , v n as also δ n and h n . This leads us to the research of some "good" upper bounds for E n , in order to deduce convergence rates for the sequences of interest w n and δ n , as Theorem 3.3.1 asserts.

To do so, we start by studying the local variation of the sequence {E n } n∈N ( i.e. the difference E n+1 -E n ). Using some Lyapunov-type analysis, for some suitable choices of parameters λ > 0 and ξ > 0, we are able to control the growth of {E n } n∈N up to a suitable order. Once this control-estimate is proven, an application of a discrete version of Gronwall's lemma ( see Lemma B.0.3 in Appendix B) will provide the bound for the sequence {E n } n∈N as given in Theorem 3.3.1.

We first present a sketch of this strategy, in order to give a better insight on the different technical Lemmas used in the forthcoming analysis.

1. We start by investigating the local variation of the sequence {E n } n≥1 .

By using the fundamental descent Lemma B.0.7 and performing some algebraic computations and a suitable value for the parameter ξ, we obtain a relation of the following form (see for example relations (3.39) and (3.40) of Lemma 3.3.1):

2γ(E n+1 -E n ) ≤ 2γα n,λ w n + β n,λ δ n + γ n,λ h n-1 (3.37)
for some suitable sequences α n,λ , β n,λ and γ n,λ .

At this point, in order to prove the second and third point of Theorem 3.3.1 it is sufficient to choose suitable values for λ in order to show that γ n,λ ≤ 0, α n,λ ≤ 0 and β n,λ ≤ 0 for all n ≥ 1, under the supplementary hypothesis that b ≥ 3.

2. For the first point (3.27) of Theorem 3.3.1, instead we are interested in the case where b ∈ (0, 3) and the terms α n,λ , β n,λ are not necessarily non-positive for all n ≥ 1. This case demands a more refined analysis as follows. Firstly we express E n as a function of w n and δ n and we find a relation of the form :

2γ(E n+1 -E n ) ≤ 2γ c t n E n + R n,λ (3.38)
3. Finally by some suitable values for λ, we show that : R n,λ ≤ 2γ a t 2 n E n , which permits us to use a recurrence relation (see Lemma B.0.3) in order to conclude point 1 of Theorem 3.3.1.

We now give formally the different Lemmas that structures the aforementioned strategy.

The first Lemma expresses the control of the local variation of E n in terms of the sequences w n , δ n and h n .

Convergence results for i-FB algorithm

Lemma 3.3.1. Let b > 0, {x n } n∈N the sequence generated by i-FB algorithm 7 and 0 ≤ λ ≤ b + 1. Then the following recursive formulas hold true for all n ≥ 1 :

1. If ξ = λ(b -λ -1) then : 2γ E n+1 -E n ≤ 2γ A 1 (λ)t n +1-λ w n + A 2 (λ)t n +(λ+1-b)(1-b) δ n (3.39) where : A 1 (λ) = 2 -λ and A 2 (λ) = λ + 1 -b. 2. If ξ = λ(λ + 1 -b) then : 2γ E n+1 -E n ≤ 2γ c(λ) t n E n +2γ B 1 (λ)t n +1-λ w n +B 2 (λ)δ n + B 3 (λ) t n h n-1 (3.40) where : c(λ) = 2(λ + 1 -b) , B 1 (λ) = (2b -3λ) , B 2 (λ) = λ + 1 -b and B 3 (λ) = 2λ(2λ + 1 -b)(b -λ -1).
Remark 8. Remark that in cases 1. and 2. of Lemma 3.3.1 the value of the parameter ξ, changes sign. Nevertheless, as we shall also see later on in the proofs, due to the restrictions on the parameters λ and b for each of the cases 1. and 2., the parameter ξ always remain non-negative.

Next we give the Lemma 3.3.2 and 3.3.3 concerning the case 0 < b < 3, which expresses the control over the local variation of E n in terms of the same energy sequence E n , which also leads to Lemma 3.3.3, expressing the order of growth of the sequence E n .

Lemma 3.3.2. Let 0 < γ ≤ 1 L , b ∈ (0, 3) λ = 2b 3 , ξ = 2b(3-b) 9
> 0 and {x n } n∈N the sequence generated by i-FB. Then the following recursive formula holds for all n ≥ 1 :

E n+1 -E n ≤ a (n + b -1) 2 + c (n + b -1) E n (3.41)
where a = (3-b)(3+b) 9 and c = 2(3-b) 3 . In particular, by a recurrence argument, from Lemma 3.3.2, for all n ≥ 1 we deduce the following : , there exists a constant C > 0, such that for all n ≥ 1, it holds :

E n+1 ≤ E 1 n i=1 1 + c t n + a t n 2 (3.
E n ≤ C(n + b -1) 2(3-b) 3 (3.43)
We are now ready to pass to the full proof of Theorem 3.3.1.

Proof of Theorem 3.3.1. We split the proof into 2-parts depending on the value of the parameter b and the value of ξ in the definition of E n (3.34).

First of all we begin by the simplest case of proving points 2. and 3. (i.e. estimates (3.28) and (3.29)).

For that if we set λ = b -1 and ξ = λ(b -λ -1) = 0 from (3.39), we have :

2γ

(E n+1 -E n ) ≤ 2γ (3 -b)t n + 2 -b w n (3.44)
Therefore, if b ≥ 3 we deduce that the right-hand part of (3.44) is nonpositive, which entails that the sequence E n is non-increasing. Hence from the definition of E n (3.34), with λ = b -1 and ξ = 0, we find :

2γt 2 n w n ≤ 2γE n ≤ 2γE 1 (3.45)
which gives the first estimation of (3.28) with

C 1 = (b-1) 2 2 x 0 -x * 2 + γb(F (x 0 ) -F (x * )).
In addition, for b ≥ 3, λ = b -1 and ξ = 0, in the definition of E n (3.36), we find :

2γE n = 2γt 2 n w n + (b -1) 2 h n-1 + (b -1)t n (h n -h n-1 ) + (t 2 n -(b -1)t n )δ n ≥ (b -1) t n h n -(t n + 1 -b)h n-1 + (t n -t 0 )t n δ n
(3.46) Since b ≥ 3 and t n ≥ t 0 , from (3.46) we find :

(b -1) t n h n -t n-1 h n-1 ≤ 2γE n ≤ 2γE 1 (3.47)
Therefore by summing over n, we deduce that for all n ≥ 1 :

t n h n ≤ (b -1)h 0 + 2γ b -1 n k=1 E 1 ≤ (b -1)h 0 + 2γE 1 b -1 t n (3.48)
which by dividing by t n , shows that sup

n≥1 x n -x n-1 2 < +∞.
Hence by injecting the last estimation (3.48) into (3.46) -and since t nt 0 ≥ tn t 0 +1 = tn b-2 , ∀n ≥ 1, we deduce that :

t 2 n δ n ≤ (b -2) 2γE n + (b -1)h n-1 ≤ (b -2) 4γE 1 + (b -1)h 0 (3.49)

Convergence results for i-FB algorithm

which gives the second estimation of (3.28) of point 2 of Theorem 3.3.1, with

C 2 = (b -2) 4γ( (b-1) 2 2 (b -1) 2 h 0 + γbw 0 ) + (b -1)h 0 . If now b > 3,
(b -3) n k=1 t 2 k w k ≤ E 1 -E n ≤ E 1 < +∞ (3.50) and b -3 b -2 n k=1 t 2 k δ k ≤ E 1 < +∞ (3.
2γt 2 n w n + 4b 2 9 h n-1 ≤ 2γE n ≤ Ct 2-2b 3 n (3.52)
for a suitable positive constant C. This gives the first estimation of (3.27).

In addition, by the definition of E n and the fact that ξ = 2b(3-b) 9 > 0, there exists some suitable positive constant C such that, for all n ≥ 1 :

h n-1 ≤ C t 2-2b 3 n (3.53)
Finally, since E n is a sum of positive terms, we also have :

v n = 2b 3 (x n-1 -x * ) + t n (x n -x n-1 ) 2 ≤ Ct 2-2b 3 n (3.54)
By taking the square roots in both sides and using the triangle inequality and estimations (3.43) and (3.53), we deduce that for all n ≥ 1 :

t n x n -x n-1 ≤ Ct 2-2b 3 n + 2b 3 h n-1 ≤ Ct 1-b 3 n (3.55)
where C > 0 is a suitable renamed constant. This concludes the proof of Theorem 3.3.1.

Finally we finish this section by giving the proof of Corollary 3.3.1, which establishes the convergence results of the i-FB Algorithm 7, when b > 3.

Proof of Corollary 3.3.1. Following the proof in the continuous setting for the differential inclusion in Chapter 2, for a sequence {x n } n≥1 generated by the i-FB algorithm 7, we define the energy sequence U n , as follows :

U n = t 2 n F (x n ) -F (x * ) + t 2 n x n -x n-1 2 2γ = t 2 n w n + 1 2 t 2 n δ n . (3.56)
From Lemma B.0.7 with y = y n and x = x n we deduce that :

w n+1 -w n ≤ α 2 n δ n -δ n+1 (3.57)
which by definition of α n = n t n+1 , multiplying by t n+1 and re-arranging the terms, is equivalent to

U n+1 -U n ≤ t 2 n -t 2 n+1 w n + (2t n+1 -1)δ n ≤ 2nw n (3.58)
The last inequality implies that the sequence Θ n = U n -2 n k=1 kw k is nonincreasing. In addition, since ∞ k=1 kw k < +∞ (by (3.29) of Theorem 3.3.1), Θ n is also bounded from below, hence convergent.

By (3.29) of Theorem 3.3.1, we also have : ), E n is bounded from below and non-increasing, hence convergent. In addition, by definition (3.34), we have :

∞ n=1 U n t n = ∞ n=1 t n (w n + δ n ) < +∞ (3.
2γE n = 2γt 2 n w n + t 2 n δ n + λ x n-1 -x * , t n (x n -x n-1 ) + (λ 2 + ξ)h n-1 (3.61
) Thus, by passing to the limit when n → ∞ and using relation (3.60), we deduce that :

lim n→∞ x n-1 -x * = lim n→∞ E n λ 2 + ξ (3.62)
which concludes the first point of the Opial's Lemma. In addition if x is a weak-cluster point of the sequence x n , then by lower-semi continuity of F , we have :

F (x) ≤ lim inf F (x n ) (3.63)
and since F (x n ) converges to F * , we deduce that x ∈ arg min F ,which shows that the second condition of Opial's lemma is satisfied, therefore we can conclude the proof of Corollary 3.3.1

Inexact version of the inertial Forward-Backward algorithm

In the previous Section we presented some results of the i-FB Algorithm 7, in the case when the proximal maps and the gradients are explicitly computed. Unfortunately, in many interesting problems the proximal operator of the function g, or the gradient of the differential part f , have not a closed form and their computation is made approximately. This procedure endows some embedded errors on the calculation of the proximal operator and gradient, of the functions g and f respectively, for every outer loop-iteration in the algorithm.

In this section we are addressing this question in the case of the i-FB algorithm 7, in presence of some error parameters on the calculation of the proximal operator of the function g and the gradient of f . Our framework and analysis follows the one considered in [START_REF] Schmidt | Convergence rates of inexact proximal-gradient methods for convex optimization[END_REF], [START_REF] Aujol | Stability of overrelaxations for the forward-backward algorithm, application to FISTA[END_REF] and [START_REF] Salzo | Inexact and accelerated proximal point algorithms[END_REF], [START_REF] Villa | Accelerated and inexact forward-backward algorithms[END_REF] (see also [START_REF] Patrick | Signal recovery by proximal forward-backward splitting[END_REF], [START_REF] Yves F Atchade | On stochastic proximal gradient algorithms[END_REF] and [START_REF] Solodov | A unified framework for some inexact proximal point algorithms[END_REF]).

In particular for the inexact version of the i-FB Algorithm 7, with b = 3, it was shown in [START_REF] Schmidt | Convergence rates of inexact proximal-gradient methods for convex optimization[END_REF], that the fast convergence rate F (x n ) -F * = O(n -2 ) is still valid, provided that the perturbation errors are sufficiently small. Several extensions for other generalized accelerated Forward-Backward inexact schemes were also made in [START_REF] Villa | Accelerated and inexact forward-backward algorithms[END_REF] and [START_REF] Devolder | Firstorder methods of smooth convex optimization with inexact oracle[END_REF], giving the convergence rates of the objective function, depending on the decay order of the errors. Lastly, in [START_REF] Aujol | Stability of overrelaxations for the forward-backward algorithm, application to FISTA[END_REF], the authors studied another similar family of an "hybrid" inertial scheme, interpolating Forward-Backward Algorithm 1 and i-FB Algorithm 7, in the same inexact setting. They proved that the corresponding results of the i-FB Algorithm 7, when b > 3, such as the improved rate o(n -2 ) and the weak convergence to a minimizer (see Corollary 3.3.1), hold also true in the inexact setting, under some proper control over the errors.

Our study completes the aforementioned ones ([SLRB11], [START_REF] Villa | Accelerated and inexact forward-backward algorithms[END_REF], [DGN14] and [START_REF] Aujol | Stability of overrelaxations for the forward-backward algorithm, application to FISTA[END_REF]) for the i-FB Algorithm 7 in the case b ≥ 3. Here we give some associated results concerning the subcritical case for the parameter b (i.e. b ≤ 3). In order to unify the different results and to give a complete presentation, we also give the proof for a general parameter b > 0 (including thus the case of low momentum, where b ≥ 3). More precisely we show that the convergence rates obtained in point 1. of Theorem 3.3.1, when b ≤ 3, hold also true for the perturbed version of the i-FB algorithm, given that the perturbation errors are sufficiently small. In what follows, we keep the same notations as in the unperturbed case for the different sequences.

Inexact computations of the proximal point

Firstly, we introduce the some notions used for the different kind of approximations of the proximal operator of a convex function, which we will use later on. The presentation of this paragraph follows the one of [START_REF] Aujol | Stability of overrelaxations for the forward-backward algorithm, application to FISTA[END_REF], [START_REF] Villa | Accelerated and inexact forward-backward algorithms[END_REF] and [START_REF] Schmidt | Convergence rates of inexact proximal-gradient methods for convex optimization[END_REF].

As recalled in the first section, if F is a proper, convex and lower semicontinuous function and a positive parameter γ > 0, the proximal operator of F , Prox γF is defined by :

Prox γF (y) = arg min x∈H F (x) + 1 2γ x -y 2
Let us denote by G γ , the upper-quadratic approximation of function

F , i.e. G γ (x) = F (x) + 1 2γ x -y 2 . (3.64)
Hence, by definition of the proximal operator, the first order optimality condition for this strongly convex minimum problem yields :

z = Prox γF (y) ⇐⇒ 0 ∈ ∂G γ (z) ⇐⇒ y -z γ ∈ ∂F (z) (3.65)
To begin with, instead of considering the classical notion of the subdifferential ∂F , we introduce the notion of limiting ε-subdifferential of F at the point z ∈ domF , which is defined as follows (see for example section 4.3 in [BNO + 03]).

For any ε > 0 take :

∂ ε F (z) = {y ∈ H | F (x) ≥ F (z) + x -z, y -ε, ∀x ∈ H} (3.66)
It is worth noticing that for any z ∈ dom(F ), the following equivalence holds true:

0 ∈ ∂ ε F (z) ⇐⇒ F (z) ≤ inf F + ε (3.67)
In that case we say that z is an ε-minimizer of F . In this sense, the notion of ε-subdifferential consists of a generalization of the classical subdifferential as given in section 2. Note also that for all ε > 0 and x ∈ H, we have :

∂F (x) ⊂ ∂ ε F (x).
We are now ready to give some definitions on the different types of approximations of the proximal operator of a function F , relying on the notion of ε-subdifferential, that one can also find in [START_REF] Aujol | Stability of overrelaxations for the forward-backward algorithm, application to FISTA[END_REF], following [START_REF] Schmidt | Convergence rates of inexact proximal-gradient methods for convex optimization[END_REF], [START_REF] Salzo | Inexact and accelerated proximal point algorithms[END_REF] and [START_REF] Villa | Accelerated and inexact forward-backward algorithms[END_REF]. 93 3.4. Inexact version of the inertial Forward-Backward algorithm Definition 3.1. We say that z ∈ H is a type 1 approximation of Prox γF (y) with ε precision and we write z ≈ 1 Prox γF (y) if and only if there exists some ε > 0, such that :

0 ∈ ∂ ε G γ (z) (3.68)
where G γ is defined by (3.64).

Definition 3.2. We say that z ∈ H is a type 2 approximation of Prox λF (y) with ε precision and we write z ≈ 2 Prox γF (y) if and only if there exists some ε > 0, such that :

y -z γ ∈ ∂ ε F (z) (3.69) Notice that if z ≈ 2 Prox γF (y), then z ≈ 1 Prox γF (y) (see Proposition 1 in [SV12]).
Finally we make call of a technical lemma taken from [START_REF] Schmidt | Convergence rates of inexact proximal-gradient methods for convex optimization[END_REF] ( see Lemma 2), that enables to consider approximations of types j = 1 or j = 2 in a unified setting, in the forthcoming analysis.

Lemma 3.4.1. If x ∈ H is a type 2 approximation of Prox γF (y) with ε precision, for some ε > 0. Then there exists r such that r ≤ √ 2γε and

y -x -r γ ∈ ∂ ε F (x) (3.70)
Notice from Lemma 3.4.1 that when r = 0, then we recover the definition of a type 2 approximation of Prox γF .

In this framework, we consider the inexact i-FB algorithm as the i-FB Algorithm 7, with presence of errors in both of calculations of the gradient of f and the proximal operator of g on every step, as follows :

Algorithm 8 Inexact i-FB Let 0 < γ ≤ 1
L and b > 0 and t n = n + b -1 for all n ≥ 1. We consider the sequences {x n } n∈N and {y n } n∈N , such that x 0 = x 1 ∈ H and for every n ∈ N * we set :

y n = x n + a n (x n -x n-1 ) where a n = t n + 1 -b t n+1 = n n + b (3.71) x n+1 = T εn en (y n ) (3.72)
where T εn en (x) ≈ εn j Prox γg x -γ(∇f (x) + e n ) where j ∈ {1, 2}

We are now ready to present the main result of this section, concerning the inexact i-FB algorithm 8, which expresses the control estimate over the sequence E n , depending on the error-terms e n ∈ H and ε n > 0 and the parameter b.

Theorem 3.4.1. Let 0 < γ ≤ 1 L , b > 0, t n = n + b -1 and p = min{1, b
3 } and {x n } n∈N the sequence generated by the inexact i-FB algorithm and E n as defined in (3.34). Then for λ = b -p and ξ = 2p(1 -p), there exist some positive constants C 1,p and C 2,p , such that for all n ≥ 1, we have :

E n ≤ C 1,p 2A n + 2(C 2,p + B n ) 2 2γ (n + b -1) 2-2p (3.

73)

where :

A n = n i=1 t p i γ e i + √ 2γε i and B n = γ n i=1 t 2p i ε i (3.74)
By proceeding in the same way as in the proof of Theorem 3.3.1, from the estimate (3.73) for E n , we deduce the following Corollary, expressing the bound estimates on

F (x n ) -F * and x n -x n-1 2 . Corollary 3.4.1. Let 0 < γ ≤ 1 L , b > 0, p = min{1, b
3 } and {x n } n∈N the sequence generated by the inexact i-FB algorithm. Then there exists some constants C 1,p > 0 and C 2,p > 0, such that for all n ≥ 1, we have :

F (x n ) -F (x * ) ≤ C 1,p 2A n + 2 C 2,p + B n 2 2γ(n + b -1) 2p and x n -x n-1 ≤ C 1,p 2A n + 2 C 2,p + B n (n + b -1) p (3.

75)

where :

A n = n i=1 t p i γ e i + √ 2γε i and B n = γ n i=1 t 2p i ε i (3.76)
Remark 9. The last Corollary asserts that under the supplementary hypothesis over the perturbation terms A n and B n , the convergence rates for the inexact i-FB algorithm remain the same as in the unperturbed case (i-FB algorithm). In fact Corollary 3.

F (x n ) -F (x * ) = O n -2p and x n -x n-1 = O n -p . (3.78)
Remark 10. Corollary 3.4.1 and Remark 9, express that even if the control over the error-terms ε n and e n is not sufficiently strong to obtain the optimal rate O(n -2 ) for the objective function, one can still expect some relatively fast convergence for the objective function and the local variation of the iterates of the algorithm (3.72) by further over-relaxing (i.e. taking b ∈ (0, 3) in algorithm 8).

Convergence analysis of the inexact i-FB algorithm

As in the unperturbed case for the i-FB algorithm, in order to prove Theorem 3.4.1, we use some Lyapunov type analysis, for the energy-sequence E n defined earlier in (3.34). For that, we begin by adapting the descent-Lemma B.0.7 of the previous section, in the case of presence of errors on the proximal operator of g and the gradient of f : Lemma 3.4.2. Let y ∈ H and γ ≤ 1 L . For all x ∈ H , we have:

F (x)-F (T ε e (y))+ε+ e+ r γ , x-T ε e (y) ≥ 1 2γ T ε e (y)-x 2 -y-x 2 (3.79)
where r ∈ H such that r ≤ √ 2γε.

For a complete proof of Lemma 3.4.2, see Lemma B.1.4 in appendix. We are now ready to present the proof of Theorem 3.4.1 : Proof of Theorem 3.4.1. For that proof, for all n ≥ 1 we will also denote the sequences

z n = λ(x n-1 -x * ) + t n (x n -x n-1 ) (here remark that z n = √ v n )
and

ζ n = e n + √ 2γεn γ t n √ v n + t 2 n ε n .
By proceeding in the same way than the one in the unperturbed case, by applying Lemma 3.4.2 to y = y n and x = 1 -λ t n+1 x n + λ t n+1 x * we obtain ( here λ ∈ (0, 1 + b)): In that case by using the last inequality, and performing the same computations as the ones made in proof of Theorem 3.3.1, for λ = b -1 and ξ = λ(b -1 -λ) = 0, we find that for all n ≥ 1, it holds:

2γ(t 2 n+1 w n+1 -t 2 n w n ) ≤ 2γk n+1 w n + (t n -1)(x n -x n-1 ) + λ(x n -x * ) 2 -v n+1 + 2γt 2 n+1 ε n+1 -2γt n+1 e n+1 + r n+1 γ , z n+1
E n+1 -E n ≤ (3 -b)t n + (2 -b) w n + (n + b) 2 ε n+1 -(n + b) e n+1 + r n+1 γ , λ(x n -x * ) + t n+1 (x n+1 -x n ) (3.81)
Since b ≥ 3, by using the definitions of z n , t n = n + b -1 and v n , as also the Cauchy-Schwartz inequality and Lemma 3.4.1, we find :

E n+1 -E n ≤ -t n+1 e n+1 + r n+1 γ , z n+1 + t 2 n+1 ε n+1 ≤ e n+1 + √ 2γε n+1 γ t n+1 √ v n+1 + t 2 n+1 ε n+1 = ζ n+1 (3.82)
Hence by summing (3.82), over n ≥ 1, we find :

E n ≤ E 1 + n k=1 ζ k (3.83)
By definitions of E n and ζ n we find :

√ v n 2 ≤ 2γE n ≤ 2γE 1 + 2 n k=1 t k γ e k + √ 2γε k √ v k + n k=1 t 2 k ε k (3.84)
Thus, by applying Lemma B.0.4 with :

u n = t n √ v n , a n = 2t n γ e n + √ 2γε n and S n = 2E 1 + 2B n (3.85)
we find :

√ v n ≤ 2A n + 2 γE 1 + B n (3.86)
Hence by injecting estimation (3.86) into (3.83) and the definitions of A n and B n , we have:

2γE n ≤ 2γE 1 + +2B n + 2A n 2A n + 2 γE 1 + B n ≤ 2A n + 2 γE 1 + B n 2 (3.
E n+1 -E n ≤ (c + a tn ) t n E n -t n+1 e n+1 + r n+1 γ , v n+1 + t 2 n+1 ε n+1 (3.88)
which by using the Cauchy-Schwartz inequality and Lemma 3.4.1, is equivalent to :

E n+1 -E n ≤ (c + a tn ) t n E n + t n+1 e n+1 + r n+1 γ √ v n+1 + t 2 n+1 ε n+1 ≤ (c + a tn ) t n E n + e n+1 + √ 2γε n+1 γ t n+1 √ v n+1 + t 2 n+1 ε n+1 (3.89) Hence by naming µ n = 1 + (c+ a tn ) tn
and the definition of ζ n , for all n ≥ 1, we find :

E n+1 ≤ µ n E n + ζ n+1 (3.90)
By using the discrete Grönwall's Lemma B.0.2, we obtain :

E n ≤ n k=1 µ k E 1 + n k=1 ζ k k+1 j=1 µ j (3.91)
In addition since t n ∼ n, by relation (B.8) of Lemma B.0.3 in Appendix B, there exist some positive constants C 1 and C 2 , such that for all n ≥ 1 it holds :

C 1 t c n ≤ n k=1 µ k ≤ C 2 t c n (3.92)
Hence by injecting (3.92) into (3.91), we find :

E n ≤ C 2 t c n E 1 + C -1 1 n k=1 t -c k ζ k (3.93)
By dividing the last inequality by t c n+1 and using the definitions of E n , ζ n and B n , we find :

t -c 2 n √ v n 2 ≤ 2γC 2 E 1 + 2C -1 1 n k=1 t 1-c 2 k γ e k + √ 2γε k t -c 2 k √ v k + t 2-c k ε k = 2γC 2 E 1 + 2C -1 1 B n + 2C -1 1 n k=1 t 1-c 2 k γ e k + √ 2γε k t -c 2 k √ v k (3.94)
Chapter 3. Inertial proximal-gradient algorithms 98 Thus, by applying Lemma B.0.4 with :

u n = t -c 2 n √ v n , a n = 2C -1 1 t 1-c 2 n γ e n + √ 2γε n and S n = 2C 2 E 1 +2C -1 1 B n (3.95)
and the definition of A n , we find that for all n ≥ 1, it holds :

√ v n ≤ 2C -1 1 A n + 2 γC 2 E 1 + C -1 1 B n t c 2 n (3.96)
By injecting the last estimation (3.96), into (3.93) and multiplying by 2γ, we find :

2γE n ≤ C 2 2C -1 1 A n 2C -1 1 A n + 2 γC 2 E 1 + C -1 1 B n + 2 γC 2 E 1 + B n t c n (3.
97) It follows that for some -renamed-suitable positive constants C 1 and C 2 , for all n ≥ 1, it holds :

2γE n ≤ C 1 A n + C 2 + B n

Concluding remarks and perspectives

In this Section we gave a unified presentation for the convergence properties of the inertial Forward-Backward Algorithm 7, for solving convex composite minimization problems. Our analysis followed the one in the continuous case in Chapters 1 and 2, for the corresponding second-order damped system. In particular we adapted the Lyapunov techniques for dynamical systems, into the discrete setting. As it is apparent from Theorem 3.3.1, the over-relaxation parameter b > 0, plays a crucial role for the convergence properties of the algorithm. It rules the order of convergence rates for both the objective function and the local variation, as also the (weak) convergence of the sequence to a minimizer. Notice also that from our analysis the tuning of the over-relaxation parameter b > 0, is "justified" via Lyapunov arguments for the energy sequence E n , in order to obtain the -worst case-optimal rate 3.5. Concluding remarks and perspectives O(n -2 ). For some other approaches, explaining the acceleration effect of inertial algorithms such as 7, linked with numerical integration of ODEs, one can also consult the work of [START_REF] Scieur | Aspremont. Integration methods and accelerated optimization algorithms[END_REF].

Like in the continuous -in time-setting, a puzzling question concerns the phase transition from high-to-low momentum b = 3. Apart from the acceleration effect from the high momentum region b < 3 to the low b ≥ 3 (see for example Figure 3.1), the weak convergence of the sequence to a minimizer still remains an open problem for the high-momenum region b ≤ 3. A partial answer to this question, is given in the very recent work of Attouch et al [START_REF] Attouch | Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3[END_REF], in the case of H = R and b = 3, but the proof uses topological arguments, which can not be directly extended to higher dimensions.

In a more general way, it would be interesting to consider inertial algorithms with a general extrapolation term α n and investigate their convergence properties with respect to the structure of the minimizing function F (this question is also partially treated in the recent work [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF]). As in the case of the i-FB Algorithm 7, this question can be of course treated in a parallel way with the study of the continuous dynamical system such as (1.18) (or (2.4) in the non-differential setting). In any case, the tuning rule of the over-relaxation sequence α n for inertial schemes, still remains a puzzling question and an active research domain. In particular, as we shall see in the next Chapter, the tuning of parameter b > 0, depends also on the possible additional geometric properties of the minimizing function F .

Chapter 4

Inertial gradient-descent algorithm under sharpness-flatness conditions

In this Chapter we study the minimization problem (M) introduced in the previous Chapter 3 under some additional assumptions on the local geometry of the minimizing function F . Of course this restrains the variety of the problems that one can handle with, but on the other hand one can expect finer results (such as strong convergence results, faster convergence rates etc). In particular we study the convergence properties of the i-FB Algorithm 7 introduced in Chapter 3 in a differential setting (i.e. when g = 0) which takes the following form (see the i-GD Algorithm 11):

y n = x n + n n + b (x n -x n-1 ) x n+1 = T γ (y n ) := y n -γ∇F (y n ) (4.1)
for a step size 0 < γ ≤ 1 L , an over-relaxation parameter b > 0 and a convex differentiable function F with L-Lipschitz gradient.

Firstly we give some preliminary notions concerning some classical geometrical assumptions on the minimizing function F , such as the strong convexity property, or more generally, growth condition (or Kurdyka-Łojasiewicz condition). We mention some of the results of the existing literature and on how these geometrical conditions can improve the convergence properties of descent schemes (i.e. schemes that produce non-increasing sequence of the objective function F (x n ) -F * ).

In our case we show that for the inertial Gradient-Descent algorithm (4.1), instead of considering only sharpness-type conditions (such as the growth Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 102 condition), we additionally exploit a supplementary condition, expressing the flatness-level of the minimizing function near the set of minimizers. In this way we are able to derive competitive rates of convergence for the i-GD algorithm (4.1), depending on both of sharpness-flatness level of the function and the inertial parameter b > 0. In particular for functions We then compare our results with the ones of the existing literature.

In particular we are treating differentiable functions F which -roughly speaking1 -satisfy near the set of their minimizers X * , a relation of the following form:

C β dist(x, X * ) β ≤ F (x) -F * ≤ C p dist(x, X * ) p , ( 4.2)
where C 1 , C 2 are some positive constants and the parameter β ≥ 1 expresses the -worst case-level of flatness of F while p ≥ 2 the -worst case-level of sharpness of F . In this framework we extend the results found for the i-FB algorithm 7 in Chapter 3 and we obtain new precise rates depending on the inertial parameter of scheme (4.1) b > 0 and the geometric parameters β and p.

More precisely the results found in Theorem 3.3.1 in the previous Chapter, hold true for a wider variety of the parameter b > 0 in the differential setting. If for example b ≤ 1 + 2 β , we show that (see Theorem 4.5.1 later on):

F (x n ) -F * + x n -x n-1 2 = O n -2βb β+2 (4.3) while if b > 1 + 2 β , then : F (x n ) -F * + x n -x n-1 2 = o n -2 (4.4)
and the sequence converges to a minimizer. If in addition the function F admits a unique minimizer and satisfies (4.2) with p = 2, then (4.2) holds true for all b > 0.

These results apply to a variety of interesting minimization problems, such as the Least-square problem or more generally minimizing quadratic functions. In these cases β = p = 2 and the order of convergence rate is

F (x n ) -F * = O n -b ,
for all b > 0. In addition, in the particular case of β = 1, the rates in (4.3) can be also extended for the i-FB Algorithm 7 in the non-differential setting. For example for the LASSO problem under uniqueness of the minimizer, if we apply the inertial scheme 4.1, by (4.3), we find that the -worst case-convergence rate for the minimizing function

4.1. Sharp-geometry condition is O n -2b 3
, for all b > 0. In such cases we obtain arbitrarily fast -but sublinear-order of convergence rate depending on the parameter b > 0, which is (theoretically) worse than the linear order of convergence that one can obtain with the Gradient Descent or other versions of inertial schemes with fixed over-relaxation sequence, such as the Heavy-Ball algorithm (see for example Algorithms 9 and 10 later on).

On the other hand, for "flatter" functions, with a unique minimizer, that satisfy (4.2) with p ≥ β ≥ 2, we show that the convergence rate for the objective function generated by (4.1) is (see Theorem 4.5.2):

F (x n ) -F * = O n -2p p-2 (4.5)
The rate (4.5) is new and consist of an improvement, in comparison to the one obtained by descent schemes, such as the Gradient Descent (see Theorem 4.2.1 later on). This fact indicates that employing inertial schemes such as (4.1) for minimizing functions with flatter-than quadratic-geometries, is a better strategy than the one of using descent schemes.

Sharp-geometry condition

We begin by defining properly the p-growth condition that we use in this Chapter.

Definition 4.1. Let p ≥ 1. The function F satisfies the condition L(p) if, for any minimizer x * ∈ X * , there exists a constant K p > 0 and ε > 0 such that:

∀x ∈ B(x * , ε), K p x -x * p ≤ F (x) -F (x * ).
Hypothesis L(p) with p ≥ 1, is a growth condition on the function F around its set of minimizers X * . Note that, when X * is a connected compact set, it can be replaced by a more general growth condition on F in the neighborhood of its minimizers : Lemma 4.1.1. Let F : R n → R be a convex differentiable function satisfying the growth condition L(p) for some p 1. Assume that the set X * = arg min F is compact. Then there exists K > 0 and ε > 0 such that, for all x ∈ R n :

d(x, X * ) ε ⇒ Kd(x, X * ) p F (x) -F * .
Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 104

Historically, the growth condition L(p) is also called p-conditioning (see [START_REF] Garrigos | Convergence of the forward-backward algorithm: Beyond the worst case with the help of geometry[END_REF] and [START_REF] Penot | Conditioning convex and nonconvex problems[END_REF]) or Hölderian error bound [START_REF] Bolte | From error bounds to the complexity of firstorder descent methods for convex functions[END_REF], and is closely related to the Łojasiewicz inequality [Łoj63, Łoj93], (or Kurdyka-Łojasiewicz inequality [START_REF] Kurdyka | On gradients of functions definable in ominimal structures[END_REF], [START_REF] Bolte | Characterizations of Łojasiewicz inequalities: subgradient flows, talweg, convexity[END_REF]) which turns out to be a key tool in the mathematical analysis of continuous and discrete dynamical systems ([ABRS10b], [START_REF] Attouch | Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods[END_REF], [START_REF] Jérôme Bolte | The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF], [START_REF] Bolte | Characterizations of Łojasiewicz inequalities: subgradient flows, talweg, convexity[END_REF]): Definition 4.2. A differentiable function F : R d → R is said to have the Łojasiewicz property with exponent θ ∈ [0, 1) if, for any critical point x * , there exist c > 0 and ε > 0 such that:

∀x ∈ B(x * , ε), ∇F (x) c|F (x) -F * | θ . (4.6)
where: 0 0 = 0 when θ = 0 by convention. More generally ([BDLM10]), F satisfies the Kurdyka-Łojasiewicz inequality (KL) on x * ∈ domF , if-f there exist some constants c > 0, r > 0 and a concave function φ ∈ C 1 ((0, r)) with φ(0) = 0, and φ > 0, such that :

1 ≤ cφ F (x) -F * ∇F (x) (4.7)
Note that in this case, this corresponds to controlling the sharpness of the function F near the set of minimizers, under the reparametrization φ. In fact the Łojasiewicz property (4.6), consists in considering the particular case φ(t) = |t| 1-θ .

In fact, in the convex setting, since the unit ball B(x * ) is ∂F invariant (see [START_REF] Brezis | Opeérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF]), the growth condition L(p), p 1, is actually equivalent to the Łojasiewicz inequality, with exponent θ = 1 -1 p ∈ (0, 1] and c = K 1 p p (see for example [BDL06, BDLM10, BNPS17], [START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the kurdyka-łojasiewicz inequality[END_REF], [START_REF] Garrigos | Convergence of the forward-backward algorithm: Beyond the worst case with the help of geometry[END_REF] or [START_REF] Kim | Adaptive restart of the optimized gradient method for convex optimization[END_REF] and [START_REF] Zhang | New analysis of linear convergence of gradient-type methods via unifying error bound conditions[END_REF] for other equivalent notions).

Typical examples of functions having the Łojasiewicz property are realanalytic functions and C 1 subanalytic functions [Łoj63], or semialgebraic functions (see [START_REF] Bolte | The łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF], [START_REF] Attouch | On the convergence of the proximal algorithm for nonsmooth functions involving analytic features[END_REF] and [START_REF] Bolte | Characterizations of Łojasiewicz inequalities: subgradient flows, talweg, convexity[END_REF]). Strongly convex functions satisfy a global Łojasiewicz property (i.e. (4.6) holds everywhere in R d ) with exponent θ = 1 2 [AB09], or equivalently a global version of the growth condition, namely:

∀x ∈ R n , F (x) -F * µ 2 d(x, X * ) 2 ,
where µ > 0 denotes the parameter of strong convexity and in this case we have K 2 = 2µ. Likewise, convex functions having a strong minimizer in the sense of [AC18d, Section 3.3], also satisfy a global version of L(2). By 4.2. Descent schemes extension, uniformly convex functions of order p 2 also satisfy the global version of the hypothesis L(p) (see for example [ABRS10a, BNPS17, GRV17] and their possible references).

As already mentioned, the geometrical interpretation of the condition L(p) is straightforward: it ensures that the function F is sufficiently sharp (at least as sharp as x → x-x * p ) in a neighborhood of its set of minimizers X * . Consistently, observe that any convex function satisfying L(p), satisfies L(p ) for all p p.

Descent schemes

First results of exploiting the strong convexity of the minimizing function F , or more generally growth condition L(2), were established in [START_REF] Goldstein | Cauchy's method of minimization[END_REF] (in the case of strongly convex functions) and by Polyak in [START_REF] Boris | Gradient methods for the minimisation of functionals[END_REF] and [START_REF] Boris | Some methods of speeding up the convergence of iteration methods[END_REF] (in the case of functions satisfying L(2)) , for the Gradient-Descent Algorithm 2 and the Heavy-Ball algorithm 9.

In particular, for the fixed-step Gradient-Descent Algorithm 2 applied to the minimization problem of a convex differentiable function F , with L-Lipschitz gradient and which satisfies globally L(2), the trajectory generated, converges linearly:

F (x n ) -F * ≤ F (x 0 ) -F * q n (4.8)
with geometric ratio q = max{|1 -γµ|, |1 -γL|}. It is then clear that the optimal value for the step size γ, is γ = 2 L+µ , which gives best linear ratio q = L-µ L+µ .

Further studies generalizing this result for convex (not necessarily differentiable) functions, as also convex functions satisfying condition L(p) for a general p ≥ 1 (or equivalently the p-Łojasiewicz property) were made in [START_REF] Tyrrell | Monotone operators and the proximal point algorithm[END_REF] [ABS13], [START_REF] Jérôme Bolte | The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF] [BL08] for the proximal point algorithm 3, or the Forward-Backward Algorithm 1 in [START_REF] Frankel | Splitting methods with variable metric for Kurdyka-łojasiewicz Bibliography functions and general convergence rates[END_REF], [START_REF] Garrigos | Descent dynamical systems and algorithms for tame optimization, and multi-objective problems[END_REF], [START_REF] Bolte | From error bounds to the complexity of firstorder descent methods for convex functions[END_REF], [START_REF] Garrigos | Convergence of the forward-backward algorithm: Beyond the worst case with the help of geometry[END_REF].

In fact, the common key-property of these schemes (such as Gradient-Descent, Proximal-point or more generally Forward-Backward algorithm), is the descent property of the generated trajectory. Roughly speaking the descent property expresses that the iterative scheme produces a non-increasing sequence of objective functions F (x n ) -F * . This property is essential, in order to derive convergence rates, by exploiting the growth condition L(p) (or equivalently the p-Łojasiewicz property).

Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 106 Formally (see for example [START_REF] Frankel | Splitting methods with variable metric for Kurdyka-łojasiewicz Bibliography functions and general convergence rates[END_REF]or [START_REF] Garrigos | Convergence of the forward-backward algorithm: Beyond the worst case with the help of geometry[END_REF]), we say that {x n } n∈N is a sequence generated by a First-order descent method for minimizing convex function F , if-f there exist some parameters α, β > 0 such that for all n ∈ N, it holds :

F (x n+1 ) -F (x n ) ≤ -α x n+1 -x n 2 ∃z n+1 ∈ ∂F (x n+1 ) s.t. z n+1 ≤ β x n+1 -x n (4.9)
It is easy to check for example from Lemma B.0.7, that the sequence generated by Forward-Backward scheme 1 verifies (4.9), for some suitable values of α > 0 and β > 0.

More generally, we have the following Theorem :

Theorem 4.2.1. [START_REF] Bolte | From error bounds to the complexity of firstorder descent methods for convex functions[END_REF], [START_REF] Frankel | Splitting methods with variable metric for Kurdyka-łojasiewicz Bibliography functions and general convergence rates[END_REF], [START_REF] Garrigos | Convergence of the forward-backward algorithm: Beyond the worst case with the help of geometry[END_REF] Let F be a convex function such that x * ∈ arg min F = ∅ , which also satisfies the growth condition L(p) (or equivalently the p-Łojasiewicz property) with p ∈ [1, +∞). Let also {x n } n∈N be a sequence generated by a first-order descent method (i.e. satisfies (4.9) for some α > 0 and β > 0) and w n = F (x n ) -F * . Then there exist some constants C 1 > 0, C 2 > 0 (in each case) and q = α 2β 2 K 2 such that :

+∞ n=0 x n+1 -x n < +∞ (4.10)
In particular : x n -→ n→∞ x * . In addition :

1. If p = 1, then : x n = x * , for all n ≥ w 0 q ( finite termination ) 2. If p ∈ (1, 2), then for all n ≥ 1 :

w n+1 ≤ (1 + q) -1 w n p 2(p-1) and x n+1 -x * ≤ C 1 √ w n (4.11)
3. If p = 2, then for all n ≥ 1 :

w n+1 ≤ (1 + q) -1 w n and x n+1 -x * ≤ C 1 √ w 0 (1 + q) n 2 (4.12)
4. If p > 2, then for all n ≥ 1 :

w n ≤ C 1 n -p p-2 and x n+1 -x * ≤ C 2 n -1 p-2 (4.13)
Remark 11. [START_REF] Bolte | From error bounds to the complexity of firstorder descent methods for convex functions[END_REF] In general, if the function F satisfies the p-Łojasiewicz property, then the order of convergence rate of x n -x * is ruled by the one of the objective function F (x n ) -F * . In particular for all n ∈ N, we have that

x n -x * ≤ C F (x n ) -F * 1 M 2,p (4.14)
for a positive constant C > 0 and M 2,p = max{2, p}

Inertial versions

It is clear by Theorem 4.2.1, that for descent methods satisfying (4.9), the more the function is sharp in the neighborhood of the minimizers (i.e. F is satisfying L(p) with p ∈ [1, 2]), the faster the convergence rates to the minimum F * are. Starting from linear convergence ( see point 3. in Theorem 4.2.1 when p = 2), passing to superlinear (see point 2. in Theorem 4.2.1 when 1 < p < 2) or even finite termination (see point 1. in Theorem 4.2.1 when p = 1).

On the contrary, for functions that are "not necessarily sufficiently sharp" (i.e. when the function F is satisfying L(p) with p > 2), the worst-case convergence rates for the objective function are sublinear of order p p-2 . This means that, the more the function is flat, the slower the rates are, with the limit case of o(n -1 ) when p → +∞ (see [START_REF] Davis | Convergence rate analysis of several splitting schemes[END_REF]).

Inertial versions

In the last section, we pointed out the importance of growth-type conditions (or Kurdyka-Łojasiewicz) for first order descent schemes (satisfying (4.9)). Nevertheless, as pointed out in Chapter 3, inertial algorithms -in generally-are not descent schemes. Therefore the analysis presented previously in Theorem 4.2.1 is not valid for these types of algorithms.

Except of some particular cases (such as the Heavy-Ball algorithm or i-FB Algorithm 10), in general, few results are known concerning the convergence of inertial schemes under additional local geometrical assumptions on the minimizing function F such as the L(p) condition for a general p ≥ 1. Most of them necessitate rather strong hypotheses such as strong convexity or strong convexity along the proximal projection (see for example [START_REF] Necoara | Linear convergence of first order methods for non-strongly convex optimization[END_REF]).

As mentioned earlier, first results concerning inertial algorithms for minimizing convex functions under additional hypotheses on the local geometry of the function near the set of minimizers, date back to the work of Polyak [START_REF] Boris | Some methods of speeding up the convergence of iteration methods[END_REF] and the Heavy-Ball algorithm: Algorithm 9 HB Let x 0 = y 0 , α > 0 and 0 < γ ≤ 2 L . For all n ∈ N consider :

y n = x n + α(x n -x n-1 ) (4.15) x n+1 = y n -γ∇f (x n ) (4.16)
Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 108

In particular for the Heavy Ball-Algorithm 9 the authors shown that for a µ-strongly convex and twice-differentiable function F (with L-bounded Hessian), and an iteration step γ =

4 ( √ L+ √ µ) 2 ≥ 1 L and α = √ L- √ µ √ L+ √ µ 2
, the convergence is linear with :

F (x n ) -F * = O √ L - √ µ √ L + √ µ n (4.17)
This consists of an improved linear ratio than the one found for Gradient Descent which is L-µ L+µ (when µ L is small). In addition as remarked by Nesterov [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF], the linear convergence rate with the ratio

√ L- √ µ √ L+
√ µ is optimal for the class of strongly convex and smooth functions. In fact the linear convergence (4.17) holds also true if the strong convexity hypothesis is relaxed to the L(2) property on F . However, a counter-example due to Lessard et al in [START_REF] Lessard | Analysis and design of optimization algorithms via integral quadratic constraints[END_REF], shows that the Heavy-Ball algorithm does not necessarily converges for a general parameter α > 0 and a general strongly convex function (which is not twice differentiable).

Another common choice of algorithm that is often taken into consideration in the case of smooth and strongly convex functions, which is close to the Heavy-Ball Algorithm 9 is the following: Algorithm 10 NSC Let x 0 = y 0 . For all n ≥ 1 consider :

y n = x n + α * (x n -x n-1 ) where : α * = √ L - √ µ √ L + √ µ x n+1 = y n - 1 L ∇F (y n ) (4.18)
In particular, for Algorithm 10 it was shown in [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF] that the convergence rate of the objective function F (x n ) -F * is also linear with :

F (x n ) -F * = O 1 - µ L n (4.19)
where µ is the strong convexity parameter and L the Lipschitz bound of the the gradient of F . Notice that even if the convergence is linear, the geometric ration is slightly worse than the one obtained for the Heavy-Ball algorithm since and 10, the hypotheses of twice differentiability as also the strong convexity property are often too restrictive.

In the recent work [START_REF] Iutzeler | On the proximal gradient algorithm with alternated inertia[END_REF] the authors propose a different application of alternated inertia to (7) (i.e. applying the inertial term α n (x n -x n-1 ) every two iterations), in a non smooth setting. Surprisingly this turns Algorithm (7) with alternated inertia, into a descent scheme and it permits to have the same convergence properties as the the ones in Theorem 4.2.1 , under the hypothesis L(p) with p 1.

Here we shall remark that even if the inertial scheme 10 or Heavy-Ball algorithm 9, give linear rates, often it is not easy to implement in practice. This is due to the fact that in most of the practical cases, the knowledge of parameters L, and mostly µ or ( K 2 in the case of L(2) property) is not necessarily known and estimating them can be a hard and costful task. Some possible approaches for estimating L is done by backtracking (see [START_REF] Nesterov | Gradient methods for minimizing composite functions[END_REF], [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF], [START_REF] Tseng | On accelerated proximal gradient methods for convex-concave optimization[END_REF] or [START_REF] Calatroni | Backtracking strategies for accelerated descent methods with smooth composite objectives[END_REF]). As for the parameter µ (or K 2 ) this question is getting more challenging (see also [START_REF] Nesterov | Gradient methods for minimizing composite functions[END_REF] for a backtracking strategy of estimating µ).

Several tentatives for estimating the strong convexity parameter µ or more generally the constant K 2 when the functions satisfies L(2), and proving linear convergence rates for FISTA Algorithm 6, have also been made by restarting techniques (see for example [START_REF] Fercoq | Restarting accelerated gradient methods with a rough strong convexity estimate[END_REF], [START_REF] Fercoq | Adaptive restart of accelerated gradient methods under local quadratic growth condition[END_REF]).

The restarting effect is a fundamental issue raising in inertial algorithms. The term refers to starting the algorithm afresh, from the last iterate (i.e. taking the "last" iteration as the new starting point). This procedure consists in resetting the inertial parameter α n equal to zero and start the algorithm again.

Restarting techniques of i-GD algorithm 11 for strongly convex functions go back to [START_REF] Nesterov | Gradient methods for minimizing composite functions[END_REF] (see also [START_REF] Gu | Parnes: A rapidly convergent algorithm for accurate recovery of sparse and approximately sparse signals[END_REF]), where a fixed-restart scheme is considered in order to obtain linear convergence rates for functions that satisfy the L(2) condition globally. Nevertheless a basic drawback of the optimal fixed-period of restarting is that it depends on parameters L and K 2 , as also it does not take into consideration better-conditioned regions ( [START_REF] Brendan | Adaptive restart for accelerated gradient schemes[END_REF]). This led to the research of adaptive methods of restarting, where the restart-rule does not depend on fixed parameters (such as the parameters L, µ or K p )

A basic palette of the existing literature on adaptive restarting scheme is given below (see [START_REF] Brendan | Adaptive restart for accelerated gradient schemes[END_REF], [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF], [START_REF] Arnar Tómasson | Afsi: Adaptive restart for fast semi-iterative schemes for convex optimisation[END_REF] and [START_REF] Kim | Adaptive restart of the optimized gradient method for convex optimization[END_REF])

1. Function scheme ([OC15]) Restart algorithm 4.1, whenever F (x n+1 ) > F (x n ) (4.20)
Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 110

2. Gradient scheme ([OC15]) Restart algorithm 4.1, whenever

∇F (y n ) • (x n+1 -x n ) > 0 (4.21)
or (see [START_REF] Arnar Tómasson | Afsi: Adaptive restart for fast semi-iterative schemes for convex optimisation[END_REF]) :

∇F (x n ) • (x n+1 -x n ) > 0 (4.22)
3. Speed-Restart scheme : [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF] Restart algorithm 4.1 whenever:

x n+1 -x n < x n -x n-1 (4.23)
Even if adaptive restarting schemes seem to behave well in practice, the theoretical guarantees of their convergence are either not proven or proven for limited cases (such as when the minimizing function F is quadratic [START_REF] Brendan | Adaptive restart for accelerated gradient schemes[END_REF][START_REF] Kim | Adaptive restart of the optimized gradient method for convex optimization[END_REF] or when F is strongly convex [START_REF] Arnar Tómasson | Afsi: Adaptive restart for fast semi-iterative schemes for convex optimisation[END_REF]) and their study is still an active area of research. In addition most of the studies are restraint to the quadratic error bound (i.e. the L(2) condition) with a few recent exceptions (see [START_REF] Liu | Adaptive accelerated gradient converging method under h\"{o} lderian error bound condition[END_REF] and [START_REF] Roulet | Aspremont. Sharpness, restart and acceleration[END_REF]) for functions satisfying L(p) for a general p ≥ 1.

From all the above discussion, it seems that -apart some particular cases -the study of convergence rates for inertial algorithms for functions that satisfy additional geometrical properties such as L(p), for a general p ≥ 1, is largely unknown and hence a challenging question. In this point, we shall mention the recent work [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF] (see also [START_REF] Su | A differential equation for modeling Nesterov's accelerated gradient method: theory and insights[END_REF]), where the authors derive some convergence rates for the i-FB Algorithm 7 in the case when F admits a strong minimum (i.e. satisfies the growth condition L(2) with a unique minimizer x * ). In that case the convergence rate of the i-FB Algorithm 7 for the objective function is

F (x n ) -F * = O n -2b 3
for any b > 0, which is sublinear but arbitrarily fast (depending on the value of b).

i-GD algorithm and geometry

In this section we shall see that by exploiting a bit more the geometry of the minimizing function F , we are able to deduce some competitive convergence properties for the smooth version of the i-FB Algorithm 7 introduced in Chapter 3. In particular instead of exploiting only sharpness condition, such as L(p), p ≥ 2, we introduce a new condition As mentioned in the beginning, in contrary to the previous Chapter, in the current one, we consider the minimization problem of a function in a 4.4. i-GD algorithm and geometry smooth and convex setting. Hence, in this section we formally consider the minimization problem (M) with g = 0 and F = f a convex, smooth function with L-Lipschitz gradient, and such that X * = arg min F = {∅}.

In that case, we call i-GD (inertial Gradient-Descent) algorithm, the i-FB Algorithm 7, which takes the following form:

Algorithm 11 i-GD Let 0 < γ < 1 L and x 0 = x 1 ∈ R d . Repeat : y n = x n + α n (x n -x n-1
) where :

α n = n n + b (4.24) x n+1 = T γ (y n ) = y n -γ∇F (y n ) (4.25)
Before giving the main results of this Chapter, we settle up the new flatness-type condition on F , that plays a key-role to our analysis.

Flat geometry of convex functions around their minimizers

In this paragraph we present a new condition for a convex and differentiable function F , describing (locally) its geometry around the set of its minimizers X * , as also the interplay between this new condition and the growth condition L(p), defined earlier in this Chapter. Roughly speaking, this condition characterizes functions behaving more "gently" than the function • β , with β ≥ 1, around its set of minimizers. While condition L(p), introduced before, indicates how sharp the minimizing function is near the set of its minimizers, this new condition assures that the function is sufficiently flat in the neighborhood of its minimizers. Formally we have the following definition: Definition 4.1. Let F : R d → R be a convex differentiable function with X * = arg min F = ∅. Let β 1. The function F satisfies the condition H(β) if, for any critical point x * ∈ X * , there exists η > 0 such that:

∀x ∈ B(x * , η), F (x) -F (x * ) 1 β ∇F (x), x -x * .
In fact the hypothesis H(β) was already introduced in [CEG09a, SBC16, AD17, ADR18] and it generalizes the notion of convexity of a differentiable function in a neighborhood of its minimizers.

Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 112

Firstly, observe that any convex and differentiable function automatically satisfies H(1). In addition any differentiable function F ensuring that (F -F * ) 1 β is convex, satisfies H(β) with β 1, which is slightly more demanding than the convexity of F .

In order to have a better insight on the local geometry of convex functions satisfying the hypothesis H(β), we need some additional results, which are formulated on the forthcoming Lemmas :

Lemma 4.4.1 ([ADR18]). Let F : R d → R be a convex differentiable func- tion with X * = arg min F = ∅. If F satisfies H(β) for some β 1, then: 1. The function F satisfies H(β ), for all 1 β β.
2. For any minimizer x * ∈ X * , there exist M > 0 and η > 0 such that:

∀x ∈ B(x * , η), F (x) -F (x * ) ≤ M x -x * β . (4.26)
In other words, as already mentioned before, the hypothesis H(β) with β 1, can be interpreted as a flatness condition: it ensures that the function F is sufficiently flat (at least as flat as x → x -x * β ) in the neighborhood of its minimizers.

The proof of Point 1 of Lemma 4.4.1 is direct since for all 1 β β x ∈ R d and x * ∈ arg min F , we have :

0 ≤ β (F (x) -F (x * )) ≤ β(F (x) -F (x * )) ≤ ∇F (x), x -x * (4.27)
For the point 2, we make use of the following Lemmas.

Lemma 4.4.2. Let g : R → R be a convex differentiable function such that: 0 ∈ arg min g and g(0) = 0. Assume that g satisfies H(β) for some β ≥ 1.

Then, there exists some constant M , such that :

∀t ∈ [0, 1], g(t) ≤ M t β .
Proof. Let t ∈ (0, 1] If we multiply H(β) by t|t| -β-2 , we find :

|t| -β g (t) -βt|t| -β-
F (x) -F * ≤ M d(x, X * ) β .
Proof. Let x * ∈ arg min F . For all x ∈ B(x * , 1), such that x = x * and t ∈ [0, 1], we define the function g x,x * (t) = F (x * + t x-x * x-x * ) -F * . By definition of g x,x * , we have that 0 ∈ arg min g x,x * and g x,x * (0) = 0. In addition g satisfies H(β) i.e. (βg x,x * (t) ≤ tg x,x * (t)). By Lemma 4.4.2, we deduce the existence of a constant M x,x * , such that :

g x * (t) ≤ M x,x * t β (4.31) Since M x,x * = F (x * + x-x *
x-x * ) -F * and F is bounded on B(x * , 1), we deduce the existence of M (independent of x and x * ), such that for all x ∈ B(x * , 1) with x = x * and t ∈ [0, 1], we have:

g x,x * (t) ≤ M t β (4.32)
Finally by choosing t = x -x * and taking the inf over X * , we conclude the proof of the Lemma.

The interplay between sharpness-flatness conditions

Before passing to the convergence results, we give some additional insight for the different interactions between the flatness condition H(β), β ≥ 1 and the sharpness condition L(p), p ≥ 2.

Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 114

For that, we consider any convex differentiable function F satisfying both hypothesis H(β) and L(p). Combining the related inequalities, namely (4.26) and the growth condition L(p), F has to be at least as flat as x -x * β and at most as sharp as x -x * p in a suitable neighborhood of the set of its minimizers.

For

Main results for the i-GD Algorithm

In our framework, the objective function F is assumed to be convex and differentiable with a Lipschitz continuous gradient. For such functions, the Lipschitz continuity of the gradient provides some additional information on the local geometry of F in the neighborhood of its minimizers. Indeed, for convex functions, the Lipschitz continuity of the gradient is equivalent to a quadratic upper bound on F :

∀(x, y) ∈ R d × R d , F (x) -F (y) ∇F (y), x -y + L 2 x -y 2 . (4.33)
Applying (4.33) at y = x * , we then deduce:

∀x ∈ R d , F (x) -F * L 2 x -x * 2 , (4.34)
which indicates that F is at least as flat as x -x * 2 around X * . More precisely:

Lemma 4.4.5. Let F : R d → R be a convex differentiable function with a L-Lipschitz continuous gradient for some L > 0.

1. If F satisfies the growth condition L(p), then necessarily p 2.

If

F satisfies L(2), then F automatically satisfies H(β) with β = 1+ K 2 2L and K 2 L 2 .
Proof. Assume that F satisfies the condition L(p). Combining the inequality (4.34) and the growth condition, we get: for any x * ∈ X * ,

K p x -x * p F (x) -F (x * ) L 2 x -x * 2 , (4.35)
for all x in some neighborhood of x * , which necessarily implies: p 2. In the particular case p = 2, we also deduce that: 2K 2 L. The second point of Lemma 4.4.5 has already been shown in [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF].

Main results for the i-GD Algorithm

In this paragraph we give the main results of this Chapter, concerning the convergence rates of the i-GD algorithm 11, depending on the geometric assumptions H(β), L(p) with p ≥ β and the inertial parameter b > 0. We also make several comments and comparisons of these results with the existing literature.

Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 116

Theorem 4.5.1. Let F : R d → R be a convex differentiable function with a L-Lipschitz continuous gradient for some L > 0. Let 0 < γ ≤ 1 L and {x n } n∈N be the sequence generated by i-GD Algorithm 11. Assume that F satisfies H(β) with β ≥ 1. Then we have the following:

1. If b < 1 + 2
β , the following convergence rates hold true asymptotically:

F (x n ) -F (x * ) = O n -2bβ β+2 and x n -x n-1 = O n -bβ β+2 (4.36) 2. (i) If b ≥ 1 + 2
β then the following convergence rate holds true asymptotically:

F (x n ) -F (x * ) = O n -2 (4.37)
If in addition b ≥ 2 then {x n } n≥1 is bounded and:

x n -x n-1 = O n -1 (4.38) (ii) If b > 1 + 2 β then: +∞ n=0 n F (x n ) -F (x * ) < +∞ and +∞ n=0 n x n -x n-1 2 < +∞ (4.39) (iii) If b ≥ 1+ 2 β and if F satisfies L(2)
and admits a unique minimizer, then the following convergence rate holds true asymptotically:

F (x n ) -F (x * ) = O n -2bβ β+2 and x n -x n-1 = O n -bβ β+2 (4.40)
In particular in the same way as in [START_REF] Attouch | The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/kˆ2[END_REF] (see Theorem 1), the results of Theorem 4.5.1, are slightly improved asymptotically. β , for n ∈ N, we actually have asymptotically :

F (x n ) -F (x * ) = o n -2 and x n -x n-1 = o n -1 (4.41)
as also the sequence x n converges to a minimizer x * .

Several remarks can be made concerning Theorem 4.5.1, which regroups some known results and extends some of them to functions satisfying hypothesis H(β) with β > 1.

Main results for the i-GD Algorithm

Remark 14. First of all in the case β = 1 (i.e. when the function F is convex), from points 1, 2(i) and 2(ii), we find the well-known estimates (3.28) of Theorem 3.3.1 in Chapter 3. In fact, points 1, 2(i) and 2(ii) improve the results of Theorem 3.3.1 (in Chapter 3), for a smooth function that satisfies H(β) with β > 1. More precisely the classical rates of O(n -2 ) or o(n -2 ) and the convergence to a minimizer, can still hold true, even if b < 3 (in particular if 1 + 2 β < b ≤ 3), given that the function F is "sufficiently" flat (i.e. β large enough). Remark 15. In the case where F satisfies both H(β) and L(2), the point 2.(iii), only makes sense, when 1 ≤ β ≤ 2. In addition, we have -at least-that β = 1 + K 2 2L (see Lemma 4.4.5). Hence from Point 2(iii) of Theorem 4.5.1) we find that

F (x n ) -F (x * ) = O n -2b 3 - 4bK 2 3(2L+K 2 )
, which is an improved result in comparison to the O n -2b/3 found in [START_REF] Attouch | Convergence rates of inertial forward-backward algorithms[END_REF] concerning functions with L-Lipschitz gradient. Furthermore, from (4.40), we can see that the rate is even better if F satisfies H(2) and L(2). In this case we have ) is sub-linear but it can be arbitrarily "fast" without any additional cost (by choosing b large enough). Nevertheless, this result is sub-optimal in comparison to the linear/geometric convergence rate which is proven for the Gradient Descent algorithm 2 (see for example (4.12) of Theorem 4.2.1), or the inertial gradient algorithm with some fixed momentum parameter 10 or the Heavy-Ball algorithm 9 (see (4.19) and (4.17) respectively). This indicates that in case of functions with sharpenough geometries, (strongly convex or satisfying at least L(2) condition), inertial Gradient Descent Algorithm 11 may not be the optimal choice (at least theoretically). In this remark we shall also add that the dependency of the hidden constant in the "big Ohs" of Theorem 4.5.1 on the parameter b > 0 can play a major role on this comparison. Remark 17. A final remark is about the necessity of the uniqueness of the minimizer in the point 2.iii of Theorem 4.5.1, which comes mostly by an algebraic argument in the proof, rather than a "counter-example intuition". Thus we conjecture that this hypothesis may not be necessary in general, but we could not omit it in our proof.

F (x n ) -F (x * ) = O n -b . Remark
In The non-differential setting In fact, in the particular case when F only satisfies H(1) (i.e. F is convex), in view of the validity of the descent Lemma B.0.7 for general convex functions (not necessarily smooth), the analysis of Theorem 4.5.1 is still applicable for the i-FB algorithm 7, i.e. :

y n = x n + n n + b (x n -x n-1 ) x n+1 = T γ (y n ) := Prox γg y n -γ∇f (y n ) (4.42)
where f and g are convex with f being smooth with L-Lipschitz gradient and γ ≤ 1 L . In this case from Theorem 4.5.1 we can recover some of the results already found in Theorem 3.3.1, concerning the i-FB Algorithm (7) studied in Chapter 3 in a more refined version (see in particular point 2.(iii) in Corollary 4.5.2). More precisely we have the following Corollary.

Main results for the i-GD Algorithm

Corollary 4.5.2. Let F = f + g , where f, g are convex, lower semicontinuous functions, with f differentiable with L-Lipschitz gradient, such that arg min F = ∅. Let also 0 < γ ≤ 1 L and {x n } n∈N be the sequence generated by Algorithm (7). Then:

1. If b < 3, the following convergence rates hold true asymptotically:

F (x n ) -F (x * ) = O n -2b 3 and x n -x n-1 = O n -b 3 (4.43)
2. (i) If b 3 then the following convergence rate holds true asymptotically:

F (x n )-F (x * ) = O n -2 and x n -x n-1 = O n -1 (4.44) (ii) If b > 3 then: +∞ n=0 n F (x n ) -F (x * ) < +∞ and +∞ n=0 n x n -x n-1 2 < +∞ (4.45) In fact : F (x n ) -F (x * ) = o(n -2 ) and x n -x n-1 = o(n -1
) and the sequence {x n } n∈N converges to a minimizer x * .

(iii) If b 3 and if F satisfies L(2) and admits a unique minimizer, then the following convergence rate holds true asymptotically:

F (x n ) -F (x * ) = O n -2b 3 and x n -x n-1 = O n -b 3 (4.46)
As we shall remark, Corollary 4.5.2, can be applied to a class of problems of a particular interest such as the (generalized) LASSO problem. In that case, the minimizing function F is convex piece-wise polynomial of degree 2, hence (see Corollary 9 in [START_REF] Bolte | From error bounds to the complexity of firstorder descent methods for convex functions[END_REF]), for all r F * , the function F satisfies L(2) on every sub-level set {F ≤ r} = {x ∈ R d : F (x) ≤ r} . This inducts that the point 2.(iii) of Corollary 4.5.2 is applicable, under the supplementary condition of the uniqueness of a minimizer of F . More precisely we have the following Corollary. Remark 18. More generally (see Corollary 4.3 in [START_REF] Drusvyatskiy | Error bounds, quadratic growth, and linear convergence of proximal methods[END_REF] or [START_REF] Garrigos | Convergence of the forward-backward algorithm: Beyond the worst case with the help of geometry[END_REF]), if F (x) = f (Ax) + g(T x), with some linear operators A ∈ R N ×M and T ∈ R N ×M and f , g are convex piecewise-linear quadratic functions (see Definition 10.20 in [START_REF] Tyrrell | Variational analysis[END_REF]), then F satisfies the L(2) condition. Hence by assuming the uniqueness of the minimizer, the point 2.(iii) of Corollary 4.5.2 is still applicable in this more general setting. Last but not least, we shall mention that for the minimization problem of F = f + g, where f, g are both convex with f differentiable with L-Lipschitz gradient and g lower semi-continuous, by making additional hypotheses, such as non-degeneracy condition on F and restricted injectivity on f (for more details see the work [START_REF] Liang | Activity identification and local linear convergence of forward-backward-type methods[END_REF]), the uniqueness of the minimizer x * is reassured. If in addition g is partially smooth at x * relative to a neighborhood of x * (see Definition 5 in [START_REF] Liang | Activity identification and local linear convergence of forward-backward-type methods[END_REF]), then F satisfies L(2) locally in x * . Thus, in that case we can recover the order of O n -2b 3 of the point 2.(iii) of Corollary 4.5.2.

The second main Theorem of this Chapter treats the case of a function F with a "sufficiently flat" behavior near the minimizer of F . In our framework this is translated to considering the case of a smooth and convex function F , verifying Hypotheses H(β) and L(p), with some parameters β and p, satisfying p ≥ β > 2.

Theorem 4.5.2. Let F : R d → R be a convex differentiable function with a L-Lipschitz continuous gradient for some L > 0. Let 0 < γ ≤ 1 L and {x n } n∈N be the sequence generated by i-GD Algorithm (11).

Assume that F satisfies H(β) and L(p) with p ≥ β > 2, and that F has a unique minimizer x * . If b ≥ β+2 β-2 , then the following estimate hold true asymptotically:

F (x n ) -F (x * ) = O n -2p p-2 (4.47)
To our knowledge, Theorem 4.5.2 is new and it is the first one that provides estimates for the objective function F (x n ) -F (x * ) for an inertial gradient descent scheme (which is not a descent scheme), for a function with sufficiently flat geometry (i.e. satisfying L(p) and H(β), with p β > 2). For some similar results concerning restarting variants of Algorithm 11 in the case when F satisfies L(p), with p > 2, one can also consult the works [START_REF] Roulet | Aspremont. Sharpness, restart and acceleration[END_REF] and [START_REF] Zhang | New analysis of linear convergence of gradient-type methods via unifying error bound conditions[END_REF]. Several comments can also be made concerning the related results in the existing literature.

First of all, Theorem 4.5.2, indicates that for b large enough (b ≥ β+2 β-2 ), the order of convergence rate for the objective function F (x n ) -F * is always better than o(n -2 ), which is the order obtained with the sole assumption of convexity of F (see Theorem 3.3.1 in Chapter 3). Unfortunately, as in 4.5. Main results for the i-GD Algorithm point 2.iii of Theorem 4.5.1, the proof of Theorem 4.5.2, necessitates the uniqueness of the minimizer x * , which constrains a bit its applicability. Nevertheless we think that this hypothesis is due to technical difficulties and may be possibly lifted.

In addition, the order of convergence rate (4.47) for the objective function F (x n )-F (x * ), consists also of an improved result, in comparison to the order obtained by general descent methods (such as the Gradient Descent algorithm (2), which is O n -p p-2 (see for example Theorem 4.2.1) and is optimal (see for example [START_REF] Merlet | Convergence to equilibrium for the backward Euler scheme and applications[END_REF]). This suggests that for functions with "sufficiently flat" geometry (i.e. F satisfying H(β) and L(p), with p β > 2), the i-GD algorithm 11, turns out to accelerate the convergence rates for the objective function F (x n ) -F * up to a factor of 2.

Finally the estimate (4.47) indicates that the order of convergence rate for the objective function F (x n ) -F (x * ) is surprisingly not ruled by the over-relaxation parameter b at least asymptotically, but only by the local sharpness of the function F (i.e. the parameter p, always under the assumption b β+2 β-2 ). As before we present an illustration of the order of the convergence rate for the objective function F (x n ) -F * = O n -p(b) , as given by Theorem 4.5.2. For simplicity reasons we consider a function F satisfying H(β) and L(β), with β > 2.

In addition in Table 4.1 we give an overall summary of the results in Theorems 4.5.1 and 4.5.2, concerning the convergence properties of the i-GD Algorithm 11, depending on the over-relaxation parameter b > 0 and the local geometry of function F (i.e. parameters β and p). b (inertia)

H(β) and L(p) rates F (x n ) -F * convergence b ≤ 1 + 2 β H(β) O n -2βb β+2 b > 1 + 2 β H(β) o(n -2 ) x n → x * b ≥ 1 + 2 β H(β) & L(2) O n -2βb β+2 x n → x * b ≥ β+2 β-2 p ≥ β > 2, H(β) & L(p) O n -2p p-2 x n → x * Table 4
.1: Summary of the convergence properties of i-GD Algorithm 11. In the last two rows the uniqueness of the minimizer of F is also assumed.

Convergence analysis of the i-GD algorithm

As made in Chapter 3, the strategy that we follow in order to prove Theorems 4.5.1 and 4.5.2 is a Lyapunov-type analysis for a suitable energysequence and the asymptotic equivalences. The choice of the Lyapunov energy-sequence, as also the asymptotic analysis are highly-inspired by the work made in the continuous-time counterpart for a solution of (1.18) in the work [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF] and briefly presented in Section 1.3.4, in Part I.

For this section we make use of the classical sequences defined in Chapter 3, i.e. for {x n } n∈N a sequence generated by i-GD Algorithm 11 and x * ∈ arg min F , for all n ≥ 1 we denote :

w n = F (x n ) -F (x * ), δ n = x n -x n-1 2 and h n = x n -x * 2 . (4.48)
and for all λ ≥ 0 :

v n = λ(x n-1 -x * ) + t n (x n -x n-1 ) 2 (4.49)
In addition -due to technical issues on the proofs-we consider a slightly modified version of the energy-sequence E n as the one defined in (3.34). In the current Chapter, for any β ≥ 1, λ > 0 and ξ ∈ R, and {x n } n∈N a sequence generated by i-GD algorithm 11 and x * ∈ arg min F , we consider the sequence {E n } n≥1 as follows :

E n = t 2 n + λβt n F (x n -F (x * ) + 1 2γ λ(x n-1 -x * ) + t n (x n -x n-1 ) 2 + λt n 2γ x n -x n-1 2 + ξ 2γ x n-1 -x * 2 = t 2 n + λβt n w n + 1 2γ v n + λt n 2γ δ n + ξ 2γ h n-1
(4.50) Observe that the energy can also be expressed as:

E n = t 2 n + λβt n w n + 1 2γ t 2 n δ n + λt n h n -h n-1 + (λ 2 + ξ)h n-1 (4.51)
Notice also that by using the basic convex inequality

u 2 ≤ 2 u -v 2 + 2 v 2 , ∀u, v ∈ R d (4.52)
Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 124

with u = t n (x n -x n-1 ) and v = λ(x n-1 -x * ), in the definition of E n (4.50), we find :

2γE n ≥ 2γ(t 2 n + λβt n )w n + t 2 n 2 + λt n x n -x n-1 2 + (ξ -λ 2 ) x n-1 -x * 2 ≥ 2γt 2 n w n + t 2 n 2 δ n + (ξ -λ 2 )h n-1
(4.53) which we will use during the several proofs.

In this context, the energy-sequence E n is of the form:

E n = ϕ n F (x n ) -F (x * ) + R n (4.54)
where R n is not necessarily non-negative (remark here that the parameter ξ can be also non-positive). In fact the exact construction of E n (see (4.50)), depends on the geometric properties of F and on the order of the convergence rate of the objective function, i.e. the value of δ such that:

F (x n ) -F (x * ) = O n -δ (4.55)
as stated in Theorems 4.5.1 and 4.5.2. In order to get the estimation (4.55), a two-step procedure is used:

1. First of all, since ϕ n ∼ n 2 , we show that the control over the growth or the decay of E n is of the following form (see for example relation (4.63) for Theorem 4.5.1 and (4.84) for Theorem 4.5.2):

E n Kn -δ+2 (4.56)

2. Since R n is not necessarily non-negative we cannot deduce (4.55) directly from (4.56). For this issue, we infer the geometric properties of F (in particular hypothesis L(p)) in order to deduce (4.55) from (4.56).

To get the appropriate control (4.56) on E n , we follow a classical strategy for bounding functions using a differential inequality, which is motivated by the continuous-time setting (see [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF]) and is similar to the one used for the proof of point 1 of Theorem 3.3.1 in Chapter 3.

More precisely, in the framework of the current chapter, we prove that the energy-sequence {E n } n 1 , satisfies the following recursive relation asymptotically: 3), for some suitable positive constants a, a 1 and a 2 . This allows to deduce the existence of a constant K such that for n ∈ N large enough, we have:

E n+1 -E n c n E n +
E n Kn c (4.58)
Finally, in order to deduce (4.55) from (4.56) we use different strategies depending on the hypotheses on the geometry of F and on the over-relaxation parameter b.

1. For the first point of Theorem 4.5.1, the sequence R n in (4.54) is positive and (4.55) holds directly.

2. For the second point of Theorem 4.5.1 and for the bound of Theorem 4.5.2, we have

R n = R n + ξ x n -x * 2
, where R n is non negative and ξ non positive. Thus, from (4.56), for n large enough, it follows that:

ϕ n (F (x n ) -F (x * )) -|ξ| x n -x * 2 Kn -δ+2 (4.59)
and we conclude, using the growth condition L(p) to bound x * -x n 2 and get inequalities such that

ϕ n (F (x n ) -F (x * )) + A 1 (F (x n ) -F (x * )) 2 p Kn -δ+2 (4.60)
which, by recalling that ϕ n = n 2 and using an appropriate strategy when p > 2 (see Lemma B.0.5), leads to (4.55).

Of course the value of δ in (4.55) and the use of conditions L(p) are different in the two theorems, which leads to different results, but the strategies in both cases are similar enough. Firstly we give the Lemma that describes the local variation of the energy E n (i.e. E n+1 -E n ) in terms of the energy E n , the objective function w n , the local variation of x n (i.e. the sequence δ n ) and the distance to the minimizer h n Lemma 4.6.1. Let F : R d → R be a convex differentiable function with a L-Lipschitz continuous gradient for some L > 0 and satisfies H(β) with β ≥ 1 and x * ∈ arg min F . Let also 0 < γ ≤ 1 L and {x n } n∈N be the sequence generated by Algorithm (11). Then for all λ ≥ 0 and ξ = λ(λ + 1 -b) in the definition of E n , the following recursive formula holds for all n ≥ 1:

2γ(E n+1 -E n ) ≤ 2γ c(λ) t n E n + 2γ A 1 (λ)t n + 1 -2λβ(λ + 1 -b) w n + A 2 (λ)δ n + A 3 (λ) t n h n-1 (4.61)
Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 126

where:

c(λ) = 2(λ + 1 -b), A 1 (λ) = 2b -(β + 2)λ, A 2 (λ) = (2λ + 1 -b)(1 -b) A 3 (λ) = -2λ(λ + 1 -b)(2λ + 1 -b)
The following Lemma gives an upper bound-estimate over the energy sequence {E n } n≥1 , which will be useful for the proof of Theorem 4.5.1. Lemma 4.6.2. Let F : R d → R be a convex differentiable function with a L-Lipschitz continuous gradient for some L > 0 and satisfies H(β) with β ≥ 1. Let also 0 < γ ≤ 1 L and {x n } n∈N be the sequence generated by Algorithm (11) with one of the following hypotheses in force:

i. b < 1 + 2 β ii. b ≥ 1 + 2
β and F admits a unique minimizer x * and satisfies L(2).

Then, for λ = 2b β+2 and ξ

= λ(λ + 1 -b) = 2bβ (β+2) 2 1 + 2 β -b in
the definition of the energy E n , there exists some n 0 ∈ N, such that for all n ≥ n 0 , the following recursive formula holds true:

E n+1 -E n ≤ a (n + b -1) 2 + c (n + b -1) E n (4.62)
for some constant a ≥ 0 and c = 2 -2bβ β+2 . In fact, the following estimate holds true asymptotically:

E n = O n 2-2bβ β+2 (4.63)
We are now ready to give the full proof of Theorem 4.5.1.

Proof of Theorem 4.5.1. We start this demonstration by proving the points 1 and 2(iii) of Theorem 4.5.1. For that, we choose:

λ = 2b β + 2 > 0, ξ = λ(λ + 1 -b) = 2bβ (β + 2) 2 1 + 2 β -b ,
in the definition (4.50) of the energy E n . Using Lemma 4.6.2, there exist n 0 ∈ N and a positive constant C such that, for all n ≥ n 0 , we have:

E n ≤ Ct 2-2bβ β+2 n . (4.64)
In order to deduce the expected convergence rates on w n = F (x n )-F (x * ), we use different strategies depending on the sign of the parameter ξ. Firstly 127 4.6. Convergence analysis of the i-GD algorithm we consider the case b < 1 + 2 β , i.e.: ξ > 0. In that case, the energy E n is a sum of non-negative terms, hence:

E n = t 2 n + λβt n w n + 1 2γ (v n + λt n δ n + ξh n-1 ) ≥ t 2 n w n .
Combining the very last inequality with (4.64) and noting that t n ∼ n asymptotically, we get the appropriate estimate:

w n = O n -2bβ β+2 for all n ≥ n 0 ,
as asserted by the first point of Theorem 4.5.1. In addition, since ξ > 0, from the definition of the energy E n (3.34) and (4.63) we find :

h n-1 2γ ξ E n Kt 2-2bβ β+2 n (4.65)
asymptotically, for some suitable positive constant K. Finally, using the inequality (4.53), we have: Consider now the case b ≥ 1 + 2 β i.e. ξ ≤ 0. In that case, the energy E n is not a sum of non negative terms anymore:

t 2 n 2 δ n ≤ 2γE n -2γt 2 n w n + (λ 2 -ξ)h n-1 ≤ 2γE n + |λ 2 -ξ|h n-1 (4.
2γE n = 2γ t 2 n + λβt n w n + v n + λt n δ n -|ξ| x n-1 -x * 2
, and an additional growth condition L(2) will be needed to bound x n-1x * 2 . First, applying the inequality (4.52), on the one hand to u = t n (x nx n-1 ) and v = λ(x n-1 -x * ) and on the other hand to u = x n-1 -x * and v = x * -x n , we have for all n ∈ N:

v n ≥ t 2 n 2 δ n -λ 2 x n-1 -x * 2 , x n-1 -x * 2 ≤ 2δ n + 2h n .
Using these two inequalities successively, we deduce:

2γE n ≥ 2γ t 2 n + λβt n w n + 1 2 + λ t n t 2 n δ n + (ξ -λ 2 ) x n-1 -x * 2 ≥ 2γ t 2 n + λβt n w n + 1 2 + λ t n - 2|ξ -λ 2 | t 2 n t 2 n δ n -2|ξ -λ 2 | x n -x * 2 ≥ 2γ t 2 n + λβt n w n + t 2 n 4 δ n -2|ξ -λ 2 | x n -x * 2 (4.67)
Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 128 since the coefficient of t 2 n δ n converges to 1 2 , and thus is greater than e.g. 1 4 for n large enough. Assuming in addition that F satisfies the growth condition L(2) and admits a unique minimizer, we then obtain for n large enough:

E n ≥ t 2 n +λβt n -2|ξ-λ 2 |K -1 2 w n + t 2 n 4 δ n = 1+ λβ t n -2 |ξ -λ 2 |K -1 2 t 2 n t 2 n w n + t 2 n 4 δ n
Hence there exists n 0 ∈ N such that for all n ≥ n 0 , we have:

E n ≥ t 2 n 2 w n + t 2 n 4 δ n .
Using finally the estimate (4.64) on the energy E n allows us to conclude the proof of the point 2(iii) of Theorem 4.5.1. Consider again the case when b ≥ 1 + 2 β . In order to prove the points 2(i) and 2(ii) of Theorem 4.5.1 with the only assumption that F satisfies the condition H(β), we choose different values of the parameters λ and ξ in the definition of the energy E n . Let us set:

λ = b -1 > 0, ξ = λ(λ + 1 -b) = 0.
In that case, the energy is again a sum of non negative terms and we then have:

E n = t 2 n + (b -1)βt n w n + 1 2γ v n + (b -1)t n δ n ≥ t 2 n w n . (4.68)
To obtain the expected convergence rate on w n as expressed in the point 2(i) of Theorem 4.5.1, it is sufficient to prove that the energy E n is bounded. For that purpose, we apply Lemma 4.6.1 with λ = b -1. Keeping in mind that b ≥ 1 + 2 β , we then have:

∀n ≥ 1, E n+1 -E n ≤ β(1 + 2 β -b)t n + 1 w n - 1 2γ (b -1) 2 δ n ≤ w n - 1 2γ (b -1) 2 δ n ≤ w n (4.69)
Injecting (4.68) into (4.69), we then obtain for all n ≥ 1,

E n+1 ≤ 1 + 1 t 2 n
E n , which implies by a recurrence argument that:

∀n ≥ 1, E n+1 ≤ E 1 n i=1 1 + 1 t 2 i
By inferring Lemma B.0.3 with c = 0 and a = 1 we deduce that the sequence (E n ) n≥1 is bounded. By (4.68), the sequence (t 2 n w n ) n≥1 is also bounded. Hence:

w n = O(t -2 n ) = O(n -2 ) asymptotically.

Convergence analysis of the i-GD algorithm

Assume in addition that: b ≥ 2. According to the definition (3.36) of the energy E n , for all n ≥ 1, we have:

2γE n = 2γ t 2 n + (b -1)βt n w n + t 2 n δ n + (b -1)t n (h n -h n-1 ) + (b -1) 2 h n-1 ≥ (b -1)t n (h n -h n-1 ) + (b -1) 2 h n-1 ≥ (b -1) t n h n -(t n -b + 1)h n-1
Observe now that, since b ≥ 2, we have:

t n -b + 1 ≤ t n -1 = t n-1 , hence: ∀n ≥ 1, 2γE n ≥ (b -1) (t n h n -t n-1 h n-1 ) .
Using the fact that {E n } n≥1 is bounded, there exist a constant C > 0 and n 0 ∈ N such that:

∀n ≥ n 0 , t n h n -t n-1 h n-1 ≤ C. (4.70)
By summing (4.70) from n 0 to N , we obtain that for all

N ≥ n 0 , t N h N ≤ t n 0 h n 0 + CN ≤ t n 0 h n 0 + Ct N .
The sequence (h n ) n is so bounded, which implies that {x n } n≥1 is also bounded. Moreover using (4.53), for all n ≥ 1, we find :

t 2 n 2 δ n ≤ 2γE n + λ 2 h n-1 ,
By using the fact that the sequences h n and E n are bounded, we obtain the boundedness of the sequence (t 2 n δ n ) n , which is equivalent to δ n = O(n -2 ) asymptotically. This concludes the proof of point 2(i) of Theorem 4.5.1.

Finally, suppose that b > 1 + 2 β . Let: η = b -(1 + 2 β ) > 0. As previously done in (4.69), we have:

E n+1 -E n ≤ -(βηt n -1)w n - 1 2γ (b -1) 2 δ n ≤ -(βηt n -1)w n (4.71)
Moreover using the fact that there exists n 0 ∈ N such that for all n ≥ n 0 , we have: β η 2 t n ≤ βηt n -1 and summing (4.71) over n ∈ {n 0 , • • • , N }, for all N > n 0 , we find:

β η 2 N n=n 0 t n w n ≤ E n 0 < +∞ (4.72)
Lastly, the proof of the summability of the (nδ n ) n is exactly the same as in [CD15, Corollary 2]. In few words, applying Lemma B.0.7, with γ ≤ 1 L , y = x n + α n (x n -x n-1 ) and x = x n , or equivalently using the Lipschitz continuity of the gradient of F , we have:

∀n ≥ 1, δ n+1 -α 2 n δ n ≤ 2γ(w n -w n+1 ),
Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 130

where: α n = n n+b . Summing the last inequality from n = 1 to N , we obtain:

(b -1) N n=2 (2t n -b + 1) δ n ≤ 2γ N n=2 (2t n + 1)w n + t 2 2 w 1 .
Observe now that for all n ≥ 1, we have:

2t n -b + 1 = 2n + b -1 ≥ 2n.
Thus the summability of nδ n follows from the summability of t n w n , which concludes the proof of point 2(ii) of Theorem 4.5.1.

For the Corollary 4.5.1, we give a brief proof, following the lines of the one made in [START_REF] Attouch | The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/kˆ2[END_REF] :

Proof of Corollary 4.5.1. The proof of this corollary is a direct consequence of the summability (4.39) and of Lemma B.0.8 in Appendix B (these two results are key elements of the convergence proof in [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF]).

We define

U n = F (x n ) -F (x * ) + xn-x n-1 2 2γ
. From (4.39) we deduce that Let us now turn our attention on the proof of Theorem 4.5.2, which takes into consideration functions F with a "flat" behavior near the minimizer. To do so we need the following Lemma : β-2 , then there exists some n 0 ∈ N and some constant C > 0 such that the following recursive formula holds true for all n n 0 :

2γ E n+1 -E n ≤ 2γ - 4 β -2 + C t n E n t n + C t 2 n h n-1 . (4.76)
Proof of Theorem 4.5.2. We split the proof into three parts. In the first part we present the analysis in order to obtain a control over the decay of the energy E n , by making use of an a-priori estimate for h n . In the second part we deduce some estimates for the sequence w n . In the last part we infer a bootstrap argument to improve the estimates for the sequence w n and to get those stated in Theorem 4.5.2. Part 1: In this part, we show that choosing λ = 2 β-2 and ξ

= λ(λ + 1 - b) = 2 β-2 β β-2 -b
in the definition of the energy E n , the control over the decay of the energy E n is given by:

E n = O n -m , with: m =    4 p if β = 6p+8 p+2 min( 4 β-2 , 1 + 4 p ) otherwise
.

In this proof we will frequently use the notation d = 4 β-2 . By Lemma 4.6.3, there exist n 0 ∈ N and a positive constant C such that

∀n n 0 , E n+1 -E n - d t n E n + C t 2 n E n + C 2γt 2 n h n-1 .
(4.77)

Denoting by

H n = C 2γt 2 n h n-1 and z n = 1 -d tn + C t 2 n
, the previous inequality (4.77) can be rewritten as:

E n+1 z n E n + H n (4.78)
which, by applying Lemma B.0.2, implies: Hence by (4.79) and the definition of

∀n n 0 , E n+1 n i=n 0 z i E n 0 + n i=n 0 H i i m=n 0 z m . ( 4
C 1 t -d n n i=n 0 z i = n i=n 0 1 - d t i + C t 2 i C 2 t -d n . ( 4
H i = C 2γt 2 i h i-1 , we find: E n+1 C 2 t -d n E n 0 + C -1 1 n i=n 0 t d i H i C 2 t -d n E n 0 + C 3 n i=n 0 t d-2 i h i-1 (4.81)
for some suitable positive constant C 3 . So to obtain an estimate on the energy, we first need an estimate on h n . Assuming that F satisfies the growth condition L(p) with p > 2, and admits a unique minimizer x * , we have:

h i-1 = x i-1 -x * 2 K -1 p F (x i-1 ) -F (x * ) 2 p = K -1 p w 2 p i-1 . (4.82)
By injecting the last inequality (4.82) into (4.81), for n n 0 we find:

E n+1 C 2 t -d n E n 0 + C 3 K -1 p n i=n 0 t d-2 i w 2 p i-1 C 2 t -d n E n 0 + C 3 K -1 p n i=n 0 t d-2-4 p i t 2 i w i-1 2 p (4.83)
Moreover, since F satisfies H(β) with β > 2, we can apply the results stated in Theorem 4.5.1. Here: b β+2 β-2 > 1 + 2 β , Hence from relation (4.39) of Theorem 4.5.1, the sequence (t 2 i w i-1 ) i is bounded. Therefore, from the previous inequality (4.83), by using the series-integral comparison test, we find that for all n n 0 , it holds:

E n+1 C 2 t -d n E n 0 + C 4 t max{d-1-4 p ,0} n , if d = 1 + 4 p or : E n+1 C 2 t -d n E n 0 + C 4 log t n C 2 t -d n E n 0 + C 4 t n , if d = 1 + 4 p
for some suitable positive constant C 4 (at each case). Therefore there exists a suitable positive constant C > 0, such that :

E n+1 Ct -m n (4.84) for m =    4 p if d = 1 + 4 p min{d, 1 + 4 p } otherwise .

Convergence analysis of the i-GD algorithm

Part 2: Once obtained the control (4.84) over the decay of E n , we now want to deduce the convergence rates on w n = F (x n ) -F (x * ) expected in Theorem 4.5.2.

Firstly, observe that when β > 2 and b ≥ β+2 β-2 , we have

λ = 2 β -2 > 0 and ξ = λ(λ + 1 -b) = 2 β -2 ( β β -2 -b) < 0.
The energy E n is not a sum of non negative terms, so that, as in Theorem 4.5.1, we will so need the growth condition L(p) to bound x n-1 -x * 2 , or more precisely x n -x * 2 in what follows. First remark that using (4.53), we get:

2γE n = 2γ(t 2 n + λβt n )w n + v n + λt n δ n -|ξ| x n-1 -x * 2 2γt 2 n w n + t 2 n 2 δ n -λ(b -1) x n-1 -x * 2 . (4.85)
By using the inequality

x n-1 -x * 2 2 x n -x * 2 + 2 x n -x n-1
2 in (4.85) we find:

2γt 2 n w n + t 2 n 2 -2λ(b -1) δ n 2γE n + 2λ(b -1)h n . (4.86)
Hence, there exists n 0 ∈ N such that for n n 0 :

t 2 n w n 2γE n + 2λ(b -1) x n -x * 2 Ct -m n + 2λ(b -1) x n -x * 2
using the control estimate (4.84) on the energy E n for some suitable positive constant C. Using the growth condition L(p) with p > 2 combined with the uniqueness of the minimizer, gives:

t 2 n w n Ct -m n + 2λ(b -1) K p w 2 p n (4.87)
Deducing now the convergence rates on w n is quite technical: multiplying (4.87) by t m n and setting g n = t m+2 n w n , we find: and α = 2 p ∈ (0, 1), we obtain:

g n C + 2λ(b -1) K t mp-2(m+2
g n 2 max C, 2 2 p 2λ(b -1) K p t mp-2(m+2) p n p p-2 = 2 max C, C t m-4 p-2 n = O t M n (4.89) where M = max 0, m - 4 p -2 = m -4 p-2
if p max(β, 4) and d = 1 + 4 p 0 otherwise.

Substituting then g n = t m+2 n w n , we finally have: Thus, we deduce the existence of an l * ∈ N, such that B 2 (l * ) does not hold true. Therefore (since the condition B 2 (l * ) does not hold true), by (4.102) we deduce that:

w n = O t M -m-2 n . ( 4
w n = O t -2p p-2 n (4.104)
for all (β, p), such that 2 < β ≤ p, which concludes the proof of Theorem 4.5.2.

Numerical examples

In this section we are presenting some numerical illustrations of applying the i-GD Algorithm 11, on some simple toy-examples of minimization problems. These examples are chosen in order to put in evidence the results of Theorems 4.5.1 or Corollary 4.5.2 and Theorem 4.5.2. We compare the i-GD Algorithm 11 with different choices for the over-relaxation parameter b > 0, as also with the Gradient Descent (GD). In particular we take a look on the different orders of convergence rates of the error F (x n ) -F (x * ), for functions that are satisfying the conditions H(β) and L(p) with β ≥ 1 and p ≥ 2.

In the following four examples (Figure 4.4) we test the i-GD Algorithm with five different choices for the over-relaxation parameter b (black,blue, green, red and magenta), for the minimization problem of the function F : R N -→ R + , where F has different expressions, that enter in the framework of Theorem 4.5.1 and Corollary 4.5.2. In all the following examples we choose x 0 = x 1 ∈ H randomly and the step parameter is fixed to γ = 1 L , where L is the Lipschitz constant of the gradient of the differentiable part in F . The different plots for these examples can be found in Figure 4.4.

Least squares:

We are interested in the minimization problem of F (x) = Ax -c 2 . In this example we choose A ∈ R 100 × R 400 with random i.i.d. centered Gaussian coefficients and c a random i.i.d. centered Gaussian vector.

Quadratic loss:

We are searching to minimize F (x) = Ax, x + c, x , where A ∈ R 200 × R 200 is a random, symmetric and positive-definite matrix and c a random vector. One can notice that in all the four cases, the over-relaxation parameter b plays a crucial role for the convergence rate of the objective function. Indeed 4.7. Numerical examples larger values of b, seem to lead to faster convergence rates at a late stage, as Theorem 4.5.1 and Corollary 4.5.2 assert. However, it is also worth mentioning that taking b very large, results to "slower" convergence behavior at the beginning. This last remark suggests that the over-relaxation parameter b, has also a serious impact in the hidden constants in the "big Ohs" of the different estimates in Theorem 4.5.1 and Corollary 4.5.2.

In the second illustration (Figure 4.5), we are considering the simple example of minimizing the function F : R 8 -→ R + , such that F (x) = x 8 . In this setting we compare the Gradient Descent (yellow), i-GD Algorithm with five different choices for the over-relaxation parameter b (black, blue, green, red and magenta) for different values of p (p = 3, p = 4, and p = 8). We always set x 0 = x 1 = 1 T ∈ R 8 and a suitable step-size γ 1 L , where L is the Lipschitz constant of the gradient of F . The different plots for these examples can be found in Figure 4.5, which seem to be bounded.

In this case we intentionally choose a parameter b = p+1 p-2 < p+2 p-2 (blue line) that violate the assumptions of Theorem 4.5.2. This seems to cause an overshoot on the minimum of the corresponding trajectory. On the contrary the trajectories that respect the hypotheses of Theorem 4.5.2 (red, green and magenta lines) seem to produce a decreasing behavior, without bump effects. From the second row, we can notice that at a final stage, the slope of the lines obtained by the i-GD algorithm, are almost the same as the one of n -2p p-2 (except the blue one). This suggests that the order of convergence rate found in Corollary 4.5.2 for the i-GD algorithm is optimal for this kind of functions, under the assumption b ≥ p+2 p-2 . Remark also in this case that the slope obtained by the GD algorithm, is less steep than n -2p p-2 , which shows that the i-GD algorithm is a better choice in this framework.

Finally, notice that in contrary with the results in Figure 4.4, this time smaller values of b seem to give slightly "better" results, given that b ≥ p+2 p-2 . This suggests that the limiting value b = p+2 p-2 may minimize the hidden constants in the "big Oh" of Theorem 4.5.2. Nevertheless the order of convergence rate seems to be independent of the parameter b, and it only varies with respect to the parameter p (larger values of p lead to slower convergence rates), as Theorem 4.5.2 asserts. here the values of the error F (x n ) -F (x * ) as a function of the iterations n, are in loglog scale. The light-blue curve corresponds to the theoretical bound n -2p p-2 as found in Theorem 4.5.2. Notice that all the slopes obtained by the i-GD algorithm with b ≥ p+2 p-2 (red, green and magenta), except the blue one (b < p+2 p-2 ) and the black one (gradient descent) seem to be identical to the light-blue one (n -2p p-2 ). In the third row : the values of the rescaled error F (x n ) -F (x * ), by ( √ γ(n + b -1))

2p p-2 . Here we can remark that all the three choices of parameter b (red, green and magenta), that enter the framework of Theorem 4.5.2, are bounded. On the contrary the blue one seems to explode periodically along the iterations. Notice also that among the three bounded lines, the one that has a better upper bound (red), corresponds to the smallest possible value of b that enters the framework of Theorem 4.5.2 (i.e. b = p+2 p-2 ).

Then for all n ≥ 1 it holds :

u n+1 ≤ C 0 n i=1
(1 + a i ) (B.4) Lemma B.0.3. Let a, c and C 0 be some real numbers such that C 0 > 0 and a > 0 and {u n } n∈N be a sequence of real numbers, and n 0 ∈ N * such that 1 + c n + a n 2 > 0 for all n ≥ n 0 . Suppose also that for all n ≥ n 0 , it holds:

u n+1 ≤ C 0 n i=n 0 (1 + c i + a i 2 )
Then there exists a positive constant C, such that for all n ≥ n 0 , it holds:

u n+1 ≤ Cn c
Proof. In fact for all n ≥ n 0 we have: By (B.6) and (B.7) we infer that there exist n 0 ∈ N and some suitable positive constants C 1 and C 2 such that for all n ≥ n 0 it holds:

n i=n 0 1 + c i + a i 2 = e
C 1 n c ≤ n i=n 0 1 + c i + a i 2 ≤ C 2 n c (B.8)
From the hypothesis we have: Proof. By applying Lemma (B.0.7) to y = y n and x = 1 -λ t n+1 x n + λ t n+1 x * , with γ ≤ 1 L , we obtain ( here λ ∈ (0, 1 + b)):

u n+1 ≤ C 0 n i=n 0 1 + c i + a i 2 (B.
2γ F 1 -λ t n+1

x n + λ t n+1

x * -F (x n+1 ) . Then the following recursive formula holds for all n ≥ 1 : 2 with the aid of E n and w n and then we regroup the different terms.

≥ x n+1 -x n + λ t n+1 (x n -x * ) 2 -
2γ(E n+1 -E n ) ≤ 2γ k n+1 -2(λ + 1 -b)t n w n + (λ + 1 -b) 2 + ξ x n -x n-1 2 - 2(λ + 1 -b)(λ 2 + ξ) t n x n-1 -x * 2 + 2γ 2(λ + 1 -b) t n E n + 2 ξ -λ(λ + 1 -b) x n -x n-1 , x n-1 -x * (B.
2γ(E n+1 -E n ) ≤ 2γ (2 -λ)(n + b) -1 -2(λ + 1 -b)t n w n + (λ + 1 -b)(2λ + 1 -b) x n -x n-1 2 + 2γ 2(λ + 1 -b) t n E n -2 λ(λ + 1 -b)(2λ + 1 -b) t n x n-1 -x * 2 = 2γ (2b -3λ)t n + 1 -λ w n + (λ + 1 -b)(2λ + 1 -b) x n -x n-1 2 + 2γ 2(λ + 1 -b) t n E n -2 λ(λ + 1 -b)(2λ + 1 -b) t n x n-1 -x * 2 (B.
E n+1 -E n ≤ a (n + b -1) 2 + c (n + b -1) E n (B.
From the convex inequality

α 2 ≤ 2 α + β 2 + 2 β 2 , ∀α, β ∈ H
and the definition of E n , we have ( for α = t n (x n -x n-1 ) and β = λ(x n-1 -x * )) we find :

2γE n ≥ 2γt 2 n w n + t 2 n 2 x n -x n-1 2 + (ξ -λ 2 ) x n-1 -x * 2 (ξ = λ(λ + 1 -b)) = 2γt 2 n w n + t 2 n 2 x n -x n-1 2 -λ(b -1) x n-1 -x * 2
(B.41) Therefore, we obtain : . Then the following recursive formula holds for all n ≥ 1 :

x n -x n-1 2 ≤ 2γ 2 t 2 n E n -4γw n + 2λ(b -1) t 2 n x n-1 -x * 2 (B.
E n ≤ C(n + b -1) 2(3-b) 3 (B.46)
for a suitable positive constant C.

Proof. From Lemma 3.3.2, without loss of generality we can suppose that for a suitable n 0 ∈ N, for all n n 0 , we have:

E n+1 -E n ≤ a (n + b -1) 2 E n + c (n + b -1) E n (B.
E n+1 ≤ 1 + c t n + a t 2 n E n (B.48)
Hence by a recurrence argument, for all n n 0 we find:

E n ≤ E n 0 n-1 i=n 0 1 + c t i + a t 2 i (B.49)
Thus, by applying Lemma B.0.3, we can conclude that there exists some n 0 ∈ N and a positive constant C > 0, such that for all n n 0 we have: E n ≤ Cn c , as expected.

Next we give a basic descent-type lemma adapted to the inexact FB algorithm 8 with the presence of perturbation errors, which is used in Section 3.4. The proof of this Lemma follows the one in [START_REF] Aujol | Stability of overrelaxations for the forward-backward algorithm, application to FISTA[END_REF]. Since α n = n n+b = n t n+1 , by using the definitions of w n , δ n and h n , we find: Then, for λ = 2b β+2 and ξ = λ(λ + 1 -b) = 2bβ (β+2) 2 1 + 2 β -b in the definition of the energy E n , there exists some n 0 ∈ N, such that for all n ≥ n 0 , the following recursive formula holds true:

2γλβt
+ (λ + 1 -b) 2 + λ(1 -b) -2λ(λ + 1 -b) + ξ δ n - 2(λ + 1 -b)(λ 2 + ξ) t n h n-1 + 2 ξ -λ(λ + 1 -b) x n -x n-1 , x n-1 -x *
E n+1 -E n ≤ a (n + b -1) 2 + c (n + b -1) E n (B.78)
for some constant a ≥ 0 and c = 2 -2bβ β+2 . In fact, the following estimate holds true asymptotically:

E n = O n 2-2bβ β+2 (B.79)
Proof. Firstly we suppose that b ≤ 1 + 2 β . Setting λ = 2b β+2 > 0, in the inequality (4.61) of Lemma 4.6.1, we find:

2γ(E n+1 -E n ) ≤ 2γ c t n E n + 2γA 1 w n + A 2 δ n + A 3 t n h n-1 (B. 80
)
where c = 2(λ + 1 -b) = 2 -2bβ β+2 and

A 1 = 1 -2βλ(λ + 1 -b) = 1 - 4bβ β + 2 1 - bβ β + 2 A 2 = (2λ + 1 -b)(1 -b) = β -2 β + 2 b 2 - 2β β + 2 b + 1 = (b -1) β -2 β + 2 b -1 and A 3 = -2λ(λ + 1 -b)(2λ + 1 -b) = - 2b β + 2 1 - bβ β + 2 1 - (β -2)b β + 2
Here we point out that in the case where b ≤ 1 + 2 β the constant A 1 is non-negative while A 2 may be positive or negative, and A 3 ≤ 0.

Without loss of generality we can suppose that the constant A 2 is nonnegative. Denoting by A = max{A 1 , A 2 } 0, from (B.80), we obtain: In this point we consider the two cases depending on the value of the parameter b.

2γ(E n+1 -E n ) ≤
Firstly we suppose that b < 1 + 2 β . In this case A 3 < 0, therefore, for n ∈ N large enough we have that :

B.2. Proofs of Lemmas in Chapter 4

For the second case we suppose that b 1 + 2 β and F satisfies L(p) with p = 2.

Remark that in this case (b 1 + 2 β ), by letting λ = 2b β+2 , the constant A 3 is eventually non-negative. In fact, if β > 2 and 1 + 2 β ≤ b ≤ β+2 β-2 , then A 3 0. Here without loss of generality we suppose that A 3 0 (the case A 3 ≤ 0 can be treated exactly in the same way as before in the case b ≤ 1 + 2 β ). In particular, by using the inequality u -v 2 ≤ 2 u -z 2 + 2 v -z 2 , for u = x n-1 , v = x * and z = x n , in (B.81) we find:

2γ(E n+1 -E n ) ≤ 2γAw n +
B 1 = 2b -(β + 2)λ = 2 b - β + 2 β -2 B 2 = (2λ + 1 -b)(1 -b) = (b -1)(b - β + 2 β -2 )
and

B 3 = -2λ(λ + 1 -b)(2λ + 1 -b) = - 4 β -2 b - β + 2 β -2 b - β β -

Conclusion and Perspectives

In this Thesis we presented a full analysis on the convergence properties of the family of Nesterov type inertial Forward-Backward Algorithms 7, for solving composite convex (non-smooth) minimization problems. We gave a unified presentation of the acceleration effects of the inertial Algorithm 7 and completed the palette of its convergence properties in the subcritical case, when the over-relaxation parameter b satisfies b ≤ 3. We additionally extended these results for the inexact version of the inertial FB algorithm 8, in the presence of perturbation errors.

Our approach followed a continuous to discrete path, which is revealed very practical and useful in the analysis. In particular we presented the study of the continuous system (1.18) (or (DI)), which can be associated to the inertial scheme 11 (or 7 in the non-differential case) with a particular finite difference scheme. We extended the known results concerning the differential equation (1.18), for the differential inclusion (DI), linked with the minimization of a non smooth convex functions.

The last years these continuous to discrete approaches are highly emerging in the field of optimization. Standard Lyapunov techniques for dynamical systems are transposed to their discrete counterparts, in order to offer a better understanding and an additional insight on the associated numerical algorithms. Nevertheless the passage from continuous dynamical systems to discrete schemes is not direct and must be treated in a delicate way every time. These observations rise up an increased interest on the different contacts between Optimization and other domains such as Numerical analysis. Notice for example, that in the current Thesis we only treated one possible discretization scheme for the differential equation/inclusion (1.18) (or (DI)). Other choices are possible and can lead to other optimal schemes. Remark for example, that even if all the convergence rates obtained for a solution of the continuous systems (1.18) and (DI) are proven to be optimal, this optimality is not yet proved for the corresponding rates obtained by the i-GD Algorithm 11 (or 7). In this type of questions, it is possible that other choices of discretization of (1.18) or (DI) might suit better, compared to Algorithm 11 (or 7) used here. In these types of questions and results, the step-size of the algorithm, linked with the time step-parameter of the associated discretized dynamical system, may play a central role, which has to be taken in consideration.

Finally, in the last Chapter we presented the convergence properties of the inertial Gradient-Descent algorithm 11, in the context of smooth convex optimization, under some additional hypotheses on the local geometry on the minimizing function F . As we mentioned in the introduction of Chapter 4, classical additional assumptions often met in practice, are the growth condition L(p), or Łojasiewicz gradient inequality (see Definition 4.2), which -roughly speaking-express the level of sharpness of the minimizing function near its minimizers. These types of conditions can render classical descent schemes, such as Gradient Descent algorithm, very efficient, according to the local sharpness of the minimizing function.

In our context, we showed that apart from growth-type conditions such as L(p), one should also exploit a flatness type condition such as H(β), in order to get competitive rates for inertial schemes such as the i-GD Algorithm 11. We believe that this study opens the way for further extensions in more general settings, and deeper investigation between the interplay of the geometrical hypotheses H(β) and L(p) and the efficiency of inertial schemes similar to Algorithm 11. For example, extending the analysis on a nonsmooth convex composite setting as the one considered in Chapter 3, or even in a non-convex one for functions that are satisfying H(β) with β > 0 and L(p), are some possible future perspectives of a great interest. Another interesting question concerns the comparison between inertial schemes such as Algorithm 11 and descent methods, depending on the local geometric parameters β and p. According to Theorems 4.5.1 and 4.5.2, it seems that for sharp geometries (at least as sharp as a quadratic function) descent schemes perform better (linear convergence rates), than the inertial scheme 11, which gives arbitrarily large, but sub-linear rates, depending on the over-relaxation parameter b. Whereas for functions with a more flat behavior the inertia turns out to accelerate the convergence rates. Of course in order to make such comparisons in a rigorous way, one shall also have knowledge of the exact dependence of the different constants of the "big Ohs" on the parameters of the algorithm, such as the step size, as also the over-relaxation parameter b. Since the proofs of Theorems 4.5.1 and 4.5.2 utilize asymptotic-equivalences arguments, this dependency is not easy to be precisely expressed and it constitutes a challenging question for future studies.



Lemma 2.3. 1 .

 1 Let 0 < b < 3 and x a solution to (DI) such that x(t 0 ) = x 0 > 0. Then lim t→∞ H(t) = l > 0 .

  Lemma A.2.2. (see Proposition 4.14 in[START_REF] Clarke | Functional analysis, calculus of variations and optimal control[END_REF]) Let g : R d -→ R be convex function and let K be a bounded set in R d . Then the set :

  min R(x) such that : Ax = y (3.3) or the regularized one: min D(Ax, y) + λR(x) (3.4)

  [CW05]),Douglas-Rachford ([LM79]) or Alternating Direction of Multipliers (ADMM) ([GM75],[GM76]), momentum methods([Pol64],[Nes83]), Primal-Dual ([CP11]) and many other variants.

  In the Figure below, we illustrate the order p(b) of convergence rates for the objective function F (x n ) -F * = O n -p(b) , depending on the overrelaxation parameter b > 0, as found in Theorem 3.3.1.

Figure 3 . 1 :

 31 Figure 3.1: Values of the order of convergence rate p(b) ,depending on the over-relaxation parameter b > 0, as given by Theorem 3.3.1. Remark that without any further assumptions on the function F , for b < 3 the order of convergence rate is sub-optimal in comparison to the worst-case optimal rate O(n -2 ) for first-order methods.

  42) with c and a as defined in Lemma 3.3.2. Hence by applying Lemma B.0.3, we obtain the following Lemma which gives the control over the order of the growth of the energy E n : Lemma 3.3.3. Let 0 < γ ≤ 1 L , b ∈ (0, 3) and {x n } n∈N the sequence generated by i-FB. Then for λ = 2b 3 and ξ = 2b(3-b) 9

  then by choosing ξ = λ(λ + 1 -b) and summing relation (3.39) over all n ≥ 1, for λ = b -1 and λ = 2, we obtain respectively :

  51) which gives the estimations of (3.29) since b > 3. Now in order to prove point 1 (i.e. (3.27)) of Theorem 3.3.1, we choose λ = 2b 3 and ξ = λ(λ + 1 -b) = 2b(3-b) 9 > 0 in the definition of E n (3.34). In that case by relation (3.43) of Lemma 3.3.3 and the definition of E n , (3.34), we have:

59) and since n k=1 1 t

 1 k is divergent, we have necessarily that U n -→ n→∞ 0. By positivity of w n and δ n we deduce that : lim n→∞ n 2 w n = 0 and lim n→∞ n 2 δ n = 0 (3.60) For the weak convergence result, we use the discretized version of Opial's Lemma (see Lemma B.0.1 in Appendix B) with S = arg min F . From (3.39) Lemma 3.3.1, for b > 3, λ = 2 and ξ = λ(b -λ -1

  4.1 expresses the trade-off between the overrelaxation sequence α n (thus the tuning of parameter b) and the error-terms e n ∈ H and ε n > 0 at each step. Formally, if 0 < γ ≤ 1 L , b > 0 and p = min{1, b 3 } and {x n } n∈N the sequence generated by the inexact i-FB algorithm. If in addition, the following assumptions hold true : +∞ n=1 n p e n ≤ A < +∞ and +∞ n=1 n p √ ε n ≤ B < +∞ (3.77) 95 3.4. Inexact version of the inertial Forward-Backward algorithm Then :

  Let us first treat the case b ≥ 3. Chapter 3. Inertial proximal-gradient algorithms 96

  87) which concludes the proof of Theorem 3.4.1 in the case b ≥ 3 (i.e. p=1), with C 1,p = 1 et C 2,p = γE 1 97 3.4. Inexact version of the inertial Forward-Backward algorithm In the same way, by (3.80) and doing similar computations as in the proof of Lemma (3.3.1) and (3.3.2), for b ∈ (0, 3), c = 2 -2b 3 λ = 2b 3 and ξ = λ(λ + 1 -b), we have :

  conclude the proof of Theorem 3.4.1 in the case 0 < b < 3 (i.e. p = b 3 ).

  the simple example of the function F : x ∈ R -→ |x| r with r > 1, a straightforward computation shows that F satisfies H(β) and L(p) if and only if 1 β r p. More generally we have the following Lemma showing the interplay of conditions H(β) and L(p): Lemma 4.4.4. If a convex differentiable function F satisfies both H(β) and L(p), with β, p 1, then necessarily the parameters β and p satisfy: p β in some suitable neighborhood of X * .Remark 13. Notice that by Lemma 4.4.4, roughly speaking, conditions H(β) and L(p) form a double-threshold for the minimizing function F in a suitable neighborhood of the set of its minimizers X * . In particular, the condition H(β) provides an upper-threshold for the function F (depending on β), while L(p) forms a lower-threshold for F (depending on p). In Figure4.1, we illustrate this remark.

Figure 4

 4 Figure 4.1: Conditions H(β) (in black dot-dashed line) and L(p) (in black dashed line), for some fixed parameters β ≥ 1 and p ≥ 2, for a function with a unique minimizer x * = 0. The blue, purple, green and red curves, represent some of the possible choices for a function F (i.e. F 1 , F 2 , F 3 , F 4 ), that has a unique minimizer x * and satisfies H(β) and L(p) in some neighborhood of x * .

Corollary 4.5. 1 .

 1 Under the same assumptions of point 2(ii) of Theorem 4.5.1, in the case b > 1 + 2

  the Figure below, we give a unified view of the order p(b) of the convergence rates for the objective function F (x n ) -F * = O n -p(b) , as given in Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 118 Theorem 4.5.1. This time the order p(b) depends both on the over-relaxation parameter b > 0 of the algorithm and the Hypotheses H(β) and L(p).

Figure 4 . 2 :

 42 Figure 4.2: Values of the order p(b) depending on b > 0 and the geometry of F . With red (and magenta) colour the rates found in Theorem 3.3.1, without supposing any geometric hypothesis apart of the convexity on F . With blue (and magenta) the corresponding order of convergence rates for a function F satisfying H(β) with β > 1 (Points 1. and 2.(i) of Theorem 4.5.1). Finally, in green colour the order obtained by supposing both H(β) and L(2), as given by Point 2.(iii) in Theorem 4.5.1.

.

  Corollary 4.5.3 (Generalized LASSO). Let F (x) = 1 2 Ax -y 2 + λ T x 1 , where A : R d -→ R M and T : R d -→ R M are some linear operators, y ∈ R M and λ > 0. Suppose also that arg min F = {x * }. Let {x n } n∈N , be the sequence generated by algorithm (4.42). Then for all b > 0, the following convergence rate holds true asymptotically :F (x n ) -F (x * ) = O n -2b 3Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 120

Chapter 4 .

 4 Figure 4.3: In red, the possible values of the order of convergence rates p(b), with respect to the over-relaxation parameter b, for a function satisfying H(β) and L(β), with β > 2. Notice that for large β, one has a larger variety of choices for b, but a smaller overall rate, with the limiting case of p(b) = 2 (red dotted line). On the other hand, when β is close to 2, the order is arbitrarily large, given that b is chosen arbitrarily big (satisfying b ≥ β+2 β-2 ). This fact goes in harmony with the corresponding result of Theorem 4.5.1, when the function is close to satisfy L(2) (but a bit flatter).

1234. 6 .

 6 Convergence analysis of the i-GD algorithm

  66)Injecting estimations (4.64) and (4.65) into (4.66) leads to: δ n = O t

Lemma 4.6. 3 .

 3 Let F : R d → R be a convex differentiable function with a L-Lipschitz continuous gradient for some L > 0. Let 0 < γ ≤ 1 L and {x n } n∈N be the sequence generated by Algorithm (11). Assume that F satisfies H(β) and L(p) with p β > 2. Let λ = 2 β-2 and ξ = λ(λ + 1 -b) = 2 β-2 β β-2 -b in the definition of the energy E n . If 131 4.6. Convergence analysis of the i-GD algorithm b β+2

  .79) By relation (B.8) of Lemma B.0.3, we deduce the existence of two positive constants C 1 and C 2 such that for all n n 0 it holds:

  .80) Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 132

  Inertial gradient-descent algorithm under sharpness-flatness conditions 134 By applying Lemma B.0.5 with z n = 2λ(b-1)

Chapter 4 .Figure 4 . 4 :

 444 Figure 4.4: Values of the error log F (x n ) -F (x * ) as a function of the iterations n, for the four minimization problems (Least squares, Quadratic, Tikonov, LASSO). The first three cases fall into the Hypotheses of Theorem 4.5.1, while the fourth (LASSO), into the ones of Corollary 4.5.2.

Chapter 4 .Figure 4 . 5 :

 445 Figure4.5: Each column correspond to the minimization of x p , with p = 3, p = 4 and p = 8 respectively. In the first row, the values of the error log(F (x n ) -F (x * )) as a function of the iterations n. In the second row : here the values of the error F (x n ) -F (x * ) as a function of the iterations n, are in loglog scale. The light-blue curve corresponds to the theoretical bound n -2p p-2 as found in Theorem 4.5.2. Notice that all the slopes obtained by the i-GD algorithm with b ≥ p+2 p-2 (red, green and magenta), except the blue one (b < p+2 p-2 ) and the black one (gradient descent) seem to be identical to the light-blue one (n -2p p-2 ). In the third row : the values of the rescaled error F (x n ) -F (x * ), by ( √ γ(n + b -1))

  basic inequality x 1+x ≤ log (1 + x) ≤ x for all x > -1 and the summation-integral comparison test, we have from the one side: c log n (B.6)where A > 0 is a (renamed at each step) suitable positive constant.From the other side: ≥ A +c log(n+c) ≥ A +c log n (B.7) where A > 0 is a (renamed at each step) suitable positive constant.

  proof of Lemma B.0.3 for a suitable positive constant C > 0.2. If ξ= λ(λ + 1 -b) then : 2γ E n+1 -E n ≤ 2γ c(λ) t n E n +2γ B 1 (λ)t n +1-λ w n +B 2 (λ)δ n + B 3 (λ) t n h n-1 (B.23) where : c(λ) = 2(λ + 1 -b) , B 1 (λ) = (2b -3λ) , B 2 (λ) = λ + 1 -b and B 3 (λ) = 2λ(2λ + 1 -b)(b -λ -1).

  36)By choosing ξ = λ(λ + 1 -b) ( here λ ≥ b -1), we obtain :2γ(E n+1 -E n ) ≤ 2γ k n+1 -2(λ + 1 -b)t n w n + (λ + 1 -b)(2λ + 1 -b) x n -x n-1 2 + 2γ 2(λ + 1 -b) t n E n -2 λ(λ + 1 -b)(2λ + 1 -b) t n x n-1 -x * 2(B.37) By definition of k n+1 (B.69), we obtain :

  38) which allows to conclude the first point (3.39) and the proof of Lemma 3.3.2. 151 B.1. Proof of Lemmas in Chapter 3 Lemma B.1.2. (Lemma 3.3.2). Let F be a function as defined in problem (M). Let also 0 < γ ≤ 1 L , b ∈ (0, 3) and {x n } n∈N the sequence generated by i-FB and E n as defined in (3.34) with λ = 2b 3 , ξ = 2b(3-b) 9

  39)where a = (3-b)(3+b) 9 and c = 2(3-b) 3 . Proof. By choosing λ = 2b 3 in (3.39) of Lemma 3.3.1, we find :2γ(E n+1 -E n ) ≤ 2γ (3 -2b) 3 w n + (3 -b)(3 + b) 9 x n -x n-1 2 -2 2b(3 -b)(3 + b) 27t n x n-1 -x * 2 + 2γ 2(3 -b) 3(n + b -1)E n (B.40) In this point, firstly we express the term x n -x n-1

  42)By injecting the inequality (B.42) into (B.36), we obtain :2γ(E n+1 -E n ) b)(3 + b) 9t 2 n E n + 2γ 2(3 -b) 3(n + b -1) E n (B.43)Therefore we have :2γ(E n+1 -E n ) ≤ 2γ (2b 2 -6b -9) 9 w n -2b(3 -b)(b + 3)n 27(n + b -1) 2 x n-1 -x * 2 + 2γ 2(3 -b)(b + 3) 9(n + b -1) 2 E n + 2γ 2(3 -b) 3(n + b -1) E n = 2γB 1 w n -B 2 n x n-1 -x * 2 (n + b -1) 2 + 2γa (n + b -1) 2 E n + 2γc n + b -1 E n (B.44) where :B 1 = 2b 2 -6b -9 9 < 0 , ∀b ∈ (0, 3) B 2 = 2b(3 -b)(b + 3) 27 > 0 , ∀b ∈ (0, 3) a = 2(3 -b)(b + 3) 9 > 0 , ∀b ∈ (0, 3) c = 2(3 -b) 3 > 0 , ∀b ∈ (0, 3)Hence it follows that for all n ≥ 1 :E n+1 -E n ≤ a (n + b -1) 2 E n + c (n + b -1) E n (B.45) which concludes the proof of Lemma 3.3.2, with a = 2(3-b)(b+3) 9 and c = 2(3-b) 3 . Lemma B.1.3. (Lemma 3.3.3). Let F be a function as defined in problem (M). Let also 0 < γ ≤ 1 L , b ∈ (0, 3) and {x n } n∈N the sequence generated by i-FB and E n as defined in (3.34) with λ = 2b 3 , ξ = 2b(3-b) 9

47) 153 B. 1 .

 1531 Proof of Lemmas in Chapter 3 with a = 4A and c = 2 -2bβ β+2 . Equivalently:

  Lemma B.1.4. (Lemma 3.4.2). Let ε > 0, e ∈ H and γ ≤ 1L . For all (x, y) ∈ H 2 and T ε e (y) ≈ ε j Prox γg y -γ(∇f (y) + e) , for j ∈ {1, 2} as defined in inexact i-FB algorithm 8, we have :2γ F (T ε e (y)) -F (x) ≤ +2γε + 2 γe + r, x -T ε e (y) + y -x 2 -T ε e (y) -x 2 (B.50) where r ∈ H such that r ≤ √2γεProof. By following the same lines of the proof of Lemma B.0.7 (see in particular the first line of (B.17)) and using the definition of the ε-subdifferential of g (Definition 3.66), for all (x, y) ∈ H, we find:F (T ε e (y)) -F (x) ≤ ∇f (y) + ∂ ε g(T ε e (y)), T ε e (y) -x + ε + L 2 T ε e (y) -y 2 (B.51) By definition of T ε e (y) ≈ ε j Prox γg y -γ(∇f (y) + e) , with j ∈ {1, 2}, we have from Lemma 3.4.1 that there exists r with r ≤ √ 2γε and: -e -T ε e (y) + r -y γ ∈ ∇f (y) + ∂ ε g(T ε e (y)) (B.52) Thus, by injecting (B.52), into (B.51), for all (x, y) ∈ H 2 we have that : F (T ε e (y)) -F (x) ≤ y -T ε e (y), T ε e (y) -x + ε + L 2 T ε e (y) -y 2 + e + r γ , T ε e (y) -x (B.53)

(B. 76 )

 76 By choosing ξ = λ(λ + 1 -b), in (B.76), we obtain:2γ(E n+1 -E n ) ≤ 2γ k n+1 -2(λ + 1 -b)t n w n + 2γ 2(λ + 1 -b) t n E n + (2λ + 1 -b)(1 -b) δ n -2λ(λ + 1 -b)(2λ + 1 -b) t n h n-1 (B.69) = 2γ 2(λ + 1 -b) t n E n + 2γ 2b -(β + 2)λ t n + 1 -2λβ(λ + 1 -b) w n + ((2λ + 1 -b)(1 -b) δ n -2λ(λ + 1 -b)(2λ + 1 -b) t n h n-1 = 2γ c(λ) t n E n + 2γ A 1 (λ)t n + 1 -2λβ(λ + 1 -b) w n + A 2 (λ)δ n Proofs of Lemmas in Chapter 4 where: c(λ) = 2(λ+1-b), A 1 (λ) = 2b-(β+2)λ, A 2 (λ) = (2λ+1-b)(1-b), and A 3 (λ) = -2λ(λ + 1 -b)(2λ + 1 -b),which concludes the proof of the Lemma 4.6.1. Lemma B.2.3. (Lemma 4.6.2) Let F : H → R be a convex differentiable function with a L-Lipschitz continuous gradient for some L > 0 and satisfies H(β) with β ≥ 1. Let also 0 < γ ≤ 1 L and {x n } n∈N be the sequence generated by Algorithm (11) and E n as defined in (4.50), with one of the following hypotheses in force: i. b < 1 + 2 β ii. b ≥ 1 + 2 β and F admits a unique minimizer x * and satisfies L(2).

  proof of the first case (b < 1 + 2 β ) of Lemma 3.3.2 with a = 2A and c = 2 -2bβ β+2 .

  Proof. In fact by Lemma 2.1.1 we have that { x γ ∞ } γ>0 and { ẋγ ∞ } γ>0 are uniformly bounded with respect to γ. Since ∇F γ (x) ≤ ∂ 0 F (x) , for all x ∈ R d , where ∂ 0 F (x) denotes the minimal norm-element of ∂F (x) (see Proposition 23.43 in[START_REF] Heinz | Convex analysis and monotone operator theory in Hilbert spaces[END_REF]), by using Lemma A.2.2 we deduce that the family { ∇F γ (x γ ) ∞ } γ>0 is also uniformly bounded with respect to γ. Finally by invoking equation (ADE), we obtain that {ẍ γ } γ>0 is uniformly bounded with respect to γ.

	Proof of Corollary 2.1.1.
	By using the estimations (2.8) and (2.10) we deduce that {x γ

10)

  then from the first point of Corollary 1.3.1 is a nonincreasing function for all γ > 0, i.e. :

	H

γ (t) ≤ H γ (s) for all t 0 ≤ s ≤ t (2.15) Let T > t 0 . By extracting a suitable subsequence when γ → 0 in (2.15), thanks to the approximation scheme (AS), we obtain :

H(t) ≤ H(

s) for a.e. t 0 ≤ s ≤ t ≤ T (2.16) Since T > t 0 is arbitrary, we deduce that H(t) ≤ H(s) for a.e. t 0 ≤ s ≤ t and in particular H(t) ≤ H(t 0 ) for a.e. t ≥ t 0 , which concludes of point 1. of Lemma 2.2.1. In the same way, for b

  we have :

	+∞		+∞	t ẋ(t) 2 dt < +∞
	t 0		t 0
					(2.52)
	In fact:
	lim	t→∞	t ẋ(t) = 0	(2.53)

t F (x(t)) -F * dt < +∞ and t→∞ t 2 w(t) = 0 and lim

  Let F : R d → R be a convex differentiable function with X * = arg min F = ∅. If F satisfies H(β) for some β ≥ 1, then there exists M > 0, such that, for all x in a neighborhood of X * , we have:

	where M = g(1). Therefore by definition of G, we find that for all t ∈ [0, 1],
	it holds :
	g(t) ≤ M |t| β	(4.30)
	Lemma 4.4.3.
	2 g(t) ≥ 0	(4.28)
	Hence if we consider the function G : (0, 1], such that G(t) = |t| -β g(t), from
	(4.28), we deduce that G is non-decreasing in (0, 1]. It follows that for all
	t ∈ (0, 1] it holds :
	G(t) ≤ M	(4.29)

  for example that this is the case when F is a quadratic function, since it satisfies H(2) and L(2).

	Remark 16. In particular from (4.40) of Theorem 4.5.1, when F is satisfying
	H(β) with β	1 and L(2) and b	1 + 2 β , the order of the convergence
	rate for the objective function (i.e. O n -2bβ β+2

  E n (see relation (3.41) of Lemma 4.6.2), while in the case of Theorem 4.5.2, r n = a 1 n 2 E n + a 2 n 2 x n-1 -x * 2 (see relation (4.76) of Lemma 4.6.

	125	4.6. Convergence analysis of the i-GD algorithm
	r n = a n 2
		r n	(4.57)
		with c = -δ + 2 and a suitable sequence {r n } n 1 that involves the geometric
		properties of F . In particular, in the context of Theorem 4.5.1, we have that

  ∈ N, for all l ≥ l * , we have µ l > 2p p-2 , which together with (4.103), leads to a contradiction.

	4.7. Numerical examples
	that after a certain rank l
	.90)

*

  a n (x n -x n-1 ) + λ t n+1 (x n -x * ) 2 ) to the left-hand side and definition of w n , we have :λt n+1 )w n -t 2 n+1 w n+1 ≥ t n+1 (x n+1 -x n ) + λ(x n -x * ) 2 -n(x n -x n-1 ) + λ(x n -x * ) 2 (B.27)which concludes the second point (3.40) of Lemma 3.3.1.For the first point, by definition of E n , we also have:2γE n = 2γt 2 n w n +(λ 2 +ξ) x n-1 -x * 2 +t 2 n x n -x n-1 2 +2λt n x n -x n-1 , x n-1 -x * -x * 2 -2λ x n -x n-1 , x n-1 -x * (B.35) By injecting this last equality into (B.32), we find:

	(B.24) (x n -x * ) 2 (B.25) (x n -x * ) 2 F (x λ By using the convexity of F we obtain : 2γ 1-λ t n+1 F (x n ) + λ t n+1 t n+1 By adding and subtracting 2γF (x 2γ 1 -λ t n+1 w n -w n+1 ≥ x n+1 -x n + λ t n+1 (x n -x * ) 2 -a n (x n -x n-1 ) + λ t n+1 (B.26) By multiplying both sides by t 2 n+1 , we obtain : 2γ (t 2 so that n+1 -(B.34) t n x n -x n-1 2 = 2γ t n E n -2γt n w n -(λ 2 + ξ) t n x n-1

* ) -F (x n+1 ) ≥ x n+1 -x n + λ t n+1 (x n -x * ) 2 -a n (x n -x n-1 ) + *

  n+1 w n+1 ≤ λt n+1 h n -h n+1 + 2λn x n -x n-1 , x n -x * + + 2λn x n -x n-1 , x n -x * -2λt n+1 x n+1 -x n , x n -x *(B.64) On the other hand, by applying (B.13) of Lemma B.0.7, with γ ≤ 1 L and y = y n , x = x n we obtain:2γ F (x n+1 ) -F (x n ) ≤ α 2 n x n -x n-1 2 -x n+1 -x nBy adding and subtracting F (x * ) on the left side and multiplying by t 2 n+1 on both sides, we find:2γt 2 n+1 w n+1 -w n ≤ n 2 δ n -t 2 n+1 δ n+1 (B.66) By adding relation (B.64) to relation (B.66), we obtain: 2γ (t 2 n+1 + λβt n+1 )w n+1 -t 2 n+1 w n ≤ -t 2 n+1 + λt n+1 δ n+1 + n 2 + (B.67) which -by adding and subtracting 2γ t 2 n +λβt n w n on both sides-is equivalent to:2γ (t 2 n+1 +λβt n+1 )w n+1 -(t 2 n + λβt n )w n ≤ 2γk n+1 w n -t 2 n+1 + λt n+1 δ n+1 + n 2 + + 2λn x n -x n-1 , x n -x * -2λt n+1 x n+1 -x n , x n -x * = 2γk n+1 w n -t 2 n+1 + λt n+1 δ n+1 + n 2 + 2λn + + 2λn x n -x n-1 , x n-1 -x * -2λt n+1 x n+1 -x n , x n -x * (B.68) where :k n+1 = t 2 n+1 -λβt n -t 2 n = (n + b) 2 -λβ(n + b -1) -(n + b -1) 2 = (2 -λβ)(n + b -1) + 1 = (2 -λβ)t n + 1 (B.69) By definition of E n (3.34), we also have 2γE n = 2γ t 2 n + λβt n w n + (λ 2 + ξ)h n-1 + t 2 n + λt n δ n + 2λt n x n -x n-1 , x n-1 -x *

							λn 2 t n+1 (B.74) δ n
	so that	(B.61) = -λt n+1 δ n+1 +	λn 2 t n+1	δ n
		t n δ n =	2γ t n	E n -2γ t n + λβ w n -	(λ 2 + ξ) t n	h n-1 -λδ n	(B.75)
			-2λ x n -x n-1 , x n-1 -x *
	By injecting the last equality into (B.73), we find:
	2γ(E n+1 -E n ) ≤ 2γ	2(λ + 1 -b) t n	E	2	(B.65)
							λn 2 t n+1	δ n
							+ 2λn x λn 2 t n+1	δ n
							λn 2 t n+1	δ n

n -x n-1 , x n -x * -2λt n+1 x n+1 -x n , x n -x * n + 2γ k n+1 -2(λ + 1 -b)(t n + λβ) w n

  In this point, firstly we express the term δ n with the aid of E n and w n and then we regroup the different terms.By relation (4.53), for ξ = λ(λ + 1 -b) we find:

			2γ	c t n	E n + 2γAw n + Aδ n +	A 3 t n	h n-1	(B.81)
	2γE n 2γt 2 n w n +	t 2 n 2	δ n -λ(b -1)h n-1	(B.82)
	Hence we have that:
	δ n ≤ 4γ	E n t 2 n	-4γw n +	2λ(b -1) n t 2	h n-1	(B.83)
									c t n	E n + 2γ	2A n t 2	E n
	+	4b(b -1)A (β + 2)t n	+ A 3	h n-1 t n	(B.84)
	≤ 2γ	c t n	E n + 2γ	2A t 2 n	E n +	4b(b -1)A (β + 2)t n	+ A 3	h n-1 t n

By injecting inequality (B.83) into (B.81), for all n 1 we find:

2γ(E n+1 -E n ) ≤ 2γ A -2A w n + 2γ

  By using again the inequalityu -v 2 ≤ 2 u -z 2 + 2 v -z 2 , with u = x n-1 , v = x * and z = x n , in (4.53) we find: 2γE n 2γ t 2 n + λβt n w n + t 2 (Lemma 4.6.3). Let F : H → R be a convex differentiable function with a L-Lipschitz continuous gradient for some L > 0. Let 0 < γ ≤ 1L and {x n } n∈N be the sequence generated by Algorithm (11) and E n as defined in (4.50).Assume that F satisfies H(β) and L(p) with p β > 2. Let λ = 2 -b in the definition of the energy E n . , then there exist n 0 ∈ N and some constant C > 0 such that the following recursive formula holds true for all n n 0 :

	Appendix B. Appendix of Part II			162
	Therefore, for n ∈ N large enough we have:
		2K -1 2	8b(b -1)A (β + 2)t 2 n	+	A 3 t n	-6γA ≤ 0
	which permits to conclude the proof of Lemma 3.3.2 with a = 4A and c =
	2 -2bβ β+2 .
	2γ ≤ 2γAw n + 2γ Lemma B.2.4. β-2 c t n E n + A + 2A 3 t n x n -x n-1 2 + 2A 3 t n x n -x * 2 c t n E n + 2Aδ n + 2A 3 t n x n -x * 2 (B.86) n 1 2 + λ t n -2λ(b -1) t 2 n δ n -2λ(b -1)h n β and ξ = λ(λ + 1 -b) = 2 β-2 β+2 β-2 If b β-2 2γ E n+1 -E n ≤ 2γ -4 β -2 + C t n E n t n + C n t 2 h n-1 . (B.91)
	2γt 2 n w n + Proof. By letting λ = 2 t 2 n δ n -2λ(b -1) x n -x * 2 β-2 in (4.61) of Lemma 4.6.1, we find: 2 Hence for n ∈ N large enough we have 2γ E n+1 -E n ≤ 2γ 2 β β -2 -b E n t n + 2γ B 1 t n + 1 -4β (B.87) β -2 β -2 β	-b w n
	δ n ≤ 4γ	E n t 2 n + B 2 x n -x n-1 -4γw n + 4λ(b -1) t 2 2 + B 3 x n -x * 2 x n-1 -x * 2 t n n	(B.88)	(B.92)
	By injecting the last inequality (B.88) into (B.86) we find: where:
	2γ(E n+1 -E n ) ≤ 2γ A -4A w n + 2γ + 2 8b(b -1)A (β + 2)t n + A 3 c t n	E n + 2γ t n x n -x * 2 4A t 2 n	E n	(B.89)
	By using Hypothesis L(p) with p = 2 and the uniqueness of the minimizer
	in inequality (B.89), we find:
	2γ(E n+1 -E n ) ≤ 2γ + 2K -1 c t n E n + 2γ 2 8b(b -1)A 2(β + 2)A t 2 n (β + 2)t 2 n + E n t n A 3	-6γA w n	(B.90)

2

  By definition of E n (3.34) we have:2γE n = 2γ t 2 n + λβt n w n + v n + λt n δ n + λ(λ + 1 -b)hHence by definition of B 1 and B 3 and relation (B.93), we find:

	163							B.2. Proofs of Lemmas in Chapter 4
	2γB 1 t n w n +	B 3 t n	h n-1 = 2γ	B 1 E n t n	-2γλβB 1 w n -λB 1 δ n -B 1	v n t n	(B.94)
	By injecting the last inequality (B.94) into (B.92) and omitting the non-
	positive term -B 1	vn tn , we find:
			2γ E n+1 -E n ≤ -2γ	d t n	E n + 2γB 1 w n + B 2 δ n	(B.95)
	where
	d =	4 β -2	, B 1 = 1 -	4β β -2	β β -2	-b -λβB 1 =	β + 2 β -2
						δ n ≤ 4γ	E n t 2 n	-4γw n +	2λ(b -1) n t 2	h n-1	(B.97)
	Hence by injecting (B.97) into (B.96) we obtain:
	2γ E n+1 -E n ≤ -2γ	d t n	E n + 2γ	2B t 2 n	E n -2γB w n +	4(b -1)B h n-1 (β -2)t 2 n
						≤ -2γ	d t n	E n + 2γ	C t 2 n	E n +	Ch n-1 t 2 n
								(B.98)
	which concludes the proof of Lemma 4.6.3 with C = max{2B , 4(b-1)B β-2 } > 0.
								n-1	(B.93)
		where λ = 2 β-2 .

2

and

B 2 = B 2 -λB 1 = b -β + 2 β -2 2 By choosing B = max B 1 , B 2 , from (B.95) we infer that: 2γ E n+1 -E n ≤ -2γ d t n E n + 2γB w n + B δ n (B.96)

By relation (4.53) (recall that λ = 2 β-2 and ξ = λ(λ + 1 -b)), for n ∈ N large enough, we have that:

Σαν τους αλγόριθμους που μελετάω.

of F (x(t)) to the minimum F * when 0 < b < 3, is optimal up to constants for the class of

3.3. Convergence results for i-FB algorithm

See Definitions 4.1 and 4.1 later on, for the exact conditions L(p) and H(β), made on the minimizing function F .

√ L-√ µ √ L-√ µ < 1 -µ L .Nevertheless, in both of two cases of Algorithms 9

Part I

The continuous setting

Appendix of Part I

Here we give the continuous version of Opial's lemma (for more details see [START_REF] Opial | Weak convergence of the sequence of successive approximations for nonexpansive mappings[END_REF] or Lemma 4.1 in [START_REF] Attouch | The heavy ball with friction method, i. the continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system[END_REF]).

Lemma A.0.1 (Opial's Lemma). Let I ⊂ R + be an interval, H a separable Hilbert space and S ⊂ H be a non-empty set and a mapx : I -→ H, such that the following conditions hold true:

Every weak-cluster point of x(t) belongs to S

Then we have that x(t) converges weakly to a point of S as t → +∞.

Remark 7. In the context of the current Thesis, we invoke Opial's Lemma with S = arg min F , in order to prove the weak convergence of x(t) to a minimizer x * ∈ arg min F . If in addition H = R d , d ≥ 1, we also deduce strong convergence of x(t) to x * .

A.1 The Yosida approximation

For a positive parameter γ > 0 and a maximally monotone operator A, one can define the resolvent of A and the Yosida approximation of A by J γA and A γ respectively as follows :

Let Φ : R d -→ R be a proper, lower semi-continuous and convex function and ∂Φ its subdifferential. Then ∂Φ is a maximally monotone operator and

Part II

The discrete setting

At this point we consider the different disjoint cases for the parameters (β, p) in order to precise the estimate (4. -If M = m -4 p-2 , then from (4.90) we find:

, thus M = 0 and from relation (4.90), we find :

The previous cases can be regrouped into two regimes B 1 and B 2 , for the parameters (β, p) ∈ (x, y) ∈ R 2 : 2 < x y with:

such that: 135 4.6. Convergence analysis of the i-GD algorithm

• If (β, p) ∈ B 1 then from (4.90) we obtain:

• If (β, p) ∈ B 2 then from (4.90) we obtain:

In the case when (β, p) ∈ B 1 , we can conclude directly the proof of Theorem 4.5.2.

Let us now treat the case of (β, p) ∈ B 2 . In this case the estimate found in (4.92) is sub-optimal, in comparison with the one stated in Theorem 4.5.2. This point is also strongly accented by the corresponding results for the continuous-time version (see Theorem 4.3 in [START_REF] François | Optimal convergence rates for Nesterov acceleration[END_REF]). This is due to the use of the a-priori estimate for (t 2 n w n-1 ) n used in (4.83), in our analysis. Nevertheless, we show that this estimate can be "improved" by inferring a bootstrap argument for a suitable amount of times. More precisely the idea is to use (4.92) as an a-priori estimate, by re-injecting it in (4.82). This idea is presented in the third part.

Part 3: First we define the sequences {µ l } l∈N , {m l } l∈N and {M l } l∈N , with µ 0 = 2, such that for all l 1 it holds:

and for all l ∈ N:

For all l ∈ N, we also define the following family of conditions B 2 (l), for the parameters (β, p):

Since B 2 (0) is in force, by relation (4.92) we have that:

Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 136

Hence, by using the hypothesis L(p) and the uniqueness of the minimizer, we find that:

By following the same procedure as before in Part 1 and injecting the inequality (4.97) into (4.81), we find :

for some suitable positive constants C 2 and C 3 and n 0 ∈ N.

Then, by using the series-integral comparison test in (4.98), we find:

By proceeding exactly in the same way as before in Part 2 of the current proof, one can deduce that:

If we suppose that B 2 (1) does not hold true (i.e. M 1 = m 1 -4 p-2 ), then the result of Theorem 4.5.2 follows directly from relation (4.100). If in the contrary B 2 (1) is in force, then from (4.100), it follows that:

In fact, in the same way as before, by a recurrence argument, we find that for all l 0 it holds:

and if B 2 (l) holds true, then :

Let us prove by contradiction that B 2 (l) cannot hold true for all l ∈ N. For that we suppose that the condition B 2 (l), holds true for all l ∈ N. In that case, since M l = 0 for all l ∈ N, we have m l < 4 p-2 which is equivalent to:

and µ l = (d -1)p 2 (4.103) Notice in that case that µ l is always an increasing sequence, converging to its supremum: 3p p-2 . In fact one can assure that there is at most one

p-2 , thus µ l+1 = 3 + 2 p µ l for all l l 1 . This entails

Concluding remarks and perspectives

In this Section we studied the Nesterov-type inertial Algorithm 11, for smooth and convex minimization problems, with some additional geometrical information over the minimizing function F . We showed that, unlike descent schemes under Growth condition L(p) for p ≥ 2, Algorithm 11, necessitates also a flatness type condition H(β), in order to deduce competitive convergence rates. In particular from Theorem 4.5.1, one can deduce that for functions with sharp enough geometry (i.e. L(2)) the inertial Gradient descent algorithm may lead to sublinear rates, in comparison to the linear rates obtained by a descent method. Nevertheless notice that for a given precision, choosing b large enough can even lead to better results than the linear convergence proven for the Gradient Descent or Heavy-Ball-type algorithms. This question is directly linked with the dependence on b, of the hidden constants in the "big Ohs", in Theorems 4.5.1 and 4.5.2. The exact knowledge of this dependence on b, can trace then the strategy of choosing (or not) a particular inertial algorithm, rather than a Gradient-Descent for minimizing F up to a given precision. Unfortunately our analysis does not allow to have an explicit formulation of these constants, which is let for future study.

On the other hand, Theorem 4.5.2 shows that for functions with flatenough behavior (i.e. H(β) and L(p) with p ≥ β > 2), the i-GD algorithm 11, performs better than a first-order descent method by factor of 2 (notice this difference between (4.47) of Theorem 4.5.2 and (4.13) of Theorem 4.2.1). In fact in the case of flat geometries (i.e. p > 2), it would be interesting to know if the flatness hypothesis H(β) is necessary for the rates proven in Theorem 4.5.2, or if we can get around with the same rates without this assumption. In addition the hypothesis of the uniqueness of the minimizer x * in Theorem 4.5.2, seems motivated more by a technical necessity, rather than a "counterexample"-intuition. This arises the question if this uniqueness-hypothesis is necessary or it can be omitted.

Another challenging task, is to extend the results of this Chapter to the inertial Forward-Backward Algorithm 7, for non-smooth convex optimization. In particular, this question is translated to extending the flatness condition H(β) for a sum of two convex functions in Lemmas B.0.7 and B.2.1. In this setting, we can further investigate the convergence properties of the i-FB algorithm 7, for convex functions that satisfy the sharpness condition L(p), for 1 ≤ p ≤ 2, as also for non-convex ones.

Moreover, we let for future study, the question of the robustness of the convergence results in Theorems 4.5.1 and 4.5.2, in the presence of perturba-Chapter 4. Inertial gradient-descent algorithm under sharpness-flatness conditions 142 tion errors in the computation of the gradient of F . As we expect, the rates found in Theorems 4.5.1 and 4.5.2 remain robust, under some summabilitycontrol conditions over the error-terms, depending on the geometry of the function (i.e. parameters β and p).

Finally an interesting question consists in proving the optimality of the rates obtained in Theorems 4.5.1 and 4.5.2. In particular, as the analysis in the continuous case for the system (1.18) shows, as also the numerical examples (see Figure 4.5), the rates found in Theorem 4.5.2, seem to be -worst case-optimal for functions such as x p , for p > 2.

Appendix of Part II

In this appendix we are giving the proofs of the different technical Lemmas used in Chapters 3 and 4.

We begin by giving the discrete version of Opial's Lemma ( [Opi67] or Lemma 4.1 in [AGR00]) : Lemma B.0.1. Let K be a separable Hilbert space and S ⊂ K be a non-empty set and x : [t 0 , +∞) such that the following conditions hold:

Every weak-cluster point of x n belongs to S

Then the sequence x n weakly converges to a point of S as n → +∞.

Remark 19. Usually we invoke the previous Lemma with S = arg min F .

The next Lemma is a discretized version of Gronwall's Lemma ( see for example Theorem 4 in [START_REF] John M Holte | Discrete Gronwall lemma and applications[END_REF] or Lemma 1 in [START_REF] Schmidt | Convergence rates of inexact proximal-gradient methods for convex optimization[END_REF] ).

Lemma B.0.2. Let C 0 a positive real number and {u n } n∈N , {u n } n∈N and {a n } n∈N three non-negative sequences such that a n = 0 for all n ≥ 1 and:

Then for all n ≥ 1 it holds:

In particular if C 0 ≥ 0, such that for all n ≥ 1 it holds :

The next Lemma is a discretized version of Gronwall's-Bellman's Lemma ( see for example Lemma 1 in [START_REF] Schmidt | Convergence rates of inexact proximal-gradient methods for convex optimization[END_REF] ). Lemma B.0.4. Let C 0 a positive real number and {u n } n∈N , {a n } n∈N two non-negative sequences, such that for all n ∈ N * it holds

where {S n } n∈N is a non-decreasing sequence such that u 2 1 ≤ S 1 . Then for all n ≥ 1, it holds :

Lemma B.0.5. Let C > 0 a positive real number, α ∈ (0, 1) and {u n } n∈N , {z n } n∈N two non-negative sequences, such that for all n ∈ N * it holds

Then for all n ∈ N * it holds:

Proof. Let n ∈ N * . We split the proof in two cases:

• Firstly we suppose that

≥ 1 2 , hence by using relation (B.10), we find:

Next, we give a basic characterization property (optimality condition) for the operator T γ , as defined in (3.6): Lemma B.0.6. Let γ > 0 and F = f + g as in (M). Let x ∈ H. Then p = T γ (x) = Prox γg x -γ∇f (x) if and only if :

Next we give a basic descent-type lemma for a function F , concerning the operator T γ (x) defined in (3.6) (see also Lemma 2.2 in [START_REF] Beck | A fast iterative shrinkagethresholding algorithm for linear inverse problems[END_REF] or Lemma 1, in [START_REF] Chambolle | On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm[END_REF]). This will play a fundamental role on the Lyapunov analysis for both of Chapters 3 and 4. Lemma B.0.7. Let γ > 0 and F = f + g with f convex, differentiable function with L-Lipschitz gradient and g ∈ Γ 0 (H). For every (x, y) ∈ H and T γ (y) = Prox γg y -γ∇f (y) , we have that:

Proof. Using the fact that ∇f is L-Lipschitz, for all (z, y) ∈ (H) 2 , one can obtain (see for example [START_REF] Bertsekas | Nonlinear programming[END_REF] or [START_REF] Heinz | Convex analysis and monotone operator theory in Hilbert spaces[END_REF]):

which -by adding g(z) on both sides-is equivalent to :

Letting z = T γ (y) and subtracting F (x) on both sides, for all (x, y) ∈ H 2 we have:

(B.16) By using the convexity of f and g, we have :

(B.17) where in the first equality we used the characterization (B.0.6) of the operator T γ and in the second equality Pythagoras identity:

By multiplying the last relation by 2γ we obtain (B.13).

B.1. Proof of Lemmas in Chapter 3

Remark 20. By choosing γ ≤ 1 L in Lemma B.0.7, it is direct that from relation (B.13) we obtain :

By using the previous Lemma, we are able to deduce the following descent property for a sequence generated by the i-FB Algorithm 7.

Lemma B.0.8. Let γ > 0 and F = f + g with f convex, differentiable function with L-Lipschitz gradient and g ∈ Γ 0 (H) and x * ∈ arg min F . Let also {x n } n≥1 be the sequence generated by the i-FB Algorithm 7. Then the energy-sequence

Proof. It suffices to apply relation (B.19) of Lemma B.0.7, with γ 1 L , y = y n and x = x n , in order to find:

By adding and subtracting F (x * ) in the left side (B.20) and rearranging the terms we find:

n+b ≤ 1, for all n ≥ 1 , from (B.21), we deduce that U n+1 U n , which concludes the proof.

B.1 Proof of Lemmas in Chapter 3

Here we give the proofs of the Lemmas used in Chapter 3. For the sake of readability we recall the statements of the Lemmas.

Lemma B.1.1. (Lemma 3.3.1). Let F be a function defined as in problem (M).

Let also b > 0, {x n } n∈N the sequence generated by i-FB algorithm 7 and 0 ≤ λ ≤ b + 1 and E n as defined in 3.34. Then the following recursive formulas hold true for all n ≥ 1 :

B.1. Proof of Lemmas in Chapter 3

By adding and subtracting 2γt 2 n w n to the left-hand side we obtain :

30) Hence by using the last inequality and the identity

and the definition of E n we have that :

32) Hence by choosing ξ = λ(b -λ -1) in (B.32), and the definitions of k n+1 and t n , we find :

The result of the Lemma follows from the fact that γ ≤ 1 L and using the following basic equality : 2 y -T ε e (y), x -T ε e (y) + T ε e (y) -y 2 = y -x 2 -T ε e (y) -x 2 (B.54)

B.2 Proofs of Lemmas in Chapter 4

First we give a basic descent-type lemma for the function F , similar to Lemma B.0.7 in the case of i-GD algorithm 11 concerning the operator T γ (x) := x -γF (x) with a function F that verifies H(β), β ≥ 1.

Lemma B.2.1. Let γ > 0 and F satisfying H(β) with β 1. For all y ∈ H and x * ∈ X * it holds:

Proof. In fact by proceeding exactly in the same way as in proof of Lemma B.0.7, with g = 0, we find :

By using hypothesis H(β) we obtain:

(B.57) By using that γ∇F (y) = y -T γ (y) and Pythagoras identity, we have: L and {x n } n∈N the sequence generated by the i-GD algorithm (11) and E n as defined in (4.50). Then for all λ ≥ 0 and ξ = λ(λ + 1 -b) in the definition of E n , the following recursive formula holds for all n ≥ 1:

Proof. For this proof we will frequently make use of the following basic identity:

Firstly, by applying (B.59) of Lemma (B.0.7) with γ ≤ 1 L and y = y n , x = x * we obtain:

which by multiplying by λβt n+1 > 0, developing the term x n + α n (x nx n-1 ) -x * 2 , is equivalent to: In addition, by developing the squares in the definition of v n+1 and v n , we have: