Enfin

Mercie Aurore

Benoit Camille Benjamin

Carlos

Elinirina, Emilien, Evrard, Ioannis, Léon, Marlène, Parissay Sergio David

These last three years have been filled up with many beautiful moments, involving love, work, passion and really important people. Let me devote some words to thank them all, by adopting the appropriate language.

Je souhaite tout d'abord remercier mon directeur de thèse Emmanuel Trélat et mon encadrant Bruno Hérissé. Ce travail n'aurait certainement pas

Introduction Générale

il a fallu attendre la fin des années 50 pour que les contributions de l'école russe de mathématiques fournissent les fondements théoriques pour le contrôle optimal de systèmes très généraux. Le travail novateur de Lev Pontryagin et de ses étudiants a apporté des conditions nécessaires d'optimalité puissantes, connues sous le nom du Principe de Maximum de Pontryagin (ou Principe de Maximum, ou encore PMP, voir, e.g. [START_REF] Pontryagin | Mathematical theory of optimal processes[END_REF]), posant les bases de conditions plus exhaustives (voir, e.g. [START_REF] Hestenes | Calculus of variations and optimal control theory[END_REF][START_REF] Clarke | The maximum principle under minimal hypotheses[END_REF][START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF][START_REF] Sussmann | A maximum principle for hybrid optimal control problems[END_REF][START_REF] Li | Optimal control theory for infinite dimensional systems[END_REF]).

Dans le même temps, l'avancée de l'informatique et des méthodes numériques a permis de résoudre concrètement presque tout problème de contrôle optimal soumis par les applications et l'industrie, à l'exception d'erreurs numériques. Aujourd'hui, un large éventail d'algorithmes existe pour déterminer la solution de problèmes de contrôle optimal. Nous pouvons les classer en deux domaines principaux: méthodes directes et indirectes (voir, e.g. [START_REF] Bryson | Applied optimal control: optimization, estimation and control[END_REF][START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF][START_REF] Rao | A survey of numerical methods for optimal control[END_REF] pour plus de détails sur ces procédures).

Les méthodes directes consistent à discrétiser les variables du problème de guidage optimal (l'état, le contrôle, etc.) pour le réduire à un problème d'optimisation nonlinéaire sous contraintes. Un haut degré de robustesse est garanti et, de manière générale, aucune connaissance du système dynamique n'est nécessaire, entraînant une utilisation simple de ces méthodes en pratique. Cependant, leur efficience dépend fortement des capacités de calcul, impliquant une utilisation hors ligne le plus souvent. Par ailleurs, les méthodes indirectes consistent à appliquer les conditions données par le Principe de Maximum en transformant le problème de guidage optimal en un problème aux valeurs limites. Les avantages de ces méthodes, dont la version basique est connue comme étant la méthode de tir, sont leur précision numérique extrêmement élevée et leur convergence rapide, si elle est atteinte. En revanche, l'initialisation de méthodes indirectes reste une tâche ardue.

Parmi les études menées par l'ONERA (Office National d'Etudes et de Recherches Aérospatiales), le guidage optimal de véhicules autonomes a acquis une position importante, à travers des applications civiles et militaires (lanceurs Ariane, systèmes de missiles, véhicules aériens sans pilote, etc.).

La plupart des travaux considèrent des calculs hors ligne à l'aide de méthodes directes pour assurer une convergence robuste vers la solution optimale. Cependant, lorsqu'on traite de problèmes dynamiques, comme, par exemple, l'interception de cibles manoeuvrantes, des trajectoires évaluées au préalable peuvent perdre rapidement leur optimalité, ou pire, mener à l'échec de la mission. Face à cela, une réévaluation rapide des strategies optimales est nécessaire pour la réussite de la mission.

Introduction Générale

Objectifs et Contributions de ce Travail

Ce travail est né d'une collaboration entre l'ONERA et le Laboratoire Jacques-Louis Lions (Sorbonne Université). L'objectif est de concevoir un algorithme autonome pour la prédiction de strategies de contrôle optimal concernant des missions à destination de véhicules lanceurs endo-atmosphériques. L'algorithme doit se baser sur des méthodes indirectes, et être capable de s'adapter à tout changement imprévu de scénario, en temps réel. L'intérêt principal se trouve dans les missions d'interception.

Le choix des méthodes indirectes dérive du fait qu'elles sont bien adaptées pour apporter une forte précision numérique et une rapidité de calculs, idéales pour des opérations embarquées. Un autre point important consiste à assurer la mise en place de strategies pour chaque scenario potentiellement réalisable. De plus, la robustesse des algorithmes concernés doit être assurée au moins statistiquement.

Pour atteindre cet objectif, nous avons identifié les étapes et contributions suivantes:

1. Appliquer le Principe du Maximum pour des stratégies globales.

Une étude précise du guidage optimal de véhicules lanceurs nécessite de prendre en compte de lourds critères de performance, ainsi que des missions difficiles. Etant donné que dans cette situation le véhicule est sujet à de nombreux efforts mécaniques, differentes contraintes de stabilité doivent être imposées, faisant ainsi apparaître des contraintes mixtes, c'est-à-dire, combinant à la fois des variables de contrôle et d'état. Ce type de problème de contrôle optimal est difficile à traiter par le Principe de Maximum (voir, e.g. [START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF][START_REF] Clarke | Optimal control problems with mixed constraints[END_REF]). En effet, d'autres multiplicateurs de Lagrange apparaissent, pour lesquels, obtenir des informations rigoureuses et utiles peut être compliqué, cela a été l'objet de multiples études (voir, e.g. [START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF][START_REF] Bonnans | Well-posedness of the shooting algorithm for state constrained optimal control problems with a single constraint and control[END_REF][START_REF] Arutyunov | The maximum principle for optimal control problems with state constraints by R.V. Gamkrelidze: revisited[END_REF]). Cette complication empêche de pouvoir considérer des méthodes indirectes basiques, compromettant une convergence rapide vers les solutions optimales. L'approche largement diffusée en aéronautique, permettant d'éviter de traiter des contraintes mixtes, consiste à reformuler le problème original de guidage en utilisant des coordonnées locales d'Euler, sous lesquelles, la contrainte de stabilité devient une contrainte pure de contrôle (voir, e.g. [START_REF] Bonnard | Singular trajectories and their role in control theory[END_REF]). Cette transformation permet de considérer le Principe de Maximum classique, donc, les méthodes usuelles de tir. En revanche, les coordonnées d'Euler ne sont pas globales et ont des singularités, empêchant la resolution de toutes configurations réalisables, et réduisant le nombre de missions achevables.

La solution que nous proposons consiste à reformuler le problème de guidage

Introduction Générale

optimal dans un point de vue intrinsèque, en utilisant le contrôle géometrique (il n'apparaît pas que ce cadre ait été systématiquement étudié dans le contexte du guidage optimal). En particulier, nous construisons des coordonnées locales supplementaires, couvrant les singularités de ces dernières, sous lesquelles la contrainte mixte peut être toujours réinterprétée comme une contrainte pure de contrôle. De plus, ces deux ensembles de coordonnées locales constituent un atlas de la variété en question, que nous utilisons aussi pour découvrir, en exploitant des outils de contrôle géométrique, le comportement complet des extrémaux de Pontryagin, aussi dans le cas d'arcs non réguliers.

L'introduction de ces coordonnées locales particulières engendre deux bénéfices principaux. D'un côté, il n'y a pas de limite aux scénarios faisables pouvant être simulés. D'un autre côté, le problème de guidage optimal n'est pas conditioné par des multiplicateurs dépendants de contraintes mixtes (ou, au moins, localement), donc, les méthodes indirectes classiques peuvent être facilement mises en pratique. C'est au prix du changement de coordonnées locales, qui complexifie légerement l'implementation de la méthode de tir, mais, n'influence pas de manière importante la vitesse et la précision computationelles.

Concevoir un chemin numérique efficace basé sur des méthodes indirectes.

La procédure détaillée précedemment nous permet d'appliquer les méthodes indirectes pour résoudre des problèmes de guidage optimal. Cependant, comme il a déjà été relevé, même si les méthodes indirectes ont une très bonne précision numérique, leur défaut principal reste leur initialisation. Cette problématique est bien connue dans la communauté aérospatiale et beaucoup d'efforts ont été faits pour permettre des procédures d'initialisation efficaces (on trouve un résumé approfondi de ces techniques dans la revue [START_REF] Trélat | Optimal control and applications to aerospace: some results and challenges[END_REF]). Le but étant de garder l'algorithme de résolution indépendant des méthodes directes, nous souhaitons éviter l'utilisation de ces dernières.

Nous proposons une procédure d'initialisation efficace pour méthodes indirectes par l'emploi des méthodes d'homotopie (voir, e.g. [START_REF] Allgower | Introduction to numerical continuation methods[END_REF]). Récemment, ces schémas numériques ont acquis une bonne réputation dans les applications aérospatiales, principalement grâce à leur haute fiabilité et polyvalence (voir, e.g. [START_REF] Graichen | A continuation approach to state and adjoint calculation in optimal control applied to the reentry problem[END_REF][START_REF] Hermant | Optimal control of the atmospheric reentry of a space shuttle by an homotopy method[END_REF][START_REF] Zhu | Minimum time control of the rocket attitude reorientation associated with orbit dynamics[END_REF]). L'idée de base des méthodes d'homotopie est de résoudre un problème difficile, étape par étape, en commençant par un problème simple (usuellement appelé problème d'ordre zéro), par déformation de paramètre. Associé au tir provenant du Principe de Maximum, une méthode d'homotopie consiste à déformer le problème aux valeurs limites en un plus simple (qui peut être facilement résolu), puis, résoudre une série de tirs, étape par étape, en remontant au système original. Dans le cas où le paramètre d'homotopie est un nombre réel et le chemin est constitué d'une combinai-

Introduction Générale

son convexe du problème d'ordre zéro et original, la méthode d'homotopie est plutôt appelée méthode de continuation.

Les méthodes d'homotopie numériques sont composées par deux principales étapes: le choix d'un problème d'ordre zéro (qui devrait être le plus facilement résoluble possible) et d'une procédure de déformation de paramètre. Nous résolvons le problème de guidage optimal en adoptant le schéma suivant: 2.a) Concevoir et résoudre le problème d'ordre zéro par une loi explicite.

Des tests expérimentaux démontrent que le problème de guidage optimal simplifié, obtenu en retirant les contributions de la poussée et de la gravité du modèle dynamique initial, conserve des régularités suffisantes, afin, qu'une fois que ce problème d'ordre zéro soit déterminé, une déformation de paramètre appropriée permet une convergence rapide de la procédure d'homotopie. En outre, l'initialisation des méthodes de tir sur ce problème de guidage optimal simplifié peut être faite analytiquement et instantanément. En effet, en manipulant le Principe du Maximum appliqué à ce problème, nous fournissons une nouvelle loi de guidage capable d'initialiser efficacement des méthodes indirectes sur le problème d'ordre zéro précédent, pour un large éventail de scénarios.

2.b) Concevoir un schéma approprié de déformation de paramètre.

Le problème original de guidage optimal est résolu en déformant le problème d'ordre zéro précédent, par un ajout itératif de la poussée et de la gravité précédemment retirées. Nous choisissons un scénario modifié capable d'initialiser analytiquement le problème simplifié (voir 2.a)) en le gardant fixe pendant cette étape d'homotopie. Par la suite, une dernière étape d'homotopie modifie le scénario temporaire pour obtenir la solution de l'intégralité du problème considérant le scenario original.

Lorsque le scénario implique que la trajectoire optimale rencontre des singularités d'Euler, dans le schéma numérique précédent, les calculs sont temporairement arrêtés et un changement de coordonnées est opéré (voir 1.). A partir de cela, l'intégration numérique reprend en évitant des échecs de convergence. Ce changement de coordonnées affecte peu le temps computationnel total, en maintenant le taux classique de convergence des méthodes indirectes.

Introduction Générale

En effet, toute la procédure d'homotopie décrite en 2., qui commence du problème simplifié et finit avec l'étape de déformation du scénario, peut contenir plusieurs itérations homotopiques (dépendant de la difficulté de la mission). Cela implique que différentes missions peuvent prendre plus ou moins de temps computationnel pour converger vers la solution optimale. En d'autres termes, le temps computationnel moyen de ce schéma d'homotopie peut être trop long pour une implémentation en temps réel.

Comme solution, nous proposons de traiter le probème lié à la haute sensibilité, et donc améliorer la robustesse du schéma d'homotopie, en utilisant une grille raffinée de guesses initiaux, évaluée hors ligne, qui contient les solutions du problème complet original (poussée et gravité incluses) pour plusieurs scénarios réalisables. Puis, la résolution de toutes missions arbitraires procède de la manière suivante. Premièrement, la grille est chargée dans la RAM (hors ligne). En suite, le scénario le plus proche (par rapport à une certaine métrique) de celui fourni par l'utilisateur est sélectionné à partir de la grille d'initialisation. Finalement, une déformation spatiale est mise en place pour récupérer la solution du problème original. Plus la grille est fine, meilleure est la chance d'obtenir une solution optimale globale.

Nous développons cette amélioration dans le contexte de l'interception optimale. Des tests statistiques montrent que seulement peu d'itérations homotopiques sont nécessaires en général pour obtenir une solution, donc, de hauts taux de convergence sont assurés. De plus, comme la réévaluation de strategies d'interception est assez élevée, la robustesse des solutions par rapport aux variations extérieures du scénario initial, comme des cibles mouvantes, augmente considérablement. 4. Caractère bien posé des méthodes d'homotopie pour les problèmes de contrôle optimal avec retards, lorsque l'homotopie est opérée sur les retards.

Les problèmes de guidage optimal se focalisent sur le contrôle de la dynamique du centre de gravité des véhicules lanceurs, en évitant de traiter le contrôle des configurations du corps rigide. Plusieurs raisons sont invoquées pour considérer séparément le problème de guidage et les manoeuvres réalisées par le pilote. Celle faisant l'unanimité atteste que, généralement, le système de guidage opère à une fréquence plus faible que celle du pilote. En revanche, certains retards apparaissent dans le suivi des consignes. Pour améliorer le modèle et les lois de contrôle, on peut étendre le problème de guidage optimal en approximant l'action du pilote dans les équations de mouvement, et en considérant des retards entre la translation et la rotation (voir, e.g. [6,[START_REF] Hanson | A plan for advanced guidance and control technology for 2nd generation reusable launch vehicles[END_REF]).

Ce contexte oblige à gérer des problèmes de contrôle optimal non linéaires avec retards et le défi consiste à résoudre efficacement ces problèmes par des

Introduction Générale

algorithmes de tir. Cette question est habituellement complexe et computationnellement coûteuse. En effet, le Principe de Maximum pour problèmes de contrôle optimal avec retards (voir, e.g. [START_REF] Kharatishvili | The maximum principle in the theory of optimal processes involving delay[END_REF][START_REF] Halanay | Optimal controls for systems with time lag[END_REF][START_REF] Banks | Necessary conditions for control problems with variable time lags[END_REF][START_REF] Göllmann | Theory and applications of optimal control problems with multiple time-delays[END_REF]) conduit à des problèmes aux valeurs finales qui combinent des termes antérieurs et postérieurs du temps, en obligeant une intégration globale des équations différentielles annexes. Un guess local concernant l'extrémale optimale n'est plus utile, un guess global doit plutôt être fourni pour permettre la convergence de la procédure. Cela représente une autre difficulté par rapport aux méthodes de tir usuelles, et affecte clairement les performances computationnelles.

Comme les méthodes précédemment adoptées permettent de résoudre le problème de guidage optimal sans retards, il semble légitime de se demander si on peut résoudre le problème de guidage optimal contenant des retards par des méthodes indirectes, en commençant une procédure d'homotopie dans laquelle le délai représente le paramètre de déformation, et le problème sans retards est considéré comme le problème d'ordre zéro. Cette approche est un moyen possible pour traiter le défaut des méthodes indirectes appliquées aux problèmes de contrôle optimal avec retards: d'une part, l'information globale donnée par le problème sans aucun retard pourrait être utilisée pour initialiser efficacémment une méthode de tir avec retards; d'autre part, nous pourrions résoudre le problème aux valeurs limites avec retards à l'aide de méthodes itératives locales pour équations différentielles. Nonobstant, différemment du contexte classique du contrôle optimal sans retards, pour lequel, sous des hypothèses appropriées, la convergence des méthodes d'homotopie est comprise et bien établie (voir, e.g. [START_REF] Trélat | Optimal control and applications to aerospace: some results and challenges[END_REF]), le caractère bien posé des méthodes indirectes combinées avec des procédures d'homotopie sur le retard n'a pas encore été bien traité dans la littérature. En particulier, des points de bifurcation ou des singularités peuvent apparaître, entraînant des échecs de convergence.

La solution que nous proposons réside sur une étude détaillée du caractère bien posé des procédures d'homotopie pour résoudre des problèmes généraux de contrôle optimal non-lineaires avec retards, par des méthodes indirectes. Notre principal resultat théorique est que, sous des hypothèses appropriées, les quantités fournies par le Principe de Maximum, incluant les vecteurs adjoints et les trajectoires, sont continues par rapport aux retards. Ce résultat assure le caractère bien posé du schéma numérique proposé précédemment: tout chemin d'homotopie sur les retards converge vers les extrémaux de Pontryagin du problème de contrôle original avec retards, si on part du problème sans aucun délai. Une fois ces propriétés de continuité établies dans un contexte général, nous exploitons ce résultat pour fournir efficacement une résolution numérique pour problèmes de guidage optimal avec retards.

Les contributions précédentes ont rendu possible le développement d'un logiciel

Introduction Générale

complètement automatique, indépendant et auto-régulé, aujourd'hui propriété de l'ONERA, pour des applications réalistes dans le cadre de véhicules lanceurs, focalisé, en particulier, sur des scénarios d'interception optimal. En une seule seconde, l'algorithme est capable de fournir des strategies optimales pour onze missions différentes avec juste un processeur de quelque mégabyte de mémoire, permettant des calculs en temps réel, idéal pour des véhicules autonomes.

Organisation du Manuscrit

Ce travail s'organise en sept chapitres et une conclusion finale.

Les chapitres 1 et 2 ont comme objectif d'introduire au lecteur la théorie classique du contrôle optimal et les problèmes de guidage sur lesquels cette étude se concentre. Dans un même temps, ces chapitres fournissent les notations et résultats théoriques standards utilisés tout au long de ce manuscrit. Dans le Chapitre 3, nous dérivons la structure des extrémaux de Pontryagin du problème de guidage optimal. Un contexte abstrait introduisant les conditions de consistance des vecteurs adjoints sous transformations locales est d'abord présenté. Le résultat est alors appliqué pour une étude correcte des contrôles réguliers et non réguliers, à travers le Principe de Maximum, dans le contexte du guidage optimal. Les chapitres 4 et 5 concernent le développement et les améliorations annexes des stratégies numériques pour le guidage optimal de missiles intercepteurs, à travers les méthodes indirectes. Dans le Chapitre 4, la méthode indirecte classique est combinée avec un schéma d'homotopie efficace, dont l'initialisation est apportée à l'aide d'une étude additionnelle de certaines formulations approchées du Principe de Maximum. Le Chapitre 5 propose l'amélioration de la robustesse numérique via la construction de grilles d'initialisation. La strategie numérique pour résoudre les problèmes de contrôle optimal avec retards, en combinant les méthodes indirectes et l'homotopie sur les retards, est introduite et analysée dans les chapitres 6 et 7. Le Chapitre 6 fournit d'abord les propriétés de continuité, par rapport au retard, des extrémals de Pontryagin liées aux problèmes de contrôle optimal avec retards. Donc, sur la base de ces résultats, l'algorithme d'homotopie annexe est introduit et commenté à l'aide de simulations numériques concernant des problèmes de guidage optimal plus généraux. D'autre part, le Chapitre 7 est consacré aux détails de la preuve des propriétés de continuité précédentes. Enfin, le manuscrit s'achève par une section dédiée aux conclusions et perspectives.

General Introduction Main Context

The optimal guidance of launch vehicles has attracted more and more attention in the past few decades, so that, nowadays, it has become literally a prerequisite in various aerospace applications, such as, orbit transfer (see, e.g. [START_REF] Kaplan | Modern spacecraft dynamics and control[END_REF]2]), atmospheric entry (see, e.g. [3,4]) and missile guidance (see, e.g. [5,6,[START_REF] Lu | Nonlinear predictive controllers for continuous systems[END_REF][START_REF] Zarchan | Tactical and strategic missile guidance[END_REF]). This problem is devoted to finding a control law enabling an aerospace vehicle to join a final target area by considering prescribed constraints as well as performance criteria. Constraints mean that only certain maneuvers (i.e. controls) and trajectories are allowed while performance criteria are demanded to reduce expensive efforts or maximize chances of success. Of course, these constraints and performance criteria depend on the considered vehicle and, as well, on the mission to accomplish. For instance, in orbit transfer applications, a typical mission consists in steering a satellite from a halo to another given orbit, with minimal effort. In this case, gravitational laws provide precise dynamical movements to which the vehicle is confined. Moreover, due to the high cost of even small maneuvers in open space, one usually tries to displace the satellite by minimal fuel consumption. On the other hand, interception missions consist in leading a missile towards a (possibly fast) target (cruise missile, ballistic missile, etc.), with the aim of neutralizing it. In this situation, constraints are given by the equations of flight dynamics and by the feasibility of certain maneuvers. Typically, one makes further efforts to maximize the velocity of the vehicle, in order to increase lethality and chances to destroy the threat.

Optimal guidance problems can be interpreted, studied and solved via the formal and well-posed mathematical representation given by optimal control. In its generality, classical optimal control deals with the problem of finding a measurable control function for a given control dynamical system, evolving within a manifold, such that a certain smooth cost functional is minimized. Constraints, limiting either the control functions or the dynamical trajectories, are usually taken into account.

General Introduction

The first answers to optimal control problems date back to the 18th century with the work done by Euler and Lagrange, which led to the calculus of variations. However, it is in the late 50s of the last century that the contributions of the Russian school of mathematics provided the theoretical fundamental results for the optimal control of very general control systems. The pioneering work of Lev Pontryagin and his students yielded powerful necessary optimality conditions, known as Pontryagin Maximum Principle (or just Maximum Principle, or PMP, see, e.g. [START_REF] Pontryagin | Mathematical theory of optimal processes[END_REF]), and put the basis for more exhaustive conditions (see, e.g. [START_REF] Hestenes | Calculus of variations and optimal control theory[END_REF][START_REF] Clarke | The maximum principle under minimal hypotheses[END_REF][START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF][START_REF] Sussmann | A maximum principle for hybrid optimal control problems[END_REF][START_REF] Li | Optimal control theory for infinite dimensional systems[END_REF]).

Simultaneously, the advent of computers and numerical methods allowed to practically solve almost every optimal control problem provided by applications and companies, up to numerical approximation errors. Nowadays, a large spectrum of numerical algorithms exist to provide solutions to optimal control problems, and we can roughly split them into two main classes: direct methods and indirect methods (see, e.g. [START_REF] Bryson | Applied optimal control: optimization, estimation and control[END_REF][START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF][START_REF] Rao | A survey of numerical methods for optimal control[END_REF] for detailed surveys on these numerical procedures).

Direct methods consist in discretizing each component of the optimal guidance problem (the state, the control, etc.) to reduce it to a nonlinear constrained optimization problem. A high degree of robustness is provided while, in general, no deep knowledge of the dynamical system is required, making these methods rather easy to use in practice. However, their efficiency strongly depends on computational capabilities, which often obliges to use them offline uniquely. On the other hand, indirect methods consists in applying the conditions given by the Maximum Principle, wrapping the optimal guidance problem into a two-point boundary value problem, which leads to accurate and fast algorithms. The advantages of indirect methods, whose more basic version is known as shooting method, are their extremely good numerical accuracy and the fact that, if they converge, the convergence is very quick. Nevertheless, initializing indirect methods still remains the hardest task.

Among research studies carried out at ONERA-The French Aerospace Lab, optimal guidance of launch vehicles has earned an important position, finding both civil and military applications (launch systems such as Ariane rockets, missile systems, unmanned aerial vehicles, etc.).

Most of the main frameworks consider offline computations via direct methods to ensure robust convergence to optimal solutions. However, when managing dynamic problems such as, for example, an interceptor missile dealing with maneuvering targets, precomputed trajectories can lose quickly their optimality, or worse, lead to a failure of the mission. In this situations, fast recomputations of optimal trajectories are needed to definitely improve the chances to achieve the mission.

For these reasons, today, ONERA-The French Aerospace Lab devotes particular attention on the development of algorithms for an onboard real-time optimal guidance of launch vehicles. Onboard means that the new methods should be able to operate

General Introduction

efficiently with a reduced charge of processors, while real-time implies the ability to provide online updates of numerical computations at frequencies compatible with the operating frequency of the guidance system (usually in the range of 1-10 Hz).

Objective and Contributions of this Work

The present work arises from a collaboration between ONERA-The French Aerospace Lab and LJLL-Laboratoire Jacques-Louis Lions (at Sorbonne University). The main objective consists in conceiving an autonomous algorithm for the prediction of optimal control strategies for endo-atmospheric launch vehicle missions, based on indirect methods, and able to adapt itself to unpredicted changes of the original scenario, in real time. Particular interest is put on interception missions.

The choice of preferring indirect methods to compute optimal trajectories derives from the fact that they are well-suited to provide both good numerical accuracy and real-time compatibility, ideal for onboard processors. Another important requirement is to ensure that the whole method provides strategies for every feasible scenario. Robustness of the concerned algorithms must be ensured at least statistically.

To this aim, we identify the following main goals and related contributions:

1. Apply the Maximum Principle and recover global strategies.

An accurate study of the optimal guidance of endo-atmospheric launch vehicles compels to consider both usually demanding performance criteria and possible onerous missions to accomplish. Since, in this situation, the vehicle is subject to several strong mechanical strains, some stability constraints must be imposed, which turn out to be modeled as mixed control-state constraints, i.e. combined constraints limiting both controls and trajectories. This kind of optimal control problems is difficult to treat by the Maximum Principle (see, e.g. [START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF][START_REF] Clarke | Optimal control problems with mixed constraints[END_REF]). Indeed, further Lagrange multipliers appear, for which, obtaining rigorous and useful information may be arduous and has been the object of many studies in the existing literature (see, e.g. [START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF][START_REF] Bonnans | Well-posedness of the shooting algorithm for state constrained optimal control problems with a single constraint and control[END_REF][START_REF] Arutyunov | The maximum principle for optimal control problems with state constraints by R.V. Gamkrelidze: revisited[END_REF]). This issue prevents from considering usual indirect methods, therefore, compromising fast convergences to optimal solutions.

A widespread approach in aeronautics to avoid to deal with these particular mixed control-state constraints consists in reformulating the original guidance problem using some local Euler coordinates, under which, the stability constraints become pure control constraints (see, e.g. [START_REF] Bonnard | Singular trajectories and their role in control theory[END_REF]). The transformation allows to consider the classical Maximum Principle, and then, usual shooting methods. However, Euler coordinates are not global and have singularities that

General Introduction

prevent from solving all reachable configurations, reducing the number of possible achievable missions.

The solution that we propose consists in reformulating the optimal guidance problem within an intrinsic viewpoint, using geometric control (it does not seem that this general framework has been systematically investigated in the optimal guidance context so far). In particular, we build additional local coordinates which cover the singularities of the previous ones and under which the mixed control-state constraints can still be reinterpreted as pure control constraints. Moreover, these two sets of local coordinates form an atlas of the configuration manifold which we use also to recover, by using some geometric control techniques, the complete behavior of Pontryagin extremals, even in the case where nonregular arcs occur.

The introduction of these particular local coordinates provides, in turn, two main benefits. On one hand, there is no limit on the feasible mission scenarios that can be simulated, and, on the other hand, the optimal guidance problem is not conditioned by multipliers depending on mixed constraints (or, at least, locally), therefore, classical standard indirect methods can be easily put in practice. This is at the price of changing local coordinates, which slightly complicates the implementation of the shooting method, but, importantly, does not affect its computational speed and precision.

Design an efficient numerical scheme based on indirect methods.

The procedure detailed previously allows to apply indirect methods to solve endo-atmospheric optimal guidance problems. However, as already pointed out, even if indirect methods have extremely good numerical accuracy, their main drawback is their initialization. This issue is well-known in the aerospace community and many efforts were gathered to provide efficient initialization procedures (a thorough summary of these techniques can be found in the survey [START_REF] Trélat | Optimal control and applications to aerospace: some results and challenges[END_REF]). Since the aim is to keep the whole resolution algorithm independent from direct methods, we desire to avoid the use of any kind of direct method.

We propose an efficient initialization procedures for indirect methods by employing homotopy methods (see, e.g. [START_REF] Allgower | Introduction to numerical continuation methods[END_REF]). Recently, these numerical schemes have acquired good reputation for aerospace applications, mostly thanks to their high reliability and versatility (see, e.g. [START_REF] Graichen | A continuation approach to state and adjoint calculation in optimal control applied to the reentry problem[END_REF][START_REF] Hermant | Optimal control of the atmospheric reentry of a space shuttle by an homotopy method[END_REF][START_REF] Zhu | Minimum time control of the rocket attitude reorientation associated with orbit dynamics[END_REF]). The basic idea of homotopy methods is to solve a difficult problem step by step starting from a simpler problem (usually called problem of order zero) by parameter deformation. Combined with the shooting problem derived from the Maximum Principle, a homotopy method consists in deforming this two-point boundary value problem into a simpler one (which can be easily integrated) and then solving a series of shooting problems step by step to come back to the original system.

General Introduction

In the case in which the homotopic parameter is a real number and when the path consists in a convex combination of the problem of order zero and of the original problem, the homotopy method is rather called continuation method.

Numerical homotopy methods are composed by two main steps: the choice of a problem of order zero (which should be as simple to solve as possible) and a procedure for the parameter deformation. We solve numerically the endoatmospheric optimal guidance problem by adopting the following scheme:

2.a) Design and solve the problem of order zero by an explicit guidance law.

Experimental tests show that the simplified optimal guidance problem obtained by removing the contribution of the thrust and the gravity from the original flight dynamical model maintains sufficient regularities, so that, once this problem of order zero is solved, appropriate parameters deformations on it result in a fast converging homotopy procedure.

Another important property of this simplified optimal guidance problem is that initializing shooting methods on it can be done analytically and instantaneously. Indeed, by manipulating the Maximum Principle applied to this problem, we provide a new approximated explicit guidance law able to efficiently initialize indirect methods on the previous problem of order zero, for a large range of initial and final conditions.

2.b) Devise an appropriate parameter deformation scheme.

The original optimal guidance problem is solved deforming the previous problem of order zero by adding iteratively the contribution of the thrust and the gravity previously removed. We choose a modified feasible scenario able to initialize the simplified problem analytically (see 2.a)) and keep it fixed during this homotopy step. Therefore, one further homotopy step deforms the first temporary scenario to obtain the optimal solution of the complete problem considering the original scenario.

When the scenario implies that the optimal trajectory encounters some Euler singularity, in the previous numerical scheme, the computations are temporarily stopped and a change of coordinates is operated (see 1.). From this, the numerical integration starts again avoiding converging failures. This change of coordinates slightly affects the total computational time, maintaining the usual fast convergence rate of indirect methods.

Robustify and speed up the convergence of the guidance problem.

Even if the numerical procedure developed in 2. provides an efficient scheme to solve the optimal guidance of launch vehicles, one main issue is not figured

General Introduction

out: the high sensitivity to initial conditions. Indeed, the whole homotopy procedure described in 2., starting from the simplified problem and finishing with the scenario deformation step, may contain several homotopic iterations (depending on the difficulty of the mission), implying that different missions may take more or less computational time to converge to the optimal solution. In other words, the average computational time of the whole homotopy scheme may be too large for a real-time implementation.

As solution, we propose to manage the high sensitivity issue, thus increasing the robustness of the homotopy scheme, by calling a refined offline precomputed grid of initial guesses of Pontryagin extremals, that contains the solutions of the complete original control problem (thrust and gravity included) for many feasible scenarios. Therefore, the resolution of any given arbitrary mission proceeds as follows. First, the grid is charged into the RAM (offline), then, the scenario the closest (with respect to some metric) to the one provided by the user is selected from the initialization grid, and finally, a spatial deformation is computed to recover the solution of the original problem. The finer the grid is, the better chance one has to obtain the global optimal solution.

We develop this improvement in the context of optimal interception missions. Statistical numerical tests show that only few homotopic iterations are needed in general to obtain a solution, thus high rates of convergence are ensured. Furthermore, since the recomputing of interception strategies is fast enough, the robustness of solutions with respect to exterior variations of the initial scenario, such as fast moving targets, increases considerably.

4. Discuss and prove the well-posedness of homotopy methods for optimal control problems with delays, when the homotopy is operated on delays.

Optimal guidance problems focus on the control of the dynamics of the center of gravity of launch vehicles, avoiding to manage the control of rigid body configurations. Several reasons are invoked to consider separately the guidance problem and the physical maneuvers realized by the pilot, and the most unanimous one states that, usually, the guidance system operates at lower frequencies than the pilot. However, some delays on the follow-up of orders occur. In order to refine the model and related control laws, one can extend the optimal guidance problem by approximating the contribution of the pilot (i.e. of the rigid body dynamics) into the equations of motion, and considering delays between the translating dynamics and the rotating dynamics (see, e.g. [6,[START_REF] Hanson | A plan for advanced guidance and control technology for 2nd generation reusable launch vehicles[END_REF]).

This framework obliges to deal with nonlinear optimal control problems with delays and the challenge consists in solving efficiently such problems by shooting algorithms. This question is usually complex and computationally demanding. Indeed, the Maximum Principle for optimal control problems with delays

General Introduction

(see, e.g. [START_REF] Kharatishvili | The maximum principle in the theory of optimal processes involving delay[END_REF][START_REF] Halanay | Optimal controls for systems with time lag[END_REF][START_REF] Banks | Necessary conditions for control problems with variable time lags[END_REF][START_REF] Göllmann | Theory and applications of optimal control problems with multiple time-delays[END_REF]) provides nonlinear two-point boundary value problems which combine backward and forward terms of time, forcing a global integration of related differential equations. Therefore, a local guess concerning the optimal extremal is no more useful, but rather, good global guesses must be provided to make the procedure converge. This represents an additional difficulty with respect to the usual shooting method, which clearly affects computational performances.

Since the methods previously adopted allow to solve the optimal guidance problems without delays, it seems legitimate to wonder if one may solve the optimal guidance problem containing delays by indirect methods starting a homotopy procedure where the delay represents the deformation parameter and the problem without delays is taken as the problem of order zero. This approach is a way to address the flaw of indirect methods applied to optimal control problems with delays: on one hand, the global information of the problem without any delay could be used to initialize efficiently a shooting method with delays and, on the other hand, we could solve the two-point boundary value problem with delays via usual local iterative methods for differential equations. Nevertheless, unlike the classical non-delayed optimal control framework, in which, under appropriate assumptions, the convergence of homotopy methods is understood and well-established (see, e.g. [START_REF] Trélat | Optimal control and applications to aerospace: some results and challenges[END_REF]), the well-posedness of indirect methods combined with homotopy procedures on delays has not been well addressed in the literature yet. In particular, bifurcation points or singularities may be encountered, causing convergence failures.

The solution that we propose resides on a detailed study of the well-posedness concerning homotopy procedures on the delay to solve general nonlinear optimal control problems with delays, via indirect methods. Our main theoretical result is that, under appropriate assumptions, the quantities provided by the Maximum Principle, including adjoint vectors and trajectories, are continuous with respect to the delays. This result ensures the well-posedness of the previously proposed numerical scheme: any homotopy path of delays converges to Pontryagin extremals of the original optimal control problem with delays, when starting from the problem without any delay. Once these continuity properties are well established in a general nonlinear optimal control framework, we make use of this result to efficiently provide the numerical resolution for more general optimal guidance problems with delays.

The previous contributions made possible the development of a fully automatic, independent and completely self-regulating software, today property of ONERA-The French Aerospace Lab, for general realistic endo-atmospheric launch vehicle applications focused, in particular, on optimal missile interception scenarios. Within one

General Introduction

second, the algorithm is able to provide optimal strategies for eleven different missions with just one processor of few megabytes of memory, allowing a real-time computation with small onboard computers, ideal for autonomous vehicles.

Organization of the Manuscript

The present thesis is organized in seven chapters and a final conclusion. Chapter 1 and Chapter 2 aim to introduce the reader to the classical theory of optimal control and the optimal guidance problems on which this study focuses, respectively. At the same time, these chapters are exploited to provide notations and standard theoretical results that are used throughout the manuscript.

In Chapter 3, we derive the structure of Pontryagin extremals of the optimal guidance problem. An abstract framework introducing consistency conditions of adjoint vectors under local change of representation is presented first. The result is then applied to correctly study, via the classical Maximum Principle, both regular and nonregular Pontryagin extremals in the optimal guidance context. Chapter 4 and Chapter 5 concern the development and related improvements of numerical strategies to solve the optimal guidance of interceptor missiles, via indirect methods. In Chapter 4, the classical indirect method is combined with an efficient homotopy scheme, whose initialization is provided under further computations via approximated Maximum Principle formulations. Chapter 5 proposes improvements on numerical robustness via the construction of initialization grids.

The numerical strategy to solve optimal control problems with delays, by combining indirect methods and homotopy on the delay, is introduced and analyzed in Chapter 6 and Chapter 7. Chapter 6 provides first the main statement concerning continuity properties, with respect to the delay, of Pontryagin extremals related to optimal control problems with delay. Therefore, based on these results, the related homotopy algorithm is introduced and commented with the help of numerical simulations on more general optimal guidance problems. On the other hand, Chapter 7 is devoted to the detailed proof of the previous continuity properties.

The manuscript ends with a section devoted to conclusions and perspectives.

Part I Optimal Control Framework and Dynamical Model 1 Elements of Optimal Control

In this chapter, we provide a brief introduction of the main mathematical results concerning optimal control theory. Particular attention is emphasized on theoretical and numerical techniques needed to deal with indirect methods, principal context of this thesis. Furthermore, several references concerning fundamental works in the area of optimal control and related aerospace applications are gradually provided. The chapter is organized as follows. In Section 1.1 some basic tools of differential geometry concerning flows of vector fields and symplectic geometry are recalled, which are essential to settle a control problem in a geometric intrinsic framework, feature that we exploit in Chapter 3. Section 1.2 provides the classical optimal control problem formulation and necessary optimality conditions in the form of a Lagrange multipliers rule. Stronger optimality conditions are given and analyzed in Section 1.3 together with an introduction to classical numerical algorithms (direct and indirect methods). Section 1.4 and Section 1.5 conclude by extending the results respectively to optimal control problems with control and state constraints and to optimal control problems with delays, nontrivial frameworks on which this study focuses.

Some Tools from Differential Geometry

The aim of this section is to introduce notations and basic results concerning manifolds and Hamiltonian systems. These frameworks are standard and well-known, and we refer the reader for any further detail to classical texts such as [START_REF] Lee | Introduction to smooth manifolds[END_REF][START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF].

Notations and Properties of Vector Fields

Throughout this dissertation, M denotes a smooth manifold of dimension n and its algebra of germs is denoted by C ∞ (M ). The notation (V, ϕ) identifies a local chart of M , whose coordinates are (x 1 , . . . ,

x n ). Every open subset V ⊆ M is a submanifold of

Chapter 1. Elements of Optimal Control

dimension n of M , and we denote by C ∞ (V ) the related algebra of germs. For a given point q ∈ M , the tangent and cotangent spaces at q of M are denoted respectively by T q M and by T * q M while the tangent and the cotangent bundles are denoted respectively by T M and by T * M . If F : M → N is a smooth mapping between smooth manifolds, we denote its differential by d F : T M → T N .

We call vector field on M every smooth section of the tangent bundle, i.e. every smooth function f : M → T M such that f (q) ∈ T q M . Let (V, ϕ) be a smooth local chart of M . Then, there exist n local functions f i (•) ∈ C ∞ (V ), that we call coordinates of the vector field, such that

f (q) = n i =1 f i (q)∂ x i (q) = n i =1 f i (q) ∂ ∂x i q , q ∈ V .
Given a vector field f , to every a(•) ∈ C ∞ (M ), we can associate the smooth germ f a defined as f a(q) = f (q)(a). Proceeding in this way, to every vector field f , we can associate a unique linear operator in C ∞ (M ), always denoted by f , defined as

f : C ∞ (M ) → C ∞ (M ) : a → f a
which is a derivation, i.e. a linear operator satisfying the Leibniz rule. From this, we define the Lie bracket of two vector fields f , g as the commutator

[ f , g ] : C ∞ (M ) → C ∞ (M ) : a → f (g a) -g ( f a) .
The operator [ f , g ] is a derivation. Indeed, it is linear and satisfies

[ f , g ](ab) = f (g (ab)) -g ( f (ab)) = f (ag (b)) + f (bg (a)) -g (a f (b)) -g (b f (a)) = a f (g b) + b f (g a) -ag ( f b) -bg ( f a) = a[ f , g ](b) + b[ f , g ](a) .
It follows that we can associate a unique vector field to [ f , g ], still denoted by [ f , g ], with the following main properties.

Proposition 1.1. Let f , g and h be vector fields on M and a, b ∈ C ∞ (M ). The Lie bracket has the following properties:

1. [ f , g ] = -[g , f ] 2. [ f , [g , h]] + [g , [h, f ]] + [h, [ f , g ]] = 0 (Jacobi identity) 3. [a f , bg ] = ab[ f , g ] + a( f b)g -b(g a) f 4. If in local coordinates we have f = h f h ∂ x h and g = k g k ∂ x k , then: [ f , g ] = n h,k=1 f h ∂g k ∂x h -g h ∂ f k ∂x h ∂ x k .

Some Tools from Differential Geometry

We call nonautonomous vector field on M (or simply, vector field on M , when no misunderstanding arises) every continuous mapping f : R× M ×R m → T M such that f (t , q, u) ∈ T q M , and which is smooth w.r.t. (t , q). The variable u is understood as a parameter and we call it control. Take a control u ∈ L ∞ l oc (R, R m ). The nonautonomous vector field f u : R× M → T M : (t , q) → f (t , q, u(t )) is measurable w.r.t. t , smooth w.r.t. q and locally bounded. Note that it is always possible to provide globally bounded vector fields from locally bounded ones by multiplying them by a smooth cut-off function. Thanks to this remark, from now on, without loss of generality, we suppose that any considered vector field on any manifold is globally bounded. For every time t 0 ∈ R and every point q 0 ∈ M , we call dynamical problem every ordinary differential equation of the following type

q(t ) = f u (t , q(t )) , q(t 0 ) = q 0 . (1.1)
As a standard result, for every couple (t 0 , q 0 ) ∈ R × M , problem (1.1) has a unique solution, that we denote by q t 0 ,q 0 (•), defined for every t ∈ R. In particular, we call complete any vector field providing solutions to (1.1) that are defined in R (remark that such vector fields do not need to be globally bounded). From this, we define the exponential mapping of f u as the flow exp f u : R 2 × M → M : (t , t 0 , q 0 ) → q t 0 ,q 0 (t ) .

This function is Lipschitz w.r.t. t and t 0 (then, absolutely continuous w.r.t. the Whitney topology of C ∞ (M )) and is smooth w.r.t. q 0 . In particular, for every (t ,

t 0 ) ∈ R 2 , exp f u (t ; t 0 , •) : M → M is a diffeomorphism whose inverse is exp f u (t 0 ; t , •) : M → M .
The mapping exp f u is known as exponential.

Standard Results on Hamiltonian Fields

We denote by Λ k (M ) the vector bundle of all k-forms on M . A differential k-form (or simply, k-form, when no misunderstanding arises) is a section of Λ k (M ), i.e. a smooth function η : M → Λ k (M ) such that η q = η(q) ∈ Λ k q (M ). The space of all kforms is denoted by A k (M ). Let N be a smooth manifold and F : M → N be a smooth mapping. We call the pull-back of

η ∈ A k (N ) the k-form F * η ∈ A k (M ) obtained by (F * η) q (v 1 , . . . , v k ) = η F (q) (d F (q)(v 1 ), . . . , d F (q)(v k ))
for every q ∈ M and every v 1 , . . . , v k ∈ T q M . The set A 1 (M ) is the space of all 1-forms on M , i.e. smooth mappings s : M → T * M such that s(q) ∈ T * q M . We recall that any local chart (V, ϕ) of M determines canonical local coordinates of the form (x, ξ) = (x 1 , . . . , x n ; ξ 1 , . . . , ξ n ) for the cotangent bundle

Chapter 1. Elements of Optimal Control

T * M . Then, any covector p ∈ T * q M has the decomposition p = n i =1 ξ i d x i | q , where we denote

d x i = (∂ x i ) * ∈ A 1 (M ).
The cotangent bundle T * M can be equipped with a canonical symplectic structure as follows. Consider the projection π : T * M → M and define the Liouville 1-form as

s : T * M → T * (T * M ) : p → π * p ∈ T * p (T * M ) .
In local coordinates, the 1-form s can be written as s(p)

= n i =1 ξ i d x i | π(p)
. Then, by taking the differential of s, we obtain the nondegenerate closed 2-form σ = d s

= n i =1 d ξ i ∧ d x i ∈ A 2 (T * M ), which makes (T * M , σ) a symplectic manifold.
We call Hamiltonian every arbitrary smooth function on the cotangent bundle. To any Hamiltonian h ∈ C ∞ (T * M ), we can associate a unique Hamiltonian vector field

h : T * M → T (T * M ) such that σ(p)(•, h) = d h(p) for every p ∈ T * M . In canonical co- ordinates, one has h = n i =1 ∂h ∂ξ i ∂ x i -∂h ∂x i ∂ ξ i .
We call Hamiltonian dynamical problem corresponding to h the following dynamical problem related to the field h

ṗ(t ) = h(p(t)) , p(t 0 ) = p 0 ∈ T * π(p 0 ) M
which, in canonical coordinates, with an abuse of notation, reads

q(t ) = ẋi = ∂h ∂ξ i = ∂h ∂p (q(t ), p(t )) , ṗ(t ) = ξi = - ∂h ∂x i = - ∂h ∂q (q(t ), p(t )) . (1.2)
In control theory, one usually considers nonautonomous Hamiltonians of type

h(t , q, p, u) = h (t ,u) (p) = 〈p, f (t , q, u)〉 , q = π(p) , p ∈ T * q M
where f is a nonautonomous vector field and (t , u) ∈ R × R m . In this case, t and u must be read as parameters. Fix a control u ∈ L ∞ l oc (R, R m ). Therefore, the Hamiltonian dynamical system takes the form

ṗ(t ) = h(t, q(t ), p(t ), u(t )) = h (t ,u(t )) (p(t )) , p(s) = p s ∈ T * q(s) M . (1.3)
Its solutions are absolutely continuous curves (in the sense of the Whitney topology of C ∞ (T * M )) and write p(t ) = (exp f u (s; t , •)) * π(p s ) • p s for every p s ∈ T * π(p s ) M . This derives immediately combining (1.2), the local coordinates version of (1.3), with the following lemma. Lemma 1.1. For almost every t ∈ R, every s ∈ R and every i = 1, . . . , n, there holds

d d t (exp f u (s; t , •)) * π(p s ) • p s ∂ ∂x i q u (t ) (t ) = - n j =1 ∂ f j ∂x i (t , q u (t ), u(t )) (exp f u (s; t , •)) * π(p s ) • p s ∂ ∂x j q u (t )
where p s ∈ T * π(p s ) M is an arbitrary covector and q u (t ) = exp f u (t ; s, π(p s )).

Classical Optimal Control Problems

Proof. For every i = 1, . . . , n, denote a i (t ) = (exp f u (s; t , •)) * π(p s ) • p s ∂ ∂x i q u (t ) and let (V, ϕ) be a local chart of π(p s ). We have

p s ∂ ∂x i π(p s ) = n j =1 a j (t ) ∂ ∂x i (x j • exp f u (t ; s, •) • ϕ -1 )(ϕ(π(p s ))) .
The term on the left does not depend on t . Therefore, by differentiating the previous expression with respect to t , we easily obtain

n j =1 ȧ j (t ) (exp f u (t ; s, •)) * q u (t ) • d x j | q u (t ) + n l =1 a j (t ) ∂ f j ∂x l (t , q u (t ), u(t )) (exp f u (t ; s, •)) * q u (t ) • d x l | q u (t ) = 0 .
Since (exp f u (t ; s, •)) * is a bundle isomorphism, by linearity, we get

n j =1 ȧ j (t )d x j | q u (t ) + n l =1 a j (t ) ∂ f j ∂x l (t , q u (t ), u(t ))d x l | q u (t ) = 0 which evaluated at ∂ ∂x i q u (t )
gives the desired result.

Classical Optimal Control Problems

Let f be a nonautonomous vector field on M which is C 1 w.r.t. the variable u. Moreover, let M f be a subset of M , U be a subset of R m and fix an initial condition q 0 ∈ M . We introduce a smooth final cost function g : R × M → R and a smooth integral cost function f 0 : R × M × R m → R. A general nonlinear Optimal Control Problem (OCP) on the manifold M consists in minimizing the cost

C (t f , u) = g (t f , q(t f )) + t f 0 f 0 (t , q(t ), u(t )) d t such that q(t ) = f (t , q(t ), u(t )) , q(0) = q 0 , q(t f ) ∈ M f among all the controls u ∈ L ∞ ([0, t f ], R m ) satisfying u(t ) ∈ U almost everywhere in [0, t f ].
Note that the final time t f may be fixed or not.

Remark 1.1. Sometimes, the initial condition is expressed more generally by asking that q(0) ∈ M 0 , where M 0 is a generic subset of M . However, in the context of this thesis, we always consider q(0) = q 0 , where q 0 ∈ M is a given fixed point. Moreover, always under smooth behaviors, the final cost g can be written as an integral cost, and we often make use of this transformation.

Chapter 1. Elements of Optimal Control

Problem (OCP) is usually called classical optimal control problem because only pure control constraints appear. The existence of solutions of (OCP), under appropriate assumptions, is a well-established result (see, e.g. [START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF][START_REF] Lee | Foundations of optimal control theory[END_REF][START_REF] Cesari | Optimization -theory and applications: problems with ordinary differential equations[END_REF]).

A natural representation of (OCP) is given by considering an infinite dimensional optimization framework as follows. The infinite dimensional constraints are imposed by considering the end-point mapping

E : R × L ∞ l oc (R, R m ) → M : (t f , u) → exp f u (t f ; 0, q 0 )
which is well-defined because the considered dynamics have compact supports. Remark that, as a classical result, this mapping is continuously Fréchet differentiable (see, e.g. [START_REF] Bonnard | Singular trajectories and their role in control theory[END_REF]). Therefore, problem (OCP) is equivalent to

min C (t f , u) : t f ≥ 0 , u ∈ L ∞ ([0, t f ],U ) , E (t f , u) ∈ M f . (1.4)
Under this new formulation, one is led to characterize the solutions of (OCP) among the critical points of (1.4), by means of the Lagrange multiplier rule (see, e.g. [START_REF] Hestenes | Calculus of variations and optimal control theory[END_REF][START_REF] Bonnans | Perturbation analysis of optimization problems[END_REF]). The idea at the basis of the Lagrange multiplier rule arises as follows.

Let us consider the situation in which

M = R n , M f = {q f } where q f ∈ M is fixed, U = R m
, the final time t f is fixed and, without loss of generality, g = 0 (see Remark 1.1). Denote f = ( f , f 0 ) and introduce the extended end-point mapping

Ẽ : R × L ∞ l oc (R, R m ) → M : (t f , u) → exp f (t f ; 0, (q 0 , 0)) . If u ∈ L ∞ ([0, t f ], R m
) is an optimal solution of (OCP) (or, equivalently, of (1.4)), it is well known (see, e.g. [START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF][START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]) that u is a critical point of the extended end-point mapping (i.e. ∂E ∂u (t f , u) is not of full rank), and then, there exists a half-space containing the image of the differential of Ẽ . In other words, there exists a nontrivial multiplier (p,

p 0 ) ∈ R n × R such that p • d E t f (u) + p 0 dC t f (u) = (p, p 0 ) • d Ẽt f (u) = 0 (1.5)
where

d E t f (•), dC t f (•) and d Ẽt f (•) denote the Fréchet derivative w.r.t. u of E t f (•) = E (t f , •), C t f (•) = C (t f , •) and Ẽt f (•) = Ẽ (t f , •), respectively. By defining the Lagrangian L t f (u, p, p 0 ) = p • E t f (u) + p 0 C t f (u)
the optimality condition (1.5) can be written in the form

∂L t f ∂u (u, p, p 0 ) = 0.
The Lagrange multiplier rule is a first-order necessary optimality condition. Although (1.5) turns out to be the most intuitive condition for optimal control problems, it represents only weak necessary conditions and it can be improved considering either high-order necessary conditions or sufficient optimality conditions, which we provide and analyze in the following section.

Optimality Conditions and Numerical Methods

Optimality Conditions and Numerical Methods

More informative relations on optimal controls related to (OCP) can be achieved by refining the Lagrange multiplier rule, arising conditions like the Maximum Principle, whose aim consists in reformulating optimal control problems into initializations of ordinary differential equation systems. A detailed analysis of optimal arcs leads also to sufficient conditions of optimality. All this information can be exploited to set up several numerical resolution strategies for (OCP), as we develop hereafter.

The Maximum Principle

First-order necessary conditions stronger than the Lagrange multiplier rule come from the Pontryagin Maximum Principle (PMP or Maximum Principle, see, e.g. [START_REF] Pontryagin | Mathematical theory of optimal processes[END_REF][START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]). For this, for every t ∈ R, u ∈ R m , introduce the Hamiltonian related to (OCP), as

h(t , q, p, p 0 , u) = h (t ,u) (p, p 0 ) = 〈p, f (t , q, u)〉 + p 0 f 0 (t , q, u) . (1.6)
Theorem 1.1. Let q(•) be an optimal trajectory for (OCP), associated to the control u(•) on [0, t f ]. There exist a nonpositive scalar p 0 and an absolutely continuous curve p : [0, t f ] → T * M called adjoint vector, for which p(t ) ∈ T * q(t ) M , with (p(•), p 0 ) = 0 and such that, almost everywhere in [0, t f ], the following relations hold:

• Adjoint Equations q(t ) = ∂h ∂p (t , q(t ), p(t ), p 0 , u(t )) , ṗ(t ) = -∂h ∂q (t , q(t ), p(t ), p 0 , u(t )) (1.7)

• Maximality Condition

h(t , q(t ), p(t ), p 0 , u(t )) = max u∈U h(t , q(t ), p(t ), p 0 , u) (1.8)

• Transversality Conditions

If M f is a submanifold of M , locally around q(t f ), then the adjoint vector can be built in order to satisfy

p(t f ) -p 0 ∂g ∂q (t f , q(t f )) ⊥ T q(t f ) M f (1.9)
and, moreover, if the final time t f is free, one has

max u∈U h(t f , q(t f ), p(t f ), p 0 , u) = -p 0 ∂g ∂t (t f , q(t f )) .
(1.10)

Chapter 1. Elements of Optimal Control

Definition 1.1. A tuple (q(•), p(•), p 0 , u(•)) satisfying the Maximum Principle is called the extremal lift of q(•) or just extremal. An extremal is said to be normal if p 0 < 0 and, in this case, it is usual to set p 0 = -1. Otherwise, it is said to be abnormal.

In the literature, many approaches to prove the Maximum Principle were developed.

The first proof was given by Pontryagin and its students and goes back to early 60s (the original procedure can be found in [START_REF] Pontryagin | Mathematical theory of optimal processes[END_REF][START_REF] Hestenes | Calculus of variations and optimal control theory[END_REF]). Their approach is based on particular variations of the control, called needle-like variations. Other works propose more complex variations, such as sliding variations or v-variations (see, e.g. [START_REF] Dubovitskii | Extremum problems in the presence of restrictions[END_REF][START_REF] Milyutin | Maximum principle for regular systems[END_REF][START_REF] Dmitruk | Maximum principle for the general optimal control problem with phase and regular mixed constraints[END_REF]), in order to apply classical Lagrange multiplier rules on modified versions of (OCP).

Finally, more general proofs exist and are based on variational principles such as the Ekeland's variational principle (see, e.g. [START_REF] Li | Optimal control theory for infinite dimensional systems[END_REF][START_REF] Ekeland | On the variational principle[END_REF]). The reader can find in [START_REF] Dmitruk | On the development of Pontryagin's maximum principle in the works of A.Y. Dubovitskii and A.A. Milyutin[END_REF] a complete survey on all these techniques.

The role of the Maximum Principle consists in transforming an optimization problem of infinite dimensional spaces into the initialization of an evolutionary problem, by means of the adjoint equations. More specifically, the adjoint vector p comes into play making the couple (q, p) the new variable, instead of the control u, whose evolution in time is completely recovered by the adjoint equations, once an appropriate initial condition (q 0 , p 0 ) is provided. Since the Maximum Principle does not furnish complete information about (q 0 , p 0 ), the problem becomes to seek, among all feasible values, the initial condition (q 0 , p 0 ) corresponding to the optimal solution of (OCP). This problem is know as two-point boundary value problem and, even if it still remains an infinite dimensional problem, it is of lower dimension.

The previous initialization problem is well-posed only if an appropriate expression of the optimal control as function of (q, p) exists. For this, it becomes necessary to work with the maximality condition. However, the maximality condition does not always provide a closed-loop expression, preventing the new formulation to be well-posed. This essentially depends on the second variation of the Hamiltonian (1.6).

Definition 1.2. Let (q(•), p(•), p 0 , u(•)) be an extremal of (OCP). Denote by

ḧ(t ) = ∂ 2 h ∂u 2 (t , q(t ), p(t ), p 0 , u(t ))
the partial second variation of the Hamiltonian of (OCP). The tuple (q(•), p(•), p 0 , u(•)) satisfies the strong Legendre condition if there exists a scalar η > 0 such that

ḧ(t )(v, v) < -η v 2 , v ∈ R m , a.e. [0, t f ] . (1.11)
Such an extremal is said to be regular. Otherwise, it is said to be nonregular.

Optimality Conditions and Numerical Methods

As a classical result (see, e.g. [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]), if an extremal of (OCP) does not satisfy the strong Legendre condition, then, the Maximality Condition may not lead to closed-loop formulas for optimal controls as functions of trajectories and related adjoint vectors.

The conditions stated by Theorem 1.1 are of first-order type. Higher-order necessary condition exist, but we do not need them explicitly in this thesis, so we avoid to report them. The reader can find a complete treatise of these conditions in [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF][START_REF] Krener | The high order maximal principle and its application to singular extremals[END_REF][START_REF] Schättler | Geometric optimal control: theory, methods and examples[END_REF].

Sufficient Optimality Conditions

Since the Maximum Principle is a necessary condition, an extremal fulfilling the conditions of Theorem 1.1 may not be related to an optimal solution of (OCP). However, under appropriate assumptions, further conditions exist to ensure the optimality of normal extremals. We recall here some basic facts concerning sufficient optimality conditions, used in Chapter 3 of this thesis (we refer to [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]Chapter 17]).

Suppose that M is simply connected and consider the following autonomous version of (OCP), for which we fix a final time t f and we minimize the cost

C (u) = t f 0 f 0 (q(t ), u(t )) d t such that q(t ) = f (q(t ), u(t )) , q(0) = q 0 , q(t f ) = q f ∈ M f among all the controls u ∈ L ∞ ([0, T ], R m ) satisfying u(t ) ∈ U a.e. in [0, t f ],
where q f is a fixed final state. The Hamiltonian corresponding to normal extremals is

h(q, p, u) = h u (p) = 〈p, f (q, u)〉 -f 0 (q, u) , q = π(p) , p ∈ T * q M , u ∈ R m .
Assume that the maximized Hamiltonian H (q, p) = max u∈U h(q, p, u) = max u∈U h u (p) is well-defined and smooth in T * M . Therefore, according to the Maximum Principle, the trajectories of the Hamiltonian dynamical system

q(t ) = ∂H ∂p (q(t ), p(t )) , ṗ(t ) = - ∂H ∂q (q(t ), p(t ))
are extremals of (OCP).

For any smooth germ a ∈ C ∞ (M ), the image of its differential

L 0 = {d a(q) | q ∈ M } is a smooth submanifold of dimension n in T * M . Moreover, by denoting L t = exp H (t ; 0, •) • L 0 , the set L = {(p, t ) | p ∈ L t , t ∈ R} is a smooth (n + 1)-dimensional submanifold in T * M × R, because it is the image of the smooth injective embedding i L : L 0 × R → T * M × R : (p 0 , t ) → (exp H (t ; 0, p 0 ), t ) .

Chapter 1. Elements of Optimal Control

Consider the 1-form

pd q -H d t = s -H d t ∈ A 1 (T * M × R) (1.12)
where s = pd q is the canonical Liouville 1-form. In mechanics, (1.12) is known as the integral invariant of Poincaré-Cartan on the extended phase space T * M × R. As a classical result, the restriction (pd q -H d t )| L is an exact 1-form.

A standard sufficient optimality condition arises from the following result.

Theorem 1.2. Assume that the restriction of the projection π :

T * M → M onto L t is a diffeomorphism for any t ∈ [0, t f ].
Then, for any p 0 ∈ L 0 , the normal trajectory

q(t ) = π • exp H (t ; 0, p 0 ) , t ∈ [0, t f ]
realizes a strict minimum for problem (OCP).

Conditions providing that the restriction of the projection π| L t is a diffeomorphism for any t ∈ [0, t f ] are usually called no-fold conditions. In general, they are not easy to verify. However, when unconstrained problems are treated, i.e. U = R m , no-fold conditions are provided by an analysis of conjugate times.

Definition 1.3. Consider an extremal (q(•), p(•), p 0 , u(•)) of (OCP). For every u ∈ R m , introduce the nonautonomous vector field

g (t , q, u) = d exp f (t f ; t , •) • f (t , q, u) -f (t , q, u(t )) ,
its first partial variation ġ (t ) = ∂g ∂u (t , q(t ), u(t )) and the spaces of variations

V t = {v ∈ L 2 l oc (R,U ) | v(s) = 0 for s > t } , K t = v ∈ V t | t 0 ġ (s) • v(s)(q 0 ) = 0 .
Denoting p0 = (p(0), p 0 ), consider the intrinsic second variation of the Hamiltonian

Q t = Q| V t = t 0 ḧ(s)(v(s), v(s)) d s + t 0 p0 s 1 0 ġ (s 2 ) • v(s 2 ) d s 2 , ġ (s 1 ) • v(s 1 ) d s 1 . A time t ∈ R is said conjugate time if the quadratic form Q t | K t is degenerate.
As a standard fact (see, e.g. [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]Chapter 20]), the absence of conjugate times implies the existence of no-fold conditions, hence, the thesis of Theorem 1.2. For further details on the conjugate time theory, the reader is invited to examine [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF][START_REF] Lee | Optimal trajectory, guidance, and conjugate points[END_REF][START_REF] Zeidan | Necessary conditions for optimal control problems: conjugate points[END_REF].

Optimality Conditions and Numerical Methods

Classical Numerical Methods in Optimal Control

In order to numerically solve (OCP), two main classes of methods are widely known, that are, direct and indirect methods. Direct methods consist in solving a nonlinear optimization problem with constraints by discretizing the state and the control. Indirect methods consist of solving numerically the boundary value problem derived from the application of the Maximum Principle. The first class of methods is usually known as first-discretize-then-optimize while the second one first-optimize-thendiscretize. Only indirect methods make use of the Maximum Principle. Here, we recall the main properties of these methods, referring to the survey [START_REF] Betts | Survey of numerical methods for trajectory optimization[END_REF] for details.

Direct Methods

These techniques consist in discretizing each component of the optimal control problem (the state, the control, etc.) to reduce it to a nonlinear constrained optimization problem. The discretization method depends on the problem under consideration. Just to give an example, consider a subdivision 0

= t 0 < • • • < t N = t f of the inter- val [0, t f ].
We discretize the control by taking piecewise constant functions on each subinterval [t i , t i +1 ], with values in U , while we model the evolution of the state by a discretization method for differential equation. For example, by taking an explicit Euler method, we obtain

x i +1 = x i + (t i +1 -t i ) f (t i , x i , u i ) = x i + h i f (t i , x i , u i ) .
Therefore, by substituting all the discretized variables within the cost, the finite dimensional optimization problem related to (OCP) is

        
min C (u 0 , . . . , u N , x 0 , . . . , x N ) subject to:

x i +1 -x i -h i f (t i , x i , u i ) = 0 , u i ∈ U , i = 1, . . . , N -1 , x 0 = q 0 , x N ∈ M f .
The numerical resolution of a nonlinear constrained optimization problem represents a widespread standard context. Many algorithms are available: gradient methods, penalization methods, quasi-Newton, dual methods, genetic and multi-starting algorithms, etc. (see, e.g. [START_REF] Wright | Numerical optimization[END_REF][START_REF] Conway | A survey of methods available for the numerical optimization of continuous dynamic systems[END_REF]). It is interesting to note that, conceptually, nothing change in the routine of direct methods if a more general control problem where control and state constraints are taken into account (see also Section 1.4).

In the aerospace context, direct methods are the most popular numerical methods and many different strategies have been developed during the last decades. By way of example, related to our launch vehicle application, we cite collocation methods [START_REF] Hargraves | Direct trajectory optimization using nonlinear programming and collocation[END_REF][START_REF] Von | Direct and indirect methods for trajectory optimization[END_REF][START_REF] Benson | Direct trajectory optimization and costate estimation via an orthogonal collocation method[END_REF] and inverse/parameter optimization approaches [START_REF] Lu | Inverse dynamics approach to trajectory optimization for an aerospace plane[END_REF][START_REF] Hull | Conversion of optimal control problems into parameter optimization problems[END_REF] for trajectory optimization, and convex/conic approximations for rendezvous problem [START_REF] Blackmore | Minimum-landing-error powereddescent guidance for mars landing using convex optimization[END_REF][START_REF] Liu | Autonomous trajectory planning for rendezvous and proximity operations by conic optimization[END_REF][START_REF] Liu | Solving nonconvex optimal control problems by convex optimization[END_REF].

Chapter 1. Elements of Optimal Control

By using these procedures, a high degree of robustness is provided while, in general, no deep knowledge of the dynamical system is required, making these methods rather easy to use in practice. However, their efficiency (numerical precision and robustness) remains proportional to the computational load: the more the discretization, and then the precision, is finer, the more memory and computational time are needed in general to seek the optimal solution.

Indirect Methods

They wrap the optimal control problem into the two-point or multi-point boundary value problem coming from the Maximum Principle, which leads to accurate and fast algorithms. The most basic version is the shooting method, which we recall below. Assume that the couple (q(•), u(•)) is an optimal solution of (OCP) whose extremal (q(•), p(•), p 0 , u(•)) is regular (i.e. the maximality condition leads to a unique solution with respect to u, see Section 1.3.1). Therefore, we are able to write the optimal control as function of the state q and the adjoint vector p by u(t ) = F (q(t ), p(t )). From this, the initial/final conditions and the transversality conditions take the form R(q(0), p(0), q(t f ), p(t f )) = 0. We obtain the following boundary value problem

     q(t ) = ∂h ∂p q(t ), p(t ), F (q(t ), p(t )) , ṗ(t ) = - ∂h ∂q q(t ), p(t ), F (q(t ), p(t )) R(q(0), p(0), q(t f ), p(t f )) = R(p(0), p(t f )) = 0 .
The previous system can be written in a more compact form. Indeed, by denoting G(q 0 , p 0 ) = R(p 0 , exp h F (t f ; 0, p 0 )), one is led to solve the following problem Find (q 0 , p 0 ) such that G(q 0 , p 0 ) = 0 .

(1.13)

Very convenient and fast algorithms to solve problem (1.13) are Newton-like methods (see, e.g. [START_REF] Galántai | The theory of newton's method[END_REF][START_REF] Kelley | Solving nonlinear equations with Newton's method[END_REF]). Their rate of convergence is high (quadratic, for the classical Newton method) and a significant level of numerical precision can be obtained with a low computation effort, useful when managing problems with high sensitivity, as in aerospace frameworks. Nevertheless, their main drawback still remains their initialization, which makes practically difficult to use them, if, no further information on the original problem is known. This classical issue is known as bad robustness.

In the aerospace context, indirect methods know a lower development than direct methods mainly because of their bad robustness. Related to our launch vehicle application, important works concern high-order geometrical analysis with applications to shuttle reentry [4,[START_REF] Hermant | Optimal control of the atmospheric reentry of a space shuttle by an homotopy method[END_REF][START_REF] Trélat | Optimal control of a space shuttle and numerical simulations[END_REF] and orbital transfer [START_REF] Haberkorn | Low thrust minimum-fuel orbital transfer: a homotopic approach[END_REF][START_REF] Bonnard | Second order optimality conditions with applications[END_REF][START_REF] Cerf | Continuation from a flat to a round earth model in the coplanar orbit transfer problem[END_REF], and indirect penalization/collocation methods for rendezvous problem [START_REF] Bonnans | Singular arcs in the generalized goddard's problem[END_REF][START_REF] Graichen | Incorporating a class of constraints into the dynamics of optimal control problems[END_REF][START_REF] Graichen | Handling constraints in optimal control with saturation functions and system extension[END_REF].

To overcome the bad robustness of classical shooting methods, one can consider multi-point boundary value problems instead of two-point boundary value problems, which raise multi-shooting methods. These algorithms add further continuity

Optimality Conditions and Numerical Methods

conditions to the previous research of zeros of a function, which improves the efficiency and the stability of the method (for further details, we refer the reader to [START_REF] Bryson | Applied optimal control: optimization, estimation and control[END_REF][START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF]). However, this increases the dimension of the problem and the original fast convergence of the shooting method may be strongly compromised.

In the literature, procedures have been developed to robustify the shooting method. Among them, recently, homotopy methods have proved to be reliable for problems in the aerospace context (see, e.g. [START_REF] Trélat | Optimal control and applications to aerospace: some results and challenges[END_REF][START_REF] Hermant | Optimal control of the atmospheric reentry of a space shuttle by an homotopy method[END_REF][START_REF] Haberkorn | Low thrust minimum-fuel orbital transfer: a homotopic approach[END_REF][START_REF] Graichen | Constructive methods for initialization and handling mixed state-input constraints in optimal control[END_REF]). From this, we have considered them to conceive a numerical procedure based on indirect methods for our launch vehicle application. In the next section, we develop fundamental results to demonstrate that, under suitable assumptions, homotopy methods coupled with indirect methods provide fine convergence properties.

Numerical Homotopy Methods

The basic idea of homotopy methods is to solve a difficult problem step by step starting from a simpler problem (that we call problem of order zero) by parameter deformation. Combined with the boundary value problem derived from the Maximum Principle, a homotopy method consists in solving a series of shooting problems step by step to come back to the original problem. In the case in which the homotopic parameter is a real number and when the path consists in a convex combination of the problem of order zero and of the original problem, the homotopy method is rather called a continuation method. For sake of clarity, in the following, we consider only continuation methods (a complete survey on homotopies can be found in [START_REF] Allgower | Introduction to numerical continuation methods[END_REF]). The continuation method can be stated in its generality as follows. Suppose to seek a solution for the problem

G 1 (x) = 0 , G 1 : R N → R N (1.14)
where G 1 is smooth. We must think to (1.14) as a problem which is particularly difficult to solve. Suppose that there exists a smooth map G 0 : R N → R N whose zeros are known. We would like to recover the zeros of G 1 by operating some kind of transformation on G 0 . For this, we define a deformation G, i.e. a map

G : R N × [0, 1] → R N , G(x, 0) = G 0 (x) , G(x, 1) = G 1 (x) . A typical choice of G is a convex continuation of type G(x, λ) = λG 1 (x) + (1 -λ)G 0 (x).
To recover a zero of G 1 , one expects to follow a zero path (see Figure 1.1) starting from

Chapter 1. Elements of Optimal Control

a point x 0 such that G 0 (x 0 ) = 0, i.e. a curve c : [s a , s b ] → G -1 (0) for which x 1 = c(s b ) is a zero of the function G 1 . These zero paths are also known as deformation paths.

A handling and fast continuation algorithm is given by the discrete continuation with acceleration step, which is presented as follows. One tries to find the zero points of G 1 via a convex continuation by discretizing the parameter λ as 0 = λ 0 < • • • < λ n c = 1 and solving the sequence of problems G(x, λ i ) = 0, i = 1, . . . , n c . The problem at the iteration i is initialized with the solution found at the iteration i -1. If the increment ∆λ i = λ i +1 -λ i is small enough, the solution x i corresponding to λ i is generally close to the solution of G(x, λ i +1 ) = 0. Thus, we can reduce ∆λ i till a solution of G(x, λ i +1 ) = 0 is found. However, if the increment ∆λ i has reduced too much, the iterative procedure may become lengthy. To overcome this, at each iteration, we can fix the length of ∆λ i by an acceleration procedure. The algorithm is given below.

Data: Solution x 0 of problem G 0 (x) = 0. Result: Solution y 1 of problem G 1 (x) = 0. begin Set i = 1, λ 0 = 0, λ Ol d = 0, ∆λ 0 = 1 and ε ∈ (0, 1] while λ i -1 ≤ 1 and ∆λ i -1 ≥ ε do Solve G(x, λ i -1 ) = 0 initialized with x i -1 .
if G(x, λ i -1 ) = 0 converges with y as solution then

x i = y , x Ol d = y , λ Ol d = λ i -1 if i > 1 and G(x, λ i -2 ) = 0 converged then ∆λ i = (1 -λ i -1 ) else ∆λ i = ∆λ i -1 end λ i = λ i -1 + ∆λ i else x i = x Ol d , ∆λ i = ∆λ i -1 2 , λ i = λ Ol d + ∆λ i end i -→ i + 1 end y 1 = x i -1 end
Algorithm 1: Discrete continuation algorithm with acceleration step.

Let us now analyze how continuation methods can solve the boundary value problem related to (OCP) and under which assumptions the problem is well-posed.

To provide global results, consider the version of (OCP) with M = R n , M f = {q f }, U = R m and with fixed final time t f . We are faced with a family of optimal control problems, parameterized by a parameter λ, that, using the infinite dimensional

Optimality Conditions and Numerical Methods

formulation (1.4), can be written in the form min

E t f ,λ (u)=q f C t f ,λ (u).
According to the Lagrange multiplier rule (1.5), if u λ is an optimal solution of the previous problem, there exists a non trivial couple (p λ ,

p 0 λ ) ∈ R n+1 satisfying ∂L t f ,λ ∂u (u λ , p λ , p 0 λ ) = p λ • d E t f ,λ (u λ ) + p 0 λ dC t f ,λ (u λ ) = 0 .
Assuming that there are no minimizing abnormal extremals in the problem, we can set p 0 λ = -1. Therefore, we are seeking a solution (u λ , p λ ) of

G(λ, u, p) = p • d E t f ,λ (u) -dC t f ,λ (u) E t f ,λ (u) -q f =   ∂L t f ,λ ∂u (u, p) E t f ,λ (u) -q f   = 0
when λ = 1, by starting from the solution of the shooting G(0, u, p) = 0 (which should be an easy task) and making λ converge to 1. Assume that G is smooth and let ( λ, u λ, p λ) be a zero of G. If the Jacobian of G with respect to (u, p), taken at the point ( λ, u λ, p λ), is invertible, then according to usual implicit function arguments, one can solve the equation G(λ, u λ , p λ ) = 0, and the solution (u λ , p λ ) depends in a smooth way on the parameter λ. Therefore, we only need to analyze the invertibility condition. The Jacobian matrix of G with respect to (u, p) is

Q t f ,λ d E t f ,λ (u) * d E t f ,λ (u) 0
where, in this case, Q t f ,λ is

∂ 2 L t f ,λ
∂ 2 u (u, p, p 0 ) restricted to ker

∂L t f ,λ
∂u and d E t f ,λ (u) * is the transpose of d E t f ,λ (u). It is easy to prove that this sensitivity matrix is invertible if and only if the linear mapping d E t f ,λ (u) is surjective and the quadratic form Q t f ,λ is nondegenerate. The surjectivity of d E t f ,λ (u) exactly means that the control u is not singular while, as long as we do not encounter any conjugate time along the continuation path, the extremals that are computed are locally optimal, which implies the nondegeneracy of Q t f ,λ . Therefore, we conclude that, as long as we do not encounter any minimizing singular control nor conjugate time along the continuation procedure, the continuation method is locally feasible, and the extremal solution (u λ , p λ ), which is locally computed as above, is smooth with respect to the parameter λ. These implicit function arguments permit to ensure the local feasibility of the continuation procedure. Nevertheless, the path may not be globally defined for every λ ∈ [0, 1]. Its global existence depends on the presence of singular minimizers and on the properness of the exponential mapping exp h u . From this, one has (see, e.g. [START_REF] Trélat | Optimal control and applications to aerospace: some results and challenges[END_REF]) Proposition 1.2. If there are no minimizing singular trajectories nor conjugate times over all the state space, the continuation procedure developed to solve the boundary value problem coming from the Maximum Principle applied to (OCP) with M = R n , M f = {q f }, U = R m and fixed final time t f is globally feasible for parameters λ ∈ [0, 1].

Chapter 1. Elements of Optimal Control

Problems with Control and State Constraints

Problem (OCP) is not general enough to describe the optimal control problems arising from our launch vehicle application. More specifically, we need to add constraints which combine control and state variables to the formulation. When this happens, the results concerning necessary conditions and indirect methods previously developed must be carefully adapted. In this section, we introduce and discuss optimal control problems with control and state constraints.

Consider the same quantities defined for problem (OCP) in Section 1.2. We introduce a smooth mixed control-state equality constraint c m e : R × M × R m → R r m e , a smooth mixed control-state inequality constraint c m i : R× M ×R m → R r m i and a smooth state constraint c s : R × M → R r s . A general nonlinear Optimal Control Problem with Control and State Constraints (OCP) m,s consists in minimizing the cost

C (t f , u) = g (t f , q(t f )) + t f 0 f 0 (t , q(t ), u(t )) d t such that q(t ) = f (t , q(t ), u(t )) , q(0) = q 0 , q(t f ) ∈ M f among all the controls u ∈ L ∞ ([0, t f ], R m ) satisfying, almost everywhere in [0, t f ],
c m e (t , q(t ), u(t

)) = 0 , c m i (t , q(t ), u(t )) ≤ 0 , c s (t , q(t )) ≤ 0
where the final time t f may be free or not. Remark that no constraints of type u(t ) ∈ U are considered because, concerning our applications, the set of control constraint U can be always parametrized, and then, it is incorporated directly within mixed control-state constraints. The existence of solutions of (OCP) m,s , under appropriate assumptions, is a well-established result (see, e.g. [START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF][START_REF] Lee | Foundations of optimal control theory[END_REF][START_REF] Cesari | Optimization -theory and applications: problems with ordinary differential equations[END_REF]).

Unlike the classical optimal control problem (OCP), necessary optimality conditions in the form of the Maximum Principle can be obtained only by considering additional assumptions and several crucial adaptations concerning the adjoint vector.

Assumption 1.1 (Mangasarian-Fromovitz Constraint Qualification). For every tuple (t , q, u) ∈ R × M × R m , denote by I (t , q, u) = { j | c j m i (t , q, u) = 0} the set of active mixed control-state inequality constraints. For every (t , q, u) ∈ R × M × R m , there are no a j ≥ 0 , j ∈ I (t , q, u) and b j , j = 1, . . . , r m e which are not all zero, such that j ∈I (t ,q,u)

a j ∂c j m i ∂u (t , q, u) + r m e j =1 b j ∂c j m e ∂u (t , q, u) = 0 .
Mixed constraints c m e and c m i satisfying this condition are said to be regular.

Problems with Control and State Constraints

General Control and State Constraints

In what follows, we provide the suitable adaptations to develop the Maximum Principle in the presence of general state and control constraints. Indeed, in this case, further multipliers appear in the Maximum Principle formulation, which obliges to adapt the Hamiltonian convention introduced in the formalism related to (OCP).

Consider a smooth map c m (t , q, u). Fix an integer r m , as well as a positive time t f , and assume to have a bounded function µ m ∈ L ∞ ([0, t f ], R r m ) and a positive measure

d µ s ∈ M ([0, t f ], R r s ). Finally, consider a control u ∈ L ∞ ([0, t f ], R m ), the related curve q u (•) = exp f u (•; 0, q 0 ) and, for t ∈ [0, t f ], take a local chart (V, ϕ) centered at q u (t ) ∈ M .
Since, for every (t , q, u) ∈ R × M × R m , the quantity µ m (t ) • ∂c m ∂q (t , q, u) can be interpreted as an element that belongs to T * q M in a standard way, we define, for every t ∈ [0, t f ], the covector curve

t f t (exp f u (s; t , •)) * q u (s) • µ m (s) • ∂c m ∂q (s, q u (s), u(s)) d s : [0, t f ] → T * M (1.15) t → n l =1 t f t (exp f u (s; t , •)) * q u (s) • µ m (s) • ∂c m ∂q (s, q u (s), u(s)) ∂ ∂x l q u (t ) d s d x l (q u (t ))
where x i are the local coordinates of the chart (V, ϕ). Let x l a and x l b be local coordinates of two different charts of q u (t ). Since d x l a (q u (t ))

= n j =1 ∂x l a ∂x j b (q u (t ))d x j b (q u (t ))
, it is straightforward that (1.15) is a globally well-defined absolutely continuous curve (with respect to the Whitney topology of C ∞ (T * M )) on the cotangent bundle.

Conversely, since for every (t , q) ∈ R × M and every j = 1, . . . , r s , the quantity ∂c j s ∂q (t , q) can be interpreted as an element that belongs to T * q M in a standard way, we define, for every t ∈ [0, t f ], another covector curve

t f t (exp f u (s; t , •)) * q u (s) • d µ s (s) • ∂c s ∂q (s, q u (s)) : [0, t f ] → T * M (1.16) t → n l =1 r s j =1 t f t (exp f u (s; t , •)) * q u (s) • ∂c j s ∂q (s, q u (s)) ∂ ∂x l q u (t ) d µ j s (s) d x l (q u (t ))
where x i are the local coordinates of the chart (V, ϕ) and the integral in (1.16) is the classical Riemann-Stieltjes integral. Exactly as done previously, it is straightforward to verify that (1.16) is a globally well-defined curve of bounded variation in T * M . For every p t f ∈ T * q u (t f ) M , the previous computations allow to define the curve

p : [0, t f ] → T * M : t → (exp f u (t f ; t , •)) * q u (t f ) • p t f + (1.17) t f t (exp f u (s; t , •)) * q u (s) •µ m (s)• ∂c m ∂q (s, q u (s), u(s)) d s+ t f t (exp f u (s; t , •)) * q u (s) •d µ s (s)• ∂c s ∂q (s, q u (s)) .

Chapter 1. Elements of Optimal Control

The curve of bounded variation defined by (1.17) is the good candidate as adjoint vector for the Maximum Principle related to (OCP) m,s . More precisely, if h(t , q, p, p 0 , u) denotes the Hamiltonian defined by (1.6),the following Maximum Principle holds.

Theorem 1.3. Let q(•) be an optimal trajectory for (OCP) m,s , associated to the control u(•) on [0, t f ]. Under Assumption 1.1, there exist a nonpositive scalar p 0 , a curve of bounded variation p : [0, t f ] → T * M , for which p(t ) ∈ T * q(t ) M , two bounded functions

µ m e ∈ L ∞ ([0, t f ], R r m e ), µ m i ∈ L ∞ ([0, t f ], R r m i )
where µ m i is nonpositive, and nonincreasing functions µ j s , j = 1, . . . , r s (generating measures d µ j s ) such that, almost everywhere in [0, t f ], the following relations hold:

• Nontriviality Condition

|p 0 | + max j ∈{0,...,n} ξ j • p(t f ) + t f 0 µ m i (t ) d t + r s j =1 |µ j s (t f ) -µ j s (0)| > 0 • Adjoint Equations q(t ) = ∂h ∂p (t , q(t ), p(t ), p 0 , u(t )) , p(t ) = (exp f u (t f ; t , •)) * q(t f ) • p(t f ) + t f t (exp f u (s; t , •)) * q(s) • µ m e (s) • ∂c m e ∂q (s, q(s), u(s)) d s + t f t (exp f u (s; t , •)) * q(s) • µ m i (s) • ∂c m i ∂q (s, q(s), u(s)) d s + t f t (exp f u (s; t , •)) * q(s) • d µ s (s) • ∂c s ∂q (s, q(s))
• Maximality Condition h(t , q(t ), p(t ), p 0 , u(t )) ≥ h(t , q(t ), p(t ), p 0 , u)

for every u : c m e (t , q(t ), u) = 0 , c m i (t , q(t ), u) ≤ 0

• Stationarity Condition ∂h ∂u (t , q(t ), p(t ), p 0 , u(t )) + µ m e (t ) ∂c m e ∂u (t , q(t ), u(t )) + µ m i (t ) ∂c m i ∂u (t , q(t ), u(t )) = 0 • Complementarity Slackness Conditions µ j m i (t )c j m i (t , q(t ), u(t )) = 0 for every j = 1, . . . , r m i , t f 0 c j s (t , q(t )) d µ j s (t )
= 0 for every j = 1, . . . , r s

Problems with Control and State Constraints

• Transversality Conditions

If M f is a submanifold of M , locally around q(t f ), then the adjoint vector can be built in order to satisfy

p(t f ) -p 0 ∂g ∂q (t f , q(t f )) ⊥ T q(t f ) M f
and, moreover, if the final time t f is free, one has

max c m e (t f , q(t f ), u) = 0 c m i (t f , q(t f ), u) ≤ 0 h(t f , q(t f ), p(t f ), p 0 , u) = -p 0 ∂g ∂t (t f , q(t f )) .
Remark 1.2. To avoid making the notations too cumbersome, in Theorem 1.3, the adjoint equation related to the evolution of p(•) is given via integral formulations. However, it is not difficult to adapt the previous arguments and provide full differential adjoint equations (we refer to [START_REF] Bourdin | Note on pontryagin maximum principle with running state constraints and smooth dynamics-proof based on the ekeland variational principle[END_REF] for any omitted detail).

The main difference with respect to the classical case is that the adjoint vector provided by this Maximum Principle may not be an absolutely continuous function. Indeed, if the jump and the singular parts of the measures d µ i s , i = 1, . . . , r s are not zero, p may exhibit discontinuities. Moreover, a serious difficulty arises, that is, no knowledge concerning the evolution of multipliers µ m e , µ m i and µ s is provided, making impossible the integration of the new adjoint equations without further information. The principal role of Assumption 1.1 consists in providing the boundness of the multipliers µ m e , µ m i related to the mixed constraints. Indeed, these multipliers are originally recovered as functionals in (L ∞ ) * , therefore, their structure may be very complex. However, thanks to Assumption 1.1, one actually proves that µ m e , µ m i ∈ (L 1 ) * . It is interesting to note that a slightly weaker version of Theorem 1.3 can be obtained without considering Assumption 1.1 (see, e.g. [START_REF] Dmitruk | Maximum principle for the general optimal control problem with phase and regular mixed constraints[END_REF]). Nevertheless, this result provides a family of partial Maximum Principles which does not reduce to the formalism of the usual Maximum Principle and the multiplicity of the conditions is the price for the nonregular mixed constraints (for further details, we refer to [START_REF] Dubovitskii | Necessary conditions for a weak extremum in optimal control problems with mixed constraints of the inequality type[END_REF][START_REF] Dubovitskii | Necessary conditions of a weak minimum in the general optimal control problem[END_REF]). Even if Theorem 1.3 is a well-established result (see, e.g. [START_REF] Dmitruk | Maximum principle for the general optimal control problem with phase and regular mixed constraints[END_REF] for the version of Theorem 1.3 in the case M = R n ), we provide a proof in Appendix A. This because the geometric version of Theorem 1.3 does not appear explicitly in the literature

Mixed Control-State Constraints

In this section we consider optimal control problems with only mixed control-state constraints, i.e. problems (OCP) m,s satisfying c s = 0. In this case, Theorem 1.3 simpli-

Chapter 1. Elements of Optimal Control

fies giving more regularity to the adjoint vector and the Lagrange multipliers. Moreover, unlike the previous case, a handling Hamiltonian formalism can be adopted.

For sake of clarity, we restate the context. A general nonlinear Optimal Control Problem with Mixed Control-State Constraints (OCP) m consists in minimizing the cost

C (t f , u) = g (t f , q(t f )) + t f 0 f 0 (t , q(t ), u(t )) d t (1.18) such that q(t ) = f (t , q(t ), u(t )) , q(0) = q 0 , q(t f ) ∈ M f (1.19) among all the controls u ∈ L ∞ ([0, T ], R m ) satisfying, almost everywhere in [0, t f ],
c m e (t , q(t ), u(t )) = 0 , c m i (t , q(t ), u(t )) ≤ 0 .

(1.20)

Consider a smooth vector function c m (t , q, u) and introduce the following family of nonautonomous Hamiltonians

h m (t , q, p, µ, u) = h m,(t ,u,µ) (p) = h(t , q, p, u) + µ • c m (t , q, u) = 〈p, f (t , q, u)〉 + µ • c m (t , q, u)
where t , u and µ are considered as parameters. Fix an integer r m , as well as a positive time t f , and let

u ∈ L ∞ ([0, t f ], R m ) a control function and µ ∈ L ∞ ([0, t f ], R r m )
p(t ) = (exp f u (t f ; t , •)) * q u (t f ) • p t f + t f t (exp f u (s; t , •)) * q u (s) • µ m (s) • ∂c m ∂q (s, q u (s), u(s)) d s
for every p t f ∈ T * π(p t f ) M . Since µ m is bounded, this curve is absolutely continuous and represents the suitable adjoint vector related to (OCP) m . To state the Maximum Principle in the presence of mixed control-state constraints, for every t ∈ R, every u ∈ R m and every µ e ∈ R r m e , µ i ∈ R r m i , define the Hamiltonian related to (OCP) m as h m (t , q, p, p 0 , µ e , µ i , u) = h m,(t ,u,µ) (p, p 0 ) = (1.22) h(t , q, p, p 0 , u) + µ e • c m e (t , q, u)

+ µ i • c m i (t , q, u) = 〈p, f (t , q, u)〉 + p 0 f 0 (t , q, u) + µ e • c m e (t , q, u) + µ i • c m i (t , q, u) .

Problems with Control and State Constraints

Theorem 1.4. Let q(•) be an optimal trajectory for (OCP) m , associated to the control u(•) on [0, t f ]. Under Assumption 1.1, there exist a nonpositive scalar p 0 , an absolutely continuous curve p : [0, t f ] → T * M , for which p(t ) ∈ T * q(t ) M , and two bounded functions

µ m e ∈ L ∞ ([0, t f ], R r m e ), µ m i ∈ L ∞ ([0, t f ], R r m i )
where µ m i is nonpositive, with (p(•), p 0 ) = 0 and such that, almost everywhere in [0, t f ], the following relations hold:

• Adjoint Equations q(t ) = ∂h m ∂p (t , q(t ), p(t ), p 0 , µ m e (t ), µ m i (t ), u(t )) , (1.23)
ṗ(t ) = - ∂h m ∂q (t , q(t ), p(t ), p 0 , µ m e (t ), µ m i (t ), u(t ))
• Maximality Condition

h(t , q(t ), p(t ), p 0 , u(t )) ≥ h(t , q(t ), p(t ), p 0 , u) (1.24)
for every u : c m e (t , q(t ), u) = 0 , c m i (t , q(t ), u) ≤ 0

• Stationarity Condition ∂h m ∂u (t , q(t ), p(t ), p 0 , µ m e (t ), µ m i (t ), u(t )) = 0 (1.25)
• Complementarity Slackness Conditions

µ j m i (t )c j m i (t , q(t ), u(t )) = 0 for every j = 1, . . . , r m i (1.26)

• Transversality Conditions

If M f is a submanifold of M , locally around q(t f ), then the adjoint vector can be built in order to satisfy

p(t f ) -p 0 ∂g ∂q (t f , q(t f )) ⊥ T q(t f ) M f (1.27)
and, moreover, if the final time t f is free, one has

max c m e (t f , q(t f ), u) = 0 c m i (t f , q(t f ), u) ≤ 0 h(t f , q(t f ), p(t f ), p 0 , u) = -p 0 ∂g ∂t (t f , q(t f )) . ( 1

Numerical Difficulties Due to Control and State Constraints

Even if the adjustments to run direct methods on (OCP) m,s and (OCP) m are provided straightforwardly by discretizing the constraints depending on the state, by analyzing both Theorem 1.3 and Theorem 1.4, it is easily understood that adapting numerical methods such as shooting or multi-shooting algorithms becomes complicated when considering control and state constraints. Indeed, even if we assume to deal with regular controls, the presence of the multipliers µ m e , µ m i and µ s prevents from integrating the adjoint equations. As pointed out previously, this is due to the fact that, usually, no knowledge concerning the evolution of these multipliers is provided.

Obtaining rigorous and useful information on the evolution and the regularity of µ m e , µ m i and µ s may be arduous and has been the object of many studies in the existing literature, both from theoretical (for example, high-order analysis [START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF][START_REF] Arutyunov | The maximum principle for optimal control problems with state constraints by R.V. Gamkrelidze: revisited[END_REF][START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF]) and numerical point of views (for example, aerospace applications [4,[START_REF] Hermant | Optimal control of the atmospheric reentry of a space shuttle by an homotopy method[END_REF]). Even if indirect methods to solve (OCP) m,s and (OCP) m have already been proposed, they are able to work under very particular assumptions and make the numerical computations more demanding, often losing the fast convergence of the original shooting method (for further details on these procedures, we refer to [START_REF] Bonnans | Well-posedness of the shooting algorithm for state constrained optimal control problems with a single constraint and control[END_REF][START_REF] Bonnans | Second-order analysis for optimal control problems with pure state constraints and mixed control-state constraints[END_REF][START_REF] Bonnans | Stability and sensitivity analysis for optimal control problems with a first-order state constraint and application to continuation methods[END_REF][START_REF] Bonnans | No-gap second-order optimality conditions for optimal control problems with a single state constraint and control[END_REF]).

Problems with Control and State Delays

We conclude this chapter by introducing necessary optimality conditions and difficulties of related indirect methods for optimal control problems with control and state delays. The interest of introducing this kind of problems arises from the fact that, in the context of our launch vehicle application, delays coming from model refinement often occur. Therefore, related to our main challenge of solely exploiting indirect methods, we need to understand whether it is possible to efficiently solve optimal control problems with control and state delays by shooting-type procedures.

Maximum Principle for Problems with Delays

For matter of concision, in what follows, all the concerned results are developed considering systems evolving in the Euclidean space, i.e. M = R n , and subject to pure control constraints. This is not limiting to address our launch vehicle application.

A nonautonomous vector field with delays on R n is a continuous vector function

f : R × R 2n × R 2m → R n : (t , x, y, u, v) → f (t , x, y, u, v)
which is smooth w.r.t. its second and third variables. Without loss of generality, we assume that any considered vector field with delays has a compact support. Fix a

Problems with Control and State Delays

positive value ∆ and consider an initial state function

Φ 1 ∈ C 0 ([-∆, 0], R n ) as well as an initial control function Φ 2 ∈ L ∞ ([-∆, 0), R m ). For every couple of delays τ = (τ 1 , τ 2 ) ∈ [0, ∆] 2 and every control u ∈ L ∞ l oc ([-∆, ∞), R m ) such that u| [-∆,0) (•) = Φ 2 (•)
, the following dynamical system with delays

ẋ(t ) = f (t , x(t ), x(t -τ 1 ), u(t ), u(t -τ 2 )) , x| [-∆,0] (•) = Φ 1 (•) (1.
τ ∈ [0, ∆] 2 , u ∈ L ∞ l oc ([-∆, ∞), R m ), Φ 1 ∈ L ∞ ([-∆, 0], R n ).

Consider a continuous integral cost function with delays

f 0 : R × R 2n × R 2m → R : (t , x, y, u, v) → f 0 (t , x, y, u, v)
which is smooth w.r.t. its second and third variables. Moreover, take subsets M f ⊆ R n and U ⊆ R m . For every couple of constant delays τ = (τ 1 , τ 2 ) ∈ [0, ∆] 2 , a general nonlinear Optimal Control Problem with Control and State Delays (OCP) τ consists in minimizing the cost

C τ (t f , u) = t f 0 f 0 (t , x(t ), x(t -τ 1 ), u(t ), u(t -τ 2 )) d t such that ẋ(t ) = f (t , x(t ), x(t -τ 1 ), u(t ), u(t -τ 2 )) , x| [-∆,0] (•) = Φ 1 (•) , x(t f ) ∈ M f among all the controls u ∈ L ∞ ([-∆, t f ], R m ) satisfying u| [-∆,0) (•) = Φ 2 (•) , u(t ) ∈ U
almost everywhere in [0, t f ], where the final time t f may be fixed or not. No general results on the existence of solutions of (OCP) τ are available and, as far as we know, only the analysis proposed by [START_REF] Nababan | A filippov-type lemma for functions involving delays and its application to time-delayed optimal control problems[END_REF] treats this issue. However, the given existence result turns out to be only partially correct. We return on this controversy in Chapter 6.

Providing a Maximum Principle for (OCP) τ is possible, up to some adaptations of the adjoint equations. The first proofs of these necessary conditions appear in [START_REF] Kharatishvili | The maximum principle in the theory of optimal processes involving delay[END_REF][START_REF] Kharatishvili | A maximum principle in extremal problems with delays[END_REF], where only either state delays or control delays are considered. The proofs concerning more sophisticated cases have been developed later in [START_REF] Halanay | Optimal controls for systems with time lag[END_REF][START_REF] Banks | Necessary conditions for control problems with variable time lags[END_REF][START_REF] Göllmann | Theory and applications of optimal control problems with multiple time-delays[END_REF][START_REF] Guinn | Reduction of delayed optimal control problems to nondelayed problems[END_REF][START_REF] Colonius | Optimal control of functional differential systems[END_REF][START_REF] Angell | On the necessary conditions for optimal control of retarded systems[END_REF]. Since we need to work directly with the variation vectors of (OCP) τ , we sketch in Chapter 7 the proof of the Maximum Principle for optimal control problems with control and state delays by means of needle-like variations, which does not appear directly in the litterature. We define the Hamiltonian related to (OCP) τ , as h(t , x, y, p, p 0 , u, v) = 〈p, f (t , x, y, u, v)〉 + p 0 f 0 (t , x, y, u, v) .

Theorem 1.5. Let x(•) be an optimal trajectory for (OCP) τ , associated to the control u(•) on [-∆, t f ].

There exist a nonpositive scalar p 0 and an absolutely continuous curve p : [0, t f ] → R n , with (p(•), p 0 ) = 0, and such that, almost everywhere in [0, t f ], the following relations hold:

• Adjoint Equations ẋ(t ) = ∂h ∂p (t , x(t ), x(t -τ 1 ), p(t ), p 0 , u(t ), u(t -τ 2 )) , ṗ(t ) = - ∂h ∂x (t , x(t ), x(t -τ 1 ), p(t ), p 0 , u(t ), u(t -τ 2 )) -1 [0,t f -τ 1 ] (t ) ∂h ∂y (t + τ 1 , x(t + τ 1 ), x(t ), p(t + τ 1 ), p 0 , u(t + τ 1 ), u(t + τ 1 -τ 2 ))
• Maximality Condition

h(t , x(t ), x(t -τ 1 ), p(t ), p 0 , u(t ), u(t -τ 2 )) + 1 [0,t f -τ 2 ] (t )h(t + τ 2 , x(t + τ 2 ), x(t + τ 2 -τ 1 ), p(t + τ 2 ), p 0 , u(t + τ 2 ), u(t )) ≥ h(t , x(t ), x(t -τ 1 ), p(t ), p 0 , u, u(t -τ 2 )) + 1 [0,t f -τ 2 ] (t )h(t + τ 2 , x(t + τ 2 ), x(t + τ 2 -τ 1 ), p(t + τ 2 ), p 0 , u(t + τ 2 ), u)
for every u ∈ U

• Transversality Conditions

If M f is a submanifold of R n , locally around x(t f ), then the adjoint vector can be built in order to satisfy

p(t f ) ⊥ T x(t f ) M f
and, moreover, if the final time t f is free and both t f and t f -τ 2 are points of continuity of u(•), one has

h(t , x(t f ), x(t f -τ 1 ), p(t f ), p 0 , u(t f ), u(t f -τ 2 )) = 0 .
Remark 1.3. To state standard transversality conditions, one needs to assume that both t f and t f -τ 2 are points of continuity of the optimal control, because the evaluation of u(•) at t f and at t f -τ 2 is required. However, with slight modifications, similar transversality conditions are recovered (we refer to [START_REF] Boccia | Free time optimal control problems with time delays[END_REF] for further details).

Problems with Control and State Delays

Numerical Difficulties Due to Control and State Delays

From the arguments presented in Section 1.3.3, it is clear that adapting direct methods to solve (OCP) τ does not produce any obstacle. However, solving (OCP) τ from a numerical point of view by means of indirect methods becomes complex because a global information on the adjoint vector p is needed. This is explained as follows.

Assume that the optimal control u(•) is regular, i.e. it can be written as a function of x(•) and p(•) (by the Maximality Condition, see Section 1.3.1). Therefore, each iteration of a shooting method consists in solving the coupled dynamics coming from the adjoint equations, where a value of p(0) is provided. In the context of Theorem 1.5, this means that one has to solve Differential-Difference Boundary Value Problems (DDBVP), where both forward and backward terms of time appear within mixed type differential equations. The difficulty in solving DDBVP is the lack of global information which forbids a purely local integration by usual iterative methods for ordinary differential equations. Techniques to solve mixed type differential equations arising from DDBVP have been analyzed, such as analytical decompositions of solutions [START_REF] Rustichini | Functional differential equations of mixed type: the linear autonomous case[END_REF][START_REF] Mallet-Paret | Mixed-type functional differential equations, holomorphic factorization and applications[END_REF], and related numerical schemes [START_REF] Lima | Analytical and numerical investigation of mixed-type functional differential equations[END_REF][START_REF] Teodoro | Numerical approach of a nonlinear forward-backward equation[END_REF]. In these approaches, the dimension of the problem may drastically increase as much as the numerical accuracy increases.

The previous considerations show that, in order to initialize correctly a shooting method on (OCP) τ , a standard guess of the initial value of the adjoint vector p(0) is not sufficient, but rather, a good numerical guess of the whole function p(•) must be provided to make the procedure converge. This represents an additional difficulty with respect to the usual shooting method and it requires a global discretization of the adjoint equations, increasing considerably the dimension of the problem.

Rendezvous Problems

This chapter is devoted to introduce the guidance of endo-atmospheric launch vehicles as an optimal control problem. Related to the applications of interest for ONERA-The French Aerospace Lab, the attention is focused on simulating rendezvous problems, which consist in steering the launch vehicle from a given initial point to some final point whose spatial coordinates are fixed, but final orientations are not necessarily imposed, and this, by minimizing some final cost. The model of the endoatmospheric flight dynamics is carefully introduced and the considered optimality criteria are initially taken to be as general as possible for an intrinsic geometric analysis of the problem. From this general framework, two applications of interest for ONERA-The French Aerospace Lab, concerning missile interception, are derived. The chapter is splitted into two sections. In Section 2.1, the physical model simulating the dynamics of an endo-atmospheric launch vehicle system is provided. In particular, after having introduced the main frames in which we establish the equations of motion, the models of the environment and the forces acting on the vehicle are given. Section 2.2 describes the main optimal control problems on which we develop our analysis: a first generic problem and two real application frameworks.

Physical Problem and Dynamical Model

The aim of this section consists in providing the equations of motion for axial symmetric endo-atmospheric vehicles, which are the dynamical systems of interest for our rendezvous problems. Some standard simplifying assumptions and approximations are taken into account to recover handy, but still, nonlinear dynamics.

Fundamental Coordinate Systems

Throughout this thesis, we consider the following assumptions:

• Since the physically meaningful trajectories have bounded range and duration,

Chapter 2. Rendezvous Problems

we assume that the Earth is a sphere fixed in the inertial space, i.e. the angular velocity of the Earth and Coriolis-type forces are considered to be zero.

• We consider a class of launch vehicles modeled as a three-dimensional axial symmetric cylinder steered by a control system (based on either steering fins or a reaction control system). The principal body axis is denoted by b.

• The mass flow of the launch vehicle is a given non-negative function of the time as well as the magnitude of its thrust, function of the mass flow. The direction of the thrust coincides with the direction of the principal body axis.

We work with the help of two main frames:

1. The inertial frame (I, J , K) which is a fixed orthonormal frame centered at O, the center of the Earth, such that K is the north-south axis of the planet and the plane (I, J ) is chosen to satisfy the right-hand rule. This is the frame where the Newton's law of motion are valid.

Physical Problem and Dynamical Model

.1 shows the relations between frame (I, J , K) and frame (e L , e , e r ). The NED frame is expressed locally as a function of the inertial frame by the relations

       e L = -sin L cos I -sin L sin J + cos L K e = -sin I + cos J e r = -cos L cos I -cos L sin J -sin L K (2.1)
where L and are local Euler coordinates representing respectively the geocentric latitude and the longitude, such that the linear transformation from frame (I, J , K) to frame (e L , e , e r ) takes the form

R(L, ) =   -sin L cos -sin L sin cos L -sin cos 0 -cos L cos -cos L sin -sin L   ∈ SO(3) .
Moreover, the evolution of frame (e L , e , e r ) have the following closed-loop form

ėL = -˙ sin L e + L e r , ė = ˙ sin L e L + ˙ cos L e r , ėr = -L e L -˙ cos L e . (2.2)
It is important to note that, since (L, l ) are local Euler coordinates, they exhibit singularities when cos L is equal to zero. However, by assumption, the physically meaningful trajectories have bounded ranges and durations, so that, starting from neighborhoods of points satisfying cos L = 0, the targets possibly reached by the vehicle are far enough from the subset satisfying cos L = 0. In any case, this is not limiting because transformations of coordinates can be operated. We return on this in Chapter 3. Chapter 2. Rendezvous Problems

Environmental and Dynamical Modeling

By assumption, we model the launch vehicle as an axial symmetric rigid body of mass m. Its motion is described by the evolution of the state variables (r, v,b,ω), where r = xI + yJ +zK is the trajectory of its center of gravity G, v = ẋI + ẏJ + żK denotes its velocity and the normal vector b corresponds to the principal body axis, whose time evolution is defined by the angular velocity ω. We proceed firstly by analyzing the environment, and thereafter, by providing the models of physical quantities.

Air Density

All the concerned aerodynamical forces depend on the interaction between the atmosphere and the vehicle. It is crucial to provide a model of the air density. We present a concise introduction and further details on atmospheric models can be found in [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF][START_REF] Anderson | Fundamentals of aerodynamics[END_REF][START_REF] Pharpatara | 3-d trajectory planning of aerial vehicles using rrt[END_REF]. In the following, we define the altitude as the scalar quantity h = rr T , where r T = r T = 6378, 145 km is the radius of the Earth. In the lower earth atmosphere (h < 35 km), the density of air and the atmospheric pressure decrease exponentially with h and approach zero at 35 km.

Physical Problem and Dynamical Model

The previous model can be put under the following closed-loop form which describes the temperature T atm and the pressure p atm of the atmosphere

T atm =       
p atm =         
101.29 For our numerical simulations, since the altitude is included between 1000 m and 20 km, denoting by ρ 0 the air density at the standard atmosphere at the sea level and by h r a fixed reference altitude, expression (2.3) is simplified as

ρ(h) = ρ 0 e -h/h r kg/m 3 .

Drag and Lift

The drag on the vehicle is the force exerted on it by the medium through which it is moving, in this case the air. Since the drag is generated by the motion of the vehicle through the air, it is naturally directed opposite to the velocity vector v.

Chapter 2. Rendezvous Problems

Usually, the effect of the drag force is lumped into one coefficient C D called drag coefficients which depends mainly on the angle of attack α, which is the angle between the velocity of the vehicle v and its principal body axis b (see Figure 2.4). The drag force is defined in terms of the drag coefficient as (see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF][START_REF] Anderson | Fundamentals of aerodynamics[END_REF][START_REF] Carlucci | Ballistics: theory and design of guns and ammunition[END_REF])

D = - 1 2 ρ(h)SC D (α) v v
where S denotes a reference area that sums up all the pointwise contributions of the interaction between the atmosphere and the vehicle. The drag coefficient is not given directly. It can be found by using the axial force coefficient C A and the normal force coefficient C N (in body frame, see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF][START_REF] Anderson | Fundamentals of aerodynamics[END_REF][START_REF] Carlucci | Ballistics: theory and design of guns and ammunition[END_REF]), both functions of the angle of attack α and other parameters. The relation connecting them reads

C D (α) = C A cos α +C N sin α . (2.4)
Under second-order approximation, the coefficient C N takes the form

C N = C N α α + C N 2 α|α|
where C N α and C N 2 are constant normal force coefficients. On the other hand, the expression of C A is much more complex and a common explicit model is given by the following conic approximation (see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF][START_REF] Anderson | Fundamentals of aerodynamics[END_REF][START_REF] Pharpatara | 3-d trajectory planning of aerial vehicles using rrt[END_REF])

C A =                  2 sin 2 α c +C A 2 α 2 , M < 0.5 2 sin 2 α c (k 1 + k 2 sin α c ) k 3 + k 4 sin α c + k 5 κ(M -0.5) +C A 2 α 2 , 0.5 ≤ M < 1.5 2 sin 2 α c (k 6 + M 2 -1 sin α c ) k 7 + M 2 -1 sin α c + κ M 2 +C A 2 α 2 , M ≥ 1.5
where α c represents the cone angle, C A 2 is the induced axial force coefficient, M is the Mach number, k 1 , . . . , k 7 represent design values depending on the missile configuration, and there holds κ = 0 for powered flight, and κ = 1 for coasting flight.

The lift is defined as the aerodynamic force that acts orthogonally to the velocity vector. The explicit expression of the lift is provided as (see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF][START_REF] Anderson | Fundamentals of aerodynamics[END_REF][START_REF] Carlucci | Ballistics: theory and design of guns and ammunition[END_REF])

L = 1 2 ρ(h)SC L (α)
where S is the same reference area appearing in the the drag force, while C L is a vector quantity whose orientation coincides with the direction of the product v ∧ (b ∧ v). As done for the drag, we can lump into one scalar coefficient C L = C L , called lift coefficient, the effect of the lift force, so that the norm of L satisfies

L = 1 2 ρ(h)SC L (α) v 2

Physical Problem and Dynamical Model

and C L (α) depends on coefficients C A and C N as follows

C L (α) = C N cos α -C A sin α . (2.5)
The complex structure of coefficients C A and C N can make the equations of motion very difficult to treat from an analytical point of view. On the other hand, considering endo-atmospheric applications, the vehicle can be stabilized only if the velocity v assumes its values inside a cone around the body axis b, of maximal amplitude α max .

The angle α max is called maximal angle of attack. In this case, the expressions of C A and C N can be simplified by considering the following assumption.

Assumption 2.1. For points (ε, x) ∈ R + × R such that (1 + ε)x 2 ≤ sin 2 (α max ) (α max > 0
is considered constant and small enough), the following approximations hold:

sin x ∼ = x , cos x ∼ = 1 -x 2 /2 , 1 -(1 + ε)x 2 ∼ = 1 -(1 + ε)x 2 /2 .
The last expression of this assumption will be crucial in Chapter 3 to derive optimal controls. From (2.4) and (2.5), Assumption 2.1 make us able to write C D (α), C L (α) as

C D (α) = C D 0 +C D α sin 2 α , C L (α) = C L α sin α (2.6)
where coefficients C D 0 , C D α and C L α are considered to be constant. This allows to contract the expression of the forces D and L, for a more handling treating of the dynamical equations, as follows (see, e.g. [4,[START_REF] Miele | Optimal trajectories for aeroassisted orbital transfer[END_REF]90,[START_REF] Pucci | Nonlinear feedback control of axisymmetric aerial vehicles[END_REF])

D m = -d (h, m) + ηc m (h, m) sin 2 α v v , L m = c m (h, m) v ∧ (b ∧ v) (2.7)
where d and c m denotes respectively the normalized drag coefficient and the normalized lift coefficient, η is an efficiency factor (which is assumed to be constant), and the following expressions hold

d (h, m) = d 0 (m)e -h/h r , c m (h, m) = c 0 (m)e -h/h r .
It is important to note that, thanks to the relation v | sin α| = b ∧ v , the drag and the lift can be expressed as implicit functions of the state variables (r, v,b,ω).

The drag and lift forces do not act on the center of gravity of the vehicle, but rather, on its center of pressure P , that is, the virtual point on which the resultant of all infinitesimal pressure forces acts. Usually, the center of pressure does not coincide with the center of gravity, therefore, turning moments arise. More precisely, drag and lift provide the overturning moment, which writes (see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF][START_REF] Anderson | Fundamentals of aerodynamics[END_REF][START_REF] Carlucci | Ballistics: theory and design of guns and ammunition[END_REF])

M Over = 1 2 ρ(h)SDC Over (α) v (v ∧ b)
where S is the same reference area for the drag and the lift, D is the largest diameter of the cylindrical vehicle and C Over ≥ 0 is an aerodynamical coefficient whose expression is function of axial and normal force coefficients C A , C N (see, e.g. [START_REF] Carlucci | Ballistics: theory and design of guns and ammunition[END_REF]).

Chapter 2. Rendezvous Problems

Further Aerodynamical Contributions

We introduce one last relevant aerodynamical contribution: the pitch damping force. Pitch damping is the tendency of the vehicle to cease its pitching motion due to air resistance. When the vehicle is spinned, it slows down because of the sticking of the fluid to the surface and the resultant viscous action. On the other hand, when the vehicle mounts until the bearing is transverse to the principal body axis and spins, one observes the viscous action slowing the vehicle down; however, this is overwhelmed by the pressure forces that retard the motion and the vehicle spins down much faster. This combination of forces is called pitch damping and, under appropriate assumptions, the resultant can be written as (see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF][START_REF] Anderson | Fundamentals of aerodynamics[END_REF][START_REF] Carlucci | Ballistics: theory and design of guns and ammunition[END_REF])

F P i t ch = D 2 ρ(h)SC F P i t ch v d b d t
where D is the largest diameter of the cylindrical vehicle, S is the same reference area appearing in the expression of the drag and the lift and C F P i t ch is a coefficient depending on many physical parameters. Generally, the pitch damping force is neglected from the equations of motion since its magnitude is considerably smaller than other considered forces (see, e.g. [START_REF] Carlucci | Ballistics: theory and design of guns and ammunition[END_REF]). Nevertheless, the moment caused by the pitch damping is frequently significant. It is described as follows (see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF][START_REF] Anderson | Fundamentals of aerodynamics[END_REF][START_REF] Carlucci | Ballistics: theory and design of guns and ammunition[END_REF])

M P i t ch = D 2 2 ρ(h)SC M P i t ch v b ∧ d b d t
and the coefficient C M P i t ch ≥ 0 may depend on several physical parameters as well.

Gravity, Thrust and Control System Effect

The gravity is taken as the usual gradient of a hyperbolic potential, because we approximate the Earth as a sphere. Its expression reads

G m = -g (r) r r = - µ 0 r 2 r r
where µ 0 = 3.986 • 10 14 N•m 2 /kg is the gravitational constant related to the Earth.

The vehicle is pushed by a thrust varying with the time. The thrust force reads

T m = f T (t ) m b = I s g (r T )q(t ) + (P e -P a )S T m b
where I s is the specific impulse, q is the mass flow, S T is the perpendicular section of the nozzle and P e , P a are the ejection pressure and the ambient pressure, respectively. For our analysis, we assume the contribution of the term (P e -P a )S T to be

Physical Problem and Dynamical Model

negligible. Remark that f T models also the contribution of the term ṁv, arising from the equations of motion (compare also with the dynamical system (2.8) below).

The control system that steers the vehicle acts by a force denoted F C S , which allows to control the translation and the rotation of the rigid body. Concerning our applications, the magnitude of this force is much weaker than the contribution of the drag and the lift, so it does not appear within the moment equation. Nevertheless, its torque has a strong effect on the dynamics. Denoting by Q the point of the vehicle at which the control system is placed, this torque writes

M C S = QG ∧ F C S .
The quantity QG can be expressed as function of the state variables (r, v,b,ω). Note that, usually when considering axial symmetric vehicles, the point Q lies along the principal body axis, so that QG = QG b and QG coincides with the lever arm.

Equations of the Dynamics and Motion Constraints

From the previous analysis, we define the rigid body motion of a launch vehicle by the following system of equation (see, e.g. [4,[START_REF] Pucci | Nonlinear feedback control of axisymmetric aerial vehicles[END_REF][START_REF] Hérissé | Asservissement et navigation autonome d'un drone en environnement incertain par flot optique[END_REF])

                           ṙ = v v = f (t , r,v,b) = f T (t ) m b -g (r) r r -d + ηc m b ∧ v v 2 v v + c m v ∧ (b ∧ v) Ṙ = Rω ∧ , b = R • (1, 0, 0) d d t (I G ω) = -ω ∧ I G ω + 1 2 ρ(h)SDC Over (α) v (v ∧ b) + d 2 ρSC M P i t ch v 2 b ∧ ω + QG ∧ F C S (2.8)
where I G denotes the inertial matrix of the vehicle at G, and we have introduced the rotational matrix R as auxiliary variable. Implicitly, these equations contain the evolution of the mass ṁ(t ) = -q(t ), where q is the mass flow. Since the evolution of q is known a priori, the evolution of I G is known as well. The dynamical system (2.8) is known as control and guidance system. Since the principal body axis is a normal vector, the natural configuration manifold, within which the state variables

(r, v,b,ω) evolve, is R 6 \ {0} × S 2 × R 3 .
Once the evolution in time of the force F C S (•) is given, the feasible trajectories of an endo-atmospheric launch vehicle are computed by (2.8).

Remark 2.1. One makes v evolve in R 3 \ {0} because, when v approaches 0, the dynamical model obtained by (2.8) is no more valid and other physical contributions emerge, modifying drastically the moment equation. Moreover, upper and lower bounds on the magnitude of r should be adopted since the model of the air density taken into Chapter 2. Rendezvous Problems account could not be fine enough for high altitudes. However, the physics of the dynamical system prevents from reaching too large distances and forces the bound r ≥ r T . From these remarks, without loss of generality, we assume that (r, v) ∈ R 6 \ {0}.

The dynamical system (2.8) turns out to be only partially appropriate to describe the complete evolution of the trajectories of the launch vehicle. Indeed, rendezvous problems must consider constraints arising from stability and structural designs.

The most important class of constraints comes from controllability issues. Indeed, in order to control efficiently the vehicle, one has to ensure a good stabilization by forcing the velocity v to take values inside a cone around the principal body axis b, of maximal amplitude α max . This consists in forcing the constraints (see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF])

cos α = v • b v ≥ 0 , b ∧ v v = | sin α| =≤ sin α max . (2.9)
We stress on the fact that the previous constraints are crucial to correctly control the endo-atmospheric launch vehicle and can neither be removed nor approximated.

Another bound which deserves to be mentioned is the load factor constraint. During endo-atmospheric flights, strong structural strains due to the aerodynamical forces may arise, thus jeopardizing the success of the mission or, at worst, risking the loss of the vehicle. To prevent this, the load factor due to aerodynamical forces must be limited according to (see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF][START_REF] Mccormick | Aerodynamics, aeronautics, and flight mechanics[END_REF])

ρ v 2 SC N (α) 2mg (r) ≤ load max = const (2.10)
where S is the same reference area appearing in the expression of the drag and the lift, and C N is the normal force coefficient. Nevertheless, for our analysis, we neglect the presence of constraint (2.10). This can be justified by the fact that, for our specific endo-atmospheric rendezvous problems, the maximal load factor load max takes large enough values so that (2.10) is automatically satisfied. Moreover, procedures adding constraint (2.10) by homotopy could be considered at a later time (see, e.g. [START_REF] Zhu | Contrôle optimal de l'attitude d'un lanceur[END_REF]).

Optimal Control Problems

Once the motion of the considered vehicle is established by dynamics (2.8) and constraints (2.9), we look for trajectories satisfying some optimality criteria. Three optimal control problems are analyzed. The first one represents an abstract launch vehicle framework, on which a detailed geometric analysis is carefully developed in Chapter 3. The remaining two problems are special instantiations of the first one. They model a particular interception scenario and they are adopted to test the numerical approaches developed in Chapter 4, Chapter 5 and Chapter 6.

Optimal Control Problems

General Optimal Guidance Problem (GOGP)

In practical rendezvous applications, the rotational dynamics related to the torque equation of system (2.8) are faster than its translational dynamics. Therefore, it is possible and more convenient to divide and treat them separately (see, e.g. [START_REF] Hérissé | Asservissement et navigation autonome d'un drone en environnement incertain par flot optique[END_REF][START_REF] Khalil | Noninear systems[END_REF]).

If we consider only the translational dynamics within (2.8), the contribution of the control force F C S disappears and what is usually controlled is the principal body axis b that, in this context, is no more a state variable but it becomes the control and we denote it by u. The variable ω does not affect anymore the evolution of the state.

Denoting by q = (r, v) the new state variables, a General Optimal Guidance Problem (GOGP) consists in minimizing the general final cost

C (t f , u) = g (t f , q(t f )) = g (t f , r(t f ), v(t f )) (2.11) such that                q(t ) = ṙ(t ) v(t ) = v(t) f (t ,

r(t),v(t),u(t))

= f (t , q(t ), u(t))

q(t ) = (r(t ), v(t)) ∈ R 6 \ {0} q(0) = q 0 = (r 0 , v 0 ) , q(t f ) = (r(t f ), v(t f )) ∈ M f ⊆ R 6 \ {0} (2.12) among all the control u ∈ L ∞ ([0, t f ], R 3 ) satisfying, almost everywhere in [0, t f ], c 0 (u(t )) = u(t) 2 -1 = 0 (i.e. u(t) ∈ S 2 ) , c 1 (q(t ), u(t)) = c 1 (v(t ), u(t)) = - v(t) • u(t) v(t) ≤ 0 (2.13) c 2 (q(t ), u(t)) = c 2 (v(t ), u(t)) = u(t) ∧ v(t) v(t) 2 -sin α 2 max ≤ 0
and the final time t f may be free or not. Motivated by the discussion of the previous section, since we take into account constraint (2.9), we do not consider the load factor constraint in problem (GOGP). Nevertheless, we let the cost (2.11) and the final target set M f be quite general, with the only requirement that the following holds.

Assumption 2.2. At least one between the following two conditions holds:

A) The final time t f is free and ∂g ∂t (t , q) = 0 for every (t , q) ∈ R × M .

B) For every local chart (x 1 , . . . , x 6 ) of R 6 \ {0}, there exists a free final variable, say

x i , such that ∂g ∂x i = 0 and (0, . . . ,

1 i-th , . . . , 0) ∈ T q(t f ) M f (in local coordinates).
As we show later (see Chapter 3), Assumption 2.2 reveals to be useful to recover a closed-loop form of optimal controls as functions of the state and the adjoint vector.

Chapter 2. Rendezvous Problems

Optimal Interception Problem (OIP)

From a numerical viewpoint, we focus on a particular subclass of (GOGP): the endoatmospheric interception. The context can be summarized as follows (see, e.g. [6]). The target is represented by a supersonic missile whose position is assumed to be known at each time step. The objective consists in intercepting the target employing a ground-to-air/air-to-air missile by maximizing the chances to neutralize the threat. Many phases occur to control the vehicle. During the first phase, when the intercepting missile is launched, some predefined controllers stabilize its critical movements due to its too low velocity. When a certain threshold value of the velocity is reached, the mid-course phase controllers start to guide the vehicle; this phase is the longest one and needs a control strategy able to provide the best conditions to intercept the target when the interceptor reach some precomputed impact point. When this point is attained, the mid-course controller stops allowing the last, usually automatized, control strategy to guide the intercepting missile to the impact with the threat.

We focus on seeking optimal strategies for the mid-course phase. This problem can be represented mathematically introducing the Optimal Interception Problem (OIP), which consists in minimizing the cost

C (t f , u) = C 1 t f -v(t f ) 2 +C 2 t f 0 u(t) ∧ v(t) v(t) 2 d t (2.14)
where coefficients 0 ≤ C 1 ≤ 1 and C 2 ≥ 0 are constant, under the dynamical system (2.12) and constraints (2.13), with free final time t f . This cost is set up to maximize the chances to reach the target with reasonable delay and with maximal velocity, following the hit-to-kill principle: the more the magnitude of the velocity is significant at the impact point, the more probably the impact succeeds (see, e.g. [6]).

According to the discussion above, the final target represents some predicted impact point whose final position and direction of the final velocity are fixed. This is due to the fact that better chances to complete the mission arise if specific orientations of the final velocity are ensured. Therefore, the target manifold M f writes

M f = (r, v) ∈ R 6 \ {0} | r = r f , v • e r v = cos ψ 1 f , v • e L v = cos ψ 2 f , v • e v = sin ψ 2 f (2.15)
where r f is a fixed position and ψ 1 f ,ψ 2 f are fixed angles. Coherently with (2.14), the magnitude of the final velocity is left free. Assumption 2.2 is satisfied (see Chapter 3). The integral term of cost (2.14) prevents problem (OIP) to be properly interpreted as a particular guidance problem (GOGP), where only Mayer-type terms appear. However, its presence is merely used, in the engineering community, as an expedient to better stabilize the optimal control problem by minimizing the number of excessively

Optimal Control Problems

abrupt maneuvers. Since our general problem (GOGP) imposes a bound on the possible values of the angle α (by means of constraints (2.13)), we remove the integral term from cost (2.14), minimizing rather the following criterion

C (t f , u) = C 1 t f -v(t f ) 2 (2.16)
where C 1 ∈ {0, 1}. When (2.16) replaces (2.14), problem (OIP) may become harder to solve because quadratic expressions able to convexifying the problem are removed. For numerical simulations, we are interested in considering a solid-fuel propelled, skid-to-turn, axial symmetric missile of low weight. In terms of the aerodynamical quantities introduced in the previous sections, the parameters of the missile are

• c m 0 (m 0 ) = 0.00075 m -1 , d 0 (m 0 ) = 0.00005 m -1 • η = 0.442 , h r = 7500 m , α max = π 6 rad • q m 0 (t ) = 0.025 s -1 , t ≤ 20 s 0 , t > 20 s , f T m 0 (t ) = 37.5 m • s -2 , t ≤ 20 s 0 , t > 20 s .
We end the paragraph by remarking the importance of solving fast problem (OIP). Indeed, the predicted impact point may change rapidly during time in the case that the threat is able to elaborate fast evasive maneuvers, in which case, high speed updates of optimal intercepting strategies are compulsory (often, in a delay of 1-10 Hz).

Optimal Interception Problem with Delays (OIP) τ

The last problem that we consider is a variant of the optimal interception problem, in which a control on the rotational velocity is introduced, considering moreover some delay on mechanical information communication. This problem allows to simulate phenomena like the non-minimum phase problem, whose principle is as follows (see, e.g. [START_REF] Isidori | On the nonlinear equivalent of the notion of transmission zeros[END_REF][START_REF] Chavez | Analytical aeropropulsive/aeroelastic hypersonic-vehicle model with dynamic analysis[END_REF]). Consider a bank-to-turn or a skid-to-turn vehicle (such as the dynamical system considered for (OIP), see, e.g. [START_REF] Zarchan | Tactical and strategic missile guidance[END_REF]6]). In order to gain altitude, the first typical maneuver executed by the control system consists in rotating the aircraft to increase the angle of attack. Rotating the vehicle produces a temporary loss of pressure that develops a downward force at the tail. This causes an overall downward force on the aircraft that initially lowers the center of gravity, before the increased upward force on the main wings from the increased angle of attack raises the vehicle. Practically, this phenomenon can be easily reproduced by inserting a delay between the variation of the angular velocity and the effect of the lift raising the vehicle.

Chapter 2. Rendezvous Problems

In this context, the vehicle is subject to the following augmented dynamics

                   ṙ = v , ḃ = u v = f τ (t , r,v,b,b(t -τ)) = f T (t ) m b -g (r) r r - d + ηc m b ∧ v v 2 v v + c m v ∧ (b(t -τ) ∧ v) (2.17)
where, now, we do not control the direction of the principal body axis b any more, but rather, its velocity, that we denote by u. Remark the presence of the state delay τ within the lift term. Denoting by x = (r, v,b) the new state variables, the Optimal Interception Problem with Delays (OIP) τ consists in minimizing the cost

C (t f , u) = -v(t f ) 2 + t f 0 u(t) 2 d t (2.18) such that                      ẋ(t ) =     ṙ(t ) v(t ) ḃ(t )     =     v(t) f τ (t , r(t),v(t),b(t),b(t -τ)) u(t)     = f (t , x(t ), x(t -τ), u(t)) x(t ) = (r(t ), v(t),b(t)) ∈ R 6 \ {0} × S 2 x(0) = x 0 = (r 0 , v 0 , b 0 ) , x(t f ) = (r(t f ), v(t f ), b(t f )) ∈ M f ⊆ R 6 \ {0} × S 2 (2.19) among all the control u ∈ L ∞ ([0, t f ], R 3 ) satisfying, almost everywhere in [0, t f ], u(t) ≤ C ω (2.20)
where C ω > 0 is constant, and the final time t f may be free or not. Similarly to the original problem without delays, the final manifold takes the form

M f = (r, v,b) ∈ R 6 \ {0} × S 2 | r = r f , v • e r v = cos ψ 1 f , v • e L v = cos ψ 2 f , v • e v = sin ψ 2 f (2.21) b • e r b = cos ψ 3 f , b • e L b = cos ψ 4 f , b • e b = sin ψ 4 f
where ψ 3 f and ψ 4 f are fixed angles related to the direction of the principal body axis.

One remarks that formulation (OIP) τ does not contain constraints (2.9), considered in both the two previous optimal control problems. Unfortunately, in this case, (2.9) arises pure state constraints, thus making harder the analysis via the Maximum Principle than in (OIP) (see Chapter 3), and consequently, the implementation of efficient shooting methods. The interest of considering (OIP) τ lies on the presence of the delay τ, which makes the analysis of the Maximum Principle and performing implementations of shooting methods more challenging (but possible, see Chapter 6).

Part II

Structure of Extremals and Numerical

Strategies of Guidance 3

Structure of Extremals for Optimal Guidance Problems

As presented in the introduction, one of the main objectives of this thesis consists in providing efficient algorithms, based on indirect methods, for the optimal guidance of launch vehicles, and for this, the first step dwells upon the analysis of necessary optimality conditions via the Maximum Principle. This chapter aims to provide the analytical structure of Pontryagin extremals, in particular of controls as functions of the state and the adjoint vector, for a broad optimal guidance framework, by investigating the general optimal guidance problem (GOGP). The abstract nature of the proposed study makes possible to apply the achieved results to more general endoatmospheric launch vehicle problems than usual rendezvous applications.

Our analysis faces two main difficulties, that we can informally illustrate as follows. As we designed in Chapter 2, formulation (GOGP) contains mixed control-state constraints. In Chapter 1, we saw that solving these control problems by means of shooting methods turns out to be a hard task because Theorem 1.4 provides closed-loop dynamical equations related to the adjoint vector, only if, the evolutions of the multipliers µ m e , µ m i related to the mixed constraints are known. In the aerospace context, a smart solution to this first issue, often taken into account by the engineering community, consists in reducing (GOGP) to a particular local formulation by using Euler coordinates. This allows to rewrite the original mixed control-state constraints as pure control constraints, which is of great advantage for applying classical shooting methods to (GOGP), as discussed in Section 1.3.3. However, this smart trick provides a second nonnegligible difficulty. Indeed, due to their local nature, Euler coordinates introduce some singularities, preventing from describing all feasible missions, which translates into the loss of possible optimal solutions, or worse, numerical failures. Any efficient solution to the previous issues should provide the possibility to use classical shooting methods avoiding to fall into singularities due to local representations. Analyzing the dynamics related to (GOGP), one suspects that there is no global trans-

Chapter 3. Structure of Extremals for Optimal Guidance Problems

formation to convert the concerned mixed constraints into pure control constraints (e.g. quaternions are not suited for). Then, a smart mix of local strategies is preferred.

The solution that we propose consists in providing an additional chart, which covers the singularities of the previous coordinates, and under which, the mixed controlstate constraints can be converted into pure control constraints. We are able to provide the consistency of these new problems, in the sense that, any global extremals of (GOGP) can be locally projected onto the extremals coming from the two local reformulations of (GOGP). This allows to entirely classify regular and nonregular arcs.

The chapter is organized as follows. In Section 3.1, we analyze the abstract framework in which an optimal control problem with mixed control-state constraints (OCP) m (see Section 1.4.2) can be splitted into several local problems with pure control constraints. We derive abstract conditions concerning the consistency of the local adjoint vectors (as explained above). Sufficient optimality conditions are analyzed as well. Section 3.2 is devoted to the development of specific local charts satisfying consistency conditions for (GOGP). First, the usual Euler coordinates for the transformation of mixed constraints are introduced from a more geometric point of view. Then, they are closed by other Euler coordinates, always providing pure control constraints. Finally, in Section 3.3, we provide a complete description of both regular and nonregular extremals for (GOGP), by exploiting the local formulations developed previously.

Local Change of Problems Under Abstract Framework

In this section, we focus on the abstract form (OCP) m of a general optimal control problem with mixed control-state constraints, as in formulation (1.18)-(1.20). We are interested in applying the Maximum Principle, given by Theorem 1.4, to conceive a shooting method able to converge efficiently and quickly. However, as we detailed in Section 1.4.2, the presence of the multiplier µ m , related to the mixed constraint, prevents from integrating the adjoint equations in an easy way. In what follows, we develop an analysis to show that, with the help of some local transformations, we are able to recover the global evolution of the adjoint vector of an extremal of (OCP) m , without any information about the multiplier µ m , by studying local problems with pure control constraints. This result is applied to (GOGP) in Section 3.2.

From now on, we denote by ( q(•), ū(•)) any optimal solution of problem (OCP) m , with final time t f . Moreover, we assume that the mixed constraint function c m satisfies Assumption 1.1, i.e. it is regular. Then, there exists a tuple (p(•), p 0 , µ e,m (•), µ i ,m (•)), with (p(•), p 0 ) = 0, satisfying the thesis of Theorem 1.4 and, in particular, conditions (1.23)- (1.28), where the Hamiltonian related to (OCP) m is given by (1.22).

Local Change of Problems Under Abstract Framework

Reduction to Local Problems with Pure Control Constraints

The local reduction that we present is based on the existence of some transformations allowing to commute locally the mixed constraint into pure control constraints.

Assumption 3.1.

There exist an atlas {(V i , ϕ i )} i ∈I of M and related smooth functions

T i : V i × R m → R m , c i ,m e : R × R m → R r m e , c i ,m i : R × R m → R r m i satisfying:
• For every q ∈ V i , the mappings T i (q, •) : R m → R m are homeomorphisms such that their inverses T -1 i (q, •) : R m → R m are continuous with respect to q.

• For every (t , q, w) ∈ R × V i × R m , there hold c m e (t , q, T i (q, w)) = c i ,m e (t , w) and c m i (t , q, T i (q, w)) = c i ,m i (t , w).

Under Assumption 3.1, the modified local problems are introduced as follows.

Since q([0, t f ]) is compact, we cover [0, t f ] by a finite number of closed intervals [r i , s i ], for which q([r i , s i ]) ⊆ V i , where (V i , ϕ i ) are the local charts given by Assumption 3.1. Our analysis is developed around the optimal trajectory q, and then, without loss of generality, we multiply the dynamics (1.19) related to problem (OCP) m by smooth cut-off functions, each of which has support contained in V i and assume 1 as value within a compact set in V i containing the local trajectory q([r i , s i ]). This allows to consider local formulations of (OCP) m , under the following compact form

(OCP) m,i                      min g (s i , q(s i )) + s i r i f 0 (t , q(t ), u(t )) d t q(t ) = f (t , q(t ), u(t )) , q(t ) ∈ V i a.e. t ∈ [r i , s i ] q(r i ) = q(r i ) , q(s i ) = q(s i ) , u ∈ L ∞ ([r i , s i ], R m ) c m e (t , q(t ), u(t )) = 0 , c m i (t , q(t ), u(t )) ≤ 0 a.e. t ∈ [r i , s i ]
which is well-defined for the construction operated above. From the local optimality of ( q(•), ū(•)), we derive that, for every index i where q| [r i ,s i ] is defined, the couples

( qi (•), ūi (•)) = ( q| [r i ,s i ] (•), ū| [r i ,s i ] (•)
) are solutions of problems (OCP) m,i , respectively. Let (q i (•), u i (•)) be any admissible trajectory of (OCP) m,i and define the curve

w i (•) = T -1 i (q i (•), u i (•)) : [r i , s i ] → R m which, under Assumption 3.1, is a measurable bounded function in [r i , s i ]. It is clear that, almost everywhere in [r i , s i ], there hold qi (t ) = f (t , q i (t ), u i (t )) = f t , q i (t ), T i (q i (t ), w i (t ))
and

c i ,m e (t , w i (t )) = c m e t , q i (t ), T i (q i (t ), w i (t )) = c m e (t , q i (t ), u i (t )) = 0 ,

Chapter 3. Structure of Extremals for Optimal Guidance Problems

c i ,m i (t , w i (t )) = c m i t , q i (t ), T i (q i (t ), w i (t )) = c m i (t , q i (t ), u i (t )) ≤ 0 .
Therefore, it follows straightforwardly that, by denoting wi (•) = T -1 i ( qi (•), ūi (•)), the couples ( qi (•), wi (•)) are optimal solutions, respectively, of the new class of local problems with pure control constraints assuming the following compact form

(OCP) i                      min g (s i , q(s i )) + s i r i f 0 t , q(t ), T i (q(t ), w(t )) d t q(t ) = f t , q(t ), T i (q(t ), w(t )) , q(t ) ∈ V i a.e. t ∈ [r i , s i ] q(r i ) = q(r i ) , q(s i ) = q(s i ) , w ∈ L ∞ ([r i , s i ], R m ) c i ,m e (t , w(t )) = 0 , c i ,m i (t , w(t )) ≤ 0 a.e. t ∈ [r i , s i ]
where the functions u are substituted by the new control functions w. In the same way as before, one proves that if ( qi (•), wi (•)) are solutions of (OCP) i , respectively, then, the couples qi (•), T i ( qi (•), wi (•)) are solutions of (OCP) m,i , respectively. Problems (OCP) i are local reformulations with only pure control constraints of the original problem (OCP) m . From this, one may wonder whether it is possible to solve (OCP) m with shooting methods, by applying the Maximum Principle to problems (OCP) i with pure control constraints, avoiding the presence of additional multipliers related to mixed constraints. In this case, an issue arises. Indeed, since the control changes when passing from (OCP) m,i to (OCP) i , it is not clear a priori how the adjoint vector of problem (OCP) m,i is related to the adjoint vector of problem (OCP) i . A solution to this controversy is provided by the following consistency result. Theorem 3.1. Suppose that ( q(•), ū(•)) is an optimal solution of (OCP) m , with final time t f . Assume that Assumption 1.1 holds for (OCP) m and let (p(•), p 0 , µ m e (•), µ m i (•)) be any tuple, with (p(•), p 0 ) = 0 and p 0 ≤ 0, satisfying conditions (1.23)

-(1.28) of The- orem 1.4. Denoting qi = q| [r i ,s i ] , ūi = ū| [r i ,s i ] and wi = T -1 i ( qi , ūi ), under Assump- tion 3.1, there exist couples (p i (•), p 0 i ) such that the tuple ( qi (•), p i (•), p 0 i , wi (•)) is an extremal of (OCP) i in [r i , s i ],

Local Change of Problems Under Abstract Framework

composed by two charts (V 1 , ϕ 1 ), (V 2 , ϕ 2 ) and that q(0) ∈ V 1 . If the optimal value of p(0) is known, from p 1 (0) = p(0), we start a shooting method on (OCP) 1 . Suppose that, at a given time t 2 ∈ (0, t f ), the optimal trajectory is such that q(t 2 )

∈ V 2 \ V 1 , i.e.
the solution crosses a singular region of the first local chart. Then, we can stop momentarily the numerical computations at a time t 1 < t 2 such that q(t 1 ) ∈ V 1 ∩ V 2 and start, from p 2 (t 1 ) = p 1 (t 1 ), a shooting method on (OCP) 2 , avoiding the geometrical singularity related to V 1 when reaching the point q(t 2 ). This procedure can be iterated every time a jump from V 1 to V 2 (as well as a jump from V 2 to V 1 ) occurs in the optimal trajectory. The adjoint vector related to (OCP) m is recovered thanks to (3.1).

Remark 3.1. The thesis of Theorem 3.1 still remains valid if we consider any extremal

(q(•), p(•), p 0 , µ m e (•), µ m i (•), u(•)
) of (OCP) m satisfying the conditions of Theorem 1.4, but not necessarily optimal (this follows easily from the proof of Theorem 3.1, below).

Proof of Theorem 3.1. Without loss of generality, we consider the case for which the integral cost function is zero, i.e. f 0 = 0, and, for sake of clarity in notations, we assume that only mixed inequality constraints appear and that we denote by c m . Let (p(•), p 0 , µ m (•)) be any tuple, with (p(•), p 0 ) = 0 and p 0 ≤ 0, satisfying conditions (1.23)-(1.28) of Theorem 1.4. In the following, (x 1 , . . . , x n ; ξ 1 , . . . , ξ n ) denote the local coordinates of T * M and we use the Einstein's notation for sums.

Fix an integer i and define p 0 i = p 0 . The stationarity condition (1.25), almost everywhere in [r i , s i ], locally reads

ξ k (t ) ∂ f k ∂u j (t , qi (t ), ūi (t )) + µ k m (t ) ∂c k m ∂u j (t , qi (t ), ūi (t )) = 0 (3.2)
for every j = 1, . . . , m. Assumption 3.1 implies that c m (t , q, T i (q, w)) = c i (t , w), for every (t , q, w) ∈ R×V i ×R m , which does not depend on the state q. Then, differentiating each coordinate of this expression with respect to x l at (t , qi (t ), wi (t )), we obtain

0 = d [c k m (t , q, T i (q, w))] d x l (t , qi (t ), wi (t )) = ∂c k m ∂x l (t , qi (t ), ūi (t )) + ∂c k m ∂u j (t , qi (t ), ūi (t )) ∂T j i ∂x l ( qi (t ), wi (t ))
for every k =, 1 . . . , r m . Multiplying the previous expression by µ k m (t ), and plugging it into (3.2), there results

µ k m (t ) ∂c k m ∂x l (t , qi (t ), ūi (t )) = ξ k (t ) ∂ f k ∂u j (t , qi (t ), ūi (t )) ∂T j i ∂x l ( qi (t ), wi (t )) . (3.3)

Chapter 3. Structure of Extremals for Optimal Guidance Problems

By definition of Hamiltonian field (see expression (1.2) and the Hamiltonian dynamical system (1.21)), the adjoint vector p, almost everywhere in [r i , s i ], satisfies

ξl (t ) = - ∂ f k ∂x l (t , qi (t ), ūi (t ))ξ k (t ) -µ k m (t ) ∂c k m ∂x l (t , qi (t ), ūi (t )) (3.4)
for every l =, 1 . . . , n. Exploiting expression (3.3), from (3.4) we finally obtain

ξl (t ) = - ∂ f k ∂x l (t , qi (t ), ūi (t ))ξ k (t ) - ∂ f k ∂u j (t , qi (t ), ūi (t )) ∂T j i ∂x l ( qi (t ), wi (t ))ξ k (t ) = - d [ f k (t , q, T i (q, w))] d x l (t ,
p i (t ), f (t , qi (t ), ūi (t )) ≥ p i (t ), f (t , qi (t ), u) (3.5) for every u ∈ R m such that c m (t , qi (t ), u) ≤ 0. If w ∈ R m satisfies c i (w) ≤ 0, setting u = T i ( qi (t ), w), one obtains c m (t , qi (t ), u) ≤ 0.
Then, from (3.5), we infer that p i (t ), f t , qi (t ), T i ( qi (t ), wi (t )) = p i (t ), f (t , qi (t ), ūi (t ))

≥ p i (t ), f (t , qi (t ), u) = p i (t ), f t , qi (t ), T i ( qi (t ), w)
which implies condition (1.8) for problem (OCP) i . The conclusion follows.

Sufficient Conditions Under Reduction to Local Problems

In the presence of constraints depending on the state, giving sufficient conditions of optimality becomes a difficult question and the framework provided in Section 1.3.2 cannot be considered. In the literature, many authors analyzed the issue and gave different sufficient conditions (the classical references are [START_REF] Maurer | First and second-order necessary and sufficient optimality conditions for infinite-dimensional programming problems[END_REF][START_REF] Maurer | First and second order sufficient optimality conditions in mathematical programming and optimal control[END_REF][START_REF] Orrell | Another jacobi sufficiency criterion for optimal control with smooth constraints[END_REF][START_REF] Pickenhain | Sufficiency conditions for weak local minima in multidimensional optimal control problems with mixed control-state restrictions[END_REF][START_REF] Zeidan | The riccati equation for optimal control problems with mixed statecontrol constraints: necessity and sufficiency[END_REF][START_REF] Maurer | Second-order sufficient conditions for control problems with mixed control-state constraints[END_REF]). However, when the problem (OCP) m with mixed constraints can be reduced, by Assumption 3.1, to the local problems (OCP) i with pure control constraints, providing sufficient conditions of local optimality becomes easier because one can adopt the structure of Theorem 1.2. In this section, we develop such sufficient conditions.

We consider the autonomous framework, i.e. we assume to deal with a problem (OCP) m with mixed control-state constraints governed by autonomous dynamics

Local Change of Problems Under Abstract Framework

f (q, u), autonomous cost f 0 (q, u) and autonomous constraints c m (q, u). Moreover, without loss of generality, we suppose that the final cost function g is zero (see Remark 1.1 of Chapter 1). Proceeding as in Section 1.3.2, we need to build submanifolds of T * M in which we assume that the canonical projections are diffeomorphisms. Under Assumption 3.1, up to multiply the vector field f by some smooth cut-off functions with compact support contained in the open sets V i , we may assume that each

q(t ) = f q(t ), T i (q(t ), w(t )) , w(t ) ∈ R m (3.6)
evolves within each V i . From this, after having defined the normal Hamiltonians

h i (q, p, w) = h i ,w (p) = 〈p, f (q, T i (q, w))〉 -f 0 (q, T i (q, w))
and their maxima

H i (q, p) = max c i ,m e (w) = 0 c i ,m i (w) ≤ 0 h i (q, p, w)
we may assume that H i : T * V i → R are smooth and such that their Hamiltonian fields

H i : T * V i → T (T * V i
) are well-defined, smooth and complete. From now on, we consider only curves starting from some fixed times r i and ending at some fixed times s i , where r i +1 = s i (the reason for this will be clear soon, see Theorem 3.2). For every

germ a i ∈ C ∞ (V i ), the images of their differentials L i r i = d a i (V i ) ⊆ T * V i are smooth submanifolds of dimension n of T * V i , respectively. It follows that the sets L i t = exp H i (t ; r i , •) • L i r i ⊆ T * V i , L i = {(p, t ) | p ∈ L i t , t ∈ R} ⊆ T * V i × R
are smooth submanifolds of dimension n and n +1, respectively. As a classical result, the restrictions of the integral invariants of Poincaré-Cartan

pd q -H i d t ∈ A 1 (T * V i × R)
onto the submanifolds L i , respectively, are exact smooth 1-forms. From these preliminary considerations, we provide the following sufficient condition framework.

Assumption 3.2.

There exists an atlas {(V i , ϕ i )} i ∈I of M satisfying Assumption 3.1 whose local charts have simply connected domains V i and, for every i ∈ I , the trajectories arising from the dynamical problem (3.6) are piecewise smooth. Moreover, for every i ∈ I , there exist germs a i ∈ C ∞ (V i ) such that the mappings π| L i t : L i t → V i are diffeomorphisms for every t ∈ R, where π : T * V i → V i are the canonical projections.

Remark 3.2. In Assumption 3.2, we ask for a greater regularity than one needs. From a practical point of view, one requires that, around a normal extremal of final time t f , a finite number of projections π|

L i t : L i t → V i are diffeomorphisms for every t ∈ (r i , s i ].
The piecewise smooth regularity of the trajectories arising from (3.6) is necessary to correctly evaluate line integrals. Related to this, a misprint is contained in the proof of Theorem 17.2 in [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]: the issue is solved by requiring this piecewise smooth regularity.

Chapter 3. Structure of Extremals for Optimal Guidance Problems

Assumption 3.2 represents sufficient prerequisites for local optimality, which hold under no-fold conditions. Our local optimality result takes the following form. Theorem 3.2. Assume that Assumption 3.2 holds and let (q * , p * , µ * m e , µ * m i , u * ) be a normal extremal of problem (OCP) m with final time t f , satisfying q * ([r i , s i ]) ⊆ V i and p * (r i ) ∈ L i r i , for integers i = 1, . . . , k such that r 1 = 0 and s k = t f . There exists an open neighborhood W ⊆ M of q * , such that, the inequality

t f 0 f 0 (q * (t ), u * (t )) d t ≤ t f 0 f 0 (q(t ), u(t )) d t (3.7)
holds for every admissible trajectory q : [0, t f ] → M of (OCP) m , i.e. satisfying dynamics (1.19) and associated to a control u : [0, t f ] → R m which satisfies constraints (1.20), that belongs to W and provides q(r i ) = q * (r i ), q(s i ) = q * (s i ) for every i = 1, . . . , k.

Proof. Without loss of generality, for sake of clarity in the exposition and notations, we assume that only mixed inequality constraints appear and that we denote by c m .

Since M is a locally compact Hausdorff space, we build k open sets W i with compact closures, such that q * ([r i ,

s i ]) ⊆ W i ⊆ W i ⊆ V i , for every i = 1, . . . , k.
Then, up to multiply the dynamics f (q, u) by some smooth cut-off functions that assume the value 1 in W i and that have supports contained in V i , respectively, thanks to Assumption 3.1, any problem (OCP) i , in which q(r i ) = q(r i ), q(r i ) = q(r i ) are substituted by

q(r i ) = q * (r i ), q(r i ) = q * (r i ), is well-defined if i = 1, . . . , k. We set W = i =1,...,k W i .
For any integer i ∈ {1, . . . , k}, since T * q V i ∼ = T * q M for q ∈ V i , the curves

q * i (•) = q * | [r i ,s i ] (•) : [r i , s i ] → V i , p * i (•) = p * | [r i ,s i ] (•) : [r i , s i ] → T * V i
are well-defined and absolutely continuous. Moreover, the control

u * i (•) = u * | [r i ,s i ] (•), locally almost everywhere in [r i , s i ], satisfies c m (q * i (t ), u * i (t )) ≤ 0 .
* i (•) = T -1 i (q * i (•), u * i (•)) makes the tuple (q * i (•), p * i (•), w * i (•)) a normal extremal of problem (OCP) i in [r i , s i ],
for any i = 1, . . . , k. Let (q(•), u(•)) be an admissible couple for (OCP) m defined on [0, t f ], satisfying the inclusion q([0, t f ]) ⊆ W and the compatibility conditions q(r i

) = q * (r i ), q(s i ) = q * (s i ) for any i = 1, . . . , k. Denoting q i (•) = q| [r i ,s i ] , u i (•) = u| [r i ,s i ] and w i (•) = T -1 i (q i (•), u i (•)
), with the same computations provided in Section 3.1.1, Assumption 3.1 gives that (q i (•), w i (•)) is an admissible couple for problem (OCP) i on [r i , s i ], for every i = 1, . . . , k.

We prove that (3.7) holds in any subinterval [r i , s i ], for i = 1, . . . , k. Fix i ∈ {1, . . . , k} and define the mapping πi :

L i → V i × R : (p, t ) → (π(p), t ) = (π| L i t (p), t )

Local Transformations for (GOGP)

which is smooth because restriction of a smooth map on L i . Under Assumption 3.2,

π| L i t : L i t → V i are diffeomorphisms for every t ∈ R, therefore, it is easily checked that πi is an injective immersion. Since dim(L i ) = n + 1, map πi is a diffeomorphism. Consider the curve p i (•) = π T * • π-1 i (q i (•), •) : [r i , s i ] → T * V i , where π T * is the projec- tion onto T * M , such that p i (t ) ∈ T * q i (t ) V i . Under Assumption 3.2, p i (•) is piecewise smooth, while p * i (•) is smooth because the Hamiltonian fields H i are smooth maps.
We define the piecewise smooth curves

γ * i : [r i , s i ] → L i : t → (p * i (t ), t ) , γ i : [r i , s i ] → L i : t → (p i (t ), t )
that have the same initial and final points. Since V i are simply connected and πi are diffeomorphisms, L i are simply connected as well, and then, γ * i (•) is homotopic equivalent to γ i (•). From the C ∞ -extension Whitney's theorems and the fact that the 1-forms (s -

H i d t )| L i = (pd q -H i d t )| L i are closed,
from the properties of the line integrals, we obtain

s i r i f 0 (q * (t ), u * (t )) d t = s i r i f 0 q * i (t ), T i (q * i (t ), w * i (t )) d t = s i r i p * i (t ), f q * i (t ), T i (q * i (t ), w * i (t )) d t - s i r i H i (q * i (t ), p * i (t )) d t = s i r i s p * i (t ) ( ṗ * i (t )) -H i (q * i (t ), p * i (t ) d t = γ * i (s -H d t ) = γ i (s -H d t ) = s i r i s p i (t ) ( ṗi (t )) -H i (q i (t ), p i (t ) d t = s i r i p i (t ), f q i (t ), T i (q i (t ), w i (t )) d t - s i r i H i (q i (t ), p i (t )) d t ≤ s i r i p i (t ), f q i (t ), T i (q i (t ), w i (t )) d t - s i r i h i (q i (t ), p i (t ), w i (t )) d t = s i r i f 0 q i (t ), T i (q i (t ), w i (t )) d t = s i r i f 0 (q(t ), u(t )) d t .
The conclusion follows.

Local Transformations for (GOGP)

Thanks to the previous treatise of mixed control-state constraints under abstract frameworks, we are able to come back and efficiently manipulate our original endoatmospheric rendezvous problem. More specifically, the aim of this section consists in showing how to build two local changes of coordinates for the general optimal

Chapter 3. Structure of Extremals for Optimal Guidance Problems

guidance problem (GOGP) with mixed control-state constraints, introduced in Section 2.2.1, that allows to convert the analysis to problems with only pure control constraints via Theorem 3.1. This turns out ot be fundamental to prepare the ground for numerical computations of classical shooting methods (that we study in Chapter 4).

We start considering the transformation coming from the trajectory reference frame (see, e.g. [4,[START_REF] Trélat | Optimal control and applications to aerospace: some results and challenges[END_REF]). These are classical local coordinates often used from the engineering community in flight contexts because of two main advantages: they allow to express all the concerned physical quantities as functions of estimable parameters, and moreover, they implicitly remove the mixed control-state constraints coming from relations (2.9). Nevertheless, due to their local nature, these coordinates introduce some Euler singularities which prevent from representing the whole configuration manifold. For the considered applications, this arises a complex issue since the vehicle is often subject to abrupt maneuvers, falling into these Euler singularities. Several choices can be made to figure out this issue. The most natural one consists in passing to global coordinates, for example by using quaternions, even if, we may lose the possibility to translate the mixed constraints into pure control constraints. The novelty of the proposed approach consists in introducing a new set of local coordinates able both to cover the singularities of the trajectory reference frame and to still provide an expression as pure control constraints of the mixed constraints (2.9).

Coordinates Under the Trajectory Reference Frame

The first local transformation is based on the trajectory reference frame. This frame is built by using Euler coordinates based on the relations between the inertial frame (I, J , K) and the NED frame (e L , e , e r ) (see Section 2.1.1). We provide a construction by an intrinsic geometric insight, useful to establish the thesis of Theorem 3.1.

The trajectory reference frame (i 1 , j 1 , k 1 ) is the unique local orthonormal frame, with origin at the center of gravity G, whose normal vector i 1 coincides with the direction of the velocity v of the vehicle, the normal vector j 1 belongs to the plane (i 1 , e r ) and is perpendicular to i 1 , such that, j 1 •e r < 0, while k 1 = i 1 ∧j 1 . By introducing the Euler coordinates (v, γ, χ), frame (i 1 , j 1 , k 1 ) can be uniquely described by the relations

           i 1 = v v = cos γ cos χe L + cos γ sin χe -sin γe r j 1 = -sin γ cos χe L -sin γ sin χe -cos γe r k 1 = -sin χe L + cos χe (3.8)

Local Transformations for (GOGP)

such that the linear transformation from (e L , e , e r ) to (i 1 , j 1 , k 1 ) is given by

R a (γ, χ) =   cos γ cos χ cos γ sin χ -sin γ -sin γ cos χ -sin γ sin χ -cos γ -sin χ cos χ 0   ∈ SO(3) .
Denote V = (0, ∞) × -π 2 , π 2 × (-π, π) 2 .
Lemma 3.1. The smooth mapping

ϕ -1 a : V → R 6 \ {0} : (r, L, , v, γ, χ) → r v = -r e r (L, ) vi 1 (L, , γ, χ) =   r cos L cos , r cos L sin , r sin L , vR T (L, ) • R T a (γ, χ)   1 0 0     is an injective embedding. Hence, denoting by V a = ϕ -1 a (V ) the open image of ϕ -1 a , the couple (V a , ϕ a ) is a local chart of R 6 \ {0}. Moreover, the following relations hold ∂ ∂r = -e r 0 , ∂ ∂L = r e L -v cos χj 1 -v sin γ sin χk 1 ∂ ∂ = r cos Le -v cos L sin χj 1 + v(cos L sin γ cos χ -sin L cos γ)k 1 ∂ ∂v = 0 i 1 , ∂ ∂γ = 0 vj 1 , ∂ ∂χ = 0 v cos γk 1
where we identify T x R 6 ∼ = R 6 by the canonical isomorphism y

→ 6 i =1 y i ∂ ∂x i x .

Chapter 3. Structure of Extremals for Optimal Guidance Problems

Proof. Evaluating the determinant of the Jacobian of the mapping ϕ -1 a by symbolic computations gives det (J ϕ -1 a ) = -r 2 v 2 cos L cos γ = 0 from which we infer that ϕ -1 a is a local diffeomorphism. Moreover, it is not difficult to show that, thanks to (2.1) and (3.8), ϕ -1 a is injective. Therefore, the first part follows.

For the second part, we remark first that, by differentiating -r e r (L, ), vi 1 (L, , γ, χ)

with respect to (r, L, l , v, γ, χ), from (2.1) and (3.8), one has

∂i 1 ∂L = -cos χj 1 -sin γ sin χk 1 , ∂i 1 ∂γ = j 1 , ∂i 1 ∂χ = cos γk 1 (3.9) ∂i 1 ∂l = -cos L sin χj 1 + (cos L sin γ cos χ -sin L cos γ)k 1 .
At this step, let f ∈ C ∞ (R 6 \ {0}). Differentiating, we obtain

d f ∂ ∂r = ∂ ∂r ( f • ϕ -1 a ) = ∇ f • ∂ϕ -1 a ∂r = d f 6 i =1 ∂(π i • ϕ -1 a ) ∂r ∂ ∂x i therefore ∂ ∂r = 6 i =1 ∂(π i •ϕ -1 a ) ∂r ∂ ∂x i .
The same computation can be operated for the other variables. Combining this with (3.9), the conclusion follows straightforwardly. This chart allows to reduce the mixed control-state constraints in problem (GOGP) to standard pure control constraint as follows. Related to the local chart (V a , ϕ a ), we define the following local smooth functions

T a : V a × R 3 → R 3 : (r, v,w) → R (L, ) • R a (γ, χ)w (3.10) c 0 : R 3 → R : w → w 2 1 + w 2 2 + w 2 3 -1 (3.11) c 1 : R 3 → R : w → -w 1 (3.12) c 2 : R 3 → R : w → w 2 2 + w 2 3 -sin 2 α max . (3.13)
where no misreadings of notations arise considering the mixed constraints in (GOGP):

the mixed constraints c 1 and c 2 related to (GOGP) take the couple (q, u) as argument.

Referring to (GOGP) as defined in Section 2.2.1, for every (r, v,w) ∈ V a ×R 3 , there hold

c 0 (T a (r, v,w)) = c 0 (w) , c 1 (v, T a (r, v,w)) = c 1 (w) , c 2 (v, T a (r, v,w)) = c 2 (w) .
Therefore, the local chart (V a , ϕ a ), together with transformations T a , c 0 , c 1 and c 2 , satisfy Assumption 3.1 (remark that T a (r, v,•) is continuously invertible), and then, is a good candidate to constitute an atlas for R 6 \ {0} to correctly apply Theorem 3.1.

Local Transformations for (GOGP)

Before passing to the next local coordinates, let us write explicitly problem (GOGP) in the local coordinates provided by (V a , ϕ a ). For this, we write the dynamics of (GOGP) along the frames (e L , e , e r ), (i 1 , j 1 , k 1 ) with respect to coordinates (r, L, l , v, γ, χ), by using the transformation T a . In particular, denoting

w = R a (γ, χ) • R(L, )u, we have v = v cos γ cos χe L + v cos γ sin χe l -v sin γe r f T (t ) m b = f T (t ) m (w 1 i 1 + w 2 j 1 + w 3 k 1 ) , g (r) r r = g (r )(sin γi 1 + cos γj 1 ) d + ηc m b ∧ v v 2 v v = d + ηc m w 2 2 + w 2 3 v 2 i 1 c m v ∧ (b ∧ v) = c m v 2 (w 2 j 1 + w 3 k 1 ) .
Thanks to Lemma 3.1 and an inversion of change of coordinates matrix, we obtain that the control system (2.12) is locally equivalent to

                       ṙ = v sin γ , L = v r cos γ cos χ , ˙ = v r cos γ sin χ cos L v = f T m w 1 -d + ηc m w 2 2 + w 2 3 v 2 -g sin γ γ = ωw 2 + v r - g v cos γ , χ = ω cos γ w 3 + v r cos γ sin χ tan L (r (t ), L(t ), l (t ), v(t ), γ(t ), χ(t )) ∈ V (3.14)
where

ω(t ) = f T (t ) m(t )v(t ) + v(t )c m (t ) > 0, subject to the pure control constraints c 0 (w) = w 2 1 + w 2 2 + w 2 3 -1 = 0 , c 1 (w) = -w 1 ≤ 0 (3.15) c 2 (w) = w 2 2 + w 2 3 -sin 2 α max ≤ 0 .
Remark 3.3. A more explicit way to obtain system (3.14) consists in identifying T (R 6 \ {0}) to R 6 and considering the time derivative of the position r and the velocity v expressed along frames (e L , e , e r ), (i 1 , j 1 , k 1 ), respectively. Indeed, exploiting relations (2.1), (2.2) and (3.8), one obtains

ṙ = d d t (-r e r ) = r Le L + r l cos Le l -ṙ e r v = d d t (vi 1 ) = vi 1 + v γ - v 2 r cos γ j 1 + v cos γ χ - v 2 r cos 2 γ sin χ tan L k 1 .
These computations come straightforwardly from (3.9), so we avoid to report them.

Chapter 3. Structure of Extremals for Optimal Guidance Problems

The Local Optimal Control Problem (GOGP) a in the local chart (V a , ϕ a ) consists in minimizing the general final cost

C a (t f , w) = g (t f , ϕ -1 a (r, L, , v, γ, χ)(t f )) (3.16)
subject to the dynamics (3.14) and the control constraints (3.15), under appropriate initial and final conditions (as in control system (2.12)). The local formulation (GOGP) a is already known and widely used in the aerospace community. However, it does not allow to describe totally the original problem (GOGP) because of its local nature. In several situations, demanding performance criteria and onerous missions force optimal trajectories to pass through points that do not lie within the domain of the local chart (V a , ϕ a ), from which, by exploiting (GOGP) a , either the optimality could be lost or, in the worst case, the numerical computations may fail. The geometric insight that we gave allows to extend (GOGP) a to any concerned mission just by introducing further local coordinates able to cover the singularity of (V a , ϕ a ).

Additional Local Euler Coordinates

The singularities introduced by the local chart (V a , ϕ a ) concerned the latitude L and the slope γ. Indeed, when either L = π/2 + kπ or γ = π/2 + kπ, this chart is no more a local diffeomorphism (see the proof of Lemma 3.1). In order to cover completely R 6 \ {0}, one need additional local charts for both L and γ. However, recalling the assumptions provided in Section 2.1.1, the trajectories usually considered for our applications extend to few hundreds of kilometers, therefore, the singularity due to the latitude is rarely encountered. Motivated from this important consideration, we seek new local coordinates able to compensate the singularities due to the slope γ only.

Local Transformations for (GOGP)

such that the linear transformation from (e L , e , e r ) to (i 2 , j 2 , k 2 ) is given by

R b (θ, φ) =   cos θ sin φ sin θ cos θ cos φ -sin θ sin φ cos θ -sin θ cos φ -cos φ 0 sin φ   ∈ SO(3) .
Comparing i 1 and i 2 from (3.8) and (3.17), we obtain cos γ cos χ = cos θ sin φ , cos γ sin χ = sin θ , -sin γ = cos θ cos φ from which cos θ = 0 when cos γ = 0, and vice versa, therefore, frame (i 2 , j 2 , k 2 ) covers the Euler singularities due to γ of the first frame (i 1 , j 1 , k 1 ). As before, the set

e L 0 , e l 0 , e r 0 , 0 i 2 , 0 j 2 , 0 k 2 is a basis of R 6 .
The second chart, with coordinates (r, L, l , v, θ, φ), arises as follows.

Lemma 3.2. The smooth mapping

ϕ -1 b : V → R 6 \ {0} : (r, L, , v, θ, φ) → r v = -r e r (L, ) vi 2 (L, , θ, φ) =   r cos L cos , r cos L sin , r sin L , vR T (L, ) • R T b (θ, φ)   1 0 0     is an injective embedding. Hence, denoting by V b = ϕ -1 b (V ) the open image of ϕ -1 b , the couple (V b , ϕ b ) is a local chart of R 6 \ {0}.
Moreover, the following relations hold

∂ ∂r = -e r 0 , ∂ ∂L = r e L v cos θk 2 ∂ ∂ = r cos Le -v(sin L sin φ + cos L cos φ)j 2 + v sin θ(cos L sin φ -sin L cos φ)k 2 ∂ ∂v = 0 i 2 , ∂ ∂θ = 0 vj 2 , ∂ ∂φ = 0 -v cos θk 2
where we identify T x R 6 ∼ = R 6 by the canonical isomorphism y

→ 6 i =1 y i ∂ ∂x i x .
Proof. The proof argues exactly as in Lemma 3.1, by noticing that

det (J ϕ -1 b ) = r 2 v 2 cos L cos θ = 0

Chapter 3. Structure of Extremals for Optimal Guidance Problems

which we provide by symbolic computations, and there hold

∂i 2 ∂L = cos θk 2 , ∂i 2 ∂θ = j 2 , ∂i 2 ∂φ = -cos θk 2 (3.18) ∂i 2 ∂l = -(sin L sin φ + cos L cos φ)j 2 + sin θ(cos L sin φ -sin L cos φ)k 2 .
To conclude, it remains to show that the new chart (V b , ϕ b ) transforms the mixed control-state constraints of problem (GOGP) into pure control constraints. Combined with the results on (V a , ϕ a ), this will allow to apply Theorem 3.1 afterwards. Define the following local smooth transformation

T b : V b × R 3 → R 3 : (r, v,z) → R (L, ) • R b (θ, φ)z (3.19)
from which, as in the previous section, for every (r, v,z)

∈ V b × R 3 , we obtain c 0 (T b (r, v,z)) = c 0 (z) , c 1 (v, T b (r, v,z)) = c 1 (z) , c 2 (v, T b (r, v,z)) = c 2 (z) .
The pure control constraints c 1 (w), c 2 (w) replace the mixed constraints c 1 (v, u), c 2 (v, u), respectively. In particular, the local chart (V b , ϕ b ), together with the transformations T b , c 0 , c 1 and c 2 , satisfy Assumption 3.1 (remark that T b (r, v,•) is continuously invertible). Then, charts (V a , ϕ a ), (V b , ϕ b ) are suitable to apply Theorem 3.1.

As done for the chart (V a , ϕ a ), we write explicitly problem (GOGP) in the local coor-

dinates provided by (V b , ϕ b ). Denoting z = R b (θ, φ) • R(L, )u, we have v = v cos θ sin φe L + v sin θe + v cos θ cos φe r f T (t ) m b = f T (t ) m (z 1 i 2 +z 2 j 2 +z 3 k 2 ) , g (r) r r = g (r )(cos θ cos φi 2 -sin θ cos φj 2 +sin φk 2 ) d +ηc m b ∧ v v 2 v v = d +ηc m z 2 2 + z 2 3 v 2 i 2 , c m v∧(b∧v) = c m v 2 (z 2 j 2 +z 3 k 2 ) .
Thanks to Lemma 3.2 and an inversion of change of coordinates matrix, we obtain that the control system (2.12) is locally equivalent to

                               ṙ = -v cos θ cos φ , L = v r cos θ sin φ , ˙ = v r sin θ cos L v = f T m z 1 -d + ηc m z 2 2 + z 2 3 v 2 + g cos θ cos φ θ = ωz 2 + v r sin θ cos φ + sin φ tan L - g v sin θ cos φ φ = - ω cos θ z 3 + v r cos θ sin φ + tan 2 θ sin φ -tan L cos φ - g v sin φ cos θ (r (t ), L(t ), l (t ), v(t ), θ(t ), φ(t )) ∈ V . (3.20)

Local Transformations for (GOGP)

Remark 3.4. Similarly to chart (V a , ϕ a ), one can obtain system (3.20) by identifying T (R 6 \ {0}) to R 6 and considering the time derivative of the velocity v expressed along frame (i 2 , j 2 , k 2 ). Exploiting relations (2.1), (2.2), (3.8) and (3.18), one obtains that

v = vi 2 + v θ - v 2 r sin θ cos φ + sin φ tan L j 2 + v 2 r cos 2 θ sin φ + tan 2 θ sin φ -tan L cos φ -v φ cos θ k 2 .
The Local Optimal Control Problem (GOGP) b in the local chart (V b , ϕ b ) consists in minimizing the general final cost

C b (t f , z) = g (t f , ϕ -1 b (r, L, , v, θ, φ)(t f )) (3.21)
subject to dynamics (3.20) and constraints (3.15), under initial and final conditions.

Remark 3.5. The mappings ϕ -1 a :

U → R 6 \ {0}, ϕ -1 b : U → R 6 \ {0}
are not defined respectively for the values χ = π, φ = π. However, these singularities can be covered by considering ϕ -1 a and ϕ -1 b also within

Ṽ = (0, ∞) × -π 2 , π 2 × (0, 2π) 2
. Moreover, as we specified previously, the Euler singularities induced by the variable L do not affect the concerned trajectories. From these remarks, without loss of generality, we can assume that the configuration manifold of (GOGP) reduces to the open set V a ∪ V b ⊆ R 6 \ {0}.

Global and Local Adjoint Formulations for (GOGP)

The previous transformations allow, under Theorem 3.1, to express the Maximum Principle formulation related to (GOGP) by the Maximum Principle formulations related to (GOGP) a , (GOGP) b . In this section, we provide explicitly this relationship.

We first apply the intrinsic Maximum Principle formulation of Theorem 1.4 to problem (GOGP). We need to verify that Assumption 1.1 holds. Since c 2 (v, u) ≤ 0 implies c 1 (v, u) = 0, it is sufficient to check that there are no coefficients a 2 ≥ 0, b 0 ∈ R, which are not both zero, such that

a 2 ∂ ∂u v ∧ u 2 v 2 -sin 2 α max + b 0 ∂ ∂u u 2 -1 = 0 .
But, it is easily verified that the matrix which arises combining these gradients

  -(v 1 u 3 -v 3 u 1 )v 3 -(v 1 u 2 -v 2 u 1 )v 2 u 1 -(v 2 u 3 -v 3 u 2 )v 3 + (v 1 u 2 -v 2 u 1 )v 1 u 2 (v 2 u 3 -v 3 u 2 )v 2 + (v 1 u 3 -v 3 u 1 )v 1 u 3  

Chapter 3. Structure of Extremals for Optimal Guidance Problems

is of full rank, therefore, Assumption 1.1 holds. Let (q(•) = (r(t ), v(t)),u(•)) be an optimal solution of (GOGP) with final time t f . Thanks to Theorem 1.4, there exist a nonpositive scalar p 0 , an absolutely continuous curve p : [0, t f ] → T * M , for which p(t ) ∈ T * q(t ) M , and three bounded functions µ 0 , µ 1 , µ 2 ∈ L ∞ ([0, t f ], R), where µ 1 , µ 2 are nonpositive, with (p(•), p 0 ) = 0 and such that, almost everywhere in [0, t f ], the relations (1.23)-(1.28) hold. The multipliers µ 0 , µ 1 , µ 2 are related to c 0 (u), c 1 (v, u) and c 2 (v, u), respectively, and multiplier µ 0 does not affect the adjoint equations (1.23).

Under the conclusions of Remark 3.5, let us assume that q([0,

t f ]) ⊆ V a ∪ V b and [0, t f ] = ∪[r i , s i ] such that either q([r i , s i ]) ⊆ V a or q([r i , s i ]) ⊆ V b . Therefore, denote x a (•) = ϕ a • q(•) , w(•) = R a (x a (•)) • R(x a (•))u(•) , (3.22) x b (•) = ϕ b • q(•) , z(•) = R b (x b (•)) • R(x b (•))u(•)
where x a and x b must be understood as disjoint pieces of absolutely continuous curves which exist only in each subinterval [r i , s i ] (since it will be clear from the context, we skip the dependence on i to keep better readability). By the optimality of (q(•), u(•)), every segment of (x a , w) and (x b , z) are optimal solutions of (GOGP) a and (GOGP) b , respectively, when concentrated on each subinterval [r i , s i ]. At this step, we apply Theorem 3.1, and it is clear that this theorem can be considered directly on the coordinate versions of the concerned local problems (OCP) i , i.e. (GOGP) a and (GOGP) b . In other words, there exist adjoint vectors

p a (•) = (p a r , p a L , p a , p a v , p γ , p χ )(•) , p b (•) = (p b r , p b L , p b , p b v , p θ , p φ )(•)
such that (x a , p a , p 0 , w) and (x b , p b , p 0 , z) are extremals of (GOGP) a and (GOGP) b , respectively, satisfying conditions (1.7)-(1.10) of the Maximum Principle for problems with pure control constraints, and such that (compare with Theorem 3.1)

p(t ) = (ϕ a ) * ϕ a (r(t ),v(t )) • p a (t ) , (r(t ), v(t)) ∈ V a (ϕ b ) * ϕ b (r(t ),v(t )) • p b (t ) , (r(t ), v(t)) ∈ V b (3.23)
where (ϕ a ) * and (ϕ b ) * are the pullbacks of charts (V a , ϕ a ) and (V b , ϕ b ), respectively.

This procedure allows to study the Maximum Principle formulation of (GOGP) by focusing only on the Maximum Principle formulations of (GOGP) a , (GOGP) b . We stress the fact that, as suggested in Section 3.1.1, numerically our approach consists in integrating the adjoint equations of p a and, as a coordinate singularity is encountered, passing to integrate the adjoint equations of p b , by first computing (compare with (3.23))

p b (t ) = (ϕ -1 b ) * (r(t ),v(t )) • (ϕ a ) * ϕ a (r(t ),v(t )) • p a (t ) . (3.24)
Transformation (3.24) can be obtained quickly enough to ensure that usual numerical rates of convergence of classical shooting methods are efficiently maintained.

Local Transformations for (GOGP)

We end by providing the adjoint equations of problems (GOGP) a , (GOGP) b , obtained by symbolic computations and used both in the following sections and in Chapter 4.

Proposition 3.1. The adjoint equations of problems (GOGP) a , (GOGP) b are as follows:

ṗa r = p a L v r 2 cos γ cos χ + p a v r 2 cos γ sin χ cos L + p γ vc m h r w 2 + v r 2 cos γ + ∂g ∂r cos γ v + p χ vc m h r cos γ w 3 + v r 2 cos γ sin χ tan L + p a v ∂g ∂r sin γ - v 2 h r d + ηc m w 2 2 + w 2 3 ṗa L = -p a l v r cos γ sin χ tan L cos L -p χ v r cos γ sin χ cos 2 L , ṗa = 0 ṗa v = -p a r sin γ -p a L cos γ cos χ r -p a cos γ sin χ r cos L + 2p a v v d + ηc m w 2 2 + w 2 3 + p γ ω v w 2 - cos γ r - g v 2 cos γ + p χ ω v w 3 cos γ - cos γ sin χ tan L r ṗγ = -p a r v cos γ + p a L v r sin γ cos χ + p a v r sin γ sin χ cos L + p χ v r sin γ sin χ tan L - ω sin γ cos 2 γ w 3 + p γ v r - g v sin γ + p a v g cos γ ṗχ = p a L v r cos γ sin χ -p a v r cos γ cos χ cos L -p χ v r cos γ cos χ tan L ṗb r = p b L v r 2 cos θ sin φ + p b v r 2 sin θ cos L + p θ vc m h r z 2 + v r 2 sin θ cos φ + sin φ tan L + ∂g ∂r sin θ cos φ v -p φ vc m h r cos θ z 3 - ∂g ∂r sin φ v cos θ - v r 2 cos θ sin φ + tan 2 θ(sin φ -tan L cos φ) -p b v ∂g ∂r cos θ cos φ + v 2 h r d + ηc m z 2 2 + z 2 3 , ṗb = 0 ṗb L = -p b l v r sin θ tan L cos L -p θ v r sin θ sin φ 1 + tan 2 L -p φ v r cosθ tan 2 θ cos φ 1 + tan 2 L ṗb v = p b r cos θ cos φ -p b L cos θ sin φ r -p b sin θ r cos L + p θ ω v z 2 + sin θ(cos φ + sin φ tan L) r + g v 2 sin θ cos φ -p φ ω v z 3 cos θ - cos θ sin φ + tan 2 θ(sin φ -cos φ tan L) r - g sin φ cos θ + 2p b v v d + ηc m z 2 2 + z 2 3 ṗθ = -p b r v sin θ cos φ + p b L v r sin θ sin φ -p b v r cos θ cos L + p b v g sin θ cos φ -p θ v r cos θ(cos φ + sin φ tan L) - g v cos θ cos φ + p φ ω sin γ cos 2 γ z 3 + g v sin θ sin φ cos 2 θ + v r sin θ sin φ + tan 2 θ(sin φ -cos φ tan L) -2 v r cos θ tan θ(sin φ -cos φ tan L)(1 + tan 2 θ) ṗφ = -p b r v cos θ sin φ -p b L v r cos θ cos φ + p b v g v cos θ sin φ -p θ g v sin θ sin φ + v r (cos φ tan L -sin φ) -p φ v r cos θ cos φ + tan 2 θ(cos φ + sin φ tan L) - g v cos φ cos θ .
Chapter 3. Structure of Extremals for Optimal Guidance Problems

Regular and Nonregular Pontryagin Extremals

Studying (GOGP) a and (GOGP) b permits to recover the whole behavior of optimal controls u of (GOGP) as functions of the state and the costate. From this, we will be able to efficiently compute shooting methods for problem (GOGP) (in Chapter 4). More precisely, in the following section, we study the maximality conditions related to problems (GOGP) a , (GOGP) b to recover related optimal controls w, z, respectively, as functions of the state and the costate. Regular and nonregular cases arise.

From now on, we skip the dependence of the time t to keep better readability. Let (q(•), u(•)) represent a solution of (GOGP) in the time interval [0, t f ], with augmented adjoint vector (p(•), p 0 ). Relations (3.22) provide that the local expressions w, z of u must maximize, respectively, the following Hamiltonians

h a = p a r v sin +p a L v r cos γ cos χ + p a v r cos γ sin χ cos L +p a v f T m w 1 -d + ηc m w 2 2 + w 2 3 v 2 -g sin γ +p γ ωw 2 + v r - g v cos γ + p χ ω cos γ w 3 + v r cos γ sin χ tan L h b = -p b r v cos θ cos φ + p b L v r cos θ sin φ + p b v r sin θ cos L +p b v f T m z 1 -d + ηc m z 2 2 + z 2 3 v 2 + g cos θ cos φ +p θ ωz 2 + v r sin θ cos φ + sin φ tan L - g v sin θ cos φ +p φ - ω cos θ z 3 + v r cos θ sin φ + tan 2 θ sin φ -tan L cos φ - g v sin φ cos θ
when evaluated at the optimal local trajectories. By denoting 2 , the maximality conditions read respectively

C a = p a v f T m , C b = p b v f T m , D a = p a v ηc m v 2 and D b = p b v ηc m v
w(t) = argmax C a w 1 -D a (w 2 2 + w 2 3 ) + p γ ωw 2 + p χ ω cos γ w 3 | (3.25) w 2 1 + w 2 2 + w 2 3 = 1 , w 1 ≥ 0 , w 2 2 + w 2 3 ≤ sin 2 α max z(t) = argmax C b z 1 -D b (z 2 2 + z 2 3 ) + p θ ωz 2 -p φ ω cos θ z 3 | (3.26) z 2 1 + z 2 2 + z 2 3 = 1 , z 1 ≥ 0 , z 2 2 + z 2 3 ≤ sin 2 α max .

Regular and Nonregular Pontryagin Extremals

As specified in Section 1.3.1, the maximality condition (1.8) does not always provide a closed-loop formula for optimal controls and this depends on the strong Legendre condition (1.11). In this context, closed-loop formula of extremal controls are given explicitly by (3.25) and (3.26) when the adjoint coordinates satisfy (p γ , p χ ) = 0 and (p θ , p φ ) = 0 almost everywhere, respectively. We speak then of regular extremals.

Otherwise, we will deal with nonregular extremals. Since a given control rule affects the dynamical system only almost everywhere in the time interval [0, t f ], we study regular and nonregular extremals within some subset E ⊆ [0, t f ] that has non-zero Lebesgue measure. Since constraint c 2 forces constraint c 1 to be active, we implicitly assume that w 1 (•) > 0, z 1 (•) > 0, respectively, without reporting it at each step.

Regular Pontryagin Extremals

Assume that along a subset E ⊆ [0, t f ], that has non-zero Lebesgue measure, either (p γ , p χ ) = 0 if the optimal trajectory crosses the domain V a or (p θ , p φ ) = 0 if, instead, the optimal trajectory crosses the domain V b . In this case, closed-loop formulas for u as function of (q, p) can be achieved by applying the Karush-Kuhn-Tucker conditions, together with Assumption 2.1, on the optimization problems (3.25), (3.26).

We start considering the case for which the optimal trajectory crosses the domain V a , that is, (p γ , p χ )| E = 0. In this framework, we always refer implicitly to problem (3.25).

If We focus now on the harder case

p a v | E (•) = 0, by definition C a | E (•) = D a | E (•) = 0,
p a v | E (•) = 0. Denote λ = p γ ω, ρ = p χ ω cos γ .
We apply the Karush-Kuhn-Tucker conditions. For this, we first remark that, if the constraints of (3.25) were active at the optimum, then, it would satisfy w 2 1 + w 2 2 + w 2 3 = 1 and w 2 2 + w 2 3 = sin 2 α max , and consequently, the gradients of constraints c 0 and c 2 evaluated at this point would satisfy the linear independence constraint qualification. From applying the Karush-Kuhn-Tucker conditions, we infer the existence of a non-zero multiplier

(η 1 , η 2 ) ∈ R × R + which satisfies    C a -2η 1 w 1 = 0 , 2(η 1 + η 2 + D a )w 2 -λ = 0 2(η 1 + η 2 + D a )w 3 -ρ = 0 , η 2 (w 2 2 + w 2 3 -sin 2 α max ) = 0 .
Since either λ = 0 or ρ = 0, necessarily η 1 + η 2 + D a = 0, therefore, any optimal control satisfies ρw 2 = λw 3 . We proceed considering λ = 0, that is, w 3 = (ρ/λ)w 2 . The

Chapter 3. Structure of Extremals for Optimal Guidance Problems

problem is reduced to the study of

max C a w 1 -1 + ρ 2 λ 2 (D a w 2 2 -λw 2 ) w 2 1 + 1 + ρ 2 λ 2 w 2 2 = 1 , 1 + ρ 2 λ 2 w 2 2 ≤ sin 2 α max .
Remark that in the case C a = 0, we seek points (w 1 , w 2 ) such that the relations Coefficient C a is zero. Since D a = 0, this case results in the maximization of a parabola under bound constraints. Coefficient C a is positive. The optimum is given by the contact point between the parabola and the ellipse coming from (3.28), that lies in the positive half-plane w 1 > 0 (see Figure 3.3). Matching the first derivatives and using Assumption 2.1, we obtain

w 1 = 1 C a 1 + ρ 2 λ 2 (D a w 2 2 -λw 2 ) + C C a , w 2 1 + 1 + ρ 2 λ 2 w 2 2 = 1 , 1 + ρ 2 λ 2 w 2 2 ≤ sin 2 α max ( 3
Denoting A = -1 + ρ 2 λ 2 D a , B = 1 + ρ 2 λ 2 λ and D = |λ| sin α max λ 2 +ρ 2 , we maximize Aw 2 2 + B w 2 such that -D ≤ w 2 ≤ D. Therefore, one has                        w 2 = -D , if A > 0 , B < 0 w 2 = D , if A > 0 , B > 0 w 2 = -B 2A , if A > 0 , -2|A|D ≤ B ≤ 2|A|D w 2 = -D , if A > 0 , B < -2|A|D w 2 = D , if A < 0 , B > 2|A|D , w 1 = 1 -1 + ρ 2 λ 2
           w 1 = 1 - λ 2 + ρ 2 (C a + 2D a ) 2 w 2 = λ C a + 2D a (3.30) if λ 2 + ρ 2 (C a + 2D a ) 2 ≤ sin 2 α max

Regular and Nonregular Pontryagin Extremals

while saturations of the control arise in the opposite case (see also Figure 3.4), that is

               w 1 = cos α max , w 2 = - |λ| sin α max λ 2 + ρ 2 , if λ C a + 2D a < - |λ| sin α max λ 2 + ρ 2 w 1 = cos α max , w 2 = |λ| sin α max λ 2 + ρ 2 , if λ C a + 2D a > |λ| sin α max λ 2 + ρ 2 .
(3.31)

Figure 3.4: Intersection between the parabola and the ellipse of (3.28).

Coefficient C a is negative. In this case, since w 1 > 0, the optimum becomes the point of intersection between the parabola and the upper part of the ellipse given by (3.28) for which C takes the maximum value.

Only saturations are allowed (see Figure 3.4). Straightforwardly, we have

         w 1 = cos α max w 2 = - |λ| sin α max λ 2 + ρ 2 , if λ D a > 0 (3.32)          w 1 = cos α max w 2 = |λ| sin α max λ 2 + ρ 2 , if λ D a < 0 . (3.33)
The same procedure holds when ρ = 0, by swapping the role of w 2 , w 3 and of λ, ρ.

At this step, we have found the explicit expression of Pontryagin extremals in the regular case for the chart (V a , ϕ a ). By the similarity of (3.25) and (3.26), analogous results hold true for the local control z using instead the chart (U b , ϕ b ), for which λ and ρ are replaced respectively by p θ ω and by -p φ ω cos θ . The computations are identical, so we avoid to report them. We have finally proved the following statement. Proposition 3.2. Under Assumption 2.1, regular optimal controls u of (GOGP) are well-defined and their local behaviors w and z are explicitly given, as function of the state and the adjoint vector, by the procedures provided in expressions (3.27)- (3.33).

We stress once more on the fact that Assumption 2.1 is not limiting because, for our applications, the maximal angle of attack α max ranges in (0, π/6]. Moreover, as further motivation for this choice, we recall that Assumption 2.1 has been already implicitly used to recover the explicit expressions of the drag and the lift (see Section 2.1.2).

Nonregular Pontryagin Extremals

Assume that along a subset E ⊆ [0, t f ], that has non-zero Lebesgue measure, either (p γ , p χ ) = 0 if the optimal trajectory crosses the domain V a or (p θ , p φ ) = 0 if, instead, the optimal trajectory crosses the domain V b . We are in the presence of nonregular extremals and optimization problems (3.25), (3.26) reduce respectively to

w(t) = argmax C a w 1 -D a (w 2 2 + w 2 3 ) | w 2 1 + w 2 2 + w 2 3 = 1 , (3.34
)

w 1 ≥ 0 , w 2 2 + w 2 3 ≤ sin 2 α max z(t) = argmax C b z 1 -D b (z 2 2 + z 2 3 ) | z 2 1 + z 2 2 + z 2 3 = 1 , (3.35
)

z 1 ≥ 0 , z 2 2 + z 2 3 ≤ sin 2 α max .
The Karush-Kuhn-Tucker conditions do not help any more because, depending on the value of C a or C b , many uncountable values (w 2 , w 3 ) or (z 2 , z 3 ) are optimal. Instead, a geometric study is required. When computing nonregular extremals, Assumption 2.2, concerning the regularity of the cost and the final target set, becomes very useful to manage hard computations, as well as the following requirement.

v > 3 2 g (r)h r 1 + 4 9 1 g (r)h r f T md -1 .
It is important to note that, for our applications, such as the interception problem (OIP), the magnitude of the velocities of the vehicle is large enough when f T > 0, so that Assumption 3.3 is always satisfied, as numerical simulations confirm. Moreover, it must be remarked that this assumption is required only for nonregular arcs, i.e., if only regular extremals arise, then, no boundaries on the velocities are imposed.

In the next paragraphs, we compute the expressions of nonregular extremals proceeding through several steps. In what follows, E ⊆ [0, t f ] represents a measurable subset of non-zero Lebesgue measure in which either p γ = p χ = 0 or p θ = p φ = 0.

Regular and Nonregular Pontryagin Extremals

Non Degeneracy of Nonregular Extremals

When Assumption 2.2 holds, it turns out that the optimization problems (3.34), (3.35) are well-defined and nondegenerate, i.e. D a = 0 and D b = 0, respectively. This feature is crucial to adopt the maximality condition to evaluate optimal controls.

Lemma 3.3. Suppose p γ | E = p χ | E = 0 if the trajectory crosses V a or p θ | E = p φ | E = 0 if the trajectory crosses V b . Under Assumption 2.2, there hold p a v | E = 0 and p b v | E = 0.
Proof. We prove the statement considering the chart (V a , ϕ a ). The same arguments hold for the chart (V b , ϕ b ). Without loss of generality, suppose that E ⊆ [r i , s i ], where [r i , s i ] is a subinterval within which (GOGP) a is locally well-defined (see Section 3.2.3). By contradiction, suppose that

p γ | E = p χ | E = p a v | E = 0.
From the adjoint equations of p γ , p χ and p a v restricted to E (see Proposition 3.1), we obtain

         -v cos γ v r sin γ cos χ v r sin γ sin χ cos L 0 v r cos γ sin χ - v r cos γ cos χ cos L -sin γ cos γ cos χ r cos γ sin χ r cos L              p a r p a L p a     =     0 0 0     . (3.36)
The determinant of the matrix in (3.36) is v 2 cos γ r 2 cos L = 0, and then, (p a r , p a L , p a )| E = 0. This implies that p a is zero everywhere in [r i , s i ] and, by Theorem 3.1, we infer that p| [r i ,s i ] = (ϕ a ) * • p a = 0, from which, p = 0 in [0, t f ]. Assumption 2.2 and the transversality conditions (1.27)-(1.28) provide p 0 = 0, a contradiction with Theorem 1.4.

Nonregular Extremals Along the Local Chart (V a ,ϕ a )

In this framework, without loss of generality, we assume that E ⊆ [r i , s i ], where [r i , s i ] is a subinterval such that the optimal trajectory satisfies q([r i , s i ]) ⊆ V a , and within which, (GOGP) a is locally well-defined (see Section 3.2.3). Moreover, we always implicitly assume that p a v | E = 0 (see Lemma 3.3). We proceed by a geometric analysis.

Consider, for the moment, a general dynamical problem on M , of the type

q(t ) = f 1 (q(t )) + Ω(t ) f 2 (q(t ))
where Ω is some measurable and bounded scalar quantity. Denoting by p(•) the curve solution of the dynamical Hamiltonian problem related to the Hamiltonian h = p, f 1 (q)+Ω f 2 (q) , assume that p(t ), f 1 (q(t )) | E = 0. Therefore, since p(t ), f 1 (q(t ))

Chapter 3. Structure of Extremals for Optimal Guidance Problems

is absolutely continuous, its derivative is zero almost everywhere in E . By using Proposition 1.1, let us evaluate explicitly this derivative in local coordinates:

d d t p(t ), f 1 (t ) = n j =1 ξj (t ) f j 1 (q(t )) + n j =1 ξ j (t ) d d t f j 1 (q(t )) = n j ,k=1 - ∂ f k 1 ∂x j (t ) -Ω(t ) ∂ f k 2 ∂x j (t ) ξ k (t ) f j 1 (t ) + n j ,k=1 ξ j (t ) ∂ f j 1 ∂x k (t ) f k 1 (t ) + Ω(t ) f k 2 (t ) = p(t ), [ f 1 , f 1 ](t ) + Ω(t ) p(t ), [ f 2 , f 1 ](t ) = Ω(t ) p(t ), [ f 2 , f 1 ](t ) .
We have finally obtained, up to reduce E , that

Ω(t ) p(t ), [ f 2 , f 1 ](q(t )) | E = 0.
The previous computations allow to recover some further relations on the adjoint vector as follows. Define the following local nonautonomous vector fields (we recall that c m and d depends on the mass m, which depends explicitly on the time)

X (t , q) = v sin γ ∂ ∂r + v r cos γ cos χ ∂ ∂L + v r cos γ sin χ cos L ∂ ∂ -d v 2 + g sin γ ∂ ∂v + v r - g v cos γ ∂ ∂γ + v r cos γ sin χ tan L ∂ ∂χ Y 1 (t , q) = f T m ∂ ∂v , Y Q (t , q) = -ηc m v 2 ∂ ∂v Y 2 (t , q) = ω ∂ ∂γ , Y 3 (t , q) = ω cos γ ∂ ∂χ .
Then, the local dynamics (3.14) of (GOGP) in the local chart (V a , ϕ a ) reads q(t ) = X (t , q(t )) + w 1 (t )Y 1 (t , q(t )) + w 2 (t )Y 2 (t , q(t ))

+w 3 (t )Y 3 (t , q(t )) + w 2 2 (t ) + w 2 3 (t ) Y Q (t , q(t )
) . As a standard result, applying iteratively the previous procedure, one has Lemma 3.4. In the local chart (V a , ϕ a ), a.e. in E , the following expressions hold:

d d t p, Y 2 = p, ∂ ∂t Y 2 + p, [X , Y 2 ] + w 1 p, [Y 1 , Y 2 ] +w 3 p, [Y 3 , Y 2 ] + (w 2 2 + w 2 3 ) p, [Y Q , Y 2 ] d d t p, Y 3 = p, ∂ ∂t Y 3 + p, [X , Y 3 ] + w 1 p, [Y 1 , Y 3 ] +w 2 p, [Y 2 , Y 3 ] + (w 2 2 + w 2 3 ) p, [Y Q , Y 3 ]

Regular and Nonregular Pontryagin Extremals

d d t p, [X , Y 2 ] = p, ∂ ∂t [X , Y 2 ] + p, [X , [X , Y 2 ]] + w 1 p, [Y 1 , [X , Y 2 ]] (3.37) +w 2 p, [Y 2 , [X , Y 2 ]] + w 3 p, [Y 3 , [X , Y 2 ]] + (w 2 2 + w 2 3 ) p, [Y Q , [X , Y 2 ]] d d t p, [X , Y 3 ] = p, ∂ ∂t [X , Y 3 ] + p, [X , [X , Y 3 ]] + w 1 p, [Y 1 , [X , Y 3 ]] (3.38) +w 2 p, [Y 2 , [X , Y 3 ]] + w 3 p, [Y 3 , [X , Y 3 ]] + (w 2 2 + w 2 3 ) p, [Y Q , [X , Y 3 ]]
where p(•) is the adjoint vector coming from the Maximum Principle formulation of (GOGP) and restricted at E , which corresponds to the local adjoint vector related to the Maximum Principle formulation of (GOGP) a in E (we implicitly use Theorem 3.1).

The idea developed in what follows consists in exploiting expressions (3.37), (3.38) to reduce the set of solutions of problem (3.34), seeking therefore the unique explicit expression of any optimal control w as function of the state and the adjoint vector. To continue our analysis, we need the following explicit expressions concerning iterated Lie brackets (that we obtained by symbolic computations).

Lemma 3.5. Considering p γ | E (•) = p χ | E (•) = 0, in the local chart (V a , ϕ a )
, there hold:

[Y 1 , Y 2 ] = f T mv vc m m - f T mv ∂ ∂γ , [Y 1 , Y 3 ] = f T mv cos γ vc m m - f T mv ∂ ∂χ [Y 2 , Y 3 ] = ω 2 tan γ cos γ ∂ ∂χ , [Y Q , Y 2 ] = -ηc m v 2 c m - f T m ∂ ∂γ [Y Q , Y 3 ] = - ηc m cos γ v 2 c m - f T m ∂ ∂χ , [Y 2 , [X , Y 3 ]] = ωC [Y 2 ,[X ,Y 3 ]] (m, r, v, γ, f T ) ∂ ∂χ 〈p, [X , Y 2 ]〉 = ω -p a r v cos γ + p a L v r sin γ cos χ + p a v sin γ sin χ r cos L + p a v g cos γ 〈p, [X , Y 3 ]〉 = ω p a L v r sin χ -p a v cos χ r cos L , 〈p, [Y 1 , [X , Y 3 ]]〉 = 2 f T c m m p a L v r sin χ -p a v cos χ r cos L 〈p, [X , [X , Y 3 ]]〉 = C [X ,[X ,Y 3 ]] (m, r, v, γ, f T ) -p a L v r sin χ + p a v cos χ r cos L 〈p, [Y Q , [X , Y 3 ]]〉 = - 2ηc 2 m v 2 m p a L v r sin χ-p a v cos χ r cos L , 〈p, [Y 3 , [X , Y 3 ]]〉 = ω 2 cos γ p a L v r cos χ+p a v sin χ r cos L 〈p, [Y 2 , [X , Y 2 ]]〉 = ω 2 p a r v sin γ + p a L v r cos γ cos χ + p a v cos γ sin χ r cos L -p a v g sin γ 〈p, [Y 2 , [Y 2 , [X , Y 2 ]]]〉 = ω 3 p a r v cos γ -p a L v r sin γ cos χ -p a v sin γ sin χ r cos L -p a v g cos γ where C [Y 2 ,[X ,Y 3 ]] , C [X ,[X ,Y 3 ]
] are coefficients depending on quantities (m, r, v, γ, f T ) and p(•) is the adjoint vector coming from the Maximum Principle formulation of (GOGP) and restricted at E , which corresponds to the local adjoint vector related to the Maximum Principle formulation of (GOGP) a in E (we implicitly use Theorem 3.1).

Chapter 3. Structure of Extremals for Optimal Guidance Problems

From Lemma 3.5, we start by considering the following statements

(A) The Lie brackets [Y 1 , Y 2 ], [Y Q , Y 2 ] are proportional to ∂ ∂γ . (B) The Lie brackets [Y 1 , Y 3 ], [Y 2 , Y 3 ], [Y Q , Y 3 ], [Y 2 , [X , Y 3 ]] are proportional to ∂ ∂χ . (C) By considering p γ | E = p χ | E = 0, the scalars p, [X , [X , Y 3 ]] , p, [Y 1 , [X , Y 3 ]] and p, [Y Q , [X , Y 3 ]] are proportional to ṗχ . (D) By considering p γ | E = p χ | E = 0, p, ∂ ∂t [X , Y 2 ] is proportional to p, [X , Y 2 ] , while p, ∂ ∂t [X , Y 3 ] is proportional to p, [X , Y 3 ] . From p γ | E = p χ | E = 0, Lemma 3.5 gives p, [X , Y 2 ] E = p, [X , Y 3 ] E = 0.
These expressions, plugged into (3.38) and using (B), (C) and (D), lead straightforwardly to

w 3 (t ) p, [Y 3 , [X , Y 3 ]] (t ) = 0 , a.e. in E . (3.39)
At this step, seeking the explicit expressions of nonregular extremals from (3.39) becomes a hard and tedious task in the case that p, [Y 3 , [X , Y 3 ]] = 0. This because more many time derivatives are required, which provide complex expressions of Lie brackets. In this situation, the environmental conditions concerning the feasibility of (GOGP) and represented by Assumption 3.3 play an important role in making these further time derivatives of Lie brackets not necessary for our purpose.

v 4 + 3g h r v 2 -g h r f T w 1 m d + ηc m w 2 2 + w 2 3 = 0 .
First of all, if f T = 0 a contradiction arises immediately. Therefore, imposing f T = 0, the only physically meaningful solution is

v = 3 2 g h r 1 + 4 9 1 g h r f T w 1 m d + ηc m w 2 2 + w 2 3 - 1
and, since 0 < w 1 ≤ 1, a contradiction arises because of Assumption 3.3.

Regular and Nonregular Pontryagin Extremals

The previous results make us able to reformulate (3.34) as

   (w 1 , w 2 )| E = argmax C a w 1 -D a w 2 2 w 2 1 + w 2 2 = 1 , w 2 2 ≤ sin 2 (α max ) w 3 | E = 0 (3.40)
that, now, we can solve easily. Notice that D a = 0, and that C a = 0 if and only if f T = 0. Suppose first that C a = 0. In this case, it is clear that component w 1 of the control does not affect the dynamics, therefore, we can chose it arbitrarily, satisfying the appropriate constraints. From (3.40), we obtain

   w 1 = 1 , w 2 = 0 , if D a > 0 w 1 = cos α max , w 2 = ± sin α max , if D a < 0 . ( 3.41)
Let now C a = 0. Exploiting a quick graphical study, it is straightforward that

   w 1 = 1 , w 2 = 0 , if C a > 0 w 1 = cos α max , w 2 = ± sin α max , if C a < 0 . (3.42)
To conclude the study with respect to the local chart (V a , ϕ a ), it remains to establish the value of the coordinate w 2 when w 1 = cos α max and w 2 2 = sin 2 α max . For this, we make use of expression (3.37). Indeed, it is clear that, when p, [Y 2 , [X , Y 2 ]] = 0, the second coordinate of the control is given by (recall statements (A)-(D)) )| E = 0. Proceeding as in the proof of Lemma 3.6, a contradiction arises straightforwardly.

w 2 = - p, [X , [X , Y 2 ]] p, [Y 2 , [X , Y 2 ]] -w 1 p, [Y 1 , [X , Y 2 ]] p, [Y 2 , [X , Y 2 ]] -w 2 2 p, [Y Q , [X , Y 2 ]] p, [Y 2 , [X , Y 2 ]] . ( 3
w 2 = - 〈p, [Y 2 , [X , [X , Y 2 ]]]〉 〈p, [Y 2 , [Y 2 , [X , Y 2 ]]]〉 -w 1 〈p, [Y 2 , [Y 1 , [X , Y 2 ]]]〉 〈p, [Y 2 , [Y 2 , [X , Y 2 ]]]〉 -w 2 2 〈p, [Y 2 , [Y Q , [X , Y 2 ]]]〉 〈p, [Y 2 , [Y 2 , [X , Y 2 ]]]〉 . ( 3
The previous analysis shows how it is always possible to identify nonregular extremals along the local chart (V a , ϕ a ) using the procedure given by expressions (3.40)- (3.44).

Nonregular Extremals Along the Chart (V b ,ϕ b )

The approach proposed for chart (V a , ϕ a ) is no more exploitable for chart (V b , ϕ b ). Indeed, the terms including the gravity, the thrust and the curvature of the Earth contained in (3.20) make the computations on the Lie algebra generated by local fields hard to treat. However, we can still recover nonregular arcs, proceeding as follows. Thanks to the previous computation, we know the explicit behavior of nonregular extremals for every point of the domain V a . Therefore, it is enough to compute possible nonregular arcs at points of the domain V b that do not belong to the domain V a . From the expressions (3.8) and (3.17) of the local frames, one sees that these points of singularity lie exactly within the following submanifold

S b = (r, v) ∈ R 6 \ {0} v // r ⊆ R 6 \ {0}
which corresponds, by forcing the coordinates of the chart (V b , ϕ b ), to points such that θ = 0, φ = 0 or θ = 0, φ = π. Following the previous argument, suppose that there exists a non-zero measure subset E ⊆ [r i , s i ] ⊆ [0, t f ] such that the optimal trajectory (r, v)(•) arisen from a nonregular extremal is such that (r, v)(t) ∈ S b ⊆ V b for every t ∈ E . In particular, suppose that θ| E = 0 , φ| E = 0 or φ| E = π. Then, from (3.20), we obtain that, almost everywhere in E , the trajectory (r, v)(•) satisfies

     ṙ = ±v , L = 0 , l = 0 , θ = ωz 2 , φ = -ωz 3 v = f T m z 1 -d + ηc m z 2 2 + z 2 3 v 2 ± g .
Since the values of θ and φ remain the same along E , their derivative with respect to the time must be zero almost everywhere in E . Nonregular extremals satisfy

(z 1 , z 2 , z 3 )| E = (1, 0, 0) . (3.45)
The whole previous analysis gives us the following conclusion.

Conclusions

Conclusions

In this chapter, we have proposed a strategy to define well-posed indirect methods on (GOGP), giving optimal controls as functions of the state and the adjoint vector.

This task is far from being easy. Indeed, problem (GOGP) contains mixed controlstate constraints and the Maximum Principle adopted to work with this kind of problems implies the existence of further multipliers whose evolution is unknown, preventing from integrating the adjoint equations without the help of more information.

The provided solution consists in reformulating locally (GOGP) via two complementary sets of Euler coordinates, so that, the original mixed control-state constraints are converted into pure control constraints, allowing the use of the usual Maximum Principle and standard shooting methods. Our main geometric result states that these two local formulations are consistent with the original global formulation, that is, the global adjoint vector arising from the mixed control-state constraints Maximum Principle can be totally retraced by studying the behaviors of the two local adjoint vectors related to the pure control constraint problems in Euler coordinates. These useful transformations allow also to recover optimal controls, related to (GOGP), as functions of the state and the adjoint vector, both in regular and nonregular cases.

The conclusions achieved throughout this chapter are fundamental to implement a numerical strategy that efficiently solves (GOGP) by indirect methods. Using the established geometric relations to build such numerical procedures will be the object of the analysis developed in Chapter 4.

General Numerical Homotopy Procedure for (GOGP)

Before focusing on the development of an efficient algorithm for (OIP), we provide a homotopy scheme to solve the general version (GOGP) by parametrized shootings. The first step consists in providing a General Optimal Guidance Problem of Order Zero (GOGP) 0 , problem from which the iterative shooting path starts. This problem should be, on one hand, handy to solve via basic shooting methods and, on the other hand, as close as possible to (GOGP) to recover easily the original solution by parameter deformation. Subsequently, a homotopy scheme, depending on some deformation parameter λ, is introduced by defining the family of deformed problems (GOGP) λ . The homotopy algorithm tries to find an optimal solution of (GOGP), by making λ converge from 0 to some value which represents problem (GOGP).

This general scheme has the advantage to be cost-independent and it may be employed for general version of (GOGP), for which (OIP) represents a particular case.

General Optimal Guidance Problem of Order Zero (GOGP) 0

The General Optimal Guidance Problem of Order Zero (GOGP) 0 consists in minimizing the final cost

C 0 (t f , u) = g 0 (t f , q(t f )) = g 0 (t f , r(t f ), v(t f )) (4.1)
such that

               q(t ) = ṙ(t ) v(t ) = v(t) f 0 (t ,

r(t),v(t),u(t))

= f 0 (t , q(t ), u(t))

q(t ) = (r(t ), v(t)) ∈ R 6 \ {0} q(0) = q 0 = (r 0 , v 0 ) , q(t f ) = (r(t f ), v(t f )) ∈ M 0 f ⊆ R 6 \ {0} (4.2)
among all controls u ∈ L ∞ ([0, t f ], R 3 ) that may satisfy or not constraints (2.13), almost everywhere in [0, t f ]. In this framework, the user has free choices concerning the cost g 0 (t f , r(t f ), v(t f )), the dynamics f 0 (t , r,v,u), the final manifold M 0 f and the fact of taking into account or not constraints (2.13), totally or even partially.

The cost and the final target of (GOGP) can take very general behaviors, which prevents from providing a suitable form of the cost (4.1) and the manifold M 0 f . However, the dynamics of (GOGP) is definitively given by system (2.12), therefore, it is legitimate to formulate convenient candidates for the dynamics of order zero f 0 , appearing in the control system (4.2), starting an analysis on the dynamics of (GOGP).

General Numerical Homotopy Procedure for (GOGP)

We stress on the fact that, in general, modeling f 0 depends on the form of the cost (4.1) and the manifold M 0 f . Nevertheless, when the geometric structures of the original cost (2.11) and the original final manifold M f are simple enough, as in the case of the optimal interception problem (OIP), a good insight on the original dynamics of (GOGP) seems to be sufficient to provide successful explicit formulations of f 0 . The most intuitive way to model the dynamics of order zero f 0 comes from the fact that, usually, the contributions of the gravity and the thrust within (2.12) make the structure of the Lie algebra generated by the vector field f complex, which prevents ordinary shooting methods to converge efficiently directly on problem (GOGP). Moreover, since, often, the feasible trajectories related to endo-atmospheric optimal guidance problems are in the range of some hundred of kilometers, to further simplify analytical computations, one may be pushed for neglecting the curvature terms arising from the rotation of the NED frame. This contribution is locally expressed via the local charts (V a , ϕ a ) and (V b , ϕ b ) respectively in (3.14) and (3.20) (in particular, compare with the local frames computed in Lemma 3.1 and Lemma 3.2). From these remarks, we propose the following model for the dynamics of order zero

f 0 (t , r,v,u) = f (t , r,v,u) - f T (t ) m u -g (r) r r -ω NED (r, v) ∧ v (4.3)
where ω NED (r, v) represents the angular velocity of the NED frame (e L , e l , e r ) with respect to the inertial frame (I, J , K). Expression (4.3) removes explicitly the contributions of the gravity and the thrust and this clearly decreases the number of achievable missions (since the thrust enlarges the admissible set). However, the applications modeled by (GOGP) need to consider large values of the velocity which makes the removal of the thrust adequate. Under the local representations of (GOGP) via the local charts (V a , ϕ a ) and (V b , ϕ b ), also the contribution of the rotation of the NED is removed. This follows either from classical relative dynamics arguments or, explicitly, by applying the following lemma to the local dynamics (3.14) and (3.20).

ω NED ∧ v = - v 2 r cos γj 1 - v 2 r cos 2 γ sin χ tan Lk 1 ω NED ∧v = - v 2 r sin θ cos φ+sin φ tan L j 2 + v 2 r cos 2 θ sin φ+tan 2 θ sin φ-tan L cos φ k 2 .
Proof. We prove the statement only for (V a , ϕ a ). By similarity, the same reasoning holds for (V b , ϕ b ). By definition, the vector ω NED satisfies the following relations ėL = ω NED ∧ e L , ė = ω NED ∧ e , ėr = ω NED ∧ e r .

(4.4)

Chapter 4. Numerical Guidance Strategy

By applying formulas (2.2) and (4.4) to the local expression of the velocity v in the coordinates of the NED frame, we obtain

ω NED ∧ v = ω NED ∧ (v cos γ cos χe L + v cos γ sin χe -v sin γe r ) = v cos γ cos χ ėL + v cos γ sin χ ė -v sin γ ėr = v( L sin γ + ˙ sin L cos γ sin χ)e L + v ˙ (cos L sin γ -sin L cos γ cos χ)e +v( L cos γ cos χ + l cos L cos γ sin χ)e r .
Writing (e L , e , e r ) along the frame (i 1 , j 1 , k 1 ) and using the evolution of coordinates L, l and r , the conclusion is easily achieved from the previous expression.

We stress on the fact that the introduction of the term ω NED ∧ v becomes useful to simplify the dynamics of (GOGP) only if the equations of motion are evaluated in the local coordinates of charts (V a , ϕ a ), (V b , ϕ b ). Indeed, it is clear that, with respect to other general coordinates such as Cartesian coordinates, adding ω NED ∧v does nothing but complexifying the whole treatise (this is verified by simple computations).

As final remark, it is recommended to select zero order costs (4.1) and zero order final manifolds M 0 f such that non-challenging maneuvers suffice to reach the target with an optimal behavior. This may imply that no change of local chart arise when solving the simplified problem (GOGP) 0 or its optimal solution has no active constraints. At this step, (GOGP) 0 can be solved by classical shooting methods, by applying the procedure described in Chapter 3. One assumes that this simplified problem is built such that it is known how to efficiently initialize a shooting method on it. The resolution of (GOGP) 0 leads to a simplified solution (r 0 (•), v 0 (•), u 0 (•)) in [0, t 0 f ], with (extended) adjoint vector (p 0 (•), p 0 0 ). The multipliers related to the mixed control-state constraints are not reported since they are not needed if one applies the geometric localization techniques described in Chapter 3 (via Theorem 3.1).

Parametrized Family of Optimal Control Problems (GOGP) λ

Once the simplified problem of order zero (GOGP) 0 is solved, its solution can be used to initialize the family of shooting methods that will converge by parameter deformation to the desire solution of the original guidance problem (GOGP).

Each instantiation (GOGP) λ of the family of parametrized problems, depending on the parameter λ, consists in minimizing the final cost

C λ (t f , u) = g λ (t f , q(t f )) = g λ (t f , r(t f ), v(t f )) (4.5)

General Numerical Homotopy Procedure for (GOGP)

such that                q(t ) = ṙ(t ) v(t ) = v(t) f λ (t ,

r(t),v(t),u(t))

= f λ (t , q(t ), u(t))

q(t ) = (r(t ), v(t)) ∈ R 6 \ {0} q(0) = q 0 = (r 0 , v 0 ) , q(t f ) = (r(t f ), v(t f )) ∈ M λ f ⊆ R 6 \ {0} (4.6) among all controls u ∈ L ∞ ([0, t f ], R 3 ) that satisfy, almost everywhere in [0, t f ], c λ 0 (u(t )) = 0 , c λ 1 (q(t ), u(t)) ≤ 0 , c λ 2 (q(t ), u(t)) ≤ 0 (4.7)
where the cost (4.5), the control system (4.6) and the constraints (4.7) are appropriate modifications of (2.11), (2.12) and (2.13), respectively, and such that the new constraints c λ 0 , c λ 1 and c λ 2 satisfy Assumption 3.1 with (V a , ϕ a ) and (V b , ϕ b ) as local charts.

There are no restrictions on the choice of the parameter λ, usually a vector of some metric space. It could be a physical parameter as well as an artificial variable. The family of problems is built such that (GOGP) λ=0 is equivalent to (GOGP) 0 , while, there exists some value λ 1 , such that (GOGP)=(GOGP) λ 1 . If one is able to solve (GOGP) λ , a solution (r λ (•), v λ (•), u λ (•)), with (extended) adjoint vector (p λ (•), p 0 λ ) in [0, t λ f ], is found. As in the case of the problem of order zero, the multipliers related to the mixed constraints are not reported since they are not needed if one applies the localization techniques described in Chapter 3. The homotopy procedure consists then in seeking the solution (r

λ 1 (•), v λ 1 (•), u λ 1 (•)) in [0, t λ 1 f ] with extremal (p λ 1 (•), p 0 λ 1 )
of the original problem (GOGP) λ 1 , starting from the solution vector (r 0 (•), v 0 (•), u 0 (•)) with extremal (p 0 (•), p 0 0 ) of the problem of order zero, by making λ converge to λ 1 .

The remarks concerning the problem of order zero (GOGP) 0 allow to introduce a particular family of parametrized problems (GOGP) λ that will be often used in the following of this thesis (see Section 4.3 and Chapter 5). We assume that the cost of order zero (4.1) and the final manifold of order zero M 0 f are chosen such that the solution of (GOGP) 0 has no active constraints along the time interval [0, t 0 f ]. Therefore, we operate with a discrete continuation by setting λ = (λ 1 , λ 2 ) ∈ [0, 1] 2 as the homotopic parameter. Its first component λ 1 acts on the cost and the dynamics as follows

g λ (t f , r,v) = g 0 (t f , r,v) + λ 1 g (t f , r,v) -g 0 (t f , r,v) (4.8) f λ (t , r,v,u) = f (t , r,v,u) -(1 -λ 1 ) f T (t ) m u -g (r) r r -ω NED (r, v) ∧ v (4.9) c λ 0 (u) = c 0 (u) , c λ 1 (q, u) = c 1 (q, u) , c λ 2 (q, u) = c 2 (q, u) (4.10)

Chapter 4. Numerical Guidance Strategy

while its second component λ 2 acts only on M f and it is chosen such that

M 0 f = M λ 2 =0 f and M f = M λ 2 =1 f
. We see that the problem of order zero corresponds to λ = 0 while the original problem corresponds to λ = (1, 1). The numerical scheme that we provide consists in applying two discrete continuations with acceleration step. First Algorithm 1 is operated on the parameter λ 1 considering a sequence of shooting methods where λ 2 = 0, and this step is initialized by the extremal of the problem of order zero (GOGP) 0 . Then, Algorithm 1 is operated on the parameter λ 2 considering rather a sequence of shooting methods where λ 1 = 1 (see Section 4.3 for details). The scheme is suppose to converge to an extremal of the original problem (GOGP) λ=(1,1) .

The idea behind the splitting of the deformation parameter λ into two components arises from the fact that treating separately the physical nature of the problem (mainly, the considered costs and dynamics) and the mission involved (that is, the imposed terminal configuration) helps numerical algorithms to converge more regularly. This can be explain empirically by remarking that the convergence of numerical simulations is more sensitive to the second component λ 2 than the first one λ 1 : adding the previously removed effect of the dynamics to a basic scenario (represented by M 0 f ) makes the iterations on λ 1 converge quickly, while modifying M 0 f to reach M f may easily lead to bifurcation points, singularities or infeasible scenarios.

The whole procedure is suppose to converge to a solution of the original problem (GOGP). Nevertheless, since we are in the presence of control constraints (recall Theorem 3.1) and final conditions M f , we cannot apply the result of Proposition 1.2, thus, obtain a rigorous optimal convergence. However, numerical simulations (see Section 4.3) show that our choice of the problem of order zero (GOGP) 0 is such that the physical structure of the solutions of the original problem (GOGP) is maintained, which makes the previous homotopy procedure converge more systematically to the optimum when starting from a solution of (GOGP) 0 . We return on this in Chapter 5.

Optimal Interception Problem of Order Zero (OIP) 0

The numerical scheme provided in the previous sections is applied to practically solve the optimal interception problem (OIP). In particular, the homotopy procedure is completely provided by the two-parameter scheme of Section 4.1.2, together with quantities (4.8)-(4.10). Moreover, the Optimal Interception Problem of Order Zero (OIP) 0 is defined as in Section 4.1.1. If one assumes that the scenarios M λ f have already been defined as explained in the previous sections, then, to give a closed-loop algorithm, we need to provide a cost function g 0 (t f , q) and final conditions M 0 f thank to which (OIP) 0 would be easy enough to solve by a standard shooting method. We propose to consider a problem of order zero satisfying the following assumption.

Optimal Interception Problem of Order Zero (OIP) 0 Assumption 4.1. The cost function of (OIP)

0 is g 0 (t f , q(t f )) = g 0 (t f , r(t f ), v(t f )) = -v(t f ) 2
while the simplified final submanifold M 0 f is given, similarly to (2.15), by

M 0 f = (r, v) ∈ R 6 \ {0} | r = r 0 f , v • e r v = cos ψ 1 f ,0 , v • e L v = cos ψ 2 f ,0 , v • e l v = sin ψ 2 f ,0
where r 0 1 , ψ 1 f ,0 and ψ 2 f ,0 are such that:

• The direction of v 0 coincides with the direction of the final velocity.

• The modulus of the difference between the initial altitude and the final altitude is bounded by small values.

• The optimal trajectory lies entirely either in the domain V a or in the domain V b and is close to the straight line joining the initial position to the final position.

• The optimal solution of (OIP) 0 has no active constraints (2.13) and the related optimal control is of class at least C 2 .

The goal of this section consists in proving that a problem of order zero (OIP) 0 satisfying Assumption 4.1 is a good candidate to correctly initialize a shooting method on (OIP) 0 . More specifically, working on (OIP) 0 with the help of Assumption 4.1, whose choice is made by considering similarities with the original cost (2.16) and the original target manifold (2.15), we aim to recover a new guidance law which is used as initial guess to run indirect methods on (OIP) 0 . Numerical simulations (see, e.g. Section 4.3) show that this guidance law often provides optimal solutions of (OIP) 0 , validating the previous homotopic approach (recall the remarks in Section 4.1.2).

From the results obtained in Chapter 3, we can localize the study of the problem of order zero (OIP) 0 in the two local charts (V a , ϕ a ) and (V b , ϕ b ). Without loss of generality, by Assumption 4.1, we assume that the optimal solution of (OIP) 0 lies entirely in the domain V a . Always by the requirements in Assumption 4.1, (OIP) 0 is equivalent to the following local problem (written in a compact form)

                               min -v 2 (t f ) , (w 2 , w 3 ) ∈ R 2 ṙ = v sin γ , L = v r cos γ cos χ , ˙ = v r cos γ sin χ cos L v = -(d + ηc m (w 2 2 + w 2 3 ))v 2 , γ = vc m w 2 , χ = vc m cos γ w 3 (r, L, l , v, γ, χ)(0) = (r 0 , L 0 , l 0 , v 0 , γ 0 , χ 0 ) (r, L, l , γ, χ)(t f ) = (r f , L f , l f , γ f , χ f ) (4.11)
where the final condition (r f , L f , f , γ f , χ f ) is chosen to satisfy Assumption 4.1. It is interesting to note that the first component of the control w 1 does not affect the problem any more. This is not surprising since controlling w 1 acts strictly on the thrust, whose contribution disappears within (OIP) 0 . The special structure of problem (4.11) allows to simplify it once more. Indeed, introducing the curvilinear abscissa s(t ) = t 0 v(t ) d t and the new variable v s = ln(v), we can neglect the evolution of the velocity in (4.11) and define a new optimal control problem equivalent to (4.11). The Optimal Interception Problem of Order Zero along Curvilinear Abscissa (OIP) s 0 consists in minimizing the cost

C s 0 (s f , w) = s f 0 (d + ηc m (w 2 2 + w 2 3 )) d t (4.12) such that                        r = d r d s = sin γ , L = d L d s = cos γ cos χ r , = d d s = cos γ sin χ r cos L γ = d γ d s = c m w 2 , χ = d χ d s = c m w 3 cos γ (r, L, l , v, γ, χ)(0) = (r 0 , L 0 , l 0 , v 0 , γ 0 , χ 0 ) (r, L, l , γ, χ)(t f ) = (r f , L f , l f , γ f , χ f ) (4.13)
among all controls w ∈ L ∞ ([0, s f ], R 3 ), and s f is the free final curvilinear abscissa.

Variable does not affect the new dynamics any more. However, removing it from the formulation prevents from recovering complete information (see the next sections). Additionally, its presence do not complexify overly the main computations.

In what follows, the objective consists in analyzing (OIP) s 0 and finding, under appropriate simplifications, an explicit guidance law, i.e. explicit expressions of w 2 and w 3 as pure functions of the state, that evaluated at the initial instant t = 0 provides a value of the adjoint vector p(0) which turns out to initialize efficiently a classical shooting method on (OIP) s 0 (therefore, on (OIP) 0 , up to time scale changes). We split the study into two main steps. First, we operate an approximation of (OIP) s 0 which induces a local controllability with small variations of the controls and their time derivatives. At a second time, an analysis of the Line Of Sight (LOS) is computed to recover suitable initial conditions (see the next sections for further definitions).

Optimal Interception Problem of Order Zero (OIP) 0

Approximated Local Controllability of (OIP) s 0

We now apply the Maximum Principle to (OIP) s 0 . In order to keep better readability, we denote p = (p r , p L , p , p v , p γ , p χ ) = (p a r , p a L , p a , p a v , p γ , p χ ). The Hamiltonian is

H = p r sin γ + p L cos γ cos χ r + p cos γ sin χ r cos L +p γ c m w 2 + p χ c m w 3 cos γ + p 0 (d + ηc m (w 2 2 + w 2 3 ))
As a standard fact (see, e.g. [START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF]), since s f is not fixed and we deal with an autonomous problem, the Hamiltonian is equal to zero along any extremal. Since no constraints are considered on w, by differentiating H with respect to w, we obtain

p γ = -2ηp 0 w 2 , p χ = -2ηp 0 cos γw 3 . (4.14)
The adjoint equations simplify as follows (compare with Proposition 3.1)

                           p r = p L cos γ cos χ r 2 + p cos γ sin γ r 2 cos L + p 0 h r (d -ηc m (w 2 2 + w 3 2 )) p L = -p cos γ sin χ tan L r cos L , p = 0 p γ = -p r cos γ + p L sin γ cos χ r + p sin γ sin χ r cos L -p χ c m w 3 tan γ cos γ p χ = p L cos γ sin χ r -p l cos γ cos χ r cos L . ( 4.15)
As done in the proof of Lemma 3.3, it is not difficult to see that, necessarily, p 0 = 0. From this, without loss of generality, we can assume that p C 0 ≤ r and, just to avoid redundant constants, we impose p 0 = -1 (up to constants, this is not limiting).

We now proceed formally. By Assumption 4.1, the considered optimal control is of class at least C 2 , therefore, we derive twice expressions (4.14 c m h r present nonlinear dependencies with respect to the state, which prevents an analytical integration of (4.16). This issue can be figured out with the help of Assumption 4.1, that is, since the values of the altitude do not have strong variations and the optimal trajectory is close to the straight line joining the initial position to the final position, we approximate these two terms by constants along the optimal trajectory. Secondly, the presence of polynomials P 1 and P 2 complexify significantly the resolution of (4.16). However, by Assumption 4.1, we impose that the optimal trajectory is close to the straight line joining the initial position to the final position, and then, we may assume that controls w 2 , w 3 take small values. Therefore, one understands that, if also the variations w 2 , w 3 are sufficiently small, then, the contributions of polynomials P 1 and P 2 can be neglected and system (4.16) can be integrated analytically. These remarks allow to focus on the resolution of the following approximated system

           w 2 = c m d 2η w 2 + cos γ c m h r w 3 = c m d 2η w 3 (4.17)
where d , c m and cos γ are maintained constant.

From what we pointed out above, in order to ensure the advantage of solving the approximated system (4.17) instead of system (4.16), we should show that problem (OIP) s 0 is controllable with controls that are close, with respect to the strong topology of the Sobolev space W 1,∞ , to the control w 0 (t ) = (0, 0, 0). In other words, we ask that (OIP) s 0 is W 1,∞ -locally controllable in s f around the solution generated by the constant zero control w 0 . It turns out that, this W 1,∞ -local controllability holds for problem (OIP) s 0 . To avoid a rough break of our treatise, the reader can find the detailed proof of this property in Appendix B.

Optimal Interception Problem of Order Zero (OIP) 0

From a LOS Analysis to a Suboptimal Guidance Law for (OIP) s 0

In order to solve system (4.17), four initial conditions are required. Two of them are obtained immediately by integrating the two differential equations of γ and χ in (4.13), by imposing c m and cos γ to be constant (see Assumption 4.1). The second set of initial conditions can be investigated exploiting an analysis of the line of sight.

The Line Of Sight (LOS) is defined as the vector joining the current position r to the desired final point, which in this case is r 0 f . We denote by R = r 0 f -r its modulus and by n = r 0 f -r R its direction. It is useful to express the vector n under local coordinates along the NED frame (e L , e , e r ), that is n = cos δ 1 cos δ 2 e L + cos δ 1 sin δ 2 e -sin δ 1 e r where δ 1 , δ 2 are Euler coordinates depending on R. The third point in Assumption 4.1 provides that the launch vehicle performs locally small maneuvers nearby the line of sight. Expressing this statement mathematically in the NED frame means that, at the first order, angles γ, χ are forced to be close respectively to δ 1 and δ 2 .

Lemma 4.2. Under Assumption 4.1, i.e. at a first order approximation, the following relations hold

δ1 = - v R sin(γ -δ 1 ) , δ2 = - v R sin(χ -δ 2 ) , Ṙ = -v cos(γ -δ 1 )
.

Notice that the time derivatives considered in Lemma 4.2 are computed with respect to the variable t . To keep continuity, the proof of this result is given in Appendix B.

The following last steps exploit Lemma 4.2 to tune the two missing initial conditions. Implicitly, Assumption 4.1 will be always adopted. We start considering control w 2 .

Integrating system (4.17), we have

w 2 (s) = Ae b(s f -s) + B e -b(s f -s) - cos γ c m h r (4.18)
where we denote b = c m d 2η , which is assumed to be constant. The third point in Assumption 4.1 makes one approximate R ∼ = (s fs). Plugging (4.18) into equation γ = c m w 2 and integrating, one obtains

c m b (e bR -1)A - c m b (e -bR -1)B = γ f -γ + cos γ h r R (4.19)
On the other hand, using Lemma 4.2 to differentiate the quantity R sin(γ -δ 1 ) with respect to t , we obtain

d d t R sin(γ -δ 1 ) = Ṙ sin(γ -δ 1 ) + R cos(γ -δ 1 )( γ -δ1 ) = Ṙ sin(γ -δ 1 ) + R cos(γ -δ 1 ) γ + v sin(γ -δ 1 ) cos(γ -δ 1 ) = R cos(γ -δ 1 )vc m w 2 = -R Ṙc m w 2

Chapter 4. Numerical Guidance Strategy

Therefore, since under Assumption 4.1 we approximate Ṙ ∼ = -v, it follows that

R sin(γ -δ 1 ) = 1 v d d t R sin(γ -δ 1 ) = Rc m w 2 .
Again, under Assumption 4.1, we impose the equality R = s fs, and, integrating the latter equation by substituting (4.18), one finds

c m b Re bR - R 0 e bR d R A + c m b R 0 e -bR d R -Re -bR B = cos γ h r R 2 2 -R sin(γ -δ 1 ) . ( 4
c m w 2 + cos γ h r = - k 1 (R) + k 2 (R) R 2 R sin(γ -δ 1 ) - cos γ h r R 2 2 - k 1 (R) R γ f -γ + cos γ h r R (4.21)
Finally, from Assumption 4.1 we approximate sin(γ -δ 1 ) ∼ = γδ 1 , deducing from (4.21) the following guidance law for control w 2

w 2 (R) = -k 1 (R) γ f -δ 1 (R) Rc m -k 2 (R) sin(γ(R) -δ 1 (R)) Rc m -k 3 (R) cos γ(R) 2h r c m (4.22)
where we denote

k 3 (R) = 2 + k 1 (R) -k 2 (R).
Under Assumption 4.1, very similar computations can be performed for the control component w 3 (that we do not report to avoid redundancy). The following guidance law for control w 3 is therefore derived

w 3 (R) = -cos γ(R) k 1 (R) χ f -δ 2 (R) Rc m + k 2 (R) sin(χ(R) -δ 2 (R)) Rc m . ( 4

.23)

The new guidance law recovered by (4.22)-(4.23) generalizes classical guidance laws for the optimal interception problem when considering endo-atmospheric flights, and this, by introducing the presence of terms depending on cos γ which, in some sense, improve the stability of the law for bounded altitudes (see, e.g. [6]).

Relations (4.22) and (4.23) can be used to provide an explicit guess for the initial value of the adjoint vector of problem (OIP) 0 and its free final time t f , as follows.

Chapter 4. Numerical Guidance Strategy

the second mission considers the same objective by combining maximal final velocity and minimal final time. Therefore, the costs of the parametrized family of optimal interception problems (OIP) λ take the form

C λ (t f , u) = g λ (t f , q(t f )) = λ 1 C 1 t f -v 2 (t f ) (4.24)
with C 1 ∈ {0, 1}. Moreover, the final target manifolds are represented in the coordinates of the local chart (V a , ϕ a ) as

M λ 2 f = (r -r T , L • r T , • r T ) = (h λ 2 f , L λ 2 T, f , λ 2 T, f ) , (γ, χ) = (γ λ 2 f , χ λ 2 f ) (4.25)
and the physical parameters of the intercepting missile are listed in Section 2.2.2. Costs (4.24) and final target manifolds (4.25) are chosen such that the original problem is given by λ = (1, 1). Missions considering minimal final time are often more challenging, therefore, in general, abrupter maneuvers are more common.

In this context, we fix a unique optimal interception problem of order zero for both missions. Represented by their local coordinates (r, v) ∼ = (r, L, l , v, γ, χ) (reported in standard units), the initial point (r 0 , v 0 ) is fixed to the value (r T + h 0 , 0, 0, v 0 , γ 0 , 0), where h 0 = 1000, v 0 = 500 and γ 0 = 0, while the values of its final point are

h 0 f = 5000 , L 0 T, f = 14000 , 0 T, f = 0 , γ 0 f = - π 6 , χ 0 f = 0 .
In particular, its optimal trajectory is supposed to lie in a plane. Two different problems, corresponding to two different scenarios, are tested. More precisely, we denote by (OIP) C 1 ,1 the optimal interception problem with the same starting point as (OIP) 0 and whose final target (4.25) is given by the quantities (in standard units)

h 1 f = 5000 , L 1 T, f = 14000 , 1 T, f = -2000 , γ 1 f = - π 6 , χ 1 f = π 6
while (OIP) C 1 ,2 denotes the optimal interception problem with the same starting point as (OIP) 0 and whose final target (4.25) is given by (in standard units)

h 1 f = 7900 , L 1 T, f = 7500 , 1 T, f = 2000 , γ 1 f = - π 4 , χ 1 f = - π 4 .
From a computational point of view, the shooting method is solved using the C routines hybrd.c (see, e.g. [START_REF] Moré | The minpack project[END_REF]), which provides modified versions of the Powell's method (see, e.g. [START_REF] Moré | The minpack project[END_REF]), while a fixed time-step explicit fourth-order Runge-Kutta method is used to integrate differential equations (whose number of integration steps varies between 250 and 350, depending on the final time t f of each problem). We use a machine Intel(R) Xeon(R) CPU E5-1607 v2 @ 3.00GHz, with 7.00 Gb of RAM. On average, final self-contained executable files are of 1.9 Mb, then, of low computational load.

Numerical Simulations for (OIP)

Homotopy Scheme and Numerical Results

In the context of (OIP), the numerical homotopy scheme presented in Section 4.1 can be resumed as in Figure 4.1, below. Here, (OIP) represents both (OIP)

λ 1 → 1 λ 2 = 0 λ 1 = 1 λ 2 → 1

Chapter 4. Numerical Guidance Strategy

the chart (V b , ϕ b ) into the coordinates of chart (V a , ϕ a ) (if needed). As expected, the two solutions do not differ so much. This can be also checked from the values of the final times and modulus of velocities (which in this case coincide with the costs), that are respectively (in standard units) Guidance Law (4.22)-( 4

.23) → → → t f = 45 v(t f ) = 223.1 , (OIP) 0 → → → t f = 44.8 v(t f ) = 241.6 .
In this case, problem (OIP) 0 does not require a change of local chart and its optimal trajectory lies in a plane. The solution provided by the guidance law does not respect the stability constraints. Nevertheless, we decide to consider constraints (2.13) in the formulation of (OIP) 0 (see Section 4.1), which clearly appears from Figure 4.2 b) (there are saturations in the terminal phase). From this, one observes that the problem of order zero that we proposed does not fulfill entirely Assumption 4.1. However, experimentally, the new guidance law (4.22)-(4.23) that we recovered seems to be robust enough to initialize a larger number of scenarios than the ones fulfilling Assumption 4.1. To show this for the present numerical simulations, we ran some Monte Carlo tests on different missions. In particular, with fixed initial point (the one used previously for these simulations), we solve the following range of scenarios

h 0 f ∈ [h 0 f ,min , h 0 f ,max ] = [2000, 4000 + k • 2000] , L 0 T, f ∈ [8000, 18000] 0 T, f ∈ [-2000, 2000] , γ 0 f ∈ - π 4 , π 4 , χ 0 f ∈ - π 4 , π
4 reported in standard units, where the integer k varies in {0, . . . , 6}. For each k, we performed 10000 Monte Carlo simulations. The simulations showed that the most sensitive variables is the maximal final altitude h 0 f ,max . However, we see that the half of the scenarios still remains solvable even

Numerical Simulations for (OIP)

when the maximal altitude h 0 f ,max is 16000 meters. Moreover, among the solvable scenarios, we notice that the number of scenarios that saturate the stability constraint increases with h and is not negligible even for small value of h 0 f ,max (19.44 % at 4000 meters). This may motivate the use of the analytical guidance law (4.22)-(4.23) to initialize many more scenarios than the ones satisfying Assumption 4.1. Following the diagram of Figure 4.1, from the solution of the problem of order zero (OIP) 0 , we start a homotopy procedure to add the contributions of the thrust, of the gravity and of the roundness of the Earth. The solid lines in Figure 4.4 show the solutions achieved both in the case for which C 1 = 0 (green) as well as C 1 = 1 (brown).

From the Problem of Order Zero to the Intermediate Problem

The following optimal values are obtained (in standard units) (OIP)

C 1 =0 Inter → → → t f = 21.3 v(t f ) = 882.4
, (OIP)

C 1 =1 Inter → → → t f = 20.7 v(t f ) = 830.2 .
Problems (OIP) Inter are solved without change of local chart and their optimal trajectories lie in a plane. Compared with the previous case, the final times are much smaller and the final velocities are much higher. This is of course caused by the presence of the thrust. Moreover, one sees from Figure 4.4 b) that the presence of the minimal final time induces abrupter maneuvers at the end of the trajectory. As expected, the problem with minimal final time is harder to solve. Indeed, 7 iterations of the homotopy procedure are needed to solve (OIP)

C 1 =0
Inter while 14 iterations occur to solve (OIP)

C 1 =1
Inter . This translates in 552.3 milliseconds to solve (OIP)

C 1 =0
Inter and 1540

Chapter 4. Numerical Guidance Strategy milliseconds to solve (OIP)

C 1 =1
Inter . We stress on the fact that the homotopy on the thrust and on the gravity are operated at the same time. Empirically, this considerably increases the number of solvable scenarios. Furthermore, we provide the contribution of the minimal final time at the same time of the thrust and the gravity. Experimentally, it seems to be more appropriate to define the structure of the solution when C 1 = 1 before focusing on the final target, because, their structure can change a lot if the mission is very different than the one employed for the problem of order zero.

From the Intermediate Problem to the Original Problem: No Minimal Final Time

Numerical Simulations for (OIP)

From the diagram of Figure 4.1, the last homotopic step consists in recovering the original scenario by spatial homotopy on the parameter λ 2 . The solid lines in figures 4.5 and 4.6 give some detail of the optimal solutions in the case where no minimal final time appears, respectively for problem (OIP) C 1 =0,1 and problems (OIP)

C 1 =0,2 .
The following optimal values are obtained (in standard units).

(OIP)

C 1 =0,1 → → → t f = 22.1 v(t f ) = 803.8 , (OIP) C 1 =0,2 → → → t f = 29.05 v(t f ) = 476.9 .
In this case, problem (OIP) C 1 =0,1 is solved without change of local chart, but for problem (OIP) C 1 =0,2 the change of variables between chart (V a , ϕ a ) and chart (V b , ϕ b ) is operated. This can be seen from Figure 4.6 a) where a subplot representing the projection of the optimal trajectory of (OIP) C 1 =0,2 onto the plane (L • r T , rr T ) is provided. Starting the computation in the local chart (V a , ϕ a ), the trajectory gets close to the singularity given by γ = 0. Therefore, the change of coordinates to the local chart (V b , ϕ b ) is operated. After, the trajectory encounters the singularity given by θ = 0, therefore, the change of coordinates to the local chart (V a , ϕ a ) is operated. In our simulations, without the change of coordinates, this scenario cannot be solved.

Concerning the computations of the spatial homotopy step on λ 2 , 5 iterations are needed to solve (OIP) C 1 =0,1 while 20 iterations occur to solve (OIP) C 1 =0,2 . The total computational time, considering both the homotopy on λ 1 and the homotopy on λ 2 , are 750 milliseconds to solve (OIP) C 1 =0,1 and 1980 milliseconds to solve (OIP)

C 1 =0,2 .
The second scenario needs much more spatial homotopy iterations and provides a much more complicated behavior of the optimal control: most of the time, this control saturates (Figure 4.6 b)). Notice that the homotopy maintains the topology of the structure of the optimal solutions (the parabolic behavior, see Figure 4.6 b)).

In order to verify the optimality of the found trajectories, the same problems are solved by exploiting direct methods. More specifically, we implement the optimal control problems in the AMPL framework, combined with IPOPT, using an explicit second-order Runge-Kutta method with 200 time steps (see, e.g. [START_REF] Fourer | Ampl[END_REF][START_REF] Wächter | On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming[END_REF]). Modifying the initial guess of IPOPT, these problems are solved by the direct method with computational times comparable or even larger to the ones given by the homotopy method, obtaining the same solutions that we presented. Moreover, when (OIP) C 1 =0,2 is considered, the computational time of the direct method increases fast because of the presence of singularities concerning the Euler coordinates. Adding the contribution of the original scenario in the case of minimal final time provides the solutions shown in Figure 4.7 for (OIP) C 1 =1,1 , and, in Figure 4.8 for (OIP)

Chapter 4. Numerical Guidance Strategy

C 1 =1,2 .
The following optimal values are obtained (in standard units).

(OIP)

C 1 =1,1 → → → t f = 21.4 v(t f ) = 753.7 , (OIP) C 1 =1,2 → → → t f = 29.03 v(t f ) = 475.2 .
Comparing Figure 4.5 with Figure 4.7, we notice that the solutions of the two problems differ, and the optimal control of problem (OIP) C 1 =1,1 contains some saturations (see Figure 4.7 b)). This is not surprising since the structure of the optimal

Conclusions solution of problem (OIP)

C 1 =1
Inter already contains saturations. Similarly to the previous case, to solve (OIP) C 1 =1,1 no change of local chart is needed. In particular, the same changes of local chart adopted to solve (OIP) C 1 =0,2 are employed to solve (OIP) C 1 =1,2 (see Figure 4.8 a)). We explain this fact empirically, by saying that, the final configuration imposed to the vehicle in problems (OIP) C 1 =0,2 and (OIP) C 1 =1,2 is restrictive enough to prevent significant modifications when the minimal final time is added. Of course, we may have obtained a local solution. The computational iterations and times for the spatial homotopy on λ 2 in the case of minimal final time are the following: 11 iterations and 1730 milliseconds to solve (OIP) C 1 =1,1 , and, 26 iterations and 2580 milliseconds to solve (OIP) C 1 =1,2 . The optimality of the solution was checked by direct methods, in the AMPL-IPOPT framework, as done previously for problem (OIP) C 1 =0,1 and problem (OIP) C 1 =0,2 . The same analysis of the case without minimal final time arises. In particular, even if we cannot ensure the global optimality of the control strategy found by AMPL-IPOPT, we can infer that common direct methods and our homotopy procedure seek the same solutions also in the presence of hard missions (like problem (OIP) C 1 =1,2 ).

Conclusions

In this chapter, we have proposed a numerical strategy which combines indirect methods with homotopy procedures to solve the optimal interception problem (OIP).

The methodology consists in deforming (OIP) by removing the contributions of the thrust, of the gravity, of the roundness of the Earth and by simplifying the original scenario. The obtained problem of order zero represents an appropriate formulation, from which, a first discrete continuation on the dynamics and a second discrete continuation on the initial and final conditions efficiently lead to obtain an optimal solution of the original problem. Our main contribution is represented by a new guidance law, conveniently developed approximating a particular Maximum Principle formulation, which correctly initializes, on the problem of order zero, the starting shooting related to the previous homotopy procedure. Numerical simulations show that this method provides optimal strategies comparable, and in some cases, better (from a computational viewpoint) than usual direct method approaches.

Although the previous procedure provides optimal strategies quickly, the average obtained computational times remain too large and still not exploitable for a real-time use. In the next chapter, we will develop a fine method to both speed-up computations and to make the algorithm more robust to radical changes of the scenario.

Numerical Robustness and Interception Software (ONERA)

In Chapter 4, we provided a numerical strategy, based on homotopy algorithms, to solve via indirect methods the optimal interception problem (OIP) (see Section 2.2.2). The idea behind this numerical approach consists in a parameter deformation procedure. First, the original optimal control problem is simplified by removing some components of the dynamics, such as the thrust and the gravity, and modifying the original scenario. We obtain an approximate solution of this simplified version thanks to an ad hoc conceived guidance law. Then, the solution of the original problem is found by deforming the approximated solution, firstly, by adding the physical components previously removed, and secondly, by recovering the original scenario. Numerical simulations provided in Chapter 4 show the efficiency of the latter procedure, that is, good computational speed and precision are obtained both on some basic and even more challenging scenario with a low computational load. Nevertheless, the high sensitivity to initial conditions is evident, that is, different scenarios may take more or less computational time to converge to the optimal solution, in particular because the integration starts from the solution provided by the guidance law on the simplified problem. In other words, roughly speaking, the width of the interval containing the computational times (obtained by Monte Carlo simulations) of the whole homotopy scheme related to complex missions is too wide and contains times that are considered too large for a real-time optimal interception. The objective of this chapter, in which we consider the simplest version of (OIP) without minimal final time, is to provide a framework to manage the high sensitivity of the homotopy scheme proposed in Chapter 4 by robustifying the entire procedure. We organize the exposition in two main parts. In Section 5.1, we propose, as solution to the high sensitivity problem discussed above, the offline development of grids storing initial values of adjoint vectors, related to many different feasible scenarios, from which the solution of current scenarios is evaluated by exploiting only

Chapter 5. Numerical Robustness and Interception Software (ONERA)

one spatial homotopic step. This strategy has three main advantages. Firstly, it helps considerably to robustify the convergence when drastic changes of the scenario occur (such as, trajectory reconfigurations of the evader). Secondly, it accelerates, on average, the converging computational time related to all kind of mission. Finally, it allows to make these computational times more uniform, i.e. quite independent on the considered scenario, which may be understood as a possible solution to the previous high sensitivity problem. On the other hand, in Section 5.2, we provide an insight on the C++ library and its methods conceived to manage optimal interception missions. Details on this code are analyzed and standard usage script are suggested.

Increasing the Robustness: Initialization Grids

Consider problem (OIP) in which one strictly maximizes the final velocity (i.e. no minimal final time is considered). The high sensitivity with respect to changes of the initial scenario and robustness improvements can be achieved by an offline computing of grids containing initial values of optimal adjoint vectors for many different feasible missions. More specifically, the idea that we propose develops as follows:

• Compute, for many possible fixed scenarios, the related initial values of the optimal adjoint vectors, where the contributions of the thrust, of the gravity and of the roundness of the Earth are already considered, and store them.

• To solve any feasible scenario given by the user, denoted by S user , proceed as follows. Choose, among all the scenarios already solved offline, the one the closest to S user (w.r.t. the euclidean norm, see Section 5.1.2), and denote it by S closest ; moreover, denote the initial value of the adjoint vector of S closest by p closest (0). Then, start a spatial homotopy procedure, whose aim is to solve the original mission by deforming S closest into S user , which is initialized by p closest (0).

Once the initialization grid is provided, the spatial homotopy can be developed, for example, as done in Chapter 4, i.e. by employing the discrete continuation of Algorithm 1, where either all the scenario variables can be deformed at the same time or sequentially. An important detail that we consider is the following. Since in the context of (OIP) usually solid-fuel propelled vehicles are employed, we look for optimal scenario whose final time t f is grater or equal than 20 seconds (see the technical values provided in Section 2.2.2). Therefore, if some scenario provides t f < 20 as optimal value, we recompute the optimal strategy by imposing a fixed final time t f = 20.

To conclude, we need to provide valid initialization grids.

Fast Initialization Grids Design

The most reliable way to proceed to design a valid initialization grid consists in computing the global optimal solution of (OIP) for a large family of fixed scenarios, for example, by using global direct methods (see Section 1.3.3). However, this procedure may take too much time (depending on the discretization step of direct methods) and may not be appropriate for the evaluation of several grids with the focus on the validation of our initializing method, and not on boosted software optimization (in the context of the present work). A fast scheme to build up these grids is required.

We propose to develop initialization grids by spatial homotopies operated on some basic scenario which is correctly initialized by the guidance law (4.

r 0 = r T + 1000 , L 0 • r T = 0 , 0 • r T = 0 , v 0 = 500 , γ 0 = π 4 , χ 0 = 0 r f = r T + 5000 , L f • r T = 14000 , f • r T = 0 , γ f = 0 , χ f = 0
for example, with the homotopy scheme provided in Section 4.3 (see Figure 4.1). Store S 0 in G 1 init and the initial value of its optimal adjoint in G 2 init .

for integer i = 2 , i ≤ 2 • I r + 1 , i → i + 1 do integer j = i /2 if i % 2 == 0 then r t emp = r f + |r f -r max | • double( j /I r ) else r t emp = r f -|r f -r min | • double( j /I

Chapter 5. Numerical Robustness and Interception Software (ONERA)

The main idea of Algorithm 2 consists in recovering solutions of many different scenarios for (OIP) by deforming, with a spatial homotopy method, the solution of (OIP) on the given starting scenario S 0 . The information concerning the initialization grid are stored in the files (G 1 init ,G 2 init ). Without loss of generality, every admissible scenario is represented by the coordinates of the local chart (V a , ϕ a ) (see Section 3.2.1). The starting scenario S 0 is chosen so that a solution of (OIP) on it can be found easily by the homotopic scheme in Figure 4.1, which is initialized by the guidance law (4.22)-(4.23). We choose those particular initial and final conditions to ascertain the convergence for S 0 (we have chosen them relying on the structural values provided in Section 2.2.2, although the choice is merely free). The initial value of the optimal adjoint vector for a given deformed scenario S t emp is sought by spatial homotopy, starting from the scenario in G 1 init the closest to S t emp , in order to maximize the chances to find a solution. The homotopy procedure makes the state variables varying between a given minimal value (i.e. r min ) and a given maximal value (i.e. r max ). The global density of the grid is chosen via the number of iterations of homotopic steps I max . Algorithm 2 makes only one variable change at time. However, it is straightforward to extend this procedure to several varying parameters, by nesting many for loops, one for each variable. Remark that one can also make initial states vary. In particular, for our numerical simulations on grids, we make the initial velocity v 0 increase. When several states vary, the domain of explored scenarios can be shown as below (in the following, without loss of generality, we represent every admissible scenario by the coordinates of the local chart (V a , ϕ a ), see Section 3.2.1).

Increasing the Robustness: Initialization Grids

The points within the rectangle of Figure 5.1 represent the scenarios that Algorithm 2 tries to solve when the varying variables are r , L and angle γ (between given minimal and maximal values, respectively). In particular, for each couple (rr T , L • r T ) a discretized range of slope angles γ is considered (the region covered by the two-head arrow). This procedure is clearly generalized to all the remaining variables. Some comments on the previous construction of the initialization grid are as follows. Firstly, it is important to note that, due to evident feasibility constraints, not all the explored scenarios may be figured out by Algorithm 2. In other words, plotting the solved scenarios as in Figure 5.1 would arise probably a geometric shape which is not an uniform box as in Figure 5.1. A first explanation may summon controllability properties of each scenario that do not depend on the numerical procedure. Secondly, the lack exposed above may not be related only to feasibility constraints. More specifically, when no constraints are imposed, under standard regularity assumptions the spatial homotopy procedure is well-posed to solve optimal control problems (see Proposition 1.2). However, in the presence of control constraints, as in our case (we implicitly assume the local forms of (OIP) by using charts (V a , ϕ a ) and (V b , ϕ b ), see Chapter 3), the spatial homotopy procedure may not converge globally, and this may reduce the number of solved scenarios. Despite this flaw, as we show hereafter, in the case of (OIP), we experimentally see that grids evaluated by Algorithm 2 which are dense enough provide a totally acceptable level of failures. Finally, the same drawback as above is summoned to explain the sub-optimality possibly provided by Algorithm 2. Indeed, by considering the argument developed in Section 1.3.4, when convergence is achieved, we ensure that only first-order necessary conditions are satisfied and it may happen that different homotopic path lead to different local solutions (see also [START_REF] Trélat | Optimal control and applications to aerospace: some results and challenges[END_REF]). To improve the chances of obtaining (at least local) optimality, one could operate with several different spatial homotopy paths and choose the one satisfying second-order necessary conditions (by improving the argument developed in Section 3.1.2), or better, proceed with globally optimal direct methods. Because of lack of time, we did not implement such improvements in the present work, leaving this issue among possible perspectives.

Numerical Time-Robustness Monte Carlo Experiments

We devote this section to numerical simulations suitable for testing initializing grids that have been computed by Algorithm 2, considering real interception scenarios. The main objective consists in testing the robustness of the grids as function of the density of points that they contain. The more points an initialization grid contains, the less computational time (and failures) the resolution method is supposed to provide, making the average computational time converge uniformly to some value depending only on the refinement of the grid. Informally, it is easily understood that Chapter 5. Numerical Robustness and Interception Software (ONERA) this kind of tests is an efficient way to check whether the sensitivity problem discussed at the beginning of this chapter can be considered statistically solved.

Monte Carlo experiments are provided with the following setting. For any tested grid, we ran 50000 uniformly randomized simulations. Each simulation consists in:

• Giving a random scenario S rand , whose points are chosen inside the physical region in which the grid was developed.

• Choosing, within the grid, the scenario S grid which is the closest to S rand and operating a spatial homotopy (via Algorithm 1) on each state variable at the same time, starting from S grid , to recover the solution of S rand .

We noticed that deforming all the state variables at the same time is usually faster and still well-posed. Moreover, considering the dimension of the tested grids (at most 10 Mb), we remarked that the time needed to find the closest scenario within the grid is negligible if compared to the time taken by the spatial homotopy procedure.

We tested six initialization grids {G 1 , . . . ,G 6 } computed offline by Algorithm 2 by varying several state variables. In particular, all the six grids only differ in their density of contained solved scenarios (which iteratively increases) and are computed by varying the following variables, within their corresponding maximal and minimal bounds (in standard units)

v 0 ∈ [500, 1500] , r f -r T ∈ [5000, 20000] , L f • r T ∈ [6000, 30000] f • r T , [-2000, 2000] , γ f ∈ - π 3 , π 3 , χ f ∈ - π 3 , π
3 where we fixed the initial variables r 0r T = 1000, L 0 = 0, 0 = 0, γ 0 = π/4 and χ 0 = 0 (as in Algorithm 2). These state bounds allow to simulate real endo-atmospheric missile interception conditions when a standard initial slope of π/4 rad is chosen. The density of points of every grids corresponds to the number of iterations I max in Algorithm 2 used for each state variables. The number of iterations used is represented in Table 5.1, in which there are reported also the total number of scenarios explored (column "Total") and the number of scenarios solved (column "Solved"). We remark that, the more points one grid has, the more explored scenarios are effectively solved (compare the percentage in column "Solved" of Table 5.1). We can explain this heuristically by saying that the presence of more points makes each spatial homotopy step shorter, helping the related convergence. From a computational point of view, the shooting methods are solved using the C routines hybrd.c (see, e.g. [START_REF] Moré | The minpack project[END_REF]), which provides modified version of the Powell's method (see, e.g. [START_REF] Moré | The minpack project[END_REF]), while a fixed time-step explicit fourth-order Runge-Kutta method is used to integrate differential equations (whose number of integration steps varies between 250 and 350, depending on the value of t f ). The calculations are done on a machine Intel(R) Xeon(R) CPU E5-1607 v2 @ 3.00GHz, with 7.00 Gb of RAM.

Increasing the Robustness: Initialization Grids

I max (v 0 ) I max (r f ) I max (L f ) I max ( f ) I max (γ f ) I max (χ f ) Total Solved G 1 3 2 2
The results obtained by Monte Carlo simulations are reported in figures 5.2 and 5.3. In Figure 5.2 a), one remarks that, following the intuition, the average percentage of non-solved scenarios statistically decreases when the number of points in the initialization grids increases. The same conclusions arise for the average computational time necessary to solve one random scenario, by analyzing Figure 5.2 b). should not be surprising since we pass from 1386 to 6652 valid scenarios in the grids. However, this does not happen when passing from grid G 3 to grid G 6 , even if we increase by six the number of valid scenarios (see Table 5.1). The same structure appears when considering the average computational time for one scenario in 5.2 b), i.e. a jump occurs between grid G 2 and grid G 3 while no particular difference exists among the provided data when switching from grid G 3 to grid G 6 .

G_1 G_2 G_3 G_4 G_5 G_6 2.
The presence of large jumps from grid G 2 and grid G 3 and the clusters created by data arising from grids G 3 -G 6 in Figure 5.2 suggests, as our initial intuition expected, that, as the number of valid scenarios of a grid increases, both the average percentage of failures converges uniformly towards a small value, which represents an acceptable threshold for our interception applications, and the average computational time needed to solve one random scenario tends uniformly to some small time. In Figure 5.3 a), the average percentage of changes of chart for successful scenario resolutions is reported. These percentages keep track of scenarios where at least one change of chart occurs. The same interpretations provided above concerning Figure 5.2 are still valid if applied to Figure 5.3 a). At the same time, we remark that, even in the best case, these percentages remain high, showing the importance of operating changes of chart to considerably increase the number of successful missions. Finally, Figure 5.3 b) shows the position of each tested initialization grid in the plane failed scenarios-computational time. We remark that a small number of failed scenarios always exists. For this, we must recall also that, among possible Monte Carlo scenarios, there are missions with conditions close to the boundary of the accessible set, which has not been studied in the present work. However, this last graph reflects our intuition on considering more valid scenarios, that is, finer grids obtain both a higher number of accomplished missions and a smaller average computational time. This empirically shows the numerical time-robustness of our initialization approach.

G_1 G_2 G_3 G_4 G_5 G_6
5.2. Software Design: a Template C++ Library (ONERA)

Software Design: a Template C++ Library (ONERA)

This final section desires to show the detailed structure of the template C++ library conceived to efficiently recover optimal strategies for optimal interception missions. This C++ library was born from the work developed in ONERA -The French Aerospace Lab, with the aim of providing an optimized and fast procedure to obtain optimal interception strategies, able to update within a range of 1-10 Hz. A template framework is employed to make as easy as possible any transition to other, possibly more performing, dynamical models and/or costs related to problem (OIP).

Library Structure (Simplified UML Class Diagram)

The main structure of the conceived template C++ library can be well summarized in the following simplified UML class diagram. The class implementing officially our indirect method coupled with homotopy procedures is RealTimeShooting. In particular, the method shoot of this class is all the user needs to call to evaluate one optimal interception strategy, when a feasible scenario is provided. The function shoot reads the file adjointGuess.txt which represents a given offline precomputed initialization grid (see Section 5.1), searching the appropriate scenario from which a spatial homotopy procedure starts (see Section 5.1.2). The performances of this library are shown by the simulations of Section 5.1.2.

Details on Classes and User Script Examples

In the following, referring to Figure 5.4, we provide some computational details about the methods contained within this template C++ library, in order to show its compatibility with other structure provided by the user. An example of main script for optimal interception strategies is furnished at the end of the present section.

123

Chapter 5. Numerical Robustness and Interception Software (ONERA)

The main characteristics of each class of the library in Figure 5.4 are in order.

• Eigen Class and CMinPack Library. The template routines composing Eigen are devoted to state-of-the-art optimized and fast linear algebra methods (details can be found at http://eigen.tuxfamily.org/). We exploit Eigen to quickly operate the change of coordinates from chart (V a , ϕ a ) to chart (V b , ϕ b ) and vice versa (see Chapter 3), and this is indeed efficiently computed because prescreening and conditioning steps are combined. The CMinPack library provides an improved version of Powell's method to find local zeros of nonlinear functions, which is computed by the routine hybrd.c. We employ this library to solve shooting problems. Further details on this procedure are given in [START_REF] Moré | The minpack project[END_REF].

• LieBracket Class. This template class (the template real stands for the numerical precision) provides public methods to compute the iterated Lie brackets needed to recover possible nonregular extremals for (OIP). We refer to Section 3.3.2 for further details on the computation of nonregular extremals.

• Model Class. A standard association relation connect Eigen and LieBracket classes to the template class Model (the template real stands for the numerical precision). This class numerically represents the endo-atmospheric dynamical model considered for (OIP) and its adjoint equations (see Section 3.2). More specifically, methods evalFirst and evalSecond evaluate dynamics (

, respectively, together with their adjoint equations, which is all a numerical solver of ordinary differential equations needs to integrate the dynamics of the Maximum Principle formulation related to (OIP). This is possible thanks to methods controlFirst and controlSecond which provide the value of the control in the coordinates of the local charts (V a , ϕ a ) and (V b , ϕ b ), respectively, and both regular and nonregular extremals are considered (with the help of LieBracket). Finally, when needed, method conversionFirstToSecond operates the change of coordinates from chart (V a , ϕ a ) to chart (V b , ϕ b ) in the cotangent space, i.e. on the adjoint vector, necessary to change coordinates within the adjoint equations. As provided in Section 3.2, this requires to invert a linear system, that we compute with the help of Eigen. Same conclusions hold for its dual method conversionSecondToFirst.

• Runge4Method Class. Class Model is passed as a template parameter to the three template class Runge4Method, whose objective consists in integrating dynamics and adjoint equations related to (OIP) (the first two template parameters represent the numerical precision and the dimension of the problem, respectively). This is operated via an explicit fourth-order Runge-Kutta algorithm, by method solve. In particular, any needed change of coordinates is computed in solve, with the help of the routines in Model, by checking whether

Software Design: a Template C++ Library (ONERA)

the numerical integration passes through some singular zone for the considered local coordinates. The flexibility of maintaining the class Model as a template parameter for this integration class comes form the fact that one can substitute Model with any other endo-atmospheric dynamical model, up to preserve the existence of the six methods previously provided in Model.

• HomotopyProcedure Class. This template class covers an important role: operating the discrete continuation with acceleration step, that is Algorithm 1, on several parameters (remark that the template real stands for the numerical precision). For this, methods from CMinPack, Model and Runge4Method are exploited via standard association relations. There exist two principal methods.

Method homotopySpGrPr operates Algorithm 1 to recover, from the problem of order zero (OIP) 0 , the contributions of the thrust, of the gravity and of the roundness of the Earth. The user has the choice to deform all these three parameters either at the same time ot sequentially. However, our tests in Section 5.1.2 have shown that the first choice is usually more efficient. Finally, method homotopySpatial is devoted to the spatial continuation on the scenario. The inputs are the initial scenario, its optimal adjoint vector and the desired final scenario and, as the previous methods, deformations can be operated either on all state variables at the same time or sequentially (we adopted simultaneous deformations). The method homotopySpatial is the one adopted for the deformation between a scenario chosen from an initialization grid and the desired final scenario, as detailed in Section 5.

• RealTimeShooting Class. Class RealTimeShooting, whose template variable represents the numerical precision, sets up the methods that are directly endorsed by the user to provide optimal strategy for (OIP). The standard association relation with class HomotopyProcedure is exploited by method shoot to read the technical details of the vehicle, load into the RAM the intialization grid (i.e. adjointGuess.txt), read and analyze the scenario provided by the user and finally evaluate optimal trajectories for interception. A typical instantiation of RealTimeShooting is provided below, via a template example script.

We conclude this section by providing a template example script for a standard usage of this C++ library for an efficient evaluation of optimal interception strategies, showing the ease of considering this C++ library. The user starts to instantiate Real-TimeShooting with a given numerical precision Shooting * shooting = / / I n i t i a l i z e an object of type Shooting new Shooting ( eta , rr , c0 , d0 ,mu0, RT, ve , q0 , tSw , alpha_m , massInit ) ;

It is at the step of declaring the object shooting that the initialization grid provided in adjointGuess.txt is read and copied into the RAM. This operation is required only once for all optimal interception scenarios the user needs to solve. The last iteration consists in declaring the scenario that the user wants to solve Real r0 = . . . , L0 = . . . , l 0 = . . . , v0 = . . . , gamma0 = . . . , chi0 = . . . ; Real r _ f = . . . , L_f = . . . , l _ f = . . . , gamma_f = . . . , c h i _ f = . . . ;

and instantiating the method shoot to recover the optimal solution Real pr , pL , pl , pgamma, pchi , pv , t _ f , / / Some parameters err_r , err_L , e r r _ l , err_gamma , err_chi , finalContinuationParam , f i n a l V e l o c i t y , deltaTReadAdjointGuess , deltaTExecution ; unsigned ret , iterContinuation , iterChartChange , i t e r S i n g u l a r , nearestRawAdjointGuess , fixedFinalTOrNot ; r e t = shooting -> shoot ( 0 , r0 , L0 , l0 ,gamma0, chi0 , v0 , / / Computation r _ f , L_f , l _ f , gamma_f , chi_f , pr , pL , pl ,pgamma, pchi , pv , t _ f , err_r , err_L , e r r _ l , err_gamma , err_chi , finalContinuationParam , iterContinuation , f i n a l V e l o c i t y , iterChartChange , i t e r S i n g u l a r , nearestRawAdjointGuess , deltaTReadAdjointGuess , deltaTExecution , fixedFinalTOrNot ) ;

Parameter ret takes 1 as value if the computation of the optimal strategy ends correctly. Parameters err_ contain the absolute errors between the state found from

Conclusions

shoot and the desired final scenario. Parameters finalContinuationParam, iterContinuation, finalVelocity and iterChartChange contain information on final values of continuation parameters, the cost, nonregular arcs and the number of taken iterations while deltaTReadAdjointGuess, deltaTExecution provide computational times. Finally, nearestRawAdjointGuess furnishes the scenario chosen from the initialization grid to solve the scenario provided by the user and fixedFinalTOrNot specifies whether a fixed final time t f is needed or not (see the beginning of Section 5.1).

Conclusions

In this chapter, we have proposed numerical improvements of the homotopy procedure of Chapter 4 to solve problem (OIP) without minimal final time, by managing high sensitivity of mission conditions and by robustifying the whole algorithm.

The homotopy scheme introduced in Chapter 4 suffers from the high sensitivity of initial conditions, that is, different scenarios may take more or less time to converge to the optimal solution and this prevents to consider one average delay in which, statistically, any shooting correctly converges. Our solution consists in computing offline an initialization grid containing many solved scenarios and the initial value of their optimal adjoint vectors. Thanks to this, any feasible mission can be solved by one spatial homotopy procedure on the scenario, which is initialized by the most suitable (in a metric sense) point of the previous grid. Monte Carlo numerical simulations show that this additional procedure provides optimal solutions for many feasible missions uniformly within a small computational delay, broadly acceptable for real-time interception missions. Moreover, the whole numerical scheme gains in robustness: in the case of quick changes of the initial scenario, such as evasive maneuvers of the target, the new optimal strategy can be computed within the same small average computational delay, allowing real-time strategy update. Finally, our self-contained C++ based code providing this shooting procedure needs only few Megabytes of memory, which is fitting for real-time onboard computations.

The numerical process developed in this chapter represents a first satisfactory answer to the problem initially proposed by ONERA-The French Aerospace Lab, concerning the onboard real-time optimal guidance of launch vehicle, focused on optimal interception missions.

Solving Optimal Control Problems with Delays

Besides the real-time optimal guidance of launch vehicles problem, which, in the context of optimal interception missions, has been satisfactorily treated in the previous chapters, ONERA-The French Aerospace Lab proposed the challenge to provide optimal solutions when delays are added to the formulation of the optimal interception problem (OIP), by maintaining the consistency with the framework proposed in Chapter 4. This translates into solving, via indirect methods, the optimal interception problem with delays (OIP) τ , introduced in Section 2.2.3. The aim of this chapter consists in proposing a numerical strategy, based on homotopy procedures on the delay, to efficiently provide optimal solutions of (OIP) τ , by using indirect methods.

The considered framework makes possible to apply the same strategy to solve optimal control problems with delays far more general than problem (OIP) τ .

The proposed approach can be explained as follows. Consider a general optimal control problem with control and state delays (OCP) τ , as introduced in Section 1.5. The Maximum Principle stated in Theorem 1.5 provides appropriate first-order necessary optimality conditions. However, a good guess of the initial condition related to the optimal adjoint vector does not suffice to make converge correctly classical shooting methods on (OCP) τ , as usual. Indeed, even in the case of regular extremals, the adjoint equations related to this problem contain both forward and backward evolutions of time, therefore, preventing from solving the related dynamical system by standard local integration methods whether only pointwise initial conditions on the adjoint vector are provided. In this context, to correctly integrate the adjoint equations, more information is required, so that, rather, a good numerical guess of the whole adjoint vector function must be provided. At this step, suppose that we are able to solve by indirect method the non-delayed version of (OCP) τ , that is, with the notations of Chapter 1, the standard optimal control problem (OCP) (as we did for (OIP) in Chapter 4). It seems legitimate to wonder if one may solve (OCP) τ by indi-

Chapter 6. Solving Optimal Control Problems with Delays

rect methods starting a homotopy method where the delay represents the deformation parameter and (OCP) is taken as the problem of order zero. This approach is a way to address the flaw of indirect methods applied to optimal control problems with delays: on one hand, the global information of the problem without any delay could be used to initialize efficiently a shooting method with delays and, on the other hand, we could solve the two-point boundary value problem with delays via usual iterative methods for differential equations. Nevertheless, unlike the classical nondelayed optimal control framework in which, under appropriate assumptions, the convergence of homotopy methods is understood and well-established (see Proposition 1.2), nowadays, the well-posedness of indirect methods combined with homotopy procedures on the delay to solve optimal control problems with delays has not been well addressed in the literature. In particular, bifurcation points, singularities or different connected components may be encountered, arising convergence failures.

Our main result consists in proving that, actually, the homotopy procedure on the delay explained above is well-posed. More specifically, we prove that, under appropriate assumptions, the quantities provided by the Maximum Principle, including optimal trajectories and related adjoint vectors, are continuous with respect to the delays. This result ensures the well-posedness of the previously proposed numerical scheme: any homotopy path of delays converges to an extremal of the original optimal control problem with delays. We exploit this result to efficiently apply indirect methods to problem (OIP) τ .

The chapter is organized as follows. Section 6.1 provides the framework to state the rigorous result concerning the continuity properties of the state, of the control and of the adjoint vector, related to problem (OCP) τ , with respect to the delay. In Section 6.2, a numerical algorithm, whose convergence is ensured thanks to the previous continuity result, is proposed to solve general optimal control problems with delays by homotopy on the delay. Moreover, some numerical simulations on nontrivial control problems are proposed. Finally, in Section 6.3 the previous numerical scheme is employed to efficiently solve problem (OIP) τ by means of shooting methods.

Continuity Properties with Respect to Delays

The objective of this section consists in introducing the theoretical result concerning the continuity of the state, of the control and of the adjoint vector, related to the very general problem (OCP) τ , with respect to the delay τ. This fundamental step ensures the well-posedness of a homotopy on the delay to solve (OIP) τ . For sake of clarity in the exposition, we first recall the notations concerning the Maximum Principle related to problem (OCP) τ , provided by Theorem 1.5. Then, the main assumptions and the sought continuity result are introduced, followed by appropriate remarks.

Continuity Properties with Respect to Delays

We consider the general optimal control problem with control and state delays (OCP) τ , introduced in Section 1.5. More precisely, fixing initial state and control functions

Φ 1 ∈ C 0 ([-∆, 0], R n ) and Φ 2 ∈ L ∞ ([-∆, 0), R m )
, respectively, and given a couple of constant delays τ = (τ 1 , τ 2 ) ∈ [0, ∆] 2 , we minimize the cost

C τ (t f , u) = t f 0 f 0 (t , x(t ), x(t -τ 1 ), u(t ), u(t -τ 2 )) d t (6.1) such that ẋ(t ) = f (t , x(t ), x(t -τ 1 ), u(t ), u(t -τ 2 )) , x| [-∆,0] (•) = Φ 1 (•) , x(t f ) ∈ M f (6.2) among all the controls u ∈ L ∞ ([-∆, t f ], R m ) satisfying u| [-∆,0) (•) = Φ 2 (•) , u(t ) ∈ U (6.3)
a.e. in [0, t f ], where the final time t f may be fixed or not. For technical reasons that will be more clear in Chapter 7, we need to work with dynamics and costs satisfying

f 0 (t , x, y, u, v) = f 0 1 (t , x, y, u)+ f 0 2 (t , x, y, v) , f (t , x, y, u, v) = f 1 (t , x, y, u)+ f 2 (t , x, y, v) where f 0 1 : R × R 2n × R m → R, f 0 2 : R × R 2n × R m → R, f 1 : R × R 2n × R m → R n and f 2 : R × R 2n × R m → R n
are continuous and of class (at least) C 2 with respect to their second and third variables. We denote by U τ t f ,R m the set of all admissible controls of (6.2) defined in [-∆, t f ] taking their values in R m while U τ t f ,U denotes the set of all admissible controls of (6.2) defined in [-∆, t f ] taking their values in U . From this,

U τ R m = t f >0 U τ t f ,R m , U τ U = t f >0 U τ t f ,U .
In what follows, optimal solutions of (OCP) τ are denoted by (x τ (•), u τ (•)), with related optimal final time t τ f . We recall that the Hamiltonian related to problem (OCP) τ is

h(t , x, y, p, p 0 , u, v) = 〈p, f (t , x, y, u, v)〉 + p 0 f 0 (t , x, y, u, v) .
For every optimal solution (x τ (•), u τ (•)) of (OCP) τ , Theorem 1.5 provides the existence of a nontrivial couple (p τ (•), p 0 τ ) = 0, where p 0 τ ≤ 0 is constant and

p τ : [0, t τ f ] → R n is absolutely continuous, such that the extremal (x τ (•), p τ (•), p 0 τ , u τ (•)) satisfies, almost everywhere in [0, t τ f ], the following adjoint equations                    ẋ(t ) = ∂h ∂p (t , x(t ), x(t -τ 1 ), p(t ), p 0 , u(t ), u(t -τ 2 )) ṗ(t ) = - ∂h ∂x (t , x(t ), x(t -τ 1 ), p(t ), p 0 , u(t ), u(t -τ 2 )) -1 [0,t τ f -τ 1 ] (t ) ∂h ∂y (t + τ 1 , x(t + τ 1 ), x(t ), p(t + τ 1 ), p 0 , u(t + τ 1 ), u(t + τ 1 -τ 2 )) (6.4)

Chapter 6. Solving Optimal Control Problems with Delays

and the following maximization condition, for every u ∈ U h(t , x(t ), x(t -τ 1 ), p(t ), p 0 , u(t ), u(t -τ 2 )) + (6.5)

1 [0,t τ f -τ 2 ] (t )h(t + τ 2 , x(t + τ 2 ), x(t + τ 2 -τ 1 ), p(t + τ 2 ), p 0 , u(t + τ 2 ), u(t )) ≥ h(t , x(t ), x(t -τ 1 ), p(t ), p 0 , u, u(t -τ 2 )) + 1 [0,t τ f -τ 2 ] (t )h(t + τ 2 , x(t + τ 2 ), x(t + τ 2 -τ 1 ), p(t + τ 2 ), p 0 , u(t + τ 2 ), u) . Furthermore, if M f is a submanifold of R n , locally around x τ (t τ
f ), then the adjoint vector can be built in order to satisfy

p τ (t τ f ) ⊥ T x τ (t τ f ) M f (6.6)
and, moreover, if the final time t τ f is free and both t τ f and t τ f -τ 2 are points of continuity of u τ (•), the extremal (x τ (•), p τ (•), p 0 τ , u τ (•)) satisfies the following final condition

h(t , x(t τ f ), x(t τ f -τ 1 ), p(t τ f ), p 0 , u(t τ f ), u(t τ f -τ 2 )) = 0 . (6.7)
It is important to remark that, when considering optimal control problems where no delays appear, i.e. (OCP) τ=0 = (OCP), this Maximum Principle formulation coincides with the classical formulation given by Theorem 1.1 for the standard problem (OCP).

As we will see in the proof of the main continuity result in Chapter 7, it is crucial to split the case in which the delay τ 2 on the control variable appears from the one which considers only pure state delays. The context of control delays reveals to be more complex, especially, in proving the existence of optimal control for (OCP) τ . Indeed, a standard approach to prove existence would consider usual Filippov's assumptions (as in the classical reference [START_REF] Filippov | On certain questions in the theory of optimal control[END_REF]) which, in the case of control delays, must be extended. In particular, using the Guinn's reduction (see, e.g. [START_REF] Guinn | Reduction of delayed optimal control problems to nondelayed problems[END_REF]), the control system with delays results to be equivalent to a non-delayed system with a larger number of variables depending on the value of τ 2 . Such extension was used in [START_REF] Nababan | A filippov-type lemma for functions involving delays and its application to time-delayed optimal control problems[END_REF]. However, it is not difficult to see that the usual assumption concerning the convexity of the epigraph of the extended dynamics is not sufficient to prove Lemma 2.1 in [START_REF] Nababan | A filippov-type lemma for functions involving delays and its application to time-delayed optimal control problems[END_REF].

We provide more details about this issue in Chapter 7 (see Section 7.3.2).

Led by this important remark, below, we provide the assumptions under which the sought continuity properties with respect to delays hold, by splitting them into three classes, that is, assumptions that must be satisfied by problems with delay appearing only in the state, assumptions that must be satisfied by problems with delay appearing both in the control and in the state and assumptions shared by both these cases.

Chapter 6. Solving Optimal Control Problems with Delays

weakened as follows. If we replace (A 2 ) and (A 3 ) with the assumption "every extremal lift of every solution of (OCP) is normal", then, the conclusion provided in Theorem 6.1 below still holds, except that the convergence properties must be written in terms of closure points. The proof of this fact follows the same guideline used to prove Theorem 6.1, without complications (see Chapter 7 and also [START_REF] Haberkorn | Convergence results for smooth regularizations of hybrid nonlinear optimal control problems[END_REF]Remark 1.11]). Assumptions (B 1 ) and (C ) play a complementary role in proving the convergence property for the adjoint vectors. In particular, Assumption (C 1 ) becomes also crucial to ensure the convergence of optimal controls and trajectories when considering delays both in state and control variables (see Section 7.3.3). Without this assumption proving these last convergences becomes a hard task. The issue is related to the following fact. Let X , Y be Banach spaces and F : X → Y be a continuous map. Suppose that (x k ) k∈N ⊆ X is a sequence such that x k x and F (x k ) F ( x) for some x, x ∈ X . Therefore, in general, we cannot ensure that the two limits coincide, that is x = x, when F is not linear. On the other hand, Assumption (C 2 ) becomes essential to correctly evaluate the convergence of Pontryagin cones when delays in the control variables appear (see Section 7.3.4) while the second statement in Assumption (C 3 ) is not fundamental to obtain the main continuity properties, but it is useful to provide more regularity to convergences related to optimal controls (see Theorem 6.1 below). The previous assumptions allow to provide the continuity of extremals of (OCP) τ with respect to the delay τ in neighborhoods of τ = 0. From the proof presented in Chapter 7, it results evident that, if assumptions (A)-(C ) are extended to some other delay τ ∈ [0, ∆] 2 , these continuity properties are valid for neighborhoods of delay τ. We are now able to state our main result, as contained in the following theorem. Theorem 6.1. Throughout the statement, suppose that assumptions (A) hold.

Consider first the context of pure state delays, that is, problems (OCP) τ for which τ = (τ 1 , 0) and f 2 (t , x, y, u) = f 0 2 (t , x, y, u) = 0, and assume also that assumptions (B ) hold. There exists τ 0 > 0 such that, for every τ = (τ 1 , 0) ∈ (0, τ 0 ) × {0}, each problem (OCP) τ has at least one solution (x τ (•), u τ (•)) defined in [-∆, t τ f ], every extremal lift of which is normal. Let (x τ (•), p τ (•), -1, u τ (•)) be such a normal extremal lift. Therefore, up to continuous extensions on [-∆, t f ], as τ tends to 0, the following convergences hold:

• The final times t τ f converge to t f .

• The trajectories x τ (•) converge uniformly to x(•).

• The adjoint vectors p τ (•) converge uniformly to p(•).

• The tangent vectors ẋτ (•) converge to ẋ(•) in L ∞ for the weak star topology.

If we work in fixed final time t τ f , then, t τ f = t f for every τ ∈ (0, τ 0 ) × {0}.

Homotopy Algorithm and Numerical Simulations

On the other hand, consider general problems (OCP) τ with delays τ = (τ 1 , τ 2 ) in both state and control variables, and, furthermore, assume that assumptions (C ) hold true. There exists τ 0 > 0 such that, for every τ = (τ 1 , τ 2 ) ∈ (0, τ 0 ) 2 , the same conclusions given above hold with t τ f = t f and, in addition, as τ tends to 0, the following holds: • Controls u τ (•), u τ (• -τ) converge to u(•) in L 2 for the weak topology. Moreover, if in addition, there holds that control u(•) assumes its values at extremal points of U , a.e. in [-∆, t f ], as τ tends to 0, one also has that:

• Controls u τ (•), u τ (• -τ) converge to u(•) almost everywhere in [0, t f ].
Lastly, for every τ ∈ [-∆, 0], by extending to the delay τ all the previous assumptions, we have that the optimal solutions (x τ (•), u τ (•)) of problems (OCP) τ (or (x τ (•), ẋτ (•)) in the case of pure state delays) and their related adjoint vectors p τ (•) are continuous with respect to delay τ at τ for the related topologies introduced above.

The most challenging and most important nontrivial conclusion achieved by Theorem 6.1 is the continuous dependence of the adjoint vectors of (OCP) τ with respect to the delay τ. It represents the essential step that allows homotopy methods on the delay based on shooting algorithms to converge robustly. The proof of this fact is not easy and an accurate analysis of the convergence of Pontryagin cones related to the Maximum Principle formulation with delays is required (for details, see Chapter 7). Remark 6.1. It is clear that, the conclusions established by Theorem 6.1 can be strengthened for some special cases (up to properly modify, sometimes, assumptions (A)-(C )). In particular, Theorem 6.1 can be extended to obtain stronger convergence conclusions, by using weaker assumptions, in the specific case of dynamics f that are affine in the two control variables, and of quadratic costs of type

t f 0 K 1 x(t ) 2 + K 2 x(t -τ 1 ) 2 + K 3 u(t ) 2 + K 4 u(t -τ 2 ) 2 d t
where K 1 , K 2 , K 3 , K 4 ≥ 0 are constant. Indeed, considering assumptions (A) and either (B ) or (C ), the convergence properties established in Theorem 6.1 for x τ (•) and p τ (•) still hold and u τ (•) converges to u(•) in L 2 for the weak topology, as the delay τ tends to 0. This fact is also true when U = R m and arises adapting the proof of Theorem 6.1 without particular complications (see Chapter 7 and the proof in [START_REF] Bonalli | Solving optimal control problems for delayed control-affine systems with quadratic cost by numerical continuation[END_REF]Theorem 1]).

Homotopy Algorithm and Numerical Simulations

The convergence properties established by Theorem 6.1 can be exploited to conceive, by adapting the structure of Algorithm 1, a general numerical scheme, based on indirect methods, capable of solving (OCP) τ by applying homotopy procedures on parameter τ, starting from the optimal extremal of its non-delayed version (OCP). Assume to have a guess p τ k+1 (•) of the optimal extremal of (OCP) τ k+1 .

Chapter 6. Solving Optimal Control Problems with Delays

Express u τ k+1 (•) as function of the state variable and of p τ k+1 (•), and solve

     ẋ(t ) = ∂h ∂p •, x(•), p τ k+1 (•), -1, u τ k+1 (•) (t ) x(t ) = φ 1 (t ) , t ∈ [-∆, 0]
From this, with x τ k+1 (•) as solution of the previous system and u τ k+1 (•)

as function of x τ k+1 (•) and of the adjoint vector variable, solve

                                       ṗ(t ) = - ∂h ∂x •, x τ k+1 (•), p(•), -1, u τ k+1 (•) (t ) - ∂h ∂y •, x τ k+1 (•), p(•), p(• -τ 1 k+1 ), -1, u τ k+1 (•) (t + τ 1 k+1 ) , t ∈ [0, t f -τ 1 k+1 ] ṗ(t ) = - ∂h ∂x •, x τ k+1 (•), p(•), -1, u τ k+1 (•) (t ) , t ∈ (t f -τ 1 k+1 , t f ] p(t f ) = p τ k+1 (t f )
.

Denote the solution of the previous system as p τ k+1 (•).

A.3) If convergence is achieved in the previous shooting, set p τ k (•) = p τ k+1 (•) and k → k + 1. Otherwise, set ∆ τ → ∆ τ /2.
end end Algorithm 3: Discrete continuation to solve (OCP) τ via indirect methods.

Homotopy Algorithm and Numerical Simulations

This Optimal Control Problem with Delays (OCP) (CSTR) τ consists in minimizing

C (CSTR) τ (t f , u) = 2 0 x(t ) 2 + 0.1 u(t ) 2 d t such that                          ẋ1 (t ) = 0.5 -x 1 (t ) -R 1 (x 1 (t ), x 2 (t )) , t ∈ [0, 2] ẋ2 (t ) = R 1 (x 1 (t ), x 2 (t )) -(u 1 (t ) + 2)(x 2 (t ) + 0.25) , t ∈ [0, 2] ẋ3 (t ) = x 1 (t -τ) -x 3 (t ) -R 2 (x 3 (t ), x 4 (t )) + 0.25 , t ∈ [0, 2] ẋ4 (t ) = x 2 (t -τ) -2x 4 (t ) -u 2 (t )(x 4 (t ) + 0.25) + R 2 (x 3 (t ), x 4 (t )) -0.25 , t ∈ [0, 2] x 1 (t ) = 0.15 , x 2 (t ) = -0.03 , t ∈ [-τ, 0] , x 3 (0) = 0.1 , x 4 (0) = 0 among all the controls u 1 (•), u 2 (•) ∈ L ∞ ([0, 2], R).
In this example, we have one fixed scalar delay τ which is chosen within the interval [0, 0.8] and acts on the state only.

Functions R 1 , R 2 are given by

R 1 (a, b) = (a + 0.5)exp 25b b + 2 , R 2 (a, b) = (a + 0.25)exp 25b b + 2 .
Since no terminal conditions are imposed, (OCP) (CSTR) τ have only normal extremals. Therefore, the Hamiltonian takes the form

H = p 1 0.5 -x 1 -R 1 (x 1 , x 2 ) + p 2 R 1 (x 1 , x 2 ) -(u 1 + 2)(x 2 + 0.25) +p 3 y 1 -x 3 -R 2 (x 3 , x 4 ) + 0.25 + p 4 y 2 -2x 4 -u 2 (x 4 + 0.25) + R 2 (x 3 , x 4 ) -0.25 -x 2 1 + x 2 2 + x 2 3 + x 2 4 + 0.1u 2 1 + 0.1u 2 2
while the adjoint equations are given by We applied Algorithm 3 to solve (OCP) (CSTR) τ , thanks to Remark 6.1. Several different delays τ were tested, as done also in [START_REF] Oh | Optimal feedback control of time-delay systems[END_REF]. We used N = 1/h = 50 Runge-Kutta time steps, and an error tolerance of 10 -10 and 1500 maximal iterations for hybrd.c routine. Results concerning optimal costs and computational times are reported in Table 6 These values are comparable with the ones obtained by [START_REF] Oh | Optimal feedback control of time-delay systems[END_REF][START_REF] Dadebo | Optimal control of time-delay systems by dynamic programming[END_REF]. Moreover, our continuation scheme finds a solution also for larger delays, i.e. τ ∈ [0.8, 1.5] (see Table 6.1). As expected, the more the delay grows the larger the number of iterations of the continuation method is. In order to check the goodness of these results, a numerical strategy for (OCP) (CSTR) τ considering an AMPL framework is developed (IPOPT solver with an explicit forward Euler scheme and N = 10000, see, e.g. [START_REF] Fourer | Ampl[END_REF][START_REF] Wächter | On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming[END_REF]). The optimal values provided for (OCP) (CSTR) τ by the direct method coincide with the ones obtained by employing Algorithm 3. Some optimal quantities recovered by using our method are shown in Figure 6

                           ṗ1 (t ) = p 1 (t ) + ∂R 1 ∂x (x 1 (t ), x 2 (t )) p 1 (t ) -p 2 (t ) + 2x 1 (t ) -1 [0,2-τ] (t )p 3 (t + τ) ṗ2 (t ) = p 2 (t )(u 1 (t ) + 2) + ∂R 1 ∂y (x 1 (t ), x 2 (t )) p 1 (t ) -p 2 (t ) + 2x 2 (t ) -1 [0,2-τ] (

Numerical Strategy to Solve (OIP) τ

The objective of this section consists in providing solutions via indirect methods for the optimal interception problem with delays (OIP) τ , which we introduced in Section 2.2.3, by making use of the homotopy procedure developed within Algorithm 3.

The state variables are represented by the tuple x = (r, v,b) ∈ R 6 \ {0} × S 2 and the delay enters within the state variable b as a time lag on the rotational velocity. In order to provide numerical results for (OIP) τ , we simplify the treatise by making use of local coordinates, needed to correctly represents the state delay. Without loss of generality, we assume that the projection of optimal trajectories onto variables (r, v) belongs to the domain of the local chart (V a , ϕ a ), introduced in Chapter 3 (see Section 3.2.1). From this, the evolution of variables (r, v) is completely described by dynamics (3.14). In particular, with respect to the notation provided by dynamics (3.14), we describe variables (w 1 , w 2 , w 3 ) as follows

w 1 = cos α , w 2 = sin α cos β , w 3 = sin α sin β , α ∈ - π 2 , π 2 , β ∈ (-π, π) (6.8)
where α and β are local coordinates for S 2 . Summing up all the previous remarks, we focus on the Local Optimal Interception Problem with Delays (OIP) a τ which consists in minimizing the cost

C (t f , u α , u β ) = -v 2 (t f ) +C u t f 0 (u 2 α + u 2 β ) d t (6.9) such that                                            ṙ = v sin γ , L = v r cos γ cos χ , ˙ = v r cos γ sin χ cos L v = f T m cos α -d + ηc m sin 2 α v 2 -g sin γ γ = ω sin α(t -τ) cos β + v r - g v cos γ χ = ω cos γ sin α(t -τ) sin β + v r cos γ sin χ tan L α = u α , β = u β (r (t ), L(t ), l (t ), v(t ), γ(t ), χ(t ), α(t ), β(t )) ∈ V × - π 2 , π 2 × (-π, π) (6.10) among all controls u α , u β ∈ L ∞ ([0, t f ], R)
, where the final time t f may be free or not, τ is a scalar constant delay and, similarly to the original global problem (OIP) τ , the

Numerical Strategy to Solve (OIP) τ

initial conditions are fixed to some given point and the final manifold is taken as in (2.21). Remark that, differently from the original problem (OIP) τ , we substitute the constraints on the controls by a direct penalization in the cost (with the help of C u ). This is not limiting and is used to give nice formulations of the Maximum Principle.

The description of problem (OIP) τ can be simplified once more, by making one further change of variable in (OIP) a τ . This simplification will help to define an initialization strategy for the shooting scheme applied to (OIP) a τ (see the following sections). As done for the initialization of the optimal interception problem without delay in Chapter 4, we introduce the new variable v s = ln(v) and, since ln is a monotonic function, we replace (OIP) a τ by the equivalent formulation (still denoted by (OIP) a τ ) which consists in minimizing the cost

C (t f , u α , u β ) = -v s (t f ) +C u t f 0 (u 2 α + u 2 β ) d t (6.11) such that                                              ṙ = e v s sin γ , L = e v s r cos γ cos χ , ˙ = e v s r cos γ sin χ cos L vs = f T e v s m cos α -d + ηc m sin 2 α e v s - g e v s sin γ γ = ω sin α(t -τ) cos β + e v s r - g e v s cos γ χ = ω cos γ sin α(t -τ) sin β + e v s r cos γ sin χ tan L α = u α , β = u β (r (t ), L(t ), l (t ), e v s (t ) , γ(t ), χ(t ), α(t ), β(t )) ∈ V × - π 2 , π 2 × (-π, π) (6.12)
among all controls u α , u β ∈ L ∞ ([0, t f ], R), where the final time t f mat be free or not, τ is a scalar constant delay and, similarly to the original problem (OIP) τ , the initial conditions are fixed to some given point and the final manifold is taken as in (2.21).

In order to efficiently solve (OIP) a τ , we propose to apply first the homotopy scheme derived in Chapter 4 on the non-delayed version of (OIP) a τ to deal with the components of thrust, of the gravity and of the roundness of the Earth, and then, recover the sought optimal solution by iterating Algorithm 3. Our aim is achieved if a good approximated initial guess for a shooting method applied on the simplified version of (OIP) a τ , where neither thrust, gravity, roundness of the Earth nor delays appear, is provided. Therefore, in what follows, we develop firstly an analysis close to the initialization scheme provided for the non-delayed problem (OIP), in Section 4.2.

Local Initialization Procedure for (OIP) τ

In what follows, we provide an efficient initial guess to correctly initialize a shooting method on the Local Optimal Interception Problem with Delays of Order Zero (OIP) a 0 = (OIP) a τ=0 , which consists in minimizing cost (6.11) under the simplified dynamics

                 ṙ = e v s sin γ , L = e v s r cos γ cos χ , ˙ = e v s r cos γ sin χ cos L γ = e v s c m sin α cos β , χ = e v s c m cos γ sin α sin β vs = -d + ηc m sin 2 α e v s , α = u α , β = u β among all controls u α , u β ∈ L ∞ ([0, t 0 f ], R)
, where the final time t 0 f is free and the delay τ is removed. The scenario of order zero is chosen, respecting (2.21), in order to make the problem simple enough to be solved (as specified within Section 4.1.1). The main idea resides on the fact that, up to denote w 2 = sin α cos β , w 3 = sin α sin β (6.13) and add the two new variables α, β to the formulation, (OIP) a 0 is structurally close to the problem of order zero derived for (OIP) in Chapter 4. Thus, it is legitimate to wonder if one may initialize indirect methods on (OIP) a 0 by using the guidance law derived throughout Section 4.2, and more precisely, its time derivatives.

We now proceed formally. Under the assumption that the time derivative of the guidance law (4.22)-(4.23) is close to the optimal control related to (OIP) a 0 , we recover an efficient guess of the initial value of the adjoint vector related to (OIP) a 0 as follows. First of all, without loss of generality, we can assume that cos α = 0 and sin α = 0 almost everywhere in [0, t 0 f ] (this depends on the simplified scenario, that we use to solve (OIP) a 0 ). From this, coupling (6. Since the behaviors of ṗα and ṗβ are known from (6.15) (by exploiting simply system (4.17 (6.17) From (6.17 Efficient initial guesses of p r (0), p L (0), p (0), p v s (0), p γ (0) and p χ (0) for a shooting algorithm on (OIP) a 0 are therefore given by solving expression (6.15)-( 6.18) at t = 0.

Numerical Simulations for (OIP) τ

The numerical scheme used to solve (OIP) a τ is close to the one provided in Figure 4.1 of Chapter 4. More specifically, we resume this homotopy procedure in Figure 6.2. First, the derivative of the guidance law (4.22)-(4.23), law recovered in Chapter 4, is used to initialize the homotopy procedure on the thrust, on the gravity and on the roundness of the Earth to solve the optimal interception problem without any delay (this is represented in Figure 4.1 of Chapter 4). More precisely, this initialization is provided by the scheme (6.15)- (6.18). Once the optimal solution of (OIP) a τ=0 is obtained, a discrete continuation starts on the parameter ε, representing the temporary delay, to make ε converges towards τ, recovering so an optimal solution of (OIP) a τ .

Derivative of the Guidance Law (4.22)-(4.23) For these numerical simulations, we adopt the first scenario provided in Section 4.3, to which we add initial and final conditions for α and β such that they are equivalent to the conditions obtained by the optimal control found solving problem (OIP) C 1 ,1 . With the notations of Section 4.3, this becomes (in standard units) h 0 = 1000 , v 0 = 500 , γ 0 = 0 , α 0 = 0.48 , β 0 = -0.43

(OIP) a ε=0 (OIP) a ε=τ λ 1 → 1 λ 2 → 1 ε → τ
h f = 5000 , L T, f = 14000 , T, f = -2000 , γ f = - π 6 , χ f = π 6 α f = -0.36 , β f = -0.9 .
This choice is made to maximize the chances to provide a good initialization of the problem without delays, by the procedure obtained thanks to relations (6.15)- (6.18). This is not limiting, because the scenario can be modified in a second time by spatial homotopy, as done in Chapter 4. For sake of concision in the exposition, without loss of generality, we avoid to apply also the deformation on the scenario.

Numerical Strategy to Solve (OIP) τ

Remark that, in this case, we do not consider any intermediate scenario, so that, only the homotopy on the thrust, on the gravity and on the roundness of the Earth appear in the first step of the diagram in Figure 6.2. We motivate our choice by the argument provided in Section 4.3.2, concerning the guidance law (4.22)-(4.23), and the fact that, empirically, for scenarios such that the guidance law provide a good initialization of the interception problem without delays, the derivative of this law should correctly initialize the augmented version (6.9)-(6.10). In order to make the numerical computations easier to manipulate, we impose a fixed final time t f = 22.1, which is equivalent to the optimal final time achieved for problem (OIP) C 1 ,1 , in Section 4.3. Finally, we impose C u = 1000, whose choice is merely practical and is needed to avoid that controls u α and u β may take too large values.

The computational framework is similar to the one in Chapter 4. In particular, we fix 60 time steps for the explicit two-order Runge-Kutta for the integration of differential equations, which gives a total dimension related to the shooting algorithm of 248 variables (see the discussion of Section 6.2.2: 60 • 4 variables due to 2 delayed adjoint variables and their numerical approximations both at integer and intermediate time steps, and 8 more variables due to the transversality conditions).

From the Derivative of the Guidance Law to the Problem without Delays

The first iteration of the numerical scheme presented in Figure 6.2 consists in using the computations arising from relations (6.15)-(6.18) to solve problem (OIP) a τ=0 . Figures 6.3 and 6.4 show some numerical quantities associated to these solutions. The orange curves in figures 6.3 and 6.4 represent the solution obtained by forcing controls (u α (•), u β (•)) obtained by differentiating the guidance law (4.22)-(4.23) (this is obtained immediately from relations (6.15)-(6.18)). Comparing figures 6.3 and 6.4 with figures 4.2 and 4.5 (remark that w 2 2 + w 2 3 = sin 2 α), one sees that the derivative of the guidance law (4.22)-(4.23) provides, via (6.15)-(6.18), a good guess to obtain an optimal solution of (OIP) a τ=0 , whose trajectory is moreover close to the one corresponding to problem (OIP) C 1 ,1 of Section 4.3 (see Figure 4.5 a)). This can be checked also by observing the optimal quantities obtained in this context (note that the derivative of the guidance law is used to intialize directly the final scenario and no other intermediate scenarios), that is (in standard units) Derivative of the Guidance Law (4.22)-(4.23)

→ → → t f = 49 v(t f ) = 190.2 , (OIP) 0 → → → v(t f ) = 803.3 .
This whole step (from (6.15)-(6.18) to the solution of (OIP) a τ=0 ) takes on average 750 milliseconds, as in the simulations of Chapter 4 related to problem (OIP). The solution of the original problem (OIP) a τ is recovered from the solution of (OIP) a ε=0 by making ε converge to τ, whose solutions are in figures 6.5 and 6.6 for different τ.

Chapter 6. Solving Optimal Control Problems with Delays

The new optimal velocities achieved are the following (in standard units)

(OIP) a τ=2 → → → v(t f ) = 789.1 , (OIP) a τ=4 → → → v(t f ) = 781.4
and the computations need one homotopy iteration on ε and 1200 milliseconds to solve (OIP) a τ=2 , while two homotopy iterations on ε and 3400 milliseconds to solve (OIP) a τ=4 . The non-negligible increase of computational times are due to the higher dimension of the shooting problem that must be solved at each iteration of the homotopy procedure on the delay. One remarks from Figure 6.6 b) that, for this particular scenario, the stability constraint is satisfied. Moreover, even for not too large Chapter 6. Solving Optimal Control Problems with Delays values of the delay, the optimal strategy change drastically from the non-delayed one, arising a discontinuity of the first time derivative of control u α at time t f -τ (see Figure 6.5 b)). The optimality of the trajectories obtained by this homotopy procedure is verified by the AMPL framework, combined with IPOPT (see, e.g. [START_REF] Fourer | Ampl[END_REF][START_REF] Wächter | On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming[END_REF]). Even if physically meaningless, we stressed this optimal control problem with delays by adding very large values of τ. Some obtained results are in figures 6.7 and 6.8. We see that the optimal solutions are found also for τ = 8 s and Figure 6.7 shows that a discontinuity of the first time derivative of control u α arises at time t f -τ. Concerning

Adding More Delay

Conclusions

the optimal velocities, we obtain the following values (in standard units)

(OIP) a τ=6 → → → v(t f ) = 836.4 , (OIP) a τ=8 → → → v(t f ) = 839.3
and the computations need two homotopy iterations on ε and 4700 milliseconds to solve (OIP) a τ=6 , while five homotopy iterations on ε and 17500 milliseconds to solve (OIP) a τ=8 . We ascertained that, in this case, the most of the used computational time is lost in the computing of failing homotopy steps on ε. This seems the price to pay for taking too large delays. The optimality of these trajectories is verified by the AMPL framework, combined with IPOPT (see, e.g. [START_REF] Fourer | Ampl[END_REF][START_REF] Wächter | On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming[END_REF]).

Conclusions

In this chapter, we have proposed a numerical scheme exploiting homotopy methods to solve the optimal interception problem with delays (OIP) τ , via indirect methods. This task is far from being easy. Indeed, solving via indirect methods general optimal control problems with control and state delays (OCP) τ requires the integration of particular adjoint equations in which forward and backward terms appear simultaneously, preventing from exploiting local information, such as an initial guess of the adjoint vector, to efficiently solve these equations. Rather, a global guess of the whole adjoint vector is needed to solve the related shooting problem. Our idea consists in solving (OCP) τ via indirect methods by operating a homotopy procedure on the delay. More specifically, with the optimal extremal of the non-delayed problem (OCP) τ=0 as global guess, we start a homotopy where τ is the deformation parameter. The algorithm makes the delay parameter converge from 0 (the non-delayed problem) towards the desired value τ (the original delayed problem), therefore, obtaining the optimal solution of (OCP) τ . Our main contribution is a theoretical result inferring that the Pontryagin extremals of (OCP) τ are continuous with respect to τ (for appropriate topologies), which motivates the previous procedure by proving that every homotopy path of delays followed by the algorithm, which starts from the extremal of (OCP) τ=0 , converges correctly to the desired extremal related to (OCP) τ . The proposed procedure holds for general optimal control problems with delays. As a particular case, we applied it to efficiently recover optimal solutions, via indirect methods, of the optimal interception problem with delays (OIP) τ .

The present chapter focused mainly on the development of this numerical homotopy algorithm starting from our theoretical continuity result. The next chapter aims to provide a complete proof of the continuity properties, with respect to τ, of extremals related to general optimal control problems with control and state delays (OCP) τ .

Continuity Properties of Pontryagin Extremals

In the previous chapter, we focused on how to efficiently solve general optimal control problems with control and state delays (OCP) τ , via indirect methods. The proposed algorithm is based on a homotopy procedure acting by deforming the delay.

The well-posedness of this numerical scheme arises from some continuity properties, with respect to τ, of extremals related to (OCP) τ . The aim of this chapter consists in proving such continuity properties, under appropriate assumptions, in the very general case of optimal control problems with control and state delays.

We focus on the proof of Theorem 6.1. As pointed out in Section 6.1, we only need to consider continuity properties around τ = 0, whose proof goes in three main steps. First, by using assumptions on the non-delayed version of the problem only, we infer the controllability of problems (OCP) τ , for every positive τ sufficiently small. The previous step requires some implicit function theorem involving parameters. This allows to proceed to the second part, which consists in showing the existence of solutions of (OCP) τ , for τ sufficiently small, and their convergences, as τ tends to 0, towards solutions of (OCP) τ=0 . In the case of both control and state delays, we will see the importance of considering affine systems throughout this step. Finally, we address the more difficult issue of establishing the convergence, as τ tends to 0, of the adjoint vectors related to (OCP) τ towards the adjoint vector of (OCP) τ=0 . For this step, a refined analysis on the convergence of Pontryagin cones is needed.

The chapter is organized as follows. Section 7.1 recalls the main steps of the proof related to the Maximum Principle for problems (OCP) τ , that is, Theorem 1.5. At the best of our knowledge, the proof of Theorem 1.5 via needle-like variations does not appear explicitly in the literature, therefore, we believe useful to report it directly. Section 7.2 provides a convenient conic version of the implicit function theorem. Finally, in Section 7.3, we report the whole proof of the result, as detailed above.

Proof of the PMP Using Needle-Like Variations

In this section we sketch the proof of the Maximum Principle for (OCP) τ (i.e. Theorem 1.5) using needle-like variations. For this, we do not rely on the assumptions of Theorem 6.1, giving the result for a large class of control systems with constant delays (as described in Section 1.5). Our reasoning is not affected for problems with free final time, since, we do not employ the well known reduction to a fixed final time problem, but rather, we modify conveniently the Pontryagin cone to keep track of the free variable t τ f , by making L 1 -variations on t τ f (as done in [START_REF] Kharatishvili | The maximum principle in the theory of optimal processes involving delay[END_REF], for pure state delays).

Preliminary Notations

Fix a constant delay τ = (τ 1 , τ 2 ) ∈ [0, ∆] 2 . Consider (OCP) τ as given by formulation (6.1)-( 6.3) and let (x τ (•), u τ (•)) be an optimal solution. For every positive final time t f , introduce the instantaneous cost function

x 0 τ (•) defined in [-∆, t f ] and solution of    ẋ0 (t ) = f 0 (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 )) , t ∈ [0, t f ] x 0 (t ) = 0 , t ∈ [-∆, 0] such that cost (6.1) provides C τ (t f , u τ ) = x 0 τ (t f ).
We defined the extended state x = (x, x 0 ) and the extended dynamics f (t , x, ỹ, u, v) = ( f (t , x, y, u, v), f 0 (t , x, y, u, v)), for which, we will often denote f (t , x, y, u, v) = f (t , x, ỹ, u, v). Therefore, consider the following extended dynamical problem in R n+1

         ẋ(t ) = f (t , x(t ), x(t -τ 1 ), u(t ), u(t -τ 2 )) , t ∈ [0, t f ] x| [-∆,0] (t ) = (φ 1 (t ), 0) , x(t f ) ∈ M f × R u(•) ∈ L ∞ ([-∆, t f ],U ) , u| [-∆,0] (t ) = φ 2 (t ) . ( 7.1)
As provided in Chapter 6, the set of all admissible controls of (7.1) defined in [-∆, t f ] taking their values in R m is denoted by Ũ τ t f ,R m , while Ũ τ t f ,U denotes the set of all admissible controls of (7.1) defined in [-∆, t f ] taking their values in U . From this

Ũ τ R m = t f >0 Ũ τ t f ,R m , Ũ τ U = t f >0 Ũ τ t f ,U .
The extended end-point mapping is defined as

Ẽτ,t f : Ũ τ t f ,R m → R n+1 : u → x(t f )
where x(•) is the unique solution of problem (7.1), related to control u(•) ∈ Ũ τ t f ,R m . As standard facts (see, e.g. [START_REF] Bonnard | Singular trajectories and their role in control theory[END_REF][START_REF] Jurdjevic | Geometric control theory[END_REF]), the set Ũ τ t f ,R m , endowed with the standard topology of L ∞ ([-∆, t f ], R m ), is open and the end-point mapping is smooth on Ũ τ t f ,R m .

Proof of the PMP Using Needle-Like Variations

For every t ≥ 0, define the extended accessible set Ãτ,U (t ) as the image of the extended end-point mapping Ẽτ,t restricted to Ũ τ t ,U , where Ãτ,U (0) = {(φ 1 (0), 0)}. The following fact is at the basis of the proof of the Maximum Principle (see, e.g. [START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF][START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]). Lemma 7.1. For every optimal solution (x τ (•), u τ (•)) of (OCP) τ defined in [-∆, t τ f ], the point xτ (t τ f ) belongs to the boundary of the set Ãτ,U (t τ f ).

Needle-Like Variations and Pontryagin Cones

In what follows we consider (OCP) τ whit free final time, remarking that all the proposed results can be easily adapted for problems with fixed final time (see, e.g. [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]). Moreover, we suppose that the optimal final time t τ f is a Lebesgue point for the optimal control u τ (•) of (OCP) τ and of u τ (• -τ 2 ). Otherwise, we can extend all the conclusions that follow by using closure points of t τ f (in the same way as in [START_REF] Lee | Foundations of optimal control theory[END_REF][START_REF] Gamkrelidze | Principles of optimal control theory[END_REF]).

For a couple of delays τ = (τ 1 , τ 2 ) ∈ [0, ∆] 2 , let (x τ (•), u τ (•)) be a solution of (OCP) τ and, without loss of generality, extend u τ (•) by some constant vector of U in

[t τ f , t τ f + τ 2 ]. Let p ≥ 1 be an integer and consider 0 < t 1 < • • • < t p < t τ
f Lebesgue points respectively of u τ (•), u τ (•-τ 2 ) and of u τ (•+τ 2 ). Choosing p arbitrary values u i ∈ U , for every η i > 0 such that -∆ ≤ t i -η i , the needle-like variation π = {t 1 , . . . , t p , η 1 , . . . , η p , u 1 , . . . , u p } of control u τ (•) is defined by the modified control

u π τ (t ) = u i t ∈ (t i -η i , t i ] ,
u τ (t ) otherwise .

Control u π τ (•) takes its values in U and, by continuity with respect to initial data, for every η i > 0 small enough, u π τ (•) ∈ Ũ τ U . Moreover, whenever (η 1 , . . . , η p ) → 0, the trajectory xπ τ (•), solution of the dynamics of (7.1) related to control u π τ (•), converges uniformly to xτ (•) = (x 0 τ (•), x τ (•)). For every value z ∈ U and appropriate Lebesgue point s ∈ (0, t τ f ), we define the vectors

ω - z (s) = f (s, x τ (s), x τ (s -τ 1 ), z, u τ (s -τ 2 )) (7.2) -f (s, x τ (s), x τ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) ω + z (s) = f (s + τ 2 , x τ (s + τ 2 ), x τ (s + τ 2 -τ 1 ), u τ (s + τ 2 ), z) (7.3) -f (s + τ 2 , x τ (s + τ 2 ), x τ (s + τ 2 -τ 1 ), u τ (s + τ 2 ), u τ (s))

Chapter 7. Continuity Properties of Pontryagin Extremals

and, given ξ ∈ R n+1 , we denote by ṽτ s,ξ (•) the solution of the following linear system

                 ψ(t ) = ∂ f ∂x (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 ))ψ(t ) + ∂ f ∂y (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 ))ψ(t -τ 1 ) ψ(s) = ξ , ψ(t ) = 0 , t ∈ (s -τ 1 , s) . ( 7
(t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 )), -f (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 )
) and vectors wτ s,z (t ) where z ∈ U and 0 < s < t is a Lebesgue point of u τ (•), u τ (•-τ 2 ) and of u τ (•+τ 2 ). The first Pontryagin cone K τ (t ) ⊆ R n and the augmented first Pontryagin cone K τ 1 (t ) ⊆ R n at x τ (t ) for the non-augmented system are defined similarly, considering dynamics f instead of the extended dynamics f . Obviously, K τ (t ), K τ 1 (t ) are the projections onto R n of K τ (t ), K τ 1 (t ), respectively.

Remark 7.1. In the case of optimal control problems without delays, i.e. (OCP) τ=0 , the definition of first Pontryagin cones is slightly different from the one obtained from Definition 7.1 by substituting τ = 0. Indeed, considering K 0 (t ), vectors wτ s,z (t ) are substituted by single variations ṽ0 s,ω z (s) (t ) for which ω z (s) = f (s, x 0 (s), x 0 (s), z, z) -f (s, x 0 (s), x 0 (s), u 0 (s), u 0 (s))

where (x 0 (•), u 0 (•)) is an optimal solution of (OCP) τ=0 (see, e.g. [START_REF] Pontryagin | Mathematical theory of optimal processes[END_REF]).

The proof of the Maximum Principle is established on the following variational result.

Lemma 7.2. Let (δ, η 1 , . . . , η p ) ∈ R × R p + small enough. For every t p < t ≤ t τ f Lebesgue point of u τ (•) and of u τ (• -τ 2 ), the following expression holds xπ τ (t + δ) = xτ (t ) + δ f (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 )) + p i =1 η i ṽτ t i ,ω - u i (t i ) (t ) + ṽτ t i +τ 2 ,ω + u i (t i ) (t ) + o δ + p i =1 η i .

Proof of the PMP Using Needle-Like Variations

Proof. The proof argues by induction. We provide computations for p = 1. The inductive step develops in the same manner, as the standard case (see, e.g. [START_REF] Pontryagin | Mathematical theory of optimal processes[END_REF]). For this, let t 1 < t ≤ t τ f Lebesgue point of u τ (•) and of u τ (• -τ 2 ). First, let us show that

xπ τ (t ) -xτ (t ) = η 1 wτ t 1 ,u 1 (t ) + o(η 1 ) = η 1 ṽτ t 1 ,ω - u 1 (t 1 ) (t ) + ṽτ t 1 +τ 2 ,ω + u 1 (t 1 ) (t ) + o(η 1 ) . (7.5)
We consider directly the case t ≥ t 1 + τ 2 (the case t < t 1 + τ 2 is similar but easier). There holds

xπ τ (t ) -xτ (t ) -η 1 wτ t 1 ,u 1 (t ) ≤ xπ τ (t 1 + τ 2 ) -xτ (t 1 + τ 2 ) -η 1 wτ t 1 ,u 1 (t 1 + τ 2 ) + t t 1 +τ 2 f (s, xπ τ (s), xπ τ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) -f (s, xτ (s), xτ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) -η 1 ẇτ t 1 ,u 1 (s) d s .
By exploiting the facts that t 1 is a Lebesgue point of u τ (•), u τ (• -τ 2 ) and of u τ (• + τ 2 ), and that xπ τ (•) converges uniformly to xτ (•), developing the extended dynamics until the second order, the first term of the expression above can be bounded as follows:

xπ τ (t 1 + τ 2 ) -xτ (t 1 + τ 2 ) -η 1 wτ t 1 ,u 1 (t 1 + τ 2 ) ≤ t 1 t 1 -η 1 f (s, xπ τ (s), xπ τ (s -τ 1 ), u 1 , u τ (s -τ 2 )) -f (s, xτ (s), xτ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) d s -η 1 ω - u 1 (t 1 ) + t 1 +τ 2 -η 1 t 1 f (s, xπ τ (s), xπ τ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) -f (s, xτ (s), xτ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) -η 1 ẇτ t 1 ,u 1 (s) d s + t 1 +τ 2 t 1 +τ 2 -η 1 f (s, xπ τ (s), xπ τ (s -τ 1 ), u τ (s), u 1 ) -f (s, xτ (s), xτ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) -η 1 ẇτ t 1 ,u 1 (s) d s-η 1 ω + u 1 (t 1 ) ≤ t 1 t 1 -η 1 f (s, xτ (s), xτ (s -τ 1 ), u 1 , u τ (s -τ 2 )) -f (s, xτ (s), xτ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) d s-η 1 ω - u 1 (t 1 ) +o(η 1 ) + t 1 +τ 2 -η 1 t 1 ∂ f ∂x (s, xτ (s), xτ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) • xπ τ (s) -xτ (s) -η 1 wτ t 1 ,u 1 (s) d s + t 1 +τ 2 -η 1 t 1 ∂ f ∂y (s, xτ (s), xτ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) • xπ τ (s -τ 1 ) -xτ (s -τ 1 ) -η 1 wτ t 1 ,u 1 (s -τ 1 ) d s + t 1 +τ 2 -η 1 t 1 1 0 d 2 f s, (σ xτ + (1 -σ) xπ τ )(s), (σ xτ + (1 -σ) xπ τ )(s -τ 1 ), u τ (s), u τ (s -τ 2 ) • • xπ τ (s) -xτ (s) 2 + xπ τ (s -τ 1 ) -xτ (s -τ 1 ) 2 + 2 xπ τ (s) -xτ (s) xπ τ (s -τ 1 ) -xτ (s -τ 1 ) d σd s +η 1 t 1 t 1 -η 1 ẇτ t 1 ,u 1 (s + τ 2 )d s + t 1 t 1 -η 1 f (s + τ 2 , xτ (s + τ 2 ), xτ (s + τ 2 -τ 1 ), u τ (s + τ 2 ), u 1 ) -f (s + τ 2 , xτ (s + τ 2 ), xτ (s + τ 2 -τ 1 ), u τ (s + τ 2 ), u τ (s)) d s -η 1 ω + u 1 (t 1 ) + o(η 1 ) .

Chapter 7. Continuity Properties of Pontryagin Extremals

Therefore, by bounding the derivatives of the extended dynamics, we have

xπ τ (t 1 +τ 2 )-xτ (t 1 +τ 2 )-η 1 wτ t 1 ,u 1 (t 1 +τ 2 ) ≤ C1 t 1 +τ 2 -η 1 t 1 -τ 1 xπ τ (s)-xτ (s)-η 1 wτ t 1 ,u 1 (s) d s+o(η 1 )
where C1 ≥ 0 is an appropriate constant. Exactly with the same technique, we can bound the remaining term as

t t 1 +τ 2 f (s, xπ τ (s), xπ τ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) -f (s, xτ (s), xτ (s -τ 1 ), u τ (s), u τ (s -τ 2 )) -η 1 ẇτ t 1 ,u 1 (s) d s ≤ C2 t t 1 +τ 2 -τ 1 xπ τ (s) -xτ (s) -η 1 wτ t 1 ,u 1 (s) d s + o(η 1 )
where C2 ≥ 0 is another appropriate constant. Coupling the two last results with the Grönwall's inequality, expression (7.5) follows straightforwardly. The conclusion comes from (7.5) and the fact that t is a Lebesgue point of u τ (•) and of u τ (• -τ 2 ).

Proof of The Maximum Principle

The proof of the Maximum Principle using needle-like variations argues by contradiction using the following classical well-established result (see, e.g. [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF][START_REF] Bonnard | Mécanique céleste et contrôle de systemes spatiaux[END_REF]).

Lemma 7.3 (Conic Implicit Function Theorem).

Let C ⊆ R m be a convex subset with non empty interior, of vertex 0, and F : C → R n be a Lipschitzian mapping such that F (0) = 0 and F is Gâteaux differentiable at 0 along admissible directions of C (in the sense of [START_REF] Bonnard | Mécanique céleste et contrôle de systemes spatiaux[END_REF]Lemma 46]). Assume that d F (0) • Cone(C ) = R n , where Cone(C ) stands for the convex cone of C . Then, 0 ∈ Int F (V ∩C ), for every neighborhood V of 0 in R m .

Consider any integer p ≥ 1 and a positive real number ε p > 0. Define

G τ p : B ε p (0) ∩ R × R p + → R n+1 : (δ, η 1 , . . . , η p ) → xπ τ (t τ f + δ) -xτ (t τ f )
where π is any variation of control u τ (•) and ε p is small enough such that G τ p is welldefined (see Section 7.1.2). The following statements hold: [START_REF] Bonnard | Mécanique céleste et contrôle de systemes spatiaux[END_REF]Lemma 46]), thanks to Lemma 7.2.

• G τ p (0) = 0 and G τ p is Lipschitz continuous. • G τ p is Gâteaux differentiable at 0 along admissible directions of R × R p + (in the sense of

The Lipschitz behavior of G τ

p is proved by a recursive use of needle-like variations at t i -η i , 1 ≤ i ≤ p (for η i small enough), Lebesgue points of u τ (•). The proof makes a recursive use of Lemma 7.2. Remark that, since t i -η i are Lebesgue points of u τ (•) only for almost every η i , the recursive use of Lemma 7.2 can be done only almost everywhere. The conclusion follows from the continuity of G τ p and density arguments.

Conic Implicit Function Theorem with Parameters

The Maximum Principle is then established as follows. Suppose, by contradiction, that the augmented first Pontryagin cone K τ 1 (t τ f ) coincides with R n+1 . Then, by definition, there would exist an integer p ≥ 1, a variation π of u τ (•) and a positive real number ε p > 0 such that

dG τ p (0) • (R × R p + ) = K τ 1 (t τ f ) = R n+1 .
In this case, Lemma 7.3 would imply that the point xτ (t τ f ) belongs to the interior of the accessible set Ãτ,U (t τ f ), which contradicts Lemma 7.1. The Hahn-Banach theorem provides therefore the following more general Lagrange multiplier rule.

Lemma 7.4. There exists ψτ

∈ R n+1 \ {0} (Lagrange multiplier) such that 〈 ψτ , f (t τ f , x τ (t τ f ), x τ (t τ f -τ 1 ), u τ (t τ f ), u τ (t τ f -τ 2 ))〉 = 0 , 〈 ψτ , ṽτ 〉 ≤ 0 , ∀ ṽτ ∈ K τ (t τ f ) .
The relations provided by Lemma 7.4 allow to derive the necessary conditions (6.4)-(6.7) arising from Theorem 1.5 (we skip these computations, referring to [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF][START_REF] Bonnard | Mécanique céleste et contrôle de systemes spatiaux[END_REF] for details). The relation between the adjoint vector satisfying (6.4) and the above Lagrange multiplier ψτ

= (ψ τ , ψ 0 τ ) is that (p τ (•), p 0 τ ) is built so that p τ (t τ f ) = ψ τ , p 0 τ = ψ 0 τ .
We will make use of the result below, which follows from the previous considerations. Lemma 7.5. Consider the free final time optimal control problem (OCP)=(OCP) τ=0 . For any optimal trajectory x(•) of (OCP), the following statements are equivalent:

• The trajectory x(•) has an unique extremal lift (x(•), p(•), p 0 , u(•)) whose adjoint (p(•), p 0 ) is unique up to a multiplicative scalar, which is normal, i.e. p 0 τ < 0.

• The first Pontryagin cone

K τ=0 1 (t f ) is a half-space of R n+1 and K τ=0 1 (t f ) = R n .

Conic Implicit Function Theorem with Parameters

The main first step of the proof of Theorem 6.1 makes use of the procedure detailed in Section 7.1. More specifically, we need the needle-like variation formula and the conic implicit function theorem. However, Lemma 7.3 is not suited to this situation because we have to take into account the dependence with respect to the delay τ. Indeed, the proof of Lemma 7.3 is based on the Brower's fixed point theorem (see, e.g. [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]) which does not consider continuous dependence with respect to parameters (which, in our case, is represented by τ). Therefore, a more general version of the conic implicit function theorem depending on parameters must be introduced.

Chapter 7. Continuity Properties of Pontryagin Extremals

When considering the delay τ as a varying parameter, the variation formula provided by Lemma 7.2 holds only for almost every τ, and this, because we need that each t i is a Lebesgue point of u τ (•), u τ (• -τ) and of u τ (• + τ). This obliges to introduce a notion of conic implicit function theorem which, on one hand, ensures a continuous dependence with respect to parameters and, on the other hand, deals with quantities defined uniquely on dense subsets. The notion of differentiability needed is the following. A function f : C ⊆ R p → R n is said almost everywhere strictly differentiable at some point x 0 ∈ C whenever there exists a linear continuous mapping

d f (x 0 ) : R p → R n such that f (y) -f (x) = d f (x 0 ) • (y -x) + y -x g (x, y)
for almost every x, y ∈ C , where g (x, y) tends to 0 as soon as xx 0 + yx 0 a.e.

--→ 0. One may remark that the notion of strictly differentiability and of conic implicit function theorem depending on parameters was already introduced by [START_REF] Antoine | Etude locale de l'ensemble des points critiques d'un problème d'optimisation paramétré[END_REF]. Nevertheless, in our framework, we need to adapt these results considering dense subsets.

+ × C → R n : (ε, x) → F (ε,
x) be a continuous mapping, for which F (0, 0) = 0, satisfying the following:

• For almost every ε ∈ R k + , F is almost everywhere strictly differentiable with respect to x at 0, and, ∂F ∂x (ε, 0) is continuous with respect to ε on dense subsets.

• For almost every ε ∈ R k + , the remainder satisfies g ε (x, y) → 0 as (x, y)

a.e.

--→ 0, uniformly with respect to ε on dense subsets.

• There holds

∂F ∂x (0, 0) • Cone(C ) = R n .
There exist a real ε 0 > 0, an open neighborhood V of 0 in R n and a continuous function h

: [0, ε 0 ) k × V → C , such that F (ε, h(ε, y)) = y for every ε ∈ [0, ε 0 ) k and every y ∈ V .
Proof. We start the proof by recalling the following standard result (see, e.g. [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]). Let L : R p → R n be a linear mapping such that L(R p + ) = R n . The following hold true:

• There holds p > n + 1 and the intersection (0, +∞) p ∩ ker L is nontrivial.

• There is a subspace S ⊆ R p , dim(S) = n, so that L| S : S → R n is an isomorphism.

In our particular case, applying the previous result to the linear mapping L = ∂F ∂x (0, 0) yields the existence of a nontrivial vector v ∈ (0, +∞) p such that L(v) = 0, and, of a n-dimensional subspace S ⊆ R p such that L| S : S → R n is an isomorphism.

Proof of Theorem 6.1

For every ε ∈ R k + and every y, u ∈ R n , set Φ(ε, y, u) = u -F (ε, L| S -1 (u)) + y. This mapping is continuous and there holds Φ(0, 0, 0) = 0. Fix ε ∈ R k + at which F is almost everywhere strictly differentiable. Then, for every y ∈ R n , one has

Φ(ε, y, u 1 ) -Φ(ε, y, u 2 ) = Id - ∂F ∂x (ε, 0) • L| S -1 (u 1 -u 2 ) + u 1 -u 2 G ε (u 1 , u 2 ) (7.6)
where G ε (u 1 , u 2 ) = g ε (L| S -1 (u 2 ), L| S -1 (u 1 )) → 0 as soon as (u 1 , u 2 )

a.e.

--→ 0. From the continuity property of ∂F ∂x (ε, 0) on dense subsets, there exists ε 0 ∈ R k + and a dense subset E ⊆ [0, ε 0 ) k , such that for every ε ∈ E there holds

Id - ∂F ∂x (ε, 0) • L| S -1 * ≤ 1 4
and there exists r ε > 0 such that

G ε (u 1 , u 2 ) ≤ 1 4 for almost every u 1 , u 2 ∈ B r ε (0) .
On the other hand, by assumption, the remainder in expression (7.6) converge to 0 uniformly with respect to ε on dense subsets. Therefore, up to reduce E , gathering the previous results with (7.6), we infer the existence of r > 0 such that

Φ(ε, y, u 1 ) -Φ(ε, y, u 2 ) ≤ 1 2 u 1 -u 2 for almost every ε ∈ E , u 1 , u 2 ∈ B r (0) .
From this last result and the continuity of mapping F , for every ε ∈ [0, ε 0 ) k and y ∈ R n , the mapping u → Φ(ε, y, u) is 1 2 -Lipschitzian on an open neighborhood of 0 in R n . At this step, for every δ > 0, denote B δ = S ∩ B δ (0) and choose δ > 0 small enough such that v + B δ ⊆ (0, +∞) p . The set U δ = L(B δ ) is a closed neighborhood of 0 in R n . With the same argument as above, it is not difficult to show that, if the quantities δ, ε and y are small enough, then, the mapping u → Φ(ε, y, u) maps U δ into itself. Lemma 7.6 follows from the application of the usual Banach fixed point theorem to the contraction mapping u → Φ(ε, y, u) with parameters (ε, y).

Proof of Theorem 6.1

From now on, assume that assumptions (A) hold. We summon the other assumptions of Theorem 6.1 when they are needed. Moreover, in what follows, (x(•), u(•)) denotes the (unique) solution of (OCP) = (OCP) τ=0 and we assume that its related final time t f is a Lebesgue point of u(•) (if not, as pointed out in Section 7.1.2, we refer the reader to the approach proposed by [START_REF] Lee | Foundations of optimal control theory[END_REF][START_REF] Gamkrelidze | Principles of optimal control theory[END_REF]). Finally, without loss of generality, we consider free final time problems (otherwise, the proof is similar but simpler), introducing later Assumption (C 2 ) for problems (OCP) τ with control and state delays.

Existence of Optimal Controls for (OCP) τ

We focus first on the existence of an optimal control for (OCP) τ , for every τ ∈ (0, τ 0 ) 2 . No other assumptions but (A) and (C 1 ) are considered. In particular, mappings f and f 0 are affine in the two control variables. Thanks to this property, the existence can be achieved by using the arguments in [START_REF] Bonalli | Solving optimal control problems for delayed control-affine systems with quadratic cost by numerical continuation[END_REF]Theorem 2]. However, we prefer to develop the usual Filippov's scheme [START_REF] Filippov | On certain questions in the theory of optimal control[END_REF] (following [START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF]) to highlight the difficulty in applying this procedure to more general systems (in particular, see Remark 7.2). Even if problems (OCP) τ with control and state delays are considered, we assume to have free final time just to use the same approach for problem with pure state delays.

Fix τ = (τ 1 , τ 2 ) ∈ (0, τ 0 ) 2 and let

α = inf u∈U τ U C τ (t f (u), u) = t f (u) 0 f 0 (t , x(t ), x(t -τ 1 ), u(t ), u(t -τ 2 )) d t Consider now a minimizing sequence of trajectories x k (•) associated to u k (•), that is C τ (t f (u k ), u k ) → α when k → ∞ and define Fk (t ) = f (t , x k (t ), x k (t -τ 1 ), u k (t ), u k (t -τ 2 )) for almost every t ∈ [0, t f (u k )]. By Assumption (A 4 ), we can extend Fk (•) by zero on (t f (u k ), b] so that ( Fk (•)) k∈N is bounded in L ∞ ([0, b], R n+1 ). Therefore, up to some subsequence, ( Fk (•)) k∈N converges to some F (•) = (F (•), F 0 (•)) ∈ L ∞ ([0, b], R n+1
) for the weak star topology of L ∞ . On the other hand, up to some subsequence, the sequence (t f (u k )) k∈N converges to some t τ f ≥ 0. Therefore, for every t ∈ [-∆, t τ f ], define

x τ (t ) = φ 1 (t )1 [-∆,0) (t ) + 1 [0,t τ f ] (t ) φ 1 (0) + t 0 F (s) d s (7.7)
Clearly, x τ (•) is absolutely continuous and, up to continuous extensions, (x k (•)) k∈N converges pointwise to x τ (•) within [-∆, t τ f ]. Moreover, by assumptions (A 1 ), (A 4 ) and the Arzelà-Ascoli theorem, up to some subsequence,

x k (•) converges to x τ (•), uniformly in [-∆, t τ f ]. From the compactness of M f , there holds x τ (t τ f ) ∈ M f .
In the next paragraph, we show that x τ (•) comes from an admissible control in U τ

t τ f ,U . For almost every t ∈ [0, t f (u k )], set Hk (t ) = f (t , x τ (t ), x τ (t -τ 1 ), u k (t ), u k (t -τ 2 )) and, if t f (u k ) + τ 2 < t τ f , extend it by 0 on (t f (u k ), b].
At this step, we need to introduce several structures to deal with the presence of the control delay τ 2 . First, let

β = max | f 0 (t , x, y, u, v)| : 0 ≤ t ≤ b , (x, y) ≤ b , (u, v) ∈ U 2 > 0 and N ∈ N such that N τ 2 ≤ t τ f < (N +1)τ 2 .
By taking continuous extensions, we clearly see that x τ (•) is well-defined in [-∆, (N + 1)τ 2 ]. Therefore, we define

G(t , u 1 , . . . , u N +1 , γ 1 , . . . , γ N +1 ) =            f (t , x τ (t ), x τ (t -τ 1 ), u 1 , φ 2 (t -τ 2 )) f 0 (t , x τ (t ), x τ (t -τ 1 ), u 1 , φ 2 (t -τ 2 )) + γ 1 f (t + τ 2 , x τ (t + τ 2 ), x τ (t + τ 2 -τ 1 ), u 2 , u 1 ) f 0 (t + τ 2 , x τ (t + τ 2 ), x τ (t + τ 2 -τ 1 ), u 2 , u 1 ) + γ 2 . . . f (t + N τ 2 , x τ (t + N τ 2 ), x τ (t + N τ 2 -τ 1 ), u N +1 , u N ) f 0 (t + N τ 2 , x τ (t + N τ 2 ), x τ (t + N τ 2 -τ 1 ), u N +1 , u N ) + γ N +1            (7.8)
almost everywhere in [0, τ 2 ], and

Ṽβ (t ) = G(t , u 1 , . . . , u N +1 , γ 1 , . . . , γ N +1 ) : (u 1 , . . . , u N +1 ) ∈ U N +1 , ∀i = 1, . . . , N + 1 : γ i ≥ 0 , | f 0 (t , x τ (t ), x τ (t -τ 1 ), u 1 , φ 2 (t -τ 2 )) + γ 1 | ≤ β ∀i = 1, . . . , N : | f 0 (t + i τ 2 , x τ (t + i τ 2 ), x τ (t + i τ 2 -τ 1 ), u i +1 , u i ) + γ i +1 | ≤ β .
Thanks to Assumption (A 1 ), Ṽβ (t ) is compact for the standard topology of R (n+1)(N +1) . Moreover, it is not difficult to see that assumptions (A 1 ) and (C 1 ) ensure that Ṽβ (t ) is convex for the same topology (remark the importance of (C 1 )). We then introduce

Ṽ = G(•) ∈ L 2 ([0, τ 2 ], R (n+1)(N +1) ) : G(t ) ∈ Ṽβ (t ) , a.e. [0, τ 2 ]
.

One checks easily that Ṽ is convex and closed in L 2 ([0, τ 2 ], R (n+1)(N +1) ) for the strong topology of L 2 , and therefore, it is convex and closed in L 2 ([0, τ 2 ], R (n+1)(N +1) ) for the weak topology of L 2 . At this step, for every i = 0, . . . , N , denote

Gi+1 k (t ) = f (t + i τ 2 , x τ (t + i τ 2 ), x τ (t + i τ 2 -τ 1 ), u k (t + i τ 2 ), u k (t + (i -1)τ 2 ))
and Gk (t ) = ( G1 k (t ), . . . , GN+1 k (t )). Therefore Gk (•) ∈ Ṽ for every k ∈ N. Moreover, since ( Gk (•)) k∈N is bounded in L 2 ([0, τ 2 ], R (n+1)(N +1) ), up to some subsequence, it converges for the weak topology of L 2 to a function G(•) that necessarily belongs to Ṽ . Therefore, for almost every t ∈ [0, τ 2 ] and i = 1, . . . , N +1, there exist

u i τ (t ) ∈ U , γ i τ (t ) ≥ 0 such that G1 (t ) = f (t , x τ (t ), x τ (t -τ 1 ), u 1 τ (t ), φ 2 (t -τ 2 )) f 0 (t , x τ (t ), x τ (t -τ 1 ), u 1 τ (t ), φ 2 (t -τ 2 )) + γ 1 τ (t )
and, for every i = 1, . . . , N ,

Gi+1 (t ) = f (t + i τ 2 , x τ (t + i τ 2 ), x τ (t + i τ 2 -τ 1 ), u i +1 τ (t ), u i τ (t )) f 0 (t + i τ 2 , x τ (t + i τ 2 ), x τ (t + i τ 2 -τ 1 ), u i +1 τ (t ), u i τ (t )) + γ i +1 τ (t )
.

Proof of Theorem 6.1

Moreover, since U is compact, functions u i τ (•), γ i τ (•) can be chosen to be measurable on [0, τ 2 ] using a measurable selection lemma (see, e.g. [START_REF] Lee | Foundations of optimal control theory[END_REF]Lemma 3A,page 161]). At this step, we come back to the whole interval [-τ 2 , t τ f ]. For this, set

u τ (t ) = φ 2 (t ) t ∈ [-τ 2 , 0] , u i +1 τ (t -i τ 2 ) t ∈ [i τ 2 , (i + 1)τ 2 ] , i = 0, . . . , N γ τ (t ) = γ i +1 τ (t -i τ 2 ) t ∈ [i τ 2 , (i + 1)τ 2 ] , i = 0, . . . , N which are measurable functions in [-τ 2 , t τ f ],
and let

H (t ) = f (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 )) f 0 (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 )) + γ τ (t )
.

From the weak star convergence in L ∞ of ( Gk (•)) k∈N towards G(•), it follows immediately that ( Hk (•)) k∈N converges to H (•) for the weak topology of L 2 . Furthermore, from the differentiability of f w.r.t. (x, y), the compactness of U and the dominated covergence theorem, there holds

lim k→∞ t τ f 0 Fk (t ) -Hk (t ) • ϕ(t ) d t = 0 for every map ϕ(•) ∈ L 2 ([0, t τ f ], R n+1 ), from which H = F almost everywhere in [0, t τ f ]
. Combining (7.7) with all the previous results, we obtain

x τ (t ) = φ 1 (t )1 [-∆,0) (t ) + 1 [0,t τ f ] (t ) φ 1 (0) + t 0 f (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 )) d s
which proves that the measurable function u τ (•) is an admissible control for (OCP) τ .

It remains to show that control u τ (•) is optimal for (OCP) τ . For this, from what we showed above and by definition of weak star convergence, we have

C τ (t f (u k ), u k ) → t τ f 0 f 0 (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 )) + γ τ (t ) d t .
Since γ τ (•) takes only non-negative values, there holds

t τ f 0 f 0 (t , x τ (t ), x τ (t -τ 1 ), u τ (t ), u τ (t -τ 2 )) d t ≤ α ≤ C τ (t f (v), v) for every v(•) ∈ U τ U .
Therefore, γ τ (•) is necessarily zero and the conclusion follows. Now, we consider pure state delays, that is, problems (OCP) τ for which τ = (τ 1 , 0). It is clear that, if Assumption (B 2 ) holds, one can proceed with the same procedure as above (which is nothing else but the usual Filippov's scheme [START_REF] Filippov | On certain questions in the theory of optimal control[END_REF]) to achieve the existence of optimal controls. Of course, Guinn's reduction (7.8) is not needed.

Chapter 7. Continuity Properties of Pontryagin Extremals

Remark 7.2. Guinn's reduction (7.8) converts the dynamics with control delays into a new dynamics without control delays but with a larger number of variables. It is clear, from the context, that the natural assumption to provide the existence of optimal controls for generic nonlinear dynamics is the convexity of system (7.8) for every N ∈ N (since the delay varies), which is a very strong assumption. From this, we ascertain that the proof of Lemma 2.1 in [START_REF] Nababan | A filippov-type lemma for functions involving delays and its application to time-delayed optimal control problems[END_REF] does not work under the weaker assumption of convexity of the epigraph of the extended dynamics.

Convergence of Optimal Controls and Trajectories for (OCP) τ

We start by considering problems (OCP) τ with pure state delays, for which τ = (τ 1 , 0) ∈ (0, τ 0 ) × {0}, by assuming that Assumption (B 2 ) holds. In this case, the classical way to proceed consists in reproducing and adapting the convexity Filippov's argument used in the previous section concerning existence of optimal controls (see, e.g. [START_REF] Haberkorn | Convergence results for smooth regularizations of hybrid nonlinear optimal control problems[END_REF]).

Let (τ k ) k∈N = ((τ 1 k , 0)) k∈N ⊆ (0, τ 0 ) × {0} an arbitrary sequence converging to 0 as k tends to ∞ and let (x τ k (•), u τ k (•)) be an optimal solution of (OCP) τ k with final time t

τ k f (u τ k ). Since t τ k f (u τ k ) ∈ [0, b],
up to some subsequence, the sequence of final times (t

τ k f ) k∈N = (t τ k f (u τ k )) k∈N converges to some t f ∈ [0, b]. Since M f is compact, up to some subsequence, the sequence (x τ k (t τ k f )) k∈N ⊆ M f converges to some point in M f . For every integer k and almost every t ∈ [0, t τ k f ], set Gk (t ) = f1 (t , x τ k (t ), x τ k (t -τ 1 k ), u τ k (t )), ∂ f1 ∂x (t , x τ k (t ), x τ k (t -τ 1 k ), u τ k (t )), ∂ f1 ∂y (t , x τ k (t ), x τ k (t -τ 1 k ), u τ k (t )) .
Thanks to Assumption (A 4 ), we extend Gk (•) by zero on (t

τ k f , b].
Assumptions (A 1 ) and (A 4 ) imply that the sequence ( Gk (•)) k∈N is bounded in L ∞ , then, up to some subsequence, it converges to some G(

•) = (G(•),G 0 (•),G x (•),G y (•)) ∈ L ∞ ([0, b], R n+1
) for the weak star topology of L ∞ . Exploiting the weak star convergence of L ∞ (and using In the next paragraph, we prove that there exists a control ū(•) ∈ L ∞ ([0, t f ],U ) such that x(•) is an admissible trajectory of (OCP) associated with this control ū(•).

1 [ t f ,b] G as test function), we get that G(t ) = 0 for almost every t ∈ [ t f , b]. From this, for every t ∈ [0, t f ], denote x(t ) = φ 1 (t )1 [-∆,0) (t ) + 1 [0, t f ] (t ) φ 1 (0) + t 0 G(s) d s . ( 7

Proof of Theorem 6.1

Using the definition of β given in Section 7.3.2, for every t ∈ [0, t f ], consider the set

Zβ (t ) = f 1 (t , x(t ), x(t ), u), f 0 1 (t , x(t ), x(t ), u) + γ, ∂ f1 ∂x (t , x(t ), x(t ), u), ∂ f1 ∂y (t , x(t ), x(t ), u) : u ∈ U , γ ≥ 0 , | f 0 1 (t , x(t ), x(t ), u, v) + γ| ≤ β .
Thanks to Assumption (B 2 ), the set Zβ (t ) is compact and convex for the standard topology of R n+1 . Therefore, we have the following statements.

• From the convexity and the compactness of Zβ (t ), for every δ > 0 and t ∈ [0, t f ],

Z δ β (t ) = x ∈ R n+1 : d (x, Zβ (t )) ≤ δ , where d (x, A) = inf y∈A x -y
is convex and compact for the standard topology of R n+1 . This fact is evident.

• For every δ > 0, the set

Zδ = F (•) ∈ L 2 ([0, t f ], R n+1 ) : F (t ) ∈ Z δ β (t ) for almost every t ∈ [0, t f ]
results to be convex and closed in L 2 ([0, t f ], R n+1 ) for the strong topology of L 2 . Then, we have that it is closed in L 2 ([0, t f ], R n+1 ) for the weak topology of L 2 .

Convexity is obvious from the previous statement. Let ( Fk (•)) k∈N ∈ Zδ such that Fk (•)

L 2 -→ F (•). Then, F (•) ∈ L 2 ([0, t f ], R n+1
) and there exists a subsequence such that Fk m (•)

a.e --→ F (•). Since Z δ β (t ) is closed for the standard topology of R n+1 , a.e. in t ∈ [0, t f ], there holds F (t ) = lim m→∞ Fk m (t ) ∈ Z δ β (t ) and the statement follows.

• For every δ > 0, there exists k δ ∈ N such that, if k ≥ k δ , there holds Gk (•) ∈ Zδ .

Indeed, thanks to assumptions (A 1 ), (A 4 ), mappings f 1 , f 0 1 are globally Lipschitz within [0, t f ] × B 2n b (0) ×U and, by using the mean value theorem, for almost every t ∈ [0, t f ], we obtain

inf z∈ Zβ (t ) Gk (t ) -z ≤ C x τ k (t ) -x(t ) + x τ k (t -τ 1 k ) -x(t )
where C > 0 is a suitable constant, which is independent from t . The conclusion follows from the uniform convergence of (x τ k (•)) k∈N towards x(•).

Using the closeness of Zδ with respect to the weak topology of L 2 , we infer that G(•) ∈ Zδ for every δ > 0. It is not difficult to see that this implies G(•) ∈ ∩ j ∈N Z1/j ⊆ Z0 .

Chapter 7. Continuity Properties of Pontryagin Extremals

We have obtained that, a.e. in t ∈ [0, t f ], there exist ū(t ) ∈ U and γ(t ) ≥ 0 such that G(t ) = f 1 (t , x(t ), x(t ), ū(t )) , f 0 1 (t , x(t ), x(t ), ū(t )) + γ(t ) , (7.10) In order to conclude, it remains to show that t f = t f , ū(•) = u(•) and then x(•) = x(•).

∂
First, the previous argument shows that

C τ k (t τ k f , u τ k ) → C 0 ( t f , ū) + t f 0 γ(t ) d t .
Thanks to the construction of the mapping Γ provided in Section 7.3.1, for every integer k, there exists a sequence (t k f , v k (•), y k (•)), respectively of final times, of admissible controls and of trajectories for (OCP) τ k , which converges to (t f , u(•), x(•)) (for the evident topologies) as k tends to ∞. Thanks to the optimality of each u τ k (•), there holds C τ k (t

τ k f , u τ k ) ≤ C τ k (t k f , v k
) and, since γ(t ) ≥ 0, passing to the limit gives C 0 ( t f , ū) ≤ C 0 (t f , u). From Assumption (A 2 ) we infer t f = t f , ū(•) = u(•) and x(•) = x(•). Remark that, from the previous argument, the following weak convergences hold .11) Let now consider problems (OCP) τ with control and state delays τ = (τ 1 , τ 2 ) ∈ (0, τ 0 ) 2 under Assumption (C 1 ). Thanks to the affine behavior of the considered mappings, the previous argument simplifies considerably, because we can transpose the weak convergence directly on controls. Adapting these proofs to more general systems turns out to be very challenging (see remarks in Section 6.1). We adopt free final time to show that, for this step, no problems arise if Assumption (C 2 ) does not hold.

         ∂ f1 ∂x (•, x τ k (•), x τ k (• -τ 1 k ), u τ k (•)) (L ∞ ) * ∂ f1 ∂x (•, x(•), x(•), u(•)) ∂ f1 ∂y (•, x τ k (•), x τ k (• -τ 1 k ), u τ k (•)) (L ∞ ) * ∂ f1 ∂y (•, x(•), x(•), u(•)) . ( 7

Proof of Theorem 6.1

Let (τ k ) k∈N = ((τ 1 k , τ 2 k )) k∈N ∈ (0, τ 0 ) 2 an arbitrary sequence of delays converging to 0 as k tends to ∞ and let (x τ k (•), u τ k (•)) be an optimal solution of (OCP) τ k with final time t On the other hand, thanks to Assumption (A 1 ), the sequence (u τ k (•)) k∈N is bounded in L 2 ([-∆, t f ], R m ). Therefore, up to some subsequence, (u τ k (•)) k∈N converges to some ū(•) ∈ L 2 ([-∆, t f ], R m ) for the weak topology of L 2 . More precisely, there holds ū(•) ∈ L ∞ ([-∆, t f ],U ). Indeed, (u τ k (•)) k∈N ⊆ L 2 ([-∆, t f ],U ) and, thanks to Assumption (A 1 ), the set L 2 ([-∆, t f ],U ) is closed and convex for the strong topology of L 2 . Therefore, it is closed and convex for the weak topology of L 2 , from which there holds ū(•) ∈ L 2 ([-∆, t f ],U ) ⊆ L ∞ ([-∆, t f ],U ) (the last inclusion still follows from (A 1 )). At this step, one crucial result is represented by the weak convergence in L 2 of the sequence (u τ k (•-τ 2 k )) k∈N towards control ū(•). To see this, consider the shift operator

S τ 2 : L 2 (R, R m ) → L 2 (R, R m ) : t → φ(t ) → t → φ(t -τ 2 ) .
Using the dominated convergence theorem, it is clear that, for every φ(•) ∈ L 2 (R, R m ), there holds S τ 2 φ-φ L 2 → 0 as soon as τ 2 → 0. At this point, extend u τ k (•), u τ k (•-τ 2 k ) and ū(•) by zero out [-∆, t f ]. Therefore, for every map ϕ(•) ∈ L 2 (R, R m ), one obtains which converges to 0, providing the weak convergence in L 2 of (u τ k (•-τ 2 k )) k∈N to ū(•).

We can now show that, under Assumption (C 1 ), the trajectory arising from control ū(•) is admissible for problem (OCP) τ=0 = (OCP), proceeding as follows. First, remark that, up to continuous extensions, for every k, there holds

x τ k (t ) = φ 1 (t )1 [-∆,0) (t ) + 1 [0, t f ] (t ) φ 1 (0) + t 0 f (s, x τ k (s), x τ k (s -τ 1 k ), u τ k (s), u τ k (s -τ 2 k )) d s . (7.13)
From this, assumptions (A 1 ), (A 4 ) ensure that (x τ k (•)) k∈N is bounded in H 1 , and then, it converges to some x(•) ∈ H 1 ([-∆, t f ], R n ) for the weak topology of H 1 . Since the immersion of H 1 into C 0 is compact, up to some subsequence, (x τ k (•)) k∈N converges to x(•) ∈ C 0 ([-∆, t f ], R n ) uniformly in [-∆, t f ], and, passing to the limit in (7. Proof of Lemma 7.7. We start by proving the first assertion. For this, denote h τ (t ) = (h τ 1 (t ), . . . , h τ n+1 (t )) = f1 (t , x τ (t ), x τ (t -τ 1 ), u τ (t )) h(t ) = (h 1 (t ), . . . , h n+1 (t )) = f1 (t , x(t ), x(t ), u(t )) .

Let us prove that, for every s ∈ (0, t f ) Lebesgue point of function h(•), for every β > 0, α s > 0 (such that s +α s < t f ), there exists γ s,α s ,β > 0 such that, for every τ 1 ∈ (0, γ s,α s ,β ), there exists s τ ∈ [s, s + α s ] Lebesgue point of h τ (•) for which h τ (s τ )h(s) < β. By contradiction, suppose that there exists s ∈ (0, t f ) Lebesgue point of h(•), α s > 0, β > 0 such that, for every integer k, there exists τ k ∈ (0, 1/k) × {0} and i k ∈ {1, . . . , n + 1} for which, for t ∈ [s, s + α s ] Lebesgue point of h τ k (•), there holds |h

τ k i k (t ) -h i k (s)| ≥ β.
From the previous results, the family (h τ (•)) τ∈(0,τ 0 )×{0} converges to h(•) in L ∞ for the weak star topology. Therefore, for every 0 < δ ≤ 1, there exists an integer k δ such that, for every k ≥ k δ , there holds )-h i (s)| < β, contradiction. Now, we consider the second statement. The case for which Assumption (C 3 ) ensures that, for every delay τ, every optimal control u τ (•) of (OCP) τ is continuous, is treated as above because of the weak convergence in L 2 of u τ (•), of u τ (• -τ 2 ) and of u τ (• + τ 2 ). Therefore, suppose that control u( and fix s ∈ (0, t f ), Lebesgue point of h(•). By contradiction, suppose that there exist β > 0 and α > 0 such that, for every integer k, there exist τ k = (τ 1 k , τ 2 k ) ∈ (0, 1/k) 2 and i k ∈ {1, . . . , 3m} for which, for every r ∈ [s, s + α] Lebesgue point of h τ k (•), there holds |h

τ k i k (r ) -h i k (s)| ≥ β.
From the arguments of the previous sections, up to some extension, the family of controls (u τ (•)) τ∈(0,τ 0 ) 2 converges to u(•) almost everywhere in [0, t f ] and the same holds true for (u τ (• -τ 2 )) τ∈(0,τ 0 ) 2 and (u τ (• + τ 2 )) τ∈(0,τ 0 ) 2 , thanks to Remark 7.3. Then, (h i τ k (•)) k∈N converges a.e. to h i (•), raising a contradiction.

Lemma 7.7 allows to prove the following convergence property for Pontryagin cones.

Lemma 7.8. For every ṽ ∈ K 0 (t f ) and every τ = (τ 1 , τ 2 ) ∈ (0, τ 0 ) 2 (as well as τ = (τ 1 , 0) ∈ (0, τ 0 )×{0} in the case of pure state delays), there exists wτ ∈ K τ (t τ f ) such that the family ( wτ ) τ∈(0,τ 0 ) 2 converges to ṽ as τ tends to 0. Proof of Lemma 7.8. We prove the statement for problems (OCP) τ with general state and control delays τ = (τ 1 , τ 2 ). If pure state delays τ = (τ 1 , 0) are considered, the same guideline can be straightforwardly employed by exploiting Lemma 7.7 and (7.11). Suppose first that ṽ = ṽ0 s,ω z (s) (t f ), where z ∈ U and 0 < s < t f is a Lebesgue point of u(•) (recall Remark 7.1). By definition, ṽ0 Moreover, from the results of the previous sections, there holds in particular

∂ f ∂x (•, x τ (•), x τ (• -τ 1 ), u τ (•), u τ (• -τ 2 )) L 2 ∂ f ∂x (•, x(•), x(•), u(•), u(•)) ∂ f ∂y (•, x τ (•), x τ (• -τ 1 ), u τ (•), u τ (• -τ 2 )) L 2 ∂ f ∂y (•, x(•), x(•), u(•), u(•)) .
By the continuous dependence w.r.t initial data for dynamical systems and since t τ f converges to t f , the family ( wτ ) τ∈(0,τ 0 ) 2 = ( wτ s τ ,z (t τ f )) τ∈(0,τ 0 ) 2 converges to ṽ as τ → 0. Take now ṽ ∈ Int K 0 (t f ). In this case, the conclusion follows straightforwardly by combining the previous case with the definition of smallest convex cone of a set. Lastly, in the case of generic ṽ ∈ K 0 (t f ), there exists a sequence ( ṽn ) n∈N ⊆ Int K 0 (t f ) converging to ṽ. Therefore, from above, for every n ∈ N, there exist vectors ṽn τ ∈ Int K τ (t τ f ) such that ṽn τ -→ τ→0 ṽn . Moreover, by removing elements in ( ṽn τ ) τ∈(0,τ 0 ) 2 if necessary, we can assume that ṽn τ -ṽn ≤ 1 for n ∈ N, τ ∈ (0, τ 0 ) 2 . Consider now, any subsequence ( ṽn k τ k ) k∈N . Since ( ṽn ) n∈N is bounded, it follows that also ( ṽn k τ k ) k∈N is bounded, and then, it has a subsequence converging necessarily to ṽ. Therefore, the whole family ( ṽn τ ) n∈N,τ∈(0,τ 0 ) 2 converges to ṽ. We finally have that the family of vectors ṽτ = lim n→∞ ṽn τ ∈ K τ (t τ f ), defined up to some subsequence, converges to ṽ.

For the last part of the proof, an iterative use of Lemma 7.8 is done. It is at this step that, for problems with general delays τ = (τ 1 , τ 2 ), Assumption (C 2 ) of fixed final time becomes fundamental to correctly derive the convergence related to adjoint vectors. Indeed, problems arise when one tries to make the final condition on the Hamiltonian (6.7) converge to the transversality condition related to problem (OCP) τ . For sake of concision, in this context, we focus only on problems (OCP) τ with general delays τ = (τ 1 , τ 2 ). The case concerning pure state delays is similar (we refer the reader to [START_REF] Haberkorn | Convergence results for smooth regularizations of hybrid nonlinear optimal control problems[END_REF]Proposition 2.15] for details). Assumptions (B ) and (C ) are implicitly used. We first prove that the following statements hold true:

Chapter 7. Continuity Properties of Pontryagin Extremals

• For every τ = (τ 1 , τ 2 ) ∈ (0, τ 0 ) 2 , every extremal lift (x τ (•), p τ (•), p 0 τ , u τ (•)) of any solution of (OCP) τ is normal.

• The set of final adjoint vectors {p τ (t f ) : τ ∈ (0, τ 0 ) 2 } is bounded.

We consider the first statement proceeding by contradiction. Assume that, for every integer k, there exist τ k = (τ 1 k , τ 2 k ) ∈ (0, 1/k) 2 and a solution (x τ k (•), u τ k (•)) of (OCP) τ k having an abnormal extremal lift (x τ k (•), p τ k (•), 0, u τ k (•)). Set ψ τ k = p τ k (t f ) p τ k (t f ) for every integer k. Therefore, there holds (ψ τ k , 0), ṽτ k ≤ 0, for every ṽτ k ∈ K τ k (t f ) and every integer k. Up to a subsequence, the sequence (ψ τ k ) k∈N ⊆ S n-1 converges to some unit vector ψ ∈ R n . Passing to the limit, by using the previous results, we infer that (ψ, 0), ṽ ≤ 0 for every ṽ ∈ K 0 (t f ). Thanks to Assumption (C 2 ), it then follows that (x(•), u(•)) has an abnormal extremal lift. This contradicts Assumption (A 3 ). For the second statement, again by contradiction, assume that there exists a sequence (τ k = (τ 1 k , τ 2 k )) k∈N ⊆ (0, τ 0 ) 2 converging to 0 such that p τ k (t f ) tends to +∞. Since the sequence p τ k (t f ) p τ k (t f ) k∈N belongs to S n-1 , up to some subsequence, it converges to some unit vector ψ. On the other hand, by construction, the following inequality (p τ k (t f ), -1), ṽτ k ≤ 0 holds for every ṽτ k ∈ K τ k (t f ) and every integer k. Dividing by p τ k (t f ) and passing to the limit, it follows that the solution (x(•), u(•)) has an abnormal extremal lift, which again contradicts Assumption (A 3 ). Now, let ψ be a closure point of {p τ (t f ) : τ ∈ (0, τ 0 ) 2 } and (τ k = (τ 1 k , τ 2 k )) k∈N ⊆ (0, τ 0 ) 2 a sequence converging to 0 such that p τ k (t f ) tends to ψ. Using the continuous dependence w.r.t. initial data and the established convergence properties, we infer that the sequence (p τ k (•)) k∈N converges uniformly to the solution z(

z(t f ) = ψ
Moreover, since (p τ k (t f ), -1), ṽτ k ≤ 0, for every ṽτ k ∈ K τ k (t f ) and every integer k, passing to the limit, thanks to the previous results, we obtain (ψ, -1), ṽ ≤ 0, for every ṽ ∈ K 0 (t f ). By definition, it follows that (x(•), z(•), -1, u(•)) is a normal extremal lift of (OCP). Using Assumption (A 3 ), we finally obtain z(•) = p(•) in [0, t f ].

This last result provides the conclusion of the proof of Theorem 6.1.

Conclusions

Conclusions

In this chapter, we have provided a complete proof of continuity properties of extremals related to optimal control problems with control and state delays (OCP) τ .

The proof develops in three main parts. The controllability of problems (OCP) τ , for every positive τ sufficiently small, is inferred first. For this, we adapt the conclusions of implicit function theorems when parameters and restriction to dense subsets are considered. After that, the existence of solutions of (OCP) τ , for τ sufficiently small, and their convergences, as τ tends to 0, towards solutions of (OCP) is recovered. For problems (OCP) τ with control and state delays, considering dynamics and costs affine with respect to the two control variables is crucial to derive these convergences properties. As last step, we prove the convergence of the adjoint vectors related to (OCP) τ towards the adjoint vector of (OCP), as τ tends to 0. This part is the most difficult and requires to show that, in a generic sense, the Pontryagin cones of problems (OCP) τ converge uniformly to the Pontryagin cone of (OCP).

The continuity properties concerning the state and the adjoint vector are the most crucial ones, because, they allow to ensure the convergence of any homotopy procedure on τ to solve (OCP) τ by indirect methods, as pointed out in Chapter 6. The proof of these important properties requires a fine use of needle-like variations for problems with state and control delays, that, since it does not appear explicitly in the literature, we propose and analyze before proving the main convergence result.

Conclusion Main Contributions

In this work, we have addressed the real-time optimal guidance of launch vehicles with the objective of designing an autonomous algorithm for the prediction of optimal control strategies, based on indirect methods, able to adapt itself to unpredicted changes of the original scenario. To this aim, we first provided an accurate geometric analysis to recover a well-posed framework and correctly apply indirect methods. We proposed a practical numerical integration of the problem by efficiently combining indirect methods with homotopy procedures, increasing robustness and computational speed. Moreover, we improved dynamical models by considering delays. More specifically, we introduced a rigorous and well-posed homotopy framework to recover solutions for optimal control problems with delays via indirect methods. All our contributions made possible the development of a fully automatic, independent and self-regulating software, today property of ONERA-The French Aerospace Lab, for general realistic endo-atmospheric launch vehicle applications focused, in particular, on optimal missile interception scenarios.

Perspectives

Hereafter, we list some possible developments for future research works.

• Considering additional structural constraints and control variables.

Concerning the dynamical model introduced in Chapter 2, to further generalize the previous treatise, one should consider additional structural state constraints. The most important one is certainly represented by the load factor (see Section 2.1.2). This constraint is imposed to bound the magnitude of the velocity and the values of normal aerodynamical coefficients within some safe range, preventing strong structural strains which could jeopardize the mission.

Conclusion

As pointed out in Section 2.1.2, for interception frameworks, this bound is negligible. On the other hand, for more general launch vehicle applications, this constraint becomes fundamental to avoid the loss of the aircraft. Integrating the load factor into our formulation and managing it by indirect methods raise new difficulties: one further mixed control-state constraint appears. However, in this case, our trick related to local transformations of coordinates provided in Chapter 3 and the homotopy scheme developed in chapters 4 and 5 cannot be exploited directly and need to be carefully revisited. This represents a limit for our approach and our guidance software. On the other hand, recent works, such as [START_REF] Zhu | Contrôle optimal de l'attitude d'un lanceur[END_REF][START_REF] Zhu | Geometric optimal control and applications to aerospace[END_REF], efficiently solve complex guidance problems in the presence of load factor constraints, by homotopy techniques, showing that similar extensions to our context may certainly be performed.

As a further development, we could consider additional controls. More specifically, in some guidance problems, it is important to control the magnitude of the thrust (see, e.g. [START_REF] Zhu | Minimum time control of the rocket attitude reorientation associated with orbit dynamics[END_REF][START_REF] Bonnans | Singular arcs in the generalized goddard's problem[END_REF]). To increase the number of feasible missions, the control of the thrust could be added to our framework and, in this case, new analysis of adjoint equations and of regular/nonregular controls are needed.

• Relaxing approximating assumptions for regular and nonregular controls.

In Chapter 3, we recovered the behavior of both regular and nonregular controls as functions of the state and the adjoint vector. This fundamental step allowed to correctly define indirect method strategies for our guidance problem. However, these computations made use of some first-order development approximations or of assumptions on the existence of particular lower bounds for the magnitude of the velocity. To generalize this context to other guidance problems, which are different from interception missions, it would be interesting to study regular and nonregular controls without considering any of the previous assumptions. Nevertheless, in this case, many further nonvanishing Lie brackets appear, making the computations harder to manage analytically.

• Improving the construction and the robustness of initialization grids.

In Chapter 5, the use of initialization grids is proposed, in the framework of optimal interception problems, to improve numerical robustness and increase computational speed. Nevertheless, two aspects certainly need improvements. First progresses should concern the generation of the grids. Indeed, a simple algorithm for their construction is provided in Chapter 5, which, however, represents a first rough attempt. Since the grids are supposed to be generated offline, we propose the use of global techniques, such as direct methods (see Section 1.3.3) or numerical resolutions of high-dimensional Hamilton-Jacobi-Bellman equations (see, e.g. [START_REF] Carlini | An efficient algorithm for hamiltonjacobi equations in high dimension[END_REF][START_REF] Bardi | Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations[END_REF]), to implement more efficient grids.

Conclusion

On the other hand, more refined results concerning the robustness of these initialization grids are necessary. Indeed, the convergence of the spatial homotopy on the scenario, initialized with these grids, has been proved only statistically, by Monte Carlo simulations. We suggest to treat more systematically the convergence of the concerned homotopies, either via more satisfactory statistical tests or the development of rigorous theoretical results.

• Extending the continuity properties of extremals to more general cases.

In chapters 6 and 7, we provided continuity properties, with respect to delays (for appropriate topologies), of extremals related to optimal control problems with control and state delays, and subject to pure control constraints. It would be interesting to extend these properties to problems with more general constraints, such as, control and state constraints (see Section 1.4). This would require to analyze the proof of the Maximum Principle with state and control constraints via sliding or v-variations (see, e.g. [START_REF] Dmitruk | On the development of Pontryagin's maximum principle in the works of A.Y. Dubovitskii and A.A. Milyutin[END_REF]), to exploit the continuous dependence with respect to parameters of implicit function theorems (Ekeland-type approaches prone to fail because this continuous dependence does not arise). Furthermore, in the case of both control and state delays, the proof that we provided needs to consider affine dynamics/costs and fixed final time. The extension to more general systems is an interesting challenge (problems related to this extensions are developed in Section 6.1).

• Considering nonconstant delays or unknown delays.

The approach developed in Chapter 6, concerning the resolution of optimal control problems with delays via indirect methods and homotopy on the delays, works in the presence of constant delays. However, we suspect that this procedure can be extended to problems with nonconstant or unknown delays, at least from a numerical point of view. Indeed, in the case of delays depending on the time and the state, Maximum Principle formulations still exist (see, e.g. [START_REF] Asher | Optimal control of systems with state-dependent time delay[END_REF]), so that, the proposed numerical continuation scheme remains welldefined. On the other hand, when delays are unknown, our homotopy approach could be coupled with estimation techniques on delays (see, e.g. [START_REF] Krstic | Delay compensation for nonlinear, adaptive, and PDE systems[END_REF]). The estimation and homotopy steps could be iterated successively in a loop until converge is obtained. [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF][START_REF] Gamkrelidze | Principles of optimal control theory[END_REF]). The main steps are sketched as follows.

A.1. Version of Theorem 1.3 Valid in Euclidean Spaces

• First, the dimension of the problem is modified by introducing, for a given integer k, fixed controls u 1 (•), . . . , u k (•), where u 1 (•) coincides with the optimal control u(•), and considering the following relaxed control system

ẋ(t ) = k j =1
a j (t ) f (t , x(t ), u j (t )) , c m e (t , x(t ), u j (t )) = 0 , c m i (t , x(t ), u j (t )) ≤ 0 for every j = 1, . . . , k, where, the controls are a j (•) ∈ L ∞ ([0, t f ], R), subject to a j (t ) ≥ 0 ,

k j =1
a j (t ) = 1 , a.e. [0, t f ] .

• The main step consists in proving that the tuple x(•), (u 1 (•), . . . , u k (•)), (1, 0, . . . , 0) is a stationary point of the relaxed version of (OCP) m,s , in which, the original control system is substituted by the previous relaxed dynamics and constraints, while, the control variable u is substituted by a 1 , . . . , a k (see [START_REF] Dmitruk | Maximum principle for the general optimal control problem with phase and regular mixed constraints[END_REF]). For this, Assumption 1.1, on the regularity of mixed constraints, is fundamental.

• The stationarity property allows to apply the classical Lagrange multiplier rule (see, e.g. [START_REF] Bonnans | Second-order analysis for optimal control problems with pure state constraints and mixed control-state constraints[END_REF]) on the previous convexified problem. The nontriviality condition, adjoint equations, the stationarity condition, slackness conditions and transversality conditions are immediately recovered. Working on the stationarity condition (which is given by the partial derivative of the Hamiltonian with respect to each a j ), the following weak maximality condition is obtained, almost everywhere in [0, t f ], for every j = 1, . . . , k (see [START_REF] Dmitruk | Maximum principle for the general optimal control problem with phase and regular mixed constraints[END_REF]) h(t , x(t ), p(t ), p 0 , u(t )) ≥ h(t , x(t ), p(t ), p 0 , u j (t )) . (A.8)

• The previous analysis shows that, for every tuple ũ(•) = (u 1 (•), . . . , u k (•)) of admissible controls of (OCP) m,s , there exists, at least, one tuple of Lagrange multipliers, of norm one (up to normalization), satisfying Theorem A.1 but the maximality condition, which must be substituted by (A.8). Denote by Λ( ũ(•)) the set of such Lagrange multipliers. It turns out that, thanks to the Banach-Alaoglu theorem, every Λ( ũ(•)) is weak star compact. Moreover, by introducing the canonical orders both on the set of controls ũ(•) and of multipliers Λ( ũ(•)), one obtains that the family {Λ( ũ(•))} ũ(•) is a centered system of nonempty compacta. Therefore, its intersection is nonempty and provides the sough Lagrange multiplier related to Theorem A.1, satisfying the strong maximality condition.

In the following, we make use of Theorem A.1 to provide a proof of Theorem 1.3.

represents the sought adjoint vector satisfying the conditions of Theorem 1.3.

Before continuing, we first derive some useful relations. For a given s ∈ [0, t f ], let λ s ∈ T * R N | x(s) and consider a local chart (V, ϕ) of x(t ) in R N such that ϕ(V ∩ M ) = ϕ(V )∩R n ×{0} N -n . Following exactly the same computations to prove Lemma 1.1 and the properties of the flow of field F , it is straightforward to verify that, with respect to the local coordinates of chart (V, ϕ), the following relation holds for every k = 1, . ∂x l (q(s), u(s))

∂ ∂x l q(s) = ∂c j m ∂q (q(s), u(s))

and similarly for constraints c k s . The adjoint equations of Theorem 1.3 then follow.

Recovering the nontriviality condition, the maximality condition, the stationarity condition and transversality conditions related to Theorem 1.3 follows trivially by the previous results. Finally, extending this context to problem with particular final conditions and free final time can be easily done as in the classical framework (see, e.g. [START_REF] Agrachev | Control theory from the geometric viewpoint[END_REF]). Therefore, the proof of Theorem 1.3 is concluded.

Figure 1 . 1 :

 11 Figure 1.1: Different zero paths.

2 .

 2 The North-East-Down (NED) frame (e L , e , e r ) that is parallel to the Earth surface and whose origin is at the center of gravity of the vehicle, denoted by G. In particular, this frame rotates with the vehicle and coincides with the orthonormal frame defined univocally as follows: -e r is the local vertical direction, (e L , e ) is the local horizontal plane and e L is pointing to the North.

Figure 2 . 1 :

 21 Figure 2.1: Graphical relations between frame (I, J , K) and frame (e L , e , e r ).

Figure 2 . 2 :

 22 Figure 2.2: Orientations of frame (e L , e , e r ) and of the principal body axis b.

Figure 2 . 3 :

 23 Figure 2.3: Air density and atmospheric pressure as functions of h (m): US-76 model.

Figure 2 . 4 :

 24 Figure 2.4: Graphical relations between the angle of attack α and the velocity v.

  Then, by coupling conditions (1.23)-(1.28) with (3.2), (3.3) and (3.4) as done in the proof of Theorem 3.1, we easily obtain that the local control w

1 Figure 3 . 1 :

 131 Figure 3.1: Graphical relations between frame (e L , e , e r ) and frame (i 1 , j 1 , k 1 ).

Figure 3 . 2 :

 32 Figure 3.2: Graphical relations between frame (e L , e , e r ) and frame (i 1 , j 1 , k 1 ).

  )

2 = sin α max p γ p 2 γ + p 2 χ cos 2 γ, w 3 = sin α max p χ cos γ p 2 γ + p 2 χ cos 2 γ, w 1 =

 2223221 therefore, from the Cauchy-Schwarz inequality, we obtain w cos α max .(3.27)

  .[START_REF] Hanson | A plan for advanced guidance and control technology for 2nd generation reusable launch vehicles[END_REF]) are satisfied with the largest possible value of C ∈ R. Three circumstances occur.

Figure 3 . 3 :

 33 Figure 3.3: Tangent point between the parabola and the ellipse of (3.28).

Assumption 3 . 3 .

 33 Let (r(•), v(•),u(•)) be an optimal solution of (GOGP) with final time t f . If the related extremal is nonregular along a subset E ⊆ [0, t f ] of non-zero Lebesgue measure, then, the optimal trajectory (r(•), v(•)) satisfies, along E ,

Lemma 3 . 6 .

 36 Under Assumption 3.3, p, [Y 3 , [X , Y 3 ]] = 0 almost everywhere in E . Proof. We implicitly refer to Proposition 3.1 and Lemma 3.5. By contradiction, suppose that p, [Y 3 , [X , Y 3 ]] = 0 a.e. in E . This implies that cos χp a L + sin χ cos L p a = 0 a.e. within E . The previous expression, combined with the adjoint equation of p χ , gives p a L | E = p a | E = 0. On the other hand, from the adjoint equation of p γ , we have (v p a rg p a v )| E = 0. Combining this expression with its derivative w.r.t. time within E and imposing p a v | E = 0 lead to

. 43 )

 43 Instead, in the case p, [Y 2 , [X , Y 2 ]] = 0 almost everywhere in E , suppose that there holds p, [Y 2 , [Y 2 , [X , Y 2 ]]] | E = 0. By differentiating with respect to the time the expression p, [Y 2 , [X , Y 2 ]] = 0, as in Lemma 3.4, we have (see also Lemma 3.5)

Proposition 3 . 3 .

 33 Under Assumption 3.3, nonregular optimal controls u of (GOGP) are uniquely well-defined and their local behaviors w and z are given, as function of the state and the adjoint vector, by the procedures provided in expressions (3.40)-(3.45).

Lemma 4 . 1 .

 41 Under the local coordinates provided by charts (V a , ϕ a ) and (V b , ϕ b ), the contribution of the rotation of the NED takes respectively the following forms

Figure 4 . 1 :From

 41 Figure 4.1: Homotopy scheme for (OIP). Continuations are done by Algorithm 1.

Figure 4 . 2 : 2 2 + w 2 3 ≤

 4223 Figure 4.2: Trajectories and constraints of the guidance law (4.22)-(4.23) and (OIP) 0 .

Figure 4 .

 4 3 resumes the results obtained.

Figure 4 . 3 :

 43 Figure 4.3: Monte Carlo simulations for the guidance law (4.22)-(4.23).

Figure 4 . 4 :

 44 Figure 4.4: Trajectories and constraints of (4.22)-(4.23), (OIP) 0 and (OIP) Inter .

Figure 4 . 5 :

 45 Figure 4.5: Trajectories and constraints of the first scenario, no minimal time.

Figure 4 . 6 :

 46 Figure 4.6: Trajectories and constraints of the second scenario, no minimal time. The black boxes within the subplot of Figure a) show changes of local chart.

FromℓFigure 4 . 7 :LFigure 4 . 8 :

 4748 Figure 4.7: Trajectories and constraints of the first scenario, with minimal time.

Figure 4 . 6 and

 46 Figure 4.8 show instead that the second scenario presents similar solutions for both cases, with or without minimal final time (compare the blue solid lines of Figure 4.6 b) and of Figure 4.8 b)).

Figure 5 . 1 :

 51 Figure 5.1: Representation of the explored scenarios of an initialization grid.

Figure 5 . 2 :

 52 Figure 5.2: Average percentage of failed scenarios and average computational time for one scenario.

6 Figure 5 . 3 :

 653 Figure 5.3: Average percentage of changes of chart and simulation/failures.

Figure 5 . 4 :

 54 Figure 5.4: UML representation of the C++ library for optimal interception strategies.

Chapter 5 .

 5 Numerical Robustness and Interception Software (ONERA) and to provide numerical parameters concerning the vehicle, with which initializing an object of type RealTimeShooting const Real pi = M_PI ; / / Vehicle parameters ( see Section 2 . 2 . 2 ) const Real eta = 0 . 4 4 2 ; const Real r r = 7500; const Real c0 = 0.00075; const Real d0 = 0.00005; const Real mu0 = 3.986 e14 ; const Real RT = 6378145; const Real ve = 1500; const Real q0 = 10 ; const Real tSw = 20 ; const Real alpha_m = pi / 6 . 0 ; const Real massInit = 400; const Real fixedT = tSw ;

Data:

  Optimal extremal (x(•), p(•), -1, u(•)) of the problem without delays (OCP). Result: Optimal extremal (x τ (•), p τ (•), -1, u τ (•)) of the problem with delays (OCP) τ . begin Set k = 0, τ k = 0, ∆ τ = τ and fix an integer k max . Solve (OCP) by indirect methods and denote (x τ k (•), p τ k (•), -1, u τ k (•)) its numerical extremal solution. while τ k < τ and k < k max do A.1) Compute τ k+1 = τ k + ∆ τ . A.2) Solve (OCP) τ k+1 by indirect methods initialized by p τ k (•), where the usual internal integration of adjoint equations is substituted by the following: Modified Internal Integration of Adjoint Equations to Compute p τ k+1 (•):

Figure 6 . 1 :

 61 Figure 6.1: Optimal quantities of (OCP) (CSTR) τ

cos α cos β cos α sin β cos γ sin α sin β - sin α cos β cos

  )), the guess concerning p γ and p χ are found by coupling the previous adjoint equations and solving the system e v s c m e v s c m η cos α sin αṗα ṗβ .

-r cos γ sin γ cos χ sin γ sin χ cos L 0 cos γ sin χ - cos γ cos χ cos L r sin γ cos γ cos χ cos γ sin χ

  ) we obtain the evolution of the time derivatives related to p γ and p χ . Therefore, coupling the following adjoint equations (related to p γ and p χ ) ṗγ = p L e v s r sin γ cos χ + p e v s r sin γ sin χ cos L p r e v s cos γ ṗχ = p L e v s r cos γ sin χp e v s r cos γ cos χ cos L with the expression of the Hamiltonian, we deduce the guess of p r , p L and p by solving the following linear system d + ηc m sin 2 α e v sp γ e v s c m sin α cos βp χ e v s c m cos γ sin α sin β -C u (u 2 α + u 2 β )

Figure 6 . 2 :

 62 Figure 6.2: Homotopy scheme for (OIP) a τ . Continuations are done by Algorithm 1.

Figure 6 . 3 :Figure 6 . 4 :Figure 6 . 5 : 4 Figure 6 . 6 :

 636465466 Figure 6.3: Three-dimensional trajectories and controls for the derivative of the guidance law (4.22)-(4.23) and (OIP) a τ=0 .

Figure 6 . 7 : 8 Figure 6 . 8 :

 67868 Figure 6.7: Three-dimensional trajectories and controls of (OIP) a τ .

Lemma 7 . 6 (

 76 Conic Implicit Function Theorem with Parameters). Let C ⊆ R p be an open convex subset with non empty interior, of vertex 0, and F : R k

  τ k ). Since t τ k f (u τ k ) ∈ [0, b],up to some subsequence, the sequence of final times (tτ k f ) k∈N = (t τ k f (u τ k )) k∈N converges to some t f ∈ [0, b]. Since M f is compact, up to some subsequence, the sequence (x τ k (t τ k f )) k∈N ⊆ M f converges to a point in M f .

t f 0 ((S τ 2 kū(S τ 2 kū

 022 u τ k (t -τ k ) -ū(t )) • ϕ(t ) d t = R (u τ k (t ) -ū(t )) • S -τ 2 k ϕ (t ) d t + R -ū)(t ) • ϕ(t ) d t = t f 0 (u τ k (t ) -ū(t )) • ϕ(t ) d t + R (u τ k (t ) -ū(t )) • S -τ 2 k ϕϕ (t ) d t + R -ū)(t ) • ϕ(t ) d t(7.12)

0 f 7 . 3 . 1 Lemma 7 . 7 .

 073177 [START_REF] Sussmann | A maximum principle for hybrid optimal control problems[END_REF] gives x(t ) = φ 1 (t )1 [-∆,0) (t ) + 1 [0, t f ] (t ) φ 1 (0) + t (s, x(s), x(s), ū(s), ū(s)) d s . Proof of Theorem 6.Consider (OCP) τ with pure state delays, i.e., τ = (τ 1 , 0) ∈ (0, τ 0 ) × {0} and assume that Assumption (B 1 ) holds. For every s ∈ (0, t f ), Lebesgue point of functionf1 (•, x(•), x(•), u(•)), there exists a family (s τ ) τ 1 ∈(0,τ 0 ) ⊆ [s, t f ), which are Lebesgue points of function f1 (•, x τ (•), x τ (• -τ 1 ), u τ (•)), such that f1 (s τ , x τ (s τ ), x τ (s τ -τ 1 ), u τ (s τ )) τ→0 ---→ f1 (s, x(s), x(s), u(s)) , s τ τ→0 ---→ s .Conversely, consider (OCP) τ with general delays τ = (τ 1 , τ 2 ) ∈ (0, τ 0 ) 2 and assume that Assumption (C 3 ) holds. For every s ∈ (0, t f ), Lebesgue point of u(•), there exists a family(s τ ) τ∈(0,τ 0 ) 2 ⊆ [s, t f ), which are Lebesgue points of u τ (•), of u τ (• -τ 2 ) and of u τ (• + τ 2 ), such that u τ (s τ ) τ→0 ---→ u(s) , u τ (s τ -τ 2 ) τ→0 ---→ u(s) , u τ (s τ + τ 2 ) τ→0 ---→ u(s) , s τ τ→0 ---→ s .

Chapter 1. Elements of Optimal Control

	.28)
	Even if the formalism has been simplified, the same difficulties, related to the inte-
	gration of the adjoint equations arising from Theorem 1.3, still appear in the formu-
	lation of Theorem 1.4. Besides, Assumption 1.1 is needed again to prove the bound-
	ness of the multipliers µ m e , µ m i related to the mixed constraints. It is crucial to note
	that, in the case where only mixed control-state constraints appear, Assumption 1.1
	provides moreover that precisely the couple (p(•), p 0 ) is not trivial, which is more in-
	formative than the weaker condition (p(•), p 0 , µ m e (•), µ m i (•)) = 0 (compare with The-
	orem 1.3). The proof of Theorem 1.4 arises easily adapting the proof of Theorem 1.3.

  T atm +273.1

	288.08		5.256	kPa	, h < 11 km (Troposphere)
	22.65e -1.73-0.000157h kPa	, 11 km ≤ h < 25 km (Lower Stratosphere)
	2.488 T atm +273.1 216.6	-11.388	kPa	, h ≥ 25 km (Upper Stratosphere)
	and, from which, we recover the air density as
	ρ =	p at m 0.2869(T atm + 273.1)	kg/m 3 .	(2.3)

  Theorem 3.1 states that there exist adjoint vectors coming from the Maximum Principles applied to (OCP) m and to (OCP) i , respectively, which coincides within each subinterval [r i , s i ]. This practically allows us to seek a global adjoint vector of (OCP) m by gluing together the local adjoint vectors related to the local problems (OCP) i . Before giving the proof of Theorem 3.1, let us clarify how one could take advantage of this to solve (OCP) m numerically by classical shooting methods (see Section1.3.3). Without loss of generality, we assume that Assumption 3.1 is satisfied by an atlas

		satisfying the Maximum Principle (1.7)-(1.10) and
	p 0 i = p 0	and	p

i (t ) = p(t ) , t ∈ [r i , s i ] . (3.1)

Structure of Extremals for Optimal Guidance Problems

	Chapter 3. cos γp a r -	sin γ cos χ r	p a L -	sin γ sin χ r cos L	p a -	g cos γ v	p a v E	= 0
	which, combined, lead to cos χp a L + expression, combined with the adjoint equation of p χ , gives p a sin χ cos L p a = 0 almost everywhere within E . This L | E = p a | E = 0. On the other hand, from the adjoint equation of p γ , we have (v p a r -g p a v
	Proof. We implicitly refer to Proposition 3.1 and Lemma 3.5. Suppose, by contradic-
	tion, that p, [Y 2 , [X , Y 2 ]] = p, [Y 2 , [Y 2 , [X , Y 2 ]]] = 0 almost everywhere in E . From
	this, one recovers respectively the following two expressions
	sin γp a r +	cos γ cos χ r	p a L +	cos γ sin χ r cos L	p a -	g sin γ v	p a v E	= 0

.44) Actually, at least one between expression (3.43) and expression (3.44) always holds. Lemma 3.7. Under Assumption 3.3, almost everywhere in E , there holds p, [Y 2 , [X , Y 2 ]] = 0 or p, [Y 2 , [Y 2 , [X , Y 2 ]]] = 0 .

  2 (w 2 , w 3 , w 2 , w 3 ) denotes again a polynomial of degree greater than one in w 2 , w 3 and in their first time derivatives), under these approximations, we easily get

	Chapter 4. Numerical Guidance Strategy
			     	w 2 =	c m d 2η	w 2 +	cos γ c m h r	+ P 1 (w 2 , w 3 , w 2 , w 3 )	.	(4.16)
			    	w 3 =	c m d 2η	w 3 + P 2 (w 2 , w 3 , w 2 , w 3 )
	At this step, in order to obtain an explicit guidance law for controls w 2 , w 3 , one pos-
	sible way consists in integrating system (4.16). Actually, two difficulties arise. First, the terms c m d 2η and cos γ
											). Combining (4.14) and
	(4.15), it is not difficult to see that the first expression in (4.14) leads to
	w 2 =	p γ 2η	=	d 2η	c m w 2 +	cos γ h r	-	1 2ηr 2 p L cos χ + p l	sin χ cos L	+ P 1 (w 2 , w 3 , w 2 , w 3 )
	where P 1 (w 2 , w 3 , w 2 , w 3 ) is a polynomial of degree greater than one in w 2 , w 3 and in their first time derivatives. Since we can assume p C 0 r 2 d 2η cos γ h r under standard
	flight conditions and Assumption 4.1, the previous expression can be formally ap-
	proximated. Iterating the same procedure on the second expression of (4.14) (here,

P

  .20) The solution of the system composed by equations (4.[START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF]), (4.20) provides the parameters A and B needed to close expression (4.18) related to control w 2 . Indeed, by defining the gain parameters k 1 (R) and k 2 (R) as

	k 1 (R) = bR	e

bRe -bR -2bR 4 + e bR (bR -2)e -bR (bR + 2) , k 2 (R) = bR e bR (bR -1) + e -bR (bR + 1) 4 + e bR (bR -2)e -bR (bR + 2) and substituting into (4.18) the expressions of A and B provided by the system (4.19)-(4.20), one obtains straightforwardly that

  C 1 ,1 and (OIP) C 1 ,2 . First, the guidance law (4.22)-(4.23) is employed to initialize the problem of order zero (OIP) 0 , as detailed at the end of Section 4.2.2. Therefore, maintaining the scenario constant, i.e. λ 2 = 0, the contributions of the thrust, of the gravity, of the roundness of the Earth and of the minimal final time (if C 1 = 0) are added to obtain the intermediate problem (OIP) Inter . Finally, the original scenario is recovered by a discrete continuation on parameter λ 2 , which leads to the solution of (OIP).

	Guidance Law (4.22)-(4.23)	(OIP) 0	(OIP) Inter	(OIP)

  [START_REF] Bonnard | Singular trajectories and their role in control theory[END_REF])-(4.23), provided in Chapter 4. We resume this procedure by the simplified scheme below, which, for sake of concision in the exposition, iterates only on one variable.

Data: Maximal value r max , minimal value r min and iterations I max ≥ 1. Result: Initialization grid files (G 1 init ,G 2 init ) on variable r for (OIP). begin Consider problem (OIP) without minimal final time. Solve (OIP) on the following scenario (in standard units) S 0 :

  Denote by S temp the scenario S 0 where r f is replaced by r t emp .Seek in G1 init the scenario S which satisfies d 2 (S, S temp ) = min Example of algorithm to develop initialization grids for (OIP).

	S ∈G 1
	Store S temp in G 1 init and the initial value of its optimal adjoint in G 2 init .
	end
	end
	end
	Algorithm 2:

r ) end init d 2 (S , S temp ), where d 2 is the euclidean norm with respect to variable r . Solve (OIP) on scenario S temp by Algorithm 1, acting on the variable r , and starting from the solution (state and adjoint variables) of scenario S.

if a solution of (OIP) for scenario S temp is found then

Table 5 .

 5 1: Number of iterations used for each state variable, for different grids.

Solving Optimal Control Problems with Delays with

  final condition p(2) = 0. Denoting by (x τ (•), p τ (•), u τ (•)) optimal extremals, from the maximization condition (6.5), optimal controls are given in the closed-loop form

	Chapter 6. u τ,1 (t ) = -5p τ,2 (t ) x τ,2 (t ) + 0.25 , u τ,2 (t ) = -5p τ,4 (t ) x τ,4 (t ) + 0.25 .
				t )p 4 (t + τ)
	ṗ3 (t ) = p 3 (t ) +	∂R 2 ∂x	(x 3 (t ), x 4 (t )) p 3 (t ) -p 4 (t ) + 2x 3 (t )
	ṗ4 (t ) = p 4 (t )(u 2 (t ) + 2) +	∂R 2 ∂y	(x 3 (t ), x 4 (t )) p 3 (t ) -p 4 (t ) + 2x 4 (t )

  .1 (the Simpson's rule to compute the cost is retained).

	τ	0	0.05	0.1	0.2	0.4
	C (CSTR) τ	(t τ f , u τ )	0.02248 0.02282 0.02313 0.02370 0.02459
	N. of Continuation Iterations	1	1	1	1	3
	Computational Time (s)	0.200	0.240	0.450	0.820	2.120
	τ	0.6	0.8	1	1.2	1.5
	C (CSTR) τ	(t τ f , u τ )	0.02516 0.02547 0.02556 0.02549 0.02527
	N. of Continuation Iterations	5	5	5	3	6
	Computational Time (s)	2.860	3.260	3.070	2.150	4.220
	Table 6.1: Optimal values of (OCP) (CSTR) τ	for different delays τ.

  .1.

	0.16
	0.14
	0.12
	0.1
	0.08
	0.06
	0.04
	0.02
	0
	-0.02	0	0.2	0.4	0.6	0.8	1	1.2	1.4	1.6	1.8	2

  +p γ e v s c m sin α cos β + p χ e v s c m cos γ sin α sin β + p α u α + p β u β -C u (u 2 thanks to the transversality conditions concerning the final manifold (2.21). To recover the guess of p γ (0) and p χ (0), we need to work we the adjoint equations of variables α and β. They assume the following form ṗα = 2p v s e v s c m η cos α sin αp γ e v s c m cos α cos βp χ e v s c m cos γ cos α sin β ṗβ = p γ ev s c m sin α sin βp χ e v s c m cos γ sin α cos β .

						6.3. Numerical Strategy to Solve (OIP) τ
									α + u 2 β )
	whose maximization condition leads to
				p α = 2C u u α , p β = 2C u u β .	(6.15)
	Since the problem is autonomous and the final time t 0 f is free, the adjoint equation
	related to p v s reads ṗv s = -H +C u (u 2 α + u 2 β ) = C u (u 2 α + u 2 β ), from which we recover
				p v s (t ) = 1 +C u	t f t 0	(u 2 α (s) + u 2 β (s)) d s	(6.16)
						13) with the guidance law (4.22)-(4.23), the
	value of α and β is obtained from
	β = arctan	w 3 w 2	, sin α =	w 2 cos β	or sin α =	w 3 sin β	.	(6.14)
	Differentiating (6.14) and w 2 2 + w 2 3 = sin 2 α with respect to time, we obtain the value
	of controls u α and u β as follows
	u β = β =	w 2 ẇ3 -w 3 ẇ2 sin 2 α	, u α = α =	w 2 ẇ2 + w 3 ẇ3 sin α cos α	.
	We focus on the Maximum Principle related to (OIP) a 0 . The normal Hamiltonian is
	H = p r e v s sin γ + p L	e v s r	cos γ cos χ + p	e v s r	cos γ sin χ cos L	-p v s d + ηc m sin 2 α e v s

  For every t ∈ (0, t τ f ], the first Pontryagin cone K τ (t ) ⊆ R n+1 at xτ (t ) for the extended system is defined as the closed convex cone containing vectors wτ s,z (t ) where z ∈ U and 0 < s < t is a Lebesgue point of u τ (•), u τ (• -τ 2 ) and of u τ (• + τ 2 ). The augmented first Pontryagin cone K τ 1 (t ) ⊆ R n+1 at xτ (t ) for the extended system is defined as the closed convex cone containing f (t , x τ

	.4)
	Vectors ṽτ s,ξ (•) are usually called variations vectors. In what follows, for sake of clarity
	in the exposition, it is useful to denote wτ s,z (t ) = ṽτ s,ω -z (s) (t ) + ṽτ s+τ 2 ,ω + z (s) (t ).
	Definition 7.1.

  Moreover, since U is compact, functions ū(•), γ(•) can be chosen to be measurable on [0, t f ] using a measurable selection lemma (see, e.g. [35, Lemma 3A, page 161]). Combining (7.10) with (7.9) providesx(t ) = φ 1 (t )1 [-∆,0) (t ) + 1 [0, t f ] (t ) φ 1 (0) +which proves that the function ū(•) is an admissible control for (OCP) τ=0 = (OCP).

	f1 ∂x	(t , x(t ), x(t ), ū(t )) ,	∂ f1 ∂y	(t , x(t ), x(t ), ū(t )) .
		t
		f 1 (t , x(t ), x(t ), ū(t )) d s
		0

Continuity Properties of Pontryagin Extremals for

  h i k (s)| by β. Firstly, since s is a Lebesgue point of h(•), there exists 0 < δ s,α s ≤ 1 such that {1, . . . , n + 1}. On the other hand, from what said above, there exists an integer k δ s,α s such that every k ≥ k δ s,α s and every i ∈ {1, . . . , n + 1}. Finally, by assumption, we have that h τ (•) is continuous for τ 1 > 0, and then, for every k ≥ k δ s,α s and every i ∈ {1, . . . , n + 1}, there exists t k,i ∈ [s, s + δ s,α s α s ] ⊆ [s, s + α s ] such that ×{0} and i ∈ {1, . . . , n +1} there exists t k,i ∈ [s, s +α s ] Lebesgue point of h τ k (•) (by continuity) such that |h

	Chapter 7. h	τ k i (t k,i ) -	1 δ s,α s α s	s	s+δ s,α s α s	h	τ k i (t ) d t <	β 3	.
	Resuming, for every τ k ∈ 0, 1 k δ s,α s						τ k i (t k,i
		1 δα s	s	s+δα s	h	τ k i (t ) d t -	s	s+δα s	h i (t ) d t <	β 3	.
			τ k i k (t ) h i (s) -1 δ s,α s α s s+δ s,α s α s s β h i (t ) d t < 3
	for every i ∈ 1 δ s,α s α s	s		s+δ s,α s α s	h	τ k i (t ) d t -	s	s+δ s,α s α s	h i (t ) d t <	β 3

for every i ∈ {1, . . . , n + 1}. We exploits this fact to bound |h

  •) assumes its values at extremal points of U , almost everywhere in [-∆, t f ]. Without loss of generality, we extend u τ (•) by some constant vector of U in [t τ f , b]. Denote h τ (t ) = (h τ 1 (t ), . . . , h τ 3m (t )) = u τ (t ), u τ (t -τ 2 ), u τ (t + τ 2 )

h(t ) = (h 1 (t ), . . . , h 3m (t )) = u(t ), u(t ), u(t )

  From Lemma 7.7, there exists a family (s τ ) τ∈(0,τ 0 ) 2 ⊆ [s, t f ), which are Lebesgue points of u τ (•), of u τ (• -τ 2 ) and of u τ (• + τ 2 ), such thatu τ (s τ ) , u τ (s τ + τ 2 ) τ +τ 2 ,ω + z (s τ ) (•), solutions of(7.4) with initial data provided by (7.2), (7.3), respectively. We denote wτ s τ ,z (t ) = ṽτs τ ,ω - z (s τ ) (t )+ ṽτ s τ +τ 2 ,ω + z (s τ ) (t ). Since the dynamics and the integral cost function related to (OCP) τ take the formsf 0 (t , x, y, u, v) = f 0 1 (t , x, y, u) + f 0 2 (t , x, y, v) , f (t , x, y, u, v) = f 1 (t ,x, y, u) + f 2 (t , x, y, v) Lemma 7.7 easily gives lim τ→0 ω - z (s τ ) + ω + z (s τ ) = f (s, x(s), x(s), z, z) -f (s, x(s), x(s), u(s), u(s)) .

						7.3. Proof of Theorem 6.1
						τ→0 ---→ u(s) , s τ	τ→0 ---→ s .
	This allows to consider ṽτ s τ ,ω -z (s τ ) (•), ṽτ
				s,ω z (s) (•) is the solution of
	   	ψ(t ) =	∂ ∂x f	(t , x(t ), x(t ), u(t ), u(t )) +	f ∂y ∂	(t , x(t ), x(t ), u(t ), u(t )) ψ(t )	. (7.14)
	  	ψ(s) = f (s, x(s), x(s), z, z) -f (s, x(s), x(s), u(s), u(s))

τ→0 ---→ u(s) , u τ (s τ -τ 2 ) τ→0 ---→ u(s) s

  •) of the Cauchy problem

	  	ż(t ) = -	∂h ∂x	(t ,
	 

x(t ), x(t ), z(t ), -1, u(t ), u(t )) -∂h ∂y (t , x(t ), x(t ), z(t ), -1, u(t ), u(t ))

  ) = (exp f u (t f ; t , •)) * x(t f ) • p(t f ) + , x(t ), p(t ), p 0 , u(t )) ≥ h(t , x(t ), p(t ), p 0 , u) (A.4)for every u : c m e (t , x(t), u) = 0 , c m i (t , x(t ), u) ≤ 0 m e (t f , x(t f ), u) = 0 c m i (t f , x(t f ), u) ≤ 0 h(t f , x(t f ), p(t f ), p 0 , u) = -p 0 ∂g ∂t (t f , x(t f )) .Several proofs of Theorem A.1 exist. Even if, the first results have been achieved by Dubovitskii and Milyutin (we found contributions only in Russian, see, e.g.[START_REF] Milyutin | Maximum principle for regular systems[END_REF]), we think that one of the clearest proofs is provided by Dmitruk in[START_REF] Dmitruk | Maximum principle for the general optimal control problem with phase and regular mixed constraints[END_REF]. Dmitruk's proof is based on a convexification of the problem by using sliding variations, tools introduced by Gamkrelidze (see, e.g.

	• Adjoint Equations
					ẋ(t ) =	∂h ∂p	(t , x(t ), p(t ), p 0 , u(t )) ,	(A.2)
		p(t t f t	(exp f u (s; t , •)) * x(s) • µ m e (s) •	∂c m e ∂q	(s, x(s), u(s)) d s (A.3)
	+	t	t f	(exp f u (s; t , •)) * x(s) • µ m i (s) •	∂c m i ∂q	(s, x(s), u(s)) d s +	t	t f	(exp f u (s; t , •)) * x(s) • d µ s (s) •	∂c s ∂q	(s, x(s))
	• Maximality Condition
				h(t

• Stationarity Condition ∂h ∂u (t , x(t ), p(t ), p 0 , u(t ))+µ m e (t ) ∂c m e ∂u (t , x(t ), u(t ))+µ m i (t ) ∂c m i ∂u (t , x(t ), u(t )) = 0 (A.5) • Complementarity Slackness Conditions

µ j m i (t )c j m i (t , x(t ), u(t )) = 0 for every j = 1, . . . , r m i , (A.6) t f 0 c j s (t , x(t )) d µ j s (t ) = 0 for every j = 1, . . . , r s (A.7)

• Transversality Conditions

If M f is a submanifold of M , locally around x(t f ), then the adjoint vector can be built in order to satisfy

p(t f )p 0 ∂g ∂x (t f , x(t f )) ⊥ T x(t f ) M f

and, moreover, if the final time t f is free, one has max c

Maximum Principle Projected onto the Manifold At

  this step, recall definitions(1.15) and(1.16) of Chapter 1 concerning integral curves in T * M . From this, easy computations in local coordinates lead to Thanks to the previous relations and the equivalence (A.14), the projected adjoint vector defined by (A.13) satisfies, for t ∈ [0, t f ],

							A.3
			∂ ∂x k q(t )	(t ) =
	n j =1 ∂x π ∂ f j -t f t (exp F u (s; t , •)) * x(s) •µ m (s)• ∂C m ∂x (x(s), u(s)) d s =	t	t f	π (exp F u (s; t , •)) * x(s) •µ m (s)•	∂C m ∂x	(x(s), u(s)) d s
	π	t f		∂C s ∂x	(x(s)) =	t f	∂C s ∂x	(x(s)) .
		p(t ) = π(λ(t )) = π	t	t f	(exp F u (s; t , •)) * x(s) • µ m (s) •	∂C m ∂x	(x(s), u(s)) d s
	200

. . , n d d t π (exp F u (s; t , •)) * x(s) • λ s k (q(t ), u(t )) π (exp F u (s; t , •)) * x(s) • λ s ∂ ∂x j q(t ) .

By the uniqueness of Hamiltonian flows, the previous expression leads immediately to the following fundamental equality (useful in what follows)

(exp f u (s; t , •)) * q(s) • π(λ s ) = π (exp F u (s; t , •)) * x(s) • λ s . (A.14) t (exp F u (s; t , •)) * x(s) • d µ s (s) • t π (exp F u (s; t , •)) * x(s) • d µ s (s) •

. Proof of Theorem 1.3 on Manifolds

  (exp F u (s; t , •)) * x(s) • d µ s (s) • ∂C s ∂x (x(s)) + (exp F u (t f ; t , •)) * x(t f ) • λ(t f ) + (exp f u (t f ; t , •)) * q(t f ) • p(t f ) .The adjoint equations follow if we prove that, for every j = 1, . . . , r m , k = 1, . . . , r s ,

			t f
			+
			t
			=	r m j =1	t	t f	µ m (s)(exp f u (s; t , •)) * j q(s) • π	j m ∂x ∂C	(x(s), u(s)) d s
		+ (x(s)) π r s j =1 t f t d µ j s (s)(exp f u (s; t , •)) * q(s) • π j ∂C s ∂x ∂C j m ∂x (x(s), u(s)) = ∂c j m ∂q (q(s), u(s)) , π ∂C k s ∂x	(x(s)) =	∂c k s ∂q	(q(s)) .	(A.15)
	But, from relations (A.9), under appropriate local coordinates, one has
	π	j m ∂x ∂C	(x(s), u(s)) =

Renforcer et accélérer la convergence pour le problème de guidage.Même si les procédures numériques développées en 2. fournissent un schéma efficace pour résoudre le problème de guidage optimal de véhicules lanceurs, une problématique reste à régler: la haute sensibilité aux conditions initiales.

Acknowledgements

pu être achevé sans leurs immenses supports. La Je tiens à le remercier encore pour son aide

Numerical Guidance Strategy

The arguments presented in this chapter concern, from a practical point of view, one of the main objectives of this thesis, that is, providing efficient indirect methods to solve rendezvous problems. We base our methodology on the previous chapter, in which we propose a strategy to define well-posed indirect methods on problem (GOGP), providing also optimal controls as functions of the state and the adjoint vector. More precisely, the aim of the present chapter consists in exploiting the previous geometric information to conceive a numerical strategy to solve problem (OIP).

The proposed numerical approach can be summarized as follows. Adopting indirect methods to solve (OIP) faces two crucial issues: the presence of mixed controlstate constraints and the initialization of shooting methods. The first difficulty can be managed by the procedure developed in Chapter 3, that, thanks to a consistent localization of the problem, allows to convert mixed control-state constraints into pure control constraints. On the other hand, initializing shooting methods directly on (OIP) is quite complicated, because, the complex structure of the Lie algebra generated by the flight dynamics and sophisticated missions may remove the intuition on the shape of optimal trajectories. The proposed idea consists in adopting homotopy methods to simplified the original problem and achieve the optimal solution by parameter deformation. More specifically, the problem of order zero is obtained by removing the contributions of the thrust, of the gravity, of the roundness of the Earth and simplifying the original scenario. This trick allows, with some manipulations on the Maximum Principle, to recover a new explicit guidance law, able to correctly initialize a shooting method on the simplified problem. In a second time, the contributions previously removed are added by an efficient deformation strategy.

The chapter is organized as follows. In Section 4.1, we provide a homotopy scheme generalized to the broader problem (GOGP). Therefore, in Section 4.2, we deal with the initialization of shooting methods on the problem of order zero. Finally, Section 4.3 provides numerical simulations testing the efficiency of our approach.

Numerical Simulations for (OIP)

Considering Assumption 4.1, we may choose as initial guess t f = R(0)/v 0 and p γ (0) = 2ηw 2 (0), p χ (0) = 2η cos(γ 0 )w 3 (0) from expressions (4.14). The guess values of p r (0), p L (0), p l (0) are easily obtained by combining the relation H (s = 0) = 0, where H is the Hamiltonian of (OIP) s 0 , with the dual system (4.15) (by differentiating successively expressions (4.14)). Concerning the initial guess of p v (0), we modify slightly the problem as follows. When introducing formulation (OIP) s 0 , the new velocity variable v s = ln(v) is adopted. Its dynamical equation is given by

We integrate the quantity v s and its dynamics into (OIP) s 0 . Therefore, it is not difficult to see that, the previous arguments concerning (OIP) s 0 still hold if we substitute p 0 by p 0p v s (in particular in expressions (4.14) and in the dual system (4.15)), where p v s is the adjoint variable related to v s . Thanks to the fact that the variable v s does not enter explicitly within this formulation, there holds

This and the transversality conditions provide immediately the equality p v s (0) = 0. All the numerical methods that we derive for (OIP) contain intrinsically this modified formulation, in which the norm v of the velocity is replaced by its natural logarithm v s = ln(v). This procedure allows to obtain a guess from (4.22)-(4.23) to initialize classical shooting methods on the problem of order zero (OIP) 0 , as shown above.

Numerical Simulations for (OIP)

Within this section, we propose numerical simulations concerning the optimal interception problem (OIP), to justify the homotopy approach previously developed. We first introduce the mathematical modelization of interception missions and numerical values of tested scenarios. In a second time, we explicitly provide the homotopy numerical scheme proposed in Section 4.1.2 and related numerical results.

Mathematical Design of the Mission

The implemented numerical algorithm solve (OIP) by combining the numerical strategies proposed in Section 4.1 and Section 4.2. In particular, the linear continuation scheme with two parameters λ = (λ 1 , λ 2 ) of Section 4.1.2 and the initialization arisen from the analysis of the problem of order zero presented in Section 4.2 are adopted. For these numerical simulations, we consider two different type of mission. The first mission consists in reaching the target with the highest possible final velocity while

Part III

Continuity of Pontryagin Extremals

with Respect to Delays

Continuity Properties with Respect to Delays

Common assumptions: (A 3 ) The optimal trajectory x(•) has a unique extremal lift (up to a multiplicative scalar) defined in [0, t f ], which is normal, denoted (x(•), p(•), -1, u(•)), solution of the Maximum Principle formulation given by Theorem 1.1.

(A 4 ) There exists a positive real number b such that, for every τ = (τ 1 , τ 2 ) ∈ [0, ∆] 2 and every v(•) ∈ U τ U , denoting x τ,v (•) the related trajectory arising from dynamics (6.2) with final time t τ,v f , one has for every t ∈ [-∆, t τ,v f ] there holds :

Assumptions in case of pure state delays:

For every delay τ, every optimal control u τ (•) of (OCP) τ is continuous.

(B 2 ) The sets

are convex for every t ∈ R and every x, y ∈ R n , where we denote f1 (t , x, y, u) = f 1 (t , x, y, u), f 0 1 (t , x, y, u) .

Assumptions in case of delays both in state and control variables:

The maps f 0 1 , f 1 and f 0 2 , f 2 are affine w.r.t. variables u and v, respectively.

(C 2 ) Problem (OCP) τ with both state and control delays has fixed final time t f .

(C 3 ) Either, for every delay τ, every optimal control u τ (•) of (OCP) τ is continuous or control u(•) assumes its values at extremal points of U , a.e. in [-∆, t f ].

To justify the use of these assumptions in the following, some remarks are in order. First of all, assumptions (A 2 ) and (A 3 ) on the uniqueness of the solution of (OCP) and on the uniqueness of its extremal lift are related to the differentiability properties of the value function (see, e.g. [START_REF] Clarke | The relationship between the maximum principle and dynamic programming[END_REF][START_REF] Aubin | Set-valued analysis[END_REF][START_REF] Rifford | On the stabilization problem for nonholonomic distributions[END_REF]). They are standard in optimization and are just made to keep a nice statement (see Theorem 6.1). These assumptions can be 6.2. Homotopy Algorithm and Numerical Simulations

Solving (OCP) τ by Shooting Methods and Homotopy on Delays

As pointed out previously, the critical behavior coming out from applying indirect methods to (OCP) τ consists of the integration of mixed-type equations that arise from system (6.4). The convergence results of Theorem 6.1 suggests the idea that, when considering indirect methods coupled to homotopies on the delay, one may solve the adjoint system (6.4) via usual iterative methods for ODEs, for example, by using the global state solution at the previous iteration. Moreover, the global adjoint vector of (OCP) could be used to initialize, from the beginning, the whole procedure. Under assumptions of nice enough extremals, these considerations lead us to Algorithm 3. For sake of brevity, we design this algorithm for problems (OCP) τ with pure state delays, i.e. τ = (τ 1 , 0) and fixed final time t f . Nevertheless, it is evident how to adapt Algorithm 3 to problems (OCP) τ with general constant delays and free final time t f . For sake of clarity, we do not consider any acceleration step in Algorithm 3, although, adapting this speed-up procedure from Algorithm 1 is straightforward.

We prove the convergence of Algorithm 3, under appropriate assumptions, by applying Theorem 6.1. Without loss of generality, we focus on the case where (OCP) τ has pure state delays. We assume that we are able to express optimal controls, via (6.5), as continuous functions of x and p (note that we do not remove nonregular extremals). Suppose that assumptions (A) and (B ) hold and that the delay

, where τ 0 is provided by Theorem 6.1. Therefore, for every ε in the open interval (0, τ), problem (OCP) ε has at least one optimal solution with normal extremal lift. This justifies the choice of setting p 0 = -1 throughout Algorithm 3. Thanks to Theorem 6.1, there hold x ε (•)

--→ u(•), as soon as ε → 0. From this, if τ k+1 -τ k is small enough, the modified internal integration of adjoint equations within Algorithm 3 results to be well-defined and well-initialized by the optimal adjoint vector p(•) of (OCP). Indeed, necessarily, the algorithm will travel backward one of the subsequence converging to the solution of (OCP). Since, for every sequence (ε k ) k∈N converging to 0, the related extremal lift (x ε (•), p ε (•), -1, u ε (•)) of (OCP) ε converges to the one of (OCP) (for the evident topologies), steps A.1), A.3) of Algorithm 3 are well-posed and make the while loop converge in a finite number of iterations. Note that, if assumptions (A 2 ) and (A 3 ) are weakened as previously explained in Section 6.1, Algorithm 3 still remains well-posed and converges to some local extremal. Remark 6.2. It is interesting to remark that, at least formally, there are no difficulties to apply Algorithm 3 to more general (OCP) τ that consider locally bounded varying delays, functions of the time and the state, i.e., τ : R × R n → [-∆, 0] 2 : (t , x) → τ(t , x). In this context, some relations close to (6.4)-(6.7) are still provided (see, e.g. [START_REF] Asher | Optimal control of systems with state-dependent time delay[END_REF]), so that, the proposed numerical continuation scheme remains well-defined.

First Numerical Tests

In order to prove effectiveness and robustness of our approach, we test Algorithm 3 preliminarily on an example, which is not related to aerospace applications and considers the nontrivial problem consisting of the optimal control of a continuous nonlinear two-stage stirred tank reactor system (CSTR), proposed by [START_REF] Oh | Optimal feedback control of time-delay systems[END_REF][START_REF] Dadebo | Optimal control of time-delay systems by dynamic programming[END_REF].

We stress the fact that, in this section, we are interested in solving an optimal control problem with delays (OCP) τ by indirect methods starting from its non-delayed version (OCP), without taking care of how (OCP) is solved, recalling that, in the case of the interception problem with delays (OIP) τ , we have already discussed in Chapter 4 how to efficiently initialize indirect methods on its non-delayed version (OIP).

The numerical examples proposed are solved applying verbatim Algorithm 3, appropriately modified by considering the same acceleration step as in Algorithm 1. An explicit second-order Runge-Kutta method is handled to solve all the ODEs coming from the Maximum Principle formulation while the routine hybrd.c (see, e.g. [START_REF] Moré | The minpack project[END_REF]) is used to solve related shootings. The procedure is initialized using the solution of (OCP) provided by the AMPL framework, combined with IPOPT (see, e.g. [START_REF] Fourer | Ampl[END_REF][START_REF] Wächter | On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming[END_REF]).

One important remark is the following. When passing the numerical approximation of the extremals in the modified internal integration of adjoint equations step of Algorithm 3, attention must be adopted. Indeed, it is known that, using collocation methods like Runge-Kutta schemes, the error between the solution and its numerical approximation remains bounded within [0, t f ] and decreases with h p , where h is the time step while p is the order of the method, only if this numerical approximation is obtained by interpolating the numerical values within each subinterval of integration with a polynomial of order p. From this remark, it is straightforward that the dimension of the shooting considered in Algorithm 3, not only increases with respect to 1/h, but is also proportional to p. In the particular case of an explicit secondorder Runge-Kutta method, the dimension of the shooting problems in Algorithm 3 is bounded above by 2n/h (where n is the dimension of the state). We use a machine Intel(R) Xeon(R) CPU E5-1607 v2 3.00GHz, with 7.00 Gb of RAM.

A Nonlinear Chemical Tank Reactor Model

The introductory numerical example that we consider concerns a two-stage nonlinear continuous stirred tank reactor (CSTR) system with a first-order irreversible chemical reaction occurring in each tank. The system was first proposed and studied by [START_REF] Oh | Optimal feedback control of time-delay systems[END_REF], and successively by [START_REF] Dadebo | Optimal control of time-delay systems by dynamic programming[END_REF] in the framework of the dynamic programming.

Chapter 7. Continuity Properties of Pontryagin Extremals

Controllability for (OCP) τ

For any integer p ≥ 1, fix 0 < t 1 < • • • < t p < t f Lebesgue points of control u(•) and p arbitrary values u i ∈ U . We denote v| n the first n coordinates of a vector v ∈ R n+1 . For an appropriate small positive real number ε p > 0, denoting by x(ε 1 ,ε 2 ) (•) the trajectory solution of (7.1) with delay τ = (ε 1 , ε 2 ) and control u (τ 1 ,τ 2 ) (•), we define the mapping

Thanks to Assumption (A 2 ) and by continuity with respect to initial data, Γ is welldefined and continuous. Moreover, Γ(0, . . . , 0) = 0 and we note that

From Lemma 7.2 and a recursive use of the needle-like variation formula (as detailed in Section 7.1.3), for almost every (ε 1 , ε 2 ) ∈ R 2 + small enough, one obtains that Γ is almost everywhere strictly differentiable w.r.t. (δ, η 1 , . . . , η p ) at 0,

) on dense subsets and, moreover, the remainder of the related Taylor development converges to zero uniformly w.r.t. (ε 1 , ε 2 ) on dense subsets. From Assumption (A 3 ), the unique extremal lift of x(•) is normal, hence, it follows from Lemma 7.5 that Int K τ=0

We recall that the dynamics and the integral cost function related to (OCP) τ take the forms

Therefore, thanks to Remark 7.1, it is straightforward to see that there exist a real number δ, an integer p ≥ 1 and a variation π = {t 1 , . . . , t p , η 1 , . . . , η p , u 1 , . . . , u p } such that the associated mapping Γ satisfies

At this point, Lemma 7.6 implies the existence of a scalar delay τ 0 > 0 such that, for every τ = (τ 1 , τ 2 ) ∈ [0, τ 0 ) 2 , there exist a real δ(τ) and positive reals η 1 (τ), . . . , η p (τ) such that Γ(τ 1 , τ 2 , δ(τ), η 1 (τ), . . . , η p (τ)) = 0. Moreover, quantities δ(τ), η 1 (τ), . . . , η p (τ) are continuous with respect to τ. From Assumption (A 4 ), it follows that, for every τ = (τ 1 , τ 2 ) ∈ [0, τ 0 ) 2 , the subset M f is reachable for the dynamics of (OCP) τ , in a final time

We have proved that, for every τ = (τ 1 , τ 2 ) ∈ (0, τ 0 ) 2 , (OCP) τ is controllable. Remark that this argument still holds for problems (OCP) τ with pure state delays τ = (τ 1 , 0).

Chapter 7. Continuity Properties of Pontryagin Extremals

In particular, there holds x( t f ) ∈ M f , then, ū(•) is admissible for (OCP) τ=0 = (OCP).

Similarly to the previous case, thanks to the achieved convergences and Assumption (C 1 ), one easily proves that C 0 ( t f , ū) ≤ C 0 (t f , u). Therefore, from Assumption (A 2 ), we infer that t f = t f , ū(•) = u(•) and x(•) = x(•), from which, the conclusion follows. In this case, not only we have weak convergence of the dynamics and of their derivatives (similarly to (7.11)), but also of optimal controls (under appropriate topologies).

The convergence almost everywhere concerning the optimal controls can be achieved when the second option of Assumption (C 3 ) holds, and more specifically, when control u(

Resuming all the previous results, for every considered case, we have shown that (t f , x(•), u(•)) (substituted by (t f , x(•), ẋ(•)) for the case of pure state delays) is the unique closure point (for the topologies used above) of (t

) k∈N for the cases of pure state delays), for any (sub)sequence of delays (τ k ) k∈N converging to 0. Then, the convergence holds as well for the whole family (t τ f , x τ (•), u τ (•)) τ∈(0,τ 0 ) 2 (substituted by (t τ f , x τ (•), ẋτ (•)) τ∈(0,τ 0 ) 2 for pure state delays).

Convergence of Optimal Adjoint Vectors for (OCP) τ

In what follows, (x τ (•), u τ (•)) will denote an optimal solution of (OCP) τ defined in the interval [-∆, t τ f ] such that, if needed, it is extended continuously in [-∆, t f ]. From the Maximum Principle given by Theorem 1.5, the trajectory x τ (•) is the projection of an extremal (x τ (•), p τ (•), p 0 τ , u τ (•)) which satisfies equations (6.4). From now on, we consider implicitly that either assumptions (B ) or assumptions (C ) are satisfied, depending on whether we consider pure state delays or not. The main step of this part consists in showing the convergence of the Pontryagin cone of (OCP) τ towards the Pontryagin cone of (OCP). Since the definition of variation vectors relies on Lebesgue points of optimal controls, we need first a set of converging Lebesgue points. Finally, for sake of concision, we do not consider final conditions on the state. Recovering the desired convergence results equipped with transversality conditions can be easily done by traveling back the arguments that follow, and using Assumption (A 1 ).

Appendices

A Geometric Maximum Principle with State Constraints

In this appendix we develop a proof of Theorem 1.3 stated in Chapter 1. In the literature, the proof of Theorem 1.3 appears explicitly only in the case M ≡ R n . In this thesis we need to work with the version of Theorem 1.3 valid on manifolds (see the arguments developed in Chapter 3), so we believe necessary providing its proof. We follow the argument proposed in [START_REF] Chang | A simple proof of the pontryagin maximum principle on manifolds[END_REF]. More specifically, the main idea consists in embedding the configuration manifold and the related vector fields into R N for an appropriate integer N ∈ N, therefore, exploiting the version of Theorem 1.3 valid in R N to recover the appropriate set of adjoint vectors and Lagrange multipliers. The appendix is organized as follows. First, we recall the version of Theorem 1.3 valid in R n . In a second time, we introduce some geometric results which allow to operate an appropriate embedding of the configuration manifold into the Euclidean space. Finally, the main part of the proof is provided with all the details. Remark that, in what follows, we make use of all the notations introduced in Chapter 1.

A.1 Version of Theorem 1.3 Valid in Euclidean Spaces

The following version of Theorem 1.3 is well-known (for the proof, see, e.g. [START_REF] Dmitruk | Maximum principle for the general optimal control problem with phase and regular mixed constraints[END_REF]).

Theorem A.1. Let x(•) be an optimal trajectory for (OCP) m,s in R n , associated to the control u(•) on [0, t f ]. Under Assumption 1.1, there exist a nonpositive scalar p 0 , a curve of bounded variation p : [0,

where µ m i is nonpositive, and nonincreasing functions µ i s , i = 1, . . . , r s (generating measures d µ i s ) such that, a.e. in [0, t f ], the following relations hold:

• Nontriviality Condition

A.2 Some Useful Geometric Results

In this section, we recall some fact from differential geometry, useful for what follows. These results are standard and, for any detail, we refer to classic texts such as [START_REF] Lee | Introduction to smooth manifolds[END_REF].

Let M be a n-dimensional manifold. The Whitney's theorem states that there exists a smooth proper embedding i : M → R N , where N = 2n + 1, such that i (M ) is a closed embedded submanifold of R N . Therefore, without loss of generality, in the following we always interpret M ⊆ R N as a closed embedded submanifold. As a further embedding result, thanks to the closedness of M , there exist an open (tubular) neighborhood A of M in R N , a smooth retraction π A : A → M , and a smooth scalar map ρ :

Consider the usual cotangent bundles T * M and T * R N , and define

Equipped with the structure of pullback bundle, the projection π :

In the proof of Theorem 1.3, we are interested in projecting adjoint vectors lying in

For this, we need to define the following orthogonal projection.

Let q ∈ M and (V, ϕ)

It follows that {d x j (•)} j =1,...,N is a local basis for T * R N and that {d x j (•)} j =1,...,n is a local basis for T * M around q. Consider the standard flat metric 〈•, •〉(•) on R N , coming from the Euclidean scalar product. By using the Gram-Schmidt process on {d x j (•)} j =1,...,N , we build a local orthonormal frame {E j (•)} j =1,...,N for T * R N around q. In particular

From the previous results, when restricted to V ∩ M , the orthogonal projection

〈v, E j (q)〉 q E j (q) (A.10) is well-defined and smooth. Finally, since the change of frame mapping between two orthonormal frames is orthogonal, it is easy to see that (A.10) is globally well-defined.

With an abuse of notation, we often identify any (q, v) ∈ T * M to its coordinate v.

A.3. Proof of Theorem 1.3 on Manifolds

A.3 Proof of Theorem 1.3 on Manifolds

For sake of concision, without loss of generality, we prove Theorem 1.3 considering the following fixed final time optimal control problem with state constraints

In what follows, we use the notations introduced in the two previous sections without reporting them continuously. In particular, M is a closed embedded submanifold of R N , where N = 2n + 1. For sake of clarity, we split the proof in several subparts.

Global Change of Variables

Up to canonical identifications, define the following smooth mappings in

Instead of analyzing (A.11), we study the following problem in

We see that, for x ∈ M , there holds F (x, u) = f (x, u) ∈ T x M . Therefore, by applying the flow theorem to the restriction of F to M × R m , since x 0 ∈ M , the feasible trajectories of problem (A.12) belong to M and are equivalent to the feasible trajectories of (A.11). Let (q(•), u(•)) be an optimal solution of problem (A.11). From the consideration given above, it is straightforward to see that (q(•), u(•)) is also optimal for problem (A.12).

Projection of Euclidean Pontryagin Extremals

We apply Theorem A.1 to problem (A.12). Therefore, we recover a nonpositive scalar p 0 , a curve of bounded variation λ : [0, t f ] → R N , a nonpositive vector function µ m ∈ L ∞ ([0, t f ], R r m ), and nonincreasing functions µ i s , i = 1, . . . , r s (generating measures d µ i s ) such that, a.e. in [0, t f ], relations (A.1)-(A.7) are satisfied (recall that, for sake of concision, we do not consider neither particular final conditions nor free final time). Since x([0, t f ]) ∈ M , there holds (x(t ), λ(t )) ∈ T * R N | M a.e. in [0, t f ]. We want to prove that the orthogonal projection of λ(•) onto T * M is the sought adjoint vector of problem (A.11). More precisely, we prove that the curve of bounded variation defined by

B Controllability Results

In this appendix, we provide results concerning the local controllability of the optimal interception problem of order zero along curvilinear abscissa (OIP) s 0 and the analysis of the LOS needed in Chapter 4. The study is organised in two sections.

B.1 Local Controllability of (OIP) s 0

Let us show first that the optimal interception problem of order zero along curvilinear abscissa (OIP) s 0 is W 1,∞ -locally controllable in s f around the solution associated with the constant zero control w 0 (•) = 0. As a standard result, it suffices to prove that the linearized dynamics of (OIP) s 0 along the reduced control (w 0 2 , w 0 3 )(•) = (0, 0) is W 1,∞ -controllable in s f . We use the following classical result (see, e.g. [START_REF] Trélat | Contrôle optimal: théorie & applications[END_REF][START_REF] Coron | Control and Nonlinearity[END_REF]).

Theorem B.1. Consider the linear dynamical control system y (s) = A(s)y(s)+B (s)u(s) in R n with control u ∈ R m , where s → A(s) and s → B (s) are of class C ∞ . We define the matrix sequence

Simple computations show that the linearized matrices evaluated at (w 0 2 , w 0 3 )(•) of the problem of order zero along curvilinear abscissa (OIP) s 0 are

B.2 Analysis of the Line Of Sight

In what follows, we provide the proof of Lemma 4.2. We introduce first the following local orthonormal frame along the direction n of the line of sight (remark the similarity with the local system (3.8) for the velocity v)

and we denote R n (δ 1 , δ 2 ) ∈ SO(3) the transformation from frame (e L , e l , e r ) to frame (i n , j n , k n ). We denote by (•) (a,b,c) the coordinates of a vector along frame (a, b,c).

B.2. Analysis of the Line Of Sight

which gives the sought relation for Ṙ.

For the expressions related to δ1 and δ2 , let us first differentiate vector n, as follows

Therefore, we have

On the other hand .

Gathering together all the previous results, the sought relations δ1 = -v R sin(γ -δ 1 ), δ2 = -v R sin(χ -δ 2 ) arise and the conclusion follows.

List of Tables

Abstract

In this work, we address the real-time optimal guidance of launch vehicles with the objective of designing an autonomous algorithm for the prediction of optimal control strategies, based on indirect methods, able to adapt itself to unpredicted changes of the original scenario. To this aim, we first provide an accurate geometric analysis in the presence of mixed control-state constraints to recover a well-posed framework and correctly apply indirect methods. A practical numerical integration of the problem is proposed by efficiently combining indirect methods with homotopy procedures, increasing robustness and computational speed. Moreover, we improve dynamical models by considering delays. More specifically, we introduce a rigorous and well-posed homotopy framework to recover solutions for optimal control problems with delays via indirect methods. All our contributions made possible the development of a fully automatic, independent and self-regulating software, today property of ONERA-The French Aerospace Lab, for general realistic endo-atmospheric launch vehicle applications focused on optimal missile interception scenarios.

Key words. Nonlinear optimal control, Geometric control, Control and state constraints, Control and state delays, Shooting methods, Homotopy and continuation methods, Optimal guidance, Endo-atmospheric launch vehicles, Interceptor missiles.

AMS subject classification. 49J15, 49M05, 49M37, 65H20.