Abdelhak Chatty
email: abdelhakchatty@ieee.org

Syed Khursheed Hasnain

Philippe Gaussier
email: gaussier@ensea.fr

Ilhem Kallel
email: ilhem.kallel@ieee.org

Philipe Laroque
email: laroque@u-cergy.fr

Adel M Alimi
email: adel.alimi@ieee.org

Effect of Low Level Imitation Strategy on an Autonomous Multi-Robot System Using On-Line Learning for Cognitive Map Building

In this paper, we present the interest of low level imitation strategy on individual and population levels in the field of Multi-Robot System. Particularly, we show that adding a simple imitation capability to our bio-inspired architecture boosts the ability of individual cognitive map building. Taking into account the notion of imitative behavior, we also show that the individual discoveries in each robot (i.e. goals) could have an effect on population level and therfore it induces a new learning capability at the individual level. To analyze and validate our hypothesis, a series of experiments have been performed with and without a low level imitation strategy.

I. INTRODUCTION

In an unknown environment, interactions among robots can be based on Stigmergy [START_REF] Holland | Stigmergy, self-organization, and sorting in collective robotics[END_REF] which reflects indirect communication through the environmental changes. To share partial knowledge of this environment in a cognitive1 multi-robot system, which is based on the cognitive map for navigating and planning, several benefits can be expected from imitation capability [START_REF] Gaussier | From perception-action loops to imitation processes: A bottom-up approach of learning by imitation[END_REF], [START_REF] Laroque | Cognitive map plasticity and imitation strategies to improve individual and social behaviors of autonomous agents[END_REF], [START_REF] Schaal | control, planning, learning, and imitation with dynamic movement primitives[END_REF], [START_REF] Chaminade | From selfobservation to imitation: visuomotor association on a robotic hand[END_REF], [START_REF] Lagarde | Proprioception and Imitation: On the Road to Agent Individuation[END_REF]. The imitation strategy can be considered as a powerful tool for autonomous robots to learn and discover new tasks and places. Learning by imitation and observation is an intuitive and natural method, it is not only the tool for learning but also a way to speed up the learning process. Therefore modern robotics considers imitation as a powerful behavior that enables learning by observation [START_REF] Kuniyoshi | Cooperation by observation -the framework and basic task patterns[END_REF] [START_REF]is imitation learning the route to humanoid robots?[END_REF]. The idea of imitation learning for robots is inspired by the notion of imitation described by developmental psychologists. According to psychologists immediate or low-level imitation corresponds to the ability of a child (few months old) to imitate (spontaneously) meaningless gestures. This low level imitation may serve to higher level functions for instance, learning by observation [START_REF] De Rengervé | Emergent Imitative Behavior on a Robotic Arm Based on Visuo-Motor Associative Memories[END_REF].

In our case, we associate this concept of very low level imitation with our architecture of exploring environments and creating cognitive maps. In the onset of the experiment, a mobile robot (Robot A) creates its own cognitive map (without the imitation strategy) while exploring the environment to discover the resources for its needs. This random exploration takes a long time to build a cognitive map. As robot A completes its cognitive map, a second mobile robot (Robot B) which is able to perform low level imitation starts exploring the environment. When Robot A is in the field of view of Robot B, it imitates the first robot by following its current direction of movement. As robot A visits resources several times depending on its needs, robot B discovers these resources during its low level imitation and builds its own cognitive map. Once the resources are discovered the imitation strategy is turned-off and robot B visits the resources independently depending on its needs.

In section 2 the theory and the bio-inspired architecture of our proposed cognitive map and its adaptive capability are presented. Section 3 describes the imitation process and finally, before concluding, section 4 details experimental results along with comprehensive discussion and analysis on positive feedback of imitation strategy in cognitive multi-robot system.

II. THE BIO-INSPIRED ARCHITECTURE

Starting from neurobiological hypothesis which highlights the importance of hippocampus in the spatial navigation, [START_REF] O'keefe | The hippocampus as a cognitive map / John O'Keefe and Lynn Nadel[END_REF], [START_REF] Burgess | Robotic and neuronal simulation of hippocampal navigation[END_REF], [START_REF] Milford | Mapping a suburb with a single camera using a biologically inspired slam system[END_REF], [START_REF] Bachelder | Mobile robot visual mapping and localization: A view-based neurocomputational architecture that emulates hippocampal place learning[END_REF] has brought to light special cells in the rat's hippocampus that fire when the animal is at a precise location. These neurons are called place cells (PC). We do not directly use them to navigate, plan or construct a map, we rather use neurons called transition cells (TC) [START_REF] Gaussier | From view cells and place cells to cognitive map learning: processing stages of the hippocampal system[END_REF]. This kind of cells is inspired by a neurobiological model of temporal sequence's learning in the hippocampus [START_REF] Alvernhe | Different CA1 and CA3 representations of novel routes in a shortcut situation[END_REF], [START_REF] Lubenov | Hippocampal theta oscillations are travelling waves[END_REF], [START_REF] Smith | Hippocampal place cells, context, and episodic memory[END_REF]. A transition cell codes a spatio-temporal transition between two PCs successively winning the competition, respectively at time t and δt. The set of the PCs and the TCs constitute a noncartesian cognitive map. The reason behind using transition cells is that their association with an action is univocal and quite straightforward. There is no need of an external algorithm to extract the action from the cognitive map. The model [START_REF] Banquet | Spacetime, order and hierarchy in fronto-hippocamal system : A neural basis of personality[END_REF], also describes the role of the hippocampus: The entorhinal cortexwhich is the main input to the hippocampus (EC) which receives signals from associative cortical areas and then filters and merges the multimodal information. In order to transfer it to CA3 pyramidal cells and dentate gyrus (DG), the DG operates an eminence between the signals and puts them together in a temporal hierarchy which later on is retransmitted on CA3 cells. This temporal hierarchy allows CA3 to be aware of past events and put them in correspondence with present events, therefore it behaves like an associative memory through stocking possible transitions between these events. The recognition of the ongoing sequence happens at the level of CA1 using EC and CA3 information. It also extends to the prefrontal cortex (PRC) to serve the higher levels of cognitive processes. Based on the same research, [START_REF] Muller | The hippocampus as a cognitive graph[END_REF] form a model of the cognitive map in the hippocampus representing the entire environment and suggest the shortest paths to a given goal. The model proposed by [START_REF] Hasselmo | Hippocampal mechanisms for the context-dependent retrieval of episodes[END_REF] provides a complete neural architecture of the learning process. His model uses a cognitive map and associates it with a mechanism of action selection. [START_REF] Martinet | Spatial Learning and Action Planning in a Prefrontal Cortical Network Model[END_REF], [START_REF] Arbib | Autonomous robots based on inspiration from biology: The relation to neuroinformatics[END_REF] also proposed a model of cognitive embedded in a parieto-fronatl network, which is based on cortical columns.

A. The cognitve map

To develop a cognitive multi-robot system we took inspiration from the model of [START_REF] Banquet | Spacetime, order and hierarchy in fronto-hippocamal system : A neural basis of personality[END_REF]. A schematic view of our architecture is shown in figure 1: To create the PC, the robot takes a visual panorama of the surrounding environment. A camera mounted on a pan system allows to perceive the environment or the surroundings. The visual images are processed to extract visual landmarks. These landmarks are learnt and a visual code is created by combining the landmarks and their corresponding azimuth. This configuration serves as a code for PCs. The signals provided by the EC are solely spatial and consistent with spatial cells' activities.

In order to select (only) the cell with the strongest response at a specific location, spatial cells' activities are submitted to a Winner-Take-All competition. We subsequently speak about the current location by indicating the spatial cell which has the highest activity at a given location. The temporal function at the level of the DG is reduced to the mere memorization the past location. The acquired association at the level of the pyramidal cells CA3 is then the transition from a location to another aside from all information concerning the time spent to carry out this transition. Once the association from the past location and the new one is learned, every new entry will reactivate the corresponding memory in the DG. During the exploration of the environment, the cognitive map is gradually created when the robot moves from one place to another. The equations governing learning in the cognitive map are given below 1 and 2:

dW CC ij (t) dt = T (t).((γ-W CC ij ).X C i (t).X C j (t)-W CC ij (t).(λ1.X C j (t)-λ2)) (1)
dW M C ij (t) dt = S(t)f ori, j = argmax k,l (X C l (t).X K M (t)) (2)
T (t) is a binary signal (0 or 1) activated when a transition is made (moving from one place to another). This signal controls the learning of recurrent connections W CC . γ is a parameter less than 1 which regulates the distribution of motivation activity on the map. λ1 and λ2 are parameters of active and passive oblivion respectively on the recurrent connections. S(t) is a signal marking the satisfaction of a goal (for example the resources discovery). This signal controls the learning of synaptic connections between neurons in W M C motivations' activity X M and neurons of the cognitive map of activity X C . After having explored the environment, the robots are in a position of predicting the locations directly reachable in each situation. Figure 2 presents an example of a cognitive map according to the perception of one robot. The environment is composed of 2 sources, 2 obstacles and 2 identical robots (robulab 10 from robosoft) that are able to avoid the stable and dynamic obstacles, to navigate, to learn and to construct their on-line own cognitive map in a unknown environment (with a threshold of vigilance to learn news places equal to 0.65 and a duration of learning equal to 30 minutes). This model has been used in Cognitive multi-robot system navigation tasks [START_REF] Chatty | Adaptation capability of cognitive map improves behaviors of social robots[END_REF] where the robots were able to build their cognitive map on-line and learn how to move towards various goals in unknown environment.

B. The selection of the shortest path

Our Cognitive robots are also able to select the shortest path [START_REF] Chatty | Adaptation capability of cognitive map improves behaviors of social robots[END_REF] in order to reach their targets: when discovering a source (food, nest or water), the motivation associated with it (thirst, hunger or stress) is associated with the cognitive map at the location where it was found. This motivation then spreads to the graph, indicating the shortest path to reach the source from any known location. We can therefore merge these motivation activities with the transition prediction activities coming from the hippocampus. The level of prediction of different transitions from the hippocampus is substantially the same. The activity derived from the cognitive map comes then to predetermine the transition choice among those that have been predicted, through selecting thereby the transition leading the fastest towards the target. A field of neurons gathers the information on the robot's direction during its entire passage of a location. It provides therefore, at the exit of a location, the direction and the length of the trajectory performed at the location. This direction is considered to be the one which is required to perform the transition. During the selection of a particular transition in order to reach the target, we can replicate, in terms of direction, the motion associated which has already been learned during the discovery of the environment.

III. THE IMITATION PROCESS

In order to share a partial knowledge of the environment, direct communication between robots could be used. However, the presented architecture is based on the stigmergy to communicate between robots. Thus, imitation seems an intersting way to strengthen this architecture. In this section, we describe a very simple architecture for imitation in a navigation perspective. The proposed model is based on dynamical interactions among mobile robots. Our aim is to provide limited capabilities to a mobile robot to interact dynamically with other robots by following their current direction of movement. Figure 3 shows the architecture of our imitation model. As shown on figure 3, perceived motions (by camera) in the visual field (of mobile robot) are estimated by a classical optical flow algorithm [START_REF] Horn | Determining optical flow[END_REF]. If the perceived motion is in upward direction, it is considered as positive activity, on the other hand, downward motion is accumulated as negative activity. Similarly, motion on the left direction leads to positive activity while movements on the right direction are considered as positive activity. Figure 4 is a snapshot taken during the experiment illustrating optical flow functioning. There are two moving objects in the field of view of a mobile robot. One moves left to right and it is transformed to negative activities by the optical flow (gray and unfilled blocks) while the other moves right to left and transformed into positive activities (shown by black color). To determine the correct direction (motion) of an interacting partner or to follow the interactor, optical flow activities are transferred to a short term memory block, this block is used to avoid the fast changes in the environment. Then, all the pixels of short term memory module are projected on the x-axis (i.e all pixels in each column are added). Than, a Winner Takes All (WTA) selects the highest activated column. This selected column indicates the direction of the movement and the robot can point and follow the interactor. For this experiment the resolution of image is 32× 24 (32 columns or location), these 32 possible locations are realized in 60 0 (-30 0 to 30 0 ) circular angles which are fed to the motor according to the corresponding columns (column zero refers to -30 0 while 32 th column corresponds to 30 0 ) and 0 0 when the agent stands in front of mobile. If two or multiple visual stimuli are present at the same time then our proposed Imitation algorithm dynamically locates and selects the interacting agent who's estimated velocity vectors are larger.

IV. EXPERIMENTATIONS, RESULTS AND DISCUSSION

Our experimental results show how imitative strategy can be useful in multi-robot system. We worked on a series of experiments to validate that the imitation strategy, which works among humans to improve the individual and social behaviors, can also be applied on a cognitive multi-robot system.

A. The analysis of the imitation behavior

In this section, we detail our experimental environment and procedures. To avoid complexities we use minimal setup which includes 2 sources, 3 obstacles and 2 robots: leader robot LR (which has already learnt the environment and the positions of both resources R1 and R2) and the imitative cognitive robot IR (which is capable of imitating and of following the other cognitive robot LR and to create its cognitive map on-line). Figure 5a, b, d ande shows that the IR tries to construct its own cognitive map by following the cognitive robot LR (when LR navigates towards the resources using its own cognitive map). Figure 5c and f demonstrate that the IR is succeeded in discovering both resources R2 and R1 during the imitation process or following the LR. Throughout the experiments, it was notable that the IR also has the capability of learning and of constructing its own cognitive map (on-line) independently, but if the same task is performed by using imitation tactic it boosts the performance (as a function of time) of discovering resources and creating a cognitive map. After discovering the positions of both resources, the IR will be able to return alone to the two resources using its own cognitive map. This experience proves the importance of imitation in multi robot system. Fig. 5. The influence of the imitation behavior in a multi-robot system. The experiment is done with two robots IR: imitator cognitive robot and a simple cognitive robot LR that has already learned the environment and the position of the resources R1 and R2. The figure is presented with chronological order and proves that thanks to imitation, the imitator robot was able to find the two resources R1 and R2. The video is available at http://perso-etis.ensea.fr/neurocyber/Videos/Cognitive Multi-Robot System /Imitation LR-IR.mov It is evident that when IR is close to the LR, its trajectory tries to follow the trajectory of LR (the two trajectories are confounded). The trajectories of the robots show that IR is able to find the two resources R1 and R2 by imitating the trajectory of LR. It is worth to notice that instead of following the complete trajectory of LR, IR imitates and follows LR only for α seconds and then stops the imitation strategy in order to detect another robot. In this case it may happen that IR do not detect LR again because LR continue to move and disappear from the visual field of IR. Since for this experiment we used only one robot (LR), IR always detects or loses the same robot.

B. The effect of imitation on the cognitive map

As the cognitive map is a way to describe the environment, we assumed that the sum of the individual cognitive maps allows to describe all the complexity of the environment. By adding a simple imitation strategy we allow the IR to share this complexity and to learn the experience of the LR to achieve its goals. Since the IR has the capability to create its cognitive map, when it follows the trajectory of the LR, its cognitive map will be influenced by the one of the LR. In order to see the difference between the two cognitive maps, we put them in the same plan (see figure 7c). It is noticeable that there is a resemblance between the two cognitive maps at the level of the PCs, TCs and the shape of the cognitive maps. Generally, in emergent collective systems, the behavior of an agent becomes a cause for the emergent property of another agent and it helps the whole system to solve a specific task. With the imitation strategy we show the same system, In our experiments, LR who had already acquired and learnt the environment, helps another robot to satisfy its motivations. Thus, adding a simple imitation strategy to our cognitive architecture allows the cognitive multi-robot system to help the individual robot.

C. The effect of imitation on the resources discovery time

Three different schemes of experiments have been tested to analyze the comparative behavior between a robot having imitative capabilities and a robot without imitation skills during the discovery of the environment and the resources (in our experiments we used 2 resources). For these experiments our main goal is to observe the time of exploration of the resources. For every experiment the robot starts with random positions and resources which are placed randomly. For the first scheme, a single robot moving randomly allows the investigation of the environment and its resources. In the second scheme, the same mobile robot along with a cognitive robot (who already knows the environment and frequently visits the resources according to its needs.) allows the discovery of the resources. The difference between this experiment and the previous one is the way of exploring and the access to resources. In the first experiment, the random robot discovers the resources without any difficulty. However, in the second experiment it is not easy to discover them because the cognitive robot (which has already learned the resources' position) acts as a dynamical obstacle. It means that when both robots are colse to a resource the random robot instead of heading towards this resource it turns in wrong direction to avoid the dynamical obstacle. In this case it loses the learning of the resource's position. Finally, in the last scheme, an imitative robot (that can follow the other robot) along with the cognitive robot allows to discover the resources and the environment. For all these three schemes we interested to compute the discovering time of both resources. For each experiment stated above, we were looking for time to find the resources and every single experiment in each category differs in time (to discover the environment). To find the average time in each category, we conducted several experiments. The number of experiments to take the average value is determined by the Fisher exact test or F test. It is a hypothesis test (statistical), it compares the difference between two variances of two sets of experiments (having the same number of observations) by taking ratio between them;

F = S 2 x /S 2 y ( 3
)
where S 2 x and S 2 y are the variances of two sets of experiments. S 2

x is the numerator and it always has the larger value between two variances. If the ratio (F ) does not exceed a certain theoretical value (available in the table of fisher), it can be accepted. On the other hand, if F is greater than the theoretical value, we reject the hypothesis of equality of two variances, in this case we increase the number of experiments to close the theoretical value of equality of two variances. For the first category, where a random robot (without imitation) looks for resources, we conducted two sets of experiments (4 experiments in each set) and computed the Fisher value (F = 5.387), it is less than 19 which is given in the table of fisher. For the second category, where a random robot explores the environment along with a cognitive robot, we conducted two sets of experiments (8 experiment in each set) and computed the Fisher value (F > 100), it is greater than 19 and unacceptable. To reduce the difference in variance, we conducted another set of experiment and computed F = 15.79, it is less than 19 (does not exceed Fisher's theoretical value). Finally, for the last category, where the imitative robot imitates the cognitive robot to find the resources, we applied two sets of experiments (8 experiments in each set) and calculated the Fisher value (F = 2.153), it is less than theoretical value [START_REF] Muller | The hippocampus as a cognitive graph[END_REF].

The average time to find two resources for each category is plotted in figure 8. It is clear from the results shown in figure 8, that the imitative robot takes less time to find both resources (R1 and R2), its accumulative time is about 5 minutes (2 min. for R1 and 3 min. for R2). However, the random robots of the first and second category take much more time (about 3 times more) to discover resources. The first category's accumulative time is 22 minutes (6 min. for R1 and 16 min. for R2) similarly, the second takes 19 minutes (6 min. for R1 and 13 min. for R2). We also have validated the positive feedback of the imitation strategy in a simulated cognitive multi-agent system based on the same bio-inspired architecture. We have shown that with imitation strategy, our cognitive agents are able to optimize the time of discovering the resources as well as increase the average number of the survivors (see figure 9). By the above stated results, we can conclude that Imitative strategy that is vital in humans for social behaviors can be equally implemented in cognitive multi-robot system for effective learning and other behaviors. We can also conclude that our bio-inspired architecture is able to maintain the same performances and results whether it is tested by simulated agents or examined by the real robots. Thus, we prove that the cognitive map aims to teach our cognitive robots the way to navigate, to reach the sources, avoid planning problems before complete exploration of the environment and adapt to a dynamic environment. Moreover, adding a simple imitation strategy to our bio-inspired architecture, allows our cognitive robots to have the same behavior as the ants algorithm (when they try to look for food) but without the need to leave physical trace in the environment.

V. CONCLUSION

In this paper, we highlighted the importance of the imitation strategy. It boosts the capability (as a function of time) of a cognitive multi-robot system to adapt to an unknown environment. It allows to solve the navigation task among various targets. We proved that combining the learning capability with simple imitation tactic leads (in a real multi-robot system) to a positive feedback in the individual and population level. Moreover, it optimizes the time to explore the resources, and it allows to create a cognitive map for the imitator robot IR (approximately the same shape as the Leader robot LR). Thus, to keep the rules of the stigmergy, imitation strategy is a better way to share the knowledge between robots without direct communication between them. As prospects, we are trying to study the effect of the imitation in cognitive multi-robot system which is able to take and withdraw objects.

Fig. 1 .

 1 Fig. 1. Model of loop hippocampo-cortical of the creation of the cognitive map

Fig. 2 .

 2 Fig. 2. From the perception of an unknown environment to the building of a cognitive map. This figure is a simplified view of the the robot's cognitive map based on transition cells. The construction process is based on: place recognition using PCs, activation of one PC and delayed activation of another allowing for transition prediction (green transition) and the need for sources (R1 and R2) which therefore triggers maximum motivation for a given goal. Figure (a) shows the real environment and figure (b) shows the cognitive map of one robot after 30 minutes of learning.

Fig. 3 .

 3 Fig. 3. Architecture to imitate or follow the other robots depending on the estimation of their velocity vectors (optical flow)

Fig. 4 .

 4 Fig. 4. Optical flow function: left side movements shows as black and realized by +ve activities while right side motion perceived as -ve activities and shown by unfilled blocks. It also shows how an imitator robot perceives the movements of the other robots (the arrows represent the robots directions).

Figure 6

 6 Figure 6 demonstrates the trajectories and the individual behavior of robots in more details. The arrows are the initial positions of the two robots. The dashed line sketches the trajectory of IR whereas trajectory of LR is shown by a continuous line. LR's trajectory shows that the robot tries to go to the position of LR when IR detects it in its visual field.It is evident that when IR is close to the LR, its trajectory tries to follow the trajectory of LR (the two trajectories are confounded). The trajectories of the robots show that IR is able to find the two resources R1 and R2 by imitating the trajectory of LR. It is worth to notice that instead of following the complete trajectory of LR, IR imitates and follows LR only for α seconds and then stops the imitation strategy in

Fig. 6 .

 6 Fig.6. The trajectories of robots to reach the resources. The cognitive map allows the LR to find the two resources. Thanks to the imitation strategy the IR is able to find the same resources when it follows the trajectory of the LR. The IR has the capability to learn the environment and the positions of the resources through creating its own cognitive map. Thus, after the first discovery of the two resources, the IR is able to return alone to the resources. Figure (a) and (b) show the discovery of the resources R1 and R2 respectively.

Figure 7 a

 7 and b show the cognitive maps of the LR and the IR respectively.

  (a) Cognitive map of the LR (b) Cognitive map of the IR (c) Resemblance of the 2 cognitive map

Fig. 7 .

 7 Fig. 7. The influence of the imitation on the creation of the cognitive map of the imitator robot. Figure (a) shows the cognitive map of the LR which already learnt the environment and the positions of both resources R1 and R2, figure (b) shows the cognitive map of IR that is created after applying the imitation strategy.Figure (c) presents the two cognitive maps in the same plan. The figure proves the positive effect of the imitation strategy which allows the IR to find the two resources and to create a cognitive map which resembles the cognitive map of the LR at the level of the PCs, TCs and the shape of the cognitive map.

  Fig. 7. The influence of the imitation on the creation of the cognitive map of the imitator robot. Figure (a) shows the cognitive map of the LR which already learnt the environment and the positions of both resources R1 and R2, figure (b) shows the cognitive map of IR that is created after applying the imitation strategy.Figure (c) presents the two cognitive maps in the same plan. The figure proves the positive effect of the imitation strategy which allows the IR to find the two resources and to create a cognitive map which resembles the cognitive map of the LR at the level of the PCs, TCs and the shape of the cognitive map.

Fig. 8 .

 8 Fig. 8. The average time of the resources discovery. The first curve (from the left) shows the time to discover the resources by a robot having imitation capability along with a cognitive robot which already knows the resources. The second curve (dotted line) describes the time taken by a robot (without imitation capability) to explore the resources along the same cognitive robot as in the previous case. The last curve demonstrates the average time needed to find the resources when single robot explores the environment randomly.

Fig. 9 .

 9 Fig. 9. The positive feedback of the imitation strategy in a simulated cognitive multi-agent system. Figure (a) shows that without imitation cognitive agent 1 is able to find the three resources (R1,R2 and R3) after 1683 time steps (5 minutes). However, in figure (b), the imitation strategy allows cognitive agent 2 to reach the resources after 505 time steps (1,5 minutes).

Here cognitive means each robot is able to build online an internal cognitive topological map of its environment.

ACKNOWLEDGMENT The authors would like to thank the financial support of the Tunisian General Direction of Scientific Research and Technological Renovation (DGRSRT), under the ARUB program 01/UR/11 02, the Institut Franc ¸ais de Tunisie, the INTERACT french project referenced ANR 09 CORD 014, the NEUROBOT french project referenced ANR-BLAN-SIMI2-L2-100617-13-01 and DIGITEO project AUTO EVAL.


Daniel Hoehener
email: hoehener@math.jussieu.fr

Geometric Approach to Second-Order Sufficient Optimality Conditions in Optimal Control *

We consider the Bolza optimal control problem with control constraints and propose new secondorder sufficient optimality conditions. These conditions are applicable under much weaker regularity assumptions than those known in the literature. In particular, the reference control may be merely measurable and the dynamics may be discontinuous in the time variable. Our approach is based on a new decomposition result for control constraints, involving second-order tangents.

Résumé

Approche géométrique des conditions d'optimalité suffisantes du second ordre pour des problèmes de contrôle optimal. Pour un problème de contrôle optimal sous forme de Bolza avec des contraintes sur les contrôles, nous proposons des nouveaux conditions d'optimalités suffisantes du second ordre. Ces conditions demandent peu de régularité des données. En particulier le contrôle de référence peut être seulement mesurable et les dynamiques peuvent être discontinues par rapport au temps. Notre approche est basée sur un nouveau résultat de décomposition pour les contraintes sur les contrôles. Cette décomposition utilise des tangents du second ordre.

Introduction

This paper is devoted to second-order sufficient optimality conditions for the following optimal control problem:

Minimize ϕ (x(1)) + 1 0 (t, x(t), u(t)) dt, (P ) over absolutely continuous x : [0, 1] → R n and measurable u : [0, 1] → R m satisfying, ẋ(t) = f (t, x(t), u(t)), u(t) ∈ U (t), a.e. x(0) = x 0 ,

where the maps f :

[0, 1] × R n × R m → R n , : [0, 1] × R n × R m → R, ϕ : R n → R, the set-valued map U : [0, 1]
R m and the initial state x 0 ∈ R n are given. Denote by W 1,1 ([0, 1]; R n ) the space of absolutely continuous mappings. The set of all measurable mappings u : [0, 1] → R m such that u(t) ∈ U (t) a.e. will be denoted by U. A process (x, u) comprises a control function u ∈ U and a state trajectory x ∈ W 1,1 ([0, 1]; R n ) satisfying the differential equation in [START_REF] Aubin | Set-Valued Analysis, Systems Control Found[END_REF]. Under standard assumptions the state trajectory x is uniquely defined by a control function u and the control system in [START_REF] Aubin | Set-Valued Analysis, Systems Control Found[END_REF]. Consequently, the cost functional J : U → R,

J(u) := ϕ (x u (1)) + 1 0 (t, x u (t), u(t)) dt, (2)
where

x u ∈ W 1,1 ([0, 1]; R n
) is the solution of (1) corresponding to the control u, is well defined.

Definition 1.1 A process (x, ū) is a (strict) weak local minimizer if there exists ρ > 0 such that ū minimizes J (strictly) over all ũ ∈ U satisfying ũ -ū ∞ ≤ ρ.
In this note we present a new approach to second-order sufficient conditions. We consider set-valued maps U (•) where the boundary of U (t) is smooth. Then we use the geometry of the sets U (t) in order to show that there exists a neighborhood of the reference control ū, such that for every ũ ∈ U in this neighborhood, one can approximate ũ-ū sufficiently well by first-and second-order tangents. Unlike in [START_REF] Bonnans | Second-order analysis for optimal control problems with pure state constraints and mixed control-state constraints[END_REF], [START_REF] Maurer | Second order sufficient conditions for optimal control problems with free final time: The Riccati approach[END_REF], [START_REF] Zeidan | The Riccati equation for optimal control problems with mixed state-control constraints: necessity and sufficiency[END_REF], where the sufficient conditions were obtained by using abstract results, our approach permits a direct and geometric proof. So far direct methods required the reference control ū to be continuous or piecewise continuous, see for instance [START_REF] Milyutin | Calculus of Variations and Optimal Control[END_REF]. With our approach we are able to state sufficient conditions when ū is merely measurable. Moreover the dynamics can be discontinuous in the time variable and no special structure of the Hamiltonian is required. In particular we do not impose a Legendre-Clebsch condition. Our sufficient optimality conditions form no-gap second-order optimality conditions with the necessary optimality conditions of [START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF]Thm. 3.2]. Finally, this approach can be extended to sets U (t) having nonsmooth boundary.

Notations and assumptions

As usual B(x, ρ) stands for the open ball with center x ∈ R n and radius ρ > 0. ∂K denotes the boundary of a set K ⊂ R n and T K (x) the Bouligand tangent cone to K at x, see [START_REF] Aubin | Set-Valued Analysis, Systems Control Found[END_REF]. Throughout the paper (x, ū) ∈ W 1,1 ([0, 1]; R n ) × U denotes a fixed reference process. In order to simplify the notations, we will abbreviate (t, x(t), ū(t)) by [t], thus for instance f [t] := f (t, x(t), ū(t)). Furthermore we define,

T B := {t ∈ [0, 1] | ū(t) ∈ ∂U (t)} .
For a map φ : [0, 1]×R n ×R m → R k we denote by φ x (t 0 , x 0 , u 0 ), φ u (t 0 , x 0 , u 0 ), φ xx (t 0 , x 0 , u 0 ), φ xu (t 0 , x 0 , u 0 ), φ uu (t 0 , x 0 , u 0 ) its partial-and second-order partial derivatives with respect to x and/or u at (t 0 , x 0 , u 0 ), assuming these derivatives exist. The derivative, resp. the Hessian, of the map (x, u) → φ(t, x, u) evaluated at (x 0 , u 0 ) is denoted by φ (t, x 0 , u 0 ), resp. φ (t, x 0 , u 0 ). We require the following regularity of the data: There exists ρ 0 > 0 such that,

(A1) (a) ∀x ∈ R n , ∀u ∈ R m , f (•, x,

  u) and (•, x, u) are measurable and for almost all t ∈[0, 1], f (t, •, •), (t, •, •) are continuously differentiable on B(x(t), ρ 0 )×B(ū(t), ρ 0 ). Moreover, ϕ ∈ C 2 (R n ; R); (b) ∀R > 0, there exists k R ∈ L 1 ([0, 1]; R + ) such that for a.e. t ∈ [0, 1], f (t, •, u) and (t, •, u) are k R(t)-Lipschitz on RB for all u ∈ B(ū(t), ρ 0 );

(c) There exists γ ∈ L 1 ([0, 1]; R + ) such that for a.e. t ∈ [0, 1] and all x ∈ R n ,

|f (t, x, u)| ≤ γ(t)(1 + |x|), ∀u ∈ B(ū(t), ρ 0 ); (d) ∀R > 0, the mapping t → sup (x,u)∈RB | (t, x, u)| is integrable on [0, 1];

* Financial support by the European Commission (FP7-PEOPLE-2010-ITN, Grant Agreement no. 264735-SADCO) is gratefully acknowledged

(A2) f x [•] is integrable and ∃a 1 > 0 s.t. for a.e. t ∈ [0, 1], ∀x, y ∈ B(x(t), ρ 0 ), ∀u, v ∈ B(ū(t), ρ 0 ), f u [t] ≤ a 1 and f (t, x, u) -f (t, y, v) ≤ a 1 (|x -y| + |u -v|);

(A3) (a) For almost all t ∈ [0, 1], f (t, •, •) is twice differentiable on B(x(t), ρ 0 ) × B(ū(t), ρ 0 );

(b) ∃a 2 > 0 such that for a.e. t ∈ [0, 1], ∀x, y ∈ B(x(t), ρ 0 ), ∀u, v ∈ B(ū(t), ρ 0 ), f [t] ≤ a 2 and f (t, x, u) -f (t, y, v) ≤ a 2 (|x -y| + |u -v|);

(A4) For all t ∈ [0, 1], U (t) := {u ∈ R m | g(t, u) ≤ 0}, where the function g

) is measurable for all u ∈ R m and there exists K g > 0 such that for a.e. t ∈ [0, 1], g(t, •) is twice continuously differentiable on B(ū(t), ρ 0 ) and for all u ∈ B(ū(t), ρ 0 ), When (A2) or (A3) are imposed, then we always assume that satisfies the same assumptions as f . Note that if (A4)(a) is satisfied, then one may also parameterize

, where d(x; K) := inf k∈K |x -k|. In this case (A4)(b) is automatically verified.

Main results

Define the Hamiltonian

Theorem 3.1 Let (x, ū) be a weak local minimizer of problem (P ) and (A1), (A2), (A4) hold true. Then there exists p

) is a process and p is the corresponding adjoint state. For a given extremal (x, ū, p), we will use the abbreviation [t] := (t, x(t), p(t), ū(t)) when evaluating the Hamiltonian or its (partial) derivatives at (t, x(t), p(t), ū(t)).

For any

is called the solution of the linearized system corresponding to w.

Having this, one uses the variational analysis performed in [START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF] in order to conclude.

Remark 2 By (A4) and Theorem 3.1 we have that for a.e.

Thus, in the case when there exists γ 1 > 0 such that,

here λ(•) is the Lebesgue measure, (4) becomes the classical sufficient condition of [START_REF] Maurer | Second order sufficient conditions for optimal control problems with free final time: The Riccati approach[END_REF]. Note that in this paper the authors impose a strengthened Legendre-Clebsch condition and assume that (A4)(b) holds on a neighborhood of T B , in order to obtain these results for an essentially bounded ū. In [START_REF] Zeidan | The Riccati equation for optimal control problems with mixed state-control constraints: necessity and sufficiency[END_REF] the same assumptions as in [START_REF] Maurer | Second order sufficient conditions for optimal control problems with free final time: The Riccati approach[END_REF] are made and in addition it is required that (5) is satisfied. Finally, in [START_REF] Bonnans | Second-order analysis for optimal control problems with pure state constraints and mixed control-state constraints[END_REF] the authors consider also pure state constraints but they impose an even stronger version of the strengthened Legendre-Clebsch condition. That is they need that there exists γ > 0 such that for almost all t ∈ [0, 1],