To formulate the autonomous driving problem as a Markovian Decision Process, one must rst describe the state of the system and its encoding in a space S. We list the main state representations found in the literature in sections 1 and 2. Then, one must dene a set of actions A that gives the agent control over the state dynamics, which we explore in section 3.

1 Representing the vehicles 1.1 Encodings 1.1.1 Continuous coordinates A vehicle driving on a road can be described in the most general way by it's continuous position, heading and velocity.

s = x y ψ v T (1)
The composite state (or joint-state) of a road trac with one ego-vehicle (denoted X 0 ) and N other vehicles can then be described by the set of the states of all vehicles.

s = {s k } k∈[0,N ] (2)
The reference frame can be absolute, but as the behaviour followed by the ego-vehicle should be the same at any given location and only depend on the relative position of entities around it, it is common to use an ego-centric reference frame. It allows to concentrate the distribution of visited states around the origin in both position, heading and velocity space, as other vehicles are often close to the ego-vehicle and with similar speed and heading. This reduces the region of state-space in which the policy must perform. The size of this state-space is R 4(N +1) . This representation is used in [START_REF] Forbes | The BATmobile: Towards a Bayesian Automated Taxi[END_REF][START_REF] Wheeler | A Probabilistic Framework for Microscopic Trac Propagation[END_REF][START_REF] Bai | Intention-Aware Online POMDP Planning for Autonomous Driving in a Crowd[END_REF][START_REF] Gindele | Learning driver behavior models from trac observations for decision making and planning[END_REF][START_REF] Song | Intention-Aware Autonomous Driving Decision-Making in an Uncontrolled Intersection[END_REF][START_REF] Sunberg | The value of inferring the internal state of trac participants for autonomous freeway driving[END_REF][START_REF] Paxton | Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments[END_REF][START_REF] Lee | A Learning-Based Framework for Handling Dilemmas in Urban Automated Driving[END_REF][START_REF] Shalev-Shwartz | On a Formal Model of Safe and Scalable Self-driving Cars[END_REF][START_REF] Galceran | Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment[END_REF][START_REF] Yu Fan Chen | Socially Aware Motion Planning with Deep Reinforcement Learning[END_REF][START_REF] Paxton | Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments[END_REF].

In some cases, this representation is used only for memory storage though the surrounding vehicles are note actually part of the optimized state, but rather used to dene constraints or penalties regarding the ego-vehicle state. See, for instance, [START_REF] Levine | Continuous Inverse Optimal Control with Locally Optimal Examples[END_REF]; [START_REF] Ziegler | Trajectory planning for Bertha -A local, continuous method[END_REF]; [START_REF] Qian | Optimal trajectory planning for autonomous driving integrating logical constraints: An MIQP perspective[END_REF]; [START_REF] Sadigh | Planning for Autonomous Cars that Leverage Eects on Human Actions[END_REF].

Discrete coordinates

To reduce the size of the state-space, any continuous variable z ∈ R of the state can be quantized to its closest value within a discrete and often nite set Z = {z i }.

This shrinks the state-space from R (N +1) to a discrete set of size |I| 4(N +1) , where I is the union quantization of the state-space of a single vehicle. Most of the time, the quantization is chosen uniform for simplicity, as in [START_REF] Gómez Plaza | Integration of cell-mapping and reinforcement-learning techniques for motion planning of car-like robots[END_REF][START_REF] Brechtel | Probabilistic MDP-behavior planning for cars[END_REF][START_REF] Du | A POMDP Approach to Robot Motion Planning under Uncertainty[END_REF][START_REF] Bandyopadhyay | Intention-Aware Motion Planning[END_REF][START_REF] Loiacono | Learning to overtake in TORCS using simple reinforcement learning[END_REF][START_REF] Osipychev | Proactive MDP-based collision avoidance algorithm for autonomous cars[END_REF]Rehder et al., 2017a).

However, it is often the case with uniform quantization that the state-space either remains too large or becomes too coarse. To address this issue, [START_REF] Tehrani | General behavior and motion model for automated lane change[END_REF] adapt the size of the grid cells with the relative speed of each other vehicles, while [START_REF] Brechtel | Probabilistic Decision-Making under Uncertainty for Autonomous Driving using Continuous POMDPs[END_REF] suggest to automatically learn a sucient and ecient discrete partition of the continuous space for a given task.

Spatial grid

The two encodings described so far are ecient in the sense that they use the smallest quantity of information necessary to represent the scene. However, they lack two important properties:

Permutation invariance

We expect a driving policy not to be dependent on the order in which all vehicles in the trac are listed. Ideally, this property should derive naturally from an architectural design and not rely only on data augmentation to cover the N ! possible permutations of any given trac state. That is, if we denote the trac state representation as s = (X 1 , .., X N ) and the policy as π(•|s), we require that

π(•|(s 1 , . . . , s N )) = π(•|(s σ(1) , . . . , s σ(N ) )) ∀σ ∈ S N (3)
This desired property can be implemented within the policy architecture, as done in [START_REF] Yu Fan Chen | Socially Aware Motion Planning with Deep Reinforcement Learning[END_REF] or [START_REF] Qi | PointNet: Deep Learning on Point Sets for 3D Classication and Segmentation[END_REF], but also directly in the state representation.

Dependency on the number of vehicles

In theses formalizations, the size of the state depends of the number N of vehicles considered. For the sake of function approximation which often expects constantsized inputs, and in order to avoid having a growing computational complexity when more vehicles are on the road, we may wish to get rid of this dependency.

These limitations are addressed by the occupancy grid representation, that uses a different approach for representing a quantity z localized in a space X. Instead of explicitly representing spatial dimensions as variables x within a state {s k = (x k , z k )} k∈[0,N ] indexed on the vehicles, they are represented implicitly through the layout of several variables z i organized in a grid-like structure indexed on a quantization of the space X.

X = ⊕ i∈I X i ( 4
)
s i = z k if ∃k ∈ [1, N ] s.t. x k ∈ X i 0 else (5)
The z variable often corresponds to mere presence information (0-1) but can also include additional channels such as heading and velocity.

The size of this state space is then |Z| |I| . This representation is used in [START_REF] Mukadam | Tactical Decision Making For Lane Changing With Deep Reinforcement Learning[END_REF][START_REF] Isele | Navigating Intersections with Autonomous Vehicles using Deep Reinforcement Learning[END_REF][START_REF] Fridman | DeepTrac: Driving Fast through Dense Trac with Deep Reinforcement Learning[END_REF].

Coordinates systems 1.2.1 Cartesian coordinates

In most cases, the vehicles locations are expressed in a Cartesian coordinates system, as seen in 1.1.

Polar coordinates

By using a coordinate system polarized at the ego-vehicle, the scene becomes explicitly described from its point of view. This scheme is consistent with the data format of many sensors in the autonomous driving industry, such as LIDARs and radars. Angular sector indexing Following the grid representation introduced in 1.1.3, instead of directly indexing on the dierent vehicles, one can bin them according to the (discretized) angular sector they belong in. By keeping a constant number of vehicle described in each angular sector (usually only one: the closest), the state representation size becomes independent on the number of vehicles, at the expense of removing some of the vehicles from the state, usually the further ones. This representation is used in [START_REF] Hadsell | Learning long-range vision for autonomous o-road driving[END_REF][START_REF] Cardamone | Evolving Competitive Car Controllers for Racing Games with Neuroevolution[END_REF][START_REF] Sharifzadeh | Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks[END_REF][START_REF] Kueer | Imitating Driver Behavior with Generative Adversarial Networks[END_REF]Pfeier et al., 2017a;[START_REF] Graf | Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing[END_REF][START_REF] Raunak | Multi-Agent Imitation Learning for Driving Simulation[END_REF] 1.2.3 Lane-centric coordinates But one can observe that the tactical-level decision making policy should also be the same on a straight or curved road, and that only low-level motion planning needs to take the shape of the road into account. Hence, instead of the Euclidean coordinate system, a lane-centric coordinate system can be used where each vehicle is described in terms of its current lane and Frenet coordinate within this lane. This representation is used in [START_REF] Coulom | Apprentissage par renforcement utilisant des réseaux de neurones, avec des applications au contrôle moteur[END_REF] for car racing, in [START_REF] Paxton | Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments[END_REF][START_REF] Wang | Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge[END_REF] for highway driving scenarios, and in (Shalev-Shwartz et al., 2016a) for a roundabout insertion scenario. In [START_REF] Riedmiller | Learning to drive a real car in 20 minutes[END_REF], the track-coordinate system is used to represent not only the vehicle's position but also heading and yaw rate, which are relative the the lane curve heading and yaw rate at the projected position.

Lane indexing Again, a grid encoding can be chosen so as to index directly the lanes on the road and describe where the vehicles are located in each lane. The state size can be made independent of the number of vehicles in the scene by keeping a constant nite number of vehicles in each lane (e.g. front, middle, rear).

This representation is used in [START_REF] Abbeel | Apprenticeship learning via inverse reinforcement learning[END_REF][START_REF] Wei | A Point-based Markov Decision Process for Robust Single-Lane Autonomous Driving Behavior under Uncertainties[END_REF][START_REF] Levine | Nonlinear Inverse Reinforcement Learning with Gaussian Processes[END_REF][START_REF] Ulbrich | Probabilistic online POMDP decision making for lane changes in fully automated driving[END_REF][START_REF] Wang | Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge[END_REF][START_REF] Altché | An LSTM Network for Highway Trajectory Prediction[END_REF][START_REF] Raunak | Multi-Agent Imitation Learning for Driving Simulation[END_REF], and is studied in [START_REF] Chen | DeepDriving: Learning Aordance for Direct Perception in Autonomous Driving[END_REF] where the authors refer to it as a direct perception approach that leverages aordance indicators of the road situation. [START_REF] Li | Game-Theoretic Modeling of Driver and Vehicle Interactions for Verication and Validation of Autonomous Vehicle Control Systems[END_REF] uses a quantization of positions in the lanes as close/nominal/far and of velocities as approaching/stable/moving away. Likewise, Wray et al. ( 2017) use a semantic quantization tailored for handling intersections: approaching/at/edged/inside/ empty.

Camera images

The information of presence and location of other vehicles can also be encoded directly through a raw camera image.

An image can be taken from dierent viewpoints, such as a top-view camera like in [START_REF] Liu | Elements of Eective Deep Reinforcement Learning towards Tactical Driving Decision Making[END_REF], or a front-view camera like in [START_REF] Mnih | Human-level control through deep reinforcement learning[END_REF] (see Enduro), and also represented in dierent spaces (RGB, HSV, etc.).

In this setting, almost no preprocessing of the raw data is needed to obtain the state representation. However, the price to pay for this lack of abstraction is high-dimensionality.

A transition model is also rarely available for planning as it requires solving the dicult task of video prediction. [START_REF] Finn | Deep visual foresight for planning robot motion[END_REF] is one of the few attempts to learn a predictive model of future video frames and use it for motion planning.

Other features 1.4.1 Intentions

The future trajectory of a vehicle often depends on the internal intentions of its driver. In order to model this dynamics in the Markov Decision Process framework, the intentions must be made part of the vehicle state, even though they are rarely directly observable. [START_REF] Bai | Intention-Aware Online POMDP Planning for Autonomous Driving in a Crowd[END_REF] and [START_REF] Bandyopadhyay | Intention-Aware Motion Planning[END_REF] both add to the state of each agent in the scene an unobserved discrete intention, representing their desired goal location coordinates. In [START_REF] Song | Intention-Aware Autonomous Driving Decision-Making in an Uncontrolled Intersection[END_REF], semantic goal locations are used instead of coordinates to represent agents intentions at intersections, among {straight, left, right, stop}.

Instead of the mere destination, other properties of the agents decision process can also be represented, such as their politeness and aggressivity in [START_REF] Sunberg | The value of inferring the internal state of trac participants for autonomous freeway driving[END_REF].

Even entire behaviours executed by the agents can be listed and represented in the state.

This is what [START_REF] Driggs | Identifying Modes of Intent from Driver Behaviors in Dynamic Environments[END_REF] call intent modes and implement with the examples of lane keeping, preparing to lane change, and lane changing. [START_REF] Forbes | The BATmobile: Towards a Bayesian Automated Taxi[END_REF] suggest to use Bayesian inference to predict these modes of behaviours based on the observations (e.g. predict a lane change when observing a blinking turn signal), just like [START_REF] Galceran | Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment[END_REF] who focus on detecting changepoints between several policies. Finally, the other agents intentions can be modelled with a high degree of expressive power as an objective function, under the hypothesis that the agents are rational and execute near-optimal policies. [START_REF] Sadigh | Planning for Autonomous Cars that Leverage Eects on Human Actions[END_REF] and [START_REF] Huang | Enabling Robots to Communicate their Objectives[END_REF] show that this approach can be used in pair with a dynamics model to predict future human actions, which yields a closed loop predictive model.

1.4.2 Lateral features [START_REF] Ulbrich | Probabilistic online POMDP decision making for lane changes in fully automated driving[END_REF] append to the state additional rule-based features, such as whether performing a lane change now is possible, and whether it is benecial. However, it is more common to see this information stored in the set of actions available in the state and their action-values, like in [START_REF] Mukadam | Tactical Decision Making For Lane Changing With Deep Reinforcement Learning[END_REF] or [START_REF] Liu | Elements of Eective Deep Reinforcement Learning towards Tactical Driving Decision Making[END_REF] where a heuristic rule-based action masker lters out apparently unsafe actions.

Longitudinal features

Driving at high speed into a distant obstacle is quite similar to driving at low speed into a close obstacle. In both situation, what matters for decision making is the time-to-collision (TTC): the decision of braking or steering out of the way has to be taken before the TTC is lower than the duration of these actions.

The longitudinal time-to-collision is dened for all i ∈ [1, N ] as

τ i = - x i -x 0 v i -v 0 (6)
Another related indicator is the time gap, dened as the time needed for the ego-vehicle to reach a vehicle's current position.

τ i = - x i -x 0 v 0 (7)
These features are used in [START_REF] Driggs | Identifying Modes of Intent from Driver Behaviors in Dynamic Environments[END_REF][START_REF] Ulbrich | Probabilistic online POMDP decision making for lane changes in fully automated driving[END_REF][START_REF] Isele | Navigating Intersections with Autonomous Vehicles using Deep Reinforcement Learning[END_REF][START_REF] Altché | An LSTM Network for Highway Trajectory Prediction[END_REF][START_REF] Liu | Elements of Eective Deep Reinforcement Learning towards Tactical Driving Decision Making[END_REF][START_REF] Raunak | Multi-Agent Imitation Learning for Driving Simulation[END_REF].

Full trajectory

Instead of only considering the current timestep in the state representation, one can decide to work directly at the scale of entire trajectories. Thus, [START_REF] Kretzschmar | Learning to predict trajectories of cooperatively navigating agents[END_REF][START_REF] Kretzschmar | Socially Compliant Mobile Robot Navigation via Inverse Reinforcement Learning[END_REF]; [START_REF] Pfeier | Predicting actions to act predictably: Cooperative partial motion planning with maximum entropy models[END_REF] use cubic splines to represent the future trajectory of each moving entity. However, this representation is not suited for sequential decision making but rather for direct optimization of the objective over the trajectory representation. The main advantage of this approach is the ability to work in continuous time instead of discrete time, wich can be very useful to access innitesimal variations of the kinematics, such as the instantaneous acceleration or jerk, which are relevant in the assessment of comfort. This is the case of [START_REF] Kuderer | Learning Driving Styles for Autonomous Vehicles from Demonstration[END_REF] who use this framework to infer comfortable driving styles from human demonstrations.

Representing the environment

The policy needs to take into account its static environment: the location of the obstacles, of the drivable space, and information regarding trac laws.

Cartesian coordinates

To represent the location of obstacles on the road around the ego-vehicle, a rst approach is simply to list each obstacle and store their location and geometry (spherical, parallelepiped, etc.) in a tuple.

This representation is compact in the sense that free space is not stored in memory. However, it has varying size and lacks spatial structure in the data structure.

It is used by [START_REF] Awais | Obstacle Avoidance in Real Time with Nonlinear Model Predictive Control of Autonomous Vehicles[END_REF].

Cartesian occupancy grid

To introduce spacial structure and xed size in the representation of obstacles, the same trick as in 1.1.3 can be used: convert the tuple of coordinates into a spatial grid. This representation is used by [START_REF] Ziebart | Planningbased Prediction for Pedestrians[END_REF][START_REF] Richter | High-Speed Autonomous Navigation of Unknown Environments using Learned Probabilities of Collision[END_REF]Chen et al., 2016;[START_REF] Tamar | Value Iteration Networks[END_REF][START_REF] Shankar | Reinforcement Learning via Recurrent Convolutional Neural Networks[END_REF][START_REF] Wulfmeier | Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environments[END_REF][START_REF] Sallab | Deep Reinforcement Learning framework for Autonomous Driving[END_REF]Rehder et al., 2017a;[START_REF] Hoermann | Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling[END_REF][START_REF] Williams | Information Theoretic MPC for Model-Based Reinforcement Learning[END_REF][START_REF] Mukadam | Tactical Decision Making For Lane Changing With Deep Reinforcement Learning[END_REF][START_REF] Rhinehart | Deep Imitative Models for Flexible Inference, Planning, and Control[END_REF].

The representation choice can be dierent for dynamic and static obstacles, as shown by Pfeier et al. (2017a) who use a Cartesian grid to encode the static environment and a polar grid for the location of pedestrians.

Polar occupancy grid

Just like in 1.2.2, the polar coordinate system can also be used to describe the environment. Thus, Pfeier et al. (2017b); [START_REF] Koutník | Evolving Large-Scale Neural Networks for Vision-Based TORCS[END_REF]; [START_REF] Manuelli | Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR[END_REF]; Plessen (2017) and [START_REF] Trehard | On line mapping and global positioning for autonomous driving in urban environment based on evidential SLAM[END_REF] describe the shape of the road as ranges to obstacles in a set of angular sectors.

This representation is very close to the natural data structure produced by LIDAR sensors or depth cameras, that measure the distance along rays in every direction.

Camera images

As mentioned in 1.3, when using raw camera images we benet from a very rich information feed. It contains information about neighbour vehicles, but also concerning the shape of the road, the location of the lanes and static obstacles. Hence, it can be used as a simple and rich representation of the environment.

The most commonly-occuring setting is the front-view camera, used in [START_REF] Dean | ALVINN, an autonomous land vehicle in a neural network[END_REF][START_REF] Cardamone | Evolving Competitive Car Controllers for Racing Games with Neuroevolution[END_REF][START_REF] Ross | A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning[END_REF][START_REF] Koutník | Evolving Large-Scale Neural Networks for Vision-Based TORCS[END_REF][START_REF] Bojarski | End to End Learning for Self-Driving Cars[END_REF][START_REF] Yu Fan Chen | Motion Planning with Diusion Maps[END_REF][START_REF] Xu | End-to-end Learning of Driving Models from Large-scale Video Datasets[END_REF][START_REF] Sallab | End-to-End Deep Reinforcement Learning for Lane Keeping Assist[END_REF][START_REF] Hesham | End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies[END_REF][START_REF] Koppula | Learning a CNN-based End-to-End Controller for a Formula Racecar[END_REF][START_REF] Sallab | Deep Reinforcement Learning framework for Autonomous Driving[END_REF][START_REF] Codevilla | End-to-end Driving via Conditional Imitation Learning[END_REF]Rehder et al., 2017c;[START_REF] Rezagholizadeh | Semi-supervised Regression with Generative Adversarial Networks for End to End Learning in Autonomous Driving[END_REF].

But the images can also come frome a top-view camera, as seen in [START_REF] Bagnell | Learning for autonomous navigation[END_REF]Rehder et al., 2017b,c).

Instead of using the raw images, a preprocessing step can be considered, usually for the sake of semantic segmentation to identify the drivable area of the image, like in [START_REF] Hadsell | Learning long-range vision for autonomous o-road driving[END_REF][START_REF] Barnes | Find your own way: Weakly-supervised segmentation of path proposals for urban autonomy[END_REF]. Another objective of preprocessing can also be the transfer of skills learnt in simulation to real use cases, as [START_REF] Pan | Virtual to Real Reinforcement Learning for Autonomous Driving[END_REF] who use style transfer to convert the simulator images distribution to a realistic images distribution that is fed to the driving policy during its training, so as to improve robustness in real usage. Sometimes, this preprocessing is not used as a transformation of the observation but rather as an auxiliary task. Thus, Eraqi et al. ( 2017) performs semantic segmentation of camera images and shows that it improves performance on the main task of steering angle regression.

Finally, camera images being high-dimensional observations, compressed representations can be learnt explicitly through the use of unsupervised learning and auto-encoders, such as in (Ha and Schmidhuber, 2018;[START_REF] Kendall | Learning to Drive in a Day[END_REF].

Road structure

The knowledge of the road network provides meaningful information. For instance, [START_REF] Van Den | Temporal exploration for reinforcement learning in continuous action spaces[END_REF] describes the track curve through its orientation, width and lateral oset with respect to a nite set of positions along a planned route. [START_REF] Se | Learning from Maps: Visual Common Sense for Autonomous Driving[END_REF] extract from camera images features describing the road structure, such as the drivable directions and distance to intersections.

However, it is rarely included directly in the state-space. [START_REF] Liniger | Optimization-based autonomous racing of 1:43 scale RC cars[END_REF]; [START_REF] Song | Intention-Aware Autonomous Driving Decision-Making in an Uncontrolled Intersection[END_REF][START_REF] Hollins Wray | Online decision-making for scalable autonomous systems[END_REF] use it as static reference information that denes constraints on the state-space, such as forbidden error states. In [START_REF] Shalev-Shwartz | On a Formal Model of Safe and Scalable Self-driving Cars[END_REF], the road geometry is used to assess responsibility in a potential collision, and to forbid only collision states where the blame is on the ego-vehicle.

More commonly, it is used to dene the reward function. For instance, more reward can be assigned to driving in some parts of the road, typically along a known navigation route and in the center of the lane, as in [START_REF] Levine | Continuous Inverse Optimal Control with Locally Optimal Examples[END_REF][START_REF] Liu | Elements of Eective Deep Reinforcement Learning towards Tactical Driving Decision Making[END_REF]. Likewise [START_REF] Abbeel | Apprenticeship Learning for Motion Planning, with Application to Parking Lot Navigation[END_REF]; [START_REF] Ziebart | Navigate like a cabbie: Probabilistic reasoning from observed context-aware behavior[END_REF]; [START_REF] Stiller | 3D perception and planning for self-driving and cooperative automobiles[END_REF]; [START_REF] Gindele | Learning driver behavior models from trac observations for decision making and planning[END_REF] represent the road network as a graph and and use it to evaluate the cost of the ego-vehicle's planned trajectory. Road boundaries can also be described by curve equations, such as in [START_REF] Williams | Robust Sampling Based Model Predictive Control with Sparse Objective Information[END_REF].

The downside of excluding road information from the state is that at test time the policy cannot rely on this information, so the optimal policy is tailored for a specic road conguration and has to be computed again whenever it changes. As a consequence, the trajectory optimization often has to be performed online in this setting.

Trac laws

In addition to the road structure, trac laws can also be encoded in the state representation. For instance, [START_REF] Paxton | Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments[END_REF] adds in the state-space some features describing the current lane's speed limit, whether vehicle has entered a stop region or has the right of way. These properties can be extracted from camera images in a perception module, like in [START_REF] Se | Learning from Maps: Visual Common Sense for Autonomous Driving[END_REF] that learns to dierentiate between one-way vs two-way roads and whether the vehicle is driving in the wrong way.

Again, these trac laws can be used as reward rather than directly in the state. [START_REF] Liu | Elements of Eective Deep Reinforcement Learning towards Tactical Driving Decision Making[END_REF] uses them to penalize entering intersections when the trac lights are red, or driving in opposite or biking lanes. As the policy has no access to this information at inference time, it has to be optimized in an online manner to stay consistent with the evolution of these data.

The action space

We now study the dierent action spaces used in the driving policy literature.

Continuous actions

When driving a car, there are only a few actuators to consider: the steering wheel angle, the acceleration and brake pedals, and the gearbox. The dierent pedals and gears are often merged into a single acceleration command for simplicity.

Hence the canonic continuous action-space is composed of the longitudinal acceleration and steering angle, and often used for low-level control tasks, such as [START_REF] Sadigh | Planning for Autonomous Cars that Leverage Eects on Human Actions[END_REF][START_REF] Cardamone | Evolving Competitive Car Controllers for Racing Games with Neuroevolution[END_REF][START_REF] Ross | A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning[END_REF][START_REF] Levine | Continuous Inverse Optimal Control with Locally Optimal Examples[END_REF][START_REF] Garcia | Safe exploration of state and action spaces in reinforcement learning[END_REF][START_REF] Koutník | Evolving Large-Scale Neural Networks for Vision-Based TORCS[END_REF][START_REF] Van Den | Temporal exploration for reinforcement learning in continuous action spaces[END_REF].

Higher-level representations can also be used if we ignore some parts of the dynamics, as showed by [START_REF] Yu Fan Chen | Socially Aware Motion Planning with Deep Reinforcement Learning[END_REF] who act directly with longitudinal and lateral velocities.

Finally, only one of these two actions can be considered if we assume that the other one is chosen according to a separate policy. For instance, [START_REF] Hester | TEXPLORE: Real-Time Sample-Ecient Reinforcement Learning for Robots[END_REF]; Shalev-Shwartz et al. (2016a) only consider the choice of continuous acceleration while assuming that a lane keeping lateral controller is available and independently sets the steering angle.

Discrete actions

To reduce complexity and accelerate the policy optimization, it is common to prefer a discrete action-space to a continuous one. From there, it is straight-forward to discretize the original continuous action-space by binning ranges of actions together.

The binning is often chosen uniform for simplicity, like in [START_REF] Isele | Navigating Intersections with Autonomous Vehicles using Deep Reinforcement Learning[END_REF][START_REF] Larry | Learning to Race: Experiments with a Simulated Race Car[END_REF][START_REF] Gómez Plaza | Integration of cell-mapping and reinforcement-learning techniques for motion planning of car-like robots[END_REF].

It is often the case that only a small number of possible actions are chosen. For instance, [START_REF] Bandyopadhyay | Intention-Aware Motion Planning[END_REF]; [START_REF] Bai | Intention-Aware Online POMDP Planning for Autonomous Driving in a Crowd[END_REF]; [START_REF] Song | Intention-Aware Autonomous Driving Decision-Making in an Uncontrolled Intersection[END_REF] use only three possible acceleration values: {decelerate a = -α, maintain velocity a = 0, accelerate a = +α}.

However, a uniform binning often suers from being either too coarse or too highdimensional. As the distribution of steering angle is heavily concentrated around the center, [START_REF] Xu | End-to-end Learning of Driving Models from Large-scale Video Datasets[END_REF] suggest to perform the binning in a log-space or even according to the data distribution, in order to have a ne sampling only around frequent actions.

Again, only one dimension can be considered. [START_REF] Brechtel | Probabilistic Decision-Making under Uncertainty for Autonomous Driving using Continuous POMDPs[END_REF]; [START_REF] Bandyopadhyay | Intention-Aware Motion Planning[END_REF]; [START_REF] Bai | Intention-Aware Online POMDP Planning for Autonomous Driving in a Crowd[END_REF]; [START_REF] Song | Intention-Aware Autonomous Driving Decision-Making in an Uncontrolled Intersection[END_REF] focus on choosing the longitudinal acceleration while assuming that a lateral control tracks a pre-planned trajectory, like simple lane keeping or following the route at intersections.

As noted by [START_REF] Sallab | End-to-End Deep Reinforcement Learning for Lane Keeping Assist[END_REF] who compares the continuous and discrete settings, the discretization implies the introduction of discontinuities in the commands, which can lead to instability and jerky trajectories. One way to adress this is to asume that discrete actions are all abstract and imply an underlying smooth continuous control policy. Alternatively we can use commands corresponding to higher order derivatives of the dynamics, so as to benet from the smoothing properties of the integration in the controlled system. This is the case of [START_REF] Riedmiller | Learning to drive a real car in 20 minutes[END_REF][START_REF] Gindele | Learning driver behavior models from trac observations for decision making and planning[END_REF][START_REF] Manuelli | Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR[END_REF] and [START_REF] Huang | Enabling Robots to Communicate their Objectives[END_REF] who use the steering rate or heading rate instead of the steering angle to generate smoother trajectories.

Temporal abstraction

The actions presented so far are commands aecting the car dynamics, which means they must be updated at a high frequency, in the order of 10Hz. In (Shalev-Shwartz and Shashua, 2016;Shalev-Shwartz et al., 2016b) and [START_REF] Shalev-Shwartz | On a Formal Model of Safe and Scalable Self-driving Cars[END_REF], the authors argue that due to this dense time resolution of decision making, the estimation of the value function faces a very small signal-to-noise ratio as its variance grows linearly with the time horizon in terms of actions count, which makes training dicult.

To address this issue, some approaches such as [START_REF] Mnih | Human-level control through deep reinforcement learning[END_REF] or [START_REF] Brechtel | Probabilistic MDP-behavior planning for cars[END_REF] suggest repeating primitive actions for several steps to reduce increase their duration and reduce the time resolution, hence decreasing the variance during training.

A more principled approach is proposed by [START_REF] Richard S Sutton | Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning[END_REF]: the options framework. In this setting, a set of sub-policies over primitive actions called options are used as high-level decisions by a policy over options. The options are often provided by the system designer as a way to introduce prior domain knowledge. Though they reduce the expressive power of the policy compared to primitive actions, they still allow to dene complex behaviours while being sample-ecient through temporally extended actions.

In the context of autonomous driving, there are some options choices that are widely shared among the community.

Indeed, the lateral behaviour of the ego-vehicle is often handled by three lane change options: {change to left lane, change to right lane, stay on current lane}. This is the case in [START_REF] Abbeel | Apprenticeship learning via inverse reinforcement learning[END_REF][START_REF] Ulbrich | Probabilistic online POMDP decision making for lane changes in fully automated driving[END_REF][START_REF] Osipychev | Proactive MDP-based collision avoidance algorithm for autonomous cars[END_REF][START_REF] Sharifzadeh | Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks[END_REF][START_REF] Li | Game-Theoretic Modeling of Driver and Vehicle Interactions for Verication and Validation of Autonomous Vehicle Control Systems[END_REF][START_REF] Mukadam | Tactical Decision Making For Lane Changing With Deep Reinforcement Learning[END_REF]. Aditionally, [START_REF] Ulbrich | Situation Assessment in Tactical Lane Change Behavior Planning for Automated Vehicles[END_REF] also consider the behaviours of preparing a lane change by edging on the side of the lane, and indicating a lane change with the blinker. [START_REF] Galceran | Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment[END_REF] also introduces separate options for handling turns at intersections: {turn right, turn left}. Specic manoeuvers can also be used in the presence of obstacles, such as a pass behavior learnt in [START_REF] Larry | Learning to Race: Experiments with a Simulated Race Car[END_REF] and an overtake behavior in [START_REF] Loiacono | Learning to overtake in TORCS using simple reinforcement learning[END_REF].

The longitudinal behavior can also be dealt with by using options. They can be used to ensure safety, like the brake option of [START_REF] Sunberg | The value of inferring the internal state of trac participants for autonomous freeway driving[END_REF] which applies a maximum safe acceleration with respect to the front vehicle. [START_REF] Isele | Navigating Intersections with Autonomous Vehicles using Deep Reinforcement Learning[END_REF] and [START_REF] Hollins Wray | Online decision-making for scalable autonomous systems[END_REF] dene three options for coming to a stop at an intersection, waiting or edging slightly, and proceeding through the intersection. They recognize that these options are less expressive than a set of discrete acceleration, but the resulting policy is easier to learn. It is also possible to use open-loop policies, like [START_REF] Wei | A Point-based Markov Decision Process for Robust Single-Lane Autonomous Driving Behavior under Uncertainties[END_REF] who dene a set of acceleration proles, such as: "keep constant acceleration for t 1 seconds, then keep constant velocity for another t 2 seconds".

Finally, options combining both lateral and longitudinal goals can be used, like in [START_REF] Shalev-Shwartz | On a Formal Model of Safe and Scalable Self-driving Cars[END_REF] where 10 4 semantic actions are generated by an option graph specifying both the lateral goal on lanes and longitudinal goal of relative positioning with respect to other vehicles and speed prole. In [START_REF] Paxton | Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments[END_REF] options generated by reinforcement learning as optimal policies with respect to specic manually-dened rewards to favour multiple behaviour such as following the front vehicle, changing lane only when it is safe, passing a vehicle, or stopping at an intersection. In [START_REF] Codevilla | End-to-end Driving via Conditional Imitation Learning[END_REF], a low-level control policy acting over acceleration and steering angle is modulated by a highlevel conditioning specifying the desired route at an intersection, among {left, right, straight}.

Conclusion

In this survey, we listed the most common representations for state and actions used in the autonomous driving literature. There is a wide variety of formulations with dierent properties in terms of size (large or small, xed or variable), continuousness or discreteness, invariance to permutations, characteristic time-scale, ease to model the dynamics, smoothness of the state-action mapping, etc. There is no best representation, and one must be chosen by considering many aspects of the intended use-case.

Figure 1 :

 1 Figure 1: The continuous kinematics representation

Figure 2 :

 2 Figure 2: The discrete kinematics representation

Figure 3 :

 3 Figure 3: The occupancy grid representation

Figure 4 :

 4 Figure 4: The polar coordinates representation

Figure 5 :

 5 Figure 5: The polar grid representation

Figure 6 :

 6 Figure 6: The lane-centric representation


O Chabani

S Abide

N Lamrous

& B Zeghmati

NUMERICAL STUDY OF NON-BOUSSINESQ CONVECTION IN A VENTILATED CAVITY

Keywords: ventilated cavity, low Mach number, numerical simulation, heat transfer

The non-Boussinesq convection in a ventilated square cavity is studied numerically. The flow is driven by a prescribed constant inlet velocity at the top-left wall and an outlet located at the bottom-right wall. The prescribed temperature difference, between the inlet flow and the walls of the cavity, involves buoyancy forces which modify the flow. The density variations are considered by means of a low-Mach-number formulation, which is discretized by a projection scheme based on a second-order finitedifference discretization. The steady solutions associated to forced and mixed convection are presented. A discussion based on the flow and heat transfer features is proposed. Especially, the deviation from the Boussinesq model is highlighted in the case of mixed convection.

INTRODUCTION

Convection heat transfer is dominant in several thermal engineering applications. Some examples include the cooling of electronic devices and the design of greenhouses. Thus, in the last 40 years, a large number of numerical studies have been reported in the literature to describe heat transfer coefficient exchange, particularly, for flows in wall bounded domain, and precisely, flows in confined domain with several inlet/outlet ports such as ventilated cavity.

A ventilated cavity refers to flows in a rectangular or parallelepipedic domain having one or more inlet/outlet ports. On the application sides, building ventilation motivates an important amount of experimental and numerical studies. For instance, [START_REF] Nielsen | The velocity characteristics of ventilated rooms[END_REF] have investigated the turbulent flow in a parallelepipedic domain with an inlet on the upper-left face and with an outlet port on the bottom-right face. This flow configuration has been considered for experiments [START_REF] Nielsen | The velocity characteristics of ventilated rooms[END_REF], large Eddy simulations [START_REF] Zhang | Large eddy simulation of indoor airflow with a filtered dynamic subgrid scale model[END_REF], or RANS computation [START_REF] Limane | Thermo-ventilation study by openfoam of the airflow in a cavity with heated floor[END_REF]. A flow in a cubical cavity has also been considered in the recent work of [START_REF] Van Hooff | Counter-gradient diffusion in a slot-ventilated enclosure assessed by LES and RANS[END_REF] or previously by [START_REF] Ezzouhri | Large eddy simulation of turbulent mixed convection in a 3D ventilated cavity: Comparison with existing data[END_REF]. These works highlight the specific features, such as transitional flows, turbulence anisotropy, and adverse pressure gradient [START_REF] Van Hooff | Counter-gradient diffusion in a slot-ventilated enclosure assessed by LES and RANS[END_REF]. Heat transfer also contributes to modify the flow structure. For instance, [START_REF] Ezzouhri | Large eddy simulation of turbulent mixed convection in a 3D ventilated cavity: Comparison with existing data[END_REF] have outlined the existence of two mean flows for an identical flow configuration.

Laminar flows in rectangular domain are also of primary interest in the understanding of heat transfer mechanisms. Applications are encountered in electronic device cooling. Heat transfers in a cavity can be controlled by introducing one or several inlet velocities; this is the consequence of the forced convection mechanism. [START_REF] Saeidi | Forced convection in a square cavity with inlet and outlet ports[END_REF] well illustrated this kind of investigation. They have proposed a parametric study to show the influence of the inlet and outlet locations on the flow features and heat transfers. Their results suggest that the Nusselt number achieves the highest values for an outlet at the vicinity of the corner. The lowest one is achieved for an outlet [START_REF] Saeidi | Forced convection in a square cavity with inlet and outlet ports[END_REF]. [START_REF] Saeidi | Transient flow and heat transfer leading to periodic state in a cavity with inlet and outlet ports due to incoming flow oscillation[END_REF] considered a time-dependent sinusoidal inlet velocity and shown that a peak of the average Nusselt number is observed for a Strouhal number close to 1.

More recently, Selimefendigil and Öztop (2014c) have investigated the flow in a ventilated cavity with two symmetrical inlets and outlet ports, and by considering buoyancy effects. They have considered several variants such as time forcing inlets and a stationary or rotating cylinder inside the cavity (Selimefendigil andÖztop, 2014c, 2015). Nanofluid and ferrofluid have also been investigated (Selimefendigil and Öztop, 2014a,b). From the amount of results, the rise of heat transfer with respect to Reynolds or Grashof numbers should be noted. Moreover, for a prescribed harmonic inlet forcing, a periodic regime is noted (Selimefendigil and Öztop, 2014b). In this case, the lowest Nusselt number is achieved for the highest frequency.

Based on the above literature review, all results involve incompressible flows with the Boussinesq approximation for the buoyancy term. However, this assumption is valid for density nearly constant. Several works based on the low-Mach-number model [START_REF] Paolucci | Filtering of sound from the Navier-Stokes equations[END_REF] show the importance of the density variation even at low speed flow. For instance, [START_REF] Nicoud | Conservative high-order finite-difference schemes for low-Mach number flows[END_REF] has performed a simulation of the turbulent flow and shown that the mean flow admits an asymmetrical feature due to the mean gradient temperature. The low-Mach-number model has been also retained to predict natural convection flow in a differentially heated cavity with prescribed large temperature gradients (Le [START_REF] Quéré | Modelling of natural convection flows with large temperature differences: A benchmark problem for low mach number solvers, Part 1. Reference solutions[END_REF]. More recently, [START_REF] Sun | On the heat and mass transfer analogy for natural convection of non-dilute binary mixtures of ideal gases in cavities[END_REF] and [START_REF] Sun | Transient double-diffusive convection in an enclosure with large density variations[END_REF] have analyzed the heat and mass transfers in a cavity based on the low-Mach assumption. Some discrepancies are noted for the local Nusselt and Sherwood numbers, in comparison with the Boussinesq limit. In addition to mass or thermal Rayleigh numbers, the low-Mach model introduces a parameter denoted ϵ, which refers to the deviation from the Boussinesq assumption, and a thermodynamic pressure for closed enclosures [START_REF] Sun | On the heat and mass transfer analogy for natural convection of non-dilute binary mixtures of ideal gases in cavities[END_REF].

Regarding the present literature review, it appears that no work is reported for mixed convection in a ventilated enclosure based on the low-Mach-number approach. So here we propose to investigate it. The geometrical configuration of [START_REF] Saeidi | Transient flow and heat transfer leading to periodic state in a cavity with inlet and outlet ports due to incoming flow oscillation[END_REF] is retained with one inlet and one outlet port. Moreover, the buoyancy effect and density variations are also considered to compute steady laminar flow. Section 2 describes the geometrical and governing equations. Next, the numerical method for the computation of solutions is detailed. The numerical results, including the code validation, are discussed in Section 4.

PROBLEM FORMULATION

Flow Configuration

A sketch of the physical problem is shown in Fig. 1. A square cavity of height H is considered with an inlet and outlet ports located at the left-top and right-bottom walls, respectively. The size of these ports is h = 0.25H. The temperature T h at the walls is constant. At the inlet port, a uniform velocity u i and temperature T c are prescribed. It is assumed that the incoming fluid flow temperature is lower than the wall temperature T c < T h . The physical properties of the fluid are assumed to be constant and determined at the reference temperature T ref = (T c + T h )/2. Depending on the inlet velocity and temperature gradient, the flow could be driven by forced, mixed, or natural convection. The governing equations are described hereafter.

Governing Equations

The flow is assumed to be two-dimensional, and the low-Mach-number model is considered for density variation. This formulation can deal with large temperature differences and/or large mass fraction differences in contrast to the Boussinesq approximation. From a practical point of view, this model is more suitable than the compressible model because acoustic waves are not considered. This relaxes the stability constraints of such numerical solvers [START_REF] Paolucci | Filtering of sound from the Navier-Stokes equations[END_REF]. This model relies on the decomposition of the motion pressure into a fluctuating part p and a thermodynamic pressure P th . The former satisfies the equation of state (ideal gas law). The dimensionless equations read

∂ρ ∂t + ∇ • ρu = 0 (1) FIG. 1: Sketch of the ventilated cavity ∂ρu ∂t + ∇ • ρu ⊗ u = -∇p + ∇τ + Ri ϵ (ρ -1) g (2) ρ ∂θ ∂t + ∇ • (ρuθ) = 1 RePr ∇ • (∇θ) + γ -1 ϵγ d t P th (3)
P th = ρ (1 + ϵθ) (4)
where the vector u = [u, v] t stands for the dimensionless velocity and p for the dimensionless pressure. These dimensionless equations are derived from the reference length and velocity, H and u i , respectively. The dimensionless density ρ stands for the ratio between the fluid density and the inlet density ρ i . The dimensionless temperature is defined by θ = (T -T c )/(T h -T c ). The Reynolds and Grashof numbers are defined by Re = u i H/ν and Gr = H 3 gϵ/ν 2 , respectively. The parameter ϵ = 1 -T c /T h is the deviation from the Boussinesq approximation. The boundary conditions are described in Section 2.1, except for the outlet port, for which the homogeneous Neumann boundary conditions are prescribed for each variable.

NUMERICAL METHODS

Method of Solutions

The numerical solution of the low-Mach-number formulation, Eqs. ( 1)-( 4), is addressed by means of a fractional step method for the incompressible Navier-Stokes equations. It should be noted that an interesting review has been proposed by [START_REF] Knikker | A comparative study of high-order variable-property segregated algorithms for unsteady low Mach number flows[END_REF]. The main idea is to write the constraint on the divergence velocity and to enforce it with a pressure correction equation. The constraint on the divergence velocity is

∇ • u = 1 V 1 P th ( 1 RePr ∇ • ∇θ - 1 γ d t P th ) ( 5
)
where V is the volume of the enclosure. The volume integration of this constraint leads to a relation for the temporal derivative of the thermodynamic pressure:

d t P th = 1 V 1 P th 1 RePr ∫ V ∇ • ∇θdV (6)
In comparison with incompressible Navier-Stokes equations, the two main differences involve the temperature equation and the velocity divergence. First, the temporal derivative of the thermodynamic pressure acts in the temperature equation and the velocity divergence. Then, the diffusive terms of the temperature appear in the velocity divergence.

The procedure to solve the low-Mach-number formulation is based on the review of [START_REF] Knikker | A comparative study of high-order variable-property segregated algorithms for unsteady low Mach number flows[END_REF]. The implemented algorithm involves the semi-explicit backward Euler/Adam Bashfort (EB2/AB) time scheme, which is second-order accurate. The main steps are noted as follows:

1. Advance the temperature. The variable θ n+1 is computed from the time-discretized temperature equation. The new thermodynamic pressure is updated from

P th = M 0 / ∫ 1/θdv.
2. Advance the density. The density ρ n+1 is evaluated from ρ n+1 = P th /(1 + ϵθ n+1 ).

3.

Compute the provisional velocity. The momentum equations are advanced in time to provide a provisional velocity field which does not fulfill the divergence constraint.

4. Pressure correction step. A pressure correction equation (Poisson) with variable coefficient is solved to enforce the divergence constraint.

5. Velocity correction. In this final step the provisional velocity is corrected by means of the gradient of pressure correction.

The above procedure is detailed by [START_REF] Knikker | A comparative study of high-order variable-property segregated algorithms for unsteady low Mach number flows[END_REF] and labeled "projection method with variable-coefficient Poisson equation." The time step is such that the CFL condition is always lower than 1/2. The staggered grid layout is retained and combined with a second-order finite-difference discretization. The mesh can be stretched by means of analytic mesh transformation [START_REF] Vinokur | On one-dimensional stretching functions for finite-difference calculations[END_REF]. The solutions to the linear systems arising from the discretization of the temperature, the momentum, and the pressure equations are tackled with a geometric multigrid solver. Particularly, the solver provided by the HYPRE library [START_REF] Falgout | The design and implementation of HYPRE, a library of parallel high performance preconditioners[END_REF] has been used in this work.

Validation

The code validation is made by considering the benchmark of the differentially heated cavity with a large temperature gradient [START_REF] Bouafia | Non-Boussinesq convection in a square cavity with surface thermal radiation[END_REF][START_REF] Knikker | A comparative study of high-order variable-property segregated algorithms for unsteady low Mach number flows[END_REF][START_REF] Sun | On the heat and mass transfer analogy for natural convection of non-dilute binary mixtures of ideal gases in cavities[END_REF]. The benchmark case is described by [START_REF] Quéré | Modelling of natural convection flows with large temperature differences: A benchmark problem for low mach number solvers, Part 1. Reference solutions[END_REF] and consists in computing the natural convection in a two-dimensional square cavity for which the isotherm left and right walls define the temperature gradient ∆T = T h -T c , while the top and bottom walls are adiabatic. The working fluid is an ideal gas; its properties, except density, remain constant. The benchmark case is uniquely defined if the Grashof, Prandtl, and the thermal expansion parameter ϵ are known. A very fine mesh of size 1024 × 1024 refined at the vicinity of the walls is considered to calculate an accurate solution for the set of parameters: Gr = 10 6 , Pr = 0.71, and ϵ = 0.6. H being the height of the cavity, the smallest and biggest cell sizes are equal to 0.00049H and 0.00149H, respectively. Such a mesh refinement is achieved by using a tangent hyperbolic mesh transformation [START_REF] Vinokur | On one-dimensional stretching functions for finite-difference calculations[END_REF]. The isotherms and the streamlines are plotted in Fig. 2. This plot, for instance, can be compared with Fig. 2 of the recent work of [START_REF] Bouafia | Non-Boussinesq convection in a square cavity with surface thermal radiation[END_REF]. Regarding the general features of this flow, the present results are in good agreement with previous works. A more specific comparison with the Nusselt number and the thermodynamic pressure is proposed in the Table 1. The quantities of interest are the average, minimum, and maximum Nusselt numbers on the hot and cold walls of the cavity. The first and second column provide the present code and Vierendeels' solutions, respectively. It should be pointed out that the data of Vierendeels are extracted from the benchmark work detailed by [START_REF] Paillère | Modelling of natural convection flows with large temperature differences: A benchmark problem for low Mach number solvers, Part 2[END_REF]. An excellent agreement is noted for all the heat transfer characteristics computed with the present code. It can be concluded that the validation of the present numerical method is satisfactory and thus can be used to evaluate the solutions of the low-Mach-number flow in a ventilated cavity.

FIG. 2:

Isotherms and streamlines of the heated cavity; Ra = 10 6 and ϵ = 0.6

RESULTS

In this section the flow and heat transfer features are discussed. The simulations of the ventilated cavity flow have been carried out for the following range of dimensionless numbers: 100 Re 600, 0 Gr 10 5 , and 10 -3 ϵ 0.6.

Numerical Details

A set of simulations has been carried out to give numerical solutions of the flow for the range of parameters previously mentioned. The criterion to stop the simulation is the residual of each variable: below 10 -6 the solution is assumed converged. The meshes are stretched such as the smallest cells are located at the vicinity of the walls, and the biggest at the center of the cavity. The flow for Re = 500, Gr = 10 5 , and ϵ = 0.6 has been computed with four different mesh grid sizes: 40 × 40, 80 × 80, 160 × 160, and 320 × 320. Then, the velocity and temperature profiles at the middle of the cavity are plotted in Fig. 3. Figure 3 indicates us that a grid size of 160 × 160 is sufficient to accurately describe the flow. In this case, the smallest and the biggest cell sizes are equal to h min = 0.00315H and h max = 0.00853H, respectively. In the following, the discussions are based on streamlines and iso-temperatures, velocity/temperature profiles, and the Nusselt number. An illustration is given in Fig. 4 for the case Re = 400, Gr = 10 3 , and ϵ = 0.4. The local Nusselt number is computed with:

Nu(s) = ∂θ ∂n s (7)
where s is a curvilinear coordinate as depicted in Fig. 4(d), and n the normal of the wall. In this study, the average Nusselt number is obtained by integration over the left and the bottom walls. For this configuration, the main stream flow, formed between two circulation regions, connects the inlet and the outlet ports. The presence of a primary vortex, 70% of characteristic length H, a secondary vortex next to the

Heat Transfers

In this section, the heat transfers at the cavity's left and bottom walls are discussed in order to compare the Boussinesq and non-Boussinesq models. Thus the mean Nusselt number is computed from the simulations performed with the Reynolds and Grashof numbers fulfilling the range 100 Re 600 and 0 Gr 10 5 . As defined previously, the non-Boussinesq parameter is set between ϵ = 10 -3 and 0.6. Such a choice allows us to consider the forced and the mixed convection regimes, while requiring the Boussinesq (ϵ = 10 -3 ) and the non-Boussinesq (ϵ = 0.6) models.

The mean Nusselt numbers gathered from the present simulations with the previously defined parameters are represented in Fig. 5.

A specific format is retained in Fig. 5: the mean Nusselt number is plotted with respect to the Richardson number and using a logarithmic scale. Using this data representation, four branches could be clearly revealed. Each branch is associated to a Grashof number, as indicated in Fig. 5. It should be noted that the plot of the branch Gr = 0 is made possible by substituting arbitrarily Gr = 0 with Gr = 10 -2 . This improves the analysis of the data without modifying the results.

The pure forced convection regime is featured by Gr = 0. The mean Nusselt number associated to this regime corresponds to the first branch at the left part of an influence is noted. Thus, for the highest Reynolds number Re = 600, the Nusselt numbers spread in the range 5.5 ≤ Nu ≤ 6, while the thermal expansion parameter ϵ is comprised between 10 -3 and 0.6. Moreover, the discrepancies between the incompressible and quasicompressible flow are quite small. A similar analysis can be achieved with the Grashof numbers Gr = 10 3 and Gr = 10 4 .

A different behavior is noted for the fourth branch Gr = 10 5 , except for the smallest Reynolds number Re = 100. For this value, the thermal expansion parameter ϵ does not influence the mean Nusselt number. A significant influence is observed for the other Reynolds numbers. Moreover, a particular behavior should be noted at Re = 600, where, unlike the other branches, the thermal expansion parameter has no significant influence. Irrespective of the density variation model (Boussinesq or non-Boussinesq), the mean Nusselt number is around 5.25. The most important influence of the thermal expansion parameter on the Nusselt number is achieved for the Reynolds numbers comprised between 200 and 400. As can be seen in Fig. 5, the branch stretches into two curves associated to the Boussinesq and the low-Mach approximations. This global analysis of the heat transfers leads us to consider the present data in two groups: the mixed convection regime (Gr = 10 5 ) and the forced convection regime (Gr = 0, Gr = 10 3 , Gr = 10 4 ). In the following section the flow is discussed for the mixed convection regime.

The Mixed Convection Regime

In this section, the mixed convection regime is considered. This corresponds to the simulations carried out with the Grashof number Gr = 10 5 . The present discussion on the influence of the parameter ϵ focuses on the Reynolds numbers Re = 100, Re = 600, and Re = 200. The chosen Reynolds numbers correspond to the bottom, top and middle of the fourth branch presented in Fig. 5.

For the Reynolds number Re = 100, Fig. 5 exhibits a small influence of the parameter ϵ on the mean Nusselt number. In this case, the local Nusselt numbers at the left and bottom walls of the cavity are investigated. The local Nusselt number is plotted in Fig. 6(a).

Its values, at the left wall, are smaller for the low-Mach approximation in comparison with the Boussinesq limit. In contrast, at the bottom wall, greater values of the local Nusselt number are observed for the low-Mach regime. This rise could be associated to the buoyancy forces induced by the temperature gradient. The balance of the local Nusselt numbers at the left and the bottom walls of the cavity leads to a small variation of the mean Nusselt number. The isotherms and the streamlines associated to the former parameters are presented in Figs. 7(a . Indeed, at the left wall of the cavity the local Nusselt heat transfer is higher for the Boussinesq limit. At the bottom wall the opposite phenomena holds. In fact, the cell confined between the jet stream and the lower left circulation has become stronger for the non-Boussinesq approximation. Moreover, its center has moved toward the bottom wall. The shear stress also becomes important at the vicinity of the bottom wall, leading to a rise of the local Nusselt number.

Here, the local Nusselt number for Re = 200 is considered. According to Fig. 5, an influence of the thermal expansion parameter is noted. To explain this, the local Nusselt numbers at the left and the bottom walls are considered [Fig. 6(b)]. The discrepancies between the Boussinesq limit (ϵ = 10 -3 ) and the non-Boussinesq approximation (ϵ = 0.6) originate from the left wall. In the first-half part of the bottom wall, the Nusselt number in the Boussinesq limit is higher than the one computed with the non-Boussinesq approximation. In the second-half part of the bottom wall, the opposite behavior is observed. Thus the average Nusselt number at the bottom wall is quite similar for the two models. The discrepancies originate from the left wall, as noted in Fig. 6(b), on which the Nusselt number in the Boussinesq limit is notably higher than the non-Boussinesq approximation. In the case of the non-Boussinesq model (ϵ = 0.6) the secondary recirculation region, which occurs at the left wall below the flow inlet, is 3 times bigger [Figs. 7(c) and 7(d)] than the one noted in the Boussinesq limit (ϵ = 0.001). The growth of this secondary recirculation has for a consequence that the wall shear stress decreases and, in a similar way, the heat transfers. The tightened isotherms in the Boussinesq limit [Fig. 7(c)], which are no longer observed in the non-Boussinesq approximation, corroborate this analysis.

The Forced Convection Regime

This section is devoted to the description of the forced convection regime. This corresponds to the Grashof numbers Gr = 0, Gr = 10 3 , and Gr = 10 4 , and the Reynolds numbers such as the Richardson number is lower than 1. Moreover, the behavior of the flow and the heat transfers is quite similar for the three Grashof numbers at the lower Reynolds numbers. A slight dependency with respect to the thermal expansion parameter is noted for increasing Reynolds number (Fig. 5). So, in the following, the discussion focuses on the Grashof number Gr = 10 A similar conclusion can be drawn for the Reynolds number Re = 200; there is no significant effect of the density variation modeling. In comparison with the Reynolds number Re = 100, the main recirculation at the bottom left is bigger and the isotherms are more stiff at the left wall [Figs. 9(c) and 9(d)].

The more significant discrepancies occur for the Reynolds number Re = 600. In this case, the mean Nusselt number is higher by using the Boussinesq model than the non-Boussinesq one. A Nusselt number Nu = 5.9 versus Nu = 5.5 for the non-Boussinesq approach is noted. The flow is also slightly modified. The main difference with the two other Reynolds numbers is that a small recirculation occurs at the vicinity of the bottom-left corner [Fig. 9(e)]. Moreover, at the left wall the isocontours of temperature are more tightened, which corroborates that the highest Nusselt number is obtained for this Reynolds number. The discrepancies between Boussinesq and non-Boussinesq come from the heat transfer at the left wall, for which the Boussinesq approach provides always a superior local Nusselt number [Fig. 8(c)].

The average Nusselt number on the left and bottom walls and for the Reynolds number Re = 600 are summarized in Table 2. This table clearly shows that there is a small influence of the density variation modeling up to Grashof number Gr = 10 4 . For the three Grashof numbers, the mean Nusselt number is around 6% more by considering the Boussinesq model. For the highest Grashof number Gr = 10 5 there is no longer a noticeable discrepancy. The difference is about 0.4%.

TABLE 2:

Mean Nusselt number on the left and bottom walls of the cavity (Re = 600) Gr 0 10 3 10 4 10 5 ϵ = 10 -3 5.87 5.87 5.85 5.21 ϵ = 0.6 5.53 5.52 5.50 5.23

CONCLUSION

A numerical study of a ventilated cavity flow has been presented. The low-Mach-number formulation has been retained to include buoyancy effects without the Boussinesq assumption. The governing equations have been discretized by means of second-order finite-difference discretization. A projection scheme to deal with the velocity/pressure and thermodynamic pressure coupling has been also considered. The method of solution has been successfully validated by comparisons with the two-dimensional low-Mach-number differentialy heated cavity. Based on this solver, simulations of the steady and two-dimensional flow in a ventilated cavity have been carried out. The range of parameters 100 ≤ Re ≤ 600, 0 ≤ Gr ≤ 10 5 , and 0 ≤ ϵ ≤ 0.6 leads to forced and a mixed convection regimes. It has been mainly shown that the thermal expansion parameter has a significant effect on mixed convection flow (Gr = 10 5 and 200 ≤ Re ≤ 400). This study could benefit from including the thermal dependency of the fluid properties, and thus to assess their influence on the heat transfers. Also, such a study could be extended to the moist air convection in order to compare the Boussinesq and non-Boussinesq model when heat and mass transfer are combined.

FIG. 3 :

 3 FIG.3: Mesh sensitivity analysis; velocity and temperature horizontal profiles at the middle of the cavity; Re = 500, Gr = 10 5 , and ϵ = 0.6

FIG. 4 :

 4 FIG. 4: Ventilated cavity for the parameters Re = 400, Gr = 10 3 , ϵ = 0.4: isotherms and streamlines (a), midline velocities (b), midline temperatures (c), and Local Nusselt number profile (d)

  FIG. 5: Mean Nusselt number versus the Richardson number for the different flow parameters Re and Gr

FIG. 6 :

 6 FIG. 6

FIG. 7 :

 7 FIG. 7: Isotherms and streamlines for the mixed convection regime (Gr = 10 5 ): Re = 100 (a,b), Re = 200 (c,d), Re = 600 (e,f); left column ϵ = 0.001 and right column ϵ = 0.6

  3 and for the Reynolds numbers Re = 100, Re = 200, and Re = 600. First, for the Reynolds number Re = 100 there is no noticeable dependency with respect to the thermal expansion parameter ϵ. The local Nusselt number at the left and the bottom walls is plotted in Fig.8(a).The local Nusselt number is very similar, whatever the thermal expansion parameter ϵ. The flow consists of a main stream connecting the inlet and the outlet ports, a main circulation at the bottom left of the cavity, and a small one at the upper right [Fig.9(a)]. The Boussinesq and non-Boussinesq flows are very similar [Figs. 9(a) and 9(b)].

FIG. 8 :FIG. 9 :

 89 FIG. 8: Local Nusselt number for the forced convection regime (Gr = 10 3 ): Re = 100 (a), Re = 200 (b), and Re = 600 (c)

TABLE 1 :

 1 Nusselt number and thermodynamic pressure; h and c refer to the hot and cold wall, respectively

		Present code	Vierendeels
	Mesh	1024 × 1024	2048 × 2048
	Nu av (h/c)	8.8597/8.8601	8.85978/8.85978
	Nu min (h/c)	1.0735/0.8551	1.0735/0.8551
	Nu max (h/c) 19.5984/16.3656 19.5964/16.3623
	P th	0.856327	0.856340


Andrea Aparicio
email: andrea.aparicio.martinez@gmail.com

Leonid Fridman

Denis Efimov

Stabilization of systems with switchings on the axis of their coordinates and its input-to-state properties

The stabilization problem for switched systems in which switchings occur on the axes of its state coordinates is considered. It is shown that a linear feedback, or a combination of linear feedback and a switching law, can be designed such that the closed-loop is stable, and has the inputto-state property, allowing to guarantee robustness against matched and unmatched perturbations. The conditions of stability are expressed in the form of linear matrix inequalities. The results are illustrated by numerical simulations.

Introduction

The study and implementation of systems that commute between different behavior schemes are strong mathematical and technical challenges. The theoretical results available are gathered in the literature of switched, hybrid, and variable structure systems; a survey on the existing stability criteria for the first can be found in [START_REF] Shorten | Stability criteria for switched and hybrid systems[END_REF]. The switched systems domain can be divided in two big groups: the systems whose switchings depend on the time, and those that operate under statedependent switching laws. The latter group can also be divided in two: the systems with given switching laws, that need to be stabilized in some way, or those for which the design of a stabilizing switching law is required. Numerous works can be found that offer stability results based on different well-known strategies, and the most utilized are Linear Matrix Inequalities (LMIs) and Lyapunov methods ( [START_REF] Pettersson | Lmi for stability and robustness of hybrid systems[END_REF][START_REF] Johansson | Computation of piecewise quadratic lyapunov functions for hybrid systems[END_REF][START_REF] Wicks | Construction of piecewise lyapunov functions for stabilizing switched systems[END_REF]). The state-dependent switching can be used to stabilize the system with the commutation law directly following the expression of a Lyapunov function ( [START_REF] Hetel | Local stabilization of switched affine systems[END_REF][START_REF] Zohra | Stabilization of switched affine systems with disturbed statedependent switching laws[END_REF]). In some cases the dwell-time restriction is imposed ( [START_REF] Allerhand | Robust state-dependent switching of linear systems with dwell time[END_REF]), then asymptotic rates of convergence can be obtained. Skipping the requirement on a minimum time between commutations it is possible to ensure a finite-time rate of convergence in the system, as in the sliding mode control framework ( [START_REF] Utkin | Sliding Mode Control in Electro-Mechanical Systems[END_REF][START_REF] Shtessel | Sliding Mode Control and Observation[END_REF]). Though in all these examples the stabilization is achieved, some perturbations might occur such that the sliding-mode is lost and the trajectories end up diverging, causing undesired or even dangerous behaviour in the related applications.

In this paper we will focus on state-dependent switching systems and, in particular, those whose switchings occur on the axes of its state coordinates, in other words, at the precise instant when a 1 change on the sign of the state variables occurs (frequently, such a condition can be ensured after a proper transformation of coordinates). An example of a physical realization of this class of systems is the ones that include relay or on-off electrical components in their configuration, and the study of the stability of such systems is one of the oldest control problems ( [START_REF] Flugge-Lotz | Discontinuous Automatic System[END_REF][START_REF] Kulebakin | On Theory of Vibration Controller for Electric Machines[END_REF][START_REF] Nikolski | On Automatic Stability of a Ship on a Given Course[END_REF][START_REF] Tzykpin | Theory of control relay systems[END_REF][START_REF]Relay control systems[END_REF]). A well-known real-world scenario of such a class of systems includes the mechanical dynamics with dry friction, when sign of the velocity determines the direction of the friction force.

The state space study of one-dimensional relay systems was performed in [START_REF] Anosov | On stability of equilibrium points of relay systems[END_REF] (see also [START_REF] Fridman | Sliding Mode Control In Engineering[END_REF]). In that work it was proven that a first order relay system with relative degree one is semi-globally finite-time stable if the switches occur on the axis of its state variable. In the same work it was shown that a system of relative degree two that switches on the axis of its first coordinate is semiglobally exponentially stable if a linear stabilizer dependent on the second coordinate is added. Moreover, it was proven that a system with relative degree greater than two that switches on the axis of its state coordinates is never stable unless some linear stabilizing terms are added.

In this paper, a particular class of systems is dealt with, whose dynamics are driven by uncertain chains of integrators with the signs of all the state variables in the last coordinate. The semi-global finite-time convergence of this kind of systems, for the case of order two, was proved in [START_REF] Emelyanov | Second order sliding modes in controlling uncertain systems[END_REF]. Almost three decades after that, in [START_REF] Sanchez | On a sign controller for the triple integrator[END_REF] it was shown that for the third order, the majority of solutions converge to zero in finite-time, but there is also a two dimensional set of equilibrium points different from zero. This set of equilibrium points of dimension n -1 also appears for chain of integrators of order n with n-relays in the dynamics of the last state variable.

The stability in the above mentioned cases is quite fragile and even a slight perturbation makes the trajectories of these systems escape to infinity. In order to ensure robustness against perturbation, the typical procedure is to design gains of a switching controller based on an upper bound of the perturbation, so it is required that this bound is known. Also, it is typically assumed that the perturbations are matched to the control input because unmatched perturbations, even when an upper bound to them is known, can also easily destroy the stability of switching systems. Some works that offer solutions to the unmatched disturbance problem are [START_REF] Castaños | Dynamic switching surfaces for output sliding mode control: An H ∞ approach[END_REF][START_REF] Aparicio | Output feedback sliding-mode control with unmatched disturbances, an iss approach[END_REF][START_REF] Osuna | L2-gain tuning of variable structure siso systems of relative degree n[END_REF].

Considering these results, one can conclude that if the system switches every time on the axes of its state space variables, it is impossible to establish its global convergence to the origin, and then some stabilizers should appear in the system's structure, or added as controllers, in order to achieve this goal. Also, that some actions must be taken in order to ensure some robustness of the closed loop to certain perturbations, at least. In this paper, for an arbitrary order system with switches on the axes of each of its coordinates (the switches can occur in any of the state variables), the conditions are established that allow the stability to be kept by adding a linear compensator. The resulting closed loop can be placed within the classifications 1 and 3, mentioned in the tutorial paper [START_REF] Decarlo | Variable structure control of nonlinear multivariable systems: a tutorial[END_REF], that refer to relays with constant gains, and linear feedbacks with switched gains, respectively. Even more, we investigate de input-to-state stability (ISS) of the closed loop, to ensure certain robustness against bounded perturbations. The main contributions of the paper are

• Sufficient conditions for the linear stabilizer design.

• The ISS properties of the closed loop.

The work is organized as follows: Section 2 includes the notation used throughout this paper, as well as some known results useful for the development of the methodology. Section 3 includes the problem statement. In Section 4 the form of the control signal is proposed, and the stability of the closed loop is proved by constructing a Lyapunov equation. The stability conditions are presented in the form of LMIs. In Section 4.1 the input-to-state stability property of the closed loop system is investigated, which guarantees robustness to external, unknown, bounded inputs that can be both matched and unmatched. Academic examples are provided in Section 5 along with some numerical simulations that validate the theoretical results. Finally, Section 6 contains some conclusions of this work.

Preliminaries

Notation

• The element-wise application of an operator • to a vector v is indicated by •(v);

• I n denotes the Identity matrix of dimension n;

• diag(A) represents a matrix where the main diagonal is the same as that of matrix A and every other element is equal to zero;

• a (n×s) , for a constant a represents a matrix of dimension n × s, whose every element is equal to a;

• A ij represents the element in the position (i, j) of a matrix A;

• A [i] denotes the ith column of a matrix A;

• A int n represents a square matrix of size n whose every element is equal to zero, except for the diagonal above the main one, which is composed of ones;

• λ min (A) and λ max (A) represent the minimal and the maximal eigenvalues of a matrix A, respectively;

• |a| represents the absolute value of a scalar a;

• v denotes the Euclidean norm of a vector v ∈ R n , v 1 = n i=1 |v i | and v ∞ = max 1≤i≤n |v i |; • A represents the induced norm of a matrix A ∈ R n×s , while A 1 = max 1≤i≤s A [i]
1 and

A ∞ = A T 1 ; • For a matrix A ∈ R n×s the following norm equivalences hold ([23]) 1 √ n A 1 ≤ A ≤ √ s A 1 . (1)
For a and b nonnegative real numbers, and if p and q are some positive real numbers such that 1 p + 1 q = 1 the following inequality holds ( [START_REF] Young | On classes of summable functions and their fourier series[END_REF])

ab ≤ a p p + b q q
• The set of all functions endowed with the (essential) supremum norm

ω ∞ = (ess) sup t≥0 ω(t) < ∞, is denoted by L m ∞ Definition 1. ISS ([25]): A system ẋ = f (x, u) is globally input-to-state stable (ISS) if there exists a KL-function β : R ≥0 × R ≥0 → R and a K-function γ : R ≥0 → R ≥0 such that, for each input ω ∈ L m ∞ and each initial state x(0) ∈ R n , it holds, for each t ≥ 0 that x(t, x(0), u) ≤ β( x(0) , t) + γ( ω ∞ )
Definition 2. ISS-Lyapunov function ( [START_REF] Sontag | On the input-to-state stability property[END_REF]): A smooth function V : R n → R ≥0 is called ISS-Lyapunov function for a system ẋ = f (x, u) if there exist K ∞ -functions α 1 , α 2 , and K-functions α 3 and χ, such that

α 1 ( x ) ≤ V (x) ≤ α 2 ( x )
for any x ∈ R n , and

∇V (x)f (x, ω) ≤ -α 3 ( x )
for any x ∈ R n and any ω ∈ R m such that x ≥ χ( ω ).

Theorem 1. Nonsmooth Lyapunov Functons (Theorem 3.1 in [START_REF] Orlov | Discontinuous Systems, Lyapunov Analysis and Robust Synthesis under Uncertainty Conditions[END_REF]) Suppose that in a domain

(x ∈ B δ , t ∈ R)
, where B δ is a ball centred at the origin with radius δ > 0, there exists a Lipschitz continuous, positive definite, decrescent function V (x, t) such that its time derivative, computed along the trajectories x(t) of the discontinuous system ẋ = f (x, t), which are initialized within B δ , is negative semidefinite almost everywhere for almost all t, then ẋ = f (x, t) is uniformly stable.

Problem Statement

Consider a system in the form

ẋ = Ā0 x + Ā1 sign(x) + Bu + ω (2) y = x,
where x ∈ R n is the state vector, sign(x) ∈ R n is a column defined as sign T (x) := sign(x 1 ) . . . sign(x n ) , u ∈ R m , with m ≤ n is the control input, and ω ∈ R p is a vector that contains all the perturbations or external, unknown inputs. Here, p ≤ n. The terms Ā0 , Ā1 ∈ R n×n , and B ∈ R n×m are known, real and constant matrices. Since sliding modes may occur, all the solutions are understood in the sense of Filippov ( [START_REF] Filoppov | Differential Equations with Discontinuous Righthand Sides[END_REF]). In its most general form, system (2) consists of a linear part and a part that switches on the axis of the states x. The perturbations are assumed bounded, that is ω ≤ ω, but ω is assumed unknown. Also, there is no restriction to the matchedness of the perturbations to the control input. The goal is to design a control u that takes the trajectories of the system to the origin, robustly in spite of the perturbations.

Control Design and Stability Analysis

In this section the nominal case of (2) will be considered. That is

ẋ = Ā0 x + Ā1 sign(x) + Bu (3) y = x,
Now, let the control signal u be

u = K 0 x + K 1 sign(x)
where K 0 , K 1 ∈ R (n×n) (see footnote 1 ). In the rest of the section a stability proof for the closed loop

ẋ = A 0 x + A 1 sign(x) (4)
where A 0 = ( Ā0 + BK 0 ), and A 1 = ( Ā1 + BK 1 ) will be provided, along with some conditions for the gain matrices K 0 and K 1 . This will be performed by establishing a sufficient LMI condition to construct a Lyapunov function for (4). To this end, the matrices

P ∈ R n×n , G ∈ R n×n , R ∈ R n×n , r ∈ R n×n and M ∈ R n×n
, and a constant µ must be defined in the following manner:

P :=    
R := 2A T 1 P + GA 0 , R d := diag(R), M := R d + |R -R d |, (5)
for some real constants g i , p ij , i, j = 1, . . . , n. Note that in the definition of M the absolute value of each element of the resulting matrix of the subtraction should be considered, and not any norm.

Theorem 2. Let the origin be the only equilibrium of (4), and the following pair of LMIs be satisfied

A T 0 P + P A 0 = -Q, M 1 (n×1) ≤0, for P = P T > 0, Q = Q T > 0, G ≥ 0, GA 1 = 0.
Then, a function V : R n → R defined as

V (x) := x T P x + n i=1 g i |x i | (6)
is a Lyapunov function for system (4), with a derivative estimate

V ≤ -x T Qx.
Proof. Since the matrix P is positive definite and G is positive semidefinite, then V is also positive definite and radially unbounded. The function V is locally Lipschitz continuous. Then, by Rademacher's theorem ( [START_REF] Morgan | Geometric Measure Theory 3rd Edition[END_REF]), it is differentiable almost everywhere in R n , and since GA 1 = 0, the derivative of V along the trajectories of ( 4) is

V = x T (A T 0 P + P A 0 )x + sign T (x)(A T 1 P + 1 2 GA 0 )x + x T (P A 1 + 1 2 A T 0 G) sign(x) = x T (A T 0 P + P A 0 )x + sign(x) T Rx. (7)
The Lyapunov equation A T 0 P + P A 0 = -Q can be solved for P = P T > 0, with Q = Q T > 0 if and only if A 0 is a stable matrix. From the second part of [START_REF] Allerhand | Robust state-dependent switching of linear systems with dwell time[END_REF] we have that sign

(x) T Rx = n i=1 x i n k=1 (sign(x k )R k,i ) ≤ n i=1 R i,i + n k =i, k=1 |R k,i | |x i |
and with M as defined above, we get

R i,i + n k =i |R k,i | ≤ 0, ∀ 1 ≤ i ≤ n ⇔ M T 1 (n×1) ≤0. Thus, V ≤ -x T Qx,
and if the conditions of Theorem 2 are satisfied, the function V is positive definite for all x, its derivative along the trajectories of ( 4) is negative definite, and by Theorem 1, ( 4) is uniformly stable.

Remark 1. The restrictions imposed on the matrix M can be replaced with the condition that

µ 1 (R) ≤ 0, where µ 1 (R) = sup i R i,i + k =i |R k,i | is the logarithmic norm of the matrix R calculated for the norm | • | 1 .

Perturbation Analysis and Input to State Stability

In this section we will prove that if system (4) satisfies the conditions of Theorem 2, it also admits the ISS property, which makes it robust to exogenous matched or unmatched bounded perturbations. It is important to note that while the boundedness of the external inputs is a fundamental condition for establishing the ISS property, the knowledge of its supremun norm is not necessary for the design. Consider system (4) with an unknown input function ω : R ≥0 → R n :

ẋ = A 0 x + A 1 sign(x) + ω. ( 8
)
Theorem 3. If ω is essentially bounded, i.e. ω ∈ L ∞ , and the conditions of Theorem 2 are satisfied, then ( 8) is ISS with an asymptotic gain γ : R ≥0 → R ≥0 given by

γ(r) = α -1 1 • α 2 • α -1 3 • α 4 (r),
where for any 1 ∈ 0, λ min (Q) λmax(P ) ,

α 1 (r) :=λ min (P ) r 2 α 2 (r) := λ max (P ) r + √ n max{g i } r α 3 (r) := (λ min (Q) -1 λ max (P )) r α 4 (r) := λ max (P ) 1 r + √ n max{g i } r.
Note that the magnitude of the gain γ(r) is proportional to the square of r, and that the established result is of qualitative nature and that quantitatively they would only represent the worst possible scenario.

Proof. The derivative of ( 6) over the trajectories of ( 8) is

V = -x T Qx + sign T (x)Rx + 2x T P ω + sign T (x)Gω.
For the above expression the following inequalities hold (see references in Section 2): sign

T (x)Rx ≤0, -x T Qx ≤ -λ min (Q) x 2 , 2x T P ω ≤λ max (P ) 1 x 2 + ω 2 1 ∀ 1 > 0, sign T (x)Gω ≤ √ n max{g i } ω , i = 1, . . . , n.
From definition 2, V is an ISS-Lyapunov function for [START_REF] Utkin | Sliding Mode Control in Electro-Mechanical Systems[END_REF] with

χ(r) = α -1 3 • α 4 (r), since α 1 ( x ) ≤ V (x) ≤ α 2 ( x ), (9)
V ≤ -α 3 ( x ) + α 4 ( ω ) (10)
with α 1 , α 2 , α 3 , α 4 ∈ K ∞ , as defined in the theorem. Then, following the arguments of [START_REF] Sontag | Smooth stabilization implies coprime factorization[END_REF], the system is ISS and the asymptotic gain function is γ(r).

Examples and numerical simulations

All simulations have been performed in Matlab software using the explicit Euler method with the discretizations step 0.001.

Interconnected mechanical systems

Consider two interconnected mechanical systems whose velocities are given by variables x 1 and x 2 respectively, and suppose that the acceleration of each of them can be controlled. The state space representation of this system can be written as

ẋ1 ẋ2 = a 11 a 12 a 21 a 22 x 1 x 2 + ω 1 ω 2 + u 1 u 2 (11)
y = 1 0 0 1 x 1 x 2 .
System (11) is linear, and the switches will be added as control signals. Two perturbations are considered, ω 1 ∈ R and ω 2 ∈ R, one on each of the systems. These perturbations are assumed to be bounded, but the value of their supremum norms is considered unknown. Suppose that the parameters of ( 11) are a 11 = -1, a 12 = 0.5, a 21 = 0.5 and a 22 = -1. If the control signals are chosen as

u 1 = -2sign(x 1 ) -sign(x 2 ) and u 2 = -sign(x 1 ) -2sign(x 2 ), (12)
then ( 11) can be written in the form (4) with

A 0 = -1 0.5 0.5 -1 , and A 1 = -2 -1 -1 -2
Matrix A 0 is Hurwitz, so the Lyapunov equation can be satisfied for a positive definite matrix P ∈ R (2×2) , with Q = I 2 . Choosing g 1 = g 2 = 0, yields the product M 1 1 = -0.67 -0.67 , whose every element is clearly negative. The conditions of Theorem 2 are satisfied. The lefthand side of Figure 1 shows the simulation results of the closed loop of ( 11) with [START_REF] Nikolski | On Automatic Stability of a Ship on a Given Course[END_REF], for the unperturbed case (upper-left), and when the perturbations are chosen as ω 1 = 1 + 0.5 sin(t) and ω 2 = 1.5 + 1.5 cos(t) (lower-left). Note that the value of these perturbations is not taken into account for the design. For comparison purposes, the right-hand side of Figure 1 shows the simulations results of the same system [START_REF] Kulebakin | On Theory of Vibration Controller for Electric Machines[END_REF] with and without perturbations, in closed loop with a linear feedback

u 1 u 2 = -2x 1 -x 2 -x 1 -2x 2 . ( 13
)
The theoretical results are validated through the simulations, which show that the trajectories of the system converge to the origin for the unperturbed case, and to a vicinity of the origin when the perturbations are present. Note that for the case when the sign controller is added to the perturbed system, the sliding mode is lost occasionally, but the trajectories remain bounded and then return to the sliding motion.

Third order system

Consider a third order system in the form [START_REF] Pettersson | Lmi for stability and robustness of hybrid systems[END_REF] with

A 0 =   0 1 0 0 0 1 0 0 0   , Ā1 =   0 0 0 0 0 0 -2 -2 -2.5   , B =   1 1 1   , (14)
where the initial conditions are given by x T 0 = x 1 (0) x 2 (0) x 3 (0) . In [START_REF] Sanchez | On a sign controller for the triple integrator[END_REF] it is proven that this system, for initial conditions that belong to a set S defined in that same work, converge to an equilibrium point different to the origin, as is shown in a) of Figure 2, for

x T 0 = 5 -5 5 ∈ S. (15)
Moreover, when a perturbation ω = 2 + 2 sin(t) 0 0 , the trajectories diverge from the origin, as is shown in b) of Figure the closed loop satisfies the conditions of Theorem 2, and the trajectories converge to the origin with ω = 0 as shown in a) of Figure 3, and to a neighborhood of the origin when ω = 2 + 2 sin(t) 0 0 as shown in b) of Figure 3, which confirms the result of Theorem 3. Note that in this case, the addition of the linear controller prevents the trajectories of the system to escape to infinity in the perturbed case, as opposed to what happens in the uncontrolled case. This is the direct consequence of having achieved an ISS behaviour through the linear control, which is impossible to achieve with a purely switched system.

Conclusions

The stabilization problem of a class of switched systems, in which the switchings occur on the axes the state coordinates is considered. Sufficient condition for the design of a linear stabilizer guaranteeing the convergence of the solutions to the origin are found. The stability proof of the closed loop is performed by means of Linear Matrix Inequalities. Moreover, the ISS property has been quantified, guaranteeing robustness against both matched and unmatched perturbations, and ensuring that the trajectories of the system will remain bounded no matter how large the present perturbation is, provided that it is bounded as well. Using the presented method, for a system whose dynamics are driven by a combination of a linear part and the signs of its state variables, a stabilizing linear feedback, a switching law, or a combination of both can be designed such that the stability of the closed loop can be checked, and its ISS properties established. Some numeric examples are provided validating the theoretical results. In particular, it is shown that for the case of a chain of three integrators, with the signs of its state variables on the last coordinate, the trajectories converge to the origin, and not to another equilibrium point, when a linear stabilizer satisfying the conditions of our main theorem is added. Moreover, it is shown that the addition of a perturbation that causes the uncontrolled trajectories to escape to infinity, do not destroy the stability of the closed loop, and that its trajectories always remain bounded. As directions of future research, the control gain tuning rules development and the robustness margin optimization can be indicated, possibly by solving an optimization problem to choose the constants of the controller matrices, as well as the matrices of the Lyapunov function.

Fig. 1 .

 1 Fig. 1. Simulations results of system (11) with control (12), and with control (13).

Fig. 2 .

 2 Fig. 2. State trajectories of (2) with initial conditions (15) and parameters (14) a) without perturbations, b) with perturbation ω .

Fig. 3 .

 3 Fig. 3. State trajectories of (2) with initial conditions (15) and parameters (14) a) without perturbations, b) with perturbation ω .

  p 11 p 12 . . . p 1n p 12 p 22 . . . p 2n

				 g 1 0 . . . 0	
	. . . p 1n p 2n . . . p nn . . . . . . . . .	   ,	G :=	   	0 . . . 0 g n 0 . . . . . . . . . . . . . . . . . . 0	   

  2. If a control law is chosen as

			unperturbed
	10
	0
					x 1 x 1	x 2 x 2	x 3 x 3
	-10	0	2	4	6	8	10
	10				u 1 u 1	u 2 u 2	u 3 u 3
	0
	-10	0	2	4	6	8	10
				time (sec.)
				perturbed
	40	x 1 x 1	x 2 x 2	x 3 x 3
	20
	0
		0	2	4	6	8	10
	0
	-20
	-40	u 1 u 1	u 2 u 2	u 3 u 3
		0	2	4	6	8	10
				time (sec.)

In Section 3 it was mentioned that the system (2) in its most general form consists of a linear part plus a switched part and it is implied that one of these parts might not be present. It is the same case for the control signal, which could be only a linear control, a switched one, or a combination of both, the restriction being that the closed loop has the form (4), for the nominal case.

7. Acknowledgment This work was supported in part by CONACyT CVU 335111; by the Government of Russian Federation (Grant 08-08) and the Ministry of Education and Science of Russian Federation (Project 14.Z50.31.0031); by Programa de Apoyo a Proyectos de Investigación e Innovación Tecnológica (UNAM) 113216 and by HoTSMoCE INRIA associate team program.