Pin Wang

Ching-Yao Chan

Arnaud De

La Fortelle

A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers

Keywords: Reinforcement Learning, Autonomous Driving, Lane Change, Vehicle Control

Lane change is a crucial vehicle maneuver which needs coordination with surrounding vehicles. Automated lane changing functions built on rule-based models may perform well under pre-defined operating conditions, but they may be prone to failure when unexpected situations are encountered. In our study, we propose a Reinforcement Learning based approach to train the vehicle agent to learn an automated lane change behavior such that it can intelligently make a lane change under diverse and even unforeseen scenarios. Particularly, we treat both state space and action space as continuous, and design a unique format of Q-function approximator to estimate the total return which is an accumulated reward over a lane changing process. Extensive simulations are conducted for training the algorithms, and the results illustrate that the Reinforcement Learning based vehicle agent is capable of learning a smooth and efficient driving policy for lane change maneuvers.

I. INTRODUCTION

Interests in autonomous vehicles have seen great increase in recent years, from automakers to high-tech companies and research institutions. While fully autonomous vehicles at Automation Level 5 per SAE-J3016 [1] may only be widely available in the more distant future, partially or highly automated features at Automation Level 2, 3, or 4 look promising to be commercially available in the near future. Experience from the mass deployment of Advanced Driver Assistance Systems (ADAS) and from extensive tests of autonomous driving show that "steady state" driving is operational and research is now focusing onto "transition" maneuvers that are intrinsically riskier. A study shows that around 10 percent of all freeway crashes are caused by lane change maneuvers [2]. Therefore, we need better algorithms to implement the challenging maneuvers. Lane changing is one fundamental and crucial function expected to be embedded in either an ADAS or a fully automated vehicle. The lane changing maneuver can be a demanding task because the vehicle needs to alertly watch the leading vehicle on its ego lane and surrounding vehicles on the target lane, and to perform proper actions according to the potential adversarial or cooperative reactions demonstrated by those relevant vehicles.

Research on automated lane changing maneuvers has been extensively conducted and the work can broadly be divided into two functional categories: a decision-making module and a control execution module [START_REF] Hidas | Modeling vehicle interactions in microscopic simulation of merging and weaving[END_REF]. A decisionmaking module can be viewed as a strategic or tactical level function which issues a lane change command according to a selected route (e.g. exiting the highway from an off-ramp) or a desired driving condition (e.g. passing a slow vehicle in front). When a lane change command is given, the ego vehicle (i.e. lane changing vehicle) performs operational control to coordinate the longitudinal and lateral movements for a safe, smooth and efficient lane change maneuver.

A considerable body of literature is existent on the topic of the decision-making function, and it generally adopts data-driven methods that are typically based on available training datasets in abundance combined with model-based optimization techniques such as Model Predicted Control (MPC). The topic deserves to be elaborated in depth to discuss how the decision-making functionality can be enhanced. However, to limit the scope of this paper, we will focus on the aspect of operational control, i.e. how the vehicle can automatically perform the lane change maneuver once it receives the command that comes from a decisionmaking module.

With regard to the lane changing maneuver, it is a typical time sequential problem where the completion of the task involves a sequence of actions, and the performance of the current action has an impact on the ultimate goal of the task (e.g. a successful lane change). Such kind of problems are quite suitable to be solved by machine learning techniques, particularly by Reinforcement Learning (RL). The focus of the paper is to demonstrate the use of a Reinforcement Learning concept to find an optimal driving policy for automated lane change maneuvers.

The remainder of the paper is organized as follows. A literature review of related work is given in Section II. Section III introduces the methodology and details of our algorithm. Simulation results are given in Section IV. Concluding remarks and discussions of future avenue of research are provided in the last section.

II. RELATED WORK

Traditional approaches for addressing the autonomous lane changing problem primarily depend on pre-defined rules and explicitly designed models. Most of these approaches introduce a virtual lane change trajectory or a series of way points for the ego vehicle to follow when a lane change process is initiated. In [START_REF] Ho | Lane Change Algorithm for Autonomous Vehicles via Virtual Curvature Method[END_REF], a virtual reference lane change trajectory was established with a polynomial equation. The virtual curvature of the intended path, along with steering angle, was fed into a bicycle model to estimate the vehicle lateral positions. In [START_REF] Choi | Lane Change and Path Planning of Autonomous Vehicles using GIS[END_REF], a number of way points with information acquired from Differential Global Positioning System and Realtime Kinematic devices were used to generate a guidance path when a vehicle embarked on a lane change. A common limitation in these approaches is the lack of flexibility in the planned trajectories under dynamic situations and diverse driving styles. Moreover, though it might work relatively well in predefined situations or within the model limits, it is far from adequate in handling situations that are out of the defined scope. This is also a clear limitation in the current time-receding optimization methods (e.g. MPC) as the optimization criteria may be too complex to be explicitly formulated for all scenarios and such methods always involves the predictions of future trajectories. A promising technique is to control all vehicles simultaneously via cooperative techniques such as in [START_REF] Qian | A hierarchical model predictive control framework for on-road formation control of autonomous vehicles[END_REF], but in the present paper we deal with a stand-alone system.

Machine learning (ML) algorithms have the capability of dealing with unforeseen situations after being properly trained on a large set of sample data without explicitly specific design and programming rules beforehand. Vallon et al. [START_REF] Vallon | A machine learning approach for personalized autonomous lane change initiation and control[END_REF] proposed to use Support Vector Machine to make the lane change decision tailored to the driver's individual driving preferences. After the lane change demand is generated, the maneuver is executed using a MPC. Bi et al. [START_REF] Bi | A Data-driven Model for Lanechanging in Traffic Simulation[END_REF] moved further to apply machine learning algorithms on both the decisionmaking process and the lane changing execution process based on traffic simulation. Specifically, they trained a randomized forest model for decision-making and a neural network for prediction of vehicle velocities at the next step, not only for the ego vehicle but also for the follower vehicle, under an assumption of connected driving environment.

Another study combined the conventional control, a PID controller, with neural networks, for designing a self-tuning and adaptive lateral controller [START_REF] Zhenhai | An Adaptive PID Controller with Neural Network Self-Tuning for Vehicle Lane Keeping System[END_REF]. The inputs to the neural network are optimal lateral acceleration, real lateral acceleration and the error between these two values. This approach should theoretically work well, but may be difficult to achieve due to the difficulty in obtaining optimal acceleration values in real-world driving situations.

Reinforcement learning, one promising category in the machine learning family, has the capability of dealing with time-sequential problems and seeking optimal policies for long-term objectives by learning from trials and errors, without resorting to an off-line collected database. RL has been extensively applied in robotics, e.g. [START_REF] Schulman | High-Dimensional Continuous Control Using Generalized Advantage Estimation[END_REF], and video games, e.g. [START_REF] Schulman | Trust Region Policy Optimization[END_REF]. Recently, it has been used in automated vehicle field [START_REF] Yu | Deep reinforcement learning for simulated autonomous vehicle control[END_REF][START_REF] Sallab | End-to-End Deep Reinforcement Learning for Lane Keeping Assist[END_REF]. However, the driving scenarios in these aforementioned studies were relatively simple because few vehicle interactions with surrounding vehicles were considered.

In one previous study [START_REF] Wang | Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge[END_REF], we used RL to train the vehicle agent to learn an optimal ramp merge driving policy under interactive driving environment. We treated both the state space and action space as continuous to more realistically represent the real-world driving situations. The results demonstrated that with the reinforcement learning approach, the automated merge actions can take place safely, smoothly and promptly under an interactive driving environment. In that work, however, we did not take into account the lateral control issue as we assumed that the agent vehicle would follow the centerline of its travel lane. In this work, we expand the methodology we used in our previous work and adopt the RL architecture for the lane change case with extended definitions in state space, action space and reward function. The design of the learning and control is kept relatively simple in this preliminary study but it would prove the same methodology could work with more complex states and actions, indicating promising perspectives.

III. METHODOLOGY

In a typical RL problem, the goal is to find an optimal policy 𝜋 * which maps the states ( 𝑠 ∈ 𝑆 ) of the environment into actions (𝑎 ∈ 𝐴) that the agent takes at the corresponding states, in a way of maximizing a total expected return 𝐺 that is a cumulative sum of immediate rewards 𝑟 received over the completion of a task [START_REF] Sutton | Book: Reinforcement Learning: An Introduction[END_REF]. 𝑆 and 𝐴 here are state space and action space, respectively, and can be discrete or continuous in a specific problem. The reward 𝑟 𝑠, 𝑎 reflects the effects of an action 𝑎 in a given state 𝑠 , whereas 𝐺 𝑠, 𝑎 is a long-term reward starting from a state 𝑠, taking an action 𝑎 and thereafter following a policy 𝜋.

In our case, the driving environment involves the interaction with other vehicles whose behaviors may be cooperative or adversarial. For example, when a vehicle reveals its intention of a lane change by turning on the turning signal, the lag vehicle on the target lane may cooperatively decelerate or change its path to yield, or it may adversarially accelerate just to deter the vehicle from cutting into its course of motion. Consequently, it is difficult to model the environment with explicitly all possible future situations. Thus, we resort to a model-free approach to find the optimal policy. Another point to adopt a RL based approach is that a traditional MPC based controller often uses explicitly defined sensor inputs, while the outcome from an image perception module is usually an extremely large feature map and might be fuzzy. MPC has difficulty in handling such a large and fuzzy set of inputs. In contrast, the RL agent can take in hundreds or even millions of features as inputs which do not necessarily have an explicit representation. In this sense, the RL approach can conveniently be connected with the perception module.

Besides, from another perspective of a joint approach, a RL/ML module can be taken as a mediator component between a perception module and a traditional MPC module, as to take the perception outcome as input and output a reference guidance for the MPC controller.

Lane Change Controllers

It's a good practice to control a vehicle in a coupled way in a MPC controller, but it is also common to decouple the complicated vehicle control into a longitudinal control module and a lateral control module for the study of vehicle maneuvers [START_REF] Ho | Lane Change Algorithm for Autonomous Vehicles via Virtual Curvature Method[END_REF]. To explore the learning ability of a RL based vehicle agent and demonstrate how the functionalities can be partitioned and addressed separately, we design decoupled controllers in our current work. Investigation of coupled controllers is also worth pursuing and we will defer it to our future work.

Since certain off-the-shelf models are ready for use in application, we choose to leverage a well-developed carfollowing model, Intelligent Driver Model (IDM), to build the longitudinal controller. The lateral control is to be learned by RL with the consideration that most of those previously proposed lateral control models are far too theoretical or empirical to be applied on an autonomous vehicle.

There is also a gap selection module working parallelly with the two controllers. After the vehicle get a lane change command, the gap selection module will check the acceptable safety distance to the leader and follower on the target lane based on all the current information (e.g. speed, acceleration, position, etc.) of the surrounding vehicles. If the gap is adequate enough to accommodate the speed difference under maximum acceleration/ deceleration and ensure minimum safety distance under current speed, it is considered as an acceptable gap, and then the lane change controllers will be initiated. The two controllers will interact with each other and function cooperatively to perform the overall lane change task while they are designed separately as individual modules.

IDM based Longitudinal Controller

IDM is a time-continuous car-following model for the simulation of highway and urban traffic. It describes the dynamics of relevant vehicles. Detailed information can be found in [16].

The IDM holds realistic properties compared with other intelligent models such as Gipps' model [START_REF] Gipps | A model for the structure of lane-changing decisions[END_REF]. But if it is applied directly, a phenomenon will appear where vehicles may travel at a relatively slow speed under moderate traffic conditions due to the safety constraint inherited in the equation, thereby, we modify the IDM to suit our purpose. Since it is not the focus of this paper and the content space is limited, the details on the formulation and the illustrations of vehicle dynamics in car-following scenarios are omitted here. But it is still worth mentioning that with the modified car-following model, the vehicle can perform reasonable longitudinal behaviors either in free traffic conditions or in interactive driving situations.

It is also worth noting that when the ego-vehicle is making a lane change, it may see two leading vehicles in the ego lane and the target lane during the transition. The IDM car-following model that we implement will allow the ego-vehicle to adjust its longitudinal acceleration by balancing between its two leaders, if observed, on its ego lane and the target lane. The smaller value will be used to weaken the potential discontinuity in vehicle acceleration incurred from lane change initiation. At the same time, the gap selection module is still working as a safety guard during the whole lane changing process to check whether the gap distance is still acceptable at each time step. If not, the decision-making module will issue a command to alter or abort the maneuver, and the control execution module will reposition in the original lane. In this way, the longitudinal controller takes the surrounding driving environment into account to ensure safety in the longitudinal direction, whereas the lateral controller directs the vehicle to intelligently merge into any accepted gap.

RL based Lateral Controller

In this section, we define the state space 𝑆, the action space 𝐴, the immediate reward function 𝑟 𝑠, 𝑎 , and a model-free approach, Q-learning, to find the optimal policy.

Action Space

In some RL studies, the action space is usually treated as discrete to make a problem more easily solvable, however, it might weaken the feasibility of the solution when applied on real-world problems. In our study, the lateral control in a lane changing process is of crucial importance since a slightly fallacious shift in steering may result in the vehicle slipping out of the lane or a significant disturbance of surrounding vehicles. Bearing this in mind, we design the lateral controller with a continuous action space to allow a realistic and smooth transition from one lane to another.

To make sure that the steering angle input is continuous and smooth, there should be no abrupt change in yaw rate or, in other words, the yaw acceleration does not fluctuate erratically. Thereby, we design the RL agent to learn the yaw acceleration, i.e., the action space is defined with vehicle yaw acceleration 𝑎 +,-.

𝑎 = 𝑎 +,-∈ 𝐴

(1)

State Space

As mentioned earlier, the ego vehicle's interaction with surrounding vehicles is taken into account by the gap selection module and longitudinal controller, the lateral controller thereby only considers the ego vehicle's motion. For a coupled controller to be developed in the future, all the relevant adjacent vehicles' state as well as the ego vehicle's will be incorporated into the state space.

A successful lane change not only relates to vehicle dynamics, but also depend on road geometry, i.e., whether a lane change is performed on a straight segment or a curve. In our study, we define the state space with both vehicle dynamics and road information. To be specific, the state space includes the ego vehicle's speed 𝑣, longitudinal acceleration 𝑎, position 𝑥, 𝑦 , yaw angle 𝜃, target lane 𝑖𝑑, lane width 𝑤, and road curvature 𝑐. 𝑠 = 𝑣, 𝑎, 𝑥, 𝑦, 𝜃, 𝑖𝑑, 𝑤, 𝑐 ∈ 𝑆 (2) In this problem formulation, we suppose the required state information is available from on-board sensors, such as GPS, IMU, radar, CAN, etc., on instrumented vehicles, and that it meets the desired accuracy requirements for the control purpose.

When the state features are from an image perception module or incorporate measurement noise, the state space here can be adaptively expanded to a large size without changing the algorithm structure, which is one superiority of the RL approach.

Reward Function

Typically, in a lane change process our attention is on the safety, smoothness and efficiency of the maneuver. As the longitudinal module and the gap selection module have taken into account the safety concern, smoothness and efficiency are considered by the lateral controller through the reward function.

The smoothness is evaluated by yaw acceleration 𝑟 ,778 due to the consideration that yaw acceleration directly affects the magnitude of a shifting in the lateral movement.

𝑟 ,778 = 𝑤 ,778 * 𝑓 ,778 𝑎 +,-

where 𝑟 ,778 represents the immediate reward obtained from yaw acceleration, 𝑤 ,778 is a weight which can be designed as a constant value or a function relevant with lateral position, and 𝑓 ,778 is a function for evaluating 𝑎 +,-. We currently use a simple format as 𝑟 ,778 = -1.0 * 𝑎 +,-.

Another indicator of smoothness is the yaw rate 𝜔 +,-, which reflects the comfort of the driver in a lane change process since a higher yaw rate will result in a significant centrifugal pull force in driving. The function is given in (4). 𝑟 ?,@8 = 𝑤 ?,@8 * 𝑓 ?,@8 𝜔 +,-

where 𝑟 ?,@8 reflects the immediate reward obtained from yaw rate, 𝑤 ?,@8 is a weight and can also be designed as a constant or a function, and 𝑓 ?,@8 is a function for evaluating 𝜔 +,-. In our study, we apply 𝑟 ?,@8 = -1.0 * 𝜔 +,-.

The efficiency is assessed by the lane changing time consumed to complete the maneuver. We take it into consideration into the reward by adding a third term to the reward function, as to avoid an overly long and extended lane change actions.

𝑟 @AB8 = 𝑤 @AB8 * 𝑑𝑡 (5)
where 𝑟 @AB8 is the immediate reward from lane changing time, 𝑑𝑡 is the time step interval, and 𝑤 @AB8 is the weight that we design as a function of the proportion of the current lateral deviation and the averaged lateral deviation and scale it with 0.01 to a similar magnitude with the other two reward parts: 𝑤 @AB8 = -∆𝑑 E,@ 𝑑 ,FG * 0.01.

The immediate reward 𝑟 in a single step is a summation of the three parts. To assess the overall performance, we also need to calculate the total reward 𝑅 that is a cumulative return of immediate rewards over a lane changing process. Equally, the total reward can also be viewed as a composition of the three aforementioned individual parts: the total reward from yaw acceleration, the total reward from yaw rate, and the total reward from lane changing time, as shown in [START_REF] Qian | A hierarchical model predictive control framework for on-road formation control of autonomous vehicles[END_REF].

𝑅 = 𝑟 ,778 A I AJK + 𝑟 ?,@8 A I AJK + 𝑟 @AB8 A I AJK (6)
It can be observed that we define the rewards with negative values. The idea is that a reward can also be considered as a penalty of an action if we want to emphasize the adverse effect from an action, such as unsmooth or inefficient lane change maneuver. In this way, the agent should be able to learn to avoid taking actions that result in a large penalty.

Q-learning

Q-learning is a model-free reinforcement learning technique which is used to find an optimal action selection policy [18] through a Q-function by estimating the value of the total return, without waiting until the end of an episode to cumulate all the rewards.

For continuous state space and/or action space, the standard Q-learning which basically maps discrete state/action pairs to Q-values will lose viability with the increased size of state/action pairs. An alternative solution is to use a Q-function approximator, normally devised as neural networks, to output the Q values. This makes it possible to apply Q-learning to large table size problems, and it even works when the state and/or action space is continuous.

The problem then becomes how to design an appropriate Q-network that is representative and learnable. In our work, we utilize a quadratic function as the Q-function approximator to learn the optimal lane changing action with the consideration that any continuous function can be approximated by a finite number of terms of Taylor series [19].

For most cases, a polynomial of degree two is enough to describe the inherent features of a problem with relatively small errors [19]. If a two-degree Taylor polynomial is expanded at its optimal, the first order would be zero, thus, we formulate the Q-function approximator as a quadratic function with the format in [START_REF] Vallon | A machine learning approach for personalized autonomous lane change initiation and control[END_REF].

𝑄 𝑠, 𝑎 = 𝐴 𝑠 * 𝐵 𝑠 -𝑎 O + 𝐶 𝑠 [START_REF] Vallon | A machine learning approach for personalized autonomous lane change initiation and control[END_REF] where 𝐴, 𝐵 , and 𝐶 are coefficients and designed with neural networks with the state information as input, as illustrated in Fig. 1.

Specifically, 𝐴 is designed with a two-layer neural network with eight neurons in the input layer (i.e. the dimension of the state space 𝑆) and 100 neurons in the hidden layer. Particularly, 𝐴 is bounded to be negative with the use of an activation function on the output layer. 𝐶 is also a two-layer neural network with the same number of neurons and layers as 𝐴, but it also takes in a terminal state as an indicator of the completion of a lane change. 𝐵 is in a heavily engineered form with three twolayer neural networks, which have the same structure of 8 neurons in the first layer and 150 neurons in the hidden layer, linked together to output the final yaw acceleration value. One neural network is for the calculation of yaw acceleration, another is for the calculation of a sensitivity factor used to evaluate the yaw acceleration variation in two consecutive steps, and the other one is for the calculation of the maximum yaw acceleration that is used to give an adjustable boundary of the yaw acceleration. Fig. 2 illustrates the structure of 𝐵.

With the above design, the optimal action for a given state can be obtained from 𝐵. In the meanwhile, Q-values are calculated with 𝐴, 𝐵, and 𝐶 , and used for weights update of the neural networks based on a loss defined between target Q-values and predicted Q-values that are calculated in a RL learning process called experience replay [START_REF] Adam | Experience replay for realtime reinforcement learning control[END_REF].

IV. SIMULATION AND RESULT

We test our proposed algorithms through a simulation platform where a learning agent is able to, on one hand, interact with the driving environment, and on the other hand improve itself by trials and errors from experiments. The simulation scenario is a highway segment with three lanes on one direction. The segment length is 1000m and each lane width is 3.75m. Traffic on the highway can be customized as needed. For example, the initial speed, the departure time, and the speed limit of an individual vehicle can all be set to a random value within a reasonable and practical range. In our current study, the departure interval is between 5s-10s, and individual speed limits are in a range of 80km/h-120km/h. All vehicles can perform practical car-following behaviors with the proposed IDM. Diverse traffic conditions can be generated with different sets of parameters. A scene of the simulation scenario is shown in Fig. 3.

In the training, we set the training steps as 40,000, the time step interval 𝑑𝑡 as 0.1s, the learning rate 𝛼 as 0.01, and the discount factor 𝛾 as 0.9. In total around 5000 vehicles executed the lane change maneuver in the training process. The training loss and Q-values gathered during training are shown in Fig. 4.

From the left graph in Fig. 4, we can observe that the loss curve shows an obvious convergence along with training steps. The Q-value graph in Fig. 4 In this work, we applied the Reinforcement Learning approach for learning the automated lane change behavior under interactive driving environment. The state space and action space are both treated as continuous to learn a more practical driving maneuver. A unique format of a quadratic function is used as the Qfunction approximator, in which the coefficients are learned from neural networks. The reward function is defined with yaw rate, yaw acceleration and lane changing time for training a smooth and efficient lane change behavior.

To implement the proposed approach, we developed a simulation platform where diverse simulation scenarios can be generated by adjusting traffic density, initial speeds, speed limit, etc. Preliminary training results showed convergence of the learning, indicating a promising attempt. The next step for our research is to first test the RL agent under different road geometries and variant traffic flow conditions, in order to enhance its robustness and adaptability in complicated driving scenarios, which is a most advantageous feature of this method to deal with. Second, the lane changing performance (e.g. lateral position deviation and steering angle, etc.) along with the convergence trend will be compared with existing optimization-based approaches such as MPC, for validation and verification. In addition, another promising enhancement is to combine RL and MPC to make the best of both approaches. The proposed architecture can be established by extending the first layer of the neural networks to accommodate a large and fuzzy input state from an image module, and then output a reference guidance for a traditional optimization-based controller to issue a quick and reliable control command to the vehicle.

Figure 1 .Figure 2 .

 12 Figure 1. Structure of the Q-function approximator

  demonstrates the distributions of Q-values at each step, and it clearly shows that Q values converge to a steady level. The training results indicate that the RL based vehicle agent is capable of learning the lane change policy with regard to our designed reward function under the proposed Qfunction approximation architecture.

Figure 3 .

 3 Figure 3. Simulation scenario

Figure 4 .

 4 Figure 4. RL training loss and Q-values V. CONCLUSION AND DISCUSSION


Géraldine Delley¹

Sébastien Plutniak²

Keywords: archives, controversy, historiography, history, history of archaeology, interdisciplinarity, reflexivity, social history of science, social studies of science, sociology

The relationship between archaeology and other sciences has only recently become a research topic for sociologists and historians of science. From the 1950s to the present day, different approaches have been taken and the aims of research studies have changed considerably. Besides methodological textbooks, which aim at advancing archaeological knowledge, historians of archaeology have tackled this question by exploring the development of archaeology as a scientific discipline. More recently, collaborations between archaeologists and other scientists have been examined as a general phenomenon regarding transfers of knowledge and power relationships between specialists, organizations, and scientific tools, where archaeology is considered as a scientific practice. Adopting a sociohistorical perspective, this entry examines the specificity of aims, facts, and procedures shared by archaeologists and other scientists regarding the crucial question of measuring time and computations.

Archaeology and sciences or archaeology as science?

From the late nineteenth to the early twentieth century, human and natural sciences were gradually reorganized into institutionalized forms. The diversification of research trends in archaeology, a trend also observed in other sciences, led to a number of ways to refer to the natural sciences, which ranged from using them as a model of scientificity to defining the difference between them and the human sciences. Local histories of scientific institutions, research traditions, and political and personal choices alike influenced the position of archaeology within the scientific field-a position that could be closer to or more distant from the human and natural sciences. The case of prehistoric archaeology highlights this crossing over of branches in the classification of sciences. The development of prehistory as a scientific practice in the second half of the nineteenth century relied on a wide range of approaches, including ethnography, history, physical anthropology, paleontology, geology, and botany, from which prehistory borrowed methodological and conceptual tools. In the nineteenth century, archaeology (see archaeology) and prehistoric archaeology were part of a large research field that encompassed geography, physical anthropology, linguistics, and ethnography and aimed to describe and understand human history from natural and cultural perspectives [START_REF] Blanckaert | 1800 -Le moment 'naturaliste' des sciences de l'homme[END_REF]. Consequently there is an intrinsic relationship between archaeology and science: the aim of producing archaeological knowledge is in itself a scientific project, and it is preferable to talk about the relationships between archaeology and other sciences.

The term "archaeosciences" is loaded with the same content: if a part of archaeological practice is scientific (the "archaeosciences"; see archaeological sciences), what about the other part? The role of historians and sociologists of science is not to find the best solution to this problem, but rather to investigate the variations in the definitions of these terms. An important issue is to identify the relationship (in terms of epistemic and social dimensions) between archaeological practices and practices in the sciences applied to archaeologyin other words, to explore the diachronic and synchronic variations in the ways archaeology has been grounded as a scientific enterprise and the controversies surrounding this topic. Indeed, the concept of archaeoscience has its own history and is closely related to the concept of archaeometry (see archaeometry).

A brief examination of the evolution of the latter can clarify the relationship between archaeology and other sciences. It is important to understand the meaning of -metry in archaeometry. The term started to become popular in the 1960s, and in 1961 the first "Symposium on Archaeometry" was held in London; it was an annual meeting that focused on the applications of physical science in archaeology. The growth of archaeometry increased rapidly in the mid-1970s. From 1975 onward, the location of the symposium changed every year; the symposium itself became international and renamed "International Symposium on Archaeometry and Archaeological Prospection. " This internationalization was also reflected in the creation of the PACT group in 1976, by the Council of Europe. This group was defined as a European study group dedicated to the study of physical, chemical, biological, and mathematical techniques applied to archaeology. From 1977 on, this group published its own journal, the eponymous PACT ; at the same time, in 1976 the Groupe des méthodes physiques et chimiques de l'archéologie (GMPCA) was founded in France in 1976 and started issuing its own journal, Revue d'archéométrie, one year later. These three cases illustrate different definitions of archaeometry: in the UK symposia and in the French GMPCA, mathematics was included only insofar as it provided methods to process the results of physical and chemical measurements; on the contrary, in the European PACT group, mathematics was an integral part of the group's definition.

The concept of measurement is relevant to the measurement of abstract properties in mathematical objects and of concrete properties in physical objects. From the 1970s onward, however, archaeometry was concerned mainly with the measurement of concrete properties; it excluded mathematics and restricted itself to physical and chemical procedures designed to investigate archaeological facts. This evolution was also related to the professionalization of archaeology, the growth of archaeometry as a specialized field, and the redefinition of the relationships between archaeologists, mathematicians, physicists, and chemists. The extensive scope and multidisciplinary nature of archaeometry and of the archaeosciences illustrate the inadequacy of an analysis based only on a normative classification of the sciences and the relevance of a sociohistorical inquiry on this issue.

The first section of this entry focuses on the different approaches taken by authors who assess the relationship between archaeology and other sciences from social and historical perspectives, and on the evolution of these approaches. The evolution of these authors' aims is also examined. The second section focuses on a long-term perspective. Since the nineteenth century, archaeological research has integrated a range of aims, facts, and procedures borrowed from or shared with other scientific disciplines.

Scientific transfers and applications in archaeology: an overview of social and historical approaches

During the twentieth and twenty-first centuries, the position that authors occupied in their discipline and the nature of their work on the history of the relationship between archaeology and other sciences can be classified into three main categories. A first group is formed by internalists: archaeologists who construct the history of their own discipline and practices, and aim to advance archaeological knowledge. The second group is formed by archaeologists who are concerned with the history of their discipline as a social practice (see scientific practice). The third group consists of externalists: historical and sociological analyses of the relationships between archaeology and other sciences are produced by historians, sociologists, or ethnologists positioned outside the field of archaeology. They aim to advance the knowledge of general phenomena related to past and present activities based on the case of archaeology.

Progress in archaeology: archaeology's relationships with other sciences

From the first decades of the twentieth century to the present day, many archaeologists have published reviews or methodological textbooks that aim to describe the diversity of tools available from the natural and formal sciences, as well as their limitations in archaeology. These contributions, assessing the relationship between archaeology and other sciences, aimed to strengthen the archaeological discipline and to support the research of archaeologists. In addition, they were conceived as a means of exposing the progress of the discipline, since they coincided with the first attempts at professionalizing prehistory (in the 1920s) and with the development of technical tools in physics, chemistry (see chemistry in archaeology), and isotope (see isotopes) research (in the 1950s). An emblematic contribution is Annette Laming's book The Discovery of the Past: Recent Progress and Techniques in Prehistory and Archaeology (published in French; see Laming-Emperaire 1952).

Archaeologists examining their discipline as sociologists or historians

Since the 1990s, the relationship between archaeology and other sciences has been investigated by archaeologists influenced by or aware of approaches developed in cultural and social history and in the history of science. Examining the institutions, the actors, and their practices, these authors have tackled the relationship between archaeology and other sciences within the context of formation and development of archaeology as a scientific discipline (Hurel and Coye 2011; Kaeser 2011). More recently, this relationship has been approached from the point of view of the development of specific tools and methods (e.g., 14C dating, dendrochronology, fluorine) that involve archaeology and natural and formal sciences, in a context of strong disciplinary affirmation [START_REF] Delley | Au-delà des chronologies. Des origines du radiocarbone et de la dendrochronologie à leur intégration dans les recherches lacustres suisses[END_REF]. Such researches focus on the behavior of the scientists and assess their actions in complex social interactions (e.g., power relations between disciplines and institutions, the role of politics in science, the relation between science and the army, medicine, culture). The use of archives, the contextualization of observations, as well as the historicization of the categories commonly used in archaeology (e.g., dating, modernity, interdisciplinarity, amateur, professional: Plutniak 2017) are constitutive of this approach, which is recognized either as a component of archaeology or as a part of the history of science.

Archaeology as a case study for social studies of science

Since the 1940s and following the pioneering work of Robert Merton, social and historical studies of science grew in number and were empirical investigations of various disciplines. Until the present day, however, archaeology has rarely been investigated by sociologists or science historians. In fact, social studies of sciences have focused mainly on natural, formal, and experimental sciences; one explanation is that archaeology is a newer discipline by comparison to physics, mathematics, or physiology. Nevertheless, there are some studies that have addressed sociological questions regarding the relationship between archaeology and other sciences. An example is the sociology of expertise developed in [START_REF] Bessy | Une collection inqualifiable. La controverse sur l'authenticité de Glozel[END_REF]. This work investigates the case of Glozel, an alleged forgery site, and shows how arguments grounded on the use of radiocarbon (see radiocarbon calibration and age estimation) and thermoluminescence (see dating ceramics and luminescence) can be controversial. The sociolinguistic approach is another example [START_REF] Goodwin | Practices of Seeing. Visual Analysis: an Ethnomethodological Approach[END_REF]. Dealing with ethnomethodological questions related to 3 the role of vision (or visual experience) in the analysis of situated practices, Goodwin analyzed how archaeologists decide the description and colors of a stratigraphy. During the 1990s, the influence of postprocessual archaeology and its particular relation to ethnography and reflexivity (see symmetrical archaeology) blurred the frontier between archaeologists analyzing their own discipline and sociologists or historians examining archaeology as a case study. An example is the collective book on the ethnography of archaeology (see ethnography and qalitative methods in the field) edited by Edgeworth, an archaeologist, which collects papers from archaeologists, sociologists, curators and linguists [START_REF] Edgeworth | Ethnographies of Archaeological Practice. Cultural Encounters, Material Transformations[END_REF].

The relationships between archaeology and other sciences: examining time

The relationship between archaeology and other sciences encompasses a wide range of scientific practices and relationship modes, for example transfer of concepts (by analogy or metaphor), methods (devices or skills), or knowledge aims. To give a short but broad example, this section focuses on the case of examining time (see time and temporality) by distinguishing three main categories of scientific methods of inquiry: observing (the effects of time), measuring (the effects of time), and computing (temporal data).

Observing the effects of time

Since the second half of the nineteenth century, observing the effect of time has been a central question in prehistory. Archaeologists expected the natural and formal sciences to devise methods to resolve this issue, in addition to the usual tools of classification (see archaeological classification) of artifacts and stratigraphy. In the early nineteenth century, the paleontologist Georges Cuvier and the mineralogist Alexandre Brongniart defined a methodological tool where faunal characterizations were used to establish a relative chronology between different stratigraphical layers. A few decades later, a scheme for recognizing and characterizing the successive stages of the development of material culture was established by Christian Jürgensen Thomsen and Jens Jakob Asmussen Worsaae on the basis of the collections stored at the National Museum of Copenhagen. Named the "three-age system, " this conceptual tool relies on the morphological and technological analysis of archaeological material. The composition of this material (stone, bronze, or iron) establishes a division of time. In the 1840s the demonstration of the antiquity of "man" by Jacques Boucher de Perthes was built on naturalist and cultural principles involving paleontological, geological, anthropological, and cultural observations (see material culture) and established the first elements of what would become the scientific practice called prehistory. At the same time, in Denmark, a commission of experts composed of archaeologists and naturalists explored the famous kitchen midden (kjoekkenmoedding), whose chronological attribution was highly problematic. Around the 1870s, Gabriel de Mortillet used in his classification of prehistoric industries paleontological stratigraphy and the "three-age system. " Naturalist and cultural conceptions are built using evolution theory, which is based on geological and paleontological evidence applied to the morphological classification of human artifacts (Hurel and Coye 2011).

Since the late nineteenth century, wetland zones-marshes, bogs, and lakeshores where organic remains are particularly well preserved-have been locations of intense exchanges between naturalists and archaeologists. During the first half of the twentieth century, these collaborations took institutionalized forms with the creation of laboratories and institutes in Tübingen, Groningen, and Basel that aimed at including botany and palynology-besides sedimentology, paleontology, and material culture-in the practice of archaeology. The history of vegetation and climate (see climate change and dating), combined with the fine analysis of stratigraphies and artifact typologies, stems from the same preoccupation of observing the effect of time from a natural and cultural perspective.

Measuring the effects of time

The introduction of measurement in archaeology was an important methodological and epistemological step (see measurement theory); furthermore, it also divided actors according to whether they could perform the method in question or not. These measurements concerned mainly the properties of cultural artifacts and organic and physical facts to the extent that they could reveal the effects of time.

A first case is the measurement of cultural assemblage properties. An example is the early development in seriation (see seriation methods), introduced by Flinders Petrie in the late nineteenth century. Petrie also authored one of the first extensive applications of mathematics to archaeological facts in his book Inductive Metrology, published in 1877. Seriation methods consider a set of objects and measure the properties of each (for instance types of potteries).

The variations observed in the targeted set lead the observer to infer the relative chronology of these objects. American archaeologists largely adopted and improved statistic-based seriation methods (see descriptive statistics), notably in the work done in the 1910s in the Pueblo and Zuni regions by researchers such as Alfred Kidder, Alfred Louis Kroeber, and Clark Wissler, who integrated these methods within an evolutionary (see evolutionary theories) framework (see [START_REF] Lyman | Measuring Time with Artifacts[END_REF].

Dendrochronology, which was introduced into Europe by Ebba Hult de Geer and Bruno Huber during the 1930s and 1940s, in botany (see charcoal and wood analysis), is at the frontier of observing the effect of time and measuring it. Dendrochronological curves can be built, and thus historical sites dated, by comparing and correlating the patterns of tree rings. The use of this method in dating prehistoric settlements was limited to establishing (although very precisely) relative chronologies until the 1980s.

Since the 1950s, radiocarbon dating has been considered, in textbooks on method, the emblematic example of a collaboration between archaeology and other sciences. This tool, developed shortly after World War II, determines the age of an object by measuring its radioactive decay. Analyses are performed in physical laboratories disconnected from the archaeological milieu. In addition, the tool was quickly presented as being "under control" by laboratories, which expected a direct correlation between the 14C decay curve and the calendar time curve. This assumption relied on the false hypothesis that the production of 14C was constant through time. Statistical and computational methods of calibration (see radiocarbon calibration and age estimation) were developed to solve this problem of the fluctuating 14C production in the atmosphere.

Time computation

During the second half of the twentieth century electronic computers became more powerful. This advancement led to redefinitions of previous methods: the research problems might have been the same, but the methods used to solve them radically changed, as computational complexity became less of a limitation (see computer applications in archaeology). The main difference was that calculations were not made on the observed properties of archaeological facts, but directly on calendar data integrated into an abstract statistical time model. A shift occurred from dealing with representations of temporal effects to dealing with representations of time, which allowed for the integration of multiple dating sources in the same formal framework. Regarding the development of dendrochronology in Europe, where climatic signals are weaker than in southwest America (where the tool was first developed by Andrew Ellicott Douglass), local dendrochronological curves can be inserted in a long referential curve, which connects the present day to prehistoric times and provides archaeologists with an absolute dating tool that necessitates the use of statistic computations. Physicians and statisticians developed Bayesian (see bayesian statistics) frameworks that allowed the calibration of unique datings and the integration and correlation of multiple dating sources (radiocarbon, dendrochronology, thermoluminescence). From the 1970s on, an important chronometer was developed on the basis of ancient DNA (see genetics: ancient and genomics: ancient) and of the known mutation rate of biological molecules. The "molecular clock" hypothesis and method offered a new way to estimate time in prehistoric archaeology. Both DNA sequencing (see dna: next generation seqencing) and the calibration of the relative duration provided by the molecular clock rely on computational techniques.

In fact, the difference between observing and measuring the effects of time on the one hand and analyzing temporal data on the other is not only a matter of methods or concepts. The introduction of new methods to investigate time in archaeology also redesigned the organization and practices of archaeology, modified its boundaries, defined new powers and relationships between archaeologists and specialists in other disciplines, and changed the image of archaeology in society. Archaeosciences and archaeometry have for instance connected archaeology with the iconic place of modern science: the laboratory [START_REF] Delley | Au-delà des chronologies. Des origines du radiocarbone et de la dendrochronologie à leur intégration dans les recherches lacustres suisses[END_REF]. These aspects are empirically investigated by social scientists and historians.


Philip Polack

Florent Altché

Brigitte D'andréa-Novel

Arnaud De

La Fortelle

Guaranteeing Consistency in a Motion Planning and Control Architecture Using a Kinematic Bicycle Model

Planning and control for autonomous vehicles are often treated as two independent problems in the literature. However, due to their level of abstraction and modeling, they are in fact highly intertwined: for example, the controller tracks the output trajectory of the motion planner without taking into account the surrounding obstacles. Therefore, an ill-designed planning and control architecture of the vehicle might lead to hazardous situations such as infeasible trajectories to follow. Some existing works in the literature have considered both problems simultaneously, but usually they either lack some guarantees on the feasibility of the computed trajectory or are non robust to modeling errors. Therefore, the present paper proposes to combine a 10Hz motion planner based on a kinematic bicycle Model Predictive Control (MPC) and a 100Hz closed-loop Proportional-Integral-Derivative (PID) controller to cope with normal driving situations. Its novelty consists in ensuring the feasibility of the computed trajectory by the motion planner through a limitation of the steering angle depending on the speed. This ensures the validity of the kinematic bicycle model at any time. The architecture is tested on a high-fidelity simulation model on a challenging track with small curve radius, with and without surrounding obstacles.

I. INTRODUCTION

As research on autonomous vehicles is getting more and more mature, the question of consistency between its different layers, namely perception, localization, planning and control, is becoming crucial to ensure the safety of the vehicle at all time. An ill-designed vehicle architecture might be very critical for its safety, even though each layer is welldesigned independently.

In this paper, we propose to focus on the problem of consistency between planning and control. Motion planning and control problems are two different but highly related problems. The first consists in computing a feasible trajectory (in terms of vehicle's dynamics) for the vehicle avoiding the surrounding obstacles such as other vehicles, pedestrians, or non moving objects. The second is acting on the actuators, i.e. the gas pedal, brake pedal and steering wheel, in order to track the trajectory obtained by the motion planner, while ensuring the stability of the system and, if possible, a smooth drive. Therefore, the properties of the design of each layer are quite different.

For motion planning purpose, the algorithm will have to explore the space of feasible solutions, which is computationally expensive. Thus, it requires both a rather simple model of the vehicle, such as a point-mass [START_REF] Nilsson | Longitudinal and lateral control for automated lane change maneuvers[END_REF] or a kinematic bicycle model [START_REF] Qian | A Hierarchical Model Predictive Control Framework for On-road Formation Control of Autonomous Vehicles[END_REF], and a rather low-frequency (around 5-10Hz). On the contrary, control is usually operating at highfrequency (around 100Hz) to ensure a good tracking of the reference trajectory and to be robust to modeling errors and disturbances. Moreover, the level of abstraction also differs: dealing with obstacles is one of the main task of the motion planner while the controller usually completely ignores them; the trajectory given by the motion planner is assumed to be safe within a certain margin and the goal of the controller is then to follow as well as possible the given trajectory, without considering the obstacles. Therefore, if the motion planner is planning outside the validity domain of its model, it might compute unfeasible trajectories from a dynamical point of view as shown in Figure 1. Then, the controller will not be able to follow the trajectories, putting the whole system in jeopardy! Fig. 1: Dynamic feasibility of motions considered as "admissible" for a kinematic bicycle model at a given speed V 0 in the physical space.

The problem of consistency, also referred to as "proper modeling" in the literature [START_REF] Ersal | A Review of Proper Modeling Techniques[END_REF], arises with the need for simple models for motion planning: there is a trade-off between the complexity (i.e. the computational time) and the accuracy of the model. It has been studied in the literature by comparing simple models with more complex ones. For example, [START_REF] Liu | The Role of Model Fidelity in Model Predictive Control Based Hazard Avoidance in Unmanned Ground Vehicles Using LIDAR Sensors[END_REF] compares a motion planner and control MPC using a 2 Degrees of Freedom (2 DoF) linear-tire dynamic bicycle model with a 14 DoF vehicle model. The validity of this dynamic bicycle model is guaranteed by constraining the lateral acceleration to 0.5g on normal road conditions, condition which has been derived in [START_REF] Park | Obstacle avoidance of autonomous vehicles based on model predictive control[END_REF]. However, the practical implementation of this system on a real vehicle is limited as it assumes a constant velocity. [START_REF] Kong | Kinematic and Dynamic Vehicle Models for Autonomous Driving Control Design[END_REF] compares the performances of MPC using a kinematic bicycle model and one using a linear-tire dynamic bicycle model. The authors conclude that the kinematic bicycle model works better in most cases, except at high-speed, and suggests a further investigation on the impact of different lateral accelerations on the validity of the kinematic bicycle model. This has been done in our previous work [START_REF] Polack | The Kinematic Bicycle Model: a Consistent Model for Planning Feasible Trajectories for Autonomous Vehicles?[END_REF] by comparing it to a high-fidelity 9 DoF vehicle model on constant radius curves. The conclusion was that the lateral acceleration should always remain lower than 0.5µg, where µ is the road friction coefficient.

In the literature, many works can be found on motion planning and control architectures. They can be classified into two categories [START_REF] Berntorp | Path planning and integrated collision avoidance for autonomous vehicles[END_REF]. In the first category, motion planning and low-level control are unified into one unique MPC formulation using different vehicle models, such as in [START_REF] Abbas | Obstacle avoidance in real time with Nonlinear Model Predictive Control of autonomous vehicles[END_REF], [START_REF] Jiang | Obstacle avoidance of autonomous vehicles with CQP-based model predictive control[END_REF] and [START_REF] Liniger | Optimization-based autonomous racing of 1:43 scale RC cars[END_REF]. However, they are not robust to modeling errors, disturbances and parameter uncertainties as they lack a realtime feedback controller (the frequency of recomputing is rather low for low-level control). Thus, for stability reasons, it is preferable to use a high-frequency closed-loop controller. The second category separates motion planning from lowlevel control to overcome this problem (see for example [START_REF] Rosolia | Autonomous Vehicle Control: A Nonconvex Approach for Obstacle Avoidance[END_REF], [START_REF] Li | Development of a new integrated local trajectory planning and tracking control framework for autonomous ground vehicles[END_REF]). However, in that case, there are no guarantees on the feasibility of the planned trajectory as the model used for motion planning might be unable to capture the admissible motions of the vehicle. This problem has not been encountered yet as the number of tested kilometers with such architecture is small and the driving strategy chosen often very conservative, but it is very critical for the vehicle safety.

Therefore, this paper proposes a simple planning and control architecture for normal driving situations that guarantees the dynamic feasibility of the planned trajectory while being robust to parameter uncertainties and disturbances. The motion planner relies on a 10Hz kinematic bicycle Model Predictive Control approach, while the low-level closed-loop control signals are computed at a high-frequency (every 10ms) and are mainly based on Proportional-Integral-Derivative (PID) controllers. The motion planner is always planning a valid dynamic reference trajectory (green region of Figure 1) according to our previous work [START_REF] Polack | The Kinematic Bicycle Model: a Consistent Model for Planning Feasible Trajectories for Autonomous Vehicles?[END_REF], by constraining the lateral acceleration to be less than 0.5µg. Our architecture is also less conservative than other ones: a local velocity planner based on the lateral acceleration constraint was designed in order to compute a heuristic speed for the motion planner, that pushes the vehicle's architecture towards its operational limits, while remaining safe.

The paper is organized as following: first, the kinematic bicycle model with constrained lateral acceleration to ensure its validity is presented. Then, the consistent planning and control architecture chosen is explained in Section III. Section IV presents the 9 Degrees of Freedom (9 DoF) model used for our simulations. Section V shows the results obtained in simulation with the proposed consistent planning and control architecture on a track with and without obstacles. Finally, Section VI concludes this paper.

II. THE KINEMATIC BICYCLE MODEL

In the kinematic bicycle model, the two front wheels (resp. the two rear wheels) of the vehicle are lumped into a unique wheel located at the center of the front axle (resp. of the rear axle) such as illustrated on Figure 2. For smoothness reasons, the control inputs correspond to the acceleration u 1 and the steering rate of the front wheel steering angle u 2 of the vehicle, when assuming that only the front wheel can be steered. The kinematic bicycle model1 can then be written [START_REF] Rajamani | Vehicle Dynamics and Control[END_REF] as:

Ẋ = V cos (ψ + β (δ )) (1a) Ẏ = V sin (ψ + β (δ )) (1b) ψ = V l r sin (β (δ )) (1c) V = u 1 (1d) δ = u 2 (1e)
where X and Y are the global coordinates of the vehicle, ψ the yaw angle, V the speed and δ the front steering angle.

The slip angle β at the center of gravity depends on δ :

β (δ ) = atan tan(δ ) l r l f + l r (1f)
This model does not take into account the vehicle's dynamics such as slipping and skidding. Therefore, when used for planning problems, it might plan unfeasible trajectories (black region of Figure 1) which are not acceptable for the safety of the vehicle as explained in Section I. However, in previous work [START_REF] Polack | The Kinematic Bicycle Model: a Consistent Model for Planning Feasible Trajectories for Autonomous Vehicles?[END_REF], we have shown that constraining the lateral acceleration of the vehicle under 0.5µg ensures the validity of the model (green region of Figure 1). This constraint was obtained by comparing2 the steering angle δ a applied on a complex 9 DoF dynamic vehicle model (presented in Section IV) and the one applied on a kinematic bicycle model δ th , see Equation ( 2), to follow a curve of radius R at different speeds V . The results are displayed on Figure 3. We chose curves as scenarii as they correspond to the situations where the vehicle is most likely to slip.

δ th = atan l f l r + 1 tan asin l r R (2)

III. THE PLANNING AND CONTROL ARCHITECTURE

The goal of the planning and control architecture is to compute the actuator inputs (steering angle and wheel

a y max = 0.5g V (m/s) δ th ( • ) 0.5 • 1 • 1.5 • 2 • 2.5 • 3 • 3.5 • 4 • 5 • 6 •
Fig. 3: Comparison between the steering δ a applied on a 9 DoF dynamic model of a vehicle (in color) and the one applied on a kinematic bicycle model δ th in order to achieve a given turn radius R, at different speeds V (cf. [START_REF] Polack | The Kinematic Bicycle Model: a Consistent Model for Planning Feasible Trajectories for Autonomous Vehicles?[END_REF]).

torques) to send to the vehicle, while setting a target speed and avoiding obstacles located on the path. Our system is composed of four main components to deal with most driving situations, namely a local velocity planner, an obstacle manager, a local MPC planner and low-level controllers (see Figure 4). These parts are detailed respectively in Subsections A-D. The main characteristics of our design are:

• A smart computation by the local velocity planner of a heuristic speed V heur based on the lateral constraint for the kinematic bicycle model, to guide the MPC towards high-speed maneuvers.

• A combination of a 10Hz MPC planner for anticipating new situations, with a high-frequency (100Hz) closedloop low-level controller for a better tracking of the reference trajectory.

• A validity guarantee of the kinematic bicycle model used in the MPC through the constraint on lateral acceleration, generating thus only feasible trajectories to follow for the low-level controllers.

A. The Local Velocity Planner

The reference input of our architecture only contains the future coordinates (x i , y i ) i∈N of the center line of the road. However, we want to drive our vehicle at the highest possible speed that keeps our architecture safe. Therefore, a velocity term should be added to our cost function that fosters higher but safe speeds. Choosing a cost such as ||V -V max || 2 with V max constant would cause the velocity cost to change too much according to the situation, thus making it difficult to tune its weight. A small weight would cause the MPC to be very conservative when it could drive at higher speeds, while a bigger weight could cause safety problems when the velocity should be low, as the optimal solution might be to drive fast rather than to respect some safety constraints.

Therefore, we introduced a local velocity planner that gives a heuristic velocity V heur defined by Equation (3). It takes into account the current speed of the vehicle V , a predefined maximum speed allowed V max and the future coordinates (x i , y i ) i∈N of the reference path. The strategy consists in increasing the actual speed of the vehicle V by ∆V while V max is not reached, except if the curvature γ max of the path in the next T prev seconds leads to a lateral acceleration a y = γ max V 2 over the authorized limit for a kinematic bicycle model of 0.5µg. V max is defined specifically for straight lines, to ensure that the vehicle is able to come to a full stop if necessary at the end of the prediction horizon of the MPC.

V heur = min( 0.5µgR min ,V max ,V + ∆V ) (3)
Remark: the local velocity planner only gives a heuristic velocity for the motion planner, in order to obtain a velocity cost that does not depend to much on the velocity. The goal of the MPC is not to track it perfectly. Therefore, it does not need to be dynamically feasible.

B. The Obstacle Manager

This module defines a parabola around each surrounding obstacle, such as explained in [START_REF] Ziegler | Trajectory planning for Bertha -A local, continuous method[END_REF]. The interior points of the parabola defines then a region that is forbidden for the vehicle, and will be used as a constraint in the local MPC planner. Please note that only static obstacles are taken into account in this paper.

C. The Local MPC Planner

The local MPC planner updates the reference trajectory every 100ms (10Hz) to take into account changes in the environment, such as obstacles. The concept of Model Predictive Control is to have a model of the plant to predict the future outputs of the system [START_REF] Camacho | Model Predictive Control[END_REF]. The MPC minimizes at each refresh time, over all possible control sequences, a cost function J while respecting some operational constraints, such as the vehicle dynamics or obstacle avoidance. Normally, only the first optimal control value computed by the MPC is sent to the plant at time t. However, the MPC control outputs here are the acceleration of the vehicle and the steering rate which do not correspond to the actual control inputs which will be used in our simulations, namely the wheel torques and the steering angle (see Section IV). Therefore, we used the predicted trajectory computed by the MPC as the reference for the low-level controllers.

At each refresh time, the optimization process is run again with a shifted prediction horizon. The main settings of a MPC is the design of the cost function J, the control horizon N u , the prediction horizon N y , and the discretization time step ∆t u . In our case, the prediction horizon is set to T H = 3s, the control discretization time step ∆t u = 0.2s, and the state prediction horizon N y is set equal to the control prediction horizon N u = T H /∆t u + 1.

Let u 1 and u 2 denote respectively the acceleration of the vehicle and the steering rate. Let U t+k,t = [u 1 , u 2 , obs tol , x tol , y tol , δ tol ] and ξ t+k,t = [s, X,Y,V, ψ, δ ] denote respectively the control input and the state of our MPC at time t + k, predicted at time t, with k = 0..N u for U t+k,t and k = 0..N y for ξ t+k,t .

1) Cost function:

We set e v = (V i -V heur i ) i∈[1:N y ] to be the speed cost; e δ = (δ i ) i∈[1:N y ] and ėδ = ( δ j ) j∈[1:N u ] respectively the cost on the steering angle and the steering angle variation; e X tol = (x tol j ) j∈[1:N u ] and e Y tol = (y tol j ) j∈[1:N u ] the cost of the slack variables associated respectively with the longitudinal and lateral offset to the reference trajectory; e Obs tol = (obs tol j ) j∈[1:N u ] the cost of the slack variables associated with the obstacle constraints; e δ tol = (δ tol j ) j∈[1:N u ] the cost of the slack variable associated with the validity of the kinematic bicycle model constraint.

We define the following cost function for a control sequence U = [U t,t ,U t+1,t , ...,U t+N u -1,t ] and a state sequence ξ = [ξ t+1,t , ξ t+1,t , ..., ξ t+N y ,t ]:

J(U, ξ ) = ||e v || 2 Q v + ||e δ || 2 Q ∆ + || ėδ || 2 Q δ + ||e X tol || 2 Q x (4) +||e Y tol || 2 Q y + ||e Obs tol || 2 Q obs + ||e δ tol || 2 Q δ tol where ||x|| 2 Q = x T Qx and Q v = q v I N y , Q ∆ = q ∆ I N y , Q δ = q δ I N u , Q x = q x I N u , Q y = q y I N u , Q obs = q obs I N u , Q δ tol = q δ tol I N u .
2) Constraints: First, we constraint the state ξ t,t to be equal to the initial state of the vehicle at time t. Then, the admissible trajectories of the vehicle are the ones that respect the equations of the kinematic bicycle model described in Equation ( 1), where we add a constraint on the curvilinear abscissa s: ṡ = V . The actuator are limited to u 1 ∈ [-8; +6]m/s 2 and u 2 ∈ [-0.5; +0.5] rad/s 2 . The road limits are encoded through a soft constraint: the lateral and longitudinal deviations from the reference path have to remain below a certain tolerance margin x tol and y tol . For each obstacle, we have an inequality constraint which is similar to the one described in [START_REF] Ziegler | Trajectory planning for Bertha -A local, continuous method[END_REF].

At last, to ensure the validity of the kinematic bicycle model, we used the criterion derived in [START_REF] Polack | The Kinematic Bicycle Model: a Consistent Model for Planning Feasible Trajectories for Autonomous Vehicles?[END_REF], i.e that the lateral acceleration a y (= V 2 /R where R is the curvature radius) should be lower than 0.5µg. Using Equation (2), we obtain the maximum authorized steering angle for a given speed V :

δ max (V ) = atan l f l r + 1 tan asin 0.5µgl r V 2 (5)
This is then expressed as a soft constraint by introducing the slack variable δ tol :

-δ max + |δ | ≤ δ tol (6)
3) Numerical resolution: The numerical optimization was done using ACADO Toolkit. The working principle of the resolution algorithm can be found in [START_REF] Houska | ACADO Toolkit -An Open Source Framework for Automatic Control and Dynamic Optimization[END_REF].

D. The Low-level Controllers

The low-level controllers are responsible for translating the trajectory computed by the local planner into actuator inputs. Many recent research on low-level control have been using MPC with complex vehicle and tire dynamics models for computing the actuators inputs (see for example [START_REF] Falcone | Predictive active steering control for autonomous vehicle systems[END_REF]). Although this method enables to anticipate the slip and skid of the vehicle, it is computationally too expensive to be computed at a high frequency. The control is thus in openloop on a long duration (around 50 to 100ms), which is less robust to modeling errors and might also be less stable.

Therefore, in our control architecture, the low-level control is done in closed-loop at a high frequency (100Hz). Longitudinal and lateral control are treated separately.

For the longitudinal controller, we implemented a simple PID controller [START_REF] Astrom | PID Controllers: Theory, Design, and Tuning[END_REF] which takes into account the difference between the reference speed V r given by the local MPC planner and the actual speed of the vehicle V , and its derivatives. V r is more precisely a linear interpolation of the reference speed output given by the MPC planner. If e = V -V r , then we have the following control law:

u(t) = -K P e(t) -K D ė(t) -K I t 0 e(τ)dτ (7)
For the lateral controller, the steering angle applied δ is composed of an open-loop part δ ol and a closed-loop part δ cl . The open-loop steering angle δ ol is computed by integrating the first value of the control obtained by the MPC planner, which corresponds to the steering rate. This value is only refreshed once every 100ms. The closed-loop steering angle δ cl is a simple PID control applied on the yaw angle error, projected one MPC time step ahead. It is computed every 10ms and uses as reference value for the yaw angle a linear interpolation of the one computed by the MPC planner.

IV. THE 9 DOF SIMULATION MODEL

In this section, we describe the 9 Degrees of Freedom (9 DoF) vehicle model used in order to simulate the dynamic of the vehicle. They correspond to 3 DoF for the whole vehicle (V x ,V y , ψ), 2 DoF for the carbody ( θ , φ ) and 4 DoF for the wheels (ω f l , ω f r , ω rl , ω rr ). The model takes into account both the coupling of longitudinal and lateral slips and the load transfer between tires. The control inputs of the simulator are the torques T ω i applied at each wheel i and the steering angle of the front wheel δ . The low-level dynamics of the engine and brakes are not considered here. The notations are given in Table I and illustrated in Figure 5.

Remark: the subscript i = 1..4 refers repectively to the front left ( f l), front right ( f r), rear left (rl) and rear right (rr) wheels.

Several assumptions were made during the modeling:

• Only the front wheels are steerable.

• The roll and pitch rotations happen around the center of gravity.

• The aerodynamic force is applied at the height of the center of gravity. Therefore, it does not involve any moment on the vehicle.

CONFIDENTIAL. Limited circulation. For review only.

Preprint submitted to 2018 American Control Conference. Received September 22, 2017. Longitudinal and lateral speed of the vehicle in its inertial frame M T Total mass of the vehicle I x , I y , I z Inertia of the vehicle around its roll, pitch and yaw axis I r i Inertia of the wheel i T ω i Total torque applied to the wheel i F xp i , F yp i Longitudinal and lateral tire forces generated by the road on the wheel i expressed in the tire frame

F x i , F y i
Longitudinal and lateral tire forces generated by the road on the wheel i expressed in the vehicle frame (x, y) F z i Normal reaction forces on wheel i l f , l r Distance between the front (resp. rear) axle and the center of gravity l w

Half-track of the vehicle h Height of the center of gravity r e f f

Effective radius of the wheel ω i Angular velocity of the wheel i V xp i

Longitudinal speed of the center of rotation of wheel i expressed in the tire frame Fig. 5: Simulation model of the vehicle.

• The slope and road-bank angle of the road are not taken into account.

A. Vehicle model

The dynamics of the longitudinal, lateral and yaw motions of the whole vehicle are given by the following Equations:

M T Vx = M T ψV y + 4 ∑ i=1 F x i -F aero (8a)
M T Vy = -M T ψV x + 4 ∑ i=1 F y i (8b)
I z ψ = l f (F y 1 + F y 2 ) -l r (F y 3 + F y 4 ) (8c)
+ l w (F x 2 + F x 4 -F x 1 -F x 3 )
where the aerodynamic drag forces are F aero = 1 2 ρ air C x SV 2

x with ρ air the mass density of air, C x the aerodynamic drag coefficient and S the frontal area of the vehicle. The position (X,Y ) of the vehicle in the ground frame is given by the following Equations:

Ẋ = V x cos ψ -V y sin ψ (8d) Ẏ = V x sin ψ +V y cos ψ (8e)

B. Carbody model

The dynamics of the roll and pitch motion of the carbody are given by the following Equations:

I x θ = l w (F z 1 + F z 3 -F z 2 -F z 4 ) + h 4 ∑ i=1 F y i (8f)
I y φ = l r (F z 3 + F z 4 ) -l f (F z 1 + F z 2 ) -h 4 ∑ i=1 F x i (8g)
where F z i are damped mass/spring forces depending on the suspension travel ζ i due to the roll θ and pitch φ angles:

F z i = -k s ζ i (θ , φ ) -d s ζi (θ , φ ) (8h)
The parameters k s and d s are respectively the stiffness and the damping coefficients of the suspensions.

C. Wheel dynamics

The dynamics of each wheel i = 1..4 expressed in the tire frame is given by the following Equation:

I r ωi = T ω i -r e f f F xp i (8i)

D. Tire Dynamics

The longitudinal force F xp i and the lateral force F yp i applied by the road on each tire i are functions of the longitudinal slip ratio τ x i , the side-slip angle α i , the normal reaction force F z i and the road friction coefficient µ:

F xp i = f x (τ x i , α i , F z i , µ) (9a)
F yp i = f y (α i , τ x i , F z i , µ) (9b)
The longitudinal slip ratio of the wheel i is defined as following:

• Traction phases (r e f f ω i ≥ V xp i ):

τ x i = r e f f ω i -V xp i r e f f |ω i | • Braking phases (r e f f ω i < V xp i ): τ x i = r e f f ω i -V xp i |V xp i |
The lateral slip-angle α i of tire i is the angle between the direction given by the orientation of the wheel and the direction of the velocity of the wheel (see Figure 5):

α f = δ -atan V y + l f ψ V x ± l w ψ ; α r = -atan V y -l r ψ V x ± l w ψ (10)
In order to model the functions f x and f y , we used the combined slip tire model presented by Pacejka in [START_REF] Pacejka | Tyre and Vehicle Dynamics[END_REF] which takes into account the interaction between longitudinal and lateral slips on the force generation. Therefore, the friction circle due to the laws of friction (see Equation ( 11)) is respected. Finally, the impact of load transfer between tires is also taken into account through F z .

|| F xp + F yp || ≤ µ|| F z || (11)
Lastly, the relationships between the tire forces expressed in the vehicle frame F x and F y and the ones expressed in the tire frame F xp and F yp are given in Equation ( 12):

F x i = (F xp i cos δ i -F yp i sin δ i ) cos φ -F z i sin φ (12a)
F y i = (F xp i cos δ i -F yp i sin δ i ) sin θ sin φ (12b) + (F yp i cos δ i + F xp i sin δ i ) cos θ + F z i sin θ cos φ
CONFIDENTIAL. Limited circulation. For review only.

Preprint submitted to 2018 American Control Conference. Received September 22, 2017.

V. RESULTS

In order to test the proposed planning and control architecture, simulations were performed in PreScan [START_REF]Tass international[END_REF] using the 9 DoF vehicle model presented in Section IV. PreScan enables to easily generate a traffic scenario in order to test algorithms for autonomous vehicles. The test track was designed to alternate between long straight lines and small radius curvatures (up to 10m only) as shown in Figure 6. This is particularly challenging as the constraint on lateral acceleration is rapidly reached in the curves if the vehicle arrives too fast. Therefore, some precautions must be taken in the choice of the parameters of the planning and control architecture. For example, the prediction horizon of the MPC depends on the maximum speed V max as the vehicle needs to be able to decelerate sufficiently in advance to reach the target speed in the curves. The road friction coefficient µ was chosen equal to 1.

A. Without obstacles

In a first step, we tested the planning and control architecture on the track without obstacles in order to show its capacity to adapt the speed accordingly to the situation. Figure 7 compares the heuristic speed V heur computed by the local velocity planner, the target speed V r computed by the MPC planner and the real speed V of the vehicle. We observe that before entering a curve, the heuristic speed computed by the local velocity planner reduces drastically in order not to exceed 0.5g for the lateral acceleration. As there are no obstacles, the speed V r of the MPC planner is guided by this heuristic, while the longitudinal low-level controller tracks sufficiently well this reference speed in curves to avoid skipping and skidding. The low-level control inputs are shown in Figure 8a and9 (in blue). The absolute value of the lateral error to the reference path given at entry of our system does not exceed 0.4m and the computational time of the MPC remains always below 100ms. Moreover, we observe in Figure 8a that δ cl is important due to the slow refresh time of the MPC.

B. With obstacles

In a second step, we tested the planning and control architecture on the track with static obstacles in order to show the capacity of the motion planner to cope with planning safe and feasible trajectories. The resulting trajectory is shown in Figure 6. The low-level control inputs are shown in Figure 8b and 9 (in red). As expected, the vehicle does almost not slip or skid thanks to the constraint on lateral acceleration. The computational time of the MPC remains below 100ms.

VI. CONCLUSION

We have presented a simple planning and control architecture for an autonomous vehicle under normal driving conditions based on a 10Hz kinematic bicycle MPC and a 100Hz closed-loop PID controller. The main characteristic of our approach is to guarantee the consistency between the two layers by ensuring the validity of the kinematic bicycle model used for planning at any time through a dynamic constraint on the maximal authorized steering angle. Therefore, only feasible trajectories will be generated. The low-level controller enables the architecture to be robust to disturbances and modeling errors. Our architecture is also less conservative as it is guided toward the highest speed that keeps the architecture safe thanks to the computation of a heuristic speed based only on the road geometry.

Our planning and control architecture was tested in simulation on a challenging track, with and without obstacles, using a high-fidelity 9 DoF dynamic vehicle model, taking into account both tire friction circles and load transfer.

Even, if the proposed architecture is able to cope with most normal driving situations, the constraint on lateral acceleration might sometimes be too harsh, especially in emergency situations. Switching to a more complex model at the planning and control level might then be necessary, for example using [START_REF] Altché | High-speed trajectory planning for autonomous vehicles using a simple dynamic model[END_REF], [START_REF] Goh | Simultaneous stabilization and tracking of basic automobile drifting trajectories[END_REF]. This will be in the scope of future work. At last, the impact of low-friction road conditions on the vehicle architecture should also be investigated for further validations.

Fig. 2 :

 2 Fig. 2: Kinematic bicycle model of the vehicle.

Fig. 4 :

 4 Fig. 4: Control architecture of the ego-vehicle.

Fig. 6 :

 6 Fig.6: Trajectories planned by MPC (green lines) and actual trajectory followed by the controller (blue dots), with obstacles (red dots).

Fig. 7 :Fig. 8 :Fig. 9 :

 789 Fig.7: Comparison between V heur (red), V r (green) and V (blue), in the case of no obstacles.

TABLE I :

 I Notations Roll, pitch and yaw angles of the carbody V x , V y

θ , φ , ψ

This work was supported by the international Chair MINES ParisTech -Peugeot-Citroën -Safran -Valeo on ground vehicle automation[START_REF] Nilsson | Longitudinal and lateral control for automated lane change maneuvers[END_REF] Philip Polack, Florent Altché, Brigitte d'Andréa-Novel and Arnaud de La Fortelle are with Center of Robotics, Mines ParisTech, PSL Reasearch University, 60 boulevard Saint-Michel, 75006 Paris, France {philip.polack, florent.altche, brigitte.dandrea-novel, arnaud.de la fortelle} @mines-paristech.fr 2 Florent Altché is also with École des Ponts ParisTech, Cité Descartes, 6-8 Av Blaise Pascal, 77455 Champs-sur-Marne, France.

This is a misuse of language, as we are actually using the acceleration instead of the speed as input for better smoothness.

The kinematic bicycle model remains valid as long as δ a = δ th to achieve the same turn radius R, where δ th is constant (does not depend on speed).

Preprint submitted to 2018 American Control Conference.Received September 22, 2017.