Tingting Yuan
email: tingt.yuan@hotmail.com.

Wilson Borba

Rocha Neto

Christian Esteve Rothenberg

Katia Obraczka

Chadi Barakat
email: chadi.barakat@inria.fr.

Thierry Turletti
email: thierry.turletti@inria.fr.

Wilson Da

Machine Learning for Next-Generation Intelligent Transportation Systems: A Survey

Keywords: Intelligent Transportation System (ITS), Machine Learning (ML), Cooperative Driving, Autonomous Vehicles

établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

I. INTRODUCTION

Intelligent Transportation Systems, or ITS for short, typically refers to the application of information, communication, and sensing technology to transportation and transit systems [1]. ITS is likely to be an integral component of tomorrow's smart cities [START_REF] Campolo | From today's VANETs to tomorrow's planning and the bets for the day after[END_REF] and will include a variety of services and applications such as road traffic management, traveler information systems, public transit system management, and autonomous vehicles, to name a few. It is expected that ITS services will contribute significantly to improved road and traffic safety, transportation and transit efficiency, as well as to increased energy efficiency and reduced environmental pollution. While ITS applications have been enabled by unprecedented advances in sensing, computing, and wireless communication technology, they will pose a variety of challenges due to their scalability and diverse quality-of-service needs, as well as the massive amounts of data they will generate. In parallel, Machine Learning (ML) techniques have recently gained significant traction enabled by a variety of technologies, notably cloud and edge computing. ML has been used by a diverse set of applications, that, similarly to ITS services, impose a wide range of requirements. In particular, ML approaches such as deep learning and reinforcement learning have been useful tools to explore patterns and underlying structures in big data sets for prediction and accurate decision making [START_REF] Mao | Deep learning for intelligent wireless networks: A comprehensive survey[END_REF][START_REF] Zhang | Deep learning in mobile and wireless networking: A survey[END_REF][START_REF] Luong | Applications of deep reinforcement learning in communications and networking: A survey[END_REF] in addition to vehicular cybersecurity [START_REF] Pethő | A survey of new orientations in the field of vehicular cybersecurity, applying artificial intelligence based methods[END_REF]. Statistics on scientific publications in the last ten years (see Fig. 1) show a clear increasing trend in the amount of research efforts leveraging ML to enable and optimize ITS tasks.

The question of how to explore and adapt ML to address ITS applications' distinctive characteristics and requirements remains challenging and offers promising research directions. Existing surveys have explored some of these challenges and solutions to address them. Table I lists the most recent ones and summarizes their scope. Notable works include [START_REF] Pethő | A survey of new orientations in the field of vehicular cybersecurity, applying artificial intelligence based methods[END_REF][START_REF] Ye | Machine learning for vehicular networks: Recent advances and application examples[END_REF][START_REF] Khan | A survey of advances in visionbased vehicle re-identification[END_REF][START_REF] Arnold | A survey on 3D object detection methods for autonomous driving applications[END_REF][START_REF] Zou | Object detection in 20 years: A survey[END_REF][START_REF] Tong | Artificial intelligence for vehicle-to-everything: A survey[END_REF][START_REF] Yang | Visual perception enabled industry intelligence: state of the art, challenges and prospects[END_REF][START_REF] Kuutti | A survey of deep learning applications to autonomous vehicle control[END_REF][START_REF] Mekrache | Deep reinforcement learning techniques for vehicular networks: recent advances and future trends towards 6G[END_REF][START_REF] Pasin | Computational intelligence and adaptation in VANETs: Current research and new perspectives[END_REF][START_REF] Liang | Toward intelligent vehicular networks: A machine learning framework[END_REF], which focus on specific aspects of ITS, such as vehicular networks [START_REF] Ye | Machine learning for vehicular networks: Recent advances and application examples[END_REF], vehicle detection [START_REF] Khan | A survey of advances in visionbased vehicle re-identification[END_REF][START_REF] Arnold | A survey on 3D object detection methods for autonomous driving applications[END_REF][START_REF] Zou | Object detection in 20 years: A survey[END_REF], safety of vehicular ad-hoc network (VANET) [START_REF] Pethő | A survey of new orientations in the field of vehicular cybersecurity, applying artificial intelligence based methods[END_REF][START_REF] Pasin | Computational intelligence and adaptation in VANETs: Current research and new perspectives[END_REF], VANET performance optimization [START_REF] Liang | Toward intelligent vehicular networks: A machine learning framework[END_REF], autonomous vehicle control [START_REF] Kuutti | A survey of deep learning applications to autonomous vehicle control[END_REF], and Deep Reinforcement Learning (DRL) for 6G VANETs [START_REF] Mekrache | Deep reinforcement learning techniques for vehicular networks: recent advances and future trends towards 6G[END_REF]. Compared to existing surveys, ours provides a broad perspective on the usage of ML in ITS and ML's role in enabling ITS services and applications.

The main goals of this paper are: (1) to provide a thorough survey of the current state-of-the-art of how ML technology has been applied to a broad range of ITS applications and services, such as co-operative driving Short survey on some ML tasks in vehicular networks. [START_REF] Khan | A survey of advances in visionbased vehicle re-identification[END_REF] 2019 Survey on vision-based vehicle re-identification.

Only for vehicle re-identification, one of perception tasks. [START_REF] Arnold | A survey on 3D object detection methods for autonomous driving applications[END_REF] 2019 Survey on 3D object detection for autonomous driving applications.

For 2D and 3D object detection, one of perception tasks.

[10] 2019 Survey on object detection. More general survey of perception tasks, not focused on ITS. [START_REF] Tong | Artificial intelligence for vehicle-to-everything: A survey[END_REF] 2019 Survey on AI-driven vehicular systems.

Focused on vehicular applications, but lacks coverage of some recent research on the topic such as 4D and 5D detection. [START_REF] Yang | Visual perception enabled industry intelligence: state of the art, challenges and prospects[END_REF] 2020 Survey on visual perception in industry intelligence.

Only for visual perception in industrial scenarios. [START_REF] Kuutti | A survey of deep learning applications to autonomous vehicle control[END_REF] 2020 Survey on deep learning for autonomous vehicle control.

Focused on autonomous vehicle control, one of the management tasks. [START_REF] Mekrache | Deep reinforcement learning techniques for vehicular networks: recent advances and future trends towards 6G[END_REF] 2021 DRL for 6G vehicular networks.

Only covered DRL techniques and 6G. [START_REF] Pethő | A survey of new orientations in the field of vehicular cybersecurity, applying artificial intelligence based methods[END_REF] 2021 Survey on ML for vehicular cybersecurity. Focus on cybersecurity. Ours 2021 Survey on ML in ITS including perception, prediction, management.

Overview of ML for main tasks in ITS.

and road hazard warning, and (2) to identify future directions for how ITS can use and benefit from ML technology. To this end, the contributions of our survey can be summarized as follows:

• First, we present an overview of ITS based on an ITS application-centric framework we propose. • Second, we explain how ML can be used by ITS applications.

• Third, based on the application-centric ITS framework, we provide a detailed review of the state-ofthe-art on the application of ML to ITS. • Finally, we discuss future trends and research directions on how ML can be applied to benefit ITS applications. The structure of the paper, which is illustrated in Fig. 2, is as follows. In Section II, we present an overview of ITS and in Section III, review ML and discuss how ML techniques can be employed by ITS applications. Sections IV, V, and VI describe studies, most of which published in the last two decades, that apply machine learning to various ITS tasks. In Section VII, we highlight several open research issues and discuss some future trends. Section VIII concludes the survey. II. ITS OVERVIEW ITS is a relatively recent-but fast-evolving area and has been attracting considerable attention from the research and practitioner communities. This section provides an overview of ITS, including: (1) defining how ITS is currently understood, (2) listing some of its more prominent applications and services, [START_REF] Mao | Deep learning for intelligent wireless networks: A comprehensive survey[END_REF] proposing an application-centric framework for ITS that will serve as the basis for the survey, and (4) identifying basic tasks that are used as building blocks by ITS applications.

A. Working Definition of ITS

Even though ITS is a relatively recent term, its definition has been evolving ever since it was first proposed in the 90's [START_REF] Fink | Intelligent transportation systems[END_REF]. Stakeholders tend to have different but not disassociated views of what ITS means. The U.S. Department of Transportation (USDOT), for example, defines ITS as a mean to achieve safety and mobility in surface transportation through the application of Fig. 2: Survey structure and organization. information and communication technologies [START_REF] Barbaresso | USDOT's intelligent transportation systems (ITS) strategic plan, 2015-2019[END_REF]. In this case, surface transportation refers to transportation by roads, rail, or water and excludes air transportation. The European Union uses a similar definition but limits surface transportation to transportation by roads [START_REF] Mandžuka | Directives of the european union on intelligent transport systems and their impact on the republic of croatia[END_REF]. This focus on surface transportation can be explained by the distinct characteristics of aerial, marine, and terrestrial transportation in terms of several aspects, including usage and security [START_REF] Williams | Intelligent transport systems standards[END_REF].

Other definitions approach ITS from different points of view, e.g., focusing on the benefits that ITS can provide to ITS users (including drivers, passengers, and pedestrians) through the use of services aiming at traffic efficiency, security [START_REF] Williams | Intelligent transport systems standards[END_REF] (e.g., speed-and road condition monitoring, weather forecasting), etc. Such services usually rely on the interaction between vehicles and road infrastructure, which in turn motivates the idea of Cooperative ITS.

Cooperative ITS (C-ITS) leverages Vehicle-to-Everything (V2X) interaction [START_REF] Platform | Platform for the deployment of cooperative intelligent transport systems in the EU (E03188)(C-ITS platform) final report[END_REF] and is consistent with current efforts towards "smart and connected vehicles" as illustrated by standardization activities worldwide [START_REF] Campolo | From today's VANETs to tomorrow's planning and the bets for the day after[END_REF]. The vision of intelligent, interconnected transportation systems is aligned with the USDOT's Connected Vehicle Pilot Program [22] which views ITS as a "mean to deploy applications utilizing data captured from multiple sources across all the elements of surface transportation systems".

Based on these current ITS definitions, in this survey, we define ITS as the means to interact with road transportation systems and deliver improved security, efficiency, and comfort to users through the deployment of applications that employ information, communication, and sensing technology. To further elaborate on this definition, we discuss notable ITS applications next.

B. ITS Applications

Different ITS stakeholders propose different classifications for ITS applications. For example, the CAR-2-CAR communication consortium groups ITS services in (1) Awareness driving (e.g., speed and position); (2) Sensing driving (e.g., pedestrian detection); and (3) Cooperative driving (e.g., turning intention) and movement coordination between vehicles [START_REF]About connected intelligent transportation systems[END_REF]. The ISO 14813-1, in turn, groups ITS services in 11 domains, ranging from traffic management operation to weather and environmental conditions monitoring [START_REF] Williams | Intelligent transport systems standards[END_REF]. The USDOT connected vehicle pilot program (CV Program) lists different applications categories, some of which (e.g., V2V/V2I safety and V2I mobility applications), have started to be implemented in US cities. Examples of CV Program's applications include forward collision warning, intelligent traffic light and pedestrian crosswalk [START_REF]Connected vehicle applications and supporting documentation[END_REF]. The European Telecommunications Standards Institute (ETSI) proposed the basic set of applications, which is illustrated in Table II. A comprehensive description of ETSI applications is presented in [START_REF] Alam | Introduction to intelligent transportation systems[END_REF]. Because ETSI's applications are well known and widely adopted, we use them in this survey to guide our exploration of ITS applications including our application-driven ITS framework, which is described in detail below.

C. Application-driven ITS Framework

We propose an application-driven ITS framework which is inspired by ETSI's applications. As illustrated in Fig. 3, the proposed framework is structured in three layers, namely: infrastructure-, resource-and orchestration, along with an application realm.

1) ITS Environment: ITS applications and services interact with the ITS physical environment which comprises: transportation infrastructure, environmental conditions, and users. Transportation infrastructure includes vehicles, traffic lights, traffic signs, roads, toll booths, road elements (e.g., speed bumps), and other road infrastructure. Examples of environmental conditions are weather, lighting, geography, and road conditions. Finally, ITS application users include drivers, passengers, pedestrians, and operation and management personnel. The interaction between transportation infrastructure, environmental elements, and users contributes to the complexity, heterogeneity, and dynamic nature of the ITS environment.

2) Infrastructure Layer: The infrastructure layer is responsible for collecting data from the ITS environment and delivering it to the other layers. Therefore, the infrastructure layer comprises both a (i) sensing infrastructure which includes all data collection devices (e.g., sensors); and (ii) communication infrastructure consisting of networking equipment responsible for enabling data access and exchange.

As roads, vehicles, pedestrians, and passengers carry an increasing number and variety of sensors (e.g., Internet of Vehicles (IoV) [START_REF] Kaiwartya | Internet of vehicles: Motivation, layered architecture, network model, challenges, and future aspects[END_REF][START_REF] Guerrero-Ibáñez | Sensor technologies for intelligent transportation systems[END_REF], On-Board Units (OBUs) [START_REF] Hernandez | Vehicle on board platform: Communications test and prototyping[END_REF]), the sensing infrastructure must be able to acquire and communicate sensed information at unprecedented scale and heterogeneity. While sensing devices such as cameras, light radars (LiDARs) and ultrasonic sensors offer visual data to ITS applications, kinetic sensors (e.g., accelerometers), magnetic sensors (e.g., compasses), and position tracking systems (e.g., global positioning system) provide scalar information. Roadside units (RSUs), access points (APs), routers, switches, and transceivers enable communication amongst ITS users and components (e.g., Vehicle-to-Infrastructure (V2I), Vehicle-to-Vehicle (V2V)) using communication standards like fifth generation (5G) mobile networks and IEEE 802.11p [START_REF] Festag | Standards for vehicular communication-from IEEE 802.11 p to 5G[END_REF].

Additionally, Unmanned Aerial Vehicles (UAVs) [START_REF] Zeng | Accessing from the sky: A tutorial on UAV communications for 5G and beyond[END_REF] are a recent addition to the ITS environment augmenting it with high mobility and agility. UAVs can play various roles in ITS [START_REF] Menouar | UAV-enabled intelligent transportation systems for the smart city: Applications and challenges[END_REF][START_REF] Yuan | Harnessing UAVs for fair 5G bandwidth allocation in vehicular communication via deep reinforcement learning[END_REF], such as aerial deliveries, aerial traffic signals, movable base stations and aerial cameras.

3) Resource Layer: ITS applications and data collected by the infrastructure layer can use a plethora of services provided by the resource layer. Such services include computing, networking, storage, and energy. Storage resources are used to store historical data and computational results locally or in clouds. Computing resources provide data processing capabilities, including fog computing and cloud computing services deployed in dedicated hardware or virtual environments (e.g., Avoid collisions and mitigate their impacts.

Across traffic turn collision risk warning, pre-crash sensing

Cooperative Traffic Efficiency

Speed management

Warn vehicles about speed discipline. Regulatory/Contextual speed limit notification Co-operative navigation Information exchange for traffic navigation coordination (e.g., intersection management and adaptive cruise control).

Traffic information and recommended itinerary, enhanced route guidance and navigation

Cooperative Local Services Location based services

Provide information for local commercial or non-commercial services (e.g., food and parking) and multimedia access.

Media downloading, automatic access control and parking management

Global Internet Services

Communities services

Enable interaction, monitoring and management of financial and insurance services provided by the wider Internet.

Fleet management, insurance and financial services

ITS station life cycle management

Manage services and the functioning of the ITS infrastructure).

Vehicle software/data provisioning, and data calibration virtual machines or containers). Networking resources are used to deliver data and include physical-and virtual networking functions performed by communication infrastructure elements [START_REF] Mijumbi | Network function virtualization: State-ofthe-art and research challenges[END_REF]. Finally, energy resources provide power to the ITS infrastructure, ensuring its continuous availability. 4) Orchestration Layer: Since ITS applications have different resource requirements, services provided by the resource layer need to be delivered to applications according to their needs. As such, resource allocation is one of the main roles of the orchestration layer, which creates abstract representations, or models, for the resource-and infrastructure layers in order to schedule their resources and services and address the different needs of ITS applications. Orchestrators then provide an interface between ITS applications and the resource-and infrastructure layers handling requests from different applications, scheduling the appropriate resources, and/or obtaining requested information to ensure applications receive the quality-of-service they need. By providing this "bridge", the orchestration layer also facilitates application development and deployment. Embodiments of the orchestration layer include Software Defined Networking (SDN) [START_REF] Kreutz | Software-defined networking: A comprehensive survey[END_REF] controllers and applications as well as network service orchestrators [START_REF] De Sousa | Network service orchestration: A survey[END_REF].

5) ITS Application Realm: As previously discussed, there is a wide variety of ITS applications ranging from driver assistance to traffic efficiency and media downloading. ITS applications need access to distinct resources, infrastructure, and data. To capture the different needs of ITS applications, we classify them into three different groups, namely: local, global, and hybrid applications.

Local applications rely solely on data collected locally. Cooperative collision avoidance is a typical example of a local application as it tries to identify possible crashes collecting and exchanging information from the vehicle and its immediate surroundings. Global applications, on the other hand, require information that transcends a specific locality. Road hazard warning services, for example, collect different kinds of traffic events and use the information obtained from the orchestration layer to enforce desired policies. For instance, in the case of impaired vehicles stopped on the road, the road hazard warning application receives relevant information (e.g., time of incident and where the vehicle is located) and decides which geographical areas should receive information about the event. Finally, hybrid applications can use both local as well as global information. Cooperative navigation services, for example, can access optimized traffic information data provided by a server connected to the orchestration layer and use the local perception of traffic and hazards to define the better route for a vehicle. Note that, as illustrated in Fig. 3, ITS applications which are represented by the application realm in the proposed ITS framework can interact with all other layers of the framework.

D. ITS Application Tasks

The wide scope of use cases and constant interactions with the highly dynamic and heterogeneous ITS environment raise a number of challenges to ITS application deployment. For example, ITS applications need to acquire, process, and take timely actions based on massive amounts of data, while providing efficiency, security and convenience to its users.

To better understand and deploy ITS applications, some studies try to divide ITS applications according to the types of tasks they perform. For example, works reported in [START_REF] Pendleton | Perception, planning, control, and coordination for autonomous vehicles[END_REF][START_REF] Sallab | Deep reinforcement learning framework for autonomous driving[END_REF][START_REF] Gruyer | Perception, information processing and modeling: Critical stages for autonomous driving appli-cations[END_REF], which focus on driver assistance and co-operative driving applications for semi-and fully autonomous vehicles, give some examples of ITS tasks. In this paper, we expand the concept of ITS tasks in order to support the wide range of ITS applications. To this end, we categorize ITS tasks tasks into (i) perception tasks, (ii) prediction tasks, and (iii) management tasks.

In the following subsections, we define and discuss the challenges raised by each task. We also showcase how ITS applications can leverage such tasks using Fig. 4 as an example of the interaction between two applications, namely cooperative collision avoidance and road hazard warning based on the proposed ITS framework.

1) Perception Tasks: Perception tasks are those that try to detect, identify and recognize data patterns in order to extract, understand, and present relevant information. These tasks are widely used in today's transportation systems due to the widespread use of sensors, shifting the challenge from how to gather to how to interpret data. Using perception, ITS applications can receive information extracted from the environment. For example, as depicted in Fig. 4, cooperative collision avoidance interacts with the sensing infrastructure of a vehicle, collecting data and using perception to raise awareness of the surrounding environment. Additionally, perception of road signs identifies signage on the side of a road, providing a decision parameter to inform management applications.

The large number and types of sensors present in the sensing infrastructure as well as the enormous amounts of data they produce [START_REF] Guerrero-Ibáñez | Sensor technologies for intelligent transportation systems[END_REF] pose data fusion [START_REF] Faouzi | Data fusion in intelligent transportation systems: Progress and challenges-a survey[END_REF] and big data problems, which impact ITS perception tasks. For example, the variety of vehicles with different mobility patterns and physical features can impact how they are identified by a co-operative collision avoidance application. On the other hand, camera images for a road sign detection task deal with signs in different physical conditions, angles, and brightness, which can change the perception of the sign [START_REF] Eykholt | Robust physicalworld attacks on deep learning visual classification[END_REF].

Even with these concerns, perception tasks are expected to be robust and stable, since their outputs are used in the applications' decision-making process. As new sensor technologies [START_REF] Guerrero-Ibáñez | Sensor technologies for intelligent transportation systems[END_REF] arrive in transportation systems, perception tasks have to deal with new features. Standardization between manufacturers [START_REF] Campolo | From today's VANETs to tomorrow's planning and the bets for the day after[END_REF] and the definition of a standard protocol [START_REF] Kaiwartya | Internet of vehicles: Motivation, layered architecture, network model, challenges, and future aspects[END_REF] are possible solutions to overcome challenges leveraged by heterogeneity. However, solely adopting a global automated data collection scheme is not enough. What ITS perception really needs is real-time and situational assessment, which can be achieved by the improvement of machine cognition [START_REF] Bengler | Three decades of driver assistance systems: Review and future perspectives[END_REF].

2) Prediction Tasks: Prediction tasks, as their name suggests, try to predict future states given historical and real-time data. Due to dynamic ITS environment, these tasks are used by proactive applications, which attempt to prepare for future states by prediction. For example, the co-operative collision avoidance application illustrated in Fig. 4 needs to predict where a vehicle will be in a future point in time, prematurely identify an accident and apply actions to mitigate or lessen impacts (e.g., pre-crash warning use-case).

However, the heterogeneous and dynamic ITS environment hinders the accuracy of prediction tasks. Besides, uncertainty, ambiguity, and quality of the information are also crucial in state prediction. For instance, in a traffic flow prediction task, a plethora of factors are relevant, including time-related (e.g., day of the week, day's schedule and holiday impact), vehicle proportion (e.g., number of cars in relation to bicycles), accidents, weather and even sociocultural ones (e.g., the behavior of drivers in a specific country). Because of this, extracting the correct features to give a precise prediction is a challenge, which restricts the scope of prediction solutions.

3) Management Tasks: Management tasks are responsible for dictating the behavior in ITS. Management tasks are needed to provide a systematic and reliable solution for a given problem. For example, in Fig. 4, a co-operative collision avoidance application, after perceiving a vehicle ahead and predicting a crash, has to use management tasks to control the vehicle's trajectory and motion to avoid the accident. If a vehicle fails to avoid the collision, the orchestration layer can gather information about the crash and provide it to a global road hazard warning application. The latter application will elect geographical locations and manage message dissemination policies to warn nearby vehicles about the crash (e.g., stationary vehicle warning use-case), leveraging the use of management tasks.

As soon as the scope covered by an ITS application keeps growing, the increased number of data necessary to deploy a management solution can compromise the scalability of the solution. Therefore, optimization in management is essential. Even for local applications, stringent time requirements demand optimal use of computational resources. Moreover, the heterogeneity of nodes and applications leverage concerns to network management, since essential nodes and sensitive applications must be prioritized to lessen the data transfer latency. The availability of data is yet another concern since a management system has to decide how the needed data can be retrieved.

III. MACHINE LEARNING FOR ITS

In this section, we discuss the potential of using ML in ITS, focusing on how ML can integrate and enhance perception, prediction, and management tasks. We provide a background on the mainstream ML approaches, introducing nomenclature and concepts typically found in the surveyed literature. An ML-experienced reader may jump the ML background and go directly to Section III-B, in which how ML works in ITS is discussed. A. Machine Learning ML is an area of computer science, which emphasizes the intelligence of machines in performing human-like tasks. In this subsection, we focus on mainstream ML approaches, including Supervised Learning (SL) [START_REF] Garcia | A survey of discretization techniques: Taxonomy and empirical analysis in supervised learning[END_REF], Unsupervised Learning (UL) [START_REF] Hastie | Unsupervised learning[END_REF], Reinforcement Learning (RL) [START_REF] Kaelbling | Reinforcement learning: A survey[END_REF], and Deep Learning (DL) [START_REF] Lecun | Deep learning[END_REF]. In order to have a better understanding of state-of-the-art ML approaches, we provide a brief review following the taxonomy presented in Fig. 5. For example, Support Vector Machine (SVM) [START_REF] Suykens | Least squares support vector machine classifiers[END_REF], and Adaptive Boosting (AdaBoost) [START_REF] Freund | A short introduction to boosting[END_REF] are representative classification algorithms. Regression algorithms work for the regression problem whose output variable is a real or continuous value, such as "salary" or "weight". Many different approaches have been proposed for the regression problem. The simplest one is the Linear Regression (LR) [START_REF] Seber | Linear regression analysis[END_REF], which tries to fit data with the best hyper-plane going through the points of training data. Another famous example is Support Vector Regression (SVR) [START_REF] Smola | A tutorial on support vector regression[END_REF]. Note that some algorithms are applied on both classification problems and regression problems, such as k-Nearest Neighbors (k-NN) [START_REF] Keller | A fuzzy k-nearest neighbor algorithm[END_REF], Random Forest (RF) [START_REF] Liaw | Classification and regression by randomforest[END_REF] and Boosted Regression Trees (BRT) [START_REF] Elith | A working guide to boosted regression trees[END_REF].

2) Unsupervised Learning: UL is a data-driven knowledge discovery approach that can infer a function describing the structure from datasets consisting of input data without labeled responses. Unsupervised algorithms can be split into two different categories: Clustering algorithms, such as K-means clustering [START_REF] Steinbach | A comparison of document clustering techniques[END_REF], discover the inherent groupings in the data. Dimension reduction algorithms, such as Principal Component Analysis (PCA) [START_REF] Jolliffe | Principal component analysis[END_REF] and Independent Component Analysis (ICA) [START_REF] Hyvarinen | Survey on independent component analysis[END_REF], find the best representation of the data with fewer dimensions.

3) Reinforcement Learning: RL aims to learn how to take a sequence of actions in an environment in order to maximize cumulative rewards. RL can be a zero-shot learning, which means it can begin to learn with no data. Fig. 6 depicts the working mechanism of RL combined with the ITS environment. The ITS environment includes all the ITS layers and the surrounding environment (for instance, the road condition). The agent in RL is the component that makes decisions on which action ought to take. To achieve it, the agent needs the ability to interact with the environment to obtain data (state, action, and reward). Then, with the obtained data, the agent can train and update itself to provide better decisions.

RL algorithms can be split into three different kinds:

Value-based algorithms are based upon temporal difference learning to obtain value function, which estimates how good to take specific actions on given states. Q-Learning [START_REF] Watkins | Q-learning[END_REF], SARSA [START_REF] Singh | Reinforcement learning with replacing eligibility traces[END_REF], and Deep Q-Network (DQN) [START_REF] Mnih | Human-level control through deep reinforcement learning[END_REF] are three typical value-based RL. Policybased algorithms directly learn optimal policy or try to obtain an approximate optimal policy based on the observation, such as Policy Gradients (PG) [START_REF] Sutton | Policy gradient methods for reinforcement learning with function approximation[END_REF] and Deterministic Policy Gradient (DPG) [START_REF] Silver | Deterministic policy gradient algorithms[END_REF]. Imitation algorithms [START_REF] Attia | Global overview of imitation learning[END_REF] (also called Apprenticeship Learning -AL) try to make decisions using demonstrations, which usually obtain a good performance when the reward function is difficult to specify or sparse and when it is challenging to optimize actions directly. These algorithms can deal with unexplored states (i.e. new states not in the training data) so that they offer more reliable frameworks for many tasks such as self-driving cars, e.g. Generative Adversarial Imitation Learning (GAIL) and Reward Augmented Imitation Learning (RAIL). Hybrid algorithms combine value-based algorithms with policybased algorithms. Their goal is to represent the policy function by policy-based algorithms, where updates of policy functions depend on value-based algorithms, e.g. Actor Critic (AC) [START_REF] Grondman | A survey of actor-critic reinforcement learning: Standard and natural policy gradients[END_REF], Asynchronous Actor-Critic Agents (A3C) [START_REF] Mnih | Asynchronous methods for deep reinforcement learning[END_REF], Deep Deterministic Policy Gradients (DDPG) [START_REF] Lillicrap | Continuous control with deep reinforcement learning[END_REF], and Soft Actor Critic (SAC) [START_REF] Haarnoja | Soft actorcritic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor[END_REF]. 4) Deep Learning and Neural Networks: DL is famous in various fields, its success mostly relies on Artificial Neural Networks (ANNs). ANNs have become a trendy method for data representation. An ANN consists of a set of interconnected nodes designed to imitate the functioning of the human brain. Each node has a weighted connection to several other nodes in neighboring layers. Individual nodes take the input received from connected nodes and use the weights together with a simple function to compute output values. ANNs, especially Deep Neural Networks (DNNs), became attractive inductive approaches owing to their high flexibility, nonlinearity, and data-driven model building.

The main kinds of neural networks are Fullyconnected Neural Networks (FNNs), Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), as shown in Table IV. CNNs achieve dominant performance on visual tasks, such as exploiting fundamental spatial properties of images and videos. RNNs can successfully characterize the temporal correlations of data, thus exhibit superior capability for time series tasks. The Long Short-Term Memory (LSTM) methods, whose units are RNNs, are capable of learning order dependence in sequence prediction problems. Graph Neural Networks (GNNs) [START_REF] Battaglia | Relational inductive biases, deep learning, and graph networks[END_REF][START_REF] Zhou | Graph neural networks: A review of methods and applications[END_REF] are a kind of graph structure, which models a set of nodes (entity) and edges (relationship). FNNs, CNNs, and RNNs are based on Euclidean data. However, GNNs use non-Euclidean data structures for deep learning.

The neural networks have a lot of extensions, such as Deep Belief Networks (DBN) [START_REF] Hinton | A fast learning algorithm for deep belief nets[END_REF], Error-feedback Recurrent Convolutional Neural Networks (eRCNNs), Fully Convolutional Neural Networks (FCNs), and Spatio-Temporal Graph Convolutional Neural Networks (STGCNs). For example, DBNs can be described as a stack of Restricted Boltzmann Machines (RBMs) [START_REF] Hinton | A practical guide to training restricted boltzmann machines[END_REF], which has a two-layer network model, consisting of visible units and hidden units.

As shown in Fig. 5, there is a new item, namely DRL, to describe the algorithms that combine RL with DL. For example, DQN, DDPG, and multi-agent RL (MARL) are DRL algorithms.

B. ML Meets ITS

Data is one of the main commodities extracted from today's ITS. Given the different scopes of ITS applications (global, local, and hybrid applications), data can be obtained from all the ITS layers. This data-heavy characteristic of ITS paves the way for the inherent ability of ML to discover knowledge from data. Regression, classification, prediction, clustering, and even decision-making, are features provisioned by ML capable of enhancing ITS and being foundations for the ITS application's building blocks, i.e., tasks. In this section, we discuss 1) how ML is integrated inside ITS, backed by an ML pipeline; and 2) how ML is harnessed by ITS tasks.

1) ML Pipeline:

In this part, we discuss the ML pipeline depicted in Fig. 7. The main objective of such a pipeline is to model desired ITS elements or behavior, which can be harnessed by ITS tasks. For example, modeling vehicle's mobility is useful for prediction tasks, whereas models to classify transportation infrastructure from images can be applied in perception. The ML pipeline consists of several steps, namely data preprocessing, feature extraction, and modeling. Data preprocessing: The raw data usually needs preprocessing; for example, data cleaning and data normalization.

Feature extraction: Feature extraction from data is a critical step. There are two ways for feature extraction, namely, hand-crafted features and deep learning features. Hand-crafted features are selected with the knowledge of human experts, which are relevant for a given task. However, even the most experienced human cannot identify all the underlying features not explicitly related to the captured data [START_REF] Bejani | Convolutional neural network with adaptive regularization to classify driving styles on smartphones[END_REF]. Therefore, the extracted features can only reflect limited aspects of the problem, which yield lower accuracy. Examples of hand-craft extractors are Gabor filter, local binary pattern (LBP), local ternary pattern (LTP), and histogram of oriented gradients (HOG) for image feature extraction. Thanks to deep learning, which has superiority in learning of deep features, the feature extraction can be automatic without any manual intervention.

Modeling: Regarding model training, ML has reached celebrity status. In particular, the advent of ML enables great strides towards better visual understanding [START_REF] Lecun | Deep learning[END_REF]. The trained ML models can be used for regression, classification, clustering, and making decisions, which can be applied to ITS tasks.

2) ML for ITS Tasks:

In this part, we introduce how ML works for each ITS task. Firstly, traditional approaches for perception were usually based on traditional sensors, such as magnetic sensors, inductive loop detectors, GPS, REID, and so on. With the widespread deployment of vision-based devices in ITS infrastructures, an unprecedented quantity of images and video data is generated, which leverages vision understanding as the crux of the perception task. Traditional techniques cannot offer the needed speed and accuracy in vision-based perception, whereas ML approaches can be used to improve these metrics. Such improvement was primarily done with hand-crafted features, which are derived from the information in the image. However, considering the growing diversity of objects and little difference between similar objects in some perceptive problems, the process of deriving hand-crafted features may not be discriminative enough. Thanks to DL, perception accuracy has been greatly improved with the extraction of deep features.

Secondly, researchers have investigated a number of parametric and non-parametric methods for the prediction problem. When the model structure is fixed and parameters are learned from data, this way of modeling is called a parametric method; examples include Grey system models, time series, and Kalman Filters. However, this method needs a good model structure in advance, based on the qualitative judgment of experts. It is highly subjective, and limited in the sense that results come under a high cost in terms of time and money. Likewise, non-parametric methods determine both parameters and their model structure from data through training. MLbased algorithms, a typical class of non-parametric methods, are driven by big data analytics, allowing ML to discover the patterns within the data automatically. For example, fuzzy logic, k-NN, and SVM are variations of this class of methods. Especially, with the development of parallel processing technology, neural networks are one of the best models for prediction [START_REF] Kumar | Short term traffic flow prediction for a non urban highway using artificial neural network[END_REF], since it can approximate almost all functions without prior knowledge of its functional form, and it is suitable for both linearity and non-linearity problems. By practice and experimentation, ML-based prediction methods can obtain accuracy with a fast learning speed [START_REF] Lee | Deep learning-based corporate performance prediction model considering technical capability[END_REF].

Finally, classical management approaches try to find a sequence of actions that transfer the environment or objects from an initial state to a desired state with some objective. In this kind of management mechanism, the problems are assumed to be fully observable (the state of environment is precisely known), finite (state space and action space is limited), deterministic (the rule of state transfer is known in advanced), and static (only the entity for which we control changes the state) [START_REF] Barták | Constraint satisfaction techniques in planning and scheduling[END_REF]. However, the environment of ITS is more complicated, being unable to meet all the assumptions of classical algorithms. ML approaches, such as RL, which offer methods dealing with infinite state and action space with uncertain effects, are more suitable for ITS management tasks.

Fig. 8 shows the evolution over the recent ten years of the trending application of different mainstream ML technologies in ITS, and this is for the main tasks of perception, prediction, and management. The data used to produce this figure is extracted from related published papers between 2010 and 2020. In this figure, the radius of a circle is proportional to the number of papers on each task and proposing the use of the associated ML technology, the center of the circle is the weighted average publication year, where the weight of a year is the ratio of the number of published papers in this year to the number of published papers over all years considered. The first observation here, which is in line with the result in Fig. 1, is that the number of circles tends to increase with years, thus highlighting the trending application of ML in ITS. Furthermore, DNN-driven technologies are becoming more and more popular. Secondly, we can also observe that some technologies are more popular for some tasks than others. For example for perception, CNN seems to be the most attractive in the recent three years, followed by R-CNNs and You only look once (Yolo) [START_REF] Bochkovskiy | Yolov4: Optimal speed and accuracy of object detection[END_REF], which got increased attention since 2019. For the prediction task, the LSTM technology has been mostly used the recent 3 years, followed by RNN and CNN. Older years than 2019 has seen a surge in the popularity of the SVM technology for the prediction task, before this popularity being decreased in the following years. For the management task, Qlearning has been first mostly used, before leaving its place to more advanced techniques such as DQN and DDPG starting from 2019.

In brief, the application of ML technologies has considerably inspired the ITS revolution and the intelligent upgrade of its main tasks, with a long list of research works being proposed to advance the state of the art on the topic. In the following sections, we review relevant ML-based works on perception, prediction and management of ITS and discuss their role in each of the specific problems.

IV. ML-DRIVEN PERCEPTION IN ITS

Given the suitability of ML approaches to deal with image processing, ML-driven perception is introduced mainly from the view of vision-based perception. Giving that perception tasks can be applied on different ITS topics and scopes, we elected the ITS topic focused by each related work focused alongside the ML approaches utilized and the role performed by ML. Considering the topics surveyed, four major categories for tasks were profiled: road, vehicles, users, and networking. These categories and all the information related to them are grouped within Table V.

A. Perception of Road

Traffic flow and behavior are affected by different road transportation elements. Roads, freeways, and bridges are full of signaling infrastructure responsible for dictating traffic flow (e.g., road surface markings and traffic signals) and enforcing the desired traffic behavior (e.g., road signs). Road condition (e.g., road integrity and wetness level) and surrounding scene (e.g., obstacles, trees, and guardrails) are also relevant, given their impact on driving behavior [START_REF] Tang | A new car-following model accounting for varying road condition[END_REF][START_REF] Kim | The impact of cooperative perception on decision making and planning of autonomous vehicles[END_REF]. Because of this, the road has useful information that can be utilized by applications. For instance, in co-operative driving, the vehicle needs to be aware of other vehicles and road conditions to define a driving policy [START_REF] Kim | The impact of cooperative perception on decision making and planning of autonomous vehicles[END_REF]. Therefore, the role of road perception is to make the information present in the road available for ITS applications.

1) Perception of Road Signs: Road signs are installed at the side or above roads to give instructions with different shapes, colors, and text. Given the high number of road signs, it is too expensive to install and maintain a sensing infrastructure in each one of them. Thus, the perception of road signs is mostly realized by visiondriven system embedded in each vehicle.

As a typical pattern recognition task, the accuracy of the road signs perception mainly depends on the feature extractor and the classifier [START_REF] Baró | Traffic sign recognition using evolutionary adaboost detection and forest-ecoc classification[END_REF]. In the beginning, ML approaches, like SVM [START_REF] Maldonado-Bascón | Road-sign detection and recognition based on support vector machines[END_REF][START_REF] Ruta | Robust class similarity measure for traffic sign recognition[END_REF] and RF [START_REF] Zaklouta | Traffic sign classification using kd trees and random forests[END_REF], were used as classifiers with hand-crafted features. These ML approaches are still insufficient to deal with the not typical (or regular or conforming) images. DNN offers methods for automatic learning of deep features, which are stored in massive data. Especially, CNN [START_REF] Jin | Traffic sign recognition with hinge loss trained convolutional neural networks[END_REF][START_REF] Lim | Real-time traffic sign recognition based on a general purpose gpu and deeplearning[END_REF][START_REF] Qian | Robust chinese traffic sign detection and recognition with deep convolutional neural network[END_REF] showed its outstanding capabilities of feature-learning in the road signs perception.

Although the CNN-based methods demonstrated their efficiency for this kind of application, they still have some drawbacks. CNN-based approaches usually deal with images in RGB space, which have a negative effect on the representation learning of CNN, in particular, due to non-uniform color distribution and information coupling of RGB space. For instance, DP-KELM [START_REF] Zeng | Traffic sign recognition using kernel extreme learning machines with deep perceptual features[END_REF], which is a kernel extreme learning machine (ELM) classifier with deep perceptual features, is a learning method from the perceptual LAB color space instead of the RGB space. On the other hand, when the sign recognition task uses a video instead of a single image, DNN-based methods may obtain good results but they require high computing resources, such as GPUs. To cope with that, incremental SVM and multi-class SVM were used in [START_REF] Yuan | An incremental framework for video-based traffic sign detection, tracking, and recognition[END_REF] alongside a scale-based voting method that combines the classification results of multi-images on the same signs in a moving vehicle.

2) Perception of Surrounding Scene and Road Conditions: The road and surrounding scene detection is an essential task for some ITS applications, such as a driving assistance application. Regarding the perception of the road scene in ITS, image segmentation is an important method. For example, classifying single image patches with CNN is an approach in which the pixels of an image are classified into the road and nonroad parts [START_REF] Brust | Convolutional patch networks with spatial prior for road detection and urban scene understanding[END_REF][START_REF] Oliveira | Efficient deep models for monocular road segmentation[END_REF]. More precise approaches, such as SegNet [START_REF] Badrinarayanan | Segnet: A deep convolutional encoder-decoder architecture for image segmentation[END_REF] and DeconvNet [START_REF] Noh | Learning deconvolution network for semantic segmentation[END_REF] use efficient encoderdecoder CNN based models for image segmentation, which have the ability to model appearance, shape and can understand the spatial relationship between different classes (such as roads and sidewalks). Some researches focused on specific object recognition in ITS, such as lane detection [START_REF] Gurghian | Deeplanes: End-to-end lane position estimation using deep neural networks[END_REF] and obstacle detection [START_REF] Dairi | Unsupervised obstacle detection in driving environments using deeplearning-based stereovision[END_REF][START_REF] Levi | Stixelnet: A deep convolutional network for obstacle detection and road segmentation[END_REF].

The recognition of vehicle parking is focused on detecting parking occupancy [START_REF] Amato | Car parking occupancy detection using smart camera networks and deep learning[END_REF][START_REF] Ling | Identifying parking spaces & detecting occupancy using vision-based iot devices[END_REF] along the road or in a parking lot. The occupation detection of parking offers visibility into parking space vacancies, which is used to assist the selection of a parking location. CNN, for example, offers advantages for occupancy detection by image processing [START_REF] Amato | Car parking occupancy detection using smart camera networks and deep learning[END_REF]. Ling et al. [START_REF] Ling | Identifying parking spaces & detecting occupancy using vision-based iot devices[END_REF], in turn, used not only ML-driven local agents but also remote ones by leveraging Amazon web services to solve the vehicle parking problem.

Furthermore, road surface conditions have a significant impact on transport safety and driving comfort. For this area, the road surface state classification (including dry, wet, snow, ice, and water) [START_REF] Zhao | Road surface state recognition based on svm optimization and image segmentation processing[END_REF] and road crack detection [START_REF] Zhang | Road crack detection using deep convolutional neural network[END_REF] were discussed.

B. Perception of Vehicles

Vehicle perception covers various aspects, such as vehicle detection, vehicle classification, vehicle identification, and driving behavior.

1) Vehicle Detection: Vehicle detection can find out vehicles in the surrounding environment without the need to distinguish vehicles. In particular, vision-based vehicle detection involves filtering vehicles from an image's background. Appearance-based methods, for example, detect vehicles directly from images. In such methods, a variety of appearance features can be used as cues for vehicle detection, from the more straightforward image features like edges and symmetry features to general and robust features like HOG features, Gabor features and Haar-like features [START_REF] Yang | Vehicle detection in intelligent transportation systems and its applications under varying environments: A review[END_REF].

Before deep learning, ANNs were thought to be out of favor for vehicle detection, since they require several parameters to tune, and the training results tend to converge to a local optimum [START_REF] Wen | A rapid learning algorithm for vehicle classification[END_REF]. Thus, researches were focused on classifiers whose training converges to a global optimum, such as SVM and AdaBoost. SVM was used as a classifier in vehicles detection with different features, such as HOG [START_REF] Sun | Monocular precrash vehicle detection: features and classifiers[END_REF] and Haarlike features [START_REF] Wen | Efficient feature selection and classification for vehicle detection[END_REF]. Compared to SVM classifiers, Ad-aBoost offers advantages in automatically finding relevant features for classification in a vast feature pool, and it was proved to have impressive performance in vehicle detection [START_REF] Tang | Vehicle detection and recognition for intelligent traffic surveillance system[END_REF]. However, the training process of AdaBoost is quite time-consuming, so to tackle this weakness, an improved AdaBoost algorithm [START_REF] Wen | A rapid learning algorithm for vehicle classification[END_REF] was proposed for vehicle detection with Haar-like features. However, in recent years, deep models have proven to be more accurate for classification and detection across almost all object types. Especially, CNN can minimize the work for designing features, model objects and the need to rely on additional sensors [START_REF] Fan | A closer look at faster R-CNN for vehicle detection[END_REF]. For example, faster R-CNN [START_REF] Fan | A closer look at faster R-CNN for vehicle detection[END_REF][START_REF] Yu | A model for fine-grained vehicle classification based on deep learning[END_REF] was adopted in vehicle detection.

2) Vehicle Classification: Vehicle classification aims to categorize vehicles into different groups according to their appearance based on vehicle detection. Compared to typical image classification, vehicle classification, especially fined-grained vehicle classification, is more challenging. The reason is that many vehicle models are similar and difficult to distinguish. However, each kind of vehicles presents some unique features, such as logos, wheels, and headlights, which makes slight differences in appearance among different but similar vehicle models. Thus, exploiting these vehicles' features can improve the classification accuracy.

Traditional vision-oriented classification uses a shallow classification model, such as SVM [START_REF] Chen | Road vehicle classification using support vector machines[END_REF] and RF [START_REF] Joshi | Application of random forest algorithm to classify vehicles detected by a multiple inductive loop system[END_REF], to classify an image based on the features extracted from the whole images. Recently, CNN [START_REF] Yu | A model for fine-grained vehicle classification based on deep learning[END_REF][START_REF] Liu | An ensemble deep learning method for vehicle type classification on visual traffic surveillance sensors[END_REF] was widely applied to vehicle classification and made a huge breakthrough in learning the feature representation from raw images automatically. Even though CNN has achieved great success in vehicle classification, each pixel of an image is treated without distinction, which limits the capability of capturing and highlighting the nuances in the critical features for clas-sification. For fine-grained classification, CNNVA [START_REF] Zhao | Deep reinforcement learning with visual attention for vehicle classification[END_REF] integrates multi-glimpse and visual attention mechanism into CNN, and it uses DRL to find the critical areas of an image to assist vehicle classification.

3) Vehicle Identification: Vehicle identification aims to identify specific vehicles. In contrast to vehicle classification, it can distinguish individuals and describe the objects in details. Vehicle re-identification (V-reID) [START_REF] Khan | A survey of advances in visionbased vehicle re-identification[END_REF] is an essential branch of vehicle identification, whose role is to identify if a particular vehicle is the same one as observed on a previous occasion.

V-reID can also be considered as a vehicle tracking problem with multi-cameras. Both hand-crafted features [START_REF] Liu | Large-scale vehicle reidentification in urban surveillance videos[END_REF] and deep features [START_REF] Liu | A deep learning-based approach to progressive vehicle re-identification for urban surveillance[END_REF] were exploited in existing vision-based researches of V-reID. Most of these researches focused on utilizing the license number plate recognition [START_REF] Masood | License plate detection and recognition using deeply learned convolutional neural networks[END_REF][START_REF] Wen | An algorithm for license plate recognition applied to intelligent transportation system[END_REF] to identify or re-identify vehicles. Liu et al. [START_REF] Liu | Large-scale vehicle reidentification in urban surveillance videos[END_REF][START_REF] Liu | A deep learning-based approach to progressive vehicle re-identification for urban surveillance[END_REF] considered both handcrafted features (color and texture features) and highlevel semantic information extracted by CNN for V-reID. Besides, they exploited Siamese Neural Network (SNN) for the verification of license number plates of vehicles, which consists of twin networks that accept distinct inputs but are joined by an energy function at the top. Liu et al. [START_REF]PROVID: Progressive and multimodal vehicle reidentification for large-scale urban surveillance[END_REF] also utilized the spatio-temporal cues of vehicles in order to improve the V-reID accuracy for vehicles that are spatially and temporally close to each other. DRDL [START_REF] Liu | Deep relative distance learning: Tell the difference between similar vehicles[END_REF] exploited a two-branch deep CNN to map vehicle images into an Euclidean space where the L2 distance can be directly used to measure the similarity of two arbitrary vehicles.

4) Driving Behavior: Driving behavior recognition is the task responsible for recognizing the actions that a vehicle makes, such as braking, steering, accelerating, and lane changing. Related to vision-enable tasks, CNN can be used to recognize a vehicle braking through its brake-lights [START_REF] Wang | Appearance-based brake-lights recognition using deep learning and vehicle detection[END_REF] and SVM can determine the vehicle orientation [START_REF] Yuan | Learning a family of detectors via multiplicative kernels[END_REF]. On the other hand, kinetics data, such as speed and acceleration, can be paired with SVM to identify abnormal lane changing [START_REF] Ramyar | Identification of anomalies in lane change behavior using one-class svm[END_REF] or with RF for identifying vehicle steering pattern [START_REF] Ouyang | Improved vehicle steering pattern recognition by using selected sensor data[END_REF]. Besides that, SVM can also be used for some abnormal driving behaviors detection [START_REF] Chen | D 3 : Abnormal driving behaviors detection and identification using smartphone sensors[END_REF], which includes weaving, swerving, side slipping, fast U-turn, turning with a wide radius and sudden braking.

C. Perception of Users

One of the main participants of ITS are the users, given their interaction as drivers and pedestrians in the ITS environment. In this subsection, we highlight the user-oriented perception tasks, grouped under driving style and pedestrian detection.

1) Recognition of Driving Style: Driving style [141] can be defined as the way the driver controls the vehicle in the context of the driving scene and external conditions, such as time, weather, and mood. Given that the driver's fault is one of the most common causes of traffic accidents [START_REF] Singh | Critical reasons for crashes investigated in the national motor vehicle crash causation survey[END_REF], driving style plays an essential role in ITS, especially for driving safety and advanced driving assistance systems. The data used to perform driving style evaluation can be collected from different sources. The most common sources are smartphones [START_REF] Engelbrecht | Survey of smartphone-based sensing in vehicles for intelligent transportation system applications[END_REF], the On Board Diagnostic system (OBD) [START_REF] Baltusis | On board vehicle diagnostics[END_REF] and embedded systems equipped with vision and kinetic sensors [START_REF] Kaplan | Driver behavior analysis for safe driving: A survey[END_REF]. Features are usually extracted from the collected data based on experiments, expertness or heuristics [START_REF] Bejani | Convolutional neural network with adaptive regularization to classify driving styles on smartphones[END_REF]. Given the variety of features, numerous researches are motivated to study ML for driving style recognition.

RF is one of the most used algorithms in this task, proving itself as a good alternative to profile driving style from smartphone data [START_REF] Júnior | Driver behavior profiling: An investigation with different smartphone sensors and machine learning[END_REF] and embedded systems [START_REF] Wang | Driver modeling based on vehicular sensing data[END_REF]. In a similar application, RF was used to identify the same driving style across multiple vehicles [START_REF] Lestyan | Extracting vehicle sensor signals from CAN logs for driver re-identification[END_REF] and identify specific drivers using data from a single accelerometer sensor [START_REF] Li | Driver identification in intelligent vehicle systems using machine learning algorithms[END_REF]. Besides that, Kmeans clustering is another widespread technique that can be used to group information in datasets accordingly to driver style. For instance, K-means was applied to classify driver aggressiveness [START_REF] Martinelli | Cluster analysis for driver aggressiveness identification[END_REF], alongside SVM to differentiate drivers [START_REF] Van Ly | Driver classification and driving style recognition using inertial sensors[END_REF] and alongside RNN to model lane-changing behavior [START_REF] Ren | A new lane-changing model with consideration of driving style[END_REF]. Another classical technique for recognition is k-NN, which was used by Vaitkus et al. [START_REF] Vaitkus | Driving style classification using long-term accelerometer information[END_REF] to classify driving style into aggressive or normal with 3-axis accelerometer signal statistical features. In a search to automate features extraction and take advantage of hidden features as well, CNN was used to classify driving styles with smartphone [START_REF] Bejani | Convolutional neural network with adaptive regularization to classify driving styles on smartphones[END_REF] data and DL was exploited to model driving risk from OBD and GPS information [START_REF] Bian | A deep learning based model for driving risk assessment[END_REF].

2) Detection of Pedestrians: Avoiding collisions with pedestrians is one of the critical aims of safe driving. The main challenges of the pedestrian detection task are due to the cluttered background and significant occlusions. As many other vision-based tasks, a breakthrough has been achieved in the field of pedestrian detection thanks to ML (especially DL) [START_REF] Brunetti | Computer vision and deep learning techniques for pedestrian detection and tracking: A survey[END_REF].

First, hand-crafted features, such as Haar-like features [START_REF] Zhang | Informed haarlike features improve pedestrian detection[END_REF] and HOG [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF], are used for this task. Recently, deep learning features have been found to be effective in pedestrian detection. Sermanet et al. [START_REF] Sermanet | Pedestrian detection with unsupervised multi-stage feature learning[END_REF] used unsupervised feature learning for a two-layer CNN based on convolutional sparse coding. On the other hand, Du et al. proposed a Fused-DNN (F-DNN) [START_REF] Du | Fused DNN: A deep neural network fusion approach to fast and robust pedestrian detection[END_REF] to improve the robustness and computational performance of pedestrian detection, while Li et al. [START_REF] Li | Scaleaware fast R-CNN for pedestrian detection[END_REF] proposed a scale-aware fast R-CNN model, which has a good performance in detecting pedestrians with different spatial scales. Besides that, some approaches focus on occlusion handling to improve the accuracy of pedestrian detection. For example, DBN [START_REF] Ouyang | A discriminative deep model for pedestrian detection with occlusion handling[END_REF] was employed to learn the visibility masks for different body parts, and FasterRCNN was proposed in [START_REF] Zhang | Occluded pedestrian detection through guided attention in CNNs[END_REF] to detect occluded pedestrians.

D. Perception of Networking Conditions in ITS

Some ITS applications are deployed in an open access data-sharing environment where huge amounts of messages of different types are exchanged. Although congestion and delay in the network cannot be avoided in such an environment, their impact can be dampened, especially for critical applications like road hazard warning. This can be done with the classification and prioritization of messages or applications, where critical ones have more access to network resources. For this task, the K-means algorithm can be used to cluster messages by classes with different access to resources [START_REF] Taherkhani | Centralized and localized data congestion control strategy for vehicular ad hoc networks using a machine learning clustering algorithm[END_REF]. Another approach would be to use SL to rank the messages according to their features, like spatial and temporal features [START_REF] Szczurek | Spatio-temporal information ranking in VANET applications[END_REF], or to rank nodes to decide the next-hop of such messages [START_REF] Zhao | A svm based routing scheme in VANETs[END_REF].

In open-access ITS networking environment, malicious nodes can insert or modify the exchanged information for their own advantage. Moreover, attackers can use the interfaces that enable V2X communication as a means to gain access to private information or even the control of a transportation system. This behavior raises security and privacy concerns in vehicular networks [START_REF] Qu | A security and privacy review of VANETs[END_REF], leveraging the detection of safety hazards as essential in ITS. ML has been exploited to improve the accuracy and speed of such detection. Some examples include LSTM for controller area network bus anomaly detection [START_REF] Taylor | Anomaly detection in automobile control network data with long short-term memory networks[END_REF], DRL for malicious network traffic detection [START_REF] Lyamin | AI-based malicious network traffic detection in VANETs[END_REF], RF for jamming detection [START_REF] Puñal | Machine learning-based jamming detection for ieee 802.11: Design and experimental evaluation[END_REF], and DBN for intrusion detection in the in-vehicle networks [START_REF] Kang | Intrusion detection system using deep neural network for in-vehicle network security[END_REF].

V. ML-DRIVEN PREDICTION IN ITS ML approaches have achieved state-of-art performance on prediction problems in ITS, mainly providing tasks that can be categorized in prediction of traffic flow, travel time, behavior of vehicles, behavior of users, and road occupancy. The ITS topics related to prediction tasks are grouped in Table VI, which also presents the ML approaches and the role performed by ML in each topic.

A. Prediction of Traffic

Forecasting traffic flows is typically a time-series problem. Therefore, traditional methods try to capture temporal dependencies in time series data using classical time-series models, such as the autoregressive moving average [START_REF] Das | Time series analysis[END_REF]. Due to the stochastic and nonlinear nature of traffic flows, traditional methods have a minimal effect. To improve the performance, some ML approaches, such as k-NN [START_REF] Chang | Dynamic near-term traffic flow prediction: system-oriented approach based on past experiences[END_REF] and SVR [START_REF] Jeong | Supervised weighting-online learning algorithm for shortterm traffic flow prediction[END_REF], were used to address the traffic prediction problem. In the last couple of decades, deep learning has drawn a lot of academic and industrial interest in this problem, which is driven by the expressive DNNs. RNN and LSTM [START_REF] Ma | Long shortterm memory neural network for traffic speed prediction using remote microwave sensor data[END_REF][START_REF] Tian | Predicting short-term traffic flow by long short-term memory recurrent neural network[END_REF] were also exploited to depict temporal dependencies. To improve the accuracy of prediction, not only the temporal dependencies but the spatial dependencies should be considered. Generally, CNN [START_REF] Ma | Learning traffic as images: a deep convolutional neural network for large-scale transportation network speed prediction[END_REF] is more suitable for finding spatial dependencies from image-like data. However, elementary ANNs, such as RNNs, LSTMs, and CNNs, fail to obtain spatial and temporal dependencies simultaneously. To deal with this challenge, some studies tried to combine the characteristics of RNN or LSTM with CNN [START_REF] Cheng | Deeptransport: Learning spatial-temporal dependency for traffic condition forecasting[END_REF][START_REF] Wang | Traffic speed prediction and congestion source exploration: A deep learning method[END_REF][START_REF] Wu | Short-term traffic flow forecasting with spatial-temporal correlation in a hybrid deep learning framework[END_REF][START_REF] Jia | Rainfall-integrated traffic speed prediction using deep learning method[END_REF]. Besides that, abundant researches exploited new architectures of neural networks, such GNN [START_REF] Shahsavari | Short-term traffic forecasting: Modeling and learning spatio-temporal relations in transportation networks using graph neural networks[END_REF], Stack Autoencoders (SAE) [189] and STGCN [START_REF] Yu | Spatio-temporal graph convolutional neural network: A deep learning framework for traffic forecasting[END_REF]. Furthermore, in addition to spatial and temporal data, external features, such as the weather [START_REF] Jia | Rainfall-integrated traffic speed prediction using deep learning method[END_REF][START_REF] Koesdwiady | Improving traffic flow prediction with weather information in connected cars: a deep learning approach[END_REF][START_REF] Soua | Big-data-generated traffic flow prediction using deep learning and dempster-shafer theory[END_REF], were considered in traffic flow prediction.

B. Prediction of Travel Time

Travel time prediction is of great importance for traffic control, path planning, vehicle dispatching (e.g., buses and trains), and so on. However, it is a complex and challenging problem, which is affected by diverse factors, including spatial correlations, temporal dependencies, and external conditions (e.g., weather and traffic lights). In regard to its implementation, there are two main approaches: segment-based estimation and pathbased estimation. Firstly, the segment-based estimation method splits a path into several road segments (or links). The prediction of travel time is based on the estimation of the travel time for each segment. Some approaches were proposed to estimate the travel time of road segments, such as SVR [START_REF] Wu | Travel-time prediction with support vector regression[END_REF], LSTM [START_REF] Duan | Travel time prediction with LSTM neural network[END_REF], Restricted Boltzmann Machine (RBM) and SVM [START_REF] Niu | DeepSense: A novel learning mechanism for traffic prediction with taxi GPS traces[END_REF], and gradient BRT [START_REF] Zhang | Urban link travel time prediction based on a gradient boosting method considering spatiotemporal correlations[END_REF]. Although these methods can estimate travel time of each segment accurately, they fail to capture the traffic conditions of the entire path, such as road turns, intersections and traffic lights. Thus, merely summing up the travel time of each road segment in the path results in low accuracy of prediction. Secondly, the path-based estimation method is to estimate the travel time of the entire path [START_REF] Wang | Travel time estimation of a path using sparse trajectories[END_REF] directly. ML approaches, such as DBN [START_REF] Siripanpornchana | Travel-time prediction with deep learning[END_REF], RNN [START_REF] Chen | An arrival time prediction method for bus system[END_REF] and Deep Extreme Learning Machines (DELM) [START_REF] Oneto | Train delay prediction systems: a big data analytics perspective[END_REF] and Graph Attention Networks (GATs) [START_REF] Fu | Compacteta: A fast inference system for travel time prediction[END_REF], showed their strength in solving this problem. However, it is challenging to find a good data set which covers all possible paths. These problems may reduce confidence in the estimation of travel time with incomplete data sets.

To address these issues of segment-based and pathbased methods, some approaches have been proposed. For example, DeepTTE [START_REF] Wang | When will you arrive? estimating travel time based on deep neural networks[END_REF] integrated the segmentbased and path-based approaches, in which a geo-based convolutional layer is used to transform the raw GPS sequence to a series of feature maps, and LSTM is used to learn the temporal dependencies of feature maps.

C. Behavior Prediction of Vehicles and Users

Behavior prediction is a fundamental task for many ITS applications, such as in the exchange of intentions performed in co-operative driving. ML offers potential for automatically predicting the behavior and inferring the action intent of vehicles and users. Vehicle behavior corresponds to actions of vehicles include braking, steering, lane change and even moving trajectory. User behavior, in turn, includes motion trajectory and actions of pedestrians (e.g., running, crossing the street, interacting with objects) and the vehicles' actions induced by drivers considering non-self-driving vehicles.

To offer better performance to ITS automation, the prediction of vehicle behavior is an important issue to tackle. Due to the complex and dynamic ITS environment, this problem is not as simple as regular moving object tracking. For example, the vehicle motion is affected by various latent factors including road conditions, traffic rules, and driver's driving style. Traditional approaches use sophisticated models to predict vehicles behavior with these factors, such as dynamic Bayesian network [START_REF] Gindele | Learning driver behavior models from traffic observations for decision making and planning[END_REF] and Gaussian mixture models [START_REF] Wiest | Probabilistic trajectory prediction with gaussian mixture models[END_REF]. Although these methods claim to have good prediction accuracy, the complexity of training and manual intervention on factor selection are their drawbacks. ML approaches offer an opportunity to such issues. For example, SVM [START_REF] Dogan | Autonomous driving: A comparison of machine learning techniques by means of the prediction of lane change behavior[END_REF][START_REF] Kim | Prediction of driver's intention of lane change by augmenting sensor information using machine learning techniques[END_REF] showed good performance in predicting lane changes. On other hand, the CNNbased approach proposed in [START_REF] Maqueda | Event-based vision meets deep learning on steering prediction for self-driving cars[END_REF] was more accurate in predicting car steering angle. Besides, the trajectory of vehicles can be considered as time sequence data. Thus, RNNs [START_REF] Ondruska | Deep tracking: Seeing beyond seeing using recurrent neural networks[END_REF] and LSTMs [START_REF] Kim | Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network[END_REF] were used to improve vehicle trajectory prediction. Considering some real-time systems have strict time constraints, CNNs [START_REF] Djuric | Motion prediction of traffic actors for autonomous driving using deep convolutional networks[END_REF] were proposed to estimate the vehicle trajectory instead of RNNs and LSTMs.

Prediction of actions of pedestrians is a prerequisite for safe driving, such as for collision avoidance applications. Traditional model-based methods use handcrafted factors, such as the walking speed of pedestrians. Furthermore, it is challenging to combine all factors (for example, road conditions, walking styles of pedestrians) into one model, which limits the task performance in complex and crowded scenes, such as in an urban environment. Subsequently, the ML approaches show

k-NN, SVR, LSTM

Learning traffic patterns with a time series of traffic data. [START_REF] Cheng | Deeptransport: Learning spatial-temporal dependency for traffic condition forecasting[END_REF][START_REF] Wang | Traffic speed prediction and congestion source exploration: A deep learning method[END_REF][START_REF] Wu | Short-term traffic flow forecasting with spatial-temporal correlation in a hybrid deep learning framework[END_REF][START_REF] Yu | Spatio-temporal graph convolutional neural network: A deep learning framework for traffic forecasting[END_REF] Traffic flow prediction depicting temporal and spatial dependencies.

CNN,RNN, GNN,SAE

Learning traffic patterns with temporal and spatial data. [START_REF] Shahsavari | Short-term traffic forecasting: Modeling and learning spatio-temporal relations in transportation networks using graph neural networks[END_REF][START_REF] Jia | Rainfall-integrated traffic speed prediction using deep learning method[END_REF][START_REF] Koesdwiady | Improving traffic flow prediction with weather information in connected cars: a deep learning approach[END_REF][START_REF] Soua | Big-data-generated traffic flow prediction using deep learning and dempster-shafer theory[END_REF] Traffic flow prediction with correlation between weather and traffic.

DBN

Learning traffic patterns considering the weather feature.

Travel Time [START_REF] Duan | Travel time prediction with LSTM neural network[END_REF][START_REF] Wu | Travel-time prediction with support vector regression[END_REF] Predicting the travel time of road segments. SVR, LSTM Learning travel time patterns with temporal feature. [START_REF] Niu | DeepSense: A novel learning mechanism for traffic prediction with taxi GPS traces[END_REF][START_REF] Zhang | Urban link travel time prediction based on a gradient boosting method considering spatiotemporal correlations[END_REF] Predicting the travel time of road segments. RBM, SVM, BRT Learning travel time patterns with temporal-spatial traffic flow feature. [START_REF] Chen | An arrival time prediction method for bus system[END_REF][START_REF] Oneto | Train delay prediction systems: a big data analytics perspective[END_REF][START_REF] Siripanpornchana | Travel-time prediction with deep learning[END_REF] Travel time prediction of paths for cars, bus and train.

DBN, RNN, DELM

Extracting features and learning travel time pattern. [START_REF] Wang | When will you arrive? estimating travel time based on deep neural networks[END_REF] Travel time prediction with segment-based and path-based approach.

LSTM, CNN Temporal dependencies learning and feature transform.

Behavior [START_REF] Dogan | Autonomous driving: A comparison of machine learning techniques by means of the prediction of lane change behavior[END_REF][START_REF] Kim | Prediction of driver's intention of lane change by augmenting sensor information using machine learning techniques[END_REF] Predicting lane change. SVM Classifying the driver's intention. [START_REF] Maqueda | Event-based vision meets deep learning on steering prediction for self-driving cars[END_REF] Predicting vehicle steering angle. CNN Finding the pattern from vision data. [START_REF] Djuric | Motion prediction of traffic actors for autonomous driving using deep convolutional networks[END_REF][START_REF] Kim | Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network[END_REF][START_REF] Ondruska | Deep tracking: Seeing beyond seeing using recurrent neural networks[END_REF] Vehicle trajectory prediction.

RNN, LSTM, CNN

Inferring future movement of vehicle.

[ [START_REF] Rehder | Pedestrian prediction by planning using deep neural networks[END_REF][START_REF] Xu | Encoding crowd interaction with deep neural network for pedestrian trajectory prediction[END_REF][START_REF] Xue | SS-LSTM: a hierarchical LSTM model for pedestrian trajectory prediction[END_REF][START_REF] Zhang | SR-LSTM: State refinement for LSTM towards pedestrian trajectory prediction[END_REF] Predicting pedestrian actions.

CNN, RNN, LSTM

Extracting of features and anticipating actions and trajectory of pedestrian. Road [START_REF] Hoermann | Dynamic occupancy grid prediction for urban autonomous driving: A deep learning approach with fully automatic labeling[END_REF][START_REF] Hoermann | Learning longterm situation prediction for automated driving[END_REF] Road occupancy prediction for urban region.

CNN

Modeling long-term motion.

[ [START_REF] Alajali | On-street car parking prediction in smart city: A multi-source data analysis in sensorcloud environment[END_REF][START_REF] Shao | Parking availability prediction with long short term memory model[END_REF][START_REF] Zheng | Parking availability prediction for sensor-enabled car parks in smart cities[END_REF] Parking occupancy prediction. SVM, FNN, BRT, LSTM Learning parking occupancy patterns with temporal data. [START_REF] Yang | A deep learning approach to real-time parking occupancy prediction in spatio-termporal networks incorporating multiple spatio-temporal data sources[END_REF][START_REF] Fu | Compacteta: A fast inference system for travel time prediction[END_REF] Parking occupancy prediction in spatiotemporal networks.

GCN, LSTM, GAT

Learning parking occupancy patterns with temporal-spatial features.

their strength on this problem, especially in vision-based prediction of human actions [START_REF] Kong | Human action recognition and prediction: A survey[END_REF]. Similar to the prediction of vehicles, CNNs can be used for image analysis of pedestrians, whereas RNNs or LSTMs are convenient to predict the action and trajectory of pedestrians [START_REF] Rehder | Pedestrian prediction by planning using deep neural networks[END_REF][START_REF] Xu | Encoding crowd interaction with deep neural network for pedestrian trajectory prediction[END_REF][START_REF] Xue | SS-LSTM: a hierarchical LSTM model for pedestrian trajectory prediction[END_REF][START_REF] Zhang | SR-LSTM: State refinement for LSTM towards pedestrian trajectory prediction[END_REF].

D. Prediction of Road Occupancy

In addition to traffic flows, travel time and behaviors of ITS users, the prediction of road occupancy and parking space are also in the scope of prediction tasks in ITS.

Road occupancy prediction is a fundamental task for various ITS applications and systems, like collision avoidance applications. The road occupancy task needs to predict the situation of a set of traffic participants (such as vehicles, pedestrians and so on) in a segment or a region. Traditional approaches can predict the occupancy of a fixed road segment with singlelane [START_REF] Althoff | Online verification of automated road vehicles using reachability analysis[END_REF] and even road segment with multi-lanes [START_REF] Althoff | Set-based prediction of traffic participants on arbitrary road networks[END_REF]. However, the occupancy prediction for a region like an urban environment is a complex problem. To tackle it, Hoermann et al. [START_REF] Hoermann | Dynamic occupancy grid prediction for urban autonomous driving: A deep learning approach with fully automatic labeling[END_REF][START_REF] Hoermann | Learning longterm situation prediction for automated driving[END_REF] proposed a CNN-based approach with an occupancy grid map.

In addition to the road occupancy, the prediction of parking occupancy is also an essential task. With a reliable parking occupancy prediction, proper recommendations and navigation of parking location can be made in advance. To support this strand, a wide range of ML-approaches, such as SVM, FNN [START_REF] Zheng | Parking availability prediction for sensor-enabled car parks in smart cities[END_REF], gradient BRT [START_REF] Alajali | On-street car parking prediction in smart city: A multi-source data analysis in sensorcloud environment[END_REF], and LSTM [START_REF] Shao | Parking availability prediction with long short term memory model[END_REF] have been used. Besides, multiple metrics can be considered in occupancy prediction, such as car parking, traffic speed, pedestrian, parking meter transactions, nearby facilities, and weather conditions. Yang et al. [START_REF] Yang | A deep learning approach to real-time parking occupancy prediction in spatio-termporal networks incorporating multiple spatio-temporal data sources[END_REF] leveraged Graph Convolutional Neural Networks (GCN) to extract the spatial relationships of traffic flows and utilized LSTM to capture their temporal features.

VI. ML-DRIVEN MANAGEMENT IN ITS

The task of management is to plan the actions and distribute resources, supporting ITS applications to achieve its objectives and fair usage of resources (e.g., for communication and computation). In this section, MLdriven ITS management is introduced from two aspects: ITS infrastructure management and ITS resource management. The related work is shown in Table VII.

A. Infrastructure Management

Among the different parts of the ITS environment, the infrastructure is the main vector of interaction between applications and the ITS environment. Because of this, the objectives of ITS applications are achieved through the management of the ITS infrastructure, mainly categorized in (1) management of traffic signals and (2) management of vehicles.

1) Traffic Signal Management: Traffic signal management is a way to alleviate traffic congestion, especially important in urban areas. In the current ITS deployment stage [START_REF] Campolo | From today's VANETs to tomorrow's planning and the bets for the day after[END_REF], advanced traffic signal management (such as the vehicle actuated signal control) is mostly implemented based on information from vehicle-actuated detectors, such as loop detectors. These approaches have a limitation in coping with the fluctuation of traffic demand, especially within short periods. Adaptive traffic signal management, which can adjust the traffic signal [START_REF] Abdoos | Holonic multiagent system for traffic signals control[END_REF][START_REF] El-Tantawy | Design of reinforcement learning parameters for seamless application of adaptive traffic signal control[END_REF][START_REF] Arel | Reinforcement learning-based multi-agent system for network traffic signal control[END_REF][START_REF] Balaji | Urban traffic signal control using reinforcement learning agents[END_REF] Traffic light management with queue and traffic.

Q-learning

Making decisions on traffic light phases. [START_REF] Gao | Adaptive traffic signal control: Deep reinforcement learning algorithm with experience replay and target network[END_REF][START_REF] Genders | Using a deep reinforcement learning agent for traffic signal control[END_REF] Traffic light management with position and speed.

DQN, CNN

Manage traffic light, and traffic information extracting. [START_REF] Zhang | Partially observable reinforcement learning for intelligent transportation systems[END_REF] Traffic light management in partial detection.

DQN

Making decisions on traffic light.

[ [START_REF] Walraven | Traffic flow optimization: A reinforcement learning approach[END_REF][START_REF] Zhu | Accounting for dynamic speed limit control in a stochastic traffic environment: A reinforcement learning approach[END_REF][START_REF] Li | Reinforcement learning-based variable speed limit control strategy to reduce traffic congestion at freeway recurrent bottlenecks[END_REF][START_REF] Wang | A new solution for freeway congestion: Cooperative speed limit control using distributed reinforcement learning[END_REF] Variable speed limit control. Q-learning, MA-DQN Making decisions on limited speed.

Vehicle [START_REF] Abbeel | Apprenticeship learning for motion planning with application to parking lot navigation[END_REF][START_REF] Huy | A practical and optimal path planning for autonomous parking using fast marching algorithm and support vector machine[END_REF] Planning vehicle path or trajectory. SVM, AL Finding trajectory and control vehicles' actions. [START_REF] Liu | Parking like a human: A direct trajectory planning solution[END_REF][START_REF] Yu | Intelligent landvehicle model transfer trajectory planning method based on deep reinforcement learning[END_REF] Planning vehicle trajectory with control motions.

DNN,DDPG

Offering optimal intelligent driving maneuver for trajectory. [START_REF] Bojarski | End to end learning for self-driving cars[END_REF][START_REF] Yang | End-to-end multimodal multi-task vehicle control for self-driving cars with visual perceptions[END_REF] End-to-end vehicle steering and speed control.

CNN

Regressing steering angles and speed from front-view cameras. [START_REF] Bhattacharyya | Simulating emergent properties of human driving behavior using multiagent reward augmented imitation learning[END_REF][START_REF] Kuefler | Imitating driver behavior with generative adversarial networks[END_REF][START_REF] Li | Humanlike driving: empirical decision-making system for autonomous vehicles[END_REF][START_REF] Xu | Establishing styleoriented driver models by imitating human driving behaviors[END_REF] Imitate human behavior for autonomous vehicle.

GAIL,RAIL, DMN

Driving behavior learning.

Networking [START_REF] Huang | Deep reinforcement learning for multimedia traffic control in software defined networking[END_REF][START_REF] Xu | Experience-driven networking: A deep reinforcement learning based approach[END_REF] Network resource management to max the QoE.

DDPG

Routing paths and bandwidth management.

[ [START_REF] An | A contextaware edge-based VANET communication scheme for ITS[END_REF][START_REF] Zhang | Artificial intelligence inspired transmission scheduling in cognitive vehicular communications and networks[END_REF][START_REF] Zhang | A deep reinforcement learning-based trust management scheme for softwaredefined vehicular networks[END_REF][START_REF] Yuan | Dynamic controller assignment in software defined internet of vehicles through multi-agent deep reinforcement learning[END_REF] Network resource management in edge and mobile network.

DRL, MARL

Path finding and resource allocation algorithm.

Resource [START_REF] Salahuddin | Reinforcement learning for resource provisioning in the vehicular cloud[END_REF] Resource provisioning in vehicular clouds.

DRL, PG Decision making of resource provisioning. [START_REF] Qi | Vehicular edge computing via deep reinforcement learning[END_REF][START_REF] Qi | Knowledge-driven service offloading decision for vehicular edge computing: A deep reinforcement learning approach[END_REF] Offload edge computing for the vehicles. A3C Optimization offloading decision. [START_REF] Tanzil | Adaptive scheme for caching youtube content in a cellular network: Machine learning approach[END_REF][START_REF] Wang | Edge caching at base stations with device-to-device offloading[END_REF] Management of the edge caching in base stations.

Q-learning, EL Caching resource provisioning policy learning. [START_REF] He | Softwaredefined networks with mobile edge computing and caching for smart cities: A big data deep reinforcement learning approach[END_REF][START_REF] He | Deep reinforcement learning (drl)-based resource management in software-defined and virtualized vehicular ad hoc networks[END_REF][START_REF] He | Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach[END_REF][START_REF] Tan | Mobility-aware edge caching and computing in vehicle networks: A deep reinforcement learning[END_REF] Optimize network, cache and computing resources in ITS.

DQN

Determining an optimal policy in resources management.

Energy [START_REF] Atallah | Deep reinforcement learning-based scheduling for roadside communication networks[END_REF][START_REF] Atallah | Scheduling the operation of a connected vehicular network using deep reinforcement learning[END_REF][START_REF] Atallah | A reinforcement learning technique for optimizing downlink scheduling in an energy-limited vehicular network[END_REF] Optimize RSU's battery usage.

Q-learning, DQN

Energy-efficient adaptive management algorithm. [START_REF] Hu | Energy management strategy for a hybrid electric vehicle based on deep reinforcement learning[END_REF][START_REF] Qi | Deep reinforcement learning-based vehicle energy efficiency autonomous learning system[END_REF] Vehicle energy management. DRL Adaptive vehicle energy usage algorithm.

according to the real-time traffic demand, is a more practical approach to alleviate traffic congestion. Among all the ML, RL is considered as one of the most promising approaches for adaptive traffic signal management. This is mainly due to the convenience of formulating signal management as a sequential decision-making problem.

Early works of RL used Q-learning for traffic light management (green, yellow and red), considering the number of waiting vehicles or the queue length [START_REF] Abdoos | Holonic multiagent system for traffic signals control[END_REF][START_REF] El-Tantawy | Design of reinforcement learning parameters for seamless application of adaptive traffic signal control[END_REF], and the statistics of traffic flow [START_REF] Arel | Reinforcement learning-based multi-agent system for network traffic signal control[END_REF][START_REF] Balaji | Urban traffic signal control using reinforcement learning agents[END_REF]. However, these parameters are unable to depict the real traffic situation accurately. With the popularization of modern sensors, more information on traffic is extracted and transmitted via the vehicular network, such as the traffic speed and vehicle waiting time. Nevertheless, more information increases the dimension of states, exponentially growing the complexity of traditional RL. To deal with this complexity, DNNs have been employed in RL, forming DRL. DQN [START_REF] Gao | Adaptive traffic signal control: Deep reinforcement learning algorithm with experience replay and target network[END_REF][START_REF] Genders | Using a deep reinforcement learning agent for traffic signal control[END_REF][START_REF] Zhang | Partially observable reinforcement learning for intelligent transportation systems[END_REF] has been proposed with information of position and speed. Besides, instead of hand-crafted features, these studies used CNN to extract machine-crafted features from raw real-time traffic data. Given the growing scale of ITS, some researches investigate promising approaches [START_REF] Zhang | Partially observable reinforcement learning for intelligent transportation systems[END_REF] in a partially observable environment.

Nowadays, modern speed limit signs can be dynamically adjusted according to various factors, such as traffic volume and weather. Variable speed limit management is a flexible way to improve road condition, increase driving safety, and reducing travel time. Some proposals used Q-Learning to estimate the optimal speed limits so as to reduce the travel time [START_REF] Zhu | Accounting for dynamic speed limit control in a stochastic traffic environment: A reinforcement learning approach[END_REF] and decrease traffic congestion [START_REF] Walraven | Traffic flow optimization: A reinforcement learning approach[END_REF][START_REF] Li | Reinforcement learning-based variable speed limit control strategy to reduce traffic congestion at freeway recurrent bottlenecks[END_REF]. Besides, in large-scale networks, Multi-Agent DQN (MA-DQN) under V2I was used for speed limit control [START_REF] Wang | A new solution for freeway congestion: Cooperative speed limit control using distributed reinforcement learning[END_REF].

2) Vehicle Management: The management of vehicles is one of the most critical tasks in modern ITS, especially for autonomous driving. It consists of two primary components: vehicle path (or trajectory) planning and motion control (such as steering angle and vehicle speed control). The scenario of vehicle management includes diverse types of events like parking, lane changing, merging, platooning, and so on.

For path planning, most existing approaches attack this problem by designing a reference path that a vehicle could approximately follow. For example, SVM was used in [START_REF] Huy | A practical and optimal path planning for autonomous parking using fast marching algorithm and support vector machine[END_REF] to provide a safe and feasible path, which has a maximum clearance from obstacles. However, a good path-planning approach needs to consider more complex objectives, including the path length, smoothness, distance to obstacles, lane-keeping, maximum curvature, and so on. Abbeel et al. [START_REF] Abbeel | Apprenticeship learning for motion planning with application to parking lot navigation[END_REF] utilized AL for trajectories planning (called the designed trajectory) considering a lot of metrics based on a demonstration set of realistic parking path trajectories. Because of dynamic constraints (for example, the limited steering angle of an autonomous vehicle) and unforeseen modifications in the environment, some deviations exist between the designed trajectory and the actual trajectory. Liu et al. [START_REF] Liu | Parking like a human: A direct trajectory planning solution[END_REF] proposed a DNN-based method to find the best parking path trajectory by connecting the candidate parking path trajectories and steering actions. Besides, DDPG was proposed to plan vehicle trajectory and decide an optimal driving maneuver [START_REF] Yu | Intelligent landvehicle model transfer trajectory planning method based on deep reinforcement learning[END_REF].

Regarding motion control, most previous approaches try to make a good decision on vehicle motion, where perception and vehicle control are two individual tasks. Inspired by the vision-based perception, motion control can be viewed as an end-to-end task, where CNN can be used to regress steering angles directly from raw pixels recorded by front view cameras [START_REF] Bojarski | End to end learning for self-driving cars[END_REF]. As an extension, speed control can also be used alongside steering angle as a feature [START_REF] Yang | End-to-end multimodal multi-task vehicle control for self-driving cars with visual perceptions[END_REF]. Besides, some works focused on how to imitate human behavior on vehicle motion control. Xu et al. [START_REF] Xu | Establishing styleoriented driver models by imitating human driving behaviors[END_REF] imitated human operations on gas and brake pedals using Partly Connected Multilayered Perceptron (PCMLP). DMN [START_REF] Li | Humanlike driving: empirical decision-making system for autonomous vehicles[END_REF], a Six-layer Decision-Making Network (SDMN), was proposed to learn human decision-making behaviors for autonomous vehicles. GAIL is an excellent method to predict and simulate human driving behavior, which was used in [START_REF] Kuefler | Imitating driver behavior with generative adversarial networks[END_REF]. In nature, human driving scenes are composed of several vehicles, which are inherently multi-agent for imitating multiple human drivers. Reliable human driver models must be capable of catching the interaction between different agents. However, GAIL cannot scale to imitating the behavior of multiple vehicles because of the problem of covariate-shift caused by multi-agent setting. Covariate-shift refers to the change in the distribution of the training data and the production data. To solve this problem, the multi-agent RAIL method was proposed in [START_REF] Bhattacharyya | Simulating emergent properties of human driving behavior using multiagent reward augmented imitation learning[END_REF] to imitate human driving behavior with emergent properties caused by multi-agent interactions.

B. Resource Management

ITS leverages ML in infrastructure management to offer services primarily for road safety and efficiency. However, resource-intensive use-cases (e.g., on-demand multimedia video and live traffic reports) require efficient resource allocation. In support of these use-cases, efficient and intelligent management of local and shared resources is required. In general, the shared resources are located remotely (cloud computing), leveraging the use of RSUs as gateways. However, in ITS, cloud resources are extended to include RSUs and OBUs, which form a vehicular cloud [START_REF] Salahuddin | Reinforcement learning for resource provisioning in the vehicular cloud[END_REF].

Resource management needs to take both the resource availability and the utility of allocation policies into consideration. The previous mainstream approaches of ITS resource management were formulated as optimization problems with objectives and constraints, i.e., the search for optimal solutions. However, this approach is not sufficient in high mobility networks, such as ITS, given the brevity of optimization results validity. Therefore, ITS needs a more dynamic and efficient resource provisioning mechanism considering high mobility environments. On the other hand, it is challenging to formulate a satisfactory objective function that simultaneously accounts for the vastly different goals of the heterogeneous vehicular links. To address these issues, ML were applied to resource management. Next, MLbased resource management is introduced considering each resource category -networking, computing, storage, and energy.

1) Networking Resource Management: The communication network in ITS is split into core networking, and the edge and mobile networking. Firstly, the core networking consists of a set of forwarding equipment with high bandwidth provided by wired links. Secondly, the edge and mobile networks consist of a set of edge nodes (e.g., RSUs) and mobile devices (e.g., vehicles and smartphones). VANET [START_REF] Hartenstein | A tutorial survey on vehicular ad hoc networks[END_REF], for example, is a typical scenario in which edge networks and mobile networks are deployed. The networking resources in VANETs include transmission power, sub-bands, connections between mobile devices and edge nodes, and connections between the mobile devices.

Concerning core networking, dynamic resource management with ML has been studied. Through proactive learning and interaction, the RL framework can manage and allocate resources automatically. Using RL, controllers can observe the changes in demand and resources; thus, they can act as agents of RL. Different objectives have been researched, such as maximal Quality of Experience (QoE) in multimedia traffic [START_REF] Huang | Deep reinforcement learning for multimedia traffic control in software defined networking[END_REF][START_REF] Zhang | DeepQoE: A multimodal learning framework for video qual-ity of experience (QoE) prediction[END_REF] and maximal network utility [START_REF] Xu | Experience-driven networking: A deep reinforcement learning based approach[END_REF] using DDPG.

How to allocate the resources of edge and mobile networks has been studied from different context information, such as communication type (V2I, V2V, unicast and broadcast), connection-dependency (connectiondependent or connection-independent), packet payload size and transmission costs. In [START_REF] An | A contextaware edge-based VANET communication scheme for ITS[END_REF], Q-learning was used to learn the best routing policy for the last twohop communications, and edge nodes work as agents, where ML-techniques were deployed. MARL was used in [START_REF] Ye | Deep reinforcement learning for resource allocation in V2V communications[END_REF] to manage sub-band and power allocation for V2V and V2I communications. In [START_REF] Zhang | Artificial intelligence inspired transmission scheduling in cognitive vehicular communications and networks[END_REF], DQN was used to optimize data transmission management with the goal of minimizing transmission costs. In [START_REF] Zhang | A deep reinforcement learning-based trust management scheme for softwaredefined vehicular networks[END_REF], Dueling Deep Q-Network (DDQN) was proposed to find the most trusted routing path in VANET.

2) Computing and Storage Resource Management: By our investigation, most of the researches focus on cloud and edge resource management. Besides the centralized cloud, which usually consists of data centers, vehicular clouds are also prominent in ITS. Edge computing, in turn, is an alternative to cloud computing, moving the computation and storage to the edge of the network. The current ITS edge computing environment usually contains a number of edge nodes, including computing nodes (located with multiple base stations), cloudlet edge computing servers (deployed with wireless access points located at RSUs), and ad hoc vehicular nodes [START_REF] Qi | Vehicular edge computing via deep reinforcement learning[END_REF].

The mainstream objectives of dynamic computing and caching resource management are threefold: (1) maximize Quality of Service (QoS) and/or QoE [START_REF] Tanzil | Adaptive scheme for caching youtube content in a cellular network: Machine learning approach[END_REF], (2) minimize overhead and (3) minimize the cost [START_REF] Wang | Edge caching at base stations with device-to-device offloading[END_REF] of dynamic resource provisioning. For resource management in vehicular clouds, RL was confirmed to be powerful with these objectives [START_REF] Salahuddin | Reinforcement learning for resource provisioning in the vehicular cloud[END_REF]. For edge com-puting, A3C [START_REF] Qi | Vehicular edge computing via deep reinforcement learning[END_REF], [START_REF] Qi | Knowledge-driven service offloading decision for vehicular edge computing: A deep reinforcement learning approach[END_REF] was used to provide offloading policy. Regarding edge caching management, Q-Learning [START_REF] Wang | Edge caching at base stations with device-to-device offloading[END_REF] and Extreme Learning (EL) [START_REF] Tanzil | Adaptive scheme for caching youtube content in a cellular network: Machine learning approach[END_REF] were used to improve the performance of caching in base stations. Furthermore, various studies jointly considered networking, computing, and caching resource in ITS using DQN [START_REF] He | Softwaredefined networks with mobile edge computing and caching for smart cities: A big data deep reinforcement learning approach[END_REF][START_REF] He | Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach[END_REF][START_REF] Tan | Mobility-aware edge caching and computing in vehicle networks: A deep reinforcement learning[END_REF][START_REF] Khan | Edge-computing-enabled smart cities: A comprehensive survey[END_REF][START_REF] He | Resource allocation in software-defined and information-centric vehicular networks with mobile edge computing[END_REF].

3) Energy Management: The current trend to reduce greenhouse gas emissions, due to climate change and air quality issues [START_REF] Bakker | Policy options to support the adoption of electric vehicles in the urban environment[END_REF], leverages the importance of Electric Vehicles (EV) in the transportation sector. However, managing the energy efficiency in EVs is a problem with a large number of pertinent factors [START_REF] Rigas | Managing electric vehicles in the smart grid using artificial intelligence: A survey[END_REF] (e.g., battery charge level and estimated trip time).

Energy management must consider energy optimization based on the current route [START_REF] Vogel | Improving hybrid vehicle fuel efficiency using inverse reinforcement learning[END_REF] to determine charge/discharge policies. Such optimization can be done with regression algorithms [START_REF] Ermon | Learning policies for battery usage optimization in electric vehicles[END_REF] and RL [START_REF] Liu | Reinforcement learning optimized look-ahead energy management of a parallel hybrid electric vehicle[END_REF]. On the other hand, management applications also have to consider energy-efficient resource management. Given that some RSUs in ITS are powered by battery, ML, such as Q-learning [START_REF] Atallah | A reinforcement learning technique for optimizing downlink scheduling in an energy-limited vehicular network[END_REF] and DQN [START_REF] Atallah | Deep reinforcement learning-based scheduling for roadside communication networks[END_REF][START_REF] Atallah | Scheduling the operation of a connected vehicular network using deep reinforcement learning[END_REF], can be used to extend the battery lifetime. Moreover, taking into account the limited power of vehicles, the energy management of hybrid electric vehicles is an important issue that involves a trade-off between gasoline and electricity. DQN, for example, was used in vehicular energy management for both electrical [START_REF] Qi | Deep reinforcement learning-based vehicle energy efficiency autonomous learning system[END_REF] and hybrid vehicles [START_REF] Hu | Energy management strategy for a hybrid electric vehicle based on deep reinforcement learning[END_REF]. VII. CHALLENGES AND FUTURE TRENDS ML are impacting a multitude of ITS applications. However, we believe that existing studies do not represent the full potential of ML-driven ITS due to both limitations of existing ML approaches and the needs of evolving ITS. In this section, we discuss challenges and some future trends of ITS that deserve further investigation, which are summarized in Table VIII. Challenges. As highlighted by our survey, ITS tasks and services have made extensive use of ML techniques. Yet, there are still a number of important challenges that need to be addressed. We discuss some of those below.

• Safe autonomous driving. Real-time visual understanding of the surrounding environment including the complex social interactions and behavior of drivers, passengers and pedestrians, which is among the basic components of autonomous driving, is still an open research question. In addition to understanding the surrounding environment, deciding how to respond in a safe manner is critical. Also, communicating with other vehicles and people for cooperation are other important considerations. Indeed, driving is a social process that frequently involves complex interactions with other drivers, cyclists and pedestrians. In many of these situations, humans rely on extensive intelligence and common sense (e.g., reasoning and anticipating) which robots are still lacking. • Efficient cooperation in ITS. ML services in ITS are no longer limited to being deployed in centralized and computationally powerful facilities in the cloud. This is largely due to the breakthroughs in edge computing that have enabled ML on edge-and end user devices [START_REF] Zhou | Edge intelligence: Paving the last mile of artificial intelligence with edge computing[END_REF]. However, due to the fact that edge devices typically exhibit energy and computing resource limitations, it is still challenging for edge devices to perform complicated ML tasks alone. Intelligence cooperation is a promising way for device cooperation. Yet, the diversity of ITS applications leads to a wide range of requirements, some of which are quite stringent. • Privacy and security concerns. As intelligent vehicles become increasingly connected, applications such as traffic flow prediction can be achieved by sharing data collected from the vehicle sensors. This, of course, brings up security and privacy concerns.

A. Deep Sensing ITS Most previous works on perception and motion prediction focused on two-dimension (2D). However, in several ITS scenarios like co-operative navigation, 2D models are not enough to describe three-dimension (3D) real-world objects. Existing works on 3D perception mainly rely on LiDAR [START_REF] Chen | Multi-view 3D object detection network for autonomous driving[END_REF][START_REF] Ku | Joint 3D proposal generation and object detection from view aggregation[END_REF] and monocular cameras [START_REF] Mousavian | 3D bounding box estimation using deep learning and geometry[END_REF][START_REF] Xu | Multi-level fusion based 3D object detection from monocular images[END_REF]. LiDAR has the following drawbacks: high cost, relatively short perception range, and sparse information. On the other hand, monocular images do not offer depth information. The shortages of LiDAR and monocular perception lead to low accuracy in 3D object perception. Currently, modern camera devices in ITS can generate stereo images that could be used to provide 3D object perception [START_REF] Li | Stereo R-CNN based 3D object detection for autonomous driving[END_REF]. Besides that, considering the hybrid ITS context where different sources of data are available, how to combine these data to improve the accuracy of 3D perception represents an exciting and critical research area. Furthermore, tasks with higher dimension, such as four-dimension (4D) perception, are still challenging and critical in ITS, especially for autonomous driving. The definition of 4D and 5D may have different definition. For example, work in [START_REF] Xue | BLVD: Building a large-scale 5D semantics benchmark for autonomous driving[END_REF] try to do 4D (3D+temporal) tracking, 5D (4D+interactive) interactive event recognition and 5D intention prediction.

Despite high dimensional sensing, social transportation will undoubtedly also be a key element of future transportation systems. Humans cooperate and interact with each other every day through virtual environments known as social networks enabling huge data exchange. In the transportation context, social networks are generally accessed through mobile personal devices which, in conjunction with data entered by the users, provide spatial, temporal and emotional information about users and their environment [START_REF] Wanichayapong | Social-based traffic information extraction and classification[END_REF]. From this information, useful models for ITS applications can be retrieved, such as models for user emotional behavior, mobility pattern, and traffic-related events (e.g., accident, street blocked, scheduled maintenance in traffic equipment) [START_REF] Zheng | Big data for social transportation[END_REF]. In social transportation, the user acts as a social sensor, perceiving the environment with a perspective different from that provided by hardware sensors. Despite being able to improve ML tasks performance, new types of data sources need to be fused with the data already in place, an endeavor that is still in early stage of development for both scientific and engineering fields [START_REF] Zheng | Big data for social transportation[END_REF]. Despite of this, the social approach for transportation data is being recognized as a field with potential for future researches with a growing number of related works [START_REF] Lv | Social media based transportation research: The state of the work and the networking[END_REF].

Emerging technologies, such as multimodal data fusion [START_REF] Gao | A survey on deep learning for multimodal data fusion[END_REF] and deep reasoning [START_REF] Tran | From deep learning to deep reasoning[END_REF], are promising avenues for deep sensing in ITS. Multimodal data fusion aims to integrate datasets from different sources, dimensions and types into a global space in which both intermodality and cross-modality can be represented in a uniform manner. Deep reasoning aims at extending neural networks to "learn-to-reason" from data, which opens up new ways to get insights from data through reasoning.

B. Cooperative ITS

One of the ITS aims is to automate the interactions among the infrastructure and vehicles to accomplish cooperative work. C-ITS [START_REF] Sjoberg | Cooperative intelligent transport systems in europe: Current deployment status and outlook[END_REF] covers a wide range of applications, relying on the perception, prediction, and management discussed in this work. Among all the cooperative applications, cooperative driving is probably the most interesting and challenging one. The idea of cooperative vehicles, jointly with the wireless communication advancements in ITS, highlights the value of interconnected devices and data sharing in vehicular networks [START_REF] Javed | Data analytics for cooperative intelligent transport systems[END_REF]. Therefore, exploring how to learn and cooperate in ITS is a meaningful trend.

Despite currently being considered to become a standard by ETSI, cooperative services in C-ITS, like the Collective Perception Services (CPS), are still discussing which message exchange methodologies or algorithms should be implemented to improve service performance [START_REF] Thandavarayan | Generation of cooperative perception messages for connected and automated vehicles[END_REF]. Also, the dependence between vehicles in C-ITS raises interesting challenges that aim at balancing the reliability gains resulting and the increased overhead caused by the cooperation [START_REF] Javed | Adaptive security for intelligent transport system applications[END_REF]. Cooperative intelligence [START_REF] Dafoe | Open problems in cooperative AI[END_REF] can integrate multiple device capabilities (e.g., computing and observing) to achieve teamwork with joint goals and shared intentions, which is a promising avenue to support C-ITS. Cooperative intelligence brings new opportunities in enhancing next-generation ITS [START_REF] Dafoe | Cooperative AI: machines must learn to find common ground[END_REF], which has been exploited in various fields, such as mobility management [START_REF] Yuan | Harnessing UAVs for fair 5G bandwidth allocation in vehicular communication via deep reinforcement learning[END_REF], network control, and resource allocation [START_REF] Yuan | Dynamic controller assignment in software defined internet of vehicles through multi-agent deep reinforcement learning[END_REF]. Existing works explored how to train multi-agent cooperatively to make learning efficient, e.g., via centralized learning and networked distributed learning [265].

C. Security and Privacy Enhancement

Because of the features of sharing data with each other in ITS, ITS exhibits a variety of vulnerabilities that can be subject to various threats and attacks. For example, the computational capability of OBUs in vehicles grant vehicles significant computing capability. However, it also enables attackers to occupy computation resources and obtain the private information from vehicles. The main security concerns in ITS include integrity, confidentiality, availability, and attacks on authentication and accountability [START_REF] Takefuji | Connected vehicle security vulnerabilities [commentary[END_REF]. Although security breaches in ITS are often critical and hazardous, deployment of comprehensive security enhancement for ITS is challenging in practice. This is due in part to the fact that typically, ITS systems are quite dynamic with frequent and instantaneous arrivals and departures of vehicles as well as short connection duration. In addition to its dynamic nature and high mobility, the use of wireless communication also makes ITS systems vulnerable to attacks that exploit the open and broadcast nature of wireless communication. Therefore, the security design in physical layer was mentioned in [START_REF] Elhalawany | Physicallayer security and privacy for vehicle-to-everything[END_REF]. However, when the number of ITS devices increases, the complex propagation environments make the security design more complex. ML thus brings new opportunities to increase ITS security, e.g., anomaly detection [START_REF] Khan | Long short-term memory neural networks for false information attack detection in software-defined in-vehicle network[END_REF][START_REF] Han | Event-triggered interval-based anomaly detection and attack identification methods for an in-vehicle network[END_REF]. Moreover, for secure message exchange, blockchain techniques that use consensus mechanisms and encryption algorithms to protect information from being tampered can be applied [START_REF] Lu | Lowlatency federated learning and blockchain for edge association in digital twin empowered 6G networks[END_REF].

The privacy concerns related to sharing individual data hampers cooperation in ITS. For example, in cooperative driving, vehicles could learn trajectory cooperatively for safe driving. However, vehicles may be unwilling to share their datasets (e.g., on-board videos and records of flying behaviors). Another example is collaborative ITS resource management, in which local information (e.g., locations and feedback) is private. Some existing approaches [START_REF] Peng | Facmac: Factored multi-agent centralised policy gradients[END_REF][START_REF] Sunehag | Value-decomposition networks for cooperative multi-agent learning based on team reward[END_REF][START_REF] Zhuo | Federated reinforcement learning[END_REF] try to offer (partial) privacy protection while promoting cooperative intelligence through cooperative learning. However, they either lack adequate privacy protection [START_REF] Peng | Facmac: Factored multi-agent centralised policy gradients[END_REF][START_REF] Sunehag | Value-decomposition networks for cooperative multi-agent learning based on team reward[END_REF] or incur high communication overheads and latency [START_REF] Zhuo | Federated reinforcement learning[END_REF]. Therefore, how to improve the privacy with lower overheads is still an open challenge. Federated learning [START_REF] Li | Federated learning: Challenges, methods, and future directions[END_REF], a promising approach for privacy protection, finds a way out of the data sharing privacy dilemma by training ML models locally at edge devices without the need to exchange data. For example, traffic flow prediction with federated learning was presented in [START_REF] Liu | Privacypreserving traffic flow prediction: A federated learning approach[END_REF].

D. 6G ITS

As described and discussed, both entities and intelligence in modern ITS need to cooperate via communication. To this end, guaranteeing the reliability and efficiency of communication is also critical for nextgeneration ITS. Fortunately, due to the innovation of network technology-the sixth generation (6G), the development of the network has provided a communication foundation for ITS.

On the one hand, as the successor to 5G cellular technology, the goal of 6G is to offer communication with microsecond latency. 6G is expected to facilitate significant improvements in the qualify of networking services for ITS, especially for data-intensive and delay-sensitive applications, such as mobile augmented reality [START_REF] Siriwardhana | A survey on mobile augmented reality with 5G mobile edge computing: Architectures, applications, and technical aspects[END_REF]. However, the emergence of 6G has raised several new challenges. For example, how to allow vehicles and users to enjoy services (e.g., automatic remote driving) while ensuring their safety with reliable networking. SDN is a solution for this challenge because it can offer logically centralized control of networking, e.g., appropriate networking resource allocation and route to trade-off between safety control and QoS. A prototype of software-defined vehicular networking was proposed in [START_REF] Sadio | Design and prototyping of a software defined vehicular networking[END_REF]. Due to the limited battery capacity on vehicles, the authors in [START_REF] Kumar | Green computing in software defined social internet of vehicles[END_REF] further improved the energy efficiency of software-defined vehicular networking.

On the other hand, 6G, working in conjunction with ML, is envisioned intelligent and innovative network. Using ML techniques in 6G vehicular networks for vehicular services received considerable attention from research and communities [START_REF] Tang | Future intelligent and secure vehicular network toward 6G: Machine-learning approaches[END_REF]. Furthermore, integration of ML with 6G can motivate and enable various technologies. For example, federated learning with innetwork computing [START_REF] He | 6G cellular networks and connected autonomous vehicles[END_REF] was applied to autonomous driving [START_REF] Zhou | Twolayer federated learning with heterogeneous model aggregation for 6G supported internet of vehicles[END_REF][START_REF] Khan | A dispersed federated learning framework for 6Genabled autonomous driving cars[END_REF] while considering security issues in 6G. In a conclusion, research about 6G ITS has just begun, and this is one of the current and future research hotspots.

VIII. CONCLUSION

ITS is a field of research and development of rapidly evolving technologies folded into different types of platforms for a myriad of advanced applications. For the deployment and run-time operation of many applications to be effective, the timely acquisition, processing, and analysis of large volumes of data become an essential cornerstone. Therefore, advances in ML are considered as key enabling technologies to drive a revolution in ITS. In this survey, we have investigated how ML has being increasingly proposed to address many of the ITS challenges. To this end, our comprehensive state of the art literature survey covers many-fold perspectives grouped into ITS ML-driven supporting tasks, namely perception, prediction, and management. We also outline some trends that are likely to contribute to the continuous shaping of the future of ITS. We expect this survey to provide basic knowledge for beginners and to encourage new research and insights to the vibrant field of ITS.

Fig. 1 :

 1 Fig. 1: Number of publications on ITS, including MLbased approaches, from 2010 to 2020.

Fig. 3 :

 3 Fig. 3: Proposed application-driven ITS framework.

Fig. 4 :

 4 Fig. 4: Example of ITS applications tasks involved in collision avoidance and road hazard warning (adapted from Alam et al. [25]).

Fig. 5 :

 5 Fig. 5: A taxonomy of mainstream ML approaches.

1 )

 1 Supervised Learning: SL models relationships and dependencies between predicted output and the input features. It does so by inferring a classification or regression from a labeled training dataset. A training dataset is composed by examples used for learning. Labeled data is a group of samples that have been tagged with target variables. Based on the function learned from the training data, SL can predict the output values for new data. According to its role, most of SL algorithms can be split into two major categories: Classification algorithms learn to predict a category as the output for a new observation, on the basis of labeled training data.

Fig. 6 :

 6 Fig. 6: The way of RL works in ITS.

Fig. 7 :

 7 Fig. 7: ML pipeline and interaction between ML and ITS.

Fig. 8 :

 8 Fig. 8: The evolution of ML technologies in ITS over years.

TABLE I :

 I Comparison with other recent surveys on ITS

	Ref. Year Scope	Comments
	[7]	2018 Survey on ML in vehicular networks.

TABLE II :

 II Examples of ETSI's applications[START_REF] Etsi | [END_REF]

	Class	Application	Objective	Use Case Examples
		Driver assistance -co-	Signal the presence of vehicles	Slow vehicle indication, emergency vehi-
		operative awareness	(e.g.,emergency) and inform surrounding	cle warning
	Active Road Safety	Driver assistance-road	vehicles. Warn surrounding vehicles of possible	Stationary vehicle warning, traffic condi-
		hazard warning	hazards (e.g., hard breaking, wrong way).	tion warning
		Cooperative collision
		avoidance or mitiga-
		tion.

TABLE III :

 III Comparison of SL, UL and RL

	Techniques	Data format	Objective	Feedback
	SL	labeled training data	predict	direct
	UL	no-labeled	explore	no feedback
	RL	zero-shot, but interact take action	reward

TABLE IV :

 IV Neural Networks Comparison

	Type	Entities	Relations	Scenario
	FNN	Units	All-to-all	-
	CNN Grid elements	Local	Spatial correlation
	RNN	Time steps	Sequential	Time correlation
	GNN	Nodes	Edges	Node, edge correlations

TABLE V :

 V Researches on ML-based Perception for ITS

	Category Research	Topic	ML	Role of ML
		[76-78]	Traffic sign and marking recognition.	SVM, RF	Classification with hand-crafted fea-
					tures.
		[79-81]	Road signs recognition with RGB single im-	CNN	Classification and recognition method.
			age.
		[82, 83]	Road signs recognition with LAB color space	ELM, SVM	Classification and recognition method.
	Road	[84-87]	and in moving vehicles. Road detection and road scene understanding. CNN	Distinguish different image patches.
		[88]	Road lane detection.	CNN	Detection position of lane.
		[89, 90]	Obstacle detection.	CNN, SVM	Solve as regression.
		[91, 92]	Detect parking occupancy.	CNN	Classifier of parking.
		[93, 94]	Road surface state and road crack recogni-	SVM, CNN	Classification of surface state and esti-
			tion.		mate cracks.
		[95-100]	Vehicle detection using appearance features.	SVM,R-	Classification method of vehicles.
				CNN,Adaboost
		[100-102]	Vehicle classification.	SVM, RF	Classification algorithm.
	Vehicles	[103, 104]	recognition. Vehicle identification with license plate	SVM, CNN	Character recognition of license plates.
		[105-108]	Vehicle re-identification.	CNN, SNN	Feature extractor and classifier.
		[109-113]	Brake, vehicle steering, lane change, orienta-	CNN, RF, SVM	Classify method for driving behaviors.
			tion, drive behavior detection.
		[71, 114-121]	Recognize driving styles of drivers.	K-means, SVM, k-	Classification driving styles into
				NN, RF, RNN	groups.
	Users	[122, 123]	Pedestrian detection using handcrafted fea-	SVM, AdaBoost	Tell pedestrians from the background
			tures.		of images.
		[124-126]	Pedestrian detection using deep features.	UL, CNN, R-CNN Feature learning and classification of
					pedestrians.
		[127-129]	Cluster or rank network messages or nodes.	K-means, SL	Classification the network messages
	Network				and nodes.
		[130-134]	Network safety hazard detection.	LSTM, DRL, RF	Feature extractor and classifier.

TABLE VI :

 VI Researches on ML-based Prediction for ITS

	Category	Research	Topic	ML	Role of ML
		[150-153]	Traffic flow prediction depicting temporal
			dependencies.
	Traffic

TABLE VII :

 VII Researches on ML-based Management for ITS

	Category	Research	Topic	ML	Role of ML
		[
	Signal

TABLE VIII :

 VIII Future Trends in ITS

	Trends	Description and Keywords	Approaches and Technologies
	Deep sensing	High dimensional perception and social transportation Multimodal data fusion and reasoning
	Cooperative ITS	Efficient and reliable cooperation	Cooperative intelligence and learning
	Privacy and security enhancement Privacy protection and anomaly detection	Federated learning and blockchain
	6G ITS	Low-latency and ultra-high speed communication	In-network computing and SDN

ACKNOWLEDGMENTS

This work was partly funded by Inria, supported by the French ANR "Investments for the Future" Program reference #ANR-11-LABX-0031-01, and UNICAMP, through the FAPESP Grant number #2017/50361-0, both in the context of the DrIVE #EQA-041801 Associated Team.

ABBREVIATIONS 5G

Fifth Generation 6G

Sixth Generation