Gérard Meunier

Jean-Michel Guichon

Olivier Chadebec

Bertrand Bannwarth

Laurent Krähenbühl

Christophe Guérin

CMP

Keywords: Volume Integral Formulation, Unstructured-PEEC, Thin regions, Losses

An unstructured-PEEC method for modelling electromagnetic thin regions is proposed. Two coupled circuits representations are used for solving both electric and/or magnetic effects in thin regions discretized by a finite element surface mesh. Dynamic effects across the the thickness of the sheet are modeled by equivalent complex conductivity and reluctivity. Non simply connected regions are treated with fundamental branch independent loop matrices coming from the circuit representation. The formulation enables the computation of eddy current losses and can be coupled with external circuits, PEEC cables or coil thanks to the circuit representation.

I. INTRODUCTION

HE electromagnetic modelling of both magnetic and conductive sheets is an important engineering problem frequently encountered in many applications such as the design of shielding devices or planar conductors. The analysis of eddy current distributions is a complex challenge because of the high ratio between the sheet main dimensions and its thickness e making the volume mesh of the active region very consuming in computing resources with e.g. the finite element method.

Volume Integral Methods (VIM), based on the solving of Maxwell's equations by the volume integration of Green's functions, are very attractive because the air region does not need to be meshed. However, the use of VIM does not prevent from meshing the thickness of the shell in order to catch the high eddy current spatial variation across it due to the skin effect.

Many works have proposed to model the shell with an averaged surface associated to an equivalent electromagnetic behavior for the material. Some formulations, limited to high frequency problems (case δ<<e), has been presented in [START_REF] Miano | A surface integral formulation of Maxwell equations for topologically complex conducting domains[END_REF]- [START_REF] Qian | Fast full-wave surface integral equation solver for multiscale structure modeling[END_REF]. However, these formulations are limited and do not take into account accurately the skin depth effect at lower frequencies. Very low frequency problems have already been solved but by assuming that the skin depth is high compared to the thickness (case δ>>e) like in [START_REF] Lopez | Multilayer integral method for simulation of eddy currents in thin volumes of arbitrary geometry produced by MRI gradient coils[END_REF]. In [START_REF] Krähenbühl | Thin layers in electrical engineering-example of shell models in analysing eddy-currents by boundary and finite element methods[END_REF], the general case is addressed (case δ<e or δ ~ e). Equivalent material behavior laws have been proposed in order to represent the field variation across the shell at every frequency. This approach has been used in the context of VIM but limited to the modelling of conductive but non-magnetic materials [START_REF] Le-Duc | A new integral formulation for eddy current computation in thin conductive shells[END_REF].

Another challenge is the field-circuit coupling which involves a fine mesh of the geometry for local effects while electrical supply and loads are represented by lumped parameters. To use PEEC (Partial Element Equivalent Circuit) method is a good solution. It is a circuit-like integral method which has been applied to shells [START_REF] Gope | S)PEEC: Time-and frequency-domain surface formulation for modeling conductors and dielectrics in combined circuit electromagnetic simulations[END_REF]. The method is very powerful because the circuit representation of the electromagnetic problem makes its coupling with an external electric circuit very easy. However, the standard PEEC approach is limited to structured meshes and the thickness of the shell has to be meshed in the general case. The PEEC method can be generalized on unstructured mesh as proposed in [START_REF] Meunier | A magnetic flux-electric current volume integral formulation based on facet elements for solving electromagnetic problems[END_REF] so it can be applied to every kind of mesh enabling the description of complex geometry. It is based on a volume integral approach based on 3D facet interpolations of the current density and of the magnetic flux density. The approaches can be seen as an unstructured-PEEC method enabling the modelling of both magnetic and conductive regions.

This paper proposes an extension of this unstructured-PEEC method adapted to the modelling of both magnetic and conductive shells in the general case (case δ<e or δ ~ e). The equivalent circuit enables coupling with external circuits, PEEC cables or coil. Finally, the use of equivalent properties for modelling dynamic effects along the thickness of the shell ensures the validity of the model in a high range of frequencies and makes the losses computation possible.

II. FORMULATION

A. Volume Integral Method

Let us consider a linear frequency-domain magnetic problem with magnetic regions ΩM, conductive region ΩJ, and source coils Ω0 (with an imposed current density J0). Based on the solution of Maxwell's equations, electric field E and magnetic field H can be linked to the current densities J and J0, and to the magnetization M with integral expression over the conducting ΩJ (J≠0) and magnetic ΩM (M≠0) domains. In the frequency domain, by neglecting capacitive and propagation effects, and thanks to Lorentz gauge, we have:

𝐄(P) = -jω𝐀(P) -𝐠𝐫𝐚𝐝V(P) 𝐇(P) = 𝐓(P) -𝐠𝐫𝐚𝐝φ(P) (1)
with

𝐓(P) = 1 4π 𝐉 × 𝐠𝐫𝐚𝐝 1 r dΩ + 𝐉 𝟎 × 𝐠𝐫𝐚𝐝 1 r dΩ 𝐀(𝐏) = 𝛍 𝟎 𝟒𝛑 𝐉 𝐫 𝛀 𝐉 𝐝𝛀 + 𝐉 𝟎 𝐫 𝛀 𝟎 𝐝𝛀 + 𝐌 ∧ 𝐠𝐫𝐚𝐝 𝟏 𝐫 𝛀 𝐌 𝐝𝛀 V(P) = 1 jω 1 4πε 𝐉. 𝐠𝐫𝐚𝐝 1 r dΩ φ(P) = 1 4π 𝐌. 𝐠𝐫𝐚𝐝 1 r dΩ (2)
In these expressions r is the distance between observation point P and integration point in media. Volume integral equations are then obtained by matching expression of E and H with the constitutive relationships E(J) and H(M) inside the media. Different formulations can be achieved by discretizing T CMP-372 the regions and by choosing adequate unknowns and interpolation basis functions. The use of 2-form Whitney face interpolation associated to both current density J and magnetic flux density B leads to the unstructured PEEC method [START_REF] Meunier | A magnetic flux-electric current volume integral formulation based on facet elements for solving electromagnetic problems[END_REF].

B. Thin regions model

In the case of thin regions, we consider that the evaluation outside of the media of A and T in equation ( 2) can be obtained by using the averaged tangential current and magnetisation Jm and Mm flowing in the thin regions:

𝐀(P) ≈ μ 4π e 𝐉 𝐦 r dΓ + 𝐉 𝟎 r dΩ + e𝐌 𝐦 ∧ 𝐠𝐫𝐚𝐝 1 r dΓ 𝐓(P) ≈ 1 4π e𝐉 𝐦 ∧ 𝐠𝐫𝐚𝐝 1 r dΓ + 𝐉 𝟎 ∧ 𝐠𝐫𝐚𝐝 1 r dΩ ( 3
)
where e is still the thickness of thin regions, ΓM and ΓJ are the average surface regions of ΩM and ΩJ. Equivalent material behavior laws, which take into account dynamic effects across the thickness, can be used. Following [START_REF] Krähenbühl | Thin layers in electrical engineering-example of shell models in analysing eddy-currents by boundary and finite element methods[END_REF], these equivalent properties link averaged surface electric and magnetic fields Esm, Hsm to averaged current and flux densities Jm, Bm:

𝐄 𝐬𝐦 = 𝐉 𝐦 Gσ 𝐇 𝐬𝐦 = 𝐁 𝐦 Gμ G = tanh (1 + j) (1 + j) (4)
where σ is the conductivity, µ=µ0µr is the permeability and δ is the skin depth. Integral equations are then obtained by matching expressions of (1) on the external surfaces of the thin region with constitutive relationships [START_REF] Lopez | Multilayer integral method for simulation of eddy currents in thin volumes of arbitrary geometry produced by MRI gradient coils[END_REF]. It leads to surface integral equations with Jm and Bm as unknowns.

C. Use of 2D face element on 3D surface

The region Γ is discretized by a surface finite element mesh (composed of triangles and/or quadrangles) on which Jm and Bm are interpolated with 2D first order face elements, i.e. 2D 2forms:

𝐉 𝐦 = 1 e 𝐰 𝐬𝐣 I 𝐁 𝐦 = 1 e 𝐰 𝐬𝐣 ϕ
where 𝐰 𝐬𝐣 is the face function related to j-th face, I and ϕ are the current (in A) and magnetic flux (in Wb) flowing through the j-th face, respectively. Let us notice that, with a surface mesh, face elements are associated to the edges of the mesh. We have:

div 𝐰 𝐬𝐢 = ± 1 s 𝐰 𝐬𝐢 . 𝐧 𝐢 = ± 1 l ( 5
)
where s is the area of the e-th element, and l and 𝐧 𝐢 are respectively the length and the normal of i-th face (geometric edge). The signs in (5) depend on the choice of the global orientation of the face. These relations ensure the continuity of Jm.n and Bm.n between adjacent elements. By using (5), considering scalar potentials V and φ and thanks to the divergence theorem, we have (Fig. 1):

𝐰 𝐢𝐬 . 𝐠𝐫𝐚𝐝 V = ΔV 𝐰 𝐢𝐬 . 𝐠𝐫𝐚𝐝 φ = Δφ (6)
Fig. 1. Primal and dual mesh (dotted lines). ΔV and Δφ are the difference between the two mean values of potentials V and φ in both adjacent elements connected to face i These properties will be useful to construct an equivalent circuit representation based on the dual mesh, as developed in the next section.

D. Unstructured PEEC model for thin regions

Applying a Galerkin projection with facet functions wi on equations (1), and by considering (3), ( 4) and ( 6), we get a circuit representation on the dual mesh: The equivalent circuit (on the dual mesh) is composed of branches connecting two adjacent elements (i.e. face of the primal mesh), where elements are the nodes of the dual mesh. {ΔVs} and {Δφs} represent the differences of electric and magnetic potentials on the branches of the dual mesh. Rs and Y are sparse finite element matrices while L , C and C are full interaction matrices. Let us notice that Rs can be physically seen as an equivalent resistance matrix, Y an equivalent reluctance matix, L the inductance matrix, C and C the interaction matrices coupling both conductive and magnetic effects.

ΔV Δφ = R + jωL jω C C Y I ϕ + S T (7) with R
In order to take into account magnetic fluxes flowing out from magnetic regions, equations ( 7) must be completed. The value of the magnetic potentials on each face element of ΓM can be obtained from equations [START_REF] Qian | Fast full-wave surface integral equation solver for multiscale structure modeling[END_REF]. Considering a constant permeability for each magnetic region, we have divM=0 inside ΩM, and then, using the divergence theorem:

𝐌. 𝐠𝐫𝐚𝐝 1 r dΩ ≈ 𝐌. 𝐧 dΓ = 1 μ μ -1 μ 1 s ϕ dΓ
where ϕ is the magnetic flux flowing out from both sides of face element k of the magnetic region. The averaged value of magnetic potentials on each element of ΓM can then be written as:

{φ } = Q {ϕ } with Q = 1 4πμ 1 s μ -1 μ 1 s 1 r dΓ dΓ
{ϕ } are the magnetic fluxes flowing out from both external faces elements of ΓM. By considering that the magnetic potential is null at infinity, the magnetic circuit is completed by adding branches which connect nodes (i.e. surface elements) of ΓM to the infinity (Fig. 2). Then, the complete set of equations which leads to the equivalent circuit interpretation of the low frequency electromagnetic problem is given by :

ΔV Δφ Δφ = R + jωL jω C 0 C Y 0 0 0 Q I ϕ ϕ + S T 0 (8)
The solution can then be obtained using a circuit solver based on an independent loop search technique which provides a unique solution. In practice fundamental circuit equations to be solved are:

[M ]{∆V } = 0 M {∆φ} = 0 with {∆φ} = ∆φ ∆φ where [M ] and M are the branch-fundamental independent loop matrices (also known as incidence matrices) of the electric and magnetic equivalent circuit representations, respectively. The unknowns of the system to be solved are the mesh currents IM and mesh flux density ϕ :

{I} = [M ]{I } with {I} = {I } {ϕ} = M {ϕ } with {ϕ} = ϕ ϕ
The final system to be solved is:

[M ][Z ][M ] [M ] Z [M ] M Z [M ] M Z [M ] I ϕ = U U (9)
with

[Z ] = [R + jωL ] Z = [jωC 0] Z = C 0 Z = Y 0 0 Q {U } = [M ] S 0 U = [M ] T 0
In the case of conducting regions only, the magnetic unknowns are not needed and the system to be solved becomes:

[M ][Z ][M ]{I } = {U } E.

Taking into account PEEC cable and circuit coupled coil

In this section, we present the extension of the proposed formulation in order to consider PEEC cable elements and/or coils with circuit coupling (region Ωc). Each coil and PEEC cable element are characterized by a vector j0j such as 𝐉 = 𝐣 𝟎𝐣 I 𝐜𝐣 .

The previous system of equations can then be easily adapted by considering the currents {I } flowing in these regions. By adapting the system of equations ( 9), we get:

{ϕ} = ϕ ϕ {I} = I I [Z ] = R + jωL jωL jωL R + jωL Z = jωC 0 jωC 0 Z = C C 0 0 Z = Y 0 0 Q with R = 𝐣 𝟎𝐢 𝐣 𝟎𝐢 σ Ω dΩ L = μ 4π 𝐣 𝟎𝐢 𝐣 𝟎𝐣 r Ω Ω dΓ dΓ L = μ 4π 𝐰 𝐢 𝐣 𝟎𝐣 r Ω dΓ dΩ L = μ 4π 𝐣 𝟎𝐢 𝐰 r Ω dΓ dΩ C = 1 4π 𝐣 𝟎𝐢 μ -1 μ 𝐰 𝐣 ∧ 𝐠𝐫𝐚𝐝 1 r dΩ dΓ C = 1 4π 𝐰 𝐢 𝐣 𝟎𝐣 ∧ 𝐠𝐫𝐚𝐝 1 r dΓ dΩ
Finally, external circuit elements composed of passive components, current or voltage sources can complete the system (9), through electric nodal connections.

F. Considerations on implementation

In order to get a good accuracy, the integrations of Green's kernels are computed by using the analytical integration techniques proposed in [START_REF] Fabbri | Magnetic flux density and vector potential of uniform polyhedral sources[END_REF] for self or very close interactions. Moreover, an adaptive mesh integration is used to compute CMP-372 cable PEEC interaction terms. Let us notice that non-simply connected electric domains are naturally treated thanks to the incidence matrix M . In practice, the circuit solver uses a specific algorithm adapted from [START_REF] Horton | A Polynomial-Time Algorithm to Find the Shortest Cycle Basis of a Graph[END_REF] with the advantage of favoring small loops. The circuit solver uses a GMRES algorithm combined with an LU decomposition-based preconditioner for sparse matrices R and Y.

III. RESULTS

We first present an academic problem constituted of a magnetic and conductive thin sphere (diameter 1m, thickness 1mm, µr = 1000, σ = 5•10 5 S/m) placed in a field produced by a cylinder coil, fed by current or voltage source thanks to the coil circuit coupling. The results are compared with a reference solution obtained with an axi-symmetrical FEM using a mesh ensuring the accuracy of the solution at any frequency. Fig. 3 compares eddy current losses obtained with both simulations. In the case of the unstructured-PEEC method, losses are computed by using the equivalent material properties [START_REF] Krähenbühl | Thin layers in electrical engineering-example of shell models in analysing eddy-currents by boundary and finite element methods[END_REF]: Results obtained are very satisfactory (difference lower than 1% at any frequencies) if we consider the difficulty of accurately calculating eddy current losses.

Losses = e Real 𝐉 𝐦 𝐉 𝐦 Gσ -jω 𝐁 𝐦 𝐁 𝐦 Gμ dΓ (10)
We also test the formulation on a more complex example dealing with the computation of eddy currents and magnetic flux density in a car body. The magnetic source field is produced by a set of cable fed by a voltage or current source. The surface mesh of the car contains about 18.000 triangles and a HCA matrix compression technique (Hybrid Cross Approximation) still associated to the GMRES linear solver has been used for the solving.

Figure 4 shows the electric loops around holes whereas Fig. 5 shows current density and tangential magnetic field on the car body.

IV. CONCLUSION

The new formulation enables the modeling of thin conductive and/or magnetic regions in order to simulate efficiently various devices with only a surface mesh. The computational effort is considerably reduced in comparison with volume approaches. Equivalent material properties enable the computation of eddy currents losses with a good accuracy at any frequency. Let us notice that the approach is limited to harmonic problems, an interesting extension could be to consider transient ones.

Fig. 2 .

 2 Fig. 2. Example of electric (left) and magnetic (right) equivalent circuits for a plate meshed with quadrangles

Fig. 3 :

 3 Fig. 3: Losses (W) in a thin sphere. Comparison between unstructured-PEEC VIM and FEM methods for various ratio e/𝛿. The two loss contributions coming from Bm and Jm solutions (10) are displayed.

Fig. 4 :Fig. 5 :

 45 Fig. 4: Mesh of car body and loops found by the circuit solver to take into account multiply connected electric problems


Florent Chiaroni
email: florent.chiaroni@vedecom.fr

Mohamed-Cherif Rahal

Nicolas Hueber
email: nicolas.hueber@isl.eu

Frédéric Dufaux
email: frederic.dufaux@centralesupelec.fr

Self-supervised learning for autonomous vehicles perception: A conciliation between analytical and learning methods

Keywords: Autonomous vehicle perception, self-supervised learning, semi-supervised learning, scene understanding

T HE interest for autonomous driving has continuously increased during the last two decades. However, to be adopted, such critical systems need to be safe. Concerning the perception of the ego-vehicle environment, the literature has investigated two different types of methods. On the one hand, traditional analytical methods generally rely on hand-crafted designs and features. On the other hand, learning methods aim at designing their own appropriate representation of the observed scene.

. Those methods have the advantage to be easily explainable. However, it is difficult to apply them on high dimensional data for semantic scene analysis. For example, identifying the other road users or understanding the large variety of situations present in an urban scene requires to extract complex patterns from high dimensional data captured by camera sensors.

.

In addition to the difficulty of obtaining a large labeled training dataset, another challenge of learning methods is to prevent unpredictable events. Indeed, some scenes unseen during the training can appear frequently in the context of the autonomous vehicle. For instance, an

accident on the road can change drastically the appearance and the location of potential obstacles. Thus, even if it is possible to predict when the model does not know what it observes, it may be interesting to confirm it through an analytical process and to adapt the learning model to this novel situation.

It turns out that self-supervised learning methods (SSL), consisting of combining analytical and learning techniques, have shown in the literature the ability to address such issues. For instance, the SSL system in [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF] won the 2005 DARPA Grand Challenge thanks to its adaptability to changing environments. SSL for autonomous driving vehicles perception is most often based on learning from data which is automatically labeled by an upstream method, similarly to feature learning in [START_REF] Jing | Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey[END_REF]. In this paper, we address the following aspects of SSL:

• abilities such as sequential environment adaptation on the application time, referred to as online learning, self-supervised evaluation, unnecessity of hand-labeled data, fostering of multimodal techniques [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF], and self-improvement. For example, iterative learning reduces progressively the corrupted predictions [START_REF] Zhong | Self-supervised learning for stereo matching with self-improving ability[END_REF]; • tasks made possible thanks to those advantages, such as depth map estimation [START_REF] Garg | Unsupervised cnn for single view depth estimation: Geometry to the rescue[END_REF], [START_REF] Zhong | Self-supervised learning for stereo matching with self-improving ability[END_REF],

temporal predictions [START_REF] Dequaire | Deep tracking in the wild: End-to-end tracking using recurrent neural networks[END_REF], moving obstacles analysis [START_REF] Bewley | Online self-supervised multi-instance segmentation of dynamic objects[END_REF], and long-range vision [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF], [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF].

For example, the SSL system in [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF] learns to extrapolate the appearance of obstacles and traversable areas observable by stereo-vision in a short-range, to identify them at a longer distance beyond the detection range of the stereo-vision.

(a) (b) (c) Fig. 1. Some self-driving cars. (a) is the self-driving car Stanley that won the DARPA Grand Challenge using a SSL system equipped with a calibrated monocular camera and a LIDAR sensor [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF]. (b) is the autonomous mobile robot LAGR. It integrates another SSL vision approach [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF] able to identify online the obstacles and road segmentation from a short-range stereovision up to a long-range monocular vision. (c) is the car equipped with the perception sensors used to generate the KITTI dataset [START_REF] Geiger | Are we ready for autonomous driving? the kitti vision benchmark suite[END_REF].

While the cited SSL techniques are respectively designed for a specific use case application, they present some similarities. In particular, a shared underlying idea is to: Learn to predict, from a given spatio-temporal information (e.g. a single camera frame [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF], [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF], [START_REF] Guizilini | Online self-supervised segmentation of dynamic objects[END_REF], [START_REF] Garg | Unsupervised cnn for single view depth estimation: Geometry to the rescue[END_REF], [START_REF] Pathak | Learning features by watching objects move[END_REF]), something (e.g. traversable area segmentation [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF], [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF], depth estimation [START_REF] Garg | Unsupervised cnn for single view depth estimation: Geometry to the rescue[END_REF], or moving obstacles segmentation [START_REF] Guizilini | Online self-supervised segmentation of dynamic objects[END_REF], [START_REF] Pathak | Learning features by watching objects move[END_REF]) that can be automatically labeled in another way using additional spatio-temporal information (e.g. stereo-vision camera [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF], [START_REF] Garg | Unsupervised cnn for single view depth estimation: Geometry to the rescue[END_REF], a temporal sequence [START_REF] Ondruska | Deep tracking: Seeing beyond seeing using recurrent neural networks[END_REF], or depth sensor [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF]).

We propose to highlight those interdependencies hereafter. In this way, we aim at providing to the reader some analytical, learning and hybrid tools which are transversal to the final application use cases. In addition, the limitations of the presented frameworks are discussed and highlighted in Table I, as well as the perspectives of improvement for self-evaluation, self-improvement, and self-adaptation, in order to address future autonomous driving challenges.

The outline of this article is as follows. After this introduction, we present in Sec. II and III some analytical and learning perception tools relevant to SSL. We follow in Sec. IV with the presentation of existing SSL techniques for some autonomous driving perception applications. Finally, we conclude with a discussion focusing on limitations and future challenges in Sec. V.

II. ANALYTICAL METHODS

Before the recent growing interest for deep learning methods, many analytical methods (without learning) have been proposed, bringing baseline reference tools for multiple challenging perception tasks in the context of autonomous driving. Some of the most investigated tasks considered in this article are briefly introduced hereafter:

• Keypoints feature detection: Before analyzing the sensor data from a relatively high level, analytical techniques often require to perform spatial or temporal data matching using feature detection methods. More specifically, these methods consist in detecting and extracting local features in the sensor data. These hand-crafted features can be small regions of interest [START_REF] Harris | A combined corner and edge detector[END_REF].

In order to enable the matching of sensor data, captured from the same scene with different spatial or temporal points of view, such features need to be as invariant as possible to scale, translation, and rotation transformations. The most common sensor data is an image captured by a camera. In this case, competitive feature detectors include SIFT [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF], SURF [START_REF] Bay | Surf: Speeded up robust features[END_REF], and ORB [START_REF] Rublee | Orb: An efficient alternative to sift or surf[END_REF]. When a depth sensor is also available, the depth information can be exploited in order to further improve feature detection. For instance, the TRISK method [START_REF] Karpushin | Keypoint detection in rgbd images based on an anisotropic scale space[END_REF] is specifically designed for RGB-D images. More recently, LIDAR has enabled the acquisition of point clouds. To tackle this new form of sensor data, some feature detection techniques are derived from image ones (e.g. Harris and SIFT). Alternatively, some new approaches such as ISS [START_REF] Zhong | Intrinsic shape signatures: A shape descriptor for 3d object recognition[END_REF] are exclusively designed for point clouds. From a practical point of view, implementations of common image feature detectors can be found in image libraries as OpenCV1 , and in point clouds libraries as PCL 2 . Feature detectors are exploited by several autonomous driving perception techniques requiring matching of sensor data, including optical flow, disparity map, visual odometry, SLAM and tracking techniques. • Optical flow is a dense [START_REF] Farnebck | Two-frame motion estimation based on polynomial expansion[END_REF] or sparse [START_REF] Lucas | An iterative image registration technique with an application to stereo vision[END_REF] motion pattern. It can be obtained by computing points or features transformations throughout a temporal images sequence captured from a static or mobile ego-camera point of view. In the context of autonomous driving perception, optical flow is relevant for background subtraction, motion estimation of the ego-vehicle and surrounding moving obstacles as proposed by Menze et al. [START_REF] Menze | Object scene flow for autonomous vehicles[END_REF]. It can also be exploited, in the case of a monocular mobile camera without any additional information, for relative depth map estimation [START_REF] Prazdny | Egomotion and relative depth map from optical flow[END_REF] of the surrounding static environment. • Depth map estimation aims at providing image pixels depths, namely the relative or absolute distance between the camera and the captured objects. Several techniques exist to address this task. One of the most common and effective approaches is to compute a disparity map from a stereo-camera. Combined with the extrinsic cameras parameters, such as the baseline distance separating both cameras, the disparity map can be converted into an inversely proportional absolute depth map. Another approach is to project LIDAR points on some of the camera image pixels. It also requires extrinsic spatial and temporal calibrations between both sensors. As mentioned previously, a relative depth map of a static scene can also be derived from the optical flow obtained with a moving camera. Under some assumptions, for example with additional accurate GPS and IMU sensors information concerning the absolute pose transformations of the moving camera, the absolute depth map can then be obtained. The depth map can also be directly obtained with some RGB-D sensors. Depth map is interesting for identifying the 3D shape of objects in the scene. More specifically, in autonomous driving, an absolute depth map is relevant for estimating the distance between the ego-vehicle and detected obstacles. However, we should note that absolute depth map estimation is more challenging compared to relative depth map, as at least two jointly calibrated sensors are necessary. Consequently, it implies a relative higher financial cost in production. Moreover, extrinsic calibrations can be sensitive to the ego-vehicle physical shocks. Finally such sensor fusions can only offer depth estimation in a limited range, due to fixed baselines with stereo cameras, or sparse point cloud projections with dispersive LIDAR sensors. Nevertheless, relative depth map is sometimes sufficient to detect obstacles and traversable areas. For example, considering the traversable area as a set of planes in the depth map 3D point cloud projection, some template matching techniques can be used [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF]. • Geometric shape detection techniques such as Hough transform and RANSAC [START_REF] Fischler | Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[END_REF] initially aimed at identifying some basic geometric shapes such as lines for lane marking detection, ellipses for traffic lights detection, or planes for road segmentation. In order to deal with sophisticated template matching tasks, techniques such as the hough transform have been generalized for arbitrary shape detection [START_REF] Ballard | Generalizing the hough transform to detect arbitrary shapes[END_REF]. Nonetheless, these techniques require an exact model definition of the shapes to be detected. Consequently, they are sensitive to noisy data and are impractical for detection of complex and varying shapes such as obstacles encountered in the context of autonomous driving. Indeed, such objects typically suffer from outdoor illumination changes, background clutter, or non-rigid transformations. • Motion tracking aims at following some data points, features or objects through time.

Tracking filters, such as the Extended Kalman Filter (EKF), predict the next motion using the prior motion knowledge. Conversely, objects tracking can be achieved by features or template matching between consecutive video frames. Pixel points and features tracking is relevant for dense or sparse optical flow, as well as visual odometry estimation [START_REF] Scaramuzza | Visual odometry [tutorial[END_REF]. Conversely, obstacle objects tracking is very important in autonomous driving for modeling or anticipating their trajectories into the ego-vehicle environment. However, on the whole, while some techniques integrate uncertainty, they remain limited when dealing with complex real motion patterns. Pedestrians and drivers behaviour prediction typically requires knowledges about the context. Moreover, mobile obstacles appearence can drastically change depending on their orientation. • SLAM techniques: The complementarity between the above enumerated concepts has been demonstrated through the problem of simultaneously localizing the ego-vehicle and mapping the surrounding environment (SLAM) [START_REF] Bresson | Simultaneous localization and mapping: A survey of current trends in autonomous driving[END_REF]. Features matching provides the pose transformations of the moving ego-vehicle. In turn, 3D scaled projections of depth maps combined with the successive estimated poses provide the environment mapping. Tracking filters and template matching may offer some robustness against sensor data noise and drifting localization estimation, as respectively proposed in EKF SLAM [START_REF] Davison | Monoslam: Real-time single camera slam[END_REF] and SLAM++ [START_REF] Salas-Moreno | Slam++: Simultaneous localisation and mapping at the level of objects[END_REF] approaches. To summarize, analytical methods can successfully deal with several perception tasks of significant interest in the context of autonomous driving. In particular, a self-driving vehicle embedding these techniques is able to carry out physical analysis such as the 3D reconstruction modelling of the environment, and dynamic estimations concerning the ego-vehicle and the encountered surrounding mobile obstacles. Moreover, these techniques have the advantage to be easily explainable in terms of design. This facilitates the identification and prevention of failure modes. However, some critical limitations persist nowadays:

• A lack of landmarks and salient features combined with the presence of dynamic obstacles may entail a severe degradation of the feature detection and matching. • Severe noisy sensor data induces the same risks.

• It is impossible to achieve dense real-time semantic scene analysis of environments including a wide range of complex shape patterns. Learning methods, by recognizing and predicting complex patterns with generalization abilities, aim at overcoming such issues, as developed in the next section.

III. LEARNING METHODS

Learning methods have demonstrated state-of-the-art prediction performances for semantic analysis tasks during the last decade. Autonomous driving is a key application which can greatly benefit from these recent developments. For instance, learning methods have been investigated in this context, for identifying the observed scene context using classification, for detecting the other road users surrounding the ego-vehicle, for delineating the traversable area surface, or for dynamic obstacles tracking.

• Classification aims at predicting, for a given input sensor sample, an output class label.

In order to deal with high dimensional data containing complex patterns, the first stage is generally to extract relevant features using hand-crafted filters or learned feature extractors.

For image feature extraction, the state-of-the-art techniques use Convolutional Neural Network (CNN) architectures. They are composed of a superposition of consecutive layers of trainable convolutional filters. Then, a second stage is to apply a learning classifier on the feature maps generated as output of these filters. Some commonly used classifiers are the Support Vector Machine (SVM) and the Multi-Layer Perceptron (MLP). Both require a training which is most of the time performed in a fully supervised way on labeled data. The CNN and MLP deep learning models are trained by backpropagating the output prediction error on the trainable weigths up to the input. Concerning the evaluation of these models, a test dataset is required, which is labeled as well. The Accuracy metric is commonly used for evaluating the prediction performances, while the F1-Score, an harmonic mean of the precision and recall, is relevant for information retrieval. An image classification application example in autonomous driving is for categorizing the context of the driven road [START_REF] Teichmann | MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving[END_REF]. • Detection generally localizes the regions of interest in a visual sensor data, which in turn can be classified. A commonly used strategy, invariant to scales and translations, applies an image classifier on sliding windows over an image pyramid. Then, several advanced competitive image detection techniques, such as Faster R-CNN [START_REF] Ren | Faster R-CNN: Towards real-time object detection with region proposal networks[END_REF] or Yolo [START_REF] Redmon | You only look once: Unified, real-time object detection[END_REF] have been more recently developed, and have been adapted for road users detection [START_REF] Teichmann | MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving[END_REF]. • Segmentation: As its name suggests, this task provides a segmentation of visual sensor data.

Three distinct applications can be considered:

-Semantic segmentation assigns a semantic class label to each pixel. An example is road segmentation [START_REF] Teichmann | MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving[END_REF]. State-of-the-art methods for autonomous vehicle perception can exploit an auto-encoder architecture, but also dilated or atrous convolutions, as well as an image context modeling strategy as reviewed in [START_REF] Garcia-Garcia | A survey on deep learning techniques for image and video semantic segmentation[END_REF]. In the context of image segmentation, these models are trained to predict as output a pixel-wise classification of the input image.

-Instance segmentation aims at detecting and segmenting each object instance. Examples include foreground segmentation and object detection of potentially moving obstacles [START_REF] He | Mask r-cnn[END_REF]. -Panoptic segmentation [START_REF] Kirillov | Panoptic segmentation[END_REF] is a unification of the two previously mentioned segmentation tasks. Some models dealing with these segmentation tasks have been adapted for performing per-pixel regression tasks such as dense optical flow estimation [START_REF] Dosovitskiy | Flownet: Learning optical flow with convolutional networks[END_REF] or depth map estimation [START_REF] Liu | Learning depth from single monocular images using deep convolutional neural fields[END_REF].

• Temporal object tracking follows the spatial location of selected objects along a temporal data sequence. State-of-the-art learning techniques use variants of the Recurrent Neural Network (RNN) model [START_REF] Milan | Online multi-target tracking using recurrent neural networks[END_REF]. Compared to standard filtering techniques, RNNs have the ability to learn complex and relatively long-term temporal patterns in the context of autonomous driving. These methods can be combined in a unified framework, for instance by sharing the same encoded latent feature maps, as proposed in MultiNet [START_REF] Teichmann | MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving[END_REF] for joint real-time scene classification, vehicle detection and road segmentation. While demonstrating competitive prediction performances, the above mentioned learning techniques are fully supervised. In other words, they have in common the limitation to require large-scale fully annotated training datasets. In order to alleviate this issue, some other learning strategies have been investigated:

• Weakly supervised learning: These techniques can be trained with a partially labeled dataset [START_REF] Niu | Theoretical comparisons of positive-unlabeled learning against positive-negative learning[END_REF], and eventually with a fraction of corrupted labels [START_REF] Ma | Dimensionalitydriven learning with noisy labels[END_REF], [START_REF] Chiaroni | Hallucinating a Cleanly Labeled Augmented Dataset from a Noisy Labeled Dataset Using GANs[END_REF]. Advantageously, these approaches drastically reduce the need of labeled data. • Clustering: These approaches can be defined as an unlabeled classification stategy that aims at gathering without supervision the data depending on their similarities. A huge advantage is that no labels are required. However, if it is necessary to associate the resulting clusters with semantic meanings understandable by human, then a final step of punctual per-cluster hand-labeling is required. State-of-the-art methods [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF] dealing with complex real images mix trainable feature extractors with standard clustering methods such as a Gaussian Mixture Model (GMM) [START_REF] Moon | The expectation-maximization algorithm[END_REF]. • Pre-training: Some relevant generic visual feature extractors can be obtained by performing a preliminary pre-training of the CNN model on unlabeled or labeled data coming from the target application domain [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF] or even from a different one [START_REF] Godard | Digging into self-supervised monocular depth estimation[END_REF]. We also note that in order to apprehend from end-to-end the learned representations, it is possible to identify training overfitting [START_REF] Houle | Local intrinsic dimensionality I: an extreme-value-theoretic foundation for similarity applications[END_REF] of deep learning models without validation test supervision. Furthermore, some learning approaches can estimate the prior of a noisy labeled training dataset [START_REF] Jain | Estimating the class prior and posterior from noisy positives and unlabeled data[END_REF] or the model uncertainty [START_REF]Uncertainty in Deep Learning[END_REF], [START_REF] Kendall | Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding[END_REF]. Now that some considered analytical and learning methods have been treated separately, the next section shows the complementarity between these two different types of approaches through several Self-Supervised Learning (SSL) systems developed in the context the perception of the autonomous driving vehicle.

IV. SSL AUTONOMOUS DRIVING APPLICATIONS

In the context of autonomous driving applications, we can organize the Self-Supervised Learning (SSL) perception techniques in two main categories:

• High-level scene understanding:

road segmentation in order to discriminate the traversable path from obstacles to be avoided dynamic obstacles detection and segmentation obstacles tracking and motion anticipation predictions

• Low-level sensor data analysis, with a particular focus on: Dense depth map estimation, which is a potentially relevant input data information for dealing with the previously enumerated scene understanding challenges.

A. Scene understanding

In order to navigate safely, smoothly, or swiftly when it is required, a self-driving car must perform a path planning adapted to the surrounding environment. The planned trajectories must pass trough traversable areas, while ensuring that surrounding static and dynamic obstacles are avoided. For this purpose, it is necessary to detect and delineate these potential obstacles in advance, but also to anticipate future positions of the mobile ones.

1) Traversable area segmentation: A traversable area can be identified by performing its segmentation over the mapped physical environment. Two different strategies have been successively applied. The former is mainly dedicated to offroad unknown terrain crossing. It entails fully self-supervised training systems (i.e. without hand-labeled data). The latter, that appeared more recently, is dedicated to urban road analysis. The main difference is that the SSL online systems are initialized with a supervised pre-training on hand-labeled data. This preliminary step aims at replacing the lack of landmarks on urban asphalt roads having uniform textures, by prior knowledge.

SSL offroad systems: a road segmentation is proposed in [START_REF] Lieb | Adaptive Road Following using Self-Supervised Learning and Reverse Optical Flow[END_REF] by exploiting temporal past information concerning the road appearence on monocular camera images. It considers the close observable area on the current monocular camera frame in front of the car as a traversable road. Next, it propagates optical flow on this area from the current frame up to the past captured frames. Then, it can deduce this close area appearence when it was spatially farther in the past. This past appearence of the actual close traversable area is exploited for producing horizontal line templates using the SSD (sum of squared differences) matching measure. It is combined with a hough transform-based horizon detector to define the image horizontal lines of pixels on which to apply the horizontal 1-D template matching. Next, with the assumption that the actual distant traversable area has roughly the same appearence as the actual close area had in the past, the 1D templates are applied over the current frame to segment the distant traversable area. If the best template matching measure changes abruptly, then it is supposed that the ego-vehicle is going out of the road or that the road appearence has suddenly and drastically changed. The approach in [START_REF] Lieb | Adaptive Road Following using Self-Supervised Learning and Reverse Optical Flow[END_REF] is relevant for providing a long-range road image segmentation using a monocular camera only. However, a major issue is the critical assumption considering the close area as always traversable. If the road aspect changes suddenly, then it is impossible with this SSL strategy to correctly segment this novel road region.

Another SSL road segmentation approach is proposed in [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF] dealing with this issue. Instead of using temporal information with the assumption that the close area is always traversable, and in addition to the monocular camera, a LIDAR sensor is used for detecting the obstacles close to the ego-vehicle. Projected on the camera images, LIDAR depth points enable to automatically and sparsely labelize the close traversable area on images pixels. Then, a learning gaussian mixture model (GMM) is trained online to recognize the statistical appearence of these sparse analytically labeled pixels. Next, the trained model is applied on the camera pixels which cannot benefit from the sparse LIDAR points projection, in order to classify them as road pixels or not. In this way, the vehicle can anticipate the far obstacles observable in the monocular camera images, but not in the dispersive LIDAR data. This SSL system enabled the Stanley self-driving car, presented in Figure 1(a), to win the DARPA Grand Challenge3 by smoothing the trajectories and increasing the vehicle speed thanks to the anticipation of distant obstacles. This highlighted the interest of combining multiple sensors in a self-driving car.

More recently, with the growing interest for deep learning methods, Hadsell et al. [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF] propose to use a CNN classifier model instead of the earlier template matching or GMM learning techniques. Moreover, an additional paired camera (i.e. stereo-camera) replaces the LIDAR sensor as in [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF]. As offroad terrain traversable areas are not always completely flat, a multi-ground plane segmentation is performed in [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF], on the short-range point cloud projection, obtained with the stereo-vision depth map, by using a hough transform plane detector. This technique provides several automatic labels for image patches which are observable in the short-range region. Then, addressing the long-range vision segmentation, the authors firstly train a classifier to predict patches labels automatically estimated within the short-range region. Next, the trained classifier predicts the same labels on the long-range observable image region patches by using a sliding window classification strategy. Concerning the prediction performances, the authors have demonstrated that the online fine tuning of the classifier and the offline pre-taining of its convolutional layers using an unsupervised autoencoder architecture can improve prediction performances. Moreover, an interesting point to note is that instead of using uncertainty or noisy labeled learning techniques, the authors created transition class labels for the boundary image surfaces separating the obstacles from the traversable area. Finally, from an initial 11-12 meters short range stereo-vision, the developed SSL system is able to extrapolate a long-range vision up to 50-100 meters. Nonetheless, in order to estimate the short-range stereo 3D reconstruction, including planar sets of points corresponding to the offroad traversable area, this approach requires the presence of salient visual features in the road regions. This may be impractical for instance on the uniform visual texture of asphalt roads commonly encountered in urban scenarios, as illustrated in Fig. 2. Pre-trained SSL urban road systems: Some other online SSL techniques deal with this issue by exploiting a classifer pre-trained offline on hand-labeled data [START_REF] Zhou | Road detection using support vector machine based on online learning and evaluation[END_REF], [START_REF] Roncancio | Traversability analysis using terrain mapping and onlinetrained terrain type classifier[END_REF].

The automatic labeling step previously performed with analytical methods is replaced in [START_REF] Zhou | Road detection using support vector machine based on online learning and evaluation[END_REF] by an SVM classifier pre-trained offline using a human annotated dataset. In this way, this approach is also compatible with uniform asphalt road surfaces. However, compared to the previously presented SSL offroad approaches, it requires hand-labeled data.

A hybrid path segmentation technique is proposed in [START_REF] Roncancio | Traversability analysis using terrain mapping and onlinetrained terrain type classifier[END_REF]. It combines a 3D traversability cost map obtained by stereo-vision, and an SVM classifier pre-trained offline over a human annotated dataset. Six different ground surfaces are considered to train the classifier: asphalt, big gravel, small gravel, soil, grass, bushes and stones. The strategy is as follows. SVM predictions refine online the cost map concerning the flat regions. In turn, the 3D traversability cost map obtained without supervision is exploited to update online some mis-classifications of the pre-trained classifier.

To sum up these road segmentation SSL approaches, we can notice that while the sensor data and the analytical and learning models are different, the online process remains essentially the same. The first stage always consists in generating automatic labels by using additional temporal [START_REF] Lieb | Adaptive Road Following using Self-Supervised Learning and Reverse Optical Flow[END_REF], sensor [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF], [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF], or prior knowledge information [START_REF] Zhou | Road detection using support vector machine based on online learning and evaluation[END_REF], [START_REF] Roncancio | Traversability analysis using terrain mapping and onlinetrained terrain type classifier[END_REF]. Then, a second stage trains or updates online a classifier, such that it can be used to provide a long-range or refine segmentation. Overall, while the short-range visions based on depth sensors aims at ensuring the reliable detection of close obstacles, using such SSL vision techniques in static environments directly enables to anticipate the path planning evolution at a long range. Consequently, it is possible to increase the maximum speed velocity of the self-driving car [START_REF] Dahlkamp | Self-supervised monocular road detection in desert terrain[END_REF], while preserving smooth trajectories [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF].

Now that we have presented some SSL techniques dealing with limited depth sensors in static environments, we focus on dynamic obstacles, as they represent the other potential road users interacting with the ego-vehicle in the shared surrounding environment.

2) Dynamic obstacles analysis: This section starts by presenting an SSL approach [START_REF] Guizilini | Online self-supervised segmentation of dynamic objects[END_REF] based on a binary per-pixel segmentation of dynamic obstacles. Then, we introduce its extension [START_REF] Bewley | Online self-supervised multi-instance segmentation of dynamic objects[END_REF] for dynamic obstacles instance segmentation, such that the different road users can be separated.

SSL for dynamic obstacles pixel-wise segmentation: a pixel-level binary segmentation of dynamic obstacles is proposed in [START_REF] Guizilini | Online self-supervised segmentation of dynamic objects[END_REF], using temporal image sequences captured with a monocular camera installed on a mobile urban vehicle. The approach firstly separates sparse dynamic keypoints features from the static ones, by applying a RANSAC technique over the optical flow between consecutive frames. Then, the automatically produced per-pixel dynamic labels are transferred as input of a learning Gaussian Process (GP) model. Next, the trained model extrapolates this knowledge to label as dynamic the pixels of the same visual properties instead of the ones previously automatically identified as dynamic. The whole process is achieved during an online procedure. The system is evaluated on a hand-labeled dataset. This SSL strategy has the advantage to provide the background subtraction from a moving camera, while extrapolating a dense per-pixel segmentation of the dynamic obstacles from sparse detected keypoints. However, this technique cannot provide per-obstacles analysis as it merely predicts a binary mask of pixels corresponding to dynamic obstacles.

The technique in [START_REF] Bewley | Online self-supervised multi-instance segmentation of dynamic objects[END_REF] extends the previous approach for SSL multi-instance segmentation by using temporal image sequences captured with a monocular camera installed on a mobile urban vehicle. The authors apply, over the mobile keypoints detected by [START_REF] Guizilini | Online self-supervised segmentation of dynamic objects[END_REF], a clustering method using the tracked keypoints information such as their spatial location and motion pattern features. The multi-instance segmentation of dynamic obstacles is evaluated on a hand-labeled video sequence of the KITTI dataset [START_REF] Geiger | Are we ready for autonomous driving? the kitti vision benchmark suite[END_REF].

Overall, the authors state that some issues shared with analytical methods persist in their approach. If the dynamic obstacles shadows are projected on the background, then the latter are considered as dynamic as well. Moreover, the segmentation of distant dynamic obstacles can be missed if the corresponding keypoints variations are considered as noise due to the difficulty to detect the corresponding slight optical flow variations. Furthermore, if a dynamic obstacle, either large or close to the sensor, represents the majority of the image keypoints, then this given obstacle is likely to be treated as the static background scene.

Nonetheless, it is important to bear in mind that these approaches present state-of-the-art competitive performances for dynamic obstacles detection and segmentation without training or pre-training on annotated data. In addition, the method in [START_REF] Bewley | Online self-supervised multi-instance segmentation of dynamic objects[END_REF] provides interesting tools to analyze on the move the dynamic obstacles, for example to separately track them and learn to predict their intention.

The next focus is on SSL techniques designed for object tracking and temporal predictions in urban road scene evolution, including dynamic obstacles.

3) Temporal tracking predictions: In order to deal with object appearence changes, a competitive SSL tracking technique [START_REF] Kalal | Tracking-learning-detection[END_REF] proposes an online adaptive strategy combining tracking, learning, and object detector real-time modules. However, in the context of autonomous driving, it may be often necessary to simultaneously track, and even anticipate the trajectories of several surrounding road users. Moreover, being able to consider the interactions between each road user requires some complex motion pattern analysis.

It turns out that some SSL approaches propose to deal with this challenge by focusing the prediction effort on the entire scene in a unified way, rather than on every obstacles independently. The SSL deep tracking system [START_REF] Ondruska | Deep tracking: Seeing beyond seeing using recurrent neural networks[END_REF] 4 learns to predict the future state of a 2D LIDAR occupancy grid. This is achieved by training an RNN on the latent space of a CNN autoencoder (AE) which is applied on the input occupancy grid considered as an image. Each cell of the grid is represented by a pixel, which can be color-coded as occluded, void, or as an obstacle surface. Consequently, the model can be trained from end-to-end by learning to predict the next occupancy grid states using the past and current grid states. Solely the prediction output error of non occluded cells is backpropagated during the training. By definition, this system can perform a self-evaluation by computing a per-pixel photometric error between the predicted occupancy grid and the real future observed occupancy grid at the same temporal instant. This technique has the advantage of being compatible with complex motion patterns compared to Bayesian and Kalman tracking techniques. In addition, the training process enables to predict the obstacles trajectories even during occlusions. The major interest of deep tracking is that, as the model learns to predict a complete scene, it naturally considers interactions between each dynamic obstacle present in the scene. In [START_REF] Dequaire | Deep tracking in the wild: End-to-end tracking using recurrent neural networks[END_REF], the deep tracking model is extended for a real mobile LIDAR sensor by adding a spatial transformer module in order to take into consideration the displacements of the ego-vehicle with respect to its environment during objects tracking.

In turn, these tracking approaches provide the tools to collect motion pattern information of surrounding dynamic obstacles, such that this information may help to classify obstacles depending on their dynamic properties [START_REF] Fathollahi | Autonomous driving challenge: To Infer the property of a dynamic object based on its motion pattern using recurrent neural network[END_REF].

B. Low-level sensor data analysis

This section addresses the sensor data analysis for low-level information estimation in the context of autonomous driving. Compared to the previous methods, the attention has mainly focused recently on SSL depth map estimation from monocular or stereo cameras.

1) SSL Depth map estimation: The self-supervised depth map estimation approach presented in [START_REF] Garg | Unsupervised cnn for single view depth estimation: Geometry to the rescue[END_REF] predicts a depth map from a monocular camera without relying on annotated depth maps. The pose transformation between both left and right cameras is known. The SSL strategy is as follows. First, the left camera frame is provided as input to a CNN model trained from scratch to predict, the corresponding depth map. Second, an inverse warping is performed by combining the predicted left depth map with the right camera frame in order to output a synthesized frame similar to the input left frame. In this way, an SSL photometric reconstruction error can be computed as output of the decoder part. Next, this per-pixel error is directly used to train the encoder weights using Stochastic Gradient Descent (SGD) optimization technique. While not requiring pre-training, nor annotated ground-truth depths, this approach presents prediction performances comparable with the state-of-the-art fully supervised monocular techniques. However, the ground truth pose transformation, related to the inter-view displacement between both cameras, is required.

Following a similar idea, another technique is proposed in [START_REF] Zhong | Self-supervised learning for stereo matching with self-improving ability[END_REF]. It is trained to reconstruct, from a given frame, the second frame taken from a different point of view. It generates a depth map using a stereo camera during the training step, but also during the prediction step. This makes the approach more robust, such that it becomes competitive with standard stereo matching techniques. Moreover, the constraint of preserving two cameras and the pose transformation ground truth for predictions, enables in counterpart to perform online learning. This may be interesting for dealing with novel ego-vehicle environments unseen during the training.

To overcome the necessity of the pose transformation ground-truth, Zhou et al. [START_REF] Zhou | Unsupervised learning of depth and ego-motion from video[END_REF] propose to predict, from a temporal sequence of frames, the depth map with a learning model, and the successive camera pose transformations with another learning model. Both models are trained together from end-to-end for making the novel view synthesis of the next frame. However, the pose transformation estimation implies that the predicted depth map is defined up to a scale factor.

A more modular technique [START_REF] Godard | Digging into self-supervised monocular depth estimation[END_REF] exploits either temporal monocular sequences of frames as in [START_REF] Zhou | Unsupervised learning of depth and ego-motion from video[END_REF], the paired frames of a stereo camera as in [START_REF] Zhong | Self-supervised learning for stereo matching with self-improving ability[END_REF], or to jointly exploit both temporal and stereo information. This framework also deals with the false depth estimation of moving obstacles by ignoring, during training, the pixels not varying between two consecutive temporal frames. It also deals with occluded pixels when the captured point of view changes by using a minimum reprojection loss.

To summarize, low-level analysis techniques for depth map estimation have demonstrated that SSL strategies without using ground truth labels can bring state-of-the-art solutions competitive with fully supervised techniques.

Overall, the SSL techniques presented in this section support the following conclusion: By exploiting the complementarity between analytical and learning methods, it is possible to address several autonomous driving perception tasks, without necessarily requiring an annotated dataset. Presented methodologies are summarized in Fig. 3 along with Table II.

V. LIMITATIONS AND FUTURE CHALLENGES

In the context of autonomous driving, some limitations remain in the presented SSL perception systems and open future research perspectives.

Catastrophic forgetting: During the online learning procedure, the trainable weights of the model may require unnecessary repetitive updates for detecting a given pattern throughout the environment exploration. In fact, when a learning model is continuously specialized for dealing

Fig. 2 .

 2 Fig. 2. Salient features location on urban ego-vehicle environment. (a) is an arbitrary frame, extracted from the KITTI dataset [20], illustrating an urban asphalt road with the surrounding environment. (b) shows keypoints detected on the left input image using SIFT detector. Keypoints distribution is dense in the offroad region, and sparse on the asphalt road in the image center.

Fig. 3 .

 3 Fig. 3. Function diagrams showing the common connections depending on the strategy. Functional blocks represent single monocular camera frame S1, additional sensor data (e.g. temporal frame sequence, stereo-camera, or lidar data) Sn, a Learning model L, an Analytical method A, and Evaluation method E.

TABLE I SUMMARY

 I OF GLOBAL TRENDS FOR ADVANTAGES AND DRAWBACKS OF CURRENT STATE-OF-THE-ART ANALYTICAL, LEARNING AND SSL METHODS FOR AUTONOMOUS VEHICLE PERCEPTION CHALLENGES.

	Methodology	no hand-labeling dense complex pattern analysis online self-evaluation and adaptation knowledge extrapolation low-cost sensor requirements
	Analytical	+++	+	++	+	+
	Supervised learning	+	+++	+	+	+++
	Self-Supervised Learning	+++	++	+++	+++	++

https://opencv.org/

http://pointclouds.org/

https://www.darpa.mil/about-us/timeline/-grand-challenge-for-autonomous-vehicles

Such an approach could be categorized as unsupervised. However, we make the choice in this article to consider that exploiting during the training an additional future temporal information, not availbable during the prediction step, is a type of self-supervision.

with the latest data, the likelihood increases that the model simultaneously forget the potentially relevant formerly learned patterns. It turns out that it is possible to deal with this catastrophic forgetting issue when using neural networks [START_REF] Kirkpatrick | Overcoming catastrophic forgetting in neural networks[END_REF]. For future research directions, it may be interesting to combine such incremental learning techniques with the presented SSL frameworks.

Concerning the scene depth map estimation solely based on temporal analysis:

• the presence of dynamic obstacles in the scene during the learning stage can result in poor estimates of the observed scene. As discussed in [START_REF] Guizilini | Online self-supervised segmentation of dynamic objects[END_REF], further research on SSL for potentially dynamic obstacles delineations on the sensor data may help to deal with this issue. • the current state-of-the-art techniques cannot estimate the real depth map without requiring a supervised scaling factor. The latter is generally obtained by estimating the real metric values of the pose transformation between two consecutive camera viewpoints. As proposed in the supervised detector Deep MANTA [START_REF] Chabot | Deep manta: A coarse-to-fine many-task network for joint 2d and 3d vehicle analysis from monocular image[END_REF], it may be interesting to recover this scale factor by using some template matching techniques on the observable objects of the scene. Concerning the online self-evaluation, some of the presented systems require a baseline reference analytically obtained [START_REF] Hadsell | Learning long-range vision for autonomous off-road driving[END_REF]. However, if we consider that the analytical processes, considered as ground-truth labeling techniques, are likely to generate some noisy labels, it may be interesting to investigate some future reasearch on how to evaluate this prior noise from the learning model point of view [START_REF] Jain | Estimating the class prior and posterior from noisy positives and unlabeled data[END_REF], and how to deal with it [START_REF] Chiaroni | Hallucinating a Cleanly Labeled Augmented Dataset from a Noisy Labeled Dataset Using GANs[END_REF].