Dr. HDR Christian Laugier
email: christian.laugier@inria.fr

Bayesian Perception & Decision-making for Autonomous Vehicles and Mobile Robots

Keywords: Embedded Bayesian Perception  Main challenges  Noisy data, Incompleteness, Dynamicity, Discrete measurements  Multi-Sensors Observations Lidar, Radar, Stereo camera, IMU …  Probabilistic Environment Model

Ownership & Feeling

Solving for each cell

Joint Probability decomposition:

P(C A O O -1 V V -1 Z) = P(A) P(O -1 V -1 | A) P(O V | O -1 V -1 ) P(C | A V) P(Z | O C)
Bayesian Filtering

(Grid update at each time step)

Bayesian Occupancy Filter (BOF) -Main Features

Autonomous

  Cars & Driverless Vehicles Driverless Taxi testing in Pittsburgh (Uber) & Singapore (nuTonomy) => Mobility Service, Numerous Sensors … Engineer in the car during testing Drive Me trials • 100 Test Vehicles in Göteborg, 80 km, 70km/h • No pedestrians & Plenty of separations between lanes Cybus experiment, La Rochelle 2012 (CityMobil Project & Inria) Tesla Autopilot based on Radar & Mobileye Costly 3D Lidar & Dense 3D mapping • Strong involvement of Car Industry & Large media coverage • An expected market of 500 B€ in 2035 • Numerous recent & on-going real-life experiments for validating the technologies Detection & Tracking of Mobile Objects (DATMO)  Objects classification (recognition)  Prediction & Risk Assessment (avoiding future collisions)  Main Models & Algorithms  Bayesian Perception Paradigm  Behaviors modeling & learning  Bayesian approaches for Prediction & Risk Assessment robot motions & Deciding of the most appropriate action to be executed by the robot (Goal & Context & Risk)  Main Difficulty & Functions  On-line Motion Planning under various constraints (time, kinematic, dynamic, uncertainty, collision risk, social)  Decision making under uncertainty using contextual data (history, semantics, prediction)  Main Models & Algorithms  Iterative Risk-based Motion Planning (e.g. Risk-RRT)  Decision making using Contextual data & Bayesian networks the robot for executing Safe & Socially Acceptable robot actions, while taking into account the related Human -Robot Interactions  Main Difficulty & Functions  Robot navigation while taking into account both Safety & Social constraints  Human in the loop  Main Models & Algorithms  Human-Aware Navigation paradigm (safety & social filters)  Intuitive Human-Robot Interaction A new framework: Dynamic Probabilistic Grids => A clear distinction between Static & Dynamic & Free components Occupancy & Velocity Probabilities Velocity field (particles)  Bayesian Occupancy Filter (BOF) => Patented by Inria & Probayes => Commercialized by Probayes => Robust to sensing errors & occultation  Used by: Toyota, Denso, Probayes, Easymile, BA-Systems, IRT Nanoelec / CEA  Free academic license available  Industrial license under negotiation with Toyota, Renault, Easymile [Coué & Laugier IJRR 05] [Laugier et al ITSM 2011] [Laugier, Vasquez, Martinelli Mooc uTOP 2015]

•=>[Approach 1 :Approach 2 :

 12 Estimate Spatial occupancy for each cell of the grid P (O | Z )• Grid update is performed in each cell in parallel (using BOF equations)• Extract Motion Field (using Bayesian filtering & Fused Sensor data)•Reason at the Grid level (i.e. no object segmentation at this reasoning level) Exploiting the Dynamic information for improving Scene Understanding !! • Dynamic part (particles) is "projected" in the grid using motion model => motion prediction • Both Dynamic & Static parts are expressed in the new reference frame => moving vehicle frame • The two resulting representations are confronted to the observations => estimation step • New representations (static & dynamic) are jointly evaluated and particles re-sampled Main steps in the updating process How to compute P(OV | Z C) in practice? HSBOF updating process (outline of the algorithm) • Data association is performed as lately as possible • More robust to Perception errors & Temporary occultation Bayesian Sensor Fusion + Detection & Tracking Fast Clustering and Tracking Algorithm (FCTA) [Mekhnacha 09, Laugier et al ITSM'11] Detected &Tracked Objects Objects classification Stereo-vision (U-disparity OG+ Road/obstacle classif.) Dynamic grid filtering using Motion data (IMU + Odometry) Drastic memory size reduction (factor 100) + Increased efficiency (complex scenes) + More accurate Velocity estimation (using Particles & Motion data from ego-vehicle )  Conditional Monte-Carlo Dense Occupancy Tracker (CMCDOT, 2015) => Increased efficiency using "state data" (Static, Dynamic, Empty, Unknown) + Integration of a "Dense Occupancy Tracker" (Object level, Using particles propagation & ID)  CMCDOT + Ground Estimator (under Patenting, 2017) => Ground shape estimation & Improve obstacle detection (avoid false detections on the ground) [Negre et al 14] [Rummelhard et al 14] [Rummelhard et al 15] Several implementations (models & algorithms) more and more adapted to Embedded constraints & Scene complexity Rummelhard et al 17] HSBOF Experimental Results in dense Urban Environments Ego Vehicle (not visible on the video) -Lecture on Bayesian Perception & Decision-making for Autonomous Vehicles and Mobile Robots Beijing Institute of Technology, Intelligent Vehicle Research Center, May 8 th 2017 51 © Inria & Laugier. All rights reserved Software / Hardware Integration -Main Features Embedded Integration for OG processing  The challenge: How to cope with contradictory requirements & constraints ? o Embedded characteristics: Low computing power, Low memory space, Low bandwidth o Embedded constraints: Low purchase cost, Low energy consumption, Small physical size o Algorithmic constraints: High computing requirement, High memory space & bandwidth requirement without filtering) on μController [1][2]  Low Cost / Energy / Size (widely used in Industrial product)  Implementation based on Integer Arithmetic (Quantized Occupancy Probability)  Time performance: Increased by a factor 5-10  Energy Consumption: Decreased by a factor 100 Software / Hardware Integration -μController Inria /Renault Zoé (4 Lidars @ 25Hz) collision risk -Main features => Grid level & Conservative motion hypotheses (proximity perception) o Detect "Risky Situations" a few seconds ahead (0.5 to 3s) o Risky situations are localized in Space & Time  Conservative Motion Prediction in the grid (Particles & Occupancy)  Collision checking with Car model (Shape & Velocity) for every future time steps (horizon h) o Resulting information can be used for choosing Avoidance Maneuvers Proximity perception: d <100m and t <5s d 0.5 s => Precrash d= 1 s => Collision mitigation d > 1.5s => Warning / Emergency Braking  Main Features  Collision Risk Estimation  Projecting over time the estimated Scene changes (DP-Grid) & Car Model (Shape + Motion)  Evaluate the Collision Risk for every next time step  Integration of risk over a time range [t t+d] • TTC • Collision Probability Generalized Risk Assessment (Object level) => Increasing time horizon & complexity using context & semantics => Key concept: Behaviors Modeling & Prediction Previous observations Highly structured environment + Traffic rules => Prediction more easy Decision-making in complex traffic situations  Understand the current traffic situation & its likely evolution  Evaluate the Risk of future collision by reasoning on traffic participants Behaviors  Takes into account Context & Semantics Context & Semantics History + Space geometry + Traffic rules + Behavior Prediction For all surrounding traffic participants + Probabilistic Risk Assessment © Inria. All rights reserved C. LAUGIER -Lecture on Bayesian Perception & Decision-making for Autonomous Vehicles and Mobile Robots Beijing Institute of Technology, Intelligent Vehicle Research Center, May 8 th 2017 66 © Inria & Laugier. All rights reserved [Tay thesis 09] [Laugier et al 11] Patent Inria & Toyota & Probayes 2010 66 Behavior-based Collision risk (Object level) Trajectory prediction & Collision Risk Assessment All rights reserved C. LAUGIER -Lecture on Bayesian Perception & Decision-making for Autonomous Vehicles and Mobile Robots Beijing Institute of Technology, Intelligent Vehicle Research Center, May 8 th 2017 67 © Inria & Laugier. All rights reserved A Human-like reasoning paradigm => Detect Drivers Errors & Colliding behaviors  Estimating "Drivers Intentions" from Vehicles States Observations (X Y θ S TS) => Perception or V2V  Inferring "Behaviors Expectations" from Drivers Intentions & Traffic rules Risk = Comparing Maneuvers Intention & Expectation => Taking traffic context into account (Topology, Geometry, Priority rules, Vehicles states) => Digital map obtained using "Open Street Map" 67 [Lefevre thesis 13] [Lefevre & Laugier IV'12, Best student paper] Patent Inria & Renault 2012 (risk assessment at road intersection) Patent Inria & Berkeley 2013 (postponing decisions for safer results) Behavior-based Collision risk (Object level) Intention & Expectation comparison => Complex scenarios with interdependent behaviors & human drivers

Decisional & Control Architecture (5) © Inria. All rights reserved

&© Inria. All rights reserved

A more efficient computation approach :

=> Sum over the particles projected in the cell & their related static parts

Previous computation approach (histograms)

New computation approach (particles)

Sum over the neighborhood, with a single velocity per antecedent

P(C A O O -1 V V -1 Z) = P(A) P(O -1 V -1 | A) P(O V | O -1 V -1 ) P(C | A V) P(Z | O C)


Fran De Aquino

Searching for the Gravific Photons

Keywords: Gravity, Gravitation, Electromagnetic Waves, Radiation Pressure

published or not. The documents may come

Introduction

Electromagnetic waves transport energy as well as linear momentum. Then, if this momentum is absorbed by a surface, pressure is exerted on the surface. In a previous paper [1] we shown that this pressure has a negative component (opposite to the direction of propagation of the photons) due to the existence of the negative linear momentum transported by the photons. Then, it was predicted the existence of photons in which the negative component of the momentum is greater than the positive one. These photons were called gravific photons [START_REF] De Aquino | The Bipolar Linear Momentum transported by the Electromagnetic Waves: Origin of the Gravitational Interaction[END_REF] and is expected that they have frequencies greater than Hz 28 10 . In addition, it was shown that the limit between the spectrum of the gravific photons and the gamma ray spectrum is defined by a characteristic frequency, Hz f g 28 10 2 ≈ (See Fig. 1). Here, we show how to produce and to detect gravific photons. We start with the wellknow process of production of neutral pions

1
Gravific Radiation is electromagnetic radiation and cannot be confused with gravitational radiation or gravitational waves, which are ripples on the space-time (predicted by the Einstein Theory).

(Mesons ), by means of the collision of highspeed protons. i.e., 0 π ( )

1 0 π + + → + p p p p
Neutral pions decay with a much shorter lifetime of 8.4×10 -17 seconds, producing 2 gamma rays. ( )

2 0 γ γ π + →
When the velocities of the protons are ultrarelativistic the energy of the neutral pion, , is

given by [ )

0 π E 2] ( ) ( )
Hz f f g 28 10 2 ≈ > 2 .
In the system shown in Fig. 2, when the Gravific Photons strike on a plate, it is exerted a pressure on opposite direction to the direction of propagation of the radiation. By measuring the pressure on the plate it is then possible to confirm the existence of the Gravific Photons. [START_REF] Boldt | Cosmic Rays Photons[END_REF] At the Large Hadron Collider (LHC) the protons each have an energy of 6.5 TeV, giving a total collision energy of 13 TeV. At this energy the protons move at about 0.999999990 c. Then the frequency of the gamma rays produced by the disintegration of the neutral pions can reach about 10 30 Hz. > the resultant momentum transported by the photon is negative. If this momentum is absorbed by a surface, pressure is exerted on the surface, in the opposite direction of propagation of the photon. This special type of photon is denominated of attractive photon or Gravific Photons.

Fig. 1 -

 1 Fig. 1 -The Gravific Photons Spectrum (above Hz f g 28 10 2 ≈

  produced gamma rays will have frequency . Therefore, by means of collision of ultra-relativistic protons it is possible to produce Gravific Photons

Fig. 2 -

 2 Fig. 2 -If the photons have frequencies