Synthèse

Au sein des systèmes avancés d'aide à la conduite (Advanced Driver Assistance Systems -ADAS) pour les systèmes de transport intelligents (Intelligent Transport Systems -ITS), les systèmes de positionnement, ou de localisation, du véhicule jouent un rôle primordial. Le système GPS (Global Positioning System) largement employé ne peut donner seul un résultat précis à cause de facteurs extérieurs comme un environnement contraint ou l'affaiblissement des signaux. Ces erreurs peuvent être en partie corrigées en fusionnant les données GPS avec des informations supplémentaires provenant d'autres capteurs. La multiplication des systèmes d'aide à la conduite disponibles dans les véhicules nécessite de plus en plus de capteurs installés et augmente le volume de données utilisables. Dans ce cadre, nous nous sommes intéressés à la fusion des données provenant de capteurs bas cout pour améliorer le positionnement du véhicule.

Parmi ces sources d'information, en parallèle au GPS, nous avons considérés les caméras disponibles sur les véhicules dans le but de faire de l'odométrie visuelle (Visual Odometry -VO), couplée à une carte de l'environnement. Nous avons étudié les caractéristiques de cette trajectoire reconstituée dans le but d'améliorer la qualité du positionnement latéral et longitudinal du véhicule sur la route, et de détecter les changements de voies possibles.

Après avoir été fusionnée avec les données GPS, cette trajectoire générée est couplée avec la carte de l'environnement provenant d'Open-StreetMap (OSM). L'erreur de positionnement latérale est réduite en utilisant les informations de distribution de voie fournies par OSM. Dans ce but, nous avons proposé une méthode basée sur les facteurs de distribution (RPDF). Tandis que le positionnement longitudinal est optimisé avec une correspondance de courbes entre la trajectoire provenant de l'odométrie visuelle et les routes segmentées décrites dans OSM par un ensemble de trois noeuds connectés.

Pour vérifier la robustesse du système, la méthode a été validée avec des jeux de données KITTI en considérant des données GPS bruitées par des modèles de bruits usuels notamment les bruits blanc, un biais de mesure et une perte de signal GPS. Plusieurs méthodes d'odométrie visuelle ont été utilisées pour comparer l'influence de la méthode sur le niveau d'amélioration du résultat après fusion des données. En utilisant la technique d'appariement des courbes que nous proposons, la précision du positionnement connait une amélioration significative, en particulier pour l'erreur longitudinale. Les performances de localisation sont comparables à celles des techniques SLAM (Simultaneous Localization And Mapping), corrigeant l'erreur d'orientation initiale provenant de l'odométrie visuelle.

Les différents bruits modélisés sur les données GPS n'influent pas sur les performances du résultat de la localisation et la robustesse de la méthode proposée a été éprouvée dans le cas de routes divergentes nécessitant un choix sur la voie suivie par le véhicule.

Declaration of Authorship

I, Dayang Nur Salmi Dharmiza AWANG SALLEH, declare that this thesis titled, "Study of Vehicle Localization Optimization with Visual Odometry Trajectory Tracking" and the work presented in it are my own. I confirm that:

• This work was done wholly or mainly while in candidature for a research degree at this University.

• No part of this thesis has previously been submitted for a degree or any other qualification at this University or any other institution.

• Where I have consulted the published work of others, this is always clearly attributed.

• Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work.

• I have acknowledged all main sources of help.

List of Figures

List of Tables

Dedicated to:

Babah, Mayya, and our new bundle of joy . . .

Chapter 1

Introduction on Vehicle Localization

"When Henry Ford made cheap, reliable cars people said, "Nah, what's wrong with a horse?" That was a huge bet he made, and it worked."

-Elon Musk

As quoted from the co-founder, CEO and product architect of Tesla Incorporation, it proves that with innovation and technology, the quality of life is improved, although it might seem dispensable in the beginning. Indeed, humans -whether consciously or nothave always desired for more comfortable life by putting great effort on technology development. That is the reason why technology development has never stopped emerging and researchers are working with each other in their own respective fields.

Vehicle technology has been developing immensely as it is one of the important necessities in daily life. It is the second most expensive commodity purchased by an adult after residential property. People commute either by public transport or personal vehicle from one point to another and safety is among the main concerns. Besides, with urban development and advanced road network, using the most efficient route in terms of journey time and fuel consumption is of the huge factor to be taken into consideration in our hectic life.

Thus, the need for high safety vehicles with accurate positioning technology and uninterrupted routing service are constantly in high demand.

Background

Recent decade has shown massive development in intelligent vehicles and Advanced Driver Assistance Systems (ADAS) with the birth of autonomous vehicles. These vehicles are capable of 'self-driving' without human input by sensing its environment from sensors. This includes Tesla cars, Google self-driving car (Waymo) and cruise-control systems introduced by high-end car manufacturers like Mercedes and Audi. These vehicles are equipped with sensors for various techniques of sensing their surroundings, such as odometry, GPS, radar, laser sensor, and computer vision. The sensed surroundings are then interpreted into information for vehicle's safety assessment in terms of collision probability, speed limit, and allows the vehicle to localize its position accurately for navigation and routing. To enable the functionality of all these assessments, it relies on the key component in an intelligent vehicle which is its navigation system. Without a correct navigation, it might affect the vehicle routing choice for the driver to arrive at certain destination. Accurate localization is also a requirement for ADAS and Location Based Services (LBS) that recently emerges to provide information or entertainment for users. Localization plays a crucial role in determining the exact position of the vehicle in the global coordinate to liaise with the navigating system and obtain accurate information for LBS.

GPS

Vehicle navigation system is part of the automobile controls (can also be a third-party device) used to find direction from a location to certain destination. It typically uses a satellite navigation device to obtain its position data which is then correlated to a position on a road.

Then, routing can be calculated when the driver needs to know the advised direction of taken route. These days, we are able to obtain our position from the widely available Global

Positioning System (GPS) devices, provided in a handheld device or equipped in the vehicle. Initially, the GPS was designed only for military purposes until in the 1980's, it was made available for public use.

The GPS communicates with satellites to estimate our coordinates on earth. It consists of 24 satellites, that circle the earth twice a day, to provide one's position, at a specific time, with velocity information. Through GPS, we are able to locate positions on the earth based on the distance measurement from the satellites. This allows us to record or find locations coordinates on the earth and with the use of a map, we can navigate from a point to another point. Therefore, GPS devices are also typically used for vehicle navigation purposes with routing services. In order to determine one's two-dimensional position (latitude and longitude), the GPS receiver must be locked with the signal of at least three satellites. With four or more satellites in view, the receiver can calculate position in three-dimensional (latitude, longitude and altitude). GPS satellites transmit at least two low-power radio signals that travel by line of sight, which means they can pass through clouds, glass and plastic. However, the signals will not go through most solid objects, such as buildings and mountains.

However, relying on GPS alone may result in localization failure due to errors in GPS signal. The factors that reduce GPS signal accuracy include:

• Ionosphere and troposphere delays: The ionosphere is the atmosphere layer from 50 to 500 km altitude that consists mostly of ionized air, while the troposphere is the lower layer of the atmosphere (below 13 km) that faces variations in temperature, pressure, and humidity associated according with weather conditions. Satellite signals tend to be refracted as they pass through the earth's atmosphere -causing the signals to slow down or speed up. Since the satellite signals contain correction information for ionospheric interference, the GPS system is able to partially correct the error, and this leaves up to a 10m of horizontal error on the ground.

• Signal multipath interference: This is caused by the reflected signals from surfaces near the GPS receiver that can either interfere with or be mistaken for the true uninterrupted signal from a satellite.

• Receiver clock errors: The receiver's built-in clock may have slight timing error because it is less accurate than the atomic clocks on GPS satellites.

• Orbital errors: Satellite orbit ('satellite ephemeris') pertains to the altitude, position and speed of the satellite and it varies due to gravitational attraction and solar pressure fluctuations.

• Number of visible satellites: GPS accuracy relies on the number of satellites that are in view with the receiver (the more, the better). When a signal is obstructed, users will get position errors or possibly no position reading at all.

• Satellite geometry: Satellite signals are more effective when satellites are located at wide angles in relative to each other, rather than in a line or tight grouping.

• Selective availability (S/A): It is the intentional degradation of the satellite signals by a time varying bias, applied by the U.S. Department of Defense before, which makes signals less accurate for non -U.S. military and government users for security reasons.

The government reduced the amount of signal interference to 0m in May 2000, which improved the accuracy of civilian GPS receivers. However, they retain the ability to reactivate S/A without notice to the GPS users.

To overcome these problems, GPS error compensation techniques have been proposed.

For instance, [START_REF] Bétaille | Applying Standard Digital Map Data in Map-aided, Lane-level GNSS Location[END_REF] presented positioning improvement by 3D geometrical city modelling. From the modelling, it is possible to identify line-of-sight or non-line-ofsight satellite signals, hence allowing correction of pseudo-range measurements. Then, the urban trenches were created automatically from a rover trajectory as a reference to match with the nearest road arc segment. It is an interesting approach of integrating trajectory, map, and GPS data correction. However, the modelling is based on the BD Topo map database for France, thus it is not practical for global use.

Besides, another option is to have a backup system by fusing GPS data with other additional information from sensors or map to obtain one's position correctly. Therefore, it is recommended to explore the available sensors typically used in vehicle that can provide additional information for localization to design a strategy in improving localization accuracy.

Sensors

Sensors are an important component of automotive electronic control systems, defined as 'devices that transform (or transduce) physical quantities such as pressure or acceleration (called measurands) into output signals (usually electrical) that serve as inputs for control systems' [START_REF] Norton | Handbook of transducers[END_REF]. In other words, it is a device that measures certain quantity and converts it into a signal that can be read by an observer, an instrument or a computer. Sensors apply physical properties of matter and laws of physics to perform and they are capable of perceiving properties or environmental attributes. These attributes include light, heat, pressure, sound, electromagnetism or motion. Generally, sensors can be classified into two categories: passive and active. A passive sensor only measures the energy or information obtained from the objects in the environment for observation. Meanwhile, an active sensor emits energy and then observes any change in the reflected energy to estimate the state of the environment.

Today, most vehicles already incorporated many types of sensors such as the Mass Air Flow Sensor (MAF), engine speed sensor, fuel temperature sensor, and voltage sensor. These sensors are mainly built into vehicle's engine to ensure that the driver can identify and prevent possible issues before they result in breakdowns that consumes expensive repairs.

Besides, these engine sensors are also to ensure that the vehicle is operating efficiently.

Aside from engine sensors, vehicles are more and more equipped with either built-in or autonomous sensor for environment perception purposes. This enables the system to obtain information from the surrounding environment, in terms of distance with other objects/vehicles, pedestrian or road signs detection, map building, and localization.

A. Wheel Odometry (ABS)

It is the most widely used method to estimate the position of mobile robots from the number of rotations of the wheels. Braking in vehicles today is commonly assisted with Anti-Lock Braking System (ABS) that uses angular encoders attached to the wheels. With this system, wheel rotation is translated into linear displacement relative to the ground while the encoder as shown in Figure 1.3 is utilized for rotation measurement. Since wheel odometry is a relative positioning technique, it faces positioning accuracy problems due to the drift caused by wheel slippage. This error is accumulated over time for both translational and orientational errors that worsens the positioning performance without proper correction.

B. INS

Inertial navigation system (INS) as shown in Figure 1.4 is a navigation tool that utilizes motion sensors (accelerometers), rotation sensors (gyroscopes) and embedded systems to calculate the position, orientation and velocity of a moving object continuously. It has the [START_REF] Wang | A Bionic Camera-Based Polarization Navigation Sensor[END_REF][START_REF] Woodman | An introduction to inertial navigation[END_REF]. These errors are also increased with time and the position needs to be corrected periodically.

C. Radar

In the beginning of the 20th century, an early implementation of radar technology (an acronym for Radio Detection And Ranging) was meant to detect the presence of ships in fog. Nowadays, radar is still highly valued because it can provide distance and velocity information, even in challenging environmental conditions. Radar can function even in fog, rain, wind, darkness, or blinding sun. As an active sensor, radar uses a transmitter that emits radio waves and a receiver that collects the reflected waves. Radar uses the Doppler Effect to accurately measure the object's velocity by detecting changes in the reflected wavelengths.

Object detection quality highly relies on the wave reflection strength which is influenced by factors such as the distance, the object's size, its absorption characteristics and the reflection angle. In automotive application of radars, challenging scenarios result in problems especially when two objects are located too close next to each other on the road. This is because it would provide a very similar reflections making it difficult to distinguish. Besides, large objects that are close to the transmitter can saturate the receiver and a radar usually cannot resolve the type of the object. Signals received from radar are not intelligent enough to resolve small details of objects which would be useful in object identification. Therefore, without this qualitative information, the application for this sensor is limited to detected object's distance and speed measurement.

There are also characteristic trade-off in radar range versus field of view as indicated in Figure 1.5. In order to identify objects at further distance (for higher-speed traffic), longrange radar systems that are capable of measuring distances exceeding 250m would have a narrow beam width of only about 15 • . As a result, although it allows measurements of objects directly in front of the car up to a long distance, it will not be able to detect vehicles or other objects nearby outside the beam angle. There are mechanical and multi-beam solutions that may increase the systems' field of view, but they require larger sensors, and more complex in computation and systems.

D. Camera

While radar is good at detecting objects physically, the systems are not able to identify objects and visible messages like road signs, road markings, and traffic light colours. Video images captured from cameras on the other hand support the driver with a better representation of the environment outside the vehicle. Today, two-dimensional cameras are widely available to display images while luxury class car makers are beginning to install cameras with virtual, three-dimensional image displays. Besides, many drivers have installed plugged in camera that records video as a proof for safety purposes. Camera also helps to overcome blind-spot problem typically faced by human drivers that contributes to the major traffic accident factors.

Aside from security measures, with computer vision, cameras are also able to automatically detect objects, classify them, and determine the distance from them, provided that the calibration parameters are carefully calculated. For example, from cameras, we are able identify pedestrians and cyclists, motor vehicles, side strips, bridge abutments, and road margins. The algorithms are also used to detect traffic signs and signals as an additional information of the road rules. These information are useful to provide knowledge on the surrounding environment to be integrated in the driver safety system.

With the technology advancement, cameras today can also be utilized for vehicle trajectory generation from visual odometry. This is achieved by the feature detection by image processing, and with vehicle movement, the same features can be matched in the preceding frames. The matched and tracked features contain the information of vehicle ego-motion, which can be translated into trajectory. This allows for enhanced navigational accuracy in vehicles on any surface for localization purposes.

Cameras utilized in vehicles can be divided into monocular and stereo vision. Monocular vision only requires a single camera for image processing while stereo vision uses two cameras, placed horizontally, in the system. With stereo camera, scene depth information can be estimated from the calculation of the object's distance from multiple captures of the same scene from different viewpoints. Although it consumes slightly higher implementation cost compared to monocular vision, the stereo vision is highly advantageous to extract information about the relative position of 3D objects in the vicinity of the system.

Since a full high-definition (HD) image equals to millions of pixels at each frame, use of cameras in ADAS produces massive amounts of data for image processing. This makes processing a computationally intense and algorithmically complex job. Despite of this, cameras are by far the cheapest and the most available sensor in automotive application. Besides, unlike radar, cameras can see colour, making them the best candidate for scene interpretation in intelligent vehicles.

E. Lidar

Lidar, an acronym for Light Detection and Ranging, is a remote sensing method that uses light in the form of a pulsed laser to measure ranges. It is originally developed over 40 years ago to track lunar and satellite distances. However, as the lidar systems become more compact, many applications have been unveiled. Lidar was first utilized on cars in the 2005

Grand DARPA challenge that makes the innovator, Stanley, won the challenge by using 5 lidar sensors mounted on the car roof.

A lidar system consists of four key components: a) a transmitter to emit laser pulses, b) a receiver to intercept pulse echoes, c) an optical analysing system to process input data, and d) a powerful computer to visualize a live, 3D image of the system's surroundings. Lidar systems emit light waves that travel outwards in all directions until making contact with an object, resulting in a reflected light wave that is redirected back to the source. The distance of that object is then calculated based on the time it took for the echo to return, in relation to the known speed of light.

These years, lidar is widely used in self-driving car, owing to its capability to generate huge 3D maps to allow the car or robot navigate within it. By using a lidar to map and navigate an environment, it allows the system to know ahead of time the bounds of a lane, or if there is a stop sign or traffic light a few hundreds of metres ahead. This kind of predictability is what a technology like self-driving cars requires, and has been a big reason for the progress over the last five years.

However, this great performance does not come without a high cost. In order to achieve a 360 • horizontal field-of-view depth map around a car, current LIDAR systems such as that from Velodyne utilize a mechanical, rotating system that rotates the laser in all directions which is neither aesthetically pleasing nor aerodynamic. These 360 • lidar systems are also quite expensive, with prototype systems today costing about $70,000. Besides, lidar faces a challenge with its vertical resolution where the Velodyne's lidar for instance, only has a vertical resolution of 64 pixels. To achieve adequate vertical pixel density, the field-of-view is limited to 26 • , resulting in detection failure of objects directly in front of a vehicle like a curb or pothole.

Harvesting Information from Digital Maps

Aside from the perceptive properties obtained from the sensors equipped to a vehicle, we also have abundant information of the surrounding environment provided by digital maps.

A digital map may contain information such as road connectivity, traffic signs, speed limit, public transport stations, and geographical structure. Data provided by the digital map is growing rapidly with the crowdsourcing and is expected to improve the map's information accuracy. This allows the possibility to harvest the information provided by the digital map as an additional input for vehicle localization by data fusion. Inputs obtained from sensors or GPS data could be a complimentary information for map matching which will further increase localization accuracy effectively.

Nowadays, the whole world map is digitalized and accessible from map service providers on the web. Among the widely known digital mapping services utilized in navigation system development are the Google Maps, Bing Maps, Apple Maps and OpenStreetMap. Here, each service will be described and compared in terms of the service availability and information accuracy.

A. Bing Maps

Bing Maps is a web mapping service provided as part of the Microsoft's Bing suite of search Road view is the default map view that displays vector imagery of roads, buildings, and geography. The data from which the default rendered road map is licensed from Navteq.

In some places, road view maps from alternative data providers are also available. For instance, in the UK, road data from the Ordonnance Survey can also be displayed. There are also 1st and 3rd party applications that can add additional functionality and content to Bing Map such as parking finder, taxi fare calculator and Facebook friends map. Bing Maps updates are being released on a monthly basis and each imagery release may contain more than 10TB of imagery data.

B. Google Maps

As recognized from its name, Google develops the Google Maps service since 2005, that currently offers satellite imagery, street maps, 360 • panoramic street view, real time traffic updates and route planning. The service's front end utilizes JavaScript, XML and Ajax. In 2008, Google Map Maker was launched to allow users to contribute in mapping service update, but it was discontinued from March 2017. However, any errors such as inaccurate geographical locations or location names can be reported by users by suggesting corrections through the 'send feedback' button which will then be reviewed before accepted or declined.

Google Maps might have a lot of useful information and services, but it is not open at the level of raw map data, in order to maintain its commercial advantage. They only expose their downstream products and services from the map data. However, with the growing tendency among developer communities to explore on the innovation possibilities from the raw map data, it is not feasible on the Google's restricted platform. However, Apple Maps has also been criticized for numerous errors and poor map detalization since its launch. Since then, some errors have been corrected, but the problem of detalization is still relevant for the USA, Western Europe, and some Asian countries. For example, in Moscow, Russia, according to Apple Maps, there are no buildings, and some cities look like cross-roads in the fields. Another drawback of Apple maps is that it is not accessible for use in the offline mode. The system can display the cached data without internet connection, but it displays only those regions that a user has viewed before and will not work at a new place.

C. Apple Maps

D. OpenStreetMaps

OpenStreetMap (OSM) is an open project that was aimed at providing a free detailed map of the world. The project is constantly updated and improved by users around the world where any registered user can edit and make additions to the map. This ensures high accuracy and detalization of cartographic data. OSM was initiated by Steve Coast in 2004, inspired by the success of Wikipedia and the predominance of proprietary map data. Since then, it has grown to over 4 million registered users as of November 2017, who can collect data using manual survey, GPS devices, aerial photography, and other free sources for map update. This crowdsourced data is made available under the Open Database Licence that allows everyone to access, download, store, and use the data freely without restrictions.

Aside from the free access, here are several advantages of OSM in comparison with other commercial map service providers:

• Users can style their map to look how they want, while with other map providers, the map style is fixed by the provider.

• With OSM, users are in control of viewing things as wished. They can choose to emphasize cycle routes (where most of other maps do not even have cycle routes) and play down motorways or only want to label subway stops but ignore bus stops by filtering the raw map data.

• Most commercial providers just do streets but OSM do much more which includes natural features, bus routes, footpaths and cycleways, administrative boundaries, shops, rivers and canals.

• Maps from commercial providers are updated every month based on the region, or less. New roads and buildings can be missing long after they are built without frequent update. OSM data is constantly updated, any minute or users can opt to update the map themselves.

• There is no limit to what users can do with OSM as the data keeps on growing every second and raw data is always easily available for use.

• Users can download all or some of the map for offline use, meaning we can safely use OSM information to find our way around without having to reveal our location to anybody.

In fact, more major organizations have been choosing OSM for their maps. It is obvious that public contribution on map data is the most effective way to acquire an up-to-date information as portrayed by maps from OpenStreetMap. This information is very useful in LBS and also aiding vehicle localization if utilized correctly. Besides, the availability of information provided by the digital maps to public allows rapid development in map-based applications.

Contributions

Certainly, an autonomous car might have solved the main problems in localization and risk assessments, but it owes to the equipped sensors performance with higher computational and financial cost. The objective in our research has been to develop an accurate low-cost localization in urban environment for common vehicle. Without requiring built-in sensors installation on vehicle, we aim to develop a standalone system that is autonomous and can be widely used in ADAS. The main problems in GPS data are studied and information provided by available sensors are carefully considered and selected to achieve our objectives.

There are three hypothesis that we can state in this work:

• GPS is the most widely used system as a base for vehicle positioning due to its low-cost and availability,

• digital map is rich with useful information for vehicle navigation which is expected to continuously improve, and

• the use of cameras on a car has become more common which can be utilized for visual odometry trajectory, aside from safety system and road features / symbolic / markings detection purposes.

In this study, we try to explore how the data fusion of low-cost GPS, digital map and visual odometry trajectory can contribute in vehicle localization improvement. As a complementary information to overcome GPS flaws, vehicle ego-motion from visual odometry trajectory is used for pose prediction. Then, the trajectory is also matched with road information from the OSM. Visual odometry has been widely studied by researchers that either improves its performance in monocular system [START_REF] Scaramuzza | Real-time monocular visual odometry for on-road vehicles with 1-point RANSAC[END_REF][START_REF] Forster | SVO: Fast semi-direct monocular visual odometry[END_REF][START_REF] Engel | Semi-dense Visual Odometry for a Monocular Camera[END_REF], or stereo system [START_REF] Kitt | Visual odometry based on stereo image sequences with RANSAC-based outlier rejection scheme[END_REF][START_REF] Geiger | StereoScan: Dense 3d reconstruction in real-time[END_REF][START_REF] Wang | Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras[END_REF].

Meanwhile, there are also works that proposed application in Simultaneous Localization and Mapping (SLAM) [START_REF] Cvišić | SOFT-SLAM: Computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles[END_REF][START_REF] Esparza-Jiménez | Visual EKF-SLAM from Heterogeneous Landmarks[END_REF][START_REF] Moreno | A constanttime SLAM back-end in the continuum between global mapping and submapping: application to visual stereo SLAM[END_REF][START_REF] Mur-Artal | ORB-SLAM: a Versatile and Accurate Monocular SLAM System[END_REF] which is particularly useful in robotic mapping of an unknown region with drift correction from loop-closure detection. However, in vehicle navigation, we can simplify the mapping problem since the map data can be obtained beforehand from the available resources. Therefore, our method only requires the visual odometry trajectory as an additional information in data fusion with GPS and map.

In the end, this thesis has contributed in:

• providing the proof of accountability in utilizing visual odometry trajectory in data fusion for positioning error optimization in both latitudinal and longitudinal,

• the probability of utilizing visual odometry trajectory curve in determining lane-changing events

• a practical low-cost solution in accurate localization with high robustness against different types of GPS error

Thesis Outline

This thesis consists of six chapters. The first chapter gives general introduction on the research background, motivation, and contribution of the thesis.

In Chapter 2, literature review of related works on visual odometry are presented, to provide deeper understanding of limitations faced and the research gap that we would like to address in our work. Here we also provide a review in multisensor data fusion, with benchmark scoreboard analysis.

Then in Chapter 3, we present the first approach in our method by further explaining about the fusion technique used on VO trajectory, noisy GPS and OSM data. We implemented the particle filter method and tested on a simple drive to study the localization performance. Then, a Road Probability Distribution Factor (RPDF) method is proposed and explained in this chapter to further improve the localization accuracy.

Meanwhile in the following Chapter 4, we continue to develop our fusion technique by adding another fusion level for trajectory curve comparison in order to optimize positioning accuracy. This is called the Segmented Curve Matching (SCM) since the visual odometry trajectory trail is compared with segmented roads from OSM. System robustness against severe GPS noise is also evaluated in this chapter. For performance comparison, we tested different VO approaches as the trajectory input to study the degree of improvement in accuracy.

Next, in Chapter 5, a study on lane-changing/lane-keeping detection by VO trajectory curve analysis is presented with two different approaches. We would like to show that it is possible to utilize VO trajectory in lane-changing detection, provided scale ambiguity and heading error are not too severe.

Finally, Chapter 6 will conclude this thesis and provides future works in this research.

As an academician, this research does not stop upon submission of this thesis, but I will seek further knowledge and looking forward to collaborate with other researchers for future research development.

Chapter 2

Related Works on Localization with Visual Odometry

Accurate vehicle localization is a fundamental challenge in intelligent vehicle applications.

It is highly desirable to have the information of one's position accurately to achieve autonomous navigation and integration with location-based services. Thus, various sensors are equipped to vehicles as mentioned in previous chapter and researchers have developed approaches for vehicle positioning from information obtained from wheel odometry, GPS, INS, radar, laser and visual odometry (VO). Each technique has its own advantages and weaknesses in terms of reliability, precision, cost, and feasibility.

In our work, we propose to focus on the employability of VO trajectory curve in our low-cost vehicle localization system. This chapter will present the related works conducted in recent years to assess merits of each technique.

Visual Odometry

The term "odometry" actually originated from the combination of two Greek words hodos that means "journey" and metron which means "measure" [START_REF] Fernandez | Visual odometry for an outdoor mobile robot[END_REF]. Thus, measuring one's journey by estimating the change in an object's pose over time is defined as odometry. Visual odometry (VO) is a positioning technique by using a stream of images acquired from visual sensor (camera). These images contain environment information presented in colour, texture and shape that are useful for feature tracking to estimate camera movement.

The idea of vehicle positioning estimation based on visual input was firstly introduced and suggested in the early 1980s by [START_REF] Moravec | Obstacle Avoidance and Navigation in the Real World by a Seeing Robot Rover[END_REF]. Since then until the new millennia, NASA dominated VO research in preparation for Mars Mission in 2004. The term 'visual odometry' coined by [START_REF] Nistér | Visual odometry[END_REF] was chosen due to its similarity with wheel odometry. However, wheel odometry incrementally estimates one's position or motion by integrating the number of turns of wheels over time, while VO integrates pixel displacements between image frames over time for state estimation.

VO is clearly an inexpensive option in odometry technique that yields higher accuracy than conventional techniques such as INS and wheel odometry. VO has a relative position error ranging from 0.1% to 2% [START_REF] Scaramuzza | Visual Odometry [Tutorial][END_REF]. It can be characterized with balanced trade-off among cost, reliability and complexity of its implementation [START_REF] Nistér | Visual odometry[END_REF]. The utilization of consumer-level camera is a straightforward and low-cost solution compared with the expensive sensors or systems in laser-based localization [START_REF] González | Combined visual odometry and visual compass for off-road mobile robots localization[END_REF][START_REF] Nourani-Vatani | Practical visual odometry for car-like vehicles[END_REF]. Although GPS are widely used and available for outdoor localization, from time to time, it still suffers from lost GPS information or signal noise that cause positioning errors. Hence, with camera as the sensor, VO can work effectively in the GPS-denied environment [START_REF] Scaramuzza | Visual Odometry [Tutorial][END_REF]. Besides, the local drift rate under VO is much smaller that drift of wheel encoders and low precision INS [START_REF] Howard | Real-time stereo visual odometry for autonomous ground vehicles[END_REF]. To achieve maximum accuracy, VO can be integrated with other existing system such as GPS or INS as an additional information input.

VO Approaches

Position estimation by VO can be mainly categorized in three approaches:

• feature-based,

• appearance-based, and

• hybrid of feature/appearance-based

A. Feature-based approach

Self-explained by its name, this method involves extracting image features such as lines, corners and curves in each image frame, that are then matched and tracked in between sequential images. This approach was previously adapted by [START_REF] Benseddik | SIFT and SURF Performance evaluation for mobile robot-monocular visual odometry[END_REF][START_REF] Cumani | Feature Localization Refinement for Improved Visual Odometry Accuracy[END_REF][START_REF] Howard | Real-time stereo visual odometry for autonomous ground vehicles[END_REF][START_REF] Jiang | Performance evaluation of feature detection and feature matching for stereo visual odometry using SIFT and SURF[END_REF][START_REF] Naroditsky | Two Efficient Solutions for Visual Odometry Using Directional Correspondence[END_REF][START_REF] Nistér | Visual odometry for ground vehicle applications[END_REF][START_REF] Parra | Robust Visual Odometry for Vehicle Localization in Urban Environments[END_REF]. Feature matching and tracking are performed only on the distinctive features to estimate the vehicle motion. In this approach, image matching with previous frame is conducted by feature points comparison in both image frames and Euclidean distance of feature vectors are calculated in order to find candidate matching features. Then, camera (vehicle) displacement is estimated from the velocity vector calculation between the pairs of matched feature points [START_REF] Lowe | Distinctive Image Features from Scale-Invariant Keypoints[END_REF][START_REF] Nistér | An efficient solution to the five-point relative pose problem[END_REF][START_REF] Nistér | Visual odometry for ground vehicle applications[END_REF].

For stereo vision system, the extracted features from the first image are matched with the corresponding points in the next frame, that will provide the 3D position of the points in space. With this, camera motion can be estimated from the feature points displacement where the camera relative pose is calculated by finding the geometric transformation between two images using a set of corresponding feature points. Nearest neighbour pairs among feature descriptors are determined to compute the matching between the feature points of two images. To achieve this, Longuet-Higgins in 1987 proposed an 8-point algorithm to compute camera pose via essential matrix [START_REF] Longuet-Higgins | Readings in Computer Vision: Issues, Problems, Principles, and Paradigms[END_REF]. Other researchers had also done studies to improve the robustness of the 8-point algorithm method [START_REF] Hartley | In defense of the eight-point algorithm[END_REF][START_REF] Wu | 8-point algorithm revisited: factorized 8-point algorithm[END_REF] and efficiently solve it in a closed-form algorithm with minimal set of 5-points as proposed by [START_REF] Nistér | An efficient solution to the five-point relative pose problem[END_REF]. Nister suggested that the relative camera pose can be obtained from five matching feature points while [START_REF] Stewénius | Recent developments on direct relative orientation[END_REF]) used 6,7 and 8 feature pairs for motion estimation. This feature-based VO approach has been succesfully implemented in the navigation system of Mars exploration rovers as reported in [START_REF] Maimone | Two years of visual odometry on the Mars Exploration Rovers[END_REF].

There are several feature detection methods usually used in this approach such as FAST, SIFT and SURF. FAST is an acronym for 'Feature from Accelerated Segment Test', which is a quick corner detection [START_REF] Rosten | Machine Learning for High-speed Corner Detection[END_REF]. It uses a circle constructed from 16 pixels (with 3 pixels radius) around a centre pixel p to determine if p is a corner. The detection is performed by comparing pixel intensity of adjacent pixels in the ring with intensity of p as shown in Figure 2.1 below.

FIGURE 2.1: FAST corner detector: Pixels 11-16 and 1-6 are brighter than p, hence p is detected as a feature point.

On the other hand, SIFT (stands for Scale Invariant Feature Transform) detector is based on Difference-of-Gaussians (DoG) where it detects the centres of blob-like structures [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF]. Even though SIFT is very efficient in object recognition applications, it requires a high computational cost which is a disadvantage for real-time applications [START_REF] Lowe | Distinctive Image Features from Scale-Invariant Keypoints[END_REF][START_REF] Miksik | Evaluation of local detectors and descriptors for fast feature matching[END_REF]. Then, SURF (Speeded Up Robust Features) was proposed to improve computation efficiency of SIFT [START_REF] Bay | Speeded-Up Robust Features (SURF)[END_REF]. SURF detect features from a blob detector based on the Hessian matrix approximation to find points of interest. However, studies conducted on feature detectors comparison show that FAST has the best overall performance above SIFT and SURF in terms of accuracy, robustness and speed [START_REF] Ali | A comparison of FAST, SURF, Eigen, Harris, and MSER features[END_REF][START_REF] Guerrero | A Comparative Study of Three Image Matcing Algorithms: Sift, Surf, and Fast[END_REF][START_REF] Işık | A comparative evaluation of well-known feature detectors and descriptors[END_REF].

B. Appearance-based approach

The appearance-based approach is a more recent approach, also known as direct-based approach, proposed in several VO techniques [START_REF] González | Combined visual odometry and visual compass for off-road mobile robots localization[END_REF][START_REF] Lovegrove | Accurate visual odometry from a rear parking camera[END_REF][START_REF] Mcmanus | Towards lighting-invariant visual navigation: An appearance-based approach using scanning laser-rangefinders[END_REF][START_REF] Nourani-Vatani | Practical visual odometry for car-like vehicles[END_REF][START_REF] Yu | Appearance-based monocular visual odometry for ground vehicles[END_REF]. Appearance-based approach generally monitors the changes in image appearance and its intensity of pixel information instead of extracting and tracking image features. While focusing on the information extracted from pixel intensity, the camera motion is estimated by using optical flow. Optical flow algorithm applies the intensity values of neighbouring pixels to find the displacement of brightness patterns from an image frame to another [START_REF] Barron | Performance of optical flow techniques[END_REF][START_REF] Campbell | Techniques for evaluating optical flow for visual odometry in extreme terrain[END_REF].

There are two types of optical flow algorithms. The first one is the dense optical flow (DOF) algorithm, introduced by [START_REF] Horn | Determining Optical Flow[END_REF] that calculates all image pixels' displacement by using global constraints. Another algorithm is called sparse optical flow (SOF) that only calculate the displacement of a selected number of pixels in the image, such as the Lucas-Kanade method [START_REF] Lucas | An Iterative Image Registration Technique with an Application to Stereo Vision[END_REF]. It was found that SOF is more desirable over DOF for many VO applications due to the fact that dense algorithms are less robust to noise compared to SOF [START_REF] Campbell | Techniques for evaluating optical flow for visual odometry in extreme terrain[END_REF][START_REF] Corke | Omnidirectional visual odometry for a planetary rover[END_REF][START_REF] Nourani-Vatani | Practical visual odometry for car-like vehicles[END_REF]. In SOF, the features are selected carefully based on higher variance between neighbour pixels, hence resulting in more reliable motion estimation.

One of the common method used in this approach is the template matching method.

The template matching method attempts to match a selected patch or template of the current image frame with the next frame. From the template matching, vehicle displacement and rotation angle can be retrieved. Thus, template matching is an important task in this approach. It is also useful in other computer vision applications such as object detection, video compression and automatic inspection. It is the process of determining the existence and position of an object (template) inside a large scene image (search area) [START_REF] Choi | A novel two stage template matching method for rotation and illumination invariance[END_REF][START_REF] Goshtasby | A Two-Stage Cross Correlation Approach to Template Matching[END_REF]. Template matching computes the similarity between the template and search area by moving the template over the search area and calculate the similarity degree in each location based on various measures. The shift position with highest degree of similarity is the most likely position of the template found in the search area. The template matching can be obtained through the normalized cross correlation [START_REF] Aqel | Adaptive-search template matching technique based on vehicle acceleration for monocular visual odometry system[END_REF].

From the identified template location, pixel displacement between the template and the maximum correlation point is calculated to obtain the horizontal and vertical pixel displacements (∆u and ∆v). In order to achieve measurements in metres, the pixel displacements are converted from the intrinsic and extrinsic camera calibration parameters through the following equations:

∆X c = -∆u Z c f x , (2.1
)

∆Y c = -∆v Z c f y , (2.2)
where f x and f y both represent the camera focal length which typically have the same value. As for the camera coordinate plane (X c , Y c , Z c ), it will be converted to the vehicle coordinate plane (X v , Y v , Z v ) based on rotation matrix R c obtained from when the camera rotation is 180 • around Z-axis and then 180 • around Y-axis. This computation is depicted in equation below:

R c = R z × R y =    cos(θ z ) -sin(θ z ) 0 sin(θ z ) cos(θ z ) 0 0 0 1    ×    cos(θ y ) 0 sin(θ y ) 0 1 0 -sin(θ y ) 0 cos(θ y )    . (2.3)
Therefore, the vehicle coordinate plane conversion at frame i can be calculated as:

   ∆X v i ∆Y v i ∆Z v i    = R c ×    ∆X c i ∆Y c i ∆Z c i    . (2.4)
Assuming the motion model is based on Ackermaan-steered model, translation ∆X i is considered from ∆X v i and rotation ∆θ i is obtained from:

∆θ i = tan -1 ∆Y v i L cam , (2.5)
where L cam is a parameter defined by the distance between camera center and the vehicle's center of rotation. Thus, the new position of the vehicle, P new is calculated from the previous position, P prev and incremental translation in the X-axis, T and rotation matrix R rotated around the Z-axis by a heading angle when θ i+1 . This final output is derived from the following equation:

P new = P prev + R × T.
(2.6)

C. Hybrid approach

This approach is a hybrid of feature-based and appearance based. The feature-based approach gives better results with textured images, such as urban environments with rough textures than in environments of a single pattern that have low textures (i.e: concrete, sandy soil). The lack of salient features to be detected in the images resulted in inefficiency of feature-based approach in such environments [START_REF] González | Combined visual odometry and visual compass for off-road mobile robots localization[END_REF][START_REF] Johnson | Robust and Efficient Stereo Feature Tracking for Visual Odometry[END_REF][START_REF] Nourani-Vatani | Correlation-based Visual Odometry for Ground Vehicles[END_REF].

On the other hand, appearance-based approach seems to be more robust and superior in low-textured images [START_REF] Kicman | Concept of integrated INS/visual system for autonomous mobile robot operation[END_REF]. Therefore, in certain scenarios, hybrid approach appears to be the best solution by combining both techniques. This approach integrates tracking features between frames and at the same time use pixel intensity information of the image to obtain the optical flow. For instance, [START_REF] Scaramuzza | Appearance-Guided Monocular Omnidirectional Visual Odometry for Outdoor Ground Vehicles[END_REF]) implemented a hybrid-based approach where they utilized image features from the ground plane in estimating vehicle translation while vehicle rotation was estimated from the image appearance.

Prior VO Works

Majority of VO methods proposed in previous literature use either stereo or monocular camera in their systems. Monocular camera means that the system only utilizes a single camera for capturing surrounding images. Meanwhile, stereo systems use two cameras that for image capture that resembles human eyes in order to obtain object information in 3D. Both methods have advantages and disadvantages that will be explained in this subchapter.

A. Stereo system

During the early days of vehicle ego-motion system by visual inputs in the 1980s introduced by Moravec, a planetary rover with a single camera sliding on a rail was used [START_REF] Moravec | Obstacle Avoidance and Navigation in the Real World by a Seeing Robot Rover[END_REF]. The slider is called a slider stereo and the rover moved and stopped periodically.

In each stop, the camera slid and captured nine images at an equal intervals. Motion was estimated by triangulation of 3D points seen at two consecutive positions after corners are detected and matched through normalized cross correlation (NCC). Although only one camera was used in this approach, it is classified as stereo VO algorithm due to the triangulation step. Basically, stereo cameras have been widely used in VO systems to capture a pair of RGB images at the same instant, thus the image scale can be retrieved by a given baseline.

Nister et al., who named the term 'visual odometry' was the first to demonstrate a realtime long-run implementation with a robust outlier rejection scheme [START_REF] Nistér | An efficient solution to the five-point relative pose problem[END_REF]. They detected features independently in all frames and only allowed matches between features to avoid feature drift during cross correlation-based tracking. Since then, VO has received an increasing interest among researchers that proposed various stereo VO implementations as presented by [START_REF] Azartash | An Integrated Stereo Visual Odometry for Robotic Navigation[END_REF][START_REF] Geiger | StereoScan: Dense 3d reconstruction in real-time[END_REF][START_REF] Gomez-Ojeda | Robust stereo visual odometry through a probabilistic combination of points and line segments[END_REF][START_REF] Howard | Real-time stereo visual odometry for autonomous ground vehicles[END_REF][START_REF] Kitt | Visual odometry based on stereo image sequences with RANSAC-based outlier rejection scheme[END_REF][START_REF] Wang | Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras[END_REF].

Algorithm proposed by Howard is generally consisted of feature detection in each frame pair, followed by feature matching. Then, the largest set of self-consistent matches (inliers) are identified. Finally, it calculates the frame-to-frame motion that minimizes re-projection error for features in the inlier set [START_REF] Howard | Real-time stereo visual odometry for autonomous ground vehicles[END_REF]. In this method, the inlier detection step is the most important to distinguish feature, hence it is best described as 'inlier detection' method rather than 'outlier rejection'. On the other hand, [START_REF] Kitt | Visual odometry based on stereo image sequences with RANSAC-based outlier rejection scheme[END_REF] suggested VO method with outlier rejection scheme by RANSAC. It is directly based on the trifocal geometry between image triples, resulting in less time consumption required for 3D scene structure recovery. Camera geometry is assumed to be known in this approach, and the authors employed an Iterated Sigma Point Kalman Filter (ISPKF) combined with RANSAC-based outlier rejection scheme to obtain robust motion estimation even in dynamic environments.

This VO algorithm (known as Libviso) is made open-source which then allows further technique improvement as proposed by [START_REF] Geiger | StereoScan: Dense 3d reconstruction in real-time[END_REF].

Geiger et al. improved Libviso's performance by implementing 'stereoscan' where they proposed a sparse feature matcher for robust VO algorithm by dense 3D reconstruction from stereo sequences. The presented reconstruction pipeline combines four stages: sparse feature matching, ego-motion estimation, dense stereo matching and 3D reconstruction. This method managed to reduce running times of feature matching while at the same time allows real-time 3D reconstructions from large-scale imagery. It is then published publicly as Libviso2 that is still widely used for benchmarking.

Later in 2013, VO with Iterative Closest Multiple Lines (ICML) algorithm was proposed for an efficient line matching [START_REF] Witt | Robust stereo visual odometry using iterative closest multiple lines[END_REF]. This method contributed in fast and robust hypothesize-and-test algorithm which can act as a fallback for challenging frame pairs with textureless scenarios where pure gradient-based optimization fails. Since lines have proven to be an interesting alternative to points in man-made environments, Azartash then explored the combination of point and line features to robustly compute the six degree of freedom motion transformation between consecutive image pairs. This method, named MEVO (stands for 'multi-environment visual odometry'), achieved real-time speed of over 10 frames per second with good localization performance compared with Libviso2.

While MEVO proposed a direct combination of point and line features, [START_REF] Gomez-Ojeda | Robust stereo visual odometry through a probabilistic combination of points and line segments[END_REF] proposed a probabilistic approach on the combination of both point and line segment for stereo VO. In order to effectively combine both types of features, projection errors of both point and line segment features were weighted according to their covariance matrices, computed from the propagation of Gaussian distribution errors in the sensor measurements. This resulted in more expensive computational cost, but it could still run in real-time and it provided an advantage of being able to be integrated straightforward into any probabilistic framework.

Recently, [START_REF] Wang | Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras[END_REF] had proposed a Stereo Direct Sparse Odometry (Stereo DSO) that aimed to integrate constraints from static stereo into the bundle adjustment pipeline of temporal multi-view stereo. With a system that combined static stereo with multi-view stereo, the absolute scale could be directly calculated, and initial depth could be estimated from the static stereo. Due to the fixed baseline, static stereo could only accurately triangulate 3D points within a limited depth range and this limit was resolved by temporal multiview stereo.

Among the proposed stereo VO, some of the works are made available to public such as Libviso [START_REF] Kitt | Visual odometry based on stereo image sequences with RANSAC-based outlier rejection scheme[END_REF], Libviso2 [START_REF] Geiger | StereoScan: Dense 3d reconstruction in real-time[END_REF] and stereo-DSO [START_REF] Wang | Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras[END_REF].

This allows researchers to do VO performance comparison to study the strength and weakness in presented works. Besides, other well-known Simultaneous Localization and Mapping (SLAM) that utilized stereo VO in their algorithms are ORB-SLAM2 as proposed by [START_REF] Mur-Artal | ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras[END_REF] and RSO by [START_REF] Moreno | A constanttime SLAM back-end in the continuum between global mapping and submapping: application to visual stereo SLAM[END_REF]. In addition to localization capability, SLAM also performs mapping and loop closure for more accurate results although this adds computation complexity in order to achieve accurate performance.

B. Monocular system

If the distance to the scene from the stereo camera is much larger than the stereo baseline, stereo vision becomes ineffectual and it can be degraded to monocular system [START_REF] Scaramuzza | Visual Odometry [Tutorial][END_REF][START_REF] Sünderhauf | Stereo odometry-a review of approaches[END_REF]. Both of the relative motion and 3D structure are computed from 2D bearing data in monocular VO. Examples of recent VO works that implemented monocular system have been conducted by [START_REF] Engel | Semi-dense Visual Odometry for a Monocular Camera[END_REF][START_REF] Engel | Large-scale direct SLAM with stereo cameras[END_REF][START_REF] Forster | SVO: Fast semi-direct monocular visual odometry[END_REF][START_REF] Forster | SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems[END_REF][START_REF] Graeter | Robust scale estimation for monocular visual odometry using structure from motion and vanishing points[END_REF]Lee et al., 2015a;[START_REF] Van Hamme | Robust monocular visual odometry for road vehicles using uncertain perspective projection[END_REF].

( [START_REF] Van Hamme | Robust monocular visual odometry for road vehicles using uncertain perspective projection[END_REF] had proposed a monocular VO algorithm that uses planar tracking of feature points on the ground plane surrounding the vehicle rather than conventional 3D pose estimation. To ease consistency of motion among features, tracking was applied in the ground plane coordinates instead of in the image coordinates of the perspective camera. Meanwhile, an online self-learning approach of monocular VO and ground classification were presented by (Lee et al., 2015a). They used a constrained kinematic model to solve the motion and structure problem besides to estimate the ground surface. The online self-learning scheme from monocular vision was achieved by a combination of probabilistic appearance-based ground classifier with geometric estimates. [START_REF] Graeter | Robust scale estimation for monocular visual odometry using structure from motion and vanishing points[END_REF] on the other hand aimed to tackle issue on scale drift in monocular VO. To do so, the scene inherent information about the ground plane to estimate the scale was leveraged for usage on Advanced Driver Assistance Systems. A ground plane estimation using Structure From Motion (SFM) techniques was complemented by a vanishing point estimation to render the proposed algorithm robust in urban scenarios. A novel pitch-and-roll-tolerant scaling algorithm for monocular visual odometry, tailored for urban environments was also proposed in their work. This was achieved by two complementary methods for ground plane normal estimation: a fit to reconstructed points and vanishing points. The scale was finally obtained from a trimmed Least Squares optimization that refined the plane.

Engel's and Forster's works are well-cited for their open-source codes on monocular VO widely known as semi-dense VO and semi-direct VO. Semi-dense VO was introduced by [START_REF] Engel | Semi-dense Visual Odometry for a Monocular Camera[END_REF], where a semi-dense depth filtering formulation was presented to significantly reduce computational complexity. Previously, it was found that only pixels with an intensity gradient provide information for motion estimation [START_REF] Dellaert | Fast image-based tracking by selective pixel integration[END_REF]. This semi-dense method was proposed by estimating the depth of all pixels with non-negligible image gradient and each estimate is represented as a Gaussian probability distribution over the inverse depth map for the current frame. Then, [START_REF] Forster | SVO: Fast semi-direct monocular visual odometry[END_REF] proposed Semi-Direct VO combined direct tracking with keypoints, which achieved high frame-rates even on embedded platforms. This owed to their algorithm that operated directly on pixel intensities. Besides, more reliable points were obtained from a probabilistic mapping method that models outlier measurements to estimate 3D points.

Engel then developed his work on Large-Scale Direct Monocular SLAM (LSD-SLAM)

which allowed to build large-scale, consistent maps of the environment. Along with highly accurate pose estimation based on direct image alignment, the 3D environment was reconstructed in real-time as pose-graph of keyframes with associated semi-dense depth maps.

Slightly different from LSD-SLAM that employed a photometric error as well as a geometric prior to estimate dense or semi-dense geometry, [START_REF] Wang | Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras[END_REF]) also proposed a novel method namely direct sparse odometry (DSO) that took full advantage of photometric camera calibration for further increase in accuracy and robustness.

Challenges and Limitations

While monocular system is admittedly the most efficient in terms of cost, calibration errors and scale uncertainty are the main challenges to obtain accurate localization with monocular camera. Furthermore, on uneven road surface, image scale has the tendency to fluctuate and the image scaling factor is also difficult to be estimated when there is a large change in the road slope or vehicle speed. The scale factor plays an important role in localization in order to compute the translation length in real world from the trajectory generated from image sequences.

Due to this, [START_REF] Kitt | Monocular Visual Odometry using a Planar Road Model to Solve Scale Ambiguity[END_REF] claimed that monocular vision systems are negatively affected by scale uncertainty that would fail the localization system without accurate scale factor. However, they also proposed a solution by using the Ackermann steering model and assumed that if the vehicle moves on a planar road surface, image scale could be recovered. [START_REF] Guo | Monocular Visual Odometry and Obstacle Detection System Based on Ground Constraints[END_REF] on the other hand proposed a monocular VO system with KLT feature tracker for feature extraction and RANSAC was used for outlier rejection, while solving scale ambiguity problem by using the constraints of camera mounting and ground planar assumption. Unfortunately, this technique does not fully solve the scale problem in monocular systems but it limits the ambiguity to a certain degree.

Stereo camera, on the other hand, has the capability to obtain scale information from the image depth calculated by triangulation process with fixed stereo baseline size. However, stereo system is more expensive than monocular system and it requires more computation in calibration to obtain camera parameters. Besides, it is crucial for the pair of stereo camera to acquire images at an exact time interval which can be achieved by shutter speed synchronization or from external trigger signal. Still, many researchers prefer stereo systems due to its accuracy in localization and ability to obtain map depth information that allows map building in SLAM development.

Multisensor Localization

Accurate vehicle localization has been immensely researched within the last decades, in the development towards autonomous vehicle. While the technology has started to develop and been in the market in recent years, it does not stop the motivation for further research.

Besides its application for autonomous vehicle, the current positioning and routing technology in vehicles should also be improved. There are still many issues need to be addressed as road network structure is becoming more complex with the urban development and this frequently cause interruptions in localization and path planning.

Although low-cost GPS are widely used for localization, it suffers from several conditions such as the multipath and Non-Line-Of-Sight (NLOS) effect especially in urban areas due to the dense buildings or other constructions like tunnels and bridges [START_REF] Kos | Effects of multipath reception on GPS positioning performance[END_REF]. [START_REF] Zair | A-Contrario Modeling for Robust Localization Using Raw GNSS Data[END_REF] proposed to overcome GPS signal problem to improve its accuracy by detecting and removing the outliers. This resulted in reliable GPS data but their method consumed complex computation and the results is inconsistent particularly in biased GPS noise.

Therefore, data fusion with other sensors is desirable to overcome this problem. Multisensor vehicle localization can be from several sensors as presented previously and among those are lidar, GNSS receiver, camera sensor, Inertial Measuring Unit (IMU), and radar sensor. In addition, digital map can also be used as an input for the data fusion. These data and information can be used together, without requiring prior computation or data compensation, to provide a new information of an estimated state.

Fusion of GPS, Camera, INS and lidar

Among the common solution approach for the GPS signal degradation problem is to fuse GPS with other sensors, such as odometer, Inertial Measurement Unit (IMU), laser scanner and mono/stereo camera. One of the earliest fusion approach was by fusing data from vehicle GPS system and odometer data of the vehicle [START_REF] Abuhadrous | Multi-sensor data fusion for land vehicle localization using/sup RT/MAPS[END_REF][START_REF] Caron | GPS/IMU data fusion using multisensor Kalman filtering: introduction of contextual aspects[END_REF][START_REF] Sasiadek | Fuzzy Adaptive Kalman Filtering for INS/GPS Data Fusion and Accurate Positioning[END_REF][START_REF] Sukkarieh | A high integrity IMU/GPS navigation loop for autonomous land vehicle applications[END_REF]. While odometer might have advantage such as minimal interference with other sensor signals, it suffers from limitations including position divergence over time, miscalculations caused by slippage or uneven roads. Besides, the accuracy is also affected by environment temperature and air pressure in tyres and the implementation requires intervention with the existing wheel sensor for data collection making it difficult to be built in standalone system.

Meanwhile, Inertial Navigation System (INS) which utilizes IMU sensor can also be used to collect the position data during GPS signal outage [START_REF] Bencheikh | Attitude observers for accelerated vehicles using IMU and GPS measurements[END_REF]. Unfortunately, its accuracy can only remain for short periods due to the accelerometer biases and gyroscope drifts [START_REF] Caron | GPS/IMU data fusion using multisensor Kalman filtering: introduction of contextual aspects[END_REF]. There are also studies on using laser scanner or lidar to obtain high resolution information of vehicle surroundings with GPS data fusion [START_REF] Chong | Synthetic 2d lidar for precise vehicle localization in 3d urban environment[END_REF][START_REF] Gao | INS/GPS/LiDAR integrated navigation system for urban and indoor environments using hybrid scan matching algorithm[END_REF][START_REF] Wolcott | Visual localization within lidar maps for automated urban driving[END_REF][START_REF] Yoneda | Lidar scan feature for localization with highly precise 3-D map[END_REF]. The use of such dimensional sensors has been studied to find vehicle pose and brings to the implementation of SLAM (Simultaneous Localization And Mapping) [START_REF] Durrant-Whyte | Simultaneous localization and mapping: part I[END_REF]. As mentioned previously, lidar is an active sensor that measures range distances by estimating the time-of-flight of laser beams to its surrounding objects. While a 2D lidar scans horizontal planes and is usually used to localize a robot on a flat area [START_REF] Zhang | Line segment based map building and localization using 2D laser rangefinder[END_REF], 3D lidar has also been studied for 6DOF localization [START_REF] Moosmann | Velodyne SLAM[END_REF].

In recent years, many works proposed VO fusion with GPS, odometer and lidar [START_REF] Graeter | LIMO: Lidar-Monocular Visual Odometry[END_REF][START_REF] Pandey | Ford Campus vision and lidar data set[END_REF][START_REF] Wolcott | Visual localization within lidar maps for automated urban driving[END_REF][START_REF] Zhang | Visual-lidar odometry and mapping: low-drift, robust, and fast[END_REF]. The proposed approaches showed performance accuracy of up to merely 0.60% in translation error as recorded from V-LOAM method by [START_REF] Zhang | Visual-lidar odometry and mapping: low-drift, robust, and fast[END_REF]. These studies exhibited interesting outcomes, but the high cost and long scanning period making it impractical for consumer level application.

On the other hand, [START_REF] Aynaud | Real-Time Multisensor Vehicle Localization: A Geographical Information System-Based Approach[END_REF] suggested a top-down approach in multisensor localization with triplets detection by lidar and choose the most efficient set of sensor, landmark, and detector. The results demonstrated that the top-down approach is applicable in vehicle localization. However, even with lidar, there were localization errors due to lack of landmarks detection. Another method was recently proposed by [START_REF] Nguyen | Improving poor GPS area localization for intelligent vehicles[END_REF] to overcome GPS signal problem by WiFi finger printing, fused with lidar-based SLAM. However, despite the use of high performance sensor like lidar, the fusion output reached up to 8m of localization error during weak GPS signal. Therefore, thiese methods might require additional sensors to improve localization accuracy.

Low-Cost Data Fusion with Camera

Since cameras have relatively lower-cost and can also be integrated in the localization system, VO has become one of the popular approaches for localization. VO can be classified into monocular or stereo vision system in which stereo system usually performs better since it can avoid scale ambiguity inherent in monocular VO. Presented works for both monocular and stereo VO systems have been explained in the beginning of this chapter with their challenges and limitations. Therefore, to overcome this problem and achieve better localization, several approaches by implementing GPS fusion with VO and other sensors were proposed.

For the sake of practicality and affordable implementation, fusion with lidar sensor is not discussed in this section.

Although VO trajectory itself is prone to accumulated drift and can be affected by scale ambiguity, by integrating it with other sensors would possibly solve this problem. For instance, in [START_REF] Peker | Fusion of map matching and traffic sign recognition[END_REF], monochrome camera and Kinect sensor were used to detect traffic sign and match with map information to improve vehicle localization. This highly relies on the availability of traffic sign recorded in map information hence is not optimized for global localization. Meanwhile, [START_REF] Suhr | Sensor Fusion-Based Low-Cost Vehicle Localization System for Complex Urban Environments[END_REF] proposed sensor fusion-based low-cost vehicle localization system that fuses data from GPS, an IMU, a wheel speed sensor, a single front camera, and a digital map via the particle filter. This requires a low-volume digital map where road markings are expressed by a minimum number of points. However, aside from VO purposes, we found that most of the works in vehicle localization utilized cameras for road markings detection to improving positioning accuracy (Lee et al., 2015b;[START_REF] Schreiber | Multi-drive feature association for automated map generation using low-cost sensor data[END_REF][START_REF] Suhr | Sensor Fusion-Based Low-Cost Vehicle Localization System for Complex Urban Environments[END_REF][START_REF] Tao | Lane marking aided vehicle localization[END_REF]. Stop lane marking or intersection detections are used to correct the longitudinal position but there is a possibility of occluded lane markings or roads without lane markings that can degrade the localization performance. Besides, we also need to consider environmental influence such as weather and construction effects on the visibility of the markings. Hence, relying on the symbolic road or lane markings has some disadvantages for a robust localization.

On the other hand, other researchers such as [START_REF] Floros | OpenStreetSLAM: Global vehicle localization using OpenStreetMaps[END_REF] used Chamfer matching on the street graph with Monte Carlo localization to match the vehicle trajectory with the road network and [START_REF] Brubaker | Map-Based Probabilistic Visual Self-Localization[END_REF] utilized road network information from Open-StreetMap by identifying the actual link between vehicle and road map through stereo vision system during GPS signal outage. Both of these methods have shown improvised localization trajectory of the vehicle on the correct road, but lane-level and longitudinal positioning accuracy has yet to be addressed. In [START_REF] Floros | OpenStreetSLAM: Global vehicle localization using OpenStreetMaps[END_REF], the method performed an exhaustive matching over all possible positions and orientations against the template edge map. Despite this, its localization error of has yet to achieve below 10m of average on two datasets tested as presented in the paper. As for [START_REF] Brubaker | Map-Based Probabilistic Visual Self-Localization[END_REF], the proposed approach managed to achieve an average of 3.7m positioning error but there were cases of ambiguous road networks that resulted in failed localization.

Datasets for VO Benchmarking

For VO localization performance benchmarking, several datasets have been published to the public that mostly contain data collected using IMU, GPS, cameras, and lidars. The datasets are KITTI dataset [START_REF] Geiger | Are we ready for autonomous driving? The KITTI vision benchmark suite[END_REF], CMU Visual Localization dataset [START_REF] Badino | The CMU Visual Localization Data Set[END_REF], and Malaga dataset [START_REF] Blanco | The Málaga Urban Dataset: High-rate Stereo and Lidars in a realistic urban scenario[END_REF]. Among these, KITTI dataset has the widest coverage, consisting sequences in city, residential, highway road and in campus environment. VO methods that has been published are compiled for performance comparison as shown in Table 2.1. Some of the methods also included their codes online for public reference.

From all test sequences provided by KITTI, the methods evaluation computes translational and rotational errors for all subsequences of length (every 100 metres). The evaluation of translational and rotation error, in accordance to system complexity. From the varying results, rotational error of VO methods is not severe as its translational error. Therefore, we can make three hypotheses that:

• the problem lies more on the scale ambiguity,

• trajectory direction maintains good accuracy due to small rotational error, and

• with the consistent direction accuracy, trajectories from VO method would mostly concede with road network shape on map.

These hypotheses are also in line with our finding regarding the higher localization error for drive sequence on a straight road -localization performs worse when there is less heading rotation during the drive.

Conclusion

Since its introduction in the early 21st century, VO has been studied vigorously for its application in mobile robotics and vehicle localization. Aside from having better performance than wheel odometer, it can even replace the use of laser scanner for map building in SLAM methods [START_REF] Cvišić | SOFT-SLAM: Computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles[END_REF][START_REF] Engel | Large-scale direct SLAM with stereo cameras[END_REF][START_REF] Fanfani | Accurate keyframe selection and keypoint tracking for robust visual odometry[END_REF][START_REF] Mur-Artal | ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras[END_REF][START_REF] Schlegel | ProSLAM: Graph SLAM from a Programmer's Perspective[END_REF]. At the same time, vehicle equipped with camera allows other intelligent systems to be deployed for advanced driver-assistance systems (ADAS) such as pedestrian detection and road signs recognition.

Although Chapter 3

Data Fusion with Road Probability Distribution Factor for Localization Improvement

As one of the approaches to non-stationary inverse problems, state estimation has been of a great interest in many practical applications [START_REF] Kaipio | Statistical and computational inverse problems[END_REF]. In such cases, the available data obtained from measurement of environment information are fused together with the prior knowledge of state condition to continuously produce estimates of the desired dynamic outputs. This can be accomplished in such a manner that the error is minimized statistically after the predicted state is corrected with additional information [START_REF] Maybeck | Stochastic models, estimation, and control[END_REF].

For instance, the position of a moving object can be estimated through the time integration of its velocity components since departure. In our case, the prediction of the vehicle position is based on visual odometry displacement. Then, this prediction will go through a correction phase, where the assumptions are measured according to the likelihood. GPS data and information from OSM are used in this phase to narrow down the possibility of obtaining the best position estimation. In short, the vehicle state estimation problem deals with the combination of the movement prediction from the displacement measurements (the integration of the velocity components may contain measurement errors), then corrected by the GPS data that also has no guarantee to be error-free, together with OSM data. This fusion is conducted in order to obtain more accurate estimations of the system variables, which is the vehicle position. In this chapter, the fusion technique is presented with our proposed Road Probability Distribution Factor (RPDF) method based on OSM data to observe Improvement the improvement in localization performance.

Filters for Data Fusion

State estimation problems are commonly solved with the Bayesian filters [START_REF] Gordon | Bayesian state estimation for tracking and guidance using the bootstrap filter[END_REF][START_REF] Kalman | A New Approach to Linear Filtering And Prediction Problems[END_REF][START_REF] Maybeck | Stochastic models, estimation, and control[END_REF] as a method in data fusion. In the Bayesian approach, an attempt is made to utilize all available information in order to reduce the amount of uncertainty present in an inferential or decision-making problem and as new information is obtained, it is combined with previous information to form the basis for statistical procedures [START_REF] Orlande | Application of Bayesian filters to heat conduction problem[END_REF]. The formal mechanism used in the information fusion with the previously available knowledge of the system state is known as Bayes' theorem [START_REF] Bayes | An Essay towards Solving a Problem in the Doctrine of Chances. By the Late Rev[END_REF][START_REF] Winkler | An introduction to Bayesian inference and decision/by Robert[END_REF].

Kalman filter is the most widely known Bayesian filter method to date which has undergone many improvements by researchers [START_REF] Kalman | A New Approach to Linear Filtering And Prediction Problems[END_REF][START_REF] Maybeck | Stochastic models, estimation, and control[END_REF][START_REF] Sorenson | Least-squares estimation: from Gauss to Kalman[END_REF][START_REF] Weiss | Improved extended Kalman filter design for passive tracking[END_REF][START_REF] Winkler | An introduction to Bayesian inference and decision/by Robert[END_REF]. However, the application of the Kalman filter is restricted to linear models with additive Gaussian noises. Some extensions of the Kalman filter were developed in the past years to partly overcome the restriction by using linearization techniques [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF][START_REF] Kaipio | Statistical and computational inverse problems[END_REF][START_REF] Ristic | Beyond the kalman filter -Book Review[END_REF]. The extended Kalman filter performs linearization around the estimation state and force the filter to use the linearized version as a model.

At the same time, Monte Carlo methods have also been developed in order to represent the posterior density in terms of random samples with associated weights [START_REF] Doucet | An introduction to sequential Monte Carlo methods[END_REF]. The Monte Carlo methods, also known as particle filter among other designations found in the literature, has the advantage of not requiring the restrictive hypotheses of the Kalman filter. Therefore, the particle filter can be applied to non-linear models with non-Gaussian errors and is more suitable for the real-world applications.

Kalman Filter

The Kalman filter [START_REF] Kalman | A New Approach to Linear Filtering And Prediction Problems[END_REF], is one of the most popular data fusion algorithms in the field of information processing. It is typically derived by using vector algebra as a minimum mean squared estimator from first principles considering a simple physical example exploiting a key property of the Gaussian distribution [START_REF] Bandemer | Prediction and improved estimation in linear models[END_REF]. The approach is mean to translate some original observables y k into Wold decomposition [START_REF] Wold | A Study in the Analysis of Stationary Time Series[END_REF] With its fusion technique development, some variants such as the Extended Kalman Filter (EKF) [START_REF] Groves | Principles of GNSS, inertial, and multisensor integrated navigation systems[END_REF][START_REF] Hoshiya | Structural identification by extended Kalman filter[END_REF][START_REF] Weiss | Improved extended Kalman filter design for passive tracking[END_REF] and unscented Kalman filter (UKF) [START_REF] Julier | Unscented filtering and nonlinear estimation[END_REF][START_REF] Wan | The unscented Kalman filter for nonlinear estimation[END_REF] that involves linearization of the problem were developed. In systems with nonlinear dynamics, the observation equations are linearized by Taylor series expansion or partial derivatives of nonlinear state functions. However, the EKF is deemed to be lack of stability and converges very slowly to the correct solution. Most of the times, it only performs well with small nonlinearities, making it impractical for highly nonlinear models.

To solve the limitations in EKF, UKF was introduced where it is based on Unscented Transformation (UT). It uses a deterministic sampling approach to obtain the mean and covariance estimates from a finite set of sample points. These points are chosen deterministically and transformed to a new space via nonlinear process to describe the distribution statistics. The UKF is an expansion of UT which consists of initialization, prediction and actualization steps. Compared with EKF, the UKF has an advantage of being independent on the local approximation, allowing it to perform in higher order nonlinear models. Unfortunately, similar with other Kalman filter, the UKF can only be used for models driven by Gaussian noises and it is not a global approximation, due to the small set of sample points.

In other cases where the hypotheses of linear Gaussian evolution-observation models are invalid, the use of the Kalman filter does not result in optimal solutions since the posterior density is not analytic -even with the use of EKF and UKF filter because they still assume Gaussian distribution in the derivation. Thus, the application of Sequential Monte Carlo technique appears as the most general and robust approach to non-linear and/or non-Gaussian distributions [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF][START_REF] Johansen | A note on auxiliary particle filters[END_REF][START_REF] Kaipio | Statistical and computational inverse problems[END_REF][START_REF] Liu | Sequential Monte Carlo methods for dynamic systems[END_REF]. More details on Sequential Monte Carlo are explained in the following Chapter 3. Data Fusion with Road Probability Distribution Factor for Localization Improvement section.

Particle Filter

The Sequential Monte Carlo technique, also known as the particle filter method, also offers a solution in the state estimation problem. Other names for the particle filter include bootstrap filter, condensation algorithm, interacting particle approximations and survival of the fittest [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF]. The main idea is to represent the required posterior density function by a set of random samples (particles) with associated weights, and to compute the estimates based on these samples and weights. When the number of samples are very large, this Monte Carlo characterization behaves as an equivalent representation of the posterior probability function, and the solution approaches the optimal Bayesian estimate.

One of the particle filtering method is the Sequential Importance Sampling (SIS) algorithm, which includes a sampling step at each instant [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF][START_REF] Ristic | Beyond the kalman filter -Book Review[END_REF]. This algorithm involves using importance density, which is a density proposed to represent the sought posterior density in the present case to solve the recursion equation.

Then, samples are drawn from the importance density. In certain importance functions, the variance of the importance weights can increase over time [START_REF] Kong | Sequential imputations and Bayesian missing data problems[END_REF]. This leads to the degeneracy phenomenon and it has a harmful effect on the estimation accuracy because after a few steps, it is possible that all but one particle will have negligible weight.

This results in a wasteful amount of computational effort that will be devoted to updating a contribution that has almost zero weight. One way to solve this degeneracy problem is by increasing the number of samples, N , which is impractical and costly. Another way is by minimizing degeneracy through a good choice of importance function and by including a resampling step in the SIS algorithm which brings the evolution to the Sequential Importance Resampling (SIR) algorithm as illustrated in Figure 3.1.

Resampling is conducted by mapping of the random measure {x i k , w i k } into a new random measure {x j k , N -1 } with uniformed importance weights. It is executed when the number of effective particles (or effective sampling size) with large weights falls below a certain threshold number. Alternatively, resampling can also be applied indistinctively at certain time t k , as in the Sampling Importance Resampling (SIR) algorithm described in [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF][START_REF] Ristic | Beyond the kalman filter -Book Review[END_REF]. SIR algorithm is summarized in the following steps, as applied in our data fusion in section 3.4.2: i In the prediction stage, generate particles based on the predictive density of respective old particles x i k-1 :

x i k ∼ p(x k | x i k-1 ). ( 3
w i k = w i k-1 p(z k | x i k )p(x i k | x i k-1 ) π(x i k | x i 0:k-1 , z 1:k ) . (3.2)
iii Total weight of all particles, T w is calculated

T w = N i=1 w i k , (3.3)
and the particles weights are normalized .4) iv Effective number of particles is computed as .5) v When the effective number of particles is smaller than given threshold, N ess < N thres , resampling occurs where samples with lower weights are removed and replaced by duplicating those with higher weights. This is done by constructing the cumulative sum of Improvement weights with c 0 = 0, for i = 1, ..., N ;

wi k = w i k T w . ( 3
N ef f = 1 N i=0 ( wi k ) 2 . ( 3
c i = c i-1 + w i k . (3.6)
Then, new particles are assigned with probabilities proportional to their importance weights while particles with lower weight are dismissed. Then, after resampling, all particles has uniformed weight of N -1 .

Summary

There has been extensive research on the usage of filter techniques for position estimation.

State estimation by means of Kalman Filter (KF), as well as Extended Kalman Filter (EKF)

and Particle Filter (PF) have been reported in studies for several applications. Brief comparison between filter techniques for position estimation was presented in [START_REF] Toro | Particle Filter technique for position estimation in GNSSbased localisation systems[END_REF]. It

shows that PF does not require the restrictive hypotheses of the KF approaches, which makes PF a better solution approach for non-linear models for multi-sensor data fusion [START_REF] Cappello | Particle filter based multi-sensor data fusion techniques for RPAS navigation and guidance[END_REF]. Besides, PF provides a natural way to incorporate road map information into vehicle position estimation and it is capable of handling multi-modal distributions.

Thus, in this research, PF is chosen over other fusion technique for its ability to perform multimodal and non-linear estimation. Besides, PF is proved to be more robust and superior than the EKF in terms of the accuracy of state vector estimation [START_REF] Lin | Comparison of EKF , Pseudomeasurement and Particle Filters for a Bearing-only Target Tracking Problem[END_REF][START_REF] Rigatos | Extended Kalman and Particle Filtering for Sensor Fusion in Motion Control of Mobile Robots[END_REF]. PF uses random samples (particles) to represent the posterior density of vehicle position in a dynamic state estimation framework such as a road network. With PF, we estimate the instate-space models inference by expressing the posterior probability density in terms of randomized particles associated with importance weights. However, due to the randomized nature of PF, it has a drawback where the exact result generation is impossible. Every new run of the program will produce different set of results for evaluation.

Conventional Visual Odometry

The initial approach of our research adopted a conventional visual odometry technique to fuse with GPS data for vehicle localization. Visual odometry, as illustrated in Figure 3.2, is the estimation of position and orientation of a moving object by analysing sequence of images from visual sensor attached to the object. The input would be a stream of images, captured at time k and k + 1, referred as I k and I k+1 . These image pairs are compared to obtain translation vector t and rotation matrix R to describe object movement whereby with heading rotation θ, rotation matrix is represented as

R =    cos θ -sin θ 0 sin θ cos θ 0 0 0 1    , (3.7)
and vehicle pose can be defined as

X k = [x k , y k , z k ] .
For image feature detection, FAST algorithm [START_REF] Rosten | Machine Learning for High-speed Corner Detection[END_REF] is applied in our approach due to its computational efficiency compared with other feature detection methods such as SIFT [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF] and SURF [START_REF] Bay | Speeded-Up Robust Features (SURF)[END_REF] as described in Chapter 2. The features detected by FAST are then tracked on the preceding image by using Kanade-Lucas-Tomasi (KLT) feature tracker algorithm [START_REF] Tomasi | Detection and tracking of point features[END_REF]. The tracker utilizes spatial intensity to approximate the search for the position that yields the best match by finding sparse pixel wise correspondences. It is expected to work well with corner-like features for matching and tracking. New feature detection in images is only repeated after number of tracked features falls below a certain value.

From the tracked features, the Essential Matrix E is computed by random sample consensus (RANSAC) method which has an iterative algorithm [START_REF] Nistér | An efficient solution to the five-point relative pose problem[END_REF]. For every iteration, five points are sampled from the set of correspondences, and the essential matrix is estimated. Then, total number of other points which are inliers when using the essential matrix is accumulated. After a fixed number of iteration, the essential matrix with maximum number of inlier points is selected. The essential matrix can be decomposed into rotation and translation as described in [START_REF] Hartley | An investigation of the essential matrix[END_REF])

E = [t] × R, (3.8)
where [t] × denotes the cross product matrix with translation vector t = [t x , t y , t z ] and can also be written as

[t] × =    0 -t z t y t z 0 -t x -t y t x 0    .
(3.9) where the columns of U and V are 3 × 3 orthogonal matrices which consist of the left and right singular vectors. Here, W is defined as

W =    0 ±1 0 ∓1 0 0 0 0 1    . (3.11)
There are two possibilities of rotation matrix, R value as

R 1 = U W V and R 2 = U W V .
However, via triangulation, only one solution is feasible from the condition called cheirality constraint and an efficient decomposition of E into R and t is further described in [START_REF] Nistér | An efficient solution to the five-point relative pose problem[END_REF].

Translation matrix can then be computed from the obtained R as shown in 3.12.

   x k y k z k    =    x k-1 y k-1 z k-1    +    cos θ k -sin θ k 0 sin θ k cos θ k 0 0 0 1       t x t y t z    . (3.12)
Then, output trajectory of the visual odometry for the current image sequence I k is presented as

X k = X k-1 + R k t, (3.13)
where the rotation matrix R k is updated from the relative rotation matrix of each image pair as

R k = R k × R k-1 . (3.14)
Here, we implemented a conventional visual odometry approach without accuracy optimization with the objective of obtaining the trajectory. We assumed that even without proper optimization, the trajectory curve has enough information to describe the position estimate of the vehicle with data fusion.

OSM Data Structure

Map-matching is a process to match a sequence of real-world coordinates into a digital map.

Its main purpose is to identify the correct road link on which the vehicle is moving and to determine the vehicle location on that link [START_REF] Quddus | A High Accuracy Fuzzy Logic Based Map Matching Algorithm for Road Transport[END_REF]. Map-matching is not only able to localize the vehicle position, but also improves the positioning accuracy as long as good spatial road network data are available.

The OSM uses a topological data structure with four core elements:

• Nodes: points with a geographic position coordinate stored in latitude and longitude according to WSG84.

• Ways: ordered lists of nodes representing a polyline, or a polygon if they form a closed loop. They are used to represent linear features such as streets and rivers, and areas, like forests and lakes.

• Relations: ordered lists of nodes, ways and relations. They are used for representing the relationship of existing nodes and ways such as turn restrictions on roads, routes that span several existing ways, and areas with holes.

• Tags: key-value pairs (both arbitrary strings). They are used to store metadata about the map object properties. Tags are always attached to a node, a way or a relation.

These ways contain node reference to locate the coordinates of road connection and tags that has the road specification. In our proposed method, other map attributes are neglected to focus on road network to be rendered by using OSM Mapnik rendering rules so only ways of roads labelled as tags with key=highway are extracted. These highway in map data is specified with information that may include road name, type, number of lanes, allowed directions, maximum speed, and even the road surface material.

An example of ways formed by connected nodes is shown in Figure 3.3 with its data structure recorded in an .osm file. Nodes are assigned unique ref each as reference number and the ways can be specified by their respective ids. Within the way description, metadata is stored in tags that contain abundant information of the way.

The first step in our approach for fusion with OSM data is by filtering only the road network and use Mapnik rendering style to estimate the road width on the map. Then, mask image is created from the rendered map to determine road and non-road area on map.

With this, the probability factor when the particle falls within the road area, is defined as α i = 1. Otherwise, the value falls exponentially with distance from road boundary, d r and calculated as α i = e -dr .

Fusion Validation of VO with GPS and OSM

The conventional VO approach is our main input for state estimation during prediction for our fusion technique. The trajectory obtained from VO is fused with noisy GPS data to obtain first estimation of vehicle pose on map. Then, further correction on the state estimation is conducted based on the road width information rendered from the OSM. The fusion is expected to be able to reduce VO drift and increase localization accuracy at the same time.

KITTI Dataset

In this work, KITTI dataset [START_REF] Geiger | Are we ready for autonomous driving? The KITTI vision benchmark suite[END_REF] was utilized to validate the efficiency of data fusion between VO, GPS and OSM. The database contains sequence of images along with data from velodyne sensor and GPS. Since the GPS data is obtained from a high precision device that has up to 0.01m accuracy, it is treated as the ground truth in method validation.

To test the ability of our method to deal with noisy low-cost GPS positions, we demonstrated the random noise of about 10m, added to the ground truth data with 1Hz of frequency. Short sequences were used to observe the accuracy and precision level of our proposed approach in lane-level localization.

Our proposed approach was performed on Dataset 1 on 400 images to verify the improved accuracy of visual odometry and noisy GPS fusion result on a straight road with double lane and two-way characteristics in Dataset 1 and a multilane road that diverged to a curved road in Dataset 2. Trajectory output was compared with ground truth data to acquire the absolute distance error. In the results section, localization accuracy and precision is defined by distance error mean and standard deviation, respectively.

Data Fusion by Particle Filter

Particle filtering method serves the purpose for performing inference interstate-space models where it represents the posterior density in terms of random samples and associated ii Prediction: For i = 1, ..., N, particles are displaced according to the VO trajectory vector (Equation 3.12) transition equation, f k with dynamic noise, v which is random and independently calculated for each particle i. This will disperse the sample movements to find the probability in a wider area.

x i k = f k (x i k-1 , v i k ) (3.16)
iii Weight update: Using GPS data and particle position on the road as the new measurement vector, weight of each particle is updated according to probability density function as

w i k = w i k-1 α i e -d 2 2σ 2 σ √ 2π , (3.17)
where

• d : relative distance between particle x i k and GPS coordinate, • σ : coefficient value of the probability distribution determined from GPS error variance in the system,

• α i : particle position whether inside/outside the road area.

Before resampling, the probability distribution function particle weight is normalized to wi k so that T w = 1.

Improvement

iv Resampling: Number of effective sampling size is calculated and if N ef f > N thres , resampling occurs. During the resampling step, we utilize the most common resampling method called 'select with replacement' by duplicating particles with higher weight and removing particles with negligible weights [START_REF] Hoover | Lecture Notes: Particle Filter[END_REF]. Firstly, the particles are sorted according to its importance weight. Then, the resampling algorithm is shown as follows:

Algorithm 1 Resampling steps

1: Q = cumsum(W );
# cumulative sum of particle weights 2: X = sort(x, w); # particles are sorted as an array, based on weights

3: i = j = 1; # array starts 4: while (i <= N ) do 5: if (X[i] < Q[j]) then 6:
index[i] = j;

7:

i = i + 1; 8:

Results and Discussion

Firstly, we observe that the visual odometry trajectory as shown in Figure 3.5a suffered from drift error, due to the inaccurate scale estimation. This was expected since no optimization was done on our visual odometry technique. However, we can see that the drift was greatly reduced with the fusion of GPS and OSM as illustrated in Figure 3.5b, despite the trajectory drift after frame 200. The distance error achieved for this dataset was (2.619±1.397)m.

The same test was conducted to a different dataset (Dataset 2) that has highway road with multiple lane and vehicle movement behaviour of turning into a diverged exit as shown in Figure 3.6a. Here, our visual odometry contained severe error that mainly affected the trajectory scale on the curved road. Besides, there was also an instant of noisy visual odometry trajectory due to the shadow and lighting condition around frame 75 to 79 but since it only occurred within a short period, this did not affect the fusion performance. The distance error comparing visual odometry and fusion output of GPS and visual odometry is displayed in Figure 3.6b, whereas the error mean and standard deviation obtained for our data fusion was (3.608 ± 1.968)m.

Road Distribution Probability Factor

Fusion with OSM Data

As mentioned in Section 3.3, OSM provides map data including road information such as road type, number of lanes, direction and maximum speed limit. Meanwhile road width Improvement is either specified in the .osm file itself or predefined by the rendering rule. This rule is constantly updated and the road width is defined according to the specified road type. Previously, the validation results were generated to exhibit the localization improvement of fusion between noisy GPS, conventional VO, and road boundary from OSM. Next, we extract number of lanes and road direction information from OSM for Road Probability Distribution Factor (RPDF) to improve positioning estimation. Generally, most of the roads are categorized into several types:

i. One-lane road that always has one way direction.

ii. Two-lane road: can be one way direction or not (bi-direction).

iii. Three-lane road or more: can be one way direction or not. (Lane direction distribution is provided by OSM)

Firstly, particles are defined to be on the road if they fall within the boundary line of roadside on the rendered map, and then confirmed true if its accumulated lateral distance between left and right boundary line is less than predefined road width. The road details will be the main contribution in determining the RPDF along with the vehicle position initialization. Then, the RPDF, represented as α, for each event is illustrated in Figure 3.7 and defined as:

α A = P (R|A) = m, (3.18
)

α B = P (R|B) = 1 -m, (3.19)
where m is the factor for the particles on the lane with rthe ight direction.The classification of each event is described as:

• event R: particles are on the road,

• event A: particles are on the road with the right direction, and

• event B: particles are on the road but in opposite direction.

If the road has two lanes that are not 'one-way', m value is set to be 0.8 as the probability on the road with correct direction. This means that we do not ignore the manuevering probability during the drive. For multiple lane roads (ie. highway), the RPDF is estimated further to improve the accuracy of being in the correct lane. Suppose L is the total number of lanes with the same direction on the road, for n = 1, ..., L, the probability factor for each lane in similar direction is When the particle position x i k is on lane n, assuming true lane l t is known, the lane difference from the true lane l t is defined as l;

P (R|A) = L n=1 P (R|A n ). ( 3
l = |l t -n| + 1, (3.21)
and finally the RPDF for each lane with similar direction can be computed as

α An = P (R|A n ) = m l+1 L-1 l=0 m l . ( 3

.22)

While P (R|B) is as defined in Equation 3.19, for multiple lane cases the probability factor P (R|B n ) for each lane n on opposite way will be defined as

α Bn = P (R|B n ) = 1 -m L . (3.23)
During the estimation step, RPDF factor α is multiplied to w i k to increase weight of particle that has more likelihood to be on the correct road lane.

w i k = αw i k . (3.24)
Meanwhile, for particles identified to be outside the road, weights are decreased exponentially as in Equation 3.17 according to its distance from the road as illustrated in Figure 3.7.

Method Validation

Localization performance evaluation was conducted on the same datasets as the previous validation to observe the positioning improvement. For fusion with OSM, the data was extracted beforehand and road networks were filtered where other information other that roads were abandoned. Then, specific useful details of the road were stored such as number Chapter 3. Data Fusion with Road Probability Distribution Factor for Localization Improvement of lanes, road type, allowed directions and speed limit. In the Dataset 2 where it contains multiple lane road, an assumption of initial was made to observe the localization accuracy after the RPDF implementation which includes probability of neighbouring lanes. From the results of Dataset 1 (Figure 3.8b), in overall, VO fusion with RPDF has relatively lower error than the others. However, between frame 123to 126, the result shows error spike for the proposed method. This is due to the existence of road divergence which affected particles weight tendency to be on the other road although the vehicle did not intend to turn. This instant is illustrated in Figure 3.9. As for the distance error, mean and standard deviation obtained for the fusion trajectory result was (2.619 ± 1.397)m. When RPDF approach was applied to the same sequence, we managed to improve its position error to (1.636 ± 0.851)m. This is about 38% improvement for the error mean and 39% improvement for the standard deviation. These results are summarized in Table 4.4 together with results achieved in Dataset 2. Despite the fact that the improvement percentage is lower than sequence with simpler road in Dataset 1 due to RPDF influence from other lanes in the same direction, the outcome is still better than localization results with other methods. Besides, our RPDF effects on multilane road almost achieved lane-level accuracy where it managed to remain on the correct lane except for two occurrences around frame 20 and 40. During these instances, the noisy GPS data were found too far from the true position that made particles on the neighbouring lanes had higher importance weights.

Results and Discussion

Synthesis

Our research had proposed visual odometry fusion of GPS, aided with map information and RPDF to improve vehicle global localization. The approach combined a conventional visual odometry system fused with GPS and road information provided by OSM to determine the Improvement probability factor for each lane and manipulate particles weight to get more accurate localization. This method had been evaluated and compared with the particle filter which showed localization improvement in terms of accuracy and precision from the error analysis. Although road width was predefined by rendering rule which is not guaranteed to be error-free, this can further be improved with the crowdsourcing OSM that constantly provides up-to-date data. Besides, a consistent lane-level accuracy has yet to be achieved due to the influence of severe GPS noise in some instances. Thus, a balance of GPS effect and particle effective sampling size need to be considered to achieve consistency in lane-level localization. Lane-keeping and lane-changing behaviour detection may also be useful in this case to avoid positioning error on the incorrect lane.

The outcome of this fusion technique shows that visual odometry trajectory can be an

interesting key input of a data fusion for vehicle localization. This is because despite the drift and scale error of our conventional method, the localization accuracy and precision can be increased with data fusion. With the fusion from OSM map data, the localization is constantly on the road, without requiring map building or road lane detection by image processing. From the error that occurred at road divergence, it might be interesting to utilize the characteristic of visual odometry trajectory to compare with road shape on the digital map. This would avoid discrepancy in vehicle route planning that affect the localization accuracy itself. Besides, the localization accuracy is expected to be further improved if tested in more mature visual odometry methods. In the next chapter, we will explore more on this possibility with longer and different datasets.

Chapter 4

Robust Localization Using VO-OSM Segmented Curve Matching

Overview

In the previous chapter, the information from OpenStreetMap (OSM) database has been used for lateral error correction by utilizing the predetermined road width to alter particle's weight distribution factor namely RPDF. This chapter proposes an improvement strategy for longitudinal error in vehicle localization. Hence, with the map data provided by OSM, our method exploits this information fused with VO trajectory and low-cost GPS without utilizing additional sensor.

In localization, position difference in lateral axis (left / right of the road) is described as lateral error, lat and longitudinal error, lon is the distance difference in longitudinal axis which makes the localization to appear ahead or behind the ground truth data (Figure 4.1).

While lateral error can easily be minimized by considering the road width factor, longitudinal position error has been a challenging task in localization especially because most of the time vehicle moves in a forward heading (longitudinal) direction, unless it turns into a junction or curve. It is not less crucial than lateral error, while the latter might affect in precise lane localization, longitudinal error also plays an important factor to determine its accurate position and whether or not the vehicle has passed certain features while moving forward.

Generally, there are two strategies in order to minimize positioning error. Firstly, by drawing a certain limit or boundary as a cutoff area for the most unlikely placement of the state -just like lateral error correction that utilizes road width boundary [START_REF] Hara | Vehicle Localization Based on the Detection of Line Segments from Multi-Camera Images[END_REF]FIGURE 4.1: Localization error [START_REF] Lu | Lane marking based vehicle localization using particle filter and multi-kernel estimation[END_REF][START_REF] Tao | Lane marking aided vehicle localization[END_REF][START_REF] Vivacqua | A Low Cost Sensors Approach for Accurate Vehicle Localization and Autonomous Driving Application[END_REF]; or secondly, by recognizing landmarks and match them into the map to obtain the true current position [START_REF] Hata | Robust curb detection and vehicle localization in urban environments[END_REF][START_REF] Mattern | High-accurate vehicle localization using digital maps and coherency images[END_REF][START_REF] Suhr | Sensor Fusion-Based Low-Cost Vehicle Localization System for Complex Urban Environments[END_REF]. Approaches have been proposed that suggested using road lane markings, stop markings, curb curve matching and so forth, but there are pros and cons of these methods which we will present in the next section.

In this chapter, a segmented curve matching (SCM) method is proposed that compares VO trajectory trail with road curve segments from the digital map to reduce the longitudinal error and simultaneously increase localization accuracy and precision. The ability of improving localization is studied with common GPS noises. It is a novel approach for curve matching as it does not require a graph-based representation of the map and the curve comparison is simplified into 'node-trios' (sets of three nodes) instead of integral calculation of the whole curve. Besides, it also does not rely on advance image processing technique to recognize any landmarks or road lanes which makes the computation even easier.

This chapter will explain how we came up with this strategy and its validation, which is organized as follows: Section 4.2 introduces previous approaches in multisensor data fusion that addressed longitudinal error evaluation, followed by Section 4.3 that presents our method in VO-OSM curve similarity matching in localization. Then the method validation will be explained in detail in Section 4.4.1 with the results evaluation and analysis. Finally, Section 4.6 concludes this chapter.

Related Works

Researchers have conducted studies on localization strategies for longitudinal error compensation. Among the most common approach is by using visual sensors to detect road markings such as lane marks, stop lines, and symbolic marks. Localization optimization using lane marks detection methods propose detecting and tracking white lines on the road to estimate lateral position of the vehicle and for global localization [START_REF] Hara | Vehicle Localization Based on the Detection of Line Segments from Multi-Camera Images[END_REF][START_REF] Lu | Lane marking based vehicle localization using particle filter and multi-kernel estimation[END_REF][START_REF] Tao | Lane marking aided vehicle localization[END_REF][START_REF] Vivacqua | A Low Cost Sensors Approach for Accurate Vehicle Localization and Autonomous Driving Application[END_REF], some works performed matching with satellite imagery of digital map [START_REF] Mattern | High-accurate vehicle localization using digital maps and coherency images[END_REF]. Although this method is most widely used for vehicle localization and while it works well for lateral position correction, the information is insufficient for longitudinal localization. This occurs especially when the specific features are not available on digital maps used for matching and obstacles during detection for accurate line matching.

Then, another method is by using stop lines and crosswalks for longitudinal error correction by restraining trajectory drift of inertial sensor [START_REF] Hata | Robust curb detection and vehicle localization in urban environments[END_REF]. The researchers only performed image processing during curve lane and stop line detection, thus it can save the computational power. Experimental results showed satisfactory localization performance of submeter accuracy but this method required in-depth road line information from pregenerated map produced by 3D lidar for curve and stop line matching. Hence, this is not feasible for global localization and consumes high cost sensor. Besides, stop line only exists at intersections, thus it only corrects the longitudinal position after some period of time.

For a more cost-effective approach, [START_REF] Suhr | Sensor Fusion-Based Low-Cost Vehicle Localization System for Complex Urban Environments[END_REF] proposed fusion of low-cost sensors -GPS, IMU, speed sensor, camera -with digital map for vehicle localization by applying symbolic road markings detection and matching with map imagery. It is an interesting approach since although the detection was less frequent than lane line detection, it was enough to generate accurate position without requiring the vehicle to be at intersections for stop line detection. However, this approach is restricted to the availability and clarity of the symbolic road markings (ie. highway roads have less symbolic marking) to be detected, and also satellite imagery could not provide information for matching in cases of the road is out of view (ie. tunnels or multilayer roads). Therefore, although it seems to be the most practical solution for longitudinal error correction, road markings detection methods prove to have a lot of limitations to achieve robust localization in various circumstances.

Multiples researchers have also proposed data fusion techniques for accurate localization without using road markings detection. For instance, [START_REF] Brubaker | Map-Based Probabilistic Visual Self-Localization[END_REF] had demonstrated an outstanding performance of map localization in 'kidnapped scenario' (unknown initial position) by only matching visual odometry curve with road network from digital map. Without any GPS data, they generated a probabilisitic map based on the travelled visual odometry trajectory and while the vehicle turns into intersections, the probability of the matched roads increases. Most of the experiments evaluated on KITTI dataset managed to localize the vehicle after a few seconds with average accuracy of 4m. Unfortunately, the system was unable to localize itself in some cases where the sequences were fundamentally ambiguous. Besides, since the information source was solely on visual odometry and digital map, failure in any of these inputs will severely affect localization performance. Hence, GPS data -albeit noisy -is required especially to provide global position of the vehicle.

Zeng et al., 2016 on the other hand implemented trajectory curve matching between GPS and road curve in map where they introduced curvedness feature constrained map matching method to retain the curveness feature in GPS tracks. This method however mainly focused on curve matching technique in localization and does not present its contribution in reducing positioning error. In addition, GPS data typically contains unpredictable noise depending on the receiver type which makes it impractical for coupling GPS tracks with road map directly for accurate localization.

Therefore, we came up with the idea of assisting GPS, and digital map data fusion with curve matching from VO trajectory for more accurate vehicle localization with the hypothesis that VO curve matching will be able to reduce longitudinal positioning error. This method does not require image processing on road lines, thus is not restricted to out-ofview of vehicle speed limitations. GPS data behaves as a global position reference and limit the curve matching searching boundary. Further details of our proposed approach will be described in the next section. The algorithm of our proposed system is illustrated in Figure 4.2. This method mainly consists of three types of input for data fusion; stream of images from camera, noisy GPS and a digital map. Firstly, the images are processed to generate trajectory based on VO that detects and tracks moving features. It is fused with the low-cost GPS data to estimate the global position. Then, applying our previous approach, road information is extracted from the map mainly for lateral error correction by road probability distribution factor (RPDF).

System Outline

Then, our method is enhanced by utilizing the result of VO and GPS fusion to act as a reference point for road segments curve estimation in the map with weight assigned based on its distance from the centre and similarity with VO curve. From VO trajectory output, a specified length of latest trajectory curve is compared with road segments from the map as a weighing factor for filtering stage during the next step of data fusion. To observe the localization robustness of our proposed method, different types of noise such as random noise, biased noise and an instant where GPS signal is lost are simulated.

VO and OSM Segmentation

In our approach, we utilize the last recent segment of visual odometry to match with road curve in OSM to find the most probable road candidates within search area. Instead of using the road curve line from the map for matching purpose, we simplify the curve matching by using node-trios provided by OSM. As explained in previous chapter, road networks in OSM consist of sets of nodes that are connected as ways. The distances in between the nodes are not fixed in specified length; the nodes are distanced further in straight roads and closer in curved roads as shown in Figure 4.3 (nodes are represented by yellow dots). In short, the nodes are spaced according to the road's shape linearity. Although at first the distance inconsistency seems to be a problem in point-to-point curve matching with visual odometry trajectory, it is in fact a merit point. This is because our visual odometry trajectory curve snippet is time-based, hence the distance travelled is proportional to the vehicle speed.

Since vehicle speed is decreased in curved roads, this will result in shorter trajectory and the shorter node-trios in curve area agrees to this situation and vice versa.

In order to match trajectory curve with map, the knowledge on curve length, orientation, and heading variation is required to optimize longitudinal positioning accuracy. Since OSM nodes are spaced unequally, we need to define the distance threshold between nodes to obtain node-trios with similar length with VO curve for the last m poses, especially in the events where the nodes are too distant on a straight road. This algorithm is illustrated in 4.2. In order to obtain the distance in metre, vehicle speed (km/h) is converted by multiplying 10 3 3600 . Since we utilize sets of three nodes to define OSM curve, the distance threshold between two nodes is divided by 2.

d vo = 1 2 k i=k-m (x i -x i-1 ) 2 + (y i -y i-1 ) 2 . ( 4
n m = d n d th + 0.5 × d th .

Curve Matching

The length of node-trio curve and VO travelled curve distance is not exactly equal all the time, but the difference is small (less than 0.05%) and negligible. To compare the curve similarity between fragment of VO trajectory curve and nodes of roads on OSM, the computation of curve similarity score, S

cs relies on two key parameters: initial orientation θ in k and curve heading variation ϑ k . Parameters for VO trajectory trail are defined as

θ in V O k = θ k-m , (4.4
)

ϑ V O k = k i=k-m 2 (θ i+1 -θ i ). (4.5)
Three nodes curve from OSM data is illustrated in cs of road segment for node n is obtained from the difference of initial orientation θ in between nodes and VO curve and its heading variation ϑ which can be written as

S (n) cs = |θ (n) -θ in V O k | × |ϑ (n) -ϑ V O k |. (4.8)
The curve matching steps from VO trajectory curve fragment to surrounding nodes are illustrated in Figure 4.6 where it starts from:

(a) extraction of VO trajectory trail since the last m poses, followed by (b) connected nodes in ways are detected within searching area from lookup centre as predetermined and explained in method validation (Section 4.4.1);

(c) then, the nodes are paired with neighbouring nodes to form 'node-trios'. All sets found within the area are compared with VO trajectory curve to obtain the most similar road segment to be matched. This approach allows the curve matching to be conducted in small fragments to obtain the most likely candidate of road segment and also reduce computational cost. This is because by assessing only small fragments of trajectory within short travel periods, the resulted curve is limited to 2nd order polynomial regardless of the overall road curve complexity. Generally, only double bend roads (as in Figure 4.7) will result in 3rd order polynomial curve or above. Still, since there is a minimum radius limit for each road curve [START_REF] Kilinc | Determination of Minimum Horizontal Curve Radius Used in the Design of Transportation Structures[END_REF] and we are only assessing the last small fragment of pose sequence, the trajectory curve will not be long enough to exhibit 3rd order polynomial curve.

Multilevel Data Fusion

Here, we used a cascaded scheme for probabilistic vehicle pose estimation by implementing particle filter (PF) technique to integrate information inputs for fusion. PF uses random samples (particles) to represent the posterior density of vehicle position in a dynamic state estimation framework such as a road network. With PF, we estimate the instate-space models inference by expressing the posterior probability density in terms of randomized particles associated with importance weights.

Our data fusion consists of three levels which are described as illustrated in Figure 4.8

with respective associated data and information used in each level. With the increasing The first level of fusion between GPS and VO is similar to the method presented in Chapter 3. Through this level, the rough estimation of vehicle initial position on the map is obtained.

This time, the study on system robustness is conducted by implementing different types of typical GPS noise that is explained in Section 4.4.1. We also utilized the conventional VO approach as described previously.

2nd Level: Lateral Position Correction by Road Probability Distribution Factor (RPDF)

Similar to our previous approach, we still use the road information obtained from OSM and RPDF to assign particle weights accordingly. The output results of the fusion, X In order to distribute the segmented roads probability factor with longitudinal accuracy optimization, new nodes are generated in between of nodes that are too distant with each other as explained in 4.3.1 with distance threshold value d th and the number of new middle nodes to be generated n m .

B. Step 2: Curve Similarity Score

To simplify calculation and improve longitudinal accuracy, the trajectory length of VO for the last m poses is set to be short and can only form a second order polynomial curve.

We compare the curve segment similarity between VO trajectory curve and node-trios with the computation of curve similarity score, S (n) cs as shown in Equation 4.8.

C. Step 3: Fusion

In the last fusion level, probability factor is assigned to each road segments constructed by node-trios for all n'th nodes. This probability factor, denoted as γ cs of the node-trio is also considered. Higher S (n) cs indicates higher similarity between both curves and this will yield an increased probability factor of the road segment, γ

(n) k . Assuming d c is the distance between X (C) k and node n, γ (n) k is calculated from Gaussian distribution as γ (n) k = e -1 2 dc σScs 2 σ √ 2π , (4.9)
and the importance weight for each particle i is updated as

w (i) k = w (i) k-1 γ (n) k , (4.10)
where n is the nearest node found from particle i.

With this approach, we will be able to obtain the best road segments candidates from the node-trio probability especially on curved roads. Although it is difficult to optimize longitudinal error on a straight road without heading variation, our fusion technique will at least limit the longitudinal error from the probability distribution of candidate ways within the lookup range. This is illustrated in Figure 4.9.

D. Step 4: Resampling

In order to eliminate particles with low importance weight, resampling is required as the final step in particle filter before repeating the pose estimation steps as above again.

The resampling steps are conducted similar to the algorithm as presented in the previous chapter.

Validation and Results Discussion

Method Validation

Our approach with SCM was validated by using KITTI dataset as the benchmark for localization by VO. We tested our method to a highway drive found in sequence 01 and residential drive (sequence 02) for longitudinal error correction while at the same time observe the localization performance in path selection during road divergence. This is especially critical in highway drive where the vehicle speed is faster than those in residential or city area that makes it more challenging to detect accurate path selection simultaneously due to small heading variation. Then, to test the localization robustness, different noise models were applied in both sequences such as an extreme random noise of up to 10m, biased GPS noise and also an event of GPS signal loss.

In this work, we implemented m = 20, which means only the last 20 poses (2s of VO trajectory with frame rate 10Hz) was considered as the curve fragment. The curve length will differ according to vehicle travel speed and this corresponds to the node-trios that are distanced based on threshold calculation from vehicle average speed or VO curve. In the case of VO trajectory with severe scale ambiguity, speed can be estimated from the road speed limit provided by OSM data. The pre-set m value was determined based on tradeoff consideration between accuracy and computational cost. In order to obtain accurate longitudinal position, the road segments should be as short as possible.

However, if we set m = 10 (last 1s of VO trajectory), most of the resulting segments would be mostly found as straight trajectory trail. For instance, assuming the vehicle moves at 30km/h, 1s of displacement is only about 8.3m in length. If it is a curve road, considering the least degree of a significant curve is 10 • , this would result in a minimum radius of 55.6m

for the road. However, according to [START_REF] Kilinc | Determination of Minimum Horizontal Curve Radius Used in the Design of Transportation Structures[END_REF], the minimum road radius with the speed limit of 30km/h is 60m. Therefore, this is sufficient for the curve analysis where the minimum curve length would be 10.5m. Besides, this would degrade the efficiency of finding the best road segment candidate with the lack of information in vehicle heading change. Hence, we concluded that 2s of travel distance is ample to obtain accurate curve segment for analysis. On the other hand, if the m value is increased to 40, the trajectory curve length is longer and the distance between nodes will increase. As a result, longitudinal positioning would be less accurate with longer road segment containing similar probability distribution.

As for the search area size, candidate ways are searched within a specified radius of about 30m from the lookup centre X (C) k since we have an approximate location from the noisy GPS of up to 10m accuracy. At the same time, we need to consider the possible bias or blunders, therefore additional 20m as a buffer zone was also considered in this study. Further distance was not considered since it might result in unnecessary road segment probability appearance in ambiguous road network. Besides, expanding the search area to a larger size would only increase computational cost with a greater number of nodes found in the area for probability calculation. Therefore, we fixed it to 30m while observing the effects of lost GPS signal on localization performance.

A. KITTI Dataset

Even though KITTI dataset contains many sequences in different types of environments, only sequence 01 exhibits an event where the vehicle entered road divergence with high speed at a highway exit. Other sequences have other scenarios such as environment with lots of shadows, traffic jam, city roads with lots of other moving vehicles, and roads with pedestrians and cyclists. In our study, we are particularly interested in this sequence. The vehicle was driven on a highway which then exited with the speed of around 60km/h.

Besides, it is reported that this sequence had always been the one with highest translation error from the works presented in [START_REF] Cvišić | Stereo odometry based on careful feature selection and tracking[END_REF][START_REF] Engel | Large-scale direct SLAM with stereo cameras[END_REF][START_REF] Mur-Artal | ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras[END_REF][START_REF] Qu | Landmark based localization in urban environment[END_REF][START_REF] Wang | Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras[END_REF]Zhu, 2017) compared with other sequences. In fact, some of the methods were not able to perform localization on this sequence due to the typical argument of being a highway dataset and vehicle moving at higher speed resulted in higher translation error. Therefore, the purpose of our study is to focus on optimizing localization on this specific sequence. At the same time, dataset 02 with a drive recorded in residential area with many intersections was also chosen to observe general improvement of localization accuracy.

B. GPS Noise Modeling

To test the robustness of the proposed system, we modelled three types of noisy GPS as one of the fusion inputs during method validation. To achieve this, random noise was added to the RTK-GPS data which is downsampled to 1~2 Hz of frequency. Albeit GPS signal noise varies based on the receiver device specification and surrounding environment, it generally can be characterized as a combination of flicker noise and white noise [START_REF] Mao | Noise in GPS coordinate time series[END_REF].

Thus, we initiated the noise with specific bias and variance with 2D Gaussian distribution that represented the white noise and a coherent Perlin noise was added to simulate the flickering noise behaviour. The noise is generated from equation ( 4.11) below, assuming (4.11) where ε G ∼ N (µ, σ 2 ) is a random Gaussian noise with mean µ and standard deviation σ.

that [x RT K k , y RT K k , z RT K k ] is the ground truth position, noisy GPS [x N k , y N k , z N k ] is defined as    x N k y N k z N k    =    x RT K k y RT K k z RT K k    + ε G    cos θ p sin θ p 0    ,
This Gaussian noise ε G acts as the white noise amplitude and θ p is generated from Perlin noise. Meanwhile, bias error in GPS can be a result from Selective Availability (SA) or some other factors. SA is the intentional degradation of the standard positioning service (SPS) signals by a time varying bias that introduce delta error resulted from dithering the satellite clock which is a common error to all users and epsilon error which is a varying orbital error.

Since SA is a changing bias with low frequency (excess of a few hours), position solutions or individual satellite pseudo-ranges cannot be effectively averaged over periods shorter than a few hours. Other factors that results in bias error include:

(a) ephemeris data errors that contains imprecise orbital data, (b) tropospheric delays which are affected by the atmospheric pressure, temperature and humidity experienced in the troposphere layer of the atmosphere, (c) unmodeled ionosphere delays where the ionosphere layer bends the GPS radio signal and changes its speed as it passes through the layer, and

(d) multipath effect caused by reflected signals from surfaces near the receiver that cause interference or be mistaken for the signal that follows the straight line path from the satellite. This source is difficult to detect and avoid.

We simulated the bias noise by shifting the GPS position in global lateral position from the ground truth and add Gaussian noise with the same 10 metres variance. This would result in higher tendency of events where the GPS data falls into the road with wrong direction or outside the road boundary.

Lastly, blunders is also one of GPS error sources where it can result in hundreds of meters of error due to user mistakes, including incorrect geodetic datum selection. This kind of large error can also be resulted from out of date GPS data due to signal unavailability resulted from blockades. To model this noise, the GPS data was turned off for a specific time range which made the localization can only rely on information from VO and map.

Results and Discussion

Sequence 01: Highway Drive Three different noise models were applied on GPS data to imitate low-cost GPS and tested on a subset of sequence 01 to emphasize on path selection event. The localization compares the performance of GPS and VO fusion, then with RPDF, and finally with the SCM. The trajectory results for each GPS noise are shown in Figure 4.11.

A. Positioning error results

We specifically chose a subset of drive in sequence 01 to observe lane changing behaviour and localization performance from curve matching method. The test was run on 200 frames with the total distance of 340m and tested with three types of GPS noise. For the highway drive during GPS with random noise (both unbiased and biased cases), it can be seen from Figure 4.12a and Figure 4.12b that the longitudinal error was improved significantly. This took place in the first 70 frames when the vehicle was driven on a straight road and facing road divergence. Regardless of the biased applied to the GPS noise, our method with curve matching proved to be unaffected by the noise. We are also able to select the best candidate road segment that matches with the true trajectory.

As for the lateral positioning error, there was a brief improvement in random noise case especially during the road divergence in between frame 45 to 55. During this time, it showed that the lateral error tested in biased noise was significantly improved, as well as its longitudinal error.

Meanwhile, in the case of lost GPS signal, as depicted in Figure 4.12c, our previous approach without road segment curve matching suffered horrendous positioning error.

This is as a result of the fact that the data fusion lost one of its important factors and it solely relied on the last GPS position and VO displacement. In fact, the localization failed to relocate its position even after the GPS signal was recovered as can be seen from the results due to the travelled distance was too far from the 'lost' particle cloud. However, with the curve matching approach, it was hardly affected by this problem owing to the curve matching of VO trajectory and road segments that enabled the localization to detect the road segment with the most probability. The longitudinal error result shows that our method managed to maintain the error below 5m for the whole drive in opposed to the error of the previous method that reached up to 20m for both lateral and longitudinal error.

The results of distance and positioning error for the compared approaches is summarized in Figure 4.13 for highway drive sequence in all noise types. In the random noise test, our proposed approach for sequence 01 showed more significant improvement in error reduction of (34.0±21.2)% in lateral error and (33.7± 30.5)% in longitudinal error.

This improvement is owing to successful path selection in diverging road. Meanwhile, we managed to achieve further improvement in biased noise where both datasets were able to reduce lateral error of (59.1±41.9)%, while the longitudinal error is reduced to (54.6±43.9)%. Finally, for the lost GPS test, the RPDF method suffered worse error than the conventional GPS and VO fusion. However, with road curve matching, we successfully maintained the localization performance with average positioning error that was constantly below 3m.

B. Path selection response

As for the path selection accuracy, we evaluated the performance of our SCM based approach according on the moment localization is corrected to the diverging road which is the true trajectory in this dataset compared with RPDF method (Awang [START_REF] Salleh | Swift Path Planning: Vehicle Localization by Visual Odometry Trajectory Tracking and Mapping[END_REF]. Since GPS and VO fusion based localization did not include road probability factor for path selection, it is not valid for comparison. In the random GPS test, our curve matching technique manages to localize itself simultaneously (it entered the diverged road in frame 32) with the ground truth while the RPDF method only corrected itself at frame 58. This was even worse when biased noise was applied to the GPS position where it simulated vehicle position to be on the straight road instead. Thus, it affected path selection performance in RPDF approach by only correcting the position at frame 71. On the other hand, the SCM showed an improved performance where it was unaffected by this as can be seen in Figure 4.11b.

Lastly, during lost GPS signal, the proposed method with curve matching outperformed the RPDF approach where the latter approach was unable to update localization without GPS data as reference while being restricted to localize on road area only. Even once the GPS signal was recovered, the particle cloud was too far from the true position due to the high speed. Interestingly, for the GPS and visual odometry fusion, the localization was less severe due to the non-existence of RPDF limitation localization on the road area.

Sequence 02: Residential Drive

Similarly, three different noise models of a low-cost GPS were tested on a subset of sequence 02 which is a residential drive sequence. The localization performance of GPS and VO fusion, then with RPDF, and finally with curve matching method were compared to study the improvement of each methods. The trajectory results for each GPS noise are shown in A Positioning error results

Our second test was conducted on a residential area for 950 frames with the total travelled distance of 924m and error results are shown in Figure 4.15a,4.15b,and 4.15c. During the random noise test, it did not show remarkable improvement in positioning error because both RPDF and SCM approaches have the similar basic approach in localization;

unless it involves severe GPS error that affects path selection. Since the vehicle speed in residential drive is lower and the intersections are easily distinguished, the localization showed a consistent performance even without curve matching method. However, in overall, our proposed approach still managed to reduce positioning error compared with other methods in some instances where the RPDF longitudinal error peaked up to more than 5m.

In the biased error test for residential drive, the results as displayed in Figure 4.15b is particularly interesting for the amount of error degradation in the result of method without curve matching in three events. The first event was during frame 70 to 120 when the vehicle moved along a straight road and there was a junction on the right where most GPS points were positioned due to the bias applied; the RPDF method accidentally assumed that the vehicle turned into the junction. While the second event happened around frame 520 to 590, where the vehicle was moving on a sharp curve that resembled making a U-turn twice, thus resulting the biased GPS fall into the previous lane. As a result, the localization mistakenly concluded that the vehicle had made a U-turn. On the other hand, our proposed approach with curve matching managed to cope with these situations by integrating the VO curve with road curve fragments for all occurrences.

Lastly, we simulated an instance of GPS signal loss to the residential drive and the results obtained are shown in Figure 4.15c. The GPS data was turned off in between frame 400 to 700 when the vehicle was moving along a curved road with multiple intersections on its sides, turning into an intersection and it travelled on a sharp curve resembling a U-turn. As shown in the error graph, it is clear that the recent SCM based approach managed to obtain a consistent low positioning error and unaffected by this lack of data compared with the results of previous fusion method. The RPDF method suffered severe error of over 40m during the signal loss, but it was able to recover the localization after two attempts. This is due to the slower vehicle speed and road curves, thus the travelled distance during GPS signal loss is still reachable by the particle cloud, unlike sequence 01.

Localization error summary for the residential drive is illustrated in Figure 4.16. The distance, lateral and longitudinal errors show declining trend except when biased noise and signal loss were applied to the GPS. This is where the RPDF approach performance was deteriorated in comparison with the conventional PF (visual odometry and GPS) outcome. However, with curve matching, we were able to reduce the error further as low as below 2m for lateral and longitudinal error in all cases. This lower mean was mainly contributed by the road network itself and vehicle speed that was slower than in highway drive (Awang Salleh et al., 2018a).

B Path selection response

Since the road network for the residential dataset had multiple intersections or curves, and vehicle speed was slower, path selection performance for localization during random GPS noise had less significant difference. However, the positioning error accuracy and precision were slightly improved from the summarized bar graph in Figure 4.16.

Nonetheless, our proposed approach showed localization improvement in both biased and lost GPS events. During biased GPS noise, the GPS+VO approach was affected by the bias which showed a shifted trajectory in lateral position. Meanwhile, the RPDF approach outcome had mistakenly selected the wrong road particularly in frame 228 to 252, 520 to 581, and 646 to 666 due to the biased noisy GPS. Lastly, for the lost GPS case, both GPS+VO and RPDF localization results were 'lost' for a moment since frame 400 and finally managed to regain its position in frame 740 for GPS+VO approach and RPDF in Summary of the localization performance with curve matching technique for both sequences is displayed in Table 4.1. Here, the distance error and path selection for our proposed approach are presented with different GPS noise types.

Comparison with other VO methods

In order to study the positioning accuracy after data fusion with curve matching, we compared the lateral and longitudinal error of the short trajectory results generated from several other VO methods -Libviso2 (monocular and stereo) [START_REF] Geiger | StereoScan: Dense 3d reconstruction in real-time[END_REF], and ORB-SLAM2 [START_REF] Mur-Artal | ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras[END_REF]. Although these VO trajectories are without fusion with GPS and OSM data, the idea for this comparison is to find the degree of improvement in localization performance of the published VO methods with conventional VO method with fusion.

The VO trajectories are obtained from the open source VO methods. In Chapter 2 we also mentioned that LSD-SLAM proposed by [START_REF] Engel | Large-scale direct SLAM with stereo cameras[END_REF] and there is also SSLAM that provides code [START_REF] Fanfani | Accurate keyframe selection and keypoint tracking for robust visual odometry[END_REF]. However, the provided code is not suitable for implementation with KITTI dataset due to the higher image frame rate required (above 30 fps). Besides, we would like to focus on VO method without mapping or loop closure (SLAM). Note that Libviso2 for both monocular and stereo were purely based on VO, but ORB-SLAM2 also performed local mapping and loop closure detection for drift correction.

Although ORB-SLAM2 also provides localization mode without local mapping and loop closure, its localization for sequence 01 suffers severe rotation error. Therefore, we leave the SLAM mode on and regard this as VO with the best performance in translation error of only 1.15%. ORB-SLAM2 ranked at the 30th place in KITTI evaluation scoreboard, while Libviso2 (S) and Libviso 2 (M) were placed at rank 65 and 87 respectively. This means that we can classify ORB-SLAM2 as a high performance localization, Libviso2 (S) as the localization with average performance and Libviso(M) as a low-performance localization. Meanwhile, our conventional VO method achieves localization performance in between Libviso(M) and Libviso(S). The comparison analysis of positioning error for each method with our proposed approach is presented in Table 4.2.

From the comparison table, we can see that our low-performance conventional VO localization accuracy is increased drastically after data fusion. The positioning accuracy is slightly better than ORB-SLAM2 that performs local mapping and loop closure. The fusion results shows over 160% of improvement during sequence 01 while for sequence 2, the improvement level average is around 550%.

Synthesis

A method with data fusion of VO, GPS and digital map for vehicle localization has been proposed in effort to optimize localization accuracy with challenging sequence that has yet to be addressed carefully in this research area. It is a difficult task to ensure the longitudinal position is not affected by GPS noise and VO drift, thus our strategy is to match the last fragment of VO curve with the road segments curve to find the most likely candidate path, namely SCM. The SCM approach reduces the computation complexity while increasing the probability of suitable candidate ways based on its prior curve similarity score. This resulted in a localization system that could withstand various types of GPS noise and VO drift with the overall mean error less than 3m. The longitudinal accuracy and precision were increased in all cases proving that the segmentation concept was able to compensate the longitudinal error. Not only that, it could also increase road selection accuracy immediately for swift path planning that had been an issue during a high speed drive with diverging roads. A quantitative analysis on the positioning accuracy in comparison with other localization techniques were also presented. Our method achieved accuracy as good as ORB-SLAM2 without requiring map building and loop detection. In fact, our localization performance is proved to be better than ORB-SLAM2 except during severe GPS noise with bias, where our mean longitudinal error was 3% worse than ORB-SLAM2's, although this is considered as really small. Therefore, we expect our method can further improve the existing VO localization accuracy, despite the accumulated drift resulted in middle or low performance VO.

Implementation with Recent VO Methods

With the developing VO methods presented by other researchers, we are interested to know the effect of employing VO trajectory with different performance on our fusion technique.

By doing this, we are able to tell how far our system can improve the localization error despite trajectory drift and scale ambiguation. Therefore, we utilized methods such as Lib-viso2 (monocular and stereo) [START_REF] Geiger | StereoScan: Dense 3d reconstruction in real-time[END_REF] and ORB-SLAM2 [START_REF] Mur-Artal | ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras[END_REF] as the VO trajectory input as low-performance, average performance and high performance visual-based localization. The system performance is evaluated from the lateral and longitudinal error of both datasets, for all different GPS noise types. This time, both sequences were tested in a longer period (1000 frames). This added a challenge in longitudinal error optimization in sequence 01 due to the long stretch of straight road during the drive.

As a reference, Table 4.3 shows the relative translation error t rel of each VO methods for sequence 01 and 02. Obviously, monocular VO by Libviso2 had the worst translation error for both sequences while ORB-SLAM2 performed best in sequence 02. The severity of monocular VO is also displayed from overall trajectory comparison shown in Figure 4.17.

Localization drift by time illustrated in Figure 4.18 shows the severe error in monocular VO mainly due to the scale inaccuracy during a high-speed drive on a highway road. Detail positioning results are summarized in Table 4.4. The localization performance for ORB-SLAM2 when applied to our fusion technique with SCM undoubtedly showed the best accuracy especially with lateral error less than 1m in sequence 01 and longitudinal error achieved as low as 1.47m in sequence 02. Meanwhile, Libviso2(S) showed impressive performance where its lateral error was constantly less than 2m and its minimum longitudinal error was 1.79m in sequence 02 during random GPS error in sequence 02. On the other hand, Libviso2(M)'s best performance was on its lateral error of 0.95m and 2.16m of longitudinal error for sequence 02, also during random GPS error.

As for the comparison in between localization results of different GPS noise models, random GPS noise and lost GPS noise showed little difference in the best positioning error recorded. This means that our fusion technique is able to compensate the absence of lost GPS data by estimating the road segments probability and limits the positioning error with particle filter. However, this is not the case for biased noise where the lowest longitudinal error was 4.80m for sequence 01 and 2.83m for sequence 02 which was achieved by ORB-SLAM2 method as trajectory input. This is due to the effect of fusion with extreme GPS bias error that resulted in unsuitable probability distribution on the road segments. In sequence 01, since the drive is mostly on a straight road with high speed, bias could easily affect the longitudinal error. However, this can be compensated when the heading rotation changed, which will correct the trajectory curve with the right node-trio match. This is the reason sequence 02 achieved better longitudinal error, because the drive itself contained multiple turning points for curve matching.

Synthesis

A method with data fusion of VO, GPS and digital map for vehicle localization has been proposed to reduce vehicle positioning error for both lateral and longitudinal. We tested our approach by comparing several SLAM/VO methods from the publicly available codes with difference performance level. The initial accuracy of each method varies greatly from the monocular Libviso2 that ranked #84 (to this date) on KITTI leaderboard score, and ORB-SLAM2 that has the highest ranking in localization without lidar usage. The robustness of our system was tested with extreme GPS noise for random, biased, and lost signal occurrences.

From our positioning error results, it supported the hypothesis where accurate VO input will contribute in better fusion performance as proved by results achieved from ORB-SLAM2. Although the difference is comparatively small with those of Libviso2(S), ORB-SLAM2 showed 100% better accuracy in sequence 01 for random GPS noise while Lib-viso2(S) achieved better lateral error in sequence 02 during biased GPS. Furthermore, considering ORB-SLAM2 consumed higher computational cost, the localization performance achieved by Libviso2(S) could be regarded as exemplary where its performance improved significantly -almost on par with SLAM method.

Meanwhile, Libviso2(M) showed good localization for sequence 02 where the lateral error average was also constantly below 2m and longitudinal error average was 3.13m. However, its performance in sequence 01 was not satisfying and we think that this is due to the scale ambiguity problem that is often faced by monocular system. In overall, it can be concluded that our fusion technique is able to improve localization accuracy despite the drift in VO trajectory but up to the limit where the initial translation error is less than 10%.

Conclusion

This chapter presented the extended approach to our previous method with RPDF fusion.

Segmented curve matching (SCM) technique was introduced to compare VO trajectory curve with road shape on digital map. Starting with a conventional VO approach tested on a challenging drive on a highway road, our proposed localization performance by multilevel data fusion achieved high accuracy, on par with localization by ORB-SLAM2 which requires higher computation cost. Besides, our method proved to be robust against different types of GPS noise, where we applied severe noise variance of up to 10m.

Then we performed further investigation on the effect of having various levels of position accuracy from several VO/SLAM methods that were made available to public for evaluation. Based on the results obtained, the localization performance was greatly improved especially on residential drive (sequence 02) regardless of VO drift and scale ambiguity, thanks to the nature of trajectory shape with multiple turns matched with road shape within the search range. This however remains a problem in the case of straight road as resulted from sequence 01 especially for monocular VO. The lack of trajectory curve in addition to bad scaling factor made it impossible to reduce the positioning error. This should be solved with scaling technique designed for monocular system as proposed by other researchers such as presented in (Fanani et al., 2017a;[START_REF] Zhou | Reliable scale estimation and correction for monocular Visual Odometry[END_REF].

As a conclusion, the key to the achievement of our localization results was the VO trajectory curve that complement the existing digital map road network for curve comparison. As long as the VO trajectory drift and rotation error is not too severe, the localization accuracy can be significantly improved with segmented curve matching. Therefore, it would be interesting to know how far the trajectory generated by VO can be used for accurate low-cost vehicle localization. Our next step is to determine whether the lane-changing behaviour can be observed from this generated trajectory.

Chapter 5

Lane-keeping / Lane-changing Detection for Lane-level Localization

Overview

Navigation systems nowadays provide lane-level instructions in taking a route for completing a journey. For instance, multi-lane roads may have different destinations for each lane, hence the routing service or navigation application could provide instructions to use the specific lane for the driver. However, vehicle localization itself has yet to provide lane-level position that would complement the service. Thus, lane-level positioning system would be highly desired in outdoor localization technology where it will estimate vehicle's current lane correctly.

Lane-level positioning is not only useful for navigation system, but it is also required in emerging technologies such as advanced driver assistance systems (ADAS) [START_REF] Hofmann | Quality Assessment of Automatically Generated Feature Maps for Future Driver Assistance Systems[END_REF], road traffic estimation in lane-level detail, and electronic toll-collection system.

The current positioning system technology can provide sub deci-meter accuracy and while it satisfies the need for ordinary location-based services [START_REF] Aly | Map++: A crowd-sensing system for automatic map semantics identification[END_REF][START_REF] Ye | Location Recommendation for Locationbased Social Networks[END_REF], it lacks information on vehicle's exact lane position. This brings to an interesting field of research where many researchers had proposed different techniques to achieve lane-level accuracy in localization [START_REF] Cui | Real-time global localization of intelligent road vehicles in lane-level via lane marking detection and shape registration[END_REF][START_REF] Tao | Lane marking aided vehicle localization[END_REF][START_REF] Toledo-Moreo | Lane-Level Integrity Provision for Navigation and Map Matching With GNSS, Dead Reckoning, and Enhanced Maps[END_REF]. Unfortunately, these systems need high specification GNSS device such as RTK-GPS or laser sensors that are expensive and not practical for ubiquitous implementation. Meanwhile, low-cost solutions as proposed in [START_REF] Cui | Real-time global localization of intelligent road vehicles in lane-level via lane marking detection and shape registration[END_REF][START_REF] Du | Comprehensive and Practical Vision System for Self-Driving Vehicle Lane-Level Localization[END_REF][START_REF] Rabe | Lane-level mapmatching based on optimization[END_REF] used camera as visual sensor to detect road and lane markings. However, these methods will face difficulties in case the road markings are faded, or camera's line-of-sight is obstructed.

Besides developing localization system that is able to sense the correct lane of the vehicle, lane-level accuracy can also be achieved if lane-changing or lane-keeping is detected and correctly identified. This information can then be fused with the available road information on lane diffusion from digital map like OpenStreetMap to obtain the lane-level position.

In this chapter, we would like to explore the possibility of employing visual odometry trajectory in lane-keeping or lane-changing detection, without utilizing any additional sensor.

This study assumes that provided that the visual odometry trajectory has acceptable drift and scale factor, the trajectory curve can provide ample information for the detection.

Related Works

Recent studies on lane-changing detection can mainly be categorized into three approaches:

trajectory prediction model and matching [START_REF] Houenou | Vehicle trajectory prediction based on motion model and maneuver recognition[END_REF][START_REF] Salvucci | Inferring Driver Intent: A Case Study in Lane-Change Detection[END_REF][START_REF] Woo | Lane-Change Detection Based on Vehicle-Trajectory Prediction[END_REF][START_REF] Yao | Learning lane change trajectories from on-road driving data[END_REF], road lane marking interception [START_REF] Satzoda | Drive Analysis Using Vehicle Dynamics and Vision-Based Lane Semantics[END_REF][START_REF] Weidl | Early Recognition of Maneuvers in Highway Traffic[END_REF], and vehicle kimatics based characterization [START_REF] Aly | LaneQuest: An accurate and energy-efficient lane detection system[END_REF][START_REF] Chen | Invisible Sensing of Vehicle Steering with Smartphones[END_REF].

The first approach requires trained lane-changing trajectory model and matched with the vehicle trajectory to observe driver's intention of changing lane. For instance, [START_REF] Houenou | Vehicle trajectory prediction based on motion model and maneuver recognition[END_REF] utilized the Constant Yaw Rate and Acceleration (CYRA) model for trajectory prediction and others trained datasets of possible trajectories from experiments with different drivers. However, the studies are more practical in the early detection of lane-changing to prevent collision with other vehicles and the confirmation on lane-changing was not observed. Besides, since predicting vehicle trajectory also relies on driver's driving habit, it is not a deterministic task to confirm any lane-change that actually took place at a specific time.

Meanwhile, the second technique detects interception or vertical movement of road lane marking to update vehicle's lane position. Again, this is prone to the visibility state of the markings and is challenging in bad weather conditions. This method was also meant for early recognition of manoeuvers and risk assessment. Lastly, in the third approach, detection by characterizing vehicle kinematics technique was proposed by observing the inertial change of the vehicle. The suggested methods used inertial sensor equipped on the vehicle's steering to observe yaw rate and lateral acceleration for lane-changing characterization. In fact, the lane detection techniques as proposed in [START_REF] Satzoda | Drive Analysis Using Vehicle Dynamics and Vision-Based Lane Semantics[END_REF][START_REF] Weidl | Early Recognition of Maneuvers in Highway Traffic[END_REF] also applied the same in-vehicle sensor to capture vehicle kinematics aside from lane-marking cross analysis for lane-change detection. The kinematics only method as presented in [START_REF] Aly | LaneQuest: An accurate and energy-efficient lane detection system[END_REF][START_REF] Chen | Invisible Sensing of Vehicle Steering with Smartphones[END_REF] is an interesting approach because it does not rely on prediction model nor affected by weather constrains.

Therefore, instead of using inertial sensor, our research will observe lane-changing characteristics and detection visually from the trajectory of the drive. Since cameras can also be used for lane-marking detection as suggested by [START_REF] Satzoda | Drive Analysis Using Vehicle Dynamics and Vision-Based Lane Semantics[END_REF][START_REF] Weidl | Early Recognition of Maneuvers in Highway Traffic[END_REF], our method with visual odometry curve exploitation could be a complimentary system for them.

In this work, we analyse visual odometry trajectory curve by considering lane-changing characteristics to study the prospective of utilizing only visual sensor to determine the completed task of lane-changes.

System Overview

Our method proposes an integration of visual odometry trajectory curve analysis with road lane distribution from digital map. Previously, we have worked on fusion between GPS, visual odometry and map in which we found that abundant information in digital map should be utilized for lane-changing detection (Awang [START_REF] Salleh | Monocular visual odometry with road probability distribution factor for lane-level vehicle localization[END_REF]. Thus, without requiring trained trajectory model and additional input sensor aside from camera, we analyse the possibility of lane-changing detection by trajectory curve analysis combined with road lane information. Unlike early detection of lane-changing possibility as presented in previous works [START_REF] Houenou | Vehicle trajectory prediction based on motion model and maneuver recognition[END_REF][START_REF] Salvucci | Inferring Driver Intent: A Case Study in Lane-Change Detection[END_REF][START_REF] Weidl | Early Recognition of Maneuvers in Highway Traffic[END_REF][START_REF] Woo | Lane-Change Detection Based on Vehicle-Trajectory Prediction[END_REF][START_REF] Yao | Learning lane change trajectories from on-road driving data[END_REF], ours is a deterministic task to confirm any lane change event for lane-level localization update that would ensure the vehicle's accurate positioning.

The curve analysis is done by CUSUM chart [START_REF] Page | Continuous Inspection Schemes[END_REF], for small change detection in yaw rate, and curve-fitting error of heading angle extracted from visual odometry trajectory points. Lane-change detection is determined from certain conditions described in Section 5.3.1 from the result of the trajectory curve analysis and the performance of both methods are compared. Details for both curve analysis methods will be further explained in Section 5.4.

Lane-changing

Firstly, in order to determine an event of lane changing, it has to meet several criteria based on lane-changing behaviour in normal circumstances. Referring to the illustration in Figure 5.1 and as reported in several studies on trajectory prediction [START_REF] Houenou | Vehicle trajectory prediction based on motion model and maneuver recognition[END_REF][START_REF] Yao | Learning lane change trajectories from on-road driving data[END_REF], lane-changing can be described when:

OpenStreetMap

From the OSM file, we extract data of ways of roads that provides many information as described in previous chapters, that includes number of lanes and the trafic distribution of each lane. For validation, we tested on 3 sets of drive, namely Drive13, Drive29 and Drive42

where there were several instances of lane changing events in the raw KITTI dataset. The road lane distribution and its specification found in OSM file are described in Figure 5.2.

As depicted in Figure 5.2b, the road has 3 lanes with one-way direction (3-lane highway road). The lane traffic distribution is described in the tag k='turn:lanes' where the value contains 'left|through|through'. This indicates that the most left lane is specifically for turning left, while middle and right lane are for going straight at the traffic light. We can confirm this from the road symbolic marking captured by camera as shown in the subfigure. This is also the case for Drive 42 (Figure 5.2c) where the road is a two-lane one-way road with different directions for left and right lane.

Thus, from the information on number of lanes for the specific road, we are able to determine whether lane-change is possible or not. Besides, it can differentiate between possible lane-changing and maneuver scenario based on the allowable driving direction obtained from OSM data. If the road is multiple lane but with bi-directional, there is a possibility that the vehicle is just taking over the front vehicle and no lane-changing will be recorded since it will return to its initial lane after maneuvering. This estimation is conducted as shown in the flowchart (Figure 5.3) with N oL w represents number of lane at road segment w.

Trajectory Analysis

For trajectory curve analysis, we noted that vehicle's vertical acceleration or steering angle is most often used to observe lane-changing event [START_REF] Aly | LaneQuest: An accurate and energy-efficient lane detection system[END_REF][START_REF] Satzoda | Drive Analysis Using Vehicle Dynamics and Vision-Based Lane Semantics[END_REF]. Since inertial sensor was adapted in this work, we attempted to extract yaw rate ψ i calculated from trajectory point at frame i as

ψ i = δθ i δt , (5.2)
where heading angle θ i is updated from the condition

θ i =    ϑ i + π 2 , if ϑ i < 0 π 2 -ϑ i , otherwise , (5.3)
ϑ i = tan -1 ∆y i,i-1 ∆x i,i-1 .
(5.4)

The heading angle and yaw rate will then be utilized for detecting any abrupt change in vehicle heading variation. This change may or may not be observed visually from a raw yaw rate graph, depending on the trajectory noise. Furthermore, a classification technique is required to extract the time and magnitude of lane-changing possibility that occurred within the drive.

In order to detect temporary change in vehicle heading variation during a lane-change, we figured out two approaches that are able to classify a time-based randomized data. As depicted in the framework overview in Figure 5.4, the trajectory analysis is conducted with two approaches. The first method is by CUSUM chart analysis which is useful in detecting small shifts in the mean of a data. As long as we can specify the shift duration and slack limit, this would be a straightforward task in determining the lane-change. The second one is by observing fitting error from curve fitting analysis. Both methods will be presented in detail in the next subsections followed by validation and results.

CUSUM Chart Method

Based on the lane-changing characteristics, it is clear that vehicle orientation change happens in a small magnitude within a short time [START_REF] Worrall | A comprehensive examination of naturalistic lane changes[END_REF]. To detect this behaviour, firstly we proposed CUSUM (cumulative sum) analysis to observe the little change in vehicle heading direction. CUSUM is a type of control chart used to monitor the deviations of individual data or subgroup averages from a target value. These deviations are summed cumulatively over time. A raw CUSUM chart illustrates change in average value of a data differs from target by its slope.

For the decision making, we plotted two 'one-side' CUSUM chart for positive deviations and negative deviations to detect any change that happened above or below respective target. With the decision target plotted on the chart, CUSUM resembles a Shewhart control chart. However, Shewhart chart sometimes failed to detect small shift because it is more effective if the shift magnitude is larger (1.5σ to 2σ).

The CUSUM value above target, C + i and below target CUSUM value, C - i can be plotted from Equation 5.5 below. With initial C i values for both sides are set to 0, the succeeding values are calculated as

C + i = max[0, C + i-1 + ψ i -µ i -k u ], (5.5)
C - i = min[0, C - i-1 + ψ i -µ i + k l ], (5.6)
where

• ψ i : observed data value (yaw rate in radian) for the i th frame,

• µ : mean value for each subgroup,

• k u : upper allowable slack,

• k l : lower allowable slack.

The allowable slack k is determined based on sample size n and standard deviation σ as

k = σ i 2 √ n .
(5.7)

Then, to detect lane-changing behaviour, the slack limit, H = hσ is defined and if CUSUM value exceeds this limit, it is considered as lane-change detected. The h value is usually set between 4 to 5 according to [START_REF] Montgomery | Introduction to statistical quality control[END_REF].

Curve-fitting Method

Our second approach is by 2nd order curve fitting method on the vehicle heading angle θ i , with the assumption that θ i change is linear on a straight road and curve road can be represented in 2nd degree polynomial curve. For quadratic regression, it has 3 coefficients (a 2 , a 1 , a 0 ) that can be obtained by solving the following linear equations by inversion method or Gaussian elimination.

   n x i x 2 i x i x 2 i x 3 i x 2 i x 3 i x 4 i       a 0 a 1 a 2    =    y i x i y i x 2 i y i    , (5.8) f (θ i ) = a 0 x 2 i + a 1 x + a 2 .
(5.9)

We choose to fit heading angle curve instead of trajectory curve because it can be plotted on time-based and for its sensitivity in sensing variation. Besides, trajectory curve is susceptible to accumulated drift which may result in inaccurate coefficient calculation results if it is to be matched with the road map where scale and rotation are the main concerns. With curve fitting, the plotted polynomial curve, f (θ i ) is compared with the actual heading angle data to find any interception within the specified time range based on the error ε i ,

ε i = θ i -f (θ i ).
(5.10)

In order to determine lane-changing behaviour, considering there is a small amount of noise in visual odometry trajectory, we assume that:

(a) the fitting error ε i has an ideal mean error of 0, (b) ε i will stay on positive/negative side within a specific time range, t lc during lanechanging, (c) average magnitude of ε i within possible lane-changing time range is more than 1 • , and (d) for any adjacent occurence, lane-changing priority to the one with maximum εi .

Validation and Results

Datasets

To compute the probability of lane-changing in a drive, we analysed visual odometry trajectory according to lane-changing characteristics as mentioned in Section 5.3.1. Three datasets (Drive 13, 29 and 42) from KITTI were identified to contain lane-changing events. Drive 42 had two lane changing events on a multi-lane highway road, while both of drive 13 and 29 changed lane once to the right and left respectively. The lane changing for each drive happens within the time range as shown in the Table 5.1 with respective direction as observed visually from the image sequences and from the ground truth trajectory. These are also illustrated as shadowed area later in results charts (Figure 5.6, 5.7, 5.8 and 5.9) for clearer view.

For validation, the subgroup mean value µ i was calculated from on a 10s window. This was defined based on our assumption in Section 5.

CUSUM Chart Results

For curve analysis by CUSUM, the trajectory was smoothened and noise in yaw rate was reduced by using moving average filter. With window size N = 2k + 1, the filtered data can be obtained from: .11) In the following Figure 5.6, a moving average with window size of 5 points was applied to yaw rate of the raw trajectories. It can be seen that the lane-changing is not easily distinguishable from the yaw rate obtained from visual odometry trajectory curve even after filtering out noise. However, by plotting CUSUM chart, we will be able to detect any small movement change in a random data.

F i = 1 N +k j=-k S i-j . ( 5
We calculated the CUSUM value C i in 100 frames cycle and the chart is plotted for each dataset with visual odometry performance comparison as shown in Figure 5.7. Window size of 100 will cause a delay of 10s from the start time of lane-change. However as mentioned earlier, this study aimed to determine a lane-changing behaviour based on visual odometry trajectory after any occurence to confirm the current lane position, thus the delay was not as crucial as for application in risk assessment for lane-changing detection. The straight lane in Figure 5.7 shows the C i limit determined from ±4σ and any value that exceeded the limit is considered as lane-change detected. The direction of lane-change was determined by the upper limit for changing to the left lane and lower limit for changing to the right lane.

From Figure 5.7a, Libviso2(S) managed to detect lane-changing between frame 35 to 80

while Libviso(M) detected 2 false positives on the opposite direction (frame 61 and 126).

This was due to the severe residual noise in heading angle obtained from visual odometry trajectory that could be seen on the yaw rate graph. Since the scale in Libviso2(M) was estimated by assuming a fixed camera height over ground, any discrepancy would result in scaling error and drift that affected vehicle trajectory points and heading angle calculated from these points individually. Besides, the lane-changing event was in fact followed by gradual change in road direction (slightly curved), which caused the heading angle became more sensitive towards the noise. On the other hand, ORB(VO) and ORB-SLAM2 both successfully detected change in trajectory yaw rate from frame 37 to 77 and frame 39 to 81 respectively. Even though ORB(VO) showed ranging noise after frame 120, the change in its yaw rate did not exceed the slack limit, hence this was not classified as a lane-change.

In drive 29, trajectory analysis from all methods were able to detect lane-changing to the left lane correctly with C i exceeding upper limit around frame 70 to 90. This dataset exhibited a lane changing to the left lane before traffic light stop and taking a left turn later.

Peaks after frame 200 were ignored for lane-changing decision because the road's NoL obtained from OSM data confirmed that it was a single-lane road that made it impossible for a lane-change or maneuver.

Lane-changing events for drive 42 were also detected by all visual odometry methods.

For Libviso(M) result, the exceeding peak was merely detected for the first lane-changing to the left at frame 62 although was not as significant as peaks detected in other visual odometry methods. There were also noise observed as small changes after frame 250 in Libviso(M) but again, since the value did not exceed ±4σ slack limit, it was not regarded as a lane-change. The second lane-change around frame 200 was detected from the peaks in all methods exceeding the lower CUSUM limit, meaning it was slipping to the right lane.

Curve-fitting Results

For lane-detection by curve-fitting analysis, heading angle data was utilized and compared with the 2nd order polynomial curve obtained from the quadratic coefficient that fitted the data of 100 window. The curve fitting error graph is shown in Figure 5.8.

Assuming the error should vary around 0 during lane-keeping, if error value ε that remained above/below 0 within t lc of 2 to 8s range and if 1 • < ε, it will be regarded as lanechanging. As depicted in Figure 5.9, the lane confidence level in the left (upper side) and right (lower side) lane-change was obtained from the ε value.

Lane-change to the right was detected in drive 13 from Libviso2(S), ORB(VO), and ORB-SLAM2 trajectory curve analysis where the most accurate lane-changing time was achieved 5.2. It can be concluded that curve-fitting method has better accuracy in lane-change detection with zero false positive and only one case of false negative in drive 13 from trajectory generated by monocular Libviso2. For this particular drive, CUSUM method unfortunately detected two false positives and one false negative resulted also from the same visual odometry approach. This is a result of the bad scale factor that contributed in severe drift even in a short drive for a monocular system. The short and straight trajectory path made the lane-changing detection by CUSUM more sensitive to any small change, causing incorrect detections. On the other hand, curve-fitting method was less sensitive due to the buffer in fitting error that should exceed 1 • to be regarded as lanechange, but this also resulted in detection failure. On top of that, raw data of monocular Libviso2 itself for drive 13 (refer Figure 5.6 and 5.8) has too much noise that was difficult to be filtered due to the window size and trajectory span which made it impossible to detect lane-change event correctly.

Implementation Challenges

Although it was proved that CUSUM and curve fitting methods were able to detect lanechanging of a vehicle from visual odometry trajectory to a certain extent, real implementation for fusion with OSM data seemed to face several difficulties. Firstly, in order to employ our previously proposed RPDF approach, we need to know the exact lane of vehicle position at least during initialization. Else, with wrong lane decision, this would affect probability distribution of the neighbouring lanes and result in inaccurate lateral position estimation.

Secondly, when there is a change in number of lanes on the road, the OSM data only provides the change in number, hence missing the information of whether the new or missing lane is on the right or left side. Without this information, we are unable to fuse with RPDF correctly. For instance, in drive 42, the drive initially started with two lanes (for RPDF calculation, we have L1 and L2) that became a single lane road (reduced to only L1) after the right lane was diverged as a highway exit. Then, there was a new additional lane on the right again (we have probability distribution for L1 and L2 again), and after some time, a new lane was added also on the right (now we have L1, L2 and L3). Although the vehicle actually stayed on the same lane after it became a single lane, the RPDF estimation would be based on L1 and with a new addition of lane on the right, it would assume of being on L1 (right lane) instead of L2 without the knowledge of on which side is the new lane. The problem worsens with further additional of the third lane that will result in incorrect estimation of probability distribution for each lane.

Synthesis

Lane-changing detection by visual odometry curve trajectory analysis from CUSUM chart and curve-fitting method was presented in this chapter with comparison on different visual odometry approaches. CUSUM chart detected small change that happened within a certain time range while 2nd order polynomial was fitted on vehicle heading angle data to detect lane-change from fitting error. The analysis were conducted on three datasets with lane-changing events and visual odometry trajectories were obtained from four VO/SLAM trajectories. From the trajectories, heading angle and yaw rate were extracted for curve analysis.

Results on both curve analysis showed consistent output particularly for Libviso(S), ORB(VO) and ORB-SLAM2 approach that achieved 100% successful detection rate. Meanwhile, Libviso(M) failed to detect lane change correctly in drive 13 for both curve analysis methods, although it performed well in the other datasets with some detection delay in curve-fitting method. This is mostly due to monocular visual odometry performance that only has scale factor from assumption and this affects noise in heading angle between each image pairs.

Chapter 6

General Conclusions and Future Works

Conclusions

The research work in this thesis was focused on the optimization of vehicle localization accuracy with low-cost sensors. Our main interest was to study how visual odometry (VO) can be used for localization with other available information such as Global Positioning System (GPS) and digital map. While the current most commonly used navigation system is provided by GPS data due to its affordability, users still find inaccuracy in the data which makes it unreliable in certain conditions due to the urban environments or signal outages.

This makes an integration with additional sensor input necessary to compensate this problem and improve vehicle localization performance.

Therefore, we proposed to utilize VO trajectory obtained from standard camera to assist localization by fusing with GPS data and publicly available digital map. Cameras can easily be found on vehicles for safety reasons (video recording) and in some high-end cars, cameras perform image processing for pedestrian or lane detection. Aside from these purposes, it can also be used to obtain the vehicle trajectory from the captured image sequences and this has been immensely developed in recent decade. VO trajectory proved to be more accurate than wheel odometry, hence making it an interesting subject to be studied as a fusion input.

Chapter 2 in this thesis provided a wide coverage on literature review of VO techniques with challenges to obtain accurate localization. Besides, we also presented some of the existing works that also utilized VO with other sensors where low-cost sensors fusion techniques have yet to achieve high localization accuracy as performed by fusion with high precision sensors such as LIDAR. To evaluate the proposed methods of vehicle localization, KITTI dataset was presented and has been widely used as a benchmark and scoreboard. From the published VO/SLAM performance, we can conclude that localization problem mostly lies in the translation error due to scale ambiguity or drift, rather than rotational error. This resulted in better localization on roads with multiple junctions or curved roads while straight roads still face higher localization error. However, its localization performance in residential drive was greatly improved, almost on par with fusion output of stereo VO methods for both lateral and longitudinal positioning error. The performance for both stereo VO methods by Libviso2 and ORB-SLAM2 in highway drive with long straight road and higher vehicle speed achieved sub-meter accuracy with longitudinal error reduced to around 2m. This is a significant achievement, considering the low-cost GPS contained severe noise, no high-precision sensor was used, and the vehicle speed was around 60 to 70 km/h.

With the difference in VO methods performance, in Chapter 5, we extended our research scope by attempting to detect lane-changing events from the VO trajectories. Since the detection was conducted on posterior poses of vehicle, it was a deterministic task which is useful for lane update in lane-level localization, to complement our RPDF approach. Two methods were adopted for trajectory curve analysis by using CUSUM chart to detect small change in yaw rate and 2nd order curve fitting of the heading angle. Again, monocular VO failed to detect lane-changing correctly in one of the three datasets that has shorter drive range while it detected lane-changing events successfully in longer drives. Meanwhile, stereo VO methods managed to detect all lane-changing events correctly in all drives, due to smaller drift and less noise in the trajectories. This concluded that it is possible to determine when a vehicle changes lane or manoeuvers, provided that detailed information on any change of road lane distribution on map is known.

Our proposed method generally fuses low-cost GPS data as a reference for global position with VO trajectory. Then, the multi-level data fusion also integrated information from digital map to obtain probabilistic position of the vehicle on the map. It was a straightforward approach that did not require complex computation and yet it was able to achieve high localization accuracy with an average performance VO method as the fusion input.

Future works

Despite good overall achievement in localization performance from our fusion technique, there are still room for improvement especially in monocular VO. Due to the scale error, monocular VO suffers from severe drift in its trajectory. This affects the fusion performance where particles are displaced based on VO trajectory vector with randomized noise. Although we also utilize GPS as position reference, the scale error is accumulated by time which worsens the result in road segment candidates. Therefore, it would be desirable to have a reliable scale factor estimation technique in monocular VO systems to, at least, have an accuracy almost equivalent to stereo system with average performance (relative translation error of less than 10%). On top of that, if this can be achieved, we can further reduce the localization system cost with only utilizing a single camera.

In terms of fusion efficiency, noisy GPS data actually affected the overall localization performance. This should be solved by reducing the dependency of particles weight on probability distribution with distance from GPS position or we can also only refer to GPS points at a longer time interval. Since this study modelled GPS noise from a high precision GPS data, it would be more realistic to use a real low-cost GPS data for validation.

In Chapter 5, challenges that we faced for the lane-changing implementation were also mentioned. Indeed, VO trajectory with acceptable accuracy can be used to determine lanechanging events for lane-level positioning system. However, in real life implementation where number of lanes on the road changes frequently, we need further information to identify the current lane position from time-to-time in order to apply RPDF approach. Thus, Abstract : With the growing research on Advanced Driver Assistance Systems (ADAS) for Intelligent Transport Systems (ITS), accurate vehicle localization plays an important role in intelligent vehicles. The Global Positioning System (GPS) has been widely used but its accuracy deteriorates and susceptible to positioning error due to factors such as the restricting environments that results in signal weakening. This problem can be addressed by integrating the GPS data with additional information from other sensors. Meanwhile, nowadays, we can find vehicles equipped with sensors for ADAS applications. In this research, fusion of GPS with visual odometry (VO) and the digital map is proposed as a solution to localization improvement with low-cost data fusion. From the published works on VO, it is interesting to know how the generated trajectory can further improve vehicle localization. By integrating the VO output with GPS and OpenStreetMap (OSM) data, estimates of vehicle position on the map can be obtained. The lateral positioning error is reduced by utilizing lane distribution information provided by OSM while the longitudinal positioning is optimized with curve matching between VO trajectory trail and segmented roads. To observe the system robustness, the method was validated with KITTI datasets tested with different common GPS noise. Several published VO methods were also used to compare the improvement level after data fusion. Validation results show that the positioning accuracy achieved significant improvement especially for the longitudinal error with curve matching technique. The localization performance is on par with Simultaneous Localization and Mapping (SLAM) SLAM techniques despite the drift in VO trajectory input. The research on the employability of VO trajectory is extended for a deterministic task in lane-change detection. This is to assist the routing service for lanelevel direction in navigation. The lane-change detection was conducted by CUSUM and curve fitting technique that resulted in 100% successful detection for stereo VO. Further study for the detection strategy is however required to obtain the current true lane of the vehicle for lane-level accurate localization. With the results obtained from the proposed low-cost data fusion for localization, we see a bright prospect of utilizing VO trajectory with information from OSM to improve the performance. In addition, to obtain VO trajectory, the camera mounted on the vehicle can also be used for other image processing applications to complement the system. This research will continue to develop with future works concluded in the last chapter of this thesis.

Universit é Paris-Saclay

Espace Technologique / Immeuble Discovery Route de l'Orme aux Merisiers RD 128 / 91190 Saint-Aubin, France

1. 1

 1 Sensors equipped on a Tesla model S . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Sensors equipped on a Waymo autonomous car . . . . . . . . . . . . . . . . . . . . 1.3 Wheel odometry encoder (Pololu Corporation 2016) . . . . . . . . . . . . . . . . 1.4 Block diagram of inertial navigation system (Gade, 2005) . . . . . . . . . . . . 1.5 Examples of medium-and long-range automotive radar sensors operating up to 250 meters. (Bosch radar) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Cartography differences at Digiteo building, Universite Paris-Saclay . . . . . 1.7 Cartography differences at Pyongyang, North Korea . . . . . . . . . . . . . . . 2.1 FAST corner detector: Pixels 11-16 and 1-6 are brighter than p, hence p is detected as a feature point. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Top view of sensors setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Front view of moving platform used to collect data . . . . . . . . . . . . . . . 3.1 Representation of the Sampling Importance Resampling (SIR) algorithm of the Particle filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Conventional VO algorithm applied in our approach . . . . . . . . . . . . . . 3.3 Example of road representation in OSM map and the data structure in .osm file 3.4 Data fusion steps in particle filter . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Localization results for Dataset 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Localization results for Dataset 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Road Probability Distribution Factor (RPDF) for different road types . . . . . 3.8 Localization results with RPDF for Dataset 1 . . . . . . . . . . . . . . . . . . . 3.9 Error spike during wrong road choice due to GPS influence . . . . . . . . . . 3.10 Localization results with RPDF for Dataset 2 . . . . . . . . . . . . . . . . . . . 4.1 Localization error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 System overview of proposed approach . . . . . . . . . . . . . . . . . . . . . .

2. 1

 1 Performance comparison on VO / SLAM methods conducted on KITTI dataset 3.1 Results summary of localization distance error . . . . . . . . . . . . . . . . . . 4.1 Localization performance with SCM technique . . . . . . . . . . . . . . . . . . 4.2 Positioning error comparison with other VO works . . . . . . . . . . . . . . . 4.3 Translation error of different VO methods . . . . . . . . . . . . . . . . . . . . . 4.4 Comparison of positioning accuracy . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Lane-changing events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Overall lane-changing detection results . . . . . . . . . . . . . . . . . . . . . .

FIGURE 1 . 1 :

 11 FIGURE 1.1: Sensors equipped on a Tesla model S

FIGURE 1 . 3 :

 13 FIGURE 1.3: Wheel odometry encoder (Pololu Corporation 2016)

FIGURE 1 . 4 :

 14 FIGURE 1.4: Block diagram of inertial navigation system[START_REF] Gade | Introduction to Inertial Navigation. Tutorial for Geodesi-og Hydrografidagene[END_REF]

FIGURE 1 . 5 :

 15 FIGURE 1.5: Examples of medium-and long-range automotive radar sensors operating up to 250 meters. (Bosch radar)

  FIGURE 1.6: Cartography differences at Digiteo building, Universite Paris-Saclay

FIGURE 2

 2 FIGURE 2.2: Top view of sensors setup

  FIGURE 3.1: Representation of the Sampling Importance Resampling (SIR) algorithm of the Particle filter

FIGURE 3 .

 3 FIGURE 3.2: Conventional VO algorithm applied in our approach

FIGURE 3 . 3 :

 33 FIGURE 3.3: Example of road representation in OSM map and the data structure in .osm file

  weights. The first step in this research after obtaining visual odometry trajectory is to fuse output from visual odometry with noisy GPS to determine the functionality of the filter and compare the accuracy with visual odometry localization. Particle filter consists of several steps as illustrated in Figure3.4 and further described below:

FIGURE 3 . 4 :

 34 FIGURE 3.4: Data fusion steps in particle filter

FIGURE 3 . 5 :

 35 FIGURE 3.5: Localization results for Dataset 1

  FIGURE 3.7: Road Probability Distribution Factor (RPDF) for different road types

  FIGURE 3.8: Localization results with RPDF for Dataset 1

FIGURE 3 . 9 :

 39 FIGURE 3.9: Error spike during wrong road choice due to GPS influence

FIGURE 4 . 2 :

 42 FIGURE 4.2: System overview of proposed approach

Figure 4 .

 4 Figure 4.4 and more details are as follow:

FIGURE 4 . 3 :

 43 FIGURE 4.3: Node placements in OSM file

  At the same time, the travelled distance of VO curve, d vo within the last m poses is computed to find the difference with d n th .

  the difference between d th and d vo is negligible, the larger distance threshold is used to minimize new middle nodes generation. Hence, d th =max(d n th , d vo ). Else, VO trail length is chosen d th = d vo to improve accuracy. (e) With the first level fusion between low-cost GPS and VO, the vehicle position can be estimated on the map and the nodes around the location are found. Distances between neighbouring two nodes n from the OSM data are recorded as d n .(f) Finally, in order to distribute road map probability with longitudinal accuracy optimization, new nodes are replicated in between of nodes that are too distant with each other. The number of new middle nodes, n m is obtained by rounding results of nodes

FIGURE 4 . 4 :

 44 FIGURE 4.4: Algorithm for nodes generation in OSM data

Figure 4 .

 4 5 and for the n'th node found within searching radius, it forms a node-trio with the neighbouring nodes referred as sets of [P n , P n+1 , P n+2 ]. The initial orientation of the first pair nodes θ n of the node-trio and curve change ϑ n are calculated as θ (n) = arctan ∆y n,n+1 ∆x n,n+1 , (4.6)

FIGURE 4 . 5 :

 45 FIGURE 4.5: Angle parameters from three nodes used for curve matching

FIGURE 4 .

 4 FIGURE 4.6: VO and OSM node-trio curve matching steps

  (d) Finally, node-trio with highest similarity score is identified and the road segment contains the highest probability that would affect particles' importance weight during data fusion.

FIGURE 4 . 7 :

 47 FIGURE 4.7: Double bend road in drive 34 with 2s of VO trajectory in quadratic curve appended in the small box

FIGURE 4 .

 4 FIGURE 4.8: Multilevel data fusion and associated inputs for each level

  lookup centre for road segments comparison with VO trajectory in the next level. that form candidate ways are searched within specified radius that contains all particles i (explained in validation part of Section 4.4.1).

  estimated based on the distance of node n from the lookup centre X (C) k . Then, the curve similarity score S (n)

FIGURE 4 .

 4 FIGURE 4.9: Probability distribution of candidate ways from fusion output as lookup centre

FIGURE 4 .

 4 FIGURE 4.10: GPS error sources

FIGURE 4 .

 4 FIGURE 4.11: Trajectory results for highway drive (subset of sequence 01)

Figure 4

 4 Figure 4.14.

  FIGURE 4.14: Trajectory results for residential drive

  FIGURE 4.15: Lateral and longitudinal localization positioning error in sequence 02

FIGURE 4 .

 4 FIGURE 4.19: Severe translation error in Libviso2(M) trajectory resulted in wrong path chosen during fusion

  (a) the vehicle movement has a small change in lateral direction (x-axis) within a limited distance d l , assuming a normal lane-change should shift just one lane in a single occurence; stereo vision, the authors demonstrated accurate dense 3d reconstructions from the image sequences to obtain vehicle trajectory.We also tested on localization trajectory generated from ORB-SLAM2 method as recently presented by[START_REF] Mur-Artal | ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras[END_REF] that contains three main parallel threads: 1) tracking thread for camera localization by feature matching to the local map and applies motion-only bundle-adjustment to minimize error, 2) local mapping thread, and 3) loop closing thread for drift error correction when loops are detected. This method has 'Simultaneous Localization and Mapping (SLAM)' mode and 'localization' mode. SLAM mode allows map building of an unknown environment and localization of the sensor within the map at the same time by performing loop closure that corrects accumulated error over time. Meanwhile, during localization mode, local mapping and loop closing threads are deactivated and tracked points are matched between ORB in the current frame and 3d points from the previous frame. We purposely tested in both modes to find the variation of yaw rate in different level of visual odometry performance.In short, the trajectory input that are used in this study greatly varies in terms of positioning accuracy. Monocular Libviso2 has the worst drift error among compared with other methods due to the scale factor limitation, which is followed by Libviso2's stereo visual odometry, ORB-VO (without loop closure), and finally ORB-SLAM2 with loop closure has recorded the highest precision in localization. With this varying visual odometry performances, we would like to analyse the feasibility of using the trajectory curve in determining vehicle lane changing occurrences.

FIGURE 5

 5 FIGURE 5.2: Road segments where lane-changing occurred (highlighted in red) with respective descriptions in OSM file

FIGURE 5 . 4 :

 54 FIGURE 5.4: Framework overview of lane-changing detection

  3.1 where considering detection delay and VO trajectory drift, a complete lane-change should take place within 2s to 8s. Thus, to detect a complete event of lane changing, 2s buffer time was added to the maximum lane-change duration. As for the slack limit, we fixed h = 4 to increase detection sensitivity.Dataset 13 had the shortest drive with 143 frames of images. It had a simple lanechanging event where the vehicle moved from the left lane to the right lane on a two-lane straight road. Meanwhile, drive 29 contained 430 poses where the vehicle moved along a two-lane road that evolved to three-lane road before a traffic light stop. The drive initially started on the left lane (L2 on two-lane) and at frame 65 it moved to the additional left lane (L3 on three-lane road) that was specifically for turning left. The vehicle stopped at the traffic light from pose 200 to 265 before turning left into a single-lane road. The third dataset, drive 42, was taken on a highway road that started with the vehicle driven on the right lane on a two-lane road (L1). Since the right lane was assigned for turning right to a highway exit, but the driver intended to drive straight, at frame 40, the vehicle changed to the left lane (L2) and kept driving on the same lane until frame 195 where it moved back to the right lane (L1) that had the same traffic direction with other lanes. This drive consisted of 500 poses for trajectory analysis and the highway road was slightly curved along the way.In order to test the feasibility of lane-changing detection by visual odometry curve analysis, these datasets were tested with four different visual odometry approaches -monocular Libviso2, stereo Libviso2, ORB based VO and ORB-SLAM2 -as explained in Section 5.3.2these are later indicated as Libviso2(M), Libviso2(S), ORB(VO) and ORB-SLAM2 in the results presentation respectively. Trajectories obtained from each dataset are shown in Figure

FIGURE 5 . 5 :

 55 FIGURE 5.5: Vehicle trajectories generated from VO/SLAM methods

FIGURE 5

 5 FIGURE 5.6: Yaw rate based on VO curve after noise filtering

FIGURE 5

 5 FIGURE 5.7: Lane-changing detection from CUSUM plots

  Our proposed technique aims to solve positioning accuracy in localization including its longitudinal error that remains difficult to overcome particularly on straight roads. This study began with the data fusion of GPS with VO and OpenStreetMap (OSM) with particle filter to verify the effectiveness of the fusion technique in improving localization accuracy as presented in Chapter 3. A conventional VO method was applied and from the information extracted from OSM, a Road Probability Distribution Approach (RPDF) was proposed to enhance lateral position of the vehicle. With the assumption that initial lane is known, and the vehicle does not change lane on a multi-lane road, our technique managed to increase the localization accuracy as tested in two datasets despite the drift and scale error of the conventional VO method. From the error spike occurred at a road divergence, we were interested to utilize the characteristic of VO trajectory curve to compare with road shape on the digital map.In Chapter 4, we developed our multi-level fusion technique by matching VO trajectory curve segments with road segments on OSM map to avoid discrepancy in road selection and increase longitudinal positioning error. The proposed method called Segmented Curve Matching (SCM), was also tested with low-cost GPS with random error, biased error and an event of GPS signal loss to observe the robustness. Then, advanced open source VO methods with different performance level were used as an input to the fusion and the results show that monocular VO still faced severe localization error in straight roads due to the scale error.

  future work might include lane position identification by image processing from the road line detection to be matched with OSM data. Besides, since OSM is an open source service with a growing community involved in its development, we can propose an additional information in their data to specify detailed information when there is a change in number of lanes. This, would be beneficial for all. Zhu, Jianke (2017). "Image Gradient-based Joint Direct Visual Odometry for Stereo Camera". In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. IJ-CAI'17. Melbourne, Australia: AAAI Press, pp. 4558-4564. ISBN: 978-0-9992411-0-3. URL: http://dl.acm.org/citation.cfm?id=3171837.3171924. Titre : Fusion de donn ées pour la localisation de v éhicule par suivi de trajectoire provenant de l'odom étrie visuelle Mots cl és : odom étrie visuelle, localisation, fusion des donn ées R ésum é : Au sein des syst èmes avanc és d'aide à la conduite (Advanced Driver Assistance Systems -ADAS) pour les syst èmes de transport intelligents (Intelligent Transport Systems -ITS), les syst èmes de positionnement, ou de localisation, du v éhicule jouent un r ôle primordial. Le syst ème GPS (Global Positioning System) largement employ é ne peut donner seul un r ésultat pr écis à cause de facteurs ext érieurs comme un environnement contraint ou l'affaiblissement des signaux. Ces erreurs peuvent être en partie corrig ées en fusionnant les donn ées GPS avec des informations suppl émentaires provenant d'autres capteurs. La multiplication des syst èmes d'aide à la conduite disponibles dans les v éhicules n écessite de plus en plus de capteurs install és et augmente le volume de donn ées utilisables. Dans ce cadre, nous nous sommes int éress és à la fusion des donn ées provenant de capteurs bas cout pour am éliorer le positionnement du v éhicule. Parmi ces sources d'information, en parall èle au GPS, nous avons consid ér és les cam éras disponibles sur les v éhicules dans le but de faire de l'odom étrie visuelle (Visual Odometry -VO), coupl ée à une carte de l'environnement. Nous avons étudi é les caract éristiques de cette trajectoire reconstitu ée dans le but d'am éliorer la qualit é du positionnement lat éral et longitudinal du v éhicule sur la route, et de d étecter les changements de voies possibles. Apr ès avoir ét é fusionn ée avec les donn ées GPS, cette trajectoire g én ér ée est coupl ée avec la carte de l'environnement provenant d'Open-StreetMap (OSM). L'erreur de positionnement lat érale est r éduite en utilisant les informations de distribution de voie fournies par OSM, tandis que le positionnement longitudinal est optimis é avec une correspondance de courbes entre la trajectoire provenant de l'odom étrie visuelle et les routes segment ées d écrites dans OSM. Pour v érifier la robustesse du syst ème, la m éthode a ét é valid ée avec des jeux de donn ées KITTI en consid érant des donn ées GPS bruit ées par des mod èles de bruits usuels. Plusieurs m éthodes d'odom étrie visuelle ont ét é utilis ées pour comparer l'influence de la m éthode sur le niveau d'am élioration du r ésultat apr ès fusion des donn ées. En utilisant la technique d'appariement des courbes que nous proposons, la pr écision du positionnement connait une am élioration significative, en particulier pour l'erreur longitudinale. Les performances de localisation sont comparables à celles des techniques SLAM (Simultaneous Localization And Mapping), corrigeant l'erreur d'orientation initiale provenant de l'odom étrie visuelle. Nous avons ensuite employ é la trajectoire provenant de l'odom étrie visuelle dans le cadre de la d étection de changement de voie. Cette indication est utile dans pour les syst èmes de navigation des v éhicules. La d étection de changement de voie a ét é r éalis ée par une somme cumulative et une technique d'ajustement de courbe et obtient de tr ès bon taux de r éussite. Des perspectives de recherche sur la strat égie de d étection sont propos ées pour d éterminer la voie initiale du v éhicule. En conclusion, les r ésultats obtenus lors de ces travaux montrent l'int ér êt de l'utilisation de la trajectoire provenant de l'odom étrie visuelle comme source d'information pour la fusion de donn ées à faible co ût pour la localisation des v éhicules. Cette source d'information provenant de la cam éra est compl émentaire aux donn ées d'images trait ées qui pourront par ailleurs être utilis ées pour les diff érentes taches vis ée par les syst èmes d'aides à la conduite. Title : Study of Vehicle Localization Optimization with Visual Odometry Trajectory Tracking Keywords : visual odometry, localization, data fusion

  engines which was initially known as Live Search Maps. With Bing Maps, users can browse and search topographically shaded street maps for many cities worldwide. It is also possible to browse public user-created points of interest and searches cover public collections, businesses or types of business, locations, and people. Five street map views are available: Road View, Aerial View, Bird's Eye View, Street Side View, and 3D View.

  Apple Inc. launched Apple Maps in September 2012 in iOS to replace Google Maps as the default mapping service for Apple operating systems. This is due to the growing tensions between Google and Apple in late 2009 when Android version of Google Maps featured Google did not immediately launch a mapping application on its own on mobile. Not until three months after Apple Maps announcement, Google Maps was released in the App Store with new features. The largest selling point for Apple Maps' turn-by-turn navigation is the 3D maps (activated by swiping up with two fingers during a journey). While it is currently available in only certain locations around the world, Apple is constantly updating its 3D maps and adding new locations. Users can see surrounding building in 3D, which makes it easier to figure out exactly where they are in relation to the position displayed on the screen.

turn-by-turn navigation, which the iOS version lacked. Thus, Apple announced the initial release of Apple Maps that include turn-by-turn navigation, 3D maps, Flyovers (3D photorealistic views of big cities and landmarks), and virtual assistance Siri. At first,

  tableshows the ranking of methods according to the average of both errors. Methods marked as

(*) denote the availability of its source code online. Meanwhile, (**) indicates that the open source methods did not use lidar in their systems. The KITTI dataset can be obtained from http://www.cvlibs.net/datasets/kitti.

TABLE 2

 2

	Method	Translation	Rotation
	V-LOAM (Zhang et al., 2015)	0.60 %	0.0014 [deg/m]
	LOAM (Zhang et al., 2014a)	0.61 %	0.0014 [deg/m]
	SOFT2 (Cvišić et al., 2017)	0.65 %	0.0014 [deg/m]
	IMLS-SLAM (Deschaud, 2018)	0.69 %	0.0018 [deg/m]
	RotRocc+ (Buczko et al., 2016a)	0.83 %	0.0026 [deg/m]
	* LIMO2_GP (Graeter et al., 2018)	0 .84 %	0.0022 [deg/m]
	GDVO (Zhu, 2017)	0.86 %	0.0031 [deg/m]
	* LIMO2 (Graeter et al., 2018)	0.86 %	0.0022 [deg/m]
	SOFT (Cvišić et al., 2015)	0.88 %	0.0022 [deg/m]
	RotRocc (Buczko et al., 2016a)	0.88 %	0.0025 [deg/m]
	DVSO (Yang et al., 2018)	0.90 %	0.0021 [deg/m]
	* LIMO (Graeter et al., 2018)	0.93 %	0.0026 [deg/m]
	Stereo DSO (Wang et al., 2017)	0.93 %	0.0020 [deg/m]
	ROCC (Buczko et al., 2016b)	0.98 %	0.0028 [deg/m]
	cv4xv1-sc (Persson et al., 2015)	1.09 %	0.0029 [deg/m]
	MonoROCC (Buczko et al., 2017)	1.11 %	0.0028 [deg/m]
	DEMO (Zhang et al., 2014b)	1.14 %	0.0049 [deg/m]
	** ORB-SLAM2 (Mur-Artal et al., 2017)	1.15 %	0.0027 [deg/m]
	NOTF (Deigmoeller et al., 2016)	1.17 %	0.0035 [deg/m]
	** S-PTAM (Pire et al., 2017)	1.19 %	0.0025 [deg/m]

.1: Performance comparison on VO / SLAM methods conducted on KITTI dataset ization on a straight road, most possibly due to drift and longitudinal error. Besides, limitations in VO systems are identified regarding the scale uncertainty and calibration errors that affects its localization performance. As presented in the evaluation table, VO methods accuracy may vary from 20.95 % & 0.0135 [deg/m] to 0.86 % & 0.0031 [deg/m]

  Thisis practical for mobile robot applications in an unknown environment and does not require high accuracy positioning in real time. However, as for vehicle localization, digital map of the world is publicly available, hence new map building is deemed unnecessary if one's position can be estimated on earth. Besides, with the abundant information provided by digital map nowadays, we believe that VO trajectory can be useful for global localization with map fusion and its performance can be on par with highly accurate GPS or lidar if optimized correctly. Furthermore, with the integration of low-cost GPS data, rough position of the vehicle can be estimated and this can reduce unnecessary computation of map area outside region of interest. The next chapter demonstrates our fusion technique that integrates the information from GPS, VO and digital map for localization accuracy improvement.

we see vast development in SLAM research with highly accurate localization results, it requires more complex computation for map building and with loop closure detection to compensate drift error that would result in delayed localization correction.

  of innovations and estimates of the state x k . From the innovations, the likelihood function of the dynamic model can be built and with the states estimates, it allows us to forecast and smooth out the stochastic process.Behaving as a recursive estimator, only the estimated state from the previous time step and the current measurement are required for the computation of the current state estimation in Kalman filter. It is different with the batch estimation techniques where it requires history of observations and its estimates. The state in this method is represented by two variables; x k , as the posteriori state estimate at time k, provided observations up to time k, and P k which denotes the posteriori error covariance matrix to measure the accuracy of the state estimation.The Kalman filter is most often conceptualized into two distinct phases: "Prediction" and "correction". The prediction phase uses the state estimation from the previous time step to calculate a new estimate of the state at the current time step. This predicted state estimation is also regarded as the a priori state estimate because although it is an estimate of the state

at the current timestep, it has yet to include observation information from the measurement at the current timestep. Meanwhile in the correction phase, the priori predicted state is combined with the current observed information to refine the state estimate. This improved updated estimate is termed as the 'posteriori state estimate'.

TABLE 3

 3

	.1: Results summary of localization distance error
	Dataset	Distance Error (µ ± σ) [m] VO+GPS VO+RPDF
	Dataset 1 2.619 ± 1.397 1.636 ± 0.851
	Dataset 2 3.608 ± 1.968 2.522 ± 1.566

TABLE 4 .

 4 1: Localization performance with SCM technique 839th frame. From the trajectory we can see that localization by RPDF snapped into nearest roads before locating the correct road. Our proposed approach with SCM however succeeded to perform localization with excellent path selection for the whole drive in all noise conditions.

TABLE 4

 4

		.2: Positioning error comparison with other VO works
	Method	Sequence 01 (Highway)	Sequence 02 (Residential)
		lat [m]	lon [m]	lat [m]	lon [m]
	Conventional VO	9.78±15.93	8.92±16.62	5.22±16.47	5.72±17.37
	Libviso2(M)	37.24±62.62	23.91±64.44	9.24±41.48	10.34±41.19
	Libviso2(S)	6.45±19.93	6.83±19.68	3.09±18.71	3.00±18.7
	ORB-SLAM2	1.77±6.66	2.22±6.62	2.31±14.83	2.38±14.81
	SCM (random)	1.40±1.06	1.37±1.03	2.05±1.62	2.08±1.58
	SCM (bias)	1.66±1.23	1.76±1.32	2.27±1.76	2.45±1.89
	SCM (lost GPS)	1.41±1.37	1.30±1.48	1.34±0.93	1.42±1.25

TABLE 4 .

 4

		3: Translation error of different VO methods
	t rel (%) Libviso2 (M) Libviso2 (S) ORB-SLAM2
	Seq. 01	25.85	5.00	1.10
	Seq. 02	8.13	1.13	0.74

TABLE 4 .

 4

			4: Comparison of positioning accuracy
			Libviso2 (M) Libviso2 (S) ORB-SLAM2
			lat	lon	lat	lon	lat	lon
	Random GPS	Seq 01 Seq 02	Mean, µ Stddev, σ 20.43 21.18 2.77 6.78 23.30 1.54 Mean, µ 0.95 2.16 1.06 Stddev, σ 0.72 1.24 0.74	2.29 5.38 1.79 1.06	0.78 0.59 1.02 0.71	1.87 1.36 1.47 0.98
	Biased GPS	Seq 01 Seq 02	Mean, µ Stddev, σ 75.42 94.79 3.07 28.31 143.59 1.81 Mean, µ 1.52 4.01 1.35 Stddev, σ 1.63 2.94 1.00	6.83 4.83 3.81 2.07	0.79 0.60 1.43 1.03	4.80 1.94 2.83 1.52
	Lost GPS	Seq 01 Seq 02	Mean, µ Stddev, σ 64.54 74.73 1.30 24.17 96.40 1.17 Mean, µ 1.30 3.22 1.14 Stddev, σ 1.24 2.31 0.68	2.63 2.47 2.05 1.45	0.78 0.61 1.07 0.77	1.75 1.18 1.50 0.91

TABLE 5 .

 5 1: Lane-changing events Drive Start frame, i s End frame, i e Lane-changing direction

	13	40	80	right
	29	65	90	left
	42	40 190	70 235	left right

Nous avons ensuite employé la trajectoire provenant de l'odométrie visuelle dans le cadre de la détection de changement de voie. Cette indication est utile pour les systèmes de navigation des véhicules. La détection de changement de voie a été réalisée par une somme cumulative et une technique d'ajustement de courbe et obtient de très bon taux de réussite.Des perspectives de recherche sur la stratégie de détection sont proposées pour déterminer la voie initiale du véhicule en particulier dans le cas d'un changement de nombres de voie sur la même route. De plus, des statégies d'optimisation des résultas sont proposés quant à la précision des données OSM, notamment sur la position de voie additionnelle ou en détectant du marquage au sol, au besoin.En conclusion, les résultats obtenus lors de ces travaux montrent l'intérêt de l'utilisation de la trajectoire provenant de l'odométrie visuelle comme source d'information pour la fusion de données à faible coût pour la localisation des véhicules. Cette source d'information provenant de la caméra est complémentaire aux données d'images traitées qui pourront par ailleurs être utilisées pour les différentes taches visée par les systèmes d'aides à la conduite. v

FIGURE 4.13: Error summary for sequence 01

Acknowledgements

First and foremost, I would like to express my sincere gratitude to my supervisor, Emmanuel Seignez, for giving me an endless support and motivation in many aspects of life, from the first day I set my foot in France. I am truly grateful for having such a great supervisor that never stops giving me encouragement during my study.

I would also like to thank my thesis reviewers, Prof. David Betaille and Prof. Ronald Chapuis, for your comments and fruitful suggestions on my work. Not to forget, my colleagues -Rokiatou, Jennifer, Hernan, Sara, Dai Duon, and many others -that always brighten up my days at lab and become my source of information whenever I have questions or doubts.

I am also debtful to my family especially my parents, who raised me up well, help me making decisions, and very supportive in my study. My siblings also deserve to be mentioned here, for their prayers for my success. Last but not least, I would like to endlessly thank my husband, who had sacrificed a lot on our journey to finish my study away fr.

Words can never really describe how thankful I am to have you by my side and only God can return your kindness.

Validation Results

From the results, we found that the positioning error for Libviso2(S) and ORB-SLAM2

achieved almost similar performance despite the differences in relative translation error of each VO trajectory without fusion with GPS and map. Even for Libviso2(M), it performed well in sequence 02 where the average positioning error could achieve as low as 1.7m during random GPS noise test. However, in sequence 01, our fusion method could not improve the localization in Libviso2(M) due to the severe drift and scale error where it resulted in earlier highway exit as shown in Figure 4.19.

(b) its heading angle θ i before and after lane-changing is almost equal (particularly on a straight road) or the yaw rate ψ i is constantly proportional (curve road): Putting these characteristics into consideration, we are able to analyse vehicle lanechanging possibility from its trajectory curve to observe its lateral speed and yaw rate. As for the lane-changing duration, t lc , according to [START_REF] Lee | A comprehensive examination of naturalistic lane changes[END_REF], the mean duration for lanechanging is around 6.28s for a single lane-change. However, this includes the cases of heavy vehicle (trucks, lorries, busses) that took longer duration for lane changing. Meanwhile, other researchers performed study on standard vehicle lane-change duration outcome that averaged at 4.6s [START_REF] Toledo | Modeling Duration of Lane Changes[END_REF]. In addition, the minimum average that can describe a lane-change is 1.25s from a study conducted by [START_REF] Worrall | A comprehensive examination of naturalistic lane changes[END_REF].

The mean differences were as a result of the definition of lane-changing start and stop time, analysis methods, and diversity of vehicle types. In our work, we specify the lanechanging characteristics observation within the given lane-changing duration average of minimum 1.25s and maximum 6.28s. Therefore, we assume the lane-changing duration, (t cl ) should be in between 2s to 8s, considering some delay in detection from trajectory curve analysis while avoiding false positive detection from the trajectory noise if the duration is too short.

Visual Odometry

In this study, the existing visual odometry algorithm proposed by [START_REF] Geiger | StereoScan: Dense 3d reconstruction in real-time[END_REF] for both stereo and monocular vision system was adapted as our visual odometry trajectory input. In their proposed monocular visual odometry, the scale is estimated by assuming that the camera is fixed on the vehicle over a certain height from the ground. Meanwhile, for Chapter 5. Lane-keeping / Lane-changing Detection for Lane-level Localization

In overall, lane-changing detection by visual odometry trajectory analysis with curvefitting technique performed better than CUSUM chart analysis technique in terms of detection rate and precision. With this method, we were able to detect lane-changes correctly with start and end time estimation especially in stereo systems. Besides, it does not require parameters settings of k or slack limit 4σ estimation like CUSUM chart does. As for the trajectory performance effects on detection accuracy, visual odometry trajectory with minimal noise are required to acquire reliable heading angle information for more accurate results.

Nonetheless, both curve analysis techniques were able to detect most lane-changes despite the overall trajectory drift.

However, in order to apply our method for lane-level localization, we need more information on the road lane distribution especially when there is a change in the number of lanes. Since OSM does not contain detailed information on this, we might have to perform image processing in road lane marking detection to determine the current lane position from time-to-time. This will only be required periodically or when the system received any change in number of lanes, hence minimizing computation effort for the line detection. Therefore, we can conclude that our lane-changing detection approach could be a complementary system with the existing road marking or line detection for an optimized and accurate lane-level vehicle localization.

Although this study focused on accuracy improvement in challenging drive sequence, the proposed system should be validated in other sequences and datasets as well. From this, we can further analyze its robustness in different environment and with its overall performance, we can join the leaderboard ranking of public dataset.

Finally, we also look forward to hardware implementation of the localization system which is also included in the research future plan. Upon successful validation on the software end, the real performance can only be evaluated on a hardware platform. For hardware implementation, we can test the system with more options of driving environment with different road network, weather conditions, and urban canyon scenarios. This will be executed once the tasks above are completed and the localization performance is validated with satisfying accuracy.