R Ésum

Cette thèse a été effectuée dans un contexte industriel et présente un framework complet pour la detection et le suivi de personnes dans un réseau de caméras de surveillance. Les trois principales étapes du processus sont traitées: la détection de personnes, le suivi de personnes dans un contexte mono-caméra et enfin la ré-identification de personnes dans le contexte multi-caméras. Les performances élevées, la généricité et la facilité de déploiement ainsi que le traitement en temps réel sont les contraintes fortes qui ont guidé ces travaux. Certaines parties du travail proposé ont déjà été intégrées et déployées dans un produit commercial de vidéo surveillance intelligente alors que les autres parties sont à l'état de prototypes et seront intégrés dans un futur proche.

La détection de personnes vise à localiser et délimiter les personnes sur les séquences vidéo ainsi que sur les images statiques. Le détecteur de personnes proposé appartient à la catégorie des détecteurs de silouhette entière et opère à l'aide d'une cascade de classifieurs, appris en utilisant l'algorithme LogitBoost sur les descripteurs de covariances de régions. Une approche de l'état de l'art, fournissant de bonnes performances mais non applicable pour le traitement en temps réel a été prise comme base de travail et a été optimisée afin de permettre le traitement en temps réel tout en améliorant légérement les performances de détection. La méthode d'optimisation proposée est généralisable à de nombreux autres types de détecteurs basés sur les cascades de classifieurs, et dont l'espace de tous les classifieurs faibles possible ne peut Ãa tre testé exhaustivement dans un temps raisonable.

Le suivi de personnes dans le contexte mono-caméra vise à fournir un ensemble d'imagette de chaque personne observé par chaque caméra, afin de permettre le calcul de la signature visuelle de ces personnes. Il fournit aussi certaines informations du monde réel qui sont tr ès utiles pour améliorer les résultats de la ré-identification, du moment que ce suivi est réalisé en utilisant des caméras statiques et calibrées. Ce suivi de personne est effectué à l'aide du suivi de points d'intérêt SIFT en utilisant un filtre à particule spécifique, prenant en compte un certain nombre d'informations utiles telles que le résultats de la soustraction de fond et la mesure de fiabilité des descripteurs SIFT que nous proposons dans ce travail, en plus d'un framework d'association de données qui permet d'inferer le suivi d'objets à partir du suivi des points SIFT, et qui permet de gérer la plus part des cas possibles, particulièrement les occultations.

Enfin, la ré-identification de personnes est effectuée à l'aide d'une approche de type apparence globale. Une approche de l'état de l'art, permettant un traitement en temps réel mais fournissant des performances très variables en fonction des données fournies en entrée, est améliorée afin de fournir de meilleures performances tout en maintenant l'avantage du traitement en temps réel. Les ameliorations ont été introduites à différents niveau du traitement de l'approche originale, soit en remplac ¸ant certaines étapes initiales ou en ajoutant de nouvelles. Une partie "connaissance du contexte" a été introduite afin de rendre la signature visuelle plus robust aux changement d'orientation des personnes, assurant de meilleures performances de ré-identification dans le cas d'applications réelles.

Cette thèse fourni les contributions suivantes: Un détecteur de personnes qui propose (1) une approche généralisable de clustering pour les données néganives avant l'apprentissage du détecteur, accélérant la phase d'apprentissage et optimisant le détecteur dont le traitement devient plus rapide (temps réel) et plus précis (de meilleurs performances). Un framework de suivi d'objets, basé sur le suivi de points d'intérêts SIFT à l'aide d'un filtre à particules, en plus d'un processus d'association de données, qui propose: (2) une méthode de détection et de sélection d'un nombre constant et correctement réparti de points SIFT sur l'objet d'intérêt, permettant une meilleure représentation de l'objet et de ce fait, de meileures performances de suivi particulièrement dans le cas d'occultations partielles, (3) une méthode hybride de pondération de particules, qui améliore le suivi des points SIFT, prenant en compte la mesure de similarité du descritpeur SIFT ainsi que les résultats de la soustraction de fond d'une manière plus complexe

ABSTRACT

This thesis is performed in industrial context and presents a whole framework for people detection and tracking in a camera network. The three main processing steps are addressed: people detection, people tracking in mono-camera context, and people re-identification in multicamera context. High performances, system genericity and ease of deployment, and the real-time processing are the most important constraints which have guided this work. Some parts of the proposed work are already integrated and deployed in a commercial product while others are in prototype state and are planned to be integrated in future.

People detection aims to localise and delimits people in video sequences and static images. The proposed people detection is a full body one and it is performed using a cascade of classifiers trained using LogitBoost algorithm on region covariance descriptors. A state of the art approach, providing good performances but not applicable for real time is taken as basis and is optimized to process in real time while detection performances are slightly improved. The optimization scheme is generalizable to many other kind of detectors based on cascade of classifiers where all possible weak classifiers cannot be reasonably tested.

People tracking in mono-camera context aims to provide a set of reliable images of every observed person by each camera, to extract his visual signature for re-identification purpose. It provides also some real world information which are useful to improve re-identification process, as long as this mono-camera tracking is performed using static and calibrated cameras. It is achieved by tracking SIFT features using a specific particle filter, taking in account many useful information like background subtraction results and a proposed reliability measure of SIFT descriptors, in addition to a data association framework which infer object tracking from SIFT points one, and which deals with most of possible cases, especially occlusions.

Finally, people re-identification is performed using an appearance based approach. A state of the art approach, which performs in real time, but provides various performances depending on the input data is improved to provide better performances while keeping the real-time processing advantage. The improvements are introduced at different levels of the original approach, by replacing some of initial steps or by adding new ones. A context-aware part is introduced to robustify the extracted visual signature against people orientations, ensuring better re-identification performances in real application case.

This thesis makes the following contributions: A people detector which proposes (1) a generalizable clustering approach for negative data before people detector training, speeding-up training process and optimizing the trained detector which performs faster (real-time) and better (performance improvements). An object tracking framework, based on SIFT feature tracking by particle filter and data association process, which proposes: (2) a method to detect and select constant number of well distributed SIFT points on the object of interest for tracking, allowing better representation of the object and thereby, a better tracking performances especially in partial occlusion situations, (3) an hybrid particle weighting method, which improves SIFT points tracking, taking in account the SIFT descriptor similarity measure and the background subtraction result in a sophisticated way (not a simple binary weighting), (4) a data association process to detect all possible situations during tracking (including partial/full occlusions) and to manage each of them. This data association process use the tracked SIFT points localisation and their reliability measures (introduced in the previous step) to identify the state of each detected/tracked object, and to update the whole tracked object states, (5) a fast (real-time) occlusion management method, using tracked SIFT points, color information, and some other "real world" information (real dimensions, real velocity), learned during object tracking, to reacquire occluded objects after their reappearance. Finally, for people re-identification, a state of the art method is strongly improved by (6) a fast method for images alignment for multiple-shot case, to reduce people delimitation error in images and allow same parts comparison (7) the add of texture information to the computed visual signatures, by adding SIFT features as a new feature in the signature and by characterizing RHSP patchs by covariance descriptors encoding both color and texture information at the same time, (8) a method for people visible side classification, allowing to compute more accurate and discriminant visual signatures for each class, and allowing a better feature weighing (9) a method to use camera calibration information to filter candidate people who does not match spatio-temporal constraints (10) an adaptive feature weighting method to allow each re-identification query to focus on the more discriminant features, and to reduce or cancel local feature weights in some cases, according to visible side classification.

qu'une simpe pondération binaire, (4) une méthode d'association de données détéctant toutes les situations possibles durant le suivi (incluant les occultations partielles et complètes) et traitant chancune d'elles. Cette méthode d'association de données utilise la position des points SIFT et leurs mesure de fiabilité (introduite dans l'étape précédente) pour identifier l'état de chaque objet détecté/suivi et de mettre à jour les états de l'ensemble des objets suivis, (5) une methode rapide (temps réelle) de gestion des occultations, utilisant les points SIFT suivis, l'information couleur, ainsi que certaines données du monde réel (véritable dimensions, véritable vitesse), apprises durat le suivi de l'objet, afin de ré-acquérir l'objet occulté lorsqu'il réapparait. Enfin, pour la détection de personnes, une méthode de l'état de l'art is fortement améliorée avec (6) une máthode temps réel pour l'allignement des images d'une même personne pour minimiser les erreurs de detection des personnes (7) l'enrichissement de la signatuer visuelle en ajoutant l'information de la texture sous la forme de descripteurs SIFT et de matrices de covariance encodant la couleur et la texture en même temps pour la description des patches RHSP, (8) la classification de la face visible de chaque personne sur chaque image, permettant de calculer des signatures visuelles pour chaque classe, augmentant l'efficacité de ces signatures visuelles et permettant aussi une meilleure pondération de chaque type d'information utilisée (9) l'utilisation de l'information fourni par la calibration des cameras et le suivi mono-camera des personnes pour filtrer les candidats dont l'état ne respecte pas les contraintes spatio-temporelles (10) une method de ponderation automatique et adaptative pour mieux focaliser l'algorithme de re-identification sur l'information la plus discriminante, et de diminuler ou supprimer l'importance de certain descripteurs locaux.

ACKNOWLEDGMENTS

First, I would like to express my gratitude to my thesis supervisor, Franc ¸ois BR ÉMOND for the quality of the supervision, for helping me to improve my research skills, for guiding me while allowing me the necessary freedom to explore different directions. I would like to thank him also for having well managed the industrial context of the thesis while ensuring the research excellence, and for being patient with me, having good as well as bad work periods. I would like to greatly thank Carlo REGAZZONI, professor at Genoa university, and James FERRYMAN, professor at Reading university, for accepting to review my PhD manuscript and to be part of the committee. Their pertinent feedbacks have been enriching and constructive and will help me to improve my future work.

I would like to thank Monique THONNAT for accepting to be the president of the committee. Her many advices on my manuscript writing and for my defence preparation have been very useful. I would also like to thank Patrick BOUTHEMY, research director at INRIA Rennes for accepting to be my thesis examiner.

I would like to thank Digital Barriers company (previously Keeneo when I started my PhD) for hosting me and allowing me to perform this work. I would like to particularly thank Benoit GEORIS, ex-Director of Keeneo who opened me the doors of the company, Alberto AVANZI without whom this adventure would not have happened and who provided me with invaluable help in both technical and daily life aspects. I would also like to thank Marie-Claude FRASSON who was my manager in the company. I thank her warmly for her effective management, balancing performance requirement and the continuous search for good and warm working conditions in the team. Special thanks to Robert STAHR also for being always available and attentive, for his valuable technical assistance and also for being the big brother when I had to go through difficult periods. I would like to thank Laurent GIULIERI for his non-negligible help during my first year of PhD. I also thank all other colleagues and friends I met in the company, Kjetil JACOBSEN with whom I was jogging to clear my mind, Guilllaume BARRELET we are missing, me and the Green King, Audrey MOLITISANTI, Olivier LEROUX, Sebastien WEBO and Dario RAVARRO for their friendship and availability and finally, Thomas HERLIN, another big brother who has given me many good advices.

Many special thanks are also to all of my colleagues in the STARS team for their kindness as well as their scientific and technical supports during my thesis period, especially Phu and Slawomir. Thanks to both Julien(s) for the pleasant moments playing table football and sharing some common hobbies. All of them have brought a lot of warmness and friendship, and have made this PhD period more pleasant.

I would like to thank also Cécile PHAM-VAN at Digital Barriers and Jane DESPLAN-QUES at INRIA for their invaluable help in the management of administrative aspect of every day, allowing me to focus on my research work.

I finish these acknowledgements by the people who matter the most to me in this world. I thank my parents for their encouragement and support, without which it would have been impossible to finish. I thank and dedicate this work to my two brothers Yacine and Walid and to my sister Yasmina. Finally, I conclude this acknowledgements with my fiancée Amel who has always been behind me, pushing me forward, and particularly constantly put up with me, because I have not always been easy to live with during this period. Thank you all.

I would like to thank and to present my excuses to all the persons I have forgotten to mention in this section. With the rapid technological advances of the last 15 years and the easy accessibility of cameras and digital media storage, video surveillance is widely used and developed in all aspects of life in modern societies. The recent global security context, with the terrorist attacks in New York (2001), Madrid (2004) and London (2005) have obviously contributed to this growth, to ensure the protection of people and assets against terrorism acts, but not only. The recent attack in Boston (2013) demonstrates the effectiveness of video surveillance, thanks to which the two authors were quickly identified.

The prevention and repression of crime and delinquency, the protection of industrial and administrative buildings, securing airports, train stations and ports, road safety, people flow management and other needs have shown the necessity to increase the capacity of video surveillance of cities, businesses, and other concerned stakeholders.

Video surveillance systems are used both in live and differed modes. Live streams are viewed by video operators in real time, allowing rapid interventions if an event of interest is detected. But the high number of deployed cameras and the volume of incoming data hinder the live processing of all streams. For these reasons, many acquired streams are stored on digital media for a predefined time, and are used a posteriori to replay any event of interest and extract useful information, like person identification or evidence extraction. This continuous increase in the number of deployed surveillance cameras increases the workload of video operators in both modes. In live surveillance mode, each video operator has to monitor more streams, causing a decrease in efficiency. The large number of video streams assigned to each video operator increases the probability of missing important events when operator is not focussing on the right camera at the right time, because he/she is monitoring many screens, or by the fact that one screen is dedicated to many cameras and displays the viewed scene of each of them periodically. In addition to that, some recent medical studies have shown that after 20 minutes of focusing on surveillance screens, a video operator lose 90 percent of his concentration and vigilance. Most of the time, no particular event occurs so after a while of monitoring images where nothing happens, sleepiness of vigilance, fatigue and decreased concentration often cause failure in some important event detection.

For the a posteriori mode, especially for government security agencies like police, searching for a given person of interest in hundreds or thousands hours of recorded videos, provided by many cameras, requires to assign a large number of enforcement officers to this task, and requires a lot of hours or days to be performed. Dedicate so much manpower is often complicated, due to the high financial cost and the unavailability of competent agents who are assigned to other tasks. In addition to that, enforcement officers are humans and thereby, they are subject to the same problems of fatigue and decreased concentration similarly to video operators.

Introduction

To overcome all these problems and process all the video data provided as live streams and recorded videos, there is a growing demand of automated analysis and understanding of video contents. Reproducing human analysis and reasoning on observed events is becoming a critical field of research. This research domain covers many tasks like object detection and recognition, tracking in mono-camera and more recently in multi-camera contexts, gesture recognition, behaviour analysis and understanding, etc. All these tasks are used in many domains like robotics, entertainment, but also and in a large proportion, in security and video surveillance.

This thesis takes place in this context and consists in answering to this question: How an automatic system can characterize a person of interest, to track him/her in real time in a camera network and determine his/her localization in a huge volume of recorded videos? Digital Barriers is a security company which provides advanced surveillance technologies to the international homeland security and defence markets. It brings innovative thinking and solutions to the protection of most critical national assets, locations and infrastructure. It combines a long heritage in the security and defence sectors, with operational expertise and an understanding of how best to apply and deploy emerging technologies. Digital Barriers is specialised in delivering intelligent surveillance information from challenging environments. It conducts advanced research in computer vision for video surveillance purpose. Among the issues that the company is interested, we can mention securing sensitive sites against all intrusion, recognizing suspicious behaviours like loitering, detection of dangerous events like abandoned luggage in public places and tracking the person who has abandoned it across the camera network which covers this public place. It is also interested on forensics fields, by finding evidences and people a posteriori in recorded videos.

Context of Study

The work presented in this thesis is also directly related to an ITEA2 European project called ViCoMo (Visual Context Modelling), in which both STARS and Digital Barriers have participated. The ViCoMo project was a 3 years project, started in September 2009 and ended in November 2012. The ViCoMo project is developing advanced videointerpretation algorithms to enhance images acquired with multiple camera systems. By modelling the context in which such systems are used, ViCoMo significantly improves the intelligence of visual systems and enables recognition of the behaviour of persons, objects and events in a 3D view. The project enables advanced content and context based applications in surveillance and security, and transport/logistics with spin-offs in the consumer & multimedia domains.

Issues in People Detection, Mono-Camera Tracking and

Person Re-identification

From Practical Point of View: Large scale video-surveillance constraints

As mentioned before, the aim of this thesis is to provide an automatic system to reidentify people across a camera network (see figure 1.3). Many studies have been done on this topic, using various techniques and under different constraints. We can divide these approaches into two main types: biometric and appearance-based approaches.

The main types of biometric techniques for re-identification in video surveillance context are face recognition and gait recognition. We can mention also iris and fingerprint analysis as other biometric techniques, but these last ones can not be considered as a mean of (re)identification for wide scale video surveillance field, because they require a cooperation and voluntary actions from individuals evolving in the monitored environment. Indeed, capturing the iris image for analysis requires the person of interest to position his eye close to a specific sensor and directly in front of it (See figure 1.4).

Similarly, the capture of fingerprint of the person of interest requires from him to put his finger on a specific sensor (See figure 1 recognition) the collaboration of observed people. They can be applied in large areas with many people and many entrance and exit possibilities as long as few constraints are verified.

Many approaches have been proposed for face recognition [Bauml 2010[START_REF] Belhumeur | [END_REF][START_REF] Kirby | [END_REF], Lee 2003a] and gait recognition [Wang 2003b, Lee 2003b, Chellappa 2007].

When the conditions and constraints are satisfied, some of these approaches provide high performance, especially for face recognition. Unfortunately, some of these constraints are not satisfied by most deployed surveillance systems.

First, for economic and optimization reasons, most of surveillance cameras cover large fields of view, providing images with small persons/objects of interest, so the extractable information is insufficient to provide acceptable performance after processing (face size under the minimum required size, indistinguishable gait, etc.).

Second, the large number of deployed cameras in a given site and the distance between this site from the monitoring or storage location may overload transfer network.

For this reason, the streams are generally compressed by the cameras, sent to the processing or storage location, before to be decompressed for display or for processing.

The storage of huge volume of videos also requires compression. Most of compression/decompression techniques introduce a loss of information and some noise. These issues directly impact the performance of face and gait recognition.

Finally, biometric approaches are significantly dependent on the point of view of the camera and the orientation of the person of interest with respect to this camera. If the face is not visible (the person is seen from behind or from the side), face recognition The person who is seen from behind (red rectangle) cannot be processed. The last person seen from the side (orange rectangle) is in the limit of constraints and processing can fail. Here, collaboration of people is not required. cannot be performed.

For these reasons, we have oriented our work on appearance-based approaches, which have less constraints than biometric ones, and are more adapted to the video surveillance requirements.

Achieving people re-identification with good performances requires to use reliable information as input. For this reason, this thesis focuses on three main parts. The first part consists in people detection on static images and video sequences. The second part consists in people tracking in mono-camera context. The last task, which is the final aim of this thesis, consists in people visual-signature extraction and comparison for re-identification.

These three parts are interlinked and the performances of any of them affect directly one or both other parts. The dependence of these three parts is shown in the figure 1.8.

People detection on static images and video sequences provides/validates the targets to track to mono-camera tracking algorithm in collaboration with a background subtraction algorithm (this collaboration will be detailed in chapter 5) and provides the candidates for re-identification to the re-identification algorithm.

Mono-camera tracking algorithm provides the different locations and images of a given person through out time to the re-identification algorithm, allowing it to learn a robust visual signature for this person, taking into account observed variations of his/her images.

In the same time, re-identification algorithm allows mono-camera tracking algorithm to deal with occlusions, by maintaining tracks of partially occluded objects thanks to their partial visual signatures and by reacquiring lost targets using their whole visual signatures.

Two important points can be noted:

The distinction between static images and video sequences in people detection, which will be more detailed in the corresponding chapter, is mainly due to the features used in each case and to the type of addressed problems.

Even if the final aim of this work is mainly dedicated to people detection, monocamera tracking and re-identification on video sequences, the second part concerning mono-camera will be generalized to all types of objects of interest, and not only people. This choice is motivated by the fact that object tracking is an important task in many security applications. Digital Barriers is interested by this generic object tracking algorithm for other purposes (for example: car tracking). Never- In this figure, two cameras are used but the architecture can be generalised for a network of many cameras.

theless, we will take care to report and discuss the specificities of people tracking whenever it is necessary.

From Computer Vision Limitations Point of View

Automated person detection finds its applications in many areas including humanrobot interaction, surveillance, pedestrian protection systems, automated image and video indexing. In the video surveillance context, detecting the whereabouts of humans is the first requirement if any event involving people has to be detected. This ranges from simple events like intrusion in forbidden area to complex events like behaviour analysis.

Object tracking is the task of following one or more objects in a scene, from their first appearance to their exit [START_REF] Forsyth | [END_REF]]. An object may be anything of interest within the scene that can be detected and depends on the requirements of the application.

Given a sequence of image frames to track a set of objects, these objects correspond to sub-images, in each frame. In general, in a dynamic environment both background and objects may vary. In principle, solving this general unconstrained problem is hard. One can add a set of constraints to help solving this problem. The more the constraints, the easier the problem is to solve.

Person re-identification consists in determining if a person of interest has already been observed over a network of cameras (see figure 1.3). As mentioned before, this task can be performed using different approaches depending on constraints and available information, using biometric-based techniques like face and gait recognition or using appearance-based approaches. Due to the cited reasons before, concerning the difficulty to ensure biometric approach constraints, appearance-based approaches are more suitable for large wide video-surveillance purpose. The global appearance of an individual has to be modelled to re-identify it. This modelling is the last topic of our work.

All these tasks are complex to achieve, due to the numerous challenges which affect their performances. Some of these challenges are common to the three tasks, and affect them with different degrees, and some other challenges are specific to a given task. In the following paragraphs, we present the most important challenges and we discuss how they affect our tasks.

• Physical variation of persons: Person appearance can vary greatly. People do not share a common body size, color, texture, and the appearance is highly influenced by the clothing they wear. This is a specific issue for people detection only, and can be considered as an advantage for mono-camera tracking and re-identification Introduction tasks.

In fact, People detection consists in finding the most common characteristics of all people which are absent in other object classes. This large variations in physical aspect requires to "enlarge" the model of persons and thereby, increase the risk to have false positives (objects which are not people but detected as such because their appearance is close to the one of a person). In the same time, trying to avoid false positives by restricting the person model may cause miss detection of persons with physical appearance out of the bounds of the model. This issue is an important but not a critical one. Most of approaches ignore color and textures, and are based on external shape or part detection in gray scale level domain. The size variation is managed by a multi-scale detection.

Concerning mono-camera object tracking and re-identification, this physical variation can be considered as an advantage due to the nature of this two tasks. unlike people detection which consists in separating a class of objects from all others, mono-camera object tracking and people re-identification require to separate an individual entity from all other entities, from the same class or not. The more physical variation are, the more likely to better characterize an individual entity is.

• Body deformations: For a task that depends highly on the shape of a person, body shape deformations can adversely affect the performances. This issue impacts the three tasks, but at different degrees.

For people detection, it is a critical issue. It is hard to build a unique model for all possible body shapes. Body deformation introduces a variation in parts localization/visibility. A people detector which is build to detect a standing person will probably fail to detect a crouching or a sitting person. A people detector based on part detection can have more chances to detect than a global person detector, but it can also fail if some important parts are not detected (legs of crouched person are hard to identify). The most used solution for this issue is to build as many detectors as identified body shapes. This solution implies a very important detection time, due to the necessity to use all detectors without any a priori information. Another issue of this solution is that some intermediate states of the body shape can be miss-detected. In fact, to pass from the standing state to the crouching one, the body part algorithm acquires many intermediate states which can not be included in existing models.

For appearance-base people re-identification also, this issue is an important one.

Most of appearance models are built by taking into account the localisation of each interesting feature on the person body. A learned visual signature of a standing person in a camera can be useless if this person appears in another camera in different position (sitting for example). Some interest features can be observed in different position than extracted/learned ones, or can not be visible.

For mono-camera tracking task, this issue can be relatively well managed in most of cases. If the video frame-rate is sufficient, the body deformation occurs continuously, and the whole deformation can be divided into small successive deformations, which can be recognized and managed. With a correct similarity measure of the used features for tracking, successive states of deformation can be linked and the tracked target identity can be maintained.

• Illumination: For a task that depends on the lighting conditions, varying illuminations and shadings in different environments can affect the performances.

For people detection, this issue is a medium one. Depending on the used features for people modelling and the use of illumination normalization step, most of state of the art approaches deal more or less well with this issue.

For mono-camera object tracking, this issue is more or less important, especially if the light variation occurs suddenly. Depending on the robustness to light changes of the used features for tracking, the similarity measures can greatly vary and cause tracking failure. Nevertheless, the temporal continuity of the processing with a sufficient frame-rate and the localisation of tracked objects before and after illumination changes provide a useful way to decrease failure probability.

For people re-identification, this issue is a critical one, especially in the case of cameras without overlapping field of view. The same person observed with two cameras under different illumination conditions can have two different appearances. This phenomenon is extreme in the case of bad light acquisition. Unfortunately, the temporal continuity of mono-camera object tracking is missing here, increasing the importance of this issue.

• Viewpoint changes: Depending on which angle people/objects are viewed, different shapes can be observed with varying aspect ratios.

For people detection task, this issue is similar to "Physical variation of persons" one.

The variation in viewpoint implies a variation in appearance and size of persons in addition to aspect ratio changes also. For the reasons cited in the "Physical variation of persons" issue, this issue is a critical one.

For mono-camera people tracking, this issue is a lower one. In the case of static cameras, this issue does not exist. In the case of moving cameras, if the motion is uniform or if the frame rate is sufficient, the temporal continuity of information Introduction allows to deal with successive small variations, and thereby, maintain the track objects identities.

For people re-identification, this issue is more important. The observed features in a given camera can not be visible in another camera, or can be observed with different size and aspect ratio, causing the failure of the re-identification.

• Crowded scene: The number of persons in the scene is an important parameter for the task performances.

For people detection, this issue can be a medium or an important one, depending on the position of all these persons. The number of persons itself in the scene is not an issue. If the scene is crowded but the persons do not occlude each others, the number of persons is not a problem. Each person is detected independently as long as their important features from the detection model are visible. On the other hand, if the important features are occluded, the detection can fail for the occluded persons.

For the mono-camera object tracking, this issue is a critical one. The high number of persons in the scene has two negative effects: first, the probability of occlusion increases with the increase of person number. Second, the high number of persons increases the risk of permutations and tracking errors due to the high probability of having close models for tracked persons/objects.

For people re-identification, this issue is an important one. The high number of persons in the scene increases the number of candidates for each re-identification query, and thereby, it increases the probability of error. This high number increases the probability to have similar visual signatures too.

• Background clutter: Sometimes background structures exhibit similar texture and shape as the one of a person or tracked objects, making distinction difficult.

This issue does not concern directly the re-identification task as long as this last one gets its inputs (request and candidates) from the two other algorithms (people detection and mono-camera object tracking), so a failure in this task because background clutter is in reality a failure in people detection or in mono-camera people tracking.

For people detection task, this issue is an important one. The desire to obtain a generic person model with all possible physical variations and a robust model against illumination changes is generally satisfied by ignoring some kinds of information like color or internal textures and by accepting large range of shape deformations. This enlargement of the model causes some wrong classification when a background part has similar shape/model to the person one (a traffic signal, tree leaves and branches, etc.).

For mono-camera object tracking, this is also an issue, but less important than for people detection. There is always a risk that a tracked model clings to a background object when the tracked object passes near this background object if this last one has a high similarity with the tracked object, but thanks to the largest amount of information which can be used to model the tracked object, this risk is minimized.

• Occlusions: Sometimes people/objects are partially or completely occluded by objects they are carrying, by overlaps with other people/objects, or by structures in the environment.

For people detection, this issue is important or critical depending if the occlusion is partial or full one. People detection becomes impossible by complete occlusion due to the unavailability of any information. In the partial occlusion case, this detection becomes difficult and depends on the used person model for detection.

If all the features of interest of the model are visible despite the partial occlusion of the person, the detection can be performed. If only some of these features are visible, it will depends on the way the detection is performed, i.e. if an inference/extrapolation process is available. This kind of process is the most difficult to provide.

For mono-camera object tracking, both partial an full occlusion are important issues, but the full occlusion is more important issue. In fact, in case of partial occlusion, depending on the used features and techniques for tracking, the still visible part of the tracked person/object can be sufficient to keep the tracking performing until the person/object re-appears entirely. In the case of full occlusion, this issue becomes a problem which can be solved by a collaboration between re-identification and spatio-temporal coherency. The results of full occlusion management depends on the re-identification performance, but also on many other parameters like occlusion duration, scene configuration (possible exits during occlusion), etc.

People re-identification task is concerned by this issue only for partial occlusion case. In full occlusion situation, it means that the request/candidate person(s)

are not available yet. The re-identification is performed when the request person reappears. The partial occlusion case is an important issue. If some important features of the visual signature are not visible, the re-identification fails.

• Shadows and reflections: Depending on illumination conditions, light angle, floor/wall smoothness, shadows and reflection can be important issues.

For people detection and mono-camera object tracking, this issue is critical. Shadows and reflections are difficult to handle during people detection and object tracking. Depending on the features (such as motion, shape and background) used for a people detection or the object tracking, a shadow on the ground or reflected by a wall/window may behave and appear like the person/object that casts it.

This issue does not concern directly the re-identification task as long as this last one gets its inputs (request and candidates) from both other algorithms (people detection and mono-camera object tracking), so a failure in this task due to shadow non-removal or reflection is in reality a failure in people detection or in monocamera people tracking.

• Different sensor response: This issue concerns people detection and re-identification tasks.

People detection is highly affected by this issue. State of the art people detectors, which are based on an off-line training, are strongly dependent on the condition of acquisition of training images. One of the most important acquisition condition is the sensor response. People detectors witch are trained using some specific cameras may not perform correctly in other situations where different kinds of sensors are used.

mono-camera object tracking is a mono-camera processing. It means that the process is performed on each camera independently, and thereby, it is not affected by external (other camera) information.

For people re-identification, this issue is a critical one. In a camera network, nothing ensures that all the cameras have the same model, and event if it is the case, the sensors may have small or large difference in their responses. The most important issue is the color response of the sensors. The same person with the same clothes can be rendered in different ways by two different cameras. Sometimes, a red pullover can be displayed as red in one camera and orange in an other one.

The same for white and yellow colors. A practical example will be presented in the chapter 6.

• Computational cost: This is an important issue for real-time processing for live video-surveillance purpose.

For people detection, techniques and methods that achieve state of the art detection usually require heavy computation time (for training and for detection) com-pared to trivial person detection method. This is a difficulty especially in video surveillance context, where it is required to have reactive response acceptable for humans. Balancing detection performance with computational requirement adds to the challenges faced in person detection. Time detection can be constant or vary according to the number of persons, depending on the used technique. For example, [START_REF] Dalal | [END_REF]] approach (HOG with SVM) takes the same time to test each candidate region while [START_REF] Tuzel | [END_REF]] approach takes different times to test each candidate region, depending on the depth of the reached level of the cascade of classifier before rejection. Nevertheless, both kinds of approach are slow.

For mono-camera object tracking, the computational cost depends on three main parameters: first, the complexity of the used model for tracked targets. Using global color histogram is faster than computing a set of features with various information types. Second, the tracking technique. A simple Kalman filter is faster than a complex particle filter. Finally, an important parameter is the number of targets which are tracked simultaneously. The more the tracked targets are, the higher the computational cost is.

For re-identification task, computational cost is an important issue too. The complexity of the computed signature impact directly the processing time. The number of candidates for a re-identification is an other important parameter, which can make processing time explode if no filtering candidate step is performed before.

The issues mentioned above show that a successful person detection based on a single sensor is very difficult. For real world scenarios, more promising approaches combine more inputs from more than one sensory channel. In multi-modal person detection, detections from the different sensors (collaboration between visible and IR cameras, fusion of several cameras with joint fields of view, etc.) can be used to cross validate the mono-modal detection to obtain a robust detector. Features that are not captured by one sensor can be captured by another one making the detection more invariant to the above listed challenges. Unfortunately, in video surveillance context, this kind of improvement, based on multi-sensors, can not be ensured most of the time.

These issues are also significant to both single-object tracking and multi-object tracking. However, multi-object tracking also requires to solve some other issues e.g. modelling multiple object interactions. Tracking methods should be able to distinguish different objects in order to keep them consistently labelled. Although during the last few years, there has been a substantial progress towards moving object detection and tracking. But tracking an object in an unconstrained, noisy and dynamic environment still makes this problem a central focus of research interest.

A summary of the different issues and challenges, and their impact on people detection, mono-camera tracking and re-identification tasks is presented in the

× × × × × × × × × Different sensor response × × × × × × Computational cost ×× ×× × × × Table 1
.1: Summary of different issues and their impact on each task. ×: low impact. ××: medium impact. × × ×: high impact. No cross: not concerned by the issue.

Hypotheses and Constraints

This thesis has been performed with some hypothesis. Some of them are common to the three presented tasks and some others are dedicated to specific tasks. These hypothesis are:

Static cameras: Due to the industrial context of Digital Barriers, and the use of background subtraction algorithm as a provider of targets to track for mono-camera object tracking, only static cameras are considered in our study.

CIF images:

For the same reason (industrial context of Digital Barriers), the necessity to run multiple analyses per server (each camera has it's own process), the network limitations (bandwidth), and for some other constraints, small size images (CIF) are used most of the time for the processing, to comply with these constraints. This implies that all the developed algorithms have to be robust enough to deal with low information amount (small people/objects sizes and resolution). When largest images are used, the performances are equivalent or better due to the availability of more information/details.

Calibrated and not calibrated cameras:

All used cameras for our work are calibrated. Detected objects by background subtraction algorithm are classified using 3D

1.5 Contributions world information, and requires calibrated cameras for that. In addition, camera calibration allows to learn fast and simple real world information of tracked objects, allowing a fast occlusion management.

For people detection part, the calibration of cameras is not critical. The detailed approach can be performed on images/videos without the availability of camera calibration, but the use of camera calibration speeds up greatly the detection process in a way which is explained in the chapter 5.

Sufficient frame-rate: This constraint is concerning exclusively mono-camera object tracking task. To be able to manage all temporal variations, a minimum frame-rate of 8fps is needed.

Contributions

Our goal is to propose the innovative ameliorations on state of the art algorithms to obtain an operational (effective and efficient) framework for tracking people through a camera network. This framework has to be a turnkey system, by being as generic as possible and by do not requiring new parametrization for each deployment case while it has to provide high performances and to process in real time, due to the industrial constraints.

The presented work brings 11 significant contributions comparing to state of the art. The first contribution consists of a general framework to process the three successive steps which are "people detection", "mono-camera tracking" and "people reidentification" (See figure 1.8). One contribution is related to people detection part, four contributions to mono-camera object tracking and finally five contributions to people re-identification. We have evaluated and compared our people detector on four datasets: INRIA, Daim-lerChrysler, Caltech and CAVIAR datasets.

Contribution to People Detection

Contributions to Mono-Camera Object Tracking

-A new method for SIFT feature detection and selection for object tracking:

We propose a new method to ensure an optimal representation of the tracked object of interest using SIFT features. This method allows better object tracking especially for partial occlusion cases. This work has been published in ICDP 2011 [START_REF] Souded | [END_REF]].

-An hybrid particle weighting method for SIFT feature particle filtering: We use a particle filter to track all SIFT features representing the object of interest. Our proposed method allows to weight the used particles using both SIFT descriptors similarity measures and background subtraction results in a sophisticated way (not a binary weighting), dealing with various background subtraction qualities. This work has been published in ICDP 2011 [START_REF] Souded | [END_REF]].

-A data association framework for object tracking: Once SIFT features are tracked from previous frames to the current one, our proposed data association method allows to infer object tracking state from SIFT features one, detecting and managing the several cases which may occur during object tracking (especially occlusions). This work has been published in ICDP 2011 [START_REF] Souded | [END_REF]].

-A fast occlusion management method: We propose a fast methods to deal with full occlusion issue. it consists in learning real world information concerning the tracked object (dimensions and velocity variations), using the dominant colors extracted during the tracking and finally, using the tracked SIFT features as additional information for object re-acquisition after occlusion. This work has been published in ICDP 2011 [START_REF] Souded | [END_REF]].

We have evaluated and compared our mono-camera object tracking algorithm on four datasets: PETS 2001, ETISEO, Caretaker and CAVIAR datasets.

Contribution to Person Re-identification

-Fast image alignments before signature computing for multiple-shot case: We propose a fast method to align automatically extracted images of each person before visual signature computing. The people detection and delimitation errors may provide some truncated or badly centred people images, which alter the computed signature if no processing is performed to deal with this issue. Our method allows to correct the delimitation of slightly bad delimited people, to remove the image 1.6 Outline with significant errors, and to align the kept images to improve the computed visual signature and the comparison.

-Use of texture information in addition to color: The baseline method we take for our work [START_REF] Farenzena | [END_REF]]) exclusively uses color information for visual signature computation. We propose to introduce texture in addition to color in two separate ways: replacing the color characterization of the signature component (RHSP features) by a color+texture characterization using covariance descriptors, and adding SIFT features to the final signature. The SIFT features are provided by mono-camera object tracking algorithm when the whole system is used.

-Visible side classification for more reliable signatures comparison: People appearance may be different according to their visible side, a unique visual signature for each person may not be efficient enough. We propose a method to detect and classify the visible side of observed people in 8 classes, based on our mono-camera object tracking algorithm, and assigning a sub-signature to each observed class, and providing a more precise signature comparison method.

-Spatio-temporal coherency filtering method: Depending if the considered cameras share overlapping fields of view or not, we propose for each case a method to exploit the camera calibration information to reduce the number of candidates for a re-identification query by filtering incoherent spatio-temporal matching, and to weight the appearance based matching by a real world distance weight in the case of overlapping field of view.

-Adaptive weights for signature components: To make the re-identification approach generic, we propose an adaptive weighting method for all used features, taking in account the amount and the quality of available information (Color/Texture) and the considered people visible side if this information is available.

We have evaluated and compared our mono-camera object tracking algorithm on four datasets: VIPeR, ETHZ, iLids and CAVIAR datasets.

Outline

This PhD manuscript is organized as follows:

• Chapter 2 presents a state of the art for each of the three studied domains: People detection, Mono-camera object tracking and People re-identification.

• Chapter 3 presents a general overview of the whole framework for person reidentification as a complete processing chain, starting by people detection, followed by people tracking in each single camera independently, and finally people re-identification through a camera network. The dependencies and the collaboration between these three processing steps including feedback are presented. After that, an overview of the proposed approaches for each processing step is presented.

• Chapter 4 details the proposed approach for people detection on static images and video sequences. First, a summary of Tuzel et al. [START_REF] Tuzel | [END_REF]] approach for people detection and its improvement by Yao et al. [Yao 2008] are presented because our work is based on these approaches. The choice of these approaches as a basis for extension and their issues are explained and detailed, introducing and justifying our contributions, consisting in an optimization process to speedup classifier training and detection process, in addition to improving detection performances.

• Chapter 5 presents the proposed approach for object tracking in mono-camera context. As mentioned in the introduction, a generic object tracking algorithm is targeted first, due to the industrial needs of Digital Barriers. We highlight all specific parts restricted to people tracking when this is the case. The presented object tracking is based on SIFT features using Particle Filtering approach, followed by a data association reasoning stage to lead to a complete object tracking. Several contributions are presented at different levels of this process.

• Chapter 6 presents the proposed improvements to state of the art algorithms for people re-identification. The baseline approach is explained, and its limitations are highlighted. The approach is based on the symmetry-driven accumulation of local features. The contributions are detailed and tested to show the improvements they provide.

• Chapter 7 is dedicated to experimental results and benchmarking with state of the art studies. Each of the presented processing steps is evaluated on several dedicated benchmarking datasets and compared with state of the art approaches.

The proposed approaches and contributions of each processing step are validated by comparing with the state of the art approaches performances.

• Chapter 8 presents the concluding remarks and limitations of the thesis contributions for each processing step. We also discuss about the short-term and long-term perspectives of this study. The second section concerns the state of the art in mono-camera object tracking, which can be classified according to two criteria: "how to represent/model the tracked object" (i.e. which features to characterize it) and "how to update search for its model over time?". The object modelling is performed in several ways, according to the used information: color, shape, texture and motion. Tracking techniques can be divided into two main categories: deterministic and probabilistic approaches.

The last section concerns the state of the art in people re-identification. Approaches are divided into two main families: biometric and appearance-based approaches. The biometric approaches include iris recognition, fingerprint recognition, face recognition and gait recognition. The appearance based approaches are classified into single-shot approaches and multiple-shot approaches, according to the number of used images per person. In both of them, the appearance modelling is performed following two main kinds of methods: Feature oriented methods and Learning methods. Some recent approaches use context information instead of focusing on people on interest images only. At the same time, it is one of the most challenging problems in computer vision due to the large number of possible situations, including variations in people appearance and poses. These challenges are further augmented in video surveillance applications due to computational time requirements for a reactive system, especially when this detection is not the final aim, but one step in a more complex processing like in our case.

People detection

Due to its importance and its several challenges, People detection is an active research area with a rapid rate of innovations.

In this section, different people detection approaches from the state of the art are The template matching approaches aims at extracting a generic template for person's class using pertinent features. Detection is performed using a direct template matching procedure, using several proposed matching measures and distances.

To help structure the flow of this chapter, figure 2 for the observed scene are acquired, then the searching areas on the images are defined as the candidate regions, either by searching on the whole image with any a priori knowledge, or by targeting specific regions using real world information. From these candidate regions, discriminative and pertinent features from the same type than the used ones for training are extracted for classification. The trained classier is finally used to decide whether the evaluated candidate region corresponds to a person or not.

In the following paragraphs, we present the used techniques in the main state of

Pertinent Features

An image consists in a array of pixels. Using each pixel independently does not allow to extract any information about image content. For this reason, many approaches for visual features extraction from groups of pixels are proposed in the state of the art.

These features enable us to extract meaningful information from image. In this section, different image features that are used in visual person detection are briefly presented.

Haar-Like Features

Haar Wavelets were first introduced in the context of Object Detection in late 90s

by Papageorgiou et al. [Papageorgiou 1998]. Viola and Jones [Viola 2001] adapted the idea of using Haar wavelets and developed the so-called Haar-like features. They introduced the notion of Integral Image so as to compute these features in a fast way.

The Haar-like features encode the relationships between average intensities of neighbouring regions along different orientations capturing edges or changes in texture. This makes them suitable to capture the structural similarities between various instances of a class. Figure 2.2a shows the three types of 2-dimensional Haar-like features used by [Oren 1997]. These features capture change in local intensity along horizontal, vertical and diagonal directions. When applied to images, the value of a two rectangle feature is the difference between the sum of the pixels lying in the unshaded area with the sum shows the complete Haar-like feature template used by [START_REF] Lienhart | [END_REF]].

Edge Orientation Histograms (EOH)

Silhouette and edge information are important features to discriminate a person in images. To encode these information, Edge Oriented Histograms (see figure 2.3) have been proposed initially for face detection by [START_REF] Levi | Learning object detection from a small number of examples: the importance of good features[END_REF]]. These features not only maintain invariance to global illumination changes, but also capture geometric properties that are difficult to capture with other features. Later, Edge Oriented Histograms have been used for people detection. In [START_REF] Gerónimo | [END_REF]], a combination between Haar-like features and Edge Oriented Histograms is used as discriminant feature for classification.

Histogram of Oriented Gradients (HOG)

Another feature for silhouette and edge information encoding, called Histogram of Orientation Gradients, is proposed by [START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF]] for people detection. The feature extraction is more complex than in Edge Orientation Histograms, increasing the discriminative power of the descriptor while ensuring a certain degree of invariance. As described in [START_REF] Dalal | [END_REF]], HOG descriptor computation is done in five steps:

1. A global image normalization equalization, using a gamma compression, is performed to reduce the effects of local shadowing and influence of illumination variation effects.

2. Computation of first order image gradients.

3. The image window is divided into small spatial regions, called "cells", and a local 1D histogram of edge orientations with K orientation bins over all the pixels in the cell is accumulated. Each edge pixel contributes to each orientation bin with a value proportional to the magnitude of its orientation.

4. A normalization step is carried out by accumulating a measure of local histogram "energy" over local groups of cells called "blocks". Each cell is normalized with respect to the block which it belongs.

5. The final HOG descriptor of the whole detection window is obtained by concatenating all HOG descriptors of all blocks of a dense overlapping grid.

The HOG feature extraction is depicted in Figure 2.4 taken from [START_REF] Dalal | [END_REF]].

Four variants of the HOG descriptor have been presented by the authors. The difference between them lies in the shape of considered cells. These four variants are:

Rectangular HOG(R-HOG), which is the original one, Circular HOG(C-HOG) where the cells are defined into grids of log-polar shape. Bar HOG where the descriptors are computed similar to the R-HOG, but use oriented second derivative filters rather than first derivatives and Center-Surround HOG which use a centre-surround style cell normalization scheme.

Many other approaches for people detection, using HOG descriptors, have been proposed. We can cite [Zhu 2006b[START_REF] Corvee | [END_REF], Bertozzi 2007]. They mainly differs in the way that the HOG descriptor is used (describing the whole person image or body parts independently) or in the used classification method (SVM, boosting, etc.).

Local Binary Pattern (LBP)

Local Binary Patterns is texture encoding feature. It is a particular case of the Texture Spectrum model proposed in [START_REF] Wang | [END_REF]] and [START_REF] He | [END_REF]]. It has been first described in [Ojala 1996]. The original version of the local binary pattern feature for each pixel is based on a 3 × 3 pixel block of an image. The pixels in this block are thresholded by its center pixel value, multiplied by powers of two and then summed to obtain a label for the center pixel. As the neighbourhood consists of 8 pixels, a total of 2 8 = 256 different labels can be obtained depending on the relative gray values of the center and the pixels in the neighborhood. A more generic LBP feature is proposed in [Ojala 2002]. It allows more information extraction from variable circular neighbourhood of the center pixel, according to two parameter which are the radius of circular neighborhood "R" and the number of considered neighborhood points "P" (see figure 2.5). Note the visual similarity of the shape contexts for •, ⋄, which were computed for relatively similar points on the two shapes. By contrast, the shape context for ⊳ is quite different. (g) Correspondences found using bipartite matching, with costs defined by the χ 2 distance between histograms.

Many people detection approaches

Region Covariance Descriptor

Region covariance descriptors have been first introduced for people detection by Tuzel et al. in [Tuzel 2006]. Each image pixel can provide a large amount of basic infor-mation like its coordinate, its intensity and color values, its first and second derivative values according to X and Y axis and many other values obtained by applying several filters and operators. By extracting a d-dimensional vector of features from each pixel of a given image region, it is possible to compute the variance of each type of considered feature and the correlation between these different features, and encapsulate all these information in a single d-dimension square matrix which will describe the considered region. More details concerning the way to compute this descriptor and its characteristics, in addition to all the related metrics are presented in the chapter 4 to avoid information redundancy, since our proposed method is based on this descriptor.

In [START_REF] Tuzel | [END_REF]], 8-dimensional covariance descriptors are used for the people detection training. The computation of these region covariance descriptors is speeded up using Integral Images. A cascade of classifiers is trained using a boosting scheme in Riemannian Manifold, due to the nature of covariance matrices. This approach provide interesting detection performances but it requires non-negligible processing time, due essentially to the eigenvectors decomposition which the basis of all metrics computation, and which is directly proportional to the region covariance descriptors dimension.

Yao and Odobez improve [START_REF] Tuzel | [END_REF]] method by introducing three modifications to the initial approach in [Yao 2008]. First, the second derivative features on X and Y axis which were used in [START_REF] Tuzel | [END_REF]] 8-dimensional features are replaced by two other features, related to the background subtraction, speeding up the detection in video sequences. Then, they greatly speed up the detection by building classifiers with 4-dimensional region covariance descriptors instead of the 8-dimensional covariance descriptor in [START_REF] Tuzel | [END_REF]], using the best subset of 4 features from the initial set of 8 features, for each considered region of interest. Finally, they increase the discriminative power of extracted features for each region of interest by concatenating the mean feature vector of the same region upon all positive training images to the projected covariance matrices on vector space before regression computation. These improvements, their justification and the way in which they are performed are detailed in chapter 4.

Candidate Region Selection

This processing step concerns the detection process only as long as the positive training dataset are provided by labelled person images.

The candidate region selection is the first step for person detection in images. In general, images contain various environment objects. They may contain none or many people. In the case of images containing people, a person is defined by a sub-image region containing a set of specific features. It is therefore necessary to select and test "some" candidate image regions to check if they correspond to persons or not. This can-didate selection presents two main advantages: First, the processing time would be decreased by checking only selected regions and second, possible false positives would be filtered out by this selection. In the same time, this selection can lead to miss-detections if relevant image regions are not checked.

Candidate region selection is done according to specific criteria and can be divided into two categories: Dense searching by sliding window and real world a priori knowledge (camera calibration).

Dense Searching by Sliding Window

This method is generally used when no information is available concerning the scene in the image. It consists in scanning the whole image using a sliding window (see figure 2.8 (a,b)). The size of the scanning window as well as the displacement step are dependent on some criteria like the sensitivity of the used features with respect to small/large shifts and rescales. To ensure the detection of people with different sizes in the image, a multi-scale scan is performed. Some approaches rescale the image by keeping the search window with a constant size, corresponding to the positive training images size, while others approaches rescale the scanning windows to detect people at various scales on the same image.

The choice between these two policies depends on the constraints on the types of used features and classifiers. For example, [START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF]] construct an image pyramid by scaling the input image by a factor of 1.2 and use a scanning window with a constant size of 64x128 pixels. They shift this scanning window by 8 pixel in both axis (the scanning shift is constrained by HOG cell dimension which is 8 pixels. Only multiples of cell dimension can be used as browsing step). The size of scanning window corresponds to the size of the images which were used to train a SVM classifier. Due to the nature of the SVM classifier, it is not possible de rescale the scanning window because the corresponding SVM classifier has to be adapted to the new window size, which is not a trivial operation (SVM hyperplane dimension is dictated by the dimension of the whole image HOG feature vector, which depends on the image size).

In opposition to this example, Tuzel et al. [START_REF] Tuzel | [END_REF]] have tested and compared both possibilities in their approach: First they rescaled input image at different scales and applied a scanning window with constant size on each scaled image, and second, they keep the original image for scanning and applied different detection window sizes.

In their approach, the nature of region covariance descriptor that they used as a feature allows to rescale classifier with a negligible variation. This is due to the fact that the information contained in a given region in term of feature variances and correlation between them does not vary by a uniform rescaling on both axis theoretically. In practice, a negligible variation occurs du to the discrete (non-continuous) nature of image pixels, and the information modification introduced by anti-aliasing filtering after rescales (bilinear filtering for example).

Note that it is better, when it is possible, to rescale the scanning window (and adapt the corresponding classifier) instead of rescaling input image. The processing time can be highly reduced by this choice. In fact, rescaling input images, in addition to the necessity to re-extract the whole image raw feature vector at each scale, are high time consuming if the number of scales is important. This rescaling time must be multiplied by the number of images in case of videos.

Rescaling scanning window is faster, even by training a classifier on one images size only and rescale this classifier directly using its mathematical properties (e.g. region covariance descriptors) or by training several classifiers on predefined set of image sizes (predefined set of detection scales). Even if the classifier adaptation (rescaling) requires computational time, this operation can be done off-line, before the on-line processing.

Unlike input images content that change over the time (and thereby the contained feature values), a classifier for a given scanning window size is constant (it corresponds to a fixed model). All the needed classifier scales are computed and stored before processing and directly used during detection process.

Filtering by Real World Knowledge

This approach uses a kind of scanning window similar to the previous method, but the main difference is that scanning is not performed on the whole image, and the searching scales are highly reduced.

This approach is strongly constrained by the camera calibration information availability, and is based on the assumption that people are on the ground floor. For a calibrated camera, rectangular regions corresponding to the aspect ratio of a person are placed on the ground floor of the 3D world up front and projected onto the image using camera transformation matrix. These regions then constitute the candidate windows for further processing (see figure 2.8 (c,d)). These approach has been applied in pedestrian detection from a vehicle by Gavrila et al. [START_REF] Gavrila | [END_REF]] and Gerónimo et al. [START_REF] Gerónimo | [END_REF]]. Using this approach, the number of possible regions for subsequent steps is highly reduced. Gerónimo et al. [START_REF] Gerónimo | [END_REF]] have shown that the performance of this candidate generation scheme is very dependant on the accuracy of the camera calibration parameters.

Classification

The classification is the final step of people detection. During the classification, a candidate region is evaluated and a decision is taken whether it is a person or not. State of the art of people detection is dominated with classifier training approaches. Most of them use variants of Boosting and SVMs machine learning. But a silhouette matching technique known as Chamfer System has also been used.

Both of these kind of classification methods are presented in the following paragraphs.

Chamfer Matching

Chamfer Matching, introduced by [Barrow 1977], is a technique used to compare the shapes of two collections of shape fragments. For example, for an edge template T composed of edge features t and an image's edge map I, the Chamfer Distance is given by the average distance dI to the nearest feature.

D Chamfer (T, I) = 1 |T | t∈T d I (t) (2.1)
Depending on how well the template represents the person's class and how good the features used are, this measure can be used for people detection. In order to make a precise decision about the object location, orientation, and scale, it may be necessary to use subsequent verification stage [START_REF] Gavrila | [END_REF]]. This method has been applied successfully for person detection from images [START_REF] Gavrila | [END_REF]].

Support Vector Machines (SVMs)

Support Vector Machines are supervised learning models with associated learning algorithms that analyse data and recognize patterns, used for classification and regression analysis. they were introduced by Vladmir Vapnik [Vapnik 1995]. SVMs are widely used in people detection approaches due to their high generalization performance without any needed a priori knowledge, even when the dimension of the input space is very high [Vapnik 1995].

The aim of SVM machine learning is to find the optimal hyperplane that separates two classes of data. Depending on the nature of data, this separation may be a linear or non-linear one. The most used non-linear kernels for SVM are polynomial and Gaussian kernels, but most of approaches for people detection assume that the class separation can be performed well with linear SVM's.

In the case of linear SVM, considering a set of N linearly separable training examples (x i , y i ) i:1..N , where x i ∈ R p are p-dimensional real vectors and y i ∈ {-1, 1} their corresponding class labels, an infinite number of separation hyperplanes can be taken (see figure 2.9(a)). SVMs aim to select the optimal hyperplane for separation. It is done by introducing "maximum-margin" notion.

Any hyperplane can be written as the set of points x satisfying w • xb = 0 where "•" denotes the dot product and w the normal vector to the hyperplane. The parameter b w determines the offset of the hyperplane from the origin along the normal vector w.

The margin maximization is performed by finding the furthest two parallel hyperplanes which delimits a points-free space between them (see figure 2.9(b)). These two hyperplane are defined by the two equations w • xb = 1 and w • xb = -1. The distance between these two hyperplanes is 2 w , so maximizing this distance is done by minimizing w . To prevent data points from falling into the margin, this minimization have to be done under the two following constrains:

w • x i -b ≥ 1
for x i of the first class, and

w • x i -b ≤ -1 for x i of the second class.
These two constraints can be grouped, using the class labelling, in the following equation:

y i (w • x i -b) ≥ 1, ∀1 ≤ i ≤ N
The optimal separation hyperplane is taken as the one between the two parallel hyperplanes, at equal distance from each of them. Several approaches for people detection, based on SVM training were proposed.

H1 H2 H3 X 1 X 2 X 2 X 1 w .x -b = 1 w .x -b = 0 w .x -b = -1 w 2 || w || b || w || (i) (ii) m a rg in
In [START_REF] Dalal | [END_REF]], people detection is performed by a linear SVM classifier trained on Histogram of Oriented Gradients features. In [START_REF] Miezianko | [END_REF]], a linear SVM classifier is trained on gradient patches extracted from low resolution infrared videos. In [START_REF] Papageorgiou | [END_REF]], haar wavelets are used to train a polynomial SVM.

In [START_REF] Ronfard | [END_REF]], a body part based people detector is trained using Relevant Vector Machines (RVM). It will be presented in subsection 2.1.4.2.

Boosting

Boosting is a field of machine learning domain. The principle consists in iterative addition of trained weak classifiers to perform a strong classifier. A weak classifier is defined as classifier which can separate two classes at least as well as a random classifier,

i.e. it does not exceed 50% of an average error rate if the classes are equivalently distributed. Each added weak classifier is weighted in the strong classifier according to its classification quality: the more it classes well, the more important is (higher his weight is).

The name of this machine learning method comes from its main common processing step: training samples witch are incorrectly classified after a given iteration are "boosted" by assigning them more important weights in the next iteration step, to make the machine learning "focus" more on them.

The final strong classifier consists in a set of weighted weak classifiers, and its clas-sification decision is performed using the sum of weighted responses from all its weak classifiers.

Many variants of Boosting algorithms were proposed, we can cite Bootstraping, Ad-aBoost, Discrete AdaBoost, Real AdaBoost, GentleBoost, BrownBoost, LogitBoost, Asym-Boost, KLBoost, FloatBoost, GloBoost, RankBoost, etc.

As the subject of our thesis does not relies to machine learning technique, we can not detail all these algorithms. We can only indicate that they all share the basic principle, but differs on the way they perform some computing step, like the new weight update for incorrectly classified training samples, the weak classifier selection criteria, the type of data for which they are dedicated and the number of initialization parameters that they requires.

Nevertheless, and due to its large use in many people detection approaches, we will focus on AdaBoost algorithm and its very close variants LogitBoost and RealAdaBoost.

AdaBoost (Adaptive Boosting) has been introduced by Freund and Schapire in [START_REF] Freund | [END_REF]]. The following paragraph summaries it's algorithm.

For a binary classification, let us consider a set of N training p-dimensional samples (x 1 , y 1 ), . . . , (x m , y m ) where x i ∈ R p , y i ∈ Y = {-1, +1}. The number of weak classifiers to train, which correspond the number of iterations of the main algorithm's loop, is fixed at the beginning. we will note it T.

The first step consists in initializing the weight distribution Di of all training samples: D 1 (i) = 1 N ,.

For each iteration t = 1,. . . ,T:

-A set H t of q weak classifiers h (j)
t is formed first (j=1,. . . ,q). The corresponding weighting error ǫ (j) t of each weak classifier h (j) t is computed with respect to D t by: ǫ

(j) t = N i=1 D t (i)I(y i = h (j) t (x i ))
, while I is the indicator function. The best weak classifier h t is then selected as the one with the maximum absolute value of the difference of the corresponding weighted error rate ǫ t and 0.5:

h t = argmax h (j) t ∈Ht 0.5 -ǫ (j) t -If |0.5 -ǫ t | ≤ β,
where β is a previously chosen threshold, then stop.

-Choose α t ∈ R, typically α t = 1 2 ln 1-ǫt ǫt . -Update all sample weights. For i = 1,. . . ,N:

D t+1 (i) = Dt(i) exp(αtI(y i =ht(x i )) i Dt(i) exp(αtI(y i =ht(x i ))
where the denominator is the normalization factor ensuring that D t+1 will be a probability distribution.

Chapter 2: State Of The Art

The final binary classifier is then obtained as: H(x) = sign T t=1 α t h t (x) AdaBoost has proven to be very useful in feature selection. At the end of the training, the features with prominent discriminative power have higher weights. It is frequently used for image classification as a result of its simplistic implementation, very good feature selection abilities, and fairly good generalization.

It has been used for person detector in many approaches. Note that all proposed approaches train final classifiers which can be either a single strong classifiers or a cascades of strong classifiers.

Cascade of classifiers perform in a rejection scheme. It is implicitly associated to a class of interest (the positive one). The tested data is evaluated by the cascade levels in increasing order. If any cascade level returns a negative response, the data is immediately considered as not belonging to the class of interest and is rejected. Is is not evaluated by the remaining levels. Only positive data (belonging to the associated class) fully pass through the whole levels of cascade when they are evaluated.

In both [Zhu 2006b] and [Jia 2007], a cascade of classifiers is trained using Ad-aBoost algorithm on Histogram of Oriented Gradients.

Other Adaboost derivatives method are used for the same purpose. In [START_REF] Gerónimo | [END_REF]], a Real AdaBoost is used train a classifier based on a combination between Haar-like features and Edge Oriented Histograms. Chen et al. [Chen 2007b] people consists in a cascade of Real AdaBoost trained classifiers, using Edge Oriented Histograms features.

LogitBoost is used by [START_REF] Tuzel | [END_REF]] and by Yao et al. [Yao 2008] to train a cascade of classifiers based on Region Covariance Descriptors. Note that in these approaches, the LogitBoost algorithm has been slightly modified to deal with the Riemannian manifold which Region Covariance Descriptors belongs.

Real Adaboost differs from the original Adaboost in the way than the original Adaboost classifier returns only a binary value (response) for a tested data, depending on this data belongs to the the first or to the second class. The Real Adaboost algorithm provide a real valued probability of class membership.

LogitBoost algorithm is first presented in [START_REF] Friedman | [END_REF]]. The authors have introduced more direct approximations to AdaBoost algorithm and have shown that they exhibit nearly identical results to boosting while the computation cab be reduced by factors of 10 to 50. The main difference between Adaboost an Logitboost resides in the way the weak classifier errors are computed and thereby the way the best weak classifier is selected at each iteration. AdaBoost minimizes an exponential loss function while LogitBoost minimizes a logistic loss (hence its name).

Person Detection Approaches

In the previous sections, we have presented the several used features and the different classification/matching methods. People detectors can also be categorized into two major types, one which searches the image for full human bodies by scanning the input images in different ways, known as full body detection approaches, and the other which tries to aggregate evidence of existence of a person by modelling and detecting human body models, known as part based approaches. Some approaches are designed to detect the full body as well as the body-parts( [START_REF] Felzenszwalb | [END_REF], Wu 2005])

Full Body Detection

In full body detection, the input image is scanned for people searching using a window with an average human aspect ratio. Most of the works in full body detection were done either from a pedestrian detection context [START_REF] Gavrila | [END_REF][START_REF] Gerónimo | [END_REF]] or from a general object detection framework context [START_REF] Dalal | [END_REF]].

In pedestrian detection, the fist promising results were reported by Oren et al. [Oren 1997].

In this approach, Haar-like templates are used to extract features, an SVM classifier is trained on a selected subset of them. Only significant features for the task, identified using a template learning stage, are used for training. [START_REF] Jones | [END_REF]] improve [Oren 1997] method by incorporating motion information to detect pedestrians. Feature vectors consist in motion information as well as intensity information, extracted using Haar-like features. AdaBoost is used to train a cascade of classifiers. This approach outperforms previous one, but due to the use of motion information, this approach can be only applicable to static camera.

In [START_REF] Gerónimo | [END_REF]], different extended Haar-like filter sets are combined with Edge Orientation Histograms (EOH) to model human body. Real AdaBoost is used for classifier training. The person detection is optimized by restricting the search area at specific image locations, determined by estimating the current ground plane. Once the ground plane is estimated, a 3D grid, sampling the road plane is projected on the 2D image defining candidate regions (subsection 2.1.2.2). The results showed that the proposed approach improves detection performances, due to two reasons. First the combination of extended Haar-like features with the Edge Oriented Histograms provides better results thant the use of Haar-like features only. Second, the ground plane estimation improves the true pedestrian location hypothesis.

In a similar context, Monteiro et al. [START_REF] Monteiro | [END_REF]] used Haar like features to detect pedestrians with an AdaBoost trained cascade of classifier. In this approach, pedestrian detection is done by a sliding window approach, and the multi-scale search is performed by and scaling the detector rather than the image.

Other approaches for person detection are performed in a general object detection context. In [START_REF] Dalal | [END_REF]], HOGs as features and a Linear SVM as a classifier, and a sliding window approach for candidate generation are used. The major contribution was the construction of particular effective features, HOGs, and a data-mining approach during training in which resulting false positives were re-introduced as hard negative examples. This approach was the winner of the 2006 PASCAL object detection challenge [Everingham 2009].

Zhu et al. [Zhu 2006b] reformulated [START_REF] Dalal | [END_REF]] detector in cascade of classifiers method to achieve fast and accurate human detection system. In this approach, variable block sizes HOG features are used. The selection of the important sets of blocks is performed using AdaBoost. Their final implementation used an integral array representation to speed-up the detection in comparison to [START_REF] Dalal | [END_REF]] detector while maintaining similar performance levels.

Laptev [Laptev 2006] used the AdaBoost framework to select prominent features for object detection. Weighted local histogram of gradient orientations in all rectangular sub-window of the object are used as features. Weighted fisher discriminant analysis is used as a weak classifier, and AdaBoost selects the best features out of all histograms computed on all sub-windows.

Body Parts Based Detection Approaches

Body parts based detection approaches are performed in two steps. First, the different parts of the body (head, face, arms, torso, leg) are detected. Then, an inference process using the detected parts and/or some geometrical constraints is performed to deduct the presence and position of a person. Performing these two steps require to model human body parts using visual features and classifiers, and to model the topology of these body parts.

In [START_REF] Forsyth | [END_REF]] body plans for people and animals detection in images are introduced. Body plans model people as an assembly of cylindrical parts, each cylinder corresponding to part of a body, where the individual geometry of the parts and the relationship between parts are constrained by the geometry of the skeleton and ligaments.

A human Body Plan is constructed by segmenting human skin using color and texture criteria, assembling the extended segments, and using a hand built body plan to support geometric reasoning. In [START_REF] Mikolajczyk | [END_REF]], the authors have reported that this body detector fails in the presence of clutter and loose clothing.

In [START_REF] Felzenszwalb | [END_REF]], authors propose a parts based detector using pictorial structures. A pictorial structure is a collection of parts arranged in a deformable configura-tion. Each deformable configuration is represented by spring-like connexions between pairs of parts. The parts are modelled as rectangle with fixed aspect ratio, average color, and color variance. The authors propose an algorithm for finding the global match of pictorial structure to an image.

Due to the use of simple color as feature for parts characterization, the proposed detector in [START_REF] Felzenszwalb | [END_REF]] is not really robust. Ronford et al. [START_REF] Ronfard | [END_REF]] improve the previous method by using better part features and detectors. A hand-labelled articulated body model with 14 joints and 15 body parts and a feature set consisting of Gaussian filtered image and its first and second derivatives are used. 15 body-parts detectors are trained using Support Vector (SVM) Machines and Relevance Vector Machines (RVM), and are applied for people detection (see figure 2.10). In [START_REF] Felzenszwalb | [END_REF]], a person detection scheme in a general object detection framework with discriminatingly trained parts based models is presented. The person detector is based on mixtures of multi-scale deformable parts models that have the ability to represent a highly variable object class like the one of a person. Used features consists in Histograms of Orientation Gradients (HOGs) with analytically reduced dimension. All model parameter learning was done by constructing a latent SVM problem and training the latent SVM using a coordinated descent approach (see figure 2.12).

In general body parts detection approaches are better suited during occlusions than full body approaches as their detection depends not only on the whole body but on the different parts of the body, head, torso, legs, detected.

In [START_REF][END_REF] standard pictorial structure models ( [START_REF] Felzenszwalb | [END_REF][START_REF] Ronfard | [END_REF]) that encode just spatial relations (see figure 2.13). All parameters, including local appearances, spatial relations, and co-occurrence relations (which encode local rigidity) are learnt with a structured SVM solver.

Discussion

In this section, we have presented the several existing approaches for people detection in images. The most used scheme fort people detection consists in two main steps:

classifiers training, and detection task. Both of these steps require to represent image region information in a significant and useful way. Many features have been used for this purpose. Some of them are specifically designed for object/people detection, like HOGs, Haar-like and Region Covariance Descriptors, and are used for other purpose later, while other features were proposed initially for different tasks, like LBP for texture analysis and Shape Context for shape matching, and have been adapted or added as additional information for people detection task.

The selection of more significant features for the classification is performed using Due to the complex separability of human and non-human classes, cascade of classifiers seems to be more adapted and more efficient in term of required processing time as long as their structures are optimal. They have to ensure a high rate of non-human data rejection in the first levels, requiring less weak classifiers evaluation than for a single strong classifiers. Only small proportion of hard negative data has to require the whole cascade evaluation.

People detection approaches either use a full body or body parts detection approaches.

An important advantage of the part based approach is its better robustness to partial occlusions than the standard approach considering the whole object, due to its object parts segmentation and their independent detection. Nevertheless, part detection methods present two main issues in comparison with full body based approaches, depending on the way they are performed and the resolution of images in which the detection is performed.

First, for the same type of classifiers trained on the same set of features, but one on full body and the other one on body parts, it is clear that body part approaches require more processing time. A full body detector will perform the classifier once on a candidate region while part based detectors require as many classifier applying as considered number of parts. This processing time is more important when no topological constraints are applied to restrict specific body part searching (topological constraints are generally used after detection to infer the existence of a person or not).

Second, according to the defined body part sizes, and the resolution of images, some parts are hard to detect if their corresponding regions in image are too small or have low resolution. We have seen than some proposed approaches ([Yang 2012]) use a large number of very small body parts to ensure a better flexibility of the built model. It may cause miss-detection of some parts and increase processing time. Most of proposed body part based approaches using small part detection, have been tested and evaluated on images in which, people have sufficient image sizes and resolution. This constrains can not be ensured in all situations. The approaches which were evaluated on low resolution images use larger and fewer parts segmentation, loosing some of the effectiveness of using more and smaller parts (for example, [Bak 2010] on iLids dataset, with 5 parts: head, torso, arms and legs).

From these observations, we can deduct that feature selection as well as detection strategy (full body VS. body parts) are dependent on the training data parameters (images resolutions and quality) and final addressed task requirements (real-time constraint for live processing VS. off-line processing).

Due to the addressed problem context, which is video surveillance, and its constraints (low resolution images and small person sizes most of the time), we have decided to focus our work on full body detection approaches. We have used cascade of classifiers structure to reduce decision time (which is a critical point for live surveillance systems)

by proposing an efficient approach to optimize the structure of the cascade and to reduce greatly training and detection times.

Mono-camera Object Tracking

Object tracking is a critical task for scene event understanding. Detecting objects of interest on independent images allows to know that these objects are present in the scene, but does not help to understand what they are doing. In computer vision systems, all kinds of event and behaviour detections require temporal information as long as an event or a behaviour has a starting and ending time (a duration) and consists in successive states. To reason at the level of scene events, it is necessary for a computer vision system to be able to continuously associate the same identifier to the detected object of interest.

The aim of an object tracking algorithm is to ensure this "object of interest"-"unique identifier" association as long as this object is present in the observed scene, and thereby, provide the full trajectory of the object in the scene. This is done by locating its position in each video frame.

Object modelling plays a crucial role in visual tracking because it helps to characterize the object of interest. The modelling is performed in a different sense than for people detection purpose. For people detection the model is a class model. It has to be as generic as possible to cover all possible variations between persons, size and like pose differences, clothing types, etc. while being discriminative against other object classes.

In object tracking task, the model is an individual model. It means that an object model has to be full-discriminative against both other type objects as well the ones of the same type. The way in which object modeling is performed can be considered as a first criteria for object tracking approach classification.

Once a model of an object of interest is computed, it is used to track this object by searching it through out the video frames. The model searching/matching over time methods constitute a second criteria for object tracking method classification.

We first present the object modelling methods for tracking. Then, we present object model searching over time methods.

Object Modelling

For object tracking purpose, a wide variety of object modelling approaches have been proposed, depending on the amount and the type of information which are extracted from the object. At the low level, the object can be represented simply by intensity value of its pixels [START_REF] Pahlavan | [END_REF]]. At the middle level it can be represented by some features used independently like, color [Pérez 2002[START_REF] Comaniciu | [END_REF]] or feature points [Yao 1995[START_REF] Tissainayagam | Object Tracking in Image Sequences using Point Features[END_REF]. At the highest level it can be represented by a global feature vector which can be boosted from many features to perform a better representation of the object, like [START_REF] Chau | [END_REF], Zhou 2006] (distance, area, shape ratio and color histogram) or [START_REF] Serby | [END_REF]] (interest points, straight and curved edges, textured regions and homogeneous regions).

In the following paragraphs, some object modelling features are presented.

Color Modelling:

Color is one of most fundamental feature to describe an object, due to its strong descriptive power. RGB color space is usually used to represent images; however, the RGB color model is perceptually not a uniform color model. Some other color spaces are more adapted for intra-color distance computation. HSV (Hue, Saturation, Value) and HSL (Hue, Saturation, Lightness) is an approximately uniform color spaces and used intensively in literature.

The color information of object can be modeled in several ways. we can cite:

Color histograms: Color histogram is the most used color encoding method for object modelling [Pérez 2002[START_REF] Comaniciu | [END_REF][START_REF] Qian | [END_REF]]. It is a representation of the distribution of colors in an image, i.e. it represents the number of pixels that have colors in each of a fixed list of color ranges (bins). It provide an interesting information about the frequency of each color in the object image, but present the inconvenient to do not encode the spacial distribution of the colors. To deal with this issue, some other color encoding methods, which keep color spatial repartition information, have been proposed. We can cite color spatiograms [START_REF] Birchfield | [END_REF]] and Maximally Stable Colour Regions for Recognition and Matching [START_REF] Forssén | Maximally Stable Colour Regions for Recognition and Matching[END_REF]].

Color spatiogram (for spatial histogram): Spatiograms capture both occurrences and spatial repartition of colors. In [START_REF] Birchfield | [END_REF]], a second-order spatiogram of an image I is defined as: h

(2)

I (b) = n b , µ b , Σ b b = 1,. . . ,B
where (2) is related to the second order of spatiogram (the proposed spatiogram can be generalized to higher orders), B is the number of used bins, n b is the number of pixels whose value is the one of the b th bin, and µ b and Σ b are the mean vector and covariance matrices, respectively, of the coordinates of those pixels.

A comparison between image generation from color histograms and from color spatiograms is provided by authors in figure 2.14.

Maximally Stable Colour Regions (MSCR):

In [START_REF] Forssén | Maximally Stable Colour Regions for Recognition and Matching[END_REF]], extends the Maximally Stable Extremal Regions (MSER) [START_REF] Matas | [END_REF]] method to color images.

Color regions are iteratively clustered until stable color regions are formed, using predefined thresholds. This method provide a feature vector containing: the color region area, the coordinates of the is color region centroid, the first and the second order moments of the color regions according to the two axis (X and Y The Color Structure Descriptor: is also based on color histograms, but aims at identifying localized color distributions using a small structuring window. To guarantee interoperability, the color structure descriptor is bound to the Hue-Min-Max-Difference (HMMD) color space.

The Color Layout Descriptor: captures the spatial layout of the dominant colors on a grid superimposed on a region or image. Representation is based on coefficients of the Discrete Cosine Transform (DCT). This is a very compact descriptor being highly efficient in fast browsing and search applications. It can be applied to still images as well as to video segments.

Shape Modelling:

The shape features are used as a powerful cue to track object in video frame sequences. Several representation are used (see figure 2.16):

Point Representation: In visual object tracking, the trivial shape is the point. An object is represented with a pixel location representing either some statistics on the object shape, such as the centroid and , or a particular characteristic of interest like the centred contact point with the floor. Many approaches use single or multiple points to represent tracked objects. For instance, it has been used for in vehicle tracking [START_REF] Kanhere | [END_REF]] or extended objects, such as ships or a convoy of vehicles moving in urban environment tracking [START_REF] Angelova | [END_REF]] and in [START_REF] Gustafsson | [END_REF] for automotive and airborne tracking applications. In [Chen 2007a], A point representation is used to represent the variance of pixels in the object of interest shape.

Points are also used in the calculation of optical flow: due to the large number of vectors to estimate, only the point representation can be afforded [Kragik 2000].

Primitive geometric shapes: The point representation of an object is a simple model. However, it does not grasp the entire dynamics of the object. To rem-edy this gap, more advanced parametric shapes are necessary. The most popular parametric shapes are primitive geometric shape (rectangle, square, ellipse and circle). The rectangle representation is frequently used for tracked objects representation like for vehicles tracking [START_REF] Melo | [END_REF][START_REF][END_REF]] or people tracking [START_REF][END_REF]]. An adaptive square shape has been used for object representation in [START_REF] Bradski | [END_REF]]. The ellipse offers the advantage of "rounding" the edges compared to the rectangle when the object does not have sharp edges [Chang 2005].

In [START_REF] Comaniciu | [END_REF], Comaniciu 2003], the author used an elliptical shapes to represent the moving object.

Articulated shape models: Articulated shapes are employed for tracking if different portions of the object of interest are to be described individually ( e.g. legs, arms and head). This kind of representation is much suitable for a human body, which is an articulated object with head, hands, legs etc. These body parts should be linked by a kinematic model. The parts can be represented by any primitive geometric shape such as rectangles, circles and ellipses. In [START_REF] Ramanan | [END_REF]], an articulated shape model, describing the body configuration and disambiguating overlapping tracks, is developed.

Skeletal models: In this representation a skeleton of object is extracted to model both articulated and rigid objects. A skeleton consists in as a set of articulations within an object that describes the dependencies and defines constraints between the representations of the parts. In [START_REF] Ali | [END_REF]], the skeletal model is used for automatic segmentation and recognition of continuous human activity.

Object silhouette: The silhouette,also called "Blob", is a dense, non-disjoint, binary mask that represents an object of interest. Blobs are of particular importance for pixel-wise processing. For instance, background subtraction provides blobs identifying the foreground or the moving objects in a scene [START_REF] Wren | Pfinder: Real-Time Tracking of the Human Body[END_REF]], [START_REF] Stauffer | [END_REF][START_REF] Elgammal | [END_REF].

Contour: In this representation the boundary of an object is defined as a contour. A non-rigid object shape can be better represented by these representations [START_REF] Yilmaz | Contour-based object tracking with occlusion handling in video acquired using mobile cameras[END_REF]].

Texture Modelling:

Texture is another important information for object characterization. It provides a useful information about the spatial arrangement of color or intensities in an image or selected region of an image. Texture is concerned with representing regular patterns in an image [START_REF] Forsyth | [END_REF]]. The texture representations can be performed in several ways, providing different features. These features may differ in the described region sizes, the main extracted information or in the level of robustness and invariance to rotations and illumination changes. Edge detection is performed by applying specific operators on image. Many edge detection operators have been proposed. We can cite Sobel Operator, Robertś cross operator, Laplacian of Gaussian, and Canny edge detection algorithm which is known to many as the optimal edge detector (see figure 2.17). The main difference between the operators resides is they handle different edge orientations.

(a) (b) (c) (d) (e) (a) (b) (c) (d)
Edges have been used for people tracking in some approaches. In [Murshed 2011],

Canny edge map is used to characterize moving object region. Curvature-based features are used for moving edge registration due to its transformation invariance nature. Each individual edge segment is tracked using a Kalman filter. Edge segments are clustered by using a weighted mean shift algorithm. The final moving object tracking is performed using a group motion tracker, applied on each cluster. Due to the robustness of edges against illumination changes and partial occlusions, the proposed tracked performs efficiently.

In [Zhu 2006a], an edge-based tracking algorithm is proposed. The feature points are extracted by efficiently utilizing the image edges in the object region. Then the parameter vector of the object's motion model is estimated based on minimizing the sum-of-squared differences between the reference feature points in the reference frame and the observed feature points in the tracking sequence frame. The SIFT and HOG descriptors perform with similar principle in the sense that both take the raw information from the magnitudes and orientations of pixel gradients in the described region of image to build histograms. The main difference consists in:

-The way the raw information (gradient magnitudes and orientations) is processed: different weights for pixel contributions, according to their position with respect to the centre of considerer region, are used for SIFT descriptor computing while similar weight for all pixel contributions are used in a HOG cell.

-The way the information is encoded: gradient histograms are extracted from independent pixel regions for SIFT, requiring and independent normalization to ensure robustness against illumination changes, while HOG use overlapped block of pixels for computation, which implicitly ensure the local invariance to illumination changes -The robustness to 2D rotations: SIFT descriptor is invariant to 2D rotations because it is computed using relative gradient orientations with respect to the main gradients orientation in the considered region. HOG descriptor does not integrate a 2D rotation invariance and is computed using absolute gradients orientations in the image region.

Note that these two descriptors were initially proposed to perform using gray scale images. Some variants SIFT descriptor integrating color information have been proposed, like in [START_REF] Verma | [END_REF]] and [Abdel-Hakim 2006] using RGB color space, and [START_REF] Bosch | [END_REF]] where HSV color space has been used.

These two descriptors were used for object tracking in a large amount of approaches.

In [Zhou 009], a SIFT based mean shift algorithm is presented for object tracking. SIFT features are used to correspond the region of interests across frames.

Meanwhile, mean shift is applied to conduct similarity search via color histograms.

The probability distributions from these two measurements are evaluated in an expectation-maximization scheme so as to achieve maximum likelihood estimation of similar regions. This mutual support mechanism provides better tracking performance if one of the two measurements becomes unstable.

In [START_REF] Fazli | [END_REF]], A color SIFT based particle filter algorithm is proposed for object tracking. It provides interesting tracking performances in partial occlusions, rotation and scale variations conditions.

Motion Modelling:

Motion detection is a critical part of the human vision system [START_REF] Sonka | [END_REF]]. Optical flow is the most widespread depiction of motion. Optical flow represents motion as a displacement vectors which defines the movement of each pixel in a region between subsequent frames [Horn 1981]. Horn and Schunk [Horn 1981] computed displacement vectors using brightness constraint, which assumes brightness constancy of corresponding pixels in consecutive frames. Lucas and Kanade [Lucas 1981] proposed a method that computes optical flow more robustly over multiple scales using a pyramid scheme.

Optical flow is used for object tracking in some approaches. In [START_REF] Shin | [END_REF]], the proposed tracking algorithm performs in three steps: First, the object of interest is localized using optical flow detection. Then prediction and correction of the object's position is performed using spatio-temporal information of the optical flow computed on beforehand detected feature points. Finally restoration of occlusion using an introduced non-

prior training (NPT) active feature model (AFM) framework. The proposed algorithm can track both rigid and deformable objects, and is robust against the object's sudden motion because both a feature point and the corresponding motion direction are tracked at the same time. The authors claim that the proposed AFM enables stable tracking of occluded objects with maximum 60% occlusion.

In [START_REF][END_REF]], object contours are tracked with optical flow. This algorithm achieves accurate, rapid and stable object tracking on the evaluated situation.

Optical flow is an interesting way to model moving object as long as the addressed situations are not complex, like the conditions in which [START_REF] Shin | [END_REF]] and [START_REF][END_REF] were evaluated. Single moving object in the scene or collusions resulting from uniform object movements can be well handled. In more complex scenes with multiple moving objects with non-uniform movements, this technique may fail.

Object Tracking Techniques

Once an object of interest is modelled, the next step consists in its localization in each frame of the video sequence. Tracking methods can be divided into two general types: deterministic and probabilistic methods. Deterministic methods look for the local maxima of a similarity measure between the object model and the considered candidate regions of current image while probabilistic methods aim at modelling object movement and to perform the object localisation through successive state prediction/update steps.

Deterministic Methods

A deterministic systems is a system in which no randomness is involved in the development of the future states. Deterministic tracking methods always provide the same tracking results on the same input data (video sequences and parameters). In this kind of approaches, the object localization may be performed either by an exhaustive search of the object model on the whole image (least-square tracking), by an iterative neighbour exploration (Mean-shift tracking), by point matching, or by many other methods.

Least-square tracking: Least-square tracking consists in an exhaustive search of the tracked object model in the whole image. A Similarity measure between the object model and candidate models extracted from all image positions is computed.

The position of the candidate model which provides the best similarity measure is considered as the new localisation of the tracked object.

Least-square tracking methods, applied on the whole image, are the most effective tracking methods in terms of correct localisation of tracked objects on video sequences since they explore all the possibilities space. However, these methods are also the most time consuming among all tracking methods. The required processing time is unacceptable for most of applications. The evaluation of the model matching in all possible positions requires many computations, especially if the model building and the similarity measure computing are complexes to perform (multi-features model for example). To avoid this critical issue, two main heuristic solutions can be used.

The first one consists in the reduction of the possibility space by performing the exhaustive search in a "local" region instead of the whole image. This really decreases the computation time in relative proportion to the region reduction, leading to another difficulty: "local" is an ambiguous term. It is hard to define an optimal local region for exhaustive search without any a priori knowledge on the object motion. Bad local region localization automatically induces a tracking failure if the correct matching region is not included.

The second heuristic solution consists in a progressive search. Instead of testing the whole possibility space, a uniform browsing step is used to evaluate a subset of positions. For example, a searching process with 2 pixels step in both directions divide the number of the tested possibilities by 4, which is not a negligible factor.

The more important the step is, the lesser the number of matching tests are. However, this solution, as the previous one, presents an important issue: how to choose the optimal browsing step? Object feature sensitivity to shifting may greatly vary.

Some features like color histograms are less sensitive to small shifting (due to the contained information nature, i.e. color frequency and non-spatial information)

than other features like Region Covariance Descriptors built with spatial information. For more sensitive feature-based object models, the maximum similarity measure may not correspond to the real object but to another image region, if the browsing process does not test the real object region.

Mean-shift tracking: Mean-shift tracking assumes that the object locations in successive images are close to each other. Thus, the searching process in the current image starts from the same location as the object location in the previous image and is performed iteratively. At each iteration, the direct neighbourhood of the current mean-shift window location is tested. A similarity measure is performed between the tracked object model, and those of candidate region located in neighbouring positions. The location with the best similarity value is taken as the new position of the mean-shift window for next iteration. The process is repeated until it stabilizes, i.e. all the similarity values of all neighbouring positions are less than the one for the current mean-shift window position. Mean shift-tracking seems to be an efficient approach, but it presents two main issues.

First, the assumption of small object displacements in successive frames can not be ensured. If the mean shift window is not correctly initialized (smaller than the object displacement), the tracking will probably fail.

Second, the convergence time is very variable and depends mainly on the object displacement magnitude and on the type and distribution of the features on which the object model has been built.

Many deterministic approaches have been proposed in the state of the art.

In [START_REF][END_REF]], moving object models are constructed using Kernel Co-occurrence Matrices (KCMs). Then those matrices are employed as the tracking cues in a mean shift framework. In [START_REF] Birchfield | [END_REF]], proposed spatiograms are used as object model for tracking. They are tracked by a proposed kernel-based tracker deriving from mean shift method. Another mean-shift based approach is proposed in [START_REF] Comaniciu | [END_REF]] to track non-rigid objects. The dissimilarity between the target model (its color distribution) and the target candidates is expressed by a metric derived from the Bhattacharyya coefficient. In [Zhou 2006], a Mean-shift method is used to track SIFT [START_REF] Lowe | [END_REF] features and color histograms, which are to model the tracked objects. Finally, in [Thayananthan 2003], a derived mean-shift method is applied to track human hand using its shape. The hand is modelled with Shape Context feature [START_REF] Belongie | [END_REF] (see section 2.1.1.5) and the used similarity measure is the Chamfer Matching method [Barrow 1977] (see section 2.1.3.1)

Previously mentioned issues for mean-shift tracking methods can affect the tracking performance since they were not handled in this proposed approach.

In [Zhu 2006a], a multi-feature points correspondences approaches is proposed. Feature points are extracted on the image edges in the object region. Then the parameter vector of the object's motion model is estimated based on minimizing the sum-of-squared differences between the reference feature points in the reference frame and the observed feature points in the tracking sequence frame. The experiments show that the edgebased tracking algorithm proposed by us can track object efficiently under uniform and varying illumination conditions.

In [START_REF] Bilinski | [END_REF]], another point matching based method is proposed. FAST points detector [START_REF] Rosten | [END_REF]] is applied to detect points of interest on moving objects, and this on each frame. Each detected point is associated to a HOG descriptor [START_REF] Dalal | [END_REF]] computed around it. The tracking is then performed by a direct point matching between object of previous frame with those of the current frame, by computing their HOG descriptors similarity. The final object movement is determined using the trajectories of all matched points of interest.

The point matching based methods provide good tracking performances, but present the main issue of high computational time consuming. These approaches requites to repeat two non-basic tasks which are interest point detection, and descriptors computation. In addition to that, the matching process consist in a kind of Cartesian product matching, i.e. each interest point from the object to track is compared to all interest points of the candidate object by their associated descriptor similarities. It is a slow task. This last issue can be partially handled by using some localisation constrains. For example, points belonging to the upper part of the object to track will be compared with those of the upper part of the candidate object only, and not with all the points. This solution decrease matching time but can cause matching fail in case of deformable or partially occluded objects.

Probabilistic Methods

They consist in a recursive estimation of a hidden state of a moving object using noisy observations. Considering that the hidden state evolves over the time, it is necessary to introduce an a priori model of displacement for the mobile, and to consider the estimation problem in a Bayesian framework.

According to the types of motion and noise, different kinds of Bayesian filters are defined. Two main Bayesian filters are used for object tracking. In the particular case of Gaussian linear systems, the filtering problem has an explicit solution, called Kalman Filter. In the case of non-linear systems with noise which is not necessary a Gaussian one, or in the general case of hidden Markonv models, some very effective Monte Carlo methods have appeared under the name of Particle Filters. Intuitively, each particle represents a possible state, explores the state space following the a priori motion model and is duplicated or eliminated at the next generation depending on its coherency with the current observations, quantified by likelihood function. This mutation/selection mechanism has the effect of automatically concentrating the particles in the regions of interest inside the state space.

The Kalman Filter

As it was indicated before, Kalman filter allows to estimate the parameters of a system which evolves over the time, using some noisy measurements.

The Kalman filter is performed in two successive steps:

-Prediction: The first step is the prediction of the estimation according to the system model. To perform it, the Kalman filter takes the previous estimation of the parameters and the error, and uses them to predict the new parameters and the error depending on the system model.

X - t = DX + t-1 + W (2.
2)

P - t = DP + t-1 D T + Q (2.3)
where: X - t and X + t-1 are respectively the predicted and corrected states at time t and t-1; P - t and P + t-1 are respectively the predicted and corrected covariances at time t and t-1; D is the state transition matrix which defines the relation between the state variables at time t and t-1; W is a noise matrix and Q is its covariance.

-Correction: The second step updates this prediction thanks to the new measurements Z t . These measure (which are noisy) allow to obtain an estimation of the parameters and the error from the performed prediction. If the model contains errors, this update step allows to correct them.

K t = P - t M T [MP - t M T + R t ] -1 (2.4) X + t = X - t + K t [Zt -MX - t ]
(2.5)

P + t = P - t -K t MP - t (2.6)
where M is the measurement prediction matrix, K is the Kalman gain and R is the covariance matrix of measurements noise.

Kalman filter has several interesting aspects. The power of this filter lies in its ability to predict the parameters and to correct errors, not only the sensor ones but also those of the model itself.

In fact, to apply a Kalman filter to estimate the parameters of a given system, it is necessary to provide a linear model before. Some variants of the Kalman filter, to deal with non-linear models, have been proposed, for example the Extended Kalman Filter which is discussed below.

In a classical estimation method, like least squares method, a simple error in the system model inevitably leads to an error in the estimation. The advantage of Kalman filter is to integrate a term of imprecision on the model itself, allowing it to correct the estimation instead of model errors (Of course, the model error has to be reasonable).

Another advantage of the Kalman filter is its ability to determine the mean error of its estimation. In fact, Kalman filter provides a vector of estimated parameters, but also the error covariance matrix. This matrix informs us about the precision of the estimation. Another interesting information is that the convergence of this error is guaranteed (in case of linear dynamics).

However, this filter is not necessarily the tool to apply in all cases. In fact, as we have seen, we need to model the system precisely to design an efficient filter. The problem is that some systems are hard to model and are not linear.

In the case where the model is a rough approximation, the filter will not be efficient enough to correct the error, which will converge quickly.

To avoid this problem of linear model of the system, the Extended Kalman filter was developed. It allows to deal with non-linear models. However, this method has some defects. First, the error covariance will not necessary converge (unlike the standard Kalman filter for linear models). The second defect is the high computing cost. In fact, some costly new matrices appear in the filtering computation (matrices of partial derivatives of the state equations and measurement of the system model).

Another important limitation of this method is that the Kalman filter allows us to consider only a Gaussian noise model. Noise can generally be modeled as Gaussian, but in some cases, other types of noise can occur. This restriction limits the use of the Kalman filter.

The Kalman filter is an interesting method of estimation, but can only be used when it is possible to accurately model the system and when the noise is a Gaussian one. When it is impossible to find a correct model for the system, it is better to refer to other methods such as Monte Carlo method, called Particle Filter, which is a statistical method, but it requires significant computing power.

The Particle Filter

Like Kalman filter, Particle filter allows to estimate the parameters of systems which evolve linearly over the time, but also non-linearly, using some noisy measures, even if this noise in not a Gaussian one. It is performed in prediction and correction steps too.

Particle filtering is a global method based on the exploration of the state space of the problem using a set of particles. These particles are distributed according to the conditional probability of the process to be estimated constrained by the observations provided by the sensors.

This method does not require the explicit resolution of the equations of the problem, so it is applicable regardless of the complexity of these equations, especially in terms of non-linear and non-Gaussian nature.

In its basic version, a particle filter consists of N particles which evolve in parallel.

Each particle evolves according to the measurements provided by the sensors at the sampling time "t" and simulates a possible trajectory, ie. the evolution of a process respecting the same equations than the process to estimate. Each particle provides two information as output:

-A state vector with the same structure than the state vector of the process to estimate.

-A scalar value called weight, representative of the probability that this vector is the one of the process to be estimated.

Sequential importance sampling (SIS) is the most basic method used for particle weighting in Particle filters. The weight of each particle is continuously updated over the time. For a given particle, its weight at current time depends on its previous weight, its previous and current state estimations, and the current observation measurements:

w i t ∼ f(w i t-1 , x i t-1 , x i t , z t ) (2.7)
where w i t and w i t-1 are respectively the weights of a given particle P i at time t and t-1; x i t-1 and x i t the estimated state of this particle at time t and t-1 respectively, and z t is the observation measurements at time t. This method present the important issue of information degeneration. In fact, only few particles may have a significant weight at each iteration. The particles corresponding to unlikely hypothesis may continuously degenerate, causing possible divergence of the particle filter, or useless processing of very low weighted particles.

One common way to deal with this degeneracy is resampling. Sampling Importance Resampling (SIR) method consists in removing less significant particles (with the lowest weights) while creating new particles at the same position than the most significant particles (with the highest weights). The number of created particles at the same position of an important particle is proportional to the weight of this important particle. The last step consist in affecting the same normalized weight 1/N to all the particles (N is the number of considered particles) (see figure 2.18).

In this method, the importance of a given hypothesis at the end of an iteration is not represented by its associated particle's weight, but by the number of particles which are related to it. The most important hypothesis at a given iteration will be the basis of more new hypothesis in the next iteration, and information degeneracy is avoided.

For a high enough number of particles, it is possible to demonstrate that the set of all the states of the weighted particles is representative of the conditional probability law of the state vector of the process. This procedure allows to concentrate the exploratory ability of the network of particles in areas where probability is maximum, thus increasing the precision of the estimate.

The most important limitation of Particle filtering method is its high computation time in comparison with other probabilistic methods (Kalman filter). It is directly proportional to the number of hypothesis to maintain and to process in parallel and to the hypothesis verification (particle weighting) complexity (complex features comparison for example).

For object tracking in video sequences, probabilistic tracking methods have been widely used. In [START_REF] Elgammal | [END_REF], Comaniciu 2003[START_REF] Melo | [END_REF], Murshed 2011], a Kalman filter is used to track the several proposed models of object of interest. In [Pérez 2002[START_REF] Serby | [END_REF][START_REF][END_REF][START_REF] Fazli | [END_REF]], the authors used particles filters for their object tracking approaches.

A summary of all cited state of the art approaches for object tracking, according to their proposed models and their tracking techniques is displayed in

Discussion

Due to its importance for scene understanding and useful event information extraction, object tracking has always highly attracted attention of researchers. Many works have been done to deal with the most important challenges (object occlusions, crowded scenes, etc.).

The object modelling is the first important task to be performed. The models have to be highly discriminative to allow a more reliable matching between the same object images over time while avoiding matching errors with a third object. It has to be robust to partial occlusions, illumination changes, and rotations.

Many feature extraction techniques have been proposed in the state of the art. All these features are extracted from one of the main image information families which are texture, color, shape and motion. Several encoding ways have been proposed for each information type, focusing more on some aspect rather than others. For example, color histograms encode the occurrence frequency of each color value without considering the spacial information of the color distribution while The Color Layout Descriptor captures the spatial layout of dominant colors only.

Some state of the art object models are built using a unique feature type while others are built using combination of different features (see Table 2.1). These second kinds of models generally use a concatenation of feature descriptors to provide the final model feature.

Using several concatenated features to build the final object model is an efficient way to improve the discriminative power and the robustness of the object of interest, but it take more time than the computation of a single feature model. In addition, the simple concatenation of feature descriptors provide a decorrelated inter-feature information, which is not efficient since the information is extracted anyway.

This last defect can be handled by using other representation that a simple concatenation. Region Covariance descriptors are a good solution since many image features can be encoded in a single structure which captures not only each feature variation but also correlation between all features. The main disadvantage of this descriptor is its high computation and comparing processing time. A detailed presentation of this descriptor is provided in the chapter 4.

Object modelling is also very dependent on the video properties (color, resolution, compression (noise)) and context (occlusions, illuminations changes, crowded scenes, restricted trajectories, et.). State of the art proposed methods perform well in some conditions while they fail in others.

For all these reasons, and due to the addressed context, which is video surveillance, and its constraints (real-time processing, small objects in large view images, both rigid and non-rigid object tracking, compressed and medium resolution images), we have decided to model our tracked objects using a single feature type which is SIFT feature.

This choice is justified by many reasons. First, the real-time processing requirement is better ensured by single feature model. Second, the necessity to have a non-rigid model and to handle partial occlusions are ensured by SIFT point representation. Finally, the robustness of the model to illumination changes, provided by the illumination invariance of SIFT features.

The weakness of single feature models are compensated by our proposed SIFT feature detection, selection, and tracking methods. The other video properties which usually affect tracking performances, like noise (due to image compression) and small object size in images (due to large view of video surveillance camera) are also handled in our proposed method which is detailed in chapter 5.

From tracking technique point of view, the state of the art approaches are divided into deterministic and probabilistic methods. Exhaustive research tracking method, which belongs to deterministic techniques, is the most effective method due to the fact that all the possibilities are evaluated. However, it is not applicable for many applications, This advantage constitutes at the same time the most important inconvenient of Particle filters. In fact, processing a large set of particles, especially during the weighting step which generally requires feature similarity measure computation, can be unmanageable for real-time tracking. However, by using a small but sufficient set of particle, the realtime processing becomes possible.

Due to its previously cited advantages, we have decided to use a particle filtering method for our object tracking. The selected parameters allow our tracking to be performed in real-time.

People Re-identification

People re-identification in camera network has become a critical task in these last years. With the increase in the number of deployed cameras in restricted and large areas, it has become important to have a global understanding of what is happening in a given location covered by many cameras. Data coming from several cameras should no longer be treated as independent information but rather as global information. Some behaviours or events of interest can only be inferred from a long term tracking of the person of the interest across the camera network (for instance, a person who leaves his/her luggage in an airport and stays more or less near it, is not a suspicious person.

On the other hand, if this person goes away from his/her luggage and leaves the airport observed by several cameras, this behaviour should attract attention). Being able to track or find a given person in a camera network on live stream or to localize him/her a posteriori on recorded videos become very important in many applications, especially for security, but not only. Many other applications like marketing for shopping mall and statistics in some sports require to track people on multiple cameras to infer the most frequent shopping paths and thereby to reorganize shops, or to calculate the travelled distance by a football player and the number of passes and shots he has made.

This global reasoning on large areas, under camera networks cannot be automatically performed without robust techniques to maintain the same identity of a given tracked person, regardless of where he/she is located and which camera is observing him/her. This identity maintaining for a given person from a camera to another one is called "re-identification". Recently, a growing number of studies have been done and this problematic is still attracting more interest from researchers. In the following sections, a literature review of different people re-identification approaches are presented.

According to the kind of used information for re-identification, we can distinguish two main families of approaches: biometric approaches and appearance-based approaches.

Biometric Approaches

Biometric approaches seem to be the most efficient techniques for people re-identification as long as they use biological characteristics of people to identify them. For instance, Pr. John DAUGMAN (Computer vision laboratory of University of Cambridge) estimates that the probability to find two identical irises is approximately 1/10 72 (even for identical twins). The same observation can be done for fingerprint comparison. Sir Francis Galton [Galton 1892] published a detailed statistical model of fingerprint analysis a in his book "Finger Prints". He had calculated that the chance to have two different individuals with the same fingerprints was about 1 in 64 billion.

Unfortunately and as it was mentioned in the first chapter of this thesis, this kind of approach cannot be used in large wide video surveillance systems due to the technical and practical constraints. In fact, this kind of approaches requires specific sensors (for iris and fingerprint recognition), or a high resolution images and sufficient framerate (for faces and gait recognition). In addition to this technical issues, this kind of approaches requires entire collaboration and voluntary actions from people for iris and fingerprint analysis, and depend highly on the orientation of the observed people for face and gait recognition.

In the following sections, we present the state of the art of the most important biometric approaches which are iris, fingerprint, face and gait recognition.

Iris Recognition

The concept of iris recognition has been proposed initially in 1936 by the ophthalmologist Frank Burch as a way for people identification. In 1987, Dr. Aran Safir and Dr.

Léonard Flom, two ophthalmologists, have patented this idea and in 1989, they asked Pr.

John Daugman (Teacher at Harvard University at this time) to develop some algorithms for iris recognition. These algorithms (based on Gabor wavelets) which Daugman have patented in 1994 became the basis for all iris recognition systems. This algorithms are implemented in a system called IrisCode R

In [Daugman 2002], the author explain his algorithm for iris recognition, which is performed in four successive steps. The approach starts by localizing, segmenting and normalizing the iris on image. To capture the rich details of iris patterns, it is necessary to ensure a minimum of 70 pixels in iris radius. Image captures are performed with sensors using NIR (near infra-red) illumination in the 700-900 nm band to be unintrusive to humans. Once iris localized and normalized, its features are extracted and encoded using 2D Gabor filters providing the iris "signature" (See figure 2.19). The dissimilarity measure between two irises is calculates using a fractional Hamming Distance. The last step consists in the final decision when two irises are compared and their dissimilarity measure is calculated. Generally, in this kind of tasks (check if two data matches), the usual way to take the decision is to define a threshold under which, a dissimilarity measure means that the two data effectively matches. This Wildes [Wildes 1997] proposed an alternative method for iris recognition, keeping the same steps than Daugman, but with different methods for each step. The localization and segmentation of the iris is performed by a circular and elliptic Hough transform, and the filtering is performed by Laplacian of Gaussian filters on four resolutions.

Miyazawa et al. [START_REF] Miyazawa | [END_REF]] introduce the concept of phase correlation for iris recognition. They use a phase correlation based on a band-limited Fourier transform to avoid low quality iris images. They proposed a method to normalize correlation scores according to the used image size after noise detection.

The IriTech company proposed another patented alternative system to IrisCode R of Daugman. In [START_REF] Kim | [END_REF]], the proposed approach uses Haar wavelets for multi- The state of the art in fingerprint recognition contains two main categories.

The first category concerns the conventional approaches which simply compare the relative positions of minutiae. In [START_REF] Jain | [END_REF]], the fist steps of processing consist in applying a directional filtering on fingerprint image, followed by a binarization and finally a by a thinning of ridges. The last step consists in determining minutiae positions on the image to quantify the similarity characteristics between two templates by "point pattern matching". Maio et al. [Maio 1998] propose an alternative method to localize minutiae in a direct way by using neural networks.

The second category concerns more complex approaches which extract and use more information from the fingerprint, like local directions of some minutiae [START_REF] Halici | [END_REF] and [Capelli 1999], or the local frequential component of textures on images [START_REF] Jain | [END_REF]]

Face Recognition

Face recognition has been the subject of many researches since many years, especially the two last decades. It has special attention from computer vision community due to its importance and effectiveness for people identification. The performances of face recognition systems have greatly increased since the first works in the 1960-1970s [Bledsoe 1964, Kelly 1971, Kanade 1977] and many new face recognition algorithms have been proposed since that time. These approaches can be categorized into three kinds according to the input information type: 2D-based approaches which use images and videos, 3D approaches based on 3D scanning, and finally hybrid approaches which combine both information. Unfortunately, these two last types of approaches, despite the highest performances that they can provide, have some drawbacks. The first one concerns high cost of 3D facial scanners. The second inconvenient is the high amount of information that have to be processed, which does not allow real-time processing. The last inconvenient concerns the unavailability of large datasets of 3D data for evaluation.

For these reasons, most of studies are carried on 2D-based approaches, which are more practical.

Face recognition methods can be divided in two main categories according to the used information level: global (holistic) methods and local methods, based on models.

Global methods are based on statistical analysis and does not requires any face features localization or extraction (like eyes, nose, mouth, etc.). In this methods, the whole face image is processed as grid of pixels which is generally transformed to vector, more practical to process (See figure 2.22). This transformation is accompanied by a dimensional reduction to model face in a low dimensional sub-space, more significant and with faster processing. Global methods are divided in two main types of techniques which linear and non-linear techniques which determine.

Linear techniques project initial information to a lower sub-space linearly. The most known technique is the Principle Component Analysis (PCA). This technique was initially used for face representation in [START_REF] Sirovich | [END_REF][START_REF] Kirby | [END_REF]] and was taken as "Eigenfaces" technique in [Turk 1991]. Some other techniques based on linear decomposition have been used, like Linear Discriminant Analysis (LDA) [START_REF] Belhumeur | [END_REF]] or Independent Component Analysis (ICA) [START_REF] Bartlett | [END_REF]]. The linear techniques present the main issue of inability to conserve the geometric manifolds contained in original face images.

This is due to the limitation to manage their non-linearity. To deal with this issue, linear methods have been extended to non-linear techniques based on "kernel" notion like

Kernel PCA [START_REF] Schölkopf | [END_REF]] and Kernel LDA [START_REF][END_REF]].

Global methods present the main advantage of being relatively faster, due to the medium complexity of required computations. However, they have are highly sensitive to variations of illumination, pose, and facial expressions, which represent an important issue. This issue is due to the fact that processed information is a low level one, directly extracted from pixel values and not encoded to more robust and invariant features. images. Other kinds of local approaches like Bayesian approaches [START_REF] Liu | [END_REF]], Support

Vector Machines (SVM) [START_REF] Guo | [END_REF]], Active Appearance Models (AAM) [START_REF] Cootes | [END_REF] or Local Binary Pattern method (LBP) [START_REF] Ahonen | [END_REF]] have been proposed. All these local methods have the advantage to model easily pose, illumination and face expression variations. Nevertheless, this kind of methods are more complex and requires more processing time in comparison with global methods.

Some works are based on a combination between global and local methods, providing hybrid method. Local Feature Analysis (LFA) [START_REF] Penev | [END_REF]] and extracted features by oriented Gabor wavelets, like Elastic Bunch Graph Matching (EBGM) [START_REF] Wiskott | [END_REF]] are some of hybrid methods examples. More recently, Log Gabor PCA (LG-PCA) [Perlibakas 2005] has been presented.

There are other approaches, based on neural networks [START_REF][END_REF]] or on Hidden Markov Models [Nefian 1998], but these approaches present important issues when the number of individuals increases, due to the complexity of computation. In addition to this, these two approaches requires many images for each face to train systems and to

Gait Recognition

Gait recognition addresses the problem of human identification by characterizing and discriminating the way they walk. Each person seems to have a distinctive way of walking. The differences in gait can be observed from many walking parameters, like gait cycle information (step frequencies and magnitudes) (See figure 2.24), person's pelvis/centroid vertical oscillations range, the maximum height of the foot when it leave the floor, etc.

One of the first attempts of automated gait analysis was performed in 1994 by Niyogi

and Adelson [Niyogi 1993]. Individual gaits is recognized by applying standard pattern recognition techniques to the contour of individual, extracted by snakes approach. Many edge.

In their approach, Canado et al. [START_REF] Cunado | [END_REF]] consider legs only. The extract lines which represent legs on images using the Hough transform. The change in inclination of these lines follows simple harmonic motion; this motion is used as the gait biometric. The method of least squares is used to smooth the data and to infill for missing points. Then, Fourier transform analysis is used to reveal the frequency components of the change in inclination of the legs. The transform data is then classified using the k-nearest neighbour rule.

Kale et al. [START_REF] Kale | [END_REF]] use two different image features: the width of the outer contour of the binarized silhouette of the walking person and the entire binary silhouette itself. From these two features, characterisation is performed following two different methods. In the first method, the high-dimensional image feature is transformed to a lower dimensional space. In second method, they work with the feature vector directly by training a Hidden Markov Model.

Wang et al. [Wang 2003a] use statistical shape analysis. Moving silhouettes of walking figures are extracted using a background subtraction algorithm. Temporal changes of the detected silhouettes are then represented as an associated sequence of complex vector configurations in a common coordinate frame, and are further analysed using the Procrustes shape analysis method to obtain mean shape as gait signature. Supervised pattern classification techniques based on the full Procrustes distance measure are finally used for recognition. [START_REF] Liu | [END_REF]] simply align and average the silhouettes over one gait cycle (See figure

2.25). The recognition is then performed using the Euclidean distance between these averaged silhouette representations.

Appearance-based approaches

Despite their effectiveness, the hard constraints of biometric approaches, presented in the previous section, make this kind of approaches inadequate for large wide videosurveillance purpose. The alternative consists in using global appearance information, which seems to be "easier" to extract in term of constraints. Nevertheless, despite the availability of this information, a major problem remains: how to use the global appearance information to characterize an individual in a robust and reliable way? As people re-identification concerns a large set of individuals acquired from different cameras, under several conditions, it is necessary to provide a distinctive and invariant to camera changes signature. This visual signature has to be invariant to some external variations, like changes in illumination, person pose and orientation and camera point of view. It has to be robust against internal camera parameters also. The sensitivity of different camera sensors and the color acquisition vary from a camera to another one, providing different images of the same person in terms of color (See figure 2.26). This last issue has been handled in two different ways in appearance-based re-identification methods.

Many approaches use some color normalization methods without any a priori information about camera parameters, while other methods are based on multi-camera color calibration, using some colorimetric transfer functions. The simplest approaches are based on statistical computations on the whole person images, using low level information and thereby, avoiding spatial information, applying a histogram representation. More complex techniques uses more sophisticated features, taking into account spatial information which is very useful to solve matching problem.

This spatial information can be classified into two categories: As a result of feature extraction techniques or as an initialization for them. In the first case, feature extraction does not use any a priori information concerning the localization of the wanted features.

The spatial information is obtained once the features are extracted, for instance the coordinates of points of interest, the coordinates of the centroid and the area of a stable color region, etc. In the second case, features are extracted in predefined regions. They can be the result of uniform subdivision of the person image (horizontal strips in [START_REF] Bird | [END_REF][START_REF] Cong | [END_REF] or grid [START_REF] Bak | [END_REF]]) or a specific region delimitation like human body parts [Bak 2010] or symmetry/asymmetry body subdivision [START_REF] Farenzena | [END_REF]].

The more sophisticated approaches can be categorized in two main groups accord-ing to the way on which they focus to perform the people re-identification. The first category concerns the Feature Oriented (FO) approaches, which focus on designing an invariant feature, which should handle viewpoint and camera changes [Bazzani 2010[START_REF] Farenzena | [END_REF]]. The second category contains approaches which concentrate on learning aspects or on feature modelling. Learning approaches use training data of different individuals to select the best features and to find the best way to combine them.

These approaches focus either on metric learning for matching appearances regardless of the representation choice [Dikmen 2011], or on discriminative methods which enhance discriminative features of a specific individual [Schwartz 2009].

Another classification of the existing approaches can be performed, based on the number of used images per person. According to this creteria, two families of approaches for appearance-base people re-identification exist: Single-shot approaches which extract information from a single image of a person and Multiple-shot approaches which use information of multiple images of a person to encode possible variations and learn a reliable representation of the person.

The two classification point of views intersect as much as some feature oriented approaches use single images per person while other feature oriented approaches are based on multiple images per person. Similarly, some learning approaches are performed using single images per person while other approaches require multiple images per person.

We take the second classification point of view as a basis for the presentation of the state of the art, as long as it relies on a primary parameter: the number of used images per person. Nevertheless, before exposing single-shot and multiple-shot state of the art, a brief presentation of existing works for colorimetric transfer function is presented first.

This task is important as long as it can allows to deal with difference in color acquisition between different cameras.

Colorimetric Transfer Function

As mentioned before, the acquired images of the same person by two cameras can present a significant difference in rendered colors (see figure 2.26). It may be due to sensor sensitivity difference or to external reasons, like difference in illumination or camera orientation. This issue becomes critical as soon as a re-identification process requires to compute a color-based people signatures.

To handle this issue, most of appearance-based people re-identification approaches integrate a color normalization step in their process, generally before the extraction of any feature. However, many researches have been done to provide a "link" between different color rendering. This task is called "Colorimetric calibration" between different cameras.

Porikli [START_REF] Porikli | [END_REF]] proposes an initial method for colorimetric calibration between different cameras, called "Brightness Transfer Function". He proposes a distance metric [ [START_REF] Gilbert | [END_REF]] approach is based on [START_REF] Porikli | [END_REF]], but integrates an on-line learning phase to update illumination variations between cameras, extending the use of correlation matrices. However, this approach is greatly sensitive to the initialisation phase of the Transfer Function and requires a large training dataset (more than 5000 trajectories).

[ [START_REF] Javed | Appearance Modeling for Tracking in Multiple Non-Overlapping Cameras[END_REF]] and later [START_REF] Javed | Modeling inter-camera space-time and appearance relationships for tracking across non-overlapping views[END_REF]] propose another extension of [START_REF] Porikli | [END_REF] approach. In this approach, a Brightness Transfer Function is estimated for each pair of These previous approaches assume that the considered objects are observed with relatively same point of view by the different cameras. Prosser et al. [START_REF] Prosser | [END_REF] address a more general case where objects are not observed with the same point of view.

In this case, the observed proportions of object colors may vary greatly from a camera to another one. [START_REF] Prosser | [END_REF]] propose a Cumulative Brightness Transfer Function (CBTF).

It cumulates many images of the same person, observed by the same camera, in the same histogram, before applying Porikli's method on the cumulated histograms, obtaining the Cumulative Brightness Transfer Function.

Colorimetric calibration presents two main issues for our addressed problematic.

First, colorimetric transfer functions are generally not bijective depending on the acquisition conditions (camera orientation, lighting conditions, etc.). Some large ranges of colors from a given camera may have unique color (in case of discretization) or a smaller range of colors as correspondence. This phenomenon is most observable for very low/high saturation values. This decrease the efficiency of colorimetric calibration.

The second issue is more important for our work since it is related to the difficulty to apply this kind of approaches in large scale video-surveillance systems. It requires to annotate a sufficient pairs of persons observed by each pair of deployed cameras. This is not conceivable as a generic method since these transfer functions are not related to the internal cameras only, but also to external conditions (light conditions, orientations, etc.).

In the following paragraphs, the main approaches of the state of the art in appearancebased re-identification are presented. These approaches are divided in two categories:

Single-shot approaches and Multiple-shot approaches.

Single-shot Approaches

In the single-shot methods, the re-identification is performed using a single image for each person. As mentioned before, two categories of approaches can be distinguishes: feature-oriented approaches and learning approaches.

Feature-oriented approaches

Feature-oriented (FO) approaches focus on finding the best feature representation to be invariant to the possible variation.

In [Park 2006], the proposed approach, called Visual Search Engine (ViSE), is based first on a segmentation of the person's body in three parts: the head, the torso, and the legs. This segmentation aims at keeping spatial correlation of feature distributions and is based on approximative dimensions of each body part with respect to the whole height (the two separation lines are at 1/5th and 3/5th of the person height). The head part is ignored due to the low discriminative power of its features when no biometric techniques are used. Each shirt and pants region In [Gallagher 2008], a collaboration between clothing segmentation and characterization in one side, and facial features in the other is performed to recognize individuals. The face detection is performed using [START_REF] Viola | [END_REF]] and the clothing segmentation is obtained using graph cuts and clothing mask. Each face is characterized by a 37-dimensional vector, obtained by projecting the face image onto a set of Fisherfaces [START_REF] Belhumeur | [END_REF]]. In [START_REF] Cai | [END_REF]], the proposed approach uses color patches to represent the person on interest. These patches are localized along edges, which are extracted using Canny edge detection algorithm [Canny 1986]. Each region is represented by the tance [Kullback 1968]. To thoroughly represent the information contained in a video sequence with as little data as possible, a key frame selection and matching scheme is proposed.

(A) (B) (C) (D) (E) (F) (G) (H)
In [START_REF] Kang | [END_REF]], they model the object of interest for reacquisition purpose using stochastic models. [START_REF] Belongie | [END_REF]], which uses a shape dictionary learnt a priori. The context of the appearance and shape is handled by using occurrence/co-occurrence function which describes probability distributions and their correlations over the image region.

[ Bak 2010] approach is base on spatial covariance regions extracted from human body parts. A human body part detector, based on Histogram of Oriented Gradient technique (HOG) [Corvee 2009] is trained and applied to detect 5 body parts: the top, the torso, legs, the left arm and the right arm. To handle color dissimilarities caused by camera and illumination differences, a color normalization technique called histogram equalization [START_REF] Hordley | [END_REF]] is applied. After that, the covariance regions of body parts are computed on normalized images to generate a human signature. The dissimilarities between these regions corresponding to different images are combined using an idea derived from the spatial pyramid match kernels [START_REF] Grauman | [END_REF]] (See figure 2.33).

Learning approaches

Learning approaches require training data to find the best way to perform appearance matching. We can distinguish two types of learning methods, according to the aspect on which they focus: Some approaches concentrate on metric learning (ML) for matching regardless of the person representation choice, while other approaches focus on the way to enhance discriminative features of an individual and are called discriminative methods (DM). [ Hirzer 2012] address the problem of cameras point of view and property differences by learning the transition from one camera to the other. The human representation is performed using HSV and Lab color channels for color information and LPB for texture information. The human image is divided on a grid of 8x16 rectangular regions, using an overlap of 50 % between regions in both directions.

In each rectangular patch, the mean values per color channel is calculated and discretized to the range 0 to 40. Additionally, a histogram of LBP codes is generated from a gray value representation of the patch. These values are then put together to form a feature vector. The vectors from all regions are concatenated to build a representation for the whole image. A PCA is applied for dimension-ality reduction. The proposed classification method is built on the ideas of Large Margin Nearest Neighbour classification solution. This is realized by learning a Mahalanobis metric using pairs of labelled samples from different cameras.

[ Ijiri 2012] proposes an other metric learning based approach to handle the same issue. To learn the optimal metric, human body images are divided into several vertically segmented regions following [START_REF] Bird | [END_REF]] approach. The color histogram of each body region is computed in HSV color space and the entire person representation is obtained by concatenating all the HSV part histograms. The corresponding label is assigned to each person histogram representation, providing a training dataset. The optimal metric learning is based on Large Margin Component Analysis (LMCA) [START_REF] Torresani | [END_REF]]. A non-linear projection is used to project the input histograms onto a higher dimensional space. For non-linear projection, several types of kernel functions were investigated (X 2 kernel, Bhattacharyya kernel, Jeffrey divergence kernel, Jensen-Shannon kernel). This approach is illustrated in figure 2.34. Concerning Discriminative methods (DM), we can cite [Lin 2008, Schwartz 2009] approaches. In [Lin 2008], the person appearance is represented with a 4-dimensional vectors, containing 3 color components and the height coordinate. Features are aggregated using the probability density function (PDF). The distance between two appearances is established using pairwise dissimilarity profiles which are learned beforehand. The nearest neighbour classification is adapted to perform re-identification.

In [Schwartz 2009], a rich set of feature descriptors based on color, textures and edges is used. Features are extracted from overlapping blocks constructing a highdimensional feature vector. The high-dimensional signature is transformed into a low-dimensional discriminant latent space using a statistical tool called Partial

Least Squares (PLS) in one-against-all scheme (the discriminative appearance of person is learned using information about the appearances of other persons). For the one-against-all scheme, PLS gives higher weights to features located in regions containing discriminative characteristics.

Opposite to Metric Learning (ML) approaches, the discriminative approaches learning is usually performed on-line.

Multiple-shot Approaches

In the multiple-shot methods, the re-identification is performed using multiple images of the same person. This allows to better take into account appearance variations of the same person and thereby, provides a more informative signature. As single-shot approaches, multiple-shot approaches can also be divided into two classes: featureoriented approaches and learning approaches.

Feature-oriented approaches

In the multiple-shot case, the availability of more than one image per person allows the use of powerful mathematical tools, such as clustering, Principal Component Analysis and many other ones, to extract most reliable features.

In [START_REF] Gheissari | [END_REF] In [Hamdoun 2008], the person signature consists in an accumulation of SURF points of interest [START_REF] Bay | [END_REF]] extracted from multiple images of each person. These cumulated points of interest are stored in KD-tree to speed-up the query processing time (see figure 2.36 (a)). The association of the models is obtained by a voting approach: every interest point extracted from the query is compared to all models points stored in the KD-tree, and a vote is added for each model containing the nearest descriptor. Finally the re-identification is performed with the highest vote for the model (see figure 2.36 (b)).

In [START_REF] Huang | [END_REF]], the person image is divided into three parts: the head, the torso, and the legs. This segmentation is performed using approximative ratios of 1/5, 2/5 and 2/5 (see figure 2.37 (c)). The head is ignored due to the low amount of extractable information without using biometric techniques. From the two remaining parts, a tree structure containing medians of RGB colors is extracted. The median value of a given node is used to separate its corresponding histogram and thereby creating the two child histograms. The final appearance feature is a vector obtained by merging median vectors. Finally, a Bayesian-based tracker combines a set of features using a multivariate normal distribution.

In [START_REF] Farenzena | [END_REF]], proposes another method to divide person body image into three main parts which are the head, the torso and the legs. The head is ignored in this approach too. This segmentation is based on maximizing the Euclidean distance between the colors of two adjacent parts (head/torso and torso/legs) while minimizing normalized areas difference between these two (the normalization is obtained by the ratio between foreground pixel of the part and its whole bounding box rectangular area, using a mask). Another segmentation, using symmetry axis, is performed on torso and legs to ensure robustness against people rota- The final matching score between two extracted signatures is the weighted sum of distances between every couple of similar features, belonging to the two signatures.

Learning approaches

In multiple-shot approaches, the different approaches often use dimensionality reduction methods, classification using SVM or Boosting and Fisher discriminants to discriminate between different individuals. mean covariance is computed for each patch location, using all patches at the same location, upon all the images of each person (see figure 2.41). A person is represented in a first time by a set of mean covariance regions of all its patches. The final person signature is obtained by selecting the most significant patch and removing most variable ones, assuming that these last patches are the noisiest ones.

Authors propose two ways to perform this selection. First, a reliability measure on patches is introduced. For each region, this reliability measure is computed using the standard deviation which is associated to the mean covariance. Only patches with a high reliability measure are used for person signature (see figure 2.42(b)).

The second way for significant patches selection is performed using a boosting scheme. Finally, a new similarity measure between signatures using Riemannian manifold theory is proposed. It enables to hold discriminative power coming from the relative position of MRC patches.

Context-Aware Approaches: Group Context / Space-time Approaches

All the previous cited approaches, both in single-shot and multiple-shot categories focus exclusively on the visual information contained in images of persons of interest only. Some other works, called Context-Aware approaches, use additional information to increase the discriminative power of their appearance-based approaches. This information can be classified into two groups: Group context information and Space-time information.

Group context information still uses visual information extracted from images, but does not focus on the person of interest silhouette only. It takes into account surrounding people also. This kind of approach is relatively recent in comparison with all previous cited techniques and thereby, very few works have been published. However, it will certainly attract more attention in the future, due to the improvements that they can provide, according to published works results, which are still medium, but which open many perspectives (keeping the main idea of using group information, but proposing more sophisticated techniques to deal with non-rigidity of people in a group, partial occlusion depending on the camera position, etc.).

In [Zheng 2009], a novel people group representation and a group matching algorithm are proposed to reduce ambiguity in person identification. Assuming that in a crowded public space, people often walk in groups, either with people they know or strangers, visual information coming from the surrounding is also used. The appearance of the group is represented by visual words. First, each pixel is represented by a feature vector consisting in a concatenation of SIFT [START_REF] Lowe | [END_REF]] features for each RGB channel of this pixel. Then, a code book of n visual words is built by quantizing the previous fea-ture vectors into n clusters by k-means. Finally, the group is represented by combining two descriptors: a center rectangular ring ratio-occurrence descriptor which aims at describing the ratio information of visual words within and between different rectangular ring regions, and a block based ratio-occurrence descriptor to explore more specific local spatial information between visual words that could be stable.

In [START_REF] Cai | [END_REF]], covariance descriptor [Tuzel 2006] are used to encode group context information to improves person re-identification performances. The appearance of a person as well as the appearance of a group have been represented using covariance descriptor.

Space-time information is another useful information which allows better people re-identification performances. It can be used either as a filtering method, to reduce the space of possible candidates for a given re-identification request before applying appearance-based matching, and this by removing all candidates which do not respect spatio-temporal coherency (A person cannot be at two different locations at the same time. He/She cannot travel a given distance in a time which corresponds to incoherent velocity, etc.), or as an identification information (especially when camera fields of view overlap) by validating the same location of the same person at the same moment in the different camera images. The spatio-temporal information may be provided as an input information (floor plans with distances and possible paths provided to the system which manage the camera network), or learned automatically by the re-identification methods.

In [START_REF] Javed | Tracking Across Multiple Cameras With Disjoint Views[END_REF]], a novel approach for establishing object correspondence across nonoverlapping cameras is proposed. The multi-camera tracking algorithm exploits the redundancy in paths that people and cars tend to follow (e.g. roads, walk-ways or corridors), by using motion trends and appearance of objects, to establish correspondence.

The proposed method does not require any inter-camera calibration, instead the system learns the camera topology and path probabilities of objects using Parzen windows, during a training phase. Once the training is complete, correspondences are assigned using the maximum a posteriori (MAP) estimation framework. The learned parameters are updated with changing trajectory patterns. [START_REF] Iwama | [END_REF]] propose a combined approach using group and space-time information. The relationships between the people in an input sequence are modelled using a graphical model. The identity of each person is then propagated to their neighbours in the form of message passing in a graph via belief propagation, depending on each personś group affiliation information and their characteristics, such as spatial distance and velocity vector difference, so that the members of the same group with similar characteristics enhance each otherś identities as group members.

Discussion

In this section, we have presented several existing approaches for people re-identification.

These approaches are classified according to a first level criterion which is the nature of used information, and each criteria is categorized using many sub-criteria.

We have seen that the first family of approaches, belonging to biometrics methods (iris, finger print, face and gait recognitions), are the most powerful methods to perform the people re-identification task, due to the discriminative power of human biometric characteristics. At the same time, these methods are the most constrained ones also. They require many conditions which cannot be ensured most of the time in video surveillance systems. The most important constraints we can cite are the voluntary collaboration of individuals (the acquisition of iris and fingerprint images by specific sensors), the restricted situation in which these approaches can be performed (visible faces for face recognition, walking direction allowing the segmentation of gait, i.e. as far as possible from to the camera optical axis direction), high resolution images (to provide face features) and high frame-rate acquisition (to capture all the gait cycle parts).

The second family of approaches, related to appearance-based approaches, are less constrained than biometric ones and thereby can be used on large wide surveillance camera systems. These approaches are separated into single-shot approaches which require only a single image of a person to model his/her appearance, and multiple-shot approaches which requires multiple-images. In both of these categorises, the appearance representation can be extracted using feature-oriented procedures or by learning approaches.

Another group of approaches, called context-aware approaches, integrate additional information coming from environment to achieve people re-identification. Group-context information is used to improve individual signature by integration his surrounding people. Spatio-temporal information is also used to increase the reliability of re-identification.

Multiple-image approaches seem to be the ones which provide better results, due to their capability to encode the possible variation of human appearance and to extract/learn the most significant appearance model. Nevertheless, they have three main issues:

Their computation cost is relatively high. The complexity of the extracted features and the learning techniques require complex computations, slowing down greatly the re-identification process. On the other hand, many approaches require the availability of a large set of images of a given person person of interest and a large set of other people images before performing the learning, i.e. they cannot compute the visual signature of a given person progressively, starting with the first image of this person and updating the signature every time a new image of him/her or of other people is provided (acquired by mono-camera tracking algorithm) . These issues make the use of this kind of approaches hard or impossible for live processing purpose. They are more suitable for off-line tasks like a posteriori people searching, where the processing time is less important and all people images can be extracted in a first processing pass, before performing re-identification in a second pass.

They are sensitive to the precision of extracted images of the same person. Images of a given person are extracted from the video sequence using background subtraction or people detectors. Due to many reasons (background clutter, occlusions, etc.), the person silhouette can be badly aligned at different location of the extracted sub-images, or may even be truncated (missing occluded part, bad background subtraction segmentation, etc.). These issues may compromise the signature extraction as long as the learning of variations of a given body part feature requires to localize it at the same position on input images. Some approaches try to deal with this issue using a pairing step between images of a same person in term silhouette localisation. They perform this pairing using some features matching increasing computation time.

We propose a fast method to perform the pairing step before signature extraction.

This method is based on a fast color region comparison in Lab space, and it is detailed in the chapter 6.

Despite they claim their robustness against people orientation variation, it seems that all the cited approaches were evaluated on datasets containing manually annotated images showing the same side of each person or with a small range of rotations in each camera. In the case of a person with a full rotation in the scene, all his/her side images are acquired. If the appearance of front side differs greatly from his/her back side (due to a backpack for example), a single signature for his/her appearance may be insufficient to allow his/her re-identification. Two cases may occur with the presented approaches: either only few features are kept if the approach is based on a selection of stable features, providing poor signature, or the variation of each feature are encoded in a large model, which may increase the re-identification failure probability due to less discriminative signature.

We propose a context-aware approach, using spatio-temporal information, to divide a person signature into several sub-signatures corresponding to different visible sides. The processing time is not increased significantly as long as the signature extraction is performed using the same approach as if no signature subdivision is done. The sub-signatures are labelled according to the viewed side from which they are computed, allowing a faster matching process, while taking into account possible error of labels.

OVERVIEW OF THE PROPOSED APPROACH

In this manuscript, we propose a complete framework for people re-identification.

The proposed approach includes three successive tasks: people detection, people tracking in mono-camera context and finally, visual signature extraction and comparison for re-identification. This chapter presents a general description of these three steps. The details of these tasks can be found in the next three chapters.

People Detection

The objective of this detection is to provide reliable inputs for both mono-camera people tracking and for people re-identification tasks (see figure 1.8). Detected persons on video sequences will be used as target initialization for mono-camera tracking in some specific cases, and as confirmation for other cases which will be more explained in mono-camera object tracking chapter (See chapter 5 ). They will also be used as request or candidates definition for re-identification task in both cases: static images and video sequences. The aim of separating these two cases is related to the type of used reidentification approach. According to some constraints which will be discussed in the chapter 6 concerning people re-identification, this task can use single-shot or multipleshots to compute the visual signature of any person of interest or any candidate.

Like the most of state of the art approaches for people detection, the proposed approach follows the same global schema which consists in two main and separate steps:

classifier training and people detection.

Chapter 3: Overview of the Proposed Approach

The proposed approach strongly optimize a state of the art approach proposed by Tuzel et al. [START_REF] Tuzel | [END_REF]] and improved by Yao et al. [Yao 2008]. It keeps the original and improvements steps, but introduce an additional processing step in the training stage which speeds-up strongly both training and detection phases and improves detection performances.

A short discussion about the generalizability of the proposed optimization and its possible application on any training approach based on boosting to learn a cascade of classifier, regardless of the type of used features, is provided at the end of chapter 4

An overview of the proposed people detection algorithm is presented in the following sections. It contains the three main parts of people detection approaches: the pertinent features extraction, the classifier training and finally the candidate region selection for detection.

Pertinent Feature Selection: Region Covariance Descriptor

To highlight the improvements provided by our optimization method to people detector approaches, we have taken Tuzel et al. [START_REF] Tuzel | [END_REF]] approach as baseline, and

we have shown the improvements of our proposed approach by comparing our results with those of [START_REF] Tuzel | [END_REF]] and also with those of [Yao 2008] which proposed other improvements to the original method, and with several other state of the art detectors.

The used features in these two approaches [START_REF] Tuzel | [END_REF], Yao 2008] and thus in our approach is Region Covariance Descriptor. Region covariance descriptor is a powerful way to encode a large amount of information inside in a given image region. It allows the encapsulation of a large range of different features in a single structure, representing the variances of each feature and the correlation between features.

Our choice to base our work on the approach of [START_REF] Tuzel | [END_REF]] is partly due to this descriptor. In fact, Region Covariance Descriptor differs from the other local descriptors in two main ways.

First, Region Covariance Descriptor can be considered more as a generic container for various features, with a powerful bag of mathematical tools than a "rigid" local descriptor like SIFT [START_REF] Lowe | [END_REF]], SURF [START_REF] Bay | [END_REF]], HOG [START_REF] Dalal | [END_REF]], LBP [Ojala 1996] and other local descriptors. In fact, while each local descriptor uses specific image information in a specific way, imposing a kind of rigidity to it, region covariance descriptor allows to use a large set of "basic" features (large amount, as there is more extractable basic features for each pixel). For example, both SIFT and HOG are based on image gradients only while LBP uses the values of direct neighbouring pixels of each pixel of interest. Some authors have combined some features to create hybrid features, by concatenating different descriptors for example but this kind of technique does not provide the relationship between the different features belonging to the hybrid one. In contrast, Region Covariance Descriptors allow to uses several features in the same structure, and provide the information about the relationship between these features.

The second main difference is the scope of the descriptors or in other terms, the area of image that they describe. Local features generally describe areas with constant sizes.

For example, unitary SIFT descriptor, in its standard version, is computed on a square region on 16x16 pixels and provide, after specific subdivisions on sub-regions of 4x4 pixels and bins separations on 8 bins, a feature vector of 128 dimensions. Of course, it is possible to enlarge or reduce the computing window size, but it requires important modifications, either in the subdivision method or in the resulting descriptor size.

It also requires to ensure some constraints concerning the size of computing window and its square shape: for example, it is impossible to divide a square region of 17x17 equivalently or to use a rectangular window without losing some robustness to rotation in comparison to the standard version. In contrast, a region covariance descriptor can describe a very small region of few pixels in the same way that it can describe the whole image, without any necessary adaptation, and independently of the shape of the region (square or rectangular one), providing descriptors with same structures and dimensions.

In fact, the dimension of a covariance descriptor relies to the size of the used feature set and not to the size of the described region.

Due to these advantages, region covariance descriptors will be used in the re-identification part by changing only the used basic features.

More details and explanations about Region Covariance Descriptors and their advantages are provided in chapter 4.

Classifier Training: Cascade of Classifiers Using LogitBoost Algorithm in Riemannian Manifold

Our

Candidate Regions Selection for People Detection: Dense Searching, Real World Candidate selection and Background Subtraction Filtering

Three methods for candidate regions selection are used. Dense searching method is the standard and the usable method in any case because it does not requires any additional external information. It can be applied on any image, but it is also the slowest method due to the highest number of tested candidate regions.

Whether camera calibration information is available, the candidate region selection is focused on the areas where it is more likely to find people, i.e. all areas of the image touching the ground floor in the real world. If scene context information is available too, this targeting is more precise by avoiding all areas that can not contain people due to the presence of scene static objects (Walls, buildings, etc.).

The last selection method, which is mainly used for our mono-camera tracking algorithm, is based on motion regions targeting. This method can be used in both previously mentioned methods, as long as it is applied on video sequences. It is more a filtering method than an independent one for candidate regions selection. It consists in focusing the detection only on moving objects, detected by background subtraction algorithm, and using camera calibration to reduce scale and width/height ratios of searching windows. Using this filtering method reduces strongly detection time, but can avoid some detection in case of static people on images.

Due to the context of our study and the work hypothesis concerning the use of static and calibrated cameras, all the possible candidate regions are pre-computed once before the detection or during the detection on the first frame of the input video sequence.

These candidate regions are stored and used on each new frame where motion is detected, avoiding the necessity to calculate the 3D-2D projections at each frame.

Mono-Camera Object Tracking

In our study, the objective of tracking people in mono-camera context consists in two main points: it allows to extract and to learn robust appearance-based visual signatures and it provides useful information for spatio-temporal filtering step during people reidentification.

To re-identify people in a camera network using appearance matching, it can be sufficient to detect all appearing persons on all the frames of available video sequences using a people detector, and extract single-shot visual signatures for each detected person. However, this method increases strongly the number of possible candidate as it considers each detection as a different person, due to the non-use of temporal information. Generally, it is better to use multiple-shot based visual signature for many reasons which are detailed in the re-identification chapter (see chapter 6). Using multiple images of a person to compute its visual signature requires to detect him on several frames on the video sequences provided by one camera, and to ensure that all these detections represent the same person. Mono-camera person tracking is the best way to perform this task.

We also use person tracking information to robustify our visual signatures by partitioning it in sub-visual signatures according to the visible side of a person in a given camera. In fact, the motion direction of a person with respect to a given camera can be easily extracted and used to infer if that this person is seen from front/behind or in profile, so the extracted visual information can be assigned to the right person position and allow more precise comparison during re-identification process. All these aspects are more discussed in the chapter 6.

Finally, mono-camera people tracking with calibrated cameras provides also useful information for spatio-temporal coherency filtering in both cases of overlapping and non-overlapping fields of view between the cameras. In case of overlapped fields of view, only people in the neighbourhood of a given person at a given time are considered as candidates for re-identification. In case of non-overlapped fields of view between cameras, using buildings plans or city maps or any contextual information, in addition to camera calibration information, allows to avoid incoherent and unlikely candidates in some cameras for a targeted person in an other camera, by checking distance/velocity coherency or possible trajectories.

We propose a mono-camera "object" tracking framework based on SIFT features (Scale-Invariant Feature Transform) [START_REF] Lowe | [END_REF]] tracking by particle filtering. It mainly use detected moving object by background subtraction as target initialisation and candidates providing, and in some cases which will be discussed in chapter 5, the people detector we present in chapter 4. A data association method, based on the reliability of tracked SIFT features, is proposed to pass reliably from point tracking to object tracking to create the temporal links between objects of each frame and thereby provide objects trajectories. This data association method also allows to detect partial and full occlusion situations, which are managed by our proposed "fast occlusion managment approach".

This last task is performed using several information: SIFT Features matching, dominant colors descriptor and "real world" object information like size and velocity (provided thanks to camera calibration).

This framework is a generic object tracking algorithm which can track any kind of objects. People tracking is performed by targeting only people among detected moving persons, either by using the "real world" information of the object (especially the real dimensions), by applying the people detector or by using both. All the cases are discussed in the chapter 5. It is also generic in term of used features for tracking as long as the object model consists in sparse local descriptors. In fact, in our presented work, we use SIFT features for this task, but any interesting local descriptor can be used as long as it ensures invariance and robustness to classical challenges.

Object Detection: Background Subtraction VS. People Detection

Our object tracking algorithm performs by creating temporal links between detected/tracked objects on the frame at time "t-1" with their corresponding objects detected on the frame at time "t". It means that objects have to be detected at each frame in a fist stage. At time Focusing on a given type of objects can be performed using camera calibration information, by checking the estimated real world dimension of any detected object and its velocity for example and verifying that it correspond to known value ranges. This method provides good results in simple situations, but it fails when some complicate cases occurs: shadows, grouped people, closest objects, etc.

We propose to use the presented person detector in collaboration with background subtraction to focus on people by applying it on the moving image regions.

SIFT Features Detection and Selection

In the proposed approach, tracked object are modelled by a set of point features.

This choice is justified by two main reasons: the flexibility of points representation of an object allow to deal with objects deformations and rotations easily since points are independent. The object occlusions can be detected easily by noting the disappearance of a part or all used points.

Once one target object detected, we model this object by a set of SIFT features. A SIFT feature [START_REF] Lowe | [END_REF]] consist in a SIFT point of interest, detected following some conditions, around which a SIFT descriptor is computed in specific way. This descriptor is assigned to the point of interest.

The several steps of SIFT feature computing, which will be recalled in chapter 5, ensure their robustness and invariance. The optimal algorithm parameters and thresholds provided in [START_REF] Lowe | [END_REF]] ensure the best robustness and invariance level. However, these parameters are too restrictive to ensure a minimal representation of object of interest with enough amount of points, and with a whole covering. Small and noisy object images provided by many video surveillance systems may be fully or partially empty in term of SIFT points.

The proposed approach changes some SIFT points detection parameters (curvature and contract thresholds) and provide more detected points. This allows to ensure a sufficient number of points on all the object image, even if the robustness of these points is decreased. The reliability decreasing of SIFT points is compensated by the proposed general data association frameworks witch use reliability measure on tracked SIFT features to create the temporal links between objects.

The approach filters the detected SIFT points using a grid subdivision of the object image. A uniform distribution of SIFT features on the whole image is then provided by this method.

SIFT Features Tracking by Particle Filtering Method

Due to the cited advantages of particle filtering methods for tracking (see sec. 2.2.2.2), the proposed approach track detected and selected SIFT points using a customized particle filter. Each SIFT point is tracked using a set of particles. The point tracking is performed by the two standard steps which are perdition and update.

Each SIFT point movement is characterised by an adaptive dynamic model, based on SIFT points position and velocity. Our approach use a linear regression function, computed on a definite number of the last positions of the SIFT point, to adapt the motion model. By this way, movement velocity and direction changes are well handled.

The prediction step for each SIFT point tracking is performed by applying its motion model to all associated particles, including a random noise estimation to each particle location projection.

Once all particles are projected in the current frame, the update (or correction) step provide the estimation of the current location of the tracked SIFT point as the centroid of all its particles. Before new SIFT point location estimation, its particles are weighted, sampled and resampled to make them focus on the most likely positions for SIFT point.

We use the Importance Sampling Resampling method due to its ability to avoid information degeneration (see section 2.2.2.2). Our method propose a new hybrid weighting method for particles.

Hybrid Particles Weighting for Sampling Resampling Step

Most of state of the art particle weighting in object tracking particle filters methods are based on a similarity measure between the used descriptors. The proposed method keeps this weighting technique since the similarity measure is an important information, but due to the addressed context (video surveillance), the low resolution of image, the small size of objects of interest, and the low contrast with background may greatly alter this similarity measure. A practical example is provided in chapter 5. To avoid this risk, a second information type is used for weighting.

The proposed approach combine background subtraction result with the similarity measure in an effective way to ensure a better weighting process. The background subtraction result is used in a continuous way even if it is provided in binary form. Our hybrid weighting method has the other advantage to be robust to background subtraction algorithm performances. It is based on the moving pixel density calculation, and is explained in chapter 5.

Data association: Temporal Links Creation

The previous steps allow to track SIFT points independently. To infer the object of interest movement and localization, a data association step, based on tracked SIFT points, is performed.

Temporal links between objects of successive frames are built depending on the localisation of tracked SIFT points on the currently detected object, weighted by their reliability measures. This method is faster than a basic detected points matching.

The proposed approach allows to detect all tracking situations: single objects moving, grouping object, separation of group of objects, and occlusions. This is performed using the tracked SIFT points localization and visibility on current frame. This method is detailed in the chapter 5.

We propose a method to build these temporal links in a reliable way, using all available information from SIFT features: (1) The spatial repartition of SIFT points to detect occlusions and to delimit the object and (2) the reliability of SIFT features to weight all the possible links between tracked objects at time "t-1" and the detected objects at time "t"

Fast Occlusion Management

The proposed data association method allows to detect occlusion situations thanks to point representation and tracking.

Partially occlusions are handled by a continuous tracking process on the remaining visible SIFT points. If the partially occluded object became fully visible, new additional SIFT points are detected and assigned to the tracked object using the same process as the one described for SIFT points detection and selection. This is due to maintain a global and well distributed representation of the object Full occluded objects are handled differently. If a tracked object is lost due to a full collusion, it is stored as an "object to reaquire".

During the tracking process of a fully visible object, some additional features are extracted and used to "learn" a coarse model for reacquisition purpose only (not for tracking). This model consists in a combination of the variation of real world object dimension and velocity extracted thanks to camera calibration in addition to the extracted dominant color descriptor (the two main colors are considered in our approach).

A matching between the last visible set of SIFT point of occluded object (taken before occlusion starting) and those of a non linked candidate object in current frame is also performed. We can afford this computational time consuming task since it is not performed in each tracking iteration (on each video frame), but on specific cases.

Note that this last matching method is used as a validation-only criteria. If a candidate object present a high matching rate of matching SIFT points with the previously occluded one, and according to defined thresholds, this candidate object can be considered as the required one of the previously occluded object. Otherwise, if SIFT point matching fails, the candidate point is not rejected. The previous cited model for reacquisition is used to test object matching. This is due to the fact that occluded object may reappear in another pose or orientation with respect to camera, hiding its previously used SIFT points. The object model for reacquisition is a more general one witch can be used even if object pose or orientation change during the occlusion.

Once a previous fully occluded object have reappeared and reacquired, new SIFT points are detected and assigned to the tracked object using the same process as the one described for SIFT points detection and selection.

Finally, non linked detected objects in the current frames, either by continuous tracking or by reacquisition, are used to initialize new object tracking since they are considered as new appearing objects in the scene.

People Re-identification

People re-identification is the final aim of the proposed framework. The proposed approach is based on [START_REF] Farenzena | [END_REF]] approach which provides good performances and which performs in real-time (or in pseudo-real time if the number of considered images per person is larger). Some of the baseline issues and more general state of the art approach issues are identified and some solutions are proposed to solve them, improving the performances of the approach, making it more generic (no off-line training or video-operator interaction are required), while maintaining low processing time requirements.

The initial approach of [START_REF] Farenzena | [END_REF]] performs in three steps:

First, human body image is divided into four parts using two asymmetry axes which separates vertically the head from the torso and the torso from the legs, and two symmetry axes which separate horizontally torso into two parts, and legs into two parts also.

The head part is ignored in this approach because of low amount of information it provides in general case (large scale video surveillance). This body subdivision is performed using both foreground pixels separation and their color values.

Once the human body separation performed, three different features are computed on each body part: Weighted color histograms (WH) in HSV space, Maximally Stable

Color Regions (MSCR) and Recurrent High-Structured Patches (RHSP). The final signature consists of the combination of these features.

Finally, the comparison between two signatures is performed by computing a dissimilarity measure between them. This dissimilarity measure is the sum of weighted dissimilarity measures between each pair of similar features (WH). The weights are fixed once by an off-line learning step on a subset of images from VIPeR dataset.

In the following paragraphs, the main identified issues of this baseline approach (and some other more general issues) as the proposed solutions are briefly exposed. We propose method to classify people images according the their visible side, using real world information, provided by our mono-camera tracking algorithm. The people trajectories are segmented according their walking directions. Assuming that people walk forward, their trajectories indicate their orientation and thereby, their visible side with respect to the cameras, thanks to some processing steps using camera calibration information. These processing steps are detailed in chapter 6. The unique visual signature is replaced then by a set of sub-visual signatures assigned to each visible side class.

Dependency of Visual Signatures to People Orientations

This classification method is accompanied by the corresponding signature comparison method, dealing with the all possible cases (common visible side classes, adjacent visible side classes, neither common nor adjacent visible side classes).

Unreliable Body Subdivision + Images Alignment Issue for Multipleshot Case

The proposed method for human body subdivision (symmetry and asymmetry axes estimation) is mainly dependent on the quality of background subtraction and to the contrast between people and the foreground. Generally, the estimated axes are not correct but more or less inaccurate. For small shifting of the asymmetry axes with respect to their real localization, the final visual signature alteration is negligible. For this reason, we have decided to avoid this processing step and to use statistical subdivision of human body, by positioning the head-torso separation line at 1/5 of the human height (starting from the top) and the torso-legs at 3/5 of the human height.

This solution may cause an important issue: it is dependent on the quality of people delimitation. People who are not well centred on the used images (or bounding boxes)

or who have some missing parts (cropped head, feet, etc.) due to bad background subtraction or people detection results provides bad statistical body subdivision. For this reason, and to improve part-to-part comparison, we propose a fast method to align correctly all the images of a given person. This allows to correct some images by shifting people to the center and by removing background in images margins, and also to remove images with significant error (cropped important parts). This image alignment is performed using fast browsing of superposed images using Lab color space for similarity maximization. Some processing steps like image downscaling and camera calibration information use speed up the alignment processing.

Exclusive Use of Unnormalized Color Information

The initial approach of [START_REF] Farenzena | [END_REF]] uses images directly without managing the difference in color rendering between cameras issue. We propose to use color histogram equalization to minimize the effect of this issue. We prefer this kind of approaches instead of colorimetric camera calibration due to the issues and constraints of this last approaches, which consist of non-bijectivity of transfer functions and the complexity of their application in large scale video-surveillance system.

On the other hand, [START_REF] Farenzena | [END_REF]] characterize a visual signature exclusively using color information. Weighted color histograms and Maximally Stable Color Regions are fully color-based features. Recurrent High-Structured Patches, even they are selected using texture information (patches entropy and LNCC maps), their final characterization is performed using simple color histograms too.

We propose to replace the simple color histograms which characterize RHSP features by covariance descriptors which are built using both color and texture information, improving the discriminative power of RHSP features.

To use additional texture information, we propose to add tracked SIFT features the the final signature too. All the required modifications in the initial approach to manage this add are performed (modification of the signature comparison by adding a weighted dissimilarity measure of SIFT features in the final dissimilarity measure between signatures).

Both SIFT and RHSP features being local descriptors, their use is controlled and managed by the adaptive feature weighing system we propose and which is described in the next paragraph.

Fixed Weights for Each Descriptor

In the baseline approach of [START_REF] Farenzena | [END_REF]], The weight of each feature in the final signature comparison is fixed after an off-line learning on a part of the used dataset. This poses the three following issues:

First, the off-line learning of the best weights is not conceivable for our work since we try to provide a generic and turnkey system for people re-identification, which does not require any external interaction after deployment.

Second, the used weights are the same for the whole dataset. We believe that this kind of weighting method is not the best one. In fact, for each person to re-identify, the system may focus more on the most discriminant information in this person appearance.

A given person appearance may be highly rich in terms of color while it is poor in terms of textures (uniform large color regions on the cloths for example) and another person appearance in the same dataset (or in the same camera network) may be poor in terms of color (a unique dark color for example) while it is rich in terms of textures (created by another unique color). These two persons may not be compared with candidate people using the same features weights. The first person re-identification may focus more on color information while the second person re-identification may focus more on texture, event whether both people are in the same dataset or observed by the same camera.

Finally, using the visible side classification, the local nature of SIFT and RHSP features implies the integration the people orientations in these two feature weighting.

It means that SIFT and RHSP feature weights vary greatly for the same query person according the compared visible sides.

To deal with these issues, we propose an adaptive weighting method, to assign the adequate weight to each feature of the visual signature, according to the richness/discriminative power of each type of information (Color and Texture) and to the considered visible sides. The assigned weights are then not dataset/camera network related, but visible side and information richness/discriminative power based for each person. The heterogeneity of the computed distances for the several people re-identifications is not an issue since the aim is to find for each query person, the most likely corresponding candidate, independently on the other re-identification queries.

EFFICIENT PEOPLE DETECTOR BASED ON COVARIANCE DESCRIPTORS

This chapter describes in detail the proposed people detector, mainly based on Tuzel et al. [START_REF] Tuzel | [END_REF]] approach and its improvements by Yao et al [Yao 2008]. This people detector is based on a cascade of classifiers, trained using LogitBoost algorithm on Region Covariance Decriptors to detect full human body on both static images and video sequences. The process is performed in two separate steps: an offline training step, and a classification step.

Region Covariance Descriptor

Region covariance descriptors are a powerful way to encode a large amount of information inside a given image region. Unlike the concatenation of several feature vectors, which provide a final vector containing independent feature information, a Region Covariance Descriptor allows the encapsulation of a large range of different features in a single structure, representing the variances of each feature in the represented image region and the correlation between these features.

Tuzel et al. [Tuzel 2006] first introduce the use of covariance matrices as a descriptor for object classification.

Let I be an image of dimension W × H. We can extract at each pixel location x = (x, y) T a set of d features such as intensity, color, gradients, filter responses, etc.

C R = 1 S -1 S k=1 (z k -µ)(z k -µ) T (4.1)
where µ is the mean vector of the points z k and S the number of pixels within R.

The diagonal entries of covariance matrix represent the variances of each feature, and the non-diagonal entries are their respective correlations.

Fast Covariance Computation Using Integral Images

A large number of covariance descriptors are required to achieve the training of classifier cascade and for an effective process. The computation of all the feature sums, means and variances for each region has a high cost in term of processing time. To deal with this issue, Integral Images are ideally suited to minimize the number of numerical operations.

Integral Images are intermediate image representations used for the fast calculation of region sums [Simard 1999, Viola 2001]. Each pixel of the integral image is the sum of all the pixels inside the rectangle bounded by the upper left corner of the image and the pixel of interest. For an image (i), the Integral Image value at a pixel coordinates (x, y) is given by:

II(x, y) = x ′ ≤x y ′ ≤y i(x ′ , y ′ ) (4.2)
The Integral Image of any image can be computed efficiently in a single pass over the considered image, using the fact that the value in the summed area table at (x, y) is just:

II(x, y) = i(x, y) + II(x -1, y) + II(x, y -1) -II(x -1, y -1) (4.3)
The main interest of Integral Image representation is the possibility to compute the sum of pixels in any rectangular region of the image (see figure 4.1) using a constant number of 4 access and 3 simple mathematical operations: Integral Images can be used to reduce greatly Region Covariance Descriptors computation. From equation 4.1, any entry C R (i, j) of the covariance matrix can be written as:

x 1 <x ′ ≤x 2 y 1 <y ′ ≤y 2 i(x ′ , y ′ ) = II(x 2 , y 2 ) + II(x 1 , y 1 ) -II(x 1 , y 2 ) -II(x 2 , y 1 ) (4.4) (0, 0) (x 2 , y 2 ) (i) i x 1 x 2 y 1 y 2 (x 1 , y 1 )
C R (i, j) = 1 S -1 S k=1 (z k (i) -µ(i))((z k (j) -µ(j)) (4.5)
This equation can be rewritten as:

C R (i, j) = 1 S -1 S k=1 z k (i)z k (j) - 1 S S k=1 z k (i) S k=1 z k (j) (4.6)
The covariance matrix in a given rectangular region R can be computed by firstly computing the sum of each feature dimension z(i) i=1...d as well as the sum of the multiplications of any two feature dimensions z(i)(j) i,j=1...d .

An integral image P(x, y, i) is computed for each sum of each feature dimension i and an integral image Q(x, y, i, j) is computed for each sum of the multiplication of any Considering any rectangular region R(x 1 , y 1 , x 2 , y 2 ) of image, and using equation (4.6) and equation (4.4), it is possible to compute the associated covariance matrix in a constant number of operations as:

C R(x 1 ,y 1 ,x 2 ,y 2 ) = 1 S -1 Q x 2 ,y 2 + Q x 1 -1,y 1 -1 -Q x 2 ,y 1 -1 -Q x 1 -1,y 2 - 1 S (P x 2 ,y 2 + P x 1 -1,y 1 -1 -P x 2 ,y 1 -1 -P x 1 -1,y 2 ) (P x 2 ,y 2 + P x 1 -1,y 1 -1 -P x 2 ,y 1 -1 -P x 1 -1,y 2 ) T (4.7)

Used Features

For people detection purpose, we use an initial set of 8-dimensional set for features, like in [START_REF] Tuzel | [END_REF]] and in [Yao 2008]. Tuzel et al. [START_REF] Tuzel | [END_REF]] use the following set of pixel features (see figure 4.2):

x y |I x | |I y | I 2 x + I 2 y |I xx | |I yy | arctan |I x | |I y | T (4.8)
where: x and y are the pixel coordinates Despite the improvements provided by the use of these two background features instead of the two second order derivative ones, we have used both set of features. The final evaluation and comparison of our people detector is performed using [Yao 2008] features but some necessary tests and comparisons to highlight our intermediate contribution are performed using [START_REF] Tuzel | [END_REF] features set. This is due to the availability of more details concerning the trained classifier in [START_REF] Tuzel | [END_REF]]: the cascade of classifiers structure, the required training time, the negative rejection rate per cascade level, the average detection time per image, etc. so to make sense to the comparison, it has to be performed in the same conditions.

In [Yao 2008], only global evaluation results and their comparison with state of the art ones are provided.

Note that in the considered case of 8-feature set, the covariance matrix will contain 36 different values (due to the symmetry), and 44 Integral Images are computed to speed up the computing process (8 integral images for the representation of the sums of each feature independently and 36 for the representation of the sums of product for each pair of features).

Covariance Normalisation

The covariance features are robust towards constant illumination changes. To enhance the robustness against local linear variations of the illumination on a given subregion r with respect to its largest containing region R, a normalization step is performed on the covariance matrix.

First, both covariance matrices C r and C R are computed using integral representation. The values of covariance matrix C r are normalized with respect to the standard deviations of their corresponding features inside the containing region R as:

Ĉr = diag(C R ) -1 2 • C r • diag(C R ) -1 2 (4.9)
where Ĉr is the normalized covariance matrix of the subregion r, C r its initial covariance matrix, C R the covariance matrix of the containing region and diag(C R ) is a matrix equal to C R at the diagonal entries and zero value at all other entries.

Region Covariance Descriptors as Riemannian Manifold

A manifold is a topological space which is locally similar to an Euclidean space. Every point on the manifold has a neighborhood for which there exists a homeomorphism (one-to-one, onto, and continuous mapping in both directions) mapping the neighborhood to R m . For differentiable manifolds, it is possible to define the derivatives of the curves on the manifold. The derivatives at a point X on the manifold lie in a vector space T X , which is the tangent space at that point.

A Riemannian manifold M is a differentiable manifold in which each tangent space has an inner product <, > X∈M , which varies smoothly from point to point. The inner product induces a norm for the tangent vectors in the tangent space such as that v

2 X = < v, v > X .
The minimum length curve connecting two points X i and X j on the manifold is called the geodesic and the distance between the points d(X i , X j ) is given by the length of this curve (see figure 4.4).

Let v ∈ T X i and X i ∈ M. From X i , there exists a unique geodesic γ v (t) starting with the tangent vector y. The exponential map exp X i : T X i -→ M maps the vector y to the point reached by this geodesic, and the distance of the geodesic is given by d(X i , exp X i (v)) = v X i . The inverse mapping is defined by log X i : M -→ T X i . In the following paragraphs, points in the vector space are noted with small bold letters while points on the Riemannian manifold are noted with capital bold letters.

The covariance matrices are symmetric positive definite matrices. The set of d × d dimensional symmetric positive definite matrices, noted Sym + d can be formulated as a connected Riemannian manifold, and an invariant Riemannian metric on the tangent space of Sym + d is given by [Pennec 2006]

< y, z > X = trace(X -1 2 yX -1 zX -1 2 ) (4.10)
The two mapping functions are:

exp X (y) = X 1 2 exp(X -1 2 yX -1 2 )X 1 2 (4.11) log X (y) = X 1 2 log(X -1 2 yX -1 2 )X 1 2
(4.12)

The exp and log are the ordinary matrix exponential and logarithm operators. Not to be confused with exp X and log X which are manifold specific operators, and which are point dependent, X ∈ Sym + d . The tangent space of Sym + d is the space of s × d symmetric matrices, and both the manifold and the tangent spaces are m = d(d+1)/2 dimensional.

The ordinary matrix exponential and logarithm of a symmetric matrix can be computed easily using its eigenvalue decomposition. Let Σ = UDU T the eigenvalue decomposition of a symmetric matrix. The ordinary exp and log matrix operators are given by:

exp(Σ) = ∞ k=0 Σ k k! = U exp(D)U T (4.13) log(Σ) = ∞ k=1 (-1) k-1 k (Σ -I) k = U log(D)U T (4.14)
exp(D) and log(D) are obtained by applying respectively exponential and logarithm functions on the diagonal entries of the diagonal matrix D.

Note that the exponential operator is always defined whereas the logarithms only exists for symmetric matrices with positive eigenvalues, Sym + d , which is the case for the considered covariance matrices.

From the definition of the geodesic given above, the distance between two points on Sym + d is measured by substituting (4.12) into (4.10)

d 2 (X, Y) =< log X (Y), log X (Y) > X = trace(log 2 (X -1 2 YX -1 2 )) (4.15)
A minimal representation of the points (covariance matrices) in the tangent space are required for classification. Tuzel et al. [START_REF] Tuzel | [END_REF]] define an orthonormal coordinate system for the tangent space with the vector operation. The orthonormal coordinates of a tangent vector y in the tangent space at point X is given by the vector operator:

vec X (y) = vec I (X -1 2 yX -1 2 ) (4.16)
where I is the identity matrix, and the vector operator at identify is defined as

vec I (y) = y 1,1 √ 2y 1,2 √ 2y 1,2 ... y 2,2 √ 2y 2,3 ... y d,d T (4.17)
Given a set of points (covariance matrices), the tangent space used for the minimal representation defined above is tangent to the Riemannian manifold of covariance matrices at a specific point which is the mean of all the considered points (covariance matrices).

The mean of a set of point {X i } i=1...N on Riemannian manifold is defined as:

µ = arg min X∈M N i=1 d 2 (X i , X) (4.18)
where d 2 is the distance metric defined in (4.15).

The mean point can be computed iteratively using the following gradient descent procedure

µ t+1 = exp µ t 1 N N i=1 log µ t (Xi) (4.19)
A weighted mean can be computed similarly, by

µ t+1 = exp µ t 1 N i=1 w i N i=1 w i log µ t (Xi) (4.20)

LogitBoost Algorithm on Riemannian Manifolds

The classification process is performed using a cascade of classifiers which is trained using a LogitBoost algorithm on Riemannian Manifolds like in [START_REF] Tuzel | [END_REF]] to allow comparison of our cascade of classifiers with the one obtained in [START_REF] Tuzel | [END_REF]]. As it was mentioned in section 2.1.3.3, the main difference between Adaboost an Logitboost resides in the way the weak classifier errors are computed and thereby the way the best weak classifier is selected at each iteration. AdaBoost minimizes an exponential loss function while LogitBoost minimizes a logistic loss.

Standard LogitBoost Algorithm on Vector Spaces

As presented in [START_REF] Friedman | [END_REF]], let {(x i , y i )} i=1...N be the set of training samples, with y i ∈ {0, 1} and x i ∈ R n . The goal is to find a decision function F which divides the input space into the 2 classes.

In LogitBoost, this function is defined as a sum of weak classifiers, and the probability of a sample x being in class 1 (positive) is represented by p(x) = e F(x) e F(x) + e -F(x) (4.21)

F(x) = 1 2 N L l=1 f l (x). (4.22)
where f l (x) is the trained weak classifier at the l th iteration.

The LogitBoost algorithm iteratively learns the set of weak classifiers {f l } l=1...N L by minimizing the negative binomial log-likelihood of the training data:

- N i [y i log(p(x i )) + (1 -y i )log(1 -p(x i ))], (4.23)
through Newton iterations. At each iteration l, this is achieved by solving a weighted least-square regression problem:

N i=1 w i f l (x i ) -z i 2 (4.24) where z i = y i -p(x i ) p(x i )(1-p(x i ))
denotes the response values, and the sample weights are given by w i = p(x i )(1p(x i )).

LogitBoost Algorithm on Riemannian Manifolds

The trained cascade consists of a list of ordered strong classifiers. Each strong classifier contains a set of weak classifiers. A weak classifier is defined by a subregion of interest inside the detection region, the corresponding mean value of covariance descriptors of all positive samples in the same subregion, and the corresponding regression function.

To train the LogitBoost cascade of classifier using covariance descriptors, the standard LogitBoost algorithm is not usable as it is. In fact, covariance descriptors do not Let r i be one of these subwindows and Ĉj r i the corresponding normalized covariance descriptor on the sample j. For each subregion r i , the weighted mean µ i of all the normalized covariance descriptors Ĉj r i of the positive samples is computed using a gradient descent procedure (eq. 4.20).

Using this mean µ i , all Ĉj r i of all the samples (positives and negatives) are projected onto the tangent space using (eq. 4.16) obtaining vectors in Euclidean space. Using these vectors and the corresponding weights of all samples, a regression fucntion g i is computed.

The best weak classifier candidate, which minimizes negative binomial log-likelihood (eq. 4.23), is added to the current training classifier. The weights and the probabilities of all the samples are updated according to the new added weak classifier. The positive and the negative samples are sorted in a decreasing order using their probabilities. The current strong classifier is considered as fully trained if the difference between the probability of the (99.8%) th positive sample and the (35%) th negative sample is greater than 0.2. This means than the current trained cascade level has to reject at least 35% of the remaining negatives while it has to correctly detect at least 99.8% of the positive samples, and the value "0.2" represent a minimum margin to ensure between the 35% rejected negatives and the 99.8% positive.

It is used to make the separation more reliable.

In this case, the training of the current cascade level is achieved. The negative sam-ples are tested with the new cascade and all correctly classified samples (recognized as negatives) are removed from the training dataset. The next cascade level is trained using remaining negatives.

The training algorithm produces a set of K LogitBoost strong classifiers.

Yao et. al [Yao 2008] have introduced two important improvements that we use also.

First, classifiers are trained on a covariance descriptors with lower dimension. The initial set of 8-dimensional features is always kept and all the corresponding Integral Images (for individual features and the feature products) are computed, but a weak classifier is defined by a lower dimensional covariance matrix, computed with the more representative subset of features in the considered image subregion, and can vary from a region to another one.

Practically, each candidate subregion among the 200 randomly selected does not provide a unique candidate weak classifier with a 8 × 8 covariance matrix, but a set of candidate weak classifiers, each of them is computed using a 4 × 4 covariance matrix.

All the possible combinations of 4 features from the 8 initial ones are considered.

This provides 8 4

= 70 covariance matrices to consider for each candidate subregion. To avoid the high computational cost required to test all these 70 covariance matrices for each subregion, a substitute of the negative binomial log-likelihood for each 4 × 4 matrix for each subregion is calculated as follow:

The 8 2 = 28 possible 2×2 covariance matrices of the all possible pairs of features (from the 8 initial ones) are computed for each subregion.

The negative binomial log-likelihood for each 2 × 2 matrix is computed using (eq.

4.23).

Instead of computing the real negative binomial log-likelihood of a given 4 × 4 matrix computed with the four features {f 1 , f 2 , f 3 , f 4 }, this value is replaced by the sum of the negative binomial log-likelihood of all the 2 × 2 matrices which are computed with at least one of {f 1 , f 2 , f 3 , f 4 } features.

The subset of 4 features which provides the minimal sum is considered as the best (more representative) subset for the considered region.

The second improvement consists in the concatenation of the mean feature vector of each random subregion to its corresponding mapped vector of each sample before regression computing, improving performances by increasing the amount of information in the final vector.

Cascade of Classifiers Optimization

Main Issues

The structure of the cascade of classifiers is as important as its content. The cascade content (weak and strong classifier discriminative power) defines the detection performances in terms of false positive (false detection) and false negative (miss detection) rates, while the cascade structure (the number of strong classifiers in the cascade and the number of weak classifiers in each of them) greatly defines the processing time cost.

The discriminative power of a weak classifier depends directly on its selected subregion (the information contained in this subregion). The main issue in the proposed method is related to the fact that each weak classifier is selected as the best one from "n" randomly selected candidate weak classifiers (n=200 is taken in the approach) but not the best possible one at all. For the iteration "l", the best weak-classifier candidate from the "n" randomly selected ones can be insufficiently discriminant, and its selection will require to add more weak-classifiers to compensate its low discriminative power, and thereby lengthening the strong classifiers of each cascade level.

The optimal cascade of classifier in term of discriminative power and time processing can be obtained by selecting the best weak classifier from all the possible ones, at each iteration "l" of each level "k", because in this case, we ensure that the selected weak classifier is the most discriminative one, but in practice, this is impossible to perform (or at least, in a reasonable training time).

In fact, if we take the INRIA training dataset as example, it contains people training images with 64 × 128 pixel size in which, the person image is surrounded by a 16 pixel margin. Considering the minimum size of subregions as 10% of image width and height as proposed in [START_REF] Tuzel | [END_REF]], the number of all possible subregions with a minimum size of 6 × 12 pixels is 12.218.310, and if only person sub-image is considered (ignoring the margin), i.e. images of 32 × 96 with subregions of 3 × 9 minimum size, the number of all possible subregions is 1.820.940 subregions.

This represent a high number of subregions to process at each iteration "l" of each level "k", knowing that even if it is possible to compute the normalized covariances of all these subregions on all training images once before training (because the covariance matrices do not change), the mean covariance of each subregion will change from training iteration to another one, due to the used weighting process which evolves according to the previous iteration and to the considered samples at a given time. This requires to compute a weighted mean of more than 1.8 million covariance matrices (or more than 12 million covariances matrices in the case of the whole images, with margin), using the gradient descent procedure (eq. 4.20) which is an iterative procedure, requiring several iterations. In addition to this, the minimal representation of each covariance matrix in tangent space and the regression function will change due to the dependency of the projection function to the mean covariance matrix.

By using only 200 subregions at each iteration, the training time indicated in [START_REF] Tuzel | [END_REF] is approximatively 2 days. We can easily see that processing more than 1.8 million subregions (or 12 millions subregions depending on the inclusion of margin) is not reasonable.

It is then necessary to select a subset of subregions at each iteration, but without any a priori knowledge concerning the more informative regions, this selection is performed randomly, increasing the probability to have more weak classifier in each cascade level than in the optimal case.

The previously described people detector provides interesting detection performances, with a lower rate of miss-detections and false positives (see 7.1), but it has the disadvantage of being highly time consuming for the detection process and not applicable for real-time processing. In [START_REF] Tuzel | [END_REF]], the authors indicate that detection time on a 320

x 240 image is approximatively 3 seconds for a dense scan, with 3 pixel jumps vertically and horizontally.

The feature subset selection approach, proposed by Yao and Odobez in [Yao 2008] allows to work in a lower dimensional symmetric positive definite matrices, making eigenvalue decomposition faster and thereby allowing real time processing, but the processing time reduction is greatly due to the use of foreground features, extracted using background subtraction algorithm. Non moving image regions are quickly rejected. Unfortunately, the unavailability of high quality foreground information (as the ones used in [Yao 2008], shown in figure 4.3), and some time the unavailability of the background subtraction (in static image case) limits this improvement dedicated to video sequences only, with the availability of an efficient background subtraction.

Note that most computationally expensive operation during the training and the classification is eigenvalue decomposition. This decomposition is the basis of all operators in Sym + d . Eigenvalue decomposition of a symmetric d × d matrix requires O(d 3 ) arithmetic operations.

We focus in our work on another way to make the classification faster while maintaining high classification performance. At the end, the obtained approach improves also the training stage. This is performed by compensating the random selection of subregions issue with a pre-training step.

In the following paragraphs, we will illustrate the improvements of our contribution using tests on INRIA Person dataset [], due to the availability of more information concerning cascade structure and processing time in [START_REF] Tuzel | [END_REF]] on this dataset, but the reasoning and the proposed method can be generalized to all datasets.

Clustering Negative Data Before Training

Using a large number of samples slows down the training process. Of course, the larger the training dataset is, the more efficient the classifier cascade is. But most of the time, especially for first cascade levels, a large number of negative samples contains very similar information. This is due to the fact that a new level is trained using false positives of previous levels, and in this case, these false positives are generally resulting from successive small shifts of testing window on the image, providing very similar content.

The trained cascade of classifiers on INRIA Person dataset by [START_REF] Tuzel | [END_REF] contains 30 strong classifiers (levels) (see figure 4.6 (a)).

A first idea to speed up the training process is to use a smaller subset of randomly selected negative samples to train a given cascade level. We can suppose that a randomly selected subset can be statistically representative of all remaining negatives, and a trained cascade level on this subset will reject a proportional number of negatives from the whole training dataset than the one rejected from the selected subset.

We have tested this approach and we have observed that random selection effectively speeds up slightly training and provides 4 less cascade levels in comparison than [START_REF] Tuzel | [END_REF]] but with longer classifiers (figure 4.6 (b)), slowing down the detection in comparison to the previous mentioned approaches. In fact, one cascade level consists of a set of weak classifiers. The response of one classifier is obtained after computing the output values of all its weak classifiers. It means than a long classifier containing a large number of weak classifiers takes more time to return a decision, so a cascade of long classifiers is very slow for detection.

Decreasing the training time by increasing the detection one is not acceptable as long as the detection process is the final aim, but the idea of training each cascade level on a subset of negative samples to speed up the training is still an interesting way to explore.

It is just necessary to find the best way to select the negative sample subsets to decrease the number of necessary weak classifiers per cascade level and to take into account the random nature of the subregion selection.

We have constructed our reasoning starting from this observation: the number of weak classifiers per cascade level depends mainly on the diversity of negative samples used for the training. The characterisation of positive samples and their separation from the negative ones require as many subregions of interest as the samples are diverse.

To illustrate the relationship between negative sample diversity and classifier cascade structure, let us use a simple example which can be generalized to understand the concept.

Suppose that we have to separate a person image from three non-person images: a For this example, there are two methods to train the classifier. The first cascade is trained with the three negative images at the same time, using appropriate parameters.

The second cascade is trained with one negative image at a time in the mentioned order.

It also specializes each cascade level faster and reduces the effect of random subregion selection method for best weak classifier extraction in this case.

We have tested two clustering methods to achieve the negative sample regrouping.

The first one is performed directly in the Riemannian manifold of covariance matrices while the second one is performed in the Tangent vector space.

Hierarchical Clustering in Riemannian Manifold of Covariance Matrices

The first clustering method to group similar negative samples in term of covariance information is performed directly using the distance between covariance matrices provided by eq. ( 4.15).

To compare two negative images in term of covariance information, and due to the unavailability of a priori knowledge concerning the most important subregions in these images, we perform a pyramidal division of the images in 4 levels where level 0 is the whole image (see figure 4.9). The distances between all pairs of negative images is performed in each pyramid level. For a given pyramid level, the distance between two negative images I 1 and I 2 is given by the sum of squared distances between all pairs of subregions at the same location from the two images. The distance between two image subregions is provided by the distance between their corresponding covariance matrices using eq. (4.15) (see figure 4.10).

A triangular matrix containing distances between all pairs of negative images is then computed for each pyramid level. From a given matrix of distances, it is easy to extract iteratively the largest cluster which contains the most similar n% of images. We take the value of 35% of remaining negative samples to be similar and comparable to [START_REF] Tuzel | [END_REF]] and [Yao 2008]. The hierarchical clustering is illustrated in figure 4.11.

From each pyramid level, we can extract the largest clusters of the most similar remaining negative samples. This provides 4 candidate largest clusters, the largest one from each pyramid level. We select the best cluster as the one which provides the lowest

The comparison between the structure of our cascade of classifiers and the one from [START_REF] Tuzel | [END_REF]], shown in figure 4.6, shows the effectiveness of our method. Our cascade of classifiers is shorter than [START_REF] Tuzel | [END_REF]] one and most levels in our cascade of classifiers are shorter than their corresponding levels in [START_REF] Tuzel | [END_REF]] one.

A more detailed comparison in terms of detection performances and processing time and a global evaluation which validates the effectiveness of the proposed people detector are provided in the chapter 7.

The second tested clustering approach consists in projecting all the remaining negative samples to the tangent space on their mean points. In this method, we consider the covariance matrix of the whole image as its representation (equivalent to the level 0 of the used pyramid in the previous method). The mean of all negative samples is computed and used to project all covariance descriptors to the Euclidean space. Finally in Euclidean space, the clustering is performed using adaptive bandwidth mean shift filtering [START_REF] Comaniciu | [END_REF]]. (See Figure . 4.12)

X 2 X 4 X 1 X 3 μ M T μ 0 x 1 x 3 x 2 x 4
the largest cluster log μ log μ The obtained cascade of classifiers, trained using clustered negative samples by this second method is substantially similar to the one we have obtained with the original method (without clustering). This is due to the negative sample sparsity.

Unlike for positive samples, the mean of negative samples does not have a physical sense (see figure 4.13, second row), or at least, it has a sense for the largest clusters of the first training levels which share some similar content. Positive samples share a similar shapes, even if variations can occur in these shapes (see figure 4.13, first row).

The mean covariance matrix of a given subregion can be considered as the covariance matrix of the mean shape of this subregion on all positive samples.

Generally, remaining negative samples after few iteration do not share similar shapes in the same image locations, so the resulting mean does not represent a physical information from the considered negative samples. Therefore, projecting negative samples using insignificant mean loses the topological distribution of these negative samples in the tangent space. This is illustrated in figure 4.14

For this reason, our final people detector is trained using the hierarchical clustering in Riemannian manifold of covariance matrices.

... ...

Conclusion

We have proposed an approach to optimize people detection using covariance descriptors. This approach consists in clustering negative data before the training step to obtain better classifier structure. The resulting detector is faster than original one and was trained in shorter time (detailed evaluations are provided in chapter 7).

Clustering negative data before training allows to reduce the effect of random subregions selection for best weak classifier training. As explained above, the exhaustive testing of all possible weak classifiers at each iteration is impossible in a reasonable processing time, and a targeted subregion selection is not possible due to the unavailability of any a priori information concerning the most interesting regions.

Of course, the unavailability of any a priori information concerning the most interesting regions can be considered as an issue for the negative image comparison and clustering. To compare two negative images and compute a distance between them, it is necessary to focus on the region of interest on each of them, which is not possible

X 2 X 4 X 1 X 3 M 0 1 x 1 x 3 x 2 x 4
the largest cluster X 5

x 5 first iteration

μ 1 log μ 1 log μ 1 T μ 1 X 2 X 4 M 0 2 x 2 x 4 the largest cluster X 5 x 5 second iteration μ 2 log μ 2 log μ 2 T μ 2 Figure 4
.14: Illustration of negative samples sparsity in a 2D Manifold: In the first iteration, many negative samples are available. The largest cluster in the Riemanian manifold M attracts the mean near to it, and the projected points keep a large amount of topological distribution. Thereby, the first cluster has a sense in term of covariance matrices similarity. In the second iteration, once the largest cluster is removed, the negative samples are sparse, and the computed mean provides a projection with a changed topological distribution. The samples X 2 and X 4 which are clearly different in M are grouped in the same cluster (x 2 and x 4 ).

for the same reason. To deal with this, a clustering stage is performed using several subdivisions of images. It is not the optimal way but it reduces significantly the random nature of subregion selection during the training.

We have shown that this clustering is better when it is performed directly in Riemannian manifold of covariance matrices, due to the fact that a distance between two covariance matrices is more precisely represented by the length of the geodesic which links them, instead of the euclidean distance between their projected points in the tangent space. This euclidean distance may be altered by a non-significant mean computed on sparse negative data.

The basic method [START_REF] Tuzel | [END_REF]] on which we have built our reasoning and improvements are not the last and the most efficient ones in the state of the art [START_REF] Dollar | The Fastest Pedestrian Detector in the West[END_REF], Walk 2010[START_REF] Felzenszwalb | [END_REF]) when this thesis has been written, but when we have performed this work, [START_REF] Tuzel | [END_REF]] was one of the most efficient approach in the state of the art , and our interest for this method is also justified by the nature of the trained classifier and the high processing time required. It allows us to highlight the contribution of our proposed method and to discuss its generalisation to other approaches based on cascade of classifier training.

In fact, clustering negative data before training is an efficient way to optimize the trained cascade of classifiers, for which the training cannot explore all the space of possible weak classifiers to select the best one at each iteration. According to the used descriptor (Haar-like, LPB, etc.), to its similarity measure, and to the way the candidate weak classifiers are preselected for the selection of the best one, it is possible to speed up the training phase using this clustering method in the appropriate descriptor space (generally in Euclidean space for most of descriptors).

ROBUST OBJECT TRACKING USING PARTICLE FILTERING

This chapter describes the proposed object tracking algorithm in mono-camera context. This tracking is performed with static and calibrated cameras. The tracked objects are modelled with a set of SIFT features, selected in a specific way. The object tracking is performed in three separate levels: first the SIFT features are tracked independently using a particle filter. Then, object localisation and temporal links are built using a data association framework based on the localisation and the reliability of the tracked SIFT features. Finally, occluded objects are managed with two methods, the first one, based on real world information and dominant color descriptor, is faster and is sufficient to deal with occlusion situations in most of the cases, but for ambiguous or complex cases, the re-identification method, presented in chapter 6 is used.

Tracked Target Initialization

In the proposed tracking algorithm, moving object detection in the scene is performed using a state of the art background subtraction algorithms based on adaptive background mixture models [START_REF] Stauffer | [END_REF]]. In this algorithm, moving image regions are extracted as group of foreground pixels. These groups of pixels are more or less connected, depending on the contrast of the corresponding object and some parameters of the background subtraction algorithms (several thresholds, number of considered Gaussian, etc.). A clustering step is performed to regroup the foreground pixels in blobs representing one or several grouped objects in the scene, and to filter too small groups of pixels which are probably noise (see figure 5.1). Finally, these blobs are delimited by minimal bounding boxes (the smallest ones which surround blobs) which will be used to define the localisation of the objects in image.

Classification by Real Dimension Estimation

As mentioned in the introduction chapter, our work is performed using calibrated cameras. The camera calibration is performed using a proprietary software developed by Digital Barriers France (previously Keeneo). The calibration is based on vanishing lines approach (see figure 5.2). It provides both extrinsic and intrinsic calibration matrices in an Y-top coordinate system. These matrices are used for real world ⇄ image projection. Note that this tool is easy to use and requires few operations from video operators (defining 4 straight lines on the ground floor and one vertical line, in addition to an estimation of the height of any object in the scene). This does not affect the "easy to use" constraint we try to respect.

A first classification of the moving objects is performed using an estimation of their real dimensions (3D width "W" and 3D height "H") provided by camera calibration matrix (see figure 5.3). All these points belong to a straight line (L) which passes through the optical center of the camera (see figure 5.4). To obtain a unique corresponding point (X, Y, Z) in the real world for a 2D point (x, y) in the image, an additional constraint is required. Generally, the ground plane is considered as the plane with coordinate Y = 0, then all 2D points on the ground image can be easily projected to real world and their real coordinates in the camera coordinate system can be computed.

To estimate the real width W of a given object in the scene, the two bottom corners of its bounding box p bl (bottom-left point) and p br (bottom-right point) are projected with Y = 0 assuming that the object is on the ground. Once the two real coordinates of the object borders P bl and P br are computed, the distance between them is considered as the object width W (see figure 5.5).

For the real height H estimation, a double constraint is considered. The bottomcenter point of the bounding box p bc is projected to real world coordinates P bc and the plane (π) containing the two points P bl and P br and which is perpendicular to the ground plane (G) is computed and considered as the wanted point P tc (real top-center point of the object). The distance between P bc and P tc is considered as the object height (see figure 5.6).

The depth of objects is ignored due to the impossibility to estimate it from a single

People Detection

The previously detailed method is fast to perform and provides, most of the time, good detected and classified objects, but can be imprecise in some situations. For example, if no shadow removal is performed, the bounding box representing the object is not precise and includes shadow pixels, providing erroneous projections and real dimensions which did not fit with known classes or fit with incorrect ones. Another example concerns grouped objects. If a group of individuals are too close to each others, the extracted bounding box will include all the people and its projected dimensions will not fit with any class or will fit with wrong one (see figure 5.8).

To deal with these possible situations for people tracking, while keeping the interesting aspect of fast detection and classification using background subtraction and camera calibration information, we use our people detector following two rules:

The people detector is applied on the moving regions on which the first classification is ambiguous (object dimensions which do not fit with any class dimensions, large objects which can be a group of people, etc.). This allows to reduce the area of searching to a subregion instead of the whole image.

The calibration information is used to limit the searching scales of the people detector. The admitted interval of people height (the human model) is used to project realistic searching window size on the moving region, avoiding useless scale detection.

These two rules allow to perform the detection in negligeable time, and thereby, did not slow all the tracking process which is still performed in real time.

Note that the real world dimension estimation and the background subtraction information are also used during the tracking itself. One for fast occlusion management and the other for the particle filtering process. This will be detailed in the following sections.

Our tracking algorithm requires the set of all detected moving objects of interest in every frame of the video sequence, in the form of bounding boxes and their corresponding extracted foreground pixels.

Object modeling

To be tracked, an object of interest has to be modeled using its discriminant features.

Among the different possible representations for an object (colors, shapes, etc.), we choose to model each tracked object by a set of features points. This choice is motivated by the independence of points between them, which allows to deal with partial occlusion and object deformations.

Our tracking algorithm is built on SIFT features for object modeling, but any other kind of point of interest around which a descriptor can be computed (SURF [START_REF] Bay | [END_REF]],

HOG [START_REF] Dalal | [END_REF]], etc.) and can be used with slightly lesser point tracking performances. SIFT features are known to be among the most robust local descriptors. Even if some authors have claimed that their proposed new descriptors, like SURF [START_REF] Bay | [END_REF] are faster than SIFT and provide comparable performances, these affirmations were probably correct since years ago but actually, using recent computers (more processing power) and optimized implementations, we have observed than no significant difference in processing time exists between SIFT and the other similar local descriptors.

SIFT features

Scale-Invariant Feature Transform (SIFT) is an algorithm in computer vision to detect and describe local features in images. It was first proposed by Lowe [START_REF] Lowe | [END_REF]]. SIFT features correspond to a set of points of interest called SIFT keypoints which are detected in a specific way, and around which a local descriptor, called SIFT descriptor, is computed and assigned to them.

SIFT Points

The SIFT point detection is performed in four successive steps. Each step provides to these points some robustness against a kind of issues or variations.

Scale-space extrema detection:

A pyramid of multiple octaves containing Difference of Gaussian images is built (see figure 5.9 (a)). The image is convolved

with Gaussian filters at different scales kσ , and then the difference of successive Gaussian-blurred (DoG) (D (x, y, σ)) images are taken. DoG images are given by: Keypoint accurate localisation: The selected extrema are (re)localised more accurately by interpolating their initial positions using the quadratic Taylor expansion of the Difference-of-Gaussian scale-space function, D (x, y, σ) with the candidate keypoint as the origin. This Taylor expansion is given by:

D (x, y, σ i ) = L (x, y, σ i+1 ) -L (x, y, σ i ) (5.
D(x) = D + ∂D T ∂x x + 1 2 x T ∂ 2 D ∂x 2 x (5.3)
where D and its derivatives are evaluated at the candidate keypoint and x = (x, y, σ) is the offset from this point. If the offset is larger than 0.5 in any dimension, then that is an indication that the extremum lies closer to another candidate keypoint. In this case, the candidate keypoint is changed and the interpolation performed instead about that point. Otherwise the offset is added to its candidate keypoint to get the interpolated estimate for the location of the extremum.

Low-contrast keypoint elimination:

The value of the second-order Taylor expansion (eq. 5.

3) is computed at the new localisation of each keypoint. If this value is less that 0.03 (this contrast threshold has been fixed in [START_REF] Lowe | [END_REF]] as the optimal one), it is considered as a low contrast point and it is eliminated.

Edge response elimination:

The keypoints that have poorly determined locations (during the keypoint accurate localisation step) but have high edge responses are eliminated. This is performed using the second-order Hessian matrix H defined as: ) yields the product. The ratio R = Tr (H) 2 / Det (H) can be shown to be equal to (r + 1) 2 /r, which depends only on the ratio of the eigenvalues rather than their individual values. R is minimum when the eigenvalues are equal to each other. Therefore the higher the absolute difference between the two eigenvalues, which is equivalent to a higher absolute difference between the two principal curvatures of D, the higher the value of R. It follows that, for some threshold eigenvalue ratios r th , if R for a candidate keypoint is larger than (r th + 1) 2 /r th , that keypoint is poorly localised and hence eliminated.

H = D xx
In [START_REF] Lowe | [END_REF]], r th = 10 is taken as the optimal value.

The remaining candidate points are the final SIFT points. These steps provide invariance to image location and scale to the detected SIFT points. They also ensure a sufficient contrast and avoid curvature to them.

SIFT Descriptors

Once SIFT points are detected, the next step consists in computing SIFT descriptors around each of them. This is performed in two steps:

Main orientation assignment: To achieve invariance of SIFT descriptor to 2D rotations (in image), each keypoint is assigned with one or more orientations based on local image gradient directions. For a SIFT point (x, y, σ), the nearest Gaussian image L (x, y, σ ′ ) to the DoG image on which this point is detected is taken to construct an orientation histogram of 36 bins (each bin covering 10 degrees). Each pixel in a neighboring window of the considered SIFT point will contribute to this histogram by its weighted gradient magnitude. The weighting is provided using a circular Gaussian windows centred on the SIFT point. The gradient magnitude and orientation of each neighboring pixel are computed as: m (x, y) = (L (x + 1, y) -L (x -1, y)) 2 + (L (x, y + 1) -L (x, y -1)) 2 (5.5) θ (x, y) = atan2 (L (x, y + 1) -L (x, y -1) , L (x + 1, y) -L (x -1, y))

(5.6)

Once the histogram of 36 bins is computed, the orientation corresponding to the highest peak is taken as the main orientation and assigned to the SIFT point. If any other local peak value is greater than or equal to 80% of the highest peak value, the corresponding orientation is also taken as another main orientation. In this case, the SIFT point is duplicated and each of these SIFT points will be assigned with one main orientation. There will be as many SIFT points at the same location and same scale as the number of main orientations.

Descriptor computing: For each SIFT point, the image closest in scale to its own scale is taken for computing the descriptor. On this image, a square region of 16 × 16 around the SIFT point is considered. This window is divided in 4 × 4 equal subregions (of 4 × 4 pixels). An orientation histogram of 8 bins is created from each subregion. These histograms are computed from magnitude and orientation values of the pixels in these subregions. The gradient orientations are reported to the main orientation computed before, to provide the invariance of the descriptor to rotations. The magnitudes are further weighted by a Gaussian function (see figure 5.10). The final SIFT descriptor is a 128 dimensional vector obtained by concatenating all these histograms (4×4×8 = 128). This vector is then normalized to unit length in order to enhance invariance to affine changes in illumination. To reduce the effects of non-linear illumination a threshold of 0.2 is applied and the vector is again normalized. SIFT descriptor is highly distinctive and partially invariant to the remaining variations such as illumination and 3D viewpoint (affine transformation).

SIFT Feature Detection and Selection For Object tracking

The detailed SIFT descriptor can describe any 16 × 16 square region of image, independently of the existence of a point of interest on its center or not, but computing it around a point of interest (SIFT point) is more efficient for matching purpose.

We want to exploit the effectiveness and the discriminative power of SIFT descriptors in our object tracking algorithm, but due to the related constraints to the addressed context (video surveillance) like small objects, low resolution and noisy images, etc.

the SIFT point detection method described below does not always ensure a sufficient number of SIFT points to describe the whole object of interest, and their localisation is not necessary uniformly distributed (see figure 5.11 (b)). This detection can also provide too many points, and tracking all these points increase the required processing time.

For these reasons, we use a modified method for SIFT point selection for the object representation. First, a more permissive SIFT point detection is performed, by avoiding low contrast and edge response filtering. More SIFT points are then detected (see figure 5.11 (c)). Once a large number of points of interest is detected inside the bounding box which delimits the object of interest, only foreground points of interest are kept. This is performed using the object mask provided by the background subtraction algorithm (see figure 5.11 (b)).

Finally, object image is divided into subregions, in which a constant number n of SIFT features is kept and the other points are rejected (see figure 5.12 (c)). If a given subregion contains more than n points, the most reliable of them, in terms of contrast and curvature values are selected. In fact, even if these two values are not used during SIFT point detection for candidate point filtering, we use them to sort detected points in each subregion and to select the most reliable of them.

The object image subdivision can be performed in several subregion sizes. We have tried many configurations and have seen that for people, a grid of 4 × 6 subregions provides the best compromise between tracking performance and processing time. The same remark can be done for the number of selected SIFT points in each subregion.

When the subregions are relatively small (like the considered 4 × 6 subregion subdivision for video surveillance images), using one point per subregion provides slightly less good results than using two points per subregions, but the processing time is multiplied by 2 for a non significative improvement in performances, especially after performing our data association framework, which will be detailed in next sections, and which compensates for the slightly decreased performances provided by only one point per subregion.

Finally, the object is represented with a sufficient number of SIFT points, uniformly distributed on the image (see figure 5.12 (d)), allowing better partial occlusions management as we will explain it in next sections. For people images, the number of representative points generally varies from 15 to 20 points.

Note that even a subset of selected points are less reliable due to the non-filtering of low contrast and edge points, they are still better than randomly selected points in SIFT points-free regions. The aim is to describe uniformly distributed local regions on the person. The low reliability level of these points is compensated by the data association method described few sections below.

SIFT Feature Tracking By Particle Filtering

After object of interest detection and modeling using SIFT features, the first level of tracking is performed on the SIFT features only. At this level, the tracking algorithm does not care about to which object belongs a given SIFT point. It tracks all SIFT features as independent entities. These points are tracked over time using a specific particle filter.

As a reminder about Bayesian filters, detailed in sec. 2.2.2.2, let x t denote the state of the system at the current time t, and y t = (y 1 , ..., y t ) the observations up to time t. The filtering problem involves the estimation of the state vector at time t, given all the measurements (observations) up to and including time t. In a Bayesian setting, this problem can be formalized as the computation of the distribution p(x t |y 1:t ), which can be done recursively in two steps. prediction

In the prediction step, p(x t |y 1:t-1 ) is computed from the filtering distribution p(x t-1 |y 1:t-1 ) at time t -1: p(x t |y 1:t-1 ) = p(x t |x t-1 )p(x t-1 |y 1:t-1 )dx t-1 (5.7)

where p(x t-1 |y 1:t-1 ) is assumed to be known due to recursion. The distribution p(x t |y 1:t-1 ) can be thought of as a prior over x t before receiving the most recent measurement y t . update The previous prior is updated with the new measurement y t using the Bayes' rule to obtain the posterior over x t : p(x t |y 1:t ) ∝ p(y t |x t )p(x t |y 1:t-1 )

(5.8)

In general, the computations in the prediction and update steps (eq. 5.7 and 5.8) cannot be carried out analytically, hence the need for approximate methods such as Monte Carlo sampling, also called Particle Filtering methods. Here, the space of hypothesis is explored using a set of particles, which are projected in the prediction step, and sampled by their importance in the update step, allowing to estimate the new system state.

The recursion requires the specification of a dynamic model describing the state evolution p(x t |x t-1 ), and a model giving the likelihood of any state in the light of the current observation p(y t |x t ). The recursion is initialized with some initial distribution p(x 0 ).

In our approach, the state of a SIFT feature x = {x, y, u, v, h, n} consists of:

The SIFT feature position (x, y).

The velocity component (u, v).

The SIFT descriptor h associated to the SIFT point.

The measurement error estimation n following a normalized distribution.

We have tested the use of acceleration component as an additional state information, but no significant improvement has been observed. This is due to the constant or slow variations of objects velocity.

In particle filtering, each hypothesis about the new state is represented by a particle which has its own state with the same structure as the SIFT feature one. Each SIFT feature is then tracked using a constant number of particles.

The prediction step consists in applying the dynamic model of each SIFT feature to all its associated particles to compute the new estimated location of each particle (see figure 5.13). This is done for each particle by:

(x, y) t = (x, y) t-1 + (u, v) t-1 .∆ t + n t-1 (x, y)
(5.9) The update step consists in estimating the new location of the tracked feature using the predicted state of all particles and some measurements provided by the current image at time t. This step is performed in three sub-steps: particle weighting, particle sampling-resampling and new state estimation.

Hybrid Particles Weighting

Generally, when a given descriptor is tracked using a particle filter, the particle weighting is performed using a similarity measure between the tracked descriptor and each particle descriptor.

In our case, we have followed this method in a first time. For a given tracked SIFT point, a SIFT descriptor is computed around each associated particle. The weight of a given particle is then provided by:

W p = 1 σ √ 2π e - d(H f ,Hp) 2 2σ 2
(5.10) where d(H f , H p ) denotes the similarity between the SIFT descriptor of the tracked point (H f ) and SIFT descriptor the considered particle descriptor (H p ). We use an Euclidean distance after having tested some other distances without getting significant improvements.

σ denotes a standard deviation computed on the tracked feature similarity variations up to time t -1.

The tracking of SIFT points using this weighting method for particle filtering performs well in most of cases, but we have observed a critical issue for some tracked SIFT points in some situations. Due to the small size of object images provided in video The intuitive idea can be to use the background subtraction results to directly avoid particles which are on the background by assigning a null weight to them. This idea is risky because the quality and the reliability of background subtraction algorithm may greatly vary according to the scene context (contrast, illumination, etc.) and to the background subtraction algorithm itself (the used method, parameters, etc.). Thereby, using a binary weighting, i.e. 1 for foreground particle and 0 for background one, is not a satisfactory solution. We have observed that using this binary weighting causes -a --b --c --d - The particle weighting used in our tracking algorithm is then given by:

W p = c σ √ 2π e - d(H f ,Hp) 2 2σ 2
(5.11)

where c ∈ {c 0 , 1} is the foreground probability weight.

If the corresponding pixel to the considered particle is an indication of a foreground one by the background subtraction result, c = 1. Otherwise, c = c 0 .

c 0 is computed for each detected object at each frame as:

c 0 = cos(ρ π 2 )
(5.12)

and:

ρ = min 1, density density 0 (5.13)
where density is the ratio between the number of foreground pixels inside the object bounding box and the area of this bounding box, and density 0 is the approximate real density of the foreground pixels of an object inside its bounding box. For a given type of objects, density is a variable value, provided at each frame by the background subtraction results while density 0 is constant and estimated off-line using some ground truth on the considered object type. For example, for a walking person, the average density 0 is approximately 0.5 (it means that in the minimal bounding box of a walking person image, approximately 50% of the pixels inside the bounding box belongs to the person).

Let us consider the possible situations of several background subtraction qualities at both localisation of a given particle: in the foreground and in the background. In all these cases, if the particle is on a foreground pixel, c = 1.0. Here, the similarity measure between the SIFT descriptor of the tracked SIFT point and the one of the particle will decid the association. The ambiguity occurs for background pixels.

For a low and medium foreground pixel detection (figure 5.16 (a,b)), the value of ρ increases with the increase of the foreground extraction quality, and thereby, particles on the background obtain decreased weight c 0 but are not discarded, to avoid the loss of useful information.

For a good foreground pixel detection (figure 5.16 (c)), the value of ρ is near 1.0 and thereby, the value of c 0 is near 0. It means that the more the background subtraction is precise, the less the particles on background are important (lower weight).

For an excessive foreground pixels detection (figure 5.16 (d)), the value of ρ is equal to 1.0 and thereby, the value of c 0 is near 0. It means that an estimated background pixel in this case has a high probability to be a real background pixel.

Real background pixels which are detected as foreground ones in this case will have a value of c = 1.0, but does not alter the particle weighting as long as this weighting is not worse than a standard weighting method which is only based on similarity measure (eq. 5.10), but equal to it. Note that we have chosen to take a cos function to encode the transition from the lower background subtraction quality to higher one smoothly (eq. 5.12). We have tested a linear function to perform this task and we have obtained satisfactory results, but the transition is encoded roughly and the precision is lower than the cos function one.

This test consists in annotating manually the localisation of a given point on the head of the walking person on each frame of a small video sequence, providing a ground truth of the real localisation of the same point in each frame. We have tested both linear and cos function by automatically tracking the same point along this sequence, by varying the background subtraction parameters, and by computing localisation error during each tracking. This error is taken as the mean of distances between the estimated position of the point by the tracking and its real position (ground truth one). We have observed that this error is lower with the cos function (see

Particles Sampling and Resampling

After weighting, all particles are sampled using a "Sampling Importance Re-sampling" (SIR) method [START_REF] Tanner | [END_REF], Smith 1992] to keep the most important ones, drop the less important and replace them by new particles generated from the kept ones. The sampling step allows the tracker to keep the more reliable particles and the re-sampling step avoids information degeneration, as explained in sec. 2.2.2.2 (Particle filters paragraph). Each feature keeps a constant number of particles over time, which makes the processing time easier to control. Finally, all particles are re-weighted with the same normalized weight.

New State Estimation

The estimation of the new location of the tracked feature is obtained as the centroid of all its particles (see figure 5.17). The descriptor of the tracked feature is computed around the new location.

A variation measure between the previous descriptor and the new one is computed for each tracked point. This variation measure is used for the feature variation learning in a Gaussian model in order to decide if a new state is acceptable or not (if it fits the Gaussian model variations according to the standard deviation). If the variation is too important the SIFT point is discarded and replaced by a new detected point in the same subregion as the discarded one (see 5.2.2 for subregion division). Otherwise, the point is kept and a reliability measure γ is computed using the variation measure as: γ t = 1 -variation measure at time t max accepted variation measure in the Gaussian model (5.14) This reliability measure γ is used for link weighting during the data association step, as described in the next section. After the update step, the velocity (u, v) t and the measurement error estimation n t (x, y) components of each SIFT feature are also updated. For each SIFT feature, a linear regression function is computed on the p last localisations of the considered SIFT features. The regression line direction provides the estimated direction of motion for next time t + 1 (velocity vector direction), the mean of displacement magnitude between these successive p positions provides the motion velocity magnitude, and the variance on these displacements provides the measurement estimation error for the next time t + 1. In our experiments, the optimal value for p is 10 (see figure 5.18(c)). We have observed that for lower values, the regression line direction, the mean and the variance of displacements vary too fast due to "non-smooth" SIFT point trajectories (see figure 5.18(a)). In the other hand, for large values of p, the regression line direction takes more time to follow the changes in motion directions (see figure 5.18(b)) causing tracking failure du to poor prediction process.

Note that the value p = 10 is extracted experimentally, and is not the optimal value for all situations. The optimal value depends strongly on the movement, but statistically, a medium value between 7 and 10 provides the best tracking results in most of the cases. The 20 last positions are used for regression computing. The resulting velocity vector does not follow movement direction variations quickly. at time t, the estimated movement direction (green vector) indicated that the movement direction is to the top-left side while the real movement direction is to the left (see the 8 last positions). (c) The 10 last positions are used for regression computing. In this case, movement direction changes are managed better than in the two previous cases

t-1 t-2 t ¡ t ¢£ t ¢¤ t ¥£ v v v v t t ¢ t ¥ t ¡ t ¢£ t ¢¤ t ¥£ v t v v v (a) (b) (c) t t ¢ t ¥ t ¡ t ¢£ t ¢¤ t ¥£ v v t

Data Association

At the end of previously detailed step, all the SIFT points of all the considered objects at time "t-1" are localised in the current video frame. Data association step consists in linking previously tracked objects at time "t-1", noted "to(t-1)", with the new detected objects at the current frame "t" , noted "do(t)", while dealing with complex situations like partial or full occlusions.

From the previous frame (at time "t-1") to the current one (at time "t"), only five general cases can occur (see figure 5.19):

In the first case, called "1 to 1" correspondence (figure 5.19 (a)), a unique detected object "do(t)" corresponds to only one previously tracked object "to(t-1)". This is the simplest case. Here the algorithm updates the "to(t-1)" localisation at time "t" by linking it directly to "do(t)".

In the second case, called "N to 1" correspondence (figure 5.19 (b)), a unique detected object do(t) corresponds to a set of Q tracked objects {to k (t -1)} k:1...Q . This situation occurs when the detection at time "t" did not correctly split detected moving objects, typically during partial occlusions or high object proximity. Here the algorithm splits the bounding box of "do(t)" into Q smaller bounding boxes using the spatial distribution of the SIFT points before the merge. This distribution is given by the ratios between feature locations and the borders of the bounding box when the objects were separated.

In the third case, called "1 to N" correspondence (figure 5.19 (c)), a set of R detected objects {do l (t)} l:1...R corresponds to a unique tracked object "to(t-1)". This situation occurs during the dispersion of a group of objects or at the end of occlusion between objects of interest. Here two situations can be distinguished:

-"to i (t-1)" can be the result of a previous merge (occlusion) of tracked objects at time "t-p" as described in the previous case (b). In this case, the tracking is resumed using the occlusion management approach described in next section (5.4.2).

-"to i (t -1)" has always been tracked as a group of objects since its appearance in the scene, new tracked objects {to l (t)} l:1...R are initialized by each {do l (t)} l:1...R after the split.

In the fourth case, called "1 to 0" correspondence (figure 5.19 (d)), no detected object "do(t)" corresponds to the tracked object "to(t-1)". This occurs in full occlusion situations or when the tracked object "to(t-1)" leaves the scene at time "t".

The two cases are managed differently:

-If the tracked object is close enough to a scene exit (image borders or known exit zones in the image) at time "t-1", the tracking algorithm considers that this object has left the scene and stops its tracking definitely.

-In the other case, the tracked object "to(t-1)" is considered as in occlusion situation. Here, the tracking algorithm stores the lost object for tracking recovery if it re-appears later. This process if performed using the occlusion management approach described in next section (5.4.2).

In the last case, called "0 to 1" correspondence (figure 5.19 (e)), a detected object "do(t)" does not correspond to any previously tracked object "to i (t -1)". This situation occures either when a new object apprears in the scene for the first time or when an previously tracked object has been occluded few time before and reappears. In the case of new object appearence, a new tracked object is initialisez.

In the case of occluded object reappearance, the occlusion managment resume its tracking as described in the occlusion management section (sec. 5.4.2).

Case Identification

The first step of our data association method consists in identifying in which case each tracked object "to(t-1)" is at time "t" before managing the case.

To do this, an M × N link score matrix, denoted S, is constructed. M is the number of tracked objects "to(t-1)" and N is the number of detected objects "do(t)".

Each element s(i, j) of S is calculated as the weighted proportion of SIFT points from the i th tracked object "to(t-1)" that geometrically belongs to the j th detected object "do(t)" using the following formula:

s(i, j) = 1 p q=1 γ q p k=1 γ k (i, j) (5.15)
where p is the number of SIFT points of the tracked object "to i (t -1)" which belongs geometrically to the detected object "do j (t)", γ k (i, j) ∈ [0, 1] is the reliability measure of the k th tracked SIFT point, computed at the end of particle filtering update step (eq.

5.14). The contribution of each SIFT point in the link score value is directly proportional to its reliability, providing a better temporal linking process.

Putting these link score values in a matrix form eases and speeds up the decision process. We use the Hungarian algorithm [START_REF] Kuhn | [END_REF]] to select the best links, i.e. we do not take the absolute best links one by one, but we select the links which provides the best global score. The case identification is performed using the link scores.

"1 to 1" correspondence is represented by a high link score (≈ 1.0) between a tracked object and a detected one.

"N to 1" correspondence is represented by a set of N close link scores (≈ 1/N) between a N tracked object and one detected one.

"1 to N" correspondence is represented by a set of N close link scores (≈ 1/N) between a tracked object and a set of N detected one.

"1 to 0" correspondence is represented by low link scores (≈ 0.0) between a tracked object and all the detected ones.

"0 to 1" correspondence is represented by remaining columns in S after linking, i.e.

detected objects "do j (t)" which are not linked with any tracked objects "to i (t-1)".

Note that after this data association step, SIFT points outside of their objects (moved onto the background or onto other objects during occlusions) are dropped and replaced by new detected SIFT features following the detection and selection process described in sec (5.2.2). Sub-regions which are common to multiple objects in the case of partial occlusions are not used for the detection to avoid ambiguous situations.

On the other hand, the tracking algorithm keeps a uniform spatial repartition of the SIFT features by filtering out too close features. The system keeps the most reliable fea-

Occlusion Management

The partially occlusions are handled by a continuous tracking process on the remaining visible SIFT points. If the partially occluded object became fully visible, new additional SIFT points are detected and assigned to the tracked object using the same process as the one described for SIFT points detection and selection. This is due to maintain a global and well distributed representation of the object.

After link creation using S, some detected objects "do(t)" may not be linked with any "to(t-1)". They can corresponds to new objects appearing for the first time in the scene or previously occluded objects which re-appear.

Before initializing new tracked objects "to(t)" with unlinked "do(t)", an attempt to match these unlinked detected objects "do(t)" with tracked objects in occlusion state is performed using two reacquisition methods. The first one is fast and requires some basic information, extracted during the tracking of objects. The second one is more sophisticated and uses the visual signatures of the occluded objects to re-identify them.

This visual signature computing is described in the re-identification chapter (chapter 6).

The second reacquisition method is performed only if the first one provides ambiguous result, i.e. if the matching is not validated and not rejected with reliable decision (score).

Most of the time, the first method is sufficient to manage the occluded object tracking recovery. It is performed as follow:

During the tracking process of a fully visible object, four kinds of information are extracted :

The last state of its associated SIFT points (the point localisations on the object and their descriptors) are stored from the last fully visible image of the object.

The variations of its estimated real world dimensions (W and H), extracted using the camera calibration matrix as explained in (sec 5.1) are encoded in a Gaussian model.

The variations of its estimated real world velocity on the ground plane, also extracted using the camera calibration matrix, are encoded in another Gaussian model.

Its n dominant colors in HSV space are stored. We take only Hue value in consideration. n depends on the class of tracked object. For people, we use n = 2 (due to the general separation of a person body in torso and legs) while we use n = 1 for vehicles.

A first matching attempt is performed using SIFT descriptors matching. A set of SIFT points are detected and their SIFT descriptors computed on the detected object. A standard point to point matching is performed. If the SIFT point matching is successful, the reacquisition is validated at this point and no other matching process is required.

This is due to the high discriminative power of SIFT descriptors. On the other side, if SIFT point matching fails, the object matching process continues. This is due to the point of view dependence of detected SIFT points. The occluded object may reappear with a different visible side than the one with which it disappears and on which the stored SIFT points have been detected.

A second matching rejection process is performed using real object dimensions. If the dimensions of the detected object "do(t)" does not fit in the computed Gaussian model of the occluded object (using the variance of dimensions of the Gaussian model), the matching is rejected.

Finally, a matching score between a detected object "do(t)" and an occluded one "to(t-p)" is computed by:

score = 1 3 (Score W + Score H + Score hue ) (5.16)
where:

Score W = 1 -(detected object Woccluded object mean W) 2 max observed W variation (5.17)

Score H = 1 - (detected object H -occluded object mean H) 2
max observed H variation (5.18)

Score hue = 1 n n i=1 1 - |θ (i) 1 -θ (i) 2 | 180 (5.19)
n is the number of considered dominant colors, θ 1 is the corresponding angle in Hue color disc to the dominant hue value of detected object, and θ 2 is the corresponding angle of the dominant hue value of occluded object. These angle values are taken in degrees.

Two remarks are to be done for this matching score.

The first one concerns the uniform weighting of each information. In future work, an importance weighting method may be proposed according to the tracking result observations.

The second remark concerns the non use of velocity information in this matching score. In fact, a detected object "do(t)" does not have a velocity as it is a static object at the current frame and no temporal information are available for it. This velocity information will be used later.

High matching scores are used to validate occluded object reacquisition.

Non linked detected objects in the current frames, either by continuous tracking or by the first reacquisition attempt, are used to initialize new object tracking since they are considered as new appearing objects in the scene.

After few frames of tracking these new object, a second attempt for occluded objects reacquisition is performed using the velocity information. where:

Score velocity = 1 -|new tracked object velocityoccluded object velocity| max observed velocity variation (5.21)

Note that each occluded object is stored for requisition purpose only for a predefined time, to avoid the combinatorial explosion which can occur when the number of occluded object to reacquire increases greatly.

In our experiments, we start this second matching attempt, based on the new object velocity, after 50 frames of tracking to have a stable and reliable object velocity to compare, and we keep occluded objects for a maximum time of 1 mn before definitely considering them as lost (out of the scene).

This method allows to deal with most of occlusion situations, and is validated by the experimental results provided in chapter 7

Conclusion

We have proposed an object tracking algorithm, based on SIFT features for object representation, particle filtering for SIFT point tracking, and a data association framework to achieve the object tracking reliably.

The use of SIFT features is justified by two main reasons: the sparse point representation of the object allows more flexible tracking and object deformations/partial occlusion management. The robustness of SIFT descriptors and their high discriminative power increase the reliability of the tracking.

Particle filtering for SIFT point tracking draws its interest in the ability of paralleled exploration of several hypotheses for the new localisation of the tracked SIFT point. The contribution which we have proposed for particle weighting and SIFT point reliability measures allows a more reliable tracking.

The final data association framework allows to detect all possible kinds of situation which can occur during object tracking, especially the occlusion case. A Weighted temporal linking process is proposed, achieving the visible object tracking.

For occluded objects, we have proposed a real time method to perform their reacquisition if they reappear in the scene.

A modified version of our tracking algorithm, using FAST interest points and HOG descriptors around them, has been integrated in the main intelligent video surveillance software of Digital Barriers company, called "SafeZone R ", which has been deployed on many video surveillance systems in the world since 2011. It provides better tracking performance in comparison with the previous used tracking algorithm. This validate also the genericity of the proposed algorithm to any kind of local descriptor computed arround some points of interest.

The evaluation and comparison with state of the art results which validate our tracking approach is be provided in chapter 7, but some issues still exists with our tracking method. The first one is the use of image intensity information only during SIFT point tracking. The use of color SIFT descriptor will be tested in future work. The use of other type of information (local/global color descriptors, covariance descriptors, etc.) is also considered and will be tested in future work.

FAST PEOPLE RE-IDENTIFICATION

Color and texture are two important pieces of information for people appearance modelling. We have built our re-identification technique based on Farenzena et al. approach [Farenzena 2010]. This approach can be used both for single-shot and multi-shot cases, the . We present both cases but we focus more on multi-shot one due to the availability of mono-camera tracking results and the better results of re-identification when multiple images of each person are used.

Our choice to take [START_REF] Farenzena | [END_REF]] as work basis is motivated by the possibility to apply it in both single and multiple shot cases and by the fast processing (real time processing for small sampled set of images per person). We start by presenting the standard approach proposed in [START_REF] Farenzena | [END_REF]]. After that, we discuss the main issues of this approach, some of them are also issues for more recent and efficient approaches. Finally, we detail our contribution to solve these issues, achieving an improved version of [START_REF] Farenzena | [END_REF]] approach which provides comparable/slightly better results than state of the art ones while keeping the real-time processing of the original approach and dealing with more complex situations (object rotations, etc.).

Person Re-identification by Symmetry-Driven Accumulation of Local Features

The basic approach requires a background/foreground separation obtained by a background subtraction algorithm in the multi-shot case, or by using STEL (Structure Element) technique [START_REF] Jojic | [END_REF]] in case of single-shot. It computes a visual signature which is exclusively based on color information. This signature consists of three different representations of the color information, each representation focuses on specific aspect of the frequency/localisation of the color. To make the representation more robust to rotations and partial occlusions, the image of a human body is divided into 4 parts, according to two symmetry and two asymmetry axes. The extracted color descriptors are then reported to the body part to which they belong

Body subdivision: Assymmetry and Symmetry Axes

To perform the body image subdivision, the two horizontal asymmetry axes which separate head from torso, and torso from legs are firstly searched. Once they are defined, two other vertical symmetry axes, one dividing the torso and the other dividing legs in two symmetric parts are searched. Farenzena et al. [Faranzena] define two operators to perform this task. Given a person image of width J and height I, the first operator is the chromatic bilateral operator defined by:

C(i, δ) = B[i-δ,i+δ] d 2 (p i , pi ) (6.1)
where d(., .) is the Euclidean distance, evaluated between HSV pixel values p i ,p i , located symmetrically with respect to the horizontal axis at height i. This distance is summed inside a vertically sliding rectangular window B[i-δ, i+δ], with the same width J, and a height of 2δ centred vertically on the height i. To achieve the independence to scale changes, δ is taken as I/8 (See figure 6.1). The second operator is the spatial covering operator, which calculates the difference of foreground areas for two regions, defined by:

S(i, δ) = 1 Jδ |A(B[i -δ, i]) -A(B[i, i + δ])| (6.2)
where A(B[iδ, i]) is the foreground area in the window of width J and height δ,

delimited by [i -δ, i].
Using these two operators (eq. 6.1 and eq. 6.2), the four axes (2 asymmetry and 2 symmetry axes) can be estimated as follow:

The asymmetry axis i TL which separate torso from legs is defined as:

i TL = argmin i (1 -C(i, δ) + S(i, δ)) (6.3)
this provides the horizontal axis which separates two neighbouring regions with strongly different appearance (colors) and similar areas. The values of C are normalized. To reduce searching time and avoid some possible errors, the search of i TL holds in the interval [δ, 1δ].

The asymmetry axis i HT , separating the head from the torso is defined as:

i TL = argmin i (-S(i, δ)) (6.4)
this provides the horizontal axis which separates two neighbouring regions which strongly differ in area. For the same reason as previously, the search for i HT is

performed in the interval [δ, i TL -δ].
The two symmetry axes separating torso and legs horizontally are defined by:

j LRk = argmin i (C(j, δ) + S(j, δ)) (6.5)
where k = 1, 2 corresponds to the regions R1 and R2, respectively the regions defining the torso and the legs. Here, the searching of the torso symmetry axis is performed in the region delimited vertically by [i HT , i TL ], and the legs symmetry axis in performed in the region delimited vertically by [i TL , I], both by a sliding window in interval [δ, 3δ], δ = J/4 for this time.

Note that the head part (delimited vertically by [0, i HT ] is ignored in this approach due to the poverty of extractable information from this region in video surveillance context (low resolution, small images, etc.) and the non-use of biometric techniques.

Feature Extraction

Three types of color features are extracted from torso and leg parts. Their distance with respect to the symmetry axes j LRk is taken into account in order to minimize the effect of pose variations.

Weighted Color Histogram

HSV weighted histograms are computed in each of the 4 body parts. The weighting is performed according to the distance of each foreground pixel to j LRk of the region to which it belongs. More precisely, each foreground pixel is weighted by a onedimensional Gaussian kernel N(µ, σ) where µ is the coordinate of the symmetry axis and σ is a priori set to J/4. In this way, pixel values near j LRk have more importance in the final histogram. At the end, four weighted color histograms are obtained (one per part).

Maximally Stable Color Regions (MSCR)

The approach segments the people images in regions with stable colors using the MSCR (Maximally Stable Color Regions) algorithm [START_REF] Forssén | Maximally Stable Colour Regions for Recognition and Matching[END_REF]] (see figure 6.2 (d)).

The extracted regions are then described by their area, centroid, second moment matrix and average color, forming 9-dimensional patterns.

MSCR algorithm is applied on foreground pixels. In the single-shot case and in order to discard outliers, only MSCRs that lay inside the Gaussian kernel used for color histograms are selected. In the multiple-shot case, the MSCRs coming from the different images are accumulated by employing a Gaussian clustering procedure [Figueiredo 2000],

which automatically selects the number of components. The clustering is carried out using the 5-dimensional MSCR sub-pattern composed by the centroid and the average color of the blob. Blobs similar in appearance and position are clustered, since they yield redundant information. This clustering operation allows to capture only the relevant information and keeps low the computational cost of the matching process, where the clustering results are used.

Recurrent High-Structured Patches

This new descriptor is proposed by Farenzena et al. [START_REF] Farenzena | [END_REF]] to highlight image patches with texture characteristics that are highly recurrent in the person appearance (see figure 6.3). The extraction of RHSP is performed in three steps:

First, a set of patches p of size [I/6×J/6] are extracted randomly on each torso and leg regions independently. These patches are mainly sampled around the j LRk -axes in order to take symmetries into consideration. The more informative patches are selected by thresholding the values of entropy of all the extracted patches. This entropy is computed as

H = H R p + H G p + H B p (6.6)
where H R p , H G p , H B p are the entropy of red, green and blue channels respectively. The entropy of one channel image is given by:

H p = - n i=1 P i log 2 P i (6.7)
where P i is the probability of occurrence of pixel value i and is provided by the corresponding bin of the image histogram.

In the presented approach, patches with H higher than a fixed threshold τ H = 13 are kept (the authors have fixed this value experimentally)

The second step consists in discarding low recurrent patches. For each patch p, a set of transformations T i , i = 1, 2, ..., N T are applied to generate a set of N T patches p i , and to obtain an enlarged set p = {p1, ..., p N T , p}. These transformations T i consists in The last step consists in clustering remaining patches p in order to avoid patches with similar content. This is done using the Gaussian clustering [Figueiredo 2000] on the HSV histogram of the patches, keeping for each final cluster the patch nearest to the cluster's centroid.

In the multi-shot case the candidate RHSPs are accumulated over all frames.

Signature Comparison

To compare images of two persons, a dissimilarity measure is computed between their signatures. The signature consists in the combination of the three extracted features. The dissimilarity measure between a signature I A and another signature I B is given by: ). This distance is defined by two components: d ab y that compares the y component of the MSCR centroids, and d ab c that compares their mean color. In both cases, the comparison is carried out using the Euclidean distance. This results:

d MSCR = b∈I B min a∈I A γ • d ab y + (1 -γ) • d ab c (6.9)
where γ is a real parameter which takes values between 0 and 1 and is used to take more importance to color position (with respect to the considered axis) than color value or vice versa.

In case of multiple-shot comparison, the first step consists in clustering all the MSCRs of the multiple images of a given person to provide a unique representation of this person in terms of MSCRs, as if it is provided by a single image.

This clustering is performed on the images of the same person using the distance d MSCR (eq. 6.9). Once each multiple-shots MSCRs has been clustered and a representation of each person as a unique image MSCR is obtained, the final distance is computed exactly in the same way as in the single-shot case, using eq.( 6.9) d RHSP is obtained by selecting the best pair of RHSP, one in I A and one in I B . The minimum Bhattacharyya distance among the RHSP's HSV histograms is evaluated. This is done independently for each body part, summing all the distances achieved and then normalizing with the number of pairs.

In their experiments, Farenzena et al. [START_REF] Farenzena | [END_REF]] have used the first 100 images of the VIPeR dataset to estimate several parameters values, and have fixed them as follows: β WH = 0.4, β MSCR = 0.4, β RHSP = 0.2 and γ = 0.4.

Approach Limitations

Fixed Weights for Each Descriptor

In the initial approach, the weight of each descriptor is fixed using an experimentation on a subset of only 100 images from a unique dataset which is VIPeR (viewpoint invariant pedestrian recognition [START_REF] Gray | [END_REF]]) dataset. These fixed weights are used for all the experiments. These parameters are "probably" the best ones for VIPeR dataset as they were ex-

tracted by experiments on it, but it is clear that each dataset has its own characteristics. More generally, each video surveillance system provides different content for analysis, depending on several conditions: depending on the season/weather, it is observed that people generally wear cloths with uniform/dark colors and poor textures (overcoat for example) in winter or when it is cold while they wear cloths with various bright colors and textures in the summer or when the weather is mild. Indoor and outdoor surveillance systems may also provide different levels of information due to illumination changes. The deployment location of the video surveillance system also may provide different types of content. For example, an airport system will provide more people with Backpacks or carrying luggages (both can be integrated in the visual signature of people if they are not correctly separated or if they occlude body parts) than a downtown surveillance system. Many other factors may provide different types of content.

In [Meden 2013], the author provides an experimentation on the whole VIPeR dataset to show that only weighted color histograms provide the most significant information and the most important discriminant power of the approach. By testing some combinations of weights for each descriptor, the author demonstrates that MSCR and RHSP does not provide a meaningful improvement. The results of this experiment is provided in the CMC curves of figure 6.4. CMC (Cumulative Matching Curve) represents the probability to find a correct matching among the r best matching, for r = 1...R (R is the number of sorted matchs). r is the rank of re-identification. In [Bazzani 2012], the nAUC (Normalized Area Under the Curve) value is used to evaluate the re-identification performances. It represents the area under the CMC curve expressed in %. More explanations and details concerning CMC cuves and nAUC are provided in sec. 7.3.1 (Experiments chapter).

Following this experiment, Meden [Meden 2013] uses the following weights: β WH = 1.0, β MSCR = 0.0 and β RHSP = 0.0. This demonstrate that the original weights provided by [START_REF] Farenzena | [END_REF]] are not necessarily the best on the whole VIPeR dataset, and thereby, may be inappropriate for other datasets, but as we will show in next sections, removing totally the MSCR and RHSP descriptors is not the best way to obtain better re-identification results. An adaptive weighting system is suitable to deal with the current conditions and to give more importance to the most discriminant descriptor for each situation.

Exclusive Use of Color Information, Without Managing Color Rendering Difference Between Cameras

In the original approach, the three extracted types of feature (Weighted color histograms, MSCRs and RHSPs) use exclusively color information to characterize a person. Even if recurrent high-structured patches (RHSP) are extracted and selected using a complex process based on textures (by selecting patches with a minimal amount of entropy and by performing some transformations to keep the most robust patches to rotation using LNCC maps), the final characterization of each selected patch is performed using a simple color histogram. Adding these histograms to weighted color histograms and MSCR's, which are also color-based descriptors, provide some kind of redundancy and may be not sufficient.

Farenzena et al. use HSV histograms instead of a more specific feature for texture describing. They claim that this is because the RHSP's content is not necessarily a texture, since it exhibits less regularity.

We agree with this last assertion in the sense that effectively, the patches may do not contain enough texture information, but in the case they do, this useful information should be exploited.

For this reason, it is necessary to characterize the patches with both color and texture to provide better results.

More generally, using only color to compute people signature is risky, due to the various situations where this information is not reliable or is poor (the same remark as in the previous paragraph, concerning frequent dark clothes in many situations). For this reason, it is suitable to integrate texture information, which is decorrelated from color information, in the computed signature.

Note that the three initial color features are extracted directly from input images, without managing one of most important color-based signatures issue which is the difference in color acquisition between cameras, i.e. the images of the same object acquired by different cameras may show color dissimilarities. This issue affect greatly the original approach results especially on some dataset like iLids, where many people clothe colors are rendered differently (see in figure 2.26)

Dependency to Orientation

Another important issue of the [START_REF] Farenzena | [END_REF]] approach, which is also a general issue, common to many state of the art approaches, is the dependency of extracted signature to the visible side of people. For example, [START_REF] Farenzena | [END_REF]] approach assumes implicitly that the extracted features from a given person in one camera will be "visible"

in the other cameras. This assumption may be verified in several situations especially for weighted color histograms and MSCR if the considered people are wearing clothes with same appearance from all sides, but most of time and depending on the context, this assumption may be not correct. A person wearing an opened jacket with different colors than his/her t-shirt will provide different color histograms and MSCRs depending on whether they are observed from front or from back. Same remark for a person having a backpack or dragging a luggage behind him/her which occlude partially his/her legs.

Concerning RHSPs, their are strongly dependent on the visible side of people from which they have been extracted especially as they are more local descriptors than the weighted color histograms and MSCRs.

More generally, we can distinguish three main types of signature computing approaches for multiple-shot case, each of them can be impacted differently with the orientation dependency issue:

Accumulation and mean/variance modelling: This kind of approach aims at modelling the people appearance by encoding the variations of the observation/features as a "most likely appearance" and an interval of admissible variations for each feature, extracted from the available images of a given person. This kind of approach has the inconvenient to be too permissive and to enlarge the possible variation interval if the accumulation of features is performed without any control process and without any a priori knowledge concerning the provenance of information (visible side of the person). A "large" model may lead to a high rate of incorrect match during the re-identification process, due to its permissivity.

Filtering and selection of recurrent/constant features: This kind of approach aims at selecting and keeping only features which are stable among all available images of a given person. For example in [START_REF] Bak | [END_REF]], all the images of a given person are resized into the same size and divided on a grid of small subregions. These subregions are characterized by covariance matrices. A mean covariance matrix is computed for each subregion location. Only subregions with salient covariance matrices are kept for the person signature, computed by the proposed reliability measure in [START_REF] Bak | [END_REF]], based on the variance of each region in terms of covariance matrices, or by automatic learning using boosting. This kind of approach is too restrictive and has the inconvenient to provide a poor signature when the different sides of a given person are significantly different and are all used to compute this kind of signature.

Use of multiple-shot case as a set of single-shot: This kind of method (mainly the case of [START_REF] Farenzena | [END_REF]] approach) does not combine all the multiple images of a given person in a unique model, but keeps all the single-shot signatures of all people and compares all possible pairs of single-shot signatures from the query person with all signatures of candidate humans, and takes the lowest distance as the one for the given person comparison. This method has the inconvenient to be highly time consuming due to the number of considered person pairs, and the number of used images per person. It is possible to decrease the processing time by sampling images of the same person and by taking a small set only, but without any a priori knowledge concerning the visible side of the person, this sampling may lead to select images of similar content and avoid those representing other situations (other visible sides).

For these reasons, it is necessary to add the people orientation information to the computed signature.

Unreliable Body Subdivision

The symmetry and asymmetry axis estimation proposed in the original approach generally provides correct results when the background subtraction is correct and when the people are wearing t-shirt and pants with different colors. Both of these constraints are not always verified. In the case of bad results of background subtraction or their unavailability (static images, or detected people using a detector instead of background subtraction process) or in the case of uniform cloth color, the torso/legs separation may be incorrect and the asymmetry axis may be shifted up or down with respect to its real position (see figure 6.5 (a)). If the asymmetry axis is positioned differently between each image of the same person in multi-shot case (due to the variation of background subtraction for example), the matching performances may be decreased proportionally to the axis estimation error, due to the weighting process with respect to the position of this axis. We have indicated the torso/legs asymmetry axis manually on a set of images from VIPeR dataset, and we have compared the provided re-identification performances with those provided by automatic axis estimation. We have observed that small errors of positioning (like in figure 6.5 (a)) provide negligible performance difference while more important axis positioning errors lead to decrease the re-identification performances. On the other hand, the symmetry division may do not have sense if people are observed from profile. The symmetry axis aims at separating torso and legs into two "similar" and symmetric parts. In the case of profile view (see figure 6.5 (b)), this subdivision may be incorrect and does not provide any useful information. It is then better to perform this second division only if the front or the back side of the person is visible, which leads to the previous mentioned issue, concerning orientation dependence of signatures.

Proposed improvements

To deal with the previously mentioned issues, we propose the following improvements:

Geometrical body subdivision and image alignment:

To deal with the unreliability of the initial body subdivision and to improve information localisation on all images of the same person, we propose to use another method to divide human body, based on statistical dimension of body parts with respect to the whole body height. We also propose a method to align all the images of a given person to extract the corresponding information from the same parts, improving the re-identification process.

Color normalization before feature extraction:

To deal with the difference in color rendering between cameras, we propose to use a color normalisation method instead of camera colorimetric calibration approaches, due to the issues and constraints of this last kind of solution, which consists in the non-bijectivity of color projections and to the practical complexity to setup a real deployed camera network system by annotating a sufficient number of people and to perform the learning on all possible pairs of cameras (see 2.3.2.1).

RHSP characterisation by color and texture:

The RHSP characterisation by color histograms is replaced by region covariance descriptors, containing both color and texture information, enriching the available information for matching.

Use of SIFT descriptors as an additional texture descriptor: An additional texture information is used in the final signature, without any additional processing cost due to the availability of this information provided by mono-camera tracking process.

Use orientation information for visible side classification:

To deal with orientation dependency of visual signatures, each person is represented by a set of sub-signatures, depending on his/her visible side. Each acquired image is assigned to the corresponding sub-signature for update.

Use real world positions to filter incoherent/impossible matching: To decrease the number of candidate for each re-identification query and thereby, the re-identification error rate, a filtering step is performed using real world informations provided by camera calibration information

Adaptive weights for each descriptor: The weights are adapted to each person to provide better re-identification performances. No off-line learning is required to extract selected the weights.

Geometrical Body Subdivision and Image Alignment

As it was mentioned, the proposed symmetry and asymmetry separation is strongly constrained by the quality of background subtraction results and the difference of colors between torso and legs. To be independent from these constraints, and to "save" some processing time which is dedicated to asymmetry/symmetry axes estimation without a guarantee of success, we propose to not perform axis estimations and replace them by a statistical and geometrical body subdivision as in [START_REF] Huang | [END_REF]], by positioning the first asymmetry axis which separates the head from torso at 1/5 of the people image height and the second axis which separates torso from legs at 3/5 of the people image height.

The symmetry axes on the torso and leg parts are kept only if the visible side of the considered person is not known (see sec. 6.3.5). In fact, in addition to the imprecision of their estimation in general case (where background subtraction and people delimitation are not precise), these axes are proposed by [START_REF] Farenzena | [END_REF]] to provide some robustness to the used features against people rotations. Our visible side classification (detailed in sec. 6.3.5) allows to deal with people rotations in a more reliable way. This decision provides another non negligible effect: The weighted color histogram (WH) size is divided by two (Only whole torso and whole legs histograms are concatenated, unlike for [START_REF] Farenzena | [END_REF]] approach where 4 body part histograms are concatenated). This divide weighted color histograms comparison time by two for two images comparison.

By multiplying this saved time per the number of testes pairs of images in multiple-shot images and per the number of considered people, the saved processing time becomes not negligible.

In the following paragraphs, we keep the name of "Weighted Histogram" (WH) in all situations to keep the same feature names as in the baseline approach of [START_REF] Farenzena | [END_REF]],

even if in practice, if the visible side is known, the removing of symmetry axes cancels the weighting of pixel colors (pixel colors are weighted with respect to the distance of these pixels to symmetry axes). In this case, we can consider that all pixels have the same weight.

Another issue still persists: most of the time, people are delimited by bounding boxes with acceptable precision, and small errors provide negligible matching errors as mentioned in sec. 6.2.4. However sometimes, the person bounding box may not be precisely computed due to an incorrect background subtraction or an error from the people detector, cropping a part of the body (missing head in figure 6.5 (a)) or adding more background area providing a bad centering of the person within the bounding box. Detecting the images which contain these situations is not a trivial task, and the use of these images for the subdivision of 1/5, 3/5 of the height, and 1/2 of width may be greatly erroneous and it may decrease significantly the re-identification performances. For this reason, we propose also a method to align all the images of a given person provided by the mono-camera algorithm.

This alignment method assumes that most of extracted images of a person are correctly delimited and tries to identify and to remove/readjust those with cropped parts of the person or with additional background.

Given two images I 1 and I 2 of the same person, obtained from the mono-camera tracking algorithm, two cases can be distinguished:

The two considered images are successive or acquired in a small time interval. In this case, the variations in person size on images, and in the pose are negligible.

The two considered images are acquired at different moments (low acquisition frame rate or a sparse sampling). In this case, the size (scale) of the person images may be different if one image is acquired when the person is near the camera and the other image when he/she is far from it. The pose of this person may have probably changed too.

Both cases are handled by our alignment algorithm. This algorithm aims at position-ing I 2 on I 1 so that the body parts (head, shoulders, torso, legs and arms) are aligned as much as possible and with the most similar sizes.

The first step consists in determining the scaling interval. If the images are provided by our mono-camera tracking, which is the case in this work, the camera calibration information and the projection functions detailed in the mono-camera tracking chapter are used to define an interval of scaling factors in which we extract the best factor to apply on I 2 to reach I 1 scale.

It is possible to compute a scaling factor directly by dividing the width and height of I 1 by those of I 2 but due to the motivation of this process, i.e. unreliability of bounding boxes which delimit person images (cropped persons or additional background), the direct scale factor may not be correct. Taking an interval around this factor is possible too and is better than the unique factor obtained by dividing, but the magnitude of this interval cannot be determined without any a priori knowledge. For this reason, using real world information (camera calibration) when it is available (as in our case) is well suitable.

In the case of unavailable real world information (no calibration information available), which is the case of many datasets on which we evaluate our methods too, like VIPeR dataset for example, we use the second method cited in the previous paragraph, that defines a sufficient large interval around the factor computed by dividing width and height of I 2 by those of I 1 .

Once the scaling interval is defined, for each scaling factor, the following process is performed:

Rescale image I 2 with the current computed factor to obtain image I ′ 2 Downscale both images I 1 and I ′ 2 using an integer factor n by dividing both images on a grid of subregions of n × n, compute the mean color inside each subregion of the grid and assign it to the corresponding pixels of the smaller image. Two corresponding smaller images i 1 and i 2 are obtained.

Considering the top-left corner of i 1 as the origin, the image i 2 is slided on i 1 browsing many possible positions (x, y), x ∈ [x min , x max ] and y ∈ [y min , y max ],

and for each position (x, y) of i 2 , a dissimilarity distance between the content of overlapped area of i 1 and i 2 is computed. This global distance of matching is given as the mean of distances between colors of each corresponding pixels in the overlapped area (following figure 6.6):

d (x,y) (i 1 , i 2 ) = 1 n x≤u<x ′ y≤v<y ′ d L * a * b * (u, v) (6.10)
where n is the number of pixels in the overlapping area and d L * a * b * (u, v) is the distance between colors of two superposed pixels in the overlapped area, computed in L * a * b * color space as follow:

d L * a * b * (u, v) = (L * 2 -L * 1 ) 2 + (a * 2 -a * 1 ) 2 + (b * 2 -b * 1 ) 2 (6.11)
where

(L * 1 , a * 1 , b * 1 ) is the color of pixel (u, v) of the image i 1 and (L * 2 , a * 2 , b * 2 ) is the color of pixel (u, v) of the image i 2 (in the same referential)
This formula (6.11) is the CIE76 formula (proposed in 1976). It is the first colordifference formula that has been verified by Lab experimentation. More recent and sophisticated formulas (CIE94 and CIEDE2000) have been proposed to improve the distance computing and to deal with saturated regions, but we keep the first formula (6.11) due to the negligible contribution of the other formulas in our context and to their higher requested processing time (more parameters and mathematical operations per pixel:

∆E * 94 = ∆L * k L S L 2 + ∆C * ab k C S C 2 + ∆H * ab k H S H 2 and ∆E * 00 = ∆L ′ k L S L 2 + ∆C ′ k C S C 2 + ∆H ′ k H S H 2 + R T ∆C ′ k C S C ∆H ′ k H S H ).
The choice of working in L * a * b * color space is motivated by the fact that this color space provides the most similar color distribution as human perception and allows Once all dissimilarity distances are computed for all possible positions of i 2 in the defined intervals, the coordinates p (1,2) (x, y) which corresponds to the lowest distance are taken as the ones providing the best alignment.

A reverse computing is performed by multiplying the found coordinates p (1,2) (x, y)

by n to obtain the superposing coordinates of the original images. For a set of q images {I 1 , I 2 , ...I q } of a given person, which is the general case, this process is performed for each pair of successive images I m , I m+1 providing q -1 superposition points p (m,m+1) (x, y). These coordinates p (m,m+1) (x, y) are relative to different origins which are the top-left corner of the first image of the considered pair of images.

Once the process is achieved, all these coordinates are reported to an absolute origin which is the top-left corner of the first image I 1 of the considered set. This is done easily by a simple summation as long as the pairs are processed in successive order.

With the assumption that a sufficient number of images are correctly delimiting the person, we have observed than these images have very close superposing coordinates, which are null or very close to the origin if the first image delimits well the person.

The largest group of closest superposed images is considered as the correct delimitation of the person. All the other images which are slightly shifted/rescaled in comparison with the largest group of superposed images are kept after being adjusted. The other images which have an important shift (i.e. important crop) are dropped and not considered for the signature computation due to the lack of useful information.

Note that the downscale step is performed to speed up the alignment process. It is possible to apply this algorithm without downscaling, but the required processing time for testing all possible superposing positions after downscaling by a factor n is ≈ n 2 lower. This factor has to be multiplied by the number of image pairs to align per person.

We can then deduct the importance of saved processing time.

In our method, n depends on the size of people images. For example, we use n = 3 for small images like for CAVIAR dataset while we use n = 5 or n = 7 for large people images like for iLids dataset. For the shifting interval, by considering W 1 and H 1 respectively the width and the height of the image i 1 (of each pair of images), we use:

x min = -W 1 /2 x max = W 1 /2 y min = -H 1 /2 y max = H 1 /2
Some examples of the image alignment results are shown in figure 6.7.

Color Normalization Before Feature Extraction

As mentioned before, the color rendering difference between cameras, due to sensor sensibility difference or to external conditions (camera point of view/orientation, illumination conditions, etc.) is an important issue for visual signature comparison as long as color is an important information for this purpose.

Due to the issues and constraints of colorimetric calibration approaches, discussed at the end of section 2.3.2.1, related to the non-bijectivity of transfer functions and to the complexity to apply this kind of method in a large scale video-surveillance system with many cameras, we prefer to use a color normalization method.

Like in [START_REF] Bak | [END_REF]], we use histogram equalization [Finlayson 2005] method. This method supposes that the rank of colors are preserved during illumination changes. The rank measured for a level "i" of a channel "k" is given by:

M k (i) = i u=0 H k (u)/ Nb u=0 H k (u) (6.12)
where H k () is the histogram of the channel k and Nb its bins number.

Histogram equalisation stretches a range of histogram to be as close as possible to a uniform histogram. It is applied to each color channel (RGB) to maximize their entropy and obtain an invariant image.

RHSP Characterisation by Color and Texture

To deal with the exclusive use of color information for the signature computation, two improvements are proposed. The first one consists in the modification of the RHSP characterization method, and the second one, which is detailed in the next section, consists in the addition of another texture-based feature to the signature.

As mentioned before, the extraction method of RHSP patches, described in 6.1.2.3, provides a set of recurrent patches, which are also robust to rotations. Unfortunately, the final characterization of these patches is performed by simple color histograms, loosing the information of color repartition inside the patchs and does not encode any texture information, despite the use of LNCC in the selection process. We keep the same method for patches selection, but we propose to replace color histograms by region covariance descriptors (see sec. 4.1), containing both color and texture information. We use the following feature vector to construct our 7×7 covariance descriptors:

x y R G B I 2 x + I 2 y arctan |I x | |I y | T (6.13)
where: x and y are the pixel coordinates, R, G and B are respectively the values of red, green and blue channels,

I 2
x + I 2 y and arctan |Ix| |Iy | are respectively the magnitude and the orientation of the gradient (the edge) computed on the green channel (to save conversion processing time).

Using these features provide the needed information: the localisation of both color and texture information (x and y coordinates), the color information (R, G, and B) and the texture information ( I 2

x + I 2 y and arctan |Ix| |Iy |). Note that contrary to the use of covariance descriptor for people detection (chapter 4), where the processing time is relatively high, the use of this descriptor here does not increase the processing time. This is due to the fact that no mean covariance is required (and thereby, no iterative gradient descent computations), but only a simple covariance matrix computing for each patch, which can be speeded up using integral images, as explained in the chapter 4.

Use of SIFT Features as an Additional Texture Descriptor

To enhance the final signature by texture information, the SIFT features which have been used in mono-camera tracking are added to the final signature. The last known state (coordinates and descriptors) of all SIFT features of a given person are stored and used to provide a partial matching score which is weighted and summed to eq. 6.8).

The new signature matching formula becomes:

d(I A , I B ) = β WH .d WH (WH(I A ), WH(I B ))+ β MSCR .d MSCR (MSCR(I A ), MSCR(I B ))+ β RHSP .d RHSP (RHSP(I A ), RHSP(I B ))+ β SIFT .d SIFT (SIFT (I A ), SIFT (I B )) (6.14)
where the SIFT (.) is the partial signature consisting of SIFT features, and β SIFT is its normalized weight in the final signature.

The distance d SIFT evaluates the SIFT features similarity. Given two persons A and B to compare, each of them has its set of SIFT features provided by mono-camera tracking algorithm.

As described in sec 5.2.2, each person image is divided into a grid of subregions and each subregion contains a constant number of SIFT features. Thanks to the images alignment algorithm we use (sec 6.3.1), we can assume that these two persons are well delimited by their bounding box. It is then possible to resize their images to the same dimensions and thereby to obtain aligned grids of subregions and to compute the corresponding SIFT feature coordinates in the new images size.

Each SIFT feature f i (A) from A is compared to a set of SIFT features {f j (B)} j:1..n B from B. This set of features from B consists of the n B SIFT features which are inside the same grid cell (subregion) than f i (A) and those inside adjacent cells (subregions) to take in account possible shifting of features located near subregions borders. We also compute a SIFT descriptor on B image at the same location than f i (A) even if is not a detected SIFT point, and compare it with f i (A). This last point has the interest to ensure one point at least for comparison, so each SIFT feature from A provides a matching distance.

The comparison between two SIFT features is performed using an Euclidean distance like during mono-camera tracking. The smallest distance is taken as the one of f i (A) with its corresponding feature in B. The final distance d SIFT is taken as the mean of all the smallest distances between all {f i (A)} i:1..n A and their corresponding features in B.

Concerning β SIFT and as it was indicated in sec 6.2.1, this weight (as all the other) is no longer constant in our approach and is fixed according to the available information as it is described in the next section (sec 6.3.7).

Note that due to the local nature of both RHSP and SIFT features, their corresponding signatures are strongly dependent on the visible side of the person. This issue is managed by the use sub-signatures per person, detailed in sec 6.3.5

Use Orientation Information for Visible Side Classification

As it was mentioned before, the orientation change of a given person is an important issue for re-identification task. Both global and local features may be affected with different levels.

It is possible to use face detection to identify whether an observed person is in front of the camera or not, but this solution presents the following issues: First, face detection requires additional processing time, slowing down the whole process. Then, the detection performances are strongly dependent on image resolution and face sizes, which are not appropriate with most of large area surveillance systems. Finally, if no face is de-tected, that mean that the person is not observed on his frontal side. It can be observed from behind or in profile. A binary classification of visible side (frontal side / not frontal side) is not sufficient.

To make the visual signature more discriminant, we have decided to consider 8 possible classes for visible sides for each person (see figure 6.10(b)). Each visible side correspond to a walking direction. These 8 side classes are: the 4 main sides which are the front side (S), the back side (N), the left profile (W) and the right profile (E), in addition to the 4 intermediate sides which are the front-left profile (SW), the front-right profile (SE), the back-left side (NW) and the back-right side (NE).

The availability of camera calibration information and the real world information concerning people movements, provided by mono-camera tracking algorithm allow us to obtain more detailed information concerning the visible side of each person, assuming that people move forward.

The used calibration tool (figure 5.2) allows to position the the coordinate system as desired on the ground floor. If the coordinate system is not positioned manually, a default position is assigned by the calibration tool as follow (see figure 6.8):

The origin "o" is positioned at the vertical projection of the optical center of the camera on the ground floor.

The "Y" axis is perpendicular to the ground floor and is directed upwards and thus it passes through the optical center of the camera (hence, the term "Y-top" for this kind of calibration).

The "X" axis is the projection of the optical axis of the camera on the ground floor, and is oriented in the camera view direction.

The "Z" axis is defined as perpendicular to the plan defined by the two axes "X" and "Y". The direction of "Z" axis is defined using the standard "right-hand rule" (see figure 6.9).

We use this default positioning and orientation of the coordinate system. If another calibration tool is used with a different coordinate system configuration, obtained extrinsic matrix has to be transformed using the necessary translation and rotation matrices to reach the desired configuration of the coordinate system.

The mono-camera tracking algorithm provides the trajectory on the ground floor of each tracked person. The trajectory of a given tracked person consists in the set of its localization coordinates {(X i , 0, Z i )} i:1...N on the ground floor, where N is the number of frames where the tracked person is tracked, obtained using the "image to world" projection process (explained in sec 5.1.1). For a given person, his/her trajectory points are used to compute a regression line.

The slope of this regression allows to classify the visible side into one of the 8 defined classes (see figure 6.10 (b)). Unfortunately, people trajectories are not always straight, so the regression line of the whole points of the trajectory may not be representative of all observed visible sides. The trajectory of "Person 4" in the figure 6.10 (a) shows that 3 different sides have been observed successively (left side, front-left side and front side) while the global regression line correspond to a unique front-left side.

To deal with this frequent issue (no straight trajectories), the trajectories are divided into subsets of "n" consecutive location points. We consider that the change in visible side may not be important in a walking interval of 2 meters. Knowing that the average human walking speed is about 5km/h ([TranSafety 1997]), a person need approximately 1.44 seconds to walk 2 meters. Depending on the acquisition frame-rate, it is then possible to define the number "n" of successive location points which are traveled by a person in this time. In our case, most our used sequences are acquired at 8 fps by deployed systems of Digital Barriers, so the corresponding position points is 11.52 which is rounded to n = 12 points. This value has to be adjusted according to the average walking/running speed in the monitored area and to the acquisition frame-rate. To increase the precision of the visible side classification, we do not assign the current person image to it's visible side class directly using the regression line on the last "n" position. A buffer of "n/2" images (6 in our case) is used to assign the image to its class with a delay of "n/2" images. This policy allows to take the "n/2" previous locations and the "n/2" next locations of a given image to decide in which class it belongs.

Otherwise, if the current image is directly classified using the "n" last locations (including the current one as the last one), there is a risk that the current image correspond To compare the signatures of two persons A and B, three cases can be distinguished:

The two persons share some classes with non-null sub-signatures (i,e. they have been observed under some similar sides).

The two persons do not share any similar classes of non-null sub-signatures, but some of their non-null sub-signatures belongs to adjacent classes (for example, the person A has a "N" sub-signature while the person B does not have a a "N" sub-signature but have a "NE" or "NW" one).

The two persons do not share any similar classes of non-null sub-signatures, and any adjacent classes of non-null sub-signatures.

In the first case, where at least one non-null sub-signature class exists for both persons, the dissimilarity measure between the signature S A of A and the signature S B of B is given by:

d(S A , S B ) = 1 n n i=1 d(SS (i) A , SS (i) B ) (6.15)
where n is the number of common classes with non-null sub-signatures, SS (i) is the sub-signature of the class i, and d(SS

(i) A , SS (i)
B ) is the dissimilarity distance between two sub-signatures, computed using eq. (6.14) as for usual signatures before.

In the second case, where only some adjacent classes of sub-signatures are shared between the two persons (no common classes), the dissimilarity measure between the signatures S A and S B is given by:

d(S A , S B ) = 1 n n i=1 d(SS (i) A , SS (j)
B ) (6.16)

where n is the number of pairs of adjacent classes with non-null sub-signatures, SS (i) and SS (i) are respectively the sub-signatures of the adjacent classes i and j, and

d(SS (i) A , SS (j)
B ) is the dissimilarity distance between two sub-signatures, computed using eq. (6.14) as for usual signatures before.

Finally, in the last case where neither common classes nor adjacent ones are shared, the dissimilarity distance between the two persons is computed as in the initial approach, by considering all the sub-signatures as a unique one, but using the new formula 6.14 including SIFT features signature. In this case, the result is similar to the one of the initial approach (not worsen). This subdivision into 8 classes of visible sides has the following three main advantages:

In the general case, when the computed visual signature consists of a large model encoding all the possible observations from all acquired images of a given person, or consists of a selection of salient features which are present in all person images, this sub-signatures subdivision allows the extraction of information from the same sides and thereby, avoids too permissive or too restrictive signatures obtained from important variations in the appearance of each person side.

In our case, where the comparison between two persons consists in keeping the minimum distance of all tested pairs for WH and RHSP, the sub-signatures decrease the number of pairs to test by focusing on smaller subset of images belonging to the selected classes.

It allows a better feature weighting (β WH , β MSCR , β RHSP , β SIFT ) in the final dissimilarity measure computation (See next section). Due to the local nature of SIFT and RHSP features, their weight may be increased, decreased or totally avoided depending on their provenance (if they are not extracted from common sides).

Two important remarks have to be taken into account here:

-If the number of available images for a given person is high, using all these images may provides the best results but due to processing time requirements, we use only a smaller subset of these images. Our criteria for images sampling is mainly based on the visible side classification: For each not empty class, we select a constant number "n" of images at constant intervals of the trajectory which belongs to this class. In our experiments, we select the value of n according to the number of non-empty classes, the aim being to take 8 to 16 images at most. The selection is done by taking images at approximately constant intervals of position in the same visible side class. If no visible side classification is available, the 8 images are sampled with the same method as they belongs to the same class.

-After a full computing of MSCR features for a given image (generally the first one of a class of visible side), the MSCR computation on the next images of the same visible side class is performed faster. For each new image, we initialize MSCR detector with the optimal parameters (number of clusters to find, initial mean colors, initial color region centers and areas) according to the detected MSCRs on the previous image, assuming that images of the same sides provides closer MSCRs. This speeds up MSCR computing process. In the case this visible side classification is not available, the same process is applied on all the acquired images.

The improvements provided by this contribution are evaluated and highlighted in section 7.3.3.2. It consists on an experimentation and performance comparison on an extracted dataset which contains mainly people who changes significantly their visible side (by rotation) with respect to the camera.

Use Real World Positions/Velocity to Filter and Weight Matching

Depending on the availability of the monitored area plan (containing the delimitation of observed area per camera and the distances between them) and/or the calibration of the deployed cameras in this are in a common coordinate system, the re-identification process can be strongly improved in terms of performances and processing time.

In our study, we use a second tool for camera network calibration, developed by Digital Barriers France for the ViCoMo project, providing calibration matrices for each camera, but reported to a unique coordinate system. This part of improvements has been tester only on ViCoMo project sequences and has proven his effectiveness. The ViCoMo project video sequences were acquired in Eindhoven airport by 10 cameras, providing both overlapping and non-overlapping fields of view situations. Unfortunately, It is not possible to evaluate it on the most used benchmarking re-identification datasets (iLids, TrecVid, Viper) due a lack of time and to the fact that dataset images are provided as detected and cropped persons, without any real world information. We are planing to evaluate this part on large video sequences like TrecVid ones in our feature work when more time will be available.

We can distinguish two different situations concerning the re-identification between two cameras: The first case concerns cameras with overlapped fields of view. The second one concerns the case of the cameras do not have overlapped fields of view.

Cameras with overlapped fields of view: in this case, the re-identification of a person from the first camera to the second one is performed when this person is observed by the two cameras simultaneously (i,e. when this person is on the overlapping area).

Due to the possible imprecision of the detection bounding box (even provided by background subtraction or by people detector), the same person may provide slightly different coordinates on the ground floor between the two cameras. This is not an issue if the person is alone in a relatively small surrounding perimeter, but if more than one person is near the provided locations (which is the case in the processed ViCoMo sequences), the re-identification became non-trivial.

For this reason, a circular perimeter with a sufficient radius "r" is centered on the person position from the first camera. All persons who are out of this perimeter are not considered as candidates. The persons inside this perimeter are tested as candidate, by comparing their visual signatures with the query person one. In our experiments on ViCoMo video sequences, we use r = 2m.

The visual signatures comparison is performed as described in previous sections.

The dissimilarity measure between two signatures is weighted by the normalized distance (on the ground floor) of the corresponding candidate location to the one of the query person. For future work, we are interested by finding a reliable way to learn automatically the possible trajectories and the distances between the covered areas of a camera network. A first idea consists in assuming that people move with constant velocities and using the elapsed time between the exit from one camera view and the appearance in another one, it is possible to estimate a traveled distance. To avoid cases where a person stops for few time between the two cameras, or he/she takes a small/large detour, it may be necessary to use a large set of manually annotated trajectory samples and to perform some clustering to eliminate outliers.

d(A, B) = d(I A , I B ). (X A -X B ) 2 + ((Z A -Z B ) 2 r (6.

Adaptive Weights for Each Descriptor

As mentioned in sec 6.2.1, Farenzena et al. [START_REF] Farenzena | [END_REF]] use fixed weights for all the experiments. These weights are extracted experimentally as the best ones for a subset of 100 persons from VIPeR dataset. This approaches is limited due to the possible diversity of situations (indoor/outdoor, weather, illumination, location, visible side of people, etc.) and their provided type of information (poor/rich amount of color/texture).

We propose to use adaptive weights according to the kind of available information and to the visible side of each person, and this without any necessary offline learning, allowing to use this method easily in live surveillance system.

The most important point in our weighting method is that the used weights are not specific for a whole dataset or for all viewed persons in a given surveillance system, but the are specific for each query person independently, and more precisely, they are specific to each considered visible side information. The weights used to re-identify a person A may probably be different from those which are used to re-identify a person B even both persons are belonging to the same dataset or are observed by the same camera.

This policy has the effect of providing heterogeneous dissimilarity distances for different query persons, but it does not constitute an issue for the re-identification task as long as the dissimilarity distances are homogeneous for a given query person, i.e, the query person is compared to all the candidates using the same weights, providing coherent ranking. The lowest distance correspond to the most likely correspondence.

The weights are assigned to a given person depending on the his/her visible side(s), the importance and the discriminative power of the visual information this person provides. SIFT and RHSP features being local features, their weights vary according to the compared visible sides. On the other hand, the more his/her appearance is rich/discriminant in terms of colors, the higher β WH and β MSCR are. The same remark concerns the richness power of his/her appearance in terms of texture and the importance of β RHSP and β SIFT .

The following paragraphs explain how a given person appearance is considered as rich or poor in terms of color/texture and how, according to this decision, the different weights are assigned. The distance between the hue histogram of the person p to re-identify and the global hue histogram is also computed and noted as Dist color (p). Our color importance estimation is based on our assumption that the higher Dist color (p) is, the more informative the color of this person is. This is due to the fact that the global hue histogram of the camera network is representing a mean of observed colors. If Dist color (p) is high, it means that this person has a high probability to be separated from most of people thanks to its colors.

For the considered person p and to quantify this importance as an importance score, we use the following formula:

Score color (p) = Dist color (p) Dist color (max) (6.18)
Score color (p) values are in [0, 1]. We are aware that computing an importance score with respect to the max distance is not the best way to obtain an absolute information about the richness of color and its discriminative power, especially if Dist color (max)

is too low, but this method provides good results as it is demonstrated in the evaluation chapter (chapter 7). We believe that the main reason for this is that when Dist color (max) is too low, it corresponds to the case where all of people have quite similar colors. In large scale video-surveillance systems or large people datasets, this case is unlikely to occur, since if only one person has significantly different colors, Dist color (max) will be high enough. Even assuming that no person has significantly different colors compared to all the other people, this case happen generally with dark colors. In this case, the texture features (RHSP and SIFT) are generally impacted too (low textures because dark cloths), and since feature importance weighting is a relative one (Color VS. texture), the computed weights still have sens even if both color and textures are not discriminative enough (in similar proportions).

The Weighted histogram and MSCR features are not fully redundant and have the same importance in our approach since they are complementary and provide different contribution to the visual signature even if both are color-based features. MSCR encodes the spacial information of color distribution and some discriminative color region shapes and orientations (color stripes, circles, etc.) but are more sensitive to deformations (region centroid and orientations may vary) while color histograms are more robust to deformations but does not encore any spacial information.

RHSP:

A similar process as for color is performed. Each time a new image of a person p is acquired by a given camera, the different features (WH, MSCR, RHSP) are computed on this image and used to update the person signature as described before. The mean entropy of the selected RHSP patches (eq. 6.6) for this person is computed and assigned to him/her as additional information (it is not used for signature comparison). This value is noted Mean RHSP (p).

The max value of of all computed Mean RHSP (p i ), noted Mean RHSP (max), is assigned to the camera network as the corresponding value to the most textured person image.

Our RHSP importance estimation is based on our assumption that the higher Mean RHSP (p)

is, the more informative the texture of this person is (both gray values and color textures since we use region covariance descriptor with both information to characterise RHSPs).

If Mean RHSP (p) is high, it means that this person has a high probability to be separated from most of people thanks to its texture.

For the considered person p and to quantify this importance as an importance score, we use the following formula:

Score RHSP (p) = Mean RHSP (p) Mean RHSP (max) (6.19) Score RHSP (p) values are in [0, 1].

SIFT:

For SFIT features, two cases can be distinguished: If the SIFT features of the considered person p are provided by mono-camera tracking algorithm, we use the mean of their reliability measures, provided also by mono-camera tracking (eq. 5. with which importance. We believe that the middle case (using these features but with a lower importance) is a good compromise to exploit their effectiveness if the compared signatures are acquired from same or close visible sides, and to do not alter the comparison process greatly if the visible sides are neither common nor adjacent, thanks to the low coefficient these local features have.

Conclusion

We have proposed a context-aware and appearance based approach for people reidentification through video camera network. This approach is fully on-line processing and satisfies the genericity and easy deployment on large scale video systems constrains.

Our approach is based on a state of the art one ( [START_REF] Farenzena | [END_REF]). The main issues of the baseline approach have been identified and some improvements have been proposed

to deal with these issues. Some other improvements have been added to increase the efficiency of the final approach.

From visual signature efficiency point of view, we have proposed an image alignment method to make the computed multi-shot based signatures more reliable by taking and comparing the information from corresponding body parts on all images. We have also increased the discriminative power of RHSP patches by characterizing them using covariance descriptors, containing both color and texture information. We have added SIFT features to the final signatures, adding more texture information. We have also proposed a classification method for people visible side, based on our object tracking algorithm and the camera calibration information, allowing more precise comparison.

We have finally proposed a method to use camera calibration information to reduce the number of candidate for re-identification and to weight some matching hypothesis using real world distance.

From processing time point of view, the baseline method [START_REF] Farenzena | [END_REF]] is a realtime method for small sets of images per person (up to 8 images) and pseudo real-time method (up to 12 images). The improvements we have proposed does not increase greatly the processing time, and the final approach requires similar processing time.

This is due to many reasons.

First, symmetry and asymmetry axes computation time is saved by the simple constant vertical division of the people bounding boxes. Second MSCR features are computed using some optimisations. In fact, after computing all MSCR features of the first image of a person, the MSCR of the following images are computed faster using better initialization parameters (centroid localisations, region areas and orientations, region colors) knowing the results of MSCR of the previous image and with the strong assump-tion that the most important MSCR vary slightly from an image to the next one.

Even if covariance matrices are the most time consuming features, the simple computation and comparison still stay in real-time (or pseudo-real time) range. In people detection approach, the most consuming process step is the covariance mean computing, which is based on iterative gradient descent process. Here, this process is not required.

Images alignment process is also fast thanks to the used downscale step and the use of calibration information for dimension range definition.

SIFT features detection and and their reliability measure computation is provided by our mono-camera tracking algorithm and do not have to be performed here. In the case of unavailability of our mono-camera tracking results, the SIFT features are detected on the last image of each person using the same detection and selection method as described in 5.2.2. This does not increase greatly the processing time. Finally, The same mono-camera tracking algorithm provides the real world trajectories which allows to perform the visible side classification and the spatio-temporal coherency filtering. The visible side classification consisting in simple regression function computation on subsets of the trajectory, the processing time is negligible.

The evaluation and comparison with state of the art results which validate our reidentification approach are provided in chapter 7, but some issues still exists with our method. The two main issues consists in the used features and also in the way they are weighted. The proposed adaptive feature weighing method, even it provides good results in our tests, is not the optimal method. Some importance scores are computed relatively (to the max distances of observed color, or to the max of observed entropy)

and provides then a measure of discriminance in comparison with other people, but not a global measure on the richness in term of a given information type which may be more informative for better weighting. On the other hand, both color and textures may not be available in enough amount in some cases (dark and uniform clothes). This is a more general issue in state of the art and does not concerns only our approach.

EXPERIMENTAL RESULTS

This chapter presents a large evaluation of the proposed methods for people detection, mono-camera tracking and re-identification. It shows the performance improvements and the remaining limitations. Due to the unavailability of a whole evaluation framework (people detection, tracking, and re-identification) evaluation in the state of the art, the evaluations are performed for each processing task independently. We compare the results of each task with the state of the art results. The evaluation datasets are selected according to their popularity (the availability of results) and/or for the challenge they provide.

For each processing task, the corresponding evaluation metrics are firstly presented.

Then, the state of the art datasets used for evaluation and benchmarking are presented.

Finally, the evaluation results on each dataset are discussed, highlighting the improvement but also the limitations, explaining the most important reasons and proposing some ideas to solve the remaining issues for future work.

Efficient People Detector

Evaluation Metrics

There are two methodologies for people detector evaluation in the literature: the "Per-Window (PW)" performance evaluation and the "Full Image" performance evaluation. The "Per-Window (PW)" methodology is more dedicated to evaluate the performances of the classifiers only, by returning a decision concerning predefined candidate regions (selected by another process or delimited manually), while the "Full Image" methodology evaluates the whole detector, including the classifier (if it exists, because some detectors are not classifer-based), the browsing image method, the scaling search, etc.

"Full Image" methodology provides a natural measure of error of an overall detection system, for this reason and because most state of the art approaches are evaluated using "Full Image" methodology, we use this methodology for our evaluations too. To decide if a region detected as a person is considered as a correct detection or not, the overlapping area between the detected bounding box (BB det ) and a ground truth one (BB gt ) has to be greater than a threshold. We use the PASCAL [Everingham 2009] measure, which states that the overlapping must exceed 50% as follow:

area(BB det ∩ BB gt ) area(BB det ∪ BB gt ) > 0.5 (7.1)
An efficient people detector has to detect the maximum number of appearing people (true positives or TP) in images while it has to make the minimum number of errors, i.e.

the number of non-people detected as people (false positive or FP).

In literature, the first variable, i.e. true positive (TP) rate is generally replaced in evaluation by the false negative (FN) one, called also miss-detection rate. These two values are correlated since: TP rate = 1.0 -FN rate False positive and miss-detection rates are also correlated in a more general sense.

In fact, the more a given people detector is "permissive" the lower the miss-detection rate is, but at the same time, the higher the false positive rate is too. In the other hand, the more this people detector is "restrictive", the lower the false positive rate is, but the higher the miss-detection rate is.

The evolution of miss-detection rate with respect to false positive one can be obtained by varying the permissivity of the evaluated people detector. This is done in several ways depending on the nature of the detector, even by varying the detection parameters (searching scales, browsing steps, width/height ratio of the detection window, etc. corresponds to the number of annotated people in the ground truth while N FP + N TN (the denominator of eq. ( 7.3)) corresponds to the number of tested negative regions.

To compare detectors we plot miss rate against false positives per image in log-log scale.

Dataset Presentation

The proposed people detector has been evaluated on four dataset: INRIA Person dataset, DaimlerChrysler dataset, Caltech Pedestrian dataset and CAVIAR dataset.

INRIA Person Dataset

The INRIA person dataset [START_REF] Dalal | [END_REF]

DaimlerChrysler Dataset

The DaimlerChrysler data set [START_REF] Munder | [END_REF]] contains gray scale images with 4,000 pedestrian (24,000 with reflections and small shifts) and 25,000 non-pedestrian anno- The sequences being acquired by a mobile camera, the annotated people are used as single images in our evaluation, due to the inability to use background subtraction The interest of this dataset for people detection evaluation in comparison with the three previous ones is the availability of video sequences. This allows to uses background subtraction based features in the feature vector for covariance computation.

Evaluation Results

In the following evaluations, we have varied the miss-detection and false positive rates by removing iteratively the last level of our cascade of classifiers. The shorter the cascade is, the more permissive the detector is, i.e. it provides lower miss-detection rate while it increase false positive rate.

INRIA Dataset

This dataset being provided as single images acquired at different locations with different backgrounds, the background subtraction features ( G and G 2 x + G 2 y , see sec. 4.1.2) are not used in the covariance descriptors due to their unavailability. The camera calibration information is not available too, avoiding the use of real world information for candidate region selections. This increases the processing time of the detection process.

We have evaluated our method on INRIA People dataset and compare the obtained results first with the ones obtained with the baseline approaches ( [START_REF] Tuzel | [END_REF]] and [Yao 2008]) to quantify the improvements and second with the results of state of the art approaches. 7.5 (a) shows the performance comparison with [START_REF] Tuzel | [END_REF]] and [Yao 2008] which have served as the basis of our work. Our method outperforms these two approaches. This is due to the more accurate strong classifiers at each cascade level. As mentioned in the explanation of our approach (see chapter 4), the random nature of candidate weak classifier selection in [START_REF] Tuzel | [END_REF]] and [Yao 2008] does not allow to select the best possible weak classifier at each iteration. Our method limits the effect of the random selection by using groups of similar negatives (in terms of content with respect to the used region covariance descriptors) to train each cascade level. Thanks to this, the characterisation of positive data (people) is achieved more reliably. The best performance on this dataset is achieved by our method and "The Fastest Pedestrian Detector in the West" (FPDW) approach [START_REF] Dollar | The Fastest Pedestrian Detector in the West[END_REF]], based on a trained classifier with Adaboost on gradients of both gray scale and color images. Our method provides the lowest false positive rate while it has the lower miss-detection rate in comparison with [START_REF] Dollar | The Fastest Pedestrian Detector in the West[END_REF]] ones. Our detector performs the detection with an average frame rate of 7 fps while [START_REF] Dollar | The Fastest Pedestrian Detector in the West[END_REF]] does it in 9 fps approximatively.

Note that by removing at least the two last levels of our cascade of classifiers, the evolution of the miss-detection rate against the false positive rate is better ensured by [START_REF] Dollar | The Fastest Pedestrian Detector in the West[END_REF]]. From a pure detection performance point of view, this last remark does not have any interest as long as the main objective is to obtain a detector with the lowest miss-detection and false positive rates, which is the case with our detector, but in practice, other parameters have to be taken into account to decide which detector is the best one for specific requirements. The most important parameter is the detection time. While [START_REF] Dollar | The Fastest Pedestrian Detector in the West[END_REF]] approach allows to vary progressively the permissivity of the classifier without impacting processing time (only by changing the classifier threshold), in our case, adding or removing some cascade levels change the processing time in a non-linear way with respect to the number of added/removed levels (due to the nonconstant number of weak classifiers per cascade level). This means that depending on the processing time requirements of a given system and its tolerance to miss-detection and false positive rates, even our method or [START_REF] Dollar | The Fastest Pedestrian Detector in the West[END_REF]] one may be more adequate.

DaimlerChrysler Dataset

As for the INRIA People dataset, we have evaluated our method on DaimlerChrysler dataset and compared the obtained results first with the ones obtained with the baseline approaches [START_REF] Tuzel | [END_REF], Yao 2008] and second with the results of state of the art approaches.

Figure 7.6 (a) shows the performance comparison with [START_REF] Tuzel | [END_REF]] and [Yao 2008]. The best performance on this dataset is achieved by [Walk 2010] approach, based on a combination of features (HOG, HOF, CSS, Optical flow) with a linear SVM classifier, and

[?] approach, based on mixtures of multiscale deformable part models trained with a latent SVM.

Our method provides interesting results as long as they are not too far from those of the best approaches on this dataset. It outperforms the well-known [START_REF] Dalal | [END_REF]] and [START_REF] Viola | [END_REF]] approaches. We keep the same evaluation schema, by evaluating our method on Caltech Pedes-trian dataset and by comparing our results first with the results obtained with [START_REF] Tuzel | [END_REF] and [Yao 2008] approaches on this dataset, and then with the results of state of the art approaches.

Caltech Pedestrian Dataset

Figure 7.7 (a) shows the performance comparison with [START_REF] Tuzel | [END_REF]] and [Yao 2008].

Our method outperforms these two approaches on this dataset. The best performance on this dataset is achieved by [Walk 2010] approach too, as for DaimlerChrysler dataset.

Our method provides the second best results on this dataset. The most miss-detections in this dataset are due to partial occlusion. More than 50% of pedestrians are occluded on at least one frame. Our detector being a full body one, it is then more sensitive to occlusions with comparison with part-based detector.

CAVIAR Dataset

This dataset is provided as full video sequences, allowing the use of background subtraction features in region covariance descriptors. These sequences are acquired by static cameras, allowing their calibration and thereby the evaluation of the use of real world information for candidate region selection.

There is no available evaluation of state of the art people detection approaches on this dataset (initially, this dataset was used for tracking evaluation). For this reason, we have evaluated our detector and compared it with [START_REF] Tuzel | [END_REF]] and [Yao 2008] approaches only. Figure 7.8 shows the results of [START_REF] Tuzel | [END_REF]] and [Yao 2008] approaches, as well as the results of two versions of our detector: the first version consists of our detector without using camera calibration information for candidate region selection while the second one uses the camera calibration information.

We can observe that even without using the camera calibration information, our detector outperforms the [START_REF] Tuzel | [END_REF]] and [Yao 2008] ones. The reason is due to the negative sample clustering to reduce the effect of random selection of candidate weak classifiers, as explained above.

The most interesting remark concerns the contribution of camera calibration information. In fact, using this information allows to strongly reduce the number of tested regions during detection. The first effect is an important processing time reduction (around 60% in average in this dataset, but may vary from a camera to another one, depending on the height of the camera and the view angle). The second effect, which is visible on the curve is an important improvement of detection performances (less missdetection with respect to false positives). This last effect is explained by the fact that less negative candidate regions are tested. During the detection process without using camera calibration information, several scales of testing windows are used. For each scale, several width/height ratios are used too (around a mean ratio of 1/3). Camera calibration information allows to reduce not only the number of localisation to test (according to the ground floor) but also the range of scales and width/height ratio for each location. This leads to a lower rate of false positives. We have conducted a last experimentation to test how a trained detector on a given dataset is efficient on other datasets. To do this experimentation, we have selected the trained detector on INRIA dataset, due to the best results it provides in comparison with our other trained detectors on other datasets (except CAVIAR dataset, but we decide to do not select it because it is background subtraction features based).

We have applied this detector on the test images of each other dataset (Daimler-Chrysler, Caltech and CAVIAR). The evaluation results are shown in figure 7.9. We can observe that the detection performances are very bad. This demonstrate the dependency of a people detector to the dataset which has served for its training.

We are convinced that this is not an overfitting issue since the application of this detector (trained on INRIA dataset) on several images which are found on internet and not in the INRIA dataset (see figure 7.10) still provides good results. We believe that some specific characteristics of each dataset are not managed well during training stage (in all state of the art approaches) and are not integrated as full part features.

Robust People Tracking Using Particle Filter

Evaluation Metrics

Object tracking performances can be evaluated under several criteria, according to all possible situations which can occur during the tracking. Most generally, an "efficient" tracking algorithm must track all objects of interest in the scene, it must start tracking them as soon as possible after they appear in the scene, it must maintain the tracking of these objects are in the scene and does not stop the tracking until they leave the scene.

It must also assign a unique ID to each tracked object all the time this object is in the scene (not necessary visible at all the time: occlusion management).

Using ground truth data, several metrics are proposed in state of the art to quantify how a given tracking algorithm complies with all or a part of these conditions.

In our evaluation, we use two types of metrics to compare our results on state of the art results:

ETISEO metrics

ETISEO dataset, as it is presented in the next section, is rich in terms of environments and external conditions, so it is an interesting dataset to evaluate our tracking algorithm.

In order to be able to compare our tracker performances with the other ones on the ETISEO videos and on Caretaker dataset (presented in the next section), we use the tracking evaluation metrics defined in the ETISEO project [START_REF] Nghiem | [END_REF]]. These metrics allows to quantify precisely how the trackers comply with all the previously mentioned conditions.

In the following paragraphs, "reference object" refers to an annotated object in ground truth (object to track), and "tracked object" refers to an object delimited and tracked by the evaluated tracker. The match between a reference object and a tracked object is done with respect to their bounding boxes.

ETISEO evaluation metrics for object tracking consists of three metrics:

Tracking time metric, denoted M 1 , measures the percentage of time during which objects of interest (reference objects) are really tracked. It is given by:

M 1 = 1 N N i=1 T i F i (7.4)
where N is the number of reference objects; T i is the number of frames in which the reference object i is tracked (by the considered tracking algorithm); F i is the number of frames for which the reference object i is annotated.

Object ID persistence metric, denoted M 2 , is used to evaluate the ID persistence.

It computes over the time how many tracked objects are associated to one reference object as follows:

M 2 = 1 N N i=1 1 Frag i (7.5)
where N is the number of reference objects, Frag i is the number of different tracked objects (provided by the evaluated tracker) which corresponds to the reference object i.

Object ID confusion metric, denoted M 3 , computes the number of reference object IDs per detected object as follows:

M 3 = 1 D D i=1 1 Conf i (7.6)
where D is the number of tracked objects (provided by the evaluated tracker) matching with ground truth data, Conf i is the number of different reference objects which corresponds to the tracked object i.

Each of these metrics helps to evaluate a given aspect of the tracking algorithm efficiency. Their values are in the interval [0, 1]. The higher the metric value is, the better the tracking algorithm performance gets.

MT, PT and ML metrics

Wu et al. have defined in [START_REF] Wu | [END_REF]] five other metrics to quantify differently how a given tracking algorithm complies with the criteria mentioned at the beginning of this section. Three of these metrics are mainly used in several state of the art publications [Xing 2009[START_REF] Huang | [END_REF][START_REF] Huang | [END_REF][START_REF] Li | [END_REF][START_REF] Kuo | [END_REF][START_REF] Chau | [END_REF]. Let Nb GT Traj be the number of trajectories in the ground-truth of the considered video sequence.

These three metrics are defined as follows :

Mostly tracked trajectories (MT) : it correspond to the number of trajectory which are tracked correctly for more than 80% divided by Nb GT Traj .

Partially tracked trajectories (PT) : it correspond to the number of trajectory which are tracked between, 20% and 80% divided by Nb GT Traj .

Mostly lost trajectories (ML) : it correspond to the number of trajectory which are tracked less than 20% divided by Nb GT Traj .

For each GT object, the trajectory of the considered tracker which overlaps the one of GT object for most of the time is considered as the correct one.

Dataset Presentation

We have conducted the experimentation of our object tracking algorithm on four datasets: PETS 2001, ETISEO, CAVIAR and Caretaker.

PETS 2001 Dataset

The

ETISEO Dataset

The ETISEO videos are provided by the ETISEO project. This project seeks to work The ETISEO project provides a set of tracker evaluation results. We test the proposed mono-camera tracking algorithm on this dataset and we compare our results with these available results and with more recent trackers results which are evaluated on this dataset.

Figure 7.12: Samples from ETISEO dataset

CAVIAR Dataset

The Caviar dataset is presented in sec. 7.1.2.4. Both INRIA Grenoble and Shopping center of Lisbon are used for the evaluation of our object tracking algorithm.

The INRIA Grenoble sequences are used as they are provided, without any rotation to obtain vertical people like for people detection evaluation. This is a deliberate choice, to show the efficiency of our approach and the used particle filter for object tracking, independently of flat world constraints, as long as the calibration is correctly done. In the ambiguous cases, where the classification by the real dimensions is not reliable (see explaination in sec 5.1.2 and figure 5.8), we apply a rotation of 90 • to the ambiguous object (unclassified bounding box) and we apply our people detector as explained in sec.

5.1.2. This does not happen frequently in this dataset, so the real-time processing is not affected.

Caretaker Dataset

The Caretaker project focuses on the extraction of a structured knowledge from large multimedia collections recorded over networks of cameras and microphones deployed in subways. This dataset is highly challenging due to many factors: the scenes are crowded most of the time, the light reflection on smooth floor deteriorates the background subtraction providing imprecise people detection and delimitation, the poor video quality (highly compressed data), and the numerous static and dynamic occlusions. The frame rate of the videos (5 frames per second) also makes this dataset challenging for our proposed approach (due to the prediction-update nature of our approach).

Evaluation Results

Comparative Evaluation on ETISEO Dataset Using ETISEO Metrics

We have evaluated our mono-camera tracking algorithm on ETISEO dataset. This dataset has been acquired in several indoor/outdoor places and it contains various environements. It has been proposed for ETISEO project and has been used by several teams to evaluate their detection and tracking algorithms. Unfortunately, the names of these teams are not available and they are identified by numbers in the ETISEO project to preserve their anonymity. Details about their tracking approaches are also missing.

Available ETISEO project data provides only the final results.

We have compared our tracking algorithm with those of 7 teams from the project and with two tracking algorithms proposed in [START_REF] Chau | [END_REF]] on two sequences of the dataset.

The first sequence, denoted ETI-VS1-BE-18-C4 shows a building entrance. This sequence provides low difficulty level. The second sequence denoted ETI-VS1-MO-7-C1 shows an underground station. It is more challenging since it contains occlusions and is acquired with low contrast and bad illumination.

The results of this comparison is shown in figure 7.14.

Our tracking algorithm outperforms all other evaluated algorithms (or is equal to the best ones in some cases) except for the Object ID persistence metric (M 2 ) in the underground station sequence. For this metric, [START_REF] Chau | [END_REF]](1) tracking algorithm performs better. This is due to the crowd environment in this sequence. [START_REF] Chau | [END_REF]](1) tracking algorithm is based on an off-line learning to extract the best features and parameters. In opposite, our algorithm does not use any off-line learning and thereby, in on-line processing, it does not have enough time to learn correct reliability measures for each tracked SIFT features in one hand, and to learn correct real world dimensions and velocity to manage occlusions. More ID switches happen due to the unreliability of these measures.

A global evaluation result on the whole dataset is shown in figure 7.15.

Evaluation on ETISEO, PETS 2001, CAVIAR and Caretaker Sequences Using ETISEO Metrics

We have also evaluated The same reason (crowd scene) explains the low score of M 1 and M 2 for Caretaker sequence (see figure 7.16), concerning the initial tracking algorithm (red bars, without offline learning). In fact, the crowded scenes at some moments of the sequence does not provide enough time to our tracking algorithm to learn (on-line) the correct reliability measures of each SIFT feature, to allow dynamic models of SIFT features to converge (for particle filtering) and to learn the real dimensions and velocity of tracked people.

This issue concerns the new appearing people in the scene directly in crowded situations.

People who appear relatively separated from crow for enough time are well managed by our tracker event if they interact with the crowd later.

Even if our presented tracking system does not include any offline learning step to be deployed, we have conducted a special experiments to highlight the fact that our tracker performs better if complex situations (crowd/oclusions) does not occure immediately when a new object of interest appear in the scene, i.e. if the tracker has enough time to "learn" SIFT features reliability measures, dynamic models and object real world dimensions and velocity in online mode. To simulate this "enough time" requirement, we have modified slightly our tracking algorithm to allow it to load tracking ground truth data from xml file when they are available and to use them exclusively as the tracking result at each frame, changing all the variables and parameters values accordingly (use ground truth objects as delemitation instead of background subtraction/people detector results, correction/removing of SIFT features if they do not coorespond to the tracking state, correct real world dimensions and velocity after ground truth data projection using camera calibration information, etc.). When the ground truth data of a given object at a given moment are no longer available, the tracker resumes it process for this object normally as it was desinged initially.

After that, we have created a partial ground truth data by taking the original dataset ground truth and by keeping only the first 10 to 15 frames data for each object of interest. This correspond to 2 to 3 seconds (this dataset is acquired at 5 frame per second).

Finally, we load this partial ground truth in our modified tracking algorithm and we start the processing (object tracking on this sequence). For the evaluation, we did not count the annotated frames of the partial ground truth in the metrics computations (it is not fair to count them as long as the output of these frames is exactly the ground truth data for each involved object), we start metric computation for each object at the first frame the tracker resumes the tracking with autonomy. The results of this evaluation are presented in figure 7.16 at the upper bar (pink bar). We can observe that if our tracking algorithm has enought time (10 to 15 frame in our experiments) to learn and stabilize the several parameters involved in the process, which occures autonomously and online (without ground truth data) if objects of interest appears and evolve without occlusion in few frames, the results of tracking may be much better. Unfortunately, the offline learning is not a part of our proposed tracking algorithm, due to the hard application in real deployed video-surveillance systems.

Comparative Evaluation on CAVIAR Dataset Using MT, PT and ML Metrics

We have evaluated our tracking algorithm on CAVIAR dataset using [START_REF] Wu | [END_REF]] metrics to be able to compare our results with state of the art trackers. The results are shown in figure 7.17. We can observe that our tracking algorithm provides good results, but it is outperformed by some state of the art tracking algorithms. The partially tracked and mostly lost people are due to the image noise and low resolution. In fact, people tracking performances depend on people detection, and in this conditions, background subtraction and people detection results are altered, impacting the tracking start of some people, especially those who come from the far end of the corridor, and then decreasing the time these people are correctly tracked.

On the other hand, the best results are provided by [START_REF] Kuo | [END_REF]], [START_REF] Li | [END_REF]] and [START_REF] Chau | [END_REF]] (a) trackers. The higher performances of [START_REF] Li | [END_REF]] approach are due to the off-line learning step (HybridBoost) performed to select the best parameters and/or the most discriminative features. This adapts the tracker for this specific sequences. Our aim being to provide a tracker which is as generic as possible and which may be deployed with a minimum parameter tuning stage by operators, this explains the lower performance of our tracking algorithm, even if they are not bad.

[ [START_REF] Chau | [END_REF]] better performances are due also to an off-line learning, but not only. [START_REF] Chau | [END_REF]] propose an hybrid-based (on line/off-line) controller to select the best features and parameters and to refine the parameters on-line according to the tracking performances and scene context changes. The on-line part of this controller may be an interesting way to investigate to improve our tracker performances without decreasing its genericity.

Fast People Re-Identification

Most of datasets for benchmarking consists of extracted (cropped) images of people of interest. Some datasets provides only one image per person and per camera while the others are provided without real world information (real world coordinates and velocity), preventing the use of some of our improvements, especially the visible side classification and the spatio-temporal coherency filtering.

However, the other improvements are evaluated and compared to the initial approach of [START_REF] Farenzena | [END_REF]] and to the other state of the art approaches.

Evaluation Metrics

The re-identification system performance is strongly dependent on the number of considered candidate people for any re-identification query. The larger the candidate number is, the higher the probability to have similar people than the query person (in terms of appearance) is, and thereby, the lower the probability to find the corresponding person among the candidate ones is. It is then necessary to associate a performance measure to the number of considered people.

For assisted people tracking under camera network, or for an off-line system of people retrieval on stored videos, re-identification systems may significantly decrease the number of possible matching per query person, by filtering the less-similar people and by proposing the most "n" likely ones for the real matching (n depends on the size number of considered people in all the available video sequences and the expected performances). This helps operators to speed up their tracking/search of people of interest.

For these reasons, the most used metric for re-identification system evaluation is the Cumulative Matching Characteristic (CMC) curve.

Cumulative Matching Characteristic (CMC) Curve

For re-identification performance evaluation, we use a metric known as the cumulative matching characteristic (CMC) curve (see figures 7.18). The CMC curve represents the expectation of finding the correct match in the top n matches. The best case is given by a re-identification rate of 100% at the first rank (see figure 7.18 (b)). The higher the re-identification rate is at the lower ranks, the more performant the re-identification algorithm is.

Normalized Area Under Curve (nAUC)

The cumulative matching characteristic (CMC) curve provides a detailed performances per re-identification rank. To be able to evaluate and compare several reidentification algorithms, [Bazzani 2012] has introduced the "Normalized Area Under the Curve" (nAUC) value which provides re-identification performance measure as a scalar. It represents the area under the CMC curve expressed in % (see figure 7.19). The best case of 100% of re-identification at the first rank corresponds to nAUC = 100% (see figure 7.19 (b)). The higher the re-identification rate is at the lower ranks, the larger the area under the CMC curve is, and thereby, the higher the nAUC value is, corresponding to the better re-identfication performances. CMC curves, and in these ranges, curves do not reach 100% of re-identification performances). For the second case, we provide a nAUC value thanks to a geometry software which allows us to estimate the area of the polygon delimited by the provided curve and its straight line extrapolation (knowing the size of the dataset and the fact that once a curve reach 100% it will be constant at this value for the remaining ranks).

Dataset Presentation

VIPeR Dataset

The Considering images from CAM B as the gallery set, and images from CAM A as the probe set, each image of the probe set is matched with the images of the gallery. This provides a ranking for every image in the gallery with respect to the probe.

This dataset is the most challenging dataset currently available for the single-shot human re-identification, but unfortunately, it does not correspond to a video surveillance case for two main reasons: first, only one image per camera is available in this dataset while many images per person and per camera may be provided by a surveillance system, and the acquisition position of the cameras is quite low, providing frontal view while in surveillance context, cameras are frequently higher and provide angled view acquisition.

(Note: the random assignment of images to one set or to the other one makes the colorimetric calibration useless (erroneous) here.) Four subsets of images have been extracted for re-identification evaluation:

i-LIDS-119

This evaluation dataset contains 476 images with 119 individuals (each individual is represented by an average number of 4 images). These images have been extracted automatically by [Zheng 2009] from the sequences provided by two different cameras. This dataset provides images where in most of which, people are well delimited and centred in images.

i-LIDS-AA-RP (Automatically Annotated with Rotating People)

This dataset contains 30 individuals extracted automatically using our people detector (chapter 4) and mono-camera tracking (chapter 5) algorithms, providing multiple images for each person. The aim of this dataset, in comparison with i-LIDS-AA one, is the fact that only people who change significantly their walking direction while they are observed by the same camera are selected. This is a more

ETHZ Dataset

This dataset is captured from a moving camera, and it has been used originally for pedestrian detection [START_REF] Ess | [END_REF]]. Schwartz and Davis in [Schwartz 2009] extract a set of samples for each different person in the videos, and use the resulting set of images to test their Partial Least Squares Analysis method for re-identification. The moving camera setup provides a range of variations in people appearance. Variation in pose is relatively small, though, in comparison with the other two datasets. The most challenging aspects of ETHZ are illumination changes and occlusions. Using the same camera to acquire all images deprives this dataset of a main challenge which is largely met in video surveillance context: rendering variation due to sensor difference.

All images are normalized to 64 × 32 pixels. The dataset is structured as follows:

SEQ. #1 contains 83 pedestrians, for a total of 4.857 images.

SEQ. #2 contains 35 pedestrians, for a total of 1.936 images.

SEQ. #3 contains 28 pedestrians, for a total of 1.762 images.

CAVIAR4REID Dataset

The CAVIAR dataset is presented in sec. This dataset is very challenging due to the perpendicular view axes of the two cameras. People are mainly observed in front/back side in one camera while they are mainly observed from profile side in the other camera.

Evaluation Results

In the following paragraphs, we detail the evaluation of our approach on several dataset and compare our results with state of the art algorithms. We provide the results in two forms: a CMC curve displaying the results on a range of ranks which is common to all the state of the art approaches with which we compare our results, and a table containing numerical values of correct re-identification rates on the first and the 5th ranks in addition to the nAUC values when they are available/estimable. "-" means that the nAUC is not available/estimable.

VIPeR Dataset

We have tested our approach on VIPeR dataset and compared our results with state of the art ones. The results are shown in figure 7.25 and Table 7.1. Our approach outperforms the initial approach proposed by [START_REF] Farenzena | [END_REF]] thanks to the use of SIFT features and covariance descriptors for Recurrent High-Structured Patches (RHSP) characterization. Unfortunately, our approach is outperformed by [Dikmen 2011] and [START_REF] Cheng | [END_REF]] approaches. VIPeR dataset consists of a set of single-shot images per person and per camera. This reduces strongly the effectiveness of our approach even it is designed for both single-shot and multiple-shot cases. Image alignment contribution cannot be applied in this case since it requires multiple images per person, reducing the performances of our approach. In addition to that, people images are acquired under viewpoint angle changes from 45 up to 180 degrees. With single image per person and without any real world information (the dataset is provided as cropped images without any additional information), the use of SIFT features and covariance descriptors for RHSP characterization by default weights (see sec. 6.3.7), may alter the final result by providing bad scores for people observed from different sides. The best performance on this dataset is achieved by LMNN-R [Dikmen 2010] and PS [START_REF] Cheng | [END_REF] This dataset is very challenging since the number of images per individual is very low (4 images in average), in addition to the many partial occlusions it contains.

It has been used to evaluate single-shot approaches as long as multiple-shot approaches. Most of state of the art authors have proposed approaches for both cases, and the evaluation of their approaches has shown that the multiple-shot approaches provide better results. For this reason, we will focus the comparison on the multiple-shot case since it corresponds to the best results that each author has obtained.

The results of our evaluation on this dataset, and the comparison to the state of the art approaches are shown in figure 7.26 and Table 7.2. We can observe that the best results are provided by our method and [START_REF] Cheng | [END_REF]] (CPS: Custom Pictural Structure) one. Our method outperforms the initial approach of [START_REF] Farenzena | [END_REF] tanks to the several improvements we propose, especially the use of SIFT features, the covariance descriptors for RHSP characterisation, and a better feature weighting instead of those which are fixed and used by [START_REF] Farenzena | [END_REF]] and which have been learnt from VIPeR dataset, demonstrating that some parameters are datasetdependent.

Another contribution which may justify the better results obtained by our approach is the image alignment method we use to used comparable information (from the same body parts). This is also the reason why CPS ( [START_REF] Cheng | [END_REF]]) provides the best results. In fact, CPS approach is based on the localization of the body parts, after a heavy learning, and on the extraction and the matching of their descriptors. The fact that our method and [START_REF] Cheng | [END_REF]] one provide the best results demonstrate the importance of precise spacial matching between body part for the re-identification task.

Approach

1st rank (%) 5th rank (%) nAUC (%) The results of the evaluation of our approach on the i-LIDS-MA dataset, displayed in figure 7.27 and Table 7.3, shows that our method outperforms the initial one proposed by [START_REF] Farenzena | [END_REF]], but it is still less efficient in comparison with [START_REF] Bak | [END_REF]] approach.

The i-LIDS-MA dataset contains 40 individuals extracted manually from two cameras. The individuals are observed from back and back-right sides in both cameras and are well delimited and centred in the images.

The back side acquisition of people in both cameras allows the use of SIFT features and RHSP with covariance descriptors in our signature computation and comparison (their weights are not null), improving the re-identification performances in comparison with [START_REF] Farenzena | [END_REF]] approach. On the other hand, the textures on people cloth are not sufficient to allow the SIFT and RHSP features to provides the required discriminative power to outperform the "Mean Riemannian Covariance

Grid" (MRCG) of [START_REF] Bak | [END_REF]]. The MRCG signature is discriminative in general, and it is especially the case in this dataset, thanks to the good delimitation and alignment of people in the provided images, and thanks to the same side acquisition of all individuals in both cameras.

Note that our approach is significantly faster than [START_REF] Bak | [END_REF]] MRCG which uses 11 × 11 covariance matrices. In fact, the computation of one person signature using 46 images requires about 6 s with MRCG approach while our method requires about 620 ms for the same 46 images and 138 ms in average using 10 sampled images by counting the computation of SIFT features which are not provided by mono-camera tracking algorithm in this case). This is due to the high computational cost of covariance mean (iterative gradient descent, with all the required eigenvalue decomposition), performed for a large number of grid cells, in comparison with the simple 7 × 7 covariance matrices computation for RHSP characterization, without any mean covariance computation. (which is a special case here, because it is supposed to be provided by monocamera tracking algorithm in the whole system use), the SIFT feature detection and selection does not require important processing time since they are performed on the last image of each visible side class, using the grid subdivision as explained in the mono-camera tracking algorithm chapter (see sec. 5.2.2). We have evaluated our approach on the iLids-AA dataset which contains 100 individuals automatically detected and tracked in two cameras. The results are shown in figure 7.28 and Table 7.4. Our approach outperforms both [START_REF] Farenzena | [END_REF] and [START_REF] Bak | [END_REF]] approaches.

Approach

As in iLids-MA dataset, the people images are acquired from the same/close sides (back and back-right sides). This allows our method to use SIFT features and covariance descriptors for RHPS characterization in the visual signature (non null weights), improving the discriminative power of the initial approach of [START_REF] Farenzena | [END_REF] and outperforming it.

Unlike for iLids-MA dataset, our approach outperforms [START_REF] Bak | [END_REF]] MRCG on this dataset. The main reason is the better management of the badly aligned and delimited images of people by our approach in comparison to [START_REF] Bak | [END_REF]]. Cropped images are noisy and many person images are not centred on the croped images or are missing some parts. The MRCG approach being based on a mean covariance computation using a grid subdivision of images, badly aligned images alter the resulting means. [START_REF] Bak | [END_REF]] proposes a method to deal with this issue by testing grid shift, but the magnitude of the shifting is not sufficient in all situations and increases greatly processing time. The results of the evaluation of our approach on the i-LIDS-AA-RP dataset is displayed in figure 7.29 and Table 7.5. We can observe that our approach outperforms both [START_REF] Farenzena | [END_REF]] and [START_REF] Bak | [END_REF]] approaches.

Approach

In this dataset, people images do not suffer a lot from bad alignement and delimitation issues like in iLids-AA dataset. This is due to two main reasons: First, we have used our own people detector (chapter 4) and mono-camera tracking (chapter 5) algorithms which outperform the used ones for iLids-AA extraction (based on HOG detector and tracker). Second, we have selected the people following the main criteria that they change their visible side by turning with at least 90 • (some people are operating rotations higher than 180 • ). The chance has made this people fully visible (it was not intentional as long as we were focusing on the visible side changes).

With this dataset, we were expecting [START_REF] Bak | [END_REF]] MRCG to provide the best results, due to the high similarity of this dataset with the iLids-MA one in terms of good people delimitation and alignment in images and the low number of people in the dataset (30 in this dataset and 40 in iLids-MA). The evaluation shows that our approach outperforms [START_REF] Bak | [END_REF]] approach, which has decreased in terms of performances for the first ranks in comparison with iLids-MA even if the number of candidates is lower (the smaller the dataset is, the better the results are expected to be). In fact, [START_REF] Bak | [END_REF]] provides 80% of correct matches at the first rank in iLids-MA while it provides 70% at the first rank for iLids-AA-RP dataset. This shows the dependence of their approach to the visible side of people. The computed mean covariances are altered due to the variation of visible information in the same grid location when a given person turns. Our approach handles better this situation and is less dependent on the orientation of people.

Approach

ETHZ Dataset

We have evaluated our approach on the ETHZ dataset too. The results are shown in figure 7.30 and Tables 7.6, 7.7 and 7.8. We can observe that all state of the art approaches perform well on this dataset. The ETHZ dataset is the less challenging one in comparison with the other used datasets, due to the image acquisition system. In fact, all images are acquired using a single camera, avoiding most of the main challenges which can be encountered for real camera network (different color rendering, image resolution, etc.). This, in addition to the low number of people (in comparison with

VIPeR dataset for example), explains the good results of our approach and those of the state of the art approaches.

CAVIAR4REID Dataset

For CAVIAR4REID dataset, we have evaluated two versions of our approach and compared them with state of the art approaches. Initially, this dataset is provided as cropped images of people, without the real world information (people positions in the scene).

We have then evaluated a first version of our approach without any real world information, only using appearance based method with the related improvements (geometric body subdivision, SIFT, Covariance descriptors for RHSP characterization and adaptive weighting). For the second version, we have extracted the same set of people using our people detector (chapter 4) and mono-camera tracking (chapter 5) algorithms. We have manually made sure that the same people are available in our extracted dataset and we have selected 10 images per person by sampling all the available images using constant sampling step. This is done to compare with the other approaches. This dataset contains additional real world information for re-identification task and allows us to have global evaluation of the whole system (people detection, mono-camera tracking and people re-identification).

The extremely low resolution of images of the corridor camera makes this dataset very challenging. As expected, our first version (without real world information) outperforms slightly the initial approach of [START_REF] Farenzena | [END_REF]] which outperforms [START_REF] Bak | [END_REF] MRCG. We suppose that the low improvement of our approach in comparison with [START_REF] Farenzena | [END_REF]] approach is due to the difference between the used weights in our approach and those used in [START_REF] Farenzena | [END_REF]] approach. SIFT features and covariance descriptors for RHSP characterization do not contribute due to the difference of 90 • between the two views. Fortunately, the low resolution of the images lowers the entropy of images, and thereby, the importance given to texture, leading to low weights for SIFT and RHSP features. We believe that otherwise, an important weight for SIFT or RHSP features would have decreased the re-identification performances of our approach due to the unavailability of matching between these features from one camera to the other one (perpendicular view axes). Note that this last point is not a real issue in real case application because the SIFT and RHSP weights would be decreased or avoided thanks to the visible side classification stage using real world information. This is demonstrated with the second version of our approach.

Our first version is outperformed by CPS [START_REF] Cheng | [END_REF]] approach, which provides the best results on this dataset (if we considered only pure appearance-base approaches).

In the second version of our approach, we use the available real world information obtained from mono-camera tracking algorithm. As expected, the performances are significantly better, outperforming CPS [START_REF] Cheng | [END_REF]] approach and providing the best re-identification performances. On the other hand, and surprisingly, the performances are lower than expected. The overlapping field of view between cameras is supposed to help strongly the re-identification process by filtering the incoherent spatio-temporal matches, but it seems that the successive algorithms (people detection and mono-camera tracking), in addition to the equal weighting between real world distance and visual signature matching scores (see eq. 6.17) when many people are too close to each other, have introduced some errors which impact the final re-identification performances.

Approach

Conclusion

In this chapter, we have presented the experimental results of the three proposed algorithms for people detection, mono-camera object tracking and people re-identification in a camera network. For each part, we have presented the usually used metrics and some benchmarking datasets.

People Detector

We have tested various challenging datasets, providing a large amount of complex Note that we have shown that people detectors are strongly dependent on the training dataset. In our experiments, we have seen that a trained detector on a given dataset is not applicable on the other datasets with acceptable performances. On one hand, images of the same dataset contain similar characteristics: the same sensor for all acquisitions in a given dataset provides the same resolution and the same noise level, and the similar acquisition conditions for the same dataset provides close people sizes and close point of view acquisition (people pictures acquired by a pedestrian in cities, walking people on sidewalk acquired by embedded camera on a car, etc.). This specializes the trained detector and enables it to integrate these characteristics in the associate thresholds and parameters. On the other hand, these acquisition characteristics differ strongly from a dataset to another one, providing bad performances when detectors are used on different datasets than those used for their training. We suppose that this is a general issue for all people detector approaches since we did not find any evaluation of a unique detector on cross datasets in the literature. For this reason, it is unfortunately not possible to quantify the dependency of the other state of the art detectors to their training datasets.

The dataset dependency of people detectors complicates strongly the wanted genericity for our system (video-surveillance conditions).

Mono-camera Tracking

We have evaluated our mono-camera object tracking algorithm on several sequences from different datasets, in indoor and outdoor situations, with many object occlusion, low object contrast and low object resolution conditions.

The results show that our tracking algorithm provides the best performances on some sequences and close performances to the best trackers on other sequences. The lower performances occur mainly due to crowded scenes. In fact, to track a given object, our tracker requires to learn on-line some parameters on few frames (correct dynamic models, SIFT feature reliability, real world dimension and velocity, dominant colors). To allow this learning, the object to track should be correctly separated from other objects (no occlusion) for the few first frames in which it appears. This condition is hardly ensured in crowded environments like subway stations.

The state of the art approaches which outperform slightly our tracker use off-line learning to extract the most discriminative features and/or the best parameters for their algorithm. Off-line learning step, even if it improves on-line tracking performances, cannot be envisaged for our tracker due to the deployment requirements of surveillance systems. It may be hard to ask video operators to acquire some sequences and to apply the off-line learning to fine tune the tracking algorithm for each camera, because this operation may be too long for networks with many cameras, but not only. The other main reason is the fact that video operators are generally not experts.

For this reason, our tracker, even if it does not provide the best results on all datasets, represents a good compromise between performances and genericity/autonomy while being deployed in several environments and context.

People re-identification

Finally, we have evaluated our people re-identification algorithm and compared it with state of the art approaches on several datasets. Our method outperforms the initial approach we have taken as basis on all tested datasets, validating our improvements. It also outperforms many other state of the art approaches on several datasets. We have demonstrated the importance of three issues/solutions: First, we have seen the importance of a correct part-to-part accumulation/comparison on human body. The most effective approach from the state of the art on the tested datasets (PS and CPS [START_REF] Cheng | [END_REF]) uses an effective body part segmentation. Our image alignment method, even if it is not as precise as a body-part segmentation, seems to be sufficient to improve the re-identification performances in comparison with approaches which do not manage this issue.

Second, we have shown the importance of obtaining the people visible side information. Unfortunately, the used datasets in the state of the art do not highlight enough this issue (even by providing few numbers of people observed from different sides or by selecting people with similar appearance from all sides when the rotation is considered), while this issue is a frequent one in many environments. The dataset we have collected especially for this issue (iLids-AA-RP) shows the effectiveness of our method to manage these cases, thanks to real-world information provided by our mono-camera tracking algorithm.

Finally, we have demonstrated the contribution of the use of context knowledge by using camera calibration allows to improve re-identification performances, by filtering candidates with incoherent real world positions.

Note that most approaches which provide the best performances (LMNN-R [Dikmen 2011]) on some datasets (VIPeR) use off-line learning step. As it was mentioned for monocamera tracking and in the objectives of our work, offline learning steps are not considered as possible stages to perform wide scale surveillance, due to the important required time it requires for a large number of cameras and to the inexperience of most video operators with respect to this task.

CONCLUSION AND FUTURE WORK

This thesis presents our work to achieve the final aim which is providing a whole framework for people tracking through camera network. In this chapter, we expose the conclusions we have drawn after conducting this work, beginning by highlighting our main contributions (sec. 8.1.1), followed by their limitations (sec. 8.2), and concluding by perspectives (sec. 8.3) which have to be investigated to improve the performances in the corresponding fields.

Conclusion

In this thesis, we present our methods for people detection, mono-camera object tracking, and people re-identification for video camera networks. The three main constraints which have guided our work, namely high performances, real-time processing, and genericity/easy deployment in industrial context have been generally respected except for the genericity of people detection part. The genericity and easy deployment constraints have led to slightly lower performances in comparison of state of the art approaches in some cases (not always) but the gap between our algorithm performances and the best ones is sufficiently low to be considered as a good compromise between all the constraints.

Wide scale video surveillance constraints and challenges have been addressed and most of them have been well managed.

Contributions

This thesis brings the following three main contributions: an efficient (fast training, real-time/pseudo-real-time detection, high performances) people detector, a robust (high performances), real-time and generic mono-camera object tracking algorithm (turnkey system, without operator configuration or offline learning steps), and a fast (real-time/pseudo-real-time) and generic people re-identification algorithm (turnkey system, without operator configuration or offline learning steps).

The 10 detailed contributions of the presented work in this thesis consists of: In order to deal with extremal cases (low resolution, noise, small object sizes and bad background subtraction results) which cause bad/lost SIFT feature tracking in some situations, the particles of each SIFT feature are weighted using two different pieces of information: the similarity measure of their descriptors and the "estimated" background/foreground state of their location. To deal with possible errors of background subtraction results, a continuous (not binary) weighting method is proposed by estimating the background subtraction quality and assigning a weight according to its reliability.

A Data Association Framework for Object Tracking

This step infers object tracking (localisations and trajectories) from SIFT feature tracking, taking into account the on-line learned reliability of tracked SIFT features and their positions, and managing all possible situations: simple isolated objects, appearing/disappearing objects (leaving the scene or being occluded) and grouping/splitting objects. It creates reliable temporal links between object delimitation on each frame, providing the final object trajectories.

A Fast Occlusion Management Method

The mono-camera tracking algorithm being dedicated to static and calibrated cameras as it was mentioned in the hypotheses and constraints (in the introduction chapter), it is able to manage occlusions by taking advantage of useful real world information. We use object real dimensions (width and height) and velocities, after learning their variations during object tracking, to estimate matching scores between occluded objects and a candidate appearing ones. This information is used in addition to the matching score of tracked/detected SIFT features and to dominant color descriptors comparison to get the final matching score and to decide whether a new appearing object corresponds to a previously occluded one. The characterisation of Recurrent High-Structured Patches (RHSP) features using covariance descriptors, based on both texture and color information, instead of simple color histograms improves the discriminative power of these features. The use of SIFT features as additional texture information with a negligible additional processing time (due to the fact that they are provided by mono-camera object tracking algorithm) increases this discriminative power and provides better re-identification results when these two features are usable. The local nature of both RHSP and SIFT makes them usable only when the same/nearest sides of people are visible in different cameras.

Visible Side Classification for More Reliable Signature Comparison

Appearance of people being different according to the their visible side in many cases, a unique visual signature for each person is not adequate. Assigning a signature to each visible side is a better way to identify a person. A visible side classification method is proposed. This method is fast and does not require complex computations. It is based on people real world trajectories subdivision and clustering, thanks to camera calibration information.

Spatio-temporal Coherency Filtering Method

Depending on whether cameras have overlapping fields of view or not, two methods using global camera calibration (in the same coordinate system) information and eventually environment maps if they are available are proposed. In the case of overlapping field of view, people move from a camera to another one by crossing the common field of view. The best opportunity to apply re-identification is then at this moment. Using camera calibration and images to world projections, a surrounding perimeter around the person to re-identify is used to filter all candidates who are too far and to weight visual signature matching scores by a real world distance. In the case of non-overlapping field of view, candidates with incoherent localisation (with respect to their velocity and the elapsed time) are filtered out, reducing the candidate number and thereby, increasing re-identification performances a speed.

Adaptive Weights for Signature Components

The proposed visual signature consisting of different features, the importance of each of them depends on the nature of used information and the considered people appearance. Off-line training being proscribed as much as possible, an adaptive method for feature weighting is proposed. This method assigns the weights per person and not per dataset. It takes in account the visible side of people to decide whether local features (SIFT and RHSP) are used or not, and to assign the several weights according to the richness/poverty of colors and textures.

Limitations

The proposed approaches still suffer from some limitations. Some of these limitations are specific to our proposed methods while others are more general limitations, impacting all state of the art approaches and constituting open issues. This section presents these limitations and the next section provides some ideas to deal with them, and which will be investigated in future work.

People Detection Limitations

Low resolution, noisy images and small people size

The combination of these three challenges demonstrate the limitation of covariance descriptors. Even if covariance descriptor is a powerful way to encode a large amount of information in a single discriminative and robust structure, it seems that it is not adequate for low resolution images. In addition, too small regions (less than 6 × 6 pixels) on noisy images are not well characterized.

Processing time

Despite the use of integral images to speed up processing, covariance matrix still have heavy computational cost (due to the several required eigenvalue decompositions). Our people detector, as a stand alone process, can perform in pseudo real time (and in real time for scenes with low complexity), but as a part of a more important framework like ours, containing mono-camera tracking and re-identification tasks which are also time consuming processes, it is not conceivable to use our people detector intensively, on each frame of each sequence, without any region targeting system.

Partial occlusions

The proposed people detector is a full-body based algorithm. It is supposed to be faster than equivalent body-part based detectors (using comparable features) due to the lower testing operations during detection and to the absence of a spatial reasoning step to infer whether detected parts correspond to a human, but present the main disadvantage of being less effective in case of partially occluded people.

Dataset/Acquisition dependency

This is a more general issue for people detection task in the state of the art. Even if the proposed approaches (including ours) are efficient and provide good results when they are tested on data belonging to the same dataset/acquisition conditions than those used for their training, these detectors cannot be used everywhere else, or on a specific dataset if they are trained on a particular other dataset. This issue compromises our search for generic solutions.

Mono-camera Object Tracking Limitations Dependency on detection results

The proposed tracking algorithm aims at creating temporal links between detected/tracked objects on successive frames. So the tracking algorithm needs to be provided by object localisation and delimitations. This task is performed by a collaboration between a state of the art background subtraction algorithm and our people detector. If some objects are not detected or are badly delimited, it impacts strongly the tracking performances.

Crowd/Occlusion object state at the starting of its tracking

The proposed algorithm being designed for generic and turnkey deployment, off-line learning steps are not considered. The algorithm learns some parameters (SIFT features reliability and real world information) on-line during the tracking. To learn correct values and dynamic models for SIFT features, the proposed algorithm requires for each tracked object to be "easily" identified and tracked for the first frames it appears. This does not necessary mean that the algorithm is only dedicated for sterile zone surveillance, it can track correctly people belonging to a group if their separation has been correctly detected by our people detector and if the group has uniform displacement over few frames (no people crossing during these first frames).

Exclusive use of a unique feature: SIFT

Even if SIFT features are known to be highly discriminative and robust against many condition changes, they still are only texture-based features. We believe that the more the object model is rich in terms in information, the better the tracking is, as long as the information is smartly used and allows real-time processing.

Processing time

The proposed tracking algorithm performs in real time for scenes with maximum number of 7 to 10 tracked objects. The grid subdivision and SIFT features selection methods we have used allow to reduce the impact of object size on processing time since the good spatial repartition of SIFT features allows to take approximatively constant number of SIFT features even if a person has a large size or not in the image. However, our tracker cannot be used for real time processing in crowded scenes.

People Re-identification Limitations

Low resolution images and small people size

As for people detection, covariance descriptors are not adequate to characterize RHSP with small dimensions (on small people images) especially when images are noisy.

Exclusive use of color for people image alignment

The proposed method to align images of the same person in multi-shot case generally provides good results. However, in some cases, when the contrast between people and the background is low, the computed color distances during the alignment process may lead to an erroneous alignment.

Processing time / sampling method for multiple-shot case

The proposed method performs in real time or pseudo real time depending on the number of used images to compute each person's signature. The more images are considered per person, the better the re-identification performances are, but the slower the processing is. Our method to samples the available images of a person, based only on selecting images after constant trajectory intervals and per visible class side may not be optimal. This sampling method does not take into account any visual information.

The sampled images may be too similar and do not represent the possible variations in appearance of the considered person.

Fully autonomous people tracking under camera network

As it was shown in the evaluation chapter, the performances of the re-identification task in the state of the art are not good enough to allow a fully-autonomous people tracking under a camera network. This kind of system requires a high re-identification rate at the first rank to be interesting (as long as the aim is to track reliably and unsupervisedly a person in the camera network). State of the art re-identification approaches

are not yet mature enough to address this task for generic and wide scale surveillances systems, with many more people than in the used datasets and with a large number of cameras (which increase the combinatorial complexity and the classical issues). More research work remains to be done to reach this objective. However, many other types of application can take advantage of proposed approaches and their current performances.

Mono-camera Object Tracking

Use of additional and "light" features for tracking initialization

To solve our issue of bad parameter learning in the few first frames when complex situation occurs, a possible solution may be to apply another kind of object tracking, based on simple information like color histograms and pseudo-exhaustive search (low processing time requirement), to provide the first positions of a tracked object and to allow the main tracking algorithm to learn the necessary values.

Integrate color information more efficiently in the tracking process

The proposed approach uses color information only for occlusion management purpose, and in a simple way (dominant color descriptor). It may be more efficient to integrate color information in the core tracking algorithm, by computing Color SIFT features or by tracking color patches using similar particle filter.

People Re-identification

Better image sampling for multiple-shot case Instead of sampling multi-shot images only by selecting images after constant trajectory intervals and per visible class side, which may take too similar images, this sampling should be performed using visual information criteria, by maximizing the variance of the considered features on the images to select. This may lead to use more representative images of possible variations in people appearance for signature computation.

Use of texture as additional information for image alignment

The presented image alignment algorithm provides good results most of the time but fails when the people cloth colors are too close to background ones. Adding texture information may solve this issue in a larger extent.

Long-term Perspectives

People Detection

Automatic Context Classification

Depending on the used features for people detection, it may be very useful to identify a way to characterize and classify the deployment environments (or datasets) by grouping them in similar context sets. Once this classification is performed, dedicated people detectors may be trained on each class of context. The final aim is to be able to identify to which class a given new deployment environment (or dataset) belongs, and incoming complex situations for a given tracked object, thanks to some predictions on object trajectories in the scene, their separating distances, their velocities, etc. For example, it may be interesting when in the same scene, some objects are fastly tracked using simple color histograms due to the fact that they are isolated while other objects are tracked using more complex and discriminative features like SIFT due to their proximity to each other or because they are partially occluded. This leads to a system in which, at the same moment, in the same scene, some objects are tracked with only their color histograms while other are tracked using SIFT features. This kind of approach allows to save some processing time in simple case tracking. This is a different approach than those [START_REF] Li | [END_REF][START_REF] Chau | [END_REF]] which use off-line learning step to select the best features which may be used later for all objects.

People Re-identification

Automatic Context Classification

As for people detection, an automatic context classification method may help to identify which features are more relevant to reach the best re-identification performances.

In our case, the feature weights used for each signature component may be more precisely selected knowing more information concerning the deployment environment (or dataset).

Group-context information as additional descriptor

In many surveillance area (airport, train station, etc.), the possible displacement ways are limited, and people move following very frequent trajectories and thereby, constituting more or less constant groups. It would be interesting to explore group-context additional information, in a more sophisticated way than in [Zheng 2009], by adding real world information combined with all other our proposed improvements.

Use object detectors for usual object removing

Many occlusions are due to usual objects which are taken by people. For example, in airport backpacks, luggages and trolleys are frequently occluding people. When the people are well delimited and the occlusion amount is low with respect to people size (small backpack, carried luggage which occlude a small part of a person, etc.), this occlusion affects the visual signature by integrating wrong information, but the effect may be low. On the other side, if the occlusion amount is high, the computed visual signature is highly altered. To deal with this issue, the use of detectors for usual objects may be an interesting solution. If these objects are well delimited also, their corresponding image regions will not be used in the visual signature computing, and may be replaced by

Figure 1

 1 Figure 1.1: Video-surveillance center: too many screens and difficulty to monitor all screens.

  My work was conducted under a collaboration between STARS team from INRIA and Digital Barriers company. STARS (Spatio-Temporal Activity Recognition Systems) team is an INRIA research team, which focuses on the design of cognitive vision systems for Activity Recognition. The research team is interested in the real-time semantic interpretation of dynamic scenes observed by video cameras and other sensors. It studies long-term spatiotemporal activities performed by agents such as human beings, animals or vehicles in the physical world. The major issue in semantic interpretation of dynamic scenes is to bridge the gap between the subjective interpretation of data and the objective measures provided by sensors. To address this problem Stars develops new techniques in the field of cognitive vision and cognitive systems for physical object detection, activity understanding, activity learning, vision system design and evaluation. Stars focuses on two principal application domains: visual surveillance and healthcare monitoring. STARS has two main research themes: Scene understanding for activity recognition and Software architecture for activity recognition.

Figure 1

 1 Figure 1.3: iLids multi-camera tracking challenge environment: 5 cameras with and without overlapping of field of views. C1, C2 and C3 are the viewed scene of the corresponding cameras. The position and orientation of each camera is displayed in the floor plan.

Figure 1

 1 Figure 1.4: Iris identification: Collaboration and specific actions from people are required. The person has to put his eye in front of the sensor.

Figure 1 . 5 :

 15 Figure 1.5: Fingerprint identification: Collaboration and specific actions from people are required. The person has to put his finger on the sensor.

Figure 1

 1 Figure1.6: Face recognition: People in front of camera (green rectangles) can be processed for face recognition. The person who is seen from behind (red rectangle) cannot be processed. The last person seen from the side (orange rectangle) is in the limit of constraints and processing can fail. Here, collaboration of people is not required.

Figure 1 . 7 :

 17 Figure 1.7: Gait recognition: gait is decomposed into different identifiable phases which are processed. No collaboration from people is required.

Figure 1

 1 Figure 1.8: Dependency between people detection, mono-camera tracking and re-identification. In this figure, two cameras are used but the architecture can be generalised for a network of many cameras.

-

  An optimization method to improve cascade of classifiers based people detectors: In order to speed-up classifier training, detection task and to improve detection performances, we propose a preprocessing step which optimizes a state of the art approach. This optimization is based on clustering negative training samples before classifier training in a specific way, which can be generalized and used for all techniques which use trained cascade of classifiers. This work has been published in VISAPP 2013 [Souded 2013].

  the state of the art of the three topics covered in this thesis.The first section concerns the state of the art in people detection on images and video sequences. This separation between static images and video sequences is mainly related to the type of used features due to the additional information provided by the movement. The state of the art in people detection can be divided into three categories, depending on the processing step for detection: candidate region selection, pertinent feature extraction and learning/classification techniques.

  presented. People detection methods can be categorized into two main families: Trained classifier approaches and template matching approaches. The trained classifier based approaches are performed in two distinguished steps: training and detection. The training step focuses on the significant features extraction using several machine learning methods to obtain a person-class model. The obtained model is then used for detection using different ways.

  .1 shows a typical flow diagram of training based people detectors procedures. In the training step, discriminative and pertinent features are extracted from a training dataset and labelled according to their class (positive or negative data). A classifier is then trained on these data using several machine learning techniques, providing a person class model. For detection step, images

Figure 2

 2 Figure 2.1: Successive steps for people detector training and detection.

  Figure 2.2: Haar like features templates.

  Figure 2.2(b)

Figure 2 . 3 :

 23 Figure 2.3: Edge orientation histogram. (Left) Example image. (Center) Edge strength image. (Right) Polar plot of edge orientation histogram. (souce [Yang 2005]).

Figure 2

 2 Figure 2.4: HOG feature extraction steps (From [Dalal 2005])

Figure 2

 2 Figure 2.5: Generic LBP: Examples of circular neighbour sets for different (P, R) ([Ojala 2002])

Figure 2 Figure 2

 22 Figure 2.6: [Mu 2008] S-LBP computing method. Step A: Calculate principle directions and lengths for each arch. Step B: Vote for corresponding histogram bins.

  Figure 2.8: Illustration of sliding window and flat world assumption for candidate regions selection ([Gerónimo 2009]).

Figure 2 . 9 :

 29 Figure 2.9: Linear SVM illustration with 2-dimensional points separation. (i) H 1 , H 2 , H 3 are three sample plans from the infinite possible plans which can separate the two point classes.(ii) Optimal plan(in red) is taken, by maximizing the margin between Support Vector points (surrounded by purple circles).

Figure 2

 2 Figure 2.10: [Felzenszwalb 2000] body parts based people detector. (a) Articulated body model with its 14 joints and 15 body parts (the whole body is considered as a part, but not displayed in the figure for illustration clarity). (b) A hand-labelled training image and its extracted body part sub-images. Reading vertically from left to right: left upper arm, forearm, hand; left thigh, calf and foot;head, torso and whole body; right thigh, calf, foot; right upper arm, forearm and hand

Figure 2

 2 Figure 2.11: [Mikolajczyk 2004] body parts. (a) Frontal head and face (inner frame). (b) Profile head and face (inner frame). (c) Frontal upper body. (d) Profile upper body. (e) Legs

Figure 2

 2 Figure 2.12: Detection obtained with a single component person model ([Felzenszwalb 2010]). The model is defined by: coarse root filter (a), several higher resolution part filters (b) and a spatial model for the location of each part relative to the root (c). The filters specify weights for histogram of oriented gradients features. Their visualisation show the positive weights at different orientations. The visualization of the spatial models reflects the "cost" of placing the center of a part a different locations relative to the root.

Figure 2 .

 2 Figure 2.13: [Yang 2012] flexible mixture-of-parts model (middle) differs from classic approaches (left) that model articulation by warping a single template to different orientation and foreshortening states (top right). Small warps are approximated by translating patches connected with a spring (bottom right). For a large warp, a different set of patches and a different spring are used. The proposed model captures the dependence of local part appearance on geometry (i.e. elbows indifferent spatial arrangements look different).

Figure 2 .

 2 Figure 2.14: Three different poses of a person (top), with images generated from the histogram (middle) and spatiogram (bottom). The spatiogram captures spatial relationships among the colors, whereas the histogram discards all spatial information ([Birchfield 2005]).

Figure 2

 2 Figure 2.15: Example of Maximally Stable Color Regions (MSCR) extraction using [Birchfield 2005] method. (a) The original image. (b) An elliptical approximation of the extracted regions

Figure 2 .

 2 Figure 2.16: Object representations. (a) Centroid, (b) multiple points, (c) rectangular patch, (d) elliptical patch, (e) part-based multiple patches, (f) object skeleton, (g) complete object contour, (h) control points on object contour, (i) object silhouette. (source [Yilmaz 2006])

Figure 2

 2 Figure 2.17: Comparison between edge detectors. (a) A test image. (b) Results of edge detection on Figure 7. Canny had the best results (source [Maini 2009]).

  has a cost in term of processing time. Features extraction as well as similarity measure

  especially video surveillance, due to the high processing time it requires. Other deterministic methods such as Mean-shift or direct point matching are interesting methods and provide good tracking results as long as the main conditions are reached (initialization of mean-shift, reduction of point matching possibilities), otherwise, tracking can fail or become very slow.Probabilistic tracking methods are widely used in object tracking approaches in the state of the art. In opposite to deterministic approaches, probabilistic approaches integrate some randomness in their processing, generally assigned to measurement uncertainty or noise modelling. The same input data may provide different results from a processing to another one.The main common principle of these methods is to model object motion and to use this model to localize tracked object in two successive steps which are the prediction and the correction (update) one.The two main used probabilistic tracking methods are Kalman filters and Particle filters. While the Kalman filter tracks an object of interest in a single hypothesis mode, Particle filters use a set of particles to perform tracking. Each particle corresponds to a hypothesis of the system state (mainly tracked object localisation). Particle filters are more interesting since they allow to explore a large space of hypotheses in a parallel way.

Figure 2

 2 Figure 2.19: Daugman's IrisCode: (i) The set of used Gabor filters with different orientations and different resolution ((a) real parts / (b) imaginary parts). (ii) Localized irises with IrisCode R . (iii) Pictorial representation of IrisCode R .

Figure 2

 2 figure 2.20). These coefficients are compared with each other by calculating the difference between coefficients of high frequences and those of low frequences which are not generated by eyelids and eyelashes.Masek[START_REF] Masek | [END_REF]] and OSIRIS (Open Source for Iris) are two open source reference systems for iris recognition benchmarking. They are built on Daugman and Wildes approaches.

Figure 2 .

 2 Figure 2.22: Global vector obtained from the whole image without any feature localization

Figure 2 .

 2 Figure 2.23: [Heisele 2003] face detection and recognition approaches: (a) The 14 components of the proposed face detector. The white crosses denote the center of each component. (b) The 10 used components for face recognition

Figure 2

 2 Figure 2.24: A complete gait cycle with its three tasks and eight phases displayed. (Source [Søndrål 2005])

Figure 2 .

 2 Figure2.25:[START_REF] Liu | [END_REF]] average silhouette computation: First and second rows show samples of the binary silhouettes over one gait cycle, for two subjects, respectively. The third row shows the averaged silhouettes for the subject in the second row; each averaged over a different gait cycle.

Figure 2 .

 2 Figure 2.26: Difference in color rendering between cameras: The same person observed by two different cameras. (Images from iLids multi-camera tracking dataset).

Figure 2

 2 Figure 2.27: [Porikli 2003] example of Brightness Transfer Function application on one channel image: (a) Reference image. (b) over-exposed image. (c) Intensity histograms of the input image (shown as black), of the overexposed image (blue), and of the compensated image (red). (d) The model function that maps the over-exposed image to the original (red). (d) The compensated image.

  images of the same person acquired by two different cameras under different illumination conditions. It obtains as many BTFs as image pairs number. The final Brightness Transfer Function, which provides the best representation of the color changes between cameras is computed by applying a Principle Component analysis (PCA) on all the individual Brightness Transfer Functions.

(

  figure 2.28

  figure 2.29.

Figure 2 .

 2 Figure 2.29: Clothing segmentation using graph cuts [Gallagher 2008]: For each group of person images, the top row shows the resized person images, the middle row shows the result of applying graph cuts to segment clothing on each person image individually, and the bottom row shows the result of segmenting the clothing using the entire group of images.

Figure 2 .Figure 2

 22 Figure2.31: Computation of the color and shape based appearance model of detected moving blobs[START_REF] Kang | [END_REF]

Figure 2

 2 Figure 2.33: Example of constructing a three-level pyramid. The left column represents two person images with the detected body parts. Level 0 corresponds to the full body part. Level 1 and level 2 correspond to the rest of detected body parts and grids inside body parts, respectively [Bak 2010]

Figure 2

 2 Figure 2.34: [Ijiri 2012] metric learning for re-identification: (a) conventional and (b) proposed color matching schemes

  figure2.35 (b)). A dynamic programming algorithm is used to fit the model to the image of the person. Image regions are compared using color and structural

Figure 2

 2 Figure2.36:[Hamdoun 2008] people re-identification process: (a) Schematic view of model building. For each tracked person, interest points are collected from multiple images, and the personś signature is the union of all these key-points stored in a global KD-tree. (b) Schematic view of re-identification of a query. Interest points are collected similarly multiple images images of the person to be re-identified, and a vote is made according to the respective identifications associated to all matching key-points

Figure 2

 2 Figure 2.38: Sketch of SDALF approach [Farenzena 2010]: (a) two instances of the same person; (b) x-and y-axes of asymmetry and symmetry, respectively; (c) weighted histogram backprojection (brighter pixels mean a more important color), (d) Maximally Stable Color Regions; (e) Recurrent Highly Structured Patches.

Figure 2 .

 2 Figure 2.39: Example obtained by the graph-based approach for nonlinear dimensionality reduction. Multiple images of 5 different persons, acquired by two cameras are used.[Truong Cong 2009]

Figure 2

 2 Figure 2.40: Color-position histogram [Truong Cong 2010]: (a) original image; (b) localization of the silhouette; (c) color distribution in the silhouette.

Figure 2 .

 2 Figure2.41:[START_REF] Bak | [END_REF]] computation of three MRC patches. Covariances gathered from tracking results are used to compute the mean covariance using Riemannian manifold space (depicted with the surface of the sphere). The mean covariance forms MRC patch.

Figure 2

 2 Figure 2.42: Illustration of patch significance: (a) one of many frames obtained during tracking; (b) reliability map obtained by the first method; (c) confidence map obtained by boosting. Colours correspond to significance of patches (for clarity only W 4 x W 4 patches, shifted by W 8 pixels are illustrated, red indicates the highest significance, blue the lowest).

  main contribution takes place in this processing step. The proposed approach uses an adapted form of LogitBoost algorithm to train a cascade of classifiers for people detection. Due to the fact that region covariance descriptors relies to Riemannian Manifolds, the original LogitBoost algorithm was modified by[START_REF] Tuzel | [END_REF]] to deal with this specificity. Like all state of the art approaches which are based on an off-line training of a binary classifier to separate persons from non-person images, two type of images datasets are used: a set of positive images corresponding to various persons with different appearances and clothes to ensure the genericity of the classifier and a set of negative images corresponding to various things except people. This second dataset must be as various as possible and in term of content, and is supposed to be many times larger than the positive dataset.In the chapter 4, the training algorithm will be entirely detailed. The trained cascade of classifiers consist in a set of ordered strong classifiers, each of which consists in a set of weak classifiers. We will see that to train one weak classifier, heavy computations are required. The processing time required for training one weak classifiers is directly proportional to the number of randomly selected weak learners, but also and in large extent to the number of used samples. It is then clear that the larger the training dataset is, the slower the training is. At the same time, the more various and numerous the negative training data are, the less false positive detection rate is.We propose a pre-training step which allows the use of large training negative datasets while it decrease significantly training time in a first stage and it provide a better cascade of classifiers in term of structure and content in a second stage, speeding-up the detection also. This step consists in clustering negative data before training in a specific way, to train each cascade level with a given cluster, specializing the corresponding strong classifier to reject a specific kind of image information.The other main reason of our choice to take Tuzel et al.[START_REF] Tuzel | [END_REF]] as a basis for our work lies in the nature of the trained classifier. Training a cascade of classifiers using a LogitBoost algorithm in Riemannian Manifolds is an interesting way to show that our method can be generalized to any kind of object detector which is based on cascade of classifier, due to the fact that both vector space and Riemannian manifolds are used during the training and in for method.

  "t", some new detected objects can appear for the first time, other objects can disappear by leaving the field of view or by being occluded. All possible cases are managed in the data association step. This object detection is performed by a state of the art background subtraction al-3.2 Mono-Camera Object Tracking 123 gorithm. It provides all moving objects delimitations on the images. These moving objects can be various things: objects of interest (people, cars, etc.), other objects (tree branches, flags, etc.), illumination change and noise.

  For a given rectangular region R of I, let {z k } k=1..S be the d-dimensional feature points inside R. The region R is represented with the d × d covariance matrix of the feature Chapter 4: Efficient People Detector Based On Covariance Descriptors points

Figure 4 . 1 :

 41 Figure 4.1: Fast computation of the sum of pixels inside any rectangular region, using Integral Image representation.

  two feature dimensions i and j Due to the symmetric nature of covariance matrices, only upper (or lower) triangle values are computed, it means that d Integral Images P(i) i=1...d and (d 2 + d)/2 Integral Images Q(i, j) i,j=1...d are computed

  I x and I xx are respectively the first and the second order intensity derivatives on X axis I y and I yy are respectively the first and the second order intensity derivatives on Y axis I 2 x + I 2 y and arctan |Ix| |Iy | are respectively the magnitude and the orientation of the gradient (the edge).

Figure 4

 4 Figure 4.2: The 8 pixel features used in [Tuzel 2007].

  Yao and Odobez[Yao 2008] replace the two second derivatives features |I xx | and |I yy | by two foreground measures G and G 2 x + G 2 y . G denotes the foreground probability value (a real number between 0 and 1 indicating the probability that the pixel x belongs to the foreground), and G x and G y are the corresponding first order derivatives. These foreground features are obtained using a background subtraction technique which is restricted to moving people (see figure4.3). In the context of human detection in videos, foreground measures should be much more informative.

Figure 4

 4 Figure 4.3: [Yao 2008] positive examples with corresponding foreground probability maps (light -high probability, dark -low probability).

Figure 4

 4 Figure 4.4: A two-dimensional manifold. The tangent plane at X i , together with the exp and log maps relating X i and X j are shown. (source [Goh 2008]).

  belong to vector spaces but to the Riemannian manifold M of d × d symmetric positive definite matrices Sym + d . Based on the previously presented invariant Riemannian metric, and the minimal representation of covariance matrices on the tangent space proposed in [Tuzel 2007], Tuzel et al. have introduced a modification to the original LogitBoost algorithm to specifically account for the Riemannian geometry. The modified LogitBoost algorithm is presented in figure 4.5 To train a level "k" of the cascade, a given number of weak classifiers are successively added. To add a weak classifier "l" to the current training classifier, 200 candidate weak classifiers are evaluated: 200 subwindows are randomly selected.

Figure 4

 4 Figure 4.5: Pedestrian detection with cascade of LogitBoost classifiers on Sym + 8 (source [Tuzel 2007]).

Figure 4 . 6 :

 46 Figure 4.6: Comparison between structures of the cascade of classifiers: (a) Tuzel et al.[Tuzel 2007]. (b) Random subset selection: shorter cascade (4 levels less than original approach) but more weak classifiers in most of cascade levels. (c) our proposed approach: less cascade levels with less weak classifiers in most of them in comparison with (a) and (b).

Figure 4 . 7 :

 47 Figure 4.7: Three possible weak classifiers to reject: (a) low texture and non vertical shapes. (b) non circular shape at the top (head). (c) not separated shape at the bottom (legs).

Figure 4 . 9 :

 49 Figure 4.9: The 4 levels pyramidal subdivision of negative images for clustering. The level 0 (the left one) is the whole image.

Figure 4 .

 4 Figure 4.12: Illustration of clustering on a tangent space to a 2D Manifold.

Figure 4 .

 4 Figure 4.13: Mean of gradient images. First row: some various positive image samples with the mean gradient image of the positive training dataset. Second row: some various negative image samples with the mean gradient image of the negative training dataset. The mean of positive images represent a mean shape of the human body while the mean of negative images does not represent any shape.

Figure 5 . 1 :

 51 Figure 5.1: Background subtraction results for real video. The top-left image is the original one, the three other images are the results of background subtraction using different parameter values [McHugh 2009]

Figure 5 . 2 :

 52 Figure 5.2: Screenshot of Digital Barriers calibration tool

Figure 5

 5 Figure 5.3: Real world dimension projection. Depth is ignored

Figure 5 Figure 5

 55 Figure 5.4: World to image and image to world projections: a unique image point p 1 (x, y) corresponds to a given real world point P 1 (X, Y, Z) but an infinite number of real world points, belonging to the straight line (L) correspond to a given image point p 2 (x, y)

Figure 5

 5 Figure 5.6: Real height H estimation

Figure 5 . 7 :Figure 5 . 8 :

 5758 Figure 5.7: Object classification by fitting estimated real dimension in known model dimensions. (a) Original frame. (b) Background subtraction results and boundingbox creation after clustering. (c) Estimation of real dimensions of delimited objects (in meter) using camera calibration. (d) Filtering of unclassed objects which did not fit with known models and validation of objects of interest (people)

  1) where L (x, y, σ) = G (x, y, σ) * I (x, y) (5.2) I (x, y) is the original image and G (x, y, kσ) is a Gaussian filer at scale σ. Candidate SIFT points are then taken as extrema of the Difference of Gaussians (DoG) that occur at multiple scales. Each pixel in the DoG images is compared with its eight neighbors at the same scale and nine corresponding neighboring pixels in each of the neighboring scales (see figure 5.9 (a)). If the pixel value is the maximum or minimum among all compared pixels, it is selected as a candidate keypoint.

Figure 5 . 9 :

 59 Figure 5.9: Extrema points detection on the DoG (Difference of Gaussian) pyramid. (a) The build multi-octave pyramid. (b) Extrema detection (source [Lowe 2004])

  H are proportional to the principal curvatures of D. The curvature value is taken as r = α/β, where α is the larger eigenvalue of H and β its smaller one. The trace of H (D xx + D yy ) gives us the sum of the two eigenvalues of H, while its determinant (D xx D yy -D 2 xy

Figure 5 .

 5 Figure5.10: A keypoint descriptor is created by first computing the gradient magnitude and orientation at each image sample point in a region around the keypoint location, as shown on the left. These are weighted by a Gaussian window, indicated by the overlaid circle. These samples are then accumulated into orientation histograms summarizing the contents over 4x4 subregions, as shown on the right, with the length of each arrow corresponding to the sum of the gradient magnitudes near that direction within the region. This figure shows a 2x2 descriptor array computed from an 8x8 set of samples, whereas the optimal and described optimal descriptor use 4x4 descriptors computed from a 16x16 sample array (source[START_REF] Lowe | [END_REF]).

Figure 5

 5 Figure 5.11: SIFT points detection results. (a) The original low resolution image. (b) Detected SIFT points using the whole detection process. 37 points are detected on the image, 16 of them are on the object of interest (the person). (c) Detected SIFT points without low contrast and edge response filtering. More points are detected: 74 points are detected on the image, 39 of them are on the object of interest (the person).

Figure 5

 5 Figure 5.12: SIFT point selection for object representation. Red points are the ones selected and the blue ones are the ones removed at each step. (a) All the detected SIFT points without low contrast and edge response filtering. (b) Background point of interest removed by background subtraction mask. (c) Object of interest subdivision and selection of constant number points of interest per subregion, by keeping the most reliable ones. (d) Final object of interest (person) representation using SIFT points

Figure 5 .

 5 Figure 5.13: Prediction step: The red point is the previous location of the SIFT point to track at time t -1, and the cyan points represents the predicted positions of all the particles at the current time t

  figure 5.14).

Figure 5

 5 Figure 5.14: SIFT point tracking failure due to small size of the person's image, the proximity of the SIFT point to the person contour and to the low texture on the containing region. The SIFT point (red cross) is tracked correctly until frame 890, where it clings on the background which presents similar descriptor region (red square) at this location

  incorrect SIFT point tracking according to the background subtraction quality. When the extracted foreground pixels are too low, like in figure5.15(b) (low contrast and hight thresholds in the background subtraction algorithm), the new localisation of the SIFT point in each new frame slides on the foreground pixels and varies greatly. The localisation is then not precise.We propose a method to take into account background subtraction results to assign a real foreground weight to particles, even if the background subtraction provides a binary separation between background and foreground. This method allows to deal with the diverse background subtraction qualities. By modifying some parameters in the used background subtraction algorithm (contrast thresholds and the number of considered gaussian per pixel), we have obtained three different background subtraction qualities (see figure5.15).

Figure 5 .

 5 Figure 5.15: Different qualities of motion detection. (a) Original image. (b) Low detection. (c) Medium detection. (d) High detection

Figure 5

 5 Figure 5.16: Hybrid particle weighting. Hybrid particle weighting, taking into account background subtraction result qualities. The density 0 value is taken as density 0 = 0.50. (a-c) From low to good foreground extraction, by varying the contrast thresholds in the background subtraction algorithm. (d) Excessive (erroneous) foreground pixel extraction by decreasing contrast threshold and considering the mean of large blocks of pixels during the background subtraction

Figure 5

 5 Figure 5.17: Tracking of one SIFT point on the head of a person by particle filter: (a) the predicted position of all particles in blue and yellow. The yellow particles are the sampled ones, which will be used to generate new ones to replace degenerated particles in blue. (b) in red, the new location of the tracked SIFT point, computed as the barycentre of all weighted sampled particles.

  t

Figure 5 .

 5 Figure 5.18: Velocity component update using regression function performed on various set of last estimated positions. The 22 last positions of a tracked SIFT point are used for illustration. (a) Only the 3 last positions are used for regression computation. The resulting velocity vector is too sensitive to local variations and the prediction at next time are erroneous. (b)The 20 last positions are used for regression computing. The resulting velocity vector does not follow movement direction variations quickly. at time t, the estimated movement direction (green vector) indicated that the movement direction is to the top-left side while the real movement direction is to the left (see the 8 last positions). (c) The 10 last positions are used for regression computing. In this case, movement direction changes are managed better than in the two previous cases

Figure 5 .

 5 Figure 5.19: The five possible cases during the object tracking from a previous frame to the current one. (a) A simple 1 to 1 correspondence. (b) Merged tracked objects due to high object proximity or partial occlusion. (c) Split tracked object due to grouped object separation or occlusion end. (d) Lost tracked object due to scene exit or full occlusion situation. (e) New detected object without correspondance with any tracked object in previous frame. It can be a new appearing object or a priviously occluded one.

Figure 5

 5 Figure 5.20: From SIFT point tracking to Object (Person) tracking: (a) four SIFT points at different location of the person body are tracked using particle filter. (b) The person tracking result after data association step.

Figure 6

 6 Figure 6.1: Symmetry-based Silhouette Partition. On the top row, overview of the method: first the asymmetrical axis i TL is extracted, then i HT ; afterwards, for each Rk region the symmetrical axis j LRk are computed. On the bottom row, examples of symmetry-based partitions on images from the datasets. As you can notice, they coherently follow the pose variation. (source [Farenzena 2010]).

Figure 6

 6 Figure 6.2: Sketch of [Farenzena 2010] approach: a) two instances of the same person; b) xand y-axes of asymmetry and symmetry, respectively; c) weighted histogram back-projection (brighter pixels mean a more important color), d) Maximally Stable Color Regions; e) Recurrent Highly Structured Patches.

  rotations along the y central axis of the patch with several angles. The Local Normalized Cross-Correlation (LNCC) is then computed for each patch in p by considering only the LNCC value of the local region containing p and not the one of the whole person image. All the N T +1 LNCC maps (matrix representations of LNCC values for each patch) are then merged together into the average map. Patches containing small values (with respect to a fixed threshold) in this map are discarded.

Figure 6

 6 Figure 6.3: [Farenzena 2010] Recurrent High-Structured Patches (RHSP) extraction. The final result of this process is a set of patches (in this case only one) characterizing each body part of the pedestrian.

Figure 6

 6 Figure 6.4: Different weighting of SDALF components: (a) Only weighted color histogram (nAUC = 89.11%). (b) only MSCR (nAUC = 83.56%). (c) Weighted color histograms and MSCR with equal weights (nAUC = 91.81%). (d) Original weighting of SDALF like in [Farenzena 2010] (WH = 0.4, MSCR = 0.4, RHSP = 0.2) (nAUC = 91.57%).

Figure 6

 6 Figure 6.5: Symmetry and asymmetry issues: (a) incorrect torso/legs separation. (b) insignificant symmetry division.

  better color comparison. The non-linear relations for L*, a*, and b* are intended to mimic the non-linear response of the eye. Furthermore, uniform changes of components in the L*a*b* color space correspond to uniform changes in perceived color, so the relative perceptual differences between any two colors in L*a*b* can be approximated by treating each color as a point in a three-dimensional space (with three components: L*, a*, b*) and taking the Euclidean distance between them ([Jain 1989]).

  Figure 6.6: Images alignment

Figure 6

 6 Figure 6.7: Image alignment process illustration and results on iLids-AA dataset samples. In the first row, the alignment process is detailed: (a) 8 sampled images of the same person obtained by automatic people detection and tracking. The person is not delimited and centred correctly in images (various shifting). (b) Samples of alignment of successive pairs of images. (c) The final alignment results. (d) Some other results of people images alignments. In this figures, red bounding box corresponds to the first image of each set of images (defining the origin of coordinate system), and the blue bounding boxes are those of the remaining images of each set. Image transparencies have been modified to show the alignment results.

Figure 6

 6 Figure 6.8: Used (default) camera calibration coordinate system.

Figure 6

 6 Figure 6.9: Right-hand rule for Cartesian 3D coordinate system.

Figure 6

 6 Figure 6.10: Visible side classification into 8 sub-classes according to the walking direction. (a) Example of trajectories in the ground floor projection plan. (b) The 8 classes of visible sides

  to the end of orientation change (see the last position of the "Person 4" in figure6.10 (a): the visible side is the front one while it may be classified as a front-left side if the orientation change is rough). Now it is possible to estimate the visible side of a given person on each acquired image, his/her global signature is replaced by a set of 8 sub-signatures, one per class of visible side. One or several visible side classes may have null signatures due to the fact that the person has never been observed by this side(s).

  17

  )where d(A, B) is the final distance used to perform the re-identification task and d(I A , I B ) is the dissimilarity distance between visual signatures, provided by eq. (6.14).Note that we are assigning the same importance to the visual signature comparison and to the real world distance. In future work, we are planning to find a better weight for real world distance, taking in account the calibration precision and the number of persons inside the perimeter (the more persons inside the perimeter are, the more the risque of imprecise real world distance computation is, due to increased risk of incorrect projections)Cameras without overlapped fields of view: if the monitored area plan is available, the velocity of tracked people and their location at the re-identification moment are used to remove all candidates who can not satisfy the spatio-temporal constraints: A person cannot be at two different locations at the same time. A person cannot travail a given distance in a given time if this displacement requires a superhuman velocity. We have used the maximum possible velocity as 44.72 km/h which was the maximum observed human velocity (reached by Usain Bolt in a 100 meters sprint). We know that we can decrease this maximum velocity because normal people, in usual monitored areas have less important velocity, but despite this high value, a large percentage of incoherent candidates have been filtered in our tests on ViCoMo sequences.

  Each time a new image of a person is acquired by a given camera, the different features (WH, MSCR, RHSP) are computed on this image and used to update the person signature as described before. At the same time, we extract a Hue histogram of this person image, we assign it to the considered person as additional information (it is not used for signature comparison) and we add it to a global histogram assigned to the camera network (no additional image browsing is needed, the WH bins which belong to the same hue value are summed). The camera network has its own normalized Hue histogram representing the frequency of all observed colors on all observed people. When a given person re-identification is required, the distance between the hue histogram of each candidate person and the global hue histogram (of the camera network) is computed. The max distance is kept as Dist color (max).

  ..) of by varying the classifier parameters: for example, in the case of SVM classifiers, the permissivity variation can be obtained by shifting the separation hyperplan more in the positive class direction (increasing FN and decreasing FP rates) or the negatvive class one (increasing FP and decreasing FN rates. SVM hyperplan separation is explained in sec 2.1.3.2). For a boosted classifier, this permissivity variation can be obtained by varying the classifier threshold which is used for decision after summing the weak classifier responses. For a boosted cascade of classifiers, the threshold of each cascade level can be varied too, obtaining a variation in the permissivity of the detector. Another method consists in changing the number of considered cascade levels. By decreasing the number of cascade levels, the permissivity of the cascade is increased too (due to the rejection mechanism of this kind of classifiers).Miss-detection and false-positive rates are given by the following equations: TN, FP and FN are the numbers of true positive, true negative, false positive and false negative respectively. N FN + N TP (the denominator of eq. (7.2))

  consists of two subsets of color images: a training set containing 2416 person annotations and 1218 person-free images and a test dataset with 1132 persons and 453 person-free images. The pedestrian annotations were scaled into a fixed size 64 × 128 window, which includes a margin of 16 pixels around the pedestrians. This dataset is quite challenging due to the various scenes, content, and persons appearence and poses (see some samples in figure 7.1).

Figure 7

 7 Figure 7.1: Samples from INRIA dataset. The first row consists of some examples of annotated people. The second row consists of some croped negative samples with the same size as annotated people (64 × 128 pixels).

  tations. As opposed to the INRIA data set, nonpedestrian annotations were selected by a preprocessing step from the negative samples, which match a pedestrian shape template based on the average Chamfer distance score. Both annotations were scaled into a fixed size 18 × 36 window, and pedestrian annotations include a margin of 2 pixels around.The data set is organized into three training and two test sets, each of them having 4,800 positive and 5,000 negative examples. The small size of the windows, combined with a carefully arranged negative set, makes detection on the Daimler-Chrysler data set extremely challenging. In addition, 3,600 person-free images with varying sizes between 360 × 288 and 640 × 480 are also supplied.

Figure 7 . 2 :

 72 Figure 7.2: Samples from DaimlerChrysler dataset. The first row consists of some examples of annotated people. The second row consists of some croped negative samples with the same size as annotated people (18 × 36 pixels)

Figure 7 . 3 :

 73 Figure 7.3: Samples from Caltech dataset.

Figure 7

 7 Figure 7.4: CAVIAR dataset: (a) INRIA Grenoble sequences. (b,c) Lisbon shopping center sequences (Front and Corridor views respectively).

Figure 7

 7 Figure 7.5: Results on INRIA dataset

Figure

  Figure7.5 (a) shows the performance comparison with[START_REF] Tuzel | [END_REF]] and[Yao 2008]

Figure 7

 7 Figure 7.5 (b) shows the performance comparison with state of the art approaches.

Figure 7

 7 Figure 7.6: Results on DaimlerChrysler dataset

Figure 7

 7 Figure 7.6 (b) shows the performance comparison with state of the art approaches.

Figure 7

 7 Figure 7.7: Results on Caltech dataset

Figure 7

 7 Figure 7.7 (b) shows the performance comparison with state of the art approaches.

Figure 7 . 9 :

 79 Figure 7.9: Results of the trained detector on INRIA dataset, applied on DaimlerChrysler, Caltech and CAVIAR datasets.

Figure 7 .

 7 Figure 7.10: Examples of detection results using our people detector trained on INRIA dataset, applied on several images that do not belong to INRIA dataset.

  Figure 7.11: PETS 2001 dataset.

  out a new structure contributing to an increase in the evaluation of video scene under-standing ; with the active participation of industrial and many research laboratories, such as French, European and International research centers.Project ETISEO focuses on the treatment and interpretation of videos involving pedestrians and (or) vehicles. The dataset contains 86 video clips. These sequences constitute a representative panel of different video surveillance areas. They merge indoor and outdoor scenes, corridors, streets, building entries, subway station, etc. They also mix different types of sensors and complexity levels. This makes this dataset relatively challenging and allows to evaluate the generecity of the proposed tracking algorithms.

Figure 7 .

 7 Figure 7.13: Samples from Caretaker dataset.

Figure 7

 7 Figure 7.14: Detailed results and comparison on two sequences from ETISEO dataset. First row: ETI-VS1-BE-18-C4 sequence results. Second row: ETI-VS1-MO-7-C1 sequence results. Bold text corresponds to the approach which provides the best results per metric. [Chau 2011](1) refers to an off-line learning based tracker; [Chau 2011](1) is and estimator-based tracker.

Figure 7

 7 Figure 7.15: Global results of the proposed approach on ETISEO, PETS 2001 and CAVIAR datasets.

Figure 7

 7 Figure 7.16: Comparative results on Caretaker sequence. [Chau 2011](1) refers to an off-line learning based tracker; [Chau 2011](2) is and estimator-based tracker.

Figure 7

 7 Figure 7.17: Comparative results on CAVIAR dataset. [Chau 2011](a) refers to the tracker with the proposed controller; [Chau 2011](b) is the tracker without the proposed controller.

Figure 7

 7 Figure 7.18: Cumulative matching characteristic (CMC) curve illustration. (a) Usual form of a CMC curve with with 25% of correct re-identification at the first rank and 80% of correct reidentification at the tenth rank; (b) The best (ideal) case with 100% of correct re-identification at the first rank.

Figure 7

 7 Figure 7.19: Corresponding nAUC value to the previous CMC curves (figure 7.18). (a) nAUC = 83.24% for the first case; (b) nAUC = 100% for the second case.

  VIPeR contains two views of 632 pedestrians. Each pair is made up of images of the same pedestrian taken from different cameras, under different viewpoint, pose and light conditions. All images are normalized to 128 × 48 pixels. The dataset contains pairs which viewpoint angle changes from 45 up to 180 degrees. Each pair is randomly split into two sets: CAM A and CAM B. The video was compressed before pairs of image extraction, so many compression artefacts are present on images.

Figure 7 .

 7 Figure 7.21: Samples from i-Lids-119 dataset

Figure 7 .

 7 Figure 7.22: Samples from i-Lids-MA dataset

Figure 7 .

 7 Figure 7.23: Samples from i-Lids-AA dataset

  7.1.2.4. For re-identification evaluation, only Lisbon shopping center sequences are used due to the availability of synchronized video sequences provided by two cameras with overlapping field of view (INRIA Grenoble sequences are provided by a unique camera).

Figure 7 .

 7 Figure 7.24: Samples from ETHZ dataset

Figure 7

 7 Figure 7.25: CMC curves obtained on VIPeR dataset.

Figure 7

 7 Figure 7.26: CMC curves obtained on iLids-119 dataset.

Figure 7

 7 Figure 7.27: CMC curves obtained on iLids-MA dataset.

  The processing time for color histogram is negligible in comparison with all other feature computation (a simple image browsing with the increase of histogram bins). The Maximally Stable Color Regions (MSCR) computation time for most images is decreased thanks to the apriori information concerning the number of required iterations and the initial clustering parameters, provided by the MCSR computation results on the first images of each visible side class. Finally, if we consider the SIFT feature computation as a part of the visual signature computation

Figure 7

 7 Figure 7.28: CMC curves obtained on iLids-AA datasets.

Figure 7

 7 Figure 7.29: CMC curves obtained on iLids-AA-RP datasets.

Figure 7

 7 Figure 7.30: CMC curves obtained on ETHZ datasets.

Figure 7

 7 Figure 7.31: CMC curves obtained on CAVIAR4REID dataset.

  situations like various scenes, content, and people appearance and poses (INRIA Person dataset), small people images, low resolution (DaimlerChrysler and CAVIAR datasets), low contrast (DaimlerChrysler dataset) and large amount of partial occlusions (Caltech dataset). In addition to the inability to use background subtraction and camera calibration for our people detector in most of these datasets (except for CAVIAR dataset) due to the single images nature or the moving camera acquisition system.The comparison result shows that the proposed detector have close performances to those of the best detectors of the state of the art. The remaining gap with the best results and with even better results is due to two main issues of our detector: First, the used covariance descriptors, even if they are very discriminant for regions with small dimensions (from 6 × 6 in our experiments on INRIA and Caltech dataset), smaller regions in the case of low resolution/noisy images alter the efficiency of this descriptor (observed especially in DaimlerChrysler dataset). The Second issue is related to the full-body type of our detector. Even if the training set of INRIA dataset and in a greater extent the one of Caltech dataset contains people with several levels of partial occlusions, the most miss-detected people in our experiments are due to the partially occluded people, especially for Caltech dataset. Body parts based detectors seems to be more adequate for this kind of situations.

  Optimization Method to Improve Cascade of Classifiers for People DetectorsThis method has been applied on a state of the art approach, improving its performances significantly while it speeds up both training and detection processing time. It consists in clustering negative data and training each cascade level by the largest remaining cluster. This step has three general and one specific benefit effects: firstly, the use of smaller subsets of negative data for each cascade training speeds up significantly the whole training process. Secondly, the clustering stage provides similar negative contents (with respect to the used information/features), specializing each cascade level to reject this type of content. Thirdly, the use of the largest remaining cluster ensures an optimized cascade level, selecting the most rejecting levels at the first positions, speeding up the detection time. Finally, and this is the most specific benefit, this clustering stage allows to reduce the effect of random selection of candidate weak classifiers when testing all possible candidate weak classifiers is not possible in reasonable time.8.1.1.2 A New Method for SIFT Feature Detection and Selection for Object TrackingIt consists in a more permissive SIFT feature detection in a first stage, providing a larger number of points on object to track, and in a reliability based selection of a subset of these points on a second stage, according to a grid subdivision of the object, the background subtraction and the SIFT feature robustness. This extension provides a good representation of the whole object to track and allows better tracking especially in partial occlusion cases. 8.1.1.3 An Hybrid Particle Weighting Method for SIFT Feature Particle Filtering

  Image Alignments Before Signature Computing for Multiple-shot CaseIn order to deal with part-to-part correct matching issue for multiple-shot signature computation, a fast image alignment algorithm is proposed. This algorithm is based on fast search of best matching score, using Lab color distance which is the closest color space to human perception. The use of camera calibration information to reduce searching scales speeds up the image alignment stage. Signatures which are computed after image alignment are more discriminative and more efficient.

table 1

 1

	.1

  Table 2.1;

	at time t-1	{	Weigted and sampled particles Resampled particles Particles before weigting (Predicted from time t-2)	weighting function
	at time t	{	Predicted particles (from t-1) Weigted and sampled particles	(eg. similarity measure between features)

Figure 2.18: Sampling Importance Resampling (SIR) process

Table 2

 2

.1: Summary of object tracking approaches, organized by the model information and the tracking technique.

Table 2

 2

		group	[Zheng 2009]
	context-aware	context	[Cai 2010]	[Iwama 2012]
		space-time	[Javed 2003]

.2: Summary of cited appearance-based approaches for people re-identification.

Table 2

 2

.3: Context-Aware approaches for people re-identification

table 5

 5 Table 5.1: Comparison between the cos function and a linear function for background quality transition encoding. The mean error of point position estimation (in pixels) is lower for cos function.

	.1).
	cos	linear

  If the computed velocity of the new tracked object during few frames does not fit the velocity Gaussian model of occluded object, the matching is avoided. Otherwise, a new matching score is computed

	by:
	score =	1 4	(Score

W + Score H + Score hue + Score velocity ) (5.20)

  14) as SIFTimportance score Score SIFT (p). Score SIFT (p) values are in [0, 1] since each SIFT feature reliability value is in [0, 1] (eq. 5.14). Otherwise, if SIFT features are detected directly on the last image of the person like for some evaluation datasets (see sec. 7.3) (due to the unavailability of tracking information), we use Score SIFT (p) = Score RHSP (p). SIFT features being texture descriptors, we assume that the importance may be quite similar.The final weights β WH , β MSCR , β RHSP and β SIFT of the used features for signature , α RHSP and α SIFT are the visible side classification coefficients. It means that depending on the availability or not of the visible side classification, and to the compared visible side classes if they are available, the feature weights are different.

	6.3.7.2 Feauture Weighting
	Now the importance of each information type measured for a given person p, the
	used features (WH, MSCR, RHSP, SIFT) have the following intermediate normalized
	weights:
	w WH (p) =	2 • Score color (p) 2 • Score color (p) + Score RHSP (p) + Score SIFT (p)	(6.20)
	w MSCR (p) =	2 • Score color (p) 2 • Score color (p) + Score RHSP (p) + Score SIFT (p)	(6.21)
	w RHSP (p) =		Score RHSP (p) 2 • Score color (p) + Score RHSP (p) + Score SIFT (p)	(6.22)
	w SIFT (p) =		Score SIFT (p) 2 • Score color (p) + Score RHSP (p) + Score SIFT (p)	(6.23)
	with w WH (p) + w MSCR (p) + w RHSP (p) + w SIFTH (p) = 1.0.
	comparison are given by:
			β WH (p) = α color • w WH (p)	(6.24)
			β MCSR (p) = α color • w MCSR (p)	(6.25)
			β RHSP (p) = α RHSP • w RHSP (p)	(6.26)
			β SIFT (p) = α SIFT • w SIFT (p)	(6.27)
	where α color

  approaches.Both approaches use color histograms as a feature representation; LMNN-R is based on metric learning while PS use body-part segmentation method.

	Approach	1st rank (%) 5th rank (%) nAUC (%)
	PS [Cheng 2011]	21.93	44.73	93.60
	Our approach	20.25	40.98	91.45
	LMNN-R [Dikmen 2011]	20.22	55.27	-
	SDALF [Farenzena 2010]	20.12	39.06	89.90
	PRDC [Zheng 2011]	16.03	38.72	-
	SCR [Bak 2011]	10.13	24.17	80.75

Table 7

 7

.1: Detailed results on VIPeR dataset.

  Table 7.5: Detailed results on iLids-AA-RP dataset.

		1st rank (%) 5th rank (%) nAUC (%)
	Our approach	80.00	90.00	80.76
	MRCG [Bak 2011]	70.00	80.00	74.96
	SDALF [Farenzena 2010]	60.00	80.00	71.36

Table 7 .

 7 Table 7.6: Detailed results on ETHZ Sequence# 1 (83 individuals). 7: Detailed results on ETHZ Sequence# 2 (35 individuals).

	Approach	1st rank (%) 5th rank (%) nAUC (%)
	CPS [Cheng 2011]	98.32	99.66	-
	LCP [Bak 2011]	95.78	97.02	-
	Our approach	93.98	97.59	99.97
	SDALF [Farenzena 2010]	90.29	93.58	98.76
	HPE [Bazzani 2010]	84.81	93.80	-
	PLS [Schwartz 2009]	79.61	88.90	-
	Approach	1st rank (%) 5th rank (%) nAUC (%)
	CPS [Cheng 2011]	95.74	100	99.93
	LCP [Bak 2011]	94.30	95.81	-
	Our approach	94.29	100	99.65
	SDALF [Farenzena 2010]	87.86	92.03	95.85
	HPE [Bazzani 2010]	81.07	92.74	-
	PLS [Schwartz 2009]	73.77	83.92	-
	Approach	1st rank (%) 5th rank (%) nAUC (%)
	CPS [Cheng 2011]	97.00	100	99.94
	LCP [Bak 2011]	96.53	100	99.82
	Our approach	96.43	100	99.67
	SDALF [Farenzena 2010]	94.15	97.29	98.96
	HPE [Bazzani 2010]	87.02	96.41	-
	PLS [Schwartz 2009]	76.71	85.27	-

Table 7 .

 7 8: Detailed results on ETHZ Sequence# 3 (28 individuals).

1.3 Issues in People Detection, Tracking and Re-identification

6.4 Conclusion

Chapter 8: Conclusion And Future Work extrapolated information if their zones are directly on the people (like backpack viewed from behind).

The first method provides "a cascade" of a unique strong classifier containing three weak classifiers at least (the number can be larger due to the possible combinations) (figure 4.8(a)). The second method provides a cascade with three levels (figure 4.8(b)), each level containing one weak classifier corresponding to one case (textured and vertical shapes only, circular shape at the top of vertical shape, separation between legs). Suppose now that we have to perform a people detection on a large image which contains only sky, a low textured road, some vertical barriers and some lampposts. Both cascades will provide equivalent detection performances, but the second one will be faster. This is because most of tested windows (sky and road) are rejected after evaluating only one covariance descriptor (the one of the first cascade level), while the second classifier cascade needs to evaluate three (or more) covariance descriptors for each tested window.

We propose an approach using a smaller subset of negatives at each cascade level training to make it faster. Our approach provides shorter cascade with smaller classifiers on average (figure 4.6 (c)) in comparison with the [START_REF] Tuzel | [END_REF]] one (figure 4.6 (a))

speeding up the detection process. At the same time, the experimental results show that our approach provides slightly better detection performance than the original one.

The idea consists in regrouping negative samples per groups containing similar content in terms of covariance information, and in training each cascade level with one group of similar samples.

The previously described Logitboost algorithm achieves characterization of people against a group of negative samples faster when these negative samples are more similar. inter-images distances. This value is computed as the sum of all distances between all pairs of images belonging to the considered cluster.

The training is now done by using the largest cluster of most similar negative images for each trained cascade level. Once a cascade level is trained using the selected cluster, the new cascade is applied to all the remaining negative samples, those used for training and the others which are not in the selected cluster. The next cluster is selected using the same method described below, applied on the remaining negative images.

Note that for a given cascade level, we observe that 80% to 95% of the negatives from the used cluster are correctly classified and removed and a small part of unused negative images (not belonging to the used cluster) also.

Chapter 6: Fast People Re-Identification

In the case of availability of visible side classifications and depending to which of the three mentioned cases in sec 6.3.5 the re-identification query belongs, the weights are assigned as follow:

Comparison of the same visible side classes:

In this case, all the features have the same importance from visible side point of view. The difference between their importance is then exclusively dictated by the intermediate weights (eq. 6.20, 6.21, 6.22 and 6.23) which are directly related to the information importance measures. The coefficients are then: α color = 1.0, α RHSP = 1.0 and α SIFT = 1.0

Comparison of the adjacent visible side classes:

In this case, Color being more global descriptors than RHSP and SIFT (which are local descriptors), the color features (WH and MSCR) take more important weights. We have decided to keep the same proportion between these two kinds of information (color and texture) as in the baseline approach (sec. 6.1.3), i.e.

80% of weight to color based features (WH and MSCRs) and 20% of weight to texture based feature (RHSPs, even if their characterisation was exclusively done by color, their selection are texture-based approach).

We assign the coefficient as follow: α color = 0.4, α RHSP = 0.05 and α SIFT = 0.15,

We assign a more important coefficient to SIFT features in comparison with RHSP due to their good invariance with respect to affine transformation (object rotation)

( [START_REF] Lowe | [END_REF]]) in comparison with covariance matrices. In fact, both SIFT features and RHSP patches may still be visible on images of adjacent classes of visible sides but with some affine transformation.

Note that in this case, another normalization step, as in equations 6.20, 6.21, 6.22 and 6.23, is required for β WH , β MSCR , β RHSP and β SIFT weights since their sum does not equal 1.0 in general case.

Comparison of signatures without common or adjacent visible side classes:

In this case, we completely avoid the use of SIFT features and RHSP for visual signature comparison, since they are local descriptors and then, the same features are not visible on the two considered signature, by setting their coefficient as α RHSP = 0 and α SIFT = 0. Color coefficient is set as α color = 1.0, and as for the previous case, β WH , β MSCR are normalized again to have β WH + β MSCR = 1.0.

In the case of unavailability of visible side classification, we do not have any information to decide if local features (SIFT and RHSP) have to be considered or not, and Chapter 8: Conclusion And Future Work

We can cite the two main examples which are "Assisted tracking" and "People retrieval aposteriori". In fact, assisted tracking is a kind of partial-autonomous system which tries to track people through the camera network, but which can be guided and corrected by the video operator by simple actions such us clicking on the person of interest. For the people a-posteriori retrieval, the contribution of re-identification algorithm may be more important by greatly reducing the number of candidates that human operators or enforcement officiers have to manually check, and this, by proposing only people in the 10% first ranks for example.

Future Work

Short-term Perspectives

People Detection

Another kind or part-based people detector for partial occlusion management

We have highlighted the fact that partial occlusion cases are better managed by body part based detectors. We have avoided this kind of approach due to the higher required processing time and to the additional spatial positioning reasoning. Noting that partial occlusions mainly occur uniformly from one of the four sides of a person (upper side, lower side, right side, left side), and instead of training a large number of body part detectors (for head, shoulders, hands, arms, legs, etc.), another way to explore this challenge may be to train only four detectors on the four global cited sides, with different levels of occlusion (for example, upper occlusion which ends at different levels, starting occlusion from waist to knee for lower body part detector, etc.). This possible solution has two main advantages: the number of parts to consider is lower (only 4) and the spatial positioning reasoning is easier due to the considered parts.

Evaluating genericity of people detectors by merging several datasets

To have more generic people detectors, it may be interesting to train the detectors on enlarged datasets containing many datasets. A preprocessing step to resize all images on the same size is necessary, and the selected size has to produce the lowest amount of information loss and image alteration. This approach may show how a given detector is able to integrate images characteristics (as resolution, noise, people size, acquisition point of view, etc.) as full part features in addition to classical features (color, textures, etc.).

Chapter 8: Conclusion And Future Work to automatically select the corresponding detector which is supposed to provide the best results. This may solve the genericity lack of current people detection systems.

Hybrid and hierarchical use of several features

Even if covariance descriptors have proven their effectiveness in most of cases, their required heavy processing time leads us to consider a smarter way to use them. Some simple cases may be easily managed with faster features like LBP or Haar-Like features.

We do not believe that the concatenation of features in the same strong classifiers is the best way to use other types of information. Quite the contrary, we believe that training each cascade level using a unique type of features, by taking care of training the first cascade levels using faster features for simple negatives fast rejection, and use more complex features (like covariance descriptor) at the end of the cascade to reject more complex negatives may be a better way to use multiple features approaches. This may be justified by two reasons: generally, most of negative regions in images are not complex cases (road, walls, sky, cars, etc.). Many fast and simple features may easily be used to reject these simple cases. The second reason is related to our desire to specialize each cascade level to reject a given kind of information.

Mono-camera Object Tracking

High level, long term on-line tracking controller

As alternative to off-line learning, which cannot be considered for the addressed requirements, it may be interesting to investigate the on-line tracking controller solutions.

In fact, in our tracking algorithm, once a decision is taken by our data association framework concerning the temporal links between objects of the previous frame and those of the current frame, this decision is no longer questioned and the tracking continues even if some links are incorrect. Using long-term control to check the coherency of trajectories provided by our tracker may help to correct bad tracks and to update the state of our tracking for better incoming results.

Multiple feature based tracking, with smart and alternating features use

Generally, the more discriminative a given feature is, the more processing time it requires to be computed. The scene complexity is not constant: sometimes people are isolated while in other times the scene is crowed. It is even possible to have both cases in the same scene at the same time, with crowded zones and relatively empty ones. Starting from these two observations, it may be interesting to find a way to have a multifeature tracker which switches easily from a feature to another one without tracking interruption and to manage this tracker under a high level controller which may predict