Paul Blondel
email: paul.blondel@u-picardie.fr

Alex Potelle

Claude Pégard

Rogelio Lozano
email: rogelio.lozano@hds.utc.fr

Fast and viewpoint robust human detection for SAR operations

There are many advantages in using UAVs for search and rescue operations. However, detecting people from a UAV remains a challenge: the embedded detector has to be fast enough and viewpoint robust to detect people in a flexible manner from aerial views. In this paper we propose a processing pipeline to 1) reduce the search space using infrared images and to 2) detect people whatever the roll and pitch angles of the UAV's acquisition system. We tested our approach on a multimodal aerial view dataset and showed that it outperforms the Integral Channel Features (ICF) detector in this context. Moreover, this approach allows real-time compatible detection.

I. INTRODUCTION

UAVs are interesting tools for finding people in distress in complex and/or in large environments because of their maneuverability, their rapidity of deployment and because they can be intelligently controlled. They can work automatically and in swarm in order to shrink the search time; but for this, each UAV should have its own detection system. Unfortunately, automatically detecting people from a UAV is not an easy task. Numerous constraints have to be taken into account in this context and the detection system has to be designed accordingly.

A. Existing work in human detection

Detection methods based on background substraction are not well suited when the camera undergoes complex movements or when people are not moving. The detection process should only rely on the visual information contained in one frame. Monolithic and part-based detectors fit this condition.

1) Monolithic detection: Monolithic detectors search for monolithic parts of the image looking like people. Gavrila et al proposed to use a hierachy of human contour templates obtained using training [START_REF] Gavrila | Shape-based pedestrian detection and tracking[END_REF]. This hierachy, used together with the chamfer matching algorithm, allows the detection of people in images. But more discriminative methods based on powerful descriptors have also been developed. The visual information is locally extracted and collected. Finally the information is compared to a general model of people with a classification algorithm. Papageroriou et al were among the first to propose this pipeline [START_REF] Papageoriou | A Trainable System for Object Detection[END_REF]. They used wavelet descriptors, a slidingwindow method to exhaustively scan the image and a SVM classifier. Many of current detectors are still based on this approach. Viola et al based their work on the work of Papageoriou et al [START_REF] Papageoriou | A Trainable System for Object Detection[END_REF]. They used integral images and a cascade classifier to speed up the computation of the Haar-like wavelet Fig. 1. First row: results obtained with our method, second row: results obtained with the ICF. Our method is less sensitive to changes of shape and angle.

features and reach real-time performance for face detection [START_REF] Viola | Rapid Object Detection using a Boosted Cascade of Simple Features[END_REF]. The Histogram of Oriented Gradients (HOG) detector of Dalal and Triggs [4] is an efficient human detector using a variant of the very well-known and quite efficient SIFT descriptor [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF]. Visual information is extracted using SIFT-like descriptors over a sliding-window. All the information is classified using a linear SVM classifier trained on images of people. The SIFTlike HOG descriptor still remains very competitive for object detection. Some detectors combine multiple descriptors, image features and/or information sources to increase the detection rate. Wojek et al showed that combining HOG, Haar-like descriptors, shaplets and the shape context outperform the HOG detector alone [START_REF] Wojek | A Performance Evaluation of Single and Multi-feature People Detection[END_REF]. Dollar et al proposed a mix between Viola et al's detector and the HOG detector [START_REF] Dollár | Integral Channel Features[END_REF]. This detector computes simple rectangular features on integral images of different channels: L,U,V, gradient magnitude and six "HOG channels". The classification is performed using a fast softcascade classifier.

2) Multiple parts detection: Instead of considering the human body as one monolithic part, some detectors consider it as a set of parts. Felzsenzwald et al proposed a method to detect people by fragments and re-build human models by using a pictural structure reprensentation [START_REF] Felzenszwalb | Pictorial Structures for Object Recognition[END_REF]. Each part of the human model is separately learned. An incorrect labelling of the fragments could decrease the performance of the detector [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF]. That is why Felzenszalb et al introduced a detector using a new classifier: the latent SVM classifier [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF]. With this classifier the most discriminative information is selected during the training to produce a more robust detection.

B. Existing work in human detection from a UAV

Detecting people is difficult and it becomes more difficult in a UAV context. Most human detectors focus on detecting upright people at nearby distances and from a more or less invariant viewpoint. The current two main applications of human detection is the security monitoring and the driving assistance. Until now little work has been done on detecting humans from a UAV. Unlike a pedestrian view, an UAV view is more complex to manage because the UAV undergoes pitching and rolling rotations. People are also on average further from the camera in this context.

Gaszczak et al proposed to use both thermal and visible imagery to better detect people and vehicles [START_REF] Gszczak | Real-time People and Vehicle Detection from UAV Imagery[END_REF]. Features extracted on thermal and visible imagery are fused together to boost the confidence level of detection. The thermal camera is used for extracting Haar-like features while the optical camera is used for a contour shape analysis as a secondary confirmation to better confirm the detection. This method permits to detect upright people at a distance of about 160m using a fixed camera pitch rotation of minus 45 degrees and in real-time. This method does not seem flexible enough for detecting people closer to the UAV.

Rudol et al also use thermal and visible imagery but in a pipeline way [START_REF] Rudol | Human Body Detection and Geolocalization for UAV Search and Rescue Missions Using Color and Thermal Imagery[END_REF]. They first identify high temperature regions from the thermal image and they reject the regions not fitting a specific ellipse. The corresponding regions are then analyzed in the visible spectrum using a relaxed Haar-like detector. Upright and seated people can be detected with this method. However, the thermal imagery can easily become very tricky to analyze with this method when the UAV is too close and the information becomes too noisy.

Reilly et al have a different approach [START_REF] Reilly | Geometric constraints for human detection in aerial imagery[END_REF]. They use people's shadows as a key clue to detect and localize people. But strong assumptions on weather conditions have to be made with this technique.

Andriluka et al evaluated various detection methods for detecting victims at nearby distances [START_REF] Andriluka | Vision based victim detection from unmanned aerial vehicles[END_REF]. They showed partbased detectors are better suited for victim detection from a UAV because they natively take into account the articulation of the human body. The authors propose the use of complementary information using several detectors and inertial sensor data to obtain a better detection rate. However, part-based detection is a slow process [START_REF] Felzenszwalb | Pictorial Structures for Object Recognition[END_REF] and this seems not suitable for detecting people too far from the camera.

C. Project context

This work is part of the French regional project SEARCH whose goal is to develop a system using several UAVs to rapidly find and rescue people in distress in the Somme estuary in northern France. The rising tide often trap walkers, bringing with it a risk of death from drowning. The current surveillance system employs helicopters to perform prevention and rescue missions. Unfortunately, this system makes use heavy equipment and is very expensive. Using a fleet of UAVs would be an interesting alternative for both the tax payers and the people in distress. This work is mainly about the development of a suitable human detection system for the SEARCH project.

The search and rescue context of the Somme estuary is the following: the environment is uncluttered, people may be close or far from the UAV (we consider a distance range of 15 to 50m), people may be walking or standing upright and the illumination may change over time and may not be the same everywhere. The use of UAVs adds some more constraints. The embedded camera of the UAV moves in a 3D world the detection system should thus be robust to the acquisition system's viewpoint. Moreover, a sufficient reactivness is required by the system in order for it to be usable in tortuous flights.

D. Content of the paper

The goal of this work is to design a detection system for addressing all of the following identified constraints: illumination robustness, detection of moving and stationary people, real-time compatibility, distance robustness and viewpoint robustness. The proposed approach is described in details in this paper.

Section II presents our approach for automatically detecting people from a UAV in open and natural environments. Section III concerns the conducted tests and the results obtained in a sequence taken in the Bay of Somme. This section also talks about the hardware. The results are discribed in this section.

II. PROPOSED APPROACH

A specific processing pipeline was adopted in order to obtain better performance: it allows faster computation and a greater viewpoint robustness (Fig. 2). At first visible and thermal images are extracted during the acquisition phase (1), a pre-processing phase follows immediately (2), the thermal images are analyzed to greatly reduce the search space (3), the spotted areas are fully analyzed in the visible light spectrum by a viewpoint robust detector to detect people (4) and to finish, the detected people are looped in the tracker [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF]. 1), a pre-processing of the data is following this step (2), a search space reduction is performed in the thermal images (3), the corresponding visible ROIs are analyzed by our viewpoint robust human detector (4) and detections are looped in the tracker [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF] This paper focuses on stage (1), ( 2), ( 3) and (4) of the proposed pipeline. The acquisition system is composed of a 3. Layout of the stereo acquisition system. C 1 and C 2 are respectively camera 1 and 2, Π 1 and Π 2 are the image planes of respectively camera 1 and 2. P is a person's location. R: rotation between the two cameras, T: translation between the two cameras. C1 is the thermal camera and C2 the visible light spectrum camera.

Π 1 Π 2 P u 1 v 1 u 2 v 2 x 1 z 1 y 1 x 2 z 2 y 2 C 2 C 1 R+T d Fig.
visible and a thermal camera. The two cameras are set one beside the other so that the two axis of the cameras are parallel and at the same height.

A. Stereo acquisition (1)

This step of the pipeline is dedicated to the registration of the visible and the thermal images: the pixels of both the cameras are spatially aligned. The goal of this step is to benefit from the discriminatory nature of thermal images and the information richness of visible images. Some knowledge about the intrinsic parameters of the cameras and the geometric relationship of the cameras are required to align the two modalities.

Because the objects of the scene are mostly far from the cameras and because the baseline is short (Fig. 3) it has been decided that the impact of the baseline may be neglected (the closest people to detect are at about 15m from the UAV). Thus, for matching the pixels of the two modalities we use the infinite homography [START_REF] Krotosky | Mutual information based registration of multimodal stereo videos for person tracking[END_REF] (equation 1 and 2). K 1 and K 2 are the matrices of the intrinsic parameters of respectively camera 1 and 2. R is the rotational matrix between the two cameras and T is the translation between the two cameras.

H ∞ = K 2 × R × K -1 1 (1) u 2 v 2 w 2 = H ∞ × u 1 v 1 1 (2)
This technique is less time-consuming but also less accurate than the full rectification of the stereo pair with respect to a known distance. Besides, we found this assumption accurate enough for our case. The regions of interest extracted from the infrared images do not have to exactly match people locations but rather match coarse areas around people. Indeed, these coarse areas and their close neighborhood are finely analyzed afterwards. Tab.I shows the average pixel error between the two cameras (along u 1 or u 2 ) and with respect to the distance to the persons. For a better accuracy, these offsets can be roughly corrected if the distance between the camera and the persons can be known at run-time. We found it better to treat the thermal images using a saliency algorithm in order to enhance the visibility of the warmest areas. We noticed that it improves considerably the job of the search space reduction: thanks to that the reduction is cleaner and more robust. It has been decided to use the Achanta's saliency algorithm [START_REF] Achanta | Saliency Detection Using Maximum Symmetric Surround[END_REF] which is fast and compute outstanding saliency maps. The algorithm has been slightly modified for the thermal case: only the L component of the Lab color space is considered.

C. Search space reduction (3)

The infrared map of a scene permits to better distinguish people from other objects of the surrounding environment. Indeed, human beings are natural Long Wave Infrared sources. But, it is not appropriate to only use this information to detect people because many objects of the scene can emit infrared on certain conditions (warm wall, stones, leafs of a tree, etc.). However, the search space can be greatly reduced for further treatments by simply analyzing these thermal images for extracting some regions of interest.

The special segmentation algorithm of San-Biagio and al [START_REF] San-Biagio | Recursive segmentation based on higher order statistics in thermal imaging pedestrian detection[END_REF] was implemented to reduce the search space. This algorithm is fast and spot relatively well warm areas of the scene that might correspond to human beings. It works recursively by extracting ROIs and refining them. It stopes when all ROIs do not change size after two iterations. The first ROI is the entire image. For each recursion the ROI is analyzed in the following way: the number of pixels with a brightness bigger than T hr step is stored for each row. The same process is repeated for each column. The boundaries of the new ROIs within this ROI are found by thresholding the previously stored numbers for the rows and the columns. The thresholds for the rows (T hr pixel,row ) and for the columns (T hr pixel,col ) depend on the size of the image and on some constants [START_REF] San-Biagio | Recursive segmentation based on higher order statistics in thermal imaging pedestrian detection[END_REF]. T hr step is refined after each recursion as showed in equation 3 and 4. w1, w2, w3 and T hr Sk are parameters.

T hr step = w1 × T hr step-1 + (1 -w1) × T hr ROI (3)
T hr ROI =w2 × maxGrayLevel(ROI)+ w3 × meanGrayLevel(ROI)+ (1 -w2 -w3) × T hr Sk (4)

D. Detection (4)

This is the most important step of the detection pipeline. The regions of interest extracted on the thermal images are analyzed in the visible light spectrum. This step is divided into two sub-steps: 1) random generation of candidate windows in and around the ROIs (Fig. 4.4) and 2) analysis of each of the candidate windows using a supervised detector (Fig. 4.5). For the first sub-step, the candidate windows are generated according to the three following rules: the center of the window is randomly chosen within the ROI, the size of the window is randomly chosen between a min and a max scale (0.5 and 1.5 respectively) and the number of windows to generate for each ROI depends on the surface of the ROI. For the second sub-step, the candidate windows are analyzed by our viewpoint robust detector: the Pitch and Roll-trained Detector (PRD). The PRD is based on the Integral Channel Features (ICF) detector of Dollar et al [START_REF] Dollár | Integral Channel Features[END_REF]. Our detector requires a specific training phase with correctly labeled training data. The detection phase and the training phase are more detailed in the following sections.

1) Detection phase: During the detection phase the PRD analyzes the visual content of the circular window. Local visual features are extracted at different places of the circular window using integral images of ten different channels, which are: L, U, V, gradient magnitude and six "HOG channels". A visual feature is simply the sum of the pixels contained within a rectangle and associated to one of the ten channels. These visual features are extracted from a resized version of the circular window (resized with a radius of 64 pixels). A person is detected in a window if the visual features match the human model previously learned during the training phase.

A coarse-to-fine approach is adopted to speed the classification: the soft-cascade. For greater performance Dollar et al [START_REF] Dollár | The Fastest Pedestrian Detector in the West[END_REF] propose to approximate the features between image scales to speed up the detection, they named this detector: The Fastest Pedestrian Detector in the West (FPDW) [START_REF] Dollár | The Fastest Pedestrian Detector in the West[END_REF]. This same technique can also be used with our PRD for even greater speed performance.

2) Training phase:

The training has been thought to deal with the change of human appearance occuring when the UAV's camera looks at people on the ground. The impact of the camera angles on the human appearance are the followings: the roll angle tends to rotate the shape of people and the pitch angle tends to change the shape of people (Fig. 5 and6). When the visual changes are too important, they cannot be managed by a detector with a classic design whose aim is to detect people in a pedestrian view. In order to overcome this problem a more general human model is trained by taking into account both the effect of the roll and the pitch during the training. A specific training dataset was built: the GMVRT-v2 training dataset 1 . This dataset contains 3846 images of people taken at different pitch angles of the camera (Fig. 7). This dataset is loaded in memory five times. Subsets of all the training images are rotated every 5deg so that all the images are spread from -90deg to 90deg. This step is necessary to minimize the effet of the roll on human apperance at training. A modified version of the Cluster Boosting Tree (CBT) algorithm [START_REF] Wu | Cluster Boosted Tree Classifier for Multi-View, Multi-Pose Object Detection[END_REF] is used to learn the multiple aspects of the human class in relation to the viewpoint. This algorithm is based on AdaBoost, which is a very famous learning method in pattern recognition. AdaBoost [START_REF] Schapire | Improved Boosting Algorithms Using Confidence-rated Predictions[END_REF] selects the best discriminant combination of visual features on positive and negative training images. Alg.1 presents our modified version of the CBT, see [START_REF] Wu | Cluster Boosted Tree Classifier for Multi-View, Multi-Pose Object Detection[END_REF] for the original implementation. Our version is lighter and proved to be more efficient for our needs. Line 1 is the angular spreading of the data. Lines 7 and 8 (Alg.1) are typical AdaBoost procedures, except that S k + is a subset of S 0 + for k>0. Line 9 is the condition to trigger the clusterization: the classification power (h(t, k).Z) of the three latest trained weak-classifiers are compared to θ Z . Lines 12 and 13 are retraining procedures of the previously trained weak-classifiers. We found it better to set θ Z to 0.98 and to authorize as many clusterizations as possible. Dollar and al recommend to use 30.000 random candidate features at training to build the weak-classifiers [START_REF] Dollár | Integral Channel Features[END_REF]. 45.000 candidate features were generated in order to keep a relatively similar density of candidates because the surface of the circular window of the PRD is 1.5 bigger than the surface of the classic detection window (64x128).

III. TESTS AND RESULTS

A. Hardware

We built our own stereoscopic system in order to conduct the tests. This system is composed of an infrared and a classic visible camera. The optical axes of the two cameras are at the same height and the baseline between the two cameras is exactly 5cm. The infrared camera is a Flir Tau2 running at 7 fps in VGA mode. The visible camera is a GoPro 3 HD (we took into account the distortion effect of the camera). Each camera is connected to a grabber. The two grabbers have the same specifications. The whole stereoscopic system weights about 470gr. The stereoscopic system is embedded in a Pelican quadrotor UAV (Fig. 8).

B. Tests

In order to build the test datasets we flied the UAV in natural and open environments. Images of people were taken for different angular configurations of the acquisition system: the UAV flied over and around the persons. Two datasets were built: a first one containing only visible images taken from very challenging viewpoints of the camera (AerialTest1 1 ), and a second dataset containing spatially and temporally syncronized visible and infrared images with some challenging viewpoints (AerialTest2 1 ). In the second case, we flied the UAV at a lower altitude. AerialTest1 is used to demonstrate the robustness of our pipeline's core detector: the PRD. AerialTest2 is used to demonstrate the feasibility of our approach and to compare the performance of it to the performance of the classic approach (ICF).

1) Test 1: general performance of our core detector in a challenging dataset: The ICF detector fails to succeed in most cases (Fig. 11): the miss-rate is constant and very high (pratically 1). Better performance are obtained with the Pitch and Roll-trained Detector (PRD). The PRD is much less sensitive to the challenging viewpoints of AerialTest1. The shape of the PRD's ROC curve is very similar to ROC curves of pedestrian detectors tested in pedestrian scenarios [START_REF] Dollar | Pedestrian detection: A benchmark[END_REF]. 2) Test 2: space reduction efficiency using infrared images: Infrared images are sometimes too bright, and therefore difficult to analyze: this leads to a non-optimal space reduction. We improved our space reduction by taking into account the saliency (Tab.II): the mean is about twice smaller. The reduction is more robust to changes, as well: the standard deviation (Sdev) is about four times smaller.

3) Test 3: detection performance of our detection pipeline: Our pipeline has better general performance than the ICF detector on AerialTest2 (Fig. 12). However, the ICF detector manages to detect the easiest cases of AerialTest2, when view is close to the pedestrian view (contrary to the cases showed in Fig. 1).

4) Test 4: computation time:

The PRD alone is 1.75 times slower than the ICF (Tab.III). The computation time of our pipeline is equivalent to the computation time of the ICF. This can be further improved by approximating features between octaves [START_REF] Dollár | The Fastest Pedestrian Detector in the West[END_REF] and/or by performing the low-level pixel treatments (such as saliency map computation) using FPGAs.

IV. CONCLUSION

In this paper, we proposed a new detection pipeline for viewpoint robust and fast human detection in search and rescue operations. Our approach combines the use of a viewpoint robust detector (PRD) and an accurate search space reduction technique. We showed that our detection pipeline outperforms traditional human detection approaches such as the Integral Channel Features (ICF) detector: one of the top-performing human detectors in pedestrian contexts.

The main contributions of this work are: 1) a search space reduction technique using syncronized infrared images optimized to lighten the work of the core detector which treats the corresponding areas but in the visible light spectrum, and 2) a viewpoint robust detector (PRD) to detect people whatever the roll and pitch angles of the acquisition system in a UAV context.

The next objective is to improve the general performance of the core detector by also taking into account the shape information available in IR images. It is also wished to improve the speed of our detection pipeline in order to improve its reactiveness even further. Test are planned to conduct tests with different types of UAVs in order to show the flexibility of our approach.

Fig. 2 .

 2 Fig.2. General pipeline of the onboard detection system. At first visible and thermal images are obtained and registered (1), a pre-processing of the data is following this step (2), a search space reduction is performed in the thermal images (3), the corresponding visible ROIs are analyzed by our viewpoint robust human detector (4) and detections are looped in the tracker[START_REF] Lowe | Object recognition from local scale-invariant features[END_REF]

Fig. 4 . 1 )

 41 Fig. 4. 1) stereo-acquisition: input visible en infrared images, 2) preprocessing: computation of the saliency map of the infrared image, 3) search space reduction: segmentation of the "warmest areas" of the saliency map, 3) search space reduction and detector: transposition of the ROIs in the visible image and random generation of candidate windows around the ROIs, 4) detector: treatment of the candidate windows by our PRD and Non-Maximum Suppression treatment.

Fig. 5 .

 5 Fig. 5. C: optical center of the camera, 1) elevation and azimuth: position angles. 2) camera's angles.

Fig. 6 .

 6 Fig. 6. Examples of people images with different roll and pitch angles.

Fig. 7 .

 7 Fig. 7. Examples of GMVRT-v2 images.

2 c ← 1 ; 3 for k ← 0 to c do 4 reset 10 h 13 ] 19 Algorithm 1 :

 21341013191 input : GMVRT-v2 training dataset 1 output: viewpoint robust classifier several angular spreading of the data; 1 extracting all candidate features for all the data; default weights of S k + and S -; 5 for t ← tinit(k) to T do 6 build best weak-classifier h(k,t); 7 update weights of S k + and S -; 8 if h(k,t).Z > θ Z and h(k,t-1).Z > θ Z and 9 h(k,t-2).Z > θ Z then split S k + into S k + and S c+1 + ; (c+1,t') = h(k,t'), ∀t' ∈ [ 0, t ] ; 11 retrain weak-classifiers h(k,t'), ∀t' ∈ [ 0, t ]12 with S k + and S -; retrain weak-classifiers h(c+1,t'), ∀t' ∈ [ 0, t with S c+1 + and S -; tinit(c+1) = t; 0, c ] compute the soft-cascade for channel k; Our CBT implementation. c: number of clusterizations, k: index of the cluster (or channel), T: maximum number of weak-classifiers, h(k,t): weak-classifier number t of channel k, tinit(k): starting index for cluster k, θ Z : clustering critera, S k + : cluster k of positive image, S -: all the negative images.

Fig. 8 .

 8 Fig.8. The Pelican UAV with the embedded acquisition system.

Fig. 9 .

 9 Fig. 9. Examples of images from AerialTest1.

Fig. 10 .

 10 Fig. 10. Examples of images from AerialTest2.

Fig. 11 .

 11 Fig. 11. ROC curves obtained by testing the ICF and the PRD detectors on AerialTest1. The ICF detector clearly fails on this dataset. On the contrary, the PRD is not affected by the challenging viewpoints of AerialTest1.

TABLE I .

 I AVERAGE PIXEL ERROR ALONG THE u AXES WITH RESPECT

	TO THE DISTANCE FROM THE CAMERA
	d (m)	10	20	30	40	50
	Average pixel error (pixels)	5.998	3.04	2.03	1.53	1.22
	B. Pre-processing (2)

TABLE II .

 II REDUCTION EFFICIENCY Fig.12. ROC curves obtained by testing the ICF and our detection pipeline on AerialTest2. Better detection performance are obtained with our detection pipeline. However, the ICF still manages to detect the easiest cases of AerialTest2.

	Reduction Technique	Min (%)	Max (%)	Mean (%)	Sdev (%)
	IR+Segmentation	0	0.6185	0.05246	0.0811
	IR+Saliency+Segmentation	0.001	0.1009	0.0266	0.0203

TABLE III .

 III COMPUTATION TIME.

	Detection Method	ICF	PRD	Our Pipeline
	Computation time	T	1.75×T	1.05×T

http://mis.u-picardie.fr/ ∼ p-blondel/papers/data


Paul Blondel

Alex Potelle

Claude Pégard

Rogelio Lozano
email: rogelio.lozano@hds.utc.fr

Fast and viewpoint robust human detection in uncluttered environments

Keywords: human detection, machine learning, multiviewpoint, viewpoint robust, supervised training

Human detection is a very popular field of computer vision. Few works propose a solution for detecting people whatever the camera's viewpoint such as for UAV applications. In this context even state-of-the-art detectors can fail to detect people. We found that the Integral Channel Features detector (ICF) is inoperant in such a context. In this paper, we propose an approach to still benefit from the assets of the ICF while considerably extending the angular robustness during the detection. The main contributions of this work are: 1) a new framework based on the Cluster Boosting Tree and the ICF detector for viewpoint robust human detection, 2) a new training dataset for taking into account the human shape modifications occuring when the pitch angle of the camera changes. We showed that our detector (the PRD) is superior to the ICF for detecting people from complex viewpoints in uncluttered environments and that the computation time of the detector is real-time compatible.

I. INTRODUCTION

The performance of object detectors can be impacted by changes in the camera's viewpoint. These algorithms can be inoperative if they are not robust enough. Most of the human detection algorithms in the literature suffer from this problem because they are designed to work with a very specific view: the pedestrian view. For numerous applications the viewpoint can be complex and/or can change over time because the camera is moving: the video surveillance, sports event filming, aerial filming, etc. Theorically there is an infinite number of possible camera's viewpoints. During the detection stage the detector should be capable of dealing with the maximum of these cases.

A. Human detection algorithms

For a more generalized use of the detection algorithm: no assumptions are made about the pose of the camera, or about the movement of people. Thus, detection methods based on background substraction are not well suited. The detection process should only be based on the visual information contained in one frame. Monolithic and part-based detectors fit this condition. 1) Monolithic detectors: Monolithic detectors search for monolithic parts of the image looking like people. Papageroriou et al [START_REF] Papageoriou | A Trainable System for Object Detection[END_REF] use wavelet descriptors, a sliding-window method to exhaustively scan the image and a SVM classifier. Many of current detectors are based on this approach. Viola et al [START_REF] Viola | Rapid Object Detection using a Boosted Cascade of Simple Features[END_REF] use integral images and a cascade classifier to speed up the computation of the Haar-like wavelet features and reach real-time performance for face detection. The Histogram of Oriented Gradients (HOG) detector of Dalal and Triggs [START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF] is an efficient human detector using a variant of the very wellknown and quite efficient SIFT descriptor [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF].

Dollár et al [START_REF] Dollár | Integral Channel Features[END_REF] proposed a mix between Viola et al's detector and the HOG detector: the ICF detector. It computes simple rectangular features on integral images of different channels. The classification is performed using a fast softcascade classifier.

2) Part-based detectors: Part-based detectors consider the human body as a set of parts. Felzsenzwald et al [START_REF] Felzenszwalb | Pictorial Structures for Object Recognition[END_REF] propose a method to detect people by fragments and re-build a human model by using a pictural structure reprensentation. Each part of the human model is separately learned. These detectors are slower than monolithic detectors and are not well adapted to detect far-off situated people.

B. The multi-viewpoint context

Let's first define the camera's viewpoint: it is the view obtained through this camera and for a specific configuration of its roll, pitch and yaw angles (Fig. 2 a). The camera has six degrees of freedom.

In addition to the perspective effect, the impact of the camera angles on the human appearance are the followings: the roll angle tends to rotate the shape of people and the pitch angle tends to change the shape of people (Fig. 3).

When the visual changes are too important, they cannot be managed by a detector designed to detect people in a pedestrian view (Fig. 2). This lack of robustness can be due to the combination of several factors: the nature of the visual descriptor, an unadapted training dataset, and/or the scanning process itself.

1) The nature of the visual descriptor: Some decriptors are natively dependant on orientation, such as: Haar-like [START_REF] Viola | Rapid Object Detection using a Boosted Cascade of Simple Features[END_REF] and HOG [START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF] descriptors. A rotation invariant descriptor should have both its shape and its metric invariant to rotation. However, in our case, rotation invariant descriptors are not a solution because we have to deal with the pitch angle of the camera which causes changes of people appearance (Fig. 2 and Fig. 3).

2) The training dataset: Most human training datasets are not adapted to deal with changing viewpoints. To the best of our knowledge only one training dataset is designed to permit an improvement of the detection capabilities for changing viewpoints: the GMVRT-v1 dataset 1 . Nevertheless, this dataset only takes into account changes implied by the pitching of the camera. The ideal dataset should allow the training algorithm to face the maximum of viewpoints during the training, i.e: with rolling and pitching combined.

3) The scanning process: This part is often designed to work for a single view: the pedestrian view. Detectors using the sliding window approach often use a vertical one-half ratio detection window to scan images for human candidates in different places, and for different depths [START_REF] Viola | Rapid Object Detection using a Boosted Cascade of Simple Features[END_REF][4] [START_REF] Dollár | Integral Channel Features[END_REF]. This technique is obviously not appropriate when there is an important roll of the camera, because, in this case, human shapes are not vertical but rotated.

C. Content of the paper

The goal of this work is to reach fast and viewpoint robust detection of upright human beings for applications in uncluttered environments. This work is mainly about a new framework designed to combine the assets of the ICF detector to the robustness of a multi-view classifier. However, detecting human people whatever the viewpoint is a pure binary problem: only the presence or the nonpresence of human people is of interest. Considering one class considerably simplify the labelization problem as well as the tuning of the parameters.

A. ICF detector

Among all the monolithic supervised pedestrian detectors the ICF detector of Dollár and al [START_REF] Dollár | Integral Channel Features[END_REF] is a good candidate to approach a fast and competitive detection [START_REF] Dollár | Pedestrian detection: A benchmark[END_REF]. This part describes the two phases of the detector.

1) The detection phase: The input image is exhaustively scanned by a sliding detection window (Fig. 4 c). To detect persons at different distances all the levels of an image pyramid are scanned as showed in Fig. 4 a.

The visual features are computed on integral images of ten different channels, which are: L, U, V, gradient magnitude and six "HOG channels" [START_REF] Dollár | Pedestrian detection: A benchmark[END_REF]. A feature is simply the sum of the pixels contained within a rectangle and associated to one of the channels. A person is detected in a window if the set of visual features matches the human model.

A coarse-to-fine approach is adopted to speed the classification: the soft-cascade. Dollár et al proposed to approximate the features between pyramid levels to speed up the detection: the Fastest Pedestrian Detector In the West (FPDW) [START_REF] Dollár | The Fastest Pedestrian Detector in the West[END_REF], and using Aggregated Channel Features (ACF) [START_REF] Dollár | Fast Feature Pyramids for Object Detection[END_REF]. These techniques can also be used with our PRD for greater speed performance.

2) The training phase: During the training phase AdaBoost [START_REF] Schapire | Improved Boosting Algorithms Using Confidence-rated Predictions[END_REF] is used to select the best discriminant features on positive and negative training images. The final classifier is a combination of weak-classifiers (depth-2 decision trees).

B. The new framework

Wu et al [START_REF] Wu | Cluster Boosted Tree Classifier for Multi-View, Multi-Pose Object Detection[END_REF] show that AdaBoost can fail to find an optimal solution when the positive training dataset contains important variations of shape. They propose a method to cope with this problem by learning the different aspects of the same object class: the Cluster Boosting Tree (CBT). The idea is to clusterize (k-means) the training positive dataset during the training process to reach an optimal solution for each cluster. A binary tree structure of the training dataset is generated by the clusterizations. This allows both sharing and optimized choices of features [START_REF] Wu | Cluster Boosted Tree Classifier for Multi-View, Multi-Pose Object Detection[END_REF].

The three following sections are about the adaptations of the ICF detector and the CBT training algorithm for viewpoint robust human detection. We named the final solution the pitch and roll-trained detector (PRD).

1) ICF adaptations: The features are computed in a circle with a radius of 64 pixels. Dollár and al recommend to use 30.000 random feature condidates [START_REF] Dollár | Integral Channel Features[END_REF]. As the surface of the circle is about 1.5 bigger than the surface of the classic detection window, 45.000 features candidates are generated in order to keep a relatively similar density of candidate features. The classification is performed by a depth-first search tree traversal of the classifier (trained with CBT).

2) CBT adaptations: Alg.1 presents our modified version of the CBT, see [START_REF] Wu | Cluster Boosted Tree Classifier for Multi-View, Multi-Pose Object Detection[END_REF] for the original implementation. Our version is lighter and proved to be more efficient for our needs. Lines 7 and 8 (Alg.1) are typical AdaBoost procedures, except that S k + is a subset of S 0 + for k>0. Line 9 is the condition to trigger the clusterization: the classification power (h(t,k).Z) of the three latest trained weak-classifiers are compared to θ Z . Lines 12 and 13 are retraining procedures of the previously trained weak-classifiers. We found it better to set θ Z to 0.98 and to authorize as many clusterizations as possible. 3) Adapted training data: The GMVRT-v1 dataset was a first attempt to deal with the pitch angle, but the lack of samples forced us to re-build a new training dataset: the GMVRT-v2 1 (Fig. 5). This new dataset contains 3846 images of people taken at different pitch angles of the camera (extracted from one hundread aerial movies). Some INRIA positive training images were added to complete the dataset.

III. TESTS

In the following section different aspects of the detector have been tested and compared: the general performance, the computation time and the angular robustness. The general performance and the computation time have been tested on a dataset of 210 images taken from complex camera's viewpoints 1 . Each image contains from 3 to 5 people. The angular robustness has been tested on two other datasets of 180 images each: a dataset for testing the rolling robustness and a dataset for testing the pitching robustness 1 . We named our final solution: the pitch and roll-trained detector (PRD).

The color code is the following: in red, the original Integral Channel Features (ICF) detector, in green, a detector trained with the GMVRT-v2 dataset for pitch robustness (pitch-trained detector or PD), in blue, a detector trained with the INRIA dataset but for different roll angles (roll-trained detector, RD) and in purple, the pitch and roll-trained detector (PRD).

A. The general performance (Fig. 7)

The ICF detector fails to succeed in most cases (Fig. 7). There is a slight improvement of the detection performance when the roll of the camera is taken into account during the training (RD). The improvement is bigger when the pitch of the camera is considered instead (PD). The best performance are obtained when both the pitch and the roll of the camera are considered (PRD).

B. The angular robustness (Fig. 8 and9) 1) The pitch angle robustness: The performance of the ICF begins to fall down from about 35 degrees (Fig. 8). The average detection rate is null at 60 degrees. Conversely, the PD and the PRD have relatively similar average rates whatever the pitch angle between 0 and 80 degrees included. 2) The roll angle robustness: The average rate of the ICF is null before -40 degrees and after 40 degrees. The average rates can be considered acceptable for human detection between -20 and 20 degrees. The RD and the PRD have relatively stable average detection rates from -90 to 90 degrees. However, the PRD seems more stable than the RD. The average speed of the RD detector is not tested in this part due to its poor detection performance. The PRD is 1.75 times slower than the ICF and the PD. This slow-down is due to the number of weak-classifiers of the PRD. The tested PRD has six times more weak-classifiers than the PD or the ICF. The FPDW optimizations allow the PRD to approximatively reach the average speed of the ICF, which is one of the fastest pedestrian detector of the state-of-the-art [START_REF] Dollár | Pedestrian detection: A benchmark[END_REF].

IV. CONCLUSION In this paper, we proposed a new detector particularly adapted to moving cameras where the viewpoint is likely to be complex and changes over time: the pitch and rolltrained detector (PRD). We showed that the PRD outperforms the Integral Channel Features (ICF) detector for complex camera's viewpoints. The main contributions of this work are: 1) a new framework based on the Cluster Boosting Tree and the ICF detector for viewpoint robust human detection, 2) a new training dataset for taking into account the human shape modifications occuring when the pitch angle of the camera changes. The next objective is to reinforce the detector with other features or signals to extend the capabilities of the detector to cluttered and more complex scenes such as urban scenes.

Fig. 1 .

 1 Fig. 1. a) results obtained with the ICF detector [1] b) and with the PRD: the detector is less sensitive to changes of shape and angle.

Fig. 2 .Fig. 3 .

 23 Fig. 2. a) camera angles, b) camera poses in a pedestrian context, c) camera poses in a general context.

  Fig. 4. a) image pyramid, b) scanning with the sliding window, c) computation of the visual features (in blue). Section II is about the ICF detector and the training framework. The second part of this section is about the adaptations proposed to reach fast and viewpoint robust human detection. Section III is about the tests conducted on a test dataset containing images with complex viewpoints. The results of the tests are also discussed in this section. II. PROPOSED APPROACH There are different approaches for improving the robustness to viewpoints with a supervised monolithic detector: training a combination of binary classifiers, training a multi-class classifier or training a view-aware binary classifier.However, detecting human people whatever the viewpoint is a pure binary problem: only the presence or the nonpresence of human people is of interest. Considering one class considerably simplify the labelization problem as well as the tuning of the parameters.

Fig. 5 .

 5 Fig. 5. Examples of GMVRT-v2 images.

Fig. 6 . 2 c ← 1 ; 3 for k ← 0 to c do 4 reset

 62134 Fig. 6. a) balanced spreading of the data with an angular step of αdeg, b) multiple balanced spreading shifted by angular offsets multiple of β (one color = one spreading).Subsets of images are rotated every 5 degrees during the training and between -90 to 90 degrees (α = 5degrees, Fig.6.a). We named it an angular spreading. Spreading the data reduces the learning robustness. We overcame this problem by repeating the same operation several times with different angular

Algorithm 1 :

 1 0, c ] compute the soft-cascade for channel k; 19 Our CBT implementation. c: number of clusterizations, k: index of the cluster (or channel), T: maximum number of weak-classifiers, h(k,t): weak-classifier number t of channel k, tinit(k): starting index for cluster k, θ Z : clustering critera, S k + : cluster k of positive image, S -: all the negative images. offsets (β = angular offset, Fig.6.b). The first spreading is from -90deg to 90deg, the second spreading is from -90βdeg to βdeg, etc. It guarantees more positive samples by degree.

Fig. 7 .

 7 Fig. 7. ROC curves for the four detectors. The two best performance are obtained with the PD and the PRD detector. The ICF detector clearly fails on this dataset.

Fig. 8 .

 8 Fig. 8. The PD and the PRD are quite robust to pitch angle variations unlike the ICF.

Fig. 9 .

 9 Fig. 9. The RD and the PRD are more robust to roll angle variations than the ICF. The PRD seems to be more stable though.

  + and S -;

	5
	6	for t ← tinit(k) to T do
	7	build best weak-classifier h(k,t);
	8	update weights of S k + and S -;
	9	if h(k,t).Z > θ Z and h(k,t-1).Z > θ Z and
		h(k,t-2).Z > θ Z then
	10	split S k + into S k + and S c+1 +	;
	11	h(c+1,t') = h(k,t'), ∀t' ∈ [ 0, t ] ;
	12	retrain weak-classifiers h(k,t'), ∀t' ∈ [ 0, t ]
		with S k + and S -;
	13	retrain weak-classifiers h(c+1,t'), ∀t' ∈ [ 0, t ]
		with S c+1 +	and S -;
		tinit(c+1) = t;

TABLE I COMPUTATION

 I TIME.


Paul Blondel
email: paul.blondel@u-picardie.fr

Alex Potelle

Claude Pégard

Rogelio Lozano
email: rogelio.lozano@hds.utc.fr

Human Detection in Uncluttered Environments: from Ground to UAV View

Nowadays pedestrian detectors are fast, scale-robust and quite efficient. Embedded within a UAV such a detector would open new possibilities. In this paper the very well known HOG detector is adapted for UAV use and a new kind of training dataset is proposed in order to increase the detector's angular robustness.

A more appropriate set of detection windows, together with a new detection pipeline, is proposed in order to reduce the search space and consequently reduce the computation time. Tests conducted using the improved detector show significantly better results on aerial images.

I. INTRODUCTION

Given the continuous fall in UAV prices and technological advances in this field, UAVs are becoming more and more accessible to laboratories and companies of every size. UAVs are more and more used for various tasks. Nowadays they are currently considering using UAVs for search and rescue missions, or monitoring specific areas such as nuclear plants or other sensitive areas. These tasks require embedded human detection algorithms to automatically detect people from the air.

A. Existing work in pedestrian-view detection 1) Detection with background substraction: The detection of moving regions is obtained by the difference between the current frame and a reference frame, often called background image. These regions are analysed in order to classify the moving objects. The analysis can be done, for instance, with the help of a visual codebook [START_REF] Zhou | Real Time Robust Human Detection and Tracking System[END_REF] or by using contour shape matching [START_REF] Toth | Detection and recognition of moving objects using statistical motion detection and Fourier descriptors[END_REF]. A more powerful classification of the moving regions can be performed using a monolithic or a multi-parts human detector. Background substraction is not suitable when the camera itself is also moving.

2) Monolithic detection: Monolithic detectors look for monolithic parts of the image that look like people. Gavrila et al proposed to use a hierachy of human contour templates obtained using training [START_REF] Gavrila | Shape-based pedestrian detection and tracking[END_REF]. This hierachy, used together with the chamfer matching algorithm, permits the detection of people in images. But more discriminative methods based on powerful descriptors have also been developed. The visual information is locally extracted and collected. Finally the information is compared to a general model of people with a classification algorithm. Papageroriou et al were among the first to propose this pipeline [START_REF] Papageoriou | A Trainable System for Object Detection[END_REF]. They used wavelet descriptors, a sliding-window method to exhaustively scan the image and a SVM classifier. Many of current detectors are still based on this approach. Viola et al based their work on the work of Papageoriou et al [START_REF] Papageoriou | A Trainable System for Object Detection[END_REF]. They used integral images and a cascade classifier to speed up the computation of the Haar-like wavelet features and reach real-time performance for face detection [START_REF] Viola | Rapid Object Detection using a Boosted Cascade of Simple Features[END_REF]. The Histogram of Oriented Gradients (HOG) detector of Dalal and Triggs [6] is an efficient human detector using a variant of the very well-known and quite efficient SIFT descriptor [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF]. Visual information is extracted using SIFT-like descriptors over a sliding-window. All the information is classified using a linear SVM classifier trained on images of people. The SIFTlike HOG descriptor still remains very competitive for object detection. Some detectors combine multiple descriptors, image features and/or information sources to increase the detection rate. Wojek et al showed that combining HOG, Haar-like descriptors, shaplets and the shape context outperform the HOG detector alone [START_REF] Wojek | A Performance Evaluation of Single and Multi-feature People Detection[END_REF]. Dollar et al proposed a mix between Viola et al's detector and the HOG detector [START_REF] Dollár | Integral Channel Features[END_REF]. This detector computes simple rectangular features on integral images of different channels: L,U,V, gradient magnitude and six "HOG channels". The classification is performed using a fast softcascade classifier.

3) Multiple parts detection: Instead of considering the human body as one monolithic part, some detectors consider it as a set of parts. Felzsenzwald et al proposed a method to detect people by fragments and re-build human models by using a pictural structure reprensentation [START_REF] Felzenszwalb | Pictorial Structures for Object Recognition[END_REF]. Each part of the human model is separately learned. An incorrect labelling of the fragments could decrease the performance of the detector [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF]. That is why Felzenszalb et al introduced a detector using a new classifier: the latent SVM classifier [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF]. With this classifier the most discriminative information is selected during the training to produce a more robust detection.

B. Existing work for detecting people from a UAV

Detecting people is difficult and it becomes more difficult in a UAV context. Most human detectors focus on detecting upright people at nearby distances and from a more or less invariant viewpoint. The current two main applications of human detection is the security monitoring and the driving assistance. Until now little work has been done on detecting humans from a UAV. Unlike a pedestrian view, an UAV view is more complex to manage because the drone undergoes pitching and rolling rotations. People are also on average further from the camera in this context.

Gaszczak et al proposed to use both thermal and visible imagery to better detect people and vehicles [START_REF] Gszczak | Real-time People and Vehicle Detection from UAV Imagery[END_REF]. Features extracted on thermal and visible imagery are fused together to boost the confidence level of detection. The thermal camera is used for extracting Haar-like features while the optical camera is used for a contour shape analysis as a secondary confirmation to better confirm the detection. This method permits to detect upright people at a distance of about 160m using a fixed camera pitch rotation of minus 45 degrees and in real-time. This method does not seem flexible enough for detecting people closer to the UAV.

Rudol et al also use thermal and visible imagery but in a pipeline way [START_REF] Rudol | Human Body Detection and Geolocalization for UAV Search and Rescue Missions Using Color and Thermal Imagery[END_REF]. They first identify high temperature regions from the thermal image and they reject the regions not fitting a specific ellipse. The corresponding regions are then analyzed in the visible spectrum using a relaxed Haar-like detector. Upright and seated people can be detected with this method. However, the thermal imagery can easily become very tricky to analyze with this method when the drone is too close and the information becomes too noisy.

Reilly et al have a different approach [START_REF] Reilly | Geometric constraints for human detection in aerial imagery[END_REF]. They use people's shadows as a key clue to detect and localize people. But strong assumptions on weather conditions have to be made with this technique.

Andriluka et al evaluated various detection methods for detecting victims at nearby distances [START_REF] Andriluka | Vision based victim detection from unmanned aerial vehicles[END_REF]. They showed partbased detectors are better suited for victim detection from a UAV because they natively take into account the articulation of the human body. The authors propose the use of complementary information using several detectors and inertial sensor data to obtain a better detection rate. However, part-based detection is a slow process [START_REF] Felzenszwalb | Pictorial Structures for Object Recognition[END_REF] and this seems not suitable for detecting people too far from the camera.

C. Paper content

Different parameters have to be considered to automatically detect people from a UAV: the position and the orientation of the embedded camera in relation to the target, the distance, the variability of human poses, the illumination, occultation, etc. The purpose of this paper is to show that it is possible to easily adapt a pedestrian-view human detector for UAV-view human detection. The aim is the detection of upright people located between 10 and 40m from the camera in a fast and robust manner and in uncluttered environments. This work provides solutions to manage the distance, the search space and the orientation of the target in relation to the camera.

It was chosen to adapt one of the most well-known pedestrian detectors: the HOG detector of Dalal and Triggs [START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF]. The section II describes the HOG detector and its key configuration parameters. The UAV context is also discussed. Section III deals with ways to adapt the detector to the UAV scenario. In section IV experimental results are presented and discussed.

II. STUDY OF THE HOG DETECTOR IN A UAV CONTEXT

A. The HOG detector 1) How it works: The input image is exhaustively scanned by a sliding detection window of a specific size and ratio as shown in Fig. 1 b. An object is detected if the combination of all the histograms computed within this detection window matches a general model of the object class. In order to detect objects of different sizes an image pyramid is built from the original input image and all the levels scanned as showed in Fig. 1 a. The configuration of this image pyramid is directly related to the expected sizes of searched objects.

For each detection window the histograms are computed in a very specific manner. The detection window is composed of overlapped blocks as shown in Fig. 1 c (in blue). A block is composed of a certain number of cells. For each cell we compute an histogram of the oriented gradients. Typically, a block is composed of four squared cells. The histogram is divided into bins, typically nine bins from 0deg to 180deg as recommended by Dalal and Triggs [START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF]. At the end, all the histograms of the blocks are locally normalized using the L2-Norm or the L2-Hys Norm.

The data computed within the sliding detection window is compared to a general model. The general model of the object class is built using a SVM classifier trained using the appropriated training images (positive and negative case images).

2) The image pyramid: An image pyramid is required to find objects of different sizes. Building the right image pyramid is very important. Three parameters are required to build an image pyramid: a number of levels or a scale factor, the minimum and the maximum scale. Objects can be missed if these parameters are maladjusted.

3) The detection window: The ratio of the detection window is important, a vertical one-half ratio is usually chosen to detect upright people in a pedestrian-view scenario. Changing the ratio with the viewpoint to better match the shape of the object could be considered, but changing the ratio of the detection window often requires changing the block configuration. Changing the block configuration tends to change the performance as well [START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF]. stabilizers do not solve all the problems: there will still be a great elevation angle between the ground and the UAV's camera. It can be summarized as follows: the rolling angle tends to rotate the shape of people and the elevation angle tends to shrink the shape of people (Fig. 2, 3 and4). In addition, as the elevation angle increases, perspective effect tends to rotate people far from the camera axis (horizontally) ; for example people in the left side of Fig. 4 a and Fig. 4 d.

2) Wide distance ranges: The greater the distance range the more scale scans are required. This increases the computation time. An image pyramid with less levels can be deduced using geometric knowledge of the scene when available. This is not the case in this work.

3) Changing weather conditions: UAVs are subject to weather conditions because they are outdoor robots. The detection should be robust to illumination changes. Fortunately the HOG detector is natively quite robust to this because of the local block normalization. But additional cues could be used in order to increase its robustness.

III. ADAPT THE DETECTOR TO THE UAV-VIEW

A. A more appropriate dataset Blondel et al showed that the robustness to the elevation angle can be improved on synthetic images by a multi-view training at different elevations [START_REF] Blondel | How to improve the HOG detector in the UAV context[END_REF]. When the elevation angle is greater than 45degrees significantly better results were obtained for detecting 3d models. We propose to extend and to evalute this approach in a real case scenario. A multi-view training dataset called GMVRT-V1 (Fig. 3) and an aerial test dataset (Fig. 4), for testing the detector performance, have been built.

1) GMVRT-V1 : Generalized multi-view real training dataset (version 1)

1 : Six different models were used for obtaining the positive samples. They were asked to mimic three different poses (relaxed, walking and making distress signs). They were also asked to change their clothes once. The acquisitions of the positive samples were made in the following manner: a GoPro Hero 3 camera was attached to the top of a triangle formed by two rods. Models were asked to stay below the camera and two others people synchronously rotated the triangle in an arc of a circle from 90 degrees 1 http://mis.u-picardie.fr/∼p-blondel/papers/data 2) Real images test dataset 1 : We took images of people with different natural backgrounds in an uncluttered environment. We made sure that the pitching angle of the camera was greater than 40 degrees. Aquisitions were made using a GoPro Hero 3 and images have been corrected to remove the barrel distortion effect.

B. Smaller detection windows for detecting smaller objects 1) Principle:

The standard HOG uses a 64x128 window with 64 pixel cells to scan all the levels of the image pyramid [START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF]. This size is not suitable in a UAV scenario because people are more likely to be far from the camera. Bigger levels are required to look for far off objects/people and this is timeconsuming. Smaller detection windows would partially help resolve this problem. But this should not alter the detection performance. A simple design is proposed to ensure that: one of the Dalal and Triggs' block configuration (2x2 cell blocks) with a number of pixels by cell always greater than the number of bins (in this case, 9). It gives three possible window configurations: 64x128 with 64 pixel cells, 48x96 with 36 pixel cells and 32x64 window with 16 pixel cells. The window configuration (and thus the associated classifier) could be dynamically switched from one configuration to another in relation to the scanning depth.

2) Complexity analysis: Taking a CCD camera with a 1/3" sensor, a 640x480 image resolution and a 8mm lens. As a first step it is considered that all the pixels of the level are first classified into the histogram bins. In a second step the block normalization is performed inside the scanning detection window. The two steps are repeated for each level. We consider a distance range from 10m to 40m with an average person size of 1m70. The scale range is from 0.53 to 1.06 for the 32x64 window, from 0.8 to 1.6 for the 49x96 window and from 1.06 to 2.13 for the 64x128 window.

The number of pixel classifications for one level (first step) is given by equation 1 (s is the scale):

Complexity(s) = totalSurf ace × s = 640 × 480 × s (1)
Figure 5 shows the complexity for a given level scale s. The total number of pixel classifications performed between two levels is given by the area under the curve. However, in practice, only a discrete approximation of the area is true.

For the 32x64 and 48x96 windows, the area under the curve is respectively 4.06 times and 1.77 times smaller than the one of the 64x128 window (Fig. 5). In conclusion, using a window size of 32x64 is faster. C. Analyse only the relevant locations 1) Principle: The exhaustive search of small objects is timeconsuming. Reducing the search space will aid in decreasing the computation time. The analysis of the saliency map permits to extract regions of interest because people's saliency is high in uncluttered environments (Fig. 6). However, this technique is not suitable when looking for closely situated people because the saliency becomes noisy and tricky to analyze. But in the last case there is no real need to reduce the search space because searching for closely situated people is quite fast.

The saliency map represents the saliency of a scene, with the concept being first introduced by Koch and Ullman [START_REF] Itti | A Model of Saliency-Based Visual Attention for Rapid Scene Analysis[END_REF]. The most salient locations extracted from this map are supposed to predict, quite well, the eye fixation locations. In uncluttered environments a small number of regions of interest can be deduced from these locations. We are particularly interested in the bottom-up saliency because it can be computed using only the pixel information [START_REF] Borji | State-of-the-art in visual attention modeling[END_REF]. But in return, the highlevel context cannot be taken into account.

The search space reduction requires a fast and discriminative method. For this task we retained four different methods. Frintrop et al's method using integral images [START_REF] Frintrop | VOCUS: A Visual Attention System for Object Detection and Goal-directed search[END_REF]. Their method is based on Itti et al's work [START_REF] Itti | Computational modelling of visual attention[END_REF]. Difference of gaussians are performed on several specific channels to mimic the visual receptive fields. The Katramados and Breckon's method [START_REF] Katramados | Real-time visual saliency by division of gaussians[END_REF]. They propose to divide two gaussian levels of two different gaussian pyramids to obtain the saliency map. The Achanta et al's method [START_REF] Achanta | Saliency Detection Using Maximum Symmetric Surround[END_REF]. They substract the pixels of the gaussian blurred version of the image to the arithmetic mean pixel value of a maximum symmetric surround. Lu et al's method [START_REF] Lu | Robust and Efficient Saliency Modeling from Image Co-occurrence Histograms[END_REF]: high-quality saliency map is deduced from image co-occurence histograms.

2) Pipeline: Reducing the search space using the saliency map requires a different approach, with a simple pipeline being Fig. 7. The proposed pipeline for using the saliency. The saliency map is computed 2) from the input image 1), blobs are extracted 3), bounding boxes of the blobs are computed 4), detection windows are generated around the boxes 5) and results are fused 6).

proposed (Fig. 7). The saliency map is first computed from the input image (Fig. 7 2) and is then thresholded. Blobs of sufficient size are retained (Fig. 7 3). Bounding boxes of all the retained blobs are computed (Fig. 7 4). Detection windows are then generated for each blob and the centers of the windows are randomly chosen inside the bounding boxes (Fig. 7 5). All the detection windows are treated separately and the results are fused with the mean-shift procedure (Fig. 7 6). Some parameters have to be considered: the threshold value, the minimum number of pixels to retain a blob, the number of detection windows to generate and the scale range.

The threshold value depends on the environment complexity. This value can be experimentally chosen. The minimum number of pixels for keeping a blob is a parameter used for filtering the noise. This value can be changed with respect to the distance to the people but most of the time a constant value permits sufficient filtering. The bigger the number of detection windows the better the detection, but the computation time will increase as well, and this must be taken into consideration when tuning this parameter.

IV. TESTING

A. A more appropriate dataset 1) Methodology: At first a test was conducted to gradually compare the detector's response to elevation degrees greater than zero. A comparaison was made between the training efficiency of the two training datasets: the INRIA and the GMVRT-V1 datasets. The purpose of this test was to know from which elevation degree the INRIA trained detector started to produce worse results. Images of people, taken from elevations between 0 and 90degrees, were used.

Secondly, the global performance of the GMVRT-V1 trained detector was evaluated using the aerial test dataset. The purpose of this test was to confirm the suitability of such a multi-view training for human detection in real flight conditions.

2) Results:

The average detection rate of the INRIA trained detector starts decreasing from 40degrees elevation and higher whilst the average detection rate of the GMVRT-V1 trained detector is not sensitive to the elevation (Fig. 8). The GMVRT-V1 trained detector is therefore more robust to elevation. The robustness has been tested on 180 cases. As far as we know the GMVRT-V1 training dataset is the first designed to manage elevation. Therefore, it is not possible to present comparaison results. However, one can easily see that the behaviour of the GMVRT-V1 trained detector, tested on the aerial dataset, is similar to the behaviour of pedestrian detectors in a pedestrian context [START_REF] Dollar | Pedestrian detection: A benchmark[END_REF] (Fig. 9). The detection results are greatly improved with this training. The INRIA trained detector never obtains a better than 0.9 miss-rate when tested on the aerial test dataset.

It is believed that better results could be obtained by increasing the diversity of the positive training images and also by adding to the training dataset more various negative images of natural and uncluttered environments. It was also observed that the perspective effect tends to slightly rotate the shape of people according to the drone's altitude and this alters the detection results.

B. Smaller detection windows for detecting smaller objects 1) Methodology: The performance of three detectors were tested: the 32x64 HOG detector, the 48x96 HOG detector and the original 64x128 HOG detector. Each detector has been trained with INRIA training images resized to fit the window size. All the detectors have been bootstrapped once using full INRIA negative images resized according to the window size (three-quarter and one-half factor for the 48x96 and the 32x64 Fig. 9. ROC curve of the detector trained with the GMVRT-V1 dataset : the shape of the curve is comparable to the shape of curves of pedestrian detectors in pedestrian-view. HOG detectors, respectively). The performance of the three detectors have been evaluated on a four times downsampled version of the INRIA test dataset. The image pyramids are configured as follows: 64 scales, a scale range from 0.6 to 4.65 for the 64x128 detector, a scale range from 0.45 to 3.2 for the 48x96 detector and a scale range from 0.3 to 2.13 for the 32x64 detector.

2) Results: The shapes of the 32x64 and the 48x96 curves are very similar to the shape of the 64x128 curve (Fig. 10). According to Dollar's critera [START_REF] Dollar | Pedestrian detection: A benchmark[END_REF] the performance of the three detectors are close: the miss-rate at 1 FPPI (Falses Positives Per Images) is similar for the three detectors. We have similar detection performance using smaller windows whilst executing fewer pixel classifications.

C. Analyse only the relevant locations 1) Methodology: Firstly, by using the aerial test dataset, the global performance and computation time of three saliency algorithms were compared: the Achanta's algorithm [START_REF] Achanta | Saliency Detection Using Maximum Symmetric Surround[END_REF], the Lu's algorithm [START_REF] Lu | Robust and Efficient Saliency Modeling from Image Co-occurrence Histograms[END_REF] and the Katramados' algorithm [START_REF] Katramados | Real-time visual saliency by division of gaussians[END_REF]. It was chosen not to use the gradients' saliency within the Lu's algorithm in order to have similar behaviors. The total surface of all the bounding boxes containing the most salient regions is computed for each image. A reduction ratio is computed Fig. 11. ROC curves of two detectors using different pipelines: the ROC curve of the detector using the saliency pipeline is similar to the curve of the detector using the classic pipeline, results are slightly better. Secondly, the global performance of two detectors, one using the saliency pipeline and the other the classic pipeline, were compared. It was chosen to use Achanta's algorithm in the saliency pipeline.

2) Results: Katramados' algorithm is the fastest (table I). It is also the most sensitive: the standard deviation (Sdev) is the biggest. And it is the one generating the most of noise : the mean is the biggest. Lu's algorithm is the slowest and it generates more noise than Achanta's. Achanta's algorithm has the best space reduction performance.

One does not necessarily choose the fastest saliency algorithm: the reduction ratio is a more important critera. A very fastly obtained but noisy saliency map leads to more blobs and the generation of unwanted windows, resulting in increased computation time.

The two curves of the Fig. 11 have a similar shape. We observe that the curve of the detector using the saliency pipeline is below the other one. The performance of this detector are better. This can be explained because the detection windows are more closely treated. In the original pipeline, detection windows are treated every 8 pixels in X and Y (8 pixels is a cell-shift).

An average computation time of 18.879 sec per image is obtained with our implementation of the HOG detector and an average computation time of 2.286 sec per image using the saliency pipeline.

V. CONCLUSION

It has been shown that the multi-view training improves the detection results when the camera is directed to the ground with a pitching angle greater than 40degrees. Two solutions with which to reduce the computation time have been proposed: smaller detection windows and a new pipeline called saliency pipeline. The smaller detection windows proposed in this paper permit to perform fewer pixel classifications whilst having relatively similar detection performance. And the saliency pipeline speeds up the detection by a factor of nine in our case and it slightly improves the robustness. Implementation of the saliency algorithm on an FPGA could improve the speed factor to obtain real-time performance for detecting far off situated people.

4 )

 4 Fig. 1. a) image pyramid, b) scanning with the sliding window, c) computation of the histograms and normalization for each overlapped block (in blue).

Fig. 2 .

 2 Fig. 2. Azimuth and elevation angles.

Fig. 3 .

 3 Fig. 3. Examples of GMVRT-V1 positive samples.

Fig. 4 .

 4 Fig. 4. Examples of images from the aerial test dataset. elevation to 0 degree elevation for each type of model pose and orientation. The positive samples were taken at different elevation angles. The negative samples are images taken from natural and uncluttered environments. The barrel distortion effect has been corrected in each image.

Fig. 5 .

 5 Fig. 5. In red : the growth of the complexity. From scale 0.53 to 1.06 : the complexity of a 32x64 window scan, for instance.

Fig. 6 .

 6 Fig. 6. a) input image, b) saliency map of the input image, c) ROIs extracted from the saliency map.

Fig. 8 .

 8 Fig. 8. Elevation robustness : the GMVRT-V1 training improves the robustness to the elevation compared to the INRIA training.

Fig. 10 .

 10 Fig. 10. ROC curves of three different detectors : the ROC curves of the 32x64 and the 48x96 detectors are similar to the ROC curve of the 64x128 detector.

TABLE I .

 I

		SALIENCY REDUCTION EFFICIENCY COMPARAISON
	Algorithm	Min (%)	Max (%)	Mean (%)	Sdev (%)	AvTime (s)
	Achanta	0.0018	0.1857	0.0178	0.0227	1.34
	Lu	0.0064	0.1625	0.0322	0.0339	4.02
	Katramados	0.0027	0.6586	0.0401	0.0560	0.08
	each time by dividing this surface by the image surface. The
	standard deviation and the mean of the reduction ratios are
	computed for each algorithm. The threshold and the minimum
	pixel number parameters have been tuned accordingly to
	the dataset to obtain the most of people's saliency and the
	least possible amount of noise. The chosen parameters are:
	a threshold of 0.18 and a minimum pixel number of 30 for
	Achanta's algorithm, a threshold of 0.20 and a minimum pixel
	number of 50 for Katramados's algorithm and a threshold of
	0.25 and a minimum pixel number of 50 for Lu's algorithm.


Imen Tayari Meftah

Chokri Ben Amar

Nhan Le Thanh

A Three-Layer Model to share emotional information

 to generate the final emotional data to be transferred through the network. We propose to modify the EmotionML vocabulary in order to represent emotions in our vector model defined in the formal computational layer.

Introduction

Human emotions are becoming a crucial aspect in human-machine interactive systems. Indeed, affective computing expands human-computer interaction by adapting the experience to the emotional state of the user. There is a rising need for emotional state recognition in several domains, such as health monitoring, video games and human-computer interaction. The emotional information exchange between applications entails many problems such as application heterogeneity, complexity of emotional states, diversification of capture tools and dependence between treatment and physical sensors. The lack of a standard in human emotions modeling hinders the sharing of affective information between applications. As part of the research project Emotica, we aim at defining a formal model allowing the communication between various multi-modal applications. In our work, we are interested in the 3D video games especially serious games like flight simulators. In this type of games we are very interested to know at each instant the felt emotion by the player and to adapt in consequence the simulation.

We target also to transfer the emotional data between the various players using different platforms and modalities.

In this paper we propose a generic computational model for the representation and exchange of emotional states and for a better multi-modal analysis. Our proposal is composed of three distinct layers: the psychological layer, the formal computational layer and the language layer.

In this paper, we explain the role of each layer of our generic model and their relationships to each other. The remainder of this paper is organized as follows. In Section 2, we give some related psychological and linguistic theories of emotion. In Section 3, we describe the different layers of our model and we conclude in Section 4.

Related Work

There is no consensus among psychological and linguistic theories on emotions. The word "emotion" comes from the Latin Latin " emovere, emotum", which means movement towards the outside. Emotion is a complex concept. Darwin [START_REF] Darwin | The expression of the emotions in man and animals[END_REF] thought emotions to be innate, universal and communicative qualities. Ekman [START_REF] Ekman | Emotion in the human face[END_REF], Plutchik [START_REF] Plutchik | Emotion, a psychoevolutionary synthesis[END_REF] and MacLean [START_REF] Maclean | Cerebral evolution of emotion, handbook of emotions[END_REF] have developed the theory that there is a small set of basic Emotions emotions all others are compounded. The most famous of these basic emotions are the Big Six, used in Paul Ekman's research on multi-cultural recognition of emotional expressions [START_REF] Ekman | The nature of emotion : Fundamental questions[END_REF].

Representing emotional states in technological environments is necessarily based on some representation format. Ball and Breese [START_REF] Ball | Modeling the emotional state of computer users[END_REF] have proposed a model of emotional states based on the Bayesian networks, which is designed to estimate the emotional state of a user interacting with a conversational compute. López et al [START_REF] Lopez | A methodological approach for building multimodal acted affective databases[END_REF] have proposed a model based on a generic ontology for describing emotions and their detection and expression systems taking contextual and multimodal elements into account. In earlier work [START_REF] Tayari Meftah | Towards an algebraic modeling of emotional states[END_REF] we have proposed an algebraic model for the representation and the exchange of emotions. Our model permits to model not only the basic emotions (e.g., anger, sadness, fear) but also different types of complex emotions like simulated and masked emotions.

The proposed model

Our proposal consists on a Three-Layer Model to share emotional information. As its name implies it is composed of three distinct layers which are interdependent to ensure a maintenance of coherence of the model: the psychological layer, the formal representation layer and the language layer.

The First layer: The psychological layer

The psychological layer represents the psychological theory adopted in our approach which is the Plutchik's theory. Indeed Plutchik model consists on a three-dimensional "circumplex model" which describes the relationships between emotions. He argued for eight primary emotion arranged as four pairs of opposites: (Joy-Sadness, Fear-Anger, Surprise-Anticipation, Disgust-Trust) [START_REF] Plutchik | Emotion, a psychoevolutionary synthesis[END_REF]. He suggested that non basic emotions are obtained through the addition of basic emotions (color analogy, Plutchik, 1962) [START_REF] Plutchik | Emotion, a psychoevolutionary synthesis[END_REF]. In his model, for instance, Love = Joy + trust and Delight = Surprise +Joy. Plutchik defined rules for building complex emotions out of basic ones.

In practice, combination of emotions follows the method "dyads and triads" [START_REF] Plutchik | Emotion, a psychoevolutionary synthesis[END_REF]. In our work, we chose the Plutchik model to represent the psychological layer because it verifies many important conditions for the elaboration of our model and it explains emotions in terms of formulas that can be universally applied to all human beings. First, the Plutchik model is based on 8 basic emotions encompassing the common five basic emotions. Then, it takes into account the intensity of emotion i.e., the level of arousal or the feeling degree of each basic emotion for example (terror, fear, apprehension). Finally, the Plutchik model is intuitive, very rich and it is the most complete model in literature because it permits to model complex emotions by using basic ones.

The second layer: The formal computational layer

The formal computational layer matches the psychological approach of the first layer. It is the formal representation of Plutchik's model and it is based on an algebraic representation using multidimensional vectors. We represent every emotion as a vector in a space of 8 dimensions where every axis represents a basic emotion defined on the Plutchik theory. We defined our base by (B) = (joy, sadness, trust, disgust, fear, anger, Surprise, anticipation). Thus, every emotion (e) can be expressed as a finite sum (called linear combination) of the basic elements. According to the Plutchik's theory, the mixture of pairs of basic emotions resulted of complex emotion. Joy and trust for example produce the complex emotion "love". "Submission" is a mixture of trust and fear. We defined the combination between emotions as the sum of two emotion vectors [START_REF] Tayari Meftah | Towards an algebraic modeling of emotional states[END_REF]. This addition is defined as the maximum value of coefficients for the same emotion. For the same axis, we keep the highest one because each modality can detect better a specific emotion. For example with the heart rate modality we can detect the fear component better than the facial expression modality.

The multidimensional representation of the formal computational layer provides the representation of an infinity of emotions and provides also a powerful mathematical tools for the analysis and the processing of these emotions. Indeed we can apply others usual basic algebraic operations on vectors like the scalar multiplication, the projection and the distance in an Euclidean space. for more detail you can see [START_REF] Tayari Meftah | Multimodal Approach for Emotion Recognition Using a Formal Computational Model[END_REF].

The language layer

The third layer of our model is the language layer. This layer provides encoding emotional information. We propose to use the Emotion Markup Language (EmotionML) for describing emotions. In fact, EmotionML provides representations of emotions and related states for technological applications. Given the lack of agreement in the community, EmotionML does not provide a single vocabulary of emotion terms, but gives users a choice to select the most suitable emotion vocabulary in their annotations. In our work, we propose to introduce in EmotionML a new element <plutchik-dimension> in order to represent emotions in our vector model defined in the formal computational layer. Therefore, we suggest to modify the EmotionML vocabulary and introduce the following item (It consists of a <vocabulary> element containing a number of <item> element) : <vocabulary type="plutchik-dimension" id="intensity-dimension"> <item name="joy"/> <item name="sadness"/> <item name="trust"/> <item name="disgust"/><item name="fear"/> <item name="anger"/> <item name="surprise"/> <item name="anticipation"/> </vocabulary>

The next example was generated using both face and voice modalities. Each modality gives a vector with different coefficients. Using the vector addition of the computational layer, we obtain the final vector describing the felt emotion <emotionml xmlns ="http ://www.w3.org/2009/10/emotionml"> <plutchik-dimension-set ="http ://www.w3.org/TR/emotion-voc/xml#intensitydimension"> <emotion expressed-through="face physiology"> <plutchik-dimension name="joie" value="0.8" /> <plutchik-dimension name="sadness" value="0.2" /> <plutchik-dimension name="trust" value="0" /> <plutchik-dimension name="disgust" value="0" /> <plutchik-dimension name="fear" value="0" /> <plutchik-dimension name="anger" value="0.7" /> <plutchik-dimension name="surprise" value="0" /> <plutchik-dimension name="anticipation" value="0" /> </emotion> </emotionml>

Conclusion

In this paper, we proposed a generic model facilitating communication between heterogeneous multi-modal applications. Our model is based on psychology research. It is composed of three distinct layers where each layer provides services that higher-level layers depend on. The middle layer provides powerful mathematical tools for the analysis and the processing of emotions. We introduce the implementation of EmotionML in the third layer and explain how it is used to represent and transfer emotional information between heterogeneous applications. We proposed to extend the current implementation of EmotionML to represent emotions in terms of our vectorial model defined in the middle layer.

α

  are scalars and u i (i=1..8) elements of the basis (B). Typically, the coordinates are represented as elements of a column vector the respective basic emotion. More the value of i α get nearer to 1, more the emotion is felt.