Bihao Wang
email: bihao.wang@hds.utc.fr

Sergio Alberto

Rodríguez Florez

Vincent Frémont
email: vincent.fremont@hds.utc.fr

Multiple Obstacle Detection and Tracking using Stereo Vision: Application and Analysis

Keywords: Stereo Vision, On-road Obstacles Detection, Particle Filter, Visual Tracking

Vision systems provide a large functional spectrum for perception applications and, in recent years, they have demonstrated to be essential in the development of Advanced Driver Assistance Systems (ADAS) and Autonomous Vehicles. In this context, this paper presents an on-road objects detection approach improved by our previous work in defining the traffic area and new strategy in obstacle extraction from U-disparity.Then, a modified particle filtering is proposed for multiple object tracking . The perception strategy of the proposed vision-only detection system is structured as follows : First, a method based on illuminant invariant image is employed at an early stage for free road space detection. A convex hull is then constructed to generate a region of interest (ROI) which includes the main traffic road area. Based on this ROI, an U-disparity map is built to characterize on-road obstacles. In this approach, connected regions extraction is applied for obstacles detection instead of standard Hough Transform. Finally, a modified particle filter framework is employed for multiple targets tracking based on the former detection results. Besides, multiple cues, such as obstacle's size verification and combination of redundant detections, are embedded in the system to improve its accuracy. Our experimental findings demonstrates that the system is effective and reliable when applied on different traffic video sequences from a public database.

I. INTRODUCTION

Obstacle detection and tracking are key issues of the Advanced driver assistance systems (ADAS). In the context of driver assistance, the purpose of obstacle detection and tracking system is to detect and monitor the dynamic behavior of one or more obstacles in the vicinity of the host vehicle. Hence, the ADAS can help to avoid potential collisions and to provide essential information for decision making. A reliable detection and tracking approach in real environments has been a challenge in the last two decades, especially considering various type of objects, their time varying number, and their states estimation from noisy observations at discrete intervals of time. Sensors, such as radar and LIDAR, have been used for this purpose using sensor data fusion approaches [START_REF] Ess | Object detection and tracking for autonomous navigation in dynamic environments[END_REF], [START_REF] Manduchi | Obstacle detection and terrain classification for autonomous off-road navigation[END_REF]. In recent years, many vision-only based approaches have been developed [START_REF] Seki | Robust obstacle detection in general road environment based on road extraction and pose estimation[END_REF], [START_REF] Jazayeri | Vehicle detection and tracking in car video based on motion model[END_REF] for the versatility of information they can provide and their low cost. In these approaches, learning-based methods focus on the detection and tracking of specific obstacles: like pedestrians and vehicles [START_REF] Fan | Pedestrian detection and tracking using deformable part models and kalman filtering[END_REF], [START_REF] Rodriguez | Densityaware person detection and tracking in crowds[END_REF]; while motion-based methods can extract the moving objects [START_REF] Jazayeri | Vehicle detection and tracking in car video based on motion model[END_REF], [START_REF] Kinjal | A survey on moving object detection and tracking in video surveillance system[END_REF]. In this paper, we present an on-road object detection approach from our previous work [START_REF] Wang | Fast road detection from color images[END_REF], [START_REF] Wang | Color-based road detection and its evaluation on the KITTI road benchmark[END_REF] which can effectively detect the road area in traffic scene and a multi-object tracking strategy based on particle filter. Since U-V-disparity map [START_REF] Hu | UV-disparity: an efficient algorithm for stereovision based scene analysis[END_REF] is an effective method to detect objects regardless their appearance and motion model, we propose to use it in the obstacle detection part. In order to predict the obstacle's position and moving direction, tracking is added in the system as a complementary to object detection [START_REF] Fan | Pedestrian detection and tracking using deformable part models and kalman filtering[END_REF], [START_REF] Okuma | A boosted particle filter: Multitarget detection and tracking[END_REF]. Especially, particle filters [START_REF] Okuma | A boosted particle filter: Multitarget detection and tracking[END_REF], [START_REF] Tehrani Niknejad | On-road multivehicle tracking using deformable object model and particle filter with improved likelihood estimation[END_REF] are widely used to solve multiple time varying obstacles tracking problems. Their strength lies in their ability to represent non-Gaussian distributions which can capture and maintain target properties.

First, stereo-vision-based obstacle detection is applied on a region of interest (ROI), which is composed of the main traffic area. To obtain the ROI, a fast road surface detection method [START_REF] Wang | Fast road detection from color images[END_REF] is firstly applied. Then, a convex hull algorithm is introduced to achieve the complete traffic area. Within the ROI, a method of connected region extraction from U-disparity map is developed to locate the obstacles in the image, and furthermore, to refine their position information from the subregion of disparity map extracted from the primary location of the obstacles. To improve the detection accuracy, multiple cues are integrated in the system, such as an adaptive height gating for obstacle detection in different distances and a combination of close detected area in the U-disparity map.

After obtaining the position, shape and depth information of the on-road obstacles, multiple target tracking hypotheses are managed by the means of a bank of particle filters. Targetto-track association is carried out following a global nearest neighbor (GNN) criterion. The tracking is performed in the image plane of the left camera in stereo vision system. In the 2D image plan, the obstacle's position and size is effected by it's distance to camera. To cope with this factor, the observed dynamics of the tracked obstacles is employed to define an adaptive association gate. Besides, a dynamic noise generation function is implemented in the filter. Considering the number of obstacles is time varying, for each iteration of the filtering, multiple hypotheses are made to create, delete and update the existing tracks. Fig. 1 shows the outline of the proposed perception system.

The approach is structured into two parts: on-road ob- The paper is organized as follows: First, a stereo-vison based on-road obstacle detection is introduced in Section II which includes two parts, traffic area extraction (Section II-A) and obstacle detection (Section II-B). Then, a modified particle filter tracking of multiple targets is presented in Section III. Experimental results and analysis on publicly available dataset [START_REF] Geiger | Vision meets robotics: The kitti dataset[END_REF] are shown in Section IV. Finally our paper ends with conclusions and future work.

II. ON-ROAD OBSTACLES DETECTION

The disparity map, I∆, can be extracted from stereo images [START_REF] Shapiro | Computer Vision[END_REF]. It refers to the displacement of the relative features or pixels between two views. A bigger disparity value corresponds to a closer distance to the camera. In I∆, an obstacle is represented as a homogeneous part with the same disparity value. U-V disparity maps are built by accumulating the pixels with same disparity value along the u, v axis of I∆ separately. The V-disparity map, I v ∆, is usually employed to estimate the longitudinal profile of the road and to detect the presence of obstacles by the means of a line extraction algorithm. However, in complex scenario, the V-disparity map I v ∆ is ambiguity prone. When obstacles are closed to each other, their representative lines in I v ∆ are mixed together. On the contrary, the U-disparity map preserves more information of the scene: the objects width, their relative positions and their depth information are kept. Therefore, in this paper, the U-disparity map leads to an accurate obstacles detection, while the V-disparity only assist in this procedure.

A. Traffic Area Extraction

In order to reduce the computational cost and to improve the detection efficiency on the road area, a suitable road ROI needs to be defined. It removes off-road information which may interfere the precision of on-road obstacle detection, for instance, continuous high walls/buildings along the road. From this consideration, a free road surface detection combined with a convex hull construction is proposed hereafter.

1) Application of Convex hull : According to [START_REF] Wang | Fast road detection from color images[END_REF], the free road surface I R can be computed by the conjunction of intrinsic road surface and the ground plane. Convex hull algorithm here provides the smallest convex area which contains all the free road surface I R . It fix up the holes and the depressions caused by on-road obstacles. A complete traffic area, i.e. the ROI, is then generated from free road surface. Therefore, obstacle detection can be focused on this approximated road traffic area. Even if the convex hull may not exactly follows the shape of the road, in most of the cases, it is sufficient to provide a satisfying ROI for further detection. For other cases, the tracking process detailed in Section III will efficiently deal with this issue.

B. U-disparity map based obstacles detection

In traffic scenes, there exist pedestrians, vehicles, traffic lights and signs, etc. For that purpose, the U-disparity map can be used to handle all the on-road obstacles, without prior knowledge on their types and motion models.

1) Connected-region extraction: In the U-disparity map I u ∆, obstacles are usually represented as straight lines. However, when an obstacle is passing near the camera side, both the frontage and side face of the obstacle are observed. The obstacle is then represented by a polyline: an horizontal part for frontage and a connected oblique part for its side face (As shown in Fig. 2 for the second obstacle in the second column). This situation happens frequently in driving scenes. To cope with this problem and to simplify the detection processing, a connected-component extraction algorithm is introduced in this paper to replace the classical Hough line extraction.

After a preprocessing using Eq. ( 1), high intensity regions are preserved in the U-disparity map I u ∆ and the other pixels are set to 0 (i.e. background).

I u ∆ = sgn(I u ∆(p) -ε) (1)
The definition of the intensity threshold, ε, is related to the camera calibration parameters and object's depth information. In our experiment, it has been set to 8 to 10 accumulated pixels.

After applying morphological operations (here, erosion and clean), noisy pixels are removed from I u ∆. Each connectedregion L being preserved in I u ∆, indicates a potential obstacle O L . Thus, the passing-by obstacle's information can be obtained by connected-component extraction algorithm. These information include: left bound u l and right bound u r of O L on the u-axis of the image; and its disparity value d O . The complementary information about O L like the height h O and the bottom position on the v-axis v b can be extracted from I v ∆ and furthermore refined by sub-region of the disparity map which contains the obstacle.

2) Obstacle localization with sub-Disparity map : For the obstacles standing at the same distance to camera, their accurate height information is mixed in I v ∆. In order to refine the location of each potential obstacle O L , a sub-region of disparity map I O ∆ for each obstacle is extracted from the complete disparity map I∆ according to their primarily N obs ← N obs + 1; end for 16: end for estimated location in the image. Pixels in these sub-regions of disparity map are classified into two classes, object or background, according to their disparity value.

I O = 1 object, if I O ∆(p) ∈ [d 1 , d 2 ] I O = 0 background, otherwise (2)
where, I O is the binary image with labeled obstacles. d 1,2 = d O ± σ where, σ is the bias of possible disparity value of the same obstacle. Obstacle's position and size information is then refined in I O . In this approach, the obstacle's information is represented by its centroid (x O , y O ), its width w O , its height h O and its disparity d O . Compare to region growing algorithms, the sub-disparity map extraction is much faster and effective. A detection example is showed in Fig. 2.

3) Multiple cues integration: In some cases, the connected component extraction may lead to false alarms or multiple detections on a single obstacle. Thus, multiple cues can be combined to refine the detection result.

• Height limitation of potential obstacles: If the object's height is smaller than a threshold δ, it will not be considered as an obstacle. This threshold is proportional to the disparity value of this potential obstacle δ ∝ d O .

The closer O L stands to the camera, the higher δ will be. Thus, the system can eliminate part of the false alarms. As in Fig. 2, the yellow lines in left middle image are the false alarms detected by I u ∆. • Combination of closely stand connected-regions: Since Udisparity image is accumulated on discrete values from the disparity map, there could exist fragments of the same obstacle using the representation of connectedregion. This will lead to redundant detections. To handle this problem, a combination operations, like bridge and dilation morphological operations are introduced. The complete on-road obstacle detection pipeline is summarized in Algorithm 1.

III. MULTIPLE OBSTACLES TRACKING

When multiple obstacles have been detected and they are tracked over frames, several unknowns must be properly handled, such as the number of targets at each time step and the data association between targets and tracks. On-road obstacles are regarded as the targets, see Fig. 1. Hereafter, a modified particle filter based on the condensation algorithm, is proposed to track every target. A dynamic noise update function is introduced and a self-adaptive gating is also designed to provide a reliable data association result.

A. Particle filter model

The condensation algorithm, as a special case of particle filtering [START_REF] Isard | Condensation-conditional density propagation for visual tracking[END_REF], provides a well-established methodology for generating samples from the required distribution without requiring assumptions about the state-space model or the state distributions. The samples from the distribution are represented by a set of particles; each particle has a weight representing the probability of that particle being sampled from the probability density function. At each time step, particles' weight and spatial distribution are used for tracker state estimation and re-sampling. In this paper, the tracking of the surrounding obstacles is carried in 2D image plane. In order to define an uniform state space, all the states are described in pixel level. The depth information is then represented by its corresponding disparity value. The filter model is build as follows:

• State vector: S = [x, y, v x , v y , w, h, d] T (3)
It is composed by the centroid object position (x, y) on the image; the velocity of the centroid v x , v y ; the width w; the height h and the disparity value d respectively.

• Observation:

Z = [x O , y O , 0, 0, w O , h O , d O ] T (4)
The values x O , y O , w O , h O , d O represent the information of detection result of Algorithm 1. Since the velocity of the object cannot be measured directly, it has been set to a 0 value.

• Estimation:

S(k) = Ns i=1 s i (k) • π i (k) (5)
where, S is the condensation state of the tracker, N s is the number of samples. s i (k) is current sample states, where, i = 1, . . . , N s , and k represents the time step. The variable π i is the normalized weight distributed for each sample.

• Prediction:

s i (k + 1) = f (s i (k)) (6) with s i (k) = [x i (k), y i (k), v i x (k), v i y (k), w i (k), h i (k), d i (k)] (7
) s i (k) and s i (k + 1) are consecutive particle states of current time and next time respectively. f is the dynamic model for evolution. In this approach, f is a constant velocity model. Thus, (6) can be written as :

                       x(k + 1) = x(k) + T • v x (k) + W x (k) y(k + 1) = y(k) + T • v y (k) + W y (k) v x (k + 1) = v x (k) + W vx (k) v y (k + 1) = v y (k) + W vy (k) w(k + 1) = w(k) + W w (k) h(k + 1) = h(k) + W h (k) d(k + 1) = d(k) + W d (k) (8)
W (k) = [Wx(k), Wy(k), Wv x (k), Wv y (k), Ww(k), W h (k), W d (k)] T (9)
where, W (k) is the noise vector at time step k that added for filter evolution.

• Update and Re-sampling:

π i (k + 1) = P (s i (k + 1 | k) | Z(k + 1)) N i=1 P (s i (k + 1 | k) | Z(k + 1)) (10)
For each evolution, particles s i (k + 1 | k) are predicted from their previous state s i (k) by Eq. [START_REF] Kinjal | A survey on moving object detection and tracking in video surveillance system[END_REF]. A new confidence density P (s i (k) | Z(k)) is then distributed to s i (k + 1 | k) through the comparison between particle states and associated observation Z(k+1). The weights of samples are then updated from the confidence density by Eq. [START_REF] Hu | UV-disparity: an efficient algorithm for stereovision based scene analysis[END_REF]. Subsequently, a new set of N s particles s i (k + 1) are constituted from the current sample set {s i (k + 1|k)} with probability proportional to the confidence distribution [START_REF] Isard | Condensation-conditional density propagation for visual tracking[END_REF].

B. Dynamic noise update

In the image coordinate frame, the scale and the displacement of an obstacle change rapidly with its distance to the camera. To handle this issue, the noise vector added in Eq. ( 8) should follow this change as well, i.e. W (k) ∝ d O (k). Therefore, the tracker is able to keep up with observation state in 2D image coordinates. At every sampling time, a dynamic noise update function is build as follows:

W (k) = C • d O (k) • Z(k) (11)
where, C is the coefficient vector that needs to be adjusted given an application. For a given equipment, these parameters can be set for once, because the variations lead by depth are related to the camera's essential matrix [START_REF] Hartley | Multiple view geometry in computer vision[END_REF]. The noise vector W (0) is initialized with high values to provide a broader range of sample distribution. During the filtering, the noise vector W (k) is set with lower order of values to provide a convergent range for particle predictions.

C. Data association

Global nearest neighbor (GNN) is the most natural data association process with a low complexity. When obstacle observations are perceived from the camera, Mahalanobis distances between each observation and prediction are calculated. In our approach, the distance between object and track is defined as follows:

dist = c 1 .∆ T x ∆ y + c 2 .∆ T w ∆ h + c 3 | ∆ d | (12)
where, c 1,2,3 are the coefficients for different measurements which indicate their contributions to the distance calculation. ∆ is the difference between estimation and observation measured on the state vector [x, y, w, h, d] separately. Thus, the obstacle's centroid (x, y) in 2D image is not the only criterion that contributes to the distance calculation, but also the width, the height (w, h) and the disparity value d are considered as well. 1) Self-adaptive gate : Target-to-Track association is limited by the use of a gate which is set to a constant value for eliminating unlikely association. In 2D image plane, the closer the obstacle stands to the camera, the greater Mahalanobis distance it might have with respect to the tracks in 2D image coordinates. For the on-road obstacles which are far from camera, they have smaller scales and stand closer to each other; a big gate will lead to mismatching. On the contrary, for nearby obstacles, they may not be able to be associated with the proper tracks because of a small gate. Thus, a constant gate is not sufficient for all the obstacles standing in different distances. In this paper, a self adaptive gate is modified with respect to every observed obstacle according to their scale and depth information:

G O = a • w O (k) T h O (k) + b | d O (k) | (13)
where, G O is the gate for each obstacle according to their observation

[x O , y O , 0, 0, w O , h O , d O ] T at time k.
Here, we set a = 0.5, b = 0.2, which is basically the radius of the circumscribed circle plus a small percentage of the disparity value. Thus, the gate is only related to the current observation's scale and depth information. This design greatly improved the reliability of data association algorithm.

2) Multiple hypotheses: Obstacles and tracks are associated by global minimal distance within the gate. If there is no association established. This leads to two possible situations: non-associated obstacle or non-associated track. In the first case, it is assumed that a new obstacle is just detected, and a new track needs to be created for this obstacle. In the second case, non-associated track will be preserved and updated for a short time period unless the tracking failed up to a threshold. In that case, the track would be pruned.

IV. EXPERIMENTAL RESULTS

The proposed algorithm has been evaluated on different sequences of KITTI dataset [START_REF] Geiger | Vision meets robotics: The kitti dataset[END_REF]. There are different types of road. Tracklet labels of the dataset are used as "ground truth" for a comparison and evaluation of this work.

• Dataset 1: urban road. Initialization: Set k = 0, generate a sample set {s i (t, k)} for each detected target/obstacle at current time k, where, i = 1, . . . , Ns, t = 1, . . . , N obs (k). N obs (k) is the number of detected obstacles at time k. Particle s i (t, k) is draw from Gaussian distribution around Z(t, k) 2: for k = 1 do last frame Evolution of frames 3:

for t = 1 do N obs (k) Tracking of each obstacle 4:

Compute adaptive gate GO(t, k) for data association by [START_REF] Geiger | Vision meets robotics: The kitti dataset[END_REF] for each obstacle 5:

Associate tracker with obstacle by GNN algorithm 6:

Update the weight of particles π i (t, k) by [START_REF] Hu | UV-disparity: an efficient algorithm for stereovision based scene analysis[END_REF] 7:

Re-sampling of particles s i (t, k) from current sample set according to π i (t, k)

8:

Estimate the tracker state by [START_REF] Fan | Pedestrian detection and tracking using deformable part models and kalman filtering[END_REF] 9:

Predict the state of particles s i (t, k + 1) by ( 8) if the tracker has not been associated for a period then 16:

Prune track hypothesis. 17:

end if 18:
end for 19: end for • Dataset 2: high way.

• Dataset 3: rural road • Dataset 4: busy urban road The algorithm is implemented in a standard PC with Windows 7 Enterprise OS, Intel CPU of 2.66 GHz. The development environment is MATLAB R2013b. Disparity map is obtained from LIBELAS toolkit [START_REF] Geiger | Efficient largescale stereo matching[END_REF] and particle filter functions from OpenCV [START_REF] Bradski | Learning OpenCV: Computer vision with the OpenCV library[END_REF] are integrated in the code. The run-time is about 3.4s per frame for on-road detection processing and 0.05s per frame for multiple obstacle tracking algorithm. The detection distance in disparity map is limited to 35m in front of the camera.

A. Experiments design KITTI 3D tracklet labels provide the position and motion history of the obstacles appeared in the scene. After projection onto the image plane, the tracklet 2D position could be seen as ground truth trajectories for object detection and tracking. However, there are some considerations that need to be made for our detection and tracking result to be evaluated based on tracklet labels. First, tracklets only labels vehicles and pedestrians, other type of obstacles are not included, such as traffic cones in Dataset 2 . However, they should be detected as on-road obstacles for ADAS. Second, the tracklet labels also provide off road information of obstacles which are beyond the traffic area considered in our approach. Third, our stereo vision based detection distance is set up to 35m, while the tracklet reaches to 70m. Thus, the evaluation is constrained to the intersection between our detection results and labeled tracklets. To establish a comparable evaluation platform, a sequence of 100 frames is chosen from the four datasets respectively. Then, the GNN association is applied to pair our experimental results with tracklets. Hence, on-road tracklets are picked out by data association. From the tracklets label list, all the obstacle presences within 35m distance to the camera are preserved as ground truth. To ensure the evaluation result integrity, some special cases, like false alarms, missed detection and redundant detections, are also noted manually during the experiment of detection and tracking. B. On-road obstacle detection Dataset 1 contains 150 detections on road, 136 of them are associated with trackelet, the rest are false alarms and redundant detections. Dataset 2 contains 363 detections on road, 48 of them are associated with tracklet. In Dataset 2, except for false alarms and redundant detections, 309 non-associated detections are traffic cones standing on the road which are not listed in tracklet. Dataset 3 contains 37 detections on road, 34 of them are associated with tracklet, 4 are false alarms; Dataset 4 contains 257 detections, 215 of them are associated with tracklet; the rest of them are composed of detection of traffic signs, false alarms and redundant detections.

The detection results are evaluated following 5 indicators stated in Table I: false alarm, missed detection, redundant detection and average error of position and size. According to the experiment observation, most of false alarms are caused by the obstacles standing beside the road edge, e.g. trees. In Dataset 2, there is no obstacle beside road edge, so false alarms rarely occur. Most of missed detections appear particularly in the two sides of the image on the bottom when obstacles closely pass by the camera. They are usually induced by the errors of the disparity map. Imprecise disparity map values can also lead to redundant obstacle detections. The average error (AEC) illustrates the average distance between detected obstacle centroid and labeled tracklet centroid (ground truth). In addition, the average error which measures the variation of size scale (AES) is also listed in Table I

C. Multiple obstacle tracking

Multiple target tracking can not only record and predict the motion of the obstacles, but also can deal with the occasionally missed detections and/or occlusions. As show in Fig. 4a, for the obstacle with tracklet id of 1, there's one frame of missed detection, while the track remains complete by filling the blank by prediction. During the tracking, four qualities are evaluated: rate of track fragmentation, rate of overlap, the average precision of tracker's position and size at each time step. As show in Table II, the tracking result

V. CONCLUSION

In this paper, a previous work for on-road obstacle detection using stereo vision was improved by a reliable definition of traffic area and a multiple object tracking scheme based on particle filtering. Road traffic area extraction is first integrated in the detection process. On this ROI, connectedcomponent extraction replaces Hough Transform for a flexible and fast detection of the obstacles. Moreover, multiple cues are considered to improve the detection accuracy. During the particle filtering, a self-adaptive gate for data association and dynamic filter noise function have been applied to enhance the tracking performance. Experimental results using a public dataset demonstrate that our detection and tracking system is efficient and reliable. Most obstacles appeared in 35m can well be detected and tracked. The proposed algorithm can work under dynamic circumstances without any prior-knowledge. Nevertheless, the use of 2D coordinates has a certain limit for further localization and tracking of the obstacles. Therefore, the next research step will be to focus on exploring the proposed approach from a 3D point of view.

Algorithm 1 7 :

 17 On-road Obstacle Detection Algorithm Input: -Stereo color images I l , Ir Output: Number of detected obstacles N obs , and their location information O1...N obs 1: for k =first frame do last frame Evolution of frames 2: Disparity map I∆ ← (I l , Ir) and free road surface IR; 3: Convex hull construction: IROI ← IR ; 4: U-V-disparity map on ROI: [Iu∆, Iv∆] ← (IROI , I∆) ; 5: Label the connected-regions L1,...,N in Iu∆: 6: for i = 1 do N Location extraction Extract primary position and disparity value [u l , ur, v b , hL, dL] ← (Li, Iu∆, Iv∆) 8: Generate sub-Disparity map for each object Oi: IO∆ ← (u l , ur, v b , hL)

9 :

 9 Extract obstacle by (2): IO ← IO∆ 10: Refine obstacle position [xO, yO, wO, hO, dO] ← IO, 11: if hO ≥ δ(dO) then Eliminate false alarm 12:

13 :

 13 ON obs = [xO, yO, wO, hO, dO]

Figure 2 :

 2 Figure 2: Procedure of on-road obstacle detection. For first two columns, from top to bottom: disparity map; U-disparity map within ROI (convex hull of the green area in detection results); detection result. Top-right is V-disparity map, examples of sub-regions of Disparity map for obstacle extraction are shown under it.

Algorithm 2

 2 Modified Particle Filter Algorithm 1:

Figure 3 : 2 Figure 4 :

 324 Figure 3: Examples of tracking results

  Generate new tracker sample set {s i (t, k)} for the obstacle, where, i = 1, . . . , Ns

	10:	if Non-associated Obstacle then	New obstacle
	11:
	12:	end if
	13:	end for
	14:	for Non-associated tracker do	Obstacle left the scene
	15:

Table I :

 I . The two indicators are measured on pixel-level of the 2D image plan. Evaluation of the on-road detection results

	Measures	false	missed	redundant	AEC	AES
		alarm	detection	detection
	Dataset 1	8.6%	4.0%	3.3%	5.3px	8.7px
	Dataset 2	1.1%	3.3%	1.3%	6.8px	12.4px
	Dateset 3	10.8%	8.1%	0%	9.6px	20.2px
	Dataset 4	4.6%	3.1%	4.2%	10.9px	17.1px

Table II :

 II Table II the missed detection and false alarms are effectively reduced by tracking process compare to Table I. Because tracking can fill up the gaps caused by instant missed detections, and prune the false alarms which happen occasionally. Evaluation of the multiple traget tracking results

	Measures	false	missed	track frag-	rate of	AEC	AES
		alarm	detection	mentation	overlap
	Dataset 1	5.3%	2.6%	2.36%	87.6%	5.5px	7.5px
	Dataset 2	0%	2.4%	6.81%	96.3%	8.6px	11.8px
	Dataset 3	7.8%	5.2%	0%	91.3%	19.5px	18.8px
	Dataset 4	1.5%	2.7%	2.12%	93.6%	11.0px	18.0px


Manh-Tuan Bui
email: manh-tuan.bui@utc.fr

Vincent Frémont

Djamal Boukerroui

Pierrick Letort

Deformable Parts Model for People Detection in Heavy Machines Applications

Keywords: Heavy machines, pedestrian detection, deformable part model, fisheye images, histogram of oriented gradients, latent support vector machine

In this paper we focus on the evaluation of the deformable part model (DPM) proposed by Felzenszwalb et al. [10] in the context of vision-based people detection in heavy machines applications. The proposed system uses a single fisheye camera to provide a wide field-of-view (FOV) at low cost. However, the fisheye optical distortions present several difficulties for image processing and object recognition. The DPM approach shows important flexibility when dealing with varying object's form. It gives good performances on people detection when images present strong fisheye distortions. Base on the analysis of DPM in the context of fisheye image, we proposed an adaptive detector which is more suitable.

I. INTRODUCTION

Construction sites are considered as a high risk working environment. People who work near heavy machines are constantly at risk of being struck by a working machine or its components. Accidents between machines and people represent a significant contribution to construction health and safety hazards. It is hard for drivers to keep watching all around their vehicle and fulfill their productive task at the same time, due to the complicated shapes of these machines. It is therefore mandatory to develop an advanced driver assistance systems (ADAS) to help the driver monitor the surrounding area and being able to raise a pertinent alarm when people are threatened. Therefore the main required functionality by such ADAS is the people detection. Notwithstanding many years of progress, safety system for people working around heavy machine is still an unresolved issue.

To solve the problem of safety on heavy machine, various kinds of sensors have been tested and compared, individually or combined. Range sensors, like radar, Light Detection And Ranging (Lidar) and ultrasonic [START_REF] Arras | Using boosted features for the detection of people in 2d range data[END_REF], which have good performance in detecting obstacles, are usually unable to distinguish between objects and people. Heavy machines often work in complicated terrains with a lot of nearby objects. In these situations, range sensors will trigger a permanent alarm, which is useless and annoying for the drivers. The last commonly used sensor is the camera. It offers the best option as a low-cost and polyvalent sensor. Moreover, computer vision and image processing tools provide the ability to recognize various kinds of objects, including people.

Recently, people detection systems on automobile, which share a lot of characteristics with the context of heavy machines, has known important progresses. Although the problematic is similar in both contexts, we can clearly distinguish between the two. In the automobile field, cars need to stop if there is an obstacle, no matter if it is a pedestrian or an object. The task of recognizing people is more important for heavy machines where the main requirement is human's safety. Besides, cars often operate at a higher speed and on straight ways. While it is important for the system on automobile to be able to detect people at far distances, heavy machines need a larger field of view (FOV) to cover the nearby area. Construction machines often have a complicated shape and large size, which can also benefit from the large FOV. This reason encourages us to use fisheye camera as the main sensor in our system. Objects detection in fisheye images is challenging. Unlike in perspective images, the appearance of objects captured by these cameras is strongly distorted. Wrapping the image into a local perspective image is the direct way to avoid non-perspective deformations. Unfortunately, besides adding computational load, this approach also creates undesirable effects. Daniilidis et al. [START_REF] Daniilidis | Image processing in catadioptric planes: Spatiotemporal derivatives and optical flow computation[END_REF] and Bülow [START_REF] Bülow | Spherical diffusion for 3d surface smoothing[END_REF] are the first researchers who argued that the warping of wideangle images should be avoided. Recently, there are others researchers who proposed approaches to increase matching rate of Scale-invariant feature transform (SIFT) for wide-angle images [START_REF] Hansen | Wide-angle visual feature matching for outdoor localization[END_REF], [START_REF] Lourenço | srd-sift: Keypoint detection and matching in images with radial distortion[END_REF].

On the other side, the Deformable Parts Model (DPM) of [START_REF] Felzenszwalb | Object detection with discriminatively trained part based models[END_REF] and its variants have recently gained a lot of attentions in object detection and recognition. The approach performs especially well in detecting people in hard conditions. Indeed, they are the winners of some recent Pascal-VOC detection challenges [START_REF] Everingham | The PASCAL Visual Object Classes Challenge[END_REF], [START_REF] Everingham | The pascal visual object classes (voc) challenge[END_REF] and have obtained the best scores in others benchmark [START_REF] Dollár | Pedestrian detection: An evaluation of the state of the art[END_REF]. This approach can be considered as the current state of the art methods. The DPM can represent an object model with different parts floating around their reference locations and finding the optimal part-configuration at every root position. This elegant way of representation brings a lot of benefit in detecting a person in different postures.

Our contribution in this paper lies in the evaluation of the DPM approach in the context of strong radial distortions in fisheye images. The analysis focus on the influence of different types of training dataset and in comparing the images features used in the DPM approach with the histogram of oriented gradient of Dalal and Triggs [START_REF]Finding people in images and videos[END_REF]. A fisheye images dataset in the context of heavy machine has also been created for testing purposes. A DPM-based people detector, adapted to fisheye images is proposed as the result of all the analysis. The paper is organized as follows. First, a brief description of the DPM is presented in section II. Section III discusses the model of fisheye camera and proposes an adaptive DPM-based detector. The evaluation results are presented in section IV. Conclusions and perspectives of our work will be presented in section V.

II. THE DEFORMABLE PART MODEL

We are mainly interested in 2 contributions proposed in the DPM framework: the feature vector reduction method and the deformable model trained by the latent SVM method. This section will recall the main ideas and clarify some details of these two methods in our research.

A. Histogram of oriented gradient dimensionality reduction

Histogram of oriented gradient (HOG) is undoubtedly the most used image feature in people recognition. The feature used in the DPM is a derived version. In this approach, the authors use the Principal Component Analysis (PCA) to justify the reduction of dimension of the HOG vector. In this paper, we call it reduced-HOG feature to distinguished from the classical HOG of Dalal and Triggs [START_REF]Finding people in images and videos[END_REF], denoted as conventional-HOG. Most of the parameters of the reduced-HOG are the same as the conventional-HOG. One cell includes 8 × 8 pixels, one block includes 4 cells and the number of angular bins of the vector gradient is 9.

Let C(i, j) the feature vector corresponding to the histogram of one cell before the block normalization. One cell belongs to 4 blocks around it (Fig. 1 left). For each block, the HOG vector of 9 cells C(i, j) is normalized with a different factor called gradient energy of the block N δ,γ = ( C(i, j)

2 + C(i + δ, j) 2 + C(i, j + γ) 2 + C(i + δ, j + γ)
2 with δ, γ ∈ {-1, 1}. The conventional-HOG vector represents one cell after normalization and is thus a 36-dimensional vector, where each 9 dimensions corresponds to one normalization block (Fig. 1 and Eq. 1).

H(i, j) =    H(i, j)-1,-1 H(i, j)+1,-1 H(i, j)+1,+1 H(i, j)-1,+1    =    C(i, j)/N-1,-1(i, j) C(i, j)/N+1,-1(i, j) C(i, j)/N+1,+1(i, j) C(i, j)/N-1,+1(i, j)    (1)
By applying PCA, Felzenszwalb et al. [START_REF] Felzenszwalb | Object detection with discriminatively trained part based models[END_REF] proposed that the 36-dimensional feature vector of one cell can be reduced to 11 without sacrificing important information. For the reason of simplicity, a method of projection to reduce the dimension of HOG vectors has been proposed (Fig. 1-right). The reduced vector includes 9 dimensions that correspond to 9 angular-bins and 4 dimensions that reflect the overall gradient energy in different areas around the cell C(i, j). The 9 bins in the HOG vector are normally contrast-insensitive. Felzenszwalb et al. suggest empirically that the detection performance on some object categories is improved using contrast-sensitive features, while other benefits from contrast-insensitive information [START_REF]Finding people in images and videos[END_REF]. Therefore they suggested the use of 31 dimensions feature vector (9 bins contrast-insensitive + 18 bins contrast-insensitive + 4 gradient energy).

B. Deformable part model

The deformable parts model is the most important contribution of the DPM over the conventional HOG. In the simple model of the conventional-HOG, the score of the detector can be considered as a scalar product between the feature map Φ(x) and a vector of parameter β:

f β (x) = β.Φ(x) (2)
where β is defined through the training process.

In the part-based model:

f β (x) = max x∈Z(x) β.Φ(x, z) (3)
where the latent variable z = (p 0 , p 1 , ..., p n ) with p i = (x i , y i , l i ) is the specification of i th -part configuration at the position x i , y i and scale l i . The purpose of the training process in the latent SVM process is also to learn the vector of the model parameters β = (F 0 , P 1 , ..., P n , b) where F 0 represents the root filter model and P i = (F i , v i , d i ) corresponds to the parameters of different parts. F i is denoted as the filter model of part i, v i is the anchor position of i th -part in the image frame and d i is the deformation features, a 4D-vector of coefficients, corresponding to x, y, x 2 , y 2 .

The idea behind the deformable part model is to represent the object of interest model using a lower-resolution "root" template F 0 , and a set of spatially flexible higher-resolution "part" template

P i = (F i , v i , d i ).
Each part captures local appearance properties of an object, and the deformations are characterized by the deformation costs. The deformation cost is calculated by a quadratic equation characterized by 4 coefficients d i . This equation and the optimal positions of each part in the model are obtained through the training process. Once the vector β is defined, the score of one object hypothesis can be computed as [START_REF] Felzenszwalb | Object detection with discriminatively trained part based models[END_REF]:

score(p0, p1, ..., pn) = n i=0 F i .φ(H, pi) - n i=1 di.φ d (dxi, dyi) + b ( 4
)
where H is the feature pyramid and φ(H, p i ) denotes the vector obtained by concatenating the feature vectors in the format of i th -part filter. (dx i , dy i ) = (x i , y i )-(λ(x 0 , y 0 )+v i ) gives the displacement if the i th -part relative to its anchor position and the deformation features are typically φ d (dx, dy) = (dx, dy, dx 2 , dy 2 ).

III. DPM ON FISHEYE IMAGE

A. Calibration model and field of deformation calculation

Wide-angle cameras have noticeable geometric distortions. While these distortions may be artistically interesting, it is generally desirable to remove them in many applications in computer vision. The geometric distortions include two major components: radial and tangential. Radial distortion causes image points to be translated by an amount proportional to their radial distance to the optical center. Tangential distortions (or decentering distortion) are generally less significant than radial distortions and are produced by the misalignment of the optical centers of various lens elements.

Given a real point P = (X, Y, Z) T , the undistorted point projected on the image sensor will be represented as p

= (u, v) T = X fx Z , Y fy Z T
with f x and f y the focal lengths of the camera optic. In the case of a wide-angle camera, the position of the distorted point on the image is given by:

p = ũ ṽ = p + δp + p0 (5)
where

δp = δu δv = δu (r) + δu (t) δv (r) + δv (t)
is the approximated distortion and p 0 = (u 0 , v 0 ) is the principal point of the camera. δu (r) , δv (r) represent radial distortions and δu (t) , δv (t) are the tangential distortions along the two image axes. Among different distortion models, the standard polynomial model is the most popular [START_REF] Hughes | Wide-angle camera technology for automotive applications: a review[END_REF]. In this paper, the standard polynomial model of third degree is used:

δu (r) δv (r) = ũ(k1r 2 d + k2r 4 d + k3r 6 d ) ṽ(k1r 2 d + k2r 4 d + k3r 6 d ) (6)
δu (t) δv (t) = 2p1 ũṽ + p2(r 2 d + 2ũ 2 ) p1(r 2 d + 2ṽ 2 ) + 2p2 ũṽ (7)
The optimal values of (f x , f y , u 0 , v 0 ) and (k 1 , k 2 , k 3 , p 1 , p 2 ) are estimated through a calibration process. It is a prerequisite for any accurate geometric measurements from image data. Most of calibration methods use known geometric patterns, such as corners, dots, circles, lines and other features that can be easily extracted from images. The method described in [START_REF] Heikkila | A four-step camera calibration procedure with implicit image correction[END_REF] and implemented in [START_REF] Bouguet | Camera calibration toolbox for matlab[END_REF] was used in our work.

B. Adaptive DPM-based model

Two main remarks can be drawn from the observation of the appearance of a person in fisheye images:

• The distortion in wide-angle camera is not identical over all the image area. It is particularly strong at close range and at the image boundaries. • The distortion of body's part is minor compared to the full-body appearance.

Starting from these observations and our understanding of the DPM (described in section II-B), we propose the following improvements:

• Since the DPM approach has good performance on people detection on perspective images, it should have equally good performance in detecting the person at a long distance to the camera in the fisheye context. • In the extreme-cases, when people are staying very close and at the border of the camera FOV, we will have low response of the root-filters F 0 but equally high response on part-filter F i . We could adjust the deformation features d i and v i (see Eq. 4) to adapt the deformable part model to the radial distortion. This adaptation will depend on the position of the object in the FOV of the camera.

We have also noticed that the radial distortions have minor effects on small local region because the relative dislocation between neighbor-points is small. Once the anchor positions v i is well defined, the deformation features d i are not sensitive to distortions. The second assumption has been developed into an adaptive DPM-based approach where we relocate the anchor position of each part in the deformation model. Wide-angle optics are very different from one to another. Given the height h c of the camera, an approximate size (W × H) of a person and his position angle θ and distance d to the camera, the center point of the person is given by

P0 =   X0 Y0 Z0   =   d sin θ H -hc d cos θ   (8)
The center point p i of the i th part in the model on a perspective image is defined by p i = p 0 + v i . Given a trained DPM model with the root filter of size (w × h), the center position p i of each part can be converted into a 3D position as

P i = P0 + vi.   W/w H/h 1   (9)
Both points P 0 and P i can be projected on the fisheye image using the camera projection equation and a given distortion model as p0 and pi (section III-A). The distortion of the fisheye optics can therefor be taken into account by an estimate of the displacement of each part relatively to the root part. We define ∆ṽ i = pi -p0 . The score of one object hypothesis (Eq. 4) in the adaptive model will have (dx i , dy i ) = (x i , y i ) -(λ(x 0 , y 0 ) + v i + ∆ṽ i ) gives the displacement if the i thpart relative to its anchor position. The displacement of each part ∆ṽ i depends on the position of the person in the camera coordinate so it evolves in function of the position of root filter and the scale in the feature pyramid. Value of ∆ṽ i can be computed offline and saved into lookup tables. In online detection, this approach intervenes in the final score computing using responses from the root by adding one lookup table operation. The computing cost of this adaptation is actually insignificant.

IV. ANALYSIS, EXPERIMENTS AND RESULTS

The dataset and the evaluation protocol are presented in sections IV-A and IV-B. Our analysis on testing the DPM approach in the context of fisheye images, begin with modifying the training database. The mix-training-dataset method [START_REF] Bui | People detection in heavy machines applications[END_REF] combined with the DPM are also taken into consideration. The dimensional-reduced-HOG is thorough evaluated in term of computational resource and performance. We also present the result of the adaptive DPM-based approach proposed in section III.

A. The dataset

Datasets take a very important role in the process of the development of a detection algorithm. A well defined dataset is not only useful for evaluating the approach but it also takes part in the training process to improve the performance of the detector. To the best of our knowledge, there are no others available dataset which provides at the same time synchronized fisheye images and LIDAR data. The LIDAR data provide precisely the distance of all objects present in the FOV. These data can be combine with the image sequence to lower the computation time of the detection algorithm and to reduce the false detection rate. The heavy machine context has some characteristics: outdoor light conditions, strong vibrations, and the presence of brutal shocks might make the detection process much harder. There are essential needs for a new dataset in order to identify conditions under which current detectors fail and focus the effort on these difficult cases.

The proposed dataset consists of images captured from one fisheye camera (Point Grey Firefly MV USB2.0), one conventional camera (Sony PlayStation Eye for PS3) and one range-sensor (LIDAR Hokuyo UTM-30LX-EW). The used sample frequencies are 10Hz for both cameras and 40Hz for the LIDAR (Fig. 2). These data are taken on board of a telescopic-forklift, namely a Bobcat-TL470, as shown in Fig. 3 (a professional driver has been recruited to operate the machine during the experiments). In most of the data sequences, the machine moves on a clear terrain. The scenarios defined within the experiment aim to simulate frequently meet situations on a construction site. People wore different kind of clothes, including helmet, reflective vests and civil clothes. Different situations of occlusion were also simulated. We defined 3 configurations sensors' positions denoted by 1, 2 and 3 as illustrated in Fig. 3. In the first configuration, all the sensors are in front of the machine at position 1. In configuration 2, we move the fisheye camera to position 2, pointing down to the ground with an angle of 30°. In the last configuration, the three sensors are positioned at the rear of the machine, at position 3.

The experiments are divided into 4 setups and 6 scenarios. All of the scenarios are pre-defined and took place under security control. The first three setups correspond to the three configurations of the system (Fig. 3). The acquisitions in the last setup were performed after sunset in order to simulate very low light conditions. The main source of light comes from the headlights of the machine in this experiments. In these lighting conditions, the reflective safety clothes appears with a high contrast on the image. Therefore it is an important image cue.

B. Evaluation protocol

The detection system takes an image and returns bounding boxes with corresponding scores or confidence indicators. A detected bounding box A and a ground truth bounding box B form a match if they have a sufficient overlap area. In the PASCAL challenge [START_REF] Everingham | The pascal visual object classes (voc) challenge[END_REF], the overlap criterion between the two bounding box A and B is t = A∩B A∪B > t 0 where t 0 is a threshold. t 0 = 0.5 is considered reasonable and is commonly used. The protocol of evaluation is adapted from the tool of Dollár which was used in [START_REF] Dollár | Pedestrian detection: An evaluation of the state of the art[END_REF]. As the context of heavy machines requires reducing the false detection rate, the results are presented in miss rate against false positive per image (FPPI). Only bounding boxes with a height more than 50 pixels are considered in the evaluation. Each detected bounding box may be matched once with the ground truth and redundant detections are considered as false positives (FP). We plot miss rate versus FPPI (lower curves indicate better performance) by varying the threshold of the detector. Decreasing the threshold level results in reducing the miss rate but in increasing the false positive per image rate. At a given FPPI, it is easy to compare the performance of different detectors. The log-average miss rate is used to summarize detectors' performance. The log-average miss rates is computed by averaging miss rate at nine FPPI evenly spaced in log in the range 10 -2 to 10 0 (for curves that end before reaching a given FPPI rate, the minimum miss rate achieved is used). When curves are somewhat linear in this range, the log-average miss rate is similar to the performance at 10 -1 FPPI [START_REF] Dollár | Pedestrian detection: An evaluation of the state of the art[END_REF], [START_REF] Hussein | A comprehensive evaluation framework and a comparative study for human detectors[END_REF]. The displayed legend entries are ordered by log-average miss rate from the worst to the best.

In the scope of our analysis, 3 different training datasets were taken into consideration: Inria [START_REF]Finding people in images and videos[END_REF], Inria-mix and Fisheye (see table I). The fisheye images (presented in section IV-A) are partially annotated and split into a train set and a testing one. The annotation for the ground truth of these image sequences are done by the labeling tool of Dollár et al. This tool requires the bounding box around objects in some keyframes and provides linear interpolation to infer the bounding boxes of the same object in intermediate frames. The objects can be labeled, in our case as: "person", "person sitting" and "occluded". The Inria-mix set have been build based on the Inria set. Starting from a training dataset without distortions, we replaced randomly 50% of undistorted images by distorted ones. The latter are simulated at different distortion angles. The total number of positive and negative sample images in the new training dataset is the same as the original one. For more details about the process of artificially generating distorted image samples, the reader is referred to [START_REF] Bui | People detection in heavy machines applications[END_REF].

The notation of different detectors used in this paper is based on the name of the dataset used for training, the name of the features extraction method and the name of the detection algorithm. The testing dataset has 7 image sequences of 5747 fisheye images.

C. Results

a) The conventional HOG vs the dimensional reduced HOG : In terms of computational cost, the reduction of the feature vector dimension improves slightly the speed of the DPM. From 36 to 32-dimentional vector, the gain is not significant. As a result, we observed a gain of about 8% in detection time (measured using 100 images at resolution VGA 480 × 640 on a PC).

The results shown in Fig. 4 are inconsistent. While the conventional-HOG feature works better with Inria and Inria- mix training dataset, the reduced-HOG features seems to work better when fisheye images are used for training. The combination of contrast-insensitive and contrast-sensitive features does not give any benefit in the general case. Contrast-sensitive features may be helpful when the object to background contrast does not change sign. This is the case for example, when the background color is always darker than the reflectivevest. The goal of the DPM approach is to detect numerous kind of objects so combining contrast-sensitive with contrastinsensitive in the feature vector is a relevant solution.

b) The influences of the training dataset: The first experiment involves the DPM approach trained with 3 different datasets: Inria, Inria-mix and Fisheye Dataset. The differences between the model trained with Inria and Inria-mix is not visually noticeable (Fig. 6) but there is a small boost of performance on the detection results (Fig. 5 left). This result conforms with the conclusion in [START_REF] Bui | People detection in heavy machines applications[END_REF]. Enriching the training dataset can handle the distortion on the people's appearance, even with the DPM approach.

Unfortunately, there is a trade-off between the image quality and the amount of distortion added to the training examples. This process has the drawback of introducing missing pixels that have to be filled by interpolation. This phenomenon is proportional to the amount of distortion and it has bad effects on the performance of the detector. In practice, image samples simulated at an angle superior to 60 • are unusable because the samples loose most of image details. The result of Inriamix detector in this experiment is the best that we can obtain from the mix-training-dataset approach. The degradation of image quality during the distortion process affects remarkably the performance of the detection. The model trained by 100% distorted sample images gives an unrecognizable person model and in experiment, the detector does not work at all.

In the perspective camera model, the HOG responses are strong on silhouette contours, especially the head, shoulders and feet [START_REF]Finding people in images and videos[END_REF]. Based on this observation, the DPM defines the anchor position of parts at the high respond region of the root filter. In the case of the fisheye detector model, although we can recognize the silhouette contours of a person but there are very little correlation between the perspective person root-filter and the fisheye person root-filter. The learned spatial model with anchors position defined by the root filter is thus different. Notice also the correspondence between human body parts and the model parts is not clear anymore.

It is worth mentioning that in the case of the fisheye detector, training and test images are extracted from the same dataset. The images were captured with the same camera and configuration. All these conditions improve the performance of the fisheye detector. Taking into account these remarks, we still believe that the classifier can learn to adapt to radial distortions. c) Adaptive DPM-based model: Here, instead of enriching the perspective training dataset by simulating fisheye type of distortions, we aim at taking into account the distortion in the detection model. The result in Fig. 5-right show a minor improvement in performance. Our proposed adaptive model show an advantage only on extreme case. In fact, even in the most extreme case, the dislocation of part's anchor ∆ṽ i is smaller than 40% of the root filter's size. Meanwhile, the distortion cost is calculated in the local region up to 80% root filter's size around the anchor point. When the response of the part-filter F i is high, the value will be propagated far enough to take part into the model. Relocating the anchor position is therefore only useful in difficult cases of a weak filter response.

V. CONCLUSIONS

Given that the DPM has a very good performance in detecting and recognizing objects on perspective images, we built a fisheye image dataset and evaluated the DPM approach in the context of people detection on fisheye images. It turned out that the deformable models can handle very well the strong radial distortions. We believe that it is possible to build an adaptive DPM-based detector which can solve the problem of object detection in all kinds of non-conventional cameras with known calibration information. The experiments on the proposed method show improvement in performance. One of the drawbacks of all DPM-based approaches is the heavy computational cost. We tent to improve it by combining the camera with a range sensor (Lidar or ultrasonic) to reduce the region of detection on the image. Indeed, this simple combination is helpful in accelerating the detection and reducing false positives in complex texture backgrounds.

Figure 1 :

 1 Figure 1: The dimensional-reduced-HOG computation process: Normalization blocks around one cell in HOG (left). Transformation projection to reduce dimension of the dimensional reduced HOG (right).

Figure 2 :

 2 Figure 2: Our proposed sensors system. The Lidar Hokuyo UTM-30LX (a) and its horizontal field of view (b). The camera PointGrey Firefly MV (c) and its example image (d)

Figure 3 :

 3 Figure 3: Top: Real image of the acquisition system setup in the configuration 1. Bottom: Sensor configurations map: h = 110cm and H = 210cm

Figure 4 :

 4 Figure 4: Detection performance of the dimensional-reduced-HOG vs the conventional-HOG using the root-filter only (left) and using the DPM approach (right).

Figure 5 :

 5 Figure 5: Detection performance of the DPM approach trained with different datasets(left) and the Adaptive DPM-based approach (right).

Figure 6 :

 6 Figure 6: Visualization of DPM models trained with different datasets. From left to right: Inria, Inria-mix, Inria-full-distorted and Fisheye. (a) The coarse root-filter. (b) All the higher resolution part-filters superposed on the rootfilter. (c) The spatial model of the location of each part relative to the root.

Table I :

 I Training datasets characteristic

	Training dataset	Positive samples	Negative sample
	Inria	1371 samples from 614 images	1218 images
	Inria-mix	1371 samples from 614 images	1218 images
	fisheye	1520 samples from 810 images	1202 images


Dingfu Zhou
email: dingfu.zhou@hds.utc.fr

Benjamin Quost
email: benjamin.quost@hds.utc.fr

Vincent Frémont
email: vincent.fremont@hds.utc.fr

Soft Label Based Semi-Supervised Boosting for Classification and Object Recognition

Supervised classification algorithms such as Boosting and SVM have achieved significant success in the field of computer vision for classification and object recognition. However, the performance of the classifier decreases rapidly if there are insufficient labeled training samples. In this paper, a semi-supervised boosting algorithm is proposed to overcome this limitation. First, a few labeled instances are use to estimate probabilistic class labels for unlabeled samples using Gaussian Mixture Models after a dimension reduction step performed via Principal Component Analysis. Then, we apply a boosting strategy on decision stumps trained using the soft labeled instances thus obtained. The performances of our strategy are evaluated on several state-of-the-art classification datasets, as well as on a pedestrian detection and recognition problem. Experimental results demonstrate the interest of taking into account additional data in the training process.

I. INTRODUCTION AND RELATED WORK

In the last decade, machine learning techniques have proven their interest for vision-based object detection. The success of these methods own to both the advance in machine learning and the breakthrough of object descriptors. Advances in image processing have established local [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] and context-based [START_REF] Ding | Contextual boost for pedestrian detection[END_REF] features as key approaches for object detection and recognition. Besides, recent works have shown the great interest of using Support Vector Machines (SVMs) [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF], Boosting [START_REF] Viola | Rapid object detection using a boosted cascade of simple features[END_REF], Random Forests [START_REF] Breiman | Random forests[END_REF] and Neural networks [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] for this purpose. Note, however, that all of these methods are sensitive to the size of the training set. More precisely, due to the phenomenon known as "curse of dimensionality", the amount of training data at hand may need to be huge.

In some practical problems there are insufficient labeled training samples, whereas labeling training data is tedious and time consuming. Many approaches have been proposed to overcome this problem by introducing an amount of unlabeled samples to help construct the classifier. This type of approach is referred to as semi-supervised learning (SSL) [START_REF] Chapelle | Semisupervised learning[END_REF]. The essential idea of SSL is to utilize the knowledge from both labeled and unlabeled training samples to build a better decision boundary between different classes. A number of approaches have been proposed e.g., self-training [START_REF] Besemer | Genemarks: a self-training method for prediction of gene starts in microbial genomes. implications for finding sequence motifs in regulatory regions[END_REF], co-training [START_REF] Blum | Combining labeled and unlabeled data with co-training[END_REF], transductive support vector machines [START_REF] Joachims | Transductive inference for text classification using support vector machines[END_REF], graph-based methods [START_REF] Pedro | Efficient graphbased image segmentation[END_REF], semi-boosting [START_REF] Leistner | Semi-supervised boosting using visual similarity learning[END_REF], [START_REF] Kumar Mallapragada | Semiboost: Boosting for semi-supervised learning[END_REF], [START_REF] Chen | Semi-supervised learning via GENOMES regularized boosting working on multiple semi-supervised assumptions[END_REF] and semi-supervised Random Forests [START_REF] Leistner | Semi-supervised random forests[END_REF]. In most of the SSL approaches, one or several semi-supervised assumptions, such as smoothness, cluster and manifold assumptions, are used to assign pseudo labels to the unlabeled samples. Then pseudo-labeled training instances are used to build the classifier. Note that SSL may be formalized using soft labels [START_REF] Wang | A weakly supervised approach for object detection based on soft-label boosting[END_REF]: for example, each training instance may be associated with a probability distribution over the set of classes. The probability of a given class label may be interpreted in a frequentist setting as the proportion of similar training instances having this label. Obviously, 0/1 probabilities may be used when the class label is known with certainty.

In a way, assigning soft labels to the (unlabeled) training instances corresponds to integrating some prior knowledge in the training set. In [START_REF] Schapire | Boosting with prior knowledge for call classification. Speech and Audio Processing[END_REF] and [START_REF] Lefort | Weakly supervised classification of objects in images using soft random forests[END_REF], the authors propose to design an objective function before using boosting. Alternatively, soft labels may be estimated from the distribution of the data. In [START_REF] Wang | A weakly supervised approach for object detection based on soft-label boosting[END_REF], soft class labels are estimated using a kernel density estimation (KDE) approach.

In this work, we use HOG descriptors to obtain training instances from the raw images. Since clustering techniques are known to be sensitive to the lack of available data, especially in high dimensions, we first select a limited amount of relevant features using PCA. We experimentally show that this step has very little influence on the clustering step. Then, we fit a Gaussian Mixture Model (GMM) [START_REF] Bilmes | A gentle tutorial of the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov models[END_REF] to the training data, and we use the posterior probabilities as soft labels. Note that as a parametric density estimation technique, GMM are known to be more robust to the lack of training data than kernel density estimation. Eventually, in the classification step, we propose to boost decision stumps trained from the probabilistic labeled training instances [START_REF] Gabbert | Decision trees from uncertain learning sets[END_REF].

Our paper is organized as follows. Section II presents the soft label estimation step using PCA and GMM. In Section III, we remind how probabilistic labels may be used in decision tree inference, and we present our boosting strategy for such a base classification algorithm. Section IV details our experiments. Eventually, we conclude the paper with some perspectives of work in Section V.

II. SOFT CLASS LABEL ESTIMATION

In our approach, the training data are expressed as D = D L +D U , where D L and D U are the labeled and unlabeled instances respectively.

D L = {(x 1 , y 1 ), • • • , (x n , y n )} ⊆ X ×Y, in which x i ∈ R d stands
U = {x 1 , • • • , x m } ⊆ X ,
in which m is the number of unlabeled instances. We also use N = m + n to represent the number of all the training instances.

A. HOG Feature Extraction

HOG features have been recognized as one of the most significant features for pedestrian detection. Owing to its excellent performance, it is also widely used for other object detection in computer vision. As described in [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF], the standard HOG descriptor for a 64*128 window size pedestrian sample will have 3780 dimension, in which we chose block-size as 16×16, cell-size as 8×8 and block-stride as [START_REF] Besemer | Genemarks: a self-training method for prediction of gene starts in microbial genomes. implications for finding sequence motifs in regulatory regions[END_REF][START_REF] Besemer | Genemarks: a self-training method for prediction of gene starts in microbial genomes. implications for finding sequence motifs in regulatory regions[END_REF]. Compare to the original image pixels, this high dimension HOG descriptor is also acceptable. However, according to [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF], the extracted HOG features also include a lot of redundant information. They have proved that a classifier trained by PCA-based reduced low dimension features has a similar performance with the classifier trained with the high dimension features. Moreover, a lower dimensional features leads to a model with fewer parameters and speeds up the learning and detection algorithms. PCA-HOG feature has also been used in [START_REF] Kobayashi | Selection of histograms of oriented gradients features for pedestrian detection[END_REF] and [START_REF] Zeng | Robust head-shoulder detection by pca-based multilevel hog-lbp detector for people counting[END_REF] for pedestrian detection and people counting. In our approach, we first extract standard HOG descriptors from all the training samples, then PCA is applied to generate lower dimension features. Experimental results detailed in Section IV, show that a PCA-HOG feature with 20-dimension can achieve good clustering results.

B. GMM based Class Label Estimation

A Gaussian Mixture Model (GMM) is a parametric density estimation technique, in which the distribution of the data is supposed to be a mixture of g multivariate Gaussians:

p(x|Ψ) = K k=1 θ k N (x|µ i , Σ i ) (1)
where, θ k is the prior probability of the kth component of the model satisfying that

K k θ k = 1 and θ k > 0;
x is a d-dimensional data feature. Each component model is parametrized by a d×1 mean vector µ k and a d×d covariance matrix Σ k . Classically, a set of unlabeled instances is considered, and maximum likelihood estimates of the parameters are computed using the EM algorithm [START_REF] Arthur P Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF], [START_REF] Richard | Mixture densities, maximum likelihood and the em algorithm[END_REF]. This algorithm considers, for each instance x i , g latent (random) variables z ik indicating from which component the instance was generated. Given a set of initial parameters Ψ = {θ k , µ k , Σ k }, k = 1, . . . , K, the algorithm iterates two successive steps:

• in the E step, the expectation of z i is computed given the current fit of the parameters:

π ik = θ k N (x|µ k ,Σ k ) g l=1 θ l N (x|µ l ,Σ l ) .
• in the M step, new estimates of the parameters are computed. When labeled instances are available, the corresponding latent variables are assumed to be known, and the expectations π ik need not to be estimated. The advantage of using such labeled instances is twofold. First, they can be used to compute (nontrivial) starting values for the parameters. Furthermore, they may guide the algorithm towards a desired solution. After the optimal parameters (Ψ

o = {θ o k , µ o k , Σ o k }
) of the GMMs model have been obtained, the optimal posterior probabilities of the unlabeled samples for both classes can also be computed via Bayes formulas:

p(y i = k|x i ) = θ o k N (x|µ o k , Σ o k ) 2 l=1 θ o l N (x|µ o l , Σ o l )
For the covariance matrices Σ, both diagonal and full covariance matrices have been investigated in our study. A GMM using diagonal matrices typically need less training data and is faster than a GMM with full matrices. The experimental results given in Section IV, also show that the diagonal matrices outperforms the full matrices for the class label estimation.

III. SEMI-SUPERVISED BOOSTING WITH PROBABILISTIC LABEL

After obtaining class probabilities for all the unlabeled samples, our classification problem can be expressed in the following form:

D = {(x 1 , π 1 ), • • • , (x N , π N ), where π i = {π i,k }, k = 1, 2, π i,k = P (Y = k|x)
being the probabilistic label estimated by GMM for class k.

A. Soft Decision Tree

Decision stumps are commonly used as base classification algorithm (or weak learner) in boosting algorithm for object detection. Here, we describe how such classifiers (and more generally decision trees) may be trained using probabilistic labeled data. More details may be found in [START_REF] Gabbert | Decision trees from uncertain learning sets[END_REF], [START_REF] Lefort | Weakly supervised classification of objects in images using soft random forests[END_REF]. We remind that this classification algorithm aims at classifying the data by splitting the feature space into subsets. For a given node, the frequency of each class is first estimated, as the proportion of the instances belonging to this class falling into the node. Then, the Gini index is computed over this distribution of frequencies. The index thus obtained may be used to evaluate the quality of a split. Let s d be a candidate split value according to the dth feature (instances such that

x d i < s d , resp. x d i ≥ s d ,
thus fall in the left-hand child node, resp. right-hand one). Then, the best split is the one that maximizes the purity criterion of both left-hand and right-hand children.

In the case of probabilistic labeled instances, the Gini index may still be used: indeed, the frequencies of the classes are obtained by averaging the probabilistic labels of the instances falling into the node. In the following text we take this soft decision stump as the weak learner algorithm.

B. Soft Label based Boosting

Boosting aims at training an accurate classifier (or strong learner) by combining a number of simple classifiers (or weak learners). Each weak learner is chosen so as to optimally reduce the training error. In this paper, we focus on the AdaBoost algorithm [START_REF] Freund | A desicion-theoretic generalization of on-line learning and an application to boosting[END_REF] for binary classification problems here. In AdaBoost, the decision H(x) made by the strong learner for a given instance x is obtained by combining the weak learners outputs h t (x) ∈ {0, 1} in the following way:

H(x) = 1, 0 if T t=1 α t h t (x i ) ≥ 1 2 T t=1 α t otherwise
where α t > 0 the weight of the tth weak learner (t = 1, . . . , T ), and T is the number of weak learners. During the training process, a set of weights are assigned to the train samples. In order to use the soft labeled training instances in the AdaBoost algorithm, several parts should be modified: 1) the weak learner should be able to handle both hard and soft labeled training instances, 2) the classification error of the weak learner should be computed for both hard and soft labeled samples. The first issue may be tackled using the soft decision tree algorithm in Section III-A. In the following, we will introduce how to calculate the classification error for soft labeled instances.

The cost-weighted misclassification error for instance x can be written as:

E Y |X=x L(y|h t (x)) = η(x).I ht(x)≤0.5 + (1 -η(x)).I ht(x)>0.5
(2) where I condition stands for the indicator function (it is equal to 1 if the condition given is met, and 0 otherwise). In Eq. ( 2), we only consider the probability of class 1 estimated by the GMM, noted as η(x) = P (Y = 1|x) (since we consider a two-class problem, the probability of class 2 is directly deduced). In the boosting algorithm, we need to compute the classification error ǫ(t) for each weak learner, which we define as a weighted version of the misclassification cost:

ǫ(t) = 1 N N i=1 w t i .ǫ xi (3)
where w t i is the weight associated with instance x i at the tth iteration, and ǫ xi = π i1 I ht(xi)<0.5 + (1π i1 )I ht(xi)≥0.5 is the misclassification cost for instance x i . Note that d er (x i ) is also used for updating the weight w t i along the iterations of the boosting algorithm:

w t+1 i = w t i β(t) 1-ǫx i (4)
where β(t) is defined as

β(t) = ǫ(t) 1-ǫ(t)
. Eventually, the weight α t of the weak learner is classically defined as α t = log 1 β(t) . Algorithm 1 summarizes the main steps of our soft label based boosting algorithm.

Remark that the proposed soft label based boosting algorithm can take both hard and soft labeled samples as inputs. If all the samples are hard labeled, it degrades into the original AdaBoost algorithm. Note furthermore that an instance with a probabilistic label close to the uniform probability distribution will have a significant classification error even if it is well classified. This reflects the difficulty to learn from instances from uncertain information. In order to let the classifier focus on the training instances classified with confidence, we set the initial weights of the instances to w 1 i = 2 * abs(π i1 -0.5).

Algorithm 1 Soft label based Boosting algorithm.

Inputs:Training data (x i , π i ), i = 1, • • • , N , π i,1 = P (y i = 1|x i ).
T , number of weak learners, weak learner algorithm . Initial: sample weight vector: w 1 i = 2 * abs(π i,1 -0.5). For t=1:T 1) Normalization sample weight:

w t i = w t i N i=1 w t i ;
2) Call W eakLearner with distribution w t i , get back a hypothesis h t : X → {0, 1}; 3) Calculate the expected classification error for sample x i :

ǫ xi = π i,1 I ht(xi)≤0.5 + (1 -π i,1 )I ht(xi)≥0.5
4) Calculate the expected error for weak learner t:

ǫ t = N i w t i .ǫ xi 5) Set β(t) = ǫt
1-ǫt and α t = log 1 β(t) ; 6) Update new weight to each sample:

w t+1 i = w t i .β(t) 1-ǫx i end Output:
The final decision:

H(x i ) = 1, 0, if T t=1 αt.ht(x i ) ≥ 1 2 T t=1 αt otherwise
For a hard labeled sample, the initial weight is 1, while a 0 initial weight will be given to a sample who has equal classes probabilities.

IV. EXPERIMENTS AND ANALYSIS

A. Performance of Probabilistic Label Estimation

In order to evaluate the performance of the GMM based probabilistic class labels estimation, we test our algorithm on the CVC [START_REF] Gerónimo | 2d-3d-based on-board pedestrian detection system[END_REF] and INRIA [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] pedestrian dataset. The CVC dataset has 3172 positive (pedestrian) and 15150 negative (non-pedestrian) samples while the INRIA dataset has 3542 positive and 4560 negative samples (randomly extracted from the negative images). In these experiments, all the samples are randomly separated into labeled and unlabeled with a ratio value γ, which is defined as: γ = m n+m , where, n and m represent the number of labeled and unlabeled samples respectively. First, the HOG features [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] are computed for each sample, then PCA is applied to reduce the features dimension to a lower one denoted as d. In this experiment, we also compare the GMM performances with diagonal and full covariance matrices. Details about the two different experiments are described below: the CVC dataset as the labeled data. Then we randomly divide the INRIA dataset into two parts with a ratio γ.

We keep a proportion of γ as the unlabeled data, while the rest is considered as labeled data. In order to evaluate the performance of the label estimation, we use a simple and rough method. A class label with the highest probability is assigned to each unlabeled sample. All the experiments have been repeated five times, and the average error rate is defined as: F -ave = mmis m , where, m mis is the number of samples that have been misclassified by the GMM.

Fig. [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] describes the average error rates of the two experiments. In the figure, the solid and dotted lines represent the results with full and diagonal covariance matrix respectively. In experiment 1, the full matrix performs a little better than the diagonal matrix when there are enough labeled samples, but it decreases rapidly with the decreasing of labeled samples. However, the diagonal matrix keeps stable with the decreasing of labeled samples. In the experiments 2, we have found something different. The performance of the diagonal matrix highly depends on the feature dimension, and we obtained that the diagonal matrix together with 20-dimension feature gives the best result. sample that can be positive or negative. The X-axis values are the estimated probabilities of a sample obtained by the GMMs model. In order to properly display the distribution of the estimated class probabilities, we uniformly divide interval [0, 1] (the class probability belongs to [0, 1]) into 10 equal parts and record the number of samples falling into each sub-interval. The Y-axis represents the ratio of the samples that fall into the corresponding sub-interval. From Fig. [START_REF] Ding | Contextual boost for pedestrian detection[END_REF], we can see that the full matrix works well for the positive samples compared to the diagonal matrix. However, it gives worse performances for the negative samples, especially when some of them have been set with high probabilities for the incorrect class (leftmost side of Fig. (2)-(b)). Compared to the full matrix, one can notice that the diagonal matrix gives an eclectic result. It has a weaker ability to classify the positive samples while it makes less clustering errors on the negative samples. Additionally, the diagonal matrix gives less average error for all the samples (see in Fig. (1)).

B. Pedestrian Recognition

We also took the INRIA pedestrian dataset to evaluate the algorithm for pedestrian recognition. Firstly we randomly separated the data into training and testing with a ratio 3 : 1. Then we randomly selected a part of the training data as the labeled sample and left the remaining data as unlabeled data. We followed the method in [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] to represent each sample by a 3780-dimensional HOG feature. Then a PCA has been applied to the resulting vector to obtain a 20-dimension feature for each sample. Guided by the labeled samples, the probabilistic class labels for unlabeled training samples are estimated by GMM using the EM algorithm. Here, the reduced 20di- mension features are only used for estimating the soft class labels. In boosting algorithm, the original HOG features are used for recognition. The AdaBoost algorithm is used as the baseline and is trained with few labeled samples. Four different kinds of classifiers have been designed in this experiment: 1) classifier 1 is a classical AdaBoost classifier trained using only few labeled samples, 2) classifier 2 is a GMM classifier using both the labeled and unlabeled samples, 3) classifier 3 is the proposed semi-supervised boosting classifier trained using all the hard and soft labeled samples, 4) classifier 4 is also the proposed semi-supervised boosting classifier trained using all the labeled and some selected unlabeled samples. Classifier 1, 3 and 4 use stump decision tree as the weak learner. Hard label decision tree is used in Classifier 1, while soft decision tree is applied in both Classifier 3 and 4.

All the experiments have been repeated five times and we calculated the average recognition rate to draw the Fig. [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF]. From this figure, we can see that the recognition rate of Classifier 1 (blue line) increases obviously with the increasing of labeled training samples, however, the recognition rate of Classifier 2 does not increase with the increase of the labeled samples. The green line represents the performance of Classifier 3. Although it increases with the increasing of labeled samples, it gives worse performance than Classifier 1 which has been trained using only few labeled samples. Through additional experiments we found that the weak performance of Classier 3 is mainly caused by the training samples who have been assigned with wrong probabilistic class labels. The GMM can give the right probabilistic class label for most of the unlabeled samples, while it also gives wrong label to some training samples. These wrong probabilistic labeled samples have a great influence on the recognition rate of Classifier 3.

In order to reduce this negative effect, we should remove some of unlabeled samples with wrong probabilistic labels. A effective strategy has been proposed as: we can predict the class labels for all the unlabeled training samples using GMM method (Classifier 2) and at the same time we can also predict another class label for each unlabeled training sample using Classifier 1 (because it is trained using only the labeled samples). For an unlabeled sample, we consider its probabilistic class label is reliable if its predicted class labels from Classifier 1 and 2 are consistent. Otherwise, it is considered as unreliable. Classifier 4 is trained using all the labeled samples and some selected unlabeled samples with reliable probabilistic class label. Although our strategy was not to get rid of all the wrong probabilistic samples from the training data, it still improve the recognition rate obviously. This improvement can be seen from the red line of Fig. [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF]. By comparing the red and blue lines in Fig. (3), we can find that our proposed boosting algorithm with additional soft labeled sample gives a better recognition rate than AdaBoost trained with only labeled samples.

C. Data Classification

In order to evaluate the proposed semi-supervised boosting algorithm further, we test our approach on public binary classification datasets in the LibSVM repository [START_REF] Chang | LIBSVM: a library for support vector machines[END_REF] As in the previous section, all the data have been randomly divided into training and testing data with a proportion 3:1. In the training data, a small part of the training samples are selected as labeled samples and the rest are considered as unlabeled samples. In this experiment, only three classifiers are designed: 1) classifier 1 is a classical AdaBoost classifier trained using only few labeled samples, 2) classifier 2 is a GMM classifier using both the labeled and unlabeled samples, 3) classifier 3 is the proposed semi-supervised boosting classifier trained using all the labeled and some selected unlabeled samples and the selection strategy is the same as in Sec. IV-B. For each dataset, the ratio of labeled samples changed from 0.05 to 0.55. Fig. [START_REF] Viola | Rapid object detection using a boosted cascade of simple features[END_REF], show the classification rate of the four different datasets in each sub-figure respectively. From the figure we can easily find that our proposed semi-supervised boosting algorithm (red line) has a higher classification rate than classical AdaBoost algorithm (blue line) most of the time. The additional unlabeled samples help to improve the classification rate in our semi-supervised boosting algorithm.

V. CONCLUSIONS AND FUTURE WORK

In this paper, we proposed a semi-supervised boosting algorithm that uses few labeled samples and an amount of unlabeled samples. Our approach provides a novel framework to take hard and soft labeled samples into the boosting algorithm. In this paper, the soft class labels of the unlabeled samples are estimated from GMM using the labeled samples. Experimental results on public datasets have shown that the performance of a boosting classifier could be improved by adding soft labeled samples. We also have found that PCAbased HOG features dimension reduction has a little influence on the results of probabilistic class labels estimation. This interesting phenomena makes it is possible to estimate soft class labels for pedestrian samples with HOG features. Finally, the results show that the recognition rate can be improved by adding an amount of soft labeled samples.

On the other hand, the soft class labels could be replaced by other prior information. In the future, we plan to design our approach to improve the classifier's performance by using additional samples with weak prior knowledge (e.g., human beliefs or uniform class probabilities). For the pedestrian detection problem, we can apply our approach to improve a generic pedestrian detector (which is trained on one or several generic public datasets) by adding plentiful unlabeled samples from some specific scenes.

  for the ddimensions feature of ith instance, y i ∈ Y is the class label and n is the number of labeled instances. Here, we only focus on the binary classification problem, therefore Y = {1, 2}. The unlabeled samples are expressed as D

Figure 1 :

 1 Figure 1: Performances of GMM based probabilistic class labels estimation.

Fig. ( 2 )

 2 shows more details about the GMM based probability label estimation results. Here, due to space limitations, we just show the results in experiment 2 with d = 20 and λ = 1.0. The red and blue columns represent the actual class of the

Figure 2 :

 2 Figure 2: Distribution of estimated probabilistic class labels.

Figure 3 :

 3 Figure 3: Recognition rates of four different classifiers. We choose T = 500 for all boosting classifiers in this experiment.

Figure 4 :

 4 Figure 4: Classification rate of four different Dataset. All the experiments have been repeated 50 times. We choose T = 100 for all boosting classifiers in this experiment.

  . Four different dataset have been chosen test our algorithm. A general description of the data is in the following table.

	Dataset	Size of the data Attribute dimension
	Australian	690	14
	Diabetes	768	8
	Heart	270	13
	Ionosphere	351	34