Remi Trichet
email: remi.trichet@gmail.com

Francois Bremond
email: francois.bremond@inria.fr

LBP Channels for Pedestrian Detection

This paper introduces a new channel descriptor for pedestrian detection. This type of descriptor usually selects a set of one-valued filters within the enormous set of all possible filters for improved efficiency. The main claim underpinning this paper is that the recent works on channel-based features restrict the filter space search, therefore bringing along the obsolescence of one-valued filter representation. To prove our claim, we introduce a 12-valued filter representation based on local binary patterns. Indeed, various improvements now allow for this texture feature to provide a very discriminative, yet compact descriptor. Filter selection boasting new combination restrictions as well as a reverse selection process are also presented to choose the best filters. experiments on the INRIA and Caltech-USA datasets validate the approach.

Introduction

With the rise of automotive, the growing need for behaviour-oriented surveillance and home-monitoring applications, pedestrian detection is one of the computer vision hot topics. With over a third [START_REF] Benenson | Ten years of pedestrian detection, what have we learned? ECCV[END_REF] of the pedestrian literature dedicated to it, features largely contributed to the progresses in the field. Integral channel features [START_REF] Dollar | Integral channel features[END_REF] are one of the keystones of this long, bumpy road. In their original work, the authors employs integral images to integrate many cues, such as gradients and colour. Each of these cue is dubbed a channel. The final descriptor is obtained by convolving a set of patches (named filters) with the different cues. By summarizing a swarm of channel region information, it provides a robust and compact descriptor. Due to their efficiency and reasonable detection time, channel features quickly became a reference. Over the years, many extensions were proposed, in terms of either speed [START_REF] Dollar | The fastest pedestrian detector in the west[END_REF] or performance [START_REF] Costea | Word channel based multiscale pedestrian detection without image resizing and using only one classifier[END_REF][START_REF] Benenson | Ten years of pedestrian detection, what have we learned? ECCV[END_REF][START_REF] Zhang | Informed haarlike features improve pedestrian detection[END_REF][START_REF] Kiani | Multi-channel correlation filters for human action recognition[END_REF][START_REF] Nam | Local decorrelation for improved pedestrian detection[END_REF][START_REF] Zhang | Filtered channel features for pedestrian detection[END_REF]. In all these works, the response of one filter region is summarized with one value, by averaging or subtracting means. Despite their low discriminative power, one-valued filters were originally chosen to explore more deeply the huge amount of possible filters and select the best performing ones. Indeed, the memory consumption bounds the process and the size of the full descriptor is the number of filters times their response size. So, the bigger the full filter response will be, the harder it will be to search the feature space for the discriminative filters. On the other hand, individual filter responses would be more discriminative if characterized by more values. Moreover, since channel features were first introduced, heuristics [START_REF] Zhang | Informed haarlike features improve pedestrian detection[END_REF][START_REF] Benenson | Ten years of pedestrian detection, what have we learned? ECCV[END_REF][START_REF] Kiani | Multi-channel correlation filters for human action recognition[END_REF][START_REF] Nam | Local decorrelation for improved pedestrian detection[END_REF] have been brought up to narrow down the filter search space. Therefore, in this paper, we propose to study the trade-off between these two factors and to show through the introduction of a new channel descriptor that several-valued filters can now reliably be extracted. This new LBP-based channel descriptor outperforms channel features [START_REF] Dollar | Integral channel features[END_REF] while requiring a fraction of the original LBP memory footprint. Uniform patterns [START_REF] Ojala | Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[END_REF] and Haarbased LBP [START_REF] Corvee | Haar like and lbp based features for face, head and people detection in video sequences[END_REF] are employed to shrink the filter dimension in accordance to our needs. Also, cell stacking and new filter combination restriction based on proposal window coverage are successfully applied. Finally, a more reliable feature selection technique is introduced. Experiments on two major pedestrian detection datasets demonstrate the robustness and discriminative power of our LBP-channel descriptor compared to the state-of-art. Our contribution is 2-fold:

• The introduction of LBP-based feature channels. This descriptor is one-of-a-kind in several respects. Its size is drastically reduced via the use of uniform patterns [START_REF] Ojala | Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[END_REF] and Haar-like descriptors [START_REF] Corvee | Haar like and lbp based features for face, head and people detection in video sequences[END_REF]. Moreover, the usage of stacked cell sizes confers to the descriptor an increased discrimination power without changing its overall size.

• We introduce reversed feature selection to construct a lower dimension final descriptor without harming its discriminability. Also, new coverage-based restrictions of filter combination allow for a sharper search.

The rest of this paper is organized as follows. Section 2 reviews the related work. Section 3 describes in details the LBP-channel feature while section 4 proves its efficacy through experimentation. Section 5 concludes this document.

Related Work

Since the Viola and Jones first pedestrian detector [START_REF] Viola | Robust real-time face detection[END_REF], the most blatant progresses in pedestrian detection concerns features. HoGs [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] are arguably the most prominent. These gradient based descriptors, densely extracted over the image at several scales are fast to compute and particularly discriminative for the human detection task. While the bulk of the literature uses this landmark feature, other noticeable work has been produced.

Another section of the literature focuses on adding extra, complementary features to the original HoG ones. In particular, the combination with colour is the first choice. Luv colour patches [START_REF] Dollr | The fastest pedestrian detector in the west[END_REF] self-similarity on colour channels [START_REF] Walk | New features and insights for pedestrian detection[END_REF] have been proposed, leading to substantial improvement. The addition of flow features [START_REF] Walk | New features and insights for pedestrian detection[END_REF][START_REF] Park | Exploring weak stabilization for motion feature extraction[END_REF][START_REF] Zhang | Filtered channel features for pedestrian detection[END_REF] has also shown success.

More recently, [START_REF] Dollar | Integral channel features[END_REF] proposed integral channel features. This approach integrates many cues, like colour or gradients, within integral images, and describes any pedestrian proposal with a swarm of convolved patch regions (named filters) over these cues. Due to its success, this efficient descriptor had many extensions. [START_REF] Dollar | The fastest pedestrian detector in the west[END_REF] improved its speed considerably, [START_REF] Costea | Word channel based multiscale pedestrian detection without image resizing and using only one classifier[END_REF] quantized the outcome of each filter therefore reducing the final descriptor size while increasing its summarization capabilities. Significant work also focussed on selecting the best filters among the plethora of possibilities. [START_REF] Benenson | Ten years of pedestrian detection, what have we learned? ECCV[END_REF] used square filters, [START_REF] Zhang | Informed haarlike features improve pedestrian detection[END_REF] combined it with Haar-type filters and cleverly derived their filters from pedestrian body parts templates to reduce the filter search space. Correlation filters [START_REF] Kiani | Multi-channel correlation filters for human action recognition[END_REF] efficiently dealt with correlations in the frequency domain whereas [START_REF] Nam | Local decorrelation for improved pedestrian detection[END_REF] learnt the filter bank as the base of a PCA over the training set.

Another recent class of approach [START_REF] Zeng | Multi-stage contextual deep learning for pedestrian detection[END_REF][START_REF] Luo | Switchable deep network for pedestrian detection[END_REF][START_REF] Ouyang | Joint deep learning for pedestrian detection[END_REF][START_REF] Angelova | Real-time pedestrian detection with deep network cascades[END_REF][START_REF] Zhang | Is faster r-cnn doing well for pedestrian detection[END_REF][START_REF] Cai | Learning complexity-aware cascades for deep pedestrian detection[END_REF]] extracts features through the training of a deep neural network. the significant performance increase that the resulting features provide has led a drastic shift in the field toward this deep learning approaches. In this domain as well, researchers enforce contextual features learning in a semi-supervised way. [START_REF] Zeng | Multi-stage contextual deep learning for pedestrian detection[END_REF] use a set of activation maps to learn each layer's features. [START_REF] Luo | Switchable deep network for pedestrian detection[END_REF] train a Restricted Boltzmann Machine providing both, images and corresponding labels as observed variables in a supervised manner. Finally, [START_REF] Ouyang | Joint deep learning for pedestrian detection[END_REF], independently define and learn 4 distinct pedestrian detection components: feature extraction, deformation handling, occlusion handling and classification. Ensemble learning has also been harnessed to guide the technique toward more meaningful features. Cascade of deep networks [START_REF] Angelova | Real-time pedestrian detection with deep network cascades[END_REF], bootstrapped boosted forests [START_REF] Zhang | Is faster r-cnn doing well for pedestrian detection[END_REF] and a pool of both, hand-crafted and CNN features [START_REF] Cai | Learning complexity-aware cascades for deep pedestrian detection[END_REF] have been employed for this purpose.

LBP-based channel extraction

This section describes how our LBP-channel descriptor is extracted. We first recall the basics of the LBP descriptor and its variants before explaining how we adjust it to the specificities of channel-structured pedestrian description. Finally, the last subsection details our feature selection methodology that minimizes the performance loss while shrinking the descriptor size.

LBP descriptor

Local Binary Patterns are binary descriptors stemming from texture analysis [START_REF] Hussain | Feature sets and dimensionality reduction for visual object detection[END_REF]. Despite their high memory impact they became a state-of-the-art descriptor for pedestrian detection [START_REF] Wang | An hog-lbp human detector with partial occlusion handling[END_REF][START_REF] Hussain | Feature sets and dimensionality reduction for visual object detection[END_REF] due to their fast calculation and their good performance for this task. Also, their ability to characterize texture makes them complementary to the HoG edge descriptor [START_REF] Wang | An hog-lbp human detector with partial occlusion handling[END_REF]. The LBP descriptor encapsulates the local intensity variations within the region to describe. Each local binary pattern, in the original descriptor, quantizes the intensity values within regions {r} of 3×3 cells, comparing the 8 border cells intensity values to the center one:

LBP (r) = ∑ i=1...8 S(I(C 0 ) -I(C i ))2 i S(x) = { 1 if |x| > T 0 otherwise (1)
with T a noise filtering threshold and I(C) the average intensity over the cell C. The 8-bit pattern is then transformed into a decimal by binomial weighting. Similarly to HoG [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF], an histogram represents the LBP pattern frequencies within a block regions b:

h(i) = ∑ r∈b and LBP (r)=i 1 (2)
and the final descriptor concatenates all the histograms computed over a grid of blocks. The original L 1 -square block normalization works best for most of the applications but is seldomly outperformed by the L 2 -norm [START_REF] Geismann | Speeding up hog and lbp features for pedestrian detection by multiresolution techniques[END_REF]. This original descriptor has been improved in several respects since its introduction in computer vision problems. In order to extend the discrimination power of the descriptor [START_REF] Tan | Enhanced local texture feature sets for face recognition under difficult lighting conditions, transactions in image processing[END_REF] extended the LBP to 3-values code. This local trinary pattern (LTP) is respectively quantized to +1 or -1 depending on whether the compared pixel intensities are above or below T ; its value remains 0 otherwise.

Memory consumption easily becomes large since there are 256 possible values for a typical 8-bit pattern, therefore leading to 256-valued block histograms. Important work has been achieved in reducing the number of available patterns without loss of discrimination. [START_REF] Ojala | Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[END_REF] stated that most patterns occurring in natural images have at most two transitions within their 8 values. They restricted the pattern extraction to this LBP subset that they called uniform patterns. [START_REF] Geismann | Speeding up hog and lbp features for pedestrian detection by multiresolution techniques[END_REF] further reduced the number of available patterns power by considering inverse patterns (with the exceptions of the informative 00000000 and 11111111 patterns) identical. In [START_REF] Xiao | mcentrist: A multi-channel feature generation mechanism for scene categorization[END_REF], authors choose to decompose the neighborhood in two sets: diagonal and horizontal plus vertical comparisons. Each of these 4-valued components has only 16 possible values. They then build the two much smaller histograms separately and concatenate them, for a total of 32 values. This strategy allows for a much more compact descriptor with little loss of discriminability. Inner LBP possible comparisons are not limited to the differences with the center cell. A significant number of alternate neighborhood topologies have been explored, their efficiency depending on the application tested on. Centersymmetric [START_REF] Zheng | Effective pedestrian detection using center-symmetric local binary/trinary patterns[END_REF][START_REF] Heikkil | Description of interest regions with center-symmetric local binary patterns[END_REF] patterns highlight symmetry, centersurround patterns [START_REF] Trefny | Extended set of local binary patterns for rapid object detection[END_REF] deal with their neighborhood outskirts and direction. Finally, [START_REF] Heng | Shrink boost for selecting multi-lbp histogram features in object detection[END_REF] combined several of them.

The original LBP cell size is 1×1 pixels. We found few authors experimenting with different cell sizes [START_REF] Geismann | Speeding up hog and lbp features for pedestrian detection by multiresolution techniques[END_REF], and none actually stacking the information of several cell sizes within the block histograms. LBP-channel descriptors have also been proposed.

[22] explored it with a variety of color and contrast channels, as well as different topologies. Their use however provided a limited success compared to channel features [START_REF] Dollar | Integral channel features[END_REF] as the amount of channels and topologies they could search was limited by the high memory size of the descriptor. This shortcoming was emphasized by the high sparsity of the initial descriptor and a fixed block grid topology. mCEN-TRIST [START_REF] Xiao | mcentrist: A multi-channel feature generation mechanism for scene categorization[END_REF] is another multi-channel LBP-based descriptor that tried to capture the complementary information across channels.

LBP-channel descriptor

This work gets down to developing a light-weight, although more discriminative, channel-based LBP descriptor. Its extraction pipeline is presented in figure 2. As the LBP large memory size impacts the feature exploration capacities, several actions are undertaken to shrink the possible number of patterns. First, we use the Haar-like descriptors [START_REF] Corvee | Haar like and lbp based features for face, head and people detection in video sequences[END_REF] topology to LBPs. The feature is composed of 4 cells of size (w, h) instead of the 9 original LBP cells. The feature is then calculated as the concatenated quantization between all adjacent vertical and horizontal cells:

S(I(C

i ) -I(C j ))2 q S(x) = { 1 if |x| > T 0 otherwise (3)
With only 16 possible patterns, this more compact representation allows for a lower memory footprint. This type of topology also allows smaller regions (with a minimal size of 2 × 2 pixels), making it a better fit to characterize small pedestrian proposals. Uniform pattern [START_REF] Ojala | Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[END_REF] restrictions are systematically applied to further reduce the number of possible patterns to 12, therefore producing a descriptor of dimensionality similar to HoG [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF]. We also substantially improved the descriptor robustness by stacking the frequencies of cells C i with various dimensions within the block histograms. A block histogram bin value is then defined as:

h(i) = ∑ i=0...n ( ∑ r Ci ∈ b and HLBP (r Ci ) = i 1 ) (4)
As shown in the experiment section, this modification deals efficiently with the pattern deformations within each block, therefore reinforcing the overall descriptor robustness.

For each channel, the feature is computed on a lattice of blocks. An histogram of pattern frequencies is further calculated for each block. All the block histograms are finally concatenated to form the final descriptor. The best results are obtained by performing a L 1 square normalization on blocks and a L 2 -normalization on the final histogram.

Training protocol

Training is performed according to the FairTrain [START_REF] Trichet | Dataset optimisation for realtime pedestrian detection[END_REF] training set generation protocol. The methodology, illustrated in figure 3, decomposes in two distinct parts: The initial training set generation and the classifier training. The initial training set generation carefully selects data from a set of images while balancing negative and positive sample cardinalities. We then train a cascade of 1 to n classifiers. This cascade could include a cascade-of-rejectors [START_REF] Costea | Word channel based multiscale pedestrian detection without image resizing and using only one classifier[END_REF][START_REF] Zhu | Fast human detection using a cascade of histograms of oriented gradients[END_REF][START_REF] Berkin | Fast human detection with cascaded ensembles on the gpu[END_REF][START_REF] Souded | Optimized cascade of classifiers for people detection using covariance features[END_REF][START_REF] Viola | Rapid object detection using a boosted cascade of simple features[END_REF], a soft cascade [START_REF] Ojala | A comparative study of texture measures with classification based on feature distributions[END_REF], or both. In addition, each independent classifier is learnt through bootstrapping [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF][START_REF] Felzenszwalb | Cascade object detection with deformable part models[END_REF] to improve performance. One key aspect is to seek balanced positive and negative sets at all time. Hence, all along the cascade, the minority class is oversampled to create balanced positive and negative sets. See [START_REF] Trichet | Dataset optimisation for realtime pedestrian detection[END_REF][START_REF] Trichet | How to train your dragon: best practices in pedestrian classifier training[END_REF] for details.

Reversed feature selection

Filters are determined according to a pre-defined dense lattice of regularly spaced positions {(x i , y i )}, as well as sets of width {w j } and heights {h k }. The filter set is the combination of all possible positions and dimensions {[(x i , y i ), w j , h k ]} with respect to a 3/1 or higher aspect ratio.

To search this vast amount of possible filter positions and dimensions {[(x i , y i ), w j , h k ]} is a complex, often underoptimal task, and experience has shown that heuristics [START_REF] Zhang | Informed haarlike features improve pedestrian detection[END_REF][START_REF] Benenson | Ten years of pedestrian detection, what have we learned? ECCV[END_REF][START_REF] Kiani | Multi-channel correlation filters for human action recognition[END_REF][START_REF] Nam | Local decorrelation for improved pedestrian detection[END_REF] Filters are typically selected using the soft cascade [START_REF] Ojala | A comparative study of texture measures with classification based on feature distributions[END_REF]. This algorithm trains a cascade of n classifiers C i , each of them being trained by adding the best filter to the feature set. In this paper, we claim that, despite its efficiency, the soft cascade is intrinsically flawed [START_REF] Trichet | How to train your dragon: best practices in pedestrian classifier training[END_REF]. Indeed, classifiers with few descriptor values lack robustness and adding iteratively the best features may lead to an under-optimal final descriptor as the combination of features might be more important than their individual efficiency. So, in this work, we instead optimize for the best filter combination. To this end, we build the soft cascade in a reverse way. Let's define F and N i as, respectively, the full filter set and the retained filter set for classifier i. We first train the final classifier harnessing N n filters and iteratively train the previous classifiers by reducing the retained filter set |N i-1 | = |N i |/2 and updating the full set F = N i . When training each classifier C i , the best filter combination N i is determined through an expectation-maximisation (EM) algorithm. Weights, initialized equal, are associated to each filter. At each EM iteration it E-step, the used filter set N it i is determined as a weighted draw without replacement over the full filter set F . During the M-step, a classifier is trained on these features and validated on a third party set. The weights of the tested filters are then updated according to the classification score. For a given iteration, a filter's weight is then the average classifier performance over all the past runs for which it was selected:

W (x i ) =    ∑ Sx=[it|x∈N it i ] score(C it i )    /|S x | (5)
Unlike the soft-cascade that selects features one by one, this algorithm selects them "as a all" and, as shown in the experiment section, leads to more robust feature selection. Pseudo-code 1 details the algorithm mechanism.

Data implementation details: Compared to soft-cascade training, where each cascade stage adds one feature to the feature pool, our approach trains many full-feature size descriptor, which is time consuming and wouldn't be tracktable on large datasets. So, to speed-up the EM-process, the first iterations are trained with a limited number of trees and bootstrapping data addition and these two parameters are quadratically increased along the iterations until the desired number of trees and data addition is reached for the last iterations. More specifically, these parameters are multiplied by (j/#It) 2 , with j the current iteration and #It the total number of iterations. Moreover, the process is multithreaded for further boost. All these improvements accelerate the algorithm by 2 order of magnitude.

Experiments

This section, dedicated to our experimental validation, breaks down into several sub-parts. After presenting the datasets and the experimental setup details, we run individual experiments on the LBP-channel main parameters. The last part compares our work to the state-of-the-art.

Datasets

We experimented on the INRIA and Caltech-USA datasets. The INRIA [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] features high resolution pictures mostly gleaned from holidays photos. The training set consists of 2416 cropped positive instances from 614 images and 1218 images free of persons. The test set contains 1132 positive instances from 288 images and 453 person free images for testing purposes. This is among the most widely used dataset for person detector validation and comparative performance analysis. Despite its small size compared to more recent benchmarks, the INRIA dataset boasts high quality annotations and a large variety of angles, scenes, and backgrounds.

The Caltech-USA [START_REF] Dollr | Pedestrian detection: A benchmark[END_REF] dataset consists of 2.3 hours of video recorded at 30fps from a vehicle driving various Los Angeles streets. It totals 350000 images and 1900 unique individual pedestrians, 300 large groups, and 110 hard to distinguish pedestrians. Despite some annotation errors [START_REF] Zhang | How far are we from solving pedestrian detection[END_REF], its large size along with crowded environments, tiny pedestrians (as low as 20 pixels), and numerous occlusions probably make the Caltech-USA dataset the most widely used one. Annotations allow to experiment on 2 different sets. Contrarily to the "full set", the "reasonable set" restricts algorithms to pedestrians over 50 pixels in size and a maximum of 35% occlusion.

With respectively a large variability and tiny occluded detections, everyday pictures and automotive application, these two datasets offer complementary settings for our experiments.

Evaluation protocol

Candidates are selected using a multi-scale sliding window approach [START_REF] Zhu | Fast human detection using a cascade of histograms of oriented gradients[END_REF] with a stride of 4 pixels. Approximately 60% are loosely filtered according to "edgeness" and symmetry. No background subtraction or motion features are employed on the Caltech dataset.

The training set size is up-bounded at 12K samples. Increasing this threshold doesn't improve performance. No generative resampling is performed. The validation set includes 240K samples for all experiments. The model height is set to 120 pixels for the INRIA dataset, 80 pixels for Caltech.

We train a cascade of 6 Adaboost classifiers with a constant threshold of 0.56 and a tree depth of 2. We utilised the openCV implementation for the Adaboost classifiers. Adaboost is initialised with 256 weak classifiers. Each extra However, in practice, increasing this value leads us to lower performance and speed. When employed, reverse feature selection applies to the first 2 classifiers of the cascade, their number of trees being then learnt during training. We use the threshold variant of non-maximum suppression (t-NMS) [START_REF] Buil | Non-maximum suppression[END_REF] that groups the detections according to the bounding boxes overlap with the group d best candidate, keeps the d candidates with highest confidence for each group, and builds the final candidate position with the group mean border positions. We set the overlap threshold to 0.5 and d = 3 for all our experiments. Log-Average Miss Rate (LAMR) [START_REF] Wojek | Pedestrian detection: An evaluation of the state of the art[END_REF] is employed as metric for all runs.

LBP channel setup

For each detection proposal window, the LBP descriptor blocks are spaced out over 8×16 positions along the x and y axis, leading to 128 different possible positions. This lattice allows blocks to be placed approximately every 2 pixels for the smallest Caltech detections and such precision is not useful for bigger pedestrian. The possible filter dimensions w j and h k are {2W/7, 3W/7, 4W/7}, with W the detection window width. In addition to that, filters covering the head, the torso, the legs and the entire On the Caltech dataset, the training of the full descriptor and classifiers takes 14 hours; the training of the reduced descriptor takes 17-24 hours with 12 cores assigned to the process. reversed feature selection runs for 200 iterations. We tested the LBP-channel descriptor with the following 7 channels: graylevel intensities I, gradient magnitude M , gradient along x and y axis G x and G y , as well as the L, U and V color channels. We purposely added gradient channels to explicitly counterbalance this descriptor's known weak characterization of edges. The combination of various channel has been a assessed on the INRIA dataset. Also, The L, U and V channels are considered separately. Results are displayed on table 1. The gradients G x , G y , L, and U channels stand out as the most discriminative. It also confirms that the grayscale intensity bears much less information than the Luv color channel. The gradient magnitude M and the V color channel are apparently detrimental to the classification.

In our implementation, we experimented with the following cell dimensions: { ( 2. Of course the more patterns, the better the performance. But this experiment also reveals that the smallest cells are more discriminative, with the detection rate being almost the same when using {(1,1); (1,2); (2,1); (2,2)} cells or the full pattern set. This is sensible as they are the majority to be extracted within small detections, few of their bigger counterparts then fitting within the reduced-sized filters. Finally, it demonstrates the efficiency of stacking cells, Method INRIA Speed(CPU/GPU) HoG [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] 46% 0.5fps HoG-LBP [START_REF] Wang | An hog-lbp human detector with partial occlusion handling[END_REF] 39% Not provided MultiFeatures [START_REF] Wojek | A performance evaluation of single and multi-feature people detection[END_REF] 36% < 1fps FeatSynth [START_REF] Bar-Hillel | Partbased feature synthesis for human detection[END_REF] 31% < 1fps MultiFeatures+CSS [START_REF] Walk | New features and insights for pedestrian detection[END_REF] 25% No Channel Features [START_REF] Dollar | Integral channel features[END_REF] 21% 0.5fps FPDW [START_REF] Dollar | The fastest pedestrian detector in the west[END_REF] 21% 2-5fps DPM [START_REF] Felzenszwalb | Object detection with discriminatively trained partbased models[END_REF] 20% < 1fps RF local experts [START_REF] Marin | Random forests of local experts for pedestrian detection[END_REF] 15.4% 3fps PCA-CNN [START_REF] Ke | Pedestrian detection via pca filters based convolutional channel features[END_REF] 14.24% < 0.1fps CrossTalk cascades [START_REF] Dollr | Crosstalk cascades for frame-rate pedestrian detection[END_REF] 18.98% 30-60fps VeryFast [START_REF] Benenson | Pedestrian detection at 100 frames per second[END_REF] 18% 8/135fps WordChannels [START_REF] Costea | Word channel based multiscale pedestrian detection without image resizing and using only one classifier[END_REF] 17% 0.5/8fps SSD [START_REF] Liu | Ssd: Single shot multibox detector[END_REF] 15% 56fps LBP-Channels full 14.3% 0.5/7.5fps LBP-Channels selected 13.6% 0.7/10fps FRCNN [START_REF] Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF] 13% 7fps RPN+PF [START_REF] Zhang | Is faster r-cnn doing well for pedestrian detection[END_REF] 7% 6fps

Method CALETCH Speed(CPU/GPU) HoG [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] 69% 0.5fps DPM [START_REF] Felzenszwalb | Object detection with discriminatively trained partbased models[END_REF] 63.26% < 1fps FeatSynth [START_REF] Bar-Hillel | Partbased feature synthesis for human detection[END_REF] 60.16% < 1fps MultiFeatures+CSS [START_REF] Walk | New features and insights for pedestrian detection[END_REF] 60.89% No FPDW [START_REF] Dollar | The fastest pedestrian detector in the west[END_REF] 57.4% 2-5fps Channel Features [START_REF] Dollar | Integral channel features[END_REF] 56.34% 0.5fps Roerei [START_REF] Benenson | Seeking the strongest rigid detector[END_REF] 48.35% 1 fps MOCO [START_REF] Chen | Detection evolution with multi-order contextual co-occurrence[END_REF] 45.5% < 1fps JointDeep [START_REF] Ouyang | Joint deep learning for pedestrian detection[END_REF] 39.32% < 1fps SquaresChnFtrs [START_REF] Benenson | Ten years of pedestrian detection, what have we learned? ECCV[END_REF] 34.8% < 1fps InformedHaar [START_REF] Zhang | Informed haarlike features improve pedestrian detection[END_REF] 34.6% < 0.63fps Spatial pooling [START_REF] Paisitkriangkrai | Strengthening the effectiveness of pedestrian detection with spatially pooled features[END_REF] 29.2% < 1fps Checkboards [START_REF] Zhang | Filtered channel features for pedestrian detection[END_REF] 24.4% < 1fps FRCNN [START_REF] Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF] 56% 7fps CrossTalk cascades [START_REF] Dollr | Crosstalk cascades for frame-rate pedestrian detection[END_REF] 53.88% 30-60fps WordChannels [START_REF] Costea | Word channel based multiscale pedestrian detection without image resizing and using only one classifier[END_REF] 42.3% 0.

Results and analysis

Reverse feature selection is compared to soft-cascade in table 3. The main difference with typical feature selection in our case is that 12-valued feature blocks are selected. We select the filter across channels rather than for one channel, which gives us a better performance in practice. The full set of unselected features is provided as baseline and scores 41.0%. During the training phase, the best reverse selection results (reported in this table) are often obtained for the highest number of trees and amount of data added during bootstrapping, which confirms the common sense advocating for more trees and data for better performance. If the soft-cascade is faster (running up to 2.2 fps) than reverse feature selection, it doesn't provide as discriminative features as the reverse feature selection, with only 1.7% improvement over the baseline. Comparatively, reverse feature selection provides at best a 5.1% performance boost while running at 0.7 fps. The "Undirected selection", referring to the same strategy crippled of the EM-algorithm, shows the overall positive impact of directing the selection process toward the best feature blocks, with a 1.8% performance in-crease. Overall, this strategy seems to learn a better information structure since the performance steadily increases with the addition of features, contrarily to the soft-cascade that quickly reaches a plateau (around 50 features). However, this is a rather small improvement compared to other channel feature selection methods [START_REF] Benenson | Ten years of pedestrian detection, what have we learned? ECCV[END_REF][START_REF] Zhang | Informed haarlike features improve pedestrian detection[END_REF][START_REF] Paisitkriangkrai | Strengthening the effectiveness of pedestrian detection with spatially pooled features[END_REF][START_REF] Zhang | Filtered channel features for pedestrian detection[END_REF]. We believe that substantial further improvement can be obtained in that direction. The computation time gain remains low even for few features selected because most of the computation lies in the integral channel calculation.

Comparison with the state-of-the-art is presented in table 4. For fair comparison, methods using optical flow features [START_REF] Benenson | Ten years of pedestrian detection, what have we learned? ECCV[END_REF][START_REF] Walk | New features and insights for pedestrian detection[END_REF][START_REF] Park | Exploring weak stabilization for motion feature extraction[END_REF][START_REF] Zhang | Filtered channel features for pedestrian detection[END_REF], are reported without optical flow or left out of this evaluation. G x + G y + L channels are employed for this experiment, giving us a better performance/speed ratio.

While running at a similar speed, LBP-channel (full) features greatly outperforms the common channel features [START_REF] Dollar | Integral channel features[END_REF], scoring 7% better on the INRIA dataset and 16.2% better on the Caltech dataset. Besides the predominance of our descriptor, it shows that one-valued filter responses should not be the de facto choice: A balance between the amount of filters and their response size must be struck. Our reverse feature selection scheme (using 150 feature blocks across channels) further improves the results down to 13.6% on INRIA and 35.9% on CalTech while making it slightly faster.

Overall, LBP-channels ranks on par or better compared to state-of-the-art pedestrian descriptors in terms of performance/speed ratio, except for [START_REF] Zhang | Is faster r-cnn doing well for pedestrian detection[END_REF]. Deep learning technique [START_REF] Liu | Ssd: Single shot multibox detector[END_REF][START_REF] Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF] often perform better but are slower. The detector speed is, of course, largely influenced by the descriptor parameters, which are the number of filters and channels.

Conclusion

In this work, we have presented a new channel descriptor based on LBPs. Stacking various cell dimensions and wise channel choices demonstrated the improved descriptor robustness and discriminability. Also, reduced block histogram size as well as a reverse feature selection alternative to the soft cascade allowed us to efficiently yield a robust set of features. Experiments show that LBP-Channels outperform their main competitors while still producing a fast and compact descriptor. Through the introduction of this new descriptor, this paper suggests that there is a balance to strike between the number of feature values used as filter response and the size of the filter space to search. Thorough study should be undertaken to determine what the appropriate trade-off is. These first feature selection results compared to the state-of-the-art imply that further research in this particular direction may lead to improved performance.

Figure 1 :

 1 Figure 1: Haar-LBP pattern divided in 4 cells. The arrows depict the 4 operation used to computer the Haar-LBP descriptor.

Figure 2 :

 2 Figure 2: LBP Channel features pipeline.

Figure 3 :

 3 Figure 3: Training pipeline. The initial training set generation selects data while balancing negative and positive sample cardinalities.A cascade of classifiers is then trained on it, each independent classifier being learnt through bootstrapping. balanced positive and negative sets is sought all along the cascade. Each circle surface is proportional to the set's cardinality that it represents.

  can help narrow down this search to the most discriminant ones. One key insight in this paper is drown from the following observation: The retained filter set always covers over 95% of the proposal window surface. Hence, To drastically restrict the possible number of filter combination to explore, we enforce a filter repartition across the positions {(x i , y i )}. More precisely, with |N | and |P | respectively being the desired number of filters and the number of positions, the amount of filters extracted per position is within the range [1, (|N |/|P |) + 2].

Table 1 :

 1 Full LBP-descriptor log-average miss rate performance (the lower the better) on the INRIA dataset when using various channel combinations. I -Grayscale intensity. {G x , G y } -gradients respectively along x and y axis. Mgradient magnitude. {L, U, V} -Luv color channels.

	Channel	LAMR
	Gx + Gy	18.8%
	Gy + L	16.7%
	Gx + Gy + I	15.8%
	Gx + Gy + I + M	15.9%
	Gx + Gy + L + U + V	15.6%
	Gx + Gy + L	14.3%
	Gx + Gy + L + U	14.1%
	Cells	LAMR
	{(3,1); (1,3); (4,4)}	16.3%
	{(1,1)}	16.0%
	{(1,2); (2,1); (2,2)}	15.4%
	{(1,2); (2,1); (2,2); (3,1); (1,3)} 15.2%
	{(1,1); (1,2); (2,1); (2,2)}	14.7%
	All	14.1%

Table 2 :

 2 Full LBP-descriptor log-average miss rate performance (the lower the better) on the INRIA dataset when using various cell set. G

x + G y + L + U channels are employed for this experiment. run adds 64 weak classifiers. This is a low number of weak classifiers compared to typical settings (i.e.

[1024, 2048]

).

Table 3 :

 3 Comparison of soft-cascade with reverse feature selection on the LBP-descriptor. Results are in terms of logaverage miss rate performance (the lower the better) on the INRIA dataset. G x + G y + L channels are employed for this experiment. See text for the methods description. body are extracted based on edge templates [52], leading to a total of 373 possible filters per channel. In these experiments, the full descriptor will refer to the full set of filter positions with the smallest dimensions 2W/7 only. With this setting, 133 filters, totalling 1596 feature values, are extracted per channel. L 1 square block as well as L 2 full histogram normalisation are performed in every case.

Table 4 :

 4 Comparison with the state-of-the-art. Near real-time methods are separated from others. Ours is in bold. Deep learning techniques are in red. Computation times are calculated according to 640×480 resolution frames. G x + G y + L channels are employed for this experiment. The used metric is the log-average miss rate (the lower the better).

	5/8fps


Multi-camera Tracklet association and fusion using ensemble of visual and geometric cues

Data association and fusion is pivot for object tracking in multi-camera network. We present a novel framework for solving online multi-object tracking in partially overlapping multi-camera network by modelling tracklet association as combinatorial optimization problem hypothesized on ensemble of cues such as appearance, motion and geometry information. Our method learns discriminant weight as a measure of consistency and discriminancy of feature patterns to make ensemble feature selection and combination between local and global tracking information. Our approach contributes uniquely in the way tracklet selection, association and fusion is done. Once multi-view correspondences are established using planar homography, Dynamic Time Warping algorithm is used to make tracklet selection for which similarity has to be calculated i.e overlapping tracklets and subtracklets. Then trajectory similarities are computed for these selective tracklets and subtracklets using ensemble of appearance and motion cues weighted by online learnt discriminative function. Later on, we tackle the association problem by building a k-partite graph and association rules to match all the pair-wise trackets. Finally, from outcome of hungarian algorithm, the associated trajectories are later fused. Fusion is done based on calculated individual tracklet reliability criteria. Experimental results demonstrate our system achieve performance that significantly improve the state of the art on PETS 2009.

Introduction

Inspite of number of solutions, object tracking across multiple camera network is still considered most challenging and unsolved computer vision problem, mainly due to placement of cameras, multi-camera calibration, fuzzy data association, fusion and person re-identification across partially or non-overlapping network of cameras. Multicamera systems help in obtaining more visual information about a same scene, thereby helps in overcoming deficits of single camera object tracking such as inter and intraocclusions, inferior visibility in crowed scenes,object reentry, abrupt movement of object, similar appearance and complex interactions among objects in dense environment. Such systems are also useful in rectifying incorrect and fragmented trajectories from individual cameras by associating and fusing collective information thereby substantially improve higher vision tasks such as activity recognition, animation, surveillance etc.

In this paper, we are trying to improve object tracking efficiency through real time multi-camera data association, fusion based on geometry and visual cues. We present a multi-camera tracking approach that associates and performs late fusion of trajectories in a centralized manner from distributed cameras.

Data from each camera in network is gathered to a central node by projecting the trajectories of people to the camera with the most inclusive view through a planar homography technique. Association and fusion is performed based on weighted combination of local and global features such as geometry,appearance and motion(Sec. 5). Association is modeled as a complete K-partite graph, K corresponds to number of cameras in network. Since we use complete K-partite graph, we have the optimal solution in real time. Whereas methods that model association as complex multivariate optimization such as [START_REF] Hofmann | Hypergraphs for joint multi-view reconstruction and multi-object tracking[END_REF], upon scaling, face the problem of being stuck at local minima, may provide sub-optimal solutions and aren't real time realizable. Fusion is performed using reliability based adaptive weighting method. Where the weights are derived from reliability attribute of each tracker from [START_REF] Chau | A multi-feature tracking algorithm enabling adaptation to context variations[END_REF]. This enables correct and consistent trajectory after fusion even if the individual trajectories have inherent noises,occlusion and false positives.

Our method has following advantages: 1.Has centralized measures that counter for noisy measurements belonging to same objects overtime by distributed local trackers. [START_REF] Berclaz | Multiple object tracking using flow linear programming[END_REF] We integrated measures during fusion to account both short and long occlusions. 3.Our method doesnt involve costly optimization,metrics and data-gathering(fusion) strategy, thus significantly influencing on the scalability of net-work and effective real-time applicability of multi-camera tracker. 4.It is modeled around real world context, giving importance to spatio-temporal pairwise relationship amongst tracklets in a bunch of frames(in our case, its 10frames/camera). 5 Our cost function allows us to model multilevel relationship amongst tracklets efficiently.

The reminder of the paper is divided into the following sections:In section 2,we review some significant previous works and how our method differs from them. In section 3,we discuss about multi-camera synchronization and multi-view geometry used in our approach. Next in section 4 we discuss how we formulate trajectory association problem. Followed by section 5 which describes calculation of trajectory similarity metrics. Trajectory fusion is introduced in section 6 , experimental results are presented in section 7 and finally section 8 concludes the paper.

Related work

In recent years, there has been variety of approaches proposed for data association in multi-object tracking. Comparatively multi-camera data association and tracking have seen less number of methods published in recent years. Mostly, multi-object temporal data association methods are extended to multi-camera data association setting. On general basis approaches can be outlined based on 1. Fusion time -either early fusion [START_REF] Evans | Suppression of detection ghosts in homography based pedestrian detection[END_REF] or late fusion [START_REF] Taj | Distributed and decentralized multicamera tracking[END_REF],

2. The search space -greedy i.e. temporally local or global optimization with longer temporal stride [START_REF] Zhang | Global data association for multi-object tracking using network flows[END_REF], [START_REF] Pirsiavash | Globallyoptimal greedy algorithms for tracking a variable number of objects[END_REF].

Approach [START_REF] Hofmann | Hypergraphs for joint multi-view reconstruction and multi-object tracking[END_REF] extends the work of [START_REF] Berclaz | Multiple object tracking using flow linear programming[END_REF] to jointly model multi-camera reconstruction and global temporal data association using MAP. They use single global min cost flow graph for tracking across multiple cameras with a good number of heuristically determined parameters, the complexity increases with more capacities and more risk of not finding a optimal solution if the graph has negative cost cycle of infinite capacity. J. Berclaz et al. [START_REF] Berclaz | Multiple object tracking using flow linear programming[END_REF] having detection based on probablity occupancy map, also uses flow graph based method for solving both mono and multicamera setup within an restricted and predetermined area of interest.

Murray Evans et al. [START_REF] Evans | Multicamera object detection and tracking with object size estimation[END_REF] uses early fusion strategy for detection inspired from [?] and extends it for multi-camera tracking and estimating object size in multi-camera environment. Their approach leverages multiview information into early stage (detection) of pipeline to remove ghosts. Since the synergy map they use for ghost suppression also suppresses existing objects in previous frame thus they cannot perform tracking by associating detections moment to moment, multivariate optimization is performed on object size together with probable location of object in next frame.

As optimizing an objective function that combines both object size estimate and tracking information is complex, the solution may be sub-optimal and wouldn't be real time realizable. By nature of their ghost suppression method which involves intricate assumptions over line from camera to object, it makes it difficult to track objects in cluttered or crowded environment. Anjum, N. and Cavallaro, A. in [START_REF] Anjum | Trajectory association and fusion across partially overlapping cameras[END_REF] have presented an unsupervised inter-camera trajectory correspondence algorithm, for the association step, they propose a hybrid approach: project the trajectories from each camera view to the ground-plane in order to find associations among trajectories, and then, make image-plane re-projections of the matched trajectories. These methods rely entirely on goodness of homography, smallest margin of error in calibration gets added up during initial projections and re-projections. Thus are susceptible to introduce errors that end up being association errors. Sheikh, Y.A. and Shah, M. in [START_REF] Sheikh | Trajectory association across multiple airborne cameras[END_REF] have proposed a target association algorithm that addressed the problem of associating trajectories across multiple moving airborne cameras with a constraint that at least one object is seen simultaneously between every pair of cameras for at least five frames. Since this method uses object centroid as feature points to recovert the homography and later use RANSAC to find out best subset of such points to find correspondence, it works well when in sparse environment, but in dense environments it may fail. Their approach assumes all the object to be tracked are on the common ground well alligned with all the cameras present in network.

In [START_REF] Chang | Tracking multiple people with a multi-camera system[END_REF] Chang,T.H. and Gong,S. present a multi-camera system based on Bayesian Networks used to match objects in multiple camera views. For mono-camera tracking they use bayesian networks to enable a full set of possible matching assignments between consecutive frames (based on motion continuity and apparent colour). When the status of segmented blobs change suddenly or the matching becomes ambiguous, the system performs multi-camera cooperative tracking to match subjects across cameras. The system employs a Bayesian modality fusion to match over several subjects such as epipolar geometry, homography, landmark modality, apparent height, and apparent colour. Drawback here is the features are weighted higher based on how recent the evidence is.

Proposed Methodology

Our approach works based on the assumption that cameras in the network are synchronized, calibrated and homography between every pair of cameras are established. If the cameras aren't synchronized, we use linear regression to find correspondence between frames.

Trajectory Association

The association problem is related with the need of establishing correspondences between pair wise similar trajectories that come from different cameras.

For simplicity purpose, we experiment using two cameras, the association or correspondence may be modelled as a bi-partite (K-partite, K=2) matching problem in which each set has trajectories that belong to one camera. For each camera C l and C r , a set of trajectories S l and S r is defined. A bi-partite graph is a graph in which the vertex set V can be divided into two disjoint subsets S lef t and S right such that every edge e ∈ E has one end point in S lef t and the other end point in S right . Each Object being tracked is denoted as T O i is the resulting observation (i.e. a track point) of the multi-target tracking algorithm presented by [START_REF] Chau | A multi-feature tracking algorithm enabling adaptation to context variations[END_REF]. The physical objects have been synchronized in terms of frame number F and they have 2D space coordinates (x, y) Thus,

T O t = (F, (x, y)) t .
Let T O i represents the i th tracked object that belongs to the trajectory T r C k j observed in the camera C k where k = l, r. Thus, each trajectory is composed by a time sequence of 3d points of physical objects:

T r C k i = {T O i 0 , T O i 1 , T O i t , ...., T O n i i }. ( 1
)
Where n j is the length of above trajectory Consequently, each camera C k has a set of N and M trajectories belonging to sets S l and S r respectively.

S l = {T r C l 0 , T r C l 1 , T r C l 2 , ....T r C l N } (2)
S r = {T r C r 0 , T r C r 1 , T r C r 2 , ....T r C r M } (3)
Once the bi-partite graph is built we need to compute overlapping trajectories across cameras and the camera pair-wise trajectory similarities. To perform this task we use spatio-temporal and appearance based trajectory features, assuming that two trajectories viewed from different cameras have to be similar both in time and space.

We abstract the trajectory association problem across multiple cameras as follows: Each trajectory T r C k j is a node of the bi-partite graph that belongs to the set S k linked with the camera C k . A hypothesized association between two trajectories is represented by an edge in the bi-partite graph, as is shown in Figure 1. The goal is to find the best match in the graph.

Time Overlapping Trajectories

For each hypothetical association we first filter and remove the associations of trajectories that do not overlap in time. In the case of time overlapping trajectories we take the intersecting time interval between them, that is, the lower, and the highest time value between both trajectories to get a new time interval in which both trajectories are contained. In the example of the figure 3 we have two trajectories T C l ri ∈ S l with 0 < j 1 < N 1 and T Cr rj ∈ S r with 0 < j 2 < N 2 , the resulting overlapping time interval is ∆t = [T r C l (t 0 ), T r C r (t f )]. In order to apply dynamic time warping, we have to have trajectories of the same size to compared frame by frame. The gaps or missing points (due to miss detections or occlusions) are completed with local linear interpolation for the mentioned time interval ∆t.

Linear Interpolation and smoothing

Object detection is not perfect due to occlusions, visibility, density of crowd, placement of camera etc, thus, a linear interpolation is applied in order to reach a more complete trajectory. We assume that a person follows uniform linear motion between the previous and next frame. Based on that a linear interpolation is performed in order to correct miss detections of time length equal to predetermined number of frame. To perform this correction, For example, for one frame miss detections, we estimate the position of the person in the current frame follows:

T r C k i (t) = T r C k i (t -1) -T r C k i (t + 1) 2 (4) Where T r C k i (t) is position of tracked object at time t, T r C k i (t -1) is position of tracked object at time t -1, T r C k i (t + 1
) is position of tracked object at time t + 1and C k is the camera number.

The 2D space of the trajectories that belongs to the left camera are projected to 2D space of right camera in order to compare it and find similar trajectories. During this task some noise can arise. Thus, in order to deal with this noise we smooth the trajectory for better results. At this time, we are almost ready to compute the trajectory similarity. However, the common tracklets between both trajectories needs to be found. Figure 3 shows a graphic interpretation of two overlapping time trajectories from the bipartite graph. The two trajectories have common tracklets in the subintervals

Find Tracklets in Common

[t 0 , t 1 ], [t 2 , t 3 ] ⊂ [t A , t B ] belonging to trajec- tories T r C l i , T r C r j , two tracklets in the subintervals [t 3 , t 4 ], [t 5 , t 6 ] ⊂ [t A , t B ] belonging to T r C r j . And finally, one tracklet [t 1 , t 2 ] ⊂ [t A , t B ] belonging to T r C l i
Later on, a trajectory similarity algorithm is applied for every pair of tracklets in common among both trajectories. It is important to note that now the tracklets contains no empty positions, have the same length, and has been synchronized.

Trajectory Similarity Calculation

The comparison of two temporal sequences (e.g. trajectory) and their similarity measurement is a multidimensional sequence data problem that has been studied in many research fields such as data mining, motion tracking, and time series analysis [START_REF] Chen | A dynamic time warping based algorithm for trajectory matching in lbs[END_REF]. There are several trajectory similarity measurements in the state of the art. Two similarity models draw our attention: Longest Common Subsequence described by Bergroth et al. [START_REF] Bergroth | A survey of longest common subsequence algorithms[END_REF], and Dynamic Time Warping introduced by Kassidas et al. [START_REF] Kassidas | Synchronization of batch trajectories using dynamic time warping[END_REF] as they are the most successful ones and widely used. Amongst these we choose the later as it offers enhanced robustness, particularly being sensible to noisy data. As our goal is to associate trajectories we need a global measurement for trajectories comparison. Which is being done using Dynamic Time Warping (henceforth DTW).

DTW has some constraints that need to be addressed: • Trajectories of equal length.

• Time Synchronized trajectories.

As a first step in DTW, is to place the trajectories in a grid in order to compares them, and initialize every element as ∞ (represent ∞ distance). Each element of the grid represent the Euclidean distance which is the alignment between two trajectories points T r

C l i (t i ), T r C r j (t j )∀t i ∈ [0...n i ], ∀t j ∈ [0...n j ]
Many paths connecting the beginning and the ending point of the grid can be constructed. The goal of Dynamic Time Warping is to find the optimal path that minimize the global accumulative distance between both trajectories:

D(T r C l i , T r C r j ) = min[ N ni,nj =1 d(T r C l i (n i ), T r C r j (n j ))]
(5) As shown in Figure 4, applying this method each grid point (n i , n j ) now represents the minimum accumulated distance from the beginning to the current point.

We can appreciate in Figure 5 that the tracklets are very similar from frame 65 to 82, but after seem like they start to be unequal. The further close the optimal path wanders around the diagonal, the more the two sequences match together. It is important to mention that for long trajectories this algorithm is very expensive in terms of memory. For this reason we adapted the algorithm to reuse a fixed matrix that is emptied and re-filled every time we reach the maximum matrix size.

Ensemble feature combination using online learnt discriminative weights

It is a known fact that feature combinations capture more underlying semantics than single feature patterns. But using less influential patterns may not improve the performance of a tracker mainly due to limited discriminability of the feature. Trajectory similarity is calculated as a two stage approach (Local and Global). An ensemble of local and global features is used instead of determining similarity score. The algorithm learns weights online based on the consistency and maximum discriminability of the feature patterns.

Local tracklet similarity. At local stage, importance is given to local frame to frame geometric information. From DTW results, we derive Euclidean Distance metric for each set of trajectories.

EDM = D(T r C l i , T r C r j ) (6)
Global tracklet similarity. At global stage, information pertaining to overall appearance of the object is taken into account for determining the similarity between tracklets. Feature patterns used for determining a overall appearance score is discussed in section 5.2. A global matching score quantified from such features represent global tracklet similarity.

Each element of trajectory association cost matrix represents weighted sum of Euclidean distance and Global Matching Score(GMS) between two trajectories T r C l i , T r C r j . An entry in association cost matrix can be defined as follows:

D (T r C l i , T r C r j ) = (1 -ω m (T r i )).EDM (T r C l i , T r C r j ) +ω m (T r i ).GM S(T r C l i , T r C r j ) ( 7
)
where GMS is defined in section 4.2 and EDM is given in equation 6.

Where ω m is appearance descriptor weight learned to specify if more importance should be given to appearance cues or geometric cue. i.e online discriminant weighting to decide a tradeoff between local information extracted from frames or global appearance information from tracklets. The learnt weight helps in better feature selection and combination to enhance inter-tracklet discrimination and also cope-up with intra-tracklet variations. In this approach both local geometric and global appearance feature patterns compliment each other and is impactfull in situations where the dataset involves significant appearance changes across object pose, illumination, viewing angle and different camera parameters.

As shown in equation 11, ω m is calculated as minimum between consistency and discriminative power of appearance based global tracklet features. To calculate consistency and discriminative power of tracklet features across cameras, we need to color calibrate the cameras for accounting color distortion between them. Therefore as a preprocessing measure before validating discriminability, we perform histogram specification and histogram matching i.e considering camera 1 as reference camera, we project and transform the histogram of camera 2 onto histogram of camera 1. Level of color distortion after matching is validated by comparing the tranformed histogram and reference histogram using correlation.

Discriminative power of the GMS features of trackets are calculated as a adopted fisher score. Fisher Score is a quantitative measure popularly used in statistics usually for numerically solving maximum likelihood problems. Fisher score gained popularity in computer vision for its ability to compare one feature subset with another in order to find most discriminating set of features [START_REF] Gu | Generalized fisher score for feature selection[END_REF]. [START_REF] Collins | Online selection of discriminative tracking features[END_REF] used fisher score to online select most discriminative set of tracking features. Since ours is a multi-camera setup, we need to adopt this fisher score to avoid certain undesirable scenarios from affecting the final discriminant score. Constraints we lay on fisher score are • In a multicamera tracking problem, the discriminating power of tracklet features should be measured across cameras and not intra-camera. Thus in equation 8, instead of calculating the mean over all trackets over both cameras, we calculate mean only on the camera with candidate matching tracklets.

• Using weighted feature mean and variance based on online discriminative descriptor weights obtained while calculating GMS. This enables most discriminating features to influence fisher score.

F S(T r i , T r j ) = F f =1 w f (µ f i -µ c j f ) 2 F f =1 w f (ρ 2 f i ) (8)
where f corresponds to any feature considered in GMS, µ f i and ρ f i are the mean and the variance of the f-th feature of i-th tracklet respectively, F is the number of features considered, w f is descriptor similarity weight of f-th feature used in GMS and µ c j f is the mean of f-th feature of over all tracklets belonging to complimentary pair of camera C j .

Similarly, Overall consistency score on entire tracklet is calculated as square root of sum of weighted consistency score of individual features over a tracklet.

SD (T r i ) = n 1 .ρ f 1 + n 2 .ρ f 2 + ...n F .ρ f F n 1 + n 2 + ... + n F (9)
Where n F is total number of F th feature instances, ρ f F is individual consistency score of F th feature.

An individual consistency score is obtained for each feature in GMS metric over the entire tracklet (T r i ). For a feature f, consistency score can be calculated as follows.

ρ f i = ni t=1 (f (T O i t ) -f (T O i )) 2 n i (10)
Where

f (T O i t ) is i th feature extracted from tracked ob- ject T O t , f (T O i ) is i th feature mean and n i is total number of i t h feature instances. Appearance descriptor discriminant weight ω m is ω m (T r i ) = min(SD (T r i ), F S(T r i , T r j )) (11)

Global Matching Score (GMS)

Appearance based cues have played a vital role in tracklet association rule mining. Given a set of appearance cues, how do we select a high quality ones for effective discrimination from other candidates for tracklet association? This is answered to an extent in mono-camera tracklet reliability descriptor work by [START_REF] Nguyen | Robust global tracker based on an online estimation of tracklet descriptor reliability[END_REF]. We select set of cues inspired from their work and extend it to suit our approach. We use k=6 cues for our work. Namely, 2D shape ratio (k=1) and 2D area (k=2) Shape ratio and area of an object are obtained from respective bounding boxes, within a temporal window, they are immune to lighting and contrast changes. Thus they are one of good cues to use.

Color histogram (k=3) and Dominant Color (k=4)

It is basically a normalized RGB color histogram of pixels inside bounding box of moving object. Dominant color descriptor is used to take into consideration only important color of object.

Motion descriptor (k=5)

Depending on the context, constant velocity model or Brownian model is used to describe motion represented by Gaussian distribution. It is useful when objects have similar appearance.

Color covariance descriptor (k=6)

Color covariance descriptor is a covariance matrix that characterizes the appearance of regions in image and is invariant to size and identical shifting of color values.

There by resisting to illumination changes

To ensure reliable tracklet association, [START_REF] Nguyen | Robust global tracker based on an online estimation of tracklet descriptor reliability[END_REF] intelligently weights the discriminative appearance and motion model descriptors and generates a global matching score (GM S) . The global matching score of tracklet T r i with each tracklet in its matching candidate list ( represented by T r j ) is

GM S = 6 k=1 w ij k .DS k (T r C l i , T r c r j ) 6 k=1 w ij k (12)
Where w ij k are corresponding weights of each feature descriptors DS k (T r C l i , T r c r j ) calculated online by modeling them directly propotional to descriptor similarity of a tracklet with its matching candidate and inversely propotional to descriptor similarity of other overlapping tracklets.

If T r i , T r j are matching candidates, T r i , T r p are other overlapping tracklets, Their discriminative descriptor weight is calculated as

w i,j k = α [DS k (T ri,T rj )-X(DS k (T ri,T rp))-1] (13)
Where α = 10 determined experimentally. The discriminative weight for motion cue alone is calculated as

w i,j 6 = 0.5 -0.5max(w i,j k ) k = 1...5 (14)
Now the bi-partite graph is complete and the weight W ij of each edge e ∈ E in G = (V ; E) is D(T r C l i , T r C r i ) given by equation 7.

Hungarian Algorithm

The task at hand is finding the maximum matching of G . Formally, maximum matching is defined as a matching with the largest possible number of edges; it is globally optimal. In other words, the goal is to find an optimal assignment: the one that minimizes total cost of a matrix. To find the maximum matching in G. we apply the Hungarian Algorithm defined by Kuhn [START_REF] Kuhn | The hungarian method for the assignment problem[END_REF] given the cost matrix built with the W ij values. The Hungarian method is a combinatorial optimization algorithm that solves the assignment problem in polynomial time O(n 3 ) , where n is number of nodes or vertexes V of the bi-partite graph G. After apply the Hungarian Algorithm to the matrix we get the maximum matching as shown in figure 6.

Trajectory Fusion

Once association is done, the next step is to fuse a pair of corresponding trajectories. In order to perform this task, a merged trajectory with the information coming from both views is built. To fuse two trajectories coming from two different cameras at a time t, e.g. T r i ∈ S lef t with 0 < i < N and T r j ∈ S right with 0 < j < M into a global one T r Gi,Gj we apply an adaptive weighting method as follows: As we defined in Eq.( 1), each trajectory is composed by a set of tracked Objects. [START_REF] Chau | A multi-feature tracking algorithm enabling adaptation to context variations[END_REF] defined a method to quantify the reliability of the trajectory of each interest point by considering the coherence of the Frame-to-Frame (F2F) distance, the direction, and the HOG similarity of the points belonging to a same trajectory. Thus, as each physical object has a reliability attribute R with values [0, 1] the weighed function is defined in terms of its R value as follows:

T r Gi,Gj (t) =            w 1 T r Cl i (t) + w 2 T
w 1 = R P On R P On +R P Om w 2 = R P Om R P On +R P Om (16)
The key idea behind this weighted approach is that resulting trajectory(merged) is in between the other trajectories and will be close to the trajectory having higher reliability values.

The fused trajectory is not smooth. In order to get a better and smoothed one, we apply a simple moving average technique (also called moving mean). The first element of the moving average is obtained by taking the average of the last N elements of the trajectory. Then the subset is modified by shifting it forward; excluding the first number of the trajectory and including the next trajectory point. This process is repeated over the entire trajectory. As result of this process we obtain a smoothed trajectory without noise.

Evaluation

Our approach is evaluated on the publicly available PETS2009 dataset [START_REF] Ferryman | Pets2010: Dataset and challenge[END_REF]. PETS2009 is a challenging dataset due to its low FPS and inter-object occlusions . We choose View1,View3 in S2.L1 scenario to evaluate. There is one static occlusions in View1 namely a pole with display board , View3 has 2 static occlusions namely a pole with display board and a tree occupying significant area in right side of video. There is substantial color tone variation between the views, making it hard for color based cues. For this reason most of the methods avoid this combination of view. For multi-camera evaluation, we use ground truth based on a common plane ie world referential of View1.

Unfortunately, the lack of common metrics for measuring the performance of multi-camera multi-object detection makes result comparison even more difficult. But for evaluating our work, we use the following metrics: 1.CLEAR [START_REF] Bernardin | Evaluating multiple object tracking performance: the clear mot metrics[END_REF] metrics namely Multiple Object Tracking Accuracy(MOTA) and Multiple Object Tracking Precision(MOTP). 2. Identity Switches(IDS), Track Fragments(FM), Mostly Tracked(MT), Partly Tracked(PT) and Mostly Lost(ML) from [START_REF] Kuo | How does person identity recognition help multi-person tracking?[END_REF] 6.1. Experimental settings //say what are the parameters to tweak ,tune and how they are obtained, their impact/how they affect on our results 1 summarizes past few results on multi-camera PETS2009 dataset. Unlike other methods which use heavy computation for best results as a trade off over real-time performance, our objective was to make the algorithm more real time and online with buffer frames at the same time making minimal sacrifice on the accuracy. This is achieved as our method uses network flow simplex method which is both effective and simple. We use buffer frame size = 20 frames in a temporal sliding window pattern to be able to perform association and fusion online

Conclusion

We introduced a multi object tracking association and fusion across multi-camera network. In order to accomplish this, we built a multi-camera framework making use of mono-camera tracking. The trajectory similarity is computed by using Dynamic Time Warping approach and online learning of discriminative weights belonging to appearance and geometric cues respectively. Afterwards, the association relies on maximum bipartite graph matching performed by Hungarian algorithm Finally, the fusion between associated trajectories is performed by an adaptive weighted method based on reliability score of individual tracklets. We evaluated the multi-camera approach and compare its results against the state of art on public dataset. Our approach outperforms considerably some existing multicamera tracking and comparable to the state of art on benchmark dataset, as has been shown in Final results and has a good occlusion management.

Figure 1 .

 1 Figure 1. Observable tracklets as Bi-partite graph. Each set contains trajectories (nodes) coming from each camera. The edges represents hypothetic correspondences between trajectories.

Figure 2 .

 2 Figure 2. The time interval for each trajectory is shown. In this step we find the most intersecting time interval for both trajectories

Figure 3 .

 3 Figure 3. Shows 2 sample tracklets in common between two cameras in the time interval [tA, tB] .

Figure 4 .

 4 Figure 4. Optimal wrapping path is shown as green.

Figure 5 .

 5 Figure 5. DTW results for tracklet 1 of two trajectories comparison. In X and Y the frames are shown. The optimal path is represented in green, and the DTW result is shown in red.

Figure 6 .

 6 Figure 6. Associations of each trajectories after hungarian algorithm.

Table 1 .

 1 Comparison of our method with previous multicamera state of art on PETS2009 dataset

	Scenario	Method	Camera ID MOTA(%) MOTP(%) MT(%) PT(%) ML(%) FM IDS
		Berclaz et al.9	1,3,5,6,8	82	56	-	-	-	-	-
		Leal-Taixe et al.25b	1,5	76	60	-	-	-	-	-
	PETS 2009 S2.L1	Leal-Taixe et al. 25b Murray Evans et al. 10 Martin Hofmann et al. 7	1,5,6 2 Cameras 1,5	71.4 63 99.4	53.4 55 82.9	--100	--0	--0	--1	--1
		Martin Hofmann et al. 7	1,5,7	99.4	83.0	100	0	0	1	2
		Our Approach(RGB)	1,3	76.33	65.28	92.59	0.035	0.714	-	2

Table