A A Mekonnen
email: aamekonn@laas.fr

F Lerasle
email: lerasle@laas.fr

A Herbulot
email: aherbulo@laas.fr

C Briand
email: briand@laas.fr

Incorporating Computation Time Measures during Heterogeneous Features Selection in a Boosted Cascade People Detector

Keywords: People Detection, Feature Selection, Binary Integer Programming, AdaBoost

In this paper, we investigate the notion of incorporating feature computation time measures during feature selection in a boosted cascade people detector utilizing heterogeneous pool of features. We present various approaches based on pareto-front analysis, computation time weighted adaboost, and Binary Integer Programming (BIP) with comparative evaluations. The novel feature selection method proposed based on BIP -the main contribution -mines heterogeneous features taking both detection performance and computation time explicitly into consideration. The results demonstrate that the detector using this feature selection scheme exhibits low miss rates with significant boost in frame rate. For example, it achieves a 2.6% less miss rate at 10 -4 FPPW compared to Dalal and Triggs HOG detector with a 9.22x speed improvement. The presented extensive experimental results clearly highlight the improvements the proposed framework brings to the table.

Introduction

In modern era computer vision plays a significant role in automated people detection. It has vast pool of applications spanning many research domains, including but not limited to: Human-Robot Interaction, Human-Computer Interaction, Pedestrian Protection Systems (part of Advanced Driver Assistance Systems), Video Surveillance, and Automated Image Indexing and Management. Automated people detection involves perceiving the whereabouts of people in the information of a scene captured by a sensor. Depending on the mode of the sensor, this can mean localizing the accurate 3D position or rough 2D position of each person in the scene. Visible spectrum cameras (mostly called classical cameras) are the most widely used sensors as they capture very informative data covering wide spatial area, with color and texture information of the scene. This work focuses on this domain -people detection using visible spectrum cameras, commonly referred as visual people detection. Unfortunately, visual people detection is by far one of the most challenging tasks in computer vision, mainly due to physical variation of people, body deformations due to articulation, illumination variation, viewpoint change, background clutter, occlusions, sensor limitations, and computational constraints. In recent years astounding progresses have been made by the scientific community [START_REF] Benenson | Ten years of pedestrian detection, what have we learned?[END_REF][START_REF] Dollár | Pedestrian detection: An evaluation of the state of the art[END_REF][START_REF] Gerónimo | Survey of pedestrian detection for advanced driver assistance systems[END_REF], but there is still room for improvement.

In many applications, e.g., robotics, video surveillance, a real time people detector is required. In these domains the frame rate of the detector is as important as the accuracy of the detector. For example, a mobile robot needs to be reactive during navigation/interaction in human occupied environments. Thus, its people detection module -which is one component of an entire functioning system -should be fast. The advent of powerful camera systems in the robotic community that provide high resolution omnidirectional images, e.g., the Ladybug series [START_REF]Ladybug2[END_REF] from Point Grey, stresses this point further urging the need to give extra focus on computation time during detector design. Hence, we propose a framework that tries to address this during detector design.

In this work, we investigate different approaches to incorporate computation time measures during people detector design. Balancing computation time and detection performance is challenging; best detection results are obtained using complex features and descriptors which are computationally expensive. As an example, Histogram of Oriented Gradients (HOG) [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] is the most discriminant single feature thus far, but it is also computationally expensive compared to simple features like Haar variants [START_REF] Viola | Robust real-time face detection[END_REF]. Furthermore, most detectors that improve over HOG either use complex human models, e.g., parts based models [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF], or consider various heterogeneous pool of features, [START_REF] Benenson | Ten years of pedestrian detection, what have we learned?[END_REF][START_REF] Dollár | Fast feature pyramids for object detection[END_REF][START_REF] Walk | New features and insights for pedestrian detection[END_REF][START_REF] Wojek | A performance evaluation of single and multi-feature people detection[END_REF], both of which contribute to added computation time unless explicit computation considerations are made. In line with this, we present investigations on a people detector that uses heterogeneous pool of features and makes explicit computation time versus detection trade-off optimization to build a performant detector that leads to a significant gain in computation time while maintaining acceptable detection performance.

This paper is structured as follows: Section 2 presents an overview and related works on visual people detection. Section 3 presents the proposed people detection and detector learning framework. Subsequent sections present the different components of the proposed framework in detail, namely: features and weak classifiers, in section 4; and feature selection and classifier learning, in section 5. All experiments carried out and obtained results are detailed in section 6. Finally, the paper finishes with discussions in section 7 and concluding remarks in section 8.

Overview and Related Work

Undoubtedly, automated people detection is a very important research area with prominent applications. All methods in the literature more or less adhere to the generic scheme depicted in figure 1. For a given input image, possible candidate windows are hypothesized. Using the person model adopted, the original raw image input is transformed into a convenient format by extracting certain types of features that capture specific cues. Finally, each hypothesis is labeled as either a person or not using a learned classification rule. Even though not shown in the figure, there is usually a last post-processing step in the form of Non-Maximal Suppression (nms) which merges multiple detections that may arise from the same person into one. The scheme shown in figure 1 shows the flow used during detection. The types of features, descriptors, classifiers, and exact person model employed is a detector design choice. But, the actual subset of features/descriptors to use and the exact classifier parameters are determined via a training, also called learning, phase using a training dataset that contains positive and negative instances in a supervised learning approach. Recently, unspervised learning approach based on convolutional neural networks have also been successful applied to person detection, notably Sermant et al. ConvNets [START_REF] Sermanet | Pedestrian detection with unsupervised multi-stage feature learning[END_REF] detector.

The entire literature in visual people detection is overwhelming and a discussion on the different techniques is beyond the scope of this paper (please refer to [START_REF] Benenson | Ten years of pedestrian detection, what have we learned?[END_REF][START_REF] Dollár | Fast feature pyramids for object detection[END_REF][START_REF] Dollár | Pedestrian detection: An evaluation of the state of the art[END_REF][START_REF] Gerónimo | Survey of pedestrian detection for advanced driver assistance systems[END_REF] for extensive surveys). The presentation here is limited to the approaches that are relevant for this work. We will primarily focus on approaches that rely on sliding window candidate window generation mode which generates candidate windows with a fixed aspect ratio by sampling at all positions and scales of the input image.

In visual people detection, a person can be modeled either as a holistic indivisible object or as a parts-based entity. The holistic representation [START_REF] Viola | Robust real-time face detection[END_REF][START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF][START_REF] Dollár | Integral channel features[END_REF], considers a person as a whole indivisible object. On the other hand, parts-based approaches [START_REF] Mikolajczyk | Human detection based on a probabilistic assembly of robust part detectors[END_REF][START_REF] Wu | Detection of multiple, partially occluded humans in a single image by bayesian combination of edgelet part detectors[END_REF][START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF], rely on detecting different parts of a body -either explicitly looking for a head, torso, arms, and legs or looking for implicit dividends -to detect a person. Holistic approaches are simple, have straight forward model training, and have reduced computation time, relatively speaking, during detection. But, they perform poorly with nonstandard poses (articulations) and partial visibility. On the other hand, parts based approaches are better suited for person detection thanks to their ability to better deal with partial occlusions, view point changes, and pose variations. However, they are difficult to train and computationally demanding during detection. They also perform perform poorly with lower resolution images as the parts require ample spatial support for robustness

The classification stage is responsible for labeling each candidate window generated and described in accordance with the abstraction adopted as either a person or not. This block can either output a binary label (person or non-person) or a continuous valued score that reflects its confidence, and can further be thresholded to provide a binary label. These classifiers are mostly trained with a discriminative learning algorithm given positive and negative example instances. Discriminative learning algorithms in addition to robust image representation are the key reasons to recent advances in people detection. The most frequently used discriminative classifiers for people detection are variants of Support Vector Machines (SVM) and Boosted classifiers. On few occasions Fisher's Linear Discriminant Analysis (LDA) [START_REF] Paisitkriangkrai | Fast pedestrian detection using a cascade of boosted covariance features[END_REF], and Artificial Neural Networks [START_REF] Szarvas | Pedestrian detection with convolutional neural networks[END_REF][START_REF] Zhao | Stereo-and neural network-based pedestrian detection[END_REF] have also been used; recently, Random Forest classifiers are also gaining a significant momentum [START_REF] Tang | Fast pedestrian detection by cascaded random forest with dominant orientation templates[END_REF].

Features enable us to capture the essence of the underlying scene by extracting meaningful information from a group of data points (pixels). Different features capture differing facets of the underlying scene and careful feature choice plays an important role on the detection performance. Early success in people detection was achieved using rudimentary Haar like features inspired by Haar Wavelets [START_REF] Papageorgiou | A trainable system for object detection[END_REF][START_REF] Viola | Robust real-time face detection[END_REF]. These features capture region intensity differences which has limited descriptive power. Especially considering the distinctive boundary of peoples' figures, which can better be captured using edges. This intuition led to the adoption of gradient based features: Edge Orientation Histogram (EOH) [START_REF] Levi | Learning object detection from a small number of examples: The importance of good features[END_REF][START_REF] Gerónimo | Haar wavelets and edge orientation histograms for on-board pedestrian detection[END_REF] and Histogram of Oriented Gradients (HOG) [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF]. Recently, variants of Local Binary Pattern (LBP) [START_REF] Ojala | A comparative study of texture measures with classification based on featured distributions[END_REF] features have also been burgeoning. Color features are rarely used in person detection because of the variability induced by clothing. But, color shows local similarity even over clothing. This notion was exploited by Color Self Similarity (CSS) features, proposed by Walk et al. [START_REF] Walk | New features and insights for pedestrian detection[END_REF], which proved successful by encoding similarities in different sub-regions. Looking at the trend in the literature, the gist in features used for people detection can be captured with two important terms: gradient and histogram. The most successful features consider image gradients with local pooling in the form of histograms. This is evident considering peoples' global silhouettes, illumination and contrast variations in imaging, and deformation in physical structure. In general, these considerations tend to lead to complex features that require increased computation time entailing more focus on computation time related optimizations. This being said, the next natural question would be, how about combination of features? Indeed, using a combination of features have shown to improve detection further, for example, the top 4 current best detectors (in terms of detection performance) in the state-of-the-art use a mixture of heterogeneous features [START_REF] Dollár | Fast feature pyramids for object detection[END_REF].

Heterogeneous features help capture complementary information useful to handle various detection challenges -the more complementary the features, the better. Many works in the literature have attested this complementary nature. Geronimo et al. [START_REF] Gerónimo | Haar wavelets and edge orientation histograms for on-board pedestrian detection[END_REF] showed this with Haar like features and EOH; Wang et al. [START_REF] Wang | An HOG-LBP human detector with partial occlusion handling[END_REF] with HOG and LBP; Wojek et al. [START_REF] Wojek | A performance evaluation of single and multi-feature people detection[END_REF] with Haar like features, HOG, and shape context features; Walk et al. [START_REF] Walk | New features and insights for pedestrian detection[END_REF] with a concatenation of HOG, Histogram of Flow (HOF [START_REF] Dalal | Human detection using oriented histograms of flow and appearance[END_REF]), and CSS. Similar conclusions were also made by Schwartz et al. [START_REF] Schwartz | Human detection using partial least squares analysis[END_REF] and Hussain and Triggs [START_REF] Hussain | Feature sets and dimensionality reduction for visual object detection[END_REF] using -HOG, color frequency, and co-occurrence features -and -HOG and LBP variant fea-tures -respectively. This is also true with the recently burgeoning integral channel features derivatives [START_REF] Benenson | Ten years of pedestrian detection, what have we learned?[END_REF].

Given heterogeneous pool of features, different ways can be used to build the final detector. Four main trends can be observed in the literature: (1) Direct concatenation [START_REF] Walk | New features and insights for pedestrian detection[END_REF][START_REF] Wojek | A performance evaluation of single and multi-feature people detection[END_REF] in which the different features are concatenated to make one high dimensional feature vector and an SVM used afterwards for classification. This is computationally costly owing to the complex feature and SVM weights applied in sliding window detection. Dimensionality reduction techniques after concatenation do improve detection performance but not detection speed [START_REF] Hussain | Feature sets and dimensionality reduction for visual object detection[END_REF][START_REF] Schwartz | Human detection using partial least squares analysis[END_REF]. (2) Direct boosting where an ensemble classifier is learned using the entire heterogeneous pool of features [START_REF] Dollár | Integral channel features[END_REF][START_REF] Wojek | A performance evaluation of single and multi-feature people detection[END_REF][START_REF] Gerónimo | Haar wavelets and edge orientation histograms for on-board pedestrian detection[END_REF]. The problem here is in boosting, on each iteration, the feature with the least weighted classification error is added to the ensemble irrespective of its computation time. This favors complex features resulting in computationally costly detector. (3) Coarse-to-fine hierarchical arrangement where a cascade is constructed using cheap features at the initial stages and using complex features at later stages [START_REF] Mogelmose | Twostage part-based pedestrian detection[END_REF][START_REF] Pan | Efficient and accurate face detection using heterogeneous feature descriptors and feature selection[END_REF]. This approach is quite advantageous and tries to find a balance between detection performance and speed. The concern is, how to decide which features to use at the different stages systematically? Both [START_REF] Mogelmose | Twostage part-based pedestrian detection[END_REF][START_REF] Pan | Efficient and accurate face detection using heterogeneous feature descriptors and feature selection[END_REF] adopt a heuristic based rule and use homogeneous family of features they deemed cheap at the initial stages, and homogeneous complex features at the latter. Finally, (4) computation time versus detection tradeoff. This notion has been considered in the works of Wu and Nevatia [START_REF] Wu | Optimizing discrimination-efficiency tradeoff in integrating heterogeneous local features for object detection[END_REF] and Jourdheuil et al. [START_REF] Jourdheuil | Heterogeneous adaboost with real-time constraints -application to the detection of pedestrians by stereovision[END_REF]. In all cases, they defined a criterion composed of feature detection performance and computation time in a multiplicative manner. But, considering a multiplicative factor masks the contributions from the corresponding objectives and is not guaranteed to be optimal.

In this paper, which is an extended version of our previous work Mekonnen et al. [START_REF] Mekonnen | People detection with heterogeneous features and explicit optimization on computation time[END_REF], we investigate different schemes to incorporate feature computation time measure during feature selection and primarily focus on Binary Integer Programming (BIP) based feature selection. The proposed BIP framework falls in the 4 th category; but, it can also be considered as a variant of coarseto-fine hierarchy in which the exact features to use at each cascade node are selected automatically via an optimization step. We use five pervasively used heterogeneous features that exhibit significant discriminative power and computation time differences, namely: Haar-like features [START_REF] Viola | Robust real-time face detection[END_REF], Edge Orientation Histogram (EOH) [START_REF] Gerónimo | Haar wavelets and edge orientation histograms for on-board pedestrian detection[END_REF], CSS [START_REF] Walk | New features and insights for pedestrian detection[END_REF], Center Surround Local Binary Patterns (CS-LBP) [START_REF] Heikkil | Description of interest rregions with local binary patterns[END_REF], and HOG [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] in a classical cascaded boosting configuration [START_REF] Viola | Robust real-time face detection[END_REF] with an added explicit optimization step based on BIP to select a subset of features that have the least combined computation time and achieve a stipulated detection performance.

Recently the trend in the community has shifted towards channel features based approaches and their derivatives thereof [START_REF] Zhang | Filtered channel features for pedestrian detection[END_REF][START_REF] Dollár | Fast feature pyramids for object detection[END_REF][START_REF] Nam | Local decorrelation for improved pedestrian detection[END_REF][START_REF] Zhang | Informed haar-like features improve pedestrian detection[END_REF][START_REF] Benenson | Seeking the strongest rigid detector[END_REF]. The basic principle is to transform the image into a set of feature channels and then to construct a feature vector by pooling over with a set of varying rectangular regions. These channel features are mostly used with boosting classifiers to learn a detector [START_REF] Benenson | Ten years of pedestrian detection, what have we learned?[END_REF]. Their conception stems back to integral channel features (ChnFtrs) [START_REF] Dollár | Integral channel features[END_REF]. The idea is quite similar to Haar like features, but in stead of sum-pooling on intensity images, it is done on heterogeneous features, for example, 6 orientation quantized HOG feature and LUV color feature channels. The majority of approaches utilizing channel features can be generalized using the concept of filtered channel features [START_REF] Zhang | Filtered channel features for pedestrian detection[END_REF] which basically considers the variants as applying a specific type of filter. Since these features are computed with the help of integral images over each channel, ignoring pre-processing steps, each feature will basically have the same computation time. These features are designed for fast computation. As a result, even though our BIP framework is quite generic (and works best with heterogeneous features in terms of both detection performance and speed), channel features and their derivatives are not considered. The majority of approaches utilizing channel features can be generalized using the concept of filtered channel features [START_REF] Zhang | Filtered channel features for pedestrian detection[END_REF] which basically considers the variants as applying a specific type of filter. For example, ChnFtrs [START_REF] Dollár | Integral channel features[END_REF] a filter composed of random rectangular shapes, ACF [START_REF] Dollár | Fast feature pyramids for object detection[END_REF] a filter with a 4 × 4 pixels pooling region, SquaresChnFtrs [3] a square pooling regions based filters of various sizes, LDCF [START_REF] Nam | Local decorrelation for improved pedestrian detection[END_REF] (short for locally decorrelated channel features) PCA bases as filter banks, and Checkerboards [START_REF] Zhang | Filtered channel features for pedestrian detection[END_REF] filters with checkerboard patterns. Even the InformedHaar [START_REF] Zhang | Informed haar-like features improve pedestrian detection[END_REF] detector that uses hand crafted (from a training sample) binary and ternary Haar like feature templates that conform to the shape averaged pedestrian gradient image obeys this generalization. Other detectors that are currently within the state-of-the-art but that do not fit in the filtered channel representation include: SketchTokens [START_REF] Lim | Sketch tokens: A learned midlevel representation for contour and object detection[END_REF], Regionlets [START_REF] Wang | Regionlets for generic object detection[END_REF], and Spatialpooling [START_REF] Paisitkriangkrai | Strengthening the effectiveness of pedestrian detection with spatially pooled features[END_REF]. Evidently, the literature also encompasses several other detectors. To name a few, using their commonly used acronyms: Po-seInv [START_REF] Lin | A pose-invariant descriptor for human detection and segmentation[END_REF], HikSvm [START_REF] Maji | Classification using intersection kernel support vector machines is efficient[END_REF], Pls [START_REF] Schwartz | Human detection using partial least squares analysis[END_REF], MultiFtr [START_REF] Wojek | A performance evaluation of single and multi-feature people detection[END_REF], and MultiFtr+CSS [START_REF] Wojek | A performance evaluation of single and multi-feature people detection[END_REF].

Furthermore, paradigms that stray from the usually manually crafted feature set selection/design with the use of deep learning frameworks, also known as deep networks, have emerged. Deep networks are gaining a strong momentum due the improved classification performance, and their ability to operate on raw input pixels [START_REF] Angelova | Realtime pedestrian detection with deep network cascades[END_REF][START_REF] Angelova | Pedestrian detection with a large-field-of-view deep network[END_REF][START_REF] Tian | Pedestrian detection aided by deep learning semantic tasks[END_REF][START_REF] Tompson | Efficient object localization using convolutional networks[END_REF]. These approaches rely on convolutional neural networks that automatically learn pertinent features, in a deep cascade, that are relevant to discriminate people from background. Even though, some authors claim better features for pedestrian detection have not yet been obtained [START_REF] Benenson | Ten years of pedestrian detection, what have we learned?[END_REF], the recent work of Angelova et al. indicate better performance along with improved speed [START_REF] Angelova | Realtime pedestrian detection with deep network cascades[END_REF][START_REF] Angelova | Pedestrian detection with a large-field-of-view deep network[END_REF] and seem to indicate promising future of deep networks in people detection. Contributions: The work presented in this paper makes two core contributions. First, it present different detector learning paradigms that incorporate computation time measures of features during feature selection -amongst which is a novel BIP formulation to mine heterogeneous features taking both detection performance and computation time into consideration. This optimization applied to heterogeneous features marks a key contribution. Second, the paper presents a thorough evaluation of the proposed person detectorusing both proprietary and public datasets -with detailed analysis of its performance compared to alternative approaches. The proprietary dataset, called Ladybug dataset (section 6.1.1), is composed of images acquired with the Ladybug2 omni-directional camera system mounted on a mobile robot. The high resolution and versatile images from the Ladybug camera series further underline the need for the proposed detector.

Proposed People Detector Framework

Evidently, the framework adopted to address the aforementioned objectives needs to be concerned by detector detection performance and its associated computation time. The most famous detector configuration suitable for these requirements is the attentional cascade detector configuration pioneered by Viola and Jones [START_REF] Viola | Robust real-time face detection[END_REF]. This configuration builds a cascade made of nodes resembling a degenerate tree. Each node rejects negative candidate windows and passes along potential positive windows onto the next stage for more scrutinized verification. Figure 2 illustrates this configuration made up of K nodes. Given a candidate window, it is passed along the cascade with a label T (for true) if it fulfills the test encountered at each node, otherwise it is rejected (labeled as F for false). A window is considered to be positive only if it makes it to the end of the cascade. This leads to an efficient structure that uses simple classifiers at the beginning of the cascade, which reject a majority of the negative samples, and complex classifiers as one progresses along the cascade speeding up detection drastically. This structure has gained wide acceptance and has even been applied in recent part-based approaches [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF].

F    TPR FPR τ    Pareto-Front Selection {(x i ,y i )} F F H bip (•)
Figure 3: Proposed cascade node training scheme using heterogeneous pool of features and binary integer programming.

Figure 3 shows a block diagram representation of the the proposed classifier learning paradigm. For a cascade node, given labeled positive and negative training sets, the heterogeneous features, F, described in section 4 are extracted and associated weak classifiers trained. Then a pareto-front extraction technique is employed to reduce the feature sets to a manageable size, F. This is followed by a multi-criteria discrete optimization based feature selection technique that identifies a subset of the features that fulfill the stipulated detection performance while exhibiting the smallest combined computation time. This is realized using Binary Integer Programming (BIP) -presented in section 5.1.2 -which furnishes a few set of selected features, F, taking the optimization objective and constraints into account. Finally, discrete AdaBoost is used to learn the nodal strong classifier, labeled as H bip (•). Each of these constituent components are presented in detail in subsequent sections.

Additionally, to highlight the pros and cons of the proposed framework, it is compared with several learning strategies, namely: a pareto-front and Ad-aBoost, a computation time weighted AdaBoost, and a random sampling and feature rearranging based learning strategies. The details of these techniques are provided in section 5.2.2.

Features and Weak Classifiers

This section presents the heterogeneous features along with corresponding weak classifiers used for each family of features and associated computation time analysis.

Heterogeneous Feature Set

In this work, we have chosen to use the following five family of heterogeneous features: Haar like features, Edge Orientation Histograms (EOH), Center-Symmetric Local Binary Patterns (CSLB), Color Self Similarity (CSS) features, and Histogram of Oriented Gradients (HOG). These choices are motivated mainly by two factors: (1) their frequent use in the literature for person detection, and (2) their complementary nature (in terms of both discrimination and computation requirements). EOH and HOG capture edge distributions, CSS focuses on color symmetry, and Haar-like and CS-LBP on intensity and texture variations. Each feature family is extracted within a given image window of 128 × 64 pixels denoted as R, a standard template size used prominently in people detection [START_REF] Dollár | Pedestrian detection: An evaluation of the state of the art[END_REF]. To generate the over-complete set of features, the position, width, and height of the region the features are computed is varied within the candidate window. In all references, (x, y) position refers to the top left corner of the region R relative to the top left corner of the candidate window, and (w, h) refers to the width and height of the region spanned for extraction.

Haar Like Features

Haar like features represent a fast and simple way to compute region differences. These features have been extensively used for face, person, and various object detections [START_REF] Papageorgiou | A trainable system for object detection[END_REF][START_REF] Viola | Robust real-time face detection[END_REF][START_REF] Lienhart | An extended set of haar-like features for rapid object detection[END_REF][START_REF] Gerónimo | Haar wavelets and edge orientation histograms for on-board pedestrian detection[END_REF]. For a given feature, the response is obtained by subtracting the sum of pixels spanned by the black region from the sum of pixels spanned by the white region, see figure 4. To incorporate various measures, we have used the extended Haar like features from Viola and Jones [START_REF] Viola | Robust real-time face detection[END_REF] and Lienhart and Maydt [START_REF] Lienhart | An extended set of haar-like features for rapid object detection[END_REF], which contain upright and tilted filters of various configurations as shown in figure 4.

If the operator Ω haar (R, x, y, w, h, ϕ) denotes the feature extracted (scalar value) in the overlaid region (x, y, w, h) within the candidate window R using Haar filter type ϕ, the over-complete set of Haar like pool of features, denoted as F haar , is obtained by extracting features for all possible combinations of x, y, w, h, ϕ in R.

Edge Orientation Histogram (EOH)

EOH is another popular feature set that has been used for people detection [START_REF] Gerónimo | Haar wavelets and edge orientation histograms for on-board pedestrian detection[END_REF]. These features represent ratios of gradients computed from edge orientation histograms. Within a given overlaid region, first gradients are computed. Then a gradient histogram is built by quantizing the gradient orientations. Finally, the ratios of each histogram bin with one another make up individual features. Similarly, let the operator Ω EOH (R, x, y, w, h, k b 1 , k b 2 ) denote the feature extracted in the overlaid region (x, y, w, h) within R by first building an edge orientation histogram and then taking the smoothed ratio of the histogram counts in any two bins k b 1 and k b 2 . Consequently, the over-complete EOH feature pool set, denoted F EOH , is constructed by extracting feature values for all possible combinations of x, y, w, h and any two bins k b 1 and k b 2 within R.

Local Binary Pattern (LBP)

LBP was initially proposed as a texture characterization features [START_REF] Ojala | A comparative study of texture measures with classification based on featured distributions[END_REF]. Since then, it has been used in many applications -primarily facial analysis [START_REF] Zhao | Dynamic texture recognition using local binary patterns with an application to facial expressions[END_REF], and person detection [START_REF] Mu | Discriminative local binary patterns for human detection in personal album[END_REF]. To date, many variants of LBPs have been proposed. In this work, we adhere to the Center-Symmetric Local Binary Pattern (CS-LBP) variant due to its small dimensional histogram and demonstrated good results on person datasets [START_REF] Heikkil | Description of interest rregions with local binary patterns[END_REF].

CS-LBP = s(n0 -n4)2 0 + s(n1 -n5)2 1 + s(n2 -n6)2 2 + s(n3 -n7)2 3 (1)
where, s(x) = 1 x ≥ 0 0 otherwise and n0,...,n7 are gray scale pixel values (figure 5a).

In our implementation, CS-LBP is computed over a 3 × 3 pixel region (best results reported in [START_REF] Heikkil | Description of interest rregions with local binary patterns[END_REF]) by comparing the opposite pixels and adding a modulated term according to equation 1 with respect to figure 5a. This gives a scalar value less than 16 which is assigned to the center pixel. This is done for all the pixels in the window. A sample raw feature image is shown in figure 5c (the values are scaled to aid visibility). Finally, the actual feature vector is extracted by constructing a CS-LBP histogram (figure 5e) over a given overlaid region, figure 5d. Let Ω CLBP (R, x, y, w, h) denote the feature vector constructed by making a histogram of all CS-LBP features within the region (x, y, w, h). Extracting feature vectors for all possible combinations of x, y, w, h within R with strides of 4 pixels in both direction gives the LBP feature pool, denoted F CLBP . The histograms have 16 bins corresponding to CS-LBP quantization levels.

Color Self Similarity (CSS)

Color features are rarely used in person detection because of the variability induced by clothing. Color, actually, shows local similarity even over clothing. CSS features, proposed by Walk et al. [START_REF] Walk | New features and insights for pedestrian detection[END_REF], encode similarities in different sub-regions. To compute the features, first the image window is subdivided into non-overlapping blocks of 8 × 8 pixels and within each block a 3 × 3 × 3 color histogram in HSV space is built with interpolation. Then similarities are computed by intersecting individual histograms. In [START_REF] Walk | New features and insights for pedestrian detection[END_REF], all histogram intersection values are concatenated to define one feature vector. But here, we define the intersection of one histogram block with the rest of the blocks as a single feature. With an 8 × 8 block size and 128 × 64 window size, there are 128 different blocks. The intersection of one block with the rest gives 127 scalar values (excluding intersection with itself). These scalar values all together make-up the feature vector computed for that specific block location. This is repeated for each block resulting in 128 different features -the CSS feature pool (F CSS ). Figure 6 shows three sample features computed at the crossed block positions; observe how neighboring blocks show similarity.

Histogram of Oriented Gradient (HOG)

As it has been mentioned, no other single feature has been able to supersede HOG features [START_REF] Dollár | Pedestrian detection: An evaluation of the state of the art[END_REF]. Hence, naturally, we have resorted to use them. In this proposed approach, we use the original HOG features proposed by Dalal and Triggs [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] along with their widely preferred/used computation, i.e., a cell size of 8 × 8 pixels, a feature block size of 2 × 2 cells and an 8 pixel horizontal and vertical stride. But, instead of using the entire descriptor as a single feature, we generate pool of features by concatenating only a subset of the block histograms. The main steps are illustrated in figure 7. Given a candidate window (figure 7a), cell histograms are computed (figure 7b), and neighboring cells are combined and normalized to compute block histograms (figure 7c). Then differing from the original proposition, a single feature is described by concatenating all block histograms inside a region parameterized by a starting location (x,y), width (w), and height (h) as shown in figure 7d. Finally, the entire feature pool F HOG is generated by varying x, y, w, and h for all possible positive values in the given candidate window. This leads to a total of 3360 features with dimensions ranging from 36 (smallest) to 7 × 15 × 36 = 3780 (highest and equivalent to the feature vector obtained when concatenating all block histograms).

Finally, the complete heterogeneous feature pool is determined by merging all heterogeneous feature pool sets, i.e., F = {F Haar , F EOH , F CLBP , F CSS , F HOG }. In consecutive sections, each individual feature is indexed by j, where j ∈ {1, 2, ..., |F|}.

Weak Classifiers

The complete heterogeneous pool of features comprises of scalar and multi-dimensional features. For all scalar features, i.e., Haar-like and EOH features, we have chosen to use decision trees as a weak classifier. A decision tree over a real valued scalar feature is equivalent to having multiple threshold values assigning different bands of the range for positive and negative samples. In light of the detection performances exhibited, linear SVM is used as weak classifier for HOG and CSS feature vectors. Unlike HOG and CSS, the total number of CS-LBP features is quite high (see table 1) and using SVM leads to an overwhelming training period. Consequently, for CS-LBP features, we employ Fisher's Linear Discriminant Analysis (LDA) [START_REF] Fisher | The use of multiple measurements in taxonomic problems[END_REF] with decision trees. Since LDA seeks to find a projection direction which better discriminates the two classes, we train an LDA projection vector for all the CS-LBP features once using the positive and negative samples provided at the first node of the cascade. Afterwards, the same projection vectors are used and only a new decision tree is trained on the projected data. Experimentally, this has lead to a good balance between weak classifier performance and training duration. Each weak classifier, associated with a unique feature, is denoted as h j and maps each instance of the training set X to a discrete label, h j : X → {-1, +1}.

Computation Time

The computation time of each feature is determined irrespective of any implementation optimization that can be done during detection, e.g., use of caches to buffer some features. This helps establish an upper bound. For each feature considered, the associated computation time is made up of two components. A part associated with image pre-processing (including rudimentary feature preparation, integral image computation, etc) and a second part pertaining to the feature extraction and necessary computation during detection (e.g., multi-dimensional feature projection). For a feature indexed by j, these are represented as τ p,j and τ e,j consecutively; the combined computation time of that feature becomes τ j = τ p,j + τ e,j . These values are determined by averaging over 1, 000 repeated iterations. The computationally cheapest feature, a two boxed horizontal Haar filter which takes a total time (τ j ) of 0.0535 µs to compute on a core i7 machine, is used as a reference to report computation time for other features. The range of computation time for each feature family is reported in table 1. The table summarizes the characteristics of the heterogeneous pool of features considered: the total number of features in each family, the minimum and maximum feature computation time (both pre-processing, τ p , and extraction, τ e ) along with the weak classifier used are listed. Table 1: Feature pool summary with minimum and maximum feature computation time in each feature family. Time is reported as a multiple of u = 0.0535 µs computed on a core i7 machine running at 2.4 Ghz. Haar like, and EOH features are computed with the help of integral images, which is accounted for in the pre-processing time component. In fact, each τ p,j of these feature families shares a fraction of the total time taken to construct the integral image proportional to the area it spans. Additionally, for Haar like features, a horizontal and vertical stride of 2 pixels are used to generate the over-complete set and for EOH gradient orientation quantization levels of 4 (give best results [START_REF] Gerónimo | Haar wavelets and edge orientation histograms for on-board pedestrian detection[END_REF]) and horizontal and vertical strides of 4 pixels are used.

Feature Selection and Nodal Strong Classifier Learning

As presented in section 3 and depicted in figure 3, the main focus of this work is a boosted cascade learning framework primarily utilizing binary integer programming for feature selection. The novelty lies on the BIP optimization formulation to select a subset of heterogeneous features with the least combined computation time that satisfy the required detection performance. Hence, in this section, the sequence of steps involved in the BIP based framework is initially presented under subsection 5.1. Consequently, to put the BIP based approach into perspective, the section continues with a presentation of other investigated boosted cascade learning alternatives in subsection 5.2.

BIP based Classifier Learning

The BIP based nodal strong classifier learning block diagram is shown in figure 3. The heterogeneous pool of features and their associated weak classifiers have been presented in section 4. The objective of this learning scheme is to build a nodal strong classifier H bip (•) from the extracted features F. This is essentially achieved with three constituent steps: Pareto-Front extraction, binary integer programming, and discrete AdaBoost training. Each of these components are presented as follows. The final cascade detector trained using this scheme is referred with the label BIP+AdaBoost.

Pareto-Front Extraction

Pareto-Front analysis deals with selecting the optimal solutions when faced with competing multi-objective optimization criteria -like in equation 2 where the objective is to minimize miss rate (MR), false positive rate (FPR), and computation time (CT). It is termed competing because one has to worsen the other objectives to improve itself. The optimal solutions for these kind of optimization are termed as the Pareto optimal solutions. Pareto-Front analysis is used to find these optimal solutions that make up the Pareto optimal set. By applying this, a subset of features that are Pareto optimal with respect to MR, FPR, and CT, are retained for further use. A feature x is said to dominate another feature x ′ , only if g l (x) ≤ g l (x ′ ) for all l ∈ {MR, F P R, CT }, and g l (x) < g l (x ′ ) for at least one l. Here, the g l (•) denotes the MR, FPR, or CT, determined using the weak learner trained with the corresponding feature x and evaluated on a validation set. On the other hand, a feature is said to be non-dominated if no other feature dominates it. The Pareto optimal features set, F, is actually the set that fulfills equation 3 which can easily be determined using algorithm 1. F ← F 1 3:

minimize g(x) = [g M R (x), g F P R (x), g CT (x)] s.t. x ∈ F (2)
F = {x ∈ F | ∀ x ′ ∈F g M R (x ′ ) ≥ g M R (x) ∨ g F P R (x ′ ) ≥ g F P R (x) ∨ g CT (x ′ ) ≥ g CT (x)} (3)
for each x ∈ F do 4:

add ← true 5:

for each x ′ ∈ F do 6:

if x dominates x ′ then 7:

F ← F\{x ′ }; continue; 8:

else if x ′ dominates x then 9:

add ← false; break; 10:

end if 11:

end for 12:

if add then F ← F {x} 13:

end for 14: return F 15: end procedure

Binary Integer Programming (BIP)

Given the substantially decreased feature set F using Pareto-Front analysis, a further subset of features F ⊆ F are selected using the BIP formulation presented here. BIP is a special case of integer programming where decision variables are required to be 0 or 1 (rather than arbitrary integers). It aims at minimizing a given linear objective function f = c T v subject to the constraints that A.v ≥ b, where v represents the vector of 0-1 variables (to be determined), c and b are known coefficient vectors, A is a matrix of coefficients (called constraint matrix). It is well-known that BIP is N P-hard in the strong sense, but in practice, branch-and-cut techniques can solve huge binary integer program very fast [START_REF] Roy | Valid inequalities for mixed 0-1 programs[END_REF][START_REF] Wolsey | Strong formulations for mixed integer programs: Valid inequalities and extended formulations[END_REF]. Here, BIP is used to select a subset of features that fulfill the detection performance stipulated (in terms of TPR and FPR) with the minimum combined computation time. With respect to the reduced feature set, F, the BIP solutions are going to be the optimal choices. The BIP selected feature set F is further used to build a strong nodal classifier with discrete AdaBoost (see subsection 5.1.3), H bip (•). The BIP formulation for feature selection is presented as follows.

Definition of parameters:. The following are list of parameters used in the optimization specification along with their definitions. A binary set is denoted as B = {0, 1}.

• N = {1, ..., n}: set of training sample indexes with n ∈ Z; a total of n training samples indexed by i;

• M = {1, ..., m}: set of weak learners indexes with m ∈ Z; a total of m weak learners indexed by j;

• y + ∈ B n , y + = y + i i∈N ; y -∈ B n , y -= y - i i∈N ; notice y - i + y + i = 1 ∀i ∈ N y + i = 1 if i is positive 0 else y - i = 1 if i is negative 0 else • H ∈ B nxm where H = {h i,j } i∈N j∈M with h i,j ∈ {0, 1}
h i,j = 1 if weak learner h j detects sample i as positive 0 else

• TPR ∈ [0, 1]: minimum true positive rate set at the considered node of the cascade;

• FPR ∈ [0, 1]: maximum false positive rate at the node;

• T ∈ R m : with T = {τ j } j∈M computation time of weak learner j.

Decision Variables:. In BIP, the decision variables are restricted to binary values, values from the set B = {0, 1}. The BIP decision variables are the following.

• v ∈ B m , v = {v j } j∈M v j ∈ {0, 1}: v j = 1 if weak learner h j is selected, else v j = 0;
• t ∈ B n , t i ∈ {0, 1}: t i = 1 if a positive sample i has been detected as positive (true positive) by at least one selected weak learner, else t i = 0;

• f ∈ B n , f i ∈ {0, 1}: f i = 1 if
a negative sample i has been detected as positive (false positive) by at least one selected classifier, else f i = 0.

Let vector p = {p i } i∈N = Hv, which denotes the total number of weak learners that have labeled each training sample i as positive.

Objective Function and Constraints:.

min T T v (1) s.t t i ≤ y + i • p i ∀i (2) f i ≥ y - i • h i,j • v j ∀(i, j) (3) t 1 ≥ y + 1 • TPR (4) f 1 ≤ y - 1 • FPR (5) v ∈ B n ; t = {t i } i∈N , f = {f i } i∈N ; t, f ∈ B n (6) •
1 is the l 1 norm. The objective function (1) aims at minimizing the computation time. Constraints (2)-( 5) express that a given rate of detection quality has to be reached (depending on the number of true and false positives). Constraints (2) link v j and t i variables (via p i ) so that t i = 0 if image i has not been well-recognized by at least one selected classifier. Since it is at least, it is not expressed with respect to each selected weak classifier j -but to the sum of all selected classifiers (sum over j, the entries of p i as it is). Constraints (3) link v j and f i variables so that f i = 1 if a negative image i has been recognized as positive by at least one selected classifier. A negative sample should be correctly labeled (as 0) by all selected weak classifiers. It is considered as a false positive (f i = 1) if at least one selected classifier labels it as positive. So in this case, there will be a distinct constraint for each sample and feature combination, hence the need for the ∀(i, j). Constraint (4) expresses that the rate TPR of true positives, obtained with the selected classifiers, has to be reached. Similarly, constraint [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] expresses that the rate FPR of false positives, obtained with the selected classifiers, must not be exceeded. In this formulation, there are a total of (n • (m + 1) + 2) number of constraints, which could be huge for large n and m values. The final subset of features F corresponds to only the selected features, i.e., non zero v entry; since each feature indexed by j is associated with a unique weak learner h j , F also represents the subset of weak learners retained. At this point, a classification rule can be materialized by looking at p i for a given sample i. If p i is greater than zero, it means one weak classifier out of the selected ones has labeled the sample as positive. But, in order to break this discrete nature, we instead use discrete AdaBoost to further train a strong classifier. This provides a means to: (1) improve the classifier's generalization to unseen samples with the help of weights assigned to the weak classifiers based on discriminative ability; and (2) provide additional flexibility to adjust the TPR and FPR margin, by varying a threshold value, with the same computation time -this could, for example, be useful to fine tune an already trained detector for a new dataset.

Discrete AdaBoost

Discrete AdaBoost is one instance of the Boosting classifier variants which build a strong classifier as linear combination (weighted voting) of a set of weak classifiers. Suppose, we have a labeled training set {(x i , y i )} {i = 1, ..., (n + + n -)} where x i ∈ X, y i ∈ Y = {-1, +1}, where n + and n -denote the number of positive and negative training samples respectively. Given a set of weak learners (features) F = {h j } j∈M , with M = {1, 2, ..., m} the total number of weak learners, that can assign a given example a corresponding label, i.e., h : x → y, Discrete Adaboost constructs a strong classifier of the form H(x) = T t=1 α t h t (x) with sign(H(x)) determining the class label. The t indexes connote the sequence of the weak learners and this specific classifier has a total of T weak learners. The specific weak learner to use at each iteration of this boosting algorithm and the associated weighting coefficients, α t , are derived minimizing the exponential loss, which provides an upper bound on the actual 1/0 loss [START_REF] Schapire | The boosting approach to machine learning: An overview[END_REF]. The complete algorithm for training a per node classifier with Discrete AdaBoost is can be found in [START_REF] Schapire | The boosting approach to machine learning: An overview[END_REF].

Other Approaches

To investigate the advantages and disadvantages of the above presented BIP based framework, we also consider other simpler variants that are essentially derived by altering some of the constituent components in the BIP based framework. In doing so, we present four different classifier learning approaches, namely: a Pareto-Front with AdaBoost, AdaBoost with random feature sampling, computation time weighted AdaBoost, and hierarchically arranged cascade. Their block diagrams are depicted in figures 9a, 9b, 9c, and 10 consecutively.

Pareto-Front with AdaBoost

In this mode, a cascade node is trained via a combination of Pareto-Front analysis and discrete AdaBoost. First, the presented Pareto-Front analysis is used to extract a subset of non-dominated features F from F, which are in turn used by discrete AdaBoost to build a strong nodal classifier H pb (•) as shown in figure 9a. This is simply jumping the BIP step in the framework presented in section 5.1 and directly using AdaBoost with the Pareto optimal

Heterogeneous

Features Extraction Weak Classifier Learning

Pareto-Front Selection

Discrete AdaBoost

F    TPR FPR τ    Pareto-Front Selection {(x i ,y i )} F F H bip (•)
(a) Pareto-Front with AdaBoost.

Heterogeneous Features Extraction

Weak Classifier Learning

Discrete AdaBoost with Random Sampling

F    TPR FPR τ    Discrete AdaBoost with Random Sampling {(x i ,y i )} F H rb (•)
(b) Random feature sampling with discrete AdaBoost.

Heterogeneous Features Extraction

Weak Classifier Learning

Computation Time Weighted AdaBoost features. The final cascade detector trained using this scheme is referred with the label Pareto+AdaBoost.

F    TPR FPR τ    Computation Time Weighted AdaBoost {(x i ,y i )} F Hct(•) (c

AdaBoost with Random Feature Sampling

Using AdaBoost directly on the extracted feature set, without any prior feature reduction, is the most classical detector learning scheme. It has been applied by various researchers following the pioneering work of Viola and Jones [START_REF] Viola | Robust real-time face detection[END_REF]. For a given cascade node, the node trains a nodal strong classifier using AdaBoost iteratively. On each iteration, AdaBoost selects a single feature that minimizes the weighted error over the training samples, assigns a proper weight to the associated weak classifier, and adds it to the ensemble. This entails evaluating the error term, ǫ j , for all the weak classifiers in the pool. Given that we have a total of 1,448,374 weak classifiers in our pool, exhaustive search is not feasible. Therefore, we randomly sample a total number of R s weak classifiers (in accordance with the relative proportion of features from each family) on each AdaBoost iteration and select the one that minimizes the classification error. The nodal strong classifier trained with this scheme is denoted as H rb (•) -figure 9b -and the corresponding cascade labeled as Random+AdaBoost.

Computation Time Weighted AdaBoost

This scheme is quite similar to the above discussed variant based on random sampling and discrete AdaBoost (subsection 5.2.2) and has appeared in our previous work [START_REF] Mekonnen | Person detection with a computation time weighted adaboost[END_REF]. We use a modified discrete AdaBoost that not only takes detection performance, i.e., minimize ǫ j for selecting the best feature, but rather selects the one that minimizes a multiplicative term composed of feature computation time and detection error:

h t = arg min h j ∈F τj * ǫ j (4) τj = τ β j l τ β max,l (5)
where τj denotes a smoothed normalized feature computation time. β ∈ [0, 1] is an exponential smoothing coefficient. τ max,l denotes the maximum computation time registered within each distinct feature pool family, i.e., l ∈ {Haar, EOH, CSLBP , CSS, HOG}.

The computation time associated with each feature, τ j = τ p,j + τ e,j , is not constant (consequentially τj changes too). The exact value evolves during the classifier learning stage. It changes in two cases. The first is when a feature that has already been selected is considered in future cascade nodes, and the second is when a feature from the same family gets selected. In the prior case, the computation time of the selected feature is replaced by a constant time, τ 0 , in future references which accounts for only memory access. In the latter case, the computation time for all of the features in the same family gets affected, specifically, the time associated with the pre-processing stage, τ p,j , is set to zero for all the features in that family. This is logical and is done to favor features of the same family. For example, if a Haar feature is selected, it will be better to consider another Haar feature so the integral image computation can be done once for the area spanned by the two features, rather than considering another feature from a different family which will require a different pre-processing step. This way the computation time of the features within the same family will be levied contributing to speed up. Accordingly, the normalized computation time of all affected features is updated.

Algorithm 2 Computation Time Weighted AdaBoost

1: procedure Train AdaBoost(F, {(x i , y i )} i∈N ) 2: Initialize: D 1 (i) = 1 (n + + n -) 3:
for t = 1,2,...,T do 4:

• Find the best weak learner h t : 5:

→ F r ← randomly sample R s features from F 6: → Compute τj = τ β j i τ β max,i 7: → h t = arg min hj ∈Fr τj * ǫ j where ǫ j = n++n- i=1 D t (i)[y i = h j (x i )]
8:

• Compute weak learner weight:

α t = 1 2 ln 1 -ǫ t ǫ t 9: • Update data weight distribution: D t+1 (i) = D t (i) exp(-α t y i h t (x i )) Z t 10:
end for 11: end procedure This modification enables AdaBoost to select the feature (weak learner) that offers a compromise between computation time and detection error. This is detailed in algorithm 2 (main modifications on the classical one are shown in bold typeface). Again, due to the huge number of features, an exhausting search is not feasible and hence R s randomly sampled features are used, as in section 5.2.2. The nodal strong classifier trained with this scheme is referred as H ct (•), see figure 9c. The cascade trained with this learning algorithm is referred as CTWeightedAdaBoost.

Coarse-to-Fine Hierarchical Arrangement

In this approach, we want to investigate the performance of a cascade detector when it is manually tailored to use specific features in specific order in coarse-to-fine hierarchy. Taking detection and computation time into consideration, the initial cascade nodes will use computationally cheap features, while trailing nodes would take on incrementally complex features (similar to [START_REF] Mogelmose | Twostage part-based pedestrian detection[END_REF][START_REF] Pan | Efficient and accurate face detection using heterogeneous feature descriptors and feature selection[END_REF]). With the help of the feature computation time characteristics highlighted in table 1, a simple cascade is envisaged with the first G nodes using Haar like features, the next EOH features, then CS-LBP, followed by

Node1,1 Node1,2 ... Node1,G Node2,1 Node2,2 ... Node2,G Node3,1 Node3,2 ... Node3,G Node4,1 Node4,2 ... Node4,G Node5,1 Node5,2 ... Node5,G T T T T T T T T T T T T T T T T T T T T F F F F F F F F F F F F F F F
Haar like EOH CS-LBP CSS HOG CSS, and finally HOG features as depicted in figure 10. On each node, discrete AdaBoost with random node specific feature sampling is utilized to learn the nodal strong classifier, denoted H mb (•). Hence, at the node level heterogeneous feature pool is no more considered, it is rather a single feature family. But, globally, the entire cascade indeed considers heterogeneous features. The cascade node trained in this manner is labeled as Hierarchi-cal+AdaBoost.

In this section, the BIP based and several other alternative nodal cascade classifier learning paradigms have been presented. They are the BIP+AdaBoost, Pareto+AdaBoost, Random+AdaBoost, CTWeightedAdaBoost, and Hierar-chical+AdaBoost. Each node k of the cascade is trained to fulfill a stipulated nodal TPR k and FPR k detection performance constraints. In all cases, the cascade construction starts with all positive training samples and a subset of the negative training samples (equivalent to the positive ones) to learn the set of relevant features and classifiers for the initial cascade node, following the learning approach specified for each method. Once this is done, all negative training samples in the dataset are tested with it. All those that get classified correctly are rejected while all those labeled as positive samples (false positives) are retained along with the positive samples for training the following nodes. This step is repeated until all negative training sample are exhausted. This data mining technique makes it possible to use vast number of negative training samples.

Experiments and Results

In this section the different experiments carried out to investigate the performance of the investigated approaches and obtained results are presented. Principally, five different cascade detector training approachesbased on five differing nodal classifier learning schemes -using heterogeneous pool of features are presented. The evaluation aims to analyze the pros and cons the presented four different classifier learning schemes referred as: BIP+AdaBoost, Pareto+AdaBoost, Random+AdaBoost, CTWeightedAd-aBoost, and Hierarchy+AdaBoost.

Visual Datasets

All experiments are carried out using two datasets: A proprietary dataset refereed as the Ladybug dataset, and the public INRIA person dataset.

Ladybug Dataset

The Ladybug dataset is a custom compiled dataset using images acquired with the Ladybug2 camera in our robotic laboratory. This dataset features images of people acquired in a very cluttered indoor environment. The Lady-bug2 (figure 11a), manufactured by Point Grey Inc [START_REF]Ladybug2[END_REF], is a polydioptric camera that provides real omni-directional view without pronounced geometric, resolution, and/or illumination artifacts. It is a spherical omni-directional camera system that has six cameras mounted in such a way to view more than 75% of a full sphere. Each camera has a maximum resolution of 1024 × 768 pixels resulting in a 3500 × 1750 pixels stitched high resolution panoramic image (figure 11b).

INRIA Person Dataset

The INRIA person dataset, introduced by Dalal and Triggs [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF], is the most important and widely used public dataset for benchmarking purposes in people/pedestrian detection works. The dataset is divided in two formats, a format which contains original images (full images with people in natural scenes) with corresponding annotations and a second format which contains cropped positive images and people free negative images.

This format consists of both training and testing sets. The training set features 2416 cropped positive instances (originals and mirrored versions) and 1218 images free of persons. The cropped instances have a resolution of 160 × 96. But, the actual size of pedestrians bounding is 128 × 64. The extra padding is provided to minimize the border effect. In this work, a total of 2.55 × 10 6 negative cropped instances are generated from these person free images. The test set contains 1132 positive instances and 453 person free images. Similarly, 2 × 10 6 cropped negative windows are uniformly sampled from the person free images during testing. Sample cropped positive images from this dataset are shown in figure 12b.

Evaluation Metrics

To evaluate our trained detectors, we use the Per Window (PW) evaluation scheme. In addition, we also define a metric which we call Average Speed Up (ASU) to characterize the speed gain with respect to a known benchmark.

PW Evaluation: Detector Error Trade-off (DET)

As the name clearly suggests, the PM method determines performance based on cropped positive and negative image windows. This approach isolates classifier performance from overall detection system, thus, making it ideal for characterizing classifier performance [START_REF] Dollár | Pedestrian detection: An evaluation of the state of the art[END_REF]. A widely used metric in this category is Detection Error Trade-off (DET). This metric depicts DET curve with miss rate (MR) versus False Positives Per Window (FPPW) on a log-log scale [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF], where M R = 1 -T P R = 1 -T P T P +F N and F P P W = F P F P +T N . This metric is principally used to report comparative results of the various investigated detectors.

Average Speed Up (ASU)

We define the ASU criterion to compare the performance of a trained detector in terms of computation time. Recall that for a cascade detector the average computation time for a given candidate window is affected by the FPR of each node. Let K be the total number of nodes in the cascade, FPR k be the false positive rate and ζ k be the total computation time of the k th node during detection. Assuming the nodal FPR characteristics hold on a generic input image, the average time spent on a test candidate window, ζ av , can be estimated as

ζ av = K k=1 ( k-1 z=0 FPR z )ζ k . ASU = ζ HOG ζ av (6)
Using Dalal and Triggs [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF] detector -which takes ζ HOG per candidate window -as a reference, the ASU over it is determined using equation 6.

Consequently, the ASU values reported henceforth are with respect to Dalal and Triggs detector.

Implementation Details and Validation

Recall that the cascade node training is governed by two parameters: the nodal FPR k and TPR k . In all the experiments a nodal TPR k value of 1.0 and FPR k of 0.5 is used unless specified otherwise (the exception is the variant discussed in section 6.4.2). During training the TPR and FPR values exhibited by the weak classifiers or cascade node is determined on a separate validation set. Actually, the training dataset is initially divided into a 60% training and a 40% validation set; the new training set is used to train the weak classifiers and nodal classifiers whereas the validation set is used to determine detection performance exhibited. Once all feature selection is done, the node is retrained using the complete training set. This is applied when considering both the Ladybug and INRIA datasets (section 6.1).

In the following subsections, the validation steps taken to determine free parameters that are crucial for the performance of the different presented nodal classifier training schemes are discussed. These parameters are: (1) the depth of the decision trees used (D), (2) number of features randomly sampled (R s ) with the Random+AdaBoost and CTWeightedAdaBoost strategies, and (3) the computation time smoothing coefficient (β) used in the CTWeightedAdaBoost variant. In all cases, a training and validation set from the Ladybug dataset is used. The depth of the decision trees for Haar like features (D haar ), EOH (D eoh ), and CS-LBP (D cslbp ) features are validated as follows. Using only a single cascade node, a strong classifier is trained using AdaBoost with each individual feature families, and its performance on a validation set analyzed. Multiple runs are performed using different decision tree depths. On each Ad-aBoost iteration, only 2, 000 randomly sampled features are used to limit the validation time to a reasonable duration. For Haar like, EOH, and CS-LBP features, the error rate on the validation set as a function of the AdaBoost iteration for different decision tree depths is shown in figures 13a, 13b, and 13c respectively. Based on this, decision tree depths of D haar = 2, D eoh = 3, and D cslbp = 3 are used. Computing Fisher LDA weights, for CS-LBP features, per each node makes the classifier over-fit on the training set with deteriorated performance on the validation set. Hence, the Fisher LDA weights computed at the first node are used throughout the cascade by learning only new decision trees.

Number of Randomly Sampled Features (R s )

At each AdaBoost iteration in Random+AdaBoost and CTWeightedAd-aBoost variants, we use a randomly selected subset of features (as is commonly done, e.g., in [START_REF] Zhu | Fast human detection using a cascade of histograms of oriented gradients[END_REF][START_REF] Wu | Optimizing discrimination-efficiency tradeoff in integrating heterogeneous local features for object detection[END_REF]). According to Scholkopf and Smola [START_REF] Scholkopf | Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond[END_REF](pp. 180), given a set of samples, it can be guaranteed to sub sample amongst the best r s percentage of estimates with a probability p by randomly sampling a sub sample of size log(rs) log(p) . This reduced set will do as well as considering the entire set with a probability p. In our case, to select amongst the best 5% features with a 99% probability, we need to sample a total of log(0.05) log(0.99) ≈ 299 samples. In our implementation we use 3000 features which is way above the suggested number of samples and guarantees inclusion of relevant features with a high probability. Hence, R s = 3000. The computation time smoothing exponential factor, β, used in the time weighted AdaBoost is determined empirically through a validation step. The CTWeightedAdaBoost is used to learn a single nodal cascade using different β values on a subset of the training set. Then the classification errors on a validation set and the conglomerated computation time of the trained node is determined to select the best value that offers a good trade-off. Figure 14 shows the validation result plots for different values of β. Clearly, higher β reduces smoothing, in effect, features with low computation time dominate improving speed but with poor detection performance. Lower values favor complex features. As a compromise, a β value of 0.2 is used to train the final cascade classifier in this scheme. The different parameters tuned in the validation step are summarized in table 2.

Results

The results corresponding to all experiments are reported in this section categorized under each dataset.

Ladybug Dataset

We train five different cascade detectors (see section 5) using its training set and test on its test set. Since this dataset is composed of cropped positive samples, we use the PW evaluation to determine detection performance. The main obtained results are depicted in figure 15 and summarized in table 3. Clearly Pareto+AdaBoost and Hierarchy+AdaBoost result in the best detection performance, 2.9% MR, followed by Dalal and Triggs detector trained on this dataset, 3.0%, at 10 -4 FPPW. CTWeightedAdaBoost shows the lowest detection performance with a 10% MR at 10 -3 FPPW, but it manages to learn a detectors that is 1.8× faster than Dalal and Triggs HOG. The Hier-archy+AdaBoost detector achieves a 8.4× speed improvement. In terms of detection, BIP+AdaBoost trails behind Random+AdaBoost with marginal loss. But, the most important result to notice is that BIP+AdaBoost results in a drastic 42.7x speed up over Dalal and Triggs with only a 7% loss in MR at 10 -4 FPPW. The main reason for this speed up is that BIP+AdaBoost systematically uses cheap features in the initial stages of the cascade and only starts using computationally expensive features at later stages. The trained classifier has 10 cascade nodes with CSS features initially appearing at the 6 th node and HOG at the final stage; figure 16 depicts this showing the selected features at some of the nodes. Apparently, Pareto+AdaBoost and Random+AdaBoost result in worsened speeds. This is because AdaBoost always privileges the most discriminant feature, irrespective of computation cost, from the pool of features passed to it, and both Pareto-Front extraction and random sampling are likely to pass such kind of complex features. Consequently, the set of features selected in the first node result in a conglomerate that is effectively computationally demanding than Dalal and Triggs detector. CTWeightedAdaBoost improves upon this by selecting a slightly less complex feature. The intuition of using cheaper features initially followed by complex features, the Hierar-chical+AdaBoost detector, pays off both in terms of detection performance and computation time improvement. But, the computation time improvement is much lower than the BIP+AdaBoost variant. Figure 17 shows the features selected in the initial cascade of the four trained detectors. The results for Hierarchy+AdaBoost are not shown as they convey no useful information -containing 14 Haar like features that clutter the window. Ran-dom+AdaBoost, Pareto+AdaBoost, and CTWeightedAdaBoost, have HOG features in this node contributing to reduced speed; on the contrary, for BIP+AdaBoost, only CS-LBP and Haar features are used. These result are obtained using a fixed nodal FPR k of 0.5 for all constructed nodes and the obtained results are very precise that altering the FPR is not necessary. The proportion of features present from each family is also consistent with the underlying training scheme adopted (see Table 4). Both CTWeightedAd-aBoost and BIP+AdaBoost emphasize on computation time, accordingly, they have the highest proportion of Haar features in their trained model. They also have the least proportion of HOG features taking only 3.3% and 2.8%, respectively, of the total proportion of features. Pareto+AdaBoost favors complex features with superior detection performance which explains the 83.7% HOG features presence. Similar results obtained for the INRIA dataset are shown in figure 18 and summarized in table 4. As this dataset is challenging, two variants of the BIP+AdaBoost classifier are trained. In the first case, a fixed nodal FPR k of 0.5 is used for all nodes, called BIP+AdaBoost (Fix). In the second case, an adaptive FPR is employed which starts at 0.3 in the initial stage and continues training nodes, whenever a solution for the BIP optimization does not exist, this constraint is relaxed/incremented by 0.1 and the procedure continues from that node likewise until all negative samples are depleted. This is called BIP+AdaBoost (Ad). The best detection results at 10 -4 FPPW are obtained by the Random+AdaBoost and Hierarchy+AdaBoost variants (Pareto+AdaBoost is edged out marginally). But, this time both variants of BIP+AdaBoost beat Dalal and Triggs detector at 10 -4 by more than 2%. On top of this, the BIP+AdaBoost(Fix) achieves a 15.6x speed up while that of BIP+AdaBoost (Ad) trails with a 9.22x speed up. As expected the Hierarchy+AdaBoost also results in improved speed (an ASU of 4.05×). Random+AdaBoost, Pareto+AdaBoost, and CTWeightedAdaBoost variants on the other hand result in increased computation time (even with respect to [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF]). The reason is all these three start off with complex features in the initial nodes. Figure 19 shows the features selected in the first node of all trained detector variants. Only the ones employing BIP do not have HOG features in the initial stage. Even though the CTWeightedAdaBoost variant does not result in a significant boost in speed, it is twice as much faster as its random counterpart (Random+AdaBoost) with marginal loss in miss rate. The explicit computation time consideration does help even in this case. Concerning the BIP+AdaBoost variants, as the initial FPR constraints are stringent on the BIP+AdaBoost (Ad) variant, it favors relatively discriminant features with increased computation time. This contributes to its superior detection performance, over BIP+AdaBoost (Fix), throughout the FPPW range shown in figure 18. Observe in table 4, there are more proportion of Haar like features (5.4% more) and less proportions of HOG features (2.0% less) in the fixed variant compared to the adaptive variant which results in the increased speed. Figure 20 illustrates a few of the selected features overlaid on an average human gradient image for BIP+AdaBoost (Ad). Observe that all selected features capture discriminant facets of people. Figure 21 shows histogram of the selected features, with relative proportions, for the first 9 nodes of both the fixed and adaptive variants. Clearly, the fixed variant initially uses cheaper features and increases along the cascade both in number and complexity. On the contrary, for the variable variant, complex features appear in the initial nodes and increase in number along the cascade.

INRIA Dataset

Discussions

The features presented in this work capture varying cues relevant to people in an image. They are highly variable in terms of discriminative ability as well as computation time. Given this kind of feature pool with diverse characteristics, extra care should be given to the way these features are mined to build a people detector. This work favors a sparser solution by employing feature selection to reduce the set to a few performant ones.

Settling on feature selection, to mine relevant features out of the pool, we have investigated five different approaches all based on the popular AdaBoost with cascade of nodes detector. The approaches are: Random+AdaBoost, which uses discrete AdaBoost to learn a strong nodal classifier by randomly sampling R s features on each boosting iterations and adding the best one to the ensemble; Pareto+AdaBoost, which uses Pareto-Front analysis to retain only non-dominated features with respect to TPR, FPR, and computation time, and then builds a nodal strong classifier with discrete AdaBoost and the retained subset of features; CTWeightedAdaBoost, which makes use of a modified AdaBoost that selects the best features using the classification error weighted by a computation time measure to enable AdaBoost to give consideration for feature computing speed; Hierarchy+AdaBoost, which takes on a coarse-to-fine hierarchical arrangement approach to use computationally cheap features in the initial nodes of the cascade and then consider incrementally complex features in subsequent cascade nodes; and finally, BIP+AdaBoost which uses a discrete optimization formulation based on BIP to retain the minimum number of features that fulfill the stipulated detection performance with the minimum combined computation time. All five approaches have been evaluated on proprietary and public datasets thoroughly and compared to the state-of-the-art. In terms of detection, all five achieve between 6.0% and 14.6% miss rates at 10 -4 FPPW on the INRIA dataset which confirms that considering heterogeneous features is relevant. The Random+AdaBoost variant avoids the need to iterate through all features in the pool, at each boosting iteration, which would have otherwise made the training non-realistic. As long as the number of randomly sampled features are high, it will lead to comparable results as the exhaustive search. In terms of detection, this approach is expected to lead to the best (compared to the four variants) result as it always picks the best discriminative features iteratively. The downside of this approach is, reflecting the properties of AdaBoost, it always selects features based on classification performance blind to feature computation time. If there are two competing features with only a slight difference in classification error but big difference in computation time, the minimum error feature will be selected. This characteristics is exemplified in the evaluations as it leads to the worst detector speed, 0.4× Dalal and Triggs HOG, with amongst the best detection performance, e.g., 6.0% MR at 10 -4 FPPW on the INRIA dataset, in all cases.

To overcome the shortcomings of Random+AdaBoost, we investigated CTWeightedAdaBoost variant. In the experimental results, this approach achieves approximately 2× and 3× as much faster detector compared to the Random+AdaBoost variant on the Ladybug and INRIA trained models respectively. But, the modification down plays its detection performance leading to reduced detection rate. For example it achieves a 5.6% MR reduction at 10 -4 FPPW compared to Random+AdaBoost on the INRIA evaluation. The third investigated scheme, Pareto+AdaBoost variant, avoids the exhaustive search that needs to be done by AdaBoost and yet is guaranteed to pass on the most performant features. This is clearly seen by the low miss rate it exhibits on the test cases, 2.9% and 7.0% MR at 10 -4 FPPW on the Ladybug and INRIA dataset evaluations respectively. But, again as long as computationally intensive discriminant features exist in the selection, which is actually the case as observed in the experiments, AdaBoost is bound to greedily favor the discriminant ones leading to computationally demanding detector. These remarks are seen on the evaluation results, for example on the INRIA test set, it achieves the second highest detection rate, 7.0% miss rate at 10 -4 FPPW, with the least frame rate, more than twice slower than Dalal and Triggs HOG.

The fourth scheme termed as Hierarchy+AdaBoost uses a unique family of features in each node. It starts with computationally cheap features, the Haar like features, and continues until HOG, training G number of nodes with each feature family, a kind of coarse-to-fine hierarchical detector. This simple approach does indeed lead to both improved detection and speed compared to Dalal and Triggs HOG -a 4.05× speed gain and a 3.2% improvement on MR at 10 -4 FPPW. But, as it has been demonstrated, the speed gain is inferior to the BIP based approach.

The main approach presented, BIP+AdaBoost, makes explicit optimization to select the features that achieve the required detection performance with the minimum possible computation time. The two modes investigated in this scheme are learned using a fixed nodal FPR and an adaptive nodal FPR. Both variants result in a detector that is most considerate as both detection and speed aspects are taken into account to come up with the best compromise. The BIP+AdaBoost (Fix) variant for example achieves a 10.0% and 8.0% MR at 10 -4 on the Ladybug and INRIA datasets respectively. It contains significant proportions (more than 54% on the Ladybug model and more than 60% on the INRIA model) of Haar features and less proportions of the costly features, e.g., only 2.8% and 11% HOG features in both datasets respectively. This helps it achieve a 42.7× and 15.6× speed up over Dalal and Triggs HOG using the Ladybug and INRIA trained models respectively. Its adaptive variant trained on the INRIA dataset improves the detection further achieving a 7.4% MR at 10 -4 FPPW with a 9.22× speed up over Dalal and Triggs HOG. The trained detector has more proportion of Haar features (55.0%) and less proportion of HOG features (13.0%). Hence, it can be safely concluded that the BIP based detector variants are the most considerate ones as they work on both detection and speed aspects to come up with the best compromise.

Another advantage of the BIP based framework is its flexibility with respect to computational resource constraints and detection requirement. On any dataset, the stipulated detection parameters used during training (nodal TPR k and FPR k ) can either be made stringent or relaxed to learn a model that can either consume more or less computational resources respectively, giving explicit control on the detection vs speed trade off. This has been demonstrated with the adaptive and fixed detector variants trained on the INRIA dataset.

Additionally, another flexibility of the framework is its ability to adapt the complexity of the learned model to the challenge inherently present in the training dataset. The framework takes advantage of the underlying challenge manifested by the training dataset to furnish an appropriate detector model. For simpler datasets, it provides simpler model with increased frame rate, e.g., the 42.7x improvement achieved with the Ladybug dataset trained model contrary to the 9.22x improvement achieved with the INRIA dataset. This quality would enable developing a detector that is suited for specific scene/domain that reflects on the detection challenge, e.g., for indoor open environment (like the hall of a shopping mall) application that might feature less background clutter with upright people having less pose variability, with faster frame rates. Most of the detectors listed in the state-of-the-art do not have the ability to automatically change the complexity of the detector based on the dataset. As an example, consider Dalal and Triggs HOG [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF], HogLbp [START_REF] Wang | An HOG-LBP human detector with partial occlusion handling[END_REF], LatSvm-V1 [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF], which have a fixed size high-dimensional classifier that is always fixed irrespective of the dataset.

Conclusions

In this work, different strategies to train a people detector using heterogeneous pool of features have been investigated. Various experiments have been carried out to investigate the advantages and shortcomings of each strategy using proprietary and multiple public datasets. The obtained results ascertain that complementary heterogeneous features lead to improved detection performance, and under explicit consideration of computation time, lead to improved frame rate as well. The different results also show the superiority of the proposed BIP based feature selection strategy. The proposed BIP strategy is quite capable in taking advantage of the diversity that exists in the feature pool from detection as well as speed perspectives. Further improved frame rates can also be achieved by parallelizing the trained model with the help of specialized hardwares like a Graphical Processing Unit (GPU).

In the near future, we plan to investigate ways to achieve more faster versions of the detector by focusing on implementation optimization and specialized accelerator hardwares. Additionally, we are in the process of integrating the proposed BIP based detector in a "tracking-by-detection" framework for multi-person tracking. This will be used in robotic applications to better identify the trajectory of each individual in the vicinity for acceptable human-aware robot navigation.

Figure 1 :

 1 Figure 1: Important components of a visual people detector.

Figure 4 :

 4 Figure 4: Set of extended Haar like feature types (configurations) used.

Figure 5 :

 5 Figure 5: CS-LBP feature extraction steps. (a) Pixel neighborhood for use with equation 1 (8-connectivity), (b) original candidate image, (c) dense CS-LBP per pixel computed values, (d) one specific feature specified by a bounding box, and (e) actual feature vector extracted from (d).

Figure 6 :

 6 Figure 6: Illustration of sample CSS features.

Figure 7 :

 7 Figure 7: Illustration of the HOG feature pool set generation.

Figure 8 :Figure 8

 88 Figure 8: Sample Pareto-Front extraction. (Best viewed in color.)

)

  Computation time weighted AdaBoost.

Figure 9 :

 9 Figure 9: Block diagrams of three other nodal strong classifier learning approaches.

Figure 10 :

 10 Figure 10: Coarse-to-fine hierarchically arranged sequence of cascade nodes.

Figure 11 :

 11 Figure 11: Ladybug2 camera and a corresponding stitched image.

Figure 12 :

 12 Figure 12: Sample positive images from the Ladybug and INRIA datasets.

6. 3 . 1 .

 31 Decision Tree Depth (D)

Figure 13 :

 13 Figure 13: Decision tree depth validation for (a) Haar like features -D haar , (b) EOH features -D eoh , and (c) CS-LBP features -D cslbp .

Figure 14 :

 14 Figure 14: Error rate on a validation set using the computation time weighted AdaBoost trained with different β values.

Figure 15 :

 15 Figure 15: DET of different detectors trained and tested on the Ladybug dataset.

Figure 16 :

 16 Figure 16: Illustration of the different features selected on different nodes of cascade (superimposed on an average human gradient image) of BIP+AdaBoost trained on Ladybug dataset. Black and white rectangular regions are Haar features, red for EOH, blue for CS-LBP, crossed white boxes for CSS, violet HOG features.

  Pareto+AdaBoost (c) CTWeightedAdaBoost (d) BIP+AdaBoost

Figure 17 :

 17 Figure 17: The features selected and used in the first node of the cascade under the different learning approaches with the Ladybug dataset superimposed on an average human gradient image. Black and white rectangular regions show Haar features, blue for CS-LBP, crossed white boxes represent CSS features and their position indicates the reference block, and finally, violet shows the spatial region spanned by the concatenated HOG blocks.

Figure 18 :

 18 Figure 18: DET of different detectors trained and tested on the INRIA dataset.

Figure 19 :

 19 Figure 19: The features selected and used in the first node of the cascade trained on the INRIA dataset superimposed on an average human gradient image. Black and white rectangular regions are Haar features, blue for CS-LBP, red for EOH, crossed white boxes for CSS, violet HOG features.

Figure 20 :Figure 21 :

 2021 Figure 20: Sample depictions of the heterogeneous features selected at different nodes of the cascade BIP+AdaBoost (Ad) trained on the INRIA dataset using an adaptive FPR. Black and white rectangular regions are Haar features, blue for CS-LBP, crossed white boxes for CSS, violet HOG features.

  ) min (τ e ) min (τ p ) max (τ e ) max

	Feature Type No of features (τ p Haar like 672,406 0.6u	τ min	0.40u	1.88u	τ max	1.60u	Weak Classifier Decision Tree
	EOH	712,960	2.72u		2.11u	315.65u		2.10u	Decision Tree
	CS-LBP	59,520	1.24u	14.26u 111.60u	282.04u	LDA + Decision Tree
	CSS	128	560.75u 457.19u 560.75u	457.19u	SVM
	HOG	3,360	10.59u 479.12u 315.75u 51103.80u	SVM

Table 2 :

 2 Values of different parameters obtained with the validation step.

	Parameter	Description	Value
	TPR k	required True Positive Rate for node k (fixed case)	1.0
	FPR k	required False Positive Rate for node k (fixed case)	0.5
	D haar	decision tree depth for Haar like features	2
	D eoh	decision tree depth for EOH features	3
	D cslbp	decision tree depth for CS-LBP features	3
	R s	number of randomly sampled features	3000
	β	computation time smoothing coefficient	0.2
	G	number of cascade nodes per feature family (for Hierarchical + AdaBoost )	3 (ladybug training) 5 (inria training)

Table 3 :

 3 Summary of the cascade detector trained on the Ladybug dataset. Miss rate is reported at 10 -4 FPPW.

	Detector		Feature Proportion		MR	ASU
		Haar	CS-LBP	CSS	EOH	HOG
	Dalal and Triggs[5]	-	-	-	-	100%	3.0%	1.0x
	Pareto + AdaBoost	10.7%	0.0%	0.0%	0.0%	83.7%	2.9%	0.7x
	CTWeightedAdaBoost	53.3%	33.3%	0.0%	10.0%	3.3%	25.0%	1.8x
	Random + AdaBoost	51.6%	6.2%	1.5%	36.0%	4.7%	8.0%	0.6x
	Hierarchy + AdaBoost 17.8%	67.4%	7.6%	5.1%	2.1%	2.9%	8.4x
	BIP + AdaBoost	54.3%	8.6%	8.5%	25.7%	2.8%	10.0%	42.7x

Table 4 :

 4 Summary of the cascade detector trained on the INRIA datasets. Miss rate is reported at 10 -4 FPPW.

	Detector		Feature Proportion		MR	ASU
		Haar	CS-LBP	CSS	EOH	HOG
	Dalal and Triggs[5]	-	-	-	-	100%	11.0%	1.0x
	Pareto + AdaBoost	42.8%	14.5%	7.8%	25.6%	9.3%	7.0%	0.4x
	Random + AdaBoost	26.3%	10.8%	3.7%	53.5%	5.6%	6.0%	0.4x
	CTWeightedAdaBoost	86.7%	9.1%	2.4%	0.0%	3.9%	14.6%	0.8x
	Hierarchy + AdaBoost 26.6%	9.7%	5.7%	24.9%	33.%	6.8%	4.05x
	BIP + AdaBoost (Fix) 60.4%	10.8%	8.0%	9.7%	11.0%	8.0%	15.6x
	BIP + AdaBoost (Ad) 55.0%	14.6%	8.1%	9.3%	13.0%	7.4%	9.22x

This is done for demonstration purposes but in the actual cascade construction all five feature pools without sub-sampling are considered.

Acknowledgment

This work was supported by a grant from the French National Research Agency (ANR) under project RIDDLE with grant number ANR-12-CORD-0003.