Emanuel E Zelniker
email: zelniker@dcs.qmul.ac.uk

Shaogang Gong

Tao Xiang Queen

Global Abnormal Behaviour Detection Using a Network of CCTV Cameras

This paper investigates the detection of global abnormal behaviours across a network of CCTV cameras. Although the problem of multiple camera tracking has attracted much attention recently, little work has been done on modelling global behaviours of objects monitored by a network of CCTV cameras with disjointed camera views, and no effort has been taken to tackle the challenging problem of detecting abnormal global behaviours, which are only meaningful and recognisable when observed over space and time across multiple camera views. To that end, we propose a novel framework, which consists of object tracking across camera views, global behaviour modelling based on unsupervised learning, and probabilistic abnormality inference. The effectiveness of the framework is demonstrated with experiments on real-world surveillance video data.

Introduction

There are now large networks of CCTV cameras collecting colossal amounts of image data for safety and security purposes. A multiple camera and multi-sensor system has the potential for gaining better viewpoints resulting in both improved imaging quality and more relevant details being captured. However, more is not necessarily better. Such a system can also cause overflow of information and confusion if data content is not analysed in real-time. There is an urgent need for the development of automated systems to monitor cooperatively the behaviours of objects across a distributed network of cameras in order to enable on-the-fly decision-making.

Despite the wide use of networked CCTV systems, there is relatively little effort made worldwide on the development of effective and robust algorithms for utilising/sharing object behaviour information across multiple cameras for abnormal behaviour detection, as the problem is potentially very difficult and unconstrained. Recent studies on be-haviour profiling and abnormality detection have been focused on a single fixed camera view. Among the existing efforts using multiple cameras, almost all have been focused on tracking. However, what has emerged to be of critical importance and urgently required for assisting decision making is the capability for monitoring and correlating local behaviours in order to infer global abnormal behaviours in a wider context beyond any single camera view. To the best of our knowledge, none of the existing work addresses the problem of correlating behaviours across different camera views and detecting subtle abnormalities that are only meaningful and recognisable when observed over space and time across multiple camera views.

To that end, we propose a novel framework, which consists of object tracking across camera views, global behaviour modelling based on unsupervised learning, and probabilistic abnormality inference. In particular, objects are tracked within each camera view and their trajectories are then connected to form a representation of the global behaviour pattern. The connected global trajectories are then profiled using a clustering technique, upon which a model for the normal behaviours are constructed. The abnormality is detected when an unseen behaviour pattern has a low likelihood of being generated by the learned normal global behaviour model.

The literature is rich in trajectory clustering methods. A recent review paper discussing various methods of trajectory clustering in outdoor surveillance scenes can be found in [START_REF] Zhang | Comparison of similarity measures for trajectory clustering in outdoor surveillance[END_REF]. Much work has been invested in clustering trajectories in a viewfield of a single camera [START_REF] Wang | Learning semantic scene models by trajectory analysis[END_REF][START_REF] Dee | Detecting inexplicable behaviour[END_REF][START_REF] Makris | Path detection in video surveillance[END_REF]. The paper in [START_REF] Wang | Learning semantic scene models by trajectory analysis[END_REF] uses two similarity measures. One similarity measure is used to cluster trajectories into vehicles or pedestrians based on the size of the targets. Another similarity measure is used to cluster vehicles and pedestrian trajectories into different clusters using spectral clustering based on the spatial and velocity information from the trajectory of the targets. Abnormal trajectories are detected as outlier tracks, i.e., those which do not match very well to any of the vehicle or pedestrian clusters learnt. The aim of this paper is to show that this can be done with multiple cameras.

In order to extend clustering approaches to a network of CCTV cameras, we must link tracks obtained in different fields of view. The correspondence (or tagging) task aims at matching the appearance of the same object in different cameras [START_REF] Gheissari | Person reidentification using spatiotemporal appearance[END_REF][START_REF] Javed | Tracking across multiple cameras with disjoint views[END_REF][START_REF] Javed | Appearance modeling for tracking in multiple non-overlapping cameras[END_REF]. This is not a trivial task as often the color histograms of the same object in different cameras are quite different. Also, different objects often have similar appearance across different camera views. The main reasons for this are due to different lighting conditions in each camera, different orientation of the object to each camera and different characteristics of the cameras themselves. As such, the correspondence problem is still very much an open ended problem and is yet to be solved well. The paper in [START_REF] Wang | Correspondence-free multi-camera activity analysis and scene modeling[END_REF] recognises the challenges of the correspondence problem and bypasses it by imposing less constraints on the topology of the camera network. The authors build a graph where each node in the graph represents a trajectory. The start and end time of trajectories are known. Then, if two trajectories are in different cameras but their temporal extents are close, within a set positive temporal threshold, it is likely that the two trajectories belong to the same object.

The input into the clustering algorithms that we will be discussing here is usually in the form of observations on trajectories, i.e., tracking, whether it be for a single field of view or multiple fields of view. The paper in [START_REF] Song | Stochastic adaptive tracking in a camera network[END_REF] looks at feature matching for tracking applications in a camera network. It looks at three main issues; treating feature similarities as random variables when feature similarities are uncertain, looking at finding features over space and time, i.e., long term interdependence of features and showing how the first two issues can be addressed by treating the tracking problem in a camera network as an optimisation problem in a stochastic adaptive framework. This is achieved by learning the probability distributions between observations at two camera nodes by treating target features (normalised color and gait width vector [START_REF] Kale | Towards a view invariant gait recognition system[END_REF][START_REF] Kale | Identification of humans using gait[END_REF]) as random variables and using dynamic programming [START_REF] Sedgewick | Algorithms[END_REF] for finding optimal paths in graphs with stochastic weights. The drift problem is dealt with by looking at long-term interdependences between features over space and time for correction and by proposing a path smoothness function to recalculate the weights for the dynamic programming procedure.

In this paper, we use temporal links [START_REF] Wang | Correspondence-free multi-camera activity analysis and scene modeling[END_REF][START_REF] Makris | Bridging the gaps between cameras[END_REF] in conjunction with trajectory color signatures to link tracks across cameras and detect abnormal global trajectories within a network of multiple cameras. We have a statistical representation of where normal tracks occur globally in the camera network from a training phase. More specifically, using our training data set of local trajectories within each camera, we first learn the entry and exit zones for each camera via Gaussian Mixture Models [START_REF] Makris | Automatic learning of an activitybased semantic scene model[END_REF] and then we learn the camera tempo-topographical model to obtain temporal links between exit and entry zones in different cameras [START_REF] Makris | Bridging the gaps between cameras[END_REF]. We use these temporal links in conjunction with a tagger in order to link tracks of the same object across cameras to obtain global trajectories.

Using the global trajectories obtained from the training phase, we obtain similarities between global tracks using spatial and velocity information and we use spectral clustering methods in order to group global tracks into clusters. For each global cluster, we learn its semantic scene model. This model can be viewed as a likelihood of a point on a trajectory at a certain global position. Then, for each global track in the testing phase, we can look at the ensemble of points which make up its trajectory, find their corresponding likelihoods from the semantic model for each cluster and determine if the trajectory is likely to fit well with at least one global cluster. If this is the case, then that global trajectory is classified as being normal but if the global track does not match well with any of the global clusters, then it is classified as an abnormal track.

Multicamera Behaviour Profiling

We start by obtaining local trajectories for training within each camera. In our experiments we used two cameras, which we call 'office' and 'foyer' respectively, as seen in Figure 2. A schematic of how the two cameras are positioned with respect to each other can be seen in Figure 1.

After the background/foreground segmentation, we use a particle filter in our experiments [START_REF] Arulampalam | A tutorial on particle filters for on-line non-linear/non-gaussian bayesian tracking[END_REF]-a multiple hypothesis tracker-to track blobs, and we group blobs into targets by employing similar techniques to those in [START_REF] Mckenna | Tracking groups of people[END_REF][START_REF] Fuentes | People tracking in surveillance applications[END_REF]. The number at one extremity of a local trajectory in Figure 2 is a unique identifier for each track. At the same time, it also marks the beginning of a track and provides information regrading the track direction.

Using the local tracks, we are able to obtain entry and exit zones into each scene using Gaussian Mixture Models (GMMs) [START_REF] Makris | Automatic learning of an activitybased semantic scene model[END_REF], depicted in Figure 3 (an 's' is an entry or start zone and an 'e' is an exit zone). It can be seen from Figure 1 and Figure 3 that the entry and exit zones of interest are 1 → 2 for tracks going from the office to the foyer and 2 → 1 for tracks going from the foyer to the office.

Using tempo-topographical cross-correlation functions described in [START_REF] Makris | Bridging the gaps between cameras[END_REF], we can learn the most popular transition time across the blind spot for each entry/exit zone combination of interest by constructing a cross-correlation function for each combination, R o→f 1→2 and R f→o 2→1 where o → f and f → o means 'office to foyer' and 'foyer to office' respectively. The cross-correlation plots can be seen in Figure 4.

From each cross-correlation function, we extract two pieces of information, µ and σ, where µ is the time of the peak in the cross-correlation function and σ is the standard deviation of the times in the discrete buffer. We use this information in conjunction with color signatures in order to associate tracks and obtain global trajectories across our two cameras. We now briefly discuss how we do this.

Track Association

Our aim is to associate tracks so that we can cluster them globally. In this section, we discuss the feature similarity scores that we use.

Since we know the start times, t s , and end times, t e , of local trajectories in each camera and we have clustered the start and end points of tracks into entry and exit zones, for each track ending in an exit zone of interest, we look at all tracks starting in an entry zone of interest and assign for each combination a temporal transitional score using

S t (e p , s q ) =        exp - (|t s p -t e q |-µ) 2 +λ σ if t s q -t e p ≤ 0 exp - (|t s p -t e q |-µ) 2 σ if t s q -t e p > 0 , (1)
where p = 1, . . . , P is the number of points ending in an exit zone cluster, q = 1, . . . , Q is the number of points starting in an entry zone cluster and µ and σ were defined above.

If the absolute time difference between an ending track and starting track pairing, t s p -t e q , is close to the peak time, µ, in the corresponding cross-correlation function, the temporal transitional score will be high (1 at the most) and as this difference starts to become larger, the temporal transitional score will become smaller following an exponential rule. If however a track in the entry zone of interest starts before a track ends in the exit zone of interest for a particular pairing of tracks, it is very unlikely that the two tracks should be associated so a positive penalty term, λ, is introduced in the equation.

We also assign for each combination a color score using the BHATTACHARYYA distance [START_REF] Bhattacharyya | On a measure of divergence between two statistical populations defined by probability distributions[END_REF] S c (e p , s q ) = N i=1

H e p (i)H s q (i), (2)
where H e p is the color histogram of a target disappearing in an exit zone of interest and H s q is the color histogram of a target appearing in an entry zone of interest. N is the number of bins used in the histograms. In our experiments, we set N = 256. Geometrically, the BHATTACHARYYA distance is a dot product between two vectors. If the two vectors (in this setting, color histograms) are normalised so that N i=1 H e p (i) = 1, and

N i=1 H s q (i) = 1, (3)
the BHATTACHARYYA distance will have a minimum value of 0 if the two vectors are orthogonal and a maximum value of 1 if the two vectors are coincident. An overall score is computed for each combination by performing a multiplication of the temporal transitional score and the BHATTACHARYYA distance, i.e.

S o (e p , s q ) = S t (e p , s q )S c (e p , s q ).

Then, only if

S o (e p , s q ) > ρ, (5)
where ρ is a positive threshold between 0 and 1, the ending track in the exit zone of interest is assigned to the starting track in the entry zone of interest with the highest overall score, i.e., max ({S o (e p , s q )}) ∀S o (e p , s q ) > ρ where p = 1, . . . , P and q = 1, . . . , Q. If the constraint in ( 5) is not met for a given track ending in an exit zone of interest and all tracks starting in the entry zone of interest, no track association will occur.

Clustering Global Trajectories

We employ the similarity measures in [START_REF] Wang | Learning semantic scene models by trajectory analysis[END_REF] with automatic selection of scaling constants and the rationale in [START_REF] Cheng | Mathching of objects moving across disjoint cameras[END_REF] to construct a similarity matrix in order to cluster global tracks. We construct a symmetric M × M affinity matrix, A, which is made up of symmetric similarity scores, A I,J , between global tracks I and J where I, J = 1, . . . , M and M is the number of global tracks in the training data set. Using the rationale in [START_REF] Cheng | Mathching of objects moving across disjoint cameras[END_REF], each entry in A is evaluated using position and velocity information of global tracks, so that A(I, J) = a min (I, J) if a min (I, J) < τ 1 -amax(I,J)-amin(I,J) amax(I,J)+amin(I,J)

if a min (I, J) ≥ τ , (6)
where a min (I, J) = min(a I→J , a J→I ), (7) a max (I, J) = max(a I→J , a J→I ).

(

) 8
The parameter τ is a chosen similarity threshold and a •→• is a directed similarity score between global tracks I and J or J and I. Looking at [START_REF] Fuentes | People tracking in surveillance applications[END_REF], if the directed similarities a I→J and a J→I are asymmetric or both low, we bound them by their lowest value, a min (I, J). If however, a min (I, J) is above or equal to τ , the symmetric similarity is defined as per [START_REF] Fuentes | People tracking in surveillance applications[END_REF] where we have some confidence that the two trajectories I and J are similar spatially and velocity wise, as we shall now see. The directed similarity score a I→J is calculated as follows

a I→J = i k ∈I γ(i k , j ψ(k) )α(i k , j ψ(k) ) i k ∈I γ(i k , j ψ(k) ) , (9)
and similarly for a J→I , where i k are observations on trajectory I and j ψ(k) is the closest observation on trajectory J to observation i k on trajectory I, i.e.,

ψ(k) = arg min m∈J i k -j m . (10)
The function γ(i k , j ψ(k) ) is a comparison confidence between the two observations i k and j ψ(k) and α(i k , j ψ(k) ) is a feature similarity between the two observations i k and j ψ(k) such that

γ(i k , j ψ(k) ) = exp - i k -j ψ(k) σ 1 , (11)
α(i k , j ψ(k) ) = exp - δ(v i k , v j ψ(k) ) σ 2 , (12)
where

δ(v i k , v j ψ(k) ) = 1 - v i k • v j ψ(k) v i k v j ψ(k) , (13)
and v i k and v j ψ(k) are the velocities at observations i k and j ψ(k) . For each observation i k on trajectory I, we find its spatially nearest observation j ψ(k) on J, compute the comparison confidence γ(i k , j ψ(k) ) and feature similarity α(i k , j ψ(k) ). Along trajectory I, feature similarities of observations are averaged and weighted by the comparison confidences to get the directed similarity score a I→J . The similarity of observations close in space has larger weight for computing global trajectory similarity. The comparison confidence indicates how far apart I is from J.

We found that the choice of scaling constants σ 1 and σ 2 in ( 11) and ( 12) is crucial for the clustering stage. As discussed in [START_REF] Zelnik-Manor | Self-tuning spectral clustering[END_REF], the selection of scaling constant is commonly done manually, as is the case in [START_REF] Wang | Learning semantic scene models by trajectory analysis[END_REF]. However, we found that manual selection of such quantities can have a very big impact on the performance of the clustering algorithm. We therefore calculate our scaling constants automatically as a function of spatial and velocity information from observations on global trajectories as follows

σ 1 = max (mean m∈J ( i k -j m ) , ζ) , ( 14
)
where ζ is a small predefined constant. We take the maximum of the two arguments in [START_REF] Makris | Bridging the gaps between cameras[END_REF] in case an observation lies on both global tracks. And

σ 2 = std m∈J   1 - v i k • v j m v i k v j m   . ( 15
)
Our aim is to group global trajectories which have similar spatial and velocity characteristics. Once we calculate the affinity matrix A, we apply a standard normalised cut clustering algorithm [START_REF] Yu | Segmentation given partial grouping constraints[END_REF][START_REF] Yu | Grouping with bias[END_REF].

Learning Path Regions for Global Clusters

Once we have trajectories grouped into clusters, we detect path regions in the scene by thresholding a density distribution proposed in [START_REF] Wang | Learning semantic scene models by trajectory analysis[END_REF]. The difference here is that we have clustered global trajectories and the density distribution, which can be viewed as a likelihood function, is for global path regions. For each global cluster, Ω, the density at each position, p, in the global scene is

L Ω (p) = i k ∈I I∈Ω exp - p -i k 2 σ 3 , (16)
and here, I and i k represent global tracks in global clusters and the observations on global clusters respectively. The scaling constant σ 3 is automatically set to the standard deviation of all the standard deviations of global tracks in the cluster, i.e., for each global cluster

σ 3 = std I∈Ω (std k∈I ( i k -c )) , ( 17
)
where c is the centroid of each global track in the global cluster

c = 1 K K k=1 i k . ( 18
)
The density for each cluster is shown in Figure 7 (7(a),7(c),7(e) and 7(g)). The path regions are obtained by thresholding the density distributions using max L Ω (p)/10 as in [START_REF] Wang | Learning semantic scene models by trajectory analysis[END_REF] (see Figures 7(b),7(d),7(f) and 7(h)).

Abnormality Detection

Abnormal trajectories are detected as outlier tracks. If a global trajectory deviates from all global path regions in all global clusters, there will be observations on that trajectory with low corresponding density values from the cluster distributions. If a certain percentage, φ, of the length of a global track deviates from the path regions, the global track is classified as an abnormal trajectory.

Experiments

A plot of the local trajectories used for training within each camera is shown in Figure 2. Using the local trajectories, we learn entry and exit zones into each scene using Gaussian mixture models [START_REF] Makris | Automatic learning of an activitybased semantic scene model[END_REF] and the expectation-maximisation algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF]. The clusters can be seen in Figure 3.

The entry and exit zones of interest are 1 → 2 for tracks going from the office to the foyer and 2 → 1 for tracks going from the foyer to the office. We use these entry and exit zones to construct tempo-topographical cross-correlation functions described in [START_REF] Makris | Bridging the gaps between cameras[END_REF] and learn the most popular transition time across the blind spot for each entry/exit zone combination of interest, R o→f 1→2 and R f→o 2→1 where o → f and f → o means 'office to foyer' and 'foyer to office' respectively. In our experiments, we used a time-search window, T , of 30 seconds and the cross-correlation functions are plotted in Figure 4. It can be seen from the plots that there are peaks at 6 seconds in R o→f 1→2 and 8 seconds in R f→o 2→1 respectively, which are the transition times from the respective exit zone to the entry zone. Of course, the more tracks that we have during training, the more robust the cross-correlation function will be. Furthermore, cross- 76 global tracks were linked correctly. The only global track that was not associated in the training set belonged to a person that went from the foyer to the office through EDBA via the male toilets in the blind gap between the cameras. That person spent more time in the blind gap than the learnt temporal connection between the respective exit zone in the foyer camera and the entry zone in the office camera (which was 8 seconds), the overall score (see Equation 5) was very low and hence, no track association occurred.

A plot of all the global associated tracks going from the office to the foyer and from the foyer to the office is in struct the affinity matrix, A, using (6) as described in Section 2.2 and use that as input to a normalised cut clustering algorithm [START_REF] Yu | Segmentation given partial grouping constraints[END_REF][START_REF] Yu | Grouping with bias[END_REF]. The results are shown in Figure 6.

As expected, we obtain four global clusters of trajectories going from the office to the foyer through CBDF (global cluster Ω 2 ) and ABDE (global cluster Ω 1 ), and trajectories going from the foyer to the office through F DBC (global cluster Ω 4 ) and EDBA (global cluster Ω 3 ).

The density for each cluster can be seen in 7(a),7(c),7(e) and 7(g) and the path regions are obtained by thresholding the density distributions using max L Ω (p)/10 as in [START_REF] Wang | Learning semantic scene models by trajectory analysis[END_REF] (see Figures 7(b),7(d),7(f) and 7(h)). Given the path regions, it is possible to use some logical operations in order to detect global anomalous trajectories in the scene. This can be done using many heuristics depending on the application at hand. It might be sufficient to observe individual positions on tracks and see if their probability is low. However, because the thresholding might not be perfect at all sections of a global path region, as can be seen in global scene is comprised of a larger set of local scenes, i.e., global path regions are longer and occur across more cameras.

If a global trajectory deviates from all global path regions in all global clusters, there will be observations on that trajectory with low corresponding density values from the cluster distributions. In our experiments, we set φ defined in Section 2.4 to 20%, i.e., if more than 20% of the total length of a global track deviates from the path regions, that track is classified as an abnormal trajectory. We tested 14 global trajectories, 2 were abnormal and 12 were normal. The global test trajectories are shown in Figure 8(a) and 8(b).

The trajectory going through F DBA and the one going through EDBC (refer to Figure 1) should be detected as abnormal trajectories as they do not fit well with any of the training path regions. The rest of the trajectories should be detected as normal as they conform well with a learnt path region. The results can be seen in Figure 8(c) and 8(d). As expected, the normal global tracks were labeled in green and the abnormal global tracks were labeled in red.

As another experiment, we let φ vary from 0%-100% in increments of 2.5% between 0%-20% and in increments of 10% between 20%-100% and plotted the ROC curve to get a relationship between the true-positive and false-positive prediction rate. The results are plotted in Figure 9. The ROC curve shows that the abnormal global tracks can be easily distinguished from the normal ones resulting in excellent detection performance.

Conclusions

We have presented a novel framework for addressing the problem of correlating behaviours across different camera views and detecting subtle abnormalities that are only meaningful and recognisable when observed over space and time across multiple camera views. The effectiveness of the framework has been demonstrated with experiments on menting with multi-modal temporal cross-correlation functions and tagging routines for global trajectory association across many disjoint cameras. Also, further experiments are required with more complicated activity across the camera network.

Figure 1 .

 1 Figure 1. Schematic of the office and foyer cameras.

  (a) Office tracks. (b) Foyer tracks.

Figure 2 .

 2 Figure 2. Local tracks in the office and foyer cameras.

  (a) Office entry zones. (b) Office exit zones. (c) Foyer entry zones. (d) Foyer exit zones.

Figure 3 .

 3 Figure 3. Entry and exit zones in the scenes. 's' stands for start and 'e' stands for end.

Fig- ure 5 .

 5 The numbers in brackets at one extremity of a global track in Figure5is a unique identifier for each trajectory. The two numbers in the brackets (I -J) specify which local tracks were associated together to obtain the global track. Our (I -J) notation also marks the beginning of a global track and that together with the order of the local scenes in the global scene provides information regarding the track direction.Having obtained the global tracks, we are able to con-(a) R o→f 1→2 . (b) R f→o 2→1 .

Figure 4 .

 4 Figure 4. Cross-correlations functions for entry/exit zones of interest.

  Figure 7(b), a more robust approach to detect anomalous tracks is to check if all observations on global trajectories are sufficiently close to path regions of global clusters. Statistically, this approach becomes increasingly more robust when the (a) Global tracks from the office to the foyer camera. (b) Global tracks from the foyer to the office camera.

Figure 5 .

 5 Figure 5. Global tracks across the cameras.

  (a) Clustered global tracks from the office to the foyer camera. (b) Clustered global tracks from the foyer to the office camera.

Figure 6 .

 6 Figure 6. Clustered global tracks across the cameras.

  (a) Density distribution for Ω 1 . (b) Path region for Ω 1 . (c) Density distribution for Ω 2 . (d) Path region for Ω 2 . (e) Density distribution for Ω 3 . (f) Path region for Ω 3 . (g) Density distribution for Ω 4 . (h) Path region for Ω 4 .

Figure 7 .

 7 Figure 7. Global density distributions and path regions for clusters.

Figure 9 .Figure 8 .

 98 Figure 9. Abnormality detection performance measured using a ROC curve.


Lebourgeois François
email: francois.lebourgeois@agroparistech.fr

THIS IS MY PERSONAL LAYOUT OF THE ARTICLE Long-term monitoring of activities of badgers (Meles meles L.) in a broadleaved forest in France

Keywords: Meles meles L., daily activities, camera-trapping, long-term monitoring, northeastern French broadleaved forest

European badgers' behaviors have been studied over a six years period (2013-2018) using camera traps settled in a main sett (400 m²; 17 holes) in an oak forest, northeastern France. I analyzed group's size, grooming, digging, bedding collecting, mating, and the emergence and return times. The burrow was inhabited by 2.8 (± 1.2) badgers. I observed between 2 to 4 new cubs each year in five out of six years, with a first emergence in mid-April. Grooming was frequent after emergence at dusk and lasted around 10 min (more frequent grooming in April after the births). Bedding collecting was a major activity in February and March during dry nights (mean duration of around 20 min) for years with births. Digging occurred only in winter 2015 and spring 2016, mainly at dusk (mean: 23 min, 2 to 90 min). Mating occurred mainly in January and February with short (< 10 min) and long (> 60 min) duration of copulation.

Emergences occurred mainly between 19 and 21h and returns between 5 and 7h. For spring and summer, badgers emerged between 30 and 60 min before sunset but 2h or more after sunset in autumn and winter. Returns occurred before sunrise in the middle of the night (1h in spring and summer and 2h or more in autumn and winter). In autumn and winter, warm days hastened emergences time (i.e., maximum temperature higher than 10°C) and frost nights hastened returns (i.e., minimum temperature below 0°C).

INTRODUCTION

The European badger (Meles meles L.) is a medium-sized carnivore species widely distributed throughout Europe (Frantz et al. 2014). Badger is a communally dwelling mammal forming well-organized family groups that live in a burrow system [START_REF] Neal | The natural history of Bagders[END_REF][START_REF] Thornton | Density and distribution of Badgers in south-west England-a predictive model[END_REF]Lebourgeois 2020a, b). The cohesion of the group calls upon many social behaviors: vocalizations [START_REF] Wong | Vocal repertoire in the European badger (Meles meles): Structure, context, and function[END_REF], grooming and scent marking [START_REF] Buesching | The social function of allo-marking in the European badger (Meles meles)[END_REF], digging, and bedding collecting [START_REF] Roper | Badger Meles meles setts architecture, internal environment and function[END_REF][START_REF] Roper | Internal structure and contents of three badger (Meles meles) setts[END_REF]. A thorough grooming ensures the elimination of the parasites and helps to share scent among the different group members. The digging ensures maintenance of the burrow and the creation of new tunnels, sleeping, and breeding chambers to welcome new animals. Digging also maintains optimal underground conditions [START_REF] Roper | Ventilation of badger Meles meles setts[END_REF] and avoid accumulation of ectoparasites (Butler and Roper 1996). The function of bedding collecting, which is a specific badger trait, may be related to preventing heat loss particularly during winter or to maintaining the cleanliness of the sleeping and breeding chambers [START_REF] Neal | The natural history of Bagders[END_REF].

I used a non-invasive monitoring method of camera trapping [START_REF] Balestrieri | Spatial organisation of European badger (Meles meles) in northern Italy as assessed by camera-trapping[END_REF]Findlay et al. 2017) to follow continuously the daily activities of a family group over six years (from April 2013 to June 2018) including grooming, digging, bedding collecting, mating, and the emergence and return times. Although these behaviors have been known for a long time, their seasonal variability remains poorly documented [START_REF] Neal | The natural history of Bagders[END_REF], as well as the importance of weather conditions in particular on bedding collecting and daily activity rhythms.

Sunrise and sunset times influence the daily activity patterns (Cresswell and Harris 1988;Kowalczyk et al. 2003;[START_REF] Neal | The natural history of Bagders[END_REF]) but many other factors such as temperature, rainfall, cloud cover, air humidity or wind speed are also cited as significant determining factors of badger's behaviors [START_REF] Martin | Weather influences trapping success for tuberculosis management in European badgers (Meles meles)[END_REF][START_REF] Noonan | Climate and the Individual: Inter-Annual Variation in the Autumnal Activity of the European Badger (Meles meles)[END_REF]). In addition to sunset and sunrise times, it was hypothesized that the badger's behavior would significantly change during warm or cold days or under dry or wet conditions and that these climatic interactions would differ throughout the seasons. To highlight the role of weather conditions on activity patterns and possible interactions between climatic parameters, I used daily meteorological data in my analyses.

MATERIALS AND METHODS

The monitored main sett (i.e., badger burrow) is located in a typical oaks (Quercus petraea and robur), beech (Fagus sylvatica) and hornbeam (Carpinus betulus) lowland forest of the « Plaine Lorraine » in the northeastern France (area of the forest ~ 1.5 km²; 48.69N -06.27E; alt. 300 m). The sett is located in the center of the forest (that included one other main sett and six outliers) [START_REF] Thornton | Density and distribution of Badgers in south-west England-a predictive model[END_REF] far from human activities and potential disturbance. The nearest edge is around 300 meters away and the furthest is around 1300 m. From 2013 to 2018, the area of the burrow increased from 150 m² with 8 entrances to 400 m² with 17 holes. The holes averaged 50 ± 20 cm wide and 40 ±15 cm high. These characteristics are typical of the main forested burrows observed in the "Plaine Lorraine" (Lebourgeois 2020a). The site has a mean annual precipitation and temperature of 774 mm and 10.5°C, respectively (Météo-France-Nancy;alt. 212 m;48.41N -06.13E;1981-2010). The coldest month is January (mean temperature of 2°C) and the warmest is July (mean temperature of 19°C). In winter, days (more rarely many weeks) can be cold (ranging from -5 to -10°C) but snow is very rare and melts quickly. Precipitation are regularly distributed over all months (50 to 70 mm per month).

To follow the different activities, three

RESULTS AND DISCUSSION

From 2013 to 2018, the burrow was regularly inhabited by 2.8 (± 1.2) badgers (maximum: 9 badgers) and cubs have been observed five years out of six (2 to 4 cubs per year). The first emergence of the cubs was usually observed in mid-April (between the first and 20 April).

Grooming has been observed in 39% of the cases and occurred mainly at dusk after the emergence of the burrow (Table 1 and Fig. 1). Grooming was more frequent at the end of winter and during spring and less important during summer and autumn (Table 2). Two peaks have been observed in April (after the emergence of the cubs) and in October. Grooming lasted usually less than 10 minutes (maximum duration: 45 min).

Bedding collecting took place almost exclusively at dusk and averaged 12 minutes (maximum duration around 150 min) (Tables 1 and2 During these two months, suitable conditions for bedding collecting correspond to dry nights. Indeed, 95% of the nights with bedding collecting had little to no rainfall (no rainfall -84% -or rainfall below 1 mm -11%). Conversely, the nights without bedding collecting were rainy in 63% of the cases (43% with rainfall above 1 mm; mean: 5.9 ± 4.3 mm). Dry bedding prevents heat loss into the burrow which increase the survival and the weight of the new-born cubs [START_REF] Tsunoda | Badger setts provide thermal refugia, buffering changeable surface weather conditions[END_REF]. Peak bedding in autumn also coincided with dry nights (90% of the nights had rainfall lower than 1 mm). It plays a major role to prepare the sleeping chambers for the cold season. Indeed, maintaining body temperature is more energetically expensive under cold and wet conditions. Thus, a "high" and stable internal sett temperature allows to minimize the body temperature loss (badgers' body were around 2 to 3 °C colder in winter than in the rest of the year) and thus to decrease the metabolic saving [START_REF] Bevanger | Body temperature changes in wild-living badgers Meles meles through the winter[END_REF].

The digging of new holes has been observed during the winter 2015-2016 with a peak in September and October 2015 followed by an important activity in April and May 2016 (Tables 1 and2). In October, badgers frequently dug but with a shorter duration which corresponded to the refreshing of the external zone of the preexisting holes. Badgers dug a total of during 58 hours during the six years of monitoring.

During the six years, we observed a total of 62 mating events, mostly at dawn with a marked peak in January and February highlighting the high sexual activity in winter (Tables 1 and2). This pattern is associated with the increasing testis weight, sperm production and testosterone levels observed in winter [START_REF] Ahnlund | Sexual maturity and breeding-season of the badger, Meles meles in Sweden[END_REF][START_REF] Buesching | Seasonal and inter-individual variation in testosterone levels in badgers Meles meles: evidence for the existence of two endocrinological phenotypes[END_REF][START_REF] Paget | Some observations on the sexual activities of badgers in yorkshire in the months december-april[END_REF] and with the post-parturient ovulation occurring after the birth of the cubs (Canivenc and Bonnin 1979;[START_REF] Wandeler | The reproductive cycle of female badgers (Meles meles) in Switzerland[END_REF]. Forty-seven percent of the mating lasted less than 20 min (n=15; mean = 7 min) and 63% more than 20 min (n=17; mean = 70 min). The longest mating lasted 150 min with two consecutive periods (90 and 60 min).

Badgers showed regular crepuscular and nocturnal activities depending on photoperiod and daily temperatures. For the long days (> 12 hours; spring and summer), badgers began their outside activity (19h30 to 21h30) between 30 and 60 min before sunset and 2 h or more after sunset for the short days (autumn and winter; 18h to 20h30) (Fig. 3). During the short days, a daily maximum temperature higher than 10°C hastened burrow emergence by around one hour compared to the emergence time observed during fresh days (0 to 5°C) (Fig. 3). Badgers were always back to the burrow one to two hours before sunrise (return times 4h30 to 5h30 during the long days of spring and summer, return times between 5h30 and 7h30 during the shorter days of autumn and winter) (Fig. 4).

As observed for the emergence, daily temperature also influenced the return. In winter, return time was hastened by around one hour during particularly frosty nights (minimum temperature between -5 and -10°C) compared to the return time observed for nights without frost (> 0°C) (Fig. 4). This clear contrast between warm and frost nights has been already reported in the literature (Do Linh San et al. 2010;Kowalczyk et al. 2003;[START_REF] Lindsay | The effects of disturbance on the emergence of eurasian badgers in winter[END_REF]. Badgers show a strong external thermal regulation with a better conservation of energy and fat reserves under warm conditions in winter [START_REF] Bevanger | Body temperature changes in wild-living badgers Meles meles through the winter[END_REF][START_REF] Mcclune | Winter Is Coming: Seasonal Variation in Resting Metabolic Rate of the European Badger (Meles meles)[END_REF][START_REF] Tanaka | Winter hibernation and body temperature fluctuation in the Japanese badger, Meles meles anakuma[END_REF]). This metabolism could explain why warm nights hastened the emergence time and frost nights quickened the return. Another explanation could be linked to the diet of badgers that fed mainly on earthworms in temperate forests [START_REF] Li | Biogeographical patterns of the diet of Palearctic badger: Is badger an earthworm specialist predator?[END_REF]. Most earthworm species are active within a temperature range of 0 to 20°C (Curry 2004;[START_REF] Nordström | Seasonall activity of lumbricids in southern Sweden[END_REF]) and soil temperature and humidity influence the presence of available earthworms on the surface in foraging areas. "Worm nights"

have been defined as warm nights (> 0°C) with the upper part of the soil slightly humid (Kruuk and Parish 1981). Thus, a linearly increase of earthworm's capture between 0 and 150 per night has been observed for an increase of external temperature from 0 to 8°C (Henry 1984;Lambert 1990). Under these favourable conditions that improve earthworm availability, badgers emerge usually earlier and devote less time to foraging (higher energetic efficiency) (Henry 1984;Kowalczyk et al. 2003).

Obviously, the foraging behaviour of animals depends not only of external conditions but also on the resource availability [START_REF] Shepherdson | Diet, food availability and foraging behaviour of badgers (Meles meles L.) in southern England[END_REF]). The forest under study is one of the most favourable contexts for badgers with high earthworm availability (i.e., broadleaved forests with rich and loose soils) (Lebourgeois 2020a).

As many other researchers, I am aware of the importance of badgers' life underground but due to difficult study conditions very few studies have been done on their subterranean behaviour [START_REF] Markham | Underground Localization in 3-D Using Magneto-Inductive Tracking[END_REF][START_REF] Noonan | A new Magneto-Inductive tracking technique to uncover subterranean activity: what do animals do underground?[END_REF]. Here, I present original results on some specific aboveground badger's traits and confirm the high interest of camera trapping to study wild animals. It appears also very important to perform a long term monitoring to highlight the seasonal variability of the different behaviors and to evaluate their duration. The study has been performed on a single burrow but the group size and the characteristics of the burrow are very representative of the main burrows observed in temperate broadleaved forests in the area (Lebourgeois 2020a, b).

Complementary studies on a larger territory (systematic cartography of the burrows in the main forest areas of the "Plaine Lorraine") and sample (seasonal monitoring of around 30 burrows) are in progress and should reinforce the results of this study. . For the year, the percentage corresponds to the ratio between the number of days of each activity and the total number of monitored days (216, 294, 290, 292, 298 and 148 from 2013 to 2018). The monthly mean duration for mating is not given as the duration was only quantified for 32 cases.

  camera traps (Bushnell Trophy Model 119477 and Cam HD Model 119877) were tied to tree trunks (1 to 1.5 m above the ground). The camera traps covered the entire area of the burrow and the overall holes (active and inactive) ensuring the reliability of the collected data. The cameras were set to record 60-s-long videoclips with a 3-s interval between two successive recordings throughout the 24-h period (day/night) (Findlay et al. 2017). The monitoring lasted 1538 days between the 1 April 2013 and the 23 June 2018 during all seasons (243 weeks and 29 648 videos). an accurate analysis at the individual level was not possible (except for mating insured by the dominant female of the group, easily recognizable by a white mark on the back). Thus, the behaviors presented must be considered at the family group level and not at the individual one. For the nightly variations of the different activities, three periods were defined: after emergence "dusk period" [18 to 24h[, during the night "night period" [0-4h] and before the return into the burrow "dawn period" ]4-9h]. The daily local sunrise and sunset times and the duration of the day and night were considered for the latitude 48.69 N. To highlight the effect of climate on the different activities, daily precipitation, temperature, wind speed, air humidity, sunshine duration, global solar radiation, and potential evapotranspiration were gathered from the station of Nancy (7 km away from the forest).

  , Fig.2). Bedding collecting was typically carried out by a single adult (rarely two in a same time). The bedding dragged into the burrow was made up exclusively of bundles of hornbeam leaves collected in the surrounding area of the burrow (between five to 20-30 meters; maximum around 50 meters). It was more important in February (mean duration 23 min) and March and in October and November (mean duration 18 min). Between mid-and the end of February, badgers collected bedding regularly between three to six days out of seven. This observation confirms the suggestion made by[START_REF] Neal | The natural history of Bagders[END_REF] that the peak in February is closely connected with preparations for the birth of the cubs which occurs during this period(Cresswell et al. 1992; Page et al. 1994).Indeed, for the five years with births, badgers devoted 23 days in average (17 to 34 days) to collect litterfall in February and March. In 2016, with no litter born, bedding collecting was observed only eight times.

  Butler JM,Roper TJ (1996) Ectoparasites and sett use in European badgers. Animal Behaviour 52: 621-629 Canivenc R, Bonnin M (1979) Delayed implantation is under environmental control in the badger (Meles meles L.). Nature 278: 849-850 Cresswell W, Harris S (1988) The effects of weather conditions on the movements and activity of badgers (Meles meles) in a suburban environment. Journal of Zoology 216: 187-194 Cresswell WJ, Harris S, Cheeseman CL, Mallinson PJ (1992) To breed or not to breed -an analysis of the social and density-dependent constraints on the fecundity of female badgers (Meles meles). Philosophical Transactions of the Royal Society of London Series B-Biological Sciences 338(1286): 393-407 Curry JP (2004) Factors affecting the abundance of earthworms in soils. In: Edwards CA (ed) Ecology, earthworm. Boca Raton, pp 91-113 Dixon DR (2003) A non-invasive technique for identifying individual badgers Meles meles. Mammal Review 33(1): 92-94 Do Linh San E, Ferrari N, Weber JM (2010) Circadian activity patterns and nocturnal resting sites of Eurasian badgers (Meles meles L.) in a rural area of western Switzerland. Revue Suisse De Zoologie 117du blaireau européen (Meles meles L.) dans une forêt du centre de la France. Mammalia 48(4): 489

Fig

  Fig. 1 Collective grooming (seven badgers) in May just after the emergence of the burrow.

Fig. 3

 3 Fig. 3 Seasonal variations of emergence time relatively to sunset curve and daily maximum temperature (in °C) (emergence time and sunset curve are expressed in local time; 48.69N -06.27E). N= 1092 observations (period April 2013 to June 2018). No camera trapping between mid-July to mid-August (Julian days: 200 to 225, excepted in 2017).

Fig. 4

 4 Fig. 4 Seasonal variations of return time relatively to sunrise curve and nightly minimum temperature (°C) (return time and sunrise curve are expressed in local time; 48.69N -06.27E). N= 1026 observations (period April 2013 to June 2018). No camera trapping between mid-July to mid-August (Julian days: 200 to 225, excepted in 2017).

Table 1

 1 Frequency of the different activities (in %) observed on the burrow from April 2013 to June 2018 per period. For each activity, the percentage of the total has been calculated from the total number of days (n=1538) and videos (n=29 648).

	). Synthèse des
	connaissances européennes. Partie 1: choix
	de l'habitat, structure et densité spatiale des
	terriers. Revue Forestière Française 72(1):
	11-32

Table 2

 2 Total number (N in days) and mean duration (D in minutes) of the different activities observed on the burrow from April 2013 to June 2018 presented by month and by year (n=1538 days)


Sofiane Yous
email: s.yous@ieee.org

Hamid Laga

Kunihiro Chihara
email: chihara@is.naist.jp

People Detection and Tracking with World-Z Map from a Single Stereo Camera

In this paper we propose a new people tracking system that uses a single stereo camera fixed at a high position and observing the scene at an oblique angle. We introduce the notion of world-Z map and show that 3D people detection and segmentation can be performed efficiently in this map and outperforms significantly the methods based on depth or plan-view. Detection and tracking play complementary roles, we derive a probabilistic framework for tracking based on motion. The system successfully deals with very complex situations without loosing track of people in highly cluttered scenes for long observation periods and achieves around 10f ps on a single PC.

Introduction

People tracking is a very active research field that aims at detecting people within a monitored space and keeping tracking them as long as they are inside. The importance of people tracking is related to the range of applications that can take benefits from it such as visual surveillance and people counting. It is needless to argue how visual surveillance is important nowadays. Cameras are more and more deployed in indoor and outdoor public spaces, such as shopping malls, retails and parking lots, for monitoring and helping the detection of abnormalities. They have been recently used not only for security purposes but also to draw some statistics useful for planning and management.

The purpose of people tracking is to provide a set of tracks that are in a one-to-one correspondence with people appearing in the monitored area [START_REF] Iocchi | Integrating plan-view tracking and color-based person models for multiple people tracking. Image Processing[END_REF]. The ideal system should satisfy the following requirements:

• Tracking in complex situations such as crowded scenes where occlusions are frequent. Previous work [START_REF] Iocchi | Integrating plan-view tracking and color-based person models for multiple people tracking. Image Processing[END_REF] identified three types of occlusions; (1) partial occlusions where only a part of the subject is visible to the sensor, (2) short-term occlusions where the subject is completely occluded for short periods of time, and (3) extended occlusions where the subject leaves the field of view for an extended period.

• Robustness against lighting changes and background complexity is a major concern. Robust systems should be able to operate in indoor and outdoor environments, and handle scenes with complex and dynamic backgrounds.

• Accuracy in recovering the trajectory of each individual with respect to the 3D space or floor plane.

• Fast processing in applications such as visual surveillance where the reaction time is critical. In these situations, the system should provide real-time information required for behavior analysis and abnormalities detection.

• Low deployment cost and easy setup and maintenance.

Ideally the system should have minimum free parameters to tune.

Although many solutions have been proposed in the literature, none of them is able to fulfill simultaneously all the requirements. In this paper we propose a new multi-people tracking system that uses a single stereo camera fixed at a high position and observing the scene at an oblique angle. We introduce the notion of world-Z map and show that people detection and segmentation can be performed efficiently outperforming significantly the existing methods in general and the plan-view-based methods [START_REF] Fleuret | Multicamera people tracking with a probabilistic occupancy map[END_REF] in particular (Section 2.1). The detection and segmentation process is described in Section 2.2. For tracking, we derive a probabilistic framework which allows the tracking of variable number of people in real-time (Section 3). We evaluated the system in very complex situations (Section 4) where the results show that it outperforms significantly the state of the art methods. Section 5 summarizes the main contributions of the paper and outlines some issues for future work.

Related work

There is a large amount of research on people tracking as it rises many challenging issues to image processing, com- Early works used a single monocular camera [START_REF] Leventon | Bayesian estimation of 3-d human motion[END_REF][START_REF] Sidenbladh | Stochastic tracking of 3d human figures using 2d image motion[END_REF]. Since a single monocular camera does not provide 3D information of scene, dealing with partial and short-term occlusions requires strong prior knowledge about the geometry, appearance and dynamics of the objects to track. This knowledge is used to build a Bayesian framework for detection and tracking [START_REF] Leventon | Bayesian estimation of 3-d human motion[END_REF][START_REF] Sidenbladh | Stochastic tracking of 3d human figures using 2d image motion[END_REF]. However, strong priors limit the flexibility of the system to handle unseen objects with high intra-class variability.

Recent advances in computing and imaging hardware had motivated the use of distributed monocular camera systems to gain access to the 3D information of the scene. Chai and Aggarwal [START_REF] Cai | Tracking human motion in structured environments using a distributed-camera system. Pattern Analysis and Machine Intelligence[END_REF] developed a distributed camera system for motion tracking in structured environments. Fleuret et al. [START_REF] Fleuret | Multicamera people tracking with a probabilistic occupancy map[END_REF] used two to four synchronized video streams taken at eye level from different angles to detect and track up to six people in the presence of occlusions. They use a probabilistic occupancy map which can be affected by: 1) the poor quality of background subtraction, 2) the presence on people in an area covered by only one camera and 3) the excessive proximity of several people. Their multi-person tracking is achieved by processing individual trajectories separately over patches of 100 frames.

Stereo cameras can be seen as a particular case of distributed monocular camera systems that produce dense depth maps of the scene. Augmenting the input space with depth maps provides better robustness to many algorithms including background substraction, and object detection and tracking. Stereo-based object detection and tracking has been previously used in stationary rig setup [START_REF] Beymer | Real-time tracking of multiple people using continuous detection[END_REF][START_REF] Darrell | Plan-view trajectory estimation with dense stereo background models[END_REF], and mobile platforms [START_REF] Agrawal | Real-time detection of independent motion using stereo[END_REF]. These works exploit the fact that stereo provides a stable basis for object segmentation, resolves to some extent the occlusions and provides useful 3D cues to the tracking stages, such as 3D positions and relative object sizes [START_REF] Beymer | Real-time tracking of multiple people using continuous detection[END_REF]. Harville and Li [START_REF] Harville | Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera[END_REF] and Darell et al. [START_REF] Darrell | Plan-view trajectory estimation with dense stereo background models[END_REF] developed a powerful tool to solve the problem of partial and short term occlusions. They use the depth maps to estimate a virtual overhead view, called the plan-view, and perform the detection and tracking in this plane based on some statistics, such as height and occupancy, recorded on this plane. For instance, Harville and Li [START_REF] Harville | Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera[END_REF] combine occupancy and height maps to eliminate the 3D noise and deal with occlusion.

Although plan-view-based methods handle many occlusions, it is a lossy representation in the sense that the rendered top view keeps only the top of objects present in the scene. This fact limits the processing that can be done on this plane. In this paper, we propose a new map, called world-Z map, that records the world-Z coordinates of the scene points into the image plane. In addition to the fact that this map has the advantage of being lossless, since it represents every point in the original depth map, we will show in Section 2.1 that it underlies very useful properties that enable high-performance segmentation and, hence, efficient and accurate detection.

Contributions

In this paper, we propose a new multi-person tracking system that deals the best with very complex situations. Specifically we make the following contributions:

• We introduce the concept of world-Z map and show that people detection and segmentation can be efficiently performed in this new plane compared to methods based on depth and plan-view. • The system we propose deals efficiently with partial and short term occlusions and computes accurate 3D trajectories of the individuals.

• Only the height and viewing angle with respect to the floor plane are required. We show that this is a very easy task and therefore, our method does not require a complex calibration step.

• The entire detection and tracking process performs in real-time. In our experiments the lowest frame-rate was 10fps on scenes of nine people.

The system operates in an online mode and, therefore, is well suited for real-time applications such as visual surveillance, people counting and crowd monitoring.

3D people detection

To address people detection challenge, depth video cameras are the most suitable. Such cameras provide an image of the scene where each pixel contains, in addition to the intensity, the distance to the subject with respect to the camera, called depth. There are many commercial cameras generating depth information at video rates [2, 3, 4, 5], and the technology is evolving rapidly. We expect that prices will continue dropping and depth video cameras will be available at reasonable costs in near future.

Using depth information, previous works compute the 3D data and map them onto an overhead virtual plane called plan-view [START_REF] Darrell | Plan-view trajectory estimation with dense stereo background models[END_REF][START_REF] Harville | Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera[END_REF]. Plan-view can record many useful statistics, such as occupancy [START_REF] Fleuret | Multicamera people tracking with a probabilistic occupancy map[END_REF][START_REF] Darrell | Plan-view trajectory estimation with dense stereo background models[END_REF] and people height [START_REF] Harville | Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera[END_REF]. Although interesting results have been achieved, the segmentation is not efficient when the scene contains adjacent objects touching each other, which occurs often in crowded scenes. Furthermore, the occupancy and height maps are lossy in the sense that not all 3D points are represented.

In this paper we propose a new map Φ on which the detection is efficient even in very crowded scenes. First we will describe the new map and then we will show how we perform people detection.

World-Z map

The world-Z map records on the image plane the world-Z coordinates of the 3D points of the scene. Let's consider that the camera is set at a height h from the floor and observing the floor at an oblique angle α. We assume also that the XY plane is parallel to the floor and the coordinates of the camera center are (0, 0, h), see Fig. 3. Given a point p with camera coordinates (x c , y c , z c ), its world coordinates (x w , y w , z w ) are given by:

x w = z c sin(α), y w = x c , z w = h -z c cos(α). (1)
The world-Z map records on the image plane, the z w coordinates of every foreground point. This representation is lossless since all 3D points computed from the initial depth map are represented. By setting the stereo camera at a high position looking towards the scene at an oblique angle with the floor, the points belonging to the closer objects with respect to the camera have larger attributes on the world-Z map, creating an implicit boundary between objects. This property is very important since it enables easier segmentation. When using the depth map, the boundaries between objects can be detected using bidirectional threshold which is not easy to set. When using the world-Z map, however, no threshold is needed as will be explained in section 2.2.2.

People detection

In this section, we formulate the people detection problem as a world-Z map segmentation. Prior to building the world-Z map, we apply a depth-based background subtraction followed by a preprocessing step. The entire process is illustrated in Fig. 4.

Preprocessing

The preprocessing step consists of eliminating the noise inherent in stereo-based depth estimation. These errors can create false connections between occluding and partially occluded objects and affect the segmentation performance. The main noise that affects segmentation is located at the object boundaries. It is mainly due to the windowing used in stereo processing. Harville and Li [START_REF] Harville | Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera[END_REF] eliminate this noise using the occupancy map. However, this method fails when the noise belongs to occupied bins.

The noisy points are characterized by a large angle of incidence with respect to the camera, see Fig. 5(a). We use this property to setup a thresholding criteria to filter this noise. We build an incidence map in the image plane and record at each pixel the angle of incidence of a point in the point set. Next, we filter this map to keep only the points having angles of incidence smaller than a predefined threshold. This map will be used as a mask for the world-Z map.

The cosine of the angle of incidence at a given point p is the dot product between the unit normal vector -→ n p at p and the unit vector -→ v p connecting the camera center c to p. The incidence map entry I(p) corresponding to the point p is given by I(p) = arccos( -→ v p • -→ n p ), where -→ n p is computed using two other valid neighbors p 1 and p 2 not aligned with p, and is given by the normalized cross product between -→ pp 1 and --→ p 1 p 2 . Fig. 5(b) shows the effect of the preprocessing step on eliminating the noise and preparing the data to the next step.

Using the world-Z map for people detection

In this section we make use of the properties of the world-Z map to design a powerful algorithm for object segmentation that deals the best with partial and short-term occlusions including situations where objects are very close or touching each other. Particularly:

1. The world-z map is a lossless representation as compared to the plan-view.

2. The points of an object (one connected component) are always ordered inversely to the Y axis of the camera.

3. Points belonging to closer objects with respect to the camera have higher values on this map. This means that if the objects in the scene have roughly similar size, as it is the case of people, the most top point in the world-Z map belongs to the farthest object in the scene with respect to the camera.

To segment the back object from the front ones (all other that are closer to the camera), we start from the top-left point p(x, y) of the World-Z map and scan the image topdown and left to right to find neighbor points p (x , y ); p and p belong to the same segment if and only if Φ(x, , y )+ ≤ Φ(x, y), where is a user defined threshold. Based on this, people segmentation can be summarized as follows:

1. Scan the world-Z map from top to down and from left to right. Exit when the last point is reached.

2. Recursively find, in the same scanning direction, all neighbors that have lower values in the world-Z map.

3. Remove the segmented region from the world-Z map and go to step 1.

For comparison, recall that the condition for the segmentation of the depth map D is: D(x´, y´) -θ l ≤ D(x, y) ≤ D(x´, y´) + θ u , which requires the setting of two thresholds θ l and θ u . Fig. 4 demonstrates how well the segmentation method deals with complex occlusions. When two persons are at the same distance from the camera and touching each other, the lower part of one object will be detected as belonging to the object segmented first. For tasks, such as people counting, this is sufficient since both objects have been detected. Such issue can be also easily solved by limiting the search space during the segmentation process. The method also can detect some false positives such as the case where an individual raises his hand. This will be resolved by the persistency test in the tracking step.

People tracking

Our goal is to track unknown number of people using the world-Z map as our observations. In this section we formulate the problem and discuss the implementation details.

Problem formulation

First, we assume that the number of people to track can vary with time. This allows the tracking of new objects entering the monitored area as well as handling situations where the track of some objects is lost. This may occur if an object exits the scene, becomes occluded by another object, or missed by the detection algorithm.

A multi-person tracking problem consists of establishing, at each time step t, an association between the state vector X t-1 = x t-1 1 , . . . , x t-1 mt-1 at time t -1 with the current measurements Y t = y t 1 , . . . , y t nt . The component x t i is the state corresponding to the i -th tracked person at time t. It encodes some of its features that are relevant for tracking. This can include position, geometry, appearance and motion. Harville and Li [START_REF] Harville | Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera[END_REF] defines x t i as the estimated position, velocity and body configuration of the i -th person. In our implementation we use the estimated position.

The measurements Y t at time t are obtained using the object detector algorithm described in Section 2.2. To handle the cases of lost objects and tracking of new arrivals, we set n t = k t + m t-1 , where:

• k t , k t ≥ 0, is the number of blobs observed at time t.

• y kt+l , l = 1, . . . , m t-1 , sometimes denoted by the additional variables φ l for clarity, is the observation corresponding to the case where the l -th object x t-1 l detected at time t -1 is lost at time t.

The tracking problem can now be reduced to the problem of finding the configuration X t that maximizes the posterior probability P (X t |Y t ). The solution is given by:

Xt = arg max X t P (Y t |X t )P (X t |X t-1 ). (2)
Equation 2 assumes that the current state depends only on the previous one. The algorithm that finds the optimal solution can be summarized in three steps:

• Step 1 -Initialization: the system remains idle until the first objects are detected. These will be the initial set of the observation vector Y 0 and state vector X 0 . At this stage, the time t is set to zero.

• Step 2 -Prediction: we predict the new state Xt that maximizes P (X t |X t-1 ). It is defined as:

Xt = f (X t-1 ). ( 3
)
where f is the prediction function called also motion model. It can be chosen to be linear such as Kalman filter [START_REF] Beymer | Real-time tracking of multiple people using continuous detection[END_REF][START_REF] Darrell | Plan-view trajectory estimation with dense stereo background models[END_REF], non-linear such as the particle filter [START_REF] Deutscher | Articulated body motion capture by annealed particle filtering[END_REF], or by assuming constant velocities [START_REF] Harville | Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera[END_REF]. In our implementation we assume a linear motion model.

•

Step 3 -Data association: assuming that the random variables x t i , i = 1, . . . , m t are independent, we have:

X t = arg max Xt P (Y t | Xt ) = mt i=1 P (Y t |x t i ). (4)
We detail this step in Section 3.2.

This algorithm establishes a correspondence between the objects tracked in the previous frame and the newly detected blobs. States x t i assigned to any of the additional variables φ l will be considered as lost. Observations y t i left without any assignment will be considered as new objects entering the scene and will be added to the state vector X t .

In our implementation we assume that: (1) at time t < 0, the scene is empty; (2) the objects should enter the scene from its boundaries; and (3) each object entering or exiting the monitored area goes first through a transit area defined along the boundaries. In this area the objects are detected but not incorporated into the tracking loop.

The second assumption allows to solve the problem of background noise and avoid the detection of static objects in the foreground. Although it appears very restrictive, it is valid in most of video surveillance applications, where the objects to track usually come from outside the monitored area. They are only tracked once they enter the scene.

Data association

The correction step requires an exhaustive search in order to find the optimal association. We start by defining a matrix M t of size n t ×m t , where the entry M t ij = P (y t i |x t j ) is the probability of the observation y t i conditioned on the state x t j estimated at the prediction step. We assume that it is a a 2D Gaussian centered at x t j and of standard deviation σ in both directions. σ is also the radius of the search area around x t j , where observations y t i that fall inside the search circle have higher probability of being associated to the configuration x t j . It is closely related to the speed of movement of the objects in the scene as well as to the frame-rate of the tracking algorithm. We define σ to be twice the expected displacement per frame. In our implementation we assume that people are moving with the average speed of v p = 3.5mph, and a tracking at an average rate of 15fps.

A configuration x i should be associated to a lost state φ i , i = 1, . . . , m t-1 if either the object has exited the tracked area or the detection failed because of occlusion, or a jump due to a movement speed that exceeds significantly the radius of the search area. Therefore, the conditional probabilities of the variables φ l are given by:

P (φ i |x t i ) = 1 -2 σ 0 N (x, 0, σ 2 )dx + N (δ(x t i ), 0, σ 2 ) (5)
where N (x, 0, σ 2 ) is the normal distribution of zero mean and variance σ 2 evaluated at point x, and δ(x) is the minimum distance of x to the boundaries of the tracking area. The first term of Equation 5 accounts for the probability that x t i has moved from frame t -1 to frame t with a distance larger than the radius σ of the search area and can be computed in advance. The smaller σ is the larger is this probability. The second term corresponds to the probability that x t i has exited the tracking area. Equation 5 gives the exact quantification of the probability of loosing an object x t i . In our experiments, however, we noticed that there was not any performance improvement when using simplified models such as considering only the first term, as well as considering only the probability of falling on the edge of the search area, i.e., P (φ i |x t i ) = N (σ, 0, σ 2 ). We used this last model in all the experiments shown in this paper.

Once the conditional probabilities are defined, the matrix M t can be fully constructed and normalized such as the columns sum to one. However M t is a full dense matrix and an exhaustive search to find the optimal association is computationally prohibited especially when tracking a large number of objects. To overcome this problem, Fleuret et al. [START_REF] Fleuret | Multicamera people tracking with a probabilistic occupancy map[END_REF] rely on the appearance model to establish an ordering of the observations that matches the ordering of the state vector X t-1 . Hence, the association problem becomes convex and can be solved in a linear time. Such solution is not guaranteed to be the optimal one for the original problem. Furthermore, the appearance model introduces high probability of identity confusion and switch.

We introduce a simple but efficient heuristic that minimizes the probability of identity confusion and switch. The main idea is to make M t sparse while maintaining the high tracking precision. We do this by setting to zero the entries where the Euclidean distance between the predicted state vector x t i and the observation y t j exceeds the radius σ of the search area. This is well justified since the Gaussian function vanishes rapidly. This allows us to achieve the tracking in crowded scenes with severe overlaps between objects at more than 10fps.

In summary, the tracking procedure has the following properties:

• Unlike previous work that uses a sliding window (of size 100 in [START_REF] Fleuret | Multicamera people tracking with a probabilistic occupancy map[END_REF] for example), the tracking at frame t depends only on the previous frame.

• False detections do not occur inside the detection area because our algorithm does not allow new detections in this area, and assumes that any new person should enter through the transition area.

• False detections may occur only inside the transition area. For example, if a person enters by the transition area with his arm up, two persons will be detected. However, because of the persistency test, the false positives will be discarded earlier in the process.

the stereo camera where depth is computed. We see also a false detection in Figure 6(f) but quickly discarded since it is not persistent in time. Such false detections occur usually in the transition area but rarely in the tracking area. This can be seen in frame 0 of Figure 1(a); although a raised hand is detected as an object and since no new detections are allowed in the tracking space, the tracking discarded it.

The second sequence that starts at Fig. 6(i) shows a scene with many people in a limited space. Even in the presence of high occlusions, the system does not lose track of them. The third sequence demonstrates how the system behaves when an individual sits down. Since no restrictions have been made on the height, unlike in [START_REF] Harville | Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera[END_REF], the system keeps tracking the individual. This sequence also demonstrates the robustness of our system to lighting changes. Although part of the room light was turned off the tracking has not been affected. This robustness allows the system to work indoor and outdoor without parameter tuning.

To further evaluate the performance of our system, we computed the False-Positive (FP) and False-Negative (FN) rates in a sequence with one to nine people moving freely in the scene. The estimated FP rate was 2.48% while the FN rate was 3.95%. These rates are, respectively, 3.99% and 6.14% in [START_REF] Fleuret | Multicamera people tracking with a probabilistic occupancy map[END_REF] where some constraint about the number and movement of people where imposed, See [1]. In contrast, in our test people were moving freely and were not aware of our system or trying extreme situations. The video sequences has been taken in an open campus event, see https://www.cs.tcd.ie/Sofiane.Yous/Projects/PeopleTracking.

The system achieves about 10fps independently of the number of individuals in the scene. Therefore, it is very suitable for applications that require real-time operation such as visual surveillance.

Conclusion

We proposed a new system for 3D people tracking using a single stereo camera. We proposed a new method for people detection based on 3D segmentation of the point sets obtained by stereo processing. We introduced the world-Z map which records the world-Z coordinates of the points of the scene on the image plane. The properties of this map allow accurate object segmentation under high partial and short-term occlusions. The performance of people detection allows a simple probabilistic framework to achieve high performance tracking. Our algorithm achieves a high performance in terms of accuracy, robustness to complex lighting changes, occlusions, and processing time.

As future work, we will explore whether introducing other object properties improves further the tracking performance. We plan also to investigate the suitability of the world-Z map for detecting and tracking other objects such as cars in roads and parking lots.

Figure 1 .

 1 Figure 1. Stereo-based 3D people tracking: the system deals with complex situations.

Figure 2 .

 2 Figure 2. Overview of 3D people tracking.

Figure 3 .

 3 Figure 3. Building the world-Z map: (a) when the two objects are clearly separated both the planview and world-Z map allow efficient segmentation. (b) The world-Z map maintains clear boundaries between very close and partially occluded objects. (c) world-Z map of a scene with two persons.

WorldFigure 4 .

 4 Figure 4. 3D segmentation-based foreground objects detection

Figure 5 .

 5 Figure 5. Preprocessing: elimination of the noise inherent to stereo processing.

Acknowledgement

Hamid Laga is supported by the Japanese Ministry of Education, Culture, Sports, Science and Technology program Promotion of Environmental Improvement for Independence of Young Researchers under the Special Coordination Funds for Promoting Science and Technology.

Sofiane Yous would like to thank Prof. Masatsugu Kidode for his invaluable advice and unlimited support.

Results

The system was tested in real situations during an open campus event. It achieved high performance when dealing with very complex situations without loosing tracks of people for long observation periods, as illustrated in Fig. 6, see the submitted video.

The sequence in Fig. 1 and Fig. 6(a) to 6(h) is of extreme complexity and no previous system can deal with. It contains scenes with very high occlusions and split-merge situations. The person on the right of the Figure 6(d) to 6(h) is not detected because it is outside the overlapping area of