Monica Martinez-Mendoza

Student Member, IEEE Fabien Seyfert

Christoph Ernst

Member, IEEE Alejandro Alvarez-Melcon

Formal Expression of Sensitivity and Energy Relationship in the Context of the Coupling Matrix

come

I. INTRODUCTION

T HE relationship between the stored energy in the el- ements of an LC N-port network and the sensitivity of the S-Parameters with respect to each L or C element was first discussed in [START_REF] Kishi | Energy theory of sensitivity in lcr networks[END_REF]. In that paper, the sensitivity was defined as the variation of the S-Parameters with respect to independent variations in the L or C elements of the network under evaluation.

Later, coupling matrices representing lowpass filter networks became fashionable and started to be widely used. The coupling matrix concept was first introduced by Atia and Williams [START_REF] Atia | Narrow-bandpass waveguide filters[END_REF] in the early 70s, where it was applied to symmetric waveguide filters. The representation of microwave filters in matrix form is specially useful because it provides a very precise model of the network and simultaneously allows to operate on the coupling matrix in a very simple way. These operations may be useful to simplify the synthesis process, to simulate complex networks in a simpler way or to reconfigure the filter topology into another configuration more suitable for the specific practical implementation. Another advantage of the coupling matrix is that it represents some of the real physical properties of the circuit elements. Each coupling matrix element is related to a specific circuit element in the final prototype. The non-diagonal matrix elements represent the couplings between the resonators of the network, and their values provide information about the magnitude of these couplings. On the other hand, the diagonal coupling elements are related to the differences in the resonant frequencies of the resonators with respect to the center frequency of the filter.

Although the initial matrices were N xN matrices, where N is the order of the filter, N + 2 coupling matrix forms were introduced some years later [START_REF] Cameron | Advanced coupling matrix synthesis techniques for microwave filters[END_REF] . The N +2 coupling matrix form includes information about the input and the output port and is widely used in the microwave community, since it presents several advantages with respect to the N xN matrix form. Among these advantages are the possibility to implement fully canonical filters with a nonzero direct coupling term, and the possibility to include frequency invariant reactance elements (FIR) in the network, allowing to synthesize symmetric or asymmetric responses.

Although the N +2 coupling matrix can be a representation of an LC network in certain cases (synchronously tuned filters), it is important to highlight that there is no direct equivalence between the variation of an element in an LC network and the variation of an element in the N +2 coupling matrix. There are two reasons for this observation. First, a coupling is related to two resonators, thereby if a coupling term M pq is varied, two elements are being affected in the LC network represented by the coupling matrix. Second, the LC network presented in [START_REF] Kishi | Energy theory of sensitivity in lcr networks[END_REF] does not consider the possibility to include FIR elements. Thereby, a variation in a FIR element, which is expressed as a variation in a diagonal coupling term M pp , cannot be translated into the variation of an element of the LC network with no FIR elements.

In this context, although there is no direct equivalence between isolated variations in L or C elements and variations in the coupling matrix terms M pp and M pq , a close relationship between the stored energy in the elements of the network and the sensitivity of the reflection S-Parameter with respect to the diagonal elements of the N + 2 coupling matrix was already noticed in [START_REF] Martínez-Mendoza | Investigation of the relationship between sensitivity and stored energy[END_REF], [START_REF] Martínez-Mendoza | Prediction of fabrication yield from low-pass prototype in a butterworth direct-coupled cavity filter[END_REF], [START_REF] Martínez-Mendoza | On the relation between stored energy and fabrication tolerances in microwave filters[END_REF]. Nevertheless, precise formulas to express this relationship were not known and the obtained results were just based on observations at that time.

This paper presents precise formulas to express the formal relationship between the time average stored energy in the resonators of a lowpass filter network, and the sensitivity of the reflection S-Parameter with respect to the coupling matrix terms considering the normalized frequency axis. Sensitivity in the real passband domain can be obtained by using the standard lowpass to bandpass transformation. These relationships are found in the modern context of the N + 2 coupling matrix, and for both diagonal and general non-diagonal coupling elements of the N + 2 coupling matrix. It is important to remark that the obtained results are based on the N + 2 coupling matrix, thus they are only valid for transfer functions fulfilling the narrow band approximation. Different examples will be used to illustrate the implications and to highlight applications derived from the novel relationships.

II. SENSITIVITY OF THE S-PARAMETERS

In this section, following the same ideas as [START_REF] Amari | Sensitivity analysis of coupled resonator filters[END_REF], [START_REF] Amari | On the sensitivity of coupled resonator filters without some direct couplings[END_REF], we derive formulas for the sensitivities of the S-parameters with respect to the couplings. The lowpass circuit we consider is detailed in Figure 1: all non-diagonal couplings are here frequency invariant admittance inverters (FIR), the diagonal ones constant susceptances modelling the frequency shift of each resonator (i.e. the FIR elements mentioned previously), the input and output loads as well as all capacitors are normalized to one. At a given frequency ω, we define the sensitivity of the scattering parameter S i,j with respect to M l,k as:

∂S i,j (ω) ∂M l,k (1)
The nodal equations ruling the behaviour of circuit in Fig. 1 are expressed in terms of the voltage vector U as follows:

[G + jωI + jM ][U ] = A[U ] = [i] (2)
where:

• [G]: Admittance matrix with all elements zero except for

G 1,1 = 1 and G N +2,N +2 = 1. Size (N + 2) x (N + 2). • [I]: Identity matrix up to I 1,1 = 0 and I N +2,N +2 = 0. Size (N + 2) x (N + 2). • [M ]: Coupling matrix. Size (N + 2) x (N + 2). • [U ] is the voltage vector, and [i] = [i 1 , 0, . . . , i 2 ] t is the excitation vector. • A = [G + jωI + jM ] by definition.
From this we compute the S-parameters as:

S 1,1 = 2[A -1 ] 1,1 -1, S 2,2 = 2[A -1 ] n+2,n+2 -1 S 1,2 = 2[A -1 ] 1,n+2 . (3)
where a similar notation as in [START_REF] Amari | Sensitivity analysis of coupled resonator filters[END_REF], [START_REF] Amari | On the sensitivity of coupled resonator filters without some direct couplings[END_REF] has been used, and [A -1 ] i,j is the i-th element of the solution vector of the system shown in (2), when the excitation is placed in the j-th element of the excitation vector

[i]. If we set W i,j = [A -1 ] i,j
, some differential calculus yields for the sensitivities:

l = k, ∂S 1,1 ∂M l,l = -2W 2 1,l , ∂S 2,2 (ω) ∂M l,l = -2W 2 N +2,l (4)
l = k, ∂S 1,1 ∂M l,k = -4W 1,l W 1,k , ∂S 2,2 (ω) ∂M l,k = -4W N +2,l W N +2,k (5)
l = k, ∂S 1,2 ∂M l,l = -2W 1,l W N +2,l (6)
l = k, ∂S 1,2 ∂M l,k = -2(W 1,l W N +2,k + W 1,k W N +2,l ) (7)
Latter formulas are, up to sign changes, equivalent to those found in [START_REF] Amari | On the sensitivity of coupled resonator filters without some direct couplings[END_REF] where a dual circuit based on impedance inverters, and inductive series resonators was implicitly used.

III. RELATIONSHIP BETWEEN SENSITIVITIES AND REACTIVE ENERGY

In this section we come to the main goal of this paper, namely the derivation of relationships between sensitivities and the reactive energy stored in the resonators. We suppose that the network is excited by only one source at a time. When source (i 1 ) is active (i.e. i 2 = 0) we define the generic parameter

T k,1 = 1/2|U k | 2 , while when source (i 2 ) is active (i.e. i 1 = 0) we define in a similar manner T k,2 = 1/2|U k | 2 .
Depending on which voltage (k) is considered, these quantities have different physical interpretations:

• For 1 < k < n + 2, the expression 1/2|U k | 2 = 1/2|U k | 2 /C k
represents the average reactive energy stored in capacitor (k).

• For k = 1, N + 2, the expression 1/2|U k | 2 represents the average power dissipated in the unit admittance at the access ports.

After some manipulations we find,

T k,1 = 1 2 |U k | 2 (8) = 1 2 |W 1,k i 1 | 2 (9) = 1 2 |W 1,k 2 a 1 | 2 (10) = ∂S 1,1 ∂M k,k |a 1 | 2 (11)
where a 1 is the amplitude of the incident wave. Equation ( 9) is obtained from the the definition of W i,j and the circuital equation (2), and ( 11) is a direct consequence of (4). We obtain in the same manner the relation between T k,2 and the sensitivity of S 2,2 . For the rest of the paper we will suppose that the input powers |a 1 | 2 , |a 2 | 2 are normalized to 1 Watt, which yields following normalized equations

T k,1 = 2|W 1,k | 2 = ∂S 1,1 ∂M k,k (12)
T k,2 = 2|W N +2,k | 2 = ∂S 2,2 ∂M k,k (13)
Combining equations ( 12),( 13) with ( 5),( 6), [START_REF] Amari | Sensitivity analysis of coupled resonator filters[END_REF] completes the correspondence between sensitivities (in fact their modulus) and the reactive stored energies (or dissipated power at the loads).

l = k, ∂S 1,1 ∂M l,k = 2 T l,1 T k,1 , ∂S 2,2 ∂M l,k = 2 T l,2 T k,2 (14)
l = k, ∂S 1,2 ∂M l,l = T l,1 T l,2 (15)
l = k, ∂S 1,2 ∂M l,k = T l,1 T k,2 + T k,1 T l,2 (16)
IV. IMPLICATIONS AND APPLICATIONS An important implication of the equations derived in previous section is to yield bounds on the sensitivities in terms of physical quantities: the reactive stored energies T k,l for 1 < k < n + 2 or dissipated powers for k = 1, N + 2. To handle the sensitivity of the modulus of S 1,1 = |S 1,1 |e j θ S 11 with respect to diagonal couplings (away from reflexion zeros) remark that

∂S 1,1 ∂M k,k 2 = ∂|S 1,1 | ∂M k,k 2 + |S 1,1 | 2 ∂Arg(S 1,1 ) ∂M k,k 2 (17) which yields ∂|S 1,1 | ∂M k,k ≤ T k,1 (18)
This is an important result, since it shows that the time average stored energy of a resonator T k,1 is a maximum boundary of the sensitivity of the |S 11 | parameter. Thus, the time average stored energy in the resonators of a filter can be used to predict a maximum deviation in the reflection parameter of the filter for a specific manufacturing tolerance affecting the resonator dimensions. This was already intuitively noticed in [START_REF] Martínez-Mendoza | Investigation of the relationship between sensitivity and stored energy[END_REF], [START_REF] Martínez-Mendoza | Prediction of fabrication yield from low-pass prototype in a butterworth direct-coupled cavity filter[END_REF], but a mathematical proof was not available at that time.

Another interesting interpretation cast some light on the global structure of the sensitivity parameters. The reactive energy derives from an energy function verifying the edge port conservation rule, provided the considered network is loss-less: the total reactive energy of a loss-less network can therefore be computed from its edge port parameters, here the S-parameters. Precisely we have (see [START_REF] Kishi | Energy theory of sensitivity in lcr networks[END_REF], [START_REF] Kishi | Relations between reactive energy and group delay in lumped-constant networks[END_REF], [START_REF] Ernst | Relationship between group delay and stored energy in microwave filters[END_REF]),

T 1 = N +1 k=2 T k,1 = -|S 1,1 | 2 ∂Arg(S 1,1 ) ∂ω -|S 1,2 | 2 ∂Arg(S 1,2 ) ∂ω (19)
T 2 = N +1 k=2 T k,2 = -|S 2,2 | 2 ∂Arg(S 2,2 ) ∂ω -|S 2,1 | 2 ∂Arg(S 2,1 ) ∂ω ( 20
)
and both expressions are equal to the group delay -∂Arg(S2,1) ∂ω for auto-reciprocal characteristics (S 1,1 = ±S 2,2 ). In view of ( 12) and ( 13) this shows that the global sensitivities, that is

N +1 1 ∂S1,1 ∂M k,k and N +1 1
∂S2,2
∂M k,k , depend only on the filter's response. In particular, they do not depend on the coupling matrix topology chosen to realize the filter. In other words, the filtering characteristic sets the global sensitivity, whereas the repartition of the latter in each single resonator is driven by the coupling topology. Equations ( 12) to (16) reveals that, coupling schemes with an equally distributed time average stored energy among the resonators, will exhibit a lower sensitivity boundary of the reflexion S-Parameters. In this sense, works targeted to equally distribute the time average stored energy in the resonators of a filter with the aim of improving the power handling capability of the structure [START_REF] Ernst | Energy Storage in Microwave Cavity Filter Networks[END_REF], [START_REF] Ernst | Comparison of the stored energy distribution in a qc-type and a tc-type prototype with the same power transfer function[END_REF], can be employed to find structures with low sensitivity.

V. VALIDATION AND REMARKS

In this section, the results previously derived will be illustrated through several simple examples. The examples will deal with different transfer functions and different filter topologies.

The first objective of the study is to show the coincidence between the absolute value of the reflection S-Parameter variation with respect to diagonal coupling matrix elements and the time average stored energy in the resonators related to that diagonal elements. In order to to this, a second order Chebyshev transfer function with 25 dB return losses has been synthesized. The transfer function is labelled as T F 1 in Fig. 2. If the filter network representing T F 1 is synthesized into an inline configuration (see Fig. 3(a)), the coupling matrix results:

M 1 =     0 1.4312 0 0 1.4312 0 2.1670 0 0 2.1670 0 1.4312 0 0 1.4312 0     (21)
On the other hand, if the filter network representing T F 1 is synthesized into a transversal configuration (see Fig. 3(b)), the coupling matrix results:

M 2 =    
0 -1.0120 1.0120 0 -1.0120 -2.1670 0 1.0120 1.0120 0 2.1670 1.0120 0 1.0120 1.0120 0 The time average stored energy in the resonators of the inline and the transversal networks, as well as the absolute value of the reflection S-Parameter variation with respect to diagonal coupling matrix elements M 11 and M 22 , are shown in Fig. 4 (inline topology) and Fig. 5 (transversal topology). As predicted by [START_REF] Ernst | Comparison of the stored energy distribution in a qc-type and a tc-type prototype with the same power transfer function[END_REF], Fig. 4 and Fig. 5 show that the sensitivity of S 11 with respect to each diagonal coupling matrix element (Sens k,k ) and the time average stored energy (T K,1 ) in its associated resonator are exactly equal. in its associated resonator (tase k ). Transfer function T F 1 (Fig. 2) in inline configuration ( Fig. 3(a)).

    (22) S L R R 2 1 S L R 1 R 2 S L R R 3 2 R 6 R R 4 1 R 5
As stated by (19) the total stored energy tase T OT is the same in both topologies (see Fig. 4 and Fig. 5). This total energy is however better balanced between resonators in the in-line topology as compared to the transversal one. The maximal sensitivity (across all diagonal couplings) is therefore, for most frequencies, lower in the in-line topology.

The second target is to show that the total sensitivity of S 11 with respect to the diagonal coupling matrix terms is different for different transfer functions, independently of their filter topology. Note that the total sensitivity of S 11 with respect to the diagonal coupling matrix terms equals the sum of the stored energy in the resonators of the networks, which is known to be constant for a given transfer function [START_REF] Ernst | Energy Storage in Microwave Cavity Filter Networks[END_REF], [START_REF] Ernst | Comparison of the stored energy distribution in a qc-type and a tc-type prototype with the same power transfer function[END_REF]. A different transfer function to the one used in the previous examples (see T F 1 in Fig. 2) have been synthesized in transversal configuration. This new transfer function exhibits a transmission zero at ω = j • 4 rad/s, and presents 25 dB of return losses. It is labelled as T F 2 in Fig. 2, and its coupling matrix results:

M 3 =     0 0.
    (23)
The total sensitivity of S 11 with respect to the diagonal coupling matrix terms (Sens 11 + Sens 22 ) for both transfer functions T F 1 and T F 2 of Fig. 2, is shown in Fig. 6. Note that the total sensitivity with respect to diagonal terms equals the total time averaged stored energy in the network, and thus it also equals the group delay of the transfer function under study if the available power is a 1 = 1 [START_REF] Ernst | Energy Storage in Microwave Cavity Filter Networks[END_REF], [START_REF] Ernst | Comparison of the stored energy distribution in a qc-type and a tc-type prototype with the same power transfer function[END_REF]. It is known that the time averaged stored energy in a network changes if the transfer function represented by the network is changed. Thus, it is expected that the total sensitivity of S 11 with respect to the diagonal coupling matrix terms varies for different transfer functions. As expected, we notice that the sensitivity of T F 2 is clearly higher than the sensitivity of T F 1 in the frequency range of the passband closer to the transmission zero of T F 2 , but lower in the other region.

The next target is to show that the sum of the coupling sensitivities is different for different topologies implementing the same transfer function. In order to do that, the total sensitivity of S 11 with respect to the general coupling matrix elements (non diagonal) Sen T ot for T F 1 of Fig. 2 implemented in inline and transversal configurations, is shown in Fig. 7. It is observed that both traces are different, being larger the total sensitivity Sen T ot with respect to the couplings in the transversal network. This was intuitively expected, since the number of couplings in both topologies is different (the transversal topology implements four main couplings, whereas the inline topology is formed with just three main couplings). To sum up, when including couplings in the global sensitivity, the sensitivity becomes dependant on the topology. Besides, there is no longer equality with the group delay, although an upper bound involving the group delay could still be computed for each specific topology. Note that the sensitivity of S 11 with respect to the individual general coupling matrix elements for transfer function T F 1 of Fig. 2 in inline and transversal configurations are also plotted in Fig. 7. Similar plots have been obtained from the time average stored energy and the dissipated power in the ports by applying ( 14), ( 15), (16), although they are not included in Fig. 7 because exactly the same results are obtained.

Finally, the last target is to compare the total sensitivity with respect to the general coupling matrix elements (i.e. nondiagonal) of two different implementations of the same transfer function with the same topology. This will allow to choose the coupling matrix with the lowest sensitivity. Note that the total sensitivity with respect to the diagonal elements will be the same in both cases, since as demonstrated in the previous section, it equals to the group delay of the response. In order to compare two different implementations of the same transfer function with the same topology, the sixth order transfer function shown in Fig. 8 will be synthesized. The selected topology is the extended box (see Fig. 3(c)). The extended box filter configuration was introduced in [START_REF] Cameron | Synthesis of advanced microwave filters without diagonal cross-couplings[END_REF], and it is known to have several solutions for a given coupling matrix synthesis problem [START_REF] Cameron | An exhaustive approach to the coupling matrix synthesis problem application to the design of high degree asymmetric filters[END_REF]. This offers some flexibility in order to select the solution with the best sensitivity behavior. Two different coupling matrices M 4 and M 5 have been synthesized using the software package Dedale-HF [START_REF] Seyfert | Dedale-hf[END_REF].

Figure. 9 shows, for both solutions, the stored energies parameters T 3,1 . . . T 6,1 with respect to the normalized frequency ω (T 2,1 and T 7,1 are the same for both solutions, hence not plotted). It is observed that the M 5 solution is extremely sensitive with respect to the resonance frequency of the second resonator, and this on the lower border of the pass-band (see T 3,1 in Figure .9 ). This is confirmed by the value of its frequency shift M 5 3,3 = 1.0014, which indicates that resonator 2 is heavily de-tuned with respect to the central frequency. It suggests that solution M 4 achieves here a better balance of the total stored energy. This in turn will result, as shown in the paper, in a lower global maximal sensitivity, obtained here in both solutions for the M 3,3 element around ω = -1. Due to formula ( 14) solution M 4 will also outperform M 5 when considering maximal sensitivities (for example over the frequency range [-1.5, 1.5]) with respect to the couplings M 2,3 and M 3,4 . Of course, due to the invariance of T 1 , the solution M 5 performs "better' than M 4 when looking at the sensitivities related to the fourth and fifth resonators.

VI. CONCLUSIONS

Precise formulas to express the formal relationship between the time average stored energy in the resonators of a lowpass filter network and the sensitivity of the reflection S parameter with respect to the coupling matrix terms have been presented in this paper. The relationships have been described in the modern context of the N+2 coupling matrix for both diagonal and general coupling elements of the matrix. Furthermore, important implications have been derived from the new relationships found in the paper, allowing to establish maximum sensitivity boundaries. The latter were obtained from energy considerations for the special case of the classical lowpass (N + 2 × N + 2) prototype yielding some insight about how to perform practically with the computation of the reactive energies. Another, but more abstract way to go is certainly to extend, in all generality, Kishi's energy theory to networks with constant inverters, and FIR elements.

Fig. 1 .

 1 Fig. 1. General low pass circuit related to the (N + 2) × (N + 2) coupling matrix framework (for simpler notations, the capacitors have been numbered starting with C 2 , C 3 . . . and ending with C N +1 ).

2 Fig. 2 .

 22 Fig. 2. Second order Chebyshev transfer function without finite transmission zeros (T F 1 ) and with one transmission zero above the passband (T F 2 )

Fig. 3 .

 3 Fig. 3. Filter topologies: a) Inline, b) Transversal, c) Extended Box.

Fig. 4 .

 4 Fig. 4. Absolute value of the S 11 variation with respect to each diagonal coupling matrix element M kk (Sens kk ) and time average stored energy (measured in Joules)

Fig. 5 .

 5 Fig. 5. Absolute value of the S 11 variation with respect to each diagonal coupling matrix element M kk (Sens kk ) and time average stored energy (measured in Joules) in its associated resonator (tase k ). Transfer function T F 1 (Fig. 2) in transversal configuration ( Fig. 3(b)).

Fig. 6 .

 6 Fig. 6. Total sensitivity of S 11 with respect to the diagonal coupling matrix terms for T F 1 and T F 2 shown in Fig. 2.

Fig. 7 .

 7 Fig. 7. Sensitivity of S 11 with respect to the general coupling matrix elements for transfer function T F 1 of Fig. 2 in inline and transversal configurations.

12 Fig. 8 .

 128 Fig. 8. Sixth order Chebyshev transfer function with two asymmetric finite transmission zeros below the passband.

Fig. 9 .

 9 Fig. 9. Stored energies for second to forth resonator and for both solution M 4 and M 5

Technical University of Cartagena, Campus Muralla del Mar s/n, Cartagena, E-30202, Spain, e-mail: shanaz00@hotmail.com, josea.lorente@gmail.com, alejandro.alvarez@upct.es

INRIA, France, e-mail: fabien.seyfert@inria.fr

TEC-ETM ESA/ESTEC Kepleerlan 1 -2202 AZ Noordwijk ZH -The Netherlands email: christoph.ernst@esa.int


Imen Halima
email: imen.halima@ecam-rennes.fr

Jean-Marc Laferté
email: jean-marc.laferte@ecam-rennes.fr

Geoffroy Cormier
email: geoffroy.cormier@neotec-vision.com

Alain-Jérôme Fougères
email: alain-jerome.fougeres@ecam-rennes.fr

Jean-Louis Dillenseger
email: jean-louis.dillenseger@univ-rennes1.fr

Depth and thermal information fusion for head tracking using particle filter in a fall detection context

Keywords: head tracking, sensor fusion, particle filter, thermal sensor, depth sensor

The security of elderly people living alone is a major issue. A system that detects anomalies can be useful for both individual and retirement homes. In this paper, we present an adaptive human tracking method built on particle filter, using depth and thermal information based on the velocity and the position of the head. The main contribution of this paper is the fusion of information to improve tracking. For each frame, there is a new combination of coefficients for each particle based on an adaptive weighting. Results show that the tracking method can deal with the cases of fast motion (fall), partial occultation and scale variation. To assess the impact of fusion on the tracking process, the robustness and accuracy of the method are tested on a variety of challenging scenarios with or without depth-thermal fusion.

Introduction

According to the French institute of health education (INPES), 9,300 people die each year from falls. These falls occur mainly at home (78% of falls) and especially at night (60%) causing physical and psychological consequences. In accordance with the World Health Organization (WHO), falling is the second leading cause of accidental or unintentional injury deaths worldwide [1]. For these reasons, an automatic system that could prevent and detect falls and call emergency services can be useful even for retirement homes. Actually, many fall detection (FD) and fall prevention (FP) systems have been presented by researchers. These systems can be classified into three categories according to the type of the sensor used: wearable technologies, ambient technologies and a combination of wearable and ambient technologies. Wearable technologies encompass two different types of hardware: inertial sensors (e.g., tri-axial accelerometer) and locating systems (GPS). Ambient technologies include vision sensors (e.g., cameras), sound sensors (e.g., microphones), radar sensors (e.g., Doppler radar), infrared sensors and pressure sensors (e.g., floor sensors) or combinations of them [START_REF] Zhang | HONEY: a multimodality fall detection and telecare system[END_REF].

In this paper, we aim to develop a person tracking algorithm in order to improve the accuracy and the sensitivity of the system proposed in [START_REF] Cormier | Analyse statique et dynamique de cartes de profondeurs : application au suivi des personnes à risque sur leur lieu de vie[END_REF] and to reduce the number of false alarms. Moreover, we would like to track the elderly person's activity in order to prevent falls.

In a previous work [START_REF] Halima | Sensors fusion for head tracking using Particle filter in a context of falls detection[END_REF], a head tracking method using the fusion of low cost thermal and depth sensors for home environments whilst preserving privacy was proposed. The addition of thermal sensor improves the tracking with depth sensor. For example, thermal information adjusts depth detection by discriminating be-0921-7126/0-1900/$35.00 c 0 -IOS Press and the authors. All rights reserved tween hot objects and cold objects moved after calculating the background image. The results demonstrate that fusion improves tracking, namely when segmentation was erroneous. However, it missed partial occluded falls, and was unable to track fast motions in real time which are interesting for fall detection. For these reasons, this paper examines the data fusion to improve fast motion tracking and partial occlusion using particle filter (PF) algorithm based on head position. Particle filtering is a sequential importance sampling method using a set of particles to estimate the posterior distribution of a Markovian process, given noisy observations. The key idea of PFs is to represent and maintain the posteriori density function by a set of random samples with associated weights and to compute the state estimate from those samples and their weights. For each depth-thermal image pair, the head position is first segmented in the depth image, and then matched with the thermal image using calibration information to predict the actual position according to the previous state. The fusion of thermal and depth information is used to update this predicted state.

This paper extends the depth-thermal tracking method based on particle filter, explained in [START_REF] Halima | Sensors fusion for head tracking using Particle filter in a context of falls detection[END_REF], by including the velocity of the head in the state vector to improve fast motion. The method was tested on several sequences, with or without depth-thermal fusion: results show its robustness and accuracy and also demonstrate that adaptive measurements of each particle by using the velocity and the position of the head improve the fast motion, partial occlusion and scale variation.

The paper is organized as follows: Section 1 contains a general introduction of fall detection system. Section 2 gives an overview of the state-of-the-art vision fall detection systems. Section 3 describes the material used, the architecture of tracking algorithm, and proposed methodology to detect falls. Section 4 discusses the experimental set-up of our dataset, the results with or without depth-thermal fusion, as well as the performance evaluation and a detailed discussion. Section 5 provides conclusion and further research potential.

Related work

A fall is defined as an event which results in a person coming to rest inadvertently on the ground or floor or other lower level. Adults older than 65 years of age suffer the greatest number of fatal falls [1]. Several FD systems have been proposed to identify and classify human activities of daily living (ADL) and to reduce the risk of elderly falls, the response and the rescue time. Many studies focusing on FD survey were increased rapidly in the world. For example, Mubashir et al. [START_REF] Mubashir | A survey on fall detection: Principles and approaches[END_REF] chose to classify the FD systems into three categories: wearable device based, ambience sensor based and camera (vision) based. However, Igual et al. [START_REF] Igual | Challenges, issues and trends in fall detection systems[END_REF] chose only two categories: context-aware systems and wearable devices. While falling detection context is promising, exciting challenges still occur. In this paper, we will study the most commonly cited works in the literature according to their advantages and their drawbacks such as cost, application, installation and privacy.

Over the last decade, the focus has been on context aware systems (vision systems especially), because the person is more independent and not constrained by the presence and the configuration of the device. Several methods use particle filters for object tracking and localization. In [START_REF] Yu | Fall detection in the elderly by head tracking[END_REF][START_REF] Debard | Camera-based fall detection using a particle filter[END_REF] the authors describe the application of particle filters for tracking moving objects using background subtraction to track human silhouettes based on color images. In [START_REF] Rougier | 3D head tracking for fall detection using a single calibrated camera[END_REF], Rougier et al. have used the head's velocity to detect the fall in visual videos by setting thresholds manually. In the same vein, [START_REF] Bouaynaya | An online motionbased particle filter for head tracking applications[END_REF] have used particle filter for head tracking based on colored histograms. In [START_REF] Łoza | Structural similarity-based object tracking in multimodality surveillance videos[END_REF], L. Loza et. al have applied PF on thermal imagining. Mubashir et al. [START_REF] Mubashir | A survey on fall detection: Principles and approaches[END_REF] have used head position to track the person's silhouette based on a Gaussian classifier. In [START_REF] Rougier | Robust video surveillance for fall detection based on human shape deformation[END_REF] the silhouette was extracted from video to localize the person which is a common strategy in the literature. However, these methods provide false alarms because it is difficult to distinguish a fall from other similar actions, e.g. sitting down. Therefore, in [START_REF] Auvinet | Fall detection with multiple cameras: An occlusion-resistant method based on 3-D silhouette vertical distribution[END_REF], Auvinet et al. have added other cameras to analyze the shape of the person in 3D and avoid hidden falls. But elderly people dislike the use of visual cameras even with local processing. They prefer noninvasive devices which preserve their privacy according to a psychosocial study done by LAUREPS laboratory at University of Rennes 2.

In order to protect user privacy, 3D fall detection systems using depth sensors were used in a fall detection context. The aim of using a depth camera like Kinect is to analyze the human shape and extract 3D features for fall detection [START_REF] Gasparrini | A depth-based fall detection system using a Kinect R sensor[END_REF]. A recent work used head position detection, extracting from depth images [START_REF] Nghiem | Head detection using Kinect camera and its application to fall detection[END_REF] and the experimental results confirm the feasibility and the effectiveness of the approach for real world applications. In [START_REF] Ballotta | Fully convolutional network for head detection with depth images[END_REF], 3D data are exploited to perform head detection for a fall detection framework. Human silhouettes, obtained by a background subtraction, are detected and all possible head positions are searched on contour segments. But in fall detection, it could not recognize correctly for instance when the person bent his knee too much to slow down the fall.

To avoid this problem, some works have used other non-invasive sensors such as thermal sensors. For example, Hayashida et al. [START_REF] Hayashida | The use of thermal ir array sensor for indoor fall detection[END_REF] integrate a thermal infrared array sensor to detect falls by computing the maximal thermal difference between the background and foreground pixels which is a technique used for static cameras. The current frame is subtracted from the model of the background scene and eventually, the difference, determines the moving objects. However, the configuration was sensitive to room temperature and brightness. In [START_REF] Mashiyama | Activity recognition using low resolution infrared array sensor[END_REF] authors proposed a system to recognize human activities, which include falls, by means of a single thermal infrared sensor. Several features based on temperature thresholds are proposed to be evaluated by a Support Vector Machine (SVM) in the classification. However, in [START_REF] Wu | Geometry based three-dimensional image processing method for electronic cluster eye[END_REF][START_REF] Wu | A multi-aperture optical flow estimation method for an ar-tificial compound eye[END_REF] authors have proposed another type of thermal sensor but relatively expensive. In [START_REF] Vadivelu | Thermal Imaging Based Elderly Fall Detection[END_REF], a very economical thermal imaging based input modality is proposed to detect falls using the optical flow of human movements tested on public datasets. These proposed methods achieved a good performance but included some confusion in distinguishing between falling and sitting.

The number of studies using analytical methods is still increasing but there is a new trend in fall detection which is the use of machine learning methods and the most popular algorithm in this context is deep learning. For instance, Quero et al. [START_REF] Quero | Detection of Falls from Non-Invasive Thermal Vision Sensors Using Convolutional Neural Networks[END_REF] detected falls from non-invasive thermal vision sensor (Heimann HTPA 32×31) using Convolutional Neural Networks (CNN). Wang et al. [START_REF] Wang | Human fall detection in surveillance video based on PCANet, Mul-Fig. 15. Row depth image with large black regions (masked information due to poor quality)[END_REF] proposed a fall detection system using a PCAnet to extract features from color images and then applied a SVM to detect falls. Nunez-Marcos et al. [START_REF] Núñez-Marcos | Vision-Based Fall Detection with Convolutional Neural Networks[END_REF] proposed a similar approach but, instead of a PCAnet, they used a modified VGG16 architecture. These methods are promising but usually require a large dataset to train a classifier and are inclined to be influenced by the image quality.

In order to efficiently improve results, some papers combined sensors. Interesting examples are provided in [START_REF] Koshmak | Challenges and Issues in Multisensor Fusion Approach for Fall Detection: Review Paper[END_REF] and [START_REF] Chaccour | From Fall Detection to Fall Prevention: A Generic Classification of Fall-Related Systems[END_REF], for example Kinect and accelerometers, or cameras with microphones plus accelerometers. In [START_REF] Huang | Learning-based Human Fall Detection using RGB-D cameras[END_REF], human silhouette was extracted using RGB-D camera. Recently RGB-T systems attracted a lot of attention, e.g. Wu et al. [START_REF] Wu | Multiple source data fusion via sparse representation for robust visual tracking[END_REF] combined RGB and thermal data into one vector which, however, intro-duces redundant information and the use of color information cannot preserve user privacy.

In this paper, we propose a combination between depth and thermal sensors, FLIR sensor (80 × 60) and Kinect sensor (640 × 480) respectively. Our method aims to track the head position in a context of daily activities classification. We apply particle filter on fusion information, include the position and the velocity of the head to the state vector and modify the modality for each particle based on an adaptive weighting of each frame. In the interest of privacy, we chose not to use color imaging.

Material and methods

The proposed system aims to track the head position using two types of sensors with different resolutions which are mounted together. The head position can be tracked according to an analytical method applying on a segmented frame. With the calibration step done before starting processing, the unidirectional thermaldepth matching can be made throughout all sequences.

In this research, we chose head position as Region Of Interest (ROI) because it is non-deformable, the hottest, highest and least hidden part of the body which can easily be approximated as an ellipse with only few parameters. Head motion is also a significant marker for fall detection.

Cameras and dataset

The fall detection system is based on thermal sensor (FLIR lepton 2.5, Focal length: 5 mm, Thermal Horizontal Field of View T HFOV : 51 o , Thermal Vertical Field of View T V FOV : 37.83 o , Thermal Resolution T Xres : 80 pixels and T Yres : 60 pixels) and a depth camera (Microsoft Kinect V1, Focal length: 6.1 mm, Depth Horizontal Field of View D HFOV : 58 o , Depth Vertical Field of View D V FOV : 45 o , Depth Resolution D Xres : 640 pixels and D Yres : 480 pixels). Figure 1 shows the sample output images of Kinect and FLIR acquisition system. These sensors can capture 3D and thermal video data under various light conditions. The extracted dataset can be divided into two principal categories (ADL and abnormal activities) to experiment the proposed method. Totally 60 video sequences with 10000 frames are recorded from 5 different human subjects in three different places with or without presence of obstacles.

Framework

Figure 2 illustrates the framework of our proposed fall detection system. This proposed system can be divided into three principal stages: calibration, segmentation and tracking. The calibration step, which is done only one time, is executed after attaching the sensors to the ceiling to be able to match a depth pixel to its corresponding thermal pixel. The segmentation step, based on acquired images, serves to detect depth foreground image by subtraction of the background and extract head position. The tracking step is based on the head position segmented on the depth image and matched to the thermal position and improved by the particle filter.

Calibration

A calibration step is required to calculate the transformation parameters (extrinsic parameters). In the literature, a conventional black and white chessboard pattern is often used in many existing methods to calibrate two cameras or more. To obtain higher accurate calibration results, this pattern needs to be kept near the cameras. The orientation could sometimes result in limiting the number of poses [START_REF] Rathnayaka | An Efficient Calibration Method for a Stereo Camera System with Heterogeneous Lenses Using an Embedded Checkerboard Pattern[END_REF]. Besides, this pattern cannot be seen by thermal sensors. For these reasons, we have decided to design a special pattern which contains several tubes of different heights mounted together on a board and different resistors fixed on each tube. The idea of this pattern is simple. The tubes will be seen by the depth sensor and the heat emitted by the resistors will be seen by the thermal sensor. Calibration pattern is shown on depth, thermal and color images in Figure 3.

The calibration operation comprises modeling the image transformation process. This process transforms points from image coordinates to a common world coordinates system. The idea is to find the relation between the coordinates of a point in the depth image with the associated point in the thermal image taking into consideration the spatial coordinates of each point (Figure 4). The estimation of the relationship between these two coordinate systems needs three steps [START_REF] Szeliski | Computer Vision: Algorithms and Applications[END_REF]:

(1) The estimation of the transformation of the depth image coordinates (u d , v d ) to the coordinate system (x d , y d , z d ) of the depth sensor. This can be done analytically from the intrinsic parameters of the depth camera, Eq. (1):

R =   cos(α) -sin(α) 0 sin(α) cos(α) 0 0 0 1   *   1 0 0 0 cos(θ) -sin(θ) 0 sin(θ) cos(θ)   *   cos(β) 0 sin(β) 0 1 0 -sin(β) 0 cos(β)   (2) T =   d x d y d z   (3)   x th y th z th   = T + R   x d y d z d   (4)
where α, θ and β are the Roll, Pitch and Yaw angles [START_REF] Vadivelu | Thermal Imaging Based Elderly Fall Detection[END_REF]. [START_REF] Cormier | Analyse statique et dynamique de cartes de profondeurs : application au suivi des personnes à risque sur leur lieu de vie[END_REF] The transformation between the coordinate system (x th , z th , z th ) of the thermal sensor to the thermal image coordinates (u th , v th ) (there are only 2 equations since the pixel value is the temperature which is not geometric information). This can be done analytically from the intrinsic parameters of the thermal camera, Eq. ( 5):

   u th = TX Res 2zth tan( T HFOV 2 ) x th + TX Res 2 v th =- TY Res 2zth tan( T V FOV 2 ) y th + TY Res 2 (5)
In our case, the intrinsic parameters are the values given by the constructor. So the purpose of the calibration is to estimate 3 parameters of rotation transformation and 3 parameters of translation transformation (α, θ, β, d x , d y , d z ) respectively using a nonlinear optimization techniques (Levenberg-Marquardt [START_REF] Mathai | A new algorithm for nonlinear least squares[END_REF]) and then to generate a one to one pixel correspondence from the depth to the thermal images (nonreciprocal correspondence).

Segmentation step

In order to improve the segmentation robustness, we calculate a reference map based on the mean and standard deviation of the N first depth images without any moving objects. Then, we subtract the reference map from depth image to extract the depth foreground image. For a new frame, each pixel p(i, j) is considered as foreground so long as it is above a threshold calculated by the variance of the reference map. Otherwise, the pixel is classified as background:

if |p(i, j)re f (i, j)| > 2σ(i, j) then p(i, j) ∈ Foreground else p(i, j) ∈ Background where re f (i, j) is the mean pixel (i, j) from reference map and σ(i, j) is the standard deviation of p(i, j).

Next we have one or more areas detected as foreground: we compare these areas and we hold only the bigger one. Following the choice of the area, we approximate the body with an ellipse.

Finally, we model the head as a smaller ellipse with the same orientation of the silhouette ellipse. Human adult body proportions are brought about by differential growth of the body segments. From 25 years of age the head is only approximately one-sixth of the total body length [START_REF] Bogin | Leg length, body proportion, and health: a review with a note on beauty[END_REF]. Therefore, we fixed the center SC of the head ellipse H at the 1/6 of half major axis SB from the upper part of major axis (see Figure 5), Eq. ( 6):

SC = 5 6 SB ( 6
)
where S is the silhouette ellipse center, B is the upper point of the major axis and BC is the head major axis knowing that the ratio between major axis and minor axis is set to 1.2 [START_REF] Yu | Fall detection in the elderly by head tracking[END_REF]. The sole use of the seg-mentation process is not a robust method to track the head position. False alarms can be trigged for any object moved after calculating the reference map. Thus, the use of a tracker is necessary.

Tracking process

The aim of the tracking is to estimate the position of the head during a sequence by considering the last movement of this ROI. Therefore, we chose a sequential Monte Carlo method which is Particle Filter (PF) method. At each frame t, the state vector x t of PF is defined by the center H, the size L and the orientation θ of the head extracted by the segmentation step in depth image, Eq. ( 7).

x(S 1) t = (x H , y H , L, θ) T

PF method seeks to estimate the hidden state vector x t from the previous state vector x t-1 , depth observation vectors

Z t = {z 1 , • • • , z t } and thermal observation vectors H t = {h 1 , • • • , h t }.
PF uses a sample of N particles

S t = S 1 t , • • • , S N t
to approximate the conditional probability p(x t /Z t , H t ).

Each particle S n t can be seen as a hypothesis about x t and is weighted by π t (n) which are normalized. Particles are resampled according to their weights and are updated according to new observations (coefficients) [START_REF] Isard | CONDENSATION-Conditional Density Propagation for Visual Tracking[END_REF]. new obsv (n) is a linear combination of coefficients tested in different forms (details in section 3.6). Thus, the PF does not consider one state vector but N particles state vectors associated with different weights. At each frame, the estimation of the head position is based on two models, the first model called AM considers the weighted average of the N particles and the second model called MM considers the particle with maximum weight.

To improve the estimation of the head position especially in cases of fast motion, we added the velocity (v xH , v yH ) on the state vector, Eq. ( 8).

x(S 2) t = (x H , y H , v xH , v yH , L, θ) T (8)
Below, we briefly define the PF algorithm. For each frame, we resample a new sample of N particles named S n t+1 , based on the previous state of each particle and the associated weights in order to prevent the problem of "particles degeneration" [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF]. Next, we predict the

S n t+1 = AS n t + w t (9)
where A stands for the transition model matrix and w t is a Gaussian noise. Finally, we update the particle weight according to observation vectors where we combine depth and thermal information in new obsv (n) (see figure 6).

Thus, the steps of iterative PF tracking algorithm are:

(1) Initialization: Generate a sample of N particles

S 1 = S 1 1 , • • • , S N 1
based on the probability of the state vector p(x 1 ), and initialize the weight of each particle by π t (n) = 1/N.

(2) Resampling: Resample particles to prevent the problem of particles degeneration, if frame > 1.

(3) Prediction: Propagate particles according to prediction model to predict the state vector x t . (4) Updating: Update the particle weight π t (n) at frame t according to observation vectors. Then normalize the weight:

π t (n) = π t (n) N k=1 π t (k) (10)
and return to step 2.

Depth-thermal fusion (first method)

Updating particle weights is a key point of PF and is specific for each application (see [START_REF] Nummiaro | An adaptive color-based particle filter[END_REF] for color information).

The weight of a particle is defined by Eq. ( 11):

π t (n) = 1 √ 2πσ exp(new obsv (n)/2σ 2 ) ( 11
)
where σ is theoretically the standard deviation of the coefficients combination. However, we have chosen a constant value of σ for computational cost reason.

We have tested several values and we returned the one that rendered the best result and new obsv (n) is a linear combination of three coefficients of thermal and depth observations: a depth distance coefficient D D , a depth gradient coefficient G D and a thermal gradient coefficient G T , where:

• D D is the distance between the center of the particle and the center of the segmented head in the depth image. • G D and G T are the gradients along the particle ellipse in the depth image and the thermal image respectively, as inspired by [START_REF] Rougier | 3D head tracking for fall detection using a single calibrated camera[END_REF].

When updating particle weight, we have observed that an occluded particle can decrease the performance of tracking. For example, it can influence the result of the AM model. To avoid this problem, we have added a flag to each particle at each frame (Flag (n) = 0 when the particle n is occluded) and we have eliminated the coefficient of this particle n on the update step following Table 1.

A particle is occluded for a sensor if it is out of the vision field of this sensor. This can occur especially after the prediction step. In this work, we tested four models of coefficient combination to update the particle weights in Eq. [START_REF] Łoza | Structural similarity-based object tracking in multimodality surveillance videos[END_REF].

The first model (M1) uses only 2 depth coefficients (D D and G D ), Eq. ( 12):

new obsv (n) 1 = αD (n) D + βG (n) D ( 12
)
where n is the particle index. There is no fusion here since we use only depth images.

The second model (M2) combines one depth coefficient (D D ) with one thermal coefficient (G T ), Eq. ( 13):

new obsv (n) 2 = αD (n) D + βG (n) T (13)
The third model (M3) combines the 3 coefficients, Eq. ( 14):

new obsv (n) 3 = αD (n) D + βG (n) D + γG (n) T (14)
We call parameters α, β, γ ∈ [0, 1] importance factors (IF). The distance coefficient is always considered because it is very discriminant.

We tested several combinations of static IF in order to estimate the impact of each coefficient (see Table 2).

Depth-thermal fusion (second method)

The use of static IF values is a general way to estimate coefficient impact because we fix a static value during the whole sequence. But at certain frames, thermal information can be more important than depth information and conversely. For instance, when the person is close to furniture that was moved after the reference map was calculated, the depth observations may not be relevant, because the silhouette can be merged with the furniture. Or if the person is close to a heater, the thermal observation cannot be efficient. Therefore, we decided to adjust the important factors dynamically and change the values at each frame according to the importance of each coefficient using these rules, Eqs. (15, 16 and 17).

if max max n G (n) D , max n D (n) D < max n G (n) D then γ = 1/2 and α = β = 1/4 (15) if max max n G (n) T , max n D (n) D < max n G (n) T then β = 1/2 and α = γ = 1/4 (16) if max max n G (n) T , max n G (n) D < max n D (n) D then α = 1/2 and β = γ = 1/4 (17)
In subsequent sections of this paper, we have compared this model called (M4) with other models defined previously.

Experimental results

In this section, we demonstrate the performance of the proposed algorithm. We have performed several sequences of people moving in a room with co-calibrated static depth and thermal cameras which were fixed in the ceiling. We have tested our system with the following objectives: (1) compare our proposal work with segmentation only and depth tracking methods, (2) evaluate the performance of the fusion algorithm, (3) evaluate each IF model, and (4) compare IF values.

In all tests, we used the following values: N = 1000 particles, σ = 0.25, and transition model matrix A = I 4 . We fixed the acquisition frequency to 8 Hz. The ground truth (GT) was established manually by setting an ellipse on each frame and the processing was performed using Matlab on Intel(R) Core(TM) i7-6700HQ CPU, 2.6 GHz.

The criteria for evaluation of our method utilizes two quantitative metrics (more details in [START_REF] Wu | Online Object Tracking: A Benchmark[END_REF]): the localization error (called precision plot) which is defined as the average Euclidean distance between the center locations of the tracked targets and the manually labeled ground truths, and the overlap score (called success plot) which is the overlap of the ground truth area and the tracking area.

Fusion of information

In this section, we illustrate, as described in section 3.4 and 3.5, the results of head segmentation (Eq. ( 6)), depth version (M1 model) (Eq. ( 12)) and first fusion model M2 (Eq. ( 13)), the difference between AM and MM models and a comparison between the first fusion model M2 and second fusion model M3 (Eq. ( 14)).

Figure 7 shows a representative a normal ADL of our datasets used in the evaluation experiment. The first three images in Figure 7 represent the results of segmentation. These results show that segmentation is wrong because the size and the position of the head do not vary whereas the silhouette's position from the captor does. In this case, the problem is caused by the segmented silhouette which does not contain legs. The second test is based on the depth method mentioned before. Comparing these results, we can see that the depth version M1 is totally erroneous because this method used segmentation to calculate the distance coefficient. In other words, the depth sensor is useless on its own. The last three images show the results of the first fusion model M2 which is able to track the head more accurately as the person moves because it employs a combination of thermal and depth imaging.

The results of MM and AM models are shown on Figure 8.a) and 8.b) respectively. We can see that the AM model provides the closest pose to GT.

In order to evaluate coefficient combination, Figure 9 shows a comparison of two fusion models a) M2 model and b) M3 model. Visually the second fusion model provides the closest pose to GT.

To validate these results, we have evaluated these models according to the precision and success metrics. The evaluation results of quantitative measurements over a sequence show in Figure 10 that fusion of 3 coefficients provides the most accurate results. As expected, considering three coefficients together gives better results than using only two coefficients.

Comparison of static IF models

As mentioned in section 3.6, the third model (M3) combines the 3 coefficients, (Eq. ( 14)).

To evaluate the importance of each IF (α, β, γ) , we have performed different tests of IF values fixed on Table 2 on several sequences.

Figure 11 illustrates a comparison between these tests on a normal ADL frame. The visual results show the impact of IF in estimating the new head position. Confirming the IF impact during a sequence using the two quantitative measurements, Figure 12 shows a clear difference between the performance of C4 (Figure 11.d) compared to other tests.

Robustness of adding velocity on state vector

In this study, we present an improved version of an algorithm initially proposed in our earlier work [START_REF] Halima | Sensors fusion for head tracking using Particle filter in a context of falls detection[END_REF]. In addition to size and orientation of the head ellipse, we have added the velocity to the state vector. Figure 13 illustrates a representative scene of fast motion. The first two images (Figure 13.a) show results of the algorithm without adding velocity on the state vector. Figure 13.b shows the impact of velocity especially in fast movement.

Robustness of adaptive weighting

As mentioned in section 3.7, the fourth model (M4) adjusts the important factors dynamically and changes the values at each frame according to the importance of each coefficient. To evaluate the impact of the adaptive combination, we have compared this model with the result of test C4 mentioned in section 4.2. Figure 14 illustrates a comparison between C4 and this model according to the success metrics. Figure 14 shows a clear difference between the performance of M4 compared to C4.

Summary

In this research, we started by testing the models of head estimation. The first model AM, which considers the weighted average of these N particles, provides the closest pose to GT as opposed to the second model MM which considers the particle with maximum weight. Then we compared segmentation and depth versions to the first and second fusion models (M1 and M2). We concluded that using both thermal and depth observations improved tracking results. The results were clearly different between each test according to the environment at time t. For this reason, we modified the static IF to dynamic according to the coefficient value at each frame.

Finally, we added velocity to the state vector to improve the estimation of the head position especially in cases of fast motion.

Conclusion

In this paper, we have detailed a tracking approach based on a particle filter using depth and thermal information fusion to detect the position of the head of a person in an indoor environment. Position, velocity, orientation, and size of the ellipse enclosing the head are used to predict the new position of the head. Furthermore, adaptive weighting was applied on the measurements of each particle according to the strength of each coefficient to update the predicted position on each frame. Consequently, this method solves the updating problem we encountered in previous tracking works, caused by changes in the background. The pro-posed framework has been tested in several situations with different models and compared with other methods to establish the accuracy of the algorithm. Moreover, results have shown that our system gave the most accurate tracking results even in critical situations with very low resolution images.

Our aim is to refine the work presented here and better address the constraints of fall detection systems. Going forward, we plan to use deep learning (DL) methods due to their performances as mentioned in recent works [START_REF] Wu | Geometry based three-dimensional image processing method for electronic cluster eye[END_REF][START_REF] Wu | A multi-aperture optical flow estimation method for an ar-tificial compound eye[END_REF] to more accurately recognize human posture. We will start by 4 postures (standing up, sitting, lying on the ground and lying on a bed or sofa) in the context of fall detection and also fall prevention by activity analysis. Before using DL, we will apply a preprocessing step on depth images to enhance their quality and avoid losing pertinent information (see Figure 15).

  Fig. 1. Camera system

Fig. 2 .

 2 Fig. 2. Fall detection framework

( 2 )

 2 Fig. 4. Calibration system

Fig. 5 .

 5 Fig. 5. Silhouette and head position

Fig. 6 .

 6 Fig. 6. Segmentation step and PF algorithm actual state x t+1 according to the propagation of particles based on the prediction equation, Eq. (9):

Fig. 7 .Fig. 8 .

 78 Fig. 7. Tracking results on different frames of depth sequence a) Segmentation only, b) Depth version (M1 model), c) First fusion model (M2). Tracking results are in white, silhouette ellipse is red and GT ellipse is black

Fig. 9 .

 9 Fig. 9. Tracking comparison results between a) first fusion model (M2) and b) second fusion model (M3). Tracking results are white, silhouette ellipse is red and GT ellipse is black.

Fig. 10 .

 10 Fig. 10. Quantitative measurements over a sequence. Localization error (a) and the overlap score (b) using the segmentation (red), the depth version (blue) the M2 model (cyan) and the M3 model (blue)

Fig. 12 .

 12 Fig. 12. IF quantitative measurement over a sequence. Localization error of C1 test (blue), C2 test (yellow), C3 test (black), C4 test (red), C5 test (cyan) and C6 test (blue).

Fig. 13 .

 13 Fig. 13. Tracking comparison results between a) algorithm without velocity b) algorithm with velocity. Tracking results are white, silhouette ellipse is red and GT ellipse is black.

Fig. 14 .

 14 Fig. 14. Quantitative measurements over a sequence. Localization error of C4 (red) and M4 (purple)

Table 1

 1 Occluded particle conditionss

Table 2

 2 Importance factor IF values

	Test	α	β	γ	more impact
	C1	1/3 1/3 1/3	equal
	C2	1/4 1/2 1/4	β
	C3	1/4 1/4 1/2	γ
	C4	1/2 1/4 1/4	α
	C5	3/8 1/4 3/8	β
	C6	3/8 3/8 1/4	γ

Acknowledgements

This work is funded under the PRuDENCE project (ANR-16-CE19-0015-02) which has been supported by the French National Research Agency. A sincere thank you to Tabitha Courbin for her diligent proofreading of this paper.