Martin Strauss

Pol Mordel

Victor Miguet

Antoine Deleforge

DREGON: Dataset and Methods for UAV-Embedded Sound Source Localization

This paper introduces DREGON, a novel publiclyavailable dataset that aims at pushing research in sound source localization using a microphone array embedded in an unmanned aerial vehicle (UAV). The dataset contains both clean and noisy in-flight audio recordings continuously annotated with the 3D position of the target sound source using an accurate motion capture system. In addition, various signals of interests are available such as the rotational speed of individual rotors and inertial measurements at all time. Besides introducing the dataset, this paper sheds light on the specific properties, challenges and opportunities brought by the emerging task of UAV-embedded sound source localization. Several baseline methods are evaluated and compared on the dataset, with real-time applicability in mind. Very promising results are obtained for the localization of a broad-band source in loud noise conditions, while speech localization remains a challenge under extreme noise levels.

I. INTRODUCTION

Unmanned aerial vehicles (UAV), commonly referred to as drones, have been of increasing influence in recent years. Applications such as autonomous human transport machines or delivery devices for postal services are being envisioned [START_REF] Andrea | Guest Editorial Can Drones Deliver?[END_REF]. Search and rescue scenarios where humans in emergency situations need to be quickly found in areas difficult to access also constitute a potentially large field of application. While a number of UAV-embedded tools to address such situations have been developed using video cameras [START_REF] Lopez-Fuentes | Review on computer vision techniques in emergency situations[END_REF], audiobased source localization from UAVs has received much less research attention [START_REF] Basiri | Robust acoustic source localization of emergency signals from micro air vehicles[END_REF], [START_REF] Ohata | Improvement in Outdoor Sound Source Detection Using a Quadrotor-Embedded Microphone Array[END_REF], [START_REF] Hoshiba | Design of UAV-Embedded Microphone Array System for Sound Source Localization in Outdoor Environments[END_REF]. Though, UAVs equipped with a microphone array may present several advantages in emergency situations, especially whenever there is a lack of visual feedback due to bad lighting conditions (night, fog, etc.) or obstacles limiting the field of view [START_REF] Basiri | Robust acoustic source localization of emergency signals from micro air vehicles[END_REF].

With that in mind, to push forward investigations of audio properties during a UAV flight and the development of new embedded sound source localization method for search and rescue, this paper presents a novel state-of-the-art dataset called DREGON for DRone EGonoise and localizatiON. A quadrotor UAV equipped with an 8-channel cube-shaped microphone array (Fig. 1) was used to record in-flight audio in different scenarios with or without a target source simulated by a loudspeaker emitting various sounds. The 1 Martin Strauss is with Friedrich-Alexander University, Erlangen, Germany. martin.strauss@fau.de 2 Pol Mordel is with CNRS/IRISA Rennes -Bretagne Atlantique, Rennes, France. pol.mordel@irisa.fr 3 Victor Miguet is with ENS Rennes, France. 4 Antoine Deleforge is with Inria Rennes -Bretagne Atlantique, Rennes, France. antoine.deleforge@inria.fr two large rooms used for the flights were low-reverberant and equipped with a motion capture system. The system was used in all flights to obtain precise ground truth 6 degrees-of-freedom (DoF) coordinates of the UAV and target source in the room at all time. Additional synchronized signals of interest obtained from sensors embedded in the UAV and referred to as log signals are also included. These include the commanded and measured speed of each of the four propellers and inertial sensor measurements at all time. External fixed viewpoint video recordings of all flights are included for reference. In addition to in-flight recordings, reference noiseless recordings of static sound sources from all-sphere directions as well as recordings of the individual propellers at various speed are included. This is the first time a dataset of this kind is publicly released, to the best of the authors' knowledge.

As an illustration, the performance of different popular sound source localization (SSL) methods, namely, variations of the GCC-PHAT [START_REF] Knapp | The generalized correlation method for estimation of time delay[END_REF], [START_REF] Aarabi | Self-localizing dynamic microphone arrays[END_REF] and MUSIC [START_REF] Schmidt | Multiple Emitter Location and Signal Parameter Estimation[END_REF], [START_REF] Argentieri | Broadband variations of the music highresolution method for sound source localization in robotics[END_REF] algorithms, are compared on the proposed dataset with a focus on real time capabilities. Encouragingly, the best performing method enables in-flight estimation of the azimuth and elevation of a broadband source with 2 • accuracy, while in-flight speech localization remains a challenge. Finally, different properties of the recorded flight noise are analyzed and some leads for future research are outlined. The dataset as well as MATLAB code for the baseline methods are publicly available on the project website: dregon.inria.fr.

A. Challenges and Opportunities of the Dataset

The SSL capability of a UAV is influenced by a variety of effects. One major issue is the noise produced by the UAV itself, generically referred to as ego-noise in robotics [START_REF] Löllmann | Challenges in acoustic signal enhancement for human-robot communication[END_REF]. Due to the quickly changing speed of motors to stabilize the vehicle in the air or to change in its position, the noise profile is highly non-stationary. Additionally, since the microphones are mounted on the drone itself, they are very close to the noise sources leading to high noise levels. Because of this, the SNR can easily reach -15 dB or less [START_REF] Wang | Microphone-Array Ego-Noise Reduction Algorithms for Auditory Micro Aerial Vehicles[END_REF] making SSL very difficult. Another factor impacting localization performance is wind noise. The wind is produced by the rotating propellers, the UAV movement in the air and may also occur naturally in outdoor scenarios. This wind noise has high power and is of low-frequency. Hence, it easily overlaps with speech signals which typically occur in a similar frequency range [START_REF] Schmidt | Wind Noise Reduction using Non-Negative Sparse Coding[END_REF]. Last, SSL must be performed using relatively short time windows, due to the fast movements of the UAV relative to potential sound sources. All these challenges need to be tackled at the same time and in near real-time for reallife SSL scenario such as search and rescue.

On the bright side however, using microphones embedded in a UAV comes with interesting opportunities. Additionally to audio signals, other signals recorded by various embedded sensors (gyroscope, motor controllers, inertial measurement unit, compass, ...) may be available. We believe that multimodal approaches fusing information from multiple sensors in order to enhance SSL and source tracking present a promising direction for future research, as investigated in [START_REF] Furukawa | Noise Correlation Matrix Estimation for Improving Sound Source Localization by Multirotor UAV[END_REF], [START_REF] Morito | Partially Shared Deep Neural Network in Sound Source Separation and Identification Using UAV-Embedded Microphone Array[END_REF].

B. Related work

Ego-noise reduction is a topic of increasing interest in robotics for several years now [START_REF] Löllmann | Challenges in acoustic signal enhancement for human-robot communication[END_REF]. Different techniques have been reported to perform well, e.g., non-negative matrix factorization [START_REF] Tezuka | Ego-motion noise suppression for robots based on Semi-Blind Infinite Non-negative Matrix Factorization[END_REF], deep neural networks [START_REF] Ito | Internal noise suppression for speech recognition by small robots[END_REF] or dictionary learning [START_REF] Schmidt | Ego-Noise Reduction Using a Motor Data-Guided Multichannel Dictionary[END_REF]. On the other hand, SSL is a long standing and extensively studied topics in robotics [START_REF] Rascon | Localization of sound sources in robotics: A review[END_REF]. However, both ego-noise reduction and SSL for the specific setting of UAV-embedded microphones are still rather new topics [START_REF] Basiri | Robust acoustic source localization of emergency signals from micro air vehicles[END_REF], [START_REF] Okutani | Outdoor Auditory Scene Analysis Using a Moving Microphone Array Embedded in a Quadcopter[END_REF], [START_REF] Ohata | Improvement in Outdoor Sound Source Detection Using a Quadrotor-Embedded Microphone Array[END_REF], [START_REF] Morito | Partially Shared Deep Neural Network in Sound Source Separation and Identification Using UAV-Embedded Microphone Array[END_REF], [START_REF] Wang | Ear in the sky: Ego-noise reduction for auditory micro aerial vehicles[END_REF], [START_REF] Hoshiba | Design of UAV-Embedded Microphone Array System for Sound Source Localization in Outdoor Environments[END_REF], [START_REF] Wang | Microphone-Array Ego-Noise Reduction Algorithms for Auditory Micro Aerial Vehicles[END_REF], [START_REF] Wang | Microphone-Array Ego-Noise Reduction Algorithms for Auditory Micro Aerial Vehicles[END_REF]. Many SSL approaches developed in recent years for robotics are different variations of the MUltiple SIgnal Classification (MUSIC) algorithm. For instance, [START_REF] Okutani | Outdoor Auditory Scene Analysis Using a Moving Microphone Array Embedded in a Quadcopter[END_REF] presents an incremental version of the generalized eigenvalue decomposition (GEVD)-MUSIC algorithm [START_REF] Nakamura | Intelligent Sound Source Localization for Dynamic Environments[END_REF] referred to as iGEVD-MUSIC. The method is showed to perform better than the original GEVD version at a lower computational cost in an outdoor scenario. In [START_REF] Ohata | Improvement in Outdoor Sound Source Detection Using a Quadrotor-Embedded Microphone Array[END_REF], it is claimed that using generalized singular value decomposition (GSVD) instead of GEVD in MUSIC is more efficient.

The proposed incremental GSVD-MUSIC (iGSVD) method shows improved performance compared to iGEVD. Additionally, a correlation matrix scaling (CMS) is applied to further improve results. Alternatively to MUSIC, Generalized Cross Correlation (GCC) methods are used for robot SSL in [START_REF] Basiri | Robust acoustic source localization of emergency signals from micro air vehicles[END_REF] and in the general framework ManyEars [START_REF] Grondin | The ManyEars open framework[END_REF].

In [START_REF] Furukawa | Noise Correlation Matrix Estimation for Improving Sound Source Localization by Multirotor UAV[END_REF], a UAV-embedded SSL method using both prerecorded and on-flight propeller speed data is proposed. These data are used to estimate an adaptive noise correlation matrix in a Gaussian process regression model. This matrix is then used in GEVD-MUSIC to improve robustness to noise. As a result, the proposed method outperforms all comparison methods especially in high SNR conditions. In the same spirit, [START_REF] Morito | Partially Shared Deep Neural Network in Sound Source Separation and Identification Using UAV-Embedded Microphone Array[END_REF] presents a Deep Neural Network (DNN) approach to UAV-embedded SSL. To overcome the large training data requirements of DNNs, a partially shared network learning multiple tasks at the same time is implemented.

The authors of [START_REF] Wang | Microphone-Array Ego-Noise Reduction Algorithms for Auditory Micro Aerial Vehicles[END_REF] compare beamforming, blind source separation (BSS) and time-frequency filtering algorithms on UAV-embedded recordings. The recordings are made with a circular microphone array placed over a UAV fixed on a tripod. The method requires the position of the sound source to be known beforehand, as opposed to the SSL scenario addressed here. BSS for UAV ego-noise reduciton is also investigated in [START_REF] Wang | Ear in the sky: Ego-noise reduction for auditory micro aerial vehicles[END_REF]. In [START_REF] Hoshiba | Design of UAV-Embedded Microphone Array System for Sound Source Localization in Outdoor Environments[END_REF], two different UAV microphone array designs are proposed and the SEVD-MUSIC and iGSVD-MUSIC algorithms are compared on an outdoor SSL scenario. SSL success rates of almost 100 % are obtained even in low SNR conditions (< 0 dB). The authors propose to adapt the algorithms depending on the considered scenario and emphasize their high computational costs as a major drawback for real-time applicability.

II. THE DREGON DATASET

This section reports on the hardware and protocols used to gather the data and on the detailed content of the dataset. A summary of the dataset content can be found in Table I.

A. Hardware and Recording Environments

A quadrotor UAV MK-Quadro from MikroKopter (HiSystems GmbH, Moormerland, Germany) (see Fig. 1) was employed. This customizable quadrotor is equipped with four MK2832-35 motors 1 . The usual MK-Quadro setup was extended with an ODROID-XU4 Linux Computer (Hardkernel co., Ltd., GyeongGi, South Korea) running ROS and the TeleKyb-genom3 framework [START_REF] Grabe | The telekyb framework for a modular and extendible ros-based quadrotor control[END_REF] for implementing the low-level flight control receiving the body-frame velocity commands [START_REF] Blandin | Multi-source tdoa estimation in reverberant audio using angular spectra and clustering[END_REF], and for exchanging data via Wi-Fi with the ground station. For the sound recordings, a specifically designed 3-D printed cube-shaped microphone array was mounted under the UAV. This placing was motivated by the assumption that a potential target source would be located under the UAV, enabling spatial filtering of ego-noise. Further, this location partially protects the microphone array from possible shocks and from the wind generated by the rotors. The array contains eight microphones (one on each vertex, see Fig. 1) and is connected to a sound card 2 mounted on the drone via a 3D-printed case. All audio recordings have a sampling frequency of 44.1 kHz. The total weight of the equipped quadrotor is 1.68 kg. The propellers' speed ranges from 15 turns/s when they starts, up to around 95 turns/s at maximum. The UAV was flown in two different carpeted indoor rooms with respective volumes 10 m×10 m×2.5 m and 12 m×12 m×3.5 m and reverberation time under 150 ms. A low reverberation time was chosen for this dataset because typical search and rescue scenarios occur in outdoor environments, where little reverberation is present. Both rooms were equipped with a twelve-camera Vicon motion capture system 3 (see Fig. 2). On top of audio and Vicon data, log signals obtained from the embedded inertial measurement unit (3D anguar velocities and accelerations) and the four propellers (commanded and feedback speed of each of the 4 propellers) are provided in the dataset. The log and Vicon signals are synchronized together via timestamps provided by the robotic middleware ROS. Synchronized timestamps were then manually created for each audio sample based on motors onsets and outsets. Maximal synchronization errors are estimated to be below 100 ms. All flights were videorecorded by a wide-angle camera mounted on a tripod for reference.

-15 • 0 • -45 • -30 • -15 • 0 • Log signals Elevation -45 • -30 • -15 • 0 • -45 • -30 • -15 • 0 • Vicon motion

B. In-flight Source Recordings

Four in-flight recordings of a single static source were performed in the largest room. The UAV was tele-operated by a human and was performing free-flights combining sequences of take-off, landing, stabilization, hovering, straight line, circles and spinning in a realistic way. The source was simulated by a Genelec loudspeaker placed on a table in the center and emitting either random speech utterances from the TIMIT dataset [START_REF] Garofolo | Timit Acoustic-Phonetic Continuous Speech Corpus LDC93S1[END_REF] or white noise at two different levels, achieving recorded audio signal-to-noise ratios (SNRs) ranging from -16 to -10 dB. The Vicon motion capture system was used to obtain the 6 DoF coordinates of both the UAV and the loudspeaker, thanks to trackers placed on each of them. These coordinates were notably used to calculate the ground truth 3D positions of the source in the UAV's frame, as showed in Fig. 5. On top of the four recordings obtained from real UAV flights, one recording of a "silent flight" with a white noise source was made. The UAV motors were off and a human operator was carrying the UAV to simulate a flight. This recording also includes 10s of "room silence", for reference.

C. In-flight Noise-Only Recordings

In order to isolate the noise characteristics of UAVembedded audio signals, a number of recordings were made with the UAV flying in the absence of a sound source. Different types of flights were performed: hovering, upand-down, spinning clockwise (cw) and counterclockwise (ccw), rectangle and free flight. While free flights were performed in both rooms, the other types were performed in Ego-noise the smallest room only. Hovering simply describes taking off and holding position, while up-and-down includes multiple vertical movements. For spinning cw and ccw the drone was taking off, holding position and spinning around the yaw angle in both directions. The fourth type of flight performed was a rectangle, where the drone takes off and flies in a horizontal rectangular shape around the room.

In addition to flight noise, other recordings were made while the UAV was fixed on a nylon string hanging freely in the room and held still by a human operator. This was done in order to record pure motor noise at a chosen speed without having the influence of balancing and stabilizing movements. Each individual motor and the four of them jointly were hence recorded for 10s at speeds of 50, 60, 70, 80 and 90 turns/s. An interesting feature of these recordings is that they contain very few wind noise.

D. Noise-Free Recordings

While testing SSL methods on realistic flight scenarios is necessary, it is also desirable to test them on more controlled settings. To this aim, the DREGON dataset includes a number of noise-free recordings of a target sound source using the microphone array of previous experiments. These recordings can be conveniently summed together and with the noise-only recordings of previous section to simulate noisy audio excerpts of any length involving any number of static sources from any fixed directions and at any SNR. They were performed in a third semi-anechoic and acoustically isolated room in order to have as little interference as possible. The microphone array was fixed on a tripod while a loudspeaker was moved to different elevation and azimuth angles and two different distances to the array. From a 1.2 m distance we used all combinations of azimuth angles {-45 • , -30 • , -15 • , 0 • } and elevation angles {-30 • , -15 • , 0 • } yielding 12 different directions. From a 2.4 m distance an additional elevation angle of -45 • was added yielding 16 different positions. Note that thanks to the rotational symmetries of the designed microphone array, over 100 source directions around the whole sphere can be conveniently simulated using these directions, by appropriately permuting the 8 channels. For each direction, a white noise sound, a chirp and random speech utterances from the TIMIT dataset [START_REF] Garofolo | Timit Acoustic-Phonetic Continuous Speech Corpus LDC93S1[END_REF] were recorded. This resulted in a total of 84 annotated clean source signals. The signal durations were 10 s for chirps, 2 s for white noise sounds and 14s in average for speech signals.

III. ACOUSTIC ANALYSIS OF UAV NOISE

The acoustic noise at microphones during UAV flights consists of several parts. As pointed out in [START_REF] Wang | Ear in the sky: Ego-noise reduction for auditory micro aerial vehicles[END_REF], three main components can be identified: mechanical noise (ego-noise), air flow noise created by the propellers and wind noise. Fig. 3 shows a noise spectrogram obtained from one of the microphones during a free flight, where these three components are highlighted. As can be seen, most of the noise's energy is located in frequencies below 3 kHz. This is particularly problematic when the target source signal is human voice, which usually covers a similar frequency range. Among the different components, wind noise, located below 1 kHz, is by far the loudest one when present. However, we observed that wind noise was rarely present in more than 4 out of 8 channels at once. This deserves further investigation and could be exploited by removing corrupted channels using a wind detection method [START_REF] Schmidt | Wind Noise Reduction using Non-Negative Sparse Coding[END_REF].

In general it can be observed that the noise contains many non-stationary components, making its modelling nonstraightforward. However there seems to exist an overall background component which is roughly stationary during most of the flight. Our hypothesis is that this component is generated mainly by the motor rotations and vibrations of the structure. Additionally, Fig. 3 shows the existence of several spectral harmonic components. To investigate this further, we used individual motor recordings of the DREGON dataset (Section II-C). In Fig. 3 it can be seen how a motor speed of 80 turns/s leads to peaks in the power spectral density along the harmonics proportional to 80 Hz. This observation paves the way for methods exploiting instantaneous motor speed logs to cancel these peaks. While these harmonic components are highly non-stationary whenever the drone undergoes rapid changes of directions, it still seems rather reasonable to assume that the noise is approximately stationary during the most steady parts of the flight (with exception to wind components). Based on this approximation, a stationary filtering technique described in Section IV-B is used to reduce noise in the SSL experiments of Section V.

IV. BASELINE SOUND LOCALIZATION METHODS

In this section some baseline sound localization methods are briefly outlined. These methods are tested on the dataset and results are reported in Section V.

A. Sound Source Localization

The most widely used feature to localize a sound source is the pairwise time difference of arrival (TDOA) of the source signal at two microphones. A number of methods exist to estimate TDOAs and directions of arrival from recorded signals. In this study, we focus on two such methods which are widely used in SSL, namely the generalized crosscorrelation phase-transform (GCC-PHAT) method [START_REF] Knapp | The generalized correlation method for estimation of time delay[END_REF], [START_REF] Aarabi | Self-localizing dynamic microphone arrays[END_REF] and the multiple signal classification (MUSIC) method [START_REF] Schmidt | Multiple Emitter Location and Signal Parameter Estimation[END_REF], [START_REF] Argentieri | Broadband variations of the music highresolution method for sound source localization in robotics[END_REF]. GCC-PHAT as described in e.g., [START_REF] Aarabi | Self-localizing dynamic microphone arrays[END_REF], [START_REF] Blandin | Multi-source tdoa estimation in reverberant audio using angular spectra and clustering[END_REF], [START_REF] Grondin | The ManyEars open framework[END_REF], takes as input a pair of time-domain audio signals, and a set of directions of arrival (DOA) to be tested. A DOA is represented by a single angle from which the source signal impinges at the microphone pair, assuming the source is in the far field. Assuming further that sound propagates in free field from the source to each microphone, a one-to-one mapping exists between DOAs and TDOAs. Based on these assumptions, the methods output a score for each tested DOA, referred to as the angular spectrum for this microphone pair [START_REF] Blandin | Multi-source tdoa estimation in reverberant audio using angular spectra and clustering[END_REF]. The Multi-Channel BSS Locate Matlab toolbox 4 (mBSSL) implements a general framework that efficiently combines estimated angular spectra from multiple microphone pairs to estimate the 2D direction (azimuth, elevation) of one or several sound sources, given the array geometry. It can be viewed as a computationally efficient variation of SRP-PHAT [START_REF] Dibiase | Robust localization in reverberant rooms[END_REF]. For convenience, we refer to mBSSL used with GCC-PHAT angular spectra as simply GCC-PHAT in the following.

The method MUSIC [START_REF] Schmidt | Multiple Emitter Location and Signal Parameter Estimation[END_REF], [START_REF] Argentieri | Broadband variations of the music highresolution method for sound source localization in robotics[END_REF] is a so-called subspace method known to be more robust to noise than GCC-PHAT. It also outputs an angular spectrum given a set of test DOAs. However, MUSIC is most effective when using an array containing more than two microphones. Because the array used in DREGON is non-linear, angular spectra estimated for this whole array must cover a two-dimensional DOA grid search (azimuth, elevation). For this reason, our implementation of MUSIC is much more computationally demanding than our implementation of GCC-PHAT, which benefits from mBSSL to efficiently aggregate one-dimensional angular estimates. Several implementations of MUSIC exist (see, e.g., [START_REF] Rascon | Localization of sound sources in robotics: A review[END_REF]). The one used in this study relies on the eigenvalue decomposition (EVD) of the frequency-domain covariance matrix of observed 8-channel signals, and is referred to as EVD-MUSIC.

B. Multichannel Wiener Filtering

The multichannel Wiener filter (MWF) is a classical signal processing technique which can be used to reduce stationary noise from a multichannel signal, see e.g. [START_REF] Löllmann | Challenges in acoustic signal enhancement for human-robot communication[END_REF]. Based on the observation that UAV-embedded noise signals are roughly stationary during steady parts of the flights (see Section III), we investigated the use of MWF as a pre-propressing step to the source localization methods described in previous section. Let

X( f ,t) = S( f ,t) + N( f ,t) (1)
where X( f ,t), S( f ,t) and N( f ,t) in C M respectively denote the M-channel observed, image-source and noise signals in the short-time Fourier domain and ( f ,t) is the frequencytime index. The MWF estimate of the source signal is given by Ŝ( f ,t) = W MW F ( f )X( f ,t) where

W MW F = R XX ( f ) -1 (R XX ( f ) -R NN ( f )) (2)
denotes the optimal Wiener filter and R XX ( f ), R NN ( f ) ∈ C M×M respectively denote the observed and the noise covariance matrices at frequency f . In practice, R XX ( f ) and R NN ( f ) cannot be computed exactly and are replaced by their sample estimates. RNN ( f ) is pre-computed using a noise-only signal of sufficient length, and RXX ( f ) is directly estimated from multichannel input data. This latter estimation can be done recursively in an efficient way for real-time applications. See [START_REF] Löllmann | Challenges in acoustic signal enhancement for human-robot communication[END_REF] for more details.

An 8-channel Wiener filter was applied to the recordings of the DREGON dataset as a pre-prossing step to GCC-PHAT for robust SSL, yielding the WF+GCC-PHAT method. Pre-processing EVD-MUSIC via Wiener filtering is in fact strictly equivalent to the so-called GEVD-MUSIC method [START_REF] Argentieri | Broadband variations of the music highresolution method for sound source localization in robotics[END_REF], which was hence also used in our experiments.

V. EXPERIMENTS AND RESULTS

In this section, the sound localization methods described in previous section are applied to the DREGON dataset and their results are compared and discussed. In all the following TABLE II: Static sound source localization of 500ms signals at 0 dB SNR. The results are displayed in the format avg ± std (fail%) where avg and std denote the mean and standard deviations of the mean azimuth and elevation absolute angular errors in degree for successful localizations only (< 10 • error), while fail denotes the percentage of failures.

A. Static-source localization

The first experiments use combinations of noise-free static source recordings (Sec. II-D) and noise-only in-flight recordings (Sec. II-C). We limited our source signals to speech emitted from a 2.4 m distance. For each test, a speech segment from a given direction and a noise segments are randomly selected and are weighted and summed to form a test mixture of desired length and SNR. Note that the speech signals used in our experiments are such that they do not contain significant pauses. The signal used to pre-compute the fixed noise covariance matrix of MWF and GEVD-MUSIC (See Sec. IV-B) is the sum of all four individual motor noise signals running at 80 turns/s. Indeed, flight logs revealed that this was the most commonly reached speed in practice.

First, we compare the 4 baseline methods GCC-PHAT, WF+GCC-PHAT, SEVD-MUSIC and GEVD-MUSIC on different flight scenarios. The analysis window, i.e., the duration of every test signal, is fixed to 500 ms. The SNR is fixed to 0 dB in all tests. For every source direction, 40 mixtures are generated from randomly chosen segments, resulting in 640 tests per scenario. For both azimuth and elevation angles, three evaluation metrics are considered: the percentage of failures, defined as localization error larger than 10 • , and the average and standard deviation of the absolute angular errors of successful estimates. The failure cases are removed from average and standard deviations in order to avoid a too strong influence of outliers. The results are showed in Table II. It can be seen that the mean absolute error of successful localizations is roughly between 2 • and 3 • for all methods and do not show a high variance, suggesting that all methods share a similar maximal angular resolution. A major difference can be found in the amount of failures. The methods with no noise pre-processing reach failure rates up to 28 % for GCC-PHAT in free-flight and 50 % in SEVD-MUSIC when the UAV spins. On the other hand, GEVD-MUSIC have less then 4 % and WF + GCC-PHAT less than 1 % outliers in all scenarios. These results are very promising. While WF+GCC-PHAT and GEVD-MUSIC show comparable performance, the former is to be preferred. Indeed, it achieved computational times that would easily allow real-time implementations with propper optimization (about 1.5s to process 1s of signal on average), while our implementations of MUSIC-based methods were about twenty times slower using the same hardware, for the reasons detailed in Section IV-A. Because of this computational bottleneck, the following experiments will focus on GCC-PHAT.

In a second experiment, the influence of analysis-window sizes and SNR levels on the GCC-PHAT and WF+GCC-PHAT methods is analyzed. The analysis window sizes varies from 100, 200, 500 and 1000 ms, and the tested SNR values vary from -25 dB to +30 dB in 5 dB steps. Hence, 28 different test conditions are created. To evaluate these, we calculate the root-mean-squared-error (RMSE) of estimated directions. Additionally, an SSL result is counted as a success only if both azimuth and elevation errors are lower than 5 • , and the success rates are displayed. As can be seen in Fig 4, Wiener pre-processing significantly boosts GCC-PHAT performance in all test conditions. The figure also reveals that the analysis window size has a strong influence on localization accuracy, larger windows systematically implying better performance. At -5 dB, a 96 % success rate is achieved using 1000 ms windows, while this falls to 84 % for 500 ms windows. Nevertheless, this experiment suggests that 500 ms is a good compromise between real-time applicability and accuracy, with a near 100 % success rate for all positive SNRs, and a relatively small decrease in performance compared to 1000 ms. In general, the experiments of this section show that for moderate SNRs (> 0 dB), stationary pre-filtering is sufficient to obtain satisfying localization performance.

B. In-flight source localization

In this section, the best performing method WF+GCC-PHAT is used on the more realistic scenarios where the UAV flies freely in the presence of a target white noise or speech source, whose range and direction are hence constantly varying in the UAV's frame. A 500 ms sliding analysis window is used over the entire recordings. The Wiener filter noise covariance matrix is estimated from a 7s noise-only recording in free-flight. For the speech scenario, the 3 channels containing heaviest wind-noise are manually removed and the search grid does not includes elevations above 20 • to avoid spurious localizations. Fig. 5 displays estimated trajectories against ground truth trajectories of azimuth and elevation angles. As can be seen, the method achieves outstanding performance in white-noise source localization, despite an average SNR of -12dB over this run. With exception of a few outliers at take off and landing, near-perfect localization is achieved with errors under 2 • in both azimuth and elevation for all test frames. However, we observe that performance drastically drops when a speech is used instead, despite a similar SNR of -13dB. This is consistent with the experiments of previous section, and can be explained by the fact that the energy of speech and UAV noise signals is mainly located in a similar frequency range. These results confirm that more accurate noise models should be developed to tackled speech localization under heavy noise for search and rescue with UAVs. Additional modalities such as motor speed and inertial measurement data could be leveraged for this aim.

VI. CONCLUSION

This paper presented DREGON, a novel state-of-the-art dataset that aims at pushing forward research in embedded sound source localization with a UAV for search and rescue. The dataset consists of noise-free, noise-only and in-flight audio recordings of a target source emitting speech or white noise signals. The position of the target source in the UAV's frame is precisely annotated in all recordings using a motion capture system. Additional modalities such as inertial measurements and motor speeds acquired by sensors embedded in the UAV as well as video recordings of all flight are included. The dataset as well as a number of baseline sound localization methods applicable to the data are made freely available online for the research community. Our preliminary investigation on the dataset revealed a number of insights on the problem at hand. Notably, the acoustic characteristics of UAV noise make speech localization particularly challenging in very low SNRs. This calls for the development of refined source and noise models. Promising directions are the use of wind noise reduction methods [START_REF] Schmidt | Wind Noise Reduction using Non-Negative Sparse Coding[END_REF] and the use of structured models [START_REF] Tezuka | Ego-motion noise suppression for robots based on Semi-Blind Infinite Non-negative Matrix Factorization[END_REF] or extra modalities to predict ego-noise [START_REF] Furukawa | Noise Correlation Matrix Estimation for Improving Sound Source Localization by Multirotor UAV[END_REF], [START_REF] Schmidt | Ego-Noise Reduction Using a Motor Data-Guided Multichannel Dictionary[END_REF].

VII. ACKNOWLEDGEMENT

Fig. 1 :

 1 Fig. 1: The MikroKopter quadrotor UAV used for the DREGON dataset, with our 3-D printed 8-channel microphone array mounted on the bottom. Green circles highlight two of the passive markers used for motion capture and yellow circles highlight two of the microphones (best seen in colors).

  capture data Video recordings Individual Motor Recordings In-Flight Source Recordings White noise (loud), speech (loud), Commanded speed [turns/s] 50 60 70 80 90 white noise (less loud), speech (less loud), "silent flight" + white noise. Log signals Motors Individual motors, all four at 70 turns/s Vicon motion capture data Video recordings

Fig. 2 :

 2 Fig. 2: Setup of the room were in-flight source recordings (Sec. II-B) were performed. The flying UAV is highlighted by a green circle, the loudspeaker simulating the source by a red circle, and two of the twelve Vicon cameras by yellow circles (best seen in colors).

Fig. 3 :

 3 Fig. 3: (a) Log-magnitude spectrogram recorded by one embedded microphone during free flight. In A harmonic components, in B structural noise and in C wind noise can be observed. (b) Mean power spectrum of the sound generated by an individual motor rotating at 80 turns/s. Peaks at harmonics proportional to 80 Hz can clearly be identified.

Fig. 4 :Fig. 5 :

 45 Fig. 4: (a) Sound localization RMSE in various SNR conditions for different analysis window sizes. Dashed lines correspond to vanilla GCC-PHAT while solid lines shows results with an additional Wiener-filter pre-processing. (b) Same as (a) but for the success ratio, were a localization is considered successful if both azimuth and elevation errors are less than 10 • .

TABLE I :

 I The DREGON dataset: content summary.

	DREGON dataset
	Noise-Free Source signals	In Flight Noise-Only Recordings

Type Chirp, white noise, speech (semi-anechoic room) Hovering, up and down, rectangle, Distance 1.2 m 2.4 m spinning cw and ccw (smallest room), free flight (both rooms) Azimuth -30 •

  ± 1.7 • (27.8%) 2.41 • ± 1.5 • (12.2%) 2.45 • ± 1.5 • (7.81%) 2.36 • ± 1.6 • (14.2%) 2.52 • ± 1.6 • (12.5%) WF + GCC-PHAT 2.45 • ± 1.8 • (1.09%) 2.31 • ± 1.5 • (0.00%) 2.36 • ± 1.6 • (0.16%) 2.35 • ± 1.6 • (0.16%) 2.41 • ± 1.6 • (0.47%) SEVD-MUSIC 3.01 • ± 2.4 • (13.6%) 3.00 • ± 2.4 • (13.4%) 3.04 • ± 2.3 • (23.6%) 3.03 • ± 2.4 • (49.8%) 2.77 • ± 2.4 • (31.6%) GEVD-MUSIC 1.98 • ± 1.9 • (0.16%) 2.05 • ± 1.9 • (3.59%) 1.93 • ± 1.9 • (2.19%) 1.93 • ± 1.9 • (2.50%) 1.84 • ± 1.8 • experiments, audio signals are downsampled to 16 kHz and the short time Fourier window is fixed to 64 ms with 50 % overlap. The global angular search space has a resolution of 5 • in both azimuth and elevation.

	Methods / Scenarios	free flight	hovering	rectangle	spin cw and ccw	up and down
	GCC-PHAT	2.43

•

http://wiki.mikrokopter.de/MK2832-35

http://bass-db.gforge.inria.fr/bss_locate/

The authors are thankful to Franc ¸ois Bodin, Guillermo Andrade-Barroso and Vincent Drevelle of IRISA (Rennes, France) for their help with the used to build this dataset. The authors are also thankful to Lupinenweg. This study has been partially funded by the Fondation Rennes 1.


Anthony Fleury
email: anthony.fleury@imt-lille-douai.fr

Edwin Lughofer
email: edwin.lughofer@jku.at

Moamar Sayed

Optimization in Dynamic Environments

published or not. The documents may come   L'archive ouverte pluridisciplinaire

During the last decade, the modeling based on learning in non-stationary environments has received more and more attention in the machine learning and data mining communities. This is because the requirements from practical (industrial) side induced a permanent paradigm shift from classical batch off-line learning procedures (as conducted throughout the 80ties and 90ties to build up accurate and high-performance machine learning models) to incremental, adaptive algorithms which are able to evolve the model structures, architectures and parameters fully autonomously and on the fly, typical in a single-pass manner. It is especially the case within the scope of dynamic data streaming context, changing environmental conditions or as part of large-scale problems, e.g. web mining, multi-sensor networks, sequential video analysis, Big Data or predictive maintenance in factories for the future (FoF) as part of the Industry 4.0 activities (even an important standpoint of the Horizon 2020 objective programme see https://ec.europa.eu/programmes/horizon2020/en/h2020-sections-projects). The necessity of such algorithms is also underlined by the increasing size of the databases and storages, which induces that conventional batch learning systems cannot be applied within a reasonable time frame and sufficient accuracy. Thus, a block-wise loading of the data has to be carried out, which again requires the usage of incremental and evolving learning algorithms.

Thereby the learning algorithms are either guided by a permanent update of model parameters and structures, leading to a so-called "any-time up-to-date" experience, or by occasional update cycles which are often explicitly selected by active and semi-supervised learning methodologies in order to keep the measurements or feedback efforts of operators at a lowest possible level (cost reduction).

The aim of this special issue is to bring recent developments in the field of adaptive and intelligent systems in non-stationary environments to a broader audience and readership, whereas the particular focus lies not only on the (adaptive, incremental) learning aspects, but also on control and optimization aspects. The two latter has been only marginally addressed so far in literature, especially optimization that is an issue which in its classical form requires multiple iterations over batch data sets in order to assure convergence. To be able perform optimization during incrementally loaded streaming data appropriately (also terms dynamic optimization), in order to not forget and outweigh older learning solutions too intensively, is still an open central challenge. Control aims at ensuring that systems execute specified or desired behavior in order to perform specific tasks for which they were designed. Because of several external perturbations as uncertainty, erroneous or incomplete measurements, unpredicted faults or the emergence of new operation modes, control strategies require to be adapted over time in order to keep the system executing desired or safe behavior and protecting it from unforseen or dangerous behavior.

Drifts in the data streams are frequently occurring, which make older relations, dependencies between input and output data (target values), adaptively learnt into the models, obsolete. Thus the model update mechanism, no matter whether used for control or optimization, has to embed some technology for an appropriate handling of drifts, such as to increase the flexibility of the models by outweighing older learned relations and concentrate more on the new ones. A comprehensive survey about available drift handling methods in literature, including the definitions of various types of drifts, detection of drifts and appropriate reactions to them, is included in this special issue.

For this issue, we received 6 submissions, all from invited authors, for their expertise and especially from getting in contact with them at the IEEE Evolving and Adaptive Intelligent Systems (IEEE EAIS) conference 2015, which have been organized by the guest editors of this special issue (Moamar Sayed-Mouchaweh being the main conference chair, Anthony Fleury the local arrangement chair and Edwin Lughofer the publication chair). Finally, 4 of them could be accepted after a complete review process and then included in this special issue.

In what follows, we provide a short overview of the basic concepts and algorithms to be found in each of these papers:

The paper "Discussion and Review on Evolving Data Streams and Concept Drift Adapting" (Paper #1, EVOS-D-16-00029) by the authors Imen Khamassi, Moamar Sayed-Mouchaweh, Moez Hammami and Khaled Ghédira comprehensively discusses several methods for detecting drifts and appropriately handling them during further model updates. It thereby exploits various concepts from literature, puts them into a red line and provides a nice and round overall picture about the current state-of-the-art in drift handling methods in data streams. As mentioned by the authors "The main objective of the survey is to present an ease understanding of the concept drift issues and related works, in order to help researchers from different disciplines to consider concept drift handling in their applications." It is thus placed as the first paper in this special issue, as any incremental, single-pass streaming learning approach (as discussed in the subsequent papers) can make use of the basic concepts.

The paper "A Dedicated State Space for Power System Modeling and Frequency and Unbalance Estimation" (Paper #2, EVOS-D-16-00030) by the authors Anh Tuan Phan, Patrice Wira and Gilles

Hermann presents a state-space representation to model power systems like electrical distribution systems. The proposed model is able to take into account all the dynamic behavior of a multiphase power system and employs an extended recursive Kalman Filter (eKF) in order to update the power quality parameters. The proposed estimation technique is thus an adaptive method able to handle the changing power system, and it is also evolving because new state-spaces may be adjoined in the eKF. Its effectiveness has been evaluated by several tests. This also has an impact on the control law embedded in the operating principles of a shunt active power filter for power quality enhancement. The new approach is successfully applied to estimate the fundamental frequency of a power system under different conditions, where it showed significantly improved results to a related well-known method.

The paper "A fuzzy approach towards inductive transfer and human-machine interface control design" (Paper #3, EVOS-D-16-00032) by the authors Rui Azevedo Antunes, Luis Brito Palma, Fernando Coito and Herminio Duarteramos proposes a new fuzzy clustering methodology to the development of a real-time inductive transfer embedded controller. It is used for improving the operator's proficiency, under a human-in-the-loop environment relying on visual feedback information. Thus the paper on the hand contributes to the field of adaptive, evolving controllers, but on the other hand also to human-computer interaction within a dynamic environments. Thereby, the operator behavior is modelled and enhanced from a human-machine interface fuzzy classifier and assisting scheme, which uses (high speed) real-time streaming data and additional information collected from an expert user. Experimental tests were performed by different participants under a driving simulator; the new approach confirmed to significantly improve the actual driving skills of the human operators.

The paper "An Artificial Immune System for Offline Isolated Handwritten Arabic Character

Recognition" (Paper #4, EVOS-D-16-00015) by the authors Chaouki Boufenar, Mohamed Batouche and Marc Schönauer investigates the applicability of a rule-based machine learning system, inspired by the principles and processes of Artificial Immune System (AIS), in isolated handwritten arabic characters, which is embedded in a dynamic environments. The developed system is composed of three main modules: preprocessing, feature extraction and recognition, and embeds some dynamic aspects in terms of memory cells generation within AIS. The obtained results of the proposed system are promising with a classification rate of 93.25% and typically outperform most well-known classifiers from Scikit Learn Library.

At the end, we hope that this special issue sheds light on some novel ideas of dynamic learning, control and optimization aspects in non-stationary environments. In particular, we would like to gratefully acknowledge and sincerely thank all the reviewers for their insightful comments and criticism of the manuscripts. Our thanks are also dedicated to the authors for their contributions and collaboration. Finally, we are grateful to the Editor-in-chief Plamen Angelov and the editorial office assistants of Evolving Systems for their suggestive and insightful editorial comments and support during the review and production process of this special issue. The second guest editor also acknowledges the support of the Austrian COMET-K2 programme of the Linz Center of Mechatronics (LCM), funded by the Austrian federal government and the federal state of Upper Austria.

Anthony Fleury Edwin Lughofer Moamar Sayed-Mouchaweh


M'hamed Bilal Abidine

Belkacem Fergani

Anthony Fleury

Integrating prior Knowledge in Weighted SVM for Human Activity Recognition in Smart Home

Keywords: Activity recognition, Feature extraction, PCA, LDA, Weighted SVM

Feature extraction and classification are two key steps for activity recognition in a smart home environment. In this work, we performed a new hybrid model using Temporal or Spatial Features (TF or SF) with the PCA-LDA-WSVM classifier. The last method combines two methods for feature extraction: Principal Component Analysis (PCA), and Linear Discriminant Analysis (LDA) followed by Weighted SVM Classifier. This classifier is used to handle the problem of imbalanced activity data from sensor readings. The experiments that were implemented on multiple real-world datasets, showed the effectiveness of TF and SF attributes combined with PCA-LDA-WSVM in activity recognition.

Introduction

Several classification algorithms have been employed for Human Activity Recognition (HAR) tasks [START_REF] Tapia | Activity recognition in the home using simple and ubiquitous sensors[END_REF][START_REF] Logan | A long-term evaluation of sensing modalities for activity recognition[END_REF][START_REF] Ordonez | Activity recognition using hybrid generative/discriminative models on home environments using binary sensors[END_REF][START_REF] Abidine | Improving Human Activity Recognition in Smart Homes[END_REF] to automatically recognize activities in intelligent manner about the occupants and ensure the comfort of older adults in smart home using sensor networks. In [START_REF] Abidine | The joint use of sequence features combination and modified weighted SVM for improving daily activity recognition[END_REF], we have developed a new classification method named PCA-LDA-WSVM based on a combination of Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA) and the modified Weighted Support Vector Machines (WSVM). We demonstrated the ability of this method to achieve good improvement over the standard used methods such as HMM, CRF, SVM, WSVM.

Classifiers address the challenge of extracting information from raw sensor data through the use of features. In this paper, we wanted to improve the classification performances of the approach in [START_REF] Abidine | The joint use of sequence features combination and modified weighted SVM for improving daily activity recognition[END_REF] by introducing prior knowledge [START_REF] Fleury | Improving Supervised Classification of Activities of Daily Activities of Daily Living Using Prior Knowledge[END_REF]. The 'Prepare breakfast' and 'Prepare dinner' activities share the same model as they involve the same set of object interactions. These two activities are distinguished by time of taking place, i.e. 'Prepare breakfast' takes place in the morning hours and "Prepare dinner" takes place in the afternoon or evening hours of the day. The location attribute can also discriminate between two different activity classes as 'Toileting' and 'Showering' that performed in two different locations.

2 The Proposed HAR system by introducing the prior knowledge

Overview

The core idea of proposed method is as follows: A dataset is divided into training and testing sets. Having defined the activities to recognize and the list of potentially interesting features, we added both temporal and spatial features (TF and SF). We then extract the features that proved to be the most useful for activity recognition. These two sets are transformed independently with PCA and LDA methods. By adding those PCF it is possible to have more than the number of classes minus one extracted features by LDA. Then WSVM method, as the latter process as follow.

Feature Representation

Sensors outputs are binary and represented in a feature space which is used by the model to recognize the activities performed. The raw data obtained from the sensors can either be used directly, or be transformed into a different representation forms (Fig. 2). We do not use the raw sensor data representation as observations; instead we use the "Change point" and "Last" representations which have been shown to give much better results in activity recognition [START_REF] Kasteren | Accurate activity recognition in a home setting[END_REF].

Feature insertion

In this work, we improve the classification performances of class activities by introducing the feature insertion stage. We added two new features to the existent data matrix. The first attribute corresponds to the hour of beginning of the activity. We extract this feature directly from the data structure. The sensor activations are collected by the state-change sensors distributed all around the environment. To find out the second feature corresponding to the room label of performed activity, we search the different objects he is manipulating in the sensors, see the below figure. ID: is a number representing the sensor ID. Each sensor has its own unique ID.

Weighted Support Vector Machines Classification (WSVM)

Osuna et al [START_REF] Osuna | Support vector machines: training and applications[END_REF] proposed a Weighted SVM (WSVM) algorithm by introducing two different cost parameter - C and  C in SVM optimization problem for the minority (d i =-1) and majority classes (d i =+1), as follow

l i b y d C C s s i i i i l i i l i ζ b, s, i i ,..., 1 , 0 , 1 ) ) ( s ( to subject 2 1 min 1 d | 1 d | i                        (1)  l (resp. 
l ) the number of positive (resp. negative) instances in the database. Solving the formulation dual of WSVM [START_REF] Abidine | The joint use of sequence features combination and modified weighted SVM for improving daily activity recognition[END_REF] gives a decision function for classifying a test point

q p R y             sv l 1 i i i i b x x K d α x ) , ( sgn ) ( f (2)
We used the Gaussian kernel as follows:

  2 2 2 / exp ) , (     y x y x K
. Huang et al. [START_REF] Huang | Weighted support vector machine for classification with uneven training class sizes[END_REF] raised a Weighted SVM algorithm. The cost coefficients are typically chosen as:

   l l C C - - (3)
To extend Weighted SVM to the multi-class scenario in order to deal with N classes (daily activities), we used different misclassification C i per class similar to [START_REF] Chen | Support vector machine soft margin classifiers: error analysis[END_REF]. By taking C -=C i and C + = C, with  l and i l be the number of samples of majority classes and number of samples in the i th class and C is the common cost parameter of the WSVM. The main ratio cost value C i for each activity can be obtained by:  )

/ ( round i l l C C i    N i ,..., 1  (4)
3 Simulation results and Assessment

Datasets

We used fully labeled datasets [START_REF] Tapia | Activity recognition in the home using simple and ubiquitous sensors[END_REF], [START_REF] Ordonez | Activity recognition using hybrid generative/discriminative models on home environments using binary sensors[END_REF], [START_REF] Kasteren | Accurate activity recognition in a home setting[END_REF] gathered by a single occupant from four houses having different layouts. We chose the ideal time slice length for discretizing the sensor data 60 t   seconds. We splitted the initial dataset into training and testing subsets using the 'leave one day out' approach, retaining one full day of sensor readings for testing and using the remaining sub-samples as training data.

Results

We optimized the SVM hyper-parameters (σ, C) for all training sets in the range [0.1-2] and {0.1, 1, 5, 10, 100}, respectively, to maximize the error rate of leaveoneday-out cross-validation technique. The number of features after extraction for PCA and LDA is mentioned in [START_REF] Abidine | The joint use of sequence features combination and modified weighted SVM for improving daily activity recognition[END_REF]. Then, for WSVM classification method, we optimized locally the cost parameter C i adapted to different classes. In table 1, the results show that the feature insertion set using either TF or SF contributes to significantly enhance the performance of PCA-LDA-WSVM classifier. One also notices that the TF is slightly better than SF for recognizing activities.

Table 1. Recall, Precision, F-measure and Accuracy results for all approaches in (%).

Bold values are the results for our approaches for each dataset.

We report in figure 4, the classification results in terms of accuracy measure for each class. In TK26M dataset, our proposed combinations outperforms the other approaches for 'Idle', 'Leaving', 'Toileting', 'Showering', 'Breakfast', 'Dinner' and 'Drink' activities. The majority activities are better for all methods over all datasets while the 'Idle' activity is more accurate for the proposed method compared to other methods. Additionally, the kitchen-related activities as 'Breakfast', 'Dinner' and 'Drink' are in general harder to recognize than other activities. In order to quantify the extent to which one class is harder to recognize than another one, we analyzed the confusion matrix of TF-PCA-LDA-WSVM for TK26M dataset in Table 2. One notices that the activities 'Leaving', 'Toileting', 'Showering', 'Sleeping', 'Dinner' and 'Drink' are better recognized comparatively with 'Idle' and 'Breakfast'. The kitchen activities seem to be more recognized using the proposed method combining TF with the PCA-LDA-WSVM classifier.

Given the considerations pointed out previously, the high performance obtained in the case of TK26M dataset, which seems to be less vulnerable to class-overlapping than others, as compared to other datasets. This overlapping between the activities is due to the layout of the house. In the TK26M House, there is a separate room for almost every activity. The kitchen activities are food-related tasks, they are worst recognized because most of the instances of these activities were performed in the same location (kitchen) using the same set of sensors. Therefore the location of sensors strongly influences recognition performance.

Conclusion

Our experiments on real-world datasets from smart home environment showed that the strategy (TF or SF)-PCA-LDA-WSVM can significantly increase the recognition performance to classify multiclass sensory data, and can improve the prediction of the minority activities. It significantly outperforms the results of the typical methods PCA-LDA-HMM and PCA-LDA-WSVM. TF-PCA-LDA-WSVM is slightly better than SF-PCA-LDA-WSVM. We added the space features needs a prior knowledge about the smart home, which makes a model very specific for that environment.

Figure 1 .

 1 Figure 1. Block diagram of the proposed activity recognition approach using the feature insertion. Att1 = TF, Att2 = SF.

Figure 2 .

 2 Figure 2. Different feature representations [8].

Figure 3 .

 3 Figure 3. TF and SF for TK26M dataset. In red, the hour of beginning activity.

Figure 4 .

 4 Figure 4. Accuracy recognition rate for each activity on TK26M dataset.

Table 2 .

 2 Confusion matrix (values in %) of activities for TF-PCA-LDA-WSVM on the TK26M dataset.

	Activity Id	Le	To	Sh	Sl	Br	Di	Dr
	Id	64.3 7.6	2.0	1.1	4.7	8.7	6.5	5.1
	Le	0.6	98.6 0.2	0.5	0.0	0.0	0.1	0.0
	To	7.8	5.2	83.2 2.1	0.7	0.6	0.1	0.3
	Sh	5.2	0.0	4.2	88.5 0.0	0.0	0.9	0.2
	Sl	0.3	0.3	0.4	0.5	98.5 0.0	0.0	0.0
	Br	16.0 0.0	0.9	0.2	0.6	63.4 11.6 7.3
	Di	5.0	0.7	0.3	0.0	0.5	2.6	83.4 3.5
	Dr	6.8	0.0	0.0	0.2	0.2	6.2	1.7	76.1