Hugo Lafaye De Micheaux
email: hugo.lafaye-de-micheaux@irstea.fr

Christophe Ducottet

Philippe Frey

Multi-model particle filter-based tracking with switching dynamical state to study bedload transport

Keywords: Visual object tracking, multiple targets tracking, particle filter, switching dynamical state, detector confidence, bedload transport

Multi-object tracking is a difficult problem underlying many computer vision applications. In this work, we focus on bedload sediment transport experiments in a turbulent flow were sediments are represented by small spherical calibrated glass beads. The aim is to track all beads over long time sequences to obtain sediment velocities and concentration. Classical algorithms used in fluid mechanics fail to track the beads over long sequences with a high precision because they incorrectly handle both miss-detections and detector imprecision. Our contribution is to propose a particle filter-based algorithm including a multiple motion model adapted to our problem. Additionally, this algorithm includes several improvements such as the estimation of the detector confidence to account for the lack of precision of the detector. The evaluation was made using two test sequences -one from our experimental setup and one from a simulation created numerically -with their dedicated ground truths. The results show that this algorithm outperforms state-of-the-art concurrent algorithms.

Introduction

Visual target tracking is a recurrent problem in computer vision, it has been used in many applications, e.g. video surveillance, sports analysis, or traffic safety. In particular, object Fig. 1 Sketch of the experimental arrangement (modified from [START_REF] Böhm | Two-dimensional motion of a set of particles in a free surface flow with image processing[END_REF][START_REF] Hergault | Image processing for the study of bedload transport of two-size spherical particles in a supercritical flow[END_REF]). Water flows from left to right, entraining a mixture of two-size spherical glass beads. Black beads of diameter d=6mm are introduced upstream and transparent beads of diameter d=4mm further downstream. The mobile bed is observed through a backlighted window and recorded by a high-speed video camera at 130Hz. tracking is used in fluid mechanics to track particles with techniques such as Particle Image Velocimetry (PIV) [START_REF] Westerweel | Particle image velocimetry for complex and turbulent flows[END_REF] and Particle Tracking Velocimetry (PTV) [START_REF] Ohmi | Particle-tracking velocimetry with new algorithms[END_REF]. We focus here on bedload sediment transport experiments conducted with a laboratory facility, as illustrated by the sketch in Fig. 1. Experiments consist in a two-dimensional steep channel where mixtures of two-size spherical glass beads are entrained by a turbulent supercritical water flow with a mobile bed. The aim is to track all beads (see Fig. 2) over a long time to obtain trajectories, particle velocities and concentrations [START_REF] Frey | Particle velocity and concentration profiles in bedload experiments on a steep slope[END_REF], for studying bedload granular rheology [START_REF] Maurin | Dense granular flow rheology in turbulent bedload transport[END_REF], size segregation [START_REF] Bacchi | The effects of kinetic sorting on sediment mobility on steep slopes[END_REF] and associated morphology [START_REF] Recking | Antidunes on steep slopes[END_REF].

In this context, PIV with background modeling-based trackers can be used to detect moving objects by background subtraction [START_REF] Honkanen | Background extraction from doubleframe PIV images[END_REF][START_REF] Theunissen | On improvement of PIV image interrogation near stationary interfaces[END_REF]. But the main problem with these methods is that it does not resolved strong gradients (e.g. of ve-Fig. 2 Example of images from the sequence recorded during an experiment. An image with a small number of transparent beads at the beginning of the sequence (top), and an image with a much higher number of transparent beads later in the sequence (bottom). The black line on top of the images is the water free surface. The image resolution is 1280x320. locity). Moreover, it is very sensitive to framerate, changing background, moving cameras and can remove locally nonmoving objects that are not really part of the background. In body-part, person or car tracking applications, these problems can be faced through tracking-by-detection approaches to gain robustness. These approaches have become increasingly popular by global optimization based on position, size and appearance similarity [START_REF] Andriluka | People-tracking-by-detection and people-detection-by-tracking[END_REF][START_REF] Henriques | Exploiting the circulant structure of tracking-by-detection with kernels[END_REF][START_REF] Leibe | Coupled object detection and tracking from static cameras and moving vehicles[END_REF]. Among these tracking methods, we focused on multiple object tracking where the aim is to follow a high number of objects in a scene individually, like in crowd tracking for example [START_REF] Ali | Floor fields for tracking in high density crowd scenes[END_REF][START_REF] Zhan | Crowd analysis: a survey[END_REF]. In fluid mechanics, PTV algorithms are deterministic (or data association-based tracking) and rely on a standard two step approach: (1) object detection, (2) object tracking by solving a data association problem [START_REF] Hergault | Image processing for the study of bedload transport of two-size spherical particles in a supercritical flow[END_REF]. However, they are not robust enough to track objects over very long sequences with a good precision for object positions. This is mainly because they fully trust the object detector output, so they incorrectly handle miss-detections and do not take into account the lack of precision of the detector.

In recent years, particle filter-based approaches have been successfully used in many applications [START_REF] Choset | Topological simultaneous localization and mapping (SLAM): toward exact localization without explicit localization[END_REF][START_REF] Lu | Tracking and recognizing actions of multiple hockey players using the boosted particle filter[END_REF][START_REF] Rathi | Tracking deforming objects using particle filtering for geometric active contours[END_REF]. One of the main benefits of these methods is to enable the estimation of the distribution of the target's system state incorporating a non linear motion model. In this context, Breitenstein et al. [START_REF] Breitenstein | Online multiperson tracking-by-detection from a single, uncalibrated camera. Pattern Analysis and Machine Intelligence[END_REF] proposed a very efficient online multi-target tracking approach accounting for the detector confidence to handle miss-detections. However, this method is not directly adapted to our problem mainly because it uses a single motion model unable to describe all occurring object interactions.

In this paper, using particle filters and with this idea of detector confidence, we propose to introduce a Bayesian mixed-state framework [START_REF] Isard | A mixed-state condensation tracker with automatic model-switching[END_REF] dedicated to our simulation of bedload transport. It is based on multiple motion models where each motion model corresponds to a specific state of motion linked with the mechanics of our objects: resting, rolling or bouncing also called saltation. Such an approach permits to follow the natural behavior of spherical beads in a fluid flow. We also introduce several improvements to correctly handle the lack of precision of the detector providing tracking over long sequences with a high precision for object positions.

The paper is structured as follows. After discussing the related work in the next section, Sec. 3 describes our proposed multiple model particle filter-based tracking algorithm and design choices. Sec. 4 gives details about the experiments and datasets we study. Sec. 5 presents the evaluation and a discussion of the results of tracking performances and a comparison with concurrent algorithms. Finally, Sec. 6 provides a conclusion of the paper.

Related work

This work stands at the intersection of computer vision, fluid mechanics and Bayesian statistical inference; it builds on previous approaches discussed below.

Various visual tracking approaches have been reviewed [START_REF] Smeulders | Visual tracking: An experimental survey. Pattern Analysis and Machine Intelligence[END_REF][START_REF] Wu | Online object tracking: A benchmark[END_REF] and they can be separated in two categories: deterministic methods and probabilistic ones. In fluid mechanics, researchers usually employ the first one, i.e. taking the object detector output as a straight truth and then solving the association problem on pairwise frames [START_REF] Böhm | Two-dimensional motion of a set of particles in a free surface flow with image processing[END_REF][START_REF] Hergault | Image processing for the study of bedload transport of two-size spherical particles in a supercritical flow[END_REF]. It is also the case for the Lagrangian Particle Tracking [START_REF] Houssais | Onset of sediment transport is a continuous transition driven by fluid shear and granular creep[END_REF][START_REF] Ouellette | A quantitative study of three-dimensional Lagrangian particle tracking algorithms[END_REF] widely used in experimental fluid dynamics. This kind of deterministic approach can lack precision since the association problem between detections and targets strongly relies on the object detector that can be unreliable and imprecise.

To handle this problem, several recent algorithms optimize detection assignments over a large temporal window in an offline step [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF][START_REF] Leibe | Coupled object detection and tracking from static cameras and moving vehicles[END_REF][START_REF] Zhang | Global data association for multiobject tracking using network flows[END_REF], i.e. by using information from future frames and after observing the complete sequence. Nevertheless, these methods are time-critical and unsuitable for online applications1, unlike Sequential Monte Carlo (SMC) methods. SMC or particle filters (PF) were introduced to provide better estimations and predictions in an online manner [START_REF] Doucet | Sequential Monte Carlo methods in practice[END_REF][START_REF] Gordon | Novel approach to nonlinear/non-Gaussian Bayesian state estimation[END_REF][START_REF] Isard | Condensation-conditional density propagation for visual tracking[END_REF]. Basically, they consist of a dynamic model for prediction and an observation model to evaluate the likelihood of a predicted state. The main steps are: random generation of a sample of potential states called particles, propagation of these particles through the motion model and resampling of the distribution according to the observation model.

In order to cope with difficult tracking situations in a multi-object context, Breitenstein et al. [START_REF] Breitenstein | Online multiperson tracking-by-detection from a single, uncalibrated camera. Pattern Analysis and Machine Intelligence[END_REF] proposed an automatic PF-based algorithm that exploits the detector confidence (i.e. the intermediate output of the object detector) as a graded observation model. It allows using a combination of unreliable information sources for robust multi-person tracking. As most particle filter-based algorithms, it relies on a single motion model which can be problematic in an environment with complex dynamics.

To face this limitation, a multiple model (MM) paradigm [START_REF] Arulampalam | Bearingsonly tracking of manoeuvring targets using particle filters[END_REF][START_REF] Li | Survey of maneuvering target tracking. Part V. Multiple-model methods[END_REF], also called switching dynamical model approach, has been introduced and is used for several applications such as robotics and navigation [START_REF] Boers | Interacting multiple model particle filter. Radar, Sonar and Navigation[END_REF] or video surveillance [START_REF] Zhai | An objecttracking algorithm based on multiple-model particle filtering with state partitioning[END_REF]. It assumes that, given a finite state of dynamic models (each of them corresponding to a particular behavior), a particular target can jump from one model to the other according to a set of transition probabilities. The interacting multiple model (IMM) algorithm [START_REF] Blom | The interacting multiple model algorithm for systems with Markovian switching coefficients. Automatic Control[END_REF][START_REF] Khalid | A low-complexity interacting multiple model filter for maneuvering target tracking[END_REF][START_REF] Mazor | Interacting multiple model methods in target tracking: a survey[END_REF] was the first to propose this approach through jump Markov process modeling. Then, the multiple model particle filter (MMPF) framework [START_REF] Isard | A mixed-state condensation tracker with automatic model-switching[END_REF][START_REF] Mcginnity | Multiple model bootstrap filter for maneuvering target tracking[END_REF] came as a promising alternative outperforming the IMM algorithm in a bearings-only tracking problem [START_REF] Arulampalam | Bearingsonly tracking of manoeuvring targets using particle filters[END_REF]. Both methods have also been recently combined in an IMMPF algorithm [START_REF] Chang | Interacting multiple model particle filtering using new particle resampling algorithm[END_REF][START_REF] Dou | Robust visual tracking based on interactive multiple model particle filter by integrating multiple cues[END_REF]. The MMPF method uses the idea of MM, i.e. object motions can be classified in different motion states depending on the situation, in a PF framework. Each motion state is dealt with a specific state evolution model for prediction. However, it requires to know all motion states beforehand with a mathematical description for each. In contrast, our method exploits the mechanistic knowledge of behaviors we have in our application.

Multi-object multi-model tracking algorithm

In this section, we first discuss about the global goal and the proposed approach. Then we give the overview of our algorithm before focusing on each if its parts.

Problem formulation and proposed approach

The purpose of the experimental setup is to study the collective behavior of beads with respect to their size. More precisely we are interested in the phenomenon of infiltration called kinetic sieving [START_REF] Bacchi | The effects of kinetic sorting on sediment mobility on steep slopes[END_REF]. At the beginning of an experiment, the bed is composed of an arrangement of large black beads. Small transparent beads are then introduced gradually and start to infiltrate in the mobile bed until a two-size equilibrium is reached (in approximately 50 min). See Sec. 4.1 for further details.

The global mechanical goal is to study the role of interactions at the bead scale in the formation of bed structures during the transition to equilibrium. This goal requires the statistical analysis of all individual trajectories along the experiment. A real time processing is not needed, but a fast post-processing is required to be able to analyze very long sequences (1 million images) in a reasonable time.

Three different motion types are used to describe the dynamics in bedload transport: resting when a bead is not moving, mostly encountered when it is deeply included inside the bed, rolling when it is rolling near the bed surface and saltating when it is bouncing on others above the bed surface. A given bead can suddenly switch from one motion type to another depending on its velocity, neighborhood and interactions with other beads.

From the tracking point of view, the main bottlenecks to solve are twofold. The first one is the diversity and abruptness of motion. Depending on the motion type, the dynamics are very different. Additionally the motion type can suddenly change and even in a given type, the bead motion is altered by interactions with neighboring beads. The second bottleneck is the weakness of the transparent beads detector. Due to 3D effects, the bead pattern is slightly changing resulting in miss-detections and lack of precision in the detection of transparent beads. Combined with the abruptness of motion, this constraint can lead to wrong associations and bad trajectories.

To face these problems, our approach is to use MMPFs. Compared to Kalman or IMM filters, the motivation to use particle filters is a non-linear observation model to handle miss-detections through the use of detector confidence information (see Sec. 3.3.2). Additionally, the multiple model approach allows incorporating the three motion models inspired by the mechanics and a switching mechanism to consider sporadic motion model changes. Compared to an IMM approach, the switching mechanism appears closer to the mechanics because a linear combination of states is not very realistic.

Algorithm overview

Our fully automatic multi-object tracking algorithm uses the same principle as [START_REF] Odobez | Embedding motion in model-based stochastic tracking[END_REF] by embedding multiple motion models in a particle filter-based approach (MMPF). Moreover, it uses the technique of switching dynamical model introduced by [START_REF] Isard | A mixed-state condensation tracker with automatic model-switching[END_REF] to increase the motion models flexibility of visual trackers. Thus, each physical trajectory is estimated by an individual particle filter (tracker), automatically initialized when a new object is detected and terminated when the object is leaving the field of view. A global overview of the algorithm is shown in Fig. 3. At each time step we perform: (1) object detection, (2) estimation of the detector confidence density, (3) association of each detection to a tracker, (4) update of the state of each particle filter according to the specific motion model and the new detection. The principle of the MMPF tracking algorithm is to predict the target state (i.e. target location, velocity and motion state) by applying an adapted motion model depending on the target behavior. This target state is then corrected with measurements made on the observed image to estimate the final target state. The target estimated at time t is further used for data association at time t + 1. Each part of the algorithm is explained below.

Object detection

For each image, the object detection phase aims at detecting all beads by image processing. It consists in 3 steps: (1) detect black beads, (2) remove them from the image, (3) detect transparent beads. Facing lacks of reliability especially with the transparent bead detector, a special treatment is applied to the detectors (cf. Sec. 3.3.2).

Object detectors

First, black beads are detected by thresholding the image at level τ b to keep only black pixels, and by taking the center of connected components having an area close to the mean bead area. This detector is very efficient as black beads are easily distinguishable from other kind of beads and they are not overlapping (6mm diameter beads in a channel width of 6.5mm). So it always returns the entire set of black bead centers D b . Then, black beads are erased by replacing their gray levels by those of the background. The background is reconstructed through a morphological closing with a disk shaped structuring element (radius just greater than bead radius). This makes transparent beads more discernible. Secondly, since transparent beads appear as faint dark rings of different shapes because of their neighboring configurations (4mm diameter beads that can be partially overlapped by others), we use a specific chain of morphological operations to detect them (initially developed by [START_REF] Hergault | Image processing for the study of bedload transport of two-size spherical particles in a supercritical flow[END_REF]). We apply a hconvex operator [START_REF] Soille | Morphological image analysis: Principles and Applications[END_REF] on the image obtained at the previous step, then a normalized cross-correlation with a ring-shape model, local maxima extraction and maxima selection according to the adjusted threshold τ t . It returns the set of transparent bead centers D t with some fallibility due to partially occluded beads. How this problem is dealt with is detailed in next section.

Detector confidence estimation

To handle missing detections, we introduce a detector confidence estimation and we use it as a graded observation model [START_REF] Breitenstein | Online multiperson tracking-by-detection from a single, uncalibrated camera. Pattern Analysis and Machine Intelligence[END_REF]. The principle is: based on the raw output of the detector, the detector confidence must give an estimation of the likelihood of a detection at each candidate pixel.

Assuming an exponential distribution for the position of each detection, we can estimate the likelihood of observing a bead at location x obs given the set of detections D as:

p(x obs |D) = d c (x obs ) = 1 k D d ∈D L(d) exp(-λ x obs -x d 2 ) (1)
where . 2 denotes the Euclidean distance between two points in Euclidean space, λ denotes the exponential decay constant (see Sec. For transparent beads, the set D corresponds to the entire set of local maxima outputs of the transparent bead detector (down to a very low threshold lower than τ t ) and L(d) is the value of the normalized cross-correlation with the ring shape model.

For black beads, D = D b and L(d) = 1 as all detections have the same reliability. In practice, black beads are never missed since the corresponding detector is efficient and very reliable.

Data association

To assign at most one detection to at most one tracker, a detection-filter association process is needed. Due to the high number of detections and the long time series, the data association was done deterministically, independently from the particle filtering. We used a greedy algorithm [START_REF] Wu | Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors[END_REF] which was found to be an effective and sufficient solution with a lower computational cost than the optimal Hungarian algorithm [START_REF] Kuhn | The Hungarian method for the assignment problem[END_REF].

Greedy data association

Given the matching cost g(tr, d) between all tracker-detection pairs computed as described below, the greedy algorithm iteratively selects the best candidate and removes the corresponding concurrent associations. To limit the number of possible tracker-detection pairs, the set of possible detections associated to a given tracker is limited to the detections located inside a circular region centered at the predicted position xtr given by this tracker. This predicted position is made assuming a motion model corresponding to the motion state of the tracker at previous time step.

Matching cost

The matching cost g(tr, d) between a detection d and a tracker tr consists of a combination of two factors: one based on the distance of the detection position x d to the predicted position of the tracker xtr , the other based on the tracker velocity at the previous time step u tr . It is defined as:

g(tr, d) = α x d -xtr 2 r s distance term + β |u tr | |u| max velocity term (2)
where |.| denotes the Euclidean norm of a vector. The distance term promotes the detections closest to the prediction. With the velocity term, we promote low velocities as it allows better associations in case of bead collisions. Indeed, for example when a bead bounces, its predicted position (assuming a constant velocity model) can fall very close to another bead leading to the same distance term for the corresponding trackers. In this case, adding a velocity term in the matching cost permits to increase the cost of higher velocities much more than lower ones, and so to favor choosing lower velocities first during the greedy data association.

When associating the first two frames, the velocity term is set to 0 since we do not have a velocity value yet. The value |u| max is a normalization coefficient equal to the highest velocity norm in the previous time step, the parameters α and β represent the weight of each term (see Sec. 4.3), and r s is the radius of the circular searching region calculated from the fluid velocity, it is used here as a normalization coefficient. Therefore, the greedy algorithm returns the subset of detections associated to a tracker. The location x d (resp. velocity u d ) of the detection is considered as the observation variable x obs (resp. u obs ) of this tracker. The velocity u obs is measured between the raw detection of the target at previous time step (the direct output of the detector before filtering) and the raw detection of the target in the current image. Additionally, motion state s obs will be associated to this tracker (see Sec. 3.5.1).

Multi-model particle filtering

Each tracker (or trajectory) is described by a separate particle filter of N particles. At time step t the state space c i t of a particle i is defined as {x i t , u i t , s i t } where x i t = (x, y) denotes the position, u i t = (u, v) the velocity and s i t the motion state. The observation variable at time t is given by z t = {x obs t , u obs t , s obs t } resulting from measurement of location, velocity and motion state. We employ the Sequential Importance Resampling filter, also known as the bootstrap filter [START_REF] Gordon | Novel approach to nonlinear/non-Gaussian Bayesian state estimation[END_REF] where the state transition density is used as importance distribution, to approximate the probability function. For one target, the algorithm goes through different steps described in Fig. 4. It works as follows: (1) Measurement of target motion state according to neighbors configuration and target velocity, (2) Random selection of new motion states according to conditional probability table, (3) Update of particle positions and velocities according to motion models, (4) Particle importance weighting, normalizing and resampling, (5) Estimation of final motion state, position and velocity of the target. The main difference with the algorithm in [START_REF] De Micheaux | Online multi-model particle filter-based tracking to study bedload transport[END_REF], is the use of the target motion state estimation as an observation in the importance weighting (see Sec. 3.5.2).

Motion models based on mechanical dynamics

In motion-based stochastic tracking [START_REF] Odobez | Embedding motion in model-based stochastic tracking[END_REF], explicit motion measurements are used to guide predictions. In our application, beads have different behaviors according to their location, velocity and neighborhood. We can distinguish three distinct motion states linked to the dynamics in bedload transport: resting (not moving), rolling (rolling/sliding on others) and saltating (bouncing on others). In the saltating state, we can discriminate two sub-states: a constant motion and a rebound motion. To propagate a particle i at time step t, we first update its previous motion state s i t-1 , and then we apply a specific motion model to update its position and velocity. The new updated motion state s i t is drawn according to a conditional probability table π (see Sec. 4.3). The motion models are:

resting: (x, y) t = (x, y) t-1 + ε (x,y) rest rolling: (x, y) t = (x, y) t-1 + (u, 0) t-1 ∆t + ε (x,y) roll (u, v) t = (u, 0) t-1 + ε (u) roll saltating-constant: (x, y) t = (x, y) t-1 + (u, v) t-1 ∆t + ε (x,y) salt (u, v) t = (u, v) t-1 + ε (u,v) salt saltating-rebound: (x, y) t = (x, y) t-1 + (u, -v) t-1 ∆t + ε (x,y) salt (u, v) t = (u, -v) t-1 + ε (u,v) salt (3) where ε (x,y) rest , ε (x,y) roll , ε (x,y)
salt are the process noises on the position and ε (u) roll , ε (u,v) salt are the process noises on the velocity (see Sec. 4.3), they are all independently drawn from zeromean normal distributions ; ∆t depends on the framerate of the sequence. At initialisation, the velocity of particles drawn in the saltating motion state is taken equal to a fraction of the velocity of the water flow.

Motion state measurement. Before applying the sequential importance resampling filtering on a target, it is possible to make an estimation of its motion state with the observations. The measurement of this motion state is computed by inspecting the neighbors configuration and velocity of the target with the help of the data association output. Indeed the idea is to use our mechanistic knowledge to best fit the mechanical dynamics of our objects. Given the number of neighboring beads nbN obs touching the observed bead, we can consider the motion state to be "resting" if either nbN obs is significantly high or the velocity norm |u obs | of the bead is low. Similarly, the motion state will be "saltating" if nbN obs is low or |u obs | is high. Formally, we have:

s obs =        resting, if nbN obs ≥ nbN r est or |u obs | ≤ u rest , saltating, if nbN obs ≤ nbN salt or |u obs | ≥ u salt , rolling, otherwise. ( 4
)
where nbN r est (resp. nbN salt ) is the threshold number of neighbors for resting (resp. saltating) and u r est (resp. u salt ) is the threshold velocity norm for resting (resp. saltating).

These values are determined experimentally on the test sequence (see Sec. 4.3).

The estimation of the motion state is used as a cue in the particle weighting (in Sec. 3.5.2) or to set the final motion state of the target (in Sec. 3.5.3).

Particle importance weighting

Observation model. The importance weight w i t for a particle i at time step t is updated from its value at previous time step w i t-1 and from the conditional likelihood of the new observation z t given the previous state c i t-1 of the particle:

w i t ∝ w i t-1 p(z t |c i t-1 ) (5)
Thanks to resampling in each time step, all weights are the same at the end and w i t-1 = 1/N, thus this term can be ignored. Moreover, in practice, the prior is usually chosen as the importance distribution to approximate the probability density function [START_REF] Gordon | Novel approach to nonlinear/non-Gaussian Bayesian state estimation[END_REF]. With these simplifications applied to Eq. ( 5), the particle weighting only depends on the propagated particle state and is so quite simpler:

w i t ∝ p(z t |c i t ) (6)
Importance weighting. Assuming independence between observation variables, we have:

p(z t |c i t ) = p(
We have chosen to give a high confidence to the observed velocity which corresponds to select only particles having a velocity component equal to u obs t . Practically, this comes down to remove the velocity likelihood in Eq. ( 7) and set each velocity component of the state to u obs t at the weighting step. This choice aims to better estimate the state density for locations and state variables with a low number of particles (e.g. 100) in particle filters.

Sometimes, there is not a detection associated to the tracker and this is most often caused by a false negative. In such cases, we need to continue having an estimation of the tracker state not to lose it until a detection is associated to it again. To do so, the likelihood of particle i at location x i t of the tracker tr is given by the detector confidence (see Eq. ( 1)). But in most cases, i.e. when a detection is associated to the tracker, it is estimated by a normal distribution centered on the corresponding detection d (see Sec. 4.3 for the variance value). Formally:

p(x obs t |c i t ) = p N (x i t -x obs t ) if I(tr) = 1, d c (x i t ) otherwise. ( 8
)
where I(tr) is an indicator function that returns 1 if a detection is associated to the tracker and 0 otherwise, it is given by the data association.

For the motion state likelihood p(s obs t |c i t ), we assume a constant value γ if s i t = s obs t and 1 -γ otherwise. This value is set experimentally (see Sec. 4.3). Formally:

p(s obs t |c i t ) = p γ (s i t ) = γ, if s i t = s obs t , 1 -γ, otherwise. (9)
Thus, the update of weights is given by:

w i t = p γ (s i t )p N (x i t -x obs t ) if I(tr) = 1, p γ (s i t )d c (x i t ) otherwise. ( 10
)
Normalizing and resampling. After weighting all particles, the weights are normalized so that N i=1 w i t = 1. Then, the resampling is used to solve particle degeneracy, namely removing particles with low weights and multiplying those with high weights. The approach used here is the multinomial resampling based on the bootstrap method [START_REF] Efron | An introduction to the bootstrap[END_REF].

Final estimation

Finally, the position of the target is estimated by averaging the resampled particles. For the velocity, according to the choice of promoting only particles having the observed velocity, the final estimate is exactly equal to this observed velocity. After all positions and velocities of the targets are estimated, we measure again their motion state the same way it was done before the particle importance weighting (see Sec. 3.5.1 and Eq. ( 4)). A classical approach could be to choose the most represented state but we chose to reassess the motion state to ensure to be consistent with the mechanical motion state. Indeed, we observed a slight mismatch between the predicted state and the re-estimated state based on the predicted position and velocity. A detailed analysis of this mismatch shows it is due to discretization issue. Thus, in some situations and when the number of particles is too low (e.g. 100) regarding the number of state variables, after importance weighting, the correct motion state is not the most represented one. This mismatch does not occur with more particles in the particle filter (e.g. 500).

Experiments

In this section, we present the detailed experimental protocol we used to produce the experimental image sequences of bedload transport. Then, we describe the datasets used for the evaluation of our algorithm. Finally, we give some details about the implementation of the algorithm with the parameter settings.

Experimental protocol

The experimental protocol to reproduce bedload transport in the experimental flume illustrated in Fig. 1 consists in forming a two-size mobile bed to promote exchanges between the moving beads and the bed. First of all, a mobile bed of black beads is created by feeding water and black beads at constant flow rates into the empty flume which ends with an obstacle. Once the equilibrium is reached (after approx. 10 min), i.e. when the output rate of black beads is equal to the input rate, the feeding of transparent beads begins at constant flow rate, while water and black beads continue to be fed. The transparent beads rebound, move and start to infiltrate in the mobile bed. This phenomenon of infiltration is called kinetic sieving [START_REF] Bacchi | The effects of kinetic sorting on sediment mobility on steep slopes[END_REF]. After a little time, a new two-size equilibrium is finally reached (after approx. 50 min). Each experiment is recorded over the entire duration and a sequence of images is obtained. These images are subsequently analysed with the procedure described in this study.

Datasets and ground truths

To test our tracking algorithm, we worked on two datasets: a short experimental sequence obtained with the previously detailed experiment and a longer numerical one obtained with a simulation. Both following datasets are freely available with their ground truth2.

Experimental sequence. The experimental sequence is a 1,000-frame sequence recorded at 130 fps with approximately 400 beads per frame (about 300 coarse and 100 small 2 https://perso.univ-st-etienne.fr/ducottet/ Fig. 5 Example of trajectories for the experimental ExpSeq (top) and simulated NumSeq (bottom) datasets (cropped images). Green color is for a coarse black bead and red is for a smaller transparent bead. The green and red color gradations from darker to lighter show respectively the three motion states (resting, rolling and saltating). beads). The image resolution is 1280x320. The sequence is split in two parts, one for optimizing and estimating the parameters, and the other part for evaluation. We created a ground truth on this sequence by first detecting beads with our object detectors on each frame (Sec. 3.3.1), and then by editing visually the detections, i.e. by removing the false positives and adding manually the false negatives. To get the ground truth of trajectories, we applied our data association algorithm (Sec. 3.4) on the ground truth of detections just created and then computed the motion states of each target by watching neighbors configuration (as in [START_REF] Hergault | Image processing for the study of bedload transport of two-size spherical particles in a supercritical flow[END_REF]). The limitation of this ground truth is the precision of the positions. Indeed, the center of missing detections were selected manually to fit visually the center of beads as good as possible. So they can be inaccurate by one or two pixels. Moreover, as we exploited the raw output of the detector to create this ground truth, the tracking algorithms cannot use it directly to study precision of estimated positions. To overcome this problem, we considered the detected locations as the truth and we added white noise on the locations given by the detector to simulate real measurements. Thus, all the classical metrics can be used to evaluate the performance of algorithms on this experimental sequence denoted ExpSeq, an example of trajectories is given in Fig. 5

(top).

Numerical sequence. In order to study a longer sequence and lower frequency phenomena, we created a numerical sequence of bedload transport of 10,000 images thanks to a model developed at Irstea based on a coupled fluid discrete element method (cf. [START_REF] Maurin | A minimal coupled fluid-discrete element model for bedload transport[END_REF], [START_REF] Maurin | Dense granular flow rheology in turbulent bedload transport[END_REF] and [START_REF] Maurin | Revisiting slope influence in turbulent bedload transport: consequences for vertical flow structure and transport rate scaling[END_REF]) and validated against our experiments (cf. [START_REF] Frey | Particle velocity and concentration profiles in bedload experiments on a steep slope[END_REF]). The idea of this simulation is to reproduce as best as possible our experiments (e.g. flume size, number of each type of beads, feeding rates, fluid velocity, image frequency, etc.). Moreover, thanks to ParaView, an open source software used for scientific visualization, we were able to have a rendering of images of the obtained sequence really close to our experimental images. Thanks to this rendering, we were able to use this numerical sequence in the same manner as an experimental sequence and apply all the previously presented algorithms. As for ExpSeq, half of the sequence is used to optimize and estimate the parameters, and the other for evaluation. The numerical sequence will be denoted NumSeq, an example of trajectories is given in Fig. 5 (bottom).

Ground truth motion states. On the obtained ground truth sequences, we need to assign a motion state to each tracker at each time step. To do this, we follow the offline method initially developed by Böhm et al. [START_REF] Böhm | Two-dimensional motion of a set of particles in a free surface flow with image processing[END_REF] which consists in combining two criteria: first testing the tracker velocity (averaged over five consecutive frames) against a threshold velocity and second testing the distance between the tracker and its nearest neighbor against a reference distance. The main difference with the way we measure the motion states in our algorithm is that here it is done after the tracking execution in a post-processing as it uses information from future frames, while we do it during the tracking execution in an online manner using current velocities. So the velocity thresholds here do not have the same values, the ones of the offline way of doing are more restrictive. We consider that the offline method gives motion states close enough to real motion behaviors to be used as ground truth.

Implementation details

Free parameter tuning. The number of particles N was fixed to N = 100, being a good compromise between computational cost and tracking reliability. The parameters in Eqs. ( 1) and ( 2) have been set experimentally and remained the same for the different experiments used in Sec. 4.2. λ was set to 1 in Eq. ( 1). In Eq. ( 2), α and β were set respectively to 0.75 and 0.25 to give more importance to the distance term. The variance of the normal distribution in Eq. ( 8) was set experimentally to 3 pixels for both x-and y-coordinates. The reward parameter γ in Eq. ( 9) was fixed to 2/3 in order to promote particles having the same motion state as the measurement.

Process noise and tracker parameters. Each motion state in Eqs. (3) has its own variance for the process noise on position and velocity. They were set automatically by analyzing representative training sequences corresponding to the first halves of the studied sequences given in Sec. 4.2. The initial particle positions and velocities were drawn from a Normal distribution with positions centered at the detection center and velocities centered to null velocity. To handle the difficulty of dealing with new trackers, we increased the variances to make the motion model more flexible during the first 3 frames. To stop bad predictions, a tracker at resting state (resp. rolling and saltating) survives only 100 frames (resp. 10 frames and 5 frames) without associated detection and is then automatically terminated.

Transition probability table.

For a given particle i, the transition from one motion state s i t-1 to another s i t is controlled with conditional probabilities. We have estimated these probabilities on representative training sequences corresponding to the first halves of the studied sequences given in Sec. 4.2. As the probabilities were found to be nearly the same for all datasets, we present only one case here. The transition probability table π required for the switching state process presented in Sec. 3.5.1, was estimated as:

π = p(s i t |s i t-1 ) =      
0.95 0.05 0.00 0.07 0.92 0.01 0.00 0.09 0.91

      (11)
As N was fixed to 100 and we want each state to have a chance to be represented at each time step, we modified the transition probability table to be slightly less restrictive by imposing a minimal probability of 0.05, leading to:

π ≈      
0.80 0.15 0.05 0.10 0.80 0.10 0.05 0.15 0.80

      (12)
The saltating state being divided in two sub-states, when it is drawn after getting through π, there is then a probability of 0.5 to be in the saltating-constant sub-state (resp. saltating-bouncing sub-state). At initialization, the particle motion states were set as unknown states. At the next step, the particles had an equivalent chance to go in any of the motion states.

Motion state measurement thresholds. To measure the motion state of a target (Eq. ( 4)), we need to calculate its number of neighbors. We consider a detection as a neighbor of the target if they are in contact, i.e. when the distance between both centers is equal to the sum of their corresponding radius. The thresholds on the number of neighbors for the resting and saltating states were set experimentally to nbN rest = 5 and nbN salt = 0 respectively. The thresholds on velocity norms were set experimentally to u rest = 0.015m/s and u salt = 0.25m/s for the sequences studied in this paper (cf. Sec. 4.2). u rest and u salt are specific to each sequence. These four thresholds were adjusted by trial and error to minimize the differences between the motion state determined by the algorithm and the motion state determined by the naked eye.

Results and discussion

In this section, we present an analysis of some execution information and the results of the proposed tracking algorithm against other algorithms of the literature. They are all compared to a ground truth of a short experimental sequence ExpSeq and a longer numerical one NumSeq (see Sec. 4.2). Indeed, as our case study is very specific, it is impossible to test our tracking algorithm on open datasets. We also use the CLEAR MOT metrics [START_REF] Bernardin | Evaluating multiple object tracking performance: the CLEAR MOT metrics[END_REF] to evaluate the tracking performance. Moreover, we study how the algorithms deal with different detector qualities and how well they estimate the motion states.

Execution analysis

Sensitivity of parameters.

As mentioned before, some parameters were individually adjusted on both ExpSeq and NumSeq. We saw that these parameters were nearly identical. So we tested the parameters used for a given dataset on the other one and vice-versa and we observed that it did not change the results. From this observation, we have chosen to use only one set of parameters for both datasets. The only parameters to adjust are the thresholds on the detectors as they depend on the imaging setup.

Execution time. For a sequence of 5,000 images and a number of N = 100 particles, the execution time on a workstation with an Intel Core i7 2.70GHz and 8GB of memory is approximately 10 hours. On this duration, 15% is used for the detection of the beads and 85% for their tracking across the sequence. Note that this duration is reasonable for the remote analysis of an experiment and it could be reduced significantly since the algorithm has been implemented in Matlab.

Algorithms

To study the reliability and efficiency of our method, we tested it against four other algorithms of the literature. First of all, we have our particle filter-based tracking algorithm (see Sec. 3) which is based on the use of three motion models (or states), the detector confidence and a measurement of the motion state as a cue in the likelihood (MMPF+). Second, we have our preliminary work on three motion models-based particle filters with detector confidence [START_REF] De Micheaux | Online multi-model particle filter-based tracking to study bedload transport[END_REF] (MMPF). Third, we have an adaptation of the algorithm of Breitenstein et al. [START_REF] Breitenstein | Online multiperson tracking-by-detection from a single, uncalibrated camera. Pattern Analysis and Machine Intelligence[END_REF] based on a single constant velocity model and detector confidence particle filters (PF). Fourth, we have an interacting multiple model tracking algorithm [START_REF] Blom | The interacting multiple model algorithm for systems with Markovian switching coefficients. Automatic Control[END_REF] that uses our three motion models described as Kalman filters (IMM). The model and noise parameters are set the same as the MMPF+ ones. Finally, we have an initial and simple deterministic tracking approach previously developed in our laboratory [START_REF] Hergault | Image processing for the study of bedload transport of two-size spherical particles in a supercritical flow[END_REF] (DET). This last algorithm blindly relies on the output of the detector, without any special treatment for the false positives and false negatives.

Evaluation of the tracking algorithm

To evaluate the tracking performance, we use the CLEAR MOT metrics [START_REF] Bernardin | Evaluating multiple object tracking performance: the CLEAR MOT metrics[END_REF]. It gives a precision score MOTP (intersection over union (IoU) of the estimated bead and the ground truth) and an accuracy score MOTA (accounts for all object configuration errors, i.e. false negatives, false positives and mismatches3). The IoU threshold used here was 0.43, which rejects the estimated beads having their center outside the area of the ground truth bead. We also use the percentage of correct tracks, where a track is considered as correct on its total length if more than 95% of it has no false negatives or mismatches. In order to see the influence of the detector output on the different tracking algorithms, we vary the threshold for the detection of the transparent beads. It gives different detector qualities and allows seeing how the tracking algorithms deal with more false negatives or false positives. At last, we study how well the multiple motion model-based algorithms estimate the motion state of the beads.

Quantitative analysis.

In Table 1, we present the evaluation results of the five tracking algorithms MMPF+, MMPF, PF, IMM and DET on both datasets ExpSeq and NumSeq. They show that MMPF+ has the best results in terms of accuracy in light of the percentage of correct tracks, number of mismatches and MOTA score. This comes from its good management of the false negatives. In terms of precision through the MOTP score, IMM and DET seem better than the others. However, with more particles (e.g. N = 500), the MMPF+'s precision climbs up to be as good as the IMM's. When compared to the other particle filter-based algorithms MMPF and PF, MMPF+ is the most performing in all metrics. For example, our precision MOTP and accuracy MOTA results are higher for both datasets. Note that all precision and accuracy scores are already high. This is due to the good quality of the transparent bead detectors for the chosen thresholds. We chose the threshold of the transparent bead detector τ t = 0.25 for ExpSeq and τ t = 0.27 for NumSeq as they give the best tracking results and as we know the importance of good detectors for reliable tracking operations. But the detector is not always that efficient as it depends on the image quality of the sequences. The analysis of the effect of different τ t is done below.

We can also observe that the number of identity switches is the lowest for MMPF+. This property is one of the most important for a good tracking algorithm. In our images, ID switches can happen for tracks changing abruptly of direction (e.g. rebounding tracks) or velocity (e.g. acceleration >> 0), or for newly appearing beads located really close to where a track should be according to its motion model. The implementation of multiple motion models in our method allows better dealing with these [START_REF] De Micheaux | Online multi-model particle filter-based tracking to study bedload transport[END_REF] 98.50% 97.21% 0.15% 0.66% 111 99.18% PF [START_REF] Breitenstein | Online multiperson tracking-by-detection from a single, uncalibrated camera. Pattern Analysis and Machine Intelligence[END_REF] 97.59% 96.40% 0.15% 0.66% 91 99.19% IMM [START_REF] Blom | The interacting multiple model algorithm for systems with Markovian switching coefficients. Automatic Control[END_REF] 98.05% 97.60% 0.16% 0.59% 144 99.23% DET [START_REF] Hergault | Image processing for the study of bedload transport of two-size spherical particles in a supercritical flow[END_REF] 98.05% 97.62% 0.17% 0.59% 145 99.23%

Table 1 Tracking evaluation results for the five algorithms (our MMPF+, the MMPF from [START_REF] De Micheaux | Online multi-model particle filter-based tracking to study bedload transport[END_REF], the PF from [START_REF] Breitenstein | Online multiperson tracking-by-detection from a single, uncalibrated camera. Pattern Analysis and Machine Intelligence[END_REF], the IMM from [START_REF] Blom | The interacting multiple model algorithm for systems with Markovian switching coefficients. Automatic Control[END_REF], the DET from [START_REF] Hergault | Image processing for the study of bedload transport of two-size spherical particles in a supercritical flow[END_REF]) on both ExpSeq and NumSeq datasets (respectively an experimental and a numerical sequence). It shows the percentage of correct tracks (i.e. track correct when more than 95% of its length is correct), the false negative rate (FN), the false positive rate (FP), the number of mismatches (Id Sw.) and the CLEAR MOT metrics [START_REF] Bernardin | Evaluating multiple object tracking performance: the CLEAR MOT metrics[END_REF] such as the precision score (MOTP) and the accuracy score (MOTA). In bold the best score for each metrics. * corresponds to values where the MMPF+ algorithm was executed with 500 particles.

situations: this was one of the motivations of the MMPF+ algorithm.

In terms of false negative rates, our algorithm surpasses the others too. Indeed, in absence of an associated detection to a tracker, the DET algorithm stops immediately the corresponding tracker leading to a false negative, contrary to particle filter algorithms that estimate its position with the help of its previous behavior. Our algorithm is better than the PF method since it uses multiple adapted motion models to predict positions. And it is also better than MMPF since it estimates the motion states more closely to the real behavior of the beads.

Nevertheless, the deterministic and IMM approaches seem to be the best in terms of false positive rate. This is mainly because of the termination of trackers at borders (i.e. side effects). Indeed, in the particle filter algorithms, when there is no associated detection to a tracker because the bead left the field of view, the tracker tries to estimate its new state. If its estimated position is still inside the field of view, the tracker continues, resulting in a false positive. It survives like this a limited number of frames without associated detection and is then automatically terminated (see Sec. 4.3 for the survival times). On the contrary, DET and IMM algorithms immediately stop a tracker when there is no associated detection. So, in this case of side effect and in both studied datasets, it appears that these two methods are better than the particle filter methods.

Impact of detector quality. In order to study the tracking performance with different levels of detection, we varied the threshold τ t of the transparent bead detector. It allows assessing how the detection quality influences the percentage of correct tracks and the accuracy MOTA on transparent beads. We made this analysis for the five algorithms and on both ExpSeq (Fig. 6, left) and NumSeq (Fig. 6, right) datasets.

The results show that our algorithm globally outperforms the four others in terms of accuracy for both datasets. Indeed, we can observe that according to the MOTA score, MMPF+ is the second best performing method for low thresholds (i.e. when more false positives) and the best performing one for high thresholds (i.e. when more false negatives). Only IMM is better for low thresholds because of the problem of false positives previously mentionned. The good results for high thresholds come from the good management of detection errors: quickly stopping trackers wrongly created by false detections and continuing predicting trackers without associated beads when missing detections.

Whatever the algorithm used, the tracking performances depend on the detector quality: the higher τ t , the smaller the percentage of correct tracks. Especially, the DET and IMM algorithms without the propagation of undetected beads are the most impacted by high values of τ t , as expected.

Motion state estimation. The motion states of trackers are not only used to set specific motion models for the particle filters. They are also used to study the behavior of trackers in each motion state. The aim is to analyze, for each motion state separately, the number of trackers at each time step, the velocity profiles, and the infiltration and concentration inside the mobile bed. For reliable analysis, we need the estimated motion states to be as close as possible to mechanical states. Therefore, we compare them against the motion states of the ground truths.

In Fig. 7, we present the F-score of the state identification obtained by our algorithm against MMPF for both ExpSeq and NumSeq. For each motion state, the F-score is a measure of accuracy of the classification in this motion state that considers both the precision4 and recall5. We choose F-score instead of accuracy since accuracy yields misleading results if the dataset is unbalanced, i.e. when the number of samples in different classes vary greatly, as it is the case for our datasets with approximately 2 times less rolling beads and 20 times less saltating beads than resting beads. Firstly, we can observe that results are almost the same for both datasets. This confirms that the numerical dataset can be used to evaluate multiple model tracking algorithms. Secondly, the MMPF+ algorithm outperforms MMPF for all states which confirms its better ability to predict the state. In particular, the accuracy of saltating beads is increased by 0.2 for the MMPF+ algorithm especially thanks to the introduction of the two saltating sub-states 'constant' and 'rebound' (see Sec. 3.5.1). Thirdly, algorithms perform more or less well depending on the motion state. Indeed, resting beads are very well classified (0.85) compared to saltating ones (0.70) or rolling ones (0.45). This is not surprising since resting beads are easy to classify thanks to their low velocity and high number of neighbors. On the contrary, rolling beads can be confused either with resting beads when they have low velocity or with saltating beads when the are moving fast on top of the bed.

Conclusions

We have presented a new online particle filter algorithm based on multiple dynamic models for automatic multi-object tracking over long sequences. Our motivation was the study of bedload transport in an experimental channel with the tracking of spherical beads entrained by a water flow. The first contribution of our work is the use of several motion models to guide bead trackers with adapted predictions. Having a priori information about the object mechanical dynamics, we are able to better estimate object trajectories. The second contribution concerns the handling of unreliable detections, i.e. false positives and false negatives (missing detections). A robust bead tracking is obtained through the estimation of the detector confidence to guide particles of the filter when no discrete high-confidence detection is issued by the detector. Furthermore, we use the state of neighboring objects as an information in the likelihood of the particles to help choosing between motion states.

To evaluate the tracking performances, we carefully prepared two ground truth datasets, one created manually on an experimental sequence of images and one created numerically through a simulation reproducing the experimental conditions. When compared to state-of-the-art algorithms, our algorithm provides the highest tracking precision and accuracy. Moreover, it appears to be the least impacted by missing detections and presents less identity switches than the others when applied on detectors of different qualities. In terms of classification of the bead behaviors in motion states, our classifier outperforms another one from a similar multiple model algorithm thanks to inspection of neighbors configurations and velocities.

In our images, the main problem is to detect efficiently the transparent beads as they appear sometimes partially occluded. A possible extension would be to train the detector on several configuration problems instead of using a single appearance model of the bead. Furthermore, particles of the filter could be used in the data association step with their specific motion model for predictions. The tracking algorithm will be applied on very long sequences to observe both high and lower frequency phenomena which appear in sediment transport.

Fig. 3

 3 Fig. 3 Overview of the different parts of the multi-object tracking algorithm.

4 . 3 )

 43 , x d is the location of a detection d ∈ D, L(d) is the likelihood of the detection at location x d and k D is the normalization coefficient.

Fig. 4

 4 Fig.[START_REF] Arulampalam | Bearingsonly tracking of manoeuvring targets using particle filters[END_REF] The multi-model particle filter-based tracking algorithm.

Fig. 6 Fig. 7

 67 Fig. 6 Evaluation of performances of the tracking of transparent beads for the ExpSeq (left graph) and NumSeq (right graph) datasets, and for different qualities of the transparent bead detector that are controlled by the detection thresholds. The percentage of correct tracks (bars, left scales) and the accuracy score MOTA (curves, right scales) are plotted.

The precision is the fraction of retrieved instances that are relevant.

Acknowledgements This research was supported by Irstea; labex OSUG@2020; INSU programme EC2CO-BIOHEFECT and DRIL Modsed; the French national research agency project SegSed ANR-16-CE01-0005; and the Rhône-Alpes region as part of its higher education, research and innovation regional Strategy (Environment Academic Research Community).


Jelte Van Waterschoot

Guillaume Dubuisson Duplessis

Lorenzo Gatti
email: l.gatti@fbk.eu

Merijn Bruijnes

Dirk Heylen
email: d.k.j.heylen@utwente.nl

HAI Alice -An Information-Providing Closed-Domain Dialog Corpus

Keywords: Corpus, Human-Agent Interaction, Wizard of Oz, Closed Domain, Information-Providing, Unexpected Situations, Verbal Alignment

The contribution of this paper is twofold: 1) we provide a public corpus for Human-Agent Interaction (where the agent is controlled by a Wizard of Oz) and 2) we show a study on verbal alignment in Human-Agent Interaction, to exemplify the corpus' use. In our recordings for the Human-Agent Interaction Alice-corpus (HAI Alice-corpus), participants talked to a wizarded agent, who provided them with information about the book Alice in Wonderland and its author. The wizard had immediate and almost full control over the agent's verbal and nonverbal behavior, as the wizard provided the agent's speech through his own voice and his facial expressions were directly copied onto the agent. The agent's hand gestures were controlled through a button interface. Data was collected to create a corpus with unexpected situations, such as misunderstandings, (accidental) false information, and interruptions. The HAI Alice-corpus consists of transcribed audio-video recordings of 15 conversations (more than 900 utterances) between users and the wizarded agent. As a use-case example, we measured the verbal alignment between the user and the agent. The paper contains information about the setup of the data collection, the unexpected situations and a description of our verbal alignment study.

Introduction

This paper presents the Human-Agent Interaction Alicecorpus (HAI Alice-corpus), a corpus of conversations between a user and an embodied conversational agent (ECA) operated by a wizard via the Wizard of Oz (WOz) method [START_REF] Dahlbäck | Wizard of Oz studies -why and how[END_REF]. The corpus has been collected as part of the ARIA-VALUSPA 1 project, in which a multimodal virtual agent for information retrieval that can deal with unexpected situations is being developed. The goal of the project is to provide interested parties, such as fellow researchers and industry, with a toolkit for building their own virtual agent [START_REF] Valstar | Ask Alice: an artificial retrieval of information agent[END_REF][START_REF] Bruijnes | Keeping up stories: design considerations for a police interview training game[END_REF]. We collected the corpus to investigate how users react to unexpected situations in a conversation with an autonomous state-of-the-art virtual agent. However, in a classic WOz approach where the wizard uses a button interface, it is nearly impossible to improvise in unexpected situations. This is why we gave our wizard the freedom to choose his own words and facial expressions, to create and respond to unexpected situations. To the best of our knowledge most state-of-the-art agents cannot yet cope with interruptions effectively, nor perform human-level verbal alignment [START_REF] Dubuisson Duplessis | Automatic Measures to Characterise Verbal Alignment in Human-Agent Interaction[END_REF]. However, we believe we are approaching such capabilities in agents, thanks to recent efforts in agent development. Therefore, we wanted to simulate an agent as closely as possible with this type of WOz setup. We analyzed the corpus for verbal alignment in Human-Agent (H-A) interaction. Verbal alignment is a process during a dialog where participants reuse lexical and syntactic structures [START_REF] Pickering | Toward a mechanistic psychology of dialogue[END_REF]. One example of reusing structures is by speaking with similar words in the conversation as other dialogue partici-1 www.aria-agent.eu [START_REF] Reitter | Computational modelling of structural priming in dialogue[END_REF]. First, we will describe some related corpora in Section 2. In Section 3 we will provide more details on the setup of our Wizard of Oz system. More details on the data collection will be given in Section 4. Section 5 will be a description of what will be released with the corpus, and an example of a dialog from the corpus. In Section 6 we briefly describe our use-case for the corpus on the topic of verbal alignment. In Section 7 we will discuss the limitations of the corpus, suggestions for its use, and some future studies.

Related Work

More and more dialog corpora are available involving Human-Human interactions as well as Human-Machine interactions; see [START_REF] Serban | A survey of available corpora for building data-driven dialogue systems[END_REF] for a recent and extensive study. However, a closer look at the Human-Machine corpora reveals corpora are mostly available for spoken dialog systems, but very few are available in the Human-Agent community. The biggest difference between these corpora is that agents are often embodied and use multi-modal information, compared to only speech in spoken dialog systems. A well-known example of a corpus in H-A is the SEMAINE corpus [START_REF] Mckeown | The SEMAINE corpus of emotionally coloured character interactions[END_REF], dedicated to emotions in H-A Interactions. Recent efforts in collecting interaction data between a chatbot and a user include the UCAR corpus [START_REF] Dubuisson Duplessis | Purely corpus-based automatic conversation authoring[END_REF] 2 , as well as the (RE-)WOCHAT international effort, which aims at collecting and annotating dialog data for chatbots and conversational agents (D 'Haro et al., 2016) 3 . We believe that more H-A corpora can be very useful to analyze H-A interaction, in order to improve agents' communicative capabilities (e.g. a corpora study about adaptation and verbal alignment in H-A Interaction [START_REF] Dubuisson Duplessis | Automatic Measures to Characterise Verbal Alignment in Human-Agent Interaction[END_REF]). This paper and the associated HAI Alice-corpus are a humble step towards sharing such valuable corpora for the H-A Interaction community.

Wizard of Oz System

The goal of the WOz system was to allow a user to have an interaction with an accomplice, who appears to the user as a virtual agent, and to record this interaction. Additionally, the user should believe (s)he is interacting with an advanced autonomous agent system4 .

Wizard and Agent

The wizard and participant were located in different rooms. The wizard could see and hear the participant through a Skype video-conference connection. The agent was rendered in real-time on the wizard's PC, and sent over the Skype connection to the participant. The wizard was trained to speak like a robot, with little pitch variation and a rhythm that is characteristic of generated speech. Additionally, using VoiceMeeter5 and Reaper6 , his voice was passed through audio filters to make it sound more robotic. The facial animations of the agent were controlled using the wizard's facial expressions. The Unity game engine, using the FacePlus plugin by Mixamo7 , mapped the wizard's expressions on to the face of the agent. The body postures and gestures of the agent were controlled by the wizard through a button interface. With these tools, our wizard was able to improvise in unexpected situations, which would not have been possible in a button interface for classic wizards.

Recording

The system allowed for multi-modal data collection on the interaction between the user and wizard. The audio-video

Domain

The conversational topic of our corpus is Alice's Adventures in Wonderland (also known as Alice in Wonderland), the famous novel of Lewis Carroll. For the data collection experiment, a virtual agent controlled by a wizard acted as an expert on Alice's Adventures in Wonderland (AiW). The agent was standing in a library and had the appearance of a middle-aged male intellectual, as can be seen in Figure 1. The wizard controlling the agent was well informed on the topic of AiW. The agent's utterances were prepared beforehand, based on possible questions that the agent might be asked during the interaction. To prepare for possible questions, a survey was held before the experiment and a pilot study was conducted via Reddit. Additionally, agent utterances to suggest topics were generated. The wizard studied these utterances, so that he could respond as fast as possible. He was allowed to paraphrase the utterances, as long as the meaning would remain the same. The participant, uninformed of the WOz setup, was instructed to ask questions about AiW that the agent had knowledge about: the events of the book, its characters, the author Lewis Carroll, and some other information around the book, such as its adaptations in films and others media.

Experimental Variations

Two variations of the AiW scenario exist. Condition A was intended to prime the user to try out the agent's capabilities rather than asking relevant questions on the book. Condition B was intended to be a credible version of an advanced and knowledgeable virtual agent that would increase the users' curiosity on the book.

• Condition A: the agent was not knowledgeable on the book, often repeated the same sentences ("I unfortunately don't know about that"), and intentional mistakes (such as using the wrong gender when saluting, or giving an incorrect answer to a question);

• Condition B, the agent answered questions to the best of its ability (and hid his ignorance when he could not answer), tried not to repeat himself, and took more time before answering.

Unexpected Situations

One aim was to record how participants respond to unexpected situations during an interaction with a virtual agent. We intentionally introduced sources of distraction and mistakes by the agent. These mistakes reflect technical limitations that exist in current state-of-the-art systems and that therefore often occur during a conversation with an autonomous agent. In particular, we created the following unexpected situations: Noise. Video and audio signals often contain recognition errors. This leads in autonomous agents to social signal processing and speech recognition with low accuracy and thus to an agent that does not understand the user. In the corpus this was mimicked by the wizard by saying "Sorry, I did not get that" when he did not understand the question (e.g., #6 in Table 1)Ṁ issing Answers. Even with perfect recognition scores, autonomous agents do not know the answer to every question the user can pose and often have to say, for example, "Sorry, I don't know that". In the corpus the wizard said this when a question or answer was not in the script. Wrong Responses.

Autonomous agents sometimes respond in an inappropriate manner. For example, in an information-retrieval or query-answer matching approach the utterance of a user is mapped to existing query-answer pairs. The QA-pair with the most similar query to the user's utterance is selected and the agent performs a response with the answer of this pair. While easy to implement and robust in simple question-answering applications, this approach sometimes selects a 'wrong' answer (see for example [START_REF] Schooten | Vidiam: Corpus-based development of a dialogue manager for multimodal question answering[END_REF]). The wizard also used query-answer pairs to select the answer for the agent. In addition, sometimes the wizard intentionally gave a wrong answer to elicit a response from the user to this type of unexpected situation. An example of an inappropriate answer is #10 in Table 1. Gender Detection. Autonomous agents' mistakes can lead to socially awkward situations. For example, referring to the gender of the user after misclassifying it. In the corpus the gender of the user was intentionally classified incorrectly (e.g., #2 in Table 1) to be able to record the reaction of users to this faux pas. Interruptions. Interruptions in human-human conversations occur frequently and often without negatively impacting the flow of conversation. In H-A conversations interruptions are uncommon and often lead to a breakdown in communication when they do occur (op den Akker and [START_REF] Op Den Akker | Computational models of social and emotional turn-taking for embodied conversational agents: a review[END_REF]). An interruption is an unexpected situation for an autonomous agent, but an agent successfully dealing with an interruption might be an unexpected situation for a user. Therefore, an (external) interruption was added to the corpus. An accomplice asked the participant whether they wanted a drink about four minutes into the conversation. This forced the participant to shift his attention and shortly lose engagement in the conversation. Additionally, the wizard commented on the interruption. This can be seen in Table 1, #14-15.

Recording Protocol

Participants were asked to read the two first pages of Alice's Adventures in Wonderland, before having a 7 minute conversation with the 'virtual expert'. Participants signed an informed consent form and filled in a short questionnaire, asking for their demographics, and knowledge on virtual agents and the AiW book. Participants were seated in front of a computer screen showing the agent and a Kinect camera. The experiment leader started the recordings and then left the room. The wizard, and thus the agent, started smiling and greeted the user, starting the conversation. After four minutes the experiment leader introduced an unexpected situation. He entered the room to interrupt the conversation by asking if the participant would like something to drink and left after the participant had responded. Three minutes later the experiment leader entered the room again to end the experiment. Finally, participants were debriefed, given the opportunity to ask questions about the experiment, and were introduced to the wizard.

Description of the Corpus

The corpus is available under the CC 4.0 license. We transcribed the corpus and distributed the transcription work amongst ourselves evenly (i.e. each author transcribing four dialogs) and one of the authors reviewed the full set of transcriptions afterwards for consistency.

Participants

A total of 16 volunteers were recruited to participate in the recording of this corpus. The participants of this study were divided across two conditions, A and B. In condition A there were 7 male and 1 female participants, of which 2 were native English speakers, with an average age of 30.38. In condition B, there were 4 male and 4 female participants, no native English speakers, with an average age of 34.13. One participant in condition A did not give permission to include their interaction in the corpus and has therefore been excluded.

Contents

The video files of the H-A Interactions are available. Anonymized transcriptions of the dialogs are included and contain the utterances of the user, the agent, and the experimenter. Annotations are provided on the unexpected situations that were introduced in the interaction, see Table 2. The corpus is quantitatively described in Tables 3 and

Verbal Alignment in H-A Interactions

One striking observation of Human-Human (H-H) Interaction is that the communicative behaviors of dialog participants (DPs) tend to converge [START_REF] Gallois | Communication accommodation theory: A look back and a look ahead[END_REF] and automatically align at several levels (such as the lexical, syntactic and semantic ones) [START_REF] Pickering | Toward a mechanistic psychology of dialogue[END_REF]. In particular, DPs tend to reuse lexical and syntactic structures [START_REF] Reitter | Computational modelling of structural priming in dialogue[END_REF]. One consequence of successful alignment at several levels between DPs is a certain repetitiveness in dialog, leading to the development of a lexicon of fixed expressions. As a matter of fact, dialog participants tend to automatically establish and use fixed expressions that become dialog routines. Alignment is a subconscious phenomenon that naturally occurs in H-H dialogs [START_REF] Pickering | Toward a mechanistic psychology of dialogue[END_REF]. It has been shown to facilitate successful task-oriented conversations [START_REF] Nenkova | High frequency word entrainment in spoken dialogue[END_REF][START_REF] Friedberg | Lexical entrainment and success in student engineering groups[END_REF]. Linguistic alignment occurs in H-A Interactions. Indeed, users adopt lexical items and syntactic structures used by a system [START_REF] Brennan | Lexical entrainment in spontaneous dialog[END_REF][START_REF] Stoyanchev | Lexical and syntactic priming and their impact in deployed spoken dialog systems[END_REF][START_REF] Parent | Lexical entrainment of real users in the Let's Go spoken dialog system[END_REF][START_REF] Branigan | Linguistic alignment between people and computers[END_REF]. However, this alignment is only one-way: the system is usually not able to align on the user. While verbal alignment has been investigated for H-H Interactions, it has been studied less for H-A Interaction.

Recent work aims at studying H-A Interaction corpora to characterize the verbal alignment process [START_REF] Dubuisson Duplessis | Automatic Measures to Characterise Verbal Alignment in Human-Agent Interaction[END_REF]. Their study contrasts H-H and H-A Interaction corpora on a negotiation task over a definite set of objects (based on the H-H and H-A negotiation corpora [START_REF] Gratch | The Benefits of Virtual Humans for Teaching Negotiation[END_REF], unfortunately not publicly available). Among other things, they show that verbal alignment is symmetrical in H-H Interactions at the level of lexical repetitions, while it is asymmetrical in H-A Interactions. More specifically, they have shown that the human participant verbally aligns more, by adopting more agent-initiated lexical patterns, and by dedicating more tokens to the repetition of previously employed lexical patterns. In this work, we aim at employing the same measures on the Alice corpus, to study the verbal alignment process in H-A interaction on an information-providing task driven by a sophisticated WOz able to adapt to the user.

Approach

Recent work proposes automatic verbal alignment measures based on the repetition between speakers at the lexical level [START_REF] Dubuisson Duplessis | Automatic Measures to Characterise Verbal Alignment in Human-Agent Interaction[END_REF]. In particular, they focus on which words and lexical patterns are shared between dialog participants. DPs are said to verbally align when they share and use common lexical pat-terns ranging from single words (e.g., "Alice", "cat") to more elaborated patterns (e.g., "white rabbit", "Alice in Wonderland", "would like to know about"). This approach describes global and speaker-specific measures of verbal alignment based on repetition at the lexical level in dialog transcripts. It operates by mining lexical patterns (following the work of (Dubuisson Duplessis et al., 2017a)), automatically building a lexicon of shared expressions and deriving verbal alignment measures. The expression lexicon keeps track of shared expressions in a dialog and valuable features about these expressions (e.g., who first produced this expression, its frequency). Then, straightforward measures are derived by leveraging both the dialog transcript and the dialog lexicon. In this paper, we focus on the following verbal alignment measures proposed in (Dubuisson Duplessis et al., 2017b): Expression Variety (EV). The total number of tokens in the dialog. This ratio indicates the variety of the expression lexicon relatively to the length of the dialog. The higher it is, the more there are different expressions established between DPs. Expression Repetition (ER). Ratio of produced tokens belonging to a repetition of shared expressions. The higher the ER is, the more DPs dedicate tokens to the repetition of expressions. Initiated Expression (IE S ). Ratio of shared expressions initiated by locutor S. Expression Repetition (ER S ). Ratio of produced tokens belonging to the repetition of a shared expressions for locutor S. The software used to compute the measures is free and open-source 8 .

Preliminary Results

We first look at the speaker-independent measures. The expression variety (EV) ranges from approx. 0.07 to 0.12 (mean=0.095, std=0.014, median=0.098). This is less than what is reported for the H-A negotiation corpus in [START_REF] Dubuisson Duplessis | Automatic Measures to Characterise Verbal Alignment in Human-Agent Interaction[END_REF]. This indicates that DPs constitute less varied expression lexicons in the informationproviding dialogs than in the negotiation ones. The expression repetition (ER) ranges from approx. 0.23 to 0.40 (mean=0.330, std=0.048, median=0.346). Once again, this is less than what is reported for the H-A negotiation corpus. This indicates that DPs in the Alice corpus dedicate fewer tokens to the repetition of shared expressions. Next, we take a closer look at each speaker in the dialog in terms of initiated expressions (IE S ) and expression repetition (ER S ). A clear asymmetry between the agent and the human appears at the initiation of shared expressions (see Figure 2). Here, the agent initiates most of the shared expressions with an IE A ranging from 0.50 to 0.72 (mean=0.61, std=0.06, median=0.61). This difference is statistically significant (Wilcoxon signed rank test, V = 120, p-value=6.104 × 10 -5 ). However, this asymmetry does not appear at other measurable levels. Indeed, the agent and the users dedicate the same number of tokens for repeating expressions (ER S ), produce a comparable number of tokens in dialog and relatively share the same amount 8 Software available at:

https://github.com/ GuillaumeDD/dialign of vocabulary with each other 9 . We have not found any significant difference in the verbal alignment measures between conditions A and B. This is possibly due to the small sizes of these groups (7 dialogs for condition A, 8 for condition B).

Discussion of the Use-Case

In this section, we have provided a preliminary quantitative study of verbal alignment on the Alice corpus with a comparison to previous work. This study is one of the few that focuses on verbal alignment in H-A interaction. Its originality lies in the nature of the corpus (informationproviding dialogs), and in the sophisticated WOz system that allows the wizard to adapt to the user. This constitutes a step towards a better understanding of verbal alignment processes in Human-Agent Interaction. Globally, our preliminary study indicates that the Alice corpus presents a quantitatively weaker verbal alignment process at the level of lexical pattern repetition than the H-A negotiation corpus studied in [START_REF] Dubuisson Duplessis | Automatic Measures to Characterise Verbal Alignment in Human-Agent Interaction[END_REF]. This is shown by the emergence between DPs of less varied expression lexicons, and by the fact that DPs dedicate less tokens to repeating lexical patterns. At the speaker level, it turns out that the Alice corpus displays an asymmetry between the agent and the human; which is a feature that seems to discriminate H-H interactions from H-A ones in terms of verbal alignment [START_REF] Dubuisson Duplessis | Automatic Measures to Characterise Verbal Alignment in Human-Agent Interaction[END_REF]. Here, the agent initiates more shared expressions than the human participant. One explanation is that the agent leads the interaction and often triggers the questions (e.g., "Is there anything you would like to know about falling jars?"). However, the agent and the user repeat lexical patterns to the same degree. This shows 9 Relative shared vocabulary for S1is computed as follow:

SV S1 = #(Tokens S1 ∩ Tokens S2 ) #(Tokens S1 )
a tendency towards a symmetrical verbal alignment, which is closer to what has been previously observed in H-H Negotiation Interactions. In other words, this indicates a more "human-like" verbal alignment in the Alice corpus compared to the H-A Negotiation corpus. One explanation can be found in the WOz system. In the Alice corpus, the WOz operator is not strictly constrained in its linguistic choices, and can thus verbally align with the user. Conversely, the WOz system in the negotiation corpus is restricted to preformatted utterances and templates, which prevent the WOz from aligning with the user. All in all, this preliminary study indicates that variations in verbal alignment at the level of shared lexical patterns not only can be quantified between H-H and H-A Interactions but also between H-A corpora. Further studies are thus required to better understand the verbal alignment processes occurring in H-A Interactions, and to improve the adaptive communicative capabilities of agents interacting with humans.

Discussion and Conclusion

We have described a Wizard of Oz setup and corpus (HAI Alice-corpus) in an information-providing and closeddomain setting, containing multiple types and variants of unexpected situations. There were two conditions, one in which the user would focus more on the agent behavior and one where the user would focus more on the content of the dialog. We expect this corpus to be a useful resource for the Human-Agent Interaction community, because few similar corpora exist. In the full paper we will describe a study on verbal alignment in human-machine interaction to showcase the usefulness of the corpus. We are aware of the limitations of this corpus. The size of the HAI Alice-corpus is quite small, it contains 15 dialogs. However, the corpus might be combined with other corpora to create a larger dataset. Furthermore, we have only transcribed the speech of the conversations and we did not annotate the data with time markers corresponding to the video. Lastly, non-verbal behaviors have not been annotated even though these could contain interesting information for other researchers. Currently, we are integrating a verbal alignment tool into the ARIA-VALUSPA platform. This will allow an autonomous agent to utilize verbal alignment strategies in conversations with a user. We will use the HAI Alicecorpus, and in particular the verbal alignment scores from the corpus, to evaluate the performance of the autonomous agent on verbal alignment.

Figure 1 :

 1 Figure 1: The conversational agent operated by the wizard

Figure 2 :

 2 Figure 2: Initiated Expressions (IE S ). "A" is the agent, "H" is the human participant. The difference is significant (p < 0.001).

Table 1 :

 1

	#	Sp. Utterance
	1	U	Hello
	2	A	Hello madam, my name is James
	3	U	I'm not a madam, but okay
	4	A	I know everything about "Alice in Wonder-
			land." Is there anything you would like to
			know?
	5	U	Who wrote it?
	6	A	Sorry, I did not get that
	7	U	Who wrote "Alice in Wonderland"?
	8	A	"Alice in Wonderland" was written by
			Charles [indistinguishable audio], or Lewis
			Carroll, which was his pen name
			[. . . ]
	9	U	So have you read the whole book, the "Alice
			in Wonderland"?
	10 A	Yes, I am an expert of "Alice in Wonderland"
	11 U	Yes, but have you read it?
	12 A	Of course.
	13 U	For how long was Alice falling?
	14 E	[interruption] I'm sorry. Do you like some
			water, coffee or tea?
	15 A	Don't you think it's rude to give attention to
			someone else during our conversation?
	16 U	Given the fact that you're not a human, no
	17 A	Okay. Is there something you would like to
			know about the animated picture?

An excerpt from the corpus. (Sp=Speaker, U=User, A=Agent, E=Experiment leader)

Table 4 :

 4 Figures about the collected dialogs.

	Tag	Attributes	Explanation
	<dialog>	id,	condition,	the type of condition (A or B), the duration of the conversation and the
		duration, gender	gender of the participant
	<part>	type		parts of the experiment (before, during or after the interruption)
	<utterance> id, speaker	the text of the dialogs and who is speaking (user or agent)
	<overlap>	id		used to annotate overlapping utterances (e.g., during interruptions)
	<noise>	n/a		tag for responses of the agent because of noise
	<missing>	n/a		tag for responses of the agent not knowing a response
	<wrong>	n/a		tag for intentionally wrong responses by the agent
			Table 2: Tags used in the XML data files
	dialogs	15
	Utterances	989 (A: 482, H: 487, E: 20)
	Unique utterances 836 (A: 373, H: 450, E: 19)
	Tokens	9900 (unique: 810)
	Table 3: Figures about the collected corpus. (A=Agent,
	H=Human, E=Experimenter)
	Type		average (std/min/max)
	Utterances		65.93 (10.34 / 48 / 80)
	Tokens		660 (93.71 / 502 / 914)
	Tokens/human		316.8 (120.64 / 131 / 633)
	Tokens/system		329.67 (51.96 / 225 / 441)
	Tokens/human utterance 9.76 (7.86 / 1 / 60)
	Tokens/system utterance 10.26 (6.07 / 1 / 50)

Available at https://ucar.limsi.fr

http://workshop.colips.org/wochat/data/

For this reason we refer to the embodiment as an agent, and not as an avatar.

http://www.vb-audio.com/Voicemeeter/

https://www.reaper.fm/

https://www.mixamo.com. Unfortunately, the Face-Plus plugin has been removed from the Mixamo package since August 2017: http://blogs.adobe.com/adobecare/ 2017/05/23/download-assets-from-mixamo/ data in this corpus consists of the audio of both the wizard and the participant and the video of the agent.4 Data CollectionIn this section we will explain the scenario in the corpus, the variations and manipulations, information about the participants and the experiment protocol.

We also include anonymized information about the participants, such as their demographic information and their familiarity with virtual agents.6 Example Use-Case: Verbal AlignmentIn this section, we will present a corpus study to show the importance of having an H-A Interaction corpus to make analysis of such interactions possible; and improve agent communicative capabilities.

Acknowledgements

This project has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No. 645378. Special thanks to Siewart van Wingerden and Adrien Luxey for their efforts in recording the corpus.


Keywords:

We describe the coherent beam combining of three high-power tapered laser amplifiers seeded by a DFB laser at λ = 976 nm, and demonstrate a combined power of 12.9 W in a close to diffraction-limit beam at a combining efficiency of > 65%. We use individual devices on C-mounts in a multi-arm interferometer. The phase pistons of each beam are matched by automatically controlling the ridge current of the amplifiers. Increased power per element was achieved by making use of a simplified, efficiently cooled single emitter-based optical system.

I. INTRODUCTION

igh power diode lasers provide the energy for many laser systems and benefit from a high electrical-to-optical efficiency. Such systems are usually based on incoherent beam combining technologies [START_REF] Fan | Laser beam combining for high-power, high-radiance sources[END_REF], and recent developments of spectrally combined diode lasers led to kW class systems suitable for direct diode laser applications such as sheet metal cutting [START_REF] Witte | kW-class direct diode laser for sheet metal cutting based on commercial pump modules[END_REF]. Even if the brightness of such systems has been dramatically increased within the last decade, it is still an important challenge to increase the available power close to the diffraction limit. First demonstrations of coherent beam combining (CBC) of diode lasers at comparably high power levels (~40 W) were based on very large arrays of low power amplifiers in Master Oscillator Power Amplifiers (MOPA) configuration [START_REF] Creedon | High efficiency coherent beam combining of semiconductor optical amplifiers[END_REF]. On the one hand, arrays of amplifiers are an elegant solution to scale the number of active devices, and the phase relation between these elements is usually more stable than for individually mounted emitters as they are exposed to approximately the same fluctuations of the temperature. On the other hand, the achievable power per amplifier of such systems is significantly lower compared to the performance of individual emitters. We are therefore currently working on MOPA-CBC architectures based on a small number of individual emitters allowing to reach comparable power levels with a significantly reduced number of elements and a smaller footprint. We investigate CBC of tapered laser amplifiers because their high power and excellent beam quality is a perfect start point for power scaling [START_REF] Crump | Development of high-power diode lasers with beam parameter product below 2 mm×mrad within the BRIDLE project[END_REF].

II. AMPLIFIER CHARACTERIZATION

The amplifiers were grown on GaAs substrates using MOVPE, fabricated into individual, 6 mm-long devices using i-line lithography. In the studies presented here, we made use of tapered amplifiers identical to lasers described in [START_REF] Fiebig | 12 W high-brightness singlefrequency DBR tapered diode laser[END_REF], but with the internal grating section replaced by a long (2mm) ridge waveguide section. Both facets were passivated and antirefection coated. The amplifiers are mounted p-side up on CuW heat spreaders and C-mounts, with separate electrical contacts used in order to control the drive current of ridge and tapered section independently.

Each of our amplifier elements reaches about 6.5 W at 10 A drive-current and a heat sink temperature of 20°C. The beam quality at this operating point is slightly degraded but at Coherent beam combining of high-power tapered amplifiers P. Albrodt least 70% of the total power (~4.5 W) was delivered within a close to diffraction limited beam (termed the central lobe, see [START_REF] Crump | Development of high-power diode lasers with beam parameter product below 2 mm×mrad within the BRIDLE project[END_REF][START_REF] Fiebig | 12 W high-brightness singlefrequency DBR tapered diode laser[END_REF]). Another important property for CBC is the stability of the phase. We therefore investigated the phase noise in a Mach-Zehnder interferometer. The measured noise spectrum for a tapered laser amplifier described above is shown in figure 2. Low frequencies (f < 10 Hz) are clearly dominating the noise spectrum and are attributed to thermal fluctuations. We measured a derivation of the phase in the range of 8 rad over an observation time of 200 s. The standard deviation over one second is in the range of 𝜎(𝜑) = 0.1 𝑟𝑎𝑑 ≈ 𝜋 30 ⁄ , which corresponds to the integrated phase noise for f > 1 Hz at the maximum operating current. Phase fluctuations in this order of magnitude can easily be controlled by a feedback loop, so integration into a mini-array on a single heatsink is not necessary to achieve sufficient phase stability for effective coherent combination.

III. COHERENT COMBINING IN MOPA ARCHITECTURE

We present MOPA-CBC of three individual amplifiers in a simple experimental setup shown in figure 2 (left), based on a multi-arm Mach-Zehnder interferometer. The MO is a narrowband distributed feedback (DFB) diode laser (λ = 976 nm) optically isolated by Faraday isolators with a total isolation of better than 50 dB. The beam is split and recombined by standard 50:50 beam splitters in three arms.

Though the 50:50 coating is not optimal for three beams, its impact on the theoretically achievable combining efficiency is only around 3%. The phase in each arm is controlled by measuring the combined intensity with a photodiode and changing the ridge currents in the range of a few tens of mA, which has a negligible effect on the power output per amplifier. The control loop is implemented in a microcontroller and uses a sequential hill-climbing algorithm with adaptive steps. The standard deviation of the power fluctuations in the stabilized regime is below 1%.

The combined power and combining efficiency were measured for several operating points (figure 2, right). With 12.9 W coherently combined in one nearly diffraction-limited beam at the maximum current, we are able to reach higher power levels than previously demonstrated by CBC of an array of five tapered lasers [START_REF] Schimmel | Coherent beam combining architectures for high power tapered laser arrays[END_REF]. The optical power was measured simultaneously at the output (combined beam, Pout) and the two unusable paths (combining losses, L1 and L2). The combining efficiency was defined as η = 𝑃 𝑜𝑢𝑡 (𝑃 𝑜𝑢𝑡 + 𝐿 1 + 𝐿 2 ) ⁄ and was 75% for low power levels and decreased to 65% at the maximum power. The relative power in the combined beam is comparable to the diffraction limited power the beam from each amplifier. The emission in the central lobe (~ 70 % of total power) has a small mismatch between beams and can efficiently combined. Emission outside the central lobe (~ 30 % of total power) shows in contrast a large mismatch between beams and leads therefore to high losses for CBC.

In summary, MOPA-CBC architectures based on individually mounted tapered laser amplifiers can deliver high power per element. The power per element is increased over that reported in our previous studies of arrays. We note however that the device design and mounting are quite different in the single emitters tested here and the arrays used in [START_REF] Schimmel | Coherent beam combining architectures for high power tapered laser arrays[END_REF], making exact comparisons challenging. We note also that the single emitter based setup presented in this paper is simple and has a high potential to be integrated in small footprint modules. In future work, we will seek increased power per element via improved packaging and further improved semiconductor device design.

5 ]

 5 fourier frequency[Hz]

Fig 1 :

 1 Fig 1: Measured phase noise spectrum of a tapered laser amplifier (ridge Irw = 400mA, taper Itp = 7A, injection Pin = 10 mW, heatsink T = 20°C).

Fig. 2 :

 2 Fig. 2: Experimental setup (left), measured combined power and combining efficiency (right)

  1* , M. Hanna 1 , F. Moron 1 , J. Decker 2 , M. Winterfeldt 2 , G. Blume 2 , P. Crump 2 , P. Georges 1 and G. Lucas-Leclin 1 1 Laboratoire Charles Fabry, Institut d'Optique Graduate School, CNRS, Université Paris-Saclay, Palaiseau, France ( * philipp.albrodt@institutoptique.fr) 2 Ferdinand-Braun-Institut, Leibniz-Institut für Höchstfrequenztechnik (FBH), Berlin, Germany