Yutong Ban

Sileye Ba

Xavier Alameda-Pineda

Radu Horaud

Tracking Multiple Persons Based on a Variational Bayesian Model

Object tracking is an ubiquitous problem in computer vision with many applications in human-machine and human-robot interaction, augmented reality, driving assistance, surveillance, etc. Although thoroughly investigated, tracking multiple persons remains a challenging and an open problem. In this paper, an online variational Bayesian model for multiple-person tracking is proposed. This yields a variational expectation-maximization (VEM) algorithm. The computational efficiency of the proposed method is due to closed-form expressions for both the posterior distributions of the latent variables and for the estimation of the model parameters. A stochastic process that handles person birth and person death enables the tracker to handle a varying number of persons over long periods of time. The proposed method is benchmarked using the MOT 2016 dataset.

Introduction

The problem of object tracking is ubiquitous in computer vision. While many object tracking methods are available, multiple-person tracking remains extremely challenging [START_REF] Luo | Multiple object tracking: a review[END_REF]. In addition to the difficulties related to single-object tracking (occlusions, self-occlusions, visual appearance variability, unpredictable temporal behavior, etc.), tracking a varying and unknown number of objects makes the problem more challenging, for the following reasons: (i) the observations associated with detectors need to be associated to objects being tracked, which includes the process of discarding detection errors, (ii) the number of objects is not known in advance and hence it must be estimated and updated over time, (iii) mutual occlusions (not present in single-tracking scenarios) must be robustly handled, and (iv) the number of objects varies over time and one has to deal with hidden states of varying dimensionality, from zero when there is no visible object, to a large number of detected objects. Note that in this case and if a Bayesian setting is being considered, as is often the case, an exact recursive filtering solution is intractable.

Several multiple-person tracking methods have been proposed within the trans-dimensional Markov chain model [START_REF] Green | Trans-dimensional Markov chain Monte Carlo[END_REF], where the dimensionality of the statespace is treated as a state variable. This allows to track a variable number of objects by jointly estimating the number of objects and their states. [START_REF] Khan | An MCMC-based particle filter for tracking multiple interacting targets[END_REF][START_REF] Smith | Using particles to track varying numbers of interacting people[END_REF][START_REF] Yang | A probabilistic framework for multitarget tracking with mutual occlusions[END_REF] exploited this framework for tracking a varying number of objects. The main drawback is that the states are inferred by means of a reversible jump Markov-chain Monte Carlo sampling, which is computationally expensive [START_REF] Green | Reversible jump Markov chain Monte Carlo computation and Bayesian model determination[END_REF]. The random finite set framework proposed in [START_REF] Mahler | Multisource multitarget filtering: a unified approach[END_REF][START_REF] Mahler | Statistics 101 for multisensor, multitarget data fusion[END_REF][START_REF] Mahler | Statistics 102 for multisensor multitarget data fusion[END_REF] is also very popular, where the targets are modeled as realizations of a random finite set which is composed of an unknown number of elements. Because an exact solution to this model is computationally intensive, an approximation known as the probability hypothesis density (PHD) filter was proposed [START_REF] Mahler | A theoretical foundation for the Stein-Winter" probability hypothesis density (PHD)" multitarget tracking approach[END_REF]. Further sampling-based approximations of random-set based filters were subsequently proposed, e.g. [START_REF] Sidenbladh | Multi-target particle filtering for the probability hypothesis density[END_REF][START_REF] Clark | Convergence results for the particle PHD filter[END_REF][START_REF] Vo | Random finite sets and sequential monte carlo methods in multi-target tracking[END_REF]. These were exploited in [START_REF] Ma | Tracking an unknown time-varying number of speakers using TDOA measurements: a random finite set approach[END_REF] for tracking a time-varying number of active speakers using auditory cues and in [START_REF] Maggio | Efficient multitarget visual tracking using random finite sets[END_REF] for multiple-target tracking using visual observations. Recently, conditional random fields have been introduced to address multiple-target tracking [START_REF] Yang | An online learned CRF model for multi-target tracking[END_REF][START_REF] Milan | Continuous energy minimization for multitarget tracking[END_REF][START_REF] Heili | Exploiting long-term connectivity and visual motion in CRF-based multi-person tracking[END_REF]. In this case, tracking is cast into an energy minimization problem. In radar tracking, popular multiple-target tracking methods are joint probabilistic data association (JPDA), and multiple hypothesis filters [START_REF] Bar-Shalom | The probabilistic data association filter: estimation in the presence of measurement origin and uncertainty[END_REF].

An interesting and less investigated framework for multiple-target tracking is the variational Bayesian class of models for tracking an unknown and varying number of persons. Although variational models are very popular in machine learning, their use for object tracking has been limited to tracking a fixed number of targets [START_REF] Vermaak | Variational inference for visual tracking[END_REF]. Variational Bayes methods approximate the joint a posteriori distribution of the complete set of latent variables by a separable distribution [START_REF] Smidl | The Variational Bayes Method in Signal Processing[END_REF][START_REF] Bishop | Pattern Recognition and Machine Learning (Information Science and Statistics)[END_REF]. In an online tracking scenario, where only past and current observations are available, this leads to approximating the filtering distribution. An interesting aspect of variational methods is that they yield closed-form expressions for the posterior distributions of the hidden variables and for the model parameters, thus enabling an intrinsically efficient filtering procedure implemented via a variational EM (VEM) algorithm. In this paper, we derive a variational Bayesian formulation for multiple-person tracking, and present results on the MOT 2016 challenge dataset [START_REF] Milan | Mot16: A benchmark for multi-object tracking[END_REF]. The proposed method extends [START_REF] Ba | An on-line variational Bayesian model for multi-person tracking from cluttered scenes[END_REF] in many apsects: (i) the assignment variables are included in the filtering equation and therefore the state variables and the assignment variables are jointly inferred, (ii) a temporal window is incorporated in the visibility process, leading to a tracker that is more robust to misdetections, (iii) death process allows to forget about old tracks and thus opens the door to large-scale processing, as needed in many realistic situations. Finally, full evaluation of the proposed tracker within the MOT 2016 challenge dataset assesses its performance against other state-of-the-art methods in a principled and systematic way. Examples of results obtained with our method and Matlab code are publicly available. 4The remainder of this paper is organized as follows. Section 2 details the proposed Bayesian model and a variational solution is presented in Section 3. In Section 4, we depict the birth, visibility and death processes allowing to handle an unknown and varying number of persons. Section 5 presents benchmarking results. Finally, 6 draws conclusions.

Variational Multiple-Person Tracking

We start by introducing our notations. Vectors and matrices are in bold A, a, scalars are in italic A, a. In general random variables are denoted with uppercase letters, e.g. A and A, and their realizations with lower-case letters, e.g. a and a.

Let N be the maximum number of persons. A track n ≤ N at time t is associated to the existence binary variable e tn taking the value e tn = 1 if the person has already been seen and e tn = 0 otherwise. The vectorization of the existence variables at time t is denoted by e t = (e t1 , ..., e tN ) and their sum, namely the effective number of tracked persons at t, is denoted by N t = N n=1 e tn . The existence variables are assumed to be observed in sections 3 and 4; Their inference, grounded in a birth stochastic process, is discussed in section 5.

The kinematic state of person n is a random vector X tn = (L tn , U tn ) ∈ R 6 , where L tn ∈ R 4 is the person location and size, i.e., 2D image position, width and height, and U tn ∈ R 2 is the person velocity in the image plane. The multipleperson state random vector is denoted by

X t = (X t1 , . . . , X tN ) ∈ R 6N .
We assume the existence of a person detector, providing K t localization observations at each time t. The k-th localization observation delivered by the detector at time t is denoted by y tk ∈ R 4 , and represents the location (2D position, width, height) of a person, e.g. Figure 1. The set of observations at time t is denoted by y t = {y tk } Kt k=1 . Associated to y tk , there is a photometric description of the person appearance, denoted by h tk . This photometric observation is extracted from the bounding box of y tk . Altogether, the localization and photometric observations constitute the observations o tk = (y tk , h tk ) used by our tracker. Definitions analogous to y t hold for h t = {h tk } Kt k=1 and o t = {o tk } Kt k=1 . The probability of a set of random variables is written as p(o t ) = p(o t1 , . . . , o tKt ).

We also define an observation-to-person assignment (hidden) variable Z tk , associated with each observation o tk . Z tk = n, n ∈ {1 . . . N } means that o tk is associated to person n. It is common that a detection corresponds to some clutter instead of a person. We cope with these false detections by defining a clutter target. In practice, the index n = 0 is assigned to this clutter target, which is always visible, i.e. e t0 = 1 for all t. Hence, the set of possible values for Z tk is extended to {0} ∪ {1 . . . N }, and Z tk = 0 means that observation o tk has been generated by clutter and not by a person. The practical consequence of adding a clutter track is that the observations assigned to it play no role in the estimation of the parameters of the other tracks, thus leading to an estimation robust to outliers.

The Online Tracking Model

The online multiple-person tracking problem is cast into the estimation of the filtering distribution of the hidden variables given the causal observations p(Z t , Z t-1 , X t , X t-1 |o 1:t , e 1:t ), where o 1:t = {o 1 , . . . , o t }. Importantly, we assume that the observations at time t only depend on the hidden and visibility variables at time t. The filtering distribution can be written as:

p(Z t , Z t-1 , X t , X t-1 |o 1:t , e 1:t ) = p(o t |Z t , X t , e t )p(Z t , X t |Z t-1 , X t-1 , e t )p(X t-1 , Z t-1 |o 1:t-1 , e 1:t ) p(o t |o 1:t-1 , e 1:t ) . (1)
The denominator of (1) only involves observed variables and therefore its evaluation is not necessary as long as one can normalize the expression arising from the numerator. Hence we focus on the two terms of the latter, namely the observation model p(o t |Z t , X t , e t ) and the dynamic distribution p(Z t , X t |Z t-1 , X t-1 , e t ).

The Observation Model. The joint observations are assumed to be independent and identically distributed:

p(o t |Z t , X t , e t ) = Kt k=1 p(o tk |Z tk , X t , e t ). (2)
In addition, we make the reasonable assumption that, while localization observations depend both on the assignment variable and kinematic state, the appearance observations only depend on the assignment variable, that is the person identity, but not on his/her kinematic state. We also assume the localization and appearance observations to be independent given the hidden variables. Consequently, the observation likelihood of a single joint observation can be factorized as:

p(o tk |Z tk , X t , e t ) = p(y tk , h tk |Z tk , X t , e t ) (3) = p(y tk |Z tk , X t , e t )p(h tk |Z tk , e t ).
The localization observation model is defined depending on whether the observation is generated by clutter or by a person:

-If the observation is generated from clutter, namely Z tk = 0, the variable y tk follows an uniform distribution with probability density function u(y tk ); -If the observation is generated by person n, namely Z tk = n, the variable y tk follows a Gaussian distribution with mean PX tn and covariance Σ: y tk ∼ g(y tk ; PX tn , Σ)

The linear operator P maps the kinematic state vectors onto the space of observations. For example, when X tn represents the full-body kinematic state (full-body localization and velocity) and y tk represents the full-body localization observation, P is a projection which, when applied to a state vector, only retains the localization components of the state vector. Finally, the full observation model is compactly defined by the following, where δ ij stands for the Kronecker function:

p(y tk |Z tk = n, X t , e t ) = u(y tk ) 1-etn u(y tk ) δ0n g(y tk ; PX tn , Σ) 1-δ0n etn . (4)
The appearance observation model is also defined depending on whether the observations is clutter or not. When the observation is generated by clutter, it follows a uniform distribution with density function u(h tk ). When the observation is generated by person n, it follows a Bhattacharya distribution with density defined by

b(h tk ; h n ) = 1 W λ exp(-λd B (h tk , h n )),
where λ is a positive skewness parameter, d B (•) is the Battacharya distance between histograms, h n is the reference appearance model of person n. This gives the following compact appearance observation model:

p(h tk |Z tk = n, X t , e t ) = u(h tk ) 1-etn (u(h tk ) δ0n b(h tk ; h n ) 1-δ0n ) etn . (5)
The Dynamic Distribution. Here we consider two hypotheses, firstly, we assume the at each time instance, assignment variable doesn't depends on the previous assignment. So we can factorize the the dynamic distribution into the observation-to-person prior distribution and the predictive distribution. Secondly, the kinematic state dynamics follow a first-order Markov chain, meaning that the state X t only depends on state X t-1 .

p(Z t , X t |Z t-1 , X t-1 , e t ) = p(Z t |e t )p(X t |X t-1 , e t ). (6)
The Observation-to-Person Prior Distribution. The joint distribution of the assignment variables can be factorized as:

p(Z t |e t ) = Kt k=1 p(Z tk |e t ). (7)
When observations are not yet available, given existence variables e t , the assignment variables Z tk are assumed to follow multinomial distributions defined as:

p(Z tk = n|e t ) = e tn a tn with N n=0
e tn a tn = 1. ( 8)

Because e tn takes the value 1 only for actual persons, the probability to assign an observation to a non-existing person is null. When person n is visible, a tn represents the probability of observation y tk to be generated from person n.

The Predictive Distribution. The kinematic state predictive distribution represents the probability distribution of the kinematic state at time t given the observations up to time t -1 and the existence variables p(X t |X t-1 , e t ). The predictive distribution is mainly driven by the dynamics of persons's kinematic states, which are modeled assuming that the person locations do not influence each other's dynamics, meaning that there is one first-order Markov chain for each person. Formally, this can be written as:

p(X t |X t-1 , e t ) = N n=1 p(X tn |X t-1n , e tn ). (9)
For the model to be complete, p(X tn |X t-1,n , e tn ) needs to be defined. The temporal evolution of the kinematic state X tn is defined as:

p(X tn = x tn |X t-1,n = x t-1,n , e tn ) = u(x tn ) 1-etn g(x tn ; Dx t-1,n , Λ n ) etn , (10)
where u(x tn ) is a uniform distribution over the motion state space, g is a Gaussian probability density function, D represents the dynamics transition operator, and Λ n is a covariance matrix accounting for uncertainties on the state dynamics. The transition operator is defined as:

D =   I 4×4 I 2×2 0 2×2 0 2×4 I 2×2  
In other words, the dynamics of an existing person n, either follows a Gaussian with mean vector DX t-1,n and covariance matrix Λ n , or a uniform distribution if person n does not exist. The complete set of parameters of the proposed model is denoted with

Θ = {Σ}, {Λ n } N n=1 , A 1:t , with A t = {a tn } N n=0 .

Variational Bayesian Inference

Because of the combinatorial nature of the observation-to-person assignment problem, a direct optimization of the filtering distribution (1) with respect to the hidden variables is intractable. We propose to overcome this problem via a variational Bayesian inference method. The principle of this family of methods is to approximate the intractable filtering distribution p(Z t , Z t-1 , X t , X t-1 |o 1:t , e 1:t ) by a separable distribution, e.g. q(Z t ) N n=0 q(X tn ). According to the variational Bayesian formulation [START_REF] Smidl | The Variational Bayes Method in Signal Processing[END_REF][START_REF] Bishop | Pattern Recognition and Machine Learning (Information Science and Statistics)[END_REF], given the observations and the parameters at the previous iteration Θ • , the optimal approximation has the following general expression: log q(Z t ) = E q(Xt)q(Xt-1)q(Zt-1) log P , [START_REF] Sidenbladh | Multi-target particle filtering for the probability hypothesis density[END_REF] log q(Z t-1 ) = E q(Xt)q(Xt-1)q(Zt) log P ,

log q(X tn ) = E q(Zt)q(Zt-1)q(Xt-1,n) m =n q(Xtm) log P ,

log q(X t-1,n ) = E q(Zt)q(Zt-1)q(Xt,n) m =n q(Xt-1,m) log P ,

where, for simplicity, we used the notation

P = p(Z t , Z t-1 , X t , X t-1 |o 1:t , e 1:t , Θ • ).
In our particular case, when these two equations are put together with the probabilistic model defined in ( 2), ( 6) and ( 9), the expression of q(Z t ) is factorized further into: log q(Z tk ) = E q(Xt)q(Xt-1)q(Zt-1) log P ,

Note that this equation leads to a finer factorization that the one we initially imposed. This behavior is typical of variational Bayes methods in which a very mild separability assumption can lead to a much finer factorization when combined with priors over hidden states and latent variables, i.e. ( 2), ( 6) and ( 9). The final factorization writes:

p(Z t , Z t-1 , X t , X t-1 |o 1:t , e 1:t ) ≈ Kt k=0 q(Z tk ) Kt-1 k=0 q(Z t-1,k ) N n=0
q(X tn )q(X t-1,n ).

(16) Once the posterior distribution over the hidden variables is computed (see below), the optimal parameters are estimated using Θ = arg max Θ J(Θ, Θ • ) with J defined as:

J(Θ, Θ • ) = E q(Z,X) {log p(Z t , Z t-1 , X t , X t-1 , o 1:t |e 1:t , Θ, Θ • )} . ( 17
)

E-Z-Step

The estimation of q(Z tk ) is carried out by developing the expectation (15) which yields the following formula:

q(Z tk = n) = α tkn = e tn tkn a tn N m=0 e tm tkm a tn , ( 18
)
and tkn is defined as:

tkn = u(y tk )u(h tk ) n = 0, g(y tk , Pµ tn , Σ)e -1 2 Tr(P (Σ) -1 PΓtn) b(h tk ; h n ) n = 0, (19)
where Tr(•) is the trace operator and µ tn and Γ tn are defined by ( 21) and ( 22) below. Intuitively, this approximation shows that the assignment of an observation to a person is based on spatial proximity between the observation localization and the person localization, and the similarity between the observation's appearance and the person's reference appearance.

E-X-Step

The estimation of q(X tn ) is derived from [START_REF] Vo | Random finite sets and sequential monte carlo methods in multi-target tracking[END_REF]. Similarly to the previous posterior distribution, which boil down to the following formula:

q(X tn ) = u(X tn ) 1-etn g(X tn ; µ tn , Γ tn ) etn , , (20)
where the mean vector µ tn and the covariance matrix Γ tn are given by:

Γ tn = Kt k=0 α tkn P (Σ) -1 P + Λ -1 n -1 , (21)
µ tn = Γ tn Kt k=0 α tkn P (Σ) -1 y tk + Λ -1 n Dµ t-1,n . (22)
Similarly, for the estimation of the distribution

q(X t-1,n ) = u(X t-1,n ) 1-etn g(X t-1,n ; µ t-1,n , Γ t-1,n ) etn , (23)
the mean and covariance are:

Γ t-1,n = D Λ -1 n D + Γ t-1,n -1 (24)
µ t-1,n = Γ t-1,n D Λ -1 n µ t,n + Γ -1 t-1,n µ t-1,n . (25)
We note that the variational approximation of the kinematic-state distribution reminds the Kalman filter solution of a linear dynamical system with mainly one difference: in our formulation, ( 21) and ( 22), the means and covariances are computed by weighting the observations with α tkn , i.e. ( 21) and ( 22).

M-step

Once the posterior distribution of the hidden variables is estimated, the optimal parameter values can be estimated via maximization of J defined in [START_REF] Milan | Continuous energy minimization for multitarget tracking[END_REF]. Concerning the parameters of the a priori observation-to-object assignment A t we compute:

J(a tn ) = Kt k=1
e tn α tkn log(e tn a tn ) s.t.

The observation covariance Σ and the state covariances Λ n can be estimated during the M-step. However, in our current implementation estimates for Σ and Λ n are instantaneous, i.e., they are obtained only from the observations at time t (see the experimental section for details).

4 Person-Birth, -Visibility and -Death Processes

Tracking a time-varying number of targets requires procedures to create tracks when new targets enter the scene and to delete tracks when corresponding targets leave the visual scene. In this paper, we propose a statistical-test based birth process that creates new tracks and a hidden Markov model (HMM) based visibility process that handles disappearing targets. Until here, we assumed that the existence variables e tn were given. In this section we present the inference model for the existence variable based on the stochastic birth-process.

Birth Process

The principle of the person birth process is to search for consistent trajectories in the history of observations associated to clutter. Intuitively, two hypotheses are confronted, namely: (i) the considered observation sequence is generated by a person not being tracked and (ii) the considered observation sequence is generated by clutter.

The model of "the considered observation sequence is generated by a person not being tracked " hypothesis is based on the observations and dynamic models defined in ( 4) and [START_REF] Mahler | A theoretical foundation for the Stein-Winter" probability hypothesis density (PHD)" multitarget tracking approach[END_REF]. If there is a not-yet-tracked person n generating the considered observation sequence {y t-L,k L , . . . , y t,k0 },5 then the observation likelihood is p(y t-l,k l |x t-l,n ) = g(y t-l,k l ; Px t-l,n , Σ) and the person trajectory is governed by the dynamical model p(x t,n |x t-1,n ) = g(x t,n ; Dx t-1,n , Λ n ). Since there is no prior knowledge about the starting point of the track, we assume a "flat" Gaussian distribution over x t-L,n , namely p b (x t-L,n ) = g(x t-L,n ; m b , Γ b ), which is approximatively equivalent to a uniform distribution over the image. Consequently, the joint observation distribution writes: [START_REF] Longyin | DETRAC filter multiple target tracker: A new benchmark and protocol for multi-object tracking[END_REF] which can be seen as the marginal of a multivariate Gaussian distribution. Therefore, the joint observation distribution p(y t,k0 , y t-1,k1 , . . . , y t-2,k L ) is also Gaussian and can be explicitly computed.

τ 0 = p(y t,k0 , . . . , y t-L,k L ) = p(y t,k0 , . . . , y t-L,k L , x t:t-L,n )dx t:t-L,n = L l=0 p(y t,k l |x t-l,n ) × L-1 l=0 p(x t-l,n |x t-l-1,n ) × p b (x t-2,n )dx t:t-L,n ,
The model of "the considered observation sequence is generated by clutter " hypothesis is based on the observation model given in [START_REF] Smith | Using particles to track varying numbers of interacting people[END_REF]. When the considered observation sequence {y t,k0 , . . . , y t-L,k L } is generated by clutter, observations are independent and identically uniformly distributed. In this case, the joint observation likelihood is

τ 1 = p(y t,k0 , . . . , y t-L,k L ) = L l=0 u(y t-l,k l ). ( 29
)
Finally, our birth process is as follows: for all y t,k0 such that τ 0 > τ 1 , a new person is added by setting e tn = 1, q(x t,n ; µ t,n , Γ t,n ) with µ t,n = [y t,k0 , 0 2 ] , and Γ tn is set to the value of a birth covariance matrix (see [START_REF] Vermaak | Variational inference for visual tracking[END_REF]). Also, the reference appearance model for the new person is defined as h t,n = h t,k0 .

Visibility Process

A tracked person is said to be visible at time t whenever there are observations associated to that person, otherwise the person is considered not visible. Instead of deleting tracks, as classical for death processes, our model labels tracks without associated observations as sleeping. In this way, we keep the possibility to awake such sleeping tracks whenever their reference appearance highly matches an observed appearance. We denote the n-th person visibility (binary) variable by V tn , meaning that the person is visible at time t if V tn = 1 and 0 otherwise. We assume the existence of a transition model for the hidden visibility variable V tn . More precisely, the visibility state temporal evolution is governed by the transition matrix,

p(V tn = j|V t-1,n = i) = π δij v (1 -π v ) 1-δij
, where π v is the probability to remain in the same state. To enforce temporal smoothness, the probability to remain in the same state is taken higher than the probability to switch to another state.

The goal now is to estimate the visibility of all the persons. For this purpose we define the visibility observations as ν tn = e tn a tn , being 0 when no observation is associated to person n. In practice, we need to filter the visibility state variables V tn using the visibility observations ν tn . In other words, we need to estimate the filtering distribution p(V tn |ν 1:tn , e 1:tn ) which can be written as:

p(V tn = v tn |ν 1:t , e 1:tn ) = p(ν tn |v tn , e tn ) vt-1,n p(v tn |v t-1,n )p(v t-1,n |ν 1:t-1,n , e 1:t-1 ) p(ν tn |ν 1:t-1,n , e 1:t ) , (30)
where the denominator corresponds to integrating the numerator over v tn . In order to fully specify the model, we define the visibility observation likelihood as: p(ν tn |v tn , e tn ) = (exp(-λν tn )) vtn (1 -exp(-λν tn )) 1-vtn

Intuitively, when ν tn is high, the likelihood is large if v tn = 1 (person is visible). The opposite behavior is found when ν tn is small. Importantly, at each frame, because the visibility state is a binary variable, its filtering distribution can be straightforwardly computed. We found this rather intuitive strategy to be somewhat "shaky" over time even taking the Markov dependency into account. This is why we enriched the visibility observations to span over multiple frames

ν tn =
L l=0 e t+ln a t+ln , so that if v tn = 1, the likelihood is large when ν tn is high and therefore the target is visible in one or more neighboring frames. This is the equivalent of the hypothesis testing spanning over time associated to the birth process.

Death Process

The idea of the person-visibility process arises from encouraging track consistency when a target disappears and appears back in the field of view. However, a tracker that remembers all the tracks that have been previously seen is hardly scalable. Indeed, the memory resources required by a system that remembers all previous appearance templates grows indefinitely with new appearances. Therefore, one must discard old information to facilitate the scalability of the method to large datasets containing sequences with several dozens of different people involved. In addition to alleviating the memory requirements, this also reduces the computational complexity of the tracker. This is the motivation of including a death process into the proposed variational framework. Intuitively one would like to discard those tracks that have not been seen during several frames. In practice, we found that discarding those tracks that are not visible for ten consecutive frames yields a good trade-off between complexity, resource demand and performance. Setting this parameter for a different dataset should not be chimeric, since the precise interpretation of the meaning of it is straightforward.

Experiments, Performance Evaluation, and Benchmark

We evaluated the performance of the proposed variational multiple-person tracker on the MOT 2016 dataset challenge [START_REF] Milan | Mot16: A benchmark for multi-object tracking[END_REF]. This dataset is composed of seven training videos and seven test videos. Importantly, we use the detections that are provided with the dataset. Because multiple-person tracking intrinsically implies track creation (birth), deletion (death), target identity maintenance, and localization, evaluating multiple-person tracking techniques is a non-trivial task. Many metrics have been proposed, e.g. [START_REF] Ristic | Performance evaluation of multi-target tracking using the OSPA metric[END_REF][START_REF] Smith | Evaluating multi-object tracking[END_REF][START_REF] Stiefelhagen | CLEAR 2006 evaluation[END_REF][START_REF] Longyin | DETRAC filter multiple target tracker: A new benchmark and protocol for multi-object tracking[END_REF].

We adopt the metrics used by the MOT 2016 benchmark, namely [START_REF] Stiefelhagen | CLEAR 2006 evaluation[END_REF]. The main tracking measures are: the multiple-object tracking accuracy (MOTA), that combines false positives (FP), missed targets (FN), and identity switches (ID); the multiple-object tracking precision (MOTP), that measures the alignment of the tracker output bounding box with the ground truth; the false alarm per frame (FAF); the ratio of mostly tracked trajectories (MT); the ratio of mostly lost trajectories (ML) and the number of track fragmentations (Frag).

Fig 2 shows sample images of all test videos: They contain three sequences recorded with static cameras (MOT16-01, MOT16-03 and MOT16-08), which contain very crowded scenes and thus are very challenging, and five sequences with large camera motions, both translations and rotations, which make the data even more difficult to process. As explained above, we use the public pedestrian detections provided within the MOT16 challenge. These static detections are complemented in two different ways. First, we extract velocity observations by means of a simple optical-flow based algorithm that looks for the most similar region of the next temporal frame within the neighborhood of the original detection. Therefore, the observations operator P is the identity matrix, project the entire state variable into the observation space. Second, the appearance feature vector is the concatenation of joint color histograms of three regions of the torso in HSV space.

The proposed variational model is governed by several parameters. Aiming at providing an algorithm that is dataset-independent and that features a good trade-off between flexibility and performance, we set the observation covariance matrix Σ and the state covariance matrix Λ n automatically from the detections. More precisely, both matrices are imposed to be diagonal; for Σ, the variances of the horizontal position, of the width, and of the horizontal speed are 1/3, 1/3 and 1/6 of the detected width. The variances of the vertical quantities are built analogously. The rationale behind this choice is that we consider that the true detection lies, more or less, within the width and height of the detected bounding box. Regarding Λ n , the diagonal entries are 1, 1 and 1/2 of the detected width, and vertical quantities are defined analogously. Furthermore, in order to eliminate arbitrary false detections, we set L = 5 in the birth process. Finally, for sequences in which the size of the bounding boxes is roughly constant, we discarded those detections that were too large or too small. Examples of the tracking results for all the test sequences except MOT16-07 are shown in Figure 3, while six frames from MOT16-07 are shown in Figure 4. In all figures, the red boxes represent our tracking result and the numbers within the boxes are the tracking indexes. Generally speaking, on one hand the variational model is crucial to properly associate detections with trajectories. On the other hand, the birth and visibility processes play a role when tracked objects appear and disappear. Regarding Figure 4, it contains 54 tracks recorded by a moving camera in a sequence of 500 frames. It is a very challenging tracking task, not only because the density of pedestrians is quite high, but also because significant camera motion makes the person trajectories to be both rough and discontinuous. One drawback of the proposed approach is that partially consistent false detections could lead to the creation of a false track, therefore tracking an inexistent pedestrian. On the positive side, the main advantage of the proposed model is that the probabilistic combination of the dynamic and appearance models can decrease the probability of switching the identities of two tracks.

Table 1 reports the performance of the proposed algorithm, which is referred to as OVBT (online variational Bayesian tracker), over the seven test sequences of the MOT 2016 challenge. The results obtained with OVBT are available on the MOT 2016 webpage. 6 One can notice that our method provides high precision (MOTP) but low accuracy (MOTA), meaning that some tracks were missed (mostly due to misetections). This is consistent with a rather low MT measure. This behavior was more extreme when the visibility process did not include any observation aggregation over time. Indeed, we observed that considering multiple observations within the visibility process leads to better performance (for all sequences and almost all measures).

Conclusions

We propose a variational Bayesian solution to the multiple-target problem. In the literature, other solutions based on sampling such as MCMC, and random finite set, such as the PHD filter have been proposed to solve the same problem.

Comparison with other state of the art methods are available [START_REF] Ba | An on-line variational Bayesian model for multi-person tracking from cluttered scenes[END_REF].

The main goal of our study was to benchmark the model on MOT Challenge 2016. Implementation issues of the tracker are discussed as well as its strengths and weaknesses regarding the absolute performance on the test sequences and the relative performance when compared with other participants to the MOT Challenges.

The presented model is free from magic parameters, since these are automatically derived from the data. Moreover, the proposed temporal aggregation for the visibility process appears to be an excellent complement to the variational Bayes EM algorithm. In the near future, we plan to derive self-paced learning strategies within this variational framework able to automatically assess which detections must be used for tracking and which should not be utilized.

Fig. 1 .

 1 Fig. 1. Examples of detected persons from the MOT 2016 dataset.

Fig. 2 .

 2 Fig. 2. Samples images from the MOT 16 test sequences.

14 Fig. 3 .

 143 Fig. 3. Sample results on several sequences of MOT16 datasets, red bounding boxes represents the tracking results, and the number inside each box is the track index.

Fig. 4 .

 4 Fig. 4. Sample results on the sequence MOT16-07, encoded as in the previous figure.Table1. Evaluation of the proposed multiple-person tracking method with different features on the seven sequences of the MOT16 test dataset.

Table 1 .

 1 Evaluation of the proposed multiple-person tracking method with different features on the seven sequences of the MOT16 test dataset.

	Sequence MOTA MOTP FAF MT ML FP	FN ID Sw Frag
	MOT16-01	23.9	71.4	1.5 13.0% 39.1% 696 4,137	35	89
	MOT16-03	46.9	75.7	4.1 17.6% 20.3% 6,173 48,631 689 1,184
	MOT16-06	32.7	73.2	0.5 3.6% 58.4% 562 7,073 124	183
	MOT16-07	33.6	73.3	2.2 9.3% 35.2% 1,077 9,605 158	272
	MOT16-08	24.6	78.4	1.7 3.2% 41.3% 1,066 11,402 150	177
	MOT16-12	32.8	76.7	0.9 10.5% 52.3% 766 4,749	63	80
	MOT16-14	18.1	74.5	1.6 2.4% 61.6% 1,177 13,866 102	155
	Over All 38.4± 8.8 75.4	1.9 7.5% 47.3% 11,517 99,463 1,321 2,140

https://team.inria.fr/perception/research/ovbt/

In practice we considered L = 2, however, derivations are valid for arbitrary values of L.

https://motchallenge.net/results/MOT16/

Support from the ERC Advanced Grant VHIA number 340113 and from MIUR Active Aging at Home CTN01 00128 is greatly acknowledged.


Fabien Gandon

Raphael Boyer

Olivier Corby

Alexandre Monnin

Wikipedia editing history in DBpedia extracting and publishing the encyclopedia editing activity as linked data

Keywords: DBpedia, editing activity, linked data I. EXTRACTING EDITION

DBpedia is a huge dataset essentially extracted from the content and structure of Wikipedia. We present a new extraction producing a linked data representation of the editing history of Wikipedia pages. This supports custom querying and combining with other data providing new indicators and insights. We explain the architecture, representation and an immediate application to monitoring events.

However, besides the content, the editing activity on Wikipedia also produces traces that are highly representative of Wikipedians' variety of focus at a given time. "Wikipedia and its sister projects develop at a rate of over 10 edits per second, performed by editors from all over the world" 3 . This happens while the "English Wikipedia alone has over 5,093,940 articles of any length, and the combined Wikipedias for all other languages greatly exceed the English Wikipedia in size, giving more than 23 billion words in 35 million articles in 291 languages" 4 . As a side effect, logs of the page changes capture the current focus and shift of interests of the contributors. Since the encyclopedia has a fast and broad coverage, this editing reflects the unfolding of events all around the world and in every do-main. Previous works thus proposed to monitor realtime editing activity of Wikipedia as a way to detect natural disaster [START_REF] Steiner | Comprehensive Wikipedia monitoring for global and realtime natural disaster detection[END_REF].

To support even more applications of the analysis of the edition activity, we designed a new DBpedia extractor that produces linked data from the history of each Wikipedia page. A historic dump of Wikipedia contains all the modifications dating back from the inception of a linguistic chapter along with some information for each and every modification. As an example, the French editing history dump represents 2TB of uncompressed data. This data extraction is performed by stream in Node.js with a MongoDB instance. It takes 4 days to extract 55 GB of RDF in turtle on 8 Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz with 68GB or RAM and using SSD disks. The result is currently published as a DBpedia test chapter with its own SPARQL end-point for beta-testing 5 .

II. RELATED WORK

In the Wikipedia community there is a long history of providing means to monitor the editing activity they include: the recent changes API, IRC streams per languages, WebSockets streams, Server-Sent Events Streams, etc. [6] Authors of [3] proposed a resource versioning mechanism based content negotiation on time as proposed in the Memento protocol (HTTP Framework for Time-Based Access to Resource States, RFC7089 6 ). They applied it to DBpedia but their approach is only archiving released dumps. In fact they are limited to five versions of the English-language of DBpedia (2.0 through 3.3) while in our work we will cover each and every single editing act since the creation of the French Wikipedia chapter.

The DBpedia Wayback machine [START_REF] Fernández | The DBpedia wayback machine[END_REF] allows to retrieve historical versions of resources at a given timestamp and to be able to reconstruct the full history of RDF documents. They offer this service through a Web user interface and a RESTful API. The major difference with what we propose here is that we focus on capturing and materializing the editing activity that led to each version (time, frequency, sizes, editors, etc.) as linked data to support any possible query. The Wayback machines provides the RDF version of a Wikipedia article at a certain time and some revision data but only through and API and for a given resource. It is not possible to have arbitrary queries such as "give me the 20 most edited actors last month". Moreover our extraction reuses as many existing vocabularies from the LOV 7 directory as possible in order to facilitate integration and reuse.

Other contributions like [START_REF] Meinhardt | TailR: a platform for preserving history on the web of data[END_REF] focus on preserving the history of linked datasets over time, providing access to prior states of contained resources and a versioning platform for linked data in general. Again a sample of 100,000 resources in releases 3.2 through 3.9 of DBpedia is used for the evaluation dataset but the goal of the authors is not to provide and support a comprehensive view of the editing activity of Wikipedia supporting open-queries as we do.

None of these previous contributions support custom querying of editing history while it is known now that the availability as linked data supports unforeseen innovative usages far beyond what is possible with constrained APIs. Moreover the potential of this data is even greater when combined with other linked data sources which again is not easily done with an API approach. These data about the activity provide indicators of interest, attention, over the set of resources they cover. Inversely, using statements of other datasets (e.g. typing) one can filter and analyze the editing history considering many dimensions. Finally we do not include here works on formats, vocabularies or algorithms to detect and describe updates to RDF datasets since at this stage what we are capturing are editing acts on Wikipedia.

III. MODELING THE EDITION HISTORY AS LINKED DATA

Figure 1 is a sample of the output of the edition history extractor for the page describing "Victor Hugo" in the DBpedia French chapter. The history data for such an entry contains one section of general information about the article history (lines 1-15) along with many additional sections corresponding to the previous revisions made to capture each change (e.g. two revisions at lines 16-24). The general information about the article includes: a) the number of revisions, b) the date of creation and last modification, c) the number of unique contributors, d) the number of revisions per year and per month and e) the average size of revisions per year and per month. Each revision section includes: a) the date and time of the modification, b) the size of the modification as a number of characters, c) the size of this previous revision as a number of characters, d) the username or IP address of the contributor, e) the optional comment of the contributor and f) if the contributor is a human or a bot. The data are fully linked to the DBpedia resources and the vocabularies used include PROV-O, Dublin Core, the Semantic Web Publishing Vocabulary, DBpedia ontologies, FOAF and SIOC. Therefore the produced linked data are well integrated to the LOD cloud. Every time we were missing a predicate we added it to DBpedia FR ontology. As exemplified in Figure 2 these data support very expressive queries such as, in this example, the ability to request most modified pages grouped by pairs of pages modified the same day.

IV. FIRST APPLICATIONS FOR VALIDATION: NEWS AND EVENT DETECTION

We prototyped two applications to demonstrate the effectiveness of the historical data captured. To do so we used the STTL template language which allows us to develop proofs of concept very quickly [START_REF] Corby | A Generic RDF Transformation Software and its Application to an Online Translation Service for Common Languages of Linked Data[END_REF]. The first application designed (Figure 2) is a visual history browser that displays images of the 50 most edited topics for every month 8 . For example, French elections, as a topic, appear in the top five of edited pages in September 2015. Then we demonstrated the ability to cross this new dataset with other linked data sources starting with DBpedia itself. We built a focus portal generator that 8 Images of editing history http://corese.inria.fr/srv/template?profile=st:dbedit reduces the monitoring activity to specific DBpedia categories of resources (e.g. companies, actors, countries, etc.). In Figure 3 the portal is focused on the artist category 9 and gives visibility to the death of Christopher Lee, an event which happened in June 2015. Many applications of the editing activity already exist [6] and these two portals are only examples of what can be done with simple SPARQL queries when the editing activity is available as linked data.

V. NEXT STEP: GOING LIVE

The extractor is being integrated to the DBpedia opensource code and in parallel the running service will migrate from the test server to the production server of the French chapter. We are now studying the adaptation of the new live mechanism dedicated both to the chapter and history extraction in order to reflect real-time changes to the content and edition history. Finally we are considering new applications, such as new linking methods [7], that could be supported by this dataset.

Fig. 2 .

 2 Fig. 2. Ten of the most modified pages grouped by pairs of pages modified the same day

Fig. 4 .

 4 Fig. 4. The French election topic in the top 5 of edited pages in September 2015

LOD Cloud Diagram http://lod-cloud.net/

French DBpedia Chapter http://fr.dbpedia.org/

Wikipedia Statistics -https://en.wikipedia.org/wiki/Wikipedia:Statistics

Comparisons https://en.wikipedia.org/wiki/Wikipedia:Size_comparisons

History endpoint http://dbpedia-test-fr.inria.fr/historique/sparql

RFC

https://tools.ietf.org/html/rfc7089 7 Linked Open Vocabularies (LOV) http://lov.okfn.org/

Category-filtered view of the History: Focusing on artists http://corese.inria.fr/srv/template?profile=st:dbedit&mode=http://dbpedia.org/ ontology/Artist

ACKNOWLEDGMENT

We thank the French Ministry of Culture for supporting the French DBpedia chapter.


Fabien Gandon
email: fabien.gandon@inria.fr

Raphael Boyer
email: raphael.boyer@inria.fr

Olivier Corby
email: olivier.corby@inria.fr

Alexandre Monnin
email: alexandre.monnin@inria.fr

Materializing the editing history of Wikipedia as linked data in DBpedia

We describe a DBpedia extractor materializing the editing history of Wikipedia pages as linked data to support queries and indicators on the history.

The different instances of the DBpedia platform typically extract RDF from Wikipedia using up to 16 extractors. The extraction focuses on structured content including infoboxes, categories, links, etc. As an example, the French chapter 1 , of which we are responsible, extracted 185 million triples in 2015. The resulting RDF graph is then published and supports up to 2.5 million SPARQL queries per day and an average of 70,000 SPARQL queries per day in 2015. But Wikipedia is a social media that produces more data than the actual content of its pages. The activity of the epistemic communities of Wikipedia produces a huge amount of traces showing, for instance, the evolution, conflicts, trends, and variety of opinions of the users. In fact, the different projects of the Wikipedia Foundation develop at a rate of over ten edits per second, performed by users from all over the world2 . And this activity is performed on broad collection of topics: the English chapter of Wikipedia alone has over 5 million articles and the combined Wikipedias for all other languages exceed the English chapter in size with more than 27 billion words in 40 million articles in 293 languages 3 . As a result the history of the editing actions captures the peaks and shifts of interests of the contributors and indirectly reflects the unfolding of events all around the world and in every domain.

Providing means to monitor the editing activity has always been important for Wikipedians to follow the changes. These means include APIs such as the recent changes API, the IRC streams per languages, the WebSockets streams, the Server-Sent Events Streams, etc. [6]. Previous works also suggested to monitor real-time editing activity of Wikipedia to detect events such as natural disasters [START_REF] Steiner | Comprehensive Wikipedia monitoring for global and realtime natural disaster detection[END_REF]. In [3] a resource versioning mechanism inspired from the Memento protocol (RFC7089) is applied but only to DBpedia dumps. In [START_REF] Fernández | The DBpedia wayback machine[END_REF] historical versions of resources are regenerated for a given timestamp with some revision data but through a RESTful API. In [START_REF] Meinhardt | TailR: a platform for preserving history on the web of data[END_REF] the preservation of the history of linked datasets is tested but only on a sample of 100,000 resources. We do not mention here works on formats, vocabularies or algorithms to detect and describe updates to RDF datasets since at this stage we are focusing on editing acts on Wikipedia.

Data about the activity provide historical indicators of interest, attention, over the set of resources they cover. They have been also used, for instance, to assess the currency of the data [7], to study conflict resolution [8], to temporally anchor data, to attribute changes and to identify vandalism [9] or to precisely attribute the authorship of content [10]. Inversely, using statements of other datasets (e.g. typing) one can filter and analyze the editing history considering chosen dimensions (e.g. focus on events about artists). But none of the previous contributions support public SPARQL querying of the full editing history. The potential of these linked data is even greater when combined with other linked data sources and this is not easily done with an API approach e.g. "give me the 10 most edited populated places in July 2012".

For this reason we designed and provide a new DBpedia extractor producing a linked data representation of the editing history of Wikipedia pages. Instead of real-time monitoring we capture the history as linked data to be able to query it, mine it and combine it with other sources to augment the dimensions we can exploit when querying linked data in general and DBpedia in particular.

A history dump of a Wikipedia chapter contains all the modifications dating back from the inception of this linguistic chapter along with some information for each and every modification. As an example, the French editing history dump represents 2TB of uncompressed data. The data extraction is performed though streams in Node.js with a MongoDB instance. It took 4 days to extract 55 GB of RDF in turtle on 8 Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz with 68GB or RAM and using SSD disks. The result is then published through a SPARQL end-point with the DBpedia chapter 4 .

The extractor reuses as many existing vocabularies from the LOV directory5 as possible in order to facilitate integration and reuse. Figure 1 is a sample of the output of the edition history extractor for the page describing the author "Victor Hugo" in the DBpedia French chapter. The history data for such an entry contains one section of general information about the article history (lines 1-15) along with as many additional sections as there are previous revisions to capture each change (e.g. two revisions at lines 16-24). The general information about the article includes: the number of revisions (line 3), the date of creation and last modification (lines 4-5), the number of unique contributors (line 6), the number of revisions per year and per month (e.g. lines 7-8) and the average sizes of revisions per year and per month (e.g. lines 9-10). In addition each individual revision description includes: the date and time of the modification (e.g. lines 17), the size of the revision as a number of characters, (e.g. lines 18) the size of the modification as a number of characters (e.g. lines 19), the optional comment of the contributor (e.g. lines 20), the username or IP address of the contributor and if the contributor is a human or a bot (e.g. line 21 or 24) and a link to the previous revision (e.g. line 22).

By construction the data are fully linked to the DBpedia resources and the vocabularies used include: PROV-O, Dublin Core, the Semantic Web Publishing Vocabulary, DBpedia ontologies, FOAF and SIOC. As a result the produced linked data are well integrated to the LOD cloud. Every time we were missing a predicate we added it to DBpedia FR ontology. As shown in Figure 2 these data support very arbitrary queries such as, in this example, the ability to request most modified pages grouped by pairs of pages modified the same day.

Fig. 2. Ten of the most modified pairs of pages modified the same day

The STTL template language [START_REF] Corby | A Generic RDF Transformation Software and its Application to an Online Translation Service for Common Languages of Linked Data[END_REF] allows to generate portals in a declarative and fast way. We used it to build two portals to show the richness of the historical data materialized. The first application designed is a visual history browser that displays images of the 50 most edited topics for every month. With the second portal we demonstrate the ability to join this new dataset with other linked data sources starting with DBpedia itself: we built a focused portal generator that reduces the monitoring activity to specific DBpedia categories of resources 6 (e.g. companies, actors, countries, etc.). Figure 3 is a screenshot of the portal focused on countries and shows the events in Ukraine in 2014. Many applications of the editing activity already exist [6] and these two portals are only a proof of concept for what can be done with SPARQL over the linked data of editing activity. The history extractor is now integrated to the DBpedia open-source code and running on the production server of the French chapter. We are studying the integration of the live change feed for both the chapter and its history in order to reflect real-time changes to the content and editing logs. We are also considering ways to represent more precisely the changes between two revisions.

Fig. 3 .

 3 Fig. 3. Portal showing Countries whose page was subject to a maximum activity.

French DBpedia Chapter http://fr.dbpedia.org/

Wikipedia Statistics -https://en.wikipedia.org/wiki/Wikipedia:Statistics

Comparisons accessed 23/08/16 https://en.wikipedia.org/wiki/Wikipedia:Size_comparisons

History endpoint http://dbpedia-historique.inria.fr/sparql

Linked Open Vocabularies (LOV) http://lov.okfn.org/ as accessed in June 2016

Category-filtered view of the History: Focusing on artists (mode=dbo:Artist) http://corese.inria.fr/srv/template?profile=st:dbedit&mode=dbo:Artist Or focusing on countries (mode=dbo:Country) using the DBpedia ontology http://corese.inria.fr/srv/template?profile=st:dbedit&mode=dbo:Country


D N Papadopoulos
email: dimitrios.papadopoulos@polytechnique.edu

P Ramirez

A Pellegrina

N Lebas

C Le Blanc

G Chériaux

J P Zou

G Mennerat

P Monot

F Mathieu

P Audebert

P Georges

F Druon

High-contrast 10-fs OPCPA-based Front-End for the Apollon-10PW laser

Keywords: Ultrafast lasers; (190.7110) Ultrafast nonlinear optics, (190.4410) Nonlinear optics, parametric processes

We present a high-contrast 10-fs Front-End for Ti:sapphire PW-lasers within the Apollon-10PW project. This injector uses OPCPA pumped at 100 Hz by Yb-based CPA chain. Combination of OPCPA and XPW permits a >10 12 contrast ratio.

The Apollon 10 PW laser

In the framework of the Apollon 10-PW French laser program, we develop a front-end delivering ultrashort pulses around 800 nm with a very-high temporal contrast to be used as seeder for high energy Ti:Sapphire amplifiers. Indeed, in the development of PW laser chains, the contrast is one of the key issues. For high-field physics applications, laser pulses are only useful with nice contrast id est with a ns-ps piedestal under 10 12 W/cm 2 . That leads, for 10 PW systems delivering 10 23 W/cm 2 , a contrast of at least of 10 11 . Moreover, to reach the 10 PW the gain narrowing in Ti:Sapphire systems is also an issue and the use of spectral filtering together with a 10 fs input pulse is the solution to reach the peak power with manageable energy (150 J). Finally, the repetition rate of this injector (typically 100 Hz) is also important to actively control the stability of the complete laser chain even if it operates below, e.g. 0.1 Hz.

In this paper, we are presenting the status of the Apollon 10-PW research status by exposing the results on this particular Front-End. The system is described and the results are analyzed in the context of PW laser facilities.

Architecture of the high-contrast short-pulse Front-End

The Front-End of the Apollon system is based on optically-synchronous OPCPA. An ultrashort-pulsed Ti:Sa oscillator (7 fs) produces a very broad spectrum that is split into two outputs : the part centered at 800 nm is sent to a commercial Ti:Sa multipass amplifier that generates 1 mJ, 25 fs pulse at 1 kHz, and the part >1020 nm is used to inject a Yb-based system that will be used to pump the OPCPA (fig. 1). The generation of the OPCPA signal is based on a specific XPW configuration providing the following characteristics: 10-fs-pulse bandwidth and contrast better than 10 10 [1,2]. The signal is then stretched to 6 ps and goes to a spectral phase controller (double-pass Dazzler) that will compensate the spectral phase distortion of the entire chain. The 1030 nm part of the oscillator spectrum is amplified in an YDFA (from Keopsys), stretched to 1.5 ns, amplified to 2 mJ in a Yb:KYW regenerative amplifier (from Amplitude Systemes), amplified in a Yb:YAG thin-disk regenerative amplifier to 150 mJ [START_REF] Tümmler | High-repetition-rate chirped-pulse-amplification thin-disk laser system with joule-level pulse energy[END_REF] and compressed down to 15 ps. The 1030 nm pulses are then frequency-double in a type-I LBO crystal producing approximately 12 ps pulses at 515 nm with energy in the 10s of mJ.

OPCPA

The approach for setting up the OPCPA stage is mainly experimental. The beam size of the pump is varied with the use of several lens combinations for the telescope and the target is to obtain a stage which when pumped with 10 mJ, will deliver a stable pre-amplified seed, with an energy around 1 mJ and bandwidth for 10-fs pulses. This best result is obtained for a pump diameter of 3 mm. The behavior of this OPCPA stage is investigated with respect to the pump beam energy, which leads to intensities ranging from 3.8 -15.1 GW/cm 2 . Regarding the input signal, it has an almost perfect Gaussian form with a 1/e 2 diameter of 5 mm. The choice of using a larger seed beam is a precaution based on a method to suppress superfluorescence implemented in [START_REF] Moses | Highly stable ultrabroadband mid-IR optical parametric chirped-pulse amplifier optimized for superfluorescence suppression[END_REF], where the larger signal beam allows the selection of the part of the signal beam with the highest signal to noise ratio and cleanest wavefront. The results are summarized in Figure 2. Under perfect phase matching conditions, the amplified seed pulse is broadened and approaches a square-shaped spectrum at 10 mJ. The optimal efficiency -chosen in order to avoid the excessive backconversion and the superfluorescence-is 9 %, which leads to a signal energy of 0.9 mJ. In order to evaluate the compressibility of the ultra-broadband generated pulses. The pulses are recompressed using a grating compressor set in an Offner configuration (close to the zero dispersion setting). The pulses are analyzed and optimized with a Wizzler/Dazzler loop. This allowed the validation of the compressibility of the amplified pulses in sub-10 fs level with a measurement of 9.5 fs to be compared to 8.1 fs FT limit (fig. 3). The residual phase is restricted to an acceptable level resulting only in minimal issues on the coherent contrast. After compression a contrast measurement is performed using a high-dynamic third order autocorrelator. A contrast ratio of 10 12 is measured, and taking into account the limit in bandwidth of our autocerrelator, we can even estimate confidently a real contraste measurement of 10 13 . Figure 4 : Contrast measurement.

Conclusion

We have presented, within the Apollon 10PW project, a high-contrast Front-End for Ti:sapphire high-energy amplifiers. A 10 13 contrast is obtained combining the XPW and OPCPA techniques. This injector demonstrated several advantages in the framework of multi-PW laser: First ultra-braodband amplification is demonstrated with a quasi-square shape compressible down to 9.5 fs. Moreover, the OPCPA efficiency is handled such as the beam profile stay of very good quality, and the contrast preserved against superfluorescence. Finally, the pump of the OPCPA is based on DPSSL Yb-based technology [START_REF] Tümmler | High-repetition-rate chirped-pulse-amplification thin-disk laser system with joule-level pulse energy[END_REF][START_REF] Papadopoulos | High Repetition Rate Yb:CaF2 Multipass Amplifiers Operating in the 100-mJ Range" Invited in Selected Topics in Quantum Electronics[END_REF], which allows the 100 Hz operation. This repetition rate will be used advantageously to drive the servo-loops of the system between high energy shoots.

Figure 1 :

 1 Figure 1 : Apollon Front-End simplified schematic (top); and close up on the OPCPA signal formatting (buttom).

figure 2 :

 2 figure 2 : spectrum (linear and log scale) and beam profile of the OPCPA.

Figure 3 :

 3 Figure 3 : characterization of the compressed pulses.

This work is supported by French state funds managed by the ANR within the "Investments for the Future" program under reference ANR-10-EQPX-25, by the ILE-APOLLON 07-CPER 017-01 contract and by the European FEDER APOLLON MPW contract.


B Yang

C.-Y Fang

H.-C Chang

F Treussart

J.-B Trebbia

B Lounis

Polarization Effects in Lattice-STED Microscopy

Massive parallelization of STED-like nanoscopies is now achievable using well-designed optical lattices for state depletion.

Yet, only the lattice intensity distribution was considered for the description of the super-resolved point spread function.

This holds for fast-rotating fluorescent emitters. Here, we study the effects of electric field topography in lattice-STED microscopy. The dependence of the super-resolved point spread function on the number of dipoles and their orientation is investigated. Single fluorescent nano-diamonds are imaged using different optical lattice configurations and the measured resolutions are compared to theoretical simulations.

Introduction

Fluorescence images of emitters separated by less than half of the wavelength are not resolvable due to the diffraction limit. By surpassing this limit and providing resolution down to a few nanometers, super-resolution microscopy techniques 1,2 have revolutionized far-field optical microscopy in the last decade. These techniques exploit the photo-physical and photo-chemical properties of emitters to make them fluoresce sequentially and therefore to optically resolve them. The sequential selective imaging is achieved either in stochastic or deterministic ways. Stochastic super-resolution methods are based on the localization of isolated single emitters 3- 9 . Deterministic methods [10][11][12][13][14][15] , such as stimulated emission depletion (STED) 10 are based on patterned laser illuminations, which restrict the fluorescence emission to pre-determined nanoscale regions. This sequential imaging modality implies that the resolution gain is at the expense of the imaging speed, which could be a severe limitation when studying dynamic structures in living cells, since many biological phenomena occur at timescales from milliseconds to seconds 16,17 . Although localization-based methods are intrinsically parallelized, their imaging speeds are limited by the density of the fluorescent emitters per raw image and by the stochastic process of emitters on/off switching. The acquisition time of a super-resolved image usually lays in the range of 1-100 s 2,[18][19][20] . Please do not adjust margins Please do not adjust margins

ARTICLE

Standard STED microscopy is a point-scanning technique 21 which requires only a limited number of fluorescence photons integrated over dwell times down to several tens of μs 22 , to obtain the local density of emitters.

However, the imaging speed of large fields of view is however strongly limited since dense pixelization is required.

An important step towards massive parallelization of STED-like nanoscopies has been achieved using welldesigned optical lattices for state depletion 23,24 . These lattices contain an array of unit cells, each of which has a point-like intensity minimum surrounded by a nearly isotropic intensity distribution. Combined with a homogeneous wide-field illumination for fluorescence excitation, each unit cell plays the same role as a doughnut beam of confocal STED, which reduces the fluorescence emission volume to a sub-diffraction scale by stimulated emission depletion. To obtain a super-resolution image, one only needs to scan the sample over a unit cell of the optical lattice rather than the entire field of view. With this Lattice-STED approach, imaging of 3 μm x 3 μm field of view with a resolution of 70 nm and a speed of 80 ms per super-resolution image has been achieved 23 . The low pulse energy of the high repetition rate laser used for the depletion and the readout time of the camera set the limit of the field of view and the imaging speed, respectively. Using a laser with lower repetition rate and higher pulse energy, larger fields of view can be imaged 25 .

In the aforementioned experiments, the polarization effects in the depletion lattice have been neglected. Only

Optical lattices

Optical lattices are periodic patterns generated by multi-beam interferences. They were first introduced in cold atoms trapping and manipulation [26][27][28] . Their applications have been extended to other fields such as photonic crystal fabrication 29 and microfluidics 30 . The electric field topology (i.e. spatial distributions of its amplitude and its polarization) of an optical lattice depends on the number of interfering beams, their directions, polarizations and relative phases 27,31 .

The total electric field ܧ ሬԦ ሺ,ݎ ሬሬԦ ݐሻ created by the interference of a number  of plane waves having the same amplitude ℰ writes:

Please do not adjust margins Please do not adjust margins

ܧ ሬԦ ሺ,ݎ ሬሬԦ ݐሻ = ℰ݁ ሺ ሬԦ ೕ . Ԧିఠ௧ାథ ೕ ሻ ݁ Ԧ ୮ ୨ୀଵ = ℰ݁ ሺ ሬԦ . Ԧିఠ௧ାథ ሻ ቌ݁ Ԧ + ݁ ሺሺ ሬԦ ೕ ି ሬԦ ሻ. Ԧାథ ೕ ିథ ሻ ୮ିଵ ୨ୀଵ ݁ Ԧ ቍ ሺ1ሻ
Where ݇ ሬԦ , ߱, ߶ , ݁ Ԧ are respectively the wave vector, the angular frequency, the phase and the polarization directions of each plane wave. The vector position ݎ Ԧ has ݍ spatial coordinates corresponding to the space dimension .ݍ Since the arbitrary choice of the time origin allows cancelling the phases ߶ , the number of independent relative phases in Equation ( 1) is ሺ -1ሻ. Furthermore, the arbitrary choice of the space origin ሺݎ Ԧ = ݎ Ԧ ᇱ + ݎ Ԧ ሻ allows rewriting the spatial dependence of the electric field as:

ܧ ሬԦ ሺݎ Ԧሻ = ℰ ቌ݁ Ԧ + ݁ ሺ ሬԦ ೕ ି ሬԦ ሻ. Ԧ ᇲ ݁ ሺሺ ሬԦ ೕ ି ሬԦ ሻ. Ԧ బ ାథ ೕ ିథ ሻ ୮ିଵ ୨ୀଵ ݁ Ԧ ቍ ሺ2ሻ
This equation clearly shows that any relative phase variation between the beams results in a global spatial translation of the optical lattice ݎ( Ԧ , without any change in the field topography) if the system of ሺ -1ሻ linear equations of ݍ variables ሺ߶ -߶ + ሺ݇ ሬԦ -݇ ሬԦ ሻ. ݎ ሬሬሬԦ = 0, 1 ≤ ݆ ≤  -1ሻ has a solution 31 . For independent wave vectors, this requirement is fulfilled if the number of interfering beams  is equal to ݍ + 1.

In the case of 2D imaging ሺݍ = 2ሻ, the interference of three beams will lead to an optical lattice which keeps both the topographies of the electric field and of the intensity independent of the beams phases. A four-beam optical lattice has an electric field topography that depends on the beams relative phases. However, in certain configurations, the intensity topography is insensitive to the beams phases and leads to better STED parallelization.

Three beam configuration

To build a three-beam optical lattice in the focal plane of a microscope objective suitable for STED parallelization, Please do not adjust margins Please do not adjust margins polarizations parallel to one of the three sides of the incidence triangle shown in Fig. 1a. Fig. 1b shows the intensity distribution of the optical lattices obtained for an incidence angle θ = 72°. At this angle, the hexagonal pattern displays a period of 355 nm and an optimal intensity contrast, better than 99.9%. The intensity profiles along two directions in the x-y plane (blue and green lines in Fig. 1c) are identical up to 150 nm away from the zero intensity position. Such depletion intensity profile should lead to an almost isotropic s-PSF when imaging fast rotating emitters. In the imaging plane (x,y), the amplitudes of the electric field projections along the x and z directions are significantly larger than that along y direction since the incident beams have the same polarization ሺ݁ Ԧ ௫ ሻ. The polarization topography of the electric field projection in the (x,z) plane is shown in Fig. 1d. Around the intensity Please do not adjust margins Please do not adjust margins minima, the polarization is elliptical except along the three directions shown by the red dashed lines in Fig. 1d where it is linear. As discussed above, the field topography is insensitive to any phase variations of the incident beams.

In practice, the back aperture of the objective and the size of the beams at this aperture limit the maximum achievable incidence angle. In the experiment, we use an oil immersion objective (N. A. = 1.49, back aperture diameter ~ 10 mm) which allows to achieve an angle θ = 60° with beam diameters of 1 mm. In this case, the optical pattern shown in Fig. 1e still displays "zero-intensity" regions with a contrast close to unity (98%).

However, as shown in the intensity profiles of Fig. 1f, the intensity gradient around the locations of the zeros is steeper in the horizontal direction.

Four beam configuration

A key point for faster Lattice-STED imaging is to generate lattices with the smallest possible unit cell. For this purpose, we use a four-beam interference configuration where a smaller unit cell can be obtained with the same angle ߠ. If one uses two orthogonally polarized pairs of beams (Fig. 2a), each pair will generate a one-dimensional interference pattern independently from the other pair of beams. The intensity distribution of the final optical lattice is just the sum of the two one-dimensional interference patterns and a square optical lattice with a periodicity ଶ୬ୱ୧୬ሺሻ = 290 ݊݉ can be obtained for ߠ = 60° and ߣ = 760 ݊݉, as shown in Fig. 2b.

Fig. 2c shows the intensity plots along the x-axis (green) and the diagonal direction (blue). The maximal intensity along the diagonal direction is twice that along the x-axis. Nevertheless the intensity gradients around the intensity "zeros" are very similar. Therefore the resolution should be almost isotropic when imaging fast-rotating molecules. The intensity topography of the lattice (Fig. 2b) is insensitive to the relative phase between the beams.

The electric field in the imaging plane is:

ܧ ሬԦ ሺ,ݔ ݕሻ = 2ℇ ቂsin ቀ ଶగ ఒ ሺݔ -ݔ ሻsin ሺߠሻቁ ݁ Ԧ ௫ + sin ቀ ଶగ ఒ ሺݕ -ݕ ሻsin ሺߠሻቁ ݁ థ ݁ Ԧ ௬ ቃ (3)
where x and y are the 2D spatial coordinate of r Ԧ and Δ߶ is the relative phase between the two orthogonal standing waves. The electric field topography only depends on Δ߶ and it is depicted in Fig. 2d-2f. For Δ߶ = 0, the polarization of the field is linear everywhere in the imaging plane. If Δ߶ ≠ 0, the polarization around the zero intensity regions is linear only along the x and y axis, while it is elliptical anywhere else. It becomes circular along the directions ±45° when Δ߶ = π/2.

Faraday Discussions Accepted Manuscript

Published , where ܫ ௦௧ is the saturation intensity which is defined as the intensity at which the probability of fluorescence intensity is reduced by half 21,32 . The image provides information about the depletion pattern (structure and periodicity). The spatial regions where the fluorescence is not depleted give directly access to the expected s-PSF.

Three different rotational mobility regimes should be considered: the first regime, introduced above, is the fastrotation regime where ܶ ௧ is much shorter than the duration of the depletion pulse. The second one corresponds to the intermediate situation where the molecules display a slow rotational diffusion, characterized by ܶ ௧ much longer than the pulse duration and much shorter than the integration time. The last one is the regime of "fixed" molecules, obtained when ܶ ௧ is much longer than the integration time.

In the fast rotation regime, the transition dipole moment of the emitters can be considered as isotropic and the effective intensity profile is simply the intensity distribution of the depletion lattice. Time averaging over all possible orientations of the dipole during each depletion pulse results in a decrease of the intensity by an overall factor of 3. Therefore, the fluorescence signal can be depleted to zero in all directions around a zero intensity region. The images in Fig. 3a present the results of such simulations. The emitting dipoles are scanned over a region slightly larger than the unit cell of the optical lattice described above (θ = 60°, intensity maxima of ܫ6 ௦௧ ).

In the case of three-beam hexagonal lattice (left image) the s-PSF is elongated reflecting directly the anisotropy of the intensity distribution of Fig. 1e-f. For the four-beams square lattice, the s-PSF is isotropic and does not depend on the relative phase difference Δ߶ between the two orthogonal standing waves (as shown in the images constructed for Δ߶ = 0, ସ and ଶ ).

In the two other regimes, the polarization pattern of the electric field has to be taken into account since dipoles perpendicular to the field polarization are not depleted, even at high STED intensities. Images of a small number of emitters exhibiting slow rotational diffusion or of a bead containing a large number of fixed emitters can be modeled in the same way. The fluorescence signal is calculated by averaging the fluorescence contributions of a large number of dipoles randomly oriented in a three-dimensional space:

ܵ ௨ ~ ݁ ିሺ ୪୬ሺଶሻ ூ ூ ೞೌ ሻ ே ୀଵ ሺ4ሻ
where ܫ is the effective field intensity which depletes the excited dipole ݀ Ԧ . The results of a calculation performed with ܰ = 500 dipoles are shown in Fig. 3b. The fluorescence images display strong correlations with the polarization patterns of Fig. 1d and Fig. 2d-f. For example, in the three beam configuration, the fluorescence image shows that from each zero-intensity region (corresponding to a fluorescence maximum in the image) there Please do not adjust margins Please do not adjust margins are three directions where the fluorescence is not efficiently depleted. They correspond to that presented in Fig. 1d giving the directions along which the lattice field is linearly polarized. An interesting situation is obtained in the four-beam configuration when Δ߶ = 0. Since the electric field polarization is linear everywhere, excited emitters are not depleted where their dipole moments are perpendicular to the local field during the scan.

Faraday Discussions Accepted Manuscript

Therefore the fluorescence signal cannot be decreased to zero even at higher STED intensities. For Δ߶ = ଶ , the emitters are efficiently depleted in the diagonal directions where the polarization of the field is close to circular.

They cannot be completely depleted along the horizontal and vertical directions, where the polarization of the field is linear. Please do not adjust margins Please do not adjust margins Polarization effects are more pronounced when the localized emission stems from a small number of fixed molecules. In this case, the s-PSF depends strongly on the orientation of dipoles as shown in Fig. 3c. The images are calculated for two configurations of 20 randomly oriented dipoles. In the four beam lattice, the shape of the s-PSF varies also strongly with the lattice relative phase Δϕ. The optimal configuration, which gives the most symmetric s-PSF is obtained for Δϕ=π/2.

Experimental Setup

Our Lattice-STED microscopy setup, sketched in Fig. 4a, is based on two synchronized laser sources delivering excitation and depletion pulses, akin to a standard STED microscope 32 . The fluorescence excitation beam (wavelength 571 nm and pulse duration 2 ps) is delivered by a frequency doubled optical parametric oscillator (Mira-OPO, Coherent) pumped by picosecond Ti-sapphire laser (Mira 900 Coherent) at a repetition rate of 76 MHz. A second Ti-sapphire active mode-locked laser (Tsunami, Spectra Physics) emitting at 760 nm, provides the depletion beam. It is synchronized with the excitation laser and delivers Fourier transform limited pulses of ~100 ps duration at an average power of 2.5 W. A Spatial Light Modulator or a set of two Wollaston prisms, conjugated with the sample plane, is used to generate multiple depletion beams (Fig. 4b). The laser beams are focused on the back focal plane of a high numerical aperture objective and illuminate a wide-field region of the sample (~7.5 μm x 7.5 μm). The optical lattice produced by interference of the depletion beams at the sample plane is superimposed on a uniform excitation beam. The fluorescence emission is collected with the same objective, filtered from excitation and depletion photons and sent to a fast CMOS camera (ORCA-Flash4.0, HAMAMATSU). A fast piezo-scanner is used for sample scanning.

Please do not adjust margins

Please do not adjust margins the chosen system, the phase difference ∆߶ could be adjusted either with the SLM hologram pattern or by the use of a tilted glass plate inserted in the beam paths.

The sample is composed of a thin layer of fluorescent nano-diamonds (size 35 nm) spin-coated on a glass coverslip. The number of NV defect centers per nano-diamond is ~15 33 . The emission from individual nanodiamonds shows an extreme photo-stability. The amplitude of the signal, which varies from a nano-diamond to another, can be used for a rough estimation of the number of emitting defects contained in the nanoparticle.

Experimental Results

Fig. 5a-5b represent a fluorescence image obtained by scanning a single nano-diamond with a piezo-stage over the field of illumination in presence of both depletion and excitation beams, while recording its fluorescence with the CMOS camera. For each scanning step (15 nm), an image is acquired and the fluorescence intensity, integrated over the image spot of the tracked nano-diamond on the camera, is plotted. The signal maxima of the images correspond to the regions of minimal depletion, which occur at the zero-intensity positions of the optical lattice. As expected for the case of three interfering beams with an incidence angle ߠ = 60°, we observed a hexagonal pattern with a periodicity of 390 nm. A field of view exceeding 6 µm x 6 µm can be achieved. The zoomed image shows that the s-PSF is elongated with an aspect ratio of ~2. Directions along which the depletion is not efficient are in agreement with the simulation of Fig. 3b for a large ensemble of emitting dipoles. Indeed, this nano-diamond was chosen among those presenting the highest fluorescence emission signal, i.e. those presenting a large number of NV defects. Using a total depletion average power of 850 mW an s-SPSF of 80 nm could be achieved.

Lattice-STED super-resolved images has been acquired following the procedure described in reference 23 . The sample containing fluorescent nano-diamonds is scanned over a unit cell of the depletion lattice in the presence of the wide field excitation and the depletion pattern, while acquiring a fluorescence image for each scanning step with the s-CMOS camera. We then overlay a binary mask on these images which acts as an array of parallelized "point detectors". The complete Lattice-STED image is then obtained by assembling the entire unit cell images together obtained for each "point detector". Please do not adjust margins Please do not adjust margins polarization distribution around the zero intensity regions: the s-PSFs are elongated along the direction where the electric field is perpendicular to the dipoles average orientation. For example, the nano-diamond used in Fig. 6c has dipoles preferentially oriented along a diagonal direction. The images could be reproduced (Fig. 6d) with a set of twenty random emitters (having a diagonal average orientation) using the depletion lattices of Fig. 2 with intensity maxima of ܫ6 ௦௧ . As ∆ϕ increases up to π/2, the anisotropy of the s-PSF gets less pronounced. The configuration for which ∆ϕ = π/2 is therefore the most favorable to perform four-beam Lattice-STED microscopy.

As for the three beams configuration, we acquire Lattice-STED super-resolved images of a dense sample of nanodiamonds for ∆ϕ = π/2 . Fig. 6e and 6f display 7.5 µm x 7.5 µm images without and with the four beams depletion pattern. A clear resolution improvement between these two images is visible. The s-PSF shown in Fig. 6g (~75nm) is well below the diffraction limit. Compared to the three beams configuration, the lattice unit cell is smaller and the intensity gradients around the zero-intensity regions are steeper, enabling a larger field of view with the same averaged depletion power.

Conclusions

In this paper, we push further the investigations and the performance of Lattice-STED which massively parallelizes STED microscopy. Super-resolution images of 75 nm resolution and a field view of 7.5 μm * 7.5 μm could be achieved with moderate powers of a high repetition rate laser. Images of fluorescent nano-diamonds acquired using different optical lattices configurations are compared to theoretical simulations. The results show that the s-PSF of Lattice-STED varies with the electric field topography of the lattices and should depend to the rotational diffusion properties of the emitters. This allowed finding the optimum configuration of four-beam lattice-STED nanoscopy.

Faraday Discussions Accepted Manuscript

Published on 17 June 2015. Downloaded by UNIV BORDEAUX 1-SCD-BIBLIO UNIVERSITAIRE on 18/06/2015 09:18:55.

View Article Online DOI: 10.1039/C5FD00092K

Journal Name 2 |

 2 J. Name., 2012, 00, 1-3 This journal is © The Royal Society of Chemistry 20xx

  the lattice intensity distribution was considered for the description of the super-resolved point spread function (s-PSF). This description holds for fast-rotating fluorescent emitters with rotation characteristic time much shorter than the depletion pulse for which the polarization effects are averaged. In this paper, we study the effects of electric field topography in lattice-STED microscopy. The dependence of the s-PSF on the number of dipoles and their orientations is investigated. Single nano-diamonds containing a few Nitrogen-Vacancy (NV) fluorescent defects are imaged using different optical lattice configurations and the measured s-PSF are compared to theoretical simulations.

4 |

 4 we consider three beams propagating in the direction of the optical axis of the objective ሺ݁ Ԧ ௭ ሻ with the same linear polarization ሺ݁ Ԧ ௫ ሻ, as shown in Fig.1a. The beams intersect the objective at the vertices of a centered equilateral triangle. After passing through the objective they are deviated by an angle ߠ toward the focal region where they interfere and form an optical lattice. The incident beams are slightly focused Gaussian beams with beam waists located at the back focal plane of the objective. The transmitted beams are collimated and deviated by the objective and have their polarizations rotated. The interference of the three beams leads to a lattice with hexagonal structures with a periodicity of ଶ ଷ୬ୱ୧୬ሺሻ, n being the refractive index of the medium and λ = 760 nm the depletion wavelength. Optical lattices useful for STED nanoscopy (local zero-intensity minima, each of which surrounded by nearly isotropic intensity distribution) can be obtained when the three incident beams have theirFaraday Discussions Accepted ManuscriptPublished on 17 June 2015. Downloaded by UNIV BORDEAUX 1-SCD-BIBLIO UNIVERSITAIRE on 18/06/2015 09:18:55. J. Name., 2012, 00, 1-3 This journal is © The Royal Society of Chemistry 20xx

Fig. 1 a

 1 Fig. 1 a) Three beam configuration. b) Resulting intensity topography of the interference pattern for ߠ = 72°. c) Cross sectional intensity plots along the blue and green lines of b). d) Electric field topography for ߠ = 72°. Only the electric field polarization in the plane (x,z) is plotted. Straight lines, ellipses and circles indicate respectively linear, elliptical and circular polarizations. Red dashed lines correspond to the directions where the electric field is linear. e)-f) Resulting intensity topography and the associated cross sectional intensity plots.

6 |

 6 on 17 June 2015. Downloaded by UNIV BORDEAUX 1-SCD-BIBLIO UNIVERSITAIRE on 18/06/2015 09:18:55. J. Name., 2012, 00, 1-3 This journal is © The Royal Society of Chemistry 20xx Please do not adjust margins Please do not adjust margins

Fig 2 aଶ.

 2 Fig 2 a) Geometrical four beams configuration. b) Resulting intensity topography of the interference pattern for θ = 60°. c) Cross sectional intensity plots along the blue and green lines of b). d)-f) plots of the electric field polarization in the plane (x,y) for Δϕ = 0, ସ , ଶ . Straight lines, ellipses and circles indicate respectively linear, elliptical and circular

Published on 17 8 |

 178 June 2015. Downloaded by UNIV BORDEAUX 1-SCD-BIBLIO UNIVERSITAIRE on 18/06/2015 09:18:55. J. Name., 2012, 00, 1-3 This journal is © The Royal Society of Chemistry 20xx

.

Fig. 3

 3 Fig. 3 Simulated fluorescence depletion patterns obtained for different dipole orientations in the three and the four beams optical lattices and for different Δϕ ሺ0, π ସ , π ଶ ) in the cases of : a fast 3D rotating dipole (a), a large number of randomly oriented dipole directions (500) (b), and two different sets of 20 randomly oriented dipoles (3D) (c) at a depletion intensity of I = 6I ୱୟ୲ .

Fig. 5c and

  Fig. 5c and 5d display 5.7 μm * 5.7 μm images of a sample containing fluorescent nano-diamonds spin-coated on a glass coverslip without and with the depletion pattern, respectively. Fig. 5c clearly shows that the PSF of the nano-diamonds are diffraction limited ~290 nm, while the resolution of the Lattice-STED image of Fig. 5d is well below this limit (typically ~80 nm).

Fig. 5 a 14 |

 514 Fig. 5 a) Experimental depletion patterns. Field of view: 6 µm x 6 µm recorded at an excitation power of 16 mW, and a depletion power of 850 mW (measured before the objective). Integration time per pixel: 20 ms. b) Zoom of the central region (white dashed square). All images are normalized with respect to their maximal intensities. c) and d) Three-beam Lattice-STED images of a sample containing 35 nm fluorescent nano-diamonds without and with the depletion optical lattice respectively. Field of view: 5.7 µm x 5.7 µm. pixel size: 30 nm, pixel dwell time: 20 ms. Inset: intensity profiles of a single nano-diamond (white dashed line) which shows a FWHM of 80 nm. Images obtained with the same excitation and depletion intensities. The scale bar corresponds to counts per unit of pixel dwell time.

Published on 17 June 2015. Downloaded by UNIV BORDEAUX 1-SCD-BIBLIO UNIVERSITAIRE on 18/06/2015 09:18:55. View Article Online DOI: 10.1039/C5FD00092K

Acknowledgements

We warmly thank Philippe Tamarat for careful reading of the manuscript. We acknowledge financial support from the Agence Nationale de la Recherche, Région Aquitaine, the French Ministry of Education and Research, the European Research Council and FranceBioImaging (Grant N° ANR-10-INSB-04-01).

Faraday Discussions Accepted Manuscript

Published on 17 June 2015. Downloaded by UNIV BORDEAUX 1-SCD-BIBLIO UNIVERSITAIRE on 18/06/2015 09:18:55. View Article Online DOI: 10.1039/C5FD00092K Journal Name ARTICLE This journal is © The Royal Society of Chemistry 20xx J. Name., 2013, 00, 1-3 | 9

ARTICLE