Athanasios Boutsikakis

Pascal Fede

Annaïg Pedrono

Olivier Simonin

Numerical Simulations of Short-and Long-Range Interaction Forces in Turbulent Particle-Laden Gas Flows

Keywords: Electrostatic interactions, Particle-turbulence interaction, Tchen-Hinze theory

The main objective of this work is to study the effects of distance-dependent interactions in turbulent gas-particle flows using an Euler-Lagrange simulation approach. The turbu-lent gas flow is accounted for via Direct Numerical Simulation to the Kolmogorov scale using a spectral method to solve the Navier-Stokes equations in a cubic computational domain with tri-periodic boundary conditions. This flow simulation is coupled (one-way) with a Lagrangian particle phase solver that performs particle trajectory tracking. Electro-static forces can be calculated using two different approaches. Firstly, the direct method consists of a sum of all inter-particle interactions for all the particles of the computational domain and their periodic images. However, this purely Lagrangian approach is computa-tionally costly for a large number of particles, therefore another approximative approach is considered. According to it, one can estimate the short-range interactions via a sum of inter-particle interactions within a cutoff distance and the long-range ones via a sum of particle interactions with clusters of particles that, from a distance greater than the cut-off, are "seen" as one pseudo-particle. This method is then adjusted in order to accommodate periodic boundary conditions, which are not trivial in the case of electrostatic interactions as periodicity entails an infinite number of periodic domain images that has to be truncated to a finite number. Simulations are then performed by varying the properties of the parti-cles in terms of diameter, density (Stokes number) and charge (electrostatic Stokes num-ber). Finally, a statistical analysis is performed in order to investigate how the dynamics of the turbulent gas-particle flow are affected by distance-dependent particle-particle interac-tions, namely electrostatic forces.

Introduction

Particulate flows are found in many practical applications from geophysical flows (pyroclastic flow, s ediments t ransport, v olcano a shes d ispersion, . ..) t o i ndustrial a pplications (pneumatic conveying, olefin polymerization, Fluid Catalytic Cracking of oil, silo discharge, ...). In these systems the inter-particle collisions or the particle-wall bouncing may lead the particles to accumulate electric charges which modify the dynamic behavior of the particles. As for example in fluidized beds [START_REF] Rokkam | Computational fluid d ynamics a nd e lectrostatic m odeling o f polymerization fluidized-bed r eactors[END_REF][START_REF] Kolehmainen | A hybrid approach to computing electrostatic forces in fluidized beds of charged particles[END_REF], charges may lead the particles to form agglomerates [START_REF] Ciborowski | On electrostatic effects in fluidized beds[END_REF] that can change the fluidization regime or to adhere on the walls [START_REF] Hendrickson | Electrostatics and gas phase fluidized bed polymerization reactor wall sheeting[END_REF]. In case of pneumatic conveying [START_REF] Joseph | Vertical gas-solid transition flow with electrostatics[END_REF] or solid entrainment [START_REF] Baron | Electrostatic effects on entrainment from a fluidized bed[END_REF], charges may also alter granular flow d ynamics. Effects of charges on particle dynamics have also been identified in geophysical flows especially for dust emission [START_REF] Esposito | The role of the atmospheric electric field in the dustlifting process[END_REF]) and for the saltating motion of grains [START_REF] Schmidt | Electrostatic force on saltating sand[END_REF][START_REF] Zheng | The effect of electrostatic force on the evolution of sand saltation cloud[END_REF]. Finally, electrostatic forces may be important in colloids, particularly regarding deposition [START_REF] Li | Aerosol particle deposition with electrostatic attraction in a turbulent channel flow[END_REF].

The present study is dedicated to the dispersion of inertial charged particles in turbulent flows. S everal l iterature studies h ave b een d edicated to charged p articles t ransported by homogeneous isotropic turbulence [START_REF] Lu | Clustering of charged inertial particles in turbulence[END_REF][START_REF] Lu | Charged particle dynamics in turbulence: theory and direct numerical simulations[END_REF][START_REF] Karnik | Mitigation of preferential concentration of small inertial particles in stationary isotropic turbulence using electrical and gravitational body forces[END_REF][START_REF] Yao | Competition between drag and Coulomb interactions in turbulent particle-laden flows using a coupled-fluid-Ewald-summation based approach[END_REF][START_REF] Di Renzo | Aerodynamic generation of electric fields in turbulence laden with charged inertial particles[END_REF] or by turbulent channel flow [START_REF] Rambaud | On the behavior of charged particles in the near wall region of a channel flow[END_REF]. These studies essentially focus on the modification of preferential concentration in the case of charged particles. Indeed, when solid non-charged particles are transported by a turbulent flow field, according to their inertia, they may accumulate in low-vorticity regions of the turbulence [START_REF] Squires | Preferential concentration of particles by turbulence[END_REF][START_REF] Fessler | Preferential concentration of heavy particles in a turbulent channel flow[END_REF]. Such a mechanism leads to large local concentrations of particles that may modify the collision rate [START_REF] Sundaram | Collision statistics in an isotropic particle-laden turbulent suspension. Part 1. Direct numerical simulations[END_REF][START_REF] Reade | Effect of preferential concentration on turbulent collisions rates[END_REF] or the coalescence rate if droplets are considered instead of solid particles [START_REF] Wunsch | Numerical simulation and statistical modeling of inertial droplet coalescence in homogeneous isotropic turbulence[END_REF]. Furthermore, external forces may modify preferential concentration, as for example, [START_REF] Fede | Effect of particle-particle collisions on the spatial distribution of inertial particles suspended in homogeneous isotropic turbulent flows[END_REF] showed that inter-particle collisions enhance preferential concentration. In addition, [START_REF] Dejoan | Preferential concentration and settling of heavy particles in homogeneous turbulence[END_REF] investigated experimentally the effect of gravity on preferential concentration. [START_REF] Lu | Clustering of charged inertial particles in turbulence[END_REF], [START_REF] Lu | Charged particle dynamics in turbulence: theory and direct numerical simulations[END_REF], [START_REF] Karnik | Mitigation of preferential concentration of small inertial particles in stationary isotropic turbulence using electrical and gravitational body forces[END_REF], [START_REF] Yao | Competition between drag and Coulomb interactions in turbulent particle-laden flows using a coupled-fluid-Ewald-summation based approach[END_REF] show that preferential concentration decreases when charges increase. However, they do not investigate how particle dispersion, in terms of agitation, is modified b y e lectrostatic interactions. Basically, particles transported by stationary homogeneous isotropic turbulence follow the Tchen-Hinze theory [START_REF] Hinze | Turbulent fluid and particle interaction[END_REF]) that entails that the particle kinetic energy is directly linked to the fluid agitation (see for example [START_REF] Zaichik | Two statistical models for predicting collision rates of inertial particles in homogeneous isotropic turbulence[END_REF][START_REF] Fede | Numerical study of the subgrid fluid turbulence effects on the statistics of heavy colliding particles[END_REF]) via fluid-particle v elocity c ovariance. H owever, L aviéville e t a l. (1997) and later [START_REF] Fede | Monte-Carlo simulation of colliding particles or coalescing droplets transported by a turbulent flow in the framework of a joint fluid-particle pdf approach[END_REF] showed that when inter-particle collisions occur, particle agitation decreases even for elastic collisions. The way that electrostatic forces act on the particles is similar to collisions via the mechanism of Coulomb collisions typically found in cold plasma [START_REF] Callen | Fundamentals Of Plasma Physics[END_REF].

In the present paper, DNS of stationary homogeneous isotropic turbulence are performed and coupled with Lagrangian tracking of the particles. Details of the physical problem are given in Sect. 2 and the numerical methods deployed are described in Sect. 3 with focus on the algorithm that accounts for electrostatic forces. The main material properties and statistics of the turbulence and particles are both given in Sect. 4. In this section, the case without charge is briefly analyzed. The main part of the paper is Sect. 5 where the effect of charges on the dispersion of inertial particles is analyzed. Conclusions are drawn in the last section of the paper.

Governing Equations for Charged Particles Transported by a Turbulent Fluid Flow

The studied configuration is a stationary homogeneous isotropic turbulent gas flow carrying mono-disperse inertial particles. The incompressible Navier-Stokes equations are solved by using a pseudo-spectral method. Periodic boundary conditions are applied on fluid and particle phase. Statistically steady flow is achieved by using the forcing scheme proposed by [START_REF] Eswaran | An examination of forcing in direct numerical simulations of turbulence[END_REF]. The solid mass loading is sufficiently low for neglecting the turbulence modulation by the presence of particles.

The particles are considered as mono-disperse, with diameter d p , and inertial, with den- sity 𝜌 p ≫ 𝜌 f . According to [START_REF] Maxey | Equation of motion for a small rigid sphere in a non uniform flow[END_REF] and [START_REF] Gatignol | The Faxen formulae for a rigid particle in an unsteady non uniform Stokes flow[END_REF], for the case of a large particle-to-fluid density ratio and for a particle diameter that is smaller than the Kolmogorov length scale, the force acting on the particles is only the drag force . In addition, if the particles are charged, then electrostatic force e may also be significant. Particle motion is governed by where p is the particle position and p the particle velocity. In Eq. ( 2), the fluid-particle drag force is calculated by where p is the particle relaxation time given by [START_REF] Schiller | A drag coefficient correlation[END_REF] with f the kinematic fluid viscosity, and Re p = d p ‖ pf @p ‖∕ f the particle Reynolds number. In Eq. (3), f @p is the undisturbed fluid velocity at the particle position. The particle positions and velocities are time-advanced by a second order Adams-Bashforth scheme.

Coulomb's law allows to calculate the electrostatic force q→p acting on p due to q as following where is Coulomb's constant (with = 1∕ 4 0 where 0 is the vacuum permittivity), Q p is the electric charge of particle p and qp = pq is the distance vector between parti- cles p and q pointing to p as depicted in Fig. 1.

(1)

d p dt = p (2) m p d p dt = e + d (3) d = - p -f @p p m p (4) p = p d p 2 18 f f 1 1 + 0.15Re p 0.687 (5) q→p = Q q Q p ‖ qp ‖ 3 qp
In a system of N p charged particles, each particle p interacts with all N p -1 particles in the computational domain, thus the total electrostatic force exerted on p is Next section is dedicated to the numerical methods for computing electrostatic force e .

Numerical Methods for the Computation of Electrostatic Forces

Direct Dipole Summation

This method consists in calculating the total electrostatic force on a particle p by directly summing all the N p -1 terms that correspond to the electrostatic interactions of the par- ticle with all particles but itself as seen by Eq. ( 6). However, one could use Newton's 3rd law for such a dipole that gives p→q = -q→p in order to perform one operation per dipole thus divide the computational cost by two resulting in N p N p -1 ∕2 ∼ N p 2 operations which is forbiddingly expensive.

Quasi-periodic Boundary Conditions

Periodic boundary conditions for the particle phase correspond to an infinite domain. Therefore, to compute the electrostatic forces it is necessary to take into account the contributions of all particles, including those not really represented/computed in the computational domain.

Let a cubic computational computational domain of length L (see Fig. 2). Consequently, consider a super-domain (due to periodic BCs) of (finite) length 2N per + 1 L , where N per is the number of domain images per direction. If d is the number of (periodic) physical dimensions, then the number of periodic domain images including the original domain is N im = 2N per + 1 d . In theory, periodic boundary conditions are exactly represented for N per → ∞ , however in practice N per will be considered finite based on a con- vergence criterion [see Eq. ( 8)] which entails an approximation error, thus these BCs are considered as quasi-periodic. Therefore, each particle in has N im -1 periodic images, and the electrostatic force acting on a particle p due to a particle q is the sum of all N im interactions due to q and its images.

(6) e = N p ∑ q = 1 q ≠ p q→p .
Fig. 1 Notations in electrostatic interaction of a dipole of isocharged particles which implies the particle electric potential energy diverges quadratically for tri-periodic boundary conditions as shown by Fig. 3.

(7) U ep = 𝜆Q p N p � q = 1 q ≠ p Q q N per � m, n, l = -N per ‖̃ qp ‖ ≤ N per L 1 ‖̃ qp ‖ ∝ N per 2 .
Fig. 2 The computational domain of interest is marked with bold contours and hatched in gray lines and its periodic images in white font. Two particles are put in the domain for simplicity and for each one, a spherical periodic volume of radius ‖ c ‖ = N per L is centered at their respective positions

Hence, in the case of tri-periodic BCs, in order to calculate the total electrostatic force exerted on p, one should perform the sum of Eq. ( 6) over all N p N im particles in the super-domain. However, the computational domain (and its periodic images) is cubic, so a particle that is away from the domain center would be subjected to a force that always points outwards of the domain due to an anisotropic long-range electrostatic force distribution (see Fig. 2). In fact, this force would be proportional to the offset of the particle from the center of the computational domain. As a result, all particles would accelerate towards the domain borders and due to periodicity reenter from the opposite side, where they would re-accelerate outwards, which would eventually result in an oscillation of the particles around the borders. This behavior is not physical and in order to avoid it, each particle p interacts with the particles (images) that are located within a sphere ( p , c ) , where ‖ c ‖ = N per L , in order to ensure an isotropic long- range electrostatic force distribution.

In practice, if the positions of N p particles in the domain are known, then for the calcula- tion of e p only N p -1 distance vectors qp are needed for N p -1 particles q in . Then, for each particle q, the N im distance vectors between p and the images of particle q (including itself) can be simply calculated as a translation of the original distance vector ̃qp = qp + mL x + nL y + lL z with m, n, l =-N per , N per . Apparently, for m = n = l = 0 the distance vector is the original one. As more periodic images are taken into consideration, one particle that lies in the computational domain interacts with more particles N im ∝ N per 3 that are placed in an increasingly greater distance ‖ ̃qp ‖ ∝ N per . One can deduce that As far as the total electrostatic force on a particle is concerned, the same dynamics are in place due to periodicity, however it should be noted that e ∝ 1∕‖ qp ‖ 2 ∝ 1∕N per 2 . In addition, although the number of electrostatic interactions scales with N per 3 , the electrostatic force converges for N per → ∞ due to its vectorial nature and the symmetry of periodicity. In fact, Fig. 4 shows that electrostatic forces converge very quickly for N per ≥ 2 , hence for the simulations carried out in this work, the domain is replicated two times towards each direction. Theoretically, from the point of view of a particle in the computational domain, as more particle images are taken into account around it, there is a cut-off distance after which the long-range electrostatic forces that are exerted on it tend to cancel out as N per → ∞ . This is due to an isotropic long-range force distribution around it, resulting in the convergence of its total electrostatic force to a finite value. From Eq. ( 6), one can deduce that For periodic BCs, the computational cost of the direct method scales with the number of periodic images as

N p N im ⏟⏟⏟ particles + images N p -1 ∕2 ∼ N im N p 2 .

Pseudo-particle Method

Let the domain be discretized in N f cells per dimension (see Fig. 5). For each particle p, its neighborhood V p of size

x v = 2N v + 1 x is defined as the ensemble of 2N v + 1 3 cells
around it. The number of cells for which the neighborhood spans towards every direction x, y, z (excluding the cell that contains the particle) is N v , as illustrated by Fig. 5. Each cell Ω k contains N k ∼ N p ∕N 3 f particles and forms a pseudo-particle, which is a cluster of particles "viewed" from distance as one particle of equivalent charge Q k eq and position k eq . The concept of pseudo-particles, inspired by [START_REF] Barnes | A hierarchical O(N log N) force-calculation algorithm[END_REF], is defined in Eqs. ( 9)-( 11).

with (8)

e = 𝜆Q p N p � q = 1 q ≠ p Q q N per � m, n, l = -N per ‖̃ qp ‖ ≤ N per L ̂ qp ‖̃ qp ‖ 2 ∝ N per � m = -N per ‖̃ qp ‖ ≤ N per L (m) = e . (9) Q k eq = N k ∑ p=1 k p Q p (10) k eq = 1 Q k eq N k ∑ p=1 k p Q p p (11) k p = 1 particle p is in cell k 0 otherwise Fig. 5 Computational domain of length L is discretized in N f cells per dimension. For each particle p, its neighborhood V p (red) is defined as the ensemble of 2N v + 1 3 cells around it.
Pseudo-particles are represented as gray circles outside of V p Therefore, each particle interacts directly with all particles in neighborhood V p (short-range interaction), as well as with the pseudo-particles that are outside of V p (long-range interaction). The electrostatic force k→p acting on particle p due to pseudo-particle k is calculated by treating k as another particle, meaning that in Eq. ( 5) Q q is replaced by Q k eq and q by k eq . Hence, the total electrostatic force on particle p is calculated by performing direct summations in V p and pseudo-particle summations outside of V p .

Quasi-periodic Boundary Conditions

Let the number of pseudo-particles plus their periodic images be

N im N f 3 . Apparently, N v must satisfy the inequality N v ≤ N f 2N per + 1 -1 ∕2 in order to fit in 2N per + 1 L , as
shown by Fig. 6. As a result, the computational cost with periodic BCs becomes

N p ⎡ ⎢ ⎢ ⎢ ⎢ ⎣ � 2N v + 1 � 3 N p ∕N 3 f -1 ⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟ inside V p + N im N f 3 - � 2N v + 1 � 3 ⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟ outside V p ⎤ ⎥ ⎥ ⎥ ⎥ ⎦
.

Error and Performance Analysis

An error and performance analysis of the proposed algorithm is presented in Fig. 7. As the error analysis is concerned, the direct method is considered to be of high fidelity as it allows for an exact calculation of the electrostatic forces and hence the results obtained with this method constitute a reference for simulations using other approximative methods such as the pseudo-particle method. Therefore, the relative error of total electrostatic force estimation for particle p at t = 0 is defined as

0 p = � ‖ 0 e � �dir -0 e � �pseudo ‖ � ∕‖ 0 e � �dir ‖.
It is obvious that the latter is performing better than the former as it approximates the longrange interactions via pseudo-particles. It should also be noted that the calculated order of complexity ∼ N p 1.3 is considerably better than the expected theoretical one ∼ N p 1.5

.

Numerical Simulation Setup and Flow Statistics

Numerical Schemes and Parameters

The numerical solver is fully parallelized for both the fluid and the particles. For the DNS the FFT are performed by the library P3DFFT [START_REF] Pekurovsky | P3DFFT: a framework for parallel computations of fourier transforms in three dimensions[END_REF]). In the present numerical ( 12)

e = N 3 f ∑ k = 1 k ∈ V p N k ∑ q = 1 q ≠ p q→p ⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟ short range 2N v + 1 3 N p ∕N 3 f -1 terms + N 3 f ∑ k = 1 k ∉ V p k→p ⏟⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏟ long range N f 3 -2N v + 1 3 terms f ,i f ,i ⟩ f rian
average on the grid used for the fluid phase.

In this work, all particles are considered to bear equal positive charges. Practically, particles are charged via the phenomenon of triboelectrification (Grosshans and Papalexandris 2017) which occurs when particles collide with walls and other particles. In the simulated periodic particulate flow, it is assumed that particles have had sufficient time to redistribute their charges among them via collisions. However, collisions and triboelectrification are neglected in this study because of the small solid volume fraction.

In fact, numerical simulations have been performed with N p = 10, 000 particles corre- sponding to a solid volume fraction p = 2.64 × 10 -6 . In addition, all particles have been considered of the same diameter so that K ∕d p = 4.56 but they differ by density. Since par- ticle diameter is smaller than the Kolmogorov scale, particles are numerically treated under the point-particle approximation. As such, the particle charge Q p is considered to be con- centrated in one point (particle's center of mass) defined as Q p = d p 2 × q , where q the particle surface charge density.

L f = 2∕3q f 2 L f ∕ f , is about 100. According to K F max = 2.
92 (where K is the Kolmogorov length scale and max the highest solved wavenumber) the small turbulent scales are well resolved. As two-way coupling is neglected, the fluid velocity seen by the particles, f @p , is computed by 3rd order polynomial interpolation of the fluid velocity field. According to the previous analysis and adjusting the parameters for optimal parallelization, the electrostatic forces have been computed with N f = 8 , N v = 1 and N per = 2.

Fluid and Particle Material Properties

The fluid material properties and main turbulence statistics are gathered in Table 1. In this table, the fluid kinetic energy is defined by q f 2 = 1∕ 2⟨u � u � where ⟨ . ⟩ f denotes an Eule- Several particle charges have been considered, all given in terms of a reference charge Q 0 = 1 × 10 -9 C . It should be noted here, that according to Hamamoto et al. (1992) there is a saturation limit of surface charge density for small spheres, which can be translated (via d p ) to a corresponding limit for point-particle charges. For the configuration presented in this work, this value can be estimated to be approximately 4 × 10 -9 C, which is of the same order of magnitude of the charges used in all simulations.

In Table 2, particle Stokes number based on the Stokes drag and the Kolmogorov time scale is given. However, as we consider non-linear drag force (see Eq. (3)), the particle relaxation time scale is introduced as F fp = � ⟨1∕ p ⟩ p � -1 where ⟨ . ⟩ p denotes a Lagrangian average on the particle distribution. Therefore, the Stokes number is defined as F fp ∕ t f @p where t f @p is the Lagrangian integral time scale of the fluid seen by the inertial particles. Such a timescale is defined from the Lagrangian auto-correlation function of the fluid velocity seen:

where q 2 f @p = 1∕2⟨u � f @p,i u � f @p,i ⟩ p the kinetic energy of the fluid seen by the particles. The Lagrangian fluid integral time scale is defined by When motionless particles are considered, Eqs. ( 13) & ( 14) allow to compute the Eulerian time scale, E , and for fluid elements the Lagrangian fluid integral time scale, t f . For the following statistical analysis we also define the particle kinetic energy, q 2 p = 1∕2⟨u � p,i u � p,i ⟩ p and the fluid-particle velocity covariance as q fp = ⟨u � f ,i u � p,i ⟩ p . Furthermore, statistical moments have been calculated over a sufficiently long duration T stat = 10 × max{ t f @p , F fp , el } in order to achieve statistical convergence, which is calcu- lated as a multiple of the maximum characteristic time scales of all the physical mechanisms involved.

(13) R t f @p ( ) = ⟨u � f @p,i (t 0 )u � f @p,i (t 0 + )⟩ p 2q 2 f @p (t 0 )

Coulomb Collisions and Characteristic Time of Electrostatic Interactions

In order to analyze the effects of the charges one should define a characteristic time scale of electrostatic interactions. This time scale is the duration of particle velocity decorrelation under the sole presence of electrostatic interactions. The mechanism of this velocity decorrelation is the Coulomb collision, which is a binary elastic collision between two charged particles interacting through their own electric field. In order to understand a Coulomb collision of two particles, we can imagine that they undergo an elastic collision with an effective Coulomb diameter d C pq as depicted by Fig. 8. These interactions are well resolved in the DNS carried out in this work and not modeled.The effective Coulomb diameter d C pq is a notion typically found in cold plasma [START_REF] Callen | Fundamentals Of Plasma Physics[END_REF] and in this case it is defined as where pq is the relative approach velocity of the particles defined as pq = pq and the Lagrangian average of its norm is approximated by ⟨‖ pq ‖⟩ p = � 16∕ × 2∕3 × q 2 p . Additionally, m pq is the reduced mass of a pair of particles p and q calculated as m pq = m p m q ∕ m p + m q . If the kinetic energy of the particles is very large with respect to the electric potential energy, the deviation from the initial trajectories of the colliding particles is small. Therefore, d C pq ≤ d p and particles would undergo a hard sphere collision, however this is not treated in our simulations, as explained in Sect. 4.2. This length scale is (14 also used to determine the size of the neighborhood x v as it should be 𝜙≫ 1 times bigger than the effective Coulomb diameter d C pq . Therefore, one can deduce the number of cells N v in the neighborhood of short-range interactions as described in Sect. 3.

) t f @p = ∫ ∞ 0 R t f @p ( )d . (15) d C pq = 1 ⟨‖ pq ‖⟩ p � Q p Q q m pq 0
The Coulomb collision frequency is written as c = n p Q c ⟨‖ pq ‖⟩ p , therefore the charac- teristic time scale of Coulomb collisions is c = 1∕ c . An estimation of this characteristic time scale can be obtained via a simple dimensional analysis using particle flow p roperties t hat yields

In order for el to be a good candidate for the characteristic time scale of electrostatic interactions, it should be of the same order of magnitude as c . This is confirmed in Table 3 that shows time and length scales for the case where Q p = 8Q 0 .

Having defined such a time scale, one could also define an electrostatic Stokes number as F fp ∕ el -in analogy with the particles Stokes number-that is directly proportional to the charge. It can be noticed that such a definition of the electrostatic Stokes number is substantially different from those found in literature. Indeed, on one hand [START_REF] Alipchenkov | Clustering of charged particles in isotropic turbulence[END_REF] uses the Coulomb number defined as the ratio of the potential energy of Coulomb inter-particle interaction to the kinetic energy at small scales of the turbulence. On the other hand, 0 defined an electric settling velocity representing the terminal velocity of a particle due to the influence of a given electric field.

Finally, one could attempt to link the two aforementioned Stokes numbers and describe their dependence on fluid-particle-electrostatics properties. In order to perform an elementary dimensional analysis one can make two crude hypotheses: F fp ≃ St p and t f @p ≃ t f that allow to rewrite the electrostatic Stokes number in terms of the particle Stokes number as Figure 9 shows that the relation between the two Stokes numbers is quadratic and that Eq. ( 16) fits well DNS data. It is evident that in order to perform simulations with adequate resolution of the Coulomb collisions, the time step of the simulation should be appropriate. In fact, the mean free path of the particles l = ⟨‖ pq ‖⟩ p t should be at least times smaller than the Coulomb effective diameter d C pq . That allows to calculate a maximum time step that ensures the resolution of electrostatic interactions as t ≤ d C pq ∕⟨‖ pq ‖⟩ p . Conse- quently, the time step should be kept below a given limit

el ∝ 1 Q p √ m p n p . ( 16
) F fp el = 2Q p d p 2 √ √ √ √ p t f 2 f f × √ √ √ √ F fp t f @p .
t max = 3 ∕32q 2 p × √ Q p Q q ∕ 0 m pq .

Analysis of Charged Particle-Laden Turbulent Gas Flows

Simulations of charged particle-laden turbulent gas flow have been performed using the pseudo-particle method described in Sect. 3 for particles of various inertia levels and different electric charges in order to investigate how electrostatic interactions affect particle dynamics.

Figure 10 shows the comparison between particle agitation and theoretical particle agitation as predicted by the Tchen-Hinze [START_REF] Hinze | Turbulent fluid and particle interaction[END_REF], and extended by [START_REF] Simonin | Eulerian prediction of the fluid/particle correlated motion in turbulent two-phase flows[END_REF]. The theoretical relations of the extension of Tchen-Hinze theory are Figure 10 shows the particle agitation and the fluid-particle velocity covariance with respect to the Stokes number in case without charge. Numerical simulations are in agreement with Eq. 17. As expected, small discrepancies are observed for particles with small relaxation time scale but this is due to from the assumption of an exponential auto-correlation function for the fluid velocity measured along the particle trajectory that is too crude.

(17)

2q 2 p = q fp = 2q 2 f @p t f @p t f @p + F fp .

Fig. Electrostatic

Stokes number with respect to the Stokes number. The symbols are the numerical simulation and the dashed lines the Eq. ( 16)

Work by [START_REF] Zaichik | Two statistical models for predicting collision rates of inertial particles in homogeneous isotropic turbulence[END_REF] showed that the Tchen-Hinze theory can be extended by using a two-exponential correlation proposed by [START_REF] Sawford | Reynolds number effects in Lagrangian stochastic models of turbulent dispersion[END_REF].

Effect of Electrostatic Interactions on Particle Agitation

In order to examine the effect of particle-particle electrostatic interactions on particle agitation, we shall reproduce Tchen-Hinze Eq. ( 17) for various electric charges. As seen by Fig. 11, the effect of electrostatic interactions on particle agitation is not straightforward. Particles with large inertia, are not particularly affected by electrostatic interactions, those with moderate inertia exhibit an increase agitation, while on the contrary the lightest particles undergo a significant decrease in their levels of agitation.

In order to understand this behavior, one should consider the equation of particle agitation in the framework of the kinetic theory under the effect of electrostatic forces that yields

The four terms of Eq. ( 18) are shown by Fig. 12 and the balance of the equation is verified in the transient regime for Q p = 8Q 0 corresponding to an electrostatic Stokes number of F fp ∕ el = 1.63 . In case of stationary flow the Eq. ( 18) becomes

(18) t q 2 p = q fp F fp - 2q 2 p F fp + ⟨ e � p ⟩ p m p . (19) q 2 p = 1 2 q fp + 1 2 F fp ⟨ e � p ⟩ p m p .
Fig. 10 agitation with respect to the inverse of the Stokes number. The dashed line corresponds to the Tchen-Hinze theory given given by Eq. ( 17)

The additional term F fp ⟨ e � p ⟩ p ∕m p represents the particle-induced electric potential energy which is equal to the time derivative of the work of electrostatic forces. This means that particle agitation depends on fluid-particle velocity covariance (i.e. the particle-turbulence interaction) and particle-induced electric potential energy. Physically, this implies that particle agitation is modified by both the drag and the electrostatic forces. As shown by Fig. 13, increasing particle charge, term F fp ⟨ e � p ⟩∕m p becomes somewhat important Fig. 11 Effect of electric charges on particle agitation. The black dashed line corresponds to the Tchen-Hinze theory given by Eq. ( 17) and the red markers to the case without charge Fig. 12 Balance of Eq. ( 18) in transient regime for

F fp ∕ el = 1.63 ( Q p = 8Q 0 )
especially for heavy particles. Furthermore, it is always positive so it acts as production of particle agitation. However, it only accounts for a ∼ [0, 0.5]% increase of particle agitation, hence it is responsible for a very small part of its increase ∼ [15, 25]% observed in Fig. 11 for particles of moderate inertia.

The physical meaning of this term could be better understood by considering Coulomb collisions. Table 3 shows that heavy particles exhibit minimum effective diameters and collision frequency. Therefore, their Coulomb collisions are more binary than those of light particles, which implies that the Coulomb collisions of the former have the tendency to modify the particles trajectories more abruptly. Finally, the small effect on particle agitation can be attributed to the elasticity of Coulomb collisions as electrostatic forces are conservative.

Figure 14 shows that q fp is the most important term of Eq. ( 19), in that it accounts for most of the variation of particle agitation. Indeed, the observed variation of particle agitation to fluid-particle velocity covariance is of the order of magnitude ∼ 0.5% of that of particle- induced electric potential energy to particle agitation seen by Fig. 13.

Effect of Electrostatic Interactions on Fluid-Particle Velocity Covariance

It is shown in Sect. 5.1 that the key to understanding the behavior of particle agitation in such a flow, is to understand fluid-particle velocity covariance q fp , which is a measure of fluid-par- ticle correlation. Figure 11 shows that particles with large inertia, are not particularly affected by electrostatic interactions, those with moderate inertia exhibit an increase both in q fp ≃ 2q 2 p and in t f @p , while on the contrary the lightest particles undergo a significant decrease in their levels of fluid-particle velocity covariance.

In the kinetic theory framework, under the effect of electrostatic forces, the transport equation of the fluid-particle velocity covariance may be derived from the joint-fluid-particle pdf equation [START_REF] Simonin | Continuum modelling of dispersed two-phase flows[END_REF]) such as The first term of the right hand side corresponds to the turbulent fluid dissipation and it is written in terms of the acceleration of the fluid seen by the particles � f @p . In literature [START_REF] Zaichik | Two statistical models for predicting collision rates of inertial particles in homogeneous isotropic turbulence[END_REF], it is usually modeled as -q fp ∕ t f @p which is not accurate in the case of light particles as shown by Fig. 15.

Figure 16 depicts all five terms of Eq. ( 20) and the power equilibrium is verified in the transient regime for F fp ∕ el = 0.92. Furthermore, for stationary flows Eq. ( 20) becomes where term F fp ⟨ e � f @p ⟩ p ∕m p represents the turbulence-induced electric potential energy. In order to explain the behavior observed in Fig. 11 by q fp , one needs to examine the three terms of the right hand side of Eq. ( 21).

Increasing particle charge, term F fp ⟨ e � f @p ⟩ p ∕m p becomes very important, especially for light particles as shown by Fig. 17. More specifically, this term is most important for particles of moderate-to-low inertia, t f ∕ F fp ∈ [1, 10] . Therefore, in presence of electric charges the fluid-particle correlation competes with the repulsive electrostatic forces, which tend to constantly decorrelate the particle velocities from the fluid velocity field. This leads to a decorrelation of fluid-particle velocity, hence the negative sign of this term that serves as a destruction term of fluid-particle velocity covariance.

(20)

t q fp = ⟨ � f @p � p ⟩ p - q fp F fp + 2q 2 f @p F fp + ⟨ e � f @p ⟩ p m p . (21) q fp = F fp ⟨ � f @p � p ⟩ p + 2q 2 f @p + F fp ⟨ e p � f @p ⟩ p m p
Fig. 14 Ratio between the particle kinetic energy and the fluid-particle velocity covariance with respect to particle inertia However, it does not account for the increase of fluid-particle velocity covariance in the case of moderate inertia particles, F fp ∕ t f ∼ 1 as shown by Fig. 11. This increase can be due to the first term of Eq. ( 21), thus due to a modification of F fp ⟨ � f @p � p ⟩ p . Indeed, Fig. 18 shows that term F fp ⟨ � f @p � p ⟩ p is modified by electrostatic forces. In particular, this term increases for an increasing electric charge that leads to an increase of fluid-particle velocity covariance especially for particles of moderate inertia,

F fp ∕ t f ∼ 1.
In order to better understand this, one could introduce a new characteristic time scale a fp = -q fp ∕⟨ � f @p � p ⟩ p that describes the dissipation rate of fluid-particle covariance by ⟨ � f @p � p ⟩ p . In Fig. 19, it is evident that this time scale increases for increasing electric charges, which essentially means that q fp dissipates in lower rates, resulting to its increase.

At this point, it would be interesting to examine the effect of electrostatic interactions on fluid acceleration seen at the position of particles ⟨ � @ 2 ⟩ p . In Fig. 20, it is apparent that for increasing electrostatic forces there is a small decrease of ⟨ � @ 2 ⟩ p for particles of moder- ate inertia and a considerable increase for light particles, but for both cases the variance of fluid acceleration measured at the particles' position tends to the value of the fluid acceleration variance measured along fluid elements. This implies that particles tend to be distributed more homogeneously. Lastly, it would be also interesting to examine the behavior of q 2 f @p . As far as the second term of Eq. ( 20) is concerned, in Fig. 21, it is shown that an increase in electric charge implies an increase in the fluid agitation from the point of view of the particles. More specifically, q 2 f @p has the tendency to flatten to the value of q 2 f for heavy particles, F fp → ∞ and to increase even further for moderate and light particles. In order to explain this behavior, we first have to examine the levels of q 2 f @p in the case of no electric charge. Indeed, inertial and light particles "see" the same fluid agitation, as the former are rather carried by large turbulent structures and the latter behave close to fluid elements. However, particles of moderate inertia get trapped in regions of low vorticity and therefore "see" slightly lower levels of fluid agitation.

Under the influence of electric charges, inertial particles are not as affected, while moderate particles are less prone to preferential concentration, thus they "see" a fluid agitation fp ⟨ e � f @p ⟩ p ∕m p with respect to particle inertia and charges close to q 2 f . However, less inertial particles exhibit an increase in their "seen" fluid agitation, which needs to be further investigated.

Finally, Fig. 22 provides an overview of what has been discussed. It shows that for a given level of particle inertia, increasing particle charge leads to an increase of fluid-particle velocity covariance due to an increase of F fp ⟨ � f @p � p ⟩ p (see Fig. 18) and q 2 f @p (see 21). This occurs up to a saturation limit where q fp starts to decrease as the third (destruction) term of Eq. ( 21), F fp ⟨ e � f @p ⟩ p ∕m p becomes more important (see Fig. 17). Increasing particle inertia, actu- ally moves that saturation limit to lower electrostatic response time scales el → 0 , thus higher electric charges.

Effect of Electrostatic Interactions on Preferential Concentration

In case of iso-charged particles, the effects of electrostatic forces on the preferential concentration are not extensively documented, however one could reflect on the nature of electrostatic interactions by observing snapshots of the particle flow in the stationary regime. In Fig. 23 it is observed that such electrostatic interactions tend to homogenize the particle distribution due to the repulsive nature of the electrostatic forces. This leads to a mitigation and eventually an elimination of preferential concentration especially for light particles as shown by Fig. 23i.

Electrostatic forces are conservative forces, thus a system of charged particles under their influence will start evolving with an initial electric potential energy, part of which will gradually transform to kinetic energy. However, in the presence of a turbulent fluid flow, the physics of the charged particle-laden turbulent gas flow are slightly more complex, so in order to gain some more insight we should consider three main mechanisms that govern it.

Firstly, according to the minimum potential energy principle, that is valid for conservative forces such as the electrostatic, the system of particles tends to an equilibrium of minimum electric potential energy which implies that the particles will try to separate themselves as much as possible. On the other hand, it has been mentioned in Sect. 2, that in turbulent particle-laden gas flows there are certain regions in the flow (high strain rate, low vorticity) that favor particle concentration resulting in a local increase of electric potential energy and thus in further production of kinetic energy. Lastly, the additional production by electrostatic effects tends to increase the average stationary level of q 2 p . This kinetic energy production is exactly compensated, in stationary flow, by an additional dissipation due to the drag force of the fluid which dissipates it in heat by viscous effect.

Conclusions and Perspectives

This study shows that the computational cost of the N p -body problem is prohibitively high ∼ N p 2 . However, separating the interactions in short-and long-range can be a viable solution. Such a method, is the pseudo-particle method that is more efficient ∼ N p 1.5

and with an acceptable error. Furthermore, in case of tri-periodic BCs, since electrostatic forces depend on the distance of particle images, electrostatic periodicity entails an additional cost which scales with the cube of the number of periodic images per direction. However, it is shown that in order to recreate accurately enough the periodic BCs, two periodic domain images per direction are enough so as to respect the isotropy of the system at large-scales. In order to ensure such periodic isotropy, every particle has to interact with particles within a sphere centered at its position. Furthermore, a simulation of particle-laden turbulent gas flow w ithout e lectric charges has been carried out. This requires the setup of a case of resolved stationary HIT and the calibration of the particle phase according to a desired range of Stokes number, which yields several particle classes based on different densities. This calibration occurs by performing a statistical analysis of the flow, by using velocity auto-correlation functions that allow to estimate characteristic time scales of the fluid and particle flow. This statistical analysis enables to recreate Tchen-Hinze theory concerning particle agitation which serves as verification of a successful particle-laden turbulent gas flow simulation.

In addition, we attempt to explain the particle-particle electrostatic interactions via the concept of Coulomb collisions borrowed from cold plasma theory. Hence, a time scale of electrostatic interactions is proposed, whose order of magnitude has been compared to the characteristic time of Coulomb collisions. This time scale allows to define an electrostatic Stokes number in analogy to the classic particle Stokes number. Then, we calibrate the electrostatics of the charged particle-laden turbulent gas flow simulation by selecting the electric charge according to a desired range of electrostatic Stokes numbers.

Following, several charged particle-laden turbulent gas flows have been simulated with different particle charges in order to investigate the effects of electrostatic interactions on particle agitation. A statistical analysis of these flows has been carried out so as to acquire an insight of the behavioral dynamics of these flows. This statistical analysis, along with a theoretical analysis enables to rewrite the transient equations of important statistical moments and effectively derive the modified Tchen-Hinze theory for the stationary regime in the presence of electrostatic forces.

Essentially, it is shown that particle agitation depends almost solely on fluid-particle velocity covariance which is a measure of fluid-particle velocity correlation. More specifically, particle-particle electrostatic forces lead to fl uid-particle ve locity decorrelation which is particularly prominent for very light particles. Furthermore, a qualitative analysis shows that electrostatic interactions of iso-charged particles mitigate preferential concentration.

Finally, these two last observations imply that particles, under the influence of electrostatic forces, "see" a slightly different flow which entails a change in the Lagrangian integral time scale f t @p as seen in Fig. 11. Therefore, in the future, more work needs to be carried out towards the investigation of the effect of electrostatic interactions on the auto-correlation functions of fluid velocity "seen" by inertial particles as well as auto-correlation functions of particle velocity. This study could allow us to further investigate particle dispersion, preferential concentration as well as modeling the electrostatic-related terms in the modified Tchen-Hinze equation.

Fig. 3 Fig. 4

 34 Fig. 3 Average particle electric potential energy with respect to the number of periodic domain images. Quadratic divergence observed as N per → ∞

Fig. 6

 6 Fig. 6 Domain of interest (solid black), periodic domain images (dashed black) with N per = 2 and spherical periodic volume (dashed red) with radius ‖ c ‖ = 2L . Neighborhood V p (solid red) with N v = 1 could span over several images of

Fig. 7

 7 Fig. 7 Error distribution (a) and performance (b) of periodic direct and pseudo-particle algorithm for N v = 1 and N per = 2

Fig. 8

 8 Fig. 8 Schematic of a Coulomb collision of two iso-charged particles and notion of Coulomb diameter. Particle diameter d p is depicted with a solid black circle, while (fictitious) effective Coulomb diameter with a dashed black circle

Fig. 13

 13 Fig. 13 Evolution of F fp ⟨ e � p ⟩ p ∕m p with respect to particle inertia and electric charges

Fig. 15 Fig. 16

 1516 Fig. 15 Comparison between DNS results (black-filled symbols) and the standard model (open symbols) for term ⟨ � f @p

Fig. 17

 17 Fig. 17 Evolution of Ffp ⟨ e

Fig. 18

 18 Fig. 18 Effect of electric charges on acceleration termF fp ⟨ � f @p

Fig. 19

 19 Fig. 19 Effect of electric charges on time scale a fp with respect to particle inertia. The red markers correspond to the case without charge

Fig. 21 )

 21 Fig.21) which are the first two (production) terms of Eq. (21). This occurs up to a saturation limit where q fp starts to decrease as the third (destruction) term of Eq. (21),

Fig. 20

 20 Fig. 20 Effect of electric charges on fluid acceleration at the particles position ⟨ � @ 2 ⟩ p with respect to particle inertia

Fig. 22

 22 Fig.22Fluid-particle covariance normalized by the value without charge with respect to the electrostatic Stokes number. Stokes numbers F fp ∕ t f @p correspond to the value without charge

Fig. 23

 23 Fig. 23 Snapshots of a charged particle-laden turbulent gas flow for various levels of inertia and electric charges for x = L f

Table 1

 1 Properties of the fluid and of the examined HIT

	Parameters	Value	Units
	Computational domain length, L x = L y = L z = L	2	m
	Fluid kinematic viscosity, f	1 × 10 -3	m 2 ∕s
	Fluid density, f	1.0	kg∕m 3
	Fluid kinetic energy, q f Eddy lifetime, T e = L f ∕ 2 √ 2∕3q 2 f	3.16 × 10 -2 4.49	m 2 ∕s 2 s
	Fluid longitudinal integral length scale, L f ∕L 1.04 × 10 -1	-
	Fluid integral time scale, t f ∕T e	6.97 × 10 -1	-
	Reynolds number, Re L f	99.39	-
	Kolmogorov length scale, K ∕L f	3.50 × 10 -2	-
	Kolmogorov time scale, K ∕T e	1.16 × 10 -1	-
	K	F max	2.92	-

Table 3

 3 Characteristic scales of Coulomb interactions forQ p = 8Q 0

	Class	F fp ∕ el	d C pq (m)	c (Hz)	d C pq ∕d p	l∕d C pq × 10 -3	el ∕ c	el ∕ t
	1	3.20	0.286	0.30	57.25	2.04	2.28	1502
	2	2.28	0.303	0.45	60.61	2.58	2.41	1062
	3	1.63	0.334	0.71	66.80	3.00	2.66	752
	4	1.22	0.379	1.08	75.76	3.14	3.02	558
	5	0.92	0.449	1.74	89.86	3.03	3.58	412
	6	0.66	0.573	3.14	114.66	2.63	4.55	290
	7	0.49	0.741	5.55	148.23	2.15	5.88	212
	8	0.34	1.011	10.70	202.11	1.64	8.04	150

Acknowledgements

The numerical simulations were performed on supercomputer Olympe (CALMIP) using time available under project P0111.

Compliance with Ethical Standards

Conflict of interest

On behalf of the co-authors, I hereby testify that the contents of the manuscript are original and have never been published or submitted elsewhere. The authors declare that they have no conflict of interest.


Baihan Lin

Xinxin Zhang
email: zhangx43@uw.edu

SPEAKER DIARIZATION AS A FULLY ONLINE LEARNING PROBLEM IN MINIVOX

Keywords: Speaker diarization, online learning, semi-supervised learning, self-supervision, contextual bandit

We proposed a novel AI framework to conduct realtime multi-speaker diarization and recognition without prior registration and pretraining in a fully online learning setting. Our contributions are two-fold. First, we proposed a new benchmark to evaluate the rarely studied fully online speaker diarization problem. We built upon existing datasets of real world utterances to automatically curate MiniVox, an experimental environment which generates infinite configurations of continuous multi-speaker speech stream. Secondly, we considered the practical problem of online learning with episodically revealed rewards and introduced a solution based on semi-supervised and self-supervised learning methods. Lastly, we provided a workable web-based recognition system which interactively handles the cold start problem of new user's addition by transferring representations of old arms to new ones with an extendable contextual bandit. We demonstrated that our proposed method obtained robust performance in the online MiniVox framework.

INTRODUCTION

Speaker recognition involves two essential steps: registration and identification [START_REF] Sremath Tirumala | Speaker identification features extraction methods: A systematic review[END_REF]. In laboratory setting, the state-ofthe-art approaches usually emphasize the registration step with deep networks [START_REF] Snyder | X-vectors: Robust dnn embeddings for speaker recognition[END_REF] trained on large-scale speaker profile dataset [START_REF] Nagrani | Voxceleb: a large-scale speaker identification dataset[END_REF]. However, in real life, requiring all users to complete voiceprint registration before a multi-speaker teleconference is hardly a preferable way of system deployment. Dealing with this challenge, speaker diarization is the task to partition an audio stream into homogeneous segments according to the speaker identity [START_REF] Anguera | Speaker diarization: A review of recent research[END_REF]. Recent advancements have enabled [START_REF] Sremath Tirumala | Speaker identification features extraction methods: A systematic review[END_REF] contrastive audio embedding extractions such as Mel Frequency Cepstral Coefficients (MFCC) [START_REF] Md Rashidul Hasan | Speaker identification using mel frequency cepstral coefficients[END_REF], ivectors [START_REF] Stephen H Shum | Unsupervised methods for speaker diarization: An integrated and iterative approach[END_REF] and d-vectors [START_REF] Wang | Speaker diarization with lstm[END_REF]; [START_REF] Snyder | X-vectors: Robust dnn embeddings for speaker recognition[END_REF] effective clustering modules such as Gaussian mixture models (GMM) [START_REF] Zajíc | Speaker diarization using convolutional neural network for statistics accumulation refinement[END_REF], mean shift [START_REF] Senoussaoui | A study of the cosine distance-based mean shift for telephone speech diarization[END_REF], Kmeans and spectral clustering [START_REF] Wang | Speaker diarization with lstm[END_REF] and supervised Bayesian non-parametric methods [START_REF] Fox | A sticky hdp-hmm with application to speaker diarization[END_REF][START_REF] Zhang | Fully supervised speaker diarization[END_REF]; and (3) reasonable resegmentation modules such as Viterbi and factor analysis subspace [START_REF] Sell | Diarization resegmentation in the factor analysis subspace[END_REF]. In this work, we proposed a new paradigm to consider the speaker diarization as a fully online learning problem of the speaker recognition task: it combines the embedding extraction, clustering and resegmentation into the same problem as an online decision making problem.

Why is this online learning problem different? The state-of-the-art speaker diarization systems usually require large datasets to train their audio extraction embeddings and clustering modules, especially the ones with deep neural networks and Bayesian nonparametric models. In many real-world applications in developing countries, however, the training set can be limited and hard to collect. Since these modules are pretrained, applying them to out-of-distribution environments can be problematic. For instance, an intelligent system trained with American elder speaker data might find it hard to generalize to a Japanese children diarization task because both the acoustic and contrastive features are different. To tackle this problem, we want the system to learn continually. To push this problem to the extreme, we are interested in a fully online learning setting, where not only the examples are available one by one, the agent receives no pretraining from any training set before deployment, and learns to detect speaker identity on the fly through reward feedbacks. To the best of our knowledge, this work is the first to consider diarization as a fully online learning problem. Through this work, we aim to understand the extent to which diarization can be solved as merely an online learning problem and whether traditional online learning algorithms (e.g. contextual bandits [START_REF] Langford | The epoch-greedy algorithm for contextual multi-armed bandits[END_REF][START_REF] Lin | Unified models of human behavioral agents in bandits, contextual bandits and rl[END_REF][START_REF] Lin | Online learning in iterated prisoner's dilemma to mimic human behavior[END_REF]) can be a practical solution.

What is a preferable online speaker diarization system? A preferable AI engine for such a realistic speaker recognition and diarization system should (1) not require user registrations, (2) allow new user to be registered into the system real-time, (3) transfer voiceprint information from old users to new ones, (4) be up running without pretraining on large amount of data in advance. While attractive, assumption (4) introduced an additional caveat that the labeling of the user profiles happens purely on the fly, trading off models pretrained on big data with the user directly interacting with the system by correcting the agent as labels. To tackle these challenges, we formulated this problem into an interactive learning model with cold-start arms and episodically revealed rewards (users can either reveal no feedback, approving by not intervening, or correcting the agent).

Why do we need a new benchmark? Traditional dataset in the speaker diarization task are limited: CALL-HOME American English [START_REF] Canavan | Callhome american english speech ldc97s42. web download[END_REF] and NIST RT-03 English CTS [START_REF] Martin | The nist 1999 speaker recognition evaluation-an overview[END_REF] contained limited number of utterances recorded under controlled conditions. For online learning experiments, a learn-from-scratch agent usually needs a large length of data stream to reach a comparable result. Large scale speaker recognition dataset like VoxCeleb [START_REF] Nagrani | Voxceleb: a large-scale speaker identification dataset[END_REF][START_REF] Nagrani | Voxceleb: Large-scale speaker verification in the wild[END_REF] and Speakers in the Wild (SITW) [START_REF] Mclaren | The speakers in the wild (sitw) speaker recognition database[END_REF] contained thousands of speaker utterances recorded in various challenging multi-speaker acoustic environments, but they are usually only used to pretrain diarization embeddings. In this work, we proposed a new benchmark called MiniVox, which can transform any large scale speaker identification dataset into infinitely long online audio streams with various configurations.

We built upon LinUCB [START_REF] Li | A contextual-bandit approach to personalized news article recommendation[END_REF] and proposed a semi-supervised learning variant to account for the fact that the rewards are entirely missing in many episodes. For each episode without feedbacks, we applied a self-supervision process to assign a pseudo-action upon which the reward mapping is updated. Finally, we generated new arms by transferring learned arm parameters to similar profiles given user feedbacks.

THE FULLY ONLINE LEARNING PROBLEM

Algorithm 1 presents at a high-level our problem setting, where c(t) ∈ R d is a vector describing the context at time t, r a (t) ∈ [0, 1] is the reward of action a at time t, and r(t) ∈ [0, 1] K denotes a vector of rewards for all arms at time t. P c,r denotes a joint probability distribution over (c, r), and π : C → A denotes a policy. Unlike traditional setting, in step 5 we have the rewards revealed in an episodic fashion (i.e. sometimes there are feedbacks of rewards being 0 or 1, sometimes there are no feedbacks of any kind). We consider our setting an online semi-supervised learning problem [START_REF] Yver | Online semi-supervised learning: Application to dynamic learning from radar data[END_REF][START_REF] Ii Ororbia | Online semi-supervised learning with deep hybrid boltzmann machines and denoising autoencoders[END_REF], where the agent learns from both labeled and unlabeled data in online setting. (c(t), r(t)) is drawn according to P c,r 3: c(t) is revealed to the player 4:

Player chooses an action i = π t (c(t))

5:
Feedbacks r a (t) for all arms are episodically revealed 6:

Player updates its policy π t 7: end for 3. PROPOSED ONLINE LEARNING SOLUTION

Contextual Bandits with Extendable Arms

In an ideal online learning scenario without oracle, we start with a single arm, and when new labels arrive new arms are then generated accordingly. This problem is loosely modelled by the bandits with infinitely many arms [START_REF] Berry | Bandit problems with infinitely many arms[END_REF]. For our specific application of speaker registration process, we applied the arm expansion process outlined in Figure 1: starting from a single arm (for the "new" action), if a feedback confirms a new addition, a new arm is initialized and stored.

Episodically Rewarded LinUCB

We proposed Background Episodically Rewarded LinUCB (BerlinUCB [START_REF] Lin | Online semi-supervised learning in contextual bandits with episodic reward[END_REF]), a semi-supervised and self-supervised online contextual bandit which updates the context representations and reward mapping separately given the state of the feedbacks being present or missing (Algorithm 2). We assume that (1) when there are feedbacks available, the feedbacks are genuine, assigned by the oracle, and (2) when the feedbacks are missing (not revealed by the background), it is either due to the fact that the action is preferred (no intervention required by the oracle, i.e. with an implied default rewards), or that the oracle didn't have a chance to respond or intervene (i.e. with unknown rewards). Especially in the Step 15, when there is no feedbacks, we assign the context x t to a class a (an action arm) with the self-supervision given the previous labelled context history. Since we don't have the actual label for this context, we only update the reward mapping parameter b a and leave the covariance matrix A a untouched. This additional usage of unlabelled data (or unrevealed feedback) is especially important in our model.

Self-Supervision and Semi-Supervision Modules

We construct our self-supervision modules given the cluster assumption of the semi-supervision problem: the points within the same cluster are more likely to share a label. As shown in many work in modern speaker diarization, clustering algorithms like GMM [START_REF] Zajíc | Speaker diarization using convolutional neural network for statistics accumulation refinement[END_REF], mean shift [START_REF] Senoussaoui | A study of the cosine distance-based mean shift for telephone speech diarization[END_REF] and spectral clustering [START_REF] Wang | Speaker diarization with lstm[END_REF] are especially powerful unsupervised modules, especially in their offline versions. Their online variants, however, often performs poorly [START_REF] Zhang | Fully supervised speaker diarization[END_REF]. As in this work, we fo-

Initialize c t ∈ R + , A a ← I d , b a ← 0 d×1 ∀a ∈ A t 2: for t = 1,2,3,• • • , T do 3: Observe features x t ∈ R d 4:
for all a ∈ A t do 5:

θa ← A -1 a b a 6: p t,a ← θ a x t + c t x t A -1 a x t 7:
end for 8:

Choose arm a t = arg max a∈At p t,a

9:

if the background revealed the feedbacks then 10:

Observe feedback r at,t 11:

A at ← A at + x t x t 12:
b at ← b at + r at,t x t 13:

elif the background revealed NO feedbacks then end if 21: end for cus on the completely online setting, we chose three popular clustering algorithms as the self-supervision modules: GMM, Kmeans and K-nearest neighbors.

Complete Engine for Online Speaker Diarization

To adapt our BerlinUCB algorithm to the specific application of speaker recognition, we first define our actions. There are three major classes of actions: an arm to denote that a new speaker is detected, an arm "No Speaker" to denote that no one is speaking, and N different arms "User n" to denote that user n is speaking. Table 1 presents the reward assignment given four types of feedbacks. Note that we assume that when the agent correctly identifies the speaker (or no speaker), the user (as the feedback dispenser) should send no feedbacks to the system by doing nothing. In another word, in an ideal scenario when the agent does a perfect job by correctly identifying the speaker all the time, we are not necessary to be around to correct it anymore (i.e. truly feedback free). As we pointed out earlier, this could be a challenge earlier on, because other than implicitly approving the agent's choice, receiving no feedbacks could also mean the feedbacks are not revealed properly (e.g. the human oracle took a break). Furthermore, we note that when "No Speaker" and "User n" arms are correctly identified, there is no feedback from us the human oracle (meaning that these arms would never have learned from a single positive reward if we don't use the "None" feedback iterations at all!). The semi-supervision by self-supervision step is exactly tailored for a scenario like this, where the lack of revealed positive reward for "No Speaker" and "User n" arms is compensated by the additional training of the reward mapping b at if context x t is assigned to the right arm.

To tackle the cold start problem, the agent grows it arms in the following fashion: the agent starts with two arms, "No Speaker" and "New"; if it is actually a new speaker speaking, we have the following three conditions: (1) if "New" is chosen, the user approves this arm by giving it a positive reward (i.e. clicking on it) and the agent initializes a new arm called "User N " and update N = N + 1 (where N is the number of registered speakers at the moment); (2) if "No Speaker" is chosen, the user disapproves this arm by giving it a zero reward and clicking on the "New" instead), while the agent initializes a new arm; (3) if one of the user arms is chosen (e.g. "User 5" is chosen while in fact a new person is speaking), the agent copies the wrong user arm's parameters to initialize the new arm, since the voiceprint of the mistaken one might be beneficial to initialize the new user profile. In this way, we can transfer what has been learned for a similar context representations to the new arm. Table 1. Routes given no feedbacks, or a feedback telling the agent that the correct label is a * . (+,+) means that the agent guessed it right by choosing the right arm; (+,-) means that the agent chose this arm incorrectly, since the correct one is another arm; (-,+) means that the agent didn't choose this arm, while it turned out to be the correct one. "-" means NA.

BENCHMARK DESCRIPTION: MINIVOX

MiniVox is an automatic framework to transform any speakerlabelled dataset into continuous speech datastream with episodically revealed label feedbacks. Since our online learning problem setting assumes learning the voiceprints without any previous training data at all, MiniVox's flexibility in length and configuration is especially important. As outlined in Figure 2, MiniVox has a straightforward data stream generation pipeline: given a pool of single-speaker-annotated utterances, randomly concatenate multiple pieces with a chosen number of speakers and a desired length. The reward stream is then sparsified with a parameter p as the percentage of time a feedback is revealed.

There are two scenarios that we can evaluate in MiniVox: if we assume there is an oracle, the online learning model is given the fixed number of the speakers in the stream; if we assume there is no oracle, the online learning model will start from zero speaker and then gradually discover and register new speakers for future identification and diarization.

EMPIRICAL EVALUATION

Experimental Setup and Metrics

We applied MiniVox on VoxCeleb [START_REF] Nagrani | Voxceleb: a large-scale speaker identification dataset[END_REF] to generate three data streams with 5, 10 and 20 speakers to simulate real-world conversations. We extracted two types of features (more details in section 5.2) and evaluated it in two scenarios (with or without oracle). The reward streams are sparsified given a revealing probability of 0.5, 0.1, 0.01 and 0.001. In summary, we evaluated our models in a combinatorial total of 3 speaker numbers × 4 reward revealing probabilities × 2 feature types × 2 test scenarios = 48 online learning environments. The online learning timescale range from ∼12000 to ∼60000 timeframes. For notation of a specific MiniVox, in this paper we would denote "MiniVox C5-MFCC-60k" as a MiniVox environment with 5 speakers ranging 60k time frames using MFCC as features.

To evaluate the performance, we reported Diarization Error Rates (DER) in the above MiniVox environments. In addition, as a common metric in online learning literature, we also recorded the cumulative reward: at each frame, if the agent correctly predicts a given speaker, the reward is counted as +1 (regardless of whether the agent observes it or not).

We compared 9 agents: LinUCB is the contextual bandit with extendable arms proposed in section 3.1. BerlinUCB is the standard contextual bandit model designed for sparse feedbacks without the self-supervision modules. We have four baseline models: Kmeans, KNN (with K=5), GMM and a random agent2 . To test the effect of self-supervision, we introduced three clustering modules in BerlinUCB (alg 2, Step 15) which we denoted: B-Kmeans, B-KNN, and B-GMM.

Feature Embeddings: MFCC and CNN

We utilized two feature embeddings for our evaluation: MFCC [START_REF] Md Rashidul Hasan | Speaker identification using mel frequency cepstral coefficients[END_REF] and a Convolutional Neural Network (CNN) embedding. We utilized the same CNN architecture as the VGG-M [START_REF] Chatfield | Return of the devil in the details: Delving deep into convolutional nets[END_REF] used in VolCeleb evaluation [START_REF] Nagrani | Voxceleb: a large-scale speaker identification dataset[END_REF]. It takes the spectrogram of an utterance as the input, and generate a feature vector of 1024 in layer fc8 (for more details of this CNN, please refer to table 4 in [START_REF] Nagrani | Voxceleb: a large-scale speaker identification dataset[END_REF]).

Why don't we use more complicated embeddings? Although more complicated embedding extraction modules such as i-vectors [START_REF] Stephen H Shum | Unsupervised methods for speaker diarization: An integrated and iterative approach[END_REF] or d-vectors [START_REF] Wang | Speaker diarization with lstm[END_REF] can improve diarization, they require extensive pretraining on big datasets, which is contradictory to our problem setting and beyond our scope.

Why do we still include this CNN? The CNN model was trained for speaker verification task in VoxCeleb and we are curious about the relationship between a learned representation and our online learning agents. Despite this note, we are most interested in the performance given MFCC features, because we aim to push the system to the limit of not having pretraining of any type before deployment -a fully online learning system.

Results

Given MFCC features without pretraining, our online learning agent demonstrated a relatively robust performance. As shown in Figure 3(a,b,c,d), in many conditions, the proposed contextual bandits significantly outperformed baseline models when the reward revealing probability is very low (p=0.01 or 0.1).

Learning without Oracle. Table 2 reports the DER in MiniVox without Oracle. In MFCC environments, we observed that in high-difficulty scenarios (such as C20), the proposed BerlinUCB variants outperformed all the baselines even when the reward revealing probability was as low as 0.01. In low-difficulty scenarios, traditional clustering methods like KNN performed the best, while this benefit was inherited by B-KNN and B-Kmeans when feedbacks were sparse (p=0.01). In the CNN environments, we discovered that Kmeans performed the best among all agents. This is expected because the CNN model was trained with the constrastive loss for a high verification accuracy [START_REF] Nagrani | Voxceleb: a large-scale speaker identification dataset[END_REF]. While the clustering modules merely classify the CNN feature by their proximity, our online learning model need to learn about their reward mapping from scratch, while maintaining a good balance between exploitation and exploration.

Learning with Oracle. Given the number of speakers, traditional clustering agents performed better (Table 3). However, the behaviors vary: we observed that GMM performed the poorest in the oracle-free environments, but performed the best in the environments with oracle; we also noted that despite the best model in many oracle-free environments, Kmeans performed poorly in the MFCC environments with oracle. Another winning algorithm, KNN, requires the model to store all historical data points and search through the entire memory, which can be computationally inhibitory in realworld applications. Our online learning models, on the other hand, maintains a relatively robust performance by keeping among the top 3 algorithms in most environments with and Is self-supervision useful? To our surprise, our benchmark results suggested that the proposed self-supervision modules didn't improve upon both the baseline models and our proposed contextual bandit models. Only in specific conditions (e.g. MiniVox C5-MFCC-60k p=0.01), the selfsupervised contextual bandits outperformed both the standard BerlinUCB and all the baseline. Further investigation into the reward curve revealed more complicated interactions between the self-supervision modules with the online learning modules (the contextual bandit): as shown in Figure 3(f,g,h), B-GMM and B-KNN maintained build upon the effective reward mapping from their BerlinUCB backbone, and benefited from the unlabelled data points to yield a fairly good performance over baselines.

App and Future work. We provided a workable webbased system [START_REF] Lin | VoiceID on the fly: A speaker recognition system that learns from scratch[END_REF] (linked before) to engage the community into this intriguing online speaker diarization task. Ongoing work includes extending this online learning framework in extraction and clustering modules [START_REF] Zhang | Fully supervised speaker diarization[END_REF] with routing [START_REF] Lin | Contextual bandit with adaptive feature extraction[END_REF].

Fig. 1 .

 1 Fig. 1. Arm expansion process of the bandit agents.

Algorithm 1

 1 Online Learning with Episodic Rewards 1: for t = 1,2,3,• • • , T do 2:

Fig. 2 .

 2 Fig. 2. (A) The flowchart of the Online Learning problem and (B) the MiniVox Benchmark.

Feedback

Fig. 3 .

 3 Fig. 3. Example reward curves. Positive: (a) C10-MFCC, p=0.01; (b) C20-MFCC, p=0.01; (c) C5-MFCC, p=0.01; (d) C5-MFCC, p=0.5; (e) C20-MFCC, p=0.01, oracle. Negative: (f) C10-MFCC, p=0.01, oracle; (g) C10-MFCC, p=0.1, oracle; (h) C5-CNN, p=0.5.

Table 2 .

 2 Diarization Error Rate (%) in MiniVox w/o Oracle MiniVox C5-MFCC-60k MiniVox C5-CNN-12k p = 0.5 p = 0.1 p = 0.01 p = 0.5 p = 0.1 p = 0

Table 3 .

 3 Diarization Error Rate (%) in MiniVox with Oracle = 0.5 p = 0.1 p = 0.01 p = 0.5 p = 0.1 p = 0.01

		MiniVox C5-MFCC-60k	MiniVox C5-CNN-12k
		p = 0.5 p = 0.1 p = 0.01 p = 0.5 p = 0.1 p = 0.01
	BerlinUCB	74.89	77.24	86.93	17.27	22.19	66.02
	LinUCB	72.83	78.12	76.80	17.73	32.73	58.98
	B-Kmeans	75.33	78.27	83.11	20.55	40.70	58.98
	B-KNN	77.39	77.97	83.99	20.47	41.33	58.98
	B-GMM	74.16	76.21	77.24	52.58	81.02	58.98
	Kmeans	78.41	82.82	83.11	4.06	7.42	39.53
	KNN	70.63	73.27	80.47	6.64	13.75	53.52
	GMM	70.34	72.54	74.74	54.38	81.02	58.98
	random	79.59	80.76	85.9	79.92	80.39	85.55
		MiniVox C10-MFCC-60k	MiniVox C10-CNN-12k
		p = 0.5 p = 0.1 p = 0.01 p = 0.5 p = 0.1 p = 0.01
	BerlinUCB	88.31	90.21	95.89	45.18	65.27	79.38
	LinUCB	84.99	91.63	97.00	50.00	72.14	65.18
	B-Kmeans	87.84	91.47	91.94	50.27	72.50	72.32
	B-KNN	86.73	85.78	92.58	49.64	72.14	77.77
	B-GMM	88.94	84.52	92.58	76.52	71.88	69.46
	Kmeans	89.42	89.57	98.74	11.16	20.27	49.49
	KNN	80.25	84.68	97.79	9.55	31.25	70.45
	GMM	90.36	79.62	91.63	76.52	78.30	77.77
	random	87.99	92.26	97.16	90.00	90.89	92.32
		MiniVox C20-MFCC-60k	MiniVox C20-CNN-12k
	p BerlinUCB 92.31	94.55	96.31	58.75	68.98	88.83
	LinUCB	89.10	93.43	95.67	53.44	70.47	83.44
	B-Kmeans	92.95	95.67	96.96	55.16	70.86	94.06
	B-KNN	91.83	92.47	97.44	54.30	89.84	96.72
	B-GMM	95.19	91.99	97.44	86.48	77.97	96.64
	Kmeans	91.67	94.23	98.08	7.66	13.75	55.63
	KNN	86.86	89.26	98.08	9.690	32.73	75.08
	GMM	98.08	94.87	98.88	93.52	95.08	97.11
	random	94.71	94.71	98.88	95.55	95.86	97.03
	without oracle.

The web-based application of a real-time system can be accessed at https://www.baihan.nyc/viz/VoiceID/. The code for benchmark evaluation can be accessed at https://github.com/doerlbh/MiniVox

In the oracle-free case, the random agent randomly selects from the "new" arm and the registered user arms, suggesting a possibility of going to infinitely (and incorrectly) many arms of registered users.