The Soledge3X-EIRENE tool: state of the art Capabilities:

Soledge3X is an edge multi fluid code, implementing the mass, momentum and energy balance for an arbitrary number of plasma species.

[N. Experiencing limitations in EIRENE coupling for 3D turbulent cases

Impossibility in running production cases with turbulence and EIRENE.

There are two main options taken into account:

Run with the newly available OpenMP version of EIRENE.

2. Change the way the couple is made, considering a coarser mesh for EIRENE respect with the one of Soledge3X.

Too large memory consumption for EIRENE parallelized in MPI mode

𝑆 𝑛 [ 𝑝𝑎𝑟𝑡 𝑠 𝑚 3 ]
The coupling works in 3D with reduced mesh (i.e. low resolutions) for any kind of geometry.

In the meantime, upgrading the fluid neutrals model in Soledge3X to stick with EIRENE.

BTW, turbulence transport required mesh at Larmor radius resolutions.

High resolution + 3D = much more points in the grid! slide 10/33 19/09/2023 V. Quadri, PET-19, 18-21 Sept. 2023, Hefei The exigence of switching to a fluid description: the embedded fluid neutrals model in SOLEDGE3X.

The goal: providing a qualitative interpretation of the physics as close as possible to EIRENE, by having a realistic localization of the recycling source and consistent momentum and energy sources. Horsten et al, 2017, Nuclear Fusion 57, 116043) kinetic description  fluid model = missing information The limit:

𝝏 𝒕 𝒏 𝒏 + 𝜵 • 𝜞 𝒏 = 𝑺
How do embedded neutrals impact the convergence and the solution in Soledge3x? The circular cases: a theoretical exercise for understanding the neutrals impact on turbulence. The gas puff does impact the toroidal symmetry

max < 𝑁 > 𝜑 -min < 𝑁 > 𝜑 < 𝑁 > 𝜑
The effect of the perturbation is strongly localized.

The entity of the perturbation can be huge: up to 40% for some field (i.e. mean density)

The gas puff introduces a non-axis-symmetry perturbation in the system along the toroidal direction.

Toroidal non uniformity of the mean plasma density

Consequences on

• the physics (locally and not)

• the way in which simulations are analyzed. < 𝑻 𝒆 > 𝒕,𝝓

• The non-axisymmetric perturbation driven by the puff has an impact on the mean plasma quantities.

• The impact of the puff is flux tube localized, the entity is more than 40% for some fields.

• Not all the fields are impacted in the same way. Local & non local effects of the gas puff

• The non-axisymmetric perturbation driven by the puff has an impact on turbulent transport.

• The impact of the puff is flux tube localized, the change in local fluctuation rates is up to 20%.

• This may be worrying for the gas puff imagining through diagnostics. Yes, the following statistical analysis will be affected by an error of 4%.

Can we drop the information along the toroidal direction?

The system is strongly axis-asymmetric due to the puff. Statistical analysis on turbulence are usually made by toroidally averaging. • Sensitive differences w/ ancestor TOKAM3X.

• Global change in the level of the fluctuations for density, up to 20% at the midplane less at the target.

• The general trend is conserved.

• The skewness does not change so much.

• More negative events at the outer target where the recycling occurs. Matching experiments and more Goal: Performing the same studies and answering the same question as the theoretical exercise. • Specific turbulent activity in the divertor region under detachment conditions: may affect gas puff imagining?

• Turbulent activity in the main SOL

Outcomes and conclusions

• Variations in poloidal equilibrium • Reorganization of the transverse transport.

Changes even more marked when dealing with higher density regimes.

The gas puff introduces a breaking in the toroidal asymmetry.

The basic features of turbulence are conserved whether neutrals or not.

• Consequences for divertor physics, due to the flux tube propagation.

• Consequences for gas puff diagnostics. Turbulence and neutrals: key players to edge plasma physics issues

Understanding the interactions among turbulence and neutrals is crucial for nuclear fusion and the challenge of handling the heat fluxes deposition at the target plates.

• Transfer the energy, feeding complex mechanism such as recycling of neutrals, leading to different plasma density states called density or diverted regimes.

• Broaden the width of the Scrape Off Layer (SOL), 𝝀 𝒒 , which is (partly?) determined by turbulence, since it influences the level of interactions between plasma and the wall.

Way out: Spreading the power radially outwards.

In this way, an investigation on the consequences of wall erosion due to particle flux and thermal stress is mandatory to model properly the SOL, taking into account both the two physics. The needing of a predictive model

Where were the interpretative models failing until now?

There is NO multi-machine scaling law for high-recycling and detached plasmas in terms of 𝜆 𝑞 .

Being interpretative means failing to capture some phenomena (like the formation of density shoulders) where the two physics, turbulence and recycling, are interconnected. The exigence of switching to a fluid description: the hierarchy of models in SOLEDGE3X.

Visible light image of filaments in

Moving from a kinetic description to a fluid model entails missing information (i.e. molecular and atomic dynamics). The goal is indeed providing a qualitative interpretation of the physics as close as possible to EIRENE, by having a realistic localization of the recycling source and consistent momentum and energy sources.

A hierarchy of fluid neutrals model is currently available in Soledge3X: the common starting point is the mass balance equation

𝝏 𝒕 𝒏 𝒏 + 𝜵 • 𝜞 𝒏 = 𝑺 𝒏 𝒏 where 𝑆 𝑛 𝑛 = 𝑛 𝑖 𝑛 𝑒 𝐾 𝑟 -𝑛 𝑛 𝑛 𝑒 𝐾 𝑖
What makes the difference is the way in which 𝜞 𝒏 and the diffusion coefficients (constant or based on a polynomial fit including cx and ionization) are computed. The convergence issue for turbulent -neutral cases

Different time scales interplay among them

Case without neutrals, source from the core Case with neutrals source 1. Fast scale: typical of turbulence, which reacts very quickly to a new equilibrium after the breaking (i.e. injection of neutrals) of the previous 2. Slow scale: it concerns the global particle balance, in terms of equilibrium between the level of gas puff and the plasma density.

The reaching of particle convergence for these kind of simulations requires ages of computation: the response of the balances is very slow and the transient extremely costly in terms of computational resources.

The turbulence lays on a total different time scale: it reaches steady state for a given plasma density very quick.

This absolutely does not prevent us to look at the impact of neutrals on turbulence through a snapshot of it at a given plasma density.

Turbulence does reach an equilibrium.

Puffing neutrals, turning off the source from the core Turbulence does reach a new equilibrium, not particles.

The reason depends on the (1-R) factor appearing in the total balance equation Reaching the steady state in theoretical exercise vs experimental matching

Respect to the given throughput, the circular case results to be in steady state. This has been possible by tuning the value of the puff: in order to do something predictive, the control parameter is no more the plasma, but the puff itself.

The simulation does reach the steady state with respect a given puff.

In order to be experimental relevant, both plasma conditions and turbulence have to match the throughputs:

Conclusions and perspectives

Turbulence has found to globally conserve the same basic properties whether neutrals recycling is accounted for or not:

• Is driven by intermittent small-scale and field aligned fluctuations propagating outwards in the parallel direction by the ExB advection in the form of filaments.

• Shows a marked ballooned character on the Low Field Side (LFS).

The presence of a localized neutral source strongely impacts the poloidal equilibrium of the plasma and the way the transverse transport is going to organize itself in the edge.

This impact is likely to become even more marked when dealing with higher density regimes in which the localization of the recycling source in the radial and poloidal directions is more pronounced.

The gas puff provides a breaking in the toroidal asymmetry: it may have consequences for divertor physics, due to the flux tube propagation.

Getting the particle balance from the 2D mean fields with drifts might help to have a proper plasma background when transiting in 3D

  The coupling has successfully made for the 2D mean field transport and running routinely for ITER cases.

  Particle balance and particle conservation Plasma Neutrals Particle balance = the amount of particle in the simulation box is not changing in time anymore. , PET-19, 18-21 Sept. 2023, Hefei Particle convergence: reaching the steady state To reach steady state: < 𝜕  𝑁+𝑁 𝑛 𝜕𝑡 > 𝑡𝑡𝑠 → 0, and so < 𝜞 𝑰𝑵 + 𝜞 𝒑𝒖𝒇𝒇 -𝟏 -𝑹 𝜞 𝒐𝒖𝒕 > 𝒕𝒕𝒔 → 𝟎

  , PET-19, 18-21 Sept. 2023, Hefei Local & non local effects of the gas puff Poloidal direction 𝜃 Toroidal direction 𝜙 Toroidal direction 𝜙 Toroidal direction 𝜙 Toroidal direction 𝜙 < 𝑵 > 𝒕,𝝓 Flux surface in the NEAR SOL Flux surface in the NEAR SOL [𝑒𝑉]

  , PET-19, 18-21 Sept. 2023, Hefei Toroidal direction 𝜙 Toroidal direction 𝜙 Toroidal direction 𝜙 Flux surface in the NEAR SOL Flux surface in the NEAR SOL 1e18 ෪ 𝑻 𝒆 𝒕,𝝓

••

  , PET-19, 18-21 Sept. 2023, Hefei Self organization of turbulence with respect neutrals recycling slide 21/33 19/09/2023 V. Quadri, PET-19, 18-21 Sept. 2023, HefeiIdentifying the recycling regime: poloidal profiles < 𝑁 𝑡,denser and cooler but probably still in a low recycling regime.Looking at specific flux surfaces the density becomes larger than it is in absence of neutrals: close to high recycling?Poloidal direction 𝜃 1𝑒18 1𝑒18 slide 22/33 19/09/2023 V. Quadri, PET-19, 18-21 Sept. 2023, HefeiImpact of neutrals on plasma overall equilibriumChanging in the radial profiles for density due to a flux re-organization after the puffing of neutrals. Poloidal asymmetries do appear, much marked with FN that without.• Strong density accumulation close to the limiter. • Flattening of the profiles in the core, and in the SOL. ,PET-19, 18-21 Sept. 2023, Hefei Impact of neutrals on plasma overall equilibrium Changing in the radial profiles for e-temperature due to a flux re-organization after the puffing of neutrals. Steepening of the profiles.• Poloidal asymmetries not so strong as for the density

  density regimes • Benchmarking fluid neutrals model with respect EIRENE slide 28/33 19/09/2023 V. Quadri, PET-19, 18-21 Sept. 2023, Hefei Modeling TCV experiments with Soledge3x TCVX23 = Extension of TCVX21 experiment up to density regimes. Long leg stable configuration at low field (RF #76142) Initial density 𝑛 𝑠𝑒𝑝 ≈ 5𝐸18, detachment at 𝑛 𝑠𝑒𝑝 ≈ 1𝐸19, P sep = 160 kW [D. Sales de Oliveira et SPC team al] [P. Tamain] case (𝑛 𝑠𝑒𝑝 = 1𝐸19): quality of FN compared with EIRENE. of the ionization source • FN is able to emulate a sort of detachment, at the condition of the experiments (TCVX23 does detach at 𝑛 𝑠𝑒𝑝 = 1𝐸19) 2D might help to have a proper plasma background in 3D! 𝑛 From 2D to 3D: the transition to turbulence.

•

  /SPPF/GTSN (Office : 513/135) Saint Paul Lez Durance, 13108, France slide 35/33 19/09/2023 V. Quadri, PET-19, 18-21 Sept. 2023, Hefei

  Ohmic L-mode on MAST. [B. D. Dudson]

  Next step: full steady state for turbulence and particle balance slide 44/33 V. Quadri, TSVV3 meeting 2D mean field case (𝑛 𝑠𝑒𝑝 = 1𝐸19): quality of FN compared with EIRENE. able to emulate a sort of detachment, at the condition of the experiments (TCVX23 does detach at 𝑛 𝑠𝑒𝑝 = 1𝐸19), with a plasma density and temperature higher with respect EIR EIRENE FN • the neutrals density distribution improved but still not realistic(𝑛 𝑛 is not the diffusive quantity in the model -𝑃 𝑛 does).• Realistic localization of the ionization source (despite is a bit more elongated all along the legs) 2D might help to have a proper plasma background in 3D!𝑛From 2D to 3D: the transition to turbulence. We are able to transit in 3D with turbulence very quick but still under investigation! ~20𝐾 time steps 𝑛 slide 46/33 19/09/2023 V. Quadri, PET-19, 18-21 Sept. 2023, Hefei

  𝒏 𝒏 where 𝑆 𝑛 𝑛 = 𝑛 𝑖 𝑛 𝑒 𝐾 𝑟 -𝑛 𝑛 𝑛 𝑒 𝐾 𝑖

	Convection -pressure diffusion
	Γ 𝑛 = 𝑛 𝑛,𝑒𝑞 𝑢 𝑖 -𝐷 𝑝 𝑛 𝛻𝑝 𝑛
	𝐷 𝑛 𝑝 =	1 𝑚 𝑛 𝑖 𝐾 𝑐𝑥,𝑚 + 𝑛 𝑒 𝐾 𝑖

hp: referred to charge exchange-dominated region (𝑇 𝑛 = 𝑇 𝑖 ) (ref.

IN OUT S OUT without neutrals with neutrals With neutrals introduction, the nature of the particle balance changes completely,

		The convergence issue for turbulent -neutral cases
		INFLUX Different time scales interplay among them; global convergence takes long [H. Bufferand, PET-19, 18/09/23]
		OUTFLUX
		SOURCE
		dndt
	Particle flux (part/s)	Convergence ratio: CR = Does it really matter being in steady state for balance investigating the properties of turbulence w/ recycling?	<	𝜕𝑡 Γ 𝐼𝑁 + Γ 𝑝𝑢𝑓𝑓 > 𝑡𝑡𝑠 𝜕  𝑁 + 𝑁 𝑛	~1 -2%
		Time (ms)
		V. Quadri, PET-19, 18-21 Sept. 2023, Hefei		19/09/2023	slide 13/33

Plot CR as function of time (negative in this case, still not steady state!)

It depends on how the simulations are driven 𝜏 𝑒𝑞 ~100 𝑚𝑠 A theoretical exercise in circular geometry Divertor simulations matching the experiments YES NO

	• Fixed plasma conditions	• Plasma conditions corresponding to throughputs
	1

. Does the gas puff break the axis-symmetry? 2. How do the fluxes change at the poloidal direction? 3. How do the turbulence properties change?

	V. Quadri, PET-19, 18-21 Sept. 2023, Hefei	19/09/2023	slide 14/33

of the transport has started.. Source and

  Out are the larger terms: even small errors on them, will dramatically impact the balance

	The convergence issue for turbulent -neutral cases Backup slides Importance of particle conservation
	Different time scales interplay among them	1. Fast scale: typical of turbulence, which reacts very
			quickly to a new equilibrium (i.e. injection of neutrals).
			2. Slow scale: it concerns the transport and the plasma 𝑛 𝑖 [𝑚 -3 ]
			particle balance.
	𝜏 𝑒𝑞 =	3. Global scale (𝜏 𝑒𝑞 ): for the whole system, with a very slow response and a long transient, costly in terms of Particles and energy sources computational resources core region not Does it really matter for this cases being in steady state? 𝜏 ~100 𝑚𝑠 1 -𝑅 1 come out from the core: Plasma Z [m] , due to numerical scheme suited.
	From theoretical to experimental relevant the point of view changes in terms of how the simulations are driven simulated.
	With neutrals introduction, the nature of the particle 1. A theoretical exercise in circular geometry, where plasma conditions are fixed and we study the balance changes completely, since the dominant behavior of turbulence in interactions with neutrals. fueling mechanism becomes different.
			R [m]
	2. Divertor simulations matching the experiments: where the aim is getting both plasma conditions and well turbulence corresponding to throughputs. Modify video by adding neutrals as A redistribution
	V. Quadri, PET-19, 18-21 Sept. 2023, Hefei V. Quadri, PET-19, 18-21 Sept. 2023, Hefei V. Quadri, PET-19, 18-21 Sept. 2023, Hefei	19/09/2023 19/09/2023 19/09/2023	slide 39/33 slide 41/33 slide 42/33

slide 40/33 19/09/2023 V. Quadri, PET-19, 18-21 Sept. 2023, Hefei Basic turbulent properties don't change: you still have a blobby transport, filamentary transport, slide 43/33 19/09/2023 V. Quadri, PET-19, 18-21 Sept. 2023, Hefei

Does the gas puff break the axis-symmetry?

How do the fluxes change at the poloidal direction?

How do the turbulence properties change?


Christine Esclapez

Joséphine Simonnot

Caroline Boë

The Corsica Sud-Archives [CO S-A] project or How to practice and share intangible heritage in a Mediterranean island territory

Capture d'écran du player son sur smartphone.

Le Chant du Geste

  Opening of the balade: 23/02/2023 « I think that for tourists, this will give them an even more cultural image and a deeper connection with the village, rather than a commercial one. In a very short time, it can give a social and cultural dimension, even without explanations. » Comments collected after the opening, 23/02/2023. Public écoutant lors de l'inauguration , 23/02/2023. Photo : C. Boë © CC BY.

https://cosa.hypotheses.org

Axis 1

Archive as a material

Creative work based on the reuse of musical archives from the Repertorium: rewritings, detour, reinterpretations.

Innovative, interdisciplinary and cross-sectoral methodological protocol

Axis 2

Archive as document

Archival appraisal of the Repertorium in order to establish a mediumand long-term heritage conservation and enhancement policy.

Axis 3

Archive as common territory

Local action with village users. Immersive listening point proposed in front of each craftsman's shop.

Le Chant du Geste

INTANGIBLE HERITAGE Axis 1

Archive as a material

Research-creation Analysis of musical practices Systematic musicology

The life of the archive : between recherche-creation and action

Axis 2

Archive as document

Historic research

Oral history Archival Sciences History Historic musicology

Axis 3

Archive as common territory

Research-action Sociology Anthropological practices

Reproductiblity Partnerships

Thank you ! @ => christine.esclapez@prism.cnrs.fr boe@prism.cnrs.fr josephine.simonnot@cnrs.fr


Stefano Fortunati

Senior, IEEE Esa Ollila

The equality between the Stochastic CRB and its semiparametric version for elliptical data

Keywords: DOA estimation, Stochastic Cramér-Rao Bound, semiparametric models

In this letter, the proof of the following quite surprising result is provided: if the data are elliptically distributed, the "parametric" Stochastic Cramér-Rao Bound (derived by assuming the a priori knowledge of the density generator) and its semiparametric counterpart (obtained by considering the density generator as an additional nuisance parameter) are equal. In other words, this means that, in the presence of finite-dimensional nuisance parameters (the source correlation matrix and noise power in the decomposition of the array covariance/scatter matrix), not knowing the data density generator does not lead to an additional loss of efficiency.

I. INTRODUCTION

The celebrated Stochastic Cramér-Rao Bound (SCRB) is a well-known tool in array processing since it represent a lower bound for the Mean Square Error (MSE) of any Direction of Arrival (DOA) estimation algorithm. While the SCRB is generally exploited in Gaussian-based scenarios, recent advances have allowed for a relaxation of this rather stringent assumption in favor of the more general hypothesis of elliptically distributed data. This letter shows a surprising equality between the parametric version of the elliptical SCRB and its semiparametric counterpart [START_REF] Fortunati | Semiparametric CRB and Slepian-Bangs formulas for complex elliptically symmetric distributions[END_REF].

Notation: In this letter, italics indicates scalar quantities (a), lower case and upper case boldface indicate column vectors (a) and matrices (A), respectively. The superscripts T , * and H indicate the transpose, the complex conjugate and the Hermitian operators, respectively. Each entry of a vector a and of a matrix A is indicated as a i ≜ [A] i and a i,j ≜ [A] i,j , respectively. Let A(θ) be a matrix (or possibly a vector or even a scalar) function of the real vector θ ∈ Θ ⊆ R d , then A 0 ≜ A(θ 0 ) while A 0 i ≜ ∂A(θ) ∂θi | θ=θ0 , where θ 0 is a particular (or true) value of θ ∈ Θ. For random variables or vectors, = d stands for "has the same distribution as".

II. DOA ESTIMATION IN ELLIPTICAL DATA

Let x ∈ C m be the observation vector collected by an array of m sensors and let ν 0 = (ν 0,1 , . . . , ν 0,q ) T ∈ Γ ⊆ R q be the vector of the location parameters (in some reference systems) of q narrowband sources. As firstly proposed in [START_REF] Ollila | Influence function and asymptotic efficiency of scatter matrix based array processors: Case MVDR beamformer[END_REF], we suppose that the observation vector is CES-distributed, Stefano Fortunati is with Université Paris-Saclay, CNRS, CentraleSupélec, Laboratoire des signaux et systèmes, 91190, Gif-sur-Yvette & DR2I-IPSA, 94200, Ivry sur Seine (e-mail: stefano.fortunati@centralesupelec.fr).

Esa Ollila is with the Department of Information and Communications Engineering, Aalto University, FI-00076 Aalto, Finland (e-mail: esa.ollila@aalto.fi).

i.e. x ∼ CES m (x; 0, Σ 0 , g 0 ), and its scatter matrix has the following peculiar structure:

Σ 0 ≡ Σ(ν 0 , Γ 0 , σ 2 0 ) = A 0 Γ 0 A H 0 + σ 2 0 I, (1)
• A 0 ≜ A(ν 0 ) = [a(ν 0,1 ) • • • a(ν 0,q )] ∈ C m×q such that (s.t.
) a(ν 0,k ) is the steering vector for the k-th source, • Γ 0 ∈ M q ⊂ C q×q is the source covariance matrix and M q stands for the set of the Hermitian, positive definite matrices of dimension q × q, • σ 2 0 ∈ R + is the noise power, • g 0 ∈ G is the density generator. To avoid the scale ambiguity between the density generator and the data covariance matrix, we define the set G as [START_REF] Fortunati | Semiparametric CRB and Slepian-Bangs formulas for complex elliptically symmetric distributions[END_REF]:

G ≜ g : R + 0 → R + 0 2 -1 s m +∞ 0 q m g(q)dq = m , (2)
where s m ≜ 2π m Γ(m) -1 and Γ(•) is the Gamma function. Note that, if x ∼ CES m (x; 0, Σ 0 , g 0 ), then it admits the following stochastic representation [START_REF] Ollila | Complex elliptically symmetric distributions: Survey, new results and applications[END_REF]:

x = d √ QΣ 1/2 0 v, (3)
where v ∼ U (CS m ) is uniformly distributed on the unit m-1 sphere and Q ∼ p Q (q) = 2 -1 s m q m-1 g 0 (q) is called the 2ndorder modular variate.

In the rest of this letter, instead of Γ 0 , we will make use of

ζ 0 ∈ Ψ ⊆ R q 2
defined as:

ζ 0 ≜ diag(Γ 0 ) T , vec l (Re(Γ 0 )) T , vec l (Im(Γ 0 )) T T , (4)
where the operator vec l (•) selects all the entries strictly below the main diagonal of Γ 0 taken in the column-wise order [4, Sec. 2.4] while diag(Γ 0 ) is a column vector collecting the diagonal elements of Γ 0 . Finally, let us define ξ 0 as:

ξ 0 ≜ (ζ T 0 , σ 2 0 ) T ∈ Ψ × R + ⊆ R q 2 +1 . (5)
The celebrated "Stochastic" Cramér-Rao Bound (SCRB) has been introduced as the CRB on the estimation of the sources location vector ν 0 in the presence of the nuisance vector ξ 0 (see e.g. [START_REF] Stoica | The stochastic CRB for array processing: a textbook derivation[END_REF]). According to the degree of a priori knowledge on g 0 ∈ G, we may have the following three results: C1 g 0 is assumed to be equal to the density generator of the Gaussian distribution. This leads to the Gaussian SCRB GSCRB(ν 0 |ξ 0 , σ 2 0 ) derived in, among others, [START_REF] Stoica | The stochastic CRB for array processing: a textbook derivation[END_REF]. C2 g 0 is assumed to be equal to a specific density generator.

This leads to the Elliptical SCRB ESCRB(ν 0 |ξ 0 , σ 2 0 ) that can be obtained by inverting the Fisher Information Matrix introduced in [START_REF] Besson | On the Fisher Information Matrix for multivariate elliptically contoured distributions[END_REF]. C3 g 0 is assumed to belong to the CES class but, unlike in C1 and C2, it is left fully unspecified, i.e. it is not assumed to be a priori known. This leads to the Semiparametric SCRB SSCRB(ν 0 |ξ 0 , σ 2 0 , g 0 ) derived in [START_REF] Fortunati | Semiparametric CRB and Slepian-Bangs formulas for complex elliptically symmetric distributions[END_REF]. In this letter, we show that the Elliptical SCRB is equal to the Semiparametric CRB for each possible g 0 ∈ G, i.e.

ESCRB(ν

0 |ξ 0 , σ 2 0 ) = SSCRB(ν 0 |ξ 0 , σ 2 0 , g 0 ), ∀g 0 ∈ G. (6)
In word, this means that if we do not know ξ 0 , not knowing the density generator g 0 does not lead to an additional loss in estimation efficiency.

To provide a proof of this (quite surprising) result, let us start by framing this estimation problem in the geometric, "Hilbert-spaces based" context of semiparametric models.

III. MODELS, TANGENT SPACES AND SCORE VECTORS

Let us consider the general semiparametric model:

P = {p X (x|ν, ξ, g) : ν ∈ Γ, ξ ∈ Ψ, g ∈ G} , (7)
where Γ ⊆ R q is the set of the (finite-dimensional) parameter vectors ν of interests, Ψ ⊆ R r is the set of (finite-dimensional) nuisance parameter vectors ξ and G is the space of the (infinite-dimensional) nuisance functions g. Note that, in the scenario described in Sect. II, r = q 2 + 1. We indicate as ν 0 ∈ Γ, ξ 0 ∈ Ψ and g 0 ∈ G the true, but unknown, related quantities and with E 0 {f } ≜ f (x)dP X (x|ν 0 , ξ 0 , g 0 ) the expectation of a measurable function f with respect to (w.r.t.) the true distribution P X (x|ν 0 , ξ 0 , g 0 ). Before moving on, let us introduce an Hilbert space that holds fundamental importance in our subsequent derivations. Specifically, we indicate with (H, ⟨•, •⟩) the (infinite-dimensional) Hilbert space of the measurable functions with i) zero-mean E 0 {h} = 0 and ii) finite variance E 0 {h 2 } = σ 2 h < +∞ ( [7, Sect. 2.2] and [8, Ch. 2]). The related inner product is given by ⟨h 1 , h 2 ⟩ = E 0 {h 1 h * 2 }. By using a self-explanatory notation, we now introduce three parametric submodels of P as:

P 1 = {p X (x|ν, ξ 0 , g 0 ) : ν ∈ Γ} , (8)
P 2 = {p X (x|ν 0 , ξ, g 0 ) : ξ ∈ Ψ} , (9)
P 1,2 = {p X (x|ν, ξ, g 0 ) : ν ∈ Γ, ξ ∈ Ψ} , (10)
along with the non-parametric model

P 3 = {p X (x|ν 0 , ξ 0 , g) : g ∈ G} . ( 11
)
Let us define the score vector s ν0 in P 1 as:

[s ν0 ] i ≜ [s ν0 (x)] i = ∂ ln p X (x|ν, ξ 0 , g 0 )/∂ν i | ν=ν0 , (12)
for i = 1, . . . , q, that represents the score vector of the parameters of interest. Similarly, the score vector s ξ0 of the finite-dimensional nuisance parameters ξ 0 in P 2 is given by:

[s ξ0 ] j ≜ [s ξ0 (x)] j = ∂ ln p X (x|ν 0 , ξ, g 0 )/∂ξ j | ξ=ξ0 , (13)
for j = 1, . . . , r. Under the standard regularity conditions discussed in [9, Sect. 6.2 and 6.3], it is immediate to verify that [s ν0 ] i , [s ξ0 ] j ∈ H, ∀i, j.

We can now introduce the (finite-dimensional) tangent space of the parametric submodels P 2 as the linear span of s ξ0 in H (see e.g. [8, Sect. 2.3] or [7, eq. ( 24)]):

H ⊇ T 2 ≜ Span{[s ξ0 ] 1 , . . . , [s ξ0 ] r }. (14)
Remark: It is worth noticing that in this letter we prefer to work only with the Hilbert space H instead of introducing the related "q-replicating space" H q ≜ H × . . . × H as in [START_REF] Fortunati | Semiparametric estimation in elliptical distributions[END_REF]Sect. 2.2]. The two approaches are fully equivalent. In particular, if we work on (subspaces of) H, instead of H q , all the operators, as expectations and projections, has to be interpreted componentwise (see e.g. [8, Sect. 2.4, Remark 2]). Specifically, for each h = (h 1 , . . . , h q ) T s.t. h i ∈ H and u ∈ U ⊆ H, we have, for i = 1, . . . , q:

[Π(h|U)] i ≜ Π([h] i |U) = Π(h i |U), (15)
[E 0 {hu * }] i ≜ E 0 {[h] i u * } = ⟨h i , u⟩ , (16)
Finally, the (infinite-dimensional) nuisance tangent space T 3 ⊆ H of the non-parametric model P 3 is defined as in [7, eq. ( 44)]. Note that, by construction, T 2 and T 3 are finiteand infinite-dimensional closed subspaces of H.

According to the previous definition, let us define the efficient score vector tν0 for the vector ν 0 of the parameter of interest in the parametric submodel P 1,2 in (10) as [7, Sect.

2.3, Def. 4]:

tν0

≜ s ν0 -Π(s ν0 |T 2 ), (17)
where the projection operator has to be interpreted componentwise as defined in (15). Since T 2 is a finite-dimensional subspace of H, the projection operator Π(•|T 2 ) can be derived in closed form as [7, eq. ( 16)], [8, Sect. 2.4, Example 1]:

[Π(h|T 2 )] i = E 0 {h i s H ξ0 }I -1 ξ0 s ξ0 , h i ∈ H ∀i, (18)
where:

I ξ0 ≜ E 0 {s ξ0 s H ξ0 }, (19)
is the FIM for ξ 0 in the parametric submodel P 2 in (9). Then, we have that tν0 can be explicitly expressed as:

tν0 = s ν0 -E 0 {s ν0 s H ξ0 }I -1 ξ0 s ξ0 = s ν0 -I ν0ξ0 I -1 ξ0 s ξ0 , (20) where I ν0ξ0 ≜ E 0 {s ν0 s H ξ0 } (21)
is the matrix of the cross-information terms in the parametric submodel P 1,2 in (10).

On the same line, the semiparametric efficient score vector sν0 for ν 0 in the semiparametric model P in [START_REF] Fortunati | Semiparametric estimation in elliptical distributions[END_REF] is given by [10, Sect. 3.4, eq. ( 18)]:

sν0 ≜ s ν0 -Π(s ν0 |T 2 + T 3 ),
where, since T 2 is a (closed) finite-dimensional subspace and T 3 is closed, then T 2 + T 3 is closed. In general, since T 2 + T 3 is infinite-dimensional, a closed form for the projection operator Π(•|T 2 + T 3 ) does not exist in general. Consequently, an explicit expression for sν0 cannot be obtained. Fortunately, some further manipulation is still possible. Since T 2 ⊆ T 2 +T 3 , by using [10, A2, eq. ( 11)], sν0 can be expressed as:

sν0 ≜ s ν0 -Π(s ν0 |T 2 + T 3 ) = s ν0 -Π(s ν0 |T 2 ) -Π(s ν0 |(T 2 + T 3 ) ∩ T ⊥ 2 ) = tν0 -Π( tν0 |(T 2 + T 3 ) ∩ T ⊥ 2 ), (22)
where the last equality holds true since, by definition,

(T 2 + T 3 ) ∩ T ⊥ 2 ⊆ T ⊥ 2 and T 2 ∋ Π([s ν0 ] i |T 2 ) ⊥ T ⊥ 2 , ∀i.
The two following Lemmas are given, without proof, in [START_REF] Bickel | Efficient and Adaptive Estimation for Semiparametric Models[END_REF]Sect. 3.4,Prop. 3]. Here, to make the presentation as selfcontained as possible, we provide a complete proof of these two key results.

Lemma 1. The efficient Semiparametric FIM (SFIM) for ν 0 in the presence of the finite-and infinite-dimensional nuisance terms ξ 0 and g 0 , can be evaluated as:

Ī(ν 0 |ξ 0 , g 0 ) = Ī(ν 0 |ξ 0 ) -E 0 ww H , ( 23
)
where w ≜ Π tν0 |(T 2 + T 3 ) ∩ T ⊥ 2 , (24)
and

Ī(ν 0 |ξ 0 ) ≜ E 0 tν0 tH ν0 . (25)
Proof. We start by recalling that

E 0 {(•)(•) * } = ⟨•, •⟩ in H (see [7, Sect. 2.2]
). From ( 22), it follows that tν0 = sν0 + w.

Then, through direct calculation, we get:

E 0 sν0 sH ν0 = Ī(ν 0 |ξ 0 ) -E 0 tν0 w H - + E 0 w tH ν0 + E 0 ww H = Ī(ν 0 |ξ 0 ) -2Re E 0 sν0 w H -E 0 ww H . (26)
Then the equality in (23) follows immediately iff E 0 sν0 w H = 0. From the componentwise application of the inner product, we have that:

E 0 sν0 w H = 0 ⇔ ⟨[s ν0 ] i , w j ⟩ = 0, (27)
∀i, j ∈ {1, . . . , q}. To show that [s ν0 ] i is orthogonal to w j , ∀i, j, we note that according to its definition given in (22), [s ν0 ] i ∈ (T 2 + T 3 ) ⊥ . Moreover, again by definition in (24),

w j ∈ (T 2 + T 3 ) ∩ T ⊥ 2 ⊆ T 2 + T 3 . Consequently, ⟨[s ν0 ] i , w j ⟩ = 0, ∀i, j since (T 2 + T 3 ) ⊥ (T 2 + T 3 ) ⊥ .
It is worth noticing that the matrix Ī(ν 0 |ξ 0 ) is the efficient FIM for ν 0 in the presence of the finite-dimensional nuisance vector ξ 0 in the parametric submodel P 1,2 in [START_REF] Bickel | Efficient and Adaptive Estimation for Semiparametric Models[END_REF]. Consequently the CRB for ν 0 in the presence of ξ 0 is given by:

CRB(ν 0 |ξ 0 ) = Ī(ν 0 |ξ 0 ) -1 = I ν0 -I ν0ξ0 I -1 ξ0 I H ν0ξ0 -1 , (28)
where:

I ν0 ≜ E 0 {s ν0 s H ν0 }, (29)
is the FIM for ν 0 in the parametric submodel P 1 in (8).

Lemma 2. The efficient SFIM Ī(ν 0 |ξ 0 , g 0 ) for the model P in [START_REF] Fortunati | Semiparametric estimation in elliptical distributions[END_REF] is equal to the parametric efficient FIM Ī(ν 0 |ξ 0 ) in P 1,2 iff E 0 ww H = 0 and consequently iff w = 0. Moreover,

w = 0 ⇔ [ tν0 ] i = [s ν0 ] i -Π([s ν0 ] i |T 2 ) ⊥ T 3 , (30)
for i = 1, . . . , q.

Proof. To prove the double implication in (30), let us start with the implication ⇐. From [10, A2, eq. ( 4)] and by recalling that T 2 + T 3 is closed, we have that

(T 2 + T 3 ) ∩ T ⊥ 2 = (T ⊥ 2 ∩ T ⊥ 3 ) ⊥ ∩ T ⊥ 2 . Then, since (T ⊥ 2 ∩ T ⊥ 3 ) ⊆ T ⊥ 2
, from [10, A2, Prop. 4], we have that:

Π( tν0 |(T 2 + T 3 ) ∩ T ⊥ 2 ) = Π( tν0 |T ⊥ 2 ) -Π( tν0 |T ⊥ 2 ∩ T ⊥ 3 ). (31) By definition, [ tν0 ] i ∈ T ⊥ 2 . Moreover, if [ tν0 ] i ⊥ T 3 , then [ tν0 ] i ∈ T ⊥ 3 . Then, [ tν0 ] i ∈ T ⊥ 2 ∩ T ⊥
3 for i = 1, . . . , q. This implies that, from (31):

Π( tν0 |(T 2 + T 3 ) ∩ T ⊥ 2 ) = tν0 -tν0 = 0. (32)
Let us now prove the second implication ⇒. If the projection Π( tν0 |(T 2 + T 3 ) ∩ T ⊥ 2 ) = 0, then, from [10, A.2, eq. ( 4)]:

[ tν0 ] i ∈ [(T 2 + T 3 ) ∩ T ⊥ 2 ] ⊥ = (T ⊥ 2 ∩ T ⊥ 3 ) + T 2 , (33)
since T 2 and (T 2 +T 3 ) ⊥ are closed subspaces. Moreover, since

(T ⊥ 2 ∩ T ⊥ 3 ) ⊆ T ⊥ 2 , we have that: [ tν0 ] i ∈ (T ⊥ 2 ∩ T ⊥ 3 ) + T 2 = (T ⊥ 2 ∩ T ⊥ 3 ) ⊕ T 2 , (34)
where ⊕ indicates the direct sum of two subspaces. Then, since by definition [ tν0 ] i ∈ T ⊥ 2 , as a consequence of (34), we need to have that [ tν0 ] i ∈ T ⊥ 3 as well, or equivalently, [ tν0 ] i ⊥ T 3 for i = 1, . . . , q.

IV. THE EQUALITY BETWEEN THE ESCRB AND SSCRB

We can now use the results of Lemma 2 to show the equality in [START_REF] Besson | On the Fisher Information Matrix for multivariate elliptically contoured distributions[END_REF]. We start by specializing the semiparametric model P in [START_REF] Fortunati | Semiparametric estimation in elliptical distributions[END_REF] for the DOA estimation described in Sect. II as:

P = {p X (x|ν, ξ, g) = CES m (x; 0, Σ(ν, ξ), g) : ν ∈ Γ, ξ = (ζ T , σ 2 ) T ∈ Ψ, g ∈ G , (35)
then, its parametric submodel P 1,2 in (10) becomes:

P 1,2 = {p X (x|ν, ξ, g 0 ) = CES m (x; 0, Σ(ν, ξ), g 0 ) : ν ∈ Γ, ξ = (ζ T , σ 2 ) T ∈ Ψ, , g 0 ∈ G. (36)
The Elliptical Stochastic CRB ESCRB(ν 0 |ξ 0 , σ 2 0 ) for a specific density generator g 0 ∈ G can then be obtained as the inverse of the efficient FIM Ī(ν 0 |ξ 0 ) for the model P 1,2 in (36). From (28), this is equivalent to calculating the first top-left block submatrix of the inverse of the joint FIM for ν 0 and ξ 0 in the model P 1,2 in (36), i.e.

I(ν 0 , ξ 0 ) = I ν0 I ν0ξ0 I H ν0ξ0 I ξ0 , (37)
that can be evaluated by means of the elliptical Slepian-Bangs formula provided in [START_REF] Besson | On the Fisher Information Matrix for multivariate elliptically contoured distributions[END_REF]. On the other hand, the Semiparametric SCRB for the model P in (35) can be expressed as SSCRB(ν 0 |ξ 0 , σ 2 0 , g 0 ) = [ Ī(ν 0 |ξ 0 , g 0 )] -1 , where Ī(ν 0 |ξ 0 , g 0 ) is given in (23) and it has been evaluated in [START_REF] Fortunati | Semiparametric CRB and Slepian-Bangs formulas for complex elliptically symmetric distributions[END_REF].

Consequently, the equality ( 6) is verified iff :

Ī(ν 0 |ξ 0 ) = Ī(ν 0 |ξ 0 , g 0 ) ⇔ [ tν0 ] i ⊥ T 3 , (38)
for i = 1, . . . , q, where the double implication comes from a direct application of Lemma 2. Moreover, from (20), confirm-ing the orthogonality condition on the RHS of equation ( 38) is equivalent to verifying the following equality:

Π( tν0 |T 3 ) = Π(s ν0 |T 3 ) -Π(I ν0ξ0 I -1 ξ0 s ξ0 |T 3 ) = 0. (39)

Let us now prove that (39) actually holds. To this end, we rely on the results already derived in [START_REF] Fortunati | Semiparametric CRB and Slepian-Bangs formulas for complex elliptically symmetric distributions[END_REF]. By adopting the same notation of [START_REF] Fortunati | Semiparametric CRB and Slepian-Bangs formulas for complex elliptically symmetric distributions[END_REF], let us introduce the quantities: ψ 0 (t) ≜ d ln g 0 (t)/dt, (40)

P 0 i ≜ Σ -1/2 0 Σ 0 i Σ -1/2 0 , ( 41
)
tr P 0 i = tr Σ 0 i Σ -1 0 = vec(Σ -1 0 ) H vec(Σ 0 i ), (42)
for i = 1, . . . , q + r where r = q 2 + 1.

As discussed in [7, Sect. 3.2.2] (see also [START_REF] Bickel | Efficient and Adaptive Estimation for Semiparametric Models[END_REF]Sect 4.2]) the projection operator Π(•|T 3 ) onto the nuisance tangent space T 3 , defined in [7, Sect. 3.2.2, Lemma 1], can be evaluated as the following conditional expectation:

[Π(h|T 3 )] i = E 0 {h i | √ Q}, ∀h i ∈ H. (43)

Consequently, from [2, eq. ( 42)], we have: 1

for i = 1, . . . , q, j = 1, . . . , r and

Moreover, let us introduce W ν0 and W ξ0 as matrices whose i-th column is given by vec(Σ 0 i ), i.e. :

Then, from (42), it is immediate to verify that eqs. ( 44) and (45) can be rewritten in vectorial form as:

From [START_REF] Besson | On the Fisher Information Matrix for multivariate elliptically contoured distributions[END_REF], the matrices I ξ0 and I ν0ξ0 can be expressed as:

where

). Following the notation of [START_REF] Stoica | The stochastic CRB for array processing: a textbook derivation[END_REF], let us introduce the matrices G ≜ KW ν0 and ∆ ≜ KW ξ0 = [V|u], where u ≜ Kvec(I) = vec(Σ -1 0 ). Then, by collecting the previous results, we have:

By substituting (49) and ( 53) in (39), and by using the 1 In [2, eq. ( 42)] there were some typos that have been fixed in [START_REF] Fortunati | Corrections to "Semiparametric CRB and Slepian-Bangs formulas for complex elliptically symmetric distributions[END_REF].

properties of the Kronecker product, we can easily verify that:

where, for notational simplicity, we introduced the orthogonal projection matrices:

where the last equality is defined by [5, eq. ( 14)].

Thus the condition (30) of Lemma 2 is equivalent to:

To conclude, we need to prove that vec(I) H Π ⊥ ∆ G = 0. Following again [START_REF] Stoica | The stochastic CRB for array processing: a textbook derivation[END_REF], let us indicate as g i the i-th column of G. Then, by using the equality in (56), the condition (57) can be written as:

i = 1, . . . , q. Now, the second term in ( 58) is equal to 0, since from [5, eq. ( 28)], u H Π ⊥ V g i = 0, i = 1, . . . , q. Then, the last step consists in proving that:

To this end, by exploiting the relation in [5, eq. ( 23)] and the properties of the vec operator, we have that the following chain of equality holds true:

i = 1, . . . , q, where the matrix Z i is given in [5, eq. ( 18)] as:

where c 0,i is the i-th column of the matrix Γ 0 in (1) while

We note that the last equality in (60) follows directly from the fact that Π ⊥ Σ -1/2 0 A0 Z i = 0, i = 1, . . . , q. This proves the identity (6).

V. CONCLUSION

We showed that the Elliptical SCRB ESCRB(ν 0 |ξ 0 , σ 2 0 ), evaluated at a specific and a-priori known density generator g 0 ∈ G, is actually equal to the Semiparametric SCRB SSCRB(ν 0 |ξ 0 , σ 2 0 , g 0 ) where, on the contrary, g 0 is assumed to be an additional unknown nuisance function:

) where D 0 is a matrix whose i-th column is d 0,i in (62).


Killian Castillon
email: killian.castillon-du-perron@univ-cotedazur.fr

Du Perron

Dino Lopez Pacheco

Fabrice Huet
email: fabrice.huet@univ-cotedazur.fr

Understanding Delays in AF XDP-based Applications

Keywords: Latency, XDP, AF XDP, High Performance, Linux, Network Drivers

Packet processing on Linux can be slow due to its complex network stack. To solve this problem, there are two main solutions: eXpress Data Path (XDP) and Data Plane Development Kit (DPDK). XDP and the AF XDP socket offer full interoperability with the legacy system and is being adopted by major internet players like Open vSwitch or Facebook. While the performance evaluation of AF XDP against the legacy protocol stack in the kernel or against DPDK has been studied in the literature, the impact of the multiple socket parameters and the system configuration on its latency has been left aside. To address this, we conduct an experimental study to understand the XDP/AF XDP ecosystem and detect microseconds delays to better architect future latency-sensitive applications. Since the performance of AF XDP depends on multiple parameters found in different layers, finding the configuration minimizing its latency is a challenging task. We rely on a classification algorithm to group the performance results, allowing us to easily identify parameters with the biggest impact on performance at different loads. Last, but not least, we show that some configurations can significantly decrease the benefits of AF XDP, leading to undesirable behaviors, while other configurations are able to reduce such round trip delays to an impressive value of 6.5 µs in the best case, including the tracing overhead. In summary, AF XDP is a promising solution, and careful selection of both application and socket parameters can significantly improve performance.

I. INTRODUCTION

Nowadays, cloud applications are deployed as chains of functions that need to communicate between each other to provide full services [START_REF] Gan | An open-source benchmark suite for microservices and their hardwaresoftware implications for cloud & edge systems[END_REF]. While the replacement of a monolithic architecture for the adoption of a microservice one (where several functions interconnect) eases development and deployment, it might also bring several challenges that need to be addressed in order to preserve the Quality of Experience required by the customers.

Hence, latency-sensitive applications, which are increasingly popular, such as live streaming, live gaming, and data query just to mention a few [START_REF] Li | Livenet: A low-latency video transport network for large-scale live streaming[END_REF], [START_REF] Gan | An open-source benchmark suite for microservices and their hardwaresoftware implications for cloud & edge systems[END_REF], call for optimizations at several layers of the service architecture. The final objective being to achieve fast communication between functions to satisfy the customers' Quality of Experience requirements. For instance, one single search at Google leads to thousands of RPCs (Remote Procedure Calls) [START_REF] Barroso | Attack of the killer microseconds[END_REF].

At the network layer, microservices can be smartly placed to bring optimized network overlays where communicating functions are close to each other on the Data Center fabric. Thus, network processing and latency is reduced while avoiding bottlenecks [START_REF] Li | Livenet: A low-latency video transport network for large-scale live streaming[END_REF].

To increase the benefits from this smart microservice placement, one can deploy fast network packet processing solutions on the servers. Note that overlay networks are frequently composed of both hardware and software networking devices. While the former already provides the fastest packet processing, servers where software networking devices live offer by default only slow packet processing capabilities [START_REF] Zhang | The Demikernel Datapath OS Architecture for Microsecond-scale Datacenter Systems[END_REF], [START_REF] Zhuo | Slim: {OS} Kernel Support for a {Low-Overhead} Container Overlay Network[END_REF].

Two main different solutions can be adopted to speed up the processing of network packets: DPDK (Data Plane Development Kit) [START_REF]DPDK Website[END_REF] and XDP (eXpress Data Path) [START_REF]XDP Project GitHub profile[END_REF]. Both solutions offer fast packet processing capabilities by sending, as fast as possible, network packets to the user space of a system, where applications are expected to live, and where parallelism and complex operations are easy to implement. Indeed, the packet processing in the network protocol stack implementation of the kernel system might be very slow due to the complexity of the current code [START_REF] Zhuo | Slim: {OS} Kernel Support for a {Low-Overhead} Container Overlay Network[END_REF], [START_REF] Zhang | The Demikernel Datapath OS Architecture for Microsecond-scale Datacenter Systems[END_REF]. The slow packet processing can moreover be exacerbated when overlay technologies are deployed, such as VxLAN [START_REF] Mahalingam | Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks[END_REF] as packets can be processed multiple times by the entire network stack implementation [START_REF] Zhuo | Slim: {OS} Kernel Support for a {Low-Overhead} Container Overlay Network[END_REF].

DPDK provides a set of libraries to create a direct path between the network interface card (NIC) and the user space programs. By doing so, DPDK completely steals the packet from the kernel, with the side effect of standard Linux networking tools not working anymore [START_REF] Tu | Revisiting the open vSwitch dataplane ten years later[END_REF].

XDP, with the help of the associated AF XDP socket, can be seen as an intermediary solution between DPDK and the default network stack implementation of current systems. AF XDP relies on the XDP hook which is implemented at the NIC driver level in Linux systems to create a short path between the NIC and user space (see Figure 1). A detailed view of the AF XDP architecture is provided in §II. Since AF XDP has full kernel support, usual networking tools (ip, ping, ...) employed in Linux systems are fully compatible with AF XDP.

Understanding every factor potentially impacting the packet processing delay is important to chase out microsecond delays in microservice-based architectures [START_REF] Barroso | Attack of the killer microseconds[END_REF]. We present in this paper an in depth study about the cost of packet processing by AF XDP in a testbed described in §IV, using two different network drivers, Mellanox mlx5 and Intel i40e. To do this, we generated close to 400 different configurations from which we analysed the results using a classification algorithm to determine what parameters lead to the best and the worst latencies ( §VI). All these aspects have not been or only partially covered by the current state of the art ( §VII).

Our results, available in §VI, show that:

• energy saving mechanism have little to no impact on the latency with AF XDP socket with the correct parameters. • with the best combination of parameters, the round-trip latency between two servers can reach 6.5 µs, which includes an approximate 5-10 µs overhead due to our performance tracing technique.

II. ARCHITECTURE OF XDP AND AF XDP A. XDP eBPF (extended Berkeley Packet Filter) is a solution to extend the capabilities of the Linux kernel, without the need to modify or write new kernel modules. eBPF allows running sandboxed program inside hooks points located at several places of the kernel architecture. One of those hooks is called XDP (eXpress Data Path) and is used to install an eBPF program at the earliest stage of the networking stack, inside the network device driver By leveraging the XDP capabilities, firewalls and loadbalancers providing very fast packet processing have been deployed on servers [START_REF]GitHub repository of Katran by Meta[END_REF].

B. AF XDP

AF XDP [START_REF] Karlsson | The Path to DPDK Speeds for AF XDP[END_REF], [START_REF]AF XDP -The Linux Kernel documentation[END_REF] is a new socket address family which leverages XDP to receive and send packets directly between a queue of the NIC and some user space process. It was proposed as a way to easily bypass the legacy network protocol stack implementation, as shown in Figure 1.

A memory area, called a UMEM, is shared by both the application and the driver. It is divided into equally sized frames which can be owned by only one of them at a time.

The UMEM relies on two memory rings, Fill (FILL) and Completion (COMP), which are single consumer / single producer. They are used to transfer ownership of the UMEM frames between user space and kernel space. The socket itself has two more rings, RX and TX, used for the actual exchange of the packets. The rings do not contain the actual packets but the addresses of the frames, as well as the length of the data for the last two rings.

Finally, the socket has two more rings, RX and TX, used for the actual exchange of the packets.

As the Linux networking stack is bypassed, most of the processing is done either in user-space by the application or in the driver of the NIC. Therefore, in a simple application, most of the possible performance improvement will be obtained by tweaking NIC hardware and driver settings as well as AF XDP socket parameters.

III. OPTIMIZATION PARAMETERS

As the Linux networking stack is out of the way when using AF XDP, the surface area for optimizations is further reduced to hardware and driver settings and the parameters of the AF XDP socket itself.

A. Hardware and driver settings

Many of the optimizations brought to the networking hardware and drivers are set with the goal to achieve higher rates, in terms of bits per second (bps) or packets per second (pps). However, our goal is to identify the different components impacting latency and its stability (jitter) independently of the throughput of the application. We found that when optimizing for throughput, one may induce higher latency especially at lower packet rates.

1) Interrupt delaying: The NIC historically used to send an interrupt to the kernel for each received packet, but this approach doesn't work well with high packet rates [START_REF] Salim | Beyond softnet[END_REF]. There are two solutions: interrupt coalescing by NAPI or by the NIC itself. NAPI (New API) is a subsystem of the Linux Kernel that improves high-speed networking by mitigating interrupts [START_REF]NAPI article on the Linux Foundation's wiki[END_REF]. Instead of sending an interrupt for each packet, NAPI triggers one interrupt to put the driver in poll mode. In this mode, the driver can be later polled by the kernel to process the packets received since the last interrupt, saving CPU time. On the hardware side, coalescing delays triggering an interrupt when receiving a packet by a certain amount of time or frames, reducing interrupt overhead. Both mechanisms involve waiting before processing a packet, which may have negligible overhead at high packet rates but can be significant at lower rates [START_REF] Salah | Performance analysis and comparison of interrupt-handling schemes in gigabit networks[END_REF].

2) Energy saving mechanisms: Modern CPUs come with energy saving mechanisms, notably C-States, which allow a CPU to enter a lower power state and disable some parts of itself when idle. This can lead to higher latency and jitter as the CPU may be in a energy saving mode when receiving a packet, especially at lower rates.

B. AF XDP socket related parameters

The socket itself and the associated UMEM are also configurable. For the UMEM and the rings, the number of frames as well as their size can be configured. However, more options are available for the socket:

1) Zero-Copy: By default, sending data from user space involves copying at least once the packet between a memory region owned by the application and one owned by the kernel for it to format the data correctly before sending it on the network. However, network drivers supporting XDP can also add support for sending and receiving packets without copying intermediate buffers into the NIC. Furthermore, zero-copy in AF XDP works both on the receiving and sending sides allowing to save latency both ways [START_REF]AF XDP: Introducing zero-copy support[END_REF].

2) XDP USE NEED WAKEUP: By setting the XDP_USE_NEED_WAKEUP option, the application enables the need_wakeup flag on the TX and FILL ring. This flag is set by the driver and when false, enables the application to write and read from the rings directly without sending a syscall to the driver beforehand. When the flag is true, the application runs in the usual mode.

3) Busy Polling: Typically, to signal an application that packets are available, the underlying network driver will send IRQs (interrupts). However, this uses CPU and can downgrade application performance if it runs on the same core as the one sending the IRQs due to context switches. With busy polling, the application has to wake up the driver so it can start processing packets both on the rx and tx sides. This enables all processing to be performed on one core and therefore prevent core-to-core cache transfers which may slow down performances [START_REF] Karlsson | RFC for busy poll support for AF XDP sockets[END_REF].

C. Application parameters 1) Polling: What we call polling is the use of the system call poll() to check for any IO activity. When we enable this option on the application, it will call poll regularly to process IO events. For our benchmarks, servers A and C will behave as traffic generators and receivers, while server B will only behave as a forwarder.

The servers interconnection is shown in Fig. 2. Servers are connected by a DAC cable, the queues (q0 and q1) being on the same port (ie. NIC interface). We estimated the latency difference due to the speed difference in both NIC models to be in the order of nanoseconds in our setup, which is negligible compared to our results. To start off, using the system's ping utility we had 191 µs between Mellanox hosts and 211 µs between Intel ones.

V. METHODOLOGY

To measure latency, we have deployed the xdpsock program[18] on every server which is the default application provided in Linux for testing AF XDP to generate, receive or forward network packets (called pings in this article). To gather delay-related statistics of our network, we added USDT (Userland Statically Defined Tracing) probes [START_REF]Using User-Space Tracepoints with BPF[END_REF] to xdpsock. Using perf [START_REF]Perf Wiki[END_REF], we obtain the time when batch of packets is sent and the time of its reply. As sending and receiving is done on the same server, we avoid clock synchronization issues when gathering timestamps.

1) Ping parameters: xdpsock generates batches of packets at fixed intervals. In our experiments we chose to use batches of 1 packet of 64 bytes length, with a 1 second inter-batch interval. Those parameters are known to be the most challenging at higher rates [START_REF] Tu | Revisiting the open vSwitch dataplane ten years later[END_REF], [START_REF] Høiland-Jørgensen | The eXpress data path: Fast programmable packet processing in the operating system kernel[END_REF], [START_REF] Rizzo | Netmap: A Novel Framework for Fast Packet {I/O}[END_REF], which will be addressed later in our experiments.

2) Tracing overhead: We chose perf and USDT probes as they enable us to add our own probes to xdpsock, while having a limited impact on the performance. Event tracing is only performed on the sender / receiver. To estimate our tracing overhead, we ran experiments with and without activating tracing on the forwarder and computed the time difference between the two versions. Depending on the interbatch interval, the typical tracing overhead was found to be between 5 µs and 10 µs.

3) Hosts configuration and run execution: We used Ansible [START_REF]Ansible official website[END_REF] to configure the servers before each run as well as to run the applications and retrieve the results. This allowed us to run all the configurations from a definition file and ensure that the system would be set up using the correct parameters before each run:

• correct number of queues set on the NIC • flow steering on the NIC to the queue on which the application was listening • rx and tx coalescing on the NIC • selected busy polling values set in the system files • setting the correct C-States level Each benchmark is run alone, during 30 seconds and at least 3 times to ensure consistency between runs. When running with busy polling enabled, we force the application to run on the same core as the driver to benefit from the better cache locality.

VI. RESULTS

In this section we investigate the impact of the various optimization parameters on the latency, for both Intel and Mellanox cards.

To do so, we generated benchmark configurations for every combination of the following parameters: zero-copy, busy poll, need wakeup, C-States level, rx and tx coalescing. This produced around 100 different configurations.

In our first set of experiments, and to avoid any interference from any kind of bottleneck, with decided to send traffic with a 1 second inter-batch interval and batch sizes of 1 packet of 64 bytes as mentioned in §V.

1) Measurements analysis: During our experiments, we noticed that some configurations with zero-copy and no polling mechanism led to very undesirable behaviors on the Intel driver, with latencies similar to the inter-batch intervals. Therefore, we decided to consider those configurations as outliers in our future analysis, while taking them into considerations for our conclusion. To further clean up our traces, we decided also to not consider configurations leading to latencies higher than the one observed with system's pings utility which was around 200 µs in our testbed.

With the remaining configurations, 69 on Mellanox and 40 on Intel, we computed general latency statistics (e.g. mean, median, standard deviations, quartiles, minimum and maximum) for every result and averaged them out per configuration for each NIC vendor (Intel and Mellanox). When analyzing the results, we found that some configurations lead to similar latency, despite having different parameters.

This suggested that one or more parameters coupled together lead to those particular results. However, identifying those clusters by hand is still challenging. Thus we decided to use rely on the k-means clustering algorithm to classify the configuration into multiple groups.

At first, we feed the algorithm with all the metrics that we had computed: mean, median, standard deviation, first and third quartiles, minimum and maximum. However, the resulting clustering failed to classify the traces under good performances and bad performance in a clear manner.

However, feeding K-means with all statistics, except the minimum and maximum observed latencies successfully grab the performance profile. To find the number of clusters, we wanted to obtain a good clustering based on the following criteria: low standard deviation among clusters, small numbers of clusters and significant statistical differences between them. This gave us a number of 4 clusters on Intel and 6 on Mellanox. To visualize the latency distribution of each configuration we decided to use Kernel Density Estimate (KDE) plots, which uses a continuous probability density curve and can be compared to a histogram but easier to read [START_REF]Seaborn's documentation on KDE[END_REF]. As we can see in Fig. 3 where we plot the KDE of our traces assigning them to its respective cluster, it is easy to identify each cluster on the x-axis, which represents the average latency for a specific configuration. According to our cluster analysis, C-States levels as well as the NIC coalescing did not have a significant impact on the results. However, enabling polling on the receiver always lead to worse results for both drivers.

Tables I and II summarize the obtained clusters, ordered from the best configuration (cluster 0) to the worst (cluster 5), along with the observed mean latency, standard deviation and relevant (i.e. always present) parameters.

2) Impact of load on best and worse configuration: Finally, we wanted to test if the best and worst configurations would be impacted by the packet rate, especially when close to the maximum line rate.

We then selected 2 configurations per cluster and re-run the tests with an inter-packet interval close to the line rate. It was computed from minimum latency previously obtained per vendor with equation ( 1), where B is the NIC bandwidth in bits/s, l min the minimal latency and P is the size of a packet in bytes, which we set to 64.

Packets per second

= B × l min 8 × P (1)
This gave us a result of around 200 pps for Intel and 1750 pps for Mellanox, which we rounded up to an inter-batch interval of 5 ms for Intel and 0.6 ms for Mellanox. This led to lower latencies across all clusters, down to a mean latency of We also tested running the application without an interbatch interval. To do so we added a subsampling argument to our tracing function to not overload perf which could alter our application's performance. This led to higher latencies for Mellanox for the previously best results as we started to have some buffering. On Intel however, this crashed the sender's network driver at least for the NIC on which the program was running, preventing us from obtaining any result at this rate.

A. Conclusion

We ran multiple experiments to find what would be the best parameters for the AF XDP sockets and the related application's datapath to lower the end-to-end latency. We identified clusters of configurations giving similar latencies with a 1 second inter batch interval and found that those combinations stayed relevant at higher packet rates. We finally found minimal latencies of 6.5 µs for Mellanox and 9.7 µs for Intel as well as the corresponding configurations. All these measurements include a measurement overhead estimated at ∼5 µs.

VII. RELATED WORK

The impact of the kernel protocol stack implementation on the packet processing delay has been analyzed in details [START_REF] Cai | Understanding host network stack overheads[END_REF], and it has been shown that the usual packet processing pipeline can be slow, especially for packets needing several rounds of processing, such as packets in a VxLAN tunnel [START_REF] Zhuo | Slim: {OS} Kernel Support for a {Low-Overhead} Container Overlay Network[END_REF]. To alleviate the problem, different strategies has been proposed, such as custom overlay networks aiming at bypassing the network stack [START_REF] Zhuo | Slim: {OS} Kernel Support for a {Low-Overhead} Container Overlay Network[END_REF], [START_REF]Cilium -Linux Native, API-Aware Networking and Security for Containers[END_REF].

One widely accepted solution to the slow packet processing at the OS kernel is, with support of the Network Interface Cards, to take a packet at the earliest stage (basically, after reception of a packet by a NIC port) and send it immediately to the user space of a system. To do so, two different solutions exist today: the Data Plane Development Kit and the eXpress Data Path.

To assess the benefits of DPDK and XDP, the networking community has carried out extensive experimental studies to compare the performance of DPDK vs the Linux kernel [START_REF] Karlsson | The Path to DPDK Speeds for AF XDP[END_REF], [START_REF] Tu | Revisiting the open vSwitch dataplane ten years later[END_REF], XDP vs the Linux kernel [START_REF] Özturk | Performance Evaluation of eXpress Data Path for Container-Based Network Functions[END_REF], [START_REF] Høiland-Jørgensen | The eXpress data path: Fast programmable packet processing in the operating system kernel[END_REF] and DPDK vs XDP [START_REF] Özturk | Performance Evaluation of eXpress Data Path for Container-Based Network Functions[END_REF], [START_REF] Høiland-Jørgensen | The eXpress data path: Fast programmable packet processing in the operating system kernel[END_REF]. Those papers report that XDP can achieve higher throughput than DPDK on a multi-core setting as well as providing more flexibility to integrate into an application on Linux.

A deep understanding of the packet processing path is important to chase out microseconds delays able to impact current latency sensitive cloud applications, frequently deployed in the form of chains of microservices [START_REF] Barroso | Attack of the killer microseconds[END_REF], [START_REF] Gan | An open-source benchmark suite for microservices and their hardwaresoftware implications for cloud & edge systems[END_REF]. Consequently, in this paper we go one step further and carry out an experimental study to understand the nature of latency added at every stage of the AF XDP architecture by exploring multiple parameters related to the socket, the NIC driver and some system parameters. As far as we know, this kind of study has not been reported yet in the literature.

VIII. CONCLUSIONS

The eXpress Data Path and AF XDP are a promising solution to the slow packet processing problem in Linux's kernel space. In this paper, we presented a comprehensive study on the performance of AF XDP from the latency point of view, as we believe, it is a key point to chase out microsecond delays and preserve the Quality of Service of latency sensitive applications. Employing xdpsock, the standard AF XDP testing program, we conducted a set of experiments to analyze every optimization parameters of the AF XDP socket, which allows to easily exchange data packet between a user space program and the driver NIC.

We have run a configuration for every combination of parameters available with the AF XDP socket and the sample AF XDP application. We then identified different clusters of configurations leading to similar results and summarized what parameters lead to those different clusters and what where the latencies we obtained with them. We have also shown that we can achieve below 10 µs latency using a basic AF XDP application when there is no buffering involved.

The results shown here are part of an ongoing project to understand the internals of AF XDP. As a next step, we plan to evaluate the impact of AF XDP on the energy consumption of servers, before and after the multiple optimizations, and evaluate the impact of the parameter optimization on real applications deployed as a chain of microservices, especially for data streaming.

Fig. 1 :

 1 Fig. 1: XDP and AF XDP in the Linux networking stack.

  IV. NETWORK TESTBEDOur testbed consists of three Dell PowerEdge R520 servers with two Intel Xeon E5-2420 v2 CPUs at 2.20 GHz, and 32 GB of 1600 MHz DDR3. Our servers feature the following NIC models: • Mellanox MT2892 ConnectX-6 100 Gbps with 2 ports • Intel X710 10 Gbps with 4 ports Those NIC are shared among the three servers (A,B,C), A with Mellanox only, C Intel only and B having both. All servers are running Ubuntu 23.04 on kernel 6.2, the driver for the Mellanox NICs is mlx5 and i40e for the Intel ones. Mellanox OFED version 23.07 has been set up on relevant servers.

Fig. 2 :

 2 Fig. 2: Network testbed. Server A and C are traffic generators and receivers. Server B forwards only traffic back to the traffic generator. q0 and q1 refer to queues.

Fig. 3 :

 3 Fig. 3: Kernel density estimate plot of the top 50 configurations for both vendors after k-means clustering. Clusters are ordered from the lowest mean latency to highest.

TABLE I :

 I Clusters, statistics and relevant parameters for Mellanox at 1 packet per second. A parameter not mentioned means that it can be either enabled or disabled.

	Cluster	Mean	Std	Relevant parameters
	0	13.36 µs	1.90 µs	Busy polling enabled, need
				wakeup set to false
	1	70.00 µs	6.17 µs	Busy polling and need wakeup
				enabled
	2	124.57 µs	8.29 µs	Busy polling and zero-copy dis-
				abled
	3	139.86 µs	36.91 µs	Need wakeup enabled, Polling
				enabled on the forwarder and
				busy polling disabled
	4	153.45 µs	38.09 µs	Polling enabled on both for-
				warder and receiver, busy polling
				and need wakeup disabled

TABLE II :

 II Clusters, statistics and relevant parameters for Intel at 1 packet per second. A parameter not mentioned means that it can be either enabled or disabled.6.5 µs on Mellanox and 9.7 µs on Intel with the best cluster, and the different identified clusters stayed relevant.

IX. ACKNOWLEDGEMENTS

This work has been supported by the French government, through the EUR DS4H Investments in the Future project managed by the National Research Agency (ANR) with the reference number ANR-17-EURE-0004.