Hadrien Hendrikx
email: hadrien.hendrikx@inria.fr

Paul Mangold
email: paul.mangold@inria.fr

Aurélien Bellet
email: aurelien.bellet@inria.fr

The Relative Gaussian Mechanism and its Application to Private Gradient Descent

The Gaussian Mechanism (GM), which consists in adding Gaussian noise to a vector-valued query before releasing it, is a standard privacy protection mechanism. In particular, given that the query respects some L2 sensitivity property (the L2 distance between outputs on any two neighboring inputs is bounded), GM guarantees Rényi Differential Privacy (RDP). Unfortunately, precisely bounding the L2 sensitivity can be hard, thus leading to loose privacy bounds. In this work, we consider a Relative L2 sensitivity assumption, in which the bound on the distance between two query outputs may also depend on their norm. Leveraging this assumption, we introduce the Relative Gaussian Mechanism (RGM), in which the variance of the noise depends on the norm of the output. We prove tight bounds on the RDP parameters under relative L2 sensitivity, and characterize the privacy loss incurred by using output-dependent noise. In particular, we show that RGM naturally adapts to a latent variable that would control the norm of the output. Finally, we instantiate our framework to show tight guarantees for Private Gradient Descent, a problem that naturally fits our relative L2 sensitivity assumption.

Introduction

Differential Privacy (DP) [START_REF] Dwork | Differential privacy[END_REF] is considered the gold standard for protecting privacy, for instance in machine learning. In this framework, a curator has a database x, and would like to answer a query R on x by releasing an output R(x). Yet, releasing R(x) might reveal sensitive information on x. Instead, the curator may use a private algorithm A to release a sanitized approximation A(R)(x) of R(x). To guarantee that the amount of information leaked by releasing A(R)(x) is limited, DP ensures that the distributions of A(R)(x) and A(R)(y) are close for any y ∼ x, i.e., that is close to x according to a neighboring relation (databases that only differ in one row for instance). Several divergences have been considered to measure the closeness between these two distributions, leading to different variants of DP. Among them, Rényi-Differential Privacy (RDP), which is based on the Rényi divergence, has become popular for its mathematical properties [START_REF] Mironov | Rényi differential privacy[END_REF]. Definition 1 (Rényi Differential Privacy). A randomized algorithm A satisfies (α, ε)-RDP for α > 1 and ε > 0 if D α (A(x)||A(y)) ≤ ε for all pairs of neighboring datasets x ∼ y, where D α (A(x)||A(y)) is the α-Rényi divergence between A(x) and A(y).

Preprint. Under review.

arXiv:2308.15250v1 [cs.LG] 29 Aug 2023

A fundamental building block for designing a private algorithm A is the Gaussian Mechanism (GM σ ) [START_REF] Dwork | Calibrating noise to sensitivity in private data analysis[END_REF][START_REF] Dwork | The algorithmic foundations of differential privacy[END_REF], which adds Gaussian noise to the private value R(x):

GM σ (R)(x) = R(x) + N(0, σ 2 ), for some σ 2 > 0 .

(1)

It is very common (e.g., in machine learning) to compose multiple calls to GM σ to build iterative algorithms like differentially private gradient descent [START_REF] Song | Stochastic gradient descent with differentially private updates[END_REF][START_REF] Bassily | Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds[END_REF]. RDP is able to tightly track the privacy guarantees of (compositions of) GM σ , and can be converted into the more classical (ϵ, δ)-DP variant [START_REF] Mironov | Rényi differential privacy[END_REF].

The noise scale σ 2 of GM σ is based on an L2 sensitivity assumption, which guarantees that for any neighboring inputs x ∼ y, the query R verifies:

∥R(x) -R(y)∥ 2 ≤ R 2 abs (2)
for some R abs > 0. In particular, for σ 2 = αR 2 abs 2ε , GM σ (R)(x) satisfies (α, ε)-RDP. It is thus crucial to estimate the L2 sensitivity precisely to achieve the best possible privacy-utility trade-off. Unfortunately, this R abs constant is often not directly known and difficult to bound tightly. In some cases, the distance between outputs is also highly correlated to the norm of these outputs, and this is the case in particular when the outputs depend on a non-private latent variable.

Consider for instance an institute that would like to assess the mean salary for different jobs in a given company. Individual salaries are sensitive information, but people's job is not secret, and the average salary per job is the desired output. If we were to use the standard Gaussian Mechanism, then we would need an absolute sensitivity bound of the form of (2) (note that other types of noises, such as Laplace, would require similar bounds in other norms, such as L1, but the absolute aspect would remain). To do this, the simplest approach is to use a bound on the maximum possible salary across all jobs in the company. However, this is not satisfactory since results for lower-paid jobs would be dominated by noise. An alternative is to restrict the neighboring relation to people that have the same job, which is possible since the job is not private. The problem is that estimating the salary per job (or a bound on it) is exactly what we would like to achieve in the first place. In this case, absolute sensitivity bounds are thus unsatisfactory, and would lead to unnecessarily high, as well as unfair (since the precision would be higher for well-paid jobs) estimates of the mean salary per job. Now consider that we know that by law, there should not be more than 10% variations in salary for a given job in a given company: this corresponds to a relative sensitivity assumption. In this case, one is tempted to calibrate the noise to the empirical mean salary for a given job, since we know that all the people with the same job in this company have comparable salaries. In this paper, we tightly characterize how to scale the noise under this relative sensitivity assumption, leading to precise and fair estimates of the mean salaries per job. Note that this simple example directly translates for instance to releasing gradients, where the job would be the point at which they are computed and the salary would be their magnitude.

Our contributions are the following: (i) We introduce the Relative L2 Sensitivity, which generalizes the standard L2 sensitivity by allowing the upper bound to depend on the norm of queries. (ii) We leverage this assumption to introduce the Relative Gaussian mechanism (RGM), in which the noise that we introduce depends on the output that we are about to release. (iii) We show tight privacy guarantees for the Relative Gaussian Mechanism. (iv) We show how the Relative Gaussian mechanism can be applied for Private Gradient Descent to provide adaptivity to the gradients' magnitude.

We first review related work in Section 2. We then define the Relative L2 Sensitivity in Section 3, and introduce RGM in Section 4. Finally, we instantiate the results for gradient descent on quadratics in Section 5, and present some corresponding numerical illustrations in Section 6.

Related work

Local sensitivity. Several classic techniques in the DP literature seek to avoid the calibration of noise to global sensitivity by relying on the notion of local sensitivity. The local sensitivity LS R (x) = max y:y∼x ∥R(x) -R(y)∥ of a dataset x measures how much R(y) can differ from R(x) for any neighbor y of x, which can be much smaller than the global sensitivity. In general however, calibrating the noise to the local sensitivity does not provide privacy, as two neighboring datasets may have very different local sensitivities. To go around this issue, previous work has proposed approaches based on smoothing the local sensitivity [START_REF] Nissim | Smooth sensitivity and sampling in private data analysis[END_REF], or proposing and privately testing the validity of a local sensitivity bound before releasing the output [START_REF] Dwork | Differential privacy and robust statistics[END_REF].

Our assumption of bounded relative sensitivity is related to local sensitivity, in the sense that it implies that neighboring datasets have similar local sensitivities (see Section 3 for details). However, our framework does not seek to reduce the noise compared to approaches based on (a tight bound on) global sensitivity, but instead address situations where (i) the scale of R(x) (and thus the global sensitivity) is not known in advance for the dataset x of interest, for instance because it depends on a latent variable (like the job type in the example of Section 1, or the distance to the optimum in gradient descent), and (ii) for any x ∼ y, ∥R(x) -R(y)∥ can be approximately bounded by a constant factor times ∥R(x)∥. In this context, RGM γ,σ naturally adapts to the latent variable, while our bounded relative sensitivity assumption ensures that it satisfies differential privacy.

Beyond absolute sensitivity. In some cases, absolute sensitivity can be high due to the presence of outliers. [START_REF] Tsfadia | Friendlycore: Practical differentially private aggregation[END_REF] proposed an algorithm to refine an absolute sensitivity bound by privately discarding outliers. [START_REF] Brunel | Propose, test, release: Differentially private estimation with high probability[END_REF] goes further and uses distributional assumptions to privately estimate queries whose absolute sensitivity is unbounded. The main drawback of their method is that, in unfavorable cases, the algorithm may stop without returning anything. In contrast to these methods, our relative Gaussian mechanism always returns a value, without relying on an absolute sensitivity bound. Instead, it uses a relative sensitivity assumption, that does not require the absolute sensitivity to be bounded.

Private gradient descent. Differentially private gradient descent (DP-GD) and its stochastic variant (DP-SGD) were first proposed by [START_REF] Song | Stochastic gradient descent with differentially private updates[END_REF]. These algorithms and further variations have been widely studied as private minimizers of the empirical risk [START_REF] Song | Stochastic gradient descent with differentially private updates[END_REF][START_REF] Bassily | Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds[END_REF], Wang et al., 2017], and of the population risk [Bassily et al., 2019[START_REF] Vitaly Feldman | Private stochastic convex optimization: optimal rates in linear time[END_REF]. All these algorithms have been formally shown to achieve the optimal utility derived by [START_REF] Bassily | Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds[END_REF]. The analysis crucially relies on an absolute L2 sensitivity bound on the gradients (typically obtained by assuming the loss function to be Lipschitz) to calibrate the noise. Unfortunately, this often leads to the injection of excessive amounts of noise. Abadi et al. [2016b] proposed a more practical version of DP-SGD (implemented notably in PyTorch Opacus [Yousefpour et al., 2021] and TensorFlow Privacy [Abadi et al., 2016a]) which uses gradient clipping to reduce gradients' L2 sensitivity. Similarly, [START_REF] Asi | Private optimization in the interpolation regime: faster rates and hardness results[END_REF] reduced this sensitivity using a clipping-like procedure. In both cases, this decrease in L2 sensitivity introduces bias in the computation [START_REF] Amin | Bounding User Contributions: A Bias-Variance Trade-off in Differential Privacy[END_REF]. This phenomenon makes the analysis of clipped algorithms significantly harder [START_REF] Chen | Understanding Gradient Clipping in Private SGD: A Geometric Perspective[END_REF][START_REF] Yang | Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimization[END_REF][START_REF] Koloskova | Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees[END_REF], and it is difficult to choose a constant clipping threshold without tuning an additional hyperparameter. [START_REF] Pichapati | AdaCliP: Adaptive Clipping for Private SGD[END_REF] and Andrew et al. [2021] proposed heuristic methods for chosing clipping thresholds adaptively, although without theoretical guarantees and with limited practical applicability. Our method can reduce the amount of injected noise, while circumventing the difficulty of setting a proper clipping threshold. Indeed, our relative sensitivity assumption allows the design of a relative Gaussian mechanism where noise naturally adapts to the gradients' norms. Unlike clipped DP-GD, the convergence analysis of DP-GD using our mechanism is very similar to the one of DP-GD without clipping (see Section 5), while allowing to reduce noise injection. In problems that do not fit our relative sensitivity assumption, we also propose a "clipping-like" procedure, which enforces a bound on the relative sensitivity under mild statistical assumptions.

Relative L2 sensitivity

As discussed in the introduction, we start by relaxing the restrictive L2 sensitivity assumption. Definition 2 (Relative L2 sensitivity). An algorithm A satisfies Relative L2 sensitivity if there exists constants η > 0 and R rel > 0 such that for any two neighboring inputs x ∼ y:

∥R(x) -R(y)∥ 2 ≤ η 2 ∥R(x)∥ 2 + R 2 rel . (3)
Note that by symmetry, this is equivalent to ∥R(x) -R(y)∥ 2 ≤ η 2 min(∥R(x)∥ 2 , ∥R(y)∥ 2 ) + R 2 rel . Besides, we recover the standard L2 sensitivity for η = 0.

Examples. This definition is particularly useful when we know relative or multiplicative bounds on inputs. As discussed earlier, this can be the case if we would like to estimate salaries for a given job, and we know that all the people we consider have salaries within 10% of each other (for instance because it is imposed by the law). We would need to know salary estimates for each job to guess the appropriate absolute sensitivity R 2 abs (or use a very imprecise global one for all jobs), whereas knowing the law directly gives us η = 0.1 and R rel = 0.

In this case, the salaries are directly correlated to a latent variable: the jobs. This is also the case for gradients, whose norm depend on the point at which they are computed. The absolute L2 sensitivity would write ∥∇f (θ) -∇f ′ (θ)∥ 2 ≤ R abs (θ) 2 , where f and f ′ are objective functions computed on neighboring datasets. Therefore, we would either need to (i) know R abs (θ) for all values of θ, which is a lot of information, or (ii) bound it uniformly, which can be very loose. In contrast, Relative L2 sensitivity can ensure ∥∇f (θ) -∇f ′ (θ)∥ 2 ≤ η 2 ∥∇f (θ)∥ 2 + R 2 rel with tight absolute (independent of θ) parameters η and R rel , see Section 5 for more details.

Links to local sensitivity. As discussed in the related work section, the motivating idea behind local sensitivity [START_REF] Nissim | Smooth sensitivity and sampling in private data analysis[END_REF] is to set the noise according to the bound on the distance between the specific output we would like to protect and all neighboring ones. This allows much lower noise in general, since some outputs might have small sensitivity. Yet, this does not guarantee differential privacy as the level of noise injected gives information about the input that is released, as two neighboring inputs might have very different local sensitivities.

Note that Definition 2 actually bounds the local sensitivity, since the bound depends on the inputs that we consider. It is stronger however, as we can show that the variations of the local sensitivity induced by the relative sensitivity for two neighboring inputs are bounded. Said differently, due to its symmetry, relative L2 sensitivity also ensures that two neighboring inputs also have comparable norms, and so comparable local sensitivities. As we will see in the remainder of this paper, Definition 2 will allow privacy guarantees to hold even though the norm of the input is partly revealed through the noising process via the local sensitivity. In particular, we will show that Definition 2 can be leveraged to release information privately even when R rel ̸ = 0, and with no additional information. Our framework thus highlights an interesting example in which a form of local sensitivity can be used while still ensuring Differential Privacy.

4 The Relative Gaussian mechanism

Mechanism and privacy guarantees

We now present our central contribution, the Relative Gaussian Mechanism (RGM γ,σ ), and derive its privacy guarantees. RGM γ,σ extends GM σ to queries that satisfy relative sensitivity. It leverages relative sensitivity to guarantee privacy while adapting the scale of the noise to the norm of the query. Definition 3 (Relative Gaussian Mechanism). Let γ > 0 and σ > 0. The Relative Gaussian Mechanism of parameters (γ, σ) is defined as:

RGM γ,σ (R)(x) = R(x) + N(0, γ ∥R(x)∥ 2 + σ 2 ) . (4)
RGM γ,σ generalizes the standard GM σ , as we recover it by setting γ = 0. When γ > 0, it controls to which extent R(x)'s norm is used to set up noise. Note that σ 2 is a baseline noise, which allows to handle inputs where the query's output has small norm. For instance, if R(x) = 0 on some input x, and R(y) ̸ = 0 on an input y ∼ x, this baseline noise is necessary to guarantee privacy.

We show that, although RGM γ,σ uses the query output to calibrate the noise, it can still guarantee privacy. This perhaps surprising result follows from the relative sensitivity assumption. Intuitively, this assumption ensures that all neighboring outputs have comparable norms, resulting in comparable levels of noise. The next theorem formalizes this intuition, deriving tight privacy guarantees for RGM γ,σ on queries that satisfy a relative L2 sensitivity assumption.

Theorem 1 (Privacy guarantees of RGM γ,σ ). Let R : D → R d be a query that verifies (η, R rel )relative L2 sensitivity (Definition 2) for some η ≤ 1 and R rel ≥ 0. Then for

1 ≤ α < (1 + η) 2 /(2η + η 2 ), and σ 2 ≥ γη -2 [1 -η(α -1)] R 2 rel , RGM γ,σ (R) satisfies (α, ϵ)-Rényi-DP with ϵ = αη 2 2γ 1 1 -η(α -1)(2 + η) + γd(2 + η) 2 . ( 5
)
The proof is mostly technical, we thus defer it to Appendix A. Theorem 1 shows that RGM γ,σ can provide meaningful privacy guarantees. For a fixed γ, the guarantee is as strong as η 2 is small. This is in line with the intuition presented above: when η 2 is small, R(x) and R(y) (for x ∼ y) have similar norms, and these norms are less sensitive.

Scale of the noise. The scale of the noise is controlled by the parameter γ. Indeed, for a fixed γ, our result suggests to set the baseline variance as σ 2 = γη -2 (1 -η(α -1))R 2 rel . As such, small values of γ will lead to small noise addition (both in the baseline and the relative term), but will decrease privacy guarantees. Conversely, higher values of γ require more noise for better privacy guarantees.

Arbitrary privacy guarantees cannot be achieved. Although the parameter γ controls the level of the privacy guarantee, not all values of α and ϵ are achievable. This is in stark contrast with the classical GM σ (η = 0), where increasing the noise σ always improves privacy. This discrepancy is due to the fact that scaling noise with ∥R(x)∥ already releases some information about the input. Sadly, this information cannot be privatized using more baseline noise σ 2 without a priori bounds on ∥R(x)∥. Nonetheless, we emphasize that when η → 0, all values of α and ϵ are possible.

Theorem 1 implies that ϵ ≥ 2αη 2 d, where d is the dimension of the output of R. Consequently, RGM γ,σ is more likely to give good privacy guarantees on small-dimensional queries. Note that this is tight, as discussed below. To mitigate this issue, one can either (i) restrict the query to a subset of its coordinates, or (ii) adapt the query to decrease the value of η (see discussion in Section 5.3).

Conversion to (ϵ, δ)-DP. Using Proposition 3 of [START_REF] Mironov | Rényi differential privacy[END_REF], we can convert the RDP guarantee given in Theorem 1 to classical DP. For clarity of discussion, we give a closed-form expression of the differential privacy guarantees for the Relative Gaussian Mechanism in Corollary 1. We stress that better guarantees can be obtained by numerically optimizing the bound obtained from Proposition 3 of [START_REF] Mironov | Rényi differential privacy[END_REF], and provide a script to choose the best values of α and γ in the supplementary.

Corollary 1 (Conversion to (ϵ, δ)-DP). Let 0 ≤ δ ≤ 1. We assume that γ -1 ≥ 4(2 + η) 2 log(1/δ)
or that d ≥ 8 log(1/δ), and use the same notations as in Theorem 1. Then, RGM γ,σ satisfies (ϵ, δ)-differential privacy with parameter

ϵ = χ + 2 χ log(1/δ), where χ = η 2 γ + 1 2 η 2 (2 + η) 2 d.
We prove this result in Appendix A.5. While this result does not allow arbitrary privacy guarantee, we stress that meaningful guarantees can still be achieved. For instance, if η = 1e-3, d = 10, δ = 1e-8, and γ = 100η 2 , the Relative Gaussian mechanism guarantees (ϵ, δ)-DP with ϵ ≈ 0.86.

Privacy Loss and Comparison with the Gaussian Mechanism

Let us consider that we use the relative sensitivity as a local sensitivity to set the noise level for disclosing output A(x). In this case, guaranteeing (α, ε ⋆ )-Rényi-DP when releasing output A(x) requires setting the noise as

σ 2 abs = α 2ε⋆ (η 2 ∥R(x)∥ 2 + R 2 rel ).
Unfortunately, as explained before, local sensitivity does not guarantee differential privacy. If we were to use the same level of noise in the Relative Gaussian mechanism, this would correspond to γ = αη 2 2ε⋆ , and σ 2 = γη -2 R 2 rel . In particular, Theorem 1 tells us that this choice actually guarantees RDP with parameter:

ε = ε ⋆ 1 -η(α -1)(2 + η) + αd 2 η 2 (2 + η) 2 (6)
The first term corresponds to the target privacy level ε ⋆ , weighted by a factor which is bounded by 2 as long as 6η(α -1) ≤ 1, and goes to 1 as η decreases (for a fixed α). The second term corresponds to the privacy loss incurred by using the norm of the current output to set the noise level. Note that we see from Theorem 1 that this term is independent of γ and σ 2 : it corresponds to a baseline loss that is paid for using a local form of sensitivity. We would get rid of this term if all possible queries R(x) had the same norm, and this norm was public. However, this is a very strong assumption that generally does not hold (or requires very high absolute sensitivity bounds R 2 abs ). Note that it is tempting to use another output R(y) to set the noise level, and thus decorrelate the noise level from the specific input that we consider. However, R(y) would not be independent from R(x) since x ∼ y.

This privacy loss term explains why using arbitrary large γ does not lead to arbitrary good privacy guarantees. However, as long as η is small enough compared to α, the privacy loss is purely additive. This means that if the dimension d is not too large (d ≤ γ -1 /9, more for small η), we are safe using the relative Gaussian mechanism with minimal privacy overhead. Note that the d term comes from the fact that we use Gaussian noise, and other noise distributions might incur other dependencies.

Standard vs. Relative Gaussian Mechanism. This "privacy loss" point of view allows us to reason about the noise introduced by the Relative Gaussian Mechanism, versus the standard one. Indeed, let us neglect the additive privacy loss term. In this case, as argued in the previous paragraph, the privacy guarantees are comparable to the standard Gaussian mechanism with local sensitivity η 2 ∥R(x)∥ 2 + R 2 rel . In particular, which mechanism yields the best utility (less noise for a given privacy level) depends on which sensitivity bound is the tightest.

If R 2 abs ≥ η 2 max x ∥R(x)∥ 2 + R 2 rel
then the relative Gaussian mechanism is always better, because it will lead to similar guarantees with less noise overall. Otherwise, some outputs might be noised more with one mechanism and less with another. This is highly application-specific, as it is conditioned by the structure of the outputs.

Tightness. One natural question that arises is the tightness of Theorem 1. Due to the parallel with local sensitivity, the first term is tight up to the (usually small) multiplicative factor. The second term is also tight up to a factor 1/2 in the limit of small η, thanks to the tightness of the inequality used to obtain it. We discuss this in Appendix A.4.

The special case of gradient descent

An important application of RGM γ,σ is private gradient descent. In this section, we describe it in the quadratic case, for which we estimate the values of η and R rel and propose a clipping-like procedure.

Gradient descent under relative sensitivity assumptions

In this section, we consider a function f : R d × D → R, where D is a set of possible datasets.

Assume that the gradients of f (w.r.t. its first parameter) verify the relative sensitivity assumption. Given a dataset D ∈ D, we can then privately minimize this function using the following private gradient descent algorithm, where γ, σ > 0 are parameters of the RGM, and τ > 0 is a step size:

θ t+1 = θ t -τ RGM γ,σ (R θt )(D) where R θt (D) = ∇f (θ t ; D) . (7)
We remark that the form of RGM γ,σ 's noise allow a tight analysis of the utility, as shown below.

Theorem 2. Let f : R d × D → R be µ-strongly-convex and L-smooth in its first parameter (see, e.g., [START_REF] Nesterov | Lectures on convex optimization[END_REF]). Let D ∈ D be a dataset, and θ ⋆ be the minimizer of f (•; D). Assume that f 's gradients satisfy (η, R rel )-relative sensitivity, and that γ, σ are set as in Theorem 1. Then if τ ≤ (L + γ) -1 , the iterates obtained by (7) satisfy, for all t ≥ 0,

E ∥θ t -θ ⋆ ∥ 2 ≤ (1 -τ µ) t ∥θ 0 -θ ⋆ ∥ 2 + τ σ 2 µ . (8)
The proof, along with a similar result in the general convex case are in Appendix B.1. The key observation is that the progress towards θ ⋆ is proportional to ∥∇f (θ t ; D)∥ 2 , so it can compensate the norm-scaled noise term, and only requires slightly decreasing the step size (we recall that γ is typically small). Contrary to the usual DP-GD, which privatizes gradients using GM σ , the variance term is τ σ 2 µ , where σ 2 now depends on R rel which can be much smaller than the absolute sensitivity. In the remainder of this section, we exhibit settings in which the gradients verify relative sensitivity.

Relative L2 sensitivity for linear regression

We now consider the specific case of quadratic objectives. More specifically, f is of the form

f (θ; X, y) = 1 2n X ⊤ θ -y 2 + µ reg 2 ∥θ∥ 2 = 1 n n i=1 1 2 X ⊤ i θ -y i 2 + µ reg 2 ∥θ∥ 2 , (9)
where X ∈ R d×n and y ∈ R n . We denote by (X i , y i ) ∈ R d × R the i-th data record (i.e., the i-th column of X and i-th element of y). Let (X ′ , y ′ ) ∼ (X, y) be a dataset that, w.l.o.g, only differs from (X, y) on its first record (X 0 , y 0 ). In the following, we denote

f = f (•; D) and f ′ = f (•; D ′ ).
Let I d ∈ R d×d be the identity matrix and let us denote

A = 1 n XX ⊤ + µ reg I d ∈ R d×d , A i = X i X ⊤ i ∈ R d×d , b = 1 n Xy ∈ R d , and b i = 1 n X i y i ∈ R d (and similarly for A ′ and b ′ ). Then for θ ∈ R d , ∇f (θ) = Aθ -b and ∇f ′ (θ) = A ′ θ -b ′ . The difference between two gradients is ∥∇f (θ)-∇f ′ (θ)∥ 2 = 1 n 2 ∥(A 0 -A ′ 0 )θ -b 0 + b ′ 0 ∥ 2 = 1 n 2 (A 0 -A ′ 0 )A -1 (Aθ -b) + (A 0 -A ′ 0 )A -1 b -b 0 + b ′ 0 2 ≤ 3 n 2 A 0 A -1 2 + A ′ 0 A -1 2 ∥∇f (θ)∥ 2 + 3 n 2 ∇f 0 (A -1 b) -∇f ′ 0 (A -1 b) 2 ,
with ∥A∥ = λ max (A) being the 2-norm for matrices. This bound hints at relative sensitivity, and we now discuss the corresponding η and R rel terms. We first define L, µ > 0 the bounds on the largest and smallest eigenvalues of all A, i.e., L ≥ ∥A∥, and µ ≤ λ min (A). Then, denote κ = L/µ.

The relative term η. The first term that we notice is ∥A 0 A -1 ∥, which can be naively bounded as

∥A 0 A -1 ∥ 2 ≤ max i ∥A i ∥ 2 /µ 2 .
However, this term can also be bounded as:

A 0 A -1 2 = ∥X 0 ∥ 2 X ⊤ 0 A -2 X 0 ≤ κ X ⊤ 0 A -1 X 0 2 . ( 10
)
Therefore, it suffices to have X ⊤ 0 A -1 X 0 ≤ L rel for some L rel (and same for X ′ 0 ), which corresponds to point X 0 belonging to the ellipse defined by A -1 and of radius L rel . We remark that in most cases, L rel does not depend on the conditioning of A. Thus, A can be highly ill-conditioned, while L rel remains small. Depending on the distribution of X 0 , ∥X 0 ∥ 2 X ⊤ 0 A -2 X 0 can also be bounded directly in a tighter way. We now discuss two examples in which we can reasonably control ∥A 0 A -1 ∥ 2 in a tight way, accentuating the relevance of this relative bound.

1 -Orthogonal data. Relative sensitivity can be easily bounded for orthogonal data, i.e. if either

X ⊤ i X j = ∥X i ∥ 2 or X ⊤ i X j = 0.
Consider that at least half of the dataset is fixed, and contains all different X i in equal proportions (so, d -1 ). In this case,

A ≽ 1 2d d i=1 X i X ⊤ i so ∥X i ∥ 2 X ⊤ i A -2 X i ≤ 2d.
Note that the relative sensitivity is independent of the scale of each X i . See Appendix B.3 for more detailed derivations.

2 -Gaussian data. If the data is Gaussian, then with sufficient regularization X ⊤ 0 A -1 X 0 ≤ L rel with L rel independent of the covariance of the data.

Proposition 1. If the columns of X are drawn i.i.d from N(0, Σ), and we set the regularization as µ reg = c ∥Σ∥ 2 [d eff + ln(d) + ln(2δ -1 )]/n then relative sensitivity is satisfied with η 2 = c ′ κ(d/n) 2 log(2n/δ) 2 with probability at least 1 -δ, where c, c ′ > 0 are small absolute constants, κ is the condition number of Σ and d eff ≤ d is the effective dimension of Σ.

The proof and details can be found in Appendix B.4. In this case, δ corresponds to a catastrophic failure mode: the assumptions are not verified, and so the privacy guarantees do not hold.

The absolute term R rel . By using the relative framework, we have gone from having to bound the difference between gradients at all points to only having to bound it at A -1 b, the rest being handled by the norm scaling. When an approximation of A -1 b is known, this gives much tighter guarantees. Otherwise, this term writes:

(A 0 -A ′ 0 )A -1 b -b 0 + b ′ 0 ≤ (A 0 -A ′ 0 )A -1 ∥b∥ + ∥b 0 -b ′ 0 ∥.
In the end, one only needs to control the norm of b and b 0 -b ′ 0 , which can be done via clipping.

General functions. All derivations above consider quadratic objectives. Yet, similar terms with corresponding intuitions can be derived for arbitrary convex functions, as presented in Appendix B.2.

Enforcing relative L2 sensitivity

In practical applications that do not verify relative sensitivity per se, we may want to enforce it, like clipping does for absolute sensitivity. In the quadratic case, the absolute term R rel can be controlled by clipping the b i 's. Yet, we should also enforce that (A 0 -A ′ 0 )A -1 ≤ η, which could be done by writing A ≽ µ reg I d , then clipping the A i 's so that ∥A i ∥ ≤ c A . In the end, we would obtain ∥A 0 A -1 ∥ = c A /µ reg . However, this bound can be quite loose, since it does not leverage relative assumptions between A 0 and A -1 . The following generic assumption can give a tighter bound.

Assumption 1. There is a matrix C ∈ R d×d , a threshold R c , and a regularization µ reg such that for any dataset X ∈ R d×n , there is a set of points

I C such that I C ⊂ {i, ∥X i ∥ 2 X ⊤ i C -2 X i ≤ R 4 c }, |I C | ≥ ωn for some ω > 0, and A C = 1 |I C | i∈I C X i X ⊤ i + µ reg I d ≽ ρC for some ρ > 0.
This assumption states that we know a matrix C, a threshold R c and a regularization µ reg such that, regardless of the specific dataset instance, if we "clip" the points that are not in I C (e.g., drop them, or reduce their norm them to meet the condition to be in I C ): (i) a constant fraction of points will not be clipped, (ii) the regularized covariance of the non-clipped points is lower bounded by ρC, for some ρ > 0 that is independent of the dataset. Intuitively, this means that although we do not know the specific points in the dataset, we know a bound on their covariance. Interestingly, using any C ̸ = I d in Assumption 1 helps obtain better values of η as long as ρ/ ∥C∥ > µ reg . Thus, even loose estimates of the covariance can be used: combining this assumption with RGM is therefore a good way to leverage expert knowledge on the matrix C to reduce noise. In the absence of such knowledge, C could also be estimated using a public subset of the X i 's. Another implication of Assumption 1 is that relative sensitivity can be enforced by a clipping-like procedure, that we describe below.

Proposition 2 (Clipping). Let Assumption 1 hold, with (C, R c ) the corresponding matrix and threshold. Let X be the clipped dataset, obtained as

Xi = R c X i / max(R c , (∥X i ∥ 2 X ⊤ i C -2 X i ) 1 4
). Then, X verifies the relative sensitivity assumption with constant η =

6R 4 c ω 2 ρ 2 n 2 .
The proof can be found in Appendix B.5. We refer to this procedure as "clipping" since relative sensitivity is enforced by shrinking the norm of the X i when they are too large. Note that the privacy guarantees in this case are quite underestimated: we consider that points that are clipped are just discarded and put to 0, whereas in reality they also contribute to the covariance. Actually discarding these points would allow to get rid of the ω 2 factor at the cost of more bias in the dataset. Importantly, the clipping only depends on C, and is independent of the other points in the dataset. This is crucial as it preserves the property that datasets that only differ in one sample still do after clipping (which would not be the case if clipping using A -1 ).

Similarly to the bound on η, we show in Proposition 3 that an alternative formulation of Assumption 1 is verified for Gaussian features where C is the covariance of the underlying Gaussian distribution. This means that we expect this kind of clipping to work well when the data "looks Gaussian". Proposition 3. Let X ∈ R d×n such that its columns X i are drawn i.i.d. from N(0, Σ). Let I C be such that

I C ⊂ {i, X ⊤ i Σ -1 X i ≤ R 2 c }.
Let δ > 0 and n ≥ 4 log(2d/δ)/9. Then, with probability at least 1 -3δ, |I C | ≥ ωn with ω = p(χ 2 (d) ≤ R 2 c ) -log(δ -1 )/2n, and

A C = 1 |I C | i∈I C X i X ⊤ i + µ reg I d ≽ ρC
for some ρ > 0 that only depends on R c and regularization

µ reg = 4 ∥Σ∥ R 2 c log(2d/δ) n
. In particular, dropping the i / ∈ I c leads to η =

6R 4 c κ ρ 2 n 2 .
The proof can be found in Appendix B.6. Note that it is generally simpler to check the condition

X ⊤ i C -1 X i ≤ R 2 c rather than ∥X i ∥ 2 X ⊤ i C -2 X i ≤ R 4 c
, as is for instance the case for Gaussian data. This is why Proposition 3 uses the first condition instead of checking Assumption 1, but arrives to a result comparable to Proposition 2 by leveraging (10).

We can then compare the result with that of Proposition 1, by setting

R 2 c = O(d) (since E χ 2 (d) = d).
We obtain that if we know the covariance Σ of the Gaussian distribution from which the data is sampled, then we can perform clipping and replace the O(log(n/δ)) term by constant terms. This is because, thanks to clipping, we do not have to increase L rel to include potential outliers.

Experiments: distributed training under local DP

We have presented several ways of estimating or enforcing the relative sensitivity parameters in the previous section. Yet, they require inverting an approximation of the covariance of the data, which may be computationally expensive. Nonetheless, an interesting use-case for the Relative Gaussian Mechanism is distributed training in the local model of differential privacy. Several nodes participate in a global training procedure, minimizing a shared objective. To this end, they periodically exchange (private) gradients, and the key bottleneck is thus communication. We remark that privatizing gradients is a local procedure, that each node completes on its own. In particular, it is reasonable to assume here that nodes can locally and efficiently estimate (η, R rel ) without having to solve the global optimization problem (which would require communication).

We argue that, contrary to the RGM γ,σ , it is impossible to effectively use the GM σ without knowledge from other nodes. To illustrate this, consider the simple example where two nodes have respective objectives f 1 (θ) = α ∥θ∥ 2 , and f 2 (θ) = β ∥θ -b∥ 2 . There, the sensitive records to keep private are α ∈ [α min , α max ] and β ∈ [β min , β max ]. Both nodes can easily compute their local parameters η 1 = α min /α max , η 2 = β min /β max , and R rel = 0. Then, they can directly use the RGM γ,σ . Consider now that nodes use GM σ with gradient clipping instead, and that for this particular instance, α = β = 1. In order not to bias the objective, the clipping threshold for node 1 would need to be set to a least ∥b∥, which is equal to the norm of the gradient of f 1 evaluated at the global optimum. However, node 1 has no knowledge of b, and has thus no way of setting a relevant clipping threshold without exchanging information with node 2. In this simple illustrative example, it would of course be enough to just exchange an approximation of b, which has a reasonable cost. Nonetheless, this highlights that setting a relevant clipping threshold in general requires knowledge of the solution to the global problem, which is generally unavailable as it depends on all nodes' data.

We illustrate this with linear regression experiments on the ijcnn1 dataset [START_REF] Chang | Ijcnn 2001 challenge: Generalization ability and text decoding[END_REF] (concatenation of train and test from LibSVM repository1 ), so the total N = 141691, and d = 22. We consider ridge linear regression, so f is of the form of ( 9) where the y i correspond to the binary classification labels. We set the regularization parameter µ reg = 0.03, and RDP parameters α = 2 and ε = 0.1. In order to avoid having to decide a clipping threshold for GM σ , we automatically set the threshold as the maximum of the individual stochastic gradients at optimum (to avoid bias). We also run experiments with c high = 10c ('Clip high') and c low = c/10 ('Clip low'). We compare this to vanilla gradient descent without noise and RGM γ,σ , where the η parameter is approximated using (10). The results are shown in Figure 1. Code is available in supplementary material, and the precise experimental details can be found in Appendix C.

We study 3 different data splits: (i) 'Random' (left plot): the data is split randomly across the two nodes. (ii) 'label' heterogeneity (center): we sample 50 points at random for each node, and then all positive labels are assigned to one node and all negative to the other. (iii) 'bias' (right): we add a bias B to the objective to recreate (with more complex data) the simple example discussed above.

We observe that, although there is generally always a clipping threshold that works well, this threshold is problem-dependent. Small thresholds work well for homogeneous objectives or with label heterogeneity, whereas larger clipping thresholds handle bias heterogeneity better. Therefore, the clipping threshold needs to be tuned, which requires more communication, and incurs additional privacy leaks. On the contrary, the Relative Gaussian mechanism is, in this case, able to deal with heterogeneity without such tuning.

Conclusion

We introduced the relative L2 sensitivity, a generalization of the usual L2 sensitivity which depends on the norm of the query. We designed the Relative Gaussian mechanism (RGM γ,σ ), a mechanism that exploits this sensitivity assumption to adapt the level of noise to the norm of the query R(x), and proved tight privacy guarantees. We then applied RGM γ,σ to private gradient descent and proposed a clipping-like procedure to enforce the relative L2 sensitivity under some statistical assumptions. An exciting and challenging direction is to generalize the relative L2 assumption and RGM γ,σ to sub-sampled queries, and apply them to private stochastic gradient descent. We also note that relative L2 sensitivity could be further refined by measuring it in other norms, possibly in combination with the use of other noise distributions.

Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, et al. Opacus: User-friendly differential privacy library in pytorch. arXiv preprint arXiv:2109.12298, 2021.

A Proofs for the Relative Gaussian Mechanism

A.1 Bounding the noise scale ratio

We start by the following simple lemma, that will allow us to bound the domain of admissible α.

Lemma 1. If σ 2 ≥ γ(1+η -1 ) 2η+η 2 R 2 rel , then (1 + η) -2 ≤ γ∥R(x)∥ 2 +σ 2 γ∥R(y)∥ 2 +σ 2 ≤ (1 + η) 2 .
Proof.

γ ∥R(x)∥ 2 + σ 2 γ ∥R(y)∥ 2 + σ 2 = γ ∥R(y) + R(x) -R(y)∥ 2 + σ 2 γ ∥R(y)∥ 2 + σ 2 ≤ γ(1 + η) ∥R(y)∥ 2 + γ(1 + η -1 ) ∥R(x) -R(y)∥ 2 + σ 2 γ ∥R(y)∥ 2 + σ 2 ≤ γ(1 + η) ∥R(y)∥ 2 + γ(1 + η -1 )(η 2 ∥R(y)∥ 2 + R 2 rel ) + σ 2 γ ∥R(y)∥ 2 + σ 2 .
The result then follows from using the bound on σ 2 to factor γ ∥R(y)∥ 2 + σ 2 in the numerator. The other side (lower bound) is obtained by inverting R(x) and R(y).

A.2 Rényi divergence of two Gaussians

Recall that for α > 1 and two distributions, P and Q, the Rényi divergence is

D α (P ||Q) = 1 α -1 log P (x) α Q(x) α dQ(x) = 1 α -1 log P (x) α Q(x) α-1 dx .
Lemma 2. Let P and Q be Gaussian distributions of dimension d centered in µ 1 and µ 2 with variance σ 2 1 and σ 2 2 . Then, assuming that ασ

2 2 + (1 -α)σ 2 1 > 0, D α (P ||Q) = α ∥µ 1 -µ 2 ∥ 2 2(ασ 2 2 + (1 -α)σ 2 1 ) + d α -1 log σ 1-α 1 σ α 2 ασ 2 2 + (1 -α)σ 2 1 . (11)
Remark that when σ 1 = σ 2 , we recover the divergence of the standard Gaussian mechanism.

Proof.

D α (P ||Q) = 1 α -1 log σ 2d(α-1) 2 (2π) d σ 2dα 1 exp - α ∥u -µ 1 ∥ 2 2σ 2 1 - (1 -α) ∥u -µ 2 ∥ 2 2σ 2 2 du .
We first compute the one-dimensional integral

+∞ -∞ exp - α(u -µ 1 ) 2 2σ 2 1 - (1 -α)(u -µ 2 ) 2 2σ 2 2 = +∞ -∞ exp - α 2σ 2 1 + 1 -α 2σ 2 2 u 2 + αµ 1 σ 2 1 + (1 -α)µ 2 σ 2 2 u - αµ 2 1 2σ 2 1 - (1 -α)µ 2 2 2σ 2 2 = +∞ -∞ exp - ασ 2 2 + (1 -α)σ 2 1 2σ 2 1 σ 2 2 u 2 + αµ 1 σ 2 2 + (1 -α)µ 2 σ 2 1 σ 2 1 σ 2 2 u - αµ 2 1 σ 2 2 + (1 -α)µ 2 2 σ 2 1 2σ 2 1 σ 2 2 Now, since +∞ -∞ exp(-(au 2 + bu + c))du = π a exp b 2 4a
-c , we have after simplification, and

assuming ασ 2 2 + (1 -α)σ 2 1 > 0, +∞ -∞ exp - α(u -µ 1 ) 2 2σ 2 1 - (1 -α)(u -µ 2 ) 2 2σ 2 2 = 2πσ 2 1 σ 2 2 ασ 2 2 + (1 -α)σ 2 1 exp - α(1 -α)(µ 1 -µ 2 ) 2 2(ασ 2 2 + (1 -α)σ 2 1 = √ 2πσ 1 σ 2 ασ 2 2 + (1 -α)σ 2 1 exp - α(1 -α)(µ 1 -µ 2 ) 2 2(ασ 2 2 + (1 -α)σ 2 1 .
Back to our divergence, we obtain

D α (P ||Q) = 1 α -1 log   σ 2d(α-1) 2 (2π) d σ 2dα 1 d j=1 √ 2πσ 1 σ 2 ασ 2 2 + (1 -α)σ 2 1 exp - α(1 -α)(µ 1 -µ 2 ) 2 2(ασ 2 2 + (1 -α)σ 2 1 )   = 1 α -1 log   d j=1 σ 1-α 1 σ α 2 ασ 2 2 + (1 -α)σ 2 1 exp - α(1 -α)(µ 1,j -µ 2,j ) 2 2(ασ 2 2 + (1 -α)σ 2 1 )   = 1 α -1 log   σ 1-α 1 σ α 2 ασ 2 2 + (1 -α)σ 2 1 d exp - α(1 -α) ∥µ 1 -µ 2 ∥ 2 2(ασ 2 2 + (1 -α)σ 2 1 )   = d α -1 log σ 1-α 1 σ α 2 ασ 2 2 + (1 -α)σ 2 1 + α ∥µ 1 -µ 2 ∥ 2 2(ασ 2 2 + (1 -α)σ 2 1 )
.

A.3 Privacy guarantees (Theorem 1)

We would now like to apply Lemma 2 where

µ 1 = R(x), σ 2 1 = γ ∥R(x)∥ 2 + σ 2 , µ 2 = R(y) and σ 2 2 = γ ∥R(y)∥ 2 + σ 2 .
Verifying the condition on α. To this end, we first need to verify that ασ 2 2 + (1 -α)σ 2 1 > 0, or equivalently that σ 2 2 /σ 2 1 ≥ 1 -α -1 . If applicable, Lemma 1 would directly give us that this is true as long as (1 + η) -2 ≥ 1 -α -1 , which leads to the bound:

α -1 ≥ 1 -(1 + η) -2 = η(2 + η) (1 + η) 2 , (12)
so in the end:

α ≤ (1 + η) 2 η(2 + η) . ( 13
) This is equivalent to α -1 ≤ 1 η(2+η)
, which is the condition from Theorem 1. In order to apply Lemma 1, we need to verify that

σ 2 ≥ γ 1 + η -1 2η + η 2 R 2 rel = γR 2 rel η 2 1 + η 2 + η . ( 14
)
This is automatically verified for the choice of σ from Theorem 1 since

1 -η(α -1) ≥ 1 -(2 + η) -1 = 1 + η 2 + η . ( 15
)
Bounding the main term in (11). Now that we verified that we can apply Lemma 2, we can use it to bound the divergence. To bound the first term in (11), we start by recalling that, by Young's inequality,

∥R(x)∥ 2 2 ≤ (1 + η) ∥R(y)∥ 2 2 + (1 + η -1 ) ∥R(x) -R(y)∥ 2 2 . ( 16
)
14

Using this inequality, we can lower bound the denominator

ασ 2 2 + (1 -α)σ 2 1 (17) = σ 2 + γα ∥R(x)∥ 2 2 -γ(α -1) ∥R(y)∥ 2 2 (18) ≥ σ 2 + γα ∥R(y)∥ 2 2 -γ(α -1)(1 + η) ∥R(y)∥ 2 2 -γ(α -1)(1 + η -1 ) ∥R(x) -R(y)∥ 2 2 (19) = σ 2 + (γα -γ(α -1)(1 + η) ∥R(y)∥ 2 2 -γ(α -1)(1 + η -1 ) ∥R(x) -R(y)∥ 2 2 (20) = σ 2 + γ(1 -(α -1)η) ∥R(y)∥ 2 2 -γ(α -1)(1 + η -1 ) ∥R(x) -R(y)∥ 2 2 (21)
Now, remark that relative sensitivity gives ∥R(y)∥

2 2 ≥ 1 η 2 ∥R(x) -R(y)∥ 2 2 -R 2 , which in turn yields ασ 2 2 + (1 -α)σ 2 1 (22) ≥ σ 2 + γ η 2 (1 -(α -1)η) ∥R(x) -R(y)∥ 2 2 -R 2 -γ(α -1)(1 + η -1 ) ∥R(x) -R(y)∥ 2 2 (23) = σ 2 - γ η 2 (1 -(α -1)η)R 2 + γ η 2 (1 -(α -1)η) -γ(α -1)(1 + η -1 ) ∥R(x) -R(y)∥ 2 2 . ( 24
)
We now use the condition on α, which is that α -1 = 1-ρ η(2+η) for some ρ < 1. If we further assume that σ 2 ≥ γ η 2 (1 -(α -1)η)R 2 , which is notably the case when σ 2 ≥ γR 2 η 2 , we obtain

ασ 2 2 + (1 -α)σ 2 1 ≥ γ η 2 (1 -(α -1)η) -γ(α -1)(1 + η -1 ) ∥R(x) -R(y)∥ 2 2 (25) = γ η 2 - γ η (α -1) (2 + η) ∥R(x) -R(y)∥ 2 2 (26) = γρ η 2 ∥R(x) -R(y)∥ 2 2 . ( 27
)
Putting everything back together, we get that for η(2+η)(α-1) ≤ 1, and

σ 2 ≥ γ(1-(α-1)η)R 2 /η 2 : α ∥R(x) -R(y)∥ 2 2 2(ασ 2 2 + (1 -α)σ 2 1 ) ≤ αη 2 2γρ = αη 2 2γ 1 1 -η(2 + η)(α -1) . ( 28
)
Bounding the log term in (11). Remark that, as a consequence of Lemma 1, we have that σ 2 2 = rσ 2 1 , for some r ≤ (1 + η) 2 . In particular, we have

log σ 1-α 1 σ α 2 ασ 2 2 + (1 -α)σ 2 1 = log r α/2 σ 1 √ 1 -α + αrσ 1 = α -1 2 log(r) - 1 2 log 1 + (r -1)α r . (29)
The two terms can be upper bounded using log(1 + x) ≤ x for x ≥ -1,

α -1 2 log(r) ≤ α -1 2 (r -1) , (30)
and log(1

+ x) ≥ x 1+x for x ≥ -1, - 1 2 log 1 + (r -1)α r = - 1 2 log 1 + (α -1)(r -1) r (31) ≤ - 1 2 (α -1)(r -1) r + (α -1)(r -1) (32) = - 1 2 (α -1)(r -1) 1 + α(r -1) . ( 33
)
Overall, we obtain

log σ 1-α 1 σ α 2 ασ 2 2 + (1 -α)σ 2 1 ≤ (α -1)(r -1) 2 1 - 1 1 + α(r -1) = α(α -1)(r -1) 2 2 , (34)
Since (1 + η) -2 ≤ r ≤ (1 + η) 2 , we have that

|r -1| ≤ max (1 + η) 2 -1, 1 -(1 + η) -2 = max η(2 + η), η(2 + η) (1 + η) 2 ≤ η(2 + η), (35)
so that in the end:

d α -1 log σ 1-α 1 σ α 2 ασ 2 2 + (1 -α)σ 2 1 ≤ αdη 2 (2 + η) 2 2 . ( 36
)

A.4 Tightness

As explained in the main text, the link with local sensitivity implies the tightness of the first term in Theorem 1 (the one which depends on γ). We now discuss the tightness of the second term, which comes from the log term in (11). In particular, we write (similarly to the previous section):

log σ 1-α 1 σ α 2 ασ 2 2 + (1 -α)σ 2 1 = α 2 log(1 + u) - 1 2 log (1 + αu) = g(u) , (37)
with u = r -1. We would like to find an expression for g(u) when u ≈ 0, which means that r ≈ 1 and so η is small. Differentiating with respect to u leads to:

g ′ (u) = α 2 1 1 + u - 1 1 + αu . (38)
Note that g(0) = 0, and g ′ (0) = 0, so we have to differentiate once again, leading to:

g ′′ (u) = α 2 α (1 + αu) 2 - 1 (1 + u) 2 . ( 39
)
In particular, g ′′ (0) = α(α-1)

2

, so when r ≈ 1,

g(r) = α(α -1) 4 (r -1) 2 + O((r -1) 3 ). ( 40
)
The leading term is the same expression as we had before, up to a factor 1/2. In particular, the bounding from the previous subsection is tight up to a factor 1/2.

Ideally, we would like to use this approximation as g(r). In order to do this, we have to show that g ′′′ (r -1) ≤ 0 for all (α, η) we consider. Let us differentiate one last time, leading to:

g ′′′ (u) = α 1 (1 + u) 3 - α 2 (1 + αu) 3 . ( 41
)
One can remark that g ′′′ (u) ≤ 0 for u ≤ 0. However, it can be that g ′′′ (u) > 0 for some u > 0.

More specifically, if α is large enough (α ≥ 2 for instance), then one can show that g ′′′ (u) ≤ 0 for the range of u that we consider (i.e., u = r -1 ≤ (1 + η) 2 -1 = η(2 + η) ≤ (α -1) -1 . Yet, this does not hold for all α, and g ′′′ ((α -1) -1 ) > 0 for α = 3/2 for instance. This is why we keep the result which is off by a factor up to 2 for large α, but works for any value of (α, η) in our range.

A.5 Comparison with differential privacy

Corollary 1. Assuming that γ ≤ 1 4(2+η) 2 log(1/δ) or d ≥ 8 log(1/δ), and that 0 < δ ≤ 1, the Relative Gaussian Mechanism is (ϵ, δ)-DP with

ϵ = χ + 2 χ log(1/δ), where χ = η 2 γ + 1 2 η 2 d(2 + η) 2 . ( 42
)
Proof. Applying the standard conversion result from RDP to (ϵ, δ)-DP, we get that the Relative Gaussian Mechanism is (ϵ, δ)-DP for

ϵ = min 1≤α≤ (1+η) 2 2η+η 2 g(α) := αη 2 2γ(1 -η(2 + η)(α -1)) + αη 2 2 d(2 + η) 2 + log(1/δ) α -1 . ( 43
)
Restricting to α ≤ 1 + 1 2η(2+η) ≤ (1+η) 2 η(2+η) , we have η(2 + η)(α -1) ≤ 1/2. We can therefore upper bound g(α) by

g(α) ≤ αη 2 γ + αη 2 2 d(2 + η) 2 + log(1/δ) α -1 = αχ + log(1/δ) α -1 . ( 44
)
This upper bound is minimal when

χ = log(1/δ) (α-1) 2 , that is α = 1 + log(1/δ) χ
. Using this value of α, we have

ϵ ≤ g(α) ≤ χ + χ log(1/δ) + χ log(1/δ) ≤ χ + 2 √ χ log(1/δ). (45)
Note that we could choose this value of α since either γ ≤ 1 4(2+η) 2 log(1/δ) or d ≥ 8 log(1/δ). These inequalities indeed implies that α = 1 + log(1/δ)

χ ≤ 1 + 1 2η(2+η) , which is equivalent to χ 4η 2 (2 + η) 2 = 1 4γ(2 + η) 2 + 1 8 d ≥ log(1/δ). (46)
B Relative L2 sensitivity for releasing gradients Before we start this section, we introduce a few notions, that will be useful in particular in subsections B.1 and B.2. The first is the notion of Bregman Divergence, which is defined for a function h and for points θ, θ ′ as:

D h (θ, θ ′ ) = h(θ) -h(θ ′ ) -∇h(θ ′ ) ⊤ (θ -θ ′ ). (47)
One can see that h is L-smooth and µ-strongly convex is equivalent to having for all θ, θ ′ ∈ R d :

µ 2 ∥θ -θ ′ ∥ 2 ≤ D h (θ, θ ′ ) ≤ L 2 ∥θ -θ ′ ∥ 2 . ( 48
)
The Bregman divergence also has nice properties w.r.t. convex conjugation. More specifically, the convex conjugate h * of h is defined as h * (θ) = arg max u θ ⊤ u -f (u). Then, it holds that:

D h (θ, θ ′ ) = D h * (∇h(θ ′ ), ∇h(θ)). (49)
Bregman divergences are often linked in convex optimization to the notion of relative smoothness, which generalizes regular smoothness. In particular, a function f is said to be L rel -smooth with respect to

h if for all θ, θ ′ ∈ R d , D f (θ, θ ′ ) ≤ L rel D h (θ, θ ′ ). (50)
This recover standard smoothness when h = 1 2 ∥•∥ 2 , and will play a key role in showing our relative sensitivity assumption for gradient descent with general functions.

B.1 Utility (Proof of Theorem 2)

We provide below the proof of Theorem 2. We also prove that under the same assumptions (but this result can also be used for µ = 0, i.e. f (•; D) is convex), we have that:

f ( θt ) -f (θ ⋆ ) ≤ ∥θ 0 -θ ⋆ ∥ 2 2τ t + τ σ 2 2 , (51)
where θt = 1 t t-1 k=0 θ k .

Consider that at least half of the dataset is fixed, and contains all different X i in equal proportions. This means that the weight of each X i is d -1 , since there are exactly d different ones. Indeed, there cannot be more than d (otherwise the orthogonality constraint would be violated), and if there are less than d we can just restrict to the relevant subspace. In this case, using that half of the data (w.l.o.g. is fixed, we have that:

A = 1 n n i=1 X i X ⊤ i ≽ 1 n n/2 i=1 X i X ⊤ i ≽ 1 2d d i=1 X i X ⊤ i ,
where the last line comes from the fact that each X i is represented n/d times, and we implicitly assume (again, w.l.o.g.) that the i first samples are all orthogonal to one another. In particular, for j ∈ {1, . . . , d},

∥X i ∥ 2 X ⊤ i A -2 X i ≤ 2d ∥X i ∥ 2 X ⊤ i   d j=1 X j X ⊤ j   -2 X i = 2d ∥X i ∥ 2 X ⊤ i X i X ⊤ i ∥X i ∥ 4 2 X i = 2d.
Note in particular that the relative sensitivity is independent of the scale of each X i .

B.4 Proof of Proposition 1

Proof. We use for the proof that A 0 A -1 ≤ √ κX ⊤ 0 A -1 X 0 . However, A is a random variable, with potentially unbounded condition number. Thus, we first need to use concentration on A to obtain the result. We could alternatively directly bound ∥X 0 ∥ 2 X ⊤ 0 A -2 X 0 , which might lead to a tighter bound (but only up to constants in the worst-case), but would require a specialized concentration result. Let L, µ > 0 be the largest and smallest eigenvalue of Σ. Let us define à = A -µ reg I d . We start by conditioning on the fact that n is large enough that A concentrates well around its mean:

p(∃i, A i A -1 > L rel ) ≤ p(∃i s.t. A i A -1 > L rel , ∥ Ã -Σ∥ ≤ ∆) + p(∥ Ã -Σ∥ > ∆). (58)
We first bound the second term, for which we apply Even and Massoulié [2021, Theorem 3], which states that:

p(∥ Ã -Σ∥ > ∆) ≤ δ 2 for ∆ = C 2 L 2 d eff + ln(d) + ln(2δ -1 ) n , (59)
where C 2 is an absolute constant and d eff = Tr(Σ)/L ≤ d is the effective dimension of Σ.

Let us now bound the first term. We set the regularization µ reg = ∆ = C 2 L 2 d eff +ln(d)+ln(2δ -1 ) n .

Since ∥ à -Σ∥ ≤ ∆ implies that à ≽ Σ -∆I d , we have that under this condition:

A = Ã + µ reg I d ≽ Σ. (60)
In particular, we obtain:

p(∃i s.t. A i A -1 > L rel , ∥ Ã -Σ∥ ≤ ∆) ≤ p(∃i s.t. X ⊤ i A -1 X i > √ κL rel , ∥ Ã -Σ∥ ≤ ∆) ≤ p(∃i s.t. X ⊤ i Σ -1 X i > √ κL rel , ∥ Ã -Σ∥ ≤ ∆) ≤ p(∃i s.t. X ⊤ i Σ -1 X i > √ κL rel ) ≤ np(X ⊤ 0 Σ -1 X 0 > √ κL rel ).
where the last inequality follows from a union bound. Now, notice that if

X 0 ∼ N(0, Σ) then p(X ⊤ 0 Σ -1 X 0 > L rel ) = p(χ 2 (d) > L rel ) ≤ 1 -F (kL rel , d) ≤ L rel d e 1-L rel d d 2 , (61)
where F (•, d) is the cumulative distribution function of the χ 2 (d) distribution. One can then show that L rel /d ≤ e

2L rel 3d

for all d, and so we have that p(X ⊤ 0 Σ -1 X 0 > L rel ) ≤ e d exp(-L rel 6 ), and so in particular, choosing the right value for L rel leads such that the probability is small enough leads to:

p(X ⊤ 0 Σ -1 X 0 > L rel ) ≤ δ 2n for L rel = 6d log 2n δ . (62)

B.5 Proof of Proposition 2

Proof. The first thing we need to argue is that we can indeed use (10) for clipping, which requires that if we have two neighboring datasets D and D ′ , then ∇f (θ; D) -∇f (θ; D ′ ) = Ã0 θ -b0 -Ã′ 0 θ + b′ 0 , where the tilde indicates the clipped dataset. This is true because C is independent of D or D ′ , and so in particular all X i are clipped in the same way whether they beyond to D or D ′ , and so all indices but 0 cancel when taking the gradients' difference. Now, let à = X X⊤ be the covariance of the clipped dataset. Then,

à = 1 n i Xi X⊤ i ≽ 1 n i∈I C Xi X⊤ i ≽ ω |I C | i∈I C X i X ⊤ i = ωA C ≽ ωρC. (63)
In particular, all points in the clipped dataset verify: 1 -Lower bound on ω. The indicator of the event {i ∈ I c } is a Bernoulli random variable with parameter p Ic = p(X ⊤ i Σ -1 X i ≤ R 2 c ). Thus, the random variable |I c | follows a Binomial distribution with parameters n, p Ic . In particular, Hoeffding inequality leads to:

∥ Xi ∥ 2 X⊤ i Ã-2 Xi ≤ ω -2 ρ -2 ∥ Xi ∥ 2 X⊤ i C -2 Xi ≤ R 4 c ω 2 ρ 2 . ( 64
p(|I c | ≥ k) = 1 -p(|I c | ≤ k) ≥ 1 -exp -2n(p Ic -k/n) 2 .
This means that taking k = np Ic -n log(δ -1 )/2 leads to p(|I c | ≥ k) ≤ δ. We finish the proof by noting that X ⊤ i Σ -1 X i follows a χ 2 (d) distribution, so that p Ic = p(χ 2 (d) ≤ R 2 c ). 2 -Concentration of the covariance. Let n c = |I c |. For i ∈ I c , X ⊤ i Σ -1 X i ≤ R 2 c , and so λ max (X i X ⊤ i ) ≤ R 2 c L. Let us define S i = X i X ⊤ i /n c . Then,

∥S k -E [S k ]∥ ≤ 2R 2 c L/n c , and ν = i∈Ic E ∥S i -E [S i ]∥ 2 ≤ 4L 2 R 4 c n . (66)
In particular, we can then use Tropp et al. [2015, Corollary 6.1.2] to bound the tails of A c = i∈Ic S i as:

p(∥A c -E [A c ]∥) ≥ t) ≤ 2d exp - nt 2 8L 2 R 4 c + 4LR 2 c t/3 . ( 67
)
In particular, if t ≤ 6LR 2 c then

p(∥A c -E [A c ]∥ ≥ t) ≤ 2d exp - nt 2 16L 2 R 4 c . (68)
In this case, we have that threshold such that none of these gradients is clipped, so c i = max j ∥∇f ij (θ ⋆ i )∥. This leverages auxiliary information, and is a very strong baseline in the homogeneous setting, but leads to small clipping thresholds in the heterogeneous setting. This leads to small noise but potentially large bias. We observe in our experiments that thresholds significantly lower than c i do not introduce such a large bias still. We conjecture that this is due to the fact that the bias introduced from clipping is small for specific distributions, for instance symmetric. Also note that per-instance clipping is in general computationally expensive, and in particular required significantly more time in our case.

C.2 Parameters for GD with relative sensitivity

In this case, the noisy gradients we release are of the form:

g (i) t = ḡt (i) + N(0, γ i ∥ ḡt (i) ∥ 2 + σ 2 i )
, where ḡt

(i) = 1 N N j=1 Ãij θ t -bij . ( 75
)
The parameters γ i and σ 2 i depend on the relative sensitivity parameters η and R rel , as specified in Theorem 1. To set η, we rely on Assumption 1. This requires some prior knowledge in order to have a covariance that is independent of the specific dataset we sample, which will be heavily application-dependent. Thus, what we do instead is that we choose this ellipse as the covariance of the full dataset, with a radius such that it contains 99% of the points in the base dataset. We then drop the remaining points (we could alternatively keep them and project them back onto the ellipse). To set R rel , we use that it is bounded by η ∥b i ∥ + 2 max ∥b ij ∥. Thus, we bound it by computing the max norm of the b ij , which can also be enforced via clipping. Although ∥b i ∥ ≤ max ∥b ij ∥, we use a separate threshold for ∥b i ∥ which usually leads to tighter guarantees. Note that we can alternatively approximate the local optimum θ ⋆ i and set R rel = max k,ℓ ∥∇f ik (θ ⋆ i ) -∇f iℓ (θ ⋆ i )∥. However, we choose not to in this case because we want something that is enforceable, and this bound is harder to enforce via clipping the b ij . We neglect the privacy loss term αη 2 d factor in our code since we only consider datasets of small dimension. Similarly, as the examples are mainly intended to be illustrative and the constants have not been optimized for, we omit constant factors when estimating η.

As in the clipping case, this specific procedure of choosing the hyperparameters does not strictly guarantee differential privacy, as it uses local datasets. Yet, in practice, application-specific knowledge can help set these parameters. We choose these favorable parameters to show what different methods can do with appropriate hyperparameters, without explicitly quantifying the privacy loss incurred by having to find these parameters. Besides, we choose parameters that can be enforced, so that strict DP is guaranteed if we assume we have some public data to estimate these parameters on.

We present experiments on the ijcnn1 dataset since the estimated η were rather small, and so could illustrate a case in which RGM γ,σ is useful. This is also the case for other LibSVM datasets that we tested, such as HIGGS or SUZY. Other datasets such as cod-rna require high levels of regularization to obtain small η, potentially trivializing the initial problem.

Figure 1 :

 1 Figure 1: Utility of several private gradient descent algorithms with equivalent RDP guarantees. (Left): 'Random', (Middle): 'label', (Right): 'bias'. Shaded areas are min/max values over 3 runs.

  )B.6 Proof of Proposition 3Proof. The proof follows the following plan:1. With probability δ, there are at least|I c | ≥ ω(R c , δ)n, with ω(R c , δ) = p(χ 2 (d) ≤ R 2 c ) -log(δ -1 )/2n.2. These points concentrate, i.e., if we defineA c = i∈Ic X i X ⊤ i /|I c | then: p(∥A c -E [A c ]∥ ≥ 4LR 2 c log(2d/δ) n ) ≤ δ for n ≥ 4 log(2d/δ)/9.(65)3. E [A c ] = ρ(R c )Σ, which finishes the proof.

p

  ∥A c -E [A c ]∥ ≥ 4LR 2 c log(2d/δ) n ≤ δ for n ≥ 4 log(2d/δ)/9.(69)

https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html

Acknowledgements

This work was supported by the Inria Exploratory Action FLAMED and by the French National Research Agency (ANR) through grant ANR-20-CE23-0015 (Project PRIDE), ANR-20-CHIA-0001-01 (Chaire IA CaMeLOt) and ANR 22-PECY-0002 IPOP (Interdisciplinary Project on Privacy) project of the Cybersecurity PEPR.

Appendix

The appendix is organized as follows. Section A contains the full proofs for the general RGM γ,σ , and in particular Theorem 1. Section B contains the proofs for the results that justify using relative assumptions when minimizing quadratics, and Section C contains the detailed experimental setting, with all the elements needed to reproduce the experiments. The code itself can be found in supplementary material.

Proof. We write f (θ) = f (•; D) for simplicity. In this case, for all t ≥ 0,

We now use the smoothness of f , which ensures that ∥∇f (θ t )∥ 2 ≤ 2LD f (θ ⋆ , θ t ), and obtain:

We can then use the µ-strong convexity of f , which yields 2D f (θ t , θ ⋆ ) ≥ µ ∥θ t -θ ⋆ ∥ 2 , and so:

By taking τ ≤ [(1 + γ)L] -1 , using that D f ≥ 0 since f is convex, and chaining the inequalities, we obtain:

In the convex case (µ = 0), we go back from (52), use the same step-size condition, and rewrite it as:

We now write (telescoping sum):

Equation ( 51) is obtained by convexity of f .

B.2 Bounding relative sensitivity for general functions

Let f, f ′ be two functions on neighboring datasets (as defined in the main text), such that f -f ′ = f 0 -f ′ 0 . Let f and f ′ be µ-strongly-convex, f 0 and f ′ 0 be L-smooth, and f and f ′ 0 be L rel -relatively smooth w.r.t. both f and f ′ . In this case, we have that:

Then, using successively the L-smoothness of f 0 , the L rel -relative smoothness of f 0 w.r.t. f , and strong convexity of f , we get:

We can then use the same bound for the f ′ 0 term, and for making f ′ appear. Compared to direct bounding without relative smoothness, we have replaced a condition number L/µ by a relative smoothness term L rel , which is generally much smaller. We then obtain that η 2 = O(κL rel /n 2 ), much like in Equation (10).

B.3 Proof for orthogonal data

Let us assume that the data is orthogonal, i.e. that either X ⊤ i X j = ∥X i ∥ 2 or X ⊤ i X j = 0. This is actually slightly stronger, because we also assume that there is no norm spread for a given direction. This could be relaxed so that results would depend on the average norm, but we keep it simple here.

E [A c

] is lower bounded by the covariance. We write, using the change of variable u = Σ -1 2 x:

where the last line follows from the fact that the expression within the integral is completely symmetric, so for any u, each direction receives a weight proportional to d -1 . In particular,

which only depends on R c , and is close to 1 for instance when R 2 c ≥ 2d. If we did not have the ∥u∥ 2 term within the integral then this would be exactly equal to the cumulated distribution function of the χ 2 (d) distribution. Yet, it is slightly different in our case and a more precise evaluation would require specific detailed derivations.

C Details on the experimental setup

In this section, we provide the various details needed to reproduce our results. Note that we also provide code in supplementary material. Our experimental setup assumes that the data is split across 2 nodes. At each step t, node i privately releases its gradient g

t , computed at point θ t . For all methods, we run the following gradient descent algorithm:

t + g

(2)

where τ is the step-size. In order to comply with the guidelines of Theorem 2, we set it as half the maximum of the largest eigenvalue of the local covariance matrices, so τ = 0.5/ max(λ max (A (1) ), λ max (A (2) )) which is an approximation for the true largest possible stepsize. This is the step-size we use regardless of the method used for noising. Then, all methods only differ in the way they add noise to the gradients. The non-private method thus releases:

C.1 Parameters for GD with clipping

For gradient clipping, each node clips all its individual gradients using a clipping threshold c (i) . In particular, the noisy gradients write:

where the variance of the noise σ 2 i is computed as

where α and ε are the Rényi-DP parameters. The results of DP-GD heavily depend on how we set the clipping threshold c i .

One way of setting it is simply to randomly try out some clipping thresholds (potentially with external knowledge), but this is quite inefficient, as each node node needss to release M times more gradients to the other node if we would like to try M thresholds. Instead, we assume in this work that we can estimate the stochastic gradients at the optimum for the function we consider, and choose the smallest


Added mass and damping of an hexagonal rod vibrating in highly confined viscous fluids

Lucia Sargentini * and Benjamin Cariteau Universit é Paris-Saclay, CEA-Saclay, Service de Thermohydraulique et de M écanique des Fluides, 91191, Gif-sur-Yvette, France lucia.sargentini@cea.fr

This paper deals with fluid-structure interaction analysis of an hexagonal rod enclosed in a narrow viscous gap.

A new analytical solution for a 2D cylindrical case is derived and described. A numerical solution of 2D Navier-Stokes equations coupled with an harmonic structure model is applied to both cylindrical and prism geometries. The comparison between the numerical tool and the analytical solution is discussed and a method to apply the analytical solution to the hexagonal case is proposed.

An original definition of the added mass and damping based on an energetic approach is provided avoiding the dependence from the geometry and the type of forcing (free or forced vibration).

An experimental facility is provided accounting for an hexagonal prism vibrating within a 7 mm enclosure. Free vibration experiments in water allow assessing the added mass and added damping effect on the modal parameters.

The fluid flow is affected by a 3D effect -named downstrokes flow -at the top and the base of the assembly because of free surface and stoky geometry. This produces a higher frequency than the 2D theoretical value given both by the analytical solution and the numerical simulation. A geometrybased correction factor is suggested to taken into account in the 2D numerical simulation the 3D effect.

Velocity measured within the gap provides further insight on this phenomenon and agrees well with the prediction of the transposed cylindrical analytical model.

Introduction

In Sodium Fast Breeder Reactor (Na-FBR) fuel assemblies are hexagonal prisms in a periodic lattice and, because of neutronic optimisation, they are arranged in a very narrow gap configuration. During vibrations, such confined configuration gives rise to high fluid forces acting on the hexagonal prisms, involving a modification on the structure's dynamic behaviour.

While in scientific literature fluid-structure interaction for cylindrical structures are largely discussed (see for in-stance the book of Païdoussis [START_REF] Paidoussis | Fluid-Structure Interactions: Slender Structures and Axial Flow[END_REF]), to the authors' knowledge only few works are specifically devoted to other geometries.

Recently, Zhong et al. [2] [3] perform some experiments to improve the comprehension of the fluid-structure interaction dynamic of the double graphite bricks in molten salt reactor. They find that finite element 2D analysis well represents the FSI experiments if bricks are sufficiently immerged in the water. Daogang et al. [4] present some free vibration tests of two neighboring hexagonal prisms. Added mass and damping coefficients are evaluated experimentally showing a large dependence on the thickness of the fluid gap. It is suggested that Wilson's laws [START_REF] Wilson | Added mass and damping coefficients for hexagonal cylinder[END_REF] are in good agreement with their experience. Nevertheless, no comparison is given. Moreover, Wilson's work shows a discrepancy between the given definition of the added mass coefficient and the experiences of Yang and Moran [START_REF] Yang | Finite-element solution of added mass and damping of oscillation rods in viscous fluids[END_REF].

In his classical study, Fujita [START_REF] Fujita | Vibrational characteristics and seimisc response analysis of colums group in liquid[END_REF] performs a seismic response analysis of a group of 7, 19 or 37 hexagonal columns immersed in water. He proposes a two-dimensional semitheoretical method based on the resolution of Euler equations coupled with mass-spring model for structure. The vertical flow of liquid is neglected and each assembly is supposed to vibrate parallel to its generating line. He suggests a value of the added mass for two concentric hexagonal rods with narrow gap showing a poor agreement with the experiences.

To study analytically a much simpler geometry, several authors investigate forced vibration of a two-dimensional cylindrical rod immersed in a fluid confined within a shell [START_REF] Chen | Vibration of nuclear fuel bundles[END_REF][START_REF] Chen | Added mass and damping of a vibrating rod in confined viscous fluids[END_REF][START_REF] Fritz | The effect of liquids on the dynamic motions of immersed solids[END_REF].

Mulcahy [START_REF] Mulcahy | Fluid forces on rods vibrating in finite length annular regions[END_REF] proposes an extension of the 2D analytical solution of Chen [START_REF] Chen | Added mass and damping of a vibrating rod in confined viscous fluids[END_REF][START_REF] Fritz | Added mass and damping of a vibratory rod in confined viscous fluids (discussion)[END_REF] to 3 dimensions for a finite length cylinder. Unfortunately, its solution is valid only far from the ends. Actually, the postulated fluid flow is two-dimensional with a vertical component of the velocity. Nevertheless, he presents a correlation between the fluid damping force and the geometrical aspect ratio of the finite cylindrical rod. We detail it later on the article.

In a previous communication [START_REF] Sargentini | Experimental and numerical analysis for fluid-structure interaction for an enclosed hexagonal assembly[END_REF], we presented an original analytical solution for the vibration of a two-dimensional cylindrical rod immersed in a fluid confined within a shell. We validated against it a coupled fluid-structure numerical solver that we used to investigate the flow differences between hexagonal and circular annulus. A transposition method was formulated to apply the cylindrical analytical solution to hexagonal prisms.

In this communication, we challenge the analytical solution against experimental data of added mass and fluid flow in the gap. In the next section we recall an analytical solution for the forced vibration of a long cylinder enclosed in a small viscous gap. Subsequently, the application to hexagonal prisms is discussed together with some three-dimensional fluid phenomena affecting short cylinders and prisms. Finally, we present some experimental results and we compare them to our analytical model. 2 Analytical solution for a cylinder vibrating in a small viscous gap and extension to hexagonal prisms An original analytical solution is obtained for the forced vibration of a cylinder in a very confined viscous fluid with initially stagnant fluid. It was presented in a previous communication [START_REF] Sargentini | Experimental and numerical analysis for fluid-structure interaction for an enclosed hexagonal assembly[END_REF]. We resume here the main features and we give in appendix the (previously unpublished) explicit expression. Subsequently, by means of an available transposition criterion, it is applied to hexagonal geometries in Section 3.

Since the fluid is stagnant at the beginning of the movement, turbulent flow regime should not be able to develop for times as short as ≈ 0.1 s. If gap fluid is water or liquid sodium, the time scale which is necessary for the development of a boundary layer is much large than 0.1 s. Nevertheless, even if viscous effects seem to be negligible a priori, we consider important to gather a solution for validation of numerical models accounting for viscosity. In fact, viscosity plays a role for the vibration of hexagonal prisms [16].

The 2D Navier-Stokes equations are solved for an incompressible and viscous fluid flow in a very thin gap for forced bi-dimensional vibrations. A sketch of the studied configuration is shown in Fig. 1.

The external cylinder of radius R 2 is fixed while the internal cylinder of radius R 1 vibrates with amplitude a(t). Noslip boundary conditions are used for the external and internal cylinder walls. Because of narrow gap hypothesis, the fluid domain can be assimilated to a developed cylinder area. Therefore a coordinate transform is applied, where x = R 2 θ and y = R 2r.

A non-dimensional analysis may be performed finding all the unknown scales [START_REF] Sargentini | Étude des mécanismes d'intéraction fluide-structure d'un coeur RNR-Na lors de l'évacuation d'une poche de gaz[END_REF]. Let's define overlined quantities as non-dimensional parameters. U and V are the scale respectively of tangential ū( x, ȳ, t) and radial v( x, ȳ, t) velocity. These velocity scales are unknown at the beginning of the analysis, as well as δp as pressure scale. Other scales are provided: the angular frequency ω is taken as time scale, the clearance of the gap b as y-scale, the external radius R 2 as

x-scale and the maximum of displacement a 0 as the scale of oscillation amplitude.

The following non-dimensional parameters are obtained from the equations :

• η = b/R 2 ≪ 1 which represents the effect of confinement; • ε = a 0 /b which represents the maximum amplitude of oscillation relating to the gap. This value is limited to 1 and can be very low;

• St = ωR 1 /U = 1/ε
   U = εR 1 ω V = ηU (1)
The complete description of the velocity and pressure scales is available in Sargentini et al. [START_REF] Sargentini | Étude des mécanismes d'intéraction fluide-structure d'un coeur RNR-Na lors de l'évacuation d'une poche de gaz[END_REF]. The dimensionless Navier-Stokes equations finally become :

                                               ∂ ū ∂ x + ∂ v ∂ ȳ = 0 ∂ p ∂ ȳ = 0 St 1 ∂ ū ∂t + v ∂ ū ∂ ȳ + ū ∂ ū ∂ x = - δp ρU 2 ∂ p ∂ x + 1 ηRe ∂ 2 ū ∂ ȳ2 v ȳ=0 = 0 ∀ x, t ū ȳ=0 = 0 ∀ x, t v ȳ=1 = -cos( x) ∀ x, t ū ȳ=1 = 0 ∀ x, t v t=0 = 1 ∀ x, ȳ (2)
The azimuthal pressure gradient is the driving term of the fluid displacement. Therefore we deduce the pressure scale (i. e. Euler number) from the other scales :

δp ρU 2 = sup St; 1; 1 σε (3)
where the first group is the scale of non-stationary term, the second one is the scale of convective term and the last one is the scale of viscous term. Fig. 2. Flow map for a solid rod vibrating in a thin fluid annulus.

From this system of equations, a flow regime map may be deduced which reveals the suitable mathematical model according to the physical condition to be described (Fig. 2). Strictly speaking, full Navier-Stokes (N-S) model is only required when St ∼ σε ∼ 1.

When σ is lower than 1, viscous effects must be taken into account and quasi-static Stokes equations apply; when σ is higher, Euler equations apply. Moreover for small amplitude oscillation (ε ≫ 1) linearized Euler equations apply. Full form of Navier-Stokes equations would be necessary when all scales equal.

The explicit solution of the equations 2 is given in appendix. Also, a solution for free decay vibration was obtained and detailed in reference [START_REF] Sargentini | Étude des mécanismes d'intéraction fluide-structure d'un coeur RNR-Na lors de l'évacuation d'une poche de gaz[END_REF]. Our solution is valide for very thin gap and, in these cases, it is in good agreement with the one provided by Chen [START_REF] Chen | Added mass and damping of a vibrating rod in confined viscous fluids[END_REF]. With respect to Chen formulation, the present one allows easier numerical calculation (because of the absence of Bessel functions) and hence easier acces to the flow field (equation 9-10-11).

3 Extension of the cylindrical analytical solution to the hexagonal prism Sargentini et al. [START_REF] Sargentini | Experimental and numerical analysis for fluid-structure interaction for an enclosed hexagonal assembly[END_REF] set up a 2D numerical tool which solves the full Navier-Stokes equation coupled to structure dynamics by means of Arbitrary Lagrangian-Eulerian mesh displacement (ALE) with the code Cast3M. A comparison between the free response of an enclosed cylinder and an hexagonal prism was studied. It was found that the 2D cylindrical analytical solution may be used to evaluate the fluid forces and flow for the hexagonal prism at middle-height. The only condition to satisfy is to set up a cylindrical analytical calculation having the same fluid cross section of the hexagonal case.

Using an energetic approach, we can give the following definitions for added mass and added damping:

• the added mass is the mass of the coupled system (liquid and solid body) which moves at the velocity of the solid body. Hence the added mass coefficient C M is:

C M = 1 ρπR 2 1 τ 0 E c, f dt τ 0 ȧ2 (t) 2 dt (4)
with τ as the time integration length and E c, f the fluid kinetic energy. • the added damping is related to the energy E dis dissipated by the coupled system. Let's define E dis as :

E dis = τ 0 C a (t) ȧ2 (t)dt (5)
with C a (t) the instantaneous added damping. This dissipation occurs in the fluid, operated by the viscous forces.

In other words:

E dis = t 0 Ω 2 ν ρ ∑ i j ∂u i (t) ∂x j D i j (t) dA dt for i, j = x, y
where

D i j = 1 2 ∂u(t) ∂y + ∂v(t) ∂x
. The instantaneous dissipated energy E dis (t) can also be calculated as:

E dis (t) = E 0 + E c, f (t) + E ts (t)
where E 0 is the initial energy of the system (i.e. the potential energy before the free vibration) and E ts is the structure total energy (i.e. kinetic and potential energy).

The average added damping coefficient C D at instant τ can be estimated as:

C D (τ) = τ ρ f πR 2 1 E dis (τ) τ 0 ȧ2 (t) dt (6)
Both the added mass and damping coefficients take the form of a power divided by an energy and are made dimensionless by using a mass. The added mass coefficient C M is an average value during the time oscillation. On the contrary, the C D depends on the time of the oscillation τ, since the damping coefficient can vary with the amplitude during vibrations.

Rather than Chen's dynamic formulation [START_REF] Chen | Added mass and damping of a vibrating rod in confined viscous fluids[END_REF], our definitions are independent on the geometry of the system and the form of vibration. Recently, some works on two-phase flow added mass also turn towards an energetic approach [18].

For the cylindrical geometry, our definition and the analytical solution of Chen are in perfect agreement (Fig. 3 and Fig. 4). For the hexagonal geometry, validation is provided in Sargentini [START_REF] Sargentini | Étude des mécanismes d'intéraction fluide-structure d'un coeur RNR-Na lors de l'évacuation d'une poche de gaz[END_REF] through a comparison with the theory of Chung and Chen [START_REF] Chung | Hydrodynamic mass[END_REF]. Results for the C M for the two geometries are close, but the hexagonal solution is always slightly higher than cylindrical one. For C D , the higher the frequency and thinner the gap, the more the difference between cylinder and hexagonal body. This discrepancy is due to the difference on the fluid velocity profiles, thus on the fluid kinetic energy, between cylindrical and hexagonal case. In fact, as the hexagon vibrates perpendicularly to one of its sides, the cylindrical analytical solution of tangential velocity describes well the fluid flow close to the mid-side of the prism. Conversely, the cylindrical analytical solution fails to estimate the flow close to the vertex of the hexagonal gap [START_REF] Sargentini | Étude des mécanismes d'intéraction fluide-structure d'un coeur RNR-Na lors de l'évacuation d'une poche de gaz[END_REF]. In Fig. 5, the 2D numerical results for the tangential velocity u in both cylindrical and hexagonal geometries is presented. The direction of vibration of the inner prism is θ = 0 • which, for the hexagonal case, is normal to the side. The cross-gap velocity profile is the same for both geometries for θ = 0 • (mid-side of the hexagon). On the contrary, the cross-gap velocity profile at the corner of the hexagon is different. In fact, the presence of these corners gives rise to recirculations (vortex) even in non-turbulent flows.

These vortices on the corners of the hexagon dissipate Equation 9 Cylinder, θ=0 Cylinder, θ=60°H exagon, corner Hexagon, mid-side Fig. 5. Numerical results for the tangential velocity u in both cylindrical and hexagonal geometries [START_REF] Sargentini | Experimental and numerical analysis for fluid-structure interaction for an enclosed hexagonal assembly[END_REF]. Comparison is given with the analytical solution of system 2 given in appendix (equation 9). Table 1. Dynamical characteristics measured on the twin bladessupported hexagonal prism enclosed in a 7 mm gap.

Frequency in air (Hz) 11.7

Damping ratio in air (%) 3.7

Frequency in water (Hz) 5.2

Damping ratio in water (%) 2.4

Added mass coefficient 5.5

energy and increase the added damping with respect to cylindrical geometry, thus explaining the discrepancies between the two geometries shown in Fig. 4.

4 Experimental results: comparison with theory

Experimental apparatus

The analytical study was used to design a test rig accounting for a single Poly-methyl methacrylate (PMMA) hexagonal prism enclosed in a 7 mm gap filled with water and named PISE-1a [START_REF] Sargentini | Experimental and numerical analysis for fluid-structure interaction for an enclosed hexagonal assembly[END_REF]. The prism is 71,4 mm of side and 500 mm tall.A picture is provided in Fig. 6.

The prism is free on top and supported by a twin blades system on the bottom. Blades are stainless-steel made, 3 mm thick and 210mm in height. This system allows solidbody, one-degree-of-freedom vibration of the prism, i.e. horizontal translation orthogonal to the generating lines. Strain gauges are fixed on the blades to recover prism displacement. Time-domain modal analysis is applied on displacement time-history to estimate the modal parameters [START_REF] Sargentini | Étude des mécanismes d'intéraction fluide-structure d'un coeur RNR-Na lors de l'évacuation d'une poche de gaz[END_REF]. Velocity fields within the gap are recorded by Particle Image Velocimetry (PIV) at a frequency of 500 Hz.

Free vibration response of the assembly are detailed in section 4.2 and Tab. 1.

Added mass and damping with 3D effects flow: discussion of the experimental results

In Sargentini et al. [17]

, the free response of the twinblades-supported enclosed prism is compared to the analytical solution. The 2D analytical model predicts a lower frequency and a higher damping ratio in water with respect to the measured one. In the next section we propose a correction factor basing on 3D flow considerations.

Free vibration response is measured both in air and in water, the results are listed in Tab. 1. Damping ratio in water is smaller with respect to air value. In fact, for narrow gaps, fluid inertia is very high and added mass is much higher than structural mass. As remarked by Daogang et al. [START_REF] Daogang | Experimental investigation on fluid-structure-coupled dynamic characteristics of the double fuel assemblies in a fast reactor[END_REF], the damping ratio of the coupled system is proportional to the added damping and inversely proportional to the added mass. Hence, if gap is small enough (as in our case), the added mass effect is dominant on the added damping effect, and the resulting damping ratio in air is higher than in water.

Mulcahy [START_REF] Mulcahy | Fluid forces on rods vibrating in finite length annular regions[END_REF] showed that the fluid damping force per unit length depends on the ratio between the internal cylindrical radius R 1 and its height H through the parameter χ:

χ = 1 - R 1 H tanh H R 1 (7)
This means that, for slender structures (H/R 1 → ∞), the damping force is due to the two-dimensional fluid flow around the 2D section, while, for stocky structures (H/R 1 → 0), the damping force is due to the vertical fluid flow along the height of the vibrating cylinder due to the conditions of constant pressure at the top of the rod. Our configuration is close to a stocky structure (H/R 1 roughly 8, and χ ≈ 0 ). This means that the damping force is due to a 3D flow. Hence, it is unlikely to predict the damping by means of 2D coupled fluid-structure simulations. According to flow visualization made with PIV, near the top of the prism the fluid flow rises up to the free surface when the gap becomes larger than the initial condition and goes down to the gap when the gap becomes thinner [17]. The fluid behavior is symmetric with respect to the perpendicular line to the assembly displacement. This behavior depends on the pressure field. The vertical flow, referred to as down-strokes flow, damps down on a distance from the free surface of about the radius of the external cylinder R 2 (Fig. 7). The effect of this fluid movement is a loss of confinement and so, a loss of added fluid inertia. As a consequence, the added mass of the fluid M add is lower than the theoretical value for a completely two-dimensional fluid-flow.

It is possible to apply a geometrical correction factor to the 2D pressure field in order to predict the frequency even in case of strong 3D effects.

This correction factor β only depends on the aspect ratio of the prism and it can be deduced from the ratio between the 2D fluid force F f ,2D (given by equation 2) and the z-averaged fluid force calculated with a 3D pressure field F f ,3D (given in [17]):

β = 2 R 1 + b H = F f ,2D -F f ,3D F f ,2D (8)
For the present experimental application β = 0.3. Sargentini et al. [17] found this correction factor effective to predict the frequency of the experimental device in water.

Fluid flow in the gap

The analytical solution of fluid flow given in the equation system 2 is now compared with the measured flow field within the experimental hexagonal assembly. As previously

Vibration direction

Laser plane for Fig. 10 Camera for Fig. 10 Laser plane for Fig. 9 Camera for Fig. 9 Vibrating prism Enclosure Fig. 8. Set-up of the PIV measurement system applied to the device of Fig. 6 (cross-section sketch). This system is used to record the velocity profiles of Fig. 9 and 10.

discussed in section 4.2, the solution for cylindrical structures can give a profitable description of flow within hexagonal gaps on some cases and zones. Fig. 8 show the position of the PIV camera for the two subsequent figures. 2D planes are registered and used to extract velocity profiles at different heights.

In Fig. 9and in Fig. 10 we compare the experimental measured velocities across the gap and the analytical results of equations 9 and 10. A good agreement is shown far from the vertex of the hexagon, where vortex are present.

In Fig. 9 we show a velocity profile at mid-height of the prism. The profile is taken in the middle of the gap, along the hexagon's side. The measured vertical velocity w is almost zero, in agreement with the 2D hypothesis. The tangential velocity u is in good agreement with the cylindrical analytical solution far from the vertex of the hexagon, where vortex are present.

In Fig. 10 we show a velocity profile closer to the top of the prism. The profile is taken across the gap at mid-side of the hexagon. The measured vertical velocity w is high due to the 3D down-strokes flow. Nevertheless, the radial velocity v is still in good agreement with the cylindrical analytical solution.

Conclusion

In this paper we presented:

• a 2D analytical solution for forced vibration of a cylindrical enclosed rod with an narrow viscous gap. With respect to existing analytical formulations, the present one allows easier numerical calculation because of the absence of Bessel functions and it can be easily adapted to other geometries if the cross fluid section is equivalent. • an energetic approach for defining the added mass and damping which is independent from geometry of the problem or if the solid is in forced or free vibrations. • a comparison between the analytical solution, 2D numerical tool and experiments on the frequency and the damping ratio. Experimental observations of the flow within the liquid gap show that 3D flow is responsible for lower added mass effects in agreement with Mulchay's theory. This 3D fluid flow depends on the pressure field at the top of the prism (free surface). Taking into account 3D effects via the aspect ratio, as in equation 7, allows extending the 2D analytical solution to the 3D stocky structure. • the comparison between the the measured flow velocity profiles for the hexagonal prism and the 2D cylindrical solutions. Radial and tangential velocities are in good agreement far from the corners.

Nomenclaturea

  Instantaneous amplitude [m]. a 0 Initial amplitude of free decay [m]. ȧ Instantaneous solid velocity [m/s]. b Gap thickness [m]. C a Added damping [kg/s]. C D Added damping coefficient [-]. C M Added mass coefficient [-]. D i, j Momentum diffusion [1/s]. E c, f Fluid kinetic energy [J]. * Address all correspondence to this author.

E

  dis Fluid dissipated energy [J]. E t,s Total solid energy [J]. E 0 Initial potential energy [J]. H Height of the prism [m]. M add Added mass [kg]. R 1 Internal cylinder radius [m]. R 2 External cylinder radius [m]. Re Reynolds number Ub/ν [-]. St Strouhal number ωR 1 /U [-]. U Tangential velocity scale [m/s]. V Radial velocity scale [m/s]. x Developed tangential coordinate [m]. y Developed radial coordinate [m]. r Radial coordinate [m]. t Time [s]. p Pressure [Pa]. δp Pressure variation scale [Pa]. Greek β 3D-to-2D correction factor [-]. η Confinement ratio [-]. ρ Fluid density [kg/m 3 ]. ν Fluid kinematic viscosity [m 2 /s]. ε Amplitude-to-gap ratio [-]. σ Stokes number ωb 2 /ν [-]. ω Angular frequency [s -1 ]. χ Mulchay slenderness parameter [-]. θ Azimuthal coordinate [rad]. τ Time integration length [s].

Fig. 1 .

 1 Fig. 1. Sketch of the cross-section of a cylinder vibrating in a thin fluid annulus.

Fig. 3 .Fig. 4 .

 34 Fig. 3. Comparison between the calculated added mass coefficient for cylindrical and hexagonal geometry for different Stokes numbers σ as a function of the confinement.

Fig. 6 .

 6 Fig. 6. Picture of the single enclosed hexagonal rod.

Fig. 7 .

 7 Fig. 7. Sketch of the flow features at the top of a rod oscillating along the x direction (θ = 0) within a fluid-filled gap. This sketch applies both to cylinder and to hexagonal prisms.

Fig. 9 .Fig. 10 .

 910 Fig.9. Velocity profiles on the face of the prism compared to the analytical solution given by equations 9 at t=109 s and at mid-height of the prism.

  which is the Strouhal number. It represents the ratio between the displacement time of a fluid particle and the displacement time of the solid body. From boundary conditions we can see that the Strouhal number depends only on the geometry and on the amplitude of vibrations; • Re = Ub/ν which is the Reynolds number. As the fluid velocity scale is imposed by the vibration, Reynolds number contains frequency information ; • σ = ωb 2 /ν which is the Stokes number.

	The resulting velocity scales are :

  PhD thesis, Pierre and Marie Curie University, Paris, France.[16] Artini, G.,Broc, D. (2017), Fluid structure interaction for tubes bundles: different homogenization methods. ASME Pressure Vessels & Piping Conference, Waikoloa, Hawaii (USA). Paper PVP2017-65727. [17] Sargentini, L., Cariteau, B., Magnaud, J-P., Monavon, A., Artini, G. (2015). 3D effects in fluid flow during 2D vibrations: experimental analysis of FSI for an enclosed hexagonal assembly. ASME Pressure Vessels & Piping Conference, Boston, MA (USA). Paper PVP2015-45904. [18] Deri, E. (2022). Some comments on added mass modeling for tube bundles under two-phase cross flow. Journal of Pressure Vessel Technology, 144:044503.A Solution of the system of equation 2The resulting velocity field if proportional to Stokes number and only depend on tangential coordinate: indeed, for small gap (high confinement) pressure doesn't vary radially. The following equations gives the dimensionless form of the developed azimuthal velocity, of the radial velocity and of the pressure: ū = e it sin x -

							p = σi cos xe it	tanh ( 2 tanh ( √ σi 2 √ σi ) + 2 ) -√ σi σi √	(11)
						√	√
				η tanh ( √ 2 σi	ȳ)	1 -tanh ( √ 2 σi	)	+
				tanh (		σi 2	)	1 -tanh ( √	σi 2	ȳ)
		√ σi + η 1 -	√ σi	1 -tanh ( √	σi 2	)
			√				√	2 tanh (	σi 2 √	)
	cosh (	σi 2	ȳ) + sinh ( 2 tanh ( √ σi σi 2 2 ) -ȳ) tanh ( σi √	σi 2	) -1	(9)
							√
	v = e it cos x	η	√ σi √	1 -tanh ( √ 2 σi	)	+
				tanh ( √	σi 2	)	1 -tanh (	σi 2	ȳ)
		ȳ 2	1 -tanh ( √ tanh ( σi σi 2 ) 2 √	)	+ 1 + η √	1 √ σi	-
	cosh (	1 -tanh ( 2 tanh ( √ σi σi 2 2 ) √ σi 2 ȳ) tanh ( √ ) σi 2 2 tanh ( √ σi ) -tanh ( sinh ( 2 tanh ( √ σi σi 2 ) -ȳ) √ σi 2 √ σi ) -√ σi ȳ 2 2 σi ) -√	+	(10)

Acknowledgments

This work has been supervised by Arnault Monavon, Jean-Paul Magnaud, Daniel Broc, whose contributions were essential.