David Ginsbourger

Xavier Bay

Olivier Roustant

Laurent Carraro

Argumentwise invariant kernels for the approximation of invariant functions

come

Introduction

Positive definite 1 (p.d.) kernels play a central role in several contemporary functional approximation methods, ranging from regularization techniques within the theory of Reproducing Kernel Hilbert Spaces (RKHS) to Gaussian Process Regression (GPR) in machine learning. One of the reason for that is presumably the following particularly elegant predictor, common solution to approximation problems in both frameworks. Indeed, if scalar responses y := (y 1 , . . . , y n ) ∈ R n are observed for n instances x 1 , . . . , x n ∈ D of a d-dimensional input variable (D is here assumed to be a compact subset of R d ), the function

m : x ∈ D -→ m(x) = k(x) T K -1 y, (1)
is at the same time the best approximation of any function f in the RKHS of kernel k subject to f (x i ) = y i (1 ≤ i ≤ n), and the GPR ("Simple Kriging") predictor of any squared-integrable centered random field (Y x∈D ) x∈D of covariance kernel k subject to Y x i = y i (1 ≤ i ≤ n). k : D × D -→ R stands here for an arbitrary p.d. kernel, with k(x) := (k(x, x 1 ), . . . , k(x, x n )) and K := (k(x i , x j )) 1,i≤j≤n (assumed invertible here and in the sequel).

In practical situations (e.g., when the y i 's steem from the output of an expensive-to-evaluate deterministic numerical simulator, say y : D → R), the choice of k is generally far from being trivial. Unless there is a strong prior in favour of a specific kernel or parametric family of kernels, the usual modus operandi to choose k in GPR (when d is too high and/or n too low for a geostatistical variogram estimation) is to rely on well-known families of kernels, and to perform classical Maximum Likelihood, Cross-Validation, or Bayesian inference of the underlying parameters based on data. For example, most GPR or Kriging softwares offer different options for the underlying kernel, often restricted to stationary but anisotropic correlations like the generalized exponential or Matérn kernels, allowing the user to choose between different levels of regularity. This is in fact based on solid mathematical results concerning the link between the regularity of covariance kernels and the mean square properties of squared integrable random fields (or even a.s. properties in the case of Gaussian random fields, see [START_REF] Cramér | Stationary and Related Stochastic Processes: Sample Function Properties and Their Applications[END_REF]).

A weak point of such an approach, however, is that not all phenomena can reasonably be approximated by stationary random fields, even with a well-chosen level of regularity and a successful estimation of the kernel parameters. In order to circumvent that limitation, several non-stationary approaches have been proposed in the recent literature, including convolution kernels (see [START_REF] Paciorek | Nonstationary Gaussian Processes for Regression and Spatial Modelling[END_REF] or [START_REF] Lee | Spatial models via convolutions of intrinsic processes[END_REF]), kernels incorporating non-linear transformations of the input space ( [START_REF] Guttorp | Nonparametric estimation of nonstationary spatial covariance structure[END_REF][START_REF] Anderes | Estimating deformations of isotropic Gaussian random fields[END_REF][START_REF] Xiong | A non-stationary covariance-based kriging method for metamodelling in engineering design[END_REF]), or treed gaussian processes ( [START_REF] Gramacy | Bayesian treed Gaussian process models with an application to computer modeling[END_REF]), to cite an excerpt of some of the most popular approaches.

Our intent here is to address a specific question related to the choice of k: Assuming a known geometric or algebraic invariance of the phenomenon under study, is it possible to incorporate it directly in a kernel-based approximation method like GPR or RKHS regularization? More precisely, given a function y invariant under a measurable action Φ of some finite group G on D, is it possible to construct a metamodel of y respecting that invariance?

Here we investigate classes of kernels leading to metamodels m inheriting known invariances from y. In the particular case of a GPR interpretation, the proposed kernels enable a deeper embedding of the prescribed invariance in the metamodel since the obtained random fields have invariant paths (up to a modification). Note that the proposed approach is complementary to the non-stationary kernels evocated above, rather than in competition with them.

Definitions and classical results

2.1. Group actions and invariant functions. Let (G, * ) be a group and D a set. We denote by e the neutral element of G. Let us recall some standard definitions from algebra [START_REF] Lang | Algebra[END_REF].

Definition 1. A (left) action of the group G on D is a map Φ : D × E -→ E (g, x) -→ g.x := Φ(g, x) such that -x ∈ D -→ Φ(e, x) is the identitity of D, i.e. ∀x ∈ D, e.x = x, -∀x ∈ D, ∀g, g ′ ∈ G, (g * g ′ ).x = g.(g ′ .x).
Definition 2. The orbit of a point x ∈ D under the action Φ is the set

O(x) := {g.x, g ∈ G}, (2)
constituted of images of x by the action of G.

Equivalently, y is said invariant whenever it is constant on the orbits of Φ.

2.2. Random fields. We borrow here a few definitions from the book [START_REF] Revuz | Continuous Martingales and Brownian Motion[END_REF], with a few minor changes in the notations.

Definition 8. Two random fields Y and Y ′ , respectively defined on probability spaces (Ω, F, P) and (Ω ′ , F ′ , P ′ ) and sharing a common measurable state space (D, D), are said equivalent if for any finite sequence of points x (1) , . . . , x (n) ∈ D and events A 1 , . . . , A n ∈ D,

P (Y x (1) ∈ A 1 , . . . , Y x (n) ∈ A n ) = P ′ Y ′ x (1) ∈ A 1 , . . . , Y ′ x (n) ∈ A n (6)
One also says in that case that each one of these random fields is a version of the other, or that both are versions of the same random field. In other words, two random fields are versions of each other whenever they have the same finite-dimensional distributions.

Definition 9. Two random fields Y and Y ′ defined on the same probability space (Ω, F, P) are said to be modifications of each other when for all x ∈ D,

P(Y x = Y ′ x ) = 1 (7)
They are said indistinguishable when

P ∀x ∈ D, Y x = Y ′ x = 1 (8)
As precised in ( [START_REF] Revuz | Continuous Martingales and Brownian Motion[END_REF]

∈ D with T h (x), T h (x ′ ) ∈ D, k(T h (x), T h (x ′ )) = k(x, x ′ ) (10)
Additionally, a centered weakly stationary random field Y defined over some subset D of a Euclidean space is said to be isotropic whenever cov(Y x , Y x ′ ) depends only on the norm-induced distance between x and x ′ , i.e. k(x, x ′ ) is a function of ||xx ′ ||. Again, this may be written as an invariance of the kernel under the simultaneous transformation of both arguments:

k(R(x), R(x ′ )) = k(x, x ′ ) (11)
where R belongs this time to the more general class of isometries. Both latter invariances can in fact be seen as particular cases (with natural actions of groups of translations or isometries, respectively) of the following definition given by Parthasarathy and Schmidt in [START_REF] Parthasarathy | Positive Definite Kernels, Continuous Tensor Products, and Central Limit Theorems of Probability Theory[END_REF]:

Definition 11. k is said invariant under the action of G on D when ∀g ∈ G, ∀x, x ′ ∈ D, k(g.x, g.x ′ ) = k(x, x ′ ) (12)
3. Main results

3.1.

A characterization of kernels leading to invariant fields. Before stating the main result of the paper, we need to introduce a new notion, generalizing the notion of invariant kernel presented in the last section.

Definition 12. A kernel k is said argumentwise invariant under Φ when ∀g, g ′ ∈ G, ∀x, x ′ ∈ D, k(g.x, g ′ .x ′ ) = k(x, x ′ ) ( 13
)
One can notice that eq.( 12) corresponds to the particular case of eq.( 13) where g = g ′ . As discussed next, this second kind of kernels corresponds to much stronger invariance properties of the associated random fields.

Remark 2. For real-valued symmetric kernels such as considered here, it is equivalent to be argumentwise invariant, left invariant, or right invariant. Indeed, assuming that k is left invariant, we get for g, g ′ ∈ G and x, x ′ ∈ D:

k(g.x, g ′ .x ′ ) = k(x, g ′ .x ′ ) = k(g ′ .x ′ , x) = k(x ′ , x) = k(x, x ′ ) (14)
∈ D ∀g ∈ G ∀ω ∈ Ω, Y x (ω) = Y g.x (ω), so that in particular, ∀x ∈ D ∀g, g ′ ∈ G: k Y (g.x, g ′ .x ′ ) = cov[ Y g.x , Y g ′ .x ′ ] = cov[ Y x , Y g ′ .x ′ ] = cov[ Y x , Y x ′ ] = k Y (x, x ′ )
Reciprocally, let us now assume that k Y is argumentwise invariant under Φ. Let us denote by A ⊂ D a fundamental domain for Φ, and by π A : D -→ A the projector mapping any x ∈ D to its representer π A (x) ∈ A, i.e. to the point of A being in the same orbit. We then define the random field Y by

∀x ∈ D Y x := Y π A (x)
By construction, Y has all its paths invariant under Φ. Now, for any x ∈ D, there exists g ∈ G such that π A (x) = g.x. Subsequently,

var[Y x -Y x ] = var[Y x -Y g.x ] = k(x, x) + k(g.x, g.x) -2k(x, g.x) = 0, so that P(Y x = Y x ) = 1
, and Y is indeed a modification of a random field with Φ-invariant paths.

Remark 3. Y x := 1 #G g∈G Y g.
x would have led to the same conclusion. Remark 4. A fundamental domain A is such that every orbit has a unique representer in A, and g∈G g.A = D. However, the g.A's (g ∈ G) are not necessarily disjoints. For example, if G = Z/2Z, D = R, and Φ :

(g, x) ∈ (Z/2Z) × R -→ R is the action defined by Φ(1, x) = -x, A = [0, +∞) is a fundamental domain containing 0, but 0 ∈ 1.A = (-∞, 0] too.
Consequently, when decomposing an invariant process over the orbits of A, one must account for the points appearing in several g.A's by dividing by the number of appearances, characterized by the cardinal of their stabilizers:

∀x ∈ D, Y x = g∈G Y x 1 g.A (x) #Stab Φ ({x}) = g∈G Z g.x (15)
where {x}) . Denoting Z's kernel by k Z , we get in particular

Z x := Y x 1 A (x) #Stab Φ (
∀x, x ′ ∈ D k Y (x, x ′ ) = cov   g∈G Z g.x , g ′ ∈G Z g ′ .x ′   = (g,g ′ )∈G 2 k Z (g.x, g ′ .x ′ ),
whereof the argumentwise invariance of k Y clearly appears.

Example 1. Let Z be a centered Gaussian process indexed by R, with covariance kernel

k Z : x, x ′ ∈ R -→ k Z (x, x ′ ) = e -|x-x ′ | ∈ R (often called the Ornstein-Uhlenbeck process, Cf. [35] sec. 1.3), and Φ : (g, x) ∈ (Z/2Z) × R -→ R the action of G = Z/2Z on R previously considered. The process Y obtained by symmetrization of Y 's restriction to A := [0, +∞[, defined by Y x = Z |x|
, has all its paths invariant under Φ. Its covariance kernel is given by ∀x,

x ′ ∈ R, k Y (x, x ′ ) = e -||x|-|x ′ || .
Let us notice that Y , symmetrized of the stationary process Z, is obviously not second order stationary.

Example 2. Let Z be a centered Gaussian field indexed by R 2 , with covariance

k Z : x, x ′ ∈ R 2 -→ e -||x-x ′ || 2 ∈ R, and Φ : (g, x) ∈ (Z/2Z) × R 2 -→ R 2 the action defined by φ(1, x) = s(x) := (x 2 , x 1 ), the symmetrized point of x = (x 1 , x 2 )
with respect to the first bisector. The process Y obtained by symmetrization of Z's restriction to

A = {x ∈ R 2 : x 1 ≤ x 2 } is defined by Y x = Z x if x ∈ A Z s(x) if x ∈ A c
Let us note that Y may also be defined as follows, in the spirit of Remark 4:

Y x = 1 1 + 1 {x∈R 2 :s(x)=x} (x) Z x 1 A (x) + 1 1 + 1 {x∈R 2 :s(x)=x} (x) Z x 1 A (s(x))
The next example illustrates that a random field with almost never Φinvariant paths may possess a modification which paths are all Φ-invariant:

Example 3. Let Ω =]0, 1[, A = B(]0, 1[), P be Lebesgue's measure on Ω , D = R, G = {e, s 0 } (s 0 be the symmetry with respect to 0), F : x ∈ R -→ x -∞ e -u 2 2 √ 2π du ∈]0, 1[, ε : ω ∈ Ω -→ ε(ω) = F -1 (ω) ∈ R, and Y : (x, ω) ∈ E × Ω -→ Y x (ω) = |x|ε(ω)1 x =ε(ω) .
The process defined by Y x (ω) = |x|ε(ω) has clearly all its paths invariant by s 0 , and

Y is a mod- ification of Y since ∀x ∈ D, P (Y x = Ỹx ) = P (ε = x) = 1. However, ω ∈ Ω / (∀x ∈ D, Y x (ω) = Ỹx (ω)) =
∈ D, Z Φ x = 1 2 (Z x + Z s(x) ) = 1 2 (Z (x 1 ,x 2 ) + Z (x 2 ,x 1 )
). The covariance kernel of this new process is given by

k Z Φ (x, x ′ ) = 1 4 [k Z (x -x ′ ) + k Z (s(x) -x ′ ) + k Z (x -s(x ′ )) + k Z (s(x) -s(x ′ ))]
One may note that here, by isotropy,

||(x 1 -x ′ 1 , x 2 -x ′ 2 )|| = ||(x 2 -x ′ 2 , x 1 - x ′ 1 )|| and ||(x 2 -x ′ 1 , x 1 -x ′ 2 )|| = ||(x 1 -x ′ 2 , x 2 -x ′ 1 )||, so that k Z Φ (x, x ′ ) = 1 2 e -||(x 1 -x ′ 1 ,x 2 -x ′ 2 )|| 2 + e -||(x 2 -x ′ 1 ,x 1 -x ′ 2 )|| 2
In more general cases (e.g., when Z has a geometrical anisotropy), however, it may be necessary to calculate a sum of four different terms.

3.2.

Kriging with an argumentwise invariant kernel. Let us now come back to our original prediction problem, and assume that we dispose of n noiseless observations

Y x i = y i (1 ≤ i ≤ n) of a square-integrable centered random field (Y x )
x∈D assumed invariant under the action Φ of a finite group G on D. As recalled in the introduction (Eq. 1), the function

m : x ∈ D -→ m(x) = k(x) T K -1 y,
is the Simple Kriging predictor (or "Kriging mean") of Y knowing the reponses at design points x 1 , . . . , x n . In addition, the Simple Kriging variance (or "Mean Squared Error") s 2 is often used as a quantifyer of m's accuracy:

s 2 : x ∈ D -→ s 2 (x) = k(x, x) -k(x) T K -1 k(x). (16)
It is well known that m interpolates the observations and s 2 vanishes at the design of experiments. As we will see now, more can be said in the case where k is argumentwise invariant. (2) basically relies on (1) in the cases where x = g.x i (i ∈ {1, . . . , n}).

In order to generalize to the conditional distribution of Y knowing

Y x i = y i (1 ≤ i ≤ n),
we can start by looking at its conditional covariance:

cov(Y x , Y x ′ |Y X = y) = k(x, x ′ ) -k(x) T K -1 k(x ′ ). (17)
In the case where Y is assumed Gaussian, the Simple Kriging mean and variance at x coincide respectively with the conditional expectation and variance of Y x knowing the observations. In addition, the Gaussian assumption makes it possible to get conditional simulations of Y , relying only on the conditional mean function and covariance kernel. The following property will play a crucial role in the applications discussed in the next section. (1) follows from the invariance of k(.) applied to Eq.17. For (2), it is useful to recall that conditional simulations are paths drawn from the conditional distribution of the considered field. Now, conditionally on the observations, this field has a mean function (the Kriging mean m) known to be Φ-invariant according to Prop.

f (g.x) = f, k(g.x, .) H = f, k(x, .) H = f (x)
It clearly appears from that representation that the left invariance is sufficient. This is of course related to the fact that we work here with symmetric kernels in the first place (in the sense that k(x, x ′ ) = k(x ′ , x)). For the reciprocal, assuming that any f ∈ H is invariant, it is straightforward that all k(x, .)'s (x ∈ D) are invariant since they belong to H. Hence,

k(g.x, g ′ .x ′ ) = k(g ′ .x ′ , .), k(g.x, .) H = k(x ′ , .), k(x, .) H = k(x, x ′ ),
which proves the argumentwise invariance of k.

Remark 6. In the case where the Mercer theorem applies, the property speaks for itself. k then possesses an orthogonal expansion of the form

k(x, x ′ ) = +∞ i=1 λ i e i (x)e i (x ′ ) ( 18
)
where the eigenfunctions e i (.) form an orthonormal basis of L 2 (D). Since the e i (.)'s are in the RKHS, they are invariant themselves, and it then appears directly that k is argumentwise invariant. straightforward way to symmetrize B is to construct SB (1) as follows:

Applications

SB (1) t = B π A (t) = B |t| (19)
The resulting process is still centered and Gaussian, with covariance

k SB (1) (t, t ′ ) := cov(SB (1)
t , SB

t ′ ) = cov(B |t| , B |t ′ | ) = min(|t|, |t ′ |) (1)
Now, as we have seen in Ex.4, another way of getting a process with symmetric paths based on B is by averaging it over the action's orbits:

SB (2) t := 1 2 (B t + B s(t) ) = 1 2 (B t + B -t ) (21)
In that case, following the way B was extended, we thus have

SB (2) t = 1 2 B |t| , (22)
so that k SB (2) = 1 4 k SB [START_REF] Abrahamsen | A review of Gaussian random fields and correlation functions[END_REF] . Simulated paths of the centered Gaussian process characterized by Eq. 20 are represented on Figure 1. Let us now consider an Orstein-Uhlenbeck (OU) process (Z t ) t∈D restricted to D := [0, 1], and s : t ∈ D -→ 1 -t ∈ D the symmetry with respect to 1 2 . This time, we choose A := [0, 1 2 ] as fundamental domain. A similar construction as for the first symmetrized BM leads to the process This centered Gaussian process is then characterized by the kernel

Y (1) t = Z π A (t) = Z min(t,s(t)) = Z min(t,1-t) (23)
k Y (1) (t, t ′ ) := cov(Z min(t,1-t) , Z min(t ′ ,1-t ′ ) ) = exp -| min(t, 1 -t) -min(t ′ , 1 -t ′ )| (24)
On the other hand, the second symmetrized OU process is obtained by averaging over the orbits of the considered group action:

Y (2) t = 1 2 (Z t + Z s(t) ) = 1 2 (Z t + Z 1-t ), (25)
and possesses the following covariance kernel:

k Y (2) (t, t ′ ) = 1 4 cov(Y t + Y 1-t , Y t ′ + Y 1-t ′ ) = 1 4 exp -|t -t ′ | + 1 4 exp -|(1 -t) -t ′ | + 1 4 exp -|t -(1 -t ′ )| + 1 4 exp -|(1 -t) -(1 -t ′ )| = 1 2 exp -|t -t ′ | + 1 2 exp -|1 -t -t ′ | (26)
Simulated paths of the centered Gaussian process defined by both eq. 24 and eq. 26 are represented on figure 2.

4.1.2. Conditional simulations of an invariant Gaussian Process. We now assume that the invariant process Y (2) was observed at the 3 points t 1 = 0.6, t 2 = 0.8, t 3 = 1, with response values y 1 = -0.8, y 2 = 0.5, y 3 = 0.9. The covariance kernel of eq. 26 is used for performing simulations of Y (2) conditionally on the latter observations. 20 such conditional simulations are represented on Figure 3. As can be seen on Figure 3, all paths are simultaneously Φ-invariant and interpolating the conditioning data, hence illustrating Property 3.3 on the conditional distribution of Gaussian Random Fields with argumentwise invariant kernel.

4.2.

Kriging with an invariant kernel. Let us now apply Kriging with an argumentwise invariant kernel to a benchmark example from the structural reliability literature exhibiting obvious symmetries.

Quoting [START_REF] Bect | Sequential design of computer experiments for the estimation of a probability of failure[END_REF] in which this test-case was recently used, "the example has been analyzed by [START_REF] Waarts | Structural reliability using finite element methods: an appraisal of DARS[END_REF] and [START_REF] Echard | Kriging-based monte carlo simulation to compute the probability of failure efficiently: AK-MCS method[END_REF] made a comparison with several meta-models proposed by [START_REF] Schueremans | Benefit of splines and neural networks in simulation based structural reliability analysis[END_REF]". The limit state function of interest reads:

y : (x 1 , x 2 ) ∈ [-5, 5] 2 -→ min        3 + 0.1(x 1 -x 2 ) 2 -(x 1 + x 2 )/ √ 2 3 + 0.1(x 1 -x 2 ) 2 + (x 1 + x 2 )/ √ 2 (x 1 -x 2 ) + 6/ √ 2 (x 2 -x 1 ) + 6/ √ 2       
Figure 4 shows the contours of y, with an illustration of the three non-trivial transformations of R 2 -denoted by s 1 , s 2 , s 3 -leaving y invariant. Actually, y can be shown to be left invariant by an action of the group (Z/2Z) 2 on R 2 . Indeed, as illustrated on Figure 4, y is invariant under s 1 , the axial symmetry with respect to the first bisector. y is also invariant under s 2 , the axial symmetry with respect to the second bisector. Finally, y is obviously invariant under their composition, s 3 , i.e. the symmetry with respect to the origin. Together with the identity of R 2 , denoted by s 0 , the latter s 1 , s 2 , s 3 forms a group of order 4, representing (Z/2Z) 2 on R 2 .

4.2.1.

Comparing three Kriging models based on different kernels. Here we investigate using argumentwise invariant kernels for approximating this function by Simple Kriging based on 30 observations at a maximin LHS Design X. The underlying Design of Experiments is generated using the R package lhs. As a preliminary step towards a comparison between different kernels, a classical Simple Kriging model with a tensor product OU kernel

k Z (x, x ′ ) = σ 2 exp - 1 θ |x 1 -x ′ 1 | + |x 2 -x ′ 2 | + τ 2 1 x=x ′ ( 27
)
is fitted to the data (see Figure 5). Here the parameter are fixed to their Maximum Likelihood estimates, σ 2 = 7.5 and θ = 20. In addition, a nugget effect with τ 2 = 0.01 is added to k Z for numerical purposes. We now consider two different argumentwise invariant kernels. To start with, using similar notations as for the 1-dimensional OU example, we define a fundamental domain for Φ (see Figure 6 for an illustration): The first argumentwise invariant kernel considered is then constructed based on the projector The second argumentwise invariant kernel considered is then constructed by averaging k Z over the orbits of Φ:

A := {x ∈ [-5, 5] 2 : x 1 ≥ 0, -x 1 < x 2 ≤ x 1 } (28)
π A : x ∈ D -→ π A (x) = O(x) ∩ A ∈ A, as follows: k Y (1) (x, x ′ ) := k Z (π A (x), π A (x ′ )) (29)
k Y (2) (x, x ′ ) := 1 16 3 i=0 3 j=0 k Z (s i .x, s j .x ′ ) (30)
The results of Kriging with kernels k Y (1) and k Y (2) based on the observations at X are illustrated on Figures 7 and8, respectively. Finally, for comparison, a Kriging model with regular OU kernel (the same as for the first model) but based on the design

X sym := 3 i=0 s i .X (31)
and with the observations at X replicated four times is considered.

-

m with OU kernel and symmetrized LHS design -

s with OU kernel and symmetrized LHS design 4.2.2. Discussion on the compared results. In order to compare the prediction abilities of the considered Kriging models, we predicted y at a 50 × 50 out-of-sample validation design X val using the four models, and compared the average prediction errors and the residuals. Figure 10 represents the mean predictions against reality (first line) and the standardized residuals

y(x (i) val ) -m(x (i) val ) s(x (i) val ) (1 ≤ i ≤ 2500) (32)
Looking at the values of the Integrated Squared Error (ISE) at X val , ISE = 2500 j=1 (y(x

(i) val ) -m(x (i) val )) 2 (33)
for the four candidate Kriging models, we first see that the first model is undoubtly dominated by the three other ones. This was to be expected since the first model is the only one which doesn't take into account the symmetry of the problem. The second model, based on a combination of the OU kernel with the projector onto the fundamental domain of Figure 6, shows significantly better performances. Indeed, the ISE drops from 433.01 to 264.33, just by playing on the underlying kernel. However, the performances of the third model, with the kernel averaged over the action's orbits, are even better. Not only does the ISE drop to 142.89, but the order of magnitude of the standardized residuals is more in accordance with what one usually expects when Kriging under the Gaussian Process assumption (even if this is not really theoretically well-founded without further ergodicity assumptions, it is customary to expect that about 95% of the sample of standardized residuals lie between the 2.5% and 97.5% quantiles of the standard Gaussian distribution, see for instance [START_REF] Jones | Efficient global optimization of expensive black-box functions[END_REF]).

Perhaps surprisingly, the last model obtained by using a regular covariance kernel with a symmetrized design gave here better performances in terms of ISE (124.53) than the two previous models with argumentwise invariant kernels. This has to be tempered by the fact that doing it this way multiplies the dimension of the covariance matrix by the order of the group (i.e. 4 here), that is to say that the total number of coefficients jumps from n 2 to n 2 × r 2 (i.e. from 900 to 14 ′ 400 here). Hence, replicating the design is likely to cause problems in terms of matrix inversion, and even in terms of data storage (for the reasonable values n = 1000 and r = 8, n 2 ×r 2 = 64 ′ 000 ′ 000). Furthermore, the test function studied here is not very smooth (so that an OU kernel was considered instead of a Gauss or a Matérn one, more commonly used in smoother cases), which may relatively hinder the benefits of taking symmetries into account, since the latter come in more regular cases with additional smoothness properties on the axes of symmetry. Concerning the second model, let us also remark that the choice of A is arbitrary, and not always without consequences on the model obtained. In the case of an anisotropic covariance, for instance, choosing the current A or its image by a rotation of center 0 and angle π 2 may lead to substantially different predictions. This has to be studied in more detail in further works.

To finish with this application, let us point out the fact that among the considered models, only the ones based on an argumentwise invariant covariance kernel enables conditional simulations with invariant paths. 4 such simulated paths with the kernel of Eq. 29 conditional on the observations at X are represented on Figure 11.

4.3.

Kriging with a wrong kind of invariant kernel. We now simulate realizations of a two-dimensional centered Gaussian Random Field with argumentwise invariant kernel, and compare the predictive performances obtained by using Kriging models with different configurations:

-Model A: Symmetrized kernel obtained by projection over a fundamental domain (analogue of the kernel defined in Eq. 29)

-Model B: Symmetrized kernel obtained by double sum over the orbits (analogue of the kernel defined in Eq. 30)

-Model C: Stationary kernel with symmetrized design (Same design as defined in Eq. 31)

However, contrarily to the previous example and in order to investigate a different class of fields for which invariances and the different kinds of symmetrization may have a more crucial impact, we chose here for the stationary kernel k Z underlying the three models above an anisotropic Gaussian kernel:

k Z (x, x ′ ) = σ 2 exp - (x 1 -x ′ 1 ) 2 20 + (x 2 -x ′ 2 ) 2 10 + τ 2 1 x=x ′ , (34)
still with σ 2 = 7.5 and τ 2 = 0.01. The same design of experiments X (up to a symmetrization in the case of model C) was used for the three models. By the way, we took the same 30-points LH Design as for the previous example.

Before analyzing statistical performance results, let us focus on the particular example addressed in Figure 12. Here, as in the other results presented in this section, the underlying model of the simulated Gaussian Field is Model B. On the right hand side of Figure 12, in the first line, one can observe that the Kriging mean surfaces obtained with the three different kernels look rather similar. Looking at the Integrated Square Error values, however, one superiority of Model B in coverage (D n statistic) clearly appears, the departure from normality of the standardized residuals being much larger for Model A and Model C. However, note that, even for Model B, the empirical distribution of standardized residuals slightly differs from a standard Gaussian one: The D n values are often higher than expected at confidence levels 0.05 or 0.01 (Figure 13, right hand side, horizontal dotted lines). The superiority of Model B is also confirmed in prediction (ISE criterion). Visible on the boxplots by checking that the notches are not overlapping (Figure 13, left), it can be checked in a quantitative way by means of a two-sample Wilcoxon test: For instance, the null hypothesis "ISE (Model B) = ISE (Model C)" with the one-sided alternative "ISE (Model B) ≤ ISE (Model C)" is strongly rejected with a p-value of 4.344e -06.

These results confirm the first impressions left by the the particular case commented above and illustrated in Figure 12: The performances of Kriging with an argumentwise invariant kernel are sensitive to the adequacy of the kernel chosen to the kind of invariant process underlying the data, at the same time in terms of prediction accuracy and of coverage. In particular, approximating an invariant Gaussian Field with a Kriging model relying on a symmetrized kernel obtained by projection over a fundamental domain may give very different results than using a "double sum over the orbits" kernel, even if the underlying stationary kernel k Z is the same in both cases.

Conclusion and perspectives

We proposed a class of covariance kernels, called argumentwise invariant kernels, characterizing (up to a modification) squared integrable random fields with invariant paths under an arbitrary action of a finite group on the index set, as well as Reproducing Kernel Hilbert Spaces of invariant functions (still with a finite group acting on the source space).

These kernels can be used for different purposes. We focused here on modeling invariant functions by Kriging. As discussed along the paper, Kriging models with an argumentwise invariant kernel have interesting properties, including the invariance of both Kriging mean and variance functions, but also the invariance of paths emanating from conditional simulations.

Among the two variants for making up invariant kernels based on arbitrary kernels proposed in the last section, summing a kernel over the orbits of the considered group action gave more convincing results than composing the basis kernel with a projection onto a fundamental domain. However, this may not hold in the general case, and further works may focus on identifying and unlocking the potential weak points of both considered approaches.

Acknowledgements: The authors would like to thank Anestis Antoniadis for a decisive question on the non necessary symmetric paths of a version of a process with symmetric paths, and Yann Richet (IRSN) for having provided them with a physical application rising the question of embedding symmetry properties within Kriging. Many thanks as well to Alain Valette for his precious advise in algebra, and to Cédric Boutillier who helped improving a former version of the present paper. Last but not least, this paper wouldn't exist in its present without the contribution of Yves Deville, who helped creating a replicate of the DiceKriging package allowing to extend the use of the prediction and simulation methods to arbitrary classes of kernels. Finally, the authors would like the anonymous referee and associate editor for their constructive suggestions and comments.

Definition 3 .Definition 4 . 3 ) 5 . 4 )

 34354 x ∈ D is a fixed point of the action when ∀g ∈ G, g.x = x. The fixator of a set S ⊂ D in G is defined by Fix Φ (S) := {g ∈ G | ∀x ∈ S, g.x = x}. (Definition The stabilizer of a set S ⊂ D in G is defined by Stab Φ (S) := {g ∈ G | ∀x ∈ S, g.x ∈ S} (Definition 6. A measurable set A ⊂ D is said to be a fundamental domain of Φ if it is a system of representatives of Φ's orbits. Remark 1. A fundamental domain is usually required to have further topolical properties, for instance to be the symmetric difference between an open set and a set of measure zero. Definition 7. Le F be an arbitrary set. A map y : D -→ F is said invariant by Φ, or invariant under the action of the group G, when ∀x ∈ D, ∀g ∈ G, y(g.x) = y(x)

Property 3 . 1 .

 31 (Kernels characterizing invariant fields) Let G be a finite group acting on D via the action Φ, and Y be a centered squared-integrable random field over D. Y has its paths Φ-invariant (up to a modification) if and only if its covariance kernel k is argumentwise invariant under Φ.Proof. Let us first assume that Y has its paths Φ-invariant, up to a modification. Then, there exist a process Y with Φ-invariant paths and such that ∀x ∈ D, P(Y x = Y x ) = 1. This implies that Y and Y are equivalent, and in particular k Y = k Y since the 2-dimensional distributions are the same. Now, by Φ-invariance of Y 's paths, we have ∀x

Property 3 . 2 .

 32 (Properties of m and s 2 when k is argumentwise invariant) (1) m and s 2 are invariant (2) ∀i ∈ {1, . . . , n}, ∀g ∈ G, m(g.x i ) = y i and s 2 (g.x i ) = 0. Proof. The covariance vector k(.) is invariant by argumentwise invariance of k. Plugging in the equality k(g.x) = k(x) in Eqs. 1 and 16, (1) follows.

Property 3 . 3 .( 1 )( 2 )

 3312 (Properties of the conditional distribution of a Gaussian Random Field with argumentwise invariant kernel) The conditional random field has an argumentwise invariant kernel All conditional simulations are Φ-invariant Proof.

4. 1 . 4 . 1 . 1 .

 1411 Invariant Brownian Motion and other elementary examples. Symmetrized BM and OU process. Let us first consider (B t ) t∈[0,+∞[ , a one-dimensional Brownian Motion (BM), and the symmetry with respect to the origin s : x ∈ D -→ -x ∈ D, where D := R. The corresponding action of the group G = Z/2Z on R is the same as in Ex. 1. In order to symmetrize B, let us first extend it to a process on the whole line by setting ∀t < 0, B t = 0. Now, relying on the fundamental domain A := [0, +∞[, a

Figure 1 .

 1 Figure 1. Symmetrization of the Brownian Motion relying on the symmetrized kernel (by projection on a fundamental domain) of Eq. 20.

Figure 2 .

 2 Figure2. Symmetrization of the OU process relying on both kernels defined by Eqs. 24, 26. Left: by projection on a fundamental domain (Eq. 24). Right: by averaging over the orbits (Eq. 26).

Figure 3 .

 3 Figure 3. Conditional Simulations of the symmetrized OU process with the kernel of Eq. 26. The black points stand for the conditioning data.

Figure 4 .

 4 Figure 4. Borri and Speranzini's function, with its two axes of symmetry (black lines). The colored arrows stand for the three non-trivial transformations leaving this function unchanged.

Figure 5 .

 5 Figure 5. Simple Kriging mean and standard deviation with the regular tensor product kernel of Eq. 27, based on observations of y at X.

Figure 6 .

 6 Figure 6. Left: The fundamental domain A for the considered action of (Z/2Z) 2 on [-5, 5] 2 . The lower left boundary is excluded. Right: Projection of the design points of X (black points) onto A (red stars).

Figure 7 .

 7 Figure 7. Simple Kriging mean and standard deviation with the symmetrized OU kernel of Eq. 29, based on observations of y at X.

Figure 8 .

 8 Figure 8. Kriging mean and standard deviation with the symmetrized OU kernel of Eq. 30, based on observations of y at X.

Figure 9 .

 9 Figure 9. Kriging mean and standard deviation with the regular tensor product kernel of Eq. 27, based on observations of y at Xsym. The solid black circles represent the LHS design X. The red squares, blue diamonds, and green triangles represent respectively the orbits of X under the transformations s1, s2, and s3.

Figure 10 .

 10 Figure 10. Comparison of prediction results at X val when using the 4 Kriging models considered for Borri and Speranzini's function.

Figure 11 .Figure 12 .

 1112 Figure 11. Four conditional simulations of Borri and Speranzini's function with symmetrized OU kernel (1), based on observations at X.

Figure 13 .

 13 Figure 13. Left: log-ISE values (at X val ) when using the three Kriging models A, B, C for predicting 100 Gaussian Field realizations generated with Model B. Right: Kolmogorov-Smirnov test statistic values for comparing the standardized residuals to a standard Gaussian distribution.

  , Y x ′ ) is a function of xx ′ (here x, x ′ ∈ D) or, in other words, when for any x ∈ D and h such that x + h ∈ D, cov(Y x+h , Y x ) depends only on h and not on x. Equivalently, the covariance kernel of Y is such that for any translation T h (x) := x + h and pair of points x, x ′

	2.3. Classical results about invariant kernels and random fields.
	2.3.1. Stationarity, isotropy: Invariance-related notions in geostatistics. A
	very classical notion in spatial statistics, and more generally in the literature
	of random processes (including time series in the first place), is the one of
	second order or weak stationarity. A centered squared-integrable random
	field Y is said weakly stationary whenever cov(Y x
	, p. 18), if Y and Y ′ are modifications of each other,
	they clearly are equivalent. A slightly less straightforward result is that if
	two random fields modifications of each other are almost surely continuous,
	then they are indistinguishable. Finally, let us add a definition which will
	play a central rôle in the sequel of the paper:
	Definition 10. Y is said to have all its paths Φ-invariant whenever
	∀ω ∈ Ω, ∀x ∈ D, ∀g ∈ G, Y x (ω) = Y g.x (ω)	(9)

  1 2 is negligible, and the two processes are hence not indistinguable.Example 4. Let us come back to the notations of example 2. One can construct a process having its paths Φ-invariant based on Z by defining ∀x

  3.2. Since the complement to this mean function is a centered Gaussian Field with argumentwise invariant kernel (from (1)), Prop. 3.1 implies that the conditional simulations are Φ-invariant, as sums of a Φ-invariant function plus Φ-invariant paths.Remark 5. In practice, the paths of Y are often simulated at a finite set of points X simu = {e 1 , ..., e m } ⊂ D based on a matrix decomposition (Cholesky, Mahalanobis) of K = (k Y (e i , e j )) 1≤i,j≤m . The Φ-invariance of the vectors simulated that way is thus sure (i.e. ∀ω ∈ Ω).3.3. What about the RKHS point of view?As a closure to the present section on the main results of the paper, let us briefly discuss the interplay between the argumentwise invariance of a p.d. kernel and the invariance of elements from the naturally associated RKHS of real-valued functions.

Property 3.4. The Reproducing Kernel Hilbert Space (H, ., . H ) with reproducing kernel k has all its functions Φ-invariant if and only if k is argumentwise invariant under Φ.

Proof. If k is argumentwise invariant and H is a RKHS of real-valued functions with kernel k, it is clear that any function f ∈ H is invariant under Φ. Indeed, taking arbitrarily x ∈ D and g ∈ G, we get

Résumé. Nous considérons le problème d'approximation par méthodes à noyaux de fonctions invariantes sous l'action d'un groupe fini. Nous introduisons les noyaux doublement invariants, et montrons qu'ils caractérisent les champs aléatoires centrés de carré intégrable à trajectoires invariantes, ainsi que les espaces de Hilbert à noyau reproduisant de fonctions invariantes. Deux classes particulières de noyaux doublement invariants sont considérées, basées respectivement sur un domaine fondamental ou sur une double somme sur les orbites. Nous établissons ensuite des propriétés d'invariance pour les modèles de Krigeage et les simulations consitionnelles associés. L'applicabilité et les avantages de tels noyaux sont illustrés sur plusieurs exemples, incluant une fonction symétrique issue d'un problème de fiabilité des structures. 1