Merci Béa

Merci Pierre-Louis

Merci Surtout À Jean

Tristan

Apo Bastien

Marie Merci

Victor

Grazie Angelino

Franco

Denis Allard
email: denis.allard@inrae.fr

Lucia Clarotto

Xavier Emery

T Opitz

T Romary

Fully nonseparable Gneiting covariance functions for multivariate space-time data

Keywords: Spatio-temporal modeling Matrix-valued covariance function Pseudo-variogram Matérn covariance Spectral simulation Composite likelihood, Block likelihood, Block approach, Tuckey g and h, Vecchia

Thank you Geir-Arne Fuglstad for motivating some of the research paths of this last year. Thank you Liliane Bel for being a fixed point in the French spatial statistics community.

These last three years have been a great journey. As always in my life, I rather travel in good company than alone. Here is my gratitude to all the people that walked by my side to get me where I am.

Merci Denis, parce que tu as vu une étincelle en moi et que tu ne l'as jamais laissé s'éteindre. Merci pour l'accueil chaleureux chaque fois que je passe par Avignon et pour m'avoir si bien comprise. Tu es un point fixe de ma carrière scientifique et j'espère que tu le seras encore longtemps.

Merci Thomas, pour ta présence constante. J'aime la précision et l'engagement que tu mets dans ton travail. Merci pour avoir créé les conditions de travail idéales et pour avoir été toujours de mon côté.

Merci Nico, pour le côté fou et en même temps génial de ton travail. J'adore l'énergie que tu mets dans les projets qui t'intéressent, et l'éclat qu'on voit dans tes yeux quand tu as une idée novatrice. Merci les doctorants (et ex) du centre de géosciences, vous faites du centre un lieu vivant et joyeux. Alexis, pour nos visions si différentes du monde, mais en même temps très ouvertes à la découverte. Charlie et Ferdinand, pour faire vivre le bureau. Antoine, Nicolas, Fabien, Léo, Jean et les autres, pour tous les moments agréables ensemble. Je ne suis pas devenue une géologue, mais au moins je sais distinguer le basalte du grès. :D Thank you Martin, to light the team up this last spring. Merci l'équipe Géostat, vous avez rendu heureux même mes aller-retours à Bleau. Nath, pour le "cafééé" de 10h et pour tout ce que tu fais pour notre bien-être. Didier, pour tes blagues et pour ton faux R roulé. Lantu, pour ne jamais nous laisser seuls au labo et pour ton dévouement infini à la recherche. Mike, pour avoir rajeuni l'équipe (ooops) et pour m'aider à résoudre les problèmes mathématiques trop complexes pour moi. Fabien, pour toute l'aide informatique indispensable, et pour être toujours aussi gentil même à la n-ème fois que je te demande comment installer les logiciels. Xavier, Émilie, Jacques, Chantal pour les pauses café, moment essentiel de la vie de l'équipe. Merci Paris, parce que tu m'as conquise avec les ruelles cachées aux touristes, les coins verts que je découvre encore comme une petite fille, la culture et la beauté qu'imbibent tes veines et la variété des gens qui t'habitent. Merci la Rooft', l'immeuble le plus cool de Paris, parce que vous m'avez accueillie il y a trois ans et m'avez accompagnée tout au long de ce voyage. Étienne, pour la naturalité des moments qu'on passe ensemble, pour ton côté bordélique et désordonné qui me fait tant marrer, pour les mille inspirations de nos discussions. Val, pour l'air agréable que tu fais vivre dans l'appart, pour ton duo de personnalité que j'ai découvert au fur et à mesure et pour les folles soirées de la Driiing. Jenn, pour nos brunchs, nos séances de ciné et nos peintures, et pour ton sourire toujours sincère. Gwen, pour nos parties aux jeux de société et pour tes enseignements écologiques et engagés. Juliette, pour nos soirées ensemble, qu'elles soient disco ou tisane. Robin, pour Disneyland et pour tes couscous. Et tous les autres… sans oublier Edit, pour les chaussons en cadeau, les douces lettres et les pots des fleurs! :D Merci au Buccrew, pour avoir accepté une fille dans votre bande de fous, pour perdre aux jeux de société avec moi, pour faire semblant que je joue suffisamment bien au tennis, pour les soirées incroyables dans la forêt.

i "Although this may seem a paradox, all exact science is dominated by the idea of approximation. When a man tells you that he knows the exact truth about anything, you are safe in inferring that he is an inexact man." 6.9 Wind residuals at 6 different spatial locations during January 1-4, 2016.

Bertrand Russell

List of Figures

6.10 Predictions of W at (T + 1), (T + 2) and (T + 3) with model [START_REF] Predictions Of | to January 31, 2003 following the Spatial Interpolation setting. Predictions (continuous line), real values (dots) and envelope of ±2 conditional standard deviations[END_REF] ("adv-diff" with α = 1 and α S = 2). . . . . . . . . . . . . . . . . . . . 6.11 Difference between MSE of model (3) (regular mesh with N S = 900) and MSE of persistence model at (T + 1), (T + 2) and (T + 3). . . . .

Difference between MSE of model (3) (handmade mesh with N S =)

and MSE of persistence model at (T + 1), (T + 2) and (T + 3). . . . .

List of Symbols

∈ R d • Euclidean distance R d → R t temporal location ∈ R t i temporal location of observation i ∈ R u = |t -t | temporal distance ∈ R E(•) expectation Cov(•, •) covariance C ST (•, •) stationary spatio-temporal covariance function (R d × R) × (R d × R) → R C S (•, •) stationary spatial covariance function R d × R d → R C T (•, •) stationary temporal covariance function R × R → R N S number of spatial mesh points ∈ R N T number of time steps ∈ R n number of observations ∈ R n S number of spatial observations ∈ R X(•, •) spatio-temporal GRF with zero-mean R d × R → R Y (•, •) spatio-temporal stochastic process R d × R → R µ(•, •) spatio-temporal trend R d × R → R W (•, •) spatio-temporal Gaussian white noise R d × R → R W S (•) spatial Gaussian white noise R d → R W T (•) temporal Gaussian white noise R → R L spatio-

Spatio-temporal statistics

Many areas of environmental science seek to predict a space-time variable of interest from observations at scattered points in the space cross time domain of study, e.g., among other possible applications, wind prediction [START_REF] Lenzi | Spatiotemporal probabilistic wind vector forecasting over Saudi Arabia[END_REF][START_REF] Huang | Forecasting high-frequency spatio-temporal wind power with dimensionally reduced echo state networks[END_REF], precipitation forecasting [START_REF] Sigrist | An autoregressive spatiotemporal precipitation model[END_REF], urban air quality inference [START_REF] Paciorek | Practical large-scale spatio-temporal modeling of particulate matter concentrations[END_REF], climate models inference [START_REF] Genton | Cross-Covariance Functions for Multivariate Geostatistics[END_REF][START_REF] Edwards | A Multivariate Global Spatiotemporal Stochastic Generator for Climate Ensembles[END_REF], meteorological predictions (Bourotte et al., 2016;[START_REF] Allard | Fully nonseparable gneiting covariance functions for multivariate space-time data[END_REF], data assimilation in oceanography [START_REF] Bertino | Sequential data assimilation techniques in oceanography[END_REF]. A more detailed list of references is available in Porcu et al. (2021).

On the one hand, when data are dense in time and sparse in space, one can work within the framework of multivariate time series. For example, this is the case of high frequency measurements arising from a small set of sensors. On the other hand, when data are dense in space and sparse in time, the analysis can be done in a multivariate geostatistical setting, where repetitions in time are treated as different variables. When data are dense both in time and space or when they show complex spatio-temporal correlations, these approaches fail due to the inability to build very large multivariate temporal or spatial processes. A much more flexible and richer approach is to consider spatio-temporal models.

Even if data for environmental processes are often taken at fixed locations and times, the processes themselves are generally defined on continuous sets in space and time. Thus, the models are defined on R d × R, where d is the spatial dimension of the process. When dealing with a spatial domain defined on a sphere, R d must be replaced by S d , the d-dimensional sphere. In this work we will consider only spatio-temporal models whose spatial domain is the d-dimensional Euclidean space R d .

We denote Y (s, t) the value of the spatio-temporal random field at location s ∈ R d and time t ∈ R. Mathematically, one may consider Y (s, t) as a random function in R d+1 and apply geostatistics techniques as usual. This approach is mathematically correct, but it misses a key point, which is the fact that time flows only in one direction. Good statistical models should take this asymmetry into account.

We assume that the spatio-temporal field Y can be decomposed as a sum of fixed and random effects Y (s, t) = µ(s, t) + X(s, t), (

where µ(s, t) = E[Y (s, t)] is the spatio-temporal trend, i.e., the deterministic part.

In this work, we will always assume, unless stated explicitly otherwise, that X(s, t) is a zero-mean second-order spatio-temporal random field, i.e., that its covariance function is

C ST (h, u) = Cov[X(s, t), X(s + h, t + u)] = E[X(s, t)X(s + h, t + u)],

(1.2) with (s, t) and (s + h, t + u) ∈ R d × R. This statement assumes that all the nonstationarities of the process are accounted for in the deterministic part, which is a quite difficult hypothesis to verify. Under the assumption of a Gaussian process X, the mean and covariance functions completely characterize its distribution (see e.g. [START_REF] Chilès | Geostatistics: Modeling Spatial Uncertainty[END_REF]).

The spatio-temporal field Y (s, t) is sampled at locations and times (s i , t i ), i = 1, . . . , n where n is the number of space-time data. Ideally, measurements are available for all times and locations, but very often data are missing for some times at some locations. Predicting the spatio-temporal field Y (s, t) at an unsampled spatiotemporal coordinate (s 0 , t 0 ) ∈ R d × R from available measurements {Y (s i , t i )} i=1,...,n requires a model for the trend µ(s, t) and a covariance model for the random component X(s, t). Several possibilities exist for modeling the deterministic part, including parametric or non-parametric, linear or non-linear models, with or without covariates.

A major step in fitting a Gaussian model to data is the estimation of the covariance function of X (s, t). Covariance functions are a measure of the linear dependence between the random variables X at the space-time locations (s, t) and (s + h, t + u).

They allow computing the covariance between any couple of points X(s i , t i ) and X(s j , t j ), hence they must be positive definite functions. Conversely, any positive definite function is the covariance function of a Gaussian process. This necessary and sufficient condition limits the choice of available models and makes the construction of models with realistic features difficult. The definition of spatio-temporal covariance functions that appropriately capture complex interactions between space and time has become a very active field of research. Important developments in this field are detailed in Section 1.2.

If X is a spatio-temporal real stationary Random Field (RF) with covariance function C ST , then, for every t ∈ R, the spatial RF X(•, t) is a stationary spatial RF, called the spatial trace of X. All spatial traces of X are stationary with same stationary spatial covariance function given by C S (h) = C ST (h, 0) for all h ∈ R d .

Analogously, for every s ∈ R 2 , the temporal RF X(s, •) is called the temporal trace of X, and all the temporal traces of X are stationary temporal RF with same covariance function given by C T (u) = C ST (0, u) for all u ∈ R.

Among modern techniques proposing efficient methods for estimation and prediction in a spatio-temporal framework, there is a distinction between two possible ways of constructing and treating spatio-temporal models [START_REF] Wikle | A general science-based framework for dynamical spatio-temporal models[END_REF]: either one follows the traditional geostatistical paradigm, using joint space-time covariance functions (see for example [START_REF] Cressie | Classes of nonseparable, spatio-temporal stationary covariance functions[END_REF], Gneiting (2002), [START_REF] Stein | Space-time covariance functions[END_REF], as well as the recent reviews Porcu et al. (2021), Chen et al. (2021)), or one uses dynamical models, including functional time series of surfaces or solutions to Stochastic Partial Differential Equations (SPDEs), see for example [START_REF] Wikle | A dimension-reduced approach to space-time kalman filtering[END_REF], [START_REF] Sigrist | A dynamic nonstationary spatio-temporal model for short term prediction of precipitation[END_REF] and [START_REF] Martínez-Hernández | Surface time series models for large spatio-temporal datasets[END_REF]. In this work, we will propose two approaches to construct spatio-temporal models: one, the unsteady advection-diffusion SPDE, belongs to the class of dynamical models; the other, the nonseparable Gneiting class, belongs to the geostatistical class of models.

Nonseparability and asymmetry

Since it is often difficult to scrutinize spatial and temporal variations simultaneously, it is tempting to focus on C ST (0, •), i.e., how the covariances at a single location vary across time, and C ST (•, 0), i.e., how the covariances at a single time vary across space. If these were the only variations that mattered, then separable models would suffice. A space-time covariance function is space-time separable if it is the product of a spatial covariance function with a temporal covariance function, i.e.

if C ST (h, u) = C ST (0, 0) -1 C S (h) C T (u). This is a valid covariance function, since when C S is positive definite on R d and C T is positive definite on R, then the product C S C T is positive definite on R d × R. The advantage of separability lies in the fact that it leads to a reduced number of parameters, allows separate estimation of the parameters relating to space and time and provides faster prediction techniques for a constant spatial design, thanks to fast computations of the inverse and of the determinant of the matrix involved in estimation and kriging.

However, using only products of spatial covariances and temporal covariances to define the spatio-temporal covariances is a severe restriction. Separable space-time covariance functions have often been used to take advantage of their computational convenience, even when they are not realistic in describing the processes due to the absence of space-time interaction. Separability implies that small changes in the locations of observations can lead to large changes in the correlations between linear combinations of observations [START_REF] Stein | Interpolation of spatial data[END_REF]. The source of this behavior is the fact that separable covariance functions can be rougher away from the origin than at the origin [START_REF] Stein | Interpolation of spatial data[END_REF]. For this reason, it is necessary to seek space-time covariance functions that are smooth everywhere except possibly at the origin, i.e., when h = 0 and u = 0. Moreover, it can be interesting to find models that allow different degrees of smoothness across space than across time. These models are called nonseparable.

A space-time covariance C ST is said to be positively (resp. negatively) space-time

nonseparable if s(h, u) = C ST (h, u)C ST (0, 0) -C S (h) C T (u) ≥ 0 (resp. ≤ 0).
(1.3)

If such inequalities hold for all (h, u) ∈ R d ×R, then C ST is called uniformly positively (resp. negatively) nonseparable (Rodrigues and Diggle, 2010;De Iaco and Posa, 2013).

The function s(h, u) will be referred to as separability function. Their empirical versions can be used as exploratory tools to characterize the type of nonseparability in a dataset. In most applications, nonseparable models show better predictions than separable models, since they are physically more realistic, albeit computationally more expensive.

Nevertheless, it is difficult to write down directly nonseparable space-time covariance functions. Recent studies have focused on constructing nonseparable models, see Gneiting (2002), De Iaco et al. (2003), [START_REF] Porcu | Nonseparable stationary anisotropic space-time covariance functions[END_REF], [START_REF] Salvaña | Lagrangian Spatio-Temporal Nonstationary Covariance Functions[END_REF], as well as Bourotte et al. (2016) and [START_REF] Allard | Fully nonseparable gneiting covariance functions for multivariate space-time data[END_REF] in a multivariate context.

Nonseparable space-time covariance models can be constructed from Fourier transforms of permissible spectral densities [START_REF] Stein | Space-time covariance functions[END_REF], mixtures of separable models (De Iaco et al., 2003), and partial differential equations (PDEs) representing physical laws [START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF][START_REF] Lindgren | A diffusionbased spatio-temporal extension of gaussian matérn fields[END_REF]. They can be stationary or nonstationary, univariate or multivariate, in the Euclidean space or on the sphere.

See Porcu et al. (2021) and Chen et al. (2021) for recent comprehensive reviews.

A notion closely related to separability is that of full symmetry. A spatio-temporal covariance function is fully symmetric [START_REF] Gneiting | Geostatistical space-time models, stationarity, separability and full symmetry[END_REF] if

C ST (h, u) = C ST (-h, u) = C ST (h, -u) = C ST (-h, -u)
for all (h, u) ∈ R d × R. A fully symmetric model is unable to distinguish possibly differing effects as time moves forward or backward. Isotropy in space and symmetry 1.3. Physics-informed statistics in time imply full symmetry. Separable covariance functions are also fully symmetric, but not vice versa. Product-sum models (De Iaco et al., 2003) and the Gneiting class (Gneiting, 2002) are fully symmetric, nonseparable (uniformly negatively nonseparable and uniformly positively nonseparable, respectively) covariance functions (Rodrigues and Diggle, 2010).

Atmospheric or environmental processes are often under the influence of air or water flows, which are incompatible with full symmetry [START_REF] Gneiting | Geostatistical space-time models, stationarity, separability and full symmetry[END_REF]. The Lagrangian reference framework, here detailed, is useful in this case. Transport effects can be modeled with the help of a stationary spatial Gaussian random field and a ddimensional velocity vector γ, possibly random. The resulting space-time covariance, which is stationary, but not fully symmetric, is defined as

C ST (h, u) = C ST (h -γ u, u).
This idea was applied to precipitation fields by [START_REF] Cox | A simple spatial-temporal model of rainfall[END_REF] and Benoit et al. (2018a), to wave heights by [START_REF] Ailliot | Spacetime models for moving fields with an application to significant wave height fields[END_REF] and to wind by [START_REF] Salvaña | Lagrangian Spatio-Temporal Nonstationary Covariance Functions[END_REF]. It will also be discussed in Section 2. 3.3. Covariance functions that are not fully symmetric can also be constructed based on advection-diffusion equations or SPDEs, as first discussed in [START_REF] Gneiting | Geostatistical space-time models, stationarity, separability and full symmetry[END_REF] and later in [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF]; [START_REF] Sigrist | Stochastic partial differential equation based modelling of large space-time data sets[END_REF]; Carrizo- [START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF]. This construction will be used in Chapter 2.

Physics-informed statistics

Understanding real-world dynamical phenomena is a challenging task. Across various scientific disciplines, machine learning has been used to analyze big data and dynamical systems, and it has become a powerful tool to make data-driven decisions based on the input data alone. However, real-world data are stochastic, noisy, and incomplete, and contain uncertainty. Statistical methods and machine learning prediction approaches ignore the fundamental laws of physics and often, although they perform well at fitting observations, they fail to make plausible predictions.

In the last years, researchers have been trying to integrate data, statistical models and physics to improve the predictive potential of traditional methods. The idea is to introduce the prior knowledge about the dynamical models into the statistical approach, in order to respect underlying physical principles. The Physics-Informed Machine Learning (PIML) models [START_REF] Raissi | Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[END_REF][START_REF] Karniadakis | Physics-informed machine learning[END_REF] often perform well for both interpolation and extrapolation, even for a small amount of noisy, missing or sparse data. The physical prior information can be derived either by Partial Differential Equations (PDEs), Ordinary Differential Equations (ODEs)

and Stochastic Differential Equations (SDEs) or by intuitive physics, and it can be incorporated into different parts of machine learning models such as data, model architecture, loss function, optimizer and inference algorithm [START_REF] Linka | Bayesian physics informed neural networks for real-world nonlinear dynamical systems[END_REF].

In this specific work, we will not make use of the most common PIML object, which is the Physics-Informed Neural Network (PINN), that is designed to be trained to satisfy the given data as well as the imposed governing equations. Instead, we will apply the Physics-Informed framework in a more general meaning, by introducing physical information into a statistical model. In this particular case, we will define a spatio-temporal random field X(s, t) as the solution of a Stochastic Partial Differential Equation (SPDE) and use this interpretation to model the physical phenomena underlying the spatio-temporal process Y (s, t) under study (see Chapter 2).

Big n problem

While the theoretical aspects of spatio-temporal geostatistics are well-developed [START_REF] Cressie | Statistics for Spatio-Temporal Data[END_REF], their implementation still faces difficulties. The geostatistical paradigm is computationally expensive for large spatio-temporal datasets, due to the factorization of dense covariance matrices, whose complexity scales with the cube of the number of observations. This well known problem is often referred to as the "big n problem" [START_REF] Banerjee | Hierarchical Modeling and Analysis for Spatial Data[END_REF].

By definition, the spatio-temporal covariance matrix contains (N S N T ) 2 covariance values, where N S and N T are the number of spatial and temporal locations, respectively. These locations can be the grid points where we want to compute a simulation or the locations of data. This means that (N S N T ) 2 values should be computed and stored, implying heavy computational and storage needs when this number becomes very large. Using separable covariance functions can significantly reduce the dimension of the covariance matrices that need to be inverted, alleviating the computational demand. This is due to the fact that, in this particular case, the large covariance matrix is defined as a Kronecker product of smaller matrices and only those smaller matrices must be inverted.

However, as detailed in Section 1.2, separable models are not always appropriate for environmental data due to their lack of flexibility, and even when they are, the "big n problem" can arise from the spatial covariance matrix only. Numerous solutions have been proposed in the stationary case to tackle this problem (see [START_REF] Sun | Geostatistics for large datasets[END_REF] for a comprehensive review). Some options are the use of compactly supported covariance functions (Gneiting, 2002;[START_REF] Bevilacqua | Estimation and prediction using generalized Wendland covariance functions under fixed domain asymptotics[END_REF], the covariance tapering [START_REF] Furrer | Covariance tapering for interpolation of large spatial datasets[END_REF][START_REF] Kaufman | Covariance tapering for likelihood-based estimation in large spatial data sets[END_REF], the Vecchia approximations and their generalizations [START_REF] Vecchia | Estimation and model identification for continuous spatial processes[END_REF][START_REF] Stein | Approximating likelihoods for large spatial data sets[END_REF]Guinness, 2018;Katzfuss and Guinness, 2021), among others. These methods limit the number of non-zero entries in the covariance matrix.

Another way of dealing with the "big n problem" is to impose that the considered Gaussian Random Field (GRF) is Markovian, i.e., a Gaussian Markov Random Field (GMRF) [START_REF] Besag | Spatial interaction and the statistical analysis of lattice systems[END_REF][START_REF] Rue | Gaussian Markov random fields: theory and applications[END_REF]. This ensures that the resulting precision matrix, the inverse of the covariance matrix, has a limited number of non-zero entries [START_REF] Rue | Gaussian Markov random fields: theory and applications[END_REF], leading to a reformulation of the problems using the sparse precision matrix instead of the covariance matrix. The computational gains from GMRFs have been exploited to provide fast and accurate Bayesian inference for latent Gaussian field models through the Integrated Nested Laplace Approximation (INLA), proposed in [START_REF] Rue | Approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations[END_REF]. However, GMRF approximations were initially criticized, because they don't allow a direct modeling of some specific correlation behaviors and because it is not possible to write down the distribution at a selected location, but only determine a conditional distribution given a set of locations. Moreover, the GMRF approximation was restricted to GRF observed over a regular lattice. Those criticisms were overcome by [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] who introduced the GMRF approximation of a GRF with Matérn covariance function as a solution of a Stochastic Partial Differential Equation (SPDE).

SPDE approach

The use of SPDEs for modeling spatial or spatio-temporal Gaussian processes is an approach that has been used in the statistical community for decades, starting with [START_REF] Whittle | On stationary processes in the plane[END_REF], [START_REF] Heine | Models for two-dimensional stationary stochastic processes[END_REF] and [START_REF] Whittle | Stochastic processes in several dimensions[END_REF]. Later works include [START_REF] Jones | Models for continuous stationary space-time processes[END_REF] and [START_REF] Brown | Blur-generated non-separable space-time models[END_REF]. Since the spectral density can be directly obtained from the SPDE, spectral methods can be considered to analyze data, as in [START_REF] Fuentes | Approximate likelihood for large irregularly spaced spatial data[END_REF] for the spatial case and in [START_REF] Sigrist | Stochastic partial differential equation based modelling of large space-time data sets[END_REF] and [START_REF] Liu | Statistical modeling for spatio-temporal data from stochastic convection-diffusion processes[END_REF] for the space-time case. [START_REF] Rozanov | Random Fields and Stochastic Partial Differential Equations[END_REF] proved that the solution of a specific class of linear SPDEs is a Markov Random Field (MRF). This occurs when the field's spectral density is the inverse of a polynomial function of the frequencies. [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] used this property to show how the approximated solution of a certain class of spatial SPDEs, obtained with the Finite Element Method (FEM), can be considered as a GMRF. The SPDE framework developed in [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] was largely adopted in spatial statistics because of two main properties. First, Lindgren et al. ( 2011) linked a specific linear SPDE to the Matérn covariance function, which is probably the most common covariance function in geostatistics. Second, the FEM that was considered to build the GMRF representation was made available for practitioners in the R-INLA and inlabru packages [START_REF] Lindgren | Bayesian spatial modelling with r-inla[END_REF][START_REF] Krainski | Advanced spatial modeling with stochastic partial differential equations using R and INLA[END_REF][START_REF] Bakka | Spatial modeling with R-INLA: A review[END_REF]. The SPDE's parameters are related to the local properties of the spatial or spatio-temporal model and are translated into the conditional distributions in the GMRF representation. One can interpret the physical parameters of the SPDE, build the prior distributions for the model parameters and obtain the corresponding spatial or spatio-temporal field. A complete review of the developments of the SPDE approach in the last decade, both from the methodological and the applied point of view, is available in [START_REF] Lindgren | The SPDE approach for gaussian and non-gaussian fields: 10 years and still running[END_REF].

Thesis statement and main contributions

This PhD dissertation starts from the considerations detailed in the sections above to propose methods for spatio-temporal inference and prediction that are physically justified (see Section 1.3), computationally efficient (see Section 1.4) and, especially, capable to model complex spatio-temporal dependence structures through nonseparable and asymmetric models (see Section 1.2).

The first contribution is the development of an SPDE approach for nonseparable asymmetric spatio-temporal models. A Spatio-Temporal (ST) advection-diffusion SPDE is proposed; its coefficients, which can be also spatially and/or temporally varying, directly influence the spatio-temporal dependencies of the process, by explaining its variability in space and time. Nonseparable and separable spatio-temporal fields can be obtained as sub-models by choosing appropriate parameters. The advectiondominated case is also discussed and dealt with by introducing the "Streamline Diffusion" stabilization technique to reduce the convergence problems related to an advection-dominated flow.

It will be shown that it is possible to build an accurate space-time approximation of the process driven by the advection-diffusion SPDE using a combination of a Finite Element Method in space and implicit Euler scheme in time. This approximation leads to sparse structured linear systems. We will obtain promising results, both in terms of precision and speed, for estimation and prediction of spatio-temporal processes based on the SPDE approach from information at scattered spatio-temporal locations. When the size of the spatio-temporal mesh is moderate, direct matrix implementation is possible through the Cholesky decomposition of sparse matrices. We will show how "matrix-free" methods (conjugate gradient method, preconditioners, etc.) can be implemented in order to obtain scalable computations even for very large meshes. The method will be applied to two different spatio-temporal environmental datasets, about solar radiation and wind speed, to show the potential of the approach. Finally, a possible generalization to nonstationary fields is proposed and some preliminary results are illustrated.

The second contribution is a generalization of the well-known Gneiting class of multivariate space-time covariance functions (Gneiting et al., 2010;Bourotte et al., 2016). The proposal is a very general parametric class of fully nonseparable direct and cross-covariance functions for multivariate random fields, where each component has a spatial covariance function from the Matérn family with its own smoothness and scale parameters and, unlike all currently available models, its own correlation function in time [START_REF] Allard | Fully nonseparable gneiting covariance functions for multivariate space-time data[END_REF]. The application of the proposed model will be illustrated on a weather trivariate dataset. It will be shown that this new model yields better fitting and better predictive scores compared to a more parsimonious model with common temporal correlation function. In this case, the main potential of the proposed approach is the possibility to obtain entirely nonseparable models in a multivariate setting.

Outline

The third contribution is a discussion about methods for approximate estimation and prediction for spatial and spatio-temporal data, motivated by the objective of reaching a trade-off between statistical efficiency and computational complexity. This

Outline

The dissertation aims at introducing and applying the SPDE approach for nonseparable advection-diffusion spatio-temporal models. In particular, we derive methods for simulation, estimation and prediction of such models. In Chapter 2 we start by setting up the mathematical framework and the main notions necessary to work with spatio-temporal SPDEs; we then present the classes of spatio-temporal SPDE models developed in the literature and the model on which this work is based: the unsteady advection-diffusion SPDE model. Chapter 3 focuses on the algorithms designed to simulate from the models via a numerical approximation of the SPDE, by first reviewing the methods that are currently used and then introducing the approach chosen in this specific work. This approach results in a convenient sparsity pattern of the global spatio-temporal precision matrix, which leads to reduced computational cost for all statistical methods (estimation, prediction, conditional simulations). Chapter 4 is devoted to the non-conditional simulation of stationary advection-diffusion SPDE models. Chapter 5 tackles the estimation and the prediction of a stationary advection-diffusion SPDE model from its partial and noisy observations. We suggest solving this problem using an approach inspired by kriging theory. In both cases, we propose practical algorithms based on the Cholesky decomposition of sparse matrices and on "matrix-free" methods. Chapter 6 presents an application of the approach for the prediction of a spatio-temporal field from its partial and noisy observations, both for solar radiation and wind speed datasets. Finally, Chapter 7 is devoted to the nonstationary generalization of the SPDE approach and to some related results, who are however still preliminary.

Moreover, the dissertation presents other contributions made during the PhD (Chapter 8). First, a different point of view on nonseparable spatio-temporal models is presented, through more classical models and their generalizations. In Section We consider a spatio-temporal model Y (s, t) observed through some measurements, with no direct measurements of the underlying stochastic process X(s, t), as presented in Equation (1.1), and reported here:

Y (s, t) = µ(s, t) + X(s, t).
Y (s, t) is the sum of fixed and random effects. The fixed effects' term can include a temporal trend, a spatial trend, and time-varying or space-varying regressions. We usually have little information about the covariance structure of the spatio-temporal random model component X(s, t), except that it should mimic the dependency structure in models of physical processes. We would like not to impose any hypothesis of separability on X, in order to be able to capture complex spatio-temporal dependencies (see Section 1.2). We use the SPDE framework, presented in Section 1.5 and detailed in the following sections. From a theoretical viewpoint, in contrast to statistically oriented constructions of nonseparable spatio-temporal models such as the product-sum models and the Gneiting class (De Iaco et al., 2003;Gneiting, 2002;[START_REF] Porcu | Nonseparable stationary anisotropic space-time covariance functions[END_REF], the SPDE approach proposes a physically grounded construction, for which the parameters carry physical interpretation such as diffusion, reaction and transport. It allows the construction of models X(s, t) with interesting nonseparability and asymmetry properties, and where nonstationarity can also easily be accounted for.

We first give a definition of Generalized Random Field (GeRF) [START_REF] Itô | Stationary random distributions[END_REF][START_REF] Rozanov | Markov Random Fields[END_REF], which is the stochastic analogous to the generalization of functions presented in Schwartz's theory of distributions [START_REF] Schwartz | Théorie des distributions[END_REF].

Definition 2.1. A generalized random field (GeRF) X on R d is a linear and continuous functional that associates to any smooth function with compact support φ ∈

C ∞ 0 (R d ) ⊂ L 2 (R d ) a random variable X (φ) ∈ R.
X is characterized by its probability distribution, which is the set of all joint distributions F φ 1 ,...,φm defined by

F φ 1 ,...,φm : (a 1 , . . . , a m ) ∈ R m → P[X (φ 1 ) ≤ a 1 , . . . , X (φ m ) ≤ a m ] for any m ≥ 1 and φ 1 , . . . , φ m ∈ C ∞ 0 (R d ).
The previous proposition implies that GeRFs have only meaning when applied to test functions in some particular functional space, and not necessarily when evaluated in points of the space.

We define a white noise as the functional W defined over R d that satisfies

E[W (φ)] = 0, ∀φ ∈ L 2 (R d ) and Cov[W (φ 1 ), W (φ 2 )] = R d φ 1 φ 2 d s , ∀φ 1 , φ 2 ∈ L 2 (R d ).
(2.1)

If, moreover, for any m ≥ 1 and any linearly independent

φ 1 , . . . , φ m ∈ L 2 (R d ), the random vector [W (φ 1 ), . . . , W (φ m )] is a Gaussian vector, then W is called Gaussian white noise.
If {Z i } i∈N is a sequence of independent, standard Gaussian variables, then the function W defined over L 2 (R d ) by

W (φ) = j∈N Z j R d φe j d s , ∀φ ∈ L 2 (R d ),
where {e j } j∈N denotes an orthonormal basis of L 2 (R d ), is a Gaussian white noise on

R d .
Let us denote ξ ∈ R d a spatial frequency and ω ∈ R a temporal frequency. The

Fourier transform of an integrable function φ(x), with x ∈ R d and φ ∈ L 2 (R d ), is F(φ)(ξ) = 1 (2π) d R d e -i x ξ φ(x) d x , ξ ∈ R d . (2.2)
Its inverse is called inverse Fourier transform and is defined as

F -1 (φ)(x) = R d e i x ξ φ(ξ) d ξ , x ∈ R d .
In this case, we necessarily have that F -1 (F(φ)) = φ.

An important result for the following sections is that the covariance structure of a stationary second-order random field X can be equivalently characterized either by its covariance function C X or by its spectral density S X . This is due to Bochner's theorem (see e.g. Donoghue (1969, Chapter 37)), that states that any continuous covariance function C X is the Fourier transform of a positive and finite measure. If we want C X to be real, the measure should be also even. In our case, the measure is absolutely continuous with respect to the Lebesgue measure. Hence, we can suppose that there exists a function S X , positive and integrable, called spectral density, such that (in the spatial case)

C X (h) = F(S(•))(h) = R d e -i h ξ S(ξ) d ξ ,
where F denotes the Fourier transform on

R d and R d S(ξ) d ξ = C X (0) < ∞.
The same result holds also for temporal and spatio-temporal covariances.

In the following sections, after presenting the spatial SPDE framework by Lind-

Spatial SPDE model

In the SPDE representation, GRFs on R d are viewed as solutions to specific SPDEs [START_REF] Whittle | On stationary processes in the plane[END_REF][START_REF] Whittle | Stochastic processes in several dimensions[END_REF]. In particular, Gaussian Whittle-Matérn fields, analyzed in details in [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] and reviewed in [START_REF] Lindgren | The SPDE approach for gaussian and non-gaussian fields: 10 years and still running[END_REF], are solutions to

(κ 2 -∆) α/2 X(s) = τ W (s), (2.3) with α > d/2 and τ > 0. ∆ = d i=1 ∂ 2 ∂s 2 i
is the Laplacian operator defining a diffusion term (see Section 2.3.1 for more physical details), and W (s) is a standard spatial Gaussian white noise. The equality is understood in a weak sense in L 2 . The spectral

density of W is S W (ξ) = (2π) -d .
The spectral density of X(s), solution to (2.3), is

S(ξ) = 1 (2π) d (κ 2 + ξ 2 ) α . (2.4)
When α is an integer, this density is the reciprocal of a strictly positive and isotropic polynomial. Rozanov's Theorem (Rozanov et al., 1982, Section 3.2.3) allows us to state that this model is an isotropic stationary Markov Rrandom Field (MRF). In Rozanov's theory, a MRF is a RF such that for every domain of R d , evaluations of the RF on the interior of the domain are independent of evaluations on the interior of the complement of the domain, conditionally on the behavior of the RF on a neighborhood of the boundary of the domain. An evaluation is the inner product between the field and a test function whose support is included in the interior of the corresponding set.

A generalization to an anisotropic model can be obtained by applying an anisotropy matrix H to ξ in Equation (2.4) and by replacing the Laplacian operator ∆ with its anisotropic form ∇ • H ∇. When α is not an integer, the SPDE (2.3) is said to be fractional and the field X is no longer a MRF. This limitation on integer α can be avoided by considering the approximation to the best Markov approximation, suggested in the reply to [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] and proposed in [START_REF] Bolin | The rational SPDE approach for gaussian random fields with general smoothness[END_REF] as a combination of the FEM discretization in space with a rational approximation. Other possible ways to tackle fractional SPDEs are proposed in [START_REF] Xiong | Covariance-based rational approximations of fractional SPDEs for computationally efficient bayesian inference[END_REF]; [START_REF] Bolin | The rational SPDE approach for gaussian random fields with general smoothness[END_REF]; [START_REF] Roques | Spatial statistics and stochastic partial differential equations: A mechanistic viewpoint[END_REF], but we will not treat this subject further in this work.

The covariance function of the Gaussian field solution to Equation (2.3) is the well known Matérn covariance function [START_REF] Matern | Spatial Variation[END_REF])

C(h) = σ 2 C M ν (κ h ) = σ 2 2 ν-1 Γ(ν) (κ h ) ν K ν (κ h ) , (2.5)
with smoothness parameter ν = α -d/2 > 0, scale parameter κ and variance

σ 2 = τ 2 (4π) -d/2 Γ(ν)Γ(ν + d/2) -1 κ -2ν
. K ν is the modified 2 nd order Bessel function and h = ss is the spatial lag between two locations s and s in R d . In particular, when ν = 1/2, we get the exponential covariance function and when ν → +∞, after proper renormalization, (2.5) tends to the Gaussian covariance function.

The spatial SPDE framework has been used for estimation and prediction of spatial processes in several scientific domains. We refer to [START_REF] Lindgren | The SPDE approach for gaussian and non-gaussian fields: 10 years and still running[END_REF] for a complete review of applications of the spatial SPDE approach in the last years.

Spatio-temporal SPDE models: state of the art

To generalize the SPDE approach to linear spatio-temporal processes X(s, t), we consider the framework proposed in Carrizo- [START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF].

Carrizo- [START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF] provided sufficient conditions to the existence and uniqueness of stationary solutions to the spatio-temporal SPDE

∂ β ∂t β + L g X(s, t) = Z(s, t), (2.6) with β > 0. L = ∂ β ∂t β + L g is a spatio-temporal operator. The spatial operator L g is defined using the spatial Fourier transform on R d , denoted F S , L g (•) = F -1 S (gF S (•)),
where g : R d → C is a sufficiently regular and Hermitian-symmetric function called the symbol function of the operator L g . The temporal operator ∂ β ∂t β is defined as

∂ β ∂t β (•) = F -1 T ((iω) β F T (•)),
where F T is the temporal Fourier transform on R and where we have used the symbol

function over R ω → (iω) β = |ω| β e i sgn(ω)βπ/2 .
These operators are called pseudo-differential operators [START_REF] Lang | Fast simulation of gaussian random fields[END_REF].

In contrast to the spatial SPDE model detailed in Section 2.1, we remark the presence of a temporal differential operator and of a spatio-temporal stochastic forcing term Z(s, t). Unless otherwise stated, we will force the stochastic source term to be a spatio-temporal white noise with unit variance

Z(s, t) = W (s, t) = W S (s) ⊗ W T (t),
but it was proved that solutions of SPDEs with white noise as source term are the starting point of more general solutions, when the source term can be any stationary GeRF [START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF] (d+1) .

The spatio-temporal symbol function of the operator involved in (2.6) is

(ξ, ω) → (iω) β + g(ξ) = |ω| β cos βπ 2 + g R (ξ) + i sgn(ω)|ω| β sin βπ 2 + g I (ξ) (2.7)
where g R and g I are the real and imaginary part of the spatial symbol function g. Remark 2.2.1. When the closed set g -1 ({0}) = {ξ ∈ R d |g(ξ) = 0} is non-empty, the non-uniqueness is due to the existence of stationary solutions to the homogeneous problem L g X(s, t) = 0.

If

In the following sections, we will review several classes of spatio-temporal SPDE models in the framework detailed above: first we introduce separable models, then we present nonseparable models in the chronological order in which they appeared in the literature, by showing their similarities and differences. We consider the real positive parameters τ, γ s , γ t , s, κ, α, β, α 1 , α 2 , α S and δ. τ is a standard deviation parameter, γ t is a time correlation parameter, γ s is a spatial range parameter, s and κ are inverse range parameters, α, β, α 1 , α 2 , α S and δ are smoothness parameters. We only consider stationary fields, where the parameters of the SPDE are constant values.

Nevertheless, we note that nonstationarity can be obtained naturally by making the operator L spatially and/or temporally varying L = L(s, t), as discussed by [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] and in Chapter 7.

Separable spatio-temporal models

The easiest way to extend the spatial SPDE model of [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] to space-time is to use the spatial SPDE as the spatial component in a spatio-temporal separable model. [START_REF] Jones | Models for continuous stationary space-time processes[END_REF] discussed how separable covariance functions can be understood through differential operators, written as L = L S L T , where L S is a purely spatial operator and L T is a purely temporal operator.

A separable space-time model can be defined as a SPDE model for the spatial trace and an autoregressive model of order 1, i.e., AR(1), for the time dimension.

The space-time precision matrix of the separable model is defined by the Kronecker product between the precision matrices of the spatial and temporal random effects [START_REF] Cameletti | Spatio-temporal modeling of particulate matter concentration through the SPDE approach[END_REF].

The same model is obtained in [START_REF] Krainski | Statistical Analysis of Space-time Data: New Models and Applications[END_REF] as a one dimensional SPDE describing the temporal evolution and another SPDE modeling the driving noise as spatially correlated. This model writes

   ∂ ∂t + a X(s, t) = Z(s, t) (κ 2 -∆) α/2 Z(s, t) = τ W (s, t)
However, as discussed in Section 1.2, separability is a strict hypothesis, rarely satisfied in the real world. For this reason, researchers have been proposing spatiotemporal nonseparable models as solutions of SPDEs.

Jones and Zhang (Jones and Zhang, 1997)

Jones and [START_REF] Jones | Models for continuous stationary space-time processes[END_REF] were the first in the spatio-temporal statistics community to define a nonseparable spatio-temporal random field as solution to the SPDE

γ t ∂ ∂t + (κ 2 -∆) α/2 X(s, t) = τ W (s, t). (2.8)
The symbol function of the spatio-temporal operator is

(ξ, ω) → [(γ t iω) + (κ 2 + ξ 2 ) α/2 ] (2.9)
and the spectral density of the stationary solution X is

S(ξ, ω) = τ 2 (2π) d+1 [γ 2 t ω 2 + (κ 2 + ξ 2 ) α ]
.

(2.10) Equation (2.8) verifies the SCEU and it can be proved that, when α > d, the spatial trace has a Matérn covariance with smoothness equal to ν = (α -d)/2 and scale parameter κ.

Stein (Stein, 2005)

Stein ( 2005) defined a nonseparable spatio-temporal model with spectral density

S(ξ, ω) = 1 (2π) d+1 [b(s 2 + |ω| 2 ) α 1 + a(κ 2 + ξ 2 ) α 2 ] δ , (2.11)
where a > 0 and b > 0 are scaling parameters, s and κ are scale parameters and α 1 , α 2 and δ are smoothness parameters.

Starting from this density, [START_REF] Krainski | Statistical Analysis of Space-time Data: New Models and Applications[END_REF][START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF] and [START_REF] Lindgren | A diffusionbased spatio-temporal extension of gaussian matérn fields[END_REF] derived the symbol function

(ξ, ω) → [b(s 2 + ω 2 ) β + a(κ 2 + ξ 2 ) α ] δ/2
(2.12)

and defined the SPDE whose solution is the spatio-temporal field with spectral density equal to (2.11)

b s 2 - ∂ 2 ∂t 2 α 1 + a(κ 2 -∆) α 2 δ/2 X(s, t) = W (s, t).
(2.13)

The model verifies the SCEU. Moreover, to obtain an integrable marginal spatial covariance, it is needed s 2 + κ 2 > 0 and a minimum smoothing condition stated by

d/α 2 + 1/α 1 < 2δ.
In this case, the marginal spatial covariance is a Matérn with scale κ and smoothness ν = α 2 (δ -1/(2α 1 )) -1 and the marginal temporal covariance (κ = 0) is a Matérn with scale s and smoothness

ν T = α 1 (δ -1/(2α 2 )) -1/2.
The interesting property of the Stein model is that, without being a separable model, the temporal and spatial smoothness of the paths of the random function can be controlled separately thanks to the parameters α 1 and α 2 . There are only a few special cases of spectral densities of the form (2.11) for which explicit expressions for the Fourier transform are available.

Having α 1 and α 2 integers and δ half-integer gives the reciprocal of Equation (2.11) a polynomial in ξ and ω and thus, the associated process is a MRF. Hence, there is an explicit GMRF representations associated, as in [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF].

Stochastic Heat Equation

The Stochastic Heat Equation was proposed by both [START_REF] Krainski | Statistical Analysis of Space-time Data: New Models and Applications[END_REF] It hence writes

γ t ∂ ∂t + (κ 2 -∆) X(s, t) = τ W (s, t).
(2.15)

Its spatio-temporal symbol function is

(ξ, ω) → [(γ t iω) + (κ 2 + ξ 2 )] (2.16)
and the spectral density of X is

S(ξ, ω) = τ 2 (2π) d+1 [γ 2 t ω 2 + (κ 2 + ξ 2 ) 2 ]
.

(2.17) Equation (2.15) verifies the SCEU.

Evolution Equation (Carrizo-Vergara et al., 2022)

In Carrizo- Two specific cases of Evolution Equations are reported here, for β = 1 and β = 2.

When β = 1, the first order evolution model has a spectral density equal to

S(ξ, ω) = 1 (2π) d+1 [(ω + g I (ξ)) 2 + g 2 R (ξ)]
,

(2.19) while, when β = 2, the second order evolution model has a spectral density equal to

S(ξ, ω) = 1 (2π) d+1 [(ω 2 -g R (ξ)) 2 + g 2 I (ξ)]
.

(2.20)

Evolving Matérn (Carrizo-Vergara et al., 2022)

An Evolving Matérn model is a stationary solution to the evolution equation (2.6) such that its spatial trace follows a Matérn model. Evolving Matérn models can be obtained by controlling the spatial symbol function g, but also the structure of the stochastic forcing term Z. We assume that the SCEU is verified, i.e., the parameters are chosen such that g R satisfies the SCEU and g R cos βπ 2 ≥ 0.

When the stochastic forcing term is a spatio-temporal white noise, i.e., Z(s, t) = W (s, t), the Evolving Matérn SPDE writes

∂ β ∂t β + s β a(κ 2 -∆) α/2 X(s, t) = W (s, t), (2.21)
where s β is a parameter that takes the value 1 or -1 depending conveniently on β in order to obtain the conditions of the SCEU. Its spatio-temporal symbol function is

(ξ, ω) → (iω) β + g(ξ) = (iω) β + s β a(κ 2 + ξ 2 ) α/2 (2.22)
and its spectral density is

S(ξ, ω) = 1 (2π) d+1 |ω| 2β + 2|ω| β a(κ 2 + ξ 2 ) α/2 |cos βπ 2 | + a 2 (κ 2 + ξ 2 ) α .
(2.23)

The solution to (2.21) with β = 1 is a symmetric nonseparable model which is a mixture of a J-Bessel model in space with an exponential model in time; it corresponds to the model of [START_REF] Jones | Models for continuous stationary space-time processes[END_REF]

DEMF (Lindgren et al., 2020)

The Diffusion-based Extension of the Matérn Field (DEMF) [START_REF] Lindgren | A diffusionbased spatio-temporal extension of gaussian matérn fields[END_REF], which was introduced in a particular case for the first time in [START_REF] Krainski | Statistical Analysis of Space-time Data: New Models and Applications[END_REF] as Iterated Heat Equation, is the SPDE

γ t ∂ ∂t + (κ 2 -∆) α/2 δ X(s, t) = τ Z(s, t).
(2.24)

The symbol function of the left-hand-side operator is

(ξ, ω) → [(γ t iω) + (κ 2 + ξ 2 ) α/2 ] δ (2.25)
and the right-hand-side stochastic forcing term is a Gaussian noise, white in time and correlated in space, i.e., Z(s, t)

= W T (t) ⊗ Z S (s), where Z S (s) is solution to the spatial SPDE (κ 2 -∆) α S /2 Z S (s) = W S (s).
The spectral density of X is then

S(ξ, ω) = τ 2 (2π) d+1 [γ 2 t ω 2 + (κ 2 + ξ 2 ) α ] δ (κ 2 + ξ 2 ) α S .
(2.26) Equation (2.24) verifies the SCEU and defines a family of spatio-temporal processes which, when restricted to space given that the smoothness parameters are chosen appropriately, gives the Matérn field. It allows us to choose how differentiable the stochastic process should be through α, α S and δ. The type of nonseparability also changes based on the chosen parameters. The DEMF family contains both a subfamily of separable models (when α = 0) and a subfamily of the Stein family [START_REF] Stein | Space-time covariance functions[END_REF] (when α S = 0), including Jones and Zhang model [START_REF] Jones | Models for continuous stationary space-time processes[END_REF] (when δ = 1 and α S = 0).

Unsteady advection-diffusion SPDE

As detailed in the previous section, Carrizo [START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF], among others, built models of spatio-temporal random fields that are stationary solutions, when they exist, of a very large class of SPDEs. They constructed very general models, that include and encompass existing ones, for example by proposing asymmetric models. They offered the possibility to build and characterize models far beyond the Matérn family which is currently the covariance model considered within most SPDE implementations [START_REF] Krainski | Statistical Analysis of Space-time Data: New Models and Applications[END_REF][START_REF] Lindgren | A diffusionbased spatio-temporal extension of gaussian matérn fields[END_REF].

In this work, we will focus on the unsteady advection-diffusion SPDE, which is explained from a physical point of view in the next section and then detailed in a statistical framework in the following sections. Unsteady (S)PDEs relate to the rate of changes of spatially distributed physical quantities interacting with one another and predict their evolution over time. Hence, at least one derivative with respect to time is present in the (S)PDE and the solution is a function of time. The steady version of the same (S)PDE is a time-independent function and it is obtained by setting the partial derivative with respect to time to constant zero. In Figure 2.1, we illustrate the nonseparable models we've presented in the last sections, along with the advection-diffusion model proposed in this work. We remark how statisticians have been trying to enlarge the family of nonseparable SPDE models to capture higher degrees of complexity.

Diffusion and advection

We here detail the physical characteristics of the advection-diffusion (S)PDE.

Diffusion is a mass transfer phenomenon that causes the distribution of a species to become more uniform in space as time passes. The species can be, for example, a chemical dissolved in a solvent. The driving force for diffusion is the thermal motion of molecules, i.e., the fact that molecules are never at rest at temperatures above absolute zero. When molecules collide with each other frequently, the direction of the motion becomes randomized. Although diffusion occurs because of statistical effects, usually diffusion is modeled through continuous partial differential equations. These PDEs include Fick's laws [START_REF] Fick | On liquid diffusion[END_REF], the advection-diffusion equation, or more complex methods for concentrated mixtures, like Maxwell-Stefan diffusion [START_REF] Maxwell | Iv. on the dynamical theory of gases[END_REF][START_REF] Stefan | Über das gleichgewicht und die bewegung, insbesondere die diffusion von gasgemengen[END_REF]). Fick's laws contain only one parameter, the diffusion coefficient, or diffusivity, which is the proportionality constant between the molar flux due to molecular diffusion and the negative value of the gradient in the concentration of the species. Its unit of measurement is m 2 /s.

Advection is a mass transfer due to the bulk motion of a fluid. For example, the flow of water transports molecules or ions dissolved in the water, or the flow of air transports molecules present in air. Sometimes the words convection or transport are used as synonyms of advection. Even though their signification is different in the context of heat transfer (where the term convection usually refers to the combination of heat transfer by conduction and advection, while the term advection refers to heat transfer due to bulk fluid motion), from now on we will use those terms as equivalent.

The velocity of a molecule undergoing mass transfer incorporates both an advective and diffusive component, since all chemical species have a nonzero diffusivity in reality. To summarize, advection is the mass transport due to the average velocity of all molecules, and diffusion is the mass transport due to the instantaneously varying, randomized velocity of individual molecules, compared to the average velocity of the fluid as a whole.

Unsteady advection-diffusion SPDE model

In this work we study the unsteady advection-diffusion SPDE, where the randomness is given by a stochastic forcing term. This SPDE writes • the operator γ •∇ models the advection, γ ∈ R d being a velocity vector;

∂ ∂t + 1 c (κ 2 -∇ • H ∇) α + 1 c γ •∇ X(s, t) = τ √ c Z(s,
• α ≥ 0 relates to the smoothness of X(•, t), κ 2 > 0 accounts for damping and c is a positive time-scale parameter;

• τ ≥ 0 is a standard deviation factor and Z is a stochastic forcing term. From now on, we will assume a Gaussian distribution for Z.

This equation was mentioned in [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF][START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF] and [START_REF] Lindgren | A diffusionbased spatio-temporal extension of gaussian matérn fields[END_REF], and was analyzed using spectral approaches in [START_REF] Sigrist | Stochastic partial differential equation based modelling of large space-time data sets[END_REF] and [START_REF] Liu | Statistical modeling for spatio-temporal data from stochastic convection-diffusion processes[END_REF]

Z(s, t) = W T (t) ⊗ Z S (s),
where Z S is a spatial GeGRF and W T is a temporal white noise. Z S is often chosen to be a spatial white noise, denoted W S in this case. To ensure a sufficient smoothness for Z, Z S can alternatively be a colored noise, such as for example the solution to the spatial Whittle-Matérn SPDE [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF])

(κ 2 -∇ • H ∇) α S /2 Z S (s) = W S (s), (2.28)
where W S is a Gaussian white noise. Notice that the parameter κ 2 in the forcing term has been set identical to that in the diffusion term in the left-hand-side of (2.27)

to ensure that the spatial marginalization of the process is a Matérn field, as detailed below.

When α > 0, X(s, t) is a stationary nonseparable spatio-temporal field with co-

variance function C ST (h, u), with (h, u) ∈ R d × R.
C S (h) = C ST (h, 0) = τ 2 Γ(α tot -d/2) 2Γ(α tot )(4π) d/2 κ 2(αtot-d/2) |H| 1/2 C M αtot-d/2 κ H -1/2 h . (2.

29)

where h = ss is the spatial lag and C M αtot-d/2 (•) is the unit variance and scale Matérn covariance function defined in (2.5) with smoothness parameter equal to ν = α tot -d/2. Proposition 2.1 is adapted from Proposition 1 in [START_REF] Lindgren | A diffusionbased spatio-temporal extension of gaussian matérn fields[END_REF]. A proof is reported below.

Proof. The covariance function of the spatial trace between X(s, t) and X(s , t) for a spatial lag h = ss does not depend on the imaginary part of the spatial symbol function [START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF], hence it can be written as (2.30) where S(ξ, ω) is the spectral density defined as

C S (h) = C ST (h, 0) = R d R exp i h ξ S(ξ, ω) dω d ξ = R d exp i h ξ R S(ξ, ω) dω d ξ = R d exp i h ξ S S (ξ) d ξ ,
S(ξ, ω) = τ 2 (2π) d+1 ω 2 + c -2 (κ 2 + ξ H ξ) 2α c(κ 2 + ξ H ξ) α S .
Integrating over ω, we obtain the spatial spectral density

S S (ξ) = τ 2 (2π) d c(κ 2 + ξ H ξ) α S R 1 2π ω 2 + c -2 (κ 2 + ξ H ξ) 2α dω = τ 2 (2π) d c(κ 2 + ξ H ξ) α S 1 2 [c -2 (κ 2 + ξ H ξ) 2α ] 1/2 = τ 2 2(2π) d (κ 2 + ξ H ξ) αtot . (2.31)
Using the change of variable ξ = κ H -1/2 w and plugging Equation (2.31) into (2.30), we obtain

C S (h) = C ST (h, 0) = τ 2 2 R d e i h ξ (2π) d (κ 2 + ξ H ξ) αtot d ξ = τ 2 2 R d e i h κ H -1/2 w |κ H -1/2 | (2π) d (κ 2 + κ 2 w w) αtot d w = τ 2 2κ 2(αtot-d/2) |H| 1/2 R d e i h κ H -1/2 w (2π) d (1 + w w) αtot d w = τ 2 Γ(α tot -d/2) 2Γ(α tot )(4π) d/2 κ 2(αtot-d/2) |H| 1/2 C M αtot-d/2 κ H -1/2 h .
The last result comes from the computation of R d (1 + w w) -αtot d w with polar coordinates.

The model reduces to a separable one in a particular case stated in the corollary below.

Corollary

C ST (h, u) = τ 2 Γ(α S -d/2) 2Γ(α S )(4π) d/2 κ 2(α S -d/2) |H| 1/2 C M α S -d/2 κ H -1/2 h exp u c ,
with smoothness parameter equal to ν = α S -d/2.

Relationship between diffusion and advection-diffusion SPDE

Let us consider the stationary solution to the spatio-temporal SPDE

∂ ∂t + L g + γ •∇ X(s, t) = W (s, t),
where L g is a purely spatial symbol operator that satisfies the SCEU. Given the stationary solution X(s, t), if we shift it by defining X γ (s, t) = X(s -γ t, t), then

X γ (s, t) satisfies the SPDE ∂ ∂t + L g X γ (s, t) = W 2 (s,

Discretization approaches for (S)PDEs

In numerical analysis, different discretization approaches can be chosen to approximate PDEs. The main ones are the Finite Difference Method (FDM), the Finite Element Method (FEM), the Finite Volume Method (FVM) and spectral methods.

The principle of the FDM is to assign one discrete unknown per each discretization point over a mesh and to write one equation per point. At each discretization point, the derivatives of the unknown are approximated by finite differences through the use of Taylor expansions. The FDM becomes difficult to use when the coefficients involved in the equation are discontinuous, or when the geometry of the domain is complex. For simplified geometries, FDM is an easy method to implement and likely leads to faster codes than FEM and FVM.

The FEM (see e.g. [START_REF] Quarteroni | Modellistica Numerica per Problemi Differenziali[END_REF]) is based on a variational formulation, which is written for both the continuous and the discrete problems. The variational formulation is obtained by multiplying the original equation by a test function. The continuous unknown is then approximated by a linear combination of the test functions for the discrete variational formulation (this is the so called Galerkin method); the resulting equation is integrated over the domain. The choice of the FEM to discretize spatial derivatives is, in general, directed by the consideration that the method fits naturally to a general spatial domain, even for complex geometries, since the triangulation of the domain can be much more precise than with FDM [START_REF] Boulakia | Simulation of SPDEs for Excitable Media Using Finite Elements[END_REF].

The FVM evaluates the unknowns at discrete locations in the spatial domain by using an integral formulation of the balance equations and a successive approximation of integrals by numerical integration at each cell of the decomposition of the domain.

The method is based on conservation laws.

Spectral methods consist in writing the solution of the differential equation as a sum of basis functions (for example, as a Fourier series) and choosing the coefficients of the terms of the sum in order to satisfy the PDE as well as possible.

In the following sections, we review the methods used in the statistical literature to solve the SPDEs whose solutions are spatio-temporal random fields; then, we present the discretization method chosen in our work. For the spatial domain, they used the Neumann boundary condition

Finite Element Method

∇X • n = 0 on ∂Ω (3.1)
where n is the normal vector to the boundary. They assumed that the domain is extended away from the region of interest to account for possible errors at the boundaries. For the temporal boundary, they ensured that the stochastic process is stationary in time, i.e.,

Cov(X(s 1 , t 1 ), X(s 2 , t 2 )) = Cov(X(s 1 , t 1 + u), X(s 2 , t 2 + u)), (3.2)
whenever t 1 , t 2 , t 1 + u, t 2 + u are in the domain.

Let Φ T = {φ j } N T j=1 denote a set of piecewise linear temporal basis functions over a regular discretization of [a, b].

Let Ψ S = {ψ i } N S
i=1 denote a set of piecewise linear spatial basis functions obtained by a triangulation of the spatial domain. They approximated the process X(s, t) in (2.24), with boundary conditions as in (3.1) and (3.2), in the Kronecker basis Ψ S ⊗ Φ T , as

X h (s, t) = N S i=1 N T j=1 x ij ψ i (s)φ j (t). (3.3)
This is the projection of the solution X(s, t) into the finite Hilbert space spanned by Ψ S and Φ T . They proved that the coefficients x ij in (3.3) have a precision matrix that is expressed as a sum of Kronecker products. More details on this approach are available in [START_REF] Lindgren | A diffusionbased spatio-temporal extension of gaussian matérn fields[END_REF].

Finite Volume Method

Other statisticians dealing with SPDEs prefer to use the FVM, that is well suited for the numerical simulation of various types (elliptic, parabolic, or hyperbolic) of conservation laws [START_REF] Eymard | Solution of Equation in R n (Part 3)[END_REF]. Some of the important features of the FVM are similar to those of the FEM: it may be used on arbitrary geometries, using structured or unstructured meshes, and it leads to robust schemes. [START_REF] Fuglstad | Approximating solutions of stochastic differential equations with gaussian markov random fields[END_REF] considered the model given in Equation (2.14) in the one-dimensional space domain to derive the precision matrix using the FVM. Then, in following works [START_REF] Fuglstad | Does non-stationary spatial data always require non-stationary random fields?[END_REF][START_REF] Fuglstad | Compression of climate simulations with a nonstationary global spatiotemporal SPDE model[END_REF][START_REF] Hu | A stochastic locally diffusive model with neural network-based deformations for global sea surface temperature[END_REF][START_REF] Berild | Spatially varying anisotropy for gaussian random fields in three-dimensional space[END_REF], the FVM was adopted for spatial SPDE models in 2D and 3D. The extension of the FVM to spatio-temporal models in 2D and 3D was never considered, and usually the temporal spatial repetitions were supposed to be independent.

Spectral methods

Spectral methods have been used in spatio-temporal statistics, mostly for approximating or solving deterministic integro-difference equations or PDEs. [START_REF] Wikle | A dimension-reduced approach to space-time kalman filtering[END_REF] introduced a dynamic spatio-temporal model obtained from an integrodifference equation that is approximated by using a reduced dimensional spectral basis. Cressie and Wikle (2011, Chapter 7) gave an overview of basis function expansions in spatio-temporal statistics.

In a stochastic framework, spectral methods are widely used to analyze approximations of solutions to specific SPDEs. When the FEM and the FVM lack in generality for cases with complicated operators involved, spectral methods can be used, even if they require a particular basis of functions to be fixed. This requisite implies that the chosen basis can be useful for a specific setting and less adapted to different equations.

One of the possible basis is the Fourier basis (see [START_REF] Chilès | Geostatistics: Modeling Spatial Uncertainty[END_REF] for a comprehensive description of the method). Efficient simulations of the SPDE models can be easily conceived using a Fourier spectral method as proposed in [START_REF] Lang | Fast simulation of gaussian random fields[END_REF] and extended in Carrizo [START_REF] Vergara | Development of geostatistical models using stochastic partial differential equations[END_REF]. [START_REF] Sigrist | Stochastic partial differential equation based modelling of large space-time data sets[END_REF] showed how to obtain a space-time Gaussian process by solving an advection-diffusion SPDE using the real Fourier transform. They also showed that, by solving the SPDE using Fourier functions, they obtained computationally efficient statistical inference methods. This computational efficiency is due to the temporal Markov property and the fact that Fourier functions are eigenfunctions of the spatial differential operators. The overall computational cost is determined by the fast Fourier transform (FFT) [START_REF] Cooley | An algorithm for the machine calculation of complex fourier series[END_REF], which is O(N T log(N S )). However, it is difficult to fit this model to sparse real data, since usually we cannot obtain a precise representation of the data in the Fourier basis. Another downside of spectral methods is that, since Fourier terms are global functions, stationarity is a necessary assumption, while this is not a requirement for the FEM and the FVM (see Chapter 7).

FEM for advection-dominated PDEs

In this section, we consider PDEs where both diffusion (second order) and advection (first order) terms are present, as in the case of Equation (2.27). We present how the FEM, used to discretize the spatial derivatives, is modified to take into account a particular characteristic of the PDE: the advection-dominated state.

The steady advection-diffusion PDE reads

-λ∆X + γ •∇X = f, with λ > 0 and γ ∈ R d .
If the problem is diffusion-dominated (λ γ ), we can safely apply the Galerkin technique, i.e., we can find a unique solution into a finite Hilbert space spanned by the basis functions {ψ i } N S i=1 defined on a triangulation with N S spatial points. However, if we consider advection-dominated problems, applying the Galerkin technique introduces significant instabilities (Quarteroni, 2008, Chapter 5). To investigate this kind of phenomena, we analyze the 1D problem (γ ∈ R)

-λ ∂ 2 X ∂x 2 + γ ∂X ∂x = f. (3.4)
We define the Péclet number as Pe h = γh 2λ , where h is a measure of the mesh of the spatial domain (e.g. the average length of the intervals in 1D). Pe h is defined to be the ratio of the rate of advection of a physical quantity by the flow to the rate of diffusion of the same quantity driven by an appropriate gradient. If Pe h > 1, that is, if h > 2λ/γ, the PDE is said to be advection-dominated and it can be shown that the numerical solution X to (3.4) is oscillating (Quarteroni, 2008, Chapter 5), thus it is not physically admissible. Then, h must be set to a small value such that Pe h < 1. However, the condition h < 2λ/γ is too strict in practice, especially for advection-dominated problems.

A way of stabilizing the advection operator γ • ∇ is to replace the diffusion co-

efficient λ with λ = λ(1 + φ(Pe h )), where lim h→0 φ(Pe h ) = 0 and φ(Pe h ) ≥ Pe h .
This method is called upwind (U) in the simplified case where φ(Pe h ) = Pe h . The idea behind the stabilization method is to add an artificial diffusion term equal to

λφ(Pe h ) ∂ 2 X
∂x 2 that depends on the size of the discretization mesh h and on the Péclet number. In this way, the equation with the additional stabilization term reads

-λ(1 + φ(Pe h )) ∂ 2 X ∂x 2 + γ ∂X ∂x = f
and it is not advection-dominated anymore, since its Péclet number Pe h is now equal

to Pe h = Pe h 1 + φ(Pe h ) ,
which always satisfies Pe h < 1.

The problem extended to dimension d, where we add also a reaction term given by bX, is the following:

-∇ • (λ∇X) + γ •∇X + bX = f. (3.5)
The weak solution X ∈ V to (3.5), where V is a Hilbert space, satisfies

Ω λ∇X • ∇v d s + Ω γ •∇Xv d s + Ω bXv d s A(X,v) = Ω f v d s G(v) ∀v ∈ V. (3.6)
We can have a possible pathological situation, when the advection coefficient γ is much larger than the diffusion coefficient λ. In this case, we can resort to a modified version of the problem, with augmented diffusion, similarly to the 1D case. Inspired by the 1D case, we consider the Galerkin method that searches

X h ∈ V h ⊂ V such that A h (X h , v h ) = G(v h ) ∀v h ∈ V h .
To choose A h (•, •), the general idea is to pick up

A h (•, •) = A(•, •) + S h (•, •)
The additional term S h (•, •) has the purpose of eliminating, or at least reducing, the numerical oscillations produced by the Galerkin method when the grid is not fine enough, and are therefore named stabilization terms. By defining h as a measure of the size of the triangulation of the spatial domain (e.g. the average diameter of the triangles), the term S h (•, •) has to vanish at the limit h → 0 to ensure weak consistency, i.e., the fact that lim

h→0 A h (X -X h , v h ) = 0, ∀v h ∈ V h . We have several possible numerical approaches: 1. Upwind (U) S U,h (w h , v h ) = Qh Ω ∇w h • ∇v h d s , Q > 0.
2. Streamline Diffusion (SD) [START_REF] Hughes | A multidimensional upwind scheme with no crosswind diffusion[END_REF])

S SD,h (w h , v h ) = Qh Ω (γ •∇w h )(γ •∇v h ) d s , Q > 0.
γ •∇w is called stream-line; it refers to all diffusion going on along the advection

direction. Generally Q is set to Q = γ -1 .
The scaling coefficient h is necessary to recover consistency. Indeed, elementsize, since ∇w h ∼ h -1 , if h was not present, we would have both S U,h ∼ h -1 and S SD,h ∼ h -1 , which implies no consistency. Both methods are only weakly consistent and provide an error that is O(h), i.e., they are first order convergent. The Upwind stabilization term can be considered as an additional artificial diffusion equal to Qh∆X in the SPDE (3.5). This extra diffusion is not only in the direction of the transport, where we aim to reduce the oscillations, but also in the orthogonal direction, where there is no problem of convergence. The SD method, on the other hand, considers only an artificial diffusion along the advection direction and it acts as an extra anisotropic diffusion Qh∇ • γ γ ∇X in (3.5).

It is also possible to add stabilization terms that ensure strong consistency, i.e.,

such that A h (X -X h , v h ) = 0, ∀v h ∈ V h , ∀h.
These methods obtain a perturbed problem in the strong form. In this case, the Galerkin method searches

X h ∈ V h such that A(X h , v h ) + S h (X h , f, v h ) = G(v h ) ∀v h ∈ V h
where S h is such that S h (X h , f, v h ) = 0, to enforce strong consistency. Two typical choices for S h are Galerkin Least Squares (GLS) and Streamline Upwind Petrov-Galerkin (SUPG) methods, detailed below.

1. Galerkin Least Squares (GLS) Tr), with Tr being a triangle in the triangulation T and where Lu = -λ∆u + γ •∇u + bu.

S GLS,h (w h , f, v h ) = Tr∈T τ Tr Ω (Lw h -f )Lv h d s , with τ Tr = δh Tr |γ| , δ > 0, h Tr = diam(

Streamline Upwind Petrov-Galerkin (SUPG)

S SU P G,h (w h , f, v h ) = Tr∈T τ Tr Ω (Lw h -f )(L SS v h ) d s ,
where L SS is the skew-symmetric part of L, i.e., L SS u = 1 2 (div(γ u) + γ •∇u), as explained in the following remark. Remark 3.2.1. We can always write an operator as the sum of a symmetric L S and a skew-symmetric L SS part:

Lu = L S u + L SS u. Let the advection-reaction-diffusion operator Lu = -λ∆u + div(γ u) + bu. Since div(γ u) = 1 2 div(γ u) + 1 2 div(γ u) = 1 2 div(γ u) + 1 2 u div(γ) + 1 2 γ •∇u,
we can decompose the operator in the following way:

Lu = -λ∆ + b + 1 2 div(γ) u + 1 2 [div(γ u) + γ •∇u] .
Note that the reaction coefficient b has become b = b+ 1 2 div(γ). The first bracket is a symmetric term, since

L S u, v V ,V = λ(∇u, ∇v) + (b u, v) = -λ u, ∆v V,V + (u, b v) = u, L S v V,V , ∀u, v ∈ V,
and the second bracket is a skew-symmetric term, since

L SS u, v V ,V = 1 2 (div(γ u), v) + 1 2 (γ •∇u, v) = - 1 2 (γ u, ∇v) + 1 2 (∇u, γ v) = - 1 2 (u, γ •∇v) - 1 2 (u, div(γ v)) = -u, L SS v V,V , ∀u, v ∈ V.
GLS and SUPG approximate the solution with error rate O(h m+ 1 2 ), where m is the degree of the polynomial approximation used in the approximation. In the simple case of piecewise linear functions, m = 1. One can ask how to choose between the several methods proposed before. On the one hand, the SD method is easy to implement, but it is only first order accurate with respect to the mesh size and much more diffusive with respect to strongly consistent methods. On the other hand, it is well known that the SUPG method has the capability of improving numerical stability for convection-dominated flows, while satisfying a strong consistency property [START_REF] Tezduyar | Stabilized finite element formulations for incompressible flow computations[END_REF]. Like many other conforming stabilized methods, the SUPG scheme contains an element-wise stabilization parameter τ Tr that has to be tuned in practice, and, except for simplified situations, the optimal value is not known. Moreover, discontinuous methods are naturally well suited for problems whose solution have low regularity, and hence they work very well for advection-diffusion problems, both in advectiondominated and diffusion-dominated regimen.

In our work, we accept to have a first order convergence and we opt for the Streamline Diffusion method, which is easier to implement. We define S = S SD for ease of notation.

In Figure 3.1, we represent how a stabilization method works. The 2D problem considered here is Equation ( 3

.5) with b = 0, λ = 0.1, f = 1. The domain is Ω = [-1, 1] × [-1, 1]
with Dirichlet boundary conditions (X = 0 on δΩ) and the mesh is a grid with h = 0.1. γ is set to γ 1 = [0.5, 0.5] for a diffusion-dominated problem and to γ 2 = [5, 5] for an advection-dominated problem. In fact, Pe h,1 = 0.35, while Pe h,2 = 3.5. Figure 3.1 shows the FEM solution to the PDE with γ 1 on the left, the FEM solution to the PDE with γ 2 without any stabilization in the middle and the FEM solution to the PDE with γ 2 with the Streamline Diffusion stabilization term on the right. In the bottom line, we show the solution along the [1, 1]-diagonal. We remark how the stabilization method in the right plot remove the oscillations present in the middle plot.

Discretization of the unsteady advection-diffusion SPDE

In the numerical analysis literature, when the (S)PDE presents a first-order derivative in time, generally it is discretized in space using either the FDM, the FEM or the FVM, and discretized in time using a standard implicit solver like Euler. The discretization of SPDE by finite differences in time and finite elements in space has been considered by several authors in theoretical studies, see for example [START_REF] Boulakia | Simulation of SPDEs for Excitable Media Using Finite Elements[END_REF]; [START_REF] Cao | Finite element methods for semilinear elliptic stochastic partial differential equations[END_REF][START_REF] Hausenblas | Approximation for semilinear stochastic evolution equations[END_REF].

In the method of lines, the (S)PDE is discretized first in space, obtaining a semidiscrete equation (system of ODEs), and then in time. This approach allows switching from one temporal discretization to another without much effort.

In the method of Rothe, the (S)PDE is discretized first in time, and then in space.

This is useful when one needs to adapt the mesh between time steps. In fact, with

the Rothe method, one can choose the spatial discretization independently at each time step. For unsteady (S)PDEs, generally simple time stepping methods, such as, for example, Crank-Nicolson or the implicit and explicit Euler schemes, are efficient enough. For these simple time integrators, it is much easier to hand-code the time integration than the spatial discretization. That might explain why most people in the finite element world prefer to think along the Rothe method.

As we have already seen in Section 3.1.1, the third possibility is to discretize the (S)PDE both in space and time simultaneously (also known as space-time discretization), as it was proposed by [START_REF] Lindgren | A diffusionbased spatio-temporal extension of gaussian matérn fields[END_REF].

Following the considerations above, the unsteady advection-diffusion SPDE (2.27) is discretized with the Rothe method, first in time and then in space, using the Finite Difference Method (FDM) and the Finite Element Method (FEM), respectively.

The temporal domain [1, T ] is discretized in N T regular time steps of length dt = T /N T . Since implicit solvers are usually less sensitive to numerical instability than explicit solvers, the implicit Euler scheme is chosen for the temporal discretization.

This choice implies unconditional stability, hence convergence towards the stationary solution. Moreover, we will show in Section 3.3.2 that the implicit Euler scheme is necessary to obtain a sparse structure of the global spatio-temporal precision matrix.

The FEM method for the spatial discretization is the Galerkin method with Neumann boundary conditions, as detailed in [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF].

The solution in two dimensions is now detailed. The solution in three dimensions involve geometrical technicalities, but is otherwise very similar. Let Ω ⊂ R 2 be a compact and connected domain of R 2 . Ω is meshed using a triangulation T with N S vertices {s 1 , . . . , s N S } ⊂ Ω. Let h := max Tr∈T h Tr , where h Tr is the diameter of the triangle Tr ∈ T . As before, we use a first order finite element representation X h of the solution to the spatial SPDE, i.e., X h = N S i=1 x i ψ i . The weights {x i } N S i=1 define uniquely the values of the field at the vertices, while the values in the interior of the triangles are determined by linear interpolation. The Galerkin solution is then obtained by finding the weights that fulfill the weak formulation of Equation (2.27) for test functions belonging to the space V h spanned by {ψ i } N S i=1 .

Proposition 3.1. Let X(s, t) be the spatio-temporal process solution to Equation

(2.27) with α ∈ {0, 1} and spatio-temporal white noise, i.e. Z(s, t) = W (s, t) = W T (t) ⊗ W S (s). Let T be a triangulation of Ω and {ψ i } N S i=1 be the piecewise linear basis functions defined over T . Let us define the mass matrix

M = [M ij ] N S i,j=1 , the stiffness matrix G = [G ij ] N S i,j=1 , the advection matrix B = [B ij ] N S i,j=1 and the matrix K = [K ij ] N S i,j=1
as follows:

M ij = Ω ψ i (s)ψ j (s) d s , G ij = Ω H ∇ψ i (s) • ∇ψ j (s) d s , B ij = Ω γ •∇ψ i (s)ψ j (s) d s , K ij = (κ 2 M ij + G ij ) α .
Then, at each time step, the continuous Galerkin finite element solution vector

x (k+1) = {x (k+1) i } N S i=1 satisfies M + dt c (K + B) x (k+1) = M x (k) + τ √ dt √ c M 1/2 z (k+1) , (3.7)
where z (k+1) ∼ N (0,

I N S ), M 1/2 is any matrix such that M 1/2 M 1/2 = M and dt = T /N T . When the noise on the right-hand side is colored in space, i.e. Z(s, t) = W T (t) ⊗ Z S (s), the discretization reads M + dt c (K + B) x (k+1) = M x (k) + τ √ dt √ c M L S z (k+1) ,
where L S is the Cholesky decomposition of Q -1 S , the covariance matrix of the discretized solution Z S of the spatial SPDE (2.28), obtained with the continuous Galerkin FEM [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF].

Proof. For the sake of a clearer exposition, we set H = I, α = 1 and we consider a spatio-temporal white noise Z(s, t) = W (s, t). The proof for the general case follows exactly the same lines as the proof below. The considered SPDE is

∂ ∂t + 1 c (κ 2 -∆) + 1 c γ •∇ X(s, t) = τ √ c W (s, t). (3.8)
For the discretization of the temporal derivative in Equation (3.8), we opt for the implicit Euler scheme, which considers the differential equation

∂X(t) ∂t = f (t, X),
with initial value X (1) = X(t 0 ). The method produces a sequence {X (k) } N T k=1 , such that X (k) approximates X(t 0 +kdt), where dt is the time step size. The approximation reads X (k+1) = X (k) + dtf (t (k+1) , X (k+1) ).

(3.9)

In the specific case of Equation (3.8), the implicit Euler discretization step reads

X (k+1) (s) -X (k) (s) + dt 1 c (κ 2 -∆) + 1 c γ •∇ X (k+1) (s) = √ dtτ √ c W (k+1) S
(s), (3.10) where W (k+1) S (s) is a spatial white noise obtained by integrating out the temporal white noise. In fact, the cumulative time-integral process between 0 and a > 0, a 0 W (s, t)dt, is a purely spatial Gaussian white noise multiplied by the standard deviation of the temporal increment, which is equal to 1/ √ a, hence the √ dt in the right-hand side term.

For ease of notation, we denote X

(k+1) = X (k+1) (s), X (k) = X (k) (s) and W S = W (k+1) S
(s), since the spatial noise is independent of the temporal step k.

At each time step of the temporal discretization, a spatial Finite Element Method method is applied. In our case, we use the continuous Galerkin with Neumann boundary condition. The weak form of Equation (3.10) is

Ω X (k+1) v d s + dt c Ω κ 2 X (k+1) v d s - Ω ∆X (k+1) v d s + Ω γ •∇X (k+1) v d s = = Ω X (k) v d s + √ dtτ √ c W S (v), ∀v ∈ V, (3.11)
where V is the Hilbert space in which we search the solution and W S (v) is the white noise applied to the test function v.

By applying Green's first identity, i.e., by writing

Ω ∆X (k+1) v d s = - Ω ∇X (k+1) • ∇v d s + ∂Ω v • (∇X (k+1) • n) dσ ,
with n being the normal vector on the boundary, and by simplifying the second term thanks to the Neumann boundary condition, we obtain

Ω X (k+1) v d s + dt c Ω κ 2 X (k+1) v d s + Ω ∇X (k+1) • ∇v d s + Ω γ •∇X (k+1) v d s A(X (k+1) ,v) = Ω X (k) v d s C(X (k) ,v) + √ dtτ √ c W S (v) E(v)
, ∀v ∈ V.

Let V h be the space of finite element solutions spanned by the basis functions

{ψ i } N S i=1 .
The Galerkin method allows us to find an approximated solution

X (k+1) h ∈ V h ⊂ V to the SPDE, such that A(X (k+1) h , v h ) = C(X (k) h , v h ) + E(v h ) ∀v h ∈ V h . (3.12)
The functions

X (k+1) h , X (k)
h and v h are linear combinations of the basis functions, with

X (k+1) h = N S i=1 x (k+1) i ψ i ; X (k) h = N S i=1 x (k) i ψ i ; v h = N S i=1 v i ψ i .
Because of the linearity in the first argument of A(•, •) and C(•, •), we get

N S i=1 A(ψ i , v h )x (k+1) i = N S i=1 C(ψ i , v h )x (k) i + E(v h ), ∀v h ∈ V h , ( 3.13)
where

A(ψ i , v h ) = M(ψ i , v h ) + dt c (K(ψ i , v h ) + B(ψ i , v h )) C(ψ i , v h ) = M(ψ i , v h ), with K(ψ i , v h ) = κ 2 M(ψ i , v h ) + G(ψ i , v h ).
Here, M and G are the mass and stiffness operators, respectively v). Since any v h can be written as a linear combination of basis functions, the formulation (3.13) is equivalent to

M(v, w) = Ω vw d s and G(v, w) = Ω ∇v • ∇w d s. B is the advection operator, i.e., B(v, w) = Ω γ •∇vw d s. Finally, E is the operator of the form E(v) = √ dtτ √ c W S (
N S i=1 A(ψ i , ψ j )x (k+1) i = N S i=1 C(ψ i , ψ j )x (k) i + E(ψ j ), ∀j. (3.14) We define M = [M ij ] N S i,j=1 = [M(ψ i , ψ j )] N S i,j=1 , G = [G ij ] N S i,j=1 = [G(ψ i , ψ j )] N S i,j=1 , B = [B ij ] N S i,j=1 = [B(ψ i , ψ j )] N S i,j=1
the mass, stiffness and advection matrices, respectively.

E(ψ j ) is a Gaussian random variable with expectation 0 and covariance equal to

Cov(E(ψ i ), E(ψ j )) = dtτ 2 c Cov [W S (ψ i ), W S (ψ j )] = dtτ 2 c Ω ψ i ψ j d s = dtτ 2 c M ij ,
by following the definition of white noise in Equation (2.1).

If k+1) is the vector containing the values {x k) is the vector containing the values {x

z (k+1) is a (N S )-Gaussian vector such that z (k+1) ∼ N (0, I N S ), x (
(k+1) i } N S i=1 and x (
(k) i } N S
i=1 , then the sparse linear system corresponding to Equation (3.14) reads (3.15) where

M x (k+1) + dt c (K + B)x (k+1) = M x (k) + √ dtτ √ c M 1/2 z (k+1) ,
K = κ 2 M + G and M 1/2 is any matrix such that M 1/2 M 1/2 = M.
When the spatial noise is colored, i.e. Z S (s), the right-hand side operator

E S (v) becomes E S (Z S , v) = √ dtτ √ c Ω Z S v d s and it satisfies E S (Z S,h , v h ) = N S i=1 M(ψ i , v h )z S,i .
Hence,

N S i=1 A(ψ i , ψ j )x (k+1) i = N S i=1 C(ψ i , ψ j )x (k) i + N S i=1 M(ψ i , ψ j )z S,i , ∀j.
If z S = {z S,i } N S i=1 has precision matrix equal to Q S , then the sparse linear system is

M x (k+1) + dt c (K + B)x (k+1) = M x (k) + √ dtτ √ c M L S z (k+1) , (3.16)
where z (k+1) ∼ N (0, I N S ) and L S is the Cholesky decomposition of Q -1 S .

Remark 3.3.1. When the diffusion term includes an anisotropy matrix H, i.e., when

∆ is replaced by ∇ • H ∇, the stiffness operator becomes G(v, w) = Ω H ∇v • ∇w d s,
and the stiffness matrix changes consequently.

Remark 3.3.2. The elements of the matrices M, G, B and K are non-zero only for pairs of basis functions which share common triangles. This implies that the matrix (M + dt c (K + B)) is sparse and that Equation (3.7) can be solved by LU decomposition [START_REF] Turing | Rounding-off errors in matrix processes[END_REF] in an efficient way.

Remark 3.3.3. The discretization error introduced by the scheme (3.12) is defined by

E (k) h = e (k) h + p (k) h , where e (k) h = X (k) h -Π h X t k , p (k) h = Π h X t k -X t k for 1 ≤ k ≤ N T . For k ∈ {1, . . . , N T }, the error e (k)
h is the difference between the approximated solution given by the scheme and the elliptic projection

Π h (•) on V h of the exact solution at time kdt. The error p (k)
h is the difference between the projection of the exact solution on V h at time kdt and the exact solution at kdt. We have the following covergence result [START_REF] Boulakia | Simulation of SPDEs for Excitable Media Using Finite Elements[END_REF]:

E(||E (k) h || 2 ) ≤ K(h + √ dt), (3.17)
where K is a constant.

Stabilization of the unsteady advection-dominated SPDE

When the advection term is too strong with respect to the diffusion term in Equation

(2.27), advection-domination occurs. In the framework outlined above, when α = 1, the asymmetric matrix M + dt c (K + B) becomes ill-conditioned, which induces oscillations and unstable solutions for the continuous Galerkin approximation. Specifically, the advection-domination occurs when the Péclet number Pe h = γ h 2λ > 1, where λ is the coefficient of the isotropic Laplacian operator (see, for example, [START_REF] Mekuria | Adaptive finite element method for steady convection-diffusion equation[END_REF] or Quarteroni (2008, Chapter 5)).

We saw in Section 3.2 that one possible solution is to decrease the diameter h, i.e., to refine the triangulation, until the advection no longer dominates on the elementlevel, with Pe h < 1. However, in many cases this is not a feasible solution because it would increase the number of vertices beyond computation limits. Another solution, adopted here, is to introduce a stabilization term. Many stabilization approaches are possible, some being more accurate than others (see Section 3.3.1 or Quarteroni (2008, Chapter 5)). In our case, we opt for the Streamline Diffusion (SD) stabilization approach [START_REF] Hughes | A multidimensional upwind scheme with no crosswind diffusion[END_REF], considered as a good trade-off between accuracy and computational complexity. Essentially, the SD approach consists in stabilizing the advection by introducing an artificial diffusion term along the advection direction.

The following proposition presents the stabilized solution to (2.27). Proposition 3.2. Assume the same hypotheses as in Proposition 3.1 with α = 1. (3.18) where S = [S ij ] N S i,j=1 is the matrix of the Streamline Diffusion stabilization operator S, such that

The solution to Equation (2.27) in presence of Streamline Diffusion stabilization is

M + dt c (K + B + S) x (k+1) = M x (k) + τ √ dt √ c M 1/2 z (k+1) ,
S ij = S(ψ i , ψ j ) = h γ -1 Ω (γ •∇ψ i )(γ •∇ψ j ) d s , and τ = τ |H +h γ -1 γ γ | -1/4
(|H|) 1/4 . When the noise on the right-hand side of Equation (2.27) is colored in space, the discretization becomes

M + dt c (K + B + S) x (k+1) = M x (k) + τ √ dt √ c M L S z (k+1) ,
where z (k+1) and L S are as in Proposition 3.1.

The proof of the discretized equation follows the same reasoning as that of Proposition 3.1 with the addition of the matrix S. The Streamline Diffusion approach can be seen as a perturbation of the original SPDE [START_REF] Bank | Some upwinding techniques for finite element approximations of convection-diffusion equations[END_REF]. Indeed, by making the classical hypothesis of Neumann boundary condition on Ω and by using the Green's first identity, we get

Ω (γ •∇X (k+1) )(γ •∇v) d s = - Ω ∇ • (γ γ )∇X (k+1) v d s .
As a consequence, the original SPDE (2.27) can be rewritten with an additional diffusion term as

∂ ∂t + 1 c κ 2 -∇ • H +h γ -1 γ γ ∇ + γ •∇ X(s, t) = τ √ c Z(s, t). (3.19)
The term (h γ -1 γ γ ) acts as an anisotropic "diffusion" matrix that is added to the anisotropy matrix H of the original diffusion. This extra diffusion stabilizes the advection directed along the direction γ. By following the proof of Proposition 2.1, we find that the marginal variance of the spatial field X(•, t) of Equation (3.19) is equal to

σ 2 = τ 2 Γ(α tot -d/2) Γ(α tot )2(4π) d/2 κ 2(αtot-d/2) |H +h γ -1 γ γ | 1/2 .
For the variance to be equal to the variance in Proposition 2.1, τ must be replaced

by τ = τ |H +h γ -1 γ γ | 1/4 (|H|) -1/4 .

Spatio-temporal GMRF approximation

Proposition 3.3. In presence of an advection-dominated flow and a spatio-temporal white noise on the right-hand side of Equation (2.27), the discretized vector x (k+1) on the mesh T at each time step is the solution to the following equation: (3.20) where k+1) . In presence of a spatiotemporal noise colored in space on the right-hand side of Equation (2.27), the matrix E reads

x (1) ∼ N (0, Σ), x (k+1) = D x (k) + E z (k+1) ,
D = M + dt c (K + B + S) -1 M, E = τ √ dt √ c M + dt c (K + B + S) -1 M 1/2 , (3.21) and z (k+1) ∼ N (0, I N S ) is independent of x (1) , . . . , x (
E = τ √ dt √ c M + dt c (K + B + S) -1 M L S ,
where L S is defined in Proposition 3.1.

Proof. Starting from Equation (3.18), which represents the numerical scheme for the advection-diffusion spatio-temporal SPDE with stabilization, it is straightforward to obtain (3.20).

When the SPDE is not advection-dominated, which implies that no stabilization term is needed, Equation (3.21) is replaced by the similar equation where the matrix S is deleted and τ is replaced by τ .

Σ, which is the covariance matrix of the spatial trace x (1) at the first time step, can be taken to be equal to any admissible positive definite matrix. The closer Σ is to the covariance C S of X(•, t), the faster the stationary solution is obtained. When the hypotheses of Proposition 2.1 are satisfied, an efficient option is to choose Σ as the Matérn covariance of Equation (2.29).

To obtain fast inference and prediction computations, the precision matrix of the spatio-temporal discretized solution x 1:N T = [x (1) , . . . , x (N T ) ] must be sparse. For this reason M is replaced by the diagonal matrix M, where M ii = ψ i , 1 [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF]. This technique is called mass lumping and is common practice in FEM (Quarteroni, 2008, Chapter 5). From now on, we always use the diagonal matrix M, but for ease of reading, it will still be denoted M. ) , . . . , x (N T ) ] be the vector containing all spatial solutions until time step N T of Equation (3.20). The global precision matrix Q of the

Proposition 3.4. Let x 1:N T = [x (1
vector x 1:N T of size (N S N T , N S N T ) reads Q =            Σ -1 + D F -1 D -D F -1 0 . . . 0 -F -1 D F -1 + D F -1 D -D F -1 . . . . . . . . . . . . . . . . . . 0 . . . . . . -F -1 D F -1 + D F -1 D -D F -1 0 . . . 0 -F -1 D F -1           
, (3.22) where F = E E .

Proof. Let us denote x 1:N T = [x (1) , . . . , x (N T ) ] the vector containing all spatial solutions until time step N T . Then,

x 1:N T = R x (1) z 2:N T , with z 2:N T = [z (2) , . . . , z (N T ) ] and R =               I N S 0 0 0 . . . 0 D E 0 0 . . . 0 D 2 D E E 0 . . . 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0 . . . . . . . . . D 2 D E              
. R has a block structure which allows easy computation of its inverse

R -1 =              I N S 0 0 0 . . . 0 -E -1 D E -1 0 0 . . . 0 0 -E -1 D E -1 0 . . . 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0 0 . . . . . . 0 -E -1 D E -1             
.

The precision matrix of x 1:N T is thus

Q = R -1        Σ -1 0 . . . 0 0 I N S . . . 0 . . . . . . . . . . . . 0 0 . . . I N S        R -1 .
By denoting F = E E , the global precision matrix reads

Q =            Σ -1 + D F -1 D -D F -1 0 . . . 0 -F -1 D F -1 + D F -1 D -D F -1 . . . . . . . . . . . . . . . . . . 0 . . . . . . -F -1 D F -1 + D F -1 D -D F -1 0 . . . 0 -F -1 D F -1           
.

By replacing the values of D and F and by defining J = M + dt c (K + B + S) , we obtain

Q = c τ 2 dt            Σ -1 + Q S -Q S M -1 J 0 . . . 0 -J M -1 Q S J M -1 Q S M -1 J + Q S -Q S M -1 J . . . . . . . . . . . . . . . . . . 0 . . . . . . -J M -1 Q S J M -1 Q S M -1 J + Q S -Q S M -1 J 0 . . . 0 -J M -1 Q S J M -1 Q S M -1 J           
.

Sparsity pattern of the precision matrix

Here, we report two examples of sparsity patterns of the spatio-temporal precision matrix, computed with N S = 5 2 and N T = 5 and N S = 20 2 and N T = 5 respectively.

The sparsity is due both to the discretization in space with the FEM and to the use of the implicit Euler scheme. In this chapter we present some examples of non-conditional simulation of stationary spatio-temporal Gaussian fields, solutions to the unsteady advection-diffusion SPDE (2.27). The main method that will be used is the FEM/FDM discretization described in Section 3.3. We will compare this method to a spectral method based on the approximation of the Fourier Transform of the field by Fast Fourier Transform. Finally, we will show how the advection-domination influences the solution of the SPDE (2.27) and how the Streamline Diffusion stabilization can tackle this issue.

Some implementation details are available in Appendix A.1.

Simulation of spatio-temporal SPDEs

We consider the advection-diffusion SPDE (2.27) with d = 2. The spatio-temporal When α tot ≤ 1, r cannot be computed; in this case, we consider that r = √ 0.8/κ.

domain Ω × [1, T ] of X(s, t), with Ω ⊂ R 2 ,
A non-conditional simulation of the spatio-temporal Gaussian vector x 1:N T on the discretized domain T with known precision matrix Q as in Equation (3.22) can be obtained through

x 1:N T = L z, (4.1)
where z is a N S N T vector with independent zero-mean, unit variance and normally distributed random components, and L is a matrix such that [START_REF] Gentle | Computational Statistics[END_REF])

L L = Q -1 . (4.2)
Since Q -1 is not known, we can compute Q chol as the Cholesky decomposition of the precision matrix Q. Then, the simulated vector x 1:N T is computed as the solution of the linear system Q chol x 1:N T = z. Two performance issues arise from this approach. First, the computation of the Cholesky decomposition of Q is intractable for large problems or when the matrix is not sparse enough; then, once computed, this decomposition must be stored, and is used to solve a linear system. Both these tasks get increasingly expensive as the size or the filling of Q chol increases.

However, we know that, by construction of Q, we only need to store the sparse spatial matrices of the tridiagonal block structure, which results in a cheaper storage.

Likewise, a non-conditional simulation of the spatio-temporal field X(s, t) on the discretized domain T , i.e., the vector x 1:N T , can be obtained by solving the sparse linear system (3.20) governing the field in space and updating the Euler scheme at each time step. In this case, only sparse (N S , N S ) spatial matrices are used, which reduces the computational cost.

The convergence of the spatio-temporal field to the stationary solution obtained from the theory is observed by comparing the empirical spatio-temporal covariance with the theoretical one.

First, we compare the empirical covariance of the spatial trace with the theoretical covariance of the corresponding spatial model, when it exists (see Proposition 2.1).

In Figure 4.1 we show the spatial trace of the stationary solution of an advection-

diffusion SPDE with α = 1, α S = 0, κ = 0.1, c = 1, γ = [1, 0] , H = 1 1 1 4 ,

Comparison FEM/FDM with spectral method

Understanding the marginal properties of the model which is solution to the SPDE is important, as they provide additional information about the process. We already know that the covariance can be directly computed from the spectral density, even though this is not always possible analytically. We consider the spectral density of the nonseparable model in order to compute the space-time covariance and marginal spatial and temporal ones as well.

An approximation of the covariances of X, solution to the advection-diffusion SPDE (2.27), can be obtained using a procedure related to the classical development on Fourier basis, for which the computations can be efficiently obtained through the use of the Fast Fourier Transform (FFT), as proved by [START_REF] Sigrist | Stochastic partial differential equation based modelling of large space-time data sets[END_REF] and Carrizo [START_REF] Vergara | Development of geostatistical models using stochastic partial differential equations[END_REF]. This method can be applied to any evolution equation presented in Section 2.2.6. We refer to Carrizo Vergara (2018, Chapter 6) for a detailed exposition of the method.

As an illustration of the method, in Figure 4.4 we plot the marginal spatial and temporal covariance functions obtained with the FFT approach from the SPDE (2.27)

with α = 0, α S = 2, κ = 0.1, τ = 1, c = 0.
2, along with the theoretical covariances of the separable model with spatial covariance equal to a Matérn with ν = α S -1 = 1 and temporal covariance equal to an exponential with scale c (see Proposition 2.1). We note that the spatial and temporal covariances computed with the FFT approximation fit well the theoretical covariances.

In Figure 4.5 we plot the marginal spatial covariance functions obtained with both the FEM/FDM and FFT approaches from the SPDE (2.27) with spatio-temporal white noise (α S = 0), κ = 0.5, τ = 1, c = 1. α is set to α 1 = 2 and α 2 = 3 in order to obtain a Matérn spatial trace with ν 1 = 1 and ν 2 = 2, respectively. The theoretical marginal variance and practical range, computed from Equation (2.29), are σ 2 1 = 0.159 and r 1 = 5.657 (and σ 2 2 = 0.318 and r 2 = 8). These values are well retrieved by both the approximation approaches, with a little error for the variances estimated by the FEM/FDM approach. This is visible in the first star on the left, which is lower than the dashed and solid lines at the same abscissa. We already remarked that the marginal variances were slightly underestimated in Figure 4.3. This is probably due to the discretization in space, which, being a bit coarse here, smooths the behavior at the origin.

Simulation of advection-diffusion SPDEs as shifted diffusion SPDEs

In the following, we illustrate with simulations the result of Section 2.3.3, i.e., the fact that the result of the advection-diffusion SPDE is equal to a shifted solution of the classical diffusion SPDE. We consider the solution of the advection-diffusion SPDE with α = 1, α S = 0, dt = 1, τ = 1, c = 1, κ = 0.3 and γ = [1, 0] , along with the solution of the corresponding diffusion SPDE (with same parameters α, α S , κ, τ and c), which has been computed on a larger grid, so that we can shift the solution along the x-axis of an interval dx = dtγ x = 1 every time step.

In Figure 4.6, we plot the covariance function of the solutions of the advectiondiffusion SPDE between 4 following time steps, when the simulation has already converged. We remark how the higher values of the covariance map follow the direction [1, 0] , as expected from the advection term with velocity parameter γ = [1, 0] .

In Figure 4.7, we plot the covariance function of the solutions of the purely diffusive SPDE with the same parameters, after being shifted along the x-axis of an interval dx = dtγ x = 1 every time step. Moreover, in Table 4.2, we report the mean and the standard devation of the differences between the two experimental spatial covariances (shifted diffusion SPDE and corresponding advection-diffusion SPDE) over 4 following time steps. We note that the plots are very similar and the differences very small, which empirically illustrates the result of Section (2.

Simulation of advection-dominated SPDEs

In the following, we simulate a spatio-temporal field from the SPDE (2.27) when the advection term is dominant with respect to the diffusive term (see Section 3.3.1. We consider the solution of the SPDE (2.27) with α = 1, α S = 0, dt = 1, τ = 1, c = 1, κ = 0.3 and γ = [0, 5] . We plot in Figure 4.8 the spatial trace of the solution to the SPDE at 4 following steps. We remark how the strong advection coming from the left side influences the spatial trace, causing problems in the solution. To better capture the amplitude of the fluctuations, we report here the average variance over 50 time steps. In the purely diffusive case, the average variance is 0.147, in the not stabilized advection-dominated case it is 0.958, while in the stabilized advection-dominated case it is 0.167. In Figure 4.9 we plot the same spatial traces but with the introduction of the Streamline Diffusion stabilization term into the FEM. We note that now the solution is stable and we remark the influence of the stabilization term on the diffusive behavior. Indeed, the additional diffusive term h/ γ ∇ • γ γ ∇X in the SPDE is visible as an anisotropic covariance structure that the spatial trace clearly shows. k) observations scattered in the spatial domain Ω. There is thus a total of n = N T k=1 n (k) spatio-temporal data collected in the vector y 1:

N T = [(y (1) ) , . . . , (y (N T ) ) ] .
We consider a statistical model with fixed and random effects. The fixed effect is a regression on a set of covariates and the random effect is modeled as the FEM discretization of a random field described by the SPDE (2.27) with the addition of a random noise: where b is the vector of q fixed effects and η is a (n, q) matrix of covariates with [η] jk = η k (s j , t j ), j = 1 . . . , n and k = 1, . . . , q. The matrix A is the (N S N T , n) projection matrix between the points in T and the data, and ε is a standard Gaussian random vector with independent components, independent of x 1:N T . When the observation locations do not change during the time window, A A is a (N S N T , N S N T ) blockdiagonal matrix with all (N S , N S ) equal blocks.

y 1:N T = η b +A x 1:N T +σ 0 ε, (5.1)
In the left panel of

Estimation of the parameters

The parameters of the SPDE are estimated using Maximum Likelihood. We collect the parameters of the SPDE in the vector θ = [κ, γ 1 , γ 2 , c, τ ], while all the parameters of the statistical model are collected in ψ = [θ , b , σ 0 ]. Following (5.1), y 1:N T is a Gaussian vector with expectation η b and covariance matrix where Q(θ) is a precision matrix of size (N S N T , N S N T ) depending on the parameters θ. For ease of notation, we use Q instead of Q(θ). The log-likelihood is equal to

Σ y 1:N T = A Q -1 (θ) A +σ 2 0 I n ,
L(ψ) = - n 2 log(2π) - 1 2 log|Σ y 1:N T | - 1 2 (y 1:N T -η b) Σ -1 y 1:N T (y 1:N T -η b). (5.2)
We use the Broyden, Fletcher, Goldfarb, and Shanno optimization algorithm [START_REF] Nocedal | Numerical Optimization[END_REF], that makes use of the first and second-order derivatives of the objective function. The gradients of the log-likelihood function (5.2) with respect to the different parameters included in ψ are approximated with finite differences. We now propose a computationally efficient formulation of each term of the log-likelihood (5.2).

Proposition 5.1. In the framework outlined above, we have

log|Σ y 1:N T | = n log σ 2 0 -log|Q| + log|Q +σ -2 0 A A |. (5.3)
Proof. To compute log|Σ y 1:N T |, let us consider the augmented matrix

Σ c =   Q -1 Q -1 A A Q -1 Σ y 1:N T   .
(5.4) Hence,

Q c = Σ -1 c = Q +σ -2 0 A A -σ -2 0 A -σ -2 0 A σ -2 0 I . (5.5)
Using block formulas, we have

log|Σ c | = -log|Q c | = -log|Q| + n log σ 2 0 ,
and

log|Σ c | = log|Σ y 1:N T | + log|Q -1 -Q -1 A Σ -1 y 1:N T A Q -1 | = log|Σ y 1:N T | -log|Q +σ -2 0 A A |,
where the last equality is a consequence of the Woodbury identity. This leads to the result.

Proposition 5.2. The term log|Q| in Equation (5.3) can be computed with the computationally efficient formula

log|Q| = log|Σ -1 | + (N T -1) log|F -1 |, (5.6)
where Σ -1 is the precision matrix of the spatial trace x (1) at the first time step, and

F -1 = c τ 2 dt (M + dt c (K + B + S)) M -1 Q S M -1 (M + dt c (K + B + S)),
where Q S is the precision matrix of the discretized spatial noise Z S defined in Proposition 3.1.

Proof. Following [START_REF] Powell | Calculating determinants of block matrices[END_REF], let

N N = [N i j] N i,j=1 be a (nN, nN ) matrix, which is partitioned into N blocks, each of size (n, n). Then the determinant of N N is |N N | = N k=1 |α (N -k) kk |,
where α (k) is defined by

α (0) ij = N ij α (k+1) ij = α (k) ij -α (k) i,N -k (α (k) N -k,N -k ) -1 α (k) N -k,j , k ≥ 1.
Q is a block-matrix organized as N N . Hence, the formula for |Q| is

|Q| = |Σ -1 ||F -1 | N -1 .
(5.7)

Applying the logarithm, we obtain Equation (5.6).

Remark 5.1.1. Note that F -1 is a (N S , N S ) sparse, symmetric and positive definite matrix. The computation of its determinant can be obtained by Cholesky decomposition of F -1 .

We here report how the log-determinant of a matrix N is computed by Cholesky decomposition. We have

log|N| = log|N chol N chol | = 2 log n i=1 N chol,ii = 2 n i=1 log(N chol,ii ),
since N chol is a triangular matrix, whose determinant is the product of the diagonal elements.

The term log|Q +σ -2 0 A A | requires a detailed analysis. The term σ -2 0 A A is an (N S N T , N S N T ) diagonal block matrix, whose (N S , N S ) blocks are sparse (see Figure 5.2). The computation of log|Q +σ -2 0 A A | is not as straightforward as in the case of log|Q|, because there is no way of reducing the computation to purely spatial matrices. Depending on the size N S N T , we can either apply a Cholesky decomposition of the (N S N T , N S N T ) matrix (Q +σ -2 0 A A ) or the matrix-free approach proposed in Pereira et al. ( 2022), here sketched. The logarithm function is first approximated by a Chebyschev polynomial P (•) [START_REF] Chebyshev | Théorie des mécanismes connus sous le nom de parallélogrammes[END_REF], then the Hutchinson's estimator [START_REF] Hutchinson | A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines[END_REF]) is used to obtain a stochastic estimate of tr[P (Q +σ -2 0 A A )]. The method is detailed in Section 5.2.

Concerning the computation of the quadratic term of the log-likelihood, we can work with the more convenient expression obtained thanks to the Woodbury formula

Σ -1 y 1:N T = σ -2 0 I n -σ -4 0 A (Q +σ -2 0 A A ) -1 A . Hence (y 1:N T -η b) Σ -1 y 1:N T (y 1:N T -η b) = σ -2 0 (y 1:N T -η b) I(y 1:N T -η b) -σ -4 0 (y 1:N T -η b) A (Q +σ -2 0 A A ) -1 A(y 1:N T -η b).
The second term can be computed either by Cholesky decomposition or using the Conjugate Gradient (CG) method, outlined in Section 5.2.1. This latter method solves N v = w with respect to v and computes v sol = w v, with N = (Q +σ -2 0 A A ) and w = A(y 1:N T -η b). In this case, it is useful to find a good preconditioner for the matrix (Q +σ -2 0 A A ) to ensure fast convergence of the CG method. We found that a temporal block Gauss-Seidel preconditioner (Young, 1971, Chapter 3) was a good choice in this case (see Section 5.2.2).

Matrix-free methods

In computational mathematics, a matrix-free method is an algorithm for solving a linear system of equations that does not store the coefficient matrix explicitly, but accesses the matrix by evaluating matrix-vector products. Such methods are preferable when the system matrix is so large that storing and manipulating it would cost too much of memory and computing time, even with the use of methods for sparse matrices. Many iterative methods allow for a matrix-free implementation.

We will benefit from these methods for the computation of the terms of the loglikelihood to obtain fast computations even for very large matrices Q, i.e., large N S N T . Some implementation details are available in Appendix A.1.

Conjugate Gradient method

Suppose we want to solve the system of linear equations N x = b for the vector x, where the known (n, n) matrix N is symmetric, positive-definite, and real, and b is known as well. We denote the unique solution of this system by x . The Conjugate Gradient (CG) method [START_REF] Hestenes | Methods of conjugate gradients for solving linear systems[END_REF]) is an iterative method that allows us to approximately solve systems where n is so large that the direct method would take too much time. It is in this sense a matrix-free method.

The solution x is the unique minimizer of the following quadratic function

f (x) = 1 2 x N x -x b, x ∈ R n .
(5.8)

The idea behind the CG method is to take the approximated solution x k at each iteration k along always different search directions p 0 , p 1 , . . . , p n-1 . We take the first basis vector p 0 to be the negative of the gradient of f at x = x 0 , leading to p 0 = b -N x 0 , which is also equal to the residual r 0 = b -N x 0 . The direction at iteration k + 1 is constructed by conjugation of all the residuals r i = b -N x i for i ≤ k. At each iteration, the residual r k is orthogonal to the previous search directions, so it is guaranteed to produce a new, linearly independent search direction unless the residual is zero, in which case the problem is solved. The conjugation constraint is an orthonormal-type constraint, which makes the algorithm an example of Gram-Schmidt orthonormalization. The number of iterations K can be either set a priori or chosen by defining an error tolerance. The algorithm is detailed in 1.

Algorithm 1 Conjugate Gradient

Require:

Matrix N ∈ R n × R n . Vector b ∈ R n . Ensure: The approximated solution x K ≈ x of N x = b. Set x 0 = 0, r 0 = b Set p 0 = r 0 for k = 1, 2, . . . , K do Compute a step length α k = r k-1 r k-1 p k-1 N p k-1 Update the approximate solution x k = x k-1 +α k p k-1 Update the residual r k = r k-1 -α k N p k-1 Compute a gradient correction factor β k = r k r k r k-1 r k-1 Set the new search direction p k = r k +β k p k-1

end for

The algorithm seems to require storage of all previous searching directions and residual vectors, as well as many matrix-vector multiplications, leading to expensive computations. However, we can show that this is not true. We define P k = span{p 0 , . . . p k-1 } the k-dimensional subspace spanned by the searching directions until iteration k -1. Since the searching vectors are built from the residuals, the subspace R k = span{r 0 , . . . r k-1 } is equal to P k . In fact, as each residual is orthogonal to the previous search directions, it is also orthogonal to the previous residuals, i.e., r i r j = 0 for i = j. Recalling that p k belongs to P k+1 , this fact implies that each new subspace P k+1 is formed from the union of the previous subspace P k and the subspace N P k . Hence,

P k = span{p 0 , N p 0 , N 2 p 0 , . . . , N k-1 p 0 } = span{r 0 , N r 0 , N 2 r 0 , . . . , N k-1 r 0 }
This subspace is a Krylov subspace, as it is created by repeatedly applying a matrix to a vector, and it has a good property: since N P k is included in P k+1 , the fact that the next residual r k+1 has to be orthogonal to P k+1 implies that r k+1 has to be N-orthogonal to P k . Gram-Schmidt conjugation becomes easy, because r k+1 is already N-orthogonal to all the previous search directions except p k . Hence, it is no longer necessary to store old search vectors to ensure the N-orthogonality of new search vectors. This feature is what makes CG a matrix-free approach, because both the space complexity and time complexity per iteration are reduced from O(n 2 ) to O(m), where m is the number of nonzero entries of N.

The condition number of a problem measures how sensitive the answer is to perturbations in the input data and to roundoff errors made during the solution process [START_REF] Demmel | On condition numbers and the distance to the nearest ill-posed problem[END_REF]. A problem with a low condition number is said to be well-conditioned, while a problem with a high condition number is said to be illconditioned. The condition number with respect to inversion of a nonsingular square matrix N is

κ(N) = N N -1 .
For a condition number κ(N), it turns out that the CG algorithm converges in O( κ(N)) iterations. However, in practice, the CG algorithm often converges faster than predicted by this upper bound.

Gauss-Seidel preconditioner

The general concept behind a preconditioner is to reduce the condition number of an ill-conditioned matrix to obtain less sensitive solutions to a linear system. Given a linear system N x = b, we want to find the matrix P R and/or P L such that the condition number of N P -1 R (right preconditioner) or P -1 L N (left preconditioner) or P -1 L N P -1 R is lower than that of N, and that we can easily solve P L w = g or P R w = g for any g. If this matrix exists, then we can solve the systems N P -1 R y = b, P R x = y (right preconditioner)

P L y = b, P -1 L N x = y (left preconditioner) P -1 L N P -1 R y = P -1 L b, P R x = y (two-side preconditioner) instead of N x = b.
Let us consider only the two-side preconditioner. The following reasoning can be adapted easily to any other type of preconditioner, by setting P L = I for a right preconditioner and P R = I for a left preconditioner. The preconditioned CG method converges in O( κ(P -1

L N P -1 R )).
Since this condition number is lower than the condition number of N, this implies fewer iterations to converge. However, each iteration is more expensive, due to the extra cost of computing P R x = y or P L y = b. This computation does not need to be explicit; it is only necessary to be able to compute the effect of applying P -1 to a vector, i.e., it is matrix-free.

Intuitively, preconditioning is an attempt to stretch the quadratic form (5.8) to make it appear more spherical, so that the eigenvalues are close to each other. The best choice for a preconditioner would be of course P R = I and P L = N (or viceversa), since in this case the condition number of P -1 L N P -1 R would be equal to 1 and the quadratic form would be perfectly spherical; however, this does not make life easier, because N is ill-conditioned and the solution of P L y = b would still be a problem. The simplest preconditioner is a diagonal matrix D whose diagonal entries are identical to those of N (i.e., P R = I and P L = D). The process of applying this preconditioner, known as Jacobi preconditioning, is equivalent to scaling the quadratic form along the coordinate axes (by comparison, the perfect preconditioner P L = N scales the quadratic form along its eigenvector axes). A diagonal matrix is trivial to invert, but is often only a mediocre preconditioner.

Another possibility is to use other iterative methods as preconditioners, such as the Gauss-Seidel method or the SOR(ω) method (Successive over-relaxation) [START_REF] Young | Iterative methods for solving partial difference equations of elliptic type[END_REF].

A more elaborate preconditioner is the incomplete Cholesky preconditioning. Incomplete Cholesky factorization is a variant of the Cholesky decomposition in which little or no fill is allowed; N is approximated by the product LL , where L might be restricted to have the same pattern of nonzero elements as N; other elements of L are thrown away. To use P L = LL as a preconditioner, the solution to LL y = b is computed by backsubstitution, i.e., the inverse of LL is never explicitly computed.

Unfortunately, incomplete Cholesky preconditioning is not always stable.

The preconditioner that we will use in this work is the Gauss-Seidel (GS) preconditioner, whose matrix form is here detailed. Given the matrix N, we have

N = L + D + U ,
where D is the diagonal of N and L and U are the lower-triangular part and upper-triangular part with zero on the diagonal, respectively. One iteration of the GS method reads

x k+1 = x k -(L + D) -1 (N x k -b)
and we refer to the preconditioner P L = L + D (and P R = I) as Gauss-Seidel preconditioner. A good property of this preconditioner is that ρ(I -(L + D) -1 N) < 1, where ρ is the spectral radius, i.e., for a positive definite matrix the GS method always converges. The preconditioned Conjugate Gradient method is described in Algorithm 2.

In the specific case of the precision matrix Q +σ -2 0 A A , all the blocks on the diagonal are equal (except the first and the last one), and the same occurs also on the upper and lower diagonals. For this reason, the Gauss-Seidel preconditioner P L can be divided in N T temporal blocks P L = [P L,1 , . . . , P L,N T ] with the following properties. For the first time step, P L,1 = D 1 = Q (1,1) +σ -2 0 A S A S , where (1, 1) is the index of the temporal block in the matrix Q. For the time steps

i = 2, . . . , N T -1 , P L,i = L i + D i = Q (i,i-1) + Q (i,i) +σ -2
0 A S A S , and the matrices N i are equal for all i. Finally, for the last temporal step,

P L,N T = L N T + D N T = Q (N T ,N T -1) + Q (N T ,N T ) +σ -2
0 A S A S , which is different from the previous blocks. This means that only a few number of spatial matrices must be stored and that the computation of P L y = b can be obtained by computing P L,j y j = b j for each temporal block j = 1, . . . , N T of the matrix and of the vectors, before reconstructing the vector y by stacking all y j . This leads to N T sparse systems of size (N S , N S ), that can be solved with a LU decomposition [START_REF] Turing | Rounding-off errors in matrix processes[END_REF] of the matrix P L,j , by remarking that the matrix P L,j is not symmetric.

Algorithm 2 Preconditioned Conjugate Gradient

Require: Matrix N ∈ R n × R n . Left preconditioner P ∈ R n × R n . Vector b ∈ R n . Ensure: The approximated solution x K ≈ x of N x = b. Set x 0 = 0, r 0 = b Compute r0 s.t. P r0 = r 0 Set p 0 = r0 for k = 1, 2, . . . , K do Compute a step length α k = r k-1 rk-1 p k-1 N p k-1 (where P rk-1 = r k-1 ) Update the approximate solution x k = x k-1 +α k p k-1 Update the residual r k = r k-1 -α k N p k-1 Solve P rk = r k Compute a gradient correction factor β k = r k rk r k-1 rk-1 (where P rk-1 = r k-1 )
Set the new search direction p k = rk + β k p k-1 (where P rk = r k ) end for

Chebyshev algorithm

Approximating a function applied to a matrix can be performed in three steps using a Chebyshev algorithm [START_REF] Fox | Chebyshev Polynomials in Numerical Analysis[END_REF][START_REF] Higham | Functions of Matrices: Theory and Computation[END_REF]. First, an interval [a, b] that contains all the eigenvalues of the matrix is defined. Then a polynomial approximation of the function over [a, b] is derived using Chebyshev sums [START_REF] Chebyshev | Théorie des mécanismes connus sous le nom de parallélogrammes[END_REF][START_REF] Clenshaw | A note on the summation of chebyshev series[END_REF]. Finally, the matrix operation is applied to this polynomial instead of the original function, using an iterative method that only involves matrixvector products. The first step is done by considering the following property for symmetric matrices.

Theorem 5.1. (Gerschgorin circle theorem [START_REF] Gerschgorin | Uber die abgrenzung der eigenwerte einer matrix[END_REF]) Any eigenvalue λ

of a symmetric matrix N ∈ R n × R n satisfies λ ∈ i∈[1:n] [N ii -r i , N ii + r i ] ,
where

r i = i =j |N ij |.
Hence, all the eigenvalues of N are contained in the interval

min i∈[1,n] (N ii -r i ), max i∈[1,n] (N ii + r i ) , i ∈ [1, n], j ∈ [1, n].
If N is positive (semi)definite, then a = 0 is a lower bound of its eigenvalues. Consequently, the interval for the eigenvalues of Q +σ -2 0 A A can be taken as 0, max i∈ [1,n]

n j=1 |(Q +σ -2 0 A A ) ij | (if the diagonal elements of Q +σ -2 0 A A are non-negative).
A different lower bound must be found for the functions that are not defined at 0, such as the logarithm. In our case, we will set a lower bound close to 0, i.e., a = 0.001. For all the tests where we could compute the eigenvalues directly, this bound was found to be adapted.

The second step, consisting in finding a polynomial approximation of the function, is obtained by computing the Chebyshev sum of the function. The Chebyshev polynomials of the first kind are a set of orthogonal polynomials defined as the solutions to the Chebyshev differential equation and denoted T n (x). They are obtained from the recurrence relation

T 0 (x) = 1 T 1 (x) = x T n+1 (x) = 2x T n (x) -T n-1 (x).
(5.9)

The Chebyshev polynomials are important in approximation theory because the roots of T n (x), which are also called Chebyshev nodes, are used as matching points for optimizing polynomial interpolation. The resulting interpolation polynomial provides an approximation that is close to the best polynomial approximation to a continuous function under the maximum norm.

The last step of the algorithm is outlined in Algorithm 3. This approach can be considered as a matrix-free algorithm. Indeed, it does not actually require the operator N to be stored in memory. Rather, it relies only on the product between the operator and vectors. This property is clearly desirable in a context where the size of the vectors and matrices may be so large that any gain in memory is appreciated.

Algorithm 3 Chebyshev algorithm

Require

: Matrix N ∈ R n × R n . Vector x ∈ R n . A set of Chebyshev coefficients c 0 , . . . , c m ∈ R for the function h(N).
Ensure: (-2) y = y +c k u u (-2) = u (-1) u = u (-1) end for This algorithm is used to approximately compute log Q +σ -2 0 A A . In fact, log Q +σ -2 0 A A needs to be computed by spectral decomposition of Q +σ -2 0 A A = U Λ U, which is computationally expensive for large matrices. We know that

The product y = P h (N) x = 1 2 c 0 T 0 (N) + m k=1 c k T k (N) x, approxima- tion of h(N) x. Set u (-2) = u (-1) = u = y = 0 P = 2 a-b N -b+a b-a I Set u (-2) = x Set u (-1) = P x Set y = 1 2 u (-2) +c 1 u (-1) for k = 2, . . . , m do u = P u (-1) -u
f (Q +σ -2 0 A A ) = U f (Λ) U, but, when f is a polynomial such that f (x) = P (x) = p i=1 a i x i , the function applied to the matrix Q +σ -2 0 A A becomes P (Q +σ -2 0 A A ) = p i=1 a i (Q +σ -2 0 A A ) i .
For this reason, it is useful to approximate the function f (•) = log(•) by a polynomial P on an interval that contains all the eigenvalues

{λ i } N S N T i=1 of Q +σ -2 0 A A .

Hutchinson's estimator

The Hutchinson's estimator is an unbiased estimator for the trace of a matrix [START_REF] Hutchinson | A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines[END_REF]. Consider the term tr(N). This can be approximated with the Hutchinson's estimator as (5.10) where z j is a vector of independent Gaussian random elements or z j = 2I(u > 0.5)-1

tr (N) ≈ 1 m m j=1 z j N z j ,
for u ∼ U(0, 1) such that z j is a vector of independent random elements -1 or 1 of equal probability (the Rademacher distribution). The approximation gets better as m increases, but even a few number of terms in the sum allows finding a good approximation of the trace.

This result implies that we do not have to compute the eigenvalues of the matrix N when it is computationally demanding, but we can only use matrix-vector multiplication between N and the vectors z j . In our case, this approximation is used to

approximately compute tr[P (Q +σ -2 0 A A )].

Prediction by Kriging

Under a Gaussian assumption, optimal prediction is the conditional expectation, also known in the geostatistics literature as kriging. We detail here two prediction settings:

space-time interpolation and temporal extrapolation.

In the space-time interpolation setting, the spatio-temporal vector x 1:N T is predicted on the entire spatial mesh during the time window [1, T ], i.e. on T , using the data y 1:N T defined in Equation (5.1). The kriging predictor is directly read from Equation (5.5):

x 1:N T = E(x 1:N T | y 1:N T ) = σ -2 0 (Q +σ -2 0 A A ) -1 A(y 1:N T -η b).
(5.11)

The computation of (5.11) requires the resolution of a system with matrix (Q +σ -2 0 A A ), as detailed in Section 5.1. The conditional variance, also called kriging variance, is

Var(x 1:N T |y 1:N T ) = (Q +σ -2 0 A A ) -1 .
The computation of the diagonal of an inverse matrix is not straightforward when only the Cholesky decomposition of the matrix is available. Among the existing methods there is the Takahashi recursive algorithm described in [START_REF] Takahashi | Formation of a sparse bus impedance matrix and its application to short circuit study[END_REF] and [START_REF] Erisman | On computing certain elements of the inverse of a sparse matrix[END_REF]. Another way of computing the kriging variance is through conditional simulations, as detailed in Section 5.4.

In the temporal extrapolation setting, the vector x (N T +1) is predicted at time step (N T + 1) on T using all the data available until time T , i.e. from y 1:N T . Following Equation (3.20), we have

x (N T +1) = D x (N T ) + E z (N T +1) ,
(5.12)

where z (N T +1) is a standardized Gaussian vector. The kriging predictor x (N T +1) is

x (N T +1) = E(x (N T +1) | y 1:N T ) = D E(x (N T ) | y 1:N T ) = D x (N T ) , (5.13)
where x (N T ) is extracted from x 1:N T . The same procedure can be iterated to predict x at further time steps.

Conditional simulations

To perform a conditional simulation, we use the conditional kriging paradigm presented below. This approach relies on the fact that kriging predictors and kriging residuals are uncorrelated (independent under Gaussian assumption, see Chilès and Delfiner (1999, Chapter 7)). First, a non-conditional simulation x

(N C)
1:N T is performed on the spatio-temporal grid T . From this simulation, kriging residuals

r 1:N T = E x 1:N T | A x (N C) 1:N T -x (N C) 1:N T
are computed over the entire spatio-temporal grid T . The conditional expectation is computed using the method presented in the previous section. In a second step, these independently generated residuals are added to the usual kriging of the data to get the conditional simulation

x (C) 1:N T = x * 1:N T + r 1:N T .
Conditional simulations at further time steps are obtained by iteratively computing x (C) N T +k using the propagation equation (5.12) with k ≥ 1. Multiple independent realizations of conditional simulations can then be used to compute approximate conditional variances or other quantities, such as probability maps of threshold exceedance.

Simulation study

Several tests will be carried out to validate the estimation technique with both methods (Cholesky and matrix-free).

We report here some results regarding the estimation of the parameters θ = [κ, γ 1 , γ 2 , c, τ ] for a spatio-temporal model simulated with the SPDE (2.27). We set Since the sizes of both the dataset and the spatio-temporal mesh are reasonable, we report the estimations computed with both the Cholesky decomposition approach and the matrix-free approach.

H = I, α =
As initial values, we use estimated values obtained from the variograms of the spatial and temporal traces of the process. Specifically, the initial value for κ is the estimated scale parameter of a Matérn covariance function with smoothness parameter ν = α + α S -1 = 2 considering independent temporal repetitions, the initial value for c is deduced from the estimated parameter of n S independent repetitions of AR(1) processes of length N T and τ 2 is computed from Equation (2.29) with σ 2 being the empirical variance computed on the data. Finally, the initial value for γ is the null vector. The parameters for the matrix-free approach are set to the following: the order of the Chebychev polynomial to approximate the logarithm is set to 30 and the number of terms in the sum of the Hutchinson's estimator is set to 10. The results are reported in Table 5.1. They show that all parameters are accurately estimated with both the approaches. In almost all cases, the true value of the parameter is within the mean ± 2 standard deviations interval. We remark how the matrix-free approach takes more time to estimate the parameters. This is due to the iterative computations, that increase the computational time. However, we know that the benefit of the matrix-free approach is the possibility of applying it to much larger spatio-temporal meshes, where the Cholesky decomposition cannot be applied at all.

Spatio-temporal statistics for environmental data

Environmental data are usually characterized by spatial, temporal, and spatio-temporal correlations. Capturing these dependencies is extremely important to model the underlying phenomena precisely. We propose to apply the SPDE methodological framework developed in the previous chapters to predict a spatio-temporal field on a fine mesh using spatially irregularly spatial data measured at regular time steps. Our approach should be able to reduce the computational demand of geostatistical methods without giving up a precise description of the spatio-temporal dependencies.

We will consider two different datasets: first, a solar radiation dataset of few spatial data (99 stations) sparse over a pretty small region (160 km 2 ) measured every minute; then, a wind speed dataset of 53333 spatial data measured every hour coming from a numerical simulation over the entire territory of Saudi Arabia (2.15 million km 2 ). In both cases, the objective is the short-term forecast at 1,2 and 3 time steps ahead on a fine spatial grid.

Application to a solar radiation dataset

The approach detailed in the previous chapters is now applied to a solar radiation dataset for which experts agree on the presence of advection due to Western prevailing winds transporting clouds from one side of the domain to the other. The HOPE campaign [START_REF] Macke | The hd(cp) 2 observational prototype experiment (hope) -an overview[END_REF] recorded Global Horizontal Irradiance (GHI) (also called SSI, Surface Solar Irradiance) over a 10 × 16 km 2 region in West Germany near the city of Jülich from April 2 to July 2, 2013. The sensors were located at 99 stations located as pictured in Figure 6.1 and GHI was recorded every 15 seconds. A detailed description of the campaign can be found in [START_REF] Macke | The hd(cp) 2 observational prototype experiment (hope) -an overview[END_REF].

G G c K c . (6.1)
The error made using this approximation depends mostly on the solar zenith angle, the ground albedo and the cloud optical depth. In most cases, the maximum errors (95th percentile) on global and direct surface irradiances are less than 15 Wm -2 and less than 2 to 5 % in relative value, as recommended by the World Meteorological Organization for high-quality measurements of the solar irradiance [START_REF] Oumbe | Decoupling the effects of clear atmosphere and clouds to simplify calculations of the broadband solar irradiance at ground level[END_REF]. Practically, it means that a model for fast calculation of surface solar irradiance may be separated into two distinct and independent models: a deterministic model for G, under clear-sky conditions, as computed according to [START_REF] Gschwind | Improving the mcclear model estimating the downwelling solar radiation at ground level in cloud-free conditions -mcclear-v3[END_REF], considered as known in this study, and a model for K c , which accounts for cloud influence on the downwelling radiation and is expected to change in time and space.

K c is modeled as a random spatio-temporal process and will be the subject of our analysis. Figure 6.2 (right panel) shows the variable K c corresponding to the variable G shown on the left panel. In general, K c lies between 0 and 1, but in rare occasions, values above 1 can be observed. This phenomenon is called overshooting [START_REF] Schade | Enhanced solar global irradiance during cloudy sky conditions[END_REF] and is due to light reflection by surrounding clouds.

A time window of 20 minutes around 4 p.m. on May 28, 2013 is extracted with observations every minute at the 73 stations with well recorded values. Parameters are estimated on this 20-minute window using the method described in Section 5.1.

The spatio-temporal grid contains N T = 20 one-minute time steps, from t = 1 to t = 20 and N S = 900 regular spatial mesh points.

Estimation and prediction

Six different models are fitted to the data and used for prediction: 3 models with advection (called "adv-diff") and 3 models without advection (called "diff") obtained by setting γ = 0. Both groups contain the three following sub-models: (i) a model The log-likelihoods of the models that include advection are within a range of variations of 10 log-likelihood units and are between 34 and 80 units larger than those with diffusion only. As a point of comparison, if all spatio-temporal dependencies were ignored, the BIC penalization for the advection parameters would be 2 ln(1460) 14.5. These results indicate strong evidence in favor of models with advection, but no significant differences among them. The parameters vary substantially from one model to the other, but it must be remembered that, when considered independently, their physical interpretation is model dependent. Some combinations are interpretable however. For example, following Proposition 2.1, the overall variance is equal to

(8π) -1 τ 2 κ -2 (or (8π) -1 τ 2 κ -2 |I +h γ -1 γ γ | -1/2 in the stabilized case) when α tot = α + α S = 2 and it is equal to (16π) -1 τ 2 κ -4 (or (16π) -1 τ 2 κ -4 |I +h γ -1 γ γ | -1/2 )
when α tot = 3. Accordingly, the estimated standard deviations for models (1), ( 3), ( 4) and ( 6) are equal to 0.160, 0.119, 0.174 and 0.199 respectively, with the experimental standard deviation being equal to 0.184. The advection-diffusion models (1) and ( 3) are advection-dominated, hence the formula for the variance is the stabilized one. For the same models, the practical ranges computed as √ 8ν/κ [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] We then perform prediction with two different validation settings containing 80% of conditioning data and 20% of validation data. In the first case (called "Uniform") the validation locations are uniformly randomly selected. In the second case (called "South-East") the validation locations are located downwind (i.e. South-East) with respect to the estimated advection direction. See Figure 6.4 for a representation of the validation settings.

Recall that a time window containing 20 time-steps, from T = 1 to T = 20, (RMSE) validation score. First, the kriging is performed spatially only (hereafter referred to as "S" kriging). Second, a temporal extrapolation is computed at the conditioning locations at time horizons (T + 1), (T + 2) and (T + 3) ("T1", "T2", "T3" kriging). Third, the spatio-temporal prediction is computed at the validation locations at time horizons (T +1), (T +2) and (T +3) ("ST1", "ST2", "ST3" kriging).

We thus have a total of 6 models × 2 validation settings × 3 prediction configurations.

RMSEs are averaged over the 10 repetitions. Results are shown in Table 6.2.

For all tested validation settings and prediction configurations, the models with advection show better RMSE scores than models without advection. This result is a confirmation of the results already observed on log-likelihoods. Models with advection have similar prediction scores in the prediction configurations S and T, model (1) having slightly better performances in the configuration S. In the T and ST configurations, models (2) and ( 3) have in general quite similar RMSEs, except in the South-East setting with ST configuration where model ( 2) is clearly the best model. In this case, prediction is made in a space-time domain lying downstream with respect to the advection. It is thus expected that the model best representing the underlying physics should lead to the best prediction performances.

An example of prediction maps on T at time horizons (T + 1), (T + 2) and (T + 3) is reported in Figure 6.5, along with the observed values (black contoured dots). The white contoured dots are the locations used for validation in the Uniform setting. of the conditional simulations lie within the envelopes in both cases and at all time horizons. The remarkable result is that the variance of the conditional simulations at the green station is smaller than that at the orange one at every time step, especially when the time horizon increases. This is due to the advection term in model ( 3), able to propagate information from North-West to South-East.

Conditional simulations

context of the increase of renewable energies in the energy mix. The evolution of solar radiation during a short period (from one minute to one hour) is essentially generated by the movement of clouds on the considered spatial domain. This phenomenon is governed by advection and diffusion processes. We therefore took this phenomenon into account by our statistical approach based on physical SPDEs.

On the HOPE dataset, we have shown that the advection-diffusion model developed in the PhD presented better prediction performances compared to simpler models, while remaining computationally fast. An in-depth analysis of the HOPE solar radiation dataset, along with the OIE research team of Mines Paris, must be undertaken. We conceive possible transformations of the dataset, which is not perfectly Gaussian, or modifications of our current approach, in order to adapt the spatio-temporal SPDE approach to this specific dataset. This work will eventually be left to a new PhD project.

Application to a wind speed dataset

The SPDE approach is now applied to a wind speed dataset, already analyzed by Here, we will apply the SPDE approach on a regular triangulation of N S = 40 2 = 1600 spatial knots by using the entire set of observations as known data. These are n S = 53333 values. The predictions will be "short-term", at 3 steps ahead, i.e., 3 hours ahead. We do not pretend to obtain better performances than the ESN proposed by [START_REF] Huang | Forecasting high-frequency spatio-temporal wind power with dimensionally reduced echo state networks[END_REF], since [START_REF] Huang | Forecasting high-frequency spatio-temporal wind power with dimensionally reduced echo state networks[END_REF] already proved that their deep learning approach is capable of capturing the nonlinear temporal dynamics of the dataset.

We only want to illustrate how the spatio-temporal advection-diffusion SPDE model works on an environmental dataset to point out the advantages and disadvantages of the methodology. For this reason, and also for computational reasons, we will restrict the temporal domain to a day, i.e., 24 temporal measurements (in particular, January 2, 2016).

We refer to Huang et al. (2022, Section 3) for the preprocessing of the dataset.

The dynamics of the square root of the wind speed is described in his deterministic part by a harmonics depending on the spatial locations and in his random part by the multiplication of a spatially varying scaling coefficient by a residual spatio-temporal random field Y (s, t), which has unit variance at each location. The square root applied to the wind speed transforms the right-skewed distribution of the wind speed to normality (Gneiting, 2002). In this work we treat the spatio-temporal random field Y (s, t) as defined in Equation (1.1) and apply the estimation and prediction approach developed in Chapter 5.

In Figure 6.7 (left panel) we plot the entire set of observations over the spatial domain, where we pick 6 locations in different areas of the domain to assess the behavior of the wind. The right panel shows the topography of Saudi Arabia. Figure 6.8 shows the wind residual at the 6 picked locations; we remark that the evolution over 24 hours changes depending on the spatial location, with some similar behaviors for stations not far from the west border of the country, i.e., the Red Sea coast (stations 0 (blue), 5000 (green) and 15650 (red)). Figure 6.9 shows the evolution of the wind residual over 24 hours during 4 following days for each of the 6 stations. It is quite hard to define if some behaviors occur repetitively at each station. It seems that station 15650 and station 35000 show a daily quasiperiodic behavior, with low and high values of wind residuals approximately at the same hours of the day every day. The other stations are much harder to analyze. It can be noted that when the wind residual is at its highest in the south of Saudi Arabia (station 5000 on day 3), it is not the case in the north of the country (station 50000 on day 3) and vice-versa.

Estimation and prediction

The parameters of the statistical model, including the parameters of the SPDE (2.27), are estimated at each time steps from the known values at the spatial observations This method results in a nonstationary spatio-temporal field, which is a good asset with respect to the stationary SPDE model. Table 6.3 reports the MSE of the different models. The results show a better performance of all the SPDE models with respect to the persistence prediction for 2 and 3 hours ahead, and similar performances to FRK. The best SPDE model are models (2) and (3), i.e., the advection-diffusion models that result in nonseparable spatio-temporal fields.

Figure 6.10 shows the forecasted wind residuals at (T + 1), (T + 2) and (T + 3)

with model ( 3) and Figure 6.11 shows the difference between the MSE of model [START_REF] Predictions Of | to January 31, 2003 following the Spatial Interpolation setting. Predictions (continuous line), real values (dots) and envelope of ±2 conditional standard deviations[END_REF] and the MSE of the persistence model. The main remark is that model ( 3) is not good at predicting the value of the wind residual in a region in the South-West part of the country. This area is characterized by high mountains (see right panel of Figure 6.7); this topographic feature results in a stronger wind with respect to other regions, but the SPDE model, which is stationary, cannot really capture this behavior.

A nonstationary model would certainly have better predictive performances in this specific area. We refer to Chapter 7 for some first results about the nonstationary extension of the spatio-temporal SPDE approach.

We know that the SPDE model performance depends on the number of mesh points used in the triangulation, and on their position. We thus perform a test with a higher number of mesh points (N S = 50 2 and N S = 60 2 ), always on a regular grid. We remark in Table 6.4 that the MSE decreases with the increase of N S and we presume that the function is decreasing with the number of mesh points. Since we know that the topographic features influence the spatial dependence structure, we decide to test a custom-made mesh, made of N S = 3273, where the majority of the points are located on the mountain areas of the map (see Figure 6.13). The results of the estimation with this irregular mesh gives the best predictive results for (T + 1) and (T + 2). This improvement can be observed especially in the mountain areas by looking at the map of the difference between the MSEs of the SPDE model and the persistence model in Figure 6.12. The yellow areas clearly visible in Figure 6.11 are now much less evident.

Model α α S T+1 T+2 T+3

(

Concluding remarks

The amount of environmental data has recently dramatically increased, making classical geostatistical approaches to fail in most cases (see the "big n problem", Section 1.4).

On the one hand, the generality of the SPDE approach we propose, and its adaptability to large datasets, favors its application to a wide class of environmental and geoscience data (wind, fine particle concentration, water resources, etc.). We pointed out the advantages of this method in terms of estimation and predictive performances on two different environmental datasets.

On the other hand, there have been several attempts to apply deep learning techniques to environmental problems. In the specific case of wind forecasting (Section 6.3), the predictive approach based on echo state networks proposed by Huang et al.

(2022) seems to obtain better forecast performances. This is due especially to the fact that the complexity of the dataset resides mainly in the temporal dynamics, that deep neural networks usually capture well, and in the nonstationary spatial dependence structure, that the current advection-diffusion SPDE model cannot reproduce.

When using deep learning techniques, researchers have generally transposed methods from computer vision to environmental raster data, coming from satellites for earth observation or from climate models outputs. Nevertheless, especially at the local scales, it is preferable to work with direct ground measurements, i.e., on a set of irregular points in space. Indeed, ground data are more precise than satellite data and climate models output, generally with a higher temporal sampling frequency of the phenomena under study and without missing data. The direct adaptation of models coming from traditional deep learning is more complex in the case of spatio-temporal data collected over sparsely distributed measurement stations, as it is the case of the solar radiation dataset analyzed in Section 6.2.

The developments of the nonstationary SPDE approach (see Chapter 7) and of computationally faster techniques for estimation and prediction will probably lead to the possibility of applying the SPDE approach to more complex scenarios, both on the already studied datasets and on new datasets. For example, the comparison between the methods applied to the wind speed dataset would be more interesting when the nonstationary behavior will be captured also by the SPDE approach. Indeed, the SPDE approach should improve its predictive performances, especially in the areas with complex topographic characteristics. This extension will be left for future work.

Chapter 7 We here sketch a possible generalization of the unsteady advection-diffusion SPDE model to nonstationary spatio-temporal fields. We first review the most used techniques to tackle nonstationarities in the spatial statistics community and then propose a way of extending our methodological framework for simulation, estimation and prediction of SPDEs with spatially and/or temporally varying parameters. This work is still ongoing.

Generalization to nonstationary fields

Résumé

Nonstationarity: state of the art

In the nonstationary case, the covariance function can no longer be expressed as a simple function of the distance between the points as in Equation (1.2). Instead, it has an expression that depends on both the location and the relative position of the considered pair of points. The first challenge that appears is to determine the expression of this covariance function from the observed data. One of the ways to tackle this problem is by imposing that the random field can be modeled in a way that allows incorporating the prior structural information about the field. In general, the different methods try to derive an expression of the covariance function for any pairs of points in the domain. A detailed review of the methods used to model nonstationary random fields is available in [START_REF] Fouedjio | Second-order non-stationary modeling approaches for univariate geostatistical data[END_REF]. We here sketch the more popular approaches in the case of a spatial field.

Space deformation

An early approach is the deformation method [START_REF] Sampson | Nonparametric estimation of nonstationary spatial covariance structure[END_REF]. A nonstationary field observed on a spatial domain can be turned into a stationary one after applying a (nonlinear) deformation to the domain. The problem is thus reformulated in a stationary framework after deformation. The covariance function of the nonstationary process is written as the composition of a stationary covariance model with a deformation function. This approach has been further developed in Schmidt (2015). The characterization of the deformation function can be made through a set of parametrized functions which are fitted to the data by minimizing an objective function [START_REF] Perrin | Reducing non-stationary random fields to stationarity and isotropy using a space deformation[END_REF]. These approaches are computationally expensive, which limit their applicability for large-scale datasets. Moreover, they do not allow to easily take into account prior structural information about the nonstationarity. Indeed, they all seek to directly characterize the overall deformation while only considering the location and the value of the data points.

Zammit-Mangion et al. ( 2022) tackled the computational problem, by expressing spatial processes with nonstationary and anisotropic covariance structure as ones that have stationary and isotropic covariance structure on a warped spatial domain as in [START_REF] Sampson | Nonparametric estimation of nonstationary spatial covariance structure[END_REF], but modeling the injective warping function through a composition of multiple elemental injective functions in a deep-learning framework.

They employed approximate Bayesian methods to make inference with these models using recent methodological and technological advances in deep learning and deep Gaussian processes.

Convolution model

Another approach is the kernel convolution method [START_REF] Higdon | Non-stationary spatial modeling[END_REF][START_REF] Paciorek | Spatial modelling using a new class of nonstationary covariance functions[END_REF][START_REF] Kleiber | Nonstationary modeling for multivariate spatial processes[END_REF][START_REF] Fouedjio | A generalized convolution model and estimation for non-stationary random functions[END_REF]. The idea behind the method is that a nonstationary field at a given point of the spatial domain can be seen as the result of the spatial convolution over the domain of a deterministic function, called kernel function, with a white noise. The kernel functions defining the process covariance can easily be made spatially varying; they control the anisotropic behavior of the process in a small neighborhood of each point. With this method, even the smoothness of the spatial field can vary across space, which is not the case for the space deformation approach. The downside of this framework is that only the information of the anisotropy at each point is taken into account in their covariance, and not the overall structure of the anisotropy, which in practice might influence the covariance. [START_REF] Bornn | Modeling nonstationary processes through dimension expansion[END_REF] proposed a different idea to model nonstationarity which assumed that the nonstationary process is a projection of a stationary process in higher dimensions. They justified their dimension augmentation procedure by stating that a realization of a nonstationary Gaussian process in R d may be interpreted as a realization of a stationary field in R d+p for p > 0 under appropriate moment constraints.

Dimension expansion

Computational complexity

All the methods mentioned above involve the covariances between any pair of locations. This means standard implementations are infeasible for large datasets. In general, increasing the degree of flexibility in the covariance structure requires increasing the number of parameters. The common isotropic Matérn covariance functions are parametrized through 3 parameters: marginal variance, range, and smoothness.

Flexible models can have dozens or even hundreds of parameters. This implies that the estimation of these parameters can be hard and computationally cumbersome.

There exist some ways to overcome these computational issues [START_REF] Heaton | A Case Study Competition Among Methods for Analyzing Large Spatial Data[END_REF]).

An appealing way to reduce dimensionality is to describe the covariance structure through covariates [START_REF] Ingebrigtsen | Spatial models with explanatory variables in the dependence structure[END_REF][START_REF] Schmidt | Considering covariates in the covariance structure of spatial processes[END_REF][START_REF] Risser | Local likelihood estimation for covariance functions with spatially-varying parameters: The convospat package for r[END_REF].

Spatio-temporal nonstationarity

Compared to the vast literature for modeling nonstationarity in space, nonstationarity in both space and time is studied much less extensively. [START_REF] Ma | Spatio-temporal covariance functions generated by mixtures[END_REF] derived nonstationary space-time covariance functions by applying a spatially and temporally varying kernel to stationary covariance functions. [START_REF] Garg | Learning non-stationary space-time models for environmental monitoring[END_REF] extended the idea of convolution processes by [START_REF] Higdon | Non-stationary spatial modeling[END_REF] and [START_REF] Paciorek | Spatial modelling using a new class of nonstationary covariance functions[END_REF] to model a nonstationary spatio-temporal Gaussian process. Set in a Bayesian framework, [START_REF] Sigrist | A dynamic nonstationary spatio-temporal model for short term prediction of precipitation[END_REF] proposed a dynamic nonstationary spatio-temporal model for short term precipitation by linking the advection parameter, describing the horizontal transport of rainfall, of the convolution kernel to an external wind vector, measured independently of precipitation. [START_REF] Huang | Modeling transport effects on ground-level ozone using a non-stationary space-time model[END_REF] extended [START_REF] Wikle | A dimension-reduced approach to space-time kalman filtering[END_REF] to develop a space-time Kalman filter where the spatio-temporal covariance function depends on covariates. [START_REF] Shand | Modeling nonstationarity in space and time[END_REF] proposed modeling nonstationarity in both space and time for a spatio-temporal process by applying the dimension expansion technique described in [START_REF] Bornn | Modeling nonstationary processes through dimension expansion[END_REF]. The computational cost of all these methods remains the main problem.

Nonstationarity in the unsteady advection-diffusion SPDE

One of the main advantages of the SPDE approach is that it directly gives rise to computationally efficient models and easily enables Gaussian random field models with spatially and/or temporal varying parameters, by offering a framework where dependence structures can be motivated from physical properties of the spatial process. [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] noted that the deformation method for nonstationary models [START_REF] Sampson | Nonparametric estimation of nonstationary spatial covariance structure[END_REF]) can be reformulated as a nonstationary SPDE in the special case of deformations within a plane. However, for deformations into a higher dimensional space the two approaches differ considerably. Since the nonstationary parameters control the local anisotropy, they must be defined on the discretization mesh. Pereira (2019) defined generalized Gaussian fields on Riemannian manifolds that ally the ease of taking into account fields of local anisotropies (as the convolution model by [START_REF] Paciorek | Spatial modelling using a new class of nonstationary covariance functions[END_REF]) to the definition of covariance functions that assimilate them as a whole (as the space deformation model by [START_REF] Sampson | Nonparametric estimation of nonstationary spatial covariance structure[END_REF]). Their approach allowed to easily take into account local anisotropies in a global model of covariance. However, they lost the closed-form expression of the covariance model, which can only be computed numerically. [START_REF] Hu | A stochastic locally diffusive model with neural network-based deformations for global sea surface temperature[END_REF] introduced spatial nonstationarity as a SPDE governed by a spatially varying metric tensor that is a combination of general functions and spatial covariates though a function which is either linear or nonlinear (neural network).

Compared to a model with no covariate [START_REF] Fuglstad | Compression of climate simulations with a nonstationary global spatiotemporal SPDE model[END_REF], the function allows to incorporate external information about the local variation not captured by the large scale general basis functions.

Nonstationary fields can thus be defined by letting the parameters (κ 2 (s, t), τ (s, t), c(s, t) and γ(s, t)) in the unsteady advection-diffusion SPDE (2.27) be space-time dependent. This generalization implies only minimal changes to the method used in the stationary case concerning the simulation, but needs more work when it must be applied to inference and prediction. We can also incorporate models of spatially varying anisotropy by replacing the Laplacian operator ∇•H ∇ with the more general operator ∇ • H(s, t)∇, where H depends on space and/or time.

The introduction of those nonstationarities in the simulation, inference and prediction approaches is the next big target of this work. This generalization could be useful for those phenomena where anisotropies are clearly present, for example for wind velocity fields. We here consider only the spatial nonstationarity. Indeed, we consider that the temporal nonstationarity can be removed by dividing the temporal domain in short time windows, where all the parameters of the SPDE can be considered temporally invariant, leading to stationarity in time on the temporal subdomain.

Simulation

Including spatially varying parameters into the non-conditional simulation approach detailed in Chapter 4 is quite straightforward. Since the simulation relies on the FEM/FDM discretization of the spatio-temporal SPDE, if we consider that the pa-

Estimation

Introducing nonstationarity in the estimation technique needs to take into account the more complex structures derived by the space-time varying parameters. Different possible ways of including nonstationarities into the estimation technique can be envisaged, such as a parameterization of the parameters based on global basis functions and on external covariates or a spatio-temporal moving neighborhood integrated into an interpolation approach.

We here consider that the only spatially varying parameter in the advectiondiffusion SPDE (2.27) is the advection vector γ. This choice is made based on two considerations: first, because the spatially varying diffusion has already been considered in several papers [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF][START_REF] Fuglstad | Does non-stationary spatial data always require non-stationary random fields?[END_REF][START_REF] Berild | Spatially varying anisotropy for gaussian random fields in three-dimensional space[END_REF]; second, because we think that the advection field in environmental datasets could be the main source of a nonstationary behavior. The complex behaviors of the advection field should be taken into account in the statistical model to obtain better predictive performances. As a first illustrative example, we assume that the advection field is a rotation around a point on the spatial domain and parameterize the 2-dimension advection However, it is not always possible to know a priori the structure of the nonstationarity, nor to parameterize it by a few parameters as in this case. If we do not have any information about the form of the nonstationarity, we can parameterize the space dependence of the velocity vector γ through a finite number of parameters by using basis functions. Let g : R 2 → R denote a generic function that we want to expand in a basis, and let p > 0 be the number of basis functions. We use basis splines similarly to [START_REF] Fuglstad | Does non-stationary spatial data always require non-stationary random fields?[END_REF] and set

vector as γ(s) = [-v 1 (s 2 -x 1 ), v 2 (s 1 -x 2 )] , where v 1 , v 2 , x 1 ,
g(s) = f (s) α g ,
where α g ∈ R p , and f (s) = (f 1 (s), . . . , f p (s)) is a p-dimensional vector with the basis functions evaluated at location s. If the domain D is rectangular, we can construct a basis as a tensor product of two one-dimensional B-splines. This means that p = m 2 , where m > 0 is the number of basis functions used in each dimension. We use clamped splines where the derivative is 0 at each boundary.

Let B x,i denote the i-th basis function of the second-order basis in the x-dimension, and similarly B y,j for the y-dimension. The resulting two-dimensional basis is then

f ij (s) = B x,i (s 1 )B y,j (s 2 ), s = (s 1 , s 2 ) ∈ D,
for all combinations i, j ∈ {1, . . . , m}. This means that α g ∈ R m 2 , and m 2 parameters must be estimated for each of the arguments of the velocity vector γ.

We use m = 3, hence p = m 2 = 9, for a total of 18 parameters for the two functions

Concluding remarks

Letting the spatial and spatio-temporal models be more flexible through nonstationary covariance functions is vital to be able to describe complex dependence structures.

This flexibility can be obtained in the SPDE approach by making the parameters of the equation to be spatially and/or temporally varying. This extension is easily introduced in the simulation of SPDE models, but needs more work when it comes to estimation of the parameters. We have proposed two different ways to estimate a spatially varying advection vector: first, a parameterization of the vector based on prior information about the structure of the nonstationarity, then a global parameterization based on a basis made of clamped B-splines over the two-dimensional domain.

In both case, the spatial field of the advection vector is well captured by the estimation technique. However, both parameterization are based on a small number of parameters (4 and 18 respectively). The implementation details of the nonstationary extension are available in Appendix A.1.

As the number of parameters to be estimated increases with the complexity of the spatial nonstationarity, the likelihood maximization becomes harder. Moreover, in our specific case, the gradients of the log-likelihood are obtained by finite differences.

This computation is cumbersome for numerous parameters. A solution would be to compute the analytical gradients of the log-likelihood, as sketched briefly in Appendix A.2.

Another possible approach to deal with spatially and/or temporally varying parameters is to use neural networks for parameter estimation. This kind of estimation approach has already been proposed by [START_REF] Zammit-Mangion | Deep compositional spatial models[END_REF].

More work is needed to obtain proper results about the estimation of nonstationary fields in the SPDE approach, and to determine if the extension to nonstationary fields is interesting for prediction performances on environmental datasets as the ones we have considered in this thesis. However, the promising results we have shown tell us that we are going in the right direction.

Chapter 8 In this chapter, we present two other contributions made as a PhD student. First, we present the fully nonseparable generalization of the Gneiting class of multivariate space-time covariance functions [START_REF] Allard | Fully nonseparable gneiting covariance functions for multivariate space-time data[END_REF]. We discuss the fitting and predictive scores of the proposed model on a weather trivariate dataset, by comparing it to a more parsimonious model. The main potential of the proposed approach is the possibility to obtain entirely nonseparable models in a multivariate setting. Then, we present some methods for approximate estimation and prediction for spatial and spatio-temporal data, motivated by the objective of reaching a trade-off between statistical efficiency and computational complexity for large spatial and spatio-temporal datasets. Some of these methods proved to be effective for parameter estimation and prediction in the context of the "Spatial Statistics Competition for Large Datasets" organized by the King Abdullah University of Science and Technology (KAUST) in 2021 and 2022.

Other contributions

Résumé

Fully nonseparable Gneiting covariance functions for multivariate space-time data

In this section, we report the main points of the paper [START_REF] Allard | Fully nonseparable gneiting covariance functions for multivariate space-time data[END_REF]

Introduction and background material

An increasing amount of multivariate data indexed by Euclidean space-time coordinates are available in numerous scientific and engineering applications, including atmospheric and environmental sciences, geosciences, as well as geological and mining engineering. For statisticians analyzing these data, one of the key issues is to model the space-time dependence structure not only for each variable separately, but also across the variables. This requires versatile models that can account for different scale and smoothness parameters for each variable, and yet whose parameters can be accurately estimated. Bourotte et al. (2016) proposed valid multivariate space-time classes based on Matérn and Cauchy spatial covariance functions, inspired by the multivariate Matérn model in Gneiting et al. (2010) and Apanasovich et al. (2012).

For both classes, each variable has its own scale and own degree of smoothness in space, while allowing for some degree of cross-correlation. However, in this construction, the marginal temporal correlation function is identical for all variables, which, as discussed in Bourotte et al. (2016), was already seen as a restrictive assumption because it was found that the time correlations of the three variables in the analyzed dataset were different from one variable to the other. To account for different time correlations, space-time linear models of coregionalizations can be proposed, as in Rouhani andWackernagel (1990), De Iaco et al. (2003), Choi et al. (2009) or Finazzi et al. (2013). However, there are two major drawbacks to this construction. First, the smoothness of any component of the multivariate field is restricted to that of the roughest underlying univariate field (Gneiting et al., 2010). Second, the number of parameters to be estimated increases quickly as the number of components of the model increases, thus raising issues of robustness and over-fitting. Gelfand et al. ( 2005) Apanasovich and Genton (2010), Rodrigues and Diggle (2010), Ip and Li (2016) and Gelfand (2021) introduced Bayesian dynamic, latent dimension, convolution and spectral approaches, respectively, to build valid matrix-valued space-time covariance functions with entries that can have different smoothness parameters. Recent comprehensive accounts of space-time univariate and multivariate covariance modeling can be found in Chen et al. (2021) and in Porcu et al. (2021).

We consider a multivariate space-time random field X(s, t) = [X i (s, t)] p i=1 defined in R d × R, where d is the space dimension and p the number of random field components, each being real-valued. Without loss of generality, we assume that all random field components are centered, i.e. E[X i (s, t)] = 0 ∀(s, t) ∈ R d × R, ∀i = 1, . . . , p. It will also be assumed that the multivariate random field X(s, t) is second-order stationary, so that its covariance functions exist and depend only on the space-time lag (h, u) ∈ R d × R [START_REF] Chilès | Geostatistics: Modeling Spatial Uncertainty[END_REF]:

C ij (h, u) = Cov(X i (s, t), X j (s + h, t + u)), ( 8
C(h, u) = [C ij (h, u)] p i,j=1 , is positive semidefinite in R d × R.
The separability and nonseparability concepts were defined in Section 1.2 for each C ij (h, u) such that i = j. The separability functions (see Equation (1.3)) are s ii (h, u), for all i = 1, . . . , p. A matrix-valued space-time covariance function is said to be a proportional model when it is obtained as the product of a (p, p) covariance matrix R and a univariate space-time correlation function ρ, i.e. Bourotte et al. (2016) proposed nonproportional matrix-valued space-time covariance functions in which the purely spatial part is a nonproportional model but the purely temporal part is a proportional model, i.e., the model is proportional-in-time but not in space.

C(h,

u) = R ρ(h, u), ∀(h, u) ∈ R d ×R. It is proportional-in-time if C T (u) = R ρ T (u) and proportional-in-space if C S (h) = R ρ S (h), ∀(h, u) ∈ R d × R.
Instead, we build valid and flexible parametric classes of matrix-valued space-time covariance functions such that both the spatial and temporal components are nonproportional. Such matrix-valued space-time covariances are referred to as being fully nonseparable. The proposed class of multivariate space-time covariance functions with Matérn spatial traces is of the form . . . , p, (8.2) where M(•; r, ν) denotes the univariate Matérn covariance function with scale and smoothness parameters r and ν, i.e., M(h; r, ν) = C M ν (r h ) , where C M is the unit variance and scale Matérn covariance function defined in (2.5). In order to build multivariate models that are neither proportional in space nor in time, an essential building block is to define admissible and relevant temporal matrix-valued covariances of the form [(γ ij (u) + 1) -d/2 ] p i,j=1 .

C ij (h, u) = σ ij (u)M(h; r ij (u), ν ij (u)), (h, u) ∈ R d × R, i, j = 1,

A general fully nonseparable Gneiting-Matérn class

Consider a p-variate temporal random field X(t) = [X i (t)] p i=1 , t ∈ R. Under the assumption that the direct and cross-increments are jointly stationary, the matrixvalued pseudo-variogram η = [η ij ] p i,j=1 is defined as (Papritz et al., 1993) (8.3) for all t, u ∈ R and any pair i, j = 1, . . . , p. Notice that the pseudo-variogram has nonnegative entries and is not necessarily an even function. For any i = 1, . . . , p, [START_REF] Chilès | Geostatistics: Modeling Spatial Uncertainty[END_REF]. Necessary and sufficient conditions for a matrix-valued function to be a pseudo-variogram are provided in Dörr and Schlather (2021). (8.4) where α and β are nonnegative.

η ij (u) = 1 2 Var X i (t + u) -X j (t) ,
η ii (direct variogram of X i ) is a conditionally negative semidefinite function (η ii c 0), i.e., such that p i=1 p j=1 ω i ω j η ij ≤ 0, ∀ ω 1 , . . . , ω p ∈ R such that p i=1 ω i = 0 (
(u) = [r ij (u)] p i,j=1 and σ(u) = [σ ij (u)] p i,j=1 , the matrix-valued function C : (h, u) → [C ij (h, u)] p i,j=1 with C ij (h, u) = σ ij (u) M(h; r ij (u), ν ij ), (h, u) ∈ R d × R, is positive semidefinite in R d × R for R(u) = αη(u) + a 2 βη(u) + b 2 and σ(u) = ρ(u)Γ(ν) exp(ν) (αη(u) + a 2 ) ν βη(u) + b 2 d/2 ,
: (h, u) → [C ij (h, u)] p i,j=1 with C ij (h, u) = τ ij (βη ij (u) + b 2 ij ) d/2 (αη ij (u) + a 2 ij ) ν ij M   h; αη ij (u) + a 2 ij βη ij (u) + b 2 ij , ν ij   , (8.5) is positive semidefinite in R d × R if the matrix τ e -ν /Γ(ν) is positive semidefinite.
We refer to [START_REF] Allard | Fully nonseparable gneiting covariance functions for multivariate space-time data[END_REF] for a proof and more mathematical details. The matrix-valued space-time covariance function (8.5) is of Gneiting-Matérn type. When α = 0, accounting for the fact that exp(-ν) is positive semidefinite, the models in (8.4) and ( 8.5) generalize the Gneiting-Matérn model (Bourotte et al., 2016)

C M ij (h, u) = σ i σ j ρ ij (γ(u) + 1) d/2 M h; r ij (γ(u) + 1) 1/2 , ν ij , (h, u) ∈ R d × R. (8.6)
If, furthermore, the direct variograms η ii are increasing functions for i = 1, . . . , p, the scale parameters r ii (u) = a ii (βη ii (u) + b 2 ii ) -1/2 decrease as u increases. The space-time direct correlations are thus higher than what would happen for a separable space-time covariance function, i.e., they are positively nonseparable. When β = 0 and under the same monotonicity assumption for the direct variograms, the scale parameters r ii (u) = αη ii (u) + a 2

ii /b ii increase with u. This a very original feature, since for all Gneiting-type spatio-temporal models of our knowledge, the scale parameter decreases with u. Applying the same reasoning as above, the resulting space-time direct covariances are thus negatively nonseparable. When α and β are both positive, the model is positively or negatively space-time nonseparable, depending on the relative values of the other parameters involved in R(u). However, in the common case where η ij (u) → ∞ as u → ∞, R(u) asymptotically tends to a matrix with all elements equal to α/β. The space-time covariance is thus asymptotically separable as |u| → ∞.

Application to a weather dataset

Data and parameterization

We now illustrate the model presented in Section 8.1.2 on a weather dataset consisting of three daily variables (solar radiation R, temperature T, and humidity H) recorded at 13 stations in Western France from 2003 to 2012. We shall compare a specific fully nonseparable multivariate model (FULL), as proposed in (8.5), to a Proportional-Intime (PIT) model as in (8.6). Both models will be applied to residuals, after centering in time and space, in order to filter out any seasonal and regional effect. For each of the three variables, the separability index S(h, u) = C(h, u)C(0, 0) -C S (h)C T (u) was computed. Empirical covariances were derived from the space-time empirical variograms computed with the function EVariogram from the package GeoModels (Padoan and Bevilacqua, 2015), using spatial bins of 50 m. Since the variograms have been computed on residuals, C(0, 0) = 1 and the separability index must be in the interval [-1, 1], with S(h, u) = 0 indicating separability of C(h, u). These analyses showed clearly that the residuals are uniformly positively space-time nonseparable for | h | ≤ 500 m and u ≤ 3, which is actually often the case for climate variables. The parameter α in (8.5) is thus set to α = 0. For identifiability reasons, we further set

β = 1 and b 2 ij = 1 -R 0 ij with max i {R ii (0)} < 1.
The pseudo-variogram η needs to be specified. For the unbounded part common to all components, we choose γ 0

(u) = (1 + |cu| 2a ) b -1, u ∈ R, with c > 0, 0 < a ≤ 1
and 0 ≤ b ≤ 1, for ease of comparison with Bourotte et al. (2016). This variogram is reminiscent of the generalized Cauchy covariance model that will also be used for R: R ij (u) = A i A j (1 + |ru| 2λ ) -b , u ∈ R, with r > 0, 0 < λ < 1 and 0 ≤ A i < 1 for i = 1, . . . , p, and where for the sake of parsimony the parameter b is set identical to the parameter b in γ 0 . In this parameterization, the space-time nonseparability parameter b acts both inside and outside the Matérn function M, implying a constant perfect correlation in time if b = 0. Following Gneiting (2002) and Bourotte et al. (2016), a reparameterization is thus useful. Multiplying the above equation by the

temporal covariance function ρij (u) = (1 + c|u| 2a ) δ -A i A j (1 + r|u| 2λ ) -δ -1
with 0 ≤ δ ≤ 1 leads to the parametric family that will be used in the rest of this section.

The parameterization is further simplified by setting

τ ii = σ 2 i a 2ν ii ii (1 -A 2 i ) 2 and, for i = j, τ ij = ρ ij √ τ ii τ jj , with [ρ ij ] i,j=1,.
..,p being a correlation matrix. With this parameterization, the matrix τ is positive semidefinite by construction. Finally, the model reads

C ij (h, u) = 1 (1 + |cu| 2a ) δ -A i A j (1 + |ru| 2λ ) -δ ρ ij a -2ν ij ij √ τ ii τ jj (1 + |cu| 2a ) b -A i A j (1 + |ru| 2λ ) -b × M h; a ij [(1 + |cu| 2a ) b -A i A j (1 + |ru| 2λ ) -b ] 1/2 , ν ij , (8.7) with (h, u) ∈ R d × R and i, j = 1, . . . , p.
The model in (8.7) provides interpretable parameters for a fully nonseparable Gneiting-Matérn model.

Estimation

Composite likelihood methods have proven efficient in the context of spatio-temporal data, with less time-consuming steps and good asymptotic properties (Bevilacqua et al., 2012). Pairwise Likelihood (PL) is the product of marginal Gaussian likelihoods computed on all pairs {Z i (s α , t α ), Z j (s

β , t β )} such that | s α -s β | ≤ d m and |t α - t β | ≤ t m
, where i, j = 1, . . . , p and α, β = 1, . . . , N . The data analyzed here are standardized residuals after centering for spatial and temporal trends. We thus set

σ i = 1, for i = 1, . . . , p.
There is a total of p(p + 2) + 6 parameters to estimate, where p is the number of variables. Here p = 3, leading to a total of 21 parameters.

PL was maximized sequentially in subspaces of the parameter space corresponding to blocks of related parameters, while keeping all other parameters fixed to the values previously attained. Finally b is the value of b among {0, 0.1, . . . , 0.9, 1} corresponding to the highest maximized PL. To perform the maximization in the subspaces of the parameter space, we used the package nlminb implemented in R with lower and upper bounds for the parameters when mathematically necessary.

All 13 stations were used to estimate the parameters of both the FULL and PIT models. For comparison purpose, PL was also maximized for the PIT model, i.e. when a = 0. For this simpler model, log PL PIT ( θb ) is monotonically decreasing from -933 961.3 to -934 072.6 as b varies from 0 to 1. FL was also computed for these two models (remember that FL is easy to compute but very long to optimize): log FL FULL ( θb=0.1 ) = -2 652.7 and log FL PIT ( θb=0.1 ) = -2 819.1, indicating strong evidence in favor of the more complex model, considering that the number of data 8.1. Fully nonseparable Gneiting covariance functions for multivariate ST data 101 is moderate with a total of 10 independent repetitions of 1209 correlated data. As a point of comparison, the (BIC) penalization to the full likelihood of the more complex model is equal to 2.5 ln(12090) 23.5.

Looking at the temporal marginals, the models show very different behaviorsas we expected. Figure 8.1(right) displays the temporal empirical covariances for variables R and H with their fitted models. Thanks to the flexibility of the FULL model, the temporal covariance functions fit the empirical values for both variables. In contrast, the unique covariance function of the PIT model lies somewhere between the empirical covariances of the two variables, thus being unable to provide a satisfactory fit to any of them. Table 8.1 reports the estimated parameters maximizing PL for both models. It is interesting to note that the parameter λ is equal 0 in the FULL model, thus implying a sort of temporal nugget effect, visible on the temporal covariances in Figure 8.1(right). Figure 8.2 shows the spatial (at time lag u = 0) and spatiotemporal covariances (at time lags u = 1, 2) for all variables and pairs of variables.

The empirical covariances between all pairs of stations and the FULL model with estimated parameters from Table 8.1 are shown. Overall, the fit is good. On the top left panel representing the covariance function for the variable R, the very high smoothness of the covariance function near the origin is clearly visible.

Prediction

We consider two different prediction settings. In the first setting, called Spatial Interpolation , prediction of the three variables at a validation station at day t is made knowing the trivariate data at the 11 estimation stations at day t, t -1 and t -2.

In the second setting, called Temporal Prediction, prediction is made with data from days t -1 and t -2 only at all 13 stations (estimation and validation stations). Other settings, such as Spatio-temporal prediction (using data at 11 stations from days t -1 and t -2 for predicting at the 2 validations stations at day t) is possible, but not shown here because they are less common in practical situations. Predictions are computed from January 3 to January 31. In the Gaussian framework, the conditional expectation is the best linear unbiased prediction, also called (co)kriging in the geostatistics literature, while the conditional covariance matrix is the covariance matrix of the cokriging errors. The conditional variances are identical for all days, since the configuration of the conditioning data remains identical. The two models (FULL and PIT) are compared by means of the Root Mean Square Error (RMSE), the Mean Absolute Error (MAE), the Continuous Ranked Probability Score (CRPS) and the Logarithmic Score (LogS). Table 8.2 reports our results. As a benchmark, we also report the scores obtained with a trivial prediction where all predicted values are set to the expectation (equal to 0) and all variances to the theoretical variance (equal to 1). deviations was added to the plot.

Concluding remarks

We have proposed new parametric classes of matrix-valued covariance functions for multivariate spatio-temporal random fields, where each component has its own smoothness and scale parameter in space and its own correlation function in time. Our constructions generalize the Gneiting class of space-time covariance functions and are fully nonseparable, in the sense that they are space-time nonseparable and that they are neither proportional-in-space nor proportional-in-time. This is a major improvement to most of the models proposed up to date (with the exceptions of the models in Dörr and Schlather (2021) and Porcu et al. (2022)), where the marginal temporal correlation function is identical for all the variables, which is overly restrictive. The main ingredient for achieving this has been the use of pseudo-variograms, as multivariate analogous of Bernstein functions as pointed out by Dörr and Schlather (2021) and Qadir and Sun (2022) in their discussion.

The main class proposed in Section 8.1.2 allows for a general behavior for the spatial scale parameter as a function of the time lag u, depending on the values taken by the parameters. It is well known that the Gneiting class of models is positively nonseparable. A very original feature of our construction is that the models can be negatively nonseparable or asymptotically separable as |u| → ∞. Since these models are continuous over R d × R, their application is not limited to data organized as regular records at measurement stations. More complex designs are in principle possible, such as irregular or incomplete records, heterotopic designs or even mobile measurement devices. We hope to see applications of our models to such datasets in a near future.

From a statistical point of view, the main challenge is to find the right balance between the flexibility of the model, which must be able to account for the complex interactions between space and time across several variables, and the number of parameters that should remain interpretable and not too numerous. More complex models should only be preferred if supported by the data. The dataset analyzed in this work is of moderate size with 10 independent repetitions of 13 × 31 = 403 spacetime trivariate data. Its analysis has shown that our model improves the fitting and the predictive performances over the more parsimonious model in Bourotte et al. (2016), in particular when the time dimension is important for prediction. In Section 8.1.3, it was shown that the maximum PL difference is around 600. Combined with the improved prediction scores in time, this provides evidence in favor of our new model for this dataset. The analysis of other datasets, in particular with more spatial locations, is necessary for confirming this first result.

Spatial and spatio-temporal statistics for large datasets

In this section we discuss a project of my PhD related to the approximate approaches for parameter estimation and prediction for spatial and spatio-temporal data. In The interest in approximate approaches is also motivated by the other research projects of the PhD, where those techniques were either part of the approach itself (the GMRF approximation of the random fields solution to the SPDE, see Chapter

3) or were found to be necessary to reduce the computational cost of the maximum likelihood (the pairwise likelihood used in Section 8.1).

State of the art

Most of the prediction approaches and estimation techniques for spatial and spatiotemporal data that have been proposed in the last decade are motivated by reaching a compromise between statistical efficiency and computational complexity (Porcu et al., 2021). This is due to the fact that massive (and/or multivariate) data sets are now available. The main estimation techniques for massive spatial datasets were gathered and reviewed by [START_REF] Sun | Geostatistics for large datasets[END_REF]. They include, among others, covariance matrix tapering, composite likelihoods, spectral techniques, and approximation of the random field with a GMRF. Estimation methods for space(-time) covariances for large data are conceptually similar to classical approaches, but they have to deal with intractable computation of likelihoods. It is statistically difficult to decide which method is the best, because approximating covariance functions with computationally tractable ones is a form of deliberate misspecification of the true underlying covariance structure. For example, in the covariance tapering framework [START_REF] Furrer | Covariance tapering for interpolation of large spatial datasets[END_REF][START_REF] Kaufman | Covariance tapering for likelihood-based estimation in large spatial data sets[END_REF], a direct product of the true covariance function and a compactly supported correlation function is used. Composite likelihood methods [START_REF] Varin | An overview of composite likelihood methods[END_REF] are built on the consideration of the likelihood function as a product of marginal or conditional densities. Low-rank approximation methods [START_REF] Banerjee | Gaussian predictive process models for large spatial data sets[END_REF][START_REF] Cressie | Fixed rank kriging for very large spatial data sets[END_REF] project the entire random process to a low-dimensional space and use the low-rank representation to approximate the original process. Spectral methods rely on truncation of the spectral expansion, hence on the loss of information on the geometric properties of the associated random field.

Finally, Markov random fields coupled with SPDEs [START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF][START_REF] Rue | Approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations[END_REF] approximate a continuous process with a process defined over a mesh for which the conditional distributions only depend on neighbors, leading to sparseness of the precision matrix.

The Spatio-Temporal Statistics and Data Science (STSDS) department of KAUST launched the first KAUST "Spatial Statistics Competition for Large Datasets" to the international spatial statistics community in 2021, followed by the second and third editions in 2022 and 2023. Several teams from all over the world participated in the competition, 21 in 2021 and 14 in 2022. The STSDS team has been exploiting the modern High-Performance Computing (HPC) systems to make large-scale exact computation of likelihoods possible and to allow processing larger data sizes more easily than before. They considered that the lack of large-scale exact computation has led to an inefficient assessment of spatial modeling approximation methods in the last decades. Indeed, on the one hand, every research group has used different datasets to assess the proposed methods; on the other hand, the few comparative studies have been limited to small and medium-sized datasets. Thus, they decided to assess the methods developed worldwide based on generated synthetic datasets and on the true and the exact estimated parameters, obtained with the ExaGeoStat software they have been developing [START_REF] Abdulah | Exageostat: A high performance unified software for geostatistics on manycore systems[END_REF]. This software can deal with millions of locations, by relying on parallel linear algebra libraries to allow fast and efficient computation of the log-likelihood function and by using modern runtime systems to make the code portable on the different parallel architectures, including

GPUs. ExaGeoStat has been coupled with a rich class of covariance models to obtain exact parameter estimation and predictions for large-scale spatial and spatio-temporal data, see e.g. [START_REF] Salvaña | Lagrangian Spatio-Temporal Nonstationary Covariance Functions[END_REF].

In the end, the goal of the competition was to achieve a comprehensive comparison between as many different methods as possible for parameter estimation and prediction on massive datasets.

2021 Edition

The 2021 edition of the "Spatial Statistics Competition for Large Datasets" featured 4 sub-competitions. The objective was to evaluate existing approximation methods on large spatial datasets. A collection of large synthetic datasets (100K/1M data) was generated from a set of real models using ExaGeoStat and each dataset was split into training and validation sets. The selected models covered disparate spatial properties to ensure a fair comparison between all competing methods. All the details on the datasets and on the scoring rules, and the main methods used by the participants are available in Huang et al. (2021). The RESSTE team was ranked first at 3 subcompetitions out of the 4 proposed.

In sub-competitions 1a and 1b, 16 datasets of 100K data each were generated from a Gaussian spatial field with a Matérn covariance function as defined in (8.2). The goal was to estimate the parameters of the covariance function from 90K training data and to make spatial predictions at 10K test locations. The four parameters determining the covariance structure are the partial sill σ 2 , the range 1/r > 0, the smoothness ν > 0, and the nugget τ 2 . Different smoothness scenarios were considered, from smooth to rough, and, by varying the range parameter, different resulting effective range were proposed, for weak, medium, and strong dependence. The 16 datasets thus covered a wide variety of covariance properties. The Mean Loss of Efficiency (MLOE)

and Mean Misspecification of the Mean Square Error (MMOM) were used to evaluate the performances of the parameter estimation in the sub-competition 1a. MLOE characterizes the average loss of prediction efficiency when the approximated model is used for predictions instead of the true model. MMOM characterizes the average misspecification of the mean square error when calculated under the approximated model. RMSE was used to evaluate the prediction accuracy in the sub-competition 1b.

As RESSTE team, we opted here for block approaches, which satisfy the following principles: (i) in each block, the estimation (by maximum likelihood) or the prediction (by kriging) is optimal; (ii) the blocks should be as large as possible while taking into account the computational problems; (iii) the blocks are assumed to be independent to each other. The approximation lies entirely in (iii), and it is easy to understand that (ii) is the key to success. We ranked first in the sub-competition 1b, related to spatial prediction, and third in the parameter estimation of sub-competition 1a. An approximation approach proposed by another team that gave good results in terms of parameter estimation and spatial prediction was the Vecchia approximations method detailed below. Generally, the parameter estimation was more difficult when the process was smoother (higher smoothness parameter) and had stronger dependence (higher effective range). Even if we did not find the best estimates of the parameters, however our prediction approach gave better results than the other teams (even those that ranked first and second in the sub-competition 1a) and than the exact kriging obtained with ExaGeoStat without any approximation. One possible reason is that the approximation of the other teams was inadequate in kriging, even though the underlying model they used was more accurate. This implies that both the model inference and the number of neighbors considered are important for local kriging predictions.

The datasets for sub-competitions 2a and 2b were univariate non-Gaussian spatial data on a unit square containing 100K and 1M observations respectively. Without giving the information about the generating process to the participants in the competition, the data were generated by the Tukey g-and-h random fields (Xu and Genton, 2017), which generalize Gaussian random fields to account for skewness and heavy tails. For a generated Gaussian field X(s), the Tukey g-and-h random process T (s) is defined by marginal transformation at each location s as

T (s) = ξ + ω e gX(s) -1 g e hX 2 (s) 2 ,
where ξ and ω are the location and scale parameters, respectively, g controls the skewness and h ≥ 0 determines the tail-heaviness. They chose two sets of values g and area to the other in the spatial domain. We therefore decided to split the spatial domain with different splitting configurations and fit an isotropic or anisotropic Matérn model for each considered region. The selection of the best splitting configuration and the best Matérn model was made by cross-validation. The option with the lowest mean square prediction error values was then finally chosen. The RESSTE team was ranked first in the sub-competition 1a and third in the sub-competition 1b. The team that obtained the best prediction accuracy in 1b adopted a deformation technique to map the locations from the original space to a latent space depending on two variables r = √ s x + s y and θ = tan -1 s y /s x , where the data were assumed to be stationary. The prediction was performed using a deep feed-forward neural network model with two inputs (r and θ), four hidden layers, and one output (the objective function,

i.e., the mean square predictive error). They used the Adam optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] for the optimization of the objective function.

The spatio-temporal datasets of the sub-competition 2a and 2b were generated with 1K and 10K locations, and 100 time-slots. The spatio-temporal models were nonseparable, with different spatial and temporal dependencies. The teams were asked to make predictions in three different scenarios of leaving out space-time points:

random spatial locations with all times left out (RS), random locations in space/time left out (RST) and all spatial locations left out on the last 10 time coordinates (T10).

We first performed an exploratory data analysis step and found that the data were generated from a positively non-separable space-time kernel. We then chose a covariance function from the Gneiting class, with a Matérn spatial covariance and a Cauchy temporal covariance. Due to the size of the given datasets, we used a block-composite likelihood to estimate the spatial and temporal parameters separately. Then, we used the estimated parameters to fit the space-time model and estimate the full set of parameters. To predict at the test locations we applied ordinary kriging conditioned on a subset of nearest data points in space and 3 preceding, current, and 3 following time-slots in time. We were ranked third in both the sub-competitions, but with a RMSE quite lower than the first one. Our results were good for the first two prediction scenarios (RS and RST), but not so good for the third scenario (T10). Indeed, the third scenario could be considered more like a multivariate time series forecast, where kriging is not necessarily the best prediction technique. The winning team, in fact, proposed a deep neural network for spatio-temporal predictions, by using basis functions to capture the spatio-temporal dependence. They used regression for the spatial interpolation step and they trained a long-short term memory network for the temporal forecast step. Their results for the first two predictive scenarios were worse than ours, but the LSTM made their temporal forecasts much more precise.

Concluding remarks

For decades, the "big n problem" in spatial data has been an active research area due to the challenges engendered by the real datasets, which often contain millions of observations, such as remote sensing data or numerical model outputs. Those datasets are sometimes multivariate and/or spatio-temporal and need efficient methods for their prediction, both in the spatial interpolation and in the temporal forecasting frameworks. The competitions proposed by KAUST in the last years are a great tool to provide a unified framework for evaluating the performances of existing approximation methods and for suggesting future research paths. We appreciated the fact that dozens of research teams participated worldwide, each one proposing different techniques. Some of them proved to be great in some contexts and worse in others, and all the pros and cons of each method are now comparable in terms of prediction accuracy on the same synthetic datasets.

The computational time was never compared in the proposed competitions, because the execution time of each participating team on different machines was not directly comparable. However, this matter should be taken into account to better evaluate a given approach. In fact, often researchers and companies do not own machines that are able to treat massive datasets, or they do not want to use excessively consuming techniques for sustainable reasons. Sometimes, as we have shown in the previous sections, approximate methods have similar performances to exact methods with a much lower computation cost, which is a great asset for a thoughtful research tool.

The spatial and spatio-temporal models proposed in the first two editions of the "Spatial Statistics Competition for Large Datasets" are only a small part of possible and more challenging models. Assessing the results of the approximate methods proposed worldwide on real environmental datasets, that show different behaviors and dependence structures, could be a great tool to choose the best approach for each peculiar type of data. Another option would be to choose a benchmark dataset that at least presents a variety of issues that arise in practice. Another important point to take into account is the experimental design: in both the competitions the data were point data. It could be interesting to evaluate if the predictive performances of the methods change in presence of gridded data or clustered data. The case of gap filling, i.e., large areas with missing data, should also be considered.

This year, KAUST hosts a third competition that focuses on the construction of confidence and prediction intervals, instead of point estimation and prediction. The datasets were generated from stationary Gaussian random fields with an isotropic Matérn covariance function, with designs of irregularly spaced locations. The team RESSTE does not participate in the 2023 competition, but the results will certainly be interesting to get useful insights on approximate methods on those datasets.

There is still room for improvement in spatial statistics to obtain more efficient approximated methods for large spatial and spatio-temporal datasets, and this is a significant goal for the research community.

Chapter 9

Discussion and perspectives

Résumé

Dans ce dernier chapitre, nous resumons les résultats de la thèse en détaillant les contributions principales. Ensuite, nous proposons des pistes de recherche futures liées aux méthodes presentées.

Discussion

This dissertation explored some approaches to deal with spatio-temporal data in environmental science. Here we make some concluding remarks on the proposed approaches and propose interesting perspectives for future work.

The spatio-temporal SPDE approach based on unsteady advection-diffusion equations that we have proposed in Chapter 2 combines elements of physics, numerical analysis and statistics. It can be seen as a first step toward physics-informed geostatistics, which introduces physical dynamics into a statistical model, accounting for possible hidden structures governing the evolution of the spatio-temporal phenomenon. The different terms of the SPDE (advection, diffusion) directly influence the spatio-temporal dependencies of the process, by controlling its variability in space and time. Compared to spatio-temporal models built on covariance functions such as the Gneiting class (Gneiting, 2002), we gain in interpretability since the parameters of the model can be linked to the physical coefficients of SPDEs.

In Chapter 3 we showed that it is possible to build an accurate space-time approximation of the process driven by the advection-diffusion SPDE using a combination of FEM in space and implicit Euler scheme in time. We dealt with the advectiondomination flow, by introducing a stabilization technique in the discretization approach. It leads to sparse structured linear systems. We obtained promising results

for the simulation (Chapter 4), the estimation and the prediction (Chapter 5) of processes both in terms of precision and speed. When the size of the spatio-temporal mesh is moderate, direct matrix implementation is possible. We showed how matrixfree methods can be implemented in order to obtain scalable computations even for very large meshes. The application to the solar radiation and to the wind speed datasets (Chapter 6) demonstrated that the nonseparable advection-diffusion model exhibited the best prediction performances on phenomenona that are governed by advection and diffusion processes.

The main results of these chapters are available in the arXiv paper [START_REF] Clarotto | The spde approach for spatio-temporal datasets with advection and diffusion[END_REF].

One of the main advantages of the SPDE formulation is that it is easy to generalize it to nonstationary settings. Nonstationary fields can be defined by letting the parameters of the advection-diffusion SPDE (κ(s, t), γ(s, t), etc.) be space-timedependent. This generalization implies only minimal changes to the method used in the stationary case concerning non-conditional simulations, but needs more work for estimation and prediction, since the maximum likelihood approach becomes much more expensive. We can also incorporate models of spatially varying anisotropy by modifying the general operator Further work is necessary to better assess the prediction accuracy and the computational complexity. Applications to larger and more complex datasets, in particular using the matrix-free approach, will be considered. Comparison to models expressing the advection in a Lagrangian framework [START_REF] Ailliot | Spacetime models for moving fields with an application to significant wave height fields[END_REF][START_REF] Benoit | Stochastic rainfall modeling at sub-kilometer scale[END_REF][START_REF] Salvaña | Lagrangian Spatio-Temporal Nonstationary Covariance Functions[END_REF] should also be performed. A frequentist maximum likelihood was implemented. As a follow-up work, it would be interesting to implement this space-time model as part of a Bayesian hierarchical construction, possibly within the INLA/SPDE framework [START_REF] Rue | Approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations[END_REF][START_REF] Krainski | Advanced spatial modeling with stochastic partial differential equations using R and INLA[END_REF].

The models and the methodological framework developed in this thesis were implemented in Python by making use of the FEniCS library [START_REF] Logg | Automated Solution of Differential Equations by the Finite Element Method[END_REF][START_REF] Alnaes | The FEniCS project version 1.5[END_REF] for the task of computing the FEM solution of (S)PDEs. The implementation details are available in Appendix A.1. Another perspective is the development of computationally efficient algorithms for inference and prediction. We envisage the possibility of introducing parallelization using MPI sub-communicators as a way of obtaining faster computations in maximum likelihood estimation and prediction. We showed in Section 8.2 that some research teams in the spatial statistics community are already dealing with parallel codes and using modern runtime systems to allow efficient computations for dense matrices [START_REF] Abdulah | Exageostat: A high performance unified software for geostatistics on manycore systems[END_REF]. Thus, we presume that extending these techniques to sparse matrices could be very useful for future applications, not only in the SPDE framework.

Another interesting consequence of defining the models through local stochastic partial differential equations is that the SPDEs still make sense when R d is replaced by a space that is only locally flat. We can define nonstationary Gaussian fields on manifolds, and still obtain a GMRF representation. Important improvements were obtained in the spatial case [START_REF] Pereira | Geostatistics for large datasets on riemannian manifolds: A matrix-free approach[END_REF]. The generalization to spacetime processes could be explored further. Possible generalization to spatio-temporal SPDEs with a fractional exponent in the diffusion term could also be considered. A development of the methods proposed by Bolin and[START_REF] Bolin | The rational SPDE approach for gaussian random fields with general smoothness[END_REF][START_REF] Vabishchevich | Numerically solving an equation for fractional powers of elliptic operators[END_REF] should be explored.

A further interesting direction is the extension of the approach to new SPDEs able to better describe some peculiar physical phenomena. The current approach allows finding a sparse structure of the precision matrix of a space-time field solution of any SPDE that can be written as a sum of differential operators. If we had any information about the physics of a given spatio-temporal phenomenon we want to model, we could replace the advection-diffusion SPDE by another SPDE closer to the true physical equation of the process. An example could be the modeling of the motion of sea waves by the wave equation, as recently proposed in [START_REF] Henderson | Characterization of the second order random fields subject to linear distributional pde constraints[END_REF]. We are increasingly interested in deep statistical learning for spatio-temporal environmental data. Although none of the works we have presented in this thesis make use of such methods, the interest of the scientific community in spatio-temporal statistics on this subject is growing and some approaches have already proven to be effective and efficient on large spatio-temporal datasets [START_REF] Zammit-Mangion | Deep compositional spatial models[END_REF][START_REF] Huang | Forecasting high-frequency spatio-temporal wind power with dimensionally reduced echo state networks[END_REF][START_REF] Hu | A stochastic locally diffusive model with neural network-based deformations for global sea surface temperature[END_REF]. We remark that, especially when we deal with a nonlinear temporal dynamics as it was the case for the wind data (see Section 6.3), deep neural networks are able to better capture a complex dynamics. As another example, we envisage to use deep learning methods to infer nonstationarities in the SPDE framework, hoping to obtain faster and more accurate estimations than with classical optimization methods. More generally, we intend to take advantage of the predictive capabilities and computational speed of deep statistical learning to deal with high-dimensional spatio-temporal datasets.

A.2 Analytical gradient of the log-likelihood

We here sketch the analytical formula of the different terms of the gradient of the log-likelihood 5.2, reported here:

L(ψ) = - n 2 log(2π) - 1 2 log|Σ y 1:N T | - 1 2 (y 1:N T -η b) Σ -1 y 1:N T (y 1:N T -η b). By setting η b = µ, Q C = Q + σ -2 0 AA and µ C = σ -2 0 Q -1 C A(y 1:N T -µ)
, we can rewrite the equation above as

L(ψ) = - n 2 log(2π)+ n 2 log σ -2 0 + 1 2 (log|Q| -log|Q C |)- σ -2 0 2 (y 1:N T -µ) (y 1:N T -µ) + σ -4 0 2 (y 1:N T -µ) A Q -1 C A(y 1:N T -µ).
By observing that

∂Q C ∂θ i = ∂Q ∂θ i and ∂Q C ∂ log σ -2 0 = σ -2 0 AA ,
we have the following formulas for the gradients of each term of the log-likelihood with respect to each θ i and to log σ -2 0 .

∂ ∂θ i (log|Q| -log|Q C |) = tr Q -1 ∂Q ∂θ i -tr Q -1 C ∂Q ∂θ i = tr (Q -1 -Q -1 C ) ∂Q ∂θ i ∂ ∂ log σ -2 0 (log|Q| -log|Q C |) = -tr   Q -1 C ∂ ∂ log σ -2 0 Q C   = -tr Q -1 C σ -2 0 AA ∂ ∂θ i σ -2 0 (y 1:N T -µ) (y 1:N T -µ) = 0 ∂ ∂ log σ -2 0 σ -2 0 (y 1:N T -µ) (y 1:N T -µ) = σ -2 0 (y 1:N T -µ) (y 1:N T -µ) ∂ ∂θ i σ -4 0 (y 1:N T -µ) A Q -1 C A(y 1:N T -µ) = = -σ -4 0 (y 1:N T -µ) A Q -1 C ∂Q ∂θ i Q -1 C A(y 1:N T -µ) = -µ C ∂Q ∂θ i µ C ∂ ∂ log σ -2 0 σ -4 0 (y 1:N T -µ) A Q -1 C A(y 1:N T -µ) = = -σ -4 0 (y 1:N T -µ) A Q -1 C   ∂Q C ∂ log σ -2 0   Q -1 C A(y 1:N T -µ) = -σ -6 0 (y 1:N T -µ) A Q -1 C AA Q -1 C A(y 1:N T -µ) = -σ -2 0 µ C AA µ C
Hence, finally, the gradient of the log-likelihood with respect to θ i is

∂ ∂θ i L(ψ) = 1 2 tr (Q -1 -Q -1 C ) ∂Q ∂θ i - 1 2 µ T C ∂Q ∂θ i µ C , (A.1)
and the gradient with respect to σ -2 0 is

∂L(ψ) ∂ log σ -2 0 = n 2 - 1 2 tr Q -1 C σ -2 0 AA - σ -2 0 2 (y 1:N T -µ) T (y 1:N T -µ)+ - σ -2 0 2 µ T C AA µ C (A.2)
We remark that in Equation (A.1), we need to compute the derivative of Q with respect to θ i . By calling J = M + dt c (K + B + S) , we recall that the matrices D and F, which are the building blocks of the precision matrix Q and were defined in Section 3.3.2, satisfy

D = J -1 M, F -1 = c τ 2 dt J M -1 Q S M -1 J, D F -1 = c τ 2 dt Q S M -1 J .
When computing the derivative of Q with respect to θ i , we can work on each block of the matrix. We observe that the derivative of the first temporal block Q (1,1) is

∂ ∂θ i Q (1,1) = c τ 2 dt ∂Q S ∂θ i
The derivative of the upper diagonal blocks Q (i,j)

for j = i + 1 is ∂ ∂θ i Q (i,j) = ∂ ∂θ i c τ 2 dt (-D F -1 ) = ∂ ∂θ i c τ 2 dt -Q S M -1 J = = - c τ 2 dt M -1 ∂Q S ∂θ i J +Q S ∂ J ∂θ i ,
and similarly for the transposed blocks Q (i,j) for i = j + 1. For the blocks on the diagonal, Q (i,i) for 2 ≤ i ≤ N T -1, we have

∂ ∂θ i Q (i,i) = ∂ ∂θ i c τ 2 dt J M -1 Q S M -1 J +Q S = = 2 c τ 2 dt ∂ J ∂θ i M -1 Q S M -1 J + J M -1 ∂Q S ∂θ i M -1 J + ∂Q S ∂θ i ,
and for the last diagonal block Q (N T ,N T ) , the last term in the above expressions is removed.

The derivatives that arise from the previous equations are those of Q S and J.

If the parameter controls part of the temporal range, e.g., τ or c, the derivative of the former is zero, and the derivative of the latter is a simple derivative of one of its matrix terms. If the parameter is part of the diffusion, then we have

∂Q S ∂θ i = 2 ∂ K ∂θ i M -1 K,
so we would have further nested derivatives here.

We don't go into further details, but there is an analytical way of computing those gradients for each parameter θ i . If we were also able to compute those gradients with respect to each of the parameters parameterizing any spatially and/or temporally varying parameters field, we could obtain analytical formulas even in the case nonstationary SPDE models. This extension is left for future work.

a b s t r a c t

We broaden the well-known Gneiting class of space-time covariance functions by introducing a very general parametric class of fully nonseparable direct and cross-covariance functions for multivariate random fields, where each component has a spatial covariance function from the Matérn family with its own smoothness and scale parameters and, unlike most of currently available models, its own correlation function in time.

We present sufficient conditions that result in valid models with varying degrees of complexity and we discuss the parameterization of those. Continuous-in-space and discrete-in-time simulation algorithms are also given, which are not limited by the number of target spatial coordinates and allow tens of thousands of time coordinates. The application of the proposed model is illustrated on a weather trivariate dataset over France.

Our new model yields better fitting and better predictive scores in time compared to a more parsimonious model with a common temporal correlation function.

Introduction

An increasing amount of multivariate data indexed by Euclidean space-time coordinates is available in numerous scientific and engineering applications, including atmospheric and * Corresponding author. D. Allard, L. Clarotto and X. Emery Spatial Statistics 52 (2022) 100706environmental sciences, geosciences, as well as geological and mining engineering. For statisticians analyzing these data, one of the key issues is to model the space-time dependence structure not only for each variable separately, but also across the variables. This requires versatile models that can account for different scale and smoothness parameters for each variable, and yet whose parameters can be accurately estimated. Bourotte et al. (2016) proposed valid multivariate spacetime classes based on Matérn and Cauchy spatial covariance functions, inspired by the multivariate Matérn model in Gneiting et al. (2010) and Apanasovich et al. (2012). For both classes, each variable has its own scale and own degree of smoothness in space, while allowing for some degree of cross-correlation. However, in this construction, the marginal temporal correlation function is identical for all variables, which, as discussed in Bourotte et al. (2016), was already seen as a restrictive assumption because it was found that the time correlations of the three variables in the analyzed dataset were different from one variable to the other. To account for different time correlations, space-time linear models of coregionalizations can be proposed, as in Rouhani andWackernagel (1990), De Iaco et al. (2003), Choi et al. (2009) or Finazzi et al. (2013). However, there are two major drawbacks to this construction. First, the smoothness of any component of the multivariate field is restricted to that of the roughest underlying univariate field (Gneiting et al., 2010). Second, the number of parameters to be estimated increases quickly as the number of components of the model increases, thus raising issues of robustness and over-fitting. Gelfand et al. (2005), Apanasovich and Genton (2010), Rodrigues and Diggle (2010), Ip and Li (2016) and Gelfand (2021) introduced Bayesian dynamic, latent dimension, convolution and spectral approaches, respectively, to build valid matrix-valued space-time covariance functions with entries that can have different smoothness parameters. Recent comprehensive accounts of space-time univariate and multivariate covariance modeling can be found in Chen et al. (2021) and in Porcu et al. (2021).

In this work, we propose new classes of matrix-valued space-time covariance functions with Matérn spatial traces that allow, for each variable, different correlation functions in time and different smoothness and scale parameters in space. Our main contribution is to establish sufficient validity conditions for a very general class of covariance functions of the type

C ij (h, u) = σ ij (u)M(h; r ij (u), ν ij (u)), (h, u) ∈ R d × R, i, j = 1, .
. . , p, where M(•; r, ν) denotes a univariate Matérn covariance function with scale and smoothness parameters r and ν. We give several constructions that offer new modeling possibilities, in particular negatively nonseparable covariance functions.

The outline of the paper is the following. Section 2 provides the necessary background material on matrix-valued space-time covariance functions and on matrix-valued pseudo-variograms. Section 3 proposes a first, parsimonious construction based on the substitution approach presented in [START_REF] Allard | Simulating space-time random fields with nonseparable Gneiting-type covariance functions[END_REF], while Section 4 proposes a more general, fully nonseparable, multivariate Gneiting-Matérn class. Section 5 shows how this model can be implemented on a trivariate weather dataset and improves the fitting and prediction performances over an existing more parsimonious model. Concluding remarks follow in Section 6. Other general forms of the Gneiting-Matérn model together with simulation algorithms, technical lemmas and proofs are given in appendices.

Background material

Throughout this work, we use roman letters for scalars and bold letters for vectors, matrices and matrix-valued functions. In particular, 0 and 1 denote vectors or matrices of all-zeros and all-ones of appropriate dimension, respectively. Also, all matrix operations (product, ratio, power, square root, exponential, etc.) are understood as elementwise.

Some matrix definitions and notation

For a = [a ij ] p i,j=1 a symmetric real matrix, we write a ⪰ 0 (resp. a ⪯ 0) to indicate that a is positive (resp. negative) semidefinite, i.e.,

∑ p i=1

∑ p j=1 ω i ω j a ij ≥ 0 (resp. ≤ 0), ∀ ω 1 , . . . , ω p ∈ R. Likewise, a ⪰ c 0 (resp. a ⪯ c 0) indicates that a is conditionally positive (resp. negative) semidefinite, D. Allard, L. Clarotto and X. Emery Spatial Statistics 52 (2022) 100706which is a second-order stationary covariance function for any continuous and completely monotone function ϕ on [0, ∞) if and only if the function γ : R → [0, ∞) is a variogram on R, i.e., a conditionally negative semidefinite function [START_REF] Chilès | Geostatistics: Modeling Spatial Uncertainty[END_REF]. The Gneiting class of spatiotemporal covariances thus involves two functions ϕ and γ : the former is associated with the spatial structure, since C S (h) = ϕ(|h| 2 ), whereas the latter is associated with the temporal structure, with C T (u) = (γ (u) + 1) -d/2 . Notice that C T (u) → 0 as |u| → ∞ if and only if γ (u) is unbounded. From a modeling point of view, the formulation (2) offers more flexibility than the original formulation in Gneiting (2002) and provides a direct geostatistical interpretation in the temporal dimension. Examples of such classes include the Gneiting-Matérn and Gneiting-Cauchy covariance functions, in the cases where ϕ(|h| 2 ) is the Matérn or the Cauchy spatial covariance, respectively. The expression of the Matérn covariance is

M(h; r, ν) = 2 1-ν Γ (ν) (r|h|) ν K ν (r|h|) , h ∈ R d , (3)
where r > 0 is a scale parameter (1/r is called the range), ν > 0 is a smoothness parameter and K ν denotes the modified Bessel function of the second kind of order ν. With α, r > 0, the Cauchy covariance is Rodrigues and Diggle (2010) and De Iaco and Posa (2013) showed that the Gneiting class of covariance functions (2) cannot accommodate negative nonseparability.

C(h; r, α) = ( 1 + r|h| 2 ) -α , h ∈ R d .

Proportional-in-time multivariate Gneiting classes

Going back to the multivariate setting, Bourotte et al. (2016) proposed classes of matrixvalued space-time covariance functions that are extensions of the Gneiting class in (2), where the completely monotone real-valued function ϕ is replaced by a matrix-valued function ϕ = [ϕ ij ] p i,j=1 , with each component ϕ ij having specific parameters. In particular, they showed that the

Gneiting-Matérn space-time model C M = [ C M ij ] p i,j=1 with C M ij (h, u) = σ i σ j ρ ij (γ (u) + 1) d/2 M ( h; r ij (γ (u) + 1) 1/2 , ν ij ) , (h, u) ∈ R d × R, (4)
is a valid second-order stationary matrix-valued covariance function if, for all i, j = 1, . . . , p,

2r 2 ij = r 2 ii + r 2 jj ; 2ν ij = ν ii + ν jj ; ρ ij = β ij Γ (ν ij ) Γ (ν ii ) 1/2 Γ (ν jj ) 1/2 r ν ii ii r ν jj jj r 2ν ij ij , ( 5
)
where Γ is the gamma function, β = [ β ij ] p i,j=1 is a correlation matrix and r ii , ν ii > 0 for i = 1, . . . , p. Using matrix notations, the model in ( 4

) is valid if the matrices ν = [ν ij ] p i,j=1 and r 2 = [r 2 ij ] p i,j=1
have positive entries and are a-separable, and if ρ r 2ν /Γ (ν) ⪰ 0, with ρ = [ρ ij ] p i,j=1 . Similarly, the Gneiting-Cauchy space-time model

C C = [ C C ij ] p i,j=1 with C C ij (h, u) = σ i σ j ρ ij (γ (u) + 1) d/2 C ( h; r ij (γ (u) + 1) 1/2 , α ij ) , (h, u) ∈ R d × R, (6)
is a valid matrix-valued covariance function if, for all i, j = 1, . . . , p,

2r -1 ij = r -1 ii + r -1 jj , 2α ij = α ii + α jj , ρ ij = β ij Γ (α ij ) Γ (α ii ) 1/2 Γ (α jj ) 1/2 r α ij ij (r α ii ii r α jj jj ) 1/2 , ( 7
)
with r ii , α ii > 0 for i = 1, . . . , p and β = [β ij ] p i,j=1 a positive semidefinite matrix. The sufficient conditions in (7) are thus that α and r -1 are a-separable and ρ r -α /Γ (α) ⪰ 0. Proofs are given in Bourotte et al. (2016). Setting h = 0 in both cases, the entries of the associated temporal matrix-valued covariance functions are Allard, L. Clarotto and X. Emery Spatial Statistics 52 (2022) 100706Example 2 (Gneiting-Cauchy Model). Let f ii : r ↦ → Γ (α ii ) -1 exp(-r/r ii ) r -α ii ii r α ii -1 be a gamma probability density on (0, ∞) with shape parameter α ii > 0 and scale parameter r ii > 0. Based on the fact that a gamma mixture of Gaussian covariances is a Cauchy covariance [START_REF] Emery | TBSIM: A computer program for conditional simulation of three-dimensional Gaussian random fields via the turning bands method[END_REF], the direct and cross-covariances are found to belong to the Gneiting-Cauchy family:

C M ij (0, u) = C C ij (0, u) = σ i σ j ρ ij (γ (u) + 1) d/2 , u ∈ R. D.
C ij (h, u) = Γ (α ij ) √ Γ (α ii ) Γ (α jj ) r α ij ij r α ii /2 ii r α jj /2 jj σ ij (η ij (u) + 1) d/2 C ( h (η ij (u) + 1) 1/2 ; r ij , α ij ) , (11)
with

2α ij = α ii + α jj , 2r -1 ij = r -1 ii + r -1 jj
and C the Cauchy covariance defined above.

The advantage of this construction is that it is easy to understand and that it is straightforward to simulate realizations of the multivariate random field Z , as shown in Appendix B. However, an implicit assumption is that ϕ

ij (t) = ∫ ∞ 0 e -tr f ij (r)dr, with f ij being p-separable: f ij = f 1/2 ii f 1/2
jj . This is a parsimonious parameterization, but also a restrictive condition. In Section 4, we propose a more general construction that encompasses the model presented in Example 1.

A general fully nonseparable Gneiting-Matérn class

We now present a very general class of matrix-valued space-time covariance functions that are neither proportional in space nor in time. Their purely spatial part belongs to the Matérn family.

The key of our construction (see Appendix D.2) is to convert a p-variate spatio-temporal random field at finitely many (say, n T ) time coordinates into a (p × n T )-variate spatial random field, and to find conditions ensuring that the latter has a multivariate Matérn covariance function for any choice of n T , based on the results established in [START_REF] Emery | New validity conditions for the multivariate Matérn coregionalization model, with an application to exploration geochemistry[END_REF]. More general classes of space-time covariance functions, with the spatial smoothness parameter depending on the temporal lag, can be elaborated in a similar way, based on findings by Gneiting et al. (2010), Apanasovich et al. (2012) and [START_REF] Emery | New validity conditions for the multivariate Matérn coregionalization model, with an application to exploration geochemistry[END_REF] (see Theorem 3 in Appendix A).

Main result

Theorem 2. Let ν, b 2 and a 2 be p × p symmetric conditionally negative semidefinite matrices, all with positive entries, such that a 2 -ν ⪯ c 0. Let u ↦ → ρ(u) be a p × p matrix-valued covariance function on R and let u ↦ → η(u) be a p × p matrix-valued pseudo-variogram on R. Then, the matrix-valued function

C : (h, u) ↦ → [C ij (h, u)] p i,j=1 with C ij (h, u) = σ ij (u) M(h; r ij (u), ν ij ), (h, u) ∈ R d × R, is positive semidefinite in R d × R for r(u) = [r ij (u)] p i,j=1 = √ αη(u) + a 2 βη(u) + b 2 and σ(u) = [σ ij (u)] p i,j=1 = ρ(u)Γ (ν) exp(ν) ( αη(u) + a 2 ) ν ( βη(u) + b 2 ) d/2 , ( 12
)
where α and β are nonnegative.

Corollary 1. Let ν, b 2 , a 2 , η, α and β be as in Theorem 2. Let τ be a p × p symmetric real matrix. The matrix-valued function C :

(h, u) ↦ → [C ij (h, u)] p i,j=1 with C ij (h, u) = τ ij (βη ij (u) + b 2 ij ) d/2 (αη ij (u) + a 2 ij ) ν ij M ⎛ ⎝ h;    √ αη ij (u) + a 2 ij βη ij (u) + b 2 ij , ν ij ⎞ ⎠ , (h, u) ∈ R d × R, (13
(u) = Y 0 (u) + Y i (u), u ∈ R,
W = [W i ] p i=1 is such that η ij (u) = γ 0 (u) + (R ii (0) + R jj (0))/2 -R ij (u), u ∈ R. In matrix notation, a valid class of unbounded matrix-valued pseudo-variogram is thus η(u) = γ 0 (u)1 + R 0 -R(u), u ∈ R, (14)
where R is any matrix-valued covariance function and R 0 is the a-separable matrix with entries

R 0 ij = (R ii (0) + R jj (0))/2. A more general model consists in defining W i (u) = Y 0 (u + δ i ) + Y i (u), u ∈ R and δ i ∈ R, which leads to the pseudo-variogram η(u) = [γ 0 (u + δ i -δ j )] p i,j=1 + R 0 -R(u), u ∈ R.

Illustration

We illustrate the flexibility of our proposed model in the univariate and bivariate case. We focus specifically on the nonseparability feature. In the univariate setting (p = 1) the space-time covariance function ( 13) is a generalization of the usual Gneiting class with

C (h, u) = 1 (βη(u) + b 2 )(αη(u) + a 2 ) ν M ⎛ ⎝ h; √ αη(u) + a 2 βη(u) + b 2 , ν ⎞ ⎠ , (15)
where η is a variogram. While the Gneiting class is uniformly positively nonseparable (Rodrigues and Diggle, 2010), the model in ( 15) allows for different nonseparability situations, depending on the values of parameters α, β, a and b, as discussed above. Fig. 1 shows the contour plots for the space-time correlation function ( 15) for different values of these parameters.

In the bivariate case, we will consider the general model in Corollary 1 with ν 11 = 1.5, ν 12 = ν 21 = 1 and ν 22 = 0.5; τ 11 = τ 22 = 1 and τ 12 = τ 21 = 0.75. For these values, the condition τe -ν /Γ (ν) ⪰ 0 is verified. We will consider the same nonseparability scenarios as in Fig. 1, i.e., (α, β) being successively equal to (1, 1) (separable), (0, 1) and (1, 0) (resp. positively and negatively nonseparable) and (0.3, 0.7). We further set γ 0 (u) = 0.5|u|, R ij (u) = A i A j (1 + 0.5|u|) -1 with A 1 = 0.8 and A 2 = 0.5, and a ij = b ij = 1 -R 0 ij . Simulations (see Supplementary Material) are performed on a 200 × 200 grid covering the square [0, 2]×[0, 2] (d = 2) with 20 regular times steps in the interval [0, 0.2]. We used the spectral approach given in Section Appendix C (Algorithm 3) with M = 20,000 cosine waves whose frequencies are drawn from the spectral density of a Matérn covariance function with ν = 0.5 and r = 2.

Application to a weather dataset

Introduction

We now illustrate the use of the fully nonseparable multivariate Gneiting-Matérn model proposed in (13) on a weather dataset consisting of three daily variables (solar radiation R, temperature T, and humidity H) recorded at n S = 13 stations in Western France from 2003 to 2012. In order to restrict ourselves to data being stationary in time, we only analyze data recorded in January. In order to filter out any seasonal and regional effect, the daily values are first centered in time and space according to ỹi (s, t) = y i (s, t) -y i (•, t) -y i (s, •) + y i (•, •), i = 1, 2, 3, where y i (•, t), y i (s, •) and y i (•, •) are the spatial average at day t, temporal average at location s and overall average of the ith variable, respectively. These centered values are then standardized every day t according to z i (s, t) = ỹi (s, t)/S i (t) with S 2 i (t) the experimental variance of the ith variable at day t.

We thus have 10 repetitions of 13 × 31 of trivariate standardized vectors, which will be supposed independent. For a detailed exposition of this dataset, we refer to a previous analysis (Bourotte et al., 2016) which has shown that a Proportional-In-Time (PIT) multivariate Gneiting-Matérn model improved the fitting and the prediction scores as compared to a model with a separable spatio-temporal covariance function. In this analysis, we shall compare a specific fully nonseparable multivariate model (FULL), as proposed in (13), to the PIT model.

For each of the three variables, the separability function

S ii (h, u) = C ii (h, u)C ii (0, 0)-C S,ii (h)C T ,ii (u)
was computed. Empirical covariances were derived from the space-time empirical variograms computed with the function EVariogram from the package CompRandFld (Padoan and Bevilacqua, 2015), using spatial bins of 50 m. Since the variograms have been computed on residuals, the separability function must be in the interval [-1, 1], with S(h, u) = 0 indicating separability of C (h, u). Fig. 2 shows the separability function of all variables at the first three time steps. Without having to compute a formal test for separability as for example in [START_REF] Mitchell | A likelihood ratio test for separability of covariances[END_REF], it is clear from these plots that the residuals are uniformly positively space-time nonseparable for |h| ≤ 500 m and u ≤ 3, which is actually often the case for climate variables. The same analysis was performed using other values for the spatial bins and similar results (not shown) were consistently obtained.

Model specification

Given the positive nonseparability of the residuals, and in light of the discussion in Section 4, the parameter α in ( 13) is set to α = 0. The temporal structure is described by a matrix-valued unbounded pseudo-variogram as given in ( 14). Since d = 2 here, the entries of the matrix-valued covariance function read:

C ij (h, u) = τ ij a -2ν ij ij βη ij (u) + b 2 ij M ⎛ ⎝ h; a ij √ βη ij (u) + b 2 ij , ν ij ⎞ ⎠ , (h, u) ∈ R d × R, (16)
with the condition that τe -ν /Γ (ν) ⪰ 0. Setting h = 0 and u = 0 in (16), one gets the generic entry

C ij (0, 0) = τ ij a -2ν ij ij /(β[R 0 ij -R ij (0)] + b 2 ij )
of the collocated covariance matrix. For identifiability reasons, we further set β = 1 and b

2 ij = 1 -R 0 ij with max i {R ii (0)} < 1.
The matrix b 2 is thus conditionally negative definite, as required. As a consequence, we get the simpler expression

C ij (0, 0) = τ ij a -2ν ij ij /[1 -R ij (0)
]. The pseudo-variogram η needs to be specified. For the unbounded part common to all components we choose γ 0

(u) = (1 + |cu| 2a ) b -1, u ∈ R with c > 0, 0 < a ≤ 1 and 0 ≤ b ≤ 1, for
ease of comparison with Bourotte et al. (2016), also advocated in Gneiting (2002). This variogram is reminiscent of the generalized Cauchy covariance model that will also be used for R: R ij (u) = A i A j (1 + |ru| 2λ ) -b , u ∈ R, with r > 0, 0 < λ < 1 and 0 ≤ A i < 1 for i = 1, . . . , p, and where for the sake of parsimony the parameter b is set identical to the parameter b in γ 0 . Notice that the matrix [A i A j ] p i,j=1 is p-separable and has nonnegative entries, thus it is positive semidefinite. R is therefore a valid matrix-valued covariance function. The covariance model that we obtain is

C ij (h, u) = τ ij a -2ν ij ij (1 + |cu| 2a ) b -A i A j (1 + |ru| 2λ ) -b × M ( h; a ij [ (1 + |cu| 2a ) b -A i A j (1 + |ru| 2λ ) -b ] 1/2 , ν ij ) , (h, u) ∈ R d × R.
(u) = [ (1 + c|u| 2a ) δ -A i A j (1 + r|u| 2λ ) -δ
] -1 with 0 ≤ δ ≤ 1 leads to the parametric family that will be used in the rest of this section. In order to get C ii (0, 0) = σ 2 i , we set τ ii = σ 2 i a

2ν ii ii (1 -A 2 i ) 2 and, for i ̸ = j, τ ij = ρ ij √ τ ii τ jj , with [ρ ij ] p i,j=1
being a correlation matrix. With this parameterization, the matrix τ is positive semidefinite by construction.

Finally, the fully nonseparable multivariate Gneiting-Matérn covariance model reads

C ij (h, u) = 1 (1 + |cu| 2a ) δ -A i A j (1 + |ru| 2λ ) -δ ρ ij a -2ν ij ij √ τ ii τ jj (1 + |cu| 2a ) b -A i A j (1 + |ru| 2λ ) -b × M ( h; a ij [ (1 + |cu| 2a ) b -A i A j (1 + |ru| 2λ ) -b ] 1/2 , ν ij ) , (17)
with (h, u) ∈ R d × R and i, j = 1, . . . , p. The model in ( 17) provides interpretable parameters for a fully nonseparable Gneiting-Matérn model:

• The collocated covariance matrix [C ij (0, 0)] p i,j=1 has elements C ij (0, 0) = ρ ij σ i σ j a ν ii ii a ν jj jj a 2ν ij ij (1 -A 2 i )(1 -A 2 j ) (1 -A i A j ) 2 , (18)
which simplify to C ii (0, 0) = σ 2 i if i = j. This is the elementwise product of a covariance matrix [ρ ij σ i σ j ] p i,j=1 with a weight matrix whose elements are ratios combining the other parameters. Notice that the quantities σ i a ν ii ii appearing in (18) are the square roots of the microergodic quantities that can be consistently estimated by Maximum Likelihood under a Gaussian assumption in an infill asymptotics setting (Zhang, 2004).

• The vector A = [A i ] p i=1 describes the multivariate aspect of the temporal covariance function.

When A i = 0 for i = 1, . . . , p, the above model reduces to the model in Bourotte et al. (2016, Equation 9), up to a slight reparameterization.

• The temporal parameters are, on the one hand, (c, a) that characterize the common part of the temporal covariance function and, on the other hand, (r, λ, A) that characterize the multivariate part of it. The parameters (c, r) are scale parameters, while (a, λ) are shape parameters relating to the regularity at the origin of C T ,ij (u). Note that the parameters (b, δ)

act on both parts of the temporal covariance function.

• The parameter b, which appears in both the spatial and temporal terms, is the separability parameter. It can vary from b = 0 (separability) to b = 1 (full spatio-temporal nonseparability). • The spatial parameters are the scale matrix a, with a 2 ⪯ c 0, and the smoothness matrix ν, which must verify a 2 -ν ⪯ c 0. For simplicity, and for a fair comparison to Bourotte et al. (2016), a 2 and ν will be chosen as a-separable. The two above conditions on a and a 2 -ν are thus always verified.

Estimation of the parameters

Recall that the data analyzed here are standardized residuals after centering for spatial and temporal trends. We thus set σ i = 1, for i = 1, . . . , p. There is a total of p(p + 2) + 6 parameters to estimate, where p is the number of variables. Here p = 3, leading to a total of 21 parameters.

Compared to the model in Bourotte et al. (2016), our more flexible model contains only 6 additional parameters describing the multivariate nature of the temporal covariance. Let us denote θ the vector of all parameters, and let Θ be the space of parameters for which the model ( 17) is valid. Optimizing a likelihood function in Θ is not straightforward. First, the space Θ has a complex shape owing to the positive semidefiniteness condition on τe -ν /Γ (ν). Second, employing blindly an optimization function to maximize the likelihood in the high-dimensional space Θ is likely to fail. As a way to alleviate this problem, the likelihood was maximized sequentially in subspaces of Θ corresponding to blocks of related parameters, while keeping all other parameters fixed to the values previously attained.

Maximum likelihood estimation requires inverting a 1209 × 1209 matrix each time a parameter is updated, which is prohibitively expensive considering the very large number of calls entailed by the optimization in a 21-dimensional space. Composite likelihood methods have been found to be efficient in the context of spatio-temporal data, with less time-consuming steps and good asymptotic properties (Bevilacqua et al., 2012). Pairwise Likelihood (PL) is the product of marginal Gaussian likelihoods computed on all pairs with a spatial separation less than d m and a temporal separation less than t m , where d m and t m are fixed thresholds. On a simulation study mimicking the sampling design of the dataset, it was shown in Bourotte et al. (2016) that, for the smoothness and scale parameters, Pairwise Likelihood (PL) provides estimates with only a slight loss in efficiency as compared to a Full Likelihood (FL) approach, with a significant gain in terms of computation. Moreover, for prediction, the difference between PL and FL was found very small on all tested prediction scores. We have therefore decided to estimate the parameters of our model using PL in this same way. We have set d m = 250 km and t m = 2 days because these values have been shown

to minimize the trace of the estimated covariance matrix, thus providing optimal estimation for all parameters (Bourotte et al., 2016). The following procedure for maximizing the likelihood builds on the one presented in Bourotte et al. (2016), which was proven to be efficient.

1. The p×p marginal empirical covariance matrix with elements ĉij , i, j = 1, . . . , p is computed.

2. The separability parameter b, known to be difficult to estimate, is successively fixed to 0, 0.1, . . . , 1.

For every fixed value of b:

(a) Initial temporal parameters (c, a, r, λ, δ, A) are estimated by maximizing the PL for the temporal covariance model C T . Using ( 17) and ( 18), one gets

C T ,ij (u) = 1 (1 + |cu| 2a ) δ -A i A j (1 + |ru| 2λ ) -δ ĉij (1 -A i A j ) 2 (1 + |cu| 2a ) b -A i A j (1 + |ru| 2λ ) -b .
(b) Using the estimates  from the previous step, initial spatial parameters (a, ν) are estimated by maximizing the PL for the spatial matrix-valued covariance C S with entries

C S,ij (h) = ĉij M ⎛ ⎜ ⎝ h; a ij ( 1 -Âi Âj ) 1/2 , ν ij ⎞ ⎟ ⎠ .
(c) The temporal parameters are updated, considering all other parameters fixed, by maximizing the PL of the multivariate spatio-temporal model ( 17). (d) The spatial parameters are updated, considering all other parameters fixed by maximizing the PL of the multivariate spatio-temporal model ( 17). (e) Steps (c)-(e) are iterated until a stopping criterion is reached.

In steps 3.(a) to 3.(d), the optimization is performed by calling the R function nlminb. At each iteration of the optimization, the estimate ρ is computed using (18) given â, ν, Â, and the definite positiveness of ν and τe -ν /Γ (ν) are enforced. Iterations are stopped when PL is increased by less than 1 unit after one iteration from (c) to (e). Finally, b is the value of b among {0, 0.1, . . . , 0.9, 1} corresponding to the highest maximized PL. To perform the maximization in the subspaces of Θ, we used the package nlminb implemented in R with lower and upper bounds for the parameters when mathematically necessary. In addition, for the sake of numerical stability, the smoothness parameter ν was upper bounded at 6.0 -which was never reached.

All 13 stations were used to estimate the parameters of both the FULL and PIT models. log PL FULL ( θb=0.1 ) = -933 337.6. For comparison purpose, PL was also maximized for the PIT model, i.e., when A = 0. For this simpler model, log PL PIT ( θb ) is monotonically decreasing from -933 961.3 to -934 072.6 as b varies from 0 to 1. FL was also computed for these two models (remember that FL is easy to compute but very long to optimize): log FL FULL ( θb=0.1 ) = -2 652.7 and log FL PIT ( θb=0.1 ) = -2 819.1, indicating strong evidence in favor of the more complex model, considering that the number of data is moderate with a total of 10 independent repetitions of 1209 correlated data. As a point of comparison, the (BIC) penalization to the full likelihood of the more complex model is equal to 2.5 ln(12090) ≃ 23.5.

Looking at the temporal marginals, the models show very different behaviors -as we expected. Fig. 3(right) displays the temporal empirical covariances for variables R and H with their fitted models. Thanks to the flexibility of the FULL model, the temporal covariance functions fit the empirical values for both variables. In contrast, the unique covariance function of the PIT model lies somewhere between the empirical covariances of the two variables, thus being unable to provide a satisfactory fit to any of them. Table 1 reports the estimated parameters maximizing PL for both models. It is interesting to note that the parameter λ is equal to 0 in the FULL model, thus implying a sort of temporal nugget effect, visible on the temporal covariances in Fig. 3 spatial (at time lag u = 0) and spatio-temporal covariances (at time lags u = 1, 2) for all variables and pairs of variables. The empirical covariances between all pairs of stations and the FULL model with estimated parameters from Table 1 are shown. Overall, the fit is good. On the top left panel representing the covariance function for the variable R, the very high smoothness of the covariance function near the origin is clearly visible.

Prediction

We consider two different prediction settings. In the first setting, prediction of the three variables at a validation station at day t is made knowing the trivariate data at the other 12 estimation stations at day t, t -1 and t -2, each station being used as validation station in turn. This setting is called Spatial Interpolation because data from the same day at different locations are used (as well as data from previous days). In the second setting, prediction is made with data from days t-1 and t-2 only at all 13 stations. In this second setting, called Temporal Prediction, purely temporal covariances are thus introduced. The first setting corresponds to interpolation at an ungauged location while the second setting corresponds to the classical one-day ahead forecast. Predictions are computed from January 3rd to January 31st since the two previous days must be used as conditioning data. In the Gaussian framework, the conditional expectation is the best linear unbiased prediction, also called (co)kriging in the geostatistics literature, while the conditional covariance matrix is the covariance matrix of the cokriging errors [START_REF] Chilès | Geostatistics: Modeling Spatial Uncertainty[END_REF]. The conditional variances are identical for all days since the configuration of the conditioning data remains identical.

The two models (FULL and PIT) are compared by means of four scores: the Mean Square Error (MSE), the Mean Absolute Error (MAE), the Continuous Ranked Probability Score (CRPS) and the D. Allard, L. Clarotto and X. Emery Spatial Statistics 52 (2022) Logarithmic Score (LogS) [START_REF] Gneiting | Strictly proper scoring rules, prediction, and estimation[END_REF]. The first two scores, MSE and MAE, compare the conditional expectation to the true value. The other two scores, CRPS and LogS, assess not only the prediction but the prediction variance as well. They both measure the discrepancy between the predictive cumulative distribution function and the true value. More details can be found in [START_REF] Gneiting | Strictly proper scoring rules, prediction, and estimation[END_REF]. Table 2 reports our results. As a benchmark, we also report the scores obtained with a trivial prediction where all predicted values are set to the expectation equal to 0. Since no prediction variance is attached in this trivial setting, CRPS and LogS are not available. Recall that lower scores indicate a better adequacy between the model and the data and that the first three scores are bounded from below by 0 whilst LogS is unbounded.

For variables R and T, Spatial Interpolation is more accurate than Temporal Prediction. For variable H, the situation is reverse, in relation with the fact that the associated MSE is close to that of the trivial prediction. Usually, spatial correlations are higher than one-day-ahead temporal correlations, but for H the spatial correlation are much weaker than for R and T (see Fig. 4). In the Spatial Interpolation setting, the two models can hardly be distinguished: the validation scores are often equal, and when they are not, they are extremely close.

In the Temporal Prediction setting however, the FULL model clearly outperforms PIT for all variables and for all scores, in accordance with the fact that the FULL model is able to better fit the data in time thanks to the extra parameters in the temporal matrix-valued covariance function. In this setting, we also computed the standard deviations of the squared and absolute prediction errors for the FULL model (those of the PIT model are very close) and divided them by √ 11310, where 11310 = 13 × 10 × 29 × 3 is the total number of predictions. From this, we see that the MSE score difference is larger than twice this value for variable R. In addition, the absolute error is smaller with FULL than with PIT for 6151 predictions out of 11310, which represents 54.4% of the predictions. As a matter of comparison, the centered 99% interval of a binomial random variable with probability of success 0.5 and sample size 11310 is [5518,5792]. The improvement brought by the FULL model as compared to the PIT model can thus be considered as significant.

For illustration purpose, Fig. 5 shows the predictions of the three variables from January 3rd to January 31st 2003 at one station (Le Rheu) following the Spatial Interpolation setting. The predicted residuals were back-transformed to the original scales to be compared to the real values.

An envelope of ±2 conditional standard deviations was added to the plot.

Concluding remarks

We have proposed new parametric classes of matrix-valued covariance functions for multivariate spatio-temporal random fields, where each component has its own smoothness and scale parameter in space and its own correlation function in time. Our constructions generalize the Gneiting class of space-time covariance functions and are fully nonseparable, in the sense that they are spacetime nonseparable and that they are neither proportional-in-space nor proportional-in-time. This is a major improvement to most of the models proposed up to date (with the exceptions of the models in Dörr andSchlather, 2021 andPorcu et al., 2022), where the marginal temporal correlation function is identical for all the variables, which is overly restrictive. The main ingredient for achieving this has been the use of pseudo-variograms, as multivariate analogous of Bernstein functions as pointed out in Dörr and Schlather (2021) and Qadir and Sun (2022) in their discussion sections.

The main class proposed in Section 4 allows for a general behavior for the spatial scale parameter as a function of the time lag u, depending on the values taken by the parameters. It is well known that the Gneiting class of models is positively nonseparable. A very original feature of our construction is that the models can be negatively nonseparable or asymptotically separable as |u| → ∞. Since these models are continuous over R d × R, their application is not limited to data organized as regular records at measurement stations. More complex designs are in principle possible, such as irregular or incomplete records, heterotopic designs or even mobile measurement devices. We hope to see applications of our models to such datasets in a near future. From a statistical point of view, the main challenge is to find the right balance between the flexibility of the model, which must be able to account for the complex interactions between space D. Allard, L. Clarotto and X. Emery Spatial Statistics 52 (2022) ] n T m,n=1

. The simulation can be done by means of the continuous spectral algorithm presented in [START_REF] Emery | An improved spectral turning-bands algorithm for simulating stationary vector Gaussian random fields[END_REF], which relies on importance sampling and on the square root decomposition of the q × q positive semidefinite matrix F (Ω ) evaluated at a random vector Ω. In addition to parameters verifying the conditions of Theorem 2, a probability density function g that does not vanish on R d is thus required to define Ω. This is summarized in Algorithm 3, where ⟨•, •⟩ stands for the usual scalar product in R d , R ∼ f is the short notation for ''R is distributed according to the distribution f '' and U (0, 2π ) is the uniform distribution on (0, 2π ).

The spectral simulation algorithm is applicable when q is not too large, say, less than a few tens of thousands, so that the square root decomposition of the spectral density matrix F (Ω ) can be achieved. There is no such restriction on the number of spatial nodes n S , which can be much higher, insofar as the simulation in space is achieved in a continuous manner by using cosine waves that can be computed at as many coordinates as desired. We refer to [START_REF] Arroyo | Algorithm 1013: An R implementation of a continuous spectral algorithm for simulating vector Gaussian random fields in Euclidean spaces[END_REF] for a proper choice of the distribution g. In particular, 1 -b ⪯ c 0, where 1 is the all-ones matrix of size p × p.

Proof. The fourth property derives from the first one, which can be proven by using Lemma 1, and from the fact that the sum of two conditionally negative (positive) semidefinite matrices is still conditionally negative (positive) semidefinite. For a proof of the second and third properties, see Berg et al. (1984, Chapter 3, Property 1.9 Since the entries of Λ add to zero, the first two terms in the last equality are zero, while the last two terms are negative or zero since, on the one hand, the entries of Λ add to zero and a is conditionally negative semidefinite and, on the other hand, C0 is a variance-covariance matrix, hence positive semidefinite. □

D.2. Proofs

Proof of Theorem 1. The proof is constructive and based on an extension of the substitution approach proposed in [START_REF] Allard | Simulating space-time random fields with nonseparable Gneiting-type covariance functions[END_REF]. Consider the p-variate space-time random field Z = [Z i ] p i=1 defined as

Z i (s, t) = √ 2f i (R) f (R) V i cos ( √ 2R ⟨Ω, s⟩ + |Ω| √ 2 W i (t) + Φ ) , (s, t) ∈ R d × R,
where R is a nonnegative random variable with a probability density f whose support contains that of f 1 , . . . , f p , V = [V i ] p i=1 is a centered Gaussian random vector with covariance matrix σ = [σ ij ] p i,j=1 , Ω is a standard Gaussian random vector with d independent components, Φ is a uniform random 21

INTRODUCTION

We congratulate the authors for organizing such a great and challenging competition. Our team brings together researchers from two French groups that have long-standing collaborations: the BioSP research unit at INRAE and the Geostatistics team at Mines ParisTech (formerly known as Ecole des Mines de Paris). They both belong to the RESSTE network 1 funded by INRAE. RESSTE organizes scientific animation around models, methods and algorithms for spatiotemporal data, and it fosters collaborations between statisticians and other scientists sharing interest in spatial and spatiotemporal data. Forced to be physically distant due to the Covid-19 sanitary crisis, we set up an efficient working environment with the help of collaborative online platforms for code, text and vivid discussions. We were thus able to contribute to all sub-competitions, including with multiple submissions for sub-competition 2. We enjoyed very much participating to this exercise. In addition to D. Allard et al. congratulate Huang, Sameh, Ying, Hatem, David and Marc for the excellent organization, we sincerely wish to thank them for the nice moments it brought to the four of us.

COMPETITION 1: BLOCK APPROACHES FOR GAUSSIAN PROCESSES

Data are known to have been generated from a Gaussian process with Matérn covariance function. Maximum likelihood (ML) would thus be the most efficient estimation method, and conditional expectation, also referred to as Kriging, is optimal for prediction. However, due to the size of the dataset, neither full ML nor unique neighborhood Kriging can be achieved, at the exception of ExaGeoStat. Efficient approximations must be sought. Here, we have opted for block approaches that satisfy the following principles: (i) in each block, estimation (ML) or prediction (Kriging) is optimal; (ii) blocks should be as large as possible while taking into account computational issues; (iii) blocks are assumed to be independent among each other. The approximation lies entirely in point (iii), and it is easy to understand that (ii) is the key for achieving good performances. What "large" means has slightly different meaning for estimation and for prediction. As regards estimation, some parameters control local properties, while others are global. Therefore, blocks for estimation need to have a large spatial extent. For plug-in Kriging, only local information is necessary. Prediction blocks are thus local, containing as many data points as possible. Details are provided below.

ESTIMATION

First, a rough estimation of the parameters was performed on each dataset with weighted least squares fits for experimental variograms using the package RGeostats. These estimates, from which an approximate effective range E R = β√ 12 ν was computed, allowed us to gain a general picture of the experimental design similar to that shown in Table 1 in Huang et al. (2021). In particular, E R was clearly close to the size of the domain for some datasets.

In a second stage, estimation of the parameters was performed using a maximum composite block-likelihood (BL) method. Blocks are characterized by their size (number of data points, N D ), shape and location of the data within the blocks. Nugget (τ 2 ≥ 0) and smoothness (ν > 0) are local parameters, whilst range (β > 0) and sill (σ 2 > 0) are non-local. When τ 2 = 0, Zhang (2004) showed that the only quantity that can be efficiently estimated in an in-fill asymptotic framework is σ 2 β -2ν . Efficient estimation of all parameters thus requires a "large domain" framework that allows sampling small distances for estimating τ 2 and ν, and intermediate to large distances with respect to E R for estimating β and σ 2 . In each block, the sub-sample must be built so as to sample all distances from 0 to a multiple of the practical range. Data separated by a distance larger than 2 to 3 times the practical range are useless for estimating nugget, regularity and range parameters and can be excluded. Here, blocks were disks with a radius set to 1.5E R, centered on a regular B × B grid covering the domain. N D points were then sampled at random within each disk with a weight decreasing linearly from the center (where it is equal to 1) to the edge of the disk (where it is equal to 0).

EXPLORATION OF THE DATA

The first step of the solution to sub-competitions 2a and 2b is related to the exploratory analysis of the dataset. Some statistical quantities such as the mean, the minimum and maximum values (Table 1) were computed along with the histograms for each of the four datasets (Fig. 1a). It is clearly visible that only one of the four datasets (2b1) could be considered as marginally Gaussian distributed. The histograms of the other three datasets (2a1, 2a2, 2b2) present heavy tails toward high values. This fact is highlighted also by the maximal values of the datasets, which are extremely far from the mean if compared to the corresponding minimum values, showing asymmetry of tails. These insights suggest that the non-Gaussian datasets could be transformed to Gaussian models before applying geostatistical inference and prediction. Various types of transformations were inspected as reported in the next section.

TRANSFORMED MARGINS

Tukey g and h Transform

A flexible parametric marginal transform of Gaussian variables was proposed by J.W. Tukey and is known as the g and h distribution [START_REF] Jorge | Some properties of the Tukey g and h family of distributions[END_REF]. It has been D. Allard et al.

Bootstrap Approach

This approach is designed to run fast with moderate computing resources, such as those available on a personal computer. We proceed as follows using the pretransformed data:

1. Bootstrap estimations (25 samples) of Matérn correlation parameters using marginally pretransformed data: subsampling (without replacement) of 500 observations, and estimation of the scale and shape parameters of the Matérn correlation function.

2. The final estimated Matérn correlation parameters are set to the median of the bootstrap estimates.

3. Simple Kriging prediction is performed on the standard Gaussian scale by using k nearest neighbors of observed locations around the location to predict.

4. Standard Gaussian predictions are transformed back to the original scale by using the direct Tukey g and h transformation in (1) with parameters estimated in Step 1.

We have used validation data to choose among several values k = 25, 50, 100 of nearest neighbors in Step 5. The implementation was realized using the R library CompRandFld for estimation of covariance parameters and Kriging.

GpGp

Vecchia approximations are a particular case of composite likelihood methods. They can also provide an approximation of the parent Gaussian process (Katzfuss and Guinness 2021). The computations are based on the Cholesky factor of the inverse covariance matrix that can be computed explicitly and that is sparse by construction. Therefore, they allow for numerically efficient inference and prediction. The package GpGp proposes an implementation of a Vecchia approximation that uses an elaborate way to order and group the data into conditionally independent blocks (Guinness 2018). It also provides an implementation of the Fisher scoring algorithm for the ML estimation of the parameters (Guinness 2021). We have used this package for parameter estimation and prediction for marginally pretransformed data in datasets 2a1, 2a2 and 2b2, or directly for dataset 2b1. Then, the predictions were transformed back into their original scale. The only parameter to be set is the number of neighbors to be considered in the groups for estimation and prediction. It has been set by trial and error regarding the prediction performances on out-of-sample validation data.

VALIDATION

After transformation of the marginals (either through a nonparametric approach or through Tukey g and h), two parameters had to be set for the Vecchia approximation approach: the number of neighbors in the estimation step, n e , and the number of neighbors in the prediction step, n p . A holdout validation method was used to define the best n e and n p . Each time 70000 (resp. 700000) data were selected as training points in the datasets of sub-competition 2a (resp. 2b), and RMSE was computed over the 20000 (resp. 200000) remaining points. The values of n e and n p leading to the best RMSE were n e = 50 for datasets in 2a, n e = 30 for datasets in 2b and n p = 100 for all datasets. In the bootstrap D. Allard et al. Future competitions could also consider more challenging types of trans-Gaussian models arising in the analysis of agricultural, biological and environmental data, such as zero-inflated data, count data and other discrete data, or compositional data. Non-Gaussian margins arising from non (trans-)Gaussian random fields such as skew-Gaussian, skew-elliptical or maxstable random fields could also be considered.

Another very interesting extension would be to increase the dimensionality of the data by considering multivariate random fields, spatiotemporal random fields, or both. In different communities, such as machine learning, sensitivity analysis and uncertainty quantification, Gaussian processes are used in high-dimensional spaces. Testing the methods that have been developed successfully in spatial statistics to the problems faced by these communities offers interesting perspectives.

To provide a more realistic setting of real data analyses, future comparisons of prediction methods could also explore nonstationary settings with trend functions that may depend on external predictors, and with nonstationary covariance functions.

2. 1

 1 Venn Diagram of nonseparable SPDE models proposed in the literature.

3. 1

 1 FEM solution to the PDE (3.5) with b = 0, λ = 0.1, f = 1, h = 1 and Dirichlet BCs. Left: γ = γ 1 = [0.5, 0.5] without stabilization; Middle: γ = γ 2 = [5, 5] without stabilization. Right: γ = γ 2 = [5, 5] with SD stabilization. . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Sparsity pattern of precision matrix. Left: N S = 5 2 and N T = 5; Right: N S = 20 2 and N T = 5. . . . . . . . . . . . . . . . . . . . . . . . 4.1 Spatial trace at 4 time steps (separated by 4dt) of the simulation of advection-diffusion SPDE for α = 1, α S = 0, κ = 0.1, c = 1, γ = [1, 0] , H = 1 11 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Spatio-temporal variogram maps of simulation of models of Table 4.1. Top row: (A), (B). Middle row: (C), (D). Bottom row: (E), (F). . . . 4.3 Spatio-temporal variogram of simulation of models of Table 4.1. Top row: (A), (B). Middle row: (C), (D). Bottom row: (E), (F). . . . . . . 4.4 Separable spatio-temporal model (α S = 2, α = 0, κ = 0.1, τ = 1, c = 0.2, γ = 0). Left: spatial trace obtained with FFT and theoretical Matérn covariance (ν = α S -1 = 1). Right: temporal trace obtained with FFT and theoretical exponential covariance. . . . . . . . . . . . . 4.5 Covariances of stationary spatial trace of nonseparable spatio-temporal model (α S = 0, κ = 0.5, τ = 1, c = 1, γ = 0) computed with FEM/FDM ("discretized"), FFT and theoretical Matérn. Left: α = 2, hence ν = 1. Right: α = 3, hence ν = 2. . . . . . . . . . . . . . . . . . 4.6 Experimental spatial covariance over 4 following time steps of the simulation of advection-diffusion SPDE for κ = 0.3, τ = 1, c = 1, α = 1 and γ = [0, 1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Experimental spatial covariance over 4 following time steps of the simulation of diffusion SPDE for κ = 0.33, α = 1, τ = 1 and c = 1, shifted at each time step of dx = 1 on the x-axis. . . . . . . . . . . . . . . . . 4.8 Advection-dominated SPDE. Spatial trace at 4 time steps (separated by dt = 1) of the simulation of advection-diffusion SPDE for α = 1, α S = 0, κ = 0.3, τ = 1, c = 1, γ = [5, 0] , without stabilization. . . . . xvi 4.9 Stabilized advection-dominated SPDE. Spatial trace at 4 time steps (separated by dt = 1) of the simulation of advection-diffusion SPDE for α = 1, α S = 0, κ = 0.3, τ = 1, c = 1, γ = [5, 0] , with Streamline Diffusion stabilization. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Left: Triangulated mesh T with N S = 5 2 and n obs = 5 scattered data (blue). Right: spatial projection matrix A S between scattered points and points T (size (n, N S )). . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Left: spatial matrix A S A S (size (N S , N S ) with N S = 5 2 ). Right: spatio-temporal block-diagonal matrix A A (size (N S N T , N S N T ) with N T = 5). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Stations capturing GHI over the spatial domain. . . . . . . . . . . . . 6.2 GHI G and Clear Sky Index K c for 4 different stations on May 28, 2013. 6.3 Histogram of K c over 20 time steps. . . . . . . . . . . . . . . . . . . . 6.4 Validation settings: Uniform (left) and South-East (right). . . . . . . . 6.5 Predictions of K c at (T + 1), (T + 2) and (T + 3) with model (3) ("adv-diff" with α = 1 and α S = 2). The black contoured dots are the conditioning locations and the white contoured dots are the validation locations for the Uniform setting. . . . . . . . . . . . . . . . . . . . . . 6.6 Real K c , mean of conditional simulations of K c and ±2σ envelope at time horizons T, (T + 1), (T + 2), . . . , (T + 6). Left: orange station. Right: green station. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7 Left: spatial locations of WRF wind simulations over the entire spatial domain. Right: topography of Saudi Arabia. . . . . . . . . . . . . . . 6.8 Wind residuals during January 2, 2016 at 6 different spatial locations.

6. 13

 13 Handmade mesh to capture complex topographic areas (the spatial domain is given in km × km). . . . . . . . . . . . . . . . . . . . . . . . 7.1 Nonstationary spatial trace given by spatially varying κ(s) = s 2 /200 (left) and spatially varying γ(s) = [-0.5(s 2 -50), 0.5(s 1 -50)] (right). The other parameters are set to κ = 0.33, c = 1, τ = 1, dt = 1, h = 1. 7.2 2D clamped B-splines basis in which are represented the spatially varying parameters γ 1 (s) and γ 2 (s). . . . . . . . . . . . . . . . . . . . . . .List of Tables4.1 Different diffusive models obtained with combinations of α and α S , along with corresponding marginal spatial variance and practical range. 48 4.2 Mean (std) ×10 -3 of differences between experimental spatial covariances over 4 following time steps with shifted diffusion SPDE and corresponding advection-diffusion SPDE. . . . . . . . . . . . . . . . . . . 52 5.1 Mean (and standard deviation) of ML estimates θ = [κ, γ1 , γ2 , ĉ, τ ] over 50 simulations for two different subsets of advection-diffusion model parameters and two different estimation approaches (Cholesky decomposition and matrix-free approach). . . . . . . . . . . . . . . . . 70 6.1 Estimated parameters and log-likelihood for 6 different models from all data on a 20-minute window. . . . . . . . . . . . . . . . . . . . . . 74 6.2 Averaged RMSE computed at 10 successive time steps for 6 different models, 2 validation settings (Uniform and South-East) and 3 prediction configurations (S, T and ST); see text for details. In each case, the best score among the models is in bold font. . . . . . . . . . . . . 77 6.3 Average MSE at (T+1), (T+2), (T+3) from n S = 53333 observations during 24 hours. Persistence, FRK, and various SPDE models (2.27) with different combinations of α and α S , with or without advection, and N S = 40 2 . The best score among the models is in bold font. . . . 82 6.4 Average MSE at (T+1), (T+2), (T+3) from n S = 53333 observations during 24 hours. Advection-diffusion SPDE model (2.27) with α = 1 and α S = 2, and N S = 40 2 , N S = 50 2 , N S = 60 2 and custom-made mesh. The best score among the different meshes is in bold font. . . . 82 7.1 Mean (and standard deviation) of ML estimates θ = (κ, v1 , v2 , x1 , x2 , ĉ, τ ) over 10 simulations for two different subsets of nonstationary advectiondiffusion model parameters (rotational advection). . . . . . . . . . . . 90 8.1 PL parameter estimates for the weather dataset with d m = 250 km, t m = 2 days. FULL: fully nonseparable model as in (8.7) with b = 0.1. PIT: simplified Proportional-In-Time model with b = 0. The indices 1, 2 and 3 refer to R, T and H, respectively. . . . . . . . . . . . . . . . 101 xx 8.2 Prediction scores at the validation stations at day t, using ML estimates. FULL: fully nonseparable model as in (8.7) with b = 0.1. PIT: simplified Proportional-In-Time model with b = 0. The best scores among FULL or PIT are shown in bold font. . . . . . . . . . . . . . . 102

  trade-off is required by the massive spatial and spatio-temporal datasets available nowadays. Those techniques will be discussed in terms of predictive capabilities based on the results of the "Spatial Statistics Competition for Large Datasets" proposed by the King Abdullah University of Science and Technology (KAUST) in 2021 and 2022. We will present the techniques that led the team RESSTE, composed of 4 researchers from Mines Paris and INRAE Avignon (Denis Allard, Thomas Romary, Thomas Opitz and myself), to rank first in some of the sub-competitions in both editions.

≥ 0 ,

 0 |g R | is inferiorly bounded by the inverse of a strictly positive polynomial and g R cos βπ 2 Theorem 1 and Proposition 3 in Carrizo-Vergara et al. (2022) state that (2.6) admits a unique stationary solution for every arbitrary g I function. For the rest of the Chapter, we will call this property the Sufficient Condition for Existence and Uniqueness (SCEU).

Figure 3 .

 3 Figure 3.1: FEM solution to the PDE (3.5) with b = 0, λ = 0.1, f = 1, h = 1 and Dirichlet BCs. Left: γ = γ 1 = [0.5, 0.5] without stabilization; Middle: γ = γ 2 = [5, 5] without stabilization. Right: γ = γ 2 = [5, 5]with SD stabilization.

Figure 3 . 2 :

 32 Figure 3.2: Sparsity pattern of precision matrix. Left: N S = 5 2 and N T = 5; Right: N S = 20 2 and N T = 5.

  is discretized in space with a triangulation T with N S nodes and discretized in time by means of N T regular time steps. This space-time discretization is denoted T = T × {T /N T , . . . , T }.When considering the SPDE (2.27) on a spatial bounded domain Ω, one has to choose some boundary conditions.[START_REF] Lindgren | An explicit link between gaussian fields and gaussian markov random fields: the stochastic partial differential equation approach[END_REF] and[START_REF] Khristenko | Analysis of boundary effects on pde-based sampling of whittle-matérn random fields[END_REF] showed that, when Neumann boundary conditions are used, one can obtain a satisfactory solution to SPDE (2.3) on Ω by using a domain Ω that is extended by a distance which is at least two times the practical correlation range r = 8(α tot -1)/κ outside the domain of interest Ω. They further showed that the error between the covariance of the solution and the covariance of the solution inside the domain Ω decreases exponentially as r increases. We chose to use the same practical framework.

  at 4 time steps separated by 4dt. The solution is the approximated field obtained with the FEM/FDM approach detailed in Section 3.3. We remark how the anisotropic matrix influences the form of the spatial field, giving an elongated vertical dependence structure, and how the spatial field moves to the right due to the advection term. Now, we restrict ourselves to the purely isotropic diffusive SPDE (γ = [0, 0] and H = I) with κ = 0.3, c = 1, τ = 1, and we compare the spatio-temporal models obtained with different combinations of α and α S . The combinations are summarized in Table4.1, along with the marginal spatial variance and the value of the practical range (when available) computed from the results of Proposition 2.1. The models (A)

Figure 4 . 1 :

 41 Figure 4.1: Spatial trace at 4 time steps (separated by 4dt) of the simulation of advection-diffusion SPDE for α = 1, α S = 0, κ = 0.1, c = 1, γ = [1, 0] , H = 1 1 1 4 .

Figure 4

 4 Figure 4.2: Spatio-temporal variogram maps of simulation of models of Table 4.1. Top row: (A), (B). Middle row: (C), (D). Bottom row: (E), (F).

Figure 4 . 3 :

 43 Figure 4.3: Spatio-temporal variogram of simulation of models of Table 4.1. Top row: (A), (B). Middle row: (C), (D). Bottom row: (E), (F).

Figure 4 . 4 :

 44 Figure 4.4: Separable spatio-temporal model (α S = 2, α = 0, κ = 0.1, τ = 1, c = 0.2, γ = 0). Left: spatial trace obtained with FFT and theoretical Matérn covariance (ν = α S -1 = 1). Right: temporal trace obtained with FFT and theoretical exponential covariance.

Figure 4 . 5 :

 45 Figure 4.5: Covariances of stationary spatial trace of nonseparable spatiotemporal model (α S = 0, κ = 0.5, τ = 1, c = 1, γ = 0) computed with FEM/FDM ("discretized"), FFT and theoretical Matérn. Left: α = 2, hence ν = 1. Right: α = 3, hence ν = 2.

Figure 4 . 6 :

 46 Figure 4.6: Experimental spatial covariance over 4 following time steps of the simulation of advection-diffusion SPDE for κ = 0.3, τ = 1, c = 1, α = 1 and γ = [0, 1] .

Figure 4 . 7 :

 47 Figure 4.7: Experimental spatial covariance over 4 following time steps of the simulation of diffusion SPDE for κ = 0.33, α = 1, τ = 1 and c = 1, shifted at each time step of dx = 1 on the x-axis.

Figure 4 . 8 :

 48 Figure 4.8: Advection-dominated SPDE. Spatial trace at 4 time steps (separated by dt = 1) of the simulation of advection-diffusion SPDE for α = 1, α S = 0, κ = 0.3, τ = 1, c = 1, γ = [5, 0] , without stabilization.

Figure 4 . 9 :

 49 Figure 4.9: Stabilized advection-dominated SPDE. Spatial trace at 4 time steps (separated by dt = 1) of the simulation of advection-diffusion SPDE for α = 1, α S = 0, κ = 0.3, τ = 1, c = 1, γ = [5, 0] , with Streamline Diffusion stabilization.

Figure 5 . 1 :

 51 Figure 5.1: Left: Triangulated mesh T with N S = 5 2 and n obs = 5 scattered data (blue). Right: spatial projection matrix A S between scattered points and points T (size (n, N S )).

  Figure 5.1, we represent a square spatial domain [1, 2] × [1, 2], discretized in space with a triangulation T with N S = 5 2 , and n S = 5 scattered observations (in blue). In the right panel, we show the sparsity pattern of the projection matrix A S between T and the scattered data. For each row i of the matrix A S , three values are different from 0: they correspond to the three vertices of the triangle in which the i-observation is located. For example, the first and the last rows correspond to the two blue points in the bottom right part of the spatial domain. Indeed, they share two common vertices. The values of the matrix are the coefficients of a convex combination (the barycentric interpolation) of the values of the discretized field at each one of the three vertices. In Figure 5.2 we present the sparsity pattern of the matrix A S A S which is the diagonal block of the spatio-temporal matrix A A with N T = 5, when the observation locations do not change with time.

Figure 5 . 2 :

 52 Figure 5.2: Left: spatial matrix A S A S (size (N S , N S ) with N S = 5 2 ). Right: spatio-temporal block-diagonal matrix A A (size (N S N T , N S N T )with N T = 5).

  The chosen interval [a, b] must be positive. More details are available in[START_REF] Pereira | Efficient simulation of gaussian markov random fields by chebyshev polynomial approximation[END_REF].

  1 and α S = 2. The spatial domain is the [0, 30] 2 square with a grid triangulation of N S = 900 spatial points. The time window is [1, 10] with unit time step and N T = 10. The n S = 100 observations are randomly located into the spatial domain and their position do not change during the N T time steps (hence n = 1000).

Figure 6 . 1 :

 61 Figure 6.1: Stations capturing GHI over the spatial domain.

Figure 6 . 2 :

 62 Figure 6.2: GHI G and Clear Sky Index K c for 4 different stations on May 28, 2013.

Figure 6 . 3 :

 63 Figure 6.3: Histogram of K c over 20 time steps.

Figure 6 . 4 :

 64 Figure 6.4: Validation settings: Uniform (left) and South-East (right).

Figure 6 . 5 :

 65 Figure 6.5: Predictions of K c at (T + 1), (T + 2) and (T + 3) with model (3) ("adv-diff" with α = 1 and α S = 2). The black contoured dots are the conditioning locations and the white contoured dots are the validation locations for the Uniform setting.

Figure 6 . 6 :

 66 Figure 6.6: Real K c , mean of conditional simulations of K c and ±2σ envelope at time horizons T, (T + 1), (T + 2), . . . , (T + 6). Left: orange station. Right: green station.

Figure 6 .

 6 Figure 6.6 shows 100 conditional simulations of K c computed at time T = 11 and horizons (T + 1), (T + 2), . . . , (T + 6) with the advection-diffusion model (3). Two validation stations have been selected: one in the North-West part of the domain (the orange star in the left panel of Figure 6.4) and one in the South-East part of the domain (the green star in the right panel of Figure 6.4). Given that there is an advection from North-West to South-East, it is therefore expected that the advection-diffusion model should be able to transport the information. The mean of the 100 simulations and the envelopes corresponding to twice the pointwise standard deviation have also been represented, along with the true values. As expected, most

  means of several prediction techniques in[START_REF] Huang | Forecasting high-frequency spatio-temporal wind power with dimensionally reduced echo state networks[END_REF]. The choice of this dataset is due to a collaboration with Marc Genton and the Spatio-Temporal Statistics and Data Science (STSDS) research group of the King Abdullah University of Science and Technology (KAUST) during a 5-week-long scientific visit there in October-November 2022.The data are hourly high-resolution simulations from the Weather Research and Forecasting model (WRF;[START_REF] Skamarock | A description of the advanced research wrf version 3[END_REF]) over the territory of Saudi Arabia. WRF was simulated from 2013 to 2016 by[START_REF] Giani | Closing the gap between wind energy targets and implementation for emerging countries[END_REF]. The hourly wind field on a 2.15 million km 2 wide country is extremely difficult to model, due especially to the nonlinear dynamics implied by the topographic characteristics of the spatial domain (two coastal areas, mountain ranges and sandy and rocky deserts) and by the massive amount of data in space and time. Standard statistical approaches such as the autoregressive integrated moving average (ARIMA) model are effective in capturing a short-range linear dependence; machine learning approaches, such as artificial neural networks, are more adapted to capture nonlinear dynamics because of their flexibility and the possibility of defining a recursive structure (recurrent neural networks, RNNs) for temporal data. The downside of RNNs is that their inference is very challenging, due to the high dimensionality of the parameter space and the impossibility to directly apply the iterative gradient computations of traditional neural networks. To reduce the computational cost of the inference,[START_REF] Jaeger | The "echo state" approach to analysing and training recurrent neural networks-with an erratum note[END_REF] proposed the echo state networks (ESNs), which is grounded on the use of sparse, random matrices instead of the dense unknown ones of RNNs. In ESNs the inputs of the neural networks are mapped to high-dimensional hidden states through fixed nonlinear dynamics. These hidden states consider sequential linkages and thus allow for a nonlinear transformation of the input history. In order to reduce the spatial dimensionality of the ESNs by preserving the spatial variability, one could use a dimensionality reduction technique, such as the empirical orthogonal function (EOF) method (grounded on the eigenvectors of the spatial covariance matrix of the space-time data) or the principal component analysis (PCA). Another possibility to reduce the spatial dimension is to consider only a small collection of knots and to recover the entire field by spatial interpolation.[START_REF] Huang | Forecasting high-frequency spatio-temporal wind power with dimensionally reduced echo state networks[END_REF] proposed a dimensionally reduced ESN model, describing the dynamics of a spatial field through a set of knots, some of which are sampled on a fix grid and others sampled in areas with complex patterns. The spatio-temporal forecast was then obtained by using the ESN for temporal forecast and a spatial interpolation approach for spatial prediction. To reflect the spatially varying dependence structure dictated by the different topographical features in Saudi Arabia, they used a nonstationary Matérn covariance function[START_REF] Paciorek | Spatial modelling using a new class of nonstationary covariance functions[END_REF] fitted to data with the R package convoSPAT[START_REF] Risser | Local likelihood estimation for covariance functions with spatially-varying parameters: The convospat package for r[END_REF].The model was compared to other approaches, including ARIMA, EOF-based ESN, Fixed Rank Kriging (FRK)[START_REF] Cressie | Fixed rank kriging for very large spatial data sets[END_REF][START_REF] Zammit-Mangion | Frk: An r package for spatial and spatio-temporal prediction with large datasets[END_REF]) and the persistence forecast, i.e., the field where the values at each spatial location at 1,2 and 3 steps ahead are set to the last known values at the same location. The ESN method proposed by[START_REF] Huang | Forecasting high-frequency spatio-temporal wind power with dimensionally reduced echo state networks[END_REF] was found to have the best prediction score (MSE) among the analyzed methods at 2 and 3 steps ahead, even though it takes into account only a subset of observations.

Figure 6 . 7 :

 67 Figure 6.7: Left: spatial locations of WRF wind simulations over the entire spatial domain. Right: topography of Saudi Arabia.

Figure 6 . 8 :

 68 Figure 6.8: Wind residuals during January 2, 2016 at 6 different spatial locations.

Figure 6 . 9 :

 69 Figure 6.9: Wind residuals at 6 different spatial locations during January 1-4, 2016.

Figure 6 . 10 :

 610 Figure 6.10: Predictions of W at (T + 1), (T + 2) and (T + 3) with model (3) ("adv-diff" with α = 1 and α S = 2).

Figure 6 . 11 :

 611 Figure 6.11: Difference between MSE of model (3) (regular mesh with N S = 900) and MSE of persistence model at (T + 1), (T + 2) and (T + 3).

Figure 6 . 12 :

 612 Figure 6.12: Difference between MSE of model (3) (handmade mesh with N S =) and MSE of persistence model at (T + 1), (T + 2) and (T + 3).

Figure 6 .

 6 Figure 6.13: Handmade mesh to capture complex topographic areas (the spatial domain is given in km × km).

  and[START_REF] Schmidt | Bayesian inference for non-stationary spatial covariance structure via spatial deformations[END_REF],[START_REF] Schmidt | Considering covariates in the covariance structure of spatial processes[END_REF][START_REF] Bornn | Modeling nonstationary processes through dimension expansion[END_REF] andFouedjio et al.

  From a technical point of view, having spatially and temporal dependent parameters only changes the elements of the precision matrix of the Markov representation of the Gaussian random field. Ingebrigtsen et al. (2014) introduced a parametric model with explanatory variables in the parameters that define the SPDE model, which results in a nonstationary spatial model. They used knowledge about the spatial process to parameterize the nonstationary dependence structure, leading to a model that can be interpreted, and they could speed up computations by sampling from the model with a fully Bayesian inference thanks to the SPDE-GMRF-INLA link.

  rameters do not vary in space inside a single triangle of the triangulation, the numerical solution of the SPDE is obtained with the same approach where the spatially varying parameters possibly change from one triangle to the other, but are constant on a single triangle. With any possible parameterization on the variables κ(s), γ(s), τ (s) and H(s), it is sufficient to know their values on each triangle and use those in the solution of the spatial sparse system (3.7).Here we report two different examples of spatio-temporal fields simulated from the SPDE with spatially varying parameters. We present a diffusion SPDE with varying κ(s) and an advection-diffusion SPDE with varying γ(s). Both simulations show the nonstationary pattern in the spatial field. In Figure7.1 we plot the spatial trace of the two spatio-temporal fields after 20 time steps of temporal simulation.

Figure 7 . 1 :

 71 Figure 7.1: Nonstationary spatial trace given by spatially varying κ(s) = s 2 /200 (left) and spatially varying γ(s) = [-0.5(s 2 -50), 0.5(s 1 -50)] (right). The other parameters are set to κ = 0.33, c = 1, τ = 1, dt = 1, h = 1.

  x 2 are 4 parameters to estimate. v 1 and v 2 define the velocity of the rotation, while x 1 and x 2 define the point (x 1 , x 2 ) on the spatial domain around which the rotation occurs. After setting the 4 parameters of the spatially varying advection vector and all the other constant parameters of the SPDE, we simulate the spatio-temporal field solution to the SPDE over the spatio-temporal domain [0, 30] × [0, 30] × [1, 10], with a triangulation of N S = 30 2 spatial points and N T = 10 time steps. We pick n S = 300 spatial locations, uniformly selected over the spatial domain and consider their values as known. We then estimate the parameters of the model from these observations and report the results of 10 different simulations in

γ 1

 1 (•) and γ 2 (•). We use the same parameters of the second rotation example, obtain a simulation with rotational advection field and estimate the parameters of the model (the constant ones and the p = 18 parameters of the B-splines basis functions) from n S = 300 observations. The constant parameters are well estimated. In Figure 7.2 we plot the two-dimensional basis made of 3 clamped B-splines on each axis on the spatial domain [0, 30] × [0, 30]. The elements γ 1 (s) and γ 2 (s) of the advection vector are represented on this basis. To determine if the advection field is well estimated, we plot the advection field recovered from the estimation with B-splines and compare it to the original one in Figure 7.3. We remark that the rotation is well retrieved, even if the center point and the intensity are slightly different.

Figure 7 . 2 :

 72 Figure 7.2: 2D clamped B-splines basis in which are represented the spatially varying parameters γ 1 (s) and γ 2 (s).

Figure 7 . 3 :

 73 Figure 7.3: Rotational advection field. Top row: simulated field (γ(s) = [0.3(s 2 -15), -0.3(s 1 -15)] ). Bottom row: estimated field with parameterization based on B-splines.

  .1) for any pair i, j = 1, . . . , p, ∀(s,s + h) ∈ R d × R d and ∀(t, t + u) ∈ R × R.The functions C ij are called direct covariance functions when i = j and cross-covariance functions otherwise. The matrix-valued covariance function C, which associates each space-time lag (h, u) with the (p, p) matrix

Theorem 8 . 1 .

 81 Let ν, b 2 and a 2 be (p, p) symmetric conditionally negative semidefinite matrices, all with positive entries, such that a 2 -ν c 0. Let u → ρ(u) be a (p, p) matrix-valued covariance function on R and let u → η(u) be a (p, p) matrix-valued pseudo-variogram on R. Denoting R

Corollary 8 . 1 .

 81 Let ν, b 2 , a 2 , η, α and β be as in Theorem 8.1. Let τ be a (p, p) symmetric real matrix. The matrix-valued function C

Figure 8 .

 8 1(left) shows the maximum log-PL as a function of the separability parameter b for the fully nonseparable Gneiting-Matérn model, referred to as FULL. The maximum is reached for b = 0.1 with log PL FULL ( θb=0.1 ) = -933 337.6.

Figure 8 . 1 :

 81 Figure 8.1: Left: PL as a function of the space-time separability parameter b. Right: (R,H) temporal correlation functions. PIT model with unique direct correlation (positive) and cross-correlation (negative) in dotted lines. FULL model with direct correlation (positive) for R and H in solid and dotdashed lines, respectively, and cross-correlation (negative) in longdashed line. Squares: empirical correlations.

  7) with b = 0.1. PIT: simplified Proportional-In-Time model with b = 0. The best scores among FULL or PIT are shown in bold font. For illustration purpose, Figure 8.3 shows the predictions of the three variables from January 3 to January 31, 2003 at one station (Le Rheu) following the Spatial Interpolation setting. The predicted residuals were back-transformed to the original scales to be compared to the real values. An envelope of ±2 conditional standard

Figure 8 . 2 :

 82 Figure 8.2: Empirical and fitted spatial-temporal correlation at u = 0 (black solid lines and circles), u = 1 (blue dashed lines and squares) and u = 2 (red dotdashed lines and triangles).

Figure 8 . 3 :

 83 Figure 8.3: Predictions of R, T and H at Le Rheu from January 3 to January 31, 2003 following the Spatial Interpolation setting. Predictions (continuous line), real values (dots) and envelope of ±2 conditional standard deviations (colored ribbon).

  particular, we examine the participation of the RESSTE team in 2021 and 2022 to the "Spatial Statistics Competition for Large Datasets" organized by the King Abdullah University of Science and Technology (KAUST) located in Thuwal, Saudi Arabia. The RESSTE team was composed of 4 researchers from Mines Paris and INRAE Avignon (Denis Allard, Thomas Romary, Thomas Opitz and myself). We present the techniques and reasoning that led the team to rank in the first positions in both the editions of the competition.

A

  short-term goal is to implement the methods in open-source softwares, in particular in the library gstlearn (Python, R, C++) developed by the Geostatistics team of Mines Paris, where most of the building blocks of the proposed method are already present, and in the library R-INLA[START_REF] Lindgren | Bayesian spatial modelling with r-inla[END_REF]. Some spatiotemporal models, such as the DEMF model described in Section 2.2.8, are already implemented in R-INLA; extending this class of models with the unsteady advectiondiffusion SPDE model would be interesting and would only require the coding of the ingredients that make our model different from DEMF (such as the FEM advection matrix, the stabilization, etc.). Linking our model to the INLA framework would benefit from all the tools already available in the library, such as the possibility of managing non-Gaussian models and Bayesian hierarchical models. Moreover, the model would be accessible to other research groups willing to model spatio-temporal datasets with advection.

  parameter estimation and prediction and compare them with those of several statistics teams worldwide, to work in a research team and to enjoy a healthy competition. It was really important for my education, because it was the first scientific project where I was a researcher at the same level as the other members of the team and where my scientific ideas counted as much as theirs. Moreover, due to the good ranking of the RESSTE team in both the editions of the competition, I had the chance to spend a visiting period of 5 weeks at KAUST, working in the STDSD research group, under the supervision of Marc Genton, and to participate in the KAUST Workshop on Statistics in November 2022. During this research visit, I worked in collaboration with Marc Genton on applications of the SPDE method to a dataset analyzed in his team (wind speed in Saudi Arabia, see Section 6.3) and on comparisons with alternative methods. I participated in the daily activities of the team (team and department seminars, workshops...) and I exchanged with researchers from the other statistics teams (Håvard Rue, David Bolin, Ying Sun, Raphaël Huser) on topics of common interest, that could possibly lead to future collaborations.

E

  -mail address: denis.allard@inrae.fr (D. Allard). 1 The author acknowledges the funding of the National Agency for Research and Development of Chile, through grants ANID/FONDECYT/REGULAR/No. 1210050 and ANID PIA AFB180004. https://doi.org/10.1016/j.spasta.2022.100706 2211-6753/© 2022 Elsevier B.V. All rights reserved.

  where Y 1 , . . . , Y p are second-order stationary random fields with direct and cross-covariance functions R : u ↦ → [R ij (u)] p i,j=1 and Y 0 is an independent intrinsic random field with unbounded variogram γ 0 . Then, the pseudo-variogram of

Fig. 1 .

 1 Fig. 1. Contour plots for the space-time covariance (15) with η(u) = √ |u|, ν = 1.5, a 2 = b 2 = 1: separability ((α, β) = (1, 1), upper left), positive nonseparability ((α, β) = (0, 1), upper right), negative nonseparability ((α, β) = (1, 0), lower left) and general nonseparability ((α, β) = (0.7, 0.3), lower right).

Fig. 2 .

 2 Fig. 2. Separability function S(h, u) computed at u = 1 (blue solid lines), u = 2 (red dashed lines) and u = 3 (green dotted lines).

D

  .Allard, L. Clarotto and X. Emery Spatial Statistics 52 (2022) 100706 In the above parameterization, the space-time nonseparability parameter b acts both inside and outside the Matérn function M, implying a constant perfect correlation in time if b = 0. FollowingGneiting (2002) andBourotte et al. (2016), a reparameterization is thus useful. Multiplying the above equation by the temporal covariance function ρij

Fig. 3 .

 3 Fig. 3. Left: PL as a function of the space-time separability parameter b. Right: (R, H) temporal correlation functions. PIT model with unique direct correlation (positive) and cross-correlation (negative) in dotted lines. FULL model with direct correlation (positive) for R and H in solid and dotdashed lines, respectively, and cross-correlation (negative) in longdashed line. Squares: empirical correlations.

Fig. 4 .

 4 Fig. 4. Empirical and fitted spatial-temporal correlation at u = 0 (black solid lines and circles), u = 1 (blue dashed lines and squares) and u = 2 (red dotdashed lines and triangles).

Fig. 5 .

 5 Fig. 5. Predictions of R, T and H at Le Rheu from January 3rd to January 31st 2003 following the Spatial Interpolation setting. Predictions (continuous line), real values (dots) and envelope of ±2 conditional standard deviations (colored ribbon).

Algorithm 3 Property 3 .

 33 Spectral simulation algorithm Require: Admissible parameters for ν, b 2 and a 2 Require: Admissible pseudo-variogram η Require: A probability density function g not vanishing on R d Require: A large integer M 1: Initialize Y (s k ) = 0 for each target spatial coordinate s k , k = 1, . . . , n S 2: for m = 1 to M do integer Q uniformly distributed in {1, . . . , q} 7: for each target spatial coordinate s k do 8:ComputeY (s k ) = Y (s k ) + √ q M H (Ω ; Q ) cos(2π ⟨Ω, s k ⟩ + Φ) where H (Ω ; Q ) denotes the Q -th column of H (Ω )9: end for 10: end for 11: Z := re-ordering of Y into a p-variate random field at n S × n T space-time coordinates Appendix D. Technical lemmas and proofs D.1. Technical lemmas Lemma 1. Let a = [a ij ] p i,j=1 be a real, symmetric conditionally negative semidefinite matrix. Then, there exist random variables Y 1 , . . . , Y p such that: i -Y j ), i, j = 1, . . . , p. Proof. The matrix b = [b ij ] p i,j=1 with b ij = a ip + a pj -a ij -a pp is positive semidefinite (Berg et al., 1984, Chapter 3, Lemma 2.1). Let Y 1 , . . . , Y p be Gaussian random variables with variance-covariance matrix b. Then 1 2 Var(Y i -Y j ) = b ii + b jj 2 -b ij = a ij -a ii + a jj 2 , i,j = 1, . . . , p. □ Property 1. A matrix is conditionally null definite if and only if it is a-separable. Property 2. All p-separable matrices with nonnegative entries are positive semidefinite. If a ⪯ c 0, then exp (-ta) ⪰ 0 for all t ≥ 0.Property 4. If a ⪯ c 0 and b is a-separable, with a and b of size p × p, then a + b ⪯ c 0 and ab ⪯ c 0.

  i λ n,j C i,j;m,n .

Figure 1 .

 1 Figure 1. Histograms of the datasets of competition 2 with superimposed Gaussian density when adequate .

  Then, Section 8.2 describes the contribution to the "Spatial Competition for Large Datasets" proposed by KAUST in 2021 and 2022.Finally, Chapter 9 presents the conclusions and the perspectives of the PhD.Several possible research directions are explored.

	Chapter 2
	Spatio-temporal SPDE models

8.1 we enlarge the Gneiting class of space-time covariance functions by introducing a very general parametric class of fully nonseparable direct and cross-covariance functions for multivariate random fields. Résumé Dans ce chapitre, nous introduisons un cadre mathématique minimal permettant d'étudier les modèles spatio-temporels qui sont solutions d'Equations aux Dérivées Partielles Stochastiques (EDPS). Nous commençons par introduire les principales classes d'EDPS spatio-temporelles dévéloppées dans la littérature statistique. Nous présentons ensuite la classe d'EDPS d'advection-diffusion en étudiant ses caractéristiques statistiques et en la comparant aux EDPS proposées précedemment.

  ). Carrizo-[START_REF] Carrizo-Vergara | A general framework for SPDE-based stationary random fields[END_REF] established that, under mild conditions, the covariance of the stationary solution to a given SPDE

with a general random source term with covariance C Z is the convolution between the covariance of the same SPDE with white noise source term and C Z . This result is a powerful tool for easily characterizing solutions of very general SPDEs. The space-time white noise W (s, t) is characterized by its spectral density S W (ξ, ω) = (2π)

.5 Damped Heat Equation (Krainski, 2018)

  One can only modulate how much comes from time and space by the parameter γ t .

	The symbol function is
			(ξ, ω) → [γ t iω + ξ 2 ].
	Hence, the stochastic heat equation defines an intrinsic random field. Equation (2.14)
	does not respect the uniqueness condition, since g(ξ) = ξ 2 has a zero at the origin
	(see Remark 2.2.1). However, there exist solutions to the Stochastic Heat Equation
	for spatial dimensions d ≥ 3, which can only be conceived as GeRFs and never as
	random functions continuous in mean-square (Carrizo-Vergara et al., 2022).
	2.2Krainski (2018) proposed the Damped Heat Equation, which can be obtained as a
	special case of the Jones and Zhang model (2.8) with α = 2 or as a generalization of
	the Stochastic Heat Equation (2.14) with the adding of the damping parameter κ 2 .
				and Carrizo-
	Vergara et al. (2022) and can be deduced from the Jones and Zhang model (2.8) with
	the absence of the damping parameter (κ = 0) and a spatial smoothness parameter
	equal to α = 2. It comes from the physical equation of heat, well known in the PDE
	literature (see e.g. Quarteroni (2008)). It writes
	γ t	∂ ∂t	-∆ X(s, t) = τ W (s, t).	(2.14)

Evolution Equation ADVECTION- DIFFUSION Stochastic Heat Equation Damped Heat Equation Jones and Zhang Iterated Heat Equation Stein DEMF Evolving Matérn

	Figure 2.1: Venn Diagram of nonseparable SPDE models proposed in the
	literature.

H ∇ is a diffusion term that can incorporate anisotropy in the matrix H. When the field is isotropic, i.e. when H = λ I, this term reduces to the Laplacian operator λ∆;

	t),	(2.27)
	where
	• the operator ∇ •

In the following |H| denotes the determinant of the square matrix H. Proposition 2.1. Let

			The advection-diffusion equation
	(2.27) is a particular first order evolution model as in Equation (2.6) with β = 1. Its
	spatial symbol function
	g(ξ) =	1 c	(κ 2 + ξ H ξ) α + i γ ξ ,
	verifies the sufficient condition for existence and uniqueness of a stationary solution
	(SCEU) recalled in Section 2.2. Carrizo-Vergara et al. (2022) showed that the ad-
	vection term does not affect the spatial trace X(•, t) of the solution (see also Section
	2.3.3). For some specific values of the parameters, the spatial trace of the solution to
	(2.27) is a Matérn field, as detailed in Proposition 2.1. Z(s, t) be a spatio-temporal noise colored in space with Z S (s)
	satisfying (2.28), and let α tot = α + α S . If α tot > d/2, the spatial trace of the
	stationary solution X(s, t) of the SPDE (2.27) is the Gaussian Matérn field with
	covariance

Chapter 3 Discretization of spatio-temporal SPDEs

	Résumé
	Dans ce chapitre nous décrivons les approches de discrétisation usuelles pour les
	EDP et les EDPS, puis nous présentons l'approche choisie pour la classe d'EDPS
	t), spatio-temporelles d'advection-diffusion. Nous discutons la notion de domination
	d'advection et proposons des méthodes de stabilisation pour les EDPS qui présentent
	where W 2 (s, t) is a shifted form of W (s, t), but still a white noise with the same cette caractéristique. Enfin, nous présentons la forme approchée du champ aléatoire
	covariance structure. For this reason, the solutions X(s, t) and X γ (s, t) have the Gaussien de Markov et la structure creuse de sa matrice de précision.
	same spatial trace and the spatio-temporal structure is influenced only by a shift
	term depending on γ. If we wanted to simulate the solution X(s, t), we could simulate
	the solution X γ (s, t) of the SPDE without advection and shift it, by ensuring that
	the domain is shifted too. This property will be illustrated in Chapter 4 through
	simulations.

  Table 4.1: Different diffusive models obtained with combinations of α and α S , along with corresponding marginal spatial variance and practical range. In Figures 4.2 and 4.3 we plot the experimental spatio-temporal variograms of the simulations of the diffusive SPDE models with the parameters of Table 4.1. The plots show that the marginal variance is slightly underestimated.

	and (B) are separable models.
	Model name α α S	σ 2	r =	√	8ν/κ
	(A)	0	2	0.442	9.428
	(B)	0	4 18.193	16.330
	(C)	1	0	/		/
	(D)	2	0	0.442	9.428
	(E)	1	2	2.456	13.333
	(F)	2	2 18.193	16.330

  3.3).

	time steps	t1,t1	t1,t2	t1,t3	t1,t4

mean (std) ×10 -3 1.19 (4.54) -0.41 (4.91) 0.86 (3.61) -1.60 (4.12) Table

4

.2: Mean (std) ×10 -3 of differences between experimental spatial covariances over 4 following time steps with shifted diffusion SPDE and corresponding advection-diffusion SPDE.

Table 5

 5

	method	κ	γ 1	γ 2	c	τ	average time (s)
	correct	0.5	2	2	1	1
	Cholesky	0.610 (0.047) 2.354 (0.515) 2.325 (0.421) 1.037 (0.218) 1.072 (0.040)	194
	Matrix-free 0.483 (0.029) 1.904 (0.178) 1.906 (0.147) 1.027 (0.046) 0.998 (0.024)	960
	correct	0.7	1	-1	2	0.5
	Cholesky	0.695 (0.067) 1.056 (0.659) -1.134 (0.631) 2.090 (0.352) 0.503 (0.022)	172
	Matrix-free 0.669 (0.052) 0.967 (0.123) -1.164 (0.142) 1.954 (0.118) 0.485 (0.013)	863

.1: Mean (and standard deviation) of ML estimates θ = [κ, γ1 , γ2 , ĉ, τ ] over 50 simulations for two different subsets of advectiondiffusion model parameters and two different estimation approaches (Cholesky decomposition and matrix-free approach).

temporelle à court terme de données de radiation solaire et de vitesse de vent. Nous montrons la flexibilité de la méthode et son interprétation statistique et physique.

  (ii) a nonseparable model whose spatial trace has no known closed form expression for the covariance function (the closed form exists only if α + α S > d/2, where d = 2 in this case, see Proposition 2.1); (iii) a nonseparable model with a Matérn spatial trace with ν = 2. In the general model of Equation (2.27) they correspond respectively to (α, α S ) = (0, 2), (1, 0), (1, 2). The parameters of the SPDE are estimated for each model separately. The results are reported in Table6.1.

with diffusion included only in the stochastic forcing term, with a Matérn spatial trace with ν = 1;

  with different combinations of α and α S , with or without advection, and N S = 40 2 . The best score among the models is in bold font.

	1) adv-diff 0	2	0.312 0.526 0.738
	(2) adv-diff 1	0	0.309 0.521 0.719
	(3) adv-diff 1	2	0.311 0.519 0.724
	(4)	diff	0	2	0.312 0.533 0.749
	(5)	diff	1	0	0.310 0.528 0.738
	(6)	diff	1	2	0.314 0.528 0.734
	(7) persistence		0.244 0.544 0.806
	(8)	FRK			0.420 0.546 0.696
	Table 6.3: Average MSE at (T+1), (T+2), (T+3) from n S = 53333
	observations during 24 hours. Persistence, FRK, and various SPDE models
	(2.27) Model α α S		N S	T+1	T+2	T+3
	adv-diff 1	2		40 2	0.311 0.519 0.724
	adv-diff 1	2		50 2	0.293 0.512 0.723
	adv-diff 1	2		60 2	0.281 0.507 0.721
	adv-diff 1	2 3273 (custom-made) 0.256 0.506 0.732

Table 6.4: Average MSE at (T+1), (T+2), (T+3) from n S = 53333 observations during 24 hours. Advection-diffusion SPDE model (2.27) with α = 1 and α S = 2, and N S = 40 2 , N S = 50 2 , N S = 60 2 and custom-made mesh. The best score among the different meshes is in bold font.

Table 7

 7

		v1	v2	x1	x2	κ	c	τ	avg time (s)
	true	0.5	0.5	12	18	0.7	1.5	1
	estimated 0.570 (0.057) 0.517 (0.072) 12.198 (0.534) 18.403 (0.688) 0.824 (0.060) 1.601 (0.160) 1.043 (0.036)	254
	true	-0.3	-0.3	15	15	0.5	2	1
	estimated -0.323 (0.038) -0.311 (0.044) 14.942 (0.946) 15.386 (0.915) 0.695 (0.048) 2.139 (0.164) 1.031 (0.029)	257
		Table 7.1: Mean (and standard deviation) of ML estimates θ =
		(κ, v1 , v2 , x1 , x2 , ĉ, τ ) over 10 simulations for two different subsets of non-
		stationary advection-diffusion model parameters (rotational advection).

.1. We remark that the parameters are pretty well estimated; only the parameter κ is slightly overestimated. This means that the model is able to capture the rotational nonstationary dependence structure.

  co-authored with Denis Allard and Xavier Emery during my PhD, and available as Supplementary Material in S.1. We broaden the well-known Gneiting class of space-time covariance functions by introducing a very general parametric class of fully nonseparable direct and cross-covariance functions for multivariate random fields, where each component has a spatial covariance function from the Matérn family with its own smoothness and scale parameters and, unlike all currently available models, its own correlation function in time. The application of the proposed model is illustrated on a weather trivariate dataset over France. Our new model yields better fitting and better predictive scores compared to a more parsimonious model with common temporal correlation function. In particular, I participated in the construction of the estimation technique, in the coding of the application of the proposed approach to real data, in the general discussions with the co-authors and in the final editing of the paper.

Table 8 .

 8

1: PL parameter estimates for the weather dataset with d m = 250 km, t m = 2 days. FULL: fully nonseparable model as in (8.7) with b = 0.1. PIT: simplified Proportional-In-Time model with b = 0. The indices 1, 2 and 3 refer to R, T and H, respectively.

.746 0.582 0.864 0.587 0.464 0.732 0.427 0.335 0.509 0.268 0.015 0.393

			RMSE			MAE		CRPS			LogS
		R	T	H	R	T	H	R	T	H	R	T	H
						Spatial interpolation
	FULL 0PIT 0.751 0.584 0.864 0.593 0.462 0.731 0.430 0.335 0.509 0.271 0.010 0.393
						Temporal prediction
	FULL 0.919 0.837 0.813 0.784 0.706 0.667 0.531 0.482 0.470 0.421 0.329 0.311
	PIT	0.933 0.835 0.825 0.780 0.714 0.684 0.537 0.483 0.479 0.437 0.331 0.330
						Trivial interpolation
		0.933 0.893 0.919 0.801 0.777 0.782 0.540 0.521 0.538 0.439 0.403 0.443

Table 8 . 2 :

 82 Prediction scores at the validation stations at day t, using ML estimates. FULL: fully nonseparable model as in(8.

  This work allowed me to see in practice how important it is to find a good trade-off between the flexibility of a multivariate spatio-temporal model, which must be able to account for the complex interactions between space and time across several variables, and the number of parameters of the model, which should remain interpretable and not too numerous. A large number of parameters results in a time-consuming estimation, which often needs to be tackled with approximate techniques, such as the pairwise likelihood method that we used in our application. More efficient methods for parameter estimation probably exist (see for example the recent approaches by Lenzi et al. (2023) and Sainsbury-Dale et al. (2023)): finding the best one for this type of model would inevitably lead to faster computations and possible applications to larger datasets. In general, we should bear in mind that a more complex model should only be preferred if it is supported by the data. Participating in the "Spatial Statistics Competition for Large Datasets" proposed by the King Abdullah University of Science and Technology (KAUST) in 2021 and 2022 (see Section 8.2) was a great way to get to know the current trends in spatial and spatio-temporal statistics for massive datasets, to propose approximate methods for

	The contributions detailed in Chapter 8 enlarge the view on spatio-temporal mod-
	eling and prediction. The class of multivariate space-time covariance functions that
	we have proposed in Allard et al. (2022) (see Section 8.1 and Supplementary Material
	S.1) allows a generalization of the Gneiting class of space-time covariance functions
	to fully nonseparable functions, i.e., functions that are space-time nonseparable and
	that are neither proportional-in-space nor proportional-in-time. This is a major im-
	provement to most of the models proposed up to date, where the marginal temporal
	correlation function is identical for all the variables, which is an overly restrictive
	hypothesis. The model we have proposed is continuous over R d × R, hence it is not
	restricted to a regular design of observations. More complex designs are in principle
	possible, such as irregular or incomplete records, heterotopic designs or even mobile
	measurement devices. We hope to see applications of our model to such datasets in
	a near future.

  Building valid unbounded matrix-valued pseudo-variograms with different diagonal entries (direct variograms) is still an open question. As a simple model for this, we propose the following construction, which is a straightforward generalization of the bivariate construction of[START_REF] Oesting | Statistical post-processing of forecasts for extremes using bivariate Brown-Resnick processes with an application to wind gusts[END_REF]. Let us define W i

	D. Allard, L. Clarotto and X. Emery	Spatial Statistics 52 (2022) 100706

)

is positive semidefinite in R d × R if the matrix τe -ν /Γ (ν) is positive semidefinite.

Table 1

 1 PL parameter estimates for the weather dataset with d m = 250 km, t m = 2 days. FULL: fully nonseparable model as in (17) with b = 0.1. PIT: simplified Proportional-In-Time model with b = 0. The indices 1, 2 and 3 refer to R, T and H,

	respectively.
		Temporal parameters
		c	a	r	λ	δ	A 1	A 2	A 3
	FULL	0.528	0.993	0.341	0.0	1.000	0.871	0.608	0.646
	PIT	4.31	0.997	N/A	N/A	0.638	0	0	0
		Spatial parameters
		a 1	a 2	a 3	ν 1	ν 2	ν 3
	FULL	15.8 10 -3	6.4 10 -3	12.4 10 -3	3.76	0.69	0.51
	PIT	18.9 10 -3	8.2 10 -3	17.0 10 -3	1.61	0.71	0.53
		Correlation parameters						PL
		ρ 12	ρ 13	ρ 23
	FULL	-0.124	-0.330	-0.099					-933 337.6
	PIT	-0.151	-0.324	-0.103					-933 961.3

Table 2

 2 100706 MSE, MAE, CRPS and LogS of predicted values. ''FULL'': fully nonseparable model as in (17) with b = 0.1 and ML estimators. ''PIT'': simplified Proportional-In-Time model with b = 0 and ML estimators. ''All 0s'': trivial prediction with 0. The best scores among FULL or PIT are shown in bold font.

		MSE			MAE			CRPS			LogS
		R	T	H	R	T	H	R	T	H	R	T	H
		Spatial interpolation
	FULL	0.594	0.482	0.814	0.574	0.508	0.714	0.412	0.371	0.505	0.188	0.091	0.382
	PIT	0.585	0.490	0.814	0.570	0.508	0.716	0.409	0.371	0.505	0.157	0.092	0.381
		Temporal prediction
	FULL	0.903	0.757	0.708	0.790	0.705	0.673	0.542	0.493	0.475	0.449	0.361	0.329
	PIT	0.943	0.763	0.722	0.795	0.717	0.684	0.553	0.497	0.481	0.474	0.367	0.342
		Trivial prediction
	All 0s	0.923	0.923	0.923	0.805	0.801	0.783	-	-	-	-	-	-
		Estimation standard deviations for FULL
		0.0194	0.0176	0.0173	0.0091	0.0088	0.0087	-	-	-	-	-	-

100706

  Algorithm 1 Substitution simulation algorithm Require: Admissible Matérn or Cauchy parameters as given in Theorem 1 Require: Admissible pseudo-variogram η Require: A probability density function f on (0, ∞) whose support contains the supports of f 1 , . . . , f p Require: A large integer M 1: Initialize Z i (s, t) = 0, ∀i and at each target space-time coordinate (s, t) ∈ R d × R 2: for m = 1 to M do At each target time coordinate t, simulate a p-variate random field W = [W i ] i=1,...,p with stationary Gaussian direct and cross-increments, with zero mean and pseudo-variogram η 8:for each target space-time coordinate (s, t) do(s, t) = Z i (s, t) + Admissible pseudo-variogram η Require: Set of time coordinates t 1 , . . . , t n T targeted for simulation 1: Calculate the (p n T ) × (p n T ) variance-covariance matrix(Papritz et al., 1993, Eq. 6)C W = [[η i1 (t m ) + η 1j (-t n ) -η ij (t m -t n )] p i,j=1 ]

	3:	Simulate R ∼ f
	4:	Simulate Ω ∼ N d (0, I d )
	5:	Simulate Φ ∼ U (0, 2π )
	6:	Simulate V ∼ N p (0, σ)
	7:
	9:	Compute
			Z i √	2 M	√	f i (R) f (R)	V i cos	(	√	2R ⟨Ω, s⟩ +	|Ω| √ 2	W i (t) + Φ	)	, i = 1, . . . , p
	10:	end for
	11: end for
	Algorithm 2 Matrix decomposition algorithm
	Require: n T
													m,n=1
					n T n=1 = C	1/2 W X , with C	1/2
	[ [ν ij ] p i,j=1	] n T m,n=1	and	[ [σ ij (u mn )] p i,j=1							i,j=1	] n T m,n=1	,

2: Simulate X ∼ N p n T (0, I p n T ) 3: Compute W = [[W i (t n )] p i=1 ] W the

positive semidefinite square root of C W Y at n S nodes of R d with a multivariate Matérn covariance with parameters [ [r ij (u mn )] p

  and Theorem 2.2). Then, for any positive integer n T and real values t 1 , . . . , t n T , the matrixC 0 = [[C ij (t m -t n )] p i,j=1] Fully nonseparable Gneiting covariance functions for multivariate ST data 159 D. Allard, L. Clarotto and X. Emery Spatial Statistics 52 (2022) 100706 Proof. It is immediate because C 0 is the variance-covariance matrix of the random vector [Y (t n )] Let a = [a ij ] p i,j=1 be a conditionally negative semidefinite matrix and η : u ↦ → [η ij (u)] p i,j=1 , u ∈ R, be the matrix-valued pseudo-variogram (dependent only on the lag vector u) of a multivariate random field Y = [Y i ] p i=1 in R with stationary direct and cross-increments. Then, for any positive integer n T and real values t 1 , . . . , t n T , the matrix η 0 = [[a ij + η ij (t m -t n )] p i,j=1 ] , i = 1, . . . , p and C i,j;m,n = Cov(Y i (t m ) -Y 1 (t 1 ), Y j (t n ) -Y 1 (t 1 )), i, j = 1, . . . , p, m, n = 1, . . . , n T . Y 1 (t 1 ) + Y 1 (t 1 ) -Y j (t n ))]

	n T n=1 . n T m,n=1 is conditionally negative
	semidefinite.
	Proof. Let Λ = [[λ m,i ] p i=1 ] C0 = [[C i,j;m,n ] p i,j=1 ] n T m,n=1 with n T m=1 be a real vector with entries adding to zero. Define Λ = [ λi ] p i=1 and
			n T
	λi = λ m,i Then: ∑ m=1
	n T		n T		p	p
	∑	∑	∑	∑
	m=1	n=1	i=1
			n T		n T	p	p
	=	∑	∑	∑	∑
		m=1	n=1	i=1
			n T		n T	p	p
	=	∑	∑	∑	∑
		m=1	n=1	i=1

□ Lemma 2. Let C : u ↦ → [C ij (u)] p i,j=1 , u ∈ R, be the matrix-valued covariance function of a p-variate second-order stationary random field Y in R. n T m,n=1 is positive semidefinite. 20 S.1. □ Lemma 3. j=1 λ m,i λ n,j [a ij + η ij (t m -t n )] j=1 λ m,i λ n,j [a ij + 1 2 Var(Y i (t m ) -j=1 λ m,i λ n,j [a ij + η i1 (t m -t 1 ) + η j1 (t n -t 1 ) -Cov(Y i (t m ) -Y 1 (t 1 ), Y j (t n ) -Y 1 (t 1 ))]

Table 1 .

 1 Mean, minimal and maximal value of the datasets of competition 2

	Dataset	Min	Mean	Max
	2a1	-7.1426	1.5141	36.0287
	2a2	-7.3185	2.2249	126.789
	2b1	-3.2676	0.0652	4.1168
	2b2	-7.2361	1.504	37.1778

© 2022 Elsevier B.V. All rights reserved.

https://informatique-mia.inrae.fr/reseau-resste/

S.2. Discussion on "Competition on Spatial Statistics for Large Datasets"

Acknowledgements

ACKNOWLEDGEMENTS

We would like to warmly thank several colleagues who have helped to improve and implement our prediction approach through discussions about methods and through technical support: N. Desassis, X. Freulon, L. Houde, M. Pereira and D. Renard. [Received June 2021. Revised June 2021. Accepted July 2021.]

List of Abbreviations

Table 6.2: Averaged RMSE computed at 10 successive time steps for 6 different models, 2 validation settings (Uniform and South-East) and 3 prediction configurations (S, T and ST); see text for details. In each case, the best score among the models is in bold font.

Discussion

A reliable and fast short-term prediction of solar radiation is not only interesting for environmental research, but also for the management of the electrical network in the Estimating the parameters of a parametric model such as the one in Section 8.1.3 is a challenge. Here, we have used Pairwise Likelihood because it was proven efficient on the proportional-in-time model in Bourotte et al. (2016). It is however relatively slow: reaching convergence with our optimization algorithm and the function nlmbinb takes typically several hours on a recent laptop computer. There is thus certainly room for improvement for an easier use of such complex models. At the cost of even larger computing times, composite likelihood with larger sets of elements than pairs and full likelihood might be interesting options for a more accurate estimation of the parameters, considering the complex interactions between space, time and the variables. Conducting a thorough comparison of several likelihood approaches was out of the scope of this work and left for future research. In our opinion, more interesting improvements can be expected by using Vecchia Gaussian process approximation (Katzfuss and Guinness, 2021). We are looking forward to (multivariate) spatiotemporal versions of the GpGp package.

All the models presented in this work can easily be generalized to R d × R , where is an integer greater than one. Other constructions based on characterizations established in Porcu et al. (2022) are also possible. Extensions of our models to other spaces that are of interest in environmental sciences, atmospheric sciences and geosciences, such as a sphere crossed with an Euclidean space (S d × R ), are left to further research.

h to consider random processes with medium and strong departure from Gaussianity.

RMSE was used to evaluate the prediction accuracy in the sub-competitions 2a and 2b.

To process these data, we first considered several methods, either from more classical geostatistical analyses or from machine learning techniques. We recognized the data distribution, as being generated by the Tukey g-and-h random fields. We thus transformed the data into Gaussian data by estimating the parameters of the Tukey g-and-h transformation and by using the inverse transform, we then estimated the parameters of the Gaussian model and finally obtained the predictions by kriging before transforming the data back with the direct transform. The estimation of the parameters and the prediction were done with the R package GpGp which uses the Vecchia approximations [START_REF] Vecchia | Estimation and model identification for continuous spatial processes[END_REF]. The principle of these approximations is similar to that implemented in the composite likelihood methods. The computations are based on the Cholesky factor of the inverse covariance matrix which can be computed explicitly and is sparse by construction. They therefore allow numerically efficient inference and prediction. The GpGp library provides an implementation of the Vecchia approximation that uses an elaborate method for ordering and clustering data into conditionally independent blocks (Guinness, 2018;[START_REF] Katzfuss | GPvecchia: Fast Gaussian-process inference using Vecchia approximations[END_REF].

We obtained the best predictive scores in both the sub-competitions 2a and 2b.

Full details on the methods that we used are provided in the Discussion on "Competition on Spatial Statistics for Large Datasets" [START_REF] Allard | Discussion on "competition on spatial statistics for large datasets[END_REF], available as Supplementary Material in S.2.

2022 Edition

The 2022 edition of the "Spatial Statistics Competition for Large Datasets" featured 6 sub-competitions. The objective was to evaluate existing approximate prediction methods on large univariate and multivariate spatial and spatio-temporal datasets.

A collection of large synthetic datasets (100K/1M data) was generated from a set of real models (2 univariate models, 2 spatio-temporal models, 2 bivariate models) using

ExaGeoStat and each dataset was split into training and validation sets. The models exhibited spatial nonstationarity. All the details on the datasets and on the scoring rules, and the main methods used by the participants, are available in [START_REF] Abdulah | The second competition on spatial statistics for large datasets[END_REF].

We already know that the computation of the log-likelihood is prohibitive for large sample size N S of spatial data, as the complexity of computing the inverse of the covariance matrix is O(N 3 S ) and requires O(N 2 S ) memory. This computation becomes even more expensive when considering the multivariate and spatio-temporal case because the sample size becomes pN S N T , where p is the number of spatial variables and N T is the number of time points.

We participated in the sub-competitions 1a and 1b (univariate spatial data): we used the R package GpGp described in the previous section for parameter estimation and spatial prediction. We noted that the nonstationary structure changed from one

Appendices

A.1 Implementation

We here sketch the main implementation features of the methodology proposed in the dissertation. The main points of the implementation concern the following procedures:

• Simulation of (S)PDE models (including the unsteady advection-diffusion model with and without stabilization);

• Estimation of SPDE model parameters;

• Prediction and conditional simulations of spatio-temporal SPDE models applied to real data;

• Nonstationary extension of the approach. Reproducible scripts will be made available at http://github.com/luciclar/ ST_advdiff_SPDE by October 2023.

A.1.1 Simulation

As detailed in Chapter 4, the simulation of SPDE models is based on the discretization approach of Section 3.3. The discretization based on the FEM in space and the FDM in time is obtained by coding the implicit Euler scheme directly in Python and by approximately solving the purely spatial SPDE at each time step by finite elements. This last task is made using the FEniCS software [START_REF] Logg | Automated Solution of Differential Equations by the Finite Element Method[END_REF][START_REF] Alnaes | The FEniCS project version 1.5[END_REF]. FEnICS is an open-source software that allows for automated ways of implementing the FEM code for partial differential equations from the variational formulation of a differential equation. Using the FEnICS platform in Python we implement the spatial (S)PDE by writing the weak form detailed in Equation (3.11). We set the parameters of the SPDE and the parameters of the numerical approximation (size of the spatial domain and of the time window, mesh structure, mesh size h, boundary conditions). The FEM matrices (mass matrix M, lumped mass matrix M, stiffness matrix G (isotropic or anisotropic), advection matrix B and Streamline Diffusion stabilization matrix S, defined in Section 3.3) are computed by FEnICS and stored as sparse matrices with the package scipy.sparse.coo_matrix. The rest of the implementation is made in Python, by using the sparse matrices retrieved from FEnICS. For the noise term it is sufficient to sample a Gaussian standard vector and to apply the corresponding multiplicative matrix. The LU decomposition of the sparse matrix in the left-hand side of system (3.7) is made using the function scipy.sparse.linalg.splu. Concerning the sparse global precision matrix Q defined in Equation (3.22), either we store it in the default Python sparse format (not necessarily the most convenient format in our specific case) or we store each of its building blocks (3 different blocks for the diagonal and 1 block for the upper and lower diagonal) in a sparse format itself.

Simulations of a spatio-temporal field, solution to the SPDE, are obtained by setting the parameters of the SPDE to estimated or chosen values. The features of the covariance structures of those fields are studied by means of comparisons with known covariance functions (the Matérn, the exponential) and with the Fourier basis functions approximation, detailed in Section 4.2 and coded entirely in Python.

A.1.2 Estimation

The estimation approach proposed in Chapter 5 is implemented in Python by coding the different functions necessary to compute the terms of the log-likelihood function.

First, the projection matrix A is created from the mesh defined in FEnICS and the locations of the data points.

The matrix-free methods (matrix-vector multiplication for block-structured matrices, temporal-block Gauss-Seidel preconditioner, Chebyschev approximation of the logarithm function, Hutchinson's estimator of the trace) are coded from scratch. The matrix-vector multiplication for block-structured matrices and the temporal-block Gauss-Seidel preconditioner are coded using the functools.partial function and by coding the single operations between each block in a custom-made function. Then, both matrix-free methods are transformed to the LinearOperator format by using the function scipy.sparse.linalg.LinearOperator. The Conjugate Gradient method is available in the function scipy.sparse.linalg.cg, where the matrix-free product and the matrix-free preconditioner are input in the LinearOperator format.

The code allows switching from the direct matrix implementation based on the Cholesky decomposition of the spatio-temporal matrices (obtained with the function scipy.sparse.linalg.cholesky) to the matrix-free approach, when the spatiotemporal mesh is too large for the Cholesky decomposition. The maximization of the log-likelihood is made with the scipy.optimize package with the BFGS method.

The variogram fit for defining the initial values of the simulation study is obtained with the Python library gstlearn.

A.1.3 Prediction and application to real dataset

The spatial meshes are constructed using the FEniCS implemented mesh function, either in the default mode or by editing it to obtain more complex meshes.

The ML optimization is obtained by using the functions detailed in the previous section.

The kriging and the conditional simulations are coded in Python by following the schemes detailed in Section 5.3 and 5.4.

The plots are made with matplotlib.

A.1.4 Nonstationarity

The simulation approach in the nonstationary extension of the SPDE model is obtained by including spatially varying parameters in the SPDE with the Expression function available in FEniCS. The corresponding spatial FEM matrices are then obtained.

The estimation approach is similar to the one detailed above, but includes more parameters in the case of the parameterization of the advection vector with a defined function of with the B-splines basis. The basic B-splines are computed by scipy.interpolate.BSpline. The 2D B-splines basis is custom-made.

Supplementary material

S.1 Fully nonseparable Gneiting covariance functions for multivariate space-time data D. Allard, L. Clarotto and X. Emery Spatial Statistics 52 (2022)

null definite matrix is both conditionally positive and conditionally negative semidefinite. An addition separable (a-separable) matrix is such that 2a ij = a ii +a jj for all i, j = 1, . . . , p, and a product separable (p-separable) matrix is such that a 2 ij = a ii a jj for all i, j = 1, . . . , p. It is straightforward to prove that any a-separable matrix is conditionally null definite and that any p-separable matrix with nonnegative entries is positive semidefinite, see Appendix D.1.

Matrix-valued space-time covariance functions

We consider a multivariate space-time random field

where d is the space dimension and p the number of random field components, each being realvalued. A space-time coordinate will be denoted (s, t), with s ∈ R d and t ∈ R. For the sake of light notation, the Euclidean norm of h will be denoted |h|. Without loss of generality, we assume that all random field components are centered, i.e.,

will also be assumed that the multivariate random field Z (s, t) is second-order stationary, so that its covariance functions exist and depend only on the space-time lag (h,

Delfiner, 2012):

for any pair i, j = 1, . . . , p, ∀(s,

The functions C ij are called direct covariance functions when i = j and cross-covariance functions otherwise.

The matrix-valued covariance function C , which associates each space-time lag (h, u) with the

is positive semidefinite [START_REF] Wackernagel | Multivariate Geostatistics: An Introduction with Applications[END_REF]. Setting u = 0 in (1) defines the purely spatial matrix-valued covariance function C S (h) := C (h, 0). Likewise, C T (u) := C (0, u) is a purely temporal matrix-valued covariance function.

A matrix-valued space-time covariance function is space-time separable if it is the elementwise product of a matrix-valued spatial covariance function with a matrix-valued temporal covariance function, i.e., when

u) for all i, j = 1, . . . , p. The direct covariance C ii is said to be positively (resp. negatively) space-time nonseparable if S ii (h, u) and Diggle, 2010;De Iaco and Posa, 2013). The functions S ii (h, u) will be referred to as separability functions. Their empirical versions can be used as exploratory tools to characterize the type of nonseparability in a dataset.

A matrix-valued space-time covariance function is said to be a proportional model when it is obtained as the product of a p × p covariance matrix R and a univariate space-time correlation function ρ, i.e., Bourotte et al. (2016) proposed nonproportional matrix-valued space-time covariance functions in which the purely spatial part is a nonproportional model but the purely temporal part is a proportional model, i.e., the model is proportional-in-time but not in space. In this work, we will build valid and flexible parametric classes of matrix-valued space-time covariance functions such that both the spatial and temporal components are nonproportional. Such matrix-valued space-time covariances are referred to as being fully nonseparable.

The Gneiting class of space-time covariance functions

We first restrict ourselves to the univariate setting, i.e., p = 1. We thus choose to drop temporarily the use of subscripts for the ease of notation. The Gneiting class of spatio-temporal covariance functions was originally presented in Gneiting (2002). For an easier exposition of our results, we follow [START_REF] Allard | Simulating space-time random fields with nonseparable Gneiting-type covariance functions[END_REF] and consider the extended class of functions of the form As discussed in Section 2.2, these are proportional-in-time models, since they are the product of a covariance matrix with a single time correlation function ρ T (u) = (γ (u) + 1) -d/2 . In order to build multivariate models that are neither proportional in space nor in time, an essential building block is to define admissible and relevant temporal matrix-valued covariances of the form

] p i,j=1 . To address this point, matrix-valued pseudo-variograms are now introduced.

Matrix-valued pseudo-variograms

Consider a p-variate temporal random field

Under the assumption that the direct and cross-increments are jointly stationary, the matrix-valued pseudo-variogram

for all t, u ∈ R and any pair i, j = 1, . . . , p. Notice that the pseudo-variogram has nonnegative entries and is not necessarily an even function. For any i = 1, . . . , p, η ii (direct variogram of Z i ) is a conditionally negative semidefinite function [START_REF] Chilès | Geostatistics: Modeling Spatial Uncertainty[END_REF]. Necessary and sufficient conditions for a matrix-valued function to be a pseudo-variogram have been provided in Dörr and Schlather (2021).

Parsimonious Gneiting-Matérn and Gneiting-Cauchy multivariate models

In this section, we construct matrix-valued space-time covariance functions that are neither proportional in space nor in time. The construction is based on the substitution approach proposed in [START_REF] Allard | Simulating space-time random fields with nonseparable Gneiting-type covariance functions[END_REF] for simulating univariate space-time Gaussian random fields with Gneitingtype covariance functions. It also uses the integral representation of completely monotone functions. Compared to the model in (4), the univariate variogram is generalized into a matrix-valued pseudovariogram, as defined in (8). This construction is less general than the class that will be presented in Section 4, but it is more direct and parsimonious.

dr, where f ii and f jj are probability density functions on (0, ∞). Then, the matrix-valued function C :

)

The following two examples generalize ( 4) and ( 6) with a matrix-valued pseudo-variogram η instead of a univariate variogram γ .

Example 1 (Gneiting-Matérn Model)

ii /(4r)) be the probability density of an inverse gamma distribution on (0, ∞) with shape parameter ν ii > 0 and scale parameter r 2 ii /4 > 0. Based on the fact that an inverse gamma mixture of Gaussian covariances is a Matérn covariance [START_REF] Emery | TBSIM: A computer program for conditional simulation of three-dimensional Gaussian random fields via the turning bands method[END_REF], the direct and cross-covariances are found to belong to the Gneiting-Matérn family:

jj and M the Matérn covariance defined in (3). The matrix-valued space-time covariance function ( 13) is of Gneiting-Matérn type. The sufficient conditions on its parameters call for several comments.

⪯ c 0 and ν is a-separable (see the technical lemmas in Appendix D.1). This more restrictive condition is also found in Apanasovich et al. (2012) and Bourotte et al. (2016). Even though a-separability is not necessary, it is convenient because it is easy to satisfy. From a statistical point of view, it reduces the number of parameters from p(p + 1)/2 to p and therefore leads to a more parsimonious model.

2. The conditional negative semidefiniteness condition on a 2 is similar to that in Apanasovich et al. (2012) in a spatial context. It is weaker than the a-separability condition in (5) that applies to the Gneiting-Matérn model in Bourotte et al. (2016).

3. Since ν ⪯ c 0, the matrix exp (-ν) is positive semidefinite (see Appendix D.1). Hence, based on the Schur product theorem, a sufficient condition for τe -ν /Γ (ν) to be positive semidefinite is that τ/Γ (ν) is positive semidefinite. 4. The positive semidefiniteness condition on τ only depends on the smoothness parameters ν. In contrast, the positive semidefiniteness condition (5) in Bourotte et al. (2016) -also necessary in the parsimonious approach presented in Section 3 -depends on both ν and r(u), which is a stronger condition. Theorem 2 therefore offers more flexibility for modeling multivariate space-time data.

The Gneiting-Matérn class ( 12) is quite versatile. The space-time properties depend very much upon the values of α and β. Obviously, if α = β = 0, the model is purely spatial. In this context, by setting b to be the all-ones matrix, the condition on σ reduces to the condition of Theorem 3B

in [START_REF] Emery | New validity conditions for the multivariate Matérn coregionalization model, with an application to exploration geochemistry[END_REF].

When α = 0, accounting for the fact that exp(-ν) and all p-separable matrices with nonnegative entries are positive semidefinite (see technical lemmas in Appendix D.1), the models in ( 12) and ( 13) generalize the Gneiting-Matérn model ( 4). If, furthermore, the direct variograms η ii are increasing functions for i = 1, . . . , p, the scale parameters r ii (u) = a ii (βη ii (u) + b 2 ii ) -1/2 decrease as u increases. The space-time direct correlations are thus higher than what would happen for a separable spacetime covariance function, i.e., they are positively nonseparable. When β = 0 and under the same monotonicity assumption for the direct variograms, the scale parameters r ii (u) = √ αη ii (u) + a 2 ii /b ii increase with u. This a very original feature, since for all Gneiting-type spatio-temporal models of our knowledge, the scale parameter decreases with u. Applying the same reasoning as above, the resulting space-time direct covariances are thus negatively nonseparable. When α and β are both positive, the model is positively or negatively space-time nonseparable, depending on the relative values of the other parameters involved in r(u). However, in the common case where η ij (u) → ∞ as u → ∞, r(u) asymptotically tends to a matrix with all elements equal to √ α/β. The space-time covariance is therefore asymptotically separable as |u| → ∞.

This model covers thus a large variety of nonseparability situations. When analyzing multivariate space-time data, an exploratory analysis can reveal the type of nonseparability that best describes the dataset, offering the possibility to set α = 0 (resp. β = 0) if uniformly positive (resp. negative) nonseparability is established. This point will be illustrated in Section 5.

Models for unbounded pseudo-variograms

The temporal structure is parameterized by a matrix-valued pseudo-variogram η : u ↦ → [η ij (u)] p i,j=1 . As discussed in Section 2.3, it is critical that this pseudo-variogram is unbounded for the direct and cross-covariances C ij (h, u) tends to 0 as |u| → ∞. On the one hand, [START_REF] Chen | Parametric variogram matrices incorporating both bounded and unbounded functions[END_REF] showed that the well-known linear model of coregionalization cannot be used to construct matrix-valued pseudo-variograms with different diagonal entries, which is required for a fully nonseparable multivariate model. On the other hand, in Papritz et al. (1993) it is shown that, for a matrix-valued pseudo-variogram η, the following large distance behavior holds: lim |u|→∞ η ij (u)/η ii (u) = 1, for any pair i, j = 1, . . . , p. In other words, all entries of a matrix-valued pseudo-variogram must have the same behavior for a very large lag separation u. and time across several variables, and the number of parameters, that should remain interpretable and not too numerous. More complex models should only be preferred if supported by the data.

The dataset analyzed in this work is of moderate size with 10 independent repetitions of 13 × 31 = 403 space-time trivariate data. Its analysis has shown that our model improves the fitting and the predictive performances over the more parsimonious model in Bourotte et al. (2016), in particular when the time dimension is important for prediction. In Section 5.3, it was shown that the maximum PL difference is around 600. Combined with the improved prediction scores in time, this provides evidence in favor of our new model for this dataset. The analysis of other datasets, in particular with more spatial locations, is necessary for confirming this first result.

Estimating the parameters of a parametric model such as the one in Section 5 is a challenge. Here, we have used Pairwise Likelihood because it was proven efficient on the proportional-intime model in Bourotte et al. (2016). It is however relatively slow: reaching convergence with our optimization algorithm and the function nlmbinb takes typically several hours on a recent laptop computer. There is thus certainly room for improvement for an easier use of such complex models. At the cost of even larger computing times, composite likelihood with larger sets of elements than pairs and full likelihood might be interesting options for a more accurate estimation of the parameters, considering the complex interactions between space, time and the variables. Conducting a thorough comparison of several likelihood approaches was out of the scope of this work and left for future research. In our opinion, more interesting improvements can be expected by using Vecchia's Gaussian process approximation (Katzfuss and Guinness, 2021). We are looking forward to (multivariate) spatio-temporal versions of the GpGp package.

In Appendix A we have proposed even more general constructions where both the spatial scale and spatial smoothness parameters can vary with the time lag u. We have also designed (see Appendix C) computationally efficient algorithms for simulating Gaussian random fields with these covariance functions, which are not restricted in the number of target spatial coordinates and allow tens of thousands of time coordinates, hence are applicable to large-scale space-time prediction problems.

All the models presented in this work can easily be generalized to R d × R ℓ , where ℓ is an integer greater than one. Other constructions based on characterizations established in Porcu et al. (2022) are also possible. Extensions of our models to other spaces that are of interest in environmental sciences, atmospheric sciences and geosciences, such as a sphere crossed with an Euclidean space (S d × R ℓ ), are left to further research.

Appendix A. General forms of Gneiting-Matérn covariances with temporal-lag dependent parameters

The following theorem provides sufficient validity conditions for a nonseparable matrix-valued space-time covariance function in which the spatial direct and cross-covariances belong to the Matérn family, with collocated correlation coefficients, scale and/or smoothness factors that depend on the temporal lag. The proof follows the line of that of Theorem 2 and is based on the sufficient validity conditions for matrix-valued spatial Matérn models given in Gneiting et al. (2010, Theorem 1), Apanasovich et al. (2012, Theorem 1) and Emery et al. (2022, Theorems 1, 2, 3). Also, the spectral algorithm given in Appendix C can be adapted to simulate space-time random fields having such a covariance. Theorem 3. Let d, p be positive integers, r and ν be positive real numbers. Let µ 0 , ν 0 and ψ 0 be p × p symmetric conditionally negative semidefinite matrices, the first two ones with positive entries and the last one with nonnegative entries. Also, let u ↦ → µ(u) -µ 0 , u ↦ → ν(u) -ν 0 and u ↦ → ψ(u) -ψ 0 be p × p matrix-valued pseudo-variograms in R and u ↦ → ρ(u) be a p × p matrix-valued covariance in R.

Then, the matrix-valued function

, with ⌊•⌋ and ⌈•⌉ standing for the floor and ceil functions.

Appendix B. A simulation algorithm for parsimonious models

We provide a simulation algorithm for the parsimonious Gneiting-Matérn and Gneiting-Cauchy multivariate models of Section 3.

A realization of a p-variate zero-mean random field Z = [Z i ] p i=1 with approximate Gaussian distribution and with covariance as in ( 10) can be constructed from the standardized sum of a large number M of independent copies of basic random fields, each of which depending only on two random variables R and Φ, two random vectors V and Ω and a temporal p-variate random field W . This is summarized in Algorithm 1 hereinafter, where ⟨•, •⟩ stands for the usual scalar product in R d , R ∼ f is the short notation for ''R is distributed according to the distribution f '', U (0, 2π ) is the uniform distribution on (0, 2π ), N d (0, Σ ) is the d-dimensional normal distribution with expectation 0 and variance-covariance matrix Σ , and I d is the d × d identity matrix. We refer to [START_REF] Arroyo | Algorithm 1013: An R implementation of a continuous spectral algorithm for simulating vector Gaussian random fields in Euclidean spaces[END_REF] for criteria on the choice of the distribution f .

The multivariate random field W can be simulated with Algorithm 2 hereinafter, which is applicable as long as the number of target time coordinates is not too large (in practice, up to a few tens of thousands), so that the positive semidefinite square root matrix can be computed (Horn and Johnson, 2013, Theorem 7.2.6). For larger numbers, iterative methods based on Gibbs sampling [START_REF] Lantuéjoul | Simulation of a Gaussian random vector: A propagative version of the Gibbs sampler[END_REF][START_REF] Arroyo | Iterative algorithms for non-conditional and conditional simulation of Gaussian random vectors[END_REF]) can be used instead. Alternatively, for pseudo-variograms in the class presented in Section 4.2, W can be constructed by combinations of stationary Gaussian random fields and intrinsic random fields with Gaussian increments, for which discrete [START_REF] Dietrich | Fast and exact simulation of stationary Gaussian processes through circulant embedding of the covariance matrix[END_REF][START_REF] Stein | Fast and exact simulation of fractional Brownian surfaces[END_REF] and continuous [START_REF] Emery | An improved spectral turning-bands algorithm for simulating stationary vector Gaussian random fields[END_REF][START_REF] Arroyo | Spectral simulation of vector random fields with stationary Gaussian increments in d-dimensional Euclidean spaces[END_REF] spectral simulation algorithms are available.

Appendix C. A spectral simulation algorithm

A realization of a p-variate Gaussian random field Z = [Z i ] p i=1 with zero mean and matrix-valued covariance function as in Theorem 2 can be constructed with a central limit approximation, by rescaling the sum of a large number M of independent copies of basic random fields. Specifically, let us consider that Z is to be simulated on a grid of n S nodes in R d crossed with a grid of n T nodes

whose generic entry is the spectral density of the Matérn covariance h ↦ → σ ij (u mn ) M(h; r ij (u mn ), ν ij ), with u mn = t m -t n , evaluated at ω [START_REF] Chilès | Geostatistics: Modeling Spatial Uncertainty[END_REF]:

In the same way as in the proof of Theorem 2 given in Appendix D.2, the p-variate random field Z to simulate at the n S × n T target nodes of R d × R can be viewed as a q-variate random field D. Allard, L. Clarotto and X. Emery Spatial Statistics 52 (2022) 100706variable in (0, 2π ), and W i , i = 1, . . . , p, are random fields on R whose direct and cross-increments are second-order stationary, characterized by the matrix-valued pseudo-variogram η, and Gaussian. The random variable R, the random vectors Ω, Φ and V and the random field W = [W i ] p i=1 are independent.

Following Allard et al. (2020, Theorem 3), it is straightforward to show that, for any i, j = 1, . . . , p, Z i and Z j have a zero mean and that the covariance function between Z i and Z j is

The cross-covariance between Z i and Z j depends only on the spatial and temporal lags h = ss ′ and u = tt ′ . Accordingly, the random field Z is second-order stationary with zero mean and covariance function C :

For C to be a positive semidefinite function, we need to show that M is positive semidefinite for any such design in R d × R. This can be done by viewing the p-variate spatio-temporal random field at n T time coordinates as a q-variate spatial random field, with q = p n T , and using the sufficient validity conditions for multivariate spatial Matérn models given in [START_REF] Emery | New validity conditions for the multivariate Matérn coregionalization model, with an application to exploration geochemistry[END_REF].

and

] n T m,n=1

are conditionally negative semidefinite (see Lemma 3). Furthermore, the matrix

is positive semidefinite (see Lemma 2). Hence, according to Emery et al. (2022, Theorem 2B),

Proof of Corollary 1. In Theorem 2, ρ(u) must be a matrix-valued covariance function. Here, we set ρ(u) = ρ, for all u ∈ R. By comparing (13) with (12), it is thus required that τe -ν /Γ (ν) = ρ is a positive semidefinite matrix. □

Appendix E. Supplementary material

Supplementary material related to this article can be found online at https://doi.org/10.1016/j. spasta.2022.100706.

Space-time simulations of the proposed model with the parameters described in Section 4.3.

We used Algorithm 3 with M = 20,000 cosine waves. The density g is the spectral density of a Matérn covariance function with ν = 0.5 and r = 2. All simulations have been obtained with the same seed for random number generation. They are thus comparable, except in terms of space-time non-separability.

Supplementary material

S.2 Discussion on "Competition on Spatial Statistics for

Large Datasets"

Supplementary materials for this article are available at https:// doi.org/ 10.1007/ s13253-021-00462-2.

Discussion on "Competition on Spatial Statistics for Large Datasets"

Denis Allard , Lucia Clarotto, Thomas Opitz, and Thomas Romary

We discuss the methods and results of the RESSTE team in the competition on spatial statistics for large datasets. In the first sub-competition, we implemented block approaches both for the estimation of the covariance parameters and for prediction using ordinary kriging. In the second sub-competition, a two-stage procedure was adopted. In the first stage, the marginal distribution is estimated neglecting spatial dependence, either according to the flexible Tuckey g and h distribution or nonparametrically. In the second stage, estimation of the covariance parameters and prediction are performed using Kriging. Vecchias's approximation implemented in the GpGp package proved to be very efficient. We then make some propositions for future competitions.

Discussion on "Competition on Spatial Statistics for Large Datasets"

After some experiments, the final setting was B = 15 and N D = 500. For datasets G4/G12, when the smoothness parameter is large, the covariance matrix was seen as singular in R; in these cases, N D was set to 400. The function nlbmin was used for minimizing the negative log-BL. Initial values for the optimization were given by the WLS estimates of the first stage. Since blocks are random, this operation is repeated 5 times for each dataset. For each parameter, the average of the estimates was computed. This average is the final estimate.

PREDICTION

A "local unique neighborhood" technique is adopted, in which Kriging is performed at all locations belonging to the same block using a common neighborhood. To this end, a K × K regular grid is defined on the domain, and one Kriging system is built for each mesh of that grid, hereafter referred to as target block. The neighborhood of a given target block is a disk of radius 1.5 √ 2/K , so that all data belonging to the 3 × 3 meshes surrounding the center of the target block are part of the Kriging system. Smaller K yields higher precision but is computationally more demanding. Larger K leads to smaller neighborhood and lower precision. Computing time decreases rapidly as K increases, roughly at a K -4 rate. A good trade-off between performance and speed was obtained with K = 31. The average number of training data per Kriging system was therefore around 1200.

DISCUSSION

Overall these block approaches performed relatively well, ranking fourth and second in sub-competitions 1a and 1b, respectively. It is noticeable that in sub-competition 1b the plug-in block Kriging described above was only outperformed by ExaGeoStat using the true model, whereas ExaGeoStat with the estimated parameters performed slightly less well. As expected, we experienced some difficulties for the simultaneous estimation of β, ν and σ 2 for smooth GPs with large effective range, i.e., when ν > 0.6 and E R > 0.1. G4 is particularly poorly estimated, with simultaneous underestimation of the sill ( σ 2 = 1.2092) and underestimation of the range ( β = 0.0486), resulting in a high MMOM and RMSEeven though MLOE is relatively small. Estimating ν when the regularity is high in the presence of a nugget is very challenging (G12, G13 and G15), as can be seen in Figures 2 andS1. In these cases, the poor estimation translated into high RMSEs for plug-in Kriging.

COMPETITION 2: TRANS-GAUSSIAN MODELS

In this competition, the generating mechanism of the data was supposed to be unknown. Therefore, we first explored marginal distributions of the data using tools such as boxplots and histograms to check if a Gaussian model for the marginal distributions makes sense. If not, we followed the well-established statistical practice of using a marginally transformed Gaussian model. This approach, among which the Box-Cox transformation is the most popular representative, allows accommodating data features such as heavier-than-Gaussian tails or asymmetry of upper and lower tails. recently studied for spatial Gaussian fields by Xu and Genton (2017). Tukey g and h transformation function is strictly monotonic and defined as follows:

(1)

Given a standard Gaussian variable W , the Tukey g and h distribution is constructed as

with parameters to control location (β 0 ), scale (β 1 ), asymmetry (g) and tail heaviness (h).

No external predictor variables were provided for the dataset, and visually we could not detect any other trends or anisotropies in the data. Therefore, we used a stationary and isotropic Tukey g and h random field model, which is obtained by applying the transformation (1) to a standard Gaussian random field with the Matérn covariance. We estimated the four parameters for each of the fields through the independence likelihood (neglecting spatial dependence) using the R library OpVaR; histograms of data after the inverse transformation to the standard Gaussian margins are shown in Figure 1b and correspond well to the superposed standard Gaussian density.

Nonparametric Transform

The estimation of the parameters of the Tukey g and h transform relies on an approximate parameter estimation procedure described below, which may conduct to underestimation or overestimation of the transformation parameters. Therefore, we also investigated the use of simple nonparametric transforms, namely a log transform for datasets 2a1 and 2b2 and a log-log transform for dataset 2a2. Their adequacy was checked by a visual inspection of the histograms of the transformed data (not represented here), in particular their symmetry.

ESTIMATION AND PREDICTION

Joint estimation of marginal and dependence parameters can be useful to allow for transfer of information between the models for margins and dependence, and for very accurate assessment of uncertainty in estimates. However, two-step approaches with separate estimation of marginal parameters, followed by marginal transformation to the standard Gaussian scale and estimation of the Gaussian correlation function, have the advantage of being more robust. In particular, they allow for the combination of different estimation techniques for margins and dependence. We here adopt two-step approaches. In the first step, which is common to two of our three approaches, we implement two substeps: (1) estimation of the marginal parameters β 0 , β 1 , g, h using the independence likelihood (i.e., by neglecting spatial dependence); (2) marginal transformation to the standard Gaussian scale using the parameters estimated in substep 1. In the last approach, we used a nonparametric transform. The following two subsections detail estimation after transformation of data to the standard Gaussian margins. approach (Sect. 3.3.1), the size of the nearest neighbor Kriging neighborhood was set to 25 (Table 2).

DISCUSSION

Several methods were initially considered to treat the datasets of sub-competitions 2a and 2b, coming either from more classical geostatistical analyses or from machine learning techniques. Data-based methods such as random forests and neural networks, even when the spatial coordinates were combined with the addition of local features (mean/min/max values computed on K nearest neighbors), led to meager results in prediction.

Regarding competition 2, the type of point prediction to use depends on the score to be optimized. The conditional mean is known to minimize mean-squared error (MSE), and it corresponds to Kriging predictions. However, when marginal transformations are involved, the transformed conditional mean prediction is not equal to the conditional mean on the transformed scale. When the target is to minimize mean absolute errors, then conditional medians provide optimal predictions. With Gaussian data, conditional means and medians coincide. To compute conditional medians, we can simply transform data to the Gaussian scale, predict on the Gaussian scale, and then transform back to the original data scale. In competition 2, the target score was MSE. Due to very small prediction variances on the Gaussian scale, we found only very small differences between conditional median and conditional mean predictions on the non-Gaussian marginal scale of the original data. In some approaches (e.g., the bootstrap approach), we have therefore submitted marginally transformed Gaussian Kriging predictions.

FUTURE DIRECTIONS

This competition has explored different methods for the estimation of the parameters, and for the prediction (Kriging), of Gaussian and Tukey g and h trans-Gaussian random fields. Among these methods, several have achieved very good performance, as shown in Huang et al. (2021). More challenging setups than classical point data could also be considered, such as preferential sampling or the addition of location errors. An interesting question is whether the methods described in this paper would be efficient on gridded data, or whether grid-specific approaches would perform better. In particular, the case of gap filling (large areas without observations) could also be investigated.

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

MOTS CLÉS

Géostatistique; Modélisation spatio-temporelle; Analyse numérique; SPDE; Statistique environmentale. RÉSUMÉ L'introduction de modèles statistiques inspirés de la physique des phénomènes sous-jacents et numériquement efficaces est d'un intérêt croissant pour la prédiction de processus spatio-temporels en sciences environnementales. Les grands jeux de données spatio-temporelles nécessitent de nouvelles méthodes numériques efficaces. L'approche par Equations aux Dérivées Partielles Stochastiques (EDPS) s'est avérée efficace pour l'estimation et la prédiction dans un contexte spatial. Nous présentons ici une EDPS d'advection-diffusion avec une dérivée de premier ordre en temps qui définit une grande classe de modèles spatio-temporels non séparables. On construit une approximation de la solution de l'EDPS par un champ aléatoire Markovien Gaussien en discrétisant la dérivée temporelle par la méthode des différences finies (Euler implicite) et en résolvant l'EDPS spatiale par la méthode des éléments finis (Galerkin continu) à chaque pas de temps. La technique de stabilisation "Streamline Diffusion" est introduite lorsque le terme d'advection domine la diffusion. Des méthodes de calcul efficaces sont proposées pour estimer les paramètres de l'EDPS et pour prédire le champ spatio-temporel par krigeage, ainsi que pour effectuer des simulations conditionnelles. L'approche est appliquée à des jeux de données de rayonnement solaire et de vitesse du vent. Ses avantages et ses limites sont examinées, et de nouvelles perspectives de travail sont envisagées, notamment afin de proposer une extension dans un cadre non stationnaire. On présente également un travail portant sur la généralisation non séparable de la classe de Gneiting des fonctions de covariance spatio-temporelles multivariées. Le principal potentiel de l'approche est la possibilité d'obtenir des modèles entièrement non séparables dans un cadre multivarié, et les avantages sont illustrés sur un ensemble de données météorologiques trivariées. De plus, on propose une analyse de méthodes d'estimation et de prédiction approximées pour les données spatiales et spatio-temporelles, motivée par l'objectif de parvenir à un compromis entre l'efficacité statistique et la complexité computationnelle. Ces méthodes se sont avérées efficaces pour l'estimation des paramètres et la prédiction dans le contexte de la "Compétition de statistiques spatiales pour les grands jeux de données" organisée par la King Abdullah University of Science and Technology (KAUST) en 2021 et 2022. Enfin, d'autres pistes de recherche sont envisagées et examinées.

ABSTRACT

In the task of predicting spatio-temporal fields in environmental science using statistical methods, introducing statistical models inspired by the physics of the underlying phenomena that are numerically efficient is of growing interest. Large space-time datasets call for new numerical methods to efficiently process them. The Stochastic Partial Differential Equation (SPDE) approach has proven to be effective for the estimation and the prediction in a spatial context. We present here the unsteady advection-diffusion SPDE which defines a large class of nonseparable spatio-temporal models. A Gaussian Markov random field approximation of the solution to the SPDE is built by discretizing the temporal derivative with a finite difference method (implicit Euler) and by solving the spatial SPDE with a finite element method (continuous Galerkin) at each time step. The Streamline Diffusion stabilization technique is introduced when the advection term dominates the diffusion. Computationally efficient methods are proposed to estimate the parameters of the SPDE and to predict the spatio-temporal field by kriging, as well as to perform conditional simulations. The approach is applied to solar radiation and wind speed datasets. Its advantages and limitations are discussed, and new perspectives for future work are envisaged, especially involving a nonstationary extension of the approach. As a further contribution of the PhD, the nonseparable generalization of the Gneiting class of multivariate space-time covariance functions is presented. The main potential of the approach is the possibility to obtain entirely nonseparable models in a multivariate setting, and this advantage is shown on a weather trivariate dataset. Finally, a review of some methods for approximate estimation and prediction for spatial and spatio-temporal data is proposed, motivated by the objective of reaching a trade-off between statistical efficiency and computational complexity. These methods proved to be effective for parameter estimation and prediction in the context of the "Spatial Statistics Competition for Large Datasets" organized by the King Abdullah University of Science and Technology (KAUST) in 2021 and 2022. Lastly, possible further research directions are discussed.

KEYWORDS

Geostatistics; Spatio-temporal modeling; Numerical analysis; SPDE; Environmental statistics.