Keywords:

and Bochner . . . . . . .

A B S T R A C T

The analysis of electrical load curves collected by smart meters is a key step for many energy management tasks ranging from consumption forecasting and load monitoring to customers characterization and segmentation. In this context, researchers from EDF R&D are interested in extracting significant information from the daily electrical load curves in order to compare the consumption behaviors of different buildings. The strategy followed by the group which hosted my doctorate is to use physical and deterministic models based on information such as the room size, the insulating materials or weather data, or to extract hand-designed patterns from the electrical load curves based on the knowledge of experts. Given the growing amount of data collected, the interest of the group in statistical or data-driven methods has increased significantly in recent years. These approaches should provide new solutions capable of exploiting massive data without relying on expensive processing and expert knowledge. My work fits directly into this trend by proposing two modeling approaches: the first approach is based on functional time series and the second one is based on non-negative tensor factorization. This thesis is split into three main parts. In the first part, we present the industrial context and the practical objective of the thesis, as well as an exploratory analysis of the data and a discussion on the two modeling approaches proposed. In the second part, we follow the first modeling approach and provide a thorough study of the spectral theory for functional time series. Finally, the second modeling approach based on non-negative tensor factorization is presented in the third part. Pie charts of the decomposition of load curves . . . . . Figure 1.7

Histograms for each features . . . . . . . . . . . . . . . Figure 1.8

Power vs temperature for Monday (in blue) and Satuday (in brown). In orange and green: linear and quadratic regression for Mondays. In purple and gray: linear and quadratic regressions for Saturdays. . . . . Figure 2 Harmonic fPCA loading filters as perturbations of the mean for opening days. The dashed black line in the mean curve μ1 and the solid blue line is the perturbation μ1 + C(δ -1 ϕ 1,-1 + δ 0 ϕ 1,0 + δ 1 ϕ 1,1 ). This part provides general information about the industrial context of my PhD and the practical objective addressed during these three years. In particular, we discuss in Chapter 1 the problem of the representation and comparison of time series and provide an overview of the functional time series framework. Next, we present EDF's strategy for extracting information from the daily load curves. In Chapter 2, we present the results of the exploratory analysis of the dataset provided by EDF. After explaining the pre-processing steps, we study the results of second order dimension reduction methods, namely functional principal component analysis and its harmonic extension. A discussion about the drawbacks of these methods is provided and we conclude that a non-negative structured decomposition that takes into account the effect of temperature should overcome these drawbacks.

1

C O N T E X T A N D B A C K G R O U N D

Context

EDF is a leading French energy company whose activity extends to many areas ranging from the production and distribution of electricity to services.

The group has a large nuclear fleet, with 58 reactors spread throughout the territory and owns numerous onshore and offshore wind farms, as well as solar fields and hydroelectric power plants. This makes it the leading European producer of renewable energies.

For several years, EDF has been participating in a process of transformation of the energy sector in order to meet the challenges of climate change and aims for CO 2 neutrality by 2050. This vision is reflected in their Cap 2030 strategic project, which is based on three pillars: innovation for customers, low carbon and international growth. In this context, EDF takes great interest in research and has eight R&D sites, including three in France: EDF Lab Paris-Saclay, EDF Lab Les Renardières and EDF Lab Chatou. In the first site, researchers focus on various topics ranging from vibrational mechanics and mathematical and digital simulation to client relations. The research carried out in the second site focuses on energy efficiency, security and reliability of the electrical network. Finally, the last site develops expertise in the fields of hydraulics, renewable and nuclear energies and the environment.

My PhD was hosted by two department: the TREE department (more specifically, the E36 group) at EDF Lab les Renardières and the PRISME department (more specifically, the P12 group) at EDF Lab Chatou.

The PRISME (Performance, Risque Industriel et Surveillance pour la Maintenance et l'Exploitation) department develops innovative solutions for data acquisition and physical and mathematical modeling to improve electricity production. The department has two main areas of expertise. The first is the life cycle of the means of production with physical measurements and simulations, but also safety control. The second expertise is data processing, in particular signal and image processing and statistical learning, in the context of energy production. To carryout its projects, the department is divided into six groups. The P12 group (Dynamic Systems, Images and Signals) is more closely linked to the data processing expertise. Its competences are thus articulated around the signal and image processing, optimization and dynamic systems. The projects carried out concern various themes such as hydraulics, thermal flexibility or life cycle.

The TREE (Technologie et Recherche pour l'Efficacité Energétique) department gathers 9 groups working on various aspects of electricity regulation, from resource optimization to building consumption modeling. The department develops low CO 2 tools for new and old buildings in the residential, tertiary and industrial sectors. In this context, artificial intelligence approaches are becoming increasingly popular with applications ranging from defect detection on heat pumps and solar panels to automatic energy regulation in Smart Buildings. The E36 group (Services & Systèmes Connectés) specializes in the Internet of Things (IoT) for Smart Buildings. In particular, the group integrates data collection and analysis in the ecosystems of EDF branches (e.g. Sowee, Dalkia, EDF ENR) and conducts numerous studies for internal and external collaborators. From the point of view of data analysis, the solutions developed by the group consist mainly in the visualization and monitoring of consumption, but also in the analysis of load curves based on the knowledge of experts. For the latter task, the group developed a web platform called ACDC (Analyse de Courbes De Charges).

My work complements those approaches in two ways: I propose a statistical and data-driven point of view on load curves analysis and I treat the multi-sites case where the consumption of several buildings is observed. In this industrial context, the practical objective of my work can be expressed as follows.

(PO) Extract information from multi-sites load curves to help understanding the differences between the sites using an unsupervised approach which does not use experts' knowledge but can still be easily interpretable.

At a higher level, the practical objective (PO) raises directly the question of the representation and comparison of time series for which many methods have been proposed in the literature of the last decades. In this Chapter, I propose an overview of these methods, as well as the methods implemented on the ACDC web platform.

Representation and comparison of time series

Comparing time series is not as trivial as comparing multivariate data and there are many methods available depending on the context, constraints and/or prior knowledge about the data. Unlike multivariate data, there is no natural distance for time series. In fact, the usual metrics used for multivariate data are often based on coordinate-wise comparison and are therefore not appropriate for comparing time series sampled differently. To address potential misalignment between two time series, metrics based on Dynamic Time Wrapping (DTW) have been proposed [START_REF] Sakoe | A Dynamic Programming Approach to Continuous Speech Recognition[END_REF]. Given a metric to compare the samples, DTW computes the best alignment by minimizing the discrepancy between the two signals. The minimum discrepancy defines a distance between the time series and can be used in many machine learning techniques. However, the computation of such a distance between two signals of lengths T 1 and T 2 has a computational complexity of O (T 1 T 2 ) and becomes too expensive to use for long time series such as the electric load curves analyzed at EDF which are sampled at 10 min intervals for 1 year and therefore have 51100 samples per site Fortunately, the temporal representation of a time series is usually very high dimensional compared to the useful information contained in the signal, in particular in the presence of noise. Therefore, it is common to rely on well-adapted representations to embed the signals in a space of lower dimension before comparing them. These representation-based approaches can be divided into three categories.

Property-based comparison

The first strategy consists in extracting specific properties from a priori assumptions based on expert knowledge or on signal processing and statistical tools. These properties can then be quantified and compared to determine the similarity of two signals. For example, in the ACDC web platform, predefined shapes are extracted from the load curves. In signal processing, Fourier or wavelets representations are also very common but dictionarybased or functional representations are also possible. Statistical methods include regression coefficients or ARMA coefficients. One of the benefits of this strategy is that using a priori knowledge can help in interpreting the results. However, quantifying such properties can be difficult in nonstationary, noisy or heterogeneous settings or with the presence of outliers or missing data. Additionally, selecting which properties to use in the com-parison can be a long and tedious task that requires several test and error steps and the help of experts' knowledge. This process is known as feature engineering and can result in representations that are very specific to the type of data being analyzed. For example, at EDF, we know that different properties are important for the residential, tertiary and industrial sectors.

End-to-end methods

The second strategy to compare signals is to use end-to-end methods. These methods take the raw signal as input and simultaneously learn a representation of the data and the parameters for a model that solves a specific task. A popular end-to-end method is deep learning [START_REF] Goodfellow | Deep Learning[END_REF] where successive representations of the data are learned in the internal layers and are used by the output layer to perform a given task (e.g. regression, classification). Contrary to the first strategy, the properties extracted by end-to-end methods are not known a priori and are learned directly from the data. Despite their versatility, these methods often present the disadvantage of being difficult to interpret because the representation is learned to perform the task and not for interpretation. For example, it is generally difficult to relate the inner layers of a neural network to the properties of the original signal. In addition, these methods require large amounts of training data because the number of parameters can be very high. It should be noted that other end-to-end methods such as task-driven dictionary learning can yield interpretable results but necessitate labeled data [START_REF] Mairal | Task-Driven Dictionary Learning[END_REF][START_REF] Mairal | Supervised Dictionary Learning[END_REF].

Pattern-based comparison

Finally, the last category of methods aims at extracting recurring patterns from the data. Contrary to end-to-end methods, this is usually done without a specific task in mind, but using general a priori assumptions on the patterns such as orthogonality, non-negativity, independence or sparsity. Since these assumptions are generally not data specific, these methods are very versatile and popular for many practical applications. Most of these methods approximate either the signal or the centered signal x t as

x t = R ∑ r=1 a r s r,t + t , (1.2.1)
where is a noise term. A common interpretation is that the a r 's represent the patterns which are modulated over time by an activation signal s r,t . For example, a binary activation would suggest that the pattern is present in the signal at specific time periods. In source separation, s r,t are interpreted as source signals which are mixed together by the coefficients a r 's. In factor analysis, a r is called the loading and s r,t the factor. The most popular example of pattern-based method is Principal Components Analysis (PCA) [START_REF] Jolliffe | Principal component analysis[END_REF] which gives orthogonal loadings and uncorrelated factors (usually called scores). Other popular methods include Independent Component Analysis (ICA) which searches for independent sources based on maximization of non-Gaussianity [START_REF] Cardoso | Source separation using higher order moments[END_REF][START_REF] Cardoso | Blind beamforming for non-Gaussian signals[END_REF] or on second-order methods [START_REF] Belouchrani | A blind source separation technique using second-order statistics[END_REF][START_REF] Yeredor | Blind separation of Gaussian sources via second-order statistics with asymptotically optimal weighting[END_REF]. Finally using non-negativity constraints on the patterns and activations leads to Non-negative Matrix Factorization (NMF) [START_REF] Lee | Algorithms for Non-negative Matrix Factorization[END_REF] and sparsity constraints leads to Sparse Dictionary Learning [START_REF] Lee | Efficient sparse coding algorithms[END_REF]. It is important to note that these methods are not intended to model the same statistical properties of the data. For example, PCA is a second-order method which models the variance while the NMF models the mean. For time series, using a convolution in (1.2.1) instead of multiplication may be interesting to take into account the dependence between the observations. For this reason, several pattern representation models have their convolutional counterpart. For example, Brillinger (2001, Chapter 9) presents a PCA in the frequency domain which, in the time domain, has a convolutional form with uncorrelated scores time series. Convolutional dictionary learning is used to capture recurring patterns from a time series in a shift-invariant way [START_REF] Grosse | Shift-Invariant Sparse Coding for Audio Classification[END_REF].

The functional time series framework

Functional data analysis (FDA) aims at treating complex data structure presenting inherent smoothness such as curves or surfaces. This field differs from the multivariate framework by this smoothness assumption and by the fact that the data are treated as belonging to an infinite dimensional space. Since the path of a continuous time stochastic process can be seen as a random function, it is legitimate to think that FDA is similar to stochastic processes analysis. However, the methodologies used in these two frameworks generally differ because, from the point of view of stochastic pro-cesses, a random function is considered as a collection of univariate random variables, while in FDA we view a random function as a random element valued in a separable Banach or Hilbert space such as L 2 (U ) where U is a bounded interval of R. Details about such random variables are recalled in Appendix B. FDA also deals with datasets consisting of a panel of random functions and the interest is focused on the analysis of the statistical properties of this panel the panel of functions. The literature on FDA has grown in the last decades and many multivariate statistical tools have been extended to this framework (see e.g. [START_REF] Ferraty | Nonparametric functional data analysis[END_REF][START_REF] Horváth | Inference for Functional Data with Applications[END_REF][START_REF] Ramsay | Functional data analysis[END_REF][START_REF] Wang | Functional Data Analysis[END_REF]. These tools have found application is various fields from medical imaging [START_REF] Lila | Statistical analysis of functions on surfaces, with an application to medical imaging[END_REF] and biophysics [START_REF] Tavakoli | Detecting and localizing differences in functional time series dynamics: a case study in molecular biophysics[END_REF] to demography [START_REF] Hyndman | Robust forecasting of mortality and fertility rates: A functional data approach[END_REF] and linguistics [START_REF] Tavakoli | A spatial modeling approach for linguistic object data: analyzing dialect sound variations across Great Britain[END_REF]. As it is already the case in the univariate and multivariate cases, the question of dependence between observations is important. Although the i.i.d. framework is widely used, the analysis of functional data with spatial and/or temporal dependence has also been an active field of research in the FDA community since the seminal work of [START_REF] Bosq | Linear processes in function spaces[END_REF]. The functional time series framework is convenient to represent temporal signals collected over several days and where the structures of intraday and interday dependence differ. This is exactly the case with the electricity consumption data analyzed at EDF which consists of electrical load curves collected over a period of 1 year at a sampling rate of 10 minutes. For this type of data, it is natural to think that intra-day dependence is more related to human activity and that the inter-day dependence is more related to seasonal variations. In this framework, we represent the load data of a given site as a panel of curves (X t ) 1≤t≤T where X t belongs to the separable Hilbert space L 2 ([0, 24)). In this case, X t (u) is the instantaneous consumption at day t and intraday time u ∈ [0, 24). In the next section, we will see that this framework is already implicitly used in the ACDC platform. In the rest of this section, we fix a probability space (Ω, F , P) and review basic aspects of FDA and functional time series.

Functional vs Hilbertian setting

There is a duality between functional data and data valued in a separable Hilbert space. In fact, from a theoretical point of view, the two settings are equivalent since any separable Hilbert space is isomorphic to the function space L 2 (U ). However, some results may be easier to interpret in one setting than the other. The Hilbertian setting is more abstract but has the advantage of providing more elegant results without using integral operators, as is often the case in the functional setting. On the contrary, the functional setting can lead to finer results, especially when continuity is involved or when the functional variable is seen as the path of a continuous-time stochastic process. A drawback of this setting is that one must keep in mind that, if X is an L 2 (U )-valued random variable, the evaluation X(u) does not make sense.

In this case, a statement like

E [X(u)] = µ(u) , u ∈ U ,
is an abuse of notation which should be understood as

E [X] = µ in L 2 (U ).
This abuse of notation is widespread and usually harmless but may, in some cases, lead to incorrect reasoning. In this manuscript, I try to be as rigorous as possible for theoretical considerations by primarily using the Hilbert setting and justifying statements made with the functional setting. For practical work, some abuse of notation is allowed as, when it comes to real data, the transition from discrete data to functional data is often done in a way which implies continuity.

From discrete data to functional data

In practice, we access a functional datum x ∈ L 2 (U ) as a collection of pairs (u i , y i ) 1≤i≤I where u i ∈ U and y i ∈ R. For example, for EDF's data I = 144 and (u i ) 1≤i≤I are evenly spaced in [0,24). If I is small enough, representing the data by the vector y := [y 1 , • • • , y I ] is not necessarily prohibitive from a computational point of view. However, this multivariate representation fails to capture the underlying structure of the data, especially smoothness, as the raw data can present errors and noise. For this reason, a first step in FDA consists in recovering the function x from the discrete observations. This process is called smoothing and usually relies on the model

y i = x(u i ) + i , i = 1, • • • , I , (1.3.1)
where ( i ) 1≤i≤I represents the noise.

Non-parametric smoothing

The model displayed in (1.3.1) is very common in non-parametric statistics and therefore many non-parametric methods can be applied to the smoothing problem. For example, one can estimate x(u) by the average of the values y i for i such that u i is in a neighborhood of u. This method, called kernel-smoothing estimates x(u) by

x(u) = I ∑ i=1 κ(u, u i )y i ,
where κ is a non-negative function such that κ(u, v) vanishes when |u -v| vanishes. A typical example is the Gaussian kernel κ(u, v) = exp -|u-v| 2 2σ 2 . Another very popular non-parametric smoothing methods is based on the addition of a penalty term to the least square loss, typically on the L 2 -norm of the second derivative of x, admitting that this second derivative is in L 2 (U ). The estimator writes as

x = argmin x n ∑ i=1 (y i -x(u i )) 2 + λ U x (u) 2 du .
This optimization problem is studied in Green and Silverman (1994, Chapter 2). In particular, the solution is necessarily a natural cubic spline and is entirely characterized by the vector x = [ x(u 1 ), • • • , x(u I )] T which is defined as x = argmin x∈R I yx 2 2 + λx Kx , with an appropriate matrix K.

Parametric smoothing

In the FDA community, the most common smoothing method consists in representing the function x as a linear combination of linearly independent (but not necessarily orthogonal) functions (v k ) 1≤k≤K ⊂ L 2 (U ) called basis functions. Namely, we assume that there exists c 1 ,

• • • , c K ∈ R such that x(u) = K ∑ k=1 c k v k (u) = c v(u) , u ∈ U , (1.3.2)
where

c := [c 1 , • • • , c K ] and v(u) := [v 1 (u), • • • , v K (u)]
. The underlying assumption behind this approach is that x belongs to (or is well approximated by its projection onto) the space V := Span (v 1 , • • • , v K ) which is a finite dimensional subspace of L 2 (U ).

The assumption underlying this approach is that x belongs to (or is well approximated by its projection onto) a finite dimensional subspace V := Span (v 1 , • • • , v K ) of L 2 (U ). This approach is more popular than the non-parametric approach because, in FDA, we are interested in studying the statistical properties of a panel of curves and these properties usually translate well to the coefficients c. The reason behind this is that, if A is a linear operator from V onto itself and A is the K × K matrix such that Av k = ∑ K k =1 A k,k v k , then for all x ∈ V decomposed as (1.3.2), we have Ax(u) = v(u) Ac , u ∈ U .

(1.3.3)

In particular, if b ∈ R K is an eigenvector of the matrix A with eigenvalue λ ∈ R, then b : u → b v(u) is an eigenvector of the operator A.

The complexity of the method lies in choosing an appropriate V, or equivalently, appropriate basis functions and their number. A possible collection of basis functions in the collection of monomials , i.e. v k (t) = t k-1 , which means that V is the set of polynomials of degree k -1. For periodic data, the Fourier basis is a standard choice which presents the advantage of being orthogonal. For non-periodic data, the B-spline [START_REF] Boor | A Practical Guide to Splines[END_REF] collection is usually the preferred choice. In this context, V becomes a subset of the set of spline functions with range U . The B-spline collection can be useful for electrical load data since all B-spline functions are valued in R + . This means that, if we take α 1 , • • • , α K ≥ 0, we are sure that x is valued in R + , although there are more accurate ways to approximate non-negative functions with B-splines (see [START_REF] Hautecoeur | Nonnegative Matrix Factorization over Continuous Signals using Parametrizable Functions[END_REF].

Given a collection of basis functions, the coefficients c 1 , • • • , c K in (1.3.2) are usually estimated by ordinary least squares. If we define the matrix V := [v(u 1 ) , • • • , v(u I ) ] , the estimator writes as ĉ = argmin c∈R K y -Vc 2 2 = (V V) -1 V y .

Registration of functional data

Similarly to time series, functional data can be subject to misalignment. When analyzing a panel of functions, we may want to take this misalignment into account, but in general, it is preferable to re-align the data first. This process is called registration. Formally, we assume that, instead of observing the functions (X t ) 1≤t≤T , we observe time-warped versions (Y t = X t • h t ) 1≤t≤T where h t : U → U is called a warping function. The goal of registration is to find the warping functions which will align the data. If the X t 's are i.i.d and

µ = E [X 1 ],
we can write

Y t = µ • h t + t , t = 1, • • • , T ,
where ( t ) 1≤t≤T is noise. Assuming we know µ, we can then estimate (h 1 , • • • , h T ) by a least-square criterion, i.e.

( ĥ1 , • • • , ĥT ) = argmin h 1 ,••• ,h T T ∑ t=1 U (Y t (u) -µ(h t (u))) 2 du .
This optimization problem can be solved for certain types of warping functions. For example, the least-square criterion is used in Ramsay and Silverman, 2005, Section 7.2.1 in the case where h t is the shift function of parameter δ t , i.e. h t (u) = uδ t . Other more advanced registration methods exist [START_REF] Srivastava | Functional and shape data analysis[END_REF] but are beyond the scope of this manuscript. In the case of EDF data, the only misalignment observed is caused by seasonal clock changes in summer and winter (see Figures 1.2 and 1.3) and can be fixed by a simple one hour shift.

Functional Principal Component Analysis

Functional Principal Component Analysis (fPCA) is one of the most used statistical tools in FDA since it provides the best finite dimensional approximation of the data for the least square error. From a high level point of view, one can think of fPCA as the succession of a linear encoding step and a linear decoding step. The role of encoding is to represent the data as a Kdimensional vector and the role of decoding is to project the encoded data back to the original space. More precisely, if X is a centered random variable in L 2 (Ω, F , H 0 , P), were H 0 is a separable Hilbert space, and K ∈ N * , the problem writes as.

Recall that L b (E, F) denotes the space of bounded operators between two Banach spaces E and F and

L b (E) = L b (E, E). min E X -ΨΦX 2 H 0 : Φ ∈ L b (H 0 , C K ), Ψ ∈ L b (C K , H 0 ) , (1.3.4) or, equivalently min E X -ΘX 2 H 0 : Θ ∈ L b (H 0 ), rank(Θ) ≤ K . (1.3.5)
Let (φ k ) k≥1 be the sequence of eigenvectors of Cov(X) ranked by decreasing order of their related eigenvalues. Then a solution of (1.3.5) is given by (1.3.6) where we recall that, for any u, v ∈ H 0 the operator u ⊗ v ∈ L b (H 0 ) satisfies that (u ⊗ v)z = z, v H 0 u for all z ∈ H 0 . Note that we have Θ = ΨΦ with

Θ = K ∑ k=1 φ k ⊗ φ k ,
Φ =      φ H 1 . . . φ H K      and Ψ = [φ 1 , • • • , φ K ] , (1.3.7)
where φ H : x → x, φ H 0 . More specifically, this means that, for all x ∈ H 0 , Φx = x, φ 1 H 0 , • • • , x, φ K H 0 ∈ C K and for all y ∈ C K , Ψy = ∑ K k=1 y k φ k ∈ H 0 . Hence (Ψ, Φ) is a solution of (1.3.4).

On other words, the best approximation X by a finite dimensional linear transformation is a truncated truncated version of the expansion (1.3.8) where the series converges in L 2 (Ω, F , H 0 , P) as a direct consequence of the convergence of the series in the eigendecomposition of Cov(X). This expansion can be seen as the Hilbertian counterpart of the Karhunen-Loève expansion of a mean square continuous stochastic process (see Bosq (2000, Theorem 1.5)). In particular, we have that ( X, φ k H 0 ) k≥1 is an uncorrelated univariate sequence. In the following, the vectors φ k and the scalar products X, φ k H 0 will be referred to a the loadings and the scores respectively.

X = +∞ ∑ k=1 X, φ k H 0 φ k ,
Similarly to PCA, if we observe a functional dataset (X t ) 1≤t≤T satisfying Cov(X t ) = Cov(X) for all 1 ≤ t ≤ T, the least squares criteria of Equations (1.3.4) and (1.3.5) and the covariance operator are replaced by their empirical counterparts. In practice, assuming H 0 = L 2 (U ) and that we only have access to a discrete version of the function X t , we need to add a second layer of approximation. In this case, there are two ways to perform fPCA. The first is to use only the sampling points and estimate the scalar product X t , φ k H 0 = U X t (u)φ k (u) du by a Riemann integral. If the sampling points are evenly spaced in U , this approach is equivalent to applying PCA to the vectors representing the discretized functions. This method only provides an estimate of the main functional component evaluated at the sampling points. To recover the function, we must use an interpolation method. An alternative is to add a penalty term on the L 2 -norm of the second derivative of ϕ k , which results in natural cubic spline estimators of the functional principal component. The other, and most used, method to perform fPCA in practice is to rely on a function basis representation of the data of the type (1.3.2). In this case, we take a basis of functions (v 1 , • • • , v K ) with, K > K and assume X t (u) = c v(u) as in (1.3.2). Then, the eigenvectors of the empirical covariance operator can be found by computing the eigenvectors of the

K × K matrix Γ = T -1 G 1/2 CC G 1/2 where C = [c 1 , • • • , c T ] ∈ R K ×T and G ∈ R K ×K is the gram matrix of (v 1 , • • • , v K ), i.e. G k,k = v k , v k H 0 .

Functional time series

A functional time series is a sequence of functional random variables (X t ) t∈Z valued in a separable Hilbert space H 0 (for example L 2 (U )) which are not assumed to be i.i.d. The study of such time series is usually done under the assumption of weak stationarity, that is we assume

(i) For all t ∈ Z, X t ∈ L 2 (Ω, F , H 0 , P). (ii) For all t ∈ Z, E [X t ] = E [X 0 ]. We say that X is centered if E [X 0 ] = 0. (iii) For all t, h ∈ Z, Cov (X t+h , X t ) = Cov (X h , X 0 ).
The most classical classes of processes in univariate and multivariate settings are linear processes and the ARMA processes. Naturally, these models have been extended to the functional setting. A (functional) linear process is a sequence of the type

X t = µ + ∑ k∈Z A k t-k , t ∈ Z , (1.3.9)
where µ ∈ H 0 , ( t ) t∈Z is an H 0 -valued white noise i.e. a centered weakly stationary H 0 -valued time series such that Cov ( 0 , h ) = 0 whenever h = 0 and (A k ) k∈Z is a sequence of L b (H 0 ) operators. A thorough study of this class of process is proposed in [START_REF] Bosq | Linear processes in function spaces[END_REF]. Beyond linear processes, extensions of the celebrated autoregressive and moving average processes to the case where the parameters are linear operators have also been studied, see e.g. [START_REF] Bosq | Linear processes in function spaces[END_REF][START_REF] Klepsch | Prediction of functional ARMA processes with an application to traffic data[END_REF][START_REF] Spangenberg | Strictly stationary solutions of ARMA equations in Banach spaces[END_REF].

Another important domain of study for weakly stationary time series is the spectral theory. The principal idea of this theory is to derive a representation of the process by an uncorrelated process indexed by frequencies. The work of Tavakoli in Panaretos and Tavakoli, 2013a,b;[START_REF] Tavakoli | Fourier Analysis of Functional Time Series, with Applications to DNA Dynamics[END_REF] provides major contributions to the generalization of the spectral theory to the functional framework. This work is based on the assumption that (X t ) t∈Z is an H 0 -valued weakly stationary time series with autocovariance operator function Γ X such that there exists

f X ∈ L 1 (T, B(T), S 1 (H 0 ), Leb) satisfying Γ X (h) = T f X (λ)e iλh dλ , h ∈ Z .
(1.3.10)

In this case, using the terminology of Tavakoli (2014, Definition 2.3.1), the function f X defines the collection of weak spectral density operators of (X t ) t∈Z .

The adjective weak emphasizes the fact that f X is only defined almost everywhere and that evaluating f X (λ) at a given frequency λ ∈ T is not possible.

In this manuscript, we call f X the spectral density operator function. In the case where Tavakoli (2014, Proposition 2.3.5) shows that the function f X exists and writes as

∑ h∈Z Γ X (h) L b (H 0 ) < +∞,
f X (λ) = 1 2π ∑ h∈Z Γ X (h)e -iλh , λ ∈ T . (1.3.11)
In particular, (1.3.11) gives that f X is continuous from T to L b (H 0 ). Provided that f X exists and is in L p (T, B(T), S 1 (H 0 ), Leb) for some p ∈ (1, +∞], Tavakoli (2014, Theorem 2.4.3) derives the functional Cramér representation of the process X = (X t ) t∈Z , that is

X t = T e iλt dZ λ , P-a.s. , t ∈ Z , (1.3.12)
where the integral is a Riemann-Stieltjes integral on L 2 (Ω, F , H 0 , P) and (Z λ ) λ∈T is an H 0 -valued process with uncorrelated increments in the sense that Cov (Z

λ 1 -Z λ 2 , Z λ 3 -Z λ 4 ) = 0 for λ 1 > λ 2 ≥ λ 3 > λ 4 .
The process (Z λ ) λ∈T is called the functional Cramér representation of X and is a key concept to define linear filtering in the spectral domain. Given a functional time series X := (X t ) t∈Z which admits a functional Cramér representation of the form (1.3.12), Tavakoli defines in Tavakoli (2014, Section 2.5) an abstract space H of transfer operator functions Φ : λ → Φ(λ) which can be used to filter the process X using a stochastic integral of the form

T e iλt Φ(λ) dZ λ .
Even though Tavakoli (2014, Definition 2.3.1) defines the weak spectral density operators without any additional assumption on the covariance structure of the process, the author proves the existence of the function f X under short-memory assumptions. However, in some cases, the f X function may exist even if the time series has long-range dependence. In the univariate setting, ARFIMA processes fall within this category and can be directly defined in the spectral domain [START_REF] Granger | An introduction to longmemory time series models and fractional differencing[END_REF]. Several generalizations of long-range dependence processes to the functional setting have been proposed recently [START_REF] Characiejus | The central limit theorem for a sequence of random processes with space-varying long memory[END_REF]Račkauskas, 2013, 2014;[START_REF]Limit theorems for Hilbert space-valued linear processes under long range dependence[END_REF]Li, Robinson, and Shang, 2020;[START_REF] Račkauskas | Operator fractional brownian motion as limit of polygonal lines processes in Hilbert space[END_REF]. Finally, in the univariate case, the Cramér representation can still be proved even when no function f X satisfy (1.3.10), see [START_REF] Holmes | Mathematical Foundations of Signal Processing[END_REF] and the references therein. In this case, the measure f X (λ) dλ in (1.3.10) is replaced by a measure on (T, B(T)) which is not dominated by Lebesgue's measure and the result is known as Herglotz's theorem. Its extension to the functional setting is discussed in Delft and Eichler, 2020.

Harmonic fPCA

Harmonic fPCA provides a way to approximate a functional time series by a finite dimensional time series and can be seen as a generalization of Brillinger (2001, Chapter 9) to the functional case. In Harmonic fPCA, instead of looking for an optimal rank-K linear projection of the data, we look for an optimal rank-K linear filter. Let X = (X t ) t∈Z be a centered H 0 -valued weakly stationary time series with functional Cramér representation (Z λ ) λ∈T and K ∈ N * , then the problem writes as (1.3.13) where F Θ (X) is the weakly stationary time series obtained by filtering X with the transfer operator function Θ, i.e. [F Θ (X)] t = T e iλt Θ(λ) dZ λ for all t ∈ Z. Theorem 2.8.2 of Tavakoli, 2014 gives that, under conditions including continuity of the spectral density operator function f X , the minimum in (1.3.13) is achieved for (1.3.14) where (φ k (λ)) k≥1 are the eigenvalues of f X (λ) ranked by decreasing order of their related eigenvalues. Since f X is the spectral counterpart of Cov(X), the transfer operator function defined by (1.3.14) is the spectral counterpart of the projection defined in (1.3.6), hence the name harmonic fPCA.

min E X t -[F Θ (X)] t 2 H 0 : rank(Θ) ≤ K, a.e. ,
Θ(λ) = K ∑ k=1 φ k (λ) ⊗ φ k (λ) , λ ∈ T ,
Similarly of fPCA, if we let K → +∞, we get an expansion of X which is the spectral counterpart of the Karhunen-Loeve expansion (1.3.8) and is referred to as the Cramér-Karhunen-Loève expansion Tavakoli (2014, Theorem 2.8.6). Namely, (1.3.15) where the series converges in L 2 (Ω, F , H 0 , P) and F φ k ⊗φ k is the filter with operator transfer function λ → φ k (λ) ⊗ φ k (λ). We also have that the time series

X t = ∞ ∑ k=1 F φ k ⊗φ k (X) t , t ∈ Z ,
F φ k ⊗φ k (X) is uncorrelated to the time series F φ ⊗φ (X) if = k.
If we write Θ(λ) = Ψ(λ)Φ(λ) in the same way as (1.3.7), we can prove that the filter F Θ is a composition of the two filters F Ψ and F Φ . For this reason the harmonic fPCA can also be seen as an encoding/decoding model where the encoder and decoder are linear filters. This is discussed in Tavakoli (2014, Section 2.8.2) and H örmann, Kidzi ński, and Hallin, 2015 where the time-domain version of the Cramér-Karhunen-Loève expansion. If we call ϕ k, the -th Fourier coefficient of the function φ k , then the k-th summand in (1.3.15) writes as the following composition of two convolutional filters. (1.3.16) and (Y k,t ) t∈Z is uncorrelated to (Y ,t ) t∈Z if = k. In the following, the vectors φ k, and the scalars Y k,t will be referred to as the loadings filters coefficients and the scores respectively. Finally, is practice, when discrete observations are available, the harmonic fPCA can be translated to a matrix form using a basis expansion as discussed in H örmann, Kidzi ński, and Hallin, 2015.

F φ k ⊗φ k (X) t = ∑ ∈Z Y k,t+ ϕ k, with Y k,t = ∑ s∈Z X t-s , ϕ k,s H 0 ,

Statistical inference for functional data

Statistical inference is an important topic in FDA and classic results from the univariate and multivariate settings have been generalized to the functional case. In this section, we assume that a collection of functional random variables (X t ) 1≤t≤T is observed and review existing results on the estimation of statistical objects from the observed data. This requires some assumptions about the dependence structure of the data. The simplest is to assume that the variables are i.i.d. For time-dependent data, the linear model assumption provides a nice framework for inference, but it may be too restrictive.

Other assumptions aim at controlling the dependency between (X t ) t<n and (X t ) t>m as mn → +∞. These assumptions are usually based on α-mixing, L p -m-approximability (see H örmann and Kokoszka, 2010) or higher order cumulant summability assumptions (see Tavakoli (2014, Condition C(l, k))). A review of recent inference results is provided in Table 1.1. Note that these results use an ideal context in which the entire function is observed. The effect of discrete observation for inference in FDA has also been studied in depth for independent data (see e.g. [START_REF] Belhakem | Minimax estimation of Functional Principal Components from noisy discretized functional data[END_REF][START_REF] Hall | Properties of Principal Component Methods for Functional and Longitudinal Data Analysis[END_REF][START_REF] Li | Uniform Convergence Rates For Nonparametric Regression And Principal Component Analysis In Functional/longitudinal Data[END_REF][START_REF] Yao | Functional Data Analysis for Sparse Longitudinal Data[END_REF] but also, more recently for dependent data (see Tavakoli (2014, Section 3.8) and [START_REF] Rubín | Sparsely observed functional time series: estimation and prediction[END_REF]. In this scenario, the estimators must be adapted and non-parametric methods are usually used.

Load curves analysis in the web platform ACDC

The web platform ACDC aims at simplifying load curve analysis carried out by the group. The platform includes tools to visualize load and temperature curves and algorithms to extract information from the load curves based on experts' knowledge. Some tools are also provided to quantify the influence of the temperature. The front-end is developed in Javascript and the backend algorithms are developed in Matlab or Python. During the second part of my PhD, I participated in the implementation of my algorithms in the ACDC platform. In this section, I give an overview of the tools which were present on the platform prior to my PhD. In all the figures, the power is in kW and the temperature is in • C.

Data visualization

When the user enters the platform, he can import the data he wants to analyze. A first graph shown on the platform represents the raw load and temperature data as shown in Figure 1.1. This representation may be useful for understanding general behavior, but other representations are better suited to highlight important signal characteristics. This is the case of the daily and weekly heatmaps displayed in Figures 1.2 and 1.3. In the graph at the center of Figure 1.2, each row represents one day of the year and the graph at the left represents the average temperature for each day. The user can select a particular day and time (here January 11, 2019 at 11:10 a.m.).

The daily load curve of the selected day is displayed in the bottom graph. The graph at the right shows the consumption observed each day at the selected intraday time (here 11:10 a.m.). In short, the bottom graph and the right graph are respectively a horizontal and a vertical slice of the central graph. The weekly heatmap of Figure 1.3 reads the same way. This latter representation is very useful for sites like the one presented because it highlights the fact that there are two regimes. Here, the site is an office and we can interpret the two regimes as working (or opening) and non-working (or closing) days. We can also see the influence of summer holidays as fewer employees are present in the office. Apart from these graphs, the platform proposes other visualization tools to analyze in more detail each day of the week (e.g. time derivative, mean and other statistics, range of values).

Decomposition of load curves

For a more quantitative study, an the platforms includes an algorithm to extract information from the load curves. Let us consider a day of the week d and the associated collection of daily load curves

X d t (u) : u ∈ [0, 24) for t = 1, • • • , T d ,
where T d is the number of times the day d is observed. Of course, in practice, we observe a sampled version of X d t (u 1 ), • • • , X d t (u I ) but seeing the load curve as a function of the intraday time is more intuitive. The algorithm decomposes the load curve as follows.

X d t (u) = P d (u) + V d t (u) , (1.4.1)
where P d (u) and V d t (u) are called the minimum profile and seasonal variation respectively. The minimum profile aims at estimating the daily load curve we would get for the weekday d by observing the consumption of the building placed in a vacuum. This way the minimum profile captures the characteristics of the building by eliminating the influence of time or any external factors which is captured by the seasonal variation. For example, for the site presented in Figure 1.3, if d is Monday, the computation of P d (u) will not take into account the few Mondays where the office is closed (e.g. in November) and summer holidays for which the seasonal variation is negative as observed in Figure 1.4.

The minimum profiles are further decomposed as follows.

P d (u) = H d + A d (u) , where H d ≈ min u∈[0,24) P d (u) . (1.4.2)
The two summands H d , and A d (u) of (1.4.2) are respectively referred to as the heel and the activity. In the example of Figure 1.4, the heel represents the consumption at night when the offices are empty and the activity reflects the effect of human activity in the office. Finally, the seasonal variations are divided into the summer seasonal variation and the winter seasonal variation to compute statistics such as the mean or some quantiles. The summer period is set by default from April 1 to September 30. The user can then visualize the effect of the seasonal variations as shown in Figure 1.5 for Mondays and Saturdays. We see, for example, that the seasonal variation is very low in summer but can be high in winter, especially on weekends. To quantify the importance of each summand in the decomposition of the load curve pie charts are provided for each weekday d individually or jointly, see

Multi-sites analysis

In a multi-sites analysis, features are taken from the decompositions (1.4.3) and (1.4.4) and are used to compare the sites. To take into account differences in size between the buildings, each feature is normalized by the surface of the building. For example, for 12 sites we get the feature matrix of Table 1.2. Some tools such as the histograms of Figure 1.7 are then provided to visualize these features. We can see some disparities in the dataset but the analysis is limited and no automatic clustering method is implemented. Depuis plusieurs années, EDF participe à un processus de transformation du secteur énergétique afin de répondre aux enjeux du changement climatique et vise la neutralité CO 2 à l'horizon 2050. Cette vision est reflétée dans leur projet stratégique Cap 2030, qui repose sur trois piliers : l'innovation pour les clients, le bas carbone et la croissance internationale. Dans ce contexte, EDF porte un grand intérêt à la recherche et dispose de huit sites de R&D, dont trois en France : EDF Lab Paris-Saclay, EDF Lab Les Renardières et EDF Lab Chatou. Sur le premier site, les chercheurs se concentrent sur des sujets variés allant de la mécanique vibratoire à la simulation mathématique et numérique en passant par les relations clients. Les recherches menées sur le deuxième site portent sur l'efficacité énergétique, la sécurité et la fiabilité du réseau électrique. Enfin, le dernier site développe une expertise dans les domaines de l'hydraulique, des énergies renouvelables et nucléaires et de l'environnement.

Mon doctorat a été accueilli par deux départements : le département TREE (plus précisément, le groupe E36) à EDF Lab les Renardières et le département PRISME (plus précisément, le groupe P12) à EDF Lab Chatou.

Le département PRISME (Performance, Risque Industriel et Surveillance pour la Maintenance et l'Exploitation) développe des solutions innovantes d'acquisition de données et de modélisation physique et mathématique pour 33 améliorer la production d'électricité. Le département a deux principaux domaines d'expertise. Le premier est le cycle de vie des moyens de production avec des mesures physiques et des simulations, mais aussi le contr ôle de la sécurité. La seconde expertise est le traitement des données, en particulier le traitement du signal et des images et l'apprentissage statistique, dans le contexte de la production d'énergie. Pour mener à bien ses projets, le département est divisé en six groupes. Le groupe P12 (Systèmes dynamiques, Images et Signaux) est plus étroitement lié à l'expertise en traitement de données. Ses compétences s'articulent donc autour du traitement du signal et de l'image, de l'optimisation et des systèmes dynamiques. Les projets menés concernent des thèmes variés tels que l'hydraulique, la flexibilité thermique ou le cycle de vie.

Le département TREE (Technologie et Recherche pour l'Efficacité Energétique) regroupe 9 groupes travaillant sur différents aspects de la régulation électrique, de l'optimisation des ressources à la modélisation de la consommation des bâtiments. Le département développe des outils à faible émission de CO 2 pour les bâtiments neufs et anciens dans les secteurs résidentiel, tertiaire et industriel. Dans ce contexte, les approches d'intelligence artificielle sont de plus en plus populaires avec des applications allant de la détection de défauts sur les pompes à chaleur et les panneaux solaires à la régulation automatique de l'énergie dans les Smart Buildings. Le groupe E36 (Services & Systèmes Connectés) est spécialisé dans l'Internet des objets (IoT) pour les Smart Buildings. Le groupe intègre notamment la collecte et l'analyse de données dans les écosystèmes des filiales d'EDF (par exemple Sowee, Dalkia, EDF ENR) et réalise de nombreuses études pour des collaborateurs internes et externes. Du point de vue de l'analyse des données, les solutions développées par le groupe consistent principalement en la visualisation et le suivi des consommations, mais aussi en l'analyse des courbes de charge basée sur les connaissances d'experts. Pour cette dernière tâche, le groupe a développé une plateforme web appelée ACDC (Analyse de Courbes De Charges).

Mon travail complète ces approches de deux manières : je propose une approche statistique et orientée données pour l'analyse des courbes de charge et je traite le cas multi-sites o ù la consommation de plusieurs bâtiments est observée. Dans ce contexte industriel, l'objectif pratique de mon travail peut être exprimé comme suit.

(PO) Extraire des informations des courbes de charge électrique multi-sites pour aider à comprendre les différences entre les sites en utilisant une approche non supervisée qui ne fait pas appel aux connaissances des experts mais qui reste facilement interprétable.

L'objectif pratique (PO) pose directement la question de la représentation et de la comparaison des séries temporelles pour lesquelles de nombreuses méthodes ont été proposées dans la littérature des dernières décennies. Dans ce chapitre, je propose un aperc ¸u de ces méthodes, ainsi que des méthodes implémentées sur la plateforme web ACDC.

Représentation et comparaison de séries temporelles

La comparaison des séries temporelles n'est pas aussi facile que celle des données multivariées et il existe de nombreuses méthodes disponibles en fonction du contexte, des contraintes et/ou des connaissances préalables sur les données. Contrairement aux données multivariées, il n'existe pas de distance naturelle pour les séries temporelles. En effet, les mesures habituelles utilisées pour les données multivariées sont souvent basées sur une comparaison par coordonnées et ne sont donc pas appropriées pour comparer des séries temporelles pouvant être échantillonnées différemment. Pour remédier aux problèmes potentiels d'alignement entre deux séries temporelles, des métriques basées sur le Dynamic Time Wrapping (DTW) ont été proposées [START_REF] Sakoe | A Dynamic Programming Approach to Continuous Speech Recognition[END_REF]. Étant donnée une métrique pour comparer les échantillons, DTW calcule le meilleur alignement en minimisant la divergence entre les deux signaux. La divergence minimale définit alors une distance entre les séries temporelles et peut être utilisée dans de nombreuses techniques d'apprentissage automatique. Cependant, le calcul d'une telle distance entre deux signaux de longueurs T 1 et T 2 a une complexité de calcul de O (T 1 T 2 ) et devient trop co ûteux pour être utilisé sur de longues séries temporelles telles que les courbes de charge électrique analysées à EDF qui sont échantillonnées à intervalles de 10 min pendant 1 an et ont donc 51100 échantillons par site. Heureusement, la représentation temporelle d'une série temporelle est généralement de très haute dimension par rapport à l'information utile contenue dans le signal, en particulier en présence de bruit. Par conséquent, il est courant de s'appuyer sur des représentations mieux adaptées qui projettent les signaux dans un espace de dimension inférieure avant de les comparer. Ces approches basées sur les représentations peuvent être divisées en trois catégories.

Comparaison par caractéristiques

La première stratégie consiste à extraire des caractéristiques spécifiques à partir d'hypothèses basées sur des connaissances d'experts ou sur des outils de traitement du signal et de statistique. Ces propriétés peuvent ensuite être quantifiées et comparées pour déterminer la similarité entre deux signaux. Par exemple, dans la plateforme web ACDC, des motifs prédéfinis sont extraits des courbes de charge. En traitement du signal, les représentations de Fourier ou d'ondelettes sont très courantes, mais des représentations basées sur des dictionnaires ou des représentations fonctionnelles sont également possibles. Les méthodes statistiques peuvent se baser, par exemple, sur des coefficients de régression ou des coefficients ARMA. L'un des avantages de cette stratégie est que l'interprétation des résultats peut être facilitée par l'utilisation de connaissances a priori. Cependant, la quantification de ces caractéristiques peut être difficile dans des contextes non stationnaires, bruités ou hétérogènes, ou en présence de valeurs aberrantes ou de données manquantes. De plus, le choix des caractéristiques à utiliser dans la comparaison peut être une tâche longue et fastidieuse qui nécessite plusieurs étapes de test et d'erreur et l'aide de connaissances d'experts. Ce processus est connu sous le nom de feature engineering et peut aboutir à des représentations très spécifiques au type de données analysées. Par exemple, à EDF, il est connu que les caractéristiques importantes différent en fonction du secteur (résidentiel, tertiaire ou industriel).

Méthodes bout-à-bout

La deuxième stratégie de comparaison des signaux consiste à utiliser des méthodes de type bout-à-bout. Ces méthodes prennent le signal brut en entrée et apprennent simultanément une représentation des données et les paramètres d'un modèle qui résout une tâche spécifique. Une méthode populaire est l'apprentissage profond [START_REF] Goodfellow | Deep Learning[END_REF] o ù des représentations successives des données sont apprises dans les couches internes et sont utilisées par la couche de sortie pour effectuer une tâche donnée (e.g. régression, classification). Contrairement à la première stratégie, les caractéristiques extraites par les méthodes de bout en bout ne sont pas connues a priori et sont apprises directement à partir des données. Malgré leur polyvalence, ces méthodes présentent souvent l'inconvénient d'être difficiles à interpréter car la représentation est apprise pour effectuer la tâche et non pour l'interprétation. Par exemple, il est généralement difficile de relier les couches internes d'un réseau neuronal aux propriétés du signal original. En outre, ces méthodes nécessitent de grandes quantités de données d'apprentissage car le nombre de paramètres peut être très élevé. Il convient de noter que d'autres méthodes bout-à-bout, peuvent donner des résultats interprétables mais nécessitent des données étiquetées [START_REF] Mairal | Task-Driven Dictionary Learning[END_REF][START_REF] Mairal | Supervised Dictionary Learning[END_REF].

Comparison par motifs

Enfin, la dernière catégorie de méthodes vise à extraire des motifs récurrents des données. Contrairement aux méthodes bout-à-bout, cela se fait généralement sans avoir de tâche spécifique en tête, mais en utilisant des hypothèses générales sur les motifs, telles que l'orthogonalité, la positivité, l'indépendance ou la parcimonie. Comme ces hypothèses ne sont généralement pas spécifiques aux données, ces méthodes sont très polyvalentes et populaires pour de nombreuses applications pratiques. La plupart de ces méthodes donnent une approximation du signal ou du signal centré x t comme suit [START_REF] Jolliffe | Principal component analysis[END_REF] qui donne des loading orthogonaux et des facteurs décorrélés (généralement appelés scores). Parmi les autres méthodes populaires, citons l'analyse en composantes indépendantes (ACI) qui recherche des sources indépendantes en se basant sur la maximisation de la non-gaussianité [START_REF] Cardoso | Source separation using higher order moments[END_REF][START_REF] Cardoso | Blind beamforming for non-Gaussian signals[END_REF] ou sur des méthodes de second ordre [START_REF] Belouchrani | A blind source separation technique using second-order statistics[END_REF][START_REF] Yeredor | Blind separation of Gaussian sources via second-order statistics with asymptotically optimal weighting[END_REF]. Enfin, l'utilisation de contraintes de positivité sur les motifs et les activations conduit à la factorisation non-négative de matrices (NMF) [START_REF] Lee | Algorithms for Non-negative Matrix Factorization[END_REF]) et les contraintes de parcimonie conduisent à l'apprentissage de dictionnaires parcimonieux [START_REF] Lee | Efficient sparse coding algorithms[END_REF] [START_REF] Grosse | Shift-Invariant Sparse Coding for Audio Classification[END_REF]. [START_REF] Ferraty | Nonparametric functional data analysis[END_REF][START_REF] Horváth | Inference for Functional Data with Applications[END_REF][START_REF] Ramsay | Functional data analysis[END_REF][START_REF] Wang | Functional Data Analysis[END_REF]. Ces outils ont trouvé des applications dans divers domaines allant de l'imagerie médicale (Lila and Aston, 2020) et de la biophysique [START_REF] Tavakoli | Detecting and localizing differences in functional time series dynamics: a case study in molecular biophysics[END_REF] à la démographie [START_REF] Hyndman | Robust forecasting of mortality and fertility rates: A functional data approach[END_REF] et à la linguistique [START_REF] Tavakoli | A spatial modeling approach for linguistic object data: analyzing dialect sound variations across Great Britain[END_REF] .

x t = R ∑ r=1 a r s

Le cadre des séries temporelles fonctionnelles

y i = x(u i ) + i ; , i = 1,
Une autre méthode de lissage non paramétrique très populaire se base sur l'ajout d'un terme de pénalité à l'erreur quadratique, typiquement la norme L 2 de la dérivée seconde de x, en admettant que cette dérivée seconde soit dans L 2 (U ). L'estimateur s'écrit alors

x = argmin x n ∑ i=1 (y i -x(u i )) 2 + λ U x (
, • • • , c K ∈ R tels que x(u) = K ∑ k=1 c k v k (u) = c v(u) ; , u ∈ U ; , (1.3.2) o ù c := [c 1 , • • • , c K ] et v(u) := [v 1 (u), • • • , v K (u)] . L'
× K telle que Av k = ∑ K k =1 A k,k v k , alors pour tout x ∈ V décomposé comme (1.3.2), nous avons Ax(u) = v(u) Ac ; , u ∈ U ; . (1.3.3)
En particulier, si b ∈ R K est un vecteur propre de la matrice A associé à la valeur propre λ ∈ R, alors b : u → b v(u) est un vecteur propre de l'opérateur A.

La complexité de la méthode réside dans le choix d'un V approprié, ou de manière équivalente, de fonctions de base appropriées et de leur nombre. Une collection possible de fonctions de base est la collection des monômes , c'est-à-dire v k (t) = t k-1 , ce qui signifie que V est l'ensemble des polyn ômes de degré k -1. Pour les données périodiques, la base de Fourier est un choix standard qui présente l'avantage d'être orthonormale. Pour les données non périodiques, les fonctions B-splines [START_REF] Boor | A Practical Guide to Splines[END_REF] sont, en général, un bon choix. Dans ce contexte, V devient un sous-ensemble de l'ensemble des fonctions splines définies sur l'intervalle U . La base des fonctions B-splines peut être utile pour les données de charge électrique elles sont à valeurs positives. Ainsi, prendre α 1 , • • • , α K ≥ 0, garantit que x est à valeurs positives. Il existe néanmoins des moyens plus précis d'approcher des fonctions non négatives avec des B-splines (voir [START_REF] Hautecoeur | Nonnegative Matrix Factorization over Continuous Signals using Parametrizable Functions[END_REF].

Étant donnée une collection de fonctions de base, les coefficients c 1 ,

V := [v(u 1 ) , • • • , v(u I ) ] , l'estimateur s'écrit comme suit ĉ = argmin c∈R K y -Vc 2 2 = (V V) -1 V y ; .

Alignement des données fonctionnelles

( ĥ1 , • • • , ĥT ) = argmin h 1 ,••• ,h T T ∑ t=1 U (Y t (u) -µ(h t (u)))
L b (E) = L b (E, E). min E X -ΨΦX 2 H 0 : Φ ∈ L b (H 0 , C K ), Ψ ∈ L b (C K , H 0 ) ; , (1.3.4) ou, de manière équivalente min E X -ΘX 2 H 0 : Θ ∈ L b (H 0 ), rank(Θ) ≤ K ; . (1.3.5)
Soit (φ k ) k≥1 la suite des vecteurs propres de Cov(X) classés par ordre décroissant de leurs valeurs propres associées. Alors une solution de (1.3.5) est donnée par

Θ = K ∑ k=1 φ k ⊗ φ k ; , (1.3.6) o ù nous rappelons que, pour tout u, v ∈ H 0 l'opérateur u ⊗ v ∈ L b (H 0 ) satisfait (u ⊗ v)z = z, v H 0 u pour tout z ∈ H 0 . Notons que Θ = ΨΦ avec Φ =      φ H 1 . . . -phi H K      et Ψ = [φ 1 , • • • , φ K ] ; , (1.3.7) o ù φ H : x → x, φ H 0 . Plus précisément, cela signifie que, pour tout x ∈ H 0 , Φx = x, φ 1 H 0 , • • • , x, φ K H 0 ∈ C K et pour tout y ∈ C K , Ψy = ∑ K k=1 y k φ k ∈ H 0 .
(v 1 , • • • , v K ) avec K > K et supposons que X t (u) = c v(u) comme dans (1.3.2).
Alors, les vecteurs propres de l'opérateur de covariance empirique peuvent être obtenus en calculant les vecteurs propres de la matrice

K × K Γ = T -1 G 1/2 CC G 1/2 o ù C = [c 1 , • • • , c T ] ∈ R K ×T et G ∈ R K ×K est la matrice de gramme de (v 1 , • • • , v K ) i.e. G k,k = v k , v k H 0 .

Séries temporelles fonctionnelles

Une série temporelle fonctionnelle est une suite de variables aléatoires fonctionnelles (X t ) t∈Z à valeurs dans un espace de Hilbert séparable H 0 (par exemple L 2 (U )) qui ne sont pas supposées i.i.d. L'étude de telles séries temporelles est généralement effectuée sous l'hypothèse de stationnarité faible, c'est-à-dire que nous supposons que

(i) Pour tout t ∈ Z, X t ∈ L 2 (Ω, F , H 0 , P). (ii) Pour tout t ∈ Z, E [X t ] = E [X 0 ]. On dit que X est centrée si E [X 0 ] = 0. (iii) Pour tout t, h ∈ Z, Cov (X t+h , X t ) = Cov (X h , X 0 ).
Dans les cadres univarié et multivarié, les types de séries temporelles les plus populaires sont les processus linéaires et les processus ARMA. Ces modèles ont été naturellement étendus au cadre fonctionnel. Un processus linéaire (fonctionnel) est une suite du type

X t = µ + ∑ k∈Z A k t-k , t ∈ Z , ( 1
-Z λ 2 , Z λ 3 -Z λ 4 ) = 0 pour λ 1 > λ 2 ≥ λ 3 > λ 4 .
The process (Z λ ) λ∈T is called the functional Cramér representation of X and is a key concept to define linear filtering in the spectral domain. Étant donnée une série temporelle fonctionnelle X := (X t ) t∈Z qui admet une représentation de Cramér fonctionnelle de la forme (1.3.12), Tavakoli définit dans Tavakoli (2014, Section 2.5) un espace abstrait H de fonctions de transfert à valeurs opérateurs Φ : λ → Φ(λ) qui peuvent être utilisées pour filtrer le processus X sous la forme d'un intégrale du type T e iλt Φ(λ) dZ λ .

Même si Tavakoli (2014, Definition 2.3.1) définit les opérateurs de densité spectrale faible sans aucune hypothèse supplémentaire sur la structure de covariance du processus, l'auteur prouve l'existence de la fonction f X sous des hypothèses mémoire courte. Cependant, dans certains cas, la fonction f X peut exister même si la série temporelle a une dépendance longue. Dans le cadre univarié, les processus ARFIMA entrent dans cette catégorie et peuvent être directement définis dans le domaine spectral [START_REF] Granger | An introduction to longmemory time series models and fractional differencing[END_REF]. Plusieurs généralisations des processus à mémoire longue ont été proposées récemment dans le cadre fonctionnel [START_REF] Characiejus | The central limit theorem for a sequence of random processes with space-varying long memory[END_REF]Račkauskas, 2013, 2014;[START_REF]Limit theorems for Hilbert space-valued linear processes under long range dependence[END_REF]Li, Robinson, and Shang, 2020;[START_REF] Račkauskas | Operator fractional brownian motion as limit of polygonal lines processes in Hilbert space[END_REF]. Enfin, dans le cas univarié, la représentation de Cramér peut être prouvée même si aucune fonction f X ne satisfait (1.3.10), voir [START_REF] Holmes | Mathematical Foundations of Signal Processing[END_REF] et les références qui s'y trouvent. Dans ce cas, la mesure f X (λ) dλ dans (1.3.10) est remplacée par une mesure sur (T, B(T)) qui n'est pas dominée par la mesure de Lebesgue et le résultat est connu sous le nom de théorème d'Herglotz. Son extension au cadre fonctionnel est discutée dans Delft and Eichler, 2020.

ACPf harmonique

L'ACPf harmonique fournit un moyen d'approcher une série temporelle fonctionnelle par une série temporelle de dimension finie et peut être considérée comme une généralisation de Brillinger (2001, Chapitre 9) au cas fonctionnel. Dans l'ACPf harmonique, au lieu de chercher une projection optimale des données de rang K, on cherche un filtre linéaire optimal de rang K. Soit X = (X t ) t∈Z une série temporelle faiblement stationnaire centrée à valeurs dans H 0 admettant pour représentation de Cramér fonctionnelle Enfin, dans la pratique, lorsque seules des observations discrètes sont disponibles, l'ACPf harmonique peut être traduite sous forme matricielle à l'aide d'une décomposition dans une base comme discuté dans H örmann, Kidzi ński, and Hallin, 2015.

(Z λ ) λ∈T et K ∈ N * , alors le problème s'écrit comme suit min E X t -[F Θ (X)] t 2 H 0 : rank(Θ) ≤ K, a.e.
Θ(λ) = K ∑ k=1 φ k (λ) ⊗ φ k (λ) , λ ∈ T , ( 1
F φ k ⊗φ k (X) t = ∑ ∈Z Y k,t+ ϕ k, with Y k,t = ∑ s∈Z X t-s , ϕ k,s H 0 , ( 1

Inférence statistiques pour les données fonctionnelles

L'inférence statistique est un sujet important dans l'ADF et les résultats classiques des contextes univariés et multivariés ont été généralisés au cas fonctionnel. Dans cette section, nous supposons qu'une collection de variables aléatoires fonctionnelles (X t ) 1≤t≤T est observée et nous passons en revue les résultats existants sur l'estimation d'objets statistiques à partir des données observées. Cela nécessite quelques hypothèses sur la structure de dépendance des données. La plus simple est de supposer que les variables sont i.i.d. Pour les données dépendantes, l'hypothèse du modèle linéaire fournit un cadre agréable pour l'inférence, mais elle peut être trop restrictive. D'autres hypothèses visent à contr ôler la dépendance entre (X t ) t<n et (X t ) t>m comme mn → +∞. Ces hypothèses sont généralement basées sur l'hypothèse α-mixing, l'approximabilité L p -m (voir H örmann and Kokoszka, 2010) ou des hypothèses de sommabilité des cumulants d'ordre supérieurs (voir Tavakoli (2014, Condition C(l, k))). Une revue des résultats d'inférence récents est fournie dans le tableau 1.1 (voir version en anglais du chapitre). Notez que ces résultats utilisent un contexte idéal dans lequel la fonction est entièrement observée. L'effet des observations discrète pour l'inférence a également été étudié en profondeur dans le cas des données indépendantes (voir [START_REF] Belhakem | Minimax estimation of Functional Principal Components from noisy discretized functional data[END_REF][START_REF] Hall | Properties of Principal Component Methods for Functional and Longitudinal Data Analysis[END_REF][START_REF] Li | Uniform Convergence Rates For Nonparametric Regression And Principal Component Analysis In Functional/longitudinal Data[END_REF][START_REF] Yao | Functional Data Analysis for Sparse Longitudinal Data[END_REF] mais aussi, plus récemment pour les données dépendantes (voir Tavakoli (2014, Section 3.8) et [START_REF] Rubín | Sparsely observed functional time series: estimation and prediction[END_REF]. Dans ce cas, les estimateurs doivent être adaptés et des méthodes non paramétriques sont généralement utilisées.

Analyse des courbes de charge sur la plateforme web ACDC

La plateforme web ACDC vise à simplifier l'analyse des courbes de charge réalisée par le groupe. La plateforme comprend des outils pour visualiser les courbes de charge et de température et des algorithmes pour extraire des informations des courbes de charge sur la base des connaissances des experts. Certains outils sont également fournis pour quantifier l'influence de la température. Le front-end est développé en Javascript et les algorithmes du back-end sont développés en Matlab ou Python. Durant la deuxième partie de mon doctorat, j'ai participé à l'intégration de mes algorithmes dans la plateforme ACDC. Dans cette section, je donne un aperc ¸u des outils qui étaient présents sur la plateforme avant mon doctorat. Dans toutes les figures, rassemblées dans la version en anglais du chapitre, la puissance est en kW et la température est en • C.

Visualisation des données

Lorsque l'utilisateur entre dans la plateforme, il peut importer les données qu'il souhaite analyser. Un premier graphique affiché sur la plateforme représente les données brutes de charge et de température comme indiqué dans la Figure 1

∑ t=1 24 0 X t (u) du = ∑ τ∈{open,closed 24H τ + A τ + ∑ s∈{winter, summer V τ s :, (1.4.4) o ù H τ = ∑ daytype(d)=τ H d et de même pour A τ et V τ s .

Analyse multi-sites

Corrélation avec la température extérieure

Un autre outil fourni par la plateforme ACDC permet à l'utilisateur de visualiser la dépendance entre la puissance et la température pour chaque jour de la semaine. La dépendance est modélisée par une régression linéaire ou polynomiale comme le montre la Figure 1.8.

When working with real-world data, it is customary to spend time exploring the dataset and the modeling options available. This exploratory step is necessary for a variety of reasons. First, real-world data generally have various flaws such as missing data or outliers which require some pre-processing before further analysis. Additionally, becoming familiar with the data is helpful in the modeling process, in particular to understand which important characteristics need to be modeled. At this stage, working with experts is an advantage because their experience can guide the models towards more interpretable results. In the case of my doctorate, modeling the data as a functional time series seemed logical and the first approach followed relied on fPCA and harmonic fPCA. For this reason, parallel to the exploratory analysis, I became interested in the study of the spectral theory of functional time series which led to the work presented in Part ii of this manuscript. This Chapter presents the successive steps taken during this exploratory analysis which led us to change our approach to better meet the challenges of our main practical objective (PO).

Presentation of the dataset and pre-processing

The dataset analyzed at EDF consists of electrical load curves of 181 supermarkets (mentioned as "sites") across France over a period of one year at a sampling rate of 10 minutes as well as daily average outdoor temperatures for each site. In this section, we mainly discuss the mono-site setting and denote, for a given site, {X t (u) : u ∈ [0, 24)} the daily load curve observed at day t ∈ 1, T and T t the average daily temperature. Figure 2.1 presents the weekly heatmaps of one site and illustrates the two main issues which were found in the data. The first one is missing values (gray in Figure 2.1) and the second one is values very close to 0 (blue in Figure 2.1). These low values correspond to periods during which the supermarket uses a personal 53 generator, for example, if the price of electricity is too high. From Figure 2.1 we see that missing values or generator usage may extend over several hours or even days. The selection of an exploitable subset of the 181 sites followed three steps after which 108 were remaining. In the first step, we manually rejected sites showing abnormal behavior such as unexpected drops in consumption that may occur in case of sensor malfunction. The second phase was to identify the periods during which the generator is used and to treat them as missing entries. Finally, we eliminate sites that had more than one week of missing entries. The generator detection is illustrated in Figure 2.2 and follows three steps.

1. We first detect which days have a low minimum value. More precisely, defining Y t := min u∈[0,24) X t (u), we select the days such that Y t ≤ τ, where

τ = 0.1 × median(Y t , t = 1, • • • , T).
2. Then for each of the selected days, we remove the intra-day times such that X t (u) ≤ τ.

3. Finally, we smooth the curve using a median filter and remove the points which are too far below the smoothed curve. This last step aims at removing points which can remain because, when the generator is used, the consumption does not drop instantaneously.

Finally, we impute missing values by a 2D interpolation of the weekly heatmap representation (see Figure 2.3). The last pre-processing step consisted in smoothing the daily load curves as discussed in Section 1.3.2 by projecting them onto a basis of 30 B-splines (see Figure 2.4). 2.2 the two approaches followed

The two approaches followed

At the beginning of my PhD, the objective was to extract features from the data using tools adapted to the functional time series framework. This framework is built upon weak-stationarity assumptions on the time series which, in particular, imply that the mean is constant over time. A look at the load data of Figures 2.1 and 2.5 shows clearly that this is not the case. The most obvious source of non-stationarity comes from the fact that two consumption regimes are observed: opening days (mostly from Monday to Saturday) and closing days (mostly on Sundays). It is therefore necessary to incorporate the consumption regime in the estimation of the expectation of the time series. To this end, let us introduce a latent series (W t ) 1≤t≤T corresponding to the consumption regime, with W t = 0 for closing days and W t = 1 for opening days. Another source of non-stationarity may come from the temperature since the power is highly correlated to the temperature. It may therefore also be useful to take the temperature into account when estimating the expectation. We propose to follow two approaches.

• First approach : without temperature. In the first approach, we estimate the expectation of X t conditionally to W t , i.e.

µ w (u) := E [ X t (u)| W t = w] , u ∈ [0, 24), w ∈ {0, 1} .
(2.2.1)

Then we apply fPCA or harmonic fPCA to the centered time series X t (u)µ W t (u).

• Second approach : with temperature. In the second approach, we estimate the expectation of X t conditionally to (W t , T t ), i.e.

µ w (u; τ) := E [ X t (u)| W t = w, T t = τ] , u ∈ [0, 24), w ∈ {0, 1}, τ ∈ R .
(2.2.2) Then we apply fPCA or harmonic fPCA to the centered time series X t (u)µ W t (u; T t ).

Before following any of these approaches, we must estimate the series (W t ) t=1,••• ,T . To this end, we focus on the behavior of the daily mean data since the two consumption regimes can be well observed in the daily mean plots of Figure 2.5. This is discussed in the next section.

First order analysis of daily means

The distinction between regimes is not as simple as the discrimination between different days of the week on the ACDC platform. For example, the supermarket can open on a few Sundays, especially at the end of the year. For this reason, we propose another method to detect consumption regimes by simultaneously improving the modeling of the dependence between daily power and temperature presented in Section 1.4.4. The model can be seen as a daily mean counterpart of (2.2.2). Denoting by Xt = 1 24 24 0 X t (u) du, the mean value of the curve X t , then we estimate μw (τ) = 24 0 µ w (u; τ) du .

(2.3.1)

In addition, for each regime, we model the variance of the residuals by a mixture of two Gaussians which aim at dividing the data into two sub-regimes corresponding to points which are close and far from the μ. We interpret these sub-regimes as "normal" and "extreme" behaviors. These sub-regimes are modeled by a latent sequence (Z t ) 1≤t≤T . The model writes as follows.

Xt = μW t (T t ) + σ W t ,Z t t , t = 1, • • • , T , (2.3.2)
where for all (w, z) ∈ {0, 1} 2 , σ w,z > 0 and ( t ) 1≤t≤T iid ∼ N (0, 1). We also assume that (W t , Z t ) 1≤t≤T are i.i.d and independent of (T t , Xt ) 1≤t≤T and that for all t, ∈ 1, T , Xt is independent of ( X , T ) conditionally to T t . The parameter of this model is denoted by η and is decomposed as η = (α, θ) where α = (α 0 , α 1 ), θ = (θ 0 , θ 1 ) and for all w ∈ {0, 1} α w = P (W t = w) and θ w = μw , β w,0 , β w,1 , σ 2 w,0 , σ 2 w,1 , with for all z ∈ {0, 1}

β w,z = P [ Z t = z | W t = w] .
We estimate the parameter η as well as the labels (W t , Z t ) 1≤t≤T using a hierarchical scheme where we alternate between the two following steps 1. Update the main clustering i.e. the estimators α and Ŵ1:T of α and W 1:T .

2.

For each value w ∈ {0, 1}, fit the regression model and clustering of sub-regimes on t : Ŵt = w .

Formally, the main iterations are described in Algorithm 2.3.1. Details of each step are provided in the next sections. The results obtained by this algorithm are illustrated in Figure 2.6 for one site of EDF's dataset. Blue and orange correspond respectively to Z t = 0 (normal) and Z t = 1 (extreme) when W t = 0 and green and red correspond respectively to Z t = 0 and Z t = 1 when W t = 1.

k ← 0 repeat αk+1 , ( Ŵk+1 t ) 1≤t≤T ← update main clustering ηk for w = 0, 1 do I k+1 w ← t ∈ 1, T : Ŵk+1 t = w . θk+1 w , Ẑk+1

Centering without temperature

Let us now present the results obtained following the first centering approach. The expectation of (2.2.1) is estimated by its empirical counterpart, i.e.

μw (u) = ∑ T t=1 1 W t =w X t (u) ∑ T t=1 1 W t =w , u ∈ [0, 24) , w ∈ {0, 1} .
The centering process is illustrated in Figure 2.7. We then apply fPCA and harmonic fPCA to the centered time series

X c t (u) := X t (u) -μW t (u) , u ∈ [0, 24) .

Results of fPCA

We extracted the K = 3 first functional principal components which explain 65, 10 and 4 percent of the variance respectively. We propose to visualize the results in three ways. Figure 2.8 presents the fPCA loadings and scores i.e the functions φ k and the coefficients X c k , φ k H 0 of Relation (1.3.8). Figure 2.9 illustrates the effect of each loadings φ k on the mean μw similarly to the visualization proposed in Ramsay and Silverman (2005, Section 8.3.1). Here we display μw + Cφ k for C ranging from min t: 2.10 shows the truncated Karhunen-Loeve expansion of three days at different truncation levels, i.e the curves μW t + ∑ K k=1 X c t , φ k φ k for K = 1, 2, 3. The first striking result is that most of the data variability is explained by the first principal component whose scores are highly correlated to the temperature. In Figure 2.8, we see that the first principal component scores are mostly negative in winter and positive in summer. It is interesting to see how the first principal component loading affects the consumption as presented in the upper left graph of Figure 2.9. Here, we see that when

W t =w X c t , φ k H 0 to max t:W t =w X c t , φ k H 0 φ k . Finally, Figure

Results of harmonic fPCA

Let us now investigate whether harmonic fPCA and the Cramér-Karhunen-Loève expansion (1.3.15) can be interesting to achieve the practical objective (PO). In practice, the Cramér-Karhunen-Loève expansion is truncated at two levels. The first is the component level where we select K = 3 as in fPCA and the second is for the convolutions in (1.3.16) where we truncate the convolutions at lag ±L with L = 19 = T 1/2 as proposed in H örmann, Kidzi ński, and Hallin, 2015. Overall, the truncated Cramér-Karhunen-Loève expansion writes as

Xt (u) = µ W t (u) + K ∑ t=1 L ∑ =-L Y k,t+ ϕ k, (u) with Y k,t = L ∑ s=-L X c t-s , ϕ k,s H 0 .
In fact, it is not necessary to take the same amount of lags in the two convolutions. It can be helpful for the interpretation to known which lags are important in the construction of Xt (u). Since

∑ ∈Z ϕ k, 2 H 0 = 1 2π T ϕ(λ) 2 H 0 dλ = 1 ,
we can select L with an elbow method on the graph of

∑ L =-L ϕ k,

2

H 0 . As we see in Figure 2.11a, most of the norm of the filter is explained by the first lags. This can also be observed in Figure 2.11b where the reconstruction with the first harmonic principal component, i.e. μW t (u)

+ ∑ L =-L Y 1,t+ ϕ 1, (u)
, is displayed for several values of L. We see that much of the variation in the curve is added by the first lags and that increasing L only brings subtle changes to the reconstructed curve. For this reason, and to facilitate the interpretation, we only display the loadings filters coefficients for lags = -1, 0, +1. These loadings filters coefficients are presented in Figure 2.12 as well as the related scores Y k,t . As an attempt to interpret the loadings filters coefficients, we using the method proposed H örmann, Kidzi ński, and Hallin (2015, Figure 6) where we display, in Figure 2.13, the curves μ1 2.10 shows the truncated Cramér-Karhunen-Loève of the same days as in Figure 2.10, at different truncation levels, i.e μW t + ∑ K k=1 ∑ L =-L Y k,t+ ϕ k, for K = 1, 2, 3. Finally, in Figure 2.15 we compare the reconstructed series obtained by the truncation of the Karhunen-Loeve and Cramér-Karhunen-Loève expansion. The relative error is defined as

+ C(δ -1 ϕ 1,-1 + δ 0 ϕ 1,0 + δ 1 ϕ 1,1 ) with C > 0 (here C = 1 T ∑ T t=1 |Y k,t |) for δ i = ±1. Finally, Figure
∑ T t=1 Xt -X t 2 H 0 ∑ T t=1 X t 2 H 0 .
Similarly to fPCA, the component explaining the most of the variability (67%) in the data is highly correlated to the temperature and the other two are not correlated to the temperature and explain only 10% and 5% of the variability respectively. The scores in Figure 2.12 look very similar to the scores of Figure 2.8 and the loadings of Figure 2.8 are also very similar to some loading filters coefficients of Figure 2.12. In fact, with a closer look, we can interpret the k-th fPCA loading φ k as a linear combination of the k-th harmonic fPCA loading filter coefficients. For example, φ 1 can be decomposed as the sum of a constant curve (i.e ϕ 1,0 ) and a curve which peaks during the afternoon (i.e ϕ 1,1 ). This additional decomposition allows harmonic fPCA to recover more subtle variation in the curves as it can be seen in Figure 2.15. However, the gain in reconstruction is not so important (0.6% of gain in the relative error) compared to the loss in the interpretation of the results. Indeed, interpreting the role of each filter coefficient is not straightforward. Looking at Figure 2.13 we can state that, when the scores at days t and t + 1 are low (i.e. δ 0 = δ 1 = -1), the consumption is below the mean and has a heating profile. On the contrary when the scores at days t and t + 1 as high (i.e. δ 0 = δ 1 = 1), the consumption is above the mean and has a cooling profile. The other cases correspond to transitions between these two cases. Again, this is consistent with the fact that the scores for the first component are mostly positive in summer and negative in winter. Finally, we see in the reconstruction plots of Figure 2.14 that the Cramér-Karhunen-Loève reconstructs more subtle changes in the curves and thus approaches better the true load curve. Moreover, the effect of the second and third components is less obvious than for the Karhunen-Loève expansions of Figure 2.10.

+ C(δ -1 ϕ 1,-1 + δ 0 ϕ 1,0 + δ 1 ϕ 1,1 ).

Centering with temperature

In the second approach, we treat the temperature as an explanatory variable for the mean and investigate how this impacts the information extracted from fPCA. We propose to define μw (•; τ) as a local average of the load curves X t such that T t is close to τ. In addition, we take advantage of the "normal" and "extreme" regimes detection discussed in the last section by only taking into account the days which are considered as "normal", i.e. Z t = 0. We get the following Nadaraya-Watson estimator.

μw (u; τ) = ∑ T t=1 κ(T t , τ)1 W t =w (1 -Z t )X t (u) ∑ T t=1 κ(T t , τ)1 W t =w (1 -Z t ) , u ∈ [0, 24), τ ∈ R, w ∈ {0, 1} ,
(2.5.1) where κ(τ, τ ) = exp -γ(ττ ) 2 and γ = 1/8. The estimated mean μw is displayed in Figure 2.16 and, as expected, for opening days we observe two behaviors when the temperature increases. The first one is a rise of the overall consumption and the second is a change of shape in the load curve from a heating profile to a cooling profile. Applying fPCA to the centered load curves X t (u) -μW t (u; T t ) then provides the loadings and scores of Figure 2.17. By removing the effect of the temperature, we see that only the constant part of the first loading from Figure 2.8 is left in the first loading of Figure 2.17. The other two loadings are very similar in both approaches, as they do not depend on the temperature.

The fact that the first principal score is almost constant means that a positive score at day t should imply that X t (u) is above µ W t (u; T t ) for all u ∈ [0, 24) and a negative t should imply that X t (u) is below µ W t (u; T t ). In fact, we see in Figure 2.17 that the points where Z t = 1 in the model of (2.3.2) are characterized by large absolute value of their scores. This is consistent with our previous observation that only the first component scores are correlated to the temperature. For another comparison, we display in Figure 2.18 the truncated Karhunen-Loève of the same three days as in Figure 2.10.

Discussion and conclusion

Overall, the exploratory analysis of the data gives rise to the following conclusions.

• Conclusion 1. We stressed the importance of taking into account the temperature to explain temporal variations. In particular, the first fPCA component falls from 67% to 35% of variance explained when the temperature is included in the centering.

• Conclusion 2. Applying harmonic fPCA does not bring a high approximation gain over fPCA and is more difficult to interpret due to its convolutional nature. On the contrary, the Karhunen-Loève decomposition is more easily interpreted by users accustomed to the ACDC platform because it provides a decomposition similar to those already implemented there. Moreover, it can be argued that the time dependence is mainly explained by the correlation with the temperature. This can be seen by noting that the decay of the autocorrelation is much faster when taking into account the temperature in the centering as observed in Figure 2. [START_REF] Yao | Functional Data Analysis for Sparse Longitudinal Data[END_REF] where the autocorrelation at lag h is defined as These first two conclusions demonstrate that the time series approach may not be the best fit for our data and that it may be more appropriate to focus on modeling temperature dependence rather than time dependence.

E X c t+h , X c t 2 H 0 E X c t 2 H 0 , where X c t is either X t -μW t or X t -μW t (•, T t ). (a) Autocorrelation of X t -μW t (b) Autocorrelation of X t -μW t (•, T t )
In addition, other types of models may be considered beyond fPCA to overcome some of its drawbacks on which we now elaborate. If we compare the fPCA approach and the experts' approach implemented on the ACDC platform, we note that both explain how the real daily load curves deviate from a reference curve. However, this reference curve is not the same in the two approaches. While the experts use the minimum profile defined in Section 1.4, fPCA uses the mean and therefore cannot explain potentially valuable information about the influence of external factors on the consumption which may be contained in the mean. This is all the more true if we take the temperature into account in the centering step. In addition, assuming that we take the temperature into account in the mean, then the mean is the most important statistic for energy monitoring. In fact, the total energy ratio explained by the variation around the mean is

∑ T t=1 24 0 |X t (u) -μW t (u; T t )| du ∑ T t=1 24 0 X t (u) du ≈ 0.075 ,
which is not significant in our industrial context. Finally, the absence of sign constraints on the loadings and scores can mislead the interpretation of the components. To illustrate this last point, Figure 2.20 shows the successive plots of µ 1 (•; 15) + C ∑ K k=1 φ k for K = 0, • • • , 3, an arbitrary C > 0 and where φ 1 , φ 2 , φ 3 are the fPCA loading displayed in Figure 2.17. Focusing on the region between 10 a.m. and 8 p.m., we observe that the components compensate each other since the consumption increases when we add Cφ 1 , then decreases when we add Cφ 2 and increases again when we add Cφ 3 . These

•; 15) + C ∑ K k=1 φ k with C > 0 for K = 0, • • • , 3.
observations lead to the final conclusion of the exploratory analysis.

• Conclusion 3. A different approach using a structured non-negative decomposition of the daily load curve taking into account the effect of temperature should allow us to address the drawbacks of fPCA.

In this Chapter, we briefly summarize the main contributions of this thesis and provide some thoughts about possible perspectives.

In parallel to the exploratory analysis described in Chapter 2, I got interested in studying the spectral theory recently introduced for functional time series, in particular in Tavakoli, 2014. Part ii of this manuscript presents our contributions to this field of research. These contributions are mainly based on revisiting the spectral theory using a Gramian isometry approach. By introducing adapted mathematical objects, we show that this theory can be clarified and completed using a minimal amount of assumptions. This part summarized in Section 3.1.1 and is based on the two following academic contributions.

C1. Amaury Durand, Franc ¸ois Roueff. Weakly stationary stochastic processes valued in a separable Hilbert space: Gramian-Cramér representations and applications. 2021. hal-02318267v4

C2. Amaury Durand, Franc ¸ois Roueff. Hilbert valued fractionally integrated autoregressive moving average processes with long memory operators. 2020. hal-02961227

In Part iii, we treat the practical objective (PO) by taking into account the conclusions of Chapter 2. A model based on a functional non-negative tensor factorization is proposed along with the updates for the corresponding optimization problem. We show empirically that this model is helpful to extract smooth intraday patterns and take into account the external temperature data and can be used to cluster the sites. Then, we propose a theoretical study of the aforementioned optimization problem by quantifying the impact of smoothness on tensor factorization with missing data. This part is summarized in Section 3.1.2 and is centered around the following academic contribution.

C3. Amaury Durand, Franc ¸ois Roueff, Jean-Marc Jicquel, Nicolas Paul. Smooth nonnegative tensor factorization for multi-sites electrical load monitoring. EUSIPCO, Aug 2021, Dublin, Ireland. hal-03167498v2

C4. Amaury Durand, Franc ¸ois Roueff, Jean-Marc Jicquel, Nicolas Paul. Smoothness constraints for non-negative tensor factorization with missing values. In progress.

In addition, I have participated in the development of the two following tools for the ACDC platform.

C5. A mono-site analysis tool based on Chapter 2 and the GUI presented in Appendix E (in progress).

C6. A multi-sites analysis tool based on non-negative tensor factorization models (in progress).

General mathematical background and algorithmic details are gathered in the appendix. Some perspectives on functional models for multi-site data are proposed in Section 3.2.

Summary of the thesis

Part ii : Theoretical aspects of functional time series.

The work presented in this part is aligned with the recent works on spectral theory for Hilbert-valued processes (see Delft andEichler, 2018, 2020;Panaretos and Tavakoli, 2013a,b;[START_REF] Tavakoli | Fourier Analysis of Functional Time Series, with Applications to DNA Dynamics[END_REF] and on functional longmemory processes (see [START_REF] Characiejus | The central limit theorem for a sequence of random processes with space-varying long memory[END_REF]Račkauskas, 2013, 2014;[START_REF]Limit theorems for Hilbert space-valued linear processes under long range dependence[END_REF]Li, Robinson, and Shang, 2020;[START_REF] Račkauskas | Operator fractional brownian motion as limit of polygonal lines processes in Hilbert space[END_REF].

Chapter 4: the general stone and bochner theorems.

In this chapter, we introduce the main tools from functional analysis and measure theory on vector and operator spaces which are needed to construct the spectral theory in its most general form. In particular, we discuss the general Bochner theorem for operator-valued functions and provide a comparison with the recently proved general Herglotz theorem of Delft and Eichler, 2020. An overview of the literature on Stone's and Bochner's theorems allow us to highlight the close relationship between these theorems and their generalizations to normal Hilbert modules and operator-valued functions.

Chapter 5: weakly stationary stochastic processes valued in a separable hilbert space.

This chapter gathers the contribution of C1. and develops a spectral theory for stochastic processes valued in a separable Hilbert space H 0 . Elements of this theory were proposed in Panaretos and Tavakoli, 2013a,b using restrictive summability assumption on the autocovariance operator function. This assumption was relaxed in Tavakoli, 2014 where the author assumes the existence of a spectral density operator function which is L p (T, B(T), S 1 (H 0 ), Leb) for some p ∈ (1, +∞]. Finally, with their general Herglotz theorem, the authors of Delft and Eichler, 2020 provide an extension to the case where no spectral density operator exist.

The approach we present in this chapter is based on the tools developed in Chapter 4 and the earlier works of [START_REF] Kakihara | Multidimensional Second Order Stochastic Processes[END_REF][START_REF] Kallianpur | Spectral theory of stationary Hvalued processes[END_REF][START_REF] Mandrekar | The Square-integrability of Operatorvalued Functions with Respect to a Non-negative Operator-valued Measure and The Kolmogorov Isomorphism Theorem[END_REF] In this approach, we develop the spectral theory with minimal assumptions while avoiding unnecessary abstractions such as the completion of a pre-Hilbert space or on the compactification of a pointed convex cone as used in Tavakoli (2014, Section 2.5) and Delft and Eichler, 2020, respectively. This approach also clarifies and completes the isomorphic relationship between the modular spectral domain and the modular time domain by exploiting the normal Hilbert module property of the Bochner space

H := L 2 (Ω, F , H 0 , P) of random variables V : Ω → H 0 such that E V 2 H 0 < ∞.
For a weakly stationary stochastic process X := (X t ) t∈G ∈ H G indexed by a locally Abelian group G, this chapter provides the construction of the modular spectral domain of the process X as a space of operator-valued transfer functions and its comparison to the spaces of operator-valued transfer functions used in recent works. This construction lead to the key result on spectral theory, namely the Gramian-Cramér representation,

X t = Ĝ χ(t) X(dχ) , t ∈ G ,
and its interpretation as a Gramian-isometric operator between the modular time domain and the modular spectral domain.

Chapter 6 : applications to linear filtering and long-mem-

ory processes

In this chapter, we use the spectral theory developed in Chapter 5 in three applications. First, we derive interesting results on the composition and inversion of linear filters with an operator-valued transfer functions. Second, we provide a version the Cramér-Karhunen-Loève decomposition and harmonic fPCA with minimal assumptions. These two first applications were presented in Contribution C1.. Finally, we define a class of fractionally integrated autoregressive moving average processes valued in a separable Hilbert space using a spectral approach as presented in Contribution C2.. In this framework, the usual univariate long memory parameter d is replaced by a long memory operator D acting on the Hilbert space. Our approach is compared to processes defined in the time domain that were previously introduced for modeling long range dependence in the context of functional time series, see [START_REF] Characiejus | The central limit theorem for a sequence of random processes with space-varying long memory[END_REF]Račkauskas, 2013, 2014;[START_REF]Limit theorems for Hilbert space-valued linear processes under long range dependence[END_REF]Li, Robinson, and Shang, 2020;[START_REF] Račkauskas | Operator fractional brownian motion as limit of polygonal lines processes in Hilbert space[END_REF]

Part iii : Multi-sites electrical load disaggregation and clustering.

In the second part of my PhD, in order to achieve the objective (PO), I proposed a model based on non-negative tensor factorization which takes into account additional temperature data and smoothness constraints. This model presents the advantage of being easily interpretable. In Part iii, we propose and study a non-negative tensor factorization model for multi-sites electrical load curves disaggregation in order to achieve the practical objective (PO).

Chapter 7 : smooth non-negative tensor factorization for multi-sites electrical load monitoring.

We propose to model the multi-sites electrical load curves using a functional formulation of nonnegative tensor factorization. Suppose we observe a dataset

{(X n,t , T n,t , n,t ) : n ∈ 1, N , t ∈ 1, T } ,
where X n,t : u → X n,t (u), T n,t and n,t represent respectively the daily load curve, the average daily temperature and the consumption regime (e.g. opening vs closing days) observed at day t for the n-th site. Then, we model the daily load curve by the factorization

X n,t (u) ≈ R ∑ r=1 a r (u)b r (T n,t )c n,t r,n , where a r (u) ≥ 0, b r (τ) ≥ 0, c r ≥ 0.
The idea is to represent the load curves as a mixture of R sources. Each source is composed of a daily consumption pattern a r (u) ≥ 0 called the signature which is modulated across temperatures by a thermal activation b r (T n,t ) ≥ 0. The weights of the mixture are represented by the site activation c n,t n,r ≥ 0 which also depends on the consumption regime to take into account that different sources may be used for opening and closing days. In this chapter, we discuss the optimization problem resulting from this model and show on concrete examples how it is useful to extract interpretable patterns from the electrical load curves.

Chapter 8 : smoothness constraints for non-negative tensor factorization with missing values.

The optimization problem resulting from the model of Chapter 7 can be seen as a weighted version of the standard NTF problem where vanishing weights represent missing values. In this chapter, we study this problem from a theoretical point of view. It has been shown in [START_REF] Lim | Optimal solutions to non-negative parafac/multilinear nmf always exist[END_REF][START_REF] Lim | Nonnegative approximations of nonnegative tensors[END_REF] that the standard NTF problem always admits a global solution. In the weighted case, we show that this it not necessarily the case and provide a sufficient condition on the smoothness of the factors to ensure this existence. Then we propose an algorithm to solve the resulting optimization problem.

Some perspective for other functional multi-sites models

The conclusions of the exploratory analysis lead us to address the practical objective (PO) using the non-negative tensor factorization model presented in Part iii. However, other ideas have been suggested from the exploratory analysis to deal with the multi-sites setting. These ideas were not pursued further but are briefly discussed in this section.

In the multi-sites setting, we observe a panel of functional time series {(X n,t ) t∈Z : n ∈ 1, N } where N is the number of sites X n,t ∈ H 0 = L 2 (U ). The ideas considered rely on a factor model with common loadings. Namely, we assume

X n,t = K ∑ k=1 a k S k,n,t + n,t , t ∈ Z , n ∈ 1, N , (3.2.1)
where a k ∈ H 0 , (S k,n,t ) t∈Z is a univariate centered weakly-stationary time series and ( n,t ) t∈Z is a centered weakly-stationary H 0 -valued noise uncorrelated to (S k,n,t ) t∈Z . This way, objective (PO) can be achieved by comparing the K-variate scores time series

([s 1,n,t , • • • , s K,n,t ]
) t∈ 1,T and interpreting the loadings a k 's as patterns. Note that, with model (3.2.1), (X n,t ) t∈Z is centered and weakly-stationary. Let us denote Γ n = Cov(X n,t ). Then we propose to estimate the a k 's and S k,n,t 's by minimizing the average mean-square errors across sites, i.e.

1

N N ∑ n=1 E   X n,t - K ∑ k=1 a k S n,k,t 2 H 0   . (3.2.2)
By the definition of the orthogonal projection on Span (a 1 , • • • , a K ), if suf- fices to take a 1 , • • • , a K orthonormal and S n,k,t = X n,t , a k H 0 and the problem reduces to the eigendecomposition of the average of the covariance operators N -1 ∑ N n=1 Γ n . This method is equivalent to applying fPCA to the whole dataset without distinction between days and sites. In the finite dimensional case where H 0 = R p , this is the Simultaneous Component Analysis with common Patterns (SCA-P) proposed in [START_REF] Kiers | Hierarchical relations between methods for simultaneous component analysis and a technique for rotation to a simple simultaneous structure[END_REF]. The advantage of this method is that it is easy to implement in the functional setting and we also get an infinite expansion

X n,t = +∞ ∑ k=1 a k S k,n,t (3.2.3)
where the series converges in L 2 (Ω, F , H 0 , P). However, contrary to the Karhunen-Loève expansion, the scores are not necessarily uncorrelated because we only have

N -1 ∑ N n=1 Cov (S k,n,t , S ,n,t ) = 0 if k = .
Several constrained versions of the SCA-P model are also proposed in Timmerman and Kiers, 2003, including one which assumes that the scores are uncorrelated. Unlike SCA-P, the addition of constraints makes the solution analytically intractable and (3.2.2) is minimized by a least-square alternating algorithm. It would be interesting to study how this constrained problem generalizes to the functional case. Namely, is there a way to estimate one of the models (3.2.1) or (3.2.3) with the additional constraint that Cov (S k,n,t , S ,n,t ) = 0 for k = ? With the more restrictive constraint of independence, this is closely linked to ICA whose functional extensions are still largely unexplored (see [START_REF] Li | Functional independent component analysis : an extension of fourthorder blind identification[END_REF] for some recent work). With the uncorrelation constraint, it seems that generalizing joint-diagonalization ideas used in second order ICA [START_REF] Belouchrani | A blind source separation technique using second-order statistics[END_REF][START_REF] Cardoso | Jacobi Angles for Simultaneous Diagonalization[END_REF] or in Common PCA [START_REF] Flury | Common Principal Components in K Groups[END_REF] could be a promising idea. In fact, ideas from Flury, 1984 have already been extended to the functional setting in [START_REF] Boente | Inference under functional proportional and common principal component models[END_REF] Finally, in the finite dimensional case, if we further decompose S k,n,t as S k,n,t = b k,n C k,t , the decomposition (3.2.1) fits into the field of tensor factorization based on the CANDECOMP/PARAFAC decomposition [START_REF] Carroll | Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition[END_REF][START_REF] Harshman | Foundations Of The Parafac Procedure: Models And Conditions For An" Explanatory" Multimodal Factor Analysis[END_REF][START_REF] Kruskal | Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics[END_REF] and functional extensions of these models could be investigated.

To conclude this section, let us note that the multi-sites setting can also be seen in the point of view of multivariate functional data analysis where other factor models have already been considered. For example, both multivariate fPCA (mfPCA) introduced in Jacques and Preda, 2014 and the factor model introduced in Tavakoli, Nisol, and Hallin, 2021 provide a decomposition of the type

X n,t = K ∑ k=1 a k,n S k,t + n,t . (3.2.4) Part II T H E O R E T I C A L A S P E C T S O F F U N C T I O N A L T I M E S E R I E S
This part gathers theoretical results on the spectral theory for weakly stationary processes valued in a separable Hilbert. These processes have known renewed interest in the last decade and we decided to revisit previously introduced mathematical approaches to provide important insights which clarify the recent literature. The first goal of this work is, therefore, to complete the understanding of the mathematical objects and relations which form this theory. This is the purpose of Chapters 4 and 5. In Chapter 4, we introduce adapted mathematical tools such as normal Hilbert modules and positive operator valued measures. As a result, we also discuss the general Bochner theorem as discussed in recent works on this topic. Then, in Chapter 5, with the help of earlier approaches, we propose to exploit the normal Hilbert module property of the space of Hilbert-valued random variables with finite second-order moment in order to describe the isomorphic relationship between the modular spectral domain and the modular time domain. The relationship takes the form of a Gramian-Cramér representation whose construction complements recent works on the subject. As a consequence, we propose three applications in Chapter 6. The first consists in providing useful results on the composition and inversion of laginvariant linear filters. Then, we derive the Cramér-Karhunen-Loève decomposition and harmonic functional principal component analysis in their most general forms. Finally, we introduce a novel class of fractionally integrated autoregressive moving average processes. These processes have been widely and successfully used, in both the time and spectral domains, to model univariate and multivariate time series exhibiting long range dependence. Functional extensions of these processes have also been studied more recently using time domain approaches and we propose to use a spectral domain approach to extend this class of models to Hilbert valued processes.

I N T R O D U C T I O N A N D M O T I VAT I O N
Spectral theory for weakly stationary time series has been originally developed in a very general fashion, starting from the seminal works by Kolmogoroff, 1941, and spanning over several decades, see [START_REF] Holmes | Mathematical Foundations of Signal Processing[END_REF] and the references therein. These foundations include time domain and frequency domain analyses, Cramér (or spectral) representations, the Herglotz theorem and linear filters. In [START_REF] Holmes | Mathematical Foundations of Signal Processing[END_REF][START_REF] Kolmogoroff | Stationary sequences in Hilbert's space[END_REF] the adopted framework is that of a bi-sequence X = (X t ) t∈Z ∈ H Z valued in a Hilbert space (H, •, • H ) and weakly stationary in the sense that X s , X t H only depends on the lag st. In this framework, a linear filter is a linear operator on H X onto H X which commutes with the lag operator U X , where H X is the closure in H of the linear span of (X t ) t∈Z and U X is the operator defined on H X by mapping X t to X t+1 for all t ∈ Z. As explained in Holmes (1979, Section 3), a complete description of such a filter is given in the spectral domain by its transfer function. Let us recall the essential formulas which summarize what this means. In [START_REF] Holmes | Mathematical Foundations of Signal Processing[END_REF], the spectral theory follows from and start with the canonical representation of the lag operator U X above, namely

U X = T e iλ ξ(dλ) , (ii.1)
where T = R/(2πZ) and ξ is the spectral measure of U X (which is a measure valued in the space of operators on H X onto itself). This corresponds to Holmes (1979, Eq. ( 8)) with a slightly different notation. Then defining X as ξ(•)X 0 (thus a measure valued in H X ), one gets the celebrated Cramér representation (see Holmes (1979, Eq. (13a)) again with a slightly different notation)

X t = T e iλ t X(dλ) , t ∈ Z . (ii.2)
An other consequence of (ii.1) is what is called the Herglotz theorem in Holmes (1979, Eq. ( 9)), summarized by the formula

X s , X t H = T e iλ (s-t) µ(dλ) , s, t ∈ Z , (ii.3)
where µ = ξ(•)X 0 , X 0 H is a non-negative measure on (T, B(T)). Interpreting the right-hand side of (ii.3) as the scalar product of the two functions e s : λ → e iλs and e t : λ → e iλt in L 2 (T, B(T), µ), Relation (ii.3) is simply 87 saying that the Cramér representation (ii.2) mapping e t to X t is isometric.

Following this interpretation, one can extend this isometric mapping to a unitary operator between the two isomorphic Hilbert spaces L 2 (T, B(T), µ) and H X , respectively refered to as the spectral domain and the time domain. In particular the output of a linear filter with transfer function Φ ∈ L 2 (T, B(T), µ) is given by (ii.4) or in other words, Y t is the image of the function e t Φ by the extended unitary operator that maps the spectral domain to the time domain. Relations (ii.2)-(ii.4) and the isometric relation between the time and spectral domains are considered as the pillars of spectral theory and apply to any Hilbert space H. This theory is applied mainly to the case where H = L 2 (Ω, F , P) the space of scalar random variables on the probability space (Ω, F , P) which have finite variance. However, it also applies to multivariate time series by taking H = L 2 (Ω, F , C q , P) = (L 2 (Ω, F , P)) q and to functional time series by taking H = L 2 (Ω, F , H 0 , P), where H 0 is an infinite dimensional separable Hilbert space, thus isomorphic to, and often taken to be, the function space L 2 (0, 1) of Lebesgue-square-integrable functions on [0, 1]. This last setting has gained renewed interest in the past decade as the field of functional data analysis becomes more popular in the statistical community. Generalizations of the spectral theory to weakly stationary functional time series have been considered in Delft andEichler, 2018, 2020;Panaretos and Tavakoli, 2013a,b;[START_REF] Tavakoli | Fourier Analysis of Functional Time Series, with Applications to DNA Dynamics[END_REF] as summarized in Section 1.3.5. At first sight, it is fair to question the difference between the functional case and the general framework studied in the original works that founded the modern theory of stochastic processes. This issue cannot be unequivocally answered because there are (many) different approaches to establishing a spectral theory for weakly stationary functional time series. Moreover, the merits and the drawbacks of a specific approach depend on the applications that one wishes to deduce from the spectral theory at hand and on the required mathematical tools in which one is ready to invest in order to rigorously employ it. Nevertheless, the limitations of the approach of [START_REF] Kolmogoroff | Stationary sequences in Hilbert's space[END_REF][START_REF] Holmes | Mathematical Foundations of Signal Processing[END_REF] to deal with multivariate time series (and functional time series even more so) are already mentioned in Holmes (1979, Section 7), where the author argues that important generalizations are needed. In order to better understand these generalizations, one must first look at the multivariate case which has been reviewed in [START_REF] Masani | Recent trends in multivariate prediction theory[END_REF] In particular, Masani stresses the importance of the Gramian structure of the product space H = L 2 (Ω, F , C q , P) = (L 2 (Ω, F , P)) q . The Gramian matrix between two vectors

Y t = e iλt Φ(λ) X(dλ) , t ∈ Z ,
V = (V (1) , • • • , V (q) ) ∈ H and W = (W (1) , • • • , W (q) ) ∈ H is the q × q matrix [V, W] H with entries V (k) , W (l) H 1≤k,l≤q
which coincides with the covariance matrix if V or W are centered. Using this Gramian structure, Relations (ii.1)-(ii.4) are easily adapted by strengthening the weak stationarity to impose that [X s , X t ] H only depends on st (see Masani (1966, Section 5)). This stronger weak stationarity assumption not only ensures that the lag operator U X is (scalar product) isometric on H X but also that it is Gramian-isometric on the larger space. Span

H (PX t , t ∈ Z, P ∈ C q×q ). Follow-
The superscript H in the notation Span H is used to emphasize the fact that the closure is taken in H ing the same approach, the development of a spectral theory of functional time series relies on exhibiting a Gramian structure for H = L 2 (Ω, F , H 0 , P) making it a normal Hilbert module and replacing the time domain space H X by the modular time domain

H X = Span H (PX t , t ∈ Z, P ∈ L b (H 0 )) , (ii.5)
where L b (H 0 ) denotes the space of bounded operators on L b (H 0 ) onto itself.

In comparison, in the definition of H X used in [START_REF] Holmes | Mathematical Foundations of Signal Processing[END_REF], P is restricted to be a scalar operator. Thus, while H X is a subspace of H seen as a Hilbert space, H X is a submodule of H seen as a normal Hilbert module. The goal of this part is to show that, by introducing suitable notions such as normal Hilbert modules and Positive Operator Valued Measures, the theory developed in Delft andEichler, 2018, 2020;Panaretos and Tavakoli, 2013a,b; Tavakoli, 2014 can be clarified and completed. More precisely, we propose the following path to achieve and fully exploit a Cramér representation on H X .

Step 1) Interpret the representation (ii.1) as the one of a Gramian-isometric operator on H X (and not only an scalar product isometric operator on H X ).

Step 2) Deduce that the Cramér representation (ii.2) can effectively be extended as a Gramian-isometric operator mapping H 0 → H 0 -operator valued functions on (T, B(T)) to an element of H X .

Step 3) As a first consequence, the scalar product isometric relation

(ii.3) is extended to [X s , X t ] H = T e iλ (s-t) ν(dλ) , s, t ∈ Z , (ii.6)
where, here, ν is an operator valued measure on (T, B(T)). This Gramian-isometric relationship corresponds to what is called the Herglotz theorem in the functional time series case.

Step 4) As a second consequence, the Cramér representation (ii.4) of a linear filter is extended to the case where the transfer function Φ is now an H 0 → H 0 -operator valued functions on (T, B(T)) (and not only a scalar valued functions on (T, B(T))). This raises the question, in particular, of the precise condition required on the transfer function to replace the condition Φ ∈ L 2 (T, B(T), µ) of the scalar case.

The results of Step 1) to Step 4) establish the spectral theory for functional time series and are addressed in Chapter 5, up to the following slight modifications.

1. We treat the more general case of a stochastic process (X t ) t∈G , where (G, +) is a locally compact Abelian (l.c.a.) group set of indices and for each t ∈ G, X t is a random variable defined on a probability space (Ω, F , P) and valued in a separable Hilbert space H 0 (endowed with its Borel σ-field). Typical examples for G and H 0 are the ones of functional time series, namely G = Z and H 0 = L 2 (0, 1) but, as far as spectral theory is concerned, the presentation of the results is not only more general (one can e.g. take G = R) but also more elegant in this general setting. We recall in Section 4.2 the definition of the dual group Ĝ of continuous characters on G. Of course, in the discrete time case G = Z, any continuity condition imposed on a function defined on G is trivially satisfied. Such continuity conditions constitute a small price to pay (and the only one) in order to be able to treat the case of a general l.c.a. group G rather than focusing on the discrete time case alone.

2. For obvious practical reasons, it is usual to treat the mean of a stochastic process separately. Therefore we will assume that the process (X t ) t∈G is centered.

3. We will consider the case where the separable Hilbert space G 0 in which the output of the filter is valued is different from H 0 , the one of the input, that is, we replace P ∈ L b (H 0 ) in (ii.5) by P ∈ L b (H 0 , G 0 ), the space of bounded operators from H 0 to G 0 . This makes the results directly applicable in the case of different input and output spaces, especially in the case where they have different dimensions (so that they are not isomorphic).

The approach to derive a spectral theory following Step 1) to Step 4) is essentially contained in [START_REF] Kakihara | Multidimensional Second Order Stochastic Processes[END_REF][START_REF] Kallianpur | Spectral theory of stationary Hvalued processes[END_REF][START_REF] Mandrekar | The Square-integrability of Operatorvalued Functions with Respect to a Non-negative Operator-valued Measure and The Kolmogorov Isomorphism Theorem[END_REF] Our main contribution concerning these steps is to introduce all the preliminary definitions required to understand them, to select the most important results, to provide detailed proofs of the key points and to bring forward this approach in order to promote what we believe to be a more powerful, complete and easy to exploit approach than the more recently proposed ones in Delft andEichler, 2018, 2020;Panaretos and Tavakoli, 2013a,b;[START_REF] Tavakoli | Fourier Analysis of Functional Time Series, with Applications to DNA Dynamics[END_REF]. A very useful benefit of the Gramianisometric approach is that it allows a concrete description of the spectral domain rather than relying on the completion of a pre-Hilbert space or on the compactification of a pointed convex cone as used in Tavakoli (2014, Section 2.5) and Delft and Eichler, 2020, respectively. A greater benefit, however, is to make the Cramér representation much easier to exploit for deriving useful general results. This will be made apparent when establishing the results gathered in the following additional steps and treated in Chapter 6.

Step 5) An interesting consequence of Step 4) is to study the composition of linear filters and deduce when and how it is possible to inverse them.

Step 6) An other interesting consequence of Step 2) is to derive the Cramér-Karhunen-Loève decomposition and the harmonic principal component analysis for any weakly stationary functional time series valued in a separable Hilbert space.

Step 7) We exploit the results of Step 4) and Step 5) to prove existence of certain classes of processes, in particular long-memory processes, based on a spectral approach.

To our best knowledge, the results on composition and inversion of filters appear to be novel in this degree of generality. Similarly, our versions of the Cramér-Karhunen-Loève decomposition and harmonic functional principal component analysis are not restricted to the case where the spectral density operator has none or finitely many points of discontinuity as in Delft and Eichler, 2020; Tavakoli, 2014. Finally, the last step introduced a new class of long-memory processes in the functional setting.

As previously mentioned, each approach has its drawbacks and the main drawback of the one we are presenting here is probably that it requires lengthier, although not intrinsically difficult, preliminaries. In particular we need to precisely recall definitions of operator valued measures and Gramian-isometric operators on normal Hilbert modules. All these definitions are assembled in Chapter 4 along with the useful facts about l.c.a. groups. In this chapter, we also discuss the generalization of Bochner's theorem for operator-valued functions which contains Step 3) and will be compared to the generalization of Herglotz's theorem proposed in Delft and Eichler, 2020. Then Step 1)-Step 4) are followed in Chapter 5 and Step 5)-

Step 7) are discussed in Chapter 6. All definitions and notation concerning operators can be found in Appendix A.

T H E G E N E R A L S T O N E A N D B O C H N E R T H E O R E M S
This chapter gathers preliminary results needed to follow Step 1) to Step 7).

In particular, we recall definitions of vector and operator valued measures, normal Hilbert modules and locally compact Abelian groups. As a consequence, we discuss the generalization of Bochner's theorem (and Herglotz's theorem a fortiori) for operator-valued functions and Stone's theorem for normal Hilbert modules. A similar generalization of Herglotz's theorem was proposed recently in Delft and Eichler, 2020 using a different, more complex, construction of operator-valued measures than the one described here (p.o.v.m.'s). Although both concepts rely on a similar idea, which is to characterize an operator-valued measure by a set of scalar measures, and turn out to be equivalent up to an isomorphism between monoids, we argue that the objects introduced in this section are more elegant and easier to work with for two reasons. The first one is that we do not rely on the compactification of a pointed convex cone. The second one is that the theory of p.o.v.m.'s has already been established decades ago (see e.g. Berberian, 1966b) and largely used in fields such as quantum mechanics. It is also a key ingredient of Stone's theorem which is used to derive Relation (ii.1). In fact, in this framework, the general Bochner theorem can be traced back to the 40's and different versions and proofs can be found in the literature, see e.g. Berberian, 1966a;[START_REF] Neumark | Positive definite operator functions on a commutative group[END_REF], Theorem VII of Fillmore (1970, Chapter 8) and Proposition 5 of Kakihara (1997, Section 2.5). It is also contained in common proofs of Stone's theorem [START_REF] Ambrose | Spectral resolution of groups of unitary operators[END_REF][START_REF] Arnous | Sur les groupes continus de transformations unitaires de l'espace de Hilbert: Une extension d'un théorème de M.H. Stone[END_REF][START_REF] Salehi | Stone's Theorem for a Group of Unitary Operators Over a Hilbert Space[END_REF] or Theorem VI of Fillmore (1970, Chapter 8). This chapter aims at introducing all the necessary tools to understand the proof of this theorem. By doing so, we highlight the close relationship between Bochner's theorem and its generalization and Stone's theorem and its generalization to normal Hilbert modules.

(A n ) n∈N ∈ A N of disjoint sets, µ( n∈N A n ) = ∑ n∈N µ(A n ),
where the series converges in the norm topology of E, that is,

lim N→+∞ µ n∈N A n - N ∑ n=0 µ(A n ) E = 0 .
Proposition 4.1.1 (Variation of a measure). Let (Λ, A) be a measurable space, (E, • E ) a Banach space and µ an E-valued measure. Then the mapping

µ E : A → sup ∑ i∈N µ(A i ) E : (A i ) i∈N ∈ A N is a countable partition of A
defines a non-negative measure on (Λ, A) called the variation measure of µ.

When µ E is finite, integrals of functions in L 1 (Λ, A, µ E ) with respect to µ are easily defined by first considering simple functions and by extending the obtained linear form to the whole space L 1 (Λ, A, µ E ) by continuity, see Dinculeanu (1967, P. 120). It is still possible to define integrals for measures with non-finite variation but this will not be necessary for our purpose.

When Λ is a locally-compact topological space and A is the Borel σ-field of Λ, endowing the class of subsets of Λ with a well-suited topology induces a notion of continuity for E-valued measures (see Dinculeanu (1967, Chapter III, §15)). This is called regularity. Definition 4.1.2 (Regularity). Let Λ be a locally-compact topological space and A the Borel σ-field of Λ. Then an E-valued measure µ defined on A is said to be regular if for all A ∈ A and > 0, there exist a compact set K ∈ A and an open set

U ∈ A with K ⊂ A ⊂ U such that µ(U \ K) E ≤ .
The definition of regularity for non-finite (non-negative) measures is similar but the property is only required for A such that µ(A) < +∞.

From the straightforward inequality µ(A) E ≤ µ E (A) for all A ∈ A, we get that, if µ is an E-valued measure with finite and regular variation, then µ is regular. An interesting result (see Kakihara (1997, Remark 3.6.2)) is that an E-valued measure µ is regular if and only if, for any element φ in the continuous dual E * of E, the mapping φ • µ is a regular complex measure.

The final notion we discuss is absolute continuity and densities of vectorvalued measures. If E is a Banach space and (Λ, A) a measurable space, then an E-valued measure is said to be absolutely continuous with respect to a non-negative σ-finite measure ν defined on the same space if for all A ∈ A,

ν(A) = 0 ⇒ µ(A) = 0. The absolute continuity of µ with respect to ν is denoted by µ ν. Note that µ ν is and only if µ E ν. We say that E has the Radon-Nikodym property if µ ν is equivalent to the existence of a function f ∈ L 1 (Λ, A, E, ν) such that µ(A) = A f dν , A ∈ A ,
In this case, f is called the density of µ with respect to ν and is unique up to a ν-null set. This is denoted by is f = dµ dν or dµ = f dν. Radon-Nikodym's theorem states that C has the Radon-Nikodym property but this is not the case of all Banach spaces. In Diestel and Uhl (1977, Chapter III) the authors study the properties that E must satisfy in order to have the Radon-Nikodym property. For example, Theorem 1 in Diestel and Uhl (1977, Chapter III, Section 3) states that any separable dual space has the Radon-Nikodym property.

Positive Operator Valued Measures (p.o.v.m.'s)

A Positive Operator Valued Measure (p.o.v.m.) is a measure valued in the space L + b (H) of positive bounded linear operators on a Hilbert space H. Similarly to vector-valued measures, defining a p.o.v.m. relies on a notion of σ-additivity for a well-suited topology. For p.o.v.m.'s, the weak operator topology (w.o.t.) is chosen and the definition writes as follows. We refer the reader to Appendix B for the definitions of the weak and strong operator topologies.

Definition 4.1.3 (Positive Operator Valued Measures (p.o.v.m.)). Let (Λ, A) be a measurable space and H be a Hilbert space. A Positive Operator Valued Measure (p.o.v.m.)

on (Λ, A, H) is a mapping ν : A → L + b (H) such that for all sequences of disjoint sets (A n ) n∈N ∈ A N , ν n∈N A n = ∑ n∈N ν(A n ) (4.1.1)
where the series converges in L + b (H) for the weak operator topology (w.o.t.).

It is interesting to note that, due to properties of positive operators, the convergence of the series in (4.1.1) in w.o.t. implies its convergence for the strong operator topology (s.o.t.), see Berberian (1966b, Proposition 1). However, the series does not necessarily converge in operator norm which implies that, in this definition, a p.o.v.m. does not need to be an L b (H)-valued measure. Therefore the above definitions of integrals and regularity cannot be applied. This is circumvented by noting that a p.o.v.m. is entirely characterized by the family of non-negative measures ν x : A → x H ν(A)x : x ∈ H . This is proved in Berberian (1966b, Theorems 1 and 2). Based on this characterization, we introduce two definitions related to p.o.v.m.'s, the first one about the regularity property and the second one about integrals of bounded scalar valued functions. Definition 4.1.4 (Regular p.o.v.m.). Let Λ be a locally-compact topological space with Borel σ-field A and H be a Hilbert space. Then a p.o.v.m. ν on (Λ, A, H) is said to be regular if for all x ∈ H, the non-negative measure ν

x : A → x H ν(A)x is regular.
An alternative equivalent definition of regular p.o.v.m.'s is Berberian (1966b, Definition 14), see also Theorem 20 in the same reference. We now define the integral of bounded functions with respect to a p.o.v.m.. Definition 4.1.5 (Integral of a scalar valued function with respect to a p.o.v.m.). Let (Λ, A) be a measurable space, H be a Hilbert space, ν be a p.o.v.m. on (Λ, A, H) and define for all x ∈ H, the non-negative measure ν x : A → x H ν(A)x. Let f : Λ → C be a bounded and measurable function. Then the integral of f with respect to ν is the unique operator in L b (H), denoted by f (λ) ν(dλ), such that for all x ∈ H,

x H f (λ) ν(dλ) x = f (λ) ν x (dλ) .
The existence of the operator f (λ) ν(dλ) in Definition 4.1.5 through the mapping x → x H f (λ) ν(dλ) x is straightforward, see Berberian (1966b, Theorem 9). The integral in Definition 4.1.5 is only valid for bounded functions. Generalizing this integral to unbounded functions is complicated. Nevertheless, when dealing with spectral operator measures of weakly stationary processes valued in a separable Hilbert space, we can rely on the additional trace-class property, which makes all the previous definitions easier to handle and extend. Trace-class p.o.v.m.'s are discussed in Section 4.5. We end this section by introducing another important class of p.o.v.m.'s which plays a role in Stone's theorem. Definition 4.1.6 (Projection-valued measure). Let (Λ, A) be a measurable space and H a Hilbert space. Then a p.o.v.m. ξ on (Λ, A, H) is called a projectionvalued measure if it satisfies the two following conditions.

(i) The operator ξ(Λ) is the identity operator on H, that is, ξ(Λ) = Id H 0 .

(ii) For all A ∈ A, ξ(A) is an orthogonal projection.

Projection-valued measures are characterized by the following proposition (see Berberian (1966b, Theorems 3, 10 and 15)). In this case, the following properties hold for all A, B ∈ A and scalar f , g bounded and measurable functions from Λ to C.

(i) ξ(A)ξ(B) = ξ(B)ξ(A) (ii) A ∩ B = ∅ ⇒ ξ(A) and ξ(B) are orthogonal. (iii) f dξ = f dξ H . (iv) f g dξ = f dξ g dξ = g dξ f dξ

Locally compact Abelian groups and the theorems of Stone and Bochner

In this section, we summarize key results from Rudin (1990, Chapter 1) on locally compact Abelian (l.c.a.) groups. Then we state the theorems of Stone and Bochner which provide characterizations of two classes of functions defined on an l.c.a. group by their spectral representations.

Basic definitions and results

Definition 4.2.1 (Topological and locally compact Abelian group). A topological group is a group (G, +) (with null element 0) endowed with a topology for which the addition and the inversion maps are continuous in G × G and G respectively. If G is Abelian (i.e. commutative) and is locally compact, Hausdorff for its topology, then it is called a locally compact Abelian (l.c.a.) group.

The Fourier theory on l.c.a. groups relies on the duality between the l.c.a. group and the space of characters. Definition 4.2.2 (Characters and dual group). A character χ of an l.c.a. group G is a group homomorphism from G to the unit circle group

U := {z ∈ C : |z| = 1} that is a mapping χ : G → U such that for all s, t ∈ G, χ(s + t) = χ(s)χ(t)
The set of continuous characters of G is called the dual group of G and is denoted by Ĝ.

It is easily checked that a character χ of an l.c.a. group G, satisfies χ(0) = 1 and χ(t) = χ(t) -1 = χ(-t) for all t ∈ G. Hence, Ĝ is a multiplicative Abelian group if we define the product of χ 1 , χ 2 ∈ G, the identity element ê and the inverse of χ ∈ G as

χ 1 χ 2 : t → χ 1 (t)χ 2 (t) , ê : t → 1 , and χ -1 : t → χ(t) -1 = χ(t) .
Then a major result on l.c.a. groups is that the dual group Ĝ becomes itself an l.c.a. group if endowed with a well-suited topology. This topology is the compact-open topology which, in this context, is equivalent to the topology for which χ n → χ in Ĝ if and only if for every compact Munkres (2000, Theorem. 46.8)). With this topology, it is straightforward to prove that Ĝ is an Abelian topological group. Then, using results from the Gelfand transform and its equivalence to the Fourier transform on G, one can show that Ĝ is also locally compact and Hausdorff, and the following result thus holds. Another straightforward result is that, for all t ∈ G, the function

K ⊂ G, χ n → χ uniformly on K i.e. sup t∈K |χ n (t) -χ(t)| ----→ n→+∞ 0 (see
e t : Ĝ → U χ → χ(t)
, is a continuous homomorphism and therefore an element of Ĝ, the set of characters of Ĝ. The next key result on l.c.a. groups states that the set {e t : t ∈ G} is exactly Ĝ and is isomorphic to G. This result is known as the Pontryagin Duality Theorem (see Rudin (1990, Theorem 1.7.2)).

Theorem 4.2.2 (Pontryagin Duality Theorem). The evaluation map

G → Ĝ t → e t .
is a bijective continuous homomorphism with continuous inverse. Hence the set {e t : t ∈ G} is exactly Ĝ and is isomorphic to G.

The most common examples of l.c.a. groups are Z, T = R/(2πZ) and R all endowed with their usual additions and topologies. Their dual can be characterized as follows. The dual group Ẑ contains all Z → U-functions χ : t → z t for some z ∈ U. Since the compact sets of Z are the finite subsets of Z, the compact-open topology on Ẑ is the same as the one induced by pointwise convergence. It is then easy to show that Ẑ, U and T are isomorphic (from Ẑ to U take χ → χ(1) and from T to U take λ → e iλ ). In this case we identify Ẑ with T. In particular, this means that an integral on χ ∈ Ẑ is replaced by an integral on λ ∈ T with χ(t) replaced by e iλ t for all t ∈ Z. The dual set of R contains all R → U-functions χ : t → e itλ for some λ ∈ R (see for example Conway (1990, Theorem 9.11.)). Hence R and R are isomorphic via the mapping λ → (t → e itλ ) and we identify R with R.

Stone's and Bochner's theorems

In the following, we consider an l.c.a. group (G, +) and a Hilbert space H. Bochner's and Stone's theorems are spectral-type theorem which aim at representing certain type of functions on G as Fourier transforms of measures on the Borel σ-field of the dual space Ĝ. Let us introduce the following notions. group and H a Hilbert space. A mapping U :

G → L b (H) h → U h
is said to be a unitary representation (u.r.) of G on H if it satisfies the two following assertions.

(i) For all h ∈ G, U h is a unitary operator from H to H.

∈ N, t 1 , • • • , t n ∈ G and a 1 , • • • , a n ∈ C, n ∑ i,j=1 a i a j γ(t i -t j ) ≥ 0.
The spectral representations of c.u.r.'s and continuous hermitian non-negative definite functions are characterized by the following theorems.

Theorem 4.2.3 (Stone). Let G be an l.c.a. group, H a Hilbert space and U : h → U h a mapping from G to L b (H). Then U is a c.u.r. of G on H if and only if there exists a regular projection-valued measure ξ on ( Ĝ, B( Ĝ)) such that

U h = Ĝ e h dξ = Ĝ χ(h) ξ(dχ), h ∈ G . (4.2.1)
In this case, ξ is uniquely determined by U.

Proof. The proof of implication (⇒) is omitted because it would require to introduce concepts about the Fourier transform on G and * -representations and Riesz-Markov representation theorem which would not be used in the rest of this manuscript. We refer the reader to §36E of [START_REF] Loomis | An introduction to abstract harmonic analysis[END_REF]. The converse inclusion is a consequence of Proposition 4.1.2 and Point (ii) of Berberian (1966b, Theorem 11). For uniqueness, it is enough to show that for all x ∈ H,

ξ x : A → x H ξ(A)x is the unique regular non-negative measure satisfying x H U h x = Ĝ χ(h) ξ x (dχ)
γ(h) = Ĝ e h dµ = Ĝ χ(h) µ(dχ), h ∈ G . (4.2.2)
In this case, µ is uniquely determined by γ.

Proof. This theorem can also be proved using Fourier theory and Riesz-Markov representation theorem as in §1.4.3 of [START_REF] Rudin | Fourier Analysis on Groups[END_REF]. However, Theorem I in Fillmore (1970, Chapter 8) gives that, for any hermitian non-negative definite function γ : G → C, there exists a Hilbert space H, a unitary representation U : t → U t of G on H and a vector x 0 ∈ H such that

γ(h) = U h x 0 , x 0 H , h ∈ G. (4.2.3)
Then, using this result Theorem 4.2.4 is a straightforward consequence of Theorem 4.2.3.

Theorem I in Fillmore (1970, Chapter 8) shows that there is a duality between hermitian non-negative definite functions and unitary representations and this duality can be used to obtain Theorem 4.2.4 as a consequence of Theorem 4.2.3. It turns out that the converse can also be done, that is Theorem 4.2.3 can be obtained as a consequence of Theorem 4.2.4. This is done in the proof of Theorem VI in Fillmore (1970, Chapter 8) which also contains the proof of a generalization of Bochner's theorem for operator-valued hermitian non-negative definite functions (see Theorem VII in Fillmore (1970, Chapter 8)). This last class of functions is actually linked to a generalization of unitary representations for normal Hilbert modules which we now introduce.

Normal Hilbert modules

Modules extend the notion of vector spaces to the case where scalar multiplication is replaced by a multiplicative operation with elements of a ring. The case where the ring is L b (H 0 ) for a separable Hilbert space H 0 is of particular interest for H 0 -valued random variables. In short, a normal Hilbert L b (H 0 )-module is a Hilbert space endowed with a module action and a Gramian. A Gramian [•, •] is similar to a scalar product but is valued in the space S 1 (H 0 ) and is related to scalar product by the relation

•, • = Tr([•, •]).
Notions such as sub-modules, Gramian-orthogonality, Gramian-isometric operators are natural extensions of their counterparts in the Hilbert framework. We give such useful definitions hereafter and refer to Kakihara (1997, Chapter 2) for details. Definition 4.3.1 (L b (H 0 )-module). Let H 0 be a separable Hilbert space. An L b (H 0 )-module is a commutative group (H, +) such that there exists a multiplicative operation (called the module action)

L b (H 0 ) × H → H (P, x) → P • x
which satisfies the usual distributive properties : for all P, Q ∈ L b (H 0 ), and x, y ∈ H,

P • (x + y) = P • x + P • y, (P + Q) • x = P • x + Q • x, (PQ) • x = P • (Q • x), Id H 0 • x = x.
Next, we endow an L b (H 0 )-module with an L b (H 0 )-valued product.

Definition 4.3.2 ((Normal) pre-Hilbert L b (H 0 )-module). Let H 0 be a separable Hilbert space. We say that (H, [•, •] H ) is a pre-Hilbert L b (H 0 )-module if H is an L b (H 0 )-module and [•, •] H : H × H → L b (H 0 ) satisfies, for all x, y, z ∈ H, and P ∈ L b (H 0 ), (i) [x, x] H ∈ L + b (H 0 ), (ii) [x, x] H = 0 if and only if x = 0, (iii) [x + P • y, z] H = [x, z] H + P[y, z] H , (iv) [y, x] H = [x, y] H H . If moreover, for all x, y ∈ H, [x, y] H ∈ S 1 (H 0 ), we say that [•, •] H is a Gramian and that H is a normal pre-Hilbert L b (H 0 )-module.
Note that an L b (H 0 )-module is a vector space if we define the scalar-vector multiplication by αx = (αId H 0 ) • x for all α ∈ C, x ∈ H and that, in the particular case where

[•, •] H is a Gramian, then •, • H := Tr[•, •] H is a scalar product. Hence a normal pre-Hilbert L b (H 0 )-module is also a pre-Hilbert space. Lemma 2 in Kakihara (1997, Section 2.1) gathers useful results about Gramians, in particular if (H, [•, •] H ) is a normal pre-Hilbert L b (H 0 ) module, we have [x, y] H 1 ≤ x H y H , for all x, y ∈ H , (4.3.1)
where • 1 is the trace-class norm defined in Appendix A. We can now define the following.

H → G an L b (H 0 )-linear operator. Then U is said to be (i) Gramian-isometric if for all x, y ∈ H, [Ux, Uy] G = [x, y] H , (ii) Gramian-unitary if it is bijective Gramian-isometric.
The space H is said to be Gramian-isometrically embedded in G (denoted by H ⊆ ∼ G) if there exists a Gramian-isometric operator from H to G. The spaces H and G are said to be Gramian-isometrically isomorphic (denoted by H ∼ = G) if there exists a Gramian-unitary operator from H to G.

The well-known isometric extension theorem can be straightforwardly generalized to the case of Gramian-isometric operators as stated in the following proposition.

Proposition 4.3.1 (Gramian-isometric extension). Let H 0 be a separable Hilbert space, H be a normal pre-Hilbert L b (H 0 )-module, and G be a normal Hilbert L b (H 0 )-module. Let (v j ) j∈J and (w j ) j∈J be two collections of vectors in H and G respectively with J an arbitrary index set. If for all i, j ∈ J, v i , v j H = w i , w j G then there exists a unique Gramian-isometric operator S : Span

H P • v j , P ∈ L b (H 0 ), j ∈ J → G
such that for all j ∈ J, Sv j = w j . If moreover H is complete then

S Span H P • v j , P ∈ L b (H 0 ), j ∈ J = Span G P • w j , P ∈ L b (H 0 ), j ∈ J (4.3.2)
Finally, orthogonal projections can be generalized in this context as follows.

Definition 4.3.6 (Gramian-projection). Let H 0 be a separable Hilbert space and H a normal Hilbert L b (H 0 )-module. Then an orthogonal projection onto a closed submodule of H is called a Gramian-projection.

Let us introduce some examples of normal Hilbert L b (H 0 )-modules that will be of interest in the following.

Example 4.3.1 (Basic examples of normal Hilbert modules). Let H 0 , G 0 be two separable Hilbert space. Then the following assertions hold.

(1) The space H 0 is a normal Hilbert L b (H 0 )-module with module action P • x = Px and Gramian [x, y] H 0 = x ⊗ y where (x ⊗ y)u = u, y H 0 x for all u ∈ H 0 as defined in (A.1.1).

(2) The space S 2 (H 0 , G 0 ) of Hilbert-Schmidt operators from H 0 to G 0 is a normal Hilbert L b (G 0 )-module with module action defined, for

P ∈ L b (G 0 ) and Q ∈ S 2 (H 0 , G 0 ), by P • Q = PQ and Gramian defined, for P, Q ∈ S 2 (H 0 , G 0 ), by [P, Q] S 2 (H 0 ,G 0 ) = PQ H .
(3) Let µ be a non-negative measure on a measurable space (Λ, A) and

H a normal Hilbert L b (H 0 )-module. Then space L 2 (Λ, A, H, µ) is an L b (H 0 )- Hilbert module with module action defined for all P ∈ L b (H 0 ) and f ∈ L 2 (Λ, A, H, µ) by P • f : λ → P • f (λ). Moreover, Relation (4.3.1) gives that, for all f , g ∈ L 2 (Λ, A, H, µ), the function λ → [ f (λ), g(λ)] H is in L 1 (Λ, A, S 1 (H 0 ), µ)
, and thus

[ f , g] L 2 (Λ,A,H,µ) := [ f , g] H dµ = [ f (λ), g(λ)] H µ(dλ)
is well defined in S 1 (H 0 ). It is easy to show that it is a Gramian and that L 2 (Λ, A, H, µ) is a normal Hilbert L b (H 0 )-module if endowed with this Gramian.

The next two example are direct applications of Point (3) in Example 4.3.1 where H is taken as in Points ( 1) and ( 2) respectively.

Example 4.3.2 (Normal Hilbert module M(Ω, F , H 0 , P)). Let (Ω, F , P) be a probability space and H 0 be a separable Hilbert space. The Bochner space L 2 (Ω,

F , H 0 , P) is the space of H 0 -valued random variables Y such that E Y 2 H 0 < +∞. Then the expectation of Y is the unique vector E [Y] ∈ H 0 satisfying E [Y] , x H 0 = E Y, x H 0 , for all x ∈ H 0 , and the covariance operator between Y, Z ∈ L 2 (Ω, F , H 0 , P) is the unique linear operator Cov (Y, Z) ∈ L b (H 0 ), satisfying Cov (Y, Z) y, x H 0 = Cov Y, x H 0 , Z, y H 0 , for all x, y ∈ H 0 .
Equivalently, we have

Cov (Y, Z) = E [(Y -E [Y]) ⊗ (Z -E [Z])] ,
and therefore, Points (1) and (3) in Example 4.3.1 give that the space M(Ω, F , H 0 , P) of all centered random variables in L 2 (Ω, F , H 0 , P) is a normal Hilbert L b (H 0 )module for the module action defined for all P ∈ L b (H 0 ) and X ∈ M(Ω, F , H 0 , P) by P • X = PX, and the Gramian

[X, Y] M(Ω,F ,H 0 ,P) = Cov (X, Y) ∈ S 1 (H 0 ) . Example 4.3.3 (Normal Hilbert module L 2 (Λ, A, S 2 (H 0 , G 0 ), µ) for a non-neg- ative measure µ).
Let µ be a non-negative measure on (Λ, A) and H 0 , G 0 be two separable Hilbert spaces. Then L

2 (Λ, A, S 2 (H 0 , G 0 ), µ) is a normal Hilbert L b (G 0 )- module with module action defined for all P ∈ L b (G 0 ) and Φ ∈ L 2 (Λ, A, S 2 (H 0 , G 0 ), µ) by P • Φ : λ → PΦ(λ) and Gramian defined, for all Φ, Ψ ∈ L 2 (Λ, A, S 2 (H 0 , G 0 ), µ), by [Φ, Ψ] L 2 (Λ,A,S 2 (H 0 ,G 0 ),µ) := ΦΨ H dµ .

Generalizations of Stone's and Bochner's theorems

The duality between the scalar product on a Hilbert space and the Gramian on a normal Hilbert module leads naturally to the following definitions. Definition 4.4.1 ((Continuous) gramian unitary representations). Let (G, +) be an l.c.a. group, H 0 a separable Hilbert space and H a normal Hilbert L b (H 0 )-

module. A mapping U : G → L b (H) h → U h
is said to be a gramian unitary representation (g.u.r.) of G on H if it is an u.r. of G on H such that for all h ∈ G, U h is gramian-unitary. A g.u.r. is continuous, then called a c.g.u.r., if it is continuous as an u.r.

Definition 4.4.2 (Gramian-projection-valued measure). Let (Λ, A) be a measurable space, H 0 a separable Hilbert space and H a normal Hilbert L b (H 0 )-module.

Then a projection-valued measure on (Λ, A, H) is called a Gramian-projectionvalued measure if for all A ∈ A, ξ(A) is a Gamian-projection.

Then the generalization of Stone's theorem for normal Hilbert modules writes as Theorem 4.2.3 where U is a c.g.u.r and ξ is a Gramian-projectionvalued measure (see Kakihara (1997, Proposition 2.5.4)).

Theorem 4.4.1 (Stone's theorem for modules). Let (G, +) be an l.c.a. group, H 0 a separable Hilbert space and H a normal Hilbert L b (H 0 )-module. Then a function U : h → U h from G to L b (H) is a c.g.u.r. if and only if there exists a regular Gramian-projection valued measure ξ on ( Ĝ, B( Ĝ), H) such that

U h = χ(h) ξ(dχ), h ∈ G . (4.4.1)
In this case, ξ is uniquely determines by U.

Proof. The idea of the proof is to apply Stone's theorem to U which is also a c.u.r and to verify that the projection-valued measure obtained is in fact Gramian-projection-valued. Now, to generalize Bochner's theorem, we need to extend the notion of hermitian non-negative definiteness to operator-valued functions. Several definitions have been used in the literature and they are not straightforwardly equivalent.

Definition 4.4.3. Let H be a Hilbert space and (G, +) an l.c.a. group. A function

Γ : G → L b (H) is said to be 1. positive definite if for all n ∈ N * , t 1 , • • • , t n ∈ G and P 1 , • • • , P n ∈ L b (H), n ∑ i,j=1 P i Γ(t i -t j )P H j 0 ; 2. of positive-type if for all n ∈ N * , t 1 , • • • , t n ∈ G and x 1 , • • • , x n ∈ H, n ∑ i,j=1 Γ(t i -t j )x j , x i H ≥ 0 ; 3. hermitian non-negative definite if for all n ∈ N * , t 1 , • • • , t n ∈ G and a 1 , • • • , a n ∈ C, n ∑ i,j=1 a i a j Γ(t i -t j ) 0.
Equivalently, Γ is hermitian non-negative definite if and only if for all x ∈ H, t → Γ(t)x, x H is hermitian non-negative definite.

It is straightforward to show that the definitions in Definition 4.4.3 are given in an increasing order of generality in the sense that 1 ⇒ 2 ⇒ 3. In the univariate case, for a function γ : G → C all these definitions are trivially equivalent to hermitian non-negative positiveness. A natural question for a general Hilbert space H is which definition should be used to extend the Bochner theorem. It turns out that this choice does not matter since, with additional continuity condition, all these definitions are equivalent. This result is essentially the Naimark's moment theorem of Berberian, 1966a. Theorem 4.4.2 (General Bochner Theorem). Let (G, +) be an l.c.a. group, H a Hilbert space and Γ : G → L b (H). Then the following assertions are equivalent. (iv) There exists a regular p.o.v.m. ν on ( Ĝ, B( Ĝ), H) such that

Γ(h) = χ(h) ν(dχ) for all h ∈ G. (4.4.2)
Moreover, if Assertion (iv) holds, ν is the unique regular p.o.v.m. satisfying (4.4.2).

Proof. The equivalence between (i) and (ii) is straightforward: to show that (i)⇒(ii), take an arbitrary x ∈ H 0 with unit norm and set P i = x x H i for i = 1, . . . , n. To show that (ii)⇒(i), take, for any x ∈ H 0 , x i = P H i x for i = 1, . . . , n. The equivalence between (ii), (iii) and (iv) is given by Berberian (1966a, Theorem 3). Recall Definition 4.1.4 of a regular p.o.v.m.. It follows that the lastly stated fact that ν is uniquely determined by (4.4.2) is a consequence of the uniqueness stated in the univariate Bochner theorem (recalled in Theorem 4.2.4) applied to ν x : A → x H ν(A) x for all x ∈ H 0 .

Most of the generalizations of Bochner's theorem aim at showing that one of the statements (i), (ii), (iii) of Theorem 4.4.2 imply (iv) and Theorem 4.4.2 is the most complete formulation possible. In fact, a closer look at the literature in operator theory shows that the implication (iii) ⇒ (iv) appears commonly as an ingredient of the proof of Stone's theorem1 , see e.g. [START_REF] Ambrose | Spectral resolution of groups of unitary operators[END_REF][START_REF] Arnous | Sur les groupes continus de transformations unitaires de l'espace de Hilbert: Une extension d'un théorème de M.H. Stone[END_REF]or Theorem VI of Fillmore (1970, Chapter 8). Since (ii) obviously implies (iii), this indicates that the implication (ii) ⇒ (iv) is a classical result which has been proved again in Delft and Eichler (2020, Theorem 3.7) for their (equivalent) notion of operator-valued measures. The same implication, (ii) ⇒ (iv), is also proved in [START_REF] Neumark | Positive definite operator functions on a commutative group[END_REF], this time as a consequence of Stone's theorem. In contrast, it seems that little attention has been given to the converse implication (iv) ⇒ (ii). The proof of this implication is included in the proof of Berberian (1966a, Theorem 2). Berberian claims there that "[He does] not know how to prove [it] without using dilation theory". The proof of the same implication given in Delft and Eichler, 2020 relies on the computation of ν(dχ)x(χ), x(χ) where ν is an operator-valued measure in the sense of their Definition 3.5, see Delft and Eichler (2020, Lines 3 and 4, Page 3695). However the rigorous definition of such an integral is unclear to us in their context. For sake of completeness, we provide a simple proof of (iv) ⇔ (ii).

Proof of (ii) ⇔ (iv) in Theorem 4.4.2. To prove the implication (ii) ⇒ (iv), we follow the classical path found in the literature. Assume that (ii) holds and denote for all x ∈ H, γ x : h → x H Γ(h)x. Then for all x ∈ H, γ x is a hermitian non-negative definite function and Theorem 4.2.4 implies that there exists a unique regular finite non-negative measure ν x on ( Ĝ, B( Ĝ)) such that, for all h ∈ G,

x H Γ(h)x = γ x (h) = Ĝ χ(h) ν x (dχ) .
Then it is easy to verify that the family {ν x : x ∈ H} satisfies the conditions of Berberian (1966b, Theorem 2) which thus provides the existence of a unique regular p.o.v.m. ν on ( Ĝ, B( Ĝ), H) such that for all A ∈ B( Ĝ),

x H ν(A)x = ν x (A) .
In this case, by Definition 4.1.5 we have, for all h ∈ G and all x ∈ H,

x H Ĝ χ(h) ν(dχ) x = Ĝ χ(h) ν x (dχ) = x H Γ(h)x ,
which gives (4.4.2).

For the converse implication, we provide a simple proof without relying on dilation theory. Suppose that (iv) holds. The continuity of Γ in w.o.t. follows immediately by dominated convergence and we now prove that it is of positive type as in Definition 4.4.3. Take some arbitrary n ∈ N * , and

x 1 , • • • , x n ∈ H 0 . Let us define the C n×n -valued measure µ on on ( Ĝ, B( Ĝ)) by µ(A) =      ν(A)x 1 , x 1 H 0 • • • ν(A)x n , x 1 H 0 . . . . . . . . . ν(A)x 1 , x n H 0 • • • ν(A)x n , x n H 0      .
Then, by the Cauchy-Schwartz inequality, for all i, j ∈ 1, n , the C-valued measure µ i,j : A → [µ(A)] i,j admits a density f i,j with respect to the nonnegative finite measure µ 1 : A → µ(A) 1 = Tr(µ(A)) and the matrixvalued function f : χ → ( f i,j (χ)) 1≤i,j≤n is µ 1 -a.e. hermitian, non-negative semi-definite since, for all a ∈ C n and A ∈ B( Ĝ),

A a H f (χ)a µ 1 (dχ) = a H µ(A)a = n ∑ i=1 a i x i H ν(A) n ∑ i=1 a i x i ≥ 0 .
Then, for all

t 1 , • • • , t n ∈ G, we have n ∑ i,j=1 Γ(t i -t j )x i , x j H 0 = n ∑ i,j=1 χ(t i )χ(t j ) µ i,j (dχ) = n ∑ i,j=1 χ(t i )χ(t j ) f i,j (χ) µ 1 (dχ) = n ∑ i,j=1 χ(t i )χ(t j ) f i,j (χ) ≥0 µ 1 -a.e.
µ 1 (dχ) ≥ 0 .

The first line follows from (iv), the definition of µ i,j above and the definition of the integral as given by Definition 4.1.5. The second line follows from the definition of f i,j and the third line from the above property of the matrixvalued function f . Hence we have shown (ii) and the proof of the implication is concluded.

Throughout this section, we have seen that Theorems 4.2.3, 4.2.4, 4.4.1 and 4.4.2 are closely related. It actually turns out that almost every result can be obtained as a consequence of any of the others. As an illustration of this conclusion, Figure 4.1 gives a graphical representation of some interesting implications found in the literature. Arrows with the same color indicate a path of implications usually followed by one or several authors.

Trace-class p.o.v.m.'s

In the case where H 0 is a separable Hilbert space, we provide the definition of trace-class p.o.v.m.'s and derive several useful properties they enjoy.

Definition 4.5.1 (Trace-class p.o.v.m.). Let (Λ, A) be a measurable space, H 0 be a separable Hilbert space and ν be a p.o.v.m. on (Λ, A, H 0 ). We say that ν is a trace-class p.o.v.m. if it is S + 1 (H 0 )-valued.

Trace-class p.o.v.m.'s are equivalent to the S + 1 (H 0 )-valued measures used in Kakihara, 1997, Section 3.4. In particular, these measures fit the framework of vector-valued measures as stated in the following Lemma.

Lemma 4.5.1. Let (Λ, A) be a measurable space and H 0 be a separable Hilbert space. Then a p.o.v.m. ν on (Λ, A, H 0 ) is trace-class if and only if ν(Λ) ∈ S 1 (H 0 ). In this case, ν is an S 1 (H 0 )-valued measure (in the sense that (4.1.1) holds in • 1 - norm) with finite variation measure ν 1 : A → ν(A) 1 . Moreover, regularity of ν as a p.o.v.m. is equivalent to regularity of ν as an S 1 (H 0 )-valued measure which is itself equivalent to regularity of ν 1 .

Proof. The first point comes from the fact that for all A ∈ A, ν(A) ν(Λ). Now, if ν is trace-class, then (4.1.1) is easily verified for the norm • 1 using the fact that • 1 = Tr(•) for positive operators. Finally, by definition of ν 1 , regularity of ν 1 is equivalent to regularity of ν as an S 1 (H 0 )-valued measure which clearly implies regularity of ν x = x H ν(•)x for all x ∈ H 0 . Suppose now that for all x ∈ H 0 , ν x is regular, then let (e k ) k∈N be a Hilbert basis of H 0 , and define for all n ∈ N, the non-negative measure µ n := ∑ n k=0 ν e k such that for all A ∈ A, ν 1 (A) = lim n→+∞ µ n (A) = sup n∈N µ n (A). Then, by Vitali-Hahn-Sakh-Nikodym's theorem (see [START_REF] Brooks | On the Vitali-Hahn-Saks and Nikodym Theorems[END_REF], the sequence (µ n ) n∈N is uniformly countably additive which implies regularity of ν 1 by Lemma 23 in Diestel and Uhl (1977, Chapter VI, Section 2).

The main advantage of trace-class p.o.v.m.'s lies in the fact that S 1 (H 0 ) is a separable dual space and therefore has the Radon-Nikodym property. Consequences of this property are discussed in the remaining of this section as well as implications of the general Bochner theorem when trace-class p.o.v.m.'s are involved. We conclude this section by deriving useful results on the eigendecomposition of a trace-class p.o.v.m..

Radon-Nikodym property of trace-class p.o.v.m.'s

Theorem 4.5.2. Let (Λ, A) be a measure space, H 0 a separable Hilbert space and ν a trace-class p.o.v.m. on (Λ, A, H 0 ). Let µ be a σ-finite non-negative measure on (Λ, A). Then ν 1 µ, if and only if there exists g ∈ L 1 (Λ, A, S 1 (H 0 ), µ) such that dν = g dµ, i.e. for all A ∈ A,

ν(A) = A g dµ .
(4.5.1)

In this case, g is unique and is called the density of ν with respect to µ and we write g = dν dµ .

Moreover, the following assertions hold.

(a) For µ-almost every λ ∈ Λ, g(λ) ∈ S + 1 (H 0 ).

(b) The mapping g 1/2 : λ → g(λ) 1/2 belongs to L 2 (Λ, A, S 2 (H 0 ), µ).

(c) The density of ν 1 with respect to µ is g 1 . In particular, g = dν

d ν 1 g 1 µ-a.e. and if µ = ν 1 , then g 1 = 1 µ-a.e.
Proof. The first part of the theorem comes from the fact that the space S 1 (H 0 ) has the Radon-Nikodym property. Indeed, since H 0 is separable, S 1 (H 0 ) is the dual of the separable space K(H 0 ). It is therefore a separable dual space and the result follows by Theorem 1 in Diestel and Uhl (1977, Chapter III, Section 3). Then for all x ∈ H 0 and A ∈ A,

A g(λ)x, x H 0 µ(dλ) = ν(A)x, x H 0 ≥ 0 ,
and there exists a set A x ∈ A with µ(A c x ) = 0 and g(λ)x, x H 0 ≥ 0 for all λ ∈ A x . Taking (x n ) n∈N a dense countable subset of H 0 we get that g ∈ S +

1 (H 0 ) on A = n∈N A x n thus proving Assertion (a). Assertion (b) then follows from Lemma B.2.1. Moreover, taking the trace in (4.5.1) gives for all A ∈ A,

ν 1 (A) = A g 1 dµ which gives Assertion (c).
From Lemma 4.5.1, we know that a trace-class p.o.v.m. ν is an S 1 (H 0 )valued measure. This allows us to integrate an unbounded scalar-valued functions f with respect to ν as soon as f ∈ L 1 (Λ, A, ν 1 ) as recalled in Section 4.1.1. With the Radon-Nikodym property, we can interpret this integral as a Bochner integral.

Corollary 4.5.3. Let (Λ, A) be a measure space, H 0 a separable Hilbert space and ν a trace-class p.o.v.m. on (Λ, A, H 0 ). Let µ be a σ-finite non-negative measure on (Λ, A) such that ν 1 µ, and let g = dν dµ . Let f : Λ → C be measurable. Then f ∈ L 1 (Λ, A, ν 1 ) if and only if λ → f (λ) g(λ) ∈ L 1 (Λ, A, S 1 (H 0 ), µ), and, in this case, we have

f (λ) ν(dλ) = f (λ) g(λ) µ(dλ) . (4.5.2)
Proof. The proof consists in extending the case f = 1 A for A ∈ A to simple functions and then using the density of simple functions.

Note that, in (4.5.2), the first integral is that of a scalar-valued function with-respect to the S 1 (H 0 )-valued measure ν as recalled above for general vector-valued measures with finite variation and the second is the Bochner integral of an S 1 (H 0 )-valued function with-respect to the non-negative measure µ as recalled in Appendix B. Of course, if f is bounded on Λ, these integrals coincide with the integral of f with respect to ν of Definition 4.1.5 in which ν is seen as a p.o.v.m.. The Radon-Nikodym property of traceclass p.o.v.m.'s is a key step to extend such integrals to operator valued functions, hence allowing us to use a handy definition of the integral of an operator valued function with respect to an operator valued measure, in the particular case where this measure is a trace-class p.o.v.m.. An example of such integrals provides a more general formulation of the space of transfer functions used in Tavakoli (2014, Section 2.5) and Delft and Eichler (2018, Appendix B.2.3) for filtering functional time series.

Example 4.5.1 (semi-Gramian [•, •] ν on L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) for a traceclass p.o.v.m. ν.). Let (Λ, A) be a measurable space, H 0 , G 0 be two separable Hilbert spaces and ν a trace-class p.o.v.m. on (Λ, A, H 0 ) with density f with respect to its finite variation ν 1 . Then the space L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) is an L b (H 0 )-module with module action defined for all

P ∈ L b (G 0 ) and Φ ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) by P • Φ : λ → PΦ(λ). Define also for all Φ, Ψ ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) [Φ, Ψ] ν := Φ f Ψ H d ν 1 .
(4.5.3)

Note that the S 1 (H 0 )-valued Bochner integral in the right-hand side of (4.5.3) is well defined because by Theorem 4.5.2 (c), we have f 1 = 1, ν 1 -a.e. and thus

Φ f Ψ H 1 ≤ Φ L b (H 0 ,G 0 ) Ψ L b (H 0 ,G 0 ) , ν 1 -a.e., which implies Φ f Ψ H ∈ L 1 (Λ, A, S 1 (G 0 ), ν 1 ). Then [•,
•] ν is a semi-Gramian in the sense that it satisfies all Assertions from Definition 4.3.2 except Assertion (ii). Indeed, we have

[Φ, Φ] ν = 0 ⇔ Φ f 1/2 = 0 , ν 1 -a.e. .
To make it a Gramian, we can quotient space L

2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) by the set { • ν = 0} where • ν is the semi-norm associated to the semi-Gramian [•, •] ν , i.e. Φ 2 ν = Tr[Φ, Φ] ν . Note that this semi-norm is different from the norm Φ 2 L 2 (Λ,A,L b (H 0 ,G 0 ), ν 1 ) .
In particular, for all Φ ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ), we have

Φ 2 ν = Tr Φ f Φ H d ν 1 = Tr Φ f Φ H d ν 1 ≤ Φ 2 L b (H 0 ,G 0 ) d ν 1 = Φ 2 L 2 (Λ,A,L b (H 0 ,G 0 ), ν 1 ) , (4.5.4)
where we used again that f 1 = 1, ν 1 -a.e. It is easy to find Φ's for which the inequality is strict.

Example 4.5.1 is pivotal for defining the modular spectral domain of a weakly stationary process with spectral operator measure ν. However, it does not suffice to describe the whole spectral domain because, as already noted in Tavakoli (2014, Section 2.5) in a similar case, this space, in general, it is not complete. As a result, unfortunately, the spectral domain is more complicated for functional time series than for (finite dimensional) multivariate time series. Of course, as proposed in Tavakoli (2014, Section 2.5), it is always possible to use topological completion under • ν . These ideas are in fact very similar to the ones of [START_REF] Kakihara | Multidimensional Second Order Stochastic Processes[END_REF][START_REF] Kallianpur | Spectral theory of stationary Hvalued processes[END_REF][START_REF] Mandrekar | The Square-integrability of Operatorvalued Functions with Respect to a Non-negative Operator-valued Measure and The Kolmogorov Isomorphism Theorem[END_REF] with the exception that the latter references provide a more general framework and lead to a modular spectral domain which is an explicit set of operator-valued functions. We will follow this approach in Chapter 5. In fact, we can already define a larger space by noting that the integral in (4.5.3) is not restricted to functions in L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ). This space is a natural extension of [START_REF] Masani | Recent trends in multivariate prediction theory[END_REF] where the case of (finite dimensional) multivariate time series is considered (see the definition of L 2,M in this reference). As introduced in Appendix B, the set

F s (Λ, A, H 0 , G 0 ) is the set of functions Φ : Λ → L b (H 0 , G 0 ) such that, the mapping λ → Φ(λ)x is measurable for all x ∈ H 0 . Example 4.5.2 (Normal pre-Hilbert module L 2 (Λ, A, L b (H 0 , G 0 ), ν), [•, •] ν for a trace-class p.o.v.m. ν.
). Let (Λ, A) be a measurable space, H 0 , G 0 be two separable Hilbert spaces and ν a trace-class p.o.v.m. on (Λ, A, H 0 ) with density f with respect to its finite variation ν 1 . Denote by

L 2 (Λ, A, L b (H 0 , G 0 ), ν) the space of functions Φ ∈ F s (Λ, A, H 0 , G 0 ) such that Φ f 1/2 ∈ L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ). Then L 2 (Λ, A, L b (H 0 , G 0 ), ν) is an L b (H 0 )-
module with the same module action as in Example 4.5.1 and the mapping [•, •] ν of (4.5.3) can be extended to a semi-Gramian on L 2 (Λ, A, L b (H 0 , G 0 ), ν). Indeed, it suffices to note that, for all

Φ, Ψ ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν), [Φ, Ψ] ν = Φ f 1/2 , Ψ f 1/2 L 2 (Λ,A,L b (H 0 ,G 0 ), ν 1 )
, (4. 5.5) where the gramian in the right side of (4.5.5) is defined in Example 4.3.3.

Then, the space

L 2 (Λ, A, L b (H 0 , G 0 ), ν) := L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) { • ν = 0} , is a normal pre-Hilbert L b (H 0 )-module.
4. (ii) Assertion (i) holds and Γ(h) ∈ S 1 (H 0 ) for all h ∈ G.

(iii) Assertion (i) holds and Γ satisfies the following continuity condition: for all

P ∈ L b (H 0 ), h → Tr(PΓ(h)) is continuous on G.
(iv) There exists a regular trace-class p.o.v.m. ν on ( Ĝ, B( Ĝ), H 0 ) such that (4.4.2) holds.

Proof. The equivalence between (i) and (iv) is a clear consequence of Theo- and take P ∈ L b (H 0 ). Then for all h ∈ G,

PΓ(h) = P Ĝ χ(h) f (χ) ν 1 (dχ) = Ĝ χ(h)P f (χ) ν 1 (dχ)
, and, since the integrand in the last integral has a S 1 (H 0 )-norm upper bounded by P L b (H 0 ) and ν 1 is finite, we get that h → PΓ(h) is continuous from Ĝ to S 1 (H 0 ) by dominated convergence. The continuity of h → Tr(PΓ(h)) follows.

Eigendecomposition of a trace-class p.o.v.m.

The goal of this section is to provide a proof of the following result which addresses the issues discussed in Section 2.7 of Tavakoli, 2014 in a more general setting.

Proposition 4.5.5 (Eigendecomposition of a trace-class p.o.v.m.). Let H 0 be a separable Hilbert space with dimension N ∈ {1, . . . , +∞}. Let ν be a traceclass p.o.v.m. on (Λ, A, H 0 ) and µ a σ-finite dominating measure of ν, e.g. its variation norm ν 1 . Then there exist sequences (σ n ) 0≤n<N and (φ n ) 0≤n<N of (Λ, A) → (R + , B(R + )) and (Λ, A) → (H 0 , B(H 0 )) measurable functions, respectively, such that the following assertions hold.

(i) For all λ ∈ Λ, (σ n (λ)) 0≤n<N is non-increasing and ∑ 0≤n<N σ n (λ) < ∞.

(ii) For all λ ∈ Λ, (φ n (λ)) 0≤n<N is orthonormal.

(iii) The trace-class p.o.v.m. ν admits the density

f : λ → ∑ 0≤n<N σ n (λ) φ n (λ) ⊗ φ n (λ) ,
with respect to µ, where the convergence holds absolutely in S 1 (H 0 ) for each λ ∈ Λ.

(iv) Moreover, if N = +∞, we have

f = +∞ ∑ n=0 σ n φ n ⊗ φ n ,
where the series converges in L 1 (Λ, A, S 1 (H 0 ), µ).

To this end, let us recall essential facts about the diagonalization of compact positive operators. Let H 0 be a separable Hilbert space of dimension N ∈ {1, • • • , +∞}, (Λ, A) be a measurable space and Φ ∈ F s (Λ, A, H 0 ) such that for all λ ∈ Λ, Φ(λ) ∈ S + 1 (H 0 ). Then, in this case, for any λ ∈ Λ, Φ(λ) admits the eigendecomposition

Φ(λ) = ∑ 0≤n<N σ n (λ)φ n (λ) ⊗ φ n (λ) , (4.5.6)
where the series converges in operator norm and the family (φ n (λ)) 0≤n<N is orthonormal. Moreover, we have

Tr(Φ(λ)) = ∑ 0≤n<N σ n (λ) < +∞ .
The following theorem shows that such a decomposition can be constructed in a way which makes the eigenvalues and eigenvectors measurable as functions of λ. The novelty of this result compared to Lemma 7 of Kakihara, 1997, Section 3.4 is that, using topological properties induced by S + 1 (H 0 ), we prove measurability of the eigenvectors and not only the eigenprojections.

We will need the following lemmas, which rely on the weak topology on H 0 , defined as the smallest topology which makes the functions x H : x ∈ H 0 continuous. Lemma 4.5.6. Let H 0 be a separable Hilbert space and denote the closed unit ball by B0,1 := x ∈ H 0 :

x H 0 ≤ 1 .
Then B0,1 endowed with the weak topology is a compact metrizable space.

Proof. By the Banach-Alaoglu theorem, B0,1 is compact for the weak topology.

Since H 0 is separable, we can choose a Hilbert basis (ψ n ) 0≤n<N for H 0 , with N ∈ {1, • • • , +∞}. It is straightforward to show that the mapping (x, y) → ∑ 0≤n<N 2 -n xy, ψ n H 0 is a metric inducing the weak topology on B0,1 .

Lemma 4.5.7. Let H 0 be a separable Hilbert space. Then the Borel σ-field B w (H 0 ) of H 0 endowed with the weak topology coincides with the (usual) Borel σ-field B(H 0 ) of (H 0 , • H 0 ).

Proof. The weak topology is included in the topology of (H 0 , • H 0 ), hence B w (H 0 ) ⊂ B(H 0 ). To prove the converse inclusion, observe that by expressing xy H 0 as the 2 -norm of the inner-products of (xy) with a Hilbert basis (ψ n ) 0≤n<N , we easily get that y → xy H 0 is measurable from (H 0 , B w (H 0 )) to (R + , B(R + )) for all x ∈ H 0 . Hence B(H 0 ) ⊂ B w (H 0 ), which concludes the proof.

Lemma 4.5.8. Let H 0 be a separable Hilbert space. If P ∈ S + 1 (H 0 ) then the mapping x → Px, x H 0 is continuous on the unit closed ball B0,1 for the weak topology. Theorem 4.5.9. Let H 0 be a separable Hilbert space and (Λ, A) be a measurable space. Let Φ ∈ F s (Λ, A, H 0 ) such that for all λ ∈ Λ, Φ(λ) ∈ S + 1 (H 0 ). Then the pairs {(σ n , φ n ) : 0 ≤ n < N} in (4.5.6) can be taken so that for all 0 ≤ n < N, σ n is measurable from (Λ, A) to (R + , B(R + )) and φ n is measurable from (Λ, A) to (H 0 , B(H 0 )).

Proof. Let us consider the eigendecomposition

P = ∑ 0≤n<N σ n φ n ⊗ φ n . Then for all x ∈ B0,1 , Px, x H 0 = ∑ 0≤n<N σ n x,
Proof. The construction of the eigenvalues and eigenvectors is done iteratively using the Measurable Maximum Theorem Aliprantis and Border (2006, Theorem 18.19) on Λ × B0,1 , where B0,1 denotes the closed unit ball of H 0 , which is compact metrizable for the weak topology by Lemma 4.5.6. As in Aliprantis and Border (2006, Definition 17.1), a correspondence ϕ from Λ to B0,1 , denoted by ϕ : Λ B0,1 , is a mapping which assigns each element of Λ to a subset of B0,1 .

Construction of (σ 1 , φ 1 ) : Define

f : Λ × B0,1 → R + (λ, x) → Φ(λ)x, x H 0 .
Then, for all x, λ → f (λ, x) is measurable and, for all λ ∈ Λ, x → f (λ, x) is continuous in x for the weak topology by Lemma 4.5.8. Moreover the correspondence

ϕ : Λ B0,1 λ → B0,1
is weakly measurable (in the sense of Aliprantis and Border (2006, Definition 18.1)) with nonempty compact values (for the weak topology). Therefore the Measurable Maximum Theorem (see Aliprantis and Border (2006, Theorem 18.19)) gives that m : λ → max x∈ B0,1 f (λ, x) is measurable and that there exists a function g : Λ → B0,1 such that for all λ ∈ Λ, g(λ) ∈ argmax x∈ B0,1 f (λ, x) and g is measurable from Λ to B0,1 endowed with the Borel σ-field generated by the weak topology. This implies the usual measurability by Lemma 4.5.7. We set σ 0 = m and φ 0 = g. Then, from the definitions of f , m and g, that σ 0 (λ) is the largest eigenvalue of Φ(λ) and that φ 0 (λ) is an eigenvector with eigenvalue σ 0 (λ).

Construction of (σ n , φ n ) : Assume we have constructed n measurable func- tions σ

0 , • • • , σ n-1 and φ 0 , • • • , φ n-1 satisfying for all λ ∈ Λ, σ 0 (λ) ≥ • • • ≥ σ n-1 (λ), and (φ 0 (λ), • • • , φ n-1 (λ)
) is an orthonormal family where for all

0 ≤ i ≤ n -1, φ i (λ) ∈ ker(Φ(λ) -σ i (λ)Id H 0 )
. Then, as in the initialization step, the function

f : Λ × B0,1 → R + (λ, x) → Φ(λ)x, x H 0 -∑ n-1 i=1 σ i (λ) x, φ i (λ) H 0 2 .
is measurable in λ and continuous in x (for the weak topology) by Lemma 4.5.8. Moreover, using Aliprantis and Border (2006, Corollary 18.8 and Lemma 18.2))

and the fact that ϕ(λ) = x ∈ B0,1 :

∑ n-1 i=0 x, φ i (λ) H 0 2
= 0 and has nonempty compact values (because ϕ(λ) is a closed subset of B0,1 for the weak topology hence is compact for this topology), we get that the correspondence

ϕ : Λ B0,1 λ → B0,1 ∩ Span (φ 0 (λ), • • • , φ n-1 (λ)) ⊥
is weakly measurable (in the sense of Aliprantis and Border (2006, Definition 18.1)). Hence, as previously, the Measurable Maximum Theorem and Lemma 4.5.7 give that m : λ → max x∈ϕ(λ) f (λ, x) is measurable and that there exists a measurable function g : Λ → H 0 such that for all λ ∈ Λ, g(λ) ∈ argmax x∈ϕ(λ) f (λ, x). We set σ n = m and φ n = g. Then, from the definitions of f , m and g, we get that σ n (λ) ≤ σ n-1 (λ) is the (n + 1)-th largest eigenvalue of Φ(λ) (because it is the largest eigenvalue of Φ(λ) -∑ n-1 i=0 σ i (λ)φ i (λ) ⊗ φ i (λ)) and that φ n (λ) is an eigenvector with eigenvalue σ n (λ) and is orthogonal to

φ 0 , • • • , φ n-1 .
With these results, the proof of Proposition 4.5.5 is easily derived.

Proof of Proposition 4.5.5. We provide a proof in the case where N = ∞ as the finite dimensional case is easier. Let f ∈ L 1 (Λ, A, S + 1 (H 0 ), µ) be the density of ν with respect to µ. We assume without loss of generality that f (λ) ∈ S 1 (H 0 ) + for all λ ∈ Ĝ (rather than for µ-almost every λ). Using Theorem 4.5.9 we can write

f (λ) = +∞ ∑ n=0 σ n (λ)φ n (λ) ⊗ φ n (λ) , (4.5.7)
where (σ n (λ)) n∈N is non-decreasing and converges to zero and (φ n (λ)) n∈N satisfies (ii). Moreover, for all λ ∈ Λ, ∑ n σ n (λ) = f (λ) 1 < ∞, and we get Assertions (i) and (iii). In remains to prove Assertion (iv). Since L 1 (Λ, A, S 1 (H 0 ), µ) is complete, it suffices to show that ∑ q n=p σ n φ n ⊗ φ n 1 dµ converges to 0 as p, q → +∞ which holds by dominated convergence because, for all λ ∈ Λ, ∑ q n=p σ n (λ)φ n (λ) ⊗ φ n (λ) 1 = ∑ q n=p σ n (λ) which converges to 0 as p, q → +∞ and is upper-bounded by f (λ) 1 .

S P E C

T R A L T H E O R Y F O R W E A K LY S TAT I O N A R Y S T O C H A S T I C P R O C E S S E S VA L U E D I N A S E PA R A B L E H I L B E R T S PA C E
In this chapter, we use the tools developed in Chapter 4 to build the spectral theory of weakly stationary stochastic processes valued in a separable Hilbert using the approach of [START_REF] Kakihara | Multidimensional Second Order Stochastic Processes[END_REF][START_REF] Kallianpur | Spectral theory of stationary Hvalued processes[END_REF][START_REF] Mandrekar | The Square-integrability of Operatorvalued Functions with Respect to a Non-negative Operator-valued Measure and The Kolmogorov Isomorphism Theorem[END_REF] In particular, we illustrate how Step 1) and Step 3) are related to Theorem 4.4.1 and Theorem 4.4.2 and how the Cramér representation can be viewed as an integral with respect to a random countably additive Gramian-orthogonally scattered measure. Such integrals are introduced in Section 5.1 which paves the way for describing the modular spectral domain. Section 5.2 contains the main results: 1) we offer a synthesis of the results of [START_REF] Kakihara | Multidimensional Second Order Stochastic Processes[END_REF][START_REF] Kallianpur | Spectral theory of stationary Hvalued processes[END_REF][START_REF] Mandrekar | The Square-integrability of Operatorvalued Functions with Respect to a Non-negative Operator-valued Measure and The Kolmogorov Isomorphism Theorem[END_REF] providing a natural and complete spectral theory for weakly stationary processes valued in a separable Hilbert space; 2) in light of these results, we re-examine the differences with the approaches proposed in Delft and Eichler, 2020;[START_REF] Tavakoli | Fourier Analysis of Functional Time Series, with Applications to DNA Dynamics[END_REF] The main proofs are postponed in Section 5.3.

Stochastic integral with respect to a random countably additive Gramian-orthogonally scattered measure

Countably additive Gramian-orthogonally scattered measures

In this section, we introduce the notion of random countably additive Gramianorthogonally scattered (c.a.g.o.s.) measures which will have an important role in the construction provided by [START_REF] Kakihara | Multidimensional Second Order Stochastic Processes[END_REF]Kallianpur and Man-drekar, 1971;[START_REF] Mandrekar | The Square-integrability of Operatorvalued Functions with Respect to a Non-negative Operator-valued Measure and The Kolmogorov Isomorphism Theorem[END_REF]. The terminologies c.a.o.s. and c.a.g.o.s. are borrowed from Definition 3 in Kakihara (1997, Section 3.1) Definition 5.1.1 ((Random) c.a.o.s. measures). Let H be a Hilbert space and (Λ, A) be a measurable space. We say that W : A → H is a countably additive orthogonally scattered (c.a.o.s.) measure on (Λ, A, H) if it is an H-valued measure on (Λ, A) such that for all A, B ∈ A,

A ∩ B = ∅ ⇒ W(A), W(B) H = 0 .
In this case, the mapping

ν W : A → W(A), W(A) H
is a finite non-negative measure on (Λ, A) called the intensity measure of W and we have that, for all A, B ∈ A,

ν W (A ∩ B) = W(A), W(B) H .
(5.1.1)

We say that W is regular if ν W is regular. When H is the space M(Ω, F , H 0 , P) of Example 4.3.2, we say that W is an H 0 -valued random c.a.o.s. measure on (Λ, A, Ω, F , P).

The generalization to a normal Hilbert module is straightforward.

Definition 5.1.2 ((Random) c.a.g.o.s. measures). Let H 0 be a separable Hilbert space, H be a normal Hilbert L b (H 0 )-module and (Λ, A) be a measurable space. We say that W : A → H is a countably additive Gramian-orthogonally scattered (c.a.g.o.s.) measure on (Λ, A, H) if it is an H-valued measure on (Λ, A) such that for all A, B ∈ A,

A ∩ B = ∅ ⇒ [W(A), W(B)] H = 0 .
In this case, the mapping

ν W : A → [W(A), W(A)] H
is a trace-class p.o.v.m. on (Λ, A, H 0 ) called the intensity operator measure of W and we have that, for all A, B ∈ A,

ν W (A ∩ B) = [W(A), W(B)] H . (5.1.2)
We say that W is regular if ν W 1 is regular. When H = M(Ω, F , H 0 , P) of Example 4.3.2, we say that W is an H 0 -valued random c.a.g.o.s. measure on (Λ, A, Ω, F , P).

The following remark will be useful.

Remark 5.1.1. Recall that any H-valued measure W is σ-additive in the sense that for any finite or countable collection (A j ) j∈J ∈ Λ J of pairwise disjoint sets we have

W   j∈J A j   = ∑ j∈J W(A j ) ,
where, in the case where J is countably infinite, the infinite sum converges in H inconditionally. When W is a c.a.o.s., the summands are moreover orthogonal. When it is a c.a.g.o.s., they are Gramian-orthogonal.

It is easy to show that a c.a.o.s. measure W as in Definition 5.1.1 can be equivalently seen as the restriction of an isometric operator

I from L 2 (Λ, A, ν W ) onto H by setting W(A) = I(1 A ) , A ∈ Λ .
This simply follows by interpreting the left-hand side of (5.1.1) as the scalar product between 1 A and 1 B in L 2 (Λ, A, ν W ) so that I above can be defined as the unique isometric extension from L 2 (Λ, A, ν W ) to H of the isometric mapping defined by

1 A → W(A) for A ∈ Λ.
(Λ, A, L b (H 0 ), ν W ) for the norm • ν .
In the case where H 0 has finite dimension, this completeness is proved in Masani (1966, Theorem 6.3) (see also Kuroda (1967, Lemma 3.2)) where the author derives a Cramér representation of the type (ii.2) for a multivariate time series (X t ) t∈Z and a c.a.g.o.s. measure X. In the infinite dimensional case, L 2 (Λ, A, L b (H 0 ), ν W ) is not necessarily complete for the norm • ν . A trivial counterexample is given in Kuroda (1967, Remark 3.1) and we will provide a necessary and sufficient condition in Theorem 5.1.5. Similarly to Tavakoli (2014, Section 2.5), Kuroda proposes to work with the completion of L 2 (Λ, A, L b (H 0 ), ν W ) under the norm • ν but highlights the inconvenience of this solution because, in this case, some elements of the completed space may not have any representative function. The approach of [START_REF] Mandrekar | The Square-integrability of Operatorvalued Functions with Respect to a Non-negative Operator-valued Measure and The Kolmogorov Isomorphism Theorem[END_REF] solved this issue by exhibiting the smallest normal Hilbert module containing the space L 2 (Λ, A, L b (H 0 ), ν W ). This space, which we will denote by L 2 (Λ, A, O(H 0 , G 0 ), ν) and define for a trace-class p.o.v.m. ν, will be introduced in Section 5.1.2. Before that, let us note that, in the case of random c.a.g.o.s. measure W, by definition of H = M(Ω, F , H 0 , P) in Example 4.3.2, the identity (5.1.2) shows that the covariance structure of the centered process (W(A)) A∈A is entirely determined by ν W . The Gaussian case is interesting as it provides a way to build W from its intensity measure. In particular, the following result will be useful.

Theorem 5.1.1. Let H 0 be a separable Hilbert space and (Λ, A) be a measurable space. Let ν be a trace-class p.o.v.m. on (Λ, A, H 0 ). Then there exist a probability space (Ω, F , P) and an H 0 -valued random c.a.g.o.s. W on (Λ, A, Ω, F , P) with intensity operator measure ν such that the process ( W(A), x ) A∈A,x∈H 0 is a (com- plex) Gaussian process.

Proof. Define γ : (H 0 × A) 2 → C by of γ((x, A); (y, B)) = x H ν(A ∩ B)y = x H 1 A , y H 1 B ν ,
where we used the Gramian (4.5.3) of Example 4.5.1 with G 0 = C. Then it is easy to see γ is hermitian non-negative definite in the sense that for all n ≥ 1, x 1 , . . . , x n ∈ H 0 , A 1 , . . . , A n ∈ A and a 1 , . . . ,

a n ∈ C, n ∑ i,j=1
a i a j γ((x i , A i ); (a j , A j )) ≥ 0 .

Let (Z x,A ) (x,A)∈H 0 ×A be the centered circularly-symmetric Gaussian process complex with covariance γ. Let (φ n ) 0≤n<N be a Hilbert basis of H 0 , with N = dim H 0 ∈ {1, 2, . . . , ∞}. It is straightforward to show that for all A ∈ A,

W(A) := ∑ 0≤n<N Z φ n ,A φ n
is well defined in H = M(Ω, F , H 0 , P) and that the so defined W is a random c.a.g.o.s. with intensity operator measure ν.

The space L

2 (Λ, A, O(H 0 , G 0 ), ν)
As discussed in the previous section, the role of c.a.o.s. and c.a.g.o.s. measures in the spectral theory of weakly stationary processes relies on their characterization by unitary or Gramian-unitary operators between the (modular) time domain and the (modular) spectral domain. This has been entirely studied in the case of univariate and multivariate time series, see [START_REF] Holmes | Mathematical Foundations of Signal Processing[END_REF][START_REF] Masani | Recent trends in multivariate prediction theory[END_REF], respectively, and the references therein. For time series valued in a general separable Hilbert space, defining the modular spectral domain requires to exhibit a suitable completion of the normal pre-Hilbert module L 2 (Λ, A, L b (H 0 , G 0 ), ν) of Example 4.5.2 where ν is a trace-class p.o.v.m. . In this section, we define such a space of operatorvalued functions which are square-integrable with respect to p.o.v.m. ν. This space was introduced in Mandrekar and Salehi, 1970 and includes the space L 2 (Λ, A, L b (H 0 , G 0 ), ν) but is, in general, larger in the case where H 0 has infinite dimension. Since this space involves functions valued in O(H 0 , G 0 ) which is not a Banach space, we need to introduce a suitable notion of measurability for such functions. The following definition is slightly adapted from Mandrekar andSalehi, 1970, Kakihara (1997, Section 3.4).

Definition 5.1.3 (O-measurability). Let (Λ, A) be a measurable space and H, G be two Hilbert spaces. Then a function Φ : Λ → O(H, G) is said to be Omeasurable if it satisfies the two following conditions.

(i) For all x ∈ H, {λ ∈ Λ : x ∈ D(Φ(λ))} ∈ A.

(ii) There exist a sequence (Φ n ) n∈N valued in F s (Λ, A, H, G) such that for all λ ∈ Λ and x ∈ D(Φ(λ)), Φ n (λ)x converges to Φ(λ)x in G as n → ∞.

We denote by F O (Λ, A, H, G) the space of such functions Φ.

Then we introduce the following definition. Definition 5.1.4. Let (Λ, A) be a measurable space, H 0 , G 0 , I 0 be three separable Hilbert spaces and ν a trace-class p.o.v.m. on (Λ, A, H 0 ) with density f with respect to its finite variation ν 1 , as defined in Theorem 4.5.2. Then, we say that the pair (Φ,

Ψ) ∈ F O (Λ, A, H 0 , G 0 ) × F O (Λ, A, H 0 , I 0 ) is ν-integrable if the three following assertions hold. (i) We have Im( f 1/2 ) ⊂ D(Φ) and Im( f 1/2 ) ⊂ D(Ψ), ν 1 -a.e. (ii) We have Φ f 1/2 ∈ S 2 (H 0 , G 0 ) and Ψ f 1/2 ∈ S 2 (H 0 , I 0 ), ν 1 -a.e. (iii) We have (Φ f 1/2 )(Ψ f 1/2 ) H ∈ L 1 (Λ, A, S 1 (I 0 , G 0 ), ν 1 ).
In the case, we define

ΦdνΨ H := (Φ f 1/2 )(Ψ f 1/2 ) H d ν 1 ∈ S 1 (I 0 , G 0 ) . (5.1.3) Moreover, we say that Φ ∈ F O (Λ, A, H 0 , G 0 ) is square ν-integrable if (Φ, Φ)
is ν-integrable and we denote by L 2 (Λ, A, O(H 0 , G 0 ), ν) the space of square νintegrable functions in F O (Λ, A, H 0 , G 0 ). 2) It is easy to show that for all Φ, Ψ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν), the pair (Φ, Ψ) is ν-integrable and thus ΦdνΨ H is well defined as above.

3) We have the following inclusions (5.1.4) and the Gramian [Φ, Ψ] ν defined on the smaller space as in (4.5.3) coincides with ΦdνΨ H defined in (5.1.3). However the equality [Φ, Ψ] ν = Φ f Ψ H is only valid when Φ, Ψ belong in one of the first two spaces of (5.1.4) where this integral is well defined.

L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) ⊂ L 2 (Λ, A, L b (H 0 , G 0 ), ν) ⊂ L 2 (Λ, A, O(H 0 , G 0 ), ν) ,
The following theorem, whose proof can be found in Section 5.3.1, shows that the same Gramian can be used over the larger space L 2 (Λ, A, O(H 0 , G 0 ), ν) and that it makes this space a normal Hilbert L b (G 0 )-module when quotiented by the set with zero norm.

Theorem 5.1.2. Let H 0 , G 0 be separable Hilbert spaces, (Λ, A) a measurable space,

ν a trace-class p.o.v.m. on (Λ, A, H 0 ) and f = dν d ν 1 . Then L 2 (Λ, A, O(H 0 , G 0 ), ν) is an L b (G 0 )-module with module action P • Φ : λ → PΦ(λ), P ∈ L b (G 0 ), Φ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν) . Moreover, we can endow L 2 (Λ, A, O(H 0 , G 0 ), ν) with the pseudo-Gramian [Φ, Ψ] ν := ΦdνΨ H Φ, Ψ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν) .
(5.1.5)

Then, for all Φ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν), we have

Φ ν = [Φ, Φ] ν 1/2 1 = 0 ⇐⇒ Φ f 1/2 = 0 ν 1 -a.e.
Let us denote the class of such Φ's by { • ν = 0} and the quotient space by

L 2 (Λ, A, O(H 0 , G 0 ), ν) := L 2 (Λ, A, O(H 0 , G 0 ), ν) { • ν = 0} . Then L 2 (Λ, A, O(H 0 , G 0 ), ν), [•, •] ν is a normal Hilbert L b (G 0 )-module. Clearly, the normal Hilbert L b (G 0 )-module L 2 (Λ, A, O(H 0 , G 0 ), ν), [•,
•] ν contains the pre-Hilbert one of Example 4.5.1. The next result, whose proof can be found in Section 5.3.1, says that it is the smallest one.

Theorem 5.1.3. Let H 0 , G 0 be two separable Hilbert spaces, (Λ, A) a measurable space, and ν a trace-class p.o.v.m. on (Λ, A, H 0 ). Then L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) is dense in L 2 (Λ, A, O(H 0 , G 0 ), ν) and the following assertions hold.

(i) The space Span (1 A P :

A ∈ A, P ∈ L b (H 0 , G 0 )) of simple L b (H 0 , G 0 )-valued functions is dense in L 2 (Λ, A, O(H 0 , G 0 ), ν). (ii) For any subset E ⊂ L 2 (Λ, A, ν 1 ) which is linearly dense in L 2 (Λ, A, ν 1 ), Span (h P : h ∈ E, P ∈ L b (H 0 , G 0 )) is dense in L 2 (Λ, A, O(H 0 , G 0 ), ν).
In some of the definitions above, it can be useful to replace ν 1 can be by any σ-finite non-negative measure µ dominating ν 1 and the following characterization hold (see Section 5.3.1 for a proof). Proposition 5.1.4. Let (Λ, A) be a measurable space, H 0 , G 0 , I 0 be three separable Hilbert spaces and ν a trace-class p.o.v.m. on (Λ, A, H 0 ). Let µ be a σ-finite nonnegative measure dominating ν 1 and set g = dν dµ , as defined in Theorem 4.5.2. Then the following assertions hold.

(a) For all (Φ, Ψ) ∈ F O (Λ, A, H 0 , G 0 ) × F O (Λ, A, H 0 , I 0 ), the pair (Φ, Ψ) is ν-integrable if and only if the three following assertions hold.

(i') We have Im(g 1/2 ) ⊂ D(Φ) and Im(g 1/2 ) ⊂ D(Ψ), µ-a.e. (ii') We have Φg 1/2 ∈ S 2 (H 0 , G 0 ) and Ψg 1/2 ∈ S 2 (H 0 , I 0 ), µ-a.e. (iii') (Φg 1/2 )(Ψg 1/2 ) H ∈ L 1 (Λ, A, S 1 (G 0 , I 0 ), µ).
In this case we have

ΦdνΨ H = (Φg 1/2 )(Ψg 1/2 ) H dµ . (5.1.6) (b) For all Φ ∈ F O (Λ, A, H 0 , G 0 ), we have Φ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν) if and only if      Im(g 1/2 ) ⊂ D(Φ) µ-a.e. Φg 1/2 ∈ L 2 (Λ, A, S 2 (H 0 , G 0 ), µ) (c) If Φ, Ψ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν), then the pair (Φ, Ψ) is ν-integrable and ΦdνΨ H = Φg 1/2 , Ψg 1/2 L 2 (Λ,A,S 2 (H 0 ,G 0 ),µ) , (5.1.7)
where the latter Gramian comes from Example 4.3.3. In particular, this means that the mapping

Φ → Φg 1/2 is Gramian-isometric from L 2 (Λ, A, O(H 0 , G 0 ), ν) to L 2 (Λ, A, S 2 (H 0 , G 0 ), µ).
Finally, we conclude this section by investigating the equality cases for the two inclusions in (5.1.4). These questions are of interest for various reasons. First, we consider the second inclusion involving L 2 (Λ, A, L b (H 0 , G 0 ), ν) and L 2 (Λ, A, O(H 0 , G 0 ), ν). Because endowing these two spaces with the same seminorm • ν makes the quotient normed space L 2 (Λ, A, O(H 0 , G 0 ), ν) the completion of the quotient normed space L 2 (Λ, A, L b (H 0 , G 0 ), ν), the equality case means that L 2 (Λ, A, L b (H 0 , G 0 ), ν) is complete. This is characterized, in Theorem 5.1.5, by having that dν d ν 1 has finite rank ν 1 -a.e. As far as we know, this is a new result. Its proof is given in Section 5.3.1. Theorem 5.1.5. Let H 0 , G 0 be separable Hilbert spaces, (Λ, A) a measurable space and ν a trace-class p.o.v.m. on (Λ, A, H 0 ) with density f with respect to its finite variation ν 1 . Then the following assertions are equivalent.

(i) We have the equality

L 2 (Λ, A, L b (H 0 , G 0 ), ν) = L 2 (Λ, A, O(H 0 , G 0 ), ν). (ii) The space L 2 (Λ, A, L b (H 0 , G 0 ), ν) is complete for the norm • ν . (iii) We have rank( f ) < +∞, ν 1 -a.e.
Moreover, if any of the above assertions holds, then rank dν dµ is finite µ-a.e. for all σ-finite non-negative measure µ which dominates ν 1 .

For sake of completeness, we also discuss the equality case for the first inclusion in (5.1.4). First note that, by (4.5.4), the inclusion defines a continuous embedding

L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) → L 2 (Λ, A, L b (H 0 , G 0 ), ν),
where L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) is endowed with its usual L 2 norm and L 2 (Λ, A, L b (H 0 , G 0 ), ν) with • ν . The comparison of the norms in (4. 5.4) relies on the point-wise inequality Tr Φ f Φ H ≤ Φ 2 L b (H 0 ,G 0 ) which follows from the fact that f ∈ S + 1 (H 0 ) with unit trace. Of course, if there exists a constant C > 0 such that f C Id H 0 ν 1 -a.e., then the opposite inequality holds up to a positive multiplicative constant and the equality case with the corresponding embedding is obtained. Note that this can only happen if H 0 has finite dimension since otherwise Id H 0 is not trace-class. It turns out that this case is the only equality case for the first inclusion in (5.1.4) as stated in the following result.

Theorem 5.1.6. Let H 0 , G 0 be separable Hilbert spaces, (Λ, A) a measurable space and ν a trace-class p.o.v.m. on (Λ, A, H 0 ) with density f with respect to its finite variation ν 1 . Consider the following assertions.

(i) There exists a constant C > 0 such that f C Id H 0 , ν 1 -a.e.

(ii) There exists a constant C > 0 such that, for all

Φ ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν) Φ 2 ν ≤ Φ 2 L 2 (Λ,A,L b (H 0 ,G 0 ), ν 1 ) ≤ C -1 Φ 2 ν . (iii) We have the equality L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) = L 2 (Λ, A, L b (H 0 , G 0 ), ν).
Then, we have (i) ⇔ (ii) ⇒ (iii). If, in addition, the measure space (Λ, A, ν 1 ) is such that

∃(A n ) n∈N ∈ A N with ∀n = m , A n ∩ A m = 0 and ∀n ∈ N µ(A n ) > 0 (5.1.8) then we have (iii) ⇒ (ii).
In particular, if the measure space (Λ, A, ν 1 ) satisfies (5.1.8) and H 0 has infinite dimension, then the first inclusion of (5.1.4) is strict.

Integration with respect to a random c.a.g.o.s. measure

Having all the necessary notions for a clear definition of the modular spectral domain, we now define the mapping which makes it Gramian-isometrically isomorphic to the modular time domain. This definition is often presented as a stochastic integral because it linearly and continuously maps a function to a random variable.

Let H 0 and G 0 be two separable Hilbert spaces, (Λ, A) be a measurable space, and let ν be a trace-class p.o.v.m. defined on (Λ, A, H 0 ). Given an H 0 -valued random c.a.g.o.s. measure W, we further set

H W,G 0 := Span G (PW(A) : P ∈ L b (H 0 , G 0 ), A ∈ A) , (5.1.9)
which is a submodule of G := M(Ω, F , G 0 , P). As in Proposition 13 in Kakihara (1997, Section 3.4) and Mandrekar and Salehi (1970, Theorem 6.9), we now define the integral of an H 0 → G 0 -operator valued function with respect to a random c.a.g.o.s. measure W as a Gramian-isometry from the normal Hilbert L b (G 0 )-module L 2 (Λ, A, O(H 0 , G 0 ), ν W ) to H W,G 0 . A proof can be found in Section 5.3.1.

Theorem 5.1.7. Let (Λ, A) be a measurable space and (Ω, F , P) a probability space. Let H 0 and G 0 be two separable Hilbert spaces. Let W be an H 0 -valued random c.a.g.o.s. measure on (Λ, A, Ω, F , P) with intensity operator measure ν W .

Let H W,G 0 be defined as in (5.1.9). Then there exists a unique Gramian-isometry

I G 0 W : L 2 (Λ, A, O(H 0 , G 0 ), ν W ) → M(Ω, F , G 0 , P)
such that, for all A ∈ A and P ∈ L b (H 0 , G 0 ),

I G 0 W (1 A P) = PW(A) P-a.s.
Moreover, L 2 (Λ, A, O(H 0 , G 0 ), ν W ) and H W,G 0 are Gramian-isometrically isomorphic.

We can now define the integral of an operator valued function with respect to W. The following remark will be useful.

Remark 5.1.3. In the setting of Definition 5.1.5, take Φ = φ Id H 0 with φ : Λ → C. Then, we have Φ ∈ L 2 (Λ, A, O(H 0 ), ν W ) if and only if φ ∈ L 2 (Λ, A, ν W 1 ). We will omit Id H 0 in the notation of the integral, writing φ dW for φId H 0 dW.

We conclude this section with a kind of Fubini theorem for interchanging a Bochner integral with a c.a.g.o.s. integral. Proposition 5.1.8. Let (Λ, A) be a measurable space and H 0 , G 0 two separable Hilbert spaces. Let W be an H 0 -valued random c.a.g.o.s. measure on (Λ, A, Ω, F , P) with intensity operator measure ν W . Let µ be a non-negative measure on a measurable space (Λ , A ). Suppose that Φ is measurable from Λ × Λ to L b (H 0 , G 0 ) and satisfies

Φ(λ, λ ) L b (H 0 ,G 0 ) µ(dλ ) 2 ν W 1 (dλ) < ∞ , (5.1.11) Φ(λ, λ ) 2 L b (H 0 ,G 0 ) ν W 1 (dλ) 1/2 µ(dλ ) < ∞ .
(5.1.12)

Then we have

Φ(λ, λ ) µ(dλ ) W(dλ) = Φ(λ, λ ) W(dλ) µ(dλ ) ,
(5.1.13) where integrals with respect to W are as in Definition 5.1.5, in the left-hand side the innermost integral is understood as a Bochner integral on L 2 (Λ , A , L b (H 0 , G 0 ), µ) and in the right-hand side, the outermost integral is understood as a Bochner integral on L 2 (Λ , A , M(Ω, F , G 0 , P), µ).

Proof. Conditions (5.1.11) and ( 5.1.12) ensure that, for

ν W 1 -a.e. λ ∈ Λ Φ(λ, •) ∈ L 1 (Λ , A , L b (H 0 , G 0 ), µ) and that, for µ-a.e. λ ∈ Λ , Φ(•, λ ) ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 )
, thus showing that the innermost integrals in both sides of (5.1.13) are well defined for adequate sets of λ and λ , respectively.

Let E 1 and E 2 denote the sets of functions Φ measurable from Λ × Λ to L b (H 0 , G 0 ) and satisfying (5.1.11) and ( 5.1.12), respectively. We denote by Φ E 1 the square root of the left-hand side of (5.1.11) and by Φ E 2 the lefthand side of (5.1.12), which make E 1 and E 2 Banach spaces. Then, for all Φ ∈ E := E 1 ∩ E 2 , concerning the left-hand side of (5.1.13), we have

Φ(•, λ ) µ(dλ ) 2 ν W ≤ Φ(•, λ ) µ(dλ ) 2 L b (H 0 ,G 0 ) d ν W 1 ≤ Φ 2 E 1 ,
as for the right-hand side, we have, setting H := M(Ω, F , G 0 , P),

Φ(λ, •) W(dλ) H dµ = Φ(•, λ ) ν W µ(dλ ) ≤ Φ E 2 ,
These two inequalities show that both sides of (5.1.13) seen as functions of Φ are linear continuous from E endowed with the norm

• E = • E 1 + • E 2 to M(Ω, F , G 0 , P). Since they coincide for Φ(λ, λ ) = 1 A (λ)1 B (λ )P with A ∈ A, B ∈ A and P ∈ L b (H 0 , G 0 )
, this concludes the proof.

The Gramian-Cramér representation

We now have all the tools to derive a spectral theory for Hilbert valued weakly-stationary processes following the approach of Kakihara (1997, Section 4.2). Let (Ω, F , P) be a probability space, H 0 be a separable Hilbert space and (G, +) be a locally compact Abelian (l.c.a.) group, whose null element is denoted by 0. Recall that Ĝ denotes the dual group of G defined in Section 4.2. Throughout this section we are interested in the spectral properties of a centered process valued in a separable Hilbert space and assumed to be weakly stationary in the following sense.

Definition 5.2.1 (Hilbert valued weakly stationary processes). Let (Ω, F , P) be a probability space, H 0 be a separable Hilbert space and (G, +) be an l.c.a. group. Then a process X := (X t ) t∈G is said to be an H 0 -valued weakly stationary process if

(i) For all t ∈ G, X t ∈ L 2 (Ω, F , H 0 , P). (ii) For all t ∈ G, E [X t ] = E [X 0 ]. We say that X is centered if E [X 0 ] = 0.
(iii) For all t, h ∈ G, Cov (X t+h , X t ) = Cov (X h , X 0 ).

(iv) The autocovariance operator function Γ X : h → Cov (X h , X 0 ) satisfies the following continuity condition: for all

P ∈ L b (H 0 ), h → Tr(PΓ X (h)) is continuous on G.
In the case of time series, G = Z, we can of course remove Condition (iv) in this definition. It is less trivial to show that, for any l.c.a. group G, we get an equivalent definition if we replace (iv) by just saying that Γ X is continuous in w.o.t. This interesting fact is explained in the following remark in a more detailed fashion.

Remark 5.2.1. Let us comment briefly the continuity assumption (iv) from Definition 5.2.1.

1) The trace appearing in Assertion (iv) of Definition 5.2.1 is well defined for any P ∈ L b (H 0 ) and any h ∈ G since the covariance operator of variables in L 2 (Ω, F , H 0 , P) lies in S 1 (H 0 ), and the composition of a bounded operator and a trace-class operator is trace-class. Furthermore, for any x, y ∈ H 0 , taking P = xy H we have Tr(PΓ X (h)) = Γ X (h)x, y H 0 . Hence Condition (iv) of Definition 5.2.1 implies the following one.

(iv') The autocovariance operator function

Γ X : h → Cov (X h , X 0 ) is contin- uous in w.o.t.
It is easy to find a mapping f : G → S 1 (H 0 ) which is continuous in w.o.t. but such that h → Tr( f (h)) is not continuous hence does not satisfy the continuity condition imposed on Γ X in (iv). However, it turns out that if Γ X is the autocovariance operator function h → Cov (X h , X 0 ) with X satisfying Conditions (i) and (iii), then Conditions (iv) and (iv') become equivalent. The reason behind this surprising fact comes from Corollary 4.5.4 as it is easily verified that Γ X satisfies Assertion (i) of Corollary 4.5.4. In other words, we can replace (iv) by (iv') without altering Definition 5.2.1.

2) The previous remark is related to a fact established in Proposition 3 of Kakihara (1997, Section 4.2), which states the equivalence between being scalar stationary and being operator stationary. The latter definition is the same as our Definition 5.2.1, and the former one amounts to replace Condition (iv) in Definition 5.2.1 by assuming that for all x ∈ H 0 , x H Γx : h → x H Γ(h)x is continuous and hermitian non-negative definite. But this amounts to says that Γ itself is continuous in the w.o.t. and hermitian non-negative definite.

Since Γ(0) ∈ S 1 (H 0 ) is a consequence of Assertion (i) in Definition 5.2.1, the equivalence between (iv) and (iv') implies the equivalence established in Proposition 3 of Kakihara (1997, Section 4.2).

As in the univariate case, the notion of weak stationarity is related to an isometric property of the lag operators, but here the covariance stationarity expressed in Condition (iii) translates into a Gramian-isometric property rather than a scalar isometric property. Namely, let X := (X t ) t∈G satisfy Conditions (i) and (ii) and take it centered so that each X t belongs to the normal Hilbert module M(Ω, F , H 0 , P) as defined in Example 4.3.2. For all h ∈ G, define the lag operator of lag h ∈ G as the mapping U X h : X t → X t+h defined for all t ∈ G. Then Condition (iii) is equivalent to saying that for all h ∈ G, the mapping U X h is Gramian-isometric on {X t : t ∈ G} for the Gramian structure inherited from M(Ω, F , H 0 , P). Thus, if this condition holds, for any lag h ∈ G, using Proposition 4.3.1, there exists a unique Gramian-unitary operator extending U X h on the modular time domain H X of X defined as the submodule of M(Ω, F , H 0 , P) generated by the X t 's, that is,

H X := Span M(Ω,F ,H 0 ,P) (PX t : P ∈ L b (H 0 ), t ∈ G) ,
which is the generalization of (ii.5) to a general l.c.a. group G. An interesting property of the lag operators is the following.

Lemma 5.2.1. Let (Ω, F , P) be a probability space, H 0 be a separable Hilbert space and (G, +) be an l.c.a. group. Let X := (X t ) t∈G be an H 0 -valued stochastic process satisfying (i) and (ii) of Definition 5.2.1 with E [X 0 ] = 0. Then X is weakly stationary if and only if U X is a c.g.u.r of G on H X .

Proof. The proof is a consequence of the definition of (U X h ) h∈G and following identity which holds for all t, h ∈ G,

Cov (X t+h , X t ) = U X h X t , X t H X = U X t X h , U X t X 0 H X .
In fact it is convenient to introduce a slightly more general definition of the modular time domain. Definition 5.2.2 (G 0 -valued modular time domain). Let (G, +) be an l.c.a. group, and H 0 and G 0 be two separable Hilbert spaces. Let X := (X t ) t∈G be a collection of variables in M(Ω, F , H 0 , P) as defined in Example 4.3.2. The G 0valued modular time domain of X is defined by

H X,G 0 := Span M(Ω,F ,G 0 ,P) (PX t : P ∈ L b (H 0 , G 0 ), t ∈ G) , (5.2.1)
which is a submodule of M(Ω, F , G 0 , P).

We now extend the (scalar) Cramér representation theorem by means of an integral with respect to a c.a.g.o.s. measure. Theorem 5.2.2 (Gramian-Cramér representation theorem). Let H 0 be a separable Hilbert space, (Ω, F , P) be a probability space and (G, +) be an l.c.a. group. Let X := (X t ) t∈G be a centered weakly stationary H 0 -valued process as in Definition 5.2.1. Then there exists a unique regular H 0 -valued random c.a.g.o.s. measure X on ( Ĝ, B( Ĝ), Ω, F , P) such that X t = χ(t) X(dχ) for all t ∈ G .

(5.

2.2)

This result is stated in Theorem 2 in Kakihara (1997, Section 4.2) without the uniqueness, which appeared to be a new result in this general setting. We provide a detailed proof in Section 5.3.2. In fact Theorem 2 in Kakihara (1997, Section 4.2) contains a converse statement, which we now state separately as a lemma with its proof. Lemma 5.2.3. Let (G, +) be an l.c.a. group, H 0 a separable Hilbert space and W be an H 0 -valued random c.a.g.o.s. measure on ( Ĝ, B( Ĝ), Ω, F , P) with intensity operator measure ν. Define, for all t ∈ G,

X t = χ(t) W(dχ) .
Then X = (X t ) t∈G is a centered H 0 -valued weakly stationary process with autocovariance operator function Γ defined by

Γ(h) = χ(h) ν(dχ) for all h ∈ G.
(5.2.3) Proof. By Definition 5.1.5, X = (X t ) t∈G is a centered H 0 -valued process satisfying (i) and (ii) in Definition 5.2.1. Using the Gramian-isometric property of integration with respect to W, we get for all t, h ∈ G, Cov (X t+h , X t ) = χ(t + h)χ(t)ν X (dχ) = χ(h)ν X (dχ) which gives (iii) in Definition 5.2.1 with auto-covariance operator function Γ given by (5.2.3). The last point of Definition 5.2.1 comes from Corollary 4.5.4 because ν is trace-class.

With Theorem 5.2.2 at our disposal, we can now define the Gramian-Cramér representation and the spectral operator measure of X. Definition 5.2.3 (Gramian-Cramér representation and spectral operator measure). Under the setting of Theorem 5.2.2, the regular c.a.g.o.s. measure X is called the (Gramian) Cramér representation of X and its intensity operator measure is called the spectral operator measure of X. It is a regular trace-class p.o.v.m. on ( Ĝ, B( Ĝ), H 0 ). By Lemma 5.2.3, we see that the auto-covariance operator function and the spectral operator measure of X are related to each other through the identity (5.2.3). As already hinted in the introduction, using the tools introduced in Section 5.1.3, we can more generally interpret the Cramér representation of Theorem 5.2.2 as establishing a Gramian-isometric mapping onto the modular time domain of X, starting from its modular spectral domain which we now introduce. Definition 5.2.4 (G 0 -valued spectral time domain). Let H 0 and G 0 be two separable Hilbert spaces and X := (X t ) t∈G be a centered weakly stationary process valued in H 0 as in Definition 5.2.1. The G 0 -valued modular spectral domain of X is the normal Hilbert L b (G 0 )-module defined by

H X,G 0 := L 2 ( Ĝ, B( Ĝ), O(H 0 , G 0 ), ν X ) , (5.2.4)
where ν X is the spectral operator measure of X introduced in Definition 5.2.3.

We can now state that the modular time and spectral domain are Gramianisometrically isomorphic, whose proof can be found in Section 5.3.2. Theorem 5.2.4 (Kolmogorov isomorphism theorem). Under the setting of Theorem 5.2.2, for any separable Hilbert space G 0 , the mapping I G 0 X : Φ → Φ d X is a Gramian-unitary operator from H X,G 0 to H X,G 0 and we have H X,G 0 = H X,G 0 . Thus, the G 0 -valued modular time domain H X,G 0 and the G 0 -valued modular spectral domain H X,G 0 are Gramian-isometrically isomorphic.

Remark 5.2.2.

There are two natural classes of Gramian-unitary operators respectively on the modular time and spectral domains, namely, for all h ∈ G, the lag operator U X h : H X → H X defined as the Gramian-unitary extension of X t → X t+h , t ∈ G, and the multiplication by M X h : H X → H X which maps Φ to χ → χ(h)Φ(χ). Then, for all h ∈ G, U X h and M X h represent the same mapping expressed either in the time domain or the spectral domain in the sense that

U X h • I H 0 X = I H 0 X • M X h for all h ∈ G .
Indeed, applying these definitions with (5.2.2), we immediately get that U X h and

I H 0 X • M X h • I H 0 X -1
are Gramian-isometric and coincide on {X t : t ∈ G}, hence, by Proposition 4.3.1, coincide on H X .

To conclude this section, let us note that Relation (5.2.3) is special case of Relation (4.4.2) from the general Bochner theorem where the function Γ is the autocovariance operator function of a weakly-stationary process X. As stated in Remark 5.2.1, the autocovariance operator function of a weaklystationary process satisfies (i) of Corollary 4.5.4 and therefore we can complete Corollary 4.5.4 as follows.

Corollary 5.2.5. Let (G, +) be an l.c.a. group, H 0 a separable Hilbert space and Γ : G → L b (H 0 ). Then the following assertions are equivalent.

(i) The function Γ is a proper auto-covariance operator function, i.e. there exists an H 0 -valued weakly stationary process with autocovariance operator function Γ.

(ii) Any of the Assertions (i)-(iv) in Corollary 4.5.4 holds. as in Theorem 4.5.2. Using that ν 1 ({g = 0}) = {g=0} g 1 dµ = 0 and g = f g 1 µ-a.e. by uniqueness of the density, we get that g 1 > 0 ν 1 -a.e. and g = f g 1 µ-a.e.

Postponed proofs

(5.3.1) (and thus also ν 1 -a.e. since ν 1 µ). Assertion (a) follows easily. Let us for instance detail the proof of the equivalence between (i') and (i) of Definition 5.1.4. The left-hand side of (5.3.1) gives that

ν 1 Im( f 1/2 ) ⊂ D(Φ) = ν 1 Im( f 1/2 ) ⊂ D(Φ) ∩ {g = 0} , ( 5
.3.2) and its right-hand side yields

µ Im( f 1/2 ) ⊂ D(Φ) ∩ {g = 0} = µ Im(g 1/2 ) ⊂ D(Φ) ∩ {g = 0} (5.3.3) = µ Im(g 1/2 ) ⊂ D(Φ) , since Im(g 1/2 ) ⊂ D(Φ) ∩ {g = 0} = ∅.
To get (i') ⇔ (i), we note that

ν 1 Im( f 1/2 ) ⊂ D(Φ) ∩ {g = 0} = {Im( f 1/2 ) ⊂D(Φ)}∩{g =0} g 1 dµ ,
and thus the right-hand side of (5.3.2) is zero if and only if the left-hand side of (5.3.3) is. Equivalences (ii) ⇔ (ii') and (iii) ⇔ (iii') and Relation (5.1.6) are easy consequences of (5.3.1). Assertions (b) and (c) come easily using the definition of L 2 (Λ, A, O(H 0 , G 0 ), ν). Measurability of Φg 1/2 and (Φg 1/2 )(Φg 1/2 ) are ensured by O-measurability of Φ, simple measurability of f and Lemma B.1.2.

We can now derive Theorem 5.1.2.

Proof of Theorem 5.1.2. All theses results are easily derived from Proposition 5.1.4 and the module nature of L 2 (Λ, A, S 2 (H 0 , G 0 ), µ). The only difficulty lies in showing the completeness of L 2 (Λ, A, O(H 0 , G 0 ), ν) which we now detailed following the proof of Theorem 11 of Kakihara (1997, Section 3.4).

Let f = dν d ν 1
and consider a Cauchy sequence (Φ n ) n∈N in the space L 2 (Λ, A, O(H 0 , G 0 ), ν). Then, by (5.1.7), the sequence (Φ n f 1/2 ) n∈N is a Cauchy sequence in L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ). Since the latter space is complete, there exists a function

Ψ ∈ L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ) such that lim n→+∞ Ψ -Φ n f 1/2 2 2 d ν 1 = 0. Now, consider the measurable eigen- decomposition f = ∑ n∈N σ n φ n ⊗ φ n
as provided by Theorem 4.5.9 and take Φ = Ψ f 1/2 † , where ( f 1/2 ) † : λ = ( f (λ) 1/2 ) † . From (A.1.5) and mea-See Appendix A.1.4 for a definition of the generalized inverse P † of an operator

P ∈ L b (H 0 )
surability of the σ n 's and φ n ⊗ φ n 's, we get that Φ f 1/2 ∈ F O (Λ, A, H 0 , G 0 ). Then, using (A.1.4), we get Φ f 1/2 = ΨΠ (ker f 1/2 ) ⊥ , where

Π (ker f 1/2 ) ⊥ is the orthogonal projection onto (ker f 1/2 ) ⊥ . Hence Φ f 1/2 2 ≤ Ψ 2 and there- fore Φ f 1/2 ∈ L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ). Hence, Φ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν) by Assertion (b) of Proposition 5.1.4. Finally, Φ -Φ n 2 ν = ΨΠ (ker f 1/2 ) ⊥ -Φ n f 1/2 2 2 d ν 1 = (Ψ -Φ n f 1/2 )Π (ker f 1/2 ) ⊥ 2 2 d ν 1 ≤ Ψ -Φ n f 1/2 2 2 d ν 1 ---→ n→∞ 0 ,
thus concluding the proof.

Proof of Theorem 5.1.3. In the first two steps of the proof of Theorem 12 in Kakihara (1997, Section 3.4) (see also Mandrekar and Salehi (1970, Theorem

4.22)), it is shown that, if Φ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν) and > 0, there exists Ψ ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) ⊂ L 2 (Λ, A, O(H 0 , G 0 ), ν) such that Φ -Ψ ν < . This implies that L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) is dense in L 2 (Λ, A, O(H 0 , G 0 ), ν).
Then Assertion (i) follows using (4.5.4) and the usual density of simple functions. Assertion (ii) then follows by approximating, for any A ∈ A and P ∈ L b (H 0 , G 0 ) the function 1 A P by gP with g ∈ Span (E) arbitrarily close to 1 A in L 2 (Λ, A, ν 1 ).

Proof of Theorem 5.1.5. The proof of the fact that we can take µ instead of ν 1 uses the same arguments we used to prove Proposition 5.1.4 and will be omitted. The equivalence between (i) and (ii) is obvious since the space L 2 (Λ, A, L b (H 0 , G 0 ), ν) is dense in L 2 (Λ, A, O(H 0 , G 0 ), ν) by Theorem 5.1.3. It remains to show the equivalence (ii) ⇔ (iii). To this end, we consider the density f : λ → ∑ n∈N σ n (λ)φ n (λ) ⊗ φ n (λ) of ν with respect to ν 1 obtained by Assertion (iii) in Theorem 4.5.9, and we let

A := {rank f 1/2 = +∞} = n∈N {σ n > 0} ∈ A. Then Assertion (iii) is equivalent to ν 1 (A) = 0 .
(5.3.4)

We now prove that (ii) is equivalent to (5.3.4).

Proof of (ii) ⇒ (5.3.4). Suppose that L 2 (Λ, A, L b (H 0 , G 0 ), ν) is complete for the norm • ν and that ν 1 (A) > 0. Then in order to get a contradiction, we will follow the following two steps.

Step

1 Construct a function Ψ ∈ L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ) such that for all λ ∈ A, Ψ(λ) / ∈ P f (λ) 1/2 : P ∈ L b (H 0 , G 0 ) . (5.3.5)
Step 2 Construct a sequence

(Φ n ) n∈N ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) N such that Φ n f 1/2 converges to Ψ in L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ).
Let us explain why these two steps lead to a contradiction.

Step

2 implies that (Φ n f 1/2 ) n∈N is Cauchy in L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ) which means that (Φ n ) n∈N is Cauchy in L 2 (Λ, A, L b (H 0 , G 0 ), ν). Since we assumed complete- ness, there exists Φ ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν) such that Φ n -Φ ν ----→ n→+∞ 0.
Again, this means that Φ n f 1/2 converges to Φ f 1/2 in L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ) and thus Φ f 1/2 = Ψ ν 1 -a.e. contradicting (5.3.5).

We now provide the constructions previously described.

Step 1 Let u ∈ G 0 with u G 0 = 1 and define

y : λ → ∑ n∈N σ n (λ)φ n (λ) , and Ψ : λ → u ⊗ y(λ) .
Then for all λ ∈ Λ, we have Ψ(λ) 2 2 = y 2 H 0 = ∑ n∈N σ n (λ) = f (λ) 1 and therefore Ψ ∈ L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ). We now show that Ψ satisfies (5.3.5). To this end, we suppose that (5.3.5) does not hold and show that this leads to a contradiction. Taking λ ∈ A and P

∈ L b (H 0 , G 0 ) such that Ψ(λ) = P f (λ) 1/2 , we have y(λ) ⊗ u = Ψ(λ) H = f (λ) 1/2 P H and thus y(λ) = (y(λ) ⊗ u) (u) = f (λ) 1/2 P H u ∈ Im( f (λ) 1/2 ) .
This means that there exists x ∈ H 0 such that y(λ) = f (λ) 1/2 x and therefore, for all n ∈ N,

σ n (λ) = y(λ), φ n (λ) H 0 = f (λ) 1/2 x, φ n (λ) H 0 = x, f (λ) 1/2 φ n (λ) H 0 = σ n (λ) x, φ n (λ) H 0 ,
where we have used the fact that f (λ) 1/2 = ∑ n∈N σ n (λ)φ n (λ) ⊗ φ n (λ). Since λ ∈ A, we have σ n (λ) > 0 and therefore x, φ n (λ) H 0 = 1 for all n ∈ N. This would mean that x H 0 = +∞, which is impossible thus concluding Step 1.

Step 2 Define

Φ n : λ → u ⊗ n ∑ k=0 φ k (λ) . Then Φ n ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ) and for all λ ∈ Λ, Φ n (λ) f 1/2 (λ) = u ⊗ ∑ n k=0 σ n (λ)φ k (λ). Hence, for all λ ∈ Λ, Ψ(λ) -Φ n (λ) f 1/2 (λ) 2 2 = +∞ ∑ k=n+1 σ k (λ) ,
which tends to 0 as n → +∞ and is bounded by f (λ) 1 which is equal to 1, ν 1 -a.e. Hence by Lebesgue's dominated converge theorem

Ψ -Φ n f 1/2 2 2 d ν 1 ----→ n→+∞ 0 , which concludes Step 2.
Proof of (5.3.4) ⇒ (ii). The proof follows the same path than the proof of Theorem 5.1.2. The only difference in the proof is the fact that, this time, we need to show that, under Assumption (5.3.4), for any

Ψ ∈ L 2 (Λ, A, S 2 (H 0 , G 0 ), ν 1 ), we have Ψ( f 1/2 ) † ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν) i.e Ψ( f 1/2 ) † ∈ F s (Λ, A, H 0 , G 0 ) and Ψ( f 1/2 ) † f 1/2 ∈ L 2 (Λ, AS 2 (H 0 , G 0 ), ν 1 ). The second point is a clear consequence of the fact that Ψ( f 1/2 ) † f 1/2 = ΨΠ (ker f 1/2 ) ⊥ , where Π (ker f 1/2 ) ⊥
is the orthogonal projection onto (ker f 1/2 ) ⊥ . For the first point, it suffices to show that we can take a representing function of ( f 1/2 ) † in F s (Λ, A, H 0 ). To this end, assume that (5.3.4) holds, i.e. that rank f 1/2 is finite ν 1a.e., and take a representing function of f 1/2 with finite rank everywhere. .1.5) and measurability of the σ n 's and φ n 's.

Then D( f 1/2 ) † = Im f 1/2 ⊗ (Im f 1/2 ) ⊥ = H 0 and the fact that ( f 1/2 ) † ∈ F s (Λ, A, H 0 ) is a consequence of (A
Proof of Theorem 5.1.6. The proof of (i) ⇔ (ii) ⇒ (iii) has already been explained before the statement of the theorem. Moreover, if H 0 has infinite dimension, then (i) does not hold. Hence, proof is completed if we show that, under (5.1.8), if (i) does not hold neither does (iii). We now assume that (5.1.8) holds and that (i) does not hold and let f = ∑ k∈N σ k φ k ⊗ φ k be the measurable eigendecomposition of f as provided by Theorem 4.5.9.

First, let us assume that we can define three measurable functions g : Λ → R + , n : λ → N and h : Λ → R + satisfying σ n + g > 0 and ν 1 -essinf(σ n + g) = 0 , (5.3.6)

h 2 d ν 1 = ∞ , (5.3.7) h 2 σ n d ν 1 < ∞ , (5.3.8)
where, here,

σ n : λ → σ n(λ) (λ). Then, defining Φ : λ → h(λ)ψ 0 φ H n(λ)
, where ψ 0 ∈ G 0 with unit norm, we get

Φ 2 ν = h 2 σ n d ν 1 < ∞ , and Φ 2 L 2 (Λ,A,L b (H 0 ,G 0 )) = h 2 d ν 1 = ∞ . Hence Φ ∈ L 2 (Λ, A, L b (H 0 , G 0 ), ν) \ L 2 (Λ, A, L b (H 0 , G 0 ), ν 1 ), which con- tradicts (iii).
To conclude the proof, we now construct such functions g : Λ → R + , h : Λ → R + and n : λ → N. Note that, if (5.3.6) holds, then defining B k = {2 -k < σ n + g ≤ 2 1-k }, for all k ≥ 1, we get that there infinitely many k's in N such that ν 1 (B k ) > 0 and therefore the function

h 2 = ∞ ∑ k=1 v k 1 B k , with v k = 2 k ( ν 1 (B k )) -1 if ν 1 (B k
) > 0 and 0 otherwise satisfies (5.3.7) and (5.3.8). Hence, it only remains to show that we can define measurable g : Λ → R + and n : λ → N such that (5.3.6) holds. To this end, let us define ρ : λ → inf n∈N σ n (λ) and let (A k ) k∈N be as in (5.1.8). Define

g = 1 {ρ≥1} + +∞ ∑ k=1 2 -k 1 {2 -k ≤ρ<2 1-k } + +a 1 {ρ=0} , with a = ∑ k∈N 2 -k 1 A k + 1 ( k∈N A k ) c .
Then, for all λ ∈ Λ, we have g(λ) > 0 and therefore the function

n : λ → min {k ∈ N : σ k (λ) ≤ ρ(λ) + g(λ)} ,
is well defined. Moreover, from the inequalities

σ n + g ≤ ρ + 2g ≤ (ρ + 2)1 {ρ≥1} + +∞ ∑ k=1 2 2-k 1 {2 -k ≤ρ<2 1-k } + 2a 1 {ρ=0} ,
we get that, for all m ≥ 1, (5.3.9) To conclude that (5.3.6) holds, it suffices to prove that one of the summands in the right-hand side of (5.3.9) not null. We distinguish two cases: first, if ν 1 ({ρ > 0}) > 0, we have ν 1 {0 < ρ < 2 -(m+1) } > 0 because (i) does not hold. On the other hand, if ν 1 ({ρ > 0}) = 0, we have (5.1.8).

ν 1 {σ n + g ≤ 2 -m } ≥ ν 1 {0 < ρ < 2 -(m+1) } + +∞ ∑ k=m ν 1 ({ρ = 0} ∩ A k ) .
∑ k∈N ν 1 ({ρ = 0} ∩ A k ) = ∑ k∈N ν 1 (A k ) > 0 by
Proof of Theorem 5.1.7. We set H = M(Ω, F , H 0 , P) and G = M(Ω, F , G 0 , P). For all A, B ∈ A and P, Q ∈ L b (H 0 , G 0 ), we have, by Theorem 5.1.2,

[1 A P, 1 B Q] ν W = Pν W (A ∩ B)Q H = P Cov (W(A), W(B)) Q H = Cov (PW(A), QW(B)) = [PW(A), QW(B)] G .
Then Proposition 4.3.1, applied to J = A × L b (H 0 , G 0 ) with v (A,P) = 1 A P and w (A,P) = PW(A), gives that there exists a unique Gramian-isometric operator .3.11) This gives that 5.3.12) Therefore, by Theorem 5.1.3, the domain of I G 0 W is equal to the whole space L 2 (Λ, A, O(H 0 , G 0 ), ν W ). Finally, (5.3.11) with (5.3.10) yields

I G 0 W from Span L 2 (Λ,A,O(H 0 ,G 0 ),ν W ) (1 A QP : A ∈ A, P ∈ L b (H 0 , G 0 ), Q ∈ L b (G 0 )) to G such that for all A ∈ A, P ∈ L b (H 0 , G 0 ), I G W (1 A P) = PW(A) and, in ad- dition, Im(I G 0 W ) = Span G (QPW(A) : A ∈ A, P ∈ L b (H 0 , G 0 ), Q ∈ L b (G 0 )) . (5.3.10) Now, note that L b (H 0 , G 0 ) = {QP : P ∈ L b (H 0 , G 0 ), Q ∈ L b (G 0 )} . ( 5
Span (1 A QP : A ∈ A, P ∈ L b (H 0 , G 0 ), Q ∈ L b (G 0 )) = Span (1 A P : A ∈ A, P ∈ L b (H 0 , G 0 )) . (
Im(I G 0 W ) = Span G (PW(A) : A ∈ A, P ∈ L b (H 0 , G 0 )) = H W,G 0 ,
which concludes the proof.

Proofs of Section 5.2

Let us start with the proof of the Gramian-Cramér representation theorem, as a consequence of Theorem 4.4.1. The proof relies on the following lemma which gives the link between Gramian-projection-valued measures and c.a.g.o.s. measures.

Lemma 5.3.1. Let H 0 be a separable Hilbert space, H an L b (H 0 )-normal Hilbert module and (Λ, A) a measurable space. Let ξ be a Gramian-projection valued measure on (Λ, A, H). Then for all x 0 ∈ H, the mapping ξx 0 : A → ξ(A)x 0 is a c.a.g.o.s. measure on (Λ, A, H) which is regular if ξ is regular.

Proof. Using the fact that ξ is a p.o.v.m. on (Λ, A, H) and Berberian (1966b, Proposition 1), it is straightforward to see that ξx 0 is an H-valued measure.

Moreover, since ξ is a Gramian-projection-valued measure, we get that for all disjoint A, B ∈ B(G)

[ξ(A)x 0 , ξ(B)x 0 ] H = [ξ(B)ξ(A)x 0 , x 0 ] H = [ξ(B ∩ A)x 0 , x 0 ] H = 0 ,
where the first equality is justified in Kakihara (1997, P. 23) and the second one by Proposition 4.1.2. This proves that ξx 0 is a c.a.g.o.s. measure on (Λ, A, H). In the following, we denote by ν its intensity operator measure.

Then, for all A ∈ A, we have

ν(A) 1 = Tr[ξ(A)x 0 , ξ(A)x 0 ] H = ξ(A)x 0 , x 0 H ,
where the last equality comes from the fact that ξ(A) is an orthogonal projection on H. Now, if ξ is regular, then the measure A → ξ(A)x 0 , x 0 is regular and so is ν 1 by the previous display. This implies that ξx 0 is regular and the proof is concluded.

Proof of Theorem 5.2.2. Suppose that X is weakly stationary as in Definition 5.2.1. Then the collection of lag operators (U X h ) h∈G of Lemma 5.2.1 is a c.g.u.r. of G on H X and therefore Theorem 4.4.1 gives that there exists a regular Gramian-projection-valued measure ξ X on ( Ĝ, B( Ĝ), H X ) such that, for all h ∈ G, U X h = χ(h) ξ X (dχ) .

(5.3.13) Then, by Lemma 5.3.1, the mapping

X : B( Ĝ) → H X A → ξ X (A)X 0 (5.3.14)
is a regular c.a.g.o.s. measure on ( Ĝ, B( Ĝ), H X ) and we denote by ν X its intensity operator measure. Since H X is a submodule of M(Ω, F , H 0 , P), X is also a regular H 0 -valued random c.a.g.o.s. measure on (Ω, F , P), see Definition 5.1.2. Relation (5.2.2) then follows by applying (5.3.13) and the fact that, for all t ∈ G, U h t X 0 = X t and, for all φ : Λ → C measurable and bounded, (5.3.15) where the integral in the left-hand side is defined as in Definition 5.1.5 (see also Remark 5.1.3) and the integral in the right-hand side as in Definition 4.1.5, for the p.o.v.m. ξ X . Relation (5.3.15) obviously holds if φ = 1 A with A ∈ A and also for φ simple by linearity. Now, for a general measurable and bounded φ : Λ → C, we can find a sequence (φ n ) n∈N of simple functions such that |φ n | ≤ |φ| for all n ∈ N and φ n (λ) → φ(λ) as n → ∞ for all λ ∈ Λ. Then, by dominated convergence, φ n converges to φ in L 2 (Λ, A, ν 1 ) and therefore φ n Id converges to φId in L 2 (Λ, A, O(H 0 ), ν). Thus φ n d X → φ d X in H X by the isometric property of the integral of Definition 5.1.5. To get (5.3.15), it now suffices to show that, for all Y ∈ H X ,

φ d X = φ dξ X X 0 ,
φ n dξ X 0 , Y H X → φ dξ X 0 , Y H X .
This follows from the polarization formula, Definition 4.1.5 and dominated convergence.

To show uniqueness, suppose there exists another regular H 0 -valued random c.a.g.o.s. measure W on ( Ĝ, B( Ĝ), Ω, F , P) satisfying the same identity as (5.2.2) with X replaced by W. Then, we get χ(t) X(dχ) = χ(t) W(dχ) for all t ∈ G .

(5.3.16)

Let η denote the Haar measure on G and denote by C c (G) the space of compactly supported functions from G to C. Then, by Rudin (1990, Theorem 1.2.4) and Rudin (1990, Section E.8), the space

E = φ : χ → φ(t)χ(t) η(dt) : φ ∈ L 1 (G, B(G), η) is dense in L 2 ( Ĝ, B( Ĝ), ν W 1 + ν X 1 ). We can thus find, for any A ∈ B( Ĝ), (φ n ) n∈N ∈ C c (G) N such that, defining φn as above, φn → 1 A both in L 2 ( Ĝ, B( Ĝ), ν W 1 ) and in L 2 ( Ĝ, B( Ĝ), ν X 1
). Then by Proposition 5.1.8, we have, for all n ∈ N,

φn (χ) W(dχ) = χ(-t) W(dχ) φ n (t) η(dt) = χ(-t) X(dχ) φ n (t) η(dt) = φn (χ) X(dχ) ,
where we have used (5.3.16) in the second equality. Letting n → ∞, we get W(A) = X(A), thus proving the uniqueness of X.

We can now prove the Kolmogorov isomorphism theorem.

Proof of Theorem 5.2.4. By Theorem 5.1.7 and (5.2.4), I G 0 X is a Gramian-unitary operator from H X,G 0 to H X,G 0 . Thus to conclude, we only need to show that H X,G 0 = H X,G 0 . By (5.2.2), we have for all P ∈ L b (H 0 , G 0 ) and t ∈ G, G 0 , where e t : χ → χ(t). Thus, by (5.2.1), we get that H X,G 0 ⊂ H X,G 0 . The definition of X in (5.3.14) gives the converse inclusion, which achieves the proof.

PX t = I G 0 X (Pe t ) ∈ H X,

Comparison with recent approaches

We can now provide a more thorough comparison with the recent works establishing a spectral theory for functional time series mentioned in the introduction. Hence we take G = Z in this section, so that χ ∈ Ĝ can be replaced by λ ∈ T = R/(2πZ) (or (-π, π]) with χ(h) replaced by e iλh . The functional case usually corresponds to setting H 0 = L 2 (0, 1) but this is unimportant for the following discussion.

Isometry vs Gramian-isometry

As hinted in the introduction, the major benefit of the construction developed in the previous sections is to clarify the spectral domain of a functional weakly-stationary process X as being defined as a set of operator-valued functions, namely L 2 ( Ĝ, B( Ĝ), O(H 0 , G 0 ), ν X ). Moreover, the Gramian-Cramér representation, as stated in Theorem 5.2.2, is a particular instance of the Gramian-unitary operator between the spectral domain and the time domain, based on the integral of Definition 5.1.5. In contrast, the isomorphism constructed in Delft and Eichler (2020, Theorem 4.4) and Panaretos and Tavakoli (2013a, Theorem 2.1) is similar to the one expressed in [START_REF] Holmes | Mathematical Foundations of Signal Processing[END_REF] and recalled in the introduction as the isometric extension of (ii.2) to H X = Span (X t , t ∈ Z). In his thesis, Tavakoli improved this construction by using the appropriate space Span (PX t : P ∈ L b (H 0 ), t ∈ Z), see Tavakoli (2014, Section 2.6), his approach assumes that the spectral density operator is dominated by Lebesgue's measure and that its density is in L p (T, B(T), S 1 (H 0 ), Leb) for p ∈ (1, +∞].

Functional orthogonal increment process vs c.a.g.o.s. measures

Moreover, in recent approaches, X in (ii.2) is defined as a functional orthogonal increment process and the integral is referred to as a Riemann-Stieltjes integral with respect to X. This notion, used for the Cramér representations exhibited in Delft andEichler, 2018, 2020;Panaretos and Tavakoli, 2013a,b;[START_REF] Tavakoli | Fourier Analysis of Functional Time Series, with Applications to DNA Dynamics[END_REF], and which follows the construction of Rosenblatt, 1985 for univariate weakly stationary time series, is based on the following definition where we prefer to use the term Gramian-orthogonal increment process to emphasize the use of the Gramian structure of M(Ω, F , H 0 , P). Definition 5.4.1 (Gramian-orthogonal increment processes). Let H 0 be a separable Hilbert space. A random process (Z λ ) λ∈ [-π,π] valued in H 0 is said to be a Gramian-orthogonal increment process if the three following assertions hold.

(i) We have Z -π = 0 a.s. and, for all λ ∈ (-π, π], Z λ ∈ M(Ω, F , H 0 , P) (as defined in Example 4.3.2).

(ii) For all λ 1 , λ 2 , λ 3 , λ 4 ∈ [-π, π], with λ 2 ≥ λ 1 and λ 4 ≥ λ 3 , we have

(λ 1 , λ 2 ] ∩ (λ 3 , λ 4 ] = ∅ ⇒ Cov (Z λ 4 -Z λ 3 , Z λ 2 -Z λ 2 ) = 0 . (iii) For all λ ∈ [-π, π], lim ↓0 E Z λ+ -Z λ 2 H 0 = 0.
Of course, Definition 5.4.1 can be related to random c.a.g.o.s. measures as in Definition 5.1.2 with Λ = (-π, π] and A = B((-π, π]). Indeed, it is straightforward to show that, if W is an H 0 -valued random c.a.g.o.s. measure on ((-π, π], B((-π, π]), Ω, F , P), then setting

Z -π = 0 and Z λ = W((-π, λ]) , λ ∈ (-π, π] ,
(5.4.1)

we get a Gramian-orthogonal increment process. The reciprocal result is stated as the following proposition. Proof. By (ii) in Definition 5.4.1, we have that, for all s < t in [-π, π],

E Z t -Z -π 2 H 0 = E Z s -Z -π 2 H 0 + E Z t -Z s 2 H 0 .
Thus, with (iii), we have that the function F : [-π, π] → R + defined by

F(λ) = E Z λ -Z -π 2 H 0
is non-decreasing and right-continuous, and it follows that there exists a finite non-negative measure ν on ((-π, π], B((-π, π])) such that, for all

s < t in [-π, π], E Z t -Z s 2 H 0 = ν((s, t]
) . Another straightforward consequence of (ii) in Definition 5.4.1 is that, for all s < t and s < t in (-π, π], we have

E Z t -Z s , Z t -Z s H 0 =      E Z t ∧t -Z s ∨s 2 H 0 if s ∨ s < t ∧ t, 0 otherwise.
Thus we can consider the mapping 1 (s,t] → Z(t) -Z(s) defined for all s < t in (-π, π] as a G := L 2 ((-π, π], B((-π, π]), ν) → H := M(Ω, F , H 0 , P) mapping, and, interpreting the right-hand side of the previous display as 1 (s,t] , 1 (s ,t ] G , we see that this mapping is isometric. Let us denote by I the unique isometric extension of this mapping on the linear closure of 1 (s,t] : s < t ∈ (-π, π] in G, which happens to be G itself. We then set, for all A ∈ B((-π, π]),

W(A) = I(1 A ) ,
and we immediately obtain that W is an H 0 -valued random c.a.o.s. measure on ((-π, π], B((-π, π]), Ω, F , P) as in Definition 5.1.1 and its intensity measure is ν. By uniqueness of the isometric extension, it only remains to show that W is moreover a c.a.g.o.s. measure, that is, for all A, B ∈ B((-π, π])

such that A ∩ B = ∅, we have [W(A), W(B)] H = Cov (W(A), W(B)) = 0 .
This is implied by showing that, for all x ∈ H 0 such that

x H 0 = 1, for all A, B ∈ B((-π, π]) such that A ∩ B = ∅, we have x H Cov (W(A), W(B)) x = Cov W(A), x H 0 , W(B), x H 0 = 0 . (5.4.2)
Now take x ∈ H 0 such that x H 0 = 1 and define

F x : [-π, π] → R + by F x (λ) = E Z λ -Z -π , x H 0 2 .
As previously with F, (ii) and (iii) in Definition 5.4.1 imply that F x is nondecreasing and right-continuous and it follows that there exists a finite nonnegative measure ν x on ((-π, π], B((-π, π])) such that, for all s < t in [-π, π],

E Z t -Z s , x H 0 2 = ν x ((s, t]) .
Again, we can extend the mapping

1 (s,t] → Z t -Z s , x H 0 defined for all s < t in (-π, π] as a G x := L 2 ((-π, π], B((-π, π]), ν x ) → M(Ω, F , C, P) isomet-
ric mapping, which we denote by I x in the following. We further denote by W x the c.a.o.s. measure defined by W x (A) = I x (1 A ) for all A ∈ B((-π, π]). This is a C-valued random c.a.o.s. measure on ((-π, π], B((-π, π]), Ω, F , P) with intensity measure ν x . Hence to obtain (5.4.2) and conclude the proof, we only need to show that for all A ∈ B((-π, π]), we have (5.4.3) We already know that this is true for A ∈ C = {(-π, λ] : λ ∈ (-π, π]} by definitions of W x , W, I x and I. The class C is a π-system of Borel sets and satisfies σ(C) = B((-π, π]). We conclude with the π-λ-theorem by observing that the class A of sets A ∈ B((-π, π]), such that (5.4.3) holds is a λ-system.

W x (A) = W(A), x H 0 .
Indeed if A ∈ A, then A c = (-π, π] \ A satisfies W x (A c ) = W x ((-π, π]) -W x (A) and W(A c ) = W((-π, π]) -W(A), so that A, (-π, π] ∈ A implies A c ∈ A. Similarly, if (A n ) n∈N ∈ A N with A n ∩ A p = ∅ for n = p then ∪ n A n ∈ A because
A P P L I C AT I O N S T O L I N E A R F I LT E R I N G A N D L O N G -M E M O R Y P R O C E S S E S
In this chapter, we illustrate the advantages of the spectral theory developed in Chapter 5 with three applications. First, we give a formal definition of linear filtering and derive interesting results on the composition and inversion of linear filters, thus establishing Step 5) of the introduction. Second, we address Step 6) by providing a version the Cramér-Karhunen-Loève decomposition with minimal assumptions. Finally, we introduce a novel class of long-memory processes with long memory operator D acting on a Hilbert space and compare our approach to processes defined in the time domain that were previously introduced for modeling long range dependence in the context of functional time series. This corresponds to Step 7) of the introduction.

Composition and inversion of linear filters

With the construction of the spectral theory for weakly-stationary processes of Section 5.2, the study of linear filters for such processes is easily derived. Indeed, we are now able to give the most general definition of linear filtering, characterize the spectral structure of the filtered process and provide results on compositions and inversion of linear filters. We start in the spectral domain by with filtering of random c.a.g.o.s. measures and then translate the results to the case where the random c.a.g.o.s. measures are Gramian-Cramér representations of weakly stationary processes.

Linear filtering of random c.a.g.o.s. measures

The definition of filtering of a c.a.g.o.s. measure based the following straightforward result.

Proposition 6.1.1. Let (Λ, A) be a measurable space, H 0 , G 0 two separable Hilbert spaces. Let W be an H 0 -valued random c.a.g.o.s. measure on (Λ, A, Ω, F , P) with intensity operator measure ν W . Let Φ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν W ). Then the mapping

V : A → A Φ dW = I G 0 W (1 A Φ) (6.1.1)
is a G 0 -valued random c.a.g.o.s. measure on (Λ, A, Ω, F , P) with intensity operator measure

Φν W Φ H : A → A Φdν W Φ H ,
which is a well defined trace-class p.o.v.m.. In particular, we have that, for all σfinite measure µ on (Λ, A) dominating ν W 1 , then µ dominates Φν W Φ H 1 and

dν W dµ = Φ dν W dµ 1/2 Φ dν W dµ 1/2 H . (6.1.2)
The c.a.g.o.s. V defined by (6.1.1) is said to admit the density Φ with respect to W, and we write dV = ΦdW (or, equivalently, V(dλ) = Φ(λ)W(dλ)).

In the following definition, we use a signal processing terminology where Λ is seen as a set o frequencies and Φ is seen as a transfer operator function acting on the (random) input frequency distribution W. Definition 6.1.1 (Filter FΦ (W) acting on a random c.a.g.o.s. measure in ŜΦ ). Let (Λ, A) be a measurable space, H 0 , G 0 two separable Hilbert spaces. For a given transfer operator function Φ ∈ F O (Λ, A, H 0 , G 0 ), we denote by ŜΦ (Ω, F , P) the set of H 0 -valued random c.a.g.o.s. measures on (Λ, A, Ω, F , P) whose intensity operator measures ν W satisfy Φ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν W ). Then, for any W ∈ ŜΦ (Ω, F , P), we say that the random G 0 -valued c.a.g.o.s. measure V defined by (6.1.1) is the output of the filter with transfer operator function Φ applied to the input c.a.g.o.s. measure W, and we denote V = FΦ (W).

The goal of this section is, given another separable Hilbert space I 0 , to characterize the transfer functions Ψ valued in O(G 0 , I 0 ) which can be used to filter the c.a.g.o.s. measure V. According to Proposition 6.1.1, Ψ must be square-integrable with respect to ν V = Φν W Φ H and this turns out to be equivalent to checking that ΨΦ is square integrable with respect to ν W as stated in the following theorem. We recall that ΨΦ is the pointwise composition, that is, ΨΦ : λ → Ψ(λ) • Φ(λ) and is defined whenever the image of Φ(λ) is included in the domain of Ψ(λ).

We can now state the main result of this section whose proof can be found in Section 6.5.1 Theorem 6.1.2. Let (Λ, A) be a measurable space, H 0 , G 0 , I 0 separable Hilbert spaces and ν a trace-class p.o.v.m. on (Λ, A, H 0 ).

Let Φ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν) and Ψ ∈ F O (Λ, A, G 0 , I 0 ). Define ΦνΦ H : A → A ΦdνΦ H = [1 A Φ, 1 A Φ] ν , which is a trace-class p.o.v.m. on (Λ, A, G 0 ). Then Ψ ∈ L 2 (Λ, A, O(G 0 , I 0 ), ΦνΦ H ) ⇔ ΨΦ ∈ L 2 (Λ, A, O(H 0 , I 0 ), ν) . (6.1.3)
Moreover, the following assertions hold.

(a) For all Ψ, Θ ∈ L 2 (Λ, A, O(G 0 , I 0 ), ΦνΦ H ), (ΨΦ)ν(ΘΦ) H = Ψ(ΦνΦ H )Θ H . (b) The mapping Ψ → ΨΦ is a well defined Gramian-isometric operator from L 2 (Λ, A, O(G 0 , I 0 ), ΦνΦ H ) to L 2 (Λ, A, O(H 0 , I 0 ), ν).
(c) Suppose moreover that Φ is injective ν 1 -a.e., then we have that

Φ -1 ∈ L 2 (Λ, A, O(G 0 , H 0 ), ΦνΦ H ) ,
where we define

Φ -1 (λ) := Φ(λ) |D(Φ(λ))→Im(Φ(λ)) -1 with domain Im(Φ(λ))
for all λ ∈ {Φ is injective} and Φ -1 (λ) = 0 otherwise.

As aconsequence of this theorem, we get the following results on the composition and inversion for random c.a.g.o.s. measures which are proved in Section 6.5.1. Corollary 6.1.3 (Composition and inversion of filters on random c.a.g.o.s. measures). Let (Λ, A) be a measurable space, H 0 , G 0 two separable Hilbert spaces, and Φ ∈ F O (Λ, A, H 0 , G 0 ). Let W ∈ ŜΦ (Ω, F , P) with intensity operator measure ν W . Then three following assertions hold.

Recall that the definitions of ⊆ ∼ and ∼ = are given in Definition 4.3. 5(i) For any separable Hilbert space I 0 , we have H FΦ (W),I 0 ⊆ ∼ H W,I 0 .

(ii) For any separable Hilbert space I 0 and all Ψ ∈ F O (Λ, A, G 0 , I 0 ), we have W ∈ ŜΨΦ (Ω, F , P) if and only if FΦ (W) ∈ ŜΨ (Ω, F , P), and in this case, we have

FΨ • FΦ (W) = FΨΦ (W). (6.1.4) (iii) Suppose that Φ is injective ν W 1 -a.e. Then W = FΦ -1 • FΦ (W)
, where Φ -1 is defined as in Assertion (c) of Theorem 6.1.2. Moreover, Assertion (i) above holds with ⊆ ∼ replaced by ∼ = .

The case of weakly stationary stochastic processes

The results of the previous section translate to H 0 -valued weakly-stationary stochastic processes as follows. Let H 0 and G 0 be two separable Hilbert spaces and Φ ∈ F O (Λ, A, H 0 , G 0 ). Then, if X = (X t ) t∈G is an H 0 -valued weakly stationary stochastic process with Gramian-Cramér representation X such that X ∈ ŜΦ (Ω, F , P) or, equivalently, Φ ∈ H X,G 0 , (6.1.5)

then we can define the random c.a.g.o.s. measure Ŷ = FΦ ( X) and Lemma 5.2.3 gives that the process Y = (Y t ) t∈G defined by

Y t = χ(t) Ŷ(dχ) , t ∈ G ,
is a G 0 -valued weakly stationary stochastic process. Moreover, applying Assertion (ii) of Corollary 6.1.

3 with Ψ = χ(t)Id G 0 yields Y t = χ(t) Φ(χ) X(dχ) , for all t ∈ G .
In this case, for convenience we write, in the time domain, X ∈ S Φ (Ω, F , P) and Y = F Φ (X) , (6.1.6)

for the assertions X ∈ ŜΦ (Ω, F , P) and Ŷ = FΦ ( X).

In this framework, using the Gramian-unitary operator between the modular time domain and the modular spectral domain, the following result holds as a direct application of Corollary 6.1.3 with Λ = Ĝ and A = B( Ĝ) and W = X. Corollary 6.1.4 (Composition and inversion of filters on weakly stationary time series). Let H 0 and G 0 be two separable Hilbert spaces and pick a transfer operator function Φ ∈ F O Ĝ, B( Ĝ), H 0 , G 0 . Let X be a centered weakly stationary H 0 -valued process defined on (Ω, F , P) with spectral operator measure ν X . Suppose that X ∈ S Φ (Ω, F , P) and set Y = F Φ (X), as defined in (6.1.6). Then the three following assertions hold.

(i) For any separable Hilbert space I 0 , we have H Y,I 0 ⊆ ∼ H X,I 0 .

(ii) For any separable Hilbert space I 0 and all Ψ ∈ F O Ĝ, B( Ĝ), G 0 , I 0 , we have X ∈ S ΨΦ (Ω, F , P) if and only if F Φ (X) ∈ S Ψ (Ω, F , P), and in this case, we have

F Ψ • F Φ (X) = F ΨΦ (X). (6.1.7) (iii) Suppose that Φ is injective ν X 1 -a.e. Then X = F Φ -1 • F Φ (X), where we define Φ -1 (λ) := Φ(λ) |D(Φ(λ))→Im(Φ(λ)) -1
with domain Im(Φ(λ)) for all λ ∈ {Φ is injective} and Φ -1 (λ) = 0 otherwise. Moreover, Assertion (i) above holds with ⊆ ∼ replaced by ∼ = .

To conclude this section, we note than many examples in the literature rely on a time-domain description of the filtering obtained as in the following interesting example.

Example 6.1.1 (Convolutional filtering). Let H 0 and G 0 be two separable Hilbert spaces. Let X = (X t ) t∈G be an H 0 -valued weakly stationary stochastic process defined on (Ω, F , P). Let η be the Haar measure on G (see Rudin (1990, Chapter 1))

and Φ ∈ L 1 (G, B(G), L b (H 0 , G 0 ), η). Define the process Y = (Y t ) t∈G by the time domain convolutional filtering Y t = Φ(s) X t-s η(ds) , t ∈ G ,
where the integral is a Bochner integral on L 1 (G, B(G), M(Ω, F , G 0 , P), η). Then, using Proposition 5.1.8 and defining

Φ : Ĝ → L b (H 0 , G 0 ) by the following Bochner integral on L 1 (G, B(G), L b (H 0 , G 0 ), η), Φ(χ) = Φ(s) χ(s) η(ds) , it is straightforward to show that Φ ∈ L 2 ( Ĝ, B( Ĝ), O(H 0 , G 0 ), ν X ) and Y = F Φ(X).

Cramér-Karhunen-Loève decomposition

Let H 0 be a separable Hilbert space with (possibly infinite) dimension N and X = (X t ) t∈G be a centered, H 0 -valued weakly-stationary process defined on a probability space (Ω, F , P) with Gramian-Cramér representation X and spectral operator measure ν X . The Cramér-Karhunen-Loève decomposition

The term "Cramér-Karhunen-Loève" was coined in Panaretos and Tavakoli, 2013a. the amounts to give a rigorous meaning to the formula

X(dχ) = ∑ 0≤n<N φ n (χ) ⊗ φ n (χ) X(dχ) , (6.2.1)
where, for all χ ∈ Ĝ, (φ n (χ)) 0≤n<N is an orthonormal sequence in H 0 chosen in such a way that the summands in (6.2.1) are uncorrelated. Such a decomposition provides a way to derive the harmonic principal component analysis of the process X, which is an approximation of X by a finite rank linear filtering. In recent works, the functional Cramér-Karhunen-Loève decomposition is achieved under additional assumptions on ν X such as having a continuous density with respect to the Lebesgue measure (in [START_REF] Tavakoli | Fourier Analysis of Functional Time Series, with Applications to DNA Dynamics[END_REF] or at most finitely many atoms (in Delft and Eichler, 2020). In fact, thanks to the Radon-Nikodym property of trace-class p.o.v.m.'s of Theorem 4.5.2 and the measurable eigendecomposition of Theorem 4.5.9, there is no need for such additional assumptions. Instead, we rely on Proposition 4.5.

(λ) = ∑ 0≤n<N σ n (λ) φ n (λ) ⊗ φ n (λ) , λ ∈ Λ
be the measurable eigendecomposition of f = dν dµ as proved in Proposition 4.5.5. Then, using the notations φ H

n : λ → φ n (λ) H and φ n ⊗ φ n : λ → φ n (λ) ⊗ φ n (λ), we have the following properties. (i) The sequence (φ H n ) 0≤n<N is orthogonal in L 2 (Λ, A, O(H 0 , C), ν). (ii) The sequence (φ n ⊗ φ n ) 0≤n<N is Gramian-orthogonal in L 2 (Λ, A, O(H 0 ), ν). (iii) The L b (H 0 )-valued mapping λ → ∑ 0≤n<N φ n (λ) ⊗ φ n (λ) belongs in L 2 (Λ, A, O(H 0 ), ν)
and is equal to the mapping λ → Id H 0 in L 2 (Λ, A, O(H 0 ), ν).

(iv) If N = +∞, we have

lim p→+∞ p ∑ n=0 φ n ⊗ φ n -Id H 0 ν = 0 . Proof. First note that the O-measurability of φ H n , φ n ⊗ φ n and ∑ 0≤n<N φ n ⊗ φ n are direct consequences of the measurability of the φ n 's. Then, since f 1/2 = ∑ 0≤n<N σ 1/2
n φ n ⊗ φ n we get the following results. Firstly, for all 0 ≤ n < N and all λ ∈ Λ, φ H n f 1/2 (λ)

2 2 = σ n (λ) ≤ f (λ) 1 . Hence φ H n f 1/2 ∈ L 2 (Λ, A, S 2 (H 0 , C), ν) and Proposition 5.1.4 gives that φ H n ∈ L 2 (Λ, A, O(H 0 , C), ν) and for all 0 ≤ n, p < N, φ H n , φ H p ν = φ H n f φ p dµ =      0 if n = p,
σ n dµ otherwise. , which proves Assertion (i).

Similarly, for all 0 ≤ n < N, φ n ⊗ φ n f 1/2 ∈ L 2 (Λ, A, S 2 (H 0 ), ν), hence by Proposition 5.1.4, we have φ n ⊗ φ n ∈ L 2 (Λ, A, O(H 0 ), ν) and for all 0 ≤ n, p < N,

φ n ⊗ φ n , φ p ⊗ φ p ν = (φ n ⊗ φ n ) f (φ p ⊗ φ p ) dµ =      0 if n = p, σ n (φ n ⊗ φ n ) dµ otherwise, which proves Assertion (ii). Moreover, for all λ ∈ Λ, ∑ 0≤n<N φ n (λ) ⊗ φ n (λ) f (λ) 1/2 = f (λ)
1/2 and therefore Assertion (iii) holds by Proposition 5.1.4.

Finally, by completeness of L 2 (Λ, A, O(H 0 ), ν), Assertion (iv) reduces showing that ∑ q n=p φ n ⊗ φ n ν converges to 0 as p, q → +∞. This holds because, for all q ≥ p ≥ 0,

q ∑ n=p φ n ⊗ φ n 2 ν = q ∑ n=p φ n ⊗ φ n f 1/2 2 2 d ν 1 = q ∑ n=p σ 1/2 n φ n ⊗ φ n 2 2 d ν 1 = q ∑ n=p σ n d ν 1 ,
where, for all λ ∈ Λ, ∑ q n=p σ n (λ) ≤ f (λ) 1 and ∑ q n=p σ n (λ) converges to 0 as p, q → 0.

We have the following remark about Assertion (iii). Remark 6.2.1. By Proposition 4.5.5, for all λ ∈ Λ, ∑ 0≤n<N φ n (λ) ⊗ φ n (λ) is the orthogonal projection onto the closure of the range of f (λ). Thus, Assertion (iii) of Lemma 6.2.1 says that this projection is equal to

Id H 0 in L 2 (Λ, A, O(H 0 ), ν). It is not equivalent to saying that ∑ 0≤n<N φ n ⊗ φ n = Id H 0 , ν 1 -a.
e. since it may happen that the range of f (λ) is dense in H 0 for none of the λ's, in which case we have Assertion (iii) of Lemma 6.2.1 at the same time as

{∑ 0≤n<N φ n ⊗ φ n = Id H 0 } = ∅.
Then the Cramér-Karhunen-Loève decomposition is derived as a consequence of Proposition 4.5.5 and Lemma 6.2.1. Corollary 6.2.2 (Cramér-Karhunen-Loève decomposition). Let H 0 be a separable Hilbert space with (possibly infinite) dimension N and X = (X t ) t∈G be a centered, H 0 -valued weakly-stationary process defined on a probability space (Ω, F , P) with Gramian-Cramér representation X and spectral operator measure ν X with

dν X d ν X 1 = ∑ 0≤n<N σ n φ n ⊗ φ n
as in Theorem 4.5.9. Then we have Proof. We treat the case N = +∞ because the other case is simpler. The first equality in (6.2.2) is a consequence of (iii) of Lemma 6.2.1 and the last one comes from Corollary 6.1.3. The only tricky part is to show that we can invert the sum and the filtering operation in the second equality, i.e. that for all A ∈ B( Ĝ),

X = F(∑ 0≤n<N φ n ⊗φ n ) ( X) = ∑ 0≤n<N Fφ n ⊗φ n ( X) = ∑ 0≤n<N Fφ n • Fφ H n ( X)
A +∞ ∑ n=0 φ n ⊗ φ n d X = +∞ ∑ n=0 A φ n ⊗ φ n d X ,
which holds by Assertion (iv) of Lemma 6.2.1 and the fact that the stochastic integral with respect to X is continuous on

L 2 ( Ĝ, B( Ĝ), O(H 0 ), ν).
Note that the equality should be understood as holding when the c.a.g.o.s. measures are evaluated at any arbitrary A ∈ B( Ĝ). The following remark provides other formulations of Relation (6.2.2) Remark 6.2.2. Depending on the setting, Relation (6.2.2) can have the following other formulations.

1. Using the density notation, we may write Relation (6.2.2) as

X(dχ) = ∑ 0≤n<N φ n (χ) ⊗ φ n (χ) X(dχ) = ∑ 0≤n<N X(dχ), φ n (χ) H 0 φ n (χ) ,
which emphasizes the fact that the Cramér-Karhunen-Loève decomposition is a Karhunen-Loève decomposition in the spectral domain. However, this notation is not rigorous and may lead to confusions.

2.

In the time domain, Relation (6.2.2) writes as

X = F (∑ 0≤n<N φ n ⊗φ n ) (X) = ∑ 0≤n<N F φ n ⊗φ n (X) = ∑ 0≤n<N F φ n • F φ H n ( X) ,
where (F φ n ⊗φ n ( X)) 0≤n<N are uncorrelated H 0 -valued weakly stationary stochastic processes and (F φ H n (X)) 0≤n<N are uncorrelated C-valued weakly stationary stochastic processes. Here the equality should be understood as holding when the time series are evaluated at any arbitrary t ∈ G.

3. Letting e t : χ → χ(t), we get the following integral formulation, for all t ∈ G,

X t = e t ∑ 0≤n<N φ n ⊗ φ n d X = ∑ 0≤n<N e t (φ n ⊗ φ n ) d X = ∑ 0≤n<N e t φ n d Fφ H n ( X) ,
which corresponds to the formulation of Tavakoli, 2014, Theorem 2.8.6.

As a consequence, we get the following general formulation of a harmonic principal components analysis for H 0 -valued weakly-stationary processes. Proposition 6.2.3 (Harmonic functional principal components analysis). Let H 0 be a separable Hilbert space and X = (X t ) t∈G be a centered, H 0 -valued weaklystationary process defined on a probability space (Ω, F , P) with spectral operator measure ν X . Let (σ n ) 0≤n<N and (φ n ) 0≤n<N be given as in Proposition 4.5.5 for some dominating measure µ of ν X , for instance µ = ν X 1 . Let q : Ĝ → N * be a measurable function. Then for all t ∈ G,

min E X t -[F Θ (X)] t 2 : Θ ∈ L 2 ( Ĝ, B( Ĝ), O(H 0 ), ν X ), rank(Θ) ≤ q is equal to Ĝ ∑ q(χ)∧N≤n<N σ n (χ) µ(dχ) ,
and the minimum is achieved for

Θ : χ → ∑ 0≤n<q(χ)∧N φ n (χ) ⊗ φ n (χ) . Proof. Let f X (χ) = ∑ 0≤n<N σ n (χ) φ n (χ) ⊗ φ n (χ)
denotes the density of ν X with respect to µ as given by Proposition 4.5.5. We have, for all t ∈ G and

Θ ∈ L 2 ( Ĝ, B( Ĝ), O(H 0 ), ν X ), [F Θ (X)] t = χ(t) Θ(χ) X(dχ) ,
and thus by isometric isomorphism between the spectral domain and the time domain,

E X t -[F Θ (X)] t 2 = (Id H 0 -Θ(χ)) f 1/2 X (χ) 2 S 2 (H 0 ) µ(dχ) .
The result is then obtained by observing that, for each χ ∈ Ĝ, the norm in the integral is minimal under the constraint rank(Θ(χ)) ≤ q(χ) for Θ(χ) = ∑ 0≤n<q(χ)∧N φ n (χ) ⊗ φ n (χ).

Some details about functional random variables

The construction of processes with long memory operators, which will be discussed in Section 6.4, relies on the singular value decomposition of a normal operator. This places us in the framework where the separable Hilbert space is H 0 = L 2 (V, V, ξ) for some measure space (V, V, ξ) and therefore, we need to introduce an appropriate formalism. This is the purpose of this section. In Section 6.3.1, we introduce notions relative to Hilbert-Schmidt integral operators. In Section 6.3.2, we show that an H 0 -valued random variable Y can be seen as the realization of some continuous time process {Y(v) : v ∈ V} in the sense that there always exists a version of Y which is jointly measurable in V × Ω as will be stated in Proposition 6.3.3. This last point allows us to define a cross-spectral density function containing the spectral information of the functional time series evaluated at two points v, v ∈ V.

In this section, we consider H 0 := L 2 (V, V, ξ) where (V, V, ξ) is a measure space. We assume that ξ is σ-finite and that the space H 0 is separable. We will denote by (φ n ) n∈N an arbitrary Hilbert basis of H 0 .

Hilbert-Schmidt integral operators

For ∈ L 2 (V 2 , V ⊗2 , ξ ⊗2 ), we define the integral operator with kernel as the unique operator K on H 0 satisfying

K f : v → V (v, v ) f (v ) ξ(dv ) , for all f ∈ H 0 . In this case, K ∈ S 2 (H 0 ) with K 2 2 = V 2 | | 2 dξ ⊗2
and, through this re- lation, the spaces L 2 (V 2 , V ⊗2 , ξ ⊗2 ) and S 2 (H 0 ) are isometrically isomorphic. In particular, any Hilbert-Schmidt operator on H 0 is an integral operator and is associated to a unique kernel in L 2 (V 2 , V ⊗2 , ξ ⊗2 ). It is easy to check that the kernel associated to K H is the adjoint kernel

(v, v ) → (v , v) and that ∈ L 2 (V 2 , V ⊗2 , ξ ⊗2
) is the kernel of a Hilbert-Schmidt operator K if and only if dv) , for all i, j ∈ N .

φ H i Kφ j = (v, v ) φ i (v)φ j (v ) ξ(dv)ξ(
A special case of interest is when we consider an operator G ∈ S + 1 (H 0 ). In this case, G is also a Hilbert-Schmidt operator and therefore is also associated to a kernel, say ∈ L 2 (V 2 , V ⊗2 , ξ ⊗2 ). However, because we can write G = HH H for some well (non-uniquely) chosen H ∈ S 2 (H 0 ), we can be more precise in describing the kernel, as stated in the following lemma, in which, for instance, one can choose

H = H H = G 1/2 . Lemma 6.3.1. Let H 0 = L 2 (V, V, ξ) be a separable Hilbert space, G ∈ S + 1 (H 0 ) and H ∈ S 2 (H 0 ) such that G = HH H . Let , ℋ ∈ L 2 (V 2 , V ⊗2 , ξ ⊗2 ) be the kernels of G and H respectively. Then for ξ ⊗2 -a.e. (v, v ) ∈ V 2 , (v, v ) = ℋ (v, v ) ℋ (v , v ) ξ(dv ) . (6.3.1)
Let us now consider an S 2 (H 0 )-valued function K defined on a measurable space (Λ, A). As explained previously, for any λ ∈ Λ, K(λ) can be seen as an integral operator associated to a kernel (•; λ) ∈ L 2 (V 2 , V ⊗2 , ξ ⊗2 ). However it is useful to consider the mapping (v, v , λ) → (v, v ; λ) and to make this mapping measurable on (V 2 × Λ, V ⊗2 ⊗ A). For convenience, we introduce the following definition to refer to such a measurable mapping. Definition 6.3.1 (Joint kernel function). Let H 0 = L 2 (V, V, ξ) be a separable Hilbert space, with (V, V, ξ) a σ-finite measured space. Let K be a measurable function from (Λ, A) to (S 2 (H 0 ), B(S 2 (H 0 ))) and

: (v, v , λ) → (v, v ; λ) be measurable from (V 2 × Λ, V ⊗2 ⊗ A) to (C, B(C)) such that, for all λ ∈ Λ and f ∈ H 0 , K(λ) f : v → (v, v ; λ) f (v ) ξ(dv ) . (6.3.2)
Then we call the Λ-joint kernel function of K.

In Definition 6.3.1, the Λ-joint kernel function of K is unique in the sense two Λ-joint kernel functions and ˜ of the same S 2 (H 0 )-valued function K must satisfy that, for all λ ∈ Λ, (•; λ) = ˜ (•; λ), ξ ⊗2a.e. The following lemma asserts that a Λ-joint kernel function of K always exists and provides additional properties in two special cases that will be of interest. Proposition 6.3.2. Let H 0 and K be as in Definition 6.3.1. Then K admits a Λ-joint kernel function . Moreover the two following assertions hold for any non-negative measure µ on (Λ, A).

(i) If K ∈ L 2 (Λ, A, S 2 (H 0 ), µ), then ∈ L 2 (V 2 × Λ, V ⊗2 ⊗ A, ξ ⊗2 ⊗ µ). (ii) If K ∈ L 1 (Λ, A, S + 1 (H 0 ), µ), then satisfies (v, v ; λ) 2 ξ(dv)ξ(dv ) 1/2 µ(dλ) < +∞ . (6.3.3) 6.3.2 L 2 (V, V, ξ)-

valued weakly stationary time series

We first show that we can always find a version of an H 0 -valued random variable which is jointly measurable on V × Ω.

Proposition 6.3.3. Let Y be an H 0 -valued random variable defined on (Ω, F , P).

Then Y admits a version (v, ω) → Ỹ(v, ω) jointly measurable on (V × Ω, V ⊗ F ).

Hence, in the following an H 0 -valued random variable Y will always be assumed to be represented by a V × Ω → C-measurable function Ỹ. If, moreover, Y ∈ L 2 (Ω, F , H 0 , P), then, by Fubini's theorem, we can see Ỹ as an element of L 2 (V × Ω, V ⊗ F , ξ ⊗ P), and we can write

Ỹ(v, ω) = ∑ k∈N Y(ω), φ k φ k (v) ,
where the convergence holds in L 2 (V × Ω, V ⊗ F , ξ ⊗ P). As expected, in this case, the covariance operator Cov(Y) is an integral operator with kernel (v, v ) → Cov Ỹ(v, •), Ỹ(v , •) . It is then tempting to write that Var Ỹ(v, •) is equal to the kernel of the integral operator Cov(Y) on the diagonal set v = v : (v, v ) ∈ V 2 . However, because this set has null ξ ⊗2 -measure, this "equality" is meaningless in the framework of Hilbert-Schmidt operators. In the following lemma we make this statement rigorous by relying on a decomposition of the form Cov(Y) = KK H for some K ∈ S 2 (H 0 ). Lemma 6.3.4. Let Y be a random variable valued in a separable Hilbert space (6.3.4) where

H 0 = L 2 (V, V, ξ), with ξ a σ-finite measure on (V, V ). Let K ∈ S 2 (H 0 ) and denote by its kernel in L 2 (V 2 , V ⊗2 , ξ ⊗2 ). Suppose that Cov(Y) = KK H . Then, we have, for ξ -a.e. v ∈ V, E Ỹ(v, •) 2 = (v, v ) 2 ξ(dv ) = (v, •) 2 H 0 ,
Ỹ is a version of Y in L 2 (V × Ω, V ⊗ F , ξ ⊗ P).
Let now X = (X t ) t∈Z be an H 0 -valued weakly stationary time series defined on (Ω, F , P) with spectral operator measure ν X and, for each t ∈ Z, denote by Xt a version of X t in L 2 (V × Ω, V ⊗ F , ξ ⊗ P). Note that, for all n ∈ N, (∑ n k=1 X t , φ k φ k (v)) t∈Z are C-valued sequences which are (v ∈ V)jointly weakly stationary. Hence, from what precedes, we get that there exists a ξ-full measure set V 0 ∈ V such that ( Xt (v, •)) t∈Z are C-valued sequences which are (v ∈ V 0 )-jointly weakly stationary. The next proposition shows that these time series admit spectral densities with respect to any non-negative measure that dominates the spectral measure of X. Proposition 6.3.5. Let H 0 = L 2 (V, V, ξ) be a separable Hilbert space, with ξ a σfinite measure on (V, V ). Let X = (X t ) t∈Z be an H 0 -valued weakly stationary time series defined on (Ω, F , P) with spectral operator measure ν X . Suppose that µ is a finite non-negative measure on (T, B(T)) that dominates ν X . Let g X = dν X dµ and ℊ X : (v, v , λ) → ℊ X (v, v ; λ) be its T-joint kernel function as in Definition 6.3.1. Then for ξ ⊗2a.e. (v, v ) ∈ V 2 , the cross spectral measure of the time series ( Xt (v, •)) t∈Z and ( Xt (v , •)) t∈Z admits the density λ → ℊ X (v, v ; λ) with respect to µ. Proposition 6.3.5 leads to the following. Definition 6.3.2 (Cross-spectral density function). Under the assumptions of Proposition 6.3.5, we call ℊ X the cross-spectral density function and with respect to µ.

Hilbert valued FIARMA processes

Although, the study of weakly-stationary time series valued in a separable Hilbert space has been an active field of research in the past decades, the literature mainly focuses on short-memory processes and the study of longmemory processes valued in a separable Hilbert space is a more recent topic, see [START_REF] Characiejus | The central limit theorem for a sequence of random processes with space-varying long memory[END_REF]Račkauskas, 2013, 2014;[START_REF]Limit theorems for Hilbert space-valued linear processes under long range dependence[END_REF]Li, Robinson, and Shang, 2020;Račkauskas andSuquet, 2011. In particular, in Li, Robinson, andShang (2020, Section 4), the authors propose a generalization of the fractionally integrated autoregressive moving average (often shortened as ARFIMA but we prefer to use the abbreviation FIARMA for reasons that will be made explicit in Remark 6.4.1) processes to the case of curve (or functional) time series. In short, they consider the functional case in which the Hilbert space is an L 2 space of real valued functions defined on some compact subset C of R, and they introduce the time series (X t ) t∈Z valued in this Hilbert space defined by (6.4.1) where -1/2 < d < 1/2, B is the backshift operator on R Z , and Y t is a functional ARMA process. As pointed out in Li, Robinson, and Shang (2020, Remark 9), taking the same d for all v ∈ C in (6.4.1) is very restrictive compared to other long memory processes recently introduced, for instance in [START_REF] Characiejus | The central limit theorem for a sequence of random processes with space-varying long memory[END_REF]Račkauskas, 2013, 2014, where they consider long-memory processes of the form

(1 -B) d X t (v) = Y t (v) , t ∈ Z, v ∈ C ,
X t (v) = ∞ ∑ k=0 (1 + k) -n(v) t-k (v) , t ∈ Z , v ∈ V ,
where (V, V, ξ) is a σ-finite measured space and ( t ) t∈Z is a white noise valued in L 2 (V, V, ξ). A formulation not restricted to an L 2 space was proposed in D üker, 2018 where the author considers long-memory processes of the form

X t = ∞ ∑ k=0 (1 + k) -N t-k , t ∈ Z , (6.4.2)
where ( t ) t∈Z is a white noise valued in a separable Hilbert space H 0 and N is a bounded normal operator on H 0 . This suggests to define FIARMA processes in (6.4.1) with d replaced by a a function d(v), or in the case where it is valued in an arbitrary separable Hilbert space H 0 , by a bounded normal operator D acting on this space.

In this section, we fill this gap by providing a definition of FIARMA processes valued in a separable Hilbert space H 0 with a long memory operator D, taken as a bounded linear operator on H 0 . If D is normal, then we can rely on its singular value decomposition and find necessary and sufficient conditions to ensure that the FIARMA process with long memory operator D is well defined. This allows us to compare FIARMA processes with the processes defined by (6.4.2) as in D üker, 2018. First we recall known results on the existence of ARMA processes.

ARMA processes

Let p be a positive integer and consider the p-order linear recursive equation (6.4.3) where = ( t ) t∈Z is an input sequence valued in H 0 and A 1 , . . . , A p ∈ L b (H 0 ). If is a white noise (that is, it is centered and weakly stationary with a constant spectral density operator), then Equation (6.4.3) is called a (functional) p-order auto-regressive (AR(p)) equation. If can be written for some positive integer q as (6.4.4) where Z = (Z t ) t∈Z is a centered white noise valued in H 0 and B 1 , . . . , B p ∈ L b (H 0 ), then is called a (functional) moving average process of order q (MA(q)) and Eq. ( 6.4.3) is called a (functional) (p, q)-order auto-regressive moving average (ARMA(p, q)) equation. Note that (6.4.4) can be written as the spectral domain filtering

Y t = p ∑ k=1 A k Y t-k + t , t ∈ Z ,
t = Z t + q ∑ k=1 B k Z t-k , t ∈ Z ,
(dλ) = θ(e -iλ ) Ẑ(dλ) with θ(z) = Id H 0 + p ∑ k=1 B k z k . (6.4.5)
Weakly stationary solutions of AR(p) or ARMA(p, q) equations are called AR(p) or ARMA(p, q) processes. The existence (and uniqueness) of a weakly stationary solution to Eq. ( 6.4.3) is given by the following result where U = {z ∈ C : |z| = 1} is the complex unit circle.

Theorem 6.4.1. Let = ( t ) t∈Z be a centered weakly stationary process valued in H 0 and A 1 , . . . , A p ∈ L b (H 0 ) satisfying the condition

φ(z) = Id H 0 - p ∑ k=1 A k z k is invertible in L b (H 0 ) for all z ∈ U. (6.4.6)
Then, setting Φ(λ) = φ(e -iλ ) for all λ ∈ R, the processes Y = F Φ -1 ( ) is well defined and is the unique weakly stationary solution Y = (Y t ) t∈Z satisfying (6.4.3). Moreover, the process Y admits the linear representation (6.4.7) where (P k ) k∈Z is a sequence valued in L b (H 0 ) whose operator norms have exponential decays at ±∞.

Y t = ∑ k∈Z P k t-k , t ∈ Z ,
Theorem 6.4.1 is usually proven in the Banach space valued case by constructing the explicit expansion (6.4.7) from algebraic arguments (see Spangenberg (2013, Corollary 2.2) and the references in the proof). In Section 6.5 we provide a very short proof relying on linear filtering in the spectral domain.

Let us introduce some notation which will be useful in the following.

Definition 6.4.1 (Polynomial sets P d (H 0 ) and P * d (H 0 )). For any integer d ∈ N, let P d (H 0 ) denote the set of polynomials p of degree d with coefficients in L b (H 0 ) and such that p(0) = Id H 0 . Further denote by P * d (H 0 ) the subset of all p ∈ P d (H 0 ) which are invertible on U (as φ in (6.4.6)).

Fractional operator integration of weakly stationary processes

In the following, we use the notation (1z) D for some D ∈ L b (H 0 ) and z ∈ C \ [1, ∞). This must be understood as

(1 -z) D = exp(D ln(1 -z)) = ∞ ∑ k=0 1 k! (D ln(1 -z)) k ,
where ln denotes the principal complex logarithm, so that Gohberg and Leiterer (2009, Chapter 1) for an introduction on this subject.

z → ln(1 -z) is holomorphic on C \ [1, ∞), and so is z → (1 -z) D , as a L b (H 0 )-valued function, see
Definition 6.4.3 (Fractional integration operator transfer function). Let H 0 be a separable Hilbert space and D ∈ L b (H 0 ). We define the D-order fractional integration operator transfer function FI D by

FI D (λ) =      1 -e -iλ -D if λ = 0, 0 otherwise.
Using the properties of z → (1z) D recalled previously, we get that FI D is a mapping from T to L b (H 0 ), continuous on T \ {0}. Then we have FI D ∈ F s (T, B(T), H 0 ) and we can define the filter F FI D as in (6.1.6) whose domain of definition are the centered weakly stationary H 0 -valued processes X ∈ S FI D (Ω, F , P). Then a fractionaly integrated autoregressive moving average (FIARMA) process is simply the output of the filter in the case where X is an ARMA process, as defined in the following. Definition 6.4.4 (FIARMA processes). Let H 0 be a separable Hilbert space and p, q be two non-negative integers. Let D ∈ L b (H 0 ), θ ∈ P q (H 0 ), φ ∈ P * p (H 0 ) and Z be an H 0 -valued centered white noise. Let X be the ARMA(p, q) process defined by X(dλ) = [φ(e -iλ )] -1 θ(e -iλ ) Ẑ(dλ) and suppose that X ∈ S FI D (Ω, F , P). Then the process defined by Y = F FI D (X), or, in the spectral domain, by

Ŷ(dλ) = FI D (λ)φ -1 (e -iλ )θ(e -iλ ) Ẑ(dλ) , (6.4.10)
is called a FIARMA process of order (p, q) with long memory operator D, shortened as FIARMA(D, p, q). Remark 6.4.1. Definition 6.4.4 extends the usual definition of univariate (C or R-valued) ARFIMA(p, d, q) processes to the Hilbert-valued case. In the general case we use the acronym FIARMA to indicate the order of the operators in the definition (6.4.10), where the fractional integration operator appears on the left of the autoregressive operator, itself appearing on the left of the moving average operator. We also respected this order in the list of parameters (D, p, q). Of course, one can also define an ARFIMA(p, D, q) process as the solution of (6.4.3) with defined as a FIARMA(0, D, q) process but the ARFIMA(p, D, q) process do not coincide with the FIARMA(p, D, q) process (this is already the case in finite dimension larger than 1). Observe that in the univariate case all the operators commute and FIARMA and ARFIMA boils down to the same definition. Note also that Definition 6.4.4 extends the definition of ARFIMA curve time series proposed in Li, Robinson, and Shang, 2020 in the case where H 0 is an L 2 (C, B(C), Leb) for some compact set C ⊂ R and where D is a scalar operator, that is D = d Id for some d ∈ (-1/2, 1/2). We will see below that, in this case, we have X ∈ S FI D (Ω, F , P) for any ARMA process X, see Remark 6.4.2 below.

Since FI D has a singularity at the null frequency, we want to provide conditions to ensure that, given a weakly stationary process X, the filter with transfer function FI D applies to X i.e. we look for conditions implying that X ∈ S FI D (Ω, F , P). For instance in the scalar case, it is well known that if X has a positive and continuous spectral density at the null frequency, then F FI d (X) is well defined if and only if d < 1/2. The Hilbert valued case relies on the singular value decomposition of D, which we will assume to be normal. Based on the spectral decomposition of a normal operator, we derive, in the following result, a necessary and sufficient condition involving the spectral operator density of X and the spectral decomposition of D which is recalled in Appendix A. Theorem 6.4.4. Let H 0 be a separable Hilbert space, D ∈ L b (H 0 ) and X = (X t ) t∈Z be an H 0 -valued weakly stationary time series defined on (Ω, F , P) with spectral operator measure ν X . Suppose that D is normal, with singular value function d on G 0 := L 2 (V, V, ξ) and decomposition operator U. Let µ be a nonnegative measure on (T, B(T)) which dominates ν X and let h ∈

L 2 (T, B(T), S 2 (G 0 ), µ) such that λ → h(λ)[h(λ)] H is the spectral operator density function of UX = (UX t ) t∈Z with respect to µ, that is, h(λ)[h(λ)] H = U g X (λ) U H for µ -a.e. λ ∈ T ,
where g X = dν X dµ . Let denote the T-joint kernel function of h. Then the three following assertions are equivalent.

(i) We have X ∈ S FI D (Ω, F , P).

(ii) There exists η ∈ (0, π) arbitrarily small such that Equation (6.4.11) holds for all η ∈ (0, π). Remark 6.4.2. If the dominating measure µ is the Lebesgue measure and if d is a constant function, d ≡ d for some d < 1/2 then the integral in (6.4.11) is bounded from above by

V 2 ×(-η,η) |λ| -2 (d(v)) (v, v ; λ) 2 ξ(dv)ξ(dv )µ(dλ) < ∞ . (6.4.11) (iii)
2η 1-2d 1 -2d Leb-essup λ∈(-η,η) V 2 (v, v ; λ) 2 ξ(dv)ξ(dv ) = 2η 1-2d 1 -2d Leb-essup λ∈(-η,η) g X (λ) 1 .
Thus, in this case, a sufficient condition for X ∈ S FI D (Ω, F , P) is to have that g X 1 is locally bounded around the null frequency. This is always the case if X is an ARMA process as in Definition 6.4.2. Theorem 6.4.5. Let H 0 be a separable Hilbert space and D ∈ L b (H 0 ). Suppose that X is an H 0 -valued ARMA(p, q) process defined by

X(dλ) = φ(e -iλ ) -1 θ(e -iλ ) Ẑ(dλ) ,
with θ ∈ P q (H 0 ), φ ∈ P * p (H 0 ) and Z a white noise with covariance operator Σ. Suppose that D is normal, with singular value function d on G 0 := L 2 (V, V, ξ) and decomposition operator U. Let W be a jointly measurable version of the G 0 -valued variable W = U[φ(1)] -1 θ(1)Z 0 and define

σ W : v → E W(v, •) 2 1/2
. Now, consider the following assertions.

(i) We have X ∈ S FI D (Ω, F , P).

(ii) We have (d) < 1/2, ξa.e. on {σ W > 0}.

(iii) We have

{ (d)<1/2} σ 2 W (v) 1 -2 (d(v))
ξ(dv) < +∞.

(iv) We have (d) < 1, ξa.e.

(v) We have φ(z) = θ(z) = Id for all z ∈ C (i.e. X = Z).

Then (i) implies (ii) and (iii). Conversely, if (iv) or (v) hold, then (i) is implied by (ii) and (iii).

Remark 6.4.3. If (d) < 1/2, ξa.e. then both (ii) and (iv) hold, and (iii) simplifies to

σ 2 W (v) 1 -2 (d(v)) ξ(dv) < +∞ .
Hence, applying our result, we get that (i) is implied by

(vi) (d) < 1/2, ξ -a.e. and σ 2 W (v) 1 -2 (d(v)) ξ(dv) < +∞,
which we think is the most useful consequence of this theorem. However it is important to note that (vi) is not necessary as our result says that, under assertion (v) (X is a white noise), only the sufficient conditions (ii) and (iii) are necessary (and it is easy to find D and Σ such that (ii) and (iii) holds but (vi) does not). Observe also that since

σ 2 W (v) ξ(dv) = E W 2 G 0 ≤ φ(1)] -1 θ(1) L b (H 0 ) E Z 0 2 H 0 < +∞ , Condition (ii) is immediately satisfied if (d) uniformly stay away from 1/2 on {σ W > 0}, that is, (d) ≤ 1/2 -η ξ -a.e. on {σ W > 0} for some η > 0.
In the n-dimensional case with n finite, we have V = {1, . . . , n}, ξ is the counting measure on V and U can be interpreted as a n × n unitary matrix, and d and σ W as n-dimensional vectors. Condition (ii) then says that (d(k)) < 1/2 on the components k ∈ {1, . . . , n} such that σ W (k) > 0, and Condition (iii) always follows from (ii). For the real univariate case (n = 1, D = d ∈ R), Condition (ii) says that d < 1/2 or σ W = 0 and the latter happens if and only if Σ = 0 (Z is the null process) or θ(1) = 0 (the MA operator contains a difference operator of order larger than or equal to 1). In particular we find the usual d < 1/2 condition for the existence of a weakly stationary ARFIMA(p, d, q) model in the case where the underlying ARMA(p, q) process is invertible (θ does not vanish on the unit circle).

Other long-memory processes

Several non-equivalent definitions of long rang dependence or long memory are available in the literature for time series. Some approaches focus on the behavior of the auto-covariance function at large lags, others on the spectral density at low frequencies, see Pipiras and Taqqu (2017, Section 2.1) and the references therein. Separating short range from long range dependence is often made more natural within a particular class of models. For instance, for a Hilbert-valued process Y = (Y t ) t∈Z , one may rely on a causal linear representation, namely

Y t = ∞ ∑ k=0 P k t-k , t ∈ Z i.e. Ŷ(dλ) = ∞ ∑ k=0
P k e -iλk ˆ (dλ) , (6.4.12) where = ( t ) t∈Z is a centered white noise valued in the separable Hilbert space H 0 and (P k ) k∈Z is a sequence of L b (H 0 ) operators. Then, by isometry, the first infinite sum appearing in (6.4.12) converges in M(Ω, F , H 0 , P) if and only if the second one converges in L 2 (T, B(T), L b (H 0 ), ν ). A sufficient condition for these convergences to hold is ∑ ∞ k=0 P k L b (H 0 ) < +∞, and this assumption is referred to as the short-range dependence (or short memory) case (for example ARMA processes), in contrast to long range dependence (long-memory) case, for which ∑ ∞ k=0 P k L b (H 0 ) = +∞, under which the convergences in (6.4.12) are no longer granted. In D üker, 2018, the case where P k = (k + 1) -N for some normal operator N ∈ L b (H 0 ) is investigated and the following result is obtained. Lemma 6.4.6. Let H 0 be a separable Hilbert space, N ∈ L b (H 0 ) be a normal operator with singular value function n on G 0 := L 2 (V, V, ξ) and decomposition operator U. Let h : v → (n(v)). Let := ( t ) t∈Z be a white noise in M(Ω, F , H 0 , P) and

σ 2 W : s → E W(v, •) 2 , where W is a jointly measurable version of W = U 0 . Suppose that h > 1 2 ξ-a.e. and V σ 2 W (v) 2h(v) -1 ξ(dv) < +∞ . (6.4.13)
Then, for all t ∈ Z,

Y t = +∞ ∑ k=0 (k + 1) -N t-k (6.4.14)
converges in M(Ω, F , H 0 , P). If, moreover, ( k ) k∈Z is an i.i.d. sequence, then the convergence also holds a.s.

In D üker (2018, Theorem 2.1), the author also studies the partial sums of the process (6.4.14) and exhibits asymptotic properties which naturally extend the usual behavior observed for univariate long-memory processes. In the following, we explain how the process (6.4.14) can be related to a FI-ARMA(D,0,0) process. First we prove the analogous of Lemma 6.4.6, namely, that that Condition (6.4.13) implies the existence of this FIARMA process. Lemma 6.4.7. Let N, , h and σ W be as in Lemma 6.4.6. Set D = Id H 0 -N. Then Condition (6.4.13) implies ∈ S FI D (Ω, F , P).

We can now state a result which shows that the two process defined by Lemmas 6.4.6 and 6.4.7 (6.4.14) are closely related up to a bounded operator C and to an additive short-memory process. Proposition 6.4.8. Under the assumptions of Lemma 6.4.6, defining Y = (Y t ) t∈Z by (6.4.14), there exists an operator C ∈ L b (H 0 ) and an sequence

(∆ k ) k∈N ∈ L b (H 0 ) N with ∑ +∞ k=0 ∆ k L b (H 0 ) < +∞ such that F FI D ( ) = C Y + Z ,
where Z is the short-memory process defined, for all t ∈ Z, by Z t = ∑ ∞ k=0 ∆ k t-k .

6.5 Postponed proofs 6.5.1 Proofs of Section 6.1

The proof of Theorem 6.1.2 relies on the following lemma.

Lemma 6.5.1. Let H 0 , G 0 , I 0 be separable Hilbert spaces and

P ∈ O(G 0 , I 0 ), Q ∈ K(H 0 , G 0 ). The following assertions hold. (i) Im( Q H ) = Im(Q). (ii) If Im(Q) ⊂ D(P), then (PQ)(PQ) H = (P Q H )(P Q H ) H . (iii) If Im(Q) ⊂ D(P), then PQ ∈ S 2 (H 0 , I 0 ) if and only if P Q H ∈ S 2 (G 0 , I 0 ).
In this case PQ 2 = P Q H 2 . Proof. For convenience, we only consider the case where the spaces have infinite dimensions. The singular values decomposition of Q yields for two orthonormal sequences

(ψ n ) n∈N ∈ G N 0 and (φ n ) n∈N ∈ H N 0 , Q = ∑ n∈N σ n ψ n ⊗ φ n and Q H = ∑ n∈N σ n ψ n ⊗ ψ n .
Proof of (i). We have

Im(Q) = ∑ n∈N σ n x n ψ n : (x n ) n∈N ∈ 2 (N) = Im( Q H ).
Proof of (ii). By the first point both compositions PQ and P Q H make sense. Consider the polar decomposition of Q

H : Q H = U Q H , with U = ∑ n∈N φ n ⊗ ψ n . Then Q = Q H U H and (PQ)(PQ) H = P Q H U H U P Q H H = P Q H P Q H H , where we used that Q H U H U = Q H .
Proof of (iii). We have that PQ ∈ S 2 (H 0 , I 0 ) if and only if (PQ)(PQ) H ∈ S 1 (I 0 ), which is equivalent to P Q H ∈ S 2 (G 0 , I 0 ) by the previous point.

We can now prove Theorem 6.1.2.

Proof of Theorem 6.1.2. Let µ be a dominating measure for ν 1 and g = dν dµ , then, by definition of ΦνΦ H , µ also dominates ΦνΦ H 1 and dΦνΦ

H dµ = (Φg 1/2 )(Φg 1/2 ) H . Hence, dΦνΦ H dµ 1/2 = (Φg 1/2
) H and we get, by Proposition 5.1.4,

Ψ ∈ L 2 (Λ, A, O(H 0 , I 0 ), ΦνΦ H ) ⇔      Im (Φg 1/2 ) H ⊂ D(Ψ) µ-a.e. Ψ (Φg 1/2 ) H ∈ L 2 (Λ, A, S 2 (G 0 , I 0 ), µ) ⇔      Img 1/2 ⊂ D(ΨΦ) µ-a.e. ΨΦg 1/2 ∈ L 2 (Λ, A, S 2 (H 0 , I 0 ), µ) ⇔ ΨΦ ∈ L 2 (Λ, A, O(H 0 , I 0 ), ν) ,
where the second equivalence comes from Lemma 6.5.1 and the fact that for all λ ∈ Λ, D(

Ψ(λ)Φ(λ)) is the preimage of D(Ψ(λ)) by Φ(λ) which gives that Im(g 1/2 (λ)) ⊂ D(Ψ(λ)Φ(λ)) if and only if Im(Φ(λ)g 1/2 (λ)) ⊂ D(Ψ(λ)).
To prove Assertion (a), note that, for all

Ψ, Θ ∈ L 2 (Λ, A, O(G 0 , I 0 ), ΦνΦ H ) and A ∈ A, we have (ΨΦ)ν(ΘΦ) H (A) = A ΨΦg 1/2 ΘΦg 1/2 H dµ = A Ψ (Φg 1/2 ) H Θ (Φg 1/2 ) H H dµ = Ψ(ΦνΦ H )Θ H (A) ,
where the second equality comes from Lemma 6.5.1. Assertion (a) follows as well as Assertion (b) by taking A = Λ. Finally, to show Assertion (c), suppose that Φ is injective ν 1 -a.e. then

Φ -1 Φ : λ → Id H 0 1 {Φ(λ) is injective} is in L 2 (Λ, A, O(H 0 ), ν) which gives that Φ -1 ∈ L 2 (Λ, A, O(G 0 , H 0 ), ΦνΦ H ) by Assertion (a).
Finally, we prove Corollary 6.1.3

Proof of Corollary 6.1.3. Assertion (i) follows from Assertion (b) of Theorem 6.1.2 and Theorem 5.1.7. The two other assertions are a bit more involved.

Proof of Assertion (ii).

If W ∈ ŜΦ (Ω, F , P), then the equivalence between W ∈ ŜΨΦ (Ω, F , P) and FΦ (W) ∈ ŜΨ (Ω, F , P) is just another formulation of the equivalence (6.1.3) with ν = ν W . Suppose that it holds and set V := FΦ (W) so that ν V = ΦνΦ H . Then, (6.1.4) means that, for all Ψ ∈ L 2 (Λ, A, O(G 0 , I 0 ), ν V ) and for all A ∈ A, A Ψ dV = A ΨΦ dW. Replacing Ψ by Ψ1 A , it is sufficient to show this identity with A = Λ. Using that the integral with respect to a random c.a.g.o.s. measure is Gramianisometric and Assertion (b) of Theorem 6.1.2, the mappings Ψ → Ψ dV and Ψ → ΨΦ dW are Gramian-isometric from L 2 (Λ, A, O(G 0 , I 0 ), Φν W Φ H ) to M(Ω, F , I 0 , P). Hence by Theorem 5.1.3, they coincide on the whole space if they coincide on all Ψ = 1 A P for A ∈ A and P ∈ L b (G 0 , I 0 ). To conclude the proof of Assertion (ii), it is thus enough to prove that, for all A ∈ A and P ∈ L b (G 0 , I 0 ),

A P dV = A PΦ dW .
This identity follows from the definition of V and the fact that on both sides the operator P can be moved in front of the integrals. This latter fact directly follows from the definition of the integral for the left-hand side and for the right-hand side when Φ = 1 B for some B ∈ A, which extends to all Φ by observing that Φ → PΦ dW and Φ → P Φ dW are continuous on

L 2 (Λ, A, O(H 0 , G 0 ), ν W ).
Proof of Assertion (iii). Continuing with the setting of the proof of the previous point, we now suppose that Φ is injective ν W 1 -a.e. Assertions (c) and (a) of Theorem 6.1.

2 give that Φ -1 ∈ L 2 (Λ, A, O(G 0 , H 0 ), ν V ) (i.e. V ∈ ŜΦ -1 (Ω, F , P)) and Φ -1 ν V Φ -1 H = ν W .
Hence, writing Relation (6.1.4) with Ψ = Φ -1 , we get FΦ -1 (V) = FΦ -1 Φ (W) = W. Moreover, reversing the roles of W and V in assertion (i) gives the embedding H W,I 0 ⊆ ∼ H FΦ (W),I 0 which, with Assertion (i), allow us to conclude that H W,I 0 ∼ = H FΦ (W),I 0 .

Proofs of Section 6.3.1

Proof of Lemma 6.3.1. We first prove that (

v, v ) → ℋ (v, v ) ℋ (v , v ) ξ(dv ) is in L 2 (V 2 , V ⊗2 , ξ ⊗2 ). By the Cauchy-Schwartz inequality, we have, for all (v, v ) ∈ V 2 , ℋ (v, v ) ℋ (v , v ) ξ(dv ) 2 ≤ ℋ (v, v ) 2 ξ(dv ) ℋ (v , v ) 2 ξ(dv )
and, integrating the right-hand side with respect to ξ(dv) and ξ(dv ) and using the fact that |ℋ | 2 dξ ⊗2 = h 2 2 we get that

ℋ (v, v ) ℋ (v , v ) ξ(dv ) 2 ξ(dv ) ξ(dv) ≤ h 4 2 < +∞ . (6.5.1) Hence (v, v ) → ℋ (v, v ) ℋ (v , v ) ξ(dv ) is well defined and is in the space L 2 (V 2 , V ⊗2 , ξ ⊗2 ). Now, for all f ∈ H 0 and v ∈ V, ℋ (v, v ) ℋ (v , v ) f (v ) ξ(dv )ξ(dv ) 2 ≤ f 2 H 0 ℋ (v, v ) ℋ (v , v ) ξ(dv ) 2 ξ(dv )
which is is finite for ξa.e. v ∈ V by (6.5.1). Hence, by Fubini's theorem, for

ξ -a.e. v ∈ V, G f (v) = HH H f (v) = ℋ (v, v ) ℋ (v , v ) f (v ) ξ(dv ) ξ(dv ) = ℋ (v, v )ℋ (v , v ) ξ(dv ) f (v ) ξ(dv )
which implies (6.3.1) by uniqueness of the kernel associated to G.

Proof of Proposition 6.3.2. Define, for all v, v ∈ V and λ ∈ T,

n (v, v ; λ) := ∑ 0≤i,j≤n φ H i K(λ)φ j φ i (v) φj (v ) ,
and, for all > 0,

N (λ) = inf n ∈ N : ∑ i or j>n φ H i K(λ)φ j 2 ≤ . Note that since ∑ i,j∈N φ H i K(λ)φ j 2 = K(λ) 2 < ∞, N (λ) is well defined and finite. Now let us define, for all v, v ∈ V and λ ∈ T, (v, v ; λ) := lim n→∞ N 2 -n (λ) (v, v ; λ) , (6.5.2)
whenever this limit exists in C and set

(v, v ; λ) = 0 otherwise. Since (φ k ⊗ φk ) k,k ∈N is a Hilbert basis of L 2 (V 2 , V ⊗2 , ξ ⊗2
), we immediately have that, for any λ ∈ Λ, N 2 -n (λ) (•; λ) converges in the sense of this L 2 space to ∑ i,j∈N φ H i K(λ)φ j φ i ⊗ φj , and so this limit must be equal to (•; λ) ξ ⊗2a.e.. It follows that, that for any λ ∈ Λ, for all i, j ∈ N, (v, v ; λ)

φi (v)φ j (v ) ξ(dv)ξ(dv ) = φ H i K(λ)φ j ,
which gives that K(λ) is an integral operator associated to the kernel (•; λ). Since (v, v , λ) → (v, v ; λ) is measurable by definition, we get that it is the Λ-joint kernel of K as in Definition 6.3.1. Assertion (i) follows by observing

that, if K ∈ L 2 (Λ, A, S 2 (H 0 ), µ), then (v, v , λ) → n (v, v ; λ) converges in L 2 (V 2 × Λ, V ⊗2 ⊗ A, ξ ⊗2 ⊗ µ)
and the limit must be equal to

ξ ⊗2 ⊗ µ - a.e. since for each λ ∈ Λ, (v, v ) → n (v, v ; λ) converges to (•; λ) in L 2 (V 2 , V ⊗2 , ξ ⊗2 ).
It only remains to prove Assertion (ii). Assume that K ∈ L 1 (Λ, A, S + 1 (H 0 ), µ) as in this assertion and let H ∈ L 2 (Λ, A, S 2 (H 0 ), µ) be such that for all λ ∈ Λ, K(λ) = H(λ)H(λ) H (for example, by Lemma B.2.1, we can take

H(λ) = K(λ) 1/2 ) . Then by Assertion (i), the Λ-joint kernel of H satisfies ℋ ∈ L 2 (V 2 × Λ, V ⊗2 ⊗ A, ξ ⊗2 ⊗ µ).
Using Lemma 6.3.1 and the Cauchy-Schwartz inequality, we get that the integral in (6.3.3) is bounded from above by |ℋ (v, v ; λ)| 2 ξ(dv)ξ(dv )µ(dλ) which is finite.

Proofs of Section 6.3.2

Proof of Proposition 6.3.3.

Decomposing Y on (φ n ) n∈N , we can define Ỹ on V × Ω by Ỹ(v, ω) =      lim n→∞ S Y N Y 2 -n (ω) (v, ω) if the limit exists in C , 0 otherwise,
where we set, for all n ∈ N, ω ∈ Ω, v ∈ V and > 0,

S Y n (v, ω) = n ∑ k=0 Y(ω), φ k φ k (v) and N Y (ω) = inf n ∈ N : S Y n (•, ω) -Y(ω) 2 H 0 ≤ .
It is easy to show that the following assertions hold for all ω ∈ Ω:

(i) N Y (ω) is well defined in N for all > 0, (ii) (N Y 2 -n (ω)) n is a non-decreasing sequence, (iii) as n → ∞, S Y N Y 2 -n (ω) (•, ω) converges to Y in H 0 ; (iv) S Y N Y 2 -n (ω) (v, ω) converges to Ỹ(v, ω) for ξ -a.e. v ∈ V, (v) Ỹ(•, ω) = Y(ω) (as elements of H 0 ). Since S Y
n is jointly measurable on V × Ω for all n ∈ N and N Y is measurable on Ω for all > 0, we get the result.

Proof of Lemma 6.3.4. As explained before the statement of the lemma, we have that

(v, ω) → ỸN (v, ω) := N ∑ n=0 Y(ω), φ n H 0 φ n (v) converges to Ỹ as N → ∞ in L 2 (V × Ω, V ⊗ F , ξ ⊗ P). Let us define, for all v, v ∈ V, N (v, v ) = N ∑ n=0 (•, v ), φ n H 0 φ n (v) Using that ∈ L 2 (V 2 , V ⊗2 , ξ ⊗2 ), it is easy to show that N converges to in L 2 (V 2 , V ⊗2 , ξ ⊗2 ) as N → +∞.
By the Cauchy-Schwartz inequality, we have that the mappings (g,

h) → [v → E g(v, •) h(v, •) ] and (g, h) → [v → g(v, •), h(v, •) H 0 ] are sesquilin- ear and continuous from L 2 (V × Ω, V ⊗ F , ξ ⊗ P) to L 1 (V, V, ξ) and from L 2 (V 2 , V ⊗2 , ξ ⊗2 ) to L 1 (V, V, ξ), respectively. This, with the two previous con- vergence result shows that [v → E ỸN (v, •) 2 ] and [v → N (v, •) 2 H 0 ] both converge in L 1 (V, V, ξ), to E Ỹ(v, •)
2 ] and (v, •) 2 H 0 , respectively, that is to the left-hand side and right-hand side of (6.3.4).

Hence to conclude we only have to show that, for all v ∈ V,

E ỸN (v, •) 2 = N (v, •) 2 H 0 . (6.5.3)
Indeed we can write

E ỸN (v, •) 2 = E N ∑ n,m=0 Y, φ n H 0 φ m , Y H 0 φ n (v)φ m (v) = N ∑ n,m=0 φ H n Cov(Y)φ m φ n (v)φ m (v) .
Using Cov(Y) = KK H and Fubini's theorem leads to

φ H n Cov(Y)φ m = (•, v ), φ n H 0 (•, v ), φ m H 0 ξ(dv ) .
Inserting this in the previous display, the double sum, put inside the integral in ξ(dv ), separates into a product of two conjugate terms and we get

E ỸN (v, •) 2 = N ∑ n=0 (•, v ), φ n H 0 φ n (v) 2 ξ(dv ) .
so that (6.5.3) is proven, which concludes the proof.

Proof of Proposition 6.3.5. For all n, n ∈ N and λ ∈ T, by the Cauchy-Schwartz inequality and since

φ n H 0 = φ n H 0 = 1, we have ℊ X (v, v ; λ) φn (v)φ n (v ) ξ(dv)ξ(dv ) ≤ ℊ X (v, v ; λ) 2 ξ(dv)ξ(dv ) 1/2
. By Proposition 6.3.2(ii), we get that

ℊ X (v, v ; λ) φn (v)φ n (v ) ξ(dv)ξ(dv ) µ(dλ) < ∞ . (6.5.4)
Therefore we can apply Fubini's theorem which gives, for all n, n ∈ N, and s, t ∈ Z,

e iλ(s-t) ℊ X (v, v ; λ) φn (v)φ n (v ) ξ(dv)ξ(dv ) µ(dλ) = e iλ(s-t) φ H n g X (λ)φ n µ(dλ) = Cov φ H n X s , φ H n X t .
On the other hand, by Fubini's theorem, we have that, for all n, n ∈ N, and

s, t ∈ Z, Cov φ H n X s , φ H n X t = Cov Xs (v, •), Xt (v , •) φn (v)φ n (v ) ξ(dv)ξ(dv ) .
This is also φ H n Cov (X s , X t ) φ n and since Cov (X s , X t ) is a trace class hence Hilbert Schmidt operator the previous display says that this operator is associated with the

L 2 (V 2 , V ⊗2 , ξ ⊗2 ) kernel (v, v ) → Cov Xs (v, •), Xt (v , •) .
The last two displays now imply that, for all n, n ∈ N, and s, t ∈ Z,

Cov Xs (v, •), Xt (v , •) φn (v)φ n (v ) ξ(dv)ξ(dv ) = e iλ(s-t) ℊ X (v, v , λ) φn (v)φ n (v ) ξ(dv)ξ(dv ) µ(dλ) = e iλ(s-t) ℊ X (v, v , λ) µ(dλ) φn (v)φ n (v ) ξ(dv)ξ(dv ) ,
where we used Fubini's theorem (justified by (6.5.4) as above). Since the

kernel (v, v ) → Cov Xs (v, •), Xt (v , •) is in L 2 (V 2 , V ⊗2 , ξ ⊗2 ) of which (φ k ⊗ φk ) k,k
∈N is a Hilbert basis, the last display shows that, for all s, t ∈ Z,

Cov Xs (v, •), Xt (v , •) = e iλ(s-t) ℊ X (v, v ; λ) µ(dλ) for ξ ⊗2 -a.e. (v, v ),
which concludes the proof.

Proofs of Section 6.4.1

Proof of Theorem 6.4.1. Denote Φ(λ) = φ(e -iλ ) for all λ ∈ R. As a trigonometric polynomial with L b (H 0 )-valued coefficients, Φ belongs to the space F s (T, B(T), L b (H 0 )). Moreover, Relation (6.4.6) directly implies that Φ -1 ∈ F s (T, B(T), L b (H 0 )). By Corollary 6.1.4, it follows that

(i) Y = F Φ -1 ( ) satisfies F Φ (Y) =
, and thus is a solution of (6.4.3);

(ii) for any centered weakly stationary process

Y such that F Φ (Y) = , we have Y = F Φ -1 • F Φ (Y) = F Φ -1 ( ).
We thus conclude that Y = F Φ -1 ( ) is the unique weakly stationary solution of (6.4.3).

Then the representation (6.4.7) holds as an immediate consequence of the fact that z → φ(z) -1 is L b (H 0 )-valued holomorhic on a ring containing the unit circle, so that

[Φ(λ)] -1 = [φ(e -iλ )] -1 = ∑ k∈Z P k e -iλk ,
where (P k ) k∈Z are the Laurent series coefficients of φ -1 (see Gohberg and Leiterer (2009, Theorem 1.9.1), hence the series in the displayed equation converges absolutely in L b (H 0 )) and it can be shown that they have exponential decay at ±∞ (as a consequence of Eq. (1.9.4) in Gohberg and Leiterer (2009, Theorem 1.9.1)). Gohberg and Leiterer (2009, Theorem 1.8.5) implies that there exists ρ > 0 and a sequence

Proof of Proposition

6.4.2. Since z → [φ(z)] -1 θ(z) is holomorphic in an open ring containing U,
(P n ) n∈N ∈ L b (H 0 ) N such that ∑ ∞ n=0 ρ n P n L b (H 0 ) < ∞ and [φ(z)] -1 θ(z) coincides with the L b (H 0 )-valued power series ∑ ∞ n=0 (z -1) n P n on the set {z ∈ C : |z -1| ≤ ρ}. Now, take η > 0 such that e -iλ : λ ∈ (-η, η) ⊂ {z ∈ C : |z -1| ≤ ρ}. Then we have, for all λ ∈ (-η, η), ∞ ∑ n=0 e -iλ -1 n P n L b (H 0 ) ≤ ∞ ∑ n=0 ρ n P n L b (H 0 ) < ∞ .
(6.5.5)

Thus we can write [φ(e -iλ )] -1 θ(e -iλ ) = P 0 + λ Ψ(λ) by setting Ψ(0) = 0 and, for all λ ∈ (-η, η),

Ψ(λ) = e -iλ -1 λ ∞ ∑ n=1 (e -iλ -1) n-1 P n ,
where the sum is absolutely convergent in L b (H 0 ) and where we used the standard convention (e -iλ -1)/λ = 1 for λ = 0 (hence Ψ(0) = 0). Since P 0 = [φ(1)] -1 θ(1), it follows by Proposition 6.1.1 that (6.4.8) holds with k(λ) := Ψ(λ)Σ 1/2 . Since Ψ is L b (G 0 )-valued, continuous and bounded on (-η, η), we get that k is continuous and bounded from (-η, η) to S 2 (H 0 ). Suppose now that H 0 = L 2 (V, V, ξ). For all n ∈ N, let n denote the kernel associated to the operator k n := P n Σ 1/2 ∈ S 2 (H 0 ). Let us introduce the following notation for all

V 2 × (-η, η) → C-measureable function f , f * = V 2 Leb-essup λ∈(-η,η) f (v, v ; λ) 2 ξ(dv)ξ(dv ) 1/2
, which allows to define a Banach space L * endowed with • * as a norm. Note that, for all n ∈ N, ( By (6.5.5) and since λ → (e -iλ -1) n /λ is bounded by ρ n-1 on (-η, η), we get that

| n | 2 dξ ⊗2 ) 1/2 = P n Σ 1/2 2 = P n L b (H 0 ) Σ 1/2 1 .
∞ ∑ n=1 (v, v λ) → (e -iλ -1) n λ (v, v ) * = ∞ ∑ n=1 Leb-essup λ∈(-η,η) (e -iλ -1) n λ P n Σ 1/2 2 ≤ Σ 1/2 1 +∞ ∑ n=1 ρ n-1 P n L b (H 0 ) < ∞ .
To conclude the proof, we observe that L * is continuously embedded in L 2 (V 2 × (-η, η), V ⊗2 ⊗ B(-η, η), ξ ⊗2 ⊗ Leb), which gives that the above series also converges in the latter space to the (-η, η)-joint kernel function of k and satifies (6.4.9).

Proof of Lemma 6.4.3. For any P, Q ∈ L b (H 0 ) such that P is invertible, we have that

UP -1 Q = [UPU -1 ] -1 [UQU -1 ]U.
Thus, we obtain, defining θ and φ as above,

UX(dλ) = U[φ(e -iλ )] -1 θ(e -iλ ) Z(dλ) = [ φ(e -iλ )] -1 θ(e -iλ ) UZ(dλ) .
It is then immediate to check that θ ∈ P q (G 0 ) and φ ∈ P * p (G 0 ), and that UZ = (UZ t ) t∈Z is a G 0 -valued white noise.

Proofs of Section 6.4.2

The proof of Theorem 6.4.4 relies on the following lemma. Lemma 6.5.2. For all z ∈ C and λ ∈ [-π, π], we have

(2/π) 2[ (z)] + |λ| 2 (z) e -π| (z)| ≤ (1 -e -iλ ) z 2 ≤ (π/2) 2[ (z)] -|λ| 2 (z) e π| (z)| .
(6.5.6)

Proof. Let z ∈ C with (z), then it can be shown that, for all λ ∈ (-π, π] \ {0},

(1 -e -iλ ) z 2 = 1 -e -iλ 2 (z) e -2 (z)b(e -iλ ) ,
where b(e -iλ ) denotes the argument of 1e -iλ that belongs to -π 2 , π 2 . It follows that e -π| (z)| ≤ e -2 (z)b(e -iλ ) ≤ e π| (z)| .

Using that |λ| π ≤ |sin(λ/2)| ≤ |λ| 2 for all λ ∈ (-π, π) and separating the cases where (z) ≥ 0 and where (z) < 0, we easily get (6.5.6).

Proof of Theorem 6.4.4. Recall that ξ is a σ-finite measure and L 2 (V, V, ξ) is separable since H 0 is by assumption and they are isomorphic. As defined in Section 6.1, X ∈ S FI D (Ω,

F , P) if and only if FI D ∈ L 2 (T, B(T), O(H 0 ), ν X ), which, by Assumption (b) of Proposition 5.1.4, is equivalent to have T (1 -e -iλ ) -D g X (λ) (1 -e -iλ ) -D H 1 µ(dλ) < +∞ . (6.5.7) We have, for all λ ∈ T \ {0}, since U is unitary from H 0 to L 2 (V, V, ξ), (1 -e -iλ ) -D g X (λ) (1 -e -iλ ) -D H 1 = U H M (1-e -iλ ) -d Ug X (λ) U H M H (1-e -iλ ) -d U 1 = M (1-e -iλ ) -d U g X (λ) U H M H (1-e -iλ ) -d 1 = M (1-e -iλ ) -d g UX (λ)M H (1-e -iλ ) -d 1 = M (1-e -iλ ) -d h(λ) 2 2 .
Hence (6.5.7) holds if and only if

T M (1-e -iλ ) -d h(λ) 2 2 µ(dλ) < +∞ , which, using the T-joint kernel of h reads (1 -e -iλ ) -d(v) (v, v ; λ) 2 ξ(dv)ξ(dv ) µ(dλ) < +∞ .
Applying Lemma 6.5.2 to z = -d(v), since d is a µ-essentially bounded function, we get that Assertion (i) is equivalent to

V 2 ×(-π,π] |λ| -2 (d(v)) (v, v ; λ) 2 ξ(dv)ξ(dv )µ(dλ) < ∞ .
This of course implies Assertion (iii), which implies Assertion (ii). Now, if Assertion (ii) holds, since |λ| -2

(d(v)) is bounded independently of v on λ ∈ (-π, π] \ (-η, η) and (v, v ; λ) 2 ξ(dv)ξ(dv )µ(dλ) = h(λ) 2 2 µ(dλ) < ∞ ,
we get back the above condition involving an integration over V 2 × (-π, π].

Proof of Theorem 6.4.5. Before proving the claimed implications, we start with some preliminary facts that are obtained from Lemma 6.4.3, Proposition 6.4.2, Lemma 6.3.4 and Theorem 6.4.4. By Lemma 6.4.3, the process UX = (UX t ) t∈Z is the G 0 -valued ARMA(p, q) process defined by UX(dλ) = [ φ(e -iλ )] -1 θ(e -iλ ) UZ(dλ), where θ := UθU -1 ∈ P q (G 0 ) and φ := UφU -1 ∈ P * p (G 0 ), and UZ = (UZ t ) t∈Z is a G 0 -valued white noise. Applying Proposition 6.4.2 with µ as the Lebesgue measure, we get that, for some η > 0,

ν UX has density h(λ)[h(λ)] H on (-η, η) with h valued in S 2 (G 0 ) satisfying, for all λ ∈ (-η, η), h(λ) = [ φ(1)] -1 θ(1)(UΣU H ) 1/2 + λ k(λ) , (6.5.8)
where k is continuous from (-η, η) to S 2 (G 0 ). Moreover, since G 0 = L 2 (V, V, ξ), Proposition 6.4.2 also gives that the joint kernel of k satisfies (6.4.9), which implies

s 2 (v)ξ(dv) ≤ Leb-essup λ∈(-η,η) (v, v ; λ)
2 ξ(dv )ξ(dv) < +∞ , (6.5.9)

where we defined, for all v ∈ V,

s(v) = Leb-essup λ∈(-η,η) (v, •; λ) G 0 .
Define, for any η ∈ (0, η),

I(η ) := V 2 ×(-η ,η ) |λ| -2 (d(v)) (v, v ; λ) 2 ξ(dv)ξ(dv ) dλ 2π (6.5.10) = V 2 ×(-η ,η ) |λ| -2 (d(v)) 0 (v, v ) + λ (v, v ; λ) 2 ξ(dv)ξ(dv ) dλ 2π ,
where is the kernel of h in (6.5.8) and 0 is the kernel of the operator k 0 := U[φ(1)] -1 θ(1)Σ 1/2 U H ∈ S 2 (G 0 ). Integrating w.r.t. v , by the Minkowski inequality, we get that (6.5.11) where we used that σ W (v) = 0 (v, •) G 0 for ξa.e. v ∈ V, which holds as a consequence of Lemma 6.3.4 since Cov(W) = k 0 k H 0 . Similarly, using the definition of s above, we can upper bound I(η ) by I(η ) ≤ 2(I 1 (η ) + I 2 (η )) , where

I(η ) ≥ V×(-η ,η ) |λ| -2 (d(v)) σ W (v) -|λ| (v, •; λ) G 0 2 ξ(dv) dλ 2π ,
I 1 (η ) = V×(-η ,η ) |λ| -2 (d(v)) σ 2 W (v) ξ(dv) dλ 2π
, and

I 2 (η ) = V×(-η ,η ) |λ| 2-2 (d(v)) s 2 (v) ξ(dv) dλ 2π .
(6.5.12)

To conclude these preliminaries, by Theorem 6.4.4, we have that Assertion (i) of Theorem 6.4.5 is equivalent to the two following assertions:

(vii) for all η ∈ (0, η), we have I(η ) < ∞;

(viii) there exists η ∈ (0, η) such that I(η ) < ∞.

We are now ready to prove the claimed implications. Proof of (i) ⇒(ii). Let us define, for any n ∈ N, thus, with (6.5.11), .5.13) Suppose that (i) holds. Then so does (vii) and thus, for all n ∈ N, the integral in (6.5.13) must be finite which implies (d) < 1/2, ξa.e. on A n (since (-η ,η ) |λ| -2d dλ = ∞ for d ≥ 1/2). On the other hand, we have n A n = {σ W > 0} ∪ {s < ∞} and, by (6.5.9), s < ∞ ξa.e.; hence we get (ii). Proof of (i) ⇒(iii). Note that, for all (v, λ)

A n = σ W > 2 -n ∩ {s ≤ 2 n } . Then, if η ∈ (0, 2 -2n-1 ], we have, for Leb -a.e. λ ∈ (-η , η ) and all v ∈ A n , |λ| (v, •; λ) ≤ η s(v) < 2 -n-1 < 2 -n < σ W (v) , which implies that σ W (v) -|λ| (v, •; λ) G 0 ≥ 2 -n -2 -n-1 = 2 -n-1 and
I(η ) ≥ 2 -2n-2 A n ×(-η ,η ) |λ| -2 (d(v)) ξ(dv) dλ 2π . ( 6
∈ V × (-η, η), σ W (v) -|λ| (v, •; λ) G 0 2 ≥ σ 2 W (v) -2 |λ| σ W (v) (v, •; λ) G 0 ≥ σ 2 W (v) -2 |λ| σ W (v) s(v) .
and thus, using (6.5.11), we get that, for all η ∈ (0, η),

I(η ) ≥ { (d)<1/2}×(-η ,η ) |λ| -2 (d(v)) σ W (v) -|λ| (v, •; λ) G 0 2 ξ(dv) dλ 2π ≥ { (d)<1/2}×(-η ,η ) |λ| -2 (d(v)) σ 2 W (v) ξ(dv) dλ 2π -2 { (d)<1/2}×(-η ,η ) |λ| 1-2 (d(v)) σ W (v) s(v) ξ(dv) dλ 2π = { (d)<1/2} η 1-2 (d(v)) 2π σ 2 W (v) 1 -2 (d(v)) ξ(dv) (6.5.14) - { (d)<1/2} η 2-2 (d(v)) 2π σ W (v) s(v) 1 -(d(v)) ξ(dv) . (6.5.15) Since d is bounded on V, we have that η 2-2 (d(v)) is upper bounded on v ∈ V and since (1 -(d(v))) -1 ≤ 1/2 on { (d) < 1/2}
, we get that the integral in (6.5.15) is bounded from above, up to a multiplicative constant, by

{ (d)<1/2} σ W (v) s(v) ξ(dv) ≤ σ W G 0 s G 0 ,
which is finite using (6.5.9) and σ W 2

G 0 = E W 2 G 0 < ∞. Using again that d is bounded on V, we have that η 1-2 (d(v))
is lower bounded by a positive constant on V. Hence, we finally get that, if (i) holds, then (vii) holds as well and what precedes yields Assertion (iii).

Proof of (ii) and (iii)⇒(i) under (iv) or (v).

To obtain (i), it is sufficient to show that Assertion (viii) holds, which, by (6.5.12), follows from I 1 (η ) < ∞ and I 2 (η ) < ∞. Under Assertion (ii), we have, for all η ∈ (0, η),

I 1 (η ) = { (d)<1/2} η 1-2 (d(v)) π σ 2 W (v) 1 -2 (d(v)) ξ(dv) ,
and since d is bounded, this integral is finite under (iii). Thus (ii) and (iii) imply that I 1 (η ) < ∞ for all η ∈ (0, η). To conclude the proof it only remains to show that I 2 (η ) < ∞ for some η ∈ (0, η) whenever (iv) or (v) holds. We have in fact I 2 (η ) < ∞ for all η ∈ (0, η) under (iv) by using (6.5.9) while under (v), we have I 2 (η ) = 0 for all η ∈ (0, η) since in this case h(λ) = h(0) so that, in (6.5.8), k(λ) = 0 for all λ ∈ (-η, η) (thus implying s = 0). This concludes the proof.

Proofs of Section 6.4.3

Proof of Lemma 6.4.6. The statement in i.i.d. case is exactly D üker (2018, Lemma A.1). The convergence in M(Ω, F , H 0 , P) follows from the proof of D üker (2018, Lemma A.1), which continues to hold under the weaker assumption that ( k ) k∈Z is a white noise.

Proof of Lemma 6.4.7. Since is a white noise, Assertion (v) of Theorem 6.4.5 holds. The result follows since the conditions in (6.4.13) imply Assertions (ii) and (iii) of Theorem 6.4.5 with and D = Id H 0 -N.

The proof of Proposition 6. Proof. By assumption on (a n ), ∑ a n z n is a power series valued in E with convergence radius at least equal to 1, hence is uniformly continuous on the open disk with radius 1. When |z 0 | = 1, the result follows using Abel's transform.

Lemma 6.5.4. Let H 0 be a separable Hilbert space, N ∈ L b (H 0 ) be a normal operator with singular value function n on G 0 := L 2 (V, V, ξ) and decomposition operator U. Define = ξ-essinf (6.5.16) where the two infinite sums on the right-hand side are L b (H 0 )-valued power series with convergence radius at least equal to 1. Moreover, if > 0, then Eq. (6.5.16) continues to hold for all z ∈ D \ {1} with the two infinite sums converging in L b (H 0 ).

v∈V (n(v)) . Then there exist C ∈ L b (H 0 ) and (∆ k ) k∈N ∈ L b (H 0 ) N with ∆ k L b (H 0 ) = O k -1-such that, for all z ∈ D, (1 -z) N-Id = C ∞ ∑ k=0 (k + 1) -N z k + ∞ ∑ k=0 ∆ k z k ,
Proof. The proof is three steps. We first show Relation (6.5.16) for all z ∈ D,

then that ∆ k L b (H 0 ) = O k -1-and finally extend the relation to z ∈ D \ {1} when > 0. Step 1. Let z ∈ D, then (1 -z) N-Id = Id + ∑ k≥1 N k z k ,
where for all k ≥ 1,

N k = ∏ k j=1 Id -N j . Let k 0 ≥ 1, such that N L b (H 0 ) /k 0 < 1 and take k ≥ k 0 , then Id - N k = exp ln Id - N k = exp -∑ j≥1 N j k j j ,
and therefore,

N k = k 0 -1 ∏ j=1 Id - N j exp -∑ j≥1 N j j k ∑ t=k 0 1 t j .
Moreover, we have the following asymptotic expansions,

k ∑ t=k 0 1 t = k ∑ t=1 1 t - k 0 -1 ∑ t=1 1 t = ln(k + 1) + γ e - k 0 -1 ∑ t=1 1 t + α k k
and for all j ≥ 2,

k ∑ t=k 0 1 t j = +∞ ∑ k=k 0 1 t j - +∞ ∑ k=k+1 1 t j = β j k j 0 + η k,j (j -1)k j-1
where γ e is Euler's constant and (α k ) k≥1 , (η k,j ) k≥1,j≥2 such that sup k≥1 |α k | < +∞ and sup k≥1,j≥2 η k,j < +∞ and β j = ∑ +∞ t=k 0 k 0 t j satisfies sup j≥2 β j < +∞. This gives, for all k ≥ k 0 ,

N k = C(k + 1) -N exp -N α k k -∑ j≥2 N j η k,j (j -1)k j-1 where C = k 0 -1 ∏ j=1 Id - N j exp -N γ e - k 0 -1 ∑ t=1 1 t exp -∑ j≥2 N k 0 j β j j .
Combining everything, we get

(1 -z) N-Id = Id + k 0 -1 ∑ k=1 k ∏ j=1 Id - N j z k + C ∑ k≥k 0 (k + 1) -N exp -N α k k -∑ j≥2 N j η k,j (j -1)k j-1 z k
which leads to Relation (6.5.16) with

∆ 0 = Id -C , ∆ k = k ∏ j=1 Id - N j -C(k + 1) -N , for all 1 ≤ k ≤ k 0 -1, ∆ k = C(k + 1) -N exp -N α k k -∑ j≥2 N j η k,j (j -1)k j-1 -Id , for all k ≥ k 0 .
Step 2. For all k ≥ k 0 , denoting by

Φ k := -N α k k -∑ j≥2 N j η k,j
(j-1)k j-1 , we get

∆ k L b (H 0 ) = C(k + 1) -N e Φ k -Id L b (H 0 ) ≤ C L b (H 0 ) (k + 1) -N L b (H 0 ) ∑ t≥1 Φ k t L b (H 0 ) t! = O k -1-,
where we used that

Φ k L b (H 0 ) ≤ N L b (H 0 ) |α k | k + ∑ j≥2 N j L b (H 0 ) η k,j (j -1)k j-1 = N L b (H 0 )   |α k | k + ∑ j≥1 N j L b (H 0 ) jk j η k,j+1   = O k -1 ,
and that (k

+ 1) -N L b (H 0 ) = (k + 1) -M n L b (H 0 ) = M (k+1) -n L b (H 0 ) = ξ-essup v∈V (k + 1) -n(v) = (k + 1) -.
Step 3. We now assume > 0 and extend (6.5.16) to D \ {1}, that is to the case z = e -iλ for some λ ∈ T \ {0}. For such a λ, we already have, for all 0 < a < 1,

(1

-ae -iλ ) N-Id = C ∑ k≥0 (k + 1) -N a k e -iλk + ∑ k≥0 ∆ k a k e -iλk . Moreover, (1 -e -iλ ) N-Id = lim a↑1 (1 -ae -iλ ) N-Id by continuity of z → (1 -z) N-Id in D \ {1} and ∑ k≥0 ∆ k e -iλk = lim a↑1 ∑ k≥0 ∆ k a k e -iλk because ∑ k≥0 ∆ k L b (H 0 ) < +∞.
It remains to show that ∑ k≥0 (k + 1) -N z is well defined on U \ {1} and that, for λ ∈ T \ {0}, ∑ k≥0 (k + 1) -N a k e -iλk converges to ∑ k≥0 (k + 1) -N e -iλk as a ↑ 1, which we prove at once by applying Lemma 6.5.3. For all k ∈ N, we have

(k + 1) -N L b (H 0 ) = ξ-essup v∈V (k + 1) -n(v) = (k + 1) -,
Since > 0, we get that (k + 1) -N L b (H 0 ) → 0 as k → ∞. Hence, to apply Lemma 6.5.3 it only remains to show

∑ k∈N (k + 1) -N -(k + 2) -N L b (H 0 ) < ∞ .
(6.5.17)

Note that we have, for all k ∈ N, v) . (6.5.18) Moreover, for all k ∈ N, and ξa.e. v ∈ V, since (n(v)) ≥ > 0, we have

(k + 1) -N -(k + 2) -N L b (H 0 ) = ξ-essup v∈V (k + 1) -n(v) -(k + 2) -n(
(k + 1) -n(v) -(k + 2) -n(v) = |k + 1| -(n(v)) 1 -exp -ln 1 + 1 k + 1 n(v) ≤ ς α(ς ln(2)) (k + 1) -ln 1 + 1 k + 1 ,
where we set ς := ξ-essup |n| and, for any r > 0,

α(r) := sup 1 -e -z z : z ∈ C 0 < |z| ≤ r .
This leads to the asymptotic bound, as k → ∞,

ξ-essup v∈V (k + 1) -n(v) -(k + 2) -n(v) = O (k + 1) --1 .
Hence, with (6.5.18) and the assumption > 0, we obtain (6.5.17) and Step 3 is completed.

Proof of Proposition 6.4.8. The processes Y and F FI D ( ) are well defined by Lemma 6.4.6 and Lemma 6.4.7 respectively. Moreover, the first condition in (6.4.13) gives ≥ 1/2 in Lemma 6.5.4 which therefore implies that there exists

C ∈ L b (H 0 ) and (∆ k ) k∈N ∈ L b (H 0 ) N with ∆ k L b (H 0 ) = O(k -3/2 ) (in particular ∑ +∞ k=0 ∆ k L b (H 0 ) < +∞) such that, for all λ ∈ T \ {0}, (1 -e -iλ ) N-Id = C ∞ ∑ k=0 (k + 1) -N e -iλk + ∞ ∑ k=0
∆ k e -iλk in L b (H 0 ) , (6. 5.19) thus concluding the proof.

Part III

M U LT I -S I T E S E L E C T R I C A L L O A D D I S A G G R E G AT I O N A N D C L U S T E R I N G
In Part iii, we propose to address the practical objective using a non-negative tensor factorization model for multi-sites electrical load curves disaggregation. This model is presented in Chapter 7, where we derive updates for the corresponding optimization problem and show on concrete examples how this formulation is helpful for exhibiting smooth intraday consumption patterns and taking into account external variables such as the outside temperature. This model satisfies the industrial expectations and is currently being implemented in a web interface to be used as an analysis tool at the TREE department of EDF R&D. Chapter 8 we study the problem of non-negative tensor factorization from a theoretical point of view. In particular, since missing data and smoothness are naturally present in the model introduced in Chapter 7, we study the link between these two characteristics and their effect on the existence of a global optimum for the resulting minimization problem. More precisely, we show that, contrarily to the case where all entries are observed, the nonnegative tensor factorization problem does not necessarily have a global optimum if some entries are missing. However, we show that adding smoothness constraints can guarantee the existence of such an optimum. Both the derivation of the constraint and the resulting optimization problem presents several algorithmic challenges which are addressed in this chapter.

I N T R O D U C T I O N A N D M O T I VAT I O N
The analysis of load curves collected from smart meters is a key step for many energy management tasks ranging from consumption forecasting to customers characterization and load monitoring. (see Wang, Chen, Hong, and Kang, 2019 for a recent review). Among these tasks, in the context of my PhD, EDF was interested in addressing the practical objective (PO). This objective can be seen a jointly performing two popular tasks from energy load curves analysis, namely load curves comparison and load curves disaggregation, in a multi-sites setting. In fact, these two tasks correspond to specific instances of the time series comparison and representation discussed in Chapter 1. The goal of load curves curves is, in general, clustering (for example for customers characterization). In this context, the meaning of the clusters themselves is more important than the characteristics of the consumption. On the other hand, load curve disaggregation aims at decomposing a load curve as a sum of curves each representing a particular consumption pattern. A popular example is Non Intrusive Load Monitoring (NILM) which was introduces in Hart, 1992 and where each consumption pattern is linked to one several devices present in the building. The two following paragraphs provide a brief overview of the methods used in the literature for these tasks.

comparing load curves.

The problem of comparing a panel of load curves

X t (u) , u ∈ U , t = 1, • • • , T ,
where U is an interval of R (e.g. U = [0, 24) for daily load curves) can be found in both the mono-site and multi-site contexts depending on the meaning given to the index t. In both cases we aim at regrouping the load curves with similar consumption patterns into one cluster. Each cluster then represents a particular consumption behavior. In the mono-site context, t usually represents the day and we aim at assigning each day to a specific cluster, which corresponds to a particular typical daily consumption behavior, (see e.g. [START_REF] Antoniadis | Clustering Functional Data Using Wavelets[END_REF][START_REF] Bouveyron | The functional latent block model for the co-clustering of electricity consumption curves[END_REF][START_REF] Richard | Daily load profiles clustering: a powerful tool for medium-sized industries demand side management[END_REF]. In the multi-site context, each t usually represents a site (or customer) as in [START_REF] Chicco | Comparisons among clustering techniques for electricity customer classification[END_REF][START_REF] Chicco | Overview and performance assessment of the clustering methods for electrical load pattern grouping[END_REF][START_REF] Wang | Load profiling and its application to demand response: A review[END_REF] In this case, if multiple load curves are observed for each site, they are aggregated to a unique "representative" load curve for each site. It is also possible to see the index t as representing both the site and a time period (days, seasons, week day) as in [START_REF] Mcloughlin | A clustering approach to domestic electricity load profile characterisation using smart metering data[END_REF][START_REF] Yilmaz | Comparison of clustering approaches for domestic electricity load profile characterisation -Impli-cations for demand side management[END_REF].

Regardless of the meaning given to the index t, this problem can be placed in the framework of time series comparison and therefore we find approaches based on the strategies discussed in Chapter 1. For example, [START_REF] Chicco | Comparisons among clustering techniques for electricity customer classification[END_REF][START_REF] Chicco | Overview and performance assessment of the clustering methods for electrical load pattern grouping[END_REF][START_REF] Mcloughlin | A clustering approach to domestic electricity load profile characterisation using smart metering data[END_REF][START_REF] Tsekouras | A pattern recognition methodology for evaluation of load profiles and typical days of large electricity customers[END_REF][START_REF] Wang | Load profiling and its application to demand response: A review[END_REF] use clustering methods directly on the raw observations while [START_REF] Albert | Smart Meter Driven Segmentation: What Your Consumption Says About You[END_REF][START_REF] Azad | Identification of typical load profiles using K-means clustering algorithm[END_REF][START_REF] Notaristefano | Data size reduction with symbolic aggregate approximation for electrical load pattern grouping[END_REF][START_REF] Yilmaz | Comparison of clustering approaches for domestic electricity load profile characterisation -Impli-cations for demand side management[END_REF] use hand-designed or model-based features. Dimension reduction methods have also been used (e.g. [START_REF] Chicco | Comparisons among clustering techniques for electricity customer classification[END_REF]. More recently, deep learning approaches [START_REF] Arechiga | Comparison of Dimensionality Reduction Techniques for Clustering and Visualization of Load Profiles[END_REF][START_REF] Ryu | Residential Load Profile Clustering via Deep Convolutional Autoencoder[END_REF][START_REF] Varga | Robust Real-Time Load Profile Encoding and Classification Framework for Efficient Power Systems Operation[END_REF] and functional data clustering [START_REF] Bouveyron | The functional latent block model for the co-clustering of electricity consumption curves[END_REF][START_REF] Dasgupta | Clustering Household Electrical Load Profiles Using Elastic Shape Analysis[END_REF][START_REF] Teeraratkul | Shape-Based Approach to Household Electric Load Curve Clustering and Prediction[END_REF] have been proposed.

load curves disaggregation.

We formulate the problem of load curves disaggregation as a decomposition

X t (u) ≈ R ∑ r=1 x (r) t (u) ,
where u ∈ U represents time (e.g. the intra-day time if U = [0, 24)) and t represent the observation (e.g. the sensor but also the day or the site in other contexts). In NILM, the variable x (r) t represents the load curve of the r-th device for the given observation t and, again, several methods have been proposed, from hand-designed methods (e.g. [START_REF] Dong | Non-Intrusive Signature Extraction for Major Residential Loads[END_REF]Hart, 1992;[START_REF] Paris | NilmDB: The Non-Intrusive Load Monitor Database[END_REF] or Hidden Markov Models (e.g. [START_REF] Aiad | Unsupervised approach for load disaggregation with devices interactions[END_REF][START_REF] Egarter | PALDi: Online Load Disaggregation via Particle Filtering[END_REF]Hart, 1992;[START_REF] Kim | Unsupervised Disaggregation of Low Frequency Power Measurements[END_REF] to recent deep learning models of [START_REF] Devlin | Non-Intrusive Load Monitoring using Electricity Smart Meter Data: A Deep Learning Approach[END_REF][START_REF] Huber | Review on Deep Neural Networks Applied to Low-Frequency NILM[END_REF][START_REF] Kelly | Neural NILM: Deep Neural Networks Applied to Energy Disaggregation[END_REF][START_REF] Singh | Deep Sparse Coding for Non-Intrusive Load Monitoring[END_REF]. Over the past decade, blind source separation methods such as Nonnegative matrix factorization (NMF) or sparse coding have gained in popularity in the NILM community (see [START_REF] Elhamifar | Energy Disaggregation via Learning 'Powerlets' and Sparse Coding[END_REF][START_REF] Figueiredo | On the Regularization Parameter Selection for Sparse Code Learning in Electrical Source Separation[END_REF][START_REF] García | Interactive visualization for NILM in large buildings using non-negative matrix factorization[END_REF][START_REF] Henriet | Independent-Variation Matrix Factorization With Application to Energy Disaggregation[END_REF][START_REF] Kolter | Energy Disaggregation via Discriminative Sparse Coding[END_REF][START_REF] Lange | BOLT: Energy Disaggregation by Online Binary Matrix Factorization of Current Waveforms[END_REF][START_REF] Miyasawa | Energy disaggregation based on smart metering data via semi-binary nonnegative matrix factorization[END_REF][START_REF] Rahimpour | Non-Intrusive Energy Disaggregation Using Non-Negative Matrix Factorization With Sum-to-k Constraint[END_REF]. The idea behind NMF for NILM is to decompose the load curve of the r-th device as a signature curve a r (u) ≥ 0 which is modulated across observations by an activation b t,r ≥ 0. The full model becomes

X t (u) ≈ R ∑ r=1 x (r) t (u) = R ∑ r=1 a r (u)b t,r ,
which corresponds to the pattern-based model of (1.2.1) and is a functional formulation of the usual NMF.

In a multi-site context, disaggregating the load curves can be a way to extract features to describe the load profiles and compare the sites, thus achieving the practical objective (PO). As discussed in Section 3.2, it is interesting to rely on representations with common loadings. An advantage of the NMF approach is that, it has a natural multi-sites extension which satisfies this property. This extension is called Nonnegative Tensor Factorization (NTF). In our context, assuming we observe a panel of daily load curves {X n,t (u) : u ∈ [0, 24), t ∈ 1, T , n ∈ 1, N } where u represents the intra-day time, j the observed day and n the site, a functional formulation of the classical NTF model writes as

X n,t (u) ≈ R ∑ r=1 a r (u)b t,r c n,r , (iii.1)
where a r (u) ≥ 0 is the signature which is modulated across days by the day activation b t,r ≥ 0 and across sites by the site activation c n,r ≥ 0. The signature and activations are refered to as the factors.

When the constraints of positivity are relaxed, the decomposition (iii.1) is know as the CANDECOMP/PARAFAC (CP) decomposition (see [START_REF] Carroll | Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition[END_REF][START_REF] Harshman | Foundations Of The Parafac Procedure: Models And Conditions For An" Explanatory" Multimodal Factor Analysis[END_REF][START_REF] Hitchcock | The Expression of a Tensor or a Polyadic as a Sum of Products[END_REF]. The CP decomposition and NTF are very popular is various domains ranging from chemometrics and psychometrics to signal processing and machine learning (see [START_REF] Cichocki | Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-Way Data Analysis and Blind Source Separation[END_REF][START_REF] Kolda | Tensor Decompositions and Applications[END_REF][START_REF] Sidiropoulos | Tensor Decomposition for Signal Processing and Machine Learning[END_REF] and the references therein). One of the strengths of these models is that they can often be adapted to take into account missing data and/or prior knowledge on the data such as non-negativity, sparsity or smoothness. This flexibility explains their popularity in real worlds applications for which interpretability of the model is necessary. For these reasons, these models seem appropriate to address the practical objective (PO). However, their use in electrical load curves analysis is still recent. For example, [START_REF] Figueiredo | Exploring the performance of non-negative multi-way factorization for household electrical seasonal consumption disaggregation[END_REF][START_REF] Figueiredo | Analysis of Trends in Seasonal Electrical Energy Consumption via Non-Negative Tensor Factorization[END_REF] use NTF to model the consumption of several devices in order to perform NILM in a supervised setting. Recently Sandoval, Barocio, Korba, and Sevilla, 2020 proposed to use PARAFAC for multi-sites load curves disaggretation in an unsupervised setting. The authors provide several applications such as data storage, data completion and, in particular, clustering of the sites by clustering the vectors of site activations, i.e. c n = (c n,1 , • • • , c n,R ) . However, they do not give any interpretation of the factors nor the clusters. In Chapter 7, we propose several modifications of the NTF model (iii.1) in order to orient the decomposition into more interpretable factors. Following the conclusions of Chapter 2, we also use additional knowledge on the outside temperature and different consumption regimes. The optimization problem related to our model is expressed as a usual NTF problem with weights and an algorithm is proposed to get an estimated solution of this problem.

S M O O T H N O N -N E G AT I V E T E N S O R FA C T O R I Z AT I O N F O R M U LT I -S I T E S E L E C T R I C A L L O A D M O N I T O R I N G
In this chapter, we introduce a model based on a functional formulation of non-negative tensor factorization (NTF) and derive updates for the corresponding optimization problem. We show on the example of multi-sites load curves disaggregation how this formulation is useful to address the practical objective (PO) by 1) exhibiting smooth intraday consumption patterns and 2) taking into account external variables such as the outside temperature. The benefits are demonstrated on simulated and real data by exhibiting a meaningful clustering of the observed sites based on the obtained decomposition. Throughout this chapter, we assume that we observe a dataset {(X n,t (u i ), T n,t , n,t ) : i ∈ 1, I , n ∈ 1, N , t ∈ 1, T } where (u i ) i∈ 1,I ∈ [0, 24) I are sampling points, and X n,t : u → X n,t (u), T n,t ∈ R and n,t ∈ 1, E respectively represent the daily load curve, the average external temperature and the consumption regime for the site n ∈ 1, N and day t ∈ 1, T . For example, for EDF's data, the regimes the opening and closing days computed in Section 2.3.

Proposed model

Based on the observations of Section 2.6, we use the a priori assumption that the dependence between X n,t (u) and t is fully explained by the variables T n,t and n,t . With this assumption in mind, we propose to replace the NTF model of (iii.1) by

X n,t (u) ≈ R ∑ r=1 a r (u)b r (T n,t )c ( n,t ) n,r , (7.1.1)
where a r (u) ≥ 0, b r (τ) ≥ 0, c ( ) n,r ≥ 0. We also assume that the functions a r and b r are smooth and that a r is periodic with a period of 24 hours (since it represents intra-day behavior). By analogy to the NMF and NTF cases we call the functions b r the thermal activations.

In order to estimate the functions a r and b r of (7.1.1) from discrete data, {X n,t (u i ) : i ∈ 1, I , t ∈ 1, T , n ∈ 1, N }, we need to rely on smoothing as discussed in Section 1.3.2. To this end, two approaches are possible: the basis approach and the penalization approach.

the basis approach.

In the basis approach, we express a r in a an adapted functional basis (v 1 , • • • , v K ) as in (1.3.2). Namely, we write

a r (u) = K ∑ k=1 α k v k (u) , (7.1.2)
and estimate the coefficients

vector α = [α 1 , • • • , α K ] .
In this framework, two options are usually used to respect the non-negativity constraints.

• The first option is to estimate α such that a r (u i ) ≥ 0 for all i ∈ 1, I . From (7.1.2), this is a linear constraint in α, but it only ensures that the function a r is non-negative at the sample points. This approach is followed in Hautecoeur and Glineur, 2021;[START_REF] Sadowski | Image Completion with Smooth Nonnegative Matrix Factorization[END_REF][START_REF] Zdunek | Alternating direction method for approximating smooth feature vectors in Nonnegative Matrix Factorization[END_REF][START_REF] Zdunek | Approximation of Feature Vectors in Nonnegative Matrix Factorization with Gaussian Radial Basis Functions[END_REF] • The second option consists in taking a basis of non-negative functions such as B-Splines and constraining the coefficients of the function to be non-negative. Namely, we assume that v k (u) ≥ 0 for all u ∈ [0, 24) and k ∈ 1, K and search for α ∈ R K + . This approach is followed in e.g. [START_REF] Yokota | Smooth nonnegative matrix and tensor factorizations for robust multi-way data analysis[END_REF][START_REF] Zdunek | B-Spline Smoothing of Feature Vectors in Nonnegative Matrix Factorization[END_REF] and ensures that a r (u) ≥ 0 for all u ∈ [0, 24). However, in the case where (v k ) k∈ 1,K is a B-spline basis, [START_REF] Boor | Splines with Nonnegative B-Spline Coefficients[END_REF] showed that estimating a non-negative function with this approach may not be optimal.

It should be noted that, recently, Hautecoeur and Glineur, 2020 proposed an algorithm which ensures that a r (u) ≥ 0 for all u ∈ [0, 24) without requiring that α ∈ R K + . Their method is based on an optimal non-negative projection onto the space Span (v 1 , • • • , v K ) in the case where the v k 's are B-splines. the penalization approach.

In the penalization approach, we add a penalty term P (a r ) to the loss which is used to estimate the factors in (7.1.1) (e.g. the mean square loss). Classical examples of penalty ensuring smoothness are the total variation norm P (a r ) = a r p TV,p = |a r | p (usually for p = 1, 2) and the L 2 -norm of the second derivative P (a r ) = |a r | 2 . The former penalty is approximated by a Riemann integral and the latter presents the advantage of implying that a r is a spline function as recalled in Section 1.3.2 which becomes entirely characterized by the points (a r (u i )) i∈ 1,I . The downside of this penalized approach is that, there is no simple method to ensure that a r is non-negative everywhere and we have to settle for non-negativity at the sampling points. This approach is followed in e.g. Cichocki and Phan, 2009;[START_REF] Cichocki | Flexible Component Analysis for Sparse, Smooth, Nonnegative Coding or Representation[END_REF][START_REF] Essid | Smooth Nonnegative Matrix Factorization for Unsupervised Audiovisual Document Structuring[END_REF][START_REF] Henriet | Independent-Variation Matrix Factorization With Application to Energy Disaggregation[END_REF] As explained in the above paragraphs, depending on the method used, we can obtain an estimate of a r which is either non-negative everywhere or non-negative only at the sampling points. The latter is, in general, sufficient all the more so as the smoothness of the curves may ensure that they do not vanish in the neighborhoods of the sampling points. It is also possible to add artificial sampling points and treat them as missing values in the optimization problem. For these reasons, we focus on a penalized approach expressed as the optimization problem proposed in the next section. Basis approaches are left to future work.

The optimization problem

Let us consider two grids (u 1 , • • • , u I ) ⊂ [0, 24) and (τ 1 , • • • , τ K ) ⊂ R which contain all the observed intra-day times and temperatures. Let also E be the number of consumption regimes so that n,t ∈ 1, E for all n ∈ 1, N and t ∈ 1, T . Then the factors a r , b r , c ( ) r , are estimated by solving the following penalized least square problem. min {F + P} , under the constraint that, for all r, i, k, n, , (7.1.3)

                   a r (u i ) ≥ 0 , b r (τ k ) ≥ 0 , c ( ) n,r ≥ 0 a r (0 + ) = a r (24 -) a r (0 + ) = a r (24 -) a r (0 + ) = a r (24 -) where F = ∑ i,j,n X n,t (u i ) - R ∑ r=1 a r (u i )b r (T n,t )c ( n,t ) n,r 2 , P = α R ∑ r=1 24 0 (a r ) 2 + β R ∑ r=1 τ K τ 1 (b r ) 2 .
The scaling constraints prevent the factors from diverging since the error F is not affected by multiplying one of the factors by a constant as soon as the other factors are scaled accordingly. These scaling constraints also make sure that we can compare the site activations.

Finally, the penalizations on the L 2 -norm of the second derivatives imply that the solutions of Problem (7.1.3) are necessarily smooth spline functions. More precisely, for all r ∈ 1, R , the function a r must be a 24-periodic cubic spline and the function b r must be a natural cubic spline. Since spline functions are characterized by their sample points, we can reformulate the problem as a weighted NTF.

Formulation as a weighted NTF problem

We reformulate Problem (7.1.3) taking advantage of the fact that spline functions are entirely characterized by their sample points. Classical results on spline (see e.g. [START_REF] Green | Nonparametric regression and generalized linear models[END_REF] imply that that there exist v 1 ∈ R I and Q 1 ∈ R I×I positive definite such that for any 24-period cubic spline a on [0, 24), we have Similarly there exist v 2 ∈ R K and Q 2 ∈ R K×K positive definite such that for any cubic spline b on [τ 1 , τ K ], we have

τ K τ 1 b = v 2 b and τ K τ 1 (b ) 2 = b Q 2 b , (7.1.5) with b = [b(τ 1 ), • • • , b(τ K )]. Now, let us define W ∈ R I×K×EN , X ∈ R I×K×EN by W i,k,( -1)N+n = T ∑ t=1 1 {T n,t =τ k } 1 { n,t = } 1/2
, and such that for all r ∈ 1, R , (7.1.6)

X i,k,( -1)N+n = ∑ T t=1 1 {T n,t =τ k } 1 { n,t = } X n,t (u i ) W 2 i,k,( -
v 1 a r = v 2 b r = 1 where f W (A, B, C) = L W (A, B, C) + αTr(A Q 1 A) + βTr(B Q 1 B) with L W (A, B, C) = W X - R ∑ r=1 a r • b r • c r 2 2 , (7.1.7)
and (v 1 , Q 1 ), (v 2 , Q 2 ) are taken as in (7.1.4) and (7.1.5) respectively.

Proof. First, we know, from the penalization terms of (7.1.3), that the solution must be such that a r and b r are 24-periodic and natural cubic splines respectively. Then, defining

A = [a 1 , • • • , a R ] ∈ R I×R + , B = [b 1 , • • • , b R ] ∈ R K×R + and C = [c 1 , • • • , c R ] ∈ R EN×R + by A i,r = a i,r = a r (u i ), B k,r = b k,r = b r (t k ), C ( -1)N+n,r = c ( -1)N+n,r = c n,r ,
we get that (7.1.3) reduces to (7.1.6) where, L is replaced by L defined as

L(A, B, C) + αTr(A Q 1 A) + βTr(B Q 2 B) , with L(A, B, C) = ∑ i,t,n X n,t (u i ) - K ∑ k=1 E ∑ =1 1 {T n,t =τ k } 1 { n,t = } A, B, C i,k,( -1)E+n 2 , where A, B, C = ∑ R k=1 a r • b r • c r .
To conclude the proof, we now show that L is equal to L up to an additive constant which does not depend on A, B, C. Noticing that

K ∑ k=1 E ∑ =1 1 {T n,t =τ k } 1 { n,t = } = 1 , we get L(A, B, C) = ∑ i,t,n K ∑ k=1 1 ∑ =0 1 {T n,t =τ k } 1 { n,t = ) X n,t (u i ) -A, B, C i,k,( -1) K+k,n 2 = 1 ∑ =0 ∑ i,k,n T ∑ t=1 1 {T n,t =τ k } 1 { n,t = ) X n,t (u i ) 2 + 1 ∑ =0 ∑ i,k,n W 2 i,k,( -1)E+n A, B, C 2 i,k,( -1)E+n -2 1 ∑ =0 ∑ i,k,n W 2 i,k,( -1)E+n X i,k,( -1)E+n A, B, C i,k,( -1)E+n = C(X) + L(A, B, C) ,
where C(X) is a constant which depends only on X. This concludes the proof.

HALS algorithm

Problem is not convex in (A, B, C) and many approaches can be found in the literature to provide an estimated local minimum, in general, in the form of an iterative procedure. In this section, we propose a Hierarchical Alternating Least Squares (HALS) methods which consists in minimizing f W alternatively in the columns of A, B, C using the fact that L W (A, B, C) in (7.1.7) writes, for all r ∈ 1, R , as

L W (A, B, C) = W (X (r) -a r • b r • c r ) 2 2 ,
where

X (r) := X -∑ s =r a s • b s • c s .
Assuming that all variables are fixed except a r for some r ∈ 1, R , we write L W (A, B, C) as a quadratic function applied to a r . Namely,

L W (A, B, C) = a r M r a r + 2u r a r + W X (r) 2 2 ,
where we have defined M r := diag(W 2

(1) (c r ⊗ b r ) 2 ) ∈ R I×I and u r := -W 2

(1)

X (r) (1) (c r ⊗ b r ) ∈ R I .
With this formulation, the update for a r writes as a r = argmin a∈R I + , v a=1 1 2 a (M r + αQ 1 )a + u r a , which can be solved using quadratic programming. However, it is common in the matrix and tensor factorization community to give an approximated solution where we solve the unconstrained problem, take the positive part and then rescale the solution. With this approximation, the update follows the two steps

a r ← -(M r + αQ 1 ) -1 u r a r ← [a r ] + v 1 [a r ] + ,
where [a] + is the vector in R I + whose i-th entry is max(a i , 0) (in practice we usually take max(a i , ) for a small > 0).

We can now derive the HALS for the optimization problem (7.1.6) as provided in Algorithm 7.1.1.

Algorithm 7.1.1: HALS algorithm for Problem (7.1.6).

Data: X, W, Q 1 , Q 2 , v 1 , v 2 and initial values for A, B, C E = X -∑ R r=1 a r • b r • c r repeat for r = 1, • • • , R do X (r) = E + a (1) r • • • • • a (N) r /* update a r */ a r = diag(W 2 (1) (c r ⊗ b r ) 2 ) + αQ 1 -1 W 2
(1)

X (r) (1) (c r ⊗ b r ) a r = [a r ] + v 1 [a r ] + /* update b r */ b r = diag(W 2 (2) (c r ⊗ a r ) 2 ) + βQ 2 -1 W 2
(2)

X (r) (2) (c r ⊗ a r ) b r = [b r ] + v 2 [b r ] + /* update c r */ c r = diag(W 2 (3) (b r ⊗ a r ) 2 -1 W 2
(3)

X (r) (3) (b r ⊗ a r ) 10 c r = [c r ] + /* update residuals */ 11 E = X (r) -a (1) r • • • • • a (N) r

Experimental results

We validate our model on two datasets. The first was extracted from energy demand data simulated for 1 year and with hourly rate by the Office of Energy Efficiency & Renewable Energy (EERE) 1 . We took a total of 775 sites in California, Arizona, Nevada, Utah, Oregon, Idaho and Washington, gathering 5 different building types. The second dataset consists of energy demand collected by EDF from 108 supermarkets across France over a period of 1 year with a sample rate of 10 minutes. The average external temperature of each day is observed for each site 2 , EERE's data have one consumption regime and EDF's data have two consumption regimes (closing and opening days). Because of space constraints, we will not discuss the detection of regimes. In order to compare sites of different sizes we scale the observed load curves by the average daily consumption.

For both NTF and our model, we use the Fast HALS updates and the algorithm is stopped when the relative improvement of the loss reaches 10 -5 . Each factors was initialized by taking the positive part of the singular vectors of the corresponding unfolding of X (e.g. a r = [φ r ] + where φ r is the r-th singular vector of X (1) ). Since there is no foolproof method to select the number of components R (see [START_REF] Bro | A new efficient method for determining the number of components in PARAFAC models[END_REF][START_REF] Ceulemans | Selecting among three-mode principal component models of different types and complexities: A numerical convex hull based method[END_REF][START_REF] Timmerman | Three-mode principal components analysis: Choosing the numbers of components and sensitivity to local optima[END_REF] for ad-hoc methods) and since cross-validation is not straightforward with tensor data (see [START_REF] Bro | Crossvalidation of component models: A critical look at current methods[END_REF][START_REF] Owen | Bi-cross-validation of the SVD and the nonnegative matrix factorization[END_REF], we take R = 6 and α = β = 3000 for EERE's data and α = β = 100 for EDF's data. This choice gives a good balance between goodness of fit and interpretation of the factors. We also found empirically that it helps to start a few iterations of the algorithm with a smaller value of the penalization to get closer to a local optimum. For clustering, we run K-means on the sites activations for all regimes. This means that site n is represented by the feature vector (c

(1) n,1 , • • • , c (1) n,R , • • • , c (E) n,1 , • • • , c (E) n,R
) where we recall that E is the number of regimes.

1 The data is available at https://openei.org/doe-opendata/dataset/commercial-and -residential-hourly-load-profiles-for-all-tmy3-locations-in-the-united-states.

2 EERE's data can be obtained from the TMY3 weather stations using the eeweather Python package.

Results on EERE's data

The factors obtained by NTF and our model are represented in Figure 7.1 where the colors correspond to the building types. The dependence on the temperature is justified by the fact that the day activations of NTF mainly indicate season changes. Moreover, the advantage of smoothing the signatures is that only the most important peaks are kept, which is valuable for interpretation. The radar plots of the site activations presented in Figure 7.2 show a better separation of the building types with our model (especially between the restaurants). To quantify this observation, we ran K-means with 5 clusters and compared the clusters with the true labels using the adjusted random index. Our model gives a perfect fit with an adjusted random index of 1 compared to 0.75 for NTF which mostly fails to separate the two types of restaurants. In our model, both hotels have a high site activation in Component 2 whose signature is typical of a hotel (high for breakfast and dinner and medium for lunch). Small Hotels tend to heat at night (Component 4) while Large Hotels tend to heat during the day (Component 1). Apartments have a high site activation in Component 4 whose signature and thermal activation are typical of heating in residential buildings. Components 5 and 6 also characterize the Apartments and can be interpreted as holidays and working days respectively. Indeed, the thermal activation of Component 5 is high when it is very cold and very hot (winter and summer holidays) where people are more at home in the middle of the day (where the signature is high) while the thermal activation of Component 6 peaks for medium temperatures and its signature presents a typical working day profile. Finally, the signature of Component 3, which peaks before lunch and dinner (and a bit before breakfast), is characteristic of restaurants. The difference between the two types of restaurants is well explained by the factorization. Indeed, Component 3 is larger for Quick Service Restaurants, thus indicating that they usually present more activity before lunch time. On the other had, Component 1 is larger for Full Service Restaurant. In particular, this indicates a high activity before dinner and larger use of heating. Finally, contrary to Quick Service Restaurants, we observe that Component 5 has some weight in Full Service Restaurants. This is consistent with the interpretation of this component as holidays during which people have more lunch time and therefore may go to bigger restaurants.

Results on EDF's data

The results obtained for EDF's data are presented in Figure 7.3 where the colors correspond to the clusters obtained by K-means. The number of clusters was selected by the Davies-Bouldin index (between 2 and 9 clusters).

The two consumption regimes can be seen in the day activations of NTF which justifies taking them into account in our model. The site activations are not very diverse in NTF and the two clusters obtained mainly differ from each other by Components 1 and 5 which respectively represent the standard profiles of opening and closing days. On the contrary, our model seems to extract more variable site activations (see also Figure 7.4) thus exhibiting more clusters. The interpretation of the components is also easier because of smoothness and of the dependence on temperature. We observe 3 heating profiles (Components 1, 3, 6) and 3 cooling profiles (Components 2, 4, 5). For both heating and cooling we observe one component which characterizes the closing days: Component 1 for heating and Component 5 for cooling. These closing days components have similar signature which peaks when the supermarket is closed, thus indicating a smaller difference between opening and closing hours as it should be expected for closing days. The signature of Component 3 is characteristic of heaters since it peaks in the morning and then slowly decays because of inertia. The signature and thermal activation of Component 4 indicate that it represents air conditioning which is activated for high temperatures and in the middle of the day. Finally, the thermal activation of Component 2 indicates that it represents food refrigeration which is activated for lower temperatures than air conditioning and reaches a saturation level for high temperatures.

To interpret the cluster, it is more convenient to look at the cluster centers whose radar plots are displayed in Figure 7.4b. By comparing the radar plots, we propose the following interpretation. Cluster 1: high heating (Component 3) and air conditioning (Component 4) profiles. Cluster 2: medium heating profile (Component 6). Cluster 3: High heating (Component 3) and food refrigeration (Component 2) and small impact of air conditioning (Component 4). Cluster 4: Small difference between opening and closing days (Components 1 and 5) and small impact of air conditioning (Component 4).

Giving a meaning to the factors and clusters is highly valuable for monitoring or maintenance purpose. For example, for high temperatures, sites in Cluster 1 should reduce their use of air conditioning while for sites in Cluster 3 should focus on food refrigeration. For low temperatures, sites in Clusters 1 and 3 should reduce their use of heating while sites in Cluster 2 present a different heating profile which should be treated separately.

Conclusion

We proposed a model based on a functional formulation of the non-negative tensor factorization model and an associated optimization algorithm for the disaggregation of multi-sites load curves. By taking into account additional information such as the outside temperature and smoothness of the factors, we showed that this model exhibits more meaningful features and clusters than previously used NTF models. In this chapter, we discuss the optimization problem expressed in Chapter 7 from a theoretical point of view. Although tensor factorization models are used in a large panel of practical studies, theoretical considerations are still rare and most of the research focuses on application or on algorithmic considerations to solve large-scale problems. The reason behind this is that tensor factorization problem are subject to various issues which makes theoretical study a challenging research topic. In particular, finding the true factorization of a tensor is often NP-Hard [START_REF] Hillar | Most Tensor Problems Are NP-Hard[END_REF] and finding and approximated factorization requires solving a non-convex optimization problem through an iterative process based on alternating minimization or on gradient descent. Since the problem is non-convex, few guarantees exist on the convergence of the optimization method used except that the objective function decreases at each iteration and that the algorithm converges to a local optima. In practical setting, these guarantees are usually enough provided a good initialization strategy for the algorithm (this can be based on high order singular values decomposition or simply using multiple initial points). A question which naturally arises in this context is the existence of a global optimum. Unfortunately, this is not always guaranteed and degeneracies have been explored in both theoretical and experimental works especially for the popular CANDECOMP/PARAFAC (CP) decomposition (see e.g. [START_REF] Lim | Nonnegative approximations of nonnegative tensors[END_REF][START_REF] Silva | Tensor rank and the ill-posedness of the best low-rank approximation problem[END_REF] and the references therein). However, it is common knowledge in the inverse problem and statistical learning communities, that solving ill-posed problems usually requires some form of control controlling on the complexity of the solution space, for example via regularization [START_REF] Tikhonov | Solutions of ill-posed problems[END_REF][START_REF] Vapnik | Statistical learning theory. Adaptive and Learning Systems for Signal Processing, Communications, and Control[END_REF]. In the context of tensor factorization, is has been shown in [START_REF] Lim | Optimal solutions to non-negative parafac/multilinear nmf always exist[END_REF][START_REF] Lim | Nonnegative approximations of nonnegative tensors[END_REF] that adding non-negativity constraints ensures exis-209 tence of a global solution for the CP decomposition. This new decomposition is known as non-negative tensor factorization (NTF).

In Chapter 7, we expressed our problem as a weighted version of the usual NTF problem with vanishing weights representing unobserved temperatures or intraday samples for a given site. This motivates us to studying the impact of missing values on the existence of a global optimum for the NTF problem. The literature on tensor factorization from the past decades has given rise to a wide range of algorithms among which some are adapted to missing values and/or additional constraints. Methods handling missing values usually fall into one of three following categories : imputation, weighted least squares and probabilistic models. In the first case, missing entries are estimated at each iteration resulting in an EM-like algorithm [START_REF] Andersson | The N-way Toolbox for MAT-LAB[END_REF][START_REF] Bro | PARAFAC. Tutorial and applications[END_REF]. In the second case the squared error is weighted with binary weights representing missing and observed entries (see e.g. [START_REF] Acar | Scalable tensor factorizations for incomplete data[END_REF][START_REF] Tomasi | PARAFAC and missing values[END_REF]. In the last case, prior distributions are proposed for the factors and their parameters are estimated from the observed data [START_REF] Rai | Scalable Bayesian Low-Rank Decomposition of Incomplete Multiway Tensors[END_REF]Xiong, Chen, Huang, Schneider, and Carbonell, 2010;[START_REF] Zhao | Bayesian robust tensor factorization for incomplete multiway data[END_REF]. The problem of missing entries is also closely related to tensor completion where, in addition to the factors, one usually also tries to estimate a full tensor which coincides with the data tensor on observed entries (see e.g Song, Ge, Caverlee, and Hu, 2019 for a recent survey). Because of the need for efficient algorithms to deal with large amount of data, the literature on tensor factorization is dominated by algorithmic considerations, especially when missing values are taken into account. The only theoretical work we could find on tensor factorization with missing entries is Jain and Oh, 2014 where a condition on the probability of missing entries is proposed for symmetric, orthonormal tensors. In this chapter, we use a different approach and study the link between smoothness and missing values. This is motivated by the model of Chapter 7 where both smoothness and missing values appear naturally. Moreover, smoothness has been thoroughly exploited in tensor factorization and tensor completion [START_REF] Henriet | Independent-Variation Matrix Factorization With Application to Energy Disaggregation[END_REF][START_REF] Imaizumi | Tensor Decomposition with Smoothness[END_REF][START_REF] Li | Low-Rank Tensor Completion with Total Variation for Visual Data Inpainting[END_REF][START_REF] Reis | PARAFAC with splines: a case study[END_REF][START_REF] Sadowski | Image Completion with Smooth Nonnegative Matrix Factorization[END_REF][START_REF] Timmerman | Three-way component analysis with smoothness constraints[END_REF]Yokota and Cichocki, 2016;[START_REF] Yokota | Smooth nonnegative matrix and tensor factorizations for robust multi-way data analysis[END_REF] as it can help solving the ill-posed optimization problem and avoiding overfitting by in-corporating prior knowledge in the model while being at the same time beneficial for the interpretation of the factors as we saw in Chapter 7.

In this chapter, we control smoothness via the total variation norm of the factors and we show the existence of an upper bound for this norm which ensures the existence of a global optimum for the NTF problem with missing values. This approach leads to a constrained optimization problem for which we propose an algorithm. However, this problem is very challenging so solve and we show, in a second phase, that it can be relaxed to penalized optimization problem which is simpler to solve while keeping the same theoretical guarantees. Considering similar results for spline constraints or using the basis approach would be an interesting and challenging topic for future work.

In the remaining of this chapter, we consider a N-way tensor

X ∈ R I 1 ו••×I N of data with I 1 , • • • , I N ∈ N * and a N-way tensor of weights W ∈ R I 1 ו••×I N + .
To avoid lengthier notations, we denote by I := ∏ N n=1 1, I n and an element i ∈ I is a vector i = (i 1 , • • • , i N ) used to index the tensors. We denote by I W := {i ∈ I : W i > 0}. Finally, for a given q ∈ [1, +∞) and I ∈ N * , we denote the positive q-norm sphere in R I by S + I,q := a ∈ R I + : a q = 1 and we recall that the total variation norm a TV,q of a vector a ∈ R I is defined as a TV,q = L I a TV,q where

L I :=         1 -1 0 • • • 0 0 1 -1 . . . . . . . . . . . . . . . . . . 0 0 • • • 0 1 -1        
∈ R (I-1)×I . (8.0.1)

Existence of a global optimum for the weighted NTF

A tensor Y ∈ R I 1 ו••×I N is said to have a non-negative tensor factorization with rank-R is there exist N factor matrices A (1) ∈ R I 1 ×R + , • • • , A (N) ∈ R I N ×R + such that Y = R ∑ r=1 a (1) r • • • • • a (N) r , (8.1.1)
where we recall that a

(n) r is the r-th column of A (n) . It is known that this formulation of the NTF is ill-posed because of permutation and scaling indeterminacy. The permutation indeterminacy indicates the fact that

R ∑ r=1 a (1) r • • • • • a (N) r = R ∑ r=1 a (1) σ(r) • • • • • a (N) σ(r) ,
for any permutation σ of 1, R and the scaling indeterminacy means that

R ∑ r=1 a (1) r • • • • • a (N) r = R ∑ r=1 (αa (1) r ) • (α -1 a (2) r ) • a (3) r • • • • • a (N) r ,
for any α ∈ R * . The permutation indeterminacy is not problematic because the user can define an order for the factors depending on the use case. On the contrary, the scaling indeterminacy implies that one of the factors can diverge without changing the factorization. To deal with this indeterminacy we need some constraint for at least N -1 of the factors. It is therefore usual to assume that the N -1 first factors have unit q-norm for some q ∈ [1, +∞). Equivalently, we can also assume that all factors are normalized and add an additional scaling parameter, thus reformulating (8.1.1) as

Y = R ∑ r=1 λ r a (1) r • • • • • a (N) r , (8.1.2)
with a

(n) r ∈ S + I n ,q for some q ∈ [1, +∞) and λ r ≥ 0. In this chapter, we consider the problem of approximating the tensor X by a tensor of the type (8.1.2). In addition, we assume that some entries of the tensor can be missing. This can be expressed by using a Hadamard product with a weights tensor where zeros represent missing entries. This leads to the following weighted least square problem. min λ,A (1)

,••• ,A (N) L W (λ, A (1) , • • • , A (N) ) s.t. ∀r ∈ 1, R , λ r ≥ 0 and ∀n ∈ 1, N , a (n) r ∈ S + I n ,q , (8.1.3)
where

L W (λ, A (1) , • • • , A (N) ) := W X - R ∑ r=1 λ r a (1) r • • • • • a (N) r 2 2 , (8.1.4) and W ∈ R I 1 ,••• ,I N +
is a binary tensor with value 0 for missing entries and 1 for observed entries. In the following we do not restrict W to be binary and consider only that W ∈ R I 1 ,••• ,I N + and that it takes the value 0 for missing entries.

A sufficient condition for the existence of a global minimum for the problem (8.1.3) is that the function L W is coercive. Assumption 8.1.1. For all (j n ) n∈N ∈ ∏ n∈N 1, I n , there exists i ∈ I , such that i n = j n , for all n ∈ N .

Then the following proposition holds. Proposition 8.1.2. Assume that I W = I and let ρ ∈ (0, +∞] N , p, q ∈ [1, +∞). Then there exists a constant C p,q (ρ, I W ) ∈ [0, +∞) such that the following assertions hold.

(i) For all 0 ≤ C < C p,q (ρ, I W ), the function L W is coercive on Θ p,q (ρ, C).

(ii) For all C p,q (ρ, I W ) ≤ C ≤ ∞, the function L W is not coercive on Θ p,q (ρ, C). Moreover, C p,q (ρ, I W ) has the following property in which we use notation {ρ = +∞} = {n ∈ 1, N : ρ n = +∞}.

(iii) We have C p,q (ρ, I W ) > 0 if and only if (I W , {ρ = +∞}) satisfies Assumption 8.1.1.

The proof of Proposition 8.1.2 is provided in Section 8.5.2 and the computation of the constant C p,q (ρ, I W ) is discussed in Sections 8.5 and 8.7. To end this section, let us comment on the result provided by Proposition 8.1.2. Remark 8.1.1.

1. The constant C p,q (ρ, I W ) describes how little we can constrain the total variation norm of the factors in order to ensure coercivity of the loss function L W . In this sense, it is an optimal smoothness constraint.

2.

If, for all N ⊂ 1, N , we define the N -cylinders as a the sets of the type {i ∈ I : i n = j n , ∀n ∈ N }, where (j n ) n∈N ∈ ∏ n∈N 1, I n , Assertion (iii) of Proposition 8.1.2 says that the only case in which adding smoothness constraints does not give guarantees on the optimization problem is the case where we have a whole {ρ = +∞}-cylinder missing in the data. In particular we have the following assertions.

a) If no mode is free, i.e. ρ n < +∞ for all n ∈ 1, N , then C p,q (ρ, I W ) = 0 if and only if W = 0 i.e. no entry is observed.

b) If only one mode is free, say ρ 1 = +∞, then C p,q (ρ, I W ) = 0 if and only if there exists a mode-1 fiber which is missing, i.e. there exists i

1 ∈ 1, I 1 such that W i 1 ,••• ,i N = 0 for all (i 2 , • • • , i N ) ∈ ∏ N n=2 1, I N . c) If only two modes are free, say ρ 1 = ρ 2 = +∞, then C p,q (ρ, I W ) =
0 if and only if there exists a mode-(1, 2) slice which is missing, i.e. there exists (i

1 , i 2 ) ∈ 1, I 1 × 1, I 2 such that W i 1 ,••• ,i N = 0 for all (i 3 , • • • , i N ) ∈ ∏ N n=3 1, I N .
d) If all modes are free, i.e ρ n = +∞ for all n ∈ 1, N , then C p,q (ρ, I W ) = 0 if and only if I W = I.

The conclusion of Proposition 8.1.2 is that we can guarantee the existence of a global solution for the problem min λ,A (1)

,••• ,A (N) L W (λ, A (1) , • • • , A (N) )
such that for all r ∈ 1, R , and all n ∈ 1, N , λ r ≥ 0 and a

(n) r ∈ S + I n ,q with a (n) r TV,p ≤ C n , ( 8

HALS Algorithm for the constrained weighted NTF

In this section, we address the optimization problem (8.1.8) which does not fit into the standard cases encountered in the tensor factorization literature because of the additional condition a

(n) r TV,p ≤ C n . There are usually two ways to solve tensor factorization problems : alternating minimization or gradient-based methods [START_REF] Acar | Scalable tensor factorizations for incomplete data[END_REF][START_REF] Tomasi | PARAFAC and missing values[END_REF]. A combination of both can also be used [START_REF] Yokota | Smooth PARAFAC Decomposition for Tensor Completion[END_REF]. In order to take into accounts additional constraints, some projection steps are also needed. Depending on the constraints, this projection can be exact or can be approximated. Usually, the projection onto S + I n ,q is approximated by taking the positive part of the tensor and dividing by its q-norm. However, recently, variable splitting methods such as Alternating Direction Method of Multipliers (ADMM, see e.g. Goldstein, O'Donoghue, Setzer, and Baraniuk, 2014 and the reference therein) have been proposed as a flexible way to include constraints in tensor factorization problems (see e.g. [START_REF] Liavas | Parallel Algorithms for Constrained Tensor Factorization via Alternating Direction Method of Multipliers[END_REF][START_REF] Sadowski | Image Completion with Smooth Nonnegative Matrix Factorization[END_REF][START_REF] Smith | Constrained Tensor Factorization with Accelerated AO-ADMM[END_REF][START_REF] Zdunek | Alternating direction method for approximating smooth feature vectors in Nonnegative Matrix Factorization[END_REF]. The use of ADMM in NTF consists in splitting the variable A (n) with itself in order to separate the minimization of the loss and the non-negativity constraints. Here, we follow a different approach where the variable splitting is used to deal with the constraint on the total variation norm. The method we propose here is based on Hierarchi-cal Alternating Least Squares (HALS) iterations. To simplify the notations let us write for r ∈ 1, R , n ∈ 1, N , and a ∈ A n ,

L (r,n) W (a) = W (X (r) -λ r a (1) r • • • • a (n-1) r • a • a (n+1) r • • • • • a (N) r ) 2 2 ,
where

X (r) = X -∑ s =r λ s a (1) s • • • • a (N) s . Also let L n = L I n ,
λ, A (1) , • • • , A (N) 1 E = X -∑ R r=1 λ r a (1) r • • • • • a (N) r 2 repeat 3 for r = 1, • • • , R do 4 X (r) = E + λ r a (1) r • • • • • a (N) r 5 for n = 1, • • • , N do 6 a (n) r = argmin a∈S + In,q , L n a p ≤C n L (r,n) W (a) 7 λ r = argmin λ≥0 W (X (r) -λa (1) r • • • • • a (N) r ) 2 2 8 E = X (r) -λ r a (1) r • • • • • a (N) r 9 until Change of W E 2 2 is sufficiently small; 10 return λ, A (1) , • • • , A (N)
In Algorithm 8.2.1, the update in λ r simply writes as

λ r =    W X (r) , W (a (1) r • • • • • a (N) r ) F W (a (1) r • • • • • a (N) r ) 2 F    + (8.2.1) For the update in a (n) r , note that L (r,n) W (a) is equal, up to an additive constant, to a Ma + 2b a, where M = λ 2 r diag   W 2 (n) m =n a (m) r 2   and b = -λ r X (r) (n) W 2 (n) m =n a (m) r .
Then the problems can be written as min

a∈S + In,q , L n a p ≤C n 1 2 a Ma + b a , (8.2.2)
This problem is quadratic and M is positive definite as soon as at the last iteration of the HALS (8.1.5) holds and λ r > 0. However, the constraints imposed are not convex and therefore finding a solution is not guaranteed. We still propose empirical ways to get an approximated solution. To this end, let us distinguish the cases where C n = +∞ and C n < +∞.

Case where C n = +∞

In the case where C n = +∞, Problem (8.2.2) reduces to minimizing a quadratic function on the positive sphere S + I n ,q . Except when q = 1, in which case the constraints are convex and a quadratic solver can be used, there is no standard method to solve this problem. For q > 1, an usual approximation in the tensor factorization literature consists in first solving the problem without constraints and then taking the positive part and normalizing the solution i.e

a = -M -1 b + [-M -1 b] + q . (8.2.3)
This approximation does not make sense when all the entries of -M -1 b are negative. In this case, several other approximations can be considered. For example one can solve the problem with only the non-negative constraint (which is convex) and then normalize the solution. A projected gradient steps can also be used as in [START_REF] Yokota | Smooth PARAFAC Decomposition for Tensor Completion[END_REF] I n ,q and therefore the method described in the previous section is applied. The update for z requires to compute the orthogonal projection onto the ball B p (C n ) = x ∈ R I n -1 : x p ≤ C n denoted by proj B p (C n ) . For p = 2 this projection has the exact form proj B 2 (C) (y) = y max(1,C -1 y 2 ) . For p = 1, an algorithm based on projection onto the simplex can be used as in [START_REF] Condat | Fast projection onto the simplex and the l 1 ball[END_REF][START_REF] Duchi | Efficient Projections onto the l1-Ball for Learning in High Dimensions[END_REF]. For p > 2, Newton's method can be used to find a zero of the gradient of the Lagrangian of the optimization problem min z∈B p (C) zy 2 2 as implemented in the Matlab BPDQ toolbox [START_REF] Hammond | The Basis Pursuit DeQuantizer (BPDQ) toolbox[END_REF] The second method to approximate a solution of Problem (8.2.2) is to solve the penalized version of the problem for increasing values of the penalty parameter until the constraints are satisfied. In order to remove the nonconvexity of the problem caused by the condition a ∈ S + I n ,q , we propose to solve the penalized version only under non-negativity constraints until L n a p ≤ C n a q and then normalize the solution. This procedure is described in Algorithm 8.2.3 where the update in a can be obtained by a first order method such as L-BFGS-B using the (sub)-gradient ∂ • p p (a) = |a| p-1 sign(a) where |•| and sign(•) are applied coordinate-wise.

Similarly to the case C n = +∞, none of the two methods proposed stands out. For example, the penalized method can be long if the target value of the penalization parameter is large. One may use a more aggressive update of this parameter in Algorithm 8.2.3 but with the risk of penalizing too much. On the other hand, ADMM for non-convex problems does not come with the same guarantees as in the convex case. In particular Wang, Yin, and Zeng, 2019 gives necessary conditions for convergence of ADMM in the non-convex case which are not satisfied by Problem (8.2.4) (see Condition A5 and Section 4 of [START_REF] Wang | Global convergence of ADMM in nonconvex nonsmooth optimization[END_REF]. In our implementation use the ADMM approach first and then the penalized approach if the first one fails at satisfying the condition under a reasonable number of iterations.

where S R denotes the set of permutations of 1, R . The similarity score quantifies how well the factors have been recovered up to scaling and permutation. As in Acar, Dunlavy, Kolda, and Mørup, 2011, we say that the factors are correctly recovered if sim(θ, θ) ≥ 0.97 and compute, for each value of I, R and p missing , the accuracy as the proportion of correctly recovered parameters. More precisely, for the given values if I,R,p missing and percentage of missing data, we have generated a set {(θ(k), X(k), W(k)) : k ∈ 1, 600 } of true parameters, noisy tensors and masks and obtained the related estimations θmodel (k) : k ∈ 1, 600 for each model (swntf and cpwopt). Then we compute accuracy(model) := 1 600 600

∑ k=1 1 {sim(θ(k), θmodel (k))≥0.97} .
The results are displayed in Table 8.1.

The first observation is that the task is more difficult when R or p missing grow or I is lower. The reason for this is that the difficulty of the problem is inversely proportional to the ratio between the number of known entries and the number of degrees of freedom which is equal to

(1-p missing )I 3 3RI
(see [START_REF] Acar | Scalable tensor factorizations for incomplete data[END_REF]. Then, both methods perform comparably well for less than 50% missing entries. However for 70% missing entries and I = 50, the accuracy of our method drops to 79.9% and 65.6% while the accuracy of cpwopt remains close 100%. A deeper investigation showed that this drop is caused by the 30 tensors which were allowed to vanish. For these tensors, Condition (8.1.5) is not necessarily satisfied and we propose two possible reasons explaining the drop of accuracy in our method. The first one is that too much bias may be added by the constraint on the TV-norm and the second is that the optimization problem (8.2.2) may be too challenging and that the algorithm proposed in Section 8.2 relies too much on approximations. It is surprising though that the cpwopt algorithm (based on gradient descent) seems to perform well even on cases where Condition (8.1.5) is not necessarily satisfied. Finding an explanation for this would be a challenging research topic which would require more investigation on the sequence of iterations of the optimization algorithm used. Table 8.1: Accuracy (in %) for randomly generated data, "both" refers to initializing swntf with cpwopt.

Discussion and perspective

In this chapter, we derived a sufficient condition based on the total variation norm to ensure existence of a global minimum for the weighted least squares problem associated to Non-negative Tensor Factorization with missing values. An HALS algorithm was proposed to solve the resulting difficult optimization problem. Unfortunately, its performance has not been validated against state-of-the-art methods in which no additional constraint is used. We argue that the drawback of our approach is mainly due to the multiplication of constraints which make the optimization very challenging. However, the main theoretical outcome of this work can help to better understand which cases are subject to degeneracies. Surprisingly, the gradient-based approach seems to work well even in these cases and further study should be considered to understand the underlying reasons for this behavior.

To open up this work to further perspectives, we propose to show that Propositions 8.1.1 and 8.1.2 can be used to guarantee the existence of a global minimum for a penalized version of the problem which is similar to the one used in [START_REF] Yokota | Smooth PARAFAC Decomposition for Tensor Completion[END_REF]. The advantage of the penalized version is that it is simpler to solve because less constraints are imposed. This result is stated as the following proposition whose proof can be found in Section 8.5.1.

Proposition 8.4.1. Let p, q, d ∈ [1, +∞) and α = [α 1 , • • • , α N ] ∈ R N + .Let Θ := R R + × ∏ N n=1 (S + I n ,q ) R and define for all θ := (λ, A (1) , • • • , A (1) ) ∈ Θ, P α (θ) := R ∑ r=1 λ d r N ∑ n=1 α n a (n) r p TV,p .
Then, if (I W , {α = 0}) satisfy Assumption 8.1.1, the function L W + P α is coercive on Θ.

Remark 8.4.1. Let us make the following remarks on Proposition 8.4.1.

1. If we take q = d = 2, we get the loss of [START_REF] Yokota | Smooth PARAFAC Decomposition for Tensor Completion[END_REF] 2. The optimization problem we considered in Section 8.2 presents the advantage that a global optimum exists but the price to pay is that, in the HALS algorithm, we have to optimize on the intersection of the positive sphere and a closed ball for the TV-norm.

N n=1 S + I n ,q → Θ (A (1) , • • • , A (N) ) → (λ, Ã(1) , • • • , Ã(N) ) , with for all n ∈ 1, N , A n ⊂ R I n + \ {0} and, for all r ∈ 1, R , λ r = ∏ n n=1 a (n) r q and ã(n) r = a (n) r a (n) r q if a (n) r q
> 0 and any element of S + I n ,q otherwise.

The reason behind this choice it that, taking d = p, we get that, for all

(A (1) , • • • , A (N) ) ∈ ∏ N n=1 A R n , L W (ψ(A (1) , • • • , A (N) )) = W X - R ∑ r=1 a (1) r • • • • • a (N) r 2 2
, and

P α (ψ(A (1) , • • • , A (N) )) = R ∑ r=1 N ∑ n=1 α n ∏ m =n a (m) r p q a (n) r p TV,p ,
hence we can easily compute a gradient. To simplify the notations, let us introduce the R + -valued functions

f W : (a (1) , • • • , a (N) ) → W a (1) • • • • • a (N) 2 2 , (8.5.1)
and

g W : (λ, A (1) , • • • , A (N) ) → W R ∑ r=1 λ r a (1) r • • • • • a (N) r 2 , (8.5.2) respectively defined on ∏ N n=1 S + I n ,q and R R + × ∏ N n=1 S + I n ,q R .
Proof of Proposition 8.1.1. First note that, by continuity of f W , (8.1.5) is equivalent to

η := inf ∏ N n=1 A n f W > 0 . (8.5.3)
Hence we show that L W is coercive on Θ if and only if Condition (8.5.3) holds.

From the two triangular inequalities, we get that L W is coercive if and only if the function g W defined in (8.5.2) is coercive and therefore we prove that Condition (8.5.3) are necessary and sufficient for g W to be coercive.

First assume that Condition (8.5.3) holds and let us show that g W is coercive. Consider a sequence (θ(m)) m∈N ∈ Θ N such that θ(m) 2 ----→ m→+∞ +∞ and write for all m ∈ N, θ(m) = (λ(m), A (1) (m), • • • , A (N) (m)). Then, since A is bounded, we must have λ(m) 2 ----→ m→+∞ +∞ and, using the fact that the entries of θ are all non-negative and Condition (8.5.3), we get

(g W (θ(m))) 2 ≥ R ∑ r=1 (λ r (m)) 2 f W (a (1) r (m) • • • • • a (N) r (m)) ≥ η λ(m) 2 2 ,
which diverges to +∞ as m → +∞ because η > 0. Thus g W is coercive. Now, for the converse statement, assume that Condition (8.5.3) does not hold and let us show that g W is not coercive. To this end, we construct a sequence (θ(m)) m∈N ∈ Θ N such that θ(m) 2 ----→ m→+∞ +∞ while g(θ(m)) remains bounded. Since (8.1.5) does not hold, we get that for all m ∈ N, there exists (a (1)

(m), • • • , a (N) (m)) ∈ ∏ N n=1 A n such that f (a (1) (m), • • • , a (N) (m)) ≤ 2 -m . Now, for all m ∈ N and n ∈ 1, N , take A (n) (m) = [a (n) (m), a (n) 2 , • • • , a (n) R ], where for r ≥ 2, a (n) r is any element of S + I n ,q and let λ(m) = [m, 0, • • • , 0]. Then θ(m) := (λ(m), A (1) , • • • , A (N) ) ∈ Θ is such that θ(m) 2 ----→ m→+∞ +∞, but (g W (θ(m))) 2 = m 2 f (a (1) (m), • • • , a (N) (m)) ≤ m 2 2 -2m ---→ m→∞ 0 .
Proof of Proposition 8.4.1. Since, for all θ ∈ Θ, we have

L W (θ) ≥ (g W (θ) -W X 2 ) 2 ,
with g W defined in (8.5.2), it suffices to show that the function g 2 W + P α is coercive. To this end, let us define the function

h α : (a (n) ) n∈ 1,N → N ∑ n=1 α n a (n) r p TV,p , from ∏ N n=1 S + I n ,q to R + , and the function ϕ : λ → λ d∧2 ∞ -1 + , from R R + → R + .
Then, for all θ = (λ, A (1) , • • • , A (N) ) ∈ Θ, since all the entries of θ are non-negative, we have

(g W (θ)) 2 + P α (θ) ≥ (λ 2 s ∧ λ d s ) f W ((a (n) s ) n∈ 1,N ) + h α ((a (n) s ) n∈ 1,N ) ,
f W is defined in (8.5.1) and s ∈ 1, N is taken such that λ s = max r∈ 1,R λ r = λ ∞ . Then, using the inequality λ 2 s ∧ λ d s ≥ ϕ(λ), we get that, for all θ = (λ, A (1) (8.5.4) where

, • • • , A (N) ) ∈ Θ, (g W (θ)) 2 + P α (θ) ≥ ϕ(λ)η ,
η := inf ∏ N n=1 S + In,q ( f W + h α ) .
Hence, to get that g 2 W + P α is coercive, it suffices to have η > 0, which we now show.

Define for all n ∈ 1, N , ρ n = α -p n with the convention that 0 -1 = +∞. Then, Assertion (ii) of Proposition 8.1.2 gives that C p,q (ρ, I) > 0 and therefore there exists 0 < C < C p,q (ρ, I). Then, taking A := ∏ N n=1 A n,p,q (ρ, C) with A n,p,q (ρ, C) defined as in (8.1.6), we get from Proposition 8.1.2 and (8.5.3) that inf A f W > 0. Moreover for all (a (n) ) n∈ 1,N ∈ A c , there ex- To prove Proposition 8.1.2, we construct the constant C p,q (ρ, I W ). To this end, we need to introduce the following notations. For any integer I ≥ 1 and J ⊂ 1, I and p, q ∈ [1, +∞), define A I,q (J ) := a ∈ S + I,q : ∀j ∈ J , a j = 0 , and m p,q,I (J ) := inf a TV,p : a ∈ A I,q (J ) , with the convention that inf(∅) = +∞. Then for all ∅ J ⊂ J 1, I , we get the two following straightforward displays. ∅ = A I,q ( 1, I n ) A I,q (J ) ⊂ A I,q (J ) A I,q (∅) = S + I,q , (8.5.5) and 0 = m p,q,I (∅) < m p,q,I (J ) ≤ m p,q,I (J ) < m p,q,I ( 1, I ) = +∞ . (8.5.6)

ists k ∈ 1, N such that a (k) / ∈ A n,p,q (ρ, C), i.e. α k a (k) r p TV,p ≥ C p and therefore h α ((a (n) ) n∈ 1,N ) ≥ α k a (k) r p TV,p ≥ C p . Hence, we get f W + h α ≥ (inf A f W ) 1 A + C p 1 A c and therefore inf( f W + h α ) ≥ (inf A f W ) ∧ C p > 0,
Now, for I ⊂ I and n ∈ 1, N , we define the projection of I onto the n-th coordinate as π n (I ) := {i n : i ∈ I } and introduce the two following sets Π(I

) := (J n ) N n=1 : ∀n ∈ 1, N , J n ⊂ π n (I ) , and
J (I ) := (J n ) N n=1 ∈ Π(I ) : ∀i ∈ I , ∃n ∈ 1, N , i n ∈ J n .
Finally, for I ⊂ I, p, q ≥ 1, and ρ ∈ (0, +∞] N , we define, for all (J n )

N n=1 ∈ Π(I ), c p,q (ρ, (J n ) N n=1 ) := max n∈ 1,N ρ -1 n m p,q,I n (J n ) . ( 8
.5.7) and C p,q (ρ, I ) := min .5.8) with the convention that 0 × inf(∅) = +∞.

(J n ) N n=1 ∈J (I ) c p,q (ρ, (J n ) N n=1 ) . ( 8
In the remaining of this section, we will show that C p,q (ρ, I W ) satisfies the assertions of Proposition 8.1.2. We first start with the following lemma which proves Assertion (iii) of Proposition 8.1.2. Lemma 8.5.1. Let p, q ∈ [1, +∞), I ⊂ I and ρ ∈ (0, +∞] N . Then the following statements hold.

(i) C p,q (ρ, I ) = +∞ if and only if I = I.

(ii) C p,q (ρ, I ) > 0 if and only if (I , {ρ = +∞}) satisfies Assumption 8.1.1. Proof. For Assertion (i), using (8.5.6), we get that

C p,q (ρ, I ) < +∞ ⇔ ∃(J n ) N n=1 ∈ J (I ), ∀n ∈ 1, N , m p,q,I n (J n ) = +∞ ⇔ ∃(J n ) N n=1 ∈ J (I ), ∀n ∈ 1, N , J n = 1, I n ⇔ ∃(J n ) N n=1 ∈ J (I ), ∃i ∈ I, ∀n ∈ 1, N , i n / ∈ J n ⇔ J (I ) ∩ (J (I)) c = ∅ ⇔ I = I .
The implication ⇒ in the last equivalence is straightforward by contraposition. For the converse implication, assume that I = I and take i ∈ I \ I . Then it is easily seen that (π n (I ) \ {i n }) N n=1 ∈ J (I ) ∩ (J (I)) c . For Assertion (ii), note that the pair (I , {ρ = +∞}) does not satisfy Assumption 8.1.1 if and only if there exists (j n ) n∈{ρ=+∞} ∈ ∏ n∈{ρ=+∞} 1, I n such that {i ∈ I : ∀n ∈ {ρ n = +∞}, i n = j n } ⊂ I . (8.5.9) Moreover, we have 8.5.10) where the last equivalence comes from (8.5.6).

C p,q (ρ, I ) = 0 ⇔ ∃(J n ) N n=1 ∈ J (I ), ∀n ∈ 1, N , ρ -1 n m p,q,I n (J n ) = 0 ⇔ ∃(J n ) N n=1 ∈ J (I ),      ∀n ∈ {ρ = +∞}, m p,q,I n (J n ) < +∞ ∀n ∈ {ρ < +∞}, m p,q,I n (J n ) = 0 ⇔ ∃(J n ) N n=1 ∈ J (I ),      ∀n ∈ {ρ = +∞}, J n = 1, I n ∀n ∈ {ρ < +∞}, J n = ∅ (
Then, using the definition of J (I ), (8.5.10) implies (8.5.9) by taking j n ∈ 1, I n \ J n for all n ∈ {ρ = +∞}. Conversely, if there exists j ∈ I which satisfies (8.5.9), we get (8.5.10) by taking J n = ∅ for n ∈ {ρ < +∞} and J n = 1, I n \ {j n } for n ∈ {ρ = +∞}.

We now prove Assertions (i) and (ii) of Proposition 8.1.2. To this end, we rely on the following lemma.

Lemma 8.5.2. Let I ⊂ I and p, q ∈ [1, +∞). Then, for all (J n ) N n=1 ∈ J (I ), we have

c p,q (ρ, (J n ) N n=1 ) = inf max n∈ 1,N ρ -1 n a (n) TV,p : (a (1) , • • • , a (N) ) ∈ N ∏ n=1 A I n ,q (J n ) . (8.5.11)
Moreover, if for all n ∈ 1, I n , J n = 1, I n , the infimum in (8.5.11) is finite and reached by an element of ∏ N n=1 A I n ,q (J n ).

Proof. If there exists n ∈ 1, I n , such that J n = 1, I n , Relations (8.5.5) and (8.5.6) give the two terms of (8.5.11) are equal to +∞. Now, if for all n ∈ 1, I n , such that J n = 1, I n , we know, by definition, that c p,q (ρ, (J n ) N n=1 ) is a lower bound for the set over which the infimum is taken in (8.5.11). Moreover, for all n ∈ 1, N , since A I n ,q (J n ) is compact and the TV-norm is continuous, we can take a (n) ∈ A I n ,q (J n ) such that a (n) TV,p = m p,q,I n (J n ) and therefore max n∈ 1,N ρ -1 n a (n) TV,p = c p,q (ρ, (J n ) N n=1 ) thus concluding the proof of Relation (8.5.11).

We can now prove Assertions (i) and (ii) of Proposition 8.1.2. Note that Lemma 8.5.1 gives that, when I W = I, we have C p,q (I, I W ) < +∞.

Proof of Assertion (i) of Proposition 8.1.2. We show that, if L W is not coercive on Θ p,q (ρ, C) then C ≥ C p,q (ρ, I W ). Assume that L W is not coercive on Θ p,q (ρ, C). Then, by Proposition 8.1.1, Condition (8.1.5) does not hold and, since the A n,p,q (ρ, C)'s defined in (8.1.6) are closed, this means that there exists (a (1)

, • • • , a (N) ) ∈ ∏ N n=1 A n,p,q (ρ, C) such that for all i ∈ I W , there ex- ists n ∈ 1, N such that a (n) i n = 0. Now, construct (J n ) N n=1 ∈ J (I W ) by the following procedure. Start with (J n ) N n=1 = (∅) N n=1
and then for each i ∈ I W select one of the n's such that a (n)

i n = 0 and put i n in J n . With this construction, we have (a (1) , • • • , a (N) ) ∈ ∏ N n=1 A I n ,q (J n ) and therefore we get

C ≥ max n∈ 1,N ρ -1 n a (n) TV,p ≥ c p,q (ρ, (J n ) N n=1 ) ≥ C p,q (ρ, I W ) ,
where the first inequality comes from (8.1.7), the second from (8.5.11) and the last from (8.5.8).

Proof of Assertion (ii) of Proposition

(J n ) N n=1 ∈ J (I W ) and (a (1) , • • • , a (N) ) ∈ ∏ N n=1 A I n ,q (J n ) such that max n∈ 1,N ρ -1 n a (n)
TV,p = C p,q (ρ, I W ). Then, this gives that for all n ∈ 1, N , a (n) TV,p ≤ ρ n C p,q (ρ, I W ) ≤ ρ n C and therefore a (n) ∈ A n,p,q (ρ, C), as defined in (8.1.6). On the other hand, (J n ) N n=1 ∈ J (I W ) means that for all i ∈ I W , there exists n ∈ 1, N such that i n ∈ J n and we get that a (n)

i n = 0 because a (n) ∈ A I n ,q (J n ). Hence (8.1.5) does not hold.

8.6 Approximations of C p,q (ρ, I W )

In this section, we consider I ⊂ I, p, q ≥ 1 and ρ ∈ (0, +∞] N and assume that we know how to compute m p,q,I (J ) for any I ≥ 1 and J ⊂ 1, I . In this case, we also know how to compute c p,q (ρ, (J n ) N n=1 ) for any (J n ) N n=1 ∈ J (I ). Then, we cannot compute C p,q (ρ, I ) using the expression (8.5.8) because it would require to iterate over all the elements of J (I ) (which is represents roughly N |I | iterations). We therefore propose a procedure to compute C p,q (ρ, I ) at a reasonable cost (in time and memory). The issue of computing m p,q,I (J ) is addressed in Section 8.7.

In the remaining of this section, we need the following definitions. Let us endow the set Π(I ) with a partial order denoted by ⊂ and defined for

(J n ) N n=1 , (J n ) N n=1 ∈ Π(I ) by (J n ) N n=1 ⊂ (J n ) N n=1 if and only if ∀n ∈ 1, N , J n ⊂ J n . Moreover, for i ∈ I and N ⊂ 1, N , (J n ) N n=1 , (J n ) N n=1 ∈ Π(I ) we denote (J n ) N n=1 = (J n ) N n=1 + N i ⇔ ∀n ∈ 1, N , J n =      J n ∪ {i n } n ∈ N J n otherwise , and
(J n ) N n=1 = (J n ) N n=1 -N i ⇔ ∀n ∈ 1, N , J n =      J n \ {i n } n ∈ N J n otherwise .
For simplicity, for k ∈ 1, N , we use the notation + k andk instead of + {k} and -{k} . Then note that, for all (J n ) N n=1 , (J n ) N n=1 ∈ J (I), we have

(J n ) N n=1 ⊂ (J n ) N n=1 ⇒ c p,q (ρ, (J n ) N n=1 ) ≤ c p,q (ρ, (J n ) N n=1 ) . (8.6.1)
As a direct consequence of (8.6.1), we get

C p,q (ρ, I ) = min c p,q (ρ, (J n ) N n=1 ) : (J n ) N n=1 ∈ J (I ) , (8.6.2)
where

J (I ) := (J n ) N n=1 ∈ J (I ) : ∀(J n ) N n=1 ∈ J (I ) \ {(J n ) N n=1 }, (J n ) N n=1 ⊂ (J n ) N n=1 .
8.6.1 Algorithm to compute C p,q (ρ, I )

The procedure to compute C p,q (ρ, I ) is provided in pseudo-code in Algorithm 8.6.1. The idea is to approach C p,q (ρ, I ) from below by computing at step k the value C p,q (ρ, I k ) where I k ⊂ I with |I k | = k. The fact that this approaches C p,q (ρ, I ) from below relies on the following result.

Proposition 8.6.1. Let I ⊂ I and i ∈ I. Then

J (I ∪ {i}) = N m=1 (J n ) N n=1 + m i : (J n ) N n=1 ∈ J m (I , i) , (8.6.3)
where J m (I , i) is the set of (J n ) N n=1 ∈ J (I ) satisfying at least one of the following assertions.

(i) i m / ∈ π m (I ).

(ii) i m ∈ J m .

(iii) i m ∈ π m (I ) and for all (J n ) N n=1 ∈ J (I ) such that i m ∈ J m , we have

(J n ) N n=1 ⊂ (J n ) N n=1 + m i.
Proof. First note that

J (I ∪ {i}) = N ⊂ 1,N N =∅ (J n ) N n=1 + N i : (J n ) N n=1 ∈ J (I ) .
To prove inclusion ⊂ of (8.6.3), take N ⊂

1, N , N = ∅ and (J n ) N n=1 ∈ J (I ) such that (J n ) N n=1 + N i ∈ J (I ∪ {i}). Note that for all m ∈ N and (J n ) N n=1 ∈ J (I ) such that (J n ) N n=1 ⊂ (J n ) N n=1 , we have (J n ) N n=1 + m i ⊂ (J n ) N n=1 + N i, and therefore (J n ) N n=1 + m i = (J n ) N n=1 + N i.
This means that we can consider without loss of generality that |N | = 1 and (J n ) N n=1 ∈ J (I ). Let us write N = {m} and assume that (J n ) N n=1 / ∈ J m (I , i). Then i m ∈ π m (I ) \ J m and there exists (

J n ) N n=1 ∈ J (I ) such that i m ∈ J m and (J n ) N n=1 ⊂ (J n ) N n=1 + m i. Since (J n ) N n=1 + m i ∈ J (I ∪ {i}), we must have (J n ) N n=1 = (J n ) N n=1 + m i. This equality gives (J n ) N n=1 ⊂ (J n ) N n=1 because that for all n = m, J n = J n and J m = J m ∪ {i m } = J m ∪ {i m }. Since (J n ) N n=1 ∈ J (I ), this implies that (J n ) N n=1 = (J n ) N n=1
and therefore i m ∈ J m which contradicts the fact that i m ∈ π m (I ) \ J m . Hence (J n ) N n=1 ∈ J m (I , i) which concludes the proof of inclusion ⊂ in (8.6.3).

To prove inclusion ⊃ of (8.6.3), take m ∈ 1, N , (J

n ) N n=1 ∈ J m (I , i) and N ⊂ 1, N , N = ∅, (J n ) N n=1 ∈ J (I ) such that (J n ) N n=1 + N i ⊂ (J n ) N n=1 + m i and let us show that (J n ) N n=1 + N i = (J n ) N n=1 + m i. Assume N = {m}, then there exists k ∈ N such that k = m and, since J k ∪ {i k } ⊂ J k , we get i k ∈ J k and therefore (J n ) N n=1 -k i (J n ) N n=1 which contradicts the fact that (J n ) N n=1 ∈ J (I ).
Hence N = {m} and we want to show

(J n ) N n=1 + m i = (J n ) N n=1 + m i. Since Assertion (ii) implies i m ∈ J m and any of the assertions (i) (iii) implies i m / ∈ J m , the fact that (J n ) N n=1 ∈ J m (I , i) gives J m ⊂ J m . Hence (J n ) N n=1 ⊂ (J n ) N n=1 and, since (J n ) N n=1 ∈ J (I ), we get (J n ) N n=1 = (J n ) N n=1
and finally (J n ) N n=1 + m i = (J n ) N n=1 + m i. This concludes the the proof of inclusion ⊃ in (8.6.3).

Observing that the set J k+1 constructed at step k of Algorithm 8.6.1 is exactly the set of (J n ) N n=1 + m i with m ∈ 1, N and (J n ) N n=1 ∈ J k such that one of the assertions (i) (ii) (iii) of Proposition 8.6.1 holds, the following result comes easily by induction as a consequence of Proposition 8.6.1 and (8.6.2).

Corollary 8.6.2. The sequence (I k , J k , C k ) k=1,••• ,|I W | constructed by Algorithm 8.6.1 is such that for all k ∈ 1, |I | , J k = J (I k ) and C k = C p,q (ρ, I k ). In particular, this implies that C 1 ≤ C 2 ≤ • • • ≤ C |I | = C p,q (ρ, I ).

The particular case where one dimension is not constrained

In the case when one of the ρ n 's is equal to +∞, the following lemma can be used to speed up the computation of C p,q (ρ, I ).

Lemma 8.6.3. Assume ρ m = +∞, then C p,q (ρ, I ) = 1 {π m (I )= 1,I m } min j∈ 1,I m C p,q (ρ (-m) , i (-m) : i ∈ I , i m = j ) , (8.6.4) where for x ∈ R N , x (-m) is the vector if R N-1 formed by removing the m-th entry of x.

Proof. We can consider without loss of generality that m = N. The fact that C p,q (ρ, I ) = 0 when π N (I ) = 1, I N is a consequence of Lemma 8.5.1. We now assume that π N (I ) = 1, I N , then the proof follows two steps. In Step 1 we prove that C p,q (ρ, I ) = min c p,q (ρ, (J n ) N n=1 ) : (J n ) N n=1 ∈ J (I ), |J N | = I N -1 , (8.6.5) and in Step 2 we show (8.6.4).

Step 1 : Since we use the convention ∞/∞ = ∞, we have c p,q (ρ, (J n ) N n=1 ) = +∞ as soon as |J N | = I N . Hence C p,q (ρ, I ) = min c p,q (ρ, (J n ) N n=1 ) :

(J n ) N n=1 ∈ J (I ), |J N | ≤ I N -1 .
Then, to prove (8.6.5), it suffices to take (J n ) N n=1 ∈ J (I ) such that |J N | < I N -1 and construct (J n ) N n=1 ∈ J (I ) which satisfies |J N | = I N -1 and c p,q (ρ, (J n ) N n=1 ) ≤ c p,q (ρ, (J n ) N n=1 ). To this end, take j ∈ 1, I N \ J N , then, since π N (I ) = 1, I N , we know that there exists i ∈ I such that i N = j and by definition of J (I ), there exists k ∈ 1, N -1 such that i k ∈ J k . Take (J n ) N n=1 = (J n ) N n=1k i + N i, then (J n ) N n=1 ∈ J (I) with |J N | = I N -1 and c p,q (ρ, (J n ) N n=1 ) ≤ c p,q (ρ, (J n ) N n=1 ) because m p,q,I N (J N ) < +∞ and m p,q,I k (J k ) ≤ m p,q,I k (J k ). This concludes the proof of (8.6.5).

Step 2 : Reformulating the right hand side of (8.6.5) and using the fact that |J N | = I N -1 ⇒ c p,q (ρ, (J n ) N n=1 ) = c p,q (ρ (-N) , (J n ) N-1 n=1 ), we get C p,q (ρ, I ) = min j∈ 1,I N c p,q (ρ (-N) , (J n ) N-1 n=1 ) : (J n ) N n=1 ∈ J (I ), J N = 1, I N \ {j} .

Then (8.6.4) follows from the fact that, for any (J n ) N n=1 ∈ J (I ) such that J N = 1, I N \ {j}, we have (J n ) N-1 n=1 ∈ J i (-N) : i ∈ I , i N = j .

Algorithm 8.6.1: Computing C p,q (ρ, I )

Data: I , ρ, p, q Result: C p,q (ρ, I ) We further denote, for any matrix M with n columns, M q,p = sup Mx p : x ∈ R n , x q ≤ 1 .

Let for ∆ ≥ 1, G p,q (∆) := m p,q,∆ ({∆}) = m p,q,∆ ({1}) and H p,q (∆) := m p,q,∆ ({1, ∆}) ,

where the equality m p,q,∆ ({∆}) = m p,q,∆ ({1}) is obtained by a symmetry argument. Also note that G p,q and H p,q are non-increasing functions of ∆ and that G p,q (1) = H p,q (1) = H p,q (2) = +∞. Then the following proposition holds.

Proposition 8.7.1. Let q ≥ p ≥ 1. Let I ≥ K ≥ 1. Let J = {j 1 , • • • , j K } ⊂ 1, I with j 1 < • • • < j K . Denote

∆ * I (J ) =      max {j k -j k-1 + 1 : 2 ≤ k ≤ K} if K ≥ 2 1 otherwise
∆ † I (J ) = max{j 1 , (Ij K + 1)} .

Then we have m p,q,I (J ) = min G p,q (∆ † I (J )), H p,q (∆ * I (J )) . (8.7.1)

Proof of Proposition 8.7.1. Since the case where K = 1 is straightforward, we assume K ≥ 2 and take a ∈ A I,q (J ). (8.7.2) 1 = a q q = 1 {j 1 >1} a 1:j 1 q q + ∑ 2≤k≤K a j k-1 :j k q q + 1 {j K <I} a j K :I q q . (8.7.3) Moreover, we have the following assertions.

(i) If j 1 > 1 and a 1:j 1 q > 0, we have a 1:j 1 / a 1:j 1 q ∈ A j 1 ,q ({j 1 }). In such a case, we get a 1:j 1 TV,p ≥ a 1:j 1 q G p,q (j 1 ) ≥ a 1:j 1 q G p,q (∆ † I (J )) .

(ii) For all 2 ≤ k ≤ K, if a j k-1 :j k q > 0, we have a j k-1 :j k / a j k-1 :j k q ∈ A j k -j k-1 +1,q ({1, j kj k-1 + 1}). In such a case, we get a j k-1 :j k TV,p ≥ a j k-1 :j k q H p,q (j kj k-1 + 1) ≥ a j k-1 :j k q H p,q (∆ * I (J )) .

(iii) if j K < I, if a j K :I q > 0, we have a j K :I / a j K :I q ∈ A I-j K +1,q ({1}). In such a case, we get a j K :I TV,p ≥ a j K :I q G p,q (Ij K + 1) ≥ a k K :I q G p,q (∆ † I (J )) .

Using these three assertions in (8.7.2), we get that, for all a ∈ A I,q (J ), a p TV,p ≥ 1 {j 1 >1} a 1:j 1 p q + ∑ 2≤k≤n a j k-1 :j k p q + 1 {j K <I} a j K :I p q m p , where m denotes the right-hand side of (8.7.1). Using that a r + b r ≥ (a + b) r for all a, b ≥ 0 and r := p/q ∈ (0, 1], we further get with (8.7.3) that, for all a ∈ A I,q (J ), a TV,p ≥ m .

Hence the left-hand side in (8.7.1) is at least equal to the right-hand side. The converse inequality is easily obtained by observing that if a ∈ A j 1 ,q ({j 1 }) then completing a with Ij 1 zeros on its right, we obtain an element of A I,q (J ). Thus the left-hand side in (8.7.1) is at most equal to G p,q (j 1 ). Similarly, it is at most equal to H p,q (j kj K-1 + 1) for all 2 ≤ k ≤ K and to G p,q (Ij K + 1). Since one of these upper bounds is m, it is at most equal to m, which concludes the proof.

Lemma 8.7.2. Let 1 ≤ q ≤ p. We have, for any even integer ∆ ≥ 2, H p,q (∆) = 2 1/p-1/q G p,q (∆/2) .

Proof. We set ∆ = 2K with K ≥ 1. For all a = a 1:∆ ∈ R ∆ , we have (8.7.4) Let a = a 1:∆ ∈ A ∆,q ({1, ∆}), then if a 1:K q > 0, we have a 1:K / a 1:K q ∈ A K,q ({1}) and thus a 1:K TV,p ≥ a 1:K q G p,q (K) , which remains true if a 1:K q = 0. Similarly, we have, a (K+1):∆ TV,p ≥ a (K+1):∆ q G p,q (K) , and finally, from (8.7.4) and the two previous displays , we get a p TV,p ≥ G p,q (K) a 1:K p q + a (K+1):n

p q + |a K+1 -a K | p .
By concavity of x → x q/p , we have, for all a = a 1:∆ ∈ R ∆ , a q q 2 = 1 2 a 1:K q q + a (K+1):n q q ≤ 1 2 a 1:K p q + a (K+1):n p q q/p .

With the previous display we get that H p,q (∆) ≥ 2 1-p/q G p,q (K) .

Take now b ∈ A K,q ({K}) and its symmetric b ∈ A K,q ({1}), so that the concatenation a = b b defined by a k = b K+1-k for k = 1, . . . , K and a k = b k-k for k = K + 1, . . . , 2K satisfies a K+1 = a K , and a q = 2 1/q . Applying (8.7.4) to a := a / a q . we get that a p TV,p = 2 2 -1/q b p TV,p = 2 1-p/q b p TV,p .

and since a ∈ A q,∆ ({1, ∆}) whenever b ∈ A q,K ({K}), this shows that the previous inequality is an equality, which concludes the proof.

Lemma 8.7.3. We have, for any n ≥ 2, G p,q (n) = B n-1 -1 p,q .

Proof. Let a = a 1:n ∈ R n + such that a n = 0. Then

a p TV,p = ∑ 2≤k≤n-1 |a k -a k-1 | p + |a n-1 | p = A n-1 a 1:(n-1) p p ,
where we used the definition of A n . Thus, by definition of A n,q ({n}), we have

G p,q (n) = inf A n-1 x p : x ∈ R n-1 + , x q = 1 = inf A n-1 x p x q : x ∈ R n-1 + \ {0} .
Since B n-1 is defined as the inverse of A n-1 , by setting x = B n-1 y and observing that, B n having non-negative entries, y ∈ R n-1 + implies x ∈ R n-1 + , we get that G p,q (n) ≤ inf y p B n-1 y q : y ∈ R n-1 + \ {0}

Using the same change of variable but letting y ∈ R n-1 , we obtain

G p,q (n) ≥ inf A n-1 x p x q : x ∈ R n-1 \ {0} ≥ inf y p B n-1 y q : y ∈ R n-1 \ {0} .
Now, for any y ∈ R n-1 , y p does not depend on the sign of its entries and, B n having non-negative entries, B n-1 y q cannot decrease when replacing its entries by their absolute values. We conclude that the inf in the previous display is achieved on y ∈ R n-1 + \ {0}, hence the inequalities in the two previous displays are equalities. Since the previous inf coincide with sup B n-1 y q y p : y ∈ R n-1 \ {0}

-1

= B n-1 -1 p,q ,
this concludes the proof.

Using Proposition 8.7.1 and the above lemmas with p = q, we get the following result. Note that a lower bound is enough for Proposition 8. where ∆ ‡ I (J ) := max {j 1 , (Ij K + 1), (j kj k-1 + 1)/2 : 2 ≤ k ≤ K} , and ∆ ‡ I (J ) := max {j 1 , (Ij K + 1), (j kj k-1 + 1)/2 : 2 ≤ k ≤ K} , and, for simplicity, we have defined B 0 -1 p,p = +∞. In particular for J / ∈ {∅, 1, I }, we have the following assertions. Proof. Assertion (i) comes from the fact that B n 1,1 = max 1≤j≤n ∑ n i=1 [B n ] i,j = n. Assertion (ii) is obtained from the fact that B n -1 2,2 is the lowest singular value of A n and A n A n is the tri-diagonal matrix with 2 on its diagonal and -1 on the upper and lower diagonals, hence its eigenvalues are 2(1 + cos( kπ n+1 )) : k = 1, • • • , n . Finally, Assertion (iii), comes from inequalities (1.2) and (1.3) of Goldberg, 1987 with q = 1 and q = 2.

Part IV

A P P E N D I C E S A B A C K G R O U N D T H E O R Y O N O P E R AT O R S
In this chapter we introduce classical definitions and results for operators on Banach and Hilbert spaces (see e.g. [START_REF] Conway | A course in functional analysis[END_REF][START_REF] Gohberg | Classes of linear operators[END_REF]Kaashoek, 1990, 2003 for details). In all of the thesis, we assume that vector spaces are defined over the field C. Recall that if (E, • E ) and (F, • F ) are two normed spaces, such that E ⊂ F, we say that E is continuously embedded in F if there exists a constant C > 0 such that, for all x ∈ E, x F ≤ C x E . In this case, we write the continuous embedding of E in F as E → F .

a.1 Basic definitions and results on operators

For two Banach spaces (E, • E ) and (F, • F ), we call O(E, F) the set of linear operators P whose domain, denoted by D(P), is a linear subspace of E and whose range is a linear subspace of F. If E = F, we simply write O(E) = O(E, E) and this short notation will be used for all operator spaces defined in this chapter. The set of linear operator with domain E is denoted by L(E, F) and we let L b (E, F) be the set of all E → F continuous operators i.e. the operator P ∈ L(E, F) for which there exists C > 0 such that for all x ∈ E, Px F ≤ C x E . In this case, we define the operator norm

P L b (E,F) := sup x E ≤1 Px F < +∞ .
Then L b (E, F), P L b (E,F) is a Banach space.

In the remaining of this section, we consider the special case where E and F are Hilbert spaces.

in decreasing order, which we will always assume. The σ n 's are called the singular valued of P, which we denote by (σ n ) n∈N = sing(P).

Note from Theorem A.1.1 that the range of a compact operator is necessarily separable. Moreover, if P = ∑ n∈N σ n ψ n ⊗ φ n , then P L b (H 0 ,G 0 ) = max n∈N σ n = σ 0 and we have the following SVD's

P H = ∑ n∈N σ n φ n ⊗ ψ n , P k = ∑ n∈N σ k n ψ n ⊗ φ n , for all k ∈ N * , P 1/2 = ∑ n∈N σ 1/2 n ψ n ⊗ φ n , |P| = ∑ n∈N σ n φ n ⊗ φ n .
When P is auto-adjoint, then (A.1.2) holds with φ n = ψ n and the SVD is referred to as the eigendecomposition and the σ n 's and φ n 's are respectively called the eigenvalues and eigenvectors of P. Finally, we define the Schatten-p spaces for p ≥ 1, S p (H 0 , G 0 ) := {P ∈ K(H 0 , G 0 ) : sing(P) ∈ p (N)} , which are Banach spaces if endowed with the norm P p := sing(P) p (N) .

If we endow the space K(H 0 , G 0 ) with the operator norm, we get the following continuous embeddings, for all 1 ≤ p ≤ p , S p (H 0 , G 0 ) → S p (H 0 , G 0 ) → K(H 0 , G 0 ) → L b (H 0 , G 0 ) . (A.1.3)

The spaces S 1 (H 0 , G 0 ) and S 2 (H 0 , G 0 ) are also respectively known as the space of trace-class and Hilbert-Schmidt operators.

It is sometimes useful to use the notation S ∞ (H 0 , G 0 ) = K(H 0 , G 0 ) and for all P ∈ S ∞ (H 0 , G 0 ), P ∞ = P L b (H 0 ,G 0 ) . This way, we can state that the Schatten spaces inherit from the H ölder inequality of the p spaces. Namely, I 0 is a third Hilbert space and 1 ≤ p, q, r ≤ +∞ are such that 1 r = 1 p + 1 q and Φ ∈ S p (H 0 , G 0 ), Ψ ∈ S q (G 0 , I 0 ), we have ΨΦ ∈ S r (H 0 , I 0 ) and ΨΦ r ≤ Ψ q Φ p . This result also holds if we replace S ∞ by L b .

a.1.2 The separable case

We now assume that H 0 and G 0 are separable and consider a Hilbert basis (e n ) n∈N of H 0 . Then, for all P ∈ L b (H 0 ) and p ≥ 1, we have From this identity, we get that S 2 (H 0 , G 0 ) is a Hilbert space if endowed with the scalar product defined, for all P, Q ∈ S 2 (H 0 , G 0 ),

P, Q 2 = Tr(Q H P) = ∑ n∈N Pe n , Qe n G 0 .

a.1.3 Isometries and unitary operators

Let H 0 , G 0 be two Hilbert spaces. If A is a linear subset of H 0 , a linear mapping U from A to G 0 such that for all x, y ∈ H 0 , Ux, Uy G 0 = x, y H 0 is called an isometry. An bijective isometry is called a unitary operator. If their exists a unitary operator between H 0 and G 0 we say that they are isometrically isomorphic. An isometry U between H 0 and G 0 in always in L b (H 0 , G 0 ) and we have U L b (H 0 ,G 0 ) = 1 and Im(Φ) is closed. The following isometric extension theorem holds.

Theorem A.1.2. Let H 0 be a pre-Hilbert space, I 0 a Hilbert spaces and G be a linear subspace of H 0 .

1. Let S : G → I 0 be an isometry. Then S admits a unique isometric extension S : G → I 0 . If moreover H 0 is a Hilbert space then S(G) = S(G).

2. Let (v t ) t∈T and (w t ) t∈T be two sets of vectors in H 0 and I 0 respectively with T an arbitrary index set. If for all s, t ∈ T, v t , v s H 0 = w t , w s I 0 then there exists a unique isometric operator S : Span (v t , t ∈ T) → I 0 such that for all t ∈ T, Sv t = w t . If moreover H 0 is a Hilbert space then S(Span (w t , t ∈ T)) = Span (w t , t ∈ T).

a.1.4 The generalized inverse

Let H 0 and G 0 be two Hilbert spaces and P ∈ L b (H 0 , G 0 ), then the linear mapping P | ker(P) ⊥ →Im(P) : ker(P) ⊥ → Im(P)

x → Px is bijective. Then, the inverse P | ker(P) ⊥ →Im(P)

-1

is a linear mapping from Im(P) to ker(P) ⊥ and we define the generalized inverse of P as the operator P † ∈ O(G 0 , H 0 ) whose domain is D(P † ) = Im(P) ⊕ Im(P) ⊥ which coincides with P | ker(P) ⊥ →Im(P) -1

on Im(P) and such that ker(P ⊥ ) = Im(P) ⊥ . In other words, ∀x ∈ D(P -), there exists (x 1 , x 2 ) ∈ Im(P) × Im(P) ⊥ such that x = x 1 + x 2 , then P -x = P | ker(P) ⊥ →Im(P) -1

x 1 .

If we denote by Π ker(P) ⊥ the orthogonal projection onto ker(P) ⊥ , then P † P = Π ker(P) ⊥ . (A.1.4)

Characterizing PP † is harder especially since it is not defined on all G 0 . However, for a compact operator P ∈ K(H 0 , G 0 ), we have D(P † ) is dense in G 0 and, if we consider the SVD P = ∑ n∈N σ n ψ n ⊗ φ n and define for all n ∈ N, σ † n = σ -1 n if σ n > 0 and 0 otherwise, we have

D(P † ) = x ∈ G 0 : ∑ n∈N (σ † n ) 2 x, ψ n G 0 2 < +∞ ,
and for all x ∈ D(P † ), (A.1.5) where the series converges in H 0 . From this identity, we get that for all x ∈ D(P † ), PP † x = Π Im(P) x . (A.1.6)

P † x = ∑ n∈N σ † n x, ψ n G 0 φ n ,

a.1.5 SVD of normal operator

We conclude this section with the singular value decomposition of a bounded normal operator on a Hilbert space H 0 (see Conway, 1990, Theorem 9.4.6, Proposition 9.4.7). Let N ∈ L b (H 0 ) be a normal operator, i.e. NN H = N H N, then, there exists a σ-finite measure space (V, V, ξ), a unitary operator U : H 0 → L 2 (V, V, ξ) and n ∈ L ∞ (V, V, ξ), such that (A.1.7) where M n denotes the pointwise multiplicative operator on L 2 (V, V, ξ) associated to n, that is M n : f → n × f . We say that N has singular value function n on L 2 (V, V, ξ) with decomposition operator U.

UNU H = M n ,

a.2 Strong and weak operator topologies

As a Banach space, L b (H 0 , G 0 ) is endowed with its natural norm topology. However, weaker topologies can be introduced and are also valid for operators in L(H 0 , G 0 ). Firstly, for two Banach spaces E, F, the strong operator topology (s.o.t.) on L(E, F) is defined as the topology such that a sequence (P n ) n∈N ∈ L(E, F) N converges to P ∈ L(E, F) in s.o.t. if and only if, for all x ∈ E,

lim n→+∞ P n x -Px F = 0 .
If H 0 and G 0 are two Hilbert spaces, then weak operator topology (w.o.t.) on L(H 0 , G 0 ) is defined as the topology such that a sequence (P n ) n∈N ∈ L(H 0 , G 0 ) N converges to P ∈ L(H 0 , G 0 ) in w.o.t. if and only if, for all x ∈ H 0 and y ∈ G 0 , lim n→+∞ Φ n y, x H 0 = Φy, x H 0 .

Note that, by the polarization identity, this is equivalent to Here we only recall some definitions and elementary properties of the Bochner integrals of functions valued in Banach spaces with respect to non-negative measures.

b.1 Measurability in Banach spaces

Let E be a Banach space and (Λ, A) a measurable space. When working with functions from (Λ, A) to E, there are different definitions of measurability which turn out to be equivalent when E in separable. We introduce these definitions in this section.

A countably-valued function from (Λ, A) to E is a function which has a countable range and such that ∀x ∈ Im( f ), f -1 ({x}) ∈ A. Such a function can be written

∑ k∈N α k 1 A k
where α k ∈ E, the A k 's are disjoints sets in A.

If, in addition, the range of f is finite, we say that f is a simple function.

The most natural notion of measurability comes from endowing E with its the Borel σ-filed B(E) generated by the norm topology. Hence, we say that a function f : Λ → E is measurable if for all A ∈ B(E), f -1 (A) ∈ A and denote by F(Λ, A, E) the set of such functions.

The following theorem introduces other measurability notions and states their equivalence in the separable case. We refer to Chapter 1 of Dinculeanu, 2011 for details.

Theorem B.1.1. Let f : Λ → E where E be a separable Banach space and (Λ, A) a measurable space. Then the following assertions are equivalent (i) We have f ∈ F(Λ, A, E).

b.2 The Bochner integral

We consider here a Banach space E and µ a σ-finite non-negative measure on a measurable space (Λ, A). Then, noting that for all f ∈ F(Λ, A, E), the function f E : λ → f (λ) E is a scalar measurable function, we can define the Bochner space as L p (Λ, A, E, µ) := { f ∈ F(Λ, A, E) : f E ∈ L p (Λ, A, µ)} , where L p (Λ, A, µ) is the usual scalar L p space. Then we define the space L p (Λ, A, E, µ) as the space obtained by quotienting L p (Λ, A, E, µ) with the µ-a.e. equality. The standard results on scalar L p spaces are transferred the Bochner spaces. In particular, L p (Λ, A, E, µ) is Banach space if endowed is the norm f L p (Λ,A,E,µ) = f p E dµ 1/p and, if E is a Hilbert space, L 2 is a Hilbert space with scalar product f , g L 2 (Λ,A,E,µ) = f , g E dµ. For p ∈ [1, ∞), the space of simple measurable functions with finite-measure support, i.e. Span (1 A x : A ∈ A, µ(A) < ∞, x ∈ E), is dense in L p (Λ, A, E, µ). For f ∈ Span (1 A x : A ∈ A, µ(A) < ∞, x ∈ E) with range {α 1 , • • • , α n }, the integral (often referred to as the Bochner integral) of the E-valued function f with respect to µ is defined by

f dµ = n ∑ k=1 α k µ f -1 ({α k }) ∈ E . (B.2.1)
This integral is extended to L 1 (Λ, A, E, µ) by continuity (and thus also to L p if µ is finite). The Bochner integral is continuous in L 1 (Λ, A, E, µ) and, for all f ∈ L 1 (Λ, A, E, µ),

f dµ E ≤ f E dµ .
Moreover, if f ∈ L 1 (Λ, A, E, µ) we have, for all Banach space F and all Φ ∈ L b (E, F), then Φ f : λ → Φ f (λ) is in L 1 (Λ, A, F, µ) and

Φ f dµ = Φ f dµ .
The following result will be useful.

Lemma B.2.1. Let Φ ∈ L 1 (Λ, A, S + 1 (H 0 ), µ) and define the function Φ 1/2 : λ → Φ(λ) 1/2 . Then Φ 1/2 ∈ L 2 (Λ, A, S 2 (H 0 ), µ).

Proof. Simple measurability of Φ 1/2 is given by Lemma 2 in Kakihara, 1997, Section 3.4 and therefore, by Lemma B.1.2, Φ 1/2 ∈ F(Λ, A, S 2 (H 0 )). The fact that Φ 1/2 ∈ L 2 (Λ, A, S 2 (H 0 ), µ) then follows from the identity Φ 1/2 (λ)

C T E N S O R A L G E B R A

In this chapter, we gather the definitions of various operations on vectors and tensors, we refer to [START_REF] Kolda | Tensor Decompositions and Applications[END_REF] for details. We take N ∈ N * , and consider I 1 , • • • , I N ∈ N * . We use bold capital letters to denote tensors and matrices and bold lowercase letters for vectors. Standard font is used for the entries of the tensors, matrices and vectors. For example A = [a 1 , • • • , a R ] ∈ R I×R means that a r ∈ R I is the r-th column of A and the (i, r)-th entry of A is denoted by A i,r or a i,r . A tensor X ∈ R I 1 ו••×I N is indexed by a vector of integers i = (i 1 , • • • , i N ) ∈ ∏ N n=1 1, I n and we write

X i = X i 1 ,••• ,i n .
For X, Y ∈ R I 1 ו••×I N , we define the Hadamard product X Y of X and Y as the element of R I 1 ו••×I N such that for all (i 1 ,

• • • , i N ) ∈ ∏ N n=1 1, I N , (X Y) i 1 ,••• ,i N = X i 1 ,••• ,i N Y i 1 ,••• ,I N .
Moreover, we denote by X 2 = X X.

The q-norm of a tensor X ∈ R I 1 ו••×I N is defined by

X q = ∑ i 1 ,••• ,i N |X i 1 ,••• ,i N | q 1/q if q ∈ N * and X ∞ = max i 1 ,••• ,i N |X i 1 ,••• ,i N | .
For q = 2, this norm comes from the scalar product defined, for X, Y ∈

R I 1 ו••×I N , by X, Y 2 = ∑ i 1 ,••• ,i N X i 1 ,••• ,i N Y i 1 ,••• ,i N .
The space R I 1 ו••×I N is isomorphic to the matrix spaces R I n ×∏ m =n I n for all n ∈ 1, N via the relation which to a tensor X ∈ R I 1 ו••×I N associates its n-th unfolding X (n) ∈ R I n ×∏ m =n I n defined such that

X (n) i n ,j = X i 1 ,••• ,i N ⇔ j = 1 + ∑ m =n (i m -1) m-1 ∏ k=1 k =n I k .
251

For N vectors a (1) ∈ R I 1 , • • • , a (N) ∈ R I N , we define their outer product a (1) • • • • • a (N) as the element of R I 1 ו••×I N such that for all (i 1 , • • • , i N ) ∈ ∏ N n=1 1, I N , by

(a (1) • • • • • a (N) ) i 1 ,••• ,i N = N ∏ n=1 a (n) i n .
The Kronecker product of two vectors a ∈ R I and b ∈ R J is defines as the vector

a ⊗ b =      a 1 b . . . a I b      ∈ R I J
Then, we can define the Kronecker product ⊗ m =n a (n) = a (N) ⊗ • • • ⊗ a (1) ∈ Be careful that the product is taken backwards.

R ∏ N n=1 I N of N vectors a (1) ∈ R I 1 , • • • , a (N) ∈ R I N recursively. This way, we have, for all n ∈ 1, N , (a (1) • • • • • a (N) ) (n) = a (n) ⊗ m =n a (m) ,

Then we define the Khatri-Rao product N n=1 A (n) between N matrices

A (1) = [a (1) 1 , • • • , a (1) R ] ∈ R I 1 ×R , • • • , A (N) = [a (N) 1 , • • • , a (N) R ] ∈ R I N ×R as N n=1 A (n) = ⊗ N n=1 a (n) 1 • • • ⊗ N n=1 a (n) R ∈ R (∏ N n=1 I n )×R .
This was, we have, for all λ ∈ R R , for all n ∈ 1, N , R ∑ r=1 λ r a

(1)

r • • • • • a (N) r (n) = R ∑ r=1 λ r a (n) r ⊗ m =n a (m) r = A (n) m =n
A (m) . ∼ N (0, 1) and (Z t ) t∈I ∈ {0, 1} I are i.i.d and independent of ( Xt , T t ) t∈I and, for t, ∈ I, Xt is independent of ( X , T ) conditionally to T t . We have σ z > 0 and μ : R → R and define θ = ( μ, β 0 , β 1 , σ 2 0 , σ 2 1 ) where β z = P (Z t = z). The goal of this section is to provide an estimator of θ. To this end, we use an EM algorithm where penalty terms are added to enforce μ to be a convex natural cubic spline. Formally, we at the -th step, given the current value of the estimate θ we solve (D.1.5) This optimization problem cannot be solved analytically but we can alternate between the maximization of each variable. The updates for this step are given in Algorithm D.1.1 where we have For the maximization with respect to μ, it is known (see [START_REF] Green | Nonparametric regression and generalized linear models[END_REF] that the L 2 penalization on second derivative of μ implies that the solution is a natural cubic spline which is entirely characterized by the vector µ = ( μ(T t )) t∈I . Defining matrices Q, K, R as in Green and Silverman, 1994, Section 2.1.2, the solution of spline smoothing (σ z , P t,z , T t , Y t ) t∈I,z∈{0,1} is entirely characterized by the solution of min

θ +1 = argmax θ: μ is convex ∑ t∈I E θ( ) [ ln p θ ( Xt , Z t |T t )| Xt , T t ] -λ ( μ ) 2 .
µ∈R |I| ∑ t∈I 1 ∑ z=0 P t,z σ 2 z ( Xt -µ t ) 2 + λµ Kµ such that R -1 Q µ 0 ,
where the constraint comes from the convexity constraint on μ. This quadratic problem can be solved using numerical solvers. In practice, we may not always want to detect two sub-regimes. To this end, in the step update regression, we compare the results obtained by fitting the model with Gaussian mixture noise (D.1.4) and the model with Gaussian noise (D.1.7) and select the one giving the best likelihood. If the model with Gaussian noise is selected, we adjust the estimators of β w,z and σ w,z similarly as in the initializing step. The colors represent the types of days. Normal opening days (W t = 1 and Z t = 0) are in blue, normal closing days (W t = 0 and Z t = 0) are in cyan, extreme opening days (W t = 1 and Z t = 1) are in red and purple (red for a consumption higher than normal and purple for a consumption lower than normal) and extreme closing days (W t = 0 and Z t = 1) are in orange and brown (orange for a consumption higher than normal and brown for a consumption lower than normal). The user can select a day t (in green) whose daily load curve X t (u) is then displayed in the lower panel and can be compared to the daily load curves (and their mean) of all days which correspond to the same weekday (here a Tuesday). Finally, the right panel displays some information about power and energy and shows the energy gain in two scenarios. The first one consists in replacing the power of extreme days by the edge of the distribution of normal days and the second consists in replacing the power of extreme days by the value of the regression curve at the related temperature. In Figure E.2, the user can visualize the functional data after projecting onto a B-spline basis and the centered data. Colored curves correspond to days during which the daily mean temperature is in a temperature window selected by the user (here 16 • C ± 3 • C). In the right panel, the daily load curve and energy of the selected day (in green) are compared to the mean curve (here µ 1 ) and mean energy ( Abstract : The analysis of electrical load curves collected by smart meters is a key step for many energy management tasks ranging from consumption forecasting and load monitoring to customers characterization and segmentation. In this context, researchers from EDF R&D are interested in extracting significant information from the daily electrical load curves in order to compare the consumption behaviors of different buildings. The strategy followed by the group which hosted my doctorate is to use physical and deterministic models based on information such as the room size, the insulating materials or weather data, or to extract hand-designed patterns from the electrical load curves based on the knowledge of experts. Given the growing amount of data collected, the interest of the group in statistical or data-driven methods has increased significantly in recent years. These approaches should provide new solutions capable of exploiting massive data without relying on expensive processing and expert knowledge. My work fits directly into this trend by proposing two modeling approaches: the first approach is based on functional time series and the second one is based on non-negative tensor factorization. This thesis is split into three main parts. In the first part, we present the industrial context and the practical objective of the thesis, as well as an exploratory analysis of the data and a discussion on the two modeling approaches proposed. In the second part, we follow the first modeling approach and provide a thorough study of the spectral theory for functional time series. Finally, the second modeling approach based on non-negative tensor factorization is presented in the third part.

Algorithm

Institut Polytechnique de Paris 91120 Palaiseau, France

  Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  Figure 1.1 Raw data. Load (in blue) is in kW and Temperature (in red) is in • C. . . . . . . . . . . . . . . . . . . . . . . . Figure 1.2 Daily heatmap of load data. At the center, heatmaps of electrical load (in kW) where rows represent days and columns represent intraday time. At the left, daily temperatures, at the right, load for each day at the selected intraday time. At the bottom, daily load curve for the selected day. . . . . . . . . . . . . . . . . . Figure 1.3 Weekly heatmap of load data. At the center, heatmaps of electrical load (in kW) where rows represent weeks and columns represent intraday time and week days. At the left, daily temperatures, at the right, load for each week at the selected week day and intraday time. At the bottom, weekly load curve for the selected week. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 1.4 Decomposition of the load curves with minimum profiles and seasonal variations . . . . . . . . . . . . . . . Figure 1.5 Effect of seasonal variation . . . . . . . . . . . . . . . . Figure 1.6 Pie charts of the decomposition of load curves . . . . . Figure 1.7 Histograms for each features . . . . . . . . . . . . . . . Figure 1.8 Power vs temperature for Monday (in blue) and Satuday (in brown). In orange and green: linear and quadratic regression for Mondays. In purple and gray: linear and quadratic regressions for Saturdays. . . . . Figure 2.1 Weekly heatmaps of power and temperature for one site. Missing data are in gray and values close to 0 (in blue) indicate the use of an external power generator. . Figure 2.2 Illustration of the generator detection method. . . . . Figure 2.3 Imputation of missing values . . . . . . . . . . . . . . . Figure 2.4 Smoothing the daily load curves for a closing day and an opening day. . . . . . . . . . . . . . . . . . . . . . . . Figure 2.5 Daily mean data . . . . . . . . . . . . . . . . . . . . . .

  of A in E. Use second notation if no ambiguity on the space.Part I B A C K G R O U N D A N D E X P L O R AT O R Y A N A LY S I S O F T H E D ATA

Figure 1 . 1 :

 11 Figure 1.1: Raw data. Load (in blue) is in kW and Temperature (in red) is in • C.

Figure 1 . 4 :Figure 1 . 5 :

 1415 Figure 1.4: Decomposition of the load curves with minimum profiles and seasonal variations

Figure 1 . 6 :

 16 Figure 1.6: Pie charts of the decomposition of load curves

Figure 1 . 7 :

 17 Figure 1.7: Histograms for each features

Figure 2 . 1 :

 21 Figure 2.1: Weekly heatmaps of power and temperature for one site. Missing data are in gray and values close to 0 (in blue) indicate the use of an external power generator.

Figure 2 . 2 :

 22 Figure 2.2: Illustration of the generator detection method.

Figure 2 . 3 :

 23 Figure 2.3: Imputation of missing values

Figure 2 . 5 :

 25 Figure 2.5: Daily mean data

  Algorithm 2.3.1: Alternating clustering and regression updates Data: Observations (T t , Xt ) 1≤t≤T and initial value η0 .

Figure 2 . 6 :

 26 Figure 2.6: Daily data regression and clustering. The left graph illustrates the result of the main clustering: blue corresponds to W t = 0 (closing days) and orange corresponds to W t = 1 (opening days). The right graph represents the results of the regression. For each main regime, the regression function is plotted and the sub-regimes are represented in different colors.Blue and orange correspond respectively to Z t = 0 (normal) and Z t = 1 (extreme) when W t = 0 and green and red correspond respectively to Z t = 0 and Z t = 1 when W t = 1.

  Figure 2.7: Illustration of the first centering approach

Figure 2 . 8 :

 28 Figure 2.8: fPCA loadings and scores. Each row corresponds to one functional principal component.

Figure 2

 2 Figure 2.9: fPCA loadings as perturbation of the mean. The black dashed curve is the mean μw , the other lines represent μw + Cφ k for C in the range of scores. Blue and red correspond to C < 0 and C > 0 respectively.

  Figure 2.10: Truncated Karhunen-Loeve expansion

  μW t (u) + ∑ L =-L Y 1,t+ ϕ 1, (u), with different number L of lags.

Figure 2 .

 2 Figure 2.11: Importance of the number of lags in the reconstruction.

Figure 2 .

 2 Figure 2.12: Harmonic fPCA loadings filters coefficients and scores. Each row corresponds to one functional principal component.

Figure 2 . 13 :

 213 Figure 2.13: Harmonic fPCA loading filters as perturbations of the mean for opening days. The dashed black line in the mean curve μ1 and the solid blue line is the perturbation μ1 + C(δ -1 ϕ 1,-1 + δ 0 ϕ 1,0 + δ 1 ϕ 1,1 ).

Figure 2 .

 2 Figure 2.14: Truncated Cramér-Karhunen-Loeve expansion

Figure 2 .

 2 Figure 2.16: Nadaraya-Watson estimate of μw (u; τ). Colors represent values of temperature τ.

Figure 2 .

 2 Figure 2.17: fPCA loadings and scores. Each row corresponds to one functional principal component.

  Figure 2.18: Truncated Karhunen-Loeve expansion

Figure 2 . 19 :

 219 Figure 2.19: Comparison of autocorrelation with the two centering methods

Figure 2 . 20 :

 220 Figure 2.20: Successive plots of µ 1 (•; 15) + C ∑ K k=1 φ k with C > 0 for K = 0, • • • , 3.

  Several denominationare used to refer to p.o.v.m.'s and we chose the most explicit.

  Proposition 4.1.2. A p.o.v.m. ξ on (Λ, A, H) satisfies Condition (ii) of Definition 4.1.6 if and only if, for all A, B ∈ A, ξ(A ∩ B) = ξ(A)ξ(B) .

  Theorem 4.2.1. The dual group Ĝ of an l.c.a. group G is an l.c.a. group when endowed with the compact-open topology.

  Definition 4.2.3 ((Continuous) Unitary representations). Let (G, +) be an l.c.a.

  (i) Γ is continuous in w.o.t. and positive definite. (ii) Γ is continuous in w.o.t. and of positive-type.(iii) Γ is continuous in w.o.t. and hermitian non-negative definite.

FourierFigure 4 . 1 :

 41 Figure 4.1: Possible proof paths for Bochner's, Stone's theorems and their generalizations.

Corollary 4. 5 . 4 .

 54 Let (G, +) be an l.c.a. group, H 0 a separable Hilbert space and Γ : G → L b (H 0 ). Then the following assertions are equivalent.(i) Any of the Assertions (i)-(iii) in Theorem 4.4.2 holds and Γ(0) ∈ S 1 (H 0 ).

  rem 4.4.2 and Lemma 4.5.1. The implications (iii) ⇒ (i) and (iii) ⇒ (ii) are straightforward. It remains to show that (iv) implies (iii) which reduces to showing that for all P ∈ L b (H 0 ), h → Tr(PΓ(h)) is continuous. To this end, let f = dν d ν 1

Remark 5 . 1 . 2 .

 512 Let us briefly comment this definition. 1) In(5.1.3), using the Radon-Nikodym property of the trace-class p.o.v.m. ν, we have thus defined an integral of operator-valued functions with respect to an operator valued measure as a simple Bochner integral in S 1 (I 0 , G 0 ). By Corollary 4.5.3 for a measurable scalar function φ : Λ → C we can interpret the integral φ dν in which ν is seen as an S 1 (H 0 )-valued measure as in Section 4.1 as the same integral as in (5.1.3) with Φ : λ → φ(λ)Id H 0 and Ψ ≡ Id H 0 . Hence the integral (5.1.3) of Definition 5.1.4 can be seen as an extension of the integral of scalar-valued functions to operator-valued functions, with respect to a trace-class p.o.v.m.

Definition 5 . 1 . 5 (

 515 Integral with respect to a random c.a.g.o.s. measure). Under the assumptions of Theorem 5.1.7, we use an integral sign to denote I G 0 W (Φ) for Φ ∈ L 2 (Λ, A, O(H 0 , G 0 ), ν W ). Namely, we writeΦ dW = Φ(λ) W(dλ) := I G 0 W (Φ) . (5.1.10)

  It is in fact better to start with the following proof because Theorem 5

Proposition 5 .

 5 4.1. Let (Z λ ) λ∈[-π,π] be a Gramian-orthogonal increment process as in Definition 5.4.1. Then there exists a unique H 0 -valued random c.a.g.o.s. W on ((-π, π], B((-π, π]), Ω, F , P) such that (5.4.1) holds.

  the c.a.o.s. measures W x and W are σ-additive in M(Ω, F , C, P) and in M(Ω, F , H 0 , P), respectively, see Remark 5.1.1.

  4.8 relies on the two following lemmas where the open and closed complex unit discs of C are respectively denoted by D := {z ∈ C : |z| < 1} and D = {z ∈ C : |z| ≤ 1}. Lemma 6.5.3. Let E be a Banach space and (a n ) n∈N ∈ E N such that a n E ----→ n→+∞ 0 and the series ∑ a na n+1 E converges. Then for all z 0 ∈ D \ {1}, the series n is uniformly continuous on [0, z 0 ].

  2 = a Q 1 a , (7.1.4) with a = [a(u 1 ), • • • , a(u I )] .

  that 0/0 = 0. Then the following result, where and • are the Hadamard and outer products defined in Appendix C. Proposition 7.1.1. The solutions of Problem (7.1.3) are entirely characterized by the solutions of the following penalized weighted NTF problem.

Figure 7

 7 Figure 7.1: EERE's Dataset : factors

  Figure 7.2: EERE's Dataset : site activations

  Figure 7.3: EDF's Dataset : factors

  Definition of C p,q (ρ, I W ) and proof of Proposition 8.1.2

5 C 1 8J 25 C

 5125 n=1 + m i : m ∈ 1, N = c p,q (ρ, (J n ) N n=1 ) : (J n ) N n=1 ∈ while k < |I W | do 9 Pick i ∈ I W \ I k I k+1 = I k ∪ {i} M = {m ∈ 1, N : i m ∈ π m (I k )} for m ∈ M do J (m) = (J n ) N n=1 ∈ J k : i m ∈ J m J k+1 = ∅ C k+1 = ∅ for (J n ) N n=1 ∈ J k do if ∃m ∈ M, (J n ) N n=1 ∈ J (m) then J k+1 = J k+1 ∪ {(J n ) N n=1 } C k+1 = C k+1 ∪ {c p,q (ρ, (J n ) N n=1 )} else for m ∈ 1, N do 22 (J n ) N n=1 = (J n ) N n=1 + m i 23 if m ∈ M and ∀(J n ) N n=1 ∈ J (m) , (J n ) N n=1 ⊂ (J n ) k+1 = J k+1 ∪ {(J n ) N n=1 } k+1 = C k+1 ∪ {c p,q(ρ, (J n ) Some results on TV norm optima Define for n ≥ 1, let us define the n × n matrices

+

  |a K+1a K | p .

  1.2 and that for p ≥ 2, getting an exact value for B n p,p is a NP-Hard problem Hendrickx and Olshevsky, 2010 Corollary 8.7.4. Let p ≥ 1. Let I ≥ K ≥ 1 and letJ = {j 1 , • • • , j K } ⊂ 1, I with j 1 < • • • < j K .Then we have B

I

  For all p ≥ 2, m p,p,I (J ) is lower bounded by 2 ∆ ‡

P

  ∈ S p (H 0 ) ⇔ ∑ n∈N Pe n , e n H 0 p < +∞ . In the case where p = 1, we call the trace of P the value Tr(P) := ∑ n∈N Pe n , e n , which does not depend on the choice of the basis (e n ) n∈N . Another characterization of Hilbert-Schmidt operators is that an operator P ∈ L b (H 0 , G 0 ) is Hilbert-Schmidt if and only if ∑ n∈N Pe n 2 H 0 < +∞. In this case, the value of the sum does not depend in the choice of the basis (e n ) n∈N and we have P

  lim n→+∞ Φ n x, x H 0 = Φx, x H 0 , for all x ∈ H 0 . B M E A S U R A B I L I T Y A N D I N T E G R AT I O N I N B A N A C H S PA C E S

DA

  L G O R I T H M I C D E TA I L S This Chapter gathers practical algorithmic details about the methods introduces throughout this thesis. d.1 Details for the daily means analysis d.1.1 Details for update main clustering The update for the main clustering is a standard Expectation-Maximization (EM) update. The parameter α is updated as αk+1 = argmax α∈R 2 + ,α 0 +α 1 =1 T ∑ t=1

P

  θ ( Z t = z | Xt , T t ) = β z p θ ( Xt |z, T t ) ∑ J z =1 β z p θ ( Xt |z , T t )

  As Algorithms 2.3.1 and D.1.1 are iterative, they require stopping criteria. In both cases the criterion used is based on the relative change in likelihood. In Algorithm 2.3.1, the likelihood is the one corresponding to model (2Xt |w, T t )α w , where p η (x|w, τ) is given by (D.1.3). In Algorithm D.1.1, the likelihood is the one of model (D.1.4), that isp θ (( Xt ) t∈I |(T t ) t∈I ) = ∏ t∈I 1 ∑ z=0 p θ ( Xt |z, T t )β z ,where p θ (x|z, τ) is given by (D.1.6). Finally, in Algorithm D.1.1, the coordinatewise maximization also requires a stopping criterion. In practice we use a fixed number of 5 iterations. d.1.3.4 Selection of the regression noise.

15

  return θ +1 , ( Ẑt ) i∈I E G R A P H I C U S E R I N T E R FA C E S F O R L O A D C U R V E A N A LY S I SAs a first step toward the implementation of the models proposed during my PhD on the ACDC platform, I developed a Graphic User Interfaces (GUI) in python to visualize the results of the algorithm as well as several diagnosis features. This GUI is aimed to be a model for future Javascript implementation on the platform.The results of the analysis on daily means are displayed in Figure E.1. In the upper panel, the user can visualize the power and temperature data as well as the regression curve and residuals obtained by Algorithm 2.3.1.

Figure E. 1 :

 1 Figure E.1: Daily mean analysis in GUI

  . . 1.2 Representation and comparison of time series . . . . . . . . . . 1.2.1 Property-based comparison . . . . . . . . . . . . . . . . . 1.2.2 End-to-end methods . . . . . . . . . . . . . . . . . . . . . 1.2.3 Pattern-based comparison . . . . . . . . . . . . . . . . . . 1.3 The functional time series framework . . . . . . . . . . . . . . . Load curves analysis in the web platform ACDC . . . . . . . . Comparaison par caractéristiques . . . . . . . . . . . . . 1.2.2 Méthodes bout-à-bout . . . . . . . . . . . . . . . . . . . . 1.2.3 Comparison par motifs . . . . . . . . . . . . . . . . . . . 1.3 Le cadre des séries temporelles fonctionnelles . . . . . . . . . . 1.3.1 Cadre fonctionnel vs cadre hilbertien . . . . . . . . . . . 1.3.2 Des données discrètes aux données fonctionnelles . . . . 1.3.3 Alignement des données fonctionnelles . . . . . . . . . . 1.3.4 Analyse en composantes principales fonctionnelle . . . 1.3.5 Séries temporelles fonctionnelles . . . . . . . . . . . . . . Inférence statistiques pour les données fonctionnelles . 1.4 Analyse des courbes de charge sur la plateforme web ACDC . 1.4.1 Visualisation des données . . . . . . . . . . . . . . . . . . 1.4.2 Décomposition des courbes de charge . . . . . . . . . . . 1.4.3 Analyse multi-sites . . . . . . . . . . . . . . . . . . . . . . 1.4.4 Corrélation avec la température extérieure . . . . . . . . Presentation of the dataset and pre-processing . . . . . . . . . . 2.2 The two approaches followed . . . . . . . . . . . . . . . . . . . . 2.3 First order analysis of daily means . . . . . . . . . . . . . . . . . 2.4 Centering without temperature . . . . . . . . . . . . . . . . . . . 2.4.1 Results of fPCA . . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 Results of harmonic fPCA . . . . . . . . . . . . . . . . . . 2.5 Centering with temperature . . . . . . . . . . . . . . . . . . . . . 2.6 Discussion and conclusion . . . . . . . . . . . . . . . . . . . . . .

	2 exploratory analysis of the data
	2.1

1.3.1 Functional vs Hilbertian setting . . . . . . . . . . . . . . 1.3.2 From discrete data to functional data . . . . . . . . . . . 1.3.3 Registration of functional data . . . . . . . . . . . . . . . 1.3.4 Functional Principal Component Analysis . . . . . . . . 1.3.5 Functional time series . . . . . . . . . . . . . . . . . . . . 1.3.6 Harmonic fPCA . . . . . . . . . . . . . . . . . . . . . . . . 1.3.7 Statistical inference for functional data . . . . . . . . . . 1.4 1.4.1 Data visualization . . . . . . . . . . . . . . . . . . . . . . 1.4.2 Decomposition of load curves . . . . . . . . . . . . . . . 1.4.3 Multi-sites analysis . . . . . . . . . . . . . . . . . . . . . . 1.4.4 Correlation with outdoor temperature . . . . . . . . . . . 1 contexte et pr éliminaires 1.1 Contexte . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Représentation et comparaison de séries temporelles . . . . . . 1.2.1 1.3.6 ACPf harmonique . . . . . . . . . . . . . . . . . . . . . . v vi contents 1.3.7

  .1 Weekly heatmaps of power and temperature for one site. Missing data are in gray and values close to 0 (in blue) indicate the use of an external power generator. . Smoothing the daily load curves for a closing day and an opening day. . . . . . . . . . . . . . . . . . . . . . . . Figure 2.5 Daily mean data . . . . . . . . . . . . . . . . . . . . . . Daily data regression and clustering. The left graph illustrates the result of the main clustering: blue corresponds to W t = 0 (closing days) and orange corresponds to W t = 1 (opening days). The right graph represents the results of the regression. For each main regime, the regression function is plotted and the subregimes are represented in different colors. Blue and orange correspond respectively to Z t = 0 (normal) and Z t = 1 (extreme) when W t = 0 and green and red correspond respectively to Z t = 0 and Z t = 1 when W t = 1. . . . . . . . . . . . . . . . . . . . . . . . . Figure 2.7 Illustration of the first centering approach . . . . . . . Figure 2.8 fPCA loadings and scores. Each row corresponds to one functional principal component. . . . . . . . . . . . Figure 2.9 fPCA loadings as perturbation of the mean. The black dashed curve is the mean μw , the other lines represent μw + Cφ

		List of Figures	xi
		Figure 2.6
	Figure 2.2	Illustration of the generator detection method. . . . .
	Figure 2.3	Imputation of missing values . . . . . . . . . . . . . . .
	Figure 2.4
	x

k for C in the range of scores. Blue and red correspond to C < 0 and C > 0 respectively. . . . . . . Figure 2.10 Truncated Karhunen-Loeve expansion . . . . . . . . . . Figure 2.11 Importance of the number of lags in the reconstruction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 2.12 Harmonic fPCA loadings filters coefficients and scores. Each row corresponds to one functional principal component. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 2.13

  . . . . . . . . . Figure 2.14 Truncated Cramér-Karhunen-Loeve expansion . . . . . Figure 2.15 Comparison of Karhunen-Loeve and Cramér-Karhunen-Loève . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 2.16 Nadaraya-Watson estimate of μw (u; τ). Colors represent values of temperature τ. . . . . . . . . . . . . . . . Successive plots of µ 1 (•; 15) + C ∑ K k=1 φ k with C > 0 for K = 0, • • • , 3. . . . . . . . . . . . . . . . . . . . . . . Figure 4.1 Possible proof paths for Bochner's, Stone's theorems and their generalizations. . . . . . . . . . . . . . . . . . Figure 7.1 EERE's Dataset : factors . . . . . . . . . . . . . . . . . . Figure 7.2 EERE's Dataset : site activations . . . . . . . . . . . . . Figure 7.3 EDF's Dataset : factors . . . . . . . . . . . . . . . . . . . Figure 7.4 EDF's Dataset : site activations of our model . . . . . . Figure E.1 Daily mean analysis in GUI . . . . . . . . . . . . . . . . Figure E.2 Visualizing functional data in GUI . . . . . . . . . . . . Figure E.3 fPCA loadings and scores in GUI . . . . . . . . . . . . Review of statistical inference for functional data . . . Table 1.2 Values are in MWh/m 2 . . . . . . . . . . . . . . . . . . . Table 8.1 Accuracy (in %) for randomly generated data, "both" refers to initializing swntf with cpwopt. . . . . . . . .

		L I S T O F TA B L E S
		Table 1.1
	Figure E.4	Truncated Karhunen-Loève expansion in GUI . . . . .
		Figure 2.17	fPCA loadings and scores. Each row corresponds to
			one functional principal component. . . . . . . . . . . .
		Figure 2.18	Truncated Karhunen-Loeve expansion . . . . . . . . . .
		Figure 2.19	Comparison of autocorrelation with the two centering

methods . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 2.20

  . Il est important de noter que ces méthodes ne sont pas destinées à modéliser les mêmes propriétés statistiques des données. Par exemple, l'ACP est une méthode de second ordre qui modélise la variance, tandis que la NMF modélise la moyenne. Pour les séries temporelles, l'utilisation d'une convolution dans (1.2.1) au lieu d'une multiplication peut être intéressante pour prendre en compte la dépendance entre les observations. Pour cette raison, plusieurs modèles de représentation de motifs ont leur contrepartie convolutive. Par exemple,Brillinger (2001, Chapitre 9) présente une ACP dans le domaine fréquentiel qui, dans le domaine temporel, a une forme convolutive avec des séries temporelles de scores décorrélés. L'apprentissage par dictionnaire convolutif est utilisé pour capturer les motifs récurrents d'une série temporelle de manière invariante par rapport au décalage

1.3.2 Des données discrètes aux données fonctionnelles En

  . Comme cela est déjà le cas dans les cas univariés et multivariés, la question de la dépendance entre les observations est importante. Bien que le cadre i.i.d. soit largement utilisé, l'analyse des données fonctionnelles avec dépendance spatiale et/ou temporelle est un domaine de recherche actif dans la communauté de l'ADF depuis les travaux séminauxde Bosq, 2000. Le cadre des séries temporelles fonctionnelles est pratique pour représenter des signaux temporels collectés sur plusieurs jours et o ù les structures de dépendance intrajournalière et interjournalière diffèrent. C'est le cas, par exemple, des données de consommation d'électricité analysées à EDF o ù l'on s'intéresse à des courbes de charge électrique collectées sur une période de 1 an à une fréquence d'échantillonnage de 10 minutes. Pour ce type de données, il est naturel pratique, nous accédons à une donnée fonctionnelle x ∈ L 2 (U ) sous la forme d'une collection de paires (u i , y i ) 1≤i≤I o ù u i ∈ U et y i ∈ R. Par exemple, pour les données d'EDF, I = 144 et (u i ) 1≤i≤I sont uniformément espacés dans [0, 24). Si I est suffisamment petit, représenter les données par le vecteur y := [y 1 , • • • , y I ] n'est pas forcément rédhibitoire d'un point de vue computationnel. Cependant, cette représentation multivariée ne permet pas de capturer la structure sous-jacente des données, notamment leur régularité, car les données brutes peuvent présenter des erreurs et du bruit.

	est un abus de notation qui doit être compris comme E [X] = µ dans L 2 (U ).
	Cet abus de notation est très répandu et généralement inoffensif mais peut,
	dans certains cas, conduire à un raisonnement incorrect.
	Dans ce manuscrit, j'essaie d'être aussi rigoureux que possible pour les
	considérations théoriques en utilisant principalement le cadre de Hilbert et
	en justifiant les déclarations faites avec le cadre fonctionnel. Pour les travaux
	pratiques, un certain abus de notation est autorisé car, lorsqu'il s'agit de
	données réelles, le passage des données discrètes aux données fonctionnelles
	se fait souvent d'une manière qui implique la continuité.
	Pour cette raison, une première étape de l'ADF consiste à récupérer la fonc-
	tion x à partir des observations discrètes. Ce processus est appelé lissage et
	s'appuie généralement sur le modèle
	ou
	lorsque la variable fonctionnelle est considérée comme la trajectoire d'un
	processus stochastique à temps continu. Un inconvénient de ce cadre est
	que l'on doit garder à l'esprit que, si X est une variable aléatoire à valeur
	L 2 (U ), l'évaluation X(u) n'a pas de sens. Dans ce cas, un énoncé comme
	E [X(u)] = µ(u) ; , u ∈ U ; ,

de penser que la dépendance intrajournalière est plut ôt liée à l'activité humaine et que la dépendance inter-journalière est plut ôt liée aux variations saisonnières. Dans ce cadre, nous représentons les données de charge d'un site donné comme un panel de courbes (X t ) 1≤t≤T o ù X t appartient à l'espace de Hilbert séparable L 2 ([0, 24)). Dans ce cas, X t (u) est la consommation instantanée au jour t et au temps intrajournalier u ∈ [0, 24). Dans la section suivante, nous verrons que ce cadre est déjà implicitement utilisé dans la plateforme ACDC. Dans le reste de cette section, nous fixons un espace de probabilité (Ω, F , P) et passons en revue les aspects fondamentaux de l'ADF et des séries temporelles fonctionnelles.

1.3.1 Cadre fonctionnel vs cadre hilbertien

Il existe une dualité entre les données fonctionnelles et les données à valeurs dans un espace de Hilbert séparable. D'un point de vue théorique, les deux cadres sont équivalents puisque tout espace de Hilbert séparable est isomorphe à l'espace de fonctions L 2 (U ). Cependant, certains résultats peuvent être plus faciles à interpréter dans un cadre que dans l'autre. Le cadre hilbertien est plus abstrait mais a l'avantage de fournir des résultats plus élégants sans utiliser d'opérateurs intégraux, comme c'est souvent le cas dans le cadre fonctionnel. Au contraire, le cadre fonctionnel peut conduire à des résultats plus fins, notamment lorsque la continuité est impliquée

i ) 1≤i≤I représente le bruit. 1.3.2.1 Lissage non paramétrique

			• • • , I ; ,	(1.3.1)
	o ù ( Le modèle présenté dans (1.3.1) est très courant en statistique non paramétrique
	et, par conséquent, de nombreuses méthodes non paramétriques peuvent
	être appliquées au problème du lissage. Par exemple, x(u) peut être estimé
	par la moyenne des valeurs y i pour les i tel que u i est dans un voisinage de
	u. Cette méthode, appelée lissage par noyau estime x(u) par
		I
	x(u) =	∑	κ(u, u i )y i ; ,
		i=1
	o ù κ est une fonction positive telle que κ(u, v) tend vers 0 lorsque |u -v| tend
	vers 0. Un exemple typique est le noyau gaussien κ(u, v) = exp -|u-v| 2 2σ 2

. 1.3.2.2 Lissage paramétrique La

  méthode de lissage la plus courante en ADF consiste à représenter la fonction x comme une combinaison linéaire de fonctions linéairement indépendantes (mais pas nécessairement orthogonales) (v k ) 1≤k≤K ⊂ L 2 (U ) appelées fonctions de base. En particulier, nous supposons qu'il existe c 1

		u)	2 du ; .
	Ce problème d'optimisation est étudié dans Green and Silverman (1994,
	Chapitre 2). En particulier, la solution est nécessairement une spline cubique
	naturelle et est entièrement caractérisée par le vecteur x = [ x(u 1 ), • • • , x(u I )] T
	qui est défini comme suit
	x = argmin x∈R I	y -x 2 2 + λx Kx ; ,
	avec une matrice appropriée K

.4 Analyse en composantes principales fonctionnelle

		Ce problème d'optimisation peut être résolu pour certains types de fonc-
		tions de déformation. Par exemple, le critère des moindres carrés est utilisé
		dans Ramsay and Silverman, 2005, Section 7.2.1 dans le cas o ù h t est la fonc-
		tion de translation du paramètre δ t , c'est-à-dire h t (u) = u -δ t . D'autres
		méthodes d'enregistrement plus avancées existent (Srivastava and Klassen,
		2016) mais dépassent le cadre de ce manuscrit. Dans le cas des données
		EDF, le seul problème d'alignement observé est causé par les changements
		d'heure d'été et d'hiver (voir Figures 1.2 and 1.3) et peut être corrigé par un
		simple décalage d'une heure.
		1.3L'analyse en composantes principales fonctionnelle (ACPf) est l'un des out-
		ils statistiques les plus utilisés dans l'ADF, car elle fournit la meilleure ap-
		proximation à dimension finie des données pour l'erreur quadratique. D'un
		point de vue général, on peut considérer l'ACPf comme la succession d'une
		étape d'encodage linéaire et d'une étape de décodage linéaire. Le r ôle de
		l'encodage est de représenter les données sous la forme d'un vecteur de di-
		mension K et le r ôle du décodage est de renvoyer les données encodées dans
		l'espace original. Plus précisément, si X est une variable aléatoire centrée
		dans L 2 (Ω, F , H 0 , P), o ù H 0 est un espace de Hilbert séparable, et K ∈ N * ,
	Nous rappelons que	le problème s'écrit comme suit : .
	L b (E, F) désigne
	l'espace des opérateurs
	bornés entre deux
	espaces de Banach E et
	F et
		2 du ; .

  ( X, φ k H 0 ) k≥1 est une suite univariée décorrélée. Dans la suite, les vecteurs φ k et les produits scalaires X, φ k H 0 seront appelés respectivement les fonctions principales et les scores.

	Par conséquent, (Ψ, Φ) est une solution de (1.3.4).
	En d'autres termes, la meilleure approximation de X par une transforma-
	tion linéaire de dimension finie est une version tronquée de la décomposition
		+∞
	X =	∑ k=1	X, φ k H 0 φ k ; ,	(1.3.8)
	o ù la série converge dans L 2 (Ω, F , H 0 , P). Cette décomposition peut être vu
	comme la version hilbertienne de la décomposition de Karhunen-Loève d'un
	processus stochastique L 2 à temps continu (voir Bosq (2000, Theorem 1.5)).
	En particulier, Comme pour l'ACP, si nous observons un ensemble de données fonction-
	nelles (X t ) 1≤t≤T satisfaisant Cov(X t ) = Cov(X) pour tout 1 ≤ t ≤ T, le
	critère des moindres carrés des équations (1.3.4) et (1.3.5) et l'opérateur de
	covariance sont remplacés par leurs équivalents empiriques. En pratique,
	en supposant que H 0 = L 2 (U ) et que nous n'avons accès qu'à une ver-
	sion discrète de la fonction X t , nous devons ajouter une deuxième couche
	d'approximation. Dans ce cas, il existe deux fac ¸ons d'effectuer une ACPf. La
	première consiste à utiliser uniquement les points d'échantillonnage et à es-
	timer le produit scalaire X t , φ k H 0 = U X t (u)φ k (u) du par une intégrale de
	Riemann. Si les points d'échantillonnage sont régulièrement espacés dans
	U , cette approche est équivalente à l'application d'une ACP aux vecteurs
	représentant les fonctions discrétisées. Cette méthode ne fournit qu'une
	estimation de la composante fonctionnelle principale évaluée aux points
	d'échantillonnage. Pour récupérer la fonction, il faut alors utiliser une méthode
	d'interpolation. Une alternative consiste à ajouter un terme de pénalité sur
	la norme L 2 de la dérivée seconde de ϕ k , ce qui permet d'obtenir des estima-
	teurs de type spline cubique naturels. L'autre méthode, la plus utilisée, pour
	effectuer une ACPf en pratique consiste à s'appuyer sur une représentation
	du type (1.3.2). Considérons une base de fonctions

  (Ω, F , H 0 , P) et F φ k ⊗φ k est le filtre de fonction de transfert λ → φ k (λ) ⊗ φ k (λ). De plus, Les séries temporelles F φ k ⊗φ k (X) et F φ ⊗φ (X) sont décorrélées pour = k.

	o ù la série converge dans L 2 En écrivant Θ(λ) = Ψ(λ)Φ(λ) comme dans (1.3.7), nous pouvons prou-
	ver que le filtre F Θ est une composition des deux filtres F Ψ et F Φ . Pour
	cette raison, l'ACPf harmonique peut également être considérée comme un
	modèle de codage/décodage o ù l'encodeur et le décodeur sont des filtres
	linéaires. Ceci est discuté dans Tavakoli (2014, Section 2.8.2) et H örmann,
	Kidzi ński, and Hallin, 2015 o ù la version dans le domaine temporel de la
	décomposition de Cramér-Karhunen-Loève est données. Si nous appelons
	ϕ k, le -ième coefficient de Fourier de la fonction φ k , alors le k-ième terme
	dans la somme de (1.3.15) s'écrit comme la composition de deux filtres con-
	volutifs.
				.3.14)
	o ù (φ k (λ)) k≥1 sont les vecteurs propres de f X (λ) classés par ordre décroissant
	des valeurs propres associées. Puisque f X est la contrepartie spectrale de
	Cov(X), la fonction de transfert définie par (1.3.14) est la contrepartie spec-
	trale de la projection définie dans (1.3.6), d'o ù le nom ACPf harmonique.
	De même que pour l'ACPf, si nous laissons K → +∞, nous obtenons une
	décomposition de X qui est la contrepartie spectrale de la décomposition
	de Karhunen-Loeve (1.3.8) et est appelé décomposition de Cramér-Karhunen-
	Loève Tavakoli (2014, Theorem 2.8.6). A savoir,
		∞
	X t =	∑ k=1	F φ k ⊗φ k (X) t , t ∈ Z ,	(1.3.15)

  .3.16) et (Y k,t ) t∈Z et (Y ,t ) t∈Z sont décorrélées si = k. Dans la suite, les vecteurs φ k, et les scalaires Y k,t seront appelés respectivement les coefficients des filtres principaux et les scores.

1.4.2 Décomposition des courbes de charge

  Dans le graphique au centre de la Figure1.2, chaque ligne représente un jour de l'année et le graphique à gauche représente la température moyenne pour chaque jour. L'utilisateur peut sélectionner un jour et une heure en particulier (ici le 11 janvier 2019 à 11h10).

	et des vacances d'été pour lesquelles la variation saisonnière est négative
	comme observé dans Figure 1.4.
	.1. Cette représentation peut être utile pour comprendre le comportement général, mais d'autres représentations sont plus adaptées pour mettre en évidence les caractéristiques importantes du signal. C'est le cas des graphes d'intensité quotidiens et hebdomadaires affichés dans les de charge journalière du jour sélectionné s'affiche dans le graphique du bas. Le graphique de droite montre la consommation observée chaque jour à l'heure intrajournalière sélectionnée (ici 11h10). En bref, le graphique du bas et le graphique de droite sont respectivement une tranche horizontale et une tranche verticale du graphique central. Le graphe d'intensité hebdomadaire de la Figure 1.3 se lit de la même manière. Cette dernière représentation est très utile pour des sites comme celui présenté car elle met en évidence le fait qu'il existe deux régimes. Ici, le site est un bureau et nous pouvons interpréter les deux régimes comme des jours ouvrables (ou d'ouverture) et non ouvrables (ou de fermeture). Nous pouvons également voir l'influence des vacances d'été car moins d'employés sont présents au bureau. Outre ces graphiques, la plateforme propose d'autres outils de visualisation pour analyser plus en détail chaque jour de la semaine (par exemple, dérivée temporelle, moyenne et autres statistiques, plage de valeurs). Pour une étude plus quantitative, la plateforme propose un algorithme per-mettant d'extraire des informations des courbes de charge. Considérons un jour de la semaine d et la collection de courbes de charge journalières X d t (u) : u ∈ [0, 24) pour t = 1, • • • , T d , o ù T d est le nombre de fois o ù le jour d est observé. Bien s ûr, en pratique, nous observons une version une fonction du temps intrajournalier est plus intuitif. L'algorithme décompose alors la courbe de charge comme suit. X d t (u) = P d (u) + V d t (u) ; , (1.4.1) o ù P d (u) et V d t (u) sont appelés respectivement le profil minimal et la vari-ation saisonnière. Le profil minimal vise à estimer la courbe de charge jour-nalière que nous obtiendrions pour le jour de la semaine d si le bâtiment était placé dans le vide. Le profil minimal capture donc les caractéristiques du bâtiment en éliminant l'influence du temps ou de tout facteur externe. Cette influence est capturée par la variation saisonnière. Par exemple, pour le site présenté dans la Figure 1.3, si d est un lundi, le calcul de P d (u) ne tiendra pas compte des quelques lundis o ù le bureau est fermé (e.g. en novembre) Les profils minimaux sont ensuite décomposés comme suit. P d (u) = H d + A d (u) ; , o ù H d ≈ min u∈[0,24) P d (u) ; . (1.4.2) Les deux termes H d , et A d (u) de (1.4.2) sont respectivement appelés le talon et le activité. Dans l'exemple de la Figure 1.4, le talon représente la con-sommation la nuit lorsque les bureaux sont vides et l'activité reflète l'effet de l'activité humaine dans les bureaux. Enfin, les variations saisonnières sont divisées en variation saisonnière d'été et variation saisonnière d'hiver pour calculer des statistiques telles que la moyenne ou certains quantiles. La période estivale est fixée par défaut du 1er avril au 30 septembre. L'utilisateur peut alors visualiser l'effet des variations saisonnières comme le montre la Figure 1.5 pour les lundis et les samedis. On constate, par exemple, que la variation saisonnière est très faible en été mais peut être élevée en hiver, no-tamment le week-end. Pour quantifier l'importance de chaque terme dans la décomposition de la courbe de charge, des diagrammes camembert sont fournis pour chaque jour de semaine d individuellement ou conjointement, voir Figure 1.6. Formellement, les diagrammes camembert des figures 1.6a et 1.6b représentent la décomposition (en kWh) T ∑ t=1 24 0 X d t (u) du = 24H d + A d + ∑ s∈{winter, summer V d s :, (1.4.3) o ù A d = 24 0 A d (u) du et V d s = ∑ season(t)=s 24 0 V d t (u) du, aux jours de la Figures 1.2 and 1.3. La courbe échantillonnée de X d t (u 1 ), • • • , X d t (u I ) mais voir la courbe de charge comme semaine d =

lundi et samedi. Le diagramme camembert de la Figure 1.6c représente la décomposition T

1 Vector valued and Positive Operator Valued Mea- sures

  In this section, we summarize important results about vector valued measures. A nice overview on the topic can be found in Dinculeanu (2011, Chapter 1) and more thorough study is proposed inDinculeanu, 1967, Diestel and[START_REF] Diestel | Vector measures. With a foreword by B. J. Pettis[END_REF]. Next, we introduce the notion of Positive Operator Valued Measures (p.o.v.m.'s) which is a key element to generalize Stone's and Bochner's theorems for normal Hilbert modules. Details can be found inBerberian,

	1966b.

93 4.4.1.1 Measures valued in a Banach space Definition 4.1.1. Let (Λ, A) be a measurable space and (E, • E ) a Banach space. An E-valued measure is a mapping µ : A → E which is σ-additive i.e. for any sequence

  (ii) The operator U 0 is the identity operator on H, that is, U 0 = Id H , and, for all s, t ∈ G, U s+t = U s U t . The mapping h → U h is continuous on G for the w.o.t. Note that a mapping valued in the set of unitary operators is continuous for the w.o.t. if and only if it is continuous for the s.o.t.Hence, a c.u.r. is continuous for the s.o.t. as a consequence of (iii). (Hermitian non-negative definite function). A function γ : G → C defined on an l.c.a. group (G, +) is said to be hermitian non-negative definite if for all n

	We say that U is a continuous unitary representation (c.u.r.) if it moreover
	satisfies
	(iii) Remark 4.2.1. Definition 4.2.4

  Definition 4.3.3 (normal Hilbert L b (H 0 )-module). A normal pre-Hilbert L b (H 0 )module is said to be a normal Hilbert L b (H 0 )-module if it is complete (for the norm defined by x 2 H = x, x H = [x, x] H 1 ). (Submodules and L b (H 0 )-linear operators). Let H 0 be a separable Hilbert space and H, G be twoL b (H 0 )-modules. Then a subset of H is called a submodule if it is an L b (H 0 )-module. An operator F ∈ L b (H, G) is said to be L b (H 0 )-linear if for all P ∈ L b (H 0 ) and x ∈ H, F(P • x) = P • (Fx).In the case where H is a normal pre-Hilbert L b (H 0 )-module, we denote, for any E ⊂ H, Span H (E) the smallest linear subspace of H which contains E and is closed for the norm • H . It is a submodule of H.

	Definition 4.3.4

Definition 4.3.5

(Gramian-isometric operators)

. Let H 0 be a separable Hilbert space, H, G be two pre-Hilbert L b (H 0 )-modules and U :

5.2 The general Bochner theorem for trace-class

p.o.v.m.'s

Let (G, +) be an l.c.a. group and H 0 a separable Hilbert space and consider a function Γ : G → L b (H 0 ) satisfying (4.4.2) where ν is a p.o.v.m. on (Λ, A, H 0 ). Then, in particular Γ(0) = ν(Λ) and Lemma 4.5.1 gives that ν is a trace-class p.o.v.m. if and only if Γ(0) ∈ S 1 (H 0 ). Interestingly, in this case, we also get that Γ(h) ∈ S 1 (H 0 ) for all h ∈ G and that Γ becomes continuous in a stronger sense. This is stated in the following corollary.

  Properties of the eigendecomposition of a trace-class p.o.v.m.). Let H 0 be a separable Hilbert space with dimension N ∈ {1, . . . , +∞}. Let ν be a trace-class p.o.v.m. on (Λ, A, H 0 ) and µ a σ-finite dominating measure of ν, e.g. its variation norm ν 1 . Let f

	5 and the
	following lemma
	Lemma 6.2.1 (

  , (6.2.2) where ( Fφ n ⊗φ n ( X)) 0≤n<N are uncorrelated random c.a.g.o.s. measures on ( Ĝ, B( Ĝ), H 0 ) and ( Fφ H n ( X)) 0≤n<N are uncorrelated C-valued c.a.o.s. measures.

  .1.8) as soon as C n = ρ n C with ρ and C satisfying the assumptions of Proposition 8.1.2. This optimization problem is non-convex and has non-convex constraints which makes it difficult to solve. In the next section, we propose a Hierarchical Alternating Least Squares (HALS) approach.

  as defined in (8.0.1), so that a

	(n) r	TV,p	= L n a	(n) r	p	. Then
	the HALS iterations are described in Algorithm 8.2.1.
	Algorithm 8.2.1: HALS algorithm
	Data: X, W, L

n , C n for n ∈ 1, N and initial values for

  . Since HALS is an iterative algorithm, it is usually not necessary to obtain an very sharp approximation at each step. For this reason we propose to first try the most straightforward update (8.2.3) and use the one of two other approaches if the first one fails.In the case where C n < +∞, the problem becomes harder because of the additional constraint. We propose two methods, the first one based ADMM and the second one based on the penalized version of the problem. C (x) = 0 if x ∈ C and +∞ otherwise. We propose to use the accelerated version of ADMM with an adaptive restart described in[START_REF] Goldstein | Fast alternating direction optimization methods[END_REF], Algorithm 8 and recalled in Algorithm 8.2.2. The update in a requires the minimization of a quadratic function on S +

	For the ADMM method, Problem (8.2.2) is reformulated as
	min a,z	1 2	a Ma + b a + I S + In,q	(a) + I [0,C n ] ( z p ) s.t. z = L n a ,	(8.2.4)
	where I

8.2.2 Case where C n < +∞

  Using Proposition 8.4.1, we can relax the constraints while keeping the existence of a global optimum by minimizing L W + P α which means that, in the HALS algorithm, we only optimize on the positive sphere. The problem is similar to the one of Yokota, Zhao, and Cichocki, 2016, but with non-negativity constraints. Investigating the effect of non-negativity constraints on the performance of the method developed in Yokota, Zhao, and Cichocki, 2016 would be interesting.3. The cpwopt method used in Section 8.3 and based on Acar, Dunlavy, Kolda, and Mørup, 2011 has the advantage of solving an optimization problem on the unnormalized factors which, therefore, only has the constraint of nonnegativity. Moreover, it seems that a gradient-based method is well suited in this context. However, with this formulation, no guarantee on the existence of a global optimum can be obtained. However, from Proposition 8.4.1, we can find an optimization problem on unnormalized factors which can be solved by a gradient method and has a global optimum. The problem consists in minimizing the function (L

W + P α ) • ψ where we take d = p and ψ : ∏

  8.1.2. Let C ≥ C p,q (ρ, I W ) and let us show that (8.1.5) does not hold. Since I W = I, Lemma 8.5.2 gives that there exist

  E ηk ln p (α, θk ) ( Xt , W t |T t ) T t , Xt , ηk ( W t = w | T t , Xt ) , t = 1, • • • , T . (D.1.2)The update (D.1.2) requires computing the distribution of W t given ( Xt , T t ) under a parameter η which. By Bayes's rule, we get for all w ∈ {0, 1},P η ( W t = w | T t , Xt ) = α w p η ( Xt |w, T t ) ∑ 1 w =0 α w p η ( Xt |w, T t ) ηk ( W t = w | T t , Xt ) ∑ 1 w =0 ∑ T t=1 P ηk ( W t = w | T t ,Xt ) Fix a w ∈ {0, 1}, then the update performed in this step correspond to fitting the following regression model with Gaussian mixture residuals on the time indices t such that Ŵk t = w. Xt = μw (T t ) + σ w,Z t t , t ∈ I k w . Equivalently, we can reason independently of the main updates Algorithm 2.3.1 by removing w in the notations and considering the model Xt = μ(T t ) + σ Z t t , t ∈ I , (D.1.4)

								(D.1.1)
	and the main clustering labels are updated as
	Ŵk+1 t	= argmax
			w∈{0,1}
								,
	where
	p η (x|w, τ) =	1 ∑ z=0	β w,z 2πσ 2 w,z	exp -	(x -μw (τ)) 2 w,z 2σ 2	.	(D.1.3)
	The update (D.1.1) is easily derived and writes as
	αk+1 w	=	∑ T t=1 P

P

, w ∈ {0, 1} .

d.1.2 Details for update regression

where ( t ) t∈I iid

  Observations ( Xt , T t ) t∈I and initial value θ(0) .P t,z ( θ ) = P θ ( Z t = z | Xt , T t ) for all t ∈ I ∑ t∈I P t,z ( θ ) Xtμ (T t ) +1 ( Z t = z | Xt , T t ) .

		← 0
		repeat
		/* Coordinate-wise maximization for regression	*/
		repeat
		for z ∈ {0, 1} do
	5
		/* Maximization with respect to σ z	*/
	6
			1/2
		σ +1	.
		/* Maximization with respect to μ	*/
		μ +1 ← spline smoothing	σ +1
			.
	11	← + 1
	12 until Stopping criterion;
		/* Cluster data	*/
	13 for t ∈ I do
	14
		Ẑt ← argmax z∈{0,1}	P θ

D.1.1: spline mixture noise Data: z ← 2 ∑ t∈I P t,z ( θ ) z , P t,z ( θ ), Xt , Z t t∈I,z∈{0,1} . until Stopping criterion; /* Maximization with respect to β 0 , β 1 */ for z ∈ {0, 1} do 10 β +1 z ← ∑ t∈I P t,z ( θ ) ∑ t∈I ∑ J z =1 P t,z ( θ )

Recall that T = R/2πZ and that S 1 (H 0 ) is the set of trace-class operators on H 0 .

(a) Summer (b) Winter (c) Open Sunday

This is because a c.u.r. clearly satisfies (iii). Hence, once we know that (iv) holds, the remaining of the proof of Stone's theorem reduces to showing that, if (Γ(h)) h∈G is an u.r., then ν is a projection-valued measure.

until Change of f W (A, B, C) is sufficiently small;

0 µ 1 (u)du). For the day displayed,

An H 0 -valued ARMA(p, q) process X can thus be concisely defined as follows.

Definition 6.4.2 (Hilbert valued ARMA process). Let H 0 be a separable Hilbert space, p, q be non-negative integers, θ ∈ P q (H 0 ), φ ∈ P * p (H 0 ) and Z be a (centered) H 0 -valued white noise. The H 0 -valued weakly stationary time series with spectral representation given by X(dλ) = [φ(e -iλ )] -1 θ(e -iλ ) Ẑ(dλ) , where Ẑ is the spectral representation of Z, is called an ARMA(p, q) process.

By Proposition 6.1.1, in this case, if Σ denotes the covariance operator of Z, then X admits the spectral density g X (λ) = [φ(e -iλ )] -1 θ(e -iλ )Σ[φ -1 (e -iλ )θ(e -iλ )] H with respect to the Lebesgue measure. The following results will be useful. Proposition 6.4.2. Let H 0 be a separable Hilbert space and X be an ARMA(p, q) process defined by X(dλ) = [φ(e -iλ )] -1 θ(e -iλ ) Ẑ(dλ) with θ ∈ P q (H 0 ), φ ∈ P * p (H 0 ) and Z an H 0 -valued white noise with covariance operator Σ. Then there exists η ∈ (0, π) and k : (-η, η) → S 2 (H 0 ) continuous and bounded such that, for Leba.e. λ ∈ (-η, η), we have g X (λ) = h(λ) [h(λ)] H with h(λ) = [φ(1)] -1 θ(1)Σ 1/2 + λ k(λ) . (6.4.8)

In the case where H 0 = L 2 (V, V, ξ) for some σ-finite measure space (V, V ), then the

(6.4.9)

The following lemma indicates that an invertible linear transform of an ARMA process is still an ARMA process. It will be useful in particular in the case where U is an isometry mapping H 0 to a space of the form G 0 = L 2 (V, V, ξ). Lemma 6.4.3. Let ξ be a σ-finite measure on (V, V ), H 0 and G 0 be two separable Hilbert spaces. Let X be an ARMA(p, q) process defined by X(dλ) = [φ(e -iλ )] -1 θ(e -iλ ) Ẑ(dλ) with θ ∈ P q (H 0 ), φ ∈ P * p (H 0 ) and Z an H 0 -valued white noise. Then, for any invertible operator U ∈ L b (H 0 , G 0 ), the process UX = (UX t ) t∈Z is the G 0 -valued ARMA(p, q) process defined by

where θ := UθU -1 ∈ P q (G 0 ) and φ := UφU -1 ∈ P * p (G 0 ), and UZ = (UZ t ) t∈Z is a G 0 -valued white noise. Definition 8.1.1. Let Θ be an unbounded subset of R d and f : R d → R. Then we say that f is coercive on Θ if

In the case where W has only non-zero entries, the arguments of [START_REF] Lim | Optimal solutions to non-negative parafac/multilinear nmf always exist[END_REF][START_REF] Lim | Nonnegative approximations of nonnegative tensors[END_REF] show that the function L W defined in (8.1.4) is coercive on the set of constraints defined in Problem (8.1.3). In the case where W has some zero entries, we have to restrict the set of constraints as stated in the following proposition whose proof can be found in Section 8.5.1. (8.1.5) where we recall that I W is the set of indices where W does not vanish and that A n is the closure of A n in S + I n ,q .

Proposition 8.1.1 implies that, in order to guarantee the existence of a global minimum, we should minimize L W with the constraint (8.1.5). Unfortunately, this constraint is not easily applicable and should be relaxed. To this end, we propose to use smoothness constraints since, intuitively, if the factors are smooth enough, they should not vanish too much and therefore (8.1.5) can hold. Formally, we look for constants

I n ,q : a TV,p ≤ C n , where p, q ∈ [1, +∞). Note that we allow C n to be equal to +∞ in case we do not want to impose any smoothness constraints on the n-th factor. To this end, we take

We assume that ρ is given by the user and look for a suitable C.

Let us define for all

Consider the following assumption for a pair (I , N ) with I ⊂ I and N ⊂ 1, N .

Algorithm 8.2.2: Fast ADMM with restart for Problem (8.2.4)

Algorithm 8.2.3: Iterating over penalized versions of Problem (8.2.2)

Experimental results

To evaluate the method, we construct a noisy tensor of the type

r and E has a standard normal entries. The factors a (2) r are constructed by taking the non-negative part of random linear combination of 7 B-Spline functions of order 4. We generate randomly 60 tensors for each dimension I = 30, 50 and rank R = 3, 6. For 30 of these tensors, the factors a

r have only strictly non-negative values (hence satisfy (8.1.5)) and for the 30 others they can vanish on some intervals. We also generate 10 binary masks representing missing values for each pattern (missing entries and missing fibers) and for several percentages p missing of missing data. Some noise is added to the true tensor and its standard deviation σ > 0 is computed as in [START_REF] Acar | Scalable tensor factorizations for incomplete data[END_REF]Tomasi and Bro, 2005, i.e. σ = (100/ν -1) -1/2 X 2 E 2 with ν = 2. With this setting, for each dimension, each rank and each missing data percentage, we run 600 experiments (60 tensors and 10 masks). In all of the experiments, we take ρ = [I -1 , I -1 , ∞] and p = q = 2 and for each mask W, we compute a lower bound on C ≤ 0.9 × C p,q (ρ, I W ) using Corollary 8.7.4, Lemma 8.6.3 and Algorithm 8.6.1.

We compare our method, referred to as swntf (for Smooth Weighted Non-Negative Tensor Factorization) with the cpwopt method of Acar, Dunlavy, Kolda, and Mørup, 2011. The cpwopt methods solves the least-squares problem related to tensor factorization and, using L-BFGS-B as a first order solver, the non-negativity constraints can be imposed. However, the unit norm and bounded TV-norm constraints are not added. In both cases, an SVD-based initialization is used. For swntf, the number of iterations and the minimum relative improvement of the residuals norm in Algorithm 8.2.1 are respectively set to 10 3 and 10 -6 . For cpwopt, the number of iterations and the minimum norm of the gradient in L-BFSG-B are respectively set to 10 3 and 10 -8 .

To evaluate the output

r , we define the similarity score between the true parameter θ = (λ, A (1) , • • • , A (N) ) and the estimated one

,

a.1.1 General definitions of operators on Hilbert spaces

Let (H 0 , •, • H 0 ) and (G 0 , •, • G 0 ) be two Hilbert spaces. Then, for any P ∈ L b (H 0 , G 0 ) there exists a unique operator P H ∈ L b (G 0 , H 0 ), called the adjoint of P, which satisfies Px, y G 0 = x, P H y H 0 , for all x ∈ H 0 and y ∈ G 0 .

In particular, we have P L b (H 0 ,G 0 ) = P H L b (G 0 ,H 0 ) . In the case where H 0 = G 0 , an operator P ∈ L b (H 0 ) satisfying Φ H = Φ is called auto-adjoint and, if for all x ∈ H 0 , Φx, x H 0 ≥ 0, we say that Φ is positive, which we write Φ 0. A positive operator is necessarily auto-adjoint. For any set E ⊂ L b (H 0 ), we denote by E + the set of positive operators in E . For any P ∈ L b (H 0 , G 0 ) we have then

. A converse property is that for any P ∈ L + b (H 0 ), there exists a unique operator in L + b (H 0 ), denoted by P 1/2 which satisfies P = (P 1/2 ) 2 . This enable us to define the absolute value the an operator P ∈ L b (H 0 , G 0 ) by |P| := (P H P) 1/2 .

We define for x ∈ H 0 and y ∈ G 0 , the operator in L b (H 0 , G 0 ).

The operator y ⊗ x has rank 1 since its range is Im(y ⊗ x) = Span (y). The set of finite rank operators can we written as Span ({y ⊗ x : x ∈ H 0 , y ∈ G 0 }).

A bounded operator P ∈ L b (H 0 , G 0 ) is said to be compact is every bounded sequence (x n ) n∈N ∈ H N 0 admits a subsequence (x n k ) k∈N such that Px n k converges in H 0 . We denote by K(H 0 , G 0 ) the set of compact operators from H 0 to G 0 . This set is characterized by the singular values decomposition theorem (see e.g. Theorem 1.1 in Gohberg, Goldberg, and Kaashoek, 1990, Chapter VI).

Theorem A.1.1. Let H 0 , G 0 be two Hilbert spaces and P ∈ L b (H 0 , G 0 ). Then P is compact if and only if there exist a sequence (σ n ) n∈N ∈ R N + and two orthonormal systems (φ n ) n∈N H 0 and (ψ n ) n∈N ⊂ G 0 such that

where the series converges in operator norm. In this case, the sets {φ n : n ∈ N, σ n > 0} and {ψ n : n ∈ N, σ n > 0} are Hilbert bases of (ker P) ⊥ and ImP respectively. We call (A.1.2) the singular values decomposition (SVD) of P. It is usual to take σ n 's measurability and integration in banach spaces

(ii) The function f is Pettis-measurable i.e. for all φ ∈ E * , φ • f is measurable.

(iii) The function f is Bochner-measurable i.e. is the pointwise limit of a sequence of simple functions (which are in particular Borel-measurable).

(iv) The function f is the uniform limit of a sequence of countably valued Borelmeasurable functions.

The equivalence (ii) ⇔ (iii) is known as Pettis's measurability theorem.

In the special case where E is a space of operators, we introduce another notion of measurability. Consider E, F two Banach spaces. Then a function Φ : Λ → L b (E, F) is said to by simply measurable if for all x ∈ E, the function Φx : λ → Φ(λ)x is Bochner-measurable. We denote by F s (Λ, A, E, F) the space of simply measurable functions from (Λ, A) to L b (E, F). Then, we clearly have, for all E → L b (E, F),

In general the equality does not hold, but it holds in some cases as stated in the following lemma.

Lemma

. By Pettis's measurability theorem, this implies that it is enough to show that for all f ∈ E * , f • Φ is a measurable complexvalued function. By Conway, 2000, Theorems 19.1, 18.14, 19.2 we get that K(H 0 , G 0 ) * , S 1 (H 0 , G 0 ) * and S 2 (H 0 , G 0 ) * are respectively isometrically isomorphic to S 1 (H 0 , G 0 ), L b (H 0 , G 0 ) and S 2 (H 0 , G 0 ) and the duality relation can be defined on E × E * as (P, Q) → Tr(Q H P). This means that we only have to show measurability of the complex-valued functions λ → Tr(P H Φ(λ)) for all P ∈ E * . Let (φ k ) k∈N be a Hilbert basis of H 0 , then Tr(P H Φ(λ)) = ∑ k∈N Φ(λ)φ k , Pφ k G 0 which defines a measurable function of λ by simple measurability of Φ.

d.1.3 Practical implementation details

In this section, we discuss practical implementation details such at the initialization and stopping criteria. The penalty parameter λ in (D.1.5) will influence the smoothness of the function μ obtained. For spline smoothing a closed form of the cross validation error is provided in Green and Silverman, 1994, Eq. (3.27) but this formula does not hold with the additional convexity constraint. Selecting the penalty parameter using cross validation would therefore be computationally costly as it requires to fit the model for multiple values of the parameter. In practice, we observed that the convexity constraint already imposed regularity on the function μ and that the value of λ did not have a great influence on the final result. In our experiments we took λ = 0.5.

d.1.3.2 Initialization.

Algorithm 2.3.1 requires an initial value η0 for the estimator of η. To this end, we perform the following steps.

1. Set Ŵ0 1:T as the result of K-means with 2 clusters performed on X1:T and set for each w ∈ {0, 1}, α0 w = 1 T ∑ T t=1 1 Ŵ0 t =w .

2. For each w ∈ {0, 1} fit a spline regression model on I 0 w with Gaussian noise (and not mixture of Gaussians). This means that we fit the model Xt = μ(T t ) + σ t t ∈ I 0 w , (D.1.7) which is the usual spline smoothing model presented in [START_REF] Green | Nonparametric regression and generalized linear models[END_REF]. We get

and set μ0 w = μ. For the values of β0 w,z and σ0 w,z , we use a priori assumptions by setting β0

w,0 = 0.8 = 1 -β0 w,0 , σ0 w,0 = √ 0.6 × σ and σ0 w,1 = σ .

These values are based on the wish to interpret regime Z = 0 as representing values close to the regression curve (i.e. "normal" behavior) and the regime Z = 1 represents values far from the regression curve (i.e "extreme" behavior).

graphic user interfaces for load curve analysis