Marie Billaud-Friess

Mi-Song Dupuy

Yvon Maday

Adrien Beguinet

Keywords: Tensor methods, Canonical Polyadic, Tesor Train, Local Tensor Methods, Model Order Reduction, Neural Networks, PINNs, Singulary Perturbed Problems

Flenghi, Olga Mula and Agustin Somacal, members of a research team that started in CEMRACS and have continued till today, conforming the last contribution of this thesis. Also for the help inside and outside the professional environment.

Also, to all the members of the COMMEDIA team, for giving me the opportunity and the support of doing this project. A special thank you to all the current and former colleagues of the team that have become close friends. You've been the best teammates that one could ask for.

Thanks to all my friends in Paris that have been more my family than my friends here and that were with me during all this three years and I'm sure for a lot more. A special thank you to my flatmates, that have lived my thesis submission from a very close and stressful perspective, as if it was theirs. Also to the ones that have been sending me love and support even in the distance. All of you have filled these three years of unforgettable moments. To my capoeira

In the second part of the work, we propose a local tensor approximation method based on clustering, that retrieves an approximated solution in the introduced local High Order Singular Value Decomposition (local HOSVD) format. The clustering method is performed direction per direction and it provides a dyadic partition tree that encloses the different possible partitions of the domain. This can be done due to the separability of the domain. In order to find the best partition to approximate the function, an extensive search between the possible partition combinations is done. Then, a HOSVD approximation of the tensor on each one of the partitions is computed. The criterion selected in order to choose one partition over the others is the memory needed to store the approximation. In the light of the numerical results obtained, we can say that the method achieves a good compression rate with respect to the HOSVD method.

In the third part of the work, a study on Deep learning-based numerical schemes, such as Physics-Informed Neural Networks (PINNs) as an alternative to classical numerical schemes for solving Partial Differential Equations (PDEs) is addressed. These methods are very appealing at first sight because of its simple implementation. Vanilla versions of PINNs, based on strong residual forms and neural networks, offer very high approximation capabilities. However, when the PDE solutions are low regular, optimization solvers are significantly challenged, and can potentially spoil the final quality of the approximated solution due to the convergence to bad local minima, and bad generalization capabilities. In this work, we present a numerical study of the merits and limitations of these schemes when solutions exhibit low-regularity, and compare performance with respect to more benign cases when solutions are very smooth. As a support for our study, we consider singularly perturbed convection-diffusion problems where the regularity of solutions typically degrades as certain multiscale parameters go to zero.

Résumé

Résumé: Dans cette thèse, nous cherchons à développer des méthodes numériques afin d'approcher des fonctions multivariées (tenseurs). Celles-ci peuvent être dérivées d'ensembles de données, de tableaux à entrée multiple , ou de solutions d'EDP qui provient de la physique, de la biologie, de l'économie ou d'autres disciplines. L'approximation de ces fonctions multivariées est abordée sous trois angles:

1. Une méthode d'approximation tensorielle, Sum of Tensor Trains (SoTT), dont la sortie est la solution approchée fournie sous la forme d'une somme de TTs.

2. Une méthode d'approximation tensorielle locale basée sur un methode de clustering, qui récupère une solution approchée dans le format local HOSVD.

3. Une méthode basée sur l'apprentissage profond qui calcule l'approximation de la solution d'une équation de convection-diffusion lorsque le paramètre de diffusion est très petit.

Dans la première partie de ce travail, la méthode proposée permet de calculer une approximation d'un tenseur donné comme une somme de Tensor Trains (TT), où pour lequel l'ordre des variables et les valeurs des rangs peuvent varier d'un terme à l'autre de manière adaptative. Le schéma numérique est basé sur un algorithme glouton et une adaptation de la méthode TT-SVD. Il permet d'obtenir de bonnes performances de bons résultats sans dépendre indépendemment de l'ordre des variables. L'approche proposée peut également être utilisée pour calculer une approximation d'un tenseur dans un format Canonical Polyadic (CP), comme alternative aux algorithmes standards. Quelques résultats numériques sont proposés, dans lesquelles la méthode développée est comparée aux méthodes Alternating Least Squared (ALS) et Alternating Singular Value Decomposition (ASVD) pour la construction d'une approximation CP, celle-ci se comporte particulièrement bien pour les tenseurs d'ordre élevé. L'intérêt d'approximer un tenseur comme une somme de Trains Tensoriels est illustré dans plusieurs cas tests numériques.

Dans la deuxième partie du travail, nous proposons une méthode d'approximation tensorielle locale basée sur le clustering, qui récupère une solution approchée dans le format introduit local High Order Singular Value Decomposition (local HOSVD). La méthode de clustering est réalisée direction par direction et fournit un arbre de partition dyadique qui englobe les partitions du domaine. Ceci peut être fait grâce à la séparabilité du domaine. Afin de trouver la meilleure partition pour approximer la fonction, une recherche extensive entre les combinaisons de partitions possibles est effectuée. Ensuite, une approche HOSVD du tenseur sur chacune des partitions est calculée. Le critère retenu pour choisir une partition plutôt que les autres qu'une autre est la mémoire nécessaire pour stocker la solution approchée. À la lumière des résultats numériques obtenus, nous pouvons dire que la méthode atteint un bon taux de compression par rapport à la méthode HOSVD.

Dans la troisième partie du travail, une étude sur les schémas numériques basés sur l'apprentissage profond, tels que les Physics-Informed Neural Networks (PINNs), comme alternative aux schémas numériques classiques pour résoudre les équations différentielles partielles (PDEs), est abordée. Ces méthodes sont très attrayantes à première vue en raison de leur mise en oeuvre très simple. Les versions anciennes des PINNs, basées sur des formes résiduelles fortes et des réseaux de neurones, offrent des capacités d'approximation très élevées. Cependant, lorsque les solutions des EDP sont peu régulières, les solveurs d'optimisation peuvent potentiellement restituer une mauvaise solution approchée en raison de la convergence vers de mauvais minima locaux et de mauvaises capacités de généralisation. Dans ce travail, nous présentons une étude numérique mettant en excergue les des mérites avantages mais aussi les limites de ces schémas lorsque les pour des solutions présentent une faible régularité peu régulières, et nous comparons également les performances pour des cas plus simples avec des solutions lisses. Comme support à notre étude, nous considérons des problèmes de convection-diffusion singulièrement perturbés où la régularité des solutions se dégrade typiquement lorsque certains paramètres multi-échelles tendent vers zéro.

Mots-clés :

Méthodes Tensorielles, Canonical Polyadic, Tensor Train, Méthodes Tensorielles Locales, Modèles d'Ordre Réduit, Réseaux Neuronaux, PINNs, Problèmes Perturbés Singuliers.

Chapter 1

Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi-modal datasets, which are often conveniently represented as multiway arrays or tensors [START_REF] Cichocki | Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[END_REF][START_REF] Kolda | Tensor decomposition: A mathematical tool for data analysis[END_REF]3]. Many problems with practical interest in physics, chemistry or mathematical applications naturally lead to high-dimensional or multivariate approximation problems. As a consequence, these can't be treated naively.

The main challenge in dealing with such amount of data is the so called curse of dimensionality, introduced by Bellman in [4], that refers to the need of using a number of degrees of freedom exponentially increasing with the dimension [START_REF] Oseledets | Breaking the curse of dimensionality, or how to use svd in many dimensions[END_REF]. In many applications, functions may depend on a potentially high number of variables d. When the dimension d increases, standard methods lead to a complexity of the numerical approximation which grows exponentially with d. As a consequence, the number of evaluations necessary to deal with such large datasets with naive tools may become prohivitive.

The numerical simulation of physical models takes today an important place in numerous branches of science and engineering. Due to the increasing complexity of models, more and more refined discretizations and robust numerical solution techniques are needed in order to obtain reliable predictions of their responses. Furthermore, in the context of optimization or model identification, the aim is not to predict the response of a unique model but of a family of models. In order to achieve these goals, traditional solution techniques require the optimal use of constantly evolving computational resources. This is closely linked with the curse of dimensionality introduced before. However, for many applications, innovative methodologies as an alternative to the brute force approach are obviously necessary to access numerical prediction. In other contexts, standard models may become multidimensional if some of the parameters that they involve are considered as new coordinates. This possibility is specially attractive when these coefficients are not well known, they have a stochastic nature, or when one is interested in optimization or inverse identification.

The concept of model reduction (MOR, [START_REF] Wilhelmus | Model order reduction: theory, research aspects and applications[END_REF][START_REF] Benner | Model reduction and approximation: theory and algorithms[END_REF]) seems to be a path for solving these computational issues. Model order reduction aims to lower the computational complexity of certain problems as simulations of largescale dynamical systems and control systems. Model reduction methods exploit the fact that the response of complex models can often be approximated with a reasonable precision by the response of a surrogate model.

For instance, in Reduced Basis (RB, [START_REF] Nc Nguyen | Reduced basis approximation and a posteriori error estimation for parametrized parabolic pdes: Application to real-time bayesian parameter estimation[END_REF][START_REF] Dv Rovas | Reduced-basis output bound methods for parabolic problems[END_REF][START_REF] Quarteroni | Reduced basis methods for partial differential equations: an introduction[END_REF][START_REF] Quarteroni | Certified reduced basis approximation for parametrized partial differential equations and applications[END_REF][START_REF] Haasdonk | Reduced basis method for finite volume approximations of parametrized linear evolution equations[END_REF][START_REF] Martin | A posteriori error bounds for reduced-basis approximations of parametrized parabolic partial differential equations[END_REF]) methods this is usually achieved by projection of full-order models. In many studies, the reduced basis is obtained by Proper Orthogonal Decomposition-based methods [START_REF] Berkooz | The proper orthogonal decomposition in the analysis of turbulent flows[END_REF][START_REF] Sirovich | Turbulence and the dynamics of coherent structures. i. coherent structures[END_REF][START_REF] Kenneth | The use of the karhunen-loeve procedure for the calculation of linear eigenfunctions[END_REF]. The dimension of reduced basis may be of several orders of magnitude lower than the dimension of the classically used numerical models. We refer to [START_REF] Benner | Model reduction and approximation: theory and algorithms[END_REF] for a complete overview of the Model Order Reduction picture.

Consequently, the goal is to provide methods for generating reduced models that are simultaneously accurate (concerning the approximation error) and performant (concerning the simulation time) independently of the complexity in parameters and the complexity in the time evolution of the original problem.

Currently, there are several classes of methods under investigation in order to propose parsimonious representations in high dimensions. They try to address an approximation in classes of functions that make it possible to circumvent the curse of dimensionality, [START_REF] Cohen | Approximation of high-dimensional parametric pdes[END_REF][START_REF] Benner | Model reduction and approximation: theory and algorithms[END_REF][START_REF] Cohen | Multiscale and highdimensional problems[END_REF][START_REF] Bui-Thanh | Model reduction for large-scale systems with high-dimensional parametric input space[END_REF]. Standard structured approximations include n-terms approximations [START_REF] Binev | Convergence rates for greedy algorithms in reduced basis methods[END_REF][START_REF] Ronald | Some remarks on greedy algorithms[END_REF][START_REF] Vladimir | The best m-term approximation and greedy algorithms[END_REF], sparse approximations [START_REF] David L Donoho | Stable recovery of sparse overcomplete representations in the presence of noise[END_REF], low-rank approximations [START_REF] Hackbusch | Tensor spaces and numerical tensor calculus[END_REF][START_REF] Nouy | Low-rank tensor methods for model order reduction[END_REF], or deep learning approximation methods [START_REF] Devore | Neural network approximation[END_REF].

This work is focused on low-rank representations of multivariate functions from three different points of view. From the so-called low-rank tensor approximations or low-rank tensor decompositions, [START_REF] Grasedyck | A literature survey of low-rank tensor approximation techniques[END_REF][START_REF] Nouy | Low-rank methods for high-dimensional approximation and model order reduction[END_REF], its local variant local low-rank tensor approximation models and from a deep learning-based approach.

Find in the present chapter the state of the art of Tensor methods in Section 1.1, an overview on clustering methods in Section 1.2 and the state of the art of Physics-Informed Neural Networks in Section 1.3. At the end of the chapter, in Section 1.4 the reader can find also the contributions of this thesis to the scientific framework.

State of the art of Tensor methods

In the literature, these high-order equivalents of vectors or matrices are called high-order tensors or multi-way arrays. For a lot of applications involving high-order tensors, the existing algebraic methods face the cited curse of dimensionality, what makes them inappropriate for some problems. This problem can be alleviated by using various tensor formats, achieved by low-rank tensor approximations, for the compression of the full tensor as described for instance in [START_REF] Kolda | Tensor decompositions and applications[END_REF][START_REF] Cichocki | Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[END_REF][START_REF] Domanov | On uniqueness and computation of the decomposition of a tensor into multilinear rank-(1, lr, lr) terms[END_REF][START_REF] Friedland | The number of singular vector tuples and uniqueness of best rank-one approximation of tensors[END_REF]. Tensors are well-studied objects in the standard mathematics literature [START_REF] William | Algebra: an approach via module theory[END_REF][START_REF] Steven | Abstract algebra[END_REF][START_REF] Lang | Algebra[END_REF] and more specifically in multilinear algebra, [START_REF] Hackbusch | Multi-grid methods and applications[END_REF][START_REF] Marcus | Finite dimensional multilinear algebra[END_REF].

Let p be the dimension of the system and let D be a subset of R p with a product structure D := D 1 × . . . × D d . Let us consider d ∈ N * . Then, for all 1 ≤ i ≤ d, D i is an open bounded subset of R pi for some p i ∈ N * . We call a tensor any real-valued multivariate function F ∈ L 2 (D 1 × • • • × D d ). The following definitions are set for continuous tensors. Let a discrete tensor be the discretizated form of a multivariate function.

For each 1 ≤ i ≤ d, let us have a certain H i Hilbert space of univariate functions defined of D i , equipped with the corresponding inner product (•, •) Hi and the associated norm ∥ • ∥ Hi . The elementary tensor product u (1) ⊗ . . . ⊗ u (d) of the d univariate functions u (i) ∈ H i has associated a multilinear mapping D 1 × . . . × D d → D 1 ⊗ . . . ⊗ D d and is defined such that for a multivariate function F and x = (x 1 , . . . , x d ) ∈ D, F (x) = u (1) (x 1 ), ⊗ . . . ⊗ u (d) (x d ).

The space L 2 (D) of square-integrable, real-valued functions defined on D equipped with a scalar product and the corresponding canonical norm ∥ • ∥ L 2 (D) that is defined for F, G ∈ L .

From now on, in order to simplify the notation of the manuscript (every time that there's no ambiguity) L 2 (D) will be simplified as D, equipped with the norm ∥ • ∥ D and its associated inner product (•, •) D .

Proper Orthogonal Decomposition (POD)

The Proper Orthogonal Decomposition (POD), [START_REF] Pearson | Liii. on lines and planes of closest fit to systems of points in space[END_REF], is a popular dimensionality reduction method used in model reduction (see [START_REF] Moore | Principal component analysis in linear systems: Controllability, observability, and model reduction[END_REF][START_REF] Willcox | Balanced model reduction via the proper orthogonal decomposition[END_REF][START_REF] Carlberg | Efficient non-linear model reduction via a least-squares petrov-galerkin projection and compressive tensor approximations[END_REF]) to define the trial subspace.

Definition. For any domain D = D x × D y , where D x and D y are open subdomains of R dx and R dy for some d x , d y ∈ N * respectively, and any W ∈ L 2 (D), it holds that there exists an orthonormal basis (U k ) k∈N * of L 2 (D x ), an orthonormal basis (V k ) k∈N * of L 2 (D y ) and a non-increasing sequence (σ k ) k∈N * of non-negative real numbers which converges to 0 as k goes to ∞, such that

W = k∈N * σ k U k ⊗ V k .
(1.1)

A decomposition of W under the form (1.1) is called a Proper Orthogonal Decomposition (POD) of W according to the separation of variables (D x , D y ). It is essentially the same as the Singular Value Decomposition (SVD) in a finite-dimensional space or in Euclidean space. The sequence (σ k ) k∈N * is known to be unique and is called the sequence of singular values of W associated to the separation of variables (D x , D y ) of the set D. The orthonormal basis (U k ) k∈N * (respectively (V k ) k∈N * ) may not be unique but is called a sequence of left (respectively right) singular vectors of W associated to this partitioning.

For any set E, we denote in the sequel by #E the cardinality of E. Assuming that N x := #D x < +∞ and N y := #D y < +∞, the complexity of the computation of an SVD decomposition of the form (1.1) scales like O max(N x , N y ) min(N x , N y ) 2 .

(1.

2)

The definition of these different tensor formats relies on the well-known separation of variables principle. We refer the reader to [START_REF] Hackbusch | Tensor spaces and numerical tensor calculus[END_REF] and [START_REF] Boris | Tensor numerical methods in scientific computing[END_REF] for extensive reviews on tensor theory and extended analysis of tensor decompositions and their numerous applications. Tensor formats are also used for solving time-dependent and stochastic/parametric PDEs ( [START_REF] Khoromskaia | Tensor-based techniques for fast discretization and solution of 3d elliptic equations with random coefficients[END_REF][START_REF] Babuška | A stochastic collocation method for elliptic partial differential equations with random input data[END_REF]).

Principal component analysis (PCA), [START_REF] Abdi | Principal component analysis[END_REF][START_REF] Hotelling | Analysis of a complex of statistical variables into principal components[END_REF], is a multivariate technique that analyzes a data set in which observations are described by several intercorrelated quantitative dependent variables. PCA is probably the most popular multivariate statistical technique and it is used by almost all scientific disciplines. Its goal is to extract the important information from these data, to compress the size of the data set by keeping only this important information, to represent them as a set of new orthogonal variables called principal components (obtained as linear combinations of the original variables), and to display the pattern of similarity of the observations and of the variables as points in certain maps. Mathematically, its components are obtained from the singular value decomposition (SVD) introduced in Equation 1.1 of the data. Thus, PCA depends on the singular value decomposition of positive semidefinite matrices, i.e. on the SVD of rectangular matrices. Its generalized form, Generalized principal component analysis (GPCA) is introduced in [START_REF] Vidal | Generalized principal component analysis (gpca)[END_REF].

In the context of approximating a multivariate function and in model order reduction, the POD is also known as PCA. It belongs to the family of linear dimensionality reduction techniques. POD-based reduced models consist in finding a low-dimensional afine subspace minimizing the projection error between the projected data and the original data. We call a solution {ϕ 1 , . . . , ϕ M } a POD basis of order M . Computing a POD basis of order M as given by Problem 1.1 is equivalent to solve a minimization problem of the form:

For X = {x i } P i=1 given, find orthonormal functions {ϕ j } M j=1 solving:

min ϕj P i=1 ∥x i - M j=1 (x i , ϕ j )ϕ j ∥ 2
Eckart-Young theorem, [START_REF] Nicholas | Computing the nearest correlation matrix-a problem from finance[END_REF], shows the optimality of the POD for two variables. Let us define then the maximal POD basis {ψ j } M j=1 and then:

P i=1 ∥x i - M j=1 (x i , ψ j )ψ j ∥ 2 ≤ min ϕj P i=1 ∥x i - M j=1 (x i , ϕ j )ϕ j ∥ 2 (1.3)
for any other {ϕ j } M j=1 and M ≤ P . Moreover the truncation error ε of using M instead of P POD basis in representing X is given by:

ε(M ) = P i=1 ∥x i - M j=1 (x i , ψ j )ψ j ∥ 2 = P k=M +1 σ 2 k
where σ k , k = M + 1, . . . , P are the P -M smaller singular values of the POD decomposition of 1.1. The corresponding M singular values kept indicate the variance that is captured, which enables to ordering the principal component and choosing their appropriate number to represent the dataset with a given level of accuracy ε.

The reduced order X derived by projecting onto the POD subspace, are represented as follows

x M = M j=1 α M j ψ j
where α M are the expansion coefficients. Solving the reduced order model 1.3 for the coefficients allow us to compute the reduced order solution x M . The different choice of basis functions will lead us to different solving strategies: Galerkin, Petrov Galerkin, reduced basis methods...

Over the past years, model reduction techniques have become a very important tool for the reduction of computational requirements in the numerical simulation of complex high-dimensional models. A technique able to alleviate the already mentioned curse of dimensionality consists of using a separated representation of the unknown multivariate function. This can be done due to the separability of the domain. A family of models that rely on the construction of separated variables representation of the solution in tensor product spaces is conformed. The methods of this family can be interpreted as generalizations of Proper Orthogonal Decomposition (POD), Singular Value Decomposition (SVD) or Principal Component analysis (PCA) in other contexts. The use of these methods for tensor decomposition in high dimensional problems can be seen in [START_REF] Nouy | A priori tensor approximations for the numerical solution of high dimensional problems: alternative definitions[END_REF][START_REF] Ladevèze | The latin multiscale computational method and the proper generalized decomposition[END_REF].

Tensor product spaces are introduced as the foundations of both a priori MOR methods and low-rank tensor approximations. Tensor Product Spaces are the appropriate framework to formalize the approximation in the parametric domain. Furthermore, the separated representations and tensor product spaces are closely related. In the following, some of the most popular low-rank tensor decompositions are introduced.

Canonical Polyadic (CP) decomposition

As it it detailed in [START_REF] Tamara | Tensor decompositions and applications[END_REF], the idea of the polyadic form of a tensor (expressing a tensor as the sum of a finite number of elementary tensors) arose initially in [START_REF] Frank | The expression of a tensor or a polyadic as a sum of products[END_REF][START_REF] Frank | Multiple invariants and generalized rank of a p-way matrix or tensor[END_REF]. This tensor format is also known as Candecomp Parafac. The form of Candecomp (canonical decomposition), was proposed in [START_REF] Carroll | Analysis of individual differences in multidimensional scaling via an n-way generalization of "eckart-young" decomposition[END_REF] and Parafac (parallel factors) in [START_REF] Richard A Harshman | Foundations of the parafac procedure: Models and conditions for an" explanatory" multimodal factor analysis[END_REF]. We refer to the Candecomp/Parafac decomposition as CP proposed independently in [START_REF] Henk | Towards a standardized notation and terminology in multiway analysis[END_REF] and [START_REF] Mocks | Topographic components model for event-related potentials and some biophysical considerations[END_REF].

Let us now introduce some notation which will be used in all the sequel. From now on, we fix some d ∈ N * and for all 1 ≤ j ≤ d, let Ω j be an open subset of R pj for some p j ∈ N * . We define Ω :

= Ω 1 × • • • × Ω d . Let F ∈ L 2 (Ω 1 × • • • × Ω d ).
The function F is said to belong to the Canonical Polyadic (CP) format [START_REF] Frank | The expression of a tensor or a polyadic as a sum of products[END_REF][START_REF] Kolda | Tensor decompositions and applications[END_REF][START_REF] Domanov | Canonical polyadic decomposition of third-order tensors: reduction to generalized eigenvalue decomposition[END_REF] with rank r ∈ N * if it reads as:

F (x 1 , x 2 , ..., x d ) = r i=1 u (1) i (x 1 )u (2) i (x 2 ) • • • u (d) i (x d ) (1.4)
for some functions u (j) i ∈ L 2 (Ω j ) for 1 ≤ i ≤ r and 1 ≤ j ≤ d. The CP decomposition factorizes a tensor into a sum of component rank-one (elementary) tensors.

The rank of a tensor is the minimum number of terms in an exact CP. A CP decomposition of a tensor F with r = rank(F ) number of components is called the rank decomposition. More details and its relation with the uniqueness can be found in [START_REF] Karlsson | Parallel algorithms for tensor completion in the cp format[END_REF][START_REF] Tamara | Tensor decompositions and applications[END_REF][START_REF] Lu | Tensor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization[END_REF] and [START_REF] Stegeman | On kruskal's uniqueness condition for the candecomp/parafac decomposition[END_REF].

The main advantage of this decomposition is the low memory cost needed to store it. Indeed, if N degrees of freedom are used per variable, the storage cost of a general function

F ∈ L 2 (Ω 1 × • • • × Ω d ) is O N d .
On the other hand, the storage cost of a CP tensor with rank r reduces to O(dN r), which scales linearly in the tensor order d and size N . However, it suffers from several drawbacks. The computation of the canonical rank is an NP-hard problem [START_REF] Håstad | Tensor rank is np-complete[END_REF] and the problem of finding a best approximation of a tensor in CP format may be ill-posed [START_REF] De | Tensor rank and the ill-posedness of the best low-rank approximation problem[END_REF], i.e. when the problem is written as a minimization problem, a minimizer may not exist; thus the numerical algorithms for computing an approximate representation in such cases might fail. The most classical algorithm in order to compute an approximation of a tensor in the CP format is the so-called Alternating Least Square (ALS) method.

See the details in Algorithm 1. Several researchers have proposed improving ALS with line searches [START_REF] Uschmajew | Local convergence of the alternating least squares algorithm for canonical tensor approximation[END_REF][START_REF] De | Decompositions of a higher-order tensor in block terms-part iii: Alternating least squares algorithms[END_REF], including the ELS [START_REF] Rajih | Enhanced line search: A novel method to accelerate parafac[END_REF], which adds a line search after each major iteration that updates all component matrices simultaneously based on the standard ALS search directions. Let us also cite the Modified Alternating Least Square (MALS), [START_REF] Holtz | The alternating linear scheme for tensor optimization in the tensor train format[END_REF], where the idea behind is to modify the least squares format used in the ordinary ALS algorithm. In addition, weighted least squares method can be found in the literature, [START_REF] Ruppert | Multivariate locally weighted least squares regression[END_REF][START_REF] Haberstich | Boosted optimal weighted least-squares[END_REF][START_REF] Asparouhov | Weighted least squares estimation with missing data[END_REF][START_REF] Cohen | Optimal weighted least-squares methods[END_REF].

Algorithm 1 Rank-one ALS greedy algorithm 1: Require: Prescribed tolerance ϵ > 0, and tensor W ∈ L 2 (Ω) 2: Output: N ∈ N * , for all 1 ≤ n ≤ N and all 1 ≤ i ≤ d, R n i ∈ L 2 (Ωi) so that the CP tensor W ∈ L 2 (Ω) defined by

W (x1, • • • , x d ) := N n=1 R n 1 (x1)R n 2 (x2) • • • R n d (x d ) ∀(x1, • • • , x d ) ∈ Ω, satisfies ∥W -W ∥ 2 L 2 (Ω) ≤ ϵ. 3: Set W 0 = W , n = 1. 4: while ∥W n-1 ∥ 2 L 2 (Ω) > ϵ do

5:

For all 1 ≤ i ≤ d, select randomly R n,0 i ∈ L 2 (Ωi) and set η := ϵ and m = 1.

6:

while η > 1 10 ϵ do 7:

for j = 1, • • • , d do 8: Compute R n,m j ∈ L 2 (Ωj ) solution to R n,m j ∈ argmin R j ∈L 2 (Ω j ) Wn-1 -R n,m 1 ⊗ • • • ⊗ R n,m j-1 ⊗ Rj ⊗ R n,m-1 j+1 ⊗ • • • ⊗ R n,m-1 d 2 L 2 (Ω)

9:

end for 10:

Compute η := R n,m 1 ⊗ • • • ⊗ R n,m d -R n,m-1 1 ⊗ • • • ⊗ R n,m-1 d 2 L 2 .
Set m := m + 1.

11:

end while

12:

Define R n i = R n,m-1 i for all 1 ≤ i ≤ d. [START_REF] Martin | A posteriori error bounds for reduced-basis approximations of parametrized parabolic partial differential equations[END_REF]:

Compute Wn(x1, • • • , x d ) = Wn-1(x1, • • • , x d ) -R n 1 (x1)R n 2 (x2) • • • R n d (x d ) for all (x1, • • • , x d ) ∈ Ω.
14:

n = n + 1
15: end while [START_REF] Kenneth | The use of the karhunen-loeve procedure for the calculation of linear eigenfunctions[END_REF]:

N = n -1
The ALS algorithm applied to CP is also used for tensor completion [START_REF] Hidasi | Fast als-based tensor factorization for context-aware recommendation from implicit feedback[END_REF], i.e. the case in which the problem of tensor factorization into CP format is applied to incomplete data. For this task, some works are proposed such as [START_REF] Karlsson | Parallel algorithms for tensor completion in the cp format[END_REF][START_REF] Acar | Scalable tensor factorizations for incomplete data[END_REF][START_REF] Tomasi | Parafac and missing values[END_REF].

CP approximations have proven useful in approximating certain multidimensional operators such as the Newton potential (see [START_REF] Hackbusch | Hierarchical kronecker tensorproduct approximations[END_REF]). CP is also used in data mining and in text analysis, [START_REF] Brett W Bader | Discussion tracking in enron email using parafac[END_REF].

Nevertheless, this separated representation is not a new proposal. It was first proposed in [START_REF] Ladevèze | The latin multiscale computational method and the proper generalized decomposition[END_REF] for performing efficient solutions of complex non-linear thermo-mechanical models and more detailed later in [START_REF] Ladevèze | Nonlinear computational structural mechanics: new approaches and non-incremental methods of calculation[END_REF] and [START_REF] Nouy | Multiscale computational strategy with time and space homogenization: a radial-type approximation technique for solving microproblems[END_REF]. Proper Generalized Decomposition is the common name for techniques using such separated representations. The general form of the separated representation of a generic multivariate function F (x 1 , . . . , x d ) in the PGD form reads as in Equation 1.4:

F (x 1 , . . . , x d ) = N i=1 f 1 i (x 1 ) . . . f d i (x d ) (1.5)
where x i denotes a scalar or vector coordinate defined in a domain Ω i of moderate dimension Ω i ∈ R d . The PGD decomposition is thus a sum of N functional products involving each a number of d functions that are unknown a priori. The case when the function F is known, has been extensively studied over the past years in multilinear algebra as the POD decomposition [START_REF] Chen | On the tensor svd and the optimal low rank orthogonal approximation of tensors[END_REF][START_REF] Kolda | A counterexample to the possibility of an extension of the eckart-young low-rank approximation theorem for the orthogonal rank tensor decomposition[END_REF][START_REF] De Lathauwer | A multilinear singular value decomposition[END_REF].

The PGD method aims to construct a decomposition of a tensor F . The solution is sought by applying a greedy algorithm (in which every iteration is solved through a fixed point algorithm, see [START_REF] Falco | A proper generalized decomposition for the solution of elliptic problems in abstract form by using a functional eckart-young approach[END_REF]), to the weak formulation of the problem. For each iteration of the algorithm, a mode consisting in a set of numerical values of the functional products f 1 i (x 1 ) . . . f d i (x d ) of the solution is computed. Each mode enriches the approximation of the solution and it gets more precise with the iterations.

There exist several approaches to the numerical analysis of PGD, [START_REF] Chinesta | The proper generalized decomposition for advanced numerical simulations: a primer[END_REF]. They combine the existence of a best approximation and a greedy algorithm. The idea of using greedy algorithms to construct successive approximations was considered in [START_REF] Ammar | A new family of solvers for some classes of multidimensional partial differential equations encountered in kinetic theory modeling of complex fluids[END_REF] in the context of the numerical solution of multidimensional PDEs. In [START_REF] Ammar | On the convergence of a greedy rank-one update algorithm for a class of linear systems[END_REF] the use of a Greedy Rank-One Update Algorithm to construct a rank-r approximate solution for a full rank linear system is proposed. Also, different greedy tensor decompositions have been used in order to compute low-rank approximations, see [START_REF] Uschmajew | Greedy rank updates combined with riemannian descent methods for low-rank optimization[END_REF][START_REF] Billaud-Friess | A tensor approximation method based on ideal minimal residual formulations for the solution of high-dimensional problems[END_REF][START_REF] Kolda | Orthogonal tensor decompositions[END_REF][START_REF] Cances | Convergence of a greedy algorithm for highdimensional convex nonlinear problems[END_REF][START_REF] Cances | Greedy algorithms for high-dimensional nonsymmetric linear problems[END_REF]. The principle of these methods is to approximate a function which depends on a large number of variates by a sum of tensor product functions, each term of which is iteratively computed via a greedy algorithm.

Between the main applications of the PGD method we can find: uncertainty quantification, where sometimes is called Generalized Spectral Decomposition, [START_REF] Nouy | A generalized spectral decomposition technique to solve a class of linear stochastic partial differential equations[END_REF], stochastic parametric analyses and numerical analysis of separated representation and its associated constructors. See more applications and details in [START_REF] Chinesta | A short review on model order reduction based on proper generalized decomposition[END_REF][START_REF] Chinesta | Recent advances and new challenges in the use of the proper generalized decomposition for solving multidimensional models[END_REF].

Let us now introduce the concept of unfoldings of a tensor. Assume a N -th order tensor X ∈ R N1×...×N N . The matrix unfolding X (n) ∈ R Nn×(Nn+1...N N N1...Nn-1) contains the element x i1...ip at the row position i n and column number equal to:

(i n+1 -1)N n+2 . . . N N N 1 . . . N n-1 + (i n+2 -1)N n+3 . . . N N N 1 . . . N n-1 + . . . . . . + (i N -1)N 1 . . . N n-1 + (i 1 -1)N 2 . . . N n-1 + . . . + i n-1
The slices of a tensor are introduced as the two-dimensional sections defined by fixing all the indices but two. In a third order tensor X ∈ R N1×N2×N3 , the horizontal, lateral, and frontal slices are denoted by X i1:: , X :i2: , and X ::i3 , respectively. They are matrices, and when we refer to the n-th frontal slice of a third order tensor we refer to its unfolding X (n) .

In general, we refer to the column and row vectors of a N -th order tensor as its n-mode vectors defined as the N n -vectors obtained by varying the index i n and keeping the rest fixed. The n-mode vectors (or just modes) of a certain tensor X are the column vectors of the unfolding X (n) .

We define the n-rank of a tensor X, denoted by R n = rank n (X) as the dimension of the vector space spanned by the n-mode vectors.

The n-mode multiplication of a tensor X ∈ R N1×...×Nn×...N d by a matrix A ∈ R In×Nn is defined by

[X × n A] i1...in-1Inin+1...i d = Nn in=1 X i1...in-1inin+1...i d a In×in (1.6)
leading to a tensor X × n A ∈ R N1×...×In×...N d . With the n-mode product, the POD decomposition introduced in Equation 1.1 is written as:

X = U ΣV T = Σ × 1 U × 2 V .

Tucker decomposition

The Tucker decomposition was first introduced by Tucker in 1963 in [START_REF] Ledyard | Implications of factor analysis of three-way matrices for measurement of change[END_REF] and then refined in [START_REF] Ledyard | The extension of factor analysis to three-dimensional matrices[END_REF][START_REF] Ledyard | Some mathematical notes on three-mode factor analysis[END_REF]. The Tucker decomposition is a form of higher-order PCA, see [START_REF] Pieter | Principal component analysis of three-mode data by means of alternating least squares algorithms[END_REF][START_REF] Kapteyn | An approach ton-mode components analysis[END_REF]. The Tucker model seeks a d-dimensional tensor F ∈ R N1×...×N d as mode products of a core tensor G ∈ R I1×...×I d and d mode matrices

U (n) ∈ R In×N1
It decomposes a tensor into a core tensor multiplied by a matrix along each mode. For a certain tensor F it reads:

F ≈ I1 i1=1
. . .

I d i d =1 G i1...i d u (1) i1 ⊗ . . . ⊗ u (d) i d (1.7)
where u (j) are the corresponding modes. G is a real (I 1 × . . . × I d )-tensor with the property of "all orthogonality" called the core tensor. When I n = N n , for all n = 1, . . . , d, the approximation becomes an equality. The Tucker format [START_REF] Ledyard | Some mathematical notes on three-mode factor analysis[END_REF][START_REF] De Lathauwer | On the best rank-1 and rank-(r 1, r 2,..., rn) approximation of higher-order tensors[END_REF] is stable but has exponential in d number of parameters, dIR + R d (for all equal ranks and degrees of freedom per dimension namely R and I respectively) parameters while the initial array has N d entries. It is suitable for "small" dimensions, especially for the three-dimensional case [START_REF] Boris | Multigrid accelerated tensor approximation of function related multidimensional arrays[END_REF][START_REF] Boris N Khoromskij | Numerical solution of the hartree-fock equation in multilevel tensor-structured format[END_REF][START_REF] Ivan V Oseledets | Tucker dimensionality reduction of three-dimensional arrays in linear time[END_REF]. Therefore, in some sense Tucker representations suffer from the curse of rank due to the need of storing the core tensor, whose size scales exponentially with the number of dimensions. For large d it is not suitable. The CP decomposition can be viewed as a special case of Tucker where the core tensor is superdiagonal and N 1 = . . . = N d . The Hierarchical Tucker format (H-Tucker), can be seen as a multilevel variant of the Tucker format, it was introduced in [START_REF] Grasedyck | Hierarchical singular value decomposition of tensors[END_REF] as a hierarchical SVD method for tensors of d ≥ 2.

Tucker representation is often computed by a generalization of the SVD, Higher-Order SVD (HOSVD) that produces quasi optimal approximations of the tensor, see [START_REF] De Lathauwer | A multilinear singular value decomposition[END_REF][START_REF] Badeau | Fast multilinear singular value decomposition for structured tensors[END_REF][START_REF] Grasedyck | Hierarchical singular value decomposition of tensors[END_REF]. For compression, it is also commonly used the HOSVD:

F = R1 i1=1 . . . R d i d =1 G i1...i d u (1) i1 ⊗ . . . ⊗ u (d) i d (1.8)
in which R 1 , . . . , R d are the n-ranks defined above and they are smaller that N 1 , . . . , N d , respectively. See more details in Algorithm 2. The core tensor G ∈ R R1...R d is computed by the n-mode multiplication defined in Equation 1.6 of the initial tensor and the mode matrices.

Algorithm 2 Truncated HOSVD algorithm

1: Require: The tensor W (x1, . . . , x d ) ∈ L 2 (Ω). The set of ranks {R k } 1≤k≤d . 2: Output: The set of truncated HOSVD matrices {U k } 1≤k≤d . The HOSVD core tensor G ∈ R R 1 ...R d . 3: for i = 1, • • • , d do 4:
Compute the SVD of W (i) :

W (i) = ŪiΣiV T i

5:

Set Ui as the Ri first column vectors of Ūi.

6: end for 7: Compute the core tensor:

G = W ×1 U T 1 . . . × d U T d
The Tucker tensors of fixed rank being an embedded manifold provide a stable parametrization, reflected in reliable practical computations. The Tucker-ALS algorithm convergence properties are very satisfactory [START_REF] Osman | Low-rank tucker decomposition of large tensors using tensorsketch[END_REF][START_REF] Kapteyn | An approach ton-mode components analysis[END_REF][START_REF] Pieter | Principal component analysis of three-mode data by means of alternating least squares algorithms[END_REF][START_REF] Ma | Fast and accurate randomized algorithms for low-rank tensor decompositions[END_REF], even if a formal analysis is not yet available.

The principle of these methods is to construct a hierarchy of optimal subspaces that results in a final tensor product of subspaces in which the function F is projected, to define its final approximation. Under strong assumptions on the estimation error made in the determination of subspaces, it is shown in [START_REF] De Lathauwer | A multilinear singular value decomposition[END_REF] that with a number of evaluations scaling in the storage complexity of the tree-based tensor format, see more in [START_REF] Falcó | Tree-based tensor formats[END_REF][START_REF] Ballani | Tree adaptive approximation in the hierarchical tensor format[END_REF], the approximation satisfies the desired error up to constants depending on some projection operators, which are not quantified.

For d-dimensional tensors with possibly large d > 3, a hierarchical data structure, called the Tree-Tucker format, is presented in [START_REF] Ivan | Breaking the curse of dimensionality, or how to use svd in many dimensions[END_REF] as an alternative to the canonical decomposition CP. It has smaller or equal number of representation parameters and viable stability properties. The approach involves a recursive construction described by a tree with the leafs corresponding to the Tucker decompositions of three-dimensional tensors, and is based on a sequence of SVDs for the recursively obtained unfolding matrices and on the auxiliary dimensions added to the initial "spatial" dimensions.

Several examples of using the Tucker decomposition in chemical analysis are provided in [START_REF] Henrion | N-way principal component analysis theory, algorithms and applications[END_REF] as part of a tutorial on N-way PCA, also applications in signal processing [START_REF] De | Dimensionality reduction in higher-order signal processing and rank-(r1,r2,. . . ,rn) reduction in multilinear algebra[END_REF], as well as several tasks in computer vision related with facial expression [START_REF] Alex | Multilinear analysis of image ensembles: Tensorfaces[END_REF][START_REF] Wang | Compact representation of multidimensional data using tensor rankone decomposition[END_REF].

Hierarchical Tucker (H-Tucker) format

The Hierarchical Tucker (H-Tucker) format from [START_REF] Grasedyck | Hierarchical singular value decomposition of tensors[END_REF][START_REF] Hackbusch | A new scheme for the tensor representation[END_REF] is a variant of the Tucker format. Hierarchical Tucker representations are an attempt to keep the generality of the Tucker representation but reducing at the same time its complexity, which was greatly affected by the size of the core tensor. In order to define it, we need to introduce a hierarchy between the modes. To do so, some preliminary concepts are needed.

Let us define the indices of a d-order tensor as D := {1, . . . , d}. And let t ⊂ D, t ̸ = ∅. Definition. A dimension tree or mode-cluster tree T for dimension d is a finite tree with root Root(T ) = D and depth p := {i ∈ N 0 |i ≤ log 2 (d)} such that each node t ∈ T is either:

• a leaf and singleton t = {µ}, with µ ∈ D.

• the union of two disjoint successors S(t) = {t 1 , t 2 } and t = t 1 ∪t 2 .

The level l of the tree is defined as a set of nodes having a distance l to the root. We denote the set of nodes in the level l as: T l := {t ∈ T |level(t) = l}. A node of the tree is called a so-called mode cluster or just cluster (a set of modes). A Canonical dimension tree is a particular case in which each node has two successors.

Definition.

Let T be a dimension tree. The hierarchical rank (H-rank) (r t ) t∈T , see [START_REF] Grasedyck | An introduction to hierarchical (h-) rank and tt-rank of tensors with examples[END_REF], of a tensor

F ∈ R N , being N := N 1 × . . . × N d , is defined by: ∀t ∈ T, r t := rank(F (t) )
the set of tensors of hierarchical rank (nodewise) at most (r t ) t∈T are the so calles H-Tucker tensors and they are denoted by:

H-Tucker(T, (r t ) t∈T ) := {F ∈ R N |∀t ∈ T : rank(F (t) ) ≤ r t }
From Definition of the H-rank of a tensor F based on a dimension tree T , one can obtain representation of the tensor. For this, we first notice that a representation matrix unfolding F (t) in the form

F (t) = U t V T t
is an exact representation of F for every node t of the tree T . Since for S(t) = t 1 , t 2 the column-vectors (U t ) i of U t ∈ R Nt×rt fulfil the nestedness property, there exists for every i ∈ {1, . . . , r t } a matrix

(B t ) i,•,• ∈ R rt 1 ×rt 2 such that (U t ) i = rt 1 j=1 rt 2 l=1 (B t ) i,j,l (U t1 ) j ⊗ (U t2 ) l
In the root and the leaves t = {µ}, the matrices are small enough to be stored in full tensor format but in the rest of the nodes we need to store sparse matrices. A tensor stored or represented in this form is said to be given in H-Tucker format. The computational complexity is in the order of O(dr 3 + dN r), where r = max t∈T r t and N = max k∈D #N k , which improves Tucker's complexity. However, operating with H-Tucker tensors is not always straightforward. For instance, the complexity of the inner product between tensors in H-Tucker format is in the order of O(dN r 2 + dr 4 ).

{1, 2, 3, 4} {1, 2} {1} {2} {3, 4} {3} {4} {1, 2, 3, 4} {1} {2, 3, 4} {2} {3, 4} {3}
The basic idea of the Hierarchical Tucker decomposition is that a tensor F can be represented in the H-Tucker format if it allows for a recursive construction out of lower-dimensional subspaces. This recursion is completely defined by the dimension tree introduced before. In this framework, a tensor is partitioned into a dimension tree with matrices at its leaves containing the modes of the tensor. These matrices are linked by the so-called transfer tensors, which are located at all inner nodes of the partition tree. Typical choices for such dimension trees are those obtained by a balanced splitting of the index set, as used, e.g., by Grasedyck [START_REF] Grasedyck | Hierarchical singular value decomposition of tensors[END_REF], or a front-to-back splitting which leads to the Tensor Trains (TT) format, [START_REF] Phan | Tensor networks for latent variable analysis: Novel algorithms for tensor train approximation[END_REF], proposed by Oseledets and Tyrtyshnikov in [START_REF] Oseledets | Tensor-train decomposition[END_REF][START_REF] Ivan | Breaking the curse of dimensionality, or how to use svd in many dimensions[END_REF].

In [START_REF] Haberstich | Adaptive approximation of high-dimensional functions with tree tensor networks for Uncertainty Quantification[END_REF], a projection method in a linear space is proposed. This method constructs the approximation of a function in tree-based tensor format, proposing a strategy to choose the tree structure in order to reduce ranks of the approximation at given precision and therefore its complexity and the required number of samples.

The use of rank-structured set of functions in tree-based tensor formats or tree tensor networks can be seen as a particular class of neural networks. It includes the Tucker format for a trivial tree, the tensor-train format for a linear binary tree [START_REF] Ivan | Breaking the curse of dimensionality, or how to use svd in many dimensions[END_REF] and the more general hierarchical format for a general dimension partition tree [START_REF] Hackbusch | A new scheme for the tensor representation[END_REF]. Any function in T l I admits a multilinear parametrization with parameters forming a tree network of low-order tensors, hence the name tree tensor networks, [START_REF] Bachmayr | Tensor networks and hierarchical tensors for the solution of high-dimensional partial differential equations[END_REF]. Considering dimension trees gives nice topological and geometrical properties [START_REF] Falcó | On minimal subspaces in tensor representations[END_REF][START_REF] Falcó | On the dirac-frenkel variational principle on tensor banach spaces[END_REF]. Tree tensor networks are particularly relevant for high-dimensional approximation, because the complexity of the parametrization of a function in tree-based tensor format is linear in the dimension d and polynomial in the ranks [START_REF] Falcó | Geometric structures in tensor representations[END_REF]. It can also be used for one-dimensional approximation [START_REF] Ali | Approximation with tensor networks. part i: Approximation spaces[END_REF][START_REF] Ali | Approximation with tensor networks. part ii: Approximation rates for smoothness classes[END_REF].

The tensor tree structure contains the Tucker format and the Hierarchical Tucker format (including the Tensor Train format). In the following scheme, Figure 1.1, we can see a 4-dimensional example of the front-to-back dimension trees representations for H-Tucker format on the left, and on the right special variant of the general H-Tucker format, the so-called Tensor Train (TT) format.

In the literature, some algorithms constructing approximations in tree-based tensor formats using points evaluations of functions have already been proposed. In [START_REF] Falcó | Geometric structures in tensor representations[END_REF], by using the tree structure and the dimensions of the associated minimal subspaces, it is proved that there exist a set of tensors that can be approximated by a tree-based format with bounded tree-based rank. There are learning approaches that use random evaluations of the functions [START_REF] Grelier | Learning with tree-based tensor formats[END_REF][START_REF] Haberstich | Active learning of tree tensor networks using optimal least-squares[END_REF]. In the numerical experiments, robustness and effectiveness of such algorithms are observed. On the other side, there are algorithms that use adaptive and structured evaluations of tensors for tree-based tensor formats, see [START_REF] Oseledets | Tt-cross approximation for multidimensional arrays[END_REF].

Tensor Train format

Let us now continue presenting the tensor format introduced before as a concrete case of the Tucker decomposition, that is going to be one of the key methods in the contributions of this manuscript. It was introduced in [START_REF] Oseledets | Tensor-train decomposition[END_REF]. The same format was introduced in the computational chemistry community under the name Matrix Product States (MPS), [START_REF] Vidal | Efficient classical simulation of slightly entangled quantum computations[END_REF]. The function F is said to belong to the Tensor Train (TT) format with ranks r 1 , . . . , r d-1 ∈ N * if and only if

F (x 1 , x 2 , ..., x d ) = r1 i1=1 ... r d-1 i d-1 =1 u (1) i1 (x 1 )u (2) i1,i2 (x 2 )u (3) i2,i3 (x 3 ) • • • u (d-1) i d-2 ,i d-1 (x d-1 )u (d) i d-1 (x d ) (1.9) with u (j) ij-1,ij ∈ L 2 (Ω j ) for 1 ≤ i j-1 ≤ r j-1 and 1 ≤ i j ≤ r j for all 1 ≤ j ≤ d (with r 0 = r d = 1
). It combines two main advantages: On the one hand, it is stable from an algorithmic point of view; on the other, it is computationally affordable provided that the TT ranks of the tensors used stay reasonably small. The ranks characterize the complexity to store a tensor in the TT format. Supposing that r 1 = . . . = r d-1 = r, its storage complexity is O(dr 2 N ). In [START_REF] Grasedyck | An introduction to hierarchical (h-) rank and tt-rank of tensors with examples[END_REF] bound for these TT ranks are provided.

One of the schemes used for the optimization of a TT tensor with a given rank is the ALS method, see details in [START_REF] Grasedyck | Stable als approximation in the tt-format for rank-adaptive tensor completion[END_REF][START_REF] Holtz | The alternating linear scheme for tensor optimization in the tensor train format[END_REF][START_REF] Rohwedder | On local convergence of alternating schemes for optimization of convex problems in the tensor train format[END_REF]. In [START_REF] Holtz | The alternating linear scheme for tensor optimization in the tensor train format[END_REF] some numerical examples that concern the stability of the TT decomposition and of ALS are shown as well as how high TT ranks are required during the iterative approximation of low-rank tensors, showing some potential of improvement. These ranks are chosen a priori and that makes the method less robust. As we will recall in Algorithm 3, other ways of computing the TT decomposition of a tensor is via the well-known TT-SVD algorithm.

We will also consider three different tensor representations which are based on slight modifications of the TT format, namely the quantized tensor-train format (QTT), (see more in [START_REF] Boris | O (dlog n)-quantics approximation of nd tensors in high-dimensional numerical modeling[END_REF]), the block tensor-train format (BTT) (see [START_REF] Sergey V Dolgov | Computation of extreme eigenvalues in higher dimensions using block tensor train format[END_REF]), and the cyclic tensor-train format (CTT) (see [START_REF] Hackbusch | Tensor spaces and numerical tensor calculus[END_REF]).

Let us make a reference also to the approximation of high-dimensional functions in a statistical learning setting: using model classes of functions in tree-based tensor format. These are particular classes of rank-structured functions that admit explicit and numerically stable representations, parametrized by a tree-structured network of low-order tensors. These nonlinear model classes can be seen as deep neural networks, see [START_REF] Grelier | Learning with tree-based tensor formats[END_REF].

Approximations in local subdomains

Nowadays, high-dimensional problems arise in a wide range of fields with practical interest such as quantum chemistry, molecular dynamics, uncertainty quantification, finance... or mathematical applications that naturally lead to high-dimensional or multivariate approximation problems. One of the main challenges that these high-dimensional and complex systems present is the already introduced curse of dimensionality [4,[START_REF] Richard | Applied dynamic programming[END_REF][START_REF] Oseledets | Breaking the curse of dimensionality, or how to use svd in many dimensions[END_REF], that refers to the need of using a number of degrees of freedom exponentially increasing with the dimension to describe them.

Dimensionality reduction methods naturally arise from this problem. They consist in finding a compressed representation of a multivariate function or dataset while limiting as much as possible the loss of information. In many applications, high-dimensional data have a low intrinsic dimension or present a certain correlation in the hidden structure of the data such as symmetry, periodicity, etc. In the case in which the method is able to detect these aspects, the curse of dimensionality is alleviated. A more efficient approach will involve approximating the function locally using a series of subspaces of smaller dimension or with less degrees of freedom on it. This idea of computing local approximations is extensively developed in the field of Model Order Reduction, see Chapter 3 of [START_REF] Benner | Model reduction and approximation: theory and algorithms[END_REF] for more details.

Algorithm 3 TT-SVD algorithm

1: Require: Prescribed tolerance ϵ > 0, the tensor W ∈ L 2 (Ω) 2: Output: K1, • • • , K d-1 ∈ N * TT-ranks, R1 ∈ L 2 Ω1, R 1×K 1 , R d ∈ L 2 Ω d , R K d-1 ×1 and for all i = 2, • • • , d -1, Ri ∈ L 2 Ωi, R K i-1 ×K i so that the Tensor Train W ∈ L 2 (Ω) defined by W (x1, • • • , x d ) := R1(x1)R2(x2) • • • R d (x d ) ∀(x1, • • • , x d ) ∈ Ω, satisfies ∥W -W ∥ 2 L 2 (Ω) ≤ ϵ 2 .
3:

Define K0 := 1, D0 = {1} and define W 0 ∈ L 2 (D0 × Ω) such that W 0(1, •) = W , I0 := {1, • • • , d}, Ω0 := Ω. 4: for j = 1, • • • , d -1 do 5: Since Dj-1 × Ωj-1 = (Dj-1 × Ωj ) × Ωj with Ωj = Ωj+1 × • • • × Ω d , compute the SVD decomposition of W j-1 according to the separation of variables (Dj-1 × Ωj , Ωj ) so that W j-1 = k∈N * σ j,k U j,k ⊗ V j,k .
6:

Select Kj ∈ N * such that Kj = inf K ∈ N * , k≥K |σ j,k | 2 ≤ ϵ 2 d -1 .

7:

Define Dj := {1, • • • , Kj } and define W j ∈ L 2 Dj × Ωj by

W j (kj , yj ) = σ j,k j V j,k j (yj )
for all (kj , yj ) ∈ Dj × Ωj .

8:

Define Rj ∈ L 2 Ωj , R K j-1 ×K j as Rj (xj ) = U j,k j (kj-1, xj ) 1 ≤ kj-1 ≤ Kj-1 1 ≤ kj ≤ Kj for all xj ∈ Ωj . 9: end for 10: Define R d ∈ L 2 Ω d , R K d-1 ×1 by R d (x d ) = σ d-1,k d-1 V d-1,k d-1 (x d ) 1≤k d-1 ≤K d-1 .
The Kolmogorov N -widths describe the rate of the worst-case error arising from a projection onto the best-possible linear subspace of dimension N ∈ N, [START_REF] Pinkus | N-widths in Approximation Theory[END_REF]. It has been proven that for certain linear, coercive, parameterized problems, the Kolmogorov N -widths decay exponentially or polynomially with high exponent as N grows, [START_REF] Buffa | A priori convergence of the greedy algorithm for the parametrized reduced basis method[END_REF]. This extremely fast decay is the base of any model reduction strategy since it guarantees that even choosing a very moderate dimension N , it is possible to achieve good approximations.

In the last years, the problem of finding a dimensionality reduction model has also been considered as a classification problem and, treated like one, studied with supervised learning algorithms, [START_REF] Al | Model order reduction based on agglomerative hierarchical clustering[END_REF][START_REF] Daniel | Model order reduction assisted by deep neural networks (rom-net)[END_REF].

Projection-based model order reduction consists in computing an approximate solution in a lowdimensional subspace of the solution space, which can give accurate predictions provided that the solution manifold is embedded in a low-dimensional space, see [START_REF] Zahm | Projection-based model order reduction methods for the estimation of vector-valued variables of interest[END_REF]. Projection-based model reduction techniques reduce the computational cost associated by restricting the solution space to a subspace of much smaller dimension. Among them, we can find POD Galerkin method [START_REF] Rowley | Model reduction for compressible flows using pod and galerkin projection[END_REF] and Reduced Basis method [START_REF] Prud | Reliable real-time solution of parametrized partial differential equations: Reduced-basis output bound methods[END_REF][START_REF] Rozza | Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations[END_REF][START_REF] Perrin | A posteriori error and optimal reduced basis for stochastic processes defined by a finite set of realizations[END_REF]. They enclose the family of models in which the approximated solution is obtained by solving the physics equations with the Galerkin method on a well chosen ROB, [START_REF] Falcó | Numerical strategies for the galerkinproper generalized decomposition method[END_REF].

Find in [START_REF] Benner | Model reduction and approximation: theory and algorithms[END_REF] some cases in which the Kolmogorov N -widths decay very slowly. In these cases, it may be very difficult to build a linear model that provides a certified approximation for a small tolerance and an affordable numerical complexity. For these classes of problems, an efficient strategy for model reduction requires to look for nonlinear models that capture the geometry of the manifold where the data lies better than linear spaces. In the situations in which we face moderate order functions is sometimes advantageous to apply a nonlinear dimensionality reduction method [START_REF] Vladimir | Nonlinear methods of approximation[END_REF][START_REF] John | Nonlinear dimensionality reduction[END_REF][START_REF] Devore | Nonlinear approximation[END_REF][START_REF] Ehrlacher | Nonlinear model reduction on metric spaces. application to one-dimensional conservative pdes in wasserstein spaces[END_REF][START_REF] Bonito | Nonlinear methods for model reduction[END_REF]. Theoretically, this is equivalent to map the data into a higher-dimensional space and then applying POD. Some methods like Kernel PCA (a nonlinear extension of PCA, see [START_REF] Schölkopf | Kernel principal component analysis[END_REF][START_REF] Hoffmann | Kernel pca for novelty detection[END_REF][START_REF] Mika | Kernel pca and de-noising in feature spaces[END_REF]) or the use of neural networks, [START_REF] Hinton | Reducing the dimensionality of data with neural networks[END_REF][START_REF] Scholz | Nonlinear pca: a new hierarchical approach[END_REF] are proposed in the literature for nonlinear problems.

Local reduced basis methods

Most MOR techniques build a single subspace that approximates the solution manifold. However, when the solution manifold is highly nonlinear and as a result a subspace of large dimension is required to approximate it globally, a more efficient approach involves approximating the manifold locally using a series of subspaces of smaller dimension or with less degrees of freedom on it. Each of the subspaces is associated with a local reduced-order basis that can be eventually stored. This methods works in such a way that the parametric domain is partitioned into subregions. Then, for a new parameter value, a local basis can be computed by simply choosing the region where the parameter belongs to. See [START_REF] Amsallem | Fast local reduced basis updates for the efficient reduction of nonlinear systems with hyper-reduction[END_REF] for a detailed analysis of nonlinear model reduction using local basis.

In [START_REF] Amsallem | Nonlinear model order reduction based on local reduced-order bases[END_REF][START_REF] Washabaugh | Nonlinear model reduction for cfd problems using local reduced-order bases[END_REF][START_REF] Amsallem | Fast local reduced basis updates for the efficient reduction of nonlinear systems with hyper-reduction[END_REF], the local subspace is selected by finding the closest cluster to the solution. The distance used is the Euclidean distance.

The MOR method introduced in [START_REF] Amsallem | Nonlinear model order reduction based on local reduced-order bases[END_REF] (and applied in [START_REF] Washabaugh | Nonlinear model reduction for cfd problems using local reduced-order bases[END_REF]) alleviates the problem of the low-efficiency of reduced models in high dimensions through the use of multiple local bases. In this approach, a local ROB is selected at each time step of the ROM simulation based on the current state of the system. Such a concept is particularly well-suited for the POD method in which the basis is built from snapshots of the system taken at various locations of the state-space.

In practice, the set of reduced bases {V j } N V j=1 , where N V is the number of subspaces, is built using a POD algorithm based on the method of snapshots. The proposed method for constructing a database of local bases consists of three steps:

• First, the states are clustered into N V subsets using an unsupervised learning algorithm, such as the kmeans. More important than the choice of algorithm is the choice of distance metric used, the Euclidean distance in this case.

• Second, the clusters are made to overlap with one another by sharing a small number of states between neighboring clusters.

• Third, the snapshots are formed from the state clusters and the individual local reduced order basis are computed.

Transitions between local subspaces require special care and updating the reduced bases associated with each subspace increases the accuracy of the reduced-order model.

Let us recall the already introduced Generalized principal component analysis (GPCA), [START_REF] Vidal | Generalized principal component analysis (gpca)[END_REF], where the alternative extension of PCA to the case of data lying in a union of subspaces is exploited. Subspace segmentation is a fundamental problem in many applications. If the segmentation of the data was known, one could simply assign a subspace to each set of points using PCA (although its effectiveness is limited by its global linearity). On the other hand, if the subspaces and its basis were known, the problem reduces to find the points that best fit the subspace. Normally, neither the subspaces or the segmentation of the data are known, most existing methods randomly choose a base for each subspace and then iterate between data segmentation and subspace estimation. In the literature, it has been done as K-subspaces, in [START_REF] Ho | Clustering appearances of objects under varying illumination conditions[END_REF], the extension of K-means to the case of subspaces; subspace growing or subspace selection, [START_REF] Leonardis | Multiple eigenspaces[END_REF]; or Expectation Maximization ( [START_REF] Todd K Moon | The expectation-maximization algorithm[END_REF]) fore mixtures of PCAs, see more in [START_REF] Michael | Mixtures of probabilistic principal component analysers[END_REF].

Splitting a non-reducible problem into multiple reducible ones can be achieved with cluster analysis. Cluster analysis belongs to unsupervised learning tasks and can be defined as the search of groups (or clusters) of similar objects in a database. The choice of the clustering algorithm depends on the underlying motivation and thus on the clusters topological properties that are expected.

State of the art of clustering methods

Clustering is a common technique for statistical data analysis, which is used in many fields, including machine learning, data mining, pattern recognition, image analysis between others. This reflects its broad appeal and usefulness as one of the steps in exploratory data analysis. Clustering is the process of grouping similar objects into different groups, or more precisely, the partitioning of a data set into subsets, so that the data in each subset according to some defined distance measure. Clustering problem is not a trivial task, especially in the case of high-dimensional data that is exactly the case that we face in most applications, and it is where conventional methods usually fail. The number of data required to define correctly the system increases exponentially with its dimension. This phenomenon is referred to as the already introduced curse of dimensionality. The difference with classification tasks is that the set of categories (or clusters) is not known a priori. Hence, clustering methods are included in the family of unsupervised learning models, where the data to be classified are not labeled. Dimensionality reduction methods are also included.

Therefore, clustering can simply be defined as the task of grouping entities in terms of a similarity measure. Here, the critical issue is to understand what is meant by "similar". Similarity is in a sense the inverse of a distance metric between two entities. The shorter the distance, the more similar the entities, and vice versa. It is important hence to note that, clustering results will be crucially dependent on the similarity notion chosen. Determining the distance between both clustering solutions is non-trivial and still subject to discussion. A conventional distance metric is the Euclidean distance. Deriving the Euclidean distance between two data points involves computing the square root of the sum of the squares of the differences between corresponding values. Many other similarity measures, e.g., [START_REF] De Maesschalck | The mahalanobis distance[END_REF][START_REF] Borgefors | Distance transformations in digital images[END_REF], could be used to tackle the broad range of domain specific clustering problems.

Clustering methods are usually categorized under four main groups: hierarchical, partitioning, overlapping, and ordination algorithms.

• The first group is based on the cluster formation methodology. Hierarchical algorithms can be agglomerative (bottom-up) or divisive (top-down), and this choice is done via analytic optimization techniques (see more in [START_REF] Allan D Gordon | A review of hierarchical classification[END_REF]). Agglomerative algorithms begin with each element as a separate cluster and merge them in successively larger clusters while divisive algorithms starts from large clusters and splits it succesively in smaller ones.

• A second group lists methods depending on the cluster model acquired such as hierarchical [START_REF] Sibson | Slink: an optimally efficient algorithm for the single-link cluster method[END_REF], representative (such that each cluster is associated to a partitioning representative, i.e. a reference point that well represents the cluster's members, they include k-means [START_REF] Lloyd | Least squares quantization in pcm[END_REF][START_REF] Macqueen | Classification and analysis of multivariate observations[END_REF] when the optimal representatives are the clusters means or centroids; k-medians [START_REF] Charu | Data mining: the textbook[END_REF] when the optimal representatives are the medians, k-medoids [START_REF] Lkpj Rdusseeun | Clustering by means of medoids[END_REF] when the optimal representatives are the medioids, in this case the representatives must be taken among the elements of the dataset; and many variants of k-means), distribution such as expectation maximization, density [START_REF] Kriegel | Density-based clustering[END_REF], subspace, group, and graph-based models, [START_REF] Novák | Graph-based clustering and characterization of repetitive sequences in next-generation sequencing data[END_REF].

• Thirdly, depending on the relationship type between entities and clusters, hard or soft clustering can be distinguished by defining binary or fuzzy relations, respectively.

• A final clustering group, based on the nature of cluster-cluster relations, defines the distinction between overlapping versus disjoint partition groups in general.

Let us make a remark on hierarchical clustering. Hierarchical techniques produce a nested sequence of partitions, with a single, inclusive cluster at the top and singleton clusters of individual points at the bottom. Each intermediate level can be viewed as combining two clusters from the next lower level (or splitting a cluster from the next higher level). The result of a hierarchical clustering algorithm can be graphically displayed as tree. This tree graphically displays the merging process and the intermediate clusters. For document clustering, this tree provides a hierarchical index. The two basic approaches to generate a hierarchical clustering: a) Agglomerative: Start with the points as individual clusters and, at each step, merge the most similar or closest pair of clusters. This requires a definition of cluster similarity or distance. b) Divisive: Start with one, all-inclusive cluster and, at each step, split a cluster until only singleton clusters of individual points remain. In this case, we will need to decide, at each step, which cluster to split and how to perform the division.

Hierarchical method suffers from the fact that once the merge/split is done, it can never be undone. This rigidity is useful in that it leads to smaller computation costs by not worrying about a combinatorial number of different choices. Divisive algorithms begin with the whole set and proceed to divide it into successively smaller clusters.

See [START_REF] Saxena | A review of clustering techniques and developments[END_REF][START_REF] Anil K Jain | Data clustering: a review[END_REF], for more information of families of clustering methods. The clustering process is considerably more complicated when dealing with clusterings of overlapping clusters. In this work we are going to restrict ourselves to the situation in which they are not overlapping. More information and an analysis of this situation can be found in [START_REF] Mark K Goldberg | Measuring similarity between sets of overlapping clusters[END_REF]. The validation of such algorithms refers to the problem of determining the ability of the methods to recover cluster configurations which are known to exist in the data. Validation approaches include mathematical derivations, analyses of empirical datasets, and Monte Carlo simulation methods. Some unsupervised learning tasks such as clustering and dimensionality reduction can be seen as data mining tasks. Dimensionality reduction is used for the construction of reduced-order models, the compression of simulation data, and the reduction of the number of variables to be processed by some predictive models; cluster analysis is used for the identification of groups of data with similar physical or mechanical behaviors, enabling the construction of dictionaries of cluster-specific models.

See [START_REF] Daniel | Machine learning for nonlinear model order reduction[END_REF] for an extensive study in model order reduction methods and its machine learning-based variants (ROM-net) and applications. In this work, local ROMs are combined with a classifier for automatic model recommendation. The classifier recommends a suitable reduced-order model. Nevertheless, as the classes are given by a clustering algorithm the computational cost is very high. The time-consuming operations required for model selection are avoided by replacing by an approximation the theoretical perfect classifier.

Physics-informed clustering

Physics-informed clustering strategies were firstly introduced in [START_REF] Daniel | Physics-informed cluster analysis and a priori efficiency criterion for the construction of local reduced-order bases[END_REF][START_REF] Daniel | Optimal piecewise linear data compression for solutions of parametrized partial differential equations[END_REF]. The focus is on finding a clustering strategy that is appropriate for model order reduction purposes. Physics-informed cluster analysis consists in clustering the parameter space by means of a dissimilarity measure which involves physical quantities obtained when solving the problem. In practice, this means that a clustering algorithm is applied in the solution space.

When using a clustering algorithm to compute partitions of the solution manifold, the quality of the partition is related with the choice of the clustering method and the measure used to group similar solutions on the manifold. Among physics-informed clustering strategies, it is common to use K-means algorithm (introduced in what follows), that measures similarities with Euclidean distances in the solution space or in a subspace of the solution space found by PCA. In [START_REF] Amsallem | Pebl-rom: Projection-error based local reduced-order models[END_REF], it was noticed that clustering based on the Euclidean distance in the solution space was not adapted for the construction of local ROMs, which led to the definition of projection-error based local ROMs where the solution space is hierarchically partitioned using the projection error as a dissimilarity criterion. In [START_REF] Daniel | Physics-informed cluster analysis and a priori efficiency criterion for the construction of local reduced-order bases[END_REF] an alternative based on the use of relative errors is proposed. In other works like [START_REF] Daniel | Model order reduction assisted by deep neural networks (rom-net)[END_REF] and [START_REF] Peherstorfer | Localized discrete empirical interpolation method[END_REF], the Grassmann distance between subspaces is used.

Other classification methods

Many algorithms exist for nonlinear classification problems, each of them having its own advantages and drawbacks. Naive Bayes classifiers [START_REF] Zhang | The optimality of naive bayes[END_REF] are well-known for their interpretability as well as Decision trees [START_REF] Vens | Decision trees for hierarchical multi-label classification[END_REF]. Decision trees are supervised learning algorithms that model the correspondence between inputs and outputs by means of simple decision rules, leading to a recursive partition of the input space. They are known to be interpretable in the sense that classification rules can be visualized as paths in a tree structure.

Other nonlinear classifiers include the k-nearest neighbors algorithm (kNN [START_REF] Cover | Nearest neighbor pattern classification[END_REF]) and quadratic discriminant analysis. See more detailed information in [START_REF] Hastie | The elements of statistical learning: data mining, inference, and prediction[END_REF][START_REF] Sotiris B Kotsiantis | Supervised machine learning: A review of classification techniques[END_REF]. In [START_REF] Lombardi | A greedy dimension reduction method for classification problems[END_REF] a greedy method to reduce dimensions in classification problems is investigated. The k-nearest neighbors method was first introduced in [START_REF] Fix | Discriminatory analysis. nonparametric discrimination: Consistency properties[END_REF], and more developed in [START_REF] Hart | The condensed nearest neighbor rule (corresp.)[END_REF] and [START_REF] Peterson | K-nearest neighbor[END_REF]. The kNN classifier [START_REF] Cover | Nearest neighbor pattern classification[END_REF] K-means algorithm is one of the best-known and simplest clustering algorithms, it is based on kNN method and it is mostly applied to solve classification problems, grouping the given data in a certain number K of clusters defined a priori.

State of the art of Deep Learning-based Schemes

Neural networks produce structured parametric families of functions that have been studied and used for almost 70 years. In the last several years, however, their popularity has surged as they have achieved state of the art in large variety of problems. We have seen an increase in the application of Deep Neural Networks (DNNs) in a wide range of disciplines. In scientific computing, computational experiments with DNNs for the numerical solution of Partial Differential Equations (PDEs) have been reported to be strikingly successful in a wide range of applications [START_REF] Beck | Solving the kolmogorov pde by means of deep learning[END_REF][START_REF] Yu | The deep ritz method: a deep learning-based numerical algorithm for solving variational problems[END_REF][START_REF] Han | Solving high-dimensional partial differential equations using deep learning[END_REF][START_REF] Sirignano | Dgm: A deep learning algorithm for solving partial differential equations[END_REF]. Moreover, deep learning-based reduced order models (DL-ROMs) have been recently proposed to overcome common limitations shared by conventional reduced order models, [START_REF] Fresca | Pod-dl-rom: enhancing deep learning-based reduced order models for nonlinear parametrized pdes by proper orthogonal decomposition[END_REF]. All these encouraging results show that DNNs are an interesting candidate for efficiently emulating other (linear or nonlinear) approximation methods.

Underlying this success, deep learning based schemes are a way to approximate functions. They embrace schemes from an additive construction commonly used in approximation theory to a compositional construction used in deep neural networks. The compositional construction seems to be particularly powerful in high dimensions. This suggests that deep neural network based models can be an interesting choice in function approximation tasks. This includes solving partial differential equations [START_REF] Kutyniok | A theoretical analysis of deep neural networks and parametric pdes[END_REF], molecular modeling [START_REF] Segler | Generating focused molecule libraries for drug discovery with recurrent neural networks[END_REF] and as mentioned before, model reduction [START_REF] Daniel | Model order reduction assisted by deep neural networks (rom-net)[END_REF], among others.

The idea of working with neural network functions to solve PDEs is by far not novel, and countless contributions have been proposed on this front in recent years, [START_REF] Kutyniok | A theoretical analysis of deep neural networks and parametric pdes[END_REF][START_REF] Han | Solving high-dimensional partial differential equations using deep learning[END_REF][START_REF] Beck | Solving the kolmogorov pde by means of deep learning[END_REF].

In high dimensional equations, the number of degrees of freedom grows exponentially fast as the dimension of PDE increases, hence we face the curse of dimensionality. One striking advantage of DNNs over classical numerical methods is that the number of degrees of freedom "only" grows (at most) polynomially. Therefore, DNNs are particularly suitable for solving high-dimensional PDEs, see [START_REF] Elbrächter | Dnn expression rate analysis of high-dimensional pdes: Application to option pricing[END_REF][START_REF] Weinan | A proposal on machine learning via dynamical systems[END_REF][START_REF] Han | Solving high-dimensional partial differential equations using deep learning[END_REF][START_REF] Han | Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations[END_REF].

Neural networks produce certified approximations of continuous functions, [START_REF] Cybenko | Approximation by superpositions of a sigmoidal function[END_REF][START_REF] Hornik | Multilayer feedforward networks are universal approximators[END_REF], in the sense that given a function f and a prescribed tolerance ε > 0, a neural network (with the right parameters) produces an approximation of f within an error smaller or equal than ε, [START_REF] Bolcskei | Optimal approximation with sparsely connected deep neural networks[END_REF].

The motivation to study the approximation of parametric PDEs by DNNs stems from the similarities between parametric problems and statistical learning problems. Assume given domains X ⊂ R d X , where

d X ∈ N and Y ⊂ R d Y , d Y ∈ N.
Let X to be the vector space of all possible inputs and Y the vector space for the possible outputs. Further assume that there exists an unknown probability distribution ρ on X × Y. Let us call loss function to J : Y × Y → R + . It measures (or penalizes, that's why it is called loss) the disparity between the estimated values and the true values for an instance of the data.

The goal of a statistical learning problem is to find a function f : X → Y such that f (x) ∼ y and that minimizes the loss function J (f (x), y) min

f ∈Y J (f (x), y) (1.10)
Since the probability measure is unknown, we have no direct access to the expected loss. Instead, we assume that we are given a set of training data, i.e., N pairs of points (x i , y i ) N i=1 , N ∈ N, called collocation points and distributed independently with respect to ρ. Then one finds f by minimizing the so-called empirical loss function:

min f ∈Y 1 N N i=1 J (f (x i ), y i ) (1.11)
We will call optimizing the empirical loss to the learning procedure. The approximation model provides a solution at a given collocation point. It will be detailed later on.

Deep learning scheme

Let us start by recalling basic concepts and notions related with neural networks. We begin by introducing feedforward neural networks and their elementary properties. The feedforward neural networks are those whose connections between neurons go from the input layer to the output layer without forming loops in the hidden layers (see [START_REF] Ma | Towards a mathematical understanding of neural network-based machine learning: what we know and what we don[END_REF] for general references). Let us have the case of a classical multilayer perceptron, that is a feedforward network composed of fully connected layers. While the latter networks are generally not the architecture of choice in most targeted applications, their architecture provides the most convenient way to understand the trade-offs between approximation efficiency and the complexity of the network.

Let X ⊂ R d X and Y ⊂ R d Y be some input and output sets of finite dimensions d X , d Y ∈ N * . A feedforward neural network ψ is a function ψ : X → Y which reads as ψ(x) = T L (σ(T L-1 (σ(. . . σ(T 0 (x))))), ∀x ∈ X , (1.12)
being, for every ℓ ∈ {0, . . . , L},

T ℓ : R p ℓ → R p ℓ+1 x ℓ → T ℓ (x ℓ ) = A ℓ x ℓ + b ℓ (1.13)
an affine function which can be expressed through a matrix A ℓ ∈ R p ℓ+1 ×p ℓ , and an offset vector b ℓ ∈ R p ℓ+1 . σ : R → R is called the (nonlinear) activation function. The smoothness of the activation function plays a key role in the accuracy of the optimization part of the algorithm, [START_REF] Devore | Neural network approximation[END_REF]. In [START_REF] Kutyniok | A theoretical analysis of deep neural networks and parametric pdes[END_REF] it is stated that, under some technical assumptions, there exists a DNN that approximates the discretized solution map.

Note that since ψ maps X onto Y, it is necessary that the number of neurons in the input layer is equal to the input dimension, p 0 = d X , and equivalently in the output layer, where p L+1 = d Y . The layers numbered from 1 to L are usually called the hidden layers of the neural network.

The main component of the neural network based methods is a nonlinear transformation with respect to its parameters. To define a class of feedforward neural networks, its architecture needs to be fixed by prescribing a given activation function σ, depth L ∈ N, and layer widths p = (p 0 , . . . , p L+1 ) ∈ (N * ) L+2 . Once the values of σ, L and p have been chosen, the coefficients (A ℓ , b ℓ ) 0≤ℓ≤L of the affine mappings T 0 , • • • , T L are viewed as parameters. We gather these coefficients in the vector of parameters, typically known as the weights in the neural network and named θ, that helps to define the transformation from x ℓ to T ℓ (x ℓ ). They read:

θ = {(A ℓ , b ℓ )} L ℓ=0
, and assume that θ takes values in a set

Θ ⊆ L ℓ=0 R p ℓ ×p ℓ+1 × R p ℓ+1 .
For any θ ∈ Θ, we define by ψ θ : X → Y the function ψ with θ = {(A ℓ , b ℓ )} L ℓ=0 ∈ Θ. The class of neural network functions with architecture (σ, L, p) and coefficient sets Θ is then defined as N (σ, L, p, Θ) := {ψ θ : θ ∈ Θ} .

(1.14)

Let us name K the set of neural networks for concrete values of the parameters, K ⊂ N . In the following scheme, Figure 1 Having made the choices of architecture and activation function of the network, the problem of design an approximation method for a target function f can be reduced to the one of choosing parameters θ.

Nevertheless, one must first specify the activation functions σ and initialize the parameters θ. The proper choice of initialization is important, because one tends to run into the problem of vanishing gradients whenever initialization is done poorly. Vanishing gradients occur when activation become saturated and thus these nodes no longer contribute to training, i.e. when the inputs x to the activation function σ(x) become so small, or large, that its gradient becomes vanishingly small. Activation functions introduce non-linearities into the neural network. Common activation functions include tanh, ReLU, swish and sigmoid. A deeper study of the use of the ReLU activation function is done in [START_REF] Devore | Neural network approximation[END_REF][START_REF] Daubechies | Nonlinear approximation and (deep) relu networks[END_REF][START_REF] Gonon | Deep relu neural networks overcome the curse of dimensionality for partial integrodifferential equations[END_REF][START_REF] Schmidt-Hieber | Nonparametric regression using deep neural networks with relu activation function[END_REF].

Physics-Informed Neural Networks (PINNs)

Being motivated by solving PDEs in the forward and inverse sense in regimes where data is sparse, combining the generality and power of machine-learning techniques with physics and domain knowledge has been deeply studied. To this end, Physics-Informed Neural Networks (PINNs, [START_REF] Raissi | Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[END_REF]) were developed. The method of PINNs is based on embedding prior knowledge about the physical systems into a neural network. The major innovation with PINNs is the introduction of a residual network that encodes the governing physics equations, takes the output of a deep-learning network, and calculates a residual value in order to minimize the loss function. In such a way, it is possible to build the space of possible solutions only with the solutions which we are interested in. Then, the information that the model obtains from the available data is magnified. This is done through a regularizer in the form of the governing equations which enforces the physical laws into the solution. As such, the model is not only more efficient with the dealt data, but also it is able to infer the solution by using the physical laws. This is especially useful in the instances where collecting the data is difficult or expensive.

PINNs methods approximate PDE solutions by training a neural network to minimize a loss function (mostly based on residuals of the equations); it includes terms reflecting the initial and boundary conditions along the space-time domain's boundary and the PDE residual at selected points in the domain (collocation points). The method retrieves the approximated function evaluated on the collocation points. One finds the coefficients of the neural network solution by minimizing a discretized version of the L 2 (Ω) norm of the strong form of the residual of the PDE. This is normally done by Gradient Descent (GD) based algorithms. One of the main advantages of this method is that it is very easily implementable but a lot of regularity in the solutions is implicitly assumed.

The basic concept behind PINN training is that it can be thought of as an unsupervised strategy that does not require labelled data, such as results from prior simulations or experiments. The PINN algorithm is essentially a mesh-free technique that finds PDE solutions by converting the problem of directly solving the governing equations into a loss function optimization problem. It works by integrating the mathematical model into the network and reinforcing the loss function with a residual term from the governing equation, which acts as a penalizing term to restrict the space of acceptable solutions.

Beyond the initial collocation version of PINNs expressed above, these methods to conservative PINNs (cPINNs, [START_REF] Ameya D Jagtap | Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems[END_REF]), variational PINNs (vPINNS, [START_REF] Kharazmi | Variational physics-informed neural networks for solving partial differential equations[END_REF][START_REF] Kharazmi | Variational physics-informed neural networks for solving partial differential equations[END_REF][START_REF] Kharazmi | hp-vpinns: Variational physics-informed neural networks with domain decomposition[END_REF]), stochastic PINNs (sPINNs, [START_REF] Zhang | Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems[END_REF]) between the developed variants.

Neural networks require a large amount of training data to perform well. For example, the image classification problem can only really be solved effectively by gathering sufficiently many labelled image examples for the neural network to train on. With today's powerful and efficient computers, training the neural network does not constitute a huge hurdle anymore, even with large data-sets and large networks. However, for many scientific applications, there can be a great difficulty and expense in gathering such great amount of data, especially for large scale systems. In those cases, data may be quite sparse in both space and time and it may be corrupted by noise. Hence, obtaining a good performance may be prohibitively expensive.

Physics-informed neural networks can address problems that are described by few data, or noisy experiment observations. They can use known data while adhering to any given physical law specified by general nonlinear partial differential equations, PINNs can also be considered as neural networks that deal with supervised learning problems. Let a differential equation on its more general form be:

R(u(x), y) = f (x), x ∈ Ω (1.15) S(u(x)) = g(x), x ∈ ∂Ω
Where x := [x 1 , . . . , x d-1 , t] indicates the space-time coordinate vector. The function u represents the unknown solution, y are the parameters related to the physics, f is the function identifying the data of the problem and R is the non linear differential operator. We denote S as the operator indicating arbitrary initial or boundary conditions related to the problem and g the boundary function.

In classical numerical methods, boundary conditions can be exactly enforced for mesh points at the boundary. Typically boundary conditions include Dirichlet, Neumann, Robin, and periodic boundary conditions [START_REF] Lawrence | Partial differential equations[END_REF]. However, it is very difficult to impose exact boundary conditions for a DNN representation. Therefore, in the loss function, it is common to add a penalty term which penalizes the difference between the DNN representation on the boundary and the exact boundary condition, typically in the sense of L 2 norm.

PDEs can describe numerous physical systems including both forward and inverse problems. The goal of forward problems is to find the function u(x) for every x, where y are specified parameters.

The forward and inverse PINNs wish to solve for two very different problems, but there are only very slight differences between how the two formulations are constructed. Recall that the forward problem is concerned with solving for u when the PDE and initial/boundary conditions are fully specified. The inverse problem however, is concerned with using data measured from the approximation of the solution and then estimating the unknown parameters in the PDE. The main difference comes from how the supervised loss function is defined, and which parameters we set as trainable variables. In this work, we are working with forward PINNs.

The usage of a DNNs to solve PDE can be summarized on three main parts:

1. Assembly of the loss function.

2. Neural network structure.

Solving the discretized minimization problem.

In the following, we introduce two of the most popular different methods to solve PDE's with the PINN method. The Deep Ritz Method and the Deep Galerkin Method.

The Deep Ritz Method

Let us consider the boundary value problem over a bounded domain Ω ⊂ R d introduced in Equation 1.15. To proceed, we assume that the problem is well-posed. The basic idea of solving a PDE using DNNs is to seek an approximated solution represented by a DNN in a certain sense [START_REF] Hornik | Multilayer feedforward networks are universal approximators[END_REF].

Let us start by recalling the Deep Ritz Method, [START_REF] Yu | The deep ritz method: a deep learning-based numerical algorithm for solving variational problems[END_REF], named so since it is based on using the neural network representation of functions in the context of the Ritz method, [START_REF] Ritz | Über eine neue methode zur lösung gewisser variationsprobleme der mathematischen physik[END_REF]. The Deep Ritz method is naturally nonlinear, naturally adaptive and its moderate computational cost gives the method the potential to work in rather high dimensions [START_REF] Han | Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations[END_REF][START_REF] Yu | The deep ritz method: a deep learning-based numerical algorithm for solving variational problems[END_REF][START_REF] Rotskoff | Active importance sampling for variational objectives dominated by rare events: Consequences for optimization and generalization[END_REF]. The framework is quite simple and fits well with the stochastic gradient descent method used in deep learning. It leverages the fact that the solution of certain PDEs is the unique minimizer to a certain energy functional.

The method is based on the following set of ideas:

• Deep neural network based approximation of the trial function.

• A numerical quadrature rule for the functional.

• An algorithm for solving the final optimization problem.

Any neural-network based numerical scheme for the solution of the kind of problems we are interested in relies on the use of a variational formulation of this problem which let us write the approximation of the solution u (or another function defined from u) as the solution of the minimization problem u = arg min v∈V J (v), (1.16) where V is a particular set of real-valued functions defined on a certain domain Ω.

The Ritz approximation, is defined by a solution u k such that for all v k ∈ V k ⊂ V, being V k the set of admissible functions, it satisfies Equation 1.15.

The minimization problem set in Equation 1.16 reads:

u k = arg min v k ∈V k J (v k ), (1.17)
and it is proved to converge. Hence, the loss function associated to Equation 1.15 reads:

J (v k ) = Ω |W (v k ) -f (x)v k | 2 dx
where we have named W (•) to the variational expression of the left-hand-side of the given PDE, R(v k ), and V k denotes the space of functions considering the boundary conditions.

Following the work of Raissi et al. [START_REF] Raissi | Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[END_REF], we approximate the solution of a PDE u(x, t) with a neural network parametrized by θ and with input variables (x, t). From now on, let us call it ψ θ (x, t). The form of this parametrized function is directly related to the architecture of the network and the activation functions used. Recall that in the PINN case, we are required to use activation functions which can be differentiated more than once. In some contemporary papers [START_REF] Krishnapriyan | Characterizing possible failure modes in physics-informed neural networks[END_REF][START_REF] Raissi | Hidden fluid mechanics: A navierstokes informed deep learning framework for assimilating flow visualization data[END_REF] the tanh(x) activation function was used.

Let the test function be now a NN, ψ θ (x), the loss function associated to the bulk term reads:

I(ψ θ ) = Ω |W (ψ θ ) -f (x)ψ θ | 2 dx
Where W (ψ θ ) is the variational expression of the left-hand-side part of the PDE, R(ψ θ ). DRM works in a weak sense that the variation of I(ψ θ ) with respect to ψ θ yields the associated Euler-Lagrange equation

R(ψ θ ) = f (x).
Since the NN doesn't necessarily belong to the space and it may not satisfy the boundary conditions, we impose them by adding them as a penalty term in the loss function

B(ψ θ ) = ∂Ω |Z(ψ θ ) -g(x)ψ θ | 2 dx
Where Z(ψ θ ) is the variational expression of the left-hand-side part of the boundary conditions of the PDE, S(ψ θ ).

And then the total loss function reads:

J (ψ θ ) = I(ψ θ ) + λB(ψ θ ) (1.18)
where λ ∈ [0, 1] is the penalization parameter. The optimal solution for the PDE is given by the solution of the minimization problem introduced in Equation 1.19.

Then, the problem of Equation 1.16 in the case in which the minimization process is computed by a Neural Network, often reads as:

θ * = arg min θ∈Θ J (ψ θ ) := 1 N N i=1 J i (ψ θ ) (1.19)
where θ * denotes the parameters of the NN that provide the approximation and each term at the right-hand side corresponds to one data point

(x i , y i ) N i=1 , N ∈ N. Conjointly, J i (ψ θ ) := J (ψ θ (x i ), y i ).
The minimization of the loss function evaluated in the collocation points (x j , y j ) M j=1 , for M ∈ N, will provide the values for the parameters that we await. In this expression, N , the number of data points, and M the number of evaluation points, are typically very large. Quadrature schemes for the high-dimensional integral run into the curse of dimensionality, and Monte-Carlo method can overcome this issue.

In order to solve the minimization problem defined in Equation 1.19, we make use of the Stochastic Gradient Descent algorithm (SGD, [START_REF] Bottou | Large-scale machine learning with stochastic gradient descent[END_REF][START_REF] Sirignano | Stochastic gradient descent in continuous time[END_REF]). The SGD algorithm and its variants play a key role in deep learning training. It is a first-order optimization method which naturally incorporates the idea of Monte-Carlo sampling and thus alleviates the curse of dimensionality. At each iteration, SGD updates the neural parameters by evaluating the gradient of the loss function at a batch of samples P ∈ N, P ≤ N , as:

θ k+1 = θ k -ϵ k 1 P P i=1 ∇ θ j i (θ k ) (1.20)
Here θ k are the parameters of neural network at the k-th iteration, ϵ k is the learning rate, and j i (θ k ) is used to approximate the loss function. The points x i are randomly generated with uniform distribution over the domain Ω and the boundary domain ∂Ω. This is the stochastic version of the gradient descent algorithm (GD). The key idea is that instead of computing the sum when evaluating the gradient of J , we simply randomly choose one term in the sum. Even though SGD requires more iterations to converge than GD, each iteration is much less computationally expensive.

Other strategies where less regular solutions are allowed are proposed in the literature. Usually, they start from its variational formulation. Let us cite the deep Galerkin method (DGM, [START_REF] Sirignano | Dgm: A deep learning algorithm for solving partial differential equations[END_REF]) that is based on a least-squares formulation, and the vPINNs (mentioned above), based on the Galerkin method. In this kind of methods, the quality of the approximation depends on the architecture of both the trial and the test neural network classes. In addition, numerous evaluations for multiple test functions need to be performed. In what follows, a brief introduction of Deep Galerkin Method is done in order to see other way to approximate the solution.

The Deep Galerkin Method

The Deep Galerkin Method, was proposed by Sirignano and Spiliopoulos [START_REF] Sirignano | Dgm: A deep learning algorithm for solving partial differential equations[END_REF]. The Galerkin method is a widely used computational method which seeks a reduced-form solution to a PDE as a linear combination of basis functions. The deep learning algorithm, DGM, uses a deep neural network instead of a linear combination of basis functions. The deep neural network is trained to satisfy the differential operator, initial condition, and boundary conditions using stochastic gradient descent at randomly sampled spatial points. In this case, the loss function is defined as the PDE residual in the least-squares sense.

In this case, the DNN approximation is substituted directly in the strong form of the equation 1.15, R(ψ θ ) = f (x) . The loss function associated to the bulk term is:

I(ψ θ ) = Ω |R(ψ θ ) -f (x)| 2 dx
DGM aims to minimize the imbalance when the approximate DNN solution is substituted into the strong form in the least-squares sense.

The inclusion of the boundary conditions in the loss function is done by adding the following penalty term. In an analogous way, the DNN is substituted in the strong form of the boundary conditions.

B(ψ θ ) = ∂Ω |S(ψ θ ) -g(x)| 2 dx
And then the total loss function reads:

J (ψ θ ) = I(ψ θ ) + λB(ψ θ ) (1.21)
where λ ∈ R is the penalization parameter. The optimal solution for the PDE is given by the solution of the minimization problem introduced in Equation 1. [START_REF] Bui-Thanh | Model reduction for large-scale systems with high-dimensional parametric input space[END_REF].

Then, the problem of Equation 1.16 is solved then in an analogous way as in DRM and the minimization process is solved by SGD method of Equation 1.20.

The highest derivative in the loss function in DRM is lower than that in DGM, thus it is thought that DGM works better for smooth solutions while DRM works better for low-regularity solutions. However, DRM can outperform DGM with a clear dependence of dimensionality even for smooth solutions and DGM can also outperform DRM for low-regularity solutions, [START_REF] Chen | A comparison study of deep galerkin method and deep ritz method for elliptic problems with different boundary conditions[END_REF].

Contributions of the thesis

In this work, the aim is to find numerical methods in order to approximate multivariate functions. They can be derived from datasets, multiway arrays, or solutions of PDEs coming from physics, biology, economy or other disciplines.

The task of approximating multivariate functions is done from three different perspectives in this thesis. The original contributions are the following:

1. A tensor approximation method, Sum of Tensor Trains (SoTT), that retrieves the approximated solution in a sum of TTs format.

2. A local tensor approximation method based on clustering, that retrieves an approximated solution in the introduced local HOSVD format.

3. A deep learning-based method that computes the approximation of the solution of a convection diffusion PDE when the diffusion parameter is very small.

Find in the following a brief overview of the contributions that can be found in the manuscript.

Contribution on Tensor Formats: The Sum of Tensor Trains method

In the first chapter of the thesis, we propose a numerical scheme, Sum of Tensor Trains (SoTT), that constructs an approximation of a tensor as a sum of tensor train decompositions, [START_REF] Ehrlacher | Sott: greedy approximation of a tensor as a sum of tensor trains[END_REF].

Among the different existing tensor formats in the literature, two of them present specific importance with respect to applications: the Canonical Polyadic (CP) and Tensor Train (TT) format. As we can see in Equation 1.4, the CP format presents a low memory cost but may be ill-posed and present instabilities. On the other hand, recalling Equation 1.9, the TT format is stable from an algorithmic point of view and it is computationally affordable (provided that the TT ranks of the tensors remain reasonably small) but the order of the variables needs to be fixed a priori.

The aim of this work is to find a method that is efficient in memory and that doesn't fix a priori the order of the variables or the ranks. This is done in such a way that the resulting approximation presents a Sum of Tensor Trains (SoTT) format.

The advantages of this procedure are:

• It selects in an adaptive way the order of the variables in each term so as to obtain favorable compressing rates with respect to pure TT approximations with an a priori prescribed order of variables.

• When the values of the ranks of the terms are fixed to be equal to one, the procedure provides a new scheme for the computation of a CP approximation of a given tensor (namely CP-TT), which appears to be more efficient than ALS for high-order tensors.

In the proposed SoTT method, the order of the variables and the ranks are greedily chosen by the method and they can vary from one term to another. They are the result of an optimization step based on the idea of an energy-balance between the accuracy of the approximation of a tensor and the computational cost needed to store it. This is done by defining a certain minimization problem in which the functional to minimize has two terms, one represents the memory spent to store the approximation and the other one is related with the energy (information) kept from the system.

The main goal of the SoTT algorithm is to compute, after n iterations, an approximation of a certain tensor F as a sum of n TTs. At iteration n, the SoTT computes an approximation of F under the form

F ≈ F n-1 + R n 1 (x τ n (1) )R n 2 (x τ n (2) ) • • • R n d (x τ n (d) ) (1.22)
where F n-1 is the approximation obtained after n -1 iterations of the algorithm, τ n ∈ {1, . . . , d} denotes the permutation of the variables on the iteration n and, for all 1

≤ j ≤ d, let R n j ∈ L 2 Ω τ n (j) , R K n j-1 ×K n j
, be the modes obtained in the SVD of the tensor:

R n j (x τ n (j) ) = U τn(j),n j,kj (k j-1 , x τ n (j) ) 1≤kj ≤K n j ,1≤kj-1≤K n j-1
where

K n 0 = K n d = 1.
The aim of the n th iteration is to choose the permutation τ n and the values of the ranks (K n j ) 1≤j≤d-1 in an appropriate way, which is done here using a greedy procedure.

Fixing the ranks as one in all the steps of the algorithm, a CP approximation of the given tensor is obtained. We name this rank-1 particular instance as the Canonical Polyadic -Tensor Train (CP-TT) method. As an output, after n iterations of the CP-TT algorithm, the method greedily produces an approximation of the tensor F under the CP format

F ≈ n k=1 R k 1 (x τ k (1) ) • • • R k d (x τ k (d) ) (1.23)
where for all 1 ≤ k ≤ n and all 1

≤ i ≤ d, R k i ∈ L 2 (Ω τ k (i) ).
We propose a proof of convergence in the general case of the SoTT algorithm, which can be extended to the case of the CP-TT algorithm.

In the particular case of the CP-TT method, we take advantage of the simplicity of the CP format and the efficiency of the TT. CP-TT is exempt of the ill-posedness of the solving methods present in the canonical format and it has the advantage of not choosing the order of the variables a priori like in the TT format. Thus, the CP-TT method is a hybrid rank-1 approximation method that gets the best of both worlds from the TT and the CP methods.

In order to check the performance of the proposed techniques, several numerical experiments were conducted. First, we compared the CP-TT to other rank-one update methods (ALS, ASVD, TTr1). Although a single iteration of CP-TT is more expensive in terms of number of operations, it presents a stability that makes it a promising candidate to compress high-dimensional tensors in CP format. We propose some tests in which we compressed the numerical solution of a parametric partial differential equation of reaction-diffusion type as well as other functions coming from different applications. In particular, we compared SoTT with the TT-SVD obtained by testing all the possible permutations of the indices. Although SoTT is suboptimal with respect to the best TT-SVD, it is independent of the order of the variables and its performances are comparable to the average TT-SVD. In this test, the SoTT method outperforms CP-TT.

The SoTT method and its rank-1 particular case show preliminary yet encouraging results in view of applications in scientific computing and compression of high order tensors. Nevertheless, the SoTT method shows some shortcomings to be addressed in further investigations: while a greedy method is appealing in view of computational tasks in which fixing the rank a priori could be cumbersome, it might be featured by a saturation effect, slowing down its convergence.

Contribution in local tensor methods: The local HOSVD method

In the second chapter of the thesis, we present a method that computes an approximation of a tensor in a local HOSVD format.

In the classical compression methods, one of the most common ways to compute the approximation of a tensor is through projection. Usually, one must choose onto which space to project and normally the basis of the space consists of functions whose support is the whole domain. Nevertheless, the choice of an appropriate projection space is crucial to obtain a good approximation.

The tensor to be approximated is a multivariate function F (x 1 , . . . , x d ) of dimension d defined in Ω := Ω 1 × . . . × Ω d . The aim of the proposed method is to provide an approximation of the tensor on the local HOSVD format with good compression rate with respect to the HOSVD decomposition.

The main idea of the method is that a tensor defined in a global domain can be expressed as the sum of its approximations on local subdomains Ω (i) :

F ≈ F (x 1 , . . . , x d ) = N i=1 F (x 1 , . . . , x d )I (i) (x 1 , . . . , x d ) = N i=1 Fi (x 1 , . . . , x d ) (1.24)
where I (i) (x) are the characteristic functions that restrict the tensor into a certain subdomain.

I (i) (x) = 1 if I (i) (x) = 1 if x ∈ Ω (i) 0 otherwise
The method is constructed in such a way that on each of the partitions of the domain, a local approximation of the function (restricted to the partition of the domain) is computed by applying the HOSVD method. Then, the approximation reads, for 1 A crucial point is to determine the subdomain partition. In the proposed method, we don't impose the partition a priori. The aim is to come up with a method that computes automatically the partition. This is done with an agglomerative clustering algorithm. The clustering process is computed in 2 different phases and it is applied independently to every direction k, with 1 ≤ k ≤ d, exploiting that the subdomains Ω (i) are separable. Henceforward, the problem of finding the partition of Ω is reduced to the problem of partitioning each of the domains per direction.

≤ i ≤ N : Fi = R1 i1=1 . . . R d i d =1 G i1...i d b (1) i1 × . . . × b (d) i d in which R 1 , . . . ,
The clustering method computes a dyadic hierarchical tree of the possible partitions of the domains Ω

(i) k 1≤k≤d
.

1. It starts by computing the clusters of fibers that will be held as the leaves of the tree. This is done by grouping the fibers considered as similar and finding the 2-dimensional basis that represents well all of them.

The partition of the set of fibers in smaller subsets is mutually exclusive (no fiber in the set is in more than one subset) and jointly exhaustive (every fiber is in some subset). The clustering provides the set of indices of the fibers that belong to each cluster, the set of bases that define them and the set of errors that we are making when we group the fibers.

2. It computes the pairwise Ward's distance [START_REF] Jr | Hierarchical grouping to optimize an objective function[END_REF], between the clusters in the leaves. Find the neighbor clusters and merge them. The resulting cluster encloses the fibers of its two sons.

The algorithm starts pairing the similar leaves and it carries on pairing till every node in the tree is fused. Each one of the levels l k of the tree denotes a possible partition of the domain on the direction k.

In the root of the tree, the point in which every cluster is fused, the POD decomposition of the unfolding transposed in the direction k is recovered.

Each of the nodes defines a partition of the domain. The total number of possible partitions in all the directions reads as d k=1 l k . The closest the nodes are to the root of the tree, the biggest the clusters are and the less the domain is divided.

In the present work, in order to find the best partition, an extensive search between the possible partition combinations is done. Then, a HOSVD approximation of the tensor on each one of the partitions is computed. The selected criterion in order to choose one partition over the others is the memory needed to store the approximation. For every possible combination of the partitions of the domain obtained, the method computes the basis that generates the subspace defined by the partition performing HOSVD on the truncated space. The one that requires less amount of memory to be stored is said to be the best of the partitions of the domain and it is the one that we are going to use to compute the approximation of the tensor. Some tests have been performed for moderate order tensors. In the light of the numerical results obtained, we can say that the method achieves a good compression rate with respect to the HOSVD method. This fact is specially remarkable for the systems that present different "regimes" and the local approximation seems to be more advantageous with respect to a global approximation. Computing partitions of the domain per direction allows the method to detect the different patterns on each direction and obtain more information spending the same amount of memory.

Contribution in Deep Learning-based Schemes for Singularly Perturbed Convection-Diffusion Problems

In the third chapter of the thesis, we propose a deep learning-based method in the context of finding an approximated solution of the convection-diffusion PDE, when the diffusion parameter tends to zero and classical methods like FEM are not able to solve it correctly, [START_REF] Beguinet | Deep learning-based schemes for singularly perturbed convection-diffusion problems[END_REF].

Let us consider the following singularly perturbed convection-diffusion equation on a given domain Ω ⊂ R d , with d ∈ N * . Let F : Ω → R d be a given force field, 0 < ϵ ≪ 1 a small parameter, and f : Ω → R be a given right-hand side function. Our goal is to find a solution u : Ω → R to

-ϵ(∆u)(x) + ∇ • (F u)(x) = f (x), ∀x ∈ Ω, (1.25)
together with general Robin boundary conditions

α(∇u • n)(x) + κu(x) = g(x) ∀x ∈ ∂Ω, (1.26)
where n refers to the outward unit vector of ∂Ω, α, κ ≥ 0 and g is a real-valued function defined on ∂Ω. In the following, we assume that the force field F derives from a potential function V : Ω → R, in the sense that

F (x) = -∇V (x), ∀x ∈ Ω.
Under appropriate assumptions on F , f and g, which are assumed to be smooth functions for the sake of simplicity, the problem can be proved to have a unique solution [START_REF] Risken | The Fokker-Planck Equation[END_REF][START_REF] Jordan | The variational formulation of the fokker-planck equation[END_REF][START_REF] Bogachev | Fokker-Planck-Kolmogorov Equations[END_REF].

The problem introduced in Equation (1.25) is challenging for classical numerical methods because it presents numerical instabilities when ϵ is small. The aim of this work is to propose and test various neural network-based schemes in order to find a strategy that is robust for small values of ϵ in the convection-diffusion problem.

Any neural-network based numerical scheme for the solution of the kind of problems we are interested in, relies on the use of a variational formulation of this problem which makes it possible to write the approximation of the solution u (or another function defined from u) as the solution of the minimization problem

min v∈V J (v), (1.27)
where V is a particular set of real-valued functions defined on a certain domain Ω. The loss function J : V → R is usually of the form

J (v) := λ Ω R(v)(x)dρ(x) + (1 -λ) ∂Ω S(v)(r)dτ (r), ∀v ∈ V, (1.28)
where λ is the penalization parameter and for every v ∈ V, R(v) and S(v) are real-valued functions defined on Ω and ∂Ω respectively. They are assumed to be measurable with respect to ρ and τ , which are defined on the domain Ω and its boundary ∂Ω respectively.

In order to assemble the loss function from the PDE and its boundary conditions, we propose different variants and we compare among the results obtained which one is more suitable for our problem. Let us start by the the most classical formulation used in neural network-based numerical schemes, in which the loss function is directly the strong formulation of the problem 1.25,

R(v)(x) = | -ϵ(∆v)(x) + ∇ • (F v)(x) -f (x)| 2
analogously for the boundary conditions

S(v)(x) = |α(∇v • n)(x) + κu(x) -g(x)|2
and therefore we refer to it as Vanilla (V) formulation. Remark that the approach requires the solution u to be more regular than the usual solution. For this reason, we develop an approach that requires less regularity in the solutions. It is computed by introducing an exponential change of variable in the vanilla formulation and solving the minimization problem. We call it the Vanilla-z (Vz) formulation. This method does not fully exploit the change of variable. Since the elliptic nature of the problem allows us to easily build a weak formulation of this equation, other strategy is then developed in which the loss function is written on its weak form, the Weak-z (Wz) formulation. Last, we introduce another formulation based on a change of scale in the original problem. We refer to this approach as the rescaled-weak-z (RWz) formulation.

The proposed method is based on Physics-Informed Neural Networks (PINN, [START_REF] Raissi | Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[END_REF]). PINN is a collocation-based method, [START_REF] Xiu | High-order collocation methods for differential equations with random inputs[END_REF], in which the problem defined in a PDE is encoded as a part of the neural network. These models transform the problem of finding a solution for the PDE (1.25) into a minimization problem, and its solving process relies on the neural network. The PINNs method finds the coefficients of the neural network solution by minimizing a discretized version of the L 2 (Ω) norm of the strong form of the residual of the PDE. This method is very easily implementable but it implicitly assumes that solutions are very regular.

One of the objectives of this work is to see how the introduced alternatives for the shape of the loss function and the strategy adopted to solve the problem affecting the neural network approach, as well as a comparison between them. Moreover, a survey on other factors that also impact the results is done. These factors include the number of K training points used, the architecture of the NN and the precision of the machine, between others.

The last step of the study is a comparison between the merits and limitations of using deep learning-based approaches (for the different formulations studied) and the classical Finite Elements Method (FEM) when the solutions become low regular. See more about FEM techniques applied to convection-diffusion equations in [START_REF] Xu | A monotone finite element scheme for convection-diffusion equations[END_REF][START_REF] Br Baliga | A new finite-element formulation for convection-diffusion problems[END_REF][START_REF] Eriksson | Adaptive streamline diffusion finite element methods for stationary convection-diffusion problems[END_REF]. Some hybrid models between NNs and FEM are proposed as well in [START_REF] He | Relu deep neural networks and linear finite elements[END_REF][START_REF] Arora | Understanding deep neural networks with rectified linear units[END_REF].

For large values of ϵ when solutions are rather regular, some PINNs perform clearly better than FEM regarding the generalization errors. The superiority is particularly remarkable for very small number of training points. However, the shapes of PINN solutions are sometimes not as satisfactory as the ones given by FEM. For the case where ϵ becomes small and solutions become less regular (which was the main motivation of our study), the accuracy of the variational neural-network methods is essentially comparable or worse to the one given by FEM in terms of generalization errors. Some PINN variational approaches become too unstable and the errors blow up. Only FEM and the vanilla PINN approach seem to be able to recover the correct shape of the exact function. The latter one has however the risk of sometimes falling into local minima with bad shapes. The runtimes are clearly in favor to PINN methods as well as the simplicity of implementation.

Organization of the manuscript

The first chapter of this work, Chapter 2, is devoted to the development of the Sum of Tensor Trains (SoTT) algorithm in which the decomposition of a tensor is obtained with the shape of a sum of Tensor Trains. It is also shown its particular case for a rank-1 approximation, the so called Canonical Polyadic -Tensor Train (CP-TT), in which the Canonical Polyadic is recovered. Some numerical tests are shown, some of them showing the compression of the algorithm in comparison with other methods and others showing its efficiency in terms of cost and memory. In Appendix A.1, some results and tests are included.

In the second chapter of the manuscript, Chapter 3 a local method for tensor compression is proposed. The approximation of the tensor is retrieved in a local HOSVD format. It is a clustering-based method and it performs in order to compute local subdomains in which the function restricted to the subspace is approximated. This is done with the aim of approximating solutions of tensor in which the global approximation was not very effective. By construction, if the best way to approximate the tensor is the global one, it recovers the HOSVD method. Some numerical examples are included in which we can see diverse applications of the method.

In the third chapter, Chapter 4, an analysis of Deep Learning-based Schemes for Singularly Perturbed Convection-Diffusion Problems is done. Specifically, the stationary Fokker-Plank equation. In order to approximate this particular equation, we make use of Physics-Informed Neural Networks (PINNs). A survey of some possible variations of the method and an anaysis of how their input format affects the results is studied. The corresponding tests in order to illustrate the different behaviors of the obtained approximation are also included in Appendix B.1.

Chapter 2

Sum of Tensor

Introduction

Nowadays, applications such as artificial intelligence and data mining algorithms are becoming more and more important in our society. The task of analyzing large volume, multi-relational and multi-modal datasets, is gaining importance too. These datasets are often represented as multiway arrays or tensors [START_REF] Cichocki | Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[END_REF][START_REF] Kolda | Tensor decomposition: A mathematical tool for data analysis[END_REF]3].

The main challenge in dealing with such data is the already cited curse of dimensionality, that refers to the need of using a number of degrees of freedom exponentially increasing with the dimension [START_REF] Oseledets | Breaking the curse of dimensionality, or how to use svd in many dimensions[END_REF]. This problem can be alleviated by using various tensor formats, achieved by low-rank tensor approximations, for the compression of the full tensor as described for instance in [START_REF] Kolda | Tensor decompositions and applications[END_REF][START_REF] Cichocki | Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[END_REF][START_REF] Domanov | On uniqueness and computation of the decomposition of a tensor into multilinear rank-(1, lr, lr) terms[END_REF][START_REF] Friedland | The number of singular vector tuples and uniqueness of best rank-one approximation of tensors[END_REF]. The definition of these different tensor formats relies on the well-known separation of variables principle. Some of the most used tensor formats have been already presented in the Introduction: Canonical Polyadic (CP), Tensor Train (TT), Tucker and High Order Singular Value Decomposition (HOSVD); and they are defined in Equation 1.4, 1.9, 1.7, 1.8, respectively. We refer the reader to [START_REF] Hackbusch | Tensor spaces and numerical tensor calculus[END_REF] and [START_REF] Boris | Tensor numerical methods in scientific computing[END_REF] for extensive reviews on tensor theory and extended analysis of tensor decompositions and their numerous applications. Tensor formats are also used for solving time-dependent and stochastic/parametric PDEs ( [START_REF] Khoromskaia | Tensor-based techniques for fast discretization and solution of 3d elliptic equations with random coefficients[END_REF], [START_REF] Babuška | A stochastic collocation method for elliptic partial differential equations with random input data[END_REF]). Other families of methods called Proper Generalized Decomposition (PGD) methods use tensor decomposition in high dimensional problems [START_REF] Nouy | A priori tensor approximations for the numerical solution of high dimensional problems: alternative definitions[END_REF], [START_REF] Ladevèze | The latin multiscale computational method and the proper generalized decomposition[END_REF].

Among the different existing tensor formats, two of them are of specific importance with respect to applications: the Canonical Polyadic (CP) and Tensor Train (TT) format. The main advantage of these decompositions is the low memory cost needed to store them. In the case of the CP format, this cost only scales linearly with the order of the tensor, whereas the memory cost for the storage of a full tensor scales 29 exponentially with its order. However, the problem of finding a best approximation of a tensor in CP format may be ill-posed [START_REF] De | Tensor rank and the ill-posedness of the best low-rank approximation problem[END_REF] and it leads to numerical instabilities. The most classical algorithm in order to compute an approximation of a tensor in the CP format is the so-called Alternating Least Square (ALS) method, which sometimes may be quite slow to converge [START_REF] Beylkin | Algorithms for numerical analysis in high dimensions[END_REF] especially for high-order tensors. Some alternative methods [START_REF] Vandecappelle | Nonlinear least squares updating of the canonical polyadic decomposition[END_REF][START_REF] Rajih | Enhanced line search: A novel method to accelerate parafac[END_REF] have been proposed in order to obtain more efficient algorithms.

The Tensor Train format is probably one of the most used tensor formats in realistic applications [START_REF] Bigoni | Spectral tensor-train decomposition[END_REF][START_REF] Wang | A distributed tensor-train decomposition method for cyber-physical-social services[END_REF][START_REF] Rakhuba | Calculating vibrational spectra of molecules using tensor train decomposition[END_REF], due to a good trade off between optimality and numerical stability. The TT format combines two advantages to take into consideration: on the one hand, it is stable from an algorithmic point of view; on the other, it is computationally affordable provided that the TT ranks of the tensors remain reasonably small. The computation of the approximation of a tensor in the TT format is usually done via the so-called TT-SVD algorithm. One of the drawback of the TT-format is that it requires a priori the choice of a particular order in the variables of the tensor, and the quality of the resulting approximation computed by a TT-SVD algorithm strongly depends on this particular choice. Even if the number of entries could be larger than in CP, the main advantage of the TT format is its ability to provide stable quasi-optimal rank reduction, obtained, for instance, by truncated singular value decompositions.

In the literature, hybrid formats combining CP with other methods have been proposed in [START_REF] Phan | Stable low-rank tensor decomposition for compression of convolutional neural network[END_REF], and described in [START_REF] Grasedyck | A literature survey of low-rank tensor approximation techniques[END_REF]. Also, CP has been combined with TT in [START_REF] Kour | Efficient structure-preserving support tensor train machine[END_REF], where it was highlighted that the combination of both methods yields interesting improvements. Fast algorithms for the rank truncation in the canonical input tensors with large CP-ranks and large mode size, have been introduced and analyzed in [START_REF] Boris | Multigrid accelerated tensor approximation of function related multidimensional arrays[END_REF]. Some other optimization-based algorithms could be seen in [START_REF] Sorber | Optimization-based algorithms for tensor decompositions: Canonical polyadic decomposition, decomposition in rank-(l r,l r,1) terms, and a new generalization[END_REF].

The main contribution of this chapter is a numerical scheme that constructs an approximation of a tensor as a sum of TTs, called the Sum of Tensor Trains (SoTT) scheme, where the order of the variables and the values of the ranks can vary from one term to another and can be adaptively chosen by an algorithm which combines the TT-SVD algorithm together with a greedy procedure (see [START_REF] Vladimir | Greedy algorithms andm-term approximation with regard to redundant dictionaries[END_REF]). The interest of such a procedure is two-fold:

• It enables to select in an adaptive way the order of the variables in each term so as to obtain favorable compressing rates with respect to pure TT approximations with an a priori prescribed order of variables.

• When the values of the ranks of the terms computed are fixed to be equal to one, the procedure provides a new scheme for the computation of a CP approximation of a given tensor (namely CP-TT), which appears to be more efficient than ALS for high-order tensors.

We also observe numerically that this algorithm performs well in practice in the sense that it provides a more accurate approximation of a given tensor, at fixed memory storage cost, than a TT-SVD algorithm, in average, when the order of the variables in the TT decomposition is chosen randomly. We also consider a particular version of the SoTT algorithm, named CP-TT, which consists in adding pure rank-1 tensor-product at each iteration of the scheme. This procedure can be used in order to compute a CP approximation of a given tensor. Such a scheme gives interesting results in comparison with other rank-1 update methods such as ALS for instance, especially when the order of the tensor is high.

Organization of the chapter

This chapter is structured as follows. In Section 2.2, some preliminary concepts about tensors are recalled or introduced. The SoTT algorithm is presented and discussed in Section 2.3. The CP-TT version of SoTT is discussed in Section 2.4 and compared with other numerical methods used for the construction of CP decompositions. Numerical experiments and results illustrating the efficiency of the approach are given in Section 2.5.

Notation and preliminaries

This section is devoted to the presentation of some concepts and notations that are going to be used in the following chapter. Let us recall some of the concepts introduced before and particularize them to its concrete use on this section. Tensors are well-studied objects in the standard mathematics literature ( [START_REF] William | Algebra: an approach via module theory[END_REF], [START_REF] Steven | Abstract algebra[END_REF], [START_REF] Lang | Algebra[END_REF]) and more specifically in multilinear algebra ( [START_REF] Hackbusch | Multi-grid methods and applications[END_REF], [START_REF] Marcus | Finite dimensional multilinear algebra[END_REF]). In this section, we introduce the tensor spaces for multivariate functions and some of the objects that concern tensors. Furthermore, we recall the concept of tensor rank and some of the tensor formats more relevant in the literature. We begin by introducing some notations together with the well-known Singular Value Decomposition in Section 2.2.1. We then recall some basic facts about the Canonical Polyadic (CP) and Tensor Train (TT) format in Section 2.2.3. The most common algorithm to compute the CP format ALS and one of its variations, the Alternating Singular Value Decomposition (ASVD) are introduced in Section 2.2.3. The classical TT-SVD algorithm is then presented in Section 2.2.3.

Tensor spaces

Let D be a subset of R p with a product structure

D := D 1 × . . . × D d . Let us consider d ∈ N * . Then, for all 1 ≤ i ≤ d, D i is
an open bounded subset of R pi for some p i ∈ N * . We call a tensor (with a slight language abuse) any real-valued function

F ∈ L 2 (D 1 × • • • × D d ).
For each 1 ≤ i ≤ d, a certain H i is a Hilbert space of univariate functions defined of D i , equipped with the corresponding inner product (•, •) Hi and the associated norm ∥ • ∥ Hi .

The elementary tensor product u (1)

⊗ . . . ⊗ u (d) of d univariate functions u (i) ∈ H i has associated a multilinear mapping D 1 × . . . × D d → D 1 ⊗ . . . ⊗ D d and is defined such that for a multivariate function F and x = (x 1 , . . . , x d ) ∈ D, F (x) = (u (1) ⊗ . . . ⊗ u (d) )(x 1 , . . . , x d ) = u (1) (x 1 ) ⊗ . . . ⊗ u (d) (x d ).
The span of these elementary tensor products is the algebraic tensor space H 1 ⊗ . . . ⊗ H d where belong all the multivariate functions that can be expressed as a finite linear combination of elementary tensors (see more in [START_REF] Werner | Linear algebra[END_REF]).

For any u (1)

∈ L 2 (D 1 ), • • • , u (d) ∈ L 2 (D d ), we denote u (1) ⊗ • • • ⊗ u (d) ∈ L 2 (D) the pure tensor product function defined by u (1) ⊗ • • • ⊗ u (d) : D = D 1 × • • • × D d → R (x 1 , • • • , x d ) → u (1) (x 1 ) • • • u (p) (x d ).
A canonical inner product over the tensor space between F and other multivariate function namely V ∈ L 2 (D 1 × . . . × D d ), whose elementary tensors are denoted by v (i) , with i ≤ i ≤ d, is defined by:

(u (1) ⊗ . . . ⊗ u (p) , v (1) ⊗ . . . ⊗ v (d) ) = (u (1) ⊗ v (1) ) H1 . . . (u (d) ⊗ v (d) ) H d .
This tensor product holds bilinearity properties and in the literature can be referred as Kronecker product.

Its associated canonical norm ∥ • ∥ Hi , verifies that:

∥u (1) ⊗ . . . ⊗ u (d) ∥ = ∥u (1) ∥ H1 . . . ∥u (d) ∥ H d . A tensor
Hilbert space H of dimension d is defined as the completion of this algebraic tensor space

H = H 1 ⊗ . . . ⊗ H d .
Moreover, we make use of the following abuse of notation. For any nonempty subset

I ⊂ {1, • • • , d} such that I c = {1, • • • , d} \ I is non-empty, and any F ∈ L 2 (D 1 × • • • D d ), we still denote by F the function F ∈ L 2 (( i∈I D i ) × ( j∈I c D j )) defined by F : ( i∈I D i ) × ( j∈I c D j ) → R ((x i ) i∈I , (x j ) j∈I c ) → F (x 1 , • • • , x d ).
For each 1 ≤ i ≤ d, we consider L 2 (D i ), the Hilbert space of sqare integrable, real-valued univariate functions defined on

D k . Its natural norm ∥ • ∥ L 2 (Di) is defined for u (i) ∈ L 2 (D i ) by ∥u (i) ∥ 2 L 2 (Di) = Di u (i) (x i ) 2 dx i .
The space L 2 (D) of square-integrable, real-valued functions defined on D equipped with the canonical norm

∥ • ∥ L 2 (D) that is defined for F ∈ L 2 (D) by ∥F ∥ 2 L 2 (D) = D F (x) 2 dx and ∥F ∥ L 2 (D) := D F 2 (x)dx 1/2
and it can be identified with the completion of the algebraic tensor spaces

L 2 (D 1 ) ⊗ . . . ⊗ L 2 (D p ). The scalar product for all F, G ∈ L 2 (D), ⟨F, G⟩ D := D F (x)G(x)dx
From now on, in order to simplify the notation of the manuscript (every time that there's no ambiguity) the norm ∥ • ∥ L 2 (D) will be denoted just as ∥ • ∥ D and its associated inner product as (•, •) D .

Let us have a discrete d-dimensional subset D h ⊂ R N1×N2×...×N d . For each direction i we fix a index set N i = {1, . . . , N i }, that denotes the degrees of freedom per direction. The Cartesian product of these index sets is denoted as

N = N 1 × . . . × N d .
Being d the dimension of the discrete tensor. Let us refer to discrete tensor (or just tensor) to the discretization of the multivariate function introduced before. A dth-order tensor is an element of the tensor product of d vector spaces D 1 × . . . × D d . Up to a certain choice of basis on the spaces (each has its own coordinate system), such an element has to be represented as a d-way array of real numbers. Tensors will carry d sub indices i 1 , . . . , i d ∈ {1, . . . , d} to denote them by elements. A certain discrete tensor x ∈ R N1×...×N d is defined by its entries, x i1...i d .

The Frobenius norm, or the l 2 norm in a discrete d-dimensional tensor space is the square root of the sum of the squares of all its elements. For a certain F ∈ L 2 (D h ):

∥F ∥ 2 l 2 = d k=1 |F i | 2
if the norm is not explicitly specified, we assume that is the Frobenius norm.

Object definition and POD decomposition

The so called fibers all along the manuscript are the higher-order analogue of matrix rows and columns, defined by fixing every index of a tensor but one. In a 3-dimensional tensor X ∈ R N1×N2×N3 , for a certain value of two of its indices i 2 ∈ {1, . . . , N 2 } and i 3 ∈ {1, . . . , N 3 }, a fiber on the i 1 -th direction is given by x (i1) = X :i2i3 , with i 1 ∈ {1, . . . , N 1 }. When fibers are extracted from tensors, they are assumed to be column vectors.

Assume a N -th order tensor X ∈ R N1×...×N N . The matrix Unfolding X (n) ∈ R Nn×(Nn+1...N N N1...Nn-1) contains the element x i1...ip at the row position i n and column number equal to:

(i n+1 -1)N n+2 . . . N N N 1 . . . N n-1 + i n+2 -1)N n+3 . . . N N N 1 . . . N n-1 + . . . . . . + (i N -1)N 1 . . . N n-1 + (i 1 -1)N 2 . . . N n-1 + . . . + i n-1
In a third order tensor X ∈ R N1×N2×N3 , the horizontal, lateral, and frontal slices are denoted by X i1:: , X :i2: , and X ::i3 , respectively. When we refer to the n-th frontal slice of a third order tensor, we obtain its unfolding X (n) .

Proper orthogonal decomposition (POD)

The Proper Orthogonal Decomposition (POD) [START_REF] Pearson | Liii. on lines and planes of closest fit to systems of points in space[END_REF] is a popular dimensionality reduction method used in model reduction (see [START_REF] Moore | Principal component analysis in linear systems: Controllability, observability, and model reduction[END_REF], [START_REF] Willcox | Balanced model reduction via the proper orthogonal decomposition[END_REF], [START_REF] Carlberg | Efficient non-linear model reduction via a least-squares petrov-galerkin projection and compressive tensor approximations[END_REF]) to define the trial subspace.

Definition. For any domain D = D x × D y , where D x and D y are open subdomains of R dx and R dy for some d x , d y ∈ N * respectively, and any W ∈ L 2 (D), it holds that there exists an orthonormal basis (U k ) k∈N * of L 2 (D x ), an orthonormal basis (V k ) k∈N * of L 2 (D y ) and a non-increasing sequence (σ k ) k∈N * of non-negative real numbers which converges to 0 as k goes to ∞, such that

W = k∈N * σ k U k ⊗ V k . ( 2

Ranks and tensor formats: Canonical Polyadic (CP) and Tensor Train (TT)

Let us define the rank of a tensor F , denoted rank(F ) or just r, as the smallest number of rank-one tensors that generate F as their sum.

rank(F ) = min{r : F ∈ R r }
where R r is the set of tensors of rank not exceeding r.

The tensor rank decomposition factorizes a tensor into a minimum-length linear combination of rank-one tensors. Let F be a real tensor obtained by the discretization of a certain d-variate function

F ∈ D 1 × • • • × D d .
Every order-d tensor in this space may then be represented with a suitably large r ∈ N * , as a linear combination of r rank-one tensors.

More details about the concepts of tensor rank and the decomposition of a tensor into a sum of outer products of elementary tensors introduced in [START_REF] Frank | The expression of a tensor or a polyadic as a sum of products[END_REF] and [START_REF] Frank | Multiple invariants and generalized rank of a p-way matrix or tensor[END_REF], can be found in [START_REF] De | Tensor rank and the ill-posedness of the best low-rank approximation problem[END_REF].

Although we focus on the previous concept of rank, there is a simpler notion of multilinear rank which directly generalizes the column and row ranks of a matrix to higher order tensors. Let us illustrate this concept with a 3-dimensional example. Let us have a certain tensor F ∈ R N1×N2×N3 . Note that the space can be viewed as R N1×N2N3 (similarly in the other directions). Then r 1 (F ) is simply the rank of the first one of its unfoldings. The same for r 2 (F ) and r 3 (F ). Then, the multilinear rank of F , is the tuple (r 1 (F ), r 2 (F ), r 3 (F )) and it was also introduced in [START_REF] Frank | The expression of a tensor or a polyadic as a sum of products[END_REF], [START_REF] Frank | Multiple invariants and generalized rank of a p-way matrix or tensor[END_REF].

The Canonical Polyadic (CP) decomposition

As it is detailed in [START_REF] Tamara | Tensor decompositions and applications[END_REF], the idea of the polyadic form of a tensor (expressing a tensor as the sum of a finite number of elementary tensors) arose initially in [START_REF] Frank | The expression of a tensor or a polyadic as a sum of products[END_REF], [START_REF] Frank | Multiple invariants and generalized rank of a p-way matrix or tensor[END_REF]. The form of CANDECOMP (canonical decomposition), was proposed in [START_REF] Carroll | Analysis of individual differences in multidimensional scaling via an n-way generalization of "eckart-young" decomposition[END_REF] and the form of parallel factors in [START_REF] Richard A Harshman | Foundations of the parafac procedure: Models and conditions for an" explanatory" multimodal factor analysis[END_REF]. We refer to the CP decomposition as proposed independently in [START_REF] Henk | Towards a standardized notation and terminology in multiway analysis[END_REF] and [START_REF] Mocks | Topographic components model for event-related potentials and some biophysical considerations[END_REF].

Let us now introduce some notation which will be used in all the sequel. From now on, we fix some d ∈ N * and for all 1 ≤ j ≤ d, let Ω j be an open subset of R pj for some p j ∈ N * . We define

Ω := Ω 1 × • • • × Ω d . Let F ∈ L 2 (Ω 1 × • • • × Ω d ).
The function F is said to belong to the Canonical Polyadic (CP) format [START_REF] Frank | The expression of a tensor or a polyadic as a sum of products[END_REF][START_REF] Kolda | Tensor decompositions and applications[END_REF][START_REF] Domanov | Canonical polyadic decomposition of third-order tensors: reduction to generalized eigenvalue decomposition[END_REF] with rank r ∈ N * if it reads as:

F (x 1 , x 2 , ..., x d ) = r i=1 u (1) i (x 1 )u (2) i (x 2 ) • • • u (d) i (x d ) (2.3)
for some functions u

(j) i ∈ L 2 (Ω j ) for 1 ≤ i ≤ r and 1 ≤ j ≤ d.
The CP decomposition factorizes a tensor into a sum of component rank-one tensors.

The rank of a tensor can be expressed as the minimum number of terms in an exact CP. A CP decomposition of a tensor F with r = rank(F ) number of components is called the rank decomposition.

More details about this decomposition and its uniqueness can be found in [START_REF] Karlsson | Parallel algorithms for tensor completion in the cp format[END_REF], [START_REF] Tamara | Tensor decompositions and applications[END_REF], [START_REF] Lu | Tensor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization[END_REF] and [START_REF] Stegeman | On kruskal's uniqueness condition for the candecomp/parafac decomposition[END_REF].

Given a certain tensor F , find the best rank approximation is equivalent to determine d rank-1 tensors that approximate well it. This can be written as a minimization problem: Find u

(1) i ∈ R N1 , . . . , u (d) i ∈ R N d , with i = 1, . . . , r, that minimize ∥F - r i=1 u (1) i (x 1 ) ⊗ u (2) i (x 2 ) ⊗ . . . ⊗ u (d) i (x d )∥
where ∥ • ∥ denotes some choice of norm on R N1×...×N d . When d = 2, the result of the minimization problem is given by the first r terms in the sum of its SVD decomposition, see [START_REF] Gene | Matrix computations[END_REF], for unitarily invariant norms. The SVD decomposition in two variables is then optimal. For higher order tensors this is a problem of central importance. It can be seen in [START_REF] De | Tensor rank and the ill-posedness of the best low-rank approximation problem[END_REF] and [START_REF] Christopher | Most tensor problems are np-hard[END_REF] that the problem is ill-posed for many r regardless the choice of norm. Moreover, if one randomly picks a tensor F in a suitable tensor space, there's a non-zero probability that F will fail to have a best rank-r approximation for some r < rank(F ). It is also known as the phenomenon underlying the concept of border rank, see [START_REF] Bini | Border rank of m× n×(mn-q) tensors[END_REF], [START_REF] Bini | Border rank of ap× q× 2 tensor and the optimal approximation of a pair of bilinear forms[END_REF], [START_REF] Landsberg | The border rank of the multiplication of 2× 2 matrices is seven[END_REF], and it is related to (but with different from) "CP degeneracy" ( [START_REF] Wim P Krijnen | On the non-existence of optimal solutions and the occurrence of "degeneracy" in the candecomp/parafac model[END_REF]).

The main advantage of the CP decomposition is the low memory cost needed to store it. Indeed, if N degrees of freedom are used per variable (N 1 = . . . = N d = N ), the storage cost of a general function

F ∈ L 2 (Ω 1 × • • • × Ω d ) is O N d .
On the other hand, the storage cost of a CP tensor with rank r reduces to O(dN r), which scales linearly in the tensor order d and size N . However, the problem of finding a best approximation of a tensor in CP format may be ill-posed [START_REF] De | Tensor rank and the ill-posedness of the best low-rank approximation problem[END_REF] and leads to numerical instabilities. The most classical algorithm in order to compute an approximation of a tensor in the CP format is the so-called Alternating Least Square (ALS) method.

Alternating Least Square (ALS) and Alternating Singular Value Decomposition (ASVD)

In general the determination of a tensor rank is an NP-hard problem. Previous works revealed us that there isn't any finite algorithm that determines the rank of a tensor. Because of that, the first problem that we face in computing a CP decomposition is how to choose the number of rank-one components. The CP decomposition with r components that best approximates F via the ALS method will be the one solving the minimization problem:

min F (∥F -F ∥)
where F is the rank-r approximation of F given by Equation 2.3. We introduce and detail the ALS algorithm in Algorithm 4 for a certain tensor namely W and a given accuracy ϵ. In order to explain the method, let us introduce some notation. In the following, W n-1 is the approximation of the tensor W obtained after n -1 iterations of the algorithm, and for all 1

≤ i ≤ d, R n i ∈ L 2 (Ω i ) and its components: R n,m i ∈ L 2 (Ω i ) .
As we can see in [START_REF] Uschmajew | Local convergence of the alternating least squares algorithm for canonical tensor approximation[END_REF], the ALS method for the computation of a CP approximation of a tensor can take many iterations to converge [START_REF] Beylkin | Algorithms for numerical analysis in high dimensions[END_REF], especially when the order of the tensor d is high. Moreover, it is not guaranteed to converge to a global minimum or even a stationary point, only to a solution where the objective function stops decreasing. The convergence properties of the ALS algorithm have been abundantly studied. We refer the reader for more details to the following series of works [START_REF] Uschmajew | Local convergence of the alternating least squares algorithm for canonical tensor approximation[END_REF][START_REF] Rohwedder | On local convergence of alternating schemes for optimization of convex problems in the tensor train format[END_REF][START_REF] Espig | On the convergence of alternating least squares optimisation in tensor format representations[END_REF][START_REF] Wang | On accelerating the regularized alternating least square algorithm for tensors[END_REF][START_REF] Ivan V Oseledets | Alternating least squares as moving subspace correction[END_REF]. The final solution can be heavily dependent on the starting guess as well. The reason for the popularity of this algorithm lies in the fact that it is very simple, conceptually and numerically, while still delivering astonishingly good results in many cases in low dimension. For higher ranks most of the difficulties with the global behavior of ALS seem to be intimately related to the fact that the approximation problem itself can be ill-posed. Some alternative methods [START_REF] Vandecappelle | Nonlinear least squares updating of the canonical polyadic decomposition[END_REF][START_REF] Rajih | Enhanced line search: A novel method to accelerate parafac[END_REF] have been proposed in order to obtain more efficient algorithms. The Alternating Singular Value Decomposition (ASVD) method is proposed in [START_REF] Friedland | On best rank one approximation of tensors[END_REF].

For the presentation of the ASVD algorithm, we need to introduce some additional notation. We denote by J := {{i, j}, 1 ≤ i < j ≤ d} the set of all possible pairs of indices between 1 and d. An ordering of the elements of J is chosen so that J = (J l ) 1≤l≤L with L := #J . We introduce the ASVD algorithm in Algorithm 5 for W and the tolerance ϵ.

Algorithm 4 ALS algorithm

1: Require:ϵ > 0, W ∈ L 2 (Ω) 2: Output: N ∈ N * , for all 1 ≤ n ≤ N and all 1 ≤ i ≤ d, R n i ∈ L 2 (Ωi) so that the CP tensor W ∈ L 2 (Ω) defined by W (x1, • • • , x d ) := N n=1 R n 1 (x1)R n 2 (x2) • • • R n d (x d ) ∀(x1, • • • , x d ) ∈ Ω, satisfies ∥W -W ∥ 2 L 2 (Ω) ≤ ϵ. 3: Set W 0 = W , n = 1. 4: while ∥W n-1 ∥ 2 L 2 (Ω) > ϵ do

5:

For all 1 ≤ i ≤ d, select randomly R n,0 i ∈ L 2 (Ωi) and set η := ϵ and m = 1.

6:

while η > 1 10 ϵ do 7:

for j = 1, • • • , d do 8: Compute R n,m j ∈ L 2 (Ωj ) solution to R n,m j ∈ argmin R j ∈L 2 (Ω j ) Wn-1 -R n,m 1 ⊗ • • • ⊗ R n,m j-1 ⊗ Rj ⊗ R n,m-1 j+1 ⊗ • • • ⊗ R n,m-1 d 2 L 2 (Ω)

9:

end for 10:

Compute η := R n,m 1 ⊗ • • • ⊗ R n,m d -R n,m-1 1 ⊗ • • • ⊗ R n,m-1 d 2 L 2 . Set m := m + 1.

11:

end while

12:

Define

R n i = R n,m-1 i for all 1 ≤ i ≤ d.
13:

Compute Wn(x1, • • • , x d ) = Wn-1(x1, • • • , x d ) -R n 1 (x1)R n 2 (x2) • • • R n d (x d ) for all (x1, • • • , x d ) ∈ Ω.
14:

n = n + 1
15: end while [START_REF] Kenneth | The use of the karhunen-loeve procedure for the calculation of linear eigenfunctions[END_REF]:

N = n -1

The Tensor Train (TT) decomposition

Let us now continue presenting other tensor format that is going to be present in this manuscript. It was introduced in [START_REF] Oseledets | Tensor-train decomposition[END_REF].

The function F is said to belong to the Tensor Train (TT) format with ranks r 1 , . . . , r d-1 ∈ N * if and only if

F (x 1 , x 2 , . . . , x d ) = r1 i1=1 . . . r d-1 i d-1 =1 u (1) i1 (x 1 )u (2) i1,i2 (x 2 )u (3) i2,i3 (x 3 ) . . . u (d-1) i d-2 ,i d-1 (x d-1 )u (d) i d-1 (x d ) with u (j) ij-1,ij ∈ L 2 (Ω j ) for 1 ≤ i j-1 ≤ r j-1 and 1 ≤ i j ≤ r j for all 1 ≤ j ≤ d (with r 0 = r d = 1).
It combines two main advantages: On the one hand, it is stable from an algorithmic point of view; on the other, it is computationally affordable provided that the TT ranks of the tensors used stay reasonably small.

One of the schemes used for the optimization of a TT tensor with a given rank is the ALS method, see details in [START_REF] Grasedyck | Stable als approximation in the tt-format for rank-adaptive tensor completion[END_REF]. In [START_REF] Holtz | The alternating linear scheme for tensor optimization in the tensor train format[END_REF] some numerical examples that concern the stability of the TT decomposition and of ALS are shown as well as how high TT ranks are required during the iterative approximation of low-rank tensors, showing some potential of improvement. These ranks are chosen a priori and that makes the method less robust. As we will recall in Algorithm 6, the most used way of computing the TT decomposition of a tensor is via the well-known TT-SVD algorithm. Let us make a special remark in this chapter to the TTr1 algorithm, proposed in [START_REF] Batselier | A constructive algorithm for decomposing a tensor into a finite sum of orthonormal rank-1 terms[END_REF]. As we will see, this is one of the closest alternatives to the proposed rank-1 particular case of the method.

TT-SVD algorithm

Let now choose and fix some W ∈ L 2 (Ω). We recall in Algorithm 6 the well-known TT-SVD algorithm for computing an approximation of the tensor W with prescribed accuracy ϵ > 0 in a TT format.

Algorithm 5 ASVD algorithm

1: Require:ϵ > 0, W ∈ L 2 (Ω) 2: Output: N ∈ N * , for all 1 ≤ n ≤ N and all 1 ≤ i ≤ d, R n i ∈ L 2 (Ωi) so that the CP tensor W ∈ L 2 (Ω) defined by W (x1, • • • , x d ) := N n=1 R n 1 (x1R n 2 (x2) • • • R n d (x d ) ∀(x1, • • • , x d ) ∈ Ω, satisfies ∥W -W ∥ 2 L 2 (Ω) ≤ ϵ.
3:

Set W 0 = W , n = 1. 4: while ∥W n-1 ∥ 2 L 2 (Ω) > ϵ do

5:

For all 1 ≤ i ≤ d, select randomly R n,0 i ∈ L 2 (Ωi) and set η := ϵ and m = 1.

6:

while η > 1 10 ϵ do 7:

Set R n,m i = R n,m-1 i for all 1 ≤ i ≤ d
8:

for l = 1, • • • , L do 9: Let 1 ≤ i l < j l ≤ d so that J l = (i l , j l ).
10:

Compute U n,m l ∈ L 2 (Ωi l × Ωj l ) solution to U n,m l ∈ argmin U l ∈L 2 (Ω i l ×Ω j l ) Wn-1 -U l ⊗ i∈{1,••• ,d}\J l R n,m i 2 L 2 (Ω)
.

11:

Compute R n,m i l , R n,m j l ∈ L 2 (Ωi l ) × L 2 (Ωj l ) solution to R n,m i l , R n,m j l ∈ argmin R i l ,R j l ∈L 2 (Ω i l )×L 2 (Ω j l ) ∥U n,m l -Ri l ⊗ Rj l ∥ 2 L 2 (Ω i l ×Ω j l
) .

12:

end for

13:

Compute

η := R n,m 1 ⊗ • • • ⊗ R n,m d -R n,m-1 1 ⊗ • • • ⊗ R n,m-1 d 2 L 2 .

14:

end while [START_REF] Sirovich | Turbulence and the dynamics of coherent structures. i. coherent structures[END_REF]:

Define R n i = R n,m i for all 1 ≤ i ≤ d.
16:

Compute Wn(x1, • • • , x d ) = Wn-1(x1, • • • , x d ) -R n 1 (x1)R n 2 (x2) • • • R n d (x d ) for all (x1, • • • , x d ) ∈ Ω.
17:

n = n + 1
18: end while 19: N = n -1

The Sum of Tensor Trains (SoTT) algorithm

The aim of this section is to present the Sum of Tensor Trains (SoTT) algorithm. We propose a method in order to greedily construct an approximation of a given tensor as a sum of Tensor Trains (TTs), where the order of the variables and the values of the ranks can be different from one term to another. The algorithm is presented in Section 2.3.1 and in a more detailed way in Algorithm 7. It is proved to converge exponentially fast in finite dimension in Section 2.3.2. Lastly, a discussion about the complexity of the method is given in Section 2.3.3.

Presentation of the SoTT algorithm

In the following, we denote S d the set of permutations of the set {1, • • • , d}.

Let us introduce a detailed scheme of the SoTT method in Algorithm 7. The different steps are explained in the following.

The aim of the SoTT algorithm is to compute, after n iterations, an approximation of a certain tensor W as a sum of n TTs. At iteration n, the SoTT computes an approximation of W under the form

W n-1 + R n 1 (x τ n (1) )R n 2 (x τ n (2) ) • • • R n d (x τ n (d) ),
where W n-1 is the approximation obtained after n -1 iterations of the algorithm, τ n ∈ S d denotes a well-chosen permutation of the variables, and for all 1

≤ j ≤ d, R n j ∈ L 2 Ω τ n (j) , R K n j-1 ×K n j
, where

K n 0 = K n d = 1.
The aim of the n th iteration is to choose the permutation τ n and the values of the ranks (K n j ) 1≤j≤d-1 in an appropriate way, which is done here using a greedy procedure.

Algorithm 6 TT-SVD algorithm

1: Require:ϵ > 0, W ∈ L 2 (Ω) 2: Output: K1, • • • , K d-1 ∈ N * TT-ranks, R1 ∈ L 2 Ω1, R 1×K 1 , R d ∈ L 2 Ω d , R K d-1 ×1 and for all i = 2, • • • , d -1, Ri ∈ L 2 Ωi, R K i-1 ×K i so that the Tensor Train W ∈ L 2 (Ω) defined by W (x1, • • • , x d ) := R1(x1)R2(x2) • • • R d (x d ) ∀(x1, • • • , x d ) ∈ Ω, satisfies ∥W -W ∥ 2 L 2 (Ω) ≤ ϵ 2 .
3:

Define K0 := 1, D0 = {1} and define W 0 ∈ L 2 (D0 × Ω) such that W 0(1, •) = W , I0 := {1, • • • , d}, Ω0 := Ω. 4: for j = 1, • • • , d -1 do 5: Since Dj-1 × Ωj-1 = (Dj-1 × Ωj ) × Ωj with Ωj = Ωj+1 × • • • × Ω d , compute the SVD decomposition of W j-1 according to the separation of variables (Dj-1 × Ωj , Ωj ) so that W j-1 = k∈N * σ j,k U j,k ⊗ V j,k .
6:

Select Kj ∈ N * such that Kj = inf K ∈ N * , k≥K |σ j,k | 2 ≤ ϵ 2 d -1 .

7:

Define Dj := {1, • • • , Kj } and define W j ∈ L 2 Dj × Ωj by

W j (kj , yj ) = σ j,k j V j,k j (yj )
for all (kj , yj ) ∈ Dj × Ωj .

8:

Define Rj ∈ L 2 Ωj , R K j-1 ×K j as Rj (xj ) = U j,k j (kj-1, xj ) 1 ≤ kj-1 ≤ Kj-1 1 ≤ kj ≤ Kj for all xj ∈ Ωj . 9: end for 10: Define R d ∈ L 2 Ω d , R K d-1 ×1 by R d (x d ) = σ d-1,k d-1 V d-1,k d-1 (x d ) 1≤k d-1 ≤K d-1 . Algorithm 7 SoTT algorithm 1: Require:ϵ > 0, W ∈ L 2 (Ω) 2: Output: N ∈ N * , for all 1 ≤ n ≤ N , τ n ∈ S d , K n 1 , • • • , K n d-1 ∈ N * TT-ranks, R n 1 ∈ L 2 Ω τ n (1) , R 1×K n 1 , R n d ∈ L 2 Ω τ n (d) , R K n d-1 ×1
and for all i = 2,

• • • , d -1, R n i ∈ L 2 Ω τ n (i) , R K n i-1 ×K n i so that the sum of Tensor Trains W ∈ L 2 (Ω)
defined by

W (x1, • • • , x d ) := N n=1 R n 1 (x τ n (1) )R n 2 (x τ n (2) ) • • • R n d (x τ n (d) ) ∀(x1, • • • , x d ) ∈ Ω, satisfies ∥W -W ∥ 2 L 2 (Ω) ≤ ϵ.
3:

Set W 0 = W , n = 1. 4: while ∥W n-1 ∥ 2 L 2 (Ω) > ϵ do 5: Define K n 0 := 1, D n 0 = {1} and define W n 0 ∈ L 2 (D0 × Ω) such that W n 0 (1, •) = W n-1 , I n 0 := {1, • • • , d}.
6:

for j = 1, • • • , d -1 do 7
:

For all i ∈ I n j-1 , since D n j-1 × i∈I n j-1 Ωi = D n j-1 × Ωi × i ′ ∈I n j-1 \{i} Ω i ′ , compute the SVD decomposition of W n j-1
according to the separation of variables (

D n j-1 × Ωi, i ′ ∈I n j-1 \{i ′ } Ω i ′ ) so that W n j-1 = k∈N * σ i,n j,k U i,n j,k ⊗ V i,n j,k .
8:

Select i n j ∈ I n j-1 and K n j ∈ N * so that i n j , K n j ∈ argmax i∈I n-1 ,r∈N * r k=1 σ i,n j,k 2 -β n i,j r,
where for all i ∈ I n j-1 , β n i,j > 0 is chosen according to (2.4).

9:

Define τ n (j) = i n j .

10:

Select K n j ∈ N * such that K n j = min K n j , inf K ∈ N * , k≥K σ τ n (j),n j,k 2 ≤ ϵ 2 d -1
.

11:

Define I n j := I n j-1 \ {τ n (j)} so that #I n j = d -j.

12:

Define D n j := 1, • • • , K n j and define W n j ∈ L 2 D n j × i∈I n j Ωi by W n j (kj , y τ n (j) ) = σ τ n (j),n j,k j V τ n (j),n j,k j (y τ n (j) )
for all (kj , y τ n (j) ) ∈ D n j × i∈I n j

Ωi.

13:

Define R n j ∈ L 2 Ω τn (j) , R K n j-1 ×K n j as R n j (x τ n (j) ) = U τn(j),n j,k j (kj-1, x τ n (j) ) 1≤k j ≤K n j ,1≤k j-1 ≤K n j-1
for all x τ n (j) ∈ Ω τ n (j) .

14:

end for 15:

Since #I n d-1 = 1, let i n d ∈ {1, • • • , n} such that I n d-1 = {i n d }. Define τ n (d) = i n d .
16:

Define R n d ∈ L 2 Ω τ n (d) , R K n d-1 ×1 by R n d (x τ n (d) ) = σ τ n (d-1),n d-1,k d-1 V τ n (d-1),n d-1,k d-1 (x τ n (d) ) 1≤k d-1 ≤K n d-1
. 17:

Compute W n (x1, • • • , x d ) = W n-1 (x1, • • • , x d ) -R n 1 (x τ n (1) )R n 2 (x τ n (2) ) • • • R n d (x τ n (d) ) for all (x1, • • • , x d ) ∈ Ω.
18:

n = n + 1
19: end while 20:

N = n -1
The idea behind the SoTT procedure is the following: the order of the variables is chosen so that it enables to obtain an interesting trade-off between accuracy and memory storage.

For instance, τ n (1) is chosen as follows. Let us denote by W n-1 := W -W n-1 the difference between the tensor and its approximation in the (n -1) th iteration and by W n 0 := W n-1 . The POD decomposition of W n 0 is computed with respect to all the unfoldings, i.e. the partitioning of the variables of the form Ω = Ω i × ( 1≤j̸ =i≤d Ω j ) for all i ∈ {1, • • • , d} = I n 0 . Denoting by σ i,n 1,k k∈N * the sequence of singular values associated to the i th partitioning of the variables, for all r ∈ N, let us define the functional

L n i,1 (r) = r k=1 σ i,n 1,k 2 -β n i,1 r
where β n i,1 is a positive real number. The function L n i,1 : N → R represents the trade-off between accuracy and memory storage on every iteration. It reads as the sum of two terms: on the one hand,

r n i,1 ∈ N solution to r n i,1 ∈ argmax r∈N L n i,1 (r)
is a value of rank which enables to obtain a reasonable trade-off between the accuracy of the truncated POD and its memory storage. Then, τ n (1) is chosen as the optimum index i ∈ I n 0 such that

τ n (1) = argmax i∈I n 0 L n i,1 (r n i,1 ) = argmax i∈I n 0 max r∈N L n i,1 (r),
and gives the index of the first variable in the TT computed at the n th iteration of the SoTT algorithm. A preliminary value of the rank

K n 1 is then chosen so that K n 1 = r n τn(1),1
. An additional step is used at line 9 for the definition of the final value of the rank K n 1 which ensures that if

k≥K n 1 σ τ n (1),n 1,k 2 ≤ ϵ 2 d -1 ,
then the final value of K n 1 is the lowest possible rank which guarantees that

k≥K n 1 σ τ n (1),n 1,k 2 ≤ ϵ 2 d -1 .
Now we need to select the values for the rest of the permutations τ n (2), • • • , τ n (d). In order to choose the complete order of the variables entering the definition of the n th TT term, one uses a similar iterative procedure applied to the d -1-order tensor W n 1 which reads as the projection of the tensor W n 0 onto the K n 1 first POD modes obtained from the τ n (1) th partitioning of variables.

We observe that the choice of the values of β n i,j > 0 at Step 8 of the SoTT algorithm is critical for its efficiency. In practice, in the case where for all 1 ≤ j ≤ d, #Ω j = N j < +∞ (i.e. when the tensor is defined on a discrete domain), we make the following choice:

β n i,j = N i + Π i ′ ∈I n j \{i} N i ′ Π i ′ ∈I n j N i ′ +∞ k=1 σ i,n j,k 2 = N i + Π i ′ ∈I n j \{i} N i ′ Π i ′ ∈I n j N i ′ ∥W n j-1 ∥ 2 ℓ 2 .
(2.4)

Introducing the function N ∋ r → L n i,j (r) := r k=1 σ i,n j,k 2 -β n i,j r, for 1 ≤ j ≤ d,
it holds that L n i,j (0) = 0, and there exists at least one

r n i,j ∈ 0, • • • , min N i , Π i ′ ∈I n j \{i} N i ′ so that r n i,j ∈ argmax r∈ 0,••• ,min Ni,Π i ′ ∈I n j \{i} N i ′ L n i,j (r),
and

L n i,j (r n i,j ) ≥ 0.

Exponential convergence of the SoTT algorithm in finite dimension

The aim of this section is to prove that the SoTT algorithm converges exponentially fast with the number of iterations in finite dimension.

Proposition 2.3.1 Let us assume that for all 1 ≤ j ≤ d, #Ω j < +∞. Then, there exists 0 < α < 1 such that for all n ∈ N * ,

∥W n ∥ 2 L 2 (Ω) ≤ α n ∥W ∥ 2 L 2 (Ω) , (2.5
)

with α ≤ 1 - 1 N ⌈d/2⌉(⌈d/2⌉+1) , where N := max 1≤i≤d #Ω i .
We observe in practice that the upper bound on the convergence rate of the SoTT algorithm given by Proposition 2.3.1 is pessimistic. We refer the reader to Section 2.5 for numerical results which illustrate this fact.

Proof 1 Let us begin by proving that for all

n ∈ N * , ∥W n ∥ 2 L 2 ≤ 1 - 1 N ⌈d/2⌉(⌈d/2⌉+1) W n-1 2 L 2 . (2.6)
Indeed, for all n ∈ N * , let us denote by

U n (x 1 , • • • , x d ) := R n 1 (x τ n (1) )R n 2 (x τ n (2) ) • • • R n d (x τ n (d) ), ∀(x 1 , • • • , x d ) ∈ Ω 1 × • • • × Ω d .
Note that, by construction and definition of the SoTT algorithm,

W n-1 -U n , U n L 2 (Ω) = 0, so that ∥W n ∥ 2 L 2 = W n-1 -U n 2 L 2 = W n-1 2 L 2 -∥U n ∥ 2 L 2 . (2.7)
By definition of the algorithm, it holds that for all 1 ≤ j ≤ d -1, K n j is lower than the minimum of the cardinality of D n j-1 × Ω i and the cardinality of i ′ ∈I n j-1 \{i ′ } Ω i ′ . Hence, we have

K n j ≤ min N K n j-1 , N d-j ,
where K n 0 = 1. Thus, by induction, we obtain that for all 1 ≤ j ≤ d -1,

K n j ≤ min N j , N d-j .
As a consequence, for all n ∈ N * , and all 1 ≤ j ≤ d -1, we obtain that for all i ∈ I n j-1 ,

#D n j-1 × Ω i ≤ N min N j-1 , N d+1-j = min N j , N d+2-j and # i ′ ∈I n j-1 \{i} Ω i ′ ≤ N d-j .
Thus, for all i ∈ I n j-1 ,

min #D n j-1 × Ω i , # i ′ ∈I n j-1 \{i} Ω i ′ ≤ min N j , N d-j . (2.8)
As a consequence, for all i ∈ I n j-1 , it holds that

σ i,n j,1 2 ≥ W n j-1 2 L 2 min (N j , N d-j ) .
Moreover, denoting by W n j :=

K n j kj =1 σ τ n (j),n j,kj U τ n (j),n j,kj ⊗ V τ n (j),n j,kj
, it holds that

W n j-1 -W n j 2 L 2 D n j-1 × i∈I n j-1 Ωi ≤ σ i,n j,1 2 .
Thus, using the fact that

W n j-1 -W n j is orthogonal to W n j , we obtain that W n j 2 L 2 D n j-1 × i∈I n j-1 Ωi = W n j-1 2 L 2 D n j-1 × i∈I n j-1 Ωi -W n j-1 -W n j 2 L 2 D n j-1 × i∈I n j-1 Ωi ≥ W n j-1 2 L 2 D n j-1 × i∈I n j-1 Ωi 1 - 1 min (N j , N d-j )
.

Lastly, using the fact that U

τ n (j),n j,kj 1≤kj ≤K j n is an orthonormal family of L 2 Ω τ n (j) and V τ n (j),n j,kj 1≤kj ≤K j n is an orthonormal family of L 2 i∈I n j Ω i , it holds that W n j 2 L 2 D n j-1 × i∈I n j-1 Ωi = W n j 2 L 2 D n j × i∈I n j Ωi = K n j kj =1 σ τ n (j),n j,kj 2 .
As a consequence, we obtain that for all n ∈ N * and for all 1

≤ j ≤ d -1, W n j 2 L 2 D n j × i∈I n j Ωi ≥ W n j-1 2 L 2 D n j-1 × i∈I n j-1

Ωi

.

In addition, it can easily be checked that

∥U n ∥ L 2 (Ω) 2 = W n d-1 2 L 2 (D n d-1 ×Ω τn(d) )
. Thus, by induction over

1 ≤ j ≤ d -1, we obtain that for all n ∈ N * , ∥U n ∥ L 2 (Ω) 2 ≥ W n 0 2 L 2 (Ω) 1 Π 1≤j≤d-1 min(N j , N d-j ) = W n-1 2 L 2 (Ω) 1 Π 1≤j≤d-1 min(N j , N d-j ) ≥ W n-1 2 L 2 (Ω) 1 N ⌈d/2⌉ j=1 j+ d-1 j=⌊d/2⌋ d-j ≥ W n-1 2 L 2 (Ω) 1 N 2 ⌈d/2⌉ j=1 j = W n-1 2 L 2 (Ω) 1 N ⌈d/2⌉(⌈d/2⌉+1) .
Collecting this estimate with (2.7), we obtain (2.6). Thus, by induction, we easily obtain the desired result (2.5).

Complexity estimate of the SoTT algorithm

The aim of this section is to provide some estimates on the complexity of the computational cost of the SoTT algorithm. Let us assume here that there exists N ∈ N * such that #Ω i ≤ N for all 1 ≤ i ≤ d.

We detail the computational cost of each iteration n ∈ N * . The computational cost is concentrated in the computation of the different POD decompositions of the tensor W n j-1 for each 1 ≤ j ≤ d -1 (l.7 of SoTT algorithm), which can be estimated using (2.2). In this process the most expensive part is the one in which we compute the SVD of the unfoldings. Exploting the computation presented in Appendix A.1.2 this cost can be reduced significantly. We consider two different cases.

Case 1: Unbounded ranks

Let us begin by giving a very pessimistic bound in the case where no upper bound on the ranks K n j is imposed for all 1 ≤ j ≤ d. From (2.8), it holds that the computational cost of each POD decomposition scales like

O max N j , N d-j min N j , N d-j 2 .
Thus, the total computational cost of the POD decompositions of one iteration of the SoTT algorithm is of the order of

O   d-1 j=1 (d -j + 1) max N j , N d-j min N j , N d-j 2   ≈ O d 2 N 3⌈d/2⌉ .

Case 2: Bounded ranks

Now, let us assume that there exists R ∈ N * such that R < N and such that for all 1 ≤ j ≤ d -1, K n j ≤ R. Then, for j = 1, the computational cost of each POD decomposition scales like

N d+1 .
Besides, for 2 ≤ j ≤ d -2, the computational cost of each POD decomposition scales like

R 2 N d-j+1 .
Lastly, for j = d -1, there is only one POD decomposition to compute, the cost of which scales like

RN 3 .
Thus, the total cost of the POD decompositions of one SoTT iteration scales like

O   dN d+1 + d-2 j=2 (d -j + 1)R 2 N d-j+1 + RN 3   ≈ O dN d+1 + dR 2 N d-1 + RN 3 .
Remark The computational cost of the SoTT algorithm is in general larger than the computational cost of the TT-SVD, as we do not fix a priori the order of the variables. In the first step of the SoTT iteration, the cost is similar to the one of the HOSVD method, in which we compute the POD for all the unfoldings. However, there is a difference in terms of the amount of memory needed to carry out the computations. In the practical implementation of SoTT, it is not necessary to store POD decompositions of all the unfoldings of the tensor, only a POD decomposition of the best one. In addition, we do not need to store the potentially dense core tensor. In the case in which the values of the ranks in SoTT are a priori fixed (for instance to 1, as will be the case in the forthcoming section), the computational cost of a POD decomposition can be reduced.

Storage

Let d be the dimension of the tensor. Let us suppose that the SoTT ranks K n j are equal for all the values of j, 1 ≤ j ≤ d, and for all the n terms of the SoTT sum. For simplicity in the notation, in the following we will denote them by K. The total storage of the SoTT tensor format is then

O(n × N × ((d -2)K 2 + 2K)).
Comparing this value with the storage of a tensor on its full format O(N d ), we obtain a compression rate of order:

O(n -1 × N d-1 × ((d -2)K 2 + 2K) -1
).

CP-TT: fixed-rank SoTT algorithm with rank 1

We make here a focus on a particular variant of the SoTT algorithm where all the ranks K n j are a priori chosen to be fixed and equal to 1 for all 1 ≤ j ≤ d and all iterations n ∈ N * . We refer the reader to [START_REF] Zhang | Rank-one approximation to high order tensors[END_REF] for a review on the stability properties of rank-1 tensor decompositions. As an output, the SoTT algorithm then computes an approximation of the tensor W in a CP format and we refer to the resulting procedure as the CP-TT algorithm. More precisely, for all 1 ≤ j ≤ d and n ∈ N * , Step 8 of the Algorithm 7 is replaced by the following step:

Select i n j ∈ I n j-1 so that i n j ∈ argmax i∈In-1 σ i,n j,1 2
and then Step 10 is not performed. As an output, after n iterations of the CP-TT algorithm, the method greedily produces an approximation of the tensor W under the CP format

W ≈ n k=1 R k 1 (x τ k (1) ) • • • R k d (x τ k (d) )
where for all 1 ≤ k ≤ n and all 1

≤ i ≤ d, R k i ∈ L 2 (Ω τ k (i) )
. Find in the following Algorithm 8 the detailed method.

We make a specific focus on this particular case because we numerically observed that this algorithm owns interesting stability and approximation properties in comparison to other more classical numerical methods like ALS or ASVD for the computation of a CP approximation of a tensor, especially when the order of the tensor d is high. For the sake of comparison, we recall the ALS and ASVD algorithm in Algorithm 4 and Algorithm 5 respectively. For more details and convergence properties of the ALS method, we refer the reader to the following series of works [START_REF] Uschmajew | Local convergence of the alternating least squares algorithm for canonical tensor approximation[END_REF][START_REF] Rohwedder | On local convergence of alternating schemes for optimization of convex problems in the tensor train format[END_REF][START_REF] Espig | On the convergence of alternating least squares optimisation in tensor format representations[END_REF][START_REF] Wang | On accelerating the regularized alternating least square algorithm for tensors[END_REF][START_REF] Ivan V Oseledets | Alternating least squares as moving subspace correction[END_REF]. The ASVD method is proposed in [START_REF] Friedland | On best rank one approximation of tensors[END_REF].

The closest method to CP-TT we found in the literature is the so called TTr1 algorithm, proposed in [START_REF] Batselier | A constructive algorithm for decomposing a tensor into a finite sum of orthonormal rank-1 terms[END_REF]. In this method, a TT-SVD decomposition with fixed ranks equal to one is computed at every stage of the algorithm. All the possible rank-1 terms are computed, stored and ordered according to the singular values of the corresponding POD decompositions. Favorable orthogonality properties of the method enable to truncate the obtained CP decomposition in order to fulfill a prescribed accuracy. The main differences of the TTr1 algorithm with respect to CP-TT are the following: first, the order of the variables is not fixed a priori in CP-TT, but is fixed beforehand in TTr1. Second, a greedy procedure is used in CP-TT computing one term at a time, whereas in TTr1 all the terms are computed at the same time before truncation. Hence, the computational cost per term is larger in CP-TT, but the storage need is much less significant, which seems beneficial for the compression of higher order tensors.

Numerical Experiments

In this section, several numerical experiments are proposed. In the first part, we compare several rank-1 update methods: ALS, ASVD, TTr1 and CP-TT, on the compression of a set of random functions belonging to certain classes of regularity. Then in the second part, we illustrate the behavior of the SoTT method with respect to the standard TT-SVD algorithm on several test cases which consist in compressing the solution of a parametric Partial Differential Equation (PDE), as well as other functions arising in different applications.

Comparison between CP-TT and other rank-one update methods

The aim of this section is to compare the efficiency of the CP-TT algorithm for the computation of approximations of a tensor in a CP format with other rank-1 methods, ALS, ASVD and TTr1.

Let (x 1 , . . . , x d ) ∈ Ω = [0, 1] d . Let (k 1 , ..., k d ) ∈ N d be
the wave numbers. The function to be compressed is assumed to be given in a Tucker format :

W (x 1 , . . . , x d ) = l1 k1=1 l2 k2=1 . . . l d k d =1 a k1...k d sin(πk 1 x 1 ) × . . . × sin(πk d x d ) (2.9)
First, the values of (l i ) 1≤i≤d ∈ N * are chosen to be a family of independent random integers uniformly distributed between 1 and 6.

Second, the coefficients (a k1...

1: Require:ϵ > 0, W ∈ L 2 (Ω) 2: Output: N ∈ N * , for all 1 ≤ n ≤ N , τ n ∈ S d and for all i = 1, • • • , d, R n i ∈ L 2 Ω τ n (i) , R 2 so that the sum of Tensor Trains W ∈ L 2 (Ω) defined by W (x1, • • • , x d ) := N n=1 R n 1 (x τ n (1) )R n 2 (x τ n (2) ) • • • R n d (x τ n (d) ) ∀(x1, • • • , x d ) ∈ Ω, satisfies ∥W -W ∥ 2 L 2 (Ω) ≤ ϵ.
3:

Set W 0 = W , n = 1. 4: while ∥W n-1 ∥ 2 L 2 (Ω) > ϵ do 5: Define K n 0 := 1, D n 0 = {1} and define W n 0 ∈ L 2 (D0 × Ω) such that W n 0 (1, •) = W n-1 , I n 0 := {1, • • • , d}.
6:

for j = 1, • • • , d -1 do 7
:

For all i ∈ I n j-1 , since D n j-1 × i∈I n j-1 Ωi = D n j-1 × Ωi × i ′ ∈I n j-1 \{i} Ω i ′ , compute the SVD decomposition of W n j-1
according to the separation of variables (

D n j-1 × Ωi, i ′ ∈I n j-1 \{i ′ } Ω i ′ ) so that W n j-1 = k∈N * σ i,n j,k U i,n j,k ⊗ V i,n j,k .
8:

Select i n j ∈ I n j-1 so that i n j ∈ argmax i∈I n-1 σ i,n j,1 2 ,

9:

Define τ n (j) = i n j .

10:

Define I n j := I n j-1 \ {τ n (j)} so that #I n j = d -j.
11:

Define D n j := 1, • • • , K n j and define W n j ∈ L 2 D n j × i∈I n j Ωi by W n j (kj , y τ n (j) ) = σ τ n (j),n j,k j V τ n (j),n j,k j (y τ n (j) )
for all (kj , y τ n (j) ) ∈ D n j × i∈I n j

Ωi.

12:

Define R n j ∈ L 2 Ω τn(j) , R 2 as R n j (x τ n (j) ) = U τn(j),n j,1 (x τ n (j) )
for all x τ n (j) ∈ Ω τ n (j) .

13:

end for 14:

Since #I n d-1 = 1, let i n d ∈ {1, • • • , n} such that I n d-1 = {i n d }. Define τ n (d) = i n d .
15:

Define R n d ∈ L 2 Ω τ n (d) , R 2 by R n d (x τ n (d) ) = σ τ n (d-1),n d-1,1 V τ n (d-1),n d-1,1 (x τ n (d) ) .
16:

Compute W n (x1, • • • , x d ) = W n-1 (x1, • • • , x d ) -R n 1 (x τ n (1) )R n 2 (x τ n (2) ) • • • R n d (x τ n (d) ) for all (x1, • • • , x d ) ∈ Ω.
17:

n = n + 1
18: end while [START_REF] Bui-Thanh | Model reduction for large-scale systems with high-dimensional parametric input space[END_REF]:

N = n -1 all 1 ≤ k 1 ≤ l 1 , ..., 1 ≤ k d ≤ l d , the value a k1...k d is then defined as: a k1...k d = α k1...k d ( k 2 1 + ... + k 2 d ) β .
For different random samples and different values of the coefficient β, we obtain different families of functions W given by (2.9). Let us remark that more detailed rank-r CP approximation of the orthogonal Tucker tensor could be seen in [START_REF] Khoromskij | Low rank tucker-type tensor approximation to classical potentials[END_REF], [START_REF] Boris | Multigrid accelerated tensor approximation of function related multidimensional arrays[END_REF].

We are testing how the four methods behave for the compression of 32 different functions generated by the random procedure described above for values of d ranging from 4 to 16. Let us point out that ALS and ASVD are both fixed point based methods, in contrast to CP-TT and TTr1, and the tolerance for the fixed point procedure has been set as η = 1.0 × 10 -4 . The maximum number of iterations of the method it max = 100. A uniform discretization grid of Ω with 25 degrees of freedom per direction is used for the discretization of W .

Description and analysis of Sobolev regularity in the d-torus

Let us establish the modes R 1 (x 1 ), . . . , R d (x d ) contained in the space L 2 (Ω). Let Ω : [0, 1] d be the domain. By simplicity of notation, let us name the product: R(x 1 , . . . ,

x d ) = R 1 (x 1 ) . . . R d (x d ).
We want to prove that the norm of the modes is finite:

∥R(x 1 , . . . , x d )∥ L 2 (Ω) < ∞.
To do so, we are going to work with the square of the norm. Writing the modes in the continuous space on the exponential representation of the sinus

Ω ∥R(x 1 , . . . , x d )∥ 2 dΩ = Ω a k1...k d e i2πk1x1 . . . e i2πk d x d 2 dΩ
Extending to a certain values of the wave number also the amplitude:

Ω ∥R(x 1 , . . . , x d )∥ 2 dΩ = Ω l1 k1=1 . . . l d k d =1 α k1,...,k d k 2 1 + . . . + k 2 d β e i2πk1x1 . . . e i2πk d x d 2 dΩ = = l1 k1=1 . . . l d k d =1 l1 m1=1 . . . l d m d =1 α k1,...,k d α m1,...,m d (k 2 1 + . . . + k 2 d )(m 2 1 + . . . + m 2 d ) β Ω e i2π(k1-m1)x1 . . . e i2π(k d -m d )x d dΩ = = l1 k1=1 . . . l d k d =1 α 2 k1,...,k d (k 2 1 + . . . + k 2 d ) β ≤ l1 k1=1 . . . l d k d =1 1 (k 2 1 + . . . + k 2 d ) β = l1 k1=1 . . . l d k d =1 1 d i=1 (k 2 i ) β < ∞
where we have used the orthogonality of the functions inside the integral. Solving this inequality in order to establish boundaries to the value of β for the functions to be regular in L 2 (Ω), we obtain a relation between the dimension and the parameter:

d -2β < 0 =⇒ β > d 2 .
If we want to extend the regularity of the functions to H 1 (Ω) the calculus is completely analogue. The relation obtained in this case:

β -1 > d 2 =⇒ β > 1 + d 2 .
In the view of this results, from the dimension of the problem we can select a value for the parameter β that will determine the regularity of the functions.

Results for functions with β = d

2 + 0.1 We begin by presenting here some numerical tests obtained with functions generated with β = d 2 + 0.1. We first present numerical experiments comparing the compression of the function given by Equation 2.9 computed by CP-TT, TTr1, ALS and ASVD in cases where d = 4 in Figure 2.1. Note that the memory required by the TTr1 method has prevented us from being able to carry out the method for higher values of d. Hence, for higher values of d, we only compare CP-TT with ALS and ASVD methods in Figure 2.2 for d = 12 and d = 16. The mean and standard deviation of the L 2 norm of the difference between the exact function W and its approximation given by any method is plotted as a function of the rank of the approximation. Let us make a remark: the negative values of the squared norm of the residual appear by subtracting the standard deviation to the mean of the results in Figure 2.1 and Figure 2.2.

Note that in the case d = 4 (Figure 2.1), ALS outperforms ASVD and CP-TT. The ALS also outperforms TTr1 for small values of the rank. However, in cases where d = 12 and d = 16 (Figure 2.2), CP-TT has a better numerical behavior when considering the decay of the norm of the residual with respect to the number of terms. In particular, the compression rate is better on average and the norm decay of the error with respect to the rank of the approximation is less subject to statistical noise. Table 2.1 summarizes the numerical results obtained with the CP-TT, ALS and ASVD methods. In particular, the mean and the standard deviation of the error (on the 32 random functions) are reported for ranks equal to 25, 50, 75 and tensor orders d = 4, 6, 8, 10, 12, 14, 16. We see here again that for low-order tensors (here when d = 4) ALS has better performances, whereas for higher order tensors CP-TT outperforms the other methods both in terms of mean and standard deviation. Mean and standard deviation of the L 2 norm of the difference between the exact function W and its approximation given by ALS (red), TTr1 (blue), CP-TT (black) and ASVD (green) as a function of the number of terms. See Table 2.1 for more detailed information. 2.1 for more detailed information. on H 1 (Ω) functions are equivalent to the ones shown for L 2 (Ω) functions, showing that the decrease in the error norm with the approximation rank is quite regular in CP-TT and behaves in a quite stable way also for higher order tensors. As a summary of the tests done we add Table A.1 as the counterpart of Table 2

.1 for β = d 2 + 1.1.

Comparison of the norm of the residual with respect to computational time

It is clear that one iteration of CP-TT is in general more costly in terms of computational time than one ALS iteration. As a consequence, even if the norm of the residual given by the CP-TT algorithm seems to decrease faster as a function of the number of terms in the approximation than with any other rank-1 update methods for high values of d, it is legitimate to compare the norm of the residual given by any method with respect to the computational time needed to compute the corresponding approximations. This is the aim of Figure 2.3, where the norm of the residual is plotted for CP-TT, ALS and ASVD as a function of the computational time. We observe in these tests that, in terms of mean of the decay of the norm of the residual as a function of the computational time, the three methods perform similarly in the sense that they are uniformly decreasing even if they present small jumps or flat intervals. However, we observe that CP-TT has a lower stochastic variability than ALS and ASVD. For functions in H 1 (Ω) the behavior is similar.

Mean

SoTT method

The goal of this section was to propose, on a simple synthetic test case, a comparison between rank-one update methods. We have also run the tests presented above by using the SoTT method. For the sake of brevity, we will synthetically comment the results in this section. For all dimensions, the SoTT method outperforms its rank-one particularization (CP-TT), as well as all the other rank-one update methods. Furthermore, it can reach a relative accuracy of 10 -3 by using 3-4 TT terms. This has two important consequences. In terms of memory, SoTT outperforms in general rank-one update methods. Concerning the computational cost: the cost per iteration of SoTT is larger than the cost of all the other methods. It is similar to the one of CP-TT (with some overheads due to the need of computing more singular vectors and values when performing the SVD, instead of just one). However, the large computational cost per iteration is counterbalanced by the fact that the method just needs three of four iterations to converge, as opposed to hundreds of iterations needed for rank-one update methods. These encouraging results motivate further investigation of the SoTT method. In the following section, we propose four different test-cases in which we compare the performances of SoTT with the TT-SVD method. Moreover, we will report the comparison between SoTT and two rank-one update methods, namely CP-TT and ALS, which is the rank-one update method which seems to perform the best for the functions of interest in the present work.

SoTT method for the compression of multivariate functions

In this section we present some numerical experiments to assess the performances of the SoTT method in compressing functions arising in different applications. Two main comparisons are shown: first, we will com- pare the SoTT method with the classical TT-SVD method, for different values of accuracy and by considering all the possible permutation of the indices (for TT-SVD); second, we will compare the performances (in terms of memory) of SoTT with its rank-one particularization, CP-TT, and the ALS method.

SoTT for the compression of the solution of a parametric reaction diffusion equation

The aim of this section is to illustrate the numerical behavior of the SoTT algorithm where the ranks are not fixed a priori but chosen according to Algorithm 7.

We consider here a fourth-order tensor obtained by solving numerically a 1D-1D parametric Fischer-Kolmogorov-Petrovsky-Piskunov (FKPP) equation. Let Ω 1 := [0, 1] be the space domain, and Ω 2 := [0, 0.25] be the time domain. Let α ∈ Ω 3 := [25, 100] be the reaction coefficient, and β ∈ Ω 4 := [0.25, 0.75] be a parameter defining the initial condition. The equation reads: for all (α, β)

∈ Ω 3 × Ω 4 , find u α,β : Ω 1 × Ω 2 ∋ (x, t) → u α,β (x, t) ∈ R solution to    ∂ t u α,β = ∂ 2 x u α,β + αu α,β (1 -u α,β ), ∀(x, t) ∈ Ω 1 × Ω 2 u α,β (0, t) = u α,β (1, t) = 0, ∀t ∈ Ω 2 , u α,β (x, 0) = exp(-200(x -β) 2 ), ∀x ∈ Ω 1 .
(2.10)

We then define, for all (

x 1 , x 2 , x 3 , x 4 ) ∈ Ω 1 × Ω 2 × Ω 3 × Ω 4 , W (x 1 , x 2 , x 3 , x 4 ) := u x3,x4 (x 1 , x 2 ).
ranks are [START_REF] Oseledets | Breaking the curse of dimensionality, or how to use svd in many dimensions[END_REF][START_REF] Oseledets | Breaking the curse of dimensionality, or how to use svd in many dimensions[END_REF]4], for the third [START_REF] Benner | Model reduction and approximation: theory and algorithms[END_REF][START_REF] Benner | Model reduction and approximation: theory and algorithms[END_REF][START_REF] Oseledets | Breaking the curse of dimensionality, or how to use svd in many dimensions[END_REF], and in general we observe that the order of the variables change.

In Figure 2.6, we compare the performance of SoTT with CP-TT, its particularization to rank-1 updates. More precisely, the logarithm of the memory is plotted as a function of the logarithm of the residual norm, for 5 iterations of SoTT and approximately 360 iterations of CP-TT and about 300 iterations of ALS. We observe that the performance of SoTT is better than the one of CP-TT and ALS.

SoTT for the compression of the parametric displacement of a cantilever beam

In this section we consider, as a 4-variate function, the displacement of the tip of a cantilever beam subjected to a uniform load. More details can be found in [START_REF] Krenk | Statics and mechanics of structures[END_REF]. Let L = 1 be the beam length, perfectly attached in x = 0, let E be the Young modulus, J x = J y = J be the inertia momentum of the beam (supposed to be symmetric), f y , f z the components along the y and z coordinates of a uniform in space load. Let u y (x), u z (x) : [0, 1] → R be the displacement fields along the y and z directions respectively. It holds:

u y (x) = f y 4EJ x 2 - f y 6EJ x 3 + f y 24EJ x 4 , (2.11) u z (x) = f z 4EJ x 2 - f z 6EJ x 3 + f z 24EJ x 4 .
(2.12)

The maximal displacement reads: u = u 2 y + u 2 z 1/2 , which equals:

u(E, J, f y , f z ) = f 2 y + f 2 z 1/2 8EJ . ( 2

.13)

For the present case we considered E ∈ [10 8 , 2

• 10 8 ], J ∈ [0.5, 1], f y , f z ∈ [0, 1] and N i = 32, 1 ≤ i ≤ 4.
In Fig. 2.7.a) we show a comparison, in terms of memory as function of the residual norm, between SoTT and the 24 possible TT-SVD, for 3 different levels of tolerance. Due to the function symmetries, the 24 TT-SVD group into 3 clusters of indices permutations. We observe that SoTT is suboptimal if compared to the best possible TT-SVDs, but it is better than the average TT-SVD. In terms of the comparison between SoTT and rank-one update methods, shown in Fig. 2.7.b), we observe, that SoTT is performing better than CP-TT and ALS, although the difference is not large for the function considered.

SoTT for the compression of the Friedman function

The Friedman function emulates the outcome of a computational process. 1 Let x ∈ [0, 1] 5 , the Friedman function f : [0, 1] 5 → R reads:

f (x) = 10 sin(πx 1 x 2 ) + 20(x 3 -0.5) 2 + 10x 4 + 5x 5 .
(2.14)

In terms of discretization, we consider N i = 20, 1 ≤ i ≤ 5. As for the other test cases, we compared the SoTT method with the TT-SVD for all 120 possible permutations of the indices, for 3 different values of tolerance, namely ε = 10 -2 , 10 -4 , 10 -6 . The results are shown in Figure 2.8.a). We observe that, due to the symmetries of the function, the 120 permutations clusters into 6 groups. SoTT is suboptimal with respect to the best TT-SVD, but it performs better than the average, requiring roughly half the memory than the worse TT-SVD. In Figure 2.8.b) we compare SoTT with its rank-1 version, CP-TT and the ALS method. As expected, SoTT outperforms both the rank-one update methods by roughly on order of magnitude in terms of storage for an accuracy of 10 -6 .

SoTT for the compression of the OTL-circuit function

In this section we consider a function relying the values of resistances and gain of a transformerless pushpull circuit (OTL) to its output2 . Let the variables be R [START_REF] Ladevèze | The latin multiscale computational method and the proper generalized decomposition[END_REF]300]. The circuit output V reads as follows:

1 ∈ [50, 150], R 2 ∈ [25, 70], R f ∈ [0.5, 3], R 3 ∈ [1.2, 2.5], R 4 ∈ [0.25, 1.2], β ∈ [
V 1 = 12R 2 R 1 + R 2 , (2.15) γ = β(R 4 + 9) + R f β(R 4 + 9) , (2.16) V (R 1 , R 2 , R f , R 3 , R 4 , β) = (V 1 + 0.74) γ + 11.35R f γβ(R 4 + 9) + 0.74R f γR 3 .
(2.17)

The results are shown in Fig. 2.9. On the left, we show the comparison, in terms of memory as function of the residual, between the TT-SVD performed by considering all the 720 permutations of the indices and the SoTT iterations. On the right, we show the results for the SoTT method compared to its rank one version, CP-TT, and the ALS method. The observed behavior is similar to the one observed and commented in previous test cases.

Computational cost

We summarize some observations on the computational cost, for all the test cases presented in this section. The computational cost per single iteration of SoTT is comparable to the one of CP-TT; both of them are in general larger than the computational cost of a single iteration of ALS, although the latter is featured by a large variance (due to the convergence of the fix point). The overall cost of SoTT to reach a given result in terms of accuracy is significantly smaller if compared to the cost of both ALS and CP-TT. This is due to the fact that, in order to achieve the target accuracy, we need to perform few iterations of SoTT (typically we computed not more than 5-6 SoTT terms on the test cases considered), contrary to ALS and CP-TT, which need a significantly larger amount of iterations. Compared to TT-SVD, the computational cost per iteration of SoTT is about d 2 times larger than the one of the TT-SVD. This computational burden could be compensated by the fact that, choosing a bad order of variables for TT-SVD could result in extra costs in terms of storage and further computational tasks.

Conclusions and perspectives

In this chapter, we proposed a method to compress a given tensor as a sum of Tensor Trains (SoTT). Neither the order of the variables nor the ranks are fixed a priori. Instead, they are the result of an optimization step. A particular instance of this method, consisting in fixing the ranks equal to one in all the steps of the algorithm, produces a CP approximation of a given tensor. A proof of convergence is proposed in the general case of the SoTT algorithm, which can be extended to the case of the CP-TT algorithm. Several numerical experiments are proposed to illustrate the properties of the methods. First, we compared the CP-TT to other rank-one update methods (ALS, ASVD, TTr1). Although a single iteration of CP-TT is more expensive in terms of number of operation, its stability makes it a promising candidate to compress high-dimensional tensors in CP format. We proposed some tests in which we compressed the numerical solution of a parametric partial differential equation of reaction-diffusion type as well as other functions coming from different applications. In particular, we compared SoTT with the TT-SVD obtained by testing all the possible permutations of the indices. Although SoTT is suboptimal with respect to the best TT-SVD, it is independent of the order of the variables and its performances are comparable to the average TT-SVD. In this test, the SoTT method outperforms CP-TT. Both methods showed preliminary yet encouraging results in view of applications in scientific computing and compression of high order tensors. The method presented shows some shortcomings, to be addressed in further investigations: while a greedy method is appealing in view of computational tasks in which fixing the rank a priori could be cumbersome, it might be featured by a saturation effect, slowing down its convergence.

Chapter 3

Local tensor methods

Introduction

In this chapter of the thesis, we propose a "local" HOSVD method to approximate tensors. Let

d ∈ N * , p 1 , • • • , p d ∈ N * and Ω 1 ⊂ R p1 , . . . , Ω d ⊂ R p d be open finite sets. Let Ω := Ω 1 × . . . × Ω d and F ∈ L 2 (Ω).
The tensor to be approximated is a multivariate function F (x 1 , . . . , x d ). This object is defined in all the domain, namely Ω := Ω 1 × . . . × Ω d . Let us introduce the domain partition.

Let N ∈ N * be the number of non-overlapping local domains Ω (i) . We have:

Ω = N i=1 Ω (i) .
To each subdomain Ω (i) , we can associate the characteristic functions I (i) (x 1 , . . . , x d ), which will be useful in what follows. These functions are one in the local subdomain and zero elsewhere. They read:

I (i) (x) = 1 if x ∈ Ω (i) 0 otherwise (3.1)
When multiplying a function by the characteristic function of a subdomain we can restrict its values to the local domain that we are considering.

55

Henceforward, a tensor defined in the global domain can be expressed as the sum of its values on N local domains as follows:

F (x 1 , . . . , x d ) = N i=1 F i (x 1 , . . . , x d ) = N i=1 F (x 1 , . . . , x d )I (i) (x 1 , . . . , x d ) (3.2)
where we have denoted F i (x 1 , . . . , x d ) the function F (x 1 , . . . , x d ) whose values are restricted to the specified subdomain Ω (i) .

Let F (x 1 , . . . , x d ) denote the approximation of F (x 1 , . . . , x d ). The goal is to provide a certified approximation such that ∥F -

F ∥ L 2 (Ω) ≤ ϵ
where ϵ > 0 is the prescribed accuracy. The approximation can be expressed by using the characteristic functions:

F (x 1 , . . . , x d ) = N i=1 F (x 1 , . . . , x d )I (i) (x 1 , . . . , x d ) = N i=1 Fi (x 1 , . . . , x d ) (3.3)
The local HOSVD method to approximate F consists in approximating F i in local domains and applying the HOSVD method on each one of the partitions. Then, the approximation reads, for 1 ≤ i ≤ N :

F i ≈ Fi = R1 i1=1 . . . R d i d =1 G i1...i d b (1) i1 × . . . × b (d) i d
in which R 1 , . . . , R d are the n-ranks, b are the modes and G is the tensor core. Their computation will be detailed later on.

The goal of this chapter is to investigate whether the local HOSVD will provide a better compression rate with respect to the HOSVD decomposition.

A crucial point is to determine the subdomain partition. In the proposed method, we don't impose the partition a priori, the aim is to come up with a method that computes automatically the partition.

Let us illustrate the partitioning on a 2-dimensional example in Figure 3

.1. As Ω = Ω 1 × Ω 2 the local partition reads Ω (i) = Ω (i) 1 × Ω (i) 2
, where 1 ≤ i ≤ N . In this case, let us say that N = 2, and let the subdomains originated by dividing the domain of x 1 be Ω

2 and Ω (2) 2 . For x 1 ∈ (0, 1) and x 2 ∈ (0, 1).

As we can see in the schematic Figure 3.1, the partition of the left is done in the axis x 1 and the partition of the right is done in the axis x 2 . One would like to guess which one of the partitions provides a better local approximation of the tensor.

Let us call N 1 ∈ N * the number of the partitions in x 1 and N 2 ∈ N * the partitions in x 2 . The total number of subdomains is N = N 1 × N 2 . In this case, N = 4. When partitioning at the same time on x 1 and x 2 we have a set of N local domains. This corresponds to the partition shown in Figure 3.2.

At this point the question of how to make the choice of the local domains arises. The main idea we investigate in this chapter is how to perform the subdivision per direction by using a clustering method. We would like to exploit the separability of the domain Ω = Ω 1 × . . . × Ω d in such a way that the partitions in the different directions provides a partition of the whole domain. The way of computing the local domains with a clustering algorithm is by applying it to the unfoldings of the different directions of the tensor. In this work we are going to restrict ourselves to the situation in which they are not overlapping. See [START_REF] Saxena | A review of clustering techniques and developments[END_REF][START_REF] Anil K Jain | Data clustering: a review[END_REF] for details of clustering methods.

Being N k the number of partitions of the domain in the k-th direction, with k = 1, . . . , d. Let J (i) k (x k ) be the characteristic functions in the direction k, J Ω 1

(i k ) k .For i k = 1, . . . , N k and i = (i 1 , . . . , i d ), they read: Ω (1) 1 Ω (2) 1 x 1 1 • • • • • • • • • 1 x 2 1 . . . . . . . . . 1 Ω (1) 2 Ω (2) 2 x 1 1 • • • • • • • • • 1 x 2 1 . . .
Ω 2 Ω 3 Ω 4 x 1 1 • • • • • • • • • 1 x 2 1 . . .
J (i k ) k (x k ) = 1 if x ∈ Ω (i k ) k 0 otherwise (3.4)
Where the domain Ω k is partitioned as:

Ω k = N k i k =1 Ω (i k ) k .
The method performs independently direction per direction. Let us recall that the characteristic functions that restrict the tensor of Equation 3.2 to the local domain read

I (i) (x 1 , . . . , x d ) = d k=1 J (i k ) k (x k ) (3.5)
The domain Ω is separable and then, the subdomain Ω (i) can be also separated

Ω (i) = Ω (i1) 1 × . . . × Ω (i d ) d .
Hence, the problem of partitioning the domain Ω can be reduced to the problem of partitioning Ω k . The partition of Ω k into local subdomains is done by a clustering method.

Clustering problem is not a trivial task, especially in the case of high-dimensional data that is exactly the case that we face in most applications, and it is where conventional methods usually fail [START_REF] Parsons | Subspace clustering for high dimensional data: a review[END_REF][START_REF] Hinneburg | Optimal grid-clustering: Towards breaking the curse of dimensionality in high-dimensional clustering[END_REF]. The number of data required to define correctly the system increases exponentially with the dimension of the system. This phenomenon is referred to as the already introduced curse of dimensionality.

Clustering is the process of grouping similar objects into different groups, or more precisely, the partitioning of a data set into subsets. This can be achieved by grouping entities in terms of a similarity (or dissimilarity) measure. Here, the critical issue is to understand what is meant by "similar". In this work, we consider similarity as the inverse of a distance metric between two entities. The shorter the distance, the more similar the entities, and vice versa. Clustering results will be crucially dependent on the similarity notion chosen. In this work, we make the choice of using a L 2 distance, and when discretizing a l 2 distance.

The clustering method proposed in this chapter is agglomerative. As mentioned, it works independently for each one of the directions of the tensor. Let us recall that the n-mode vectors (or just modes), in this work also called fibers, of a certain unfolding F (k) are its column vectors, when k ∈ {1, . . . , d}. In the new partitioning of Ω k , we will apply the clustering to the fibers of the unfolding transposed. Let us now introduce some notation which will be used in all the sequel. Let x k and y k = (x 1 , . . . , x j , . . . , x d ) j̸ =k . The set of fibers of F (k)T reads as:

{f k p (y k )} 1≤p≤N k .
If we face a d-dimensional tensor, F ∈ R N1×...×N d and we compute the unfolding transposed corresponding to the first one of the directions F (1) ∈ R (N2×...×N d )×N1 , the system presents a set of N 1 mode vectors:

f 1 1 (y 1 ), . . . , f 1 n1 (y 1 ) of size N 2 × . . . × N d .
Then, for the k-th one of the unfoldings of the tensor, k ∈ {1, . . . , d}, the N k fiber modes read as:

f k 1 (y k ), . . . , f k N k (y k ), of size N yk = d k̸ =j N j .
This clustering method will allow us to subdivide the fibers of a tensor in different groups (clusters). The intuition suggest that in a physical context, we are grouping the fibers that belong to the same "regime".

Remark:

In this case, we are going to work from the unfoldings of the tensor transposed, F (k)T . If we want to work with the unfoldings directly, let us remark that the partitions that we are computing are done not in the direction k but in the rest of the directions 1 ≤ j ̸ = k ≤ d. The subdomains in which we divide the multivariate functions reads as Ω

(i) j̸ =k 1≤j≤d instead of Ω (i) k 1≤k≤d
. It is indeed, an equivalent way of proceeding. In this work, we have chosen to work with the transpose of the unfoldings.

The output of the clustering method is a hierarchical tree of the possible partitions of the domains Ω (i) k 1≤k≤d

. It starts pairing the similar leaves and it carries on pairing the nodes till everything is fused.

Each one of the levels l k of the tree denotes a possible partition of the domain on the direction k. In the root of the tree, the point in which every cluster is fused, the POD decomposition of the unfolding transposed in the direction k is recovered. Each of the nodes defines a partition of the domain. The closest the nodes are to the root of the tree, the biggest the clusters are.

Let #l k be the number of levels of the tree T I k on the direction k. Each level l k denotes a possible partition of the domain Ω k on that direction. The total number of possible combinations of partitions in all the directions reads as d k=1 #l k . On each level l k , where l k = 0, . . . , #l k -1, of the tree, the number of subdomains in which the domain Ω k is divided is denoted by N k,l k .

In this work, in order to find the best partition we perform an extensive search between the possible partition combinations and we perform the HOSVD on each of them. In order to determine which partition of the domain is the best we need to establish a criterion in order to select one partition over the others. This is the memory needed to store the approximation. For every possible combination of the partitions of the domain obtained, the method computes the basis that generates the subspace defined by the partition performing HOSVD on the truncated space. The one that requires less amount of memory to be stored is said to be the best of the partitions of the domain.

Clustering methods that focus on finding a clustering strategy that is appropriate for model order reduction purposes have been already used and can be found in [START_REF] Daniel | Physics-informed cluster analysis and a priori efficiency criterion for the construction of local reduced-order bases[END_REF][START_REF] Daniel | Optimal piecewise linear data compression for solutions of parametrized partial differential equations[END_REF], where a dictionary of clusters is assembled and when a new instance of the data needs to be classified, the method finds the cluster to which it belongs. In [START_REF] Amsallem | Nonlinear model order reduction based on local reduced-order bases[END_REF][START_REF] Washabaugh | Nonlinear model reduction for cfd problems using local reduced-order bases[END_REF] a multiple local bases method is introduced. The main difference with the method proposed in this work is that in [START_REF] Amsallem | Nonlinear model order reduction based on local reduced-order bases[END_REF], the local ROB are computed by the method of snapshots in the intersection between overlapping clusters.

Organization of the chapter

After the presented overview of the method proposed in this chapter, it will be studied in detail in the following sections. The organization followed in the chapter reads as follows:

Find in Section 3.2 some notation and preliminary concepts that are used in the present chapter. In Section 3.3 the process of construction of the local domains is detailed. In Section 3.4, the computation of the approximation of the tensor in the local subspaces obtained is computed. It is in this section where the local HOSVD method is presented. Section 3.5 breaks down the cost and complexity of the proposed algorithm. A summary recalling the important steps of the method can be found in Section 3.6. To finish, the numerical results of the method applied to practical cases appear in Section 3.7.

Notation and preliminaries

The notation here recalls the one used in Section 2.2. Let us look back on some of the concepts introduced before and particularize them to its concrete use on this section. New notation and concepts are introduced as well.

Let us begin by recalling some definitions. Let us start by the definition of a tensor as a multivariate function and its associated preliminary concepts. Let p be the dimension and let D be a subset of R p with a product structure

D := D 1 × . . . × D d . Let us consider d ∈ N * . Then, for all 1 ≤ i ≤ d, D i is
an open bounded subset of R pi for some p i ∈ N * . We call a tensor any real-valued multivariate function

F ∈ L 2 (D 1 × • • • × D d ).
For each 1 ≤ i ≤ d, a certain H i a Hilbert space of univariate functions defined of D i , equipped with the corresponding inner product (•, •) Hi and the associated norm ∥ • ∥ Hi .

The space L 2 (D) of square-integrable, real-valued functions defined on D equipped with a scalar product and the corresponding canonical norm

∥ • ∥ L 2 (D) that is defined for F, G ∈ L 2 (D) by ⟨F, G⟩ D := D F (x)G(x)dx and ∥F ∥ L 2 (D) := D F 2 (x)dx 1/2 .
From now on, in order to simplify the notation of the manuscript (every time that there's no ambiguity) L 2 (D) will be denoted as D, equipped with the norm ∥ • ∥ D and its associated inner product (•, •) D . In the cases in which we work with discrete objects in l 2 (D), the norm used will be the Euclidean norm and will be denoted by

∥ • ∥ l 2 .
We recall here some well-known definitions and introduce some notation about tensor fibers and tensor unfoldings:

The so called fibers are the higher-order analogue of matrix rows and columns, defined by fixing every index of a tensor but one. In a 3-dimensional tensor F ∈ R N1×N2×N3 , for a certain value of two of its indices i 2 ∈ {1, . . . , N 2 } and i 3 ∈ {1, . . . , N 3 }, a fiber on the i 1 -th direction is given by f i1 = F :i2i3 , with i 1 ∈ {1, . . . , N 1 }. When fibers are extracted from tensors, they are assumed to be column vectors.

Unfoldings. Assume a N -th order tensor X ∈ R N1×...×N N . The matrix unfolding X (n) ∈ R Nn×(Nn+1...N N N1...Nn-1) contains the element x i1...ip at the row position i n and column number equal to:

(i n+1 -1)N n+2 . . . N N N 1 . . . N n-1 + i n+2 -1)N n+3 . . . N N N 1 . . . N n-1 + . . . . . . + (i N -1)N 1 . . . N n-1 + (i 1 -1)N 2 . . . N n-1 + . . . + i n-1
The unfolding transpose of a N -th order tensor X ∈ R N1...N d tensor is the matrix X (n)T ∈ R (Nn+1...N N N1...Nn-1)×Nn that contains the element x i1...ip at the column position i n and row number equal to:

(i n+1 -1)N n+2 . . . N N N 1 . . . N n-1 + i n+2 -1)N n+3 . . . N N N 1 . . . N n-1 + . . . . . . + (i N -1)N 1 . . . N n-1 + (i 1 -1)N 2 . . . N n-1 + . . . + i n-1
As a remark, notice that the fibers are the column vectors of the unfolding matrices.

Given two fibers,

f i1 1 ∈ R N1 and f (i1) 2 ∈ R N1 , for i 1 ∈ {1, . . . , N 1 }, let us define the concatenation of them as a matrix F ∈ R N1×2 : F = (f i1 1 f i1 2 ) (3.6)
such that its first column F :,1 , is the fiber f i1 1 and its second column

F :,2 , is f i1 2 .
Following this definition, we extend it to a concatenation of matrices. Let F ∈ R N1×N2 and X ∈ R N1×N3 . The concatenation of them results other matrix A ∈ R N1×(N2+N3) reads:

A = (F X)
such that: for 1 ≤ i ≤ N 2 , A :,i = F :,i and for N 2 ≤ j ≤ N 4 , A :,j = X :,j-N2 .

Partitioning of tensors

In this chapter, the notation followed in order to explain the concepts related with local tensor decomposition is the one introduced in [START_REF] Ehrlacher | Adaptive hierarchical subtensor partitioning for tensor compression[END_REF].

Let d ∈ N * be the dimension of the system and p

∈ N * . Let D 1 , . . . , D p be open subsets of R di with d i ∈ N * and 1 ≤ i ≤ p. We denote D := D 1 × . . . × D p .
A domain partition {D (k) } 1≤k≤K of D satisfies:

• For all 1 ≤ k ≤ K, there exists

D (k) 1 ⊂ D 1 , . . . , D (k) d ⊂ D d open subsets such that D (k) := D (k) 1 × . . . × D (k) d . • For all 1 ≤ k ̸ = l ≤ K, D (k) ∩ D (l) = ∅. • D = K k=1 D (k) .
A domain partition of D for 1 ≤ j ≤ d can be: a set of subdomains P j = D (kj ) j ∈ D j , with 1 ≤ k j ≤ K j , that satisfies the properties enhanced before, such that the partition reads

P = × d j=1 D (kj ) j 1≤k1≤K1,...,1≤k d ≤K d
In [START_REF] Ehrlacher | Adaptive hierarchical subtensor partitioning for tensor compression[END_REF] we can also see that under appropriate assumptions, it can be proved that there exists an admissible domain partition D (k) ⊂ D such that the restricted tensor F (x k 1 , . . . , x k d ), with x k j the restricted values of the variable x j , can be represented in some tensor formats with low ranks.

Partition trees

Let T I be a tree with nodes V(T I ) and edges E(T I ). For all J ∈ V(T I ) we denote the set of sons of the node J as:

S J (T I ) := {J ′ ∈ V(T I ), (J, J ′ ) ∈ E(T I )}
and equivalently we can define the set of sons of J of the k-th generation, S k J (T I ) and the nodes in the k-th generation as V k (T I ) . The set of leaves reads,

L(T I ) := {J ∈ V(T I ), S J (T I ) = ∅}
The set of parents of leaves of a tree T I is defined as L p (T I ) ⊂ V(T I ) which have at least one son that is a leaf of T I .

With all this concepts set we are in position to recall the definition of partition tree. A tree T I is called a partition tree if:

• I is the root of T I .

• For all J ∈ V(T I )\L(T I ), S J (T I ) is an admissible partition and |S J (T I )| ≥ 2.

• For all J ∈ V(T I ), J ̸ = ∅.

In this work we are interested in the particular case in which T I is a dyadic partition tree. In this case, for all J ∈ V(T I )\L(T I ), S J (T I ) is an admissible partition and |S J (T I )| = 2.

The High Order Singular Value (HOSVD) decomposition

Let us recall from Equation 1.8 the already introduced HOSVD format altogether with the concept of n-rank of a tensor. It is the generalization of the SVD decomposition presented in Equation 2.1.

Let us refer to the column, row, ... vectors of a N -th order tensor as its n-mode vectors defined as the N n -vectors obtained by varying the index i n and keeping the rest fixed. The n-mode vectors (or just modes) of a certain tensor X are the column vectors of the unfolding X (n) . We define the n-rank of a tensor X, denoted by R n = rank n (X) as the dimension of the vector space spanned by the n-mode vectors. The HOSVD of a certain tensor F reads:

F = R1 i1=1 . . . R d i d =1 G i1...i d u (1) i1 × . . . × u (d) i d (3.7)
in which R 1 , . . . , R d are the n-ranks defined above and they are smaller than N 1 , . . . , N d , respectively. For compression, it is also commonly used the Truncated HOSVD, in which the summation is truncated when the approximation reaches a certain tolerance. The representation of a tensor in this format requires to store N d + N dR elements, supposing that all the ranks R 1 = . . . = R d = R are equal.

Local tensor spaces

The objective is to propose a partition of the domain in which the function is well approximated by HOSVD method. This is done automatically by the method and it doesn't require the choice of the ranks of the approximation a priori.

From now on, we fix some d ∈ N * and for all 1 ≤ j ≤ d, let Ω j be an open subset of R pj for some p j ∈ N * . We define

Ω := Ω 1 × • • • × Ω d .

Construction of local subdomains

Recalling Equation 3.2, a multivariate function F (x 1 , . . . , x d ) can be expressed as a sum on N local domains:

F (x 1 , . . . , x d ) = N i=1 F i (x 1 , . . . , x d ) = N i=1 F (x 1 , . . . , x d )I (i) (x 1 , . . . , x d )
where, for N number of local subdomains and i = 1, . . . , N , F i (x 1 , . . . , x d ) is the function whose values are restricted to the specified subdomain defined by the characteristic functions defined in Equation 3.1.

In order to exploit that Ω is a separable domain, the clustering method is applied independently to each one of the unfoldings of the tensor F (1) , . . . , F (d) . On each direction, a partition tree is computed in an agglomerative way. It will provide the possible partitions of the domain that will be considered. Let N k,l k be the number of subdomains. The approximation of the unfolding transposed reads:

F (k)T (x k , y k ) = N k,l k i k =1 F (k)T i k (x k , y k )I (i k ) k (x k )
where the characteristic functions per direction are defined in Equation 3.4.

As introduced before, in this part of the work for u and v given fibers of the unfolding transposed, we set the distance between them as dist(u, v) = ∥u -v∥ L 2 ( j̸ =k Ωj ) , being j̸ =k Ω j with j = 1, . . . , d, j ̸ = k the product of the spaces different from Ω k , when we are dealing with continuous functions and dist(u, v) = ∥u -v∥ l 2 , where l 2 is on R N yk , when we are working with discrete vectors. Let us recall that we have denoted

N yk = d j̸ =k N j .
The method introduced in this work proceeds by computing an agglomerative tree. Let us start introducing how to compute the leaves. They are clusters of fibers represented by 2-dimensional bases. This grouping process relies on the notion of distance between the fibers on the different unfoldings of the tensor.

Computation of the leaves

Let us now explain how the method proceeds for one of the directions of the system. The extension to others is identical. As we have introduced before, for a certain direction k of the tensor we have a set of fibers depending of the rest of the variables,

f k 1 (y k ), . . . , f k N k (y k ) of size N yk .
Let us define a set of indices per direction T k = {1, . . . , N k } that denotes the index of the fibers. At the beginning of the method, it denotes the whole set and at the end an empty set T k = ∅.

Each one of the fibers is then a vector of N yk components. Choosing randomly one of the fibers, let us say without loss of generality the first of them f k 1 (y k ), we take its index out of the set, T k is updated by T k \ {1}, and we compute its nearest neighbor.

q * = arg min q∈T k dist(f k 1 , f k q ) where dist(f k 1 (y k ), f k q * (y k )) = ∥f k 1 -f k q * ∥ l 2
This pairing of two fibers is the first step in the grouping process. A SVD of the matrix composed by the two fibers as its columns will allow us to compute the basis.

Let us suppose that q * is such that f k q * (y k ) is linearly independent from f k 1 (y k ). Then, let us define the matrix F k(1,q * ) ∈ R N yk ×2 as introduced in 3.6:

F k(1,q * ) = (f k 1 (y k )f k q * (y k )).
Recalling the previously defined SVD decomposition, it holds that there exists an orthonormal basis (U i ) i∈N * ∈ R N yk ×N yk , an orthonormal basis (V i ) i∈N * ∈ R 2×2 , and a decreasing sequence (σ i ) i∈N * ∈ R min(N yk ,2)×min(N yk ,2) of non-negative real numbers, such that

F k(1,q * ) = U k(1,q * ) Σ k(1,q * ) V k(1,q * )T = i∈N * σ k(1,q * ) i U k(1,q * ) i ⊗ V k(1,q * ) i . (3.8)
If the fibers f k 1 and f k q * are linearly dependent, one of the singular values is going to be zero and a one-dimensional basis will be retrieved. If that occurs, we look for the next neighbor to perform this step.

The orthonormal basis of the joint subspace is given by the matrix U k (1,q * ) . From now on, let us call it B [START_REF] Cichocki | Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[END_REF] k , where the superindex of the basis is the first selected fiber. Let us define now a certain set of indices I

(1) k = {1, q * } that is composed by the indices of the fibers in a certain cluster. The superindex corresponds to the one used for the basis. Every time that one of the fibers is added to a cluster, we take its index out of T k in order to avoid repetition, and T k is updated by

T k \ I (j) k , being I (j) k
the set of indices of the fibers in the j-th cluster in the k-th direction that is being used to construct a cluster.

If some neighbors with zero distance had been found when constructing the 2-dimensional basis process, its indices need to be included also in the set because it is obvious that they are well represented by the basis. An example of this situations is the case in which the solution of a PDE is periodic and some of the fibers are consecutively repeated. If they are linearly dependent, the index of the neighbor will be also added to the set.

In the clusters, not only the fibers that we used to compute the basis or the ones that are linearly dependent (and that belong to the subspace) can be included, but also the ones that are close enough to the subspace. In this case, a fiber is considered close enough to the subspace if it can be represented on it with an error smaller than the one needed to guarantee the prescribed accuracy.

The error for a certain fiber f k p (y k ) that is approximated in the space spanned by a basis B (h) k is defined as the difference between the projection of the fiber and the fiber itself:

∥f k p (y k ) - j̸ =k Ωj B (h)T k f k p (y k )dy k ∥ L 2 = ϵ (h) k (3.9)
Let N k be the number of fibers in the k-th direction. In order to consider a certain fiber of the direction k, f k p (y k ), where p / ∈ I

k , well represented by the basis B

(1)

k , the error defined in Equation 3.9 needs to be smaller than the squared tolerance ε 2 (prescribed) over the number of fibers (δ

(1) k ) 2 = ε 2 /N k .
Let us define the error made in the cluster q of the direction k, namely ϵ (q) k,l , as the sum of the errors made each time that a fiber is added to it:

h∈#I (q) k ϵ (h) k = ϵ (q)
k,l . In order to obtain a guaranteed approximation, we need that ϵ

(h) k < δ (1)
k . At the beginning, this error is zero because the cluster consists only in the two vectors themselves, the fibers with indices {1, q * } in the previous example. When we add fibers to the cluster, they are represented by the basis with an error ϵ [START_REF] Cichocki | Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[END_REF] k . This error is such that it guarantees that the global error made in all the clusters respects the prescribed tolerance.

When a certain fiber is included in a cluster, we modify its basis in order to reduce the approximation error in the cluster. When just the two neighbor fibers were considered, the error was zero but when we include other fiber in the cluster it may be represented for the basis of the cluster with a certain error given by Equation 3.9. With the objective of minimizing the error in the cluster, an update of the basis is computed. In [START_REF] Brand | Fast low-rank modifications of the thin singular value decomposition[END_REF][START_REF] Brand | Fast online svd revisions for lightweight recommender systems[END_REF] the author develops an identity for additive modifications of a singular value decomposition (SVD) to reflect updates, downdates, shifts, and edits of the data matrix. This situation has our particular interest because it allows us to compute the new SVD decomposition by adapting the old one instead of recomputing the whole SVD. In our concrete case, we are interested on the SVD of the matrix after a rank-1 modification. Let a real matrix F ∈ R Ni×p , with N i , degrees of freedom in the direction i and p ∈ N * , have rank R and SVD F = U ΣV T with Σ ∈ R R×R the diagonal matrix of singular values. Let A ∈ R Ni×c , B ∈ R p×c be arbitrary matrices of rank c. We are interested in the SVD of the real matrix F ∈ R Ni×(p+1) resulting from the sum:

F = F + AB T = U A Σ 0 0 I V B T (3. 10
)
Where I is the identity matrix. In our case the application of this method for the joint function F k(1,q * ,p) as a rank-1 modification of F k(1,q * ) reads:

F k(1,q,p) = F k(1,q) + f k p b T = B (1) k f k p Σ k(1,q * ) 0 0 I V k(1,q * ) b T (3.11)
where b T = [0, 0, . . . , 1] is the rank-1 value of B and A on its particular rank-1 case is chosen as f k p . Find more details of the choice of the values for A and B and of the method itself in the cited work, [START_REF] Brand | Fast low-rank modifications of the thin singular value decomposition[END_REF]. In our case, the new basis of the cluster with the new fiber included obtained by the SVD of the expression 3.11 will update the old B [START_REF] Cichocki | Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[END_REF] k . The error budget, that at the beginning of the method was equal to the tolerance squared, b k = ε 2 , will be also updated because making some error in the projection means spending part of the error budget. It becomes now b

k = b k -(ϵ (1) k ) 2 . The quantity (δ (1)
k ) 2 used to determine if a fiber is well represented or not by a certain basis, is updated as: (δ

(1) k ) 2 = b k #T k .
This process is repeated till there are no fibers left to be checked and T k = {0}. We have obtained a set of 2-dimensional basis that well represents all the fibers in a certain direction.

It could happen that at the end of the clustering process, a fiber is not well represented by any of the existing bases. The fiber is then the generator of its own 1-dimensional space.

Let us introduce N k,0 as the number of clusters that we obtain as the leaves for the tree and let us remark that 1 ≤ N k,0 ≤ ⌈ N k 2 ⌉. The case in which, N k,0 = ⌈ N k 2 ⌉ is the one in which all the fibers of the direction k have been paired two by two. The other endpoint, N k,#l k -1 = 1 represents the case in which the first basis originated by two fibers represents well the rest of the fibers on the set. Let us name N k the set of all the partitions of the domain Ω

k , N k = #l k -1 j=0 N k,j .
A necessary condition to guarantee that we are obtaining a certified approximation is that sum of all the errors squared is smaller or equal that the prescribed tolerance:

N k,0 q=1 (ϵ (q) k,l ) 2 ≤ ε 2 .
The error satisfies also that

N k,0 q=1 (ϵ (q) k,l ) 2 + b k = ε 2 .
Where b k denotes the part of the error squared that hasn't been made in the clustering process, or what is the same, the difference between the prescribed tolerance and the errors made on each cluster.

The first part of the method, developed till here, can be found more detailed in Algorithm 9.

Algorithm 9 Computation of the leaves algorithm

1: Require: Unfolding of a tensor in the direction k, F (k)T (x1, . . . , x d ) ∈ R d j=1,j̸ =k N j ×N k , for 1 ≤ k ≤ d. Tolerance ε. 2: Output: Set of indices {I (q) k } 1≤q≤N k,0 . Set of bases {B (q) k } 1≤q≤N k,0 . Set of local error budgets {ϵ (q) k,b } 1≤q≤N k,0 and an error budget b k . 3: Compute the set of N k fibers f k 1 (y k ), . . . , f k N k (y k ) 4: Initalize n = 1 5: Initialize T (1) k = {1, . . . , N k } 6: Initialize i = T (1) k,1 7: Initialize the error budget: b k = ε 2 8: while T (i) k ̸ = {0} do 9:
for n > 1 do 10: 111: end for [START_REF] Haasdonk | Reduced basis method for finite volume approximations of parametrized linear evolution equations[END_REF]:

Set i = T (i) k,
for q ∈ T (i) k do 13:
Compute the distance between the fibers and select the closest one:

q * = arg min q∈T (i) k ,q̸ =i dist(f k i (y k ), f k q (y k )), where dist(f k i (y k ), f k q (y k )) = ∥f k i,j -f k q,j ∥ l 2

14:

end for

15:

Set

I (i) k = {i, q * } 16: Update T (i) k = T (i) k -{q * }

17:

Compute the joint matrix

F k(i,q * ) = (f k i (y k )f k q * (y k ))
18:

SVD of F k(i,q * ) = j∈N * σ k(i,q * ) j U k(i,q * ) j ⊗ V k(i,q * ) j 19: Set B (i) k = U k(i,q * ) 20: Set δ 2 = b k /#T (i) k

21:

Initialize local error:

ϵ (i) k,l = 0 22: for p ∈ T (i) k do 23: if |f k p (y k ) - j̸ =k Ω j B (i)T k f k p (y k )dy k | 2 = (ϵ (i) k ) 2 < δ 2 then

24:

The fiber is well represented by the basis

25:

Update

I (i) k = I (i) k + {p} 26: Update T (i) k = T (i) k -{p}

27:

Recompute the SVD of F

k(i,q * ,p) i = (F k(i,q * ) f k p (y k )).

28:

Update the basis

B (i) k = U k(i,q,p) i

29:

Update the error budget:

b k = b k -(ϵ (i) k ) 2

30:

Update the local error (ϵ

(i) k,l ) 2 = (ϵ (i) k,l ) 2 + (ϵ (i) k ) 2

31:

end if

32:

end for

33:

n = n+1

34: end while

This process is computed independently for each of the directions. For 1 ≤ k ≤ d, the output of the method is:

1. The sets of indices {I

(q) k } 1≤q≤N k,0
2. The sets of local error budgets {ϵ

(q) k,b } 1≤q≤N k,0 3. The set of bases {B (q) k } 1≤q≤N k,0 4. The global error budget squared b k
where N k,0 is the number of the leaves.

Let us remark that for the moment the basis obtained are 2-dimensional ones. We now consider that option and let us remember that we may still have some budget that we can spend in compressing more the tensor.

Merging local subdomains

In this section we are going to introduce a second phase of the proposed method: starting from the leaves obtained from Algorithm 9, a merging process between clusters is performed. The merging criteria will rely on the notion of similarity between clusters. This process provides a hierarchical bottom-to-top binary partition tree with an agglomerative structure, [START_REF] Sibson | Slink: an optimally efficient algorithm for the single-link cluster method[END_REF], per direction. This procedure is computed independently for each of the directions of the tensor.

Let us denote the set of clusters obtained from Algorithm 9 as {C (q) k } 1≤q≤N k,0 . In order to start the merging phase, the notion of distance between clusters needs to be set.

The computation of the distances between clusters have been widely studied and some methods are proposed in the literature, [START_REF] Mark K Goldberg | Measuring similarity between sets of overlapping clusters[END_REF]. Between the alternatives we find in the literature, let us cite some works on distances between subspaces in [START_REF] Ye | Distance between subspaces of different dimensions[END_REF][START_REF] Ye | Schubert varieties and distances between subspaces of different dimensions[END_REF]. In this work, the notion of distance used is the so called Ward's distance, denoted by ∆, (see [START_REF] Jr | Hierarchical grouping to optimize an objective function[END_REF] for more details). Find more choices of distances and its analysis in [START_REF] Murtagh | Algorithms for hierarchical clustering: an overview[END_REF].

Ward's method is used in order to find the nearest neighbor of a centroid c k p . This is performed by finding the smaller distance with respect to the others centroids weighted with a certain constant. Let us name the partition tree that we are computing per direction k as T I k . This part of the algorithm begins by computing the centroids or baricenters of the clusters. These centroids of the leaves of the T I k denoted as c k 1 , . . . , c k N k,0 are computed as the mean of the fibers well represented by the basis of the clustering, they are the barycenters of the C k 1 , . . . , C k N k ,0 clusters. For q = 1, . . . , N k,0 :

c k q = p∈I (q) k f k p (y k ) #I (q) k
Let us introduce the set of indices C (q) k of clusters that are not merged (as the analogous set to T (q) k for the fibers). At the beginning of this merging phase, the set is complete C (q) k = {1, . . . , N k,0 } and while we are merging, we are removing the indices of the clusters that are being paired. For an intermediate iteration let us have

C (q) k = {1, . . . , N k }. The process finishes when C k = {0}. q * , p * = arg min q,p∈C (q) k ∆(C k p , C k q )
Ward's method says that the distance between two clusters,C k p and C k q , is how much the sum of squares will increase when we merge them:

∆(C k p , C k q ) = i∈I (p) k ∪I (q) k ∥f k i -c k C k p ∪C k q ∥ 2 - i∈I (p) k ∥f k i -c k p ∥ 2 - i∈I (q) k ∥f k i -c k q ∥ 2 = #I (p) k #I (q) k #I (p) k + #I (q) k ∥c k p -c k q ∥ 2
where #I

(p) k , #I (q) k
is the number of elements of each set of indices and then of each cluster. Let us remark that in this case, the coefficient that multiplies the distance between the centroids promotes to the fusion of small clusters, considering them closer than if we were computing the Euclidean distance. Or what is the same, given two pairs of clusters whose centers are equally far apart, Ward's method will prefer to merge the smaller ones.

Other difference between the computation of the leaves and the merging process apart from the notion of distance chosen, is that in this case instead of selecting the nearest neighbor between one fiber and the rest, we perform a pairwise computation of the distance. This means that we compute the Ward's distances between all the clusters: for all 1 ≤ p, q ≤ N k,0 , the distances ∆(C k p , C k q ) are computed, and among all of them we select the smallest. We select the pair that provides it as the neighbor clusters.

In the leaves level of the tree l k = 0, in order to compute the pairwise distance operation, (N k,0 -1) × (N k,0 -2) × . . . × 1 distances need to be computed, taking into account that we don't recompute distances between the same two clusters. As we go up in the levels of the tree, some of the clusters are merged and this quantity diminishes. In Algorithm 10, this procedure is explained in a more detailed way.

Merging the two neighbor clusters C k q * and C k p * found, they become sons of a certain node of the tree of the level l k = 1. This node defines a new cluster C k p * ,q * that holds the fibers whose indices are in

I (q * ,p * ) k .
This set of indices

I (q * ,p * ) k
is the joint set of the ones of the sons of the node, such that

I (q * ,p * ) k = I (q * ) k ∪ I (p * ) k
. Then, it is possible to compute its basis by a POD, let us name it A k(p * ,q * ) . In this case, we truncate the SVD, in order for our approximation to have an error squared smaller or equal to the sum of the local errors squared obtained in both clusters.

Let e (q)

k,l be the error for the cluster C q k . Before starting the merging process, it is equal to the one that the computation of the leaves outputted, e

(q) k,l = ϵ (q)
k,l . Let us recall that at the end of the previous grouping given by Algorithm 9, a global error budget b k such that (ϵ

(1) k,l ) 2 + . . . + (ϵ (N k,0 ) k,l ) 2 + b k = ε 2 ,
may be left. Now, this budget can be used by equally distributing it between the clusters. A proportional part of the budget, b k /N k,0 , is added to the local budgets for each cluster. Hence, the error budget that we have for a cluster C q k reads as e

(q) k,l = ϵ (q) k,l + b k /N k,0 .
Notice that the error budget has been distributed between the clusters but the errors keep satisfying that (e

(1)
k,l ) 2 + . . . + (e (N k,0 ) k,l ) 2 ≤ ε 2 .
Respecting the prescribed accuracy is a necessary condition and this inequality will held in all the directions and for all the iterations of the method.

The error of the resulting merged cluster e (q * ) k is given by the tail of the singular values of its SVD, the sum of the singular values that we depreciate in the truncation, (e

(q * ) k ) 2 = h∈N * ,h>j (σ k(q * ,p * ) h ) 2
being j the number of terms that we need to keep in the truncated SVD in order to ensure that (e

(q * ) k ) 2 ≤ (e (q * ) k,l ) 2 + (e (p * ) k,l ) 2 .
At this stage of the algorithm, let us have the part of the error budget that has not been spent: g

(q) k = (e (q) k,l ) 2 + (e (p) k,l ) 2 -(e (q)
k ) 2 . This value is equally redistributed into all the errors (e (q) k,l ) 2 . They are updated by its value plus a factor

g (q) k #C (q) k
. Now that the two clusters are merged, the centroid of this new cluster needs to be recomputed as well as the values for the distances between clusters. The process of finding its nearest neighbor and merging the clusters they refer to is performed again. This process originates a new merge. Notice that this procedure is agglomerative and it is repeated till we have consecutively fused all the clusters in one. In this situation, the POD of the unfolding transposed is recovered. Find more details about the computation of the joint bases in Algorithm 10, where the process of merge clusters is shown in a schematic way.

Once the method explained in Algorithm 9 and Algorithm 10 finishes for all the different directions, we obtain a set of d partition trees. Each one of the trees corresponds to the succession of #l k possibilities of partitions of the domain with respect to one direction. In a certain level l k of the tree, we have N k,l k partitions of the domain. A node h, with 1 ≤ h ≤ N k , of the tree has associated sets of indices I (h) k that denote synthetically which fibers are well represented by the basis of a certain cluster and the error ϵ (h) k,l that we are making when we compute the approximation of a fiber by projecting it into the space generated by the basis of the cluster. We have obtained the different subdomains, Ω

(i) k , for 1 ≤ i ≤ N k .
Algorithm 10 Merging clusters algorithm

1: Require: Unfolding of a tensor in the direction k, F (k) (x1, . . . , x d ) ∈ R d j=1,j̸ =k N j ×N k , for 1 ≤ k ≤ d. Tolerance ε. Set of indices I (1) k , . . . , I (N k ) k . Set of local errors ϵ (1) k,l , . . . , ϵ (N k ) k,l
and error budget b k .

2:

Output: Set of indices on each node I (1) k , . . . , I

(N k ) k . Local errors ϵ (1) k,l , . . . , ϵ (N k ) k,l for 1 ≤ k ≤ d. 3: Compute the set of N k fibers f k 1 (y k ), . . . , f k N k (y k ), with 1 ≤ k ≤ d 4: Initialize the set of indices of centroids: C (0) k = {1, . . . , N k } 5: Initialize counter of iterations n = 1 6: for i = 1, . . . , N k do 7: Compute centroid: c k i = p∈I (i) k f k p (y k ) #I (i) k 8:
Distribute the budget equally in the local errors: (e

(i) k,l ) 2 = (ϵ (i) k,l ) 2 + b k /N k 9: end for 10: for i = 1, . . . , N k do 11:
Compute the Ward's distance between clusters and select the closest one:

q * = arg min q∈C (i) k ∆(C k i (x k ), C k q (x k ))
where

∆(C k i , C k q ) = #I (i) k #I (q) k #I (i) k + #I (q) k ∥c k C k i -c k C k q ∥ 2

12:

Compute joint basis matrix:

A k(i,q * ) = (B (i) k B (q * ) k ) 13:
Compute its basis by the truncated SVD:

A k(i,q * ) = j∈N * σ k(i,q * ) j U k(i,q * ) j ⊗ V k(i,q * ) j + (e (i) k ) 2 , , where (e (i) k ) 2 ≤ (e (i) k,l ) 2 + (e (q) k,l ) 2 14:
Merge clusters and set the basis of the joint

A (i) k = U k(i,q * )

15:

Set the error budget of the joint: (e

(i) k ) 2 = h∈N * ,h>j (σ k(i,q * ) h ) 2 16:
Update the local error: e

Compute the non used error budget: g

(i) k = (e (i) k,l ) 2 + (e (q) k,l ) 2 -(e (i) k ) 2

18:

Update the set of indices of centroids:

C (i) k = C (i-1) k -q 19: Redistribute g (i) k : (e (i) k,l ) 2 = (e (i) k,l ) 2 + g (i) k /#C (i) k 20:
Update the centroid:

c k i = p∈I (i) k ∪I (q) k f k p (y k ) #I (i) k •#I (q) k
21: end for

Computing local HOSVD method

In the following, in order to determine the best partitioning to approximate the tensor, we perform an extensive search for all the possible combinations. If we have l k different possibilities of partitions of the domain in the direction k, with 1 ≤ k ≤ d, the whole set of combinations is then d k=1 l k . Each of this combinations provides a certain partitioning for computing a certified approximation of the tensor F in the local domain. Between all these combinations, we select as the best the one that provides the most compressed approximation. Let us explain the method in more detail. As we have seen along the chapter, the clustering method proposed in this work, performs independently for each one of the directions. From the clustering method, d binary partition trees are obtained {T I k } 1≤k≤d . Being l k the number of partitions of the domain in the k-th dimension, with k = 1, . . . , d.

The cost of computing and comparing the HOSVD of the

Each level l k of the partition tree obtained, T I k in the direction k, defines a set of sets of indices that determine a possible partition of the domain. The possible combinations of indices are given by the number of nodes in the different generations of the tree, N k . The number of nodes per generation is equal to the total number of levels #l k minus the level of the generation l k for this binary tree structure.

We recall that the proposed clustering method provides N k sets of indices per direction namely I (i) k , with i = 1, . . . , N k , that denote the fibers that belong to the cluster C i k . During the merging process, the number of clusters, and its associated set of indices, decrease in one unit as we get closer to the root of the tree. See in the following Figure 3.3 , an schematic partition tree T I in the k-th direction for #l k = 4. As we can see, in the leaves N k,0 = 4, and with the first fusion, N k,1 = 3 and successively till arriving to the root. As we can see in Figure 3.3, as the method goes on and the levels of the tree increase, the number of partitions of the domain in one direction changes. The number of subdomains is reduced in one after each merge.

I (q,m,p,h) k I (h) k I (h) k I (h) k I (q,m,p) k I (p) k I (p) k I (q,m) k I (q) k I (m) k x k y k x k y k x k y k x k y k
For each one of the trees, each level defines one possible partition of the domain. As we have seen, each tree presents #l k levels. As we can easily see in the scheme of the partition tree, the level l k = 0 corresponding to the leaves, provides one possible partition of the domain, in which L(T

I k ) = I (h) k ∪ I (p) k ∪ I (q) k ∪ I (m) k .
In the previous generation a different partition of the domain is provided,

V 1 (T I k ) = I (h) k ∪ I (p) k ∪ I (q,m) k
, in which the partitions have decreased in one unit. It continues till we arrive to the root.

Remark:

In general, the best partition of the domain is different from the intersection of the best partitions per direction.

The sets of indices

I (i)
k are closely related with the characteristic functions J (i) k defined in Equation 3.4. They denote which fibers of the unfolding belong to the same cluster and hence, they can be well represented on a certain subdomain. Let us recall from Equation 3.5 that the characteristic functions per direction are related with the characteristic functions that restrict the tensor of Equation 3.2 to the local domain as

I (i) (x 1 , . . . , x d ) = d k=1 J (i) k (x k ) (3.12)
To illustrate the main idea of the method, let us start with a 2-dimensional example. The merging process provides two trees T I1 and T I2 . Let us say that, a certain partition of the domain provides N 1 = 2 and N 2 = 2. The local domains that appear look as follows: The combination of the partitions J 2 the red domain Ω 4 . In the figure below we can see a schematic example of how these domains can be placed in the 2-dimensional set.

Ω 1 Ω 2 Ω 4 Ω 3 x 1 1 • • • • • • • • • N 1 x 2 1 . . . . . . . . . N 2
In a different iteration of the method, we place ourselves in different levels of the trees. Let us say that they provide a partition in which the domain is more divided than before (the case is in a generation close to the leaves of the tree) and we have that N 1 = 2 and N 2 = 3 and for a certain combination of partitions, the local domains that appear in this case look as follows: As before, The combination of the partitions J In the schematic figure below we can see how these domains are distributed.

Ω 1 Ω 2 Ω 6 Ω 5 Ω 4 Ω 3 x 1 1 • • • • • • • • • N 1 x 2 1 . . . . . . . . . N 2
Among all the possible partitions, we pick the best one in terms of memory. That means that we compute the POD basis of all the non-zero subdomains and we keep the partition per direction that provides the subdomain with the smaller amount of memory needed. We have named it Ω * . In the following schematic figure, an example of the local domains obtained is presented following the simple 2-dimensional case. Without loss of generality we can say that in this case, Ω * = Ω 2 and then, the domain in which we are considering our approximation will look:

Ω * x 1 1 • • • • • • • • • N 1 x 2 1 . . . . . . . . . N 2
The aim of the method will be to find out the partitions of the domain given by J

(i1 * ) 1 (x 1 ) . . . J (i d * ) d (x d
) that provide the best approximation Fi1 * ...i d * (x 1 , . . . , x d ) in terms of memory. This means that for all

1 ≤ i 1 ≤ #l 1 , . . . , 1 ≤ i d ≤ #l d ,
between all the Fi1...i d we select as the best Fi1 * ...i d * , the one that requires less memory to store its HOSVD basis. For the sake of simplicity, in the following let us denote the set {i 1 * . . . i d * } just * . Then, the best one of the approximations becomes F * (x 1 , . . . , x d ).

After computing the memory needed to store the basis on each of the #l 1 × . . . × #l d possible partitions of the domain and selecting the best in terms of memory, we define the partition of the domain associated to the best approximation. The best of the possible partitions of the domain is given by the set of indices:

I ( * ) (x k , y k ) = d k=1 J (i k * ) k (x k )
where

J (i k * ) k (x k ), with 1 ≤ i k * ≤ N k
is the value of the characteristic function in the direction k that appears in the best combination of partitions. Then, the number of different subdomains in which the function is locally approximated reads as

N = d k=1 N k,l k * .
The best approximation obtained by the proposed method is given by Equation 3.2 and it reads:

F * (x k , y k ) = F (x k , y k )J (i1 * ) 1 (x 1 ) . . . J (i d * ) d (x d ) (3.13)
where F is the HOSVD approximation given by:

F (x k , y k ) = R1 i1=1 . . . R d i d =1 G i1...i d b (1) i1 ⊗ . . . ⊗ b (d) i d (3.14)
in which R 1 , . . . , R d are the n-ranks and they are smaller that N 1 , . . . , N d , respectively. Given the function restricted in the local subdomain F * , the ranks of the approximation are chosen by truncating the HOSVD approximation with a certain error that has been uniformly distributed in the local domains ε N k . In this way, the approximation is certified under the prescribed tolerance.

The core tensor G ∈ R R1×...×R d is stored in full format. The basis functions are written by terms, introducing the set of R k modes of each basis B k as: b k,1 , b k,2 . . . , b k,R k . These modes are the columns of the basis:

B k = (b k,1 b k,2 . . . b k,R k ).
The approximation in local HOSVD format of the tensor reads:

F (x 1 , . . . , x d ) ≈ Fi * ,...,j * J (i * ) 1 . . . J (j * ) d = R d i d =1 . . . R1 i1=1 G (i * ,...,j * ) i1,...,i d ⊗ b (i * ) 1,i1 ⊗ . . . ⊗ b (j * ) d,i d J (i * ) 1 . . . J (j * ) d (3.15)
As a remark, if we observe that in some region, the function is zero (or close to zero), we don't spend memory on that region. We consider it well approximated by the zero function. The approximation

∥F i ∥ L 2 (Ωi) < ε
N k and then Fi = 0. Imagine that following with the previous example, the function in the region Ω 4 in the example turns out to have values equal or very close to zero and its best approximation is the zero function. Then, the domain can be considered as:

Ω 1 Ω 2 Ω 5 Ω 6 0 Ω 3 x 1 1 • • • • • • • • • N 1 x 2 1 . . . . . . . . . N 2

Discretization

If we proceed with the discretization of the function by points, the variables will read: y k,p k ∈ y k for p k = 1, . . . , N yk . The approximated discretized function then reads:

F (x 1,p1 , . . . , x d,p d ) ≈ R d i d =1 . . . R1 i1=1 G (i * ,...,j * ) i1,...,i d (x 1,p1 , . . . , x d,p d ) × b (i * ) 1,i1 (x 1,p1 ) × . . . × b (j * ) d,i d (x d,p d ) J (i * ) 1 (x 1,p1 ) . . . J (j * ) d (x d,p d )

Cost and complexity of the algorithm

Let us compute the complexity of the algorithm with respect to one of the unfoldings of the input tensor. Taking into account that there are d of them, the final result will be d times the result obtained for the unfolding k, where k ∈ [1, . . . , d]. In the following we will consider the number of degrees of freedom of the fibers equal for all of them and let us call it N such that N = N yk .

Cost of computing the leaves

We start by computing the cost of its first phase of clustering/regrouping the fibers. The number of operations of this part will depend on how long it takes to find the "closest" fiber, computing its basis and then checking again the similarity between basis in the sense of the l 2 distance that we are considering.

In the first iteration, in order to compute the distance to find the nearest neighbors, the computation requires O(N N 2 k ) operations. Then, on each iteration the number is decreasing by at least 2 units. To write so, let us use the already introduced cardinality of the set of indices of the fibers that are not yet included in a cluster #T (it) . At the beginning #T (0) = N k , and it will decrease with the iterations.

This numbers are specific for the case in which the partition of the set of fibers in smaller subsets is mutually exclusive (no fiber in the set is in more than one subset) and jointly exhaustive (every fiber is in some subset).

It would be possible to accelerate this first phase constructing k -d trees (this data structure splits the space and allows faster search for neighbors), or with the ball tree method (that also divides the space but with in contrast to k-d trees, which divides the space with median value "cuts", ball tree groups points into "balls" organized into a tree structure). See more details and alternatives in [270].

After the computation of the nearest neighbors and the SVD in order to compute the basis that generates the subspace needs to be computed. Computing a full SVD of a matrix M ∈ R p×q is fundamentally an O(pq • min(p, q))-time problem. Normally, we are facing "tall and skinny" matrices, because the number of columns is equal to two and the number of rows is equal to the degrees of freedom N (normally larger than 2). The cost of an SVD of one of them is equal to 4N , i.e. linear on the degrees of freedom O(N ). The next step is checking if some of the fibers left can be well represented by the basis. This operation is composed by a scalar product between the 2d basis, a subtraction and a squared. To compute the scalar product, we must perform 2N multiplications and 2(N -1) additions. The cost reads O(N ).

The last operation that eventually we need to compute is the modification of the basis with the already introduced Brand's method. In the concrete case in which we are computing an additive rank-1 modification, the new SVD can be computed in O(R 2 b ) time, being R b the rank of the matrix.

In a certain iteration of the method in which we have included some fiber into the cluster, it, the cost of the subdivision in 2-dimensional domains will be:

O(#T (it) N N 2 k ) + O(N ) + O((#T (it) -2)N ) + O(R 2 b ) = = O((#T (it) (N 2 k + 1) -1)N + R 2 b )
In the case in which the method doesn't compute the tilting part, the expression won't have the last term. This process is done d times, one per direction. Considering all N k equal. The total cost of this phase reads:

O((#T (it) (N 2 + 1) -1)dN k + dR 2 b )

Cost of the merging

In this second part of the proposed method, a partition tree is computed. Starting from the clusters set as leaves and finishing when all the local domains have been fused and the HOSVD is recovered.

In order to compute the neighbors between centroids we compute its pairwise distance. In the leaves level of the tree, iteration it l = 0, (N k,0 -1) × (N k,0 -2) × . . . × 1 distances need to be computed, taking into account that we don't recompute distances between the same two clusters. let us name the quantity of distances that we need to compute on each iteration it l as #D (it l ) k . In the level l k , reached at it l = l k , the number of distances: (l k -1) × (l k -2) × . . . × 1. As we iterate, we go up in the levels of the tree, and this quantity diminishes because a pair of clusters are merged. The distance computation requires O(#D

(it l )2 k N k )
The next operation in the algorithm is the computation of the SVD of the joint basis matrix. In this case the matrices are still "tall and thin" because the number of columns is normally smaller that N . In the first merging, the number of columns of the matrix is at most 4 (due to the fact that in the previous part of the method the bases were at most 2-dimensional), in the second is at most 6, etc. For each iteration it l on the tree, the number of columns of the matrix obtained by SVD has at most 2 • it l + 2 columns. Hence, the order is O((2 • it l + 2) 2 N ), linear on the degrees of freedom. This way, the cost per direction reads as:

O((#D (it l )2 k + (2 • it l + 2) 2 )N k )
As before, the merging process is done independently for each one of the d directions. If we consider the value of #D (it l ) k equivalent in all the directions, the total cost:

O((#D (it l )2 k + (2 • it l + 2) 2 )dN k )

Cost of the basis computation

The last operation in the algorithm is the computation of the SVD of the d k=1 l k possible partitions of the domain.

Compute the cost of the computation of the basis is difficult a priori because it depends on the partitions produced by the method. Let us estimate this value considering generic partitions of the domains. Let us suppose that for the k-th direction, the partition of the subdomain Ω k that provides the best partition of the whole domain Ω involves N k p k fibers of size N . Let p k be a generic fractional coefficient to describe the partition. The cost of one computation of the HOSVD for the local domain is:

O( d k=1 N k p k )
The total cost of the algorithm depending on the iterations of the first and second phase results:

O((#T (it) (N 2 k + 1) -1 + #D (it l )2 + (2 • it l + 2) 2 )dN k + dR 2 b + d k=1 N k p k ) (3.16)
Let us also remark that in the subdomains in which the value for the tensor is zero, or very close to zero, no memory is spent, neither the computation of the SVD is done once we have identified the null regions. Even though, the cost is very high. In this work a first approach to a good local approximation is studied by computing all the possibilities, computing their memories and keeping the most compressed one.

Summary

The clustering method proposed performs independently for each one of the unfolding matrices on the different directions. Its aim is to compute a partition tree per direction that encloses the different partitions of the domain in subdomains. Among them, the one in which the compression ratio for a given tensor is larger is selected.

In broad terms, the process can be divided in three different phases: the initial one that computes 2-dimensional subdomains, leaves of the partition trees; the second phase in which the tree is computed that merges consecutively them and saves information on every merging; and the last one in which, once the two previous phases of the method have finished for all the different dimensions, between all the possible partitions of the domain, selects the one in which the tensor is more compressible. Let us recapitulate the main steps performed by the method.

For 1 ≤ k ≤ d, let F (k) be the unfolding of the k-th direction of the tensor F .

• Computation of the leaves -Starting from the fibers (column vectors) of the unfolding, find the nearest neighbor.

-Compute 2-dimensional basis of the space spanned by the two neighbor fibers as the matrix of modes of a POD of the two vectors.

-Check if other fibers are well represented by the basis. If it is so, modify the POD with Brand's method of Equation 3.11.

At this point, a set of clusters defined by 2-dimensional basis that well represents some of the fibers (with a certain error) is obtained. They are the leaves of the partition tree constructed. In the following phase, the tree is constructed.

• Merging clusters -Compute centroids of each cluster.

-Compute pairwise distances between clusters and select the smallest.

-Compute the joint subspace, a new cluster resulting from the merging of two clusters.

-Recompute centroids and distances for the new cluster.

This process is repeated till all the clusters have been fused. On each fusion, the resulting partition of the domain is saved.

• Selecting the best partition of the domain

The task of selecting the best partition of the domain is done by computing all the possible combinations of partitions of the domain between the different directions. The method computes all the HOSVD and selects as the best the one that requires less memory to store the approximation, i.e. the one that provides higher compression ratio with respect to the HOSVD of the initial tensor.

Numerical results

In the present section, we propose several test cases to asses the properties of the proposed method. The tensors we are going to approximate are of moderate order because, as we have already introduced, the computational complexity of the method is high.

Compression of a Gaussian function

The initial test consists in computing the approximation of a Gaussian function. The function to approximate reads as: f (x, t) = e -500 * (x-0.05-t) 2 (3.17)

where x ∈ [0, 1] and t ∈ [0, 0.9]. The resolution for the variable x is fixed to N x = 500 points and the resolution in the variable t, N t is going to be changed in order to see how its value affects the results. We consider a finite difference approach of the function f . The compression ratio (Cr) shown in the following results is defined as the memory employed to store the compressed tensor obtained by the local HOSVD method over the memory needed to store the tensor in full format: 3.17 with an error of ϵ = 1.0 • 10 -2 and resolution N x = 500 and N t = 400. In the figure, the exact function is reconstructed with the HOSVD method and with its local version. It is known that the HOSVD method is not optimal for the reconstruction of this function, leading to some oscillations that are not present in the exact function. We can observe that in the reconstruction made via the local HOSVD method, these oscillations have disappeared. The exact solution of the Equation 3.17 is shown, as well as its reconstructions by local HOSVD and HOSVD methods.

As we can see, the reconstruction with local HOSVD presents some ondulatory interferences, this can be due to the large value for the error that we are considering. On the other hand, the reconstruction computed with HOSVD presents several oscillations is all the domain, more concentrated close to the solution and wider as we are going further.

In the following table, Table 3.1 some values of the lowest memory storage required obtained from the extensive research between the possible partitions of the domain in which the HOSVD method has been applied. They are computed for different resolutions in the domain, changing the number of degrees of freedom in the variable x, N x , and in t, N t , as well as the values for the error. In this test, the error adopts values from 1.0 • 10 -4 to 5.0 • 10 -2 . It is shown as well, the compression ratio. In this case, the ratio that appears in the table is not with respect to the memory needed to store the function in the full format but to the one needed to store the HOSVD (in this 2-dimensional case, SVD) decomposition of the function.

Moreover, the number of partitions per direction that provides the best compression ratio is shown.

As we can see in Table 3.1, the compression ratio obtained with the proposed method varies with relation to the HOSVD, providing a larger compression of the data with an approximately factor two. This ratio stays stable even when changing the tolerance or the resolution. Nevertheless, the optimal number of partitions of the domain changes with the prescribed tolerance and also with the resolution.

Compression of the solutions of the Fitz-Hugh-Nagumo equation

The Fitz-Hugh-Nagumo model (FHN), named after Richard FitzHugh (1922-2007) who suggested the system in 1961 and J. Nagumo et al. who created the equivalent circuit the following year, describes a prototype of an excitable system (e.g., a neuron).

The FHN Model is an example of a relaxation oscillator because, if the external stimulus I ext exceeds a certain threshold value, the system will exhibit a characteristic excursion in phase space, before the variables u and w relax back to their rest values. This behavior is typical for spike generations (a short, nonlinear The pair of equation describing the phenomena:

∂ t u(x, t) = -ku(x, t)(u(x, t) -a)(u(x, t) -1) -w(x, t) + ν∆u(x, t) + I ext (3.18) ∂ t w(x, t) = ϵ(u(x, t) -γw(x, t))
where ∆ : The parameters ϵ > 0, the time of the pulse, and γ ≥ 0, the strength of the recovery of the system. The different choices of the parameters in the equations make the solution u(x, t) adopting different shapes: from the stationary wave to a labyrinth structure, passing trough certain values that produce a spiral. See [START_REF] Zheng | Pattern formation in the fitzhugh-nagumo model[END_REF] for more information about the values of the parameters and the different patterns that they produce. In the tests performed, we are facing a 2-dimensional space variable, the time variable and other variable, let us call it θ, that refers to the parameters of the system. In this case, the variable θ encloses a linear combination of the parameters k and ϵ. In some tests, the variable θ will adopt a fixed value and it will behave as a parameter of the system. In other tests, it will adopt values inside a certain domain and it will be treated as a third variable.

= ∂ 2 ∂ 2 x + ∂ 2 ∂ 2
The system is simulated numerically by a finite differences scheme on a (256 × 256) two-dimensional spatial grid. The number of time steps is being changed on the different tests performed. The final time has been set as T = 800. The initial conditions from which we have recovered the spiral shape have been: u(x, 0) = 1 for x ∈ ([0, 0.5] × [0, 0.5]) and 0 in the rest, being the left down quarter of the square 1 and the rest 0. The values for the parameters chosen in all the simulations are: γ = 1, a = 0.1, ν = 1/62500, k = 0.96 and ϵ = 0.0015. In the case in which they are treated as variables and take values in the intervals they read k ∈ [0.96, 1.20] and ϵ ∈ [0.0015, 0.0060] respectively.

In Figure 3.7 we can see how from the initial conditions the system evolves in time. In order to compute this simulation, the number of time steps is N t = 400. From Figure 3.10, we can see how the local HOSVD method has divided the spatial domain in order to compute the approximation. We can also see how the shape of the partitions is adapted to the initial condition and its initial translation. The Figure 3.11 shows three analogous plots for three consecutive partitions on the spatial domain in a longer time T = 800. In this case, the system has evolved and we can identify the spiral structure that characterizes the solution of the equation. We can see that the method is capable of capturing the behavior of the system and adapting the partitions made to it.

In Table 3.2 and Table 3.3 a summary of the memory compression for the obtained best number of divisions of the space is included. In Table 3.2 we start by changing the final time of the experiment, T = {200, 400, 800} and fixing the time step n t = 1 in such a way that the degrees of freedom of the time variable change with the final time.

In the following table, Table 3.3 resolution in time. Let us fix the final time T = 800 and let us change the time step n t = {4, 2, 1} in a way that the degrees of freedom in this variable also change as N t = {200, 400, 800}, respectively.

As we can see in the previous results, the compression rate with respect to the HOSVD is notorious. Starting from a factor two of compression and reaching so far a factor eight in some cases. The results are very optimistic. When the prescribed tolerance decreases, more costly the HOSVD becomes, and the advantage of the use of the local approximation method arises. The number of optimal partitions of the domain is augmenting when decreasing the tolerance for the same resolution what was expected because the number of partitions obtained before starting the merging phase was also larger.

Conclusions and perspectives

In this chapter a method that computes the local HOSVD approximation of a tensor is introduced. This is done by computing a HOSVD approximation of the tensor restricted to local subdomains, obtained by dividing the whole domain. A key point of the method is how to compute the subdomain in which the function is well approximated. What we propose in this work is that it is obtained automatically by applying a clustering method independently on each direction. This can be done due to the separability of the domain and it provides a certain partition of the domains on each direction. In this work a first approach is obtained by computing all the possible partitions of subdomains and keeping the one that provides the most compressed approximation. The main negative aspect of the proposed method is that the cost of the algorithm is very high, even though the numerical tests have been restricted to moderate order tensors. In further works a less expensive approach to the current method will be studied and eventually a good trade-off between memory and computational cost will be studied.

From the numerical results of the tests computed, we can infer that the partitions of the domain obtained with the clustering method are able to provide a certified approximation by spending less memory. This aspect increases notoriously when the tensor we want to approximate presents different regimes.

Introduction

Singularly perturbed differential equations are typically characterized by a small parameter ϵ > 0 multiplying some of the highest order terms in the differential equation. In general, the solutions to such equations exhibit multiscale phenomena, and this raises significant challenges to classical numerical methods such as finite elements or finite volumes. To build accurate and robust approximations with these methods as ϵ decreases, it is necessary to develop elaborate numerical discretizations. In addition to the mathematical difficulties of the formulation, the resulting numerical schemes are often not entirely trivial to implement: they often require mesh adaptation, and working on complicated geometries is challenging. These difficulties motivate the search for new discretization schemes, hopefully mesh-free, with potential to deliver good quality approximations with easier implementation techniques. In this chapter, we explore this research direction, and consider strategies based on deep learning techniques. Our main goal is to test various neural network-based schemes, so as to design a strategy which should be robust when ϵ → 0, easily implementable even for complicated geometries, and with potential to scale in high dimension.

The idea of working with neural network functions to solve PDEs is by far not novel, and countless contributions have been proposed on this front in recent years. The strategies can be roughly classified into two categories:

1. In the first category, deep neural networks are employed to assist classical numerical methods by improving some limitations, or accelerating certain steps (see, e.g., [START_REF] Lee | Neural algorithm for solving differential equations[END_REF][START_REF] Wang | Structured trainable networks for matrix algebra[END_REF]). It has notably been used to assist in the construction of numerical fluxes adapted to Finite Volumes (see, e.g., [START_REF] Després | Machine learning design of volume of fluid schemes for compressible flows[END_REF]). They can also be used in order to compute reduced-order models of parametric problems: for each value of the parameters, the solution (or other quantities of interest) of the model of interest is computed by means of a standard numerical scheme, and the values of the solutions are interpolated by mean of a deep neural network over the whole range of parameter values [START_REF] Sumeet Trehan | Error modeling for surrogates of dynamical systems using machine learning[END_REF][START_REF] Freno | Machine-learning error models for approximate solutions to parameterized systems of nonlinear equations[END_REF].

2. In the second category, neural networks are used to directly approximate the solution of PDEs. The solution schemes become in this case an optimization problem where it is crucial to design appropriate loss functions. The loss functions are mostly based on residuals of the equations, and yield to different methods depending on the specific choice:

(a) Physics-informed neural networks (PINNs, [START_REF] Raissi | Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[END_REF]) is a collocation-based method. One finds the coefficients of the neural network solution by minimizing a discretized version of the L 2 norm of the strong form of the residual of the PDE. This method is very easily implementable but it implicitly assumes that solutions are very regular.

(b) Other strategies leverage weak variational formulations where less regular solutions are allowed. On this front, most of the classical methods originally formulated for piecewise polynomial functions have by now been tested with trial and test spaces of neural network functions. In this respect, the deep Galerkin method (DGM, [START_REF] Sirignano | Dgm: A deep learning algorithm for solving partial differential equations[END_REF]) is based on a least-squares formulation, and the variational PINNS (vPINNs, [START_REF] Kharazmi | Variational physics-informed neural networks for solving partial differential equations[END_REF][START_REF] Kharazmi | hp-vpinns: Variational physics-informed neural networks with domain decomposition[END_REF]) is based on the Galerkin method. The main drawback of this approach is that the approximation quality depends on the architecture of both the trial and the test neural network classes. In addition, numerous evaluations for multiple test functions need to be performed. Also, strategies involving the minimization of weak-form residuals are usually not trivial to implement because they involve the computation of norms in very weak spaces which necessitate extra discretization steps.

(c) Another approach based on weak variational formulations is the so-called deep Ritz method (DRM, [START_REF] Yu | The deep ritz method: a deep learning-based numerical algorithm for solving variational problems[END_REF]). It leverages the fact that the solution of certain PDEs is the unique minimizer to a certain energy functional. When possible, this approach seems the most appealing: the loss function is naturally given by the problem, it can accommodate low regular solutions, and the computational cost is moderate in the sense that it only requires to handle test functions (no trial functions). It also carries potential to address high dimensional problem as illustrated in [START_REF] Han | Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations[END_REF][START_REF] Yu | The deep ritz method: a deep learning-based numerical algorithm for solving variational problems[END_REF][START_REF] Rotskoff | Active importance sampling for variational objectives dominated by rare events: Consequences for optimization and generalization[END_REF].

The goal of this part of the work is to develop and compare several neural network schemes for singularly perturbed problems when ϵ → 0. We focus more particularly on convection-diffusion (or stationary Fokker-Planck) problems with vanishing diffusion for which we explore schemes from the second category according to the above distinction. In other words, we approximate solutions of singular PDEs with feed-forward neural network functions. When ϵ → 0, the regularity of the solutions is deteriorated because of local or boundary thin layers.

The classical formulation commonly used in neural network based schemes is constructed from the strong formulation of the problem, where the analytical solution is approximated by the one generated by evaluations on the sampling points. It is referred in the following as vanilla PINNs, which has been introduced in the (2)-(a) subcategory on the introduction. More details could be found in Section 4.2.3. Therefore, it is expected to perform poorly for small values of ϵ because it commits a "variational crime" (and this is actually confirmed in our numerical experiments). By "variational crime", we mean that the norm of the residual which is traditionally used in vanilla PINNs methods requires that the solution of the PDE has to be more regular than what would naturally be expected from the theory.

In our case (convection-diffusion problems), the Vanilla-PINN method requires the solution to belong to an H 2 space, whereas it is more natural from a theoretical point of view to look for an approximation of the solution in a set of H 1 functions.

Methods based on weak variational formulations seem better adapted, and on that front, it is desirable to work with the deep Ritz method. However, finding energy formulations is not straightforward due to the non-symmetric nature of convective effects. We show how this method can be applied in this context thanks to a change of variable. We compare its numerical robustness with respect to the PINNs method, and a naive finite element discretization with a uniform grid. In the present study, our tests are performed on a 1D example. Despite its simplicity, the example exhibits all the features that are challenging for numerical schemes. For our purposes, the example also presents the important advantage of having analytic solutions which we can leverage in our error analysis, and our validations. Higher-dimensional tests involving also more elaborate sampling strategies are left for future work.

Organization of the chapter

This chapter is organized as follows. In Section 4.2, various formulations of the convection-diffusion problem we are interested in are introduced. In Section 4.3, we introduce various neural networks-based schemes which are inspired from the various formulations introduced in Section 4.2. The reader is encouraged to observe that an expert mathematical insight is required in order to build formulations that do not incur in variational crimes. In Section 4.4, these various schemes are compared for one-dimensional problem. To conclude, in Section 4.4.4, after the presentation of the numerical results, it is summarized how each one of the PINN methods behave for small values of ϵ and its comparison with the FEM method.

A singularly perturbed convection-diffusion equation

The aim of this section is to introduce the singularly perturbed convection-diffusion equation we consider in this work, and various formulations of the problem which will be used in Section 4.3 so as to design various neural networks-based schemes for its numerical solution.

Problem definition

As a prototypical example, we consider the following singularly perturbed convection-diffusion equation on a given domain Ω ⊂ R d , with d ∈ N * . Let F : Ω → R d be a given force field, 0 < ϵ ≪ 1 a small parameter, and f : Ω → R be a given right-hand side function. Our goal is to find a solution u : Ω → R to

-ϵ(∆u)(x) + ∇ • (F u)(x) = f (x), ∀x ∈ Ω, ( 4.1)
together with Robin boundary conditions

α(∇u • n)(x) + κu(x) = g(x) ∀x ∈ ∂Ω, (4.2)
where n refers to the outward unit vector of ∂Ω, α, κ ≥ 0 and g is a real-valued function defined on ∂Ω. In the following, we assume that the force field F derives from a potential function V : Ω → R, in the sense that

F (x) = -∇V (x), ∀x ∈ Ω.
Under appropriate assumptions on F (or V ), f and g, which are assumed to be smooth functions for the sake of simplicity, problem (4.1)-(4.2) can be proved to have a unique solution [START_REF] Risken | The Fokker-Planck Equation[END_REF][START_REF] Jordan | The variational formulation of the fokker-planck equation[END_REF][START_REF] Bogachev | Fokker-Planck-Kolmogorov Equations[END_REF]. Note that, more generally, α and κ could also be given as real-valued functions defined on ∂Ω, instead of constants, and our subsequent developments could be easily adapted.

The equation represents the change in the concentration u of a quantity in a given medium in presence of convective and diffusive effects. The force field F represents the drag force while the singular perturbation parameter ϵ represents the diffusivity of the medium. In the limit of an inviscid medium as ϵ → 0, the equation changes from elliptic to hyperbolic nature, and from second to first order. For Dirichlet boundary conditions u = 0 on ∂Ω, the solution can develop sharp boundary layers of width ϵ near the outflow. We refer the reader to [START_REF] Roos | Robust numerical methods for singularly perturbed differential equations: convection-diffusion-reaction and flow problems[END_REF] for general references on this equation regarding its analysis and numerical methods.

Classical numerical methods are challenged by problem (4.1) when ϵ is small. In the case of the Galerkin finite element method, the poor performance for this problem is reflected in the bound on the error in the finite element solution. For Ω = (0, 1) and Dirichlet boundary conditions, a standard Galerkin method with a uniform grid of size h delivers a solution u h on a finite element space P h that satisfies

∥u -u h ∥ H 1 (0,1) ≤ C(ϵ) inf w h ∈P h ∥u -w h ∥ H 1 (0,1) , ( 4.3)
where C(ϵ) ∼ ϵ -1 , so that the constant blows up as ϵ → 0 (see [START_REF] Roos | Robust numerical methods for singularly perturbed differential equations: convection-diffusion-reaction and flow problems[END_REF]Theorem 2.49]). The dependence of C on ϵ is usually referred to as a loss of robustness in the sense that, as ϵ decreases, the Galerkin method is bounded more and more loosely by the best approximation error. As a consequence, on a coarse mesh and for small values ϵ, the Galerkin approximation develops spurious oscillations everywhere in the domain. This very well-known behavior will actually be observed later on in our numerical tests.

Numerous methods have been proposed in order to address this loss of robustness in finite element methods. An important family of methods is based on using residual-based stabilization techniques. Given some variational form, the problem is modified by adding to the bilinear form the strong form of the residual, weighted by a test function and scaled by a stabilization constant τ . The most well-known example of this technique is the streamline upwind Petrov-Galerkin (SUPG) method (see [START_REF] Brooks | Streamline upwind/petrov-galerkin formulations for convection dominated flows with particular emphasis on the incompressible navier-stokes equations[END_REF]). The addition of the residual-based stabilization term, can be interpreted as a modification of the test functions which means that these methods seek stabilization by changing the test space, and motivates to search for optimal test spaces in the spirit of [START_REF] Demkowicz | Robust DPG method for convection-dominated diffusion problems[END_REF][START_REF] Chan | A robust dpg method for convection-dominated diffusion problems ii: Adjoint boundary conditions and mesh-dependent test norms[END_REF].

Other classical discretization methods such as finite volumes suffer from similar issues, and strategies involving layer-adaptive grids such as Shishkin meshes have been proposed (see, e.g., [START_REF] Kopteva | Shishkin meshes in the numerical solution of singularly perturbed differential equations[END_REF]).

The aim of this work is to explore the potential of approximating solutions of such problems with neural network functions, and the next section presents several options for this, with a discussion on their merits and limitations.

General formulation

Any neural-network based numerical scheme for the solution of (4.1)-(4.2) relies on the use of a variational formulation of this problem which enables to write u (or another function defined from u) as a minimizer of a problem of the form min

v∈V J (v), (4.4)
where V is a particular set of real-valued functions defined on Ω. The loss function J : V → R is usually of the form

J (v) := Ω R(v)(x)dρ(x) + ∂Ω S(v)(r)dτ (r), ∀v ∈ V, (4.5)
where for every v ∈ V, R(v) and S(v) are real-valued functions defined on Ω and ∂Ω respectively. They are assumed to be integrable with respect to the measures ρ and τ , which are defined on Ω and ∂Ω respectively. Note that the measures ρ and τ have to be chosen a priori, and they can greatly affect the final result. The question of discovering the optimal weights is beyond the scope of our present investigation.

The aim of the next sections is to introduce various formulations of problem (4.1)-(4.2) under the form (4.4)-(4.5). This requires appropriate definitions of the set V, the functions R(v) and S(v) for any v ∈ V and the unknown function solution of (4.4). Unless otherwise stated, the measures ρ and τ will be defined as the Lebesgue bulk measure and Lebesgue surface measure respectively.

Vanilla (V) formulation

We begin by introducing the most classical formulation used in neural network-based numerical schemes such as PINNs. For the reasons that we outline next, different aspects of this formulation can be improved, therefore we refer to it as vanilla (V) formulation in the following.

The formulation consists in interpreting the solution u of (4.1)-(4.2) as the unique solution of a minimization problem of the form (4.4) with V = H 2 (Ω) and to define for all v ∈ V,

R(v)(x) := λ |-ϵ(∆v)(x) + ∇ • (F v)(x) -f (x)| 2 , for all x ∈ Ω, S(v)(x) := (1 -λ) |α(∇v • n)(x) + κv(x) -g(x)| 2 , for all x ∈ ∂Ω, ( 4.6)
for some λ ∈ (0, 1). In this approach, the parameter λ enables to tune the respective weight of the contributions of the bulk and boundary terms in the total functional J to be minimized. In practice, in the numerical tests presented in Section 4.4, λ will always be chosen to be equal to 1 2 .

Note that such an approach requires the solution u to belong to H 2 (Ω), which implies that the solution has to be sufficiently regular. When ϵ → 0, this assumption becomes less and less realistic due to the formation of boundary layers. This raises the question as to whether it is possible to introduce another formulation of problem (4.1)-(4.2) which would allow for less regular solutions. The goal of the next section is to introduce such an alternative formulation.

Weak variational (W) formulation

In this section we develop an avenue based on an energy minimization approach which requires less regularity in the solutions than the vanilla formulation. To this aim, we introduce the change of variable

u(x) = e cV (x) z(x), ( 4.7)
where c ∈ R is a constant yet to be determined. Taking first and second derivatives in (4.7) yield that for all x ∈ Ω,

∇u(x) = e cV (x) (c∇V (x)z(x) + ∇z(x)) ∆u(x) = e cV (x) c∆V (x)z(x) + |c∇V (x)| 2 z(x) + 2c∇V (x) • ∇z(x) + ∆z(x) .
Now, setting the value of c to be

c = 1 2ϵ ,
and inserting the change of variable into (4.1), we conclude that u is a solution to (4.1) if and only if z is a solution to the elliptic problem

-∆z(x) + ∆V (x) 2ϵ + |∇V (x)| 2 4ϵ 2 z(x) = f (x) e -V (x) 2ϵ ϵ , ∀x ∈ Ω, (4.8)
with Robin boundary conditions

α(∇z(x) • n(x)) + κ + α 2ϵ ∇V (x) • n(x) z(x) = e -V (x) 2ϵ g(x), ∀x ∈ ∂Ω. ( 4.9)
At this stage, one could of course apply the vanilla formulation to solve (4.8)-(4.9) and compute z solution of a minimization problem of the form (4.4) with V = H 2 (Ω) and the functionals R and S defined by

     R(v)(x) := λ ∆v(x) + ∆V (x) 2ϵ + |∇V (x)| 2 4ϵ 2 v(x) -f (x) e -V (x) 2ϵ ϵ 2 , for all x ∈ Ω, S(v)(x) := (1 -λ) α(∇v(x) • n(x)) + κ + α 2ϵ ∇V (x) • n(x) v(x) -e -V (x) 2ϵ g(x)
2 , for all x ∈ ∂Ω, (4.10) for all v ∈ V = H 2 (Ω) and some λ ∈ (0, 1). The value of λ chosen in our numerical tests is λ = 0.5. We will refer to this approach as the vanilla-z (V z) formulation.

Note that this method does not fully exploit the change of variables since the elliptic nature of problem (4.8) allows us to easily build a weak formulation of this equation. Testing against a smooth test function v and integrating by parts we obtain the weak formulation

Ω ∇z • ∇v - ∂Ω v∇z • n + Ω ∆V 2ϵ + |∇V | 2 4ϵ 2 zv = Ω f e V 2ϵ v ϵ
Using equality (4.9), we get

Ω ∇z • ∇v + Ω ∆V 2ϵ + |∇V | 2 4ϵ 2 zv + ∂Ω κ α + 1 2ϵ ∇V • n zv = Ω f e -V 2ϵ ϵ v + ∂Ω 1 α e -V 2ϵ gv.
Therefore the weak formulation of problem (4.8) is to find z ∈ H 1 (Ω) such that

a(z, v) = ℓ(v), ∀v ∈ H 1 (Ω) (4.11) with a(z, v) = Ω ∇z • ∇v + Ω ∆V 2ϵ + |∇V | 2 4ϵ 2 zv + ∂Ω κ α + 1 2ϵ ∇V • n zv l(v) = Ω f e -V 2ϵ ϵ v + ∂Ω 1 α e -V
2ϵ gv.

To ensure that the symmetric bilinear form a is continuous and coercive, we assume in the sequel that the following conditions are satisfied:

       ∆V (x) 2ϵ + |∇V (x)| 2 4ϵ 2 ≥ a 0 > 0, ∀x ∈ Ω κ α + 1 2ϵ ∇V (x) • n(x) ≥ 0, ∀x ∈ ∂Ω f ∈ L 2 (Ω), g ∈ L 2 (∂Ω) (4.12)
In that case, all the hypothesis of the Lax-Milgram theorem are satisfied for a and ℓ. Notice that, if V is a regular confining potential (i.e. if ∇V (x) • n(x) ≥ 0 for all x ∈ ∂Ω), then there exists ϵ 0 > 0 such that conditions (4.12) are satisfied for all ϵ ≤ ϵ 0 .

If conditions (4.12) are satisfied, z can be equivalently rewritten as the unique solution of a minimization problem of the form

z = argmin v∈H 1 (Ω) 1 2 a(v, v) -ℓ(v). ( 4

.13)

This implies that z can equivalently be recast as the unique solution of a minimization problem of the form (4.4) with V = H 1 (Ω) and

   R(v)(x) = 1 2 |∇v(x)| 2 + ∆V (x) 2ϵ + |∇V (x)| 2 4ϵ 2 |v(x)| 2 -f (x) e -V (x) 2ϵ ϵ v(x), ∀x ∈ Ω, S(v)(x) = 1 2 κ α + 1 2ϵ ∇V (x) • n(x) |v(x)| 2 -1 α e -V (x) 2ϵ g(x)v(x), ∀x ∈ ∂Ω. ( 4.14)
We will refer to this approach as the weak-z (Wz) formulation.

Moreover, using (4.7), we can equivalently express u as a solution of a minimization problem of the form (4.4) with

V := v = e V 2ϵ v, v ∈ H 1 (Ω) , ( 4.15)
and rewriting the Equation (4.14)

   R(v)(x) = 1 2 |∇v(x)| 2 + ∆V (x) 2ϵ + |∇V (x)| 2 4ϵ 2 |v(x)| 2 -f (x) e -V (x) 2ϵ ϵ v(x), ∀x ∈ Ω, S(v)(x) = 1 2 κ α + 1 2ϵ ∇V (x) • n(x) |v(x)| 2 -1 α e -V (x) 2ϵ g(x)v(x), ∀x ∈ ∂Ω, (4.16)
for all v ∈ V with v := ve -V 2ϵ the change of variable suggested above. We will refer to this formulation as the weak (W) formulation.

Note that in the non-discretized case, formulations (W) and (W-z) are equivalent up to the exponential change of variable. However, when the minimizer of both formulations is computed by means of a neural network, the corresponding approximations will be different and have different accuracies. The (W) formulation has the advantage to avoid potential machine precision issues linked to the presence of the exponential term when ϵ becomes small.

Rescaled formulation

In this section, we introduce another formulation based on a change of scale in the original problem. More precisely, introducing Ω ϵ := 1

ϵ Ω, we introduce auxiliary functions u : Ω ϵ → R, z : Ω ϵ → R and V : Ω ϵ → R defined so that for all x ∈ Ω,

u(x) = ϵ u x ϵ , z(x) = ϵ z x ϵ , V (x) = ϵ V x ϵ . ( 4
v ∈ H 1 (Ω ϵ ), Ωϵ ∇ z(y) • ∇ v(y) dy + ∂Ωϵ ϵκ α + 1 2 ∇ Ṽ (y) • n(y) z(y) v(y) dy + Ωϵ ∆ Ṽ (y) 2 + |∇ Ṽ (y) | 2 4 z (y) v(y) dy = (4.20) = Ωϵ f (y)e -1 2 Ṽ (y) v(y) dy + ∂Ωϵ 1 α e -1 2 Ṽ (y) g(y) v(y)dy.
The result is valid provided that the following assumptions on the coefficients hold

     ∆ Ṽ (y) + |∇ Ṽ | 2 (y) ≥ 0, ∀y ∈ Ω ϵ , κϵ α + 1 2 ∇ Ṽ (y) • n(y) ≥ 0, ∀y ∈ ∂Ω ϵ , f ∈ L 2 (Ω ϵ ), g ∈ L 2 (∂Ω ϵ ).
The above conditions are equivalent to the assumptions (4.12) stated in Section 4.2.4.

As in the previous section, the function z is then the unique solution of a minimization problem of the form (4.4) with

V = H 1 (Ω ϵ ) with      R(v)(y) = 1 2 |∇v(y)| 2 + ∆ V (y) 2 + |∇ V (y)| 2 4 |v(y)| 2 -f (y)e -V (y) 2 v(y), ∀y ∈ Ω ϵ , S(v)(y) = 1 2 ϵκ α + 1 2 ∇ V (y) • n(y) |v(y)| 2 -1 α e -V (y) 2
g(y)v(y), ∀y ∈ ∂Ω ϵ .

(4.21)

We will refer to this approach as the rescaled-weak-z (RWz) formulation.

Summary of the methods

For the sake of clarity, we summarize here the main features of each method.

Method

Neural networks based numerical schemes

In this section we describe the numerical approach used in order to compute an approximation of the solution of a minimization problem of the form (4.4) by means of a neural-network based method. We first present in Section 4.3.1 the general principle of such approaches. The main ingredients to design a neural-network based method consist in the choice of a class of neural network functions and of sampling schemes in order to approximate the integrals involved in the definition of the loss function J defined by (4.5). These two ingredients are detailed respectively in Section 4.3.2 and Section 4.3.3. Finally, some details on the numerical implementation are given in Section 4.4.2.

General principle

The numerical solution of a minimization problem of the form (4.4) usually requires to consider alternatives to V and J that are amenable for practical implementation. • J is an approximation of the loss function J where the integrals are approximated using some particular quadrature or sampling schemes.

More precisely, for given integers K, M ∈ N * , given sets of points (x k ) 1≤k≤K ⊂ Ω, (y m ) 1≤m≤M ⊂ ∂Ω, and given sets of weights (ρ k ) 1≤k≤K ⊂ R and (τ m ) 1≤m≤M ⊂ R, for all v ∈ K, the functional J (v) is defined by

J (v) := K k=1 ρ k R(v)(x k ) + M m=1 τ m S(v)(y m ). ( 4

.23)

As a consequence, the definition of a neural-network based numerical scheme for the approximation of a problem of the form (4.4) requires the definition of two ingredients:

• the class K ⊂ V of neural network functions;

• the sampling scheme, i.e. the choice of K, M , (x k ) 1≤k≤K , (y m ) 1≤m≤M , (ρ k ) 1≤k≤K and (τ m ) 1≤m≤M in order to define the approximate functional J given by (4.23).

The set of neural network functions K we consider in our numerical experiments is presented in Section 4.3.2. The various sampling schemes tested here are given in Section 4.3.3.

Neural Network classes of functions

In this work, we only consider classes of functions defined by means of feedforward neural networks whose definition was already introduced and we recall next (see [START_REF] Ma | Towards a mathematical understanding of neural network-based machine learning: what we know and what we don[END_REF] for more detailed definitions).

Let X ⊂ R d X and Y ⊂ R d Y be some input and output sets of finite dimensions d

X , d Y ∈ N * . A feedforward neural network is a function ψ : X → Y which reads as ψ(x) = T L (σ(T L-1 (σ(. . . σ(T 0 (x))))), ∀x ∈ X . ( 4

.24)

Recalling Equation 1.13, for every ℓ ∈ {0, . . . , L},

T ℓ : R p ℓ → R p ℓ+1 x ℓ → T ℓ (x ℓ ) = A ℓ x ℓ + b ℓ (4.25)
is an affine function which can be expressed through a matrix A ℓ ∈ R p ℓ+1 ×p ℓ , an offset vector b ℓ ∈ R p ℓ+1 , and the so-called (nonlinear) activation function σ : R → R. By a slight abuse of notation, for all p ∈ N * and for any vector w := (w i ) 1≤i≤p ⊂ R p , the notation σ(w) actually denotes the vector of R p with entries σ(w i ), that is, σ(w) = (σ(w i )) p i=1 . Note that since ψ maps X onto Y, it is necessary that p 0 = d X and p L+1 = d Y . The layers numbered from 1 to L are the hidden layers of the neural network.

To define a class of feedforward neural networks, we fix an architecture by prescribing a given activation function σ, depth L ∈ N, and layer widths p = (p 0 , . . . , p L+1 ) ∈ (N * ) L+2 . Once the values of σ, L and p have been chosen, we view the coefficients (A ℓ , b ℓ ) 0≤ℓ≤L of the affine mappings T 0 , • • • , T L as parameters. We gather these coefficients in the vector of parameters

θ = {(A ℓ , b ℓ )} L ℓ=0 ,
and assume that θ takes values in a set

Θ ⊆ L ℓ=0 R p ℓ ×p ℓ+1 × R p ℓ+1 .
For any θ ∈ Θ, we define by ψ θ : X → Y the function ψ defined by (4.24) with θ = {(A ℓ , b ℓ )} L ℓ=0 ∈ Θ. The class of neural network functions with architecture (σ, L, p) and coefficient sets Θ is then defined as N (σ, L, p, Θ) := {ψ θ : θ ∈ Θ} In our context, the input and output sets X and Y are respectively given by X = Ω (or Ω ϵ ) and Y = R, so that d X = d and d Y = 1. In all the numerical tests presented below, the class K is chosen as

K := N (σ, L, p, Θ), with σ = tanh, L = 2, p = (d, 10, 10, 1) and Θ = L ℓ=0 R p ℓ ×p ℓ+1 × R p ℓ+1 .
Note that the set K is then a subset of V for all the formulations of the convection-diffusion problem we introduced in Section 4.2. Moreover, the solution of the approximate problem (4.22) is equivalent to finding a minimizer θ * ∈ Θ solution to min θ∈Θ J (ψ θ ).

Remark: In many machine learning applications, the choice of relu activation functions is very common due to its low computational cost when performing evaluation or first order differentiation. However, in our problem, second order derivatives are needed to calculate the loss function. If relu activation functions were used, then the second order derivative terms would be 0, and no good approximation could be learned. This reason motivates our choice of tanh as the activation function. In Figure B.2 some results concerning the choice of the architecture of the network are attached.

Sampling schemes

We detail in this section the various sampling schemes we considered in our numerical tests in order to define the approximate loss function J .

Since we work with one-dimensional examples, we carry the discussion for dimension one. In fact, we consider problem (4.1)-(4.2) with Ω = (0, 1) so that ∂Ω = {0} ∪ {1} (and ∂Ω ϵ = {0} ∪ {1/ϵ}). Thus, for all our tests, the domain boundary has M = 2 points y 1 = 0 and y 2 = 1 (or y 2 = 1 ϵ for the RWz method). Taking τ 1 = τ 2 = 1 for the surface weights, the surface term in (4.23) takes the simple form

M =2 m=1 τ m S(v)(y m ) = ∂Ω S(v) dτ ∀v ∈ V (or ∂Ωϵ S(v) dτ for the RWz formulation).
We consider three different sampling schemes for the approximation of the bulk term Ω R(v) dρ:

1. The first choice is a simple uniform sampling scheme (labeled -u in our tests). For a given K ∈ N * , we set ρ k = 1 K and (x k ) 1≤k≤K as the centers of the intervals given by a uniform discretization grid of the interval (0, 1).

2. The second sampling scheme, called random (-r) scheme, consists in choosing ρ k = 1 K and the points (x k ) 1≤k≤K as a collection of random points, identically independently distributed according to the uniform distribution on (0, 1).

3. We lastly consider a third sampling scheme, called exponential (-e) scheme, which is specific to the Wz formulation. Recall that in this case, for all v ∈ K, the expression of R(v) is given by (4.14), namely

R(v)(x) = R (1) (v)(x) + R (2) (v)(x), ∀x ∈ Ω with    R (1) (v)(x) := 1 2 |∇v(x)| 2 + ∆V (x) 2ϵ + |∇V (x)| 2 4ϵ 2 |v(x)| 2 R (2) (v)(x) := -f (x) e -V (x) 2ϵ ϵ v(x), ∀x ∈ Ω.
Thus, the bulk integral term:

Ω R(v)(x)dρ(x) = Ω R (1) (v)(x)dρ(x) + Ω R (2) (v)(x)dρ(x),
and we approximate each component separately as follows. For the first term, we draw a collection of K 1 ∈ N * independent identically distributed (iid) random points (x

k ) 1≤k≤K1 from the uniform distribution on (0, 1) and for all 1 ≤ k ≤ K 1 , the weights ρ [START_REF] Cichocki | Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[END_REF] k are chosen to be equal to 1 K1 . For the second term, we draw K 2 ∈ N * iid random points (x

Setting now ρ

(2) k = Zϵ K2 for all 1 ≤ k ≤ K 2 , the integral Ω R(v) is then approximated by

K1 k=1 ρ (1) k 1 2 |∇v(x (1)
k )| 2 + ∆V (x (1)
k ) 2ϵ + |∇V (x (1)
k )| 2 4ϵ 2 |v(x (1)
k )| 2 - K2 k=1 ρ (2) k f (x (2)
k )

1 ϵ v(x (2)
k ) .

In the following, we use the notation -u (respectively -r and -e), after the name of a formulation, in order to refer to the numerical method obtained by using this formulation, together with a uniform (respectively random or exponential) sampling scheme. For instance, the V -u method refers to the vanilla formulation used in conjunction with a uniform sampling scheme.

Note that to tackle higher-dimensional problems, special sampling techniques such as adaptive Markov-Chain Monte-Carlo or Quasi Monte Carlo would be required.

Comparison with finite element schemes

One important point in the investigation of the merits and limitations of deep learning-based numerical schemes is to understand how they compare with respect to other existing schemes. In our tests, we provide a numerical comparison with a vanilla finite element Galerkin scheme involving a uniform mesh. For the sake of completeness, we briefly recall the main steps of our finite element Galerkin approach.

Integrating the original equation We numerically solve this problem by Galerkin projection. For this, we consider a mesh (T n ) N n=1 of Ω and define the associated P 1 finite element space We then search for a solution u N ∈ V N ⊂ H 1 (Ω) by Galerkin projection, that is, we search for u N ∈ V N such that a(u N , v) = l(v), ∀v ∈ V N .

We next take as a basis of V N the set of tent functions defined as φ i (x j ) = δ ij , for 1 ≤ i, j ≤ N, and we express the solution as u N = N i=1 c i φ i . Gathering the expansion coefficients in the vector c = (c i ) N i=1 , and injecting the expansion of u N in the variational formulation, we are led to the system of equations M c = q where M = (M i,j ) 1≤i,j≤N , M i,j := a(φ i , φ j ), and q = (q i ) N i=1 , q i := l(φ i ).

Numerical Results

Test case and comparison criteria

In this section we show the results obtained by approximating the exact solution of the problem described in equation (4.1) using the methods introduced above.

Here, we work in the case when d = 1, Ω = (0, 1), and F , f are assumed to be equal to some constant real numbers. Then, the solution of (4.1)-(4.2) has an analytic expression which is given hereafter. Let us also introduce g 0 , g 1 ∈ R so that g(0) = g 0 and g(1) = g 1 . The problem then reads as follows: find u : (0, 1) → R solution to    -ϵu ′′ (x) + F u ′ (x) = f, ∀x ∈ (0, 1), -αu ′ (0) + κu(0) = g 0 , αu ′ (1) + κu(1) = g 1 .

(4.29)

Then, it can be easily checked that the solution to this equation reads as

u(x) = C 1 + C 2 e F x ϵ + f F x
where C 1 and C 2 are constants that are determined with the Robin boundary conditions. They satisfy the system κ κ -αF ϵ κ κe

F ϵ + α F ϵ e F ϵ C 1 C 2 = g 0 + α f F g 1 -f F (κ + α) which is invertible except for κ = 0, or α κ = ϵ(1 -e F ϵ ) F (1 + e F ϵ )
.

In the following, the values of κ and α are always chosen so that the above system is invertible.

In the numerical tests presented below, we fix F = 1, f = 1. We choose Robin boundary conditions that mimic Dirichlet conditions and we set α = 10 -3 , κ = 1, g 0 = g 1 = 0. Note that we cannot take α = 0 since all variational methods are not well defined for pure Dirichlet boundary conditions. With these choices, the equation reads    -ϵu ′′ (x) + u ′ (x) = 1, ∀x ∈ (0, 1), -10 -3 u ′ (0) + u(0) = 0, 10 -3 u ′ (1) + u(1) = 0.

(4.30)

Since the exact solution has an analytic form, we can thus easily compare the approximation quality of the output functions û from our methods by computing a discrete version of their L 2 (Ω) error norm with respect to the exact solution:

e 2 L 2 := ∥u -û∥ 2 L 2 (Ω) ≈ 1 K K-1 k=0 (u(x k ) -û(x k )) 2 =: e 2 ℓ 2 .
The points x k are sampled uniformly as defined in 4.3.3. We use 10 times more points than the ones used for approximating the integral, so K = 10K. Similarly, we also compute the error with respect to the H 1 (Ω) semi-norm:

e 2 H 1 := ∥u ′ -û′ ∥ 2 L 2 (Ω) ≈ 1 K K-1 k=0 (u ′ (x k ) -û′ (x k )) 2 =: e 2 h 1 .
Note that one can obtain the H 1 error by adding the above error components.

We study the impact on the errors of the following parameters:

• The values of ϵ. They range from 5 • 10 -3 to 10.0 with a logarithmic spacing.

• The number K of training points (or collocation points). We consider K = 10, 10 2 , 10 3 , 10 4 .

• The choice of the sampling method for the training points (uniformly spaced or uniformly random, labelled as -u and -r).

• The impact of the machine precision (Float16, Float32, Float64).

Due to the randomness in the initialization of weights on the neural networks, for each combination of parameters (ϵ, K, sampling type, and machine precision), we perform 10 repetitions with different initialization. Since we didn't notice a big difference between the l 2 error and the h 1 error, we keep just the second one for clarity and put in the Appendix B.1 the plots in l 2 error. Moreover, in the Section B.3 of the Appendix, some results concerning the training of the NN.

Our code and practical implementation details

All our neural network based numerical tests were performed in Python 3.6 and using the TensorFlow 1.13.1 library [START_REF] Abadi | Tensorflow: Large-scale machine learning on heterogeneous distributed systems[END_REF]. The code provided in the original paper on PINNs [START_REF] Kharazmi | Variational physics-informed neural networks for solving partial differential equations[END_REF] was used as the starting point for our own code developments, and we have followed similar guidelines to generalize and enlarge it where needed. In the same way, for each numerical method, derivatives of functions v ∈ K are computed using automatic differentiation. The numerical optimization procedure used in order to compute an approximation of θ * a minimizer of problem (4.26) is given by the quasi-Newton L-BFGS algorithm [START_REF] Nocedal | On the limited memory bfgs method for large scale optimization[END_REF]. The code used to generate the examples shown here is available at https://github.com/agussomacal/ConDiPINN

The interested reader can reproduce our results and test the impact of the variations of certain parameters such as ϵ, K, the sampling method, and the machine precision.

Discussion

Impact of the number K of training points

In this section we discuss the impact of the number K of training points. We fix the machine precision to Float32, and the uniform sampling -u. From these figures, we first notice that the approximation of FEM degrades when ϵ decreases. However, the accuracy improves when the number of discretization points increases (see, e.g., Figure 4.2 -left plot). The rate of improvement is linear as we can see from the right plot in Figure 4.2, as expected. In addition, when looking at the runtimes (Figure 4.2 -right) we observe the expected linear increase with respect to the number K of discretization points.

We can next study the behavior of Vanilla PINN and compare to FEM. We observe that it performs at an almost constant accuracy for any number of training points until around ϵ = 0.027 where it stops producing reliable approximations (see Figure 4.1). One remarkable observation is that the Vanilla PINN error for large values of ϵ and small number of training points K = 10 is comparable to the FEM errors with a much larger number of degrees of freedom K > 10 3 (see left plot in Figure 4.2). However, we observe that the running times of FEM computations remain much lower than the ones of PINN-based methods (right plot in Figure 4.2). Note that the low runtime of the FEM approach is due to the fact that the associated resulting linear system is tridiagonal, and this allows to solve with a linear cost w.r.t. the number of degrees of freedom.

We next comment on the other PINN-based variational methods. For ϵ large enough, we observe that all the variational based methods follow the same error trend as FEM both with respect to ϵ and K and for K < 10 4 they even perform marginally better. With respect to the computing time, all the methods perform with almost constant time with respect to K and similarly to a FEM method with K = 100 degrees of freedom. However, for ϵ < 0.63, the methods W -z, W -z -e and V -z blow up and lose completely their approximation capabilities. We conjecture that this is due to the fact that the neural network is used to approximate the solution z from the transformed problem, and there is an exponential term to go back from z to u (see equation (4.7)). This may lead to machine precision overflows (in the exponential computation) and underflows (the neural network has to learn very small values of z which also are in the limits of precision). To address this issue, we have explored two possible strategies: one was by directly minimizing over u while maintaining the weak formulation which accounts for the method W . The second approach is to perform the re-scaling of the domain RW -z. In both cases the blow up caused by the exponential is solved although the re-scaling method RW -z doesn't perform as good as others in the region with large ϵ values.

We finish this section by plotting in Figure 4.3 the best approximated solution for each model, and different values of ϵ. The interested reader may experiment other configurations in our provided code. The most striking observation is that only FEM and the vanilla PINN method recover the final shape of the exact solution when ϵ is small. The other variational PINN methods fail despite that some of them exhibit comparable values to FEM in the generalization errors as Figure 4.1 illustrates. This observations suggests that perhaps other types of error metrics should be introduced in order to be able to better distinguish between "good solution shapes" and "bad ones".

Impact of Machine Precision

Figure 4.4 shows the h 1 -error of the different approximated solution by changing the machine precision in the parameters of the neural networks for the different values of ϵ: Float16, Float32 and Float64. There is an improvement when going from Float16 to Float32 in all methods. Interestingly, we did not obtain very satisfactory results when working with Float64 precision. This precision seems to difficult the convergence to good quality minima: even after 10 repetitions, we failed to find good results. However, as the plots show, when a good minimum is found, it delivers slightly better approximation than lower machine precisions. For these reasons we have performed our experiments using the Float32 which seemed the most stable choice.

Impact of Sampling Strategy

.5 shows the h 1 -error of the different approximated solution by changing the sampling strategy. For all models, the unif orm strategy is found to be either as good as the random or slightly better. For this reason we performed all the experiments using the unif orm strategy.

Conclusions from the numerical experiments

The above numerical experiments depict a contrasted landscape concerning the merits and limitations of deep learning-based approaches when the solutions become low regular:

• For large values of ϵ when solutions are rather regular, some PINNs perform clearly better than FEM regarding the generalization errors. The superiority is particularly remarkable for very small number K of training points. However, the shapes of PINN solutions are sometimes not as satisfactory as the ones given by FEM.

• For the challenging case where ϵ becomes small and solutions become less regular (which was the main motivation of our study), the accuracy of the variational neural-network methods is essentially comparable or worse to the one given by FEM in terms of generalization errors. Some PINN variational approaches become too unstable and the errors blow up. Only FEM and the vanilla PINN approach seem to be able to recover the correct shape of the exact function. The latter one has however the risk of sometimes falling into local minima with bad shapes.

• The runtimes are clearly in favor to FEM methods as Figure 4.2 illustrates, but the simplicity of implementation is in favor to all PINN methods.

Future research directions and extensions

One important point to explore in future works concerns the choice of the loss function for the training, and also the metric to evaluate generalization errors. It will also be interesting to explore if adaptive sampling strategies during the training could help to recover good solutions in a more stable manner. Finally, the impact of the machine precision in some steps involving exponential transformations seems also to be an important obstacle to retrieving stable solutions. It would be interesting to develop strategies that circumvent this issue. All these developments will play a crucial role in order to address higher dimensional problems with similar characteristics as the one considered here.

Find in Figure B.2 in the Appendix a comparison between different architectures (depth and width) and the activation function of the network that have been tested. In the future, a possible improvement would be test more architectures in order to obtain better results.

We also refer the reader to

Conclusions and perspectives

In the present work, we propose and study three numerical methods to approximate functions from different perspectives.

1. A tensor approximation method, Sum of Tensor Trains (SoTT), whose output is the approximated solution in a sum of TTs format.

2. A local tensor approximation method based on clustering, that retrieves an approximated solution in the introduced local HOSVD format.

3. A deep learning-based method that computes the approximation of the solution of a convection diffusion PDE when the diffusion parameter is very small.

Conclusions of SoTT

In the first chapter of the thesis, we proposed a method to compress a given tensor as a sum of Tensor Trains (SoTT). Neither the order of the variables nor the ranks are fixed a priori. They are the result of an optimization step. A particular instance of this method, consisting in fixing the ranks equal to one in all the steps of the algorithm, produces a CP approximation of a given tensor, we have introduced it as CP-TT. A proof of convergence is proposed in the general case of the SoTT algorithm, which can be extended to the case of the CP-TT algorithm. Moreover, several numerical experiments are proposed in order to illustrate the properties of the methods:

• First, we compared the CP-TT to other rank-one update methods (ALS, ASVD, TTr1). Although a single iteration of CP-TT is more expensive in terms of number of operations, its stability makes it a promising candidate to compress high-dimensional tensors in CP format.

• Then, we did some tests in which we compressed the numerical solution of a parametric partial differential equation of reaction-diffusion type as well as other functions coming from different applications.

In particular, we compared SoTT with the TT-SVD obtained by testing all the possible permutations of the indices. Although SoTT is suboptimal with respect to the best TT-SVD, it is independent of the order of the variables and its performances are comparable to the average TT-SVD. In this test, the SoTT method outperforms CP-TT.

Both methods showed preliminary yet encouraging results in view of applications in scientific computing and compression of high order tensors. The method presented shows some shortcomings, to be addressed in further investigations: while a greedy method is appealing in view of computational tasks in which fixing the rank a priori could be cumbersome, it might be featured by a saturation effect, slowing down its convergence.

Conclusions of local HOSVD method

In the second chapter of the thesis, a method to compress tensors is proposed. In this case, the approximation is done locally and it is computed by the HOSVD method in a local subdomain of the original domain. The 99 subdomain is obtained automatically with a clustering method, that exploits the separability of the domain to compute partitions of the domain per direction. The different combinations of these partitions provide local subdomains. The HOSVD approximation of the tensor restricted to the local subdomain is computed. Among the partitions obtained, an extensive search of the subdomain that provides the smaller compression ratio with respect to the classical HOSVD method (in all the domain) is chosen to approximate the function. To the approximation of the tensor with the HOSVD method in this partition of the domain we have called local HOSVD method.

The process is computationally expensive and that is the reason why the numerical tests have been restricted to moderate order tensors. Nevertheless, some numerical tests for different applications and with different input formats have been performed and the results are optimistic.

• The partitions of the domain obtained with the clustering method are able to detect symmetry or periodicity in the system.

• An advantage in memory storage with respect to the HOSVD method is observed. This becomes more remarkable when the tensor we want to approximate presents different regimes.

Some other alternatives of how to select the subspace in which we compute the approximation have been explored but the results were not as encouraging as the ones obtained with the extensive search. An approach to the extensive search reducing the computational cost is a subject under study and it will be addressed in further investigations.

Moreover, there are other alternatives in order to use the idea of computing local approximations produced by the proposed clustering method. Once the clustering part is finished and the partitions of the domains per direction have been obtained, the objects obtained as an output can be used to construct a local tensor approximations with other format different to HOSVD. This idea is still under study but we are encouraged about it.

Conclusions of Deep Learning-based schemes

In the third chapter of the thesis, the task of approximating a multivariate function is tackled from a deep learning based method. The function to approximate is the solution of a convection-diffusion equation, known to be difficult to solve with the classical methods like Finite Element Method (FEM) when the diffusion coefficient ϵ adopts small values. Some experiments are done in order to see how different parameters affect the neural network (the format of the equation input, number of training points, etc). The experiments done, depict a contrasted landscape concerning the merits and limitations of deep learning-based approaches when the solutions become low regular:

• For large values of ϵ when solutions are rather regular, some PINNs perform clearly better than FEM regarding the generalization errors. The superiority is particularly remarkable for very small number K of training points. However, the shapes of PINN solutions are sometimes not as satisfactory as the ones given by FEM.

• For the challenging case where ϵ becomes small and solutions become less regular (which was the main motivation of our study), the accuracy of the variational neural-network methods is essentially comparable or worse to the one given by FEM in terms of generalization errors. Some PINN variational approaches become too unstable and the errors blow up. Only FEM and the vanilla PINN approach seem to be able to recover the correct shape of the exact function. The latter one has however the risk of sometimes falling into local minima with bad shapes.

• The runtimes and simplicity of implementation are clearly in favor to PINN methods.

In future works, a study of the choice of the loss function for the training and the metric to evaluate generalization errors will be performed. It will also be interesting to explore if adaptive sampling strategies during the training could help to recover good solutions in a more stable manner. Finally, the impact of the machine precision in some steps involving exponential transformations seems also to be an important obstacle to retrieving stable solutions. It would be interesting to develop strategies that circumvent this issue. We refer the reader to Appendix B to see other points in which the method can improve. Mean and standard deviation of the L 2 norm of the difference between the exact function W and its approximation given by ALS (red), ASVD (blue) and CP-TT (black) as a function of the number of terms. See Table A

FF 2 (

 2 2 (D) by ⟨F, G⟩ D := D (x)G(x)dx and ∥F ∥ L 2 (D) := D

{4}Figure 1 . 1 :

 11 Figure 1.1: In this schematic trees we can see two 4-dimensional examples of dimension trees. In the right, a hierarchical binary tree, H-Tucker. In the left the representation of the TT format as a binary dimension tree.

  R d are the n-ranks, b are the modes and G is the tensor core.

  ℓ 2 norm of the rank-r truncated POD of W n 0 and increases with r; on the other hand, β n i,1 r is a term which reflects the memory need related to the storage of a rank-r truncated POD of W n 0 . An integer

  k d ) 1≤k1≤l1,••• ,1≤k d ≤l d are randomly chosen as follows: Let β > 0. Let (α k1...k d ) 1≤k1≤l1,...,1≤k d ≤l d be a family of independent random variables uniformly distributed in [-1, 1]. For Algorithm 8 CP-TT algorithm

  Similar numerical tests have been performed in the case where the value of the parameter β is chosen to be equal to d 2 +1.1. We refer the reader to the Appendix A.2 to see them in more detail. The results obtained m of the esidual Dec easing of the no m of the esidual. Mean and Std fo the diffe ent methods.

Figure 2 . 1 :

 21 Figure 2.1: Case d = 4 and β = d 2 + 0.1.Mean and standard deviation of the L 2 norm of the difference between the exact function W and its approximation given by ALS (red), TTr1 (blue), CP-TT (black) and ASVD (green) as a function of the number of terms. See Table2.1 for more detailed information.

  m of the esidual Dec easing of the no m of the esidual. Mean and Std fo the diffe ent methods. m of the esidual Dec easing of the no m of the esidual. Mean and Std fo the diffe ent methods. Mean of ASVD Mean of CPTT Mean of ALS Std of ASVD Std of CPTT Std of ALS

Figure 2 . 2 :

 22 Figure 2.2: Case β = d 2 + 0.1. Mean and standard deviation of the L 2 norm of the difference between the exact function W and its approximation given by ALS (red), ASVD (blue) and CP-TT (black) as a function of the number of terms. Left: case d = 12. Right: case d = 16. See Table2.1 for more detailed information.

Figure 2 . 3 :

 23 Figure 2.3: Functions with β = d 2 + 0.1. From left to right in the top d = 4, 12. In the bottom: d = 16. Mean and standard deviation of the norm of the residual as a function of the accumulated time of computation for ALS (red), ASVD (blue) and CP-TT (black).

Figure 2 . 6 :

 26 Figure 2.6: Compression test performed in Section2.5.2, double logarithmic plot of the memory as function of the residual norm for the SoTT, the CP-TT and ALS algorithms. For the rank-1 update methods we plotted the result every 16 iterations for the sake of clarity in the graphical representation.

Figure 2 . 7 :

 27 Figure 2.7: Cantilever beam case, presented in Section 2.5.2, memory as function of the residual norm, in logarithmic scale for: a) and b).

Figure 2 . 8 :

 28 Figure 2.8: Friedman function case, presented in Section 2.5.2, memory as function of the residual norm, in logarithmic scale for: a) and b).

Figure 2 . 9 :

 29 Figure 2.9: OTL circuit case, presented in Section 2.5.2, memory as function of the residual norm, in logarithmic scale for: a) SoTT and the 720 possible TT-SVD, for 3 different values of tolerance, b) ALS, CP-TT and SoTT methods.

3 . 4

 34 Computing local HOSVD method . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Cost and complexity of the algorithm . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.1 Compression of a Gaussian function . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.2 Compression of the solutions of the Fitz-Hugh-Nagumo equation . . . . . . . . . . 3.8 Conclusions and perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 ,

 1 and the ones obtained by dividing x 2 , Ω

1 Figure 3 . 1 : 2 -

 1312 Figure 3.1: 2-dimensional schematic example of a possible partition of the domain with respect to its 2 directions.

1 Figure 3 . 2 : 2 -

 1322 Figure 3.2: 2-dimensional schematic example of the combination of the partitions of Figure 3.1.

d

  k=1 l k possible combinations of partitions is what makes this alternative very costly and in some situations not feasible.

Figure 3 . 3 :

 33 Figure 3.3: In the left an example of the partition tree in the direction k with N k,0 = 4 and in parallel on the right, the evolution of the partitions of the domain with the mergings.

  the green domain Ω 3 and the red domain Ω 4 is given by J

  Cr = Memory(F locHOSV D ) Memory(F ) Let us show in Figure 3.4 the compression rate for different values of the tolerance of the approximation of the tensor computed by HOSVD method and the local HOSVD alternative. The values from the tolerance in this test hold ϵ = {1.0 • 10 -2 , ϵ = 1.0 • 10 -3 , ϵ = 1.0 • 10 -4 }. And the values for the time resolution N t = {200, 400, 600, 800, 1000}.

Figure 3 . 4 :

 34 Figure 3.4: The compression rate for the function computed by the local HOSVD approximation method (in orange) and also the compression rate for HOSVD method (in blue). From left to right, the tolerance: ϵ = 1.0 • 10 -2 , ϵ = 1.0 • 10 -3 and ϵ = 1.0 • 10 -4 . In all of them the resolution in space is fixed to N x = 500 and in time N t = 200, 400, 600, 800, 1000 points.

Figure 3 . 5 :

 35 Figure 3.5: Local reconstruction for the Gaussian function (in orange) in comparison with the HOSVD (in blue) and the analytic expression (in yellow) for an error of ϵ = 1.0 • 10 -2 and resolution N x = 500 and N t = 400. In Figure 3.6, the solution of the transport function is shown for the same values of error and resolution. The value for the error is ϵ = 1.0 • 10 -2 and for the resolution the values taken are N x = 500 and N t = 400.The exact solution of the Equation3.17 is shown, as well as its reconstructions by local HOSVD and HOSVD methods.

Figure 3 . 6 :

 36 Figure 3.6: Reconstruction of the exact solution with an error of ϵ = 1.0 • 10 -2 and resolution N x = 500 and N t = 400. In the left, the reconstruction of the exact solution is computed with the local HOSVD method and in the right with the HOSVD method.

  y the Laplace operator. The domain is defined as Ω = ([0, 1] × [0, 1], T ). Being T the final time whose value change depending on the test performed. The boundary conditions have been set as Neumann homogeneous boundary conditions ( zero-flux boundary conditions.): ∂u(x,t) ∂⃗ n = 0 in ∂Ω.

Figure 3 . 7 :

 37 Figure 3.7: Solution of the Fitz-Hugh-Nagumo equation, 3.18, for N x = 256 × 256 square and time step equal to 2, N t = 400. The initial conditions are set as: 1 for x ∈ ([0, 0.5] × [0, 0.5]). The values for the parameters k = 1 and ϵ = 0.005.

Figure 3 . 8 :

 38 Figure 3.8: One of the 2-dimensional basis obtained for 3.18 with respect to the variable x. From left to right, the first and the second modes of the basis. For N x = 256 × 256 square and time step equal to 2. The values for the parameters are: 2 values of k = [0.96, 0.98] and 5 values of ϵ with ϵ ∈ [0.0050, 0.00375].

Figure 3 . 9 :

 39 Figure 3.9: From left to right, three different time consecutive captures of the initial condition square moving to the right and vanishing into a vertical structure that will originate the spiral shape. The simulation is produced for final time T = 200.

Figure 3 . 10 :

 310 Figure 3.10: Three different consecutive partitions of the spatial domain. Each partition is colored different. From left to right, the number of color blocks is reduced by one. Plots for T = 200. These partitions show the domains that represent the sequence of Figure 3.9

  , let us see how the values for the memory have changed if we change the

Figure 3 . 11 : 10

 31110 Figure 3.11: Three different consecutive partitions of the spatial domain. Each partition is colored differently. From left to right, the number of color blocks is reduced by one. Plots for T = 800. T ϵ Best Memory Mem POD/ Best Mem Best # clusters (x-t) 5 • 10 -2 239082 3,30 1-19 200 1 • 10 -2 305741 5,59 1-20 5 • 10 -3 315965 6,45 1-181 • 10 -3 459811 6,43 1-30 5 • 10 -4 497865 6,87 1-70 1 • 10 -4 604648 7,28 1-42 5 • 10 -2 204283 2,16 1-29 400 1 • 10 -2 483252 6,55 1-42 5 • 10 -3 535816 7,14 1-30 1 • 10 -3 697919 8,03 1-44 5 • 10 -4 884585 7,23 1-30 1 • 10 -4 1017435 8,17 1-69 5 • 10 -2 1605626 2,02 1-28 800 1 • 10 -2 2410667 2,96 1-60 5 • 10 -3 2450132 3,46 1-60 1 • 10 -3 3367752 3,51 1-68 5 • 10 -4 2952861 5,22 1-50 1 • 10 -4 3022160 6,15 1-50

( 4 . 1 ) 1 Ω

 411 against a sufficiently smooth function v ∈ C ∞ (Ω), and integrating by parts, it follows that a weak formulation of problem (4.1) is to findu ∈ H 1 (Ω) such that a(u, v) = l(v), ∀v ∈ H 1 (Ω) with a(u, v) = Ω ∇u • ∇v + ϵ -

V

  N := {v ∈ C 0 (R) : ∀0 ≤ s ≤ N -1, v| [xs,xs+1) ∈ P 1 ([x s , x s+1 ])} with P 1 ([x s , x s+1 ]) := {v : [x s , x s+1 ] → R, v(x) = ax + b, (a, b) ∈ R 2 }.

Figure 4 .

 4 [START_REF] Cichocki | Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[END_REF] shows the best result obtained in the tests, i.e., the minimum value of the h 1 norm obtained in the 10 different simulations, plotted against the values of ϵ. In Figure4.2, we fix ϵ = 10, and plot statistics on the accuracy e h 1 (left plot) and computation runtimes for different K (right plot), and for the different methods. Find in the Appendix in Section B.1 the equivalent plot for the l 2 error.

Figure 4 . 1 :

 41 Figure 4.1: Comparison of the behavior of the h 1 error for the different methods and different number of sampling points in training. From top to bottom and from left to right, the first figure is produced for K = 10, the second for K = 100, the third for K = 1000 and the last one for K = 10000. The set of points to train and test have been chosen with the uniform sampling method. The precision has been fixed to Float32 for all the tests.

Figure 4 . 2 :

 42 Figure 4.2: For ϵ = 10 (region where all methods work well), we look at the comparison between methods and the difference with respect to the number of training points K. The h 1 error (left) and the computation times (right). The set of points to train and test have been chosen with the uniform sampling method. The precision has been chosen as Float32 for all the tests.

Figure 4 . 3 :

 43 Figure 4.3: The best approximated solution out of 10 repetitions, for each model, and with K = 100 training samples. From left to right: ε = 0.039, 0.18, 10. The interested reader may experiment other configurations in our provided code.

Figure 4 . 4 :

 44 Figure 4.4: Here, the comparison of the behavior of the model for different float precision. The tests have been performed for K = 100 and uniform sampling.

Figure 4 . 5 :

 45 Figure 4.5: Here, the comparison of the behavior of the model for the two different sampling strategies. The tests have been performed for K = 100 and the float precision equal to Float32.

  Figure B.3 and Figure B.4 to see the number of iterations that the network takes to converge.

Figure A. 1 :

 1 Figure A.1: Case β = d 2 + 1.1.Mean and standard deviation of the L 2 norm of the difference between the exact function W and its approximation given by ALS (red), ASVD (blue) and CP-TT (black) as a function of the number of terms. See TableA.1 for more detailed information.

Figure B. 3 :

 3 Figure B.3: For ϵ = 0.014 (left) and ϵ = 0.108 (right) we see the approximation obtained by Vanilla PINN with K = 100. In the left the local minimum that was found correspond to a straight line that plays a trade-off between minimizing the residuals (with ∂u ∂ 2 x = 0 and ∂u ∂x ≈ 1 = f ) and the loss in the boundary conditions.

Figure B. 4 :

 4 Figure B.4: Each curve correspond to the evolution of the train loss for a realization Vanilla PINN for different choices of ϵ and K = 100.

  belongs to the family of instance-based learning algorithms, which store training data in memory and compare test data with training examples to make predictions. The label for a given observation is obtained by a majority vote among the k-nearest training examples, understanding nearest as the smallest Euclidean distance.

  .2, let us show the possible architecture of a feedforward neural network with one hidden layer composed by n neurons and input and output dimension d X and d Y , respectively.

		Input	Hidden	Ouput
		layer	layer	layer
	x 1
			h 1
	x 2				y 1
	x 3	. . .	. . . h n	. . .	y d Y
	x d X
	Figure 1.2: Schematic example of the architecture of a feedforward neural network with one hidden layer
	composed by n neurons and input dimension d X and output dimension d Y .

Trains: SoTT Contents 2.1 Introduction

  Object definition and POD decomposition . . . . . . . . . . . . . . . . . . . . . . . 32 2.2.3 Ranks and tensor formats: Canonical Polyadic (CP) and Tensor Train (TT) . . . . 33 2

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.1.1 Organization of the chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.2 Notation and preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.2.1 Tensor spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.2.2

.3 The Sum of Tensor Trains (SoTT) algorithm

  Exponential convergence of the SoTT algorithm in finite dimension . . . . . . . . . 40 2.3.3 Complexity estimate of the SoTT algorithm . . . . . . . . . . . . . . . . . . . . . . 41 2

. . . . . . . . . . . . . . . . . . . . 36 2.3.1 Presentation of the SoTT algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.3.2

.4 CP-TT: fixed-rank SoTT algorithm with rank

1 . . . . . . . . . . . . . . . . . . 42 2.5 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.5.1 Comparison between CP-TT and other rank-one update methods . . . . . . . . . . 43 2.5.2 SoTT method for the compression of multivariate functions . . . . . . . . . . . . . 47 2.6 Conclusions and perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

  .1)A decomposition of W under the form (2.1) is called a Singular Value Decomposition (SVD) (or Proper Orthogonal Decomposition (POD)) of W according to the separation of variables (D x , D y ). The sequence (σ k ) k∈N * is known to be unique and is called the sequence of singular values of W associated to the separation of variables (D x , D y ) of the set D. The orthonormal basis (U k ) k∈N * (respectively (V k ) k∈N * ) may not be unique but is called a sequence of left (respectively right) singular vectors of W associated to this partitioning.For any set E, we denote in the sequel by #E the cardinality of E. Assuming that N x := #D x < +∞ and N y := #D y < +∞, the complexity of the computation of an SVD decomposition of the form (2.1) scales like O max(N x , N y ) min(N x , N y ) 2 .

	(2.2)

Table 2 .

 2

	Std

1: Mean and standard deviation of the norm of the residual for 32 random functions in the case where β = d 2 + 0.1. For TTr1, due to memory issues, it wasn't possible to obtain definite results for order higher than four.

Table 3 .

 3 1: Comparison between the memory obtained when computing the POD and the memory computed for the best of the possible partitions of the domain for different values of the error tolerance and different resolutions in the domain.

	(N x , N t )	ϵ	Best Memory Mem POD/ Best Mem Best # clusters (x-t)
		5 • 10 -2	4576	2,07	3-4
	(250,200) 1 • 10 -2	5015	2,43	7-9
		5 • 10 -3	6087	2,15	12-15
		1 • 10 -3	6866	2,23	8-8
		5 • 10 -4	7326	2,21	6-8
		1 • 10 -4	8438	2,13	6-8
		5 • 10 -2	6477	2,92	6-7
	(500,400) 1 • 10 -2	9433	2,58	9-9
		5 • 10 -3	12013	2,17	9-13
		1 • 10 -3	13894	2,20	11-17
		5 • 10 -4	14931	2,17	6-8
		1 • 10 -4	18144	1,98	7-8
		5 • 10 -2	9768	2,90	7-8
	(750,600) 1 • 10 -2	12806	2,85	10-10
		5 • 10 -3	15899	2,46	8-10
		1 • 10 -3	23477	1,96	11-14
		5 • 10 -4	21883	2,22	7-11
		1 • 10 -4	25134	2,15	8-9
		5 • 10 -2	14559	2,60	6-6
	(1000,800) 1 • 10 -2	17367	2,80	7-9
		5 • 10 -3	19723	2,65	9-10
		1 • 10 -3	29826	2,05	7-7
		5 • 10 -4	34050	1,90	11-13
		1 • 10 -4	35581	2,02	7-10

Table 3 .

 3 2: Comparison between the memory obtained when computing the POD and the memory computed for the best of the possible partitions of the domain for different values of the error tolerance and different resolutions in the domain. In this case the final time T = N t , i.e. what we are reconstructing are parts of the whole simulation with a times tep n t = 1. The resolution in x, is N x = 256 × 256.

Table 3 .

 3 3: Comparison between the memory obtained when computing the POD and the memory computed for the best of the possible partitions of the domain for different values of the error tolerance and different resolutions in the domain. In this case the final time T = 800 and the resolution N t = {200, 400, 800}, i.e. what we are reconstructing is the whole simulation changing the time step.

	(N x , N t )	ϵ	Best Memory Mem POD/ Best Mem Best # clusters (x-t)
		5 • 10 -2	1258991	2,51	1-46
	(256 × 256,200) 1 • 10 -2	1431429	3,48	1-50
		5 • 10 -3	1571165	4,98	1-50
		1 • 10 -3	2334709	4,67	1-50
		5 • 10 -4	2424712	4,56	1-50
		1 • 10 -4	3204997	4,02	1-50
		5 • 10 -2	1392122	2,27	1-30
	(256 × 256,400) 1 • 10 -2	1528037	3,15	1-30
		5 • 10 -3	1859227	4,27	1-50
		1 • 10 -3	1893692	6,16	1-50
		5 • 10 -4	1942698	6,07	1-80
		1 • 10 -4	3915820	4,39	1-70
		5 • 10 -2	1605626	2,02	1-28
	(256 × 256,800) 1 • 10 -2	2410667	2,96	1-60
		5 • 10 -3	2450132	3,46	1-60
		1 • 10 -3	3367752	3,51	1-68
		5 • 10 -4	2952861	5,22	1-70
		1 • 10 -4	3022160	6,15	1-70

  (ϵy) for all y ∈ Ω ϵ , it holds that u is solution to (4.1)-(4.2) if and only if u is solution to-∆ u(y) + ∇ • F u (y) = f (y), ∀y ∈ Ω ϵ (4.18)where f (y) := f (ϵy) for all y ∈ Ω ϵ with boundary conditions for all y ∈ Ω ϵ . Using similar calculations to the ones done in Section 4.2.4, the Lax-Milgram theorem guarantees that z is the unique solution in H 1 (Ω ϵ ) of the following variational problem: for all

	α(∇ u • n)(y) + ϵκ u(y) = g(y), ∀y ∈ ∂Ω ϵ ,	(4.19)
	with g(y) := g(ϵy)

.17)

Notice that if u and z satisfy (4.7), then

z(y) = u(y)e 1 2 V (y) , ∀y ∈ Ω ϵ .

Denoting by F (y) := -∇ V (y) = F

Table A .

 A .1 for more detailed information. 1: Mean and standard deviation of the norm of the residual for 32 random functions in the case where β = d 2 + 1.1.

				Mean		Std
	Dimension (d) Rank (r)	ALS	CPTT ASVD	ALS	CPTT ASVD
		25	0.1722 0.2261 0.1759 0.0643 0.2261 0.1759
	4	50	0.0572 0.1382 0.0590 0.0220 0.1382 0.0232
		75	0.0252 0.0948 0.0262 0.0103 0.0948 0.0110
		25	0.3741 0.2938 0.4171 0.1158 0.0942 0.1341
	6	50	0.2037 0.1507 0.2281 0.0655 0.0523 0.0791
		75	0.0851 0.0579 0.1045 0.0334 0.0233 0.0493
		25	0.3676 0.2560 0.3977 0.1361 0.0905 0.1517
	8	50	0.2136 0.1229 0.2413 0.0807 0.0451 0.1023
		75	0.1046 0.0455 0.1145 0.0437 0.0195 0.0631
		25	0.4574 0.3737 0.4753 0.1235 0.1548 0.1817
	10	50	0.2613 0.3483 0.3193 0.0809 0.1825 0.1648
		75	0.1168 0.3332 0.2352 0.0628 0.2034 0.1865
		25	0.4634 0.2505 0.5182 0.1681 0.0842 0.2116
	12	50	0.2889 0.1141 0.3922 0.1421 0.0384 0.2170
		75	0.1278 0.0382 0.3144 0.0671 0.0126 0.2502
		25	0.5943 0.2169 0.4386 0.2043 0.1262 0.2014
	14	50	0.2841 0.0779 0.3132 0.1277 0.0686 0.1915
		75	0.1422 0.0244 0.2021 0.0814 0.0227 0.2192
		25	0.4598 0.2460 0.5543 0.1496 0.0726 0.1603
	16	50	0.2861 0.1108 0.3936 0.1268 0.0348 0.2022
		75	0.1395 0.0438 0.3181 0.0552 0.0153 0.2477

1.3. STATE OF THE ART OF DEEP LEARNING-BASED SCHEMES

We consider the function as reported in http://www.sfu.ca/ ˜ssurjano/fried.html.

The function can be found in http://www.sfu.ca/ ˜ssurjano/otlcircuit.html.

Acknowledgments

I start acknowledging my thesis directors: Virginie Ehrlacher and Damiano Lombardi. Thank you for the inspiration, the example, the professional advice, and for all the things you have thought me.

In Figure 2.5, the memory of the computed approximation (i.e. the number of stored double precision numbers) is plotted as a function of the residual norm, for the TT-SVD approximations corresponding to all the possible 24 choices of permutations of the variable indices and the SoTT approximation. These approximations are computed in all cases for several residual tolerances, ranging from 10 -2 to 5•10 -4 . Remark here that the results obtained by the TT-SVD algorithm heavily depend on the order of the variables chosen. The difference in memory between the best and the worst TT-SVD is roughly one order of magnitude, for all the tolerances tested.

We observe in this test case that the SoTT method produces a sub-optimal compression with respect to the best TT-SVD compression. However, it performs better than the average TT-SVD and in general better than the canonical order 1, 2, 3, 4. The first term computed is a rank-1 update, for the second term the TT

Optimization of the coefficients

In this section we try to maximize the coefficients c in every term of the CP decomposition given by Equation 2.3, in order to have a better result. This is possible because we can add some small α such that |α| ≪ 1 to the coefficient c to reach a better approximation. The goal of the coefficient optimization is to retrieve the best projection on the subspaces selected by the algorithm. Slightly modifying the coefficients c that appear in every term of the CP decomposition, what we are doing is letting the parameters that define our result a small range of movement in order to make the approximation closer to the exact projection of the tensor in the subspace.

The optimum value of the CP coefficients is given by the following minimization problem:

Where F is the CP approximation of the tensor F given by Equation 2.3. It is possible to rewrite the expression as a minimization problem. Let us define the functional to minimize J(c) : L 2 (Ω) → R + as:

Then, by finding the minimum of the functional with respect to the coefficients, δJ δc (l) = 0, a linear system for the coefficients has to be solved. For j, l = 1, . . . , r, the system written by terms:

Where:

A.1.2 Computing the SVD of the unfolding without explicitly computing and assembling the unfolding

As we can see in Section 2.3.3, one of the most computationally expensive operations in the computation of a tensor approximation via the SoTT method is the one that computes the unfolding matrices of the tensor and does an SVD of them on each iteration. For this reason, we have developed a numerical scheme that computes the SVD of each one of the unfoldings needed in the SoTT method without computing explicitly all of them.

Let us explain this. By simplicity let us particularize for one of the dimensions of the tensor. Without loss of generality, let us choose the one that has been identified as the first. Let M ∈ R N1×N2...N d , be an unfolding of a certain tensor F with respect to the first one of the variables, M (1) . The CP form of F , defined in Equation 2.3 reads:

A first approximation of the tensor F is given by the projection of F into the subspace generated by the first modes of its SVD decomposition u

The expression above could be interpreted as a CP tensor decomposition if we consider a new set of coefficients for 1

Multiplying the matrix unfolding and its transpose M (1) M (1) T and using the change of variable in the coefficients

. . .

. . dx d

We can identify in the result the form T (1) KT (1) T , where K ∈ R r×r is a covariance matrix. By components defined as:

Its SVD is T (1) = Ū (1) ΣV (1) T , where we can also define Z (1) = ΣV (1) T . Then, M (1) M (1) T = T (1) KT (1) T = Ū (1) Z (1) KZ (1) T Ū (1) T If we identify the product K = Z (1) KZ (1) T = W ΛW T , being Λ the diagonal matrix of eigenvalues, we can compute the largest singular value and its associated singular vector of K, u (1) 1 = Ū (1) w (1) σ (1) = λ 1 By doing this computation, we can obtain all the information of the needed unfolding without explicitly computing it. In terms of the computational cost, instead of computing as much SVD decompositions as unfoldings, we can obtain directly the mode associated to the largest singular value of each one of them.

A.2 Results for functions in H1

Similar numerical tests have been performed to the ones in Section 2.5.1 for different regularity of the functions. In this case, the value of the parameter β is chosen to be β = d 2 + 1.1. The results on the compression of the multivariate function given in Equation 2.9 obtained for H 1 (Ω) functions are equivalent to the ones shown for L 2 (Ω) functions, in which the decrease of the error norm with the approximation rank is quite regular in CP-TT and behaves in a quite stable way also for higher order tensors.

Let us show in

B.2 Architecture of the NN plots

The choice of the architecture (depth and width) and activation function (tanh, sigmoid, etc) of the underlying neural network have an effect. For the present work, we decided that this investigation was not going to be our main focus, and that we wanted to concentrate mainly in the different formulations of PINNs. Note that we originally choose the activation function to be tanh because it was used in the original paper of PINNs. Our architecture with two hidden layers of twenty neurons each was fixed to have a trade-off between expressivity and computation time. To illustrate that the choice of architecture and activation function is not obvious, we have drawn the four plots of figure B.2 below. They show that increasing the depth effectively increases in general the computation time to arrive to the optimum, but the choice of width and the activation function doesn't seem to shed clear tendencies as to which one should be chosen.

B.3 Training of the PINN

As the neural networks are neither very deep nor very wide, the training typically doesn't take too much time. In general we have observed that for initializations that finish by converging to a good approximation the time spent in training is bigger than those in which the optimization got stuck into a local minima B.3 (left). But it can be otherwise too as shown in figure B.4.