Alice Saiteja

Yann

Thomas

Jonas Hadi

Sylvain, Daniel, Niklas Stefan Yann

Raphaël, Shuman, Wen Julian

Nicoco, Qiao Puis Pawel

Clément, Jaume, Tania, Benoît, Zihao, Gaëtan, Florent Luigi

Santi

Françoise Renaudat

Philippe Maisonobe

Ann Lemahieu

Isabelle Delorme

Er- Wan Aubry

Christian Pauly

François Delarue D'

Je remercie Sofía pour nos échanges passionnés sur la géométrie riemannienne et les espaces barycentriques, Bastien pour ses explications patientes sur la théorie des représentations et la topologie, Dimby pour nos discussions sur les connexions, ses questions profondes qui furent la source de grand tracas cependant fructueux et ses conseils sur le monde académique, Anna pour nos discussions sur les graphes, les arbres, les forêts et autres espaces stratifiés. Je tiens à les remercier ainsi qu'Elodie, Morten, James, Josquin, Luís et Tom pour nos échanges stimulants et pour l'ambiance très amicale de nos rendez-vous. Je remercie aussi les participants aux hackathons geomstats, particulièrement

Géométries riemanniennes et stratifiées des matrices de covariance et de corrélation

Dans de nombreuses applications, les données sont des matrices de covariance ou de corrélation entre plusieurs signaux (EEG, MEG, fMRI), grandeurs physiques (cellules, gènes) ou instants (autocorrélation). L'ensemble des matrices de covariance est un cône convexe qui est un espace stratifié non euclidien : il a un bord qui est lui-même un espace stratifié de dimension inférieure. Ses strates sont les variétés de matrices de covariance de rang fixé et la strate principale des matrices Symétriques Définies Positives (SPD) est dense dans l'espace total. L'ensemble des matrices de corrélations admet une structure similaire.

Les concepts géométriques comme les géodésiques, le transport parallèle ou la moyenne de Fréchet permettent de généraliser les opérations classiques (interpolation, extrapolation, recalage) et statistiques (moyenne, analyse en composantes principales, classification, régression) à ces espaces non linéaires. Cependant, ces généralisations reposent sur le choix d'une géométrie supposée connue à l'avance, c'est-à-dire d'un opérateur de base tel qu'une distance, une connexion affine, une métrique riemannienne, une divergence. En général il n'existe pas une unique géométrie adaptée à une application mais plutôt une famille de géométries à explorer pour faire ce choix. D'abord, la géométrie doit correspondre au problème. Par exemple, si les matrices de covariance doivent être inversibles, les matrices dégénérées doivent être rejetées à l'infini. Ensuite, elle doit satisfaire aux invariances naturelles du problème par des groupes de transformations : si multiplier chaque variable par un facteur indépendant n'a pas d'influence, alors il faut une métrique invariante par le groupe des matrices diagonales strictement positives, par exemple une métrique produit qui découple les échelles et les corrélations. Enfin, de bonnes propriétés numériques (formes closes, algorithmes efficaces) sont essentielles pour utiliser cette géométrie en pratique.

Dans ma thèse, j'étudie des géométries sur les matrices de covariance et de corrélation suivant ces principes. En particulier, je fournis les opérations géométriques associées qui sont les briques élémentaires pour calculer avec ces matrices.

Sur les matrices SPD, je m'inspire de la caractérisation des métriques affine-invariantes pour caractériser les métriques continues invariantes par O(n) au moyen de trois fonctions multivariées continues. Je construis ainsi une classification de métriques : les contraintes imposées sur ces fonctions définissent des classes emboîtées vérifiant des propriétés de stabilité. En particulier, je réinterprète la classe des "kernel metrics", j'introduis la famille des métriques "mixed-Euclidean" dont je calcule la courbure, et je résume et complète les connaissances sur les métriques classiques (log-euclidien, Bures-Wasserstein, BKM, power-Euclidean).

Sur les matrices de corrélation de rang plein, je calcule les opérations riemanniennes de la métrique quotient-affine et je montre que, malgré sa construction intéressante et son invariance par permutations, sa courbure est non majorée et de signe non constant, ce qui rend sa géométrie très complexe en pratique. Pour pallier ce défaut majeur, j'introduis des métriques Hadamard ou même log-euclidiennes ainsi que leurs opérations géométriques. Pour retrouver l'invariance par permutations perdue, je définis deux nouvelles métriques log-euclidiennes invariantes par permutations, l'une d'elle étant invariante par une involution naturelle de l'espace. Je fournis aussi un algorithme efficace pour calculer les opérations géométriques associées, qui s'appuie sur le "scaling" de matrices SPD.

Enfin, j'étudie la structure riemannienne stratifiée de la distance de Bures-Wasserstein sur les matrices de covariance. Je calcule le domaine de définition des géodésiques et le domaine d'injection dans chaque strate, puis je caractérise les courbes minimisant la longueur entre toutes les strates.

Mots clés: Géométrie riemannienne, Matrices de covariance, Matrices de corrélation, Familles de métriques, Géodésiques, Espaces stratifiés.

Riemannian and stratified geometries of covariance and correlation matrices

In many applications, the data can be represented by covariance matrices or correlation matrices between several signals (EEG, MEG, fMRI), physical quantities (cells, genes), or within a time window (autocorrelation). The set of covariance matrices forms a convex cone that is not a Euclidean space but a stratified space: it has a boundary which is itself a stratified space of lower dimension. The strata are the manifolds of covariance matrices of fixed rank and the main stratum of Symmetric Positive Definite (SPD) matrices is dense in the total space. The set of correlation matrices can be described similarly.

Geometric concepts such as geodesics, parallel transport, Fréchet mean were proposed for generalizing classical computations (interpolation, extrapolation, registration) and statistical analyses (mean, principal component analysis, classification, regression) to these non-linear spaces. However, these generalizations rely on the choice of a geometry, that is a basic operator such as a distance, an affine connection, a Riemannian metric, a divergence, which is assumed to be known beforehand. But in practice there is often not a unique natural geometry that suits the application. Thus, one should explore more general families of geometries that exploit the data properties.

First, the geometry must match the problem. For instance, degenerate matrices must be rejected to infinity whenever covariance matrices must be non-degenerate. Second, we should identify the invariance of the data under natural group transformations: if scaling each variable independently has no impact, then one needs a metric invariant under the positive diagonal group, for instance a product metric that decouples scales and correlations. Third, good numerical properties (closedform formulae, efficient algorithms) are essential to use the geometry in practice.

In my thesis, I study geometries on covariance and correlation matrices following these principles. In particular, I provide the associated geometric operations which are the building blocks for computing with such matrices.

On SPD matrices, by analogy with the characterization of affine-invariant metrics, I characterize the continuous metrics invariant by O(n) by means of three multivariate continuous functions. Thus, I build a classification of metrics: the constraints imposed on these functions define nested classes satisfying stability properties. In particular, I reinterpret the class of kernel metrics, I introduce the family of mixed-Euclidean metrics for which I compute the curvature, and I survey and complete the knowledge on the classical metrics (log-Euclidean, Bures-Wasserstein, BKM, power-Euclidean).

On full-rank correlation matrices, I compute the Riemannian operations of the quotient-affine metric. Despite its appealing construction and its invariance under permutations, I show that its curvature is of non-constant sign and unbounded from above, which makes this geometry practically very complex. I introduce computationally more convenient Hadamard or even log-Euclidean metrics, along with their geometric operations. To recover the lost invariance under permutations, I define two new permutation-invariant log-Euclidean metrics, one of them being invariant under a natural involution on full-rank correlation matrices. I also provide an efficient algorithm to compute the associated geometric operations based on the scaling of SPD matrices.

Finally, I study the stratified Riemannian structure of the Bures-Wasserstein distance on covariance matrices. I compute the domain of definition of geodesics and the injectivity domain within each stratum and I characterize the length-minimizing curves between all the strata.

Keywords: Riemannian geometry, Covariance matrices, Correlation matrices, Families of metrics, Geodesics, Stratified spaces.

Je voudrais adresser toute ma reconnaissance à mon directeur de thèse Xavier Pennec pour m'avoir accompagné et guidé pendant ces cinq années depuis mon premier stage jusqu'à l'aboutissement de cette thèse. Je me suis tout de suite senti en confiance tant scientifiquement qu'humainement, j'ai apprécié sa direction souple, c'est à dire le parfait équilibre entre d'un côté la présence infaillible, le suivi attentif et soucieux, la clarté des grandes lignes, la vision, et de l'autre la liberté laissée à l'expression personnelle jusqu'à l'humble effacement pour laisser l'oiseau voler de ses propres ailes. Ces qualités induisent un climat de confiance qui permet l'accomplissement personnel et le dépassement de soi. J'ai eu la chance de bénéficier de ce même climat de confiance avec mes parents, et à plusieurs reprises dans mon parcours, notamment à Centrale. Je suis heureux de l'avoir retrouvé ici auprès de Xavier et je l'en remercie infiniment. J'ai aussi apprécié les efforts constants pour nous emmener à de nombreux workshops et conférences, notamment Nicolas et moi au début, lorsque cela était possible, et plus généralement le fait de nous avoir généreusement présentés aux communautés de statistiques géométriques et cousines. La constitution de l'équipe G-Statistics au sein d'Epione et l'organisation des hackathons geomstats furent une façon de restaurer les interactions perdues à cause de la crise sanitaire. Je mesure bien la valeur de ces riches échanges qui, outre l'intérêt crucial de contrecarrer la dangereuse distanciation « sociale », m'ont beaucoup apporté dans mes recherches. Un immense merci pour tout cela.

I warmly thank Pierre-Antoine Absil and Marc Arnaudon for having accepted to review this manuscript and for their very careful and detailed reports and observations. I would like to thank Rodolphe Sepulchre who accepted to be the president of the jury and Rajendra Bhatia, Minh Ha Quang, Huiling Le and Frank Nielsen for being part of the jury. I am especially grateful to Frank for his strong support throughout the thesis, the many references he pointed to me and for being part of my PhD committee with Marco Lorenzi, whom I also thank a lot for his support and accurate suggestions. I would like to thank Minh for his invitation to spend some time in Japan, which unfortunately did not occur, and for sending me Rajendra's paper on the Bures-Wasserstein metric, without which I may have never had a look at it. In the same spirit, I thank Estelle Massart for showing me Takatsu's and her work on this Bures-Wasserstein metric, and Peter Michor for helping me understand the geometry of orbit spaces.

Je remercie Nicholas, grâce à qui tout a commencé, ainsi que Xavier, Hervé, Maxime, Marco et Irene de faire vivre l'équipe Epione de façon aussi dynamique et passionnée. Je mesure la chance que nous avons d'en faire partie, de pouvoir interagir avec tant de collègues et aussi, simplement, de pouvoir se faire des amis parmi eux. Je remercie tout particulièrement Isabelle pour son aide précieuse sur les missions, les conférences, les paiements, les contrats, les journaux, les réservations, la soutenance et tout ce que l'on ne voit pas mais qui est si important.

Au sein de la petite entreprise de Xavier, G-Statistics, qui ne connaît pas la crise, j'aimerais avant tout remercier Nicolas qui fut mon compagnon de route, qui m'a parlé Gallier et Postnikov tandis que je lui répondais Paulin et O'Neill -entre autres Sharpe, Lee, Kobayashi et Nomizu -, qui a eu la patience de m'initier à geomstats, PyCharm, et même au cluster, et avec lequel j'ai eu plaisir à découvrir quelques opéras et instruments de musique. Je remercie Nina pour avoir initié l'aventure geomstats et pour nous y avoir vite associés. leur amour inconditionnel, ma Tati-marraine qui veille toujours sur moi, Laurent pour notre complicité qui supporte la distance, Jean-Michel, Marie-Cécile, Tonton Jacky, Claude, Rémi, Juliet, Joël, Taoufik, Khadija, Marie-Louise, Jeannine pour leurs pensées toujours tendres et affectueuses. J'ai moi-même une pensée émue pour Jean qui restera une grande source d'inspiration, et une pensée espiègle pour Leila et Yannis. Je remercie mes parents pour leur soutien indéfectible et leur présence réconfortante. Merci d'avoir toujours encouragé ma curiosité, mon indépendance, et de m'avoir transmis tant de vous, en particulier le goût des mathématiques, le dépassement de soi et le respect de chacun. Je remercie Sara pour son soutien quotidien, sa tendresse, sa profonde gentillesse, sa fraîcheur innocente, son humour décalé, son honnêteté et son sens de l'honneur, sa curiosité et sa culture. Volim te.

Funding

This thesis was funded by the French government, through the UCAJEDI Investments in the Future project managed by the National Research Agency (ANR) with the reference number ANR-15-IDEX-01. The contract extensions, travels, equipment, participation to conferences and publication fees were funded by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant G-Statistics agreement No 786854).

Part I

Introduction

Riemannian manifolds

Data science consists in extracting knowledge from the data collected in a broad variety of contexts. The data may take multiple forms such as signals (brain activity, radar), images (MRI, scanner), videos, shapes (organs, bones, proteins, cells), networks, etc. These data are generally modelled by mathematical objects such as structured matrices, trees, graphs, meshes, shapes, curves, diffeomorphisms. To analyze the data, one needs to compute with this kind of objects. However, these mathematical objects often live in non-linear spaces where the classical statistical operations are not defined. Indeed, without a vector space structure, it seems impossible to compute means, to perform principal component analyses, to do interpolations or extrapolations for example.

Therefore, a natural idea that became popular in data science over the last 20 years is to consider that the data live in a Riemannian manifold, that is a locally Euclidean space. It allows to generalize a lot of Euclidean operations, in particular statistical operations. Firstly, thanks to the differential structure, one can differentiate curves, functions and define the gradient. If the manifold is oriented, which is often the case in practice, there is a canonical volume form which defines the Riemannian measure. Thus, absolutely continuous measures and especially densities of probability measures can be defined. Secondly, the Riemannian metric defines a canonical affine connection called the Levi-Civita connection. Thus, the Riemannian manifold inherits the notions of geodesics (or self-parallel curves), exponential map, parallel transport and Riemann curvature. Other notions of curvature may be defined such as the scalar curvature, the Ricci curvature and especially the sectional curvature which characterizes the Riemann curvature. Thirdly, the Riemannian metric defines a notion of length and an intrinsic distance called the Riemannian distance on each connected component of the manifold. The locally minimizing geodesics of the Riemannian distance coincide with the geodesics of the Levi-Civita connection. Moreover, the distance allows to define (globally) minimizing geodesics and Riemannian logarithms. In addition, the Hopf-Rinow theorem ensures that the metric space is complete if and only if the Riemannian manifold is geodesically complete. All this is summarized on the left part of Table 1.1.

This very rich set of geometric tools led to generalize many statistical concepts and algorithms from Euclidean spaces to Riemannian manifolds. In a metric space (M, d), a

Fréchet mean [Fréchet, 1948] of the points x 1 , ..., x k ∈ M is a minimizer of the continuous map f : x ∈ M -→ k i=1 a i d(x, x i ) p with p = 2 and a 1 = ... = a k = 1. More generally, this defines the notion of Riemannian L p center of mass for any p 1 and the notion of Riemannian barycenter for any coefficients a 1 , ..., a k . Thus, these notions are defined in Riemannian manifolds via the Riemannian distance. Moreover, precise conditions were derived to ensure their local existence, uniqueness and convexity [Karcher, 1977, Buser and Karcher, 1981, Kendall, 1990, Le, 2004, Yang, 2010, Afsari, 2011, Arnaudon and Miclo, 2014] and the consistence and robustness of empirical estimators [Bhattacharya andPatrangenaru, 2003, Arnaudon et al., 2013]. In addition, several gradient descent algorithms were proposed to compute them [Pennec, 2006, Arnaudon et al., 2012, Arnaudon et al., 2013, Afsari et al., 2013, Bonnabel, 2013]. More generally, many optimization algorithms (Riemannian Newton methods, line-search methods, trust-region methods) based on the concept of retraction were developed on matrix manifolds to tackle classical problems such as the eigenvalue problem [START_REF] Absil | Optimization algorithms on matrix manifolds[END_REF], Boumal et al., 2014]. Other main statistical operations were defined such as the covariance of a sample, the Mahalanobis distance, the normal law or the χ 2 law [Pennec, 2006]. A central limit theorem was established [START_REF] Bhattacharya | Large sample theory of intrinsic and extrinsic sample means on manifolds-II[END_REF] and several generalizations of Principal Component Analysis (PCA) were imagined: tangent PCA, Principal Geodesic Analysis [Fletcher andJoshi, 2004, Sommer et al., 2014], Geodesic PCA [START_REF] Huckemann | Intrinsic shape analysis: Geodesic PCA for Riemannian manifolds modulo isometric Lie group actions[END_REF]. Approximated parallel transport such as Schild's ladder [START_REF] Lorenzi | Schild's Ladder for the Parallel Transport of Deformations in Time Series of Images[END_REF] and pole ladder [Lorenzi andPennec, 2013, Guigui andPennec, 2021a] were also proposed for registration purposes. This is summarized on the right part of Table 1.1. operations to non-linear spaces. However, it is not always possible to equip a space with a differential structure so the hypothesis of working in a Riemannian manifold may be a bit restrictive. A more general framework is given by metric spaces.

Metric spaces

Spaces of trees and graphs are typical examples of spaces that are too complex to admit a manifold structure. We can cite the BHV space of phylogenetic trees [START_REF] Billera | Geometry of the Space of Phylogenetic Trees[END_REF], the QED space of unlabeled trees [START_REF] Feragen | Geometries on spaces of treelike shapes[END_REF] or more recently the Graph space [START_REF] Calissano | Populations of Unlabeled Networks: Graph Space Geometry and Geodesic Principal Components[END_REF] and the Wald space of forests [START_REF] Garba | Information geometry for phylogenetic trees[END_REF]. The three first spaces are complete length spaces and even geodesic spaces. A length space is a metric space such that the distance between any two points coincides with the infimum of lengths of curves between them. A geodesic space is a metric space such that between any two points, there exists a (globally minimizing) geodesic. Beware that in the context of metric spaces, a geodesic often designates a globally minimizing geodesic whereas in Riemannian manifolds, it rather stands for a locally minimizing geodesic, or equivalently a self-parallel curve. The Hopf-Rinow theorem ensures that any complete and locally compact length space is a geodesic space. A notion of curvature due to Aleksandrov also exists in metric spaces [START_REF] Aleksandrov | Generalized Riemannian spaces[END_REF], Alexander et al., 2019, Feragen and Nye, 2020]. It consists in comparing the areas of triangles with the model spaces of constant curvature: the sphere, the Euclidean space and the hyperbolic space. It defines upper or lower bounds for the curvature of the metric space.

The notions of minimizing geodesics, Fréchet means and its variants are defined in a metric space (M, d). The consistency of the Fréchet mean and median empirical estimators still holds [Bhattacharya andPatrangenaru, 2003, Arnaudon et al., 2013]. More generally, a distance defines a measure of dissimilarity between points, which can be taken as a loss function. Thus, one can define least square models [START_REF] Feragen | Chapter 8. Statistics on stratified spaces[END_REF] and therefore geodesic regression or principal geodesics. However, the main difference with Riemannian manifolds is the scarce quantity of available tools to compute these statistical operations. The tools for metric spaces are more specific to a given space, such as the computation of geodesics in the BHV space [Kupczok et al., 2008, Owen andProvan, 2011], the phylogenetic PCA [Nye, 2011, Nye et al., 2017] or the computation of Fréchet means for persistence diagrams [START_REF] Turner | Fréchet Means for Distributions of Persistence Diagrams[END_REF] for example.

Fortunately, there are subclasses of metric spaces where we retrieve generic results and methods. For instance, there exist many references on non-positively curved spaces [START_REF] Bridson | Metric Spaces of Non-Positive Curvature[END_REF], results on their probability measures [Sturm, 2003] and algorithms to compute medians and means in Hadamard spaces [Bacák, 2014]. Hadamard spaces are complete non-positively curved geodesic spaces, they are generalizations of Hadamard manifolds and satisfy analogous properties. For example, there exists a unique geodesic segment between any two points. In contrast, it seems that few tools exist on non-negatively curved spaces. Another example of notable metric spaces is given by quotients of metric spaces by isometric group actions. When the total space is a Riemannian manifold, Geodesic PCA is still defined [START_REF] Huckemann | Intrinsic shape analysis: Geodesic PCA for Riemannian manifolds modulo isometric Lie group actions[END_REF], potentially with generalized geodesics. When the group is finite like for the Graph space, there exist algorithms of type "Align All and Compute" to compute the Fréchet mean or the geodesic components [START_REF] Calissano | Populations of Unlabeled Networks: Graph Space Geometry and Geodesic Principal Components[END_REF]. They converge in finite time since the group is finite.

Thus, the framework of metric spaces is much more limited than the one of Riemannian manifolds. However, there exist subclasses of convenient spaces and examples such as spaces of trees that were extensively studied. Most of examples of metric spaces that are not Riemannian manifolds actually convey more structure than a simple distance: they are often stratified spaces.

Stratified spaces

Stratified spaces were defined and studied by Hassler Whitney, René Thom and John Mather [Thom, 1969, Mather, 1970]. A C p stratification of a closed set X in a manifold M is a sequence of nested spaces X = X k ⊃ X k-1 ⊃ ... ⊃ X 0 ⊇ X -1 := ∅ such that for all i ∈ {0, ..., k}, the set M i = X i \X i-1 is a C p manifold and the map i ∈ {0, ..., k} -→ dim M i is increasing. (As in [Trotman, 2020], we can always impose dim M i = i by adding copies of X i 's and by allowing that M i might be empty.) The connected components of the M i 's are called the strata. Therefore M i can be split into strata S j i for 1 j α i where α i is the number of connected components of M i = 1 j α i S j i . Thus the space X is split into connected manifolds, X = k i=0 α i j=1 S j i . We say that the stratum S is adjacent to the stratum T and we write T < S when T ⊂ S\S. This defines the partial order of adjacency on the set of strata. We can represent the adjacency by the Hasse diagram of this partial order.

We say that a pair of strata (S, T ) satisfies Whitney's condition (b) at t ∈ T when for all sequences (s n ) ∈ S N and (t n ) ∈ T N tending to t, if in a chart of M around t, the sequence of tangent spaces (T sn S) tends to a space τ in the Grassmannian Gr(dim M, dim S) and the sequence of lines (s n t n ) tends to a line λ in the Grassmannian Gr(dim M, 1), then λ ⊆ τ . A Whitney stratification is a C 1 stratification such that the number of strata is locally finite (each point has a neighborhood that meets a finite number of strata) and such that for all strata T < S, the pair (S, T ) satisfies the Whitney's condition (b) at all t ∈ T . Therefore, one can show that T < S implies dim T < dim S [Trotman, 2020], so if T ⊆ M i and S ⊆ M i , then i < i . In other words, the map S ⊆ M i -→ i ∈ {0, ..., k} is increasing. Let us illustrate the definition on examples.

A manifold with boundary M (that is locally diffeomorphic to an open set of a half-space R n-1 × [0, +∞)) is a stratified space with X 0 = ∂M (the boundary) and X 1 = M (the manifold without boundary). For example, the closed ball of R n for the Euclidean norm Bn 2 (0, 1) = {x ∈ R n | x 2 1} is composed of the sphere M 0 = S n 2 (0, 1) and the open ball M 1 = B n 2 (0, 1). The dimensions are dim(M 0 ) = n -1 and dim(M 1 ) = n. The closed ball of R 3 for the infinite norm B3 ∞ (0, 1) = {x ∈ R 3 | x ∞ = max |x i | 1} is a cube where the strata are the 8 vertices M 0 , the 12 edges M 1 , the 6 faces M 2 and the open cube M 3 . The dimensions are dim(M i ) = i. The Hasse diagram of adjacency of the closed unit square of R 2 is represented on Figure 1.1. Note that when the strata are connected manifolds, the diagram is a trivial line M 0 → ... → M k . Other popular examples are the spider and the open book exposed in [START_REF] Feragen | Chapter 8. Statistics on stratified spaces[END_REF]. The spider is a set of half-lines connected at their finite extremity. The open book is the Cartesian product of R and the spider, that is a set of half-planes sharing their boundary line. Their Hasse diagrams of adjacency are represented on Figure 1.2.

Vertices

M 0 (1, 1) $ $

(1, -1) 1}. This is an upper semi-lattice. On the diagram, the free variables x and y stand for all the interval (-1; 1). Point

z z % % (-1, 1)
x x & & M 0 Line x x & &
Half line 1 Metric stratified spaces are stratified spaces endowed with a distance inducing the same topology. The BHV space [START_REF] Billera | Geometry of the Space of Phylogenetic Trees[END_REF] and orthant spaces [START_REF] Barden | The logarithm map, its limits and Fréchet means in orthant spaces[END_REF], the QED space [START_REF] Feragen | Geometries on spaces of treelike shapes[END_REF], the Graph space [START_REF] Calissano | Populations of Unlabeled Networks: Graph Space Geometry and Geodesic Principal Components[END_REF], the Wald space [START_REF] Garba | Information geometry for phylogenetic trees[END_REF] are important examples of metric stratified spaces used in applications. They offer more structure than simple metric spaces. For example, the BHV space is a collection of Euclidean orthants in which the topology of the tree is preserved. They are glued together so that the intersection between two orthants corresponds to the shrinking of an edge in the tree. Therefore, minimizing geodesics are piecewise Euclidean [START_REF] Billera | Geometry of the Space of Phylogenetic Trees[END_REF], Nye, 2011, Feragen and Nye, 2020]. A logarithm, generalizing the Riemannian logarithm, is defined and the Fréchet mean and its limiting distribution can be precisely characterized [START_REF] Barden | The logarithm map, its limits and Fréchet means in orthant spaces[END_REF]. We would like to go beyond spaces of trees and graphs where methods seem to be mainly ad hoc. Thus, we promote an interesting subclass of metric stratified spaces with more structure which is the class of Riemannian orbit spaces [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF].

• • • Half line k M 1 Half plane 1 • • • Half plane k

Riemannian orbit spaces

In this thesis, we choose to work with a particular type of metric stratified spaces that we call Riemannian orbit spaces. A Riemannian orbit space is the quotient of a Riemannian manifold by a proper and isometric Lie group action [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF], Huckemann et al., 2010]. (Note that the condition of a proper group action is classical to get a Hausdorff separated quotient space.) When the action is free in addition, the quotient space has a unique structure of smooth manifold such that the canonical projection is a submersion [Lee, 2012]. It can be turned into a Riemannian submersion by descending the Riemannian metric. The theory of Riemannian submersions apply so the geodesics and the curvature can be deduced from the ones of the total space [O'Neill, 1966]. When the action is not free, the isotropy group is not reduced to identity: the points can be grouped by conjugation class of isotropy group, this creates strata in the total space and in the quotient space, called orbit space [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF], Michor, 2008]. The canonical projection becomes a stratified Riemannian submersion in a sense. Moreover, the Riemannian distance of the total space descends to a quotient distance on the orbit space so we get a metric stratified space. The strata are proved to be Riemannian manifolds [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF], Michor, 2008].

Riemannian orbit spaces can be seen as the stratified generalization of quotients of Riemannian manifolds by a free, proper, isometric Lie group action. Some results on complete Riemannian quotient manifolds still hold in complete Riemannian orbit spaces. For example, any minimizing geodesic segment of the orbit space is the projection of a horizontal geodesic segment, called a horizontal lift [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF]. More precisely, the length of a horizontal curve coincides with the length of its projection as long as it does not meet an orbit of more singular type [Michor, 2008]. Moreover, all the horizontal lifts of a geodesic differ by the action of an element in the Lie group [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF]. These nice properties allow to characterize the geodesics in a complete Riemannian orbit space. Hence, they are a particularly convenient type of metric stratified spaces.

How to choose the geometry

After detailing these possible choices of geometric structures, how can we choose between them? Moreover, even within one of these classes of structures, does there exist a canonical geometric structure to use? Otherwise, how can we choose it? In Euclidean spaces, the notions of straight line or mean do not depend on the choice of the inner product because they all share the same canonical affine connection. In non-linear spaces, all these notions strongly depend on the choice of the geometric structure, should it be a Riemannian metric or a distance. We identify three principles to choose the geometric structure for non-linear data.

First principle: adequacy of the model to the data

The first principle is the adequacy of the model to the data. Do they live in a manifold or are there singularities to take into account in a more complex model such as a stratified space? For example, in the set of covariance matrices, rank-deficient matrices may correspond to a non-physical reality. For instance, in diffusion tensor imaging or electro-encephalography, these matrices are often considered as degenerate and there exist tricks to make them positive definite such as adding εI n to the dataset. In this case, the model should be the manifold of symmetric positive definite (SPD) matrices with a Riemannian metric that rejects rankdeficient matrices to infinity. However, in other contexts, rank-deficient matrices should be kept at finite distance because they correspond to a physical reality. It occurs when covariance matrices are of the type Σ = XX with X ∈ R n×k with n > k, i.e. empirical covariance matrices between many features with few data points. So the treatment of singularities depends on the application. Another example of singularity on SPD matrices is given by matrices with two equal eigenvalues. For instance, in diffusion tensor imaging (DTI), they may indicate the crossing of two fibers. Therefore, considering this space as stratified by eigenvalue multiplicity is more suited to this kind of problem [START_REF] Groisser | Geometric foundations for scaling-rotation statistics on symmetric positive definite matrices: Minimal smooth scaling-rotation curves in low dimensions[END_REF].

Another important element is the notion of invariance under the action of a group of transformations or under a set of symmetries of the space. Being invariant under a transformation means that if it is applied jointly to all the dataset, then the results of geometric operations such as the Fréchet mean will be transformed accordingly. Therefore, if the inverse transformation is applied at the end, the result will be the same as the one without any transformation so the statistical analysis will be unchanged. Let us take the example of permuting the nodes in an undirected graph with vertices {A, B, C} for three types of electronic device which exchange information. The attribute of an edge is the quantity of information exchanged between the two types of device, allowing self loops. The adjacency matrix of the graphs is built by associating A, B, C to 1, 2, 3 respectively. A distance on adjacency matrices with non-negative coefficients M ∈ Sym 0 (3) is invariant under permutations if and only if the coefficients M 11 , M 22 , M 33 (resp. M 12 , M 23 , M 31 ) are treated equally. For example, d(M, M

) = i |M ii -M ii | + 3 i =j |M ij -M ij | is permutation-invariant but d (M, M ) = i,j i|M ij -M ij | is not.
Moreover, the choice of associating A to 1, B to 2 and C to 3 is arbitrary so we probably want the statistical analysis to be independent from this choice. That is exactly what a permutation-invariant distance allows to do. If the same permutation is applied to the whole dataset, then the Fréchet mean will be affected by the same permutation so we will be able to retrieve the information on A, B, C respectively since we know how they are associated to 1, 2, 3. Thus, a permutation-invariant distance preserves the difference between A, B and C, despite what one could think.

Invariance is not independence. Being independent from a transformation is much stronger: it means that if a set of different transformations is applied independently to each data within the dataset, then the result of the geometric operations will not even be affected. It means that the geometry has to be defined on the quotient of the space by this group of transformations. Here, the distance does not distinguish anymore between A, B and C. Actually, the graph itself is defined up to permutation, the vertices have no label here. For example, the Graph space is defined as the quotient of graphs by the permutation of nodes [START_REF] Calissano | Populations of Unlabeled Networks: Graph Space Geometry and Geodesic Principal Components[END_REF].

In addition to singularities, invariance or independence under natural transformations of the space, another type of hypothesis consists in decoupling variables and turning the space into a Cartesian product endowed with a product metric or a product distance. This can come from the observation that two variables seem to be unrelated. This can also be a refinement of the assumption of independence under one of the two variables. Indeed, the respective weights in the product metric can give more or less importance to each variable.

In general, these assumptions lead to families of admissible structures rather than one unique structure. Thus, we need other principles to discriminate between them.

Second principle: good theoretical properties

We saw that for metric spaces, some hypotheses such as completeness, non-positive curvature or quotient structure, lead to important theoretical results of existence and uniqueness. We also argued that Riemannian manifolds convey many more geometric tools and results which allow to generically perform some statistical operations. Let us specify some particular types of Riemannian manifolds where there are even more tools and results to use.

When a manifold M is embedded in a Riemannian manifold (M 0 , g 0 ), the ambiant Riemannian metric g 0 induces a Riemannian metric g on the submanifold. Gauss-Codazzi equations provide the Levi-Civita connection and the curvature in (M, g) in function of those in the embedding space (M 0 , g 0 ).

Another typical situation is when a submersion π : M 0 -→ M is given. The submersion π and the metric g 0 allow to define a vertical distribution V = ker dπ, a horizontal distribution H = V ⊥ , a horizontal lift # x = (d x π) -1 |Hx : T π(x) M -→ H x at each point x ∈ M 0 , and finally an induced metric g π(x) = g 0 x • (# x , # x ) on M. The map π : (M 0 , g 0 ) -→ (M, g) is called a Riemannian submersion and O'Neill's equations provide the Levi-Civita connection, the curvature and the exponential map in (M, g) in function of those in the total space (M 0 , g 0 ) [O 'Neill, 1966]. More precisely, the geodesics in M are the projections of horizontal geodesics in M 0 so the exponential map writes Exp π(x) = π • Exp 0

x • # x . In particular, if (M 0 , g 0 ) is geodesically complete, then (M, g) is geodesically complete.

Many particular cases of Riemannian submersions can be met such as Riemannian quotient manifolds, Riemannian homogeneous manifolds, naturally reductive homogeneous manifolds or Riemannian symmetric spaces. Each of these classes is included in the previous one and brings more properties to the space. They are summarized along with the embedded spaces in Table 1.2.

Riemannian manifold Theoretical properties Embedded

Levi-Civita connection, curvature (if known in embedding space) Submersed/quotient + exponential map (if known in the total space) Homogeneous + completeness, transitive isometric group action Naturally reductive + geodesics at identity are one-parameter subgroups Symmetric + ∇R = 0, parallel transport is a composition of two symmetries Euclidean + everything in closed form Table 1.2: Theoretical properties of Riemannian manifolds implying practical properties.

Apart from Lie group actions that are compatible with the Riemannian metric, Riemannian manifolds may also carry other compatible structures such as a pair of dual connections or a divergence, or it may be seen as the parameter space of a statistical model. This is well studied in information geometry [START_REF] Amari | Methods of Information Geometry[END_REF], Amari, 2016, Nielsen, 2022]. Two connections ∇, ∇ * are dual with respect to the metric g if they satisfy X(g(Y, Z)) = g(∇ X Y, Z) + g(Y, ∇ * X Z) for all vector fields X, Y, Z. A divergence is a distance-like function D : M 2 -→ [0, +∞) satisfying separation and which induces by differentiation a dualistic structure (g, ∇, ∇ * ), that is a Riemannian metric g with dual connections ∇, ∇ * [START_REF] Amari | Methods of Information Geometry[END_REF].

A statistical model, or family of probability densities, induces by differentiation a Riemannian metric called the Fisher-Rao metric g. Then, the natural gradient is defined as the gradient associated to the Fisher-Rao metric. A statistical model also induces a pair of dual connections (∇ (1) , ∇ (-1) ), or more generally the one-parameter family of Amari-Chentsov α-connections (∇ (α) ) α∈R [START_REF] Amari | Methods of Information Geometry[END_REF]. The dualistic structure (g, ∇ (α) , ∇ (-α) ) can also be viewed as emanating from a divergence. If the statistical model is an exponential family or a mixture family, then ∇ (1) and ∇ (-1) are flat [START_REF] Amari | Methods of Information Geometry[END_REF].

Information-geometric tools are used in many algorithms for parameter estimation [Amari, 1995], computing centroids [START_REF] Nielsen | Sided and Symmetrized Bregman Centroids[END_REF], clustering [START_REF] Liu | Shape Retrieval Using Hierarchical Total Bregman Soft Clustering[END_REF] or function optimization [START_REF] Ollivier | Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles[END_REF], to cite a few.

To summarize, the Riemannian geometry can be enriched with other compatible geometric structures that provide more tools to compute on the manifold.

Third principle: good practical properties

Our third principle is the ability of the geometry to be implemented efficiently, for example with closed-form formulae or fast algorithms to compute the geometric operations such as the Fréchet mean, the geodesics or the parallel transport. As we argued earlier, this ability is often a consequence of a rich geometry. As shown in Table 1.2, the simpler is the richer. Note that it does not mean that the simplest geometry is the best suited for the application at hand. Application constraints and practical constraints might be in conflict so one might need to find a trade-off between them. This supports the idea of defining families of Riemannian metrics encompassing different geometries to be able to go continuously from one to the other.

This principle also pushes us to give a particular attention to numerical questions. The python package geomstats [Miolane et al., 2020a[START_REF] Miolane | Introduction to Geometric Learning in Python with Geomstats[END_REF] aims at offering generic implementations for certain classes of manifolds such as Riemannian quotient spaces, Lie groups, or more generally Riemannian manifolds and manifolds equipped with an affine connection. For example, the exponential map of an affine connection is computed by integrating the Hamiltonian flow. These generic methods can be overridden by specific methods, for example when one has closed-form formulae or more efficient algorithms. This encourages us to push the theoretical computations as far as possible because we often get formulae that can be computed faster and with more precision than the generic implementations. The geometric operations being the building blocks of more complex algorithms, it is crucial to simplify their implementation when it is possible.

Spaces of covariance and correlation matrices

We now focus on the spaces studied in this thesis: the spaces of covariance and correlation matrices.

Two stratified spaces

In many applications, data are modelled by covariance matrices or correlation matrices for example between signals, physical quantities or within a time window (auto-correlation). The application domains are electro-encephalography (EEG) [START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF], Barachant et al., 2013], magneto-encephalography (MEG), functional Magnetic Resonance Imaging (fMRI) [START_REF] Marrelec | Partial correlation for functional brain interactivity investigation in functional MRI[END_REF], Varoquaux et al., 2010], diffusion tensor imaging (DTI) [START_REF] Pennec | A Riemannian Framework for Tensor Computing[END_REF], Lenglet et al., 2006, Fletcher and Joshi, 2007, Moakher, 2005, Batchelor et al., 2005], trees and graphs [START_REF] Journée | Low-Rank Optimization on the Cone of Positive Semidefinite Matrices[END_REF], Severn et al., 2019, Garba et al., 2021], signal processing, radar [Barbaresco, 2013], computer vision, genomics [de la Fuente et al., 2004, Peng et al., 2009], etc. Covariance matrices are symmetric positive semi-definite matrices. They don't form a manifold but a stratified space whose strata are the manifolds of symmetric positive semi-definite matrices of fixed rank. Since these manifolds are connected, the Hasse diagram of adjacency is trivial. The principal stratum is the manifold of Symmetric Positive Definite (SPD) matrices, it is dense in the set of covariance matrices. Covariance matrices form a convex cone in the vector space of symmetric matrices.

Correlation matrices are covariance matrices with unit diagonal. The correlation coefficient between two variables is the quotient of their covariance by the product of their standard deviations: Cor(X i , X

j ) = Cov(X i ,X j ) √ Cov(X i ,X i ) √ Cov(X j ,X j )
. Thus correlation matrices C = Cor(X) are built from covariance matrices Σ = Cov(X) with non-zero variances by dividing each entry by the product of square-roots of the diagonal elements on its row and its column:

C ij = Σ ij √ Σ ii √ Σ jj
or C = Diag(Σ) -1/2 Σ Diag(Σ) -1/2 . Correlation matrices form a convex space called the elliptope. They admit a similar stratified structure where the strata are also the manifolds of correlation matrices of fixed rank. In terms of strata, a first difference with covariance matrices is that the null matrix is not a correlation matrix. A second difference is that the correlation matrices of rank one form a discrete manifold corresponding to the corners of the elliptope [Tropp, 2018]. Back to the difference explained earlier between invariance and independence, let us consider the action of the Lie group of positive diagonal matrices on SPD matrices. We can decompose the space of SPD matrices into the Cartesian product of positive diagonal matrices and full-rank correlation matrices. Therefore, the action of the positive diagonal group only acts on the diagonal component. If an experiment with covariance matrices as data is very weakly sensitive to the scales of variables, it means that the correct model to use should be correlation matrices instead of covariance matrices.

To study the geometry of these stratified spaces, it seems natural to proceed step by step by studying firstly the principal stratum, secondly the other strata, thirdly the whole space. Since the principal stratum is dense in the whole space, it is worth spending some time studying it because it is a good approximation of the whole space. In other words, random generic covariance matrices are almost surely SPD. However, covariance matrices may not be generic in many practical problems. For example, when empirical covariance matrices are computed with less data than the number of features, they are mandatorily degenerate and the previous statement fails. That is precisely why it is also important to have theoretical tools on the boundary.

The many geometries of SPD and PSD matrices

Many Riemannian metrics were proposed on the manifold of SPD matrices, emanating from different contexts from algebra to information geometry or quantum information geometry: affine-invariant [Siegel, 1943, Skovgaard, 1984, Pennec et al., 2006, Lenglet et al., 2006, Fletcher and Joshi, 2007, Moakher, 2005], log-Euclidean [START_REF] Arsigny | Log-Euclidean metrics for fast and simple calculus on diffusion tensors[END_REF], Fillard et al., 2007, Hà Quang et al., 2014], Bures-Wasserstein [Dowson and Landau, 1982, Olkin and Pukelsheim, 1982, Takatsu, 2010, Takatsu, 2011, Malagò et al., 2018, Bhatia et al., 2019], Bogoliubov-Kubo-Mori [Petz andToth, 1993, Michor et al., 2000], power-Euclidean [Hiai andPetz, 2009, Dryden et al., 2010], alpha-Procrustes [Hà Quang, 2022], power-affine [START_REF] Thanwerdas | Is affineinvariance well defined on SPD matrices? A principled continuum of metrics[END_REF], Cholesky [START_REF] Wang | A constrained variational principle for direct estimation and smoothing of the diffusion tensor field from complex dwi[END_REF], log-Euclidean-Cholesky [START_REF] Li | Local Log-Euclidean Multivariate Gaussian Descriptor and Its Application to Image Classification[END_REF] log-Cholesky [Pinheiro andBates, 1996, Lin, 2019], etc. We generically call them the noted metrics. Let us follow the general method we sketched to choose a Riemannian metric. On the manifold of SPD matrices, the singularities are the degenerate matrices, those which have at least one null eigenvalue and which are located on the boundary. Therefore, depending on the problem, we may either want these degenerate matrices to be at finite or infinite distance of SPD matrices.

One natural transformation of the space of SPD matrices is given by the invertible affine transformation of the feature vector X -→ AX + X 0 with A ∈ GL(n). It is well known that the new covariance matrix is congruent to the original one via the linear part A of the transformation: Cov(AX + X 0 ) = A Cov(X)A . Therefore, choosing an affine-invariant metric means that the geometry (and the associated geometric operations) will not change if we apply jointly an affine transformation to the whole set of data. For example, in braincomputer interfaces, we may assume that between two different sessions, the electrode cap is not positioned at the same location and that this induces a linear transformation in the signals acquired at first order. In contrast, within a session, we may assume that the transformation remains the same for all the experiments. Then, under this hypothesis, an affine-invariant metric seems to be a good choice to analyze experiments within a session but there should be an alignment step or a change of reference across different sessions [START_REF] Barachant | Classification of covariance matrices using a Riemannian-based kernel for BCI applications[END_REF].

Furthermore, all the subgroups of the general linear group define a notion of invariance that may be of interest or not depending on the application: orthogonal transformations (orthogonal group), independent positive scalings of variables (positive diagonal group), permutations between variables (permutation group) or global positive scaling (positive real group) for instance. All the metrics cited above are invariant under orthogonal transformations, except those named after Cholesky. Their quantity suggests to propose other theoretical properties to classify and distinguish them.

On singular PSD matrices of fixed rank, it seems that four Riemannian metrics were proposed [START_REF] Bonnabel | Riemannian Metric and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank[END_REF], Vandereycken et al., 2009, Vandereycken et al., 2013, Massart and Absil, 2020]. These spaces are a bit more difficult to handle since they are not open sets of vector spaces. On PSD matrices, the invariance under affine transformations or even under the positive diagonal group is impossible to get for any distance [START_REF] Bonnabel | Riemannian Metric and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank[END_REF]. However, it might be quite difficult to glue these Riemannian manifolds together to form a global interesting geometry on covariance matrices of any rank. Therefore, it appears that it is important to think to a geometry on the whole space before studying it on each stratum separately.

The only distance defined on the whole space of covariance matrices that we are aware of is the Bures-Wasserstein distance [START_REF] Dowson | [END_REF]Landau, 1982, Olkin andPukelsheim, 1982]. The stratified topology was described in [Takatsu, 2011] as well as the tangent cone (which replaces the notion of tangent space at singular points). It was also shown to be an Aleksandrov space of non-negative curvature. There seems to be no other work on stratified geometries of covariance matrices so far.

The few geometries of correlation matrices

In contrast, very few Riemannian metrics or more general geometric structures were proposed in the literature on the manifold of correlation matrices. The oldest geometry seems to be the quotient by the orthogonal group of a product of n spheres S n-1 of dimension n -1. It was used to simulate correlation matrices in [START_REF] Rebonato | The most general methodology to create a valid correlation matrix for risk management and option pricing purposes[END_REF]. The quotient topology was later described in [Kercheval, 2008]. However, a complete description of the geometry does not exist neither on strata nor on the whole space of correlation matrices so far. Cholesky parametrizations were used to compute the nearest low-rank correlation matrix [START_REF] Grubišić | Efficient rank reduction of correlation matrices[END_REF].

On full-rank correlation matrices, we can cite the Hilbert projective geometry, which is a distance on any open bounded convex subset of R n . It was applied to the open elliptope of full-rank correlation matrices for clustering [START_REF] Nielsen | Clustering in Hilbert's Projective Geometry: The Case Studies of the Probability Simplex and the Elliptope of Correlation Matrices[END_REF]. A Riemannian metric was proposed by taking the quotient of the affine-invariant metric on SPD matrices by the action of positive diagonal matrices [David and[START_REF] David | [END_REF]. More recently, a Euclidean bijective parametrization of the open elliptope was proposed via the matrix exponential [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF].

The same remark as for SPD matrices can be made about singularities. If one wants to avoid singular matrices, then the open elliptope should be considered without its boundary. Since it is an open set of an affine space, the simplest Riemannian geometry should be diffeomorphic to a Euclidean geometry. Therefore, one should find natural diffeomorphisms between the open elliptope and Euclidean spaces. The recent bijective parametrization of [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF]] could be a nice tool to do so.

About the natural group transformations of the space, the situation is very different than the one of SPD matrices. Indeed, the affine transformation of the feature vector does not induce a group action on correlation matrices. Correlation matrices were often treated as covariance matrices in the applications. However, their structure is completely different. Indeed, among the groups cited above for covariance matrices, only the permutation group stabilizes the manifold of full-rank correlation matrices. Following the discussion on invariance and independence, it seems natural to use permutation-invariant Riemannian metrics when the features of the feature vector should be treated equally, for example for electrodes, brain regions, cells or genes.

One could also require the weaker invariance by groups of features, that is by block permutation matrices with fixed sizes of blocks. However, the invariance under permutation might not be relevant for features such as times in auto-correlation matrices because contrarily to the previous cases, the order between times is probably a key element of the analysis.

Comprehensive presentation of the contributions

Purposes and methodology

Our purposes follow the analysis we made of theoretical and practical tools available on covariance and correlation matrices. On SPD matrices, there is a need of classification to understand the common points and the differences between all the Riemannian metrics proposed in the literature. In particular, the invariance under orthogonal transformations seems to be a key classification criterion to study. In other words, the identified need is essentially theoretical since there exist already many tools to compute with SPD matrices.

On the contrary, there is an obvious lack of intrinsic and efficient tools even to interpolate between two full-rank correlation matrices. Our goal is to define several types of geometries with different properties so that people have the choice in function of their application. Furthermore, proposing families of metrics rather than isolated metrics is a leitmotiv to offer more flexibility and allow optimizing the parameters of the family under other constraints for example. In practice, the goal is to get simple formulae of geodesics or parallel transport to perform the basic geometric and statistical operations on the manifold of full-rank correlation matrices.

About the stratified spaces of covariance and correlation matrices, we aim at proposing a first example of stratified structure with a Riemannian metric on each stratum and a global distance between any two points of the space. This structural description should be enriched by some practical tools such as the geodesics between any two points.

Our methodology is the following. From the literature on applications using covariance and correlation matrices, we extract some assumptions that may be relevant in different contexts such as invariances, symmetries, singularities, independent variables. Then, we abstract these considerations and enrich them with other theoretical constraints on the algebraic or topological structure of the space, on the groups acting on it, on the relevant functions that were used in other domains for example. After that, we try to define new structures by manipulating all these concepts together. A crucial element of our methodology is to push the computations as far as they can be. We noticed that in several works, calculations were left with implicit formulae that are difficult to grasp and implement. Moreover from a computational point of view, a closed-form formula often offers faster and more precise results than a composition of generic functions that may propagate numerical errors. Following these guidelines, we got several results from theorems of classification, characterization or stability to practical formulae (geodesics, curvature, parallel transport, etc.) or algorithms. The package geomstats offers a place to gather all these tools on manifolds. Our methodology is schematized on Figure 1.3.

Classification of Riemannian metrics on SPD matrices

In order to classify Riemannian metrics on SPD matrices, we first characterize the continuous O(n)-invariant Riemannian metrics by means of three continuous multivariate functions α, β, γ : R n -→ R satisfying three conditions of symmetry, compatibility and positivity (Theorem 4.20). This shows that the class of O(n)-invariant metrics is much wider than the class of affine-invariant metrics, which is indexed by only two real parameters. Secondly, in light of this characterization, we reinterpret the classes of kernel and mean kernel metrics introduced in [Hiai andPetz, 2009, Hiai andPetz, 2012] (permutation matrices). This is shown on Figure 1.5. We characterize LT + (n)-invariant metrics as pullbacks by the Cholesky map of left-invariant metrics on the Lie group LT + (n) (Theorem 7.4). This gives a Lie group structure to the manifold of SPD matrices for which the affine-invariant metrics are left-invariant metrics. This is an unexpected result since this Riemannian manifold has only been considered so far as a Riemannian symmetric space.

To prove the continuity of the O(n)-invariant metric defined by the three continuous maps α, β, γ, we were brought to answer the following question: given two symmetric matrices, how can we choose their matrices of eigenvectors so that they are the closest possible? Indeed, the matrix of eigenvectors is not unique. We can reduce the degrees of freedom by imposing the order of the eigenvalues but we are left with rotations and symmetries in the eigenspaces. We answer this question in Chapter 2 as a mathematical preliminary.

These theoretical results came with additional practical results. We provide the complete formula of the sectional curvature of the affine-invariant metrics in Table 4.5 (one term was forgotten in [Skovgaard, 1984]). We give new formulae for the Bures-Wasserstein metric, especially an explicit equation for the parallel transport which allowed to implement it in geomstats (Table 4.7 and Proposition 4.12). We derived the curvature of the Mixed-Euclidean metrics (Theorem 5.25) based on the derivation of the BKM curvature in [START_REF] Michor | The Curvature of the Bogoliubov-Kubo-Mori Scalar Product on Matrices[END_REF]. In addition, we prove that all these curvatures take negative values and we show numerically that all the Mixed-Power-Euclidean metrics but the Euclidean, log-Euclidean and affine-invariant metrics also have positive curvature. We show that the necessary and sufficient condition for geodesic completeness on mean kernel metrics [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF] is still valid for the wider class of extended mean kernel metrics. We also show that the classes of kernel metrics and extended kernel metrics are stable by taking the cometric (if we identify the vector space of symmetric matrices and its dual via the Frobenius inner product). This is an important advantage to compute the geodesics by integration of the Hamiltonian equations of geodesics since they require to compute the cometric. These results can be found in Chapters 4 and 5 in Part III.

New families of Hadamard metrics on full-rank correlation matrices

We started by studying the recently introduced quotient-affine metric on full-rank correlation matrices [David and[START_REF] David | [END_REF]. It relies on the quotient space structure of the open elliptope. Indeed, the orbits of the action of positive diagonal matrices on SPD matrices are in bijection with correlation matrices so it suffices to find a metric invariant under the positive diagonal group to define a metric on the open elliptope, for example the affine-invariant metric [START_REF] David | A Riemannian structure for correlation matrices[END_REF]. Thanks to O'Neill's fundamental equations of Riemannian submersions, we compute the quotient geometry, the Riemannian metric, the geodesics, the Levi-Civita connection and the curvature in Chapter 6. We also show that Riemannian logarithms exist but the uniqueness remains an open question as well as closedform formulae. Furthermore, we show that the curvature is not of constant sign and that it is unbounded from above (Theorem 7.1). This argues for the definition of other metrics with a better behavior, for example Hadamard or even flat since the open elliptope is diffeomorphic to a Euclidean space. We show that the construction of quotient-affine metrics generalize to LT + (n)-invariant metrics (Section 7.3), which unfortunately suffer the same drawbacks. However, this generalization puts in light the interesting idea of transforming correlation matrices by the Cholesky map. This was done for SPD matrices but not precisely for fullrank correlation matrices. This allows us to define very convenient geometries by pullback via the Cholesky map in Section 7.

Part I. Introduction

Although these Riemannian metrics are very practical, they miss an ingredient that could be useful in some applications: the invariance by permutations. Therefore, we propose two permutation-invariant metrics defined completely differently in Chapter 8. The first one relies on a bijective parametrization recently proposed thanks to the matrix exponential map [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF]. The authors proved that given a symmetric matrix, there exists a unique diagonal matrix such that the exponential of their sum is a correlation matrix. This defines a bijection between symmetric hollow (with null diagonal) matrices and full-rank correlation matrices. We prove that this bijection is a smooth diffeomorphism (Theorem 8.6), therefore it allows to push permutation-invariant inner products forward to the open elliptope. We call these Riemannian metrics off-log metrics and, as a log-Euclidean metric, all the geometric operations are known in closed form modulo the computation of the bijection. It is proved in [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF] that the bijection can be computed in logarithmic time complexity.

To find all the permutation-invariant inner products on the vector space of symmetric hollow matrices, we were brought to answer the more general question of finding all the Ginvariant inner products on a vector space, knowing one of them. Although this seems to be a well-known result in representation theory, we were not able to find this result in reference books. Therefore, we answer this question and we present some examples in Chapter 3 as another mathematical preliminary.

In parallel, we explored an analogous idea which is somehow the symmetric or the dual of the previous one. We showed that given a correlation matrix there exists a unique positive diagonal matrix such that the logarithm of their composition by congruence belongs to a vector space, namely the vector space of symmetric matrices with null row sums (Theorem 8.15). This defines a bijection between full-rank correlation matrices and symmetric matrices with null row sums. Analogously, we show that it is a diffeomorphism (Theorem 8.19) and thus, permutation-invariant inner products can be pulled back to the elliptope. We call these pullback metrics log-scaled metrics because our result is based on the well-known concept of "scaling" of an SPD matrix [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF]. These metrics have an additional characteristic that only the quotient-affine metric has: they are inverse-consistent (Theorem 8.18). Correlation matrices are not stable by inversion but the correlation matrix of the inverse is of course a correlation matrix. This leads to a map that we call the corinversion and that is closely related to the notion of partial correlation, well known in signal processing or Gaussian graphical networks. Indeed, the opposite off-diagonal coefficients of the cor-inversion are exactly the partial correlation coefficients (or parcors). This important map defines notions of intrinsic inversion of correlation matrices and inverse-consistency of Riemannian metrics, that are satisfied by the log-scaled metrics. Quotient-affine metrics and log-scaled metrics intrinsically satisfy this property because their construction relies on the action of positive diagonal matrices. The properties of all these metrics are summarized on Table 1.3.

All the geometric operations of log-scaled metrics are known in closed form modulo the computation of the bijection, that is the computation of the scaling of an SPD matrix. Many algorithms exist to compute the scaling. However, most of them are not specific to SPD matrices. Surprisingly, the specific algorithms seem not to rely on the proof of existence of the scaling [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF] although it involves the minimization of a strictly convex map. We show that the optimization problem amounts to minimize this even strongly

(n)-inv. is S(n)-invariant, Inv.-cons. is Inverse-consistent.
convex map on a convex set. Thus we propose in Section 8.5 a method relying of the wellknown Projected Gradient Descent (PGD) algorithm. Since the projection is not known in closed form, we contribute a simple algorithm to compute the projection that we plug in the PGD algorithm. We prove the convergence of our approximated PGD algorithm in logarithmic time and we argue that its total complexity and precision is competitive with respect to SPD-specific and unspecific algorithms. Finally, we show that all these metrics split in two groups in dimension 2, that is with only one correlation coefficient (Theorem 8.23). This provides two different interpolations of the correlation coefficient, including one closely related to the Fisher z-transformation. There results are presented in Chapters 6, 7 and 8 in Part IV.

Bures-Wasserstein geodesics in the orbit space of covariance matrices

Before studying the geometry of stratified spaces, it is important to notice that it seems unrealistic to build a distance on the whole space that coincides with a complete Riemannian distance on the principal stratum. Indeed, the boundary is considered as a singularity rejected infinitely far so these viewpoints seem irreconcilable. Therefore, we are now more interested in non-complete metrics on SPD matrices that should be completed thanks to the stratified boundary of singular matrices. Many Riemannian metrics on SPD matrices such as the Euclidean or the power-Euclidean metrics have geodesics that leave the space of covariance matrices. We know only one metric whose geodesics bounce on the boundary and come back into SPD matrices: this is the Bures-Wasserstein metric. Thus we study the same metric stratified space as in [Takatsu, 2011].

We propose to apply the richer framework of Riemannian orbit spaces [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF] to the quotient of the Euclidean space of square matrices equipped with the Frobenius inner product by the right action of the orthogonal group. With this viewpoint, covariance matrices form a stratified space whose strata are the manifolds of covariance matrices of fixed rank, as explained in Section 9.3. It is well known that the quotient distance is exactly the Bures-Wasserstein distance. The geodesics were studied in the principal stratum of SPD matrices [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF], Bhatia et al., 2019] and in other strata [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF]. However, we noticed that several questions were still open on the injectivity domain of the exponential map, the set of preimages or the set of logarithms. We answer precisely all these questions in Sections 9.4 and 9.5. We also compute the horizontal lift, which allows us to give explicit formulae of the Riemannian metric and the exponential map.

Furthermore, we characterize precisely the minimizing geodesics of the total metric space between any two covariance matrices (Theorem 9.31). We prove that they are of constant rank on the interior of the segment. We show that the minimizing geodesic segments between two covariance matrices Σ and Λ are parametrized by the closed unit ball or R (k-r)×(l-r) for the spectral norm, where k, l, r are the respective ranks of Σ, Λ, ΣΛ (Theorem 9.34). We give their expression in function of this parameter. In particular, we show that there is either exactly one or an infinity of geodesics between two covariance matrices, depending on the rank of their product. We also give the number of geodesics of minimal rank and we give a simple formula of the minimizing geodesic when it is unique. Finally, we show that this formula actually defines a minimizing geodesic between any two matrices so we call it the canonical geodesic. These results are presented in Chapter 9.

Organization of the thesis 1.4.1 Overview

This thesis is organized in parts divided in chapters. In Part II, we give two mathematical preliminaries. They result from the resolution of two problems that came naturally throughout our research and that revealed to be non-trivial. We put them aside to ensure subject homogeneity within chapters. Chapter 2 tackles the problem of regularity of eigenvalues and eigenvectors of symmetric matrices. It allows us to show the continuity of a Riemannian metric defined via eigenvalue decomposition in Chapter 4. Chapter 3 answers the question of finding all G-invariant inner products on a vector space, based on representation theory. We use this result to build families of permutation-invariant metrics in Chapter 8.

Part III focuses on the geometries of the open cone of SPD matrices, that is full-rank covariance matrices. Since many geometries were previously defined on this manifold, we survey the literature in Chapter 4 and we complete it wherever needed with new formulae. Since these geometries were essentially O(n)-invariant, we characterize this general family and we show that it is much wider than the family of affne-invariant metrics. We promote the idea of defining families of Riemannian metrics encompassing existing ones and having nice stability properties. Thus, we rely on the previous work on kernel metrics of [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF] to define new subclasses and super-classes of metrics. In particular in Chapter 5, we explore the information geometry and the Riemannian geometry of the families of Mixed-Power-Euclidean and Mixed-Euclidean metrics, which encompass many noted metrics with few parameters.

We identified in the introduction the need to define geometric tools on correlation matrices for intrinsic computation and also to endow covariance matrices with product metrics. Therefore in Part IV, we define new geometries on the open elliptope of full-rank correlation matrices. After computing the Riemannian operations of the recently introduced quotient-affine metric [START_REF] David | A Riemannian structure for correlation matrices[END_REF] in Chapter 6 and generalizing it to quotient-Lie-Cholesky metrics, we show that their geometry is quite complex. Since the manifold is diffeomorphic to R n(n-1)/2 , we argue that there should exist simpler Hadamard or flat metrics on this manifold. Therefore, we define three new geometries satisfying these requirements in Chapter 7 and two flat geometries that are additionally invariant under permutations in Chapter 8. The geometric operations of the former metrics can be computed in closed form modulo Cholesky decomposition, as well as the geometric operations of the latter metrics modulo the computation in logarithmic time complexity of the diffemorphisms towards the Euclidean spaces.

Beyond Riemannian geometry, we study the Bures-Wasserstein stratified geometry of covariance matrices in Part V. We complete the work of [Takatsu, 2011] by viewing this space as the Riemannian orbit space [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF] of square matrices quotiented by the right action of the orthogonal group. After giving complements on the geometry of each stratum following [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF] and [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF], we characterize all the minimizing geodesic segments between two covariance matrices of any rank.

We conclude the thesis in Part VI and we identify some perspectives for future works on correlation matrices, stratification by eigenvalue multiplicity or spaces of matrices of different sizes.

Part VII is an appendix composed of Chapter 11, which gathers all the proofs of the results of the thesis. We decided to defer to this chapter almost all the proofs for the sake of brevity and readability of the other chapters. Only the very short proofs or the ones that we considered as essential to understand the chapter were kept in their original place. Chapter 11 is self-contained, all the results are recalled before their proofs. For readers interested in reading the proof immediately, it suffices to click on "Y " in the sentence "See the proof of Result X in Section Y." below the result. Then it suffices to click on the number "X " of the result to come back to the result within the concerned chapter.

Summary of contributions by chapter

Part II. Mathematical preliminaries

Chapter 2. Regularity of eigenvalues and eigenvectors. On the set of symmetric matrices with distinct eigenvalues Sym = (n), we prove that the function of ordered eigenvalues val : Sym = (n) -→ Diag(n) is smooth and that there exist local smooth maps vec :

U ⊂ Sym = (n) -→ O(n) satisfying vec(Σ)val(Σ)vec(Σ) = Σ for all Σ ∈ Sym = (n) that
cannot be extended globally to Sym = (n) in a continuous way. We show an inequality on Sym(n) that is weaker than a continuity result but that allows us to prove the continuity of our Riemannian metrics in Chapter 4.

Chapter 3. Characterization of invariant inner products. We give a general characterization of invariant inner products on a completely reducible Euclidean space. We apply it to the congruence action on square matrices of the orthogonal group O(n), the permutation group S(n) and the block-orthogonal group O(I) where I = {k 1 , ..., k p } is a partition of n, i.e. k i ∈ {1, ..., n} such that p i=1 k i = n. We use the characterization of S(n)-invariant inner sign and is unbounded from above. We show that the manifold of SPD matrices can be endowed via the Cholesky map with a Lie group structure under which the affine-invariant metric is a left-invariant metric. This allows us to generalize the construction of quotientaffine metrics. However, unsatisfied by these complex geometries, we propose three families of much more convenient Hadamard metrics on full-rank correlation matrices by pullback via the Cholesky map. This chapter was submitted to the SIAM Journal on Matrix Analysis and its Applications in January 2022 under the title "Theoretically and computationally convenient geometries on full-rank correlation matrices" [START_REF] Thanwerdas | Theoretically and computationally convenient geometries on full-rank correlation matrices[END_REF].

Chapter 8. Permutation-invariant Log-Euclidean metrics. We define two new families of permutation-invariant log-Euclidean metrics on full-rank correlation matrices called the off-log metrics and the log-scaled metrics. We define a natural involution called the cor-inversion and we show that the log-scaled metrics are inverse-consistent. We provide all the Riemannian operations in closed form modulo the computation of the bijections between the open elliptope and the vector spaces in consideration. We propose a new algorithm to compute the scaling of an SPD matrix and thus the log-scaled bijection.

Part V. Stratified spaces of covariance and correlation matrices

Chapter 9. Bures-Wasserstein stratified geometry of covariance matrices. We explain the global stratified geometry of covariance matrices endowed with the Bures-Wasserstein distance relying on the theory of Riemannian orbit spaces. We complete the literature on geodesics in the strata of covariance matrices of fixed rank, specifying the injectivity domain of the exponential map, the set of preimages, the set of logarithms and giving explicit expressions based on the expression of the horizontal lift. We contribute the expression of all minimizing geodesics between two covariance matrices of any rank, we give the exact number of geodesics and we provide the expression of a canonical geodesic between any two covariance matrices.

Part VI. Conclusion

Chapter 10. Conclusion and perspectives. We conclude the thesis and we propose some perspectives for future works.

Part VII. Appendix

Chapter 11. Proofs. For readability, we decided to defer to this chapter almost all the proofs that were not essential to the comprehension of the chapters, that is most of them. After each result, a sentence "See the proof of Result X in Section Y." invites the reader to click on "Y " to see the proof. The result is restated there with the complete proof. It suffices to click on "X " there to go back to the result in the chapter.

Part II

Mathematical preliminaries

Chapter 2

Regularity of eigenvalues and eigenvectors

Abstract

Some functions on symmetric matrices are defined via an eigenvalue decomposition. In this context, it is important to know the regularity of eigenvalues and eigenvectors to infer the regularity of such functions. In this work, we recall and show several results on this topic with an elementary approach. We show that there is no global continuous map of eigenvectors on the subspace of symmetric matrices with distinct eigenvalues. However, we show that there exist a global smooth map of eigenvalues and local smooth maps of eigenvectors on this subspace. We explain how it completes the literature. We also show an inequality on the whole space of symmetric matrices involving their eigenvector matrices, chosen to satisfy a condition of proximity. This inequality allows us to prove the continuity of a map defined via an eigenvalue decomposition, which seems to be new.

Following our policy on when to keep or to defer proofs, we kept the majority of the proofs within this chapter because they help understand the results.

Introduction

In the context of differential geometry on subspaces of symmetric matrices such as Symmetric Positive Definite (SPD) matrices or correlation matrices, one may want to define objects that depend on the eigenvalues or the eigenvectors of a symmetric matrix. For example, let F be a generic space, e.g. a topological space or a smooth manifold, and let f : O(n)×Diag(n) -→ F be a function with some regularity, e.g. continuous or smooth. We assume that f is invariant under the choice of an eigenvalue decomposition: for all

P, Q ∈ O(n), for all D, ∆ ∈ Diag(n), if P DP = Q∆Q , then f (P, D) = f (Q, ∆).
We say that f is a spectral map. Therefore, one can define the quotient map f : Σ = P DP ∈ Sym(n) -→ f (P, D) ∈ F. To study the regularity of f in function of the regularity of f , it seems useful to study the regularity of eigenvalues and eigenvectors.

However, eigenvalues and eigenvectors are a priori multi-valued functions. Indeed, although symmetric matrices are diagonalizable in an orthonormal basis and their eigenvalues are real which are very nice properties, one symmetric matrix has several vectors of eigenvalues (up to permutation) and several bases of eigenvectors (up to symmetry and rotation in the eigenspaces). Thus it is worth formalizing eigenvalues and eigenvectors in terms of functions before studying their regularity.

The eigenvalues of symmetric matrices are real so it suffices to order them to neutralize the degree of freedom, that is the permutation. Therefore, we can define the map val : Sym(n) -→ Diag(n) which associates the diagonal matrix of non-decreasingly ordered eigenvalues to any symmetric matrix. It is well known that this map is 1-Lipschitz [Bhatia, 1997, Theorem III.4.4, Equation IV.62], in particular it is continuous. Now we can call "map of eigenvectors" any map vec : U ⊆ Sym(n) -→ O(n) satisfying vec(Σ)val(Σ)vec(Σ) = Σ for all Σ ∈ U, where U is an open subset of symmetric matrices. By analogy with eigenvalues, one could also neutralize the degree of freedom to define a map of eigenvectors vec : Sym(n) -→ O(n). However, such a map cannot be continuous at any Σ ∈ Sym(n) having two equal eigenvalues [Katō, 1995]. Denoting Diag = (n) and Sym = (n) = val -1 (Diag = (n)) the open subsets of diagonal matrices and symmetric matrices with n distinct eigenvalues, we show in this work that there is no continuous map of eigenvectors on Sym = (n) either. Nevertheless, we show that for all Σ ∈ Sym = (n), there exists an open neighborhood U ⊂ Sym = (n) of Σ and a continuous map of eigenvectors vec : U -→ O(n) which is even smooth. In other words, we show that there exist local smooth maps of eigenvectors on Sym = (n). This also allows us to show that the global map of increasingly ordered eigenvalues is smooth on Sym = (n).

Going back to our initial problem with the spectral map f : O(n)×Diag(n) -→ F and the quotient map f : Σ = P DP ∈ Sym(n) -→ f (P, D) ∈ F, the above results show that if f is smooth, then f is smooth on Sym = (n). Stating properties on the whole vector space Sym(n) seems much more difficult. We propose the beginning of an answer by showing a result on eigenvectors of any pair of symmetric matrices, including those with repeated eigenvalues. Given two symmetric matrices Σ, Λ ∈ Sym(n), given a matrix of eigenvectors Q ∈ O(n) of Λ, the result essentially explains how to choose the matrix of eigenvectors P ∈ O(n) of Σ that is the closest to Q. For example, if Σ = I n , the closest-to-Q matrix of eigenvalues P ∈ O(n) of Σ is clearly P = Q. These matrices P and Q satisfy an inequality that is useful to study the regularity of f on Sym(n). In particular, we show that if f is continuous, then f is continuous.

These results are close to already known results. In perturbation theory [Katō, 1995, Chapter II], these results are often formulated in function of a real parameter t ∈ R which captures the perturbation of the matrix Σ ∈ Sym(n) into Σ(t) = Σ + tX ∈ Sym(n). For example in [Katō, 1995, Theorem II.6.8] and in [Serre, 2010, Section 6.2

, Comment 3], it is stated that if a curve on symmetric matrices t ∈ I -→ Σ(t) ∈ Sym(n) is of class C k , then there exists a map of (non necessarily ordered) eigenvalues (λ 1 , ..., λ n ) : I -→ R n of class C k .
It is also stated that this does not generalize to two parameters or to eigenvectors. This is probably why we did not manage to use this kind of results to prove the continuity of f . Here in contrast, we prove regularity results depending directly on the matrix Σ (not on a real parameter), using differential calculus.

Besides perturbation theory, the book of Denis Serre [Serre, 2010] summarizes some results on eigenvalues and eigenvectors of complex/real matrices in general and more specifically of Hermitian/symmetric matrices. In [Serre, 2010, Theorem 5.3], it is stated that given a matrix M 0 with complex coefficients, given an algebraically simple eigenvalue λ 0 ∈ C of M 0 , there exist a neighborhood U of M 0 and analytic functions λ : U -→ C and x : U -→ C n such that λ(M ) is an eigenvalue of M , x(M ) is an eigenvector of M associated to λ(M ), and λ(M 0 ) = λ 0 . When M 0 ∈ Sym = (n), all eigenvalues are algebraically simple so we prove in this specific case that the map λ can even be defined globally. Another difference is that we don't use the implicit function theorem to define the map x. On the contrary, we give a semi-constructive approach in the sense that once chosen an x 0 = x(M 0 ), we explain how to build the neighborhood U and how to make a consistent choice of sign for the eigenvector x(M ) so that x : U -→ C n is smooth.

Thus, our main contribution is probably the inequality we mentioned on the whole space Sym(n) allowing to show the continuity of f . The other results bring complements to known results with an elementary point of view. In the remainder of this section, we introduce some notations. In Section 2.2, we prove that there is no continuous map of eigenvectors vec : Sym = (n) -→ O(n). In Section 2.3, we prove that there exist local Lipschitz maps of eigenvectors on Sym = (n). In Section 2.4, we prove that the map of ordered eigenvalues on Sym = (n) and the local maps of eigenvectors are not only Lipschitz but also smooth. In Section 2.5, we prove that f is continuous if f is continuous. We conclude in Section 2.6.

Notations and preliminary notions

We denote R + (resp. R * ) the set of positive (resp. non null) real numbers. We denote Mat(n) the vector space of real square matrices of size n, Sym(n) the vector subspace or real symmetric matrices, Diag(n) the vector subspace of real diagonal matrices. We denote Diag : Mat(n) -→ Diag(n) the linear operator pinching the diagonal of a matrix. The Frobenius norm is denoted • on Mat(n) and on subspaces. We denote O(n) the orthogonal group.

Definition 2.1 (Spectral map, quotient map) Let F be a set. We say that f

: O(n) × Diag(n) -→ F is a spectral map if for all P, Q ∈ O(n), for all D, ∆ ∈ Diag(n), if P DP = Q∆Q , then f (P, D) = f (Q, ∆
). This automatically defines the quotient map f : Sym(n) -→ F.

We denote Diag = (n) the subset of diagonal matrices with n distinct eigenvalues, Diag < (n) the subset of diagonal matrices with n distinct eigenvalues ordered increasingly, Diag (n) the subset of diagonal matrices with non-decreasingly ordered eigenvalues, and Diag = (n) = (Diag = (n)) c the subset of diagonal matrices with repeated eigenvalues. In other words, if D ∈ Diag = (n), then there exist i = j ∈ {1, ..., n} such that

d i = d j . Note that Diag = (n) and Diag < (n) are open in Diag(n) since they are the respective preimages of the open sets (R * ) n-1 and (R + ) n-1 by the continuous map D ∈ Diag(n) -→ (d 2 -d 1 , ..., d n -d n-1 ) ∈ R n-1 .
We denote analogously Sym = (n) the subset of symmetric matrices with n distinct eigenvalues, Sym = (n) = (Sym = (n)) c the subset of symmetric matrices with repeated eigenvalues.

We introduce the map val : Sym(n) -→ Diag (n) associating to a matrix Σ ∈ Sym(n) the diagonal matrix D = val(Σ) = Diag(d 1 , ..., d n ) of its (real) eigenvalues (with multiplicity) ordered non-decreasingly, i.e. d 1 ... d n . We recall that val is a 1-Lipschitz map.

Theorem 2.2 (val is 1-Lipschitz) [Bhatia, 1997, Theorem III.4.4, Equation IV.62] For all Σ, Λ ∈ Sym(n), val(Σ) -val(Λ) Σ -Λ . In other words, the map val : Sym

(n) -→ Diag (n) is 1-Lipschitz.
In particular, val is continuous so the set of symmetric matrices with n distinct eigenvalues Sym

= (n) = val -1 (Diag < (n)) = val -1 (Diag = (n)) is open in Sym(n).
For Σ ∈ Sym(n), we denote sp(Σ) the set of (real) eigenvalues of Σ.

Definition 2.3 (Partition of n) A partition of n is a vector I = (k 1 , ..., k p ) ∈ {1, ..., n} p such that k 1 + • • • + k p = n,
for some p ∈ {1, ..., n}. We say that I = (k 1 , ..., k p ) is finer than (or is a subdivision of) J = (l 1 , ..., l q ) and we denote I J when there exists a partition of p denoted (m 1 , ..., m q ) such that

l i = k m 1 +•••+m i-1 +1 + • • • + k m 1 +•••+m i for all i ∈ {1, ..., q}, denoting m 0 = 0.
It is a partial order on the set of partitions of n. The finest partition of n is (1, ..., 1) ∈ R n and the coarsest is (n).

Note that our notion of partition of n slightly differs from the ones in [START_REF] Groisser | Geometric foundations for scaling-rotation statistics on symmetric positive definite matrices: Minimal smooth scaling-rotation curves in low dimensions[END_REF] because our diagonal matrices are ordered while they also want to handle matrices such as Diag(0, 1, 0). Definition 2.4 (Signature) The signature of a symmetric matrix Σ ∈ Sym(n) is the vector of multiplicities of the ordered eigenvalues in val(Σ) ∈ Diag (n). It is a partition of n. Signatures inherit the order of partitions of n. Definition 2.5 (Maximal radius to coarser signature) The maximal radius to coarser signature is the map defined by ρ :

Σ ∈ Sym(n) -→ 1 √ 2 min λ =µ∈sp(Σ) |λ -µ| ∈ (0, +∞].
It is the largest number r such that the symmetric matrices in the open ball of Sym(n) centered on Σ of radius r have a finer signature than Σ. The closed ball of radius ρ(Σ) contains a symmetric matrix of strictly coarser signature. For all Σ ∈ Sym = (n), it means that d(Σ, Sym = (n)) = ρ(Σ) [START_REF] Breiding | On the Geometry of the Set of Symmetric Matrices with Repeated Eigenvalues[END_REF]. Note that ρ(λI n ) = +∞.

More precisely, the open ball of center Σ ∈ Sym = (n) and radius r > 0:

• is included in Sym = (n) if and only if r ρ(Σ),

• is adherent to Sym = (n) without intersecting it if and only if r = ρ(Σ),

• intersects Sym = (n) if and only if r > ρ(Σ).

No continuous map of eigenvectors on Sym = (n)

We start with a lemma that we use several times in the chapter.

Lemma 2.6 Let P, Q ∈ O(n) such that P -Q < √ 2. Then Diag(P Q) > 0.
Proof. We denote

P = [p 1 • • • p n ] and Q = [q 1 • • • q n ] with p i , q i ∈ R n . Since p i -q i 2 = p i 2 + q i 2 -2 p i |q i = 2 -2 p i |q i and p i -q i P -Q < √ 2, we have 1 -p i |q i < 1 so [P Q] ii = p i q i = p i |q i > 0. Hence Diag(P Q) ∈ Diag + (n).
Theorem 2.7 (No continuous map of eigenvectors on Sym = (n)) Let n 2. There exists no continuous map vec :

Sym = (n) -→ O(n) such that for all Σ ∈ Sym = (n), vec(Σ)val(Σ)vec(Σ) = Σ.
See the proof of Theorem 2.7 in Section 11.1. This forces to define maps of eigenvectors only locally around symmetric matrices with n distinct eigenvalues. We use the distance from Σ ∈ Sym = (n) to Sym = (n) to define appropriate neighborhoods around symmetric matrices with n distinct eigenvalues and local Lipschitz maps of eigenvectors in Sym = (n).

Local Lipschitz maps of eigenvectors in Sym

= (n)
Given Σ ∈ Sym = (n), we would like to define a Lipschitz map of eigenvectors vec : U ⊂ Sym = (n) -→ O(n) on a neighborhood U around Σ. In particular, for all Λ ∈ U, we would like to have an equality of the form vec(Σ) -vec(Λ) c U Σ -Λ . First, we find a majoration of this type. Second, we show how it helps define a map of eigenvectors around Σ.

Majoration of the distance between eigenvector matrices in

Sym = (n)

Theorem 2.8 (Majoration of the distance between eigenvector matrices in Sym

= (n)) For all Σ ∈ Sym = (n), for all Λ ∈ Sym(n), for all matrices of eigenvectors P, Q ∈ O(n) such that Σ = P val(Σ)P and Λ = Q val(Λ)Q , if the matrix U = P Q has non-negative diagonal entries, then P -Q 2 ρ(Σ) Σ -Λ .
Corollary 2.9 (Existence of eigenvector matrices satisfying the majoration in Sym

= (n)) Given Σ ∈ Sym + (n), Λ ∈ Sym(n) and P ∈ O(n) (resp. Q ∈ O(n)) satisfying Σ = P val(Σ)P (resp. Λ = Q val(Λ)Q ), there exists Q = Q Λ,P ∈ O(n) (resp. P = P Σ,Q ∈ O(n)) such that Λ = Q val(Λ)Q (resp. Σ = P val(Σ)P ) and P -Q 2 ρ(Σ) Σ -Λ .
Proof of Corollary 2.9. Given Σ, Λ and

P = [p 1 • • • p n ], take Q = [q 1 • • • q n ] ∈ O(n) such that Λ = Q val(Λ)Q
and change the signs of the q i 's such that for all i ∈ {1, ..., n}, p i q i 0. If Q is given instead, choose the signs of p i 's the same way.

Proof of Theorem 2.8. We denote D = val(Σ) and ∆ = val(Λ). On the one hand, using Theorem 2.2 at line (2.1) and triangular inequality at line (2.2):

2 Σ -Λ Σ -Λ + D -∆ = P DP -Q∆Q + Q(D -∆)Q (2.1) P DP -QDQ = P (P DP -QDQ )Q = DU -U D . (2.2) Hence: 4 Σ -Λ 2 DU -U D 2 = i,j [DU -U D] 2 ij = i,j (d i -d j ) 2 U 2 ij (2.3) = i =j (d i -d j ) 2 U 2 ij (2.4) 2ρ(Σ) 2 i =j U 2 ij = 2ρ(Σ) 2 Ç i,j U 2 ij - i U 2 ii å = 2ρ(Σ) 2 tr(I n -Diag(U ) 2 ). (2.5)
On the other hand:

P -Q 2 = P (P -Q) 2 = I n -U 2 = 2tr(I n -U ) (2.6) Since U ∈ O(n), each column of U is of norm 1 so all entries belong to [-1, 1]. The hypothesis ensures that for all i ∈ {1, ..., n}, U ii 0 so U ii ∈ [0, 1]. Hence U 2 ii U ii so tr(Diag(U ) 2 ) = i U 2 ii i U ii = tr(U ). Finally: 4 ρ(Σ) 2 Σ -Λ 2 2tr(I n -Diag(U ) 2 ) 2tr(I n -U ) = P -Q 2 , (2.7) so P -Q 2 ρ(Σ) Σ -Λ .
Remark 2.10 Note that these results are valid for Λ ∈ Sym(n), not only for Λ ∈ Sym = (n).

Existence of local Lipschitz maps of eigenvectors in Sym

= (n) Theorem 2.11 (Existence of local Lipschitz maps of eigenvectors in Sym = (n)) For all Σ ∈ Sym = (n), there exists an open ball U = B(Σ, ρ(Σ) 2 √ 2 ) ⊂ Sym = (n) around Σ on which there exists a Lipschitz map of eigenvectors vec = vec U : U -→ O(n), i.e. for all Λ ∈ U, vec(Λ)val(Λ)vec(Λ) = Λ. In other words, there exists a constant c U ∈ R + such that for all Λ, Λ ∈ U, vec(Λ) -vec(Λ ) c U Λ -Λ . Proof. Let Σ ∈ Sym = (n). Let U be the open ball centered on Σ of radius ε = ρ(Σ) 2 √ 2 . Since ε ρ(Σ), U ⊂ Sym = by Definition 2.5. We choose P = [p 1 • • • p n ] ∈ O(n)
such that Σ = P val(Σ)P . There are 2 n such matrices because each p i can be multiplied by -1: for example, we can choose the one for which on each column, the first non null coefficient is positive.

Let Λ ∈ U. Among the

2 n matrices Q = [q 1 • • • q n ] ∈ O(n) such that Λ = Q val(Λ)Q
, there is only one such that Diag(P Q) 0. Indeed by Corollary 2.9, such a matrix Q exists and by Theorem 2.8, it satisfies P -Q

2 ρ(Σ) Σ-Λ < 2 ρ(Σ) ε = 1 √ 2 <
√ 2 so by Lemma 2.6, Diag(P Q) > 0. In other words, p i q i cannot be null (because Λ ∈ U) so the signs of q i 's are imposed by P . Thus we can define the map of eigenvectors vec

= vec U : Λ ∈ U -→ Q ∈ O(n) where Q is the unique orthogonal matrix such that Λ = Q val(Λ)Q and Diag(P Q) > 0. Let us show that vec is Lipschitz. Let Λ, Λ ∈ U and Q = vec(Λ), Q = vec(Λ ) ∈ O(n)
. By triangular inequality and by Theorem 2.8 applied to (Σ, Λ, P, Q) and (Σ, Λ , P, Q ), we have:

Q -Q P -Q + P -Q 2 ρ(Σ) ( Σ -Λ + Σ -Λ ) < 4ε ρ(Σ) = √ 2. (2.8)
Hence, by Lemma 2.6, for all i ∈ {1, ..., n}, q i |q i > 0. Therefore, by Theorem 2.8 applied to

(Λ, Λ , Q, Q ), we have Q-Q 2 ρ(Λ) Λ-Λ . Finally, inf Λ∈U ρ(Λ) = inf Λ∈U d(Λ, Sym = (n)) = ρ(Σ)-ε = 2 √ 2-1 2 √ 2 ρ(Σ) > 0 so we can define c U = sup Λ∈U 2 ρ(Λ) = 4 √ 2 2 √ 2-1 1 ρ(Σ) < +∞ and conclude that the map vec : U -→ O(n) is c U -Lipschitz. Note that c U 2 ρ(Σ)
tends to +∞ when Σ tends to a matrix in Sym = (n) since ρ(Σ) tends to 0.

Smoothness of eigenvalues and local eigenvectors

on Sym = (n)

In this section, we prove the smoothness of the map of eigenvalues on symmetric matrices with distinct eigenvalues, val |Sym = (n) : Sym = (n) -→ Diag < (n), and the smoothness of the local maps of eigenvectors vec :

U = B(Σ, ρ(Σ) 2 √ 2 ) ⊂ Sym = (n) -→ O(n).
Theorem 2.12 (Smoothness of the map val on Sym = (n) and smoothness of the maps vec) The maps

val |Sym = (n) : Sym = (n) -→ Diag < (n) and vec : U = B(Σ, ρ(Σ) 2 √ 2 ) -→ O(n) are smooth. Their differentials at Σ ∈ U ⊂ Sym = (n) are for all X ∈ Sym(n), denoting D = val(Σ) = diag(d 1 , ..., d n ) and P = vec(Σ): d Σ val(X) = Diag(P XP ),
(2.9)

d Σ vec(X) = P A,
(2.10)

where

A ∈ Skew(n) is defined by A ij = [P XP ] ij d j -d i
for all i = j (and A ii = 0).

Proof. Let us show that if val |Sym = (n) and vec are differentiable, then necessarily their differentials satisfy Equations (2.9) and (2.10). We differentiate the equality Σ (2.11)

= vec(Σ) val(Σ) vec(Σ) in the direction X ∈ Sym(n), denoting Ṗ = d Σ vec(X) ∈ T P O(n) = P Skew(n) and Ḋ = d Σ val(X) ∈ T D Diag(n) = Diag(n): X = Ṗ DP + P D Ṗ + P ḊP ,
P XP = P Ṗ D + D Ṗ P + Ḋ.
(2.12)

We denote A = P Ṗ ∈ Skew(n). Then P XP = AD -DA + Ḋ so Ḋ is the diagonal part of P XP and AD -DA = [A ij (d j -d i )] 1 i,j n is the off-diagonal part, as expected.

Let us show that val |Sym = (n) and vec are differentiable. Let Σ ∈ U and X ∈ Sym(n). Now we denote Ḋ = Diag(P XP ) and Ṗ = P A with A ∈ Skew(n) defined by

A ij = [P XP ] ij d j -d i for all i = j. Let us show that: val(Σ + tX) = D + t Ḋ + o(t),
(2.13) To do so, it suffices to define Σ(t) = P (t)D(t)P (t) with P (0) = P , D(0) = D, Ṗ (0) = Ṗ = P A and Ḋ(0) = Ḋ. For example, let P (t) = P exp(tA) and

vec(Σ + tX) = P + t Ṗ + o(t
D(t) = D + t Ḋ. For t sufficiently small, D(t) ∈ Diag < (n) since Diag < (n) is open so D(t) = val(Σ(t)
). For t sufficiently small, P P (t) = exp(tA) has positive diagonal since exp(tA) = I n + o(1), so P (t) = vec(Σ(t)). Hence:

val(Σ + tX) = val(Σ(t)) + o(t) = D(t) + o(t) = D + t Ḋ + o(t), (2.15) vec(Σ + tX) = vec(Σ(t)) + o(t) = P (t) + o(t) = P + t Ṗ + o(t) (2.16)
So val and vec are differentiable on U.

Let us assume that val and vec are differentiable k times on U. We denote L(E, F ) the vector space of linear maps from the vector space E to the vector space F . Then for all i = j, the map Mat(n)) are differentiable k times, so val and vec are differentiable k +1 times. Since val and vec are differentiable on U, by recurrence they are smooth on U. Since val |Sym = (n) is locally smooth, it is globally smooth.

A ij : Σ ∈ U -→ Ä X ∈ Sym(n) -→ [vec(Σ) Xvec(Σ)] ij [val(Σ)] jj -[val(Σ)] ii ä ∈ L(Sym(n), R) is differentiable k times so A : U -→ L(Sym(n), Skew(n)) is differentiable k times. Since d Σ val(X) = Diag(vec(Σ) Xvec(Σ)) and d Σ vec(X) = vec(Σ)A(Σ, X), the maps d val : U -→ L(Sym(n), Diag(n)) and d vec : U -→ L(Sym(n),
We have the following immediate corollary.

Corollary 2.13 (Smoothness of a spectral map) Let M be a smooth manifold, let f : O(n) × Diag(n) -→ M be a smooth spectral map and let f : Sym(n) -→ M be the associated quotient map. Then f|Sym = (n) is smooth.

Proof. It suffices to write f|U = f • (vec U , val |U ).
Note that Corollary 2.13 could already be deduced from [Serre, 2010, Theorem 5.3]. The notable results up to now are the existence of a global smooth map of eigenvalues on Sym = (n) and the fact that the choice of the neighborhood and the definition of the smooth map of eigenvectors rely on the assumption Diag(vec(Σ) vec(Λ)) > 0. In the next section, we generalize Theorem 2.8 to symmetric matrices with repeated eigenvalues, that is we give an analogous inequality with a more general condition.

Majoration of the distance between eigenvector matrices in Sym(n)

Back to our initial problem, when one wants to prove regularity properties on objects defined via eigenvalue decomposition, it is important to know how to choose the decomposition. The answer is now rather clear on Sym = (n). However, the regularity on the whole vector space Sym(n) is more difficult to handle than on the open set Sym = (n). Although there is no local continuous map of eigenvectors around Σ ∈ Sym = (n), given a symmetric matrix Λ close to Σ, there is a way of choosing matrices of eigenvectors P, Q that are the closest possible. We explain this choice in the following result. It allows us to give the beginning of an answer to our initial problem. Indeed, we further prove that if the spectral map is continuous, then the corresponding function defined on symmetric matrices is continuous on Sym(n).

This condition generalizes Diag(P Q) > 0. The following corollary ensures that given Q, there exists P satisfying this condition.

Corollary 2.15 (Existence of eigenvector matrices satisfying the majoration in Sym

(n)) Given such Σ, Λ ∈ Sym(n) and Q ∈ O(n) such that Λ = Q vec(Λ)Q , there exists P = P Σ,Q ∈ O(n) such that Σ = P val(Σ)P and P -Q 2 ρ(Σ) Σ -Λ .
The proofs consist in replacing Diag(U ) = diag(U 11 , ..., U nn ) in the proof of Theorem 2.8 by the block-diagonal matrix W = Diag(U 1 , ..., U p ) pinched from U .

Proof of Corollary 2.15. Given Σ, Λ, Q, take P such that Σ = P val(Σ)P . Let S ∈ Sym + (n) and R ∈ O(n) be block diagonal matrices of same signature as W such that W = SR . Hence R val(Σ)R = val(Σ) so P can be replaced by P R. Since R W = S, the diagonal blocks of the new U := (P R) Q = R U are the diagonal blocks of R W . Hence P R satisfies the hypothesis of Theorem 2.14.

Remark 2.16 Beware that it is not true that given Σ, Λ ∈ Sym(n) and P ∈ O(n) such that Σ = P val(Σ)P , there always exists an appropriate Q. This is because the symmetry between Σ and Λ is broken by the (arbitrary) choice of ρ(Σ) rather than ρ(Λ). Therefore, the condition on P Q relies on the signature (n 1 , ..., n p ) of Σ (see proof of Theorem 2.14). Since the signature of Λ can be completely different, once any eigenvector matrices P and Q of Σ and Λ are respectively chosen, we can only modify P in general so that P Q satisfies the condition of positivity (see proof of Corollary 2.15).

A counter-example of the existence of Q = Q Λ,P is Σ = 0 (hence ρ(Σ) = +∞), Λ = diag(t, -t) with t = 0 and P = Å cos θ ± sin θ sin θ ∓ cos θ ã with θ ∈ R\ π 2 Z. However, if the signature of Σ is finer than the signature of Λ, then Q can be adapted so that P Q satisfies the condition. Hence in this case, it is true that given Σ, Λ, P , there exists an appropriate Q. This was the case in Theorem 2.8 because Σ ∈ Sym = (n), i.e. the signature of Σ was (1, ..., 1) which is the finest signature. This allows to explain Remark 2.10.

Proof of Theorem 2.14. Lines (2.1) to (2.3) of the previous proof are still valid. Hence:

4 Σ -Λ 2 i,j|d i =d j (d i -d j ) 2 U 2 ij (2.17) 2ρ(Σ) 2 i,j|d i =d j U 2 ij = 2ρ(Σ) 2 i,j U 2 ij - λ∈sp(Σ) i,j|d i =d j =λ U 2 ij (2.18) = 2ρ(Σ) 2 tr Ç U U - p k=1 U k U k å = 2ρ(Σ) 2 tr(I n -W W ). (2.19) Moreover, line (2.6) is still valid so P -Q 2 = 2tr(I n -U ) = 2tr(I n -W ).
The equality U U = I n ensures that for all k ∈ {1, ..., p}, 0 U k U k I n k for the Loewner order. Indeed, for k = 1 for example, if we write

U = Å U 1 A B C ã with A, B, C of appropriate sizes, then I n = U U = Å U 1 U 1 + AA * * * ã so U 1 U 1 + AA = I n 1 . Hence U 1 U 1 0 and I n 1 -U 1 U 1 0 as claimed. Otherwise said, 0 W W I n , i.e. the eigenvalues of W W are in [0, 1]. Hence W W √ W W . The hypothesis ensures that W is symmetric positive semi-definite so W = √ W W . Finally, tr(W ) = tr( √ W W ) tr(W W ) so 4 ρ(Σ) 2 Σ -Λ 2 2tr(I n -W W ) 2tr(I n -W ) = P -Q 2 as expected.
Remark 2.17 Theorem 2.14 is not a continuity result. Let Q(t) be a curve on O(n) for t ∈ (0, 1] and D(t) be a curve on Diag(n

) for t ∈ [0, 1] with D = D(0) ∈ Diag = (n). Assume that Λ(t) = Q(t)D(t)Q(t) converges to Σ ∈ Sym = (
n) when t tends to 0. The result does not mean that Q(t) converges. It only states that for all t ∈ (0, 1], one can find P ∈ O(n) such that Σ = P DP and P -Q(t)

2 ρ(Σ) Σ -Λ(t)
. So the choice of P depends on t! Thus, this does not contradict the fact that eigenvectors are not continuous at symmetric matrices with repeated eigenvalues Σ ∈ Sym = (n).

Theorem 2.18 (Continuity of a spectral function) Let F be a topological space, let f : O(n) × Diag(n) -→ F be a continuous spectral map and let f : Sym(n) -→ F be the associated quotient map. Then f is continuous.

Proof. Let Σ ∈ Sym(n) and let ε > 0. By Heine's theorem, f is uniformly continuous on O(n) × B(val(Σ), 1) where B is a closed ball in Diag(n). Hence, let δ > 0 such that if (P, D) -(Q, ∆) = P -Q 2 + D -∆ 2 < δ, then f (P, D) -f (Q, ∆) < ε. Let η = min(1, ρ(Σ) 2 ) δ √ 2 . Let Λ ∈ Sym(n) such that Σ -Λ < η. By Corollary 2.15, let P, Q ∈ O(n) such that Σ = P val(Σ)P , Λ = Q val(Λ)Q and P -Q 2 ρ(Σ) Σ -Λ . Then P -Q δ √ 2 and val(Σ) -val(Λ) Σ -Λ δ √ 2 . Therefore, (P, D) -(Q, ∆) = P -Q 2 + D -∆ 2 < δ so f (P, D) -f (Q, ∆) < ε. For all Λ ∈ Sym(n), if Σ -Λ < η, then f (Σ) -f (Λ) < ε so f is continuous at Σ. So f is continuous.
We said that Theorem 2.14 only gives the beginning of an answer because it would be interesting to know sufficient conditions on f so that f is of class C k on Sym(n). We think that there must be conditions on the limit on the differential of f at points (P, D) ∈ O(n) × Diag = (n).

Conclusion

We proved that increasingly ordered eigenvalues are smooth on the open subset of symmetric matrices with distinct eigenvalues Sym = (n). We showed that there is no global continuous map of eigenvectors but there exist local smooth maps of eigenvectors. This allowed to show that the quotient map of a smooth spectral map is smooth on Sym = (n). We also proved an inequality on the whole vector space of symmetric matrices, allowing to show that the quotient map of a continuous spectral map is continuous. We will use these results in Chapter 4. A more difficult problem is to find conditions on the smooth spectral map f and on its successive differentials for the associated spectral map f to be smooth. Such a result would be very appealing.

Introduction

When one looks for appropriate metrics on a given space representing some data, it is natural to require them to be invariant under a certain group action. For example, when data are represented by Symmetric Positive Definite (SPD) matrices, one can use Riemannian metrics that are invariant under the congruence action of the general linear group (affineinvariant metrics [Skovgaard, 1984, Pennec et al., 2006, Lenglet et al., 2006, Fletcher and Joshi, 2007, Moakher, 2005, Batchelor et al., 2005]), the orthogonal group [START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]] (e.g. log-Euclidean [START_REF] Arsigny | Log-Euclidean metrics for fast and simple calculus on diffusion tensors[END_REF], Bures-Wasserstein [Bhatia et al., 2019, Dryden et al., 2009, Takatsu, 2011, Malagò et al., 2018], Bogoliubov-Kubo-Mori metrics [Petz andToth, 1993, Michor et al., 2000]), the group of positive diagonal matrices [START_REF] David | A Riemannian structure for correlation matrices[END_REF], David, 2019, Thanwerdas and Pennec, 2021], the permutation group. In several situations, the question of finding all G-invariant Riemannian metrics on a manifold reduces to finding all H-invariant inner products on a vector space where H is a Lie subgroup of G. For example, bi-invariant metrics (or pseudo-metrics) on a Lie group G are characterized by Ad(G)-invariant inner products (or non-degenerate symmetric bilinear forms) on its Lie algebra g. On a homogeneous space M = G/H, G-invariant metrics are characterized by Ad(H)-invariant inner products on the tangent space m = T H M at the equivalence class H ∈ G/H.

Another frequent case is when the manifold M is diffeomorphic to a vector space V . For example, the cone of SPD matrices is diffeomorphic to the vector space of symmetric matrices by the symmetric matrix logarithm log : Sym + (n) -→ Sym(n). Similarly, the elliptope of full-rank correlation matrices Cor + (n) is diffeomorphic to a vector space of dimension n(n-1) 2 . Hence, given a G-equivariant diffeomorphism φ : M -→ V between the manifold M and a vector space V , the G-invariant inner products on V provide natural flat G-invariant Riemannian metrics on M by pullback. These three examples of bi-invariant metrics on Lie groups, Riemannian homogeneous manifolds and "Euclideanized" manifolds motivate to find all G-invariant inner products on a vector space V . Indeed, if one wants to impose an invariance on a space, this requirement defines a family of metrics in general, rarely a unique metric. Therefore, there is no reason a priori to distinguish between all the metrics that satisfy this requirement. Then, it is possible to reduce the choice by requiring other invariance or constraints, or to optimize within the family in function of the data for example.

To answer this question, the central notion is the reducibility or irreducibility of a vector space under a group action. We say that V is G-irreducible when there is no other subvector space than {0} and V that is stable under the action of G. It can easily be proved that inner products on an irreducible space are positive scalings of one another. Hence, when V is completely reducible, i.e. V can be expressed into a direct sum of irreducible subspaces, then any positive linear combination of inner products on each irreducible subspace is an invariant inner product. For example, if G = {-1, 1} × {-1, 1} acts on V = R 2 = R ⊕ R component by component, then one can easily check that all G-invariant inner products are given by ϕ((x, y), (x, y)) = αx 2 + βy 2 with α, β > 0.

However, there are other invariant inner products in general. Take another example where G = {-1, 1} acts on V = R 2 = R ⊕ R globally. It is clear that all the G-invariant inner products are given by ϕ((x, y), (x, y)) = αx 2 + 2γxy + βx 2 with α, β > 0 and αβ > γ 2 . In the first example, G acts differently on each copy of R: for example (1, -1) • (x, y) = (x, -y). In the second example, G acts identically on each copy of R, they are indistinguishable with respect to the action. This is why another coefficient is allowed between the two components. Therefore, in the irreducible decomposition, one has to group all the irreducible spaces on which G acts the same way to find all G-invariant inner products. That is what we explain in Section 3.2 and what is summarized in Theorem 3. 4. In this work, we assume that we know a G-invariant inner product on V and that V is completely reducible, i.e. there exists an irreducible decomposition of V . This is the case when G is finite [Artin, 2011, Maschke's Theorem 10.2.10] and also when the group action is unitary and continuous [Artin, 2011, Corollary 10.3.5]. In particular, when G is finite or compact, it is well known that one can find one G-invariant inner product by taking any inner product and averaging over G with the counting measure or the Haar measure µ: x|y := 1 |G| a∈G ax • ay or G (ax • ay)µ(da). So the two hypotheses are automatically satisfied when G is finite or when G is compact and the action is continuous. The examples we give all belong to one of these two situations although the method we give applies more generally as soon as the two mentioned hypotheses are satisfied together.

Our method relies on basic representation theory. Indeed, any G-invariant inner product on V is characterized, via the given inner product •|• , by a G-invariant automorphism of V . Then, representation theory allows to find the general form of this automorphism, hence the general form of G-invariant inner products on V . Note that this characterization of invariant inner products is well known in algebra although we could not find it explicitly in the references we read. Yet, it seems to be much less known in the communities of applied mathematics which use differential geometry, although it seems quite useful. That is why we present it with the fewest details on representation theory and why we study thoroughly some examples.

The general method is exposed in Section 3.2. In the examples, we focus on the actions of the orthogonal group, the block orthogonal group and the permutation group. In Section 3.3, we apply this method to several examples. In Section 3.3.1 we recall the general form of invariant inner products on R n under these three group actions. In Section 3.3.2, we recall the general form of invariant inner products on square matrices under the congruence action of the orthogonal group. Then in Section 3.3.3, we characterize invariant inner products on square matrices under the block orthogonal group. In particular, this allows to characterize O(n)-invariant metrics on the homogeneous space of flags Flag(I) = O(n)/O(I) where I is a partition of n, which is new. Finally in Section 3.3.4, we characterize permutationinvariant inner products on square matrices. We give a natural instantiation of the irreducible decomposition with coordinate-free orthogonal projections. This allows to define Euclidean metrics on SPD matrices and correlation matrices. We summarize and conclude in Section 3.4.

Characterization of invariant inner products of a Euclidean space using representation theory

A representation of a group G on a vector space V is a group homomorphism ρ : G -→ GL(V ). It is equivalent to a linear group action of G on V , i.e. a map ρ : G × V -→ V such that for all a, b ∈ G and all x ∈ V , ρ(a, ρ(b, x)) = ρ(ab, x), ρ(e, x) = x (where e is the neutral element of G) and ρ a :

x ∈ V -→ ρ(a, x) ∈ V is linear. Let (V, •|•
) be a Euclidean space (i.e. a real vector space of finite dimension endowed with an inner product), V = {0}. We denote • the associated norm. The set O

(V ) = {f ∈ GL(V )|∀x ∈ V, f (x) = x } is the subgroup of isometries of GL(V ). Let ρ : G -→ O(V )
be a representation of a group G acting isometrically on V .

Preliminaries

Vocabulary of representation theory

The following terminology is neither unique in representation theory nor exclusive to this branch of mathematics. We use the term "module" for simplicity, in reference to "G-module" which is frequent. (In particular, a "module" here does not designate an abelian group (M, +) endowed with a monoid action (R, •) × M -→ M that is distributive on additions, given a ring (R, +, •).)

Definition 3.1 (Vocabulary of representation theory)

• A (G-)module is a vector space W on which G acts continuously. For instance, the vector space V introduced above is a G-module. We omit the group G when it is clear from the context.

• A submodule of W is a G-stable subvector space of W .

• A module homomorphism (resp. endomorphism, isomorphism, automorphism) is a ρequivariant linear map (resp. endormorphism, isomorphism, automorphism) between modules. Given modules W, W , we denote W W when there exists a module isomorphism from W to W and we say that W and W are isomorphic modules.

• A module W is irreducible if it has no other submodules than {0} and W . Otherwise, it is reducible.

• A module W is completely reducible if it is the direct sum of irreducible modules.

From now on, we assume that V is completely reducible. For example, this is the case when G is finite [Artin, 2011, Maschke's Theorem 10.2.10] or when ρ is unitary and continuous [Artin, 2011, Corollary 10.3.5].

Note that if W is a submodule of V , then W ⊥ (where orthogonality refers to the inner product

•|• ) is also a submodule of V . Indeed, if x ∈ W ⊥ and a ∈ G, then for all y ∈ W , ρ(a -1 )(y) ∈ W because W is a module so ρ(a)(x)|y = x|ρ(a -1 )(y) = 0, so ρ(a)(x) ∈ W ⊥
. Therefore, we can assume that the decomposition of V into irreducible submodules is orthogonal.

The big picture

We split the method into three steps:

1. Transform the problem of finding all ρ-invariant inner products on V into finding all ρ-equivariant automorphisms of V .

2. Find all ρ-equivariant automorphisms of V using representation theory.

3. Go back to the initial problem.

Before the general case (V completely reducible), it is natural to start with the particular case where V is irreducible (Section 3.2.3.1). Then, we explain why the general case does not reduce to the irreducible case but only to an intermediate case (Section 3.2.3.2). Therefore, we treat this intermediate case (Section 3.2.3.3) and we conclude with the general case (Section 3.2.3.4). So we do the 1 st step globally (Section 3.2.2), then we do the 2 nd and 3 rd steps for each case (Section 3.2.3).

Step 1: from invariant inner products to equivariant automorphisms

Let ϕ be an inner product on V . We introduce the musical isomorphisms of ϕ:

• "Flat" ϕ : x ∈ V -→ ϕ(x, •) ∈ V *
which is a linear isomorphism by the Riesz representation theorem,

• "Sharp" # ϕ = -1 ϕ : V * -→ V the inverse linear isomorphism. We denote = •|• and # = # •|• . Then ϕ is a ρ-invariant inner product on V if and only if f ϕ := # • ϕ : V -→ V is a ρ-equivariant automorphism of V . We retrieve ϕ from f := f ϕ by ϕ(x, y) = f (x)|y for all x, y ∈ V .
The core of the method is to determine the general form of f .

3.2.3

Steps 2 and 3: find all equivariant automorphisms and all invariant inner products 3.2.3.1 The particular case: V irreducible

We start with the simplest case where V is irreducible. The key result to characterize all ρ-equivariant automorphisms of V is Schur's lemma.

Lemma 3.2 [Artin, 2011, Schur's Lemma 10.7.6] Let V, W be irreducible modules.

1. A module homomorphism f : V -→ W is either null or a module isomorphism.

A module endomorphism

f : V -→ V is a scaling, i.e. there exists α ∈ R such that f = α Id V .
Indeed, the first statement holds because f (V ) is a submodule of W so f (V ) = {0} or W and ker(f ) is a submodule of V so ker(f ) = V or {0}. The second statement holds because if α ∈ C is an eigenvalue of f , then f -α Id V is not a module isomorphism so it is null by the first statement. Since V is a real vector space and V = {0}, α has to be in R.

As a consequence, if V is an irreducible module, then:

• Module automorphisms on V are the non null scalings.

• Module isomorphisms between V and an irreducible module W are unique up to scaling. Indeed, if f, f 0 : V -→ W are module isomorphisms, then f -1 0 •f : V -→ V is a scaling, i.e. there exists α ∈ R * such that f = αf 0 .

• The ρ-invariant inner products on V are positive multiples of •|• .

The general case does not reduce to the previous case

We continue with the case where V is completely reducible. Therefore, let

V 1 , ..., V m be irreducible submodules of V such that V = V 1 ⊥ ⊕ • • • ⊥ ⊕ V m . From the previous case, it is clear that the map f : x = x 1 + • • • + x m ∈ V -→ α 1 x 1 + • • • + α m x m ∈ V with α 1 , ..., α m ∈ R * is a module automorphism of V .
The question is: are they the only ones? The answer is no and the following consequence of Schur's lemma allows to explain precisely why. Lemma 3.3 (Consequence of Schur's lemma on the irreducible decomposition) We group

V 1 , ..., V m by classes C 1 , ..., C p of isomorphic irreducible modules. Each class C k is of the form C(W ) = {j ∈ {1, ..., k}|V j W } for W ∈ {V 1 , ..., V m }. The decomposition becomes V = V 1 ⊥ ⊕ • • • ⊥ ⊕ V p with V k = ⊥ i∈C k V i . Let f : V -→ V be a module automorphism. Then f (V k ) = V k for all k ∈ {1, ..., p}. Proof. For i, j ∈ {1, ..., m}, let f ij = proj V j • f |V i : V i -→ V j . Let k ∈ {1, .
.., p} and let i ∈ {1, ..., n} such that V i ∈ C k . By Schur's lemma, for all j ∈ {1, ..., n}, f ij is null or it is an isomorphism. Since f is an isomorphism, there exists j ∈ {1, ..., m} such that f ij is non null, thus an isomorphism. Hence

V j ∈ C k so V j ⊆ V k . Therefore, f (V i ) = V j ⊆ V k . This inclusion is valid for all V i ∈ C k so f (V k ) ⊆ V k and f (V k ) = V k by equality of dimensions because f is bijective.
In other words, the study of ρ-invariant automorphisms of V cannot be reduced to the V i 's but only to the V k 's: there is no reason that f (V i ) = V i for all i ∈ {1, ..., m} (unless all classes are singletons). So we need to study the case V = V mW with W irreducible, where mW is a notation for the direct sum of m irreducible modules isomorphic to W .

The intermediate case: V mW with W irreducible

We assume that

V = V 1 ⊥ ⊕ • • • ⊥ ⊕ V m where V 1 • • • V m are isomorphic irreducible modules.
Let W be an irreducible module isomorphic to them, endowed with a G-invariant inner product (•|•). Let ψ i : V i -→ W be the unique module isomorphism which is an isometry. Indeed, the module isomorphism is unique up to scaling and the pullback of the inner product (•|•) onto V i is necessarily a scaling of the restriction of •|• to V i so there is a unique choice of ψ i such that it is an isometry. For example, W can be taken as one of the V i 's.

Let f be a module automorphism of V . We define the module endomorphisms

f ij = ψ j • proj V j • f • ψ -1 i : W -→ W
. By Schur's lemma, they are scalings: there exists S ij ∈ R such that f ij = S ij Id W . This defines a matrix S = (S ij ) 1 i,j m ∈ Mat(m). Then, f writes:

f (x) = (Id V • f )(x) = Ç m j=1 proj V j å • f Ç m i=1 x i å = m i=1 m j=1 (proj V j • f )(x i ) = m i=1 m j=1 (ψ -1 j • f ij • ψ i )(x i ) = m i=1 m j=1 S ij ψ -1 j • ψ i (x i ). (3.1)
When f comes from an inner product ϕ as explained in Section 3.2.2, we have for all x, y ∈ V :

ϕ(x, y) = f (x)|y = m i=1 m j=1 S ij ψ -1 j • ψ i (x i )|y j = m i=1 m j=1 S ij (ψ i (x i )|ψ j (y j )),
because the V i 's are orthogonal and

ψ j : (V j , •|• ) -→ (W, (•|•)
) is an isometry. This implies that S is symmetric and positive definite. Indeed, let w ∈ W \{0}, a ∈ R m and x = k i=1 a i ψ -1 i (w). Then: • if a = (1, ..., 1), then ϕ(x i , x j ) = S ij w 2 and by symmetry of ϕ, we have ϕ(x i , x j ) = ϕ(x j , x i ) = S ji w 2 so S ij = S ji ,

• we have ϕ(x, x) = i,j S ij a i a j w 2 so for all a ∈ R m \{0}, we have i,j S ij a i a j > 0.

Conversely, if S ∈ Sym + (m), then ϕ(x, y) = f (x)|y defines an inner product. It is clearly symmetric and if x = 0, there exists w ∈ W \{0} and a ∈ R m \{0} such that x = k i=1 a i ψ -1 i (w) so the equality above proves that ϕ(x, x) > 0 (for the existence of w and a, take i ∈ {1, ..., p} such that x i = 0 and define w = ψ i (x i ) and a j = ψ j (x j ) ψ i (x i ) for j ∈ {1, ..., m}).

So f is a module isomorphism of V = V 1 ⊥ ⊕• • • ⊥ ⊕V m if
and only if there exists S ∈ Sym + (m) such that f writes as in Equation (3.1). Now we have all the ingredients to state the global result.

The general case:

V completely reducible Theorem 3.4 (General form of a ρ-invariant inner product on V ) Let V = p k=1 V k , with V k = m k i=1 V k i , be an orthogonal decomposition where V k 1 • • • V k m k are irreducible modules. For all k ∈ {1, ..., p}, let (W k , (•|•) k ) be a Euclidean space and ψ k i : V k i -→
W k be the unique module isomorphism which is an isometry. Then, an inner product ϕ on V is ρ-invariant if and only if there exist p SPD matrices S k ∈ Sym + (m k ) for k ∈ {1, ..., p} such that for all x = p k=1

m k i=1 x k i ∈ V and y = p k=1 m k i=1 y k i ∈ V : ϕ(x, y) = p k=1 1 i,j m k S k ij (ψ k i (x k i )|ψ k j (y k j )) k . (3.2)
The number of parameters is

p k=1 m k (m k +1) 2
and the number of positivity constraints is p k=1 m k .

Proof. We assemble the pieces of demonstration of the previous sections together. Let ϕ be a ρ-invariant inner product on V . Then, the map

f = # • ϕ : V -→ V is a module automorphism (Section 3.2.2). Hence for all k ∈ {1, ..., p}, f (V k ) = V k (Section 3.2.3.2).
Therefore, there exists a matrix

S k ∈ Sym + (m k ) such that for all x k , y k ∈ V k , f (x k )|y k = m k i=1 m k j=1 S k ij (ψ k i (x k i )|ψ k j (y k j )) k (Section 3.2.3.3).
Hence, the inner product ϕ writes:

ϕ(x, y) = f (x)|y = p k=1 f (x k )|y k = p k=1 m k i=1 m k j=1 S k ij (ψ k i (x k i )|ψ k j (y k j )) k . ( 3
i : (V k i , •|• ) -→ (W k , (•|•) k ) instead of λ i ψ k i with λ i > 0 for all i ∈ {1, ..., m k }: it suffices to replace S k ∈ Sym + (m k ) by ΛS k Λ ∈ Sym + (m k ) where Λ = Diag(λ 1 , ..., λ m k ).

Examples

We recall that this theorem is based on the complete reducibility of V . To be applied on examples, it means that we have to find explicit irreducible decompositions. This is our main task in this section. From now on, we apply the previous theorem to several situations. The groups we focus on are:

• the orthogonal group O(n) = {U ∈ Mat(n)| U U = I n }, which is a compact subgroup
of the real general linear group GL(n),

• the block orthogonal group:

O(I) =      Ö R 1 0 . . . 0 R p è ∈ O(n)|∀i ∈ {1, ..., p}, R i ∈ O(k i )      O(k 1 ) × • • • × O(k p ),
which is a subgroup of O(n) where I = (k 1 , ..., k p ) is a "partition of n", i.e. such that k 1 + ... + k p = n with k 1 , ..., k p ∈ N * ,

• the permutation group S(n), which is a finite subgroup of O(n) denoting either the set of permutations σ of {1, ..., n} or the set of permutation matrices P σ defined by [P σ ] ij = δ i,σ(j) . (Beware that S(n) SO(n), especially transpositions are not in SO(n).)

The Euclidean modules we focus on are:

• R n with the canonical inner product and the isometric action (R,

x) ∈ O(n) × R n -→ Rx ∈ R n ,
• the vector space Mat(n) of n×n squared real matrices endowed with the Frobenius inner product and the isometric congruence action (R,

X) ∈ O(n) × Mat(n) -→ RM R ∈ Mat(n)
, which is also the tangent space of the general linear group GL(n),

• the Euclidean submodule Skew(n) = {X ∈ Mat(n)|X = -X} of skew-symmetric matrices, which is the tangent space at I n of the orthogonal group O(n),

• the Euclidean submodule Sym(n) = {X ∈ Mat(n)|X = X} of symmetric matrices, which is the tangent space of the manifold of Symmetric Positive Definite (SPD) matrices Sym + (n) = {X ∈ Sym(n)|X > 0} (where > denotes the strict Löwner order),

• (only for the action of the permutation group) the Euclidean submodule Hol(n) = {X ∈ Sym(n) | Diag(X) = 0}, which is the tangent space of the manifold of full-rank correlation matrices Cor

+ (n) = {Σ ∈ Sym + (n)|Diag(Σ) = I n }
We denote (e 1 , ..., e n ) the canonical basis of R n and E ij = e i e j + e j e i ∈ Sym(n) which form a basis of Sym(n) for 1 i j n. These examples being trivial for n = 1, we consider n 2. We structure each example as follows:

• Find the Direct sum and compute the projections,

• Prove the stability and Irreducibility of the terms in the decomposition,

• Determine classes of Isomorphic modules,

• Write the General form of invariant inner product/quadratic form.

For readibility, some proofs are moved in appendix of the thesis.

Invariant inner products on R n

In this section, we illustrate the previous theorem on simple examples. The irreducible orthogonal decompositions of R n with respect to the action of O(n), S(n) and O(I) are given in Table 3.1 for I = (k 1 , ..., k p ). The following notations are used:

• 1 ∈ R n is the vector with all entries equal to one,

• R n 0 = span(1) ⊥ = {x ∈ R n | sum(x) := n i=1 x i = x 1 = 0}, • R k i
abusively denotes the subspace of R n where the (k 1 + ... + k i-1 ) first entries and the (k i+1 + ... + k p ) last entries are null,

• p k i : R n -→ R k i denotes the canonical projection,

• O n (k i ) O(k i ) denotes the subgroup of O(I) acting on R k i . Group Decomposition O(n) R n O(I) p i=1 R k i S(n) R n 0 span (1)
Table 3.1: Irreducible orthogonal decompositions of R n with respect to several groups.

Direct sum

The decompositions are clearly direct orthogonal sums. The orthogonal projections on R n 0 and span(1

) are x ∈ R n -→ x -x|1 n 1 ∈ R n 0 and x ∈ R n -→ x|1 n 1 ∈ span(1).
Irreducibility The action of O(n) on R n is transitive so R n is stable and irreducible with respect to the action of O(n). Therefore, R k i is stable and irreducible with respect to O n (k i ), thus to O(I).

Regarding the action of S(n), the vector spaces R n 0 and span(1) are stable and span(1) is irreducible because of dimension 1. Moreover, let W ⊆ R n 0 be a submodule such that W = {0}. Let x ∈ W , then there exist i < j such that x i = x j . Hence, applying the permutation σ = (i, j), the vector e ij = e i -e j = 1

x i -x j (x -σ • x) ∈ W . Hence with a permutation τ ∈ S(n) such that τ (i) = k and τ (j) = k + 1, we have e k,k+1 = τ • e ij ∈ W for all k ∈ {1, ..., n -1}. As (e k,k+1 ) k∈{1,...,n-1} is a basis of R n 0 , we have R n 0 ⊆ W so W = R n 0 and R n 0 is irreducible.

Isomorphic modules

Regarding S(n), the two modules are clearly not isomorphic for n 3 because they have different dimensions. For n = 2, R 2 0 = span 1 -1 and the permutation (1, 2) does not act the same way on 1 -1 and 1. Indeed, (1, 2)•1 = 1 and (1, 2)• 1 -1 = -1 -1 . So R 2 0 and span(1) are not isomorphic as modules even if they are isomorphic as vector spaces. Analogously, even if k i = k j with i = j, the group O(I) does not act the same way on R k i and R k j . Thus, the modules are not isomorphic.

General form Thus, the O(n)-invariant inner products on R n are positive scalings of the canonical one. The O(I)-invariant and S(n)-invariant inner products on R n are respectively given for all x ∈ R n by:

ϕ(x, x) = p i=1 α k i p k i (x) 2 , (3.4) ϕ(x, x) = α 1 x - x|1 n 1 + α 2 x|1 2 n = α x 2 + β x|1 2 , (3.5) with α k 1 , ..., α kp > 0, α 1 = α > 0 and α 2 = α + nβ > 0.

O(n)-invariant inner products on Mat(n)

In this section, we illustrate the theorem with the congruence action of the orthogonal group O(n) on squared matrices Mat(n). This example is also well known although we did not find a reference book containing it. It allows us to prepare the following sections.

O(n)-invariant inner products on Sym(n)

The irreducible decomposition of Sym(n) with respect to the congruence action of O(n) (and

SO(n)) is Sym(n) = Sym 0 (n) ⊥ ⊕ span(I n ) where Sym 0 (n) = Sym(n) ∩ ker tr.

Direct sum

The decomposition is clearly a direct irreducible sum. The orthogonal projections are

X ∈ Sym(n) -→ X -tr(X) n I n ∈ Sym 0 (n) and X ∈ Sym(n) -→ tr(X) n I n ∈ span(I n ).
Irreducibility Firstly, Sym 0 (n) and span(I n ) are stable and span(I n ) is irreducible because of dimension 1. Secondly, let W ⊆ Sym 0 (n) be a submodule such that W = {0}. Let X ∈ W , X = 0. Then there exist P ∈ SO(n) and D ∈ Diag(n) ∩ ker tr such that

X = P DP so D ∈ W \{0}. If n = 2, then d 1 = 0 and E 11 -E 22 = 2 d 1 D ∈ W . By applying the rotation Å 0 1 -1 0 ã ∈ SO(2), we get E 12 ∈ W so Sym 0 (2) = W and Sym 0 (2) is irreducible. Let us
prove it for n 3. There exist i < j such that d i = d j because tr(D) = 0 and D = 0. It is tempting to apply the same method as in the example of S(n) acting on R n to conclude that Diag(n) ∩ ker tr ⊆ W . However, permutations are not all in SO(n), especially transpositions do not belong to SO(n). So we need to find another way. We define the matrix

R = R ij (θ) ∈ SO(n) by R ii = R jj = cos θ, R ij = -R ji = sin θ, R kk = 1 for k /
∈ {i, j} and zero elsewhere.

With θ = π 4 , the submatrix

Å d i 0 0 d j ã is transformed into 1 2 Å d i + d j d j -d i d j -d i d i + d j ã . Hence D 1 = 1 d j -d i (D -RDR ) = 1
2 (e i e i + e i e j + e j e i -e j e j ) ∈ W . Since n 3, we can take a third index k = i, j and act by the diagonal matrix diag(1, ..., 1, -1, 1, ..., 1, -1, 1, ..., 1) ∈ SO(n) where the -1 are at indexes i and k. We obtain D 2 = 1 2 (e i e i -e i e j -e j e i -e j e j ). Hence e i e i -e j e j = D 1 +D 2 ∈ W . Then for all k ∈ {1, ..., n-1}, it is easy to find even permutations sending i on k and j on k+1 so that e k e k -e k+1 e k+1 ∈ W and Diag(n)∩ker tr ⊆ W . Moreover, E ij = D 1 -D 2 ∈ W so with appropriate even permutations, we can prove that all E kl with k = l belongs to W . Hence, Sym 0 (n) ⊆ W and Sym 0 (n) is irreducible.

Isomorphic modules

The two modules are clearly not isomorphic because they have different dimensions: dim(Sym

0 (n)) = n(n+1) 2 -1 > 1 = dim span(I n ) for n 2.
General form Thus, the O(n)-invariant (and SO(n)-invariant) inner products on Sym(n) are given for all X ∈ Sym(n) by:

ϕ(X, X) = α 1 X - tr(X) n I n 2 + α 2 tr(X) 2 n = α tr(X 2 ) + β tr(X) 2 , ( 3.6)
with α 1 = α and α 2 = α + nβ > 0.

O(n)-invariant inner products on Mat(n)

The irreducible decomposition of Mat(n) with respect to the congruence action of O(n) is

Mat(n) = Sym 0 (n) ⊥ ⊕ Skew(n) ker tr ⊥ ⊕ span(I n ).

Direct sum

The decomposition is clearly a direct orthogonal sum. The orthogonal projections are

X ∈ Mat(n) -→ X+X 2 -tr(X) n I n ∈ Sym 0 (n), X ∈ Mat(n) -→ X-X 2 ∈ Skew(n) and X ∈ Mat(n) -→ tr(X) n I n ∈ span(I n ). Irreducibility Let us prove that Skew(n) is O(n)-irreducible. Indeed, let W = {0} be a submodule of Skew(n) and let M ∈ Skew(n), M = 0. Then, M is orthogonally congruent to B = á B λ 1 0 . . . B λp 0 0 n-2p ë ∈ Skew(n)
where ±iλ 1 , ..., ±iλ p ∈ iR * (and 0 if n = 2p) are its complex eigenvalues and

B λ = Å 0 λ -λ 0 ã ∈ Skew(2). Applying the permutation σ = (1, 2), we get e 1 e 2 -e 2 e 1 = Å B 1 0 0 0 ã = 1 2λ 1 (M -P σ M P σ ) ∈ W .
With appropriate permutations, all e i e j -e j e i ∈ W so W = Skew(n) and Skew(n) is O(n)-irreducible.

Isomorphic modules

For n 3, the modules have different dimensions: dim(Sym

0 (n)) = n(n+1) 2 -1 > dim Skew(n) = n(n-1)
2 > dim span(I n ) = 1. So they are clearly not isomorphic. For n = 2, dim(Sym 0 (2)) = 2 > 1 = dim Skew(2) = dim span(I 2 ), but the permutation (1, 2) acts differently on e 1 e 2 -e 2 e 1 and I 2 so the modules are not isomorphic either.

General form Thus, the O(n)-invariant inner products on Mat(n) are given for all X ∈ Mat(n) by:

ϕ(X, X) = α 1 X + X 2 - tr(X) n I n 2 + α 2 X -X 2 2 + α 3 tr(X) 2 n = α 1 tr Å X 2 + (X ) 2 + 2XX 4 + tr(X) 2 n 2 I n - tr(X) n (X + X ) ã + α 2 4 tr(XX + X X -X 2 -(X ) 2 ) + α 3 tr(X) 2 n = α tr(XX ) + β tr(X 2 ) + γ tr(X) 2 , with α = α 1 +α 2 2 , β = α 1 -α 2 2 and γ = α 3 -α 1 n . The inverse relations are α 1 = α + β > 0, α 2 = α -β > 0 and α 3 = α + β + nγ > 0, as expected.

O(I)-invariant inner products on Mat(n)

In this section, we consider the congruence action of the block orthogonal group O(I) on skewsymmetric matrices Skew(n), on symmetric matrices Sym(n) and finally on squared matrices Mat(n). In particular, O(I)-invariant inner products on Skew(n) = T In O(n) characterize O(n)-invariant Riemannian metrics on the homogeneous space of flags of signature I denoted Flag(I) = O(n)/O(I). In this context, the signature is sometimes called the nationality of the flag.

We recall that

I = (k 1 , ..., k p ) is a partition of n, i.e. k 1 , ..., k p ∈ N * with k 1 + • • • + k p = n.
We consider the group of block orthogonal matrices of signature I:

O(I) =      Ö R 1 0 . . . 0 R p è ∈ O(n)|∀i ∈ {1, ..., p}, R i ∈ O(k i )      O(k 1 ) × ... × O(k p ). We denote O n (k 1 ) = ßÅ R 0 0 I n-k 1 ã ∈ O(I)|R ∈ O(k 1 ) ™ O(k 1
) and analogously O n (k i ) for i ∈ {1, ..., p}. They are subgroups of O(I). In the same block-wise spirit, we also denote for i = j:

• Mat n (k 1 ) = ßÅ X 0 0 0 ã ∈ Mat(n)|R ∈ Mat(k 1 )
™ with appropriate sizes of null matrices, and analogously Mat n (k i ) Mat(k i ) for i ∈ {1, ..., p},

• Mat n (k 1 , k 2 ) = ßÅ 0 X 0 0 0 0 ã ∈ Mat(n)|X ∈ Mat(k 1 , k 2 )
™ again with appropriate sizes of null matrices, and analogously Mat n (k i , k j ) Mat(k i , k j ) for i, j ∈ {1, ..., p},

• Skew n (k i ) = Skew(n) ∩ Mat n (k i ) Skew(k i ), • Skew n (k i , k j ) = Skew(n) ∩ (Mat n (k i , k j ) ⊕ Mat n (k j , k i )) Mat n (k i , k j ), • Sym n (k i ) = Sym(n) ∩ Mat n (k i ), • Sym n (k i , k j ) = Sym(n) ∩ (Mat n (k i , k j ) ⊕ Mat n (k j , k i )) Mat n (k i , k j ), • Sym 0 n (k i ) = Sym n (k i ) ∩ ker tr, • Sym 0 n (k i , k j ) = Sym n (k i , k j ) ∩ ker tr. Whenever i = j, we also denote Mat n (k i , k i ) = Mat n (k i ), Sym n (k i , k i ) = Sym n (k i ) and Skew n (k i , k i ) = Skew n (k i ).

O(I)-invariant inner products on Skew(n)

We denote X ∈ Skew(n) -→ X ij ∈ Skew n (k i , k j ) the canonical projection for i, j ∈ {1, ..., p}.

(n) = 1 i p k i 2 Skew n (k i ) ⊕ 1 i<j p Skew n (k i , k j ).
The decomposition is orthogonal for the Frobenius inner product. The O(I)-invariant inner products on Skew(n) are given for all X ∈ Skew(n) by: ϕ(X, X) =

1 i p k i 2 α ii X ii 2 + 1 i<j p α ij X ij 2 , ( 3.7)
where α ij > 0 for all 1 i j p.

See the proof of Example 3.5 in Section 11.2.

O(n)-invariant Riemannian metrics on Flag(I)

The homogeneous manifold of flags of signature

I is Flag(I) = O(n)/O(I) = {U O(I), U ∈ O(n)}. Since T In O(n) = Skew(n) and T In O(I) = Skew(I) := 1 i p Skew n (k i ) = 1 i p k i 2 Skew n (k i ),
the tangent space T O(I) Flag(I) can be identified to m :=

1 i<j p Skew n (k i , k j ). Therefore, a Riemannian metric g on Flag(I) is O(n)-invariant if and only if g O(I) : T O(I) Flag(I) × T O(I) Flag(I) -→ R is O(I)-invariant.
Such inner products are characterized by the p(p-1)/2 positive coefficients α ij > 0 for 1 i < j p. From Equation (3.7), the general O(n)invariant Riemannian metric on Flag(I) is given for all X ∈ T U O(I) Flag(I) U m by:

g U O(I) (X, X) = 1 i<j p α ij (U -1 X) ij 2 .
(3.8)

The manifold of flags Flag(I) endowed with one of these Riemannian metrics is a Riemannian symmetric space. They all share the same Levi-Civita connection, thus the same geodesics. We retrieve a result on Grassmannians Gr(n, k) = Flag(k, n-k) = O(n)/(O(k)×O(n-k)) (for which p = 2): there exists a unique O(n)-invariant Riemannian metric on Gr(n, k) up to a scaling factor α 12 > 0.

O(I)-invariant inner products on Sym(n)

We denote X ∈ Sym(n) -→ X ij ∈ Sym n (k i , k j ) the canonical projection for i, j ∈ {1, ..., p}.

Example 3.6 (O(I)-invariant inner products on Sym(n)) The irreducible decomposition of Sym(n) with respect to the congruence action of O(I) is Sym

(n) = p i=1 span(I ii n ) ⊕ 1 i p k i 2 Sym 0 n (k i ) ⊕ 1 i<j p Sym n (k i , k j ).
The decomposition is orthogonal for the Frobenius inner product and V. The O(I)-invariant inner products on Sym(n) are given for all X ∈ Sym(n) by: (3.9) where S ∈ Sym + (p) and α ij > 0 for all 1 i j p.

ϕ(X, X) = 1 i,j p S ij tr(X ii )tr(X jj ) k i k j + 1 i p k i 2 α ii X ii - tr(X ii ) k i I ii n 2 + 1 i<j p α ij X ij 2 ,
See the proof of Example 3.6 in Section 11.2.

O(I)-invariant inner products on Mat(n)

To distinguish between the projections X -→ X ij onto symmetric matrices and skewsymmetric matrices for 1 i j p, we denote:

• X ∈ Mat(n) -→ X ij sym = 1 2 (X + X ) ij ∈ Sym n (k i , k j ), • X ∈ Mat(n) -→ X ij skew = 1 2 (X -X ) ij ∈ Skew n (k i , k j ).
Example 3.7 (O(I)-invariant inner products on Mat(n)) The irreducible decomposition of Mat(n) with respect to the congruence action of O(I) is:

Mat(n) = p i=1 span(I ii n ) ⊕ 1 i<j p (Skew n (k i , k j ) ⊕ Sym n (k i , k j )) ⊕ 1 i p k i 2 Skew n (k i ) ⊕ 1 i p k i 2 Sym 0 n (k i ) (3.10)
Direct sum It is the direct sum of the two previous decompositions.

Irreducibility

We already know that all the terms are stable and irreducible.

Isomorphic modules

We grouped Skew n (k i , k j ) and Sym n (k i , k j ) because they are isomorphic modules. According to the previous study of the action of O(n) and O(I) on Skew(n) and Sym(n), it is clear that there is no module isomorphism between other terms in the direct sum.

General form Hence, the O(I)-invariant inner products on Mat(n) are given for all X ∈ Mat(n) by: ϕ(X, X) = .11) where S ∈ Sym + (p), α ii , β ii > 0 for all i ∈ {1, ..., p} such that k i 2, α ij > 0 and α ij β ij -γ 2 ij > 0 for all 1 i < j p.

1 i,j p S ij tr(X ii )tr(X jj ) k i k j + 1 i p k i 2 Ç α ii X ii sym - tr(X ii ) k i I ii n 2 + β ii X ii skew 2 å + 1 i<j p Ä α ij X ij sym 2 + β ij X ij skew 2 + γ ij X ij sym |X ij skew ä , ( 3

S(n)-invariant inner products on Mat(n)

In this section, we consider the congruence action of the group of permutation matrices S(n) ≡ {P σ = (δ i,σ(j) ) 1 i,j n ∈ O(n), σ ∈ S(n)} on squared matrices Mat(n). We especially focus on the action on symmetric hollow matrices Hol(n) = Sym(n) ∩ ker Diag (without isomorphic irreducible submodules) and symmetric matrices Sym(n) (with isomorphic irreducible submodules). Since Hol(n) and Sym(n) are respectively the tangent spaces of full-rank correlation matrices Cor + (n) = {Σ ∈ Sym + (n)|Diag(Σ) = I n } and SPD matrices Sym + (n), this provides permutation-invariant Euclidean metrics on full-rank correlation and covariance matrices.

The case n = 2 is degenerate so we treat it here. We have:

Mat(2) = span(I 2 ) ⊥ ⊕ span Å 0 1 1 0 ã ⊥ ⊕ span Å 1 0 0 -1 ã ⊥ ⊕ span Å 0 1 -1 0 ã . (3.12)
The two first terms are isomorphic modules with isotropy group S(2), the two last terms are isomorphic modules with isotropy group {Id}. So the S(2)-invariant inner products on Mat(2) are given for all

X = Å x y z t ã ∈ Mat(2) by: ϕ(X, X) = α (x + t) 2 + β (y + z) 2 + 2γ (x + t)(y + z) + α (x -t) 2 + β (y -z) 2 + 2γ (x -t)(y -z), (3.13)
where

Å α γ γ β ã ∈ Sym + (2) and Å α γ γ β ã ∈ Sym + (2).
In the following, we assume that n 3.

S(n)-invariant inner products on Hol(n), n 3

We denote sum(X) = 1 X1 = i,j X ij the sum of entries of X ∈ Mat(n) and analogously sum(µ) = 1 µ for µ ∈ R n .

Example 3.8 (S(n)-invariant inner products on Hol(n)) The irreducible decomposition of Hol(n) with respect to the congruence action of S(n), i.e. of permutation matrices, is

Hol(n) = ker φ 1 ⊥ ⊕ im ψ 2 ⊥ ⊕ im ψ 3 where: 1. φ 1 : X ∈ Hol(n) -→ X1 ∈ R n , 2. ψ 2 : µ ∈ R n 0 -→ 1 n-2 (µ1 + 1µ -2diag(µ)) ∈ Hol(n), 3. ψ 3 : x ∈ R -→ x n(n-1) (11 -I n ) ∈ Hol(n)
, with φ 1 surjective, ψ 2 and ψ 3 injective. The corresponding orthogonal projections are:

• p 3 : X ∈ Hol(n) -→ X 3 = ψ 3 (sum(X)) ∈ im ψ 3 , • p 2 : X ∈ Hol(n) -→ X 2 = ψ 2 ((X -X 3 )1) ∈ im ψ 2 , • p 1 : X ∈ Hol(n) -→ X 1 = X -X 2 -X 3 ∈ ker φ 1 .
The S(n)-invariant inner products on Hol(n) are given for all X ∈ Hol(n) by: (3.15) where

ϕ(X, X) = α 1 tr(X 2 1 ) + α 2 tr(X 2 2 ) + α 3 tr(X 2 3 ) (3.14) = α tr(X 2 ) + β sum(X 2 ) + γ sum(X) 2 ,
α 1 = α > 0, α 2 = α + n-2 2 β and α 3 = α + (n -1)(β + nγ) > 0.
See the proof of Example 3.8 in Section 11.2. Remark 3.9 The case n = 3 is also degenerate because ker φ 1 = {0}. The inner product writes ϕ(X, X) = β sum(X 2 ) + γ sum(X) 2 with β > 0 and β + 3γ > 0.

S(n)-invariant inner products on Sym(n), n 3

Example 3.10 (S(n)-invariant inner products on Sym(n)) The irreducible decomposition of Sym(n) with respect to the congruence action of S(n), i.e. of permutation matrices, is

Sym(n) = ker φ 1 ⊥ ⊕ im ψ 2 ⊥ ⊕ im ψ 3 Hol(n) ⊥ ⊕ im ψ 4 ⊥ ⊕ im ψ 5 Diag(n)
ker φ 1 ⊕ 2R n 0 ⊕ 2R where φ 1 , ψ 2 , ψ 3 were defined in the previous section and:

4. ψ 4 : λ ∈ R n 0 -→ diag(λ) ∈ Diag(n), 5. ψ 5 : y ∈ R -→ y n I n ∈ Diag(n)
, with ψ 4 , ψ 5 injective. The corresponding orthogonal projections are:

• π 5 : X ∈ Sym(n) -→ X 5 = ψ 5 (tr(X)) ∈ im ψ 5 , • π 4 : X ∈ Sym(n) -→ X 4 = Diag(X -X 5 ) ∈ im ψ 4 , • π 3 : X ∈ Sym(n) -→ X 3 = p 3 (X -Diag(X)) ∈ im ψ 3 , • π 2 : X ∈ Sym(n) -→ X 2 = p 2 (X -Diag(X)) ∈ im ψ 2 , • π 1 : X ∈ Sym(n) -→ X 1 = p 1 (X -Diag(X)) ∈ ker φ 1 .
The S(n)-invariant inner products on Sym(n) are given for all X = X 1 + ψ 2 (µ) + ψ 3 (x) + ψ 4 (λ) + ψ 5 (y) ∈ Sym(n) by: (3.17) where α 1 = α > 0 and the SPD matrices are S =

ϕ(X, X) = α 1 tr(X 2 1 ) + S 11 µ 2 + Σ 11 x 2 + S 22 λ 2 + Σ 22 y 2 + 2S 12 λ µ + 2Σ 12 xy (3.16) = α tr(X 2 ) + β sum(X 2 ) + γ sum(X) 2 + δ tr(Diag(X) 2 ) + ε sum(Diag(X)X) + ζ tr(X) 2 + η tr(X) sum(X),
Å 2 n-2 α + β β + ε 2 β + ε 2 α + β + δ + ε ã ∈ Sym + (2) and Σ = 1 n Å 1 n-1 α + β + nγ β + ε 2 + n(γ + η 2 ) β + ε 2 + n(γ + η 2 ) α + β + δ + ε + n(γ + ζ + η) ã ∈ Sym + (2).
Mat(n) = ker φ 1 ⊥ ⊕ im ψ 2 ⊥ ⊕ im ψ 3 Hol(n) ⊥ ⊕ im ψ 4 ⊥ ⊕ im ψ 5 Diag(n) Sym(n) ⊥ ⊕ ker φ 6 ⊥ ⊕ im ψ 7 Skew(n) (3.18) ker φ 1 ⊕ ker φ 6 ⊕ 3R n 0 ⊕ 2R, (3.19)
where φ 1 , ψ 2 , ψ 3 , ψ 4 , ψ 5 were defined in the previous sections and:

6. φ 6 : X ∈ Skew(n) -→ X1 ∈ R n 0 , 7. ψ 7 : ν ∈ R n 0 -→ 1 n (ν1 -1ν ) ∈ Skew(n)
, with φ 6 surjective and ψ 7 injective. The corresponding orthogonal projections are:

• Π 7 : X ∈ Mat(n) -→ X 7 = ψ 7 Ä X-X 2 1 ä ∈ im ψ 7 , • Π 6 : X ∈ Mat(n) -→ X 6 = X-X 2 -X 7 ∈ ker φ 6 , • Π 5 : X ∈ Mat(n) -→ X 5 = π 5 Ä X+X 2 ä = ψ 5 (tr(X)) ∈ im ψ 5 , • Π 4 : X ∈ Mat(n) -→ X 4 = π 4 Ä X+X 2 ä = Diag(X -X 5 ) ∈ im ψ 4 , • Π 3 : X ∈ Mat(n) -→ X 3 = π 3 Ä X+X 2 ä ∈ im ψ 3 , • Π 2 : X ∈ Mat(n) -→ X 2 = π 2 Ä X+X 2 ä ∈ im ψ 2 , • Π 1 : X ∈ Mat(n) -→ X 1 = π 1 Ä X+X 2 ä ∈ ker φ 1 .
The S(n)-invariant inner products on Mat(n) are given for all

X = X 1 + ψ 2 (µ) + ψ 3 (x) + ψ 4 (λ) + ψ 5 (y) + X 6 + ψ 7 (ν) ∈ Mat(n) by: ϕ(X, X) = α 1 tr(X 2 1 ) + S 11 µ 2 + Σ 11 x 2 + S 22 λ 2 + Σ 22 y 2 + α 2 tr(X 2 6 ) + S 33 ν 2 + 2S 12 λ µ + 2S 23 ν λ + 2S 13 µ ν + 2Σ 12 xy (3.20) = α tr(XX ) + α tr(X 2 ) + β sum(X 2 ) + β sum(XX ) + β sum(X X) + γ sum(X) 2 + δ tr(Diag(X) 2 ) + ζ tr(X) 2 + η tr(X) sum(X) + ε sum(Diag(X)X) + ε sum(Diag(X)X ), (3.21)
where

α 1 = α+α 2 > 0, α 2 = α-α 2
> 0 and the SPD matrices are:

S = Ñ 2 n-2 α+α 2 + β + β + β β + β + β + 1 2 (ε + ε ) β -β 4 β + β + β + 1 2 (ε + ε ) α+α 2 + β + β + β + δ + ε + ε β -β 4 + ε -ε 2 β -β 4 β -β 4 + ε -ε 2 2α + β -β -β é ∈ Sym + (3), Σ = 1 n Ç 1 n-1 α+α 2 + β + β + β + nγ β + β + β + 1 2 (ε + ε ) + n(γ + η 2 ) β + 1 2 (ε + ε ) + n(γ + η 2 ) α+α 2 + β + β + β + δ + ε + ε + n(γ + ζ + η) å ∈ Sym + (2).
See the proof of Example 3.12 in Section 11.2.

Remark 3.13 The case n = 3 is degenerate because ker φ 1 = {0}. In this case, one has to replace α 1 and α by 0 (and n by 3) in the previous formulae.

Remark 3.14 Equation (3.19) can be found in [Ramgoolam, 2019]. It gives the irreducible decomposition of Mat(n) up to module isomorphism. Our Equation (3.18) instantiates an irreducible decomposition by choosing specific spaces. The advantage is that it allows to manipulate coordinate-free equations and explicit spaces. Moreover, it puts the light on interesting spaces that are stable by permutations.

Conclusion

We formalized a general method to determine all G-invariant inner products on a completely reducible Euclidean space V . We gave the general form of inner products on R n and Mat(n) that are invariant under the orthogonal group O(n), the block orthogonal group O(I) and the permutation group S(n). They are summarized in Table 3.2. Beyond linear algebra, this characterization is interesting when one wants to characterize invariant Riemannian metrics on Lie groups and homogeneous spaces. It is well known that the O(n)-invariant inner product on Grassmannians Gr

(n, k) = O(n)/(O(k) × O(n -k))
is unique up to scaling. More generally, our method allowed us to characterize all O(n)invariant metrics on the homogeneous manifold Flag(I) = O(n)/O(I) of flags of signature I (see Equation (3.8)). The characterization is also interesting when there exists a global diffeomorphism from a manifold to a vector space, such as for symmetric positive definite matrices or full-rank correlation matrices.

This method is based on representation theory. It would be interesting to investigate non-linear methods to characterize invariant Riemannian metrics on manifolds when this problem does not reduce to a linear problem.

Action

Inner product ϕ(x, x) or ϕ(X, X)

Positivity condition O(n) R n α x 2 α > 0 O(I) R n p i=1 α k i p k i (x) 2 α k i > 0 S(n) R n α x 2 + β (x) 2 α > 0, α + nβ > 0 O(n) Mat(n) α tr(XX ) + β tr(X 2 ) + γ tr(X) 2 α > |β|, α + β + nγ > 0 O(I) Skew(n) 1 i p k i 2 α ii X ii 2 + 1 i<j p α ij X ij 2 α ij > 0 O(I) Sym(n) 1 i,j p S ij tr(X ii )tr(X jj ) √ k i k j + 1 i p k i 2 α ii X ii -tr(X ii ) k i I ii n 2 + 1 i<j p α ij X ij 2 S ∈ Sym + (p), α ij > 0 O(I) Mat(n) 1 i,j p S ij tr(X ii )tr(X jj ) √ k i k j + 1 i p k i 2 α ii X ii sym -tr(X ii ) k i I ii n 2 + β ii X ii skew 2 S ∈ Sym + (p), α ii > 0, β ii > 0 + 1 i<j p Ä α ij X ij sym 2 + β ij X ij skew 2 + γ ij X ij sym |X ij skew ä α ij > 0, α ij β ij -γ 2 ij > 0 S(n) Hol(n) α tr(X 2 ) + β sum(X 2 ) + γ sum(X) 2 α > 0, 2α + (n -2)β > 0, α + (n -1)(β + nγ) > 0 S(n) Sym(n) α tr(X 2 ) + β sum(X 2 ) + γ sum(X) 2 + δ tr(Diag(X) 2 ) α > 0, S 1 , Σ 1 ∈ Sym + (2) +ε sum(Diag(X)X) + ζ tr(X) 2 + η tr(X) sum(X) S(n) Mat(n) α tr(XX ) + α tr(X 2 ) + β sum(X 2 ) + β sum(XX ) + β sum(X X) α > |α | +γ sum(X) 2 + δ tr(Diag(X) 2 ) + ζ tr(X) 2 + η tr(X) sum(X) S 2 ∈ Sym + (3) +ε sum(Diag(X)X) + ε sum(Diag(X)X ) Σ 2 ∈ Sym + (2) S 1 = Ç 2 n-2 α + β β + ε 2 β + ε 2 α + β + δ + ε å and Σ 1 = 1 n Ç 1 n-1 α + β + nγ β + ε 2 + n(γ + η 2 ) β + ε 2 + n(γ + η 2 ) α + β + δ + ε + n(γ + ζ + η) å , S 2 = Ö 2 n-2 α+α 2 + β + β + β β + β + β + 1 2 (ε + ε ) β -β 4 β + β + β + 1 2 (ε + ε ) α+α 2 + β + β + β + δ + ε + ε β -β 4 + ε -ε 2 β -β 4 β -β 4 + ε -ε 2 2α + β -β -β è , Σ 2 = 1 n Ç 1 n-1 α+α 2 + β + β + β + nγ β + β + β + 1 2 (ε + ε ) + n(γ + η 2 ) β + 1 (ε + ε ) + n(γ + η ) α+α + β + β + β + δ + ε + ε + n(γ + ζ + η) å .
Table 3.2: General form of invariant inner products, where x ∈ R n and X ∈ Mat(n). Therefore in this work, we investigate super-classes of kernel metrics and we study which key results remain true. We also introduce an additional key result called cometric-stability, a crucial property to implement geodesics with a Hamiltonian formulation. Our method to build intermediate embedded classes between O(n)-invariant metrics and kernel metrics is to give a characterization of the whole class of O(n)-invariant metrics on SPD matrices and to specify requirements on metrics one by one until we reach kernel metrics. As a secondary contribution, we synthesize the literature on the main O(n)-invariant metrics, we provide the complete formula of the sectional curvature of the affine-invariant metric and the formula of the geodesic parallel transport between commuting matrices for the Bures-Wasserstein metric.

This chapter was resubmitted in January 2022 to the journal Linear Algebra and its Applications after minor revisions [START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]. Here is the revised version with an additional modification specific to this manuscript. Indeed, our proof of continuity of the O(n)-invariant metrics in Section 4.5.2 is trivialized by our Theorem 2.18 in Chapter 2 so we replace the sequence of majorations that proved the continuity by a simple reference to this Theorem 2.18.

Introduction

Symmetric Positive Definite (SPD) matrices are ubiquitous in data analysis because in many situations, the data (signals, images, diffusion coefficients...) can be represented by their covariance matrices. This is the case in the domains of Brain-Computer Interfaces, diffusion and functional MRI, Computer Vision, Diffusion Tensor Imaging (DTI), etc. SPD matrices form a cone in the vector space of symmetric matrices so a first idea to compute with SPD matrices could be to perform Euclidean computations on symmetric matrices. However, this method has several drawbacks. As geodesics are straight lines, they leave the SPD cone at finite time so extrapolation methods could lead to non admissible matrices, namely with negative eigenvalues. Moreover, the trace is linearly interpolated but other invariants such as the determinant are not monotonically interpolated along geodesics. For example in DTI, where SPD matrices are represented by 3D ellipsoids, the ellipsoids along the geodesic can have a larger volume than the two ellipsoids at extremities, which leads to non realistic predictions in fiber tracking (swelling effect).

Hence, other Riemannian metrics were used in applications to solve these problems. The affine-invariant/Fisher-Rao metric [Skovgaard, 1984, Pennec et al., 2006, Lenglet et al., 2006, Fletcher and Joshi, 2007, Moakher, 2005, Batchelor et al., 2005, Varoquaux et al., 2010, Barachant et al., 2013] provides a Riemannian symmetric structure to the SPD manifold: it is negatively curved, geodesically complete (matrices with null eigenvalues are rejected to infinity), it is invariant under the congruence action (which, in the context of covariance matrices, corresponds to the invariance of the feature vector under affine transformations) and it is inverse-consistent. The log-Euclidean metric [START_REF] Arsigny | Log-Euclidean metrics for fast and simple calculus on diffusion tensors[END_REF] is diffeomorphic to a Euclidean inner product: it also provides a Riemannian symmetric space, it is geodesically complete and inverse-consistent. It is not curved and it is not affine-invariant although it is still invariant under orthogonal transformations and dilations. The Bures-Wasserstein/Procrustes metric [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF], Dryden et al., 2009, Takatsu, 2011, Malagò et al., 2018] is a positively curved quotient metric which is also invariant under orthogonal transformations. It is not geodesically complete but geodesics remain in the cone with boundaries: this means that this metric is suited for computing with Positive Semi-Definite (PSD) matrices. Many other interesting metrics exist with different properties: Bogoliubov-Kubo-Mori [Petz andToth, 1993, Michor et al., 2000], polar-affine [START_REF] Su | Fitting smoothing splines to time-indexed, noisy points on nonlinear manifolds[END_REF], Euclidean-Cholesky [START_REF] Wang | A constrained variational principle for direct estimation and smoothing of the diffusion tensor field from complex dwi[END_REF], log-Euclidean-Cholesky [START_REF] Li | Local Log-Euclidean Multivariate Gaussian Descriptor and Its Application to Image Classification[END_REF], log-Cholesky [Pinheiro andBates, 1996, Lin, 2019], power-Euclidean [START_REF] Dryden | Power Euclidean metrics for covariance matrices with application to diffusion tensor imaging[END_REF], and more recently power-affine [START_REF] Thanwerdas | Is affineinvariance well defined on SPD matrices? A principled continuum of metrics[END_REF], alpha-Procrustes [Hà Quang, 2019], mixed-power-Euclidean [Thanwerdas and Pennec, 2019a].

Except those named after Cholesky, all the other Riemannian metrics cited above are invariant under orthogonal transformations. If we consider SPD matrices as covariance matrices, this transformation corresponds to a rigid-body transformation of the feature vector X ∈ R n -→ RX + X 0 where R is an orthogonal matrix. In 2009, Hiai and Petz introduced the subclass of kernel metrics [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF], which are O(n)-invariant metrics indexed by smooth symmetric maps φ : (R + ) 2 -→ R + . This class satisfies key results: it contains most of the cited O(n)-invariant metrics, it is stable under a certain class of diffeomorphisms and it provides a sufficient condition for geodesic completeness. This sufficient condition becomes necessary if we restrict the class to the subclass of mean kernel metrics which is indexed by kernel maps of the form φ = m θ where m : (R + ) 2 -→ R + is a symmetric homogeneous mean and θ ∈ R is a power. However, the class of kernel metrics does not contain all the aforementioned O(n)-invariant metrics. The main goal of this chapter is to study the super-classes of kernel metrics, especially the whole class of O(n)-invariant metrics for which we give a characterization. More precisely, our objective is to determine which key results on kernel metrics can be generalized and thus to understand better the specificity of kernel metrics within these super-classes.

Results and organization of the chapter

In the remainder of the Introduction, we give the notations and conventions used in the chapter. In Section 4.2, we introduce two preliminary concepts and one result. The first concept is the notion of O(n)-equivariant map on symmetric matrices. We especially explain how to build them from a map defined on diagonal matrices via the spectral theorem because this is a procedure we need several times in the chapter. Then the second concept is a particular case of the previous one, called univariate map. These are maps characterized by a map on positive real numbers. They are particularly interesting because their differential is known in closed form modulo eigenvalue decomposition and because the class of kernel metrics is stable under univariate diffeomorphisms. Finally the result is the characterization of O(n)-invariant inner products on symmetric matrices. These inner products are composed of two terms, the Frobenius term and the trace term, which have different weights so they form a two-parameter family. In the proof, we give elementary tools that we reuse when we characterize O(n)-invariant metrics on SPD matrices.

To explain why kernel metrics do not encompass all the O(n)-invariant metrics cited above, we need to present them or at least the most important ones. One can notice that many metrics and families of metrics are actually based on five of them, namely the Euclidean, the log-Euclidean, the affine-invariant, the Bures-Wasserstein and the Bogoliubov-Kubo-Mori metrics. That is why in Section 4.3, we synthesize the literature on these five noted metrics. For each of them, we give the fundamental Riemannian operations (squared distance, Levi-Civita connection, curvature, geodesics, logarithm map, parallel transport map) when they are known. As a secondary contribution, we give the complete formula of the sectional curvature of the affine-invariant metric and we also give, for the Bures-Wasserstein metric, the new formula of the parallel transport between commuting matrices and simpler formulae of the Levi-Civita connection, the curvature and the parallel transport equation.

In Section 4.4, after reviewing kernel metrics and their key properties, we give two new observations on them. Firstly, the cometric of a metric on SPD matrices can be considered itself as a metric on SPD matrices by identifying the vector space of symmetric matrices and its dual via the Frobenius inner product. Therefore we observe that the cometric of a kernel metric defined by the kernel map φ is a kernel metric characterized by 1/φ. This remarkable result has an important consequence for the numerical computation of geodesics. Indeed, the geodesic equation ∇ γ γ = 0, which is a second order equation, has a Hamiltonian version which is a first order equation that only involves the cometric, not the Christoffel symbols. The Hamiltonian equation is much simpler to integrate and numerically more stable, that is why it is often preferred in numerical implementations, for instance in the Python package geomstats [Miolane et al., 2020a]. Hence knowing a simple explicit formula for the cometric helps to compute numerically the geodesics. Secondly, there is a natural extension of kernel metrics that encompasses all the aforementioned O(n)-invariant metrics, which still satisfies the key properties of kernel metrics including the cometric stability. Roughly speaking, kernel metrics look like the Frobenius inner product on symmetric matrices where the elementary quadratic forms (the X 2 ij ) are weighted by a coefficient involving the kernel map φ and depending on the point. Since the Frobenius inner product is not the only O(n)-invariant inner product on symmetric matrices as explained above, the trace term can be added to the framework of kernel metrics to form extended kernel metrics.

In Section 4.5, we characterize the class of O(n)-invariant metrics on SPD matrices by means of three multivariate maps α, β, γ : (R + ) n -→ R operating on the eigenvalues (d 1 , ..., d n ) of the SPD matrix and which satisfy three conditions of symmetry, compatibility and positivity (Theorem 4.5). Then, we observe that kernel metrics are characterized by two properties within this family. They are ortho-diagonal: it means that the metric matrix is diagonal, i.e. β = 0. They are bivariate: it means that the remaining functions α and γ do not depend on their n -2 last terms, and the compatibility condition imposes that they are equal so we can write γ = α = 1/φ : (R + ) 2 -→ R + . Since the term "kernel" is quite overloaded in many different contexts (such as in Reproducing Kernel Hilbert Spaces in machine learning or in kernel density estimation/regression in statistics), we propose to designate them as Bivariate Ortho-Diagonal (BOD) metrics. Afterwards, we give key properties of O(n)-invariant metrics in analogy with the key properties of BOD (kernel) metrics. Since we do not have a closed-form expression for the cometric anymore, we introduce the intermediate class of bivariate separable metrics which is cometric-stable and we give the expression of the cometric. A summary of the classes of metrics defined in this chapter is shown on Figure 4.1.

Section 4.6 is dedicated to the conclusion.

Notations and conventions

Manifolds Our manifold-related notations are summarized in Table 4.

1. A chart ϕ : U ⊂ M -→ R N provides a local basis of vectors (∂ 1 , ..., ∂ N ) where ∂ k = ∂ ∂ϕ k is a short notation defined for all differentiable maps f : M -→ R and at each point x ∈ U by (∂ k f ) |x = ∂(f •ϕ -1 ) ∂x k ϕ(x)
. A vector field X can be locally decomposed on this basis, X = X k ∂ k , where X k : U -→ R are the coordinate functions of X and where we used Einstein's summation convention. As we deal with matrices, the coordinates often have two indices:

X = X ij ∂ ij . T x M, T M Tangent space at x, tangent bundle d x f, df Differential of map f at x, differential of map f f * , f * Pullback via f , pushforward via f γ Derivative of curve γ g, G Metric on Sym + (n), metric on another space d Riemannian distance on Sym + (n) ∇ Levi-Civita connection R Curvature R(X, Y )Z = ∇ X ∇ Y Z -∇ Y ∇ X Z -∇ [X,Y ] Z γ (Σ,X) (t)
Geodesic at time t with γ(0) = Σ and γ(0) = X Exp, Log Riemannian exponential and logarithm maps Π γ;Σ→Λ X Parallel transport of X along curve γ from Σ to Λ Table 4.1: Notations in a manifold.

Manifolds of matrices

We denote the matrix spaces as shown in Table 4

.2. The (i, j)- coefficient of a matrix M is denoted M ij , [M ] ij or M (i, j) depending on the context. To build a matrix from its coefficients, we denote M = [M ij ] 1 i,j n or simply M = [M ij ] i,j . We denote (C ij ) the canonical basis of matrices, E ii = C ii , E ij = 1 √ 2 (C ij + C ji ) and F kl = 1 2 (C kl + C lk ) for i = j and k, l ∈ {1, ..., n}. The norms are denoted M 1 = i,j |M ij | and M 2 = tr(M M ).
Vector space of matrices Manifold of matrices Mat

(n) n × n real matrices GL(n) General Linear group GL + (n) Positive determinant Sym(n) Real symmetric Sym + (n) Symmetric positive definite Skew(n) Real skew-symmetric O(n) Orthogonal group SO(n) Rotation group Diag(n) Diagonal Diag + (n) Positive diagonal
Table 4.2: Notations for matrix spaces.

The congruence action is the following action of the general linear group on matrices :

(A, M ) ∈ GL(n) × Mat(n) -→ AM A ∈ Mat(n) which leaves stable the spaces of symmetric matrices and SPD matrices. Then, A ∈ GL(n) naturally acts on M = M i 1 j 1 ,...,iqjq C i 1 j 1 ⊗ • • • ⊗ C iqjq ∈ Mat(n) ⊗q by: A M = M i 1 j 1 ,...,iqjq (A C i 1 j 1 ) ⊗ • • • ⊗ (A C iqjq ) ∈ Mat(n) ⊗q .

GL(n) also acts by on any

Cartesian product r i=1 Sym(n) ⊗q i component-wise, especially on Sym(n) p .
Let E and F be two spaces on which GL(n) acts by .

Let G ⊆ GL(n) be a subgroup of GL(n). A map f : E -→ F is: • G-equivariant if f (A M ) = A f (M ) for all A ∈ G, for all M ∈ E, • G-invariant if f (A M ) = f (M ) for all A ∈ G, for all M ∈ E.
In particular, a Riemannian metric g : Sym

+ (n) × Sym(n) × Sym(n) -→ R (or an inner product) is G-invariant if g AΣA (AXA , AXA ) = g Σ (X, X) for all A ∈ G, Σ ∈ Sym + (n) and X ∈ Sym(n).
The symmetric group of order n is denoted by S n and the permutations by small greek letters σ, τ.... The permutation matrix associated to the permutation σ, which sends any basis (e 1 , ..., e n ) of R n to the permuted basis (e σ(1) , ..., e σ(n) ), is denoted P σ . We have P σ (i, j) = δ σ(i),j where δ is the Kronecker symbol. Given a matrix M ∈ Mat(n), we have

(P σ M P σ )(i, j) = M (σ(i), σ(j)).
The manifold of SPD matrices The manifold Sym + (n) is an open set of the vector space of symmetric matrices Sym(n). Hence, the canonical immersion id : Sym + (n) → Sym(n) provides:

• An identification between the tangent space T Σ Sym + (n) and the vector space Sym(n) at any point

Σ ∈ Sym + (n) by d Σ id : T Σ Sym + (n) ∼ -→ Sym(n). Thus, any tangent vector X ∈ T Σ Sym + (n) is considered as a symmetric matrix: X ≡ d Σ id(X) ∈ Sym(n). • A global chart (id, Sym + (n)) of the manifold Sym + (n), thus a global derivation ∂ X Y = X ij (∂ ij Y kl )∂ kl defined by derivation of coordinates in this global chart. More generally, if f : Sym + (n) -→ Sym(n) is a diffeomorphism on its image, it provides a global derivation denoted ∂ f .
Another important tool is the matrix exponential exp(X) = +∞ k=0 X k k! which is a diffeomorphism between Sym(n) and Sym + (n), and therefore its inverse, the symmetric matrix logarithm log :

Sym + (n) -→ Sym(n).
The spectral theorem ensures that symmetric matrices are orthogonally congruent to a diagonal matrix. If the symmetric matrix is SPD, then the diagonal matrix has positive elements on the diagonal. Most of the time in this chapter, for an SPD matrix Σ ∈ Sym + (n), we denote Σ = P DP one spectral decomposition with

P ∈ O(n) and D = diag(d 1 , ..., d n ) ∈ Diag + (n).
When we consider tangent vectors X, Y, ... ∈ T Σ Sym + (n), we denote X = P XP so that every matrix expressed in the orthogonal basis given by P is denoted with a prime:

X = P X P , Y = P Y P , etc.
Products of symmetric matrices share two nice properties with symmetric matrices. First, if X, Y ∈ Sym(n), then sp(XY ) ⊂ R where sp denotes the spectrum, i.e. the set of complex eigenvalues. Second, if Σ, Λ ∈ Sym + (n), then ΣΛ has a unique square-root matrix that represents a positive definite self-adjoint endomorphism, it is denoted (ΣΛ)

1/2 = √ ΣΛ = Σ 1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 Σ -1/2 = Λ -1/2 (Λ 1/2 ΣΛ 1/2 ) 1/2 Λ 1/2
. This definition is used in [Bhatia et al., 2019, Equation (10)] for example.

Preliminary concepts and results

Extending maps defined on diagonal matrices

Thanks to the spectral theorem, O(n)-equivariant maps f : Sym + (n) -→ F are characterized by their values on positive diagonal matrices. A question that arises several times in this chapter is: are we allowed to extend a map f :

Diag + (n) -→ F into an O(n)-equivariant map f : Sym + (n) -→ F by the formula f (P DP ) = P f (D)?
To do so, we need to show that given two eigenvalue decompositions Σ = P DP = Q∆Q , we have P f (D) = Q f (∆). Note that (Q, ∆) is highly constrained by (P, D). The following lemma gives explicitly the possible cases, hence it tells exactly what is to be checked in such an extension process.

Lemma 4.1 (Relation between two eigenvalue decompositions of an

SPD matrix) Let D, ∆ ∈ Diag + (n) and P, Q ∈ O(n) such that P DP = Q∆Q . Let τ ∈ S(n) be a permutation that orders the values of D decreasingly, i.e. such that D = P τ Diag(λ 1 I m 1 , ..., λ p I mp )P τ with λ 1 > ... > λ p > 0. Then, there exists a permutation σ ∈ S(n) and a block-diagonal orthogonal matrix R = Diag(R 1 , ..., R p ) ∈ O(n) with j-th block R j ∈ O(m j ) such that ∆ = P σ P τ DP τ P σ and Q = P P τ RP σ . Proof. ∆ is clearly a permutation of D so there exists σ ∈ S(n) such that ∆ = P σ P τ DP τ P σ . Let R = P τ P QP σ . Then P DP = Q∆Q is equivalent to Diag(λ 1 I m 1 , ..., λ p I mp )R = R Diag(λ 1 I m 1 , ..., λ p I mp ).
Decomposing R by blocks, the off-diagonal blocks have to be null since the λ i 's are distinct. Since RR = I n , the diagonal blocks are orthogonal.

This result tells what is to be checked to extend f : Diag + (n) -→ F. In this chapter, we only need to extend tensorial maps T : Diag + (n) -→ Sym(n) ⊗q ⊗(Sym(n) * ) ⊗p or equivalently T : Diag + (n) × Sym(n) p -→ Sym(n) ⊗q for p, q ∈ N. Hence, we state the result in this particular case though it is valid for F.

Lemma 4.2 (Spectral extension) Let

T : Diag + (n) × Sym(n) p -→ Sym(n) ⊗q be a map such that for all D 0 = Diag(λ 1 I m 1 , ..., λ p I mp ) with λ 1 > ... > λ p > 0 and for all X ∈ Sym(n) p : (a) T (D 0 , X) = P σ T (P σ D 0 P σ , P σ X) for all permutations σ ∈ S(n), (b) T (D 0 , X) = R T (D 0 , R X) for all block-diagonal orthogonal matrices R ∈ O(n), R = Diag(R 1 , ..., R p ) with R j ∈ O(m j ). Then, T : Sym + (n) × Sym(n) p -→ Sym(n) ⊗q defined by T (P DP , X) := P T (D, P X) extends T , with D ∈ Diag + (n), P ∈ O(n) and X ∈ Sym(n) p .
Proof. Assume that P DP = Q∆Q . Then by Lemma 4.1, let σ, τ ∈ S(n) and R as in (b) such that D 0 = P τ DP τ = Diag(λ 1 I m 1 , ..., λ p I mp ), ∆ = P σ D 0 P σ and Q = P P τ RP σ . Then, by applying (a) with σ, (b) with R and (a) with τ , we easily see that

Q T (∆, Q X) = P T (D, P X). Thus T : Sym + (n) × Sym(n) p -→ Sym(n) ⊗q is well defined.
In practice in this chapter, we use Lemma 4.2 for:

• p = 0, q = 1 for f : Diag + (n) -→ Sym(n) in Section 4.2.2, • p = 1, q = 1 for Φ : Diag + (n) × Sym(n) -→ Sym(n) in Section 4.4.2.3, • p = 2, q = 0 for g : Diag + (n) × Sym(n) × Sym(n) -→ R in Section 4.5.2.

Univariate maps

We apply Lemma 4.2 to a map defined on positive real numbers f : R + -→ R and extended to positive diagonal matrices f :

Diag + (n) -→ Diag(n) by f (Diag(d 1 , ..., d n )) := Diag(f (d 1 ), ..., f (d n )). (a) Since f is defined component-wise, we have f (D) = P σ f (P σ DP σ )P σ . (b) As f (λI m j ) = f (λ)I m j , the matrix Rf (D)R is a block diagonal matrix with j-th block f (λ j )R j R j = f (λ j )I m j , which corresponds to f (D)'s j-th block so R f (D) R = f (D). Therefore f can be extended into an O(n)-equivariant map f : Sym + (n) -→ Sym(n) by f (P DP ) = P f (D)P .
This extension is called the functional calculus of f in Functional Analysis. We call it a univariate map. The symmetric matrix logarithm log :

Sym + (n) -→ Sym(n), the power diffeomorphisms pow p : Sym + (n) -→ Sym + (n) with p = 0 or the constant map pow 0 : Σ ∈ Sym + (n) -→ I n ∈ Sym(n) are examples of univariate maps.

Definition 4.3 (Univariate maps) A univariate map is the extension of a map on positive real numbers

f : R + -→ R into an O(n)-equivariant map f : Sym + (n) -→ Sym(n) by the equality f (P DP ) = P Diag(f (d 1 ), ..., f (d n )) P . Moreover [Bhatia, 1997, Theorem V.3.3], if f ∈ C 1 (R + ), then its extension f is differentiable and the differential df : Sym + (n) × Sym(n) -→ Sym(n) is O(n)-equivariant
, thus it is characterized by its values at diagonal matrices D ∈ Diag + (n), given by:

∀X ∈ Sym(n), [d D f (X)] ij = f [1] (d i , d j )X ij , ( 4.1)
where f [1] is the first divided difference defined below. Thus, a

C 1 -diffeomorphism f : R + -→ R + is extended into a diffeomorphism f : Sym + (n) -→ Sym + (n). Definition 4.4 (First divided difference) [Bhatia, 1997] Let f ∈ C 1 (R + ). The first divided difference of f is the continuous symmetric map f [1] : (R + ) 2 -→ R defined for all x, y ∈ R by: f [1] (x, y) = ® f (x)-f (y) x-y if x = y f (x) if x = y
´.

(4.2)

O(n)-invariant inner products on symmetric matrices

To characterize the O(n)-invariant metrics on SPD matrices, an appropriate starting point is the characterization of O(n)-invariant inner products on the tangent space, i.e. on symmetric matrices. The following theorem states that such inner products form a two-parameter family indexed by a Scaling factor α > 0 and a Trace factor β > -α/n.

Theorem 4.5 (Characterization of O(n)-invariant inner products on symmetric matrices)

Let •|• : Sym(n) × Sym(n) -→ R be an inner product on symmetric matrices. It is O(n)- invariant if and only if there exists (α, β) ∈ ST := {(α, β) ∈ R 2 | min(α, α + nβ) > 0} such that: ∀X ∈ Sym(n), X|X = α tr(X 2 ) + β tr(X) 2 . (4.3)
Moreover, the linear isometry that pulls the Frobenius inner product back onto this one is F p,q (X) = q X + p-q n tr(X)I n with p = √ α + nβ and q = √ α.

There are several proofs of this elementary result. We already gave one in Section 3.3.2.1. Here we give one based on the following lemma because we reuse it to characterize O(n)invariant metrics on SPD matrices. This lemma gives the characterization of inner products on symmetric matrices which are respectively invariant under two subgroups of O(n):

(a) the group D ± (n) := {ε = Diag(±1, ..., ±1)} ∼ = {-1, +1} n of diagonal matrices taking their diagonal values in {-1, +1}, (b) the group S ± (n) := {εP σ ∈ Mat(n)|(ε, σ) ∈ D ± (n) × S(n)} ∼ = D ± (n) × S(n) of signed permutation matrices.
Lemma 4.6 (Characterization of inner products on symmetric matrices invariant under

D ± (n) or S ± (n)) Let •|• : Sym(n) × Sym(n) -→ R be an inner product on symmetric matrices. (a) It is D ± (n)-invariant if and only if there exist n(n-1)
2 positive real numbers α ij = α ji > 0 for i = j and a matrix S ∈ Sym + (n) such that:

∀X ∈ Sym(n), X|X = i =j α ij X 2 ij + i,j S ij X ii X jj . (4.4) (b) It is S ± (n)-invariant if and only if there exist (α, β, γ) ∈ R 3 with α > 0, γ > β and γ + (n -1)β > 0 such that: ∀X ∈ Sym(n), X|X = γ n i=1 X 2 ii + α i =j X 2 ij + β i =j X ii X jj . (4.5)
See the proof of Lemma 4.6 in Section 11.3.

Proof of Theorem 4.5. An O(n)-invariant inner product on symmetric matrices is S ± (n)invariant so it is of the form of Equation (4.5). We define the rotation matrix 2) and we apply it to the matrix

R = Å R π/4 0 0 I n-2 ã ∈ O(n) with R π/4 = √ 2 2 Å 1 1 -1 1 ã ∈ O(
X = Å M Y Y Z ã ∈ Sym(n) with M = Å a b b c ã ∈ Sym(2). Since R π/4 M R π/4 = 1 2 Å a + c + 2b c -a c -a a + c -2b ã , the coefficient of b 2 in X|X in Equation (4.5) is 2α and the coefficient of b 2 in RXR |RXR is 2γ -2β.
Hence by invariance, γ = α + β and the positivity condition becomes α > 0 and α + nβ > 0. Conversely, Equation ( 4.3) clearly defines O(n)-invariant inner products.

Main O(n)-invariant metrics on SPD matrices with new formulae

The goal of this section is to describe the main O(n)-invariant metrics on SPD matrices that can be found in the literature, namely the Euclidean (abbreviated 'E', Section 4.3.1), the Log-Euclidean ('LE', Section 4.3.2), the Affine-invariant ('A', Section 4.3.3), the Bures-Wasserstein ('BW', Section 4.3.4) and the Bogoliubov-Kubo-Mori ('BKM', Section 4.3.5) metrics. For each metric, we give a short explanation on the way it was introduced, some useful references and a synthetic table that summarizes its fundamental Riemannian operations: squared distance, Levi-Civita connection, curvature, geodesics, logarithm map, parallel transport map.

Our contributions are (1) the synthesis of many results scattered in the literature especially for the Bures-Wasserstein metric, (2) the complete formula of the sectional curvature of the affine-invariant metric, (3) the new formula of the parallel transport between commuting matrices and new expressions of the Levi-Civita connection, the curvature and the parallel transport equation of the Bures-Wasserstein metric.

O(n)-invariant Euclidean metrics

A Euclidean metric on SPD matrices is the pullback of an inner product •|• on symmetric matrices by the canonical immersion id : Sym + (n) -→ Sym(n). As we know O(n)-invariant inner products on symmetric matrices from Theorem 4.5, we know all the O(n)-invariant Euclidean metrics on SPD matrices.

Definition 4.7 (O(n)-invariant Euclidean metrics on SPD matrices) An O(n)-invariant

Euclidean metric on SPD matrices is a Riemannian metric of the following form for all Σ ∈ Sym + (n) and X ∈ Sym(n):

g E(α,β) Σ (X, X) = α tr(X 2 ) + β tr(X) 2 , ( 4.6)
with (α, β) ∈ ST, i.e. α > 0 and β > -α/n. Its Riemannian operations are detailed in Table 4.3.

O(n)-invariant log-Euclidean metrics

A log-Euclidean metric on SPD matrices [START_REF] Arsigny | Log-Euclidean metrics for fast and simple calculus on diffusion tensors[END_REF] is the pullback of an inner product •|• on symmetric matrices by the symmetric matrix logarithm log : Sym + (n) -→ Sym(n). Hence the SPD manifold endowed with the log-Euclidean metric is isometric to a Euclidean space, thus geodesically complete. From Theorem 4.5 and the fact that d log :

Sym + (n) × Sym(n) -→ Sym(n) is O(n)-equivariant, we know all the O(n)-invariant log- Euclidean metrics.
Definition 4.8 (O(n)-invariant log-Euclidean metrics on SPD matrices) An O(n)-invariant log-Euclidean metric on SPD matrices is a Riemannian metric of the following form for all Σ ∈ Sym + (n) and X ∈ Sym(n):

g LE(α,β) Σ (X, X) = α tr(d Σ log(X) 2 ) + β tr(Σ -1 X) 2 , (4.7) Metric g Σ (X, X) = α X 2 2 + β tr(X) 2 Squared distance d(Σ, Λ) 2 = α Λ -Σ 2 2 + β(tr(Λ) -tr(Σ)) 2 Levi-Civita ∇ X Y = ∂ X Y Curvature R = 0 Geodesics γ (Σ,X) (t) = Σ + tX for t ∈ I
where I depends on λ min = min sp(Σ -1 X) and λ max = max sp(Σ -1 X) as follows:

• If λ min < 0 < λ max , then I = (-1/λ max , -1/λ min ). • If 0 λ min , then I = (-1/λ max , +∞). • If λ max 0, then I = (-∞, -1/λ min ). Logarithm Log Σ (Λ) = Λ -Σ Parallel transport
Does not depend on the curve:

Π Σ→Λ : ß T Σ Sym + (n) -→ T Λ Sym + (n) X -→ (d Λ id) -1 (d Σ id(X)) ≡ X Table 4.3: Riemannian operations of O(n)-invariant Euclidean metrics on SPD matrices.
with (α, β) ∈ ST, i.e. α > 0 and β > -α/n. Moreover, this metric is the pullback of the Frobenius log-Euclidean metric (α = 1 and β = 0) by the isometry

f p,q : Σ ∈ Sym + (n) -→ exp(F p,q (log Σ)) = det(Σ) p-q n Σ q ∈ Sym + (n) with p = √ α + nβ and q = √ α,
where F p,q was defined in Theorem 4.5. Its Riemannian operations are detailed in Table 4

.4. Metric g Σ (X, X) = α d Σ log(X) 2 2 + β tr(Σ -1 X) 2 Squared distance d(Σ, Λ) 2 = α log Λ -log Σ 2 2 + β log(det(Λ)/ det(Σ)) 2 Levi-Civita ∇ X Y = ∂ log X Y Curvature R = 0 Geodesics ∀t ∈ R, γ (Σ,X) (t) = exp(log(Σ) + t d Σ log(X)) Logarithm Log Σ (Λ) = (d Σ log) -1 (log Λ -log Σ) Parallel transport
Does not depend on the curve:

Π Σ→Λ : ß T Σ Sym + (n) -→ T Λ Sym + (n) X -→ (d Λ log) -1 (d Σ log(X))
Table 4.4: Riemannian operations of O(n)-invariant log-Euclidean metrics on SPD matrices.

Affine-invariant metrics

Affine-invariant metrics were introduced in many different ways. We adopt here the most recent viewpoint [Pennec, 2009], which underlies the term "affine-invariant". Consider SPD matrices Σ ∈ Sym + (n) as empirical covariance matrices of a random vector

X ∈ R n , namely Σ = 1 n (X -X)(X -X) . Define the affine action on vectors ((A, B), X) ∈ (GL(n) R n ) × R n -→ AX + B ∈ R n . Then, the induced action on SPD matrices is ((A, B), Σ) ∈ (GL(n) R n ) × Sym + (n) -→ AΣA ∈ Sym + (n).
It is simply the congruence action of GL(n) on matrices. Hence an affine-invariant metric on SPD matrices simply designates a GL(n)-invariant metric.

Historically, Siegel introduced a metric on the half space S = {X + iΣ| X ∈ Sym(n), Σ ∈ Sym + (n)} which is invariant under the action of the symplectic group [Siegel, 1943]. As a consequence, the restriction of this metric to SPD matrices by the immersion Σ ∈ Sym + (n) → iΣ ∈ S was proved to be invariant under GL(n) and under inversion and to provide a Riemannian homogeneous structure to Sym + (n). The expression of this metric is g

Σ (X, Y ) = tr(Σ -1 XΣ -1 Y ).
Rao considered the Fisher information of a family of densities as a Riemannian metric on the space of parameters [Rao, 1945] and Skovgaard detailed all the properties of the Fisher-Rao metric of the family of multivariate Gaussian densities [Skovgaard, 1984]. By restriction to the family of centered multivariate Gaussian densities, we get the same metric as Siegel's scaled by a factor 1/2, namely g Σ (X, Y ) = 1 2 tr(Σ -1 XΣ -1 Y ). In addition, Amari and Nagaoka stated that the canonical immersion id :

Σ ∈ Sym + (n) -→ Σ ∈ Sym(n) and the inversion inv : Σ ∈ Sym + (n) -→ Σ -1 ∈ Sym(n)
give two dual coordinate systems with respect to this metric [START_REF] Amari | Methods of Information Geometry[END_REF].

Between 2005 and 2007, this metric was used in many computational methods for Diffusion Tensor Imaging [START_REF] Pennec | A Riemannian Framework for Tensor Computing[END_REF], Lenglet et al., 2006, Fletcher and Joshi, 2007, Moakher, 2005, Batchelor et al., 2005], in functional MRI [START_REF] Varoquaux | Detection of brain functional-connectivity difference in post-stroke patients using group-level covariance modeling[END_REF] and in Brain-Computer Interfaces [START_REF] Barachant | Classification of covariance matrices using a Riemannian-based kernel for BCI applications[END_REF]. It was claimed to be the unique affineinvariant metric. However, Pennec showed that GL(n)-invariant metrics are characterized by O(n)-invariant inner products on the tangent space at I n , that is on symmetric matrices. Hence from Theorem 4.5, there is actually a two-parameter family of affine-invariant metrics [Pennec, 2009]. Definition 4.9 (Affine-invariant metrics on SPD matrices) An affine-invariant metric on SPD matrices is a GL(n)-invariant Riemannian metric. It is of the following form for all Σ ∈ Sym + (n) and X ∈ Sym(n):

g A(α,β) Σ (X, X) = α tr((Σ -1 X) 2 ) + β tr(Σ -1 X) 2 , ( 4.8)
with (α, β) ∈ ST, i.e. α > 0 and β > -α/n. The Fisher-Rao metric often refers to the affine-invariant metric with (α, β) = (1/2, 0). Moreover, given α > 0, this metric is the pullback of the affine-invariant metric with β = 0 by the isometry

f p,1 : Σ ∈ Sym + (n) -→ det(Σ) p-1 n Σ ∈ Sym + (n) with p = » α+nβ α .
The following proposition details the characteristics of homogeneity and symmetry of these Riemannian metrics. The Riemannian operations, essentially due to Skovgaard [Skovgaard, 1984], are detailed in Table 4.5. The second term of the sectional curvature is part of our contributions as it seems to be forgotten in [Skovgaard, 1984]. We prove the formula in Section 11.3. Proposition 4.10 (Riemannian symmetric structure of the affine-invariant metric) The Riemannian manifold (Sym + (n), g A(α,β) ) is a Riemannian symmetric space, hence it is geodesically complete. The underlying homogeneous space is GL + (n)/SO(n) and g A(α,β) is a quotient metric obtained by the submersion π :

A ∈ GL + (n) -→ AA ∈ Sym + (n) from the left-invariant metric G A (M, M ) = 4α tr(A -1 M (A -1 M ) ) + 4β tr(A -1 M ) 2 for A ∈ GL + (n) and M ∈ T A GL + (n). The symmetries are s Σ : Λ ∈ Sym + (n) -→ ΣΛ -1 Σ ∈ Sym + (n). Metric g Σ (X, X) = α Σ -1 X 2 2 + β tr(Σ -1 X) 2 Squared distance d(Σ, Λ) 2 = α log(Σ -1/2 ΛΣ -1/2 ) 2 2 + β log(det(Σ -1 Λ)) 2 Levi-Civita (∇ X Y ) |Σ = (∂ X Y ) |Σ -1 2 (XΣ -1 Y + Y Σ -1 X)

Curvature

The sectional curvature κ ∈ [-1/2α; 0]. More precisely, the Riemann and sectional curvatures are:

R Σ (X, Y, Z, T ) = α 2 (XΣ -1 Y Σ -1 (ZΣ -1 T -T Σ -1 Z)Σ -1 ) κ Σ (Σ 1/2 E β ii Σ 1/2 , Σ 1/2 E β ij Σ 1/2 ) = -1/4α for i = j κ Σ (Σ 1/2 E β ij Σ 1/2 , Σ 1/2 E β ik Σ 1/2 ) = -1/8α for i = j = k = i where E β ij = E ij -1-p np δ ij I n . Other terms are null. Geodesics ∀t ∈ R, γ (Σ,X) (t) = Σ 1/2 exp(t Σ -1/2 XΣ -1/2 )Σ 1/2 Logarithm Log Σ (Λ) = Σ 1/2 log(Σ -1/2 ΛΣ -1/2 )Σ 1/2
Parallel transport Depends on the curve. Along a geodesic:

Π Σ→Λ : ß T Σ Sym + (n) -→ T Λ Sym + (n) X -→ (ΛΣ -1 ) 1/2 X(Σ -1 Λ) 1/2
Table 4.5: Riemannian operations of affine-invariant metrics on SPD matrices.

Another metric that also provides a Riemannian symmetric structure on Sym + (n) was used in [START_REF] Su | Fitting smoothing splines to time-indexed, noisy points on nonlinear manifolds[END_REF], Zhang et al., 2018]. It was introduced directly by the quotient structure detailed in Proposition 4.10 but with the submersion

√ π : A ∈ GL + (n) -→ √ AA ∈ Sym + (n)
based on the polar decomposition of A (and without the coefficient 4). We called it the polar-affine metric in [START_REF] Thanwerdas | Is affineinvariance well defined on SPD matrices? A principled continuum of metrics[END_REF]

. It is GL(n)-invariant with respect to the action (A, Σ) ∈ GL(n) × Sym + (n) -→ √ AΣ 2 A ∈ Sym + (n). Hence it is O(n)-invariant

Bures-Wasserstein metric

The L 2 -Wasserstein distance between multivariate centered Gaussian distributions is given by d(Σ, Λ) 2 = tr(Σ) + tr(Λ) -2 tr((ΣΛ) 1/2 ). It corresponds to the Procrustes distance between square-root matrices, namely

d(Σ, Λ) 2 = inf U ∈O(n) Σ 1/2 -Λ 1/2 U 2
Frob . The second order approximation of this squared distance defines a Riemannian metric called the Bures metric (or the Helstrom metric) in quantum physics. All these viewpoints are explained in details with modern notations in [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF]. In particular, the expression of the Riemannian metric is derived in [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF] and we take it as a definition. ,d n ) and X = P X P , its expression is:

g BW Σ (X, X) = g BW D (X , X ) = 1 2 i,j 1 d i + d j X 2 ij . (4.9)
The Bures-Wasserstein metric can also be expressed by means of the linear map S Σ : Sym(n) -→ Sym(n) implicitly defined by the Sylvester equation X = ΣS Σ (X) + S Σ (X)Σ for X ∈ Sym(n). More explicitly with the previous notations, we have S Σ (X) = P

X ij d i +d j i,j P . Then we have g BW Σ (X, Y ) = 1 2 tr(XS Σ (Y )) = tr(S Σ (X)ΣS Σ (Y )), where X, Y ∈ T Σ Sym + (n) are canonically identified with d Σ id(X), d Σ id(Y ) ∈ Sym(n),
as explained in the introduction. This is a common expression in recent papers [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF], van Oostrum, 2020]. However, in [Takatsu, 2011] which is a reference paper on the Bures-Wasserstein metric, Takatsu gives the expression g Σ (X, Y ) = tr(XΣY). The trick comes from the identification S Σ (X) ≡ X ∈ Sym(n) that differs from the canonical one d Σ id(X) ≡ X ∈ Sym(n). As this could be confusing when the formula is written without this precision (and without bold letters), we adopt the same formalism as in [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF], Bhatia et al., 2019, van Oostrum, 2020].

Bundle GL(n) Group action ρ : (A, U ) ∈ GL(n) × O(n) -→ AU ∈ GL(n) Submersion π : A ∈ GL(n) -→ Σ := AA ∈ Sym + (n) Vertical space V A = ker d A π = Skew(n) A - Bundle metric G A (M, M ) = tr(M M ) Hor. space H A = V ⊥ G A = Sym(n)A Hor. isometry (d A π) |H A : ß H A = Sym(n)A -→ T Σ Sym + (n) X h = X 0 A -→ X = ΣX 0 + X 0 Σ Sym. lift X 0 S Σ : ® T Σ Sym + (n) -→ H In = Sym(n) X -→ X 0 = P X 0 P with X 0 ij = X ij d i +d j Hor. lift X h X ∈ T Σ Sym + (n) -→ X h = X 0 A ∈ H A Table 4
.6: Quotient structure of the Bures-Wasserstein metric.

We recall the quotient structure of the Bures-Wasserstein metric [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF] The proofs of the formulae of the distance and the logarithm can be found in [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF]. The Levi-Civita connection and the exponential map were computed in [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF]. We computed the Levi-Civita connection independently using a more geometric proof that we provide in Section 11.3. We get a simpler formula.

Takatsu computed the curvature in [Takatsu, 2010] in a basis of vectors and gave a general formula in [Takatsu, 2011]. However, we argued above that the notations of [Takatsu, 2011] could be confusing because of the chosen identification. Moreover, the expression of the curvature given there is a bit implicit since it is

R Σ (X, Y, X, Y ) = 3 4 tr((|Y, X] -S)Σ([Y, X] - S) ) where S = S Σ ([X, Y ]Σ + Σ[Y, X]) ∈ Sym(n)
. Therefore, we prove in Section 11.3 the compact and explicit formula provided in Table 4.7 using the same method: O'Neill's equations of submersions [O'Neill, 1966].

Finally, the geodesic parallel transport between commuting SPD matrices is new. The proof is given in Section 11.3. We provide a new formulation of the equation of the parallel transport between any two SPD matrices in the following proposition. It is used in the Python package geomstats [Miolane et al., 2020a] to compute the parallel transport.

Metric

g Σ (X, X) = g Σ 1/2 (X h , X h ) = 1 2 i,j 1 d i +d j X 2 ij Squared distance d(Σ, Λ) 2 = trΣ + trΛ -2tr((ΣΛ) 1/2 ) Levi-Civita (∇ X Y ) |Σ = (∂ X Y ) |Σ -(X 0 ΣY 0 + Y 0 ΣX 0 ) Curvature The sectional curvature is non-negative. More precisely R Σ (X, Y, X, Y ) = 3 2 i,j d i d j d i +d j X 0 , Y 0 2 ij where [V, W ] = V W -W V is the Lie bracket of matrices.
Geodesics γ (Σ,X) (t) = Σ + tX + t 2 X 0 ΣX 0 for t ∈ I where I depends on λ max = max sp(X 0 ) and λ min = min sp(X 0 ) as follows:

• If λ min < 0 < λ max , then I = (-1/λ max , -1/λ min ). • If 0 λ min , then I = (-1/λ max , +∞). • If λ max 0, then I = (-∞, -1/λ min ). Logarithm Log Σ (Λ) = (ΣΛ) 1/2 + (ΛΣ) 1/2 -2Σ Parallel transport
Depends on the curve. Along a geodesic between commuting matrices Σ = P DP and Λ = P ∆P :

Π Σ→Λ : T Σ Sym + (n) -→ T Λ Sym + (n) X -→ P δ i +δ j d i +d j [P XP ] ij P Table 4
.7: Riemannian operations of the Bures-Wasserstein metric on SPD matrices.

Proposition 4.12 (Parallel transport equation of Bures-Wasserstein metric) Let γ(t) the geodesic between γ(0) = Σ and γ(1) = Λ, and a vector

X ∈ T Σ Sym + (n). We denote γ h (t) = (1 -t)Σ 1/2 + tΣ -1/2 (Σ 1/2 ΛΣ 1/2
) 1/2 the horizontal lift of the geodesic γ. The two following statements are equivalent.

(i) The vector field X(t) defined along γ(t) is the parallel transport of X.

(ii) X(t) = γ(t)X 0 (t) + X 0 (t)γ(t) where X 0 (t) is a curve in Sym(n) satisfying the following ODE: γ(t) Ẋ0 (t) + Ẋ0 (t)γ(t) + γ h (t) γh X 0 (t) + X 0 (t) γh γ h (t) = 0. (4.10)
See the proof of Proposition 4.12 in Section 11.3.

Interestingly, this equation resembles the parallel transport equation on the Kendall shape space, which is also a quotient space, derived in [Kim et al., 2021, Proposition 3.1].

Bogoliubov-Kubo-Mori metric

The Bogoliubov-Kubo-Mori metric is a Riemannian metric used in quantum physics [START_REF] Petz | The Bogoliubov inner product in quantum statistics[END_REF], given by g

BKM Σ (X, X) = tr( ∞ 0 (Σ + t I n ) -1 X(Σ + t I n ) -1 Xdt).
It can be seen as the integration of the affine-invariant metric on a half-line included in the SPD cone. It can be rewritten thanks to the differential of the logarithm and we take this other expression as a definition.

Definition 4.13 (Bogoliubov-Kubo-Mori (BKM) metric) The Bogoliubov-Kubo-Mori metric is the O(n)-invariant Riemannian metric defined for Σ ∈ Sym + (n) and X ∈ T Σ Sym + (n) by: g BKM Σ (X, X) = tr(X d Σ log(X)). (4.11)
Important functions related to this metric are defined by [START_REF] Michor | The Curvature of the Bogoliubov-Kubo-Mori Scalar Product on Matrices[END_REF] to get simple expressions of the Levi-Civita connection and the curvature.

Given Σ = P DP ∈ Sym + (n), they define m ij = ∞ 0 (d i + t) -1 (d j + t) -1 dt which is symmetric in (i, j) and m ijk = ∞ 0 (d i + t) -1 (d j + t) -1 (d k + t) -1 dt which is symmetric in (i, j, k). They also denote g Σ (X) = d Σ log(X) whose expression is g Σ (X) = P g D (X ) P and [g D (X )] ij = m ij X ij where X = P XP . This g Σ is defined so that g Σ (X, Y ) = tr(X g Σ (Y ))
. By differentiating this equality and using the definition of the BKM metric, they get the differential of Σ -→ g Σ :

d Σ g(P F ij P )(P F kl P ) = d D g(F ij )(F kl ) = - 1 2 (δ jk m ilj F il + δ jl m ikj F ik + δ il m jki F jk + δ ik m jli F jl ), or more compactly [d Σ g(P XP )(P XP )] ij = -2 n k=1 m ijk X ik X jk .
The Levi-Civita connection and the curvature can be expressed in closed forms by means of g and dg, as shown in Table 4.8. Note that the sign of the sectional curvature is not known. The distance, exponential, logarithm and parallel transport maps are not known either.

Metric g Σ (X, X) = tr(X d Σ log(X)) Levi-Civita (∇ X Y ) |Σ = (∂ X Y ) |Σ + 1 2 g -1 Σ (d Σ g(X)(Y )) Curvature R Σ (X, Y )Z = -1 4 g -1 Σ (d Σ g(X)(g -1 Σ (d Σ g(Y )(Z)))) + 1 4 g -1 Σ (d Σ g(Y )(g -1 Σ (d Σ g(X)(Z))))
Table 4.8: Riemannian operations of the BKM metric on SPD matrices.

In this section, we reviewed five of the mainly used O(n)-invariant Riemannian metrics and we contributed new formulae. We also highlighted that the O(n)-invariant Euclidean, the O(n)-invariant log-Euclidean and the affine-invariant metrics are actually two-parameter families of Riemannian metrics indexed by (α, β) ∈ ST while this extra term weighted by the trace factor β is never defined in the literature for the Bures-Wasserstein and the Bogoliubov-Kubo-Mori metrics. Actually, there does not seem to exist a natural way of extending them with a trace term. Indeed, under the Bures-Wasserstein metric, there is a choice of an O(n)right-invariant inner product on GL(n) but they differ from O(n)-invariant inner products on symmetric matrices given in Theorem 4.5. Indeed, any inner product on GL(n) of the form

X|X = tr(X SX) with S ∈ Sym + (n) is O(n)-right-invariant.
As for the BKM metric, we could change the inner product in the integral but after computation, we would obtain this metric: α g BKM Σ (X, X) + β i,j log [1] (d i , d j )X ii X jj . The fact that we cannot separate the indices i and j in the trace term differs from the previous situations.

In the next section, we recall the definition of the class of kernel metrics [Hiai andPetz, 2009, Hiai andPetz, 2012] and a selection of its key properties. Since this class of Riemannian metrics contains all the previously introduced metrics without trace term, we show that this is the right framework to define the trace term extension. We show that this new class of extended kernel metrics still satisfies the key results on kernel metrics we selected. We also prove another property of these two classes: the stability under the cometric.

The interpolating class of kernel metrics: new observations

Kernel metrics were introduced by Hiai and Petz in 2009 [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF]. It is a family of O(n)-invariant metrics indexed by smooth bivariate functions φ : (R + ) 2 -→ R + called kernels. It has several key properties and it encompasses all the O(n)-invariant metrics introduced in Section 4.3 without trace term (β = 0). After recalling these key results (Section 4.4.1), we provide new observations on kernel metrics (Section 4.4.2), especially the trace term extension and the stability under the cometric.

The general class of kernel metrics

Definition 4.14 (Kernel metrics, mean kernel metrics) [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF] A kernel metric is an O(n)-invariant metric for which there is a smooth bivariate map φ

: (R + ) 2 -→ R + such that g Σ (X, X) = g D (X , X ) = i,j 1 φ(d i ,d j ) X 2 ij
, where Σ = P DP with P ∈ O(n) and D = Diag(d 1 , ..., d n ), and X = P X P .

A mean kernel metric is a kernel metric characterized by a bivariate map φ of the form φ(x, y) = a m(x, y) θ where a > 0 is a positive coefficient, θ ∈ R is a homogeneity power and m : (R + ) 2 -→ R + is a symmetric homogeneous mean, that is:

1. symmetric, i.e. m(x, y) = m(y, x) for all x, y > 0, 2. homogeneous, i.e. m(λx, λy) = λ m(x, y) for all λ, x, y > 0, 3. non-decreasing in both variables, 4. min(x, y) m(x, y) max(x, y) for all x, y > 0. It implies m(x, x) = x.

As the goal of this chapter is to extend the class of kernel metrics, we selected from [Hiai andPetz, 2009, Hiai andPetz, 2012] the results that we found simple and powerful to be able to generalize them later on. It would be interesting to study other properties such as monotonicity and comparison properties but it is beyond our scope. Our selection of results is in Proposition 4.15. Proposition 4.15 (Key results on kernel metrics) [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF] 1. (Generality) The Euclidean, log-Euclidean and affine-invariant metrics without trace term (β = 0), the polar-affine, the Bures-Wasserstein and the Bogoliubov-Kubo-Mori metrics are mean kernel metrics. The kernels and the names of the corresponding means are given in Table 4.9.

(Stability)

The class of kernel metrics is stable under univariate diffeomorphisms. More precisely, if g is a kernel metric with kernel function φ and if f is a univariate diffeomorphism (defined in Section 4.2.2), then the pullback metric f * g is a kernel metric with bivariate function (x, y)

-→ φ(f (x),f (y)) f [1] (x,

Arithmetic mean 1 BKM

x-y log(x)-log(y)

Logarithmic mean 1 Table 4.9: Bivariate functions of all the O(n)-invariant metrics of Section 4.3.

(Completeness)

A mean kernel metric with homogeneity power θ is geodesically complete if and only if θ = 2. Therefore this result provides a sufficient condition for kernel metrics to be geodesically complete.

Another property that we left for a different reason is the attractivity of the Log-Euclidean metric, i.e. the fact that the log-Euclidean metric is the limit when p tends to 0 of the pullback of a kernel metric by a power diffeomorphism pow p : Σ ∈ Sym + (n) -→ Σ p ∈ Sym + (n), scaled by 1 p 2 . However, it is not specific to kernel metrics since this is the case for any metric g.

New observations on kernel metrics 4.4.2.1 Kernel metrics form a cone

The class of kernel metrics is a sub-cone of the cone of Riemannian metrics on the SPD manifold. Indeed, it is stable by positive scaling and it is convex because if g, g are kernel metrics associated to φ, φ , then (1 -t)g + tg is a kernel metric associated to φφ /((1 -t)φ + tφ) > 0 for t ∈ [0, 1].

Cometric stability of the class of kernel metrics

A Riemannian metric g : T M × T M -→ R on a manifold M defines a cometric g * :

T * M × T * M -→ R defined for all covectors ω, ω ∈ T * M by g * (ω, ω ) = ω(x ) where x ∈ T M is the unique vector such that for all vectors x ∈ T M, g(x, x ) = ω (x) (Riesz's theorem).
On the manifold of SPD matrices M = Sym + (n), we have a canonical identification of T Σ M with Sym(n) given by d Σ id. Hence by duality, we also have a canonical identification between T * Σ M and Sym(n) * . So to identify T Σ M with T * Σ M, we only need an identification between Sym(n) and Sym(n) * . This is provided by the Frobenius inner product. To summarize, there is a natural identification between the tangent space and the cotangent space given by:

ß T Σ Sym + (n) -→ T * Σ Sym + (n) X -→ (Y ∈ T Σ Sym + (n) -→ tr(d Σ id(X)d Σ id(Y ))) . (4.12)
Hence, a cometric on SPD matrices can be seen as a metric. Back to kernel metrics, it is interesting to note that this class is stable under taking the cometric and that the cometric has a simple expression. Proposition 4.16 (Cometric stability of kernel metrics) Let g be a kernel metric with kernel function φ. Then the cometric g * seen as a metric through the identification explained above is a kernel metric with kernel function φ * = 1/φ. This elementary fact is interesting from a numerical point of view. Indeed, to compute numerically the geodesics, one can either integrate the geodesic equation involving the Christoffel symbols (which is of second order) or integrate its Hamiltonian version involving the cometric (which is of first order). Hence, the fact that the cometric of a kernel metric is available is a quite important result that appeared to be previously unnoticed. More precisely, the geodesic equation writes ẍk + Γ k ij ẋi ẋj = 0 where x(t) is a curve on the manifold M and Γ k ij are the Christoffel symbols related to the metric by

Γ k ij = 1 2 g kl (∂ i g jl + ∂ j g il -∂ l g ij ).
By considering a curve p(t) on the cotangent bundle T * M instead, and x(t) the curve on the manifold M such that p(t) ∈ T *

x(t) M, the geodesic equation admits the following Hamiltonian formulation:

® ẋk = g kl p l ṗl = -1 2 ∂g ij ∂x l p i p j . (4.13)
The Hamiltonian equation is often preferred to compute the geodesics numerically since the integration is simpler and more stable. It only involves the cometric g * = (g ij ) i,j , which is very easy to compute for a kernel metric. This is how geodesics are generically computed in the python package geomstats for example.

Canonical Frobenius-like expression of a kernel metric

An expression of kernel metrics was given in [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF]] by means of the operators

L Σ : X -→ ΣX, R Σ : X -→ XΣ and φ(L Σ , R Σ ) : Sym(n) -→ Sym(n) defined for Σ = P DP ∈ Sym + (n) by φ(L Σ , R Σ )X = P [φ(d i , d j )] i,j
• (P XP ) P , where • denotes the Schur (entry-wise) product. This expression is

g φ Σ (X, X) = tr(Xφ(L Σ , R Σ ) -1 (X)). The existence of the map Φ : Sym + (n) × Sym(n) -→ Sym(n) hidden in φ(L Σ , R Σ ) := Φ Σ is ensured by extending the O(n)-equivariant map Φ : Diag + (n) × Sym(n) -→ Sym(n) defined by [Φ D (X)] ij = φ(d i , d j )X ij .
Indeed, one can easily check that Φ satisfies the two hypotheses of Lemma 4.2. In this work, we even prefer to define the bivariate map ψ = φ -1/2 and define in a analogous way the map Ψ : Sym + (n) × Sym(n) -→ Sym(n) so that we can write the kernel metric with a suitable Frobenius-like expression:

g φ Σ (X, X) = tr(Ψ Σ (X) 2 ). (4.14)
We can give explicitly Ψ in some particular cases:

1. Euclidean metric: Ψ E Σ (X) = X; 2. log-Euclidean metric: Ψ LE Σ (X) = d Σ log(X); 3. affine-invariant metric: Ψ A Σ (X) = Σ -1/2 XΣ -1/2 .
This is an important step towards the trace term extension.

Kernel metrics with a trace term

The class of kernel metrics does not encompass the O(n)-invariant Euclidean, O(n)-invariant log-Euclidean and affine-invariant metrics with a trace factor β = 0. However, thanks to the previous canonical expression, we can define a natural extension of a kernel metric with a trace term.

Definition 4.17 (Extended kernel metrics) Let g φ be a kernel metric associated to the kernel function φ : (R + ) 2 -→ R + . We define the map ψ = φ -1/2 and the map Ψ : Sym + (n) × Sym(n) -→ Sym(n) as described above so that g Σ (X, X) = tr(Ψ Σ (X) 2 ). We define a twoparameter family which extends the kernel metric g φ for all Σ ∈ Sym + (n) and X ∈ Sym(n) by:

g φ,α,β Σ (X, X) = α tr(Ψ Σ (X) 2 ) + β tr(Ψ Σ (X)) 2 , (4.15)
where (α, β) ∈ ST, i.e. α > 0 and α + nβ > 0.

We can apply this definition to the Bures-Wasserstein and the BKM metrics. One can show that the trace term such defined is β tr(Σ -1/2 X) 2 . Contrarily to the log-Euclidean and the affine-invariant cases, there is no isometry a priori between two metrics of the family. It is interesting to note that Propositions 4.15 and 4.16 are still valid for these extended kernel metrics. We omit the proofs since they are analogous to the ones given for kernel metrics in [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF]. 3. (Completeness) An extended mean kernel metric with homogeneity power θ is geodesically complete if and only if θ = 2.

(Cometric)

The class of extended kernel metrics is cometric-stable and the correspond-

ing transformation is (φ, α, β) -→ ( 1 φ , 1 α , -β α(α+nβ) ).
In this section, we recalled the definition of kernel metrics and three key properties. We added the property of stability under the cometric with an explicit expression and we argued that it is an interesting property from a numerical point of view to compute geodesics. We found a wider class of metrics which satisfies the same key properties and which encompasses all the O(n)-invariant metrics defined in Section 4.3. It is now tempting to look for wider classes of O(n)-invariant metrics and to determine if these properties are still valid.

In the next section, we characterize O(n)-invariant metrics by means of three multivariate functions satisfying conditions of symmetry, compatibility and positivity. This result allows to understand better the specificity of kernel metrics and extended kernel metrics within the whole class of O(n)-invariant metrics. Then we give a counterpart of Proposition 4.18 and we propose a new intermediate class of O(n)-invariant metrics which is cometric stable.

Characterization of O(n)-invariant metrics

In this section, we give a characterization of O(n)-invariant metrics on SPD matrices. We present it as an extension of Theorem 4.5 characterizing O(n)-invariant inner products on symmetric matrices. Instead of two parameters α, β which satisfy a positivity condition, an O(n)-invariant metric is characterized by three multivariate functions α, β, γ : (R + ) n -→ R which satisfy a positivity condition plus a symmetry condition and a compatibility condition. This is explained in Section 4.5.1. We also give two corollary results which characterize two subclasses of O(n)-invariant metrics with additional invariances: scaling invariance and inverse-consistency. Section 4.5.2 is dedicated to the proof of the theorem. In Section 4.5.3, we reinterpret kernel metrics in light of the theorem. In Section 4.5.4, we give key results on O(n)-invariant metrics and we compare them to those on kernel metrics given in Proposition 4.15. In particular, we state that the cometric can be difficult to compute. Hence in Section 4.5.5, we introduce the class of bivariate separable metrics which is an intermediate class between O(n)-invariant and extended kernel metrics, which is cometric-stable and for which the cometric is known in closed-form.

Theorem and corollaries

Let us rephrase the characterization of O(n)-invariant inner products on Sym(n) (Theorem 4.5). An inner product The characterization of O(n)-invariant metrics on Sym + (n) has an analogous form where real numbers are replaced by n-multivariate functions and where there is an additional property of symmetry of these functions. We introduce this notion of symmetry before stating the theorem. The proof is in Section 4.5.2.

•|• on Sym(n) is O(n)-invariant if and only if there exist real numbers γ, α > 0 and β ∈ R such that: X|X = γ i X 2 ii + α i =j X 2 ij + β i =j X ii X jj , ( 4
Definition 4.19 ((k, n -k)-symmetric functions) We say that a function f : (R + ) n -→ R is (k, n -k)-symmetric if it
) := f (σ • d) where σ({1, ..., k}) = I and (σ • d) i = d σ(i) . Theorem 4.20 (Characterization of O(n)-invariant metrics) Let g be a Riemannian metric on Sym + (n). If g is O(n)-invariant, then there exist three maps γ, α : (R + ) n -→ R + and β : (R + ) n -→ R such that for all Σ = P DP ∈ Sym + (n) and X = P X P ∈ T Σ Sym + (n): g Σ (X, X) = g D (X , X ) (4.17) = i γ(d i , d k =i )X 2 ii + i =j α(d i , d j , d k =i,j )X 2 ij + i =j β(d i , d j , d k =i,j )X ii X jj , 0. (Symmetry) γ is (1, n -1)-symmetric and α, β are (2, n -2)-symmetric, 1. (Compatibility) γ equals α + β on the set D = {d ∈ (R + ) n |d 1 = d 2 }, 2. (Positivity) for all d ∈ (R + ) n , the symmetric matrix S(d) defined by S ii (d) = γ(d i , d k =i ) and S ij (d) = β(d i , d j , d k =i,j ) is positive definite.
Conversely, if there exist such maps α, β, γ, then Equation (4.17) correctly defines an O(n)invariant Riemannian metric that we denote g α,β,γ or equivalently g α,S . Moreover, g is continuous if and only if α, β, γ are continuous.

Before giving the proof, we observe that this theorem allows to characterize subclasses of O(n)-invariant metrics as well. Here we give the general form of O(n)-invariant metrics that are invariant under scaling and under inversion respectively. We omit the proof.

1. g is invariant under scaling if and only if f (λd) = 1 λ 2 f (d) for f ∈ {α, β, γ}, for all d ∈ (R + ) n and for all λ > 0. 2. g is invariant under inversion if and only if γ(d -1 1 , ..., d -1 n ) = d 4 1 γ(d 1 , ..., d n ) and f (d -1 1 , ..., d -1 n ) = d 2 1 d 2 2 f (d 1 , ..., d n ) for f ∈ {α, β}, for all d ∈ (R + ) n .

Proof of the theorem

Proof of Theorem 4.20 (Characterization of O(n)-invariant metrics). Let g be an O(n)-invariant metric on Sym + (n). Since any diagonal matrix D is invariant under the subgroup D ± (n), the inner product g D is D ± (n)-invariant. Hence, Lemma 4.6 (a) ensures that there are positive coefficients α ij (D) = α ji (D) and a matrix S(D)

∈ Sym + (n) s.t. g D (X, X) = i =j α ij (D)X 2 ij + i,j S ij (D)X ii X jj .
Then, we define the three maps:

• α : d ∈ (R + ) n -→ α 12 (Diag(d)) > 0, • β : d ∈ (R + ) n -→ S 12 (Diag(d)), • γ : d ∈ (R + ) n -→ S 11 (Diag(d)) > 0.
Following the same idea as in the proof of Lemma 4.6 (b), we use the invariance under permutations since Diag + (n) is stable under this action. Then, one easily checks that α, β are (2, n -2)-symmetric and γ is (1, n -1)-symmetric and that we can express the other coefficients in function of α, β, γ by permuting the d i 's. We get for i = j:

• α ij (Diag(d)) = α(d i , d j , d k =i,j ), • S ij (Diag(d)) = β(d i , d j , d k =i,j ), • S ii (Diag(d)) = γ(d i , d k =i ).
So we get the expression (4.17), the symmetry and the positivity conditions. We only miss the compatibility condition so let d

= (d 1 , ..., d n ) ∈ (R + ) n such that d 1 = d 2 . Since D = Diag(d) is stable under any block-diagonal orthogonal matrix R = Diag(R θ , I n-2 ) ∈ O(n) with R θ ∈ O(2)
, with the same computations as in the proof of Theorem 4.5, we get γ

(d) = α(d) + β(d).
Conversely, if α, β, γ are three maps satisfying the conditions of symmetry, compatibility and positivity, then we define g D (X, X)

= i γ(d i , d k =i )X 2 ii + i =j α(d i , d j , d k =i,j )X 2 ij + i =j β(d i , d j , d k =i,j )X ii X jj .
In other words, we define a map g : Diag + (n) × Sym(n) × Sym(n) -→ R and we would like to extend it by defining g P DP (X, X) = g D (P XP, P XP ). According to Lemma 4.2, we have two cases to study. One can easily show that the first condition with permutations is satisfied. The non-trivial condition is the second one, involving a diagonal matrix

D = Diag(λ 1 I m 1 , ..., λ p I mp ) with sorted diagonal values λ 1 > ... > λ p > 0 and a block-diagonal orthogonal matrix R = Diag(R 1 , ..., R p ) ∈ O(n) with R k ∈ O(m k ). So we have to show that g D (R XR, R XR) = g D (X, X) for all matrix X ∈ Sym(n), since R DR = D. We denote Xkl ∈ Mat(m k , m l ) the (k, l) block matrix defined by Xkl ij = X n k-1 +i,n l-1 +j where n k = k j=1 m j . Note that Xkk ∈ Sym(m k ) is the k-th diagonal block of X and Xlk = ( Xkl ) . Therefore R XR kl = R k Xkl R l .
In the following, we split the sums between the blocks with multiplicity 1 and the blocks with higher multiplicity and we use the compatibility condition. The notation α(λ k , λ l , ...) stands for α(d i , d j , d m =i,j ) where λ k = d i and λ l = d j , i.e. n k-1 + 1 i n k and n l-1 + 1 j n l . We compute the difference:

g D (R XR, R XR) -g D (X, X) = k:m k =1 γ(d n k , d m =n k )((R XR) 2 n k n k -X 2 n k n k ) 0 + k =l m k =m l =1 α(λ k , λ l , ...)((R XR) 2 n k n l -X 2 n k n l ) 0 + k =l m k =m l =1 β(λ k , λ l , ...)((R XR) n k n k (R XR) n l n l -X n k n k X n l n l ) 0 + k:m k >1 γ(λ k , λ k , ...) α(λ k ,λ k ,...)+β(λ k ,λ k ,...) n k i=n k-1 +1 ((R XR) 2 ii -X 2 ii ) + k,l m k or m l >1 α(λ k , λ l , ...) n k-1 +1 i n k n l-1 +1 j n l i =j ((R XR) 2 ij -X 2 ij ) + k,l m k or m l >1 β(λ k , λ l , ...) n k-1 +1 i n k n l-1 +1 j n l i =j ((R XR) ii (R XR) jj -X ii X jj ).
Hence the missing term i = j in the two last sums is provided by the sum weighted by γ.

After a change of indexes based on the equality R XR kl = R k Xkl R l , we get:

g D (R XR, R XR) -g D (X, X) = k,l m k or m l >1 α(λ k , λ l , ...) m k i=1 m l j=1 ((R k Xkl R l ) 2 ij -( Xkl ) 2 ij ) tr(R k Xkl R l (R k Xkl R l ) )-tr( Xkl ( Xkl ) )=0 + k,l m k or m l >1 β(λ k , λ l , ...) m k i=1 m l j=1 ((R k Xkk R k ) ii (R l Xll R l ) jj -Xkk ii Xll jj ) tr(R k Xkk R k )tr(R l Xll R l )-tr( Xkk )tr( Xll )=0 = 0.
This proves that g Σ is well defined for all Σ ∈ Sym + (n) and O(n)-invariant by construction. The positivity condition ensures that g is a metric.

Finally, it is clear that α, β, γ have at least the same regularity as the metric g since they are coordinates of the map D ∈ Diag + (n) -→ g D . By Theorem 2.18, if α, β, γ are continuous, then the metric g is continuous.

The smoothness seems to be more complicated to study. We suspect additional conditions of compatibility on the derivatives of the smooth maps α, β, γ at the singular set of SPD matrices with repeated eigenvalues in order to make the metric g is smooth.

Reinterpretation of kernel metrics

Theorem 4.20 allows to reinterpret kernel metrics. The curiosity of this theorem is the function γ because we have no information on it as soon as the d i 's are distinct. If α, β, γ do not depend on their n -2 last arguments, i.e. if they are bivariate, then γ does not depend on its second argument and γ(d 1 ) must be equal to α(d

1 , d 1 ) + β(d 1 , d 1 ). Hence g Σ (X, X) = i,j α(d i , d j )X 2 ij + i,j β(d i , d j )
X ii X jj with α > 0 and α + nβ > 0, which is much more tractable. Moreover, if β = 0, then the quadratic form has a diagonal expression (sum of squares X 2 ij , no mixed terms X ii X jj ) in the basis of matrices induced by the orthogonal matrix P ∈ O(n) in the eigenvalue decomposition of Σ. In this case, we say that the metric is ortho-diagonal.

To sum up, the subclass of kernel metrics has two fundamental properties: it is bivariate (α = γ -β = 1/φ) and ortho-diagonal (β = 0). This is the reason why we propose to designate kernel (resp. mean kernel) metrics as Bivariate Ortho-Diagonal or BOD metrics (resp. Mean Ortho-Diagonal or MOD metrics), as summarized in Table 4.10. We say that the metric is Bivariate Ortho-ST when it is the extension by Definition 4.17 of a Bivariate Ortho-Diagonal metric with the Scaling and Trace factors α > 0 and β > -α/n. Hence, the extended (mean) kernel metrics can also be designated as BOST (and MOST) metrics.

Key results on O(n)-invariant metrics

In Section 4.

+ (n) -→ Sym + (n)
and in this case, the pullback metric f * g α,β,γ is characterized by the three maps:

(a) α f : d ∈ (R + ) n -→ α(f (d)) f [1] (d 1 ,d 2 ) 2 , (b) β f : d ∈ (R + ) n -→ β(f (d)) f [1] (d 1 ,d 2 ) 2 , (c) γ f : d ∈ (R + ) n -→ γ(f (d)) f (d 1 ) 2 .
3. (Completeness) Let g = g α,β,γ be an O(n)-invariant metric. We assume that α, β, γ satisfy a homogeneity property which is similar to the one assumed for mean kernel metrics: there exists θ ∈ R such that for f ∈ {α, β, γ}, x ∈ (R + ) n and λ > 0, we have

f (λx) = λ -θ f (x).
If the metric g is geodesically complete, then θ = 2.

(Cometric)

The class of O(n)-invariant metrics is obviously cometric-stable. The cometric is characterized by α * = 1/α and S * = S -1 where S(d)

∈ Sym + (n) is defined by S ij (d) = β(d i , d j , d k =i,j ) and S ii (d) = γ(d i , d k =i ) for all d ∈ R + and i = j.
We omit the proof since it consists in elementary verifications for all but the third statement, whose proof is analogous to the one given in [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF].

About completeness, the result is much weaker for general O(n)-invariant metrics. Indeed, we lost the converse implication: "if θ = 2, then the metric is geodesically complete". According to the proof of [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF], the key element to prove this converse implication is exactly the bivariance, plus the fact that a symmetric homogeneous mean satisfies m(x, x) = x. It is worth noticing that θ = 2 is still necessary though.

About the cometric, we lost the closed-form expression we had for BOD and BOST metrics. Computing the cometric is numerically quite heavy in general because it is equivalent to invert the matrix S(d) for all d ∈ (R + ) n . However, note that when β = 0, the cometric is obviously given by the triple (1/α, 0, 1/γ). These ortho-diagonal metrics can be seen as the multivariate generalization of BOD metrics. In the next section, we give a cometric-stable extension of the class of BOST metrics for which the cometric can be computed in closed form: the class of bivariate separable metrics.

Bivariate separable metrics

We argued in Section 4.5.3 that bivariate metrics are of the form g Σ (X, X) = i,j α(d i , d j )X 2 ij + i,j β(d i , d j )X ii X jj with α > 0 and α + nβ > 0. Then, the first term corresponds to a BOD metric and it can be rewritten tr(Ψ Σ (X) 2 ), but it is still difficult to write the second term in a more compact way. If the function β is separable, i.e. if β can be written β(x, y) = ψ (1) (x)ψ (2) (y), then the second term is simply tr(Ψ

(1) Σ (X))tr(Ψ (2) Σ (X)). Indeed, we can define Ψ (k) D (X) = Diag(ψ (k) (d i )X ii ) and extend it into Ψ (k)
Σ as explained in Section 4.4.2.3. In particular, BOST metrics correspond to the case when β(x, y) = λ α(x, x)α(y, y) with 1 + nλ > 0. The wider class of bivariate separable metrics is actually cometric-stable and the cometric can be computed quite easily. This is stated in Proposition 4.23. Proposition 4.23 (Cometric of bivariate separable metrics) Let ψ : (R + ) 2 -→ R + be a symmetric map and let ψ (1) , ψ (2) : R + -→ R + be two maps on positive real numbers. As explained above, we define their extensions Ψ, Ψ (1) ,

Ψ (2) : Sym + (n) × Sym(n) -→ Sym(n). The quadratic form defined by g Σ (X, X) = tr(Ψ Σ (X) 2 ) + tr(Ψ (1) Σ (X))tr(Ψ (2)
Σ (X)) automatically satisfies the symmetry and compatibility conditions of Theorem 4.20. Then g is positive definite if and only if the vectors

x = x(d) = Ä ψ (1) (d i ) ψ(d i ,d i ) ä 1 i n and y = y(d) = Ä ψ (2) (d i ) ψ(d i ,d i ) ä 1 i n satisfy the inequality x y -x|y < 2 for all d ∈ (R + ) n .
In this case, we say that g is a Bivariate Separable metric. As an O(n

)-invariant metric, it is characterized by α(d) = ψ(d 1 , d 2 ) 2 and the matrix S = S(d) = ∆(I n + 1 2 (xy + yx ))∆ with ∆ = Diag(ψ(d i , d i ))
. This class of metrics is cometric-stable. If x = 0 or y = 0, the cometric at Σ is simply characterized by S -1 = ∆ -2 . Otherwise, the cometric is given by:

S -1 = ∆ -1 ï I n - 1 4c (2 + x|y )(xy + yx ) + 1 4c ( y 2 xx + x 2 yy ) ò ∆ -1 , (4.18) with c = 1 + x|y -1 4 ( x 2 y 2 -x|y 2 ) > 0.
Proof of Proposition 4.23. To determine when g is a metric, we express the functions α, β, γ, S of Theorem 4.20 in function of ψ, ψ (1) , ψ (2) :

1. α(d 1 , ..., d n ) = ψ(d 1 , d 2 ) 2 > 0, 2. β(d 1 , ..., d n ) = 1 2 (ψ (1) (d 1 )ψ (2) (d 2 ) + ψ (1) (d 2 )ψ (2) (d 1 )), 3. γ(d 1 , ..., d n ) = ψ(d 1 , d 1 ) 2 + ψ (1) (d 1 )ψ 2 (d 1 ), 4. hence S ij (d) = ∆ 2 ij + 1 2 (ψ (1) (d i )ψ (2) (d j ) + ψ (2) (d i )ψ (1) (d j )), so we have S = ∆(I n + 1 2 (xy + yx ))
∆ with the notations of the proposition. The symmetry and compatibility conditions of Theorem 4.20 are trivially satisfied. The positivity condition reduces to S ∈ Sym + (n), i.e.

I n + 1 2 (xy + yx ) ∈ Sym + (n).
As the eigenvalues of M = xy +yx are 0 (with multiplicity n-2) and x|y ± x y , S is positive definite if and only if 2 + x|y ± x y > 0. But x|y + x y 0 so there is only one condition: 2 > x y -x|y ( 0), as announced. Now, we want to compute S -1 . If x = 0 or y = 0, the result is obvious so we assume that x, y = 0. As M is of rank 2 at most, there exists a polynomial P of degree 3 at most such that

P (I n + 1 2 M ) = 0.
Let us find such a polynomial to compute S -1 . Since M 2 = x|y M + N with N = y 2 xx + x 2 yy and N M = x 2 y 2 M + x|y N , we have:

Å I n + 1 2 M ã 2 = I n + Å 1 + x|y 4 ã M + 1 4 N, Å I n + 1 2 M ã 3 = Å I n + 1 2 M ã 2 + 1 2 Å I n + 1 2 M ã 2 M = Å I n + 1 2 M ã 2 + 1 2 M + 1 2 Å 1 + x|y 4 ã M 2 + 1 8 N M = Å I n + 1 2 M ã 2 + 4 + 4 x|y + x|y 2 + x 2 y 2 8 M + 1 4 (2 + x|y )N = a Å I n + 1 2 M ã 2 + b 2 M -(2 + x|y )I n = a Å I n + 1 2 M ã 2 + b Å I n + 1 2 M ã + c I n , with      a = 3 + x|y b = -12-8 x|y -x|y 2 + x 2 y 2 4 c = 1 + x|y + x|y 2 -x 2 y 2 4 = 1 -a -b > 0 . Indeed, c > 1 + x|y -1 2 ( x y + x|y ) = 1 -1 2 ( x y -x|y ) > 0. Hence, denoting S 0 := I n + 1 2 M , we have S -1 0 = 1 c (S 2 0 -a S 0 -bI n ) = I n + 1 4c (N -(2+ x|y )M ) and S -1 = ∆ -1 I n + 1 4c (N -(2 + x|y )M ) ∆ -1 which is exactly Equation (4.18).
Finally, we want to prove that the cometric is bivariate separable. Regarding Equation (4.18), we look for x = Ax+By 4c and y = Cx + Dy for A, B, C, D ∈ R such that:

x y + y x = - 1 2c (2 + x|y )(xy + yx ) + 1 2c ( y 2 xx + x 2 yy ) (4.19) It is satisfied if AC = y 2 , BD = x 2 and AD + BC = -2(2 + x|y ), or equivalently (AX +B)(CX +D) = y 2 X 2 -2(2+ x|y )X + x 2 . This is a second-order polynomial with roots λ = 2+ x|y + √ δ y 2 and µ = 2+ x|y - √ δ y 2 where δ = (2+ x|y + x y )(2+ x|y -x y ) > 0 is the discriminant. Hence, it suffices to define A = y , B = -λ y , C = y and D = -µ y , so that S -1 = ∆ -1 I n + 1 2 (x y + y x ) ∆ -1 .
Hence, the cometric is bivariate separable and this class of metrics is cometric-stable.

Conclusion

To encompass all the O(n)-invariant metrics summarized in Section 4.3, including the ones with a trace term (β = 0), we defined the class of extended kernel metrics. This class satisfies the key results of stability and completeness we selected from [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF] plus the cometric-stability with cometric in closed form, which is important to compute geodesics numerically via the Hamiltonian formulation. Then, from the characterization of O(n)-invariant metrics in terms of three continuous maps α, β, γ : (R + ) n -→ R + satisfying properties of symmetry, compatibility and positivity, we were able to characterize kernel metrics as Bivariate Ortho-Diagonal (BOD) metrics. Among the key results on mean kernel metrics, the sufficient condition of completeness and the closed-form expression of the cometric disappear for general O(n)-invariant metrics. We finally defined the intermediate class of bivariate separable metrics which is cometric-stable and for which the cometric has a simple expression.

Since kernel metrics encompass very different metrics regarding curvature and completeness, it would be nice to introduce some more requirements on metrics to perform the opposite work of defining principled sub-classes of (mean) kernel metrics. In the next chapter, we propose some principled subfamilies of kernel metrics. It would also be interesting to rely on the cometric-stability of kernel metrics or super-classes to effectively compute the geodesics numerically and to investigate their properties regarding statistical analyses.

Another interesting direction would be to consider other properties of kernel metrics that were described in [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF], namely monotonicity and comparison properties. It would be challenging to understand how they could be generalized to BOST metrics or even to O(n)-invariant metrics. Furthermore, to our knowledge there is no trace of families of non O(n)-invariant metrics in the literature. However, there exist some situations where the O(n)-invariance is not relevant, for example on correlation matrices because the space is not stable under this group action. This is investigated in Part IV.

Chapter 5

Geometry of Mixed-Euclidean metrics Abstract

Several Riemannian metrics and families of Riemannian metrics were defined on the manifold of Symmetric Positive Definite (SPD) matrices. Firstly, we formalize a common general process to define families of metrics: the principle of deformed metrics. We relate the recently introduced family of alpha-Procrustes metrics to the general class of mean kernel metrics by providing a sufficient condition under which elements of the former belongs to the latter. Secondly, we focus on the principle of balanced bilinear forms that we recently introduced. We give a new sufficient condition under which the balanced bilinear form is a metric. It allows us to introduce the Mixed-Euclidean (ME) metrics which generalize the Mixed-Power-Euclidean (MPE) metrics. We unveal their link with the (u, v)-divergences and the (α, β)-divergences of information geometry and we provide an explicit formula of the Riemann curvature tensor. We show that the sectional curvature of all ME metrics can take negative values and we show experimentally that the sectional curvature of all MPE metrics but the log-Euclidean, power-Euclidean and power-affine metrics can take positive values.

This chapter was published in the journal Differential Geometry and its Applications in April 2022 under the title "The geometry of mixed-Euclidean metrics on symmetric positive definite matrices" [Thanwerdas and Pennec, 2022a]. All the proofs, originally in appendix of the paper, are deferred to Section 11.4.

Introduction

The convex cone of Symmetric Positive Definite (SPD) matrices is a manifold on which several Riemannian metrics were defined: Euclidean, Fisher-Rao/affine-invariant [Skovgaard, 1984, Amari and Nagaoka, 2000, Moakher, 2005, Pennec et al., 2006, Lenglet et al., 2006, Fletcher and Joshi, 2007], log-Euclidean [START_REF] Arsigny | Log-Euclidean metrics for fast and simple calculus on diffusion tensors[END_REF], Bures-Wasserstein [START_REF] Dowson | The Fréchet distance between multivariate normal distributions[END_REF], Olkin and Pukelsheim, 1982, Dryden et al., 2009, Takatsu, 2010, Takatsu, 2011, Bhatia et al., 2019], Bogoliubov-Kubo-Mori [Petz andToth, 1993, Michor et al., 2000], log-Cholesky [Lin, 2019], etc. Several families of metrics encompassing them were defined to understand their common properties, their differences and the level of generality of each property: kernel metrics and mean kernel metrics [Hiai andPetz, 2009, Hiai andPetz, 2012], power-Euclidean [START_REF] Dryden | Power Euclidean metrics for covariance matrices with application to diffusion tensor imaging[END_REF], alpha-Procrustes [Hà Quang, 2019], deformed-affine [START_REF] Thanwerdas | Is affineinvariance well defined on SPD matrices? A principled continuum of metrics[END_REF], mixed-power-Euclidean [Thanwerdas and Pennec, 2019a], extended kernel metrics, bivariate separable metrics [START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF], etc. In particular, kernel metrics form a very general family of O(n)-invariant metrics indexed by kernel maps φ : (0, ∞) 2 -→ (0, ∞) acting on the eigenvalues of SPD matrices. This family contains many O(n)-invariant metrics and it has good stability properties. The subclass of mean kernel metrics, for which the kernel maps have monotonicity properties, is interesting because it provides a necessary and sufficient condition for geodesic completeness. Hence, kernel metrics and mean kernel metrics appear as sufficiently general families with interesting properties so it is a natural framework to work in. However, this class contains metrics with very different geometries so it motivates us to define subfamilies of metrics which share more geometric properties with one another.

In previous works, we introduced two principles for building families of Riemannian metrics that share interesting properties: the principle of deformed metrics [START_REF] Thanwerdas | Is affineinvariance well defined on SPD matrices? A principled continuum of metrics[END_REF]] and the principle of balanced bilinear forms [Thanwerdas and Pennec, 2019a]. Deforming metrics (or datasets of SPD matrices) via a diffeomorphism is a very common procedure to define families of metrics. In particular, kernel metrics are stable by univariate diffeomorphisms, those which are characterized by their action on eigenvalues. However, mean kernel metrics are not stable by all univariate diffeomorphisms because of the monotonicity requirement. In this work, we gather many constructions of deformed metrics and we contribute a sufficient condition under which alpha-Procrustes metrics are mean kernel metrics.

The balanced bilinear form of two flat metrics is defined by composing the Frobenius inner product with the parallel transport of each flat metric [Thanwerdas and Pennec, 2019a]. When the bilinear form is a metric, it forms a dually-flat manifold along with the two flat Levi-Civita connections of the flat metrics. In the case where the two flat metrics are power-Euclidean metrics, the balanced bilinear form is a metric called the mixed-power-Euclidean metric. In this work, we give a new sufficient condition for a balanced bilinear form to be a metric, namely that the flat metrics are univariately-deformed-Euclidean metrics, which allows to define the new family of Mixed-Euclidean metrics. Then, we provide the geometric operations of Mixed-Euclidean metrics regarding information geometry and Riemannian geometry. In particular, our main contributions are on the one hand the link we establish between Mixed-Euclidean/Mixed-Power-Euclidean metrics and the (u, v)/(α, β)-divergences of information geometry, and on the other hand the expression of the Riemann curvature tensor of Mixed-Euclidean metrics.

In Section 5.2, we present our notations and the preliminary concepts of univariate maps and kernel metrics. In Section 5.3, we study deformed metrics and we relate the family of alpha-Procrustes metrics to the class of mean kernel metrics. In Section 5.4, we recall the main concepts of information geometry, we state the principle of balanced bilinear forms and we explain the relation between the two. In Section 5.5, we introduce the new family of Mixed-Euclidean metrics and we study its geometry. We conclude and discuss some perspectives in Section 5.6. The proofs of the results are presented in appendix.

Notations and preliminary concepts

Section 5.2 partly summarizes Chapter 4 to allow Chapter 5 to be read independently. The skilled reader may, after a glance at the definition of the second divided difference (Definition 5.2), skip this part to resume at Section 5.3.

In this section, we introduce some notations and we recall two concepts that are used throughout the chapter. The first one is the concept of univariate map on SPD matrices: it is a map acting on the eigenvalues, such as the symmetric matrix logarithm or the power maps. The successive differentials of smooth univariate maps can be expressed in closed form modulo eigenvalue decomposition thanks to the functions called divided differences [Bhatia, 1997]. This main advantage explains why they are ubiquitous as indexing collections of families of metrics. Secondly, we recall the main facts about classes of kernel and mean kernel metrics introduced in [Hiai andPetz, 2009, Hiai andPetz, 2012].

Notations

We denote Sym(n) the vector space of real symmetric matrices of size n, Sym + (n) the manifold of SPD matrices, O(n) the orthogonal group, Diag + (n) the group of positive diagonal matrices.

On the manifold Sym + (n), we denote T Σ Sym + (n) the tangent space at Σ ∈ Sym + (n). Given a metric g I on the manifold Sym + (n) where I is any index characterizing the metric, we denote ∇ I its Levi-Civita connection, R I the Riemann curvature tensor, T I the torsion tensor, Π I the parallel transport. We omit the index when the context is clear.

Given a matrix M , we denote

M ij or [M ] ij the (i, j)-th coefficient of M . Given coefficients (M ij ) 1 i,j n ∈ R n 2 , we denote [M ij ] i,j the matrix with (i, j)-th entry M ij . Given (d 1 , ..., d n ) ∈ R n , we denote diag(d 1 , ..., d n ) the corresponding diagonal matrix.
We recall that exp :

Σ ∈ Sym(n) -→ +∞ k=0 1 k! Σ k ∈ Sym + (n) is a diffeomorphism whose inverse is the symmetric matrix logarithm denoted log : Sym + (n) -→ Sym(n).

Univariate maps

We call O(n)-equivariant map a map f : Sym

+ (n) -→ Sym(n) such that f (RΣR ) = R f (Σ)R for all Σ ∈ Sym + (n) and R ∈ O(n).
Among O(n)-equivariant maps, we focus on the class of univariate maps.

Definition 5.1 (Univariate maps) A univariate map is an O(n)-equivariant map f : Sym + (n) -→ Sym(n) such that there exists a map on positive real numbers also denoted f : (0, ∞) -→ R such that f (P DP ) = P Diag(f (d 1 ), ..., f (d n )) P for all P ∈ O(n) and D ∈ Diag + (n) with D = Diag(d 1 , ..., d n ).
Any f : (0, ∞) -→ R can be extended into a univariate map and if the former is of class C 1 (resp. C 2 , resp. a C 1 -diffeomorphism), then the latter is differentiable (resp. two times differentiable, resp. a diffeomorphism) [Bhatia, 1997, Thanwerdas and[START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]. We denote Univ the set of smooth univariate diffeomorphisms. In addition, the differential and the Hessian of a smooth univariate map can be expressed thanks to the first and second divided differences as follows.

Definition 5.2 (Divided differences) [Bhatia, 1997] 1. Let f ∈ C 1 (R, R). The first divided difference of f is the continuous symmetric map f [1] : R 2 -→ R defined for x, y ∈ R by:

f [1] (x, y) = ® f (x)-f (y) x-y if x = y f (x) if x = y
´.

(5.1)

2. Let f ∈ C 2 (R, R).
The second divided difference of f is the continuous symmetric map f [2] : R 3 -→ R defined for x, y, z ∈ R by:

f [2] (x, y, z) =            (f [1] (x, •)) [1] (y, z) = f [1] (x,z)-f [1] (x,y) z-y if y = z (f [1] (y, •)) [1] (z, x) = f [1] (y,x)-f [1] (y,z) x-z if z = x (f [1] (z, •)) [1] (x, y) = f [1] (z,y)-f [1] (z,x) y-x if x = y 1 2 f (x) if x = y = z            . (5.2) If f ∈ C 2 (R, R), then one can check that the differential of f [1] at (x, y) ∈ R 2 is: d (x,y) f [1] (h, k) = ® f (x)h-f (y)k x-y + f (x)-f (y) (x-y) 2 (h -k) if x = y f (x) 2 (h + k) if x = y
´, (5.3) so one can prove that df [1] is continuous and f [1] ∈ C 1 (R 2 , R). This also proves that ∂f [1] ∂x (x, x) = f (x) 2 and that f [2] is continuous. From now on, all univariate maps are assumed to be smooth.

Lemma 5.3 (Differential and Hessian of a univariate map) [Bhatia, 1997] The differential and the Hessian of a univariate map f are O(n)-equivariant: d P DP f (P XP ) = P d D f (X) P and H P DP f (P XP , P Y P ) = P H D f (X, Y ) P . Hence, they are determined by their values at diagonal matrices D ∈ Diag + (n), which are given by the following formulae:

[d D f (X)] ij = f [1] (d i , d j )X ij , (5.4) [H D f (X, X)] ij = 2 n k=1 f [2] (d i , d j , d k )X ik X jk .
(5.5)

Classes of O(n)-invariant metrics

The class of kernel metrics is a subclass of O(n)-invariant metrics on SPD matrices indexed by smooth bivariate symmetric maps φ : (0, ∞) 2 -→ (0, ∞) [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF]. The advantages of this class are the simple formulation of its elements, some important results on the metrics (completeness, cometric) and some important stability properties of the class. Hence, it is a good ambient class to define subfamilies of metrics. Therefore in this section, we recall the definition of kernel metrics, the refinement of mean kernel metrics and the results on completeness, cometric and stability.

Kernel metrics

Definition 5.4 (Kernel metric) A kernel metric [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF]] is an O(n)-invariant metric for which there is a smooth bivariate map φ

: (0, ∞) 2 -→ (0, ∞) such that g Σ (X, X) = g D (X , X ) = i,j 1 φ(d i ,d j ) X 2 ij , where X = P X P , Σ = P DP with P ∈ O(n) and D = Diag(d 1 , ..., d n ).
Important examples of kernel metrics are the Euclidean, the log-Euclidean, the affineinvariant, the Bures-Wasserstein and the Bogoliubov-Kubo-Mori metrics:

(Euclidean) g E Σ (X, X) = tr(X 2 ), (5.6) (Log-Euclidean) g LE Σ (X, X) = tr(d Σ log(X) 2 ), (5.7) (Affine-invariant) g A Σ (X, X) = tr((Σ -1 X) 2 ), (5.8) (Bures-Wasserstein) g BW Σ (X, X) = tr(ΣS Σ (X) 2 ), (5.9) (Bogoliubov-Kubo-Mori) g BKM Σ (X, X) = tr(d Σ log(X)X), (5.10)
where S Σ (X) denotes the solution of the Sylvester equation X = ΣS Σ (X) + S Σ (X)Σ. A review of the definitions, geometric properties and main references on these five metrics can be found in [START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF].

The refinement of mean kernel metrics

There is a refinement of kernel metrics where the bivariate function φ relies on a function called a symmetric homogeneous mean [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF]. These subclasses provide a nice necessary and sufficient condition for geodesic completeness.

Definition 5.5 (Mean kernel metrics) [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF] A mean kernel metric is a kernel metric characterized by a bivariate map φ of the form φ(x, y) = a m(x, y) θ where a > 0 is a positive coefficient, θ ∈ R is a homogeneity power and m : (0, ∞) 2 -→ (0, ∞) is a symmetric homogeneous mean, that is:

1. symmetric, i.e. m(x, y) = m(y, x) for all x, y > 0, 2. homogeneous, i.e. m(cx, cy) = c m(x, y) for all c, x, y > 0, 3. non-decreasing in both variables, 4. min(x, y) m(x, y) max(x, y) for all x, y > 0. It implies m(x, x) = x.

Theorem 5.6 (Completeness of mean kernel metrics) [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF] Mean kernel metrics are geodesically complete if and only if θ = 2.

Main results

The five kernel metrics cited above are mean kernel metrics. The mean functions are summarized in Table 5.1. Moreover, the class of kernel metrics is stable under pullback by univariate diffeomorphisms [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF] univariate diffeomorphisms, essentially because of the third condition of a symmetric homogeneous mean. Indeed, the mean has to be non-decreasing in both variables which is neither a differential nor a Riemannian property.

In addition, the class of kernel metrics is cometric stable [START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]. Indeed, the cometric is a metric on the cotangent bundle

T * Sym + (n) Sym + (n) × Sym(n) * .
Thanks to the Riesz theorem, the Frobenius inner product provides the identification Sym(n) * Sym(n) so the cometric can be considered as a metric. The cometric of the kernel metric characterized by φ is the kernel metric characterized by 1 φ .

Deformed metrics

Log-Euclidean metrics on SPD matrices are pullback metrics of Euclidean metrics on the vector space of symmetric matrices via the symmetric matrix logarithm log : Sym + (n) -→ Sym(n). This geometric construction of a metric on SPD matrices based on a diffeomorphism f is commonly used to define families of metrics on SPD matrices indexed by automorphisms of SPD matrices. Indeed, even if these metrics are isometric, they do not give the same results in data analyses. It is actually equivalent to compute with the metric g on the transformed dataset [f (Σ 1 ), ..., f (Σ N )] or to compute with the pullback metric f * g on the initial dataset [Σ 1 , ..., Σ N ].

In this section, we give examples of situations in the literature where such transformations are applied to the data (Section 5.3.1), then we unify them into our principle of deformed metrics and we give the fundamental Riemannian operations (distance, geodesics, curvature, parallel transport) of the deformed metrics (Section 5.3.2). In Section 5.3.3, we contribute the new family of deformed-Wasserstein metrics based on this principle which comprises the family of alpha-Procrustes metrics [Hà Quang, 2019]. We also give a sufficient condition under which alpha-Procrustes metrics are mean kernel metrics.

Use of a deformation in the literature

As mentioned before, the class of kernel metrics is stable by pullback under univariate diffeomorphisms [Hiai andPetz, 2009, Thanwerdas and[START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]. In particular, pullbacks of the Euclidean metric and the affine-invariant metric under power diffeomorphisms are detailed in the original paper on kernel metrics [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF]. They were later called power-Euclidean [START_REF] Dryden | Power Euclidean metrics for covariance matrices with application to diffusion tensor imaging[END_REF] and power-affine metrics, or more generally deformed-Euclidean and deformed-affine metrics for an arbitrary diffeomorphism [START_REF] Thanwerdas | Is affineinvariance well defined on SPD matrices? A principled continuum of metrics[END_REF]. Moreover, power-Euclidean metrics are mean kernel metrics for any power and power-affine metrics are mean kernel metrics if and only if the power belongs to [-2, 2] [Hiai and Petz, 2009]. Since power-Euclidean metrics interpolate between the log-Euclidean, the Wigner-Yanase/square-root and the Euclidean metrics, an optimization procedure was proposed on the parameter to choose the most appropriate metric on a dataset of covariance matrices for Diffusion Tensor Imaging (DTI) [START_REF] Dryden | Power Euclidean metrics for covariance matrices with application to diffusion tensor imaging[END_REF]. It is common in DTI to compute with precision matrices which are the inverses of covariance matrices, inv(Σ) = Σ -1 [START_REF] Lenglet | Inferring White Matter Geometry from Diffusion Tensor MRI: Application to Connectivity Mapping[END_REF], or with other transformations of the covariance matrices such as the adjugate function adj(Σ) = det(Σ)Σ -1 [START_REF] Fuster | Adjugate Diffusion Tensors for Geodesic Tractography in White Matter[END_REF]. More recently, the family of alpha-Procrustes metrics was introduced by pullback under power diffeomorphisms of the Bures-Wasserstein metric, as for power-Euclidean and power-affine metrics, and it was extended to the infinite dimension in the context of Reproducing Kernel Hilbert Spaces (RKHS) [Hà Quang, 2019].

In papers where the power diffeomorphisms are used to define power-Euclidean, poweraffine and alpha-Procrustes metrics [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF], Dryden et al., 2010, Hiai and Petz, 2012[START_REF] Thanwerdas | Is affineinvariance well defined on SPD matrices? A principled continuum of metrics[END_REF], Hà Quang, 2019], it is often noticed that the limit when the power tends to 0 is the log-Euclidean metric. This is actually a general fact. Indeed, from a Riemannian metric g, it is possible to construct a one-parameter family of metrics (g (p) ) p∈R * by taking the pullback by the power diffeomorphism pow p : Σ

∈ Sym + (n) -→ Σ p ∈ Sym + (n) for p = 0 and to scale it by 1 p 2 , that is g (p) Σ (X, X) = 1 p 2 g Σ p (d Σ pow p (X), d Σ pow p (X)
) for all Σ ∈ Sym + (n) and all X ∈ T Σ Sym + (n). Then when p tends to 0, g (p) tends to the log-Euclidean metric associated to the inner product g In , that is

g (p) Σ (X, X) -→ p→0 g In (d Σ log(X), d Σ log(X))
for all Σ ∈ Sym + (n) and all X ∈ T Σ Sym + (n).

Principle of deformed metrics

Principle 5.7 (Principle of deformed metrics) Let g be a Riemannian metric on Sym + (n) and f : Sym + (n) -→ Sym + (n) be a diffeomorphism. Then the f -deformed metric is defined as the pullback metric f * g. It is a Riemannian metric on Sym + (n) which is isometric to g and whose expression is:

(f * g) Σ (X, X) = g f (Σ) (d Σ f (X), d Σ f (X)).
(5.11)

All the Riemannian operations of a deformed metric are obtained by pulling back the formulae that are known for the initial metric, as shown in Table 5.2.

A fundamental stability property is that if g is O(n)-invariant and if f is O(n)-equivariant, then the deformed metric f * g is also O(n)-invariant. Moreover, as mentioned before, if g is a kernel metric and if f is univariate, then the deformed metric f * g is a kernel metric and the set {f * g, f ∈ Univ} forms a family of kernel metrics that is closed under pullback by univariate diffeomorphisms [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF]. The Riemannian operations that are known in closed form for g are also known in closed form for f * g.

Metric

g f Σ (X, X) = g f (Σ) (d Σ f (X), d Σ f (X)) Distance d f (Σ, Λ) = d(f (Σ), f (Λ)) Levi-Civita d Σ f (∇ f X Σ Y ) = ∇ d Σ f (X) (df (Y )) Curvature d Σ f (R f Σ (X, Y )Z) = R f (Σ) (d Σ f (X), d Σ f (Y ))d Σ f (Z) Geodesics f (γ f (Σ,X) (t)) = γ (f (Σ),d Σ f (X)) (t) Logarithm d Σ f (Log f Σ (Λ)) = Log f (Σ) (f (Λ)) Parallel transport d Λ f (Π f γ;Σ→Λ X) = Π f •γ;f (Σ)→f (Λ) (d Σ f (X))
Table 5.2: Riemannian operations of deformed metrics on SPD matrices.

The new family of deformed-Wasserstein metrics

Definition 5.8 (Deformed-Wasserstein metrics) A deformed-Wasserstein metric is the pullback metric by a univariate diffeomorphism of the Bures-Wasserstein metric (Formula (5.9)).

The family of deformed-Wasserstein metrics contains the family of alpha-Procrustes metrics since they are pullbacks of the Bures-Wasserstein metric by the power diffeomorphism pow 2α scaled by 1 4α 2 [Hà Quang, 2019]. In this work, we designate alpha-Procrustes metrics as power-Wasserstein metrics to be consistent with power-Euclidean and power-affine metrics and to parameterize the family by p ∈ R * , the correspondence being p = 2α. As argued earlier, we can say that the log-Euclidean metric belongs to deformed-Wasserstein metrics so we can designate it as power-Wasserstein with power p = 0.

Since the Bures-Wasserstein metric is a mean kernel metric, it is tempting to determine when a power-Wasserstein metric is a mean kernel metric, in analogy to the work done for the power-Euclidean and the power-affine metrics [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF]]. Here we give a sufficient condition under which a power-Wasserstein metric is a mean kernel metric. The proof is in Section 11.4.

Theorem 5.9 (Sufficient condition for power-Wasserstein to be mean kernel) The power-Wasserstein metric of parameter p 1 is a mean kernel metric. See the proof of Theorem 5.9 in Section 11. 4. This condition does not seem to be sufficient. Indeed, after numerical simulations, we conjecture that there exists p 0 ∈ (2.61, 2.611) such that the power-Wasserstein metric of parameter p is a mean kernel metric if and only if p ∈ (-∞, 1] ∪ [p 0 , +∞). Moreover, the proof actually tells that if p ∈ (1, 2], then it is not a mean kernel metric.

In this section, we gathered the deformations of Riemannian metrics or of SPD datasets under our principle of deformed metrics. Therefore from a metric we can define the family of power deformations of this metric, which tends to a log-Euclidean metric when the power tends to 0. Moreover, the family of univariate deformations of a kernel metric is a stable subfamily of kernel metrics and it is interesting to determine when these metrics are mean kernel metrics. It seems to be a quite difficult problem for general univariate deformations. On the example of power deformations of the Bures-Wasserstein metric, we gave the sufficient condition p 1. To the best of our knowledge, determining necessary and sufficient conditions for deformed-Euclidean, deformed-affine and deformed-Wasserstein metrics to be mean kernel metrics remains an open problem.

Balanced metrics

The affine-invariant and the Bogoliubov-Kubo-Mori metrics were shown to provide a duallyflat structure, that is a couple of flat affine connections which are dual with respect to the metric. This is a rich geometric structure which provides a so called canonical divergence, potentials and specific algorithms [START_REF] Amari | Methods of Information Geometry[END_REF], Banerjee et al., 2005, Nielsen and Nock, 2009]. A dually-flat manifold is a Hessian manifold where the potential is defined globally [START_REF] Shima | [END_REF]Yagi, 1997, Amari and[START_REF] Amari | [END_REF]. Inspired by the characterization of the duality based on parallel transport, we introduced a preliminary version of the principle of balanced bilinear forms in [Thanwerdas and Pennec, 2019a] which allows to define a bilinear form g 0 on SPD matrices from two flat Riemannian metrics g, g * by

g 0 Σ (X, Y ) = tr((Π Σ→In X)(Π * Σ→In Y ))
where Π, Π * denote the respective parallel transports. The term "balanced" was chosen because the bilinear form relies half on each of the two flat metrics. We showed that if the two flat metrics are power-Euclidean metrics, then the balanced bilinear form is symmetric and positive definite, i.e. a metric. In this section, we give a weaker condition under which the bilinear form is a metric.

To ease the comprehension of this section, we recall the main concepts of information geometry, especially dually-flat manifolds and related notions, in Section 5.4.1. In Section 5.4.2, we provide a new condition on the two flat metrics so that the balanced bilinear form is a metric: it is sufficient to assume that the flat metrics are univariately-deformed-Euclidean metrics.

Information geometry and dually-flat manifolds

Before introducing the specific concepts of information geometry, we recall the definition of an affine map between manifolds equipped with affine connections and the definition of a flat affine connection. We denote ∂ the canonical affine connection on a vector space. Definition 5.10 (Affine map) Let M, M be two manifolds with respective affine connections ∇, ∇ . We say that f : M -→ M is an affine map if for all vector fields X, Y on M, we have In the following, we call 1-flat (resp. 2-flat) a connection that is flat according to the sense 1 (resp. 2) of the previous definition. A 2-flat connection is clearly 1-flat. Conversely, a 1-flat connection is locally 2-flat, i.e. each point has a neighborhood U such that ∇ is 2-flat on U. A 1-flat connection is a priori not globally 2-flat because the manifold need not be an open set of R n (e.g. the circle S 1 ). The obstruction is topological.

∇ f * (X) f * (Y ) = f * (∇ X Y ).
In this work, the flat metrics we introduce on Sym + (n) (which is an open set of the vector space of symmetric matrices) are actually 2-flat.

Dual connections with respect to a metric

Definition 5.12 (Dual connections) [START_REF] Amari | Methods of Information Geometry[END_REF]] Let (M, g) be a Riemannian manifold, ∇ g the Levi-Civita connection of g and ∇, ∇ * be affine connections on M. We say that ∇ * is the dual connection of ∇ with respect to g if one of the following equivalent requirements is satisfied:

1. ∂ k g ij = g lj Γ l ki + g il (Γ * ) l kj for all i, j, k ∈ {1, ..., dim M} in any chart, where Γ k ij and (Γ * ) k ij are the Christoffel symbols of ∇ and ∇ * , 2. Z g(X, Y ) = g(∇ Z X, Y ) + g(X, ∇ * Z Y ) for all vector fields X, Y, Z on M, 3. g(X, Y ) = g(ΠX, Π * Y ) for all vector fields X, Y on M.
Hence given a metric g, ∇ uniquely determines ∇ * and (∇ * ) * = ∇ so that ∇+∇ * 2 is a metric connection. We call (g, ∇, ∇ * ) a dualistic structure. Note that if ∇ and ∇ * are torsion-free, then ∇+∇ * 2 = ∇ g . Definition 5.13 (Dually-flat manifold) [START_REF] Amari | Methods of Information Geometry[END_REF] We say that (M, g, ∇, ∇ * ) is a dually-flat manifold (or a Hessian manifold) when ∇ and ∇ * are dual with respect to the metric g and when ∇ and ∇ * are flat (in the sense 2 of Definition 5.11).

Divergence

Definition 5.14 (Divergence) [START_REF] Amari | Methods of Information Geometry[END_REF] A divergence is a distance-like smooth map D : M × M -→ R + such that:

1. (separation) D(x, y) = 0 if and only if x = y, 2. (non-degenerate) the symmetric positive semi-definite bilinear form

g D : z ∈ M -→ -∂ x | x=z ∂ y | y=z D is positive definite. It is called the induced Riemannian metric. We denote : T M -→ T * M and # = -1 : T * M -→ T M the musical isomorphisms associated to the metric g D , defined by (X)(Y ) = g(X, Y ).
We can also define the dual divergence D * : (x, y) -→ D(y, x) and the induced connection by

∇ D X Y : z ∈ M -→ (Z -→ ∂ 2 x | x=z ∂ y | y=z D(X, Y, Z)).
Lemma 5.15 (Dual connections induced by a divergence) [START_REF] Amari | Methods of Information Geometry[END_REF] Let D be a divergence on M. Then the connections ∇ := ∇ D and ∇ * := ∇ D * are dual with respect to the induced metric g D : a divergence induces a dualistic structure.

In general, there is not a canonical way to define a divergence from a dualistic structure, except if it is dually-flat.

Canonical divergence of a dually-flat manifold

Definition 5.16 (Canonical divergence) [START_REF] Amari | Methods of Information Geometry[END_REF]] Let (M, g, ∇, ∇ * ) be a dually-flat manifold where M is simply connected. Let u, v : M -→ R n be two smooth coordinate systems such that u is ∇-affine, v is ∇ * -affine and g( ∂ ∂u i , ∂ ∂v j ) = δ ij . The canonical divergence D is defined by D(x, y) = ψ(x) + ϕ(y) -u(x)|v(y) for all x, y ∈ M where •|• is the canonical inner product on R n and ψ, ϕ : M -→ R are smooth maps called potentials defined as follows:

1. dψ = i v i du i for all i ∈ {1, ..., n} or equivalently without coordinates

d x ψ(X) = v(x)|d x u(X) for all x ∈ M and X ∈ T x M, 2. dϕ = i u i dv i for all i ∈ {1, ..., n} or d x ϕ(X) = u(x)|d x v(X) , 3. ψ(x) + ϕ(x) = u(x)|v(x) for all x ∈ M.
The equation dψ = i v i du i has a solution by Poincaré's lemma because M is simply connected and the differential form

ω = i v i du i is closed. Indeed, g( ∂ ∂u i , ∂ ∂u j ) = ∂v k ∂u j g( ∂ ∂u i , ∂ ∂v k ) = ∂v i
∂u j and by symmetry of g, g( ∂ ∂u i , ∂ ∂u j ) = ∂v j ∂u i so ∂v i ∂u j = ∂v j ∂u i . So ψ is well defined up to an additive constant and ϕ as well. Finally,

d x (ψ + ϕ)(X) = v(x)|d x u(X) + u(x)|d x v(X) = d x ( u|v )(X) so there exists a constant c ∈ R such that ψ(x) + ϕ(x) = u(x)|v(x) + c for all x ∈ M.
We can impose c = 0 by choosing the constant in ϕ appropriately.

Principle of balanced bilinear forms

The principle of balanced bilinear forms [Thanwerdas and Pennec, 2019a] provides a bilinear form by combining the parallel transports of two flat metrics via the Frobenius inner product. We can give a more general definition of a balanced bilinear form by choosing any inner product on symmetric matrices, although we focus on the Frobenius inner product afterwards.

Principle 5.17 (Principle of balanced bilinear forms) We fix •|• an inner product on Sym(n). Let g + , g -be two flat Riemannian metrics on Sym + (n). We denote ∇ + , ∇ -their Levi-Civita connections and Π + , Π -their associated parallel transport maps that do not depend on the curve since the metrics are flat. Then the balanced bilinear form associated to g + and g -is defined by:

g 0 Σ (X, Y ) = Π + Σ→In X|Π - Σ→In Y .
(5.12)

Theorem 5.18 (Relation between balanced metric and dually-flat manifold) [Thanwerdas and Pennec, 2019a] Let g + , g -be two flat Riemannian metrics on Sym + (n). We denote ∇ + , ∇ -their Levi-Civita connections. If the balanced bilinear form g 0 is a metric, then (Sym + (n), g 0 , ∇ + , ∇ -) is a dually-flat manifold, which automatically comes with a canonical divergence D according to the previous section.

It would be nice to have a sufficient condition under which a balanced bilinear form is a metric. In [Thanwerdas and Pennec, 2019a], we proved that, with the Frobenius inner product, if g + and g -are power-Euclidean metrics with powers α and β, then g 0 is a metric. In the following theorem, we give a weaker sufficient condition which allows to define the new family of Mixed-Euclidean metrics. The proof is in Section 11.4.

Theorem 5.19 (Sufficient condition for a balanced bilinear form to be a metric) Let •|• = Frob be the Frobenius inner product. Let g + , g -be deformed-Euclidean metrics respectively associated to univariate diffeomorphisms u and v. Then the balanced bilinear form g 0 is a metric. See the proof of Theorem 5.19 in Section 11.4.

The new family of mixed-Euclidean metrics

Definition

Definition 5.20 (Mixed-Euclidean metric ME(u, v)) The (u, v)-Mixed-Euclidean metric is the balanced metric g 0 defined in Theorem 5.19. It is given by: (5.13) where X = P X P , Σ = P DP with

g ME(u,v) Σ (X, X) = 1 u (1)v (1) i,j u [1] (d i , d j )v [1] (d i , d j )X 2 ij ,
P ∈ O(n), D = diag(d 1 , ..., d n ).
Remark 5. 21 We notice that if we denote

φ u = u (1) u [1] and φ v = v (1) v [1]
the kernel maps associated to the u, v-deformed Euclidean metrics, the balanced metric is a kernel metric characterized by

φ u,v = √ φ u φ v .
Hence, the principle of balanced bilinear forms seems to appear as a principle of mean of metrics.

The family of Mixed-Euclidean metrics contains the family of Mixed-Power-Euclidean metrics [Thanwerdas and Pennec, 2019a] for u = F α and v = F β where F α = pow α if α = 0 and F 0 = log.

(Log-Euclidean) g

MPE(0,0) Σ (X, X) = tr(d Σ log(X) 2 ), (5.14) (Power-Euclidean) g MPE(α,α) Σ (X, X) = 1 α 2 tr(d Σ pow α (X) 2 ), (5.15) (Power-affine) g MPE(α,-α) Σ (X, X) = 1 α 2 tr((Σ -α d Σ pow α (X)) 2 ), (5.16) ("Power-BKM ) g MPE(α,0) Σ (X, X) = 1 α tr(d Σ pow α (X)d Σ log(X)), (5.17) (General MPE) g MPE(α,β) Σ (X, X) = 1 αβ tr(d Σ pow α (X)d Σ pow β (X)).
(5.18)

As mentioned in [Thanwerdas and Pennec, 2019a], this family interpolates between the log-Euclidean metric (0, 0), the power-Euclidean metrics (α, α), the power-affine metrics (α, -α) (including the affine-invariant metric (1, -1)) and the Bogoliubov-Kubo-Mori metric (1, 0).

Information geometry of Mixed-Euclidean metrics

As said in Theorem 5.18, balanced metrics come with a canonical divergence. As Mixed-Euclidean metrics are the balanced metrics of two deformed-Euclidean metrics u * g E and v * g E , it is straightforward that u : Sym + (n) -→ Sym(n) and v : Sym + (n) -→ Sym(n) provide flat coordinate systems for these respective metrics. The canonical divergence of this structure is known as the (u, v)-divergence in Information Geometry [Amari, 2016, Section 4.5.2].

The novelty here is the relation we establish between Mixed-Euclidean metrics and (u, v)divergences. In particular, the Mixed-Power-Euclidean metrics come with the so-called (α, β)divergences on SPD matrices [Amari, 2014]. This family contains the well known families of α-divergences and β-divergences [Amari, 2014, Formulae 69,70]. We state the correspondence between Mixed-Euclidean metrics and (u, v)-divergences in the following corollary of Theorem 5.18. We recall the formulae of (α, β)-divergences with the corresponding potentials and we illustrate the correspondence with two charts.

Corollary 5.22 (Mixed-Euclidean metrics and (u, v)-divergences) Let u, v be two univariate diffeomorphisms u, v : Sym

+ (n) -→ Sym + (n). Then the manifold (Sym + (n), g ME(u,v) , u * ∇ E , v * ∇ E )
is dually-flat and its canonical divergence is the (u, v)-divergence of Information Geometry [Amari, 2016]. In particular, the manifold (Sym + (n), g MPE(p,q) , ∇ PE(p) , ∇ PE(q) ) is dually-flat and its canonical divergence is the (α, β)-divergence [Amari, 2014, Formulae 51,54,56,66]. The (α, β)-divergences and the corresponding potentials (up to an additive constant, see Section 5.4.1.3) are:

(α = β = 0) D 0,0 (Σ|Σ ) = 1 2 log(Σ) -log(Σ ) 2 Frob , (5.19) (α = β = 0) D α,α (Σ|Σ ) = 1 2α 2 Σ α -Σ α 2 Frob , (5.20) (α = -β = 0) D α,-α (Σ|Σ ) = - 1 α 2 tr (I n + α log Σ) -α log Σ -Σ α Σ -α , (5.21) (α = β = 0) D α,0 (Σ|Σ ) = 1 α tr ïÅ Σ α log Σ - 1 α Σ α ã + 1 α Σ α -Σ α log Σ ò , (5.22) (α, β, α ± β = 0) D α,β (Σ|Σ ) = 1 αβ tr ï α α + β Σ α+β + β α + β Σ α+β -Σ α Σ β ò , (5.23) (α = β = 0) ψ 0,0 (Σ) = 1 2 tr(log(Σ) 2 ), (5.24) (α = β = 0) ψ α,α (Σ) = 1 2α 2 tr(Σ 2α ), (5.25) (α = -β = 0) ψ α,-α (Σ) = - 1 α tr(log Σ) = - 1 α log(det Σ) (5.26) (α = β = 0) ψ α,0 (Σ) = 1 α tr(Σ α log Σ - 1 α Σ α ), (5.27) (α, β, α ± β = 0) ψ α,β (Σ) = 1 β(α + β)
tr(Σ α+β ).

(5.28)

The (α, β)-divergences on SPD matrices can also be obtained by extending the (α, β)divergences on positive discrete measures [START_REF] Cichocki | Generalized alphabeta divergences and their application to robust nonnegative matrix factorization[END_REF]. Indeed, a positive discrete measure is a vector of positive numbers so the diagonal matrix of eigenvalues of an SPD matrix can be considered as a positive discrete measure. Then the (α, β)-potential on positive diagonal matrices is extended by O(n)-invariance, which defines the (α, β)-potential and the (α, β)-divergence on SPD matrices [Amari, 2014]. Conversely, the (α, β)-divergences on SPD matrices define divergences on positive discrete measures when restricted to positive diagonal matrices. So there is a one-to-one correspondence between (α, β)-divergences on SPD matrices [Amari, 2014] (or Mixed-Power-Euclidean metrics) and (α, β)-divergences on positive discrete measures [START_REF] Cichocki | Generalized alphabeta divergences and their application to robust nonnegative matrix factorization[END_REF]. This correspondence is given on Figure 5.1. The graph on the right is essentially borrowed from [START_REF] Cichocki | Generalized alphabeta divergences and their application to robust nonnegative matrix factorization[END_REF] with complements from [START_REF] Cichocki | Families of Alpha-Betaand Gamma-Divergences: Flexible and Robust Measures of Similarities[END_REF]. The (u, v)-divergences can be expressed via an integral formula [Amari, 2016, Formula (4.170)] following Definition 5.16. The formulae of the previous corollary can thus be computed either from that formula or directly.

Riemannian geometry of Mixed-Euclidean metrics

Another immediate consequence of the relation between balanced metrics and dually-flat manifolds is that the Levi-Civita connection of the Mixed-Euclidean metric MPE(u, v) is simply the arithmetic mean of the Levi-Civita connections of the deformed-Euclidean metrics u * g E and v * g E . Corollary 5.24 (Levi-Civita connection of Mixed-Euclidean metrics)

∇ ME(u,v) X Σ Y = ∂ X Σ Y + 1 2 ((d Σ u) -1 (H Σ u(X, Y )) + (d Σ v) -1 (H Σ v(X, Y ))) (5.29)
It is even possible to compute the curvature following the same ideas as for the BKM metric in [START_REF] Michor | The Curvature of the Bogoliubov-Kubo-Mori Scalar Product on Matrices[END_REF]. The proof is given in Section 11.4.

Theorem 5.25 (Curvature of Mixed-Euclidean metrics) Let u, v : Sym + (n) -→ Sym + (n) be two univariate diffeomorphisms. We define the univariate diffeomorphism w = v • u -1 so that u : (Sym + (n), g ME(u,v) ) -→ (Sym + (n), w (1) u (1)v (1) g ME(Id,w) ) is an isometry. For Σ = P DP ∈ Sym + (n), we denote X = P X P ∈ T Σ Sym + (n) and analogously for Y, Z, T ∈ T Σ Sym + (n), we denote u ij = u [1] (d i , d j ), u ijk = u [2] (d i , d j , d k ) and analogously for v, w. We denote

m ij = w [1] (u(d i ), u(d j )) = v ij u ij and m ijk = w [2] (u(d i ), u(d j ), u(d k )).
Then the curvature of the mixed-Euclidean metric g ME(u,v) is:

R ME(u,v) Σ (X, Y, Z, T ) = 1 u (1)v (1) i,j,k,l ρ ijkl (X ij Y jk Z kl T li -Y ij X jk Z kl T li (5.30) + X ij Z jk Y kl T li -Y ij Z jk X kl T li ),
where

ρ ijkl = m ijl m jlk 2m jl u ij u jk u kl u li = 1 2u jl v jl (u ij v ijl -v ij u ijl )(u jk v jkl -v jk u jkl ) is symmetric in i ↔ k, in j ↔ l and in u ↔ v.
In particular, at Σ = I n , the curvature is:

R ME(u,v) In (X, Y, Z, T ) = 1 4 ïÅ ln v u ã (1) ò 2 R A In (X, Y, Z, T ), (5.31)
where A stands for the affine-invariant metric (Formula 5.8). Therefore, the sectional curvature of the mixed-Euclidean metric at I n takes non-positive values. In particular, for mixedpower-Euclidean metrics MPE(α, β) with α 2 = β 2 (thus excluding log-Euclidean, power-Euclidean and power-affine metrics), since κ

MPE(α,β) λΣ (X, Y ) = λ -(α+β) × κ MPE(α,β) Σ
(X, Y ) for all λ > 0, the lower bound of the sectional curvature is -∞. See the proof of Theorem 5.25 in Section 11.4.

It seems difficult to determine theoretically whether the sectional curvature of mixed-Euclidean metrics (again, excluding MPE(α, β) with α 2 = β 2 ) can take positive values. On Figure 5.2, we show numerical results which make us think that this is the case. Indeed, we observe numerically that for all α, β ∈ {0.05 k| k ∈ {-40, ..., 40}} such that α 2 = β 2 , we have κ MPE(α,β) min < 0 and κ MPE(α,β) max > 0. These simulations also tend to show that, at a given point Σ, the negative values taken by the sectional curvature are much larger in absolute value than the positive values taken by the sectional curvature.

From Figure 5.2, it appears that power-Euclidean metrics (flat), power-affine metrics (Hadamard) and the log-Euclidean metric at the intersection play a special role among the family of Mixed-Power-Euclidean since all others apparently admit positive and negative sectional curvature.

In addition, for Mixed-Power-Euclidean metrics, we can also compute the geodesics, the logarithm map and the distance between commuting matrices. These formulae are proved in Section 11.4.

Theorem 5.26 (Riemannian operations of MPE metrics) Let α, β ∈ R such that α + β = 0, thus excluding log-Euclidean and power-affine metrics. Table 5.3 summarizes the formulae of the geodesics, the logarithm map and the distance in the particular case where Σ, Λ ∈ Sym + (n) and V ∈ T Σ Sym + (n) commute. They essentially reduce to the formulae of the α 0 -power-Euclidean metric with α 0 = α+β 2 . These formulae are generally not valid for noncommuting matrices. See the proof of Theorem 5.26 in Section 11. [START_REF]A quotient-affine logarithm of C ∈ Cor + (n) at I n is a tangent vector X ∈ Hol(n) of minimal length such that Exp QA In (X) = Cor(exp(X)) = C[END_REF].

MPE(α,β) D (X, Y ) = R D (X,Y,X,Y ) g D (X,X)g D (Y,Y )-g D (X,Y ) 2 for (α, β) ∈ [-2, 2]
Geodesics γ (Σ,V ) (t) = (Σ α 0 + t d Σ pow α 0 (V )) 1/α 0 Logarithm Log Σ (Λ) = (d Σ pow α 0 ) -1 (Λ α 0 -Σ α 0 ) Distance d(Σ, Λ) = 1 α 0 Λ α 0 -Σ α 0 Frob
Table 5.3: Riemannian operations of Mixed-Power-Euclidean metrics for commuting matrices.

It would be tempting to generalize the formulae of geodesics, logarithm and distance between commuting matrices to Mixed-Euclidean metrics. However, if we consider two diffeomorphisms u, v : (0, ∞) -→ (0, ∞), the map f = √ uv (which generalizes pow α 0 = √ pow α pow β ) is not a diffeomorphism of (0, ∞) in general. For example, take u(x) = x(x + 1) and v(x) = 1

x 2 . So the generalization is not straightforward.

Conclusion

Deforming a Riemannian metric is a general way of defining new metrics and new families of metrics on SPD matrices. In particular, using power diffeomorphisms defines one-parameter families which tend to the log-Euclidean metric when the power tends to 0. The class of kernel metrics is stable by univariate diffeomorphism whereas the class of mean kernel metrics is not. We showed that the alpha-Procrustes (or power-Wasserstein) metrics are mean kernel metrics when the power p = 2α 1.

We extended the principle of balanced bilinear forms and we gave a new sufficient condition under which the bilinear form is a metric. This allowed to define the new family of Mixed-Euclidean metrics which extends the two-parameter family of Mixed-Power-Euclidean metrics. Since balanced metrics define dually-flat manifolds which are characterized by a canonical divergence, Mixed-Euclidean metrics are in one-to-one correspondence with the (u, v)-divergences of information geometry. In particular, Mixed-Power-Euclidean metrics are in bijection with (α, β)-divergences. Finally, we computed the curvature of all Mixed-Euclidean metrics.

Some questions remain open. What are the conditions on the univariate diffeomorphisms u, v for the u-deformed Euclidean, affine, Wasserstein or the (u, v)-mixed-Euclidean metric to be a mean kernel metric? Are there more general conditions on two flat metrics for their balanced bilinear form to be a metric? What if we replace the Frobenius inner product by another one? Does the operation (φ, φ ) -→ √ φφ on flat kernel metrics generalize the principle of balanced metrics?

More generally, since the two-parameter family of (α, β)-Mixed-Power-Euclidean metrics interpolate between the Euclidean, the log-Euclidean, the affine-invariant and the Bogoliubov-Kubo-Mori metrics, does there exist a principled family with three or four parameters which additionally includes the Bures-Wasserstein metric? It is not difficult to build parametric families of metrics which interpolate between all of them whereas it is more difficult to find interpolations with an interesting geometry such as the dually-flat geometry.

Part IV

Correlation matrices of full rank:

the open elliptope

Introduction

Correlation matrices are used in many domains with time series data such as functional brain connectivity in functional MRI, electroencephalography (EEG) or magnetoencephalography (MEG) signals. Full-rank correlation matrices form a strict sub-manifold of the cone of Symmetric Positive Definite (SPD) matrices sometimes called the (open) elliptope [Tropp, 2018]. However, very few geometric tools were defined for intrinsic computations with correlation matrices. For example, [START_REF] Rebonato | The most general methodology to create a valid correlation matrix for risk management and option pricing purposes[END_REF] rely on a surjection from a product of n spheres of dimension n -1 onto the space of correlation matrices in order to sample valid correlation matrices for financial applications: a point in the former space can be represented by an n × n matrix A with normed rows and therefore encodes a correlation matrix AA . Since low-rank matrices have null measure, one gets a full-rank correlation matrix almost surely. More recently, the open elliptope was endowed with the Hilbert projective geometry [START_REF] Nielsen | Clustering in Hilbert's Projective Geometry: The Case Studies of the Probability Simplex and the Elliptope of Correlation Matrices[END_REF] which relies on its convexity.

Since there exist efficient tools on SPD matrices (affine-invariant/Fisher-Rao metric, log-Euclidean metric, etc.), correlation matrices are often treated as SPD matrices [START_REF] Varoquaux | Detection of brain functional-connectivity difference in post-stroke patients using group-level covariance modeling[END_REF]. Nevertheless, these extrinsic tools do not respect the geometry of correlation matrices. Moreover, most of these tools on SPD matrices are invariant under orthogonal transformations which is not compatible with correlation matrices. The elliptope is not even stable by the orthogonal action. Hence using the tools designed for SPD matrices may not be relevant for correlation matrices. One could restrict a Riemannian metric from the cone of SPD matrices to the open elliptope to define intrinsic tools by embedding. To the best of our knowledge, this hasn't been studied in depth. However, a key property of correlation matrices in applications with respect to covariance matrices is that the scale is renormalized independently for each axis. This physically corresponds to an invariance under a group action, namely the action of diagonal matrices on SPD matrices. The previously cited structures do not rely on this physical reality. This is why we are more interested in the recently introduced quotient-affine metric which corresponds to the quotient of the affine-invariant metric on SPD matrices by the action of diagonal matrices [David and[START_REF] David | [END_REF].

In this work, we investigate the geometry of this quotient-affine metric and we contribute additional tools to compute on this manifold. Based on the formalization of the quotient manifold with vertical and horizontal distributions, we compute in closed form some fundamental Riemannian operations of the quotient-affine metrics, notably the exponential map and the curvature. In Section 6.2, we recall how quotient-affine metrics are introduced, as well as the basics on quotient manifolds. In Section 6.3, we provide the following fundamental quotient and Riemannian operations of quotient-affine metrics: the vertical and horizontal projections, the metric, the exponential map, the Levi-Civita connection and the sectional curvature. This opens the way to many practical algorithms on the open elliptope for different applications. Considering SPD matrices as the Cartesian product of positive diagonal matrices and full-rank correlation matrices, it also allows to define new Riemannian metrics which preserve the quotient-affine geometry on correlation matrices. Thus in Section 6.4, we illustrate the quotient-affine metric in dimension 2 by coupling it with the diagonal power-Euclidean metrics g E(p) D (∆, ∆) = tr(D 2(p-1) ∆ 2 ) for p ∈ {-1, 0, 1, 2}, where D is positive diagonal and ∆ diagonal, and then by comparing it with the affine-invariant and the log-Euclidean metrics on SPD matrices.

Quotient-affine metrics

The quotient manifold of full-rank correlation matrices

The group of positive diagonal matrices Diag + (n) acts on the manifold of SPD matrices Sym + (n) via the congruence action (Σ,

D) ∈ Sym + (n) × Diag + (n) -→ DΣD ∈ Sym + (n).
The manifold of full-rank correlation matrices Cor + (n) can be seen as the quotient manifold Sym + (n)/Diag + (n) via the invariant submersion Cor which computes the correlation matrix from a covariance matrix, Cor :

Σ ∈ Sym + (n) -→ Diag(Σ) -1/2 Σ Diag(Σ) -1/2 ∈ Cor + (n),
where Diag(Σ) = diag(Σ 11 , ..., Σ nn ). Hence, any Riemannian metric G on Sym + (n) which is invariant under Diag + (n) induces a quotient metric g on Cor + (n). The steps to define it are the following.

1. Vertical distribution. V Σ = ker d Σ Cor for all Σ ∈ Sym + (n).

Horizontal distribution. H

Σ := V ⊥ Σ for all Σ ∈ Sym + (n)
, where the orthogonality ⊥ refers to the inner product G Σ .

3. Horizontal lift. The linear map d Σ Cor restricted to the horizontal space H Σ is a linear isomorphism onto the tangent space of full-rank correlation matrices (d Σ Cor) |H Σ :

H Σ ∼ -→ T Cor(Σ) Cor + (n).
The horizontal lift # is its inverse:

# : X ∈ T Cor(Σ) Cor + (n) ∼ -→ X # ∈ H Σ . (6.1)
4. Quotient metric. It is defined by pullback through the horizontal lift:

∀C ∈ Cor + (n), ∀X ∈ T C Cor + (n), g C (X, X) = G Σ (X # , X # ), (6.2)
where Σ ∈ Cor -1 (C) and the definition does not depend on the chosen Σ.

So the only missing ingredient is a Riemannian metric on SPD matrices which is invariant under the congruence action of positive diagonal matrices. In [David and[START_REF] David | [END_REF], the authors chose to use the famous affine-invariant/Fisher-Rao metric.

The affine-invariant metrics and the quotient-affine metrics

The affine-invariant metric is the Riemannian metric defined on SPD matrices by [Siegel, 1943, Skovgaard, 1984, Amari and Nagaoka, 2000]. It is invariant under the congruence action of the whole real general linear group GL(n) which contains Diag + (n) as a subgroup. It provides a Riemannian symmetric structure to the manifold of SPD matrices, hence it is geodesically complete and the geodesics are given by the group action of one-parameter subgroups. We recall the exponential map, the Levi-Civita connection and the sectional curvature below for all Σ ∈ Sym + (n) and vector fields V, W ∈ Γ(T Sym + (n)) where we also denote V ≡ V Σ and

G Σ (V, V ) = tr(Σ -1 V Σ -1 V ) for all Σ ∈ Sym + (n) and V ∈ T Σ Sym + (n)
W ≡ W Σ ∈ T Σ Sym + (n): Exp G Σ (V ) = Σ 1/2 exp(Σ -1/2 V Σ -1/2 )Σ 1/2 , (6.3) (∇ G V W ) |Σ = d Σ W (V ) - 1 2 (V Σ -1 W + W Σ -1 V ), (6.4) κ G Σ (V, W ) = 1 4 tr((Σ -1 V Σ -1 W -Σ -1 W Σ -1 V ) 2 ) G(V, V )G(W, W ) -G(V, W ) 2 0. (6.5)
The metrics that are invariant under the congruence action of the general linear group GL(n) actually form a two-parameter family of metrics indexed by α > 0 and

β > -α/n [Pen- nec, 2009]: G α,β Σ (V, V ) = α tr(Σ -1 V Σ -1 V ) + β tr(Σ -1 V ) 2 .
We call them all affine-invariant metrics. In particular, these metrics are invariant under the congruence action of diagonal matrices so they are good candidates to define Riemannian metrics on full-rank correlation matrices by quotient. In [David and[START_REF] David | [END_REF], the authors rely on the "classical" affine-invariant metric (α = 1, β = 0). We generalize their definition below. Definition 6.1 (Quotient-affine metrics on full-rank correlation matrices) The quotientaffine metric of parameters α > 0 and β > -α/n is the quotient metric g α,β on Cor + (n) induced by the affine-invariant metric G α,β via the submersion Cor :

Σ ∈ Sym + (n) -→ Diag(Σ) -1/2 Σ Diag(Σ) -1/2 ∈ Cor + (n).

Fundamental Riemannian operations

In this section, we detail the quotient geometry of quotient-affine metrics g α,β . We give the vertical and horizontal distributions and projections in Section 6.3.1. We contribute the formulae of the metric itself in Section 6.3.2, the exponential map in Section 6.3.3, and finally the Levi-Civita connection and the sectional curvature in Section 6.3.4. To the best of our knowledge, all these formulae are new. They are proved in Section 11.5.

Vertical and horizontal distributions and projections

• Let • be the Hadamard/Schur product on matrices defined by [X

• Y ] ij = X ij Y ij . • Let A : Σ ∈ Sym + (n) -→ A(Σ) = Σ • Σ -1 ∈ Sym + (n)
. This smooth map is invariant under the action of positive diagonal matrices. The Schur product theorem ensures that A(Σ)

∈ Sym + (n). A fortiori, I n + A(Σ) ∈ Sym + (n). • Let ψ : µ ∈ R n -→ (µ1 + 1µ ) ∈ Sym(n)
. This is an injective linear map. This function was met in Chapter 3 in the S(n)-irreducible decomposition of Sym(n).

• Let S Σ (V ) the unique solution of the Sylvester equation ΣS

Σ (V ) + S Σ (V )Σ = V for Σ ∈ Sym + (n) and V ∈ Sym(n).
• Let Hol(n) be the vector space of symmetric matrices with vanishing diagonal (symmetric hollow matrices). Each tangent space of the manifold of full-rank correlation matrices can be seen as a copy of this vector space.

Theorem 6.2 (Vertical and horizontal distributions and projections) The vertical distribution is given by V Σ = Σ•ψ(R n ) and the horizontal distribution is given by H Σ = S Σ -1 (Hol(n)).

The vertical projection is:

ver : V ∈ T Σ Sym + (n) -→ Σ • ψ((I n + A(Σ)) -1 Diag(Σ -1 V )1) ∈ V Σ . (6.6)
Then, the horizontal projection is simply hor(V ) = V -ver(V ).

See the proof of Theorem 6.2 in Section 11.5.

Horizontal lift and metric

Theorem 6.3

(Horizontal lift) Let Σ ∈ Sym + (n) and C = Cor(Σ) ∈ Cor + (n). The horizon- tal lift at Σ of X ∈ T C Cor + (n) is X # = hor(∆ Σ X∆ Σ ) with ∆ Σ = Diag(Σ) 1/2 . In particular, the horizontal lift at C ∈ Sym + (n) is X # = hor(X).
See the proof of Theorem 6.3 in Section 11.5.

Theorem 6.4 (Expression of quotient-affine metrics) For all C ∈ Cor + (n) and

X ∈ T C Cor + (n), g α,β C (X, X) = α g QA C (X, X) (independent from β) where: g QA C (X, X) = tr((C -1 X) 2 ) -21 Diag(C -1 X)(I n + A(C)) -1 Diag(C -1 X)1. (6.7)
See the proof of Theorem 6.4 in Section 11.5.

Geodesics

The geodesics of a quotient metric are the projections of the horizontal geodesics of the original metric. This allows us to obtain the exponential map of the quotient-affine metrics.

Theorem 6.5 (Geodesics of quotient-affine metrics) The geodesic from

C ∈ Cor + (n) with initial tangent vector X ∈ T C Cor + (n) is: ∀t ∈ R, γ QA (C,X) (t) = Exp QA C (tX) = Cor(C 1/2 exp(t C -1/2 hor(X)C -1/2 )C 1/2 ). (6.8)
In particular, the quotient-affine metric is geodesically complete.

See the proof of Theorem 6.5 in Section 11.5.

The Riemannian logarithm between C 1 and

C 2 ∈ Cor + (n) is much more complicated to compute. It amounts to find Σ ∈ Sym + (n) in the fiber above C 2 such that Log G C 1 (Σ) is horizontal. Then we have Log QA C 1 (C 2 ) = d C 1 Cor(Log G C 1 (Σ))
. This means finding Σ that minimizes the affine-invariant distance in the fiber:

D = arg min D∈Diag + (n) d(C 1 , DC 2 D),
from which we get Σ = DC 2 D. This is the method used in [David and[START_REF] David | [END_REF].

During the conference GSI 2021, Jonas Lueg had the idea to prove the existence of a minimizer by showing the coercivity of the smooth map f :

D ∈ Diag + (n) -→ tr(log(ΣDΛDΣ) 2 ) for all Σ, Λ ∈ Sym + (n), i.e.
show that f (D) -→ +∞ when one of the d i 's tends to 0 or to +∞. Thus we proved the following theorem. Note that the uniqueness remains an open problem.

Theorem 6.6 (Existence of a logarithm) For all C 1 , C 2 ∈ Cor + (n), there exists X ∈ T C Cor + (n) such that Exp QA C 1 (X) = C 2 . See the proof of Theorem 6.6 in Section 11.5.

Levi-Civita connection and sectional curvature

In this section, we give the Levi-Civita connection and the curvature. The computations are based on the fundamental equations of submersions [O'Neill, 1966]. We denote sym(M ) = 1 2 (M + M ) the symmetric part of a matrix. Theorem 6.7 (Levi-Civita connection and sectional curvature of quotient-affine metrics) The Levi-Civita connection of quotient-affine metrics is:

(∇ QA X Y ) |C = d C Y (X) + sym[Diag(X # )Y # + Diag(Y # )X # + Diag(X # C -1 Y # )C -X # C -1 Y # - 1 2 Diag(X # )CDiag(Y # ) - 3 2 Diag(X # )Diag(Y # )C]. (6.9)
The curvature of quotient-affine metrics is:

κ QA C (X, Y ) = κ G C (X # , Y # ) + 3 4 G C (ver[X # , Y # ], ver[X # , Y # ]) g C (X, X)g C (Y, Y ) -g C (X, Y ) 2 (6.10) = 2 tr((C -1 X # C -1 Y # -C -1 Y # C -1 X # ) 2 ) + 3 1 D(I n + A(C)) -1 D1 8(g C (X, X)g C (Y, Y ) -g C (X, Y ) 2 ) , (6.11) where [V, W ] = dW (V )-dV (W ) is the Lie bracket on Sym + (n) and D = D(X, Y )-D(Y, X) with D(X, Y ) = Diag(C -1 Diag(X # )Y # -C -1 Y # C -1 Diag(X # )C).
There is a slight abuse of notation because ver[X # , Y # ] induces that X # and Y # are vector fields. Indeed here, they are horizontal vector fields extending the horizontal lifts at C. See the proof of Theorem 6.7 in Section 11.5.

The first term of the sectional curvature is negative, the second one is positive so we don't know in general the sign of the curvature of quotient-affine metrics.

The quotient-affine metrics not only provide a Riemannian framework on correlation matrices but also provide correlation-compatible statistical tools on SPD matrices if we consider that the space of SPD matrices is the Cartesian product of positive diagonal matrices and full-rank correlation matrices. We give a taste of such a construction in the next section.

Illustration in dimension 2

In dimension 2, any correlation matrix

C ∈ Cor + (2) writes C = C(ρ) := Å 1 ρ ρ 1
ã where ρ ∈ (-1, 1) is the correlation coefficient between the two variables. In the following theorem, we give explicitly the logarithm, the distance and the interpolating geodesics of the quotientaffine metric.

Theorem 6.8 (Quotient-affine metrics in dimension 2

) Let C 1 = C(ρ 1 ), C 2 = C(ρ 2 ) ∈ Cor + (n) with ρ 1 , ρ 2 ∈ (-1, 1). We denote f : ρ ∈ (-1, 1) → 1+ρ 1-ρ ∈ (0, ∞) which is a smooth increasing map. We denote λ = λ(ρ 1 , ρ 2 ) = 1 2 log f (ρ 2 ) f (ρ 1 )
which has the same sign as ρ 2 -ρ 1 . Then the quotient-affine operations are:

1. (Logarithm) Log QA C 1 (C 2 ) = λ Å 0 1 -ρ 2 1 1 -ρ 2 1 0 ã , 2. (Distance) d QA (C 1 , C 2 ) = √ 2|λ|, 3. (Geodesics) γ QA (C 1 ,C 2 ) (t) = C(ρ QA (t)) where ρ QA (t) = ρ 1 cosh(λt)+sinh(λt) ρ 1 sinh(λt)+cosh(λt) ∈ (-1, 1) is mono- tonic (increasing if and only if ρ 2 -ρ 1 > 0). Let Σ 1 , Σ 2 ∈ Sym + (2)
and C 1 , C 2 their respective correlation matrices. We denote γ AI , γ LE the geodesics between Σ 1 and Σ 2 for the affine-invariant and the log-Euclidean metrics respectively. We define ρ AI , ρ LE such that the correlation matrices of γ AI (t), γ LE (t) are C(ρ AI (t)), C(ρ LE (t)). On Figure 6.2, we compare the geodesics of these metrics. The determinant is the area of the ellipsis and the trace is the sum of the lengths of the axes. Thus the product metrics of the form power-Euclidean on the diagonal part and quotient-affine on the correlation part can be seen as performing a correlation-monotonic trade-off between the trace-monotonicity and the determinant-monotonicity.

Conclusion

We investigated in this chapter the very nice idea of quotienting the affine-invariant metrics on SPD matrices by the action of the positive diagonal group to obtain the principled quotientaffine metrics on full-rank correlation matrices. The quotient-affine metric with α = 1 and β = 0 was first proposed in Paul David's thesis [David, 2019] and in the subsequent journal paper [START_REF] David | A Riemannian structure for correlation matrices[END_REF]. We contribute here exact formulae for the main Riemannian operations, including the exponential map, the connection and the sectional curvature. The exponential map is particularly interesting to rigorously project tangent computations to

Time t t=0 t=1

Eucl.

E(2)xQA E(1)xQA LExQA E(-1)xQA Aff-inv.
Log-Eucl. the space of full-rank correlation matrices. This opens the way to the implementation of a number of generic algorithms on Riemannian manifolds. However, we could not find a closed form expression for the logarithm which remains to be computed through an optimization procedure. Thus, computing distances with these metrics remains computationally expensive. In order to obtain more efficient methods, this leads us to look for other principled Riemannian metrics on correlation matrices for which the logarithm could be expressed in closed form.

Introduction

The (open) elliptope is the set of full-rank correlation matrices, it is open in the affine space of symmetric matrices with unit diagonal. Its geometry has been much less studied than the one of the cone of Symmetric Positive Definite (SPD) matrices. Nevertheless, several applications could benefit from well suited geometries on this manifold: graphical networks, brain connectomes [START_REF] Varoquaux | Detection of brain functional-connectivity difference in post-stroke patients using group-level covariance modeling[END_REF], Dodero et al., 2015], finance [Rebonato andJaeckel, 2001, Marti et al., 2021] or phylogenetic trees [START_REF] Garba | Information geometry for phylogenetic trees[END_REF]. A few operations have been proposed such as sampling by projecting samples on spheres [Rebonato andJaeckel, 2001, Kercheval, 2008], computing distances via the Hilbert geometry of convex sets [START_REF] Nielsen | Clustering in Hilbert's Projective Geometry: The Case Studies of the Probability Simplex and the Elliptope of Correlation Matrices[END_REF] or via a recent parametrization by R n(n-1)/2 [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF].

The open elliptope was also recently characterized as the quotient manifold of SPD matrices by the smooth, proper and free congruence action of positive diagonal matrices [David and[START_REF] David | [END_REF]. Indeed, a correlation matrix is obtained from the covariance matrix by dividing by the standard deviation of each variable. This is equivalent to multiply the covariance matrix on left and right by the diagonal matrix of the inverse square roots of the variances. Hence all covariance matrices that are congruent up to a diagonal matrix represent the same correlation matrix. This structure allowed to quotient the well known affine-invariant metrics on SPD matrices to the so called quotient-affine metrics on full-rank correlation matrices [START_REF] David | A Riemannian structure for correlation matrices[END_REF], David, 2019, Thanwerdas and Pennec, 2021]. They only differ by a scaling factor so they are often referred as the quotient-affine metric. It offers promising perspectives since it is geodesically complete with closed form expressions for the exponential map, the Levi-Civita connection and the sectional curvature. However, it was difficult to determine the sign and potential bounds of the curvature of the quotient-affine metric. In this chapter, we show that the sectional curvature can take both positive and negative values, that it is bounded from below and unbounded from above. Therefore, the elliptope of full-rank correlation matrices endowed with the quotient-affine metric is neither a Hadamard space nor a CAT(k) space for any k ∈ R.

This calls for new Riemannian metrics on the elliptope. Indeed, we could expect to find suitable metrics with non-positive or even null curvature since the elliptope is an open set of an affine space. The structure of the space is an important element of modeling because simple structures often bring good theoretical properties and better computability of the geometric operations. For example, in a Hadamard space, the Fréchet mean is unique. Recall that in a metric space (M, d), a Fréchet mean of points x 1 , ..., x k ∈ M is a point x ∈ M which minimizes the function x ∈ M -→ k i=1 d(x, x i ) 2 0. This is the case of the affine-invariant metric on SPD matrices [Skovgaard, 1984] or the Fisher metric of beta and Dirichlet distributions [Le Brigant et al., 2021]. Moreover, if one has a Euclideanization of the space, i.e. a smooth diffeomorphism to a Euclidean space, then all operations become trivial. This is the case of the log-Euclidean [START_REF] Arsigny | Log-Euclidean metrics for fast and simple calculus on diffusion tensors[END_REF] or the log-Cholesky [Lin, 2019] metrics on SPD matrices.

Another strong element of modeling is the invariance of the geometry under a given group action. Let us give some examples. First, on SPD matrices, one could require the invariance of the geometry under all affine transformations of the feature vector. Indeed in EEG, if we assume that there is an affine transformation from one brain to another at first order, then electromagnetic fields are transformed similarly since they satisfy linear equations. The affine-invariant metric significantly improved the results of classification in BCI [START_REF] Barachant | Classification of covariance matrices using a Riemannian-based kernel for BCI applications[END_REF]. Second, one could want the analysis to be invariant from the scale of each variable. As explained previously, this corresponds to the invariance under the congruence action of positive diagonal matrices and it suggests to focus more on correlation matrices than on covariance matrices. Third, all the operations mentioned above on correlation matrices are invariant under permutations. It means that the statistical analyses are invariant under any joint permutation of the rows and columns of the correlation matrices in the dataset. It is an advantage if the way the variables (or channels) are ordered is arbitrary. It is a drawback if the order is chosen for a certain reason. For instance, in the auto-correlation matrix of a signal, the variables represent different times, which are not exchangeable. One could want a structure on correlation matrices that respects the time structure instead of being invariant under permutations. In this chapter, we focus on non-permutation-invariant geometric structures.

Moreover, an invariance may also lead to simple structures with good properties. For example, a Riemannian homogeneous space is geodesically complete, which is well suited for interpolation and extrapolation. The geodesics of naturally reductive homogeneous spaces are the orbits of the one-parameter subgroups so they are known in closed form. In a Riemannian symmetric space, the parallel transport is obtained in closed form by composition of two symmetries. So it is another good reason to study Lie group actions on our space.

Overview of the results

In this chapter, we define new non-permutation-invariant geometric structures on full-rank correlation matrices following two directions. Firstly, we define a non-permutation-invariant generalization of quotient-affine metrics. Our approach consists in studying the congruence action of several matrix Lie groups on SPD matrices. Our main result is a characterization of affine-invariant metrics, i.e. metrics that are invariant under the congruence action of the general linear group GL(n), by the joint invariance of a pair of subgroups. These are the group of permutation matrices S(n) and the group of lower triangular matrices with positive diagonal LT + (n). In other words, the affine-invariant metrics are the unique (S(n)×LT + (n))invariant metrics on SPD matrices. Therefore, the family of LT + (n)-invariant metrics appears as a natural non-permutation-invariant generalization of affine-invariant metrics. Moreover, we show that such metrics are exactly the pullback metrics of left-invariant metrics on the Lie group LT + (n) by the Cholesky map, so we call them Lie-Cholesky metrics. In particular, there exists a Lie group structure on SPD matrices (namely, the product of the Cholesky factors) such that affine-invariant metrics are left-invariant metrics on that Lie group. Finally, since the Lie group LT + (n) contains positive diagonal matrices, Lie-Cholesky metrics descend to quotient metrics on full-rank correlation matrices by the same procedure as affine-invariant metrics. We show that several Riemannian operations are numerically computable such as the Riemannian metric, the exponential map, the logarithm map or the Riemannian distance. However, despite the nice theoretical results and the computability of the main geometric operations, quotient-Lie-Cholesky metrics don't have obvious nice geometric properties. For example, it is not clear whether the Riemannian logarithm and the Fréchet mean are unique or not.

Hence, secondly, to solve the drawbacks of quotient-affine and quotient-Lie-Cholesky metrics, we propose a series of new geometries on full-rank correlation matrices for which the mean of finite samples is unique. They are built in a different way than the Lie-Cholesky metrics, the only common point being the use of the Cholesky map to convey the structures from triangular matrices to correlation matrices. Hence they are not permutation-invariant either for n 3. We define the poly-hyperbolic-Cholesky metrics as the pullbacks of weighted products of hyperbolic spaces H 1 × • • • × H n-1 . The metrics of this family provide a structure of Riemannian symmetric space with non-positive bounded curvature, thus Hadamard. All operations are known in closed form. We define the Euclidean-Cholesky and the log-Euclidean-Cholesky metrics as the pullbacks by two maps derived from the Cholesky map of Euclidean structures on the vector space LT 0 (n) of strictly lower triangular matrices. The metrics in these two families are flat and geodesically complete so all Riemannian operations are known in closed form, the former being less costly than the latter. Finally, we define a Lie group structure as the pullback of the Lie group (for matrix multiplication) LT 1 (n) of lower triangular matrices with unit diagonal. Here, we consider the geometric structure induced by the canonical Cartan-Schouten connection rather than by a Riemannian metric. Indeed, the (group) exponential map allows to define a group mean, which is unique here because the Lie algebra of the Lie group is nilpotent [START_REF] Buser | Gromov's almost flat manifolds[END_REF]. Finally, in dimension 2, we show that on the one hand, the quotient-affine and the poly-hyperbolic-Cholesky metrics coincide, and on the other hand, the Euclidean-Cholesky and the log-Euclidean-Cholesky metrics coincide and their geodesics coincide with the group geodesics. In particular, these geodesics provide a new interpolation of the correlation coefficient, different than the one proposed in [START_REF] Thanwerdas | Geodesics and Curvature of the Quotient-Affine Metrics on Full-Rank Correlation Matrices[END_REF].

In the remainder of this section, we introduce the matrix notations, the notions of covariance and correlation matrices and the Cholesky map. In Section 7.2, we recall the definition of the quotient-affine metrics and we prove that its sectional curvature takes both negative and positive values and is unbounded from above. In Section 7.3, we give a characterization of affine-invariant metrics in function of the congruence action of other groups. This allows us to introduce Lie-Cholesky metrics on SPD matrices and quotient-Lie-Cholesky metrics on fullrank correlation matrices. In Section 7.4, we introduce four new non-permutation-invariant structures on full-rank correlation matrices for which the mean is unique. We conclude in Section 7.5.

Concepts and notations

Matrix notations

Our main matrix space notations are given in Table 7.1. In this chapter, we also use the following linear maps:

• (A, B) ∈ Mat(n)×Mat(n) -→ A•B = [A ij B ij ] 1 i,j n ∈ Mat(n) is the Hadamard/Schur product of matrices, • Diag : M ∈ Mat(n) -→ [δ ij M ij ] 1 i,j n ∈ Diag(n) selects the diagonal terms, • Off = Id Mat(n) -Diag : Mat(n) -→ ker Diag selects the off-diagonal terms, • Low : M ∈ Mat(n) -→ [δ i j M ij ] 1 i,j n ∈ LT(n)
selects the lower triangular terms, including the diagonal terms,

• Low 0 = Low -Diag : Mat(n) -→ LT 0 (n) selects the strictly lower triangular terms, excluding the diagonal terms,

• Low S = Low 0 + 1 2 Diag : Sym(n) -→ LT(n) selects the strictly lower triangular terms and half of the diagonal terms, so that when a symmetric matrix writes

M = L + L ∈ Sym(n) with L ∈ LT(n), then L = Low S (M ),
• Sum : M ∈ Mat(n) -→ i,j M ij = 1 M 1 sums the terms of the matrix, where 1 = (1, ..., 1) ∈ R n . The possibly uncommon notations in Table 7.1 are:

• sum : x ∈ R n -→ i x i = 1 x
• Sym + (n) = {Σ ∈ Sym(n)|Σ > 0}
where > is the Loewner order,

• Cor + (n) = {Σ ∈ Sym + (n)|Diag(Σ) = I n }, • Hol(n) = {M ∈ Sym(n)|Diag(M ) = 0}
where "hollow" means vanishing diagonal,

• LT(n) = {Low(M )|M ∈ Mat(n)}, • LT + (n) = {L ∈ LT(n)|Diag(L) ∈ Diag + (n)}, • LT 0 (n) = {L ∈ LT(n)|Diag(L) = 0}, • LT 1 (n) = {L ∈ LT(n)|Diag(L) = I n } = I n + LT 0 (n).
We denote S(n) the permutation group of order n.

Covariance and correlation matrices

Given an invertible covariance matrix Σ = (Cov(X i , X j )) 1 i,j n ∈ Sym + (n) of a random vector X, the corresponding correlation matrix is defined by C = Cor(Σ) = (Cor(X i , X j )) 1 i,j n where:

Cov(X i , X j ) = E(X i X j ) -E(X i )E(X j ), (7.1) Cor(X i , X j ) = Cov(X i , X j ) Cov(X i , X i ) Cov(X j , X j ) = Σ ij √ Σ ii Σ jj (7.2) = [Diag(Σ) -1/2 Σ Diag(Σ) -1/2 ] ij , (7.3) with Diag(Σ) = diag(Σ 11 , ..., Σ nn ) ∈ Diag + (n). Moreover, if D ∈ Diag + (n)
, then the correlation matrix associated to DΣD is again Cor(Σ). It is known as the invariance of the correlation matrix under the scaling of each component of the random vector. In other words, the surjective map Cor :

Σ ∈ Sym + (n) -→ Cor(Σ) ∈ Cor + (n) is invariant under the group action (D, Σ) ∈ Diag + (n) × Sym + (n) -→ DΣD ∈ Sym + (n)
and the orbit space Sym + (n)/Diag + (n) can be identified with Cor + (n). Note that, via this identification, the induced topology Cor + (n) → Sym(n) coincides with the quotient topology of Sym + (n)/Diag + (n). The action being proper [START_REF] David | A Riemannian structure for correlation matrices[END_REF], the quotient manifold theorem [Lee, 2012] states that the orbit space Sym + (n)/Diag + (n) has a unique smooth manifold structure such that the canonical surjection Sym + (n) -→ Sym + (n)/Diag + (n) is a smooth submersion. Since Cor : Sym + (n) -→ Cor + (n) is a smooth submersion and since Cor + (n) is Sym + (n)/Diag + (n) as a topological space, this smooth structure coincides with the induced smooth structure of Cor + (n).

The Cholesky map

A Cholesky decomposition of a symmetric positive semi-definite matrix Σ is a factorization of the form Σ = LL where L ∈ LT(n) is a lower triangular matrix. If Σ is positive definite, then there exists a unique triangular matrix with positive diagonal L ∈ LT + (n) such that Σ = LL . This allows to define the Cholesky bijective map:

Chol : Σ ∈ Sym + (n) -→ L ∈ LT + (n), (7.4)
whose inverse is the smooth map φ :

L ∈ LT + (n) -→ LL ∈ Sym + (n).
Moreover, the Cholesky map is smooth. Indeed, it is the product of two smooth maps Chol(Σ) = L(Σ) D(Σ) where L ≡ L(Σ) ∈ LT 1 (n) and D ≡ D(Σ) ∈ Diag + (n) are recursively defined for all (i, j) ∈ {1, ..., n} 2 with i > j by:

D ii = Σ ii - i-1 j=1 L 2 ij D jj > 0, (7.5
)

L ij = 1 D jj Ç Σ ij - j-1 k=1 L ik L jk D kk å , (7.6)
the order of the computations being

{D 11 } → • • • → {L i1 → L i2 → • • • → L i,i-1 → D ii } → • • • → {L n1 → • • • → L n,n-1 → D nn }.
We denote:

L = Chol(Cor + (n)) =      L = Ö L 1 . . . L n è ∈ LT + (n)|∀i ∈ {1, ..., n}, L i 2 = L i L i = 1     
, which is diffeomorphic to Cor + (n) via the Cholesky map. We also define a Cholesky-based diffeomorphism between Cor + (n) and LT 1 (n):

Θ : C ∈ Cor + (n) -→ Γ = Diag(Chol(C)) -1 Chol(C) ∈ LT 1 (n), (7.7) Φ : Γ ∈ LT 1 (n) -→ C = Diag(ΓΓ ) -1/2 ΓΓ Diag(ΓΓ ) -1/2 ∈ Cor + (n). (7.8)
We clearly have the relations Φ = Θ -1 = Cor • φ.

We compute the differentials of φ : LT + (n) -→ Sym + (n) and Chol : Sym

+ (n) -→ LT + (n). For all Z ∈ T L LT + (n) LT(n): V := d L φ(Z) = ZL + LZ , (7.9) L -1 V L -= L -1 Z + (L -1 Z) , Low S (L -1 V L -) = L -1 Z, Z = d Σ Chol(V ) = L Low S (L -1 V L -). (7.10)
In particular, d In Chol = Low S .

Quotient-affine metrics

In this section, we briefly recall how to build quotient metrics to fix the notations (Section 7.2.1), then we recall the definition of quotient-affine metrics (Section 7.2.2) and finally we show that the sectional curvature takes both negative and positive curvature and is unbounded from above (Section 7.2.3). Sections 7.2.1 and 7.2.2 partly summarize Chapter 6 to allow Chapter 7 to be read independently. The skilled reader may skip these sections to resume at Section 7.2.3.

Quotient metrics

Given smooth manifolds M, M and a smooth submersion π : M -→ M , we can define the vertical space V x = ker d x π ⊂ T x M, where d x π : T x M -→ T π(x) M is the differential of the map π at point x ∈ M. The horizontal space H x can be any supplementary vector space, i.e. such that V x ⊕ H x = T x M. Given a horizontal distribution x -→ H x , there exist vertical and horizontal projections ver x : T x M -→ V x and hor x : T x M -→ H x . Moreover, the linear map (d x π) |Hx : H x -→ T π(x) M is an isomorphism. Its inverse isomorphism is called the horizontal lift and denoted # x : X ∈ T π(x) M -→ X #

x ∈ H x . When M is endowed with a Riemannian metric, there is a canonical choice of horizontal space which is the orthogonal

H x = V ⊥
x . In this case, the projections are orthogonal. Given a smooth manifold M on which a Lie group G acts smoothly, properly and freely, the quotient space M/G admits a unique smooth manifold structure that turns the canonical projection π : M -→ M/G into a smooth submersion [Lee, 2012, Theorem 21.10]. The vertical distribution is G-equivariant, i.e. V a•x = a • V x for all a ∈ G (where • is the group action on M and T M). Given a G-invariant Riemannian metric g on M, the horizontal distribution is G-equivariant and the metric descends to a metric g on M/G defined by g π(x) (X, X) = g x (X #

x , X # x ).

Definition of quotient-affine metrics

Applying this to M = Sym + (n), G = Diag + (n) and M/G Cor + (n), the submersion Cor : [David and[START_REF] David | [END_REF] allows to descend any Diag + (n)-invariant Riemannian metric on Sym + (n) to a Riemannian metric on Cor + (n) [O'Neill, 1966].

Σ ∈ Sym + (n) -→ Diag(Σ) -1/2 Σ Diag(Σ) -1/2 ∈ Cor + (n)
A natural example of Diag + (n)-invariant Riemannian metric on SPD matrices is provided by the affine-invariant metric defined for all Σ ∈ Sym + (n) and V ∈ T Σ Sym + (n) Sym(n) by: g

AI(α,β) Σ (V, V ) = α tr(Σ -1 V Σ -1 V ) + β tr(Σ -1 V ) 2 (7.11)
where α > 0 and β > -α n . Its sectional curvature is for V, W ∈ Sym(n) [Skovgaard, 1984, Thanwerdas and[START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]:

κ AI(α,β) Σ (V, W ) = 1 4α tr((Σ -1 V Σ -1 W -Σ -1 W Σ -1 V ) 2 ) ∈ ï - 1 2α ; 0 ò (7.12)
The quotient-affine metric is the quotient of the affine-invariant metric via the submersion Cor : Sym + (n) -→ Cor + (n). It does not depend on β and it writes α g QA with, for all

C ∈ Cor + (n) and X ∈ T C Cor + (n) Hol(n): g QA C (X, X) = tr((C -1 X) 2 ) -21 Diag(C -1 X)(I n + C • C -1 ) -1 Diag(C -1 X)1. (7.13)
Note that it is invariant under permutations. The vertical/horizontal distributions/projections and the horizontal lift are for all Σ ∈ Sym + (n), V ∈ T Σ Sym + (n) Sym(n) and X ∈ T Cor(Σ) Cor + (n) Hol(n) [START_REF] Thanwerdas | Geodesics and Curvature of the Quotient-Affine Metrics on Full-Rank Correlation Matrices[END_REF]:

• V Σ = {DΣ + ΣD|D ∈ Diag(n)}, • H Σ = {V ∈ Sym(n)|Σ -1 V + V Σ -1 ∈ Hol(n)} = S Σ -1 (Hol(n)), • ver Σ (V ) = DΣ + ΣD ∈ V Σ with D = diag ((I n + Σ • Σ -1 ) -1 Diag(Σ -1 V )1), • hor Σ (V ) = V -ver Σ (V ) ∈ H Σ , • X # Σ = hor Σ (Diag(Σ) 1/2 XDiag(Σ) 1/2 ) ∈ H Σ ,
where S A (X) is the unique solution of the Sylvester equation AS A (X) + S A (X)A = X. These operations allow to write the sectional curvature for all C ∈ Cor + (n) and X, Y ∈ T C Cor + (n) Hol(n) [START_REF] Thanwerdas | Geodesics and Curvature of the Quotient-Affine Metrics on Full-Rank Correlation Matrices[END_REF]:

κ QA C (X, Y ) = κ AI C (X # C , Y # C ) ∈[-1 2 ;0] + 3 8 µ (I n + C • C -1 ) -1 µ g QA C (X, X)g QA C (Y, Y ) -g QA C (X, Y ) 2 0 , (7.14) with µ ∈ R n defined by µ = [D(X, Y ) -D(Y, X)]1 ∈ R n and D(X, Y ) ∈ Diag(n) defined by D(X, Y ) = Diag([C -1 Diag(X # C )C, C -1 Y # C ])
, where [A, B] = AB -BA is the commutator of squared matrices.

Complement: bounds of curvature

Theorem 7.1 (Bounds of curvature) The sectional curvature of the quotient-affine metric takes positive and negative values. It is bounded from below and unbounded from above. See complete proof of Theorem 7.1 in Section 11.6. [START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]. So the curvature takes negative values. Third, let X = 11 -I n and Y

Sketch of the proof. First of all, κ C (X, Y ) κ AI C (X, Y ) -1 2 so the curvature is bounded from below. Second, at C = I n , X # = X and Y # = Y so Diag(X # ) = Diag(Y # ) = 0 and µ = 0. Hence, κ In (X, Y ) = κ AI In (X, Y ) 0 and for example κ In (E ij , E ik ) = -1 8 < 0 with i = j = k = i ∈ {1, ..., n}
= µ1 + 1µ -2 diag(µ) with sum(µ) = 1 µ = 0 where µ ∈ R n . Let C = (1 -ρ)I n + ρ11 ∈ Cor + (n) for ρ ∈ (-1
n-1 , 1). We show in Section 11.6 that κ C (X, Y ) tends to +∞ when ρ → -1 n-1 , which proves that the curvature takes positive values and it is not bounded from above.

Hence, the Riemannian manifold of full-rank correlation matrices endowed with the quotient-affine metric is geodesically complete but it is not a CAT(k) space for any k ∈ R. In particular, it is not a Hadamard space as one could have hoped. It may be a problem for many algorithms because the Riemannian logarithm and the Fréchet mean are not ensured to be unique.

In Section 7.3, we relax the invariance under permutations that might be unsuitable in some contexts. In Section 7.4, we define non-permutation-invariant metrics which in addition bring uniqueness of the mean.

Generalization of quotient-affine metrics

In this section, we define a family of quotient metrics which generalize quotient-affine metrics and which are not invariant by permutations. In Section 7.3.1, we start by giving an overview of the congruence action of several subgroups of the general linear group GL(n) on SPD matrices. In particular, we show that affine-invariant metrics are exactly (S(n) × LT + (n))invariant metrics so the family of LT + (n)-invariant metrics appears as a natural generalization of affine-invariant metrics. In Section 7.3.2, we show that LT + (n)-invariant metrics are exactly pullbacks by the Cholesky map of left-invariant metrics on the Lie group LT + (n). We call them Lie-Cholesky metrics. In particular, we define a Lie group structure on SPD matrices such that the affine-invariant metric is a left-invariant metric on that Lie group. This is an unexpected result as the SPD cone endowed with an affine-invariant metric is always seen as a Riemannian homogeneous (symmetric) space. In Section 7.3.3, we define quotient-Lie-Cholesky metrics and we show that we can compute numerically the exponential map, the logarithm map and the Riemannian distance.

Congruence actions of matrix Lie groups on SPD matrices

The action of congruence of the real general linear group GL(n) on SPD matrices is:

ß GL(n) × Sym + (n) -→ Sym + (n) (A, Σ) -→ AΣA . (7.15)
In the following theorem, we focus on the subactions given by the following subgroups.

1. The group of matrices with positive determinant GL + (n).

The special linear group SL

(n) = {A ∈ GL(n)| det(A) = 1} ⊂ GL + (n)
, which is interesting for the invariance of covariance matrices under volume-preserving linear transformations of the feature vector. 4. The group of lower triangular matrices with positive diagonal LT + (n), which is interesting because an LT + (n)-invariant metric on SPD matrices is a left-invariant metric on a Lie group (cf. Section 7.3.2).

5. The positive diagonal group Diag + (n) ⊂ LT + (n), which is interesting for the invariance of covariance matrices under scalings on each variable.

6. The group of positive real numbers R + which injects itself into the general linear group via the map λ ∈ R + -→ λI n ∈ GL + (n). It is interesting for the invariance of covariance matrices under global scaling.

7. The permutation group S(n) which injects itself into the orthogonal group via the map σ ∈ S(n) → P σ ∈ O(n) defined by [P σ ] ij = δ i,σ(j) for all i, j ∈ {1, ..., n}. It is interesting for the invariance of covariance matrices under permutation of the axes.

In the proof, we also use the following notations: R * is the group of invertible scalar matrices, Diag * (n) is the group of invertible diagonal matrices, LT * (n) is the group of invertible lower triangular matrices, UT + (n) is the group of upper triangular matrices with positive diagonal.

Theorem 7.2 (Characterization of affine-invariant metrics) Let g be a Riemannian metric on SPD matrices. The following statements are equivalent:

1. g is GL(n)-invariant, 2. g is GL + (n)-invariant, 3. g is SL(n)-invariant and R + -invariant, 4. g is SO(n)-invariant and Diag + (n)-invariant, 5. g is S(n)-invariant and LT + (n)-invariant.
See the proof of Theorem 7.2 in Section 11.6.

One interpretation of this result is that these pairs of invariance (SL(n) and R + ; SO(n) and Diag + (n); S(n) and LT + (n)) are incompatible except in the affine-invariant metrics. In other words, the family of LT + (n)-invariant metrics is a natural non-permutation-invariant extension of the family of affine-invariant metrics. In the next section, we characterize them as pullback metrics of left-invariant metrics on the Lie group LT + (n) by the Cholesky map.

Lie-Cholesky metrics

LT + (n)-invariant metrics are Lie-Cholesky metrics

The manifold LT + (n) is an open set of the vector space LT(n). It is a Lie group where the internal law is the matrix multiplication.

Consequences

Corollary 7.5 The Riemannian metrics on Sym + (n) which are LT + (n)-invariant are geodesically complete.

Corollary 7.6 Permutation-invariant Lie-Cholesky metrics are affine-invariant metrics.

Corollary 7.7 (Affine-invariant metrics on SPD matrices are left-invariant metrics on a Lie group) The manifold Sym + (n) has a structure of Lie group given by the matrix multiplication of the Cholesky factors, i.e. for all Σ, Σ ∈ Sym + (n), denoting L = Chol(Σ) and L = Chol(Σ ), the internal law is:

Σ Σ = (LL )(LL ) = LΣ L .
(7.16)

The affine-invariant metrics g Σ (X, X) = α tr(Σ -1 XΣ -1 X) + β tr(Σ -1 X) 2 are left-invariant metrics for this Lie group structure. They are pullbacks of the left-invariant metrics on the Lie group LT + (n) characterized by the following inner products at

I n ∈ LT + (n), where Z ∈ T In LT + (n) LT(n): Z|Z α,β = g In (Z + Z , Z + Z ) = α tr((Z + Z )(Z + Z )) + β tr(Z + Z ) 2 = 2α tr(ZZ ) + 2α tr(Diag(Z) 2 ) + 4β tr(Z) 2 . (7.17) We denote •|• = •|• 1/2,0 , i.e. Z|Z = tr(ZZ ) + tr(Diag(Z) 2 ) for Z ∈ LT(n).
The left-invariant metrics on LT + (n) are parametrized by the inner products at I n . Hence, they can be parametrized by self-adjoint positive definite linear maps f : LT(n) -→ LT(n) as follows, for all Z, Z ∈ LT(n):

Z • Z = f (Z)|Z = Z|f (Z ) . (7.18)
In other words, if vec : LT(n) -→ R n(n+1)/2 denotes the linear map defined by vec(Z) = (Z 11 , Z 21 , Z 22 , ..., Z n,n-1 , Z nn ) for all Z ∈ LT(n), then there exists an SPD matrix A ∈ Sym + ( n(n+1)

2

) such that Z • Z = vec(Z) A vec(Z ). Thus the Lie-Cholesky metrics are parametrized by self-adjoint positive definite linear maps f : LT

(n) -→ LT(n) or SPD matrices A ∈ Sym + ( n(n+1)

2

). They are denoted LC(f ) or LC(A). Therefore, we can see the family of Lie-Cholesky metrics as a natural non-permutationinvariant extension of affine-invariant metrics. Since they are left-invariant metrics on a Lie group, the numerical computation of the geodesics (exponential map, logarithm map) and the parallel transport is simpler and much more stable and precise than for other metrics [Guigui and Pennec, 2021b]. The curvature is known in closed form modulo the computation of the operator ad * : LT [Besse, 1987, Theorem 7.30]. The operator ad * for Lie-Cholesky metrics is given in the following lemma.

(n) × LT(n) -→ LT(n) defined for all X, Y, Z ∈ LT(n) by ad * (X)(Y )|Z = Y |[X, Z]
Lemma 7.8 (Operator ad * for Lie-Cholesky metrics) Let LC(A) be a Lie-Cholesky metric characterized by

A ∈ Sym + ( n(n+1) 2 ). Then for X, Y ∈ LT(n), vec(ad * (X)(Y )) = A(I n ⊗X - X ⊗ I n )A -1 vec(Y ).
See the proof of Lemma 7.8 in Section 11.6.

Quotient-Lie-Cholesky metrics

Lie-Cholesky metrics being LT + (n)-invariant, they are in particular Diag + (n)-invariant as the affine-invariant metric. As recalled earlier, the smooth map Cor : Sym + (n) -→ Cor + (n) is a Diag + (n)-invariant submersion so Lie-Cholesky metrics descend to quotient-Lie-Cholesky metrics on full-rank correlation matrices. Quotient-Lie-Cholesky metrics naturally extend quotient-affine metrics. To use them, one needs to compute the vertical and horizontal projections. After the definition of quotient-Lie-Cholesky metrics, the following lemma gives the expression of the vertical and horizontal distributions. Definition 7.9 (Quotient-Lie-Cholesky metric) A quotient-Lie-Cholesky metric is the quotient metric of a Lie-Cholesky metric by the submersion Cor : Sym + (n) -→ Cor + (n). Equivalently, it is the pushforward metric by the diffeomorphism Φ :

L ∈ LT 1 (n) -→ Diag(LL ) -1/2 LL Diag(LL ) -1/2 ∈ Cor + (n) of a quotient metric on LT 1 (n) Diag(n)\LT + (n) defined by quotient via the submersion π : L ∈ LT + (n) -→ Diag(L) -1 L ∈ LT 1 (n).
In this section, we express all the Riemannian operations of quotient-Lie-Cholesky metrics LT 1 (n). It suffices to push them forward by Φ to get them on Cor + (n).

Lemma 7.10 (Vertical distribution, horizontal distribution) The vertical and horizontal distributions associated to the quotient-Lie-Cholesky associated to f are

V L = Diag(n)L and H LC(f ) L = L f -1 (L -1 d LL Chol(S (LL ) -1 (Hol(n)))).
See the proof of Lemma 7.10 in Section 11.6.

Then, to compute the vertical and horizontal projections, it suffices to take bases of the vertical space V L and the horizontal space H L , to orthonormalize them by Gram-Schmidt process and to project onto these orthonormal bases. Then the horizontal lift can be computed as follows, which allows to compute the metric and the exponential map.

Lemma 7.11 (Horizontal lift, Riemannian metric, exponential map) Given the horizontal projection hor L : T L LT + (n) -→ H L and the exponential map of the Lie-Cholesky metric Exp LC : LT(n) -→ LT + (n), we have for all L ∈ LT + (n), for all Γ = Diag(L) -1 L ∈ LT 1 (n), for all ξ ∈ T Γ LT 1 (n) LT 0 (n):

• (Horizontal lift) ξ # L = hor L (Diag(L)ξ), • (Riemannian metric) g QLC Γ (ξ, ξ) = f (Γ -1 hor Γ (ξ))|Γ -1 hor Γ (ξ) , • (Exponential map) Exp QLC Γ (tξ) = Diag(Exp LC Γ (t hor(ξ))) -1 Exp LC Γ (t hor(ξ)).
In particular, quotient-Lie-Cholesky metrics are geodesically complete.

See the proof of Lemma 7.11 in Section 11.6.

The Riemannian logarithm and the Riemannian distance can then be computed by minimizing the Lie-Cholesky distance along a fiber. The Lie-Cholesky distance is itself computed numerically with efficient tools on Lie groups [Guigui and Pennec, 2021b].

In this section, we gave an overview of the congruence action of several matrix Lie groups on SPD matrices. It allowed to understand that the natural extension of affine-invariant metrics to non-permutation-invariant metrics is the family of LT + (n)-invariant metrics. We showed that such metrics are pullbacks of left-invariant metrics on the Lie group LT + (n) by the Cholesky diffeomorphism. Hence, the space of lower triangular matrices and the Cholesky map naturally appear when one wants to get rid of the invariance under permutations on SPD matrices. Following the same construction as for the quotient-affine metric, we built quotient-Lie-Cholesky metrics and we showed that most of the interesting Riemannian operations can be computed numerically. However, this geometry is not completely satisfying since we have no formula in closed form and no obvious nice geometric properties except geodesic completeness. Moreover, the numerical computation of the Riemannian logarithm and the Riemannian distance are done with the generic methods in quotient manifolds which can be heavy and unstable. Furthermore, the quotient-affine metric has unbounded curvature as shown in Section 7.2 and it is difficult to say something about the curvature of general quotient-Lie-Cholesky metrics.

Hence in the following section, we continue to rely on lower triangular matrices and the Cholesky map to define Riemannian metrics with simpler geometries than the general quotient geometry of quotient-Lie-Cholesky metrics.

New geometric structures with unique mean

In this section, we define new families of metrics on the open elliptope of full-rank correlation matrices. They are based on simple geometries of subspaces of lower triangular matrices and transported to the elliptope via the Cholesky map or the derived map Θ. In Section 7.4.1, we introduce the poly-hyperbolic-Cholesky metrics which provide Riemannian symmetric structures. Then we introduce the Euclidean-Cholesky metrics (Section 7.4.2) and the log-Euclidean-Cholesky metrics (Section 7. [START_REF]A quotient-affine logarithm of C ∈ Cor + (n) at I n is a tangent vector X ∈ Hol(n) of minimal length such that Exp QA In (X) = Cor(exp(X)) = C[END_REF].3) that provide Euclidean structures. Equipped with a metric of one of these three families, the open elliptope is Hadamard so the Riemannian logarithm and the Fréchet mean are unique. In Section 7.4.4, we introduce a Lie group structure that allows to define Cartan-Schouten affine connections and left-invariant metrics. The group mean of the canonical Cartan-Schouten connection is unique. In Section 7.4.5, we give the geodesics in dimension 2, which correspond to interpolations of one correlation coefficient.

Symmetric space: poly-hyperbolic-Cholesky metrics

In this section, we use the diffeomorphism Chol |Cor + (n) : Cor + (n) -→ L where L is the set of lower triangular matrices with positive diagonal such that each row is unit normed for the canonical Euclidean norm. Thus, the k-th row of L = Chol(C) ∈ L writes (L k1 , ..., L k,k-1 , L kk , 0, ..., 0) with L kk > 0. It belongs to the open hemisphere HS

k-1 = {x ∈ R k | x = 1 and x k > 0}. So each L ∈ L is a point in HS 0 × • • • × HS n-1
. This construction is clearly bijective and diffeomorphic. Note that since L 11 = 1 and HS 0 = {1}, we can remove it from the Cartesian product. Hence, we can define the diffeomorphism Ψ : L -→ HS 1 × • • • × HS n-1 . Moreover, an open hemisphere is one of the avatars of the hyperbolic space, which is the Riemannian manifold of negative constant curvature.

Before introducing the Riemannian metric induced by the diffeomorphism Ψ, we recall the definition and Riemannian operations of the hyperbolic space in the model of the hyperboloid. Theorem 7.12 (Hyperbolic geometry of the hyperboloid) The vector space R k+1 is endowed with the non-degenerate quadratic form Q(x) = k i=1 x 2 i -x 2 k+1 . We denote (x, y) -→ Q(x, y) the associated symmetric bilinear form. The hyperboloid H k is the Riemannian manifold defined by H k = {x ∈ R k+1 |Q(x) = -1} endowed with the induced pseudo-metric, which is a Riemannian metric on H k . The tangent space writes T x H k = {v ∈ R k+1 |Q(x, v) = 0}. For all x, y ∈ H k , v ∈ T x H k and w ∈ T x H k non colinear to v, the Riemannian operations are:

• (Riemannian metric) g H x (v, v) = Q(v), • (Riemannian norm) v H = Q(v), • (Riemannian distance) d H (x, y) = arccosh(-Q(x, y)), • (Exponential map) Exp H x (v) = cosh( v H )x + sinh( v H ) v v H , • (Logarithm map) Log H x (y) = d H (x, y) y+Q(x,y)x y+Q(x,y)x H , • (Sectional curvature) κ H x (v, w) = -1.
The formulae on the other models of the hyperbolic space can be obtained by pullback via the appropriate diffeomorphism. In particular, the diffeomorphism between the open hemisphere HS

k = {(x 1 , ..., x k+1 ) ∈ R k × R + | k+1 i=1 x 2 i = 1}
and the hyperbolic space H k is:

ϕ SH : (x 1 , ..., x k+1 ) ∈ HS k -→ 1 x k+1 (x 1 , ..., x k , 1) ∈ H k . (7.19)
Thus the natural metric on the open hemisphere is the pullback metric g HS = (ϕ SH ) * g H . Definition 7.13 (Poly-hyperbolic-Cholesky metrics) Let α 1 , ..., α n-1 > 0 be positive coefficients. A poly-hyperbolic-Cholesky metric on Cor + (n) is the pullback metric g PHC = (Ψ • Chol) * (α 1 g HS 1 ⊕ ... ⊕ α n-1 g HS n-1 ) by the map Chol • Ψ of a weighted product metric on the product of hyperbolic spaces HS 1 × • • • × HS n-1 . The PHC metric with all weights equal to 1 is called the canonical PHC metric.

Theorem 7.14 (Symmetric space structure) The manifold of full-rank correlation matrices Cor + (n) equipped with a poly-hyperbolic-Cholesky metric is a Riemannian symmetric space of non-positive sectional curvature bounded by [a, 0] with a = -1 min i 2 α i . For n 3, it is not of constant curvature. The canonical PHC metric writes for all C ∈ Cor + (n) and (7.20) where L = Chol(C) ∈ L. The square distance between C and C = φ(L ) writes: (7.21) where L i , L i are the i-th rows of L, L respectively. See the proof of Theorem 7.14 in Section 11.6.

X ∈ T C Cor + (n) Hol(n): g CPHC C (X, X) = Diag(L) -1 L Low S (L -1 XL -) 2 ,
d CPHC (C, C ) 2 = n i=2 arccosh(-Q(L i , L i )) 2 ,

Vector space: Euclidean-Cholesky metrics

The Cholesky map was already used on SPD matrices [START_REF] Wang | A constrained variational principle for direct estimation and smoothing of the diffusion tensor field from complex dwi[END_REF]. The Euclidean-Cholesky metric is defined as the Euclidean metric on the Cholesky factor belonging to LT + (n). However, it is not complete because LT + (n) is open in LT(n). A solution to this problem is to take the logarithm of the diagonal before taking the Euclidean metric [Pinheiro andBates, 1996, Lin, 2019]. It amounts to define the product metric of a Euclidean metric on the strictly lower part and a log-Euclidean metric on the diagonal part.

For correlation matrices, we can use the diffeomorphism Θ : Cor + (n) -→ LT 1 (n). Since the diagonal is I n , the two mentioned metrics reduce to the same metric on the open elliptope. We call it the Euclidean-Cholesky metric.

Definition 7.15 (Euclidean-Cholesky metrics) The Euclidean-Cholesky metrics on full-rank correlation matrices are pullback metrics by Θ :

Cor + (n) -→ LT 1 (n) of inner products on LT 1 (n) = I n + LT 0 (n).
Theorem 7.16 (Riemannian operations) Let • be a Euclidean norm on LT 0 (n). The Riemannian operations of the Euclidean-Cholesky metric associated to this norm are, for all

C, C , C i ∈ Cor + (n), X ∈ T C Cor + (n) Hol(n), t ∈ R: • (Exponential map) Exp C (tX) = Θ -1 (Θ(C) + t d C Θ(X)), • (Logarithm map) Log C (C ) = (d C Θ) -1 (Θ(C ) -Θ(C)), • (Geodesic) γ C→C (t) = Θ -1 ((1 -t)Θ(C) + t Θ(C )), • (Distance) d(C, C ) = Θ(C ) -Θ(C) , • (Parallel transport) Π C→C X = (d C Θ) -1 (d C Θ(X)), • (Curvature) Null, • (Fréchet mean) C = Θ -1 ( 1 n n i=1 Θ(C i )), where d C Θ(X) = Θ(C)Low S (L -1 XL -) -1
2 Diag(L -1 XL -)Θ(C) and L = Chol(C). The Euclidean-Cholesky metrics are geodesically complete. These metrics are flat, geodesically complete and the Riemannian operations are trivial. Since they reduce to (the pullback of) an inner product on a vector space, we prefer not to use the term Lie group for them, contrarily to the terminology of [START_REF] Li | Local Log-Euclidean Multivariate Gaussian Descriptor and Its Application to Image Classification[END_REF], Lin, 2019]. We prefer to reserve it for Lie groups that are not vector spaces, such as the natural Lie group structure of LT + (n) (with matrix multiplication) underlying Lie-Cholesky metrics.

Vector space: log-Euclidean-Cholesky metrics

Another map was used to Euclideanize the manifold LT + (n): the matrix logarithm [START_REF] Li | Local Log-Euclidean Multivariate Gaussian Descriptor and Its Application to Image Classification[END_REF]. Indeed, the matrix exponential is a smooth diffeomorphism from LT(n) to LT + (n) [Gallier, 2008]. We can use the same idea for correlation matrices since the matrix exponential is a smooth diffeomorphism from LT 0 (n) to LT 1 (n). Moreover it has a particularly simple expression: (7.22) because LT 0 (n) is a nilpotent algebra. Then the logarithm log : LT 1 (n) -→ LT 0 (n) is simply:

exp(ξ) = n-1 k=0 1 k! ξ k ,
log(Z) = n-1 k=1 (-1) k-1 k (Z -I n ) k . (7.23)
Therefore, the differential of the logarithm writes:

d Z log(ξ) = n-1 k=1 (-1) k k [(Z -I n ) k-1 ξ + (Z -I n ) k-2 ξ(Z -I n ) + ... + ξ(Z -I n ) k-1 ]. (
∈ Cor + (n), X ∈ T C Cor + (n) Hol(n), t ∈ R: • (Exponential map) Exp C (tX) = Θ -1 • exp(log(Θ(C)) + t d C (log • Θ)(X)), • (Logarithm map) Log C (C ) = (d C (log • Θ)) -1 (log(Θ(C )) -log(Θ(C))), • (Geodesic) γ C→C (t) = Θ -1 • exp((1 -t) log(Θ(C)) + t log(Θ(C ))), • (Distance) d(C, C ) = log(Θ(C )) -log(Θ(C)) , • (Parallel transport) Π C→C X = (d C (log • Θ)) -1 (d C log • Θ(X)),
• (Curvature) Null,

• (Fréchet mean) C = Θ -1 • exp( 1 n n i=1 log(Θ(C i ))),
where

d C (log • Θ)(X) = d Θ(C) log(d C Θ(X)
). The log-Euclidean-Cholesky metrics are geodesically complete.

The computation of the Riemannian operations of the Euclidean-Cholesky metrics is more straightforward than those of the log-Euclidean-Cholesky because one has to compute the differential of the triangular matrix logarithm for the latter.

Nilpotent Lie group structure

Another interesting structure is given by the natural Lie group structure of LT 1 (n) for the matrix multiplication. This equips full-rank correlation matrices with a Lie group structure via the diffeomorphism Θ : Cor + (n) -→ LT 1 (n). Hence, left-invariant metrics can be defined. In analogy with Sym + (n) LT + (n), they can also be called Lie-Cholesky metrics. Then all Riemannian operations can be computed numerically [Guigui and Pennec, 2021b] and the space is ensured to be geodesically complete. However, this doesn't give information on the sign of the curvature.

More interestingly, one can rely on the canonical Cartan-Schouten connection to define the group exponential and the notion of group mean. We can also name them after Lie-Cholesky.

Theorem 7.19 (Group operations) The group operations associated to the Lie-Cholesky group structure on full-rank correlation matrices are, for all C, C ,

C i ∈ Cor + (n), X ∈ T C Cor + (n) Hol(n), t ∈ R: • (Exponential map) Exp C (tX) = Θ -1 (Θ(C) exp(t Θ(C) -1 d C Θ(X)), • (Logarithm map) Log C (C ) = (d C Θ) -1 (Θ(C) log(Θ(C) -1 Θ(C ))), • (Geodesic) γ C→C (t) = Θ -1 (Θ(C)(Θ(C) -1 Θ(C )) t ), • (Group mean) Unique, characterized by k i=1 log(Θ( C) -1 Θ(C i )) = 0.
See the proof of Theorem 7.19 in Section 11.6.

Explicit geodesics in dimension 2

In dimension 2, the elliptope is reduced to one parameter. All full-rank correlation matrices

write C = C(ρ) = Å 1 ρ ρ 1 ã with ρ ∈ (-1, 1)
. Therefore, the quotient-affine metric and the metrics defined in Section 7.4 only depend on one scaling parameter. They actually split in two groups and the geodesics can be computed in closed forms. The two formulae in the following result provide two different interpolations of the correlation coefficient. The proof is in the supplementary material.

Theorem 7.20 (Geodesics in dimension

2) Let C 1 = C(ρ 1 ), C 2 = C(ρ 2 ) ∈ Cor + (2) with ρ 1 , ρ 2 ∈ (-1, 1).
1. Quotient-affine metrics and poly-hyperbolic-Cholesky metrics coincide (up to a scaling factor). The geodesic between C 1 and C 2 is C(ρ(t)) for t ∈ R where:

ρ(t) = ρ 1 cosh(λt) + sinh(λt) ρ 1 sinh(λt) + cosh(λt) , (7.25) where λ = log » 1+ρ 2 1-ρ 2 -log » 1+ρ 1
1-ρ 1 is known as the difference of the Fisher transformation of the correlation coefficients ρ 1 and ρ 2 .

2. Euclidean-Cholesky and log-Euclidean-Cholesky metrics coincide. The geodesic between C 1 and C 2 is C(ρ(t)) for t ∈ R where: (7.26) where

ρ(t) = F (t) 1 + F (t) 2 ,
F (t) = (1-t) ρ 1 √ 1-ρ 2 1 +t ρ 2 √ 1-ρ 2 2
. This geodesic also coincides with the Lie-Cholesky group geodesic of Section 7.4.4.

See the proof of Theorem 7.20 in Section 11.6.

Conclusion

In this work, we proposed new Riemannian metrics on the open elliptope of full-rank correlation matrices that are not invariant under permutations. To the best of our knowledge, all the existing geometric structures were invariant under permutations. Thus the geometries we propose significantly departs from the classical structures. This can be a good assumption in some applications and an irrelevant characteristic in some others. We generalized the recently introduced quotient-affine metrics by studying the congruence action of several matrix Lie groups on SPD matrices. We showed that the family of LT + (n)-invariant metrics is a natural non-permutation-invariant generalization of affine-invariant metrics. Moreover, they are pullbacks of left-invariant metrics on the Lie group LT + (n) by the Cholesky map. They are invariant under the congruence action of positive diagonal matrices so they descend to the elliptope. We explained that the main Riemannian operations can be computed numerically for these quotient-Lie-Cholesky metrics. However, we also showed that the curvature of quotient-affine metrics is unbounded and we can conjecture that the situation is not better for quotient-Lie-Cholesky metrics. In addition, the Riemannian operations are not computable in closed form a priori.

That is why we introduced new Riemannian metrics on the elliptope in a different way. We kept the Cholesky map which seems to be a good alternative to the invariance under permutations since the space of lower triangular matrices is not stable by permutations. Thus we defined the poly-hyperbolic-Cholesky (PHC) metrics which provide non-positively curved Riemannian symmetric space structures. We also defined two kinds of vector space structures that are flat, geodesically complete and for which all operations are known in closed form. Thus, these three families of metrics provide a Hadamard structure, in particular the Riemannian logarithm and the Fréchet mean are unique. We also put forward a nilpotent Lie group structure for which the group mean is unique. Finally, we proved that in dimension 2, the PHC geodesics are the quotient-affine geodesics and the geodesics of the three last structures coincide. This provides a new interpolation of the correlation coefficient.

It would be nice to test these new metrics on different kinds of data in future works. Moreover, all metrics on correlation matrices provide new product metrics on covariance matrices by decoupling the scales of the variables and the correlations between them. This approach seems promising since in many problems, the correlation gives more information than the covariance on the strength of the relations between the variables, although the scales can remain interesting. Thus, one question could be to adjust the weights between the two components and also between the scales of the variables. The possibilities are multiplied now we have many metrics on correlation matrices. Another direction of research is to investigate permutation-invariant Riemannian metrics on correlation matrices with a simpler geometry than the one of the quotient-affine metrics, for example Hadamard or even flat.

Chapter 8

Permutation-invariant Log-Euclidean metrics Abstract

There is a growing interest in defining specific tools on correlation matrices which depart from those suited to SPD matrices. Several geometries have been defined on the open elliptope of full-rank correlation matrices: some are permutation-invariant, some others are log-Euclidean, i.e. diffeomorphic to a Euclidean space. In this work, we merge these two properties by defining the families of off-log metrics and log-scaled metrics. Firstly, we prove that the recently introduced off-log bijection is a diffeomorphism, allowing to pullback (permutation-invariant) inner products. We introduce the "cor-inverse" involution on the open elliptope which can be seen as analogous to the inversion of SPD matrices. We show that off-log metrics are not inverse-consistent. That is why secondly, we define the log-scaling diffeomorphism between the open elliptope and the vector space of symmetric matrices with null row sums. This map is based on the congruence action of positive diagonal matrices on SPD matrices, more precisely on the existence and uniqueness of a "scaling", i.e. an SPD matrix with unit row sums within an orbit. Thanks to this multiplicative approach, log-scaled metrics are inverse-consistent. We provide the main Riemannian operations in closed form for the two families modulo the computation of the respective bijections. In particular, we define a new algorithm that computes the scaling of an SPD matrix in logarithmic time complexity.

Introduction

In many domains such as Diffusion Tensor Imaging, Brain-Computer Interfaces, brain connectomes or radar signals, the data are time series which are often represented by their covariance matrices. They encode the dependence between the variables and the scale of intensity of these variables. Many Riemannian geometries were proposed to compute with covariance matrices with more natural tools that the Euclidean ones. The use of the affineinvariant metric was shown to outperform many results based on Euclidean metrics such as fiber reconstruction in DTI [START_REF] Pennec | A Riemannian Framework for Tensor Computing[END_REF], movement classification in BCI [START_REF] Barachant | Classification of covariance matrices using a Riemannian-based kernel for BCI applications[END_REF] or detection of brain functional connectivity [START_REF] Varoquaux | Detection of brain functional-connectivity difference in post-stroke patients using group-level covariance modeling[END_REF]. Shortly after, the log-Euclidean metric [START_REF] Arsigny | Log-Euclidean metrics for fast and simple calculus on diffusion tensors[END_REF], Fillard et al., 2007] was shown to be a more efficient alternative to the affine-invariant metric with similar results. The Bures-Wasserstein metric was also proposed to deal with low-rank matrices since the two previous ones are only defined on the space Sym + (n) of Symmetric Positive Definite (SPD) matrices. All these metrics belong to the wide families of kernel metrics [START_REF] Hiai | Riemannian metrics on positive definite matrices related to means[END_REF] and O(n)-invariant metrics [START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]. Non O(n)-invariant metrics were also proposed such as the Cholesky [Wang et al., 2004, Grubišić andPietersz, 2007], log-Euclidean-Cholesky [START_REF] Li | Local Log-Euclidean Multivariate Gaussian Descriptor and Its Application to Image Classification[END_REF] and log-Cholesky [Lin, 2019] metrics.

In the previously cited domains and in other ones such as phylogenetic trees [START_REF] Garba | Information geometry for phylogenetic trees[END_REF] or Gaussian graphical networks [Lauritzen, 1996, Epskamp and[START_REF] Epskamp | [END_REF], another possible and natural choice to represent the data is the correlation matrix instead of the covariance matrix. However, the geometries of correlation matrices have been much less studied. Hence they are often considered as covariance matrices on which one can use the classical tools. Nevertheless, these tools are not adapted to correlation matrices, at least the O(n)-invariant ones. Indeed, firstly, the manifold of full-rank correlation matrices is not stable by the congruence action of the orthogonal group so this action has no sense for them. Secondly, it is not a totally geodesic submanifold for neither of the noted O(n)invariant metrics on SPD matrices, except the Euclidean metric. This motivates the study of intrinsic geometries of correlation matrices. Such geometries could also have a great impact on applications with covariance matrices since they would provide product metrics with one part on diagonal matrices and the other part on correlation matrices. It would thus allow to decouple the scale of the variables from the dependence between the variables.

Among the geometries proposed on correlation matrices, one involves a surjection from a product of spheres [Rebonato andJaeckel, 2001, Kercheval, 2008]. It is an orbit space [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF], this construction is quite analogous to the Bures-Wasserstein geometry of covariance matrices. However, to our knowledge, it has not been precisely described yet. A metric space structure called the Hilbert geometry relies on the convexity of the set [START_REF] Nielsen | Clustering in Hilbert's Projective Geometry: The Case Studies of the Probability Simplex and the Elliptope of Correlation Matrices[END_REF]. Among the Riemannian structures, the recently introduced quotient-affine metric is obtained by taking the quotient of the affine-invariant metric under the congruence action of positive diagonal matrices : (∆, Σ) [David andGu, 2019, Thanwerdas andPennec, 2021]. Indeed, full-rank correlation matrices can be seen as the orbits of this action so the space Cor + (n) of such matrices is the quotient manifold Sym + (n)/Diag + (n) and any invariant metric on Sym + (n) Cor + (n) × Diag + (n) descends to a Riemannian metric on Cor + (n). These constructions have the common property to be invariant under permutations. It means that the statistical analysis is invariant under reordering the variables, which can be a relevant hypothesis when the order is arbitrary. When the order of the variables is meaningfully chosen depending on the application (e.g. for auto-correlation matrices), other Riemannian metrics that are not permutation-invariant can be considered. The metrics proposed in [START_REF] Thanwerdas | Theoretically and computationally convenient geometries on full-rank correlation matrices[END_REF]] provide a Hadamard structure or even a vector space structure, which are very convenient for computing with correlation matrices.

∈ Diag + (n) × Sym + (n) -→ ∆Σ∆ ∈ Sym + (n)
Given this short survey on geometries of correlation matrices, there is an obvious gap to fill in: no permutation-invariant log-Euclidean metrics have been derived yet on the space Cor + (n) of full-rank correlation matrices. By log-Euclidean, we mean the pullback of an inner product on a vector space V by a diffeomorphism referred to as a logarithm and denoted Log : Cor + (n) -→ V. In this work, we propose two approaches to define such a diffeomorphism. The first one is based on a recent bijective parametrization of full-rank correlation matrices by the space LT 0 (n) of lower triangular matrices with null diagonal introduced by Archakov and Hansen [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF]. The second one is entirely new. We rephrase their framework to present the two approaches in a similar way to facilitate the comparison between them and the comprehension of the second one by analogy with the first one. These two methods are summarized in Table 8.1 and explained below.

First approach (additive)

Second approach (multiplicative) Off-log diffeomorphism Cor-exp diffeomorphism Action + :

ß Diag(n) × Sym(n) -→ Sym(n) (D, S) -→ D + S : ß Diag + (n) × Sym + (n) -→ Sym + (n) (∆, Σ) -→ ∆Σ∆ Claim ∀S ∈ Sym(n), ∃! D := D(S) ∈ Diag(n) : ∀Σ ∈ Sym + (n), ∃! ∆ := D (Σ) ∈ Diag + (n) : exp(D + S) ∈ Cor + (n) log(∆ Σ) ∈ V Status
Claim proved in [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF] Claim proved for V = Row 0 (n) in Section 3 The left part is based on [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF]. The right part is new; V is a vector space stable by permutations satisfying Sym(n) = V ⊕ Diag(n).

Diagram Cor + (n) Log=Off • log Hol(n) / / Exp ? ? Sym(n) D / / exp • (D+Id Sym(n) ) O O Diag(n) Cor + (n) / / Log ! ! Sym + (n) D / / log • (D Id Sym + (n) ) Diag + (n) V Exp =Cor

Results and organization of the chapter

In the remainder of this section, we introduce the necessary notations. We also define a natural involution on the open elliptope called the cor-inversion, which allows to define a notion of inverse-consistency for Riemannian metrics.

In [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF], the authors show the claim on the left of Table 8.1, that is for all symmetric matrix S, there exists a unique diagonal matrix D = D(S) such that exp(D + S) is a full-rank correlation matrix. Thus it defines a surjective map π : S ∈ Sym(n) -→ exp •(D(S) + S) ∈ Cor + (n) which is equivariant under permutations, and a bijective map L ∈ LT 0 (n) -→ π(L+L ) ∈ Cor + (n). We astutely replace the space LT 0 (n) by the space V = Hol(n) of symmetric matrices with null diagonal (which is of same dimension) so that the restriction Exp = π |V : V -→ Cor + (n) is also equivariant under permutations. Note that Hol(n) is actually the tangent space of Cor + (n). Moreover, we trivially observe that π is invariant by the additive action of a diagonal matrix. Our contribution is to show that the bijection Exp is a smooth (that is C ∞ ) diffeomorphism, to define by pullback the family of off-log metrics and to provide all the Riemannian operations in closed form modulo the computation of D. An algorithm with logarithmic time complexity is already defined in [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF] to compute D. This additive approach is summarized on the left part of Table 8.1 and exposed in Section 8.2.

Our second approach consists in inverting the roles played by the vector spaces Sym(n) = V ⊕ Diag(n) and the manifolds Sym + (n) = Cor + (n) × Diag + (n), as well as the matrix exponential and the matrix logarithm, and especially to replace the additive action of Diag(n) on Sym(n) by the congruence action : (∆, Σ)

∈ Diag + (n)×Sym + (n) -→ ∆Σ∆ ∈ Sym + (n).
In this work, we find a vector space V such that for all Σ ∈ Sym + (n), there exists a unique ∆ = D (Σ) ∈ Diag + (n) such that log(∆Σ∆) ∈ V . This allows to define the surjective map π : Σ ∈ Sym + (n) -→ log(D (Σ) Σ) ∈ V and the bijective map Log = π |Cor + (n) : Cor + (n) -→ V which are equivariant under permutations. This multiplicative approach is summarized on the right part of Table 8.1 and exposed in Section 8.3. One major advantage of this multiplicative approach is that it intrinsically respects the structure of correlation matrices since Cor + (n) = Sym + (n)/Diag + (n), contrarily to the additive approach. The main consequence is the compatibility with the inversion, i.e. π (C -1 ) = -π (C) for all C ∈ Cor + (n).

More precisely, we try to prove the claim with V = Hol(n) and V = Row 0 (n), where Row 0 (n) is the vector space of symmetric matrices with null row sums. With the first choice, we only manage to prove the existence. We actually prove that the uniqueness would imply the uniqueness of the Riemannian logarithm at identity of the quotient-affine metric mentioned above, which is an open problem. This is a secondary contribution that relates two problems on full-rank correlation matrices. In contrast, we prove the claim with V = Row 0 (n). Indeed, we show that exp(Row 0 (n)) = Row + 1 (n), where Row + 1 (n) is the submanifold of SPD matrices with unit row sums. This reduces our question to the famous problem of scaling an SPD matrix to prescribed row sums by congruence of a positive diagonal matrix: for all SPD matrix Σ, does there exist a positive diagonal matrix ∆ = D (Σ) such that ∆Σ∆ ∈ Row + 1 (n). The answer is yes [Marshall andOlkin, 1968, Johnson and[START_REF] Johnson | [END_REF]] so the claim is true.

In Section 8.4, similarly to the additive approach, we prove that the bijection Log is a smooth diffeomorphism and we define by pushforward the family of log-scaled metrics. Contrarily to off-log metrics, they are inverse-consistent. We provide all the Riemannian operations in closed form modulo the computation of D , that is the computation of the scaling of an SPD matrix. In Section 8.5, we design a new algorithm to compute the scaling of an SPD matrix which is more efficient than generic algorithms to compute the scaling such as [Khachiyan andKalantari, 1992, O'Leary, 2003] (see surveys [START_REF] Johnson | Scaling of symmetric matrices by positive diagonal congruence[END_REF], Idel, 2016, Allen-Zhu et al., 2017]). We build it by following the proof of existence given by [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF] and adapting the efficient Projected Gradient Descent (PGD) algorithm [Bubeck, 2015]. We conclude in Section 8.6.

Notations

Matrices

Tables 8.2 and 8.3 summarize our notations for matrix spaces. We also use the following constant and linear operators on vectors and matrices:

• 1 ∈ R n is the vector with all entries equal to 1;

• diag : R n -→ Diag(n) makes a diagonal matrix from a vector;

• sum : R n -→ R computes the sum of the entries of a vector;

• Diag : Mat(n) -→ Diag(n) extracts the diagonal matrix from a matrix;

• Off : Mat(n) -→ ker Diag substracts the diagonal matrix from a matrix;

• Sum : Mat(n) -→ R computes the sum of entries of a matrix;

• • : Mat(n) × Mat(n) -→ Mat(n) computes the Hadamard/Schur (entry-wise) product of matrices. Squared of size n Mat(n) = {M = [M ij ] 1 i,j n |M ij ∈ R} Skew-symmetric Skew(n) = {Y ∈ Mat(n)|Y = -Y } Symmetric Sym(n) = {X ∈ Mat(n)|X = X} Diagonal Diag(n) = {Diag(X)|X ∈ Mat(n)} Symmetric hollow (null diagonal) Hol(n) = {X ∈ Sym(n)|Diag(X) = 0} Symmetric null-row-sum Row 0 (n) = {X ∈ Sym(n)|X1 = 0} Table 8.2: Matrix vector spaces. Invertible GL(n) = {A ∈ Mat(n)| det(A) = 0} Orthogonal O(n) = {A ∈ GL(n)|AA = I n } Symmetric Positive Definite Sym + (n) = {AA |A ∈ GL(n)} Positive diagonal Diag + (n) = Sym + (n) ∩ Diag(n) Full-rank correlation Cor + (n) = {C ∈ Sym + (n)|Diag(C) = I n } SPD unit-row-sum Row + 1 (n) = {Σ ∈ Sym + (n)|Σ1 = 1} Table 8.3: Matrix manifolds.
We recall the definition of the congruence action :

(A, M ) ∈ GL(n) × Mat(n) -→ AM A ∈ Mat(n).
Since the permutation group S(n) is a subgroup of the orthogonal group via the canonical injection σ ∈ S(n

) → P σ = [δ i,σ(j) ] 1 i,j n ∈ O(n), the permutation action (σ, M ) ∈ S(n) × Mat(n) -→ σ • M = P σ M P σ ∈ Mat(n) is the congruence action of the subgroup S(n).
We recall the definition of the matrix exponential exp :

M ∈ Mat(n) -→ +∞ k=0 1 k! M k ∈ GL(n)
which is a smooth map. Its restriction to symmetric matrices is a smooth diffeomorphism onto SPD matrices, exp : Sym(n) -→ Sym + (n). The symmetric matrix logarithm is its smooth inverse, log : Sym + (n) -→ Sym(n). The computation of exp, log and their differentials are particularly simple modulo eigenvalue decomposition. Given Σ (8.4) where

= P DP ∈ Sym + (n), X = Q∆Q , Y ∈ Sym(n) where P, Q ∈ O(n), D ∈ Diag + (n) and ∆ ∈ Diag(n): exp(X) = Q exp(∆)Q , (8.1) log(Σ) = P log(D)P , (8.2) d X exp(Y ) = Q Ä [exp [1] (δ i , δ j )] 1 i,j n • (Q Y Q) ä Q , (8.3) d Σ log(Y ) = P Ä [log [1] (d i , d j )] 1 i,j n • (P Y P ) ä P ,
f [1] (x, y) = ® f (x)-f (y) x-y if x = y f (x) if x = y
´is the first divided difference of f ∈ {exp, log} [Bhatia, 1997]. In other words, the maps exp, log, d exp, d log are O(n)-equivariant, and therefore S(n)-equivariant.

Correlation matrices

The

∈ Sym + (n) -→ Diag(Σ) -1/2 Σ Diag(Σ) -1/2 ∈ Cor + (n). Given Σ ∈ Sym + (n), X ∈ Sym(n), denoting ∆ = Diag(Σ) -1/2 , its differential is: d Σ Cor(X) = ∆ ï X - 1 2 (∆ 2 Diag(X)Σ + ΣDiag(X)∆ 2 ) ò ∆. (8.5)
We introduce a notation for equicorrelation matrices

C(ρ) = (1 -ρ)I n + ρ11 ∈ Cor + (n) where ρ ∈ (-1 n-1 , 1
). Given a correlation matrix C ∈ Cor + (n), there exist partitions of n, i.e. sets I = {i 1 , ..., i p } satisfying i 1 , ..., i p 1 and i 1 + • • • + i p = n, partitioning the matrix C into equicorrelation diagonal blocks and constant off-diagonal blocks. The signature of C is the maximum I C of such sets I with respect to the natural order on partitions of n. We say that C is a block equicorrelation matrix of signature I C (see Table 8.4). For example, an equicorrelation matrix is a block equicorrelation matrix with signature {n}. The maps introduced in this chapter preserve the signature.

C(ρ) = á 1 ρ • • • ρ ρ 1 . . . . . . . . . . . . . . . ρ ρ . . . ρ 1 ë á C(ρ 1 ) ρ 12 11 • • • ρ 1p 11 ρ 12 11 C(ρ 2 ) . . . . . . . . . . . . . . . ρ p-1,p 11 ρ 1p 11 • • • ρ p-1,p 11 C(ρ p )
ë Table 8.4: Equicorrelation and block equicorrelation matrices.

The elliptope is stable by the permutation action, it is not stable by the congruence action of the orthogonal group O(n). It is not stable by inversion either. However, the correlation matrix of its inverse, namely Cor(C -1 ), contains the same information as the partial correlation matrix defined below.

Definition 8.1 (Cor-inversion, partial correlation matrix)

• The cor-inversion is the smooth involution

I : C ∈ Cor + (n) -→ Cor(C -1 ) ∈ Cor + (n). • The partial correlation matrix of an SPD matrix Σ ∈ Sym + (n) is the matrix Γ ∈ Sym(n) defined by Γ = I n -Off(I(C)) = 2I n -I(C), where C = Cor(Σ).
The bijective parametrization C -→ Γ is used in the theory of stationary stochastic processes where the (potentially complex and infinite-dimensional) matrices are Toeplitz. The set of partial correlation coefficients (along with the common variance) is considered as an alternative "represention of the second-order statistics" [Burg, 1975, Section II.B.5] of the process with respect to the traditional "auto-correlation" (or auto-covariance) function. This characterization is used in signal processing, especially in radar signal processing where the manifold of SPD Toeplitz matrices is traditionally endowed with the Poincaré polydisk geometry [Barbaresco, 2013]. In Gaussian graphical networks, the partial correlation between two variables indicates the correlation between them conditionally to the other variables. bijection they define, that we call the off-log bijection and that we denote Log : Cor + (n) -→ Hol(n), is actually a diffeomorphism. It allows to pullback inner products on full-rank correlation matrices. Since the off-log diffeomorphism is equivariant under permutations, we give a characterization of permutation-invariant inner products on Hol(n) so that their pullbacks provide permutation-invariant log-Euclidean metrics on Cor + (n). Then, we detail the Riemannian operations of these metrics. We prove that, as expected, the log-Euclidean metrics such defined are not inverse-consistent with respect to the cor-inversion. In Section 8.2.3, we simply recall the algorithm of [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF] to compute the inverse diffeomorphism Exp = Log -1 , the speed of convergence and the complexity.

The off-log bijection

Theorem 8.3 states that the claim in the left part of Table 8.1 is true. It allows to define the off-log bijection Log : Cor + (n) -→ Hol(n). Theorem 8.4 states some interesting properties of the off-log bijection. These results are due to [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF].

Theorem 8.3 (Definition of D) [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF] For all S ∈ Sym(n), there exists a unique D ∈ Diag(n) such that exp(D + S) ∈ Cor + (n). This allows to define:

• the surjective map D : S ∈ Sym(n) -→ D ∈ Diag(n),
• the surjective map π : S ∈ Sym(n) -→ exp(D(S) + S) ∈ Cor + (n) which is invariant under the additive group action + : Diag(n) × Sym(n) -→ Sym(n),

• the bijective map Exp = π |Hol(n) : Hol(n) -→ Cor + (n) (note that π = Exp • Off),

• the smooth bijective inverse map Log = Exp -1 = Off • log : Cor + (n) -→ Hol(n) that we call the off-log bijection.

Theorem 8.4 (Properties of the off-log bijection) [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF] 1. (Equivariance) Log and Exp are equivariant under permutations.

(Equicorrelation matrix) For all

ρ ∈ (-1 n-1 , 1), Log(C(ρ)) = 1 n ln Ä 1+(n-1)ρ 1-ρ ä (11 -I n ). In dimension n = 2, Log(C(ρ)) = Å 0 F (ρ) F (ρ) 0 ã where F (ρ) = 1 2 log( 1+ρ 1-ρ ) ∈ R is the Fisher transformation of the correlation coefficient ρ ∈ (-1, 1).

(Block equicorrelation matrix) If

C is a block equicorrelation matrix of signature I = {i 1 , ..., i p }, then Log(C) is a block symmetric hollow matrix of signature I with multiples of 1 i j 1 i j -I i j on diagonal blocks and multiples of 1 i j 1 i k on off-diagonal blocks.

4. (Generalization) For all ∆ ∈ Diag + (n), for all S ∈ Sym(n), there exists a unique D ∈ Diag(n) such that Diag(exp(D + S)) = ∆.

Note that Theorem 8.3 is a particular case of Theorem 8.4 item 4 with ∆ = I n . The result in dimension 2 was stated as a motivation in [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF] to use the map Log = Off •log in higher dimensions since it gives in dimension 2 a well known transformation of the correlation coefficient. Interestingly, the same coefficient appears in dimension 2 for the quotient-affine metric [START_REF] Thanwerdas | Geodesics and Curvature of the Quotient-Affine Metrics on Full-Rank Correlation Matrices[END_REF].

By analogy with the symmetric matrix logarithm log : Sym + (n) -→ Sym(n) satisfying log(Σ -1 ) = -log(Σ), one could expect that the off-log bijection Log : Cor + (n) -→ Hol(n) "commutes with inversion", i.e. satisfies Log(I(C)) = -Log(C). We show that it is not the case as we argued in the introduction.

Theorem 8.5 (Incompatibility between cor-inversion and off-log bijection) Let n 3. There exists C ∈ Cor + (n), such that Log(I(C)) = -Log(C). Otherwise said, the following diagram does not commute.

Cor

+ (n) I / / Log Cor + (n) Log Hol(n) -Id / / Hol(n) (8.7)
See the proof of Theorem 8.5 in Section 11.7.

This incompatibility is one of the justifications of the multiplicative approach that we present in Section 8.3. Still, this bijection Log : Cor + (n) -→ Hol(n) remains a very nice tool that allows to define permutation-invariant log-Euclidean metrics on full-rank correlation matrices. Let us show this.

Permutation-invariant pullback metrics via the off-log diffeomorphism

This section is part of our contributions. We prove that the off-log bijection Log : Cor + (n) -→ Hol(n) is actually a diffeomorphism (Section 8.2.2.1). Then we characterize all permutationinvariant inner products on Hol(n) (Section 8.2.2.2) and we pull them back to permutationinvariant log-Euclidean metrics on Cor + (n) (Section 8.2.2.3).

The off-log bijection is a diffeomorphism

Theorem 8.6 (Log = Off • log is a diffeomorphism) The off-log bijection Log : Cor + (n) -→ Hol(n) is a smooth diffeomorphism. We give the differentials of Log and Exp in function of the differentials of the symmetric matrix logarithm and exponential maps log and exp. For all C ∈ Cor + (n) and S, X, Y ∈ Hol(n):

d C Log(X) = Off(d C log(X)), (8.8) d S Exp(Y ) = d log(Exp(S)) exp(Y -Diag(Exp(S)Y )).
(8.9)

See the proof of Theorem 8.6 in Section 11.7.

Permutation-invariant inner products on Hol(n)

The characterization of permutation-invariant inner products on Hol(n) can be found in Section 3.3.4.1 of Chapter 3.

Theorem 8.7 (Permutation-invariant inner products on Hol(n)) [Section 3.3.4.1] For n 4, permutation-invariant inner products on Hol(n) are the symmetric bilinear forms associated to the following positive definite quadratic forms defined for X ∈ Hol(n):

q(X) = α tr(X 2 ) + β Sum(X 2 ) + γ Sum(X) 2 (8.10)
with α > 0, 2α + (n -2)β > 0 and α + (n -1)(β + nγ) > 0. For n = 3, the permutationinvariant inner products have the same form with α = 0, i.e. q(X) = β Sum(X 2 )+γ Sum(X) 2 with β > 0 and β + 3γ > 0. For n = 2, they have the same form with α = β = 0, i.e. q(X) = γ Sum(X) 2 with γ > 0.

Pullback metrics via the off-log diffeomorphism

Definition 8.8 (Off-log metrics) An off-log metric on Cor + (n) is the pullback metric of an inner product characterized by a quadratic form q as in Definition 8.7. For all C ∈ Cor + (n) and

X ∈ T C Cor + (n) = Hol(n), it writes g C (X, X) = q(d C Log(X)) where d C Log(X) = Off(d C log(X)).
Theorem 8.9 (Riemannian operations of off-log metrics) We consider an off-log metric characterized by the quadratic form q. Let C, C , C 1 , ..., C k ∈ Cor + (n), X ∈ Hol(n). The Riemannian operations of this metric are summarized in Table 8.5.

Exponential map Exp

C (X) = Exp(Log(C) + d C Log(X))) Logarithm map Log C (C ) = d Log(C) Exp(Log(C ) -Log(C)) Geodesic γ(t) = Exp((1 -t)Log(C) + t Log(C )) Squared distance d(C, C ) 2 = q(Log(C ) -Log(C)) Fréchet mean C = Exp( 1 k k i=1 Log(C i )) Curvature R = 0 Parallel transport Π C→C X = (d C Log) -1 (d C Log(X))
Table 8.5: Riemannian operations of off-log metrics.

Beware that the Riemannian exponential and logarithm maps only coincide with the diffeomorphisms Exp : Hol(n) -→ Cor + (n) and Log : Cor + (n) -→ Hol(n) at C = I n introduced in Theorem 8.3. They differ from the symmetric matrix diffeomorphisms exp :

Sym(n) -→ Sym + (n) and log : Sym + (n) -→ Sym(n).
Therefore, the off-log diffeomorphism provides a closed-form distance between two fullrank correlation matrices (modulo the computation of a symmetric matrix logarithm, i.e. modulo an eigenvalue decomposition). Moreover, all the other Riemannian operations can be computed in closed form modulo the computation of Exp, i.e. the computation of D. In the next section, we recall how it is computed in [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF].

Numerical computation of the off-log inverse map

The implicit functions D and Exp = exp • (D + Id Sym(n) ) can be computed very efficiently, as proved in [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF].

k -D * L k 1-L D 1 -D 0 .
3. (Time complexity) For all ε > 0, the sufficient number of iterations so that

D k -D * < ε is k ε = O(ln( n ε )). 4. (Complexity) At each iteration, a symmetric matrix exponential is computed which is a O(n 3 ). Hence the complexity with precision ε > 0 is O(n 3 k ε ) = O(n 3 ln( n ε )).
In this section, we recalled the main facts on the off-log parametrization introduced in [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF]] and we transformed it as a geometric tool to introduce log-Euclidean metrics on full-rank correlation matrices. We also formalized this tool in terms of invariance under a group action and we showed that off-log metrics are not inverse-consistent. In the next sections, we rely on this formalization to introduce the family of log-scaled metrics which are permutation-invariant, log-Euclidean and inverse-consistent.

The log-scaling bijection

In this section, we examine two versions of the following conjecture: for all Σ ∈ Sym + (n), there exists a unique ∆ ∈ Diag + (n) such that log(∆Σ∆) ∈ V . This conjecture depends on V , which is a vector space stable by permutations satisfying Sym(n) = V ⊕ Diag(n). In Section 8.3.1, we relate the conjecture with V = Hol(n) to the problem of the quotient-affine logarithm [David andGu, 2019, Thanwerdas andPennec, 2021]. We prove the existence and we explain why the uniqueness remains difficult to prove. In Section 8.3.2, we explain why V = Row 0 (n) is a good candidate for the conjecture to be true and in Section 8.3.3, we prove the conjecture thanks to a result known as the existence and uniqueness of the scaling of SPD matrices [Marshall andOlkin, 1968, Johnson and[START_REF] Johnson | [END_REF]. We explain the proof of existence because it can be difficult to understand in [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF]] and because we need it in Section 8.5 to design the algorithm that computes the scaling. In Section 8.3.4, we give the properties of our new Euclideanization called the log-scaling bijection.

Is the conjecture true with V = Hol(n)?

Before relating the conjecture with V = Hol(n) to the problem of existence and uniqueness of the Riemannian logarithm at I n of the quotient-affine metric, we first recall the conjecture and the definition of the quotient-affine metric.

Conjecture 8.11 (The result is true with V = Hol(n)) For all Σ ∈ Sym + (n), there exists a unique matrix ∆ ∈ Diag + (n) such that log(∆Σ∆) ∈ Hol(n). Definition 8.12 (Quotient-affine metric) The quotient-affine metric on Cor + (n) is the quotient metric of the affine-invariant metric on Sym + (n) by the congruence action of Diag + (n) [START_REF] David | A Riemannian structure for correlation matrices[END_REF]. At I n , the horizontal space is H QA In = Hol(n), the quotient-affine metric writes g QA In (X, X) = tr(X 2 ) and the exponential map writes Exp QA In (X) = Cor(exp(X)) for all X ∈ Hol(n) [START_REF] Thanwerdas | Geodesics and Curvature of the Quotient-Affine Metrics on Full-Rank Correlation Matrices[END_REF]. Theorem 8.13 (Existence and equivalence of conjectures) We define the smooth map f : ∆ ∈ Diag + (n) -→ d AI (I n , ∆C∆) 2 = tr(log(∆C∆) 2 ). It gives the affine-invariant squared distance between I n and all points of the fiber Cor

-1 (C) = {∆C∆ ∈ Sym + (n)|∆ ∈ Diag + (n)}.
1. The smooth map f has a global minimizer.

2. For all Σ ∈ Sym + (n), there exists ∆ ∈ Diag + (n) such that log(∆Σ∆) ∈ Hol(n).

The following conjectures are equivalent for all C ∈ Cor + (n).

(i) There exists a unique ∆ ∈ Diag + (n) such that log(∆C∆) ∈ Hol(n) (Conjecture 8.11).

(ii) There exists a unique X ∈ Hol(n) such that Exp QA In (X) = C. (iii) There exists a unique local minimizer of the smooth map f , which is actually the global minimizer ensured by statement 1.

4. The previous conjectures imply the uniqueness of the quotient-affine logarithm at I n .

See the proof of Theorem 8.13 in Section 11.7.

Otherwise said, Conjecture 8.11 is stronger than the conjecture stating the uniqueness of the quotient-affine logarithm at I n . On the one hand, this could provide a new path to prove the latter. However, Conjecture 8.11 seems difficult to prove because the manifold exp(Hol(n)) is hard to describe in terms of properties on the coefficients of the matrices. Thus it is difficult to determine whether its intersection with the fiber Diag + (n) Σ = Cor -1 (Cor(Σ)) is reduced to one point or not. On the other hand, this could also help to show that Conjecture 8.11 is false. Indeed, the quotient-affine metric has both positive and negative curvature so the quotient-affine logarithm might not be unique. Hence, this seems to be a difficult problem.

Why V = Row 0 (n) seems to be a better choice

Nevertheless, another interesting decomposition of symmetric matrices where each subspace is stable by permutations is given by Sym

(n) = Row 0 (n) ⊕ Diag(n), where Row 0 (n) = {S ∈ Sym(n)|S1 = 0}
is the vector space of symmetric matrices with null row sums. That is why we propose to examine V = Row 0 (n). In the following theorem, we show that exp(Row 0 (n)) has a nice form.

Theorem 8.14 (exp : Row

0 (n) -→ Row + 1 (n) is a smooth diffeomorphism) The symmetric matrix logarithm is a smooth diffeomorphism from Row + 1 (n) = {Σ ∈ Sym + (n)|Σ1 = 1} onto Row 0 (n) = {S ∈ Sym(n)|S1 = 0}. Proof. It is clear that exp(Row 0 (n)) ⊂ Row + 1 (n) since if S ∈ Row 0 (n), then exp(S)1 = ∞ k=0 1 k! S k 1 = 1. Conversely, let Σ ∈ Row + 1 (n). Then the Lagrange polynomial P (X) = λ∈eig(Σ) log(λ) µ∈eig(Σ),µ =λ X+1-µ λ-µ satisfies P (Σ -I n ) = log Σ. Since 1 ∈ eig(Σ), P (0) = log(1) + λ =1 log(λ) 1-1 λ-1 µ =λ,1 1-µ λ-µ = 0. Hence log(Σ)1 = P (Σ -I n )1 = P (0)1 = 0. So log(Row + 1 (n)) ⊂ Row 0 (n). Finally, Row + 1 (n) = exp(Row 0 (n)) so log : Row + 1 (n) -→ Row 0 (n) is a smooth diffeomorphism.
Hence, the question becomes: for all Σ ∈ Sym + (n), does there exist a unique ∆ ∈ Diag + (n) such that ∆Σ∆ ∈ Row + 1 (n)? The answer is yes [Marshall andOlkin, 1968, Johnson and[START_REF] Johnson | [END_REF], let us explain why.

The conjecture is true with

V = Row 0 (n)
We recall that we denote : Diag + (n) × Sym + (n) -→ Sym + (n) the congruence action of positive diagonal matrices on SPD matrices.

Theorem 8.15 (Definition of D ) For all Σ ∈ Sym + (n), there exists a unique ∆ ∈ Diag + (n) such that log(∆Σ∆) ∈ Row 0 (n) [Marshall andOlkin, 1968, Johnson and[START_REF] Johnson | [END_REF]. This allows to define:

• the surjective map D : Σ ∈ Sym + (n) -→ ∆ ∈ Diag + (n), • the surjective map π : Σ ∈ Sym + (n) -→ log(D (Σ) Σ) ∈ Row 0 (n) which is invariant under the congruence group action of Diag + (n) on Sym + (n),
• the bijective map Log = π |Cor + (n) : Cor + (n) -→ Row 0 (n) that we call the log-scaling (note that π = Log • Cor),

• the smooth bijective inverse map Exp = (Log )

-1 = Cor • exp : Row 0 (n) -→ Cor + (n).
Proof of Theorem 8.15 (Definition of D ). The existence and uniqueness are due to [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF], the uniqueness has been proved differently later in [START_REF] Johnson | Scaling of symmetric matrices by positive diagonal congruence[END_REF]. Before explaining the proof of existence which is important to compute the solution numerically, let us explain the properties of the functions D , π , Log and Exp .

• The map D is surjective because D (∆ -2 ) = ∆ for all ∆ ∈ Diag + (n).

• The map π is surjective because if S ∈ Row 0 (n), then π (exp(S)) = S.

• The map Log is surjective because Log (Cor(exp(S))) = S and injective because if

C, C ∈ Cor + (n) are such that Log (C) = Log (C ), then D (C) C = D (C ) C so C = Cor(D (C) C) = Cor(D (C ) C ) = C .
• We just showed that (Log ) -1 = Cor • exp so Exp = (Log ) -1 is bijective and smooth.

The existence of ∆ ∈ Diag + (n) such that log(∆Σ∆) ∈ Row 0 (n) comes from the following more general result from Marshall and Olkin [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF]] that we explain below.

Theorem 8.16 (Existence of a scaling for prescribed row sum) [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF] Let S ∈ Sym(n) and µ ∈ (R + ) n . We define f :

x ∈ (R + ) n -→ x Sx ∈ R. We denote M µ = {x ∈ (R + ) n | n i=1 x µ i i = 1} which is a smooth hypersurface of R n . We assume that f |M µ is: 1. (bounded away from 0) ∃c > 0, ∀x ∈ M µ , |f (x)| c, 2. (unbounded above when x → ∞) ∀C 0, ∃M 0, ∀x ∈ M µ , x > M ⇒ f (x) > C.
Then there exists ∆ = ∆ µ ∈ Diag + (n) such that ∆S∆1 = µ. Theorem 8.16 implies the existence in Theorem 8.15 with S = Σ ∈ Sym + (n). Indeed, for all x ∈ M µ , f (x) λ min x 2 where λ min = min eig(Σ) > 0 so Assumption 2 is satisfied. Moreover, there exists ε > 0 such that the ball B(0, ε) satisfies B(0, ε)∩M µ = ∅ so if x ∈ M µ , then x ε. Thus for all x ∈ M µ , f (x) λ min ε 2 > 0 so Assumption 1 is satisfied. These assumptions are actually satisfied for any strictly copositive matrix [Marshall and Olkin, 1968, Theorem 1].

Detailed proof of Theorem 8.16. We follow the path sketched in [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF] giving more details. The proof consists in rewriting the equation ∆S∆1 = µ into a minimization problem: (8.11) where δ = ∆1 = (δ 1 , ..., δ n ) . Now we can easily find a function whose gradient is (twice) the left side of this equation, namely:

Sδ - Å µ 1 δ 1 , ..., µ n δ n ã = 0
F (∆) = δ Sδ -2 Ç n i=1 µ i log(δ i ) -log(c) å , (8.12)
where c > 0 is a constant. Equivalently, if we denote m = n i=1 µ i , x i = c -1/m δ i and λ = c -2/m , then the following function has the same gradient up to a scaling factor:

G(∆) = λF (∆) = x Sx -2λ n i=1 µ i log(x i ).
(8.13)

This function can be seen as the Lagrangian L(x, λ) of the minimization problem min x∈M µ f (x) = min f |M µ . Therefore, we prove that f |M µ reaches its infimum at a certain x * ∈ M µ (thanks to Assumption 2) and that this provides a solution ∆ * ∈ Diag + (n) to our problem (thanks to Assumption 1).

To prove that f |M µ reaches its infimum, it suffices to prove that the level sets

S f |M µ (a) = {x ∈ M µ |f (x) a} are compact. Indeed, the level sets S f |M µ (a) = f -1 |M µ (] -∞, a]) are closed in M µ since f |M µ is continuous on M µ and ] -∞, a] is closed in R. As M µ is closed in R n , S f |M µ (a) is closed in R n .
The level sets are bounded because according to Assumption 2, there exists M 0 such that for all x ∈ S f |M µ (a), x M . Hence the level sets are compact. Since at least one of them is non-empty and f is continuous on this compact set, it reaches its minimum at a certain x * ∈ M µ .

According to the Lagrange multiplier theorem, there exists a unique λ * such that (x * , λ * ) is a critical point of the Lagrangian. Since ∇L(x, λ) = (2Sx-2λ( µ 1

x 1 , ..., µn xn ), -2 n i=1 µ i log(x i )), the critical point (x * , λ * ) has to satisfy λ * = f (x * ) m . Thus according to Assumption 1, we have λ * = 0. Hence it provides a solution ∆ * = 1 √ λ * diag(x * ) to Equation (8.11). This demonstration not only allows to prove the result but also shows that

D (Σ) = 1 √ λ * diag(x * ) with x * = min M f and λ * = f (x * ) n where f : x ∈ (R + ) n -→ x Σx ∈ R + and M = M 1 = {x ∈ (R + ) n | n i=1 x i = 1}.
We use this fact in Section 8.5 to compute D (Σ).

Properties of the log-scaling bijection

Let us give properties of Log that are analogous to the properties of Log.

Theorem 8.17 (Properties of the log-scaling bijection) The log-scaling bijection satisfies the following properties.

1. (Equivariance) Log and Exp are equivariant under permutations.

2. (Equicorrelation) For all ρ ∈ (-

1 n-1 , 1), Log (C(ρ)) = 1 n ln Ä 1+(n-1)ρ 1-ρ ä (11 -nI n ). In dimension n = 2, Log (C(ρ)) = Å -F (ρ) F (ρ) F (ρ) -F (ρ) ã where F (ρ) = 1 2 log( 1+ρ 1-ρ ) ∈ R is the Fisher transformation of the correlation coefficient ρ ∈ (-1, 1).

(Block equicorrelation matrix) If

C is a block equicorrelation matrix of signature I = {i 1 , ..., i p }, then Log (C) is a block symmetric matrix with null row sum of signature I with diagonal blocks of the form (α j -β j )I i j + β j 1 i j 1 i j and off-diagonal blocks of the form β jk 1 i j 1 i k .

4. (Generalization) For all x ∈ (R + ) n , for all Σ ∈ Sym + (n), there exists a unique ∆ ∈ Diag + (n) such that log(∆Σ∆)x = 0.

See the proof of Theorem 8.17 in Section 11.7.

The log-scaling bijection has an additional property: it is compatible with the corinversion. It is a corollary of Theorem 8.2, the key reason being the use of the congruence action of positive diagonal matrices instead of the additive action of diagonal matrices. Cor

+ (n) I / / Log Cor + (n) Log Row 0 (n) -Id / / Row 0 (n) (8.14)
In this section, we proved the existence in the conjecture with V = Hol(n) and we proved the conjecture for V = Row 0 (n). This provides a bijective map Log : Cor + (n) -→ Row 0 (n) called the log-scaling bijection.

Permutation-invariant log-Euclidean metrics via the log-scaling bijection

In this section, we use the log-scaling bijection to define log-Euclidean metrics on Cor + (n). More precisely, in Section 8.4.1, we prove that the log-scaling bijection is a diffeomorphism. In Section 8.4.2, we characterize all permutation-invariant inner products on Row 0 (n). In Section 8.4.3, we define permutation-invariant log-Euclidean metrics by pullback and we give their geometric properties.

The cor-exp bijection is a diffeomorphism

d S Exp (Y ) = ∆ -1 ï d S exp(Y ) - 1 2 (∆ -2 Diag(d S exp(Y )) Σ + Σ Diag(d S exp(Y )) ∆ -2 ) ò ∆ -1 , (8.15) d C Log (X) = d Σ log Å ∆X∆ + 1 2 (X 0 Σ + ΣX 0 ) ã , (8.16)
where ∆ = Diag(Σ) 1/2 and X 0 = -2 diag((

I n + Σ) -1 ∆X∆1).
See the proof of Theorem 8.19 in Section 11.7.

Permutation-invariant inner products on Row 0 (n)

Theorem 8.20 (Permutation-invariant inner products on Row 0 (n)) For n 4, permutationinvariant inner products on Row 0 (n) are the symmetric bilinear forms associated to the following positive definite quadratic forms q defined for Y ∈ Row 0 (n): (8.17) with α > 0, nα + (n -2)δ > 0 and nα + (n -1)(δ + nζ) > 0. For n = 3, the permutationinvariant inner products have the same form with α = 0. For n = 2, they have the same form with α = δ = 0. See the proof of Theorem 8.20 in Section 11.7.

q (Y ) = α tr(Y 2 ) + δ tr(Diag(Y ) 2 ) + ζ tr(Y ) 2 ,

Pullback metrics via the log-scaling diffeomorphism

Definition 8.21 (Log-scaled metrics) An log-scaled metric on Cor + (n) is the pullback metric of an inner product characterized by a quadratic form q as in Definition 8.20. For all

C ∈ Cor + (n) and X ∈ T C Cor + (n) = Hol(n), it writes g C (X, X) = q (d C Log (X)) where d C Log (X) = d C log(X + 1 2 (X 0 C + CX 0 )) with X 0 = -2diag((I n + C) -1 X1).
Theorem 8.22 (Riemannian operations of log-scaled metrics) We consider a log-scaled metric characterized by the quadratic form

q . Let C, C , C 1 , ..., C n ∈ Cor + (n), X ∈ Row 0 (n).
The Riemannian operations of this metric are summarized in Table 8.6. Moreover, the metric is permutation-invariant and inverse-consistent, i.e. it is invariant under the pullback by the cor-inversion

I : Cor + (n) -→ Cor + (n).
Beware not to confuse the Riemannian maps Exp C :

T C Cor + (n) = Hol(n) -→ Cor + (n) and Log C : Cor + (n) -→ T C Cor + (n) = Hol(n) with the diffeomorphisms Exp : Row 0 (n) -→ Cor + (n) and Log : Cor + (n) -→ Row 0 (n).
Since one motivation behind the off-log bijection in [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF] was the closed form expression in dimension 2 (cf. Theorem 8.4.2), it seems important to note the following result.

Exponential map Exp

C (X) = Exp (Log (C) + d C Log (X))) Logarithm map Log C (C ) = d Log (C) Exp (Log (C ) -Log (C)) Geodesic γ (t) = Exp ((1 -t)Log (C) + t Log (C )) Squared distance d (C, C ) 2 = q (Log (C ) -Log (C)) Fréchet mean C = Exp ( 1 k k i=1 Log (C i )) Curvature R = 0 Parallel transport Π C→C X = (d C Log ) -1 (d C Log (X))
Table 8.6: Riemannian operations of log-scaled metrics.

Theorem 8.23 (Coincidence of the metrics in dimension 2) In dimension 2, up to a positive scaling factor, the quotient-affine metric, the off-log metric and the log-scaled metric coincide.

We recall that the Fisher transformation is the increasing map

F : ρ ∈ (-1, 1) -→ 1 2 ln( 1+ρ 1-ρ ) ∈ R + . Let C = C(ρ) and X = Å 0 x x 0 ã with ρ ∈ (-1, 1) and x ∈ R. Then: 1. (Metric) g C (X, X) = x 2 (1-ρ 2 ) 2 (up to a scaling factor α > 0), 2. (Geodesic) γ(t) = C(ρ(t)) where ρ(t) = ρ 1 cosh(λt)+sinh(λt) ρ 1 sinh(λt)+cosh(λt) with λ = F (ρ 2 ) -F (ρ 1 ) is monotonic (increasing if ρ 1 < ρ 2 , decreasing if ρ 1 > ρ 2 , constant if ρ 1 = ρ 2 ), 3. (Distance) d(C 1 , C 2 ) = |λ| = |F (ρ 2 ) -F (ρ 1 )| (up to a scaling factor √ α).
See the proof of Theorem 8.23 in Section 11.7.

We showed in this section that the log-scaling bijection is a diffeomorphism. Therefore it provides a family of permutation-invariant inverse-consistent log-Euclidean metrics by pulling back permutation-invariant inner products on Row 0 (n). Thus, the Riemannian operations are trivial. As shown in Table 8.6, they are known in closed form modulo the computation of Log : Cor + (n) -→ Row 0 (n) or equivalently the computation of D : Sym + (n) -→ Diag + (n). That is the goal of the next section.

Numerical computation of the log-scaling

The topic of scaling a matrix to prescribed row and column sums is much broader than the use we make of it in this chapter. The research on this topic in linear algebra started in the 1960's with the Sinkhorn theorem stating that squared positive matrices A ∈ Mat(n), i.e. with positive entries A ij > 0, admit a bistochastic scaling D 1 AD 2 where D 1 , D 2 are positive diagonal matrices. After several extensions of this result, the novelty of [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF] was to gather three ingredients: to tackle symmetric matrices A ∈ Sym(n), to impose D 1 = D 2 and to abandon the positivity of the coefficients. As recalled in Theorem 8.16, they proved the sufficiency of two hypotheses which are in particular satisfied for positive definite matrices and even for strictly copositive matrices. See the introduction of [START_REF] Johnson | Scaling of symmetric matrices by positive diagonal congruence[END_REF] for a brief survey or [Idel, 2016] for a thorough one.

According to [START_REF] Allen-Zhu | Much faster algorithms for matrix scaling[END_REF], this problem actually appeared in different scientific communities from telephone traffic computation to probability theory. Moreover, the results have been applied to many fields from image reconstruction to computer science. Thus, the question of the computation of the scaling has always been a burning topic. Examining the literature, there seems to exist very few algorithms specifically designed for SPD matrices with respect to positive matrices or other classes of matrices. That is surprising since the proof of Theorem 8.16 [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF] suggests that the positive definiteness could improve significantly the performance of an optimization algorithm since the map to minimize f : x ∈ R n -→ x Σx ∈ R + is strictly convex. We can cite [O'Leary, 2003] with polynomial complexity (without more information) and [START_REF] Khachiyan | Diagonal Matrix Scaling and Linear Programming[END_REF] with time complexity O( √ n ln(n) + ln(ln( 1 ε ))) via a Newton method (as proved in [Kalantari, 2005, Theorem 8.4 SP]), where ε is the desired precision. For the latter, each step requires to solve a linear system, which is O(n 2 ), so the total complexity is O(n 5/2 ln(n) + n 2 ln(ln( 1 ε ))). As for generic methods, the best one seems to be from [START_REF] Allen-Zhu | Much faster algorithms for matrix scaling[END_REF] where the total complexity is O(m + n 1/4 ) where m is the number of non-zero entries in the matrix.

In this section, we provide an algorithm to compute the scaling of an SPD matrix with precision ε, therefore to compute D : Sym + (n) -→ Diag + (n) and Log : Cor + (n) -→ Row 0 (n), with complexity O(m ln( n ε ) + n ln( n ε ) ln(ln( n ε ))) which seems quite competitive. In Section 8.5.1, we explain the convex optimization problem we need to solve to compute D , based on the proof of Theorem 8.16 [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF]. In Section 8.5.2, we explain that this kind of problem can be solved with logarithmic time complexity by Projected Gradient Descent (PGD) if one knows the projection [Bubeck, 2015]. Since we don't have the projection in closed form, we propose in Section 8.5.3 an efficient algorithm to compute it. Finally in Section 8.5.4, we modify the PGD algorithm with the projection approximation and we show that it still converges with logarithmic time complexity.

The optimization problem

From the proof of Theorem 8.16, we know that

D (Σ) = 1 √ λ * diag(x * ) with x * = min M f and λ * = f (x * ) n where f : x ∈ (R + ) n -→ x Σx ∈ R + and M = {x ∈ (R + ) n | n i=1 x i = 1}.
Since the map f is a positive definite quadratic form, it is natural to think about convex optimization to compute x * . However, this map has to be minimized on the set M which is not convex. In the following result, we show that x * is also the minimum of f on the convex hull of M, which is

Ω = {x ∈ (R + ) n | n i=1 x i 1}.
Theorem 8.24 (Optimization problem to compute the scaling) arg min Ω f ∈ M and Ω = conv(M). Thus x * is the global minimizer of the convex map f on the closed convex set Ω.

See the proof of Theorem 8.24 in Section 11.7.

Optimizing an α-strongly convex β-smooth map on a closed convex set

There are many algorithms to optimize convex maps on closed convex sets, as described in [Bubeck, 2015]. The optimal algorithm depends on the properties of the map. The simplest one is the Projected Gradient Descent (PGD): it consists in alternating an unconstrained gradient descent and a projection onto the convex set of constraints. More precisely, if f : R n -→ R is a map to minimize on the closed convex set C ⊆ R n , the algorithm takes

x 0 ∈ C, computes y k+1 = x k -η∇f |x k and x k+1 = Π C y k+1
where η > 0 is a fixed parameter and Π C is the projection onto the closed convex set C. The PGD algorithm is shown to converge with logarithmic time complexity when the map is α-strongly convex and β-smooth. [Bubeck, 2015] In this section, we recall the definitions of an α-strongly convex map and a β-smooth map and we show that f : x -→ x Σx satisfies these two properties. Then we recall the definition of the projection onto a closed convex set C and the theorem of convergence of the PGD algorithm for an α-strongly convex and β-smooth map.

α-strongly convex β-smooth maps

Definition 8.25 (α-strongly convex map) [Bubeck, 2015] Let C ⊆ R n be a convex set. We say that the map f : C -→ R is α-strongly convex if one of the following equivalent assertions is satisfied:

(i) f ((1 -t)x 1 + tx 2 ) (1 -t)f (x 1 ) + tf (x 2 ) -α 2 t(1 -t) x 1 -x 2 2 for all t ∈ [0, 1], for all x 1 , x 2 ∈ C, (ii) the function f -α 2 • 2 is convex.
If f is differentiable, then these assertions are equivalent to a third one:

(iii) f (x 1 ) -f (x 2 ) ∇f |x 1 (x 1 -x 2 ) -α 2 x 1 -x 2 2 for all x 1 , x 2 ∈ C.
Definition 8.26 (β-smooth map) [Bubeck, 2015] We say that f : R n -→ R is β-smooth if f is of class C 1 and its gradient is β-Lipschitz, i.e. for all x 1 , x 2 ∈ R n , we have

∇f |x 1 -∇f |x 2 β x 1 -x 2 .
We state an important inequality satisfied by β-smooth maps from [Bubeck, 2015], that we use later.

Lemma 8.27 (Important result on β-smooth maps) [Bubeck, 2015]

Let f : R n -→ R be a β-Lipschitz map. Then for all x 1 , x 2 ∈ R n , f (x 1 ) -f (x 2 ) -∇f |x 2 (x 1 -x 2 ) β 2 x 1 -x 2 2 .
Now, we state that f : x -→ x Σx satisfies these two properties.

Lemma 8.28 (The map f is 2λ min -strongly convex and 2λ max -smooth) Let Σ ∈ Sym + (n). We denote λ min , λ max the minimum and maximum eigenvalues of Σ. The map f :

x ∈ R n -→ x Σx ∈ R is 2λ min -strongly convex and 2λ max -smooth. Proof. The map f -λ min • 2 : x ∈ R n -→ x (Σ -λ min I n )x is convex because Σ -λ min I n is symmetric positive semi-definite. So f is 2λ min -strongly convex. Moreover, ∇f |x = 2Σx so ∇f |x 1 -∇f |x 2 = 2 Σ(x 1 -x 2 ) 2λ max x 1 -x 2 . Hence, f is 2λ max -smooth.

Projection and Projected Gradient Descent

We recall the definition of the projection onto a closed convex set and the theorem of convergence of the Projected Gradient Descent for α-strongly convex β-smooth maps. Theorem 8.30 (Projected gradient descent for α-strongly convex β-smooth maps) [Bubeck, 2015] Let f : C -→ R be a convex map on a convex set C. We assume that f is α-strongly convex and β-smooth and we denote κ = β/α. Let x 0 ∈ C. We define

y k+1 = x k -1 β ∇f |x k and x k+1 = Π C (y k+1 ). Then (x k ) k∈N converges to x * = arg min C f . Moreover, x k -x * exp(-k 2κ ) x 0 -x * for all k ∈ N.
Thus, this theorem provides a very efficient algorithm to compute arg min Ω f , and thus D , if we know the projection onto Ω = {x ∈ (R + ) n | n i=1 x i 1}. However, we don't know it in closed form. Hence, we have to approximate it and to show that this approximation preserves the previous theorem, i.e. that the new algorithm with approximated projection still converges, hopefully with logarithmic time complexity. That is the goal of the two next sections.

Approximate the projection

In this section, we define an approximation of the projection Π Ω : R n -→ Ω which is not generic but very specific to the space Ω = {x ∈ (R + ) n | n i=1 x i 1}. We consider the hypersurfaces

M α = ∂Ω α = {x ∈ (R + ) n | n i=1 x i = α} for α ∈ R + . Our method to project z ∈ R n \Ω onto Ω consists in defining a map F z : a ∈ R + -→ x(a) ∈ M α(a)
such that α(a) = n i=1 x i (a) increases with a. Thus, it suffices to do a line search on a > 0 to find the one such that α(a) = 1. This is illustrated on Figure 8.1.

The tangent hyperplane at

x ∈ M α is T x M α = ( 1 x ) ⊥ where 1 x = ( 1 x 1 , ..., 1 xn ). Thus, the orthogonal projection x ∈ M α of z / ∈ Ω onto Ω α = conv(M α ) satisfies x -z 1 x . Otherwise said, there exists a ∈ R + such that x -z = a 1 x . Given z / ∈ Ω and α ∈ R + , it is difficult to compute x ∈ M α and a ∈ R + such that x -z = a 1
x . On the contrary, given z / ∈ Ω and a ∈ R + , it is very easy to find α ∈ R + and x ∈ M α such that x -z = a 1 x . Indeed, x i satisfies the equation

x 2 i -z i x i -a = 0 so x i = 1 2 (z i + z 2 i + 4a) and α = n i=1 x i . Therefore, for z ∈ R n \Ω, we define the smooth map F z : a ∈ R + -→ x = (x 1 , ..., x n ) ∈ R n and the smooth increasing map P F z : a ∈ R + -→ n i=1 x i ∈ R + where x i = 1 2 (z i + z 2 i + 4a
) for all i ∈ {1, ..., n}. Thus, x := F z (a) and α := P F z (a) satisfy x -z = a 1

x and x ∈ M α . In other words, x = Π Ωα (z) or F z (a) = Π Ω P Fz (a) (z).

Theorem 8.31 (Approximate the projection) We use the above notations.

1. There exists a unique a * ∈ R + such that P F z (a * ) = 1. It satisfies F z (a * ) = Π Ω (z) and for all a a * , P F z (a) 1 and F z (a) ∈ Ω. 2. We define I 0 = [b 0 ; c 0 ] where b 0 = 0 and c 0 = 1 -min i z i > 0 so that P F z (b 0 ) < 1 and P F z (c 0 ) 1. We define recursively for all k ∈ N:

a k = b k + c k 2 , ( 8.18
)

I k+1 = [b k+1 ; c k+1 ] = ß [b k ; a k ] if P F z (a k ) 1 [a k , c k ] if P F z (a k ) < 1 ™ . (8.19) Then lim k→∞ a k = a * , lim k→∞ P F z (a k ) = 1 and lim k→∞ F z (a k ) = Π Ω (z). Moreover, |a k -a * | c 0 2 -k . The time complexity is k ε = 1 ln(2) ln( c 0 ε ) and the complexity is O(n ln( c 0 ε )).
Proof of Theorem 8.31 (Approximate the projection).

1. Note that lim

a→0 P F z (a) = n i=1 z i +|z i | 2 . It is null (thus inferior to 1) if one of the z i 's is non-positive. Otherwise if all z i > 0, it is equal to n i=1 z i < 1 since z /
∈ Ω. Thus, since P F z is continuous and increasing from lim a→0 P F z (a) < 1 to +∞, there exists a unique a * ∈ R + such that P F z (a * ) = 1. Therefore, F z (a * ) = Π Ω 1 (z) and for all a a * ,

P F z (a) P F z (a * ) = 1 so F z (a) = Π Ω P Fz (a) (z) ∈ Ω P Fz(a) ⊆ Ω.
2. We just proved that P F z (b 0 ) < 1. Without loss of generality, let us assume that min i z i = z 1 . We have

z 1 < 1 because n i=1 z i < 1. Hence c 0 = 1 -z 1 > 0 and z 1 + z 2 1 + 4c 0 = 1. Since for all i ∈ {1, ..., n}, z i + z 2 i + 4c 0 z 1 + z 2 1 + 4c 0 , we have P F z (c 0 )
1. Moreover, the decreasing sequence of segments (I k ) k∈N has its diameter tending to 0 so their intersection is a singleton.

Since P F z (a * ) = 1, a * ∈ I k for all k ∈ N so k∈N I k = {a * }. For all k ∈ N, |a k -a * | c k -b k = c 0
2 k so the sequence (a k ) converges to a * . By continuity, lim k→∞ P F z (a k ) = 1 and lim k→∞ F z (a k ) = Π Ω (z).

The time complexity k ε for precision ε is given by c 0 2 -kε = ε, i.e. k ε = 1 ln(2) ln( c 0 ε ). (We keep c 0 because c 0 = c 0 (z) will change at each step of the minimization algorithm since z will change.) At each step, we only compute P F z (a k ) which requires 4n + n -1 = 5n -1 = O(n) operations. Hence, the complexity is the product O(n ln( c 0 ε )).

Theorem 8.32

(Stopping criterion) ∀z ∈ R n \Ω, ∀α > 1, Π Ωα z -Π Ω z (α 1/n -1) Π Ω z . Proof. Let z ∈ R n \Ω. We denote x = Π Ω z and a ∈ R + such that x -z = a 1
x . Thus it is equivalent to prove that for all x ∈ M and a ∈ R + , Π Ωα (xa x ) -x (α 1/n -1) x . Hence, we define the smooth maps:

z : a ∈ R + -→ x - a x ∈ R n \Ω, y : a ∈ R + -→ Π Ωα (z(a)) ∈ M α , D : a ∈ R + -→ y(a) -x 2 = n i=1 (y i (a) -x i ) 2 ∈ R + .
We want to prove that for all a ∈ R + , D(a) (α 1/n -1) 2 x 2 . We proceed in two steps. Firstly, we prove that lim a→+∞ D(a) = (α 1/n -1) 2 x 2 . Secondly, we prove that D is increasing. The proofs of these two facts being quite technical, we put them in Section 11.7. This theorem states that given ε > 0, one can compute a k 's until 1 P F z (a k ) 1 + ε n to ensure that F z (a k ) ∈ Ω and F z (a k ) -Π Ω z ε Π Ω (z) . Thus we define the projection approximation as follows.

Definition 8.33 (Projection approximation) Let ε > 0. We define Π ε Ω : z ∈ R n \Ω -→ F z (a k ) ∈ Ω where k = min{i ∈ N|1 P F z (a i ) 1 + ε n }. Note that Π ε Ω (z) = Π Ω P Fz (a k ) (z).
Now we can use this projection approximation to modify the algorithm in Theorem 8.30 and prove the convergence of this new algorithm.

Approximated projection gradient descent

In this section, we define the new algorithm and we prove that it still converges at exponential rate. We recall that Ω

= {x ∈ (R + ) n | n i=1 x i = 1} and f : x ∈ R n -→ x Σx ∈ R where Σ ∈ Sym + (n).
We denote λ min , λ max the minimum and maximum eigenvalues of Σ.

Theorem 8.34 (Algorithm to compute D : convergence, speed, complexity) Let

x 0 = 1 ∈ Ω. We denote κ = λmax λ min and R = √ nκ. Let ε ∈ (0, 1). Let δ = ε 2 72nκ 2 . Let (u k ) k∈N ∈ (R + ) N is a decreasing sequence with ∞ k=0 u k 1, e.g. u k = 6 π 2 (k+1) 2 , and δ k = δu k . We define y k+1 = (I n -1 λmax Σ)x k and x k+1 = Π δ k+1
Ω (y k+1 ). Then:

1. (Convergence) (x k ) k∈N converges to x * = arg min Ω f .

(Speed of convergence)

The convergence is exponential:

x k -x * ε + exp(-k 2κ ) x 0 - x * .
3. (Time complexity) For all ε > 0, the sufficient number of iterations so that

x k -x * < 2ε is k ε = O(ln( n ε )).
4. (Complexity) The total complexity with precision 2ε is O n 2 ln n ε + n ln n ε ln ln n ε .

We recall that given x * , we then have

D (Σ) = 1 √ λ * diag(x * ) where λ * = 1 n (x * ) Σx * .
Proof. The proof of Theorem 8.30 strongly relies on the characterization of the projection onto a closed convex set (item 2 of Definition 8.29) [Bubeck, 2015]. Therefore, it is not obvious that replacing the projection by an approximation leads to the same convergence.

There is actually an extra term that appears and that we need to bound carefully. We proceed in three steps:

1. We bound all what should be bounded.

2. We define the extra term and we bound it.

3. We prove the convergence.

We need to introduce the theoretical sequences of Theorem 8.30 by x 0 0 = x 0 = 1, y 0 k+1 = (I n -1 λmax Σ)x 0 k and x 0 k+1 = Π Ω y 0 k+1 . We also introduce the sequences (a k ) k∈N , (α k ) k∈N and (x k ) k∈N defined by

x k = F y k (a k ), α k = P F y k (a k ) and xk = Π Ωα k y 0 k . Note that x k = Π Ωα k y k . First step: bounds Note that x 0 R since x 0 2 = n nκ. Let us show that x * R. We denote E Σ = {x ∈ R n |x Σx 1} the ellipsoid characterizing the SPD matrix Σ. We clearly have B(0, 1 √ λmax ) ⊆ E Σ ⊆ B(0, 1 √ λ min ), where B(0, d) = {x ∈ R n | x d} is the centered closed ball of radius d. Thus 1 2 √ nλ min E Σ ∩ Ω ⊆ B(0, √ n
2 ) ∩ Ω = ∅. In addition, r r if and only if rE Σ ⊆ r E Σ . Therefore, there exists (a unique) r * ∈ R + such that for all r ∈ R + , r < r * if and only if

rE Σ ∩ Ω = ∅. Note that rE Σ = f -1 ((0, r 2 ]). Since x * is the unique minimizer of f on Ω, we have r * = √ min Ω f and r * E Σ ∩ Ω = {x * }. Since 1 ∈ B(0, √ n) ∩ Ω = √ nλ max B(0, 1 √ λmax ) ∩ Ω ⊆ √ nλ max E Σ ∩ Ω, this proves that √ nλ max E Σ ∩ Ω is non-empty so √ nλ max r * . Hence x * ∈ r * E Σ ⊆ √ nλ max E Σ ⊆ B(0, R) so x * R.
From Theorem 8.30, we have

x 0 k -x * exp(-k 2κ ) x 0 0 -x * 2R so x 0 k x 0 k - x * + x * 3R and y 0 k+1 x 0 k 3R because the eigenvalues of I n -1 λmax Σ are in [0, 1).
Moreover, using Theorem 8.32 and the fact that Π Ωα k+1 is 1-Lipschitz, we have:

x 0 k+1 -x k+1 x 0 k+1 -xk+1 + xk+1 -x k+1 = Π Ω y 0 k+1 -Π Ωα k+1 y 0 k+1 + Π Ωα k+1 y 0 k+1 -Π Ωα k+1 y k+1 (α 1/n k+1 -1) x 0 k+1 + y 0 k+1 -y k+1 δ k+1 3R + x 0 k -x k = 3Rδu k+1 + x 0 k -x k , x 0 k -x k k-1 i=0 3Rδu i+1 3Rδ R, x k x k -x 0 k + x 0 k 4R, y k+1 x k 4R.
Second step: extra term. By Definition 8.29, we have (x 0 k+1 -y 0 k+1 ) (x 0 k+1 -x * ) 0. We would like to have an equality of the form (x k+1 -y k+1 ) (x k+1 -x * ) c ε where c ε = ε→0 o(1). We introduce x 0 k+1 and y 0 k+1 in this expression to use the inequality:

(x k+1 -y k+1 ) (x k+1 -x * ) = (x k+1 -x 0 k+1 + x 0 k+1 -y 0 k+1 + y 0 k+1 -y k+1 ) (x k+1 -x 0 k+1 + x 0 k+1 -x * ) (x k+1 -x 0 k+1 ) (x k+1 -y k+1 + x 0 k+1 -x * ) + (y 0 k+1 -y k+1 ) (x 0 k+1 -x * ) x k+1 -x 0 k+1 ( x k+1 + y k+1 + x 0 k+1 -x * ) + y 0 k+1 -y k+1 x 0 k+1 -x * x k+1 -x 0 k+1 ( x k+1 + y k+1 + 2 x 0 k+1 -x * ) 3Rδ(4R + 4R + 4R) = 36R 2 δ = ε 2 2κ =: c ε .
This implies an inequality that we use to prove the convergence and which is a modification of a similar inequality (namely with c ε = 0) from [Bubeck, 2015]. Note that y k+1 = (I n -

1 λmax Σ)x k = x k -1 β ∇f |x k since ∇f |x = 2Σx and β = 2λ max .
Hence the previous inequality rewrites:

(x k+1 -(x k - 1 β ∇f |x k )) (x k+1 -x * ) c ε ⇐⇒ ∇f |x k (x k+1 -x * ) βc ε + β(x k -x k+1 ) (x k+1 -x * ).
Third step: convergence. We use the fact that f is α-strongly convex and β-smooth with α = 2λ min and β = 2λ max as in [Bubeck, 2015]:

0 f (x k+1 ) -f (x * ) = f (x k+1 ) -f (x k ) + f (x k ) -f (x * ) ∇f |x k (x k+1 -x k ) + β 2 x k+1 -x k 2 + ∇f x k (x k -x * ) - α 2 x k -x * 2 = ∇f |x k (x k+1 -x * ) + β 2 x k+1 -x k 2 - α 2 x k -x * 2 βc ε + β(x k -x k+1 ) (x k+1 -x * ) + β 2 x k+1 -x k 2 - α 2 x k -x * 2 β ε 2 2κ + β(x k -x k+1 ) (x k -x * ) - β 2 x k+1 -x k 2 - α 2 x k -x * 2 , so: x k+1 -x k 2 -2(x k -x * ) (x k -x k+1 ) ε 2 κ - α β x k -x * 2 .
Hence:

x k+1 -x * 2 = x k+1 -x k + x k -x * 2 = x k -x * 2 + x k+1 -x k 2 -2(x k -x * ) (x k -x k+1 ) ε 2 κ + Å 1 - 1 κ ã x k -x * 2 , x k+1 -x * 2 -ε 2 Å 1 - 1 κ ã ( x k -x * 2 -ε 2 ). Let k 0 = min{k ∈ N| x k -x * ε}.
Then for all k k 0 :

x k -x * 2 -ε 2 Å 1 - 1 κ ã k ( x 0 -x * 2 -ε 2 ) x 0 -x * 2 exp Å - k κ ã ,
and this inequality is trivially true for

k k 0 since x k -x * -ε 0 x 0 -x * 2 exp(-k κ ). To conclude, x k -x * ε + x 0 -x * exp -k 2κ . Finally, the time complexity k ε is obtained by x 0 -x * exp(-kε 2κ ) = ε, i.e. k ε = 2κ ln( x 0 -x * ε ) 2κ ln( 2R ε ) = O(ln( n ε )
). At each step, there is a multiplication matrixvector (n 2 operations) and a projection approximation with complexity 5n-1 ln(2) ln( n δ j ) since (c 0 ) j = 1 -min i [y k ] i 1 + 4R = 1 + √ nκ (with the notation c 0 from Theorem 8.31). Therefore, the total complexity is bounded by the following quantity:

kε j=1 Å n 2 + n ln Å n δ j ãã n 2 k ε + nk ε ln Å n δu kε ã = O n 2 ln n ε + n ln n ε ln ln n ε .
From the proof, we can notice that the choice of δ ∝ ε 2 seems necessary to get an approximation of the result up to ε. Numerically, it means that if real numbers are implemented with precision 10 -16 , then the precision on D cannot exceed 10 -8 . Moreover, the other parameters n, κ and u k 's strengthen this observation. For example, if n = 100, κ = 10 and u 10 = 1 100 , then δ 10 < ε 2 × 10 -6 so ε > 10 -5 . However, the inequalities in the proof are far from optimal. It means that at the beginning of the algorithm, x k might be far from x k+1 and x * so the inequalities used in the third step are large. In addition, when x k becomes close to x * , the inequalities used in the first step are overestimated of a factor about 18. These observations can help gain in precision in practice.

The combination of this approach with the Newton method of [START_REF] Khachiyan | Diagonal Matrix Scaling and Linear Programming[END_REF] can also help gain in precision. Indeed, as soon as

∆ k Σ∆ k 1 -1 c < 1 (where ∆ k = 1 √ x k Σx k diag(x k ))
, the Newton method ensures that for all s ∈ N, ∆ k+s Σ∆ k+s 1 -1 c 2 s [START_REF] Khachiyan | Diagonal Matrix Scaling and Linear Programming[END_REF]] so one could reach any precision ε in an additional O(ln(ln( 1 ε ))) of steps. The total complexity becomes O(n 2 ln(n ln( 1 ε ))) since each step of Newton requires to solve a linear system which is a O(n 2 ). Although the complexity is worse, the precision could be better. However, this is conditioned to the possibility to reach ∆ k Σ∆ k 1 -1 < 1 with our method. One can show that ∆ k Σ∆ k 1 -1 2κR x k -x * so one has to choose ε < 1 8κR . With the values of the previous paragraph, 1 8κR = 5×10 -4 which is greater than 10 -5 so one can actually reach any precision ε by plugging Khachiyan's method [START_REF] Khachiyan | Diagonal Matrix Scaling and Linear Programming[END_REF] after ours, with a better complexity than Khachiyan's. Note that the precision ε is limited by the resolution of the linear system (∆ -2 k+s +Σ)h = -Σ∆ k+s 1 where h ∈ R n is unknown, the next step being ∆ k+s+1 = ∆ k+s + diag(h).

To compare with algorithms for which the complexity depends on the number of nonzero entries m n 2 of Σ, we can notice that the multiplication matrix-vector at each step of the algorithm only requires m operations. Therefore, the complexity of our algorithm is actually O m ln( n ε ) + n ln( n ε ) ln ln( n ε ) . In comparison with the generic algorithm with complexity O(m + n 4/3 ) of [START_REF] Allen-Zhu | Much faster algorithms for matrix scaling[END_REF], it is competitive for sparse SPD matrices with m < n 4/3 .

Conclusion

In this work, we introduced two families of permutation-invariant log-Euclidean metrics on full-rank correlation matrices. The first family was built via the off-log diffeomorphism whose algebraic properties were introduced in [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF]. The second family was built via the log-scaling diffeomorphism thanks to the result on the scaling of symmetric matrices [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF]]. In addition, the log-scaled metrics are inverse-consistent contrarily to the off-log metrics. These metrics allow to compute with correlation matrices very efficiently since they are flat and the main Riemannian operations are known in closed form modulo the computation of the maps D and D . That is why we also provide efficient algorithms to compute them. These permutation-invariant log-Euclidean metrics fill a gap in the study of Riemannian metrics on the open elliptope since they gather the invariance under permutations of some structures [Kercheval, 2008, Nielsen and Sun, 2019, David and Gu, 2019, Thanwerdas and Pennec, 2021] and the log-Euclidean geometry of some others [START_REF] Thanwerdas | Theoretically and computationally convenient geometries on full-rank correlation matrices[END_REF]. More generally, we tried to introduce a comprehensive formalism on full-rank correlation matrices in terms of stability under the action of permutations and under the cor-inverse involution. This systematic approach allowed to satisfy intrinsically the requirement of inverse-consistency in the construction of log-Euclidean metrics. We hope that this presentation will help to manipulate correlation matrices as easily as we work with SPD matrices. This approach could also help to define appropriate geometries of block equi-correlation matrices of a given signature.

Part V

Stratified spaces of covariance and correlation matrices 9.1 Introduction

Many data can be represented as covariance matrices. They are often assumed to be Symmetric Positive Definite (SPD) because it is much more convenient from the geometric point of view. Indeed, the set of SPD matrices is an open convex cone in the vector space of symmetric matrices so it has a canonical differential structure. The induced Euclidean metric is not satisfying to compute with SPD matrices because geodesics leave the space in finite time and interpolations are often non-realistic. To solve this problem, a lot of Riemannian metrics were proposed on SPD matrices, mainly O(n)-invariant metrics [START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF] (affine-invariant [Siegel, 1943, Skovgaard, 1984, Pennec et al., 2006, Lenglet et al., 2006, Fletcher and Joshi, 2007, Moakher, 2005], log-Euclidean [START_REF] Arsigny | Log-Euclidean metrics for fast and simple calculus on diffusion tensors[END_REF], Fillard et al., 2007, Hà Quang et al., 2014], Bures-Wasserstein [Dowson and Landau, 1982, Olkin and Pukelsheim, 1982, Takatsu, 2010, Takatsu, 2011, Malagò et al., 2018, Bhatia et al., 2019], Bogoliubov-Kubo-Mori [Petz andToth, 1993, Michor et al., 2000], etc.), Cholesky-like metrics [START_REF] Li | Local Log-Euclidean Multivariate Gaussian Descriptor and Its Application to Image Classification[END_REF], Lin, 2019] or product metrics with one metric on positive diagonal matrices and one metric on full-rank correlation matrices [Thanwerdas andPennec, 2021, Thanwerdas and[START_REF] Thanwerdas | Theoretically and computationally convenient geometries on full-rank correlation matrices[END_REF]]. However, this viewpoint often forgets about singular covariance matrices, that is covariance matrices with non-full rank. Altogether, they form a closed convex cone which is not anymore a manifold. First, it can be equipped with distances to provide a metric space structure. The Euclidean distance is not satisfying either here because geodesics leave the closed cone in finite time again. A very interesting alternative is the Helstrom/Bures distance defined in quantum information geometry [Helstrom, 1967, Bures, 1969]. It was also introduced in optimal transport as the L 2 Wasserstein/Kantorovitch distance between multivariate centered Gaussian distributions, possibly degenerate [START_REF] Dowson | [END_REF]Landau, 1982, Olkin andPukelsheim, 1982]. This is why it is now called the Bures-Wasserstein distance. From our viewpoint, it allows to define a distance on the whole set of covariance matrices, contrarily to the aforementioned affine-invariant and log-Euclidean metrics. The Bures-Wasserstein distance is also the quotient distance of the Euclidean metric on square matrices by the right action of the orthogonal group (U,

M ) ∈ O(n) × R n×n -→ M U ∈ R n×n
. This is why it was also called the Procrustes distance [START_REF] Dryden | Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging[END_REF], Hà Quang, 2022]. This viewpoint allows to split the closed cone into strata that are Riemannian manifolds whose induced geodesic distance is precisely the Bures-Wasserstein distance. In particular, the space of covariance matrices equipped with this distance is a complete geodesic metric space.

The geometry of stratified spaces is a topic of interest in the community of statistics in non-linear spaces. Examples of popular stratified spaces are the Kendall shape spaces [Kendall, 1984], the BHV space of trees [START_REF] Billera | Geometry of the Space of Phylogenetic Trees[END_REF], the QED space of trees [START_REF] Feragen | Geometries on spaces of treelike shapes[END_REF], the Graph space [START_REF] Calissano | Populations of Unlabeled Networks: Graph Space Geometry and Geodesic Principal Components[END_REF], the Wald space of forests [START_REF] Garba | Information geometry for phylogenetic trees[END_REF], the correlation matrices or the symmetric/diagonal matrices stratified by eigenvalue multiplicity. Moreover, the space of covariance matrices with the Bures-Wasserstein distance is a metric space of non-negative curvature [Takatsu, 2011]. Spaces of this type have been much less described than metric spaces of non-positive curvature [START_REF] Bridson | Metric Spaces of Non-Positive Curvature[END_REF]. These are two motivations to study the Bures-Wasserstein geometry of covariance matrices. In this work, we focus on geodesics.

There are three new elements with respect to the previous chapters that we would like to highlight. Firstly, most of the previous Riemannian manifolds were geodesically complete. In this chapter, we need the notion of definition domain of the exponential map because it is not R in general. Secondly, the geodesics were minimizing on R most of the time. Here, we need the notions of cut time and injectivity domain to specify when the geodesic stops to be minimizing. Thirdly, we talk about geodesics in a metric space, not only in a Riemannian manifold as previously.

In the principal (or regular) stratum of SPD matrices, the Bures-Wasserstein Riemannian metric was extensively studied. Since X ∈ GL(n) -→ XX ∈ Sym + (n) is a Riemannian submersion, many geometric operations can be computed thanks to O'Neill's equations [O'Neill, 1966]. Therefore, the curvature was derived in [Takatsu, 2010, Takatsu, 2011[START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF], the quotient geometry was described in [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF], Bhatia et al., 2019, van Oostrum, 2020], the exponential map was computed in [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF], a Riemannian logarithm was given in [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF], the injectivity radius was computed in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF] and a simplified equation of the geodesic parallel transport was proposed in [START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]. In contrast to geodesically complete Riemannian metrics, it is important to specify the definition domain of the exponential map. It was characterized in [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF] as the connected component of 0 in a subset of R, which could be specified more explicitly. Moreover, the uniqueness of the Riemannian logarithm is not established and the injectivity domain seems to be unknown. Therefore in Section 9.4, we clarify the definition domain of the exponential map, we prove the uniqueness of the logarithm thanks to a result from [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF] and we prove that the geodesics are minimizing on their domain of definition, which also provides the injectivity domain.

In each other stratum of PSD matrices of fixed rank k < n, the Bures-Wasserstein Riemannian metric was studied via the analogous Riemannian submersion defined by

X ∈ R n×k * -→ XX ∈ Sym + (n, k), where R n×k *
is the open set of matrices of full rank k in R n×k and Sym + (n, k) is the set of PSD matrices of size n and rank k. The curvature was computed in [START_REF] Massart | Curvature of the Manifold of Fixed-Rank Positive-Semidefinite Matrices Endowed with the Bures-Wasserstein Metric[END_REF], the exponential map, its domain of definition, the logarithm map and the injectivity radius were derived in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF]. The horizontal lift was kept implicit so these results are formulated in the total space R n×k * . We think that it is easier to understand the geometry with formulae depending directly on the tangent vector and not on its horizontal lift. Therefore, in Section 9.5, we compute the horizontal lift and we give the expressions of the Riemannian metric, the exponential map and its definition domain in function of vectors tangent to the manifold Sym + (n, k). Moreover, we solve the problem of characterization of the preimages of the exponential map, which is knowingly left open in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF] where they focus on the characterization of logarithms, that is preimages with minimal norm. In addition, we give an explicit bijective parametrization of the Riemannian logarithms, which allows us to count them. When it is unique, we give an explicit formula of the corresponding minimizing geodesic in function of the end points. This finally allows us to compute the injectivity domain which is kept implicit in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF].

Beyond the clarification and completion of the literature on geodesics in each stratum, our main objective is to characterize the minimizing geodesics between strata. Our main results are the following. (1) Any minimizing geodesic segment between two covariance matrices Σ and Λ is of constant rank on the interior of the segment. It is called the rank of the minimizing geodesic and it is greater than the ranks of Σ and Λ. (2) We give the explicit formula of all the minimizing geodesic segments in Theorem 9.31. (3) They are parametrized by the vectors of the closed unit ball of R (k-r)×(l-r) for the spectral norm, where k, l, r are the respective ranks of Σ, Λ, ΣΛ. In other words, they are parametrized by matrices R 0 ∈ R (k-r)×(l-r) with singular values in [0, 1]. ( 4) The minimizing geodesic segment is unique if and only if r = min(k, l) (this includes max(k, l) = n). Otherwise, there are infinitely many. (5) The number of minimizing geodesics of minimal rank (i.e. of rank equal to max

(k, l)) is 1 if r = min(k, l), otherwise it is 2 if k = l, otherwise it is infinite. (6) Assuming k l, if R 0 belongs to the Stiefel manifold St(k -r, l -r), that is R 0 R 0 = I l-r ,
then the corresponding geodesic is of minimal rank. (7) The choice of parameter R 0 = 0 leads to the geodesic

γ 0 Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t)sym(Σ 1/2 ((Σ 1/2 ΛΣ 1/2 ) 1/2 ) † Σ 1/2 Λ
) whose expression does not depend on the ranks of Σ and Λ. It is called the Bures-Wasserstein canonical geodesic.

In the remainder of this section, we introduce some matrix notations. In Section 9.2, we introduce the important concepts of geodesics and quotient space in metric spaces and manifolds. We give a particular attention to the notions that characterize if a geodesic (selfparallel curve) is minimizing: cut time, injectivity domain, difference between preimages of the exponential map and Riemannian logarithms. In Section 9.3, we recall the algebraic structure, metric topology and differential geometry of the convex cone of covariance matrices seen as the quotient of square matrices by the orthogonal group. In Sections 9.4 and 9.5, we complete the literature on the geodesics of the Bures-Wasserstein metric on SPD matrices and on singular matrices of fixed rank respectively. In Section 9.6, we give our main results on minimizing geodesics in the whole Bures-Wasserstein metric space of covariance matrices. We conclude in Section 9.7 and we give a glance at an analogous structure on correlation matrices. The main proofs are deferred to Section 11.8.

Matrix notations

Let n, k ∈ N. In this work, we use the following manifolds of matrices:

• the vector space of n × k matrices R n×k , • the open subset R n×k * ⊂ R n×k of full-rank matrices,
• in particular, the vector space of square matrices Mat(n) = R n×n and the general linear group GL(n) = R n×n * ,

• the orthogonal group O(n),

• the Stiefel manifold St(n, k) = O(n)/O(n -k),
• the manifold of symmetric positive definite matrices Sym + (n),

• the manifold of symmetric positive semi-definite matrices of fixed rank k, Sym + (n, k),

• the vector space of diagonal matrices Diag(n),

• the groups of invertible diagonal matrices Diag * (n) = Diag(n) ∩ GL(n) and positive diagonal matrices Diag

+ (n) = Diag(n) ∩ Sym + (n).
We use the following notations.

• I n denotes the identity matrix of size n.

• 0 n denotes the null matrix of size n × n. 0 n,k denotes the null matrix of size n × k. We may simply denote them 0 when sizes are obvious in the context.

• (Sylvester equation) S A (B) is the unique solution Z of the Sylvester equation AZ + ZA = B for A ∈ Sym + (k) and B ∈ Sym(k).
• (Löwner order) For all Σ ∈ Sym(n), we say that Σ is positive definite (resp. positive semi-definite) and we denote Σ > 0 (resp. Σ 0) when Σ has positive (resp. nonnegative) eigenvalues. Given Λ ∈ Sym(n), we denote Σ > Λ (resp. Σ Λ) when Σ -Λ > 0 (resp. Σ -Λ 0).

We recall basic facts on symmetric matrices.

1. (Eigenvalue decomposition) For all Σ ∈ Sym(n), there exist U ∈ O(n) and D ∈ Diag(n) such that Σ = U DU . By removing null eigenvalues, given r = rk(Σ), there also exist U ∈ St(n, r) and D ∈ Diag * (r) such that Σ = U DU .

2. For all X ∈ R n×k , XX is symmetric positive semi-definite and rk(X) = rk(XX ) = rk(X X). In particular, if X ∈ R n×k * , then X X ∈ GL(k).

(Singular value decomposition

) For all M ∈ R n×k , denoting r = rk(M ) min(n, k),

there exist U ∈ O(n), V ∈ O(k) and D = Å D r 0 r,k-r 0 n-r,r 0 n-r,k-r ã with D r ∈ Diag + (r) such that M = U DV .
The diagonal entries of D, that is the D ii s for i ∈ {1, ..., min(n, k)} are called the singular values of M .

4. (Moore-Penrose inverse) For all M ∈ R n×k , the unique matrix

M † ∈ R k×n satisfying M M † M = M , M † M M † = M † , M M † ∈ Sym(n) and M † M ∈ Sym(k) is
∈ St(n, r), P = [U U ⊥ ] ∈ O(n), D ∈ Diag * (r), it is easy to check that Σ † = U D -1 U = P Diag(D -1 , 0)P . Moreover, for all V ∈ St(n, k), (V ΣV ) -= V Σ -V .
5. (Symmetric square root) For all Σ ∈ Sym(n), if Σ 0, then there exists a unique matrix A ∈ Sym(n), A 0 such that A 2 = Σ. It is called the (symmetric) square root of Σ and it is denoted Groetzner and Dür, 2020, Lemma 2.6].

√ Σ or Σ 1/2 . 6. (Non-symmetric square roots) For all X, Y ∈ R n×k , XX = Y Y if and only if there exists U ∈ O(k) such that XU = Y [
We use the following notations for norms of vectors x ∈ R k and matrices M ∈ R n×k :

1. (Euclidean norm) M 2 = tr(M M ) 1/2 and x 2 = (x x) 1/2 .

(Spectral norm or Schatten's infinite norm) M

S = sup x∈R k x 2 1 M x 2 = max 1 i n σ i (M ),
where σ 1 (M ), ..., σ n (M ) are the singular values of M .

Without index, • generically denotes the norm on the tangent spaces associated to the Riemannian metric at hand.

Preliminary concepts

In this section, we recall the basic definitions of geodesics and quotient spaces. We depart from the previous chapters where most of the Riemannian metrics were geodesically complete and where the injectivity domain of the exponential map was the entire manifold. We also need to introduce the notions of length and geodesic in a metric space [Bridson andHaefliger, 1999, Paulin, 2014] and the notion of Riemannian orbit space [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF]. In Section 9.2.1, we introduce the concepts needed to study geodesics and minimizing geodesics. We recall that geodesics are defined in a metric space, in a manifold endowed with an affine connection and in a Riemannian manifold, where the two previous notions coincide. Then we introduce our notations for preimages of the exponential map, logarithms and related notions. We define the cut time and the injectivity domain, extending the definition usually given in complete metric spaces, and we explain where to be cautious. In Section 9.2.2, we recall the notions of quotient metric space, quotient Riemannian manifold and Riemannian orbit space. We introduce the vocabulary, notations and results that we use in the next sections.

Geodesics

Definition 9.1 (Curve) Let M be a topological space. A curve on M is a continuous map c : I -→ M, where I is an interval of R. When I is a segment of R, we may call c a segment. 9.2.1.1 Geodesics in a metric space Definition 9.2 (Length, length distance, length space) [Paulin, 2014] Let (M, d) be a metric space.

1. (Length) Let c : [a, b] ⊆ I -→ M be a curve. The length of c is defined by L(c) = sup p k=0 d(c(t k ), c(t k+1 )) ∈ [0, +∞] over all subdivisions a = t 0 t 1 ... t p t p+1 = b. We say that c : I -→ M is rectifiable when for all a b in I, L(c |[a,b]
) is finite.

(Length distance)

The length distance between x ∈ M and y ∈ M is defined by

d L (x, y) = inf L(c) d(x, y) over all rectifiable curves c : [0, 1] -→ M from x = c(0) to y = c(1). If M is connected by rectifiable curves, then d L is a distance. 3. (Length space) We say that (M, d) is a length space when d = d L .
Lemma 9.3 (Length is additive and continuous) [Bridson and Haefliger, 1999, Proposition 1.20] Let c : [a, b] -→ M be a rectifiable curve.

1. For all t ∈ [a, b], L(c) = L(c |[a,t] ) + L(c |[t,b] ). 2. The map f : t ∈ [a, b] -→ L(c |[a,t]
) is non-decreasing and continuous.

A classical example is the sphere S 2 ⊂ R 3 endowed with the Euclidean distance d. The distances between the north pole N = (0, 0, 1) and the south pole S = (0, 0, -1) are d(N, S) = 2 and d L (N, S) = π. The metric space (S 2 , d) is not a length space while the metric space (S 2 , d L ) is a length space. Definition 9.4 (Geodesics in a metric space) [Paulin, 2014] Let (M, d) be a metric space.

1. (Constant speed) We say that c : I -→ M is a curve parameterized at constant speed when there exists v 0 such that for all t t in I, L(c

|[t,t ] ) = v(t -t).
2. (Unit speed) We say that c : I -→ M is a curve parameterized at unit speed (or by arc length) when for all t t in I, L(c

|[t,t ] ) = t -t.

(Locally minimizing)

We say that c : I -→ M is locally minimizing when for all t ∈ I, there exists a neighborhood I 0 ⊆ I of t such that c |I 0 is minimizing.

(Geodesic)

A geodesic is a locally minimizing curve of constant speed. A minimizing geodesic is a globally minimizing curve of constant speed.

6. (Geodesic space) A geodesic (metric) space is a length space such that there exists a minimizing geodesic between any two points.

Geodesics of an affine connection

In the following notions, the definition interval and the definition domain are of interest when the manifold is not geodesically complete and the set of preimages is of interest when the exponential map is not injective.

Definition 9.5 (Geodesics of an affine connection) [Paulin, 2014] Let (M, ∇) be a smooth manifold equipped with an affine connection ∇ : Γ(T M) × Γ(T M) -→ Γ(T M).

1. (Geodesic) A geodesic (or self-parallel curve) is a solution γ : I -→ M of the secondorder equation ∀t ∈ I, ∇ γ(t) γ = 0. The maximal solution satisfying the initial condition γ(0 v) . It is the maximal interval of R on which the geodesic γ (x,v) is defined.

) = v for v ∈ T x M is denoted γ (x,v) : I x,v -→ M. 2. (Definition interval) We call I x,v ⊆ R the definition interval of γ (x,
In a geodesically complete manifold, I x,v = R.

(Exponential map) The exponential map is defined by Exp

: v -→ γ (x,v) (1) on the open set x∈M {v ∈ T x M| 1 ∈ I x,v } ⊆ T M. The exponential map at x, defined by Exp x (v) = Exp(v) for v ∈ T x M such that 1 ∈ I x,v
, is a diffeomorphism from a neighborhood of 0 in T x M to a neighborhood of x in M.

(Definition domain)

The definition domain of the exponential map if

D x = {v ∈ T x M| 1 ∈ I x,v }. 5. (Preimage) A preimage of y ∈ M by Exp x is a vector v ∈ T x M such that 1 ∈ I x,v and Exp x (v) = y. The set of preimages of y from x is denoted Pre x (y) = Exp -1 x ({y}) = {v ∈ T x M| 1 ∈ I x,v and Exp x (v) = y}.
Note that there might be none, one, several or infinitely many elements in Pre x (y).

6. (Geodesic from x to y) A geodesic from x to y is a geodesic γ (x,v) such that v ∈ Pre x (y). They are bijectively indexed by Pre x (y) so we denote them

γ v x→y = γ (x,v) for v ∈ Pre x (y).
When there exists a unique preimage of y from x, we simply denote the geodesic γ x→y .

Geodesics of a Riemannian metric

Any Riemannian manifold is equipped with a natural affine connection called the Levi-Civita connection. Moreover, we recall in the following definition that a connected Riemannian manifold is a metric space and even a length space. Definition 9.6 (Length, Riemannian distance) [Paulin, 2014] Let (M, g) be a Riemannian manifold. For v ∈ T M, we denote its norm v = g (v, v).

1. (Length) The length of a C 1 curve c : [a, b] -→ M is defined by L(c) = 1 0 ċ(t)
dt. This definition extends to piecewise C 1 curves. The length is independent from the parametrization of the curve.

(Distance)

The Riemannian distance between x, y ∈ M is defined by d(x, y) = inf L(c) over all piecewise C 1 curves c : [0, 1] -→ M from x = c(0) to y = c(1). If M is connected, then the Riemannian distance is a distance (defining the topology of M), the Riemannian length and the metric length coincide on piecewise C 1 curves, and (M, d) is a length space.

Fortunately, the two notions of geodesics coincide: a piecewise C 1 curve is a geodesic for the Levi-Civita connection if and only if it is a geodesic in the metric space (M, d). Then, it is natural to ask what are the globally minimizing geodesics. A few concepts can be introduced to formalize this question: injectivity radius, cut time, cut locus, injectivity domain. Moreover, among the preimages v ∈ Pre x (y) of the exponential map of the Levi-Civita connection, those which satisfy v = d(x, y) are called Riemannian logarithms. When there exists a unique Riemannian logarithm, the logarithm map can be defined. Definition 9.7 (Geodesics of a Riemannian metric) Let (M, g) be a connected Riemannian manifold. Let d be the Riemannian distance and ∇ be the Levi-Civita connection. We denote

B x (ε) = {v ∈ T x M| v < ε} ⊂ T x M the centered open ball of T x M of radius ε > 0.
1. (Geodesics) A curve γ : I -→ M is called a geodesic of (M, g) if one of the two following equivalent statements is satisfied:

(a) γ is a geodesic of (M, d) (locally length-minimizing curve of constant speed), (b) γ is a geodesic of (M, ∇) (self-parallel curve).

The equivalence is shown in [Paulin, 2014, Proposition 3.14] for example.

2. (Injectivity radius) [Paulin, 2014] The injectivity radius at x ∈ M is defined by inj

(x) = sup ε over all ε > 0 such that exp x is a diffeomorphism from B x (ε) ⊂ T x M to its image. The injectivity ball is B x (inj(x)) ⊂ T x M. For v ∈ T x M of norm 1 and t 0 ∈ [0, inj(x)), the map γ (x,v) : t ∈ [0, t 0 ] -→ exp x (tv) ∈ M
is the unique geodesic between x and γ (x,v) (t 0 ). It is globally minimizing.

The injectivity radius of M is defined by inj(M) = inf x∈M inj(x).

(Cut time) The cut time at

x ∈ M in the direction v ∈ T x M, v = 1, is defined by t cut (x, v) = sup{t ∈ I x,v |d(x, Exp x (tv)) = t} ∈ (0, +∞]. Note that the geodesic γ (x,v)
need not be minimizing on (-t cut (x, -v), t cut (x, v)).

(Tangential cut locus)

The tangential cut locus at x is the set

T CL(x) = {t cut (x, v)v| v ∈ T x M, v = 1, t cut (x, v) < +∞}.

(Injectivity domain)

The injectivity domain of the exponential map at x is the set

Inj(x) = {tv|t ∈ [0, t cut (x, v)), v ∈ T x M, v = 1} ⊆ D x .
The injectivity ball is included in the injectivity domain.

6. (Logarithms) We call (Riemannian) logarithm of y ∈ M from x ∈ M a preimage v ∈ Pre x (y) ⊆ T x M of y from x by the exponential map such that v = d(x, y). We denote Log x (y) ⊆ Pre x (y) the set of logarithms of y from x. In particular, the geodesic γ v x→y joining x to y with initial speed v ∈ Log x is minimizing on [0, 1].

7. (Logarithm map) Denoting U x ⊆ M the subset of points y such that there exists a unique logarithm of y from x, this defines a map Log x : U x -→ T x M. In particular, B x (inj(x)) ⊆ U x and Log x : Exp

x (B x (ε)) -→ B x (ε) is a diffeomorphism for all ε < inj(x).
Remark 9.8 The cut time is usually defined in complete manifolds only [do Carmo, 1992].

Although the definition still holds, some basic results may fail in non-complete manifolds. For example, if

I x,v = R and t cut (x, v) < +∞, then t cut (x, v) need not belong to I x,v and Exp x (t cut (x, v)v)
need not be defined. Then the cut locus, which is the image of the tangential cut locus by the exponential map in complete manifolds, should be defined differently. One possible definition could simply forget the vectors v ∈ T x M such that t cut (x, v) / ∈ I x,v as well as the definition in complete manifolds forgets about the vectors v ∈ T x M such that t cut (x, v) = +∞. A maybe more satisfying definition could rely on the metric completion of the space: the cut point in direction v could be the limit of Exp x (tv) when t tends to t cut (x, v) if this limit exists.

Studying the geodesics of a space means (at least) determining precisely an expression of the geodesic γ (x,v) , the maximal domain I x,v , the injectivity radiuses inj(x) and inj(M), the cut time t cut (x, v), the preimages of the exponential map Pre x (y), an expression of the geodesics γ v x→y , the Riemannian logarithms Log x (y), the logarithm map Log x and its definition domain U x .

The goal of Sections 9.4 and 9.5 is to clarify and complete the knowledge on geodesics of the Bures-Wasserstein metric on the manifolds Sym + (n) and Sym + (n, k). 9.2.2 Quotient spaces 9.2.2.1 Quotient distance Definition 9.9 (Quotient distance) Let (M, d) be a metric space. Let G be a group acting isometrically on (M, d). We denote M 0 = M/G and we define the map

d 0 : (Gx, Gy) ∈ M 0 × M 0 -→ d(Gx, Gy) = inf g∈G d(gx, y) ∈ [0, +∞).
If the orbits are closed, then d 0 is a distance on M 0 called the quotient distance.

Remark 9.10 There exists a more general notion of quotient pseudo-distance when the action is not isometric. In this chapter, the actions are isometric so we don't detail it.

Remark 9.11

The hypotheses of the definition are satisfied if the following conditions hold together:

• G is a locally compact topological group (e.g. a Lie group),

• (M, d) is a metric space (e.g. a Riemannian manifold),

• the action of G on (M, d) is continuous and isometric,

• the action is proper, i.e. (with the previous assumptions [Bourbaki, 1971, III.4.4 Proposition 7]) for all x, y ∈ M, there exist respective neighborhoods V x , V y such that the set K = {g ∈ G|gV x ∩ V y = ∅} is relatively compact.

Indeed, let (g n x) n∈N be a sequence in Gx tending to y ∈ M. Then there exists n 0 ∈ N such that for all n n 0 , g n x ∈ V y so g n V x ∩ V y = ∅, i.e. g n ∈ K. Since M is a metric space and K is relatively compact, there exists a subsequence (g ϕ(n) ) converging to g ∈ G. Since the action is continuous, g ϕ(n) x converges to gx. Therefore, y = gx ∈ Gx and Gx is sequentially closed, hence closed.

Definition 9.12 (Registered points) We say that x, y ∈ M are registered points when d(x, y) = d 0 (Gx, Gy).

Lemma 9.13 (Length in a quotient metric space) We denote L the length on both M and M 0 = M/G, and π : M -→ M 0 the canonical projection. For all curve c : [0, 1] -→ M, we have L(π • c) L(c). In particular, if x = c(0) and y = c(1) are registered and if c is minimizing, then π • c is minimizing and

L(π • c) = L(c) = d(x, y) = d 0 (π(x), π(y)). Proof. By definition L(π•c) sup p k=0 d 0 (π(c(t k )), π(c(t k+1 ))) sup p k=0 d(c(t k ), c(t k+1 )) = L(c). If x = c(0) and y = c(1) are registered, then d(x, y) = d 0 (π(x), π(y)) L(π • c). If c is minimizing, then d(x, y) = L(c) L(π • c). Thus if both hold, then L(π • c) = L(c) = d(x, y) = d 0 (π(x), π(y)) and π • c is minimizing.

Quotient Riemannian metric

Definition 9.14 (Quotient metric) Let (M, g) be a Riemannian manifold. Let G be a Lie group acting smoothly, properly, freely and isometrically on (M, g). Then there exists a unique smooth structure on M 0 = M/G such that the quotient map π : M -→ M 0 is a submersion [Lee, 2012]. Thus, one can define for all x ∈ M:

1. (Vertical space) V x = T x M x = ker d x π where M x = π -1 (x) is a submanifold of M, 2. (Horizontal space) H x = V ⊥ x so that T x M = V x ⊕ H x , 3. (Horizontal lift) # x : T π(x) M 0 -→ H x the inverse isomorphism of (d x π) |Hx : H x -→ T π(x) M 0 , 4. (Quotient metric) g 0 π(x) : (v, w) ∈ T π(x) M 0 × T π(x) M 0 -→ g x (v # x , w # x ) ∈ R
. This is a particular case of a Riemannian submersion [O'Neill, 1966]. The Riemannian distance of the quotient metric g 0 is the quotient distance of d, defined by d 0 (π(x), π(y)) = d(Gx, Gy) = inf g∈G d(gx, y).

The fundamental theorem on geodesics of a quotient metric is the following. Theorem 9.15 (Geodesics of a quotient metric) [O'Neill, 1966] The projection of a horizontal geodesic is a horizontal geodesic and their lengths coincide on any segment. More precisely, let x ∈ M and v ∈ T π(x) M 0 . Then

I x,v # x ⊆ I π(x),v and for all t ∈ I x,v # x , Exp π(x) (tv) = π(Exp x (tv # x )).

Riemannian orbit spaces

When the action of G on M is not free, the orbit space M/G is not a Riemannian manifold in general. This was studied in [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF]. We briefly recall the main facts that we use in this chapter.

Theorem 9.16 (Riemannian geometry of orbit spaces) [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF] Let (M, g) be a connected complete Riemannian manifold. Let G be a Lie group acting smoothly, properly and isometrically on (M, g). We denote π : M -→ M/G the canonical surjection.

For a Lie subgroup H of G, we denote (H) = {gHg -1 |g ∈ G} the conjugacy class of H and M (H) the set of points x ∈ M such that the stabilizer of x,

Stab(x) = {g ∈ G|gx = x}, belongs to (H). 1. M (H) is a smooth submanifold of M. 2. We denote (M/G) (H) = π(M (H) ) = M (H) /G the (isotropy) stratum of type (H). Then π (H) := π |M (H) : M (H) -→ (M/G) (H)
is a smooth fiber bundle with fiber type G/H.

3. The isotropy strata form a partition of M/G.

4. (M/G, d 0 ) is a complete metric space and a length space.

The definition of the vertical space still holds while the horizontal space is replaced by the normal space [Michor, 2008, VI.29.2]. Definition 9.17 (Vertical space, normal space) We take the notations of Theorem 9.16. Let

x ∈ M (H) . Thus M x = π -1 (π(x)) is a submanifold of M (H) . 1. (Vertical space) V x = T x M x ⊆ T x M (H) ⊆ T x M. 2. (Normal space) N x = V ⊥ x ⊆ T x M so that T x M = V x ⊕ N x .
Note that V x and N x need not have a constant dimension. We recall a result that we use later.

Lemma 9.18 (Geodesics in a Riemannian orbit space) [Alekseevsky et al., 2001, Lemma 3.5] We take the notation of Theorem 9.16. Given Lie subgroups

H 1 , H 2 of G, we denote (H 1 ) (H 2 ) if H 1 is conjugated so a subgroup of (H 2 ). Let γ : [0, 1] -→ M/G be a minimizing curve. For t ∈ [0, 1], let (M/G) (Ht) denote the stratum of γ(t).
Then, for all t ∈ (0, 1), (H t ) (H 0 ) and (H t ) (H 1 ).

We are now well prepared to study the construction of the orbit space of covariance matrices (Section 9.3), the geodesics within each stratum (Sections 9.4 and 9.5) and the minimizing geodesics in the whole space (Section 9.6).

Bures-Wasserstein geometry of covariance matrices

We denote the set of symmetric positive semi-definite matrices or covariance matrices by Cov

(n) = {Σ ∈ Sym(n)|Σ 0} = {XX |X ∈ Mat(n)} ⊂ Mat(n).
It is a complete metric subspace of the vector space of n × n square matrices equipped with the Euclidean distance

d E (Σ, Λ) = Σ -Λ 2 = tr((Σ -Λ) 2 ) 1/2 .
In this section, we recall that this set can also be described as the orbit space of the manifold Mat(n) of n × n matrices quotiented on the right by the orthogonal group O(n). We recall that the quotient topology coincides with the Euclidean topology and we recall the expression of the quotient distance known as the Bures-Wasserstein distance. We insist on the definition of the strata, which are investigated in the following sections.

The quotient geometry of covariance matrices

The group action of the orthogonal group O(n) on the vector space of square matrices Mat

(n) is (X, U ) ∈ Mat(n) × O(n) -→ XU ∈ Mat(n).
It is smooth, proper and isometric for the Euclidean distance.

The stabilizer (or isotropy group) of a matrix

X ∈ Mat(n) is Stab(X) = {U ∈ O(n)|XU = X}. If we denote k = rk(X), it is well known that X is equivalent to the matrix J k = Å I k 0 0 0 ã , i.e. there exist P ∈ GL(n) and Q ∈ O(n) such that X = P J k Q. Then it is clear that H k := Stab(J k ) = { Å I k 0 0 U ã |U ∈ O(n -k)} and Stab(X) = Q Stab(J k ) Q with dim Stab(X) = dim O(n -k) = (n-k)(n-k-1)

2

. Hence two matrices have conjugate stabilizers if and only if they have the same rank. Note that (H k ) (H l ) if and only if k l.

The The orbit strata of Mat(n)/O(n) are the sets of points that have conjugate stabilizers [START_REF] Alekseevsky | The Riemannian geometry of orbit spaces. The metric, geodesics, and integrable systems[END_REF], i.e. that have the same rank here. This gives a manifold structure to Mat(n

orbit of X is Orb(X) = {XU |U ∈ O(n)} O(n)/Stab(X). Its dimension is dim Orb(X) = dim O(n) -dim Stab(X) = nk -k(k+1)
) (H k ) = R n×n k . The strata of Mat(n)/O(n) are R n×n k /O(n),
or equivalently the strata of covariance matrices are the sets of symmetric positive semi-definite matrices of fixed rank Sym

+ (n, k) = Cov(n) ∩ R n×n k . The principal/regular stratum is the set of Symmetric Positive Definite (SPD) matrices Sym + (n) = Cov(n) ∩ GL(n). Finally, π k := π (H k ) : R n×n k -→ Sym + (n, k) is a smooth fiber bundle with fiber type St(n, k) = O(n)/O(n -k), in particular it is a submersion.

The Bures-Wasserstein distance

Since the group action is continuous, proper and isometric, the Euclidean distance descends to a distance on Mat(n)/O(n). Via the bijection Orb(X)

∈ Mat(n)/O(n) -→ XX ∈ Cov(n),
it is usually expressed as a distance on covariance matrices. It is known as the Bures-Wasserstein distance [START_REF] Dowson | The Fréchet distance between multivariate normal distributions[END_REF], Olkin and Pukelsheim, 1982, Bhatia et al., 2019]. Definition 9.19 (Bures-Wasserstein distance) [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF] The Bures-Wasserstein distance between Σ and Λ is defined by:

d BW (Σ, Λ) = inf X,Y ∈Mat(n) XX =Σ,Y Y =Λ d E (X, Y ) = inf R∈O(n) d E (Σ 1/2 , Λ 1/2 R) (9.1) = tr(Σ + Λ -2(Σ 1/2 ΛΣ 1/2 ) 1/2 ) 1/2 . (9.2) If XX = Σ and Y Y = Λ, let R ∈ O(n) such that X Y = (X ΛX) 1/2 R. Then d E (X, Y R ) 2 = Y R -X 2 2 = tr(XX + Y Y -2X Y R ) = d BW (Σ, Λ) 2 .
As the quotient of a length space, the space of covariance matrices endowed with the Bures-Wasserstein metric is a length space. It is even a complete geodesic metric space [Alekseevsky et al., 2001, Proposition 3.1.(1)].

The following result seems elementary although we did not find a clear reference in the literature. However, the Riemannian geometry is difficult to study via the submersion π k : R n×n k -→ Sym + (n, k). Indeed, it is the projection of a bundle of fiber St(n, k) O(n)/O(n -k) which is not a Lie group. Fortunately, the set Sym + (n, k) is also in bijection with the quotient manifold R n×k * /O(k) [Massart and Absil, 2020, Proposition 2.1], where R n×k * is the open set of matrices of full rank in R n×k . We recall this quotient geometry in Table 9.1. The quotient distance induced on Sym + (n, k) is the Bures-Wasserstein distance [Massart and Absil, 2020, Proposition 5.1]. In particular, the quotient topology coincides with the previous ones. This bijection naturally provides a smooth structure on Sym + (n, k). Above all, the submersion πk : R n×k * -→ Sym + (n, k) is the projection of a principal fiber bundle. Hence, it is much more convenient to study the Bures-Wasserstein Riemannian geometry of Sym + (n, k) via the Riemannian submersion πk . This is exactly what is done in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF].

To summarize, the strata Sym + (n, k) are smooth connected manifolds and the regular stratum Sym + (n) is a dense open set in Cov(n).

The Riemannian geometry of the principal stratum was extensively studied [Takatsu, 2010, Takatsu, 2011, Malagò et al., 2018, Bhatia et al., 2019, van Oostrum, 2020[START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF] and the Riemannian geometry of the other strata was recently well

Set Sym + (n, k) = Cov(n) ∩ R n×n k Smooth manifold R n×k * /O(k) Group action ß R n×k * × O(k) -→ R n×k * (X, U ) -→ XU Orbit Orb(X) = {Y ∈ R n×k * |Y Y = XX } Identification ß R n×k * /O(k) -→ Sym + (n, k) Orb(X) -→ XX Submersion π Sym + (n,k) : ß R n×k * -→ Sym + (n, k) X -→ XX Table 9.1: Smooth manifold structure of Sym + (n, k).
detailed [START_REF] Massart | Curvature of the Manifold of Fixed-Rank Positive-Semidefinite Matrices Endowed with the Bures-Wasserstein Metric[END_REF]Absil, 2020]. However, there remain missing formulae and open questions about the geodesics in each stratum: injectivity domain, preimages, and explicit formulae of the horizontal lift, the exponential map and logarithms in the base space Sym + (n, k) of the principal fiber bundle R n×k * -→ Sym + (n, k). We precisely answer these questions in Section 9.4 (full-rank matrices) and Section 9.5 (low-rank matrices). Furthermore, we contribute in Section 9.6 the minimizing geodesics for the Bures-Wasserstein distance between different strata and the condition of uniqueness of the geodesic between two points.

Geodesics of the Bures-Wasserstein metric on Sym + (n)

In this section, we give complements and new results on the Bures-Wasserstein geodesics on SPD matrices Sym + (n). The quotient structure is well known, as well as the exponential map [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF] and the injectivity radius [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF]. The definition interval of the geodesic was implicitly described in [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF] as the connected component of 0 in a subset or R so we give it explicitly here. It was proved in [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF] that there exists a preimage which is a logarithm. We prove the uniqueness of the preimage and the logarithm based on a result of [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF] on Sym + (n, k) applied for k = n. Moreover, we contribute the cut time, thus the injectivity domain. The proof is deferred to Section 11.8. After Theorem 9.22 on Bures-Wasserstein geodesics, we show on an example that we already know some geodesics between degenerate matrices that cross the main stratum of SPD matrices. Definition 9.21 (Bures-Wasserstein metric on Sym + (n)) [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF], Bhatia et al., 2019, van Oostrum, 2020] The Bures-Wasserstein metric on Sym + (n) is the quotient Riemannian metric induced by the submersion π : X ∈ GL(n) -→ XX ∈ Sym + (n) and the Frobenius metric on GL(n). Let X ∈ GL(n) such that XX = Σ and let V ∈ T Σ Sym + (n) ≡ Sym(n). The quotient operations are:

1. (Vertical space) V X = ker d X π = X Skew(n), 2. (Horizontal space) H X = Sym(n) X, 3. (Horizontal lift) V # X = S Σ (V )X ∈ H X , 4. (Bures-Wasserstein metric) g BW (n) Σ (V, V ) = tr(S Σ (V )ΣS Σ (V )),
where S Σ (V ) ∈ Sym(n) is the unique solution of the Sylvester equation ΣS Σ (V ) + S Σ (V )Σ = V .

Theorem 9.22 (Bures-Wasserstein geodesics on Sym + (n)) Let Σ ∈ Sym + (n).

1. (Exponential map) [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF] For all V ∈ T Σ Sym + (n) ≡ Sym(n), the geodesic from Σ with initial speed V writes γ (Σ,V ) (t) = Σ + tV + t 2 S Σ (V )ΣS Σ (V ) ∈ Sym + (n).

2. (Definition interval) Let λ max = max sp(S Σ (V )) and λ min = min sp(S Σ (V )). The definition interval of the geodesic γ (Σ,V ) is the interval I Σ,V defined by:

• I Σ,V = (-1 λmax , -1 λ min ) if λ min < 0 < λ max , • I Σ,V = (-∞, -1 λ min ) if λ min < 0 and λ max 0, • I Σ,V = (-1 λmax , +∞) if λ min 0 and λ max > 0, • I Σ,V = R if λ min = λ max = 0 (which only happens for V = 0). 3. (Cut time) The cut time is t cut (Σ, V ) = -1 λ min if λ min < 0 or +∞ otherwise. The geodesic γ (Σ,V ) : I Σ,V -→ M is even minimizing on I Σ,V .
4. (Logarithm map) For all Λ ∈ Sym + (n), there exists a unique preimage V ∈ Pre Σ (Λ).

It writes

V = 2 sym(Σ 1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 Σ -1/2 )-2Σ
, where we denote sym(M ) = 1 2 (M + M ). The geodesic joining Σ to Λ writes:

γ Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t) sym(Σ 1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 Σ -1/2 ). (9.3)
Moreover, it is a logarithm: V ∈ Log x (y). Thus the logarithm map is defined on U Σ = Sym + (n) and it writes:

Log Σ : ß Sym + (n) -→ T Σ Sym + (n) Λ -→ 2sym(Σ 1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 Σ -1/2 ) -2Σ . (9.4)
See the proof of Theorem 9.22 in Section 11.8.

Remark 9.23

The minimizing geodesic γ (Σ,V ) : I Σ,V -→ Sym + (n) clearly has a limit at the finite boundaries of I Σ,V . When I Σ,V is bounded, we can define Σ 0 = lim t→-1/λmax γ (Σ,V ) (t), Σ 1 = lim t→-1/λ min γ (Σ,V ) (t) and the extended curve γ : ĪΣ,V = [-1/λ max , -1/λ min ] by γ(t) = γ (Σ,V ) (t) for t ∈ I Σ,V , γ(-1/λ max ) = Σ 0 and γ(-1/λ min ) = Σ 1 . The curve γ is a minimizing geodesic on I Σ,V . Thus, by Lemma 9.

Å (1 + t) 2 0 0 (1 -t) 2 ã for t ∈ (-1, 1) with V = Å 2 0 0 -2 ã .
Moreover, from the viewpoint of the Levi-Civita connection, the curve t -→ Exp Σ (tV ) is a geodesic (self-parallel curve) on each subinterval of the set J

Σ,V = {t ∈ R|Exp Σ (tV ) ∈ Sym + (n)} = R\{t ∈ R| -1 t ∈ sp(S Σ (V )
)}, which is R without a maximum of n points. Theorem 9.22 actually states that every geodesic is minimizing on its domain. Hence, the minimizing geodesic γ (Σ,V ) : I Σ,V -→ Sym + (n) naturally extends to a curve γ (Σ,V ) : R -→ Cov(n) which is a minimizing geodesic on the segments delimited by two consecutive values in R\J Σ,V . 9.5 Geodesics of the Bures-Wasserstein metric on Sym + (n, k)

In this section, we give complements and new results on the Bures-Wasserstein geodesics on the manifold of PSD matrices of fixed rank k, Sym + (n, k). They were mainly studied in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF]. The formulae of the exponential map and its definition domain were kept implicit because they were formulated in function of horizontal vectors in the total space R n×k * of matrices of full-rank k. We compute the horizontal lift, which allows us to express the Bures-Wasserstein metric, the exponential map and the definition interval directly in function of the tangent vector. Moreover, we characterize the preimages of the exponential map. In [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF], they characterize the logarithms by solving the underlying matrix equation and then imposing the condition of minimizing norm. Based on their resolution of the matrix equation, we impose the weaker condition that the preimage v ∈ Pre x (y) must satisfy 1 ∈ I x,v ; otherwise, the geodesic γ x,v : I x,v -→ M could leave the space before reaching y. This is how we characterize all the preimages. We also give an explicit formula of the minimizing geodesic joining two points when it is unique and we specify the number of minimizing geodesics between two points otherwise. Finally, we compute the injectivity domain that was kept implicit in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF]. After Theorem 9.26, we precisely specify the novelty of our result with respect to the reference work [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF]. Then, we give examples to illustrate the possible cases for the number of preimages and logarithms. Definition 9.24 (Bures-Wasserstein metric on Sym + (n, k)) [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF] The Bures-Wasserstein metric on Sym + (n, k) is the quotient Riemannian metric induced by the submersion π : X ∈ R n×k * -→ XX ∈ Sym + (n, k) and the Frobenius metric on R n×k . Let X ∈ R n×k * such that XX = Σ and let V ∈ T Σ Sym + (n, k). The vertical and horizontal spaces are:

1. (Vertical space) V X = ker d X π = X Skew(k), 2. (Horizontal space) H X = {X(X X) -1 F +X ⊥ K, F ∈ Sym(k), K ∈ Mat(n-k, k)} where X ⊥ ∈ Mat(n, n -k) has orthonormal columns (X ⊥ X ⊥ = I n-k
) that are orthogonal to the columns of X (X X ⊥ = 0).

T Σ Sym + (n, k) = {V ∈ Sym(n)|X ⊥ V X ⊥ = 0}, 2. (Horizontal lift) V # X = X(X X) -1 S X X (X V X)+(I n -X(X X) -1 X ) X ⊥ X ⊥ V X(X X) -1 , 3. (Bures-Wasserstein metric) g BW (n,k) Σ (V, V ) = tr(S Σ,V ΣS Σ,V + V Σ † V (I n -U U )).
See the proof of Theorem 9.25 in Section 11.8. 1. (Exponential map) For all V ∈ T Σ Sym + (n, k), the geodesic from Σ with initial speed

V is γ (Σ,V ) : t ∈ I Σ,V -→ Σ + tV + t 2 W Σ,V , where W Σ,V = S Σ,V ΣS Σ,V + S Σ,V V (I n - U U )+(I n -U U )V S Σ,V +(I n -U U )V Σ + V (I n -U U ) and S Σ,V = U S D (U V U )U . 2. (Definition interval) Let F 0 X,V = S X X ((X X) -1/2 X V X(X X) -1/2 ) and M 0 X,V = (X X) -3/2 X V (I n -X(X X) -1 X )V X(X X) -3/2 ∈ Sym(n). Let E Σ,V = {λ ∈ sp(F 0 X,V )| ker(λI k -F 0 X,V ) ∩ ker(M 0 X,V ) = {0}} ⊆ sp(F 0 X,V ) = sp(S Σ,V ). If E Σ,V is non-empty, then let λ + = max E Σ,V and λ -= min E Σ,V .
The definition interval of the geodesic γ (Σ,V ) is the interval I Σ,V defined by:

• I Σ,V = (-1 λ + , -1 λ -) if λ -< 0 < λ + , • I Σ,V = (-∞, -1 λ -) if λ -< 0 and λ + 0, • I Σ,V = (-1 λ + , +∞) if λ -0 and λ + > 0, • I Σ,V = R if E Σ,V is empty.
Applying this to X = U D 1/2 without loss of generality, F 0 X,V = S D (U V U ) and M 0 X,V = D -1 U V (I n -U U )V U D -1 which is a bit more tractable to compute E Σ,V .

3. (Cut time) Let λ max = max sp(F 0 X,V ) and λ min = min sp(F 0 X,V ). Note that if E Σ,V = ∅, then we have (λ -, λ + ) ⊆ (λ min , λ max ). The cut time is t cut (Σ, V ) = -1 λ min if λ min < 0 or +∞ otherwise. Symmetrically, we have t cut (Σ, -V ) = 1 λmax if λ max > 0 or +∞ otherwise.

(Preimages) We define the indexing set I Pre

X,Y by:

I Pre X,Y = {R ∈ O(n)|H := X Y R ∈ Sym(n) and ∀µ < 0, ker(µI k -(X X) -1/2 H(X X) -1/2 ) ∩ ker(µ 2 I k -(X X) -1/2 RY Y R (X X) -1/2 ) = {0}}. For R ∈ I Pre X,Y , we denote H = H X,Y,R = X Y R so that X Y = HR. Then, the map R ∈ I Pre X,Y -→ V = 2 sym(XRY ) -2Σ ∈ Pre Σ (Λ) is a bijection whose inverse is V ∈ Pre Σ (Λ) -→ R = (Y Y ) -1 Y (X + V # X ) ∈ I Pre X,Y . The geodesic joining Σ to Λ parametrized by R ∈ I Pre X,Y writes: ∀t ∈ [0, 1], γ R Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t)sym(XRY ). (9.5) 5. (Logarithms) Let I Log X,Y = {R ∈ O(n)|H X,Y,R = X Y R ∈ Cov(n)} = {R ∈ O(n)|H X,Y,R = (X ΛX) 1/2 } = {R ∈ O(n)| X Y = (X ΛX) 1/2 R} ⊆ I Pre X,Y . Then, the map R ∈ I Log X,Y -→ V = 2 sym(XRY ) -2Σ ∈ Log Σ (Λ) is a bijection whose inverse is V ∈ Log Σ (Λ) -→ R = (Y Y ) -1 Y (X + V # X ) ∈ I Log X,Y .
6. (Logarithm map) Let r = rk(ΣΛ) = rk(X Y ) = rk(H).

(a) If r = k, then there exists a unique logarithm of Λ from Σ. In this case, the minimizing geodesic joining Σ to Λ writes:

γ Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t)sym(Σ 1/2 ((Σ 1/2 ΛΣ 1/2 ) 1/2 ) † Σ 1/2 Λ). (9.6) (b) If r = k -1
, then there exist exactly two logarithms of Λ from Σ.

(c) If r < k -1, then there is an infinity of logarithms of Λ from Σ.

Therefore, the logarithm map is defined on

U Σ = {Λ ∈ Sym + (n, k)|rk(ΣΛ) = k} and it writes Log Σ : Λ ∈ U Σ -→ 2 sym(Σ 1/2 ((Σ 1/2 ΛΣ 1/2 ) 1/2 ) † Σ 1/2 Λ) -2Σ ∈ T Σ Sym + (n, k).
See the proof of Theorem 9.26 in Section 11.8.

Remark 9.27 Let us clarify our contributions in Theorem 9.26 with respect to the reference paper [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF].

1. (Exponential map) The formula is new. Only the exponential map of a horizontal vector in the total space R n×k * was given in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF].

(Definition interval)

The definition interval was formulated in the total space R n×k * in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF]. The novelty here is to formulate it in function of V thanks to the horizontal lift.

3. (Cut time) This is new.

(Preimages

) Note that I Pre X,Y ⊆ I Sol X,Y := {R ∈ O(n)|X Y R ∈ Sym(n)}.
As shown in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF], the set I Sol X,Y indexes the solutions of the logarithm equation Exp [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF]. The curve "γ R " with R ∈ I Sol X,Y may hit the boundary before reaching Λ. Therefore, among these candidate R's such that X Y R ∈ Sym(n), we specify the set of R's that really define a geodesic from Σ to Λ in the manifold Sym + (n, k), based on the condition underlying the definition interval.

Σ (V ) = Λ. The characterization of I Pre X,Y = {R ∈ O(n)|1 ∈ I Σ,V } (with V = XRY + Y R X -2Σ) is explicitly dismissed in
5. (Logarithms) It was stated in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF] that if R leads to a logarithm then H 0, and that if H 0, then R leads to a preimage which is additionally a logarithm. However, it is not stated clearly that in this case, H has to be equal to (X ΛX) 1/2 . It is important for the next point though. The expression of the logarithms is new, although very straightforward. It was not given in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF] because they prefer to work in the total space R n×k * . 6. (Logarithm map) It was stated that the logarithm is unique if and only if r = k in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF]]. However, it was not stated that there are exactly two logarithms when r = k -1 and that there is an infinity of logarithms when r < k -1. The expression of the minimizing geodesic when it is unique is also new.

In other words, we have a minor contribution on the reformulation in Sym + (n, k) of results stated in the total space, and more important contributions on the injectivity domain (cut time), the expression of the minimizing geodesic when it is unique and the clarification between the three sets

I Log X,Y ⊆ I Pre X,Y ⊆ I Sol X,Y := {R ∈ O(n)|X Y R ∈ Sym(n)}.
We give several examples below to illustrate the differences between these three sets.

Remark 9.28 The definitions of the sets E Σ,V and I Pre Σ,V might seem intricate. It is difficult to simplify them though. Nevertheless, the set E Σ,V is easy to determine numerically. Analogously, it is easy to determine numerically if a candidate R ∈ I Sol X,Y given in [Massart and Absil, 2020, Lemma 4.1]

(t) = (1 + t) 2 Σ ∈ Sym + (3, 2) for t ∈ (-1, +∞) ⊃ [0, 1],
(b) there is no non-minimizing geodesic,

(c) R = -I 2 or R = R - θ lead to curves that hit Sym + (3, 1) at t = 1 3 < 1, e.g. R = -I 2 leads to the curve γ(t) = (1 -3t) 2 Σ ∈ Sym + (3, 2) only for t ∈ (-∞, 1 3 ). 2. Let Σ = Ñ 1 0 0 0 1 0 0 0 0 é and Λ = Ñ 1 0 0 0 1 1 0 1 1 é in Sym + (3, 2) with r = 2 again. Then, let X = Ñ 1 0 0 1 0 0 é and Y = Ñ 1 0 0 1 0 1 é .
Then X Y = I 2 so the candidate pairs for (H, R) are (I 2 , I 2 ), (-I 2 , -I 2 ) and (R - θ , R - θ ). One can show that:

(a) R = I 2 leads to the geodesic γ

I 2 Σ→Λ (t) = Ñ 1 0 0 0 1 t 0 t t 2 é ∈ Sym + (3, 2) for t ∈ R which is minimizing on î -2 1+ √ 2 , 2 √ 2-1 ó ⊃ [0, 1], (b) R = R - θ for θ = 0 lead to non-minimizing geodesics, e.g. R = R - π leads to the curve γ R - π Σ→Λ (t) = Ñ 1 0 0 0 (1 -2t) 2 -t(1 -2t) 0 -t(1 -2t) t 2 é ∈ Sym + (3, 2) for t ∈ [0, 1], (c) R = R - 0 and R = -I 2 lead to curves that hit Sym + (3, 1) at t = 1 2 < 1, e.g. R = R - 0 leads to the curve γ(t) = Ñ (1 -2t) 2 0 0 0 1 t 0 t t 2 é ∈ Sym + (3, 2) only for t ∈ (∞, 1 2 ). 3. Let Σ = Ñ 1 0 0 0 1 0 0 0 0 é and Λ = Ñ 1 0 0 0 0 0 0 0 1 é in Sym + (3, 2) with r = 1. Then, let X = Ñ 1 0 0 1 0 0 é and Y = Ñ 1 0 0 0 0 1 é . Then X Y = Å 1 0 0 0 ã so the candidate values of R
are Diag(±1, ±1). One can show that:

(a) R 0 ± = Diag(1, ±1) lead to two minimizing geodesics whose expressions are γ

R 0 ± Σ→Λ (t) = Ñ 1 0 0 0 (1 -t) 2 ±t(1 -t) 0 ±t(1 -t) t 2 é ∈ Sym + (3, 2) for t ∈ [0, 1],
(b) there is no non-minimizing geodesic,

(c) R 1 ± = Diag(-1, ±1) lead to curves that hit Sym + (3, 1) at t = 1 2 < 1, namely γ R 1 ± (t) = Ñ (1 -2t) 2 0 0 0 (1 -t) 2 ±t(1 -t) 0 ±t(1 -t) t 2 é ∈ Sym + (3, 2) only for t ∈ (-∞, 1 2 ). 4. Let Σ = Å I 2 0 0 0 ã and Λ = Å 0 0 0 I 2 ã in Sym + (4, 2) with r = 0. Then, let X = Å I 2 0 ã and Y = Å 0 I 2 ã . Then X Y = 0 so every R ∈ O(2) is a candidate. One can show that any R ∈ O(2) leads to a minimizing geodesic γ R Σ→Λ (t) = Å (1 -t) 2 I 2 t(1 -t)R t(1 -t)R t 2 I 2 ã ∈ Sym + (4, 2) for t ∈ R.
In the two last sections, we studied the geodesics and the minimizing geodesics within each stratum. In the next section, we turn to the study of the minimizing geodesic segments in the Bures-Wasserstein metric space (Cov(n), d BW ), that is between any two covariance matrices of any rank.

Minimizing geodesics of the Bures-Wasserstein distance on Cov(n)

In this section, we completely characterize the Bures-Wasserstein minimizing geodesic segments between any two covariance matrices. We show that they have constant rank on the interior of the segment and we give an explicit expression. Moreover, we show that the number of geodesics depends on the ranks of the extremities and we give this number in all cases. More precisely, we show that minimizing geodesics between Σ and Λ ∈ Cov(n) are parametrized by the closed unit ball of R (k-r)×(l-r) for the spectral norm, where k, l, r are the respective ranks of Σ, Λ, ΣΛ. We also give the number of geodesics of minimal rank. Finally, we show that there exists a canonical geodesic with an expression that does not depend on the ranks of the extremities. This expression coincides with the formula in low rank when the minimizing geodesic is unique and with the formula in full rank.

Characterization of minimizing geodesics

The following lemma states that the rank of a minimizing geodesic segment is constant on the interior of the segment. Then, Theorem 9.31 characterizes the Bures-Wasserstein minimizing geodesic segments.

Lemma 9.30 (Rank of minimizing curve) Let γ : [0, 1] -→ Cov(n) be a minimizing curve from Σ to Λ. Then γ has constant rank p max(rk(Σ), rk(Λ)) on (0, 1).

Proof. Let p = max t∈[0,1] rk(γ(t)) and let t 0 ∈ [0, 1] such that rk(t 0 ) = p. By Lemma 9.18, for all t ∈ (0, t 0 ) ∪ (t 0 , 1), (H rk(γ(t)) ) H rk(γ(t 0 )) so rk(γ(t)) rk(γ(t 0 )) = p so rk(γ(t)) = p. (i) the curve γ : [0, 1] -→ Cov(n) is a minimizing geodesic segment from Σ to Λ,

(ii) there exists R ∈ O(n) such that H X,Y,R := X Y R ∈ Cov(n) and for all t ∈ [0, 1], γ(t) = γ R Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t)sym(XRY ).
Moreover, H X,Y,R = (X ΛX) 1/2 and the minimizing geodesic γ R Σ→Λ is of constant rank p max(k, l) on (0, 1). See the proof of Theorem 9.31 in Section 11.8. Remark 9.32 In the previous theorem, X and Y may be respectively taken as X = Σ 1/2 and Y = Λ 1/2 .

Number of minimizing geodesics

In this section, we count the number of minimizing geodesic segments between two covariance matrices. We start with an elementary lemma. 1. For all X, Y ∈ Mat(n) such that XX = Σ and Y Y = Λ, r = rk(X Y ).

2. We have l -r n -k.

See the proof of Lemma 9.33 in Section 11.8. In the following theorem and especially in its proof, we need to distinguish cases where matrices may have one or two null dimensions, i.e. belonging to R n×0 , R 0×k or R 0×0 . Thus we recall that these spaces are equal to the vector space {0}. Indeed, there is a unique linear map from R n to R 0 or from R 0 to R k , which is the identically null map. The canonical basis of R 0 is empty and the corresponding matrix in the canonical bases is called the empty matrix. It is practical to treat these spaces as non-trivial spaces to avoid writing particular cases. In particular, St(n, 0) = R n×0 and O(0) = GL(0) = Mat(0) = Diag(0) are sets of cardinal 1. Theorem 9.34 (Number of Bures-Wasserstein minimizing geodesic segments in Cov(n)) Let Σ, Λ ∈ Cov(n) with rk(Σ) = k and rk(Λ) = l. We assume that k l without loss of generality. We denote r = rk(ΣΛ). We have l -r n -k.

1. There exists a bijection between the set of minimizing geodesics from Σ to Λ and the closed unit ball of R (k-r)×(l-r) for the spectral norm BS (0, 1)

= {R 0 ∈ R (k-r)×(l-r) | R 0 S 1} = {R 0 ∈ R (k-r)×(l-r) | 0 R 0 R 0 I l-r }.
More precisely, this bijection is given by: r) , D r ∈ Diag + (r), R 0 ∈ BS (0, 1). 2. The minimizing geodesic is unique if and only if r = l. This includes the cases k = n.

γ R 0 Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t) sym(X r Y r + X k-r R 0 Y l-r ), ( 9
Ñ I r 0 0 0 R 0 * 0 * * é ∈ O(n), with X r , Y r ∈ R n×r , X k-r ∈ R n×(k-r) , Y l-r ∈ R n×(l-
3. There are infinitely many minimizing geodesics if and only if r < l.

The minimizing geodesics corresponding to the choices

R 0 ∈ St(k -r, l -r) (including the empty matrix if r = l) have rank exactly k on [0, 1) (on [0, 1] if l = k). Note that St(k -r, l -r) is included in the unit sphere S S (0, 1) = {R 0 ∈ R (k-r)×(l-r) | R 0 S = 1}.
5. The minimizing geodesic corresponding to the choice R 0 = 0 (or the empty matrix if r = l) writes for all t ∈ [0, 1]: The number of minimizing geodesic segments in Sym + (n, k) and in Cov(n) is summarized in Table 9.2 with n k l r.

γ 0 Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t) sym(Σ 1/2 ((Σ 1/2 ΛΣ 1/2 ) 1/2 ) † Σ 1/2 Λ). ( 9
See the proof of Theorem 9.34 in Section 11.8.

Σ ∈ Λ ∈ r = rk(ΣΛ) Number of minimizing geodesics in Sym + (n, k) in Cov(n) Sym + (n) Sym + (n) n 1 1 Sym + (n) Sym + (n, k) k 1 1 Sym + (n, k) Sym + (n, k) k 1 1 k -1 2 ∞ < k -1 ∞ ∞ Sym + (n, k) Sym + (n, l) l 1 1 < l ∞ ∞ Table 9
.2: Number of Bures-Wasserstein minimizing geodesic segments (n k l r).

Remark 9.35

The canonical minimizing geodesic (Equation 9.8) is defined between any two covariance matrices. This formula can always be used to interpolate between two covariance matrices. In particular, we easily check that it reduces to the formula of the unique minimizing geodesic between two SPD matrices (Equation 9.3). Moreover, Equation 9.8 is logically the same as the formula of the minimizing geodesic in Sym + (n, k) when it is unique (Equation 9.6).

Conclusion

In this work, we have answered several open questions on geodesics of the Bures-Wasserstein distance on covariance matrices. Beyond geodesics, a very important element of Riemannian geometry is the curvature. We know that the space of covariance matrices with the Bures-Wasserstein distance is an Aleksandrov space of non-negative curvature [Takatsu, 2011] and we know the curvature tensor in each stratum [Takatsu, 2010, Takatsu, 2011, Massart et al., 2019]. However, we lack a comprehensive and global approach of the curvature of the whole metric space. In particular, what is the appropriate notion of curvature to use to go from a stratum to another?

In the community of geometric statistics, most of the stratified spaces that were studied from the viewpoint of geodesics or curvature are very singular (spiders, trees) or a bit complex to start with (BHV space, Wald space) [START_REF] Feragen | Chapter 8. Statistics on stratified spaces[END_REF]. Thus, the familiar example of the Bures-Wasserstein Riemannian orbit space appears to be a good basis to generalize concepts defined in Riemannian statistics. Indeed, after studying the geometry of these non-Riemannian spaces, what statistical tools should we define on them to generalize the Euclidean and Riemannian ones? This is probably the main question to investigate for the future.

As a perspective for future works, we quickly introduce an ongoing work that is closely related to the Bures-Wasserstein geometry of covariance matrices. The set of correlation matrices, known as the elliptope, was mainly studied on the open stratum of full-rank correlation matrices Cor [START_REF] David | A Riemannian structure for correlation matrices[END_REF], David, 2019, Nielsen and Sun, 2019, Thanwerdas and Pennec, 2021[START_REF] Thanwerdas | Theoretically and computationally convenient geometries on full-rank correlation matrices[END_REF], Archakov and Hansen, 2021, Thanwerdas, 2022]. The geometry of the whole elliptope Cor(n) = {C ∈ Cov(n)|Diag(C) = I n } was described topologically in [Kercheval, 2008] Chapter 10

+ (n) = {C ∈ Sym + (n)|Diag(C) = I n } [

Conclusion and perspectives 10.1 Summary of contributions and beyond

In this thesis, we studied geometries of covariance and correlation matrices from different angles. On SPD matrices, we formalized a dictionary of Riemannian metrics enriched with new formulae and with a novel approach of classes based on the characterization of O(n)-invariant metrics. In particular, we interpolated the noted Euclidean, log-Euclidean, power-Euclidean, affine-invariant, power-affine and Bogoliubov-Kubo-Mori metrics by the two-parameter family of Mixed-Power-Euclidean metrics. We enlightened the bridge between the wider family of Mixed-Euclidean metrics and the (u, v)-divergences of information geometry and we computed their curvature by generalizing the computation known for the BKM metric, that is with u = Id and v = log. Besides O(n)-invariant metrics, we also characterized LT + (n)invariant Riemannian metrics as pullbacks of left-invariant metrics on the Lie group LT + (n) via the Cholesky map. These Lie group metrics form a family of Riemannian metrics parametrized by inner products on LT(n), that is by Sym + ( n(n+1)

2

). They belong to the wider family of Diag + (n)-invariant metrics which descend to the open elliptope of full-rank correlation matrices. This exploration of the space of Riemannian metrics on SPD matrices with parametric and non-parametric subfamilies offers a new promising setting for choosing a metric satisfying given requirements or for optimizing the metric.

We also followed this systematic approach on full-rank correlation matrices. Our work on Lie group actions on SPD matrices allowed us to generalize the quotient-affine metric with the family of quotient-Lie-Cholesky metrics. However, we observed that these are probably not the best suited metrics on the open elliptope. Thus we grasped the relevant concepts behind them such as the permutation action, the positive diagonal action on SPD matrices, the reduced Cholesky map Θ : Cor + (n) -→ LT 1 (n), the cor-inverse involution or the block-equicorrelation matrices. In view of proposing better metrics, the characterization of permutation-invariant inner products on the tangent space Hol(n) = T In Cor + (n) turned out to be crucial for understanding the natural vector spaces to work with. This made us realize the module isomorphism between Hol(n) and Row 0 (n), which supported the idea of an analogous correspondence at the manifold level between Cor + (n) and Row + 1 (n). Thanks to these suited tools, we built five families of Hadamard and flat metrics on full-rank correlation matrices. In particular, the log-scaled metrics are flat, geodesically complete, permutationinvariant and inverse-consistent, which are quite satisfying properties. All the geometric operations can be computed via the diffeomorphism onto the Euclidean space at hand, in finite time for the ones based on the Cholesky map, in logarithmic time complexity for the others. For the log-scaled metrics, we proposed a new algorithm to compute the scaling of an SPD matrix which seems competitive with respect to existing ones.

These Riemannian metrics also lead to brand new approaches on SPD matrices since they allow to decouple the scales of variables and the correlations between variables. These two components of the data are completely mixed so far because of the use of O(n)-invariant metrics such as the affine-invariant or the log-Euclidean metrics. These new product metrics define new classes of means and midpoints with different monotonicity properties that could be compared with existing ones [START_REF] Bhatia | Riemannian geometry and matrix geometric means[END_REF], Mostajeran and Sepulchre, 2018, Mostajeran et al., 2020]. In practice, they offer the possibility for researchers to make other types of assumptions than before on their data and experiments. For example, if the scales of variables do not matter in first approximation, the data can be normalized to correlation matrices. Then one could add a metric λg on the scales, that is on positive diagonal matrices, with a parameter λ 0 to tune in function of the relative importance of scales to correlations. Let us give the example of single-cell analysis in genetics. Cells are sampled from a tissue, the goal is to identify types of different cells. For each cell i, the number of mRNA segments belonging to the gene j is counted after a polymerase chain reaction (PCR). Thus the raw data are count matrices of size n cells × n genes . An independent normalization of each row is usually applied to neglect effects related to the size of the cell: a big cell tends to have more mRNA segments than a smaller cell. Thus the correlation matrix of the count matrix could be a relevant representation of the data. However, the size of the cell may also contain information on the type of cell. Therefore, one could assume that choosing a product metric with a low coefficient on the diagonal part could better fit the application.

It is even possible to tune the metric g to give more or less importance to the scales of some variables. For example, in brain-computer interfaces, we may assume that the scales of the signals at the electrodes located in the brain region of the response to the stimulus contain more information than elsewhere. Therefore, one would rather choose a metric on diagonal matrices that is not invariant by permutations to take into account this asymmetry. By analogy with [Guigui et al., 2021, Guigui, 2021], the product metrics could also initiate new methods in the registration of covariance matrices with parallel transport by allowing a step of normalization of the scales, especially when the scales significantly differ within a dataset of covariance matrices.

On covariance matrices of any rank, we promoted the viewpoint of Riemannian orbit spaces to compute all the minimizing geodesics of the Bures-Wasserstein geometry. It was well known that covariance matrices form a quotient space, the strata had been described in [Takatsu, 2011], the geometry had been studied on each stratum [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF], Bhatia et al., 2019, Massart and Absil, 2020], and we finally brought a bigger picture with the characterization of minimizing geodesics. This work departs from the current research on stratified spaces for applications. Indeed, it is an example of a non-negatively curved metric stratified space that is not a tree space or a graph space and for which the minimizing geodesics are known in closed form. Besides these three major differences, the simple formulae of geodesics supports the idea of using Riemannian orbit spaces when it is possible to get more structure in the space. This class contains Kendall shape spaces, the space of correlation matrices stratified by the rank, the space of diagonal matrices stratified by eigenvalue multiplicity and probably other examples. Moreover, Geodesic PCA [START_REF] Huckemann | Intrinsic shape analysis: Geodesic PCA for Riemannian manifolds modulo isometric Lie group actions[END_REF] is defined in this kind of spaces via generalized geodesics, which are simply projections of horizontal geodesics. We note that the Bures-Wasserstein metric is more and more used in applications [START_REF] Severn | Manifold valued data analysis of samples of networks, with applications in corpus linguistics[END_REF], Chewi et al., 2020, Kroshnin et al., 2021, Han et al., 2021] since its geometric operations were derived. In this aspect, our formulae open new uses of the Bures-Wasserstein distance in applications where covariance matrices may be either regular or singular.

Riemannian orbit spaces could be a nice first class of stratified spaces to implement in geomstats with generic geometric and statistical operations. Complete Riemannian orbit spaces should inherit all the methods known in complete metric spaces and should be the right class to generalize many statistical methods used in complete Riemannian (quotient) manifolds. Although mainly ad hoc methods are currently used in stratified spaces such as the BHV space or the QED space, it might be possible to adapt some of them to the richer structure of Riemannian orbit spaces. Therefore, the Bures-Wasserstein geometry of covariance matrices offers a new playground to test generic tools on orbit spaces and stratified spaces.

During my PhD thesis, I also took part in the development of the geomstats package with about 100 commits. I also had a less visible role in the reflection of the class architecture, trying to mathematically categorize the types of spaces of interest with my colleague Nicolas Guigui. Thanks to my bibliographic work on Riemannian metrics, especially on the affineinvariant, the Bures-Wasserstein and the quotient-affine metrics, we had long discussions on all sorts of quotient spaces and homogeneous spaces before he implemented the generic class of (principal) fiber bundle. I personally left the code aside momentarily to allocate time for the research of suitable structures on covariance and correlation matrices and to push the computations as far as they could be. I can now implement all these tools in geomstats and make them available for everyone.

Future works 10.2.1 Other geometries for the rank stratification

The first burning perspective to this work is probably the study of the Riemannian orbit space of correlation matrices endowed with the quotient-polysphere structure that we briefly described at the end of Chapter 9. Contrarily to the Bures-Wasserstein geometry, it has not been described on each stratum so it is a very challenging problem.

It would also be nice to introduce families of stratified geometries on covariance and correlation matrices to allow some flexibility. In this direction, generalized Bures-Wasserstein metrics were recently introduced on SPD matrices [START_REF] Han | Generalized Bures-Wasserstein geometry for positive definite matrices[END_REF]. They are defined as the quotient metrics of the O(n)-invariant metrics g GBW(S) A (X, X) = tr(X SX) for A ∈ GL(n) and X ∈ Mat(n), where S ∈ Sym + (n) is a parameter. The Bures-Wasserstein metric corresponds to S = I n . This metric turn out to be simply the pullback metric of the Bures-Wasserstein metric by the diffeomorphism Σ ∈ Sym + (n) -→ S 1/2 ΣS 1/2 ∈ Sym + (n). In other words, it is a change of reference point. This construction could be extended to the whole space of covariance matrices to define the family of generalized Bures-Wasserstein stratified geometries indexed by S ∈ Sym + (n). With S ∈ Diag + (n)\R + I n , this introduces weights on the features, thus the geometry is not anymore invariant under permutations. It could be interesting if one wants to give more or less importance to some features for example.

Analogously, a generalized quotient-polysphere distance could be defined by introducing weights in the product metric of spheres (S n-1 ) n . Another question that we left aside is the research of geometries for the manifolds Cor(n, k) of fixed-rank correlation matrices, independently from other strata. It could be useful in applications where the rank is almost surely equal to k. Note that contrarily to the full-rank case, the manifold Cor(n, k) is not the quotient of Sym + (n, k) by positive diagonal matrices. Indeed, singular covariance matrices include covariance matrices with zero variance, which are excluded from Cor(n, k). We suspect that several quotient geometries on Sym + (n, k) = M/G may pass to Cor(n, k) not by simple quotient but by an operation of type (Diag + (n)\M)/G, that is removing the scales before quotienting by G. That is exactly the case for the quotient-polysphere structure: the Bures-Wasserstein geometry on covariance matrices is not invariant under positive diagonal matrices so instead of defining something like Cor(n

) = Diag + (n)\Cov(n) = Diag + (n)\(Mat(n)/O(n)), the quotient-polysphere is defined by Cor(n) = (S n-1 ) n /O(n) = (Diag + (n)\Mat(n))/O(n).
Covariance and correlation matrices are also embedding spaces for spaces of trees or graphs such as the Laplacian matrices [START_REF] Ginestet | Hypothesis testing for network data in functional neuroimaging[END_REF] or the Wald space [START_REF] Garba | Information geometry for phylogenetic trees[END_REF], which are themselves stratified spaces. It could be interesting to study the geometry induced by the Bures-Wasserstein geometry of these spaces. For example, a condition satisfied by Laplacian matrices is that the row sums are null. We notice that this condition is preserved all along a minimizing geodesic if it is satisfied by the end points. We could extend the notion of totally geodesic submanifold to metric spaces and study if these spaces are totally geodesic subspaces of covariance matrices or correlation matrices.

Geometries of matrices of any size and any rank

Distances between Grassmannians and between SPD matrices of different dimensions were proposed in [Ye andLim, 2016, Lim et al., 2019]. For two SPD matrices Σ ∈ Sym + (n) and Λ ∈ Sym + (k) with k n represented as ellipsoids E Σ and E Λ , the general methodology consists in finding the ellipsoid E Σ 0 ⊆ E Σ of dimension k which is the closest to E Λ for a given distance d in Sym + (k). The distance between Σ and Λ is defined as the distance d(Λ, Σ 0 ) in Sym + (k). For Grassmannians, it consists in aligning vector spaces in a similar way. This method seems quite general, it could probably be applied to PSD matrices represented as ellipsoids for SPD matrices and ellipsoidal cylinders for singular matrices, as well as for correlation matrices. This would provide distances between covariance matrices of different dimensions and different ranks. If the dimension is fixed, it could be interesting to study the geometry of such defined distances on the spaces of covariance matrices and correlation matrices.

This raises the problem of the geometry of spaces of matrices of different dimensions. Beyond the distance, is it possible to glue the manifolds together in a stratified way? Could they be seen as orbit spaces? These challenging theoretical questions may meet practical needs in graph theory for example, were it is difficult to compare graphs with a different number of nodes. These graphs are often represented by adjacency or Laplacian matrices so adding an isolated node corresponds to add zeros in the matrix, which artificially introduces singularities. This also obliges to fix the number of nodes beforehand, which is a strong constraint in applications where nodes appear and disappear.

Eigenvalue multiplicity and block-wise transformations

The stratification of SPD matrices by eigenvalue multiplicity was investigated in [START_REF] Groisser | Geometric foundations for scaling-rotation statistics on symmetric positive definite matrices: Minimal smooth scaling-rotation curves in low dimensions[END_REF]. To understand eigenvalue multiplicity, it may be easier to consider the simpler stratified space of positive diagonal matrices. It is the orbit space of SPD matrices by the congruence action of the orthogonal group. This framework can probably be extended to covariance matrices of any rank by allowing null eigenvalues. This would create a framework where both multiple eigenvalues and singular matrices are allowed.

Another direction to investigate could be the invariance under the action of block-wise orthogonal transformations or block-wise permutations. This could help identify groups of variables that must be equally treated within groups but potentially not across different groups. This is related to eigenvalue multiplicity since block-orthogonal matrices are precisely the stabilizers of diagonal matrices with repeated (non-decreasingly ordered) eigenvalues. This is also related to the study of flag manifolds, which are quotients of the orthogonal group by a block-orthogonal group [Monk, 1959, Alekseevsky, 1997] and which generalize the Grassmannians. Moreover, thanks to Chapter 3, we have all the necessary tools to determine all block-permutation-invariant inner products on symmetric matrices so this could help define suitable Riemannian metrics to handle this kind of invariance.

Applications

One of the most exciting challenges is to apply these new tools on covariance and correlation matrices in the domains we identified: brain-computer interfaces, brain connectivity, signal processing, genomics, shapes of proteins, finance, etc. Although the affine-invariant and the log-Euclidean metrics were shown to outperform the results with respect to the Euclidean metric in many contexts, there are still many unsolved issues with this kind of data. While discussing with researchers from these domains, the three problems we heard the most were the fact that the rank might be deficient, the fact that the rank might be non-constant within a dataset and the lack of tools for correlation matrices. Therefore, the tools we propose will at least allow to do new operations on the data. Note that many of the new geometries defined in this thesis could probably be extended to complex matrices, which opens the way to other applications in complex signal processing for instance.

Furthermore, applying our metrics to these problems will allow us to test our assumptions and to compare the results obtained with different metrics. There are many simple tests to do on synthetic data of full-rank correlation matrices such as computing means, principal components and distances to do clustering. For example in BCI, it is frequent to classify mental tasks by comparing distances to reference points. Then, the interactions with researchers of these domains will help us get other assumptions on the data and understand at which step of the workflow the geometry may intervene. These are appealing perspectives to Preprints • Thanwerdas, Yann and Pennec, Xavier (2022)

= (n) -→ O(n) such that for all Σ ∈ Sym = (n), vec(Σ)val(Σ)vec(Σ) = Σ.
Proof of Theorem 2.7 (No continuous map of eigenvectors on Sym = (n)). It is well known that there is no continuous map of eigenvectors on the whole vector space of symmetric ma-

trices Sym(n). Indeed, if Σ(t) = t Å -1 0 0 1 ã = t I 2 Å -1 0 0 1 ã I 2 and Λ(t) = t Å 0 1 1 0 ã = t 2 Å 1 1 -1 1 ã Å -1 0 0 1 ã Å 1 -1 1 1 ã , then
: t ∈ R -→ vec(Σ(t)) ∈ O(2) and R : t ∈ R -→ P (t)Q(0) ∈ O(2). Note that Q(0) = Q(1) = R(0) = -R(1) ∈ Diag(±1, ±1) and Σ(t) = Q(t)DQ(t) = R(t)DR(t) for all t ∈ [0, 1].
Thanks to Lemma 2.6, we will build a sequence 0 = t 0 < ... < t N t N +1 = 1 and show by recurrence that Q(t k ) = R(t k ) for all k ∈ {0, ..., N + 1}. Thus we will show that Q(1) = R(1) which will be a contradiction.

By the Heine theorem applied to

Q on [0, 1], let δ ∈ (0, 1 4 ) such that if t, t ∈ [0, 1] satisfy |t -t | < 2δ, then Q(t) -Q(t ) < √ 2. We denote t k = kδ for k ∈ {0, ..., N } where N = 1 δ and t N +1 = 1. Since N 1 δ < N + 1, we have t N 1 < t N + δ so 0 t N +1 -t N < δ. Assume that k ∈ {0, ..., N } is such that Q and R coincide on t 0 , ..., t k . Since Q(t k+1 )DQ(t k+1 ) = R(t k+1 )DR(t k+1 ) and D has distinct eigenvalues, we have Q(t k+1 ) = R(t k+1 )ε with ε ∈ Diag(±1, ±1). Let us show that ε = I 2 . Since |t k+1 -t k | δ < 2δ, we have Q(t k+1 ) -Q(t k ) < √ 2 so by Lemma 2.6, Diag(Q(t k ) Q(t k+1 )) > 0. We also have Diag(R(t k ) R(t k+1 )) = Q(0)Diag(P (t k ) P (t k+1 ))Q(0) = cos(π(t k+1 -t k ))I 2 > 0 since |t k+1 -t k | < 1 2 . Since Diag(Q(t k ) Q(t k+1 )) = Diag(R(t k ) R(t k+1 ))ε, it is only possible if ε = I 2 so Q(t k+1 ) = R(t k+1 ). By recurrence, since R(0) = Q(0), we have R(t N +1 ) = Q(t N +1 ), i.e. R(1) = Q(1)
. This is a contradiction.

Proofs of Chapter 3

In this section, we prove the general form of invariant inner products given in Chapter 3.

(n) = 1 i p k i 2 Skew n (k i ) ⊕ 1 i<j p Skew n (k i , k j ).
The decomposition is orthogonal for the Frobenius inner product. The O(I)-invariant inner products on Skew(n) are given for all X ∈ Skew(n) by: ϕ(X, X) =

1 i p k i 2 α ii X ii 2 + 1 i<j p α ij X ij 2 ,
where α ij > 0 for all 1 i j p.

Proof of Example 3.5: O(I)-invariant inner products on Skew(n).

Direct sum

The orthogonal direct sum is the sum of blocks and the orthogonal projections are the canonical projections. When k i = 1, Skew n (k i ) = {0} so these spaces are removed from the direct sum.

Irreducibility By block matrix multiplication, the terms are clearly stable by O(I). Moreover, we already showed that Skew(n) is O(n)-irreducible in the previous section for n 2. Hence, for k

i 2, Skew n (k i ) is clearly irreducible with respect to the group O n (k i ). Since it is a subgroup of O(I) and since Skew n (k i ) is stable under O(I), Skew n (k i ) is O(I)- irreducible. Moreover, we can show that Mat(k, l) is O(k)×O(l)-irreducible for the action ((U, V ), X) ∈ (O(k) × O(l)) × Mat(k, l) -→ U XV and this will analogously prove that Skew n (k i , k j ) Mat(k i , k j ) is O n (k i ) × O n (k j )-irreducible, hence O(I)-irreducible. Indeed, if k
l (without loss of generality), then by singular value decomposition, for all X ∈ Mat(k, l), there exists U, V ∈ O(k) × O(l) such that U XV = (D 0) with D ∈ Diag(k). If X = 0, then there exists i ∈ {1, ..., k} such that d i = 0, for example i = 1. Hence, applying the action of (I k , diag(-1, 1, ..., 1)) ∈ O(k)×O(l), we get D = diag(-d 1 , d 2 , ..., d n ). Then E 11 = 1 d i (D-D ) and all the matrices e i e j of the canonical basis can be obtained by permuting the lines and columns by (P (1,i) , P (1,j) ). So from any X ∈ Mat(k, l)\{0}, we retrieve Mat(k, l) so Mat(k, l) is O(k) × O(l)-irreducible.

Isomorphic modules

The group clearly acts differently on each block so the irreducible modules of the decomposition are not isomorphic.

General form Therefore, we are again in the simple case where an invariant inner product is the positive weighted sum of the restrictions of the Frobenius inner product to each irreducible submodule in the decomposition.

Proof of

(n) = p i=1 span(I ii n ) ⊕ 1 i p k i 2 Sym 0 n (k i ) ⊕ 1 i<j p Sym n (k i , k j ).
The decomposition is orthogonal for the Frobenius inner product and V. The O(I)-invariant inner products on Sym(n) are given for all X ∈ Sym(n) by:

ϕ(X, X) = 1 i,j p S ij tr(X ii )tr(X jj ) k i k j + 1 i p k i 2 α ii X ii - tr(X ii ) k i I ii n 2 + 1 i<j p α ij X ij 2 ,
where S ∈ Sym + (p) and α ij > 0 for all 1 i j p.

Proof of Example 3.6: O(I)-invariant inner products on Sym(n).

Direct sum

The orthogonal direct sum is the sum of blocks, refined on the diagonal blocks with the O(k i )-irreducible decomposition on Sym n (k i ) Sym(k i ), i.e. the trace part span(I ii n ) and the traceless part Sym 0 n (k i ). When k i = 1, Sym 0 n (k i ) = {0} so these spaces are removed from the direct sum.

Irreducibility

(LT) part of X ∈ Sym n (k i , k j ) to get UT(X) -LT(X) ∈ Skew n (k i , k j ). We just proved that Skew n (k i , k j ) if O(I)-irreducible, hence so does Sym n (k i , k j ).

Isomorphic modules

The main difference with the previous examples is that the spaces span(I ii n ) are all isomorphic as modules. Indeed,

ψ i : λ √ k i I ii n ∈ span(I ii n ) -→ λ ∈ R for i ∈ {1, ...,
k i I ii n ä 2 = λ 2 .
The other terms of the irreducible decomposition are clearly not isomorphic as modules because the action of O(I) differs on each block. If there were an off-diagonal block of dimension 1, it couldn't be isomorphic as a module to span(I ii n ) since it can be changed into its opposite by an appropriate element of O(I).

General form Therefore, according to Theorem 3.4, the O(I)-invariant inner products on Sym(n) are given for all X ∈ Sym(n) by: ϕ(X, X) =

1 i p k i 2 α ii X ii - tr(X ii ) k i I ii n 2 + 1 i<j p α ij X ij 2 + 1 i,j p S ij ψ i Å tr(X ii ) k i I ii n ã tr(X ii ) √ k i ψ j Å tr(X jj ) k j I jj n ã tr(X jj ) √ k j , ( 11.1)
where α ij > 0 for 1 i j p and S = (S ij ) 1 i j p ∈ Sym + (p), as expected.

Proof of Example 3.8

Example 3.8 (S(n)-invariant inner products on Hol(n)) The irreducible decomposition of Hol(n) with respect to the congruence action of S(n), i.e. of permutation matrices, is

Hol(n) = ker φ 1 ⊥ ⊕ im ψ 2 ⊥ ⊕ im ψ 3 where: 1. φ 1 : X ∈ Hol(n) -→ X1 ∈ R n , 2. ψ 2 : µ ∈ R n 0 -→ 1 n-2 (µ1 + 1µ -2diag(µ)) ∈ Hol(n), 3. ψ 3 : x ∈ R -→ x n(n-1) (11 -I n ) ∈ Hol(n)
, with φ 1 surjective, ψ 2 and ψ 3 injective. The corresponding orthogonal projections are:

• p 3 : X ∈ Hol(n) -→ X 3 = ψ 3 (sum(X)) ∈ im ψ 3 , • p 2 : X ∈ Hol(n) -→ X 2 = ψ 2 ((X -X 3 )1) ∈ im ψ 2 , • p 1 : X ∈ Hol(n) -→ X 1 = X -X 2 -X 3 ∈ ker φ 1 .
The S(n)-invariant inner products on Hol(n) are given for all X ∈ Hol(n) by:

ϕ(X, X) = α 1 tr(X 2 1 ) + α 2 tr(X 2 2 ) + α 3 tr(X 2 3 ) = α tr(X 2 ) + β sum(X 2 ) + γ sum(X) 2 , where α 1 = α > 0, α 2 = α + n-2 2 β and α 3 = α + (n -1)(β + nγ) > 0. Proof of Example 3.8: S(n)-invariant inner products on Hol(n), n 3.
Direct sum Here, the decomposition and the projections are not trivial so let us check all the statements made above. Indeed, ψ 3 is clearly injective and for all µ ∈ R n , with µ 0 = µ-sum(µ) n 1 ∈ R n 0 , we have ψ 2 (µ 0 )1 = µ 0 and [ψ 2 (µ 0 )+ψ 3 (sum(µ))]1 = µ, which proves that ψ 2 is injective and ψ 1 is surjective. By definition of X 3 , 1 (X -X 3 )1 = sum(X) -sum(X 3 ) = 0 so (X -X 3 )1 ∈ R n 0 and p 2 is well defined. Then X1 = (X -X 3 )1 -ψ 2 ((X -X 3 )1) = 0 so p 1 is well defined and Hol(n) = ker φ 1 +im ψ 2 +im ψ 3 . Since dim ker φ 1 +dim im ψ 2 +dim im ψ 3 = (dim Hol(n) -n) + (n -1) + 1 = dim Hol(n), the sum is a direct sum. Now if X ∈ ker φ 1 , Y = ψ 2 (µ) and Z = ψ 3 (x), we have:

tr(XY ) = 1 n -2 tr(Xµ1 + X1µ -2Xdiag(µ)) = - 2 n -2 tr(Xdiag(µ)) = - 2 n -2 n i=1 µ i n j=1 X ij = 0, tr(XZ ) = x n(n -1) tr(X11 -X) = 0, tr(Y Z ) = x n(n -1)(n -2) tr(ψ 2 (µ)11 -ψ 2 (µ)) = x n(n -1)(n -2)
tr(µ1 ) = 0, so the direct sum is orthogonal for the Frobenius inner product.

Irreducibility

The vector spaces ker φ 1 , im ψ 2 , im ψ 3 are stable because φ 1 , ψ 2 , ψ 3 are equivariant. Then, im ψ 2 , im ψ 3 are irreducible as images of irreducible modules by injective maps. Let us show that ker φ 1 = {X ∈ Hol(n)|X1 = 0} is irreducible. Note that ker φ 1 is generated by matrices E ijkl with distinct i, j, k, l ∈ {1, ..., n}, defined by:

E ijkl = E ij + E kl -E il -E jk = (e i -e k )(e j -e l ) + (e j -e l )(e i -e k ) = E jilk = -E kjil = -E ilkj . Indeed, the subfamily {E 12kl | 3 k < l n} ∪ {E 132l | 4 l n} is free because if (λ kl ) 3 k<l n ∈ R ( n-2 2 ) and (λ 2l ) l 4 n ∈ R n-3 are coefficients such that 3 k<l n λ kl E 12kl + 4 l n λ l E 132l = 0, then for 3 k < l n, the (k, l)-th coefficient is λ kl = 0 and the (2, l)-th coefficient is λ l = 0. Since it is of cardinal n-2 2 + n -3 = n(n-3) 2 = dim ker φ 1 ,
this subfamily is a basis so the whole family generates ker φ 1 . Moreover, all the matrices E ijkl are in the same orbit under the action of the permutation group S(n). Indeed, if E rstu is another one, it suffices to apply a permutation σ ∈ S(n) such that σ

(i) = r, σ(j) = s, σ(k) = t and σ(l) = u to get E rstu = P σ E ijkl P σ .
We are ready to prove that ker φ 1 is irreducible. Let W ⊆ ker φ 1 , W = {0}, and let X ∈ W , X = 0. It suffices to show that one E ijkl ∈ W and we will have ker φ 1 ⊆ W . First, there exist distinct indexes i, j, k ∈ {1, ..., n} such that X ik = X jk , otherwise X would belong to span(11 -I n ) so it would be null. Up to permutation, we can as-

sume that i = 1, j = 2 and k = 3. We define Y = X -(1, 2) • X = Å 0 A A 0 ã ∈ W where A = Å a 3 • • • a n -a 3 • • • -a n ã ∈ Mat(2, n -2)
with n i=3 a i = 0 and (a 3 , ..., a n ) = 0 since a 3 = X 13 -X 12 = 0. Hence, there exist 3 k < l n such that a k = a l so we can define

Z = 1 a k -a l (Y -(k, l) • Y ) = E 1k2l ∈ W .
Therefore, W = ker φ 1 and ker φ 1 is irreducible.

Isomorphic modules A simple verification on the dimensions shows that the three terms cannot be isomorphic. Indeed, dim ker φ 1 = n(n-3) 2 , dim im ψ 2 = n -1 > dim im ψ 3 = 1 so for n 5, dim ker φ 1 > dim im ψ 2 and for n ∈ {3, 4}, dim im ψ 2 > dim ker φ 1 = 1.

General form So the invariant inner products are positive weighted sums of the restriction of the Frobenius inner product to each term. Hence, the S(n)-invariant inner products on Hol(n) are given for all X ∈ Hol(n) by:

ϕ(X, X) = α 1 tr(X 2 1 ) + α 2 tr(X 2 2 ) + α 3 tr(X 2 3 ) (11.2) = α 1 tr(X 2 -X 2 2 -X 2 3 ) + α 2 tr(X 2 2 ) + α 3 tr(X 2 3 ) (11.3) = α 1 tr(X 2 ) + (α 2 -α 1 )tr(X 2 2 ) + (α 3 -α 1 )tr(X 2 3 ), (11.4)
with α 1 , α 2 , α 3 > 0. We would like to get a more comprehensive formula without X i 's, with only X. To do so, we compute tr(X 2 2 ) and tr(X 2 3 ) in function of X in two steps. First, we compute these quantities in function of µ ∈ R n 0 and x ∈ R, taking X 2 = ψ 2 (µ) and

X 3 = ψ 3 (x): tr(X 2 2 ) = 1 (n -2) 2 tr((µ1 + 1µ -2diag(µ)) 2 ) = 1 (n -2) 2 tr[sum(µ) 0 (µ1 + 1µ ) + 4diag(µ) 2 + nµµ + µ 2 11 -4µµ -2(µ 2 1 + 1(µ 2 ) )] = 1 (n -2) 2 (2n -4) µ 2 = 2 n -2 µ 2 , tr(X 2 3 ) = x 2 n 2 (n -1) 2 tr((11 -I n ) 2 ) = x 2 n 2 (n -1) 2 n(n -1) = x 2 n(n -1)
.

Second, we need to express µ and x in function of X = X 1 + ψ 2 (µ) + ψ 3 (x). On the one hand, sum(X) = 0 + 0 + x so x = sum(X). On the other hand, X1 = 0 + µ + x n 1 so µ = X1 -sum(X) n 1. We finish by computing µ 2 in function of X:

µ 2 = µ µ = Å 1 X - sum(X) n 1 ã Å X1 - sum(X) n 1 ã = sum(X 2 ) - 1 n sum(X) 2 .
Altogether, the inner product writes:

ϕ(X, X) = α tr(X 2 ) + β sum(X 2 ) + γ sum(X) 2 , (11.5) with α = α 1 , β = 2 n-2 (α 2 -α 1 ) and γ = 1 n(n-1) (α 3 -α 1 ) -2 n(n-2) (α 2 -α 1 ). The inverse relations write α 1 = α, α 2 = α + n-2
2 β and α 3 = α + (n -1)(β + nγ) as expected.

Proof of Example 3.10

Example 3.10 (S(n)-invariant inner products on Sym(n)) The irreducible decomposition of Sym(n) with respect to the congruence action of S(n), i.e. of permutation matrices, is

Sym(n) = ker φ 1 ⊥ ⊕ im ψ 2 ⊥ ⊕ im ψ 3 Hol(n) ⊥ ⊕ im ψ 4 ⊥ ⊕ im ψ 5 Diag(n)
ker φ 1 ⊕ 2R n 0 ⊕ 2R where φ 1 , ψ 2 , ψ 3 were defined in the previous section and: 4.

ψ 4 : λ ∈ R n 0 -→ diag(λ) ∈ Diag(n), 5. ψ 5 : y ∈ R -→ y n I n ∈ Diag(n)
, with ψ 4 , ψ 5 injective. The corresponding orthogonal projections are:

• π 5 : X ∈ Sym(n) -→ X 5 = ψ 5 (tr(X)) ∈ im ψ 5 , • π 4 : X ∈ Sym(n) -→ X 4 = Diag(X -X 5 ) ∈ im ψ 4 , • π 3 : X ∈ Sym(n) -→ X 3 = p 3 (X -Diag(X)) ∈ im ψ 3 , • π 2 : X ∈ Sym(n) -→ X 2 = p 2 (X -Diag(X)) ∈ im ψ 2 , • π 1 : X ∈ Sym(n) -→ X 1 = p 1 (X -Diag(X)) ∈ ker φ 1 . The S(n)-invariant inner products on Sym(n) are given for all X = X 1 + ψ 2 (µ) + ψ 3 (x) + ψ 4 (λ) + ψ 5 (y) ∈ Sym(n) by: ϕ(X, X) = α 1 tr(X 2 1 ) + S 11 µ 2 + Σ 11 x 2 + S 22 λ 2 + Σ 22 y 2 + 2S 12 λ µ + 2Σ 12 xy = α tr(X 2 ) + β sum(X 2 ) + γ sum(X) 2 + δ tr(Diag(X) 2 ) + ε sum(Diag(X)X) + ζ tr(X) 2 + η tr(X) sum(X),
where α 1 = α > 0 and the SPD matrices are S =

Å 2 n-2 α + β β + ε 2 β + ε 2 α + β + δ + ε ã ∈ Sym + (2) and Σ = 1 n Å 1 n-1 α + β + nγ β + ε 2 + n(γ + η 2 ) β + ε 2 + n(γ + η 2 ) α + β + δ + ε + n(γ + ζ + η) ã ∈ Sym + (2).
Proof of Example 3.10: S(n)-invariant inner products on Sym(n), n 3.

Direct sum

Based on the study of the action of S(n) on Hol(n) and R n Diag(n), the direct sum and the orthogonal projections are clear.

Irreducibility

The terms in Hol(n) were treated in the previous example. The spaces im ψ 4 and im ψ 5 are irreducible modules as images by equivariant maps of irreducible modules.

Isomorphic modules On the one hand, im ψ 2 and im ψ 4 are isomorphic to R n 0 as modules. On the other hand, im ψ 3 and im ψ 5 are isomorphic to R as modules. None of them is isomorphic to ker φ 1 because they have different dimensions.

General form Therefore, Theorem 3.4 ensures that the S(n)-invariant inner products on Sym(n) are given for all (11.7) where α 1 > 0, S ∈ Sym + (2) and Σ ∈ Sym + (2), because tr(X 2 1 ) = tr(X 2 ) -5 i=2 tr(X 2 i ), because the computations of tr(ψ 2 (µ) 2 ) = 2 n-2 µ 2 and tr(ψ 3 (x) 2 ) = x 2 n(n-1) made previously are still valid and because tr(X 2 4 ) = λ 2 and tr(X 2 5 ) = y 2 n . To get an explicit formula in function of X, we need to compute µ, λ ∈ R n 0 and x, y ∈ R:

X = X 1 + ψ 2 (µ) + ψ 3 (x) + ψ 4 (λ) + ψ 5 (y) ∈ Sym(n) by: ϕ(X, X) = α 1 tr(X 2 1 ) + S 11 µ 2 + Σ 11 x 2 + S 22 λ 2 + Σ 22 y 2 + 2S 12 λ µ + 2Σ 12 xy (11.6) = α 1 tr(X 2 ) + Å S 11 - 2 n -2 α 1 ã µ 2 + Å Σ 11 - 1 n(n -1) α 1 ã x 2 + (S 22 -α 1 ) λ 2 + Å Σ 22 - 1 n α 1 ã y 2 + 2S 12 λ µ + 2Σ 12 xy,
tr(X) = 0 + 0 + 0 + 0 + y, Diag(X) = 0 + 0 + 0 + diag(λ) + y n I n , sum(X) = 0 + 0 + x + 0 + y, X1 = 0 + µ + x n 1 + λ + y n 1, so y = tr(X), diag(λ) = Diag(X) -y n I n , x = sum(X) -tr(X) and µ = X1 -λ -x+y n 1. Then: µ 2 = 1 X -λ - x + y n 1 X1 -λ - x + y n 1 (11.8) = Å sum(X 2 ) + λ 2 - (x + y) 2 n -2λ X1 ã , ( 11.9)
where: (11.11)

λ 2 = (1 Diag(X) - y n 1 )(Diag(X)1 - y n 1) (11.10) = tr(Diag(X) 2 ) - 1 n tr(X) 2 ,
λ X1 = (1 Diag(X) - y n 1 )X1 = sum(XDiag(X)) - 1 n sum(X)tr(X).
(11.12)

So we obtain: (11.16)

µ 2 = sum(X 2 ) + tr(Diag(X) 2 ) - 1 n tr(X) 2 - 1 n sum(X) 2 -2sum(XDiag(X)) + 2 n sum(X)tr(X), (11.13) x 2 = sum(X) 2 + tr(X) 2 -2sum(X)tr(X), (11.14) λ 2 = tr(Diag(X) 2 ) - 1 n tr(X) 2 , (11.15) y 2 = tr(X) 2 ,
λ µ = λ X1 -λ 2 (11.17) = sum(XDiag(X)) - 1 n sum(X)tr(X) -tr(Diag(X) 2 ) + 1 n tr(X) 2 , (11.18) xy = sum(X)tr(X) -tr(X) 2 . (11.19)
Altogether, the inner product writes:

ϕ(X, X) = α tr(X 2 ) + β sum(X 2 ) + γ sum(X) 2 + δ tr(Diag(X) 2 ) + ε sum(Diag(X)X) + ζ tr(X) 2 + η tr(X) sum(X), with: α = α 1 , β = - 2 n -2 α 1 + S 11 , γ = Å 2 n(n -2) - 1 n(n -1) ã 1 (n-2)(n-1) α 1 - 1 n S 11 + Σ 11 , δ = - Å 1 + 2 n -2 ã n n-2 α 1 + S 11 + S 22 -2S 12 , ε = 4 n -2 α 1 -2S 11 + 2S 12 , ζ = Å 2 n(n -2) - 1 n(n -1) + 1 n - 1 n ã 1 (n-2)(n-1) α 1 - 1 n S 11 + Σ 11 - 1 n S 22 + Σ 22 + 2 n S 12 -2Σ 12 , η = Å - 4 n(n -2) + 2 n(n -1) ã -2 (n-2)(n-1) α 1 + 2 n S 11 -2Σ 11 - 2 n S 12 + 2Σ 12 .
The inverse relations are:

α 1 = α, S 11 = 2 n -2 α + β, Σ 11 = γ + 1 n β + 1 n(n -1) α, S 12 = β + ε 2 , S 22 = α + β + δ + ε, Σ 12 = 1 n β + ε 2 + γ + η 2 , Σ 22 = 1 n (α + β + δ + ε) + γ + ζ + η,
as expected.

Proof of Example 3.12

Example 3.12 (S(n)-invariant inner products on Mat(n)) The irreducible decomposition of Mat(n) with respect to the congruence action of S(n), i.e. of permutation matrices, is:

Mat(n) = ker φ 1 ⊥ ⊕ im ψ 2 ⊥ ⊕ im ψ 3 Hol(n) ⊥ ⊕ im ψ 4 ⊥ ⊕ im ψ 5 Diag(n) Sym(n) ⊥ ⊕ ker φ 6 ⊥ ⊕ im ψ 7 Skew(n) ker φ 1 ⊕ ker φ 6 ⊕ 3R n 0 ⊕ 2R,
where φ 1 , ψ 2 , ψ 3 , ψ 4 , ψ 5 were defined in the previous sections and: 6.

φ 6 : X ∈ Skew(n) -→ X1 ∈ R n 0 , 7. ψ 7 : ν ∈ R n 0 -→ 1 n (ν1 -1ν ) ∈ Skew(n),
with φ 6 surjective and ψ 7 injective. The corresponding orthogonal projections are:

• Π 7 : X ∈ Mat(n) -→ X 7 = ψ 7 Ä X-X 2 1 ä ∈ im ψ 7 , • Π 6 : X ∈ Mat(n) -→ X 6 = X-X 2 -X 7 ∈ ker φ 6 , • Π 5 : X ∈ Mat(n) -→ X 5 = π 5 Ä X+X 2 ä = ψ 5 (tr(X)) ∈ im ψ 5 , • Π 4 : X ∈ Mat(n) -→ X 4 = π 4 Ä X+X 2 ä = Diag(X -X 5 ) ∈ im ψ 4 , • Π 3 : X ∈ Mat(n) -→ X 3 = π 3 Ä X+X 2 ä ∈ im ψ 3 , • Π 2 : X ∈ Mat(n) -→ X 2 = π 2 Ä X+X 2 ä ∈ im ψ 2 , • Π 1 : X ∈ Mat(n) -→ X 1 = π 1 Ä X+X 2 ä ∈ ker φ 1 .
The S(n)-invariant inner products on Mat(n) are given for all

X = X 1 + ψ 2 (µ) + ψ 3 (x) + ψ 4 (λ) + ψ 5 (y) + X 6 + ψ 7 (ν) ∈ Mat(n) by: ϕ(X, X) = α 1 tr(X 2 1 ) + S 11 µ 2 + Σ 11 x 2 + S 22 λ 2 + Σ 22 y 2 + α 2 tr(X 2 6 ) + S 33 ν 2 + 2S 12 λ µ + 2S 23 ν λ + 2S 13 µ ν + 2Σ 12 xy = α tr(XX ) + α tr(X 2 ) + β sum(X 2 ) + β sum(XX ) + β sum(X X) + γ sum(X) 2 + δ tr(Diag(X) 2 ) + ζ tr(X) 2 + η tr(X) sum(X) + ε sum(Diag(X)X) + ε sum(Diag(X)X ),
where

α 1 = α+α 2 > 0, α 2 = α-α 2
> 0 and the SPD matrices are:

S = Ñ 2 n-2 α+α 2 + β + β + β β + β + β + 1 2 (ε + ε ) β -β 4 β + β + β + 1 2 (ε + ε ) α+α 2 + β + β + β + δ + ε + ε β -β 4 + ε -ε 2 β -β 4 β -β 4 + ε -ε 2 2α + β -β -β é ∈ Sym + (3), Σ = 1 n Ç 1 n-1 α+α 2 + β + β + β + nγ β + β + β + 1 2 (ε + ε ) + n(γ + η 2 ) β + 1 2 (ε + ε ) + n(γ + η 2 ) α+α 2 + β + β + β + δ + ε + ε + n(γ + ζ + η) å ∈ Sym + (2).
Proof of Example 3.12: S(n)-invariant inner products on Mat(n), n 3.

Direct sum

Given the previous sections, we only need to check the direct sum Skew(n) = ker φ 6 ⊕ im ψ 7 . Since ψ 7 (ν)1 = ν for all ν ∈ R n 0 , φ 6 is surjective and ψ 7 is injective. Since

1 X-X 2 1 = 1 2 (sum(X) -sum(X )) = 0, X-X 2 1 ∈ R n 0 so Π 7 is well defined. Hence, Ä X-X 2 -X 7 ä 1 = X-X 2 1 -ψ 7 Ä X-X 2 1 ä 1 = 0 so Π 6 is well defined and Skew(n) = ker φ 6 + im ψ 7 . Since dim ker φ 6 + dim im ψ 7 = (dim Skew(n) -(n -1)) + (n -1) = dim Skew(n), the sum is a direct sum. Moreover, if X ∈ ker φ 6 and Y = ψ 7 (ν) ∈ im ψ 7 with ν ∈ R n 0 , then: tr(Y X ) = 1 n tr(ν1 X -1ν X ) = 1 n tr(1ν X) = 0,
so the sum is orthogonal for the Frobenius inner product. The other projections Π 1 , ..., Π 5 are the compositions of the previous projections π 1 , ..., π 5 with the projection onto symmetric matrices X ∈ Mat(n) -→ X+X 2 .

Irreducibility

The vector spaces ker φ 6 and im ψ 7 are stable because φ 6 and ψ 7 are equivariant. Then, im ψ 7 is irreducible as image of an irreducible module by an injective map. Let us show that ker φ 6 = {X ∈ Skew(n)|X1 = 0} is irreducible. Note that ker φ 6 is generated by matrices F ijkl with i, j, k, l ∈ {1, ..., n}, i = k and j = l and (i = j or k = l), defined by:

F ijkl = (e i -e k )(e j -e l ) -(e j -e l )(e i -e k )
= (e i e j -e j e i ) + (e k e l -e l e k ) -(e i e l -e l e i ) -(e k e j -e j e k )

= F jilk = -F kjil = -F ilkj . Indeed, the subfamily (F 11kl ) 2 k<l n is free because if (λ kl ) 2 k<l n ∈ R ( n-1 2 ) are coefficients such that 2 k<l n λ kl F 11kl = 0, then the (k, l)-th coefficient is λ kl = 0. Since it is of cardinal n-1 2 = (n-1)(n-2) 2
= dim ker φ 6 , this subfamily is a basis so the whole family generates ker φ 6 .

For n = 3, dim ker φ 6 = 1 so ker φ 6 is irreducible. We assume that n 4. Let W ⊆ ker φ 6 be a submodule, W = {0}. Let X ∈ W , X = 0. By absurdum, if for all distinct i, j, k, l ∈ {1, ..., n}, X ik -X jk = X il -X jl , then:

∀i = j = l = i, 0 = n k=1 (X ik -X jk ) = -X ji + X ij + k =i,j (X ik -X jk ) = 2X ij + k =i,j (X il -X jl ) = 2X ij + (n -2)(X il -X jl ), ∀j = l, 0 = n i=1 2X ij = 2X lj + i =j,l 2X ij = 2X lj - i =j,l (n -2)(X il -X jl ) = 2X lj -(n -2)[(0 -X jl ) -(n -2)X jl ] = n(n -3)X jl , so X would be 0. Therefore, there exist distinct i, j, k, l ∈ {1, ..., n} such that X ik -X jk = X il -X jl . Up to a permutation, we can assume that (i, j, k, l) = (1, 2, 3, 4). Let Y = X -(1, 2)• X = Å B A -A 0 ã ∈ W where B = Å 0 a 2 -a 2 0 ã ∈ Skew(2) and A = Å a 3 a 4 • • • a n -a 3 -a 4 • • • -a n ã ∈ Mat(2, n -2). Hence a 3 = X 13 -X 23 = X 14 -X 24 = a 4 so F 1324 = 1 a 3 -a 4 (Y -(3, 4) • Y ) ∈ W .
By permutations, all F ijkl ∈ W with distinct i, j, k, l ∈ {1, ..., n}.

Moreover, there exist i = j such that X ij = 0. Up to a permutation, we can assume

that (i, j) = (1, 2). Let Y = X -(1, 2) • X = Å B A -A 0 ã ∈ W where B = Å 0 a 2 -a 2 0 ã ∈ Skew(2) and A = Å a 3 a 4 • • • a n -a 3 -a 4 • • • -a n ã ∈ Mat(2, n -2) with a 2 = 2X 12 = 0. Let σ = (3, 4, ..., n) ∈ S(n), let Z k = σ k • Y for k ∈ {0, ..., n -3} and let Z = 1 a 2 n-3 k=0 Z k = â 0 n -2 -1 • • • -1 -(n -2) 0 1 • • • 1 1 -1 0 • • • 0 . . . . . . . . . . . . . . . 1 -1 0 • • • 0 ì . Finally, F 1123 = 1 n-2 (Z -n k=4 F 132k ) ∈ W and by
permutations, all F iikl ∈ W for distinct i, k, l ∈ {1, ..., n}. So W = ker φ 6 and ker φ 6 is irreducible.

Isomorphic modules

The module im ψ 7 is isomorphic to R n 0 , thus to im ψ 2 and im ψ 4 . We still have im ψ 3 ψ 5 . For n 4, the other dimensions don't match. For n = 3, dim ker φ 6 = 1 but the permutation (1, 2) changes X ∈ ker φ 6 into -X while it does not act on im ψ 3 or im ψ 5 . So there is no other isomorphism.

General form Therefore, Theorem 3.4 ensures that the S(n)-invariant inner products on Mat(n) are given for all

X = X 1 + ψ 2 (µ) + ψ 3 (x) + ψ 4 (λ) + ψ 5 (y) + X 6 + ψ 7 (ν) ∈ Mat(n) by: ϕ(X, X) = α 1 tr(X 2 1 ) + S 11 µ 2 + Σ 11 x 2 + S 22 λ 2 + Σ 22 y 2 + α 2 tr(X 2 6 ) + S 33 ν 2 + 2S 12 λ µ + 2S 23 ν λ + 2S 13 µ ν + 2Σ 12 xy = (α 1 + α 2 )tr(XX ) + (α 1 -α 2 )tr(X 2 ) + Å S 11 - 2 n -2 α 1 ã µ 2 + Å Σ 11 - 1 n(n -1) α 1 ã x 2 + (S 22 -α 1 ) λ 2 + Å Σ 22 - 1 n α 1 ã y 2 + (S 33 -2α 1 -2α 2 ) ν 2 + 2S 12 λ µ + 2S 13 ν λ + 2S 23 µ ν + 2Σ 12 xy,
where α 1 , α 2 > 0, S ∈ Sym + (3) and Σ ∈ Sym + (2), because:

• tr(X 2 1 ) = tr(XX ) -5 i=2 tr(X 2 i ) -X-X 2 2 = tr(XX )+tr(X 2 ) 2 -5 i=2 tr(X 2 i ), • tr(ψ 2 (µ) 2 ) = 2 n-2 µ 2 , tr(ψ 3 (x) 2 ) = x 2 n(n-1) , tr(X 2 4 ) = λ 2 , tr(X 2 5 ) = y 2 n as seen earlier, • tr(X 6 X 6 ) = X-X 2 2
-tr(X 7 X 7 ) = tr(XX )-tr(X 2 ) 2 -tr(X 7 X 7 ) and tr(X 7 X 7 ) = 2 ν 2 .

The previous expressions of µ, λ ∈ R n 0 and x, y ∈ R in function of X are still valid if we replace X by X+X Updating Equations (11.8) to (11.19), we get:

2 ∈ Sym(n). So y = tr( X+X 2 ) = tr(X), diag(λ) = Diag( X+X 2 ) -y n I n = Diag(X) -y n I n , x = sum( X+X 2 ) -tr( X+X 2 ) = sum(X) -tr(X) and µ = X+X 2 1 -λ -x+y n 1.
µ 2 = 1 2 sum(X 2 ) + 1 4 sum(XX ) + 1 4 sum(X X) + tr(Diag(X) 2 ) - 1 n tr(X) 2 - 1 n sum(X) 2 -sum(Diag(X)X) -sum(Diag(X)X ) + 2 n sum(X)tr(X), x 2 = sum(X) 2 + tr(X) 2 -2sum(X)tr(X), λ 2 = tr(Diag(X) 2 ) - 1 n tr(X) 2 , y 2 = tr(X) 2 , λ µ = 1 2 sum(Diag(X)X) + 1 2 sum(Diag(X)X ) - 1 n sum(X)tr(X) -tr(Diag(X) 2 ) + 1 n tr(X) 2 , xy = sum(X)tr(X) -tr(X) 2 .
In addition, ν = X-X 2 1 so:

ν 2 = 1 4 sum((X -X ) (X -X )) = 1 2 sum(X 2 ) - 1 4 sum(XX ) - 1 4 sum(X X), ν λ = 1 2 sum Å (X -X ) Å Diag(X) - tr(X) n I n ãã = 1 2 sum(Diag(X)X ) - 1 2 sum(Diag(X)X), ν µ = -ν λ + 1 2 sum Å (X -X ) Å X + X 2 - x + y n I n ãã = 1 2 sum(Diag(X)X) - 1 2 sum(Diag(X)X ) + 1 4 sum(X X) - 1 4 sum(XX ).
Altogether, the inner product writes:

ϕ(X, X) = α tr(XX ) + α tr(X 2 ) + β sum(X 2 ) + β sum(XX ) + β sum(X X) + γ sum(X) 2 + δ tr(Diag(X) 2 ) + ζ tr(X) 2 + η tr(X) sum(X) + ε sum(Diag(X)X) + ε sum(Diag(X)X ), (11.20)
where:

α = α 1 + α 2 , α = α 1 -α 2 , β = Å - 1 n -2 -1 ã α 1 -α 2 + 1 2 (S 11 + S 33 ), β = 1 2 Å - 1 n -2 + 1 ã α 1 + 1 2 α 2 + 1 4 (S 11 -S 33 -2S 13 ), β = 1 2 Å - 1 n -2 + 1 ã α 1 + 1 2 α 2 + 1 4 (S 11 -S 33 + 2S 13 ), γ = Å 2 n(n -2) - 1 n(n -1) ã α 1 - 1 n S 11 + Σ 11 , δ = Å - 2 n -2 -1 ã α 1 + S 11 + S 22 -2S 12 , ε = 2 n -2 α 1 -S 11 + S 12 -S 23 + S 13 , ε = 2 n -2 α 1 -S 11 + S 12 + S 23 -S 13 , ζ = Å 2 n(n -2) - 1 n(n -1) + 1 n - 1 n ã α 1 - 1 n S 11 + Σ 11 - 1 n S 22 + Σ 22 + 2 n S 12 -2Σ 12 , η = Å - 4 n(n -2) + 2 n(n -1) ã α 1 + 2 n S 11 -2Σ 11 - 2 n S 12 + 2Σ 12 .
The inverse relations are:

α 1 = α + α 2 , α 2 = α -α 2 , S 13 = β -β 4 , S 11 = 2 n -2 α + α 2 + β + β + β , S 33 = 2α + β -β -β , Σ 11 = 1 n(n -1) α + α 2 + 1 n (β + β + β ) + γ, S 12 = β + β + β + 1 2 (ε + ε ), S 22 = α + α 2 + β + β + β + δ + ε + ε , S 23 = β -β 4 + ε -ε 2 , Σ 12 = 1 n Å β + β + β + 1 2 (ε + ε ) ã + γ + η 2 , Σ 22 = 1 n Å α + α 2 + β + β + β + δ + ε + ε ã + γ + ζ + η.

Proofs of Chapter 4

In this section, we prove the results of Chapter 4: O(n)-invariant metrics.

Proof of Lemma 4.6

Lemma 4.6 (Characterization of inner products on symmetric matrices invariant under

D ± (n) or S ± (n)) Let •|• : Sym(n) × Sym(n) -→ R be an inner product on symmetric matrices.
(a) It is D ± (n)-invariant if and only if there exist n(n-1) 2 positive real numbers α ij = α ji > 0 for i = j and a matrix S ∈ Sym + (n) such that:

∀X ∈ Sym(n), X|X = i =j α ij X 2 ij + i,j S ij X ii X jj .
(b) It is S ± (n)-invariant if and only if there exist (α, β, γ) ∈ R 3 with α > 0, γ > β and γ + (n -1)β > 0 such that:

∀X ∈ Sym(n), X|X = γ n i=1 X 2 ii + α i =j X 2 ij + β i =j X ii X jj . with E ii , E ij for i = j defined by E ii (k, l) = δ ik δ il and E ij (k, l) = δ ik δ jl +δ il δ jk √ 2 . As κ Σ (X, Y ) = R Σ (X,Y,X,Y ) X 2 Y 2 -X|Y 2 , we have κ Σ (Σ 1/2 E ij Σ 1/2 , Σ 1/2 E kl Σ 1/2 ) = 1 2α tr((E ij E kl ) 2 -(E ij E kl )(E ij E kl
) ) so we only need to compute a few expressions. In the following equalities, when an elementary matrix E has two different indexes, they are assumed to be distinct:

• E ii E jj = δ ij C ij hence E ii E jj 2 2 = δ ij , • E ii E jk = 1 √ 2 (δ ij C ik + δ ik C ij ) hence E ii E jk 2 2 = 1 2 (δ ij + δ ik ), • E ij E kl = 1 2 (δ jk C il + δ ik C jl + δ jl C ik + δ il C jk ) hence E ij E kl 2 2 = 1 4 (δ ik + δ il + δ jk + δ jl ), • (E ii E jj ) 2 = δ ij C ij hence tr((E ii E jj ) 2 ) = δ ij , • (E ii E jk ) 2 = 0 hence tr((E ii E jk ) 2 ) = 0, • (E ij E kl ) 2 = 1 4 (δ jk δ il (C il + C jk ) + δ jl δ ik (C ik + C jl )), hence tr((E ij E kl ) 2 ) = 1 2 (δ jk δ il + δ jl δ ik ). • κ In (E ii , E jj ) = 0, • κ In (E ii , E jk ) = -1 4α (δ ij + δ ik ), • κ In (E ij , E kl ) = -1 8α ((δ ik -δ jl ) 2 + (δ il -δ jk ) 2 ). Hence the non null terms are κ In (E ii , E ij ) = -1 4α and κ In (E ij , E ik ) = -1 8α .
Secondly, for β = 0, we use the isometry f p,1 : the values are the same if we replace Σ

1/2 E ij Σ 1/2 by (d Σ f p,1 ) -1 (f p,1 (Σ) 1/2 E ij f p,1 (Σ) 1/2 ) = Σ 1/2 E β ij Σ 1/2 .
To prove that κ ∈ [-1/2α; 0], it suffices to note that for normed and orthogonal

X, Y ∈ Sym(n), we have κ In (X, Y ) = -1 4α [X, Y ] 2 2 . Diagonalizing X = P ∆P and denoting Z = P Y P , from (d i -d j ) 2 2(d 2 i + d 2 j ) 2 D 2 , we get κ In (X, Y ) = κ In (∆, Z) = -1 4α i =j (d i -d j ) 2 Z 2 ij -1 2α D 2 Z 2 = -1 2α . This bound is reached for X = 1 √ 2 (E ii -E jj ) and Y = E ij .

Proof of Bures-Wasserstein Levi-Civita connection

We prove that the Levi-Civita connection of the Bures-Wasserstein metric is (∇ Table 4.7. Let X, Y be vector fields on Sym + (n). The Levi-Civita connection is computed in [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF]. With our notation X 0 = S Σ (X) defined by X = ΣX 0 + X 0 Σ, their result writes ∇

X Y ) |Σ = (∂ X Y ) |Σ -(X 0 ΣY 0 + Y 0 ΣX 0 ) for all X, Y ∈ T Σ Sym + (n) at Σ ∈ Sym + (n).

Proof of Levi-Civita connection in

X Y = ∂ X Y -{X 0 Y + Y 0 X} S + {ΣX 0 Y 0 + ΣY 0 X 0 } S where {A} S = 1 2 (A + A ) is the symmetric part of the matrix A. It is easy to see that it rewrites ∇ X Y = ∂ X Y -(X 0 ΣY 0 + Y 0 ΣX 0 ) which is a simpler expression.
We would like to give a different proof that relies on the geometry of the horizontal distribution. According to [O'Neill, 1966,

Lemma 1], dπ(∇ G X h Y h ) = ∇ X Y
, where ∇ G = ∂ is the Levi-Civita connection of the Frobenius metric G on GL(n), i.e. the derivative of coordinates in the canonical basis of matrices. We differentiate the equality

X h = (X 0 • π) × Id GL(n) on GL(n): (∇ G X h Y h ) |A = ∂ X h A (Y 0 • π)A + Y 0 π(A) X h A = (∂ X π(A) Y 0 )A + Y 0 π(A) X 0 π(A) A, (∇ X Y ) |AA = d A π((∇ G X h Y h ) |A ) = AA (∂ X π(A) Y 0 ) + (∂ X π(A) Y 0 )AA + AA X 0 π(A) Y 0 π(A) + Y 0 π(A) X 0 π(A) AA , (∂ X Y ) |Σ = Σ(∂ X Σ Y 0 ) + (∂ X Σ Y 0 )Σ + X Σ Y 0 Σ + Y 0 Σ X Σ = Σ(∂ X Σ Y 0 ) + (∂ X Σ Y 0 )Σ + ΣX 0 Σ Y 0 Σ + Y 0 Σ X 0 Σ Σ + X 0 Σ ΣY 0 Σ + Y 0 Σ ΣX 0 Σ = (∇ X Y ) |Σ + X 0 Σ ΣY 0 Σ + Y 0 Σ ΣX 0 Σ . Finally, we find ∇ X Y = ∂ X Y -(X 0 ΣY 0 + Y 0 ΣX 0 ) as expected.

Proof of Bures-Wasserstein curvature

We prove that the Riemann curvature of the Bures-Wasserstein metric is R

Σ (X, Y, X, Y ) = 3 2 i,j d i d j d i +d j X 0 , Y 0 2 ij for all X, Y ∈ T Σ Sym + (n) at Σ ∈ Sym + (n), where [V, W ] = V W - W V is the Lie bracket of matrices. Proof of curvature in Table 4.7. Let X, Y ∈ T Σ Sym + (n) be tangent vectors at Σ ∈ Sym + (n). We would like to compute the sectional curvature κ(X, Y ) = R(X,Y,X,Y ) X 2 Y 2 -X|Y 2 , i.e. R(X, Y, X, Y ). Let X h , Y h ∈ H Σ 1/2
be the horizontal lifts of X, Y at Σ 1/2 and X 0 , Y 0 ∈ Sym(n) defined as explained above. We extend X h , Y h into vector fields by X h A := X 0 A and Y h A := Y 0 A. We do so because the formula we use to compute the curvature is based on a Lie bracket and can only be computed with fields. As the curvature is a tensor, it only depends on the values of X and Y at Σ so the way we extend the fields does not influence the result (but it simplifies the computation).

A first strategy to compute the curvature is to use the Levi-Civita connection via the

definition R(X, Y )Z = ∇ X ∇ Y Z -∇ Y ∇ X Z -∇ [X,Y ] Z.
It is tedious but doable. Another one consists in using the relation between the curvatures of the quotient metric (here, Bures-Wasserstein) and the original metric (here, Frobenius) found in [O'Neill, 1966], formula {4}. According to this formula, since the Euclidean metric is flat, the formula is

R Σ (X, Y, X, Y ) = 3 4 ver X h , Y h Σ 1/2
2 where ver :

X v + X h ∈ T GL(n) -→ X v ∈ V
is the vertical projection and •, • denotes the Lie bracket on vector fields of GL(n), which must be distinguished from the matrix Lie bracket

[V, W ] = V W -W V . Note that the right term only depends on X h Σ 1/2 and Y h Σ 1/2 because if f : GL + (n) -→ R is a map, then ver f X h , Y h Σ 1/2 = f (Σ 1/2 )ver X h , Y h Σ 1/2 + d Σ 1/2 f (Y h )ver(X h ) 0 .
The rest of the proof consists in computing ver

X h , Y h = X h , Y h -hor X h , Y h . On the one hand, X h , Y h A = Y 0 X h A -X 0 Y h A = -[X 0 , Y 0 ]A. On the other hand, let Z h A := hor X h , Y h A =: Z 0 AA A ∈ H A . Now, we can fix Σ ∈ Sym + (n) and A = Σ 1/2
. We take a spectral decomposition Σ = P DP and we denote with a prime all the previous matrices taken in the basis P of eigenvectors of Σ, e.g. X 0 = P X 0 P . Then:

Z Σ := d Σ 1/2 π( X h , Y h Σ 1/2 ) = Σ[X 0 , Y 0 ] -[X 0 , Y 0 ]Σ, Z h Σ 1/2 = (d Σ 1/2 π |H Σ 1/2 ) -1 (d Σ 1/2 π( X h , Y h Σ 1/2 )) = (d Σ 1/2 π |H Σ 1/2 ) -1 (Z Σ ), [Z 0 Σ ] ij = 1 d i + d j (D[X 0 , Y 0 ] -[X 0 , Y 0 ]D) ij = d i -d j d i + d j [X 0 , Y 0 ] ij , [Z h Σ 1/2 ] ij = d j [Z 0 Σ ] ij = d j d i -d j d i + d j [X 0 , Y 0 ] ij , (ver X h , Y h Σ 1/2 ) ij = ( X h , Y h Σ 1/2 ) ij -[Z h Σ 1/2 ] ij = -[X 0 , Y 0 ] ij d j -d j d i -d j d i + d j [X 0 , Y 0 ] ij = - 2d i d j d i + d j [X 0 , Y 0 ] ij , R Σ (X, Y, X, Y ) = 3 4 (ver X h , Y h Σ 1/2 ) 2 = 3 i,j d 2 i d j (d i + d j ) 2 î X 0 , Y 0 ó 2 ij = 3 2 i,j d i d j d i + d j î X 0 , Y 0 ó 2 ij ,
where P XP = DX 0 + X 0 D and P Y P = DY 0 + Y 0 D.

Proof of Bures-Waserstein geodesic parallel transport between commuting matrices

We prove that along a geodesic between commuting matrices Σ = P DP and Λ = P ∆P ∈ Sym + (n), the parallel transport of the Bures-Wasserstein metric writes:

Π Σ→Λ : T Σ Sym + (n) -→ T Λ Sym + (n) X -→ P δ i +δ j d i +d j [P XP ] ij P .
Proof of geodesic parallel transport between commuting matrices in Table 4.7. We want to prove that the geodesic parallel transport of the Bures-Wasserstein metric between two commuting matrices is Π Σ→Λ X = P

δ i +δ j d i +d j [P XP ] ij i,j
P where Σ = P DP and Λ = P ∆P ∈ Sym + (n). The geodesic parallel transport is O(n)-invariant so we only need to prove that

[Π D→∆ X] ij = δ i +δ j d i +d j X ij . The geodesic from D to ∆ is γ(t) = ((1 -t) √ D + t √ ∆) 2 . Let us define X(t) = ((1-t)d i +tδ i ) 2 +((1-t)d j +tδ j ) 2 d i +d j X ij i,j
and let us check that ∇ γ X = 0. We compute:

[X 0 (t)] ij = [X(t)] ij γ i (t) + γ j (t) = 1 d i + d j ((1 -t)d i + tδ i ) 2 + ((1 -t)d j + tδ j ) 2 X ij , γ(t) = 2( √ ∆ - √ D)((1 -t) √ D + t √ ∆), γ0 (t) = 1 2 γ(t)γ -1 (t) = 1 2 γ -1 (t) γ(t) = ( √ ∆ - √ D)((1 -t) √ D + t √ ∆) -1 , [ Ẋ(t)] ij = 2( √ δ i - √ d i )((1 -t)d i + tδ i ) + 2( δ j -d j )((1 -t)d j + tδ j ) 2 d i + d j ((1 -t)d i + tδ i ) 2 + ((1 -t)d j + tδ j ) 2 X ij = ( δ i -d i )((1 -t)d i + tδ i )[X 0 (t)] ij + [X 0 (t)] ij ( δ j -d j )((1 -t)d j + tδ j ) = [ γ0 (t)γ(t)X 0 (t) + X 0 (t)γ(t) γ0 (t)] ij , ∇ γ(t) X = Ẋ(t) -( γ0 (t)γ(t)X 0 (t) + X 0 (t)γ(t) γ0 (t)) = 0. So the geodesic parallel transport from Σ = P DP to Λ = P ∆P is Π Σ→Λ X = P δ i +δ j d i +d j [P XP ] ij i,j P .

Proof of Proposition 4.12

Proposition 4.12 (Parallel transport equation of Bures-Wasserstein metric) Let γ(t) the geodesic between γ(0) = Σ and γ(1) = Λ, and a vector X ∈ T Σ Sym + (n). We denote γ h (t) = (1 -t)Σ 1/2 + tΣ -1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 the horizontal lift of the geodesic γ. The two following statements are equivalent.

(i) The vector field X(t) defined along γ(t) is the parallel transport of X.

(ii) X(t) = γ(t)X 0 (t) + X 0 (t)γ(t) where X 0 (t) is a curve in Sym(n) satisfying the following ODE:

γ(t) Ẋ0 (t) + Ẋ0 (t)γ(t) + γ h (t) γh X 0 (t) + X 0 (t) γh γ h (t) = 0.
Proof of Proposition 4.12 (Parallel transport equation of Bures-Wasserstein metric). The geodesic parallel transport equation is ∇

γ(t) X = 0 along the geodesic γ(t) = γ h (t)γ h (t) between Σ and Λ ∈ Sym + (n), where γ h (t) = (1 -t)Σ 1/2 + tΣ -1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 . For a vector field X(t) on Sym + (n) defined along γ(t), we can define the horizontal lift X h (t) = X 0 (t)γ h (t) ∈ H γ h (t)
where X 0 (t) is defined by X(t) = γ(t)X 0 (t) + X 0 (t)γ(t). We are going to prove that X(t) is the geodesic parallel transport of X ∈ T Σ Sym + (n) if and only if X 0 (t) satisfies the following ODE: (11.21) To rewrite the geodesic parallel transport equation ∇ γ(t) X = 0, we need to compute the following derivatives:

γ(t) Ẋ0 (t) + Ẋ0 (t)γ(t) + γ h (t) γh X 0 (t) + X 0 (t) γh γ h (t) = 0.
Ẋ(t) = γ(t) Ẋ0 (t) + Ẋ0 (t)γ(t) + γ(t)X 0 (t) + X 0 (t) γ(t), γ(t) = γh γ h (t) + γ h (t) γh where γh = γ0 (t)γ h (t).
We can notice that g 1 (1) = 0 and g 1 (1) = 0. So if we prove that g 1 < 0 on (0, 1), then g 1 is decreasing thus positive, so g 1 is increasing thus negative, and finally g 0 is positive so g 0 increases.

i. If p + 1 = 0, then g 1 has the sign of p 2 (p -1) < 0, ii. if p + 1 > 0, then g 1 is positive before x 0 := p(p-1) (p+1)(p-2) < 0 and negative after, iii. if p + 1 < 0, then g 1 is negative before x 0 := p(p-1) (p+1)(p-2) > 1 and positive after. So we proved that F is increasing on (0, 1).

(b) Let us prove that F is increasing on (1, ∞). As x -→ 1 (x p -1) 2 is increasing, we only need to prove that h 0 is increasing on (1, ∞) so that F is increasing on (1, ∞) as product of two positive increasing functions. We derive successively:

• h 0 (x) = (x -1)h 1 (x) with h 1 (x) = (p + 2)x p -px p-1 + 2, • h 1 (x) = px p-2 ((p + 2)x -(p -1)).
We need to prove that h 1 > 0 on (1, ∞). As before, we distinguish the cases: i. If p + 2 < 0, then h 1 is negative before x 0 := p-1 p+2 > 1 and positive after. As

h 1 (x 0 ) = 2 -x p-1 0 > 1, we have h 1 > 0 on (1, ∞). ii. If p + 2 0, then h 1 is negative on (1, ∞). As lim x→∞ h 1 (x) = 2, we have h 1 > 0 on (1, ∞).
So we proved that F is increasing on (1, ∞) and therefore on (0, ∞).

Finally, we proved that F is non-decreasing if and only if p 1. As f = pow 1 2-p

• F , we can assert that if p 1, then f is non-decreasing, as expected.

Proof of Theorem 5.19

Theorem 5.19 (Sufficient condition for a balanced bilinear form to be a metric) Let •|• = Frob be the Frobenius inner product. Let g + , g -be deformed-Euclidean metrics respectively associated to univariate diffeomorphisms u and v. Then the balanced bilinear form g 0 is a metric.

Proof of Theorem 5.19 (Sufficient condition for a balanced bilinear form to be a metric). Let u, v : Sym + (n) -→ Sym(n) be two univariate diffeomorphisms onto the respective image of Sym + (n) by u and v. Let g + , g -be the respective deformed-Euclidean metrics by u and v. Hence for all Σ ∈ Sym + (n) and X, Y ∈ T Σ Sym + (n):

g + Σ (X, X) = tr(d Σ u(X) 2 ), g - Σ (Y, Y ) = tr(d Σ v(Y ) 2 ).
Hence the flat parallel transports Π + and Π -do not depend on the curve, they are simply given by the differentials of u and v:

Π + Σ→Λ X = (d Λ u) -1 (d Σ u(X)), Π - Σ→Λ Y = (d Λ v) -1 (d Σ v(Y )), d Σ v(X) = d u(Σ) w(d Σ u(X)) with w = v • u -1
, the map u is an isometry between g ME(u,v) and c g ME(Id,w) with c = w (1) u (1)v (1) . So it suffices to compute the curvature of g ME(Id,w) and to conclude by pullback and scaling.

Let Σ = P DP ∈ Sym + (n). We denote u ij = u [1] (d i , d j ), u ijk = u [2] (d i , d j , d k ) and analogously for v and w.

The curvature of g := g ME(Id,w) can be computed the same way as shown in [START_REF] Michor | The Curvature of the Bogoliubov-Kubo-Mori Scalar Product on Matrices[END_REF] for the metric BKM = MPE(1, 0) = ME(Id, log). Following [START_REF] Michor | The Curvature of the Bogoliubov-Kubo-Mori Scalar Product on Matrices[END_REF], we introduce G Σ (X) = d Σ w(X) and Γ Σ (X, Y ) such that:

g Σ (X, Y ) = 1 w (1) tr(d Σ w(X)Y ) = 1 w (1) tr(G Σ (X)Y ), ∇ X Σ Y = d Σ Y (X) + Γ Σ (X, Y ),
where ∇ is the Levi-Civita connection of g. Note that

G Σ : Sym(n) -→ Sym(n) is a linear isomorphism and Γ Σ is symmetric. According to Lemma 5.3, [G D (X)] ij = w ij X ij . Then the Riemann curvature tensors are defined by R(X, Y )Z = ∇ X ∇ Y Z -∇ Y ∇ X Z -∇ [X,Y ] Z and R(X, Y, Z, T ) = -g(R(X, Y )Z, T
) so we can write R in function of Γ and dΓ or in function of G and dG [START_REF] Michor | The Curvature of the Bogoliubov-Kubo-Mori Scalar Product on Matrices[END_REF]:

R(X, Y )Z = dΓ(X)(Y, Z) -dΓ(Y )(X, Z) + Γ(X, Γ(Y, Z)) -Γ(Y, Γ(X, Z)) = - 1 4 G -1 (dG(X)(G -1 (dG(Y )(Z)))) + 1 4 G -1 (dG(Y )(G -1 (dG(X)(Z)))), R(X, Y, Z, T ) = 1 4w (1) tr dG(X)(G -1 (dG(Y )(Z))) -dG(Y )(G -1 (dG(X)(Z))) T .

So we only need to express

d Σ G(X)(Y ) = H Σ w(X, Y ). Lemma 5.3 gives H Σ w(X, Y ) = P H D w(X , Y )P and [H D (X , Y )] ij = k w ijk (X ik Y jk + X jk Y ik ). Hence: tr(d Σ G(X)(G -1 Σ (d Σ G(Y )(Z)))T ) = tr(d D G(X )(G -1 D (d D G(Y )(Z )))T ) = i,j [d D G(X )(G -1 D (d D G(Y )(Z )))] ij T ij = i,j,k w ijk (X ik [G -1 D (d D G(Y )(Z ))] jk + X jk [G -1 D (d D G(Y )(Z ))] ik )T ij = 2 i,j,k w ijk X ik [G -1 D (d D G(Y )(Z ))] jk T ij = 2 i,j,k w ijk w jk X ik [d D G(Y )(Z )] jk T ij = 2 i,j,k,l w ijk w jkl w jk X ik (Y jl Z kl + Y kl Z jl )T ij . Therefore: R(X, Y, Z, T ) = 1 w (1) i,j,k,l w ijk w jkl 2w jk (X ik Y kl Z lj T ji + X ik Z kl Y lj T ji -Y ik X kl Z lj T ji -Y ik Z kl X lj T ji ).

Proof of Theorem 5.26

Theorem 5.26 (Riemannian operations of MPE metrics) Let α, β ∈ R such that α + β = 0, thus excluding log-Euclidean and power-affine metrics. Table 5.3 summarizes the formulae of the geodesics, the logarithm map and the distance in the particular case where Σ, Λ ∈ Sym + (n) and V ∈ T Σ Sym + (n) commute. They essentially reduce to the formulae of the α 0 -power-Euclidean metric with α 0 = α+β 2 . These formulae are generally not valid for noncommuting matrices. 5.3: Riemannian operations of Mixed-Power-Euclidean metrics for commuting matrices Proof of Theorem 5.26 (Riemannian operations of MPE metrics). We compute the geodesics, the logarithm map and the distance between commuting matrices. We show that the geodesics of the Mixed-Power-Euclidean metrics MPE(α, β) with α + β = 0 when the base point Σ ∈ Sym + (n) and the initial tangent vector

Geodesics γ

(Σ,V ) (t) = (Σ α 0 + t d Σ pow α 0 (V )) 1/α 0 Logarithm Log Σ (Λ) = (d Σ pow α 0 ) -1 (Λ α 0 -Σ α 0 ) Distance d(Σ, Λ) = 1 α 0 Λ α 0 -Σ α 0 Frob Table
X ∈ T Σ Sym + (n) commute is γ(t) = (Σ α 0 + t α 0 Σ α 0 -1 X) 1/p 0 where α 0 = α+β 2 = 0.
Once this is shown, the formulae of the logarithm and the distance are obvious so we omit the proofs. As the metric is O(n)invariant, we can assume that Σ and X are diagonal matrices.

First, we assume that α, β = 0. As MPE(α, β) is a balanced metric, the Levi-Civita connection is

∇ MPE(α,β) = 1 2 (pow * α ∇ E + pow * β ∇ E )
where ∇ E is the Euclidean connection on symmetric matrices. Since for any curve γ on Sym + (n), we have:

(pow * α ∇ E ) γ (t) γ = (d γ(t) pow α ) -1 (∇ E (d γ(t) pow α )(γ (t)) dpow α (γ )) = (d γ(t) pow α ) -1 (∇ E (γ α ) (t) (γ α ) ) = (d γ(t) pow α ) -1 ((γ α ) (t)), the geodesic equation ∇ MPE(α,β) γ γ = 0 rewrites: (d γ(t) pow α ) -1 ((γ α ) (t)) + (d γ(t) pow β ) -1 ((γ β ) (t)) = 0.
We compute: (11.25) As this expression is skew-symmetric in (α, β), the curve γ satisfies the geodesic equation.

γ(t) α = Σ α (I n + t α 0 Σ -1 X) α α 0 , (γ α ) (t) = α Σ α-1 X(I n + t α 0 Σ -1 X) α α 0 -1 , (γ α ) (t) = α(α -α 0 ) Σ α-2 X 2 (I n + t α 0 Σ -1 X) α α 0 -2 , (d γ(t) pow α ) -1 ((γ α ) (t)) = 1 α γ(t) 1-α (γ α ) (t) = α -β 2 Σ -1 X 2 (I n + t α 0 Σ -1 X) 1 α 0 -2 .
+ (n). The horizontal lift at Σ of X ∈ T C Cor + (n) is X # = hor(∆ Σ X∆ Σ ) with ∆ Σ = Diag(Σ) 1/2 . In particular, the horizontal lift at C ∈ Sym + (n) is X # = hor(X).
Proof of Theorem 6.3 (Horizontal lift). The horizontal lift is

X # = hor(∆ Σ X∆ Σ ) since d Σ π(∆ Σ X∆ Σ ) = ∆ -1 Σ V -1 2 (∆ -2 Σ Diag(V )Σ + ΣDiag(V )∆ -2 Σ ) ∆ -1 Σ = X, where V = ∆ Σ X∆ Σ .
11.5.3 Proof of Theorem 6.4

Theorem 6.4 (Expression of quotient-affine metrics) For all C ∈ Cor + (n) and

X ∈ T C Cor + (n), g α,β C (X, X) = α g QA C (X, X) (independent from β) where: g QA C (X, X) = tr((C -1 X) 2 ) -21 Diag(C -1 X)(I n + A(C)) -1 Diag(C -1 X)1.
Proof of Theorem 6.4 (Expression of quotient-affine metrics). We use the definition of the quotient metric.

g α,β C (X, X) = G α,β C (hor(X), hor(X)) = G α,β C (X, X) -G α,β C (ver(X), ver(X)).
First, we observe that tr(C -1 X) = tr(C -1 ver(X)) since tr(C -1 hor(X)) = 0 so the terms in β vanish and g

α,β C (X, X) = α(tr(C -1 XC -1 X) -tr(C -1 ver(X)C -1 ver(X))) does not depend on β. We denote ver(X) = C • ψ(µ) with µ = (I n + A(C)) -1 Diag(C -1 X)1. Then: tr(C -1 ver(X)C -1 ver(X)) = ijkl [C -1 ] ij C jk (µ j + µ k )[C -1 ] kl C li (µ l + µ i ), = 2 ij (δ ij + [C -1 ] ij C ij )µ i µ j , = 2µ (I n + A(C))µ, = 2 1 Diag(C -1 X)(I n + A(C)) -1 Diag(C -1 X)1.
11.5.4 Proof of Theorem 6.5 Theorem 6.5 (Geodesics of quotient-affine metrics) The geodesic from

C ∈ Cor + (n) with initial tangent vector X ∈ T C Cor + (n) is: ∀t ∈ R, γ QA (C,X) (t) = Exp QA C (tX) = π(C 1/2 exp(t C -1/2 hor(X)C -1/2 )C 1/2 ).
In particular, the quotient-affine metric is geodesically complete.

Proof of Theorem 6.5 (Geodesics of quotient-affine metrics). The geodesics are the the projections of horizontal geodesics [START_REF] Gallot | Riemannian Geometry[END_REF]] so they are given by Exp

QA C (X) = π(Exp G C (X # )) = π(C 1/2 exp(C -1/2 hor(X)C -1/2 )C 1/2 ).
11.5.5 Proof of Theorem 6.6 Theorem 6.6 (Existence of a logarithm) For all C 1 , C 2 ∈ Cor + (n), there exists X ∈ T C Cor + (n) such that Exp QA C 1 (X) = C 2 . Proof of Theorem 6.6 (Existence of a logarithm). We prove the coercivity of f : [DΛD] 11 -→ 0 so DΛD tends to a singular matrix and so does ΣDΛDΣ. Hence f (D) -→ +∞.

D ∈ Diag + (n) -→ tr(log(ΣDΛDΣ) 2 ). Since f (D) = tr(log(Σ D -1 Λ D -1 Σ ) 2 ) with Σ = Σ -1 and Λ = Λ -1 , it suffices to prove that if d 1 -→ 0, then f (D) -→ +∞. Indeed, if d 1 -→ 0, then
11.5.6 Proof of Theorem 6.7 Theorem 6.7 (Levi-Civita connection and sectional curvature of quotient-affine metrics) The Levi-Civita connection of quotient-affine metrics is:

(∇ QA X Y ) |C = d C Y (X) + sym[Diag(X # )Y # + Diag(Y # )X # + Diag(X # C -1 Y # )C -X # C -1 Y # -1 2 Diag(X # )CDiag(Y # ) -3 2 Diag(X # )Diag(Y # )C].
The curvature of quotient-affine metrics is:

κ QA C (X, Y ) = κ G C (X # , Y # ) + 3 4 G C (ver[X # , Y # ], ver[X # , Y # ]) g C (X, X)g C (Y, Y ) -g C (X, Y ) 2 , = 2 tr((C -1 X # C -1 Y # -C -1 Y # C -1 X # ) 2 ) + 3 1 D(I n + A(C)) -1 D1 8(g C (X, X)g C (Y, Y ) -g C (X, Y ) 2 ) where [V, W ] = dW (V ) -dV (W ) is the Lie bracket on Sym + (n) and D = D(X, Y ) -D(Y, X) with D(X, Y ) = Diag(C -1 Diag(X # )Y # -C -1 Y # C -1 Diag(X # )C).
There is a slight abuse of notation because ver[X # , Y # ] induces that X # and Y # are vector fields. Indeed here, they are horizontal vector fields extending the horizontal lifts at C.

Proof of Theorem 6.7 (Levi-Civita connection and sectional curvature of quotient-affine metrics). According O'Neill's equations of submersions, the Levi-Civita connection of quotient-affine metrics is given by:

(∇ QA X Y ) |C = d Σ π((∇ G X # Y # ) |C ), = d C π(d C Y # (X # ) - 1 2 (X # C -1 Y # + Y # C -1 X # )), = d C Y # (X # ) - 1 2 (Diag(d C Y # (X # ))C + CDiag(d C Y # (X # )) (11.26) - 1 2 (X # C -1 Y # + Y # C -1 X # -Diag(X # C -1 Y # )C -CDiag(X # C -1 Y # )). Since Y π(Σ) = ∆ -1 Σ Y # Σ ∆ -1 Σ -sym(∆ -3 Σ Diag(Y # Σ )Σ∆ -1 Σ ) where ∆ Σ = Diag(Σ) 1/2 , then: d π(Σ) Y (X) = d Σ (Y • π)(X # ), = ∆ -1 Σ (d Σ Y # (X # ))∆ -1 Σ -sym(∆ -3 Σ Diag(d Σ Y # (X # ))Σ∆ -1 Σ ) (11.27) -sym[∆ -3 Σ Diag(X # )Y # ∆ -1 Σ - 3 2 ∆ -5 Σ Diag(X # )Diag(Y # )Σ∆ -1 Σ + ∆ -3 Σ Diag(Y # )X # ∆ -1 Σ - 1 2 ∆ -3 Σ Diag(Y # )ΣDiag(X # )∆ -3 Σ ].
Since line (11.26) is line (11.27) at Σ = C, we finally get:

(∇ QA X Y ) |C = d C Y (X) + sym[Diag(X # )Y # + Diag(Y # )X # + Diag(X # C -1 Y # )C -X # C -1 Y # -1 2 Diag(X # )CDiag(Y # ) -3 2 Diag(X # )Diag(Y # )C].
The curvature in formula (6.10) directly comes from the fundamental equations of submersions [O'Neill, 1966]. The curvature of the affine-invariant metric comes from [Skovgaard, 1984]. Hence we only have to compute G

C (ver[X # , Y # ], ver[X # , Y # ]) where G is the affine-invariant metric, [•, •]
is the Lie bracket on the manifold of SPD matrices (it is not the matrix commutator), C ∈ Cor + (n) and X # , Y # are horizontal vector fields on Sym + (n) extending X # C and Y # C respectively, where X, Y ∈ T C Cor + (n) are tangent vectors at C. For example, we can consider that X, Y are constant vector fields on Cor + (n) and simply define X # , Y # as their horizontal lifts everywhere:

Y # Σ = hor(∆ Σ Y ∆ Σ ), = ∆ Σ Y ∆ Σ -ver(∆ Σ Y ∆ Σ ), = ∆ Σ Y ∆ Σ -2Σ • ψ((I n + Σ • Σ -1 ) -1 Diag(Σ -1 ∆ Σ Y ∆ Σ )1), = ∆ Σ Y ∆ Σ -2Σ • ψ((I n + Σ • Σ -1 ) -1 Diag(π(Σ) -1 Y )1), with ∆ Σ = Diag(Σ) 1/2 and ψ(µ) = µ1 + 1µ for µ ∈ R n . Now we can compute [X # , Y # ] = ∂ X # Y # -∂ Y # X #
and evaluate it at C. We have:

∂ X # Y # = 1 2 (∆ -1 Σ Diag(X # )Y ∆ Σ + ∆ Σ Y Diag(X # )∆ -1 Σ ) -X # • ψ((I n + Σ • Σ -1 ) -1 Diag(Σ -1 Y )1 + v Σ (X, Y ),
where v Σ (X, Y ) ∈ V Σ is a short notation for the following expression:

Σ • ψ[(I n + Σ • Σ -1 ) -1 (X # • Σ -1 -Σ • Σ -1 X # Σ -1 )(I n + Σ • Σ -1 ) -1 Diag(Σ -1 Y )1 + (I n + Σ • Σ -1 ) -1 Diag(π(Σ) -1 Xπ(Σ) -1 Y )1]
We evaluate it at C ∈ Cor + (n) so that ∆ C = I n . We denote B = (I n + C • C -1 ) -1 to simplify the notations. Note that the last term is symmetric in X and Y so we can define v 0 The second term is symmetric in X and Y . The first term in null, the third term is symmetric in X and Y .

C (X, Y ) = C • ψ[B(X # • C -1 -C • C -1 X # C -1 )BDiag(C -1 Y )1]. Then: [X # , Y # ] = 1 2 (Diag(X # )Y + Y Diag(X # ) -Diag(Y # )X -XDiag(Y # )), ( 11
The expression composed of the third term of Equation (11.31) plus the second term of Equation (11.32) is symmetric in X and Y . Hence, the vertical projection of line (11.28) reduces to the vertical projection of 1 2 (Diag(X # )Y # -Diag(Y # )X # ).

Proofs of Chapter 7

In this section, we prove the results of Chapter 7: Theoretically and computationally convenient Cholesky-based geometries. 1 n-1 , 1). Let us show that κ C (X, Y ) tends to +∞ when ρ → -1 n-1 , which proves that the curvature is not bounded from above. The symmetric matric 11 has two eigenvalues: 0 with multiplicity n-1 and n with multiplicity 1. Since (11 ) 2 = n11 , the minimal polynomial is P 11 (x) = x(x-n) for x ∈ R. For all α, β ∈ R, the symmetric matrix Σ = αI n + β11 has minimal polynomial P Σ (x) = (x -α)(x -(α + nβ)) for x ∈ R, which is of degree 2. Hence Σ is positive definite if and only if α > 0 and α + nβ > 0. In this case, its inverse is a polynomial in 11 of degree 1. More precisely, Σ Finally:

3 8

1 D(I n + C • C -1 ) -1 D1 g C (X, X)g C (Y, Y ) -g C (X, Y ) 2 = 3n(n -2) 8(n -1) αβ 4 α + nβ Å 2α + nβ 2α(α + nβ) + nβ 2 ã 2 .
When β → -1 n-1 (and α = 1 -β → n n-1 ), we have α + nβ → 0 and we have αβ 4 Ä (11.34) This proves that the quotient-affine sectional curvature is not bounded from above.

κ C (X, Y ) - 1 2 + 3 8 1 D(I n + C • C -1 ) -1 D1 g C (X, X)g C (Y, Y ) -g C (X, Y ) 2 -→ ρ→-1 n-1 +∞.
11.6.2 Proof of Theorem 7.2 Theorem 7.2 (Characterization of affine-invariant metrics) Let g be a Riemannian metric on SPD matrices. The following statements are equivalent:

1. g is GL(n)-invariant, 2. g is GL + (n)-invariant, 3. g is SL(n)-invariant and R + -invariant, 4. g is SO(n)-invariant and Diag + (n)-invariant, 5. g is S(n)-invariant and LT + (n)-invariant.

Proof of Theorem 7.2 (Characterization of affine-invariant metrics). The first statement clearly implies the others. To prove 2 =⇒ 1, we need to take a general GL + (n)-invariant metric on SPD matrices and prove that it is GL(n)-invariant. It amounts to prove that SO(n)-invariant inner products on symmetric matrices are O(n)-invariant, which is well known so 2 =⇒ 1. To prove 3 =⇒ 2; 4 =⇒ 2 and 5 =⇒ 1, it suffices to prove that the pairs of groups respectively generate GL + (n), GL + (n) and GL(n).

The group generated by SL(n) and R + is GL + (n) so 3 =⇒ 2. The group generated by SO(n) and Diag + (n) is also GL + (n) (it is clearly included in GL + (n) and conversely it contains Sym + (n) by the spectral theorem and GL + (n) by polar decomposition) so 4 =⇒ 2.

To prove that 5 =⇒ 1, let us show that the group generated by S(n) and LT + (n) is GL(n). First, the LU decomposition exists for any square matrix modulo a permutation. More precisely, for all A ∈ GL(n), there exists a permutation σ ∈ S(n), a lower triangular matrix L ∈ LT * (n) and an upper triangular matrix U ∈ UT + (n) with Diag(U ) = I n such that A = P σ LU . If we permute the rows and columns of U according to the permutation σ 0 : k -→ n + 1 -k, then P σ 0 U P σ 0 is lower triangular with ones on the diagonal so it is in LT + (n). Hence the group generated by S(n) and LT * (n) is GL(n). Since LT * (n) is generated by LT + (n) and Diag * (n), it suffices to prove that Diag * (n) is generated by S(n) and LT + (n). Since Diag * (n) is generated by Diag + (n) and matrices of the form diag(±1, ..., ±1) and since Diag + (n) ⊂ LT + (n), it suffices to prove that matrices of the form diag(±1, ..., ±1) are generated by S(n) and LT + (n). By matrix product and permutations, it suffices to prove that diag(-1, 1, ..., 1) is generated by S(n) and LT + (n). The following product of matrices:

∈LT + (2) Å 1 0 -1 1 ã ∈UT + (2) Å 1 1 0 1 ã ∈LT + (2) Å 1 0 -1 1 ã ↓ ↓ ↓ = S(2) Å 0 1 1 0 ã Å -1 1 1 0 ã Å -1 0 1 1 ã Å -1 0 0 1 ã (11.35)
can be generalized in dimension n by adding a diagonal block I n-2 . Hence, the matrix diag(-1, 1, ..., 1) is generated by S(n) and LT + (n) so LT * (n) as well and finally GL(n) entirely.

11.6.3 Proof of Theorem 7.4

Theorem 7.4 (Lie-Cholesky are LT + (n)-invariant metrics) A Riemannian metric on Sym + (n) is a Lie-Cholesky metric if and only if it is LT + (n)-invariant.

Proof of Theorem 7.4 (Lie-Cholesky are LT + (n)-invariant metrics). Let g be a metric on Sym + (n). Let Σ ∈ Sym + (n), V ∈ T Σ Sym + (n), L = Chol(Σ) and Z = d Σ Chol(X). Note that Equation (7.10) rewrites L -1 Z = d In Chol(L -1 V L -). Thus, by definition of the pushforward, we have the following equalities: ). Then for X, Y ∈ LT(n), vec(ad * (X)(Y )) = A(I n ⊗ X -X ⊗ I n )A -1 vec(Y ).

(Chol * g) L (Z, Z) = g Σ (V, V ) (11.36) (Chol * g) In (L -1 Z, L -1 Z) = g In (L -1 V L -, L -1 V L -) (
Proof of Lemma 7.11 (Horizontal lift,Riemannian metric,exponential map). For all ξ ∈ LT 0 (n), we have d L π(Diag(L)ξ) = Diag(L) -1 (Diag(L)ξ-Diag(ξ)L) = ξ. Hence ξ # L = hor L (Diag(L)ξ). In particular, ξ # Γ = hor Γ (ξ). Then the metric and the exponential map simply write g QLC Γ (ξ, ξ) = g LC Γ (ξ # Γ , ξ # Γ ) and Exp QLC Γ (tξ) = π(Exp LC Γ (tξ # Γ ).

11.6.7 Proof of Theorem 7.14 Theorem 7.14 (Symmetric space structure) The manifold of full-rank correlation matrices Cor + (n) equipped with a poly-hyperbolic-Cholesky metric is a Riemannian symmetric space of non-positive sectional curvature bounded by [a, 0] with a = -1 min i 2 α i . For n 3, it is not of constant curvature. The canonical PHC metric writes for all C ∈ Cor + (n) and X ∈ T C Cor + (n) Hol(n): α k and b = 0 are clearly reached for n 3 by bivectors (X, Y ) and (X, Z) respectively, where X = (0, ..., 0, X k , 0, ..., 0), Y = (0, ..., 0, Y k , 0, ..., 0) and Z = (Z 1 , 0, ..., 0) ∈ T (HS 1 × • • • × HS n-1 ). To express the Riemannian metric, we take the pullback of the Riemannian metric on the hyperboloid H n defined by g x (v, v) = n k=1 v 2 k -v 2 n+1 by the diffeomorphism ϕ SH : (x 1 , ..., x n+1 ) ∈ HS n -→ 1

x n+1 (x 1 , ..., x n , 1) ∈ H n . We compute for all x ∈ HS n and all v ∈ T x HS n , using n+1 k=1 x 2 k = 1 and n+1 k=1 x k v k = 0:

d x ϕ SH (v) = 1 x n+1 Å v 1 -x 1 v n+1 x n+1 , ..., v n -x n v n+1 x n+1 , - v n+1 x n+1 ã , g HS x (v, v) = g H ϕ(x) (d x ϕ SH (v), d x ϕ SH (v)) = 1 x 2 n+1 Ç n k=1 Å v k -x k v n+1 x n+1 ã 2 - v 2 n+1 x 2 n+1 å = 1 x 2 n+1 Ç n k=1 Å v 2 k -2x k v k v n+1 x n+1 + x 2 k v 2 n+1 x 2 n+1 ã - v 2 n+1 x 2 n+1 å = 1 x 2 n+1 Ç n k=1 v 2 k + 2v 2 n+1 + (1 -x 2 n+1 ) v 2 n+1 x 2 n+1 - v 2 n+1 x 2 n+1 å = v 2 x 2 n+1 .
Hence, for all C ∈ Cor + (n), X ∈ Hol(n), L = Chol(C) ∈ L and Y = d C Chol(X) = L Low S (L -1 XL -) with Y 11 = 1 2 L 11 [L -1 XL -] 11 = X 11 2L 11 = 0, we have:

g PHC C (X, X) = g HS 1 ו••×HS n-1 L (Y, Y ) = n i=2 Y i• 2 L 2 ii = n i=1 Diag(L) -1 ii Y i• 2 = Diag(L) -1 Y 2 = Diag(L) -1 L Low S (L -1 XL -) 2 .
Note that the general PHC metric writes g PHC • (Geodesic) γ C→C (t) = Θ -1 (Θ(C)(Θ(C) -1 Θ(C )) t ),

• (Group mean) Unique, characterized by k i=1 log(Θ( C) -1 Θ(C i )) = 0. Proof of Theorem 7.19 (Group operations). The exponential map, logarithm map and geodesics are pullbacks by Θ of corresponding operations in LT 1 (n), which are for all Γ, Γ ∈ LT + (n), ξ ∈ T Γ LT + (n) LT(n) and t ∈ R:

• (Exponential map) Exp Γ (tξ) = Γ exp(t Γ -1 ξ),

• (Logarithm map) Log Γ (Γ ) = Γ log(Γ -1 Γ ),

• (Geodesic) γ Γ→Γ (t) = Exp Γ (t Γ -1 Log Γ (Γ )) = Γ(Γ -1 Γ ) t .

Since the Lie algebra LT 0 (n) is nilpotent, the group mean Γ of the finite sample Γ 1 , ..., Γ k ∈ LT 1 (n) is unique [Buser and Karcher, 1981, Example 8.1.8] 1-ρ 1 is known as the difference of the Fisher transformation of the correlation coefficients ρ 1 and ρ 2 .

2. Euclidean-Cholesky and log-Euclidean-Cholesky metrics coincide. The geodesic between C 1 and C 2 is C(ρ(t)) for t ∈ R where:

ρ(t) = F (t) 1 + F (t) 2 ,
where

F (t) = (1-t) ρ 1 √ 1-ρ 2 1 +t ρ 2 √ 1-ρ 2 2
. This geodesic also coincides with the Lie-Cholesky group geodesic of Section 7.4.4.

Proof of Theorem 7.20.

1. The formula of the geodesic is known for the quotient-affine metrics in dimension 2. Hence it suffices to show that the quotient-affine metrics and the poly-hyperbolic-Cholesky metrics coincide up to a scaling factor. Let C = C(ρ) and X = Å 0 x x 0 ã ∈ T C Cor + (2). We compute the quotient-affine metric g QA C (X, X) = tr(C -1 XC -1 X)-2 sum(D(I n +C•C -1 ) -1 D) and the canonical PHC metric g CPHC C (X, X) = Diag(L) -1 L Low S (L -1 XL -) 2 where D = Diag(C -1 X) and L = Chol(C).

C -1 = 1 1 -ρ 2 Å 1 -ρ -ρ 1 ã , C -1 X = x 1 -ρ 2 Å -ρ 1 1 -ρ ã , C -1 XC -1 X = x 2 (1 -ρ 2 ) 2 Å 1 + ρ 2 -2ρ -2ρ 1 + ρ 2 ã , tr(C -1 XC -1 X) = 2(1 + ρ 2 ) (1 -ρ 2 ) 2 x 2 , I n + C • C -1 = 1 1 -ρ 2 Å 2 -ρ 2 -ρ 2 -ρ 2 2 -ρ 2 ã , (I n + C • C -1 ) -1 = 1 4 Å 2 -ρ 2 ρ 2 ρ 2 2 -ρ 2 ã , D = Diag(C -1 X) = - ρx 1 -ρ 2 I 2 , sum(D(I n + C • C -1 ) -1 D) = ρ 2 x 2 (1 -ρ 2 ) 2 , g C (X, X) = 2x 2 (1 -ρ 2 ) 2 , L = Chol(C) = Å 1 0 ρ √ 1 -ρ 2 ã , L -1 XL -= Ç 1 0 -ρ √ 1-ρ 2 1 √ 1-ρ 2 å Å 0 x x 0 ã Ñ 1 -ρ √ 1-ρ 2 0 1 √ 1-ρ 2 é = Ç 0 x x √ 1-ρ 2 -ρx √ 1-ρ 2 å Ñ 1 -ρ √ 1-ρ 2 0 1 √ 1-ρ 2 é = Ñ 0 x √ 1-ρ 2 x √ 1-ρ 2 -2ρx 1-ρ 2 é , Diag(L) -1 L Low S (L -1 XL -) = Ç 1 0 ρ √ 1-ρ 2 1 å Ç 0 0 x √ 1-ρ 2 -ρx 1-ρ 2 å = Ç 0 0 x √ 1-ρ 2 -ρx 1-ρ 2 å , g CPHC C (X, X) = Å 1 1 -ρ 2 + ρ 2 (1 -ρ 2 ) 2 ã x 2 = x 2 (1 -ρ 2 ) 2 .
2. Euclidean-Cholesky and log-Euclidean-Cholesky metrics coincide in dimension 2 because exp(ξ) = I n + ξ and log(Γ) = Γ -I 2 for ξ ∈ LT 0 (2) and Γ ∈ LT 1 (2). Thus their common Riemannian exponential and logarithm in LT 1 (2) are simply Exp Γ (ξ) = Γ + ξ and Log Γ (Γ ) = Γ -Γ. On the other hand, the group exponential in LT 1 (2) is Exp

LT 1 (2) Γ

(ξ) = Γ exp(Γ -1 ξ) = Γ(I n + Γ -1 ξ) = Γ + ξ = Exp Γ (ξ). Hence, the group geodesics coincide with the (log-)Euclidean-Cholesky geodesics. Let us compute them.

Γ 1 := Diag(Chol(C 1 )) -1 Chol(C 1 ) = Ç 1 0

ρ 1 √ 1-ρ 2 1 1 å , Γ 2 := Diag(Chol(C 2 )) -1 Chol(C 2 ) = Ç 1 0 ρ 2 √ 1-ρ 2 2 1 å , C(t) = Θ -1 ((1 -t)Γ 1 + tΓ 2 ) = Θ -1 Å 1 0 F (t) 1 ã = Cor Å 1 F (t) F (t) 1 + F (t) 2 ã = Ñ 1 F (t) √ 1+F (t) 2 F (t) √ 1+F (t) 2 1 é .
We can also compute the (log-)Euclidean-Cholesky metric in dimension 2.

g C (X, X) = d C Θ(X) 2 = (Θ • C) (ρ) x 2 = f (ρ) 2 x 2 , where f (ρ) = ρ √ 1-ρ 2 . So f (ρ) = √ 1-ρ 2 + ρ 2 √ 1-ρ 2 1-ρ 2 = 1
(1-ρ 2 ) 3/2 and:

g C (X, X) = x 2 (1 -ρ 2 ) 3 .

Proofs of Chapter 8

In this section, we prove the results of Chapter 8: Permutation-invariant Log-Euclidean metrics.

Ñ 0 √ 2 √ 2 √ 2 1 -1 √ 2 -1 1 é , D = diag(1, 1 + √ 2 √ 7 , 1 - √ 2 √ 7 ), I(C) = Ñ 1 -a a -a 1 -a 2 a -a 2 1 é = Q∆Q with Q = 1 √ 14 Ñ 0 √ 6 2 √ 2 √ 7 -2 √ 3 √ 7 2 - √ 3
é , ∆ = 1 6 diag(5, 10, 3). For n 4, it suffices to take the block diagonal matrix Diag(C, I n-3 ).

between I n and all points of the fiber Cor -1 (C) = {∆C∆ ∈ Sym + (n)|∆ ∈ Diag + (n)}.

1. The smooth map f has a global minimizer.

2. For all Σ ∈ Sym + (n), there exists ∆ ∈ Diag + (n) such that log(∆Σ∆) ∈ Hol(n).

3. The following conjectures are equivalent for all C ∈ Cor + (n).

(i) There exists a unique ∆ ∈ Diag + (n) such that log(∆C∆) ∈ Hol(n) (Conjecture 8.11).

(ii) There exists a unique X ∈ Hol(n) such that Exp QA In (X) = C. (iii) There exists a unique local minimizer of the smooth map f , which is actually the global minimizer ensured by statement 1.

4. The previous conjectures imply the uniqueness of the quotient-affine logarithm at I n .

Proof of Theorem 8.13 (Existence and equivalence of conjectures).

1. The smooth map f has a global minimizer because it is coercive (Theorem 6.6).

2. Hence, there exists ∆ ∈ Diag + (n) such that f (∆) = min f . In other words, ∆C∆ is "in optimal position" to I n [Huckemann et al., 2010, Definition 2.3]. Thus [Huckemann et al., 2010, Theorem 2.4], the geodesic from I n to ∆C∆ is horizontal, i.e. Log AI In (∆C∆) ∈ H QA In , i.e. log(∆C∆) = Hol(n). For Σ = Diag(Σ) 1/2 C Diag(Σ) 1/2 , it suffices to take ∆Diag(Σ) -1/2 ∈ Diag + (n). Hence, if ∆ ∈ Diag + (n) is such that log(∆C∆) ∈ Hol(n), then the Hessian of f at ∆ is positive definite so f has a local minimum at ∆. Thus, if proposition (iii) is true, then ∆ has to be the global minimizer so it is unique. Conversely, if f has a local minimum at ∆ ∈ Diag + (n), then d ∆ f = 0 so log(∆C∆) ∈ Hol(n). Thus, if proposition (i) is true, then ∆ is unique. Therefore, assertions (i) and (iii) are equivalent.

Since Exp

an SPD matrix with row sums prescribed by X 2 1. Thus the existence and uniqueness is ensured by [START_REF] Marshall | Scaling of matrices to achieve specified row and column sums[END_REF]. 2. (Geodesic) γ(t) = C(ρ(t)) where ρ(t) = ρ 1 cosh(λt)+sinh(λt) ρ 1 sinh(λt)+cosh(λt) with λ = F (ρ 2 ) -F (ρ 1 ) is monotonic (increasing if ρ 1 < ρ 2 , decreasing if ρ 1 > ρ 2 , constant if ρ 1 = ρ 2 ), Proof of Theorem 8.23 (Coincidence of the metrics in dimension 2). It suffices to use the 2nd statement of Theorems 8.4 and 8.17 and the formulae of distances of the off-log metric and the log-scaled metric. For the former, d(C 1 , C 2 ) = q(F (ρ 2 )

Å 0 1 1 0 ã -F (ρ 1 ) Å 0 1 1 0 ã ) ∝ λ 2
and similarly for the ladder. Up to a multiplicative constant, these distances are equal to the quotient-affine distance in dimension 2 [START_REF] Thanwerdas | Geodesics and Curvature of the Quotient-Affine Metrics on Full-Rank Correlation Matrices[END_REF]. Therefore, the Riemannian metrics coincide up to a constant and the geodesics coincide. The formulae can be found in [START_REF] Thanwerdas | Theoretically and computationally convenient geometries on full-rank correlation matrices[END_REF].

Proof of Theorem 8.24

Theorem 8.24 (Optimization problem to compute the scaling) arg min Ω f ∈ M and Ω = conv(M). Thus x * is the global minimizer of the convex map f on the closed convex set Ω.

Proof of Theorem 8.24 (Optimization problem to compute the scaling). For all x ∈ Ω, the vector x 0 = 1 α x ∈ M with α = ( n i=1 x i ) 1/n 1 satisfies f (x 0 ) = x 0 Σx 0 = 1 α 2 x Σx f (x). Moreover, if α > 1, then f (x 0 ) < f (x). Hence if x minimizes f on Ω, then f (x) f (x 0 ) f (x) so f (x 0 ) = f (x) so α = 1 and x = x 0 ∈ M. Otherwise said, arg min Ω f = arg min M f ∈ M.

Let us show that Ω = conv(M). Let x, y ∈ Ω and t ∈ [0, 1]. Then since log is concave, n i=1 log((1 -t)x i + ty i ) (1 -t) n i=1 log(x i ) + t n i=1 log(y i ) 0 so (1 -t)x + ty ∈ Ω. Hence Ω is convex and it contains M so conv(M) ⊆ Ω.

Let x ∈ Ω\M. Let F : a ∈ (-x 2 , x 1 ) -→ log(x 1 -a)+log(x 2 +a)+ n i=3 log(x i ) ∈ R. Note that F (0) > 0. Then F (a) = 1

x 2 +a -1 x 1 -a = x 1 -x 2 -2a (x 1 -a)(x 2 +a) . Hence with a 0 = x 1 -x 2 2 ∈ (-x 2 , x 1 ), F (a) is positive when a < a 0 and negative when a > a 0 . Thus F increases on (-x 2 , a 0 ) and decreases on (a 0 , x 1 ) with F (a 0 ) F (0) > 0, F -→ a→-x 2 -∞ and F -→ a→x 1 -∞. So, since F is continuous, there exist a 2 ∈ (-x 2 ; a 0 ) and a 1 ∈ (a 0 ; x 1 ) such that F (a 1 ) = F (a 2 ) = 0. Note that a 2 < 0 < a 1 because F (0) > 0. Therefore, x = (1 -t)y + tz with y = (x 1 -a 1 , x 2 + a 1 , x 3 , ..., x n ) ∈ M, z = (x 1 -a 2 , x 2 + a 2 , x 3 , ..., x n ) ∈ M and t = a 1 a 1 -a 2 ∈ (0, 1). Finally, Ω ⊆ conv(M) so Ω = conv(M). 11.7.9 Proof of Theorem 8.32 Theorem 8.32 (Stopping criterion) ∀z ∈ R n \Ω, ∀α > 1, Π Ωα z -Π Ω z (α 1/n -1) Π Ω z . Proof of Theorem 8.32. Let z ∈ R n \Ω. We denote x = Π Ω z and a ∈ R + such that x-z = a 1

x . Thus it is equivalent to prove that for all x ∈ M and a ∈ R + , Π Ωα (x-a x )-x (α 1/n -1) x .

Hence, we define the smooth maps:

z : a ∈ R + -→ x - a x ∈ R n \Ω, y : a ∈ R + -→ Π Ωα (z(a)) ∈ M α , D : a ∈ R + -→ y(a) -x 2 = n i=1
(y i (a) -x i ) 2 ∈ R + .

We want to prove that for all a ∈ R + , D(a) (α 1/n -1) 2 x 2 . We proceed in two steps. Firstly, we prove that lim a→+∞ D(a) = (α 1/n -1) 2 x 2 . Secondly, we prove that D is increasing.

First step: we compute the limit of D at +∞. We recall that x ∈ M, z(a) = x -a

x , y(a) = Π Ωα z(a) and D(a) = y(a) -x 2 . We denote u = 1 Now, let (a k ) k∈N ⊂ (R + ) N be a sequence such that lim k→∞ a k = +∞. Let y k = y(a k ). Let Y be a subsequential limit of (y k ) k∈N . Then 1 Y u so Y x. Moreover Y ∈ M α since M α is closed so Y = α 1/n x = y 0 . Otherwise said, (y k ) k∈N is a bounded sequence with only one subsequential limit y 0 so (y(a k )) k∈N converges to y 0 . This being valid for any sequence (a k ) k∈N tending to +∞, we proved that y(a) tends to y 0 = α 1/n x when a tends to +∞. Thus lim a→+∞ D(a) = (α 1/n -1) 2 x 2 .

Second step: we prove that D is increasing. ). This also gives the diagonal cofactors [com(M )] ii = j =i (m j -µ)(1 + µ j =i 1 m j -µ ). The off-diagonal cofactors are, up to a sign, the determinant of a matrix of the same type, where m i = µ, which is µ j =i (m j -µ). Hence the off-diagonal cofactors for i < j are [com(M )] ij = [com(M )] ji = (-1) i+j (-1) j-1-i µ -µ j =i (m j -µ) because the (j -1)-th column has to be put in i-th position (by inverting consecutive columns) so that we are in the previous situation. This requires j -1 -i transpositions. Finally:

M -1 = 1 det(M ) com (
(y i -x i )y i = δ Å y -x y ã v = δ Å y -x y ã M -1 u = δ n i=1 c i (1 - x i y i )(s - y i x i ) + 1 n i,j c i c j (1 - x i y i )(s - y j x j ) = n i=1 c i (1 - x i y i )(s - y i x i ) - 1 n n j=1 c j n i=1 c i (1 - x i y i )(s - y i x i ) + 1 n i,j c i c j (1 - x i y i )(s - y j x j ) = i c i (1 - x i y i )(s - y i x i ) + 1 n i,j c i c j (1 - x i y i )( x i y i - y j x j ) = i c i (1 - x i y i ) 1 n j ( y j x j - y i x i ) - i c i (1 - x i y i ) 1 n j c j ( y j x j - x i y i ) = 1 n i,j c i (1 -c j )(1 - x i y i )( y j x j - y i x i ) = 1 2n i,j ( y j x j - y i x i )(1 -c i )(1 -c j )[ c i (1 -x i y i ) 1 -c i - c j (1 -x j y j ) 1 -c j ] = 1 2nb i,j
(1 -c i )(1 -c j ) 0 ( y j x j -y i x i )[y i (y i -x i ) -y j (y j -x j )]

Since b = y i (y i -z i ) = y i (y i -x i ) + a y i x i and a > 0, we have y i (y i -x i ) -y j (y j -x j ) = a( y j

x j -y i x i ). Finally:

D (a) = a nbδ i,j (1 -c i )(1 -c j )( y j x j - y i x i ) 2 > 0 (11.39)
The inequality is strict because the sum is null if and only if x and y are colinear, i.e. y = α 1/n x, which corresponds to a = +∞. So D is increasing so D lim a→+∞ D(a) = (α 1/n -1) 2 x 2 .

Proofs of Chapter 9

In this section, we prove the results of Chapter 9: Bures-Wasserstein stratified geometry of covariance matrices. 1. (Exponential map) [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF] For all V ∈ T Σ Sym + (n) ≡ Sym(n), the geodesic from Σ with initial speed V writes γ (Σ,V ) (t) = Σ + tV + t 2 S Σ (V )ΣS Σ (V ) ∈ Sym + (n).

2. (Definition interval) Let λ max = max sp(S Σ (V )) and λ min = min sp(S Σ (V )). The definition interval of the geodesic γ (Σ,V ) is the interval I Σ,V defined by:

• I Σ,V = (-1 λmax , -1 λ min ) if λ min < 0 < λ max , • I Σ,V = (-∞, -1
λ min ) if λ min < 0 and λ max 0, • I Σ,V = (-1 λmax , +∞) if λ min 0 and λ max > 0, • I Σ,V = R if λ min = λ max = 0 (which only happens for V = 0).

(Cut time)

The cut time is t cut (Σ, V ) = -1 λ min if λ min < 0 or +∞ otherwise. The geodesic γ (Σ,V ) : I Σ,V -→ M is even minimizing on I Σ,V .

4. (Logarithm map) For all Λ ∈ Sym + (n), there exists a unique preimage V ∈ Pre Σ (Λ).

It writes V = 2 sym(Σ 1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 Σ -1/2 ) -2Σ, where sym(M ) = 1 2 (M + M ). The geodesic joining Σ to Λ writes: γ Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t) sym(Σ 1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 Σ -1/2 ). Moreover, it is a logarithm: V ∈ Log x (y). Thus the logarithm map is defined on U Σ = Sym + (n) and it writes Log Σ : Λ ∈ Sym + (n) -→ 2sym(Σ 1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 Σ -1/2 )-2Σ ∈ T Σ Sym + (n).

Proof of Theorem 9.22 (Bures-Wasserstein geodesics on Sym + (n)). We prove statement 3 in the end because it requires statement 4.

(Exponential map)

The expression of the exponential map comes from [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF].

(Definition domain)

The domain I Σ,V is described in [START_REF] Malagò | Wasserstein Riemannian geometry of Gaussian densities[END_REF] as the connected component of 0 in J Σ,V = {t ∈ R|I n + tS Σ (V ) ∈ Sym + (n)}. Since t ∈ J Σ,V if and only if 0 / ∈ {1 + tλ|λ ∈ sp(S Σ (V ))} if and only if t / ∈ {-1 λ |λ ∈ sp(S Σ (V ))}, we have max(-∞, 0] ∩ {-1 λ |λ ∈ sp(S Σ (V ))} = -1 λmax if λ max > 0 and min[0, +∞) ∩ {-1 λ |λ ∈ sp(S Σ (V ))} = -1 λ min if λ min < 0. Therefore, we have the following cases:

• if λ min < 0 < λ max , then I Σ,V = (-1 λmax , -1 λ min ), • if λ min < 0 and λ max 0, then (-∞, 0] ⊆ J Σ,V so I Σ,V = (-∞, -1 λ min ), • if λ min 0 and λ max > 0, then [0, +∞) ⊆ J Σ,V so I Σ,V = (-1 λmax , +∞), • if λ min 0 and λ max 0, which means λ min = λ max = 0, then V = 0 and I Σ,V = J Σ,V = R.

(Logarithm map)

The existence of a preimage V ∈ Pre Σ (V ) and even a logarithm V ∈ Log Σ (V ) (because it satisfies V = d(Σ, Λ)) is due to [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF]. The geodesic joining Σ to Λ (Equation 9.3) is derived in [START_REF] Bhatia | On the Bures-Wasserstein distance between positive definite matrices[END_REF] and it suffices to derive the expression at t = 0 to compute V = γΣ→Λ (0) = 2 sym(Σ 1/2 (Σ 1/2 ΛΣ 1/2 ) 1/2 Σ -1/2 )-2Σ.

The uniqueness of the preimage comes from [Massart and Absil, 2020, Proposition 4.4] on Sym + (n, k) applied to k = n. Indeed, it is stated that there exists a unique W ∈ H Σ 1/2 such that:

(a) for all t ∈ [0, 1], Σ 1/2 (Σ 1/2 + tW ) ∈ GL(n), (b) (Σ 1/2 + W )(Σ 1/2 + W ) = Λ.

Therefore, there exists a unique

V = d Σ 1/2 π(W ) = Σ 1/2 W + W Σ 1/2 ∈ T Σ Sym + (n) (and W = V # Σ 1/2
) such that 1 ∈ I Σ,V (for all t ∈ [0, 1], Exp Σ (tV ) ∈ Sym + (n)) and Exp Σ (V ) = Λ, i.e. V ∈ Pre Σ (Λ). Thus the logarithm map Log Σ is defined on U Σ = Sym + (n).

(Cut time)

Let us prove that γ (Σ,V ) is minimizing on I Σ,V . This will prove in particular that t cut (Σ, V ) = sup I Σ,V . Let t, t ∈ I Σ,V , t < 0 < t , let Λ = Exp Σ (tV ) and Λ = Exp Σ (t V ). Changing the base point of the geodesic, we have Λ = Exp Λ ((t -t)V ) with V = -γ(Σ,V ) (t) ∈ T Λ Sym + (n). Since for all s ∈ [0, 1], (1 -s)t + st ∈ I Σ,V and Exp Λ (s(t -t)V ) = Exp Σ (((1 -s)t + st )V ) ∈ Sym + (n), we have 1 ∈ I Λ,(t -t)V so (t -t)V ∈ Pre Λ (Λ ). By uniqueness of the preimage of Λ from Λ, Log Λ (Λ ) = (t -t)V and γ Λ,(t -t)V is minimizing on [0, 1]. Equivalently, γ (Σ,V ) is minimizing on [t, t ] so it is minimizing on I Σ,V . t) for all t ∈ [0, 1]. Indeed, it is equal to c 0 (t)c 0 (t) on (0, 1) and the equality is clear for t ∈ {0, 1}. Moreover, H X,Y,R = X Y R = U Diag(X 0 Y 0 , 0)U ∈ Cov(n) so H X,Y,R = ((X Y )(X Y ) ) 1/2 = (X ΛX) 1/2 . Finally, γ(t) = c(t)c(t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t) sym(XRY ).

V # X = X(X X) -1 S X X (X V X)+(I n -X(X X) -1 X ) X ⊥ X ⊥ V X(X X) -
(Sufficiency) Let γ R (Σ,Λ) (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t)sym(XRY ) with H = H X,Y,R = X Y R ∈ Cov(n) and let us prove that it is a minimizing geodesic segment. We define W = Y R -X and c(t) = X + tW = (1 -t)X + tY R for t ∈ [0, 1]. The curve c is a geodesic of Mat(n) such that c(t)c(t) = γ R (Σ,Λ) (t) for all t ∈ [0, 1]. Moreover,

L(c) = W = tr(XX + Y Y -2X Y R ) 1/2 = tr(Σ + Λ -2H) 1/2 . Let Q ∈ O(n) such that X = Σ 1/2 Q. Since H 0, H = (X ΛX) 1/2 = Q (Σ 1/2 ΛΣ 1/2 ) 1/2 Q. Therefore, L(c) = W = Y R -X = tr(Σ + Λ -2H) 1/2 = d BW (Σ, Λ).
In other words, c : [0, 1] -→ Cov(n) is a minimizing curve between two registered points X and Y R so by Lemma 9.13, its projection γ : [0, 1] -→ Cov(n) is a minimizing curve and L(γ) = L(c) = d BW (Σ, Λ).

By Lemma 9.30 again, γ has constant rank p max(k, l) on (0, 1) so γ |(0,1) : (0, 1) -→ Sym + (n, p) is a minimizing curve of Sym + (n, p). Since c |(0,1) has constant speed, so does γ |(0,1) . By continuity of the length, γ has constant speed on [0, 1] so γ : [0, 1] -→ Cov(n) is a minimizing geodesic segment. 1. For all X, Y ∈ Mat(n) such that XX = Σ and Y Y = Λ, r = rk(X Y ).

2. We have l -r n -k.

Proof of Lemma 9.33 (Elementary algebra).

1. Let X 0 ∈ R n×k * and Y 0 ∈ R n×l * such that X 0 X 0 = Σ and Y 0 Y 0 = Λ. Thus there exist P, Q ∈ O(n) such that X = [X 0 0]P and Y = [Y 0 0]Q. Since ΣΛ = X(X Y )Y , we have r rk(X Y ) = rk(X 0 Y 0 ). Since X 0 Y 0 = (X 0 X 0 ) -1 X 0 ΣΛY 0 (Y 0 Y 0 ) -1 , we have rk(X 0 Y 0 ) rk(ΣΛ). Finally, r = rk(X 0 Y 0 ) = rk(X Y ).

2. Let f, g : R n -→ R n be linear endomorphisms respectively represented by Σ and Λ is the canonical basis. From the rank-nullity theorem applied to f |im (g) : im (g) -→ R n , the restriction of f to im (g), and since im (f |im (g) ) = im (f •g) and ker(f |im (g) ) ⊆ ker(f ), we have rk(g) = rk(f |im (g) )+dim ker(f |im (g) ) rk(f •g)+dim(ker f ) = rk(f •g)+n-rk(f ). This writes l -r n -k.

11.8.7 Proof of Theorem 9.34 Theorem 9.34 (Number of Bures-Wasserstein minimizing geodesic segments in Cov(n)) Let Σ, Λ ∈ Cov(n) with rk(Σ) = k and rk(Λ) = l. We assume that k l without loss of generality. We denote r = rk(ΣΛ). We have l -r n -k.

1. There exists a bijection between the set of minimizing geodesics from Σ to Λ and the closed unit ball of R (k-r)×(l-r) for the spectral norm BS (0, 1) = {R 0 ∈ R (k-r)×(l-r) | R 0 S 1} = {R 0 ∈ R (k-r)×(l-r) | 0 R 0 R 0 I l-r }. More precisely, this bijection is given by:

γ R 0 Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t) sym(X r Y r + X k-r R 0 Y l-r ),
where X = (X r X k-r 0) ∈ R n×n and Y = (Y r Y l-r 0) ∈ R n×n are such that XX = Σ, Y Y = Λ and X Y = Diag(D r , 0), and R = r) , Y l-r ∈ R n×(l-r) , D r ∈ Diag + (r), R 0 ∈ BS (0, 1). 2. The minimizing geodesic is unique if and only if r = l. This includes the cases k = n.

Ñ I r 0 0 0 R 0 * 0 * * é ∈ O(n), with X r , Y r ∈ R n×r , X k-r ∈ R n×(k-
3. There are infinitely many minimizing geodesics if and only if r < l.

4. The minimizing geodesics corresponding to the choices R 0 ∈ St(k -r, l -r) (including the empty matrix if r = l) have rank exactly k on [0, 1) (on [0, 1] if l = k). Note that St(k -r, l -r) is included in the unit sphere S S (0, 1) = {R 0 ∈ R (k-r)×(l-r) | R 0 S = 1}.

5. The minimizing geodesic corresponding to the choice R 0 = 0 (or the empty matrix if r = l) writes for all t ∈ [0, 1]: γ 0 Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t) sym(Σ 1/2 ((Σ 1/2 ΛΣ 1/2 ) 1/2 ) † Σ 1/2 Λ).

If r = l, it has rank exactly k on [0, 1).

The number of minimizing geodesic segments in Sym + (n, k) and in Cov(n) is summarized in Table 9.2 with n k l r. , X k-r ∈ R n× (k-r) and Y l-r ∈ R n× (l-r) .

Σ ∈ Λ ∈ r = rk(ΣΛ) Number of minimizing geodesics in Sym + (n, k) in Cov(n) Sym + (n) Sym + (n) n 1 1 Sym + (n) Sym + (n, k) k 1 1 Sym + (n, k) Sym + (n, k) k 1 1 k -1 2 ∞ < k -1 ∞ ∞ Sym + (n, k) Sym + (n, l) l 1 1 < l ∞ ∞ Table 9
Necessarily, H = (X Y Y X) 1/2 = Diag(D r , 0). The possible R ∈ O(n) such that HR = X Y are R = Diag(I r , R n-r ) with R n-r = [R l-r R n-l ] ∈ O(n -r) where R l-r ∈ St(n -r, l -r) and R n-l ∈ St(n -r, n -l). We denote -r) and R 1 ∈ R (n-k)×(l-r) with R 0 R 0 + R 1 R 1 = I l-r . Note that both R 0 and R 1 have more rows than columns.

R l-r = Å R 0 R 1 ã with R 0 ∈ R (k-r)×(l
A simple calculus gives XRY = X r Y r + X k-r R 0 Y l-r . Given R, R satisfying HR = HR = X Y , we have XRY = XR Y if and only if R 0 = R 0 since X k-r X k-r and Y l-r Y l-r are invertible. We even have sym(XRY ) = sym(XR Y ) if and only if

R 0 = R 0 . Indeed, if sym(XRY ) = sym(XR Y ), then X k-r (R 0 -R 0 )Y l-r = Y l-r (R 0 - R 0 ) X k-r .
Since X k-r Y l-r = 0, it suffices to multiply on the left by X k-r and on the right by Y l-r to conclude that R 0 = R 0 . Thus there is a bijection between minimizing geodesic segments and submatrices R 0 ∈ R (k-r)×(l-r) of R l-r = Å R 0 R 1 ã ∈ St(n -r, l -r).

Since R 1 has more rows than columns, any R 0 ∈ R (k-r)×(l-r) such that R 0 R 0 I l-r can be completed by an appropriate R 1 = Å (I l-r -R 0 R 0 ) 1/2 0 n-k-(l-r),l-r ã .

Therefore, the minimizing geodesic segments are in bijection with the matrices R 0 ∈ R (k-r)×(l-r) such that R 0 R 0 I l-r , that is the closed unit ball for the spectral norm BS (0, 1).

2. When r = l, the component Y l-r of Y is the empty matrix. In other words, the dependence of the minimizing geodesic on R 0 vanishes so the minimizing geodesic is unique. In particular when k = n, r = rk(ΣΛ) = rk(Λ) = l.

3. On the contrary, when r < l (thus n > k l), there is an infinite number of convenient R 0 's. For example, R 0 = Å cos θ 0 1,l-r-1 0 k-r-1,1 0 k-r-1,l-r-1 ã

and R 1 = Ñ sin θ 0 1,l-r-1 0 l-r-1,1 I l-r-1 0 n-k-(l-r),1 0 n-k-(l-r),l-r-1 é .

4. Since R 0 has more rows than columns, R 1 may be null which means than R 0 ∈ St(kr, l -r), that is R 0 R 0 = I l-r . A simple calculus shows than Y R = [Y r Y l-r R 0 0 n-k ]. Therefore, the curve c(t) = (1 -t)X + tY R has its n -k columns identically null so it has rank less than k. But it also has rank at least k because rk(Σ) = rk(X) = k. So c and γ R 0 Σ→Λ are of rank exactly k on [0, 1) (and on [0, 1] if l = rk(Λ) = k).

5. At the other extremity, there is R 0 = 0 (and R 1 ∈ St(n -k, l -r)). For r = l, it corresponds to the empty matrix. In this case, XRY = X r Y r . Inspired by the case of the unique geodesic in Sym + (n, k) (with k = l = r), we notice that XH † X Y Y = XH † HY = XDiag(I r , 0)Y = X r Y r = XRY . Therefore, denoting X = U D 1/2 V with U, V ∈ St(n, k) and D ∈ Diag + (k), we have:

XRY = XH † X Λ = U D 1/2 V ((V D 1/2 U ΛU D 1/2 V ) 1/2 ) † V D 1/2 U = U D 1/2 ((D 1/2 U ΛU D 1/2 ) 1/2 ) † D 1/2 U = U D 1/2 U ((U D 1/2 U ΛU D 1/2 U ) 1/2 ) † U D 1/2 U = Σ 1/2 ((Σ 1/2 ΛΣ 1/2 ) 1/2 ) † Σ 1/2 Λ.
Thus the minimizing geodesic writes:

γ 0 Σ→Λ (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t) sym(Σ 1/2 ((Σ 1/2 ΛΣ 1/2 ) 1/2 ) † Σ 1/2 Λ).
When the geodesic is unique, i.e. when r = l, i.e. when R 0 is the empty matrix, it has rank exactly k on [0, 1).

Figure 1 . 1 :

 11 Figure 1.1: Hasse diagram of adjacency of the canonical stratification of the closed unit square B2∞ (0, 1) = {(x, y) ∈ [-1; 1] 2 | max(|x|, |y|)1}. This is an upper semi-lattice. On the diagram, the free variables x and y stand for all the interval (-1; 1).

Figure 1 . 2 :

 12 Figure 1.2: Hasse diagrams of adjacency of the spider (left) and the open book (right). These are lower semi-lattices.

Figure 1 . 4 :

 14 Figure 1.4: Elements of classification of Riemannian metrics on SPD matrices.

Figure 1

 1 Figure 1.5: Invariance under subgroups of GL(n). Each color represents a class of Riemannian metrics that are invariant under the associated group. Affine-invariant metrics belong to all other classes. Some pairs of other invariances (O(n) and Diag(n); S(n) and LT + (n)) imply affine-invariance. This is explained in Chapter 7.

  Example 3.5 (O(I)-invariant inner products on Skew(n)) The irreducible decomposition of Skew(n) with respect to the congruence action of O(I) is Skew

  (SPD) matrices are ubiquitous in data analysis under the form of covariance matrices or correlation matrices. Several O(n)-invariant Riemannian metrics were defined on the SPD cone, in particular the kernel metrics introduced by Hiai and Petz. The class of kernel metrics interpolates between many classical O(n)-invariant metrics and it satisfies key results of stability and completeness. However, it does not contain all the classical O(n)-invariant metrics.

Figure 4 .

 4 Figure 4.1: Super-classes of kernel metrics.

  in the usual sense. It is the pullback metric of the affine-invariant metric via the square diffeomorphism pow 2 : Σ -→ Σ 2[START_REF] Thanwerdas | Is affineinvariance well defined on SPD matrices? A principled continuum of metrics[END_REF].

Definition 4 .

 4 11 (Bures-Wasserstein metric) The Bures-Wasserstein metric is the Riemannian metric associated to the Bures-Wasserstein distance. It is O(n)-invariant and given an eigenvalue decomposition Σ = P DP ∈ Sym + (n) with P ∈ O(n) and D = diag(d 1 , ...

Proposition 4 .

 4 18 (Key results on extended kernel metrics) 1. (Generality) All the metrics in Section 4.3 are extended kernel metrics.2. (Stability)The class of extended kernel metrics is stable under univariate diffeomorphisms and the transformation is the same as in Proposition 4.15.

  .16) 1. (Compatibility) γ = α + β, 2. (Positivity) the symmetric matrix S defined by S ii = γ and S ij = β is positive definite.

Proposition 4 .

 4 21 (Characterizations of subclasses of O(n)-invariant metrics) Let g be an O(n)-invariant metric characterized by the maps α, β, γ.

Definition 5 .

 5 11 (Flat affine connection) Depending on domains of research and authors, a flat affine connection is a connection such that: 1. (Affine geometry) R = 0 and T = 0, 2. (Information geometry) [Amari and Nagaoka, 2000, Section 1.7] there exists a global chart f : (M, ∇) -→ (R dim M , ∂) which is an affine map, i.e. M can be seen as an open set of R dim M via f .

Figure 5 . 1 :

 51 Figure 5.1: Correspondence between MPE metrics on SPD matrices and (α, β)-divergences on positive discrete measures.

Figure 5 . 2 :

 52 Figure 5.2: Lower and upper bounds of the sectional curvature of the mixed-power-Euclidean metrics. Left: lower bound. Right: upper bound. The lower bound of the power-affine metrics (β = -α) is known asα 2 2 [Thanwerdas and Pennec, 2022b]. The bounds were obtained in dimension 3 by taking 1000 random positive diagonal matrices D of determinant 1 (to avoid scaling effects), 1000 random pairs of symmetric matrices (X, Y ) and computing the sectional curvature κ

  2 with a step ∆α = ∆β = 0.05. Diagonal matrices are taken instead of SPD matrices because the MPE metrics are O(n)-invariant.

  Figure 6.1.(a) shows ρ AI , ρ LE , ρ QA with Σ 1 = numerically Σ 1 and Σ 2 , it seems that ρ LE and ρ AI always have three inflection points. In contrast, ρ QA always has one inflection point since (ρ QA ) = -2λ 2 ρ QA (1 -(ρ QA ) 2 ). Analogously, we compare the interpolations of the determinant (Fig.6.1.(b)) and the trace (Fig.6.1.(c)) using several Riemannian metrics: Euclidean (tracemonotonic); log-Euclidean and affine-invariant (determinant-monotonic); power-Euclidean × quotient-affine (correlation-monotonic).

  Figure 6.1: Extrapolation and interpolation between the SPD matrices Σ 1 and Σ 2 using various Riemannian metrics. The metrics E(p) × QA refer to the p-power-Euclidean metric on the diagonal part and the quotient-affine metric on the correlation part. When p tends to 0, E(p) tends to the log-Euclidean metric LE ≡ E(0).

Figure 6 . 2 :

 62 Figure 6.2: Interpolations between SPD matrices Σ 1 and Σ 2 .

3.

  The orthogonal group O(n) = {R ∈ GL(n)|RR = I n } and the special orthogonal group SO(n) = O(n) ∩ SL(n), which are interesting for the invariance of covariance matrices under rotations and symmetries.

  Definition 7.3 (Lie-Cholesky metrics on Sym + (n)) A Lie-Cholesky metric on Sym + (n) is the pullback of a left-invariant metric on the Lie group (LT + (n), ×) via the Cholesky diffeomorphism. It is geodesically complete. Theorem 7.4 (Lie-Cholesky are LT + (n)-invariant metrics) A Riemannian metric on Sym + (n) is a Lie-Cholesky metric if and only if it is LT + (n)-invariant. See the proof of Theorem 7.4 in Section 11.6.

Theorem 8 .

 8 10 (Algorithm to compute D: convergence, speed, complexity) [Archakov and Hansen, 2021] Let S ∈ Sym(n) and D 0 ∈ Diag(n), e.g. D 0 = 0. The sequence (D k ) k∈N ⊂ Diag(n) is recursively defined by D k+1 = ϕ S (D k ) where ϕ S : D ∈ Diag(n) -→ Dlog(Diag(exp(D + S))) is a smooth L-contractant map with L ∈ [0, 1). Then: 1. (Convergence) (D k ) converges to the fixed point D * = D(S) ∈ Diag(n) of ϕ S . 2. (Speed of convergence) The convergence is exponential: D

Theorem 8 .

 8 18 (Compatibility between inversion and log-scaling bijection) For all C ∈ Cor + (n), Log (I(C)) = -Log (C). Otherwise said, the following diagram commutes.

Theorem 8 .

 8 19 (Exp = Cor • exp is a diffeomorphism) The log-scaling bijection Log : Cor + (n) -→ Row 0 (n) is a smooth diffeomorphism. We give the differentials of Log and Exp in function of the differentials of the symmetric matrix logarithm and exponential maps log and exp. For all C ∈ Cor + (n), S, Y ∈ Row 0 (n) and X ∈ Hol(n) such that Σ = D (C) C = exp(S):

Definition 8 .

 8 29 (Projection onto a closed convex set) Let C ⊆ R n be a closed convex set. For all z ∈ R n , there exists a unique x ∈ C satisfying the following equivalent conditions: (i) for all y ∈ C, z -x z -y , (ii) for all y ∈ C, z -x|y -x 0. This point x ∈ C is called the projection of z onto C and denoted Π C (z). It defines a 1-Lipschitz map Π C : R n -→ C which satisfies (Π C ) |C = Id C and Π C (R n \C) ⊆ ∂C.

Figure 8 . 1 :

 81 Figure 8.1: Approximation of the projection.

  called the pseudoinverse or Moore-Penrose inverse. In this work, we only use it for symmetric matrices Σ ∈ Sym(n). Given r = rk(Σ) and an eigenvalue decomposition Σ = U DU = P Diag(D, 0)P with U

  3. (Globally minimizing) We say that c : I -→ M is (globally) minimizing when for all a b in I, L(c |[a,b] ) = d(c(a), c(b)).

2.

  Given that there exists U ∈ O(n) such that Y = XU if and only if XX = Y Y [Groetzner and Dür, 2020, Lemma 2.6], we have Orb(X) = {Y ∈ Mat(n)|Y Y = XX }. The orbit space Mat(n)/O(n) = {Orb(X)|X ∈ Mat(n)} is thus in bijection with the set of covariance matrices Cov(n) = {XX |X ∈ Mat(n)} by the map Orb(X) -→ XX .

Lemma 9 .

 9 20 (Euclidean and Bures-Wasserstein topologies coincide) The Euclidean distance d E and the Bures-Wasserstein distance d BW define the same topology on Cov(n). See the proof of Lemma 9.20 in Section 11.8.9.3.3 Topology, metric and smooth structure of the strataThe set of symmetric positive definite (SPD) matrices Sym+ (n) = {Σ ∈ Sym(n)| sp(Σ) ⊂ (0, +∞)} isan open set of the vector space of symmetric matrices, hence it has a natural structure of smooth manifold. This topology clearly coincides with the topology induced by (Cov(n), d E ), thus it also coincides with the topology induced by (Cov(n), d BW ). The set Sym + (n, k) is in bijection with R n×n k /O(n). The Euclidean distance on R n×n k descends to the Bures-Wasserstein distance on Sym + (n, k) thanks to Equation (9.1). Therefore, the quotient topology coincides with the Bures-Wasserstein topology induced by (Cov(n), d BW ), thus also with the Euclidean topology induced by (Cov(n), d E ) and (Sym(n), d E ).

ã

  3 (continuity of the length), the curve γ is a minimizing geodesic on [-1/λ max , -1/λ min ]. So we already have examples of minimizing geodesics between two degenerate matrices which pass through the principal stratum Sym + (n). For instance, we have Σ = which are linked by the geodesic Exp I 2 (tV ) =

Theorem 9 .

 9 25 (Horizontal lift, tangent space, metric) Let Σ ∈ Sym + (n, k), let X ∈ R n×k * such that Σ = XX ∈ Sym + (n, k) and let V ∈ T Σ Sym + (n, k). Let Σ = U DUbe a singular value decomposition with D ∈ Diag + (k) and U ∈ St(n, k). We denote S = S Σ,V = U S D (U V U )U , where S A (B) denotes the unique solution Z of the Sylvester equation AZ + ZA = B. Note that S Σ,V and (I n -U U ) are independent from the chosen decomposition. 1. (Tangent space)

Theorem 9 .

 9 26 (Bures-Wasserstein geodesics on Sym + (n, k)) Let Σ, Λ ∈ Sym + (n, k) and X, Y ∈ R n×k * such that XX = Σ and Y Y = Λ. Let U ∈ St(n, k) and D ∈ Diag + (k) such that Σ = U DU .

  belongs to I Pre X,Y . Examples 9.29 Let us denote R - θ = Å cos θ sin θ sin θ cos θ ã ∈ O(2) ∩ Sym(2) with det(R - θ ) = -1. XX = Σ, Y Y = Λ and X Y = 2I 2 .Thus the candidate pairs for (H, R) are (2I 2 , I 2 ), (-2I 2 , -I 2 ) and (2R - θ , R - θ ). One can show that:(a) R = I 2 leads to the unique minimizing geodesic γ Σ→Λ

Theorem 9 .

 9 31 (Bures-Wasserstein minimizing geodesic segments in Cov(n)) Let Σ, Λ ∈ Cov(n) with rk(Σ) = k and rk(Λ) = l. Let X, Y ∈ R n×n such that XX = Σ and Y Y = Λ. The two following statements are equivalent:

Lemma 9 .

 9 33 (Elementary algebra) Let Σ, Λ ∈ Cov(n) with rk(Σ) = k and rk(Λ) = l. Let r = rk(ΣΛ).

  .7) where X = (X r X k-r 0) ∈ R n×n and Y = (Y r Y l-r 0) ∈ R n×n are such that XX = Σ, Y Y = Λ and X Y = Diag(D r , 0), and R =

  .8) If r = l, it has rank exactly k on [0, 1).

  vec(Σ(t)) = ε where ε = Diag(±1, ±1) and vec(Λ(t)) ε = Diag(±1, ±1). If the map vec were continuous, then vec(0) = lim t→0 vec(Σ(t)) = ε and vec(0) = lim t→0 vec(Λ(t)) = is a contradiction. In other words, one can reach a symmetric matrix with repeated eigenvalues (here 0) from different bases in the eigenspace corresponding to the repeated eigenvalue.Removing the matrices with repeated eigenvalues is not sufficient because one can still turn around the axis RI 2 . That is what we prove now. Assume there exists a continuous map of eigenvectors vec : Sym= (n) -→ O(n). Let D = Diag(-1, 1) ∈ Diag(2), P (t) = Å cos(πt) sin(πt) -sin(πt) cos(πt) ã ∈ O(2) and Σ(t) = P (t)DP (t) ∈ Sym = (n). Then P (0) = I 2 , P (1) =-I 2 and Σ(0) = Σ(1) = D. We define the continuous maps Q

  5 (O(I)-invariant inner products on Skew(n)) The irreducible decomposition of Skew(n) with respect to the congruence action of O(I) is Skew

  O(I)-invariant inner products on Sym(n)) The irreducible decomposition of Sym(n) with respect to the congruence action of O(I) is Sym

  The terms of the direct sum are stable. The terms span(I ii n ) and Sym 0n (k i ) are O(I)-irreducible because they are O n (k i )-irreducible. The modules Sym n (k i , k j ) and Skew n (k i , k j ) are isomorphic as O(k i ) × O(k j )-modules to Mat(k i , kj ): it suffices to keep the Upper Triangular (UT) part and take the opposite of the Lower Triangular

  p} are clearly module isomorphisms with respect to O(I), where O(I) trivially acts on R by (R, λ) ∈ O(I) × R -→ λ ∈ R. They are isometries since tr Ä λ √

  Horizontal lift) Let Σ ∈ Sym + (n) and C = Cor(Σ) ∈ Cor

  .28) -(X # • ψ(BDiag(C -1 Y )1) -Y # • ψ(BDiag(C -1 X)1)), computation of ver[X # , Y # ] = C • ψ(BDiag(C -1 [X # , Y # ])1) brings some simplifications. We proceed line by line. 1. To simplify line (11.28), we plugY = Y # -1 2 (Diag(Y # )C + CDiag(Y # )). We proceed term by term.

  -1 Diag(X # )Y ) = Diag(C -1 Diag(X # )Y # ) (11.31) -1 2 Diag(C -1 Diag(X # )Diag(Y # )C) -1 2 Diag(C -1 Diag(X # )CDiag(Y # ))

  -1 Y Diag(X # )) = Diag(C -1 Y # Diag(X # )

2.

  To simplify line (11.29), we can show by computing coordinate by coordinate that Diag(C -1 (X # • ψ(BDiag(C -1 Y )1)))1 = (C -1 • X # )BDiag(C -1 Y )1.Hence, line (11.29) cancels the first term of v 0 .3. Finally, a nicer expression of the second term of v 0 can be obtained by noticing that Diag(Y# )1 = -2BDiag(C -1 Y )1. After simple calculations, we get (C • C -1 X # C -1 )BDiag(C -1 Y )1 = -1 2 Diag(CDiag(X # )C -1 Y # C -1 )1. To summarize, we have ver[X # , Y # ] = C • ψ( 1 2 BD1) with D = D(X, Y ) -D(Y, X) ∈ Diag(n) and D(X, Y ) = Diag(C -1 Diag(X # )Y # -C -1 Y # C -1 Diag(X # )C). Finally, since G C (C • ψ(µ), C • ψ(µ)) = 2µ Bµ for any vector µ ∈ R n , we get G C (ver[X # , Y # ], ver[X # , Y # ]) = 12 1 DBD1, as expected.

  1 (Bounds of curvature) The sectional curvature of the quotient-affine metric takes positive and negative values. It is bounded from below and unbounded from above.Proof appendix of Theorem7.1 (Bounds of curvature). First of all, κ C (X, Y ) κ AI C (X, Y ) -1 2 so the curvature is bounded from below. Second, at C = I n , X # = X and Y # = Y so Diag(X # ) = Diag(Y # ) = 0 and µ = 0. Hence, κ In (X, Y ) = κ AI In (X, Y ) 0 and for example κ In (E ij , E ik ) = -1 8 < 0 with i = j = k = i ∈ {1, ..., n}[START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF]. So the curvature takes negative values. Third, let X = 11 -I n and Y = µ1 + 1µ -2 diag(µ) with sum(µ) = 1 µ = 0 where µ ∈ R n . Let C = (1 -ρ)I n + ρ11 ∈ Cor + (n) for ρ ∈ (-

  -1 = α I n + β 11 with α = 1 α and α + nβ = 1 α+nβ , i.e. β = -β α(α+nβ) . Note that αβ + βα + nββ = 0.Moreover, for all i = j ∈ {1, ..., n},[Σ•Σ -1 ] ii = (α+β)(α +β ) and [Σ•Σ -1 ] ij = ββ . Therefore, I n + Σ • Σ -1 = AI n + B11 with A = 1 + (α + β)(α + β ) -ββ = 2 + αβ + βα = 2α(α+nβ)+nβ 2 α(α+nβ)and B = ββ . Note that A + nB = 2. And (I n + Σ • Σ -1 ) -1 = A I n + B 11 with A = 1 A and B = -B 2A . We compute κ C (X, Y ) where C = αI n + β11 ∈ Cor + (n) with α + β = 1. C -1 X = (α I n + β 11 )(11 -I n ) = -α I n + (α + (n -1)β -1 X) = (n -1)β I n , (I n + C • C -1 ) -1 Diag(C -1 X)1 = (n -1)β (A I n + B 11 )1 = n -1 2 β 1, X # = X -(n -1)β C, Diag(X # ) = -(n -1)β I n , D(X, Y ) = Diag(C -1 Diag(X # )Y # -C -1 Y # C -1 Diag(X # )C) = 0, C -1 Y = (α I n + β 11 )(µ1 + 1µ -2diag(µ)) = α µ1 + (α + (n -2)β )1µ -2α diag(µ), Diag(C -1 Y ) = (n -2)β diag(µ), (I n + C • C -1 ) -1 Diag(C -1 Y )1 = (n -2)β (A I n + B 11 )µ = (n -2)β A µ, Y # = Y -(n -2)β A (diag(µ)C + Cdiag(µ)), Diag(Y # ) = -2(n -2)β A diag(µ), C -1 Diag(Y # )X # = -2(n -2)β A (α I n + β 11 )diag(µ)(11 -I n ) = -2(n -2)β A (-α diag(µ) + α µ1 -β 1µ ), C -1 X # C -1 Diag(Y # )C = -2(n -2)β A (-α I n + (α + (n -1)β )11 )(α I n + β 11 )diag(µ)(αI n + β11 ) = -2(n -2)β A (-α I n + (α + (n -1)β )11 )(diag(µ) + α βµ1 + αβ 1µ ) = -2(n -2)β A [-α (diag(µ) + α βµ1 + αβ 1µ ) + (α + (n -1)β )(1 + nαβ )1µ ], C -1 Diag(Y # )X # -C -1 X # C -1 Diag(Y # )C = -2(n -2)β A [α (1 + α β)µ1 -(α + (n -1)β )(1 + nαβ )1µ ], nβ) + nβ 2 2α + nβ α 2 (α + nβ) 2 diag(µ), 1 D(I n + C • C -1 ) -1 D1 = A Å 2n(n -2) β 2 2α(α + nβ) + nβ 2 2α + nβ α 2 (α + nβ) 2 ã 2 µ 2 = 4n 2 (n -2) 2 β 4 (2α + nβ) 2 [α(α + nβ)(2α(α + nβ) + nβ 2 )] 3 µ 2 ,where we used α + β = 1 fromEquation (11.33). Now, we compute g QA C (X, X), g QA C (Y, Y ) and g QA C (X, Y ).

  g C (X, X) = tr((C -1 X) 2 ) -21 Diag(C -1 X)(I n + C • C -1 ) -1 Diag(C -1 X)1 = tr Ä (-α I n + (α + (n -1)β )11 ) 2 ä -2(n -1) 2 β 2 sum(A I n + B 11 ) = tr(α 2 I n + (n(α + (n -1)β ) 2 -2α (α + (n -1)β ))11 ) -2(n -1) 2 β 2 n(A + nB ) = n((n -1)α 2 + 2(n -1) 2 α β + n(n -1) 2 β 2 ) -n(n -1) 2 β 2 = n(n -1) α 2 (α + nβ) 2 ((α + nβ) 2 -2(n -1)β(α + nβ) + (n -1) 2 β 2 ) = n(n -1) α 2 (α + nβ) 2 , g C (Y, Y ) = tr((C -1 Y ) 2 ) -21 Diag(C -1 Y )(I n + C • C -1 ) -1 Diag(C -1 Y )1 = tr Ä (α µ1 + (α + (n -2)β )1µ -2α diag(µ)) 2 ä -2(n -2) 2 β 2 µ (A I n + B 11 )µ = tr(4α 2 diag(µ) 2 + α (α + (n -2)β )(nµµ + µ 2 11 )) -2α tr((2α + (n -2)β )µµ + α (µ • µ)1 + (α + (n -2)β )1(µ • µ) )) -2(n -2) 2 β 2 A µ 2 = µ 2 (4α 2 + 2nα (α + (n -2)β ) -4α (2α + (n -2)β ) -2(n -2) -2) µ 2 α 2 (α + nβ)(2α(α + nβ) + nβ 2 ) ((1 + β)(2α(α + nβ) + nβ 2 ) -(n -2)αβ 2 ) 2α(1+β)(α+nβ)+2β 2 (α+nβ)=2(α+nβ) = 4(n -2) µ 2 α 2 (2α(α + nβ) + nβ 2 ) , g C (X, Y ) = tr(C -1 XC -1 Y ) -21 Diag(C -1 X)(I n + C • C -1 ) -1 Diag(C -1 Y )1 = tr((-α I n + (α + (n -1)β )11 )(α µ1 + (α + (n -2)β )1µ -2α diag(µ)))-2(n -1)(n -2)β 2 1 (A I n + β 11 )µ = constant × sum(µ) = 0.

  1) 3 so this quantity tends to +∞. Finally withC = (1ρ)I n + ρ11 ∈ Cor + (n) with ρ ∈ (-1 n-1 ; 1), X = I n -11 ∈ T C Cor + (n) and Y = µ1 + 1µ -2diag(µ) ∈ T C Cor + (n), we have:

  X) = Diag(L) -1 L Low S (L -1 XL -) 2 ,where L = Chol(C) ∈ L. The square distance between C and C = φ(L ) writes:d CPHC (C, C ) 2 = n i=2 arccosh(-Q(L i , L i )) 2 , where L i , L i are the i-th rows of L, L respectively.Proof of Theorem 7.14 (Symmetric space structure). The product of Riemannian symmetric spaces is a Riemannian symmetric space. The product of manifolds with sectional curvature bounded by [a, b] with a 0 b has its sectional curvature bounded by [a, b]. (This is also valid for (-∞, b] and [a, +∞).) Let k 2 such that α k = min i 2 α i . The values a = -1

C

  (X, X) = n i=2 Diag(L) -1 ii α i Y i• 2 and the general PHC distance writes d(C, C ) 2 = n i=2 α i arccos(-Q(L i , L i )) 2 . 19 (Group operations)The group operations associated to the Lie-Cholesky group structure on full-rank correlation matrices are, for all C, C ,C i ∈ Cor + (n), X ∈ T C Cor + (n) Hol(n), t ∈ R: • (Exponential map) Exp C (tX) = Θ -1 (Θ(C) exp(t Θ(C) -1 d C Θ(X)), • (Logarithm map) Log C (C ) = (d C Θ) -1 (Θ(C) log(Θ(C) -1 Θ(C ))),

  . It is characterized by 0 = k i=1 LogΓ(Γ i ) = Γ k i=1 log( Γ-1 Γ i ), which is equivalent to k i=1 log( Γ-1 Γ i ) = 0. (Geodesics in dimension 2) Let C 1 = C(ρ 1 ), C 2 = C(ρ 2 ) ∈ Cor + (2) with ρ 1 , ρ 2 ∈ (-1, 1).1. Quotient-affine metrics and poly-hyperbolic-Cholesky metrics coincide (up to a scaling factor). The geodesic between C 1 and C 2 is C(ρ(t)) for t ∈ R where:ρ(t) = ρ 1 cosh(λt) + sinh(λt) ρ 1 sinh(λt) + cosh(λt) , where λ = log » 1+ρ 2 1-ρ 2 -log » 1+ρ 1

  5 (Incompatibility between cor-inversion and off-log bijection) Let n 3. There exists C ∈ Cor + (n), such that Log(I(C)) = -Log(C). Otherwise said, the following diagram does not commute.Cor + (n)Proof of Theorem 8.5 (Incompatibility between cor-inversion and off-log bijection).It is easy to see it numerically. For a formal proof, one can look for a matrix C ∈ Cor + (3) such that log(C) and log(I(C)) are easy to compute manually. We propose the following example, with x

  kk )P 1k P 2k + ln(∆ kk )Q 1k Q 2k )

  QAIn = Cor • exp, we clearly have (i) ⇐⇒ (ii) because X = log(∆C∆) and ∆ = Diag(exp(X)) 1/2 . To prove (i) ⇐⇒ (iii), let us compute the differential and the Hessian of f . Let ∆ ∈ Diag + (n) and D, D ∈ Diag(n). We denote E = D∆ -1 ∈ Diag(n) andA = ∆C∆ = P BP ∈ Sym + (n) with P ∈ O(n) and B ∈ Diag + (n). d ∆ f (D) = 2 tr(log(∆C∆) d ∆C∆ log((DC∆ + ∆CD))) = 2 tr(log(A) d A log(EA + AE )) = 2 tr(log(B) d B log(P EP B + BP E P )) = 2 tr(log(B)B -1 (P EP B + BP E P )) = 4 tr(log(A)E) = 4 tr(log(∆C∆)D∆ -1 ), d ∆ f = 0 ⇐⇒ Diag(log(∆C∆)) = 0 ⇐⇒ log(∆C∆) ∈ Hol(n), H ∆ f (D, D ) = 4 tr(d A log(EA + AE )D ∆ -1 -log(A)D ∆ -1 D∆ -1 ) =4tr(2∆ -2 DD -Diag(log(∆C∆))∆ -2 DD ), H ∆ f (D, D) = 4 tr((2I n -Diag(log(∆C∆)))∆ -2 D 2 ).

  19 (Exp = Cor • exp is a diffeomorphism) The log-scaling bijection Log : Cor + (n) -→ Row 0 (n) is a smooth diffeomorphism. We give the differentials of Log and Exp in function of the differentials of the symmetric matrix logarithm and exponential maps log and exp. For allC ∈ Cor + (n), S, Y ∈ Row 0 (n) and X ∈ Hol(n) such that Σ = D (C) C = exp(S): d S Exp (Y ) = ∆ -1 ï d S exp(Y ) -1 2 (∆ -2 Diag(d S exp(Y )) Σ + Σ Diag(d S exp(Y )) ∆ -2 ) ò ∆ -1 , d C Log (X) = d Σ log Å ∆X∆ + 1 2 (X 0 Σ + ΣX 0 ) ã ,where ∆ = Diag(Σ) 1/2 and X 0 = -2 diag((I n + Σ) -1 ∆X∆1). Proof of Theorem 8.19 (Exp = Cor • exp is a diffeomorphism).It suffices to show that D is smooth. We apply the implicit function theorem to the smooth function Φ :(Σ, ∆) ∈ Sym + (n) × Diag + (n) -→ ∆Σ∆1 -1 ∈ (R + ) n which satisfies ∆ = D (Σ) if and only if Φ (Σ, ∆) = 0. Let us prove that for all (Σ, ∆) ∈ (Φ ) -1 (0), the differential of the partial function Φ Σ : ∆ ∈ Diag + (n) -→ Φ (Σ, ∆) ∈ (R + ) n is invertible. In the direction D ∈ T ∆ Diag + (n) = Diag(n): d ∆ Φ Σ (D) = DΣ∆1 + ∆ΣD1 = D∆ -1 1 + ∆ΣD1 = ∆(∆ -2 + Σ)D1. Since ∆(∆ -2 + Σ) ∈ GL(n), the differential is invertible so D is smooth. Since Exp = Cor • exp, we have d S Exp (Y ) = d Σ Cor(d S exp(Y )) with Σ = exp(S). Using d Σ Cor(Z) = ∆ -1 Z -1 2 (∆ -2 Diag(Z)Σ + ΣDiag(Z)∆ -2 ) ∆ -1 with ∆ = Diag(Σ) 1/2 and Z = d S exp(Y ) ∈ T Σ Row + 1 (n) = Row 0 (n),we get the expected result. Now we want to invert the relation X = d S Exp (Y ) to get Y = d Σ Log (X). We use the intermediate matrices Σ ∈ Row + 1 (n) and Z ∈ Row 0 (n) and the relations X = d Σ Cor(Z) and Y = d Σ log(Diag(Z)1 = -2(I n + Σ) -1 ∆X∆1, ∆ -2 Diag(Z) = X 0 , Z = ∆X∆ + 1 2 (X 0 Σ + ΣX 0 ), which allows to conclude with d Σ Log (X) = Y = d Σ log(Z).

  3. (Distance) d(C 1 , C 2 ) = |λ| = |F (ρ 2 ) -F (ρ 1 )| (up to a scaling factor √ α).

  n-1 . Let us prove that y is bounded. Let y 0 = α 1/n x. We consider the line D = y 0 + R 1 y 0 and the infinite right circular cylinder C centered on D and passing through x. More precisely, denoting R = y 0 -x -y 0 -x|u u , the cylinder isC = {p ∈ R n |d(p, D) R}. Since the line D = x + R 1 x is parallel to D and since x ∈ C, we have D ⊂ C. For z(a) ∈ D , we define p = Π D z(a) = z 0 + y 0 -x -y 0 -x|u u ∈ D. Then: y(a) -y 0 = Π Ωα (z(a)) -Π Ωα (p) z(a) -p = R.So y is bounded. Let us prove that lim a→+∞ y(a) = y 0 . We denote v(a) = y(a)-x-y(a)-x|u u y(a)-x-y(a)-x|u u ∈ S n-1 so that u ⊥ v(a) and the triangle T (a) with vertices x, y(a), z(a) is in the plane (x, u, v(a)). In T (a), since x = Π Ω (z(a)) and y(a) ∈ Ω, we have x -z(a)|x -y(a) 0, i.e. cos( Ÿ z(a)xy(a)) 0, i.e. Ÿ z(a)xy(a) ∈ [ π 2 , π). Hence the angles θ(a) = Ÿ xz(a)y(a) and ϕ(a) = Ÿ xy(a)z(a) are in (0, π 2 ]. Moreover, the law of sines ensures that y(a)-x sin θ(a) = z(a)-x sin ϕ(a) z(a) -x -→ a→+∞ +∞.Since y(a) -x is bounded, lim a→+∞ sin θ(a) = 0 so lim a→+∞ θ(a) = 0. Hence, since v is bounded and θ(a)

.

  Since y = y(a) is characterized by y(a) -z(a) 1 y(a) and n i=1 y i (a) = α, it is implicitly Let us prove that D : a∈ R + -→ y(a) -x 2 = n i=1 (y i (a) -x i ) 2 ∈ R + is increasing. a)y i (a), u(a) = M (a)v(a),where u(a) = (1n n j=1 y j (a)x j )1 -y(a) x , v(a)= y(a)y (a) (product of vectors are Hadamard products, i.e. coordinate-wise) and: Thus we can get y (a) by inverting M (a). Let us compute the inverse with the formula M -1 = 1 det(M ) com(M ) where com(M ) is the comatrix of M . First, one can show that the matrix with diagonal Diag(m 1 , ..., m n ) and off-diagonal terms all equal to µ / ∈ {m 1 , ..., m n }, then its determinant is det(M ) = n i=1 (m i -µ)(1 + µ n i=1 1 m i -µ

  M ) = Diag(c 1 , ..., c n ) + 1 nδ [c i c j ] 1 i,j n ,wherec i = c i (a) = 1 m i -µ = 1 1+ b(a) y i (a) 2 and δ = δ(a) = 1 -1 n n i=1 c i (a) > 0. Now it suffices to compute D (a)and show that it is positive. We remove the dependence in a for readability.We denote s = 1

11. 8 . 1

 81 Proof of Lemma 9.20 Lemma 9.20 (Euclidean and Bures-Wasserstein topologies coincide) The Euclidean distance d E and the Bures-Wasserstein distance d BW define the same topology on Cov(n).Proof of. The map π :X ∈ Mat(n) -→ XX ∈ (Cov(n), d E) is continuous so the quotient topology, i.e. the topology induced by the Bures-Wasserstein distance, is finer than the Euclidean topology. Conversely, let U be an open set for the Bures-Wasserstein distance. Let Σ ∈ U. Let ε > 0 such that the Bures-Wasserstein ballB BW (Σ, ε) is included in U. The set V = pow 2 (B E (Σ 1/2 , ε)) is open for the Euclidean distance because the map pow 2 : Σ -→ Σ 2 is a homeomorphism of (Cov(n), d E ). Moreover, if Λ ∈ V, then d BW (Σ, Λ) d E (Σ 1/2 , Λ 1/2 ) ε so Λ ∈ B BW (Σ, ε) ⊆ U. So V ⊆ U is a Euclidean neighborhood of Σ,so U is open for the Euclidean distance. Therefore the two topologies coincide. (Bures-Wasserstein geodesics on Sym + (n)) Let Σ ∈ Sym + (n).

  (Horizontal lift, tangent space, metric) Let Σ ∈ Sym + (n, k), let X ∈ R n×k * such that Σ = XX ∈ Sym + (n, k) and let V ∈ T Σ Sym + (n, k). Let Σ = U DU be a singular value decomposition with D ∈ Diag + (k) and U ∈ St(n, k). We denote S = S Σ,V = U S D (U V U )U ,where S A (B) denotes the unique solution Z of the Sylvester equation AZ + ZA = B. Note that S Σ,V and (I n -U U ) are independent from the chosen decomposition. 1. (Tangent space) T Σ Sym + (n, k) = {V ∈ Sym(n)|X ⊥ V X ⊥ = 0}, 2. (Horizontal lift)

  1 , a pair of matrices R a,b ∈ I Log c 0 (a),c 0 (b) and H a,b = H c 0 (a),c 0 (b),R a,b = c 0 (a) c 0 (b) ∈ Cov(p). We compute H a,b :H a,b = c 0 (a) c 0 (b) = [(1 -a)X 0 + aY 0 ] [(1 -b)X 0 + bY 0 ] = (1 -a)(1 -b)X 0 X 0 + abY 0 Y 0 + (1 -a)bX 0 Y 0 + a(1 -b)Y 0 X 0 . Therefore, lim a→0 b→1 H a,b = X 0 Y 0 so X 0 Y 0 ∈ Cov(p). Since [X 0 0][X 0 0] = Σ and [Y 0 0][Y 0 0] = Λ, there exist P, Q ∈ O(n) such that X = [X 0 0]P and Y = [Y 0 0]Q. Thus the curve c : t ∈ [0, 1] -→ (1 -t)X + tY R ∈ Mat(n) with R = P Q ∈ O(n) satisfies γ(t) = c(t)c(

  (Elementary algebra) Let Σ, Λ ∈ Cov(n) with rk(Σ) = k and rk(Λ) = l. Let r = rk(ΣΛ).

  .2: Number of Bures-Wasserstein minimizing geodesic segments (n k l r) Proof of Theorem 9.34 (Number of Bures-Wasserstein minimizing geodesic segments in Cov(n)).1. Without loss of generality, let us choose X, Y ∈ Mat(n) with an convenient form. We choose X 0 ∈ R n×k * and Y 0 ∈ R n×l * as in the proof of the previous lemma. Given a singular value decomposition ofX 0 Y 0 = U k DV l with U k ∈ O(k), V l ∈ O(l) and D = Diag(D r , 0) with D r ∈ Diag + (r), we define P = Diag(U k , I n-k ) ∈ O(n) and Q = Diag(V l , I n-l ) ∈ O(n). We choose X = [X 0 0]P = [X k 0] with X k = X 0 U k ∈ R n×k * and Y = [Y 0 0]Q = [Y l 0] with Y l = Y 0 V l ∈ R n×l * . Therefore we have XX = Σ, Y Y = Λ, X = [X k 0], Y = [Y l 0] and X Y = Diag(D r , 0). We denote X = [X r X k-r 0] and Y = [Y r Y l-r 0] with X r , Y r ∈ R n×r *

  and we propose intermediate classes of metrics with stability properties. The smaller the class is, the more properties are shared by all the metrics of the class. So we propose a sequence of nested classes defined by conditions on the three functions α, β, γ ranging from the noted metrics (affine-invariant, log-Euclidean, Bures-Wasserstein, BKM, etc.) to the whole class of O(n)-invariant metrics. Notable classes are the families of Mixed-Power-Euclidean and Mixed-Euclidean metrics (Section 5.5) which encompass with few parameters most of the noted metrics. This is represented on Figure 1.4 and on Figure 4.1 in Chapter 4. Thirdly, we also show that affine-invariant metrics are characterized by the pairs of invariance under R

+ and SL(n)

, or Diag + (n) and O(n), and less evidently under LT + (n) (lower triangular matrices with positive diagonal) and S(n)

Table 1 .

 1 3: Properties of Riemannian metrics on the open elliptope of full-rank correlation matrices. Abbreviations: [D19] is[START_REF] David | A Riemannian structure for correlation matrices[END_REF], [A21] is[START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF], Ch. is Chapter, Comp. is Complete, S

  is invariant), the ψ i 's are equivariant and the inner products (•|•) k are invariant. So this bilinear form is a ρ-invariant inner product on V .Note that the choice of the inner product (•|•) k on W k instead of λ(•|•) k with λ > 0 does not affect the general form of the inner product: it suffices to replace S k ∈ Sym + (m k ) by λS k ∈ Sym + (m k ). Neither does the choice of the isometric parameterization ψ k

	projections are equivariant (because •|•
	.3)
	Conversely, this bilinear form is an inner product since it is the sum of inner products on
	the V k 's (Section 3.2.3.3) which are supplementary. It is ρ-invariant because the orthogonal

  in Table 4.6. The Riemannian operations are detailed in Table 4.7. Let us precise what was known and what is new in Table 4.7.

  y) 2 . Note that the class of mean kernel metrics is not stable under univariate diffeomorphisms because of the non-decreasing property required for mean kernel metrics.

	Metric		φ(x, y)	Mean m	θ
	Euclidean		1	Any mean	0
	Log-Euclidean	(	x-y log(x)-log(y) ) 2 Logarithmic mean 2
	Affine-invariant		xy	Geometric mean 2
	Polar-affine Bures-Wasserstein		( 2xy x+y ) 2 4 x+y 2	Harmonic mean	2

  4, we gave four key results on BOD/MOD metrics in Propositions 4.15 and 4.16, and four key results on BOST/MOST metrics in Proposition 4.18. Here we give the

	Previous description	New designation
	Kernel metric	BOD metric
	Mean kernel metric	MOD metric
	Extended kernel metric	BOST metric
	Extended mean kernel metric	MOST metric
	Table 4.10: Name correspondences for kernel metrics and sub/super-classes.
	counterpart of these propositions for O(n)-invariant metrics.
	Proposition 4.22 (Key results on O(n)-invariant metrics)
	1. (Generality) The class of O(n)-invariant metrics obviously contains the classes of BOD,
	MOD, BOST, MOST metrics, hence it contains all the metrics in Section 4.3.

2. (Stability)

The class of O(n)-invariant metrics is obviously stable by O(n)-equivariant diffeomorphisms of Sym + (n). Hence it is stable by univariate diffeomorphisms f : Sym

  . However, the class of mean kernel metrics is not stable under

	Metric	Kernel φ(x, y)	Mean m	Power θ
	Euclidean		1	Any mean	0
	Log-Euclidean	(	x-y log(x)-log(y) ) 2 Logarithmic mean	2
	Affine-invariant		xy	Geometric mean	2
	Bures-Wasserstein Bogoliubov-Kubo-Mori		4 x+y 2 x-y log(x)-log(y)	Arithmetic mean Logarithmic mean	1 1
	Table 5.1: Bivariate functions of the main O(n)-invariant metrics on SPD matrices.

  sums the terms of the vector,

	Matrix vector spaces		Matrix manifolds
	Mat(n)	Squared	GL(n)	General linear group
	Skew(n)	Skew-symmetric	O(n) SO(n)	Orthogonal group Special orthogonal group
	Sym(n)	Symmetric	Sym + (n)	Sym Positive Definite cone
	Hol(n)	Symmetric hollow	Cor + (n) Full-rank correlation elliptope
	LT(n)	Lower Triangular	LT + (n)	LT with positive diagonal
	LT 0 (n) LT with null diagonal	LT 1 (n)	LT with unit diagonal
	Diag(n)	Diagonal	Diag + (n)	Positive diagonal group
		Table 7.1: Matrix space notations.

Table 8

 8

.1: Two approaches to define permutation-invariant log-Euclidean metrics on Cor + (n).

  manifold of full-rank correlation matrices is called the open elliptope. It it relatively open in Sym(n), i.e. open in I n + Hol(n).We introduce the smooth submersion Cor : Σ

  . Bures-Wasserstein minimizing geodesics between covariance matrices of different ranks. Preprint. Submission planned in April 2022 to SIMAX. (No continuous map of eigenvectors on Sym = (n)) Let n 2. There exists no continuous map vec : Sym

	• Thanwerdas, Yann (2022). Permutation-invariant log-Euclidean geometries on full-rank Chapter 11
	correlation matrices. Preprint. Submission planned in May 2022 to SIMAX.
	Proofs
	11.1 Proofs of Chapter 2
	Part VII 11.1.1 Proof of Theorem 2.7
	Theorem 2.7
	Appendix

  11.37) Hence, g is a Lie-Cholesky metric if and only if Chol * g is left-invariant if and only if the left terms are equal if and only if the right terms are equal if and only if g is invariant under the action of LT + (n).

	11.6.4 Proof of Lemma 7.8

Lemma 7.8 (Operator ad * for Lie-Cholesky metrics) Let LC(A) be a Lie-Cholesky metric characterized by A ∈ Sym + ( n(n+1) 2

ρ(Σ) Σ -Λ .

Remerciements

Thus the partial correlations are the weights of the arrows in the network [Lauritzen, 1996, Koller and Friedman, 2009, Epskamp and Fried, 2018]. This approach is applied in many domains such as genomics [de la Fuente et al., 2004, Peng et al., 2009] or brain connectomics [START_REF] Marrelec | Partial correlation for functional brain interactivity investigation in functional MRI[END_REF]. Hence the importance of partial correlations confirms that the corinversion is a relevant concept.

We consider the cor-inversion as analogous to the matrix inversion for SPD matrices inv : Sym + (n) -→ Sym + (n). The cor-inversion commutes with permutations on full-rank correlation matrices as well as the inversion commutes with the congruence by O(n) on SPD matrices. Moreover, for all signature I, the space of block equicorrelation matrices of signature I is stable by the cor-inversion. In addition, in relation to the problem of this chapter, we have the following result.

Theorem 8.2 (Compatibility between the second approach and the cor-inversion) Let V be a vector space stable by permutations and such that Sym(n) = V ⊕ Diag(n). We assume that the claim on the right part of Table 8.1 is true, i.e. for all C ∈ Cor + (n), there exists a unique ∆ ∈ Diag + (n) such that log(∆C∆) ∈ V . This defines the inverse bijections Log : C ∈ Cor + (n) -→ log(∆C∆) ∈ V and Exp = Cor • exp : V -→ Cor + (n). Then we automatically have Log (I(C)) = -Log (C), i.e. the following diagram commutes.

Proof. Note that for all Σ ∈ Sym + (n), we have Cor(Σ -1 ) = Cor(Cor(Σ) -1 ). Indeed, if Σ = DCD with C = Cor(Σ) ∈ Cor + (n), then Cor(Σ -1 ) = Cor(D -1 C -1 D -1 ) = Cor(C -1 ). Therefore, for all X ∈ V we have Exp (-X) = Cor(exp(-X)) = Cor(exp(X) -1 ) = Cor(Exp (X) -1 ) = I(Exp (X)). Thus with C = Exp (X), we have Log (I(X)) = -Log (C).

Otherwise said, the multiplicative approach is automatically compatible with the corinversion. This is due to the use of the congruence action of positive diagonal matrices on SPD matrices instead of the additive action of diagonal matrices on symmetric matrices. Indeed, the former is intrinsically related to the definition of a correlation matrix. On the contrary, we can expect that the bijections built via the additive approach are not compatible with the cor-inversion in general.

Thus, if one finds a vector space V satisfying the claim and if the bijections are smooth, the log-Euclidean metrics defined by pullback will automatically be inverse-consistent. This is quite satisfying for log-Euclidean metrics on full-rank correlation matrices in analogy with SPD matrices.

Permutation-invariant log-Euclidean metrics via the off-log diffeomorphism

In this section, we rephrase the framework of [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF] (Section 8.2.1) to ease the comprehension of the next sections by analogy. In Section 8.2.2, we prove that the

Part VI Conclusion

close the loop initiated in the introduction and reproduced on Figure 10.1. To conclude, the concepts and geometric tools introduced in this thesis could be the basis of many future developments to tackle challenging theoretical and practical problems.

Publications

Journal papers • Thanwerdas, Yann and Pennec, Xavier (2022). The geometry of mixed-Euclidean metrics on symmetric positive definite matrices. Differential Geometry and its Applications, 81.

• Miolane, N., Guigui, N., Le Brigant, A., Mathe, J., Hou, B., Thanwerdas, Y., Heyder, S., Peltre, O., Koep, N., Zaatiti, H., Hajri, H., Cabanes, Y., Gerald, T., Chauchat, P., Shewmake, C., Brooks, D., Kainz, B., Donnat, C., Holmes, S., and Pennec, X. (2020). Geomstats: A Python Package for Riemannian Geometry in Machine Learning. Journal of Machine Learning Research, 21(223):1-9.

Journal submissions

• Thanwerdas, Yann and Pennec, Xavier (2022). Theoretically and computationally convenient geometries on full-rank correlation matrices. Submitted to the SIAM Journal on Matrix Analysis and Applications.

• Thanwerdas, Yann and Pennec, Xavier (2022). O(n)-invariant Riemannian metrics on SPD matrices. Submitted to Linear Algebra and its Applications (2 nd round after minor revisions). Among the five running papers for the best paper award.

Conference articles

• Thanwerdas, Yann and Pennec, Xavier (2019). Is affine-invariance well defined on SPD matrices? A principled continuum of metrics. In Proceedings of GSI 2019 -4th conference on Geometric Science of Information, volume 11712 of Lecture Notes in Computer Science, pages 502-510, Toulouse, France. Springer International Publishing.

Proof of Lemma 4.6. (a) We write X|X = i,j,k,l a ij,kl X ij X kl a general inner product. Note that a ij,kl = a ji,kl = a ji,lk = a ij,lk by symmetry of X and a ij,kl = a kl,ij by symmetry of the inner product. We use the invariance under the matrix ε m ∈ D ± (n) with -1 on the m-th component and 1 elsewhere, for m ∈ {1, ..., n}. We denote P XOR Q = 1 if the "exclusive or" between propositions P and Q holds, and otherwise

Therefore, if there exists m ∈ {1, ..., n} such that θ ijklm = -1, then a ij,kl = 0. One can easily show that θ ijklm = -1 if and only if m equals exactly one or exactly three index(es) among i, j, k, l. There exists such an m if:

• card({i, j, k, l}) = 4, i.e. i, j, k, l are distinct,

• card({i, j, k, l}) = 3,

• card({i, j, k, l}) = 2 and three of them are equal.

Thus we are left with X|X = i<j 4a ij,ij X 2 ij + i,j a ii,jj X ii X jj . Then, we get the expression (4.4) by denoting α ij = 2a ij,ij and S ij = a ii,jj = S ji . Since the quadratic form splits into two quadratic forms defined on supplementary vector spaces (off-diagonal and diagonal terms), it is positive definite if and only if these two quadratic forms are positive definite, i.e. α ij > 0 for all i = j and S is positive definite. Conversely, Equation (4.4) clearly defines D ± (n)-invariant inner products.

(b) A S ± (n)-invariant inner product on symmetric matrices is D ± (n)-invariant so it is of the form of Equation (4.4). Since it is invariant under permutations, we have α ij = α kl =: α and S ij = S kl =: β for all i = j and k = l and S ii = S jj =: γ for all i, j. Under these notations, Equation (4.4) becomes Equation (4.5). Since S = (γ -β) I n + β 11 , then S ∈ Sym + (n) if and only if γ -β > 0 and γ -β + nβ > 0 as expected. Conversely, Equation (4.5) clearly defines S ± (n)-invariant inner products.

Proof of affine-invariant curvature

We prove that the sectional curvature of the affine-invariant metric satisfies κ ∈ [-1/2α; 0]. More precisely, the Riemann and sectional curvatures are:

where

Other terms are null. Proof of sectional curvature in Table 4.5. Firstly, we compute the sectional curvature of the affine-invariant metrics for β = 0 at Σ ∈ Sym + (n) in the orthonormal basis (Σ 1/2 E ij Σ 1/2 ) 1 i j n , Now, we simply rewrite the equation:

Proofs of Chapter 5

In this section, we prove the results of Chapter 5: Geometry of Mixed-Euclidean metrics.

11.4.1 Proof of Theorem 5.9 Theorem 5.9 (Sufficient condition for power-Wasserstein to be mean kernel) The power-Wasserstein metric of parameter p 1 is a mean kernel metric.

Proof of Theorem 5.9 (Sufficient condition for power-Wasserstein to be mean kernel). Let us 1. We denote f 0 (x) = p x-1

x p -1 > 0 and g 0 (x) = x p f 0 (x) 2 so that F (x) = f 0 (x) 2 +g 0 (x). Note that f 0 is non-decreasing if and only if p 1. We also introduce h 0 (x) = (x p + 1)(x -1) 2 so that F (x) = p 2 h 0 (x) (x p -1) 2 .

1. If p -1 > 0, then F (0) = -2p 2 < 0 so F cannot be positive around 0 because it is smooth at 0. So F is not non-decreasing.

2. If p ∈]0, 1], then F is non-decreasing as product of three non-decreasing positive functions.

3. We assume that p < 0. Let us prove separately that F is increasing on (0, 1) and on (1, ∞). As F is continuous (at 1), it will prove that F is increasing on (0, ∞).

(a) Let us prove that F is increasing on (0, 1). We only need to prove that g 0 is increasing on (0, 1). We successively derive:

where

Id. The same is valid for v. Therefore, since the parallel transport is O(n)-equivariant, the balanced bilinear form g 0 is defined by:

First, g 0 is symmetric. Second, since u : (0, ∞) -→ R is a diffeomorphism, either u > 0 or u < 0 and by the mean value theorem, the sign of u [1] is the sign of u . Hence

> 0 so the coefficients of the quadratic form g 0 Σ (X, X) are positive. So the balanced bilinear form g 0 is a Riemannian metric.

Proof of Theorem 5.25

Theorem 5.25 ,w) ) is an isometry. For Σ = P DP ∈ Sym + (n), we denote X = P X P ∈ T Σ Sym + (n) and analogously for Y, Z, T ∈

Then the curvature of the mixed-Euclidean metric g ME(u,v) is:

where

where A stands for the affine-invariant metric (Formula 5.8). Therefore, the sectional curvature of the mixed-Euclidean metric at I n takes non-positive values. In particular, for mixedpower-Euclidean metrics MPE(α, β) with α 2 = β 2 (thus excluding log-Euclidean, power-Euclidean and power-affine metrics), since κ

(X, Y ) for all λ > 0, the lower bound of the sectional curvature is -∞.

Proof of Theorem 5.25 (Curvature of Mixed-Euclidean metrics). We compute the curvature of the metric g ME(u,v) for univariate diffeomorphisms u, v : Sym

To get the curvature of g ME(u,v) , we scale this formula by c = w (1) u (1)v ( 1) and we pull it back via the map u. The coefficients w ij = w [1] (d i , d j ) and w ijk = w [2]

where

we can write:

Since the two expressions of ρ ijkl are continuous in (d i , d j , d k , d l ), they also coincide when

Part VII. Appendix

The curvature at Σ = D = I n follows from the following computations:

Hence, according to Formula (11.22), the curvature at I n writes: (11.23) because the second and fourth terms cancel. Recognizing the curvature of the affine-invariant [Skovgaard, 1984, Pennec et al., 2020[START_REF] Thanwerdas | O(n)-invariant Riemannian metrics on SPD matrices[END_REF], we can finally write:

Second, we assume that α = 0 and β = 0. Similarly, the Levi-Civita connection is

This expression cancels Equation (11.25) with β = 0 so the curve γ is the geodesic.

Proofs of Chapter 6

In this section, we prove the results of Chapter 6: Geometry of Quotient-affine metrics.

11.5.1 Proof of Theorem 6.2 Theorem 6.2 (Vertical and horizontal distributions and projections) The vertical distribution is given by V Σ = Σ•ψ(R n ) and the horizontal distribution is given by

The vertical projection is:

Then, the horizontal projection is simply hor(V ) = V -ver(V ).

Proof of Theorem 6.2 (Vertical and horizontal distributions and projections). We denote ∆ Σ = Diag(Σ) 1/2 . Using Section 6.2.1, we have:

Thus, we have computed the horizontal space for the affine-invariant metric α = 1 and β = 0. It is still valid for all α > 0 and β > -α/n since the latter is included in the former (because tr(Σ -1 W ) = 0) and they have the same dimension so they are equal. Now we compute the vertical projection. Let

Proof of Lemma 7.8 (Operator ad * for Lie-Cholesky metrics). In the matrix Lie group LT(n), the Lie bracket is the commutator [X, Z] = XZ -ZX. Therefore:

11.6.5 Proof of Lemma 7.10

Lemma 7.10 (Vertical distribution, horizontal distribution) The vertical and horizontal distributions associated to the quotient-Lie-Cholesky associated to f are

Proof of Lemma 7.10 (Vertical distribution, horizontal distribution). The vertical space is the tangent space of the fiber Diag + (n)L or the kernel of the differential of the submersion

. Indeed, it is the pushforward by the Cholesky map of the horizontal space on SPD matrices (cf. Section 7.2.2). In other words, for all Z ∈ H AI L , for all Z ∈ V L , we have L -1 Z|L -1 Z = 0. Therefore, the horizontal space of the metric LC(f ) is:

In particular, quotient-Lie-Cholesky metrics are geodesically complete. [START_REF] Archakov | A New Parametrization of Correlation Matrices[END_REF]] that for all D ∈ Diag(n), the linear map d D ϕ S : Diag(n) -→ Diag(n) has its eigenvalues in [0, 1) ⊂ R. For all (S, D) ∈ Φ -1 (0) and all ∆ ∈ Diag(n), we have:

Therefore, the linear map d D Φ S = Id -d D ϕ S has its eigenvalues in (0, 1] so it is invertible. Hence, the implicit function

Proof of Theorem 8.13

Theorem 8.13 (Existence and equivalence of conjectures) We define the smooth map f : ∆ ∈ Diag + (n) -→ d AI (I n , ∆C∆) 2 = tr(log(∆C∆) 2 ). It gives the affine-invariant squared distance 11.7.4 Proof of Theorem 8.17

Theorem 8.17 (Properties of the log-scaling bijection) The log-scaling bijection satisfies the following properties.

1. (Equivariance) Log and Exp are equivariant under permutations.

2. (Equicorrelation) For all ρ ∈ (-

is the Fisher transformation of the correlation coefficient ρ ∈ (-1, 1).

(Block equicorrelation matrix) If

C is a block equicorrelation matrix of signature I = {i 1 , ..., i p }, then Log (C) is a block symmetric matrix with null row sum of signature I with diagonal blocks of the form (α j -β j )I i j + β j 1 i j 1 i j and off-diagonal blocks of the form β jk 1 i j 1 i k .

4. (Generalization) For all x ∈ (R + ) n , for all Σ ∈ Sym + (n), there exists a unique ∆ ∈ Diag + (n) such that log(∆Σ∆)x = 0.

Proof of Theorem 8.17 (Properties of the log-scaling bijection). 1. This is clear.

Let C = C(ρ).

The result is clear for ρ = 0 so we assume that ρ = 0. One easily checks that ∆ = √ aI n with a = 1 1+(n-1)ρ satisfies Σ :

3. If C is a block equicorrelation matrix of signature I = {i 1 , ..., i p }, it is clear that the matrix ∆ is a block diagonal matrix of signature I with scalar blocks because the sums of all the rows belonging to the same interval [i j +1; i j+1 ] are equal. The matrix product preserves the signature and the form of the blocks so the logarithm as well.

4. Similarly to Theorem 8.14, one can prove that given x ∈ (R + ) n , the map exp : Theorem 8.20 (Permutation-invariant inner products on Row 0 (n)) For n 4, permutationinvariant inner products on Row 0 (n) are the symmetric bilinear forms associated to the following positive definite quadratic forms q defined for Y ∈ Row 0 (n): (11.38) with α > 0, nα + (n -2)δ > 0 and nα + (n -1)(δ + nζ) > 0. For n = 3, the permutationinvariant inner products have the same form with α = 0. For n = 2, they have the same form with α = δ = 0.

Proof of Theorem 8.20 (Permutation-invariant inner products on Row 0 (n)). We rely on the characterization of permutation-invariant inner products on Hol(n) and on the equivariant isomorphism Θ : [START_REF] Kurata | Moore-Penrose inverse of a hollow symmetric matrix and a predistance matrix[END_REF], whose inverse isomorphism is given by

2 Diag(Y )1. Indeed, let q : Row 0 (n) -→ R be a permutation-invariant quadratic form. Then q is positive definite if and only if q • Θ : Hol(n) -→ R is a permutation-invariant quadratic form on Hol(n). Hence, by Theorem 8.7, q is of the form q

We compute X 2 with X = Θ -1 (Y ), and the three terms:

, which gives the expected positivity condition.

Proof of Theorem 8.23

Theorem 8.23 (Coincidence of the metrics in dimension 2) In dimension 2, up to a positive scaling factor, the quotient-affine metric, the off-log metric and the log-scaled metric coincide. We recall that the Fisher transformation is the increasing map

(1-ρ 2 ) 2 (up to a scaling factor α > 0), defined by the following equations:

Let us apply the implicit function theorem to the smooth map G : (a,

where u i = z i -2y i . So by the implicit function theorem, y : R + -→ R n is smooth. We introduce an intermediate map b : R + -→ R + defined by z i (a) = x i -a

x i = y i (a) -b(a) y i (a) . It does not depend on i since y -z 1 y . It is smooth since it writes b = y i (y i -z i ).

3. (Bures-Wasserstein metric) g

Proof of Theorem 9.25 (Horizontal lift,tangent space,metric). 1&2. We prove the expression of the tangent space and the horizontal lift together. Let V ∈ T Σ Sym + (n, k).

The horizontal lift is defined by:

When we plug the second equality in the first one and we multiply by X on the left and X on the right, since X X ⊥ = 0, we get immediately X V X = X XF + F X X so F = S X X (X V X). By multiplying by X ⊥ on the left instead, we get

We compute d X π(V # ) to check that it is equal to V :

3. The quotient metric is defined by g

) so we only need to compute V # (V # ) for any X, for example X = U D 1/2 , and its trace.

). If E Σ,V is non-empty, then let λ + = max E Σ,V and λ -= min E Σ,V . The definition interval of the geodesic γ (Σ,V ) is the interval I Σ,V defined by:

3. (Cut time) Let λ max = max sp(F 0 X,V ) and λ min = min sp(F 0 X,V ). Note that if E Σ,V = ∅, then we have (λ -, λ + ) ⊆ (λ min , λ max ). The cut time is t cut (Σ, V ) = -1 λ min if λ min < 0 or +∞ otherwise. Symmetrically, we have t cut (Σ, -V ) = 1 λmax if λ max > 0 or +∞ otherwise.

(Preimages) We define the indexing set I Pre

X,Y by:

(a) If r = k, then there exists a unique logarithm of Λ from Σ. In this case, the minimizing geodesic joining Σ to Λ writes:

, then there exist exactly two logarithms of Λ from Σ.

(c) If r < k -1, then there is an infinity of logarithms of Λ from Σ.

Therefore, the logarithm map is defined on

Proof of Theorem 9.26 (Bures-Wasserstein geodesics on Sym + (n, k)). We prove statement 3 in the end because it requires statements 4 and 5.

1. (Exponential map) The exponential map is simply Exp

was already computed in the proof of Theorem 9.25.

(Definition domain) As in Sym

According to [Massart and Absil, 2020, Proposition 3.2] applied to tV #

Its computation is analogous to the one in the proof of Theorem 9.22.

To get a condition on the kernels of symmetric matrices in E Σ,V , it suffices to note that the condition rewrites ker(λI

so the condition does not depend on X.

(Preimages)

In [Massart and Absil, 2020, Propositions 4.4 & 4.5], the solutions of the equation Exp Σ (V ) = Λ are computed in a wider set than the definition domain D Σ of the exponential map because they did not need more to characterize the logarithms. They are actually computed in the set

However, the geodesic γ (Σ,V ) may leave the manifold Sym + (n, k) before reaching Λ. Therefore, we complete their work with the additional condition 1 ∈ I Σ,V to characterize the preimages of Λ from Σ.

From [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF], we know that preimages V necessarily satisfy

We can now compute E Σ,V and I Σ,V . For all λ ∈ R, for all Z ∈ R n :

Therefore:

Thus, denoting λ -= min E Σ,V , the condition 1 ∈ I Σ,V rewrites:

To conclude, with the notations of statement 4, V = 2 sym(XRY ) -2Σ ∈ Pre Σ (Λ) if and only if R ∈ I Pre X,Y . 5. (Logarithms) In [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF], it is stated that the shortest vectors

0. Therefore, we necessarily have H = (X ΛX) 1/2 . From Definition 9.19, they even satisfy V = V # = d BW (Σ, Λ). Moreover, if H 0, the condition 1 ∈ I Σ,V is automatically satisfied, as stated in [Massart and Absil, 2020, Corollary 3.3 (5)]. So with the notations of statement 5, the logarithms are indexed by I Log X,Y . 6. (Logarithm map) It is clear that rk(ΣΛ) rk(X Y ) since ΣΛ = X(X Y )Y . We also have X Y = (X X) -1 X (ΣΛ)Y (Y Y ) -1 so rk(X Y ) rk(ΣΛ). Finally, rk(ΣΛ) = rk(X Y ) = rk(H). We denote it r = rk(ΣΛ).

(a) As stated in [START_REF] Massart | Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices[END_REF], if r = k, then there exists a unique logarithm of Σ from Λ. Moreover, we can compute an explicit expression. Indeed, R = H -1 X Y so XRY = XH -1 X Λ = X(X ΛX) -1/2 X Λ. Since the choice of X is free, let us take X = U D 1/2 where Σ = U DU with U ∈ St(n, k) and D ∈ Diag + (k). Therefore:

So the unique minimizing geodesic joining Σ to Λ writes: γ (Σ,Λ) (t) = (1 -t) 2 Σ + t 2 Λ + 2t(1 -t)sym(Σ 1/2 ((Σ 1/2 ΛΣ 1/2 ) 1/2 ) † Σ 1/2 Λ).

Besides, X X + tX V # = (X X) 1/2 (I n + tS 0 X,V )(X X) 1/2 . Let λ min = min sp(S 0 X,V ) = min sp(S Σ,V ). Therefore, γ (Σ,V ) is minimizing on [0, t] if and only if H 0 if and only if I n + tS 0 X,V 0 if and only if 1 + tλ min 0. If λ min 0, the condition is empty so t cut (Σ, V ) = +∞. If λ min < 0, the condition writes t -1 λ min so t cut (Σ, V ) = -1 λ min .

Moreover, H X,Y,R = (X ΛX) 1/2 and the minimizing geodesic γ R Σ→Λ is of constant rank p max(k, l) on (0, 1).

Proof of ). (Necessity) Let γ : [0, 1] -→ Cov(n) be a minimizing geodesic segment from Σ = γ(0) to Λ = γ(1). Let p = max t∈[0,1] rk(γ(t)). By Lemma 9.30, γ is of constant rank p max(k, l) on (0, 1). In other words, γ |(0,1) : (0, 1) -→ Sym + (n, p) is a minimizing geodesic of Sym + (n, p). Let c 0 : (0, 1) -→ R n×p * be a horizontal lift of γ |(0,1) . Necessarily, c 0 (t) = (1 -t)X 0 + tY 0 with X 0 , Y 0 ∈ R n×p with X 0 X 0 = Σ and Y 0 Y 0 = Λ since (X 0 , Σ) = lim t→0 (c 0 (t), γ(t)) and (Y 0 , Λ) = lim t→1 (c 0 (t), γ(t)). Let us show that X 0 Y 0 ∈ Cov(p).

For all [a, b] ⊂ (0, 1), the tangent vectors