3.6.3 Other extensions 44 3.7 Additional proofs 45 3.7.1 Proof of Lemma 1 45 3.7.2 Proof of Theorem 5 48 3.7.3 Proof of Theorem 6 60 3.7.4 Proof of technical lemmas 61 4 clustering multilayer graphs with missing nodes 71 4.1 Introduction 71 4.1.1 Related work 72 4.1.2 Contributions 73 4.2 Problem setup 73 4.2.1 Multilayer Stochastic Block Model (MLSBM) 74 4.2.2 Missing nodes 74 4.3 Final aggregation methods 74 4.3.1 A method based on a variant of k-means for incomplete data 75 4.4 Early fusion methods: spectral clustering on sum of adjacency matrices 80 4.4.1 Imputing missing entries with zeros 81 4.4.2 Iteratively imputing the missing entries 84 4.5 Intermediate fusion methods: OLMF estimator 85 4.5.1 The complete data setting 86 4.5.2 Extension to the missing nodes setting 86 4.6 Numerical experiments 87 4.6.1 Synthetic data 87 4.6.2 MIT Reality Mining dataset 88 4.6.3 Malaria parasite genes network 90 4.7 Possible extensions and concluding remarks 90 4.7.1 Extension of IR-LS to the multilayer setting 90 4.7.2 Other research direction 92 4.8 Additional proofs 93 4.8.1 Auxiliary Lemmas 93 4.8.2 Comparison between misclustering bound under MLSBM in the complete setting 95 5 minimax optimal clustering of high-dimensional bipartite graphs 97 5.1 Introduction 97 5.1.1 Main contributions 98 5.1.2 Related work 99 contents xi 5.2 The statistical framework 100 5.3 Algorithm 103 5.3.1 Initialization with a spectral method 103 5.3.2 Iterative refinement with GPM 103 5.4 Spectral initialization 104 5.5 Analysis of GPM 106 5.5.1 Error decomposition 106 5.5.2 Oracle error 107 5.5.3 Contraction of the error at each step 108 5.5.4 Application to BiSBM 109 5.6 Minimax lower bound 112 5.7 Numerical experiments 119 5.8 Additional proofs 122 5.8.1 Concentration of the oracle error 122 5.8.2 Proof of Lemma 18 124 ii graph matching 127 6 introduction to graph matching 129 6.1 Information theoretic limits of matching 129 6.1.1 Correlated Wigner Model (CoWM) 130 6.1.2 Correlated Erdös-Renyi Model (CoERM) 130 6.2 Matching methods 132 6.2.1 Convex Quadratic Programming (CQP) relaxation 132 6.2.2 SDP relaxation 132 6.2.3 Spectral methods 133 6.2.4 Message passing algorithm 134 6.2.5 Signature matching 135 7 seeded graph matching with the generalized power method 137 I N T R O D U C T I O N T O

A B S T R A C T

Graph-structured datasets arise naturally in many fields including biology with protein-to-protein interaction networks, ecology with predator-prey networks, and economy with financial market networks. In order to extract relevant information from these networks, one often uses clustering methods to gather nodes having similar connectivity patterns into communities. Numerous clustering algorithms have been proposed in the past decade and have been analyzed under the Stochastic Block Model (SBM), a popular random graph model. However, in practice, one often has access to side information, and it is typically unclear how this information can be incorporated into existing methods and to what extent it can help to improve the clustering performance.

We will first address this question under the Contextual Stochastic Block Model (CSBM) -a simple extension of the SBM with independent Gaussian covariates associated with each node -and propose an iterative refinement method that is fast and achieves the informationtheoretic threshold for exact recovery. Our method is inspired by the Generalized Power Method (GPM) and principles of Expectation Maximization (EM) type algorithms.

Next, we extend the method to be applied to networks with heterogeneous degrees or mixed-membership, but also with different kinds of covariates like multilayer graphs with possibly missing values, or high dimensional bipartite graphs, hence showing the flexibility of the approach.

Lastly, we consider the graph matching problem where the additional graphs can be considered as correlated side information. We show that we can also use the GPM for this problem to significantly improve the matching obtained by state-of-the-art methods, and provide consistency guarantees for the GPM under the Correlated Wigner Model (CoWM).

v 野を横に馬牽むけよほととぎす -松尾芭蕉 R E M E R C I E M E N T S
Par ordre chronologique, il me faut d'abord remercier Matsuo Basho, le célèbre poète de la grenouille qui plonge, sans lequel je n'aurais sans doute pas entrepris cette thèse. C'est en déchiffrant l'un de ses haïkus gravé sur une pierre recouverte de mousses évoquant un cavalier excité par le cri d'un coucou que j'ai réalisé que la poésie me serait toujours plus étrangère que les bornes minimax.

Il me faut ensuite remercier les personnes qui m'ont mené vers l'équipe Modal grâce à leurs conseils, encouragements et contacts : Philippe, David et Pascal.

Bien évidemment, ce travail n'aurait pas vu le jour si Christophe ne m'avait pas chaleureusement accueilli et proposé un sujet de recherche en collaboration avec Hemant. Je leur suis particulièrement redevable du temps et de l'énergie qu'ils ont consacré à me former. En particulier à Hemant qui a fait preuve d'une rare patience à relire et corriger mes preuves.

Je tiens également à exprimer ma gratitude aux rapporteurs, Catherine Matias et Marc Lelarge qui m'ont fait l'intimidant honneur de se pencher sur ma thèse. Je suis également reconnaissant à Olga Klopp et Alexandre d'Aspremont d'avoir très gentiment accepté de participer au jury.

Si j'ai pu mener ce travail à son terme c'est également grâce au personnel administratif de l'Inria (Anne et Charlotte en particulier) ainsi que quelques anciens collègues de l'Insee qui m'ont aidé dans les procédures administratives. Enfin, je dois également saluer tous les membres de l'équipe Modal qui ont supporté mes jérémiades dans la bonne humeur pendant trois ans. En particulier à Cristian qui a accepté le rôle de souffre douleur aux échecs et m'a été d'un soutien précieux durant ma thèse.

Il me reste encore quelques lignes pour remercier, même s'ils ne liront vraissemblablement jamais ce manuscrit, ma famille, mes amis plus ou moins lointains, et Agathe, l'araignée vivant derrière le réfrigérateur, pour leurs présences réconfortantes. 0) , such that overlap(x (0) , x * ) = 0.08. 155

Performance of PPMGM with different initializations. Here in.1,in.2,in.3,in.4 corresponds to and overlap of x (0) with the ground truth of 0.05, 0.1, 0.15 and 0.5 respectively. 156 Figure 23 PPMGM with an initialization such that overlap(x (0) , x * ) = 0.1. Here it.1, it.2, it.3, it.4, it.5 corresponds to 1, 2, 4, 8 and 30 iterations respectively. 157 Figure 24 Here it.1, it.2, it.3, it.4 corresponds to 1, 2, 4 and 8 iterations respectively. 157 Figure 25 Initial overlap is equal to 0.5 158 Figure 26 Initial overlap is equal to 0.1 159 Figure 27 Heatmap for the recovery rate of PPMGM algorithm with input (A spar 1 , B spar 1 ) for different threshold values τ i (y axis); i = 1, • • • , 6, and different values of σ (x axis). Here τ i corresponds to the solution of (7.5.2) with n = 1500 and p i for i = 1, 2 • • • , 6 in a uniform grid between p 1 = 42 × 10 -3 and p 6 = 54 × 10 -3 . 160 • The all-ones vector of size n is denoted by 1 n .

L I S T O F TA B L E S
• The indicator function of an event A is denoted by 1 A .

• The set of permutation over n elements is denoted by S n and the corresponding set of matrices is denoted by P n

• The empty set is denoted by ∅.

• The column j of A will be denoted by A :j , and the (i, j)th entry by A ij . The i-th row of a matrix A will be denoted as A i: and depending on the context can be interpreted as a column vector.

• The transpose of A is denoted by A ⊤ and A ⊤ :j corresponds to the jth row of A ⊤ by convention.

• I k denotes the k × k identity matrix.

• The number of non zero entries of a matrix A is denoted by nnz(A).

• For matrices, we use ||.|| and ||.|| F to respectively denote the spectral norm (or Euclidean norm in case of vectors) and Frobenius norm.

• The trace of a matrix A is denoted by Tr(A).

• We use lowercase letters (ϵ, a, b, . . .) to denote scalars and vectors, except for universal constants that will be denoted by c 1 , c 2 , . . . for lower bounds and C 1 , C 2 , . . . for upper bounds. We use uppercase letters to denote matrices and random variables (with possible local exceptions). 1

I N T R O D U C T I O N
Identifying and recognizing patterns is a natural task for the human brain. It helps to extract relevant information from one's environment and discover general laws in nature. Hence, clustering -the act of gathering objects by similarities -is a fundamental activity that lies at the root of knowledge elaboration. This is probably why one spends so much time trying to organize one's library by sorting one's books by topic, color, shape, etc.

However, even if the ability of human brain to categorize and organize information is remarkable, it is not capable of processing large amounts of data and can fail to detect some patterns. This is a reason why one has tried to automate this procedure and started to develop clustering algorithms. The first systematic numerical approaches were developed for applications in social sciences [START_REF] Czekanowski | Zur Differentialdiagnose der Neandertalgruppe[END_REF][START_REF] Zubin | A technique for measuring like-mindedness[END_REF]. More historical references can be found in the introduction of the book of [START_REF] Bouveyron | Model-Based Clustering and Classification for Data Science: With Applications in R[END_REF]. Most of the initially proposed methods were based on the definition of a distance between the objects of interest. Then, for a given distance, one can use optimization methods to obtain a partition of the objects such that the objects belonging to the same class have a small distance between them and a large distance with objects from another class. It is exemplified by the k-means algorithm, one of the most popular clustering methods.

The previous heuristic approach has several drawbacks. The choice of the distance is arbitrary and it is impossible to assess the quality of the resulting clustering. Also, methods that usually work in practice can fail in particular instances. This is for example the case of the spectral method applied on the cockroach graph [START_REF] Luxburg | A Tutorial on Spectral Clustering[END_REF].

A way to overcome these issues is to introduce a probabilistic model for the data and formulate the clustering problem as a statistical inference problem.

a statistical approach of clustering

The first generative model proposed for clustering numerical vector data (X i ) n i=1 ⊂ R d of dimension d is the Gaussian Mixture Model (GMM). Each vector X i is assigned to a community k ∈ [K] and then generated from a Gaussian probability distribution than only depends on k, independently of the other vectors. More formally, for each i, assign to i a label z i ∈ [K] and generate X i by

X i = µ z i + ϵ i , where ϵ i ind. ∼ N(0, Σ z i )
where for all k ⩽ K, µ k ∈ R d and Σ k ∈ R d×d is a positive definite matrix. Clustering (X i ) n i=1 is now equivalent to estimate the latent partition given by z : [n] → [K] that associates i to z i .

A natural approach is to maximize the likelihood associated with the problem. For example, under the isotropic case where all Σ k = Id, one can easily check that the (complete) Maximum Likelihood Estimator (MLE) associated to the GMM is related to the one provided by the k-means optimization problems arg min

µ 1 ,...,µ K ∈R d ,C K k=1 i∈C k ∥X i -µ k ∥ 2 (1.1.1)
where C is a partition of [n] into K sets C 1 , . . . , C K . Unfortunately, solving (1.1.1) requires doing an exhaustive search over all the possible K n partitions and cannot be done in a reasonable time (i.e. polynomial in n). Even if k is fixed to 2, k-means clustering is a NP-hard problem as shown in [START_REF] Dasgupta | The Hardness of K-means Clustering[END_REF] So in practice, one has to use other strategies. For example, there are approximate k-means algorithms, see for example [START_REF] Kumar | A Simple Linear Time (1+ϵ)-Approximation Algorithm for k-Means Clustering in Any Dimensions[END_REF], that run in polynomial time in n and produce an output with an estimation error proportional to the error associated with the optimal solution. In the following section, we will describe an alternative strategy, particularly useful when one can obtain a first rough estimate of the latent partition.

alternating optimization

A popular approach to solve (1.1.1) is to use Lloyd's algorithm: starting from a first estimate C (0) = {C

µ (0) k = 1 |C (0) k | i∈C (0) k X i
and then update the partition to obtain C (1) by assigning each X i to the closest centroid. One can check that this is equivalent to solve (1.1.1) by alternatively fixing C or µ.

This approach for solving nonconvex optimization problems is referred to as alternating optimization. It has been used with success in discrete optimization problems including approximate ranking, group synchronization, and multireference alignment (see [START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF] as well as for continuous optimization problems, e.g. matrix completion, phase retrieval [START_REF] Chi | Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview[END_REF] or Principal Component Analysis (PCA) with missing data [START_REF] Zhu | Highdimensional principal component analysis with heterogeneous missingness[END_REF]. This principle is also at the core of model-based approaches: EM-like algorithms [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF] use alternating optimization to maximize a proxy of the likelihood, the Evidence Lower Bound (ELBO).

One of the drawbacks of this approach is that it requires an initial estimate of the partition. But one can often use spectral methods to obtain a cheap initial estimate. One can also rely on random sampling strategies. This is a very common approach in the model-based literature and it can be proved to work under some assumptions, see for example Wu et al., 2022. 1.3

network structured data

The past two decades have witnessed the increase of network structured data with the rise of social networks. Contrary to the previously discussed vector data setting where each measurement X i is independent of the other, networks encode pairwise interactions between the agents of interest represented by nodes, and hence there are structural dependencies among the observations. This is a fundamental difference. But before discussing its algorithmic implications, let us first introduce a generative model to give a statistical meaning to the clustering task.

The most popular random graph model that incorporates a community structure is the SBM. It was introduced by [START_REF] Holland | Stochastic blockmodels: First steps[END_REF] and is often used as a benchmark to evaluate clustering algorithms on graphs. It is defined by the following parameters.

• The set of nodes N = [n].

• Communities C 1 , . . . , C K , of respective sizes n 1 , . . . , n K , forming a partition of N.

• A membership matrix Z ∈ M n,K where M n,K denotes the class of membership matrices. Here, Z ik = 1 if node i belongs to C k , and is 0 otherwise. Each membership matrix Z can be associated bijectively with a function z :

[n] → [K] such that z(i) = z i = k
where k is the unique column index satisfying Z ik = 1.

• A symmetric, connectivity matrix of probabilities between communities

Π = (Π kk ′ ) k,k ′ ∈[K] ∈ [0, 1] K×K .
Denoting by P = (p ij ) i,j∈[n] := ZΠZ T , a graph G is distributed according to SBM(Z, Π) if the entries of the corresponding symmetric adjacency matrix A are generated by

A ij ind. ∼ B(p ij ), 1 ⩽ i ⩽ j ⩽ n,
where B(p) denotes a Bernoulli distribution with parameter p. Hence the probability that two nodes are connected depends only on their respective community membership. Here we allowed self-loop for analysis convenience, but usually the diagonal of the matrix A is removed.

Usually observed networks are sparse -most of the entries are zeros -and the sparsity level of a SBM is measured by p max = max i,j p ij .

A direct application of Lloyd algorithm to this setting will require to estimate the centroids µ k := Π k: Z ⊤ by averaging the rows of the adjacency matrix A that belong to the same community. By doing that we completely ignore the fact that the columns of A have a similar block structure. Typically, we would expect that such an approach won't be robust to noise since it doesn't take into account the lowrank structure of the problem.

An alternative is to solve the following optimization problem that takes into account the row and columns block structure: arg min

Z∈M n,K ,Π∈[0,1] K×K A -ZΠZ ⊤ 2 . (1.3.1)
Unfortunately, this is again an hard combinatorial optimization problem. But we can try an alternating optimization strategy. If the partition Z were known, the solution Π of (1.3.1) will be given by Z ⊤ AZ.

But for a fixed Π, (1.3.1) is still difficult to solve due to the quadratic nature of the objective function in Z. We will show in Chapter 3 that it can be done by using a low-rank embedding of A computed from an initial estimate of the latent partition. Another possible approach would be to relax the condition Z ∈ M n,K to Z ∈ O n,K where

O n,K = {M ∈ R n×K such that M ⊤ M = I K }
is the set of matrices with orthonormal columns and then use continuous optimization tools such as gradient descent. This approach is a special case of the Orthogonal Linked Matrix Factorization (OLMF) method developed for multilayer graphs or multiview clustering, see e.g. [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF][START_REF] Dong | Clustering on Multi-Layer Graphs via Subspace Analysis on Grassmann Manifolds[END_REF]. We will discuss in more detail the existing method for clustering graphs in Chapter 2.

One can also wonder why one should use a least square criterion instead of the likelihood as an objective function. We will show in Chapter 3 that using a least square proxy of the likelihood doesn't induce any loss of accuracy -under some parameter regimes -compared to the solution obtained by solving the MLE using a similar alternating optimization strategy. Moreover, it leads in practice to a significant gain in term of computational time. Popular model-based approaches rely on variational EM algorithm [START_REF] Celisse | Consistency of maximum-likelihood and variational estimators in the Stochastic Block Model[END_REF] and stochastic variant [START_REF] Brault | Estimation and Selection for the Latent Block Model on Categorical Data[END_REF].

side information

In practice, real-world networks often come with side information in the form of nodes or edges covariates. For example, when analyzing interactions among people on a social network, we have access to additional features such as gender, age, or ethnicity that can be relevant for the clustering task. Whereas several algorithms have been proposed to incorporate nodes or edges side information, little is known about the value of this side information to improve the clustering performance. The question has not only a theoretical interest but also practical implications on the design of algorithms that should incorporate optimally the available side information.

In this dissertation we will consider several kinds of side information

• nodes side information provided by a GMM;

• edges side information that can be represented by a multilayer graph;

• edges side information that comes from correlated but unlabeled graphs.

We will further discuss the last setting in the next section.

graph matching

In many applications, one observes several correlated realizations of a graph. For example, in computer vision, one can observe different images of an object -than can be modeled by a graph -in different positions. In order to connect these observations it is necessary to align the graphs, that is to say, find the permutations that maximize the similarity between each pair of graphs. When there are only two graphs, this optimization problem can be formulated as follows. Let A and B two adjacency matrices associated with graphs G and H. The permutation that best aligns G and H is given by the solution of max

x∈S n i,j A ij B x(i)x(j) (1.5.1)
where S n denotes the set of permutations over [n] or equivalently min

X∈S n A -XBX ⊤ 2 F . (1.5.2)
This is a hard combinatorial optimization problem known as the Quadratic Assignment Problem (QAP). Indeed the problem is known to be NP-hard [START_REF] Sahni | P-Complete Approximation Problems[END_REF]. Fortunately, under some generative models it can be solved exactly. The most studied models for correlated random graphs are the following.

correlated wigner model CoW M(n, σ, x * ). The following Gaussian model has been proposed in [START_REF] Ding | Efficient random graph matching via degree profiles[END_REF].

A ij ∼    N(0, 1 n ) if i ̸ = j, N(0, 2 n ) if i = j, and B x * (i)x * (j) = √ 1 -σ 2 A ij + σZ ij , where Z d = A.
Both A and B are distributed as the Gaussian Orthogonal Ensemble (GOE). Here the parameter σ > 0 should be interpreted as the noise parameter and in that sense, B can be regarded as a "noisy perturbation" of A. Moreover, x * ∈ S n is the ground-truth (or latent) permutation that we seek to recover. It is not difficult to verify that the problem (1.5.1) is in fact the MLE of x * under the CoWM.

correlated erdös-renyi model CoERM(n, q, s, x * ). For q, s ∈ [0, 1], the correlated Erdos-Renyi model with latent permutation x * ∈ S n can be described in two steps.

1.

A is generated according to the Erdös-Renyi model ERM(n, q): for all i, j ∈ [n] such that i < j, A ij ind.

∼ B(q).

2. Conditional on A, the entries of B are i.i.d according to the law

B x * (i),x * (j) ∼    B(s) if A ij = 1, B q 1-q (1 -s) if A ij = 0.
(1.5.3)

The graph matching problem (1.5.2) share a lot of similarities with the clustering problem (1.3.1). The main difference is that the latter involves a latent low-dimension structure since usually K ≪ n. But if K = n and the matrix Π were given, the two problems would be equivalent. So it is not surprising that there are strong links between methods used for addressing each of these problems. However, graph matching is generally harder. It is usually difficult to find a good enough initial matching. This can be explained intuitively by the absence of a latent low-dimension structure. We will discuss the information-theoretic limits and techniques used for graph matching in more detail in Chapter 6.

thesis contribution and outline

In this dissertation, we investigate the value of side information in clustering and matching problems on graphs from an informationtheoretic perspective but also from a practical aspect by developing efficient algorithms. The first part of this thesis is devoted to clustering problems where independent side information is available in the form of nodes or edges covariates.

In Chapter 2, we provide an overview of the relevant literature on graph clustering-with a focus on theoretical results -and present the principal methods: spectral and Semi-Definite Programming (SDP) relaxations, variational methods, and approximate message passing algorithms.

In Chapter 3 we study a simple model -the CSBM -where each node of a graph generated from a SBM is associated with Gaussian covariates generated from an GMM, independent of the graph but sharing the same latent partition. We propose an algorithm that iteratively estimates the model parameters and refines the partition based on a least square criterion. We show that our algorithm achieves the threshold for exact recovery under a symmetric version of the CSBM, and that up to a constant factor, its rate of convergence is minimax optimal in the general setting. The technical tools introduced in this chapter will be further used in the following chapters.

In Chapter 4, we consider multilayer graphs where each layer describes a different modality of the pairwise interactions between the agents of interest. Multilayer graphs can be thought of as a way to represent networks with edges side information. We further assume that each node doesn't appear on each layer and propose a generative model for this setting based on the Multi-Layer Stochastic Block Model (MLSBM). We study an extension of classical methods used for clustering multilayer graphs and derive consistency guarantees. We also explain how the method developed in Chapter 3 to this setting can be applied to the multilayer setting and provide some numerical experiments.

In Chapter 5, we study an extension of our iterative refinement method to graphs that are not generated by a SBM. First, we consider high-dimensional bipartite graphs -generated from the Bipartite Stochastic Block Model (BiSBM) -and show that our method can achieve the minimax optimal rate of convergence, up to a constant factor. Secondly, we show how to adapt the algorithm to graphs with heterogeneous degrees and overlapping communities.

The second part of the thesis is devoted to graph matching problems. In this setting, we observe two, or more, graphs that are generated from the same parent graphs but where the node labels are not known and consequently the graph cannot be directly aligned. Finding the permutation that best aligns these graphs is the goal of matching algorithms. If this permutation could be found, then we would obtain a multilayer graph. But contrary to the setting developed in Chapter 4, the layers are no longer independent but correlated. Hence, graph matching can be used as a subroutine for clustering tasks. For example, it has recently been shown [START_REF] Racz | Correlated Stochastic Block Models: Exact Graph Matching with Applications to Recovering Communities[END_REF][START_REF] Gaudio | Exact Community Recovery in Correlated Stochastic Block Models[END_REF] that the side information provided by correlated graphs can help to improve the threshold for exact recovery under the Correlated Stochastic Block Model (CoSBM). Unfortunately, the proposed algorithm to achieve this task is not implementable in polynomial time.

In Chapter 6 we further discuss the literature on graph matching and present the main methods and associated theoretical guarantees.

In Chapter 7 we analyze an iterative refinement matching method under the CoWM. Provided that one can have access to an initial estimate of the ground-truth permutation that correctly matches a suffi-ciently large proportion of nodes, one can show that this refinement method -based on the GPM -significantly improves the matching.

Finally, the appendix gathers some general results that are used in different chapters.

publications

This dissertation is in large parts based on the following publications and technical reports. The publications relevant to each chapter will be precised at the appropriate places. [START_REF] Araya | Seeded graph matching for the correlated Wigner model via the projected power method[END_REF] Part I

C L U S T E R I N G G R A P H S W I T H S I D E I N F O R M AT I O N
In the first part of this dissertation, we will investigate the adding value of side information for clustering graphs. First we will study a simple setting where each node is associated to vector covariates. Then we will show that the proposed optimal strategy to tackle this problem can be extended to other settings involving edges covariates (multilayer graphs) and also adapted to more general random graphs models incorporating degree heterogeneity and overlapping communities.

This model will be referred to as SSBM(n, K, p, q). It is often used in theoretical analysis since the simplicity of the model allows some simplification in the calculations due to symmetries. Also, the different information-theoretic thresholds are easier to interpret for this model since they only depend on the difference p -q, than for the general SBM that involves the Chernoff-Hellinger divergence. A more complete presentation can be found in Abbe, 2018.

The usual way to quantify (for theoretical purposes) the accuracy of an estimate ẑ of the latent partition is the following quantity referred to as the misclustering rate

r(ẑ, z) = 1 n min π∈S i∈[n] 1 { ẑ(i)̸ =π(z(i))} .
One can have different recovery requirements.

• Exact recovery: r(ẑ, z) = 0 with probability 1 -o(1) when n → ∞, that is to say one recovers exactly the partition w.h.p. This condition is also referred to in the literature as strong consistency.

• Almost exact recovery: r(ẑ, z) = o(1) with probability 1 -o(1) when n → ∞, that is to say the proportion of misclustered nodes goes to zero when n → ∞. This condition is also referred to in the literature as weak consistency.

• Partial recovery: r(ẑ, z) ⩾ α with probability 1 -o(1) when n → ∞, that is to say one can correctly classify at least a proportion α of the nodes w.h.p. This regime is interesting only if α > 1/K, i.e. we can do better than a random guess of the communities. This last regime is referred to as the detection regime in the literature.

For exact recovery, we need to be in a regime where p max = Θ( log n n ). The following theorem gives a more precise statement.

Theorem 1 (Abbe et al., 2015a). Under the SSBM(n, K,

a log n n , b log n n ), exact recovery is possible only when ( √ a - √ b) 2 K > 1.
Moreover, in this parameter regime there are polynomial time algorithms that efficiently recover the latent partition.

For almost exact recovery, the necessary condition is relaxed to np max → ∞.

Theorem 2 (Zhang et al., 2016a). Assume that K is a constant. Under the SSBM(n, K, p, q), a sufficient and necessary condition for almost exact recovery is n(p -q) → ∞.

On the other hand, partial recovery only requires p max = Ω( 1 n ).

Theorem 3 [START_REF] Abbe | Detection in the stochastic block model with multiple clusters: proof of the achievability conjectures, acyclic BP, and the information-computation gap[END_REF]. Under the SSBM(n, K, a n , b n ), it is possible to detect communities in polynomial time if

SNR := (a -b) 2 K(a + (K -1)b) > 1.
When K > 4 and SNR < 1, it is still possible to detect communities with a non-efficient algorithm.

Remark 1. There are several algorithms, including spectral methods and SDP relaxation that have been shown to achieve the threshold for exact recovery or lead to almost exact recovery. However, the partial recovery regime is usually more challenging. We will discuss these methods in Section 2.2.

clustering methods

Spectral methods

Spectral methods rely on the computation of the eigenvectors of a wisely chosen matrix that contains information about the community structure. Their popularity comes from the fact that they are usually fast (one can compute quickly the top K eigenvalues and associated eigenvectors of a matrix), easy to implement, and often accurate. It is also common to use an additional refinement step to the estimate provided by a spectral method.

Let us explain the method when directly applied to the adjacency matrix. In this case, the method will be referred to as the vanilla spectral method. Assume that A ∼ SSBM(n, 2, p, q) where p -q = Ω(p) and np → ∞. In expectation, A has a block structure1

E(A) =   p1 n/2 1 ⊤ n/2 q1 n/2 1 ⊤ n/2 q1 n/2 1 ⊤ n/2 p1 n/2 1 ⊤ n/2   .
It is easy to check that E(A) has two non zero eigenvalues λ 1 = n(p + q)/2 and λ 2 = n(p -q)/2 associated with eigenvectors

u 1 = 1 n and u 2 = ( 1, . . . , 1 n/2 times , -1, . . . , -1 n/2 times ) ⊤ .
Hence if there were no noise, communities could be identified by the sign of the entries of u 2 . One can show that this argument also works when there is noise. Let û2 the eigenvector associated with the second largest eigenvalue of A denoted by λ2 . By using Weyl's inequality [START_REF] Weyl | Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung)[END_REF] we have for k = 1, . . . , n

|λ k -λk | ⩽ ∥A -E(A)∥
where λk is the kth largest eigenvalue of A. By concentration inequalities, when np max ≳ log n, ∥A -E(A)∥ ≲ √ np w.h.p. This shows that the spectrum of A is close to the spectrum of E(A) because the largest eigenvalues are of magnitude np ≫ √ np in the almost exact recovery regime. Moreover Davis-Kahan theorem [START_REF] Yu | A useful variant of the Davis-Kahan theorem for statisticians[END_REF] implies that the angles θ i between the eigenvectors corresponding to the ith eigenvalue have a sinus bounded by

sin θ i ≲ ∥A -E(A)∥ min i̸ =j | λj -λi | ≲ 1 √ np .
Consequently ∥ û2 ± u 2 ∥ = o(1) and one can consistently use the sign of the entries of û2 to estimate the community membership. This idea can be generalized to general SBM, see [START_REF] Lei | Consistency of spectral clustering in stochastic block models[END_REF] This proof technique only leads to weak consistency guarantees. But one can use recent results (Abbe et al., 2020c;[START_REF] Lei | Unified ℓ 2→∞ Eigenspace Perturbation Theory for Symmetric Random Matrices[END_REF] on entrywise perturbation of eigenvectors to show that the vanilla spectral method achieves the threshold for exact recovery.

A popular alternative is to use the Laplacian L = D -A or the symmetric2 Laplacian L sym = I -D -1/2 AD -1/2 where D is a diagonal matrix defined by D ii = A i: 1 n . Spectral clustering on the Laplacian can be interpreted as a relaxation of the (normalized or not) Ratio Cut optimization problem as explained in [START_REF] Luxburg | A Tutorial on Spectral Clustering[END_REF]. Strong consistency guarantees for this algorithm have been derived in the recent work of [START_REF] Deng | Strong Consistency, Graph Laplacians, and the Stochastic Block Model[END_REF].

However, in the detection regime, the vanilla spectral method will fail because nodes with high degrees prevent the eigenvectors from concentrating on the communities. A simple way to fix this problem is to use regularization by reducing the degree of the problematic nodes of A (Le et al., 2017) (or adding ϵ1 n 1 ⊤ n for an appropriate choice of ϵ to the Laplacian). Unfortunately, this approach doesn't achieve the information-theoretic threshold for detection. But instead of using the adjacency matrix or a regularized version, on can use the non-backtracking matrix [START_REF] Bordenave | Nonbacktracking spectrum of random graphs: Community detection and nonregular Ramanujan graphs[END_REF]) -a matrix indexed by oriented edges where cycles of order two are removed -or the powered adjacency (Abbe et al., 2020a). The main idea is to incorporate more information about the neighborhood of a node at a certain depth and use the fact that Erdös-Renyi like graphs have a local tree structure.

Semi-definite programming (SDP)

A semi-definite program is an optimization program of the form min

X∈R n×n ⟨A, X⟩ s.t. X ⪰ 0 and ⟨B i , X⟩ ⩽ b i for i = 1, . . . , m where A, B 1 , . . . , B m ∈ R n×n and b i ∈ R for all i ∈ [m].
It is a convex optimization problem since the set of positive semi-definite matrices is a cone. Consequently, such optimization problems can be solved in polynomial time by using for example interior point methods ( [START_REF] Vandenberghe | Semidefinite Programming[END_REF][START_REF] Huang | Solving SDP Faster: A Robust IPM Framework and Efficient Implementation[END_REF]. But the computational cost of SDP methods is usually expensive. Recent works proposed different strategies to solve SDP optimization problem more efficiently. In particular, the Burer-Monteiro method [START_REF] Burer | A Nonlinear Programming Algorithm for Solving Semidefinite Programs via Lowrank Factorization[END_REF] has surged in popularity during the past decade.

A common strategy to solve NP-hard combinatorial problems is to relax them into a SDP optimization problem. This approach has been used for example to solve phase synchronization problem [START_REF] Gao | SDP Achieves Exact Minimax Optimality in Phase Synchronization[END_REF], clustering sub-Gaussian mixtures [START_REF] Giraud | Partial recovery bounds for clustering with the relaxed K-means[END_REF] or community detection [START_REF] Li | Convex Relaxation Methods for Community Detection[END_REF]. Let us illustrate the principle of the method for a SSBM(n, 2, p, q) where the diagonal elements are removed.

Let us define the binary variable X ij equal to 1 when i and j are in the same cluster and 0 otherwise and X = (X ij ) i,j∈[n] ∈ R n×n be the corresponding partition matrix. The set of all partition matrices is denoted by X n . It is easy to show [START_REF] Li | Convex Relaxation Methods for Community Detection[END_REF] that maximizing the likelihood of the model is equivalent to max

X∈X n i<j A ij -λ X ij (2.2.1)
where λ =

log 1-p 1-q log p(1-q) (1-p)q
and the maximum is taken over all partition matrices X. We can now relax the constraints on X and only use the following properties of X:

1. X is positive semi-definite;

2. X ij ⩾ 0 for all i, j;

3. X ii = 1 for all i.

Then the optimization problem (2.2.1) can be relaxed to max

X∈R n×n ⟨A -λJ n , X⟩ (2.2.2)
s.t. X ⪰ 0, X ii = 1 for all i and X ⩾ 0

where J n = 1 n 1 ⊤ n . Under some noise regime, one can show that the solution X of (2.2.2) is equal to X * , the ground truth partition matrix. The proof strategy is as follows. First one can show by using KKT conditions and constructing a dual certificate (it is generally difficult) that X * is indeed a solution of (2.2.2). It remains then to show that the solution is unique. One can also have weaker requirement and show that X is close to X * .

Variational Expectation Maximization (VEM)

EM algorithms are a family of iterative methods for finding a local maximum of the log-likelihood when the statistical model depends on unobservable latent variables. VEM algorithms involve an additional approximation to approximate a posterior distribution which computation is usually intractable. Let us detail the algorithm for the SBM. It was first proposed by [START_REF] Daudin | A mixture model for random graph[END_REF] Let assume that A ∼ SBM(Z, Π) and that

Z i i.i.d. ∼ Multi(α 1 , . . . , α K ) where Multi(α 1 , . . . , α K ) is a multinomial distribution with parame- ters (α k ) k∈[K] ∈ [0, 1] K such that k α k = 1.
In other words, we added a prior distribution over the partition. The coefficients α K can be interpreted as the proportion of each community. In the approximately equal size case, we will choose α 1 = . . .

= α K = 1/K. Let us denote θ = (Π, α 1 , . . . , α K ).
The complete log-likelihood of the model log L θ (A, Z) can be written as

log L θ (A, Z) = i,k Z ik log α k + i,k,k ′ j>i Z ik Z jk ′ log b(A ij , Π kk ′ ) (2.2.3) where b(A ij , Π kk ′ ) = Π A ij kk ′ (1 -Π kk ′ ) 1-A ij is the density of a Bernoulli with parameter Π kk ′ .
One can show that for every distribution

Q on Z log L θ (A) = log E Z L θ (A, Z) ⩾ E Z∼Q (log L θ (A, Z)) -E Z∼Q (log Z) J θ (A,Q)
. This lower bound of the observed likelihood is often called the evidence lower bound (ELBO). By using the identity

J θ (A, Q) = log L θ (A) -KL(Q||L θ (Z|A))
one can see that the ELBO is maximised when Q correspond to the posterior distribution L θ (Z|A). Unfortunately, due to dependencies in the model one cannot compute this posterior distribution. This is why we use a variational approximation: we will restrict the optimization problem to the probability distributions Q that can be factorized

Q(Z) = i Q i (Z i ).
This approximation is also often referred to as the mean field approximation. Now we can use alternative optimization to find the couple

(θ, Q) that maximizes J θ (A, Q).
ve-step. Assume that the model parameter θ is fixed. The distribution Q is by definition fully characterised by the the variational parameters

τ ik = P Q (Z ik = 1) for all i ∈ [n], k ∈ [K] that satisfies the constraint k τ ik = 1 for all i. It is easy to check that J θ (A, Q) = i,k τ ik (log α k -log τ ik ) + i,k,k ′ j>i τ ik τ jk ′ log b(A ij , Π kk ′ ).
By the KKT conditions, the solution τ = ( τik ) i,k that maximizes

J θ (Q, A) satisfies for all i ∈ [n] and k ∈ [K] τik ∝ α k j̸ =i k ′ b(A ij , Π kk ′ ) τjk ′ .
Consequently, τ can be estimated by solving a fixed point problem and using the constraint k τ ik = 1. m-step. When the distribution Q is fixed, it is easy to check that the parameter θ that maximizes J θ (Q, A) is given by

αk = i τ ik n Πkk ′ = i<j τ ik τ jk ′ A ij i<j τ ik τ jk ′ ∀k, k ′ ∈ [K].
The VE-step and the M-steps are repeated until convergence of the algorithm.

The consistency of the estimator provided by this VEM-algorithm has been analyzed in [START_REF] Celisse | Consistency of maximum-likelihood and variational estimators in the Stochastic Block Model[END_REF] then extended by [START_REF] Bickel | Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels[END_REF]. The analysis has been extended to more general models including SBM with missing observations [START_REF] Mariadassou | Consistency and asymptotic normality of stochastic block models estimators from sampled data[END_REF][START_REF] Gaucher | Optimality of variational inference for stochasticblock model with missing links[END_REF], the Latent Block Model [START_REF] Brault | Consistency and asymptotic normality of Latent Block Model estimators[END_REF] and dynamic SBM [START_REF] Longepierre | Consistency of the maximum likelihood and variational estimators in a dynamic stochastic block model[END_REF]. Whereas the previously mentioned work focus on estimating the model parameters, the accuracy of the method for estimating the partition has been investigated in Zhang et al., 2020a. An alternative to the mean field approximation has been proposed by [START_REF] Yin | A Theoretical Case Study of Structured Variational Inference for Community Detection[END_REF] The effect of random initialization has been studied in [START_REF] Sarkar | When random initializations help: a study of variational inference for community detection[END_REF] In practice, the VEM-algorithm can handle graphs with tens of thousands of nodes, but the running time becomes prohibitive for very large networks.

Modularity based clustering

A popular way to measure the quality of the partition of a graph into communities is to rely on the quality function Q [START_REF] Newman | Finding and evaluating community structure in networks[END_REF] defined by

Q = 1 |E| i,j (A ij -P ij )1 z i =z j
where |E| is the number of edges in A, z i ∈ [K] denotes the community i belongs to, and P ij correspond to a choice of a null model (a common choice is deg(i)deg(j)

2|E|

). In general, it is NP-hard to find the partition that minimizes Q. However, there is a popular heuristic to solve this problem: the Louvain algorithm [START_REF] Blondel | Fast unfolding of communities in large networks[END_REF]. It consists of several steps that are repeated iteratively.

Start with an initial partition

C = (C 1 , . . . , C K ).

2.

For each vertex i and k ∈ [K] define a partition C k,i where i is removed from the cluster it was previously associated with and is added to the cluster k.

3. Define Q k,i the modularity function associated with C k,i minus Q and let be k * i = arg max k Q k,i (breaking ties arbitrarily).

4.

For all vertices i such that C z i ̸ = C k * i pick a pair (i, j) at random such that C z i = C k * j and C z j = C k * i , and define a new partition C ′ by moving i to C k * i and j to C k * j .

5.

Repeat the previous steps until convergence.

This is a local search algorithm that aims at optimizing the modularity Q. In practice, the algorithm is fast and can be parallelized. Recently, the work of Cohen-Addad et al., 2020 established consistency guarantees of this algorithm under the SBM.

Methods for partial recovery

The previously described will usually fail in the very sparse regime where p max = O(1/n). Even if SDP approaches can achieve detection in some regimes, they don't reach the threshold for partial recovery [START_REF] Ricci-Tersenghi | Performance of a community detection algorithm based on semidefinite programming[END_REF]. One of the invoked reason of this failure is that these methods were developed to approximate the Maximum A Posteriori (MAP) (or minimum bisection when K = 2), a global criterion that is not suitable for this regime as explained in Abbe, 2018, Section 2.6. Instead of maximizing a global criterion, one should try to maximize the posterior distribution of each singular vertex max ẑi P( ẑi = z i |A). One can approximate this posterior distribution by using a belief propagation algorithm. A linear approximation of this algorithm is related to the nonbacktracking operator studied in [START_REF] Bordenave | Nonbacktracking spectrum of random graphs: Community detection and nonregular Ramanujan graphs[END_REF]. See also [START_REF] Abbe | Achieving the KS threshold in the general stochastic block model with linearized acyclic belief propagation[END_REF]. Instead of counting nonbacktracking walks, Abbe et al., 2020a showed that one can use spectral method on the new adjacency matrix where two nodes are connected if there is a path of a certain length connecting the two nodes in the original graph. Another simpler matrix related to the nonbacktracking operator is the Bethe-Hessian matrix [START_REF] Saade | Spectral Clustering of graphs with the Bethe Hessian[END_REF] 2.3 beyond sbm: other random graphs models

Degree Corrected SBM (DCSBM)

Under the SBM assumptions, all the nodes within the same community are exchangeable. But in real-life networks, nodes often have heterogeneous degrees. For example, in social networks, there are often hubs corresponding to very popular people. A way to integrate this property is to add a degree correction parameters θ i > 0 for each node i ∈ [n]. A graph G is distributed according to DCSBM(Z, Θ, Π) if the entries of the corresponding symmetric adjacency matrix A are generated by

A ij ind. ∼ B(θ i θ j Π z(i)z(j) ), 1 ⩽ i ⩽ j ⩽ n.
So in expectation we have E(A) = ΘZΠZ ⊤ Θ where Θ is the diagonal matrix with Θ ii = θ i . For identifiability, we can further assume i∈C k θ i = 1 for all k (if it were different from 1, then we can absorb this factor in the connectivity matrix Π). A particular case of this model is analyzed in [START_REF] Gao | Community detection in degree-corrected block models[END_REF] They proposed a feasible two-stage algorithm that achieves the threshold for exact recovery.

Mixed Membership SBM (MMSBM)

The assumption that each node belongs to only one community can be too restrictive. For example, in a citation network, it would be more realistic to assume that some articles can cover several topics. To overcome the expressive limitation of the SBM, [START_REF] Airoldi | Mixed Membership Stochastic Blockmodels[END_REF] introduced the MMSBM. Here, each node is associated with a probability distribution over the different communities. This can be interpreted geometrically. Pure nodes -the ones that are associated with only one community -correspond to the extremal points of a polytope, and the positions of the other nodes on this polytope are given by their associated probability distributions that can be interpreted as barycentric coordinates.

More formally, instead of requiring Z ∈ M K we will consider matrices Z ∈ M c K where M c K is the set of mixed membership matrix defined as follows: Z ∈ M c K if and only if Z ∈ [0, 1] n×K and:

• ||Z i: || 1 = 1 for all 1 ⩽ i ⩽ n;
• for all k ∈ [K], there is at least one index i such that Z ik = 1 (these indices correspond to pure nodes).

Other models like the Overlapping SBM (OSBM) [START_REF] Latouche | Overlapping stochastic block models with application to the French political blogosphere[END_REF], the Overlapping Continuous Community Assignment Model (OCCAM) (Zhang et al., 2020b), or SBM with Overlap (SBMO) [START_REF] Kaufmann | A spectral algorithm with additive clustering for the recovery of overlapping communities in networks[END_REF] have also been proposed, but we will focus on the MMSBM because it is easier to interpret than OCCAM and more general than the OSBM or SBMO. The MMSBM can also be interpreted in certain cases as a RDPG [START_REF] Athreya | Statistical Inference on Random Dot Product Graphs: a Survey[END_REF].

Random Dot Product Graph (RDPG)

RDPGs are a particular case of latent position graphs where each node is associated with an unknown latent position in some space and the connectivity probability between two nodes is determined by the distance (or a measure of similarity) between the corresponding latent positions. Formally, let us define X = (X 1 , . . . , X n ) ⊤ ∈ R n×d where X i is the latent position associated with the node i. Further assume that for each i, j ∈

[n], X ⊤ i X j ∈ [0, 1]. Then an adjacency matrix A is generated from a RDPG(n, X) if A ij ind. ∼ B(X ⊤ i X j ), 1 ⩽ i ⩽ j ⩽ n.
RDPG encompasses the DCSBM and MMSBM as special cases. More information can be found in the survey of [START_REF] Athreya | Statistical Inference on Random Dot Product Graphs: a Survey[END_REF] This chapter is based on Braun et al., 2021a where we introduce a new clustering method for graphs that incorporate nodes side information.

In contrast to previously proposed methods, our algorithm comes with strong consistency guarantees under the CSBM and is minimax optimal under the Contextual Symmetric SBM (CSSBM). The theoretical results are confirmed by experiments on synthetic data that show that our algorithm significantly outperforms concurrent methods. We also show that it can be applied to weighted graph and demonstrate the practical interest of our method on real data. The chapter is organized as follows. In Section 3.1 we introduce the problem and discuss the related work. The statistical framework is presented in Section 3.2 and the algorithms in Section 3.3. We then state our main results and outline the proofs in Section 3.4. Experimental results are presented in Section 3.5. Possible extensions of our framework is discussed in Section 3.6. The additional proofs details are gathered in Section 3.7.

introduction

Real-world networks often come with side information in the form of nodes covariates which can be used to improve clustering performance. For example, when analyzing interactions among people on a social network, we have access to additional features such as gender, age, or ethnicity that can be relevant for the clustering task. Other examples, including biological networks and predator-prey interaction networks are discussed in Newman et al., 2015. The Contextual Stochastic Block Model (CSBM) is a simple extension of the SBM that incorporates such side information: each node is associated with a Gaussian vector of parameters depending only on the community to which the node belongs; see Section 3.2 for details.

Several variants of this model and clustering algorithms have been proposed in the literature to incorporate side information. These methods include model-based approaches [START_REF] Yang | Community Detection in Networks with Node Attributes[END_REF][START_REF] Weng | Community detection with nodal information: Likelihood and its variational approximation[END_REF][START_REF] Hric | Network Structure, Metadata, and the Prediction of Missing Nodes and Annotations[END_REF][START_REF] Emmons | Map equation with metadata: Varying the role of attributes in community detection[END_REF][START_REF] Stanley | Stochastic Block Models with Multiple Continuous Attributes[END_REF][START_REF] Contisciani | Community detection with node attributes in multilayer networks[END_REF][START_REF] Fajardo-Fontiveros | Node Metadata Can Produce Predictability Crossovers in Network Inference Problems[END_REF], spectral methods [START_REF] Binkiewicz | Covariateassisted spectral clustering[END_REF][START_REF] Mele | Spectral Inference for Large Stochastic Blockmodels With Nodal Covariates[END_REF]Abbe et al., 2020b), modularity based optimization methods (Zhang et al., 2016b), belief propagation [START_REF] Deshpande | Contextual Stochastic Block Models[END_REF] and SDP based approaches [START_REF] Yan | Covariate Regularized Community Detection in Sparse Graphs[END_REF]. Even if some of these algorithms come with certain theoretical guarantees, the added value of side information is in general not well understood. The recent works of Abbe et al., 2020b[START_REF] Lu | Contextual Stochastic Block Model: Sharp Thresholds and Contiguity[END_REF]and Ma et al., 2021 clarify the situation by establishing information theoretic thresholds for exact recovery and detection in a special case with two communities. However, the algorithm presented in Abbe et al., 2020b is not likely to be extended to a general CSBM with more than two (possibly unequal-sized) communities, while the latter two results focus on detection rather than consistency.

Our contributions

We make the following contributions in this paper.

• We propose a new iterative algorithm for clustering networks that is fast and is applicable to various settings including the general CSBM and also signed weighted graphs as shown in experiments.

• The proposed algorithm is theoretically analyzed under the CSBM, and we show that its rate of convergence is statistically optimal under the CSSBM. As a byproduct, we derive the threshold for exact recovery with K communities under the CSSBM, thus extending the recent result of Abbe et al., 2020b which was obtained for K = 2.

• We confirm the theoretical properties of our algorithm through experiments on simulated data showing that our method outperforms existing algorithms, not only under the CSBM but also under the Signed SBM the latter of which models community structure in signed networks [START_REF] Cucuringu | SPONGE: A generalized eigenproblem for clustering signed networks[END_REF]. Finally, we provided a real data application of our algorithm.

Related work.

As outlined earlier, covariate-assisted clustering methods have been studied from various perspectives and it is outside the scope of this work to provide an exhaustive survey. Here, we will discuss the literature that is most relevant to our work.

Our iterative method can be thought of as a Classification-EM algorithm [START_REF] Celeux | A Classification EM algorithm for clustering and two stochastic versions[END_REF], hereafter referred to as C-EM, where instead of using the likelihood we use a least squares criterion. Such ideas were first applied and analyzed under various models including associative SBM by [START_REF] Lu | Statistical and Computational Guarantees of Lloyd's Algorithm and its Variants[END_REF] and then extended to a general framework by [START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF] Recently, such ideas were also successfully applied to the Gaussian Tensor Block Model [START_REF] Han | Exact Clustering in Tensor Block Model: Statistical Optimality and Computational Limit[END_REF] and a general GMM [START_REF] Chen | Optimal Clustering in Anisotropic Gaussian Mixture Models[END_REF]. However, the above results can not be directly applied to the CSBM due to the heterogeneity of the data.

Iterative refinement methods can also be derived naturally from the Power Method [START_REF] Wang | Optimal Non-Convex Exact Recovery in Stochastic Block Model via Projected Power Method[END_REF][START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] or alternative optimization methods [START_REF] Chi | Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview[END_REF]. They have been successfully deployed in other settings as well, e.g., SBM [START_REF] Wang | Optimal Non-Convex Exact Recovery in Stochastic Block Model via Projected Power Method[END_REF], group synchronization [START_REF] Boumal | Nonconvex Phase Synchronization[END_REF], joint alignment from pairwise differences [START_REF] Chen | The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences[END_REF], graph matching [START_REF] Onaran | Projected power iteration for network alignment[END_REF] and low-rank matrix recovery [START_REF] Chi | Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview[END_REF].

Despite the huge amount of work on covariate-assisted clustering, there are only limited consistency results. [START_REF] Binkiewicz | Covariateassisted spectral clustering[END_REF][START_REF] Yan | Covariate Regularized Community Detection in Sparse Graphs[END_REF] obtained some weak consistency guarantees that depend on both sources of information, but as noted in Abbe et al., 2020b, those bounds are not tight. Abbe et al., 2020b were the first to propose a method that achieves the threshold for exact recovery, but their algorithm only works when K = 2 and it seems difficult to extend it to a more general setting.

the statistical framework

The CSBM consists of a graph encoded in an adjacency matrix A ∈ {0, 1} n×n and nodes covariates forming a matrix X

= [X 1 X 2 • • • X n ] ⊤ ∈ R n×d
where d is the dimension of the covariate space. The graph and the covariates are generated as follows.

• The graph part of the data is generated from a SBM(Z, Π) with Z ∈ M n,K as defined in Section 1.3.

We further assume in this chapter that the communities are approximately well balanced and that the sparsity level of the graph p max is of order log n/n. Assumption A1. There is a constant α such that for all k ∈

[K] n αK ⩽ n k ⩽ αn K .
This is a common assumption in the literature. The problem becomes harder when the communities have very unbalanced sizes. Assumption A2. We will assume throughout that p max ≍ log n/n.

If the graph is denser, we are in the exact recovery regime and the problem is easy. If we are in a sparser regime, we would need to regularize the adjacency matrix to enforce concentration, or use completely different methods (see e.g. [START_REF] Lu | Contextual Stochastic Block Model: Sharp Thresholds and Contiguity[END_REF].

For the analysis, we will also consider a special case of the SBM where the communities are equal sized, i.e., n k = n/K for all k ∈ [K], and the connectivity matrix is given by

Π =        p q . . . q q p . . . q . . . . . . . . . . . . q q . . . p        ∈ [0, 1] K×K .
We will further assume that p = p ′ log n n and q = q ′ log n n for constants p ′ , q ′ such that p ′ > q ′ > 0. This model will be referred to as the Symmetric SBM (SSBM) and denoted by SSBM(n, K, p, q).

• The nodes covariates are generated by a GMM, independent of A conditionally on the partition Z. More formally, for each i,

X i = µ z i + ϵ i , where ϵ i ind. ∼ N(0, σ 2 I d )
with µ k ∈ R d for all k ⩽ K and σ > 0. We assume that d = O(n).

Remark 2. For ease of exposition we further assume that σ is known but our method can be extended to anisotropic GMM with unknown variance as in [START_REF] Chen | Optimal Clustering in Anisotropic Gaussian Mixture Models[END_REF] We also assume K to be known -this assumption is common in the clustering literature. Estimating K is a non-trivial task which is outside the scope of this work, see [START_REF] Jin | Optimal adaptivity of signed-polygon statistics for network testing[END_REF] for a procedure for SBM. We also leave as further work the incorporation of other forms of covariates, e.g., discrete covariates as in [START_REF] Ahn | Binary Rating Estimation with Graph Side Information[END_REF] 3.3 how to integrate heterogeneous sources of information?

The use of side information should intuitively help to recover clusters that are not well separated on each individual source of information. However, it is not well understood how to integrate two heterogeneous sources of information in the clustering process. Previous attempts [START_REF] Binkiewicz | Covariateassisted spectral clustering[END_REF][START_REF] Yan | Covariate Regularized Community Detection in Sparse Graphs[END_REF] proceed by directly aggregating the adjacency matrix and a Gram matrix (or Kernel matrix) formed by the covariates, but a lot of information can be lost in the aggregation process. Moreover, it is not clear what is the best linear combination of the two matrices. Here, we propose a different approach based on a two step algorithm (see Algorithm 1) that fully exploits all information. In the first step, we obtain a rough estimate of the model parameters from the previous estimate of the partition; the initialization methods that can be used are discussed in Section 3.3.2. Then, in the second step, we iteratively refine the partition, as further explained in Section 3.3.1. In Section 3.5.1 we illustrate via experiments that Algorithm 1 outperforms existing methods for cluster recovery in the setting where the clusters are insufficiently separated on a single source of information.

Algorithm 1 Iterative Refinement with Least Squares

Input: A ∈ R n×n , X ∈ R n×d ,K ∈ N * , σ > 0, Z (
0) ∈ {0, 1} n×K a membership matrix and T ⩾ 1.

1: for 0 ⩽ t ⩽ T -1 do 2:
Given Z (t) , estimate the model parameters: (t) , and µ

n (t) k = |C (t) k |, W (t) = Z (t) (D (t) ) -1 where D (t) = diag(n (t) k ) k∈[K] , Π (t) = W (t) ⊤ AW
(t) k = W (t)⊤ k:
X, for all k ⩽ K.

3:

Refine the partition by solving for each i ∈

[n] z (t+1) i = arg min k ||(A i: W (t) -Π (t) k: ) Σ (t) k || 2 + ||X i -µ (t) k || 2 σ 2
where

Σ (t) k =              diag( n (t) k ′ Π (t) kk ′ ) k ′ ∈[K] (IR-LS) min k ′ n (t) k ′ max k ′ ,k ′′ Π (t) k ′ k ′′ I K (sIR-LS) n K(p (t) -q (t) ) log( p (t) (1-q (t) ) q (t) (1-p (t) ) )I K (IR-LSS) with p (t) = K -1 k∈[K] Π (t) kk and q (t) = (K 2 - K) -1 k̸ =k ′ ∈[K] Π (t)
kk ′ .

4:

Form the matrices Z (t+1) from z (t+1) .

5: end for Output: A partition of the nodes Z (T ) .

The refinement mechanism

At each step t, Algorithm 1 estimates the model parameters given a current estimate of the partition (W (t) ), then updates the partition by reassigning each node to its closest community. Here, the proximity of a node i to a community k is measured by the distance between its estimated (graph) connectivity profile (A i: W (t) ) and its covariates (X i ) to the current estimate of the community parameters (Π

(t) k: , µ (t) k ).
Instead of using the MAP estimator as in C-EM algorithms, we use a least-square criterion. In a model-based perspective, this can be interpreted as a Gaussian approximation of the connectivity profile of each node. We will see later in the experiments that this doesn't lead to a loss of accuracy (see Section 3.5), and is also faster (see Table 1).

Different variants of our algorithm are possible depending on the way the variance of each community is estimated and integrated in the criterion used for the partition refinement. The general method will be referred as IR-LS, the simplified spherical version is denoted by sIR-LS (it assumes implicitly that each community has the same variance) and the version of the algorithm used for CSSBM is denoted by IR-LSS(the optimal choice in this setting to balance the weight attributed to each source of information).

Computational cost. In each iteration, the complexity of estimating the parameters is O(nnz(A) + nd) while that of estimating the partition is O(nK(K + d)). So the total cost of IR-LS is O(T (nnz(A) + nK(max(K, d))). In our setting A is sparse, hence nnz(A) ≍ n log n.

Remark 3. Algorithm 1 can also be used for clustering weighted signed graphs, as shown later in the experiments. Moreover, it is interesting to note that when there are no covariates, the algorithm can be applied to graphs generated from a general SBM. This is in contrast to the iterative algorithm proposed by [START_REF] Lu | Statistical and Computational Guarantees of Lloyd's Algorithm and its Variants[END_REF] that can only be applied to assortative SBMs (see appendix).

Initialization

Different strategies can be adopted for initialization. If we assume that the communities are separated on each source of information and that the Signal-to-Noise Ratio (SNR) is large enough to recover a sufficient proportion in each cluster, we can use a spectral method on one source of information (the graph for example). However, it is in general better to combine both sources of information. While one could use the methods proposed in [START_REF] Yan | Covariate Regularized Community Detection in Sparse Graphs[END_REF][START_REF] Binkiewicz | Covariateassisted spectral clustering[END_REF] that also come with some theoretical guarantees, we instead use Algorithm 2 to initialize the partition. This algorithm will be referred to as EM-Emb. In our experiments, we used the package clusterR (Mouselimis, 2022) for estimating the Gaussian mixture with an EM algorithm. This algorithm is fast, provides a sufficiently accurate estimate of the partition, and avoids hyperparameter tuning. As shown in the experiments (see Section 3.5.4.2), under some parameters regime, we can also use a random initialization strategy.

main results and analysis principle

In this section, we analyze the variants sIR-LS and IR-LSS of Algorithm 1. While it is possible to extend the analysis to IR-LS, it would be considerably more technical and tedious due to its non-spherical structure. Hence, we will assume here that the covariance matrix

Σ (t) k
in Algorithm 1 has the form λ (t) I k where λ (t) is an appropriate scalar depending on whether we use sIR-LS or IR-LSS.

In Section 3.4.1 we will present the general principle for the analysis. Then we will specialize it for analyzing IR-LSS (under the CSSBM) Algorithm 2 EM on graph embedding and covariates (EM-Emb) Input: The number of communities K, the adjacency matrix A, covariates X.

1: Compute U K ∈ R n×K the matrix formed by the eigenvectors associated with the top-K eigenvalues (in absolute order) of A.

2: Merge the columns of U K with the columns of X to obtain a matrix X ′ .

3: Cluster the rows of X ′ by using an EM algorithm for GMM. Output: A partition of the nodes Z (0) . in Section 3.4.2, and prove that the convergence rate obtained is optimal in Section 3.4.3. Finally we show that the same framework can be used to bound the convergence rate of sIR-LS (under the CSBM) in Section 3.4.4. The details of the proofs are outlined in the appendix.

Analysis principle

Our analysis is motivated by the general framework recently developed by [START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF], and also borrows some decomposition techniques used for analyzing Gaussian tensors from [START_REF] Han | Exact Clustering in Tensor Block Model: Statistical Optimality and Computational Limit[END_REF] However, these results are not directly applicable to our setting due to dependencies arising from symmetry in the SBM. Moreover, we need tighter control of the error terms then provided by these works.

We will assume w.l.o.g. that σ = 1 (since σ is assumed to be known in our framework) and that the permutation π that minimizes the distance between z (0) and z is the identity (if not, then replace z by π -1 (z)). Hence there is no label switching ambiguity in the community labels of z (t) because they are determined from z (0) .

The first step is to analyze the event "after one refinement step, the node i will be incorrectly clustered given the current estimation of the partition z (t) at time t". This corresponds to the condition

a ̸ = arg min k ||X i -µ (t) k || 2 + λ(t) ||A i: W (t) -Π (t) k: || 2 for a node i such that z i = a. One can see that this condition is equivalent to the existence of b ∈ [K] \ a such that ⟨ϵ i , μa -μb ⟩ + λ⟨E i: W, Πa: -Πb: ⟩ C i (a,b) ⩽ -∆ 2 (a, b) 2 + Err (t) ib .
Here,

∆ 2 (a, b) = ||µ a -µ b || 2 + λ||Π a: -Π b: || 2 , μk = X ⊤ W :a , Πk: = W ⊤ k: AW, and λ = n min p max or n K(p -q) log p(1 -q) q(1 -p)
depending whether we are analyzing sIR-LS or IR-LSS. Moreover, Err

ib is an error term that can be further decomposed as a sum

F (t) ib + G (t) ib + H (t)
ib of different kinds of error terms which will be controlled in different ways. If we ignore the error term, we obtain the condition corresponding to having an incorrect result after one iteration starting from the ground truth partition. The errors occurring in this way will be quantified by the ideal oracle error

ξ(δ) = n i=1 b∈[K]\z i ∆ 2 (z i , b)1 {C i (a,b)⩽ -(1-δ)∆ 2 (z i ,b) 2 } .
Let us denote

∆ min = min a̸ =b∈[K] ∆(a, b)
to quantify the separation of the parameters associated with the different communities. If ∆ min = 0, it would imply that at least two communities are indistinguishable and the model would not be identifiable. For t ⩾ 1 and δ ∈ [0, 1), let

δ (t) = max 7 8 τ (t-1) τ (0) , δ , τ (t) = τ (0) δ (t)
be sequences where τ (0) = ϵn∆ 2 min /K for a small enough constant ϵ > 0.

In general the rate of decay of ξ(δ) leads to the convergence rate of iterative refinement algorithms, hence it is important to control this quantity.

Condition C1 (ideal error). Assume that ξ(δ (t) ) ⩽ 3 4 τ (t-1) , for all t ⩾ 1 holds with probability at least 1 -η 1 .

We now have to analyze the error terms and prove that their contribution is negligible compared to the ideal oracle error rate. Let

l(z, z ′ ) = i∈[n] ∆ 2 (z i , z ′ i )1 {z i ̸ =z ′ i }
be a measure of distance between two partitions z, z ′ ∈ [K] n . We will control the error terms by showing that the following conditions are satisfied.

Condition C2 (F-error type). Assume that max

{z (t) :l(z,z (t) )⩽τ (0) } n i=1 max b∈[K]\z i (F (t) ib ) 2 ∆ 2 (z i , b)l(z, z (t) ) ⩽ δ 2 256
for all t ⩾ 0 holds with probability at least 1 -η 2 .

Condition C3 (GH-error type). Assume that t+1) 4

max i∈[n] max b∈[K]\z i |H (t) ib | + |G (t) ib | ∆(z i , b) 2 ⩽ δ (
holds uniformly on the event {z (t) : l(z, z (t) ) ⩽ τ (t) } for all t ⩾ 0 with probability at least 1 -η 3 .

We can now show under these conditions that there is a contraction of the error if the initial estimate of the partition is close enough to the ground truth partition.

Theorem 4. Assume that l(z (0) , z) ⩽ τ (0) and δ < 1. Additionally assume that Conditions C1, C2, and C3 hold. Then with probability at least 1 -

3 i=1 η i l(z (t) , z) ⩽ ξ(δ (t) ) + 1 8 l(z (t-1) , z), ∀t ⩾ 1. (3.4.1)
Remark 4. This is an adaptation of Theorem 3.1 in [START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF] where we allow at each step to choose a different δ (t) . It allows us to obtain a weaker condition for initialization than the one used in Theorem 4.1 in Gao et al.,

2019. Indeed, they require l(z (0) , z) = o(

n∆ 2 min K
) in order to have δ = o(1), but we only need l(z (0) , z) = O(

n∆ 2 min K
).

Proof of Theorem 4. By definition δ (t) < 1 for all t ⩾ 1. Let i ∈ [n] such that z i = a and assume that l(z (t-1) , z) ⩽ τ (t-1) for some given t ⩾ 1. Denoting

I (t) i (a, b) := 1 {C i (a,b)⩽ -(1-δ (t) )∆ 2 (a,b) 2 } , observe that 1 {z (t) i =b} (1) ⩽ 1 {C i (a,b)⩽ -∆ 2 (a,b) 2 +F (t-1) ib +G (t-1) ib +H (t-1) ib } (2) ⩽ I (t) i (a, b) + 1 { δ (t) 2 ∆ 2 (a,b)⩽F (t-1) ib +G (t-1) ib +H (t-1) ib } (3) ⩽ I (t) i (a, b) + 1 { δ (t) 4 ∆ 2 (a,b)⩽F (t-1) ib } ⩽ I (t) i (a, b) + 1 { δ 4 ∆ 2 (a,b)⩽F (t-1) ib } (4) ⩽ I (t) i (a, b) + 32(F (t-1) ib ) 2 δ 2 ∆ 4 (a, b)
.

The inequality (1) follows from the definition of z (t) i and the error decomposition. Inequality (2) comes from a union bound while (3) uses Condition C3. Finally, (4) follows from Markov inequality. Hence,

l(z (t) , z) = i∈[n] b∈[K]\{z i } ∆ 2 (z i , b)1 {z (t) i =b} ⩽ i∈[n] b∈[K]\{z i } ∆ 2 (z i , b)1 {C i (a,b)⩽ -(1-δ (t) )∆ 2 (z i ,b) 2 } + i∈[n] b∈[K]\{z i } ∆ 2 (z i , b)1 {z (t) i =b} 32(F (t-1) ib ) 2 δ 2 ∆ 4 (z i , b) ⩽ ξ(δ (t) ) + i∈[n] max b∈[K]\{z i } 32(F (t-1) ib ) 2 δ 2 ∆ 2 (z i , b) ⩽ ξ(δ (t) ) + 1 8 l(z (t-1) , z).
Using Condition C1, we hence obtain

l(z (t) , z) ⩽ ξ(δ (t) ) + 1 8 τ (t-1) ⩽ 7 8 τ (t-1) .
Thus τ (t) is an upper bound for l(z (t) , z) and the theorem is proved by induction.

By iteratively unwrapping (3.4.1) we obtain for t large enough the following bound on l(z (t) , z).

Corollary 1. Assume that the assumptions of Theorem 1 hold. Denoting

t * = ⌈ log(1/δ) log(8/7) ⌉, we have for all t ⩾ t * that τ (t) = τ, δ (t) = δ and l(z (t) , z) ⩽ 8 7 ξ(δ) + 3 28 τ (0) 1 8 t-t * .
This bound shows that if t is suitably large, then the estimation error is of the order of the oracle error.

Proof of Corollary 1. For convenience, denote τ = τ (0) δ. Let t * be the smallest integer such that τ ⩾ ( 7 8 ) t τ (0) ; clearly, t * = ⌈ log(1/δ) log(8/7) ⌉. Then for t ⩾ t * , we have τ (t) = τ = τ (0) δ, and hence (from the definition of δ (t) ), δ (t) = δ. Therefore Theorem 4 implies that for t ⩾ t * ,

l(z (t) , z) ⩽ ξ(δ (t) ) + 1 8 ξ(δ (t-1) ) + . . . + 1 8 t-t * ξ(δ (t * ) ) + 1 8 t-t * +1 ξ(δ (t * -1) ) + . . . + 1 8 t-1 ξ(δ (1) ) ⩽ 8 7 ξ(δ) + 1 8 t-t * +1 ξ(δ (t * -1) ) + . . . + 1 8 t-1 ξ(δ (1) ) (since δ (t) = δ for t ⩾ t * ) ⩽ 8 7 ξ(δ) + 3 4 τ (0) 1 8 t-1 + . . . + 1 8 t-t * +1
(since for all t, ξ(δ

= 1 8 min k̸ =k ′ ||µ k -µ k ′ || 2 + log n K ( p ′ -q ′ ) 2 .
It is easy to see that ∆ ≍ ∆ min . The following lemma shows that ξ(δ) decreases exponentially fast in ∆ provided ∆ min is suitably large.

Lemma 1. Assume that K 1.5 /∆ min → 0 and δ = δ(n) → 0 at a suitably slow rate. Then with probability at least 1exp(-∆), we have

ξ(δ) ⩽ n exp(-(1 + o(1)) ∆2 ).
The following theorem shows that if z (0) is close enough to z, then the misclustering rate decreases exponentially fast with the SNR ∆

after O(log n) iterations.

Theorem 5. Assume that K 1.5 /∆ min → 0 and ∆2 ≍ log n/K. Under the CSSBM(n, K, p, q) assumption, if z (0) is such that

l(z, z (0) ) ⩽ ϵn∆ 2 min K
for a constant ϵ small enough, then with probability at least 1 -n -Ω(1) we have for all t ≳ log n r(z (t) , z) ⩽ exp(-(1 + o(1)) ∆2 ).

Sketch of proof. We first show that Conditions C2 and C3 are satisfied.

Then we show that Condition C1 is satisfied for the sequences δ (t) and τ (t) , hence Theorem 4 can be applied to obtain a contraction of the error at each step.

Remark 5. By assumption ∆2 ≳ log n/K, and so the condition ∆2 ≍ log n/K is not very restrictive. Indeed, if the information provided by the GMM part was not of the same order as the graph part, it would not be useful to aggregate information. If ∆2 ≫ log n then we would be in the exact recovery setting and the problem becomes easy.

Remark 6. The initial condition implies that h(z (t) , z)/n ⩽ ϵ/K where h denotes Hamming distance, see appendix for details. This is a detection condition. It is outside the scope of this work to analyze an algorithm that achieves this condition under CSBM. The numerical experiment done in Section 3.7 suggests that the algorithm can still work with a random initialization, at least in particular cases.

Minimax lower-bound for CSSBM

We are going to establish that the convergence rate established in Theorem 5 is optimal. Let

Θ = {(µ k ) k∈[K] ∈ R K , p, q ∈ [0, 1] such that p > q}
be the admissible parameter space.

Theorem 6. Under the assumption ∆/ log K → ∞, we have

inf ẑ sup θ∈Θ E(r(ẑ, z)) ⩾ exp(-(1 + o(1)) ∆2 ). If ∆ + log K = O(1), then inf ẑ sup θ∈Θ E( r( ẑ,z) n ) ⩾ c for some positive constant c.
Remark 7. This lower-bound shows that if ∆2 < log n then every estimator fails to achieve exact recovery with a probability bounded below from zero because sup θ∈Θ E(r(ẑ, z)) > n ϵ for some ϵ > 0. On the other hand, Theorem 5 shows that when ∆2 > log n then IR-LSS achieves exact recovery. Hence the threshold for exact recovery is ∆2 / log n. When K = 2 and µ 1 = -µ 2 = µ this matches the result obtained by Abbe et al., 2020b. Sketch of proof. We can use the same argument as in Theorem 3. 3 Lu et al., 2016 to reduce the problem to a hypothesis testing problem. The solution of the latter is given by the maximum likelihood test according to the Neyman-Pearson lemma. Then the probability of error can be controlled by using concentration inequalities.

Convergence guarantees for sIR-LS under the CSBM

The proof techniques used in the previous section can be extended in a straightforward way to obtain consistency results for sIR-LS under the CSBM. The main difference is that the specialized concentration inequality used to prove Lemma 1 can no longer be applied to this setting.

Theorem 7. Assume that

K 1.5 ∆ min → 0, ∆ 2 min ≍ log n/K and max a,b∈[K] ∆ 2 (a, b) ≲ ∆ 2 min .
Under the CSBM with approximately balanced communities, if for some small enough constant ϵ > 0, l(z, z (0) ) ⩽ , then with probability at least 1 -n -Ω(1) we have

r(z (t) , z) ⩽ exp - 1 8 ∆ 2
min for all t ≳ log n.

Comparison with existing theoretical results

Abbe et al., 2020b obtain the same bound as us when K = 2. Their method -which is an aggregated spectral method -requires computing the largest eigenvector of XX ⊤ (with diagonal set to zero) which has a complexity O(n 2 ). In contrast, our method has complexity O(n log n + nd) and is thus faster when d = o(n).

numerical experiments

We now empirically evaluate our method on both synthetic and real data1 . Section 3.5.1 contains simulations for the CSBM and Section 3.5.2 contains results for clustering signed networks under a Signed SBM. In Section 3.5.3, we test our method on a dataset consisting of a (weighted) signed graph along with covariate information for the nodes.

CSBM with not well separated communities

In this experiment the graph is generated from a SBM with parameters n = 1000, K = 3, Z i i.i.d ∼ Multinomial(1; 1/3, 1/3, 1/3), and

Π = 0.02 *     1.6 1.2 0.05 1.2 1.6 0.05 0.05 0.05 1.2     .
The covariates are generated from a GMM with variance σ 2 = 0.2 and class centers µ 1 = (0, 0, 1), µ 2 = (-1, 1, 0), µ 3 = (0, 0, 1). Note that C 1 , C 3 cannot be separated by the covariate information, while C 1 , C 2 are not well separated in the graph information (as seen from Π). Hence, one would expect in this example that using only a single source of information should not yield good clustering results. To demonstrate this, we use the Normalized Mutual Information (NMI) criterion to measure the quality of the resulting clusters. It is an information theoretic measure of similarity taking values in [0, 1], with 1 denoting a perfect match, and 0 denoting the absence of correlation between partitions.

Performance comparison

We will use K-SC and L-SC to denote respectively the results obtained by applying spectral clustering on the Gaussian kernel matrix K formed from the covariates, and spectral clustering (SC) applied on the Laplacian of the graph. Additionally, SDP-Comb refers to the method proposed by Yan et al., 2020; IR-MAP is similar to IR-LS but with the least-square criterion replaced by the MAP to update the partition; ORL-SC (Oracle Regularized Laplacian SC) corresponds to SC applied on A + λK where λ is chosen to maximize the NMI between the (oracle known) true partition and the one obtained by using SC on A + λK. For the implementation of SDP-Comb, we used the Matlab code provided by Yan et al., 2020 with λ given by ORL-SC. We limited our comparison to the aforementioned methods for concreteness; a comparison with all existing methods from the literature would need a separate study and is outside the scope of the paper.

Figure 1 shows that the three iterative methods considered (IR-MAP, IR-LS, sIR-LS), initialized with EM-Emb, provide significantly better clustering performance compared to the other methods. The variance of sIR-LS is a bit larger than IR-LS. On the other hand, other methods based on aggregating the two sources of information (SDP-Comb and ORL-SC ) lead to a limited improvement in clustering performance. Additional experiment results in the appendix suggest that the iterative methods considered also work with random initialization.

Computational cost

We took the average of CPU time (in seconds) over 20 repetitions. There is an important gain in speed obtained by replacing the MAP objective by a least square criterion. Moreover, the initialization obtained with EM-Emb is very fast. The results are gathered in Table 1.

L-SC ORL-SC EM-Emb IR-LS IR-MAP Time 1.4 7.9 0.5 1.2 37.1 Ratio 2.7 15 1 2.

70

Table 1: Comparison between computation times (averaged over 20 runs).

Signed SBM

A graph is generated from the Signed SBM as follows. First we generate an Erdös-Renyi graph where each edge appears with probability p and each edge takes the value 1 if both extremities are in the same community and -1 otherwise. Then we flip the sign of each edge independently with probability η ∈ [0, 1/2). Our method sIR-LS can be directly applied to this setting, but we can also use the fact that the connectivity matrix Π is assortative to design a more specialised algorithm IR-SSBM (see Algorithm 3) that assigns a node to the community which maximizes its intra-connectivity estimated probability. For initialization, we use Sponge-sym [START_REF] Cucuringu | SPONGE: A generalized eigenproblem for clustering signed networks[END_REF] for clustering signed graphs. Figure 2 shows that 20 refinement steps improve the clustering.

Algorithm 3 IR-SSBM

Input: The number of communities K, initial partition z (0) , T ⩾ 1.

1: for 0 ⩽ t ⩽ T -1 do 2: Compute W (t) = Z (t) (D (t) ) -1 where D (t) = diag(n (t) k ) k∈[K]
, and C (t) = AW (t) .

Australia Rainfall Dataset

We consider the time series data of historical rainfalls in locations throughout Australia, this was also studied in [START_REF] Cucuringu | SPONGE: A generalized eigenproblem for clustering signed networks[END_REF].

Edge weights are obtained from the pairwise Pearson correlation, leading to a complete signed graph on n = 306 nodes. We use the longitude and latitude as covariates X, and Sponge [START_REF] Cucuringu | SPONGE: A generalized eigenproblem for clustering signed networks[END_REF] to obtain an initial partition for sIR-LS and Iter-SBM (the version of sIR-LS without covariates). We exclude IR-LS here due to its relative instability on this dataset (see appendix). This shows that in some situations it can be better to use sIR-LS rather than IR-LS.

Random initialization

In this subsection, we show that under some parameters regime, our method can work in practice with a random initialization. But in general it seems better to use EM-Emb.

Comparison between IR-MAP and IR-LS

In this experiment we fix K = 2 and n = 1000. Let c > 0 and define p = 4c log n/n and q = c log n/n. Also define a Gaussian mixture in R with centers 1 (for community 1) and 2 (for the second community). The variance parameter σ is chosen such that the information provided by the Gaussian part of the model is equal to the one provided by the graph. More precisely, we set σ 2 = 1 2c log n . The value c = 0.5 corresponds to the threshold for exact recovery. Figure 7 shows that when initialized with a partition Z (0) where each entry is generated independently from a Multinomial(1; 1/2, 1/2), sIR-LS performs slightly better than IR-MAP, has less variability, and is robust to random initialization, hence justifying the Gaussian approximation.

Comparison between random initialization and initialization with

EM-Emb

We use a similar experimental setting as the one described in Section 5.1. We only slightly change the connectivity matrix

Π = 0.02 *     1.6 1.2 0.5 1.2 1.6 0.5 0.5 0.5 1.2     .
Figure 8 shows that when randomly initialized ours algorithms IR-LS and sIR-LS can suffer from numerical instability. That's why we recommend to use EM-Emb instead. But it could interesting to develop a strategy based on random initialization by identifying and disregarding the random initialization that lead to atypical results.

Heterophilic SBM

If we disregard the covariates, our algorithm can be used for inference under a general SBM, in contrast to the method proposed by [START_REF] Lu | Statistical and Computational Guarantees of Lloyd's Algorithm and its Variants[END_REF] which was restricted to the assortative setting. In particular, our algorithm also works for networks with heterophilic communities. The following experiment illustrates the gain in term of accuracy for IR-LS initialized with A-SC (spectral clustering on the adjacency matrix). It also shows the interest of using more than one iteration in the refinement step with the MAP (this corresponds to IR-MAP(1)).

We consider n = 1000, K = 3, Z i i.i.d ∼ Multinomial(1; 1/3, 1/3, 1/3) andΠ

=     0.2 0.05 0.1 0.05 0.15 0.05 0.1 0.05 0.03     .
The NMI is averaged over 40 repetitions; the results are shown in Figure 9. We also considered the Variational Expectation Maximization (VEM) algorithm implemented in the R package blockmodels [START_REF] Léger | Blockmodels: A R-package for estimating in Latent Block Model and Stochastic Block Model, with various probability functions, with or without covariates[END_REF], but the running time was prohibitive (approximately one hour for a single Monte Carlo run, whereas our algorithm takes a few seconds). It nevertheless returned the exact partition as IR-LS.

possible extensions and future work

Although commonly used as a benchmark for clustering on graphs, the SBM doesn't encode several properties often observed in real-life networks such as degree heterogeneity, mixed-membership, powerlaw, and so on. In this section, we will discuss how the method we proposed can be extended to other models.

Degree-corrected SBM

The DCSBM has been defined in Section 2.3.1. Due to the degree heterogeneity, the refinement mechanism used in IR-LS won't work. But it can be fixed in the following manner. Due to the property that t) . But now we have to take into account the fact that each node connectivity profile A i: W (t) scale differently. A solution could be to update according to the following rule arg min

Z T ΘW = I K since i∈C k θ i = 1 for all k, we can still estimate Π (t) by W (t) ⊤ AW (
k∈[K],θ∈R A i W (t) -θΠ (t) k: 2 .
It can be solved by testing all the possibilities for k and computing the corresponding θ (solution of a least square problem). A similar idea using angles (but that could be rewritten in this form) has been proposed by the independent work of [START_REF] Hu | Multiway Spherical Clustering via Degree-Corrected Tensor Block Models[END_REF] in the same time as we released the present work. They used this method for clustering tensors with heterogeneous degrees and their results don't apply directly to DCSBM but we believe that one could show that by incorporating in this way nodes heterogeneity in the refinement process, the output of the algorithm is minimax optimal under the model considered in [START_REF] Gao | Community detection in degree-corrected block models[END_REF]

Mixed-Membership SBM

We can also adapt IR-LS to MMSBM (see Section 2.3.2 for the definition of the model) by Algorithm 4. This leads to an algorithm close to the one proposed by [START_REF] Arroyo | Overlapping Community Detection in Networks via Sparse Spectral Decomposition[END_REF], but that doesn't involve hyperparameter tuning.

Algorithm description

Algorithm 4 Generalized Power Method (GPM)

Input: A ∈ R n×n , K ∈ N * , Z (0) ∈ [0,
1] n×K a mixed membership matrix, a threshold parameter δ and T ⩾ 1, the number of iterations.

1: for 0 ⩽ t ⩽ T -1 do 2:
Given Z (t) , form the matrix

W (t) = Z (t) (Z (t)⊤ Z (t) ) -1 esti- mate the model parameters: Π (t) = W (t)⊤ AW (t) 3:
Refine the partition by solving for each i ∈

[n] Z (t+1) i = P C (A i W (t) Π (t) -1 ) (3.6.1)
where

C = {x ∈ R K + : ||x|| 1 = 1}. 4:
Form the matrices Z (t+1) from (Z (t+1) i

) i .

5: end for Output: Z (T ) .

At each step t, Algorithm 4 first estimates the model parameters (Π (t) ) given a current estimate of the partition (W (t) ), and then updates the latent position of each node by projecting A i W (t) Π (t) -1 onto the simplex C. It can be shown using some properties of the pseudo-inverse that the matrix T t in the work of [START_REF] Arroyo | Overlapping Community Detection in Networks via Sparse Spectral Decomposition[END_REF] corresponds to AW (t) Π (t) -1 . This latter way of writing T t can be interpreted in a easy way: A i W (t) corresponds to the estimated connectivity profile of node i. In the noiseless case it would be equal to P i W = Z i Π. So in order to obtain Z i , it only remains to multiply by right with the inverse of Π.

The main difference with the GPM presented in Arroyo et al., 2021 lies in the projection step. We directly project on the simplex, whereas Arroyo et al., 2021 first applies a thresholding operator and then normalize the rows of the resulting matrix. It is unclear to us if the composition of these operations is a projection. Moreover, due to a difference in the model considered, we use the l 1 norm to impose a constraint on the rows of Z instead of the l 2 norm. computational complexity. Computing Z (t)⊤ Z (t) has a complexity O(K2 n) and the inverting a K × K matrix has complexity O(K 3 ), so this operation is negligible. Hence, computing W (t) has a complexity O(nK 2 ). The complexity of estimating

Π (t) is O(Knnz(A) + K 2 n).
Under the sparsity assumption p max ≍ log c n/n, c > 1, we have nnz(A) ≍ n log c n and since K ≪ log n the dominant term in the complexity is Knnz(A). So computing AW (t) Π (t) -1 has complexity O(Knnz(A)). Finally, since there are T iterations we get a total complexity of O(T Knnz(A)). Usually T is of the order log n, so in a relatively sparse regime, the computational cost of the refinement procedure is O(npolylog(n)). As shown in the experiments, there are fast spectral methods to get a rough initial estimate Z (0) , but still good enough to lead to accurate estimate after applying GPM.

Numerical experiments

We considered the following methods to compare with: SPACL (Mao et al., 2021b) a fast spectral method, M-SCORE [START_REF] Jin | Estimating network memberships by simplex vertex hunting[END_REF] a method based on a vertex hunting algorithm that is shown to outperform previously proposed methods and sPCA [START_REF] Arroyo | Overlapping Community Detection in Networks via Sparse Spectral Decomposition[END_REF] an algorithm based on a generalization of the power method.

We didn't add the method proposed by [START_REF] Mao | Overlapping Clustering Models, and One (Class) SVM to Bind Them All[END_REF] since the authors only provide a Matlab implementation of their algorithm requiring a licence to run. We also tried the method proposed by Majmudar et al., 2020 with inputs A and the normalized correlation matrix D -1/2 A 2 D -1/22 , but the algorithm didn't return any estimate of the latent positions.

The accuracy of the clustering is measured by

|| Ẑ -Z|| 1 := min σ∈S K i || Ẑi: σ -Z i: || 1
where S K is the set of permutation matrices in [0, 1] K×K . The permutation that best aligns Ẑ and Z is found by solving a Linear Assignment problem with the cost matrix C ∈ R K× defined by

C kk ′ = ||Z :k -Ẑ:k ′ ||.
The latent positions (Z i ) are generated by independent Dirichlet random variables with parameter α = (1/K, . . . , 1/K). We then apply T 1/K so that ||Z i || 1 = 1. The connectivity matrix Π as the symmetric form Π = (p -q)1 K 1 ⊤ K + qI K where p > q > 0. In the following, the parameter δ of our algorithm is set to 1/(1.1K).

initialization We considered two methods for initializing our algorithm. First we used SCORE+, a spectral method proposed by Jin et al., 2022 for clustering SBM. When initialized with this method, Algorithm 4 will be referred as IR-SCORE. Since SCORE+ is not designed for MMSBM, we also considered the fast spectral method SPACL (Mao et al., 2021b) to initialize Algorithm 4. In this case, the algorithm is referred as IR-SPACL. influence of the number of anchor nodes For this experiment we fixed K = 3. As shown in Figure 10, SPACL is not sensitive to the number of pure nodes. While having better performances, M-SCORE suffers from the same drawback. In particular, its performances deteriorate when almost all the nodes are pure. On the other hands, the iterative methods IR-MM, IR-SPACL and sPCA have better performances and achieve exact recovery when there are only pure nodes. Even in the case when n 0 = 0 our methods perform well suggesting that the assumption that there exists pure node is not necessary provided that the columns of Z are enough separated.

influence of the number of communities Since the computational complexity of M-SCORE is exponential in the number of communities, we didn't integrate this algorithm in this experiment.

For this experiment we fix n = 5000. Figure 11 shows that IR-MM has the best performances when the number of communities increases.

When the number of communities is small, whether we initialize our algorithm with Score+ or SPACL we get the same result. On the other hand, if the number of communities is large and the latent positions not enough separated (n 0 small) iterative methods can suffer from numerical instability and does no longer provide accurate estimates.

Other extensions

In Chapter 4 we study a setting where partial contextual information is provided by multilayer graphs. The iterative strategy considered in this chapter can be adapted to this setting, see Section 4.7. Instead of considering Gaussian covariates, we can have access to discrete, high-dimensionnal covariates. A first step to extend the method to this setting is presented in Chapter 5.

It would be also interesting to further study when random initialization based strategies could work and develop variant of IR-LS robust to missing values and outliers.

additional proofs

Proof of Lemma 1

Let δ, δ > 0 . The ideal oracle error term can be upper bounded as follows

ξ(δ) ⩽ n i=1 b∈[K]\z i ∆ 2 (z i , b)1 {⟨ϵ i ,µ z i -µ b ⟩+λ⟨E i: W,Π z i : -Π b: ⟩⩽ -(1-δ-δ)∆ 2 (z i ,b) 2 } M 1 + n i=1 b∈[K]\z i ∆ 2 (z i , b)1 {⟨ϵ i , μz i -µ z i ⟩+λ⟨E i: W, Πz i : -Π z i : ⟩⩽ -δ∆ 2 (z i ,b) 4 } M 2 + n i=1 b∈[K]\z i ∆ 2 (z i , b)1 {⟨-ϵ i , μb -µ b ⟩-λ⟨E i: W, Πb: -Π b: ⟩⩽ -δ∆ 2 (z i ,b) 4 } M 3
.

We will first obtain upper bounds for each E(M i ), i = 1 . . . 3. In particular we will show that the dominant term is E(M 1 ). Then, we will use Markov inequality to control ξ(δ) with high probability.

upper bound of E(M 1 ). Let us denote for any given i ∈

[n] and b ∈ [K] \ z i the event Ω 1 = ⟨ϵ i , µ z i -µ b ⟩ + λ⟨E i: W, Π z i : -Π b: ⟩ ⩽ -(1 -δ -δ)∆ 2 (z i , b) 2 .
By using an analogous argument as the one presented in Lemma 7 we obtain

P(Ω 1 ) ⩽ exp(-(1 + o(1)) ∆2 ).
Thus by taking δ = δ going to zero as n → ∞, we obtain

E(M 1 ) ⩽ n i=1 b∈[K]\z i ∆ 2 (z i , b) exp(-(1 + o(1)) ∆2 ) ⩽ nK exp(-(1 + o(1)) ∆2 ) ⩽ n exp(-(1 + o(1)) ∆2 ).
In the second line we used the fact that

∆ 2 (z i , b) = ∆ min ≲ ∆2 for all z i ̸ = b and ∆ ≍ log n/K → ∞.
In the third line we used the assumption ∆2 / log(K) → ∞.

upper bound of E(M 2 ). Let us denote for any given i ∈

[n] and b ∈ [K] \ z i the events Ω 2 = ⟨ϵ i , μz i -µ z i ⟩ + λ⟨E i: W, Πz i : -Π z i : ⟩ ⩽ -δ∆ 2 (z i , b) 4 , Ω ′ 2 = ⟨ϵ i , μz i -µ z i ⟩ ⩽ -δ∆ 2 (z i , b) 8 ,
and

Ω ′′ 2 = λ⟨E i: W, Πz i : -Π z i : ⟩ ⩽ -δ∆ 2 (z i , b) 8 .
Clearly P(Ω 2 ) ⩽ P(Ω ′ 2 ) + P(Ω ′′ 2 ) by a union bound argument. Let us first upper bound P(Ω ′ 2 ). Recall that n k = n/K under the CSSBM by assumption, let us also define n min := min k n k . We keep this general notation because it is shorter and indicates how the proof can be generalized to the unbalanced setting. By definition μz i -

µ z i = j∈C z i ϵ j n z i , hence ⟨ϵ i , μz i -µ z i ⟩ = ||ϵ i || 2 + ϵ ⊤ i j∈C z i j̸ =i ϵ j n z i .
This last quantity is lower bounded by ϵ ⊤ i η i where η i = j∈Cz i , j̸ =i ϵ j n z i . In particular ϵ i and η i are independent and their entries are also independent. Moreover η i is a centered gaussian random variables with independent entries such that Var((η i ) k ) ⩽ 1/n k . So by Bernstein inequality, it holds for all x > 0 that

P ||η i || 2 ⩾ 1 n k (K + 2 √ Kx + 2x) ⩽ exp(-x)
which in turn implies

P ϵ ⊤ i η i ⩽ - δ∆ 2 min 8 ⩽ P ϵ ⊤ i η i ⩽ - δ∆ 2 min 8 ||η i || 2 ⩽ 1 n k (K + 2 √ Kx + 2x) + P ||η i || 2 ⩾ 1 n k (K + 2 √ Kx + 2x) ⩽ exp -c n k ( δ∆ 2 min ) 2 K + 2 √ Kx + 2x + exp(-x). Setting x = √ n k δ∆ 2 min we obtain P(Ω ′ 2 ) ⩽ 2 exp(-C δ√ n k ∆ 2 min ). Since δ → 0 (as n → ∞) at a suitably slow rate, we have δ n/K → +∞. Consequently, P(Ω ′ 2 ) = o(exp(-(C + o(1))∆ 2 min ) = o(exp(-(1 + o(1)) ∆2 )).
Let us now bound

P(Ω ′′ 2 ). Since Πkk ′ -Π kk ′ = i∈C k , j∈C k ′ E ij n k n k ′ = W ⊤ :k EW :k ′ , we obtain the decomposition λ⟨E i: W, Πz i : -Π z i : ⟩ = λ⟨E i: W, W ⊤ z i : E (i) W⟩ + λ⟨E i: W, W ⊤ z i : E (-i) W⟩
where E (i) is obtained from E by only keeping the ith row and column, and E -i is the matrix obtained from E by replacing the ith row and column by zero. In particular, E (-i) is independent from E i: . The second term can be controlled by using the same techniques as before. Indeed, the entries of

W ⊤ z i : E (-i) W are independent and Var(W ⊤ z i : E (-i) W :k ) ⩽ C p max n 2 min for all k. Denoting η ′ ik = W ⊤ z i : E (-i) W :k and η ′ i = (η ′ ik ) k∈[K] , this implies P λ⟨E i: W, η ′ i ⟩ ⩽ - δ∆ 2 min 16 ⩽ P λ⟨E i: W, η ′ i ⟩ ⩽ - δ∆ 2 min 16 ∀k, |η ′ ik | 2 ⩽ C p max n 2 k (K + x) + P ∃k, |η ′ ik | 2 ⩾ C p max n 2 min (K + x) ⩽ K exp -C n 3 min ( δ∆ 2 min ) 2 p max λ 2 (K + x) + K exp(-x).
Here, we used a union bound argument for the first inequality. The second inequality uses Lemma 33 -which provides a concentration bound for binomial random variables -along with the fact λ ≍ n Kp max . Setting x = C √ n k p max δ∆ 2 min we obtain

P λ⟨E i: W, η ′ i ⟩ ⩽ - δ∆ 2 min 16 ⩽ 2 exp(-c √ n k p max δ∆ 2 min ) = o(exp(-∆2 )) since δ can be chosen such that √ n k p max δ → +∞ (because by as- sumption np max ≍ log n ≫ K 4 ) and ∆ min ≍ ∆. It remains to control ⟨E i: W, W ⊤ z i : E (i) W⟩. Using the fact (W ⊤ z i : E (i) W) k =      j ′ ∈C k E ij ′ n z i n k if k ̸ = z i 2 j ′ ∈C k E ij ′ n 2 z i if k = z i
we have

⟨E i: W, W ⊤ z i : E (i) W⟩ = k̸ =z i j∈C k j ′ ∈C k E ij n k E ij ′ n z i n k + 2n z i   j∈C z i E ij ′ n 2 z i   2 = 1 n z i   j∈C k E ij n k   2 + 2n z i   j∈C z i E ij ′ n 2 z i   2 ⩾ 0.
Consequently, P Ω ′′ 2 can be bounded as

P Ω ′′ 2 ⩽ P λ⟨E i: W, η ′ i ⟩ ⩽ - δ∆ 2 min 16 = o(exp(-∆2 )).
upper bound of E(M 3 ). Let us denote for any given i ∈

[n] and b ∈ [K] \ z i the event Ω 3 = ⟨-ϵ i , μb -µ b ⟩ -λ⟨E i: W, Πb: -Π b: ⟩ ⩽ -δ∆ 2 (z i , b) 4 .
First observe that

⟨ϵ i , μb -µ b ⟩ = ϵ ⊤ i j∈C b ϵ j n z i ,
therefore this term can be handled in the same way as before. Moreover, we have

λ⟨E i: W, Πb: -Π b: ⟩ = λ⟨E i: W, W ⊤ b: E (i) W⟩ + λ⟨E i: W, W ⊤ b: E (-i) W⟩.
The second term can be handled in the same way as before by using a conditioning argument. Now observe that

⟨E i: W, W ⊤ b: E (i) W⟩ = 1 n 2 z i n b   j∈C b E ij     j ′ ∈C z i E ij ′  
where j∈C b E ij and j ′ ∈C z i E ij ′ are independent subgaussian random variables. Thus this term can also be controlled by using the same conditioning argument as before.

conclusion. The previously obtained upper bounds imply

E(ξ(δ)) ⩽ 3E(M 1 ) ⩽ n exp(-(1 + o(1)) ∆2 ).
Finally, by Markov inequality, we obtain

P(ξ(δ) ⩾ exp( ∆)Eξ(δ)) ⩽ exp(-∆). But since exp( ∆)Eξ(δ) ⩽ n exp(-(1 + o(1)) ∆2 )
we obtain that with probability at least 1exp(-∆)

ξ(δ) ⩽ n exp(-(1 + o(1)) ∆2 ).

Proof of Theorem 5

The general proof strategy has been presented in Section 3.4.1. In Section 3.7.2.1 we will make the error decomposition explicit. Then, we will control the different error terms in Sections 3.7.2.2, 3.7.2.3 and 3.7.2.4. Finally, we will conclude by applying Theorem 4 in Section 3.7.2.5.

Error decomposition for the one-step analysis of IR-LSS

We will assume without lost of generality that σ = 1 to simplify the exposition. Let i ∈ [n] and a ∈ [K] be such that3 z i = a, and let

λ (t) = n K(p (t) -q (t) ) log p (t) (1 -q (t) ) q (t) (1 -p (t) )
denote the scalar corresponding to the diagonal entry of the inverse covariance matrix

Σ (t) k . Similarly, let us denote λ = n K(p -q) log p(1 -q) q(1 -p) .
Given the current estimator of the partition Z (t) , node i will be incorrectly estimated after one refinement step if

a ̸ = arg min k ||X i -µ (t) k || 2 + λ(t) ||A i: W (t) -Π (t) k: || 2 or equivalently, if there exists b ∈ [K]\a such that ||X i -µ (t) b || 2 + λ(t) ||A i: W (t) -Π (t) b: || 2 ⩽ ||X i -µ (t) a || 2 + λ(t) ||A i: W (t) -Π (t) a: || 2 .
The above inequality is equivalent to

⟨ϵ i , μa -μb ⟩ + λ⟨E i: W, Πa: -Πb: ⟩ ⩽ -∆ 2 (a, b) 2 + F (t) ib + G (t) ib + H (t) ib
where

∆ 2 (a, b) = ||µ a -µ b || 2 + λ||Π a: -Π b: || 2 , μk = X ⊤ W :a ,
and Πk:

= W ⊤ k: AW for all k ∈ [K]. Furthermore, the terms F (t) ib , G (t)
ib and

H (t)
ib are given by

F (t) ib = ⟨ϵ i , ( μa -µ (t) a ) -( μb -µ (t) b )⟩+ λ (t) ⟨E i: W (t) , ( Πa: -Π (t) a: ) -( Πb: -Π (t) b: )⟩ + λ (t) ⟨E i: (W -W (t) ), Πa: -Πb: ⟩ + (λ -λ (t) )⟨E i: W, Πa: -Πb: ⟩, 2G (t) ib = (||µ a -µ (t) a || 2 -||µ a -μa || 2 ) -(||µ a -µ (t) b || 2 -||µ a -μb || 2 ) + λ (t) (||P i: W (t) -Π (t) a: || 2 -||P i: W (t) -W ⊤ a: AW (t) || 2 ) -λ (t) (||P i: W (t) -Π (t) b: || 2 -||P i: W (t) -W ⊤ b: AW (t) || 2 ) and 2H (t) ib = ||µ a -μa || 2 -||µ a -μb || 2 + ||µ a -µ b || 2 + λ (t) (||P i: W (t) -W ⊤ a: AW (t) || 2 -||P i: W (t) -W ⊤ b: AW (t) || 2 + ||Π a: -Π b: || 2 ) + (λ -λ (t) )||Π a: -Π b: || 2 .
The main term in this decomposition is

⟨ϵ i , μa -μb ⟩ + λ⟨E i: W, Πa: -Πb: ⟩ ⩽ -∆ 2 (a, b) 2
and corresponds to the error when the current estimation of the partition is the ground truth partition. It is controlled by Lemma 1

The three error terms will be controlled in different ways. The error term

F (t)
ib depends in a crucial way on i and t, it will be controlled with a l 2 -type norm (see Condition C2). The square of the error terms G

Bounding the error term F (t) ib

In this section we are going to show that Condition C2 is satisfied.

Lemma 2. Under the assumptions of Theorem 7 (that are also satisfied by Theorem 5) we have w.h.p. that for all z (t)

such that l(z (t) , z) ⩽ τ (0) , n i=1 max b∈[K]\z i (F (t) ib ) 2 ∆ 2 (z i , b)l(z, z (t) ) ⩽ δ 2 256 .
Proof. We need to upper-bound of

F = max {z (t) :l(z,z (t) )⩽τ (0) } n i=1 max b∈[K]\z i (F (t) ib ) 2 ∆(z i , b) 2 l(z, z (t) )
.

To this end, we can decompose

F (t) ib = F 1,(t) ib + F 2,(t) ib
where

F 1,(t) ib = ⟨ϵ i , ( μa -µ (t) a ) -( μb -µ (t) b )⟩
is the error arising from the GMM part of the model and

F 2,(t) ib = λ (t) ⟨E i: W (t) , ( Πa: -Π (t) a: ) -( Πb: -Π (t) b: )⟩ + λ (t) ⟨E i: (W -W (t) ), Πa: -Πb: ⟩ + (λ -λ (t) )⟨E i: W, Πa: -Πb: ⟩
is the error coming from the SBM part of the model. We have

F ⩽ 2 max {z (t) :l(z,z (t) )⩽τ (0) } n i=1 max b∈[K]\z i (F 1,(t) ib ) 2 ∆(z i , b) 2 l(z, z (t) ) F (t) 1 + 2 max {z (t) :l(z,z (t) )⩽τ (0) } n i=1 max b∈[K]\z i (F 2,(t) ib ) 2 ∆(z i , b) 2 l(z, z (t) ) F (t) 2
and it is sufficient to individually control each term. control of F 1 . We follow the same steps as in [START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF], the only difference is that we use a different definition for ∆. To begin with,

F (t) 1 ⩽ n i=1 b∈[K]\z i ⟨ϵ i , ( μz i -µ (t) z i ) -( μb -µ (t) b )⟩ 2 ∆(z i , b) 2 l(z, z (t) ) ⩽ b∈[K] a∈[K]\b i∈C a ⟨ϵ i , ( μa -µ (t) a ) -( μb -µ (t) b )⟩ 2 ∆(a, b) 2 l(z, z (t) ) ⩽ b∈[K] a∈[K]\b || i∈C a ϵ i ϵ ⊤ i || ||( μa -µ (t) a ) -( μb -µ (t) b )|| 2 ∆(a, b) 2 l(z, z (t) )
.

We first need to control || i∈C a ϵ i ϵ ⊤ i || which can be done using the following lemma.

Lemma 3. Let ϵ i i.i.d ∼ N(0, I d ). With probability at least 1 -exp(-0.5n), we have || i∈[n] ϵ i ϵ ⊤ i || ≲ n + d.
Proof. See Lemma A.2 in [START_REF] Lu | Statistical and Computational Guarantees of Lloyd's Algorithm and its Variants[END_REF].

Next, we need to control || μa -µ

(t)
a || 2 for all a ∈ [K], this can be done with the following lemma.

Lemma 4. Under the assumptions of Theorem 5, the following holds with

probability at least 1 -n -Ω(1) . If z (t) satisfies l(z (t) , z) ⩽ τ (0) = ϵn∆ 2 min K then it implies 1. max k∈[K] || μk -µ k || ≲ K(d+log n) n , 2. max k∈[K] ||E(X) ⊤ (W (t) :k -W :k )|| ≲ K n∆ min l(z (t) , z), 3. max k∈[K] ||(X -E(X)) ⊤ W (t) :k || ≲ K (d + n)l(z (t) , z) n∆ min + K K(d + log n)l(z (t) , z) n √ n∆ 2 min , 4. || μk -µ (t) k || ⩽ C 3 K √ (d+n)l(z (t) ,z) n∆ min .
Proof. Straightforward adaptation the proof of Lemma 4.1 in [START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF].

By combining the different bounds, we can now conclude that with high probability, max

{z (t) :l(z (t) ,z)⩽τ (0) } F (t) 1 ≲ K 2 (Kd/n + 1) ∆ 2 min 1 + K(d/n + 1) ∆ 2 min .
This quantity goes to zero when ∆ 2 min /K 3 → +∞.

control of

F (t)
2 . Here we can not directly apply the framework developed by [START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF] Different changes are necessary and we need to deal with additional dependencies.

Let b ∈ [K] ̸ = z j , we then have the bound

(F 2,(t) ib ) 2 ⩽ 3(λ (t) ⟨E i: W (t) , ( Πa: -Π (t) a: ) -( Πb: -Π (t) b: )⟩) 2 + 3(λ (t) ⟨E i: (W -W (t) ), Πa: -Πb: ⟩) 2 + 3(λ -λ (t) ) 2 ⟨E i: W, Πa: -Πb: ⟩ 2 = F 2 21 + F 2 22 + F 2 33 .
We drop the superscript (t) in the notation for the terms F 21 , F 22 and F 23 for convenience, but clearly they depend on t as well. We will now bound each of the terms F 2i for i = 1 . . . 3 separately. Starting with F 21 , first note that

|⟨E i: W (t) , ( Πa: -Π (t) a: ) -( Πb: -Π (t) b: )⟩| 2 ⩽ 4||E i: W (t) || 2 max k || Πk: -Π (t) k: || 2 .
With high probability, for all z (t) such that l(z (t) , z) ⩽ τ (0) we have max

k∈[K] || Πk: -Π (t) k: || 2 ⩽ 2 max k∈[K] ||(W :k -W (t) :k ) ⊤ AW|| 2 + 2 max k∈[K] ||W (t)⊤ :k A(W -W (t) )|| 2 ≲ K 2 √ p max l(z (t) , z) n 1.5 ∆ min 2 (by Lemma 11) ≲ K 4 p max l(z (t) , z) 2 n 3 ∆ 2 min and since λ (t) ≲ λ (by Lemma 13) it follows that n i=1 max b∈[K]\z i F 2 21 ∆ 2 (z i , b)l(z, z (t) ) ≲ λ 2 i ||E i: W (t) || 2 K 4 p max l(z (t) , z) 2 n 3 ∆ 2 min ≲ λ 2 ||EW (t) || 2 F K 4 p max l(z (t) , z) n 3 ∆ 4 min ≲ λ 2 K||EW (t) || 2 K 4 p max l(z (t) , z) n 3 ∆ 4 min ≲ λ 2 K 2 p max K 4 p max l(z (t) , z) n 3 ∆ 4 min
(by a consequence of Lemma 11, fifth item)

≲ K 4 l(z (t) , z) n∆ 4 min ( since λ ≲ n Kp max ) ≲ K 3 ∆ 2 min → 0
where we used the fact Kl(z (t) ,z) n∆ 2 min ⩽ ϵ for the last line. Indeed, Kl(z (t) ,z)

n∆ 2 min ⩽ Kτ (0) n∆ 2 min = ϵ. Let us define ∆ 2 2 (a, b) := ∥Π a: -Π b: ∥ 2 for a, b ∈ [K]. Since F 2 22 ≲ λ 2 ||E i: (W -W (t) )|| 2 || Πz i : -Πb: || 2 ≲ λ 2 ||E i: (W -W (t) )|| 2 ||Π z i : -Π b: || 2
hence we have w.h.p. for all z (t) such that l(z

(t) , z) ⩽ τ (0) n i=1 max b∈[K]\z i F 2 22 ∆(z i , b) 2 l(z, z (t) ) ≲ λ i ||E i: (W -W (t) )|| 2 1 l(z (t) , z) max b∈[K]\z i λ||Π z i : -Π b: || 2 ∆(z i , b) 2 ≲ ||E(W -W (t) )|| 2 F λ l(z (t) , z) (because ∆ 2 (z i , b) ⩾ λ∆ 2 2 (z i , b)) ≲ K||E(W -W (t) )|| 2 λ l(z (t) , z) ≲ λKnp max K 3 l(z (t) , z) n 3 ∆ 4 min ( by Lemma 11) ≲ K 3 ∆ 2 min → 0.
Using the same proof technique as in Lemma 1

(⟨E i: W, Πa: -Πb: ⟩) 2 ≲ Kp max ∆ 2 2 (a, b)
holds w.h.p. Since by Lemma 13 we have w.h.p. that for all z (t) such that l(z (t) , z) ⩽ τ (0)

|λ (t) -λ| ≲ λ K 2 l(z (t) , z) √ np max n∆ min , we obtain n i=1 max b∈[K]\z i F 2 23 ∆(z i , b) 2 l(z, z (t) ) ≲ λ 2 K 2 l(z (t) , z) √ np max n∆ min 2 i ||E i: W|| 2 max b∈[K]\z i || Πz i : -Πb: || 2 ∆ 2 (z i , b)l(z (t) , z) ≲ λ K 4 l(z (t) , z) n 3 p max ∆ 2 min ||EW|| 2 F max b∈[K]\z i λ||Π z i : -Π b: || 2 ∆ 2 (z i , b) ≲ n Kp max K 4 l(z (t) , z) n 3 p max ∆ 2 min Kp max (by Lemma 35) ≲ K 4 l(z (t) , z) n 2 p max ∆ 2 min ≲ K 3 np max ≲ K 3 ∆ 2 min → 0.
Consequently, we have established that Condition C2 holds for all δ = o(1) such that δ 2 = ω(K 3 /∆ 2 min ).

Error term G (t) ib

In this section we are going to show that the G -error term satisfied condition C3.

Lemma 5. Under the assumptions of Theorem 7 (that are also satisfied by Theorem 5) we have w.h.p. for all z (t) such that l(z

(t) , z) ⩽ τ (t) max i∈[n] max b∈[K]\z i |G (t) ib | ∆(z i , b) 2 ⩽ δ (t+1)
8 .

Proof. As for

F (t) ib we can split G (t) ib = G 1,(t) ib + G 2,(t) ib where G 1,(t) ib = ||µ a -µ (t) a || 2 -||µ a -μa || 2 ) -(||µ a -µ (t) b || 2 -||µ a -μb || 2 ) G 2,(t) ib = λ (t) (||P i: W (t) -Π (t) a: || 2 -||P i: W (t) -W a: AW (t) || 2 ) -λ (t) (||P i: W (t) -Π (t) b: || 2 -||P i: W (t) -W b: AW (t) || 2 ).
By the proof of Lemma 4.1 in Gao et al., 2019 (last inequality of page 46, equations ( 115) and ( 118)), we have w.h.p. for all z (t) t) , ϵδ (using τ = τ (0) δ and also the definition of δ (t) )

such that l(z (t) , z) ⩽ τ (t) , |G 1,(t) ib | ∆ 2 (a, b) ≲   Kl(z (t) , z) n∆ min + K Kl(z (t) , z) n∆ min   2 ∆ -2 min +   Kl(z (t) , z) n∆ min + K Kl(z (t) , z) n∆ min   Kl(z (t) , z) n∆ min ∆ -2 min +   Kl(z (t) , z) n∆ min + K Kl(z (t) , z) n∆ min   ∆ -1 min ≲ Kl(z (t) , z) 2 n 2 ∆ 4 min + K l(z (t) , z) n∆ 2 min ≲ K l(z (t) , z) n∆ 2 min (since K l(z (t) ,z) n∆ 2 min < 1 and Kl(z (t) ,z) 2 n 2 ∆ 4 min < 1 K ) ≲ Kτ (t) n∆ 2 min ≲ max 7 8 Kτ (t-1) n∆ 2 min , Kτ n∆ 2 min (by definition of τ (t) ) ≲ max 7 8 ( 8 7 ϵ)δ (
≲ ϵδ (t) .

(since δ ⩽ δ (t) )
Now by choosing ϵ to be a suitably small constant (< 1), since δ (t+1) ⩽

7 8 δ (t) we obtain |G 1,(t) ib | ∆ 2 (a, b) ⩽ δ (t+1) 16 .
To bound G

2,(t) ib

we will adapt the method developed in Han et al., 2020. We have by direct calculation

G 2,(t) ib λ (t) = (||Π (t) a: -W ⊤ a: AW (t) || 2 -||Π (t) b: -W b: AW (t) || 2 ) + 2⟨P i: W (t) -W a: AW (t) , W ⊤ :a AW (t) -Π (t) a: ⟩ -2⟨P i: W (t) -W b: AW (t) , W ⊤ :b AW (t) -Π (t) b: ⟩ ⩽ ||Π (t) a: -W ⊤ :a AW (t) || 2 -||Π (t) b: -W ⊤ :b AW (t) || 2 + 4 max a∈[K] ⟨W ⊤ :a EW (t) , (W :b -W (t) :b ) ⊤ AW (t) ⟩ + 2 ⟨(Π a: -Π b: )Z ⊤ W (t) , (W :b -W (t) :b ) ⊤ AW (t) ⟩ =: G 21 + G 22 + G 23 .
We drop the superscript (t) in the notation for the terms G 21 , G 22 and G 23 for convenience, but clearly they depend on t as well. First observe that w.h.p, it holds for all z (t)

such that l(z (t) , z) ⩽ τ (t) that G 21 ⩽ max a∈[K] ||Π (t) a: -W ⊤ :a AW (t) || 2 = max a∈[K] ||(W (t) a: -W :a ) ⊤ AW (t) || 2 ≲ K 3 p max l(z (t) , z) 2 n 3 ∆ 2 min
where we used Lemma 11 for the last inequality. This implies by Lemma 12 that

max b∈[K]\z i λ (t) G 21 ∆ 2 (z i , b) ≲ K 2 l(z (t) , z) 2 n 2 ∆ 4 min ⩽ K 2 (τ (0) δ (t) ) 2 n 2 ∆ 4 min which implies max b∈[K]\z i λ (t) G 21 ∆ 2 (z i ,b) ⩽ ( δ (t+1) 16
) 2 using the same argument used earlier (for suitably small constant ϵ < 1). Next, in order to bound G 22 , note that w.h.p, it holds for all z (t) such that l(z (t) , z) ⩽ τ (t) that

G 22 ⩽ 4 max a∈[K] ||W ⊤ :a EW (t) || max a∈[K] ||(W :b -W (t) :b ) ⊤ AW (t) || ≲ K √ p max √ n K 1.5 √ p max l(z (t) , z) n 1.5 ∆ min (by Lemma 11) ≲ K 2.5 p max n l(z (t) , z) n∆ min which in turn implies max b∈[K]\z i λ (t) G 22 ∆(z i , b) 2 ≲ √ K ∆ min Kl(z (t) , z) n∆ 2 min (since λ (t) ≲ n/(Kp max )) = o(δ) as √ K/∆ min → 0 and K l(z (t) ,z) n∆ 2 min < 1.
Finally, in order to bound G 23 , note that w.h.p, it holds for all z (t) such that l(z (t) , z) ⩽ τ (t) ,

G 23 ≲ ||Π a: -Π b: )Z ⊤ W (t) || max b∈[K] ||(W :b -W (t) :b ) ⊤ AW (t) || ≲ ∆ 2 (a, b) K 1.5 √ p max l(z (t) , z) n 1.5 ∆ min which implies max b∈[K]\z i λ (t) G 23 ∆(z i , b) 2 ≲ λ ∆ 2 (z i , b) ∆(z i , b) K 1.5 √ p max l(z (t) , z) n 1.5 ∆ 2 min ≲ √ λ √ Kp max √ n Kl(z (t) , z) n∆ 2 min ≲ Kl(z (t) , z) n∆ 2 min . (since λ ≲ n Kp max )
For a suitably constant ϵ < 1, this then implies

max b∈[K]\z i λ (t) G 23 ∆(z i , b) 2 ⩽ δ (t+1)
16 .

Error term H (t) ib

In this section we are going to show that the H -error term satisfied condition C3.

Lemma 6. Under the assumptions of Theorem 7 (that are also satisfied by Theorem 5) we have w.h.p. that for all z (t) such that l(z (t) , z) ⩽ τ (t) ,

max i∈[n] max b∈[K]\z i |H (t) ib | ∆(z i , b) 2 ⩽ δ (t+1)
8 .

Proof. As before, we can split

H (t) ib = H 1,(t) ib + H 2,(t) ib where 2H 1,(t) ib = ||µ a -μa || 2 -||µ a -μb || 2 + ||µ a -µ b || 2 2H 2,(t) ib = λ (t) (||P i: W (t) -W ⊤ a: AW (t) || 2 -||P i: W (t) -W ⊤ b: AW (t) || 2 + ||Π a: -Π b: || 2 ) + (λ -λ (t) )||Π a: -Π b: || 2 .
By an immediate adaptation of Lemma 4.1 in Gao et al., 2019 it holds w.h.p. that for all z (t)

such that l(z (t) , z) ⩽ τ (t) , |H 1,(t) ib | ∆(z i , b) 2 ≲ K(d + log n) n∆ 2 min + K(d + log n) n∆ 2 min → 0 as long as K/∆ 2 min → 0. It remains to uniformly control H 2,(t) ib , let us split it 4 as H 2,(t) ib = λ (t) H 1 + H 2 .
First note that by Lemma 13 it holds w.h.p. that for all z (t) such that l(z (t) , z) ⩽ τ (t) ,

H 2 := (λ -λ (t) )||Π a: -Π b: || 2 ≲ λ Kl(z (t) , z) n∆ 2 min ∆ 2 2 (a, b) which implies |H 2 | ∆(z i , b) 2 ≲ Kl(z (t) , z) n∆ 2 min . Then, we obtain |H 2 | ∆(z i ,b) 2 ⩽ δ (t+1)
16 for a small enough constant ϵ < 1. Now observe that

H 1 : = ||P i: W (t) -W ⊤ a: AW (t) || 2 -||P i: W (t) -W ⊤ b: AW (t) || 2 + ||Π a: -Π b: || 2 = ||W ⊤ a: EW (t) || 2 + (||Π a: -Π b: || 2 -||P i: W (t) -W ⊤ b: PW (t) || 2 ) -(||P i: W (t) -W ⊤ b: AW (t) || 2 -||P i: W (t) -W ⊤ b: PW (t) || 2 ) = (||Π a: -Π b: || 2 -||P i: W (t) -W ⊤ b: PW (t) || 2 ) + (||W ⊤ a: EW (t) || 2 -||W ⊤ b: EW (t) || 2 ) + 2⟨P i: W (t) -W ⊤ b: PW (t) , W ⊤ :b EW (t) ⟩ = H (t) 11 + H (t) 12 + H (t) 13 .
By writing

P i: W (t) -W ⊤ b: PW (t) = (Π a: -Π b: )Z ⊤ W (t)
, we obtain w.h.p. that for all z (t) such that l(z (t) , z) ⩽ τ (t) ,

|H (t) 13 | ≲ ||P i: W (t) -W ⊤ b: PW (t) ||||W ⊤ :b EW (t) || ≲ ||Π a: -Π b: || K √ p max √ n ≲ ||Π a: -Π b: || 2 K √ np max .
In particular,

λ (t) |H (t) 13 | ∆ 2 (z i , b) ≲ λ∆ 2 2 (z i , b)K ∆ 2 (z i , b) √ np max ≲ K √ np max → 0.
Next observe that w.h.p., it holds for all z (t) such that l(z

(t) , z) ⩽ τ (t) that ||W ⊤ a: EW (t) || 2 -||W ⊤ b: EW (t) || 2 ⩽ max k∈[K] ||W ⊤ k: EW (t) || 2 ≲ K 2 p max n
where the last inequality uses Lemma 11. This implies

λ (t) |H (t) 12 | ∆ 2 (z i , b) ≲ K ∆ 2 min → 0.
|H (t) 11 | := ||Π a: -Π b: || 2 -||P i: W (t) -W ⊤ b: PW (t) || 2 = ||Π a: -Π b: || 2 -||(Π a: -Π b: )Z ⊤ W (t) || 2 .
Using the fact

||Π a: -Π b: || 2 -||(Π a: -Π b: )Z ⊤ W (t) || 2 ⩽ Z ⊤ W (t) -I 2 + 2 Z ⊤ W (t) -I ∥Π a: -Π b: ∥ 2 ,
we obtain by the proof of part 1 of Lemma 11 that w.h.p., it holds for all z (t) such that l(z

(t) , z) ⩽ τ (t) |H (t) 11 | ≲ ||Π a: -Π b: || 2 Kl(z (t) , z) n∆ 2 min which implies λ (t) |H (t) 11 | ∆ 2 (z i , b) ≲ Kl(z (t) , z) n∆ 2 min .
Then as before, for a suitably small constant ϵ < 1, this implies

λ (t) |H (t) 11 | ∆ 2 (z i , b) ⩽ δ (t+1) 16 .
By summing all these inequalities we see that

G (t)
ib and

H (t)
ib satisfy Condition C3.

Conclusion

By Lemma 2, 5 and 6 the Conditions C2 and C3 are satisfied. In order to apply Theorem 4, we also need to show that Condition C1 is satisfied. To this end, let us define the events

Ω(δ (t) ) := {C i (a, b) ⩽ - 1 -δ (t) 2 ∆ 2 (a, b)}, t ⩾ 1,
where we recall that

C i (a, b) = ⟨ϵ i , μa -μb ⟩ + λ⟨E i: W, Πa: -Πb: ⟩.
Note that by definition of δ (t) , we have for all t ⩾ 1

Ω(δ (t) ) ⊆ Ω(δ (1) ) =⇒ 1 Ω(δ (t) ) ⩽ 1 Ω(δ (1) ) =⇒ ξ(δ (t) ) ⩽ ξ(δ (1)
).

(3.7.1) Hence it suffices to bound ξ(δ (1) ). To achieve this, it is crucial to bound P(Ω(δ (1) )) since the bound on ξ(δ (1) ) then follows via an easy adaptation of Lemma 1.

We can bound P(Ω(δ (1) )) by an immediate adaptation of Lemma 7 -we just need to replace ∆ 2 /4 by (1 -δ (1) )∆ 2 /4 in the last step of the lemma, leading to

P(Ω(δ (1) )) ⩽ exp(-(1 -δ (1) + o(1)) ∆2 ).
Then as mentioned earlier, an easy adaptation of Lemma 1 along with (3.7.1) implies that w.h.p., it holds for all t ⩾ 1 that

ξ(δ (t) ) ⩽ n exp(-(1 -δ (1) + o(1)) ∆2 ).
But we know that

n exp(-(1 -δ (1) + o(1)) ∆2 ) ⩽ 3 4 τ = 3 4 τ (0) δ
for n large enough, by a suitable choice of δ = o(1), and the fact that ∆2 ≍ log n/K → ∞. Hence the assumptions of Theorem 4 are satisfied. Moreover, for all t ≳ log(1/δ), Corollary 1 yields the bound

l(z (t) , z) ≲ ξ(δ) + τ (0) (1/8) t-Θ(log(1/δ)) ≲ ne -(1+o(1)) ∆2 + τ (0) (1/8) t-Θ(log(1/δ)) .
Since τ (0) ≲ n log n/K and δ can be chosen such that log(1/δ) ≲ log n, we obtain that for t ≳ log n l(z (t) , z) ⩽ ne -(1+o(1)) ∆2 .

Proof of Theorem 6

Proof. The proof follows the same steps as in the proof of Theorem 3.3 in [START_REF] Lu | Statistical and Computational Guarantees of Lloyd's Algorithm and its Variants[END_REF], only the last part needs to be changed. For the sake of completeness, we reproduce the arguments below. Let us denote

h(z ′ , z ′′ ) = i∈[n] 1 {z ′ (i)̸ =z ′′ (i)} (3.7.2) to be the unnormalized Hamming distance between z ′ , z ′′ ∈ [K] n .
Without lost of generality we can assume that

min k,k ′ ||µ k -µ k ′ || = ||µ 1 -µ 2 ||. For each k ∈ [K], let T k a subset of C k with cardinality 3n 4K . Define T = ∪ K k=1 T k and Z = {ẑ : ẑi = z i for all i ∈ T }.
For all ẑ ̸ = z ∈ Z we have

h(ẑ, z) n ⩽ 1 4
and for all permutations σ ∈ S K , σ ̸ = Id ( where Id denotes the identity permutation) we have

h(σ(ẑ), z) n ⩾ 1 2 .
Thus r(ẑ, z) = h( ẑ, z) n . Then following the same arguments as in the proof of Theorem 2 in [START_REF] Gao | Community detection in degree-corrected block models[END_REF] we can obtain

inf ẑ sup θ∈Θ E(r(ẑ, z)) ⩾ 1 6|T c | i∈T c 1 2K 2 inf ẑi (P 1 (ẑ i = 2) + P 2 (ẑ i = 1))
(3.7.3) where P k denotes the probability distribution of the data when z i = k. By the Neyman Pearson Lemma, the infinimum of the right hand side of (3.7.3) is achieved by the likelihood ratio test. From Section 3.1 in Zhang et al., 2016a, the log-likelihood of the SBM part can be rewritten as

log p(1 -q) q(1 -p) i<j A ij 1 {z i =z j } + f(A)
where f(A) doesn't depend on z. Consequently,

1 2 inf ẑi (P 1 (ẑ i = 2) + P 2 (ẑ i = 1)) = P(-0.5||ϵ i || 2 + log p(1 -q) q(1 -p) j∈C 1 A ij ⩽ -0.5||µ 1 + ϵ i -µ 2 || 2 + log p(1 -q) q(1 -p) j∈C 2 A ij ) (3.7.4) Let us denote Z i = log( p(1-q) q(1-p) )( j∈C 2 A ij -j∈C 1 A ij ), this is a random variable independent of ϵ i . So we get (3.7.4) = P(0.5||µ 1 -µ 2 || 2 -Z i ⩽ -⟨ϵ i , µ 1 -µ 2 ⟩) ⩾ P(0.5||µ 1 -µ 2 || 2 -Z i ⩽ -⟨ϵ i , µ 1 -µ 2 ⟩ | Z i > 0)P(Z i > 0) ⩾ P(||µ 1 -µ 2 || 2 ⩽ -2⟨ϵ i , µ 1 -µ 2 ⟩)P(Z > 0) ⩾ exp - ∆ 2 1 8 exp -n (1 + o(1))( √ p - √ q) 2 K ⩾ exp(-(1 + o(1)) ∆2 ).
Here we used for the penultimate inequality a result from the proof of Theorem 3.3 in Lu et al., 2016 and also use Lemma 5.2 in Zhang et al., 2016a.

Proof of technical lemmas

Lemma 7 (Concentration rate for the ideal oracle error under CSSBM). Recall that

Ω 1 = ⟨ϵ i , µ z i -µ b ⟩ + λ⟨E i: W, Π z i : -Π b: ⟩ ⩽ -(1 -δ -δ)∆ 2 (z i , b) 2
and suppose that δ, δ = o(1). Then under the assumptions of Theorem 5, we have

P(Ω 1 ) ⩽ exp(-(1 + o(1)) ∆2 ) where ∆2 = 1 8 min k,k ′ ||µ k -µ k ′ || 2 + log n K ( p ′ -q ′ ) 2 .
Proof. We are going to bound the m.g.f of

Z = ⟨ϵ i , µ z i -µ b ⟩ + λ⟨E i: W, Π z i : -Π b: ⟩
and use Chernoff method. We have for all t < 0

log Ee tZ ⩽ log Ee t⟨ϵ i ,µ z i -µ b ⟩ + log Ee tλ⟨E i: W,Π z i : -Π b: ⟩ (by independence) ⩽ ||µ z i -µ b || 2 t 2 2 + n K log(pe tλ(p-q)K/n + 1 -p) + n K log(qe -tλ(p-q)K/n + 1 -q) -tλ(p -q) 2 .
By choosing t = -1/2 we get e tλ(p-q)K/n = q(1 -p) p(1 -q)

and thus log(pe tλ(p-q)K/n + 1 -p)(qe -tλ(p-q)K/n + 1 -q)

= log(pq

+ (1 -p)(1 -q) + 2 √ pq (1 -p)(1 -q)). log(pe tλ(p-q)K/n + 1 -p)(qe -tλ(p-q)K/n + 1 -q) = log(pq + (1 -p)(1 -q) + 2 √ pq (1 -p)(1 -q)).
This last quantity is equal to

-(1 + o(1))( √ p - √ q) 2 .
We can now conclude by remarking that

P(Ω 1 ) ⩽ P - 1 2 ⟨ϵ i , µ z i -µ b ⟩ - 1 2 λ⟨E i: W, Π z i : -Π b: ⟩ ⩾ (1 + o(1)) ∆ 2 min 4 hence P(Ω 1 ) ⩽ Ee -Z 2 - ∆ 2 min 4 ⩽ exp( ||µ z i -µ b || 2 8 - n K (1 + o(1))( √ p - √ q) 2 + λ 2 (p -q) 2 -(1 + o(1)) ∆ 2 min 4 ) ⩽ exp(-(1 + o(1)) ∆2 ), since ∆ 2 min = ||µ z i -µ b || 2 + 2λ(p -q) 2 .
Lemma 8. Recall the definition of Ω 1 from the previous lemma. Under the assumptions of Theorem 7 we have

P(Ω 1 ) ⩽ exp - 1 8 ∆ 2 min .
Proof. First observe that

tλ⟨E i: W, Π z i : -Π b: ⟩ = tλ k∈[K] (Π z i k -Π bk ) j∈C k E ij n k = k∈[K] tλ(Π z i k -Π bk ) j∈C k A ij -Π z i k n k .
The sum over k involves independent random variables so in order to bound the m.g.f. of

⟨E i: W, Π z i : -Π b: ⟩ it is sufficient to control the m.g.f. of j∈C k A ij -Π z i k for each k. Setting t ′ = λt |Π z i k -Π bk | n k , we have log E(e t ′ j∈C k (A ij -Π z i k ) ) = log(Π z i k e t ′ + 1 -Π z i k ) -n k t ′ Π z i k ⩽ n k Π z i k (e t ′ -t ′ -1) ⩽ n k Π z i k e(t ′ ) 2 2 ( by Taylor-Lagrange inequality) ⩽ 1.5n k p max (λt Π z i k -Π bk n k ) 2 ⩽ 1.5λ|Π z i k -Π bk | 2 t 2 .
For the second inequality we used the fact that for 0

< x < 1, log(1 - x) ⩽ -x. Consequently, log Ee tZ ⩽ ||µ z i -µ b || 2 t 2 2 + 1.5λ||Π z i : -Π b: || 2 t 2 and P(Ω 1 ) ⩽ e ||µ z i -µ b || 2 t 2 2 +1.5λ||Π z i : -Π b: || 2 t 2 - ∆ 2 (z i ,b) 4
.

For t = 1/2 we then obtain

P(Ω 1 ) ⩽ e - ||µz i -µ b || 2 8 -λ 8 ||Π z i : -Π b: || 2 ⩽ e -1 8 ∆ 2 min .
Recall the Hamming distance h defined in (3.7.2).

Lemma 9. For all z, z ′ ∈ [K] n we have h(z, z ′ ) ⩽ l(z, z ′ ) ∆ 2 min . Proof. i∈[n] 1 z i ̸ =z ′ i ⩽ i∈[n] ∆(z i , z ′ i ) 2 ∆ 2 min 1 z i ̸ =z ′ i = l(z, z ′ ) ∆ 2 min . Lemma 10. Assume that for some α > 1 n αK ⩽ n k ⩽ αn K . If l(z, z (t) ) ⩽ n∆ 2 min /(2αK) then for all k ∈ [K] n 2αK ⩽ n (t) k ⩽ 2αn K . Proof. Since for all k ∈ [K] we have n/(αK) ⩽ n k ⩽ αn/K, i∈C (t) k 1 ⩾ i∈C k ∩C (t) k 1 ⩾ i∈C k 1 - i∈[n] 1 z i ̸ =z (t) i ⩾ n αK -h(z, z (t) ) Lemma 9 ⩾ α n K - l(z, z (t) ) ∆ 2 min ⩾ αn 2K
by assumption. The other inequality is proved in a similar way.

Lemma 11. Assume that A ∼ SBM(Z, Π) with equal size communities and suppose that the conditions of Theorem 7 are satisfied 5 . Then with probability at least 1 -n -Ω(1) the following holds for all z (t) such that l(z (t) , z) ⩽ τ (0) .

1. max k∈[K] ||W (t) :k -W :k || ≲ K 1.5 n 1.5 ∆ 2 min l(z (t) , z), 2. max k∈[K] ||(W (t) :k -W :k ) ⊤ AW|| ≲ K 1.5 √ p max n 1.5 ∆ min l(z, z (t) ), 3. max k∈[K] ||W (t)⊤ :k A(W -W (t) )|| ≲ K 2 √ p max l(z (t) ,z) n 1.5 ∆ min , 4. max k∈[K] ||(W (t) :k -W :k ) ⊤ AW (t) || ≲ K 1.5 √ p max l(z (t) ,z) n 1.5 ∆ min , 5. ||Z ⊤ W (t) || ≲ 1.
Proof. This is a rather straightforward adaptation of Lemma 4 in Han et al., 2020, but for completeness we include a proof adapted to our setting with our notations.

proof of 1. First observe that Z is rank K and λ K (Z) = √ n min so that ||W (t) :k -W :k || ⩽ n -1/2 min ||I -Z ⊤ W (t) ||. For any k ∈ [K], denote δ k = 1 -(Z ⊤ W (t) ) kk . Since for all k, k ′ ∈ [K] (Z ⊤ W (t) ) kk ′ = i∈C k 1 z (t) i =k ′ n (t) k ′ , we have 0 ⩽ δ k ⩽ 1, k ′ ∈[K]\k (Z ⊤ W (t) ) k ′ k = δ k .
Therefore,

||Z ⊤ W (t) -I|| = k∈[K]   δ 2 k + k ′ ∈[K]\k (Z ⊤ W (t) ) 2 k ′ k   ⩽ k∈[K]   δ 2 k +   k ′ ∈[K]\k (Z ⊤ W (t) ) k ′ k   2    ⩽ 2 k∈[K] δ 2 k ⩽ √ 2 k∈[K] δ k = √ 2 k∈[K] i∈C (t) k 1 z i ̸ =k n (t) k ⩽ √ 2 max k (n (t) k ) -1 i∈[n] 1 z i ̸ =z (t)
i 5 These assumptions are clearly satisfied by Theorem 5 as well.

Lemma 10

≲ K n h(z, z (t) ) Lemma 9 ≲ K l(z, z (t) ) n∆ 2 min .
(3.7.5) proof of 2. Observe that with probability at least 1 -n -Ω(1) we have max

k∈[K] ||(W (t) :k -W :k ) ⊤ AW|| ⩽ max k∈[K] ||(W (t) :k -W :k ) ⊤ PW|| + max k∈[K] ||(W (t) :k -W :k ) ⊤ EW|| ⩽ max k∈[K] ||(W (t) :k -W :k ) ⊤ ZΠ|| + ||EW|| max k∈[K] ||(W (t) :k -W :k )|| ⩽ ||Π b: - j∈C (t) b Π z j : n (t) b || + C Kp max max k∈[K] ||(W (t) :k -W :k )|| ≲ ||Π b: - j∈C (t) b Π z j : n (t) b || + K 2 √ p max n 1.5 ∆ 2 min l(z (t) , z). Recall that ∆ 2 2 (a, b) := ∥Π a: -Π b: ∥ 2 . Then we have Π b: - j∈C (t) b Π z j : n (t) b = j∈C (t) b b ′ ∈[K]\b 1 {z j =b ′ } n (t) b (Π b: -Π b ′ : ) ⩽ C K n j∈C (t) b b ′ ∈[K]\b max b,b ′ ∆ 2 (b, b ′ )1 {z j =b ′ } ⩽ C K n max b,b ′ ∆ 2 (b, b ′ )h(t, t (t) ) ⩽ C K∆ min √ λn∆ 2 min l(z, z (t) ) ( since max b,b ′ ∆ 2 (b, b ′ ) ≲ ∆ min √ λ for SBM) ⩽ C K 1.5 √ p max n 1.5 ∆ min l(z, z (t) ).
Consequently, by summing the previous bounds and using the first inequality of the Lemma we get max

k∈[K] ||(W (t) :k -W :k ) ⊤ AW|| ≲ K 1.5 √ p max n 1.5 ∆ min l(z, z (t) ) + K 2 √ p max n 1.5 ∆ 2 min l(z (t) , z).
In our setting ∆ 2 min ≍ log n so the first term is dominant.

proof of 3. First let's bound max k∈[K] ||W (t)⊤ :k P(W -W (t) )||. By Lemma 10 we have ||W (t) k || ≲ K/n, so max k∈[K] ||W (t)⊤ :k P(W -W (t) )|| ⩽ max k∈[K] ||W (t)⊤ :k Z||||ΠZ ⊤ (W -W (t) )|| ≲ ||ΠZ ⊤ (W -W (t) )|| F ≲ √ K max k∈[K] ||(W (t) :k -W :k ) ⊤ ZΠ|| ≲ K 2 √ p max n 1.5 ∆ min l(z, z (t) ).
(by the proof of part 2)

We now give an upper bound for max k∈

[K] ||W (t)⊤ :k E(W -W (t) )||. By triangle inequality, ||W (t)⊤ :k E(W -W (t) )|| ⩽ ||W ⊤ :k E(W -W (t) )|| + ||(W (t) :k -W k: ) ⊤ E(W -W (t) )||.
First we have

||W ⊤ :k E(W -W (t) )|| ⩽ ||W :k ||||E||||(W -W (t) )|| ≲ K 2 √ p max n 1.5 ∆ 2 min l(z (t) , z).
On the other hand, we also have

||(W (t) :k -W k: ) ⊤ E(W -W (t) )|| ⩽ ||W :k -W (t) :k ||||E(W -W (t) )|| ⩽ ||E|| √ K max k ||W :k -W (t) :k || 2 ≲ K 3.5 √ np max l(z (t) , z) n 2 ∆ 2 min l(z (t) , z) n∆ 2 min ≲ K 3.5 √ p max l(z (t) , z) n 1.5 ∆ 2 min
where the last inequality comes from the fact that by assumption

l(z, z (t) ) ⩽ τ ⩽ ϵ n∆ 2 min K . Thus it follows that max k∈[K] ||W (t)⊤ :k P(W -W (t) )|| ≲ K 2 √ p max l(z (t) , z) n 1.5 ∆ min . proof of 4. First note that ||(W (t) :k -W :k ) ⊤ PW (t) || ⩽ ||(W (t) :k -W :k ) ⊤ ZΠ||||Z ⊤ W (t) || ≲ K 1.5 √ p max l(z (t) , z) n 1.5 ∆ min .
Furthermore, by the same argument as before,

||(W (t) :k -W :k ) ⊤ EW (t) || ⩽ ||(W (t) :k -W :k ) ⊤ E(W (t) -W)|| + ||(W (t) :k -W :k ) ⊤ EW|| ≲ K||E|| max k ||W :k -W (t) :k || 2 + K 2 √ p max n 1.5 ∆ 2 min l(z (t) , z) ≲ K 2 √ p max n 1.5 ∆ 2 min l(z (t) , z).
We obtain the result by triangle inequality.

proof of 5. Since Z ⊤ W = I K we have ||Z ⊤ W (t) || ⩽ 1 + ||Z ⊤ (W (t) -W)|| ≲ 1 + ||I -Z ⊤ W (t) || ≲ 1 + K l(z (t) , z) n∆ 2 min (by Equation (3.7.5)) ≲ 1.
(by assumption on τ (0) )

Lemma 12. For sIR-LS we have with probability at least 1 -n -Ω(1) that for all z (t) such that l(z

(t) , z) ⩽ τ (0) max k∈[K] |n (t) k -n k | ⩽ l(z (t) , z) ∆ 2 min , |λ (t) -λ| ≲ λ Kl(z (t) , z) n∆ 2 min . Proof. First observe that max k∈[K] |n (t) k -n k | = max k∈[K] | i 1 {Z (t) i =k} -1 {Z i =k} | ⩽ h(z (t) , z).
Moreover by Lemma 9 we have h(z (t) , z) ⩽ l(z (t) ,z)

∆ 2 min . Then note that we have |p (t) max -p max | ⩽ max k,k ′ ||(W (t) :k ) ⊤ AW (t) :k ′ -W ⊤ :k PW :k ′ || (the max is 1-Lipschitz) ⩽ max k,k ′ (||(W (t) :k ) ⊤ EW (t) :k ′ || + ||(W (t) :k -W :k ) ⊤ PW (t) :k ′ || + ||W ⊤ :k P(W (t) :k ′ -W :k ′ )||) ≲ max k ||W (t) :k || 2 ||E|| + K 1.5 √ p max l(z (t) , z) n 1.5 ∆ min (by the proof of Lemma 11) ≲ K √ p max √ n + K 1.5 √ p max l(z (t) , z) n 1.5 ∆ min ≲ K 1.5 √ p max l(z (t) , z) n 1.5 ∆ min ≲ K np max Kl(z (t) , z) n∆ min p max ≲ Kl(z (t) , z) n∆ 2 min p max since ∆ min ≍ log n/K ≍ np max /K. Consequently, λ (t) λ -1 ⩽ n (t) min n min p max p (t) max -1 ⩽ n (t) min -n min n min p max p (t) max + p max -p (t) max p (t) max ≲ Kl(z (t) , z) n∆ 2 min + Kl(z (t) , z) n∆ 2 min ≲ Kl(z (t) , z) n∆ 2 min .
Lemma 13. For IR-LSS, we have with probability at least 1 -n -Ω(1) that for all z (t) such that l(z (t) , z) ⩽ τ (0)

|λ (t) -λ| ≲ λ Kl(z (t) , z) n∆ 2 min
Proof. By a similar argument as the one used in Lemma 12 we have with probability at least 1 -n -Ω(1) that

|p (t) -p| ≲ Kl(z (t) , z) n∆ 2 min p, |q (t) -q| ≲ Kl(z (t) , z) n∆ 2 min q.
This implies

p (t) -q (t) p -q = 1 + O (p + q)Kl(z (t) , z) (p -q)n∆ 2 min = 1 + O Kl(z (t) , z) n∆ 2 min because p -q ≳ p. Thus, log p (t) q (t) -log p q = log p (t) p q q (t) = 2 log 1 + O Kl(z (t) , z) n∆ 2 min = O Kl(z (t) , z) n∆ 2 min . Hence log( p (t) q (t) ) log( p q ) -1 = O K 1.5 l(z (t) , z) n∆ 2
min since log(p/q) is bounded above by assumption. Consequently,

λ (t) λ -1 = O Kl(z (t) , z) n∆ 2 min . C L U S T E R I N G M U LT I L AY E R G R A P H S W I T H M I S S I N G N O D E S
In this chapter we consider partial side information given by graphs instead of Gaussian covariates as in the previous chapter. This work is based on Braun et al., 2021b where we propose different extensions of existing clustering methods for multilayer graphs to the setting were each node doesn't appear in all the layers. We then complete this work by showing how the method studied in Chapter 1 can be extended to this setting.

The chapter is organized as follows. Section 4.1 motivates the problem and discusses the related work. The generative model is introduced in Section 4.2. In Section 4.3 we propose a final aggregation method for the missing nodes setting. In Section 4.4 we study an early aggregation method based on the sum of adjacency matrices and in Section 4.5 we propose an extension of an intermediate fusion method to the missing nodes setting. We then explain how the method of Chapter 3 can be applied to this problem in Section 4.7. Finally we perform numerical experiments in Section 4.6. Technical results are gathered in Section 4.8

introduction

Graphs are a powerful tool to represent relationships between agents. Due to applications in a wide array of fields including biology, sociology, ecology and economics (see for e.g., [START_REF] Braun | Dynamic reconfiguration of frontal brain networks during executive cognition in humans[END_REF][START_REF] Han | Consistent Estimation of Dynamic and Multi-Layer Block Models[END_REF][START_REF] Kivelä | Multilayer networks[END_REF][START_REF] Kim | Community Detection in Multi-Layer Graphs: A Survey[END_REF], the analysis of networks has received significant interest over the last two decades. One fundamental problem of network analysis is clustering which involves detecting communities by regrouping nodes having similar connectivity properties. Numerous clustering algorithms have been developed over the years based on different approaches such as modularity maximization, maximum likelihood, random walks, semi-definite programming and spectral clustering (see for instance the survey articles by [START_REF] Fortunato | Community detection in graphs[END_REF][START_REF] Abbe | Community Detection and Stochastic Block Models[END_REF].

Often, relationships are better understood through different modalities. These multiple aspects of relationships can be represented by a multilayer graph where each layer is a graph representing the interactions between agents for one modality. For e.g., social interaction between a set of people can be recorded via email exchanges, phone calls, professional links, and so on. Each level of interaction can be encoded into a simple graph and the collection of these graphs leads to a multilayer representation. Another important example of a multilayer graph is given by a time-varying network where each view of the network at a given time corresponds to a different layer.

Over the last decade, many methods have been proposed for clustering multilayer graphs such as those based on matrix factorization, spectral methods, maximisation of a modularity function or probability model-based approaches; see [START_REF] Kim | Community Detection in Multi-Layer Graphs: A Survey[END_REF] for a survey. Consistency results for the recovery of the partition under a stochastic generative model have also been shown for some algorithms, see for example [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF][START_REF] Pensky | Spectral clustering in the dynamic stochastic block model[END_REF][START_REF] Lei | Bias-Adjusted Spectral Clustering in Multi-Layer Stochastic Block Models[END_REF][START_REF] Bhattacharyya | Spectral Clustering for Multiple Sparse Networks: I[END_REF] Most existing approaches assume that all the layers share the same set of nodes. In practice, however, data are often incomplete; in particular, the set of observed nodes can clearly vary across layers. For example, in social networks evolving over time, the set of nodes can change due to people leaving/joining the network. This is the setting considered in the present paper.

Related work

clustering on multi-layer graphs. As noted by [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF], clustering strategies for multilayer graphs can be roughly categorized into three groups: early fusion methods where all views are aggregated and then clustering is performed, intermediate fusion methods where the algorithm finds a factor common to all the views, and final aggregation methods where each individual view is processed separately and a consensus partition is formed. In the complete setting, different algorithms have been proven to be consistent under a multilayer stochastic block model assumption (see Section 4.2.1). Among them are spectral clustering on the sum of adjacency matrices (e.g., [START_REF] Bhattacharyya | Spectral Clustering for Multiple Sparse Networks: I[END_REF][START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF] or on the sum of squared adjacency matrices with bias correction (e.g., [START_REF] Lei | Bias-Adjusted Spectral Clustering in Multi-Layer Stochastic Block Models[END_REF][START_REF] Bhattacharyya | General Community Detection with Optimal Recovery Conditions for Multirelational Sparse Networks with Dependent Layers[END_REF], orthogonal linked matrix factorization (e.g., [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF], and co-regularized spectral clustering (e.g., [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF]. Existing misclustering bounds for these methods are gathered in the supplementary material.

incomplete multi-view clustering (imvc). Recently a similar problem has been addressed in the context of IMVC, see for example [START_REF] Liu | Efficient and Effective Regularized Incomplete Multi-View Clustering[END_REF][START_REF] Hu | One-Pass Incomplete Multi-View Clustering[END_REF] and references therein. To the best of our knowledge, no consistency results for the recovery of the ground truth clustering structure are shown in this line of work. Algorithms designed for the IMVC framework cannot be directly applied to our setting since they apply to a collection of feature vectors. However they could possibly be adapted, in a non trivial manner, to our framework. For example, in the complete setting, the OMVC method proposed by Hu et al., 2019 can be considered as a variant of the OLMF estimator proposed by [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF] where the optimization problem is modified in order to take into account the symmetry of the inputs. Similarly, if there were no missing views, the algorithm proposed by [START_REF] Liu | Efficient and Effective Regularized Incomplete Multi-View Clustering[END_REF] resembles a variant of the co-regularized spectral clustering method of Paul et al., 2020 for clustering multilayer graphs. We leave the adaptation of the algorithm proposed by [START_REF] Liu | Efficient and Effective Regularized Incomplete Multi-View Clustering[END_REF] to our setting for future work.

Contributions

We consider the problem of clustering multilayer graphs with missing nodes under a Multi-Layer Stochastic Block Model (MLSBM) described in Section 4.2. Our contributions are as follows.

• In Section 4.3.1 we propose a final aggregation method based on a variant of k-means for incomplete data (Algorithm 5), and derive a bound for the misclustering rate.

• Section 4.4 extends a popular early fusion method -based on spectral clustering applied to the sum of adjacency matrices -to the missing nodes setting. Section 4.4.1 studies this by imputing the missing entries with zeros (Algorithm 6), and contains an upper bound for the misclutering rate. Section 4.4.2 proposes an alternative method (Algorithm 7) wherein the missing entries are imputed iteratively. This method is shown to perform well in our experiments.

• Section 4.5.2 proposes an extension of an intermediate fusion method -namely the Orthogonal Linked Matrix Factorization (OLMF) method studied by Paul et al., 2020 -to the missing nodes setting.

• In Section 4.6 we empirically evaluate our algorithms on synthetic data, and also on real datasets.

problem setup

A multilayer graph is a sequence of graphs G = (G (1) , . . . , G (L) ). If all the graphs are defined on the same set of nodes N indexed by [n], then G is said to be pillar. Throughout, we will assume that for all l ⩽ L each graph G (l) is undirected and has no self-loop. This implies that its associated adjacency matrix

A (l) ∈ {0, 1} n×n is symmetric with A (l)
ii = 0 for all i. Given G as input, our goal is to recover a partition of N into K disjoint sets (or communities), so that nodes belonging to the same community share a similar connectivity profile. To make the setup more precise, we will study this problem in the setting where G is generated via an underlying (unknown) stochastic model, with a latent community structure. This model is a common extension of the well-studied stochastic block model (SBM) for the unilayer case which we now describe.

Multilayer Stochastic Block Model (MLSBM)

We now describe the multilayer stochastic block model (MLSBM), which is a common extension of the SBM to the setting of multilayer graphs (see for e.g., [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF][START_REF] Bhattacharyya | Spectral Clustering for Multiple Sparse Networks: I[END_REF][START_REF] Lei | Consistent community detection in multi-layer network data[END_REF]. The MLSBM is parametrized by the number of layers L, a common block membership matrix Z ∈ M n,K , and connectivity matrices

Π (1) , . . . , Π (L) ∈ [0, 1] K×K .
Similar to the unilayer case, let us denote P (l) = ZΠ (l) Z T for l = 1, . . . , L. A multilayer graph G is distributed according to the model MLSBM(Z, Π (1) , . . . , Π (L) ) if the adjacency matrix A (l) of each layer is distributed according to a SBM(Z, Π (l) ) for l = 1, . . . , L. Hence, while the probability that two nodes are connected can vary across layers, the block membership of each node remains unchanged. As in the unilayer case we can define the quantities p

(l) max = max i,j p (l) ij , p max = max l p (l) max .

Missing nodes

The assumption that all the layers share the same set of nodes is quite restrictive since real world multilayer networks are often 'nonpillar'. We propose to deal with such networks by considering nodes present in some layers but not in others as missing. Let w (l) i be a binary variable that records the presence of node i in the layer l where w (l) i = 1 if node i is observed in layer l and 0 otherwise. Denoting w (l) = (w

(l) 1 , . . . , w (l) n ) T , let Ω (l) = w (l) (w (l)
) T be the mask matrices and Ã(l) = A (l) ⊙ Ω (l) for l ⩽ L where ⊙ is the usual Hadamard product. Let J l denote the set of non-missing nodes in layer l with n J l = |J l |. By a slight abuse of notation we will denote by A J l the matrix A (l) J l . The number of observed nodes in C k will also be denoted by n J l ,k . Throughout, we assume that the missing nodes are generated as w

(l) i ind.
∼ B(ρ) for i = 1, . . . , n.

final aggregation methods

A natural way to extend unilayer graph clustering to the multilayer setting is to analyze each layer separately and then find a consensus partition -such approaches are referred to as final aggregation methods. For example, one can apply any clustering method on each individual layer, take one layer's labels as a reference, find for each remaining layer the permutation of its labels that maximizes the agreement with the reference layer, and then define a consensus community by majority voting as discussed in [START_REF] Han | Consistent Estimation of Dynamic and Multi-Layer Block Models[END_REF] There exist alternative ways to avoid the cumbersome issue of label switching ambiguity such as the 'aggregate spectral kernel' considered in [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF]. Such methods rely on the quality of each individual layer and are often empirically outperformed by other methods as shown in [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF][START_REF] Han | Consistent Estimation of Dynamic and Multi-Layer Block Models[END_REF] Final aggregation methods are still relevant in the missing nodes context. Indeed, if we have exact recovery for each layer, and if for all k there is at least one common node between two layers belonging to C k , then we can easily reconstruct the whole partition even when the set of common nodes is very small. Hence such methods can be considered as baseline methods.

A method based on a variant of k-means for incomplete data

We now propose a final aggregation method for clustering multilayer graphs in the incomplete setting; it avoids the aforementioned label switching problem.

For each layer l, we can compute the matrix ÛJ l of size |J l | × K corresponding to the eigenvectors associated with the top K eigenvalues (in absolute value) of A J l ∈ R |J l |×|J l | . The matrix ÛJ l can be transformed to a matrix Û(l) of size n × K by completing with zeros the rows of the nodes that haven't been observed1 . Let Û be the n × KL matrix obtained by stacking Û(l) .

Analogously, let U J l be the matrix formed by the K eigenvectors corresponding to non-zero eigenvalues of Z J l Π (l) Z T J l , U (l) be the n × K matrix obtained from U J l by filling the rows corresponding to unobserved nodes with the row corresponding to an observed node (belonging to the same community), and U be the matrix obtained by stacking all the matrices U (l) . For each l, let O l be a K × K orthogonal matrix such that

O l ∈ argmin O T O=I k || ÛJ l -U J l O|| F .
As in the unilayer setting, k-means could be applied on the rows of Û(l) in order to recover the community structure for each l. But in order to avoid the label switching problem we propose to apply on the rows of Û a variant of k-means described in Chi et al., where

Ω U = (w (1) ⊗ 1 K • • • w (L) ⊗ 1 K ) is the n × KL mask matrix with 1 K ∈ R 1×K
denoting the all ones vector. It is a matrix composed of L blocks where the rows of each block are 1 if the corresponding node is observed and 0 otherwise.

Algorithm 5 k-pod clustering

Input: The number of communities K, the sets J l and the adjacency matrices A J l .

1: Form Û(l) from A J l as explained at the beginning of Section 3.1. 2: Form the matrix Û by stacking the matrices Û(l) .

3: Initialize the partition Ẑ and the centroid matrix M. 4: repeat 5:

Replace Û by Û ⊙ Ω U + ( Ẑ M) ⊙ (11 T -Ω U ).

6:

Apply k-means on the complete matrix Û and update M and Ẑ.

7: until convergence. Output: A partition of the nodes N = ∪ K i=1 C i based on Ẑ.

In the worst case, the complexity of the algorithm is O((L + K)n2 ). But in practice the layers are often sparse and so the complexity will be much less 2 . Theorem 8. Consider the missing nodes MLSBM in Section 4.2.2, and suppose that ρL

⩾ 1, KL ⩽ C 0 n, ρn min ⩾ C 1 K 2 max(log 2 n, √ np max ) and np (l) max ⩾ C 2 ρ -1 log n. Let λ (l)
K be the K-th largest singular value of Π (l) and recall that β = n max /n min . If

1 ρLn l p (l) max (λ (l) K ) 2 < (30C 3 β 4 K 3 ) -1
then with probability at least 1 -O(n -1 ), it holds that the solution Ẑ ∈ M n,K of (4.3.1) satisfies

r( Ẑ, Z) ⩽ C 4 exp(-c ′ ρL) + C 5 β 3 K 2 ρLn l p (l) max (λ (l) K ) 2
.

Remark 8. The assumption ρL ⩾ 1 is natural since ρL corresponds to the expected total number of times a node is observed, and a node needs to be observed at least once in order to be classified. The condition ρn min ⩾ C 1 K 2 log 2 n ensures that ρ and n min are not too small. If the communities are well-balanced and the parameters ρ and K are fixed independently of n, then the previous condition is satisfied for n large enough.

Remark 9. Our analysis assumes that each layer is sufficiently informative, and doesn't use the fact that there is more information contained in the whole set of layers than in individual layers. This is why the bound does not improve when L increases. The obtained upper-bound is unlikely to be optimal since as shown in the experiments, the clustering performance does seem to improve a bit when L increases.

Proof. Let Ẑ and Ĉ be solutions of the optimization problem (4.3.1) and write Ū := Ẑ Ĉ. Define U ′ as the block matrix obtained by stacking the matrices U (l) O l , L i = {l ∈ [L] : i ∈ J l } be the indices of layers where the node i appears, and N u = {i : |L i | ⩾ ρL/c} where c > 1 is a constant that will be fixed later. Let S k be the set of 'bad nodes' defined as

S k := {i ∈ C k ∩ N u : ∀l ∈ L i , ||U (l) i * O l - Ū(l) i * || ⩾ δ (l) k /2}
where

δ (l) k := min i∈C k i ′ ∈C k ′ k ′ ̸ =k ||U (l) i ′ * -U (l) i * || = min i∈C k i ′ ∈C k ′ k ′ ̸ =k ||U (l) i ′ * O l -U (l) i * O l ||
is the smallest distance between two rows of U (l) corresponding to different communities. Let

T k := (C k \ S k ) ∩ N u be the complement of S k in N u ∩ C k and T = ∪ k T k .
Step 1. First let us show by contradiction that if for all k, |T k | > n k /30 and n k satisfies the assumptions of the theorem, then all the nodes in T are well classified with probability at least 1 -O(n -1 ). Assume that there exist i ∈ T k and j ∈ T k ′ such that Ūi = Ūj . If

L i ∩ L j ̸ = ∅, every l ∈ L i ∩ L j satisfies max(δ (l) k , δ (l) k ′ ) ⩽ ||U (l) i * -U (l) j * || ⩽ ||U (l) i * - Ū(l) i * || + ||U (l) j * - Ū(l) j * || < δ (l) k 2 + δ (l) k ′ 2 contradicting the fact that i ∈ T k and j ∈ T k ′ . It remains to treat the case L i ∩ L j = ∅.
Let C 1 be a cluster induced by Ū containing the nodes i and j. If there were other nodes belonging to C k and C k ′ but appearing in a common layer, the previous argument can be used to obtain a contradiction. So we can assume that all the nodes of community C k in C 1 and all nodes of community C k ′ in C 1 appear on distinct layers. We are going to show this property implies that for all k the size of C k ∩ C 1 , and thus the size of C 1 , is small with high probability. Let l 1 be a layer where a node in C k ′ ∩ C 1 appears.

The probability that none of the nodes in

C k ∩ C 1 appears in l 1 is (1 -ρ) |C k ∩C 1 | and this probability is O(1/n 2 ) if |C k ∩ C 1 | ⩾ 2ρ -1
log n (we used the fact thatlog(1 -ρ) ⩾ ρ). By symmetry, the result holds for every k such that |C k ∩ C 1 | > 0. Therefore we can assume that

|C 1 ∩ C k | ⩽ 2ρ -1 log n. Since for all k, |T k | ⩾ n k /30 ⩾ 3K 2 ρ -
|T k ′ | = k |C k ∩ T k ′ | ⩽ 2K 2 ρ -1 log n contradicting the fact that |T k | ⩾ 3K 2 ρ -1 log n.
We are now going to show that under the assumptions of the theorem, for all k, T k satisfies |T k | > n k /30 with probability at least 1 -O(n -1 ). In order to prove this result we will first show that

|S k | is small (Step 2) and then show that N u ∩ C k is large (Step 3).
Step 2. Observe that if i ∈ S k then ∀l ∈ L i , 4(δ

(l) k ) -2 ||(U (l) O l ) i * - Ū(l) i * || 2 ⩾ 1. So for all k, |S k |δ 2 k ⩽ 4 i∈C k ∩N u min l∈L i ||(U (l) O l ) i * - Ū(l) i * || 2 ⩽ 4 i∈C k ∩N u l∈L i ||(U (l) O l ) i * - Ū(l) i * || 2 |L i | (4.3.2)
where we used the fact δ

(l)
k ⩾ δ k for the first inequality, and the fact that the minimum is always bounded by the mean for the second inequality.

By summing over k, and using the fact that

|L i | ⩾ ρL/c for i ∈ N u , we get k |S k |δ 2 k ⩽ 4c ρL i∈N u l∈L i ||(U (l) O l ) i * - Ū(l) i * || 2 ⩽ C ρL ||(U ′ -Ū) ⊙ Ω U || 2 F . (4.3.3)
Using triangular inequality we get

||(U ′ -Ū) ⊙ Ω U || 2 F ⩽ ||(U ′ -Û) ⊙ Ω U || 2 F + ||( Û -Ū) ⊙ Ω U || 2 F ⩽ 2||( Û -U ′ ) ⊙ Ω U || 2 F (4.3.4)
where the second inequality follows from the fact that U ′ is feasible for (4.3.1), i.e., it can be written as a product of a membership matrix Z and a centroid matrix C ∈ R K×KL .

Notice that

||( Û -U ′ ) ⊙ Ω U || 2 F = l || ÛJ l -U J l O l || 2 F .
Let λ K,J l be the Kth largest singular value of Z J l Π (l) Z T J l . This last quantity depends on the missing patterns, but the concentration results established in Lemma 14 shows that for all l, n J l ⩽ 1.5ρn with probability at least 1 -O(n -1 ) and Lemma 17 applied with Z J l instead of Z and n J l ,min instead of n min shows that λ K,J l ⩾ n J l ,min λ (l)

K ⩾ 0.5ρn min λ (l)
K with probability at least 1 -O(n -1 ). The concentration inequality used in Lemma 16 and Lemma 14 shows that with probability at least

1 -O(n -1 ), ||A J l -E(A J l )|| ⩽ C n J l p (l) max ⩽ C ρnp (l) max . But ρn min λ (l) K ⩾ 4C ρnp (l)
max for all l due to our assumptions. Moreover, since with high probability, n J l p (l) max ⩾ c log n for each l (using the fact that w.h.p, n J l ⩾ c ′ ρn for each l, the condition in the theorem statement suffices), hence Lemma 16 applies and we get that for for each l that with probability

1 -O(n -2 ) || ÛJ l -U J l O l || 2 F ⩽ C||A J l -E(A J l )|| 2 F λ 2 K,J l ⩽ CK n J l p (l) max λ 2 K,J l . (4.3.5)
So by Lemma 17 and Lemma 14 there exists C > 0 such that with probability at least 1 -O(Ln -2 ) (via union bound), we have for all

l ⩽ L that n J l p (l) max λ 2 K,J l ⩽ C np (l) max ρ(n min λ (l) K ) 2 . ( 4
-O(n -1 ) k |S k |δ 2 k ⩽ CK l np (l) max ρ 2 L(n min λ (l) K ) 2 . We have δ k = min l δ (l) k = min l 1 n k,J l by Lemma 2.1 in Lei et al., 2015. Moreover min l 1 n k,J l ⩾ c √ ρn k with probability at least 1 -O(n -1 ) by Lemma 14 since ρn k ⩾ C log 2 n by assumption. Thus we obtain k |S k | ⩽ k |S k |(c -1 √ ρn k ) 2 (δ k ) 2 ⩽ CKn max l np (l) max ρL(n min λ (l) K ) 2 . Observe that n max n ⩽ β K . If l np (l) max ρL(n min λ (l) K ) 2 < (30Cβ 2 K) -1 , then |S K | < n k /30
for all k By using n min ⩾ n βK this last condition can be simplified as

1 ρLn l p (l) max (λ (l) K ) 2 < (30Cβ 4 K 3 ) -1 .
Step 3. We are now going to show that

|N u ∩ C k | is large. Let p(ρ, L) = P(|L i | < ρL/c).
For the choice c = 25, we always have p < 8/10 since ρL ⩾ 1 by assumption. Chernoff bound (Lemma 32) shows that p(ρ, L) ⩽ e -ρL(1-c -1 )/3 . If ρL > 12 log n then with probability at least 1 -O(n -2 ), N u = N and |N c u | = 0. Let us assume that ρL < 12 log n. The number of nodes in N c u ∩ C k can be written as a sum n k independent Bernoulli variables with parameter p = p(ρ, L) (we will omit the dependence on ρ and L in the following for notation convenience):

|N c u ∩ C k | = i⩽n k b i .
In expectation

E(|N c u ∩ C k |) = pn k and Hoeffding's bound implies that P(||N c u ∩ C k | -pn k | ⩾ t) ⩽ 2e -t 2 /n k for any choice of t > 0. So we can take t = C n k log n = o(n k ) and obtain that with probability at least 1 -O(Kn -2 ) for all k |N c u ∩ C k | ⩽ n k p + C n k log n. Thus |C k ∩ N u | ⩾ n k (1 -p - C log n n k ). If n is large enough, then C log n n k
< 1/30. Since the sets S k have cardinalities at most n k 30 we obtain that |T k | ⩾ 5n k 30 .

Conclusion.

Steps 1,2 and 3 show that all nodes that belong to T k are well classified with probability at least 1 -O(n -1 ). Hence the number of misclustered nodes is bounded by the sum of the cardinalities of S k plus |N c u |. So with probability at least 1 -O(n -1 ) we get

r( Ẑ, Z) ⩽ 1 n (|N c u | + k |S k |) ⩽ 31 30 p(ρ, L) + Cβ l np (l) max ρL(n min λ (l) K ) 2 ⩽ C exp(-c ′ ρL) + Cβ 3 K 2 ρLn l p (l) max (λ (l) K ) 2
.

early fusion methods: spectral clustering on sum of adjacency matrices

Late fusion methods rely heavily on the quality of each layer. However, by simultaneously using all the information contained in all lay-ers, the clustering performance can be improved in some settings (see the numerical experiments in [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF][START_REF] Han | Consistent Estimation of Dynamic and Multi-Layer Block Models[END_REF]. One way to do this is to aggregate the information across layers and then apply a suitable clustering method. This approach will be referred to as an early fusion method. One simple but popular way to do this is to take the mean of the adjacency matrices (see for e.g., [START_REF] Bhattacharyya | Spectral Clustering for Multiple Sparse Networks: I[END_REF][START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF]. Then, the k-means algorithm can be applied to the rows of the n × K eigenvector matrix associated with the top K eigenvalues (in absolute value) of A = L -1 l A (l) .

Imputing missing entries with zeros

A natural way to extend the aforementioned approach to the setting of missing nodes is to fill the missing entries with zeros, thus leading to Algorithm 6. The worst-case complexity of the algorithm is O((L + K)n 2 ).

Algorithm 6 Sum of adjacency matrices with missing entries filled with zeros

Input:

The number of communities K, the matrices A (l) and Ω (l) . l) . 2: Compute the eigenvectors u 1 , . . . , u K associated with the K largest eigenvalues of A (ordered in absolute values) and form

1: Compute A = L -1 l A (l) ⊙ Ω (
U K = [u 1 u 2 • • • u K ].
3: Apply k-means on the rows of U K to obtain a partition of N into K communities.

Output: A partition of the nodes

N = ∪ K i=1 C i .
Let us denote à = ρ -2 L -1 l A (l) ⊙ Ω (l) (clustering on A or à is equivalent since the two matrices are proportional, but for the analysis it is more convenient to work with Ã). Since the diagonal entries of A (l) are zero, E( Ã) = L -1 l E(A (l) ). Denote by E(X|Ω) to be the expectation of X conditionally on Ω = (Ω (1) , . . . , Ω (L) ) and let λ K denote the Kth largest singular value of E( Ã). We have l) . Using the same kind of perturbation arguments and concentration inequalities as in [START_REF] Lei | Consistency of spectral clustering in stochastic block models[END_REF] we can relate à to E( Ã|Ω) and then use Bernstein inequality to relate E( Ã|Ω) with E( Ã). This leads to the following bound on the misclustering rate.

E( Ã|Ω) = ρ -2 L -1 l E(A (l) ) ⊙ Ω (
Theorem 9. Under the missing nodes MLSBM in Section 4.2.2, there exist constants C 0 , C 1 > 0 such that with probability at least 1 -O(n -1 ), the solution Ẑ ∈ M n,K obtained from Algorithm 6 satisfies

r( Ẑ, Z) ⩽ C 0 K ρ 4 λ 2 K np max L + log n L noise error + C 1 K (ρ -2 -1) 2 λ 2 K (np max ) 2 log(n) L + np max log n L 2 missing data error .
If L is small then the missing data error could be larger than one making the upper bound trivial. In the best case scenario, we expect that λ K scales as np max . So we need at least C log n layers to get a non trivial upper bound. In order to obtain asymptotic consistency, it is necessary that L ≫ log n. However, experiments show that even when L is small, Algorithm 6 gives good results as long as the layers are dense enough and the number of missing nodes is not too large.

When ρ = 1 and np max ⩾ log n, r( Ẑ, Z) = O( 1 Lnp max ). That improves the one obtained by Bhattacharyya et al., 2018 under a more general setting. See the supplementary material for other comparisons.

In order to prove Theorem 9, we are going to show that à is close to E( Ã|Ω) with high probability for every realization of Ω and that E( Ã|Ω) concentrates around E( Ã) if L is large enough. These results are summarized in the following proposition.

Proposition 1. There exist constants c 1 and c 2 such that the following holds.

1. P || Ã -E( Ã|Ω)|| ⩾ c 1 ρ -2 np max L + log n L |Ω ⩽ n -1 ; 2. ||E( Ã|Ω) -E( Ã)|| ⩽ c 2 (ρ -2 -1) np max log n L + log n L with probability at least 1 -o(n -1 ).
Proof. The proof of the first statement is the same as the proof of the corresponding inequality if there are no missing values. Since we reason conditionally to the missingness mechanism, the zero entries of à can also be considered as the realization of independent Bernoulli variables with parameter zero. Let E = ρ 2 ( à -E( Ã|Ω)) and E ′ be an independent copy of E. Define E s = E -E ′ as the symmetrized version of E. Jensen's inequality implies that

||E|| = ||E(E -E ′ |E)|| ⩽ E(||E s || | E), so it is enough to control ||E s ||.
The ψ 2 norm (see for example Vershynin, 2016, Proposition 1.2.1) of each entry of E s is bounded by

K L := C √ L -1 K where K = max i,j,l ||A (l) ij || ψ 2 and A (l)
ij are centered Bernoulli random variables with parameters p (l) ij . By definition of the ψ 2 norm there exists a constant c 0 such that for each i, j ⩽ n

P(|E s ij | ⩾ c 0 K L log n) ⩽ n -4 . Define T ij = E s ij 1 |E s ij |⩽c 0 K L √ log n and let T = (T ij ) ∈ R n×n .
By a union bound argument the matrix E s -T has entries that are not zero with probability at most n -2 , thus ||E s || = ||T || with probability at least 1 -O(n -2 ). Since the entries of E s are symmetric, the matrix T is centered and has entries bounded by c 0 K L log n by construction. So we can apply the bound from Lemma 15 to T and obtain

||T || ⩽ C np max L + K L log n
with probability at least 1 -O(n -1 ). We can use the following theorem to get a sharp bound for K L .

Theorem 10 (Buldygin et al., 2013, Theorem 2.1, Lemma 2.1 (K6)). Let Y be a centered Bernoulli random variable with parameter p, i.e., Y = 1 -p with probability p, and Y = -p with probability 1 -p. Then,

∥Y∥ 2 ψ 2 =        0 ; p ∈ {0, 1} , 1/4 ; p = 1/2, 1-2p 2 log( 1-p p )
; p ∈ (0, 1) \ 1 2 .

In particular, it holds that

∥Y∥ ψ 2 ⩽ 1 √ 2|log(min{2p,2(1-p)})|
.

If np max ⩽ log 2 n, then K L ⩽ C(L log n) -1/2 and we obtain the first part of the proposition by dividing by ρ 2 . If np max ⩾ log 2 n then we can bound use the trivial bound K L ⩽ CL -1/2 to see that

K L log n ⩽ C np max L . Hence || Ã -E( Ã|Ω)|| ⩽ Cρ -2 np max L + log n L with probability at least 1 -O(1/n) for all Ω.
It remains to bound the difference between E( Ã|Ω) and E( Ã). We do so using the matrix Bernstein inequality (Lemma 34).

Let X l := ρ -2 E(A (l) ) ⊙ Ω (l) -E(A (l) ); clearly each X l is centered. Moreover ||X l || ⩽ ||X l || F ⩽ p max n(ρ -2 -1).
For notation convenience, we will write X instead of X l . We have

E(X 2 ) ij = k⩽n X ik X jk because X is symmetric. Recall that X ik = a ik (ρ -2 ω i ω k -1) where a ik corresponds to A (l) ik . A simple calcula- tion shows that E(X 2 ) ij = k E(a ik a jk (ρ -2 ω i ω k -1)(ρ -2 ω j ω k -1))) = k a ik a jk E((ρ -2 ω i ω k -1)(ρ -2 ω j ω k -1))). If i = j, E((ρ -2 ω i ω k -1) 2 ) = ρ -2 -1 and if i ̸ = j, E((ρ -2 ω i ω k - 1)(ρ -2 ω j ω k -1))) = ρ -1 -1. So in both cases, |E(X 2 ) ij | ⩽ np 2 max (ρ -2 - 1). We can now bound ||E(X 2 l )|| by ||E(X 2 l )|| F ⩽ [np max (ρ -2 -1))] 2 and σ 2 := || l E(X 2 l )|| by L[np max (ρ -2 -1)] 2 . Therefore matrix Bernstein inequality implies that || l X l || ⩽ C(ρ -2 -1)(np max L log n + np max log n)
with probability at least 1 -O(n -1 ) for a constant C chosen appropriately.

Proof of Theorem 9. Triangle inequality gives

|| Ã -E( Ã)|| ⩽ || Ã -E( Ã|Ω)|| + ||E( Ã|Ω) -E( Ã)||
and we can use Proposition 1 to bound with high probability each term. So with probability at least

1 -O(n -1 ) || Ã -E( Ã)|| ⩽ C ρ 2 np max L + log n L + C(ρ -2 -1) np max log n L + np max log n L .
We can now use the relation established in [START_REF] Lei | Consistency of spectral clustering in stochastic block models[END_REF], Lemma 2.1, and an immediate adaptation of Lemma 16 to conclude as in Theorem 8.

Iteratively imputing the missing entries

When the number of missing nodes is important, filling missing entries with zero can lead to a huge bias and hence poor clustering performances. In order to reduce the bias we propose an alternative way of imputing the missing values (outlined as Algorithm 7) based on the fact that each adjacency matrix is a noisy realization of a structured matrix. At iteration t, given an initial estimate Ût K ∈ R n×K of the common subspace we can estimate the membership matrix Ẑt by applying kmeans on Ût K . Then, we can estimate the connectivity matrix Π(l),t for each l as

Π(l),t = (( Ẑt ) T Ẑt ) -1 ( Ẑt ) T A (l),t Ẑt (( Ẑt ) T Ẑt ) -1 . (4.4.1)
Given Ẑt and Π(l),t we estimate the rows and columns corresponding to missing nodes. Indeed, the connectivity profile of a node i in layer l is given by the ith row of Ẑt Π(l),t ( Ẑt ) T . By replacing the rows and columns of missing nodes by their estimated profiles, and leaving the value of observed nodes unchanged, we obtain the updated imputed matrix A (l),t+1 . Applying spectral clustering on L -1 l A (l),t+1 then leads to an updated estimate Ût+1 K of the common subspace. The procedure can be repeated using Ût+1 K and A (l),t+1 , thus iteratively imputing the missing values in order to obtain "completed" adjacency matrices that share the same K rank structure across layers. In the worst case, the complexity of the algorithm run with T iterations is O((K + L)n 2 T + LKnT ).

Similar iterative imputation methods have been studied in the context of principal component analysis, see for e.g., [START_REF] Zhang | Heteroskedastic PCA: Algorithm, optimality, and applications[END_REF][START_REF] Zhu | Highdimensional principal component analysis with heterogeneous missingness[END_REF] In our experiments, Algorithm 7 is seen to perform Algorithm 7 Sum of adjacency matrices with missing entries filled iteratively Input: Number of communities K; J l and A J l ∈ R n×n for each l; initial estimate of the common subspace Û0

K ∈ R n×K (with orthonormal columns) obtained from Algorithm 6; number of iterations T .

1: Initialize t = 0 and A (l),0 = A J l for all l. 2: repeat 3:

Given Ût

K , estimate the membership matrix Ẑt and the connectivity parameters Π(l),t for all l by using (4.4.1).

4:

For each l, replace rows (and corresponding columns) of A (l) corresponding to a missing node i by the ith row of Ẑt Π(l),t Ẑt T to form A (l),t+1 .

5:

Compute the eigenvector matrix Ût+1

K = [u t+1 1 u t+1 2 • • • u t+1
K ] associated with the K largest (in absolute order) eigenvalues of L -1 l A (l),t+1 . Update t ← t + 1. 6: until t ⩽ T 7: Apply K-means on ÛT K to get a partition of N. Output: A partition of the nodes N = ∪ K i=1 C i .

significantly better than other methods when ρ decreases. While we do not currently have any statistical performance guarantee for Algorithm 7, establishing this is an interesting direction for future work.

intermediate fusion methods: olmf estimator

Orthogonal Linked Matrix Factorization (OLMF) is a clustering method for multilayer graphs that originated in the work of [START_REF] Tang | Clustering with Multiple Graphs[END_REF] in the complete data setup, and was later analysed in [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF].

It shows good performance in various settings and outperforms spectral clustering when the multilayer network contains homophilic and heterophilic communities (see the numerical experiments in [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF].

The complete data setting

In the complete data setting, the OLMF estimator is a solution of the following optimization problem

( Q, B(1) , . . . , B(L) ) ∈ argmin Q T Q=I k B (1) ,...,B (L) l ||A (l) -QB (l) Q T || 2 F , (4.5.1)
where Q ∈ R n×K , B (l) ∈ R K×K . Note that there is no constraint on the values taken by the entries of B (l) . A little algebra (see [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF] shows that the optimization problem (4.5.1) is equivalent to

Q ∈ argmax Q T Q=I k l ||Q T A (l) Q|| 2 F , B(l) = QT A (l) Q (4.5.2)
for l = 1, . . . , L. The OLMF estimator can be computed with a gradient descent on the Stiefel manifold (see [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF] and supplementary material therein). The community estimation is then obtained by applying K-means on the rows of Q.

Extension to the missing nodes setting

We now present an extension of the OLMF estimator to the setting of missing nodes. By replacing the matrices A (l) , Q in the objective function in (4.5.1) with A J l ∈ R n×n , Q J l ∈ R n×K , we end up with the following modification for the incomplete setting

( Q, B(1) , . . . , B(L) ) ∈ argmin Q T Q=I k B (1) ,...,B (l) l ||A J l -Q J l B (l) Q T J l || 2 F . (4.5.3)
In our experiments, we employ a BFGS algorithm for solving (4.5.3). The worst-case complexity of the algorithm is O(LK(n 2 + Kn)). Denoting the objective function in (4.5.2) by F, its gradients are given by

∂F ∂Q = -2 l (A J l -Q J l B (l) Q T J l )Q J l B (l) , ∂F ∂B (l) = -Q T J l (A J l -Q J l B (l) Q T J l )Q J l .
We relax the constraint that the gradient remains on the Stiefel manifold of n × k matrices, and initialize the parameters using Algorithm 6.

The optimization problem in (4.5.2) can be motivated via the missing nodes MLSBM as follows. If we replace the noisy realization A J l with (ZΠ (l) Z T ) ⊙ Ω (l) then one can show (under some conditions) that the solution Q of (4.5.3) has the same column span as the ground truth Z ∈ M n,K . This is shown formally in the following proposition.

Proposition 2. Assume that Π (l) is full rank for each l, and that for each l, l ′ the sets J l ∩ J l ′ intersect all communities. Then if A J l = (ZΠ (l) Z T ) ⊙ Ω (l) , it holds that the solution of (4.5.3) is given by Q = Z(Z T Z) -1/2 and B(l) = (Z T Z) 1/2 Π (l) (Z T Z) 1/2 and is unique up to an orthogonal transformation. Moreover if i, j belong to the same community, then Qi * = Qj * .

The matrix E(A (l) ) can be considered as a slight perturbation of ZΠ (l) Z T since the former has zeros on the diagonal. Thus the proposition shows that when there is no noise, the column-span of Q (the solution of (4.5.3)) is the same as the ground truth partition Z.

numerical experiments

Synthetic data

We now describe simulation results when the multilayer graph is generated from the missing nodes MLSBM. The Normalized Mutual Information (NMI) criterion is used to compare the estimated community to the ground truth partition. It is an information theoretic measure of similarity taking values in [0, 1], with 1 denoting a perfect match, and 0 denoting completely independent partitions. Nodes that are not observed at least once are removed. The diagonal (resp. off-diagonal) entries of the connectivity matrices are generated uniformly at random over [0.18, 0.19] (resp. 0.7 * [0.18, 0.19]). The ground truth partition is generated from a multinomial law with parameters 1/K. While K = 3 is fixed throughout, the parameters n, ρ and L are varied suitably. The average NMI is reported over 20 Monte Carlo trials. As shorthand, we denote Alg. 5 by k-pod, Alg. 6 by sumAdj0, Alg. 7 by sumAdjIter, and (4.5.3) by OLMFm.

Figure 12 shows that sumAdj0 gives good results unless ρ is too small. Then, the performance of this method decreases quickly. This suggests that there is a threshold involving ρ and the difference between intra and inter connectivity parameters. Figure 14 supports this claim. When ρ is small, the performance of sumAdj0 doesn't improve when n increases. So even if the separation between communities improves, the intra and inter connectivity parameters remain the same suggesting a link between these parameters and ρ.

When L increases (see Figs. 12 and 13), the performance of all methods improves. However, performance of k-pod improves less quickly than other methods. This is expected since contrary to other methods, k-pod relies more on the quality of each individual layer. OLMFm and sumAdjIter exhibit better performance than others in the challenging situation when ρ is small, and perform as well as the others when ρ ≈ 1. They perform significantly better than k-pod, especially when L is large.

MIT Reality Mining dataset

This dataset records interactions (measured by cell phones activities) between 96 students and staff at MIT in the 2004-05 school year (see [START_REF] Eagle | Reality Mining: Sensing Complex Social Systems[END_REF]. We used the dataset as provided by the R package 'GreedySTBM'. As in [START_REF] Han | Consistent Estimation of Dynamic and Multi-Layer Block Models[END_REF] we removed the first and last layers, then discretized the time into one week intervals. The number of times two persons had an interaction during the week is not conserved in order to have a simple undirected graph corresponding to each layer. In total we obtained 32 layers. For different values of ρ, we randomly removed nodes in each layer of the multilayer network. The average NMI over 50 Monte Carlo trials is reported in Table 2 for our methods. The ground truth partition here is taken to be that obtained from sumAdj0 when ρ = 1. We disregarded k-pod because even when ρ = 1, its performance was disappointing and very sensitive to the initialization. This is not very surprising since this method works only if each layer is informative enough while we have a multilayer network where individual layers can be very sparse. The performance of the other three methods studied are quite similar when ρ is not too small (ρ ⩾ 0.4). However, the performance of OLMFm seems to be quite sensitive to initialization since for ρ ∈ {0.6, 0.8} its performance is worse than sumAdj0 and sumAdjIter. Even if we remove half of the nodes in each layer we can still approximately recover the partition.

ρ sumAdj0

Malaria parasite genes network

The dataset was constituted by [START_REF] Larremore | A Network Approach to Analyzing Highly Recombinant Malaria Parasite Genes[END_REF] to study the var genes parasite Plasmodium falciparum involved in Malaria. The nodes of the dataset correspond to 307 different amino acid sequences and each of the 9 layers corresponds to a highly variable region (HVR). Two nodes are linked in a given layer if there is a common block sequence between the corresponding amino acid sequences within the HVR associated to the layer. The analysis in [START_REF] Larremore | A Network Approach to Analyzing Highly Recombinant Malaria Parasite Genes[END_REF][START_REF] Jing | Community detection on mixture multilayer networks via regularized tensor decomposition[END_REF] shows that the first six layers share the same community structure with K = 4. Hence we restrict our study to the first six layers with K = 4. We use the same procedure as before to delete nodes and to select the ground truth partition. k-pod was disregarded for the same reason as the previous experiment. As ρ de- creases, the clustering performance decreases rapidly due to a weak separation between the clusters.

ρ sumAdj0

possible extensions and concluding remarks

Extension of IR-LS to the multilayer setting

In this section, we will describe how to extend the algorithm designed for clustering the CSBM to the multilayer graph with missing nodes setting.

The principle of the method is similar to the one exposed in Section 3.3. Instead of having Gaussian covariates, we have graph side information given by the other layers. Consequently, we need to change the criterion used to refine the partition at each step. But this is quite straightforward, it is sufficient to take the sum of the least square criterion used for individual SBM in Algorithm 1. Since a node i may not appear in each layer, we should disregard the layer were i is missing since these layers doesn't provide any information about the community i belong to.

Algorithm 8 Iterative Refinement for MLSBM (IR-MLSBM)

Input: Adjacency matrices A (l) ∈ R n×n for l ⩽ L and a initial estimate of the latent partition Z (0) ∈ {0, 1} n×K .

1: for 0 ⩽ t ⩽ T -1 do 2:
Given Z (t) , estimate the model parameters: t) for all l ⩽ L.

n (t) k = |C (t) k |, W (t) = Z (t) (D (t) ) -1 where D (t) = diag(n (t) k ) k∈[K] , Π (l,t) = W (t) ⊤ A (l) W (

3:

Refine the partition by solving for each i ∈

[n] z (t+1) i = arg min k l⩽L ||(A (l) i: W (t) -Π (l,t) k: ) Σ (l,t) k || 2 1 i∈J l where Σ (l,t) k =              diag( n (t) k ′ Π (l,t) kk ′ ) k ′ ∈[K] (IR-LS') min k ′ n (t) k ′ max k ′ ,k ′′ Π (l,t) k ′ k ′′ I K (sIR-LS')
n K(p (l,t) -q (l,t) ) log( p (l,t) (1-q (l,t) ) q (l,t) (1-p (l,t) ) )I K (IR-LSS')

with p (l,t) = K -1 k∈[K] Π (l,t) kk and q (l,t) = (K 2 - K) -1 k̸ =k ′ ∈[K] Π (l,t)
kk ′ .

4:

Form the matrices Z (t+1) from z (t+1) .

5: end for Output: A partition of the nodes Z (T ) . This algorithm can be thought as an intermediate fusion method. But contrary to OLMF that optimizes on the set of orthogonal matrices of rank K, Algorithm 8 optimize directly on the discrete set M n,K .

We have shown in Chapter 3 that under the SBM, the misclustering rate associated with the output of Algorithm 1 decreases exponentially in np max (up to a constant factor). So it is natural to conjecture that under the MLSBM the signal contained in each layer will add up and we would obtain a convergence rate of order e -Θ(Lnp max ) . When we delete some node with probability ρ, in average each node will appear in ρL layers and will have an average degree of order ρnp max instead of np max . This leads to the following conjecture.

Conjecture 1. Under the MLSBM with missing nodes, if ρ 2 Lnp max → ∞ and L is large enough, the output Z (T ) of Algorithm 8 initialized with a estimate Z (0) such that r(Z (0) , Z) = O(1/K) satisfies r(Z (T ) , Z) ⩽ e -Θ(ρ 2 Lnp max ) .

To prove this conjecture one could use the same proof techniques as used in Chapter 3. We believe that the adaptation is quite straightforward, but requires some effort to be presented with all the details. We leave that for further work. However, the preliminary numerical experiments results presented in Figure 15 are not very encouraging. We considered a similar setting as in Section 4.6.1: L = 3, K = 3, ρ = 0.9 and the connectivity matrices are the diagonal (resp. off-diagonal) entries of the connectivity matrices are generated uniformly at random over [0.18, 0.19] (resp. 0.7 * [0.18, 0.19]). We initialized Algorithm 8 with sumAdj0. The performances are averaged over 20 Monte-Carlo simulations. Contrary to our expectation, the performance of the clustering slightly decreases after using IR-MLSBM.

Other research direction

We assumed for simplicity that the nodes are missing under a Bernoulli sampling scheme, but other missing patterns could be considered. Another important direction would be to relax the strong condition imposed by MLSBM that all layers share the same common partition. For example, it would be more realistic to assume that the partition of networks evolving over time also evolves slowly. It would also be interesting to study the case where missing values are missing not at random (MNAR). In this case, the positions where there is a missing value could contain information about the underlying communities and hence be considered as side information.

additional proofs

Auxiliary Lemmas

We first recall the standard Chernoff bound for sum of independent Bernoulli random variables.

Lemma 14. Under the assumptions of Theorem 8, with probability at least 1 -O(KL/n 2 ), it holds for each k = 1, . . . , K and l = 1, . . . , ⩽ L that

ρ 2 n k ⩽ n k,J l ⩽ 2ρn k . Proof. Recall that n k,J l = i∈C k 1 i∈J l is a sum of n k independent
Bernoulli random variables with parameter ρ. By applying Lemma 32 with δ = 1/2 we get

n k,J l ⩾ ρ 2 n k and n k,J l ⩽ 2n k with probability at least 1 -O(1/n 2 ), provided that n k ρ ⩾ C log n
for a constant C large enough as assumed in Theorem 8. The lemma follows from a union bound.

Lemma 15. Let X be an n × n symmetric matrix whose entries X ij are independent centered random variables. Then there exists for any 0 < ϵ ⩽ 1/2 a universal constant c ϵ such that for every t ⩾ 0

P(||X|| ⩾ 2(1 + ϵ) σ + t) ⩽ exp - t 2 cϵ σ * where σ = max i j E(X 2 ij ) and σ * = max i,j E||X ij || ∞ .
Proof. See Bandeira et al., 2016, Corollary 3.12 and Remark 3.13.

Lemma 16. Let A ∈ R n×n be an adjacency matrix generated by a SBM(Z, Π). Denote λ K to be the Kth largest singular value of

P = ZΠZ T . If λ K > 2||A -E(A)||, then with probability at least 1 -O(n -2 ) || Û -UO|| F ⩽ C √ K max(np max , log n) λ K
where Û is the matrix formed by the first K left singular vectors of A, U = Z(Z T Z) with probability at least 1 -O(n -2 ). Moreover, since Û and UO are at most rank K matrices we have

|| Û -UO|| F ⩽ √ 2K|| Û -UO||.
Wedin's theorem (see [START_REF] Wedin | Perturbation bounds in connection with singular value decomposition[END_REF] implies that

|| Û -UO|| ⩽ ||A -E(A)|| δ (4.8.1)
where δ := |λ K (A) -λ K+1 (E(A))| represents the spectral gap. By Weyl's inequality,

|λ K+1 (E(A)) -λ K+1 (ZΠZ T )| ⩽ ||E(A) -ZΠZ T ||.
Since E(A) -ZΠZ T is a diagonal matrix, its spectral norm is bounded by its largest coefficient that is bounded by p max . Moreover since λ K+1 (ZΠZ T ) = 0 we get λ K+1 (E(A)) ⩽ p max . The same argument can be used to show that λ K (E(A)) ⩾ λ K (ZΠZ T ) -p max .

Weyl's inequality also implies that

|λ K (E(A)) -λ K (A)| ⩽ ||A -E(A)||. Thus λ K (A) ⩾ λ K (E(A)) -||A -E(A)|| ⩾ λ K (ZΠZ T ) -p max -||A -E(A)|| ⩾ 1 2 λ K (ZΠZ T ) -p max .
The last inequality follows from the assumption that λ

K (ZΠZ T ) ⩾ 2||A -E(A)||. Since p max ⩽ ϵ(n)λ K (ZΠZ T ) where ϵ(n) → 0 when n → ∞, λ K (A) ⩾ cλ K (ZΠZ T
) and then δ ⩾ cλ K (ZΠZ T ). Therefore the concentration bound stated at the beginning of the proof and (4.8.1) implies

|| Û -UO|| F ⩽ √ 2K ||A -E(A)|| δ ⩽ C √ K max(np max , log n) λ K with probability at least 1 -O(n -2 ).
Lemma 17. We have λ K (ZΠZ T ) ⩾ n min λ K (Π).

Proof. Let µ 1 ⩾ µ 2 ⩾ • • • ⩾ µ K be the K non-zero eigenvalues of ZΠZ T .
By the variational characterization of eigenvalues we have for all k = 1, . . . , K µ k (ZΠZ T ) = min

V⊂G n-k+1 max x∈V ||x||=1 x T ZΠZ T x (4.8.2)
where G n-k+1 denotes the set of n -k + 1 dimensional subset of R n . Observe that ker(Z T ) = Im(Z) ⊥ . An element x ∈ ker(Z T ) cannot be a solution because ZΠZ T is a rank K matrix and thus µ k (ZΠZ T ) ̸ = 0, so for k ⩽ K the optimization problem (4.8.2) is equivalent to

µ k (ZΠZ T ) = min V⊂G n-k+1 max x∈V∩Im(Z) ||x||=1
x T ZΠZ T x.

(4.8.3)

It implies in particular that any eigenvector of ZΠZ T associated with µ k belongs to Im(Z), so it has a block structure. Let v be an eigenvector associated with µ k (ZΠZ T ) for 1 ⩽ k ⩽ K. Then v = Zu where u ∈ R K . In particular ZΠZ T v = Zw where w = ΠZ T Zu. Thus

µ 2 k (ZΠZ T ) = ||ZΠZ T v|| 2 ⩾ n min ||w|| 2 ⩾ n min λ 2 K (Π)||Z T Zu|| 2 ⩾ n 2 min λ 2 K (Π)||v|| 2
because the least singular value of Z is √ n min . Clearly, this in particular implies that λ K (ZΠZ T ) ⩾ n min λ K (Π).

Comparison between misclustering bound under MLSBM in the complete setting

Here we compare existing bounds for the misclustering rate under the MLSBM in the complete data setting. In order to simplify the comparison between the existing bounds, we will assume that K is a constant, the communities are well balanced and p (l) max ≈ p max for each l.

• Co-regularized spectral clustering. This algorithm was introduced by [START_REF] Kumar | Co-regularized Multi-view Spectral Clustering[END_REF] (l) is full rank then w.h.p.

r OLMF = O 1 √ np max max 1, (log n) 2+ϵ log L L 1/4 .
• Sum of adjacency matrices. It was shown by [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF] that if Lnp max ⩾ log n and λ K ( l Π (l) ) ≈ Lp max then w.h.p. [START_REF] Bhattacharyya | Spectral Clustering for Multiple Sparse Networks: I[END_REF] showed that if Lnp max ⩾ log n and λ K ( l Π (l) ) ≈ Lλ K (Π (1) ) then w.h.p.

r sum = O log n Lnp max .
r sum = O 1 √ Lnp max .
The condition Lnp max ⩾ log n is not stated in [START_REF] Bhattacharyya | Spectral Clustering for Multiple Sparse Networks: I[END_REF] and is only assumed here for simplification. This last bound is better than the former in the sparse case when Lnp max ≈ log n. But when np max ≫ log n 2 the first bound is sharper.

• Bias adjusted sum of the squared adjacency matrices. Sum of adjacency matrices performs badly when some layers are associative and other disassociative. Taking the sum of the square of adjacency matrices instead permits us to overcome this issue. However the diagonal entries of these squared matrices introduce bias, so they are often removed. More involved debiasing strategies have also been considered by [START_REF] Zhang | Heteroskedastic PCA: Algorithm, optimality, and applications[END_REF][START_REF] Giraud | Partial recovery bounds for clustering with the relaxed K-means[END_REF]. Assume L = O(n). In the sparse case when √ Lnp max ⩾ C log n and np max = O(1), Lei et al., 2022 showed that w.h.p.

r sq = O 1 n + log n L(np max ) 2 .
If np max ⩾ C log n they showed that w.h.p.

r sq = O log n √ Lnp max .
This method was also analyzed by [START_REF] Bhattacharyya | General Community Detection with Optimal Recovery Conditions for Multirelational Sparse Networks with Dependent Layers[END_REF] They showed that if Lnp max ⩾ C log n then w.h.p.

r sq = O 1 (Lnp max ) 1/2 .

M I N I M A X O P T I M A L C L U S T E R I N G O F H I G H -D I M E N S I O N A L B I PA R T I T E G R A P H S

This chapter is based on [START_REF] Braun | Minimax Optimal Clustering of Bipartite Graphs with a Generalized Power Method[END_REF] We show that a similar iterative refinement algorithm as the one used for clustering CSBM can be used to cluster bipartite graphs in the high-dimensional regime where the number of columns n 2 is larger than the number of rows n 1 . Our work extends the work of [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] in several ways. First our algorithm can be applied to a more general bipartite graph model contrary to the one proposed by [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] that is limited to a particular case where the columns (resp. rows) can be partitioned into L (resp. K) communities with K = L = 2. Secondly we derive a minimax lower bound on the misclustering on the misclustering error when K = L = 2, which matches the corresponding upper bound up to a constant factor.

In Section 5.1 we introduce the problem and discuss the related work. The statistical framework is presented in Section 5.2 and the algorithms in Section 5.3. In Section 5.4 we analyze the spectral method used for initialization. In Section 5.5 we analyze the GPM algorithm used to refine the estimate of the partition. In Section 5.6 we derive a minimax lower bound on the misclustering error. Experimental results are presented in Section 5.7 and the additional proofs details are gathered in Section 5.8.

introduction

The interactions between objects of two different types can be naturally encoded as a bipartite graph where nodes correspond to objects and edges to the links between the objects of different type. One can find examples of such data in various fields, e.g., interactions between customers and products in e-commerce [START_REF] Huang | Analyzing Consumer-Product Graphs: Empirical Findings and Applications in Recommender Systems[END_REF], interactions between plants and pollinators [START_REF] Young | Reconstruction of plant-pollinator networks from observational data[END_REF], investors and assets networks [START_REF] Squartini | Enhanced capital-asset pricing model for bipartite financial networks reconstruction[END_REF], judges vote predictions [START_REF] Guimerà | Justice Blocks and Predictability of U.S. Supreme Court Votes[END_REF] and constraint satisfaction problems [START_REF] Feldman | Subsampled Power Iteration: A Unified Algorithm for Block Models and Planted CSP's[END_REF].

Clustering is one of the most important analysis tasks on bipartite graphs aimed at gathering nodes that have similar connectivity profiles. To this end, several methods have been proposed in the literature, e.g., convex optimization approaches [START_REF] Lim | A Convex Optimization Framework for Bi-Clustering[END_REF], spectral methods [START_REF] Zhou | Analysis of spectral clustering algorithms for community detection: the general bipartite setting[END_REF], modularity function maximization [START_REF] Beckett | Improved community detection in weighted bipartite networks[END_REF], pseudo-likelihood [START_REF] Zhou | Optimal Bipartite Network Clustering[END_REF] and variational approaches [START_REF] Keribin | Estimation and selection for the latent block model on categorical data[END_REF]. The performance of the algo-rithms are generally evaluated under the Bipartite Stochastic Block Model (BiSBM), a variant of the SBM, where the partitions of the rows and the columns are decoupled. In particular, edges are independent Bernoulli random variables with parameters depending only on the communities of the nodes.

When the number of rows n 1 (corresponding to type I objects) and the number of columns n 2 (corresponding to type II objects) of the adjacency matrix associated to a bipartite graph are of the same order, the BiSBM behaves similarly to the SBM. However, in the high dimensional setting where n 2 ≫ n 1 , classical methods that are applicable when n 1 is of the same order as n 2 can fail. In particular, when the bipartite graph is very sparse, it becomes impossible to consistently estimate the latent partition of the columns, whereas it is still possible to estimate the latent partition of the rows. Hence, methods based on estimating the latent partitions of both rows and columns will necessarily fail in the high dimensional regime when the bipartite graph is very sparse. However, the high dimensional regime appears in many applications, e.g. hypergraphs where the number of columns corresponds to the number of hyperedges, or in e-commerce where the number of customers could be much smaller than the number of products (or vice-versa). Hence it is important to understand how to design statistically optimal algorithms in this regime.

Recently, the work of [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] improved the state-ofthe-art conditions for exact recovery of the latent partition of the rows under the BiSBM when n 2 ≳ n 1 log n 1 . Unfortunately, their method, which can be understood as a generalized power method, uses a centering argument that only works in the special case where there are K = 2 latent communities for the rows and L = 2 latent communities for the columns. Moreover, they only consider a Symmetric BiSBM (SBiSBM) where the edge probabilities can take two values (see Section 5.2). It is not clear how their method can be extended to the more general setting where K ̸ = L ⩾ 2, and with more general connectivity matrices. To overcome this limitation, we propose a new algorithm -also based on the generalized power method -that can be applied to general BiSBMs, and has similar theoretical guarantees when specialized to the setting of [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF]

Main contributions

Our results can be summarized as follows.

• We present a novel iterative clustering method that can be applied to general BiSBMs, unlike the one proposed by [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF]. We analyze our algorithm under the BiSBM without restrictions on K and L, and derive an upper bound on the misclustering error. In particular, we show that our algorithm achieves exact recovery when the sparsity level p max of the graph satisfies p 2 max = Ω( log n 1 n 1 n 2 ) for fixed K, L. This is the same sparsity regime obtained in [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] for the SBiSBM with K = L = 2. We remark that our bounds are non-asymptotic and showcase the full dependence on K, L.

• We derive a minimax lower bound for the misclustering error in the special case of an SBiSBM with L = K = 2 that matches the corresponding upper bound of our algorithm, and is the first such lower bound for this problem. It completes the work of [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] which only shows that an oracle version of their algorithm fails to achieve exact recovery when p 2 max ⩽ ϵ log n 1 n 1 n 2 where ϵ is a small enough constant.

• As part of our analysis, we derive a concentration inequality for matrices with independent centered Binomial entries (see part 3 of Lemma 36) that could be of independent interest.

Our findings are complemented by numerical experiments on synthetic data.

Related work

A clustering strategy based on the MAP estimate for discrete weighted bipartite graphs that can encompass the BiSBM as a special case has recently been proposed by [START_REF] Jo | Discrete-Valued Latent Preference Matrix Estimation with Graph Side Information[END_REF]. However, their method requires estimating the latent partition of the columns and hence requires the sparsity level of the graph p max to satisfy p max ≳ log n 2 n 1 . In contrast, we need a far weaker condition p max ≫ 1 √ n 1 n 2 in the high dimensional regime. This highlights one of the difficulties we face in the high dimensional regime -while it is impossible to correctly estimate all the model parameters, it is still possible to exactly recover the row partition. A similar phenomenon is known for Gaussian mixture models, see [START_REF] Ndaoud | Sharp optimal recovery in the two-component Gaussian Mixture Model[END_REF] Our algorithm design is based on the GPM which has been applied successfully in various statistical learning problems in recent years. This includes, e.g., group synchronization [START_REF] Boumal | Nonconvex Phase Synchronization[END_REF], joint alignment from pairwise difference [START_REF] Chen | The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences[END_REF], graph matching [START_REF] Onaran | Projected power iteration for network alignment[END_REF], low rank matrix recovery [START_REF] Chi | Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview[END_REF] and SBM [START_REF] Wang | Optimal Non-Convex Exact Recovery in Stochastic Block Model via Projected Power Method[END_REF].

The work of Ndaoud et al., 2022 which we extend in the present paper is also based on the GPM. However, there are significant differences between the algorithms. We do not need the centering step used in [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] and since we encode the community memberships in a n 1 × K matrix (instead of using the sign), our algorithm can be applied when K > 2. Our algorithm is closer to the one proposed by [START_REF] Wang | Optimal Non-Convex Exact Recovery in Stochastic Block Model via Projected Power Method[END_REF] for clustering graphs under the SBM. In contrast to Wang et al., 2021, we do not add any constraints on the columns, and instead of solving a linear assignment problem, we directly project on the extreme points of the unit simplex.

The GPM is also related to alternating optimization, a common strategy used to solve non-convex optimization problems in an iterative way. For example, EM-type algorithms [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF] have been used since decades. In general, these methods are not guaranteed to achieve a global optimum. However, a recent line of research has show that under various statistical models, alternating optimization can actually lead to consistent estimators [START_REF] Lu | Statistical and Computational Guarantees of Lloyd's Algorithm and its Variants[END_REF][START_REF] Chen | Gradient descent with random initialization: fast global convergence for nonconvex phase retrieval[END_REF][START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF][START_REF] Han | Exact Clustering in Tensor Block Model: Statistical Optimality and Computational Limit[END_REF][START_REF] Chen | Optimal Clustering in Anisotropic Gaussian Mixture Models[END_REF]Braun et al., 2021a). Our proof techniques are based on the work of Braun et al., 2021a which itself is based on a general framework developed by [START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF]

the statistical framework

The BiSBM is defined by the following parameters.

• A set of nodes of type I, N 1 = [n 1 ], and a set of nodes of type II,

N 2 = [n 2 ]. • A partition of N 1 into K communities C 1 , . . . , C K and a partition of N 2 into L communities C ′ 1 , . . . , C ′ L . • Membership matrices Z 1 ∈ M n 1 ,K and Z 2 ∈ M n 2 ,L where M n,K
denotes the class of membership matrices with n nodes and K communities. Each membership matrix Z 1 ∈ M n 1 ,K (resp. Z 2 ∈ M n 2 ,L ) can be associated bijectively with a partition function z :

[n] → [K] (resp. z ′ : [n] → [L]) such that z(i) = z i = k
where k is the unique column index satisfying (Z 1 ) ik = 1 (resp. (Z 2 ) ik = 1). To each matrix Z 1 ∈ M n 1 ,K we can associate a matrix W by normalizing the columns of Z 1 in the ℓ 1 norm:

W = Z 1 D -1 where D = Z ⊤ 1 1 n 1 . This implies that W ⊤ Z 1 = I K = Z ⊤ 1 W.
• A connectivity matrix of probabilities between communities

Π = (π kk ′ ) k∈[K],k ′ ∈[L] ∈ [0, 1] K×L . Let us write P = (p ij ) i,j∈[n] := Z 1 Π(Z 2 ) ⊤ ∈ [0, 1] n 1 ×n 2 . A graph G is distributed according to BiSBM(Z 1 , Z 2 , Π) if

the entries of the corresponding bipartite adjacency matrix A are generated by

A ij ind. ∼ B(p ij ), i ∈ [n 1 ], j ∈ [n 2 ],
where B(p) denotes a Bernoulli distribution with parameter p. Hence the probability that two nodes are connected depends only on their community memberships. We will frequently use the notation E for the centered noise matrix defined as E ij = A ij -p ij , and denote the maximum entry of P by p max = max i,j p ij . The latter can be interpreted as the sparsity level of the graph. We make the following assumptions on the model.

Assumption A3 (Approximately balanced communities).

The communities C 1 , . . . , C K , (resp. C ′ 1 , . . . , C ′ L ) are approximately balanced, i.e., there exists a constant α ⩾ 1 such that for all k ∈ [K] and l ∈ [L] we have

n 1 αK ⩽ |C k | ⩽ αn 1 K and n 2 αL ⩽ C ′ l ⩽ αn 2 L .
Assumption A4 (Full rank connectivity matrix). The smallest eigenvalue of ΠΠ ⊤ , denoted by λ K (ΠΠ ⊤ ), satisfies

λ K (ΠΠ ⊤ ) ≳ p 2 max and ΠΠ ⊤ ≲ p 2 max .
Assumption A5 (Diagonal dominance). There exist β > 0 and η ⩾ 1 such that for all

k ′ ̸ = k ∈ [K],
we have

(ΠΠ ⊤ ) kk -α 2 (ΠΠ ⊤ ) kk ′ ⩾ βp 2 max and (ΠΠ ⊤ ) kk - 1 α 2 (ΠΠ ⊤ ) kk ′ ⩽ ηβp 2 max .
Remark 10. When all communities C ′ l have size equal to n 2 /L, the first condition in Assumption A5 simplifies to (ΠΠ ⊤ ) kk -(ΠΠ ⊤ ) kk ′ ⩾ βp 2 max and corresponds to a diagonal dominance assumption. The second condition in Assumption A5 is useful to show that β is (up to a parameter η that could depend on L as discussed in the remark below) the parameter that measures the minimum difference between the diagonal and off-diagonal entries of ΠΠ ⊤ . symmetric bisbm. A particular case of interest is where L = K and Π = (p -q)I K + q1 K 1 ⊤ K where 1 ⩾ p > q ⩾ 0 and q = cp for some constant 0 < c < 1. This model will be referred to as the SBiSBM, denoted by SBiSBM(Z 1 , Z 2 , p, q). For K = 2 this corresponds to the model analyzed in the work of [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] Since we now have (for

k ′ ̸ = k) (ΠΠ ⊤ ) kk = p 2 + (K -1)q 2 and (ΠΠ ⊤ ) kk ′ = 2qp + (K -2)q 2
the following observations are useful to note.

• We have (ΠΠ ⊤ ) kk -(ΠΠ ⊤ ) kk ′ = (p -q) 2 ⩽ p 2
max , and also

(p -q) 2 ⩾ (1 -c) 2 p 2 max . Hence Assumption A5 is satisfied for η = 1 (1-c) 2 and β = (1 -c) 2 in the equal-size community case (α = 1). Additionally, λ K (ΠΠ ⊤ ) = (1 -c)p max .
• In the unequal-size case, we need to show choices of β, η under which Assumption A5 holds. To this end, note that

(ΠΠ ⊤ ) kk -α 2 (ΠΠ ⊤ ) kk ′ = p 2 max ((1 -c) 2 -(α 2 -1)K),
and

(ΠΠ ⊤ ) kk - 1 α 2 (ΠΠ ⊤ ) kk ′ = p 2 max ((1 -c) 2 -( 1 α 2 -1)K) ⩽ p 2 max ((1 -c) 2 + (α 2 -1)K).
Hence if, for instance, α 2 = 1 + (1-c) 2 2K , we see that Assumption A5 is satisfied for β = (1-c) 2 2 and η = 3.

Remark 11. One of the parameters β or η can scale with L as shown in the following examples. Assume L is even and α = 1. Let us consider p = (p 1 , . . . , p L/2 ), q = (q 1 , . . . , q L/2 ) where p l > q l for all l ⩽ L/2 and

Π = p q q p ∈ [0, 1] 2×L . Then L max l (p l -q l ) 2 ⩾ min k̸ =k ′ (ΠΠ ⊤ ) kk -(ΠΠ ⊤ ) kk ′ = ∥p -q∥ 2 ⩾ L min l (p l -q l ) 2 , hence β = Θ(L) and η = Θ(1). Let us define q ′ = (q ′ 1 , q ′ 2 , q 3 , . . . , q L/2 ) for q ′ 1 = 0.5q 1 and q ′ 2 = 0.5q 2 If we now consider Π =     p q q p q ′ p     ∈ [0, 1] 3×L .
where q i and p i are as before and q ′ 1 ̸ = q 1 , q ′ 2 ̸ = q 2 . Then it is easy to check that max k̸ =k

′ (ΠΠ ⊤ ) kk -(ΠΠ ⊤ ) kk ′ ⩽ Lp 2 max but min k̸ =k ′ (ΠΠ ⊤ ) kk -(ΠΠ ⊤ ) kk ′ = (ΠΠ ⊤ ) 22 -(ΠΠ ⊤ ) 23 = q 2 1 + q 2 2 2
is independent of L. Hence β = Θ(1) and η = Θ(L).

Remark 12. As already mentioned in the introduction, we will focus in this work on the regime where n 2 ≫ n 1 log n 1 and √ n 1 n 2 p max ≳ log n 1 . In this parameter regime there is no hope to accurately recover Z 2 because the columns of A are too sparse. Indeed, consider the setting where √ n 1 n 2 p max ≍ log n 1 . Then in expectation, the sum of the en-

tries of each column is n 1 p max ≍ n 1 log n 1 n 2 → 0.
But by analogy to the SBM, we would need a condition n 1 p max → ∞ in order to recover Z 2 . This is actually a necessary condition obtained in a related setting by [START_REF] Jo | Discrete-Valued Latent Preference Matrix Estimation with Graph Side Information[END_REF] While it is impossible to estimate Z 2 in this sparsity regime, it is still possible to accurately estimate Z 1 . We will focus on this problem from now onwards.

algorithm

Initialization with a spectral method

We can use a spectral method on the hollowed Gram matrix B to obtain a first estimate of the partition Z 1 . This is similar to the algorithm in [START_REF] Florescu | Spectral thresholds in the bipartite stochastic block model[END_REF] we only use a different rounding step so the algorithm can be applied to general bipartite graphs with K > 2 communities, in contrast to [START_REF] Florescu | Spectral thresholds in the bipartite stochastic block model[END_REF] Algorithm 9 Spectral method on H(AA ⊤ ) (Spec)

Input: The number of communities K and the adjacency matrix A.

1: Form the diagonal hollowed Gram matrix B := H(AA ⊤ ) where H(X) = Xdiag(X).

2: Compute the matrix U ∈ R n 1 ×K whose columns correspond to the top K-eigenvectors of B.

3: Apply approximate (1 + ε) approximate k-means on the rows of U and obtain a partition z (0) of [n 1 ] into K communities.

Output: A partition of the nodes z (0) .

computational complexity of spec. The cost for computing [START_REF] Kumar | A Simple Linear Time (1+ϵ)-Approximation Algorithm for k-Means Clustering in Any Dimensions[END_REF] In practice, this operation is fast and the most costly operation is the computation of B which has complexity O(n 2 1 n 2 p max ), since one can show that nnz(A) = O(n 1 n 2 p max ) with high probability.

B is O(n 1 nnz(A)) and for U is 1 O(n 2 1 K log n 1 ). Applying the (1 + ε) approximate k-means has a complexity O(2 (K/ ε) O(1) n 1 K), see

Iterative refinement with GPM

Our algorithm is based on the Generalized Power Method. In contrast to the power method proposed recently by [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] we do not require to center the adjacency matrix A and, instead of using the sign to identify the communities, we project on M n 1 ,K . Consequently, our algorithm can be applied to bipartite graphs with K > 2 and

K ̸ = L while Ndaoud et al., 2022 require K = L = 2.
In the first step we form the diagonal hollowed Gram matrix B. This is natural since in the parameter regimes we are interested in, there is no hope to consistently estimate Z 2 . Then, we iteratively update the current estimate of the partition

Z (t)
1 by a row-wise projection of 1 The log n 1 term comes from the number of iterations needed when using the power method to compute the largest (or smallest) eigenvector of a given matrix.

BZ (t)

1 onto the extreme points of the unit simplex of R K denoted by S K . The projection operator P : R K → S K is formally defined by P(x) := arg min y∈S K ∥x -y∥ for all x ∈ R K .

( 5.3.1) This implies that (Z (t+1) 1

) ik = 1 for the value of k that maximizes 1: Form the diagonal hollowed Gram matrix B := H(AA ⊤ ) where H(X) = Xdiag(X) and compute

W (0) = ( Ẑ(0) 1 ) ⊤ (D (0) ) -1 where D (0) = diag(( Ẑ(0) 1 ) ⊤ 1 n 1 ). 2: for 0 ⩽ t ⩽ T do 3:
Update the partition Z (t+1) 1 = P(BW (t) ) where P is the operator in (5.3.1) applied row-wise.

4:

Compute

D (t+1) = diag((Z (t+1) 1 ) ⊤ 1 n 1 ). Then form W (t+1) = (Z (t+1) 1 ) ⊤ (D (t+1) ) -1 .
5: end for Output: A partition of the nodes Z (T +1) 1 . computational complexity of gpm. Computing B has a cost O(n 1 nnz(A)), as mentioned earlier. For each t, the cost of computing BW (t) is O(n 2 1 K) and the cost of the projection is O(Kn 1 ). The cost of computing D (t+1) and W (t+1) is O(Kn 1 ). So the total cost over T iterations is O(T n 2 1 K) and doesn't depend on n 2 .

Remark 13. The recent work of [START_REF] Wang | Optimal Non-Convex Exact Recovery in Stochastic Block Model via Projected Power Method[END_REF] proposed a power method for clustering under the SBM, but instead of using a projection on the simplex as we did, they add an additional constraint on the column on Z so that each cluster has the same size. They showed that computing this projection is equivalent to solving a Linear Assignment Problem (LAP). But the cost of solving this LAP is O(K 2 n 1 log n 1 ) whereas the cost of the projection on the simplex is O(Kn 1 ). Moreover, their algorithm requires to know in advance the size of each cluster and it is not straightforward to extend their theoretical guarantees to the approximately balanced community setting.

spectral initialization

We show that Algorithm 9 can recover an arbitrary large proportion of community provided that the sparsity level p max is sufficiently large, and n 2 /n 1 → ∞ sufficiently fast, as n 1 → ∞.

Proposition 3. Assume that A ∼ BiSBM(Z 1 , Z 2 , Π) and Assumptions A3, A4 are satisfied. For any 0 < ε < 1, suppose additionally that

n 2 n 1 ≳ max(L log n 1 , log 2 n 2 ), p 2 max ⩾ KL log n 1 εn 1 n 2 , (5.4.1) KL log n 1 = o(1), (KL) 3 n 2 n 1 log n 1 = o(1) as n 1 → ∞.
(5.4.2)

Then the output z (0) of Algorithm 9 satisfies with probability at least 1n -Ω(1) the bound r(z (0) , z) ≲ Kε 2 .

Proof. First, we will control the noise B -PP ⊤ using Lemma 36.

Note that the conditions in (5.4.1) imply the conditions on n 1 , n 2 and p max in Lemma 36. Since H(•) is a linear operator and B = H(B), we have the decomposition

B = H(PP ⊤ ) + H(PE ⊤ ) + H(EP ⊤ ) + H(EE ⊤ ),
which leads to the following bound by triangle inequality

B -PP ⊤ ⩽ H(PE ⊤ ) + H(EP ⊤ ) + H(EE ⊤ ) + H(PP ⊤ ) -PP ⊤ ⩽ 4 ∥EZ 2 ∥ Π ⊤ Z ⊤ 1 + H(EE ⊤ ) + H(PP ⊤ ) -PP ⊤ .
(5.4.3)

Now let us observe that

H(PP ⊤ ) -PP ⊤ = H(Z 1 ΠZ ⊤ 2 Z 2 Π ⊤ Z ⊤ 1 ) -Z 1 ΠZ ⊤ 2 Z 2 Π ⊤ Z ⊤ 1 = diag(Z 1 ΠZ ⊤ 2 Z 2 Π ⊤ Z ⊤ 1 ) ⩽ αn 2 L (Lp 2 max ) ⩽ αn 2 p 2 max ,
while,

Π ⊤ Z ⊤ 1 ⩽ ∥Z 1 ∥ ∥Π∥ ⩽ n 1 K ∥Π∥ F ⩽ n 1 Lp max .
Using these bounds along with the bounds on ∥EZ 2 ∥ and H(EE ⊤ ) from Lemma 36, and applying them in (5.4.3), it holds with probability at least 1 -n -Ω(1) that

B -PP ⊤ ≲ max(log n 1 + √ n 1 n 2 p max ) + √ n 2 n 1 p 1.5 max + n 2 p 2 max .
Let us denote by Û (resp. U) to be the matrix of top-K eigenvectors of B (resp. PP ⊤ ). Denoting λ K (PP ⊤ ) to be the Kth largest eigenvalue of PP ⊤ , it is not difficult to verify that

λ K (PP ⊤ ) ⩾ n 2 L λ K (Z 1 ) 2 λ K (Π) 2 ≳ n 1 n 2 p 2 max KL .
Then by the Davis-Kahan Theorem [START_REF] Yu | A useful variant of the Davis-Kahan theorem for statisticians[END_REF], there exists an orthogonal matrix (5.4.4) where we bounded the max operation by the sum. Now on account of the conditions in (5.4.1) and (5.4.2), it is easily seen that KL log n 1 n 1 n 2 p 2 max ≲ ε, while the remaining three terms in the RHS of (5.4.4) are o(1) as n 1 → ∞. Finally, we conclude by using the same argument as in the proof of Theorem 3.1 in [START_REF] Lei | Consistency of spectral clustering in stochastic block models[END_REF]

Q ∈ R K×K such that Û -UQ ≲ B -PP ⊤ λ K (PP ⊤ ) ≲ KL log n 1 n 1 n 2 p 2 max + KL √ n 1 n 2 p max + KL √ n 2 p max + KL n 1 .
to show that r(ẑ, z) ≲ Û -UQ 2 F ≲ K Û -UQ 2 ≲ Kε 2 .
Remark 14. As shown in Section 5.7, Spec has very good empirical performance. This suggests that the previous proposition is far from being optimal and doesn't capture the true rate of convergence of this spectral method. Also note that Proposition 3 gives a meaningful bound only when ε = O( 1 √ K ).

analysis of gpm

Our analysis strategy is similar to the one recently considered by Braun et al., 2021a for clustering under the contextual SBM, which in turn is based on the framework recently developed by [START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF] There are however some additional technical difficulties due to the fact that there are more dependencies in the noise since the matrix B is a Gram matrix.

We will assume w.l.o.g. that π * the permutation that best aligns z (0) with z is the identity, if not, then replace z by (π * ) -1 (z). Hence there is no label switching ambiguity in the community labels of z (t) because they are determined from z (0) . First we will decompose the event "after one refinement step, the node i will be incorrectly clustered given the current estimation of the partition z (t) at time t" into an event independent of t, and events that depend on how close z (t) is from z. Then we will analyze these events separately. Finally, we will use these results to show that the error contracts at each step.

Error decomposition

By definition, a node i is misclustered at step t + 1 if there exists a

k ̸ = z i ∈ [K] such that B i: W (t) :k ⩾ B i: W (t) :z i .
(5.5.1)

By decomposing B as

B = H(PP ⊤ ) P + H(EP ⊤ + PE ⊤ + EE ⊤ )
Ẽ one can show that condition (5.5.1) is equivalent to Ẽi:

(W :z i -W :k ) ⩽ -∆ 2 (z i , k) + F (t) ik + G (t) ik (5.5.2)
where

∆ 2 (z i , k) = Pi: (W :z i -W :k ), F (t) ik = ⟨ Ẽi: (W (t) -W), e z i -e k ⟩, and
G (t) ik = ⟨ Pi: (W (t) -W), e z i -e k ⟩.
Here e 1 , . . . , e K denotes the canonical basis of R K . The terms

F (t) ik and G (t)
ik can be interpreted as error terms (due to W (t) ̸ = W) while ∆ 2 (z i , k) corresponds to the signal. Indeed, let us denote

Q = Π(Z ⊤ 2 Z 2 )Π ⊤ .
Then we obtain

PP ⊤ = Z 1 QZ ⊤ 1 which implies ∆ 2 (z i , k) = Pi: (W :z i -W :k ) = Q z i z i -Q z i k -(PP ⊤ ) ii (W iz i -W ik ) = (1 - K n 1 )Q z i z i -Q z i k .
We have for all k, k ′ ∈ [K] that

n 2 αL (ΠΠ ⊤ ) kk ′ ⩽ Q kk ′ ⩽ αn 2 L (ΠΠ ⊤ ) kk ′ which implies n 2 αL (ΠΠ ⊤ ) z i z i -α 2 (ΠΠ ⊤ ) z i k ⩽ ∆ 2 (z i , k) ⩽ αn 2 L (ΠΠ ⊤ ) z i z i - 1 α 2 (ΠΠ ⊤ ) z i k
for n 1 large enough. By Assumption A5 this implies

β n 2 p 2 max αL ⩽ ∆ 2 (z i , k) ⩽ ηβ αn 2 p 2 max L .
(5.5.3)

when n 1 is large enough.

Oracle error

We want to show that the condition (5.5.2) cannot occur with high probability. First we will show that this is indeed the case when we ignore the F and G error terms; the subsequent error will be referred to as the oracle error

ξ(δ) = n 1 i=1 k∈[K]\z i ∆ 2 (z i , k)1 Ω i,k (δ) for δ ∈ (0, 1)
where

Ω i,k (δ) = Ẽi: (W :z i -W :k ) ⩽ -(1 -δ)∆ 2 (z i , k) .
Let us denote

∆ min = min a̸ =b∈[K] ∆(a, b)
to be the minimal separation of the parameters associated with the different communities. We will also denote by

∆2 := β 2 12eLα 3 n 1 n 2 p 2 max KL ,
to be the approximate signal-to-noise ratio (SNR) associated with the model.

In general the rate of decay of ξ(δ) leads to the convergence rate of iterative refinement algorithms, hence it is important to control this quantity by showing that the following condition is satisfied. Let us define

τ (0) = ϵ ′ δ min(1, β 2 )
n 1 ∆ 2 min K for a small enough constant ϵ ′ > 0. This parameter will be referred to as the minimal error required for the initial estimate of our algorithm.

Condition C4 (oracle error). Assume that there exists δ ∈ (0, 1) such that

ξ(δ) ⩽ 3 4 τ (0)
holds with probability at least 1 -η 1 .

Contraction of the error at each step

Let l(z, z ′ ) = i∈[n] ∆ 2 (z i , z ′ i )1 {z i ̸ =z ′ i }
be a measure of the distance between two partitions z, z ′ ∈ [K] n . We want to show that l(z (t) , z) decreases until reaching the oracle error.

To this end, we will need to control the noise level. In particular, we are going to show that the following two conditions are satisfied.

Condition C5 (F-error type). Assume that max

{z (t) :l(z,z (t) )⩽τ (0) } n i=1 max b∈[K]\z i (F (t) ib ) 2 ∆ 2 (z i , b)l(z, z (t) ) ⩽ δ 2 256
holds with probability at least 1 -η 2 .

Condition C6 (G-error type). Assume that

max i∈[n] max b∈[K]\z i |G (t) ib | ∆ 2 (z i , b) ⩽ δ 4
holds uniformly on the event {z (t) : l(z, z (t) ) ⩽ τ (0) } with probability at least 1 -η 3 .

Under these conditions, we can show that the error contracts at each step.

Theorem 11. Assume that l(z (0) , z) ⩽ τ (0) . Additionally assume that Conditions C4, C5, and C6 hold. Then with probability at least 1

-3 i=1 η i l(z (t) , z) ⩽ ξ(δ) + 1 8 l(z (t-1) , z), ∀t ⩾ 1.
(5.5.4)

In particular, we have for all t ≳ log(1/δ) that

l(z (t) , z) ≲ ξ(δ) + τ (0) (1/8) t-Θ(log(1/δ)) .
Proof. It is an immediate adaptation of Theorem 3.1 in Gao et al.,

2019.

The last part can be derived in the same way as in Corollary 1 in Braun et al., 2021b.

Application to BiSBM

When applied to the BiSBM the previous theorem leads to the following result.

Theorem 12. Assume that A ∼ BiSBM(Z 1 , Z 2 , Π), K 2 L ⩽ ∆2 , ∆ → ∞,
(5.4.1) and (5.4.2) are satisfied. Then, under Assumption A3, A4 and A5, if GPM is initialized with a z (0) such that

l(z, z (0) ) ⩽ τ (0) = ϵ ′ δ min(1, β 2 ) n 1 ∆ 2 min K
for a small enough constant ϵ ′ > 0 and δ = 1 4ηα . Then with probability at least 1 -n

-Ω(1) 1 we have for all t ≳ log n 1 r(z (t) , z) ⩽ exp(-(1 -o(1)) ∆2 ).
In particular, if ∆2 > log n 1 , we can exactly recover Z 1 .

Corollary 2. Under the assumptions of Theorem 12, Spec returns an estimate z (0) such that

l(z, z (0) ) ⩽ ηβ αn 1 n 2 p 2 max L r(z, z (0) ) ≲ ηβK 2 ε 2 n 1 ∆ 2 min K and hence satisfies l(z, z (0) ) ⩽ τ (0) for ε = O( min(1,β 2 ) Kη √ ηβ ).
Remark 15. Specialized to the SBISBM case where K = 2 = L, q = cp for some 0 < c < 1, the condition ∆2 = Ω(log n 1 ) needed for exact recovery in the above theorem is equivalent to the condition p max = Ω( log n 1 n 1 n 2 ) needed by [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] We will show in the next section that this rate is optimal. Remark 16. The condition n 1 n 2 p 2 max ≳ KL log n 1 is needed because we used the concentration inequality ||H(EE ⊤ )|| ≲ max(log n 1 , √ n 1 n 2 p max ) in Lemma 36. If this last condition could have been replaced by ||H(EE ⊤ )|| ≲ √ n 1 n 2 p max then we would only need the condition ∆ → ∞. This is the principal step that needs to be improved in order to get more general weak consistency guarantees. A possible way to obtain this tighter concentration inequality would be to adapt the combinatorial method of [START_REF] Friedman | On the Second Eigenvalue of Random Regular Graphs[END_REF] similarly to what have been done to obtain a tight concentration inequality for similarity matrices formed from an hypergraph in [START_REF] Lee | Robust Hypergraph Clustering via Convex Relaxation of Truncated MLE[END_REF] Proof of Theorem 12. In order to apply Theorem 4, we need to show that Conditions C4, C5 and C6 are satisfied. oracle error. Let δ = 1 4ηα . We obtain by Lemma 19 (see Section 5.8) and Assumption A5 that

E(ξ(δ)) ⩽ αηβKn 1 n 2 L p 2 max e -∆2 ≲ α 4 β K 2 L ∆2 e -∆2 = e -(1-o(1)) ∆2
by assumptions on ∆ and KL. Then, by Markov inequality, we obtain

P ξ(δ) ⩾ exp( ∆)Eξ(δ) ⩽ exp(-∆).
Consequently we get

exp( ∆)Eξ(δ) ⩽ exp(-(1 -o(1)) ∆2 )
and hence with probability at least 1exp(-∆)

ξ(δ) ⩽ exp(-(1 -o(1)) ∆2 ) ⩽ 3 4 τ (0) because exp(-(1 -o(1)) ∆2 ) = o(1) ≪ τ (0) = Ω(1)
for n 1 large enough. This shows that Condition C4 is satisfied.

f-error term. W.h.p, for all z (t) such that l(z (t) , z) ⩽ τ (0) we have

n i=1 max b∈[K]\z i (F (t) ib ) 2 ∆ 2 (z i , b)l(z, z (t) ) ⩽ i Ẽi: (W (t) -W) 2 max b∈[K]\z i ∥e z i -e b ∥ 2 ∆ 2 (z i , b)l(z, z (t) ) (by Cauchy-Schwartz) ⩽ 2 Ẽ(W (t) -W) 2 F ∆ 2 min l(z (t) , z) ⩽ 2 Ẽ 2 W (t) -W 2 F ∆ 2 min l(z (t) , z).
By Lemma 36, we have w.h.p.

H(EE ⊤ ) ≲ EE ⊤ -E(EE ⊤ ) ≲ max(log n 1 , √ n 1 n 2 p max )
and

EP ⊤ ⩽ ∥EZ 2 ∥ ΠZ ⊤ 1 ≲ n 1 n 2 p max L p max n 1 K = ( √ n 1 n 2 p max ) n 1 p max KL . Since Ẽ ≲ H(EE ⊤ ) + EP ⊤ we obtain n i=1 max b∈[K]\z i (F (t) ib ) 2 ∆ 2 (z i , b)l(z, z (t) ) ≲ max log 2 n 1 , n 1 n 2 p 2 max , (KL) -1 n 2 1 n 2 p 3 max K 3 l(z (t) , z) n 3 1 ∆ 6 min ≲ K 2 max log 2 n 1 , n 1 n 2 p 2 max , (KL) -1 n 2 1 n 2 p 3 max n 2 1 ∆ 4 min Kτ (0) n 1 ∆ 2 min ≲ K 2 L 2 max log 2 n 1 , n 1 n 2 p 2 max , (KL) -1 n 2 1 n 2 p 3 max β 2 n 2 1 n 2 2 p 4 max ϵ ′ δβ 2
(by definition of τ (0) and (5.5.3))

≲ K 2 L 2 log 2 n 1 + n 1 n 2 p 2 max + (KL) -1 n 2 1 n 2 p 3 max n 2 1 n 2 2 p 4 max ϵ ′ δ.
By assumptions, we have KL n 2 p max = o(1) and

K 2 L 2 log 2 n 1 n 2 1 n 2 2 p 4 max = O(1), K 2 L 2 n 1 n 2 p 2 max ≲ KL log n 1 = o(1).
Hence for an appropriately small choice of ϵ ′ Condition C5 is satisfied.

g-error term. W.h.p, for all z (t) such that l(z (t) , z) ⩽ τ (0) we have

|G (t) i | ∆ 2 (z i , b) ⩽ √ 2 Pi: (W (t) -W) ∆ 2 min .
( by Cauchy-Schwartz)

But since Pi:

≲ P i: P ⊤ ⩽ ∥Π∥ 2 Z ⊤ 2 Z 2 ∥Z 1 ∥ ⩽ αn 2 L p 2 max n 1 K we obtain |G (t) i | ∆ 2 (z i , b) ≲ n 2 p 2 max √ n 1 (W (t) -W) L √ K∆ 2 min ≲ n 2 p 2 max L∆ 2 min Kl(z (t) , z) n 1 ∆ 2 min ( by Lemma 11) ≲ K l(z (t) , z) βn 1 ∆ 2 min ≲ Kτ (0) βn 1 ∆ 2 min .
Now by choosing ϵ ′ to be a suitably small constant (< 1), we obtain

|G (t) i | ∆ 2 (z i , b) ⩽ δ 4 .
This shows that Condition C6 is satisfied.

minimax lower bound

Let us denote the admissible parameters space by

Θ = {P ∈ [0, 1] n 1 ×n 2 : P = Z 1 ΠZ ⊤ 2 where Π = q1 K 1 ⊤ K + (p -q)I K , 1 > p > 0, q = cp for some constant c ∈ (0, 1), Z 1 ∈ M n 1 ,K , Z 2 ∈ M n 2 ,K with α = 1 + O log n 1 n 1 }.
We want to lower bound inf ẑ sup θ∈Θ E(r(ẑ, z)) where ẑ is an estimator of Z 1 . For simplicity, we will assume that K = 2, but we believe that the same argument can be extended to SBiSBM with an arbitrary number of communities K. In the supervised case, i.e. when Z 2 is known, we can use the same strategy as the one use for the degreecorrected SBM by [START_REF] Gao | Community detection in degree-corrected block models[END_REF] and obtain a lower bound of the order e -(1+o(1))n 2 (p-q)/2 corresponding to the failure probability of the optimal test associated to the following two hypothesis problem

H 0 : ⊗ n 2 /2 i=1 B(p) ⊗ n 2 i=n 2 /2+1 B(q), vs H 1 : ⊗ n 2 /2 i=1 B(q) ⊗ n 2 i=n 2 /2+1 B(p).
However when n 2 ≫ n 1 log n 1 the error associated with this hypothesis testing problem is of order exp(-n 2 (p -q)/2) ≈ exp(-

n 2 log n 1 n 1
) but this is far smaller than exp(-n 1 n 2 p 2 max ), the misclustering rate obtained for our algorithm. A similar phenomena appears for highdimensional Gaussian mixture models. Indeed, as shown in [START_REF] Ndaoud | Sharp optimal recovery in the two-component Gaussian Mixture Model[END_REF], it is essential to capture the hardness of estimating the model parameters in the minimax lower bound in order to get the right rate of convergence. The argument developed for obtaining a lower bound of the minimax risk in Ndaoud, 2018 relies heavily on the Gaussian assumption (they set a Gaussian prior on the model parameters and use the fact that the posterior distribution is also Gaussian) and cannot directly be extended to this setting.

Our lower bound result is summarised in the following theorem.

Theorem 13. Suppose that

A ∼ SBiSBM(Z 1 , Z 2 , p, q) with K = L = 2, n 2 ≫ n 1 log n 1 , n 1 n 2 p 2 max → ∞ and n 1 n 2 p 2 max = O(log n 1 ). Then there exists a constant c 1 > 0 such that inf ẑ sup θ∈Θ E(r(ẑ, z)) ⩾ exp(-c 1 n 1 n 2 p 2 max )
where the infimum is taken over all measurable functions ẑ of A. Moreover, if n 1 n 2 p 2 max = Θ(1), then inf ẑ sup θ∈Θ E(r(ẑ, z)) ⩾ c 2 for some positive constant c 2 .

Remark 17. This lower-bound shows that the rate of convergence of our estimator is optimal up to a constant factor. Indeed, for exact recovery, the minimax lower bound implies that n 1 n 2 p 2 max ≳ log n 1 is necessary, while for weak recovery, we need n 1 n 2 p 2 max → ∞. It also shows that when n 1 n 2 p 2 max = O(1) it is not possible to consistently estimating Z 1 .

Remark 18. For simplicity, we only write the proof for K = L = 2 but we believe it can be extended to the case K = L > 2 by reducing the problem to test between two fixed communities as in [START_REF] Gao | Community detection in degree-corrected block models[END_REF] Proof. The general idea of the proof is to first lower bound the minimax risk by an error occuring in a two hypothesis testing problem and then to replace this hypothesis testing problem by a simpler one.

The steps are detailed below.

step 1. Recall that z, z ′ denote the partition functions associated with Z 1 and Z 2 respectively. We choose as a prior on z and z ′ a product of independent, centered Rademacher distributions. Since the marginals of z, z ′ are sign invariant, By using standard arguments the results in [START_REF] Gao | Community detection in degree-corrected block models[END_REF][START_REF] Ndaoud | Sharp optimal recovery in the two-component Gaussian Mixture Model[END_REF] show that

inf ẑ sup θ∈Θ E(r(ẑ, z)) ≳ 2 n 1 ⌈n 1 /2⌉ i=1 inf ẑi E z,z ′ E A|z,z ′ (ϕ i (A))
where ẑ is a measurable function in A and ϕ i (A) = 1 ẑi ̸ =z i .

step 2. We can write

E z,z ′ E A|z,z ′ (ϕ i (A)) = E z -i E z i E z ′ E A|z,z ′ (ϕ i (A)) = E z -i E z i E A|z (ϕ i (A)) R i
where z -i := (z j ) j̸ =i and E X|Y means that we integrate over the random variable X conditioned on Y. Since z ′ is random, the entries of A are no longer independent (this is the reason why we use an uninformative prior on z ′ ). Note that the quantity

E z ′ E A|z,z ′ (ϕ i (A)) = E A|z (ϕ i (A)
) only depends on f(A|z) the density of A conditionally on z. Since the columns of A are independent conditionally on z (because (z ′ j ) n 2 j=1 are independent) we have that

f(A|z) = j f(A :j |z),
i.e. f(A|z) is the product of the densities of the columns A :j conditionally on z. Let us denote

A -ij := (A i ′ j ) i ′ ̸ =i and A -i: = (A i ′ j ) i ′ ̸ =i,j .
Now for each j we can write

f(A :j |z) = f(A ij |A -ij , z)f(A -ij |z). Since A -ij doesn't depend on z i we have f(A -ij |z) = f(A -ij |z -i ). Assume that z i = 1. Then A ij ∼ B(p) if z ′ j = 1 or A ij ∼ B(q) if z ′ j = -1.
This in turn implies

P(A ij = 1|A -ij , z) = pP(z ′ j = 1|A -ij , z) + qP(z ′ j = -1|A -ij , z) = pP(z ′ j = 1|A -ij , z -i ) + qP(z ′ j = -1|A -ij , z -i )
Let us denote by α j the random variable P(z ′ j = 1|A -ij , z -i ). When z i = -1, similar considerations imply

P(A ij = 1|A -ij , z -i ) = qα j + p(1 -α j ).

This shows that the conditional distribution

A ij |A -ij , z ∼ B(α j p + (1 -α j )q) when z i = 1
(5.6.1)

and

A ij |A -ij , z ∼ B(α j q + (1 -α j )p) when z i = -1.
(5.6.2)

We can write R i as

R i = E z -i E z i E A -i: |z E A i: |z,A -i: (ϕ i (A)) = E z -i E z i E A -i: |z -i E A i: |z,A -i: (ϕ i (A)) = E z -i E A -i: |z -i E z i E A i: |z,A -i: (ϕ i (A)) R ′ i .
The term R ′ i corresponds to the risk associated with the following two hypothesis testing problem (conditionally on

(A i ′ j ) i ′ ̸ =i and z -i ) H 0 : ⊗ n 2 j=1 B(α j p + (1 -α j )q) vs H 1 : ⊗ n 2 j=1 B(α j q + (1 -α j )p). (5.6.3)
Our goal is to replace this two-hypothesis testing problem by a simpler hypothesis test associated with a smaller error. The proof strategy is the following. When α j is very close to 1/2 it is not possible to statistically distinguish (B(α j p + (1 -α j )q) from (B(α j q + (1 -α j )p) and these factors can be dropped. When α j is significantly different from 1/2, then the risk associated to the test can be lower bounded by a the risk of testing between a product of B(p) vs. a product of B(q). More precisely, we will show that the number of indices j for which α j is significantly different from 1/2 is of order n 1 n 2 p and hence the risk is lower bounded by the one associated by the two hypothesis testing problem

H ′ 0 : ⊗ n 2 n 1 p i=1 B(p) ⊗ n 2 n 1 p i=1 B(q) vs H ′ 1 : ⊗ n 2 n 1 p i=1 B(q) ⊗ n 2 n 1 p i=1 B(p).
It is well known that the error associated with the above testing problem is of the order e -Θ(n 2 n 1 p 2 ) (see e.g. [START_REF] Gao | Community detection in degree-corrected block models[END_REF] which corresponds to the rate of convergence of our algorithm. Let us now formalize the above argument.

step 3. First, let us define

C + = {i ′ ̸ = i : z i ′ = 1}, C -= {i ′ ̸ = i : z i ′ = -1}, θ j = α j 1 -α j for j = 1 . . . n 2 , ϵ = Θ(p n 1 log n 1 ) = o(1), J b = {j ∈ [n 2 ] : θ j ∈ [1 -ϵ, 1 + ϵ]}, J g = {j ∈ [n 2 ] : θ j / ∈ [1 -ϵ, 1 + ϵ]}, T j ind.
∼ B(α j ), for all j ∈ J g , and the events

E 1 = |C + | -|C -| ∈ [-C n 1 log n 1 , C n 1 log n 1 ]
(for some constant )C > 0,

E 2 =    j 1 { i ′ ∈C + A i ′ j -i ′ ∈C -A i ′ j ̸ =0} = Θ(n 2 n 1 p)    , E 3 =    j∈J b A ij = Θ(n 2 p)    .
We will show later that these events occur with high probability. They are useful for obtaining a lower bound on the densities f(A ij |A -ij , z). Since we are integrating over positive functions one can write

R i ⩾ E z -i 1 E 1 E A -i: |z i 1 E 2 E z i E A i: |z,A -i: (1 E 3 ϕ i (A)).
(5.6.4) step 4. For all j ∈ J b , the set for which α j ≈ 1 2 , we are going to lower bound the densities f(A ij |A -ij , z) by g(A ij ) corresponding to the density of B( p+q 2 ). A simple calculation shows that

θ j = α j 1 -α j = i ′ ̸ =i:z i ′ =1 p A i ′ j (1 -p) 1-A i ′ j i ′ ̸ =i:z i ′ =-1 q A i ′ j (1 -q) 1-A i ′ j i ′ ̸ =i:z i ′ =1 q A i ′ j (1 -q) 1-A i ′ j i ′ ̸ =i:z i ′ =-1 p A i ′ j (1 -p) 1-A i ′ j .
The previous expression can be rewritten as

θ j = p(1 -q) q(1 -p) i ′ ∈C + A i ′ j -i ′ ∈C -A i ′ j 1 -p 1 -q |C + |-|C -|
.

We also have the relation α j = θ j 1+θ j , so α j is close to 1/2 if and only if θ j is close to 1. If z -i were an exactly balanced partition, i.e.,

|C + | -|C -| = 0, then θ j = 1 would be equivalent to i ′ ∈C + A i ′ j - i ′ ∈C -A i ′ j = 0. However the contribution of the term 1-p 1-q |C + |-|C -|
is small under E 1 . Indeed, we have

|C + | -|C -| ∈ [-C n 1 log n 1 , C n 1 log n 1 ]
. Note that log( 1-p 1-q ) ∈ [-c(p -q), c(p -q)] by using Taylor's formula for some constant c > 0. Hence, under E 1

1 -p 1 -q |C + |-|C -| ∈ [e -c ′ √ n 1 log n 1 p , e c ′ √ n 1 log n 1 p ] for some constant c ′ > 0. But we have max(|e c ′ p √ n 1 log n 1 -1|, |e -c ′ p √ n 1 log n 1 -1|) ⩽ c ′ p n 1 log n 1 := ϵ.
Therefore, it follows that

( 1-p 1-q ) |C + |-|C -| ∈ [1 -ϵ, 1 + ϵ] under E 1 . It is easy to check θ j ∈ [1 -ϵ, 1 + ϵ] implies α j ∈ [1/2 -ϵ ′ , 1/2 + ϵ ′ ] for ϵ ′
proportional to ϵ. Since the constant involved here doesn't matter, we won't make a distinction between ϵ and ϵ ′ . Now recall that by (5.6.1) and (5.6.2) we have when

z i = 1 f(A ij |A -ij , z) = α j p A ij (1 -p) 1-A ij + (1 -α j )q A ij (1 -q) 1-A ij
and when

z i = -1 f(A ij |A -ij , z) = α j q A ij (1 -q) 1-A ij + (1 -α j )p A ij (1 -p) 1-A ij . • When z i = 1, we have for all j ∈ [n 2 ] such that α j ∈ [1/2 - ϵ, 1/2 + ϵ] that α j p A ij (1 -p) 1-A ij + (1 -α j )q A ij (1 -q) 1-A ij ⩾ (1 -ϵ) p + q 2 1 A ij =1 + (1 -pϵ) 1 -p + 1 -q 2 1 A ij =0
and

j∈J b α j p A ij (1 -p) 1-A ij + (1 -α j )q A ij (1 -q) 1-A ij ⩾ (1 -ϵ) j∈J b A ij (1 -pϵ) j∈J b (1-A ij ) × j∈J b p A ij (1 -p) 1-A ij + q A ij (1 -q) 1-A ij 2 . • When z i = -1, we have for all j ∈ [n 2 ] such that α j ∈ [1/2 - ϵ, 1/2 + ϵ] α j q A ij (1 -q) 1-A ij + (1 -α j )p A ij (1 -p) 1-A ij ⩾ (1 -ϵ) p + q 2 1 A ij =1 + (1 -qϵ) 1 -p + 1 -q 2 1 A ij =0
and

j∈J b α j q A ij (1 -q) 1-A ij + (1 -α j )p A ij (1 -p) 1-A ij ⩾ (1 -ϵ) j∈J b A ij (1 -qϵ) j∈J b (1-A ij ) × j∈J b p A ij (1 -p) 1-A ij + q A ij (1 -q) 1-A ij 2 . Since on E 3 , j∈J b A ij = Θ(n 2 p) we obtain that (1 -ϵ) j∈J b A ij ⩾ 1 -n 2 ϵp = 1 -o(1)
and similarly

(1 -pϵ) j∈J b (1-A ij ) ⩾ 1 -o(1) and (1 -qϵ) j∈J b (1-A ij ) ⩾ 1 -o(1).
This implies the lower bound

1 E 3 j∈J b f(A ij |A -ij , z) ⩾ (1 -o(1))1 E 3 j∈J b g(A ij ).
Consequently, we obtain the lower bound

E z i E A i: |z,A -i: (1 E 3 ϕ i (A))
⩾ (1 -o(1))

z i A i: 1 E 3 ϕ i (A) j∈J b g(A ij ) j∈J g f(A ij |A -ij , z)dA i: dP(z i ) ≳ z i (A ij ) j∈J b (A ij ) j∈Jg j∈J b g(A ij )   ϕ i (A) j∈J g f(A ij |A -ij , z)d(A ij ) j∈J g   d(A ij ) j∈J b dP(z i ) -PA i: |A -i: ,z (E c 3 ) ≳ z i (A ij ) j∈J b j∈J b g(A ij ) (A ij ) j∈Jg   ϕ i (A) j∈J g f(A ij |A -ij , z)d(A ij ) j∈J g   d(A ij ) j∈J b dP(z i ) -PA i: |A -i: ,z (E c 3 ) (since J b and J g are disjoint) ≳ (A ij ) j∈J b j∈J b g(A ij )   z i (A ij ) j∈Jg ϕ i (A) j∈J g f(A ij |A -ij , z)d(A ij ) j∈J g   R ′′ i d(A ij ) j∈J b dP(z i ) -PA i: |A -i: ,z (E c 3 ) (since g(A ij ) is independent from z i )
where PA i: |A -i: ,z is the conditional distribution corresponding to the density product

j∈J b g(A ij ) j∈J g f(A ij |A -ij , z)dP(z i ).
step 5. R ′′ i corresponds to risk associated to the testing problem (5.6.3) where the product is restricted to the set J g . One can lower bound this risk as follows.

By definition of T j , we have when z i = 1 by equation (5.6.1)

f(A ij |A -ij , z) = E T j T j p A ij (1 -p) 1-A ij + (1 -T j )q A ij (1 -q) 1-A ij :=E T j f T j (A ij )
.

A similar result holds when z i = -1. Since T j is independent of z i (it only depends on A -ij and z -i through α j ) we can write

R ′′ i = E T E z i (A ij ) j∈Jg ϕ i (A) j∈J g f T j (A ij )d(A ij ) j∈J g R ′′′ i = E T (R ′′′ i )
where T = (T j ) j∈J g . But now R ′′′ i is the risk associated with the following two hypothesis testing problem

H ′′ 0 : ⊗ T j =1 B(p) ⊗ T j =0 B(q), vs H ′′ 1 : ⊗ T j =1 B(q) ⊗ T j =0 B(p).
The error associated with this test is lower bounded by the error associated with the test

H ′′′ 0 : ⊗ j∈J g B(p) ⊗ j∈J g B(q), vs H ′′′ 1 : ⊗ j∈J g B(q) ⊗ j∈J g B(p).
since adding more information can only decreases the error (more formally, the lower bound follows from the fact that we multiply by density functions inferior to one). Under E 2 ∩ E 1 we have

|J g | = Θ(n 1 n 2 p) and |J b | = (1 -o(1))n 2 .
(5.6.5) Hence, by Lemma 4 in [START_REF] Gao | Community detection in degree-corrected block models[END_REF] the risk is lower bounded by

R ′′ i ⩽ e -Θ(n 1 n 2 p 2 ) .
conclusion. It remains to integrate over all the events we conditioned on. We have shown sor far that

R i ⩾ E z -i 1 E 1 E A -i: |z -i 1 E 2 e -Θ(n 1 n 2 p 2 ) -E z -i 1 E 1 E A -i: |z -i 1 E 2 PA i: |A -i: ,z (E c 3 ) ⩾ e -Θ(n 1 n 2 p 2 ) E z -i (1 E 1 P A -i: |z -i (E 2 )) -E z -i 1 E 1 E A -i: |z -i 1 E 2 PA i: |A -i: ,z (E c
3 ).

(5.6.6)

We can control these probabilities with the following lemma proved in the appendix, see Section 5.8.2.

Lemma 18. We have

1. P z -i (E 1 ) ⩾ 1 -n -Ω(1) 1 ; 2. for all realizations z -i , P A -i: |z -i (E 2 ) ⩾ 1 -e -Ω(n 1 n 2 p) ; 3. for all z -i ∈ E 1 and A -i: ∈ E 2 , PA i: |A -i: ,z (E 3 ) ⩾ 1 -e -Ω(n 2 p) .
Lemma 18 and (5.6.6) directly imply R i ≳ e -Θ(n 1 n 2 p 2 ) since e -Θ(n 2 p) = o(e -Θ(n 1 n 2 p 2 ) )

under the assumption on p.

When n 1 n 2 p 2 max = Θ(1), we can use the same proof except that now the final two hypothesis testing problem we reduced to has a non vanishing risk (see e.g. Zhang et al., 2016a).

numerical experiments

In this section, we empirically compare the performance of our algorithm (GPM) with the spectral algorithm (Spec) and the algorithm introduced by Ndaoud et al., 2022 (referred to as HL).

case K = L = 2. In this setting, we generate a SBiSBM with parameters n 1 = 500, n 2 = ⌈Cn 1 log n 1 ⌉ (where C ⩾ 1 is a constant), p ∈ (0, 1), and q = cp where c > 0 is a constant. The accuracy of the clustering is measured by the NMI; it is equal to one when the partitions match exactly and is zero when they are independent. The results are averaged over 20 Monte-Carlo runs. For the experiment presented in Figure 16, we fixed C = 10; for the experiment in Figure 17, we fixed C = 3 and c = 0.5. For the experiment presented in Figure 18, we fixed p = 0.01 and c = 0.5. We observe that HL and GPM have similar performance in all the aforementioned experiments. Thus, there is no gain in using the specialized method HL instead of the general algorithm GPM. The spectral method Spechas only slightly worst performance than the iterative methods HL and GPM. In particular, when approaching the threshold for exact recovery, the performance gap disappears. This suggests that Spec also reaches the threshold for exact recovery. It would be interesting to obtain stronger theoretical guarantees to explain the good performance of Spec.

case K = L ̸ = 2. We fix n 1 = 1000, n 2 = 10000, p = 0.05 and c = 0.5 and vary K from 2 to 10. As can be seen from Figure 19, the performance of Spec decreases faster than GPM when K increases.

Ω i,k (δ) = Ẽi: (W :z i -W :k ) ⩽ -(1 -δ)∆ 2 (z i , k) .
We have for all

0 < δ ⩽ 1 4ηα , k ̸ = z i ∈ [K] and i ∈ [n 1 ] P(Ω i,k (δ)) ⩽ e -β 2 12eLα 3 n 1 n 2 p 2 max KL = e -∆2 .
Proof. The event Ω i,k (δ) holds if and only if

B i: (W :k -W :z i ) ⩾ -δ∆ 2 (z i , k).
But we can decompose this quantity as

B i: (W :k -W :z i ) = j̸ =i B ij (W jk -W jz i ) = |C k | -1 j∈C k \{i} ⟨A i: , A j: ⟩ -|C z i | -1 j∈C z i \{i} ⟨A i: , A j: ⟩ = ⟨A i: , Ãk -Ãz i ⟩ where Ãk = 1 |C k | j∈C k A j: and Ãz i = 1 |C z i |
j∈C z i \{i} A j: are independent random variables. Since the index i doesn't appear in the second sum, Ãz i is independent from A i: . By definition the entries of Ãk and Ãz i are independent normalized binomial random variables whose parameters vary depending on the community associated with the entry.

We are now going to bound the moment generating function of M := ⟨A i: , Ãk -Ãz i ⟩, conditionally on A i: . Observe that M is a sum of independent random variables. Recall that if X ∼ B(p) then E(e tX ) = (e t p + 1 -p). Hence, for t > 0, conditionally on A ij , each summand has a m.g.f equal to log E(e tA ij ( Ãkj -

Ãz i j ) |A ij ) = |C k | log(e tA ij |C k | Π kz ′ j + 1 -Π kz ′ j ) + (|C z i | -1) log(e - tA ij |Cz i | Π z i z ′ j + 1 -Π z i z ′ j ) ⩽ |C k |Π kz ′ j (e tA ij |C k | -1) + (|C z i | -1)Π z i z ′ j (e - tA ij |Cz i | -1)
by using the fact that log(1 + x) ⩽ x for all x > -1. Fix t = t * = ϵ n 1 αK for a parameter ϵ ∈ (0, 1) that will be fixed later. We have by Taylor Lagrange formula for all t ⩽ n 1 αK e

t |C k | -1 ⩽ t |C k | + e 2 t |C k | and e -t |Cz i | -1 ⩽ - t |C z i | + e 2 t |C z i | 2 .
By using these upper bounds we get log E(e

t * A ij ( Ãkl -Ãz i j ) |A ij ) ⩽ Π kz ′ j - |C z i | -1 |C z i | Π z i z ′ j t * A ij + e 2 (t * ) 2 A ij Π kz ′ j |C k | + Π z i z ′ j |C z i | .
Hence by using independence we obtain

log E(e t * M |A i: ) ⩽ j∈[n 2 ]       Π kz ′ j - |C z i | -1 |C z i | Π z i z ′ j t * + e 2 (t * ) 2 Π kz ′ j |C k | + Π z i z ′ j |C z i | t ij       A ij .
Using Markov inequality and the fact that ∆ 2 (z i , k) ⩽ ηβαn 2 p 2 max /L by (5.5.3) leads to

P(Ω i,k (δ)|A i: ) ⩽ e δϵ n 1 αK ∆ 2 (z i ,k) E(e t * M |A i: ) ⩽ e δϵ βn 1 n 2 p 2 max LK j E(e t ij A ij ).
But since A ij is a Bernoulli random variable with parameter Π z i z 2j

we have E(e t ij A ij ) = (e t ij -1)Π z i z 2j + 1.

By our choice of t * , t ij = O(n 1 p max ) = o(1) so that j (e t ij -1)Π z i z 2j + 1 ⩽ e j (e t ij -1)Π z i z 2j .

Here we use the fact that for x 1 , . . . , x n > -1 we have i∈ [n] (1 +

x i ) ⩽ e i∈ [n] x i . But again, by using Taylor Lagrange formula, we have

e t ij -1 = t ij + O(t 2 ij ). Consequently j E(e t ij A ij ) ⩽ e j (t ij +o(t ij ))Π z i z 2j .
We can write j t ij Π z i z 2j as t * A 1 + e(t * ) 2 2 A 2 where

A 1 = l∈[L] |C ′ l | Π z i l Π kl -Π 2 z i l |C z i | -1 |C z i | and A 2 = l∈[L] |C ′ l | Π z i l Π kl |C k | + Π 2 z i l |C z i | -1 |C z i | 2 .
By Assumption A3 and A5 we have

-A 1 ⩾ n 2 αL βp 2 max + o(p 2 max )
and

A 2 ⩽ αK n 1 n 2 p 2 max Let ϵ = β 3eLα (recall that t * = ϵ n 1 αK )
. By this choice of ϵ we have

eϵn 1 2αK A 2 ⩽ |A 1 | 2 .
Consequently we have for all δ ⩽ 1/(4ηα)

P(Ω i,k (δ)) ⩽ e -ϵβ 2α 2 n 1 n 2 p 2 max KL +δϵ n 1 αK ∆ 2 (z i ,k) ⩽ e -ϵβ 4α 2 n 1 n 2 p 2 max KL ⩽ e -β 2 12eLα 3 n 1 n 2 p 2 max KL .

Proof of Lemma 18

The first inequality is a direct consequence of Hoeffding concentration inequality. For the second inequality, observe that

P A -i: |z -i (E 2 ) = E z ′ P A -i: |z -i ,z ′ (E 2 ) and P A -i: |z -i ,z ′   i ′ ∈C + A i ′ j - i ′ ∈C - A i ′ j = 0   ⩾ P(A i ′ j = 0 for all i ′ ) ≳ (1 -p) n 1 = 1 -n 1 p.
To obtain an upper bound on this probability, we can use Paley-Zigmund inequality as follows. Let us denote

Z = | i ′ ∈C + A i ′ j - i ′ ∈C -A i ′ j |. We then have P A -i: |z -i ,z ′ (Z = 0) = 1 -P A -i: |z -i ,z ′ (Z > 0) ⩽ 1 -P A -i: |z -i ,z ′ (Z > θEZ) ⩽ 1 -(1 -θ) 2 (E A -i: |z -i ,z ′ (Z)) 2 E A -i: |z -i ,z ′ (Z 2 )
for θ ∈ (0, 1) by Paley-Zigmund inequality. It is easy to check that

E A -i: |z -i ,z ′ Z ≳ n 1 (p -q) and E A -i: |z -i ,z ′ (Z 2 ) ≍ n 1 p.
Consequently, P A -i: |z -i ,z ′ (Z = 0) = 1 -Θ(n 1 p). By using independence over j and Chernoff's multiplicative bound we obtain that

P A -i: |z -i ,z ′   j 1 { i ′ ∈C + A i ′ j -i ′ ∈C -A i ′ j ̸ =0} = Θ(n 2 n 1 p)   ⩾ 1 -e -cn 2 n 1 p
for any realization of z -i and z ′ . Hence the stated bound follows.

Finally it remains to control the probability of E 3 . Under PA i: |A -i: ,z , for each j ∈ J b , A ij are independent and distributed as a mixture of Bernoulli of parameters p and q. The size of the set J b is also controlled by the assumption A -i: ∈ E 2 (see equation (5.6.5)). Since under PA i: |A -i: ,z for all j ∈ J b , A ij are independent r.v. with density g, we obtain by stochastic domination (we can replace a mixture of Bernoulli with parameters p and q by a Bernoulli of parameter p) and Chernoff's bound that PA i:

|A -i: ,z (E 3 ) ⩾ 1 -e -Θ(n 2 p) .
The objective of the graph matching problem is to find a bijective correspondence between the vertices of two graphs G and H such that the alignment between the edges of G and H is maximized (see Section 1.5). This problem appears in many applications such as computer vision (H. [START_REF] Sun | A Survey On Graph Matching In Computer Vision[END_REF], network de-anonymization [START_REF] Narayanan | De-anonymizing Social Networks[END_REF], pattern recognition [START_REF] Conte | Thirty Years Of Graph Matching In Pattern Recognition[END_REF][START_REF] Emmert-Streib | Fifty Years of Graph Matching, Network Alignment and Network Comparison[END_REF], protein-protein interactions and computational biology (Zaslavskiy et al., 2009b;[START_REF] Singh | Global alignment of multiple protein interaction networks with application to functional orthology detection[END_REF]. In computer vision, for example, it is used as a method of comparing two objects (or images) encoded as graph structures or to identify the correspondence between the points of two discretized images of the same object at different times. In network de-anonymization, the goal is to learn information about an anonymized (unlabeled) graph using a related labeled graph as a reference, exploiting their structural similarities. For example, [START_REF] Narayanan | How To Break Anonymity of the Netflix Prize Dataset[END_REF] show that it was possible to effectively de-anonymize the Netflix database using the IMDb (Internet Movie Database) as the "reference" network. It also contains the ubiquitous graph isomorphism as a special case.

In general, the graph matching problem is NP-hard. But under generative models such as the CoERM and CoWM (see Section 1.5 for a precise description) there exists efficient algorithms. Recently, other models of correlation have been proposed for random graphs with a latent geometric structure [START_REF] Kunisky | Strong recovery of geometric planted matchings[END_REF]Wang et al., 2022a), community structure [START_REF] Racz | Correlated Stochastic Block Models: Exact Graph Matching with Applications to Recovering Communities[END_REF] and with power law degree profile (Yu et al., 2021b).

In this thesis, we will focus on the Correlated Erdös-Renyi Model (CoERM) and the Correlated Wigner Model (CoWM). In Section 6.1, we will present the information-theoretic limits of the graph matching problem under these generative models. Then, in Section 6.2 we will discuss the principal efficient methods that have been proposed in the literature.

information theoretic limits of matching

The accuracy of an estimated permutation x to the ground-truth permutation x * is usually measured by the fraction of well-matched nodes overlap

(x, x * ) := |{i ∈ [n] : x(i) = x * (i)}| n . (6.1.1)
Similarly to the clustering problem, one can have different recovery requirements.

• Exact recovery: overlap(x, x * ) = 1 with probability 1 -o(1) when n → ∞, i.e. one can recover exactly the latent permutation w.h.p.

• Almost exact recovery: overlap(x, x * ) = 1 -o(1) with probability 1 -o(1) when n → ∞, i.e. the proportion of correctly matched nodes goes to zero when n → ∞. 1

• Partial recovery: overlap(x, x * ) ⩾ α with probability 1 -o(1) when n → ∞, i.e. one can correctly classify at least a proportion α of the nodes w.h.p.

Correlated Wigner Model (CoWM)

In the Gaussian setting, a sharp phase transition phenomenon occurs.

There is a threshold involving n and the noise level σ above which exact recovery of x * is possible and below which even partial recovery is impossible. Recall that the MLE is given by xMLE ∈ arg max

x∈S n i,j A ij B x(i)x(j) .
Theorem 14 [START_REF] Wu | Settling the sharp reconstruction thresholds of random graph matching[END_REF]. Assume that (A, B) ∼ CoWM(n, σ, x * ) and let us define ρ = √ 1 -σ 2 . If for some constant ϵ > 0

ρ 2 ⩾ (4 + ϵ) log n n ,
then xMLE = x * w.h.p. Conversely, if for some constant ϵ > 0 ρ 2 ⩽ (4 -ϵ) log n n , then for any estimator x and fixed constant δ > 0, P(overlap(x, x * ) ⩽ δ) = 1 -o(1).

Correlated Erdös-Renyi Model (CoERM)

In the dense regime, the recovery of x * exhibits a sharp phase transition similar to the Correlated Wigner model: there is no partial recovery regime.

Theorem 15 [START_REF] Wu | Settling the sharp reconstruction thresholds of random graph matching[END_REF]. Assume that (A, B) ∼ CoERM(n, q, s, x * ) where q is bounded away from 1 and q = n -o(1) . If for some ϵ > 0

nqs 2 ⩾ (2 + ϵ) log n log 1 q -1 + q , 1
There is an alternative definition where we only require to be able to correctly match an arbitrary large fraction of nodes. See [START_REF] Wu | Settling the sharp reconstruction thresholds of random graph matching[END_REF] Here we preferred to use an analogous definition to the one used for clustering.

then for any constant δ < 1, P(overlap(x MLE , x * ) ⩾ δ)) = 1 -o(1).

Conversely, if for some ϵ > 0 nqs 2 ⩽ (2 -ϵ) log n log 1 q -1 + q , then for any estimator x and any constant δ > 0, P(overlap(x MLE , x * ) ⩽ δ) = 1 -o(1).

The threshold for exact-recovery in the Erdös-Renyi model when q is bounded away from 1 is given by

nqs 2 (1 - √ q) 2 log n = 1
as shown in Theorem 4 in [START_REF] Wu | Settling the sharp reconstruction thresholds of random graph matching[END_REF] However, in the sparse regime, there is an interesting regime where partial recovery is possible but almost exact recovery impossible.

Theorem 16 [START_REF] Wu | Settling the sharp reconstruction thresholds of random graph matching[END_REF]. Assume that (A, B) ∼ CoERM(n, q, s, x * ) where q = n -Ω(1) . If for some constant ϵ > 0

nqs 2 ⩾ (2 + ϵ) max( log n log(1/q) , 2),
then there exists a constant δ > 0 such that P(overlap(x MLE , x) ⩾ δ) = 1 -o(1). Conversely, assuming nq = ω(log 2 n), if for some ϵ > 0 nqs 2 ⩽ 1 -ϵ, then for any estimator x and any constant δ > 0, P(overlap(x, x) ⩽ δ) = 1 -o(1).

Indeed, this theorem implies that the threshold for partial recovery is at nqs 2 ≍ 1, but it has been shown by [START_REF] Cullina | Partial Recovery of Erdős-Rényi Graph Alignment via k-Core Alignment[END_REF] that one needs nqs 2 → ∞ in order to get almost exact recovery. So there is a regime where partial recovery is possible but not almost exact recovery.

Remark 19. While the statistical limits of graph matching under the CoERM or CoWM are well understood, it is not known if there are polynomial time algorithms that achieve the threshold for exact recovery. Usually the existing methods that come with consistency guarantees assume that the correlation level is asymptotically one. The only exception is the recent work of Mao et al., 2021a, that proposed a polynomial time algorithm that comes with consistency guarantees for a constant level of correlation s < 1 under the CoERM.

matching methods

Convex Quadratic Programming (CQP) relaxation

As saw in Chapter 1, Section 1.5, QAP is a NP-hard problem. A popular approach is to consider a relaxation of the set of permutations to a continuous domain, leading to a tractable optimization problem. There are several ways to do this, we recall some of them below. First observe that (1.5.2) can be rewritten in the following form min

X∈S n A -XBX ⊤ 2 F ⇔ min X∈S n ∥AX -XB∥ 2 F .
If we replace the set of permutations S n by the space of doubly stochastic matrices Since P n is a convex set (it is the Birkhoff polytope and is the tightest convex relaxation possible), the previous optimization problem can be solved in polynomial time and leads to a solution X. Since in general X is not a permutation, one needs to project X on S n . This can be done by solving the following LAP X = arg max

P n = {X ∈ R n×n : X1 n = 1 n , X ⊤ 1 n = 1 n , X ij ⩾ 0}

X∈S n

Tr(X ⊤ X).

In practice, the method performs well. But it remains an open problem to derive consistency guarantees for this algorithm. Besides, the computational cost of this method can be prohibitive for large graphs.

SDP relaxation

The QAP can be rewritten in a vectorial form arg max

XS n Tr((B ⊗ A)vec(X)vec(X) ⊤ )
where ⊗ denotes the Kronecker product and vec(X) ∈ R n 2 is the column-wise version of X. [START_REF] Zhao | Semidefinite Programming Relaxations For The Quadratic Assignment Problem[END_REF] proposed to relax the constraints on X by replacing S n by P n as before and replace vec(X)vec(X) ⊤ by a matrix Q satisfying some constraints. More precisely, they consider the following optimization problem. max

Q,P∈R n×n Tr((B ⊗ A)Q) s.t. P ∈ P n Q -vec(P)vec(P) ⊤ ⪰ 0, i Q(i, i) = I n , Tr(Q(i, j)) = 0 for i ̸ = j, Tr(Q(i, j)1 n 1 ⊤ n ) = 1 for all i, j, all entries of Q are non-negative, (Q(i, i)) jj = P ji ,
where Q(i, j) ∈ R n×n is the block (i, j) of size n × n that constitutes Q. See also S. [START_REF] Ferreira | Semidefinite Programming Approach for the Quadratic Assignment Problem with a Sparse Graph[END_REF] for other relaxations.

From a computational point of view, this approach is feasible since it requires solving an SDP problem in a space of dimension n 2 .

Spectral methods

The spectral method GRAMPA introduced by Fan et al., 2019 can be related view as a solution of a relaxed and regularized version of the QAP for a given hyperparameter η > 0

min X∈R n×n ∥AX -XB∥ 2 F + η 2 2 ∥X∥ 2 F -1 ⊤ n X1 n . (6.2.2)
Since it is a convex quadratic optimization problem, we can write the solution X in closed form. Write the spectral decomposition of A and

B as A = i∈[n] λ i u i u ⊤ i and B = j∈[n] µ j v j v ⊤
X = i,j∈[n] w(λ i , µ j )u i u ⊤ i 1 n 1 ⊤ n v j v ⊤ j
where w(x, y) is the Cauchy Kernel of bandwidth η defined by

w(x, y) = 1 (x -y) 2 + η 2 .
It can be shown that under the correlated Wigner model W(n, σ, x * ), with a noise level σ of order at most O( 1 log n ), GRAMPA exactly recovers the ground-truth permutation x * for a suitable choice of η (see Theorem 2.1 in [START_REF] Fan | Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model[END_REF]. In practice, GRAMPA has good performance, but its computational cost is O(n 3 ). A faster (but less robust to noise) spectral method has also been proposed by Ganassali et al., 2022a. Another popular approach is Umeyama's algorithm [START_REF] Umeyama | An eigendecomposition approach to weighted graph matching problems[END_REF]. Here we construct a similarity matrix X based on the absolute value of the eigenvectors of A and B. More precisely

X = n i=1 |u i ||v i | ⊤
where |v| denotes the entrywise absolute value of vector v.

Message passing algorithm

Let (A, B) ∼ CoERM(n, q, s, x * ). Recently Ganassali et al., 2022b andindependently Piccioli et al., 2022 proposed a matching method based on a message passing algorithm for Correlated ER model that are sparse enough, i.e. when the sparsity q satisfies q = λ n for some constant λ. The idea, as explained in [START_REF] Piccioli | Aligning random graphs with a sub-tree similarity message-passing algorithm[END_REF], is to take advantage of the local tree-like structure of the extended neighborhood of each node in order to approximate the posterior distribution P(x * (i) = i ′ |A, B) for each i, i ′ (where x * is generated uniformly at random). If instead of conditioning on A and B we only condition on A i,d (resp. B i ′ ,d ) the set of nodes at distance d from i (resp i ′ ) in A (resp. B) one can show that the posterior distribution

P(x * (i) = i ′ |A i,d , B i ′ ,d
) can be written in close form when n → ∞ and d is chosen such that the neighborhoods of i and i ′ at distance d are trees. More precisely for any trees T , T ′ of length d By (6.2.3), determining if the posterior distribution

lim n→∞ P(x * (i) = i ′ |A i,d = T , B i ′ ,d = T ′ ) = n P (d) 1 (T , T ′ ) P (d) 0 (T )P (d) 0 (T ′ ) (6.2.
P(x * (i) = i ′ |A i,d = T , B i ′ ,d = T ′ )
is superior or inferior to 1/2 is equivalent to testing between the distributions P

L (d) (T , T ′ ) = P (d) 1 (T , T ′ ) P (d) 0 (T )P (d) 0 (T ′ )
in a recursive form so that it can be computed in polynomial time. But in practice, the algorithm has limited applications because due to its complexity that is exponential in λ. From a theoretical point of view, Ganassali et al., 2022b have shown that a one-sided test between the two hypotheses can succeed when the Kullback-Leibler (KL) divergence between the two probabilities goes to infinity with d and λs > 1.

Signature matching

In their groundbreaking work, Mao et al., 2021a proposed a matching method based on signatures which are statistics associated with each node that captures some of their particularities. The basic idea shares some similarities with the message passing approach: locally most of the nodes' neighborhood is a tree. Hence, in order to match nodes, it would be sufficient to match trees. Unfortunately, since these trees can have at each level number of nodes, it is not possible to compute all the possible permutations to find the one that best matches the tree. The idea is to construct for node i a partition tree rooted at i and use the information provided by the degrees of the neighbors to label the tree.

More formally, let (A, B) ∼ CoERM(n, q, s, x * ) and G, G ′ the associated graphs. Given a node i and a depth m we iteratively construct sets T k s for k ∈ [m] and s ∈ {-1, 1} k as follows. We start with the root T 0 ∅ = {i}. Assume that all T k s have been constructed. Let us denote by N(T ) the set of neighbors in G of all the nodes belonging in the set T and S(i, l) the sets of nodes that are exactly at distance l from i in G. Then we can define the two children T k+1 (s,-1) and T k+1 (s,+1) of T s k by

T k+1 (s,-1) = j ∈ N(T k s ) ∩ S(i, k + 1) : deg(j) < np and T k+1 (s,+1) = j ∈ N(T k s ) ∩ S(i, k + 1) : deg(j) ⩾ np .
The collection of sets (T k s ) s,k encodes information about the path of length m starting from i. These paths are labeled with s depending on the degrees of the nodes on the path. Based on these paths we can associate to each node i a signature (f(i),

v(i)) ∈ R 2 m × R 2 m defined by f(i) = (f s (i)) s and v(i) = (v s (i)) s with f s (i) = j∈N(T m s )∩S(i,m+1) (deg(j) -1 -np) and v s (i) = np(1 -p)|N(T m s ) ∩ S(i, m + 1)|. Then two nodes i ∈ G and i ′ ∈ G ′ are matched if T := 1 |J| s∈J (f s (i) -f ′ s (i ′ )) 2 v s (i) + v ′ s (i ′ ) < 1 - 1 log n
where J is a subset of {-1, 1} m chosen uniformly at random and

(f ′ (i ′ ), v ′ (i ′ ))
is the signature associated to the node i ′ in G'. The intuition is as follows. When there is no noise, s = 0, the graphs G and G ′ are the same and consequently if i ′ = x * (i) the signature of i and i ′ should be the same. On the other hand, if i ′ ̸ = x * (i) the signatures are likely to be different and hence the variance measured by the statistic T should be larger than in the case where i ′ = x * (i).

It has been shown by Mao et al., 2021a that even when the noise s is of constant order, the previously described method works. This is the first theoretical result in graph matching that allows a constant order noise, all the previous results assumed that s = o(1). The computational cost of the algorithm is O(n 2 log n c ) for some constant c > 0. So it is faster than GRAMPA.

S E E D E D G R A P H M AT C H I N G W I T H T H E G E N E R A L I Z E D P O W E R M E T H O D

This chapter is based on the article [START_REF] Araya | Seeded graph matching for the correlated Wigner model via the projected power method[END_REF]

introduction

In this chapter, we will focus on the setting where side information is provided under the form of a noisy seed, which is a partial matching between the graphs. The seeded version of the problem is motivated by the fact that in many applications, a set of correctly matched vertices can be available as prior information. Several algorithms, based on different techniques, have been proposed for seeded graph matching. In [START_REF] Pedarsani | On the privacy of anonymized networks[END_REF][START_REF] Yartseva | On the Performance of Percolation Graph Matching[END_REF], the authors use a percolation based method to "grow" the seed to recover (at least partially) the ground-truth matching. Other algorithms [START_REF] Lubars | Correcting the Output of approximate Graph matching algorithms[END_REF]Yu et al., 2021a) construct a similarity matrix between the vertices of both graphs and then solve the maximum linear assignment problem (either optimally or by a greedy approach) using the similarity matrix as the cost matrix. The latter strategy has also been successfully applied in the case described below, when no side information is provided.

In this chapter we analyse the performance of the projected power method (PPM) for the seeded graph matching problem in the context of the correlated Wigner model. This family of iterative algorithms has recently been successfully applied to several problems in machine learning and statistics [START_REF] Chen | The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences[END_REF][START_REF] Boumal | Nonconvex Phase Synchronization[END_REF][START_REF] Wang | Optimal Non-Convex Exact Recovery in Stochastic Block Model via Projected Power Method[END_REF], in clustering in particular (see Chapter 3). We prove that PPM can exactly recover the ground-truth permutation provided that a sufficiently good initial permutation is provided. Our analysis extends the analysis of the refinement algorithm Mao et al., 2021a, Alg.4 to the case of (dense) Wigner graphs and represents, to best of our knowledge, the first analysis of PPM in the dense regime. The main technical difficulty in proving the convergence of PPM lies in proving that each step of the algorithm is a contraction, which requires establishing a uniform bound for the error in a neighborhood of the ground truth. As a byproduct of our analysis, we see that PPM provides a general framework which generalizes some of the state-of-the-art algorithms in the seeded case, such as Yu et al., 2021a, Alg.1, Lubars et al., 2018, Alg.2 and Mao et al., 2021a, Alg.4. contributions. The main contributions of this work can be summarized as follows.

• We provide (see Theorems 17, 18) exact and partial recovery guarantees under the Gaussian Wigner model when the PPM is initialized with a given data-independent seed, and only one iteration of the PPM algorithm is performed. For this result to hold, it suffices that the overlap of the seed with the groundtruth permutation is Ω( n log n). This matches the best-known bound for the sparse Erdös-Renyi case (Yu et al., 2021a), for which an overlap of Ω( n log n) is required to obtain exact recovery.

• We prove (see Theorem 19) that when multiple iterations are allowed, then PPM converges to the ground-truth matching in O(log n) iterations provided that it is initialized with a seed with overlap Ω (1 -κ)n , for a constant κ small enough, even if the initialization is data-dependent or adversarial. This extends the results in (Mao et al., 2021a) from the sparse Erdös-Renyi setting, to the dense Wigner case.

• We complement our theoretical results with experiments on synthetic data showing that PPM can help to significantly improve the accuracy of the matching (for correlated Wigner model) compared to that obtained by a standalone application of existing seedless methods.

Related work

projected power method (ppm). PPM, which is also often referred to as a generalized power method (GPM) in the literature, is a family of iterative algorithms for solving constrained optimization problems. It has been used with success for various tasks including clustering SBM [START_REF] Wang | Optimal Non-Convex Exact Recovery in Stochastic Block Model via Projected Power Method[END_REF], group synchronization [START_REF] Boumal | Nonconvex Phase Synchronization[END_REF][START_REF] Gao | Optimal Orthogonal Group Synchronization and Rotation Group Synchronization[END_REF], joint alignment from pairwise difference [START_REF] Chen | The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences[END_REF], low rank-matrix recovery [START_REF] Chi | Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview[END_REF] and the generalized orthogonal procrustes problem [START_REF] Ling | Generalized Power Method for Generalized Orthogonal Procrustes Problem: Global Convergence and Optimization Landscape Analysis[END_REF]. It is a useful iterative strategy for solving non-convex optimization problems as shown in the previous chapters, but it usually requires a good enough initial estimate. The use of PPM for graph matching was first proposed and experimentally analysed in [START_REF] Onaran | Projected power iteration for network alignment[END_REF] and it has been subsequently been analysed in the case of sparse Erdös-Renyi graphs in [START_REF] Lubars | Correcting the Output of approximate Graph matching algorithms[END_REF]Yu et al., 2021a) (only for one iteration) and in (Mao et al., 2021a) (although the connection with PPM is not mentioned in those works).

graph matching. When no side information is available, several polynomial time algorithms have been proposed recently relying on spectral methods [START_REF] Umeyama | An eigendecomposition approach to weighted graph matching problems[END_REF][START_REF] Fan | Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model[END_REF]Ganassali et al., 2022a;[START_REF] Feizi | Spectral Alignment of Graphs[END_REF][START_REF] Cour | Balanced Graph Matching[END_REF], degree profiles [START_REF] Ding | Efficient random graph matching via degree profiles[END_REF][START_REF] Dai | Analysis of a Canonical Labeling Algorithm for the Alignment of Correlated Erdős-Rényi Graphs[END_REF], other vertex signatures (Mao et al., 2021a), random walk based approaches [START_REF] Singh | Global alignment of multiple protein interaction networks with application to functional orthology detection[END_REF][START_REF] Kazemi | On the Structure and Efficient Computation of IsoRank Node Similarities[END_REF][START_REF] Gori | Graph matching using random walks[END_REF], convex and concave relaxations [START_REF] Aflalo | On convex relaxation of graph isomorphism[END_REF][START_REF] Lyzinski | Graph Matching: relax at your own risk[END_REF]Zaslavskiy et al., 2009a), and other nonconvex methods [START_REF] Yu | Generalizing graph matching beyond quadratic assignment model[END_REF][START_REF] Xu | Scalable Gromov-Wasserstein Learning for Graph Partitioning and Matching[END_REF][START_REF] Onaran | Projected power iteration for network alignment[END_REF].

Most of the previous algorithms have theoretical guarantees only in the low noise regime. A more complete discussion of the problem can be found in Chapter 6. seeded algorithms. Different algorithms have been proposed when a seed of the the form

S = {(i, i ′ ) : i ∈ V(G), i ′ ∈ V(H)}
is given as side information [START_REF] Pedarsani | On the privacy of anonymized networks[END_REF][START_REF] Yartseva | On the Performance of Percolation Graph Matching[END_REF][START_REF] Mossel | Seeded Graph Matching via Large Neighborhood Statistics[END_REF][START_REF] Fishkind | Seeded graph matching[END_REF][START_REF] Lubars | Correcting the Output of approximate Graph matching algorithms[END_REF]Yu et al., 2021a). Many algorithms in this class work under the assumption that the information provided by the seed corresponds perfectly to the ground truth permutation, i.e., (i, i ′ ) ∈ S if and only if x * (i) = i ′ . Some algorithms relax this requirement allowing "noisy" seeds, where for some (i, i ′ ) in S it happens that x * (i) ̸ = i [START_REF] Yartseva | On the Performance of Percolation Graph Matching[END_REF][START_REF] Kazemi | Growing a Graph Matching from a Handful of Seeds[END_REF][START_REF] Lubars | Correcting the Output of approximate Graph matching algorithms[END_REF]Yu et al., 2021a;Mao et al., 2021a). Most of the previous work on the seeded version of the problem has been devoted to the Erdös-Renyi model, under different assumptions on the sparsity. To the best of our knowledge, the state-of-art algorithm in this category is the j-hop algorithm (Yu et al., 2021a, Alg.1), although it shares similarities with (Lubars et al., 2018, Alg.2) and (Mao et al., 2021a, Alg.4). On the other hand, it will be evident from our analysis of PPM for graph matching that those algorithms can also be seen as examples of the PPM.

7.2 algorithm 7.2.1 Projected power method for Graph matching

The projected power method (PPM) has been used to solve the graph matching problem, and its variants, by several authors [START_REF] Onaran | Projected power iteration for network alignment[END_REF]. Most of the work so far has been empirical and, to the best of our knowledge, theoretical guarantees have been obtained only in the case of sparse Erdös-Renyi graphs, such as in (Mao et al., 2021a, Thm.B) in the case of multiple iterations, and (Yu et al., 2021a;[START_REF] Lubars | Correcting the Output of approximate Graph matching algorithms[END_REF] in the case of one iteration. Interestingly, the connection with the PPM is not explicitly stated in any of these works. We start by defining the projection operator onto P n for a matrix C ∈ R n×n . We will use the greedy maximum weight matching (GMWM) algorithm introduced in [START_REF] Lubars | Correcting the Output of approximate Graph matching algorithms[END_REF], for the problem of graph matching with partially correct seeds, and subsequently used in (Yu et al., 2021a). The steps are outlined in Algorithm 11. No-Algorithm 11 GMWM (Greedy maximum weight matching)

Input: A cost matrix C ∈ R n×n .
1: Select (i 1 , j 1 ) such that C i 1 j 1 is the largest entry in C (break ties arbitrarily). Define C (1) ∈ R n×n by C

(1)

ij = C ij 1 i̸ =i 1 ,j̸ =j 1 -∞ • 1 i=i 1 or j=j 1 . 2: for k = 2 to N do 3: Select (i k , j k ) such that C (k-1) i k ,j k is the largest entry in C (k-1) . 4: Define C (k) ∈ R n×n : C (k) ij = C (k-1) ij 1 i̸ =i k ,j̸ =j k -∞ • 1 i=i k or j=j k . 5: end for 6: Define X ∈ {0, 1} n×n : X ij = N k=1 1 i=i k ,j=j k . 7: return X Output: A permutation matrix X.
tice that the original version of GMWM works by erasing the row and column of the largest entry of the matrix C (k) at each step k. We change this to assign -∞ to each element of the row and column of the largest entry (which is equivalent), mainly to maintain the original indexation. The output of Algorithm 11 is clearly a permutation matrix, hence we define which provides an orthogonal projection, while τ corresponds to an oblique projection in general.

Algorithm 12 PPMGM (PPM for graph matching)

Require: Matrices A, B, an initial point X (0) and N the maximum number of iterations. Ensure: A permutation matrix X.

1: for k = 0 to N -1 do 2: X (k+1) ← τ(AX (k) B). 3: end for 4: return X = X (N)
The PPM is outlined in Algorithm 12. Given the estimate of the permutation X (k) from step k, the power step corresponds to the operation AX (k) B while the projection step is given by the application of the projection τ on AX (k) B. The similarity matrix C k+1 := AX (k) B is the matrix form of the left multiplication of [X (k) ] by the matrix B ⊗ A. Indeed, given that A and B are symmetric matrices, we have [AX (k) B] = (B ⊗ A)[X (k) ], by (Schäcke, 2004, eqs. 6 and 10). All previous works related to the PPM for graph matching use (B ⊗ A)[X (k) ] in the power step, which is highly inconvenient in practice. Also, a power step of the form AX (k) B connects the PPM with the seeded graph matching methods proposed for correlated Erdös-Renyi graphs [START_REF] Lubars | Correcting the Output of approximate Graph matching algorithms[END_REF]Yu et al., 2021a;Mao et al., 2021a) where related similarity matrices are used, thus providing a more general framework. Indeed, the set of elements correctly matched by the initial permutation x (0) ∈ S n can be considered as the seed of the problem, i.e., we take the set of seeds S := {(i, i ′ ) : x (0) (i) = i ′ }. Thus, the number of correct seeds will be the number of elements i ∈ [n] such that x (0) (i) = x * (i). Observe that the definition of the seed as a permutation is more general than a set S of bijectively pre-matched vertices, because S can be augmented (arbitrarily) to a permutation. initialization. We prove in Section 7.3 that Algorithm 12 recovers the ground truth permutation x * provided that the initialization x (0) is sufficiently close to x * . The initialization assumption will be written in the form ∥X

(0) -X * ∥ F ⩽ θ √ n (7.2.2) for some θ ∈ [0, √ 2).
Here, the value of θ measures how good X (0) is as a seed. Indeed, (7.2.2) can be equivalently stated as: the number of correct seeds is larger than n(2 -θ 2 ). The question of finding a good initialization method can be seen as a seedless graph matching problem, where only partial recovery guarantees are necessary. In practice, we can use existing seedless algorithms such as those in [START_REF] Umeyama | An eigendecomposition approach to weighted graph matching problems[END_REF][START_REF] Fan | Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model[END_REF][START_REF] Feizi | Spectral Alignment of Graphs[END_REF] to initialize Algorithm 12. We compare different initialization methods numerically, in Section 7.5.

Remark 20 (PPM as a gradient method). The projected power method can be seen as a projected gradient ascent method for solving the MLE formulation in (1.5.1). This can be reformulated in a vectorial form as max

[X]∈[P n ] [X] T B ⊗ A[X] (7.2.3)
where [P n ] is the set of permutation matrices in vector form. From the formulation (7.2.3) it is clear that the gradient of the likelihood evaluated on

X ∈ P n is 2B ⊗ A[X]
or, equivalently, 2AXB in matrix form. This interpretation of PPM has been acknowledged in the context of other statistical problems [START_REF] Journée | Generalized Power Method for Sparse Principal Component Analysis[END_REF][START_REF] Chen | The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences[END_REF].

Remark 21 (Optimality). Algorithms based on PPM or GPM have been shown to attain optimal, or near-optimal, statistical guarantees for several problems in statistics, including community detection [START_REF] Wang | Optimal Non-Convex Exact Recovery in Stochastic Block Model via Projected Power Method[END_REF]Wang et al., 2022b), group syncronization [START_REF] Boumal | Nonconvex Phase Synchronization[END_REF][START_REF] Gao | Iterative Algorithm for Discrete Structure Recovery[END_REF] and generalized orthogonal procrustes problem [START_REF] Ling | Generalized Power Method for Generalized Orthogonal Procrustes Problem: Global Convergence and Optimization Landscape Analysis[END_REF].

Remark 22 (Complexity). The computational time complexity of Algorithm 12 is O(n ω log n + n2 log 2 n), where O(n ω ) is the matrix multiplication complexity and O(n 2 log n) is the complexity of Algorithm 11 (Yu et al., 2021a). In (Le [START_REF] Gall | Algebraic Complexity Theory and Matrix Multiplication[END_REF], the authors establish the bound ω ⩽ 2.373.

main results

Our goal in this section is to prove recovery guarantees for Algorithm 12 when the input matrices A, B are realizations of the correlated Wigner model. In what follows, we will assume without loss of generality that X * = I n .

Exact recovery in one iteration

For any given seed x (0) that is close enough to x * , the main result of this section states that x * is recovered exactly in one iteration of Algorithm 12 with high probability. Let us first introduce the following definition: we say that a matrix M is diagonally dominant 1 if for all i, j with i ̸ = j we have M ii > M ij . This notion will be used in conjunction with the following lemma, its proof is in Appendix 7.8.

Lemma 20. If a matrix C satisfies the diagonal dominance property, then the greedy algorithm GMWM with input C will return the identical permutation. Consequently, for C = AXB and Π = τ(C), we have

P(Π ̸ = I n ) ⩽ P(C is not diag. dominant) (7.3.1)
The next theorem allows us to control the probability that C is not diagonally dominant and, in turn, proves that Algorithm 12 recovers the ground truth permutation with high probability. The proof of Theorem 17 is outlined in Section 7.4.1.

Theorem 17. Let A, B ∼ CoWM(n, σ, id) and X ∈ P n with ∥X -Id ∥ F ⩽ θ √ n, with 0 ⩽ θ ⩽ 2(1 -10 n )
and n ⩾ 10. Then the following holds.

1. For C = AXB we have

P(C is not diag. dominant ) ⩽ 5n 2 e -c(σ) 1-θ 2 2 2 n where c(σ) = 1 384 ( 1-σ 2 1+2σ √ 1-σ 2 ).
2. Denote Π as the output of Algorithm 12 with input (A, B, X (0) = X, N = 1), then

P(Π = Id) ⩾ 1 -5n 2 e -c(σ) 1-θ 2 In particular, if ∥X -Id ∥ 2 F ⩽ 2 n - 1 c(σ) n log (5n 3 ) then P(Π = Id) ⩾ 1 -n -1 . Remark 23. The assumption ∥X -Id ∥ 2 F ⩽ 2(n - 1 c(σ) n log (5n 3 )) can be restated as |S X | ⩾ 1 c(σ) n log 5n 3
, where S X is the set of fixed points of X. That is, for this assumption to hold, we need that X has a number of fixed points of order Ω σ ( n log n). Also note that c(σ) is decreasing with σ, which is consistent with the intuition that larger levels of noise make it more difficult to recover the ground truth permutation. We include a plot of c(σ) (rescaled) in Figure 20. discussion. Given an initial seed X (0) ∈ P n , the case N = 1 in Algorithm 12 can be alternatively interpreted as the following two step process: first, compute a similarity matrix AX (0) B and then round the similarity matrix to an actual permutation matrix. This strategy has been frequently applied in graph matching algorithms in both the seeded and seedless case [START_REF] Umeyama | An eigendecomposition approach to weighted graph matching problems[END_REF][START_REF] Fan | Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model[END_REF][START_REF] Lubars | Correcting the Output of approximate Graph matching algorithms[END_REF]Yu et al., 2021a). In terms of the quality of the seed, Theorem 17 gives the same guarantees obtained by Yu et al., 2021a, Thm.1 which requires Ω( n log n) vertices in the seed to be correctly matched. However the results of (Yu et al., 2021a) are specifically for the correlated Erdös-Renyi model.

Partial recovery in one iteration

In the partial recovery setting, we are interested in the fraction of nodes that are correctly matched. To this end let us define the following measure of performance overlap(ν, ν

′ ) := 1 n |{i ∈ [n] : ν(i) = ν ′ (i)}| (7.3.2)
for any pair ν, ν ′ ∈ S n . Recall that we assume that the ground truth permutation is x * = id and π is the output of Algorithm 12 with input (A, B, X (0) = X, N = 1) where Π = GMWM(AXB). Observe that overlap(π, x * = id) = s π is the fraction of fixed points of the permutation π. It will be useful to consider the following definition. We say that

C ij is row-column dominant if C ij > C i ′ j for all i ′ ̸ = i and C ij > C ij ′ , for all j ′ ̸ = j.
The following lemma relates the overlap of the output of GMWM with the property that a subset of the entries of C is row-column dominant, its proof is outlined in Appendix 7.8.

Lemma 21. Let C be a n × n matrix with the property that there exists a

set {i 1 , • • • , i r }, with 1 ⩽ r ⩽ n such that C i k ,i k is row-column dominant for k ∈ [r]. Let π ∈ S n be permutation corresponding to GMWM(C) ∈ P n . Then it holds that π(i k ) = i k for k ∈ [r]
and, in consequence, the following event inclusion holds

{overlap(π, id) < r/n} ⊂ I r ⊂[n] |I r |=r i∈I r
{C ii is not row-column dominant }.

(7.3.3)

Equipped with this lemma, we can prove the following generalization of Theorem 17, its proof is detailed in Section 7.4.2.

Theorem 18. Let A, B ∼ W(n, σ, id) and X ∈ P n with ∥X -Id ∥ F ⩽ θ √ n,
where 0 ⩽ θ ⩽ 2(1 -10 n ) and n ⩾ 10. Let π ∈ S n be the output of Algorithm 12 with input (A, B, X (0) = X, N = 1). Then, for r ∈

[n] P(overlap(π, id) > r/n) ⩾ 1 -16rne -c(σ) 1-θ 2 2 2 n .
In particular, if x ∈ S n is the map corresponding to X and

|S X | ⩾ 1 c(σ) n log (16rn 2 ), then P(overlap(π, id) > r/n) ⩾ 1 -n -1 .

Exact recovery after multiple iterations, uniformly in the seed

The results in Sections 7.3.1 and 7.3.2 hold for any given seed X (0) , and it is crucial that the seed does not depend on the graphs A, B. In this section, we provide uniform convergence guarantees for PPMGM which hold uniformly over all choices of the seed in a neighborhood around x * .

Theorem 19. Let σ ∈ [0, 1), A, B ∼ W(n, σ, id) and let X (0) be a -possibly random and data dependent -permutation such that |S X

(0) | ⩾ (1 -κ)n for a constant κ > 0 such that √ 1 -σ 2 > 48κ.
Then by applying PPMGM with input (H(A), H(B), X (0) , N = 2 log n) where H(X) corresponds to the matrix X with the diagonal removed, when n is large enough, we obtain a permutation X (N) such that P(X (N) ̸ = Id) ⩾ 1 -6 log n n 2 . The diagonal of the adjacency matrices A and B in Algorithm 12 was removed in the above theorem only for ease of analysis. Its proof is detailed in Section 7.4.3.

Remark 24. Contrary to our previous theorems, here the strong consistency of the estimator holds uniformly over all possible seeds that satisfy the condition |S X (0) | ⩾ (1 -κ)n. For this reason, we need a stronger condition than |S X | = Ω( n log n) as was the case in Theorem 17. Our result is non trivial and cannot be obtained from Theorem 17 by taking a union bound. The proof relies on a decoupling technique adapted from Mao et al., 2021a that used a similar refinement method for Erdös-Renyi graphs.

Remark 25. Contrary to the results obtained in the seedless case that require σ = o(1) for exact recovery [START_REF] Fan | Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model[END_REF], we can allow σ to be of constant order. The condition √ 1 -σ 2 > 48κ seems to be far from optimal as shown in the experiments in Section 7.5. For example, PPMGM can achieve exact recovery when κ = 0.08 and σ = 0.6. But interestingly, this condition shows that when the noise σ increases, PPMGM needs a more accurate initialization, hence a larger κ, to recover the latent permutation. This is confirmed by our experiments.

proof outline

Proof of Theorem 17

For A, B ∼ W(n, σ, id), the proof of Theorem 17 relies heavily on the concentration properties of the entries of the matrix C = AXB, which is the matrix that is projected by our proposed algorithm. In particular, we use the fact that C is diagonally dominant with high probability, under the assumptions of Theorem 17, which is given by the following result. Its proof is delayed to Appendix 7.7.1.

Proposition 4 (Diagonal dominance property for the matrix C = AXB). Let A, B ∼ W(n, σ, id) with correlation parameter σ ∈ [0, 1) and let X ∈ P n with S X the set of its fixed points and s x := |S X |/n. Assume that s x ⩾ 10/n and that n ⩾ 10. Then the following is true.

1. Noiseless case. For a fixed i ∈ [n] it holds that

P ∃j ̸ = i : (AXA) ij > (AXA) ii ⩽ 4ne -s 2 x 96 n . 2. For C = AXB and i ∈ [n] it holds P(∃j ̸ = i : C ij > C ii ) ⩽ 5ne -c(σ)s 2 x n where c(σ) = 1 384 1-σ 2 1+2σ √ 1-σ 2 .
With this we can proceed with the proof of Theorem 17.

Proof of Theorem 17. To prove part (i) of the theorem it suffices to notice that in Proposition 4 part (ii) we upper bound the probability that C = AXB is not diagonally dominant for each fixed row. Using the union bound, summing over the n rows, we obtain the desired upper bound on the probability that C is not diagonally dominant. We now prove part (ii). Notice that the assumption ∥X -I n ∥ F ⩽ θ √ n for θ < √ 2 implies that s x is strictly positive. Moreover, from this assumption and the fact that ∥X -

I n ∥ 2 F = 2(n -|S X |) we deduce that s x ⩾ 1 - θ 2 2 . (7.4.1)
On the other hand, we have

P(Π ̸ = Id) ⩽ P(C is not diag.dom) = P(∃i, j ∈ [n], i ̸ = j : C ii < C ij ) ⩽ 5n 2 e -c(σ)s 2 x n ⩽ 5n 2 e -c(σ) 1-θ 2 2 2 n
where we used Lemma 20 in the first inequality, Proposition 4 in the penultimate step and, (7.4.1) in the last inequality.

Proof of Proposition 4

In Proposition 4 part (i) we assume that σ = 0. The following are the main steps of the proof.

1. We first prove that for all X ∈ P n such that s x = |S X |/n and for i ̸ = j ∈ [n] the gap C ii -C ij is of order s x in expectation.

2. We prove that C ii and C ij are sufficiently concentrated around its mean. In particular, the probability that C ii is smaller than s x /2 is exponentially small. The same is true for the probability that C ij is larger than s x /2.

3. We use the fact

P(C ii ⩽ C ij ) < P(C ii ⩽ s x /2) + P(C ij ⩾ s x /2) to control the probability that C is not diagonally dominant.
The proof is mainly based upon the following two lemmas.

Lemma 22. For the matrix C = AXA and with s x = |S X |/n we have

E[C ij ] =    s x + 1 n 1 i∈S X for i = j, 1 n 1 x(j)=i for i ̸ = j,
and from this we deduce that for i, j ∈

[n] with i ̸ = j s x - 1 n ⩽ E[C ii ] -E[C ij ] ⩽ s x + 1 n .
Lemma 23. Assume that s x ∈ (10/n, 1] and n ⩾ 10. Then for i, j ∈ [n] with i ̸ = j we have (7.4.3) With this we can prove Proposition 4 part (i).

P(C ii ⩽ s x /2) ⩽ 4e -s 2 x 48 n , (7.4.2) P(C ij ⩾ s x /2) ⩽ 3e -s 2 x 96 n .
Proof of Prop. 4 (i). Define the event E j = {C ii < s x 2 } ∪ {C ij > s x 2 } and note that for j ̸ = i, we have {C ij > C ii } ⊂ E j . With this and the bounds (7.4.2) and (7.4.3) we have

P ∃j ̸ = i : C ij > C ii = P(∪ j̸ =i {C ij > C ii }) ⩽ P(∪ j̸ =i E j ) ⩽ P(C ii ⩽ s x 2 ) + j̸ =i P(C ij ⩾ s x 2 ) ⩽ 4e -s 2 x 96 n + 3(n -1)e -s 2 x 96 n ⩽ 4ne -s 2 x 96 n .
The proof of Lemma 22 is short and we include it in the main body of the paper. On the other hand, the proof of Lemma 23 mainly uses concentration inequalities for Gaussian quadratic forms, but the details are quite technical. Hence we delay its proof to Section 7.7.1.1. Before proceeding with the proof of Lemma 22, observe that the following decomposition holds for the matrix C.

C ij = k,k ′ A ik X k,k ′ A k ′ i =    k∈S X A 2 ik + k/ ∈S X A ik A ix(k) for i = j, n k=1 A ik A x(k)j for i ̸ = j. (7.4.4)
Proof of Lemma 22. From (7.4.4) we have that

E[C ii ] = k∈S X E[A 2 ik ] + k/ ∈S X E[A 2 ik ] = |S X | n + 1 i∈S X n .
Similarly, for j ̸ = i it holds

E[C ij ] = n k=1 E[A ik A x(k)j ] = 1 n 1 i,j/ ∈S X ,x(j)=i = 1 x(j)=i
n from which the results follows easily.

The proof of Proposition 4 part (ii) which corresponds to the case σ ̸ = 0 uses similar ideas and the details can be found in Appendix 7.7.1.2.

Proof of Theorem 18

The proof of Theorem 18 will be based on the following lemma, which extends Proposition 4.

Lemma 24. For a fixed i ∈ [n], we have

P(C ii is not row-column dominant) ⩽ 16ne -c(σ)s 2 x n .
The proof of Lemma 24 is included in Section 7.7.2. We now prove Theorem 18. The main idea is that for a fixed i ∈ [n], with high probability the term C ii will be the largest term in the i-th row and the i-th column, and so GMWM will assign π(i) = i. We will also use the following event inclusion, which is direct from (7.3.3) in Lemma 21.

{overlap(π, id) < r/n} ⊂ r i=1 {C ii is not row-column dominant }. (7.4.5) Proof of Theorem 18 . Fix i ∈ [n]
. By (7.4.5) we have that

P(overlap(π, id) ⩽ r/n) ⩽ r i=1 P(C ii is not row-column dominant) ⩽ r i=1 P(∃j ̸ = i, s.t C ij ∨ C ji > C ii ) ⩽ 16rne -c(σ)s 2 x n
where we used Lemma 24 in the last inequality.

Remark 26. Notice that the RHS of (7.4.5) is a superset of the RHS of (7.3.3). To improve this, it is necessary to include dependency information. In other words, we need to 'beat Hölder's inequality'. To see this, define

E i := 1 C ii is not row-column dominant , ε I := 1 i∈I E i >0 , for I ⊂ [n]; then ε I ′ , for I ′ = [r],
is the indicator of the event in the RHS of (7.4.5). On other hand, the indicator of the event in the RHS of (7.3.3) is

I⊂[n],|I|=r ε I .
If E ε I is equal for all I, then Hölder inequality gives

E I⊂[n],|I|=r ε I ⩽ E[ε I ′ ]
which does not help in quantifying the difference between (7.3.3) and (7.4.5). This is not surprising as we are not taking into account the dependency between the events ε I for the different sets I ⊂ [n], |I| = r.

Proof of Theorem 19

The general proof idea is based on the decoupling strategy used by (Mao et al., 2021a) for Erdös-Renyi graphs. To extend their result from binary graphs to weighted graphs, we need to use an appropriate measure of similarity. For i, i

′ ∈ [n], W ⊂ [n] and g ∈ P n , let us define ⟨A i: , B i ′ : ⟩ g,W := j∈W A ig(j) B i ′ j
to be the similarity between i and i ′ restricted to W and measured with a scalar product depending on g (the permutation used to align A and B). When g = id or W = [n] we will drop the corresponding subscript(s). If A and B were binary matrices, we would have the following correspondence

⟨A i: , B i ′ : ⟩ g,W = |g(N A (i) ∩ W) ∩ N B (i ′ )|.
This last quantity plays an essential role in Proposition 7.5 of (Mao et al., 2021a). Here g(S) denotes the image of a set S ⊆ [n] under permutation g.

step 1. The algorithm design relies on the fact that if the matrices A and B were correctly aligned then the correlation between A i: and B i: should be large and the correlation between A i: and B i ′ : should be small for all i ̸ = i ′ . The following two Lemmas precisely quantify these correlations when the two matrices are well aligned.

Lemma 25 (Correlation between corresponding nodes). Let (A, B) ∼ W(n, σ, x * = id) and assume that the diagonals of A and B have been removed. Then for n large enough, we have with probability at least

1 -n -2 that ⟨A i: , B i: ⟩ ⩾ 1 -σ 2 (1 -ϵ 1 ) -σϵ 2 for all i ∈ [n],
where

ϵ 1 , ϵ 2 = O( log n n ).
Lemma 26 (Correlation between different nodes). Let (A, B) ∼ W(n, σ, id) and assume that the diagonals of A and B have been removed. Then for n large enough, we have with probability at least 1 -n -2 that

|⟨A i: , B i ′ : ⟩| ⩽ 1 -σ 2 ϵ 2 + σϵ 3 for all i, i ′ ∈ [n] such that i ′ ̸ = i, where ϵ 3 = O( log n n ).
The proofs of Lemmas 25 and 26 can be found in Section 7.8.1.

step 2. Since the ground truth alignment between A and B is unknown, we need to use an approximate alignment (provided by X (0) ). It will suffice that X (0) is close enough to the ground truth permutation. This is linked to the fact that if |S X (0) | is large enough then the number of nodes for which there is a substantial amount of information contained in S c X (0) is small. This is shown in the following lemma.

Lemma 27 (Growing a subset of vertices). Let G a graph generated from the Wigner model with self-loops removed, associated with an adjacency matrix A, and let I be a random subset of [n] (possibly depending on A) with |I| ⩾ (1 -κ)n where κ ∈ (0, 1/2). Let δ = 8κ and define a random subset of vertices Ĩ = {i ∈ [n] : ∥A i: ∥ 2 I c < δ}. Then for n large enough, we have

P | Ĩc | ⩽ 1 4 |I c | ⩾ 1 -e -c ′ κn
for some constant c ′ > 0.

In order to prove this lemma we will need the following decoupling lemma.

Lemma 28 (An elementary decoupling). Let M > 0 be a parameter and G be a weighted graph on [n], with weights of magnitude bounded by 1 and without self loops, represented by an adjacency matrix A ∈ [-1, 1] n×n . Assume that there are two subsets of vertices Q, W ⊂ [n] such that

∥A i: ∥ 2 W ⩾ M for all i ∈ Q.
Then there are subsets

Q ′ ⊆ Q and W ′ ⊆ W such that Q ′ ∩ W ′ = ∅, |Q ′ | ⩾ |Q|/5 and ∥A i: ∥ 2 W ′ ⩾ M/2 for all i ∈ Q ′ . Proof. If |Q \ W| ⩾ |Q|/5 then one can take Q ′ = Q \ W and W ′ = W. So we can assume that |Q ∩ W| ⩾ 4|Q|/5. Let W := W \ Q and Q be a random subset of Q ∩ W where each element j ∈ Q ∩ W is selected independently with probability 1/2 in Q. Consider the random disjoint sets Q and W ′ := W ∪ ((Q ∩ W) \ Q).
First, we will show the following claim.

Claim 1. For every i ∈ Q ∩ W, we have P(∥A i: ∥ 2 W ′ ⩾ M/2|i ∈ Q) ⩾ 1/2. Indeed, we have by definition ∥A i: ∥ 2 W ′ = j∈W ′ A 2 ij = j∈W∩Q A 2 ij 1 j̸ ∈ Q + j∈ W A 2 ij .
By taking the expectation conditional on i ∈ Q, we obtain

E ∥A i: ∥ 2 W ′ i ∈ Q = j∈W∩Q A 2 ij 2 + j∈ W A 2 ij ⩾ 1 2 j∈W A 2 ij ⩾ M 2 . But since j∈W∩Q A 2 ij (1 j̸ ∈ Q -1 2 ) is a symmetric random variable we have that P ∥A i: ∥ 2 W ′ ⩾ E(∥A i: ∥ 2 W ′ ) i ∈ Q = 1/2
and hence

P ∥A i: ∥ 2 W ′ ⩾ M 2 i ∈ Q ⩾ 1/2.
Consequently, we have

E   i∈Q∩W 1 {∥A i: ∥ 2 W ′ ⩾M/2} 1 i∈ Q  = i∈Q∩W P(i ∈ Q)E 1 {∥A i: ∥ 2 W ′ ⩾M/2} i ∈ Q ⩾ |Q ∩ W| 4 ⩾ |Q| 5 .
Therefore, there is a realization Q ′ of Q such that Q ′ and W ′ satisfy the required conditions.

Proof of Lemma 27. By considering sets W = I c and Q ⊂ Ĩc we obtain the following inclusion

{| Ĩc | > 1 4 |I c |} ⊂ E := {∃ Q, W ⊂ [n] : |W| ⩽ κn, |Q| ⩾ |W|/4 ̸ = 0, ∥A i: ∥ 2 W ⩾ δ for all i ∈ Q}.
According to Lemma 28, E is contained in

E ′ := {∃ Q ′ , W ′ ⊂ [n] : |W ′ | ⩽ κn, |Q ′ | ⩾ |W|/20 ̸ = 0, Q ′ ∩ W ′ = ∅, ∥A i: ∥ 2 W ′ ⩾ δ/2 for all i ∈ Q ′ }.
For given subsets Q ′ and W ′ , the random variables (∥A i: ∥ 2 W ′ ) i∈Q ′ are independent. So, by a union bound argument we get

P | Ĩc | > 1 4 |I c | ⩽ ⌈κn⌉ w=1 |W ′ |=w n k=⌈w/20⌉ n k P ∥A i: ∥ 2 W ′ ⩾ δ/2 k .
According to Lemma 37, for the choice t = κn we have for all

W ′ P ∥A i: ∥ 2 W ′ ⩾ δ/2 ⩽ P n ∥A i: ∥ 2 W ′ ⩾ |W| + |W|t + 2t ⩽ e -κn .
Consequently, for n large enough, we have where c > 0 is a constant.

P | Ĩc | > 1 4 |I c | ⩽
step 3. We are now in position to show that at each step the set of fixed points of the permutation obtained with PPMGM increases.

Lemma 29 (Improving a partial matching). Let G and G ′ two graphs as before, and g be a random permutation possibly depending on G and G ′ . Further assume that √ 1 -σ 2 > 48κ. Let

E := {|i ∈ [n] : g(i) = i| ⩾ (1 -κ)n}
be the event that the number of fixed point of g is large enough. Define a random permutation g and a random set J as follows. Let δ = 8κ, we say that a vertex i ∈

[n] belongs to J if there is a unique i ′ ∈ [n] such that • ⟨A i: , B i ′ : ⟩ g ⩾ 3δ; • |⟨A i: , B j: ⟩ g | < 3δ for all j ̸ = i ′ ; • |⟨A j: , B i ′ : ⟩ g | < 3δ for all j ̸ = i.
Then we set g(i) = i ′ for any such pair of vertices. We complete g into a permutation in an arbitrary way.

If n is sufficiently large and κ sufficiently small, we have with probability at least P(E) -

3 n 2 , |{i ∈ [n] : g(i) = i}| ⩾ n 2 + |{i ∈ [n] : g(i) = i}| 2 .
It implies in particular that the set of fixed points of g is strictly larger than the set of fixed points of g.

Remark 27. The description of GMWM doesn't involve the use of a threshold, but for the nodes that satisfy the conditions described in Lemma 29, GMWM provides by definition the same matching (this can be seen using the notion of row-column dominance and Lemma 21). Since the nodes that do not satisfy these conditions can be matched in an arbitrary way, we can use GMWM instead of the thresholding procedure and the analysis remains valid.

Proof. Define the random sets

I :={j ∈ [n] : g(j) = j}, Ĩ :={j ∈ [n] : ∥A i: ∥ 2 I c < δ}, Ĩ′ :={j ∈ [n] : ∥B i: ∥ 2 I c < δ},
where δ = 8κ and consider the event

E ′ = E ′ 1 ∩ E ′ 2 ∩ E ′ 3 where E ′ 1 := {| Ĩc | ∨ |( Ĩ′ ) c | ⩽ 1 4 |I c |} E ′ 2 := {∀i ∈ [n] : ⟨A i: , B i: ⟩ ⩾ 0.9 1 -σ 2 } E ′ 3 := {∀i ̸ = i ′ ∈ [n] : |⟨A i: , B i ′ : ⟩| < C log n/
⟨A i: , B i: ⟩ g ⩾ ⟨A i: , B i: ⟩ g,I -|⟨A i: , B i: ⟩ g,I c | ⩾ ⟨A i: , B i: ⟩ -|⟨A i: , B i: ⟩ I c | -|⟨A i: , B i: ⟩ g,I c | ⩾ 3δ.
Here we used the fact that for all permutations g, |⟨A i:

, B i: ⟩ g,I c | ⩽ ∥A i: ∥ I c ∥B i: ∥ I c (because g(I c ) = I c by definition of I).
On the other hand, for every i ′ ∈ [n] \ i we have

|⟨A i: , B i ′ : ⟩ g | ⩽ |⟨A i: , B i ′ : ⟩ g,I | + |⟨A i: , B i ′ : ⟩ g,I c | < 3δ.
Similarly we have |⟨A i ′ : , B i: ⟩ g | < 3δ. Hence Ĩ ∩ Ĩ′ ⊂ J and g(i) = i for all i ∈ Ĩ ∩ Ĩ′ . Moreover we have by the first condition on

E ′ | Ĩ ∩ Ĩ′ | ⩾ n -| Ĩc | -|( Ĩ′ ) c | ⩾ n - |I| c 2 = n 2 + |I| 2 ,
so the result follows.

conclusion. By Lemma 29, if the initial number of fixed points is (1 -κ)n then after one iteration step the size of the set of fixed points of the new iteration is at least (1 -κ/2)n with probability greater than 1 -3 n 2 . So after 2 log n iterations the set of fixed points has size at least (1 -κ/2 2 log n )n > n -1 with probability greater than 1 -6 log n n 2 .

numerical experiments

In this section, we present numerical experiments to assess the performance of the PPMGM algorithm and compare it to the state-of-art algorithms for graph matching, under the correlated Wigner model. We divide this section into two parts. In Section 7. 0) , such that overlap(x (0) , x * ) = 0.08.

runs. In Figure 22 we plot the performance of the PPMGM algorithm for randomly chosen seeds and with different number of correctly pre-matched vertices. More specifically, we consider an initial permutation x is chosen uniformly at random in the subset of permutations that satisfy each overlap condition. We observe that initializing the algorithm with an overlap of 0.1 with the ground truth permutation already produces perfect recovery in one iteration for levels of noise as high as σ = 0.6.

varying the number of iterations N. We experimentally evaluate the performance of PPMGM when varying the number of iterations N in Algorithm 12. In Figures 23 and24 we plot the recovery rate of PPMGM, initialized with x (0) , with an overlap of 0.1 with the ground truth. In Figure 23 we see that adding more iterations increases the performance of the algorithm for n = 500; however the improvement is less pronounced in the higher noise regime. In other words, the number of iterations cannot make up for the fact that the initial seed is of poor quality (relative to the noise level). We use N = 1, 2, 4, 8, 30 iterations and we observe a moderate gain between where ϵ > 0 is arbitrary and R is an absolute constant. We refer the reader to (Mao et al., 2021a) for details. For each p in the range defined by (7.5.1) we solve the equation

P(|A ij | ⩽ τ p ) = 2Φ(-τ p √ n) = p (7.5.2)
where Φ is the standard Gaussian cdf (which is bijective so τ p is well defined). In our experiments, we solve (7.5.2) numerically. Notice that A spar 1 and B spar 1 are sparse correlated Erdös-Renyi graphs with a correlation that depends on σ. For the value of k that defines A spar 3 , B spar 3 we choose k = Ω(log n) or k = Ω(n o(1) ), to maintain the sparsity degree in (7.5.1). In Figures 25 and 26 we plot the performance comparison between the PPMGM without sparsification, and the different sparsification strategies (initialized with overlap 0.5 and 0.1). We see that the use of the full information A, B outperforms the sparser versions in the higher noise regimes and for small overlap of the initialization. On the other hand, the performance tends to be more similar for low levels of noise and moderately large number of correct initial seeds. In theory, sparsification strategies have a moderate denoising effect (and might considerably speed up computations), but this process seems to destroy important correlation information. To choose between them, we use a simple grid search where we evaluate the recovery rate for each sparsification parameter on graphs of size n = 1500, and take the mean over 25 independent Monte Carlo runs. In Figure 27, we plot a heatmap with the results. We see that the best performing parameter in this experiment was for τ 5 corresponding to a probability p 5 = 51 × 10 -3 , although there is a moderate change between all the choices for p.

concluding remarks and futur work

In this work, we analysed the performance of the projected power method (proposed in [START_REF] Onaran | Projected power iteration for network alignment[END_REF] as a seeded graph matching algorithm, in the correlated Wigner model. We proved that for a non-data dependent seed with O( n log n) correctly pre-assigned vertices, the PPM exactly recovers the ground truth matching in one iteration. This is analogous to the state-of-the-art results for algorithms in the case of relatively sparse correlated Erdös-Renyi graphs. We additionally proved that the PPM can exactly recover the optimal matching in O(log n) iterations for a seed that contains Ω (1 -κ)n correctly matched vertices, for a constant κ ∈ (0, 1), even if the seed can potentially be dependent on the data. For the latter result, we extended the arguments of Mao et al., 2021a from the (sparse) correlated An open question is to find an efficient initialization method which outputs a permutation with order (1 -κ)n correctly matched vertices in regimes with higher σ (say for σ > 1/2). For those noise levels, spectral methods do not seem to perform well (at least in the experiments). An idea could be to adapt the results Mao et al., 2021a from the sparse Erdös-Renyi case to the Wigner case. In that paper, the authors construct for each vertex a signature containing the neighborhood information of that vertex and which is encoded as tree. Then a matching is constructed by matching those trees. It is however unclear how to adapt those results (which heavily rely on the sparsity) to the dense Wigner case.

It would also be interesting to compare in numerical experiments the performance of the algorithm proposed by [START_REF] Ma | Community Detection with Contextual Multilayer Networks[END_REF] to other state-of-the-art methods for graph matching to see if the main contribution of Mao et al., 2021a is on the theoretical analysis or if their work can also lead to an efficient algorithm in practice. We are currently working on the implementation of this algorithm and possible improvements.

additional proofs

Proof of Proposition 4

We divide the proof into two subsections. In Appendix 7.7.1.1 we prove Lemma 23 and in Appendix 7.7.1.2 we prove part (ii) of Proposition 4. Before proceeding, let us introduce and recall some notation. Define C ′ := AXA and C ′′ := AXZ, then C = AXB = √ 1 -σ 2 C ′ + σC ′′ . Recall that for a permutation x, S X will denote the set of fixed points of x (the set of non-zero diagonal terms of its matrix representation X) and we will often write s x = |S X |/n = T r(X)/n. We will say that a real random variable Y ∼ χ 2 K if it follows a central Chi-squared distribution with K degrees of freedom.

Proof of Lemma 23

The proof of Lemma 23 mainly revolves around the use of concentration inequalities for quadratic forms of Gaussian random vectors. For that, it will be useful to use the following representation of the entries of C. C ij = ⟨A :i , XA :j ⟩ (7.7.1)

where we recall that A :k represents the k-th column of the matrix A.

Proof of Lemma 23.

high probability bound for C ii . Define ãi to be a vector in R n such that ãi (k) =    A ki , for k / ∈ i, x -1 (i),

1 √ 2 A ii , for k ∈ i, x -1 (i).

Using representation (7.7.1) we have

C ii = ⟨ ãi , X ãi ⟩ + Z i where Z i := 1 2 A ii A x(i)i ) + A x -1 (i)i .
It is easy to see that √ n ãi is a standard Gaussian vector. Using Lemma 38 we obtain n⟨ ãi , X ãi

⟩ d = n 1 i=1 µ i g 2 i - n 2 i=1 ν i g ′2 i
where (µ i ) n 1 i=1 , (-ν i ) n 2 i=1 , (with µ i ⩾ 0, ν i ⩾ 0 and n 1 + n 2 = n) is the sequence of eigenvalues of 1 2 (X + X T ) and g = (g 1 , • • • , g n 1 ), g ′ = (g ′ 1 , • • • , g ′ n 2 ) are two independent sets of i.i.d standard Gaussians.

Lemma 38 tell us in addition that ∥µ∥ 1 -∥ν∥ 1 = s x n, ∥µ∥ 2 + ∥ν∥ 2 ⩽ √ 2n and ∥µ∥ ∞ , ∥ν∥ ∞ ⩽ 1. Using Corollary 3 (A.0.2), we obtain P(n⟨ ãi , X ãi ⟩ ⩽ s x n -2 √ 2nt -2t) ⩽ e -t (7.7.2) for all t ⩾ 0. To obtain a concentration bound for Z i we will distinguish two cases.

(a)Case i ∈ S X . In this case, we have Z i = a 2 i (i), which implies that C ii ⩾ ⟨ ãi , X ãi ⟩. Hence

P(nC ii ⩽ s x n -2 √ 2nt -2t) ⩽ 2e -t .
Replacing t = t := n 2 ( 1 + s x 2 -1) 2 in the previous expression, one can verify 2 that t ⩾ n 48 s 2 x , for s x ∈ (0, 1], hence

P(C ii ⩽ s x /2) ⩽ 2e -s 2 x 48 n
which proves (7.4.2) in this case.

(b) Case i / ∈ S X . Notice that in this case, a i (i) is independent from (a i (x(i)) + a i (x -1 (i)), hence nZ i d = g 1 g 2 , where g 1 , g 2 are independent standard Gaussians. Using the polarization identity g 1 g 2 = 1 4 (g 1 + g 2 ) 2 -1 4 (g 1 -g 2 ) 2 , we obtain

nZ i d = 1 2 ( g2 1 -g2 2 )
where g1 , g2 are independent standard Gaussians. 2 Indeed, the inequality ( √ 1 + x -1) 2 ⩾ 1 6 x 2 , follows from the inequality x 2 + (2 √ 6 -6)x ⩽ 0, which holds for 0 < x ⩽ 1. Contrary to a i and a j which share a coordinate, the vectors ãi and ãj are independent. With this notation, we have the following decomposition C ij = ⟨ ãi , X ãj ⟩ + A ji A x(j)j + A x -1 (i)i .

For the first term, we will use the following polarization identity (7.7.5) By the independence of ãi and ãj , it is easy to see that ãi + X ãj and ãi -X ãj are independent Gaussian vectors and E[⟨ ãi , X ãj ⟩] = 0.

⟨ ãi , X ãj ⟩ = ∥ 1 2 ( ãi + X ãj )∥ 2 -∥ 1 2 ( ãi -X ãj )∥ 2 .
Using (7.7.5) and defining Z ij := A ji A x(j)j + A x -1 (i)i n, it is easy to see that

nC ij d = n-1 i=1 µ i g 2 i - n-1 i=1 ν i g ′2 i + Z ij (7.7.6)
where g 1 , • • • , g n-1 and g ′ 1 , • • • , g ′ n-1 are two sets of independent standard Gaussian variables and µ i , ν i ∈ { 1 2 , 3 4 , 1}, for i ∈ [n -1]. The sequences (µ i ) n-1 i=1 , (ν i ) n-1 i=1 will be characterised below, when we divide the analysis into two cases x(j) = i and x(j) ̸ = i. We first state the following claim about Z ij .

Claim 2. For i ̸ = j, we have

Z ij d =    q ij (ζ 1 -ζ 2 ) if x(j) ̸ = i, 2ζ 3 if x(j) = i,
where ζ 1 , ζ 2 and ζ 3 are independent Chi-squared random variables with one degree of freedom and

q ij =            3 2 if i ∈ S X , j / ∈ S X or i / ∈ S X , j ∈ S X , √ 2 if i, j ∈ S X , 1 √ 2 if i, j / ∈ S X .
We delay the proof of this claim until the end of this section. From the expression (7.7.6), we deduce that the vectors g = (g 1 , • • • , g n-1 ),

g ′ = (g ′ 1 , • • • , g ′ n-1
) and Z ij are independent. Hence, by Claim 2 the following decomposition holds

nC ij d = n i=1 µ i g 2 i - n i=1 ν i g ′2 i where µ n =    q ij if x(j) ̸ = i, 2 if x(j) = i,
and

ν n =    q ij if x(j) ̸ = i, 0 if x(j) = i.
Let us define µ := (µ 1 , • • • , µ n ) and ν := (ν 1 , • • • , ν n ). We will now distinguish two cases.

(a) Case x(j) ̸ = i. In this case, we can verify that one of the µ 1 , • • • , µ n-1 is equal to 0 (and the same is true for the values ν 1 , • • • , ν n-1 ). Assume without loss of generality that µ 1 = ν 1 = 0. Also, one of the following situations must happen for the sequence µ 2 , • • • , µ n-1 (resp. ν 2 , • • • , ν n-1 ): either n -3 of the elements of the sequence are equal to 1 2 and one is equal 1 or n -4 are equal to 1 2 and two are equal to 3 4 or n -3 are equal to 1 2 and one is equal to 3 4 . In either of those cases, the following is verified

∥µ∥ 1 -∥ν∥ 1 = 0, ∥µ∥ 2 + ∥ν∥ 2 ⩽ √ 2n, ∥µ∥ ∞ , ∥ν∥ ∞ ⩽ √ 2,
where the first equality comes from Lemma 22, the inequality on the norm ∥ • ∥ 2 comes from the fact that in the worst case ∥µ∥ 2 = ∥ν∥ 2 ⩽ n+1 4 . The statement about the norm ∥ • ∥ ∞ can be easily seen by the definition of µ and ν. Using (A.0.1), we obtain

P(nC ij ⩾ 4 √ nt + 4t) ⩽ 2e -t .
Replacing t = t := n 4 ( 1 + s x 2 -1) 2 in the previous expression and noticing that t ⩾ 1 96 s 2 x n for s x ∈ (0, 1] leads to the bound P(C ij ⩾ s x /2) ⩽ 2e -s 2 x 96 n .

(b) Case x(j) = i. In this case, we have that for the sequence µ 1 , • • • , µ n-1 (resp. ν 1 , • • • , ν n-1 ): either n -2 of the elements of the sequence are equal to 1 2 and one is equal 1 or n -3 are equal to 1 2 and two are equal to 3 4 . In either case, the following holds

∥µ∥ 1 -∥ν∥ 1 = 2, ∥µ∥ 2 + ∥ν∥ 2 ⩽ 2 √ n, ∥µ∥ ∞ , ∥ν∥ ∞ ⩽ 2. E ij := { 1 -σ 2 C ′ ij ⩾ s σ,x /2} ∪ {σC ′′ ij ⩾ s σ,x /2}
, for i ̸ = j. One can easily verify that {C ii ⩽ s σ,x } ⊂ E i δ , hence it suffices to control the probability of E i δ . For that we use the union bound and the already established bounds in Lemmas 23 and 30. To attack the offdiagonal case, we observe that the following holds {C ij ⩾ s σ,x } ⊂ E ij . The following lemma allows us to bound the probability of the events E i δ and E ij .

Lemma 31. Let δ be such that 0 ⩽ δ ⩽ s x 2 √ 1 -σ 2 . Then for i, j ∈ [n] with i ̸ = j have the following bounds

P(E i δ ) ⩽ 4e -1 96 ( sx 2 -δ √ 1-σ 2
) 2 n + 2e -1 96 ( δ σ ) 2 n (7.7.7)

P(E ij ) ⩽ 4e -1 384 s 2
x ( 1-σ 2 σ 2 ∧1)n . (7.7.8)

In particular, we have

P(E i δ σ,x ) ⩽ 6e -1 384 s 2 x ( 1-σ 2 1+2σ √ 1-σ 2
)n (7.7.9)

where δ σ,x = σ √ 1-σ 2 σ+ √ 1-σ 2 s x 2 .
Proof. Using (7.7.4), we have that 1 -σ 2 , we obtain

P 1 -σ 2 C ii ⩽ 1 -σ 2 s x -2( √ 2 + 1/ √ n) t n -3 t n ⩽ 4e -
P 1 -σ 2 C ′ ii ⩽ s σ,x + δ ⩽ 4e -1 6 ( sx 2 -δ √ 1-σ 2 ) 2 n
.

Using this and Lemma 30 we have

P(E i δ ) ⩽ P( 1 -σ 2 C ′ ii ⩽ s σ,x + δ) + P(σC ′′ ii ⩽ -δ) ⩽ 4e -1 6 ( sx 2 -δ √ 1-σ 2 ) 2 n + 2e -1 96 ( δ σ ) 2 n .
Similarly, to prove (7.7.8) we verify that

P(E ij ) ⩽ P(C ′ ij ⩾ s x 4 ) + P(C ′′ ij ⩾ √ 1 -σ 2 σ s x 4 ) ⩽ 2e -1 384 s 2 x n + 2e -1 384 s 2 x ( 1-σ 2 σ 2 )n ⩽ 4e -1 384 s 2 x ( 1-σ 2 σ 2 ∧1)n .
To prove (7.7.9) it suffices to use (7.7.7) with the choice of δ = δ σ,x =

σ √ 1-σ 2 σ+ √ 1-σ 2 s x 2 .
With this we prove the diagonal dominance for each fixed row of C. )n

Proof of

+ 4(n -1)e

-1 384 s 2 x ( 1-σ 2 σ 2 ∧1)n ⩽ 5ne -1 384 s 2 x ( 1-σ 2 1+2σ √ 1-σ 2
)n

where in the third inequality we used Lemma 31, and in the last inequality we used the fact that 1-σ 2 σ 2 ∧ 1 ⩾ 1-σ 2 1+2σ √ 1-σ 2 .

Proof of Lemma 24

The proof of Lemma 24 uses elements of the proof of Proposition 4.

The interested reader is invited to read the proof of Proposition 4 first.

Proof of Lemma 24. It will be useful to first generalize our notation.

For that, we denote

C ij,x = (AXB) ij , C ′ ij,x = (AXA) ij , C ′′ ij,x = (AXZ) ij
for x ∈ S n , and

E ij x -1 := { 1 -σ 2 C ′ ij,x -1 ⩾ s σ,x /2} ∪ {σC ′′ ij,x -1 ⩾ s σ,x /2}
where x -1 is the inverse permutation of x. The fact that P(C ii,x < C ij,x ) ⩽ 8e -c(σ)s 2 x n follows directly from the bound for Ẽj derived in the proof of Proposition 4 part (ii). To prove P(C ii,x < C ji,x ) notice that C ′ ji,x = C ′ ij,x -1 and that C ′′ ji,x d = C ′′ ij,x -1 . On the other hand, notice that s x = s x -1 (hence s σ,x = s σ,x -1 ). Arguing as in Lemma 31 it is easy to see that P(C ii,x < C ji,x ) ⩽ 8e -c(σ)s 2

x n .

The bound on P(∃j, s.t C ij,x ∨ C ji,x > C ii,x ) then follows directly by the union bound.

proofs of lemmas 20 and 21

Proof of Lemma 20. By assumption C is diagonally dominant, which implies that ∃i 1 such that C i 1 i 1 = max i,j C ij (in other words, if the largest entry of C is in the i 1 -th row, then it has to be C i 1 i 1 , otherwise it would contradict the diagonal dominance of C). In the first step of GMWM we select C i 1 i 1 , assign π(i 1 ) = i 1 and erase the i 1 -th row and column of C. By erasing the i 1 -th row and column of C we obtain a matrix which is itself diagonally dominant. So by iterating this argument we see ∃ i 1 , • • • , i n ⊂ [n] such that π(i k ) = i k , for all k, so π has to be the identical permutation. This proves that if C is diagonally dominant, then Π = Id. By using the contrareciprocal, (7.3.1) follows.

Proof of Lemma 21. We argue by contradiction. Assume that for some 1 ⩽ k ⩽ r, we have π(i k ) ̸ = i k (and π -1 (i k ) ̸ = i k ). This means that at some some step j the algorithm selects either C (j)

i k π(i k ) or C (j) π -1 (i k )π(i k )
as the largest entry, but this contradicts the row-column dominance of i k . This proves that that if there exists a set of indices I r ⊂ [n] of size r such that for all i ∈ I r , C ii is row-column dominant, then that set is selected by the algorithm, which implies that π(i) = i for i ∈ I r , thus overlap(π, id) ⩾ r. (7.3.3) follows by the contrareciprocal.

Concentration inequalities used in Theorem 19

In this section we provide proofs of Lemma's 25 and 26 used to prove Theorem 19.

Proof of Lemma's 25 and 26. Recall that

B ij = √ 1 -σ 2 A ij + σZ ij .
step 1. First let us consider the terms of the form ⟨A i: , A i: ⟩. We can write

⟨A i: , A i: ⟩ = n-1 i=1 µ i g 2 i
where g i are independent standard Gaussian random variables and µ i = 1/n for all i. Observe that ||µ|| 2 = n-1 n 2 . By Lemma 37 we have for i ∈ [n] and all t > 0

P ⟨A i: , A i: ⟩ ⩽ n -1 n -2 t(n -1) n 2 ⩽ e -t .
For the choice t = 5 log n we obtain

⟨A i: , A i: ⟩ ⩾ 1 -O log n n
with probability at least 1 -e -5 log n .

step 2. Let us consider now terms of the form ⟨A i: , Z i: ⟩. We can write

⟨A i: , Z i: ⟩ = 1 n n-1 i=1 (g i g ′ i ) = 1 n G ⊤ G ′
where G = (g i ) n-1 i=1 and G ′ = (g ′ i ) n-1 i=1 are i.i.d. standard Gaussian random variables. We can write

G ⊤ G ′ = ∥G∥ G ∥G∥ ⊤ G ′ .
Since G ′ is invariant by rotation ( G ∥G∥ ) ⊤ G ′ is independent from G and has distribution N(0, 1). By Gaussian concentration inequality we hence have

G ∥G∥ ⊤ G ′ ⩽ C log n
with probability at least 1 -e -5 log n for a suitable choice of C. Similarly, by Lemma 37 we have

∥G∥ ⩽ 2 √ n
with probability at least 1 -e -5 log n . Hence with probability at least 1 -2e -5 log n we have

1 n G ⊤ G ′ ⩽ 2C log n n .
step 3. The same argument can be used to show that for i ̸ = j P ⟨A i: , A j: ⟩ ⩾ C log n n ⩽ e -5 log n .

conclusion. We can conclude by using the identity ∼ B(ρ). Then P(X ⩽ (1 -δ)nρ) ⩽ e -nρδ 2 /2 and P(X ⩾ (1 + δ)nρ) ⩽ e -nρδ 2 /3 for all 0 < δ < 1.

B ij = 1 -σ 2 A ij + σZ ij
Proof 3. The last inequality can be obtained by using the proof techniques in [START_REF] Bandeira | Sharp nonasymptotic bounds on the norm of random matrices with independent entries[END_REF] as follows. In order to extend the concentration result from a matrix Y with independent standard Gaussian entries to a matrix X with symmetric sub-Gaussian entries, the key is to upper bound all the moments of X ij by moment of Y ij . This can be done by using the boundedness of Z ij as in Corollary 3.2, or the sub-Gaussian norm of X ij as in Corollary 3.3 of [START_REF] Bandeira | Sharp nonasymptotic bounds on the norm of random matrices with independent entries[END_REF]. But in our case, none of these bounds gives a good result. However, the proof of Theorem 1.1 (and its extensions) only requires control of the moments of the order log n 1 . For a Binomial r.v. X with parameters n 2 /L and p max , we have, according to Theorem 1 in Ahle, 2022, for all c ∈ N * , E(X c ) ⩽ (n 2 p max /L) c e c 2 /(2n 2 p max /L) .

Let X ′ be an independent copy of X. Since n 2 p max /L ≳ log n 1 by assumption, e c 2 /(2n 2 p max /L) ⩽ e γc for c ≍ log(n 1 ) and an absolute constant γ > 0. Hence

E((X -X ′ ) c ) ⩽ i⩽c c i E(X i )E(X c-i ) ⩽ 2 c (n 2 p max /L) c e γc .
Observe that (2e γ ) c ≲ EY 2c ij = O((2c) c ) for every c, so that for all even c we have

E L(X -X ′ ) 2e γ n 2 p max c ≲ EY 2c ij .
We can now use the same argument as in Corollary 3.2 of Bandeira et al., 2016 to conclude that the matrix M with independent entries generated with the same law as X -X ′ satisfies with probability at least 1 -O(n Ω(1) 1

) (since n 1 n 2 p max /L ≳ log n 1 by assumption) ∥M∥ ⩽ n 1 n 2 p max /L. ).

Remark 28. The second concentration inequality of the above lemma slightly improves Proposition 1 and Theorem 4 in [START_REF] Ndaoud | Improved Clustering Algorithms for the Bipartite Stochastic Block Model[END_REF] The third concentration could be of independent interest and be applied for example in multilayer network analysis where matrices with independent Binomial entries arise naturally as the sum of the adjacency matrices of the layers, see e.g. [START_REF] Paul | Spectral and matrix factorization methods for consistent community detection in multilayer networks[END_REF]Braun et al., 2021b. using the rotation invariance of the standard Gaussian vectors. Notice that

|S X | = T r(X) = T r 1 2 (X + X T ) = n i=1 λ i which leads to ∥µ∥ 1 -∥ν∥ 1 = n i=1 λ i = |S X | = s x n.
The fact that ∥µ∥ ∞ , ∥ν∥ ∞ ⩽ 1 follows easily since X is a unitary matrix. The inequality ∥µ∥ 2 + ∥ν∥ 2 ⩾ √ n follows from the fact that ∥µ∥ 2 2 + ∥ν∥ 2 2 = n. From the latter, we deduce that ∥µ∥ 2 + ∥ν∥ 2 ⩽ ∥µ∥ 2 2 + n -∥µ∥ 2 2 ⩽ 2 n 2 , and the result follows.

  η (noise) under signed SBM, K = 20, n = 10000, p = 0.01. 36 Figure Sorted adjacency matrices and maps for Australian rainfall dataset (K = 5). 36 Figure Sorted adjacency matrices of the Australian rainfall data set and corresponding maps for K = 3. 37 Figure Sorted adjacency matrices of the Australian rainfall data set and corresponding maps for K = 7. 37 Figure Sorted adjacency matrices of the Australian rainfall data set and corresponding maps for K = 10. 38 Figure Average performance measured by NMI obtained with random initialization over 20 runs. 38 Figure Average performance over 20 runs of our algorithms under the experimental setting of Section 5.1. Here (rd) indicates random initialization. 39 Figure Average performance of different algorithms on a heterophilic SBM, over 40 Monte Carlo runs. 40 Figure Estimation error associated to a MMSBM with parameters n, K = 3, p, q and n 0 (x-axis) averaged over 20 runs. 43 Figure Estimation error associated to a MMSBM with parameters n = 5000, K (x-axis), p, q and n 0 averaged over 20 runs. 44 Figure NMI vs ρ for different values of L. 88 Figure NMI vs L for different values of ρ. 88 Figure NMI vs n for different values of ρ. 89 Figure NMI for varying n with ρ = 0.9. 92 Figure The constant c(σ) (re-scaled multiplying by 384) appearing in Theorem 17. 143 Figure Performance of PPMGM as a refinement of Grampa and Umeyama algorithms, compared with PPM with a random initialization x (

•

  For any sequences (a n ) n⩾1 and (b n ) n⩾1 if there is a constant C > 0 such that a n ⩽ Cb n (resp. a n ⩾ Cb n ), we will use the notation a n ≲ b n (resp. a n ≳ b n ). If the inequalities only hold xvi acronyms xvii for n large enough, we will use the notation a n = O(b n ) (resp. a n = Ω(b n )). If a n ≲ b n (resp. a n = O(b n )) and a n ≳ b n (resp. a n = Ω(b n )), then we write a n ≍ b n (resp. a n = Θ(b n )).

K

  } of the latent partition one can estimate the centroids of the partition by

Figure 1 :

 1 Figure 1: Average performance over 40 runs of different algorithms under CSBM.

  partition of the nodes z(T ) .

Figure 2 :

 2 Figure 2: NMI versus η (noise) under signed SBM, K = 20, n = 10000, p = 0.01.

Figure 3 :

 3 Figure 3: Sorted adjacency matrices and maps for Australian rainfall dataset (K = 5).

Figure 3 Figure 4 :

 34 Figure 4: Sorted adjacency matrices of the Australian rainfall data set and corresponding maps for K = 3.

Figure 5 :

 5 Figure 5: Sorted adjacency matrices of the Australian rainfall data set and corresponding maps for K = 7.

Figure 6 :

 6 Figure 6: Sorted adjacency matrices of the Australian rainfall data set and corresponding maps for K = 10.

Figure 7 :

 7 Figure 7: Average performance measured by NMI obtained with random initialization over 20 runs.

Figure 8 :

 8 Figure 8: Average performance over 20 runs of our algorithms under the experimental setting of Section 5.1. Here (rd) indicates random initialization.

Figure 9 :

 9 Figure 9: Average performance of different algorithms on a heterophilic SBM, over 40 Monte Carlo runs.

Figure 10 :

 10 Figure10: Estimation error associated to a MMSBM with parameters n, K = 3, p, q and n 0 (x-axis) averaged over 20 runs.

Figure 11 :

 11 Figure11: Estimation error associated to a MMSBM with parameters n = 5000, K (x-axis), p, q and n 0 averaged over 20 runs.

  ib | will be controlled uniformly (see Condition C3).

Finally

  , it remains to bound |H (t) 11 |. To begin with,

Figure 12 :

 12 Figure 12: NMI vs ρ for different values of L.

Figure 13 :

 13 Figure 13: NMI vs L for different values of ρ.

Figure 14 :

 14 Figure 14: NMI vs n for different values of ρ.

Figure 15 :

 15 Figure15: NMI for varying n with ρ = 0.9.

  Ties are broken arbitrarily. Algorithm 10 Generalized Power Method (GPM) Input: Number of communities K, adjacency matrix A, an estimate of the row partition Ẑ(0) 1 and, number of iterations T .

Figure 16 :Figure 17 :

 1617 Figure 16: Relative performance of Spec, HL and GPM for varying c.

Figure 18 :

 18 Figure 18: Relative performance of Spec, HL and GPM for varying C.

Figure 19 :

 19 Figure 19: Relative performance of Spec, HL and GPM for varying K.

  Concentration of the oracle error Lemma 19. Assume that the assumptions of Theorem 12 hold. Recall that

j

  where the eigenvalues λ 1 ⩾ . . . ⩾ λ n of A (resp. µ 1 ⩾ . . . ⩾ µ n of B) are ordered in decreasing order. Then, as shown in Corollary 2.2 in[START_REF] Fan | Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model[END_REF]

  is the distribution of a Galton-Watson (GW) tree (of length d) with offspring given by a Poisson law with parameter λ and P (d) 1 (T , T ′ ) is the distribution associated to a colored GW tree where edges can be red, blue or bicolored. The offsprings associated with red edges (resp. blue and bicolored) are drawn by independent Poisson law with parameter λ(1 -s) (resp. λ(1 -s) and λs).

  , T ′ ) (T and T ′ are generated in a correlated way) and P ′ ) (T and T ′ are generated independently). One can write the likelihood ratio

  τ(C) := {Output of GMWM with input C} (7.2.1) which can be considered a projection since τ 2 (C) = τ(C) for all C ∈ R n×n . Notice that, in general, the output of GMWM will be different from solving the linear assignment problem τ(C) := argmin{∥C -X∥ F | X ∈ P n } = argmax Π∈P n ⟨Π, C⟩ F

Figure 20 :

 20 Figure 20: The constant c(σ) (re-scaled multiplying by 384) appearing in Theorem 17.

  e κn/20 ⩽ e -cκn

Figure 25 :

 25 Figure 25: Initial overlap is equal to 0.5

Figure 26 :

 26 Figure 26: Initial overlap is equal to 0.1

Figure 27 :

 27 Figure 27: Heatmap for the recovery rate of PPMGM algorithm with input (A spar 1 , B spar 1 ) for different threshold values τ i (y axis); i = 1, • • • , 6, and different values of σ (x axis). Here τ i corresponds to the solution of (7.5.2) with n = 1500 and p i for i = 1, 2 • • • , 6 in a uniform grid between p 1 = 42 × 10 -3 and p 6 = 54 × 10 -3 .

A

  high probability bound for C ij , i ̸ = j. Let us first define the vectors ãi , ãj ∈ R n as ãi (k) :=    A ki , for k / ∈ {j, x -1 (i)}, 0, for k ∈ {j, x -1 (i)}, kj , for k / ∈ {j, x -1 (i)}, 0, for k ∈ {j, x -1 (i)}.

  2 ) 2 , which is valid for 0 ⩽ δ ⩽ s x 2 √

  Prop. 4 part (ii). Define Ẽj := {C ii ⩽ s σ,x } ∪ {C ij ⩾ s σ,x }, which clearly satisfies {C ii ⩽ C ij } ⊂ Ẽj .Then by the union bound,P(∪ j̸ =i Ẽj ) ⩽ P(C ii ⩽ s σ,x ) + j̸ =i P(C ij ⩾ s σ,x )

  and taking the union bound over all indicesi ̸ = j. E R A L C O N C E N T R AT I O N I N E Q U A L I T I E S Lemma 32 (Chernoff bound -multiplicative form). Let X = i⩽n X iwhere X i ind.

  When the random variables are only centered, we can use the symmetrization argument of Corollary 3.3 to finally obtain||EZ ⊤ 2 || ≲ n 1 n 2 p max /L,with probability at least 1 -

Table 1

 1

		Comparison between computation times (av-
		eraged over 20 runs).	35
	Table 2	NMI vs ρ for MIT Reality Mining dataset.	89
	Table 3	NMI vs ρ for Malaria parasite genes network.	90
	L I S T O F A L G O R I T H M S
	Algorithm	Iterative Refinement with Least Squares	25
	Algorithm	EM on graph embedding and covariates (EM-Emb)	27
	Algorithm	IR-SSBM	35
	Algorithm	Generalized Power Method (GPM)	41
	Algorithm	k-pod clustering	76
	Algorithm	Sum of adjacency matrices with missing en-
		tries filled with zeros	81
	Algorithm	Sum of adjacency matrices with missing en-
		tries filled iteratively	85
	xiv

CSBM Contextual Stochastic Block Model CSSBM Contextual Symmetric SBM MLSBM Multi-Layer Stochastic Block Model DCSBM Degree Corrected SBM MMSBM Mixed Membership SBM RDPG Random Dot Product Graph BiSBM Bipartite Stochastic Block Model

  . Seeded graph matching for the correlated Wigner model via the projected power method. arXiv: 2204.04099 [math.ST].Braun, Guillaume and Hemant Tyagi (2022a). Minimax Optimal Clustering of Bipartite Graphs with a Generalized Power Method. arXiv:

2205.12104 [math.ST].

  2015 that can handle missing values, see Algorithm 5. Let us describe the principle behind this algorithm. The classical k-means problem seeks a partition Z and centroid values (encoded in the matrix C) that solves

	min Z∈M n,K	|| Û -ZM|| 2 F .
	C∈R K×KL
	When there are missing values one can instead solve
	min Z∈M n,K	||( Û -ZM) ⊙ Ω U || 2 F	(4.3.1)
	C∈R K×KL

  1 log n by assumption, there are nodes in T k and T k ′ that are not in C 1 . Hence there is another cluster C 2 induced by Û containing nodes from two different communities. The same argument can be applied to C 2 and iteratively to C 3 , . . . , C K . At the end, since the C k form a partition of the set of nodes, we obtain

Table 2 :

 2 NMI vs ρ for MIT Reality Mining dataset.

			OLMFm sumAdjIter
	1	1.00	1.00	1.00
	0.9	0.99	0.96	0.99
	0.8	0.97	0.86	0.97
	0.7	0.96	0.93	0.96
	0.6	0.94	0.79	0.94
	0.5	0.89	0.91	0.90
	0.4	0.76	0.73	0.78
	0.3	0.56	0.57	0.62
	0.2	0.26	0.41	0.36
	0.1	0.09	0.10	0.11

Table 3 :

 3 NMI vs ρ for Malaria parasite genes network.

			OLMFm sumAdjIter
	1	1.00	0.99	1.00
	0.9	0.75	0.75	0.72
	0.8	0.63	0.62	0.58
	0.7	0.47	0.49	0.47
	0.6	0.32	0.37	0.34
	0.5	0.22	0.20	0.26
	0.4	0.13	0.07	0.16

  1/2 and O is the orthogonal matrix that aligns Û and U.

Proof. By Remark 3.13 in Bandeira et al., 2016 we get that ||A -E(A)|| ⩽ C max(np max , log n)

  This estimator was discussed earlier in Section 4.5.1. It was shown by Paul et al., 2020 that if np max ⩾ C log n and at least one of the matrices Π

	r coreg = O	log n Lnp max	.
	• OLMF.

. It is an intermediate fusion method that aims to find the best set of eigenvectors that simultaneously approximate the set of eigenvectors associated with each individual layer. It was shown later by Paul et al., 2020 that if Lnp max ⩾ C log n and Π

(l)

is full rank for all l, then with high probability (w.h.p)

  by combining Lemma 25, 26 and 27. Condition on any realization of G, G ′ , g such that the event E ′ ∩ E holds. Let i ∈ Ĩ ∩ Ĩ′ . By definition of E ′ ∩ E, we have

		n}
	for a suitably large constant C > 0 (which is the constant hidden in
	the O(•) symbol in Lemma 26). If n is sufficiently large and κ satisfies √ 1 -σ 2 > 48κ, one can show that
	P(E ′ ∩ E) ⩾ P(E) -	3 n 2

  Performance of PPMGM as a refinement of Grampa and Umeyama algorithms, compared with PPM with a random initialization x

	1				Grampa+PPMGM
	0.9				Grampa Umeyama
					Umeyama+PPMGM
	0.8				PPMGM rand.init
	0.7
	0.6
	0.5
	0.4
	0.3
	0.2
	0.1
	0
	0	0.2	0.4	0.6	0.8	1
	Figure 21:
					5.1 we generate

  Figure23: PPMGM with an initialization such that overlap(x (0) , x * ) = 0.1.Here it.1, it.2, it.3, it.4, it.5 corresponds to 1, 2, 4, 8 and 30 iterations respectively.Figure24: Here it.1, it.2, it.3, it.4 corresponds to 1, 2, 4 and 8 iterations respectively.

		1
	1	0.9			PPMGM it.1
	0.8 0.9	0.8 0.7	PPMGM PPMGM thr.1 PPMGM thr.2 PPMGM topk		PPMGM it.2 PPMGM it.3 PPMGM it.4 PPMGM it.5
	0.7	0.6
	0.6	0.5
	0.5	0.4
	0.4	0.3
	0.3	0.2
	0.2	0.1
	0.1	0
	0	0	0.2	0.4	0.6	0.8	1
	0	0.2	0.4	0.6	0.8	1
	1
	0.9	PPMGM it.1 PPMGM it.2
		PPMGM it.3
	0.8	PPMGM it.4
	0.7
	0.6
	0.5
	0.4
	0.3
	0.2
	0.1
	0
	0	0.2	0.4	0.6	0.8	1

  . SeeMitzenmacher et al., 2005, Theorem 4.5 and Corollary 4.6 .Lemma 33 (Hoeffding inequality). Let X 1 , . . . , X n Lemma 34 (Matrix Bernstein inequality). Let X 1 , . . . , X n be a sequence of independent zero-mean random matrices of size d 1 × d 2 . Suppose that ||X i || ⩽ M almost surely, for all i. Then for all positive t, ⩽ (d1 + d 2 ) exp -t 2 2σ 2 + 2M/3twhere σ 2 = max(|| i E(X i X * Then with probability at least 1 -n -Ω(1) the following holds. ||EW|| 2 F ≲ K 2 p max .Proof. The first inequality is a classical result used for SBM in the relatively sparse regime p max = ω(log n). It can obtained as a consequence of Remark 3.13 in[START_REF] Bandeira | Sharp nonasymptotic bounds on the norm of random matrices with independent entries[END_REF] The second inequality follows from||EW|| 2 F ⩽ K||EW|| 2 ⩽ K||E|| 2 ||W|| 2 ≲ K 2 p max .Lemma 36. Assume that A ∼ BiSBM(Z 1 , Z 2 , Π), and recall E = A -E(A). For n 1 large enough, the following holds true.1. P(||E|| ≲ √ n 2 p max ) ⩾ 1 -n -Ω(1) 1 if n 2 p max ≳ log n 2 and n 2 ⩾ n 1 . 2. P(||EE ⊤ -E(EE ⊤ )|| ≲ max(log n 1 , √ n 1 n 2 p max )) ⩾ 1 -n . Moreover, this event implies H(EE ⊤ ) ≲ max(log n 1 , √ n 1 n 2 p max )).3. P(||EZ2 || ≲ n 1 n 2 p max /L) ⩾ 1 -nThe first inequality follows from classical proof techniques. We first convert E into a square symmetric matrix as follows It is easy to verify that Ẽ = ∥E∥. Since Ẽ is a square symmetric matrix with independent entries, we can use the result of Remark 3.13 in Bandeira et al., 2016 and obtain with probability at least 1 -n max since n 2 p max ≳ log(n 1 + n 2 ). This condition is ensured since n 2 ⩾ n 1 and n 2 p max ≳ log n 2 . 2. The second inequality follows from the recent work of Cai et al., 2022. First observe that ||H(EE ⊤ )|| ⩽ 2||EE √ n 1 n 2 p max , log n 1 ). It is however more difficult to get a high probability bound from this last inequality since we can no longer use Talagrand's inequality as in Bandeira et al., 2016. However, we can use the moments to obtain a tail bound as in Theorem 5 in Cai et al., 2022. This theorem is stated for matrices with Gaussian entries, but if instead of Lemma 1 we use Lemma 9 of Cai et al., 2022, we obtain a similar result for bounded sub-Gaussian entries. Since the variance parameters σ R and σ C that appear in the statement of Theorem 5 in Cai et al., 2022 satisfy σ 2 R ⩽ n 1 p max and σ 2 C ⩽ n 2 p max we obtain the result.

	ind. ∼ B(p). Then -Ω(1) 1 if n 2 p max ≳ L log n 1 1 0 E E ⊤ 0 . 2. -Ω(1) P   1 n i∈[n] P(|| 1. ||E|| ⩽ √ np max , and n 1 n 2 p max L ≳ log n 1 . Proof. 1. Ẽ = -Ω(1) 1 that Ẽ ≲ √ n 2 p

X i -p ⩾ t   ⩽ 2 exp(-2nt 2 ).

Proof. See

Boucheron et al., 2013, Theorem 2.8

.

i X i || ⩾ t) i )||, || i E(X * i X i )||).

Proof. See Tropp, 2012, Theorem 1.6 Lemma 35. Assume that A ∼ SBM(Z, Π). Let E = A -E(A). ⊤ -E(EE ⊤ )|| since ||H(A)|| ⩽ ||A|| + || diag(A)|| ⩽ 2||A|| for all square matrices A and H(EE ⊤ ) = H(EE ⊤ -E(EE ⊤ )). Theorem 4 in Cai et al., 2022 shows that E(||EE ⊤ -E(EE ⊤ )||) ≲ max(

We can assume w.l.o.g that the first n/2 entries are in community 1.

One can also use the random-walk Laplacian L w = I -D -1 A.

The source code is available on https://github.com/glmbraun/CSBM

The algorithm proposed by[START_REF] Majmudar | Provable Overlapping Community Detection in Weighted Graphs[END_REF] is for weighted adjacency matrix, and the matrix D -1/2 A 2 D -1/2 can be interpreted as a Gram matrix

Depending on the context we will interchangeably use the notation z i and a.

We drop the superscript (t) in the notation for H 1 , H 2 for convenience, but clearly they both depend on t as well.

It is easy to verify that Û(l) is also the eigenvector matrix corresponding to the top K eigenvalues (in absolute value) of A (l) ⊙ Ω(l) .

This remark regarding the complexity applies to our other methods as well.

n .1 This is weaker than the usual notion of diagonal dominance, where for all i ∈[n] |M ii | ⩾ j̸ =i |M ij |.

correlated Wigner graphs A, B ∼ W(n, σ, x * ) for a random permutation x * , and apply to A, B the spectral algorithms Grampa [START_REF] Fan | Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model[END_REF] and the classic Umeyama [START_REF] Umeyama | An eigendecomposition approach to weighted graph matching problems[END_REF], both of which work in the seedless case. As a second step, we apply algorithm PPMGM with the initialization given by the output of Grampa and Umeyama. We show experimentally that by applying PPMGM the solution obtained in both cases improves, when measured as the overlap (defined in (7.3.2)) of the output with the ground truth. We also run experiments by initializing PPMGM with X (0) randomly chosen at a certain distance of the ground truth permutation X * . Specifically, we select X (0) uniformly at random from the set of permutation matrices that satisfy ∥X (0) -X * ∥ F = θ ′ √ n, and vary the value of θ ′ ∈ (0, 1). In Section 7.5.2 we run algorithm PPMGM with different pairs of input matrices. We consider the Wigner correlated matrices A, B and also the pairs of matrices (A spar 1 , B spar 1 ), (A spar 2 , B spar 2 ) and (A spar 3 , B spar 3 ), which are produced from A, B by means of a sparsification procedure (detailed in Section 7.5.2). The main idea behind this setting is that, to the best of our knowledge, the best theoretical guarantees for exact graph matching have been obtained in Mao et al., 2021a for relatively sparse Erdös-Renyi graphs. The algorithm proposed in Mao et al., 2021a has two steps, the first of which is a seedless type algorithm which produces a partially correct matching, that is later refined with a second algorithm Mao et al., 2021a, Alg.4. Their proposed algorithm RefinedMatching shares similarities with PPMGM and with algorithms 1-hop [START_REF] Lubars | Correcting the Output of approximate Graph matching algorithms[END_REF]Yu et al., 2021a) and 2-hop (Yu et al., 2021a). Formulated as it is, RefinedMatching (Mao et al., 2021a) (and the same is true for 2-hop for that matter) only accepts binary edge graphs as input and also uses a threshold-based rounding approach instead of Algorithm 11, which might be difficult to calibrate in practice. With this we address experimentally the fact that the analysis (and algorithms) in Mao et al., 2021a do not extend automatically to a simple binarization of the (dense) Wigner matrices, and that specially in high noise regimes, the sparsification strategies do not perform very well.

Performance of PPMGM

In Figure 21 we plot the recovery fraction, which is defined as the overlap (see (7.3.2)) between the ground truth permutation and the output of: Grampa, Umeyama, Grampa+PPMGM, Umeyama+PPMGM and PPMGM. The algorithms Grampa+PPMGM and Umeyama+PPMGM use the output of Grampa and Umeyama as seeds for PPMGM, which is performed with N = 1. In the algorithm PPMGM, we use an initial permutation x (0) ∈ S n chosen uniformly at random in the set of permutations such that overlap (x (0) , x * ) = 0.08; this is referred to as 'PPMGM rand.init'. We take n = 800 and consider the average overlap over 25 Monte Carlo 1, in.2, in.3, in.4 corresponds to and overlap of x (0) with the ground truth of 0.05, 0.1, 0.15 and 0.5 respectively.

N = 8 and N = 30. In Figure 24 we use a matrix of size n = 1000 and we see that the difference between using N = 1 and N > 1 is even less pronounced ( we omit the case of 30 iterations for readability purposes, as it is very similar to N = 8).

Sparsification strategies

Here we run PPMGM using different input matrices which are all transformations of the Wigner correlated matrices A, B. Specifically, we compare PPMGM with A, B as input with the application of PPMGM to three different pairs of input matrices (A spar 1 , B spar 1 ), (A spar 2 , B spar 2 ) and (A spar 3 , B spar 3 ) that are defined as follows.

A

where τ > 0 and for k ∈ N and a n × n matrix M, top k (M i: ) is the set of the k largest elements (breaking ties arbitrarily) of M i: (the i-th row of M). The choice of the parameter τ is mainly determined by the sparsity assumptions in (Mao et al., 2021a

Replacing t = t := n 4 ( 1 + s x 2 -2 n -1) 2 in the previous expression and noticing that t ⩾ 1 20 s 2 x n for s x ∈ (10/n, 1] we get

where we used that n ⩾ 10.

Proof of Claim 2. Observe that when x(j) = i (or equivalently x -1 (i) = j) we have

In the case x(j) ̸ = i, let us define

which are all independent Gaussians random variables. Moreover, ψ 1 ∼ N(0, 1) and

Consider the case i, j / ∈ S X . In this case, it holds

, where ζ 1 and ζ 2 are independent χ 2 1 random variables. The proof for the other cases is analogous.

Proof of Proposition 4 part (ii)

Now we consider the case where σ ̸ = 0. It is easy to see that here the analysis of the noiseless case still applies (up to re-scaling by √ 1 -σ 2 ) for the matrix C ′ = AXA. We can proceed in an analogous way for the matrix C ′′ = AXZ which will complete the analysis (recalling that

Before we proceed with the proof, we explain how the tail analysis of entries of C ′ in Prop.4 part (i) helps us with the tail analysis of C ′′ . Observe that for each i, j ∈ [n] we have

The term C ′′ ij , for all i, j ∈ [n], can be controlled similarly to the term C ′ i ′ j ′ (when i ′ ̸ = j ′ ). Indeed, we have the following Lemma 30. For t ⩾ 0 we have

Consequently,

Proof. We define h 1 := 1 2 (A :i + XZ :j ) and h 2 := 1 2 (A :i -XZ :j ). It is easy to see that h 1 and h 2 are two i.i.d Gaussian vectors of dimension n. By the polarization identity, we have

4 , 1}. For µ i (and the same is true for ν i ) the following two cases can happen: either n -1 of its entries are 1/ √ 2 and one entry takes the value 1 (when i = j) or n -2 of its entries are 1/ √ 2 and two entries take the value 3/4 (when i ̸ = j). In any of those cases, one can readily see that

Using Corollary 3 we obtain

Arguing as in the proof of Proposition 4 part (i) we obtain the bound

Now we introduce some definitions that will be used in the proof. We define s σ,x := 1 2 √ 1 -σ 2 s x , and for δ > 0, i, j ∈ [n], we define the following events

The following lemma corresponds to [START_REF] Laurent | Adaptive estimation of a quadratic functional by model selection[END_REF], Lemma 1.1) and controls the tails of the weighted sums of squares of Gaussian random variables.

Lemma 37 (Laurent-Massart bound). Let X 1 , • • • , X n be i.i.d standard Gaussian random variables. Let µ = (µ 1 , • • • , µ n ) be a vector with nonnegative entries and define ζ = n i=1 µ i (X 2 i -1). Then it holds for all t ⩾ 0 that

An immediate corollary now follows.

The next lemma give us a distributional equality for terms of the form ⟨g, Xg⟩ where g is a standard Gaussian vector and X is a permutation matrix.

Lemma 38. Let X ∈ P n and g = (g 1 , • • • , g n ) be a standard Gaussian vector. Then is holds

where λ i are the eigenvalues of 1 2 (X + X T ) and g ′ = (g 1 , • • • , g n ) is a vector of independent standard Gaussians. Moreover, if |S X | = s x n for s x ∈ (0, 1], µ ∈ R n 1 is a vector containing the positive eigenvalues of 1 2 (X + X T ), and -ν ∈ R n 2 is a vector containing the negative eigenvalues of 1 2 (X + X T ), then

Proof. Notice that ⟨g, Xg⟩ = ⟨g, 1 2 (X + X T )g⟩ and given the symmetry of the matrix 1 2 (X + X T ) all its eigenvalues are real. Take its SVD decomposition 1 2 (X + X T ) = VΛV T . We have that ⟨g, 1 2 (X + X T )g⟩ = (V T g)