Comme tout étudiant lisant une thèse, je me suis souvent arrêté sur les remerciements sans aller plus loin. Parfois court, souvent long, c'est enfin le lieu pour remercier tout ceux qui ont généreusement et passionnément contribué à cette petite fin qu'est la thèse de doctorat. On en a souvent les larmes aux yeux d'ailleurs, car on meurt toujours un peu quand quelquechose se termine.

J'aimerais ici remercier tout ceux qui de près ou de loin ont contribué à cette thèse et dont la rencontre à fait de moi ce que je suis.

Naturellement je commence par le good cop et le bad cop, alias le codeur fou et le fou codeur, sans qu'un de ces sobriquets ne désigne l'un plutôt que l'autre. Si je dis le petit et le grand, là par contre... Je parle bien entendu de Julien et Pierre. HDR donnant la primeur, mille MERCIS cher directeur sosie du fabuleux Marty McFly, grand maître codeur et pourvoyeur de champagne aux fines bulles. À la fois guide, bienveillant et botteur de fesses tu as tout du parfait directeur de thèse. Évidemment sans toi cette thèse n'aurait jamais eu lieu. D'abord pourvoyeur d'HDR manquante tu m'as par la suite beaucoup appris, sur la forme comme sur le fond. À coder bien sûr, proprement et efficacement. À écrire aussi et surtout à chercher comme tu sais bien le faire. Je n'oublierai pas ce que tu m'as enseigné, conscient de la chance que j'ai je garde ce que tu m'as appris comme un trésor.

Pierre ! La tranquillité en complément de la vivacité. Le brave et le fort, l'ombre et la flamme, chevalier des temps modernes alias Ned Stark. Colosse et virevoltant que tu es, ton jeu physique au foot restera dans les mémoires. Codeur génial à la répartie sanglante, tes "non mais tu rigoles" passeront à la postérité. Un grand MERCI. Pour avoir proposé avec Sophie et Sarah le sujet de stage qui est devenu sujet de thèse ensuite et pour avoir bien voulu co-encadrer ma thèse. Pour ta bienveillance au cours de ces 3 dernières années, le temps passé à discuter et à répondre à mes nombreuses questions. J'ai beaucoup appris en te regardant raisonner et je suis très admiratif de ta compréhension fine des modèles et des données. Souvent j'arrivais avec un problème et je repartais plein d'espoir, à raison, en ce qu'on s'était dit pour le résoudre. Tout comme Julien tu fais un directeur comme beaucoup aimeraient et auraient la chance d'en avoir un. Pour tout ce que je garde de nos échanges et de ce que j'ai appris en travaillant avec vous deux, encore merci.

Merci chaleureusement à Eric Kolaczyk et à Pierre Latouche d'avoir eu la gentillesse d'accepter de rapporter ma thèse et d'avoir été présents le jour de la soutenance. Merci pour vos relectures soigneuses et bienveillantes. J'ai personnellement beaucoup étudié et été très inspiré par vos travaux de recherche respectifs. Merci en particulier à Eric pour les discussions à Eindhoven où j'ai pu profiter en direct de vos talents de chercheur et de votre pédagogie.

Merci à Julie Josse et à Tabea Rebafka d'avoir accepté d'être dans mon jury de thèse. Merci en particulier à Julie de m'avoir invité à parler au séminaire des doctorants du CMAP.

Merci à Sophie et Sarah de m'avoir encadré pendant mon stage de M2, je suis très reconnaissant de toute l'attention et de la gentillesse que vous m'avez montrées pendant ces 3 ans et demi. J'ai beaucoup aimé parler avec chacune et faire de la recherche avec vous.

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Merci à Mahendra d'avoir accepté de collaborer avec moi et de m'avoir accueilli à Jouy régulièrement. Merci de ta disponibilité et de ta grande gentillesse, c'était une vrai chance de travailler avec toi.

Un grand merci au groupe MIRES de m'avoir accueilli au cours de ces 3 dernières années. J'ai été grâce à vous le témoin privilégié de beaux moments de partage et de collaboration scientifique. Un merci particulier à Vanesse, Christian, Mathieu et Matthieu pour vos échanges passionnants et sincères.

Je voudrais ici remercier chaleureusement tout le laboratoire de statistique de l'Agro dans lequel j'ai passé ces 3 dernières années. L'ambiance de travail, vos qualités de chercheurs et vos amitiés ont réellement contribué à me faire grandir à bien des niveaux. Soyez tous assurés de ma gratitude et de mon amitié. Merci Émilie, Maud, Liliane, Christophe, Christophe, Christophe, Marie-Laure, Julie, Éric, Céline, Tristan, Michel, Laure, Jessica, Isabelle, Séverine. Une mention spéciale pour le grand chef Gabriel sans qui les conseils de labo seraient moroses, là où tu passes la folie et les rires suivent ! Un merci particulier à mes co-bureaux successifs et amis. Je pense en premier à Pierre, Marie et Anna qui m'ont accueilli en début de thèse, votre gentillesse et vos bons conseils sans oublier ce tour du monde en bateau virtuel resteront de super souvenirs. Merci Marie, Mathieu, James et Raphaëlle pour avoir partagé avec enthousiasme le BDDSS à tour de rôle. Marie, ta joie permanente et ta gentillesse me resteront à jamais, merci de ton amitié. Mathieu, pour ton amitié et toutes nos discussions je te remercie chaleureusement, j'espère que nos routes se recroiseront à l'avenir :) Et le dernier et non le moindre, merci mon cher Félix pour ta gentillesse et ton amitié sans faille, je te dois beaucoup dans mon admission en médecine et te suis à jamais reconnaissant. Merci Paul, pour ta discrétion et ton exemple, c'est toujours une joie de te croiser. Merci enfin à Saint-Clair mon petit frère de thèse, je te souhaite le meilleur pour la suite, à Yann pour ton amitié et ta gentillesse, à Jade, Rhana, Annarosa, Martina, et Joe pour votre grande gentillesse.

Merci à tous ceux que j'ai rencontrés au cours de séminaires et conférences, qui par leurs idées et questions, m'ont aidés à mieux comprendre et à me poser plus de questions sur mes travaux. Je pense en particulier à Vincent, toujours souriant et gentil, dont les travaux m'ont beaucoup apporté, tu es sûrement la personne dont je connais le mieux la thèse :) Sylvain, grand frère de tous à l'Agro, généreux et souriant, tu inspires beaucoup de monde. J'aurais beaucoup aimé continuer et apprendre avec toi. Jean-Benoist l'irréductible, ta passion et vision profonde des statistiques et de l'informatique forcent le respect ! Merci de ton invitation à Compiègne dont je garde un très bon souvenir :) Merci enfin à Valérie pour ton sourire et ta gentillesse.

Je voudrais remercier maintenant avec beaucoup de gratitude et d'émotion tout ceux à qui je dois d'être là aujourd'hui. Je pense à tous mes professeurs de lycée qui ont cru en moi, et à mes professeurs à l'université. Je pense à Mme Carré qui m'a encouragé et a cru en moi alors que je redoublais et surtout à Mme Berton grâce à qui je suis allé étudier à Orsay. Un grand merci à Dominique Hulin et Jean-Christophe Léger, responsable respectivement des L3 et des L1, à l'époque, à Orsay. La gentillesse et l'énergie que vous avez déployées à croire en chacun de vos étudiants sont pour beaucoup dans ma réussite. J'ai une pensée pour tous les professeurs passionnés qui m'ont enseigné durant mon cursus, soyez tous sincèrement remerciés. Je pense en particulier à Aurélien Galateau qui a été pour moi un exemple et une motivation, merci pour votre gentillesse et votre attention. Un grand merci aussi à toute l'équipe des matheux de l'IUT d'Orsay qui m'ont accueilli avec beaucoup de gentillesse pour faire mon monitorat de thèse.

Je voudrais ici remercier tous mes amis et co-étudiants de mathématiques avec qui j'ai buché et souffert pendant mes années d'études et de préparation de l'agrégation. Je pense à Arnaud, Justine, Victor, Nadège, Yvann, Vincent, Ridwann, Antoine, Antoine, Amandine, Dimitri, Éric, Laurent et Ghislain. Pour vos amitiés qui durent et tous les bons moments partagés je vous dis un grand MERCI. Je n'oublie pas la petite Margaux qui ira loin c'est sûr :) Merci de ton amitié et de ton sourire ;) Enfin, merci à toute la promo du M2 MSV, 4 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q c'était un réel plaisir de travailler avec vous et de vous connaître, je pense en particulier à Christophe dont j'espère que les aventures Montpelliéraine se passent bien :) Je n'oublie pas Christophe Giraud, merci de votre attention et de ne jamais compter votre temps pour répondre à nos questions et nous guider.

Merci à Matéo pour ton soutient et ton amitié ;) Merci à ma belle famille au complet, pour votre soutient sans faille. Pour me suivre dans tout ce que je fais de fou et pour ne jamais cesser de chercher à comprendre. Merci pour votre amour et confiance, c'est très important pour moi.

Merci à mes frères et soeurs, à Laure, Emmanuelle, Emmanuel, Tiphaine, Elise et Briac. Pour tout ce qui nous rassemble et tout ce que vous êtes, pour votre amour et parce que je vous aime, MERCI.

Merci à mes parents, sans qui je ne serai rien. Pour votre exemple en tout, votre soutient et votre amour inconditionnel. Parce que je vous doit tout et bien plus encore, MERCI du fond du coeur.

Merci à celui qui guide nos pas sans jamais nous abandonner et donne sens à toutes choses.

Merci à mon petit Octave, si adorable. T'avoir parmi nous est un vrai bonheur.

Enfin merci à celle qui m'accepte comme je suis, qui met de la lumière chaque jour que Dieu fait et m'aime peu importe le temps qu'il fait en moi. MERCI mon amour, cette thèse t'es dédiée évidemment. 5 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q CONTENTS CONTENTS 8 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q List of Symbols q n is the number of nodes in the network, q N = 1, n the set of nodes, q D = N × N the set of dyads, q Y ∈ M n (R) the adjacency matrix of the network, q Q ∈ N the number of blocks (or groups), q Q = 1, Q the set of blocks, q z ∈ Q n the vector of block memberships, q Z i ∈ {0, 1} Q the vector such that (Z i ) q = Z iq = 1 {zi=q} , q X i ∈ R N some covariates of node i such that (X i ) q = X iq , q X ij ∈ R m some covariates of dyad (i, j), q X the set of covariates ({X i } i on nodes and {X ij } ij on dyads),

q R ∈ M n (R) the sampling matrix : R ij = 1 si Y ij sampled, 0 otherwise, q V ∈ {0, 1} n the sampling vector indicating which node is sampled (V i = 1) or not (V i = 0), q Y o = {Y ij : R ij = 1} the observed part of the adjacency matrix, q Y m = {Y ij : R ij = 0} the non-observed part of the adjacency matrix, q D o = {(i, j) : R ij = 1} the observed dyads, q D m = {(i, j) : R ij = 0} the non-observed dyads.

CONTENTS CONTENTS 10 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction Introduction Introduction

ÉTAT DE L'ART INTRODUCTION

réduit, nous ne sommes pas en mesure de dire quels ont été les échanges ; seuls leurs liens avec les interrogés sont connus.

Le second exemple est celui d'un réseau de co-régulation de gènes. Le réseau extrait de la plateforme string [START_REF] Szklarczyk | String v10: protein-protein interaction networks, integrated over the tree of life[END_REF], accessible à l'adresse http://www.string-db. org, est le réseau de voisinage de la protéine ER (pour Estrogen Receptor) codé par le gène ESR1 (pour EStrogen Receptor 1). Ce réseau est composé de 741 protéines (ou gènes codant ces protéines) et chaque dyade (i.e. une paire de noeud) est pondérée par un score appartenant à [0, 1] reflétant le niveau de confiance calculé sur la base d'expériences ou de travaux de recherche. Plus la valeur est proche de 1 plus la probabilité que cette arête existe est grande. Notant ω ij le poids associé à la dyade (i, j), nous définissons la matrice dépendant du seuil γ :

A γ = (A γ ) i,j =    1 if ω ij > 1 -γ, NA if γ ≤ ω ij ≤ 1 -γ, 0 if ω ij < γ.
(1.1)

Cette façon d'échantillonner la matrice des poids associés aux arêtes est cette fois-ci centrée sur les dyades.

Remarque 1. Dans la suite on notera les entrées manquantes dans les dyades par NA.

État de l'art

Dans cette section, nous allons présenter des modèles de graphes aléatoires, leurs spécificités et quelques résultats théoriques justifiant l'emploi de certains plutôt que d'autres. Ensuite, nous présenterons quelques méthodes d'inférence de ces modèles et leurs limites. Enfin nous introduirons la problématique des données manquantes et l'échantillonnage des réseaux.

Modèles probabilistes de graphes aléatoires

Nous présentons ici une revue de modèles probabilistes de graphes aléatoires. Les modèles génératifs de graphes comme les modèles petit monde (E J [START_REF] Newman | Random graph models of social networks[END_REF] et d'attachement préférentiel [START_REF] Barabási | Emergence of scaling in random networks[END_REF] ne seront pas présentés ici. Nous commençons par le premier modèle (historiquement parlant) défini par Erdős et Rényi. Puis nous continuons avec les modèles de graphes aléatoires géométriques et exponentiels. Enfin nous aborderons la grande classe des modèles de graphes aléatoires à espaces d'états latents, utilisés pour modéliser l'hétérogénéité des connexions observée dans un réseau. Rappelons qu'en mathématiques, un graphe G est la donnée du couple ({Noeuds}, {Arêtes}). L'ensemble des arêtes est constitué des dyades dont la valeur associée est non nulle. On représente G par une matrice d'adjacence Y = (Y ij ) 1≤i,j≤n ∈ M n (R) où n est le nombre de noeuds et Y ij = 1 s'il y a une arête entre les noeuds i et j, Y ij = 0 sinon. Cette matrice peut contenir d'autres valeurs que 0 ou 1. C'est le cas des graphes pondérés qui ne sont pas le sujet principal de cette thèse. D'éventuelles covariables associées aux noeuds seront désignées par la variable X = (X i ) 1≤i≤n . Si en revanche elles sont associées aux dyades, on les notera X = (X ij ) 1≤i,j≤n . Elles peuvent être catégorielles, quantitatives ou bien un mélange des deux. Rappelons que N := 1, n désigne l'ensemble des noeuds du graphe et D := N × N l'ensemble des dyades. Finalement nous noterons θ et Θ respectivement le paramètre associé à un modèle et l'espace des paramètres. Par la suite nous nous intéresserons plus particulièrement au cas de graphes binaires (sauf mention du contraire) non-orientés (i.e. dont la matrice d'adjacence Y est symétrique) et sans boucles (i.e. la diagonale de Y est nulle). Ce choix est fait pour plus de simplicité. Cependant tout ce qui suit s'étend naturellement aux cas de graphes pondérés et/ou nonsymétriques et/ou avec des boucles. 12 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q INTRODUCTION 2. ÉTAT DE L'ART

Modèle d'Erdős-Rényi

Soit p ∈ [0, 1] une probabilité. Ce modèle couramment noté G(n, p) est défini comme suit :

(Y ij ) i,j ∼ iid B(p)
où B est la loi de Bernoulli. On note que la loi des degrés est binomiale : D i := j Y ij ∼ Bin(n -1, p). Citons l'article [START_REF] Gilbert | Random graphs[END_REF] et le livre plus récent [START_REF] Bollobas | Random Graphs[END_REF] dans lequel ce modèle a été largement étudié.

Modèles de graphes aléatoires géométriques

Les modèles de graphes aléatoires géométriques sont généralement utilisés pour modéliser des réseaux de télécommunications sans fil ou bien des propagations de phénomènes naturels tels que des incendies de forêt ou encore des épidémies. Nous citons le livre de [START_REF] Penrose | Random Geometric Graphs[END_REF] qui est une référence incontournable concernant cette classe de modèles. Dans ces modèles, les noeuds sont identifiés à des positions spatiales aléatoires (N i ) 1≤i≤n dans un espace métrique (E, d) suivant une certaine distribution (uniforme ou gaussienne par exemple). De plus, conditionnellement à la position des noeuds et en se fixant un seuil r > 0, la matrice d'adjacence Y est déterministe. Plus précisément on a (Y ij ) i,j = 1 d(Ni,Nj )≤r .

À noter le lien entre ces modèles et le précédent : on peut montrer que localement, les graphes géométriques contiennent un graphe d 'Erdős-Rényi (voir Channarond, 2013).

Modèle Expected Degree (EDD)

Adapté des modèles définis dans [START_REF] Chung | Connected components in random graphs with given expected degree sequences[END_REF] et [START_REF] Newman | The structure and function of complex networks[END_REF], le modèle EDD permet de contrôler en espérance les degrés de chaque noeud. Dans la suite on notera (K i ) i les « degrés attendus » (non nécessairement entiers) associés à chaque noeud et G une densité de probabilité. Alors,

(K i ) i ∼ iid G (Y ij ) i,j | (K i ), (K j ) ∼ ind B(K i K j /κ), κ ∈ R. De cette façon, E[D i |K i ] ∝ K i , ∀i ∈ 1, n .

Modèle de graphes aléatoires exponentiels (ERGM)

La famille exponentielle permet à elle seule de définir un grand nombre de lois classiques telles que les lois de Bernoulli, binomiale, Poisson et gaussienne par exemple. L'idée des ERGM (Exponential Random Graph Models) est d'utiliser la caractère générique de l'écriture de la famille exponentielle pour modéliser un graphe. Ce modèle exprime la loi d'un graphe en fonction de statistiques exhaustives, telles que le nombre d'arêtes dans le graphe, le nombre de triangles pour décrire tout type de dépendance entre les arêtes. Leur loi est de la forme

P(Y = y) = 1 Λ θ exp C∈D θ C g C (y)
où y est une réalisation de Y et (i) chaque C est appelé une configuration, i.e. un sous-ensemble d'arêtes ou de dyades parmi un sous-ensemble de noeuds du graphe ;

(ii) g C (y) = (i,j)∈C y ij indique si la configuration C apparaît dans Y ; 13 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction

ÉTAT DE L'ART INTRODUCTION

(iii) les paramètres θ C sont associés à la configuration C. Si θ C = 0 alors les arêtes de la configuration sont indépendantes conditionnellement aux arêtes restantes du graphe. Une valeur non nulle de θ C induit de la dépendance entre les arêtes de C conditionnellement aux restes des arêtes. Enfin, quand une valeur positive de θ C encourage la configuration, une valeur négative la pénalise ;

(iv) Λ θ est une constante de normalisation dépendant de θ,

Λ θ = y exp C∈D θ C g C (y) . (1.2)
On remarquera que toute collection de paramètres {θ C , C ∈ D} ne conduit pas automatiquement à un modèle où la distribution jointe de Y est bien définie. Les conditions de validité sont décrites dans le théorème de Hammersley-Clifford (voir Besag, 1974). On notera certaines limites théoriques et pratiques importantes évoquées dans [START_REF] Chatterjee | Estimating and understanding exponential random graph models[END_REF] à propos des ERGM (ce travail concerne exclusivement le cas des graphes denses et s'inscrit dans le cadre de la théorie limite des graphes développée dans Lovász, 2012) comme la dégénérescence de modèles (beaucoup de réalisations sont en fait soit des graphes vides d'arêtes soit complets), ou l'inutilité des statistiques exhaustives prises en comptes dans le modèle [START_REF] Bhamidi | Mixing time of exponential random graphs[END_REF], ou encore le fait que beaucoup de réalisations des ERGM ne se distinguent pas d'un graphe simulé suivant le modèle d'Erdős-Rényi.

Modèles de graphes aléatoires à espaces d'états latents

Dans cette partie nous allons présenter en détail la classe des modèles à espaces d'états latents. Ces modèles ont pour principal objectif la modélisation de l'hétérogénéité des profils des individus dans un graphe. Ils permettent aussi de faire du clustering en regroupant les noeuds dans des groupes homogènes d'un point de vue de leurs connectivité. Nous noterons Z := {Z i , i ∈ N } les variables latentes associées aux noeuds. Dans la suite deux cas se dissocieront : quand l'espace latent est fini et quand il est continu. Quand il est fini, nous noterons Q le nombre d'états latents (ou groupes) et Q := 1, Q . Les variables latentes prennent alors la forme de Q-uplets composés de 1 et de 0. La valeur 1 à la coordonnée k signifie que le noeud est dans l'état latent k. Un noeud peut appartenir à 0, 1 ou plusieurs états latents suivant les modèles. Quand l'espace latent est continu, nous considérerons que les variables latentes appartiennent à R d . D'après la Figure 1.1, on a P(Y |Z) = ⊗ (i,j)∈D P(Y ij |Z i , Z j ). Ainsi on définit le Stochastic 14 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q INTRODUCTION 2. ÉTAT DE L'ART Block Model dans le cas général comme suit :

Stochastic Block Model (SBM)

(Z i ) i ∼ iid M(1, α) (Y ij ) i,j | (Z iq = 1), (Z j = 1) ∼ ind f (•; δ q )
où α = (α 1 , ..., α Q ) est le paramètre de mélange et δ = (δ q ) (q, )∈Q 2 la matrice des paramètres de connexion intra et inter-groupes. Cette matrice est aussi appelée matrice de connectivité dans le cas de données binaires. Le cas du SBM binaire (ou Bernoulli) a été étudié dans [START_REF] Snijders | Estimation and prediction for stochastic blockmodels for graphs with latent block structure[END_REF] et [START_REF] Daudin | A mixture model for random graphs[END_REF]. Il correspond au cas où la loi de Y ij sachant Z i , Z j est la loi de Bernoulli (voir Table 1.1) et

∀y ∈ {0, 1} et ∀π ∈ [0, 1], f (y; π) = π y (1 -π) 1-y .
C'est ce cas qui concentrera l'essentiel de notre attention dans la thèse.

Le cas des SBM pondérés a été largement développé dans [START_REF] Mariadassou | Uncovering latent structure in valued graphs: A variational approach[END_REF]. On y trouve par exemple les cas détaillés (modèle et estimation) des lois d'émissions Poisson, régression poissonnienne avec effets homogène (PRMH) ou inhomogène (PRMI), multinomiale, gaussienne (univariée, bivariée ou encore la régression linaire). Nous rappelons quelques lois classiques et les paramètres associés dans la Table 1.1. Nous citerons aussi les modèles Multiplex SBM [START_REF] Barbillon | Stochastic block models for multiplex networks: an application to networks of researchers[END_REF] et Multipartite SBM [START_REF] Bar-Hen | Block model for multipartite networks[END_REF] dans lesquels des liens de différentes natures entre des individus sont modélisés ; le Stochastic Topic Block Model [START_REF] Bouveyron | The stochastic topic block model for the clustering of vertices in networks with textual edges[END_REF][START_REF] Corneli | The dynamic stochastic topic block model for dynamic networks with textual edges[END_REF] dans lequel les arêtes représentent un échange de texte et enfin les modèles de SBM dynamiques [START_REF] Matias | Statistical clustering of temporal networks through a dynamic stochastic block model[END_REF][START_REF] Matias | A semiparametric extension of the stochastic block model for longitudinal networks[END_REF][START_REF] Ludkin | Dynamic stochastic block models: parameter estimation and detection of changes in community structure[END_REF] tous directement inspirés du SBM.

Modèle

Loi de probabilité Paramètre δ q Bernoulli B(π q ) π q Bernoulli avec covariables B(g(γ q + X T ij β))

(γ q , β)

Poisson P(λ q )
λ q PRMH P(λ q e β T X ij ) (λ q , β)

PRMI P(λ q e β T q X ij ) (λ q , β q ) Multinomial M(1; p q ) p q
Gaussien univarié N (µ q , σ 2 q )

(µ q , σ 2 q )

Gaussien bivarié N (µ q , Σ q ) (µ q , Σ q )

Régression linéaire simple

N (a q + bX ij , σ 2 ) (a q , b, σ 2 ) Régression linéaire N (β T q X ij , σ 2 q )
(β q , σ 2 q )

Table 1.1 -Modèles de SBM et lois de probabilités de : Y ij |Z iq = 1, Z j = 1, sauf pour le modèle gaussien bivarié dont la loi est celle de :

(Y ij , Y ji )|Z iq = 1, Z j = 1.
Le SBM est identifiable (voir [START_REF] Celisse | Consistency of maximum-likelihood and variational estimators in the stochastic block model[END_REF] sauf sur un ensemble de mesure de Lebesgue nulle et à permutation des labels près.

Degree Corrected Stochastic Block Model (DCSBM).

Une limite du SBM tel que défini précédemment concerne le fait que tous les noeuds d'un même groupe sont stochastiquement équivalents. En particulier E[Y ij |Z iq = 1, Z j = 1] = π q ne dépend que des classes des noeuds i et j et en espérance 2 noeuds d'un même groupe ont les mêmes degrés. Ainsi, dans le but de prendre en compte la propension de chaque noeud à se connecter aux autres et introduire plus d'hétérogénéité dans les degrés des noeuds du graphe, les auteurs de [START_REF] Karrer | Stochastic blockmodels and community structure in networks[END_REF] ont proposé le DCSBM dans lequel chaque noeud i a un paramètre γ i qui lui est propre et influe sur son degré de sorte que E[Y ij |Z iq = 1, Z j = 1] = γ i γ j π q . Ces paramètres de degré sont soumis à la contrainte i γ i 1 {Ziq=1} = 1, ∀q ∈ Q pour que le modèle soit identifiable. 15 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction

ÉTAT DE L'ART INTRODUCTION

Ce modèle a été étudié dans le cas d'une loi d'émission de Bernoulli ou Poisson. La loi de Poisson permet une mise à jour explicite des paramètres du modèle a contrario de la loi de Bernoulli. La loi de Poisson est donc utilisée pour approcher la loi de Bernoulli dans le cas d'une faible espérance correspondant à des données parcimonieuses. Une étude théorique de ces modèles est menée dans Zhao et al. (2012b).

Popularity-Adjusted Block Model (PABM). La « popularité » des noeuds est fortement liée à la notion de structure en communauté (ou affiliation). La popularité observée d'un noeud i au sein de la communauté (ou groupe) q est donnée par P iq = j, Zj =q Y ij et son espérance µ iq = E[P iq ]. On remarque que, dans le cas du SBM, µ iq = #{j, Z jq = 1}π ciq et que pour le DCSBM, µ iq = γ i π ciq où c i = Q q=1 q1 {Ziq=1} . Donc, dans le SBM, les noeuds d'une même communauté ont tous la même popularité et dans le DCSBM, la popularité des noeuds est modulée par leurs degrés. Ainsi, deux noeuds de forts degrés ont plus de chance de se connecter entre eux que deux noeuds de faibles degrés. Ce point de modélisation n'est pas très réaliste si par exemple les communautés représentent des clans opposés. En effet, dans ce cas les noeuds de forts degrés représentent les « chefs » opposés et il est peu probable qu'ils se connectent entre eux. Au contraire, les noeuds de faible degré représentant des gens dont la position sociale est moins extrême se connecteront plus volontiers. Le PABM [START_REF] Sengupta | A block model for node popularity in networks with community structure[END_REF]Chen, 2018 et Noroozi et al., 2019) prend en compte cet aspect de modélisation dans sa définition. Soit (λ iq ) i∈N ,q∈Q les paramètres de popularité associés à chaque noeud et vis-à-vis de chaque classe. Alors,

(Z i ) i ∼ iid M(1, α) (Y ij ) i,j | (Z iq = 1), (Z j = 1) ∼ ind B(λ i λ jq ).
Le modèle est identifiable sous la contrainte que Λ q = Λ q où Λ q := j, Zj =q λ j . De plus, sous le PABM, µ iq = λ iq Λ qci .

Overlapping Stochastic Block Model (OSBM). Une hypothèse du SBM est que chaque noeud appartient à un et un seul groupe. L'overlapping SBM [START_REF] Latouche | Overlapping stochastic block models with application to the french political blogosphere[END_REF] est une adaptation du SBM qui permet à chaque noeud d'appartenir à plusieurs groupes. Plus précisément, la loi de Z est donnée par

(Z i ) i ∼ iid ⊗ Q q=1 B(α q ).
On remarque que cette définition autorise toutes les composantes de Z i à être à zéro, ce qui permet de ne pas avoir de classe composée de noeuds de faible degré comme on peut souvent observer dans les problèmes de classification. En effet, ces « outliers » ne sont pas classés par OSBM, i.e. Z i = 0. On peut maintenant donner la loi conditionnelle de

Y ij sachant Z i et Z j : (Y ij ) i,j | (Z iq = 1), (Z j = 1) ∼ ind B(g(a q )) (1.3) où g(x) = (1 + e x ) -1 . De plus a ZiZj = Z T i W Z j + Z T i U + Z T j V + W * avec W ∈ M Q (R) et U, V ∈ M Q1 (R).
Le premier terme de (1.3) décrit les interactions entre les noeuds i et j alors que les second et troisième termes modélisent respectivement les capacités des noeuds i et j à se connecter aux autres noeuds. Le dernier terme est scalaire et représente un biais pour modéliser la parcimonie. Enfin, nous noterons que l'OSBM est identifiable [START_REF] Latouche | Overlapping stochastic block models with application to the french political blogosphere[END_REF], à échange des numéros des groupes près et sauf sur un ensemble de mesure de Lebesgue nulle.

Mixed Membership Stochastic Block Model (MMSBM).

Ce modèle est défini pour la première fois dans [START_REF] Airoldi | Mixed membership stochastic blockmodels[END_REF] avec pour but de prendre en compte dans chaque relation possible la variabilité propre à chaque noeud de se connecter aux autres. Dans le 16 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q INTRODUCTION 2. ÉTAT DE L'ART modèle OSBM, chaque noeud appartient à plusieurs classes et la probabilité que deux noeuds se connectent dépend de toutes les classes auxquels appartiennent les deux noeuds. Dans MMSBM, pour chaque dyade, la probabilité que les deux noeuds en question se connectent ne dépend que d'un seul des groupes auxquels appartient chaque noeud. Plus précisément,

(γ i ) i ∼ iid Dirichlet(α) (Z i→j ) i,j ∼ ind M(1, γ i ) (Z j→i ) i,j ∼ ind M(1, γ j ) (Y ij ) i,j | (Z i ), (Z j ) ∼ ind B(Z T i→j πZ j→i ).
Le paramètre de mélange est donc α ∈ Q, et la matrice Q × Q de connectivité est π, comme dans le SBM. Notons que l'identifiabilité de ce modèle n'a pas encore été démontrée.

Latent Block Model (LBM).

Le LBM est un modèle fréquemment utilisé pour faire du clustering de ligne et colonne de matrice a priori non carrée (ou bi-clustering), pas nécessairement une matrice d'adjacence au sens classique. Il a été introduit par [START_REF] Govaert | Clustering with block mixture models[END_REF]. Nous noterons M ∈ M nd (R) une telle matrice. Ce modèle est très lié au SBM qui correspond au cas où les groupes en colonnes sont identiques aux groupes en lignes. Il permet en particulier de modéliser un graphe bipartite. On définit Z et W les variables latentes respectivement associées aux lignes et aux colonnes, leur loi étant donnée par

(Z i ) i ∼ iid M(1, α) (W j ) j ∼ iid M(1, ρ). On a donc (M ij ) i,j | (Z i ), (W j ) ∼ ind f (•; Z T i πW j ).
Les paramètres α = (α 1 , ..., α Q1 ) et ρ = (ρ 1 , ..., ρ Q2 ) sont les paramètres de mélange associés respectivement aux lignes et aux colonnes et π ∈ M Q1Q2 (R) la matrice des paramètres de connexions inter et intra-groupes. Comme dans le cas du SBM la densité f peut par exemple appartenir à la famille exponentielle ou être une densité quelconque.

Le LBM est identifiable sous certaines conditions décrites dans [START_REF] Keribin | Estimation and selection for the latent block model on categorical data[END_REF] et à permutation près des labels en lignes et en colonnes. [START_REF] Handcock | Model-based clustering for social networks[END_REF] et inspiré du SBM, le LPCM a pour but de modéliser un réseau basé sur les distances sociales entre individus. Ceci implique que deux personnes ayant un profil social proche auront une plus grande probabilité de se connecter. Cette tendance à s'affilier à ses semblables s'appelle l'homophilie. On associe (comme dans le SBM) à chaque noeud du graphe une variable latente continue à valeurs dans R d correspondant aux caractéristiques sociales des noeuds :

Latent Position Cluster (LPCM). Introduit par

(Z i ) i ∼ iid q∈Q α q N (µ q , σ 2 q I d ).
(1.4)

Les noeuds sont donc d'abord placés dans des groupes à la façon du SBM avec probabilités α, puis leurs caractéristiques sociales -qui dépendent du groupe auxquels ils appartiennent -suivent une certaine loi normale multi-dimensionnelle. Les variables latentes sont ainsi indépendantes conditionnellement aux groupes auxquels chaque noeud appartient (correspondant à la première étape du processus). Finalement,

(Y ij ) i,j | (Z i ), (Z j ) ∼ ind B(g(β 0 c ij -β 1 Z i -Z j )) (1.5)
avec g = (1 + e x ) -1 et β 1 ≥ 0 pour que la probabilité soit faible quand la distance sociale est grande entre i et j et élevée quand ils sont proches socialement. Les (c ij ) (i,j)∈D jouent le rôle de covariables observées sur les arêtes.

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction

(Z i ) i ∼ iid f (Y ij ) i,j | (Z i ), (Z j ) ∼ ind B k Z i -Z j h n avec h n > 0 et k : R → [0,
(Z i ) i ∼ iid U(I) (Y ij ) i,j | (Z i ), (Z j ) ∼ ind f (Z i , Z j ).
(1.6)

Le modèle défini ainsi n'est pas identifiable. En effet, pour toute mesure σ préservant les fonctions définies et à image dans [0, 1], composer la fonction f avec σ donne exactement le même modèle qu'avec la fonction f . Pour pallier ce problème, on peut par exemple supposer que x → f (x, y)dy est croissante [START_REF] Latouche | Goodness of fit of logistic regression models for random graphs[END_REF]. Enfin, pour relier le SBM au graphon, on remarquera que l'équation (1.6) permet de définir le SBM comme un graphon. En effet, si la fonction f est constante sur chaque bloc rectangulaire de taille α q ×α et vaut π q , alors ceci correspond au SBM de paramètres α et π tel que défini précédemment. La variable latente Z i associée au noeud i est alors simplement le numéro du sous-intervalle dans lequel tombe la variable aléatoire Z i . (ii) La loi du sous-graphe induit pas l'ensemble de noeuds allant de 1 à n -1 est la même que la loi de G n-1

Garanties théoriques : théorème de représentation et loi limite

(iii) Pour tout 1 < k < n, les sous-graphes de G n induits par les ensembles de noeuds 1, i et i + 1, n sont indépendants. 18 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q INTRODUCTION 2. ÉTAT DE L'ART

Limite de graphes aléatoires

Le chapitre 11 du livre [START_REF] Lovász | Large Networks and Graph Limits[END_REF] est consacré à l'étude de la convergence des suites de graphes aléatoires denses et à l'identification de la loi (ou graphe) limite. Dans le cadre théorique qui y est développé il est montré que le graphon est le bon modèle limite pour des suites de graphes aléatoires tels que ceux satisfaisant les conditions du Théorème 1.

Inférence des modèles de graphe

Il existe globalement 2 grandes familles de méthode pour estimer les modèles décrits à la section 2.1 : (i) les méthodes particulaires de type MCMC (Markov Chain Monte Carlo) et (ii) les méthodes dérivées de l'algorithme EM (Expectation -Maximization, [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF]. Concernant les méthodes MCMC, les algorithmes de Metropolis-Hastings ou l'échantillonneur de Gibbs (voir [START_REF] Gilks | Markov Chain Monte Carlo in Practice[END_REF] sont les plus couramment utilisés. Leur but est d'estimer une loi a posteriori. Concernant les dérivées de l'EM, il existe de nombreux algorithmes tels que le CEM [START_REF] Celeux | A classification em algorithm for clustering and two stochastic versions[END_REF], le VEM [START_REF] Jordan | An introduction to variational methods for graphical models[END_REF][START_REF] Jaakkola | Advanced Mean Field Methods: Theory and Practice, chapter : Tutorial on variational approximation methods[END_REF], ou encore le VBEM [START_REF] Latouche | Variational bayesian inference and complexity control for stochastic block models[END_REF][START_REF] Brault | Estimation and model selection for the latent block model[END_REF], dont l'objectif est d'estimer le maximum de vraisemblance. Ces deux familles de méthodes sont parfois couplées, comme dans les algorithmes SEM [START_REF] Celeux | The sem algorithm : a probabilistic teacher algorithm derived from the em algorithm for the mixture problem[END_REF], MCEM [START_REF] Wei | A monte carlo implementation of the em algorithm and the poor man's data augmentation algorithms[END_REF], SAEM [START_REF] Delyon | Convergence of a stochastic approximation version of the em algorithm[END_REF] ou encore SEM-Gibbs et SEM+VEM (voir [START_REF] Brault | Estimation and model selection for the latent block model[END_REF]. Certains algorithmes de clustering comme les k-means [START_REF] Steinhaus | Sur la division des corps matériels en parties[END_REF] et MacQueen ( 1967)) et le Spectral Clustering [START_REF] Donath | Lower bounds for the partitioning of graphs[END_REF] et [START_REF] Rohe | Spectral clustering and the high-dimensional stochastic block model[END_REF] appliqué au SBM) sont fréquemment utilisés pour initialiser les algorithmes MCMC ou les dérivées de l'EM. L'algorithme des k-means divise les données en k groupes en minimisant la distance d'un point à la moyenne des points de son groupe. Le spectral clustering quant à lui applique une méthode de clustering comme les k-means aux vecteurs propres de la matrice Laplacienne associée à un graphe.

Concernant les modèles de graphes aléatoires à variables latentes, il est impossible de factoriser la loi de Z conditionnellement à Y (voir [START_REF] Lauritzen | Graphical models[END_REF]. Ces modèles ne peuvent donc pas être estimés avec l'algorithme EM. Les algorithmes MCMC ont d'abord été utilisés pour estimer ces modèles on citera [START_REF] Celeux | A classification em algorithm for clustering and two stochastic versions[END_REF], [START_REF] Nowicki | Estimation and prediction for stochastic blockstructures[END_REF] pour le SBM et [START_REF] Govaert | Clustering with block mixture models[END_REF] pour le LBM. Ils souffrent cependant d'un manque de rapidité computationnelle et ne s'appliquent qu'à des réseaux composés de plusieurs centaines de noeuds. Les algorithmes avec approximation variationnelle [START_REF] Jordan | An introduction to variational methods for graphical models[END_REF], [START_REF] Jaakkola | Advanced Mean Field Methods: Theory and Practice, chapter : Tutorial on variational approximation methods[END_REF]), basés sur l'EM et dans lesquels la loi de Z|Y est approchée ont ensuite été utilisés : voir [START_REF] Daudin | A mixture model for random graphs[END_REF] pour le SBM, [START_REF] Keribin | Estimation and selection for the latent block model on categorical data[END_REF] pour le LBM, [START_REF] Latouche | Variational bayesian inference and complexity control for stochastic block models[END_REF] pour le OSBM, [START_REF] Airoldi | Mixed membership stochastic blockmodels[END_REF] pour le MMSBM, [START_REF] Latouche | Variational Bayes model averaging for graphon functions and motif frequencies inference in W-graph models[END_REF] pour le graphon. Ceci est vrai aussi pour toutes les versions du SBM (dynamiques ou pas).

Pour l'estimation de très grands graphes, on citera le Largest Gaps Algorithm (LGA) [START_REF] Channarond | Clustering in a random graph : models with latent space[END_REF] pour le SBM et [START_REF] Brault | Estimation and model selection for the latent block model[END_REF] pour le LBM) qui estime les paramètres du modèle ainsi que la classification des noeuds à partir de la densité empirique des degrés des noeuds en évaluant les plus grands « gaps » dans la densité. Cependant cet algorithme s'applique principalement à de gros graphes composés d'au moins plusieurs milliers de noeuds.

L'estimation des modèles ERGM quant à eux se fait essentiellement sur la base d'algorithmes MCMC, voir [START_REF] Snijders | Markov chain monte carlo estimation of exponential random graph models[END_REF], Hunter and Handcock (2006a) et [START_REF] Kolaczyk | Statistical analysis of network data, methods and models[END_REF] pour plus de détails. On remarquera que la constante de normalisation Λ θ dépend de θ, c'est un problème pour utiliser des algorithmes MCMC comme le Metropolis-Hastings.

Dans la suite nous présentons en détail l'algorithme VEM avec comme fil rouge son application au SBM.

Variational EM. Cet algorithme a pour but de maximiser la vraisemblance des données observées. On décompose pour cela la log-vraisemblance des données log p θ (Y) de la façon 19 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction

ÉTAT DE L'ART INTRODUCTION

suivante : log p θ (Y) = log p θ (Y, Z) -log p θ (Z|Y).

(1.7)

Comme dans le raisonnement amenant à l'algorithme EM décrit dans [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF], on intègre (1.7) de chaque coté par rapport à une loi Q portant seulement sur Z. On obtient :

log p θ (Y) = E Q [log p θ (Y, Z)] + H(Q) + KL[Q p θ (Z|Y)] (1.8)
où H est l'entropie de Shannon (positive, maximale pour la loi uniforme) et KL(Q, P) la divergence de Kullback-Leibler (positive et nulle si et seulement si P loi = Q). Si Q était égale à p θ (Z|Y), alors maximiser (1.8) en θ reviendrait à utiliser exactement l'algorithme EM. Cependant, à cause de la structure de dépendance existante dans les modèles de graphes aléatoires à variables latentes entre les variables Y et Z (Figure 1.1), le calcul de l'étape E de l'algorithme EM, autrement dit le calcul de E Y|Z [log p θ (Y, Z)], se trouve compromis car la distribution de P(Z|Y) ne se factorise d'aucune façon. Finalement, en utilisant la décomposition (1.8) et la positivité de la divergence de Kullback-Leibler, on obtient : (voir Jaakkola, 2000[START_REF] Daudin | A mixture model for random graphs[END_REF] de prendre la classe des distributions factorisables, i.e. qui s'écrivent sous la forme

log p θ (Y) ≥ J θ,Q (Y) := E Q [log p θ (Y, Z)] + H(Q) (1.9) = log p θ (Y) -KL[Q p θ (Z|Y)]. (1.10) Améliorer la borne inférieure (1.10) équivaut à minimiser KL[Q p θ (Z|Y)] en Q dans l'en- semble des distributions, c'est-à-dire trouver Q = p θ (Z|Y)
Q(Z, τ ) = i∈N Q(Z i , τ i ).
(1.11)

Cette approximation est aussi appelée « champs moyens ». Dans le cadre précis du SBM on prendra (voir Lauritzen, 1996). Finalement en utilisant les équations (1.9) et (1.10), en remplaçant l'étape E de l'algorithme EM par une étape dite variationnelle E (ou VE) on obtient l'algorithme VEM qui consiste à itérer les 2 étapes suivantes jusqu'à convergence vers un maximum local :

Q(Z, τ ) = ⊗ i∈N M(Z i ; τ i ) avec τ i = (τ i1 , ..., τ iQ ) ∈ [0, 1] Q . On
• VE-step : avec l'estimation courante θ h of θ, calculer

τ h+1 = arg min τ ∈[0,1] Q KL[Q(Z; τ )||p θ h (Z|Y)].
• M-step : mettre à jour l'estimation courante θ h de θ

θ h+1 = arg max θ E Q(Z;τ h+1 ) [log(p θ (Y, Z))].
Cet algorithme produit une séquence {τ h , θ h , h ≥ 0} croissante pour la fonction J θ,Q . 20 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q INTRODUCTION 2. ÉTAT DE L'ART Sélection de modèle. L'estimation du nombre de groupes dans les modèles de mélanges avec variables latentes se fait généralement en utilisant le critère ICL (Integrated Completed Likelihood) proposé dans [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF]. Ce critère est un dérivé du critère BIC (Bayesian Information Criterion) de [START_REF] Schwarz | Estimating the dimension of a model[END_REF] et pénalise la vraisemblance complète classifiante des données.

Théorie des données manquantes

La problématique des données manquantes est centrale en statistique. Nous rappelons dans cette partie les rudiments de la théorie introduite dans [START_REF] Rubin | Inference and missing data[END_REF] adaptée aux données « réseaux ». Rappelons les notations suivantes : (ii) Missing At Random (MAR), si l'échantillonnage ne dépend que des valeurs des données observées mais pas des valeurs des données non-observées. En termes mathématiques,

Définition 1 (Matrice et vecteur d'échantillonnage). Soit R = (R ij ) (i,j)∈D , alors pour tout (i, j) ∈ D, R ij = 1 si la dyade (i,
D o := {(i, j) : R ij = 1}, D m := {(i, j) : R ij = 0}, Y o := {Y ij : (i, j) ∈ D o } et Y m := {Y ij : (i, j) ∈ D m }.
R |= Y m | Y o .
(iii) Not Missing At Random (NMAR), si l'échantillonnage dépend des valeurs de toutes les données. C'est le cas par exemple de la censure.

On remarque que (i) est un cas particulier de (ii). L'intérêt de la dichotomie entre le cas MAR et NMAR réside dans la proposition suivante.

Proposition 1. Si la loi de R satisfait (i) ou (ii), alors pour tout ψ ∈ Ψ tel que p θ,ψ (Y o , R) = 0 : arg max θ p θ,ψ (Y o , R) = arg max θ p θ (Y o ),
où p désigne la vraisemblance des données. Ceci revient à dire que la loi du processus d'échantillonnage ne perturbe pas l'inférence des paramètres du modèle génératif des données.

Échantillonner un réseau

Dans le domaine de l'échantillonnage de réseau, nous allons aborder deux problématiques partant de postulats différents. D'un coté la théorie des données manquantes développée par D. Rubin rappelée dans la section 2.4, qui s'inscrit dans un cadre « model-based » au sens où Y est une variable aléatoire. On s'intéresse dans ce cas explicitement à ce qui caractérise sa loi. D'un autre coté une approche dite « design-based » (voir [START_REF] Kolaczyk | Statistical analysis of network data, methods and models[END_REF] dans laquelle 21 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction C'est un estimateur sans biais de s. En effet :

E[ŝ µ ] = E i∈E t i µ i = E i∈N t i µ i 1 i∈E = i∈N t i µ i E [1 i∈E ] = s.
Pour la suite, nous noterons µ ij = P(i ∈ E, j ∈ E).

Nous allons maintenant donner deux exemples de stratégies d'échantillonnage ainsi que les probabilités associées aux estimateurs d'Horvitz-Thompson.

Sous-graphe induit. On sélectionne aléatoirement et uniformément un sous-ensemble de noeuds d'un graphe G = (N, A) et on observe tout le sous graphe induit par ces noeuds (i.e. le sous-graphe G = (N , A ) de G composé des noeuds échantillonnés (N de cardinal n ) ainsi que les arêtes de G entre eux (A )). On a donc

µ i = n n et µ ij = n (n -1) n(n -1) .
Sous-graphe incident. On sélectionne par un tirage sans remise un sous-ensemble d'arêtes A (de cardinal N A ) d'un graphe G et on observe tous les noeuds incident à ces arêtes : N . On a donc

µ i =      1 - N A -Di n N A n , si n ≤ N A -D i 1 , sinon et µ ij = n N A ou D i est le degré du noeud i.
On trouvera dans [START_REF] Kolaczyk | Statistical analysis of network data, methods and models[END_REF] d'autres exemples d'échantillonnages centrés sur les noeuds ainsi que les probabilités d'être échantillonné µ i et µ ij respectivement associées aux noeuds et aux arêtes.

22

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q INTRODUCTION 2. ÉTAT DE L'ART

P(R ij = 1|Y, ψ) = 1 -(1 -ψ) 2 , ∀i < j.
De plus, en notant 1 n le vecteur colonne de dimension n composé de 1, le vecteur V définie dans la définition 1 est donnée pas la relation logique :

V = [R1 n = (n -1)1 n ]. Réciproque- ment on peut retrouver R à partir de V avec la relation R = 1 n • V + V • 1 n -V • V,
P(R = r|Y, ψ) = P(V = v|Y, ψ) = ψ v T 1n (1 -ψ) n-v T 1n .
Cet échantillonnage est MCAR car la probabilité d'observer une dyade est complètement indépendante de sa valeur et de la valeur des autres dyades du graphe. Ce qui n'est pas le cas des deux stratégies d'échantillonnage que nous allons définir dans la suite (qui sont MAR mais pas MCAR). Dans ces exemples, les dyades seront observées par vagues successives et la probabilité d'observer une dyade dépend des dyades vues aux vagues précédentes.

One-wave link-tracing design. Pour cet échantillonnage, on sélectionne indépendamment des noeuds du graphe avec probabilité ψ. Ceci constitue un premier groupe de noeuds V 0 . Puis on observe toutes les dyades dont au moins l'une des extrémités inclue un des noeuds sélectionnés auparavant. Les voisins des noeuds du premier groupe (i.e. les noeuds partageant une arête avec les noeuds du premier groupe) constituent alors un deuxième groupe de noeuds que l'on notera V 1 . Finalement, on observe toutes les dyades incluant les noeuds du deuxième groupe.

On a donc la relation

V = V 0 + V 1 et V 1 est relié à V 0 et Y par la relation logique V 1 = [YV 0 (1 -V 0 ) > 0] où
est le produit de Hadamard de deux matrices. Par ailleurs les variables R et V sont reliées de la même façon que pour l'ego-centric design. En conséquence, la loi du one-wave link-tracing design est donnée par

P(R = r|Y, ψ) = v0, v0+[Yv0 (1-v0)>0]=v ψ v T 0 1n (1 -ψ) n-v T 0 1n .
Multi-wave link-tracing design. Cet échantillonnage consiste en l'application successive du one-wave link-tracing design. Nous noterons k le nombre de « vagues » successives d'échantillonnages. En utilisant les notations précédemment définies, pour m ∈ 1, k on définit l'ensemble des noeuds échantillonnés à la k-ième vague sachant toutes les vagues précédentes, par la relation logique

V m = [YV m-1 (1 - m-1 t=0 V t ) > 0]
. Finalement, la loi du k-waves link tracing design est donnée par

P(R = r|Y, ψ) = v0, v0+v1+•••+v k =k ψ v T 0 1n (1 -ψ) n-v T 0 1n .
23 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction 3. CONTRIBUTIONS DE LA THÈSE INTRODUCTION

Contributions de la thèse

Nous décrivons dans cette section ce que nous avons apporté d'original dans l'étude du SBM avec données manquantes.

Données manquantes dans le SBM

Le SBM est un modèle à variables latentes correspondant aux groupes auxquels appartiennent les noeuds. Cela signifie que sans la connaissance des groupes, la vraisemblance de Y n'a pas de forme explicite. La notion de « Missing At Random » telle que définie dans [START_REF] Rubin | Inference and missing data[END_REF] n'est donc pas applicable directement et nécessite une adaptation. Considérons la vraisemblance des données observées :

p θ,ψ (Y o , R) = p θ (Y o , Y m , Z)p ψ (R|Y o , Y m , Z)dZdY m .
(1.12)

Z Y R Z Y R (a) (b) Z Y R Z Y R (c) (d)

Stratégies d'échantillonnages

Nous avons vu dans la partie 1 des exemples de réseaux échantillonnés par des stratégies respectivement noeuds-centrées et dyades-centrées. Nous donnons maintenant des exemples de stratégies d'échantillonnages d'un réseau binaire inspirées des données décrites dans la section 1.

Stratégies dyades-centrées :

Random-dyad sampling : chaque dyade (i, j) ∈ D a la même probabilité P(R ij = 1) = ρ := ψ d'être observée, indépendamment les unes des autres.

• Double standard sampling : soit ψ := (ρ 1 , ρ 0 ) ∈ [0, 1] 2 . L'échantillonnage double standard (ou deux poids deux mesures) est une stratégie pour laquelle la probabilité 24 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q INTRODUCTION 3. CONTRIBUTIONS DE LA THÈSE d'observer une dyade dépend de sa valeur :

(R ij ) i,j | (Y ij ) ∼ ind B(ρ 1 1 {Yij =1} + ρ 0 1 {Yij =0} ).
Le cas ρ 1 = ρ 0 correspond au random-dyad sampling.

• Block-dyad sampling : soit ψ := (ρ q ) (q, )∈Q 2 ∈ M Q ([0, 1]). L'échantillonnage blockdyad (ou bloc-dyade) est une stratégie où la probabilité d'observer une dyade (i, j) dépend des classes des noeuds i et j :

(R ij ) i,j | (Z i ), (Z j ) ∼ ind B(ρ ZiZj ).

Stratégies noeuds-centrées :

Star and snowball samplings : l'échantillonnage star (ou en étoile) consiste à sélectionner aléatoirement un ensemble de noeuds, puis à observer les lignes correspondantes de la matrice Y. L'échantillonnage snowball (ou boule de neige) quant à lui se déroule en plusieurs « vagues » dont la première correspond à un échantillonnage en étoile. Les vagues successives consistent à observer les voisins des noeuds sélectionnés à la vague précédente. Ces échantillonnages correspondent au multi-waves link tracing design défini dans la sous-sous section 2.5 pour respectivement k = 0 et k ≥ 1.

• Star degree sampling : l'échantillonnage star degree (ou en étoile basé sur les degrés) est une stratégie en étoile dans laquelle les probabilités d'échantillonner les noeuds {ρ 1 , . . . , ρ n } dépendent des degrés des noeuds de la façon suivante :

(V i ) i | (D i ) ∼ ind B(logistic(a + bD i )) où ψ := (a, b) ∈ R 2 , D i = j Y ij .
• Block-node sampling : block-node sampling (en étoile basé sur les groupes) est aussi un échantillonnage en étoile dans lequel la probabilité d'observer un noeud dépend du groupe auquel il appartient. On a donc

(V i ) i | (Z i ) ∼ ind ρ Zi où ψ := (ρ 1 , ..., ρ Q ) ∈ [0, 1] Q .
Les échantillonnages marqués par une étoile sont MAR (et même MCAR) si les probabilités de sélectionner les noeuds ou les dyades ne dépendent pas de la valeur des dyades ou des groupes auxquels appartiennent les noeuds. Ceux marqués par un rond plein • sont quant à eux NMAR car dépendant de la valeur des dyades : double standard sampling, star degree sampling ; ou des groupes des noeuds : class-dyad sampling, class sampling.

Inférence MAR et NMAR dans le SBM

Inférence dans le cas MAR.

L'inférence du SBM dans le cas ou les données manquantes sont MAR (a fortiori MCAR) est identique à celle du SBM sans données manquantes, à ceci près que l'on ne considère plus toutes les entrées de la matrice Y mais seulement l'ensemble des dyades observées, noté dans la thèse D o . De plus, le nombre de noeuds observés est composé de l'ensemble des noeuds pour lesquels au moins une dyade est observée : N o := {i : ∃ j ∈ N s.t. R ij = 1}. 25 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction

CONTRIBUTIONS DE LA THÈSE INTRODUCTION

Inférence dans les cas NMAR.

Nous ne traiterons que du cas des réseaux binaires. La principale difficulté est de trouver une approximation de loi type « champs moyens » de la loi conditionnelle (Y o , R)|(Y m , Z). Dans Tabouy et al. (2019a), nous avons proposé l'approximation suivante :

Q(Z, Y o ) = i∈N M(Z i , τ i ) (i,j)∈D m B(Y ij ; ν ij ),

Garanties théoriques

Identifiabilité.

Nous avons montré que les paramètres du SBM ainsi que ceux des échantillonnages MCAR random dyad sampling et star sampling ainsi que de l'échantillonnage NMAR class sampling sont identifiables. Dans le cas des échantillonnages MCAR, comme il y a indépendance entre R et Y, l'identifiabilité se montre en deux temps. Dans un premier temps on montre que les paramètres ψ sont identifiables puis que θ l'est aussi. Proposition 2. Le paramètre d'échantillonnage ρ > 0 du random-dyad (resp. star) sampling est identifiable par rapport à la distribution de l'échantillonnage. Théorème 2. Soit n ≥ 2Q. Supposons que pour tout 1 ≤ q ≤ Q, ρ > 0 les coordonnées de πα sont deux à deux différentes. Alors, sous le random-dyad (resp. star) sampling, les paramètres du SBM sont identifiables par rapport à la distribution de la partie observée du SBM, à la permutation des labels près.

Le théorème suivant établit l'identifiabilité du SBM échantillonné suivant le class sampling. À la différence du cas MCAR, on ne peut pas prouver l'identifiabilité de ψ et de θ séparément, cela doit être fait conjointement à cause de la dépendance entre R et Y.

Théorème 3. Soit n ≥ 2Q et supposons que pour tout 1 ≤ q ≤ Q, ρ q > 0, α q > 0 les coordonnées de o = πα et de t = ( Q k=1 π 1k ρ k α k , . . . , Q k=1 π Qk ρ k α k ) sont deux à deux différentes.
Alors, sous le class sampling, les paramètres du SBM sont identifiables par rapport à la distribution du SBM, à la permutation des labels près. 26 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q INTRODUCTION 3. CONTRIBUTIONS DE LA THÈSE Les résultats d'identifiabilité associés aux échantillonnages double standard sampling et star degree sampling ne sont pas encore prouvés.

Consistance et normalité asymptotique.

Suivant les travaux de Celisse et al. (2012), [START_REF] Bickel | Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels[END_REF] et de Brault et al. (2017), nous avons montré que dans le cadre du SBM avec une loi d'émission appartenant à la famille exponentielle à un paramètre et sous les conditions d'échantillonnage du random dyad sampling, les estimateurs du maximum de vraisemblance et les estimateurs issus de l'approximation variationnelle du SBM sont consistants et asymptotiquement normaux. De plus, les variances asymptotiques de ces estimateurs sont identiques et explicites. Finalement, nous avons aussi montré que le SBM est identifiable pour toute loi d'émission des dyades appartenant à la famille exponentielle à un paramètre en présence de données manquantes générées par un échantillonnage random-dyad sampling. Ces résultats sont développés dans le chapitre 3.

Prise en compte de covariables

Dans la section 2.1, nous avons vu qu'il était possible d'intégrer la connaissance de covariables sur les noeuds ou sur les dyades dans la modélisation du SBM. Cette inclusion des covariables dans la loi d'émission des dyades conditionnellement aux variables latentes est souvent source d'ambiguïté. Il s'agit non pas de prendre en compte l'effet des covariables sur la structure du graphe, mais bien de l'enlever.

Le but de notre travail est d'inclure dans le SBM la connaissance de covariables, ainsi que de définir des stratégies d'échantillonnage de graphes dépendants des covariables. Nous montrons qu'un SBM binaire avec données manquantes NMAR peut être équivalent à un SBM prenant en compte les covariables avec données manquantes MCAR. Ce travail est motivé par le fait que l'on n'a pas toujours accès aux covariables pendant l'échantillonnage ou dans les données.

Données manquantes, SBM et covariables.

Nous avons étudié trois modèles de SBM avec covariables. Dans le premier, les variables latentes dépendent directement des covariables, dans le second la loi des dyades dépend directement des covariables, finalement le troisième est la combinaison du modèle 1 et 2. L'échantillonnage quant à lui dépend directement des covariables. Les liens de dépendance conditionnelle entre les variables aléatoires Y, Z et R associées à ces modèles sont décrits dans la figure 1

.3. X Z Y R X Z Y R X Z Y R
(modèle 1) (modèle 2) (modèle 3) Les modèles 1 et 2 sont définis ainsi : 27 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction

3. CONTRIBUTIONS DE LA THÈSE INTRODUCTION Modèle 1. Soit α • = (α •1 , ..., α •Q ) où α •q ∈ [0, 1] Q pour tout q ∈ Q, on définit α iq = e β T q Xi1 {q =Q} 1 + Q-1 k=1 β T k X i , ∀(i, q) ∈ N × Q, avec β q ∈ R N pour tout q ∈ 1, Q -1 and β Q = 0. De plus nous avons (Z i ) i | (X i ) ∼ iid M(1, α i ), (Y ij ) i,j | (Z i ), (Z j ) ∼ ind B(π ZiZj ), avec π ZiZj ∈ [0, 1] pour tout (i, j) ∈ N 2 .
Modèle 2.

(Z i ) i ∼ iid M(1, α), (Y ij ) i,j | (Z i ), (Z j ), (X i ), (X j ) ∼ ind B(g(γ zizj + β T φ(X i , X j ))), où γ q ∈ R, β ∈ R m , α ∈ [0, 1] Q , g(x) = (1 + e -x ) -1 et φ(•, •) : R N × R N → R m est une
fonction symétrique (i.e. φ(x, y) = φ(y, x)) mesurant la proximité entre deux vecteurs.

Le modèle suivant est la combinaison des deux modèles précédents.

Modèle 3. Soit α • = (α •1 , ..., α •Q ) ∈ [0, 1] Q alors α iq = e β t q Xi1 {q =Q} 1 + Q-1 k=1 β t k X i , ∀(i, q) ∈ N × Q, avec β q ∈ R N pour tout q ∈ 1, Q -1 and β Q = 0. Alors Z i | X i ∼ iid M(1, α i ), ∀i ∈ N , Y ij | Z i , Z j , X ij ∼ ind B(g(γ zizj + β t X ij )), ∀(i, j) ∈ N 2 ,
les paramètres et les notations sont identiques au modèle 2.

Stratégies d'échantillonnage.

Comme dans la section 3.2 pour le SBM, nous définissons deux stratégies d'échantillonnage (une dyade-centrée et une autre noeud-centrée), pour lesquelles la probabilité d'échantillonner un noeud ou une dyade dépend de la valeur des covariables.

• Dyad covariates sampling :

soient α ∈ R, κ ∈ R p et ψ(•, •) : R N × R N → R p . On a donc (R ij ) i,j | (X i ), (X j ) ∼ iid B(g(α + κ T ψ(X i , X j ))).
• Node covariates sampling : soient ν ∈ R et η ∈ R N . La probabilité d'observer toutes les dyades associées à un noeud est donnée par

(V i ) i | (X i ) ∼ iid B(g(ν + η T X i )).
Considérant les modèles 1 et 2 décrit ci-dessus et conditionnellement à X, les stratégies dyad covariates sampling et node covariates sampling sont MCAR. En particulier, d'après la proposition 1, l'inférence des paramètres des modèles 1 et 2 avec des données manquantes produites suivant ces stratégies d'échantillonnage se fait sur la partie observée des données. 28 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q INTRODUCTION 3. CONTRIBUTIONS DE LA THÈSE Cas d'équivalence de modèles.

La proposition suivante illustre l'article [START_REF] Molenberghs | Every missing not at random model has got a missing at random counterpart with equal fit[END_REF]. En effet, nous montrons qu'un modèle de SBM avec covariables et données manquantes MCAR peut dans certains cas être équivalent à un SBM sans covariables et données manquantes NMAR.

Proposition 3. Soit X ∈ M 1×n (Q), on modélise des covariables et des variables latentes Z de la façon suivante :

X i ∼ iid M(1, ν), ∀i ∈ N , P(Z ia = 1|X ia = 1) = δ et P(Z ib = 1|X ia = 1) = 1 -δ Q -1 , b = a, ∀i ∈ N .
De plus, on définit la probabilité d'observer une dyade comme suit :

p q = P(R ij = 1|X iq = 1, X j = 1).
On utilise enfin le modèle 1 pour la loi des dyades conditionnellement à X et Z.

Alors ce modèle est équivalent à un SBM binaire échantillonné avec une stratégie block-dyad sampling dont les paramètres d'échantillonnage seraient les suivants :

ρ q = P(R ij = 1|Z iq = 1, Z j = 1), = δ 2 p q ν q ν + 1-δ Q-1 2 a =q b = p ab ν a ν b + 2δ 1-δ Q-1 a =q b= p ab ν a ν b (δν q + 1-δ Q-1 c =q ν c )(δν + 1-δ Q-1 c = ν c )
.

Ce résultat théorique est illustré par des simulations dans le chapitre 4.

Package R : missSBM

Une partie des développements méthodologiques de la thèse ont été implémentés dans le package R missSBM (Tabouy et al., 2019c) 29 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Introduction 3. CONTRIBUTIONS DE LA THÈSE INTRODUCTION 30 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 Chapter 2

Variational inference of Stochastic Block Model from sampled data

This chapter has been published as an article in the Journal of the American Statistical Association (Tabouy et al., 2019a). This paper deals with non-observed dyads during the sampling of a network and consecutive issues in the inference of the Stochastic Block Model (SBM). We review sampling designs and recover Missing At Random (MAR) and Not Missing At Random (NMAR) conditions for the SBM. We introduce variants of the variational EM algorithm for inferring the SBM under various sampling designs (MAR and NMAR) all available as an R package on the CRAN (see chapter 5 for more details). Model selection criteria based on Integrated Classification Likelihood are derived for selecting both the number of blocks and the sampling design. We investigate the accuracy and the range of applicability of these algorithms with simulations. We explore two real-world networks from ethnology (seed circulation network) and biology (protein-protein interaction network), where the interpretations considerably depends on the sampling designs considered.

Chapter 2

1. INTRODUCTION

Introduction

Networks arise in many fields of application for providing an intuitive way to represent interactions between entities. In this paper, a network is composed by a fixed set of nodes, and an interaction between a pair of nodes (dyad) is called an edge. We consider undirected binary networks with no loop, which can be represented by symmetric adjacency matrices filled with zeros and ones.

Various statistical models exist for depicting the probability distribution of the adjacency matrix (see, e.g. [START_REF] Goldenberg | A survey of statistical network models[END_REF][START_REF] Snijders | Statistical models for social networks[END_REF], for a survey). A highly desirable feature is their capability to describe the heterogeneity of real-world networks. In this perspective, the family of models endowed with a latent structure (reviewed in Matias and Robin, 2014) offers a natural way to introduce heterogeneity. Within this family the Stochastic Block Model (in short SBM, see Frank and[START_REF] Frank | Cluster inference by using transitivity indices in empirical graphs[END_REF][START_REF] Holland | Stochastic blockmodels: First steps[END_REF] describes a broad variety of network topologies by positing a latent structure (or a clustering) on the nodes, then making the probability distribution of the adjacency matrix dependent on this latent structure. In order to estimate SBMs, Bayesian approaches were first developed (Snijders and[START_REF] Snijders | Estimation and prediction for stochastic blockmodels for graphs with latent block structure[END_REF]Nowicki and[START_REF] Nowicki | Estimation and prediction for stochastic blockstructures[END_REF] prior to variational approaches [START_REF] Daudin | A mixture model for random graphs[END_REF][START_REF] Latouche | Variational bayesian inference and complexity control for stochastic block models[END_REF]. On the theoretical side, [START_REF] Celisse | Consistency of maximum-likelihood and variational estimators in the stochastic block model[END_REF] study the conditions for identifiability and the consistency of the variational estimators; [START_REF] Bickel | Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels[END_REF] prove their asymptotic normality. Several generalizations are possible such as weighted or directed variants [START_REF] Mariadassou | Uncovering latent structure in valued graphs: A variational approach[END_REF], mixed-membership and overlapping SBM [START_REF] Airoldi | Mixed membership stochastic blockmodels[END_REF][START_REF] Latouche | Overlapping stochastic block models with application to the french political blogosphere[END_REF], degree-corrected SBM [START_REF] Karrer | Stochastic blockmodels and community structure in networks[END_REF], dynamic SBM [START_REF] Matias | Statistical clustering of temporal networks through a dynamic stochastic block model[END_REF], or multiplex SBM [START_REF] Barbillon | Stochastic block models for multiplex networks: an application to networks of researchers[END_REF].

This paper deals with inference in the SBM when the network is not fully observed. We consider cases where all the nodes are observed but information regarding the presence/absence of an edge is missing for some dyads. In other words the adjacency matrix contains missing values, a situation often met with real-world networks. For instance in social sciences, network data consists in interactions between individuals: the set of individuals is fixed, possibly known from a census. Information about the presence/absence of an edge is only available when at least one of the two individuals is available for an interview, otherwise it is missing. See [START_REF] Thompson | Model-based estimation with link-tracing sampling designs[END_REF], [START_REF] Thompson | Adaptive Sampling[END_REF], [START_REF] Kolaczyk | Statistical analysis of network data, methods and models[END_REF] and [START_REF] Handcock | statnet: Software tools for the representation, visualization, analysis and simulation of network data[END_REF] for a review of network sampling techniques. Even though some papers deal with SBM inference under missing data condition [START_REF] Aicher | Learning latent block structure in weighted networks[END_REF][START_REF] Vinayak | Graph clustering with missing data: Convex algorithms and analysis[END_REF], the sampling mechanism responsible for the missing values is overlooked in the inference, contrary to the approach developed in our paper.

Our contributions. A typology of sampling designs is introduced in Section 2.2. We adapt the theory developed in [START_REF] Rubin | Inference and missing data[END_REF] and [START_REF] Little | Statistical analysis with missing data[END_REF] to the SBM by splitting the sampling designs into the three usual classes of missing data: i) Missing Completely At Random (MCAR), where the sampling does not depend on the data, neither on the observed nor on the unobserved part of the network.

ii) Missing At Random (MAR), where the probability of being sampled is independent on the value of the missing data. For network data, the sampling does not depend on the presence/absence of an edge of an unobserved (or missing) dyad. MCAR is a particular case of MAR.

iii) Not Missing At Random (NMAR), where the sampling scheme is guided by unobserved dyads in some way.

Section 2.3 introduces several examples of sampling designs (MAR and NMAR) for which we derive conditions for identifiability of the SBM parameters. Estimation of the SBM in the MAR cases can be handled with the Variational EM (VEM) of [START_REF] Daudin | A mixture model for random graphs[END_REF] by conducting the inference only on the observed part of the 32 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 2. STATISTICAL FRAMEWORK network (Section 3.1). NMAR is more difficult to deal with as the sampling design must be taken into account in the inference. We introduce in Section 3.2 a general variational algorithm [START_REF] Jordan | An introduction to variational methods for graphical models[END_REF] to deal with NMAR cases when the sampling design relies on a probability distribution which is explicitly known 1 . Our variational approach is based on a double mean-field approximation applied to the latent distribution of the clustering and to the distribution of the missing dyads. We implement VEM algorithms that produce unbiased estimators for three natural NMAR sampling designs: a dyad-centered strategy, a node-centered strategy, and a block-centered strategy. We also derive an Integrated Classification Likelihood criterion (ICL, [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF] for selecting the number of blocks.

Although it is not possible to distinguish whether the sampling is MAR or NMAR [START_REF] Molenberghs | Every missing not at random model has got a missing at random counterpart with equal fit[END_REF], the ICL can also be used to select which sampling design is the best fit for the data.

In Section 4.2 we show the good performance of our VEM algorithms on simulations for both MAR (Section 4.1) and NMAR conditions. Finally we investigate two very different real-world networks with missing values, namely a Kenyan seed exchange network (Section 5.1), and a protein-protein interaction (PPI) network (Section 5.2).

Related works.

In the few papers dealing with missing data for networks, the sampling design is rarely discussed. Even if not explicitly stated they all assume MAR conditions. [START_REF] Aicher | Learning latent block structure in weighted networks[END_REF] propose a weighted SBM modeling simultaneously the presence/absence of an edge and its weight. Missing data are handled by dropping the corresponding terms in the likelihood and the inference is conducted by a variational algorithm. In [START_REF] Vincent | Estimating the size and distribution of networked populations with snowball sampling[END_REF] a Bayesian augmentation procedure is introduced to estimate simultaneously the size of the population and the clustering when the sampling design is a one-wave snowball. Apart from the SBM, the exponential random graph model has been studied in the MAR setting in [START_REF] Handcock | statnet: Software tools for the representation, visualization, analysis and simulation of network data[END_REF].

The matrix completion literature brings additional insights since SBM inference can be seen as a low-rank matrix estimation. [START_REF] Vinayak | Graph clustering with missing data: Convex algorithms and analysis[END_REF] introduce a convex program for the matrix completion problem where the underlying matrix has a simple affiliation structure defined via an SBM. The entries are sampled independently with the same probability, corresponding to a MAR case. In [START_REF] Davenport | 1-bit matrix completion[END_REF] the case of noisy 1-bit observations is studied and a likelihood-based strategy is developed with theoretical justifications ensuring good matrix completion. [START_REF] Chatterjee | Matrix estimation by universal singular value thresholding[END_REF] proves strong results for large matrices with noisy entries estimation, by means of a universal singular value thresholding.

Another related question is when the status of some dyads (absence/presence) is not clear in errorfully observed graph. Such uncertainties can be taken into account [START_REF] Priebe | Statistical inference on errorfully observed graphs[END_REF][START_REF] Balachandran | On the propagation of low-rate measurement error to subgraph counts in large networks[END_REF]. The latter reference studies the error propagation made by using estimators computed on observed sub-graphs, in order to estimate the number of existing edges in the real underlying graph.

Statistical framework 2.1 Stochastic Block Model

In an SBM, nodes from a set N {1, . . . , n} are distributed among a set Q {1, . . . , Q} of hidden blocks that model the latent structure of the graph. The blocks are described by the latent random vectors

Z i q = (Z i1 , . . . , Z iQ ) i∈N with multinomial distribution M(1, α = (α 1 , . . . , α Q ))
. The probability of an edge between any dyad in D N × N only depends on the blocks the two nodes belong to. Hence, the presence of an edge between i and j, indicated by the binary variable Y ij , is independent on the other edges conditionally on the 1 More complex sampling schemes -for instance adversarial strategies -are thus not handled 33 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 2. STATISTICAL FRAMEWORK latent blocks:

Y ij | Z iq = 1, Z j = 1 ∼ ind B(π q ), ∀(i, j) ∈ D, ∀(q, ) ∈ Q × Q,
where B stands for the Bernoulli distribution. In the following, π = (π q ) (q, )∈Q×Q is the

Q × Q matrix of connectivity probabilities, Y = (Y ij ) (i,j
)∈D is the n × n adjacency matrix of the random graph, Z = (Z iq ) i∈N ,q∈Q is the n × Q matrix of the latent blocks and θ = (α, π) are the unknown parameters. In the undirected binary case, Y ij = Y ji for all (i, j) ∈ D and Y ii = 0 for all i ∈ N . Similarly, π q = π q for all (q, ) ∈ Q × Q.

Sampled data in the SBM framework

The sampled data is an n×n matrix with entries in {0, 1, NA}. It corresponds to the adjacency matrix Y where unobserved dyads have been replaced by NA's. More formally, let R be the n × n sampling matrix recording the data sampled during this process, such that

R ij = 1 if Y ij is observed and 0 otherwise; also define D o = {(i, j) : R ij = 1}, D m = {(i, j) : R ij = 0}, Y o = {Y ij : (i, j) ∈ D o } and Y m = {Y ij : (i, j) ∈ D m }
to denote the sets of variables respectively associated with the observed and missing data. The number of nodes n is assumed to be known. The sampling design is the description of the stochastic process that generates R. It is assumed that the network exists before the sampling design acts upon it. Moreover, the sampling design is fully characterized by the conditional distribution p ψ (R|Y), the parameters of which are such that ψ and θ live in a product space Θ × Ψ.

Hence the joint probability density function of the observed data satisfies

p θ,ψ (Y, R) = p θ (Y o , Y m , Z)p ψ (R|Y o , Y m , Z)dY m dZ. (2.1)
Simplifications may occur in (2.1) depending on the sampling design, leading to the three usual types of missingness (MCAR, MAR and NMAR). This typology depends on the relations between the adjacency matrix Y, the latent structure Z and the sampling R, so that the missingness is characterized by four directed acyclic graphs displayed in Figure 2.1. On the basis of these DAGs, the sampling design is Proof.

Z Y R Z Y R (a) (b) Z Y R Z Y R (c) (d)
MCAR if R |= (Y m , Z, Y o ), MAR if R |= (Y m , Z) | Y o ,
To prove i), if R satisfies MAR conditions, then p ψ (R|Y o , Y m , Z) = p ψ (R|Y o ).
Moreover, θ and ψ lie in a product space so that (2.1)

factorizes into p θ,ψ (Y o , R) = p θ (Y o )p ψ (R|Y o
). This corresponds to the ignorability condition of [START_REF] Rubin | Inference and missing data[END_REF] and [START_REF] Handcock | statnet: Software tools for the representation, visualization, analysis and simulation of network data[END_REF]. The proof of ii) is postponed to the supplementary materials. 34 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 2. STATISTICAL FRAMEWORK

Sampling design examples

MAR examples

Definition 1 (Random-dyad sampling). Each dyad (i, j) ∈ D has the same probability P(R ij = 1) = ρ to be observed independently of the others.

This design is trivially MCAR because each dyad is sampled with the same probability ρ which does not depend on Y. Definition 2 (Star and snowball sampling). The star sampling consists in selecting uniformly a set of nodes, then observing corresponding rows of matrix Y. Snowball sampling is initialized by a star sampling which gives a first "wave" of nodes. The second wave is composed by the neighbors of the first. Successive waves can then be obtained. The final set of observed dyads corresponds to all dyads involving at least one of these nodes.

These two designs are node-centered and MAR. Indeed, selecting nodes independently in star sampling or in the first wave of snowball sampling corresponds to MCAR sampling. Successive waves are then MAR since they are built on the basis of the previously observed part of Y. Expressions of the corresponding distributions p ψ (R|Y o ) are given in [START_REF] Handcock | statnet: Software tools for the representation, visualization, analysis and simulation of network data[END_REF].

Identifiability of random-dyad and star sampling designs. Since random-dyad and star samplings are MCAR, the identifiability is assessed in two steps by proving the identifiability of, first, the sampling parameter ψ = ρ and second, the SBM parameters θ = (α, π) given ρ. Our proofs, postponed to the supplementary materials, follow [START_REF] Celisse | Consistency of maximum-likelihood and variational estimators in the stochastic block model[END_REF] who established the identifiability of the SBM without missing data.

Proposition 5. The sampling parameter ρ > 0 of random-dyad (resp. star) sampling is identifiable w.r.t. the sampling distribution.

Theorem 1. Let n ≥ 2Q and assume that for any 1 ≤ q ≤ Q, ρ > 0, α q > 0 and that the coordinates of πα are pairwise distinct. Then, under random-dyad (resp. star) sampling, SBM parameters are identifiable w.r.t. the distribution of the observed part of the SBM up to label switching.

NMAR examples

Definition 3 (Double standard sampling). Let ρ 1 , ρ 0 ∈ [0, 1]. Double standard sampling consists in observing dyads with probabilities

P(R ij = 1|Y ij = 1) = ρ 1 , P(R ij = 1|Y ij = 0) = ρ 0 . (2.2) Denote S o = (i,j)∈D o Y ij , So = (i,j)∈D o (1 -Y ij )
and similarly for S m , Sm . In this dyad-centered sampling design satisfying DAG (b), the log-likelihood is

log p ψ (R|Y ) = S o log ρ 1 + So log ρ 0 + S m log(1 -ρ 1 ) + Sm log(1 -ρ 0 ), with ψ = (ρ 0 , ρ 1 ).
(2.3) Definition 4 (Star sampling based on degrees -Star degree sampling). Star degree sampling consists in observing all dyads corresponding to nodes selected with probabilities {ρ 1 , . . . , ρ n } such that

ρ i = logistic(a+bD i ) for all i ∈ N where (a, b) ∈ R 2 , D i = j Y ij and logistic(x) = (1 + e -x ) -1 .
In this node-centered sampling design satisfying DAG (b), the log-likelihood is

log p ψ (R|Y ) = i∈N o log ρ i + i∈N m log(1 -ρ i ), with ψ = (a, b).
(2.4) 35 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2

VARIATIONAL INFERENCE

Definition 5 (Class sampling). Class sampling consists in observing all dyads corresponding to nodes selected with probabilities {ρ

1 , . . . , ρ Q } such that ρ q = P(i ∈ N o | Z iq = 1) for all (i, q) ∈ N × Q.
In this node-centered sampling design satisfying DAG (d), the log-likelihood is

log p ψ (R|Z) = i∈N o q∈Q Z iq log ρ q + i∈N m q∈Q Z iq log(1 -ρ q ), with ψ = (ρ 1 , . . . , ρ Q ).
(2.5)

Identifiability of class sampling. Theorem 2 establishes the identifiability of the SBM sampled under NMAR class sampling design (see the supplementary materials for the proof).

Note that the identifiability of the sampling parameters ψ = (ρ 1 , . . . , ρ Q ) and of the SBM parameters must be proved jointly because of the dependence between the network and the sampling. It is worth mentioning that both α q and ρ q are identifiable and not only their product. Although somewhat counter-intuitive, this fact is supported by the inference algorithm for class sampling in Section 3.3, which weights the recovery of the latent clusters by taking the unbalanced sampling into account.

Theorem 2. Let n ≥ 2Q and assume that for any 1 ≤ q ≤ Q, ρ q > 0, α q > 0, and that the coordinates of o = πα and t = (

Q k=1 π 1k ρ k α k , . . . , Q k=1 π Qk ρ k α k ) are pairwise distinct.
Then, under class sampling, SBM and class sampling parameters are identifiable w.r.t. the distributions of the SBM and the sampling up to label switching.

Variational Inference

Derivations of the practical variational algorithms considerably change depending on the missing data condition at play. We start by MAR to gently introduce the variational principle for SBM, then develop algorithms in a series of NMAR conditions

MAR inference

By Proposition 4 part (i), inference in the MAR case is conducted on Y o . The EM algorithm is unfeasible since it requires the evaluation of the conditional mean of the complete log-likelihood

E Z|Y o [log p θ (Y o , Z)]
which is intractable when Y comes from an SBM. The variational approach circumvents this limitation by maximizing a lower bound of the loglikelihood based on an approximation pτ of the true conditional distribution p θ (Z|Y o ),

log p θ (Y o ) ≥ J τ,θ (Y o ) log(p θ (Y o )) -KL[p τ (Z)||p θ (Z|Y o )], = E pτ [log(p θ (Y o , Z))] -E pτ [log pτ (Z)], (2.6)
where τ are some variational parameters and KL is the Kullback-Leibler divergence. The approximated distribution is chosen so that the integration over the latent variables simplifies by factorization. Recall from Section 2.1 that the latent vectors Z i q = (Z i1 , . . . , Z iQ ) i∈N are independent with a multinomial prior distribution. Thus, in order to factorize the likelihood in a convenient way, a natural variational counterpart to

p θ (Z|Y o ) is pτ (Z) = i∈N m(Z i q; τ i ),
where τ i = (τ i1 , . . . , τ iQ ), and m(•; τ i ) is the multinomial probability density function with parameters τ i . The VEM sketched in Algorithm 1 consists in alternatively maximizing J w.r.t. τ = {τ 1 , . . . , τ n } (the variational E-step) and w.r.t. θ (the M-step). The two maximization problems are solved straightforwardly following [START_REF] Daudin | A mixture model for random graphs[END_REF]:

1. The parameters θ = (α, π) maximizing J θ(Y o ) when τ is held fixed are αq = i∈N o τiq card (N o ) , πq = (i,j)∈D o τiq τj Y ij (i,j)∈D o τiq τj
. 36 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2

VARIATIONAL INFERENCE

2. The variational parameters τ maximizing J τ (Y o ) when θ is held fixed are obtained with the following fixed point relation:

τiq ∝ α q   (i,j)∈D o ∈Q b(Y ij ; π q ) τj   ,
where b(x, π) = π x (1 -π) 1-x the Bernoulli probability density function.

Algorithm 1: Variational EM for MAR inference in SBM Initialization: Set up τ (0) with some clustering algorithm repeat

θ (h+1) = arg max θ J Y o ; τ (h) , θ M-step τ (h+1) = arg max τ J Y o ; τ , θ (h+1) variational E-step until θ (h+1) -θ (h) < ε
Algorithm 1 generates a sequence {τ (h) , θ (h) ; h 0} with increasing J(Y o ; τ (h) , θ (h) ). Since there is no guarantee for convergence to the global maximum, we run the algorithm from several different initializations to finally retain the best solution.

Model selection of the number of blocks. The Integrated Classification Likelihood (ICL) criterion of [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF] is relevant for latent variable models where the likelihood -and thus BIC -is intractable. [START_REF] Daudin | A mixture model for random graphs[END_REF] derive a variational ICL for the SBM which we adapt to missing data conditions:

if θ = arg max log p θ (Y o , Z) then ICL(Q) = -2E pτ log p θ (Y o , Z; Q) + Q(Q + 1) 2 log card (D o ) + (Q -1) log card (N o ) .
Note that each dyad is only counted once since we work with symmetric networks.

NMAR inference: the general case

In contrast to the MAR case, conducting inference on the observed dyads only may bias the estimates in the NMAR case. In fact, all observed data (including the sampling matrix R in addition to Y o ) must be taken into account. The likelihood of the observed data is thus log p θ,ψ (Y o , R) and the corresponding completed likelihood has the following decomposition:

log p θ,ψ (Y o , R, Y m , Z) = log p ψ (R|Y o , Y m , Z) + log p θ (Y o , Y m , Z), (2.7)
where an explicit form of p ψ (R|Y o , Y m , Z) requires further specification of the sampling. The joint distribution p θ (Y o , Y m , Z) has a form similar to the MAR case. Now, the approximation is required both on latent blocks Z and missing dyads Y m to approximate

p θ (Z, Y m |Y o ).
We suggest a variational distribution where complete independence is forced on Z and Y m , using a multinomial (resp. Bernoulli ) distribution for Z (resp. for Y m ):

pτ,ν (Z, Y m ) = pτ (Z) pν (Y m ) = i∈N m(Z i• ; τ i ) (i,j)∈D m b(Y ij ; ν ij ), (2.8)
where τ and ν = {ν ij , (i, j) ∈ D m } are two sets of variational parameters respectively associated with Z and Y m . This leads to the following lower bound for log p θ,ψ (Y o , R):

J τ ,ν,θ,ψ (Y o , R) = E pτ,ν [log p θ,ψ (Y o , R, Y m , Z)] -E pτ,ν [log pτ,ν (Z, Y m )] .
37 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2

VARIATIONAL INFERENCE

By means of Decomposition (2.7) of the completed log-likelihood, variational approximation (2.8) and entropies of multinomial and Bernoulli distributions, one has

J τ ,ν,θ,ψ (Y o , R) = E pτ,ν [log p ψ (R|Y o , Y m , Z)] + (i,j)∈D o (q, )∈Q 2 τ iq τ j log b(Y ij , π q ) + (i,j)∈D m (q, )∈Q 2 τ iq τ j log b(ν ij , π q ) + i∈N q∈Q τ iq log(α q /τ iq ) - (i,j)∈D m ν ij log(ν ij ) + (1 -ν ij ) log(1 -ν ij ). (2.9) In (2.9), E pτ,ν [log p ψ (R|Y o , Y m , Z)
] can be integrated over the variational distribution pτ,ν (Z, Y m ), as expected. The practical computations depend on the sampling design.

The general VEM algorithm used to maximize (2.9) is sketched in its main lines in Algorithm 2. Both the E-step and the M-step split into two parts: the maximization must be performed on the SBM parameters θ and the sampling design parameters ψ respectively. The variational E-step is performed on the parameters τ of the latent block Z and on the parameters ν of the missing data Y m . Algorithm 2: Variational EM for NMAR inference in SBM Initialisation: set up τ (0) , ν (0) and ψ (0) repeat (h) , ν (h) , ψ (h) , θ M-step a)

θ (h+1) = arg max θ J Y o , R; τ
ψ (h+1) = arg max ψ J Y o , R; τ (h) , ν (h) , ψ, θ (h+1) M-step b) τ (h+1) = arg max τ J Y o , R; τ , ν (h) , ψ (h+1) , θ (h+1) VE-step a) ν (h+1) = arg max ν J Y o , R; τ (h+1) , ν, ψ (h+1) , θ (h+1) VE-step b) until θ (h+1) -θ (h) < ε
Interestingly, resolution of the two steps concerned with the optimization of the parameters related with the SBM -that is to say, θ and τ -can be stated almost independently of any further specification of the sampling design. Proposition 6. Consider the lower bound J τ ,ν,θ,ψ (Y o , R) given by (2.9).

1. The parameters θ = (α, π) maximizing (2.9) when all others are held fixed are

αq = 1 n i∈N τiq , πq = (i,j)∈D o τiq τj Y ij + (i,j)∈D m τiq τj νij (i,j)∈D τiq τj .
2. The optimal τ in (2.9) when all other parameters are held fixed verifies

τiq ∝ λ iq α q   (i,j)∈D o ∈Q b(Y ij ; π q ) τj     (i,j)∈D m ∈Q b(ν ij ; π q ) τj  
with λ iq a simple constant depending on the sampling design.

Proof. These results are simply obtained by differentiation of (2.9).

The two steps concerned with ψ and ν are specific to the sampling designs used to describe R. Further details are provided below for the designs presented in Section 2.3. 38 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2

VARIATIONAL INFERENCE

NMAR: specificities related to the choice of the sampling

In light of Figure 2.1, NMAR conditions specified by DAGs (b), (c) or (d) induce different simplifications for the conditional distribution of the sampling design R:

DAG (b) p ψ (R|Y o , Y m , Z) = p ψ (R|Y o , Y m ), DAG (c) p ψ (R|Y o , Y m , Z) = p ψ (R|Y o , Y m , Z), DAG (d) p ψ (R|Y o , Y m , Z) = p ψ (R|Z). This induces different evaluations of E pτ,ν [log p θ,ψ (Y o , R, Y m , Z)]
in the lower bound (2.9) for double standard sampling, star degree sampling and class sampling. We obtain below explicit formulas of ψ and ν by differentiation of the corresponding variational lower bounds. The computations are tedious but straightforward and thus eluded in the following.

Double-standard sampling. Let s m = (i,j)∈D m ν ij , sm = (i,j)∈D m (1 -ν ij )
be the variational counterparts of S m and Sm . From ( 2.3) we have

E p log p ψ (R|Y) = S o log ρ 1 + So log ρ 0 + s m log(1 -ρ 1 ) + sm log(1 -ρ 0 ).
Proposition 7 (double standard sampling).

1. The parameters ψ = (ρ 0 , ρ 1 ) maximizing (2.9) when all others are held fixed are

ρ0 = So So + sm , ρ1 = S o S o + s m .
(2.10)

2. The optimal ν in (2.9) when all other parameters are held fixed are

νij = logistic   log 1 -ρ 1 1 -ρ 0 + (q, )∈Q 2 τ iq τ j log π q 1 -π q   . Moreover, λ iq = 1 ∀(i, q) ∈ N × Q for optimization of τ in Proposition 6.b).
Class sampling. According to (2.5) we have

E p log p ψ (R|Y) = i∈N o q∈Q τ iq log(ρ q ) + i∈N m q∈Q τ iq log(1 -ρ q ).

Proposition 8 (class sampling).

1. The parameters ψ = (ρ 1 ...ρ Q ) maximizing (2.9) when all others are held fixed are

ρq = i∈N o τ iq i∈N τ iq . (2.11)
2. The optimal ν in (2.9) when all other parameters are held fixed verify

νij = logistic   (q, )∈Q 2 τ iq τ j log π q 1 -π q   . Moreover λ iq = ρ 1 {i∈N o } q (1 -ρ q )
1 {i∈N m } for optimization of τ in Proposition 6.b). 39 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2

VARIATIONAL INFERENCE

Star degree sampling. From Expression (2.4) of the likelihood, one has a+bDi) ) has no explicit form, an additional variational approximation is needed [START_REF] Jordan | An introduction to variational methods for graphical models[END_REF]. This technique was recently used in random graph framework [START_REF] Latouche | Goodness of fit of logistic regression models for random graphs[END_REF]. It relies on the following approximation of the logistic function:

E p log p ψ (R|Y) = - i∈N m a + b Di + i∈N E p -log(1 + e -(a+bDi) ) , where Di = E p [D i ] = i∈N m ν ij + i∈N o Y ij is the approximation of the degrees. Because E p -log(1 + e -(
g(x) ≥ g(ζ) + x -ζ 2 + h(ζ)(x 2 -ζ 2 ), h(ζ) = -1 2ζ logistic(ζ) - 1 2 (2.12) for all (x, ζ) ∈ R × R + .
This leads to a lower bound of the initial lower bound:

log p θ,ψ (Y o , R) ≥ J τ ,ν,θ,ψ (Y o , R) ≥ J τ ,ν,ζ,θ,ψ (Y o , R), (2.13) with ζ = (ζ i , i ∈ N ) such that ζ i > 0 is an additional set of variational parameters used to approximate -log(1 + e -x
). The second lower bound J τ ,ν,ζ,θ,ψ is derived from Equation (2.12) and given in the supplementary materials for completeness . At the end of the day, we have an additional set of variational parameters to optimize, and a corresponding additional step in Algorithm 2. Expression of all the parameters specific to star degree sampling by differentiating J τ ,ν,ζ,θ,ψ .

Proposition 9 (star degree sampling).

Let D 2 i = E p D i 2 and D- k = Dk -ν k . 1. The parameters ψ = (a, b) maximizing J τ ,ν,ζ,θ,ψ (Y o , R) when others are held fixed are b = 2 n 2 -card (N m ) i∈N (h(ζ i ) Di ) -1 2 i∈N Di -i∈N m Di × i∈N h(ζ i ) 2 i∈N (h(ζ i ) D 2 i ) × i∈N h(ζ i ) -2 i∈N h(ζ i ) Di 2 , â = - b i∈N h(ζ i ) Di + n 2 -card (N m ) i∈N h(ζ i ) . 2. The parameters ζ maximizing J τ ,ν,ζ,θ,ψ (Y o , R) when others are held fixed are ζi = a 2 + b 2 D 2 i + 2ab Di , ∀i ∈ N . 3. The optimal ν in J τ ,ν,ζ,θ,ψ (Y o , R
) when all other parameters are held fixed verify

νij = logistic (q, )∈Q 2 τ iq τ j log π q 1 -π q -b + 2h(ζ i ) ab + b 2 (1 + D-j i ) + 2h(ζ j ) ab + b 2 (1 + D-i j ) . (2.14) Moreover, λ iq = 1 ∀(i, q) ∈ N × Q for optimization of τ in Proposition 6.b).
Model selection. In NMAR cases, ICL can be useful not only to select the appropriate number of blocks but also for selecting the most appropriate sampling design when it is unknown. Contrary to the MAR case, ICL is no longer a straightforward generalization of [START_REF] Daudin | A mixture model for random graphs[END_REF]. Indeed, the complete likelihood and thus the penalization needs to take into account the sampling design. Let us consider a model with Q blocks and a sampling design with K parameters (i.e. the dimension of ψ). The ICL criterion is a Laplace approximation of the complete likelihood p(Y o , Y m , R, Z|Q, K) with p(θ, ψ|Q, K) the prior distributions on the parameters such that

p(Y o , Y m , R, Z|Q, K) = Θ×Ψ p θ,ψ (Y o , Y m , R, Z|Q, K)p(θ, ψ|Q, K)dθdψ.
40 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 4. SIMULATION STUDY Proposition 10. For a model with Q blocks, a sampling design with a vector of parameters

ψ ∈ R K and ( θ, ψ) = arg max (θ,ψ) log p θ,ψ (Y o , Y m , R, Z), then ICL(Q) = -2E pτ,ν ; θ, ψ log p θ, ψ (Y o , Y m , R, Z|Q, K) + pen ICL (Q), pen ICL =    K + Q(Q+1) 2 log n(n-1) 2 + (Q -1) log(n) for dyad-centered sampling Q(Q+1) 2 log n(n-1) 2 + (K + Q -1) log(n) for node-centered sampling
Note that an ICL criterion for MAR sampling designs can be constructed in the same fashion for the purpose of comparison with NMAR sampling designs.

Simulation study

In this section, we illustrate the relevance of our approaches on network data simulated under the SBM and sampled under MAR and NMAR conditions. The quality of the inference is assessed by computing the distance between the estimated and the true connectivity matrices π in terms of Frobenius norm. The quality of the clustering recovery is measured with the adjusted Rand index (ARI, [START_REF] Rand | Objective criteria for the evaluation of clustering methods[END_REF] between the true classification and the clustering obtained by maximum posterior probabilities for each τ i .

MAR condition

Algorithm 1 for MAR samplings is tested on affiliation networks with 3 blocks. The number of blocks is assumed to be known. For this topology the probability of connection within a block is η and is ten times stronger than the probability of connections between nodes from different blocks. We generate networks with n = 200 nodes and marginal probabilities of belonging to blocks α = (1/3, 1/3, 1/3). The sampling design is chosen as a random-dyad sampling with a varying ρ. The difficulty is controlled by two parameters: the sampling effort ρ and the overall connectivity in matrix π, defined by c = q α q α π q , which is directly related to the choice of η: the lower the η, the sparser the network and the harder the inference. The simulation is repeated 500 times for each configuration (c, ρ). Figure 2.2 displays the results in terms of estimation of π and of classification recovery, for varying connectivity c and sampling effort ρ. Our method achieves good performances even with a low sampling effort provided that the connectivity is not too low.

NMAR condition

Under NMAR conditions we conduct an extensive simulation study by considering various network topologies (namely affiliation, star and bipartite), the connectivity matrix of which are given in Figure 2.3. We use a common tuning parameter to control the connectivity of the networks in each topology: the lower the , the more contrasted the topology.

Among the three schemes developed in Section 2.3, we investigate thoroughly the double standard sampling, for which we exhibit a large panel of situations where the gap is large between the performances of the algorithms designed for MAR or NMAR cases. Other sampling designs are explored in the supplementary materials.

Simulated networks have n = 100 nodes, with varying in {0.05, 0.15, 0.25}. Prior probabilities α are chosen specifically for affiliation, star and bipartite topologies, respectively (1/3, 1/3, 1/3), (1/6, 1/3, 1/6, 1/3) and (1/4, 1/4, 1/4, 1/4). The exploration of the sampling parameters ψ = (ρ 0 , ρ 1 ) is done on a grid [0.1, 0.9] × [0.1, 0.9] discretized by steps of 0.1. Algorithm 2 is initialized with several random initializations and spectral clustering.

In Figure 2.4, the estimation error is represented as a function of the difference between the sampling design parameters (ρ 0 , ρ 1 ): the closer this difference to zero, the closer to the MAR case. As expected, Algorithm 1 designed for MAR only performs well when 41 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 4. SIMULATION STUDY The adjacency matrix Y is generated under random-dyad sampling strategy for various connectivity c = q α q α π q .

π -π F / π F Adjusted Rand Index
1 - 42 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 4. SIMULATION STUDY ρ 1 -ρ 0 ≈ 0. Algorithm 2 designed for NMAR double-standard sampling shows relatively flat curves which means that its performances are roughly constant no matter the sampling condition. Model selection. Simulations are also conducted to study the performances of ICL. We compare results for the different topologies described in Figure 2.3 for = 0.05. Rates of correct answers for selecting the number of blocks Q under a double standard sampling with different sampling rates are displayed in Table 2.1. The ARI is also provided. The ICL shows a satisfactory ability to select the true Q even if the selection task obviously needs a larger sampling effort than the estimation task. It is worth mentioning that a whole block may not be sampled, which leads the ICL to select a lower number of blocks. In such a case the ARI is a meaningful additional information to demonstrate that the clustering remains coherent with the true one. In Table 2.2, results concern the rates of correct selections of the sampling design when the two designs in competition are the random-dyad and the double standard samplings. 43 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 4. SIMULATION STUDY Each configuration is simulated 500 times. 44 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2

1 - 1 - (a) affiliation    1 - 1 - 0 0 1 - 0 0 0 1 - 1 - 0 0 1 - 0    (b) star    1 - 1 - 1 - 1 -    (c) bipartite
|ρ 0 -ρ 0 |/|ρ 0 | |ρ 1 -ρ 1 |/

IMPORTANCE OF ACCOUTING FOR MISSING VALUES IN REAL NETWORKS

As expected, the rate of correct answers increases with the sampling rate.

Importance of accouting for missing values in real networks

Seed exchange network in the region of Mount Kenya

In a context of subsistence farming, studies which investigate the relationships between crop genetic diversity and human cultural diversity patterns have shown that seed exchanges are embedded in farmers' social organization. Data on seed exchanges of sorghum in the region of Mount Kenya were collected and analyzed in [START_REF] Labeyrie | Seed exchange networks, ethnicity, and sorghum diversity[END_REF][START_REF] Labeyrie | Influence of ethnolinguistic diversity on the sorghum genetic patterns in subsistence farming systems in eastern kenya[END_REF]. The sampling is node-centered since the exchanges are documented by interviewing farmers who are asked to declare to whom they gave seeds and from whom they receive seeds. Since an interview is time consuming, the sampling is not exhaustive. A limited space area was defined where all the farmers were interviewed. The network is thus collected with missing dyads since information on the potential links between two farmers who were cited but not interviewed is missing. With the courtesy of Vanesse Labeyrie, we analyzed the Mount Kenya seed exchange network involving 568 farmers among which 155 were interviewed.

Although other farmers in this region might be connected to non-interviewed farmers, we focus on this closed network of 568 nodes. Since we only know that the sampling is node-centered, we fit SBM under the three node-centered sampling designs presented in Section 2.2 (star (MAR), class and star degree sampling). The ICL criterion is minimal for 10 blocks under the star degree sampling and for 11 blocks under the class degree sampling. The clusterings between the SBMs obtained with either class or star degree sampling remain close from each other (ARI: 0.6) and both unravel a strong community structure. The model selected by ICL for MAR sampling is composed by 11 blocks. The ARIs between MAR clustering and the two other clusterings are lower (around 0.4). Finally, note that interviewed and non-interviewed farmers are mixed up in the blocks of the three selected models. The ICL criteria computed for the three sampling designs are a slightly in favor of the MAR sampling.

On top of network data, categorical variables are available for discriminating the farmers such as the ntora 2 they belong to (10 main ntoras plus 1 grouping all the others) and the dialect they speak (4 dialects). In Figure 2.6, we compute ARIs between the ntoras (left panel), the dialects (right panel) and the clusterings obtained with the SBM under the three node-centered sampling designs for a varying number of blocks. Even though the ARIs remain low, the clusterings from class or star degree sampling seem to catch a non negligible fraction of the social organization, larger than the one caught by the clustering 2 The ntora is a small village or a group of neighborhoods 45 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2

IMPORTANCE OF ACCOUTING FOR MISSING VALUES IN REAL NETWORKS

from the MAR sampling. These two categorical variables, reflecting some aspects of the social organization, could partially explain the structure of the exchange network.

ARI q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.00 0.05 0.10 0.15 4 8 12 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.00 0.05 0.10 4 8 12 number of blocks sampling q q q Class Degree MAR

ER (ESR1) Protein-Protein Interaction network in breast cancer

Estrogen receptor 1 (ESR1) is a gene that encodes an estrogen receptor protein (ER), a central actor in breast cancer. Uncovering its relations with other proteins is essential for a better understanding of the disease. To this end, various bioinformatics tools are available to centralize knowledge about possible relations between proteins into networks known as Protein-Protein Interaction (PPI) networks. The platform string [START_REF] Szklarczyk | String v10: protein-protein interaction networks, integrated over the tree of life[END_REF] accessible via http://www.string-db.org is one of the most popular tools for this task.

Given a set of one (or several) initial protein(s) provided by the user, it is possible to recover a valued network between all proteins connected to the initial set. The value of an edge in this network corresponds to a score obtained by aggregating different types of knowledge (wet-lab experiments, textmining, co-expression data, etc. . . ), reflecting a level of confidence. Thus, it is possible for a given protein -we choose ER here -to recover the PPI network between all proteins involved. Our ambition is to rely on a SBM with missing data to finely analyze such networks: we rather describe a dyad as missing (thus not choosing between 0 or 1) if its level of confidence is too low. The PPI network in the neighborhood of ER is composed by 741 proteins connected by edges with values in (0, 1]. We remove ER from this set of proteins, as well as the zinc finger protein 44. Indeed, they were both connected to most of the other proteins and would thus only blur the underlying clustering structure. We denote ω ij the weight associated with dyad (i, j). By means of a tuning parameter γ reflecting the level of confidence, the adjacency matrix is defined as follows:

A γ = (A γ ) ij =    1 if ω ij > 1 -γ, NA if γ ≤ ω ij ≤ 1 -γ, 0 if ω ij < γ.
(2.15)

In order to analyze the ER-centered network, Algorithm 1 (random-dyad MAR sampling) and Algorithm 2 (double-standard NMAR sampling) were applied on A γ for γ varying in {.15, .25, .35}, hence taking the uncertainties on the missing dyads into account with various thresholds. The ICL criterion in Figure 2.7 systematically chooses the NMAR modeling against the MAR modeling, whatever the value of γ.

46

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 6. CONCLUSION We study the best MAR and NMAR models associated with γ = 0.35, which value exhibits a clearer choice of the ICL than for γ = {.15, .25} for both MAR and NMAR modelings. The two corresponding SBMs have 11 clusters for MAR sampling and 13 clusters for NMAR sampling. The ARI between the two clusterings is around 0.39: this is mainly due to a large block in the random-dyad MAR clustering which contains much more nodes than any of the blocks in the NMAR clustering. The latter dispatches many of these nodes in four blocks (see the supplementary materials for a more detailed exposition of results). To prove that this finest clustering of the nodes is more relevant from the biological point of view, we propose a validation based on external biological knowledge. To this end, we rely on the Gene Ontology (GO) annotation [START_REF] Ashburner | Gene ontology: tool for the unification of biology[END_REF] which provides a DAG of ontologies to which genes are annotated if the proteins encoded by these genes are involved in a known biological process. Here, we use GO to perform enrichment analysis (that is to say identifying classes of genes that are over-represented in a large set of genes, via a simple hypergeometric test) on genes corresponding to the proteins present in the large block for MAR, and the corresponding four blocks for NMAR. Interestingly, at a significance level of 1%, we find a single significant biological process for MAR modeling while 13 are found significant in the NMAR case. We check that it is not due to a simple threshold effect by looking at the ranks of the p-values of the 13 NMAR significant processes in the 100 first most significant terms found in the MAR model: only 5 of the NMAR processes are found, with high ranks (24, 33, 39, 56 and 77) far from the smallest MAR p-values.

Conclusion

This paper shows how to deal with missing data on dyads in the SBM. We study MAR and NMAR sampling designs motivated by network data and propose variational approaches to perform inference under this series of designs, accompanied with model selection criteria. Relevance of the method is illustrated on numerical experiments both on simulated and real-world networks. An R-package called missSBM is available on the CRAN (see chapter 5 for more details).

This work focuses on undirected binary networks. However, it can be adapted to other SBMs, in particular those developed in [START_REF] Mariadassou | Uncovering latent structure in valued graphs: A variational approach[END_REF] for (un)directed valued networks with a distribution of weights belonging to the exponential family. It could also be adapted to the degree-corrected SBM [START_REF] Karrer | Stochastic blockmodels and community structure in networks[END_REF], where the sampling design would depend on the degree correction parameters. This should lead to a design close to the star degree sampling. In future works, we plan to investigate the consistency of 47 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 6. CONCLUSION the variational estimators of SBM under missing data conditions, looking for similar results as the ones obtained in [START_REF] Bickel | Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels[END_REF] for fully observed networks. Another path of research is to consider missing data where we cannot distinguish between a missing dyad and the absence of an edge like in [START_REF] Priebe | Statistical inference on errorfully observed graphs[END_REF] and [START_REF] Balachandran | On the propagation of low-rate measurement error to subgraph counts in large networks[END_REF]. 48 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 7. SUPPLEMENTARY 7 Supplementary

Proof of Proposition 4.ii)

Recall the following result from graphical model theory (see Giraud, 2014, Formula 7.1).

Lemma. Let X, Y, W and Z be random variables, then for all h measurable,

X |= (Y, W ) | Z ⇒ X |= Y | (h(W ), Z).
Applying this lemma with h the identity, function,

we get R |= (Y m , Z) | Y o ⇒ R |= Z | (Y o , Y m ) , and then R |= Z | (Y o , Y m ) implies R |= Z | Y.

Identifiability

When the sampling is node-centered, we denote

V i = 1 if node i is observed and V i = 0 otherwise. Proof of Proposition 5. Let ρ, ρ > 0 be such that p ρ (R) = p ρ (R) (resp. p ρ (V) = p ρ (V)). Since R (resp.V) does not depend on Y, then P ρ (R ij = 1) = ρ = ρ = P ρ (R ij = 1) (resp. P ρ (V i = 1) = ρ = ρ = P ρ (V i = 1)).
Proof of Theorem 1. Let P o

[n] denote the probability distribution function of Y o . We show that there exists a unique (α, π)

corresponding to P o [n] . Define s q = P(Y ij R ij = 1|Z iq = 1) = ρ(π α) q (resp. s q = P(Y ij = 1|Z iq = 1) = (π α) q
for star sampling). Up to reordering, s 1 < s 2 < . . . < s Q are the coordinates of the vector s in the increasing order. Let S denote the Vandermonde matrix defined by S i,q = s i q , for 0 ≤ i < Q and 1 ≤ q ≤ Q. S is invertible since the coordinates of s are all different. For i ≥ 1, S i,q = P(Y 12 R 12 = 1, . . . , Y 1i+1 R 1i+1 = 1|Z 1q = 1) for random-dyad sampling (resp. S i,q = P(Y 12 = 1, . . . , Y 1i+1 = 1|Z 1q = 1) for star degree sampling). Let us also define

u i = 1≤k≤Q α k s i k (resp. u i = 1≤k≤Q ρα k s i k ), i = 0, . . . , 2Q -1 . For i ≥ 1, u i = P(Y 12 R 12 = 1, . . . , Y 1i+1 R 1i+1 = 1) (resp. u i = P(Y 12 = 1, . . . , Y 1i+1 = 1, V 1 = 1)
). Note that n ≥ 2Q is a necessary requirement on n since Y i,i = 0 by assumption. Hence, given P o

[n] and ρ, u 0 = 1 and u 1 , . . . , u 2Q-1 are known. Furthermore, set M the (Q + 1) × Q matrix given by M i,j = u i+j for every 0 ≤ i ≤ Q and 0 ≤ j < Q, and let M i denote the square matrix obtained by removing the row i from M . The coefficients of M Q , for 0 ≤ i, j < Q, are

M i,j = 1≤k≤Q s i k α k s j k (resp. M i,j = 1≤k≤Q ρs i k α k s j k ) , with 0 ≤ i, j < Q . Defining the diagonal matrix A = Diag(α), it comes that M Q = SAS t (resp. M Q = ρSAS t )
, where S and A are invertible, but unknown at this stage, and

ρ > 0. With D k = det(M k ) and the polynomial B(x) = Q k=0 (-1) k+Q D k x k , it yields D Q = det(M Q ) = 0 and the degree of B is equal to Q. Set C i = (1, s i , . . . , s Q i ) t
and let us notice that B(s i ) is the determinant of the square matrix produced when appending C i as the last column to M . The Q + 1 columns of this matrix are linearly dependent, since they are all linear combinations of the

Q vectors C 1 , C 2 , . . ., C Q . Hence B(s i ) = 0 and s i is a root of B for every 1 ≤ i ≤ Q. This proves that B = D Q Q i=1 (x -s i ).
Then, one knows that s = (s 1 , . . . , s Q ) (as the roots of B defined from M ) and S. It results that

A = S -1 M Q (S t ) -1 , which yields a unique (α 1 , . . . , α Q ) (resp. A = ρ -1 S -1 M Q (S t ) -1
). 49 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2

It only remains to determine π. For 0 ≤ i, j < Q, let us introduce U i,j the probability that the first row of Y o begins with i+1 occurrences of 1, and the second row of Y o ends up with j occurrences of 1

(i + 1 + j ≤ n -1 implies n ≥ 2Q). Then, U i,j = k,l S i,k α k π k,l α l S j,l (resp. U i,j = k,l ρ 2 S i,k α k π k,l α l S j,l ), for 0 ≤ i, j < Q, and the Q × Q matrix U = SAπAS t . The conclusion results from π = A -1 S -1 U (S t ) -1 A -1 (resp. π = ρ -2 A -1 S -1 U (S t ) -1 A -1 ).
Proof of Theorem 2. Let P [n] denote the probability distribution function of (Y o , R). We show that there exists a unique (α, π, ρ) corresponding to P [n] .

Identifiability of α. Up to reordering, let t 1 < t 2 < . . . < t Q denote the coordinates of the vector t in the increasing order, we have :

t q = P(Y ij = 1, V j = 1|Z iq = 1).
Let T denote the Vandermonde matrix defined by T i,q = t i q , for 0

≤ i < Q and 1 ≤ q ≤ Q. T is invertible since the coordinates of t are all different. For i ≥ 1, T i,q = P(Y 12 = 1, . . . , Y 1i+1 = 1, V 2 = 1, . . . , V i+1 = 1|Z 1q = 1). Let us also define v i = 1≤k≤Q α k t i k , i = 0, . . . , 2Q -1 . For i ≥ 1, v i = P(Y 12 = 1, . . . , Y 1i+1 = 1, V 2 = 1, . . . , V i+1 = 1
). Hence given P [n] , v 0 = 1 and v 1 , . . . , v 2Q-1 are known. Furthermore, set N the (Q + 1) × Q matrix given by N i,j = v i+j for 0 ≤ i ≤ j ≤ Q, and let N i denote the square matrix obtained by removing the row i from N . The coefficients of N Q are

N i,j = v i+j = 1≤k≤Q t i k α k t j k , with 0 ≤ i, j < Q . Defining the diagonal matrix A = Diag(α), it comes that N Q = T AT t , where T and A are invertible. With D k = det(N k ) and the polynomial B(x) = Q k=0 (-1) k+Q D k x k , it yields D Q = det(N Q ) = 0 and the degree of B is equal to Q.
Set C i = (1, t i , . . . , t Q i ) t and let us notice that B(t i ) is the determinant of the square matrix produced when appending C i as last column to N . The Q + 1 columns of this matrix are linearly dependent, since they are all linear combinations of the

Q vectors C 1 , C 2 , . . ., C Q . Hence B(t i ) = 0 and t i is a root of B for every 1 ≤ i ≤ Q. This proves that B = D Q Q i=1 (x -t i ).
Then, one knows that t = (t 1 , . . . , t Q ) (as the roots of B defined from N ) and T . It results that

A = T -1 N Q (T t ) -1 , which yields a unique (α 1 , . . . , α Q ).
Identifiability of ρ. Up to reordering, let o 1 < o 2 < . . . < o Q denote the coordinates of the vector o in the increasing order, then s q = P(Y ij = 1, V i = 1|Z iq = 1) = ρ q o q . Let O denote the Vandermonde matrix defined by O i,q = o i q , for 0 ≤ i < Q and 1 ≤ q ≤ Q. O is invertible since the coordinates of o are all different. For i ≥ 1, O i,q = P(Y 12 = 1, . . . , Y 1i+1 = 1|Z 1q = 1). Let us also define

u i = 1≤k≤Q ρ k α k o i k , i = 0, . . . , 2Q -1 . For i ≥ 1, u i = P(Y 12 = 1, . . . , Y 1i+1 = 1, V 1 = 1). Hence given P [n]
, u 0 = 1 and u 1 , . . . , u 2Q-1 are known. Furthermore, set M the (Q + 1) × Q matrix given by M i,j = u i+j for every 0 ≤ i ≤ Q and 0 ≤ j < Q, and let M i denote the square matrix obtained by removing the row i from M . The coefficients of M Q are

M i,j = u i+j = 1≤k≤Q o i k α k ρ k o j k , with 0 ≤ i, j < Q .
Defining the diagonal matrix B = Diag(ρ), it comes that M Q = OABO t , where O, B and A are invertible. Using the same algebraic argument than for the identifiability of α, it results that

B = A -1 O -1 M Q (O t ) -1
, which yields, because of the identifiability of α, a unique (ρ 1 , . . . , ρ Q ). 50 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 Identifiability of π. For 0 ≤ i, j < Q, let us introduce U i,j the probability that the first row of Y o begins with i + 1 occurrences of 1, and the second row of Y o ends up with j occurrences of 1.

U i,j = P {Y 12 = 1, . . . , Y 1i+2 = 1, V 2 = 1, . . . , V i+2 = 1} {Y 2n-j = 1, . . . , Y 2n = 1, V n-j = 1, . . . , V n = 1} , Then, U i,j = k,l T i,k α k π k,l ρ k α l T j,l , for 0 ≤ i, j < Q, and the Q × Q matrix U = T AπABT t . The conclusion results from π = A -1 T -1 U (T t ) -1 B -1 A -1 .
7.3 Derivation of second lower bound in star degree sampling

J τ ,ν,ζ,θ,ψ = C miss + n i=1 E p logistic(ζ i ) + (a + bD i ) -ζ i 2 + h(ζ i )((a + bD i ) 2 -ζ 2 i ) , = C miss + n i=1 logistic(ζ i ) + (a + b Di ) -ζ i 2 + h(ζ i )(a 2 + 2ab Di + b 2 Di -ζ 2 i ) ,
where

C miss = i∈N m a + b Di and Di = E p D i 2 = V p (D i ) + E p [D i ] 2 = j∈N m ν ij (1 - ν ij ) + j∈N m ν ij + j∈N o Y ij 2 .

Proof of Proposition 10

If the sampling design is node-centered (respectively dyad-centered), the first term in (2.7) consists in n Bernoulli random variables (respectively n(n-1)/2 Bernoulli random variables). The term log p ψ (R|Y o , Y m , Z, Q) can be derived using a BIC approximation:

log p ψ (R|Y, Z, Q) arg max ψ log p(R|Y, Z, ψ, Q) - 1 2 pen BIC ,
where pen BIC = K × log(n) if the sampling design is node-centered,

K × log n(n-1)
2 otherwise.

Simulations for star degree and class samplings

With node-centered samplings such as star-degree and class samplings, it is more difficult to find configurations different from MAR sampling designs: with dyad-centred doublestandard design, the sampling parameters were directly related to the probability of an edge conditionally on the observation of the corresponding dyad, which is no longer the case for the node-centered designs. The sampling designs being hardly different from a MAR design, the NMAR inference does not show much improvement compared to the MAR inference. Still, we exhibit some interesting situations where star degree and class samplings deserve an appropriate treatment, presented herein. We simulate networks with n = 100 nodes under an affiliation topology with intra-community probability (resp. inter community probability) equal to 0.5 (resp. 0.05) and α = (0.25, 0.5, 0.25). Sampling parameters are chosen such that ψ = (a, b) = (-3.6, 0.1) for star degree sampling, which makes nodes with highest degrees preferably selected. In class sampling, these parameters are set to ψ = (ρ 1 , ρ 2 , ρ 3 ) = (0.75, 0.5, 0.05), which makes nodes from the largest block and from a small block preferably selected while the other small block is under-sampled. In Figure 2.8, the estimation errors and the ARI are pictured for both cases. The sampling rates (i.e. rates of observed dyads over total number of dyads) lie in the intervals [0.558, 0.844] for 51 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 class sampling and [0.162, 0.622] for star degree sampling. These two intervals arise from the values of the parameter ψ explored for these two designs. We compare the performances of Algorithm 2 to an oracle (when inference is conducted via a classical VEM algorithm on a fully observed network) and with Algorithm 1. When facing NMAR condition, Algorithm 2 shows a slight improvement over Algorithm 1 even if it remains far from the oracle.

ππ F / π F ARI(Z, Ẑ) class q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.0 0.2 0.4 0.6

[0.558,0.844] q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.80 star degree q q q q q q q q q q q q q q q q q q q 0.0 0.5 1.0 1.5 (0.162,0.622] q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.2

Additional results for the ER Protein-Protein Interaction network

In this appendix, we provide more details on the NMAR clustering of the ER network discussed in Section 5.2. We represent the estimated connectivity matrix π in Figure 2.9b for NMAR, which exhibits a network-structure with 13 blocks (or sets of proteins) the sizes of which can be sketched from Figure 2.9a. In Figure 2.9c, we represent on the same matrix the network with the original missing data and the imputed missing dyads with the variational parameters ν. Interestingly, many of the imputed values are close to 1, which might help validating some relationships which were still uncertain in the biological literature. Figure 2.9a shows that the MAR clustering leads to a large cluster (block 3) which is mainly split into 4 blocks in the NMAR clustering (blocks 3, 4, 6 and 13). 52 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 MAR estimated clusters .9 -ER PPI network analysis with SBM under missing data conditions 53 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 2 54 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

Chapter 3

Consistency and Asymptotic Normality of Stochastic Block Models Estimators from Sampled Data

This chapter is a joint work with Mahendra Mariadassou and is available as a preprint on arXiv [START_REF] Mariadassou | Consistency and asymptotic normality of stochastic block models estimators from sampled data[END_REF]. Statistical analysis of network is an active research area and the literature counts a lot of papers concerned with network models and statistical analysis of networks. However, very few papers deal with missing data in network analysis and we reckon that, in practice, networks are often observed with missing values. In this paper we focus on the Stochastic Block Model with valued edges and consider a MCAR setting by assuming that every dyad (pair of nodes) is sampled identically and independently of the others with probability ρ > 0. We prove that maximum likelihood estimators and its variational approximations are consistent and asymptotically normal in the presence of missing data as soon as the sampling probability ρ satisfies ρ log(n)/n.

Introduction

For the last decade, statistical network analyses has a been a very active research topic and the statistical modeling of networks has found many applications in social sciences and biology for example [START_REF] Aicher | Learning latent block structure in weighted networks[END_REF], [START_REF] Barbillon | Stochastic block models for multiplex networks: an application to networks of researchers[END_REF], [START_REF] Mariadassou | Uncovering latent structure in valued graphs: A variational approach[END_REF], [START_REF] Wasserman | Social Network Analysis: Methods and Applications[END_REF] and [START_REF] Zachary | An information flow model for conflict and fission in small groups[END_REF].

Chapter 3

INTRODUCTION

Many random graphs models have been widely studied, either from a theoretical or an empirical point of view. The first model studied was Erdős-Rényi model [START_REF] Erdős | On random graphs[END_REF] which assumes that each pair of nodes (dyad) is connected independently to the others with the same probability. This model assumes homogeneity of all nodes across the network. In order to alleviate this constraint, many families of models have been introduced. Most are endowed with a latent structure (reviewed in Matias and Robin, 2014) to capture heterogeneity across nodes. Among those, the Stochastic Block Model (in short SBM, see Frank and[START_REF] Frank | Cluster inference by using transitivity indices in empirical graphs[END_REF][START_REF] Holland | Stochastic blockmodels: First steps[END_REF]) is one of the oldest and most studied as it is highly flexible and can capture a large variety of structures (affiliation, hub, bipartite and many other). In order to estimate this model, Bayesian approaches were first proposed (Snijders and[START_REF] Snijders | Estimation and prediction for stochastic blockmodels for graphs with latent block structure[END_REF]Nowicki and[START_REF] Nowicki | Estimation and prediction for stochastic blockstructures[END_REF] but have been superseded by variational methods [START_REF] Daudin | A mixture model for random graphs[END_REF][START_REF] Latouche | Variational bayesian inference and complexity control for stochastic block models[END_REF]. The former class of approaches are exact but lack the computational efficiency and scalability that the latter offers.

Theoretical guarantees concerning maximum likelihood estimators (in short MLE) and variational methods for the binary SBM estimation is not an easy task and have been widely studied. In [START_REF] Celisse | Consistency of maximum-likelihood and variational estimators in the stochastic block model[END_REF], consistency of MLE and variational estimates is proven but asymptotic normality requires that the estimators converges at rate at least n -1 , which is not proven in the paper, although some results were available for some particular cases (affiliation for example). [START_REF] Ambroise | New consistent and asymptotically normal parameter estimates for random-graph mixture models[END_REF] tackles the specific case of affiliation model with equal group proportion and proves the consistency and asymptotic normality of parameter estimates. [START_REF] Bickel | Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels[END_REF] extends those results to arbitrary binary SBM graphs and improves [START_REF] Celisse | Consistency of maximum-likelihood and variational estimators in the stochastic block model[END_REF] by removing the condition on the convergence rate. Following along the path of [START_REF] Bickel | Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels[END_REF], [START_REF] Brault | Consistency and Asymptotic Normality of Latent Blocks Model Estimators[END_REF] proved consistency and asymptotic normality of estimators (MLE and variational) to weighted Latent Block Models where the weights distribution belongs to a regular one-dimensional exponential family. In particular, considering non-bounded edge values invalidates several parts of the proofs for binary graphs and requires substantial adaptations and additional results, notably concentration inequalities for sums of unbounded, non-gaussian random variables.

Some results are also available for the related semi-parametric problem of assignment reconstruction. [START_REF] Mariadassou | Convergence of the groups posterior distribution in latent or stochastic block models[END_REF] show that the conditional distribution of the (latent) assignments converge to a degenerate distribution and [START_REF] Rohe | Spectral clustering and the high-dimensional stochastic block model[END_REF] prove that, when the data are generated according to a SBM model, spectral methods are consistent. Choi et al. (2012a) extend those results to settings where the density of the graph goes to 0 as Ω(log α (n)/n) (for α large enough) and/or the number of groups goes to +∞ as √ n. Finally, [START_REF] Wang | Likelihood-based model selection for stochastic block models[END_REF] and [START_REF] Hu | On consistency of model selection for stochastic block models[END_REF] also show that model selection for the number of groups is consistent for dense graphs, they suggest using a penalized likelihood criteria with penalty of the form k(k+1) 2 log(n) + λn log(k) where λ is a tuning parameter. In this paper we consider a simple setting with fixed number of groups and fixed density but weighted edges and missing values. In most network studies, there is a strong asymmetry between the presence of an edge and its absence: the lack of proof that an edge exists is taken as proof that the edge does not exist and edges with uncertain status are considered as non existent in the graph. This is the strategy adopted in most sparse asymptotic settings where the density of edges goes to 0 asymptotically [START_REF] Bickel | Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels[END_REF]. We adopt a different point of view where edges with uncertain status are considered as missing, rather than absent and explicitly accounted for their missing nature. We use the framework of [START_REF] Rubin | Inference and missing data[END_REF] and its application to network data, see [START_REF] Kolaczyk | Statistical analysis of network data, methods and models[END_REF] and [START_REF] Handcock | statnet: Software tools for the representation, visualization, analysis and simulation of network data[END_REF], for parameter inference in presence of missing values and more specifically its applications to SBM Tabouy et al. (2019a). We prove that, in the MCAR setting where each dyad is missing independently and with the same probability, the MLE and variational estimates are still consistent and asymptotically normal.

The article is organized as follows. We first present the model and missing data theory applied to our context with some examples of sampling designs. We then posit some defi-56 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 2. STATISTICAL FRAMEWORK nitions and discuss the assumptions required for our results in Section 2. In Section 3 we establish asymptotic normality for the complete-observed model (i.e. observed SBM where latent variables are known). Section 4 is the main result of this paper and states that the observed-likelihood behaves like the complete-observed likelihood (i.e. joint likelihood of the observed data and latent variables) close to its maximum. The proof is sketched in Section 5. Consequences for the MLE and variational estimator, as well as comparison to existing results, are in discussed in Section 6. Technical lemmas and details of the proofs are available in the appendices.

Statistical framework 2.1 Stochastic Block Model

In SBM, nodes from a set N {1, . . . , n} are distributed among a set Q {1, . . . , Q} of hidden blocks that model the latent structure of the graph. The block-memberships are encoded by (z i , i ∈ N ) where the z i are independent random variables with prior probabilities α = (α 1 , . . . , α Q ), such that P(z i = q) = α q , for all q ∈ Q. The value y ij of any dyad (i, j) in D = N × N , with i = j, only depends on the blocks i and j belong to. The variables (y ij )s are thus independent conditionally on the (z i )s:

y ij | z i = q, z j = ∼ ind ϕ(., π q ), ∀(i, j) ∈ D, i = j, ∀(q, ) ∈ Q × Q.
In the following, y = (y ij ) i,j∈D is the n × n adjacency matrix of the random graph, z = (z 1 , . . . , z n ) the n-vector of the latent blocks. With a slight abuse of notation, we associate to z i a binary vector (z i1 , . . . , z iQ ) such that z i = q ⇔ z iq = 1, z i = 0, for all = q. In this case z is a n × Q matrix.

We note the complete parameter set as θ = (α, π) ∈ Θ where Θ stands for the parameter space. When performing inference from data, we note θ = (α , π ) the true parameter set, i.e. the parameter values used to generate the data, and z the true (and usually unobserved) memberships of nodes. For any z, we also note:

• z +q = i z iq the size of block q for membership z

• z +q its counterpart for z .

Missing data for SBM

Regarding SBM inference, a missing value corresponds to a missing entry in the adjacency matrix y, typically denoted by NA's. We rely on the n × n sampling matrix r to record the missing state of each entry:

(r ij ) = 1 if y ij is observed, 0 otherwise. (2.1)
As a shortcut, we use y o = {y ij : r ij = 1} and y m = {y ij : r ij = 0} to respectively denote the observed and missing dyads. The sampling design is the description of the stochastic process that generates r. It is assumed that the network exists before the sampling design acts upon it, which is fully characterized by the conditional distribution p ψ (r|y), the parameters of which are such that ψ and θ live in a product space Θ × Ψ. In this paper we are going to focus on a specific type of missingness, called missing completely at random (MCAR) for which p ψ (r|y) = p ψ (r) and leave aside more complex forms of dependencies such as Missing at random (MAR) and Not missing at random (NMAR). 57 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

STATISTICAL FRAMEWORK

We then follow the framework of [START_REF] Rubin | Inference and missing data[END_REF] and Tabouy et al. (2019a) for missing data and define the joint probability density function as

p θ,ψ (y o , z, r) = p θ (y o , y m , z)p ψ (r|y o , y m , z)dy m .
(2.2)

Property 1. According to Equation (2.2), if the sampling design is MCAR, then maximizing p θ,ψ (y o , z, r) or p θ,ψ (y o , r) in θ is equivalent to maximizing p θ (y o ) in θ, this corresponds to the ignorability notion defined in [START_REF] Rubin | Inference and missing data[END_REF].

Sampling design examples

We present here some examples of sampling designs to illustrate differences between notions of MCAR, MAR and NMAR.

Definition 3 (Random dyad sampling). Each dyad (i, j) ∈ D has the same probability P(r ij = 1) = ρ of being observed, independently of the others. This design is MCAR.

Definition 4 (Random node sampling). The random node sampling consists in selecting independently with probability ρ a set of nodes and then observing the corresponding rows and columns of matrix y.

The major point in both examples is that the probability (ρ in random dyad sampling and 1 -(1 -ρ) 2 in the random node sampling) of observing a dyad does not depend on its value. In contrast, the following dyad-centered sampling design adapted to binary networks is NMAR since the probability to observe a dyad depends on its value: Definition 5 (Double standard sampling). Each dyad (i, j) ∈ D is observed, independently of other dyads, with a probability depending on its value: P(r ij = 1|y ij = 0) = ρ 0 and P(r ij = 1|y ij = 1) = ρ 1 .

For non-binary networks, specifying the sampling design is more involved and requires defining the sampling density for every possible value of y ij , e.g. (P(r ij = 1|y ij = k)) k∈N for Poisson-valued edges.

Observed-likelihoods

When the labels are known, the complete-observed log-likelihood is given by:

L co (z; θ) = log p(y o , z; θ) = i,q z iq log α q + i,j,q, i =j z iq z j r ij log ϕ(y ij ; π q ) (2.3)
But the labels are usually unobserved, and the observed log-likelihood is obtained by integration over all memberships:

L o (θ) = log p(y o ; θ) = log z∈Z p(y o , z; θ) .
(2.4)

Models and Assumptions

We focus here on parametric models where ϕ belongs to a regular one-dimension exponential family in canonical form: ϕ(y, π) = b(y) exp(πy -ψ(π)), (2.5) where π belongs to the space A, so that ϕ(•, π) is well defined for all π ∈ A. Classical properties of exponential families ensure that ψ is convex, infinitely differentiable on Å, that (ψ ) -1 is well defined on ψ ( Å). Furthermore, when y π ∼ ϕ(., π), E[y π ] = ψ (π) and 58 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 2. STATISTICAL FRAMEWORK

V[y π ] = ψ (π).
In the following, we assume that missing data are produced according to a random dyad sampling with parameter ρ > 0.

Moreover, we make the following assumptions on the parameter space :

A 1 : There exists a positive constant c, and a compact interval

C π such that ρ ∈ [c, 1 -c], Θ ⊂ [c, 1 -c] Q × C Q×Q π with C π ⊂ Å.
A 2 : The true parameter θ = (α , π ) lies in the interior of Θ.

A 3 : The map π → ϕ(•, π) is injective.
A 4 : The coordinates of π ψ (α ), where ψ is applied component-wise, are pairwise distinct.

The previous assumptions are standard. Assumption A 1 ensure that the group proportions and the sampling parameter are bounded away from 0 and 1 so that no group disappears when n goes to infinity. It also ensures that π is bounded away from the boundaries of the A. This is essential for the sub-exponential properties of Propositions 13 and 14. A 2 and A 3 are necessary for identifiability purposes: the model is trivially not identifiable if the map π → ϕ(., π) is not injective. A 4 states the identifiability of SBM parameters under random dyad sampling. Note that, combined with A 3 , it implies that all columns and all rows of π are distinct and therefore there are no two groups with identical connectivity profiles. In the following, we consider that Q, the number of classes (or groups) is known.

Identifiability

Since r is independent on y, the identifiability of SBM with emission law in the one-dimension exponential family under random dyad sampling can be stated in two steps. First the sampling parameter ρ and secondly the SBM parameters θ = (α , π ) given ρ.

Proposition 11. The sampling parameter ρ > 0 of random dyad sampling is identifiable w.r.t. the sampling distribution.

Proof. See Tabouy et al. (2019a). The proof does not depend on y being binary but also holds for y distributed as in Eq. (2.5).

Proposition 12. Let n ≥ 2Q and assume that for any 1 ≤ q ≤ Q, ρ > 0, π q > 0 and that the coordinates of α ψ (π ), where ψ is applied component-wise, are pairwise distinct. Then, under random dyad sampling, SBM parameters are identifiable w.r.t. the distribution of the observed part of the SBM up to label switching.

Proof. The proof is nearly identical to the one written in Tabouy et al. (2019a) and inspired by [START_REF] Celisse | Consistency of maximum-likelihood and variational estimators in the stochastic block model[END_REF] for the binary SBM under random dyad sampling. However, substituting E[y ij |z i = q] to s q in the proof ensures that α is identifiable. Finally, the fact that (ψ ) -1 is a one-to-one map ensures that π is identifiable.

Note that asymptotically, the assumption n ≥ 2Q is always satisfied since Q is fixed and n grows to infinity. 59 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 2. STATISTICAL FRAMEWORK

Sub-exponential variables

Remarque 2. Since we restricted π in a bounded subset of Å, the variance of y π is bounded away from 0 and +∞. We note

σ2 = sup π∈Cπ V(y π ) < +∞ and σ 2 = inf π∈Cπ V(y π ) > 0.
(2.6)

Similarly, since π belongs to a bounded subset of a open interval, there exists a constant κ > 0, such that [π -κ, π + κ] ⊂ Å uniformly over all π ∈ C π Proposition 13. With the previous notations, if π ∈ C π and y π ∼ ϕ(., π), then y π is sub-exponential with parameters (σ 2 , κ -1 ).

Proposition 14. Considering x = y π r ij + λr ij (we recall that r ij ∼ B(ρ)), with r ij independent of y π and λ ∈ R bounded. There are non-negative numbers ν and b such that x is sub-exponential with parameters (ν 2 , b -1 ).

Proof. These results derive directly from theorem 20 (statement 2.).

Symmetry

We now introduce the concepts of assignments and parameter symmetries, that must be accounted for when studying the asymptotic properties of the MLE. Complications stemming from symmetries are related to but no equivalent to the problem of label-switching in mixture models.

Definition 6 (permutation). Let s be a permutation on {1, . . . , Q}. If A is a matrix with Q columns and n rows, we define A s as the matrix obtained by permuting the columns of A according to s, i.e. for any row i and column q of A, A s iq = A is(q) . If C is a matrix with Q rows and Q columns, C s is defined similarly:

A s = A is(q) i,q C s = C s(q)s( ) q,
Definition 7 (equivalence). We define the following equivalence relationships:

• Two assignments z and z are equivalent, noted ∼, if they are equal up to label permutation, i.e. there exists a permutation s such that z = z s .

• Two parameters θ and θ are equivalent, noted ∼, if they are equal up to label permutation, i.e. there exists a permutation s such that (α s , π s ) = (α , π ).

• (θ, z) and (θ , z ) are equivalent, noted ∼, if they are equal up to label permutation on π and z, i.e. there exists a permutation s such that (π s , z s ) = (π , z ). This is label-switching.

Definition 8 (symmetry). We say that the parameter θ exhibits symmetry for the permutation s if (α s , π s ) = (α, π).

θ exhibits symmetry if it exhibits symmetry for any non trivial permutations s. Finally the set of permutations for which θ exhibits symmetry is noted Sym(θ).

Remarque 3. The set of parameters that exhibit symmetry is a manifold of null Lebesgue measure in Θ. The notion of symmetry allows us to deal with a notion of non-identifiability of the class labels that is subtler than and different from label switching. More precisely Label switching is when :

p(y o , z, θ) = p(y o , z s , θ s ), θ = θ s ∀s Symmetry is when : p(y o , z, θ) = p(y o , z s , θ), ∀s ∈ Sym(θ)
In particular, in label-switching, z and z s have the same likelihood but under equivalent yet different parameters θs. In contrast, in the presence of symmetry, multiple assignments can have exactly the same likelihood under θ. 60 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

COMPLETE-OBSERVED MODEL

The issue of symmetry forces us to use a notion of distance between assignment that is invariant to label permutation. Definition 9 (distance). We define the following distance, up to equivalence, between configurations z and z :

z -z 0,∼ = inf z ∼z z -z 0
where, for all matrix z, we use the Hamming norm • 0 defined by

z 0 = 1 2 i,q
1{z iq = 0}.

Definition 10 (Set of local assignments). We note S(z , r) the set of configurations that have a representative (for ∼) within relative radius r of z :

S(z , r) = {z : z -z 0,∼ ≤ rn}

Other definitions

We finally introduce a few useful notions that will be instrumental in the proofs. The first is "regular" assignments, for which each group has "enough" nodes:

Definition 11 (c-regular assignments). Let z ∈ Z. For any c > 0, we say that z is c-regular if min q z +q ≥ cn.

(2.7)

Class distinctness δ(π) captures the differences between groups: lower values of δ(π) means that at least two classes are very similar. δ(π) is intrinsically linked to the convergence rate of several estimates. Definition 12 (class distinctness). For θ = (α, π) ∈ Θ. We define:

δ(π) = min q,q max KL(π q , π q ) with KL(π, π ) = E π [log(ϕ(Y, π)/ϕ(Y, π ))] = ψ (π)(π -π ) + ψ(π ) -ψ(π)
the Kullback-Leibler divergence between ϕ(., π) and ϕ(., π ), when ϕ comes from an exponential family.

Remarque 4. Since all π have distinct rows and columns, δ(π) > 0.

Finally, the confusion matrix allows to compare groups between assignments: Definition 13 (confusion matrix). For given assignments z and z , we define the confusion matrix between z and z , noted IR(z), as follows:

IR(z) qq = 1 n i z iq z iq (2.8)
Definition 14. For more conciseness, we define S = (S q ) q = ψ (π q ) q (2.9)

Complete-observed Model

In the following we study the asymptotic properties of the complete-observed data model, i.e. when the true assignment z is known. 61 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 3. COMPLETE-OBSERVED MODEL Proposition 15. Under random dyad sampling, defining N i = i,j R ij and Ω 0,n = {∀i ∈ {1, ..., n}, N i 1} the set of nodes with at least one dyaddy observed. Then

P lim n→+∞ Ω 0,n = 1.
Proof. This proposition is a direct consequence of Borel-Cantelli's theorem. Details are available in appendix 10.

Remarque 5. This result shows that, with high probability, the network has no unobserved node. In the remainder, we work conditionally on Ω 0,n .

Let θ c = ( α, π) be the MLE of θ in the complete-observed data model. Simple manipulations of Equation ( 2.3) yield:

αq = α q (z) = z +q n y q (z) = i =j y ij r ij z iq z j i =j r ij z iq z j π q = π q (z) = (ψ ) -1 ( y q (z)) (3.1)
Since there are missing values in the adjacency matrix, we need the following technical lemma to prove asymptotic normality of π q 's in the complete data model.

Lemme 1.

U

n = 1 n(n -1) i =j r ij z iq z j P -----→ n→+∞ ρα q α l
Proof. The proof of this lemma is based on Hoeffding's decomposition for U-statistics and on the proof of Hoeffding's concentration inequality. Details are postponed to appendix 10.

Proposition 16. Let Σ α = Diag(α ) -α (α ) T . Σ α is semi-definite positive, of rank Q -1, and α is asymptotically normal:

√ n ( α (z ) -α ) D ----→ n→∞ N (0, Σ α ) (3.2)
Similarly, let V (π ) be the matrix defined by [V (π )] q = 1/ψ (π q ) and Σ π = ρ -1 Diag -1 (α )V (π ) Diag -1 (α ). Then the estimates πq (z ) are independent and asymptotically Gaussian with limit distribution:

n(n -1) ( π q (z ) -π q ) D ----→ n→∞ N (0, Σ π ,q ) for all q, (3.3)
Proof. The proof is postponed to appendix 10. The first part is a direct application of central limit theorem for i.i.d. variables and the second part relies on a variant of the central limit theorem for random sums of random variables.

Proposition 17 (Local asymptotic normality). Let L co be the complete likelihood function defined on Θ by L co (α, π) = log p (y o , z ; θ). For any s, t and u in a compact set, we have:

L co α + s √ n , π + u n(n -1) = L co (θ ) + s T Y α + Tr(u T Y π ) - 1 2 s T Σ α s + 1 2 Tr (u u) T Σ π + o P (1)
62 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

MAIN RESULT

where denote the Hadamard product of two matrices (element-wise product) and Σ α and Σ π are defined in Proposition 16. Y α is asymptotically Gaussian with zero mean and variance matrix Σ α . Y π is a random matrix with independent entries that are asymptotically Gaussian zero mean and variance Σ π .

Proof. This result is based on a Taylor expansion of L co in a neighborhood of (α , π ). Details are available in appendix 10.

Main Result

Our main result compares the observed likelihood ratio p(y o ; θ)/p(y o ; θ ) with the complete likelihood p(y o , z ; θ )/p(y o , z ; θ ) to show that they have the same argmax. To ease the comparison, we work only on the high probability set Ω 1 of c/2-regular configurations, i.e. that have Ω(n) nodes in each group as defined in Section 2, Proposition 18. Define Z 1 as the subset of Z made of c/2-regular assignments, with c defined in assumption H 1 . Note Ω 1 the event {z ∈ Z 1 }, then:

P θ Ω1 ≤ Q exp - nc 2 2 .
Proof. This proposition is a consequence of Hoeffding's inequality. See appendix 10 for more details.

We can now state our main result:

Theorem 15 (complete-observed). Assume that A 1 to A 4 with random-dyad sampling hold for the Stochastic Block Model of known order with n × n observations coming from an univariate exponential family and define # Sym(θ) as the set of permutation s for which θ = (α, π) exhibits symmetry. Then, for n tending to infinity, the observed likelihood ratio behaves like the complete likelihood ratio, up to a bounded multiplicative factor:

p(y o ; θ) p(y o ; θ ) = # Sym(θ) # Sym(θ ) max θ ∼θ p(y o , z ; θ ) p(y o , z ; θ ) (1 + o P (1)) + o P (1)
where the o P is uniform over all θ ∈ Θ.

The maximum over all θ that are equivalent to θ stems from the fact that because of label-switching, θ is only identifiable up to its ∼-equivalence class from the observed likelihood, whereas it is completely identifiable from the complete likelihood. The multiplicative factor arises from the fact that equivalent assignments have exactly the same complete likelihood and contribute equally to the observed likelihood.

Corollary 16. If Θ contains only parameters with no symmetry:

p(y o ; θ) p (y o ; θ ) = max θ ∼θ p(y o , z ; θ ) p(y o , z ; θ ) (1 + o P (1)) + o P (1)
where the o P is uniform over all Θ.

Proof Sketch

The proof of theorem relies on controlling deviations of the log-likelihood ratios from their expectations. We introduce a few notations for those quantities. 63 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 5. PROOF SKETCH

log-likelihood ratios

Definition 17. We define the conditional log-likelihood ratio LR and its expectation ELR as:

LR(θ, z) = log p(y o |z; θ) p(y o |z ; θ ) and ELR(θ, z) = E θ [ LR(θ, z)| z ] (5.1)
We also define the profile ratio Λ and its counterpart Λ as:

Λ(z) = max θ LR(θ, z) and Λ(z) = max θ ELR(θ, z).
(5.2)

Proposition 19. Conditionally on z , we have

ȳq (z) := E θ [ y q (z)|z ] = IR(z) T S IR(z) q α q (z) α (z) (5.3)
with ȳq (z) = 0 for z such that α q (z) = 0 or α (z) = 0.

Remarque 6. Note the absence of the random variable r in ȳq (z).

The following decomposition of p(y o ; θ) highlights the importance of F n (θ, z):

p(y o ; θ) = (z) p(y o , z; θ) = p(y o |z ; θ ) (z)
p(z; θ) exp(LR(θ, z)).

Since LR(θ, z) ≤ Λ(z), the profile ratio is useful to remove the dependency on θ and reduce the study to a series of problems depending only on z. The following propositions show when those quantities reach their maximum values and what the corresponding values are.

Proposition 20 (maximum of ELR and Λ in θ). The functions LR(θ, z) and ELR(θ, z) are maximum respectively in π for π(z) and π(z) defined by: π(z) q = (ψ ) -1 ( y q (z)) and π(z) q = (ψ ) -1 (ȳ q (z)) so that Λ(z) = LR( π(z), z) and Λ(z) = ELR( π(z), z).

Proposition 21 (Local upper bound for Λ). Conditionally upon Ω 1 , there exists a positive constant C such that for all z ∈ S(z , C):

Λ(z) ≤ -cρn 3δ(π ) 4 z -z 0,∼ (5.4)
Proposition 22 (maximum of ELR and Λ in (θ, z)). ELR can be written:

ELR(θ, z) = -ρn 2 q,q ,
IR(z) q,q IR(z) , KL(π q , π q ) ≤ 0.

(5.5)

Conditionally on the set Ω 1 of regular assignments and for n > 2/c, (i) ELR is maximized at (π , z ) and its equivalence class and ELR(π , z ) = 0.

(ii) Λ is maximized at z and its equivalence class and Λ(z ) = 0.

(iii) The maximum of Λ (and hence the maximum of ELR) is well separated.

Proofs of Propositions 19, 20, 22 and 21 are postponed to Appendix 11. 64 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 5. PROOF SKETCH

High level view of the proof

The proof proceeds with an examination of the asymptotic behavior of LR on three types of configurations that partition Z:

1. global control: for z such that Λ(z) = Ω(-n 2 ), Proposition 23 proves a large deviation behavior and shows that LR = -Ω P (n 2 ). In turn, those assignments contribute a o P of p(y o , z ; θ )) to the sum (Proposition 24).

2. local control: a small deviation result (Proposition 25) is needed to show that the combined contribution of assignments close to but not equivalent to z is also a o P of p(y o , z ; θ ) (Proposition 26).

3. equivalent assignments: Proposition 27 examines which of the remaining assignments, all equivalent to z , contribute to the sum.

These results are presented in next section 5.3 and their proofs postponed to Appendix 11. They are then put together in section 5.4 to prove our main result. The remainder of the section is devoted to the asymptotic of the ML and variational estimators as a consequence of the main result.

Different asymptotic behaviors

Λ(z) -Λ(z ) n z -z 0,∼ = o P (1) (5.7)
The next proposition uses Propositions 25 and 22 to show that the combined contribution to the observed likelihood of assignments close to z is also a o P of p(z , y o ; θ ): q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

VARIATIONAL AND MAXIMUM LIKELIHOOD ESTIMATES

Equivalent assignments

It remains to study the contribution of equivalent assignments.

Proposition 27 (contribution of equivalent assignments). For all θ ∈ Θ, we have

z∼z p(y o , z; θ) p(y o , z ; θ ) = # Sym(θ) max θ ∼θ p(y o , z ; θ ) p(y o , z ; θ ) (1 + o P (1))
where the o P is uniform in θ.

Proof of the main result

Proof. We work conditionally on Ω

Variational and Maximum Likelihood Estimates

This section is devoted to the asymptotic of the ML and variational estimators in the incomplete data model as a consequence of the main result 15. Note that, with high probability, ML and variational estimators have no symmetry since the set {θ : # Sym(θ) > 1} is a manifold of null Lebesque's measure in Θ. 66 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 7. DISCUSSION

ML estimator

The asymptotic behavior of the maximum likelihood estimator in the incomplete data model is a direct consequence of Theorem 15 and Proposition 17.

Corollary 18 (Asymptotic behavior of θ M LE ). Denote θ M LE the maximum likelihood estimator and use the notations of Proposition 16. There exist permutations s of {1, . . . , Q} such that

α (z ) -α s M LE = o P n -1/2 , π (z ) -π s M LE = o P n -1 .
Hence, the maximum likelihood estimator for the SBM under random-dyad sampling condition is consistent and asymptotically normal, with the same behavior as the maximum likelihood estimator in the complete data model. The proof is postponed to appendix 11.10.

Variational estimator

Due to the complex dependency structure of the observations, the maximum likelihood estimator of the SBM is not numerically tractable, even with the Expectation Maximization algorithm. In practice, a variational approximation is often used (see [START_REF] Daudin | A mixture model for random graphs[END_REF]: for any joint distribution Q ∈ Q on Z a lower bound of L(θ) is given by

J (Q, θ) = L(θ) -KL (Q, p (.; θ, y o )) = E Q [L co (z; θ)] + H (Q) .
where

H (Q) = -E Q [log(Q)]. Choosing Q to be the set of product distributions, such that for all z Q (z) = i,q Q (z iq = 1)
ziq allows us to obtain tractable expressions of J (Q, θ). The variational estimate θ var of θ is defined as

θ var ∈ argmax θ∈Θ max Q∈Q J (Q, θ) .
The following corollary states that θ var has the same asymptotic properties as θ M LE and θ M C , in particular is consistent and asymptotically normal.

Theorem 19 (Variational estimate). Under the assumptions of Theorem 15, there exist permutations s of {1, . . . , Q} such that

α (z ) -α s var = o P n -1/2 , π (z ) -π s var = o P n -1 .
The proof is very similar to the proof of Theorem 18 and postponed to appendix 11.10.

Discussion

Close examination of the different proofs, especially of Prop. 26, reveals that the quantities driving convergence of the estimates are ρnδ(π ), which must go to +∞ with n to ensure validity of Prop. 24, and ρnt n δ(π ), which must be larger than log(n) while t n → 0, to ensure validity of Prop. 26. Both conditions are met as soon as ρ log(n)/n, allowing for a 67 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 8. ACKNOWLEDGMENT large fraction of missing edges. Note that this limiting rate for missingness is the same as the one found for graph density in sparse settings to achieve consistency and local asymptotic normality of θ [START_REF] Bickel | Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels[END_REF].

In this paper, we focused on data sampled according to random dyad sampling. However, as described in section 2.3, there are many other ways to sample a network. In the case of node-centered sampling design, like random node sampling, the main difficulty to prove consistency and asymptotic normality is the dependency between the r ij variables. Indeed, in random node sampling, the variable r i0j0 depends on all r ij0 and r i0j (for all i, j ∈ N ). As a consequence, many results proved in this paper are not valid under random node sampling. NMAR sampling designs raises problem of their own: each design requires its own estimation procedure (Tabouy et al., 2019a) and therefore its own analysis. For example, even parameter estimation under the double standard sampling for binary networks mentioned in section 2.3 is still an unsolved problem: numerical experiments suggest that θ = (π, α) and ψ = (ρ 0 , ρ 1 ) are jointly identifiable but there is no formal proof. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 9. SUPPLEMENTARY 9 Supplementary 10 Technical results

Proof of proposition 15

Proof. Noticing that

N i ∼ Bin(n -1, ρ), then P(N i 1) = 1 -(1 -ρ) n-1 . As a con- sequence P(Ω 0,n ) i P(N i = 0) = n(1 -ρ) n-1 -→ n→+∞ 0
, and P(Ω 0,n ) -→ n→+∞ 1. Then P(lim sup(Ω 0,n )) = 0 by Borel-Cantelli's theorem (because n P(Ω 0,n ) converge), and as lim sup Ω 0,n = n 0 q n Ω 0,n = n 0 q n Ω 0,n = lim inf Ω 0,n , the result follow.

Proof of lemma 1

Proof. Noticing that E[r ij z iq z j ] = ρα q α l and defining q q, i,j = r ij z iq z j -ρα q α l . By Hoeffding decomposition for U-statistics (see [START_REF] Hoeffding | A class of statistics with asymptotically normal distribution[END_REF])

U n = 1 n(n -1) i =j (r ij z iq z j -ρα q α l ) = 1 n! σ∈Sn 1 n 2 n 2 i=1 q q, σ(i),σ(i+ n 2 ) , (10.1)
where for each permutation σ ∈ S,

n 2 i=1 q q, σ(i),σ(i+ n
2 ) is a sum of independent r.v. Then, for γ > 0 by Jensen's inequality and Hoeffding's lemma about bounded r.v.

E [exp(γU n )] ≤ 1 n! σ∈Sn E exp   γ n 2 n 2 i=1 q q, σ(i),σ(i+ n 2 )   ≤ exp γ 2 8 n 2 .
Finally, using the same proof than Hoeffding's inequality allows us to conclude.

Proof of proposition 16

Proof. Since α (z ) = (α 1 (z ) , . . . , αg (z )) is the sample mean of n i.i.d. multinomial random variables with parameters 1 and α , a simple application of the central limit theorem (CLT) gives:

Σ α ,qq = α q (1 -α q ) if q = q -α q α q if q = q which proves Equation (3.2) where Σ α is semi-definite positive of rank Q -1.
Similarly, ψ ( π q (z )) is the average of i =j r ij z iq z j i.i.d. random variables with mean ψ π q and variance ψ π q . i =j r ij z iq z j is itself random but thanks to lemma 1 :

1 n(n-1) i =j r ij z iq z j P -----→ n→+∞ ρα q α l
. Therefore, by Slutsky's lemma and the CLT for random sums of random variables [START_REF] Bibliography | A central limit theorem for random sums of random variables[END_REF], we have:

n(n -1)ρα q α (ψ ( π q (z )) -ψ (π q )) = n(n -1)ρα q α i =j y ij r ij z iq z j i =j r ij z iq z j -ψ (π q ) D -----→ n→+∞ N (0, ψ (π q ))
69 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

MAIN RESULTS

The differentiability of (ψ ) -1 and the delta method then gives:

n(n -1) ( π q (z ) -π q ) D -----→ n→+∞ N 0, 1 ρα q α ψ (π q )
and the independence results from the independence of π q (z ) and π q (z ) as soon as q = q or = , as they involve different sets of i.i.d. variables.

Proof of proposition 17

Proof. By Taylor expansion,

L co α + s √ n , π + u n(n -1) = L co (θ ) + 1 √ n s T ∇L coα (θ ) + 1 n(n -1) Tr u T ∇L coπ (θ ) + 1 n s T H α (θ ) s + 1 n(n -1) Tr (u u) T H π (θ ) + o P (1)
where ∇L coα (θ ) and ∇L coπ (θ ) denote the respective components of the gradient of L co evaluated at θ and H α and H π denote the conditional hessian of L co evaluated at θ . By inspection, H α /n and H π /(n(n -1)) converge in probability to constant matrices Σ α , Σ π and the random vectors ∇L coα (θ ) / √ n and ∇L coπ (θ ) / n(n -1) converge in distribution by central limit theorem.

Proof of proposition 18

Proof. In regular configurations, each group has Ω(n) members, where u n = Ω(n) if there exists two constant a, b > 0 such that for n enough large an ≤ u n ≤ bn. c/2-regular assignments, with c defined in Assumption H 1 , have high P θ -probability in the space of all assignments, uniformly over all θ ∈ Θ.

Each z +q is a sum of n i.i.d Bernoulli r.v. with parameter α q ≥ α min ≥ c. A simple Hoeffding bound shows that

P θ z +q ≤ n c 2 ≤ P θ z +q ≤ n α q 2 ≤ exp -2n α q 2 2 ≤ exp - nc 2 2
taking a union bound over Q values of q leads to Proposition 18.

Main Results

Proof of proposition 19)

Proof. First of all we will prove equation 5.3,

ȳq (z) = E θ i =j z iq z j r ij y ij i =j z iq z j r ij z = E θ E θ i =j z iq z j r ij y ij i =j z iq z j r ij R, z z = E θ i =j z iq z j r ij S ZiZj i =j z iq z j r ij z ,

70

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

MAIN RESULTS

where Z i = q ⇔ z iq = 1. Noticing that the (i, j) for which z iq z j = 0 does not contributes in any of the two terms of the ratio. The calculus of this expectation is then equivalent to calculate an expectation of the general form

E θ n i=1 aiRi n i=1
Ri , (a i ) i∈{1,..,n} ∈ R n and

T i iid ∼ B(ρ). Lemme 2. E θ n i=1 a i T i n i=1 T i = n i=1 a i n . Proof. Define N = n i=1 T i and noticing that E[T i |N = k] = k n . Conditionally to N ≥ 1 E n i=1 a i T i n i=1 T i = E E n i=1 a i T i N N = n i=1 a i n .
Now, applying lemma 2 with N o q (z) = i =j z iq z j r ij leads to

E θ i =j z iq z j r ij S ZiZj i =j z iq z j r ij z , N o q (z) ≥ 1 = IR(z) T S IR(z) q α q (z) α (z) 1 N o q (z)≥1 .
Finally, E θ [ y q (z)|z , N o q (z) = 0] can be arbitrarily defined at the same value than E θ [ y q (z)|z , N o q (z) ≥ 1] which conclude the proof.

Proof of proposition 20

Proof. Defining ν(y, π) = yπ -ψ(π). For y fixed, ν(y, π) is maximized at π = (ψ ) -1 (y). Manipulations yield

LR(θ, z) = log p(y o ; z, θ) -log p(y o ; z , θ ) = q N o q (z)ν( y q (z), π q ) - q N o q (z )ν( y q (z ), π q )
which is maximized at π q = (ψ ) -1 ( y q (z)). Similarly with

N q (z) = i =j z iq z j , ELR(θ, z) = E θ [log p(y o ; z, θ) -log p(y o ; z , θ )|z ] = ρ q N q (z)ν(ȳ q (z), π q ) - q N q (z )ν(ψ (π q ), π q )
is maximized at π q = (ψ ) -1 (ȳ q (z)). 71 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 11. MAIN RESULTS

Proof of Proposition 22 (maximum of ELR and Λ)

Proof. We condition on z and prove Equation (5.5):

ELR(θ, z) = E θ log p(y o ; z, θ) p(y o ; z , θ ) z = i j q,q , E θ y ij (π q -π q ) -(ψ(π q ) -ψ(π q )) ρz iq z iq z j z j = n 2 ρ q,q ,
IR(z) q,q IR(z) , ψ (π q )(π q -π q ) + ψ(π q ) -ψ(π q ) = -n 2 ρ q,q , IR(z) q,q IR(z) , KL(π q , π q )

If z is regular, and for n > 2/c, all the rows of IR(z) have at least one positive element and we can apply Lemma 3.2 of [START_REF] Bickel | Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels[END_REF] to characterize the maximum for ELR.

The maximality of Λ(z ) results from the fact that Λ(z) = ELR( π(z), z) where π(z) is a particular value of π, Λ is immediately maximum at z ∼ z , and for those, we have π(z) ∼ π .

The separation and local behavior of G around z is a direct consequence of the proposition 21.

Proof of Proposition 21 (Local upper bound for Λ)

Proof. We work conditionally on z . The principle of the proof relies on the extension of Λ to a continuous subspace of M Q ([0, 1]), in which the confusion matrix is naturally embedded. The regularity assumption allows us to work on a subspace that is bounded away from the borders of M Q ([0, 1]). The proof then proceeds by computing the gradient of Λ at and around its argmax and using those gradients to control the local behavior of Λ around its argmax. The local behavior allows us in turn to show that Λ is well-separated.

Note that Λ only depends on z through IR(z). We can therefore extend it to matrix U ∈ U c where U is the subset of matrices M Q ([0, 1]) with each row sum higher than c/2.

Λ(U ) = -ρn 2 q,q , U qq U KL π q , πq where πq = πq (U ) = (ψ ) -1 U T S U q [U T 1U ] q
and 1 is the Q × Q matrix filled with 1. Confusion matrix IR(z) satisfy IR(z)1I = α(z ), with 1I = (1, . . . , 1) T a vector only containing 1 values, and are obviously in U c as soon as z is c/2 regular. The maps f q,q , , : (U ) → KL(π q , πq (U )) are twice differentiable with second derivatives bounded over U c and therefore so is Λ(U ). Tedious but straightforward computations show that the derivative of Λ at D α := Diag(α(z )) is:

A qq (z ) := -1 n 2 ∂ Λ ∂U qq (D α ) = 2ρ α (z )KL π q , π q A(z ) is the matrix-derivative of -Λ/n 2 at D α . Since z is c/2-
regular and by definition of δ(π ), A(z ) qq ≥ cρδ(π ) if q = q and A(z ) qq = 0 for all q. By boundedness of the second derivative, there exists C > 0 such that for all D α and all H ∈ B(D α , C), we have:

-1 n 2 ∂ Λ ∂U qq (H) ≥ ρ 7cδ(π ) 8 if q = q ≤ ρ cδ(π ) 8
if q = q 72 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

MAIN RESULTS

Choose U in U c ∩ B(D α , C) satisfying U 1I = α(z ). U -D α have non-negative off diagonal coefficients and negative diagonal coefficients. Furthermore, the coefficients of U, D α sum up to 1 and Tr(D α ) = 1. By Taylor expansion, there exists H also in

U c ∩ B(D α , C) such that -1 n 2 Λ (U ) = -1 n 2 Λ (D α ) + Tr (U -D α ) -1 n 2 ∂ Λ ∂U (H) ≥ ρ cδ(π ) 8 [7 q =q (U -D α ) qq - q (U -D α ) qq = cρ 3δ(π ) 4 (1 -Tr(U ))
To conclude the proof, assume without loss of generality that z ∈ S(z , C) achieves the . 0,∼ norm (i.e. it is the closest to z in its representative class). Then U = IR(z) is in (U c ∩B(D α , C) and satisfy U 1I = α(z ). We just need to note n(1-Tr(IR(z))) = z-z 0,∼ to end the proof.

Proof of Proposition 23 (global convergence LR)

Proof. Conditionally upon z ,

LR(θ, z) -Λ(z) ≤ LR(θ, z) -ELR(θ, z) = i j (π zizj -π z i z j ) y ij r ij -ψ (π z i z j )ρ + i j (ψ(π zizj ) -ψ(π z i z j )) (r ij -ρ) = qq (π q -π q ) W qq ≤ sup Γ∈IR Q 2 ×Q 2 Γ ∞ ≤Diam(Θ) qq Γ qq W qq := Z
uniformly in θ, where the W qq are independent and by Taylor expansion defined by:

W qq = i j z iq z j z i,q z j (y ij r ij -ψ (π q )ρ -(r ij -ρ)C qq ) , C qq ∈ ψ (Θ)
is the sum of n 2 IR(z) qq IR(z) sub-exponential variables with parameters (ν 2 , 1/b) and is therefore itself sub-exponential with parameters (n 2 IR(z

) qq IR(z) ν 2 , 1/b). According to Proposition B.3 of Brault et al. (2017) , E θ [Z|z ] ≤ Q 2 Diam(Θ) √ n 2 ν 2 and Z is sub- exponential with parameters (n 2 Diam(Θ) 2 (2 √ 2) 2 ν 2 , 2 √ 2 Diam(Θ)/b).
In particular, for all ε n < νb

P θ Z ≥ νQ 2 Diam(Θ)n 1 + √ 8n 2 ε n Q 2 z ≤ P θ Z ≥ E θ [Z|z ] + ν Diam(Θ)n 2 2 √ 2ε n z ≤ exp - n 2 ε 2 n 2
We can then remove the conditioning and take a union bound. 73 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 11. MAIN RESULTS

Proof of Proposition 24 (contribution of far away assignments)

Proof. Conditionally on z , we know from proposition 22 that Λ is maximal in z and its equivalence class. Choose 0 < t n decreasing to 0 but satisfying nρtn √

log(n) → +∞. According to 22 (iii), for all z / ∈ S(z , t n ) Λ(z) ≤ -cρn 3δ(π ) 4 z -z 0,∼ ≤ -cρ 3δ(π ) 4 n 2 t n (11.1) since z -z 0,∼ ≥ nt n . Set ε n = inf(5cρδ(π )t n /( √ 2ν Diam(Θ)), νb) and n large enough that n ≥ Q 2 n √ 8
. By proposition 23, and with our choice of ε n , with probability higher than 1

-∆ 1 n (ε n ), z / ∈S(z ,tn) p(y o , z; θ) = p(y o |z , θ ) z / ∈S(z ,tn) p(z; θ)e LR(θ,z)-Λ(z)+ Λ(z) ≤ p(y o |z , θ ) z p(z; θ)e LR(θ,z)-Λ(z)-3n 2 tncρδ(π )/4 ≤ p(y o |z , θ ) z p(z; θ)e -n 2 tncρδ(π )/8 = p(y o , z ; θ ) p(z ; θ ) e -n 2 tncρδ(π )/8 ≤ p(y o , z ; θ ) exp -n 2 t n cρδ(π ) 8 + n log 1 c = p(y o , z ; θ )o(1)
where the second line comes from inequality (11.1), the third from the global control studied in Proposition 23 and the definition of ε n , the fourth from the definition of p(y o , z ; θ ), the fifth from the bounds on α and the last from nρtn √ log(n)

→ +∞.

In addition, with our choice of t n , we have ε n log(n)/n so that the series n ∆ 1 n (ε n ) converges and:

z / ∈S(z ,t nd ) p(y o , z; θ) = p(y o ; z , θ )o P (1)

Proof of Proposition 25 (local convergence LR)

Proof. We work conditionally on z ∈ Z 1 . Choose ε ≤ κσ 2 small. Assignments z at . 0,∼ -distance less than c/4 of z are c/4-regular. According to Proposition B.1 of [START_REF] Brault | Consistency and Asymptotic Normality of Latent Blocks Model Estimators[END_REF] , y q and ȳq are at distance at most ε with probability higher than 1 -

2 exp -n 2 c 2 ε 2 32(ν 2 +b -1 ε) . Defining Λ(z) = q N o q (z)ν(ȳ q (z), π q ) - q
N o q (z )ν(ψ (π q ), π q ), 74 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

MAIN RESULTS

where Λ(z) = E Λ(z)|z . Manipulation of Λ, Λ and Λ yield

Λ(z) -Λ(z) n 2 ≤ Λ(z) -Λ(z) n 2 + Λ(z) -Λ(z) n 2 = 1 n 2 q N o q (z) [f ( y q ) -f (ȳ q )] -N o q (z )π q ( y q -ȳ q ) + 1 n 2 q f (ȳ q ) N o q (z) -N o q (z ) -ρ(N q (z) -N q (z )) =A q + 1 n 2 q [N o q (z) -ρN q (z)](f (ȳ q ) -f (ȳ q ))
where

f (x) = x(ψ ) -1 (x) -ψ • (ψ ) -1 (x), y q = y q (z ) and ȳ q = ψ (π q ).
Concerning the first term. The function f is twice differentiable on Å with f (x) = (ψ ) -1 (x) and f (x) = 1/ψ • (ψ ) -1 (x). f (resp. f ) are bounded over I = ψ (C π ) by C π (resp. 1/σ 2 ) so that:

f ( y q ) -f (ȳ q ) = f (ȳ q ) ( y q -ȳq ) + Ω ( y q -ȳq ) 2
By Proposition B.1 (adapted for SBM) of [START_REF] Brault | Consistency and Asymptotic Normality of Latent Blocks Model Estimators[END_REF] , ( y q -ȳq )

2 = O P (1/n 2 )
where the O P is uniform in z and does not depend on z . Similarly,

f (ȳ q ) = f (ȳ q ) + Ω(ȳ q -ȳ q ) = π q + Ω(ȳ q -ȳ q )
ȳq is a convex combination of the S q = ψ (π q ) therefore,

|ȳ q -ȳ q | = IR(z) T S IR(z) q α q (z) α (z) -ȳ q ≤ 1 - IR(z) qq IR(z) α q (z) α (z) (S max -S min )
Note that:

q, N o q (z) 1 - IR(z) qq IR(z) α q (z) α (z) = n 2 ρ(1 + o P (1)) q, [1 -IR(z) qq IR(z) ] = n 2 ρ(1 + o P (1))[1 -Tr(IR(z)) 2 ] ≤ nρ(1 + o P (1))2 z -z 0,∼
and y q -ȳq = o P (1). Therefore

1 n 2 q, N o q (z)Ω(ȳ q -ȳ q ) × ( y q -ȳq ) = o P z -z 0,∼ n
The remaining term writes

1 n 2 q, π q N o q (z)( y q -ȳq ) -N o q (z )( y q -ȳ q )
and is also o P (( zz 0,∼ /n) uniformly in z and z ∈ Ω 1 by Proposition 28. 75 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

MAIN RESULTS

Concerning the second term. For all q, , defining

N + q (z, z ) = n 2 q IR(z) qq (z) IR(z) (z) -n 2 IR(z) qq IR(z) N - q (z, z ) = n 2 q IR(z) qq (z) IR(z) (z) -n 2 IR(z) qq IR(z)
and noticing that N + q (z, z ) = #{(i, j) : z iq = 1, z j = 1, (z q , z j ) = (z q , z j )} and N - q (z, z ) = #{(i, j) : z iq = 1, z j = 1, (z q , z j ) = (z q , z j )}. Using the following notations

ρ+ q = 1 N + q (z, z ) (i,j)∈N + q (z,z ) R ij , ρ- q = 1 N - q (z, z ) (i,j)∈N - q (z,z ) R ij
we are able to write

A q = i<j ziq=1,z j =1 (R ij -ρ) - i<j z iq =1,z j =1 (R ij -ρ) =N + q (z, z )(ρ + q -ρ) -N - q (z, z )(ρ - q -ρ).
Where the second equality is the sum of independent random variables. Note that :

q N + q (z, z ) = q N - q (z, z ) = n 2 q, [1 -IR(z) qq IR(z) ] = n 2 [1 -Tr(IR(z)) 2 ]
≤ n2 zz 0,∼ also that ρ+ q -ρ = o P (1) and ρq -ρ = o P (1). Therefore

1 n 2 q f (ȳ q )A q = o P z -z 0,∼ n .
Concerning the third term. Using arguments developed previously leads to the same conclusion than before :

1 n 2 q [N o q (z) -ρN q (z)](f (ȳ q ) -f (ȳ q )) = o P z -z 0,∼ n .
As a conclusion, writing

sup z z Λ(z) -Λ(z ) n z -z 0,∼ = sup z z Λ(z) -Λ(z) n z -z 0,∼ + Λ(z) -Λ(z ) n z -z 0,∼
and noticing that Λ(z)-Λ(z ) n z-z 0,∼ ≤ 0 since Λ is maximized in z (see 22). We have

sup z z Λ(z) -Λ(z ) n z -z 0,∼ = o P (1).
76 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 11. MAIN RESULTS

Proof of Proposition 26 (contribution of local assignments)

Proof. By Proposition 18, it is enough to prove that the sum is small compared to p(z , y o ; θ ) on Ω 1 . We work conditionally on z ∈ Z 1 . Choose z in S(z , C) with C defined in proposition 24. For C small enough, we can assume without loss of generality that z is the representative closest to z and note r = zz 0 . Then:

LR(θ, z) ≤ Λ(z) -Λ(z) + Λ(z) ≤ Λ(z) -Λ(z) -cρ 3δ(π ) 4 nr ≤ cρ 3δ(π ) 4 nr(1 + o P (1))
where the first line comes from the definition of Λ, the second line from Proposition 22 and the third from Proposition 25. Thanks to proposition 29, we also know that:

log p(z; θ) p(z ; θ ) ≤ O P (1) exp M c/4 r
There are at most n r Q r assignments z at distance r of z and each of them has at most

Q Q equivalent configurations. Therefore, z∈S(z ,c) z z p(z, y o ; θ) p(z , y o ; θ ) ≤ O P (1) r≥1 n r Q Q+r exp rM c/4 -cρ 3δ(π ) 4 nr(1 + o P (1)) = O P (1) 1 + e (Q+1) log Q+M c/4 -cρn 3δ(π )(1+o P (1)) 4 n -1 ≤ O P (1)a n exp(a n )
where a n = ne (Q+1) log Q+M c/4 -cρn 3δ(π )(1+o P (1))

4

= o P (1).

Proof of Proposition 27 (contribution of equivalent assignments)

Proof. Choose s permutations of {1, . . . , Q} and assume that z = z ,s . Then p(y o , z; θ) = p(y o , z ,s ; θ) = p(y o , z ; θ s ). If furthermore s ∈ Sym(θ), θ s = θ and immediately p(y o , z; θ) = p(y o , z ; θ). We can therefore partition the sum as 77 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 11. MAIN RESULTS

Proof of Corollary 18: Behavior of θ M LE

We may prove the corollary by contradiction. Note first that unless Θ is constrained and with high probability, θ M LE and θ(z ) exhibit no symmetries. Indeed, equalities like y q = y q , have vanishingly small probabilities of being simultaneously true when y ij is discrete, and even null when y ij is continuous. Assume then min s (

α s M LE -α (z )) = o P (1/ √ n) or min s ( π s M LE -π (z )) = o P (1/n)
p(y o , z ; θ ) -max s p y o , z ; θ s M LE p(y o , z ; θ ) = o P (1)
which contradicts Eq (11.2) and concludes the proof.

Proof of Corollary 19: Behavior of J (Q, θ)

Remark first that for every θ and for every z,

p (y o , z; θ) ≤ exp [J (δ z , θ)] ≤ max Q∈Q exp [J (Q, θ)] ≤ p (y o ; θ)
where δ z denotes the dirac mass on z. By dividing by p (y o ; θ ), we obtain

p (y o , z; θ) p (y o ; θ ) ≤ max Q∈Q exp [J (Q, θ)] p (y o ; θ ) ≤ p (y o ; θ) p (y o ; θ ) .
78 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

SUB-EXPONENTIAL RANDOM VARIABLES

As this inequality is true for every couple z, we have in particular:

max z∼z p (y o , z; θ) p (y o ; θ ) = max θ ∼θ p y o , z ; θ p (y o ; θ ) ≤ max Q∈Q exp [J (Q, θ)] p (y o ; θ ) .
Noticing that p (y o ; θ ) = # Sym(θ )p (y o , z ; θ ) (1 + o p (1)), Theorem 15 therefore leads to the following bounds:

max θ ∼θ p y o , z ; θ p (y o , z ; θ ) (1 + o P (1)) ≤ max Q∈Q exp [J (Q, θ)] p (y o , z ; θ ) ≤ # Sym(θ)max θ ∼θ p y o , z ; θ p (y o , z ; θ ) (1 + o P (1)) + o P (1).
Again unless Θ is constrained, θ V AR exhibits no symmetries with high probability and the same proof by contradiction as in appendix 11.10 gives the result.

12 Sub-exponential random variables

We now prove two propositions regarding sub-exponential variables. Recall first that a random variable X is sub-exponential with parameters (τ

2 , b) if for all λ such that |λ| ≤ 1/b, E[e λ(X-E(X)) ] ≤ exp λ 2 τ 2 2 .
In particular, all distributions coming from a natural exponential family are sub-exponential. Sub-exponential variables satisfy a large deviation Bernstein-type inequality:

P(X -E[X] ≥ t) ≤ exp -t 2 2τ 2 if 0 ≤ t ≤ τ 2 b exp -t 2b if t ≥ τ 2 b (12.1) So that P(X -E[X] ≥ t) ≤ exp - t 2 2(τ 2 + bt)
The sub-exponential property is preserved by summation and multiplication.

• If X is sub-exponential with parameters (τ 2 , b) and α ∈ R, then so is αX with parameters (α 2 τ 2 , αb)

• If the X i , i = 1, . . . , n are sub-exponential with parameters (τ 2 i , b i ) and independent, then so is X = X 1 + • • • + X n with parameters ( i τ 2 i , max i b i ) Theorem 20 (Equivalent

characterizations of sub-exponential variables). For a zero-mean random variable X, the following statements are equivalent:

1. There are non-negative numbers (ν, b -1 ) such that

E[e λX ] ≤ exp λ 2 ν 2 2 for all |λ| < b.
2. There is a positive number c 0 > 0 such that E[e λX ] < ∞ for all |λ| < c 0 .

3. There are constants c 1 , c 2 > 0 such that P(|X| ≥ t) ≤ c 1 e -c2t for all t > 0. 79 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 13. LIKELIHOOD RATIO OF ASSIGNMENTS

The quantity γ

:= sup k≥2 E[X k ] k! 1/k is finite.
Proof. A proof of this theorem can be found in Wainwright (2015).

Proposition 28 (Maximum in z). Let (z be any configuration and z the ∼-equivalent configuration that achieves zz 0 = zz 0,∼ let y q = ŷq, (z) (resp. ȳq (z)) and y q = ŷq, (z ) (resp. ȳ q = ȳq (z ) = ψ (π q )) be as defined in Equations (3.1) and (5.3).

Under the assumptions of the section 2.5, for all ε ≤ κσ 2 , P max

z z max k,l N o q (z)(ŷ q, -ȳq ) -N o q (z )( y q -ȳ q ) n z -z 0 > ε = o(1)
Proof. Note r = zz 0 . The numerator within the max in the fraction can be expanded to

Z q (z) = i,j (z iq z j -z iq z j )(y ij r ij -π z iq z j ρ)
and is thus a sum of at most N = nr non-null centered sub-exponential random variables with parameters (a 2 , 1/w). It is therefore a centered sub-exponential with parameters (N a 2 , 1/w). By Bernstein inequality, for all ε ≤ κa 2 we have

P(Z ≥ εnr) ≤ exp - nrε 2 2a 2 .
There are at most n r Q r Q Q z at . 0,∼ distance r of z . An union bound shows that:

P max z z max q, Z q (z) n z -z 0 ≥ ε ≤ r≥1 r= z-z 0,∼ Q 2 P(Z q (z) ≥ εnr) ≤ r≥1 Q Q exp -nrε 2 /2a 2 + r log(nQ) + 2 log(Q) = o(1)
where the last equality is true as soon as nε n log n.

13 Likelihood ratio of assignments Proof. Note then that:

p(z; θ) p(z ; θ ) = p(z; α) p(z ; α ) = p(z; α) p(z ; α(z )) p(z ; α(z )) p(z ; α ) ≤ p(z; α(z)) p(z ; α(z )) p(z ; α(z )) p(z ; α ) ≤ exp M c/4 z -z 0 × p(z ; α(z )) p(z ; α ) ≤ O P (1) exp M c/4 z -z 0
where the first inequality comes from the definition of α(z) and the second from Lemma B.6 of [START_REF] Brault | Consistency and Asymptotic Normality of Latent Blocks Model Estimators[END_REF] and the fact that z and z are c/4-regular. Finally, local asymptotic normality of the MLE for multinomial proportions ensures that p(z ; α(z )) p(z ;α ) = O P (1). 80 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 14. GENERAL TECHNICAL RESULTS

General technical results

Proposition 30 (Maximum in z). Let z be a configuration and ŷq, (z) resp. ȳq (z) be as defined in Equations (3.1) and (5.3). Under the assumptions of the section 2.5, for all ε > 0

P max z max k,l α q (z) α (z)|ŷ q, -ȳq | > ε ≤ Q n+2 exp - n 2 ε 2 2(σ 2 + κ -1 ε) . (14.1)
Additionally, the suprema over all c/2-regular assignments satisfies:

P max z∈Z1 max k,l |ŷ q, -ȳq | > ε ≤ Q n+2 exp - n 2 c 2 ε 2 8(σ 2 + κ -1 ε) . (14.2)
Note that equations 14.1 and 14.2 remain valid when replacing c/2 by any c < c/2.

Proof. The random variables Y ij R ij are sub-exponential with parameters (ν 2 , 1/b). Conditionally to z , z +q z + (ŷ q, -ȳq ) is a sum of z +q z + centered sub-exponential random variables. By Bernstein's inequality [START_REF] Boucheron | Concentration Inequalities: A Nonasymptotic Theory of Independence[END_REF], we therefore have for all t > 0

P(z +q z + |ŷ q, -ȳq | ≥ t) ≤ 2 exp - t 2 2(z +q z + ν 2 + b -1 t) In particular, if t = n 2 y, P ( α q (z) α (z)|ŷ q, -ȳq | ≥ x) ≤ 2 exp - n 2 y 2 2( α q (z) α (z)ν 2 + b -1 y) ≤ 2 exp - n 2 y 2 2(ν 2 + b -1 y)
uniformly over z. Equation (14.1) then results from a union bound. Similarly,

P (|ŷ q, -ȳq | ≥ x) = P ( α q (z) α (z)|ŷ q, -ȳq | ≥ α q (z) α (z)y) ≤ 2 exp - n 2 y 2 α q (z) 2 α (z) 2 2( α q (z) α (z)ν 2 + b -1 y α q (z) α (z)) ≤ 2 exp - n 2 c 2 y 2 8(ν 2 + b -1 y)
Where the last inequality comes from the fact that c/2-regular assignments satisfy α q (z) α (z) ≥ c 2 /4. Equation (14.2) then results from a union bound over Z 1 ⊂ Z.

Lemme 3. If X is a zero mean random variable, sub-exponential with parameters

(σ 2 , b), then |X| is sub-exponential with parameters (8σ 2 , 2 √ 2b). Proof. Note µ = E|X| and consider Y = |X| -µ. Choose λ such that |λ| < (2 √ 2b) -1 . We need to bound E[e λY ]. Note first that E[e λY ] ≤ E[e λX ] + E[e -λX
] < +∞ is properly defined by sub-exponential property of X and we have

E[e λY ] ≤ 1 + k=2 |λ| k E[|Y | k ] k!
where we used the fact that E[Y ] = 0. We know bound odd moments of |λY |.

E[|λY | 2k+1 ] ≤ (E[|λY | 2k ]E[|λY | 2k+2 ]) 1/2 ≤ 1 2 (λ 2k E[Y 2k ] + λ 2k+2 E[Y 2k+2 ])
81 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3

GENERAL TECHNICAL RESULTS

where we used first Cauchy-Schwarz and then the arithmetic-geometric mean inequality. The Taylor series expansion can thus be reduced to

E[e λY ] ≤ 1 + 1 2 + 1 2.3! E[Y 2 ]λ 2 + +∞ k=2 1 (2k)! + 1 2 1 (2k -1)! + 1 (2k + 1)! λ 2k E[Y 2k ] ≤ +∞ k=0 2 k λ 2k E[Y 2k ] (2k)! ≤ +∞ k=0 2 3k λ 2k E[X 2k ] (2k)! = E cosh 2 √ 2λX = E e 2 √ 2λX + e -2 √ 2λX 2 ≤ e 8λ 2 σ 2 2
where we used the well-known inequality

E[|X -E[X]| k ] ≤ 2 k E[|X| k ] to substitute 2 2k E[X 2k ] to E[Y 2k ].
Proposition 31 (concentration for sub-exponential). Let X 1 , . . . , X n be independent zero mean random variables, sub-exponential with parameters

(σ 2 i , b i ). Note V 2 0 = i σ 2 i and b = max i b i .
Then the random variable Z defined by:

Z = sup Γ∈IR n Γ ∞≤M i Γ i X i is also sub-exponential with parameters (8M 2 V 2 0 , 2 √ 2M b). Moreover E[Z] ≤ M V 0 √ n so that for all t > 0, P(Z -M V 0 √ n ≥ t) ≤ exp - t 2 2(8M 2 V 2 0 + 2 √ 2M bt) (14.3) Proof. Note first that Z can be simplified to Z = M i |X i |. We just need to bound E[Z].
The rest of the proposition results from the fact that the |X i | are sub-exponential (8σ 2 i , 2 √ 2b i ) by Lemma 3 and standard properties of sums of independent re-scaled subexponential variables.

E[Z] = E    sup Γ∈IR n Γ ∞≤M i Γ i X i    = E i M |X i | ≤ M i E[X 2 i ] = M i σ i ≤ M i 1 1/2 i σ 2 i 1/2 = M V 0 √ n using Cauchy-Schwarz.

Lemme 4.

Let Z 1 be the subset of Z of c-regular configurations, as defined in Definition 11. Let

S Q = {α = (α 1 , α 2 , . . . , α Q ) ∈ [0, 1] Q : Q k=1 α k = 1} be the Q-dimensional simplex and note S Q c = S Q ∩ [c, 1 -c] Q .
Then there exists two positive constants M c and M c such that for all z, z in Z 1 and all α ∈ S Q c |log p(z; α(z)) -log p(z ; α(z ))| ≤ M c zz 0 82 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 14. GENERAL TECHNICAL RESULTS

Proof. Consider the entropy map H

: S Q → R defined as H(α) = - Q k=1 α k log(α k ). The gradient ∇H is uniformly bounded by Mc 2 = log 1-c c in . ∞ -norm over S Q ∩ [c, 1 -c] Q . Therefore, for all α, α ∈ S Q ∩ [c, 1 -c] Q , we have |H(α) -H(α )| ≤ M c 2 α -α 1
To prove the inequality, we remark that z ∈ Z 1 translates to α(z)

∈ S Q ∩ [c, 1 -c] Q , that log p(z; α(z))-log p(z ; α(z )) = n[H( α(z))-H( α(z ))] and finally that α(z)-α(z ) 1 ≤ 2 n z -z 0 .
83 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 3 14. GENERAL TECHNICAL RESULTS 84 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 4

Introduction

This chapter deals with missing data in the Stochastic Block Model (in short SBM) with covariates. The case with no covariates and missing data in the SBM has already been treated in Chapter 2. We follow this work and introduce three different models of SBM with covariates on nodes or dyads. In order to explore the link between the sampling design and the covariates we define one sampling design centered on dyads and another one centered on nodes, both depending on covariates. We exhibit a specific case where a sampling design which is MAR conditionally on covariates becomes NMAR when the covariates are not observed. We show explicitly the equivalence between the two models and run simulations to compare the inference with a MAR model conditionally on covariates, NMAR model and a MAR model. We also propose another case where there is no explicit equivalence but where the NMAR model leads to better inference than a MAR model when the covariates are missing.

Related works. Stochastic Block Models with the adding of covariates are popular to model heterogeneity in networks. The main purpose of accounting for covariates in the model is to remove covariates effects on clustering. In Tallberg ( 2004), the authors proposed a Bayesian approach to model an SBM with covariates with a multinomial probit model where distribution of the block given by the latent variable Z i depends on the vector of covariates X i . As a consequence, in this model covariates act on dyads through nodes memberships. When covariates are associated to dyads, [START_REF] Mariadassou | Uncovering latent structure in valued graphs: A variational approach[END_REF] combined these covariates with the hidden structure using the framework of the generalized linear model, for example in the binary SBM with covariates

Y ij |z i = q, z j = , X ij ∼ ind B(logistic(γ q + β T X ij )). (1.1)
Chapter 4

STOCHASTIC BLOCK MODELS WITH COVARIATES AND SAMPLING MODELS

A very close model is proposed in [START_REF] Zanghi | Clustering based on random graph model embedding vertex features[END_REF] with Gaussian distribution on dyads. In these frameworks, heterogeneity is modeled by parameters (γ q ) q corresponding when X = 0 to inter and intra block probabilities (or connectivity matrix) in the binary SBM. It accounts for heterogeneity that is not explained by the regression term β T X ij . A similar interpretation is given in Choi et al. (2012b) who first apply a regression step and then perform SBM inference on the residuals. SBM with covariates has also been applied in [START_REF] Miele | Spatially constrained clustering of ecological networks[END_REF] in a spatial data context where nodes correspond to entities that have explicit geographic locations. If no hidden structure is supposed, [START_REF] Robins | An introduction to exponential random graph (p*) models for social networks[END_REF] show how to incorporate covariates into a graph model. Alternative models are used to model covariates in SBM, like in [START_REF] Sweet | Incorporating covariates into stochastic blockmodels[END_REF] where authors use a hierarchical Bayesian model. When the principal purpose is community detection, the previous models enable to detect structure in the network beyond the effect of covariates while the following references focus on detecting structure by using both covariates and the network. We cite [START_REF] Zhang | Community detection in networks with node features[END_REF] which is based on a joint community detection criterion and [START_REF] Binkiewicz | Covariate-assisted spectral clustering[END_REF] using spectral properties of the adjacency matrix of a network.

Stochastic Block Models with covariates and sampling models

We will consider three different structures in which covariates and missing data impact the SBM. Conditional dependencies of these models are represented with a DAG in Figure 4.1.

We choose to study only cases where covariates represented by X impact either the sampling design or the network model directly. Furthermore, note that the systematic edge between Z and Y is part of the SBM. Finally, we do not consider any edge between nodes Y, Z and node R since we require that conditionally on X the missing data are MAR. In the first model, covariates influence directly latent variables and the sampling. In the second model covariates influence also the sampling and the distribution of edges. Finally, in the last model, covariates influence latent variables, the sampling and the distribution of edges.

X Z Y R X Z Y R X Z Y R (model 1) (model 2) (model 3) Figure 4
.1 -DAGs of relationships between Y, Z, R and X considered in the framework of missing data for SBM with covariates.

Model 1 Writing α • = (α •1 , ..., α •Q ) ∈ [0, 1] Q we define α iq = e β t q Xi1 {q =Q} 1 + Q-1 k=1 β t k X i , ∀(i, q) ∈ N × Q, with β q ∈ R N for all q ∈ 1, Q -1 and β Q = 0. Then we have Z i | X i ∼ iid M(1, α i ), ∀i ∈ N , Y ij | Z i , Z j ∼ ind B(π zizj ), ∀(i, j) ∈ N 2 ,
with π q ∈ [0, 1], (q, ) ∈ Q 2 . 86 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 4

STOCHASTIC BLOCK MODELS WITH COVARIATES AND SAMPLING MODELS

Model 2

Z i ∼ iid M(1, α), ∀i ∈ N , Y ij | Z i , Z j , X ij ∼ ind B(g(γ zizj + β t X ij )), ∀(i, j) ∈ N 2 ,
where

γ ∈ M Q ([0, 1]), β ∈ R m , α ∈ [0, 1] Q , g(x) = (1 + e -x
) -1 . Note that if the covariates are linked with nodes, the covariates are transferred to dyads by using a "similarity" symmetric function φ(•,

•) : R N × R N → R m such that X ij = φ(X i , X j ) for all (i, j) in D. Model 3 Writing α • = (α •1 , ..., α •Q ) ∈ [0, 1] Q we define α iq = e β t q Xi1 {q =Q} 1 + Q-1 k=1 β t k X i , ∀(i, q) ∈ N × Q, with β q ∈ R N for all q ∈ 1, Q -1 and β Q = 0. Then we have Z i | X i ∼ iid M(1, α i ), ∀i ∈ N , Y ij | Z i , Z j , X ij ∼ ind B(g(γ zizj + β t X ij )), ∀(i, j) ∈ N 2 ,
with parameters and notations defined as in Model 2.

In the three previous models, we need to specify the distribution R|X. This distribution can be naturally one of the two following distributions depending whether X is a set of covariates on nodes or on dyads. We then define two sampling designs, one dyad-centered and the other node-centered. In both sampling, the probability to observe a dyad (resp. node) depends on the value of the covariate. If the covariates have no effect, then the probability to observe a dyad (resp. node) is independent of the dyad (resp. node).

Definition 6 (Dyad sampling). Let δ ∈ R, κ ∈ R m . The probability to observe a dyad is

P(R ij = 1|X ij ) = g(δ + κ t X ij ), with ψ = {δ, κ}.
For all i ∈ N , we denote by

V i = 1 if the node i is observed, V i = 0 otherwise. Then, if V i = 1 we have R ij = 1 for all j ∈ N .
Definition 7 (Node sampling). Let ν ∈ R and η ∈ R N . The probability to observe all dyads corresponding to a node is

P(V i = 1|X i ) = g(ν + η t X i ),
with ψ = {ν, η}.

In the next paragraph, we exhibit a particular case where we can provide an equivalence between a MAR model conditionally on covariates and a NMAR model when covariates are not observed. Indeed, when sampling a network, some covariates may not be available. In this case, a NMAR model may compensate for the absence of covariates.

A case of equivalence. In this paragraph, we demonstrate on a specific case that an SBM with covariates where the missing data are MAR conditionally on covariates can be equivalent to an SBM without covariates with missing data NMAR. We first recall the definition of the block-node sampling design where nodes are sampled depending on the group they belong to. 87 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 4

STATISTICAL INFERENCE

Definition 8 (Block-node sampling). Let ρ = (ρ q ) 1≤q≤Q ∈ [0, 1] Q . In block-node sampling, the conditional distribution of V|Z is given by

V i |Z i ∼ ind B(ρ zi ). (2.1)
Proposition 32. Let's define the covariates matrix

X := [X 1 , ..., X n ] ∈ M Q×n ({0, 1}) for Q > 1.
The model is generated as follow

X i ∼ iid M(1, α), ∀i ∈ N , P(Z ia = 1|X ia = 1) = δ and P(Z ib = 1|X ia = 1) = 1 -δ Q -1 , b = a, ∀i ∈ N .
The probabilities to observe a dyad are given by

p q = P(V i = 1|X iq = 1).
Then, this model corresponds to a conventional SBM under block-node sampling, with parameters

ρ q = P(V i = 1|Z iq = 1), = δp q α q + 1-δ Q-1 =q p α δα q + 1-δ Q-1 (1 -α q ) . Remark 1. δ = 1 ⇒ ρ q = p q .
This result illustrate the article [START_REF] Molenberghs | Every missing not at random model has got a missing at random counterpart with equal fit[END_REF] which states that any model generating missing data that are MAR has a dual model generating missing data NMAR and fitting equally the data.

Statistical inference

On the basis of Figure 4.1, the type of missingness for SBM is defined as follows:

Sampling design for SBM is

     MCAR if R |= (Y m , Z, Y o ) | X, MAR if R |= (Y m , Z) | (Y o , X), NMAR otherwise. (3.1) Proposition 33. From (3.1), if the sampling is MAR or MCAR then maximizing p θ,ψ (Y o , R) or p θ (Y o ) in θ is equivalent.
In the light of (3.1), sampling designs defined in definitions 6 and 7 are missing completely at random (MCAR) conditionally on covariates.

Inference of Model 1

In the MAR case, inference can be conducted directly and without bias on the observed part of the adjacency matrix. We start by recalling the complete likelihood p θ (Y o , Z|X) which has an explicit form contrary to the likelihood of the observed data p θ (Y o |X).

The complete log-likelihood restricted to the observed variables is

log p θ (Y o , Z|X) = (i,j)∈D o q, Z iq Z j log b(Y ij , π q ) + i∈N o q Z iq log (α iq ) , (3.2)
with b(x, π) = π x (1 -π) 1-x the Bernoulli probability density function. 88 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 4

STATISTICAL INFERENCE

By Equation (3.2) and "mean-field" approximation, the close form of the lower bound is

J τ ,θ (Y o |X) = (i,j)∈D o q, τ iq τ j log b(Y ij , π q ) + i∈N o q τ iq log(α iq /τ iq ).
(3.3)

We refer to the notations defined page 5 for D o and N o . The two maximization problems are solved as stated in the following proposition, straightforwardly derived from [START_REF] Daudin | A mixture model for random graphs[END_REF].

Proposition 34. Consider the lower bound J τ ,θ (Y o |X) given by (3.3).

The parameter

π maximizing J π (Y o |X) when τ is held fixed is πq = (i,j)∈D o τiq τj Y ij (i,j)∈D o τiq τj , α = arg max α J τ ,θ (Y o |X)
has no explicit form and must be estimated with an optimization algorithm. Corresponding gradient is

∂J τ ,θ (Y o |X) ∂α iq = τ iq α iq . (3.4)
2. The variational parameters τ maximizing J τ (Y o |X) when θ is held fixed are obtained thanks to the following fixed point relation:

τiq ∝ α iq   (i,j)∈D o ∈Q b(Y ij ; π q ) τj   .

Inference of Model 2

The complete log-likelihood for Model 2 is

log p θ (Y o , Z|X) = (i,j)∈D o Y ij g Z i t γZ j + β t X ij + (1 -Y ij ) log(1 -g Z i t γZ j + β t X ij ) + i∈N o Z t i log (α) , (3.5) with h(x) = log(g(x)) = -log(1 + e -x
). As a consequences the variational lower bound is given by 1. The parameters γ and β maximizing J τ ,θ (Y o |X) when all other parameters are held fixed have no explicit forms and must be estimated with an optimization algorithm.

J τ ,θ (Y o |X) = (i,j)∈D o q, τ iq τ j (Y ij -1)(γ q + β t X ij ) + h(γ q + β t X ij ) (3.6) + i,q τ iq log α q τ iq . ( 3
Corresponding gradients are

∂J τ ,θ (Y o |X) ∂γ q = (i,j)∈D o τ iq τ j Y ij -1 + e -(γ q +β t Xij ) 1 + e -(γ q +β t Xij ) , (3.8) ∂J τ ,θ (Y o |X) ∂β k = (i,j)∈D o q, τ iq τ j (X ij ) k Y ij -1 + e -(γ q +β t Xij )
1 + e -(γ q +β t Xij ) . (3.9) 89 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 4

3. STATISTICAL INFERENCE Concerning α, αq = i∈N o τiq card (N o ) .
2. The optimal τ in J τ ,θ (Y o |X) when all other parameters are held fixed verify

τiq ∝ α q (i,j)∈D o exp (Y ij -1)(γ q + β t X ij ) + h(γ q + β t X ij ) τj .
(3.10)

Inference of Model 3

The inference of Model 3 can be done directly by combination of Model 1 and Model 2 estimation procedures. The complete log-likelihood for Model 3 is

log p θ (Y o , Z|X) = (i,j)∈D o Y ij g Z i t γZ j + β t X ij + (1 -Y ij ) log(1 -g Z i t γZ j + β t X ij ) + i∈N o Z t i log (α i ) , (3.11) with h(x) = log(g(x)) = -log(1 + e -x
). As a consequences the variational lower bound is given by 1. The parameters α, γ and β maximizing J τ ,θ (Y o |X) when all other parameters are held fixed have no explicit forms and must be estimated with an optimization algorithm.

J τ ,θ (Y o |X) = (i,j)∈D o q, τ iq τ j (Y ij -1)(γ q + β t X ij ) + h(γ q + β t X ij ) (3.12) + i,q τ iq log α iq τ iq . ( 3
Corresponding gradients are the same as in Equations (3.4), (3.8) and (3.9).

2. The variational parameters τ maximizing J τ ,θ (Y o |X) when θ is held fixed are obtained thanks to the following fixed point relation:

τiq ∝ α iq (i,j)∈D o exp (Y ij -1)(γ q + β t X ij ) + h(γ q + β t X ij ) τj .
(3.14)

Model selection

The Integrated Classification Likelihood criterion was introduced by [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF] for mixture models, where the likelihood -and thus BIC -is usually intractable. [START_REF] Daudin | A mixture model for random graphs[END_REF] adapted a variational ICL in the context of SBM. Under MAR condition, this criterion requires a slight adaptation stated in the following proposition.

Proposition 37 (Model selection). For an SBM with Q blocks and for θ = arg max log p θ (Y o , Z), the ICL criterion is given by

ICL(Q) = -2E pτ log p θ (Y o , Z; Q, X) + pen Q , and pen Q =          Q(Q+1) 2 log card (D o ) + N (Q -1) log card (N o ) (in Model 1), Q(Q+1) 2 + m log card (D o ) + (Q -1) log card (N o ) (in Model 2), Q(Q+1) 2 + m log card (D o ) + N (Q -1) log card (N o ) (in Model 3), (3.15)
Note that a dyad is only counted once since we work with symmetric networks. The number of blocks chosen is the one associated to the lowest ICL. 90 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 4 4. ILLUSTRATIONS

Illustrations

We consider in this section two cases where the NMAR modeling appears like a good alternative when some covariates are not available.

Example 1

In this Section we illustrate Proposition 32 on a simple example available in Figure 4.2. In this example, we consider an SBM with covariates with n = 100 nodes following Model 1. Furthermore, the network is sampled according to a block-node sampling with sampling parameters p = (.8, .2, .5) corresponding to probabilities to observe a node conditionally to his covariate. As a consequence, with these parameters, the sampling rate is varying in the interval (.46, .828]. Finally the parameter δ is varying and takes three different values : 1, 2/3 and 1/3. The greater δ the less X depends on Z.

In Figure 4.2, boxplots correspond to estimations of the connectivity matrix on the left and to the adjusted Rand index (in short ARI Rand, 1971) between the true clustering and the predicted one on the right. We run the VEM algorithms for SBM with block-node sampling (NMAR) and with node sampling (MAR) described in (Tabouy et al., 2019a) and finally VEM for SBM with covariates corresponding to Model 1 (see Section 3.1).

Figure 4.2 shows that in this example, making the MAR assumption is not the best way to deal with missing data when covariates are missing. Furthermore, we see that the estimation of the clustering and of π is made with the same accuracy for VEM_NMAR and VEMCov1, moreover results do not depend on the value of the parameter δ. Finally, note that the estimations of parameter ρ (results not reported here), linked to parameter p as described in Proposition 32, appears to be good in terms of Frobenius norm when δ is equal to 1 and 2/3 but is degrading when δ gets low as when it is equal to 1/3.

Example 2

Considering an SBM with covariates on dyads with missing entries, we explore various network topologies (namely affiliation, star and bipartite). The connectivity matrix represented in SBM with covariates context by the matrix g -1 (γ) of which are given in Figure 4.3. This parameter is classically considered (see [START_REF] Mariadassou | Uncovering latent structure in valued graphs: A variational approach[END_REF] as monitoring heterogeneity in the SBM and then the topology of the network.

In the following, we will consider that each dyad has one covariate and that they are generated conditionally to latent variables Z as following :

X ij = 1 {zi=zj } (1 -B ij ) + (1 -1 {zi=zj } )B ij , (4.2)
where B ij are independent Bernoulli variables with the tuning parameter p. According to these covariates, the SBM is as in model 3 with the specificity that we give here the conditional distribution of X|Z and not the inverse as suggested in Figure 4.1. A simple application of Bayes formula allow us to find the conditional distribution of Z|X if wanted.

Coming back to the definition of X ij , note that the smaller the parameter p, the more X depend on Z. The value p = 0 corresponds to the case where X encode exactly Z. This particular case when X is observed is called "Oracle" in order to show the response of the VEM described in Section 3.2 when covariates are available. Cases corresponding to p = .2 and p = .5 are explored in Figure 4.5. In parallel of the Oracle we run VEM for SBM without 91 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 4 4. ILLUSTRATIONS δ = 2/3 q q q q q q q q q q q q 0.00 δ = 1/3 q q q q q q q q q 0.00

π -π F / π F ARI(Z, Ẑ) δ = 1 q q q 0.

92

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 4 4. ILLUSTRATIONS covariates and missing data from a double-standard sampling. This design states that every edge is sampled with probability ρ 1 and every non-edge is sampled with probability ρ 0 . It implies that missing data are NMAR. We also run random-dyad sampling where every dyad is sampled with probability ρ. It implies that missing data are MAR. More details are available in Tabouy et al. (2019a) about these sampling designs and estimation methods. They are respectively called "NMAR" and "MAR". Simulated networks have n = 100 nodes. Marginal probabilities α are chosen specifically for affiliation, star and bipartite topologies, respectively (1/3, 1/3, 1/3), (.15, .35, .15, .35) and(1/4, 1/4, 1/4, 1/4). The sampling parameters δ is fixed at 1 and κ takes iteratively values (-1.4, -.8, -.25, .1) in order to have a panel of sampling rates. Algorithms are initialized with spectral clustering.

Ŷm -Y m F /nm ARI(Z, Ẑ) affiliation q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.0 0.2 0.4 0.6 0.8

[0.169,0.294] (0.294,0.432] (0.432,0.587] (0.587,0.745] q q q q q q q q q q q q q q q q q q q q q q q q q q 0.00 0.25 0.50 0.75 1.00

[0.169,0.294] (0.294,0.432] (0.432,0.587] (0.587,0.745] bipartite q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.0 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.00 method Oracle VEM_mar VEM_nmar star q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.0 0.1 0.2 0.3 0.4 0.5 [0.137,0.254] (0.254,0.488] (0.488,0.695] (0.695,0.732] q q q q q q q q q q q q q q q q q q q q q q q 0.00 Simulations on Figure 4.4 show that "NMAR" algorithm is a good alternative when covariates are unknown whereas "MAR" algorithm does not estimate well parameters and does not predict correctly the clustering. However, the more the sampling rate increases, the more similar are the performance of algorithms "NMAR" and "MAR". On Figure 4.5, where parameter p denoting how much X encode Z, the difference between "NMAR" and "MAR" decrease when p increase. Indeed, "NMAR" algorithm accuracy is linked to parameter p in the way that it is greater as much as p is close to 0, which corresponds to the case where X 93 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 4 5. CONCLUSION depends the most on Z. We have not yet fully understood the reason for this phenomenon, it would be worth studying.

Ŷm -Y m F /nm ARI(Z, Ẑ) p = 0.2 q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.2 0.4 0.6 0.8

[0.203,0.341] (0.341,0.486] (0.486,0.631] (0.631,0.764] q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.00 p = 0.5 q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.3 0.4 0.5 0.6 [0.249,0.419] (0.419,0.559] (0.559,0.686] (0.686,0.809] q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.00

Conclusion

In this paper we described several combining sampling of a network generated under an SBM with covariates. Algorithms have been proposed to estimate parameters of the models and recover the clustering. Furthermore, we investigate cases where some covariates are not available and show through two examples where NMAR modeling can be a better alternative than MAR modeling. During our exploration, we encounter many simulation settings leading to similar performances for MAR and NMAR models (simulations not reported here). These considerations would deserve further investigations to understand the relatively good robustness of the MAR modeling. Note that however, we never found MAR approximation to be better that NMAR approximation. 94 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 Chapter 5 missSBM: An R Package for

Handling Missing Values in the Stochastic Block Model

This chapter is available as a preprint on arXiv (Tabouy et al., 2019b). The Stochastic Block Model (SBM) is a popular probabilistic model for random graph.

It is commonly used to perform clustering on network data by aggregating nodes that share similar connectivity patterns into blocks. When fitting an SBM to a network which is partially observed, it is important to account for the underlying process that originates the missing values, otherwise the inference may be biased. This paper introduces missSBM, an R-package fitting the SBM when the network is partially observed, i.e. the adjacency matrix contains not only 1 or 0 encoding presence or absence of edges but also NA encoding missing information between pairs of nodes. It implements a series of algorithms for the binary SBM, with the possibility of accounting for covariates if needed, by performing variational inference for several sampling mechanisms, the methodology of which is detailed in Tabouy et al. (2019a). Our implementation automatically explores different block numbers to select the most relevant according to the Integrated Classification Likelihood (ICL) criterion. The ICL criterion can also help to determine which sampling mechanism fits the best the data. Finally, missSBM can be used to perform imputation of missing entries in the adjacency matrix. We illustrate the package on a network data set consisting in interactions between blogs sampled during the French presidential election in 2007.

INTRODUCTION 1 Introduction

In many fields of science, networks are a natural way to represent interaction data. To cite a few examples, a network may represent social interactions such as friendship or collaboration between people in a social network, regulation between genes and their products in a gene regulatory network, or predation between animals in a food web. Notice that we only consider here networks which can be represented by graphs composed by simple edges connecting pairs of nodes (also referred to as dyads in the following). At this day, there exist many softwares performing network-related analyzes. Unsurprisingly, the R community is extremely active in this area. Indeed, the R programming language is especially well-designed for performing data manipulation and visualization, and thus appropriate for handling network data. Among the many available R packages related to networks, we suggest a classification into three groups: i) Packages performing representation, manipulations, visualization tasks, and/or packages computing descriptive statistics on networks. This group clearly occupies the first place in terms of number of packages. We may cite non exhaustively the following top representatives: igraph [START_REF] Csardi | The igraph software package for complex network research[END_REF], network (Butts, 2008a), statnet [START_REF] Handcock | statnet: Software tools for the representation, visualization, analysis and simulation of network data[END_REF], or sna (Butts, 2008b).

ii) Packages fitting (probabilistic) models on network data. Most of the existing packages in this group are dedictated to the estimation of a specific network model: important examples include ergm [START_REF] Hunter | ergm: A package to fit, simulate and diagnose exponential-family models for networks[END_REF], fitting the family of exponential random graph models introduced in Hunter and Handcock (2006b); mixer [START_REF] Ambroise | MixeR: Random Graph Clustering[END_REF] and blockmodels [START_REF] Leger | Blockmodels: A r-package for estimating in latent block model and stochastic block model, with various probability functions, with or without covariates[END_REF], fitting the family of Stochastic Block Models [START_REF] Holland | Stochastic blockmodels: First steps[END_REF]Nowicki and[START_REF] Nowicki | Estimation and prediction for stochastic blockstructures[END_REF]; or latentnet [START_REF] Krivitsky | Fitting latent cluster models for networks with latentnet[END_REF], implementing the latent space approach of [START_REF] Hoff | Latent space approaches to social network analysis[END_REF].

iii) Packages learning the structure of a network from an external source of data, such as huge (Zhao et al., 2012a), or bnstruct [START_REF] Franzin | bnstruct: an r package for bayesian network structure learning in the presence of missing data[END_REF]. These packages generally rely on an specific graphical modeling of the data (e.g., Gaussian graphical models [START_REF] Lauritzen | Graphical models[END_REF] in huge, or Bayesian networks [START_REF] Pearl | Bayesian networks[END_REF] in bnstruct).

In addition to this brief typology, the interested reader may consult the CRAN task view on the related topic of graphical modeling [START_REF] Hojsgaard | Cran task view: graphical models in R[END_REF].

The package missSBM that we introduce here belongs to the second category, that is, softwares that fit a probabilistic model to network data. missSBM is dedicated to the estimation of the stochastic block model (SBM). The SBM is a mixture of Erdős-Rényi random graphs [START_REF] Erdős | On random graphs[END_REF] that allows a high degree of heterogeneity in connectivity profiles. As a consequences it generally fits well real-world network data, while retaining the advantage of being a generative model (contrary to mechanistic approaches such as the Barabási-Albert model (Albert and Barabási, 2002), defined by a preferential attachment algorithm). The main outcome of the SBM inference on a network is a clustering of its nodes -or "blocks" sharing the same connectivity properties.

Even though there already exist efficient R packages for SBM inference such as Blockmodels [START_REF] Leger | blockmodels: Latent and Stochastic Block Model Estimation by a 'V-EM' Algorithm[END_REF] and mixer [START_REF] Ambroise | MixeR: Random Graph Clustering[END_REF], they can only be applied to complete data sets. The main feature introduced in missSBM is to deal with cases where the network data is only partially observed. More precisely, we consider situations where the adjacency matrix encoding the network data contains contains not only 1 or 0 encoding presence or absence of edges but also NA encoding missing information between dyads. In the presence of missing data, it is important to account for the underlying process that originates the missing values in the estimation of a probabilistic model, otherwise estimation of the model parameters may be biased. In particular, one has to take the type of missing data mechanisms into account (Missing at Random or Not, see [START_REF] Rubin | Inference and missing data[END_REF]. This issue has been 96 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 2. STATISTICAL FRAMEWORK studied in the context of network data in [START_REF] Handcock | statnet: Software tools for the representation, visualization, analysis and simulation of network data[END_REF] for exponential random graph models and in our previous paper Tabouy et al. (2019a) for stochastic block models. The package missSBM is an implementation of the methodology developed therein.

Specifically, missSBM implements variational algorithms in the vein of [START_REF] Daudin | A mixture model for random graphs[END_REF] for estimating the SBM for binary network data, with or without covariates, under various missing data mechanisms. This includes cases of incomplete data where the inference can be made only on the observed part of the data (Missing at Random), or cases where it is necessary to take the sampling design in the inference into account (Not Missing at Random). Although version 0.2.0 of missSBM only deals with binary networks (either directed or not), we deploy a structure that let the possibility to easily include other variants in the future by adopting an oriented-object programming spirit thanks to R6-classes and the R6 package of [START_REF] Chang | R6: Classes with Reference Semantics[END_REF]. In particular, extending missSBM to weighted SBM with exponential distributions of the edges [START_REF] Mariadassou | Uncovering latent structure in valued graphs: A variational approach[END_REF]) should be straightforward.

The paper is organized as follows: Section 2 briefly introduces the statistical framework of the binary SBM, with or without covariates, and summarizes the key points for estimating the SBM under missing data condition, further detailed in Tabouy et al. (2019a). Section 3 presents the package structure; Section 4 provides basic user guidelines. We finally detail in Section 5 a case study which analyzes a network data set describing the French blogosphere during French presidential election of 2007, illustrating the most striking features of the package.

Statistical Framework

Binary Stochastic Block Model (SBM)

In an SBM, nodes from a set N 1, n are distributed among a set Q 1, Q of hidden blocks that model the latent structure of the graph. The group memberships are described by categorical variables (Z i , i ∈ N ) with multinomial distribution M(1, α = (α 1 , ..., α Q )). The probability of an edge between any pair of nodes (or dyad) in D N × N only depends on the blocks the two nodes belong to. Hence, the presence of an edge between i and j, indicated by the binary variable Y ij , is independent on the other edges conditionally on the latent blocks:

Y ij |Z i , Z j ∼ ind B(π ZiZj ), ∀(i, j) ∈ N 2 , ( 2.1)
where B stands for the Bernoulli distribution. In the following, we denote by π = (π q ) (q, )∈Q 2 the connectivity matrix, α the mixture parameters, Z = (Z 1 , ..., Z n ) T the n×Q membership matrix and Y = (Y ij ) (i,j)∈D the n × n adjacency matrix. The vector encompassing all the unknown model parameters is θ = (α, π). A schematic representation of the binary SBM in the undirected case is given in Figure 5.1, where we highlight the latent clustering.

Accounting for External Covariates

On top of information about connections between nodes, it is common for network data to be accompanied with additional information on nodes or dyads, that we call covariates: for instance in social network, nodes may belong to different categories (gender, occupation, nationality). Covariates on dyads usually represent distances between nodes covariates: for example in a context of spatial data where nodes corresponds to entities that have explicit geographic locations, dyad covariates may be the distances between the nodes. Depending on the analysis, we may want to detect a connectivity pattern beyond the covariate effect. To do so, we present here a variant of the SBM implemented in missSBM that allows the user to include covariates in the model. Let X ij denote the vector of length m of covariates for dyad (i, j). If the covariates correspond to the nodes, i.e. X i ∈ R N is associated with node i for all i ∈ N , they are transferred on the dyad level through a symmetric "similarity" function 97 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 2. STATISTICAL FRAMEWORK

A1 A2 A3 π •• B1 B2 B3 B4 B5 π •• C1 C2 π •• π •• π •• π •• • Q = {•, •, •} blocks • α • = P(i ∈ •), • ∈ Q, i = 1, . . . , n • π •• = P(i ↔ j|i ∈ •, j ∈ •)
φ(•, •) : R N × R N → R m : X ij φ(X i , X j ). In the following, X [X ij ] i,j∈N ∈ (R m
) n×n denotes the covariates. An SBM accounting for covariates is

Z i ∼ iid M(1, α), ∀i ∈ N , Y ij | Z i , Z j , X ∼ ind B(g(γ zizj + β X ij ), ∀(i, j) ∈ N 2 , ( 2.2)
where

γ q ∈ R, β ∈ R m , α = (α 1 , ..., α Q ), g(x) = (1 + e -x
) -1 . In this case the vector of unknown parameters is defined by θ = (γ, β, α). Note the connection between this model and logistic regression. In this framework, heterogeneity is modeled by parameters (γ q ) q corresponding to inter and intra block probabilities (or connectivity matrix) in the binary SBM. It accounts for heterogeneity that is not explained by the regression term β T X ij .

Missing Data and SBM

The main feature of missSBM is to deal with missing values to perform unbiased estimation of the parameters of the model underlying an incompletely observed network. The sampled data can be encoded in an adjacency matrix Y where missing information -here dyads not sampled -is encoded by NA's. We also define the n × n sampling matrix R such as R ij = 1 if the dyad Y ij is sampled and R ij = 0 otherwise. For convenience we define Y o = {Y ij : R ij = 1} and Y m = {Y ij : R ij = 0} the respective sets of observed and non-observed dyads. At this stage it is important to notice that the number of nodes n is assumed to be known.

In our framework, a sampling design is a stochastic process that generates R. We then rely on the standard missing data theory of [START_REF] Rubin | Inference and missing data[END_REF] to classify those designs either into Missing Completely At Random (MCAR), Missing At Random (MAR) or Not Missing At Random (NMAR) cases. We summarize the analyzes conducted in Tabouy et al. (2019a) on missing data for network as follows:

Sampling design for SBM is

     MCAR if R |= (Y, Z), MAR if R |= (Y m , Z) | Y o , NMAR otherwise. (2.3)
Note that MCAR missingness is a particular case of MAR missingness. Denoting by ψ the set of parameters associated with the sampling design distribution that generates R, we assume that ψ and θ are living in a product space, so that we can derive the following proposition: 98 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 In words, the inference can be conducted on the observed part of the network data when the sampling is MAR or NMAR without incurring any biased. In these cases, adaptation of existing algorithm for SBM inference is straightforward. NMAR sampling designs require, however, more refined inference strategies.

Examples of Sampling Designs for Networks

This section reviews a series of stochastic processes -or sampling designs -available in missSBM either for sampling an existing network or to be accounted for in the inference of an SBM model. Note that the sampling design may depend either on i) the values of the dyads in the network; ii) the latent clustering of the nodes; or iii) some covariates. We specify for each case when the sampling is MAR or NMAR. The sampling design examples detailed in the following assume the independence between dyad observation for dyad-centered sampling designs and between node observation for node-centered sampling design conditionally on Y, Z and X.

Dyad-Centered Sampling Designs

• Dyad sampling (MAR): each dyad (i, j) ∈ D has the same probability P(R ij = 1) ψ to be observed.

• Double standard sampling (NMAR): let ψ (ρ 1 , ρ 0 ) ∈ [0, 1] 2 . Double standard sampling consists in observing dyads with probabilities

P(R ij = 1|Y ij = 1) = ρ 1 , P(R ij = 1|Y ij = 0) = ρ 0 .
The probabiliy for sampling a dyad thus intrinsically depend on the presence/absence of the corresponding edge.

• Block-dyad sampling (NMAR): this sampling consists in observing all dyads with probabilities ψ (ψ q ) (q, )∈Q 2 such that

ψ q = P(R ij = 1 | Z iq = 1, Z j = 1).
Thus sampling thus depends on the underlying clustering of the network.

• Covar-dyad sampling (MAR):let us define ψ (α, κ) ∈ R × R m . In this sampling, the probability to observe a dyad is driven by the effect of a given covariate:

P(R ij = 1|X ij ) = g(α + κ t X ij ).

Node-Centered Sampling Designs

A node-centered sampling consists in drawing some nodes to observe with probabilities given by the sampling design. Observing a node means observing all the dyads implying this node. For all i ∈ N , we denote by V i = 1 if the node i is observed, V i = 0 otherwise. Then, if V i = 1 we have R ij = 1 for all j ∈ N .

• Node sampling (MAR): the probabilities for observing nodes are uniform: P(V i = 1) = ψ for all i ∈ N .

• Degree sampling (NMAR): for all node i ∈ N , P(V i = 1) = ρ i where (ρ 1 , . . . , ρ n ) ∈ [0, 1] n are such that ρ i = g(a + bD i ) for all i ∈ N where ψ (a, b) ∈ R 2 and D i = j Y ij . 99 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5

STATISTICAL FRAMEWORK

• Block-node sampling (NMAR): this sampling consists in observing all dyads corresponding to nodes selected with probabilities ψ (ψ 1 , . . . , ψ Q ) ∈ [0, 1] Q such that ψ q = P(V i = 1 | Z iq = 1) for all (i, q) ∈ N × Q.

• Covar-node sampling (MAR): let ψ (ν, η) ∈ R × R N . The probability to observe a node is

P(V i = 1|X i ) = g(ν + η t X i ).

Estimation Procedure

Optimization: a Variational EM

The SBM is a latent state space model which can be seen as a mixture model for random graphs. As such the EM algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF] is the natural choice for the inference. It is based on the evaluation of the expectation of the complete loglikelihood of the model, with respect to the conditional distribution of the latent variables given the data. However, this expectation is intractable in the SBM due to the structure of dependency between the latent variables Z and the network Y. In fact, it would require to sum over all possible clustering for all pairs of nodes, which is out of reach even for a moderate number of nodes or clusters. To address this shortcoming in the complete data situation, [START_REF] Daudin | A mixture model for random graphs[END_REF] introduced a variational EM, based on the variational principles of [START_REF] Jordan | An introduction to variational methods for graphical models[END_REF]. The idea is to maximize a lower bound of the loglikelihood based on an approximation of the true conditional distribution of the latent variable Z.

In the case of an SBM with missing data, the level of difficulty is higher since the set of latent variables encompasses both Z (the latent clustering) and Y m (the missing dyads). We propose here a variational distribution of the conditional distribution p θ (Z, Y m |Y o ) where complete independence is forced on Z and Y m , using a multinomial m(•), respectively a Bernoulli b(•) distribution for Z and Y m :

pτ,ν (Z, Y m ) = pτ (Z) pν (Y m ) = i∈N m(Z i ; τ i ) (i,j)∈D m b(Y ij ; ν ij ),
where τ = {τ i , i ∈ N } and ν = {ν ij , (i, j) ∈ D m } are two sets of variational parameters respectively associated with Z and Y m . Interestingly, τ 's are proxies for the posterior probabilities of memberships for all nodes, and ν's correspond to the imputed values of the missing dyads in the network data. This approximation leads to the following lower bound J of the loglikelihood, where KL is the Kullback-Leibler divergence between the true and approximated conditional distribution:

log p θ,ψ (Y o , R) ≥ J τ ,ν,θ,ψ (Y o , R) log p θ,ψ (Y o , R) -KL(p τ ,ψ (Z, Y m ) p θ (Z, Y m Y o )), = E pτ,ν [log p θ,ψ (Y o , R, Y m , Z)] -E pτ,ν [log pτ,ν (R, Y m )] .
Note that when we chose p to be the true conditional distribution of the latent variables Z, Y m , we meet again the quantity maximized in the standard EM algorithm.

Based on this approximation, the variational EM algorithm consists in alternatively updates of the variational parameters {τ , ν} (the VE-step) and updates of θ, ψ maximizing J τ ,θ (the M-step). Steps VE and M are iterated until convergence like in a standard EM. The algorithm converges to a local maximum of the lower bound of the loglikelihood. This variational principle is translated into a series of algorithms for handling missing data with all sampling designs introduced in Section 2.4. We underline that some time consuming parts of the VEM algorithms are coded in C++ using the Rcpp package [START_REF] Eddelbuettel | Rcpp: Seamless R and C++ integration[END_REF] to interface C++ with R. 100 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5

STRUCTURE OF THE PACKAGE

Initialization

It is well known that algorithms based on EM algorithm have a great sensitivity to initialization. This step therefore requires a special attention. In missSBM we implemented several methods for initializing the variational EM: the Absolute Eigenvalues Spectral Clustering (detailed in [START_REF] Rohe | Spectral clustering and the high-dimensional stochastic block model[END_REF]; a Hierarchical Ascending Classification (HAC) based on Manhattan norm between nodes; and the K-means algorithm. Experiences showed us that the HAC algorithm is sometimes more robust in cases where the network contains a large proportion of missing dyads. However, in general, we recommend the use of the Absolute Eigenvalues Spectral Clustering, which is used by default in the package.

Selection of the Number of Blocks

A main difficulty met when conducting SBM inference lies in the estimation of the number of blocks, generally unknown to the user. To remedy this problem we use the Integrated Classification Likelihood (ICL) criterion defined in [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF] and routinely used in the framework of mixture models. More precisely, we implement in missSBM an exploration procedure designed to produce a smooth and robust ICL curve, by avoiding to get stuck in local minima. This "smoother" is divided into two steps, forward and backward:

The forward step creates new initializations for each number Q of block considered, by splitting blocks obtained from estimations with Q -1 blocks. On the other hand, the backward step tries new initializations for each Q by merging groups of the model with Q + 1 groups. The best model in terms of ICL is always retain. The procedure can be iterated until an satisfying smoothing is reached.

Structure of the Package

The R package missSBM is coded using object-oriented programming with R6 package [START_REF] Chang | R6: Classes with Reference Semantics[END_REF]. 101 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5

GUIDELINES FOR USERS

The package contains two general R6 classes: SBM and networkSampling. These two classes instantiate objects that collect all parameters needed to respectively define an SBM and a sampling design, as depicted in Sections 2.1 and 2.2 for SBMs and in Section 2.4 for the sampling design. These classes both give birth to two child classes, a sampler class and a fit class, which inheritate of all methods and fields of their respective mother classes. The sampler class allows the user to obtain a realization of network data sampled under an SBM (i.e. an adjacency matrix Y), and a realization of the sampling of a network (i.e. a sampling matrix R). The fit classes contains methods for estimating the SBM and the sampling design parameters. Finally, SBM_fit gives birth to two child classes: SBM_fit_nocovariates and SBM_fit_covariates, respectively dedicated to the estimation of binary SBM with or without covariates. Note that the two classes SBM and network_Sampling can be used totally independently. In particular, it is possible to fit SBM without missing data. In order to perform SBM inference when the network is partially observed, two others classes of object are introduced: sampledNetwork and missSBM_fit. The former instantiates an object which collects any information about presence or absence of dyads and nodes in a sampled network. The latter makes the connection between objects of classes SBM and network_sampling, in order to make it possible to infer SBM from partially-observed network data, under several sampling designs.

Guidelines for Users

The package missSBM allows the user to do three differents actions: simulate a network under the SBM, sample a network according to a panel of sampling designs, and estimate an SBM from partially observed network data. To do so, three standard R functions are exported to the user, which use internally the classes of object detailed in Section 3. We describe in the following the usage of these functions plus some additional auxiliary functions, as well as their connections with models defined in Section 2.

Simulate a network

The function simulate draws a realization of an SBM, with the following usage: The arguments of simulate and their mathematical counterpart from (2.1) and (2.2) are described in Table 5 Note that the network simulated are by default undirected and without covariates. In the case of an SBM with covariate(s), covarParam must not include intercepts since this role is played by parameters (γ q ) q, in model (2.2). 102 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5

GUIDELINES FOR USERS

The output of simulate is an R6 object belonging to the class SBM_sampler (see Figure 5.2). An non-exhaustive list of the most useful fields, accessible via $ are presented in Table 5.2. A plot method is also available which allows the user to either draw the network by reordering rows and columns of the adjacency matrix Y according to the nodes memberships, or draw the predicted connectivity matrix ZYZ , i.e. the probability of connection for each nodes once reordered block-wise. This drawing is a simple yet powerful way to visualize the structure of large networks.

Sampling a network

The function sample draws a realization of a sampling matrix R from a distribution chosen from the series of sampling designs defined in Section 2.4. The usage is the following:

φ R N × R N → R m Table 5
.3 -Arguments of sample and their counterparts in the SBM. ( ) possible values for sampling are characters string in "dyad", "double-standard", "block-dyad", "node", "blocknode", "degree", "covar-node" and "covar-dyad" for SBM without covariate, and "dyad", "node", "covar-node" and "covar-dyad" SBM with covariates.

Note that the dimension (or length) of parameters depends on the sampling design selected, as described in Section 2.4. Argument clusters only needs to be specified for "block-dyad" and "block-node" sampling designs. Arguments covarMatrix is by-default NULL, covarSimilarity is set to the l1_similarity function defined as follows: (

x, y) ∈ R d × R d → -|x -y| 1 ∈ R d
and finally intercept is set to 0. These last three arguments only needs to be specified in a context of SBM with covariate(s). Note that the intercept is not included in parameters and must be specified independently.

The output of sample is an R6 object belonging to the class sampledNetwork (see Figure 5.2). An non-exhaustive list of the most useful fields is given in Table 5.4, where stands for the Hadamard (or element-wise) product. A plot method is defined for sampledNetwork objects, drawing the adjacency matrix Y of the network with missing entries induced by the sampling matrix R. 103 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5

GUIDELINES FOR USERS

Parameters estimation, prediction and clustering

The purpose of the function estimate is to perform variational inference of a SBM from sampled adjacency matrix, with the following usage: missSBM::estimate(sampledNet, vBlocks, sampling, clusterInit = "spectral", useCovariates = TRUE, control = list())

Data preparation.

When the network data is not obtained from the function sample but from real-world network, the user has to format this data to an object with class sampledNetwork. We implemented a simple function for this task, taking as an input an adjacency matrix filled with {0, 1, NA}:

missSBM::prepare_data(adjacencyMatrix, covariates = NULL, similarity = missSBM:::l1_similarity)

Tuning the optimization. The argument control is a list to finely tune the variational EM algorithm, with the following entries:

(i) threshold: optimization stops when a V-EM step changes the objective function by less than threshold, (default = 1.10 -3 );

(ii) maxIter: optimization stops when the number of iteration exceeds maxIter, default = 50 if useCovariates is TRUE, 100 otherwise;

(iii) fixPointIter: number of iterations in the fix point algorithm used to solve the Variational E step, = 2 if useCovariates is TRUE, 5 otherwise; 104 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 4. GUIDELINES FOR USERS (iv) cores: number of thread be ran in parallel, = 1 by default;

(v) trace: integer for verbosity (0, 1, 2), useless when cores > 1.

Estimation ouputs: classes missSBM_collection and missSBM_fit. The output of estimate is an R6 object with class missSBM_collection, fields of which are described in Table 5.6. Among the fields of missSBM_collection, the field models is a list with length(vblocks) elements which are R6 objects with class missSBM_fit (see Figure 5.2). These missSBM_fit objects, fields of which are detailed in 5.7, contain all the results of the inference for a fixed number of block Q. Smoothing and model exploration. At the end of the estimation process, it is common that the algorithm get stuck in some local minima for some values of Q, the number of blocks. The consequence is an "non-smooth" ICL curve, which is theoretically supposed to be convex. We thus provide to the user a post-processing function to "smooth" the ICL by exploring additional models with new initialization. Smoothing of the ICL curve is performed with function smooth, detailed in the following : 105 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 5. ILLUSTRATION: THE 2007 FRENCH POLITICAL BLOGOSPHERE missSBM::smooth(Robject, type = c("forward", "backward", "both"), control = list())

The basic idea is to apply a split and/or merge strategy to the path of models in a collection of SBM, in order to find better initialization. This should result in a "smoothing" of the ICL, that should be close to convex. Arguments of smooth are described in Table 5.9.

Argument Description

R6 object / class Robject an output from estimate missSBM_collection type kind of ICL smoothing : "forward", "backward" or "both" character control controlling the variational EM algorithm list Table 5.9 -Arguments of smooth function with short descriptions and their types.

This function acts directly on Robject (i.e. by reference). The argument type is bydefault set to "forward". Finally, control is a list composed of 3 elements : (i) cores the number of R processes allowed set by-default to 1, (ii) iterates the number of iteration by-default set to 1 and (iii) trace which is by default TRUE.

Illustration: the 2007 French political blogosphere

This section illustrates the features of missSBM by conducting an analysis of a real-world network data. It is a sub-network of the French political blogosphere, extracted from a snapshot of over 1100 blogs collected during a period preceding the 2007 French presidential election, and manually classified by the "Observatoire Présidentielle project" (see [START_REF] Zanghi | Fast online graph clustering via erdős-rényi mixture[END_REF], . The network is composed of 196 blogs representing nodes in the network and 1432 edges indicating that at least one of the two blogs references the other. On top of missSBM, our analysis relies on magrittr and igraph for standard (network) data manipulation. We also fix the seed for reproducibility:

set.seed(1234) library(missSBM) library(igraph) library(magrittr)

The frenchblog2007 data set is shipped with missSBM 1 . We provide the data as an igraph object, which is the most convenient way of manipulating network data in R, in our opinion. We extract the adjacency matrix corresponding to the blog network and the vertex attribute party, providing the political party of each blog, also giving a natural classification of the nodes that could be used as a reference in our analyzes. data("frenchblog2007", package = "missSBM") class(frenchblog2007)

[1] "igraph" adjacencyMatrix <-frenchblog2007 %>% as_adj(sparse = FALSE) party <-vertex.attributes(frenchblog2007)$party Once reordered row-wise and column-wise according to party, the adjacency matrix shows us that a part of the underlying structure is indeed supported by the political party (see Figure 5.3).

1 earlier versions of this data set were available in packages mixer and sand 106 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 The global sampling rate obtained for this of the network data is

sampledNet$samplingRate ## [1] 0.7950811
The sampleNet object owns a plot method representing the network data in the missing data context (see right-panel in Figure 5.3): plot(sampledNet, clustering = factor(party), main = "") Estimation of a partially observed network. We are now in position of performing inference of SBM under missing data condition. We fit two types of model: first the SBM under the NMAR block-node sampling design, i.e. under the design that truly generated the missing entries; second the SBM under the star sampling, a.k.a MCAR node sampling in the package. The estimation is ran on both models with the same setting as for the fully observed data (i.e., by varying the number of blocks in 1, 15 and using the same tuning parameters for the initialization step and the optimization procedure). The ICL curve is smoothed with a forward-backward smoothing.

We first run the estimation for the block-node sampling and report ICL values before and after smoothing: sbm_block <-estimate(sampledNet, vBlocks, "block-node", control = control) ICL_block_nsm <-sbm_block$ICL missSBM::smooth(sbm_block, smoothing_type, control) Figure 5.4 illustrates the dramatic change in the model selection process due to the smoothing. 108 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 Sampling design comparison. Now, we consider the simple MCAR node sampling which basically performs inference only on the observed part of the network, neglecting the process that originates the missing values:

ILLUSTRATION: THE 2007 FRENCH POLITICAL BLOGOSPHERE

sbm_node <-estimate(sampledNet, vBlocks, "node", control = control) missSBM::smooth(sbm_node, smoothing_type, control)

Figure 5.5 shows how ICL can be used to select which sampling design fit at best the data by comparing the smoothed ICL curves for "block-node" and "node" sampling designs. Note that the curve associated with the block-node sampling uniformly dominates the curve associated with the star sampling, showing that this sampling design is more adapted to the network data at play. We also represent the ICL curve of the SBM estimated on the fully observed network: although the values of the ICLs cannot be compared with the ones obtained for the partially observed network (indeed, data are not the same in this case), the number of block selected in the different cases remains comparable. The ICL criterion selects 11 blocks for an SBM adjusted on the fully observed network, while the SBM with missing entries accounted for by block-node sampling only selects 9 blocks, with a more flatter ICL curve. Indeed, due to the partial sampling, some blocks are less well represented than others; and it seems more likely to gather some blocks together considering the information available. 109 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5

ILLUSTRATION: THE 2007 FRENCH POLITICAL BLOGOSPHERE

Regarding the clusterings obtained by the three variants (fully observed, missing entries with MCAR modeling and missing entries with NMAR block-node sampling), we compare them with the Adjusted Rand Index (ARI, [START_REF] Rand | Objective criteria for the evaluation of clustering methods[END_REF], computed with the aricode package [START_REF] Chiquet | aricode: Efficient Computations of Standard Clustering Comparison Measures[END_REF]. We use the classification of the SBM fitted on the fully observed network as a reference, since its clustering was used to sample the network with the blocknode sampling design. We typically expect that an SBM relying on a better modeling of the missing values shall lead to a clustering closer to the reference. This is indeed the case when looking at the next piece of code, where it is shown that the ARI with the reference clustering is 50% higher for the SBM with block-sampling than for SBM with MCAR node sampling: aricode::ARI(sbm_block$bestModel$fittedSBM$memberships, cl0)

[1] 0.6031785 aricode::ARI(sbm_node$bestModel$fittedSBM$memberships , cl0)

[1] 0.4412234 Extraction of the SBM with block-sampling design. The model that we finally retain it thus block-sampling with 9 blocks.

myModel <-sbm_block$bestModel

As seen from Figure 5.2, myModel is an object with class missSBM_fit with two important fields used for storing the results of the estimation of both the SBM (field fittedSBM) and the sampling design (fittedSampling). The important fields and methods are recalled to the user thanks to the print methods: $type, $parameters, $prob, $df $penalty, $vExpec 110 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 Representation and validation With myModel, we now have at hand a tool for analyzing the clustering of the French political blogosphere. The first output is the connectivity matrix of the network, which puts into light the community structure of the blogosphere. Indeed, it is revealed by a diagonal filled with high probabilities and off-diagonal with low probabilities. Thus, nodes (blogs) into blocks connects with high probability with other nodes of the same block and with low probability with nodes of other blocks. Such a network concentrates most of its edges between nodes of the same blocks. This can be seen by displaying the probability of connection predicted by the SBM at the whole network scale:

plot(myModel$fittedSBM, type = "connectivity")

For validation, we suggest to compare the clustering of the model with the node attribute corresponding to the political parties to which blogs belong to. First, we remark that the SBM fitted on missing entries carry the same amount of information regarding the political party than the SBM adjusted on the fully observed network: aricode::ARI(party, myModel$fittedSBM$memberships)

[1] 0.4279665 aricode::ARI(party, sbm_full$bestModel$fittedSBM$memberships)

[1] 0.4244866 A more detailed comparison between blocks inferred by the SBM and political parties is reported in Figure 5.7 with an alluvial diagram.

Also remember that missSBM performs imputation of the missing dyads in the adjacency matrix. Thus, we can compare the imputed values with the value sof the dyad in the fully observed network to validate the performance of our approach. Using the R package pROC [START_REF] Robin | proc: an open-source package for r and s+ to analyze and compare roc curves[END_REF], we check the quality of the imputation. The following piece of code generates a singe ROC curve for the current sampling and imputation. Results in Figure 5.8 display ROC curves for 100 samplings of missing entries (always with block-sampling design). The sampling rate varies between ≈ 0.4 and ≈ 0.9. It shows the robustness and the good performance of the imputation method. 111 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 Finally, the Area Under the Curve (AUC) when the sampling rate is varying between 0.43 and 0.9 is plotted in Figure 5.9. We can see that the more the sampling rate increases the more the AUC increases. 112 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 5. ILLUSTRATION: THE 2007 FRENCH POLITICAL BLOGOSPHERE AUC q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.6 0.7 0.8 0.9 0.4 0.6 0.8 sampling rate Figure 5.9 -Area Under the Curve (AUC) in function of the sampling rate.

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Chapter 5 5. ILLUSTRATION: THE 2007 FRENCH POLITICAL BLOGOSPHERE 114 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q

Conclusion

Conclusion et perspectives

Conclusion

Dans le chapitre 2 nous avons étudié les différents types de données manquantes selon la théorie développée dans [START_REF] Rubin | Inference and missing data[END_REF] et nous l'avons adaptée au cas du SBM. Fort de la dichotomie MAR/NMAR nous avons donné des exemples de stratégies d'échantillonnage aussi bien centrées sur les noeuds que sur les dyades illustrant les cas de données manquantes MAR et NMAR pour le SBM. Un algorithme EM avec approximation variationnelle a été proposé pour s'adapter à tous les cas de données manquantes dont la loi du processus d'échantillonnage suit un modèle connu. Une étude de simulation poussée a permis de montrer la pertinence de la prise en compte de la nature des données manquantes dans l'inférence. De plus elle a permis de montrer qu'il était possible de choisir parmi plusieurs stratégies laquelle expliquait le mieux les données. Enfin l'identifiabilité du SBM sous trois des modèles de données manquantes définis dans ce chapitre a été établie. Dans le chapitre 3 nous nous sommes intéressés à l'étude asymptotique des estimateurs du maximum de vraisemblance et de l'approximation variationnelle du SBM en présence de données manquantes. Le cadre considéré est celui d'un SBM dont la loi d'émission d'une dyade appartient à la famille exponentielle à un paramètre avec des données manquantes telles que chaque dyade est observée indépendamment et avec même probabilité. Ainsi, les estimateurs du maximum de vraisemblance et de l'approximation variationnelle sont consistants et asymptotiquement normaux. De plus, la variance asymptotique est explicite. Enfin, nous avons montré l'identifiabilité du SBM pour toute la famille exponentielle à un paramètre en présence de données manquantes.

Le chapitre 4 introduit différents modèles de SBM avec covariables et données manquantes. Les liens entre données manquantes et covariables sont définis ainsi que des exemples de stratégies d'échantillonnage. Nous avons montré que sous certaines conditions, un SBM sans covariable avec données manquantes NMAR est équivalent à un SBM avec covariables et données manquantes MAR. De plus, à travers des simulations nous montrons que sans la connaissance de covariables, les algorithmes NMAR développés dans le chapitre 2 sont dans certains cas une alternative intéressante pour l'estimation des paramètres du modèle et la prédiction de la classification par rapport au fait de considérer les données manquantes MAR pour un SBM sans covariables.

Finalement, le chapitre 5 décrit le package missSBM, développé dans la language R et actuellement sur le CRAN. Chaque fonction est précisément présentée ainsi que son utilisation. Plusieurs exemples reproductibles fondés sur des jeux de données disponibles directement dans le package sont proposés pour permettre une meilleure introduction et prise en main de missSBM.

PERSPECTIVES Conclusion et perspectives

Une extension naturelle serait d'étudier d'autres lois pour les dyades du SBM comme par exemple la loi de Poisson pour des données de comptage et la loi normale pour des données continues. En effet, définir des stratégies d'échantillonnage et une méthode d'inférence pour les cas NMAR n'est pas évident et ne découle pas directement de notre travail. Il apparaît que l'approximation variationnelle que nous avons proposée (i.e. supposer les dyades manquantes indépendantes, de même lois avec leurs propres paramètres) pour le cas binaire avec des données manquantes NMAR ne produise pas des estimateurs explicites des paramètres variationnels et du SBM. Ceci résulte du fait que dans le cas binaire, la vraisemblance est linéaire en les dyades et s'intègre directement, comme pour la loi binomiale et la loi multinomiale. Cependant ce n'est le cas par exemple pour la loi de Poisson où la vraisemblance d'une dyade donnée par log p θ (Y ij |Z iq = 1, Z j = 1) = -π q + Y ij log(π q ) -log(Y ij !) dont l'espérance quand Y ij est manquant n'est pas directement calculable avec une approximation de loi de type "champs moyen". De plus, il n'est pas évident que des algorithmes d'optimisation soient efficaces compte tenu de la non convexité en général du problème. Il pourrait être intéressant aussi de se demander si des méthodes comme des algorithmes consistant à simuler conditionnellement au reste les Y ij manquants permettraient de résoudre ce problème.

De nombreux problèmes théoriques persistent encore à propos de l'étude asymptotique des estimateurs du maximum de vraisemblance, ainsi que de celle des estimateurs du maximum de vraisemblance issus de l'approximation variationnelle pour des stratégies d'échantillonnage noeuds centrés MAR et toutes les stratégies NMAR. Ce problème est difficile et nécessite probablement pour chaque stratégie une preuve spécifique. De plus, il serait intéressant de montrer l'identifiabilité des modèles définis lorsqu'elle n'est pas connue. Il semble que les preuves classiques d'identifiabilité du SBM ne s'appliquent pas pour toutes les stratégies que nous avons définies. Notre intuition est que ces modèles sont identifiables, les simulations allant dans ce sens. Finalement, il nous semblerait important de mieux comprendre pourquoi dans les cas où des noeuds sont échantillonnés, faire l'hypothèse que des données manquantes sont MAR quand elles sont NMAR constitue une hypothèse robuste dans de nombreux cas que nous avons pu observer sur des simulations. Il serait important de pouvoir identifier ces cas de manière à pouvoir prévoir et prévenir quand l'hypothèse MAR est crédible. D'autres stratégies d'échantillonnage pour le cas de réseaux binaires qui paraîtraient naturelles pourraient aussi être définies. Le package missSBM a d'ailleurs été pensé pour pouvoir accueillir facilement de nouvelles stratégies.

Dans toute cette thèse, la taille des réseaux étudiés est supposé connue. Cette hypothèse pose naturellement une question récurrente que nous n'avons pas explorée qui est de ne plus supposer la taille du réseau connue. C'est ce qu'il est proposé de faire dans l'article de [START_REF] Vincent | Estimating the size and distribution of networked populations with snowball sampling[END_REF] dans lequel les auteurs estiment conjointement la taille du réseau et les paramètres du SBM qu'ils explorent avec un échantillonnage one-wave snowball sampling. On pourrait se demander si leurs méthodes s'appliquent à d'autres cas comme ceux que nous étudions. Notre intuition est que cela ne devrait pas modifier la nature des données manquantes, ce qui devrait rendre simple l'inférence des cas MAR. Les cas NMAR nécessite d'estimer la taille du réseau, on pourrait imaginer par exemple que n soit aléatoire avec une loi puissance et l'estimer alternativement avec les paramètres du modèle.

Il serait intéressant aussi de se poser la question de l'impact des "liens fallacieux" (i.e. observés avec du bruit) décrits dans Clauset et al. (2008) et Guimerà and[START_REF] Guimerà | Missing and spurious interactions and the reconstruction of complex networks[END_REF] sur la modélisation et la nature des données manquantes ainsi que leur prise en compte dans l'inférence.

Enfin, faisant le lien avec des problèmes venant de l'écologie, on pourrait se demander comment prendre en compte des stratégies NMAR pour le SBM qui échantillonnent uniquement des liens existant (i.e. des arêtes) et ceci en tenant compte éventuellement de covariables. Ceci consisterait a priori à appliquer une stratégie d'échantillonnage comme le 116 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Conclusion et perspectives 2. PERSPECTIVES deux poids deux mesures en prenant en compte à la fois la valeur des covariables associées à une dyade et la valeur de la dyade dans la probabilité de l'échantillonner. 117 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 2. PERSPECTIVES Conclusion et perspectives 118 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q BIBLIOGRAPHY BIBLIOGRAPHY C. Matias and S. Robin. Modeling heterogeneity in random graphs through latent space models: a selective review. ESAIM Proc. Sur., 47:55-74, 2014. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Titre : Impact de l'échantillonnage sur l'inférence de structures dans les réseaux : application aux réseaux d'échanges de graines et à l'écologie Title : Impact of sampling on structure inference in networks: application to seed exchange networks and to ecology Keywords : Networks, Stochastic Block Model, missing data Abstract : In this thesis we are interested in studying the stochastic block model (SBM) in the presence of missing data. We propose a classication of missing data into two categories Missing At Random and Not Missing At Random for latent variable models according to the model described by D. Rubin. In addition, we have focused on describing several network sampling strategies and their distributions. The inference of SBMs with missing data is made through an adaptation of the EM algorithm: the EM with variational approximation. The identiability of several of the SBM models with missing data has been demonstrated as well as the consistency and asymptotic normality of the maximum likelihood estimators and variational approximation estimators in the case where each dyad (pair of nodes) is sampled independently and with equal probability. We also looked at SBMs with covariates, their inference in the presence of missing data and how to proceed when covariates are not available to conduct the inference. Finally, all our methods were implemented in an R package available on the CRAN. A complete documentation on the use of this package has been written in addition.

Figure 1

 1 Figure 1.1 -Modèle graphique réduit associé au SBM.

  Dans la suite nous noterons ψ ∈ Ψ respectivement le paramètre d'échantillonnage et l'espace des paramètres associé. Nous supposerons tout le temps que les paramètres θ et ψ vivent dans un espace produit. On considérera aussi que les données précèdent toujours l'échantillonnage, c'est pourquoi nous nous intéressons à la loi de R sachant Y. Ainsi, en reprenant les travaux de D.B. Rubin nous dirons que les données manquantes sont (i) Missing Completely At Random (MCAR), si l'échantillonnage ne dépend ni des valeurs des données observées ni des valeurs des données non-observées. Mathématiquement cela revient à dire que R |= Y.

Figure 1 . 2 -

 12 Figure 1.2 -Graphes acycliques dirigés de dépendances conditionnelles entre Y, Z et R dans un cadre de données manquantes dans le SBM.

Figure 1 . 3 -

 13 Figure 1.3 -Graphes acycliques dirigés de dépendances conditionnelles entre Y, Z et R dans un cadre de données manquantes dans le SBM avec covariables.

Figure 2 . 1 -

 21 Figure 2.1 -DAGs of relationships between Y, Z and R in the framework of missing data for SBM. DAG where R is a parent node are not reviewed since the network exists before the sampling design acts upon it. The systematic edge between Z and Y is due to the definition of the SBM. Note that the DAG (b) may correspond to MAR or NMAR samplings.

  and NMAR otherwise. We derive Proposition 4 from these definitions. Proposition 4. If the sampling is MCAR or MAR then i) arg max θ p θ,ψ (Y o , R) = arg max θ p θ (Y o ) for any ψ such that p θ,ψ (Y o , R) = 0 and ii) the sampling design necessary satisfies DAG (a) or (b).

Figure 2 . 2 -

 22 Figure 2.2 -Estimation error of π and Adjusted Rand Index averaged over 500 simulations in the MAR setting.The adjacency matrix Y is generated under random-dyad sampling strategy for various connectivity c = q α q α π q .

Figure 2 .

 2 Figure 2.3 -Matrix π in different topologies with inter/intra block probabilities.

Figure 2 .Figure 2

 22 Figure 2.4 -Double standard setting: estimation error of π and adjusted Rand index averaged over 500 simulations for affiliation, bipartite and star topologies.

Figure 2 .

 2 Figure 2.6 -ARIs computed between the clusterings given by an SBM under class, star degree and MAR samplings with a varying number of blocks Q and ntora of farmers (lefthand-side) or dialect spoken by farmers (right-hand-side)

Figure 2

 2 Figure 2.7 -ICL criteria for SBMs with random-dyad MAR sampling and double-standard NMAR sampling in the thresholded ER network.

Figure 2 . 8 -

 28 Figure 2.8 -Estimation error of π and ARI averaged over 500 simulations in star degree and class settings. The topology is affiliation with = 0.05.

  (b) Matrix of connectivity π for NMAR inference (double standard) ; intensity of the color is proportional to the probability of connection between blocks. (c) ER PPI network reordered by blocks inferred with SBM with NMAR modeling. Left panel: original data with NA entries colored in gray (upper triangle) and data imputed with νij (lower triangle); right panel: zoom of blocks (1,2,3).

Figure 2

 2 Figure 2.9 -ER PPI network analysis with SBM under missing data conditions

Proposition 26 (

 26 contribution of local assignments). With the previous notations and C the positive constant defined in Proposition 21: sup θ∈Θ z∈S(z ,C) z z p(z, y o ; θ) = o P (p(z , y o ; θ )) 65

  o , z ; θ ) p(y o , z ; θ) uni-modal in θ, with a mode in θ M C . By consistency of θ M C , either p(y o , z ; θ) = o P (p(y o , z ; θ )) or p(y o , z ; θ) = O P (p(y o , z ; θ )) and θ → θ . In the latter case, any θ ∼ θ other than θ is bounded away from θ and thus p(y o , z ; θ ) = o P (p(y o , z ; θ )). In summary, θ ∼θ p(y o , z ; θ ) p(y o , z ; θ ) = max θ ∼θ p(y o , z ; θ ) p(y o , z ; θ ) (1 + o P (1))

. 7 )

 7 Proposition 35. Consider the maximization of the lower bound (3.7).

. 13 )

 13 Proposition 36. Consider the lower bound J τ ,θ (Y o |X) given by (3.13).

Figure 4 . 3 -

 43 Figure 4.3 -Matrix g -1 (γ) in different topologies with inter/intra block probabilities.

  Figure 4.4 -On the left side we estimate the prediction error of missing dyads where n m is the number of missing dyads and on the right side the ARI between the predicted clustering and the true one. In this case p = 0.

Figure 4 . 5 -

 45 Figure4.5 -On the left side we estimate the prediction error of missing dyads where n m is the number of missing dyads and on the right side the ARI between the predicted clustering and the true one. In these cases p = .2 and p = .5.

Figure 5 . 1 -

 51 Figure 5.1 -Schematic representation of an undirected network following the stochastic block model with 3 blocks. Colors are blocks in which nodes are dispatched with probabilities α and dyads distribution between nodes depends on colors of nodes with probabilities π.

Figure 5 . 2 -

 52 Figure 5.2 -Diagram of relation between classes of objects in missSBM, with three levels of inheritance.

  missSBM::simulate(nNodes, mixtureParam, connectParam, directed = FALSE, covariates = NULL, covarParam = NULL)

  missSBM::sample(adjacencyMatrix, sampling, parameters, clusters = NULL, covariates = NULL, similarity = l1_similarity, intercept = 0)

a

  list of models with class missSBM_fit ICL the vector of ICL associated to the models in the collection bestModel best model according to the ICL (a missSBM_fit object) optimizationStatus a data.frame summarizing the optimization process for all models

  Figure 5.4 -ICL criterion for block-sampling design with/without smoothing of the ICL.

  Figure 5.5 -ICL criterion curves for th block-node and node sampling designs.

Figure 5

 5 Figure 5.6 -Probability predicted by the SBM with block-node sampling

Figure 5 Figure 5 . 8 -

 558 Figure 5.7 -Alluvial plot between block-node sampling clustering and political parties.

  Mots Clefs : Réseaux, modèle à blocs stochastiques, données manquantes Résumé :Dans cette thèse nous nous intéressons à l'étude du modèle à bloc stochastique (SBM) en présence de données manquantes. Nous proposons une classication des données manquantes en deux catégories Missing At Random et Not Missing At Random pour les modèles à variables latentes suivant le modèle décrit par D. Rubin. De plus, nous nous sommes attachés à décrire plusieurs stratégies d'échantillonnages de réseau et leurs lois. L'inférence des modèles de SBM avec données manquantes est faite par l'intermédiaire d'une adaptation de l'algorithme EM : l'EM avec approximation variationnelle. L'identiabilité de plusieurs des SBM avec données manquantes a pu être démontrée ainsi que la consistance et la normalité asymptotique des estimateurs du maximum de vraisemblance et des estimateurs avec approximation variationnelle dans le cas où chaque dyade (paire de n÷uds) est échantillonnée indépendamment et avec même probabilité. Nous nous sommes aussi intéressés aux modèles de SBM avec covariables, à leurs inférence en présence de données manquantes et comment procéder quand les covariables ne sont pas disponibles pour conduire l'inférence. Finalement, toutes nos méthodes ont été implémentées dans un package R disponible sur le CRAN. Une documentation complète sur l'utilisation de ce package a été écrite en complément.

5

  Illustration: the 2007 French political blogosphere . . . . . . . . . . . . . . . 106 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 2 Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

	6 Conclusion et perspectives	115
	1
	Bibliography	119

  Très proche dans sa définition du LPCM, le LPM (voir Channarond, 2013, pour une définition et une étude théorique du modèle) modélise aussi la notion d'homophilie. Dans ce modèle, les variables latentes sont continues et appartiennent à l'espace R d . Soit f une densité par rapport à la mesure de Lebesgue sur R d , on considère le modèle suivant :

	2. ÉTAT DE L'ART	INTRODUCTION
	Latent Positions Model (LPM).

  1] une fonction isotrope (i.e. ne dépendant pas de la direction) et décroissante par rapport à la norme euclidienne. Ce modèle de graphe introduit par[START_REF] Lovász | Limits of dense graph sequences[END_REF] est très populaire car très général dans sa formulation. De plus il peut être vu comme une limite pour les graphes denses. Il est très activement étudié. On cite par exemple le livre de[START_REF] Lovász | Large Networks and Graph Limits[END_REF] qui est une référence incontournable concernant ce modèle.

Graphon (ou

W -graph). Un graphon est une fonction symétrique et mesurable f : (I 2 , B(I 2 )) → (I, B(I)) avec I = [0, 1]. Comme dans les modèles LPCM et LPM, les variables latentes sont continues :

Théorème de représentation par un modèle de Graphon Le

résultat suivant justifie l'étude des modèles à variables latentes et particulièrement du SBM. Il établit le modèle de graphon -qui est une version générale et continue d'un modèle à variable latente -comme une représentation possible de tous ces modèles.

Théorème 1.

[START_REF] Lovász | Limits of dense graph sequences[END_REF]

, Théorème 2.7) Pour toute suite de modèles (G n ) de graphes aléatoires de taille n vérifiant les conditions suivantes, il existe une fonction symétrique κ : [0, 1] 2 → [0, 1] telle que le modèle a la même loi que H n,κ :

(i) La loi de G n est invariante par permutation des numéros des noeuds (on dit que c'est un modèle de graphe échangeable).

  j) est échantillonnée au cours du processus d'échantillonnage et R ij = 0 dans le cas contraire. Pour les échantillonnages centrés sur les noeuds, nous utiliserons plus spécifiquement la variable V = (V i ) i∈N = (1 {i est échantillonné} ) i∈N indiquant quel noeud a été échantillonné.

  2. ÉTAT DE L'ART INTRODUCTION Y n'est pas une variable aléatoire, i.e. l'aléa est dû à l'échantillonnage seul. Dans ce second cas, les quantités d'intérêts sont des caractéristiques structurelles du réseau telles que le nombre d'arêtes, la moyenne des degrés des noeuds, la loi des scores de centralité associés aux noeuds ou encore des covariables associés aux noeuds. Dans Handcock and Gile (2010) les auteurs traitent du cas « model-based » pour des échantillonnages MAR dans le cadre des ERGM et du cas « design-based » pour les mêmes échantillonnages. Imaginons que l'on veuille estimer la somme de quantités d'intérêts associées aux noeuds d'un réseau (âge, taille, sexe, etc.) notée s = i∈N t i . Soit E = {i 1 , ...i n E } un échantillon de taille n E de l'ensemble des noeuds tel que pour chaque i dans E la quantité t i est observée.Supposons de plus que l'échantillonnage utilisé consiste en un tirage uniforme avec remise. Si désormais l'échantillonnage n'est pas un simple tirage uniforme avec remise, on se demande si l'estimateur ŝ de s est encore sans biais, si oui comment le débiaiser. La réponse est apportée dans[START_REF] Horvitz | A generalization of sampling without replacement from a finite universe[END_REF]. Supposons que pour tout noeud i dans N la probabilité qu'il soit dans E est donnée par µ i . Alors, l'estimateur d'Horvitz-Thompson de s est

	Cadre « design-based » : estimateurs d'Horvitz-Thompson
	Alors ŝ = n n E
	ŝµ =	i∈E	t i µ i	.

i∈E t i est un estimateur sans biais de s (i.e. E[ŝ] = s).

  Dans cette section, les données Y suivent un modèle probabiliste. Le but est d'inférer les paramètres de la loi de Y à partir d'un échantillon incomplet de Y. Suivant le travail de[START_REF] Handcock | statnet: Software tools for the representation, visualization, analysis and simulation of network data[END_REF], nous allons définir quelques stratégies d'échantillonnage Missing at Random de graphe et nous donnerons leurs lois. Rappelons que dans le cas de données manquantes MAR, l'inférence des paramètres de la loi de Y se fait sur la partie observée des données uniquement et sans biais induit par la stratégie d'échantillonnage. Nous supposerons par la suite les graphes symétriques et sans boucles, cependant tout ce qui suit se transpose directement au cas des graphes non-symétriques avec ou sans boucles.

	Cadre « model-based »
	Ego-centric design. Cet échantillonnage consiste à sélectionner indépendamment des
	noeuds du graphe avec probabilité ψ et d'observer toutes les valeurs des dyades entre les
	noeuds sélectionnés. Ainsi :

Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2 Statistical framework . . . . . . . . . . . . . . . . . . . . . . . . . 33

	2.1	Stochastic Block Model . . . . . . . . . . . . . . . . . . . . . . . 33
	2.2	Sampled data in the SBM framework . . . . . . . . . . . . . . . . 34
	2.3	Sampling design examples . . . . . . . . . . . . . . . . . . . . . . 35
	3

Variational Inference . . . . . . . . . . . . . . . . . . . . . . . . . 36

	3.1	MAR inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
	3.2	NMAR inference: the general case . . . . . . . . . . . . . . . . . 37
	3.3	NMAR: specificities related to the choice of the sampling . . . . 39
	4

Simulation study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

	4.1	MAR condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
	4.2	NMAR condition . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
	5

Importance of accouting for missing values in real networks . 45

	5.1	Seed exchange network in the region of Mount Kenya . . . . . . 45
	5.2	ER (ESR1) Protein-Protein Interaction network in breast cancer	46
	6

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 7 Supplementary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

	7.1	Proof of Proposition 4.ii) . . . . . . . . . . . . . . . . . . . . . . 49
	7.2	Identifiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
	7.3	Derivation of second lower bound in star degree sampling . . . . 51
	7.4	Proof of Proposition 10 . . . . . . . . . . . . . . . . . . . . . . . 51
	7.5	Simulations for star degree and class samplings . . . . . . . . . . 51
	7.6	Additional results for the ER Protein-Protein Interaction network 52

Table 2 .

 2 |ρ 1 | 1 -Performance of the ICL criterion: rates of correct answers when choosing the number of blocks and Adjusted Rand Indexes. Tested configurations are different sampling rates and three topologies (affiliation, bipartite and star) under a double standard sampling.

					0.25
	0.20				0.20
	0.15				0.15
	0.10				0.10
	0.05				0.05
	0.00				0.00
	-0.75 -0.5 -0.25	0	0.25	0.5	0.75	-0.75 -0.5 -0.25	0	0.25	0.5	0.75
					ρ 1 -ρ 0
	topology affiliation bipartite star	ε 0.05 0.15 0.25

Figure 2.5 -Double standard setting: estimation error of ρ 0 and ρ 1 averaged over 500 simulations for affiliation, bipartite and star topologies.

Table 2 .

 2 2 -Rates of correct answers of the ICL criterion when choosing between Randomdyad sampling (MAR) and double standard sampling (NMAR) in each of the 18 configurations. A configuration is the combination of a topology (affiliation, bipartite and star), a sampling rate and a sampling design. Each configuration is simulated 500 times.

	sampling rate sampling affiliation bipartite star
	(0.096, 0.367]	MAR	0.73	0.67	0.63
		NMAR	0.72	0.75	0.75
	(0.367, 0.638]	MAR	1	1	1
		NMAR	0.91	0.78	0.82
	(0.638, 0.909]	MAR	1	1	1
		NMAR	0.91	0.8	0.95

  1 . Choose z ∈ Z 1 and a sequence t n decreasing to 0 but satisfying ρnt n / log(n) → +∞. According to Proposition 24,

		sup
		θ∈Θ z / ∈S(z ,tn)
	And Proposition 27 allows us to conclude
	p(y o ; θ) p(y o ; θ )	=	# Sym(θ) # Sym(θ )	θ ∼θ max	p(y

p(z, y o ; θ) = o P (p(z , y o ; θ )) Since t n decreases to 0, it gets smaller than C (used in proposition 26) for n large enough. As this point, Proposition 26 ensures that: sup θ∈Θ z∈S(z ,tn) z z p(z, y o ; θ) = o P (p(z , y o ; θ )) And therefore the observed likelihood ratio reduces as: p(y o ; θ) p(y o ; θ ) = z∼z p(y o , z; θ) + z z p(y o , z; θ) z∼z p(y o , z; θ ) + z z p(y o , z; θ ) = z∼z p(y o , z; θ) + p(y o ; z , θ )o P (1) z∼z p(y o , z; θ ) + p(y o ; z , θ )o P (1) o , z ; θ ) p(y o , z ; θ ) (1 + o P (1)) + o P (1).

with covariates and sampling models 86 3 Statistical inference

  Inference of Model 1 . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.2 Inference of Model 2 . . . . . . . . . . . . . . . . . . . . . . . . . 89 3.3 Inference of Model 3 . . . . . . . . . . . . . . . . . . . . . . . . . 90 3.4 Model selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

		Chapter 4 SBM with covariates and
		missing values
	Contents
	1	Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
	2	Stochastic Block Models

. . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.1 4 Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 4.1 Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 4.2 Example 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

  The network topology is affiliation, which means that diagonal parameters of the connectivity matrix π are greater than extra-diagonal parameters. Indeed we choose

			.2 .05 .05	
	π =		.05 .2 .05	 and α = (1/4, 1/2, 1/4).	(4.1)
			.05 .05 .2

  From (2.3), if the sampling is MAR or MCAR then maximizing p θ,ψ (Y o , R) or p θ (Y o ) in θ is equivalent.

	2. STATISTICAL FRAMEWORK
	Proposition 38.

  The hierarchical structure and inheritance relations are represented in Figure5.2.

	sampledNet	missSBM fit		Inheritance
				Has fields
				Levels
				Model
				Realization
	SBM	networkSampling	Estimation Data
	SBM sampler	networkSampling sampler
		SBM fit	networkSampling fit
	SBM fit nocovariates	SBM fit covariates

  .1.

	Argument Description	Correspondence	Lives in
	n	number of nodes	n	N
	alpha	mixture parameters	α	[0, 1] Q
	pi	matrix of inter and intra clusters probabilities	π	MQ([0, 1])
	directed	whether the network is directed or not		{T, F}
	covariates list of covariates	(Xi)i∈N , (Xij)ij	( )
	covarParam regression parameters of the covariates	β	R m
	Table 5.1 -Arguments of simulate and their counterparts in the SBM. ( ) is a list with
	N entries (the N covariates corresponding to nodes) or a list of m n × nmatrices (covariates
	corresponding to dyads).

  draw Y re-ordered by blocks if type="network" and ZYZ if type="connectivity"

	Field	Description	Correspondence
	adjacencyMatrix	adjacency matrix of the simulated network	Y
	blocks	matrix of blocks memberships	Z
	memberships	vector of blocks memberships	(which.max(τ i )) i∈N
	Method	Description
	plot(object, type)

Table 5 .

 5 2 -Selection of important fields and methods in class SBM_sampler with their mathematical counterparts

Table 5 .

 5 3 describes the arguments of sample and their mathematical counterparts.

	Argument	Description	Correspondence	Lives in
	adjacencyMatrix adjacency matrix the network	Y	Mn({0, 1})
	sampling	name of the sampling design		( )
	parameters	sampling parameter(s)	ψ	Ψ
	clusters	vector of blocks memberships	(which.max(τ i))i∈N	Q n
	covariates	list of covariates	(Xi)i∈N , (Xij)ij	( )
	similarity	similarity function for covariates on dyads

  Table 5.4 -Selection of important fields and methods in class sampledNetwork.

	Field	Description	Correspondence
	adjacencyMatrix sampled adjacency matrix with NA at missing entries	Y R
	covarMatrix	the matrix of covariates (if applicable)	X
	covarArray	the array of covariates (if applicable)	(X ij ) ij
	missingDyads	the set of dyades not sampled in the network	Y m
	observedDyads	the set of dyades sampled in the network	Y o
	observedNodes	vector of sampled nodes	V
	samplingRate	rate of observed dyades	mean(R)
	samplingMatrix	sampling matrix	R
	Method	Description
	plot(object)	draw the adjacency matrix Y and the sampling matrix R

Table 5 .

 5 5 provides a basic description of the arguments of estimate.

	Argument	Description

Table 5 .

 5

5 -Arguments of sample with short descriptions and types.

Table 5 .

 5 6 -Structure of missSBM_collection (output of estimate).

  Table 5.8 -Selection of fields in fittedSBM and fittedSampling with their mathematical counterparts.

	Field	Description		R6 object / class
	fittedSBM	the adjusted Stochastic Block Model	SBM_fit_(no)covariates
	fittedSampling the estimated sampling process	networkSampling_fit
	sampledNet	the sampled network data	sampledNetwork
	imputednetwork the adjacency matrix with imputed values	matrix
	monitoring	status of the optimization process	data.frame
	vICL	ICL criterion associated to Q	double
	R6 object	Field	Description	Correspondence
		blocks	estimated probability of block belonging	{τ i } i∈N
		connectParam connectivity matrix		π
	fittedSBM	connectProb covarParam	imputed adjacency matrix regression parameter	Y o ∪ {ν ij } (i,j)∈D m β
		memberships	vector of blocks memberships	(which.max(τ i )) i∈N
		mixtureParam mixture parameters		α
	fittedSampling	parameters	sampling parameter		ψ

vBound value of the variational bound J τ ,θ at each step double vExpec value of E pτ [log(p θ (Y, Z))] at each step double penalty penalty of the model with Q blocks double

Table

5

.7 -Selection of fields in object missSBM_fit with descriptions and types.

We finally give additional details on fields fittedSBM and fittedSampling in Table

5

.8.

  For illustrative purpose, we sample the blog network to mimic missing data and create a new adjacency matrix with missing entries. Since the data are interactions between blogs (who references who), it is natural to sample the network with a node-centered sampling design: the following code generates a realization of a 196 × 196 sampling matrix according to the block-node sampling, where blocks corresponds to the clustering estimated by SBM on the full data set and where sampling rate is either low (0.2) or high (0.8) in each block.

	5. ILLUSTRATION: THE 2007 FRENCH POLITICAL BLOGOSPHERE
	## * Useful fields (most are special object themselves)
	##	$fittedSBM (the adjusted stochastic block model)
	##	$fittedSampling (the estimated sampling process)
	##	$sampledNetwork (the sampled network data)
	##	$imputedNetwork (the adjacency matrix with imputed values)
	##	$monitoring, $vICL, $vBound, $vExpec, $penalty
	Sampling the network data. cl0 <-sbm_full$bestModel$fittedSBM$memberships
	samplingParameters <-base::sample(
	x		= c(0.2, 0.8),
	size	= sbm_full$bestModel$fittedSBM$nBlocks,
	replace = TRUE)
	sampledNet <-sample(adjacencyMatrix, "block-node", samplingParameters, cl0)

Motivations : des données manquantes dans les réseauxLes réseaux sont un moyen naturel de représenter les interactions entre des individus ou plus généralement des entités (protéines, espèces, sites web, etc.). On les rencontre dans de nombreux domaines comme en biologie, en sociologie, en écologie, en industrie, ou Internet. Il existe en statistique de nombreux modèles de réseaux -dont certains seront présentés plus loin -et techniques permettant d'analyser des réseaux(Kolaczyk

, 2009 etMatias and Robin, 2014). L'estimation de ces modèles n'est pas en général un problème facile.Une difficulté supplémentaire souvent rencontrée par le statisticien est de n'avoir accès qu'à un échantillon incomplet du graphe à étudier. En effet, pour des raisons simples telles que le manque de temps, le coût ou la malchance, un observateur peut être conduit à collecter des données incomplètes. On est donc amené à étudier un réseau partiellement observé tout en voulant connaître les propriétés du réseau complet sous-jacent. Dans la suite nous allons motiver notre travail avec deux exemples, le premier concernant des données d'ethnobiologie avec un réseau d'échanges de semences et le second des données génomiques avec un réseau de co-régulation de gènes (ou réseau protéine-protéine).Le premier exemple que nous étudions est celui d'un réseau d'échanges de semences entre agriculteurs de la région du Mont Kenya. Nous remercions chaleureusement Vanesse Labeyrie d'avoir partagé ces données (voir[START_REF] Labeyrie | Influence of ethnolinguistic diversity on the sorghum genetic patterns in subsistence farming systems in eastern kenya[END_REF][START_REF] Labeyrie | Seed exchange networks, ethnicity, and sorghum diversity[END_REF] et pris du temps pour en discuter. L'étude de ce réseau a pour but de mieux comprendre l'impact des échanges sur la diversité génétique cultivée. Des études cherchant à lier la diversité génétique cultivée et la diversité culturelle humaine ont montré que les échanges de semences sont intimement liés à l'organisation sociale entre les agriculteurs. C'est un exemple de résultat auquel nous souhaiterions apporter une confirmation (ou non) d'ordre statistique. Ces données sont échantillonnées suivant un processus noeud-centré au sens où certains agriculteurs ont été interrogés pour renseigner à qui ils donnaient et de qui ils recevaient des semences. Ainsi, 155 agriculteurs ont été interrogés, renseignant leurs liens avec un total de 777 agriculteurs (dont les 155 interrogés). Puis, en accord avec les ethnologues, pour simplifier l'étude, nous avons réduit le réseau à 568 des 777 agriculteurs. Pour les 413 non interrogés du réseau

PerspectivesAu cours de cette thèse, nous avons abordé la question des données manquantes dans le SBM. Cependant de nombreuses questions que nous n'avons pas eu le temps d'aborder restent en suspens et mériteraient de l'attention.

Remerciements 8 Acknowledgment

The authors thank Pierre Barbillon (INRA-MIA, AgroParisTech), Julien chiquet (INRA-MIA, AgroParisTech), Stéphane Robin (INRA-MIA, AgroParisTech) and James Ridgway (CFM) for their helpful remarks and suggestions. This work is supported by two public grants overseen by the French National research Agency (ANR): first as part of the « Investissement d'Avenir » program, through the « IDI 2017 » project funded by the IDEX Paris-Saclay, ANR-11-IDEX-0003-02, and second by the « EcoNet » project.

Acknowledgments

The authors thank all members of MIRES group for fruitful discussions on network sampling designs. This work is supported by public grants overseen by the French National research Agency (ANR) as part of the "Investissement d'Avenir" program, through the "IDI 2017" project funded by the IDEX Paris-Saclay, ANR-11-IDEX-0003-02, and second by the "EcoNet" project, ANR-18-CE02-0010.

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ?

? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ?

? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? ? ? ?

? ?

Figure 5.3 -Adjacency matrix of the French blogosphere reordered according to political party; left: fully observed; right: sampled according to a block-node sampling design. Blue color range corresponds to edges between nodes from the same cluster; red color range between node from different clusters; gray color correspond to missing dyads.

Standard SBM estimation. At this stage, the data set has no missing entries: every dyads and nodes are observed. The adjacency matrix Y of the fully-observed network is adjacencyMatrix. We can first perform a standard SBM estimation on the fully observed network. To do so we set first the algorithm parameters that will be common to all analyzes:

## Algorithm parameters : ## vBlocks <-1:15 control <-list(cores = 10, trace = 0, iterates = 2) smoothing_type <-"both"

We then proceed to the estimation of the fully-observed network, including smoothing of the ICL.