oo

  Groupe conjoint OGC -Organisation Météorologique Mondiale (OMM / WMO)  Cible : les standards pour les données de l'eau => WaterML 2.0 suite : https://www.ogc.org/standard/waterml/  Organisant des tests d'interopérabilité (Interoperability Experiments -IEs) focalisés sur les différents sous-domaines de l'eau  Point d'entrée https://external.ogc.org/twiki_public/HydrologyDWG/WebHome  Chairs o Tony Boston (Australia) o Silvano Pecora (Italy) -WMO Liaison o David Blodgett (USA) Iterative Development http://www.opengeospatial.org/projects/groups/hydrologydwg Dynamique du groupe  Un long historique d'activités conjointes o -Earth Systems Science Domain Working Group (DWG) o -Hydrology DWG o -Groundwater Interoperability Experiment (IE) o -Water Information Services Concept Development Study o -Surface Water Interoperability IE o -Hydrology Forecasting IE o -Climate-Hydrology Information Sharing Pilot o -GroundWater IE2 o -Water ML 2.0 Standards Working Group (SWG) o -Hydrographic Features SWG o -Research Data Alliance Global Water Information IG (Hydro DWG sister group) o -Groundwater SWG o -Geoscience DWG o -Environmental Linked Features IE (ELFIE) o -Borehole IE o -Second ELFIE (SELFIE) o -Water Quality IE => en cours Dynamique du groupe  Communauté o des représentants de part le monde o des membres de différents types o établissements publics o organismes publics de recherche o organisations internationales o entreprises privées  Quelques exemples o NR-Can, GSC, USGS, US EPA, CUASHI, SDSC, BRGM, UK CEH, DELTARES, GRDC, BaFG, Univ Tartu, Fraunhofer IOSB, DataCove, NIWA/LAWA (NZ), Federation University (Australia), BoM Australia, CSIRO, … o WMO, UNESCO, … o Kisters, 52°N, Aquatic Informatics, … Dynamique du groupe  Standards WaterML2.0 Repris dans le système d'information de l'OMM/WMO o Dans ceux de nombreux organismes : UNESCO, USGS, US EPA, NrCan, NIWA, BRGM, etc… o Dans des outils opensource : CUASHI Hydro-Server, Kisters, 52°N etc… o Contribution régulière de projets impliquant les partenaires 6 https://portal.ogc.org/files/?artifact_id=57222 Adoption de WaterML2: Part 1 Annonce concernant la stratégie américaine pour l'usage de WaterML2.0 pour l'observation civile de la terre (2013) o http://www.opengeospatial.org/pressroom/pressreleases/1831  L'US Federal Geographic Data Committee (FGDC) approuve WaterML2: Part 1 (2014) o http://www.fgdc.gov/standards/news/WaterML  WaterML2: Part 1 recommandé par la legislation Européenne sur l'échange de données (2013) => INSPIRE o http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:331:0001:0267:EN:PDF  L'USGS l'implémente dans son système d'information sur l'eau (2014) o http://help.waterdata.usgs.gov/news/april-10-2014  Le service météorologique australien (BoM) implémente WaterML2: Part 1 via son Water Data Online (2017) o http://www.bom.gov.au/waterdata/  La WMO/OMM l'implémente en natif dans son système d'information. WaterML2: Part 2 -Ratings, Gaugings and Sections  Standardisation des courbes de tarage, des conversions et des sections de rivières (cross sections)  https://docs.ogc.org/is/15-018r2/15-018r2.html WaterML2: Part 3 -Surface Hydrology Features  Modèle conceptuel décrivant les objets hydrographiques de surface : rivière, lac, bassin versant et réseau de drainage  https://docs.ogc.org/is/14-111r6/14-111r6.html Modèles conceptuel et logique décrivant les objets hydrogéologiques et les observations associées  https://docs.ogc.org/is/16-032r2/16-032r2.html • WaterML-WQan O&M and WaterML 2.0 profile for water quality data  https://docs.ogc.org/bp/14-003/14-003.html  Périmètre o Spécialisation d'O&M et de WaterML2 -Part 1 pour les séries temporelles de concentrations chimiques o Début de spécification d'un vocabulaire 'Qualité des eaux' basé sur QUDT, ChEBI, NVS P01, S06 I-ADOPT  En cours de mise à jour par le test d'interopérabilité « Water Quality » WaterML2: Part 5 -Water Quality Best Practice  OGC Water Quality Interoperability Experiment : https://github.com/opengeospatial/WaterQualityIE o Co-Lead : BRGM, Internet of Water o Membres: USGS, US EPA, WMO, UNESCO, BfG, BRGM, OFB -Pôle INSIDE, University of Tartu, Australia & NZ participants o Contexte o Partage d'observations de qualité des eaux (de surface ou souterraine) entre organisations o Types de données : capteurs in-situ, mesures ponctuelles, échantillons (et sous-échantillons), analyses au laboratoire, observation de biodiversité et les vocabulaires contrôlés associas o Colonne vertébrale issue de "l'OGC Baseline" o Sémantique: "Observations, Measurements & Samples" o API : SensorThings API, OGC API -Features La Dordogne -Février 2006 La Dordogne -Pont de Lalinde -Août 2005 OneWater Data / Water4All  Deux projets de plateforme Interopérable sur l'Eau  o 53 millions €, 10 ans, programme de recherche français o https://www.onewater.fr/fr (ANR project : 22-PEXO-0009) o Pour contribuer à accélérer la transition et mesurer l'impact des changements mondiaux sur les socio-hydro systèmes à travers 6 challenges scientifiques. Un étant : l'échange de données eau FAIR.  o https://www.water4all-partnership.eu/ o 31 pays, 7 ans, 86 m€ pour la premières phase o To tackle water challenges as means to face climate change, help to achieve the United Nations' Sustainable Development Goals and boost the EU's competitiveness and growth (EU Green Deal). One being FAIR Water Data Exchange OneWater Data / Water4All  Les éléments partagés o Mettre en place des FAIR Water Data Platforms  Basées sur les pratiques OGC, W3C, INSPIRE, pratiques RDA  Autant d'aspects d'ingénierie de l'échange de données eau (Re, l'implementation des standards et leur evolution) que d'aspects dynamique de communauté (conduite du changement, formation, …) o Même période d'activité o Des projets à long terme : ~ 9 et 7 ans o Démarrage en ce moment même o Typologie d'acteur : mélange d'organismes publiques et d'organismes de recherche o Financement: les deux ont un financement 'décent' pour faire le travail : sous réserve de bien nous synchroniser entre les projets.  Synergies avec le groupe OGC Hydro DWG o Les deux projets sont dans la phase "A. Stakeholders needs and interaction" (expression des besoins utilisateurs) Pour identifier les Use Case métiers sur lesquels partir o Première liste (Mai dernier) Synergies déjà identifiées o Water Quality IE o WMO hydrology expert groups discussion o Et des échanges directs avec certains membres de l'HydroDWG => Comment peut on mieux échanger pour faire évoluer les standards et bonnes pratiques ? o Ajout de "sang-neuf" dans l'HydroDWG issu de ces deux projets o En complément de discussions bilatérales pour faire entrer ces partenaires dans la dynamique OGC/FAIR o Intégration des résultats du Water Quality IE dans la production des projets OneWater Data et Water4All  Implémentations chez des partenaires non historiquement dans l'HydroDWG  Formation chez ces partenaires à ces pratiques techniques et de groupes OGC

Merci

s.grellet@brgm.fr


Albane Saintenoy

Sheila Bonde

Clark Maines

Detecting remains of a centrally-planned kitchen at the Abbey of Tiron (Thiron-Gardais, Eure-et-Loire) with Ground-Penetrating Radar

The possibility of the remains of a centrally-planned, medieval kitchen at the abbey of the Holy Trinity of Tiron (Thiron-Gardais, Eure-et-Loire, France), led us to conduct a Ground-Penetrating Radar survey in July 2018. The joint interpretation of 2D profiles and 3D temporal slices using Python scripts has highlighted the remains of three walls of a hexagonal building at the location indicated on the 1651 plan drawn by the Maurist architect, dom Hilaire Pinet. The geophysical prospection also made it possible to detect the passage of a water conduit, with a change of direction possibly linked to the presence of latrines, that would have been part of the monastery's complex water management system.

Introduction

Substantial remains of the twelfth-century church are all that survive above ground of the reformist Benedictine abbey of the Holy Trinity of Tiron, located in the village Thiron-Gardais (Eure-et-Loir, France). Pinet's measured site plan shows a large hexagonal building, labelled by Pinet "ancienne cuisine", projecting off the northern claustral range. Only about a dozen centrally-planned, medieval monastic kitchens are known in France, the most famous one being at the abbey of Fontevraud, a marvel of Romanesque architecture.

Ground-Penetrating Radar (GPR) has been extensively used for mapping archeological structures [START_REF] Saintenoy | Ground Penetrating Radar prospection at the charterhouse of Bourgfontaine[END_REF]Trinks at al., 2018). Confirmation of Pinet's drawing, and better understanding of the history of centrally-planned kitchens in western France, led us to conduct a Ground-Penetrating Radar survey in July 2018.

Method

A Malå CUII system with 250 MHz antennas was used over three zones positioned according to Pinet's drawing (Fig. 1): zone 1 over the suspected kitchen remains, zone 2 over the suspected latrines remains and zone 3 over the chapter room. Each zone is surveyed by parallel profiles acquired every 0.5 m with a 5 cm distance step between each trace.

Each profile is processed using the Reflexw Software (Sandmeier, 2012) using DEWOW, time zero setting to first amplitude peak, Stolt migration (Stolt, 1978) using a velocity of 0.1 m/ns estimated from hyperbola analysis (Sham and Lai, 2016), and AGC gain. In all profiles, the penetration depth is exceptionally good, giving imaging possibilities down to 3 m.

In each zone, profiles were merged into a single 3D data cube. A python script using the libraries numpy, matplotlib and ipywidget was written to slice the data cube in the twoway traveltime (TWT) direction and to visualize map of the summed absolute amplitudes between two TWT.

Results

In zone 1, profiles were acquired from a West toward East direction, first profile on the south side of the zone. Fig. 2 shows the profile acquired 21 m north of profile 1. A strong reflection is visible about 25 m from the beginning of the profile. Looking at the map obtained from adding amplitude in between 13 and 20 ns (Fig. 3) similar reflections align NNW and can be interpreted as coming from a large pipe. The time slice obtained between 33 and 43 ns (Fig. 4) shows an interesting arrangment as part of an hexagonal shape. Its placement let us to believe that we were seeing reflections of the centrally-planned kitchen foundations. Similarly, combined interpretation of profiles and time slices of the 3D data cubes acquired in zones 2 and 3 can be summarized in Fig. 5 and6. A large pipe (dashed blue line) continued from zone 1 to zone 2. A smaller one exists only in zone 1. A pair of pipes was detected in zone 2. The change in angle is best explained by the presumed presence of latrines, also indicated on Pinet's plan. We think that the limits of the chapter room were detected in zone 3.

Conclusions

The GPR data survey at the site of the Abbey of Tiron shows exceptional penetration depth (about 3 m) especially using an old GPR Mala system as we did. For this work we developed some scripts using python3 that greatly aided the visualisation and interpretation of GPR data cubes. We detected several pipes and more interestingly, three sides of an hexagonal shape at the position of the centrally planned kitchen given by the Pinet's 1651 plan. This discovery will encourage further archaeological and architectural research at the site.

Figure 1 :

 1 Figure 1: Aerial photography showing the three prospected zone, north of the church.

Figure 2 :

 2 Figure 2: Profil 43 acquired in zone 1. Red dashed lines delimit the time section shown in Fig. 3. Black dashed lines delimit the time section in Fig. 4.

Figure 3 :

 3 Figure 3: Map of summed absolute amplitude of each trace in between 13 and 20 ns.

Figure 6 :

 6 Figure 6: Results of GPR prospections with interpretations Figure 5: Results of GPR prospections

  Trinks, I., Hinterleitner, A., Neubauer, W., Nau, E., Löcker, K., Wallner, M., Gabler, M., Filzwieser, R., Wilding, J., Schiel, H. and Jansa, V., 2018.

	Sandmeier, K.J., 2012. REFLEXW Version 7.0, Program
	for the processing of seismic, acoustic or
	electromagnetic reflection, refraction and
	transmission data. User's Manual, p.578.
	Sham, J.F. and Lai, W.W., 2016. Development of a new
	algorithm for accurate estimation of GPR's wave
	propagation velocity by common-offset survey
	method. NDT & E International, 83, 104-113.
	Stolt, R.H., 1978. Migration by Fourier transform.
	Geophysics, 43 (1), 23-48.
	Large area high resolution ground penetrating -area hi -area hi -area hi
	radar	measurements	for	archaeological
	prospection. Archaeological Prospection, 25(3),
	171-195.

Acknowledgments

We are pleased to thank Romain Sarian and Marine Elias for their efficient help with the data acquisitions.


domain: stat info

text:
Rémi Boutin
email: remi.boutin.stat@gmail.com

Pierre Latouche

Charles Bouveyron

The Deep Latent Position Topic Model for Clustering and Representation of Networks with Textual Edges

Numerical interactions leading to users sharing textual content published by others are naturally represented by a network where the individuals are associated with the nodes and the exchanged texts with the edges. To understand those heterogeneous and complex data structures, clustering nodes into homogeneous groups as well as rendering a comprehensible visualisation of the data is mandatory. To address both issues, we introduce Deep-LPTM, a model-based clustering strategy relying on a variational graph auto-encoder approach as well as a probabilistic model to characterise the topics of discussion. Deep-LPTM allows to build a joint representation of the nodes and of the edges in two embeddings spaces. The parameters are inferred using a variational inference algorithm. We also introduce IC2L, a model selection criterion specifically designed to choose models with relevant clustering and visualisation properties. An extensive benchmark study on synthetic data is provided. In particular, we find that Deep-LPTM better recovers the partitions of the nodes than the state-of-the art ETSBM and STBM. Eventually, the emails of the Enron company are analysed and visualisations of the results are presented, with meaningful highlights of the graph structure.

Topic Modelling

Like network analysis, the field of topic modelling was first developed using heuristics and was only studied from a statistical perspective at the end of the 20th century. [START_REF] Papadimitriou | Latent Semantic Indexing: A Probabilistic Analysis[END_REF] provided statistical foundations for the latent semantic index (LSI) and [START_REF] Hofmann | Probabilistic latent semantic analysis[END_REF] introduced a mixture model to represent the vocabulary distribution where each component modelled a specific topic. Ultimately, the latent Dirichlet allocation model (LDA) was proposed by [START_REF] Blei | Latent dirichlet allocation[END_REF]. They assumed that the topic proportions of the documents are sampled from a Dirichlet distribution. Many extensions of LDA have been proposed, including the use of a normal-logistic prior to better model the correlations between topics [START_REF] Blei | Correlated topic models[END_REF]. In the end, the gap between deep generative modelling and topic models was bridged by [START_REF] Srivastava | Autoencoding Variational Inference For Topic Models[END_REF] who introduced a variational distribution parametrised by a DNN. More recently, the embedded topic model (ETM, [START_REF] Dieng | Topic modeling in embedding spaces[END_REF]). allowed to use embeddings in order to represent both the words and the topics in the same vector space. They are used as part of the decoder and can be pre-trained on large datasets to incorporate semantic meaning, as illustrated in the original paper with examples relying on continuous bag of words [START_REF] Mikolov | Efficient estimation of word representations in vector space[END_REF].

Joint analysis of network and texts

While both topic modelling and statistical network analysis gave rise to many publications over the last 20 years, only few works have combined the two approaches. The community-user topic model (CUT, [START_REF] Zhou | Probabilistic models for discovering e-communities[END_REF]) added a latent variable to the author-topic model (AT, [START_REF] Rosen-Zvi | The Author-Topic Model for Authors and Documents[END_REF] to represent the communities either as a set of co-authors or as a set of topics. Thereafter, the community-authorrecipient-topic model (CART, [START_REF] Pathak | Social topic models for community extraction[END_REF]) used communities both at the document generation level and at the author and recipient generation level which corresponds to the network generation. However, the high number of parameters combined with the inference based on a Gibbs sampler does not allow to scale those models to large datasets. The topic-link LDA, presented in [START_REF] Liu | Topic-link LDA: joint models of topic and author community[END_REF], also offers a joint-analysis of texts and links in a unified framework by conditioning the generation of a link on both the topics within the documents and the community of authors. The inference relies on a variational EM algorithm to scale the approach to large datasets. However, this method only deal with undirected networks. Finally, the topic-user-community models (TUCM) was introduced in [START_REF] Sachan | Using content and interactions for discovering communities in social networks[END_REF] and was able to discover topic-meaningful communities. The main feature of this model was its capacity to incorporate different types of interactions, well-suited for social networks applications. The inference relied on Gibbs sampling approach which can be limiting when dealing with large datasets. More recently, the stochastic topic block model (STBM) presented in [START_REF] Bouveyron | The stochastic topic block model for the clustering of vertices in networks with textual edges[END_REF] was the first model to handle the simultaneous clustering of nodes and edges while keeping the inference tractable to large datasets thanks to a variational classification EM based inference. This model was extended in [START_REF] Bergé | The latent topic block model for the co-clustering of textual interaction data[END_REF] for the simultaneous clustering of bipartite networks with textual edges. It was also adapted for dynamic networks in [START_REF] Corneli | The dynamic stochastic topic block model for dynamic networks with textual edges[END_REF]. Recently, [START_REF] Boutin | Embedded topics in the stochastic block model[END_REF] introduced the embedded topic in the stochastic block model (ETSBM) to incorporate embeddings and semantic meanings into the word representations. However, despite their modelling qualities, all these approaches do not allow for the construction of a meaningful representation of the data which characterises both the connections and the documents exchanged. Only Deep-LPTM is able to provide node positions incorporating information about the network structure as well as the documents content. The four figures were obtained with the gplot function from the sna library [START_REF] Pavel | Statnet: Tools for the Statistical Modeling of Network Data[END_REF]. While the first network is plotted manually to highlight the generative structure, the two networks in the middle are based on the Fruchterman-Reingold algorithm while the fourth graph uses the node positions estimated by Deep-LPTM. Contrary to former methods STBM and ETSBM, which require the use of an exterior method, our methodology provides an end-to-end method tailored for representing networks with clustering properties.

Main contributions and organisation of the paper

The model proposed in the present paper is the first to simultaneously cluster the nodes of a network, uncover the topics in the texts exchanged between the nodes and to output a representation of both the topics and the edges in a Euclidean space. To this aim, (i) we propose a generative model assuming that each node and each edge is represented in a latent space by a mixture of Gaussians. By doing so, we incorporate the clustering in the generative process such that each mixture component models either a cluster of nodes or the documents exchanged between a pair of clusters of nodes. Moreover, our model distinguishes itself from former methods by allowing each node or edge to be represented by a latent position and not only by the group it belongs to. This is illustrated in Figure 1, where the first graph depicts the true node clusters and main topics of the corresponding documents of a simulated dataset.The second graph gives, for this dataset, the node clusters provided by SBM and the topics estimated by LDA. SBM does not retrieve the true node partition and does not provide node positions to apprehend the results. An external algorithm, namely the Fruchterman-Reingold algorithm [START_REF] Fruchterman | Graph drawing by force-directed placement[END_REF], considering the presence of connections only in the network, had to be used for graph representation. The third graph is based on state-of-the-art ETSBM and does not recover the node partition either. Moreover, ETSBM is not able to render a comprehensive representation of the full network. Again, as for SBM, the Fruchterman-Reingold algorithm had to be used for graph representation Finally, the figure on the right-hand side presents the Deep-LPTM results. As opposed to the previous methods, Deep-LPTM is able to gather the information about the network structure as well as the exchanged documents into the node positions while finding the true node partition and topics. A representation of the graph is directly obtained by the estimation procedure such that no external graph representation algorithm is needed. As we shall see, the node positions are computed by considering both the connections and the content of the corresponding documents. (ii) We derive a two-stage variational expectation-maximisation (VEM) algorithm for estimating the model parameters as well as the posterior parameters of the latent positions. The first stage relies on analytical formulas to update the cluster probabilities as well as the mixture parameters. The second stage uses a stochastic gradient descent algorithm to update the expected lower bound with respect to the VGAE parameters and the deep topic model parameters. In particular, the deep topic model can make the best out of pre-trained embeddings. Thus, introducing semantic meaning into the word representations is possible as well as learning the representation from scratch. (iii) In order to choose the relevant numbers of clusters and latent space dimensions, we introduce the integrated classification and latent likelihood (IC2L) for model selection.

It extends the integrated classification likelihood criterion, which was conceived for mixture models, to account for the latent representations of the nodes and of the edges. The criterion relevance is strongly upheld by the evaluation on synthetic data as well as the provided real-word use case. Moreover, by selecting a low dimensions regarding the node embedding space, IC2L praises for models with a strong and direct capacity of representation.

We close this section with the organisation of the paper. In Section 2, we present the assumptions concerning the generation of the data. The inference as well as the model selection criterion are presented in Section 3. In Section 4, Deep-LPTM and the impact of the initialisation are evaluated on synthetic data. An extensive benchmark study against state-of-the-art methods is also provided. Eventually, the emails of the Enron company are analysed with Deep-LPTM in Section 5. The results as well as the visualisations are presented to illustrate the ease of interpretation of the model outcomes.

Notations

In this paper, we are interested in data represented by a graph G := {V, E} where V = {1, . . . , N} denotes the set of vertices.

The set E denotes the edges between the nodes with M = |E| the number of edges. We focus on binary a adjacency matrix A ∈ M N×N ({0, 1}) such that A ij equals 1 if (i, j) ∈ E, and 0 otherwise. The graph is assumed to be directed and without any self loop. Therefore A ii = 0 for all i ∈ V. Finally, Q denotes the number of clusters of nodes. Each edge in the graph represents a textual document sent from one node to another. An edge from node i to node j exists or equivalently (i, j) ∈ E , if and only if node i sent a textual document to node j, denoted W ij . We use a bag-of-word representation of the texts where

W ij = W 1 ij , . . . , W V ij ∈ N V
denotes the vector of word occurrences in the document between nodes i and j such that W v ij is the number of times word v appears in the document,

M ij = V v=1 W v ij
is the total number of words in document W ij and V the size of the vocabulary. Hence, A ij = 1 if W ij exists. The set of documents will be denoted W := (W ij ) (i,j)∈E and the number of topics is denoted by K. Eventually, the simplex of dimension d will be denoted ∆ d-1 .

MODEL

In the following, the assumptions about the graph generation as well as the hypothesis concerning the documents construction are presented.

Graph generation

Assuming that the number of clusters Q is fixed before hand, each node i is assumed to belong to a cluster, represented by the cluster membership variable C i . The variables C i , for any i ∈ V, are assumed to be independent and identically distributed (i.i.d) according to a multinomial distribution such that for any node i ∈ {1, . . . , N}:

C i ∼ M Q (1, π), (1)
with π ∈ ∆ Q-1 and C i ∈ {0, 1} Q being one hot encoded so that C iq = 1 if node i belongs to cluster q and C iq = 0 otherwise. Thus, denoting C = (C 1 , . . . , C N ) T ∈ M N×Q ({0, 1}) the cluster membership matrix, we have:

p(C | π) = N i=1 Q q=1 π Ciq q . ( 2
)
Moreover, given its cluster membership, the node i is assumed to be represented by a Gaussian vector Z i in a p dimensional latent space such that:

Z i | C iq = 1 ∼ N µ q , σ 2 q I p . (3)
Eventually, the connection between two nodes is assumed to depend on the closeness of the node representations in the latent space. Therefore, denoting η ij := κ -∥Z i -Z j ∥, which accounts for the opposite of the distance between the node latent representations, the probability for node i to be connected to node j is:

A ij | Z i , Z j , κ ∼ Ber 1 1 + e -ηij , (4)
where a logistic function is used as a link function. For the sake of brevity, we will denote p ij = (1 + e -ηij ) -1 . The probability of the entire adjacency matrix is given by:

p(A | Z, κ) = i̸ =j p Aij ij (1 -p ij ) 1-Aij .
Finally, the joint-distribution of the adjacency matrix, the latent node vectors, as well as the cluster memberships can be factorised as follow:

p(A, Z, C | κ, µ, σ, π) = p(A | Z, κ)p(Z | C, µ, σ)p(C | π). (5)
where µ = (µ q ) q and σ = (σ q ) q . It is worth noticing that the model described in Equations ( 1), ( 3) and (4) corresponds to the latent position cluster model [START_REF] Handcock | Model-based clustering for social networks[END_REF]. The fundamental difference with our approach for this part of the model will arise in the inference, as discussed in Section 3.

Generation of the texts on the edges

At the core of our approach is the motivation to be able to use textual data to obtain more homogeneous and meaningful clusters.

To begin with, we make the assumption that each edge can be represented in a latent space by a Gaussian vector, depending only on the node cluster memberships. Thus, given (C i ) i∈V , the latent variables Y ij are assumed to be i.i.d such that:

Y ij | A ij C iq C jr = 1 ∼ N (m qr , s 2 qr I K ), ∀(i, j) ∈ E, (6)
where m qr ∈ R K , s qr ∈ R + . Moreover, we assume that the topic proportions of the document W ij , denoted θ ij , can be deduced from the latent variables such that:

Ỹij = softmax(Y ij ), (7)
where softmax(x) =

F f =1 e xf -1
(e x1 , . . . , e xF ) ⊤ if x ∈ R F . Hence, assuming that the documents are i.i.d given their corresponding topic proportions, we have for any edge (i, j) ∈ E:

W ij | A ij = 1, θ ij ∼ M V M ij , β ⊤ θ ij , (8)
where

β k = softmax(ρ ⊤ α k ) ∈ R V , β = (β 1 . . . β K ) ⊤ ∈ M K×V (R), ρ ∈ M L×V (R), α k ∈ R L and α = (α 1 . . . α K ) ∈ M L×K (R).
Thus, denoting m = (m qr ) qr , s = (s qr ) qr , the joint likelihood of the texts, the latent representation of the documents, as well as the clusters memberships can be computed as:

p(W, Y | A, C, ρ, α, m, s) = p(W | A, Y, ρ, α)p(Y | A, C, m, s), (9)
where m = (m qr ) 1≤q,r≤Q and s = (s qr ) 1≤q,r Q . Figure 2 gives a graphical representation of the generative assumptions presented above. We omitted the parameters for the sake of clarity but the full version can be found in Appendix A1. Interestingly enough, those assumptions can be linked to existing models as we shall see in the next section.

Link with other models

On the one hand, if the topic modelling alone is considered, restricting all topic proportions to be equal for all (i, j) such that C iq C jr = 1 corresponds to the text modelling in ETSBM [START_REF] Boutin | Embedded topics in the stochastic block model[END_REF]. In that sense, Deep-LPTM increases the freedom of each edge representation compared to ETSBM. Additionally, Deep-LPM corresponds to Deep-LPTM when the textual data present on the edges are disposed of (or LPCM if no GCN-based encoder is used in the inference strategy). Accordingly, Deep-LPTM prolongs Deep-LPM and LPCM to networks with textual data. On the other hand, discarding the information provided by the graph and the clustering of the nodes would correspond to ETM applied on the observed documents. Therefore, Deep-LPTM extends ETM to texts with a connectivity structure to improve the topic modelling.

W ij A ij Observed data Z i Z j Y ij
Latent representation in a vector space

C j C i Clusters (i, j) F I G U R E 2
Graphical representation of the model without the parameters for the sake of clarity.

INFERENCE

In the next section, the inference of the model is presented as well as the model selection criterion.

Likelihood

In this work, we consider the marginal likelihood of the network and the texts for parameter estimation. The latent variables are denoted by

C = (C i ) N i=1 , Z = (Z i ) N i=1 and Y = (Y ij ) (i,j)∈E
, and the set of parameters is Θ = {π, µ, σ, κ, m, s, α, ρ} .Thus, the marginal log-likelihood is given by:

L(Θ; A, W) = log p(A, W | Θ) = log C Z Y p(A, W, C, Z, Y | Θ)dZdY . (10)
Unfortunately, this quantity is not tractable since the sum over C requires to compute Q N terms. Besides, it involves integrals that cannot be computed analytically. Therefore, we choose to rely on a variational inference approach for approximation purposes.

Decomposition of the marginal log-likelihood

For any distribution R(C, Z, Y), the following decomposition holds:

L(Θ; A, W) = L (R(•); Θ) + KL(R(•)||p(C, Z, Y | A, W)), (11)
where

L (R(•); Θ) = E R log p(A, W, C, Z, Y | Θ) R(C, Z, Y) . ( 12
)
Since the Kullback-Leibler divergence is always positive in Equation [START_REF] Zhang | Network representation learning: A survey[END_REF], the ELBO L (R(•); Θ) is a lower bound of the marginal log-likelihood. Moreover, the closer R(•) is to the posterior distribution of the latent variables, in terms of Kullback-Leibler divergence, the closer the ELBO is to the marginal log-likelihood. Since the marginal log-likelihood does not depend on R(•), maximizing the ELBO with respect to R(•) is equivalent to minimizing the Kullback-Leibler divergence between R(•) and the posterior distribution. To make the ELBO tractable, we restrict the family of variational distributions by considering a mean field assumption. Moreover, the assumptions in Equations ( 14) to ( 16) allow us to take full advantage of the SGD efficiency to optimise the variational distribution while keeping a high flexibility thanks to the neural network parametrisations of the distributions (see, section 2.5 and 2.6 of [START_REF] Kingma | An introduction to variational autoencoders[END_REF]). The assumptions are given by:

R(C, Z, Y | A, W) = R(C)R(Z | A)R(Y | A, W), (13)
R(C) = N i=1 R τi (C i ) = N i=1 M Q (C i ; 1, τ i ), (14)
R(Z | A) = N i=1 R ϕZ (Z i | A) = N i=1 N (Z i ; µ ϕZ (A) i , σ 2 ϕZ (A) i I P ), (15)
R(Y | A, W) = i̸ =j R ϕY (Y ij | W ij ) Aij = i̸ =j N Y ij ; µ ϕY (W ij ), diag σ 2 ϕY (W ij ) Aij , ( 16
)
where τ = (τ i ) N i=1 with ∀i ∈ {1, . . . , N}, τ i ∈ ∆ Q-1 .
Moreover, in Equation 15, the mapping

µ ϕZ : M N×N (R) → M N×P (R) (σ 2 ϕZ : M N×N (R) → (R + ) N respectively)
is the mapping normalising the adjacency matrix by its degree, D -1/2 AD -1/2 , and encoding the normalised adjacency matrix into the approximated posterior means (standard deviations) of the node latent positions. The diagonal matrix D is filled with D ii , the degree of node i, for all nodes. The two mappings µ ϕZ and σ 2 ϕZ rely on a GCN parametrised by ϕ Z [START_REF] Kipf | Variational graph auto-encoders[END_REF]. Similarly, in Equation 16,

µ ϕY : M M×V (R) → M M×K (R) (σ 2 ϕY : M M×V (R) → M M×K (R +
) respectively) encodes the documents into the approximated posterior means (standard deviations) of their corresponding latent vectors. However, the two functions rely on the ETM encoder with parameter ϕ Y [START_REF] Dieng | Topic modeling in embedding spaces[END_REF]. In practice, in all the experiments we carried out, we used a feed-forward neural network to encode the documents, with three layers and 800 units on each layer. The first two layers are shared to encode the variances and the means while the last layer is specific to each vector. Regarding the encoder of the adjacency matrix, we rely on [START_REF] Kipf | Variational graph auto-encoders[END_REF] such that,

µ ϕZ (A) = à ReLU( ÃW 0 )W µ and log σ 2 ϕZ (A) = à ReLU( ÃW 0 )W σ , where à = D -1/2 AD -1/2 and ReLU(x) = (max(0, x 1 ), . . . , max(0, x F )) if x ∈ R F . µ ϕZ (•) and log σ 2
ϕZ (•) share the first-layer parameter W 0 ∈ M N×D with D = 10 in all the experiments we carried out, and W µ , W σ ∈ M D×P . For the sake of brevity, we will take the exponential of the encoder of the log variance and consider σ 2 ϕZ (•). Thus, the ELBO can be decomposed as follow :

L (R(•); Θ) = E R log p(A | Z, κ) + E R log p(W | A, Y, ρ, α) + E R log p(C | π) + E R log p(Z | C, µ, σ) + E R log p(Y | A, C, m, s) -E R log R(C) -E R log R(Z | A) -E R log R(Y | A, W) . ( 17
)
The computation of each term in Equation ( 17) is detailed in Appendix B. To optimise the ELBO, we propose here to alternate between closed form updates and stochastic gradient descent steps thanks to the results presented in the next section.

Optimisation

Analytical updates

Given a variational distribution R(•) complying with Equations ( 13) to ( 16), the model parameters can be updated using Propositions ( 1) and ( 2).

Proposition 1. Let R(•) be a variational distribution complying with Equations (13) to [START_REF] Mehta | Stochastic blockmodels meet graph neural networks[END_REF]. The parameters of the nodes embeddings distributions maximising the ELBO are given by:

µ q = 1 N q N i=1 τ iq µ ϕZ (A) i , (18)
σ 2 q = 1 pN q N i=1 τ iq pσ 2 ϕZ (A) i + ∥µ ϕZ (A) i -µ q ∥ 2 2 , ( 19
)
where N q = N i=1 τ iq is the posterior mean of the number of nodes in cluster q. The proof is given in Appendix C.1.0.3. Interestingly, this proposition states that the µ q are the weighted mean of the (approximated) posterior mean nodes positions µ ϕZ provided by the DNN. It also indicates that the σ q are updated as the sum of two terms: the first one corresponds to a weighted mean of the posterior variances while the second one is the intra cluster variance weighted by the posterior clusters membership probabilities τ i .

Proposition 2. For a given variational distribution R(•) complying with Equations (13) to [START_REF] Mehta | Stochastic blockmodels meet graph neural networks[END_REF], with parameters τ , µ ϕY , σ ϕY , the parameters of the edge embeddings distributions maximising the ELBO are given by:

m qr = 1 N qr N i,j=1 A ij τ iq τ jr µ ϕY (W ij ), (20)
s 2 qr = 1 KN qr N i,j=1 A ij τ iq τ jr K k=1 σ 2 ϕY (W ij ) k + ∥µ ϕY (W ij ) -m qr ∥ 2 2 , ( 21
)
where N qr = N i,j=1 A ij τ iq τ jr denotes the expected number of documents sent from cluster q to cluster r under the approximated posterior distribution.

The proof is given in Appendix C.1.0.4 The interpretation following Proposition 1 can also be applied to Proposition 2 while a second DNN is used. The main difference lies in the weighting that here corresponds to the probability that the pair of nodes composing each edge belong to a pair of cluster. For instance, for the edge (i, j), the posterior probability that node i belongs to cluster q and node j to cluster r is given by τ iq τ jr .

3.2.2

The stochastic gradient descent

The other model parameters κ, ρ and α, and variational parameters ϕ Z and ϕ Y cannot be updated with analytical formulas because of the integral involving the variational distribution R(•) in the ELBO. In the following section, we aim at deriving estimates of the gradients of the ELBO with respect to these parameters, to perform stochastic gradient descent.

Model parameters

The partial derivatives of the ELBO with respect to each parameter κ, ρ and α are obtained thanks to Monte-Carlo estimates. For instance, the gradient of the ELBO with respect to κ is estimated by:

∂ ∂κ L (R(•); Θ) = ∂ ∂κ E R log p(A | Z, κ) = E R ∂ ∂κ log p(A | Z, κ) ≈ 1 S S s=1 ∂ ∂κ log p(A | Z (s) , κ),
where

Z (s) = (Z (s) 1 , . . . , Z (s) N ) and Z (s) i i.i.d ∼ N (µ ϕZ (A) i , σ 2 ϕZ (A) i I p ).
The same computations result in estimates for the partial derivatives of the ELBO with respect to ρ and α. In practice, we rely on the common practice in the field of VAE and set S = 1.

Variational parameters

The last parameters to update are the variational parameters ϕ Y and ϕ Z . Ideally, we would like to use the same computation as in the previous section. For instance, we would like to compute the following partial derivates of the ELBO with respect to ϕ Z :

∂ ∂ϕ Z L (R(•); Θ) = ∂ ∂ϕ Z E R log p(A | Z, κ) + log p(Z | C, µ, σ) -log R(Z) = ∂ ∂ϕ Z E R log p(A | Z, κ)] (22)
-

N i=1 Q q=1 τ iq ∂ ∂ϕ Z KL Z iq µ ϕZ (A) i , σ ϕZ (A) i , µ q , σ q , ( 23
)
with

KL Z iq µ ϕZ (A) i , σ ϕZ (A) i , µ q , σ q = log σ p q σ ϕZ (A) p i - p 2 + pσ 2 ϕZ (A) i + ∥µ ϕZ (A) i -µ q ∥ 2 2 2σ 2 q .
See Appendix B1 for the computational details. Unfortunately, the expectation of the term on the left-hand side of Equation ( 22), is taken with respect to the variational distribution which depends on ϕ Z . Therefore, the derivation of this quantity is not straightforward. Thanks to [START_REF] Kingma | Auto-Encoding Variational Bayes[END_REF] and [START_REF] Rezende | Stochastic backpropagation and approximate inference in deep generative models[END_REF], this difficulty can be tackled by using the reparametrisation trick. In particular, let ε i be a centred, normalised and P-dimensional Gaussian vector. Hence, the vectors Z i and µ ϕZ (A) i ⊕ σ ϕZ (A) i ⊙ ε i have the same distribution. Therefore, the expectation can be taken with respect to ϵ = (ε i ) i=1...,N which gives:

∂ ∂ϕ Z E R log p(A | Z, κ)] = ∂ ∂ϕ Z E ε log p A | µ ϕZ (A) ⊕ σ ϕZ (A) ⊙ ϵ , κ where µ ϕZ (A) ⊕ σ ϕZ (A) ⊙ ϵ = µ ϕZ (A) i ⊕ σ ϕZ (A) i ⊙ ε i i=1,...,N
, ⊙ denotes the Hadamard product and ⊕ the element-wise sum. A Monte-Carlo estimate of this quantity is derived by sampling S centred and reduced P-dimensional Gaussian vectors ϵ (s) i with s = 1, . . . , S, i = 1, . . . , N and with ϵ (s) = (ε (s) i ) i=1,...,N . Plugging it back into [START_REF] Blei | Correlated topic models[END_REF] gives the following estimate:

∂ ∂ϕ Z L (R(•); Θ) ≈ 1 S S s=1 ∂ ∂ϕ Z log p A | µ ϕZ (A) ⊕ σ ϕZ (A) ⊙ ϵ (s) , κ - N i=1 Q q=1 τ iq ∂ ∂ϕ Z KL Z iq µ ϕZ (A) i , σ 2 ϕZ (A) i , µ q , σ 2 q .
The same derivation steps lead to a similar estimate for the partial derivatives of the ELBO with respect to ϕ Y . Thanks to the low variances of the gradients estimated with the reparametrisation trick and to avoid increasing the computations, we use a sample size S = 1, as advised in the VAE literature. In addition, the computation of the partial derivatives is implemented with Pytorch automatic differentiation framework [START_REF] Paszke | PyTorch: An Imperative Style, High-Performance Deep Learning Library[END_REF] to take fully advantage of the computational efficiency of GPUs. Moreover, we rely on the Adam optimiser [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] to carry out the stochastic gradient descent with a learning rate of 0.002 (0.005 respectively) for the optimiser of κ and ϕ Z (ϕ Y respectively).

Model selection

To complete the inference, we present IC2L, a new model selection criterion accounting for both the clustering partition as well as the latent representations. In all previous sections, we considered the number of clusters Q, the number of topics K and the dimension of the node latent space P fixed before hand. In this section, we aim at selecting the triplet (K, P, Q) that captures the most information out of the data without over-parametrisation. The integrated complete (or classification) likelihood [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF], denoted ICL, was introduced for mixture models and is now a common model selection criterion in this context. First, considering a mixture model M , with observed data X, latent cluster memberships C, Q clusters and distribution parameters in the set Θ, the complete likelihood refers to p(X, C | M , Q, Θ). This quantity depends on both the clustering and the model parameters. Then, to account for the uncertainty over the set of parameters and to penalise the model complexity, [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF] proposed to integrate over Θ and to evaluate the quantity log p(X, C | M , Q) = θ∈Θ log p(X, C | M , Q, θ)dθ. Since the integral is not tractable for many statistical models, the authors relied on a BIC-like approximation of this quantity. In the present paper, we propose to extend ICL to include the evaluation of the node embeddings Z, as well as the edge embeddings Y. Denoting M the model presented in Sections 2.1 and 2.2, we are interested in:

log p(A, W, Z, Y, C | M , Q, K, P) = log θ p(A, W, Z, Y, C | θ, M , Q, K, P)p(θ)dθ. ( 24
)
Since this quantity is not tractable, we derive an estimate in the following proposition.

Proposition 3. Let us consider a model M , as described in Section 2, with Q denoting the number of clusters, K the number of topics and P the dimension of the node latent space. In addition, let us assume that the prior of the model parameters fully factorises, as p(κ, π, µ, σ, m, s, ρ, α) = p(κ)p(π)p(µ)p(σ)p(m)p(s)p(ρ)p(α). Then, the IC2L(M , Q, K, P, Ẑ, Ŷ, Ĉ) criterion, denoted IC2L, is given by:

IC2L = max θ log p(A, W, Ẑ, Ŷ, Ĉ | θ, M , Q, K, P) -Ω(M , Q, K, P) = max κ log p(A | Ẑ, κ, M ) - 1 2 log(N(N -1)) + max µ,σ log p( Ẑ | Ĉ, µ, σ, M , Q, P) - QP + Q 2 log(N) + max ρ,α log p(W | A, Ŷ, ρ, α, M ) - VL + KL 2 log(M) + max m,s log p( Ŷ | A, Ĉ, m, s, M , K) - Q 2 K + Q 2 2 log(M) + max π log p( Ĉ | π, M , Q) - Q -1 2 log(N), (25)
with Ẑ, Ŷ and, Ĉ the maximum-a-posteriori estimates, and

Ω(M , Q, K, P) = 1 2 log(N(N -1)) + Q(P + 2) -1 2 log(N) + L(V + K) + Q 2 (K + 1) 2 log(M), (26)
where each term in Equation (26) corresponds to the penalisation of the BIC-like approximation of the probabilities, as detailed in Equation [START_REF] Mikolov | Efficient estimation of word representations in vector space[END_REF].

Proof. See Appendix C.2.
Ultimately, the relevance of this criterion as well as the parameters estimation are assessed in the next section on synthetic data. Moreover, an extensive comparison with baseline methods is provided.

NUMERICAL EXPERIMENTS

This section is dedicated to the assessment of the proposed methodology. We start with an introductory example to illustrate the results obtained with Deep-LPTM. We continue with an evaluation of the initialisation impact on our method. Then, we move on to Section 4.4 to provide numerical evidence of the robustness of IC2L against the dimensions of the parameter spaces. We close this section with a benchmark study to compare Deep-LPTM with the state-of-the-art ETSBM and STBM. ‡

Simulation settings

To begin with, we introduce three simulation scenarios to be used for evaluating the methodology on different conditions detailed hereinafter.

Scenarios

• Scenario A is constituted of three communities, each defining a cluster, and four topics. By definition, a community is a group of nodes more densely connected together than with the rest of the network. For each cluster, a specific topic is employed to sample the documents associated with the intra-cluster connections. Besides, an extra topic is employed to model documents ‡ Our code is available at https://plmlab.math.cnrs.fr/rboutin/deeplptm_package.

Scenario A Scenario B Scenario C

F I G U R E 3 Networks sampled from each scenario. The node colours denote the node cluster memberships and the edge colours denote the majority topic in the corresponding documents.

sent between nodes from different clusters. Hence, by construction, the clustering structure can be retrieved either using the network or the texts only. • Scenario B is made of a single community and three topics. Consequently, all nodes connect with the same probability. Then, the nodes are spread into two clusters using distinct topics. An extra topic is used to model documents exchanged between the two clusters. Accordingly, the network itself is not sufficient to find the two clusters but the documents are. • Scenario C comprises three communities and three topics. Two of the communities are associated with their respective topics, say t 1 and t 2 . Furthermore, following Scenario B, the third community is split in two clusters, one being associated with topic t 1 and the other with t 2 . Thus, considering both texts and topology, each network is actually made up of four node clusters. Consequently, both textual data and the network are necessary to uncover the clusters. This scenario will be of major interest in this experiment section since it ensures that the two sources of information are correctly used to uncover the node partition.

For all scenarios, networks with 100 nodes are sampled and the edges holding the documents are constructed by sampling words from four BBC articles, focusing each on a given topic. The first topic deals with the UK monarchy, the second with cancer treatments, the third with the political landscape in the UK and the last topic deals with astronomy. In the general setting, for all scenarios, the average text length for the documents is set to 150 words. The parameters used to sample data from the three scenarios are given in Table 1. Moreover, three examples of networks generated from A, B and C are presented in Figure 3. To summarise, the three proposed scenarios inspect different facets of the model. Scenario A insures that the model rightfully uses the network structure, Scenario B focuses on the usage of the topics to recover the node partition. Finally, Scenario C combines the two scenarios to guarantee that both sources of information are correctly utilised simultaneously.

Scenario A Scenario B Scenario C Q (clusters) 3 2 4 K (topics) 4 3 3 Communities 3 1 3
πqr (connection probabilities) ψ = 0.25, ϵ = 0.01   ψ ϵ ϵ ϵ ψ ϵ ϵ ϵ ψ   ψ ψ ψ ψ     ψ ϵ ϵ ϵ ϵ ψ ϵ ϵ ϵ ϵ ψ ψ ϵ ϵ ψ ψ    
Topics matrix T between pairs of clusters (q, r)

  t 1 t 4 t 4 t 4 t 2 t 4 t 4 t 4 t 3   t 1 t 3 t 3 t 2     t 1 t 3 t 3 t 3 t 3 t 2 t 3 t 3 t 3 t 3 t 1 t 3 t 3 t 3 t 3 t 2    
T A B L E 1 Detail of the three simulation scenarios used to evaluate our model.

Clustering performance evaluation

The adjusted rand index (ARI) is used as a measure of the closeness between two partitions. In this paper, ARI compares the true node labels with the node partition provided by a model. In particular, obtaining an ARI of 0 suggests that the clustering is as close to the true node labels as a random cluster assignment of the nodes. On the contrary, the closer the ARI is to 1, the better the results are. Ultimately, an ARI of 1 signifies that the true partition was perfectly recovered (up to a label permutation).

Level of difficulty

In order to generate more situations from the three scenarios, we introduce the Hard difficulty to test the model robustness against two aspects. First, we want to test the model against documents using several topics. Thus, in the Hard difficulty, the documents are formed of multiple topics such that, for any edge (i, j) with node i in cluster q and node j in cluster r, the topic proportions are computed as a ratio between the pure topic proportions Ỹ⋆ qr ∈ {0, 1} K , with zeros everywhere except at the coordinate corresponding to the true topic, and between the uniform distribution over the topics. This combination is controlled by a parameter ζ such that ζ = 0 corresponds to a pure topic case while ζ = 1 leads to a uniform distribution over the topics. This translates into:

Ỹqr = (1 -ζ) Ỹ⋆ qr + ζ * 1 K , . . . , 1 K ⊤ , (27)
with ζ = 0.7 in the Hard setting. The second aspect tested by the Hard setting is the robustness in the presence of less connected communities. Consequently, the intra-cluster connection probability is decreased from ψ = 0.25 in the classical setting to ψ = 0.1 in the Hard one.

Main features of Deep-LPTM

This section gives an overview of the main features of Deep-LPTM on one network simulated according to Scenario A, with an intra cluster connection probability ψ equal to 0.15, an inter-cluster connection probability ϵ fixed to 0.05 and the parameter controlling the topic proportions ζ set to 0.5. In addition, ϕ Y , ρ, α, the parameters referring to the topic modelling, are pre-trained for only 5 epochs with ETM alone. Conversely, the parameters ϕ Z , κ, µ, σ, related to network modelling, are randomly initialised without pre-training to illustrate the evolution of the node embeddings during the optimisation. In the rest of the paper, those parameters will be pre-trained by running ETM and Deep-LPM independently before hand. On the one hand, the evolution of the ELBO as well as the ARI of the nodes and the edges are presented in Figure 4. The node ARI and the edge ARI increase to reach an ARI of 1 following the evolution of the ELBO. We only display the first 200 epochs for the sake of clarity, but the entire training is provided in the appendix.

On the other hand, Figure 5 features the evolution of the node latent positions during the training. Interestingly enough, Deep-LPTM finds a meaningful representation of the network even with a random initialisation. This difficult problem requires to train the model longer when no initialisation is provided. Hence, the ELBO continues to increase as displayed in Figure D2, presented in the appendix. In the rest of the paper, the GCN parameters as well as the topic model parameters are initialised before hand. In addition, the node cluster memberships probabilities are initialised with a similarity-based method between the topic proportions Ỹ of the node neighbours [START_REF] Bouveyron | The stochastic topic block model for the clustering of vertices in networks with textual edges[END_REF]. F I G U R E 6 The meta-network is a representation of the network at the cluster level based on Deep-LPTM estimates. Each cluster is represented by a node, the node sizes depend on the number of nodes assigned to each cluster, the node positions as well as the major topics between two connected clusters (denoted by the colours of the edges) are estimated by the parameters (µ q ) q as well as (m qr ) qr respectively and the sizes of the edges depend on the number of connections between two clusters. The meta-network, represented in Figure 6, describes the connectivity at the cluster level. The node sizes depend on the number of nodes assigned to each cluster q and are given by N i=1 Ĉiq , with Ĉiq = 1 if argmax τ i = q and 0 otherwise. The cluster positions are estimated by (µ q ) q and the major topics (denoted by the colours of the edges on the figure) between two clusters is estimated by the argmax m qr for all pairs of clusters (q, r). Finally, the sizes of the edges depend on the number of connections between clusters, given by N i,j=1 Ĉiq Ĉjr for all pairs of clusters (q, r).

ScenarioA

ScenarioB ScenarioC Node ARI Node ARI Node ARI Easy Random init 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 Dissimilarity init 0.97 ± 0.06 1.00 ± 0.00 0.98 ± 0.03 Deep-LPTM random 1.00 ± 0.00 1.00 ± 0.01 0.63 ± 0.20 Deep-LPTM dissim 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 Deep-LPTM -PT random 1.00 ± 0.00 0.90 ± 0.30 0.55 ± 0.15 Deep-LPTM -PT dissim 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 Hard Random init 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 Dissimilarity init 0.31 ± 0.14 1.00 ± 0.00 0.38 ± 0.24 Deep-LPTM random 0.80 ± 0.21 0.95 ± 0.05 0.47 ± 0.02 Deep-LPTM dissim 0.99 ± 0.02 1.00 ± 0.00 0.89 ± 0.15 Deep-LPTM -PT random 0.95 ± 0.05 0.73 ± 0.30 0.45 ± 0.04 Deep-LPTM -PT dissim 1.00 ± 0.01 1.00 ± 0.00 0.85 ± 0.18

T A B L E 3 Adjusted rand index (ARI) of the initialisations and the results of Deep-LPTM in terms of node clustering, without and with pre-trained embeddings (denoted PT in that case). ARI is averaged over 10 graphs, for each scenario and difficulty.

Eventually, Table 2 presents the topics obtained by Deep-LPTM. They are both very interpretable and distinguishable one from another which is crucial to understand complicated datasets. This will be stressed in the analysis of Enron email dataset in Section 5.

Impact of the initialisation and the pre-trained embeddings

This section aims at evaluating the improvement of our method upon the initialisation, with a warm start and without. In this regard, Table 3 presents the ARI of a random initialisation as well as a dissimilarity initialisation [START_REF] Bouveyron | The stochastic topic block model for the clustering of vertices in networks with textual edges[END_REF] ,denoted random and dissimilarity respectively, in Table 3. The initialisation alone (without any model name preceding it in the table) as well as the model with the initialisations (with the model preceding the initialisation) are provided. Moreover, the Deep-LPTM node clustering is evaluated with and without pre-trained skipgram embeddings [START_REF] Mikolov | Efficient estimation of word representations in vector space[END_REF], denoted PT in the table. The results are obtained by averaging the ARI over 10 graphs for each scenario and difficulty and can be summarised in three points.

First, in all cases where the initialisation has not already reached an ARI of 1, Deep-LPTM improves the node clustering, even in difficult settings with no warm-start. For instance, in Scenario A with the Hard setting, the model starts from an ARI of 0.31, with the dissimilarity initialisation, to reach 0.99 and 1.00 without and with pre-trained embeddings respectively.

Second, the improvement provided by the pre-trained embeddings depends on the scenario. On the one hand, Scenario A benefits from the usage of pre-trained embeddings which always improves the results. For instance, in the Hard setting with a random initialisation, the ARI increases from 0.80 ± 21 to 0.95 ± 0.05. On the other hand, Scenario B and C always favour the results without pre-trained embeddings. As an example, with a random initialisation in the Hard setting, the ARI decreases from 0.95 ± 0.05 to 0.73 ± 0.30 and 0.47 ± 0.02 to 0.45 ± 0.04 in Scenario B and C respectively. The same deduction can be made with the dissimilarity initialisation. Since Scenario B and C are the ones evaluating the text contribution to the clustering, we advise not to use pre-trained embeddings.

Finally, the best ARI are all obtained with the dissimilarity initialisation. Consequently, Deep-LPTM will only be initialised with it in the rest of the paper.

Model selection

In order to assess the model selection criterion, this section provides two experiments. First, we evaluate IC2L relevancy to select Q on all three scenarios. Second, we test IC2L efficiency to select the triplet (K, P, Q) on Scenario C specifically.

4.4.0.1

Selection of Q with P = 2 and the true K Keeping P set to 2 and K fixed to its true value for the moment, Table 4 assesses the effectiveness of IC2L to select the number of clusters Q. In all three scenarios, IC2L selects the true model 10 times out of 10.

Scenario A Scenario B Scenario C Q ⋆ = 3 Q ⋆ = 2 Q ⋆ = 4 Q = 2 0 10 0 Q = 3 10 0 0 Q = 4 0 0 10 Q = 5 0 0 0 Q = 10 0 0 0 T A B L E 4
Number of times a value Q is selected by the IC2L criterion over 10 graphs with the true value of K and P = 2. Let us now consider selecting the triplet (K, P, Q) simultaneously. Table 5 displays the number of times a triplet is selected by IC2L for 10 graphs simulated according to Scenario C (with Q ⋆ = 4 and K ⋆ = 3 the true values). The selected node embedding dimension is always P = 2, thus, we only provide K and Q in the table. First, it is satisfactory that IC2L always selects the true number of topics and clusters. Second, by always picking the dimension P = 2, IC2L favours models with a lower complexity. In our case, this translates into choosing models able to directly visualise the data in two dimensions, simply by plotting the latent vectors Z i . This suits our purpose of building an explainable model.

K = 2 K = 3 K = 4 K = 5 K = 6 Q = 2 0 0 0 0 0 Q = 3 0 0 0 0 0 Q = 4 0 10 0 0 0 Q = 5 0 0 0 0 0 Q = 6 0 0 0 0 0 T A B L E 5

Benchmark

ScenarioA

F I G U R E 7
Deep-LPTM representation of Enron email network. The nodes positions, the node cluster memberships (denoted by the colour of the nodes) as well as the majority topic in the documents (denoted by the colour of the edges) are estimated by Deep-LPTM.

We close this section with a benchmark study presented in Table 6 to compare Deep-LPTM with state-of-the-art ETSBM and STBM. We also provide SBM and Deep-LPM as baselines even though they are not able to take into account the text edges. As in the previous sections, the table presents the average of the ARI over 10 graphs. Each graph result is obtained by running each method with five different initialisations and by taking the one resulting in the highest ELBO. The table is presented for three different models, namely STBM, ETSBM and Deep-LPTM. The last two models are evaluated with and without pre-trained embedding. In all cases, Deep-LPTM is either as good as or better than other models. In particular, in Scenario C with difficulty Hard, the ARI of Deep-LPTM node clustering is higher than all other methods, by at least 0.15. Likewise, in Scenario A with difficulty Hard, Deep-LPTM always recover the true partition while STBM only reaches an ARI of 0.66 ± 0.18.

APPLICATION TO THE ANALYSIS OF THE ENRON EMAIL NETWORK

Enron, formed in 1985, was an American company selling natural gas in North America. In 2001, the securities and exchange commission (SEC) opened an investigation on October, 31th, for fraud, while on August, 14th, the company was "probably in the strongest and best shape that it [had] probably ever been in" according to its CEO. On early December the same year, the company filed for the largest bankruptcy at that time. We propose here to concentrate on the critical period September, 1st to December, 31th leading to the downfall of the company in order to understand the organisation of the company during this crucial period. Thanks to the decision of the federal energy regulatory commission (FERC), the dataset is publicly accessible, and contains all 20,940 emails exchanged between 149 employees. All edges holding multiple messages were coerced into a single meta-message by stacking the documents together. As a result, the network holds 1,234 edges between the 149 employees. The dataset can be found at https://www.cs.cmu.edu/~./enron/. The estimation of Deep-LPTM is conducted for all triplets (Q, K, P) where Q ∈ {5, 7, 10}, K ∈ {3, 5, 7, 10} and P ∈ {2, 4, 8, 16}. The highest value of IC2L corresponds to the triplet (Q, K, P) = (7, 10, 2). Deep-LPTM clustering results are displayed in Figure 7. The node cluster memberships as well as the edge majority topics are represented by their respective colours. In addition, the topics are interpreted by looking at their corresponding most probable words in Figure 8. Interestingly, IC2L selected a low-dimensional node latent space, which fits with the observations made in [START_REF] Hoff | Latent space approaches to social network analysis[END_REF].

Topics analysis

The topics can be depicted as follow:

• Topic 1 concerns Charles Watson, Dynegy CEO at the time, that negotiated a deal to finance Enron, involving the transwestern pipeline, and to merge the companies • Topic 2 refers to regional energy • Topic 3 deals with business operations • Topic 4 is related to office supplies and day-to-day work • Topic 5 mentions the energy usage and delivery • Topic 6 is related to legal and strategical aspects of Enron business, involving Sara Shakleton (vice president of Enron North America Corporation), and Debra Perlingiere, from the legal department • Topic 7 is concerned with infrastructures and geographical projects • Topic 8 corresponds to a discussions about Enron activities in Afghanistan, which may be seen as sensitive given the american situation in 2001 • Topic 9 focuses on financial aspects • Topic 10 mentions the California electricity crisis, which almost led to the bankruptcy of the Southern California Edison corporation

The topics as well as the visualisation provide significant information on the dataset. In particular, Deep-LPTM identifies different departments and cases of the company through the topics and successfully represent it in the graph structure as we shall detail. The colours of the edges refer to the majority topic between the respective clusters and the sizes of the edges depend on the number of connections between the clusters. We keep only the edges corresponding to more than five connections for readability purposes.

Cluster analysis

The meta-network in Figure 9 gives an overview of the results to analyse the clusters. We keep only the edges corresponding to more than five connections for readability purposes. However, important details provided by Figure 7 could not be uncovered with the meta-network only. First, Clusters 1 and 2 are well separated from the rest of the graph. Moreover, each one is characterised by a specific topic. Cluster 1 is involved in discussions about a financial deal and a possible merge (Topic 1) and more than half of the employees in Cluster 1 have a managerial position or work in the legal department (vice presidents, directors, lawyer). Cluster 2 refers to the regional energy business (Topic 2) and nine out of the 15 nodes correspond to either employees or a traders.

Second, Cluster 6 displays a high level of internal connectivity and is essentially featured by discussions involving the legal department (Topic 6) as well as financial aspects of the company (Topic 9). The status of the employees in that cluster are mainly composed of directors, vice presidents as well as presidents of the company. Interestingly, Deep-LPTM placed a president, corresponding to the node with the highest degree in the graph, at the centre of the network, and allocated it to the cluster of managers of the company. This graphical property, unique to Deep-LPTM, stresses the incorporation of the connections, as well as the topics in the emails, to obtain a meaningful representation of the network. Conversely, in Figure 9, the meta-network is not able to dissociate the connectivity of a node with a different connectivity than the rest of the cluster. Thus, Cluster 6 is central as a whole in the network in Figure 9.

Third, the topics involved in emails of Clusters 3 and 4 are the main drivers of the characterisation of the two clusters. For instance, Cluster 3 is highly connected to the graph and is involved in discussions about infrastructures and geographical aspects (Topic 7), as well as financial aspects (Topic 9). Emails related to Afghanistan were exchanged, mainly involving nodes from Cluster 3. It is worth noticing that nodes in cluster 3 correspond to people with a high position in the hierarchy of the company (almost two third of the employees in the clusters are either managers, directors, vice presidents, presidents or CEO). In addition, although many nodes in Cluster 4 are involved in regional energy discussions (Topic 2) like Cluster 2, they also discuss about day to day business questions (Topic 4), and are highly connected with the rest of the network, unlike Cluster 2.

Finally, Cluster 7 is composed of four nodes that are at the crossroads between several clusters and topics. We close this analysis with Cluster 5 that is composed of nodes mainly receipting emails and poorly connected to the graph, highlighted in the representation by the node positions being on the rim of the network.

CONCLUSION

We introduced a novel deep probabilistic model, named the deep latent position topic model (Deep-LPTM) to analyse networks with textual edges. Deep-LPTM allows to simultaneously cluster nodes, to model the topics in the documents, and to provide a network visualisation. To represent networks into a Euclidean space, most methodologies rely either on heuristic method used a posteriori or only focus on the connectivity of the graph. On the contrary, this work tackles this problem by incorporating the network representation into the modelling, enabling the clustering as well as the topic modelling to be included in the calculation of the node positions. To benefit from the flexibility of deep neural networks, our methodology is based on a graph convolutional network (GCN) to encode the nodes into a vector space using the connectivity of the graph as well as a neural topic model to encode the documents and the topics into a vector space. Even though we focused on directed networks, the extension to undirected networks is straightforward. The applications of this methodology are numerous, including social sciences, journalism or social network analysis. The proposed methodology relies on a variational inference algorithm to maximise the marginal likelihood. The optimisation combines analytical formulas and stochastic gradient descent steps to estimate the parameters. In this paper, we also derived the integrated classification and latent likelihood (IC2L) criterion to choose relevant numbers of clusters and topics as well as the dimension of the node latent space. Both the extensive benchmark study as well as the Enron emails analysis highlighted the visualisation power and the clustering efficiency of the proposed methodology. In future work, it is crucial to adapt this methodology to networks with multiple edges, corresponding to the exchange of several documents between nodes.

APPENDIX A GRAPHICAL MODEL

Figure A1 provides the graphical representation of the model with its parameters.

B COMPUTATION OF THE ELBO TERMS

In this section, computational details regarding the ELBO are provided term by term. First, E R log p(A | Z, κ) is given by:

E R log p(A | Z, κ) = N i,j=1 A ij E R log p ij + (1 -A ij )E R log 1 -p ij ,
where p ij = (1 + e -ηij ) -1 and η ij := κ -∥Z i -Z j ∥.

W ij A ij
Observed data

Z i Z j Y ij
Latent representation in a vector space

C j C i Clusters (i, j) π M ρ α N µ σ N m s F I G U R E A1
Graphical model with the parameters of Deep-LPTM where the Z i s denote the latent node vectors, Y ij s the latent document vector, C i s the node cluster memberships, A = (A ij ) ij ∈ M N×N ({0, 1}) the binary adjacency matrix and W ij the document sent by node i to node j.

Second, denoting

β k = softmax(ρ ⊤ α k ) ∈ R V , β = (β 1 . . . β K ) ⊤ ∈ M K×V (R) and w v ij = (β ⊤ Ỹij ) v , the probability for the word v to appear in document W ij ,for any v ∈ {1, . . . , V}, E R log p(W | A, Y, ρ, α) is: E R log p(W | A, Y, ρ, α) = N i,j=1 A ij E R log M V W ij ; M ij , β ⊤ softmax(Y ij ) = i,j A ij E R log M ij ! V v=1 (W v ij )! V v=1 (w v ij ) W v ij .
The difference between the terms related to the cluster memberships, E R log p(C | π) and E R log R(C) , gives the following:

E R log p(C | π) -E R log R(C) = M i=1 Q q=1 E R C iq log π q -E R C iq log τ iq = N i=1 Q q=1 τ iq log π q τ iq .
The difference between the generative distribution of the node positions term E R log p(Z | C, µ, σ) and the one related to the variational distribution of the node positions E R log R(Z | A) gives:

E R log p(Z | C, µ, σ) -E R log R(Z | A) = N i=1 Q q=1 E R C iq log N Z i ; µ q , σ 2 q I p (B1) - N i=1 E R log N Z i ; µ ϕZ (A) i , σ 2 ϕZ (A) i I p = - N i=1 Q q=1 τ iq KL N µ ϕZ (A) i , σ 2 ϕZ (A) i I p || N µ q , σ 2 q I p = - N i=1 Q q=1 τ iq log σ p q σ ϕZ (A) p i - p 2 + pσ 2 ϕZ (A) i + ∥µ ϕZ (A) i -µ q ∥ 2 2 2σ 2 q KL Z iq (µϕ Z (A)i,σ ϕ Z (A)i,µq,σq) = - N i=1 Q q=1 τ iq KL Z iq µ ϕZ (A) i , σ ϕZ (A) i , µ q , σ q . (B2)
Symmetrically, the term regarding the edge positions is obtained as follow:

E R log p(Y | A, C, m, s) -E R log R(Y | A, W) = N i,j=1 Q q,r=1 E R A ij C iq C jr log N Y ij ; m qr , s 2 qr I K - N i,j=1 E R A ij log N Y ij ; µ ϕY (W ij ), diag σ 2 ϕY (W ij ) = - N i,j=1 Q q,r=1 A ij τ iq τ jr KL N µ ϕY (W ij ), diag σ 2 ϕY (W ij ) || N m qr , s 2 qr I K = - N i,j=1 Q q,r=1 A ij τ iq τ jr KL Y ijqr µ ϕY (W ij ), σ ϕY (W ij ), m qr , s qr , where KL Y ijqr µ ϕY (W ij ), σ ϕY (W ij ), m qr , s qr = K log s qr - K k=1 log σ ϕY (W ij ) k - K 2 + K k=1 σ 2 ϕY (W ij ) k + ∥µ ϕY (W ij ) -m qr ∥ 2 2 2s 2
qr .

C INFERENCE C.1 Optimisation

In this section, we provide the optimisation steps to recover the parameters maximising the ELBO. To begin with, let us recall the ELBO expression:

L (τ , ϕ Z , ϕ Y ; π, µ, σ, κ, m, s, α, ρ) := L (R(•), Θ) = i,j A ij E R log p ij + (1 -A ij )E R log 1 -p ij + i,j A ij E R log M ij ! V v=1 (W v ij )! V v=1 (w v ij ) W v ij - N i=1 Q q=1 τ iq KL Z iq µ ϕZ (A) i , σ 2 ϕZ (A) i I p , µ q , σ 2 q I p - N i,j=1 Q q,r=1 A ij τ iq τ jr KL Y ijqr µ ϕY (W ij ), diag σ 2 ϕY (W ij ) , m qr , s 2 qr I K + N i=1 Q q=1 τ iq log π q τ iq . C.1.0.1 Update of τ
First, we optimise the ELBO with respect to τ iq . Since

τ i ∈ ∆ Q-1 , the term c i (1 - Q q=1 τ iq
) is added to the ELBO, giving the Lagrangian of the function. Thus, the derivative of the Lagrangian with respect to τ iq gives:

∂ ∂τ iq L(R(•); Θ) = -KL Z iq - N j=1 Q r=1 A ij τ jr KL Y ijqr +A ji τ jr KL Y jirq + log π q τ iq -1 -c i .
Setting this partial derivative to zero gives:

log τ iq = -KL Z iq - N j=1 Q r=1 A ij τ jr KL Y ijqr +A ji τ jr KL Y jirq + log π q -1 Tiq -c i .
Thus τ iq = e Tiq e -ci . Moreover, since Q q=1 τ iq = 1, we have e ci = Q q=1 e Tiq . Therefore, τ iq = e Tiq /( Q q=1 e Tiq ). The complete form is:

τ iq = π q e -KL Z iq -j̸ =i Q r=1 AijτjrK Y ij,qr +Ajiτjr KL Y jirq Q l=1 π l e -KL Z il -j̸ =i Q l ′ =1 Aijτ jl ′ K Y ij,ll ′ +Ajiτ jl ′ KL Y jil ′ l . (C3) C.1.0.2 Update of π Since π ∈ ∆ Q-1
, this constraint is added to the function to obtain the lagrangian. This corresponds to adding the term c 1 -Q q=1 π q to the ELBO. Thus, the partial derivative of the Lagrangian L with respect to π q of the lagrangian is:

∂ ∂µ q L(R(•); Θ, c) = N i=1 τ iq π q -c.
Setting this to zero gives:

1 π q N i=1 τ iq = c.
Multiplying by π q and summing over q gives that c = N. Therefore, after plugging it back into the previous expression, the following holds:

π q = 1 N N i=1 τ iq . (C4)
C.1.0.3 Updates of µ q and σ q

Taking the partial derivative of the ELBO with respect to µ q gives the following:

∂ ∂µ q L (R(•); Θ) = - N i=1 τ iq 2σ 2 q 2µ q -2µ ϕZ (A) i .
Therefore, setting this quantity to zero gives the following update for µ q :

µ q = N i=1 τ iq -1 N i=1 τ iq µ ϕZ (A) i . (C5)
The partial derivate of the ELBO with respect to σ q is:

∂ ∂σ q L (R(•); Θ) = - N i=1 τ iq p σ q - pσ 2 ϕZ (A) i + ∥µ ϕZ (A) i -µ q ∥ 2 2 2 2σ q σ 4 q .
Thus, the first order condition on σ q gives the following:

p σ q N i=1 τ iq = 1 σ 3 q N i=1 τ iq pσ 2 ϕZ (A) i + ∥µ ϕZ (A) i -µ q ∥ 2 2 σ 2 q = p N i=1 τ iq -1 N i=1 τ iq pσ 2 ϕZ (A) i + ∥µ ϕZ (A) i -µ q ∥ 2 2 . (C6) C.1.0.

Updates of m and s

As in the previous sections, we optimise the ELBO with respect to m and s with the first order conditions. The partial derivate of L with respect to m qr is:

∂ ∂m qr L (R(•); Θ) = - 1 2s 2 qr N i,j=1
A ij τ iq τ jr 2m qr -2µ ϕY (W ij ) .

Therefore, setting this expression to zero gives the following update for m qr :

m qr =   N i,j=1 A ij τ iq τ jr   -1 N i,j=1
A ij τ iq τ jr µ ϕY (W ij ). (C7)

The partial derivate of the ELBO with respect to s qr is: Thus, the first order condition on s qr gives the following:

s 2 qr K N i,j=1 A ij τ iq τ jr = N i,j=1 A ij τ iq τ jr K k=1 σ 2 ϕY (W ij ) k + ∥µ ϕY (W ij ) -m qr ∥ 2 2 .
Hence, the update of s 2 qr is given by:

s 2 qr =   K N i,j=1
A ij τ iq τ jr (C8)

Unfortunately, this expression cannot be computed since each term requires an integral with respect to the corresponding parameter. For instance, p(A | Z, M ) cannot be integrated analytically because of the logistic link function. Fortunately, a BIClike approximation can be derived for p(A|Z, M ), p(Z|C, M , Q, P), p(Y|A, C, M , K) and p(W|A, Y, M ). For instance, p(A|Z, M ) can be approximated by: where ν A,M = 1 denotes the number of free components in κ and n A = N(N -1) denotes the number of observations in A. This can be applied to all terms except p(C | M , Q) since the posterior cluster memberships probabilities τ i can be on the boundary of the parameter space. Fortunately, this term can be computed analytically. By assuming a Dirichlet prior D Q (δ 1 , . . . , δ Q ) on the topic proportions π:

p(C | M , Q) = p(C | π, M , Q)p(π)dπ = Γ Q q=1 δ q Q q=1 Γ δ q Q q=1 Γ(n q + δ q ) Γ Q q=1 n q + δ q ,
where n q := N i=1 C iq . In this paper, we consider the non informative Jeffreys prior distribution (δ q = 1/2), as in [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF] and [START_REF] Daudin | A mixture model for random graphs[END_REF]. Moreover, since C i is not available, we replace it with its maximum-a-posteriori estimate Ĉi where Ĉiq = 1 if q = argmax(τ i1 , . . . , τ iQ ), and 0 otherwise, which in turn gives nq := N i=1 Ĉiq . Using Stirling formula to approximate the Gamma function for a large value of N, we obtain:

p( Ĉ | M , Q) ≈ p( Ĉ | π, M , Q) - Q -1 2 log(N). ( C9
)
To conclude, since Z and Y are not available, we replace the missing data with their maximum-a-posteriori estimates Ẑ and Ŷ. Denoting IC2L the quantity log p(A, W, Ẑ, Ŷ, Ĉ | M , Q, K, P), we obtain:

IC2L = max

D NUMERICAL EXPERIMENTS

This section provides the entire ELBO evolution as well as the evolutions of the node ARI and the edge ARI, corresponding to the example in Section 4.2.

E APPLICATION TO THE ANALYSIS OF THE ENRON EMAIL NETWORK

To compare our results with a state-of-the-art approach, we provide an analysis of the Enron email network with ETSBM. The node positions displayed in Figure E3 were obtained using a Freichterman-Reingold algorithm [START_REF] Fruchterman | Graph drawing by force-directed placement[END_REF] and the top words associated to ETSBM topics are provided in Figure E4.

1

 1 Illustration of Deep-LPTM main contributions on a synthetic network. The colours of the nodes and of the edges denote the node clusters and the main topics of the corresponding documents respectively. The true node partition as well as the true topics of the documents are represented on the left-hand side. The three other figures are based on the respective results of SBM and LDA (second figure), ETSBM (third figure) and Deep-LPTM (fourth figure).

4

 4 Evolution of the ELBO, as well as the node and edge ARI during the optimisation of Deep-LPTM.

10 F I G U R E 9

 109 Enron email meta-network, based on Deep-LPTM estimates, represents the clusters and their interactions. The node sizes depend on the number of email accounts assigned to each cluster and the node positions are estimated by Deep-LPTM.

2 s 3

 23 ∂ ∂sqr L (R(•); Θ) = -N i=1 A ij τ iq τ jr K s qr -K k=1 σ 2 ϕY (W ij ) k + ∥µ ϕY (W ij ) -m qr ∥ 2 qr .

2 .C. 2

 22 ij ) k + ∥µ ϕY (W ij ) -m qr ∥ 2 Derivation of the selection model criterion Proof of Proposition 3. Assuming a fully factorised prior distribution p(κ, π, µ, σ, m, s, ρ, α) = p(κ)p(π)p(µ)p(σ)p(m)p(s)p(ρ)p(α), Lemma 3.1 in[START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF] can be directly extended to our case to decompose the integral in[START_REF] Dieng | Topic modeling in embedding spaces[END_REF]:log p(A, W, Z, Y, C | M , Q, K, P) = log p(A | Z, M ) + log p(Z | C, M , Q, P) + log p(W | A, Y, M ) + log p(Y | A, C, M , K) + log p(C | M , Q).

  log p(A | Z, M ) = log κ p(A | κ, Z, M )p(κ)dκ ≈ max κ log p(A | Z, κ, M ) -ν A,M 2 log(n A ),

θ 2 K + Q 2

 22 log p(A, W, Ẑ, Ŷ, Ĉ | θ, M , Q, K, P) -Ω(M , Q, K, P) log p( Ẑ | Ĉ, µ, σ, M , Q, P)log p(W | A, Ŷ, ρ, α, M )log p( Ŷ | A, Ĉ, m, s, M , K) -Q + K) + Q 2 (K + 1) 2 log(M).

  U R E D2 Evolution of the ELBO, as well as the nodes ARI during the optimisation.

  Number of times a triplet (K, P, Q) is associated with the highest IC2L over 10 graphs simulated according to Scenario C (Q ⋆ = 4 and K ⋆ = 3). All the models with the highest IC2L value correspond to P = 2. Therefore, only the table corresponding to this value is shown.

4.4.0.2

Selection of the triplet (P, K, Q)

  ARI of the node clustering averaged over 10 graphs in all three scenarios for the two levels of difficulty Easy and Hard. Deep-LPTM, as well as ETSBM, are presented with and without pre-trained embeddings (denoted PT). Moreover, STBM and SBM are also provided as baselines.

	Topic 1
	Topic 2
	Topic 3
	Topic 4
	Topic 5
	Topic 6
	Topic 7
	Topic 8
	Topic 9
	Topic 10
	ScenarioB 1.00 ± 0.00 -0.00 ± 0.01 0.73 ± 0.05 ScenarioC 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.01 0.99 ± 0.03 1.00 ± 0.00 0.96 ± 0.04 1.00 ± 0.00 1.00 ± 0.00 0.96 ± 0.05 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 Deep-LPTM -PT 1.00 ± 0.00 SBM STBM ETSBM ETSBM -PT Deep-LPTM 1.00 ± 0.00 1.00 ± 0.00 SBM 0.97 ± 0.03 0.00 ± 0.00 0.62 ± 0.1 STBM 0.63 ± 0.23 1.00 ± 0.00 0.66 ± 0.19 ETSBM 0.96 ± 0.10 0.90 ± 0.30 0.72 ± 0.25 ETSBM -PT 0.99 ± 0.01 1.00 ± 0.00 0.74 ± 0.21 Deep-LPTM 0.99 ± 0.02 1.00 ± 0.00 0.89 ± 0.15 Deep-LPTM -PT 1.00 ± 0.01 1.00 ± 0.00 0.85 ± 0.18 T A B L E 6 Cluster 1 Easy Hard Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7

  F I G U R E E3 ETSBM clustering results on the Enron email network. The node positions were obtained using the Fruchterman-Reingold algorithm.

		Topic 1	Topic 2	Topic 3		Topic 4	Topic 5
			tw		enron	mike		enron	business
		watson	message	message		message	interview
		message	original	original		master		enron
			gas	company	grigsby		corp		friday
		mmbtu		mail	desk		original		phase
		capacity	november	john		agreement	interviewers
		Topic 1 deliveries		pmto	pmto		attached		unit
		Topic 2 original		amto	october		october		super
		Topic 3 lynn Topic 4 Topic 5 socalgas		fw trading	daily deals		america north		units dinner
		Topic 6 Topic 7 Topic 6	Topic 7	Topic 8		Topic 9	Topic 10
		Topic 8 message	backup	message		day		jeff
		Topic 9 enron Topic 10 original		plan seat	enron original		ofo gas	state california
			jim		work	gas		cycle		edison
			ferc	location	november		storage		power
			steffes		west	october		usage		puc
			rto		enron	pmto		daily	dasovich
			group		day	amto		socal		davis
		market		team	mail		mmcf	message
		energy		move	monday		scheduled	original
	F I G U R E	E4 The	10	most	probable	words	of	each	topic	according	to	ETSBM.
										Cluster 1
										Cluster 2
										Cluster 3
										Cluster 4
										Cluster 5
										Cluster 6
										Cluster 7