Eric, Polina Annika Luiza

Géraud Le Falher

In this thesis, we develop methods to efficiently and accurately characterize edges in complex networks. In simple graphs, nodes are connected by a single semantic. For instance, two users are friends in a social networks, or there is a hypertext link from one webpage to another. Furthermore, those connections are typically driven by node similarity, in what is known as the homophily mechanism. In the previous examples, users become friends because of common features, and webpages link to each other based on common topics. By contrast, complex networks are graphs where every connection has one semantic among k possible ones. Those connections are moreover based on both partial homophily and heterophily of their endpoints. This additional information enable finer analysis of real world graphs. However, it can be expensive to acquire, or is sometimes not known beforehand. We address the problems of inferring edge semantics in various settings. First, we consider graphs where edges have two opposite semantics, and where we observe the label of some edges. These so-called signed graphs are a convenient way to represent polarized interactions. We propose two learning biases suited for directed and undirected signed graphs respectively. This leads us to design several algorithms leveraging the graph topology to solve a binary classification problem that we call EDGE SIGN PREDICTION. Second, we consider graphs with k ≥ 2 available semantics for edge. In that case of multilayer graphs, we are not provided with any edge label, but instead are given one feature vector for each node. Faced with such an unsupervised EDGE ATTRIBUTED CLUSTERING problem, we devise a quality criterion expressing how well an edge k-partition and k semantical vectors explains the observed connections. We optimize this goodness of explanation criterion in vectorial and matricial forms, and show how those two methods perform on synthetic data.

1 https://www.inria.fr/en/teams/magnet iii and Émilie. On my way to organize an elusive café lingua, I also enjoyed the company of many members of a meetup group, including Samir, Nathalie, Marie-Pierre, Anne Charlotte, Léa, Géraldine, Fabienne, George, Norida and Thomas, as well as Rashida, Mairead and Yash, who bore with me during circumstances better not written here. Another great event in Lille is undeniably the "apéro culture", where I was privileged to meet Cyrille, Élise, Audrey, Wasilla and Olmo. This picture wouldn't be complete without Stéfana, Arthur, Esmeralda, Daniel, and my flatmate Édouard for some nice video games nights.

I also received support from outside Lille, thanks to Tristan, Olivier

Résumé

Dans cette thèse, nous proposons des méthodes pour caractériser efficacement et précisément les arêtes au sein de réseaux complexes. Dans les graphes simples, les noeuds sont liés au travers d'une sémantique unique. Par exemple, deux utilisateurs sont amis dans un réseau social, ou une page web contient un lien hypertexte pointant vers un autre page. De plus, ces connexions sont généralement guidées par la similarité entre les noeuds, au travers d'un mécanisme appelé homophilie. Dans les exemples précédents, les utilisateurs deviennent amis à cause de caractéristiques communes, et les pages web sont reliées les unes aux autres sur la base de sujets communs. En revanche, les réseaux complexes sont des graphes où chaque connexion possède une sémantique parmi k possibles. Ces connexions sont en outre basées à la fois sur une homophilie et une hétérophilie partielle des noeuds à leurs extrémité. Cette information supplémentaire permet une analyse plus fine des graphes issus d'applications réelles. Cependant, elle peut être coûteuse à acquérir, ou n'est pas toujours disponible a priori. Nous abordons donc le problème d'inférer la sémantique des arêtes dans plusieurs contextes. Tout d'abord, nous considérons les graphes où les arêtes ont deux sémantiques opposées, et où nous observons l'étiquette de certaines arêtes. Ces « graphes signés » sont une façon élégante de représenter des interactions polarisées. Nous proposons deux biais d'apprentissage, adaptés respectivement aux graphes signés dirigés et non dirigés. Ceci nous amène à concevoir plusieurs algorithmes utilisant la topologie du graphe pour résoudre un problème de classification binaire que nous appelons EDGE SIGN PREDICTION. Deuxièmement, nous considérons les graphes avec k ≥ 2 sémantiques possibles pour les arêtes. Dans ce cas, nous ne recevons pas d'étiquette d'arêtes, mais plutôt un vecteur de caractéristiques pour chaque noeud. Face à ce problème non supervisé d'EDGE ATTRIBUTED CLUSTERING, nous concevons un critère de qualité exprimant dans quelle mesure une k-partition des arêtes et k vecteurs sémantiques expliquent les connexions observées. Nous optimisons ce critère « qualité explicative » sous une forme vectorielle et matricielle et illustrons le comportement de ces deux méthodes sur des données synthétiques.

First and foremost, I would like to express my deepest gratitude to my supervisor, Pr. Marc Tommasi. From the day we first got in touch in that spring four years ago, and all the way towards my defense, he has provided me with thorough guidance. In terms of research of course, but on many other topics as well, from professional development to folkloric music. Moreover, this has always been done in the kindest way possible. Therefore, I'm doubly indebted to him, for not only I wouldn't have complete my PhD without him, but he made it a very pleasant experience. Second, such feelings carries over to include my advisor, Dr. Fabio Vitale. There has been time when his sense of rigor has challenged me. But at the end, it was an important learning experience, and his ability to crack a joke at the least expected moment of a long research meeting has proved very useful: the fact we went to bars in at least three different countries speaks for it. I would also like to thank Pr. Claudio Gentile for hosting me in Varese, and giving me sharp advice along those three years.

Then I want to thank Alessandro Provetti and Liva Ralaivola who kindly agreed to review my manuscript, as well as Elisa Fromont and Sophie Tison for having accepted to be part of my committee. To conclude this academic paragraph, I'd like to sincerely acknowledge how lucky I was to collaborate with all my talented co-authors: Nicolò Cesa-Bianchi, Emre Çelikten, Claudio Gentile, Aristides Gionis, Michael Mathioudakis and Fabio Vitale.

Although doing a PhD is a rather personal enterprise, I was fortunate to be part of the awesome Magnet team 1 , whose past and current members I met include, in seating order: Thomas, Pauline, David, Mathieu (Thursday French Fries forever), Thanh, Carlos, Pierre, Paul, Juhi, Quentin, William, Mikaela, Pascal (special thanks for great accommodation and sportive spirit), Rémi, Aurélien and Hippolyte. Thanks for stimulating discussions on many topics, constant readiness to help and answer my questions, life long lessons of table football, and quite importantly, thanks for patiently bearing with me while I was finishing lunches (and/or fiddling with my phone)! Of course, this wouldn't be proper acknowledgment without mentioning the best office in the world, namely B224, and the best possible office mates: Nathalie (for all the cookies) and Thibault (because there were some genuinely good jokes among all of them). Whereas I've always been slightly worried about logistics, I never had to think about it thanks to the fantastic work of Julie. Finally, one last Magnet member, Antonino, left before I joined but nonetheless deserve my gratitude for he helped me find my next job. Same credit goes to Michal, in addition to being a great company in all occasions. At this stage, I might as well thanks the rest of the Sequel team, and especially Julien, Alexandre, Émilie, Frédéric, Romaric, Matteo, Daniele, Julien, Bilal, Marta, Tomáš, Daniele, Florian, Jean-Bastien, Olivier and Ronan (too bad my defense is not on Friday, for I know you wouldn't be long to hit the dance floor). Finally I want to thank all the support crews from Inria, as they made it a great place for doing research, as well as the University Lille 3 for making my short teaching experience so pleasant. Doing a PhD is also supposed to be a rather labor intensive enterprise, but again, I was fortunate to live in the student city of Lille. There I met many wonderful people, each of them who contributed in a way to this manuscript. It all started when a young version of myself attended a student event organized by Tilda and later had to pleasure to collaborate with Guillaume, Antoine, Fabien, Alexandre, and by extension with Carmelo, Jason, Cindy, Antoine (Dujardin), Clément, Ilkay, Benjamin 1 from [START_REF] Abraham | Using petal-decompositions to build a low stretch spanning tree[END_REF], showing the evolution of the best asymptotic average stretch over time. . . . . . . . . . . . . . . . . 3.9 A summary of the lowest stretches achievable for various problems.

List of Figures

LIST OF TABLES

In this thesis we will sometimes write remarks in a smaller font and with a light blue edging.

Such remarks provide additional information to the topic discussed above, but can be skipped without harming the comprehension of the main material.

Table 1 -List of notations used in this thesis

Symbol Meaning

K

The natural integers from 1 to K, i.e. {1, 2, . . . , K}

I {p}

An indicator function evaluating to 1 if the predicate p is true and 0 otherwise x ;i For a vector x ∈ R d , x ;i denotes the value of its i th component.

S d

The sphere of 2 unit vectors in dimension d, i.e. x ∈ R d :

x 2 = 1
Chapter 1

Introduction

Graphs are a natural way to represent the relationships over a set of entities. Because of the simplicity and flexibility of this formalism, graphs are ubiquitous and have been used in countless fields. To make the rest of our explanations more concrete, we now give three examples of graphs that we consider later on. 1. In a social network1 , the nodes of the graph are human users, and relationships between nodes denote interactions between the corresponding users. One social network we study in Chapter 2 summarizes the connections between the editors of WIKIPEDIA. These editors can be promoted to administrators after a vote from their peers. An edge u → v in this network means that user u has voted on the possible promotion of v. 2. In computer vision, we can represent an image as a graph. Each pixel is a node, and those nodes are connected to four neighbors, namely the adjacent pixels from the top, bottom, left and right sides. The relationship in this graph is therefore adjacency in an image. 3. In e-commerce, we can consider a co-purchase network. Nodes are products being sold on a website, and two nodes are connected if the corresponding products are frequently bought together by customers, for instance a phone and a memory card. Not only are there graphs in many domains, but the progress of technology in the last few decades has made it easier to collect many graphs in every single domain, sometimes with up to billions of nodes and hundreds of billions of edges. The availability of such large amounts of structured data has prompted the development of automated methods to extract insights from them. For instance, it is possible to cluster nodes into coherent groups, predict the category of the nodes or study how to best propagate information within a graph. We describe in more details these possibilities and others in Section 1.1. By doing so, our goal is to illustrate the wide potential of learning in graphs.

At this point of the discussion though, we have only considered the most common and simple kind of graph, one representing a single type of relation between nodes.2 However, in many situations, there are two dimensions along which graphs exhibit more complexity. First, nodes have more than one type of relation among each other. Second, two nodes are not only connected because of their global similarity but also for more nuanced reasons. As a case in point, let us look again at our three examples. In the WIKIPEDIA network, a vote can support the promotion or oppose it. This additional information enrich our understanding of the relationships among the nodes in the graph. It also points out that two editors can be connected because they share a common topic of interest, but come from different socio-economic backgrounds and therefore cannot agree on this topic.

Likewise in an image, an edge between two pixels is positive if the two pixels belong to the same object (say a car or a building) and negative otherwise. This information could be used to segment the image. In the co-purchase network, there are even more than two types of relation. Assuming the products are movies, two movies can be frequently bought together because they are part of a series (like Star Wars), because they have the same director but different genre, because they receive the same prestigious award in different years, and so on. The second type of relation ("same director" and "different genre") is actually an example of a mixed relation, for it combines similarity and dissimilarity over several features of the nodes.

We call complex networks graphs where edges have different semantics, or types, and where connections are explained by more than simple global similarity. In Section 1.2, we give additional examples of such complex networks. As showed in our three previous examples, we distinguish between two cases. The first is when there are two types of edge having opposite semantics. Such graphs are called signed graphs and have been extensively studied since the fifties, for we shall see they have many applications. Their name comes from the fact that edges are typically labeled +1 and -1. The second case is when there are more than two types of edge. Such graphs have also been studied for a long time under different names and we refer to them as multilayer graphs.

It is natural to expect that we can extract finer insights from graphs with edge semantics. However, in many cases, the information about edge types is not available, at least not in a convenient form. We therefore present in Section 1.3 the problem of characterizing edges in complex networks. Informally, given an input graph and possibly some extra information, we want to predict the type of every edge. This problem can take several forms depending on what information is available as input. First, the graph may be directed or undirected. Second, there might be two edge types (in the case of signed graphs) or more (for general multilayer graphs). Third, the problem can be supervised or not. In the supervised setting, we are provided with labels for some of the edges. In the unsupervised setting, there is no label at all. We thus make the additional assumption that we observe some attributes of the nodes. Once again, we return to our three examples to illustrate what those attributes can be. In the WIKIPEDIA graph, we could have for each user data about her age, experience and area of expertise. In an image, each pixel is associated with a color, along with higher order visual features. Each product of a co-purchase network comes with information about, e.g., its price, popularity, category and availability. In the absence of label, our intuition is that these attributes can inform us about the types of edge among nodes.

We list in Section 1.4 three concrete instantiations of this general problem that we consider in this thesis. Our solutions to these three problems offer evidence in the defense of our thesis statement:

There exist efficient and accurate methods to predict edge type in complex networks, relying only on the graph topology or also on node attributes.

Learning in graphs

The birth of graph theory is credited to Euler [START_REF] Euler | Solutio problematis ad geometriam situs pertinentis[END_REF] in 1736 for his elegant solution of the Seven Bridges of Königsberg problem 3 . Since then, it has been a very active branch of mathematics [START_REF] Biggs | Graph Theory[END_REF]. Indeed, it provides a conceptually simple yet immensely rich framework to model phenomena where entities are connected with each other [START_REF] Da Fontoura | Analyzing and modeling real-world phenomena with complex networks: a survey of applications[END_REF]. Coupled with the increasing availability of large amount of relational data, learning on graphs has recently spurred a lot of interest across various lines of research, with tangible benefits.

community detection [START_REF] Fortunato | Community detection in graphs[END_REF] The goal is to cluster nodes in tightly connected groups that are loosely connected with the rest of the graph. This allows for a better understanding of the graph organization, and present a higher level view than looking at the individual node relationships. For instance, it has been used to identify proteins functional groups [START_REF] Spirin | Protein complexes and functional modules in molecular networks[END_REF], or to see how different scientific fields relate based on publication data [START_REF] Rosvall | Maps of random walks on complex networks reveal community structure[END_REF].

semi-supervised learning [CSZ06; ST14]

In addition to labeled data, the learner is also provided with unlabeled data at training time, and its goal is to classify nodes.

Connecting similar instances allow propagating information along the graph. This has found applications in classifying text documents [START_REF] Talukdar | New regularized algorithms for transductive learning[END_REF] or aligning categories and relations across multiple knowledge bases [START_REF] Wijaya | Pidgin: ontology alignment using web text as interlingua[END_REF].

node embedding [START_REF] Hamilton | Representation learning on graphs: methods and applications[END_REF] The goal is to find a low dimensional representation of the nodes, based on their structural patterns. This usually performed in an unsupervised way, although it is also possible to include problem supervision when available. Such representation can then be used in downstream tasks, for instance visualization [START_REF] Tang | Line: large-scale information network embedding[END_REF] or the aforementioned node classification, even in the inductive setting where new nodes can join the graph after training [START_REF] Hamilton | Inductive representation learning on large graphs[END_REF].

link prediction [MBC16]

Given a snapshot of the graph at time t, the goal of link prediction is to return a set of links that do not exist at time t but will be created by time t + ∆ t . Most methods are based on the assumption that link creation is driven by node similarity. It has been successfully applied to inferring potential interactions between proteins without expensive experiments [START_REF] Qi | Evaluation of different biological data and computational classification methods for use in protein interaction prediction[END_REF] and uncovering hidden associations in criminal networks [START_REF] Clauset | Hierarchical structure and the prediction of missing links in networks[END_REF].

information and influence propagation [CLC13]

The study of processes by which content is spread across networks, and how such processes can be influenced to speed them up or slow them down. Two representative applications are selecting the best seeds in a social network to promote a viral marketing campaign [START_REF] Kempe | Maximizing the spread of influence through a social network[END_REF] and containing more effectively the diffusion of actual biological viruses [START_REF] Prakash | Fractional immunization in networks[END_REF].

network evolution [START_REF] Aggarwal | Evolutionary network analysis: a survey[END_REF] These methods focus on the mechanisms and consequences of the growth of networks. They also seek ways to keep the results of some data mining algorithms up to date and relevant. Monitoring the changes in the interactions of proteins can indeed be used as an early indicator for some kind of diabetes [START_REF] Beyer | Mechanistic insights into metabolic disturbance during type-2 diabetes and obesity using qualitative networks[END_REF]. Furthermore, sudden changes in a network of computers are usually worth investigating, for they might signal external attacks [START_REF] Idé | Eigenspace-based anomaly detection in computer systems[END_REF].

This list of graph learning problems and their applications to real world scenario is incomplete. Yet it already demonstrates the impact of inferring patterns in relational data over many aspects of our lives. However, we argue that more can be done. Indeed, all the methods presented above only consider graphs with a single type of edge and where nodes are connected based on some domain-dependant notion of similarity.

Graph with several edge semantics

The three graphs we described in the introduction are examples of what we call complex networks. Such graphs have more than one type of edge, and edges are not explained by mere global node similarity. The point of this section is to illustrate that this notion is not simply a mathematical variation of a "simple" graph, where the adjacency matrix would take values in {-1, 0, 1} or {0, 1, . . . , k} instead of {0, 1}.

Rather, we review many applications in various domains where complex networks are the right model to represent rich networked data. First, we showcase many uses of signed networks. In that case, we do not insist on the fact that similarity is not the only driver of node connections, for it is implicit that negative edges actually denote dissimilarity. Second, we present multilayer graphs, that is graphs with more than two types of edges. When applicable, we highlight how the connections in such graphs indeed rely on a nuanced similarity or dissimilarity across some of the nodes attributes.

To elaborate on this last point, our hypothesis is that connections in complex networks are the results of both partial homophily (that is, nodes are connected when they are similar on a subset of the attributes) and partial heterophily (that is, nodes are connected when they are dissimilar on a subset of the attributes). As examples of the latter, think of dating websites -where most users are linked with users of the opposite gender [START_REF] Thelwall | Homophily in myspace[END_REF][START_REF] Tyson | A first look at user activity on tinder[END_REF]; diffusing innovations -where meeting people with different backgrounds and point of views is crucial to favor diversity and creativity [START_REF] Rogers | Diffusion of Innovations[END_REF]; and online news consumption -where connecting people from different sides of the political spectrum helps to avoid echo chambers and instead fuel a democratic debate [START_REF] Garimella | Balancing information exposure in social networks[END_REF].

Signed graphs

In this section, we present a list of signed graph usages, sorted by domains [START_REF] Tang | A survey of signed network mining in social media[END_REF]. Many of those signed graphs are the input of some clustering algorithms. In the context of signed graph, the clustering task can be captured by the CORRELA-TION CLUSTERING problem. We provide a thorough overview of this problem in Chapter 3. Here we simply give a broad, informal definition. The objective in CORRELATION CLUSTERING is to cluster the nodes of a signed graph in a way that minimizes the number of positive edges across clusters and the number of negative edges within clusters. Such edges are called disagreements.

Computer Vision

The ubiquitous task of segmenting an image into homogeneous regions is a prerequisite for many further processing. As we mentioned earlier in one example, building a signed graph can help, although it might be costly to do it at the pixel level. For instance, to segment cells in microscopy imagery, Zhang et al. [START_REF] Zhang | Cell detection and segmentation using correlation clustering[END_REF] first use generic image features to classify pixel in belonging to region boundaries or not. Then, they extract small scale regions called superpixels. After building the adjacency graph of these superpixels, they assign edge weights by averaging the boundary probabilities of the pixels separating adjacent superpixels. They also add strong negative constraints between distant superpixels, and lastly cluster these superpixels according to the CORRELATION CLUSTERING objective to obtain the final segmentation. A similar approach was used earlier in Kim et al. [START_REF] Kim | Higher-order correlation clustering for image segmentation[END_REF], who stress the importance of considering such higher order constraints between distant superpixels in order to achieve good performance. This was also extended to 3D segmentation [And+12], where additional tuning allows to segment a volume image of a mouse cortex with up to billions voxels. Beier et al. [START_REF] Beier | Fusion moves for correlation clustering[END_REF] segment 2D and 3D images with an energy based formulation of CORRELATION CLUSTERING and iteratively improve their solution by merging it with another clustering given by a proposal generator. By developing another scalable energy based optimization procedure, and with the help of few user-provided cues, Bagon et al. [START_REF] Bagon | Large scale correlation clustering optimization[END_REF] are able to apply CORRELATION CLUSTERING directly at the pixel level.

Beyond image segmentation, Gori et al. [START_REF] Gori | Flowrep: descriptive curve networks for free-form design shapes[END_REF] develop a method to extract a network of descriptive curves from 3D shapes. After an initial stage of generating many such flowlines, they describe in Section 6 a CORRELATION CLUSTERING formulation to extract reliable representative flowlines, using geometric constraints to obtain positive or negative cues that two flowlines are from the same reliable representative.

Finally, in order to track several targets across sequential video frames, Solera et al. [START_REF] Solera | Learning to divide and conquer for online multi-target tracking[END_REF] propose a multistage framework. One step revolves around a matrix A that defines the cost of assigning an object tracked in previous frames to an object detected in the current frame. This matrix is turned into a symmetric affinity matrix Āsym that can be seen as a signed graph adjacency matrix. CORRELATION CLUSTERING is then used to extract clusters (called zones), in which local processing is performed. This is beneficial since the complexity of these local methods can be adapted to the difficulty of each zone.

Natural Language Processing Coreference resolution is the task of finding all expressions that refer to the same entity in a text. Like image segmentation, it is a preprocessing step that can later be used in document summarization, question answering, and information extraction. Furthermore, in that case, it is also natural to build a graph of words. One then add negative edges between words that cannot refer to the same entity (for instance because they are of different gender) and positive edges between words with linguistic cues indicating they might refer to the same entity. McCallum et al. [MW05, Section 2.3] instead tackle coreference resolution by using an undirected graphical model on which performing inference is equivalent to CORRELATION CLUSTERING. On small scale instances, Elsner et al. [START_REF] Elsner | Bounding and comparing methods for correlation clustering beyond ilp[END_REF] use the signed graph procedure outlined above. They first obtain an upper bound of the optimal solution by solving a SDP relaxation of the problem. They then compare various heuristics and show that the best performing ones are within few percents of the optimum, provided they are followed by a local search step, such as the Best One Element Move [START_REF] Gionis | Clustering aggregation[END_REF]. Further NLP tasks amenable to a signed graph representation are referenced in their paper. Another task is to cluster words based on distributional embedding vectors while adding antonym constraints [START_REF]Predicting Emotional Word Ratings using Distributional Representations and Signed Clustering[END_REF].

Biology Signed graphs are also abundant in biology. A typical input is a similarity matrix between genes expression level in various experimental conditions, and the goal is to cluster those genes into groups that react similarly. Ben-Dor et al. [START_REF] Ben-Dor | Clustering gene expression patterns[END_REF]Section 4] gives three examples: 112 genes involved in the rat central nervous system, 1246 genes of the roundworm C. elegans and 2000 human genes obtained from 40 tumorous and 22 normal colon samples. Mason et al. [START_REF] Mason | Signed weighted gene co-expression network analysis of transcriptional regulation in murine embryonic stem cells[END_REF] analyze a signed co-expression network of genes involved in embryonic stem cells to find which genes are related to pluripotency (the ability to differentiate into any type of cell in the body) or self-renewal (the ability to replicate indefinitely). Another application is to study the variation of one individual DNA [START_REF] Das | Sdhap: haplotype assembly for diploids and polyploids via semi-definite programming[END_REF]. In the human organism, chromosomes are organized in pair, and both chromosomes of a pair have similar but not identical DNA sequences. This is mostly because of single nucleotide polymorphisms (SNPs), where a single base differs between the two DNA sequences, leading to different alleles of the corresponding gene. A haplotype is the list of all alleles at a contiguous region of a single chromosome, and this information is used in several medical applications. The high-throughput sequencing of one individual genome yields many short reads that provides information about the order of nucleotides in a fragment of one chromosome and that can be used to assemble haplotypes. To do so, the authors build a graph of reads and define a similarity function between reads to assign weights on the edges. The clusters of that graph correspond to haplotypes, and are obtained by a SDP relaxation of the CORRELATION CLUSTERING objective. DasGupta et al. [START_REF] Dasgupta | Algorithmic and complexity results for decompositions of biological networks into monotone subsystems[END_REF] also consider graphs whose nodes are genes, but in a different context. In this case, positive edges represent an activating connection, while negative edges represent inhibiting connection. They also define a monotone system as a balanced subgraph, that is a subgraph which does not contain a cycle with an odd number of edges. Such monotone system are stable, in the sense that modifying the concentration of one gene will have a predictable effect, even ignoring the precise kinematics of the chemical reactions involved. Their goal is to find the minimum number of edges to remove in order to decompose a dynamics system into a collection of monotone system. This allows to study the complete system more easily. More applications of weighted complete signed graphs in biology are presented in [START_REF] Böcker | Cluster editing[END_REF]Section 6].

Network science One early use of signed graphs was to model social interactions [START_REF] Harary | On the notion of balance of a signed graph[END_REF]Hei58]. Here we present some recent references within this line of research. For instance, one can extract all the votes of the members of a political parliament and form a graph whose nodes are politicians and edge weights quantify how much they agree or disagree on various issues they have been voting on. This can be used to study various social science questions such as loyalty, leadership, coalitions, political crisis and polarization. It has been applied to the European parliament [START_REF] Mendonca | Relevance of negative links in graph partitioning: a case study using votes from the european parliament[END_REF], Slovenian parliament [START_REF] Jiang | Stochastic block model and exploratory analysis in signed networks[END_REF] and the Brazilian parliament [START_REF] Levorato | Brazilian congress structural balance analysis[END_REF]. This can also be used at international level. For instance, by considering a dataset of military alliances and disputes, Traag et al. [START_REF] Traag | Community detection in networks with positive and negative links[END_REF] cluster countries into blocks that resemble those identified by Huntington in his Clash of Civilizations book. Another source of data is the vote on resolutions during the United Nations General Assembly [START_REF] Macon | Community structure in the united nations general assembly[END_REF]. Finally, one can also study how to exploit the information contained within the negative links to enhance the visualization of social networks [START_REF] Kunegis | Spectral analysis of signed graphs for clustering, prediction and visualization[END_REF][START_REF] Kermarrec | Can everybody sit closer to their friends than their enemies?[END_REF].

Others

• Deduplication, also called duplicate detection or entity resolution, is the process of identifying objects from a real-world, noisy database that refer to the same entity. On a high level, a solution to this problem is to build a graph of all the available objects, define a similarity between them and run a CORRELATION CLUSTERING algorithm. The main challenge thus lies in devising an appropriate similarity measure, given that objects can have very different features from one database to another. Arasu et al. [START_REF] Arasu | Large-scale deduplication with constraints using dedupalog[END_REF] propose a declarative language, expressing both hard constraints (that have to be satisfied) and soft constraints (that can be seen as cues guiding the process). Because of these hard constraints that admissible clusterings have to respect, the authors have to modify in nontrivial ways an existing CORRELATION CLUSTERING algorithm. This was extended to weighted and partial constraints by Shen et al. [START_REF] Shen | Entity resolution with weighted constraints[END_REF]. Another example is given by Gael et al. [START_REF] Gael | Correlation clustering for crosslingual link detection[END_REF], who cluster together news articles in different languages covering the same event. CORRELATION CLUSTERING was also evaluated among other solutions to that problem by Hassanzadeh et al. [START_REF] Hassanzadeh | Framework for evaluating clustering algorithms in duplicate detection[END_REF], who note that their non optimized implementation does not perform best.

• Given an electrical circuit layout, Chiang et al. [START_REF] Chiang | Fast and efficient bright-field aapsm conflict detection and correction[END_REF] extract a graph of its components (called shifter) that must be assigned one of two possible phases.

Because two shifters next to some specific shape must be in opposite phase and two shifters separated by less than a specified distance must be of the same phase, the authors look for a two-clustering of the nodes that will minimize the number of disagreements.

• In finance, one can represent an investment portfolio as a signed graph [START_REF] Harary | Signed graphs for portfolio analysis in risk management[END_REF].

Each node is a security, and the edge between two securities is weighted by their correlation, which can be negative. For instance, a graph with only positive edges is speculative, as all the securities move in the same direction, either up or down. On the other hand, if the securities can be partitioned in groups without disagreement, the risk is limited, for two clusters will move in opposite directions, providing the investors with a hedging guarantee.

• In wireless networks, signed graphs can be used to solve optimization problems involved in determining energy-saving routes [START_REF] Rath | Weighted signed graph (wsg) power-aware routing in distributed wireless networks[END_REF] or to exchange cryptographic keys in a secure and efficient manner [START_REF] Sarkar | Secure iot using weighted signed graphs[END_REF].

Multilayer graphs

Besides signed graphs, in this thesis we also consider multilayer graphs [START_REF] Kivela | Multilayer networks[END_REF][START_REF] Boccaletti | The structure and dynamics of multilayer networks[END_REF]. Those are graphs with k edge types, and the name refer to the fact we can see them as the superposition of the k subgraphs induced by each edge type. Even when k = 2, we make a distinction between signed graphs and multilayer graphs.

In signed graphs, the two types of edge have reverse semantic, whereas in a 2-layers graph, they simply denote two possible interactions, for instance advisor-advisee or regular coauthors relation in a citation network [START_REF] Wang | Mining advisor-advisee relationships from research publication networks[END_REF]. In general though, we focus on cases where k is larger than 2, and not larger than a few dozen in order to preserve interpretability. Like signed graphs, these multilayer graphs are versatile enough to be used in many fields. Another example is photo sharing website Flickr and its users. They can interact in eleven ways, either directly or through comments, shared tags, groups membership and so on [START_REF] Kazienko | Multidimensional social network in the social recommender system[END_REF]. Again, while being part of a common group denotes shared interests between two users, overall they must also differ in some other attributes (for instance location) in order to bring diversity to this group.

Social networks

Citation networks

In these networks, nodes are research papers or authors, and edges typically connect two nodes whenever one cites the other. For instance, using the DBLP dataset, Cai et al. [START_REF] Cai | Community mining from multi-relational networks[END_REF] connect two authors if they have co-authored a paper in one the 1 000 conferences appearing in the data. One can then consider that the resulting graph was obtained as the superposition of these 1 000 subgraphs. Besides direct citations, Dunlavy et al. [START_REF] Dunlavy | Multilinear algebra for analyzing data with multiple linkages[END_REF] consider four others reasons to connect 5 000 SIAM papers, based on their similarity in terms of abstract words, title words, keywords and authors.

Economic networks

In our ever increasingly globalized economy, entities around the world are getting more and more tightly connected. However, those connections take on many different forms simultaneously. For something seemingly as simple as the connections between the largest 951 ports in the world, one must already notice that these connections can be implemented by any combinations of three kinds of ships: bulk dry carriers, container ships and oil tankers [START_REF] Kaluza | The complex network of global cargo ship movements[END_REF]. Likewise, the 162 countries of the International Trade Network are connected by 96 kinds of commodities they can exchange [START_REF] Barigozzi | Multinetwork of international trade: a commodity-specific analysis[END_REF]. Finally, Cranmer et al. [START_REF] Cranmer | Kantian fractionalization predicts the conflict propensity of the international system[END_REF] studies international relations through the lens of Kant's three folded program for peace, based on democracy, economic dependence and supra national governance. They build the graph of all countries and connect them in three layers. All democracies form a clique in the first layer, countries are connected with weight proportional to the amount of yearly trade in the second layer and with weight proportional to the number of international organizations they belong to together in the third layer. While the essence of commerce is to exploit differential between the partners involved (i.e. heterophily), trade intensity and participating in common institutions also involve geographical, historical and cultural ties (i.e. homophily).

Biological networks

The interactions in biology also take several forms and studying them as a whole has proved fruitful. One example is a genes co-expression network, where each connection was tested under 130 different experimental conditions, providing as many layers [START_REF] Li | Integrative analysis of many weighted co-expression networks using tensor computation[END_REF]. Multilayer graphs are also a relevant way to represent ecological networks [START_REF] Pilosof | The multilayer nature of ecological networks[END_REF]. For instance, Kéfi et al. [START_REF] Kéfi | Network structure beyond food webs: mapping non-trophic and trophic interactions on chilean rocky shores[END_REF] build the graph of more than 100 species living on the Chilean coast4 and divide their interactions in three categories: trophic (i.e. one specie eating another one), positive non-trophic (e.g. refuge providing) and negative non-trophic (e.g. competition for shared resources). Finally, in neuroscience, multilayer graphs have recently emerged as a useful tool to better understand the human brain [START_REF] Muldoon | Network and multilayer network approaches to understanding human brain dynamics[END_REF]. The nodes of such graphs are neurons, and the edges can be labeled in various ways: some correspond to actual physical links while others are functional (i.e. neurons responding in the same way to external stimuli), some are present in healthy subjects and others in treated patients, some are acquired through MRI and others by EEG.

Predicting edge type

Let us summarize in one sentence the two previous sections. Learning in graphs provides many insights, and many graphs are complex, in the sense of having edges expressing different semantics and created for mixed reasons. The logical conclusion is that we want to learn in complex graphs. However, this requires the edges of such graphs to be labeled with one of k types of relationship. In the examples presented above, this was already done. Yet we argue that efficiently labeling the edges is an interesting problem, especially in the following three situations:

1. There exists a function that can perfectly label any edge because it is tailored to this specific graph, but calling it is expensive. For instance, say we want to know whether two connected users of a social network are tied to each other through family, work, school or hobby. The labeling function in this case simply asks users to label their relationships, assuming the answers do not contain any noise. At the scale of Facebook, this would require asking hundred of questions to each user on average, which is quite time-consuming. Furthermore, it would also cost marketing resource to convince users this is beneficial for them, and engineering time to ensure this information remain confidential. Likewise, in a biological network, determining whether two proteins interact positively or negatively with each other is achieved by a lab experiment, which requires time and material.

2. All edges are already labeled, perfectly and without any cost, but the number k of edge types is very large. Indeed, we have seen examples where they are hundreds or even thousands of edge types. Similarly to what happen in dimensionality reduction [START_REF] Burges | Dimension reduction: a guided tour[END_REF], to make intuitive sense of such data, we want to reduce the number of edge types to less than ten. A natural way is to ask a domain expert to cluster edge types together. However, this is again time consuming, and does not necessarily make use of the topology of the graph.

3. The input graph is actually unlabeled. That is, we observe interactions between the nodes, and we assume from domain knowledge that these interactions fall into k categories. However, there is no reasonable way to come up with a specific labeling function. In Table 4.1 on page 92, we present several such examples, but for now we simply recall our earlier co-purchase network. As we mentioned, there are several reasons why two products could be bought together. Yet it is unlikely that customers will provide this type of feedback, for it does not bring them immediate advantage.

Predicting edge type

In this thesis, we consider three versions of the problem of predicting edge types.

1. The EDGE SIGN PREDICTION problem, which takes as input the topology of a directed signed graph, and the label of some of the edges. The output is the label of the remaining edges. Therefore, it can be seen as a supervised binary classification problem.

2. The same EDGE SIGN PREDICTION problem, where the input graph is undirected. In this case, we consider an active variant where, instead of being given a random training set of labels, we have a budget of queries we can use to request arbitrary labels.

3. The EDGE ATTRIBUTED CLUSTERING problem, which takes as input an unlabeled, undirected multilayer graph, the attributes of all its nodes, and a number of edge types k. The output is a k clustering of the edges, and a set of k vectors describing the clusters in terms of node attributes. Therefore, it can be seen as an unsupervised clustering problem with side information.

We now briefly review existing approaches addressing these problems, in order to highlight our contribution in the next section.

The modern formulation of the EDGE SIGN PREDICTION on directed graphs can be attributed to Leskovec et al. [START_REF] Leskovec | Predicting positive and negative links in online social networks[END_REF]. Their idea is to compute local features of the nodes based on the training labels. Such features include variations on the node degree, such as the number of positive outgoing edges or the number of negative incoming edges. These node features are combined into edge features, and a supervised classification algorithm is trained. Several works have devised additional features [SM15; Yua+17], most notably based on scoring the nodes in a Page Rank fashion [TNV10; SJ14; WAS16]. These methods are accurate, fast -for they are local, and interpretable -for the features are hand crafted. The drawback is that this feature engineering is done mostly in an ad hoc way. On the other hand, it is also possible to look at the problem from a global point of view, by completing the signed adjacency matrix through low rank factorization [START_REF] Chiang | Prediction and clustering in signed networks: a local to global perspective[END_REF][START_REF] Wang | Online matrix completion for signed link prediction[END_REF]. These methods are also accurate, and bypass feature engineering, but by nature, they require careful algorithms to scale with larger networks. Finally, during the writing of this manuscript, several papers have been published, which use embedding of the nodes in a low dimensional space, based on the training labels [START_REF] Islam | Signet: scalable embeddings for signed networks[END_REF][START_REF] Yuan | Sne: signed network embedding[END_REF]. This also avoids feature engineering but is global in nature and not so interpretable.

One way to solve the problem on undirected graphs would be to apply the previous methods, having first replaced every edge (u, v) by two directed edges u → v and v → u. This is not quite satisfactory, both from a theoretical but also practical point of view (see Section 3.1.2). Instead, Cesa-Bianchi et al. [START_REF]A correlation clustering approach to link classification in signed networks[END_REF] draw a connection between the EDGE SIGN PREDICTION problem and the CORRELATION CLUSTERING problem. Recall that given a fully labeled signed graph, the solution to CORRELATION CLUSTERING is a partition of the nodes that minimize the number of disagreement edges [START_REF] Bansal | Correlation clustering[END_REF]. Cesa-Bianchi et al. [START_REF]A correlation clustering approach to link classification in signed networks[END_REF] assume that the signs are originally consistent with an underlying, hidden 2-clustering, but that we only observe the signs after they have been flipped uniformly at random. In this case, they show that the optimal number of disagreements is a lower bound of the number of mistakes made by any active EDGE SIGN PREDICTION algorithm. One natural approach would then be to solve CORRELATION CLUSTERING based on the observed signs, and predict the remaining signs consistently with the inferred clusters. At first, it seems hopeless, as CORRELATION CLUSTERING is difficult to approximate on general graphs [START_REF] Charikar | Clustering with qualitative information[END_REF], even when there are only two clusters [START_REF] Giotis | Correlation clustering with a fixed number of clusters[END_REF]. However, this worst case analysis does not forbid more positive results on signed graphs that exhibit stability under perturbation [Ben15; BB09; NJ09] or are obtained through perturbations from an ideal case [AL09; MMV15]. Furthermore, in the active setting, the learner gets to choose which signs are observed. The general idea of Cesa-Bianchi et al. [START_REF]A correlation clustering approach to link classification in signed networks[END_REF] is thus to use the query budget to build fully labeled paths between each connected nodes. Those paths must be as short as possible, since the predicted sign of (u, v) is the product of the signs along the path from u to v. Therefore, the shorter the path and the less influence of the random perturbations. At the same time, those paths must span the whole graph. This is the topic of an active research area [START_REF] Abraham | Using petal-decompositions to build a low stretch spanning tree[END_REF][START_REF] Elkin | Efficient algorithms for constructing very sparse spanners and emulators[END_REF].

As for the EDGE ATTRIBUTED CLUSTERING problem, to the best of our knowledge, it has not been studied under our assumptions. A more common problem in attributed graphs is to cluster nodes into communities [LM12; YML13; Xu+14a; ZLZ16]. However, it is not immediate how such methods, generally based on generative models, could be adapted to our problem. Direct approaches to classify edges have been proposed, based on graphical models [START_REF] Tang | Learning to infer social ties in large networks[END_REF], nearest neighbors with a customized distance [START_REF] Aggarwal | Edge classification in networks[END_REF] and edge embedding in knowledge graphs [START_REF] Bordes | Translating embeddings for modeling multi-relational data[END_REF]. Yet they all rely on having training labels as supervision. A simple unsupervised method is to cluster the line graph of the input graph [START_REF] Evans | Line graphs, link partitions, and overlapping communities[END_REF], but this does not take advantage of attributes. Another method uses topological features [START_REF] Ahmed | Edge role discovery via higher-order structures[END_REF] but is rather complex and thus not very interpretable. If we further assume that each edge type is associated with a Euclidean space, and that the position of nodes in one space are not correlated with their position in another space, then it is possible to recover an approximation of these Euclidean metrics in polynomial time without any supervision [START_REF] Abraham | Low-distortion inference of latent similarities from a multiplex social network[END_REF].

Outline

The contributions of this thesis are organized as follow:

• We start in Chapter 2 by addressing the EDGE SIGN PREDICTION problem in Directed Signed Social Networks. Our goal is to design a method that is scalable, principled and accurate. For that, we start by introducing a generative model for signs, and derive approximations of the optimal Bayes predictor and of the maximum likelihood predictor. We confirm the theoretical soundness of these approaches by performing extensive synthetic and real world experiments. Finally, to the best of our knowledge, we are the first to give an online algorithm for the EDGE SIGN PREDICTION problem on directed graphs. This chapter is based on an existing publication [Le +17].

• We then move from directed to undirected signed graphs, and explain in Chapter 3 why this requires another learning bias. Namely, we assume that the nodes belong to k groups, and that the sign between two nodes is positive when both nodes belong to the same group, negative otherwise. We describe how this new bias is related to the CORRELATION CLUSTERING problem, and present a thorough overview of existing approaches. Finally, we provide the first implementation of an existing spanner construction [START_REF] Vitale | [END_REF], and give a preview of its performance on synthetic and real signed graphs.

• In Chapter 4, we consider the EDGE ATTRIBUTED CLUSTERING problem on multilayer graphs with node attributes. Our approach is to seek a small number of vectors that, once assigned to every edge, best explain the graph (i.e. maximize a score function between the vector assigned to an edge and the profiles of this edge's endpoints). From this initial formulation, we derive two optimization problems, and show how to solve them on synthetic data.

• Finally, reflecting on our treatment of the three previous problems, we discuss in Chapter 5 other settings and methods that might extend the problem of characterizing edges in complex networks beyond the frame of this thesis.

Chapter 2

On the Troll-Trust Model for edge sign prediction in Social Networks

As we saw in the introduction, many situations in various fields can be modeled by signed graphs. Nonetheless, one of the most natural usage of signed graphs is the study of social networks. It is indeed what first motivated their inception. Quite naturally, we thus begin our exploration of the role of edges in complex networks by a binary classification problem in signed graphs. Namely, we want to predict whether an observed interaction between two nodes is positive or negative, and we call this problem EDGE SIGN PREDICTION. 1 To motivate this problem from a practical point of view, let us first define precisely the type of social networks we consider. We call Directed Signed Social Network (DSSN) a directed graph whose nodes are either human beings (often called users if they are the members of an online community) or artifacts directly created by individuals2 . Furthermore, a directed edge from node u to node v denote an interaction initiated by the user u and whose object is user v. Such an interaction can be positive (to praise, to support, to trust, to befriend) or negative (to criticize, to oppose, to distrust, to make enemy with). Known examples are EBAY, where users trust or distrust agents in the network based on their personal interactions, SLASHDOT, where each user can tag another user as friend or foe, BLABLACAR, a carpooling website where users can evaluate drivers and passengers as pleasant travelers or not, and EPINION, where users can rate positively or negatively not only products, but also other users. In such examples, the sign of the interaction is explicit. However, there are other cases where such interactions are only displayed in their raw form and need further processing to be given a sign. The typical situation where this happens is when interactions are mediated through text, for instance in Twitter or in the comment section of some online content. It is then possible to extract signs using sentiment analysis methods [START_REF] Hassan | Extracting signed social networks from text[END_REF].

Another distinction is that in most cases, only the owner of the network can observe the detail of all those interactions. Think for instance of YOUTUBE, where users can like or dislike the video of another user but where only aggregate anonymous statistics are available publicly. The situation is similar on the STACK OVER-FLOW community, where users can upvote or downvote answers according to their perceived quality, but where only the total score are available.

With this definition of social networks at hand, we now look at how the learning problem of EDGE SIGN PREDICTION can be used in real world applications. More precisely, we present four possible applications, and order them by how severely negative interactions impact the well-being of users of the network. First, it could help improve the quality of link recommendation in solely positive networks.

Chapter 2. On the Troll-Trust Model for edge sign prediction in Social Networks Such networks indeed contain implicit negative links [START_REF] Yang | Friend or frenemy?[END_REF]. By asking users to label a small proportion of existing links as truly positive or actually negative, recommended links could later be discarded if they are predicted to be negative. Second, this could be used to monitor news textual comments in a scalable fashion and thus ensure that online debates remain courteous and constructive. For instance, Manosevitch et al. [START_REF] Manosevitch | Promoting online deliberation quality: cognitive cues matter[END_REF] show that by visually reminding comment authors of the importance of respecting the plurality of opinions, the quality of debates is improved (other references can be found in [START_REF] Diakopoulos | Picking the nyt picks: editorial criteria and automation in the curation of online news comments[END_REF]). In a more extreme case, banning users with aggressive and hateful behavior has proved effective in the case of Reddit [START_REF] Chandrasekharan | You can't stay here: the efficacy of reddit's 2015 ban examined through hate speech[END_REF]. Third, like many other fields, human resource management is undergoing a transformation through the increasing use of Machine Learning [START_REF] Tomassen | Exploring the black box of machine learning in human resource management: an hr perspective on the consequences for hr professionals[END_REF]. Among other tools, assessing personal relationships between employees is crucial to maintain a productive work environment [START_REF] Mohammed | Technical-administrative task performance, leadership task performance, and contextual performance: considering the influence of team-and task-related composition variables[END_REF]. Whereas this can be done by invasive wearable sensors [START_REF] Olguín | Capturing individual and group behavior with wearable sensors[END_REF], our methods could predict which employees would form the tighter teams from a small amount of labeled data. Fourth, and most critical, predicting the sign of interactions could be a tool to detect users with a high proportion of negative interactions. Indeed, because people tend to be less inhibited in their online interactions [START_REF] Suler | The online disinhibition effect[END_REF], some users may join an online community with the main goal to disrupt it, by engaging into anti-social behavior and creating conflictual relationships with other members. This kind of attitude expressed publicly on social media leads to the following definition of trolls: "users whose real intentions are to cause disruption and/or to trigger or exacerbate conflict for the purposes of their own amusement" [START_REF] Hardaker | Trolling in asynchronous computer-mediated communication: from user discussions to academic definitions[END_REF]. Shachaf et al. [START_REF] Shachaf | Beyond vandalism: wikipedia trolls[END_REF] elaborate on their motives, adding that boredom, attention seeking, and revenge motivate trolls; they find pleasure from causing damage to other people or to the community as a whole. An extreme form of trolling is bullying (i.e. the behavior of someone intentionally and repeatedly harming a victim that is unable to defend himself or herself) and just like anything else, it has an online version called cyber bullying [START_REF] Slonje | The nature of cyberbullying, and strategies for prevention[END_REF]. Such behavior is rather widespread, one study revealing that 72% of 1 454 surveyed teenagers reported at least one occurrence of cyberbullying, most of them through instant messaging [START_REF] Juvonen | Extending the school grounds?-bullying experiences in cyberspace[END_REF]. While we have not conducted experimental study on this topic, we believe that being able to predict such harmful interactions would be beneficial for the majority of users.

Such applications motivate studying EDGE SIGN PREDICTION, which is the problem of classifying the positive or negative nature of the links based on the network topology. Prior knowledge of the network topology is often a realistic assumption, for in several situations the discovery of the link sign can be more costly than acquiring the topological information of the network. For instance, when two users of an online social network communicate on a public web page, we immediately detect a link. Yet, the classification of the link sign as positive or negative may require more involved techniques.

From the modeling and algorithmic viewpoints, because of the huge amount of available networked data, a major concern in developing learning methods for EDGE SIGN PREDICTION is algorithmic scalability. Many successful, yet simple heuristics for EDGE SIGN PREDICTION are based on the troll-trust features, i.e. on the fraction of outgoing negative links (trollness) and incoming positive links (trustworthiness) at each node. We first define these notions and more notations in Section 2.1, along with others tools and precise problem statements. We also introduce suitable graph transformations defining reductions from EDGE SIGN PREDICTION to node sign prediction problems. Then we study such troll-trust heuristics by defining in Section 2.2 a probabilistic generative model for the signs on the directed links of a given network. We also show that these heuristics can be understood and analyzed as approximators to the Bayes optimal classifier for our generative model. In this context, we design our first batch algorithm and show in Section 2.3.1 that it provably approximates the Bayes classifier on dense graphs.

We furthermore gather empirical evidence supporting our probabilistic model by observing that a logistic model trained on trollness and trustworthiness features alone is able to learn weights that, on all datasets considered in our experiments, consistently satisfy the properties predicted by our model.

Our graph transformations opens up the possibility of using the arsenal of known algorithmic techniques developed for node classification. In particular, we introduce in Section 2.3.2 our second batch algorithm. It takes the form of a Label Propagation algorithm that, combined with our problem reduction, approximates the maximum likelihood estimator of our probabilistic generative model. In order to compare our two algorithms with existing work, we then describe with more details previous heuristics and related methods in Section 2.4. We then experimentally evaluate our proposed approach in Section 2.5. On synthetic data, we confirm the quality of our approximation of the Bayes predictor. More importantly, on real-world data, we show the competitiveness of our approach in terms of both prediction performance (especially in the regime where training data are scarce) and scalability.

Finally, in Section 2.6, we point out that the notions of trollness and trustworthiness naturally define a measure of complexity, or learning bias, for the signed network that can also be used to design online (i.e. sequential) learning algorithms for the EDGE SIGN PREDICTION problem. The learning bias encourages settings where the nodes in the network have polarized features (e.g., trollness/trustworthiness are either very high or very low). Our online analysis holds under adversarial conditions, namely, without any stochastic assumption on the assignment of signs to the network links.

Notation and Preliminaries

In this chapter 3 , we let G = (V, E) be a directed graph, whose edges (u, v) ∈ E carry a binary label y u,v ∈ {-1, +1}. The edge labeling will sometimes be collectively denoted by the

|V | × |V | matrix Y = [y u,v ], where Y u,v = y u,v if (u, v) ∈ E,
and y u,v = 0 otherwise. The corresponding edge-labeled graph will be denoted by G(Y ) = (V, E(Y )). We use E in (u) and E out (u) to denote, respectively, the set of edges incoming to and outgoing from node u ∈ V , with d in (u) = E in (u) and d out (u) = E out (u) being the in-degree and the out-degree of u. Moreover, d + in (u) is the number of edges (w, u) ∈ E in (u) such that y w,u = +1. We define d - in (u), d + out (u), and d - out (u) similarly, so that, for instance, d - out (u)/d out (u) is the fraction of outgoing edges from node u whose label in G(Y ) is -1. We call tr(u) = d - out (u)/d out (u) the trollness of node u, and un(u) = d - in (u)/d in (u) the untrustworthiness of node u. Finally, we also use the notation N in (u) and N out (u) to represent, respectively, the in-neighborhood and the out-neighborhood of node u ∈ V . Most of these notations are illustrated on Figure 2.1.

Given the directed graph G = (V, E), we define two edge-to-node reductions transforming the original graph G into other graphs. As we see later, these reductions are useful in turning the EDGE SIGN PREDICTION problem into a node sign prediction problem (often called node classification problem), for which many algorithms are indeed available [BC01; ZGL03; BDL06; HP07; HLP09; Vit+11; HPV12; Ces+13; HPG15]. Although any node classification method could in principle be used, the reductions we describe next are essentially aimed at preparing the ground for quadratic energy-minimization approaches computed through a Label Propagation algorithm [START_REF] Zhu | Semi-supervised learning using gaussian fields and harmonic functions[END_REF][START_REF] Bengio | Label propagation and quadratic criterion[END_REF].

The first reduction builds an undirected graph G = (V , E ) as follows. Each node u ∈ V has two copies in V , call them u in and u out . Each directed edge (u, v) in E is associated with one node, call it e u,v , in V , along with the two undirected edges (u out , e u,v ) and (e u,v , v in ).

Hence |V | = 2|V | + |E| and |E | = 2|E|. Moreover, if G =
3 And in the next chapter when applicable.

13

Chapter 2. On the Troll-Trust Model for edge sign prediction in Social Networks - G(Y ) is edge labeled, then this labeling transfers to the subset of nodes e u,v ∈ V , so that G is a graph G (Y ) = (V (Y ), E ) with partially-labeled nodes. The second reduction builds an undirected and weighted graph G = (V , E ). Specifically, we have V ≡ V and E ⊃ E , where the set E also includes edges (u out , v in ) for all u and v such that (u, v) ∈ E. The edges in E have weight 2, whereas the edges in E \E have weight -1. We provide in Section 2.9.1.3 an analytic justification for this weights choice in the context of Label Propagation. Finally, as in the first reduction, if G = G(Y ) is edge labeled, then this labeling transfers to the subset of nodes e u,v ∈ V . Graph G , which will not be used in the following, is an intermediate structure between G and G and provides a conceptual link to the standard cutsize measure in node sign classification, as we will describe shortly. Figure 2.2 illustrates these two reductions. Note that because |V | = 2|V | and |E | = 3|E|, the reduction yielding G can be computed in linear time, and does not require the knowledge of the edge label, meaning it can be done in parallel with potential label querying. Furthermore, it can be updated incrementally if new nodes or edges are added to the original graph G. These reductions serve two purposes: First, they allow us to use the many algorithms designed for the better studied problem of node sign prediction. Second, the reduction yielding G , along its specific choice of edge weights, is designed to make the Label Propagation solution approximate the maximum likelihood estimator associated with our generative model (see Section 2.3.2). Note also that efficient Label Propagation implementations exist that can leverage the sparsity of G .

+ N out (u) E out (u) N in (u) E in (u) u v + y u,v =+ + - - - + tr(u) = 1 4 un(u) = 3 5 Figure 2.1 -Part of a DSSN centered on node u. 1 2 3 4 + + - - + - (a) 1 out 2 out 3 out 4 out 1 in 2 in 3 in 4 in 1 -1 -1 1 -1 1 (b) 1 out 2 out 3 out 4 out 1 in 2 in 3 in 4 in 1 -1 -1 1 -1 1 2 2 2 2 2 2 2 2 2 2 2 2 -1 -1 -1 -1 -1 -1 (c)
We consider two learning settings associated with the problem of EDGE SIGN PREDICTION: a batch setting and an online setting. In the batch setting, we assume that a training set of edges E 0 has been drawn uniformly at random without replacement from E, we observe the labels in E 0 , and we are interested in predicting the sign of the remaining edges E \E 0 by making as few prediction mistakes as possible.

The specific batch setting we study here assumes that labels are produced by a generative model which we describe in the next section, and our label regularity measure is a quadratic function (denoted by Ψ 2 G (Y ) and defined in Section 2.5 as a regularized energy function of G 4 ) related to this model. Ψ 2 G (Y ) is small just when all nodes in G tend to be either troll or trustworthy.

On the other hand, the online setting we consider is the standard mistake bound model of online learning [START_REF] Littlestone | Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm[END_REF] where all edge labels are assumed to be generated by an adversary and sequentially presented to the learner according to an arbitrary permutation. More precisely, in this setting, learning is split into a sequence of rounds: At each round t = 1, 2, . . . a learning algorithm A outputs a prediction y t ∈ {-1, +1} for the label y t ∈ {-1, +1} of an edge arbitrarily selected by the adversary. After the prediction is made, label y t is revealed to the algorithm, hence allowing it to change its internal state. In the next round, a new edge is selected by the adversary, and so on. Notice that since the underlying labeling over the edges is decided by the adversary once and for all, 5 all edges occur exactly once within the sequence to be predicted (so that this game lasts exactly |E| rounds). The adversary decides both the underlying labeling Y over the edges of G and the order of their presentation to the learning algorithm. We say that A has made a mistake at time t if y t = y t , and we measure A's prediction performance simply through the total number of mistakes M A (Y ) it makes over G(Y ) when the worst possible presentation order of the edge labels in Y is selected by the adversary. As it is standard practice, we contrast M A (Y ) to some kind of regularity measure Ψ G (Y ) of the labeling Y over G, so that we are in fact aimed at bounding the (cumulative

) regret M A (Y ) -Ψ G (Y ) .
We want to design our label regularity measure such that it is small when nodes in G tend to be either troll or trustworthy. Indeed, when this happens, few labels from the incoming and outgoing edges of each node are sufficient to predict the labels on the remaining edges and the EDGE SIGN PREDICTION problem is to some extent "easy". Formally, for fixed G and Y , let

Ψ in (v, Y ) = min d - in (v), d + in (v) and Ψ out (u, Y ) = min d - out (u), d + out (u)
be respectively the number of the least used label in the incoming edges to v and the outgoing edges from u. Let also

Ψ in (Y ) = v∈V Ψ in (v, Y ) and Ψ out (Y ) = u∈V Ψ out (u, Y )
be the sum of irregularity over all the nodes of the graph. Then we define Ψ G (Y ) = min Ψ in (Y ), Ψ out (Y ) . The two measures Ψ G (Y ) and Ψ 2 G (Y ) are conceptually related and their value on real data is quite similar (see Table 2.4 on page 25).

The reductions presented above are meaningful only if they are able to approximately preserve the two label regularity measures Ψ G (Y ) and Ψ 2 G (Y ) when moving from edges to nodes. That is, if the EDGE SIGN PREDICTION problem is easy for a given G(Y ) = (V, E(Y )), then the corresponding node sign prediction problems on G (Y ) = (V (Y ), E ) and G (Y ) = (V (Y ), E) are also easy, and vice versa. 4 If we denote the value of the square node eu,v of G as 1+yu,v 2

, the value of the circle node uout as pu and the value of the circle node vin as qv, we have that

Ψ 2 G (Y ) = min (p,q) (u,v)∈E 1+yu,v 2 -pu+qv 2 2
, where (p, q) = {pu, qu}

|V |
u=1 . 5 For simplicity, we assume the adversary is deterministic. Let us first introduce the notion of cutsize of an undirected node-labeled graph G (Y ), which is the number of edges in G connecting nodes that have mismatching labels. Now, because the only nodes in G we are interested in predicting are those corresponding to the edges in G (the colored squares in Figure 2.2b), the online prediction problem on the edges of G translates to a node sign prediction problem on a subset of V . As for the remaining nodes in V (the circles in Figure 2.2b), we are free to assign arbitrary labels so as to minimize the corresponding mistake bound over G . Such an assignment is constructed as follow: we set the labeling Y on V in such a way that y uout = +1 if tr(u) ≤ 1 /2 and -1 otherwise. Similarly, we have y u in = +1 if un(u) ≤ 1 /2, and -1 otherwise. For instance, given the trollness and untrustworthiness values for the graph G of Figure 2.2a reported in Table 2

1 out +1 2 out -1 3 out +1 4 out +1 1 in +1 2 in +1 3 in +1 4 in +1 1 -1 -1 1 -1 1 Figure 2.
Ψ out (u, Y ) 0 0 1 0 1 Ψ in (u, Y ) 1 1 0 1 3 tr(u) 0 1 1 /3 1 - un(u) 1 /2 1 /2 1 /2 1 /2 -

Generative Model for Edge Labels

u (pu, qu) ∼ µ(p, q) v (pv, qv) ∼ µ(p, q) Pr(yu,v = +1) = 1 2 (pu + qv) Figure 2.4 -The sign y u,v of the edge u → v is positive with prob- ability 1 2 (p u + q v ).
We now define the stochastic generative model for edge labels we use in the batch learning setting. Given the graph G = (V, E), let the label y u,v ∈ {-1, +1} of directed edge (u, v) ∈ E be generated as follows. Each node u ∈ V is endowed with two latent parameters p u , q u ∈ [0, 1], which we assume to be generated, for each node u, by an independent draw from a fixed but unknown joint prior distribution µ(p, q) over [0, 1] 2 . Each label y u,v ∈ {-1, +1} is then generated by an independent draw from the mixture of p u and q v Pr y u,v = 1 = pu+qv 2 This process is illustrated in Figure 2.4.

The basic intuition is that the nature y u,v of a relationship u → v is stochastically determined by a mixture between how much node u tends to like other people (p u ) and how much node v tends to be liked by other people (q v ). In a certain sense, 1tr(u) is the empirical counterpart to p u , and 1un(v) is the empirical counterpart to q v . One might view our model as reminiscent of standard models for link generation in social network analysis, like the classical p 1 model from [START_REF] Holland | An exponential family of probability distributions for directed graphs[END_REF]. However, note that the similarity falls short, for all these models aim at representing the likelihood of the network topology, rather than the probability of edge signs, once the topology is given.

Notice that the Bayes optimal prediction for y u,v is y * (u, v) = sign η(u, v) -1 2 , where η(u, v) = Pr y u,v = 1 . Moreover, once all the signs have been generated, we can compute the expected number of positive edges outgoing from u as the expected value of the random variable P u = v∈Nout(u) I {y u,v = +1}. Using this, we have that the probability of drawing at random a +1-labeled edge from E out (u) equals

E (P u ) d out (u) = 1 d out (u) v∈Nout(u) p u + q v 2 = 1 2 p u + 1 d out (u) v∈Nout(u) q v = 1 2 (p u + q u ) ,
(2.1) where q u = 1 dout(u) v∈Nout(u) q v is the average q value of u out neighbors. Similarly, the probability of drawing at random a +1-labeled edge from E in (v) equals

1 2 q v + 1 d in (v) u∈N in (v) p u = 1 2 (q v + p v ) (2.2)

Algorithms in the Batch Setting

Given G(Y ) = (V, E(Y )), in the batch setting we have at our disposal a training set E 0 of labeled edges from E(Y ). Formally, we have a training set E 0 = (u 1 , v 1 ), y u 1 ,v 1 ), ((u 2 , v 2 ), y u 2 ,v 2 ), . . . , ((u m , v m ), y um,vm , that has been drawn from E × Y uniformly at random without replacement, with m = |E 0 |. We want to build a predictive model for the labels of the remaining edges. We present two algorithms to do so, which both compute estimates of the all parameters p and q of our generative model. They differ in the approximation guarantees they provide, and in the class of graphs to which they apply. The first algorithm runs on the graph G and estimates locally the parameters by their empirical means in the training set, which under some density assumptions are showed to concentrate around their true values. The second algorithm, on the other hand, exploits the reduced graph G and computes a maximum likelihood estimation of the parameters through a global label propagation approach, without making any density assumption.

Approximation to Bayes via dense sampling

Our first algorithm is an approximation to the Bayes optimal predictor y * (u, v). Let us denote by tr(u) and un(u) the trollness and the untrustworthiness of node u when both are computed on the subgraph induced by the training edges. Recall that the Bayes optimal predictor classifies an edge u → v using the following rule:

y * (u, v) = sign pu+qv 2 -1 2 .
Our approximation thus consists in using the quantities tr(u) and un(u) to estimate pu+qv 2 . This results in the following rule:

sign 1 -tr(u) + 1 -un(v) -τ -1 2 , (2.3)
where τ ≥ 0 is the only parameter to be trained. We now give an intuition to justify this equation, and defer the technical arguments to end of the chapter, on page 37, where we will formalize what it means for a quantity to be "close" to another.

Note first that 1tr(u) =

d + out (u) dout(u)
is the empirical mean 7 of the probability of a random edge outgoing from u to be positive, and is therefore "close" to 1 2 (p u + q u ) according to (2.1). By the same reasoning, 1un(v) ≈ 1 2 (q v + p v ) in accordance with (2.2). At this stage, 1tr(u) + (1un(v)) -pu+qv 2 ≈ 1 2 (p v + q u ). Since p v is a sample mean of i.i.d. [0, 1]-valued random variables independently drawn from the prior marginal 1 0 µ •, q dq, it concentrates around its expectation µ p . Likewise, q u ≈ µ q . Now we seek an estimation of 1 2 (p v + q u ) ≈ 1 2 (µ p + µ q ). For that, we compute the expected number of positive edges in the graph, which, as in the previous section, is the expected value of the random variable Z = u→v∈E I {y u,v = +1}. Let V out be the subset of node of V with at least one outgoing edge, and define similarly V in as {v ∈ V :

d in (v) > 0}. According to our generative model, E (Z) = u→v∈E pu+qv 2 = u∈Vout v∈Nout(u) pu+qv 2 . Observe that u∈Vout   v∈Nout(u) p u   = v∈V in   u∈N in (v) p u   = v∈V in d in (v)p v ≈ v∈V in d in (v)µ p = |E|µ p and u∈Vout   v∈Nout(u) q v   = u∈Vout d out (u)q u ≈ u∈Vout d out (u)µ q = |E|µ q .
Therefore, letting τ be the fraction of positive edge in the graph, we have that

τ = E(Z) |E| ≈ 1 2 (µ p + µ q )
. This means that if τ is the quantity τ computed on the training, then τ is close to (p v + q u ). Putting together all this approximations thus provide an informal justification of the rule (2.3).

We are now ready to describe our algorithm, which follows naturally from the formula derived above. It takes as input a training set E 0 drawn at random without replacement and outputs a sign prediction for all the edges in E \ E 0 . We call it BLC(tr, un), which stands for Bayes Learning Classifier based on trollness and untrustworthiness.

1.

For each u ∈ V , let tr(u) = d - out (u)/ d out (u), i.e. the fraction of negative edges found in E out (u) ∩ E 0 .

For each

v ∈ V , let un(v) = d - in (v)/ d in (v), i.
e. the fraction of negative edges found in E in (v) ∩ E 0 .

3. Let τ be the fraction of positive edges in E L ∩ E 0 (this set E L is a technical requirement that we shall define shortly).

Any edge

(u, v) ∈ E \ E 0 is predicted as y(u, v) = sign 1 -tr(u) + 1 -un(v) -τ -1 2
Despite its apparent simplicity, BLC(tr, un) works reasonably well in practice, as demonstrated by our experiments (see Section 2.5). Moreover, unlike previous edge sign prediction methods for directed graphs, our classifier comes with a rigorous theoretical motivation, since it approximates the Bayes optimal classifier y * (u, v) with respect to the generative model defined in Section 2.2. Indeed, we quantify this approximation on nodes whose in-degree and out-degree are not too small in the next result. However, this first requires a few extra assumption on the training set. Namely, given parameters Q (a positive integer) and α = |E 0 | /|E| ∈ (0, 1), we assume there exists a set E L ⊆ E of size 2Q α where each vertex u ∈ V appearing as an endpoint of some edge in E L occurs at most once as origin -i.e. u → vand at most once as destination -i.e. v → u. While the definition of E L is not immediately intuitive, this set is needed to find an estimate τ of τ in (2.3) during

Step 3 of BLC(tr, un), and its definition allows us to applying an Hoeffding bound on independent variables. Any undirected matching of G of size O(log |V |) can be used, obtained for instance by the blossom algorithm [START_REF] Edmonds | Paths, trees, and flowers[END_REF]. In practice, however, we never computed E L , and estimated τ on the entire training set E 0 (instead of E L ∩ E 0 ). Theorem 1. Let G(Y ) = (V, E(Y )) be a directed graph with labels on the edges generated according to the model in Section 2.2. If the algorithm is run with parameter Q = Ω(log |V |), and α ∈ (0, 1) such that the above assumption about E L is satisfied, then y(u, v) = y * (u, v) holds with high probability simultaneously for all test edges (u, v) ∈ E such that d out (u), d in (v) = Ω(log |V |), and η(u, v) = Pr(y u,v = 1) is bounded away from 1 2 .

While we defer the full proof of Theorem 1 to the additional material at the end of this chapter (Section 2.9.1.1), here we give a sketch of the method used. First, by Lemma 2 on negatively associated random variables (stated on page 37), we show that with our choice of Q, for θ = 2Q α and for any u having enough out neighbors (i.e. d out (u) ≥ θ), at least Q edges of E out (u) are in the training set with high probability. Likewise, for any v with enough in neighbors (i.e. d in (v) ≥ θ), at least Q edges of E in (v) are in E 0 with high probability. With this number of samples, we then show a chain of concentration results roughly following the outline given earlier to justify the equation (2.3), culminating in proving that for any 0 < ε < 1 16 ,

1 -tr(u) + (1 -un(v)) -τ - p u + q v 2 ≤ 8ε
simultaneously holds with high probability for each

(u, v) ∈ {(u, v) ∈ E : d in (v) ≥ θ, d out (u) ≥ θ} \ E 0 .
The approach leading to Theorem 1 requires the graph to be sufficiently dense. At first sight, having Q = Ω(log |V |) training edges per nodes appears to be a reasonable assumption. Consider for instance Facebook -which is neither signed nor directed though. It has two billion users as of 20178 , each of them having 155 ≈ 7.2 log |V | friends on average [START_REF] Dunbar | Do online social media cut through the constraints that limit the size of offline social networks?[END_REF]. However, the constant in the Ω notation is a trade off between the number of edges to sample per node, and the quality guarantee of the pu+qv 2 estimation. In the Facebook example, having a good guarantee that holds simultaneously for all the test edges might require more than 7.2 log |V | samples per node. We will nonetheless see in the experiments that we can still apply BLC(tr, un) with satisfying results, especially since its simplicity makes it very scalable. Additionally, and in order to sidestep this density limitation, we now introduce a second method based on label propagation.

Approximation to Maximum Likelihood via Label Propagation

Remember we suppose that the training set E 0 has been drawn uniformly at random without replacement, with m = |E 0 |. Then a reasonable approach to approximate y * (u, v) would be to resort to a maximum likelihood estimator of the parameters {p u , q u } |V | u=1 based on E 0 . If we further assume, in order to make the computation more tractable, that the joint prior distribution µ(p, q) is uniform over [0, 1] 2 with independent marginals,9 we show in the supplementary material on page 40 that for ∈ {1, . . . , |V |} the gradient of the log-likelihood function with respect to a given p and q satisfies

∂ log Pr E 0 {p u , q u } |V | u=1 ∂p = m k=1 I {u k = , y ,v k = +1} p + q v k - m k=1 I {u k = , y ,v k = -1} 2 -p -q v k (2.4) ∂ log Pr E 0 {p u , q u } |V | u=1 ∂q = m k=1 I {v k = , y u k , = +1} p u k + q - m k=1 I {v k = , y u k , = -1} 2 -p u k -q ,
(2.5) where I {•} is the indicator function of the event at argument. Unfortunately, equating (2.4) and (2.5) to zero, and solving for parameters {p u , q u } |V | u=1 gives rise to a hard set of nonlinear equations. Moreover, some such parameters may never occur in these equations, namely whenever E out (u) or E in (v) are not represented in E 0 for some u, v ∈ V .

Our first approximation is therefore to replace the nonlinear equations resulting from (2.4) and (2.5) by a set of linear equations. In the case of (2.4), for a given ∈ V , we make the assumption that

(p + q v k )(2 -p -q v k ) is constant for every k ∈ [1, . . . , m].
Multiplying (2.4) by this constant quantity and setting the resulting equation to zero, we obtain for each ∈ V :

m k=1 I {u k = , y ,v k = +1} (2 -p -q v k ) = m k=1 I {u k = , y ,v k = -1} (p +q v k ) (2.6)
We apply a similar transformation to (2.5) in order to obtain, for each ∈ V :

m k=1 I {v k = , y u k , = +1} (2 -p u k -q ) = m k=1 I {v k = , y u k , = -1} (p u k + q )
(2.7) At this point, we find convenient to take a step back and define one of the label regularity measure we introduced in Section 2.1. Namely, recall we said Ψ 2 G (Y ) could be seen as a soft quadratic version of the cutsize. With the notation of our generative model, we can write it as

Ψ 2 G (Y ) = min (p,q) (u,v)∈E 1 + y u,v 2 - p u + q v 2 2 = min (p,q) f E (p, q) ,
where (p, q) = {p u , q u } |V | u=1 is the set of the model parameters. Intuitively, minimizing f E with respect to (p, q) is similar to the maximum likelihood approach, as we seek the parameters (p, q) that "best agree" with the observed signs. It also turns out that if we restrict the minimization problem to the observed edges (i.e. min (p,q) f E 0 (p, q)) and set the derivative of f E 0 with respect to p and q to zero, we recover the equations (2.6) and (2.7) respectively.

To include the full topology of the graph and not restrain ourselves to the observed edges, we follow a label propagation approach by adding to f E 0 the corresponding test set function f E\E 0 , and treat the sum of the two as the function to be minimized during training w.r.t. both (p, q) and all

y u,v ∈ [-1, +1] for (u, v) ∈ E \ E 0 : min (p,q),yu,v∈[-1,+1], (u,v)∈E\E 0 f E 0 (p, q) + f E\E 0 (p, q) (2.8)
Binary ±1 predictions on the test set E \ E 0 are then obtained by thresholding the computed values y u,v .

We now proceed to solve (2.8) via label propagation [START_REF] Zhu | Semi-supervised learning using gaussian fields and harmonic functions[END_REF] on the graph G obtained through the second reduction of Section 2.1. Indeed, one can show 10 that this objective is equal -up to a regularization term-to the quadratic energy objective minimized by label propagation methods. However, because of the presence of negative edge weights in G , we first have to symmetrize 11 variables p i , q i and y u,v so as they all lie in the interval [-1, +1]. After this step, one can see that, once we get back to the original variables, label propagation computes the harmonic solution minimizing the function

f p, q, y u,v (u,v)∈E\E 0 =f E 0 (p, q) + f E\E 0 (p, q)+ 1 2 u∈V d out (u) p u - 1 2 2 +d in (u) q i - 1 2 2
The function f is thus a regularized version of the target function f E 0 + f E\E 0 in (2.8), where the regularization term tries to enforce the extra constraint that whenever a node u has a high out-degree then the corresponding p u should be close to 1 /2. Thus, on any edge (u, v) departing from u, the Bayes optimal predictor y * (u, v) = sign(p u + q v -1) will mainly depend on q v being larger or smaller than 1 /2 (assuming v has small in-degree). Similarly, if u has a high in-degree, then the corresponding q u should be close to 1 /2, implying that on any edge (v, u) arriving at u, the Bayes optimal predictor y * (v, u) will mainly depend on p v (assuming v has small out-degree). Put differently, a node having a huge out-neighborhood makes each outgoing edge "count less" than a node having only a small number of outgoing edges, and similarly for in-neighborhoods.

The label propagation algorithm operating on G does so (see again Figure 2.2c) by iteratively updating as follows:

p u ← -v∈Nout(u) q v + v∈Nout(u) (1 + y u,v ) 3 d out (u) ∀u ∈ V q v ← -u∈N in (v) p u + u∈N in (v) (1 + y u,v ) 3 d in (v) ∀v ∈ V y u,v ← p u + q v 2 ∀(u, v) ∈ E \ E 0 .
The algorithm is guaranteed to converge [START_REF] Zhu | Semi-supervised learning using gaussian fields and harmonic functions[END_REF] to the minimizer of f . Notice that the presence of negative weights on the edges of G does not prevent label propagation from converging. In fact, any node classification algorithm handling both positive and negative weights on the edges of G could be used instead of label propagation. One alternative would thus be the WTA algorithm from [START_REF]Random spanning trees and the prediction of weighted graphs[END_REF]. However, our label propagation algorithm is the one we will be championing in our experiments of Section 2.5.

Related work

Interest in signed networks can be traced back to the psychological theory of structural balance [CH56; Hei58] and its weak version [START_REF] Davis | Clustering and structural balance in graphs[END_REF], that we will describe with more details in Section 3.1.3 on page 48. The advent of online signed social networks has enabled a more thorough and quantitative understanding of that phenomenon. In this section, we provide an overview of methods tackling the same EDGE SIGN PREDICTION problem as us. Along the way, we give five of them a name in small capitals, for they are recent and effective. Therefore, we will compare our approaches with those methods in the experiments of the next section. At the end, we mention some closely related variants of the original EDGE SIGN PREDICTION problem.

Existing methods can be broadly divided into three strategies, which share some similarities:

1) embedding the nodes of the graph in a low dimensional space using spectral or neural techniques, before using node positions as features for a classifier;

2) completing the adjacency matrix through a global optimization algorithm; and 3) computing local features of the edges with several heuristics and train a classifier such as logistic regression or SVM. In the first direction, the spectral embedding is illustrated by works from Kunegis et al. [START_REF] Kunegis | The slashdot zoo: mining a social network with negative edges[END_REF] and Zheng et al. [START_REF] Zheng | Spectral embedding of signed networks[END_REF]. We shall describe them more thoroughly in Section 3.2.4, but here we note that, following the natural orientation of the study of graph spectrum, they focus more on clustering than EDGE SIGN PRE-DICTION. Furthermore, the use of the adjacency matrix usually requires a quadratic running time in the number of nodes, which makes those methods hardly scalable to large graphs. More recently, there has been great interest in adapting word embedding techniques such as word2vec [START_REF] Mikolov | Distributed representations of words and phrases and their compositionality[END_REF] to a "corpus" of random walks that are considered as documents, while nodes play the role of word. This allows the unsupervised extraction of node features that can then used to train downstream classifier, see [CO16; Wil17; CZC17] for three recent surveys. It can also be tailored to exploit the specificity of signed graphs. The goal is to find for every node u a vector x u ∈ R d in such a way that in this new space, nodes are close to their positive neighbors and far from their negative neighbors. For instance in SiNE [START_REF] Wang | Signed network embedding in social media[END_REF], for a node triplet u, v, w such that the edge (u, v) is positive and the edge (u, w) is negative, the objective is to find vectors maximizing f (x u , x w ) + δf (x u , x v ). The similarity function f is chosen to be a Siamese multilayers neural network, whose parameters are learned by back propagation. In SIGNet [START_REF] Islam | Signet: scalable embeddings for signed networks[END_REF], the similarity function is a sigmoid whose argument x T u x v is weighted by the sign of the corresponding edge, with an optimization technique closer to word2vec. Finally, SNE [START_REF] Yuan | Sne: signed network embedding[END_REF] uses a log-bilinear model, and the training objective is to predict, given a path P, which node should follow P, weighting differently the vector nodes in P depending of whether they are the source of a positive or negative edge in P.

Next we look at matrix completion approaches, which are global by nature. For instance, Chiang et al. [START_REF] Chiang | Prediction and clustering in signed networks: a local to global perspective[END_REF] tackle the EDGE SIGN PREDICTION problem through this lens, restricting themselves to undirected graphs. They consider the observed adjacency matrix A, made of the edges in the training set E 0 , as a noisy sampling of the adjacency matrix A of an underlying complete graph satisfying the weakly balance condition (that is with no cycle containing only one negative edge, see Definition 3.1.4 on page 49). This condition implies the existence of a small number k of node clusters with positive links within clusters and negative links across clusters, which in turn implies rank A = k. By recovering a complete matrix à that matches the non-zeros entries of A, it is possible to predict the sign of (u, v) / ∈ E 0 as y u,v = sign( Ãu,v ). Although the exact version of this problem is NP-HARD, the authors assume that k is an hyperparameter known beforehand and look for two matrices W, H ∈ R k×|V | that minimise a sigmoidal reconstruction loss of A, subject to a nuclear norm regularization term. The minimization is carried out by Stochastic Gradient Descent and we refer to this method as LOWRANK. The approach of Wang et al. [START_REF] Wang | Online matrix completion for signed link prediction[END_REF] is similar, but they consider directed graphs, use the logistic loss and compute at each iteration a threshold optimizing the F 1score on the observed signs. Furthermore, they argue that to better handles class imbalance, it is preferable to minimize the maxnorm of the recovered matrix, which is a tighter approximation to the rank function than the nuclear norm. We thus refer to this method as MAXNORM.

Finally, the last set of methods are based on the computation of local features of the graph. These features are evaluated on the subgraph induced by the training edges, and the resulting values are used to train a supervised classification algorithm. The most basic set of local features used to classify a given edge u → v are defined by d + in (v), d - in (v), d + out (u), d - out (u) computed over the training set E 0 , and by the embeddedness coefficient E out (u) ∩ E in (v) , which is the number of common neighbors of u and v. In turn, these degree features can be used to define more complicated features, such as a notion of similarity between two nodes based on how they rate and are rated by their neighbors [START_REF] Yuan | Negative sign prediction for signed social networks[END_REF]. Another way of looking at neighborhoods and degrees is to mine ego networks12 with a SVM [START_REF] Papaoikonomou | Predicting edge signs in social networks using frequent subgraph discovery[END_REF]. Bachi et al. [Bac+12b] also use an approach based on ego networks, in a data mining fashion. Namely, they extract frequent small subgraphs from the collection of all ego networks of G. Then, they construct rules, which are made of two frequent subgraphs differing by a single edge.

A sophisticated take on degree features is presented by Song et al. [START_REF] Song | Link sign prediction and ranking in signed directed social networks[END_REF], who note that a node can belong to one of the 16 node-types based of whether the number of its positive (respectively negative) outgoing (respectively incoming) edges is zero or not. The number of unobserved incoming and outgoing edges of each node u let us define a 16-dimensional vector V u containing the probability of transitioning to any other type once the unobserved signs are revealed. Then each edge u → v is associated with a feature vector consisting of the outer product of V u with V v and also including additional features such as triads count and degree information before training a Logistic Regression model. We refer to this method as BAYESIAN.

Other types of features are derived from social status theory, which posits that a positive link from u to v denotes that user u considers user v as having a higher status or skill than herself [START_REF] Leskovec | Predicting positive and negative links in online social networks[END_REF]. This has implications on the distribution of the so called triads in the network. A triad is a triangle formed by u → v together with u → w (or w → u) and v → w (or w → v) for any w ∈ N out (u) ∩ N in (v). Taking signs and directions into account, there are 16 possible triads but according to the status theory some must be more represented than others. The 16 TRIADS method [START_REF] Leskovec | Predicting positive and negative links in online social networks[END_REF] exploits this fact by counting for each edge in the training set how frequently it is involved in each of the 16 triad types. It also adds 7 degree features before training a Logistic Regression model.

A third group of features is based on node ranking scores. These scores (usually one or two per node) are computed using a variety of methods, including

• PageTrust [START_REF] De Kerchove | The pagetrust algorithm: how to rank web pages when negative links are allowed?[END_REF], which adapts the random walk of PageRank [START_REF] Page | The pagerank citation ranking: bringing order to the web[END_REF],

by making walkers keep in memory a list of nodes they do not like. If they ever reach such a node, their walk stop. The final PageTrust is computed iteratively until convergence.

• Prestige [START_REF] Zolfaghar | Mining trust and distrust relationships in social web applications[END_REF], which can be seen as a compounded degree feature, as it is defined by

P (u) = d + in (u)-d - in (u) d in (u)
.

• exponential ranking [START_REF] Traag | Exponential ranking: taking into account negative links[END_REF], which is the fixed point of the equation π =

A T exp 1 µ π || exp 1 µ π|| 1
. This follows from the discrete choice theory and by assuming that we observe the reputation with some noise that is double exponentially distributed with parameter µ.

• Bias and Deserve [START_REF] Mishra | Finding the bias and prestige of nodes in networks based on trust scores[END_REF], which are defined in terms of each other in weighted graphs. The bias is the tendency of a node u to trust/mistrust others, that is the difference between its opinion of a neighbor v and what v truly deserves according to the network:

bias(u) = 1 d out (u) v∈Nout(u) (w u,v -deserve(v)).
The deserve of a node v is the aggregated opinion of its in-neighbors, discounted by their bias:

deserve(v) = 1 d in (v) u∈N in (v) (w u,v (1 -max{0, sign(w u,v ) × bias(u)})) .
• Reputation and Optimism [START_REF] Shahriari | Ranking nodes in signed social networks[END_REF], defined for a node u by v∈N in (u) yv,uσ (v) v∈N in (u) σ(v) and v∈N out (u) yu,vσ(v) v) , where σ(v) is the ranking score assigned to node v. The former can be seen as a weighted version of Prestige, while the latter is its outgoing counterpart.

v∈N out (u) σ(
• TrollTrust [START_REF] Wu | The troll-trust model for ranking in signed networks[END_REF], which builds upon [START_REF] Shahriari | Ranking nodes in signed social networks[END_REF] but defines the ranking σ(u)

as the trustworthiness of u. It follows from a recursive definition of trollness based on the opinion of one node's neighbors weighted by their own trollness, which allow to assign a σ(u) to each node u through a set of non linear equations solved by an iterative method. These σ values are used to compute Reputation and Optimism scores, thus providing four features for each edge, which are in turn used to train a Logistic Regression model for the classification task. We refer to this method as RANKNODES.

Other works have also considered versions of the problem where side information related to the network is available to the learning system. For instance, [START_REF] Papaoikonomou | Trust inference in online social networks[END_REF] uses the product purchased on EPINION in conjunction with a neural network, [START_REF] Cheng | Antisocial behavior in online discussion communities[END_REF] identifies trolls by analysing the textual content of their post, [START_REF] Wang | Attributed signed network embedding[END_REF] improves the embedding approach of Wang et al. [START_REF] Wang | Signed network embedding in social media[END_REF] by considering the words written in reviews as attributes of the users, and [START_REF] Ye | Predicting positive and negative links in signed social networks by transfer learning[END_REF] uses SVM to perform transfer learning from one network to another. [START_REF] Tang | Exploiting homophily effect for trust prediction[END_REF] uses a matrix completion approach, approximating A by P CP T , where the row P u of the matrix P ∈ R n×d is the low rank representation of the node u and C ∈ R d×d is the correlation matrix between these representations. They also assume they are given a symmetric matrix Z of the homophily coefficients ζ u,v between u and v and add the following regularization term to be minimized:

u<v ζ u,v ||P u -P v || 2 2 = T r(U T L Z U )
, where L Z is the Laplacian of Z. While many of these approaches have interesting performances, they often require extra information which is not always available (or reliable) and, in addition, may face severe scaling issues.

Whereas our focus is on binary prediction, researchers have also considered a weighted version of the problem, where edges measure the amount of trust or distrust between two users. Note that typically, the embedding methods we discussed at the beginning of this section are able to handle weighted networks. One of the early and influential work on modeling how distrust propagate among users is [START_REF] Guha | Propagation of trust and distrust[END_REF]. They propose to represent atomic conclusions (such as if u trusts v and v trusts w then u is likely to trusts w) as matrix operators and define four of them that are assembled by a weighted linear combination. Starting from the observed adjacency matrix A, they repeatedly apply this operator (with potentially a discount factor) to obtain a final weigh matrix F (that can also be rounded to provide sign prediction). Qian et al. [START_REF] Qian | Foundations of trust and distrust in networks: extended structural balance theory[END_REF] extend the binary case with categorical relationships (such as strongly positive or weakly negative) and describe how every unbalanced triads experience some stress depending on the strength of its contradictory relationships. Arguing that the network should converge to a balanced state with as little change as possible, they express finding this minimal transformation as a Multidimensional Scaling problem, and the resulting graph to characterize unlabeled edges. Finally, Kumar et al. [START_REF] Kumar | Edge weight prediction in weighted signed networks[END_REF] explicitly consider the weight prediction problem in signed graphs. They use a procedure similar to the ranking methods discussed above. Namely, they defined two scores for each node that are computed iteratively from a uniform initialization. Assuming that weights lie in [-1, 1], the fairness f (u) of u is a measure of how fair or reliable u is when rating others nodes, while the goodness g(v) of v measure how trustworthy is v when evaluated with complete fairness. Formally, f (u) = 1 -

1 2Nout(u) v∈Nout(u) |wu,v-g(v)| 2 and g(v) = 1 N in (v) u∈N in (v) f (u)w u,v .
Note that these definitions are very close to bias and deserve from Mishra et al. [START_REF] Mishra | Finding the bias and prestige of nodes in networks based on trust scores[END_REF], but the absolute value in the factor 2 in the expression of fairness allows for better convergence property. Finally, the weight of a test edge u → v is predicted as f (u) × g(v).

While we presented methods that operate in the batch setting, and will present an online algorithm as well in Section 2.6, other works have addressed the EDGE SIGN PREDICTION problem from an active learning point of view [START_REF] Cesa-Bianchi | A linear time active learning algorithm for link classification[END_REF][START_REF]A correlation clustering approach to link classification in signed networks[END_REF]. Recall that in the active setting, we are given a budget of edge labels to observe, and are free to select them the way we want within E(Y ). Again, the goal is to make as few mistakes as possible when predicting the sign of the remaining edges. These two methods build a spanning subgraph of G and query all its edge, but they differ in its the construction. Cesa-Bianchi et al. [START_REF] Cesa-Bianchi | A linear time active learning algorithm for link classification[END_REF] partition G into stars and connect them in a tree, whereas Cesa-Bianchi et al. [START_REF]A correlation clustering approach to link classification in signed networks[END_REF] cover the graph with cycles, each containing one test edge and being queried for the other, while respecting the user specified query budget. This relies on a different bias than our generative model, and this will be the subject of our next chapter. One might also consider an online version of the problem where the topology is not known in advance but discovered as prediction are made [START_REF] Gentile | Online similarity prediction of networked data from known and unknown graphs[END_REF]. This naturally increases and the difficulty of the problem, as reflected by the computational cost of the solution proposed, which is quadratic in |V | at each prediction.

The survey [START_REF] Tang | A survey of signed network mining in social media[END_REF] contains pointers to many papers on signed networks, in particular for the EDGE SIGN PREDICTION problem.

Experimental Analysis

We now evaluate our EDGE SIGN PREDICTION methods on representative realworld datasets of varying density and label regularity. After presenting the data and our evaluation criterion, we proceed in two steps. First, we simulate our generative model on real networks to give them signs, then we study to which extent we can recover the parameters p and q of each node, and how predictions based on these estimation compare with the Bayes optimal. Second, we select random training sets from the actual signs. This shows that our methods compete well against existing approaches in terms of both predictive and computational performance. We are especially interested in small training set regimes, and have restricted our comparison to the batch learning scenario since all competing methods we are aware of have been developed in that setting only.

Datasets

We consider six real-world classification datasets. The first four are Directed Signed Social Networks widely used as benchmarks for this task [LHK10; SJ14; WAS16; Wan+17a]. In ADVOGATO, a trust-based social network for open source developers, a user u can certify another user v with different degrees of trust: "Observer", "Apprentice" (both of which we consider negative), "Journeyer" and "Master" (both of which we consider positive). 13 A full description of this trust metric, and its resistance to attacks, is available in the PhD thesis of the website's creator [Lev02, Section 4]). In WIKIPEDIA, there is an edge from user u to user v if v applies for an admin position and u votes for or against that promotion. In SLASHDOT, a news sharing and commenting website, member u can tag other members v as friends or foes. Finally, in EPINION, an online shopping website, user v reviews products and, based on these reviews, another user u can display whether he considers v to be reliable or not. In addition to these four datasets, we considered two other signed social networks where the signs are inferred automatically, rather than given explicitly by the users. In WIK. EDITS [START_REF] Maniu | Casting a web of trust over wikipedia: an interaction-based approach[END_REF], an edge from Wikipedia user u to user v indicates whether they edited the same article in a constructive manner or not. 14 Finally, in the CITATIONS [START_REF] Kumar | Structure and dynamics of signed citation networks[END_REF] network, an author u cites another author v by either endorsing or criticizing v's work. The edge sign is derived by classifying the citation sentiment with a technique using a list of positive and negative words; see [START_REF] Kumar | Structure and dynamics of signed citation networks[END_REF] for more details. 15Table 2.2 summarizes statistics for these datasets. We note that most edge labels are positive. Hence, test set accuracy is not an appropriate measure of prediction performance. We instead evaluated our performance using the so-called Matthews Correlation Coefficient (MCC) [START_REF] Baldi | Assessing the accuracy of prediction algorithms for classification: an overview[END_REF], defined as

MCC = tp × tn -f p × f n (tp + f p)(tp + f n)(tn + f p)(tn + f n) (2.9)
MCC combines all the four quantities found in a binary confusion matrix (true positive, true negative, f alse positive and f alse negative) into a single metric which ranges from -1 (when all predictions are incorrect) to +1 (when all predictions are correct) through 0 (when predictions are made uniformly at random).

Although the semantics of the edge signs is not the same across these networks, we can see from Table 2.2 that our generative model essentially fits all of them. Specifically, two columns of the table report the rate of label (ir)regularity, as measured by Ψ 2 G (Y )/|E| (6 th column) and Ψ G (Y )/|E| (7 th column), where

Ψ 2 G (Y ) = min (p,q) (u,v)∈E 1 + y u,v 2 - p u + q v 2 2
as first described in Section 2.3.2 and Ψ G (Y ) is the label regularity measure adopted in the online setting, as defined in Section 2.1. It is reasonable to expect that higher label irregularity corresponds to lower prediction performance. This trend is in fact confirmed by our experimental findings: whereas EPINION tends to be easy, CITATIONS tends to be hard, and this holds for all algorithms we tested, even if they do not explicitly comply with our inductive bias principles. Moreover, Ψ 2 G (Y )/|E| tends to be proportional to Ψ G (Y )/|E| across datasets, hence confirming the anticipated connection between the two regularity measures.

Finally, there is a low fraction of reciprocal edges (i.e. both u → v ∈ E and v → u ∈ E), which is a common mechanism of link formation in directed networks [GL04; Squ+13]). Moreover, in most cases, such reciprocal edges do not disagree, i.e. they have the same sign. In practice, we can use this fact to improve our prediction at no additional computational cost: when predicting u → v ∈ E test , if the reciprocal edge v → u is part of the training set, we set y u,v = y v,u . For clarity, when comparing our methods with existing approaches, we dot not use that heuristic. But afterwards, we show in Table 2.6 when it can be beneficial.

Synthetic signs

Recall that according to our generative model of Section 2.2, each node u has a parameter p u governing its sending behavior and each node v has a parameter q v governing its receiving behavior, such that the sign of the edge u → v is positive with probability pu+qv 2 = η(u, v). Given a topology G = (V, E), we start by assigning a p and q values to each node. If we want η(u, v) to be uniform over E, we have to take into account the out-degree of nodes with at least one outgoing edge, and likewise the in-degree of nodes with at least one incoming edge. For that, in the case of p, we partition the interval [0, 1] into |V | segments of size proportional to d out (u 1 ), d out (u 2 ), . . . , d out (u |V | ). We shuffle these segments and draw a number uniformly at random from each of them, which we set as the initial p value of the corresponding node. Then, because we want to model real sign distribution and have more positive than negative edges, we apply an exponential transformation

p = 1 1 -e -λ 1 -e -λp ,
where we choose λ = 3. We do the same for q, then for every edge u → v, we set it positive with probability η(u, v). An example of the distributions we obtain for the WIKIPEDIA graph is showed on Figure 2.516 , giving 71.9% of positive edges. Once the signs are generated, we select a training set E 0 uniformly at random, and predict the signs of the remaining edges in the testing set E \ E 0 . The Bayes optimal predictor classifies an edge u → v as positive if η(u, v) ≥ 1 /2 and negative otherwise. We compare it with the BLC(tr, un) algorithm analyzed in Section 2.3.1. Recall that BLC(tr, un) proceeds as follow: after computing tr(u) and un(u) on training set E 0 for all u ∈ V (or setting those values to 1 /2 in case there is no outgoing or incoming edges for some node), we use the equation (2.3) 17 , having estimated τ on E 0 .

Chapter 2. On the Troll-Trust Model for edge sign prediction in Social Networks The result of this comparison on the six datasets is showed in Table 2.3. For each network, we generated the signs once. Then, for different training size (20%, 40% or 80% of E), we sampled a training set and predict using the knowledge of the true η (Bayes predictor) or its estimated value (BLC(tr, un)). The MCC of the Bayes predictor is the same on all datasets (around 3018 ), and so is its accuracy (around 75%). The corresponding values for BLC(tr, un) are close on all datasets (although some are easier) and the gap naturally decreases as the training size increases. Another interesting quantity is how close can BLC(tr, un) estimates η using η = 1tr(u) + (1un(v))τ as defined in Section 2.3.1. We compute the mean absolute error (MAE) between η and η for all testing edges, and then specifically for the testing edges whose both endpoints have been sampled above a certain threshold in the training set. As expected, the estimation is more accurate with increasing training set size, and with increasing number of samples for a given edge.

Finally, we can also look in more details at the role of p and q by building a 2D histogram of the testing edges u → v based on their coordinate (p u , q v ). For instance on Figure 2.6a, we see that the MAE between η and η is not uniform over the p, q-space. It seems to decrease along the diagonal from (0, 0) to (1, 1). Note though that Figure 2.6b suggests this is likely due to the higher number of edges in the top right corner (reflecting the imbalance toward positive edges), which allows better estimation. In this case, we cannot use the MCC because outside the diagonal, the Bayes predictor classifies edges either all positive or all negative, resulting in a division by zero in the definition of MCC. We thus fall back on accuracy and display in Figure 2.6c the difference between the accuracy of of BLC(tr, un) and the accuracy of the Bayes predictor. This time, the anti diagonal where p+q 2 = 1 2 seems to play a special role. Indeed, as showed more clearly in Figure 2.6d, the gap in accuracy between BLC(tr, un) and the Bayes predictor increases symmetrically as p+q 2 moves away from 1 /2. The gap also tends to disappear when

Real signs

We compared the following algorithms:

1. The label propagation algorithm of Section 2.3.2 (referred to as L. PROP.). The actual binarizing threshold was set by cross-validation on the training set.

2.

The BLC(tr, un) algorithm described in Section 2.3.1.

3.

A logistic regression model where each edge (u, v) is associated with the features [1tr(u), 1un(v)] computed again on E 0 (we call this method LOGREG). The best binary thresholding is again computed on E 0 . Experimenting with this logistic model serves to support the claim we made in the introduction that our generative model in Section 2.2 is a good fit for the data.

4.

The solution obtained by directly solving the unregularized problem (2.8) through a constrained minimization algorithm (referred to as UNREG.). Again, the actual binarizing threshold was set by cross-validation on the training set.19

5.

The matrix completion method from [START_REF] Chiang | Prediction and clustering in signed networks: a local to global perspective[END_REF] based on LOWRANK matrix factorization. Since the authors showed their method to be robust to the choice of the rank parameter k, we picked k = 7 in our experiments.

6.

The other MAXNORM matrix completion method from [START_REF] Wang | Online matrix completion for signed link prediction[END_REF], setting the parameter λ to 1.2 as advised in their paper.

7.

A logistic regression model built on 16 TRIADS features derived from status theory [START_REF] Leskovec | Predicting positive and negative links in online social networks[END_REF].

8. The TrollTrust algorithm from [START_REF] Wu | The troll-trust model for ranking in signed networks[END_REF], naming it RANKNODES. As for hyperparameter tuning (β and λ 1 in [WAS16]), we closely followed the authors' suggestion of doing cross validation.

9.

The last competitor is the logistic regression model whose features have been build according to [START_REF] Song | Link sign prediction and ranking in signed directed social networks[END_REF]. We call this method BAYESIAN.

The above methods can be roughly divided into local and global methods. A local method hinges on building local predictive features, based on neighborhoods: BLC(tr, un), LOGREG, 16 TRIADS, and BAYESIAN essentially fall into this category. The remaining methods (L. PROP., LOWRANK, MAXNORM and RANKNODES) are global in that their features are designed to depend on global properties of the graph topology.

Our main results are summarized in Table 2.4, reporting MCC test set performance after training on sets of varying size (from 5% to 25%, plus 50% and 90%). Results have been averaged over 12 repetitions. Because scalability is a major concern when training on sizeable datasets, we also give an idea of relative training times by reporting separately in Table 2.5 the time (in milliseconds) it took to train a single run of each algorithm on a training set of size20 15% of |E|, and then predict on the test set. Some conclusions emerge from those experiments:

1. Global methods tend to outperform local methods in terms of prediction performance, but are also significantly (or even much) slower (running times can differ by as much as three orders of magnitude). This is not surprising, and is in line with previous experimental findings (e.g., [SJ14; WAS16]). BAYESIAN looks like an exception to this rule, but its running time is indeed in the same ballpark as global methods.

L. PROP. almost always ranks first or at least second in this comparison when

MCC is considered, at least in the small training set regime (that is when having access to at most 25% of the labels, which might be more realistic in many real world situations). On top of it, L. PROP. is fastest among the global methods (one or even two orders of magnitude faster), thereby showing the benefit of our approach to EDGE SIGN PREDICTION.

3.

Two methods are competitive with us, BAYESIAN and RANKNODES. BAYESIAN achieves its best results on the larger dataset (SLASHDOT, EPINION and WIK. EDITS) when the training set size is large. Indeed, it learns a much more complex model than our methods, with 256 parameters21 , which also requires a lot of time to be trained. RANKNODES also shines in those large datasets, even at small training size. Note however that the difference with L. PROP. is usually less than one point, at the expense of a five times longer learning time.

4.

The regularized solution computed by L. PROP. is always better than the unregularized one computed by UNREG. in terms of both MCC and running time.

5.

As claimed in the introduction, our Bayes approximator BLC(tr, un) closely mirrors in performance the more involved LOGREG model. In fact, supporting our generative model of Section 2.2, the logistic regression weights for features 1tr(i) and 1un(j) are almost equal (see Table 2.11 in the supplementary material), thereby suggesting that predictor (2.3), derived from the theoretical results in Section 2.3.1, is also the best logistic model based on trollness and untrustworthiness.

CITATIONS

5% 24.2 ± 0.9 19 .8 ± 0.5 19.8 ± 0.5 15.9 ± 0.5 12.4 ± 0.7 1.2 ± 1.4 11.4 ± 1.1 17.5 ± 1.0 15.2 ± 1.4 10% 31.7 ± 0.8 28 .0 ± 0.6 27.9 ± 0.7 26.0 ± 0.7 17.9 ± 0.7 12.6 ± 0.8 17.2 ± 1.0 25.1 ± 0.9 25.5 ± 0.9 15% 36.1 ± 0.7 33.1 ± 0.8 33 .2 ± 0.7 31.6 ± 0.7 22.0 ± 0.6 22.2 ± 0.9 21.0 ± 1.0 31.2 ± 1.0 32.0 ± 0.8 20% 38.9 ± 0.8 37 .1 ± 0.6 36.9 ± 0.6 35.6 ± 0.5 25.7 ± 0.9 30.3 ± 0.8 24.3 ± 0.7 35.2 ± 0.7 36.7 ± 0.5 25% 41.1 ± 0.7 39.7 ± 0.8 39.7 ± 0.7 38.4 ± 0.9 29.0 ± 0.6 36.5 ± 0.9 27.0 ± 0.6 37.8 ± 0.9 39 .8 ± 1.0 50% 47.2 ± 0.6 48.1 ± 0.6 47.7 ± 0.5 46.7 ± 0.7 44.9 ± 1.4 55.9 ± 0.9 35.4 ± 1.0 46.0 ± 0.6 50 .9 ± 0.8 90% 51.6 ± 2.1 52.6 ± 2.2 52.5 ± 1.7 52.3 ± 1.9 60.4 ± 1.6 66.7 ± 1.0 47.2 ± 1.9 51.5 ± 2.1 61 .7 ± 1.3 ADVOGATO 5% 40.9 ± 0.7 36.5 ± 0.7 36 .8 ± 0.8 30.2 ± 0.8 25.1 ± 1.0 4.7 ± 3.3 29.5 ± 3.7 32.2 ± 1.0 19.8 ± 0.8 10% 46.8 ± 0.6 44.7 ± 0.6 45 .3 ± 0.7 42.8 ± 0.9 28.3 ± 0.7 24.2 ± 1.6 37.6 ± 1.4

38.8 ± 1.0 30.7 ± 0.9 15% 50 .4 ± 0.5 49.5 ± 0.6 50.4 ± 0.6 47.9 ± 0.6 30.3 ± 0.8 30.1 ± 1.1 42.0 ± 0.8 45.1 ± 1.6 38.5 ± 0.8 20% 52 .2 ± 0.6 51.9 ± 0.5 53.0 ± 0.5 51.1 ± 0.7 31.5 ± 0.7 33.5 ± 1.3 44.4 ± 0.6 48.9 ± 0.9 43.4 ± 0.6 25% 54 .2 ± 0.7 54.1 ± 0.5 55.3 ± 0.5 53.7 ± 0.6 32.7 ± 0.7 35.6 ± 1.1 46.7 ± 0.9 51.7 ± 1.1 47.6 ± 0.7 50% 59.0 ± 0.5 59.8 ± 0.5 60.7 ± 0.4 60 .3 ± 0.4 37.1 ± 0.8 42.8 ± 1.2 52.2 ± 0.9 58.2 ± 0.6 56.5 ± 0.5 90% 63.3 ± 1.3 64.0 ± 1.2 64 .5 ± 1.2 65.0 ± 1.1 44.0 ± 2.0 49.5 ± 1.9 57.9 ± 1.3 63.3 ± 1.1 63.0 ± 1.2 WIKIPEDIA 5% 39.8 ± 0.7 38.2 ± 1.0 39 .0 ± 0.7 36.0 ± 0.7 24.7 ± 0.8 24.6 ± 0.9 9.7 ± 1.0 33.4 ± 0.6 26.2 ± 1.4 10% 46 .7 ± 0.6 45.9 ± 0.5 46.8 ± 0.5 44.4 ± 0.6 31.5 ± 0.7 34.9 ± 0.4 26.8 ± 1.5 43.2 ± 0.6 40.3 ± 0.6 15% 50 .3 ± 0.5 49.6 ± 0.4 50.5 ± 0.4 48.8 ± 0.4 35.4 ± 0.7 39.4 ± 0.6 34.2 ± 0.6 48.4 ± 0.5 46.7 ± 0.4 20% 52 .5 ± 0.4 51.9 ± 0.5 52.8 ± 0.5 51.7 ± 0.5 38.0 ± 0.6 42.2 ± 0.5 38.2 ± 0.7 51.2 ± 0.5 50.2 ± 0.4 25% 54 .2 ± 0.6 53.6 ± 0.6 54.6 ± 0.5 53.6 ± 0.4 40.2 ± 0.6 44.2 ± 0.6 41.2 ± 0.6 53.5 ± 0.5 53.0 ± 0.6 50% 57 .8 ± 0.3 56.9 ± 0.5 57.9 ± 0.4 57.6 ± 0.4 47.3 ± 0.7 48.8 ± 0.7 48.9 ± 0.5 57.8 ± 0.4 57.4 ± 0.5 90% 60.5 ± 0.8 59.5 ± 0.9 60.4 ± 0.9 59.9 ± 1.1 53.5 ± 0.9 51.3 ± 0.8 55.6 ± 0.8 60 .4 ± 1.0 59.6 ± 1.2 SLASHDOT 5% 41 .0 ± 0.2 36.3 ± 0.5 37.1 ± 0.3 33.6 ± 0.2 36.9 ± 0.4 14.9 ± 0.8 27.8 ± 1.0 45.3 ± 0.4 29.4 ± 0.3 10% 46 .6 ± 0.2 42.2 ± 0.2 43.1 ± 0.2 40.9 ± 0.3 39.7 ± 0.3 25.7 ± 0.7 40.3 ± 1.1 47.8 ± 0.5 38.2 ± 0.2 15% 49.7 ± 0.2 45.4 ± 0.3 46.5 ± 0.2 45.4 ± 0.2 41.3 ± 0.3 32.5 ± 0.3 45.5 ± 1.3 48 .7 ± 0.7 43.4 ± 0.2 20% 51 .8 ± 0.2 47.6 ± 0.3 48.7 ± 0.1 48.3 ± 0.2 42.9 ± 0.2 36.6 ± 0.3 49.0 ± 0.7 52.0 ± 0.3 47.0 ± 0.3 25% 53 .3 ± 0.2 49.4 ± 0.2 50.2 ± 0.1 50.0 ± 0.1 44.3 ± 0.6 39.7 ± 0.3 50.6 ± 0.4 53.3 ± 0.3 49.4 ± 0.2 50% 57.2 ± 0.1 54.2 ± 0.2 55.0 ± 0.1 54.6 ± 0.1 52.9 ± 0.5 48.2 ± 0.1 55.2 ± 0.4 56 .8 ± 0.2 56.3 ± 0.1 90% 59 .6 ± 0.5 57.4 ± 0.5 58.0 ± 0.5 57.7 ± 0.4 59.0 ± 0.4 53.5 ± 0.4 58.0 ± 0.4 59.2 ± 0.6 61.1 ± 0.5 EPINION 5% 54 .6 ± 0.3 46.9 ± 0.6 48.9 ± 0.3 42.8 ± 0.4 39.9 ± 0.3 28.9 ± 0.3 41.4 ± 2.0 56.0 ± 0.6 37.9 ± 0.4 10% 59 .0 ± 0.4 54.1 ± 0.4 55.9 ± 0.2 53.3 ± 0.4 44.4 ± 0.3 36.7 ± 0.4 50.8 ± 1.5 60.5 ± 0.3 49.9 ± 0.4 15% 61 .5 ± 0.3 57.9 ± 0.3 59.3 ± 0.1 58.7 ± 0.5 48.6 ± 0.7 41.0 ± 0.5 54.5 ± 1.8 62.7 ± 0.2 56.5 ± 0.7 20% 63 .0 ± 0.3 60.3 ± 0.3 61.4 ± 0.1 61.8 ± 0.2 52.0 ± 0.9 43.9 ± 0.4 56. 45.8 ± 0.3 50% 39.5 ± 0.5 42.8 ± 0.2 42.6 ± 0.2 38.9 ± 0.2 41.0 ± 0.7 41.0 ± 0.3 27.4 ± 0.5 42 .9 ± 0.3 53.2 ± 0.2 90% 41.7 ± 0.5 44.5 ± 0.6 43.9 ± 0.4 41.0 ± 0.5 45.3 ± 0.4 45.8 ± 0.3 31.6 ± 0.5 46 .0 ± 0.5 58.5 ± 0.4

Table 2.5 -The time taken (in milliseconds) to train, for each dataset, on a 15% training set and predict the remaining 85%. The experimental setting is the same as in Table 2

Additional experiments

We perform two other sets of experiments. The first one evaluates the effect of predicting reciprocal edges by their value if available. As expected, the results in Table 2.6 demonstrate that it improves MCC when the network has enough reciprocal edges with low disagreements, like in SLASHDOT and EPINION. It has no effect when there are few reciprocal edges, like in CITATIONS, WIKIPEDIA and WIK. EDITS. And it is detrimental when there are many reciprocal edges that do not agree enough, like in ADVOGATO, in which case it is better to rely solely on the learned model.

Table 2.6 -100 time the difference of MCC between using the twin heuristic and not using it for our two main methods. Those values are computed on the same training/testing split as of Table 2.4.

5% 10% 15% 20% 25% 50% 90% CITATIONS L. PROP. 0.0 0.0 0.0 0.0 0.0 -0.1 -0.7 BLC (tr, un) 0.0 0.0 0.0 0.0 0.1 -0.2 -0.7 ADVOGATO L. PROP. -0.2 -0.6 -1.0 -1.6 -2.2 -5.4 -10.8 BLC (tr, un) -0.1 -0.5 -1.0 -1.6 -2.2 -5.7 -11.1 WIKIPEDIA L. PROP. 0.0 0.0 0.0 0.0 0.0 -0.1 -0.2 BLC (tr, un) 0.0 0.0 0.0 -0.1 -0.1 -0.2 -0.2 SLASHDOT L. PROP. 0.3 0.5 0.8 0.9 1.1 2.0 3.3 BLC (tr, un) 0.3 0.6 0.9 1.1 1.3 2.2 3.5 EPINION L. PROP. 0.3 0.5 0.7 0.8 1.0 1.9 3.0 BLC (tr, un) 0.4 0.6 0.8 1.0 1.2 1.8 2.8 WIK. EDITS L. PROP. 0.0 0.0 0.0 0.0 -0.1 -0.1 -0.3 BLC (tr, un) 0.0 0.0 0.0 0.0 0.0 -0.2 -0.5
In the second set of experiments, we study the effect of our hypothesis that the labeled edges of the training set are chosen uniformly at random. In two of our datasets (WIKIPEDIA and EPINION), edges come with the timestamp at which they were created. A more realistic way of choosing the training set is 2.6. Algorithms in the Online Setting therefore, for a given training size m, to let E 0 be the m oldest edges, and try to predict the remaining, newest ones. This is a common experimental setting in link prediction, where the goal is to infer which pair of nodes will be connected in the future [START_REF] Martínez | A survey of link prediction in complex networks[END_REF]. However, as showed in Table 2.7, this makes the problem much more challenging, both for our methods and our competitors. It is quite surprising that even when the training set is as large as 90% of all edges, the MCC are so low compared with those reported in Table 2.4. Table 2.7 -Same as Table 2.4, but when selecting the training set by the timestamp of the edges.

|E 0 | |E| L. PROP. BLC(tr, un) LOGREG UNREG. LOWRANK MAXNORM 16 TRIADS RANKNODES BAYESIAN WIKIPEDIA 5% 2.2 ± 0.0 4 .0 ± 0.0 4.0 ± 0.0 -1.0 ± 1.0 3.0 ± 0.4 0.5 ± 0.4 1.5 ± 0.1 2.8 ± 0.2 2.0 ± 0.2 10% 3.1 ± 0.0
5.6 ± 0.0 5 .9 ± 0.0 0.4 ± 0.9 5.8 ± 0.5 0.8 ± 0.3 2.2 ± 0.3 6.3 ± 0.5 4.2 ± 0.2 15% 5.2 ± 0.0 5.7 ± 0.0 6 .5 ± 0.1 0.7 ± 0.7 6.2 ± 0.5 1.4 ± 0.4 3.4 ± 0.2 7.4 ± 0.5 3.4 ± 0.1 20% 5.5 ± 0.0 6.9 ± 0.0 7 .4 ± 0.1 0.7 ± 0.7 5.3 ± 0.6 1.6 ± 0.2 3.9 ± 0.2 10.1 ± 0.2 2.7 ± 0.1 25% 5.8 ± 0.0 6.5 ± 0.0 7.2 ± 0.0 2.6 ± 0.9 7 .2 ± 0.7 1.9 ± 0.3 6.6 ± 0.3 9.4 ± 0.1 3.9 ± 0.3 50% 7.0 ± 0.0 7.3 ± 0.0 8.0 ± 0.0 4.7 ± 0.6 8.1 ± 1.1 3.3 ± 0.5 17.1 ± 1.7 12 .4 ± 0.0 5.2 ± 0.4 90% 20.9 ± 0.0 14.7 ± 0.0 13.3 ± 0.0 13.5 ± 0.9 10.5 ± 0.7 7.4 ± 1.7 20 .2 ± 0.5 18.7 ± 0.1 13.2 ± 0.3 EPINION 5% 26.8 ± 0.0 23.2 ± 0.0 23.9 ± 0.0 20.9 ± 0.5 23.0 ± 0.3 13.9 ± 0.2 28.9 ± 0.2 31.9 ± 0.2 29 .8 ± 0.8 10% 26.3 ± 0.0 25.0 ± 0.0 26.0 ± 0.1 22.4 ± 0.5 28.4 ± 0.4 16.7 ± 0.3 29.3 ± 0.3 34.3 ± 0.0 33 .7 ± 0.5 15% 23.2 ± 0.0 23.0 ± 0.0 25.2 ± 0.0 21.0 ± 0.4 28.7 ± 0.4 17.1 ± 0.5 28.1 ± 0.3 32 .6 ± 0.2 33.1 ± 0.3 20% 18.4 ± 0.0 21.3 ± 0.0 24.2 ± 0.0 19.5 ± 0.3 26.2 ± 0.4 16.6 ± 0.3 26.8 ± 0.3 29 .3 ± 0.0 32.7 ± 0.3 25% 30.4 ± 0.0 25.1 ± 0.0 27.9 ± 0.0 20.5 ± 0.4 25.7 ± 0.3 17.0 ± 0.4 26.4 ± 0.3 35.6 ± 0.1 30 .4 ± 0.3 50% 21.2 ± 0.0 25.5 ± 0.0 29.1 ± 0.0 19.2 ± 0.5 19.9 ± 1.0 12.2 ± 0.8 31 .3 ± 0.4 34.7 ± 0.0 22.3 ± 0.1 90% 38.6 ± 0.0 32.1 ± 0.0 33.9 ± 0.0 25.2 ± 0.8 24.9 ± 0.8 8.2 ± 1.2 40 .8 ± 0.5 42.5 ± 0.0 36.8 ± 0.3

The explanation of why these two sampling strategies produce so different results can be inferred from Table 2.8. Letting V out be the subset of node of V with at least one outgoing edge, and defining similarly V in , we see that the node coverage is naturally larger when sampling at random. Yet, when the training set reaches 90% size, both sampling strategies cover roughly 90% of the nodes in V out and in V in . The difference in MCC is thus explained by the last three columns. It shows, for testing edges u → v, the breakdown between three situations, from the least to the most informative:

(i) no sampled edge outgoing from u nor incoming to v;

(ii) some sampled edges, but only either outgoing from u or incoming to v; and (iii) some sampled edges, both outgoing from u and incoming to v; When the training set is build uniformly at random, the vast majority of testing edges falls into the case (iii). This is no longer true with temporal training set, where situations (ii) and even (i) are more common, making prediction more difficult. The difference of repartition is particularly strong in WIKIPEDIA, justifying the better performance in EPINION. This conclusion is not in contradiction with our previous experiments, since our algorithms were developed under the assumption that both the revealed labels and the ones to predict were distributed uniformly within the graph. Recall for instance that Theorem 1 requires to have Ω(log |V |) labels outgoing from u and incoming to v to guarantee an accurate prediction of the sign of u → v. Therefore in the next section, we present an online algorithm, whose goal is to guarantee good performance when the labeling is regular enough, despite the adversarial order in which the predictions are asked.

Algorithms in the Online Setting

In the online scenario, recall we do not assume anymore that the signs are generated by the model of Section 2.2. Instead, they are chosen adversarially, and presented sequentially in order to force as many mistakes as possible. This is achieved by deviating from the regular labeling where every user is either perfectly troll or Table 2.8 -The effect of not sampling edges at random on WIKIPEDIA and EPINION. In case of random sampling, all values are averaged over 20 trials. Refer to the main text for an interpretation of how this affects the predictive performance. trustworthy, as measured by the regularity measure Ψ G (Y ) described at the end of Section 2.1. We start by presenting an algorithm that combines randomized Weighted Majority instances and makes little more than Ψ G (Y ) mistakes on any edge-labeled graph

G(Y ) = (V, E(Y )).
We then show there is not much room for improvement, for as long as Ψ G (Y ) is within a budget of K irregularities, any online algorithm is condemned to err at least K /2 times.

Theorem 2. There exists a randomized online prediction algorithm A whose expected number of mistakes satisfies

EM A (Y ) = Ψ G (Y ) + O |V |Ψ G (Y ) + |V | on any edge- labeled graph G(Y ) = (V, E(Y )).
Proof. Let each node u ∈ V host two instances of the randomized Weighted Majority (RWM) algorithm [START_REF] Littlestone | The weighted majority algorithm[END_REF] with an online tuning of their learning rate [Ces+97; ACG02]: one instance for predicting the sign of outgoing edges (u, v), and one instance for predicting the sign of incoming edges (v, u). Both instances simply compete against the two constant experts, predicting always +1 or always -1. Denote by M (u, v) the indicator function (zero-one loss) of a mistake on edge (u, v). Then the expected number of mistakes of each RWM instance satisfy [Ces+97; ACG02]:

v∈Nout(u) E M (u, v) = Ψ out (u, Y ) + O Ψ out (u, Y ) + 1 and u∈N in (v) E M (u, v) = Ψ in (v, Y ) + O Ψ in (v, Y ) + 1 .
We then define two meta-experts: an ingoing expert, which predicts y u,v using the prediction of the ingoing RWM instance for node v, and the outgoing expert, which predicts y u,v using the prediction of the outgoing RWM instance for node u. The 2.7. Open questions number of mistakes of these two experts satisfy

u∈V v∈Nout(u) E M (u, v) = Ψ out (Y ) + O |V |Ψ out (Y ) + |V | v∈V u∈N in (v) E M (u, v) = Ψ in (Y ) + O |V |Ψ in (Y ) + |V | ,
where we used v∈V Ψ in (v, Y ) ≤ |V |Ψ in (Y ), and similarly for Ψ out (Y ). Finally, let the overall prediction of our algorithm be a RWM instance run on top of the ingoing and the outgoing experts. Then the expected number of mistakes of this predictor satisfies

(u,v)∈E E M (u, v) = Ψ G (Y ) + O |V |Ψ G (Y ) + |V | + Ψ G (Y ) + |V | + |V |Ψ G (Y ) = Ψ G (Y ) + O |V |Ψ G (Y ) + |V | as claimed.
We complement the above result by providing a mistake lower bound. Like Theorem 2, the following result holds for all graphs, and for all label irregularity levels Ψ G (Y ).

Theorem 3. Given any edge-labeled graph

G(Y ) = (V, E(Y )) and any integer K ≤ |E| 2 , a randomized labeling Y ∈ {-1, +1} |E| exists such that Ψ G (Y ) ≤ K,
and the expected number of mistakes that any online algorithm A make can be forced to satisfies

EM A (Y ) ≥ K 2 .
Proof. Let Y K be the set of all labelings Y such that the total number of negative and positive edges are K and |E| -K, respectively (without loss of generality we will focus on negative edges). Consider the randomized strategy that draws a labeling Y ∈ {-1, +1} |E| of the edges of the input graph uniformly at random from Y K . For each node u ∈ V , we have

Ψ in (u, Y ) ≤ d - in (u), which implies Ψ in (Y ) ≤ K.
A very similar argument applies to the outgoing edges, leading to

Ψ out (Y ) ≤ K. The constraint Ψ G (Y ) ≤ K is therefore always satisfied.
The adversary will force on average 1 /2 mistakes in each one of the first K rounds of the online protocol by repeating K times the following: (i) A label value ∈ {-1, +1} is selected uniformly at random. (ii) An edge (u, v) is sampled uniformly at random from the set of all edges that were not previously revealed and whose labels are equal to .

The learner is required to predict y u,v and, in doing so, 1 /2 mistakes will be made on average because of the randomized labeling procedure. Observe that this holds even when A knows the value of K and Ψ G (Y ). Hence, we can conclude that the expected number of mistakes that A can be forced to make is always at least K/2, as claimed.

In fact, we can refine the above statement by proving that, as K |E| → 0, the lower bound gets arbitrarily close to K for any G(Y ), hence asymptotically matching the upper bound of Theorem 2. A sketch of the proof can be found on page 41.

Open questions

Given the fundamental role of our generative model, we would like to adapt it in order to handle the three problem extensions presented in the related works of Section 2.4.

1. When available, using side information about users is an alluring option to alleviate the cold start problem. Recall that in our experiments, we saw that if we learn our model on historical data, we cannot successfully predict the interactions involving new users, for their trollness and trustworthiness

are not yet known to us. However, we expect that for existing users, side information is correlated with their value of p and q. This knowledge thus provides a parameterised prior for the p and q values of news users. As we observe their behavior in the network, our estimation of p and q would rely less and less on prior information and more and more on observed data, as it is common in a Bayesian approach. Note that this introduces an additional learning stage.

2. Our generative model, and in particular the prior distribution µ(p, q), could be exploited to guide label queries in an adaptive active learning setting. Given a budget of queries B, and after an initial phase of querying edge labels uniformly at random, one might use concentration results on µ22 to devise an informativeness criterion, indicating for which node the value of p and q are the most uncertain. Combined with the graph topology, this would suggest which edge is the most important to query next, until the budget is exhausted.

3. So far we discussed binary classification of edges. In weighted graphs though, a refined problem would be to predict the weight of unlabeled edges. Let us assume for simplicity that the weights are bounded and within [-1, 1]. The most immediate solution would be, for an edge u → v, to linearly shift our estimation of pu+qv

2 = η(u, v) from [0, 1] to [-1, 1
] and use this quantity as our prediction. We could also change the semantic of η(u, v) from being the probability of u → v to be positive to the mean of a narrow Gaussian from which the weight of u → v is drawn. Finally, by letting p u range from -1 to +1, we note a similarity with fairness and goodness, suggesting p u × q v could approximate the weights.

Summary

In this chapter, we started our characterization of edges in complex networks by studying the EDGE SIGN PREDICTION problem in Directed Signed Social Networks. As mentioned in the introduction, given a network topology and a few labeled interactions, being able to infer the polarity of the remaining interactions is valuable to improve user experience. Our main insight is that two features of users behavior, trollness and (un)trustworthiness, are key to understand their interactions. This leads us to design a simple sign generative model. Such a model serves both as a theoretical justification for many successful heuristics and as the underpinning of our BLC(tr, un) algorithm. This algorithm is fast, trivially parallelizable, provably close to the Bayes optimal predictor on dense graphs and experimentally accurate even on sparse graphs. We further exploit this model in the batch setting through an approximation of the maximum likelihood estimation procedure. While making this approximation, we moreover cast the problem as a node prediction problem. This can be efficiently tackled by standard Label Propagation algorithms.

We validate our theoretical results by experimentally assessing these two methods in the small training set regime, on synthetic and real-world datasets. We draw two main conclusions from our experiments. First, our generative model is robust across domains. Indeed, it produces predictors that are empirically both close to the Bayes optimal and to general linear models trained on trollness and trustworthiness features. This is notable as this result is achieved by simply counting negative edges, without resorting to Stochastic Gradient Descent methods typically involved in training large scale linear models. Second, our methods are in practice either strictly better than their competitors in terms of prediction quality or, when they are not, they are faster.

Finally, we study the online adversarial setting, where trollness and (un)trustworthiness naturally lend themselves to define a notion of edge sign regularity. Based on this regularity, we provide an upper and an (almost matching) lower bounds on the expected number of prediction mistakes. We conclude by presenting three directions in which our generative model could be extended: namely use side information as priors, guide budgeted queries in active learning and generate weights in addition to signs.

In the next chapter, we will go even further than this generative model and explore another learning bias for EDGE SIGN PREDICTION, which is not based on trollness and (un)trustworthiness, but rather on the social balance theory.

Additional material

2.9.1 Proofs from Section 2.3

Proof of Theorem 1

The following ancillary results will be useful.

Lemma 1 (Hoeffding's inequality for sampling without replacement). Let X = {x 1 , . . . , x N } be a finite subset of [0, 1] and let

µ = 1 N N i=1 x i .
If X 1 , . . . , X n is a random sample drawn at random from X without replacement, then, for every ε > 0,

Pr 1 n n t=1 X t -µ ≥ ε ≤ 2e -2nε 2 .
Lemma 2. Let N 1 , . . . , N n be n subsets of a finite set E. Let E 0 ⊆ E be sampled uniformly at random without replacement from E, with |E 0 | = m. Then, for δ ∈ (0, 1), Q > 0, and θ ≥ 2 × max Q, 4 log n δ , we have

Pr ∃i : |N i | ≥ θ, |N i ∩ E 0 | < Q ≤ δ provided |E| ≥ m ≥ 2|E| θ × max Q, 4 log n δ . Proof of Lemma 2. Set for brevity p i = |N i | /|E|. Then, due to the sampling without replacement, each random variable |N i ∩ E 0 | is the sum of m dependent Bernoulli random variables X i,1 , . . . , X i,m such that Pr(X i,t = 1) = p i , for t = 1, . . . , m. Let i be such that |N i | ≥ θ. Then, since θ ≥ 2Q, we may assume that m 2Q θ ≤ |E|, which implies Q ≤ mθ 2|E| ≤ m p i 2 = E |N i ∩ E 0 | 2 .
Since the variables X i,j are negatively associated, we may apply a (multiplicative) Chernoff bound [DP09, Section 3.1]. This gives

Pr |N i ∩ E 0 | < Q ≤ e -m p i 8 ≤ e -m θ 8|E| so that Pr ∃i : |N i | ≥ θ, |N i ∩ E 0 | < Q ≤ n e -m θ
8|E| , which is in turn upper bounded by δ whenever m ≥ 8|E| θ log n δ (again a natural assumption since θ ≥ 8 log n δ ).

Let now

E θ = {(u, v) ∈ E : d in (v) ≥ θ, d out (u) ≥ θ} \ E 0
, where E 0 ⊆ E is the set of sampled edges provided to the learning algorithm of Section 2.3.1. Then Theorem 1 in the main paper is an immediate consequence of the following lemma.

Lemma 3. Let G(Y ) = (V, E(Y )) be a directed graph with labels on the edges generated according to the model in Section 2.2. For all 0 < α, δ < 1 and 0 < ε < 1 16 , if the learning algorithm of Section 2.3.1 is run with parameters Q = 1 2ε 2 log 4|V | δ and α, then with probability at least 1 -9δ the predictions y(u, v) satisfy y(u, v) = y * (u, v) for all

(u, v) ∈ E θ such that η(u, v) -1 2 > 8ε.
Proof of Lemma 3. We apply Lemma 2 with θ

= 2Q α ≥ 2 × max Q, 4 log 2|V |+1 δ to the 2|V | + 1 subsets of E consisting of E L and, for all u ∈ V , of E in (u) and E out (u).
We have that, with probability at least 1δ, at least Q edges of E L are sampled, at least Q edges of E in (v) are sampled for each v such that N in (v) ≥ θ, and at least Q edges of E out (u) are sampled for each u such that N out (u) ≥ θ. For all (u, v) ∈ E θ let

p v = 1 d in (v) u∈N in (v) p u and q u = 1 d out (u) v∈Nout(u) q v
and set for brevity δ in (v) = 1un(v) and δ out (u) = 1tr(u). We now prove that δ in (v) and δ out (u) are concentrated around their expectations for all (u, v) ∈ E θ . Consider δ out (u) (the same argument works for

δ in (v)). Let V 1 , . . . , V Q be the first Q draws in E 0 ∩ N out (u) and define µ p (u) = 1 Q Q t=1 p u + q Vt 2 .
Applying Lemma 1 to the set pu+qv 2 : v ∈ N out (u) , and using our choice of Q, we get that µ p (u)µ p (u) ≤ ε holds with probability at least 1δ/(2|V |), where

µ p (u) = 1 d out (u) v∈Nout(u) p u + q v 2 = p u + q u 2 .
Now consider the random variables Z t = I {y u,vt = 1}, for t = 1, . . . , Q. Conditioned on V 1 , . . . , V Q , these are independent Bernoulli random variables with

E[Z t | V t ] = pu+q V t 2
. Hence, applying a standard Hoeffding bound for independent variables and using our choice of Q, we get that

1 Q Q t=1 Z t -µ p (u) ≤ ε with probability at least 1 -δ/(2|V |) for every realization of V 1 , . . . , V Q . Since δ out (u) = (Z 1 + • • • + Z Q )/Q, we get that δ out (u) -µ p (u) ≤ 2ε with probability at least 1 -2δ/(2|V |).
Applying the same argument to δ in (v), and the union bound 23 on the set δ in (v), δ out (u) : (u, v) ∈ E θ , we get that

δ out (u) + δ in (v) - p v + q u 2 - p u + q v 2 ≤ 4ε (2.10)
simultaneously holds for all (u, v) ∈ E θ with probability at least 1 -4δ. Now notice that p v is a sample mean of Q i.i.d. [0, 1]-valued random variables drawn from the prior marginal 1 0 µ •, q dq with expectation µ p . Similarly, q u is a sample mean of Q 23 The sample spaces for the ingoing and outgoing edges of the vertices occurring as endpoints in E θ overlap. Hence, in order to prove a uniform concentration result, we need to apply the union bound over the random variables defined over these sample spaces.

i.i.d. [0, 1]-valued random variables independently drawn from the prior marginal 1 0 µ p, • dp with expectation µ q . By applying Hoeffding bound for independent variables, together with the union bound to the set of pairs of random variables whose sample means are p v and q u for each (u, v) ∈ E θ (there are at most 2|V | of them) we obtain that p vµ p ≤ ε and q uµ q ≤ ε hold simultaneously for all (u, v) ∈ E θ with probability at least 1 -2δ. Combining with (2.10) we obtain that

δ out (u) + δ in (v) - µ p + µ q 2 - p u + q v 2 ≤ 5ε (2.11) simultaneously holds for each (u, v) ∈ E θ with probability at least 1 -6δ. Next, let E L be the set of the first Q edges drawn in E L ∩ E 0 . Then E τ = 1 Q (u,v)∈E L Pr y u,v = 1 = 1 Q (u,v)∈E L p u + q v 2 ,
where the expectation is w.r.t. the independent draws of the labels y u,v for (u, v) ∈ E L . Hence, by applying again Hoeffding bound (this time without the union bound) to the

Q = 1 2ε 2 log 4|V | δ independent Bernoulli random variables I {y u,v = 1}, (u, v) ∈ E L , the event τ -E τ ≤ ε holds with probability at least 1 -δ. Now, introduce the function F (p, q) = E τ = 1 Q (u,v)∈E L p u + q v 2 .
For any realization q 0 of q, the function F 1 (p) = F (p, q 0 ) is a sample mean of

Q = 1 2ε 2 log 4|V | δ i.i.d. [0, 1]-valued random variables {p u : (u, v) ∈ E L } (recall that if u ∈ V is the origin of an edge (u, v) ∈ E L ,
then it is not the origin of any other edge (u, v ) ∈ E L ). Using again the standard Hoeffding bound, we obtain that

F (p, q) -E p F (p, q) ≤ ε
holds with probability at least 1δ for each q ∈ [0, 1] |V | . With a similar argument, we obtain that

E p F (p, q) -E p,q F (p, q) ≤ ε also holds with probability at least 1 -δ. Since E p,q F (p, q) = µ p + µ q 2
we obtain that

τ - µ p + µ q 2 ≤ 3ε
(2.12) with probability at least 1 -3δ. Combining (2.11) with (2.12) we obtain

δ out (u) + δ in (v) -τ - p u + q v 2 ≤ 8ε
simultaneously holds for each (u, v) ∈ E θ with probability at least 1 -9δ. Putting together concludes the proof.

Derivation of the maximum likelihood equations

Recall that the training set

E 0 = u k , v k ), y u k ,v k : k = 1, .
. . , m is drawn uniformly at random from E without replacement. We can write

Pr E 0 {p u , q u } |V | u=1 = 1 |E| m m! m k=1 p u k + q v k 2 I{yu k ,v k =+1} × m k=1 1 - p u k + q v k 2 I{yu k ,v k =-1} = 1 |E| m m! |V | =1 m k=1 p + q v k 2 I{u k = , y ,v k =+1} × m k=1 1 - p + q v k 2 I{u k = , y ,v k =-1} so that log Pr E 0 {p u , q u } |V | u=1 is proportional to |V | =1 m k=1 I {u k = , y ,v k = +1} log p + q v k 2 + m k=1 I {u k = , y ,v k = +1} log 1 - p + q v k 2 and ∂ log Pr E 0 {p u , q u } |V | u=1 ∂p = m k=1 I {u k = , y ,v k = +1} p + q v k - m k=1 I {u k = , y ,v k = -1} 2 -p -q v k .
By a similar argument,

Pr E 0 {p u , q u } |V | u=1 = 1 |E| m m! |V | =1 m k=1 p u k + q 2 I{v k = , y u k , =+1} × m k=1 1 - p u k + q 2 I{v k = , y u k , =-1} so that ∂ log Pr E 0 {p u , q u } |V | u=1 ∂q = m k=1 I {v k = , y u k , = +1} p u k + q - m k=1 I {v k = , y u k , = -1} 2 -p u k -q .

Label propagation on G

Here we provide more details on the choice of weight for the edges of G , as well as an explanation on why we temporarily use symmetrized variables lying in [-1, 1] (which we will denote with primes, so that for instance p u = 2p u -1). Since only the ratio between the negative and positive weights matters, we fix the negative weight of the edges in E \ E to be -1 and we denote by the weight of edges in E . With these notations, Label Propagation on G seeks the harmonic minimizer of the following expression

1 16 u,v∈E y u,v -p u 2 + y u,v -q v 2 + (p u + q v ) 2
which can be successively rewritten as

1 16 u,v∈E (y u,v + 1 -2p u ) 2 + (y u,v + 1 -2q v ) 2 + (2p u + 2q v -2) 2 = 1 8 u,v∈E 2 y u,v + 1 2 -p u 2 + 2 y u,v + 1 2 -q v 2 + 8 p u + q v -1 2 2 = 1 8 u,v∈E 2 y u,v + 1 2 2 -p u (1 + y u,v ) + p 2 u + 2 y u,v + 1 2 2 -q v (1 + y u,v ) + q 2 v + 8 p u + q v 2 2 - p u + q v 2 + 1 4 = 1 8 u,v∈E 4 y u,v + 1 2 2 -2 y u,v + 1 2 p u + q v 2 + 2 p u + q v 2 2 + u,v∈E 2 p 2 u -4p u + 1 + 2 q 2 v -4q v + 1
By setting = 2, we can factor this expression into u,v∈E

y u,v + 1 2 - p u + q v 2 2 + 1 2 u,v∈E p u - 1 2 2 + q v - 1 2 2 .

Proof from Section 2.6

Proof sketch that, in the context of Theorem 3, as K |E| → 0 then EM A (Y ) = K . Let E be the following event: There is at least one unrevealed negative label. The randomized iterative strategy used to force a number of mistakes arbitrarily close to K is identical to the one described in the proof of Theorem 3 on page 35, except for the stopping criterion. Namely, the adversary draws a labeling Y uniformly at random from Y K , the set of all labelings Y such that the total number of negative and positive edges are K and |E| -K. Then the adversary repeats the following two steps: (i) A label value ∈ {-1, +1} is selected uniformly at random. (ii) An edge (u, v) is sampled uniformly at random from the set of all edges that were not previously revealed and whose labels are equal to . This is done until E is true.

Let m r,c be defined as follows: For c = 1 it is equal to the expected number of mistakes forced in round r when K = 1. For c > 1 it is equal to the difference between the expected number of mistakes forced in round r when K = c and K = c -1. One can see that m r,c is null when r < c. When K = 1, the probability that E is true in round r is clearly equal to 1 2 r-1 . Hence, the expected number of mistakes made by A when K = 1 in any round r is equal to 1 2 1 2 r-1 = 1 2 r . We can therefore conclude that m r,1 = 1 2 r for all r. A simple calculation shows that if r = c then m r,c = 1 2 r . Furthermore, when r > 1 and c > 1, we have the following recurrence:

m r,c = m r-1,c + m r-1,c-1 2 .
In order to calculate m r,c for all r and c, we will rest on the ancillary quantity s j (i), defined recursively for any positive integer variables i and j by s 0 (i) = 1 and

s j (i) = i k=1 s j-1 (k) .
(see Table 2.9 -Values of s j (i) for i ≤ 7 and j ≤ 3.

It is not difficult to verify that m r,c = s c-1 (r -c + 1) 2 r . Since s j (i) = i j j!
, where i j is the rising factorial i(i + 1)(i + 2) . . . (i + j -1), we have When K |E| → 0, given any integer K > 1, the difference between the expected number of mistakes forced when

m r,c = r -c + 1 c-1 (c -1)!2 r . (see Table 2.10) for examples) a a c r 1 2 3 4 1 1/2 1 0 0 0 2 1/2 2 1/2 2 0 0 3 1/2 3 2/2 3 1/2 3 0 4 1/2 4 3/2 4 3/2 4 1/2 4 5 1/2 5 4
K = K and K = K -1 is equal to ∞ r=K m r,K = 1 (K -1)! ∞ r=K r -K + 1 K -1 2 r = 1 (K -1)!2 K -1 ∞ r =1 r K -1 2 r ,
where we set r = r -K + 1. Setting i = i -1 and recalling that

i j = j! i + j -1 i -1 , we have 1 j! ∞ i=1 i j 2 i = ∞ i=1 i+j-1 i-1 2 i = ∞ i =0 i +j i 2 i +1 .
Now, using the identity

i + j + 1 i = i + j i + i + j i -1 ,
we can easily prove by induction on j that

∞ i =0 i +j i 2 i +1 = 2 j .
Hence, we have

∞ r=K m r,K = 1.
Moreover, as shown earlier, m r,1 = 1 2 r for all r. Hence we can conclude that when

K |E| → 0 EM A (Y ) ≥ ∞ r=1 1 2 r + K K =2 ∞ r=K m r,K = K
for any edge-labeled graph G(Y ) and any constant K, as claimed.

Further Experimental Results

This section contains more evidence related to the experiments in Section 2.5. In particular, we experimentally demonstrate the alignment between BLC(tr, un) and LOGREG.

After training on the two features 1tr(u) and 1un(v), LOGREG has learned three weights w 0 , w 1 and w 2 , which allow to predict y u,v according to

sign w 1 (1 -tr(u) + w 2 1 -un(v) + w 0 .
This can be rewritten as

sign 1 -tr(u) + w 2 1 -un(v) -1 2 -τ ,
with w 2 = w 2 w 1 and τ = -1 2 + w 0 w 1 . As shown in Table 2.11, and in accordance with the predictor built out of Equation (2.3), w 2 is almost 1 on all datasets, while τ tends to be always close the fraction of positive edges in the dataset. Recovering such clusters from a given signed graph has been studied extensively in the last decades under the CORRELATION CLUSTERING name. In Section 3.2, we show how this difficult combinatorial clustering problem is connected to our learning bias. We survey a wide range of methods to solve it exactly, approximately or heuristically, paying special attention to settings where such recovery is easier, such as noisy or stable instances. Finally, focusing on the important and convenient special case K = 2, we develop in Section 3.3 an algorithm that, given an undirected general graph, produce a spanning tree designed to support fast and accurate edge sign prediction.

A bias for general graphs

Let us recall first the sign model we used for Directed Signed Social Networks. Each node u is endowed with two parameters p u and q u drawn from an arbitrary joint distribution µ over [0, 1] × [0, 1]. p u can be interpreted as the tendency of u to send positive edges to other nodes (i.e. the "niceness" of u), while q u can be interpreted as the tendency of u to receive positive edges from other nodes (i.e. the "popularity" of u). The sign of an edge u → v is then chosen to be positive with probability pu+qv 2 . This suggests that nodes in the graph have a form of agency, and further imposes that edges are directed. We show experimentally that failing to meet these two assumptions harms the performance of our previous methods. This is not surprising, for our bias is no more justified in that case. Therefore, we suggest a different bias, drawing heavily on social balance theories, although we shall see later this holds for many nonsocial graphs as well.

Sign generative model and behavior

We can interpret the description of our sign model as if, when establishing a new link, nodes were deciding the sign of this link based on their individual preferences. While this makes sense in social networks where nodes represent human beings, We presented a way to circumvent this notion of node behavior with our online algorithm, where this time, signs are generated by an arbitrary adversary. However, even in that case, our bias remains that the labeling is regular. Recall this means informally that all the outgoing signs from a given node tend to be the same, and likewise for the incoming signs. Indeed, irregularities are the cost payed by the adversary to make our algorithm mispredict. In other words, regularity hints at a consistent sign behavior from nodes, that we cannot assume carry out outside of social networks. Pursuing the proteins example, we can imagine that proteins interact half of the time with proteins of their own group, and half of time with proteins from other groups. This would correspond to maximum irregularity, yet it is a plausible situation. We now show experimentally, that in fact, biological networks do not lend themselves to our sign model bias. Namely, we consider gene regulatory networks. The nodes of such directed graphs are various chemicals (such as genes, proteins or messenger RNAs) and a directed edge u → v indicates that one node u influences the concentration of another node v through a chemical reaction. This influence can be positive (that is, an increase of u concentration results in an increase of v concentration) or negative (that is, an increase of u concentration results in a decrease of v concentration). In this context, we could now interpret the "niceness" of u as it ability to accelerate chemical reactions, and its "popularity" as its propensity to take part in reactions accelerated by other nodes. Intuitively, this is quite far-fetched. To demonstrate this, we borrow the following three gene regulatory networks from [KvK09, Table 1], and display their statistics in Table 3.1:

1. The signaling pathways of the HIPPOCAMPAL CA1 neuron [START_REF] Ma'ayan | Formation of regulatory patterns during signal propagation in a mammalian cellular network[END_REF].

The interactions between genes and their products that are involved in CAN-

CER development in humans and mouses [START_REF] Cui | A map of human cancer signaling[END_REF].

3. The REGULONDB database [START_REF] Gama-Castro | Regulondb version 9.0: high-level integration of gene regulation, coexpression, motif clustering and beyond[END_REF], summarizing the connection between transcription factors and their targets in the Escherichia coli K-12 bacteria. We download four files1 containing experimental evidence and keep only the positive and negative edges with Strong support.

Compared with the Directed Signed Social Networks of the previous chapter, those biological networks are smaller, have lower average degree and little to no reciprocal edges. Trying to solve the EDGE SIGN PREDICTION problem using the same procedure as in Section 2.5, we read in Table 3.2 that our methods have lower absolute MCC performance (for instance, L. PROP. with a 15% training size achieves 20, 24 and 45 compared with values between 36 and 61 on Directed Signed Social Networks). To further illustrate the mismatch between the troll bias and biological networks, notice that the standard deviation figures are several times larger than those reported in Table 2.4. 3.0 ± 3.4 15.9 ± 3.6 26.7 ± 3.5 23 .2 ± 3.5 25% 24.4 ± 3.8 23.2 ± 3.2 21.5 ± 3.6 19.2 ± 2.6 21.5 ± 3.5 3.2 ± 3.4 18.5 ± 3.6 28.6 ± 4.7 24 .7 ± 3.7 50% 30.5 ± 4.5 32 .1 ± 4.4 32.0 ± 4.8 28.1 ± 4.2 31.9 ± 4.0 16.0 ± 3.9 31.5 ± 4.1 36.4 ± 3.9 31.2 ± 4.1 90% 34.4 ± 5.8 35.7 ± 7.1 35.5 ± 7.4 31.3 ± 7.8 40.8 ± 12.7 39 .9 ± 11.4 36.3 ± 9.0 35.9 ± 6.6 37.2 ± 9.3 CANCER 5% 15.6 ± 4.0 12.9 ± 4.1 10.6 ± 3.9 9.1 ± 2.9 13.0 ± 2.4 0.7 ± 1.9 5.7 ± 4.6 14.2 ± 5.1 14 .7 ± 3.3 10% 21 .7 ± 2.6 18.0 ± 3.2 15.9 ± 3.1 12.1 ± 2.6 16.9 ± 2.7

1 40.0 ± 6.7 90% 56.4 ± 6.2 50.4 ± 5.6 47.9 ± 6.1 47.2 ± 5.6 65.8 ± 5.5 47.2 ± 6.4 59.7 ± 6.1 64 .0 ± 4.7 49.5 ± 6.5

Directed edges requirement

Having two parameters per node, one for outgoing edges and another one for incoming edges, clearly targets directed graphs. Many online interactions are inherently directed, for instance friendship, trust or communication. On the other hand, predicting edge signs in undirected graphs is an equally relevant objective. A prime example of such a situation is when we are given n objects, some pairwise similarities them, and the similarity function itself, which unfortunately takes an exorbitant time to be evaluated. There is an underlying graph and being able to predict the sign of its undirected edges would save us expensive evaluations of the symmetric similarity function.

A trivial way to turn an undirected graph G into a directed graph G is to let V = V and, for every edge (u, v) in E, to add both u → v and v → u to E with the same sign as (u, v). In terms of our generative model, this corresponds to putting all the probability mass of µ on the p = q diagonal. This is clearly not a very satisfactory solution, for it removes one degree of freedom from the model. To illustrate this point, we conduct the following experiment. We use our previous Directed Signed Social Networks datasets and remove the edge direction. As mentioned earlier, for a few pair of nodes, there are reciprocal edges of different signs, in which case we pick a sign arbitrarily. Given those undirected graphs, we orient them using the approach described above and compare our method with the LOWRANK approach ran directly on the undirected graphs. This is a fair comparison, for LOWRANK is designed to works solely on undirected graphs.

As we can see in Table 3.3, our methods perform worse than when running on the original directed graphs. Looking at L. PROP. on a 15% training set as an example, we observe that the MCC decreases by one (WIK. EDITS) to almost ten (EPINION) points. On the other hand, the LOWRANK seems to perform better. Recall however that because it is an undirected method in the first place and because we roughly double the number of signed edges, a 15% training size means the algorithm gets to observe more signs and therefore perform better. This small experiment thereby shows that although the performance remains decent, not having direction information hurts our bias.

Social balance as a learning bias

From a narrow practical point of view, balance theories specify which sign assignments of a triangle are possible. By forbidding triangles with a single negative edge, weak balance gives rise to a K-consistent clusters structure on complete graphs.

After making these notions more detailed in the rest of this section, we conclude by precisely exposing our learning bias.

We focus here on balance theory, for it is both compelling and well-established. Indeed, as illustrated by the account of Zheng et al. [START_REF] Zheng | Social balance in signed networks[END_REF], it has been used extensively in the last half century to study signed networks and their dynamics (an example of this far reaching influence is the analysis of the relation between characters in fairy tales [START_REF] Auster | Balance theory and other extra-balance properties: an application to fairy tales[END_REF]). However, it has shortcomings, the main one being its inability to deal with directed graphs. Therefore, alternative theories governing signs formation have been proposed, as surveyed in [START_REF] Yap | Why does everybody hate me? balance, status, and homophily: the triumvirate of signed tie formation[END_REF].

In his seminal work on interpersonal relations, Heider [Hei46] posits through psychological and sociological arguments that in order to reduce their cognitive dissonance, three people always interact in a way that preserve social balance. These social balance requirements can be succinctly summarized by four statements [Hei58]:

1. my friend's friend is my friend + + + 3. my enemy's friend is my enemy -+ -

my enemy's enemy is my friend --+

This can be readily translated into graph properties. Given a path of length two over three nodes, note that the first part of each statement (in blue and orange) defines the four possible sign assignments of such a path. The last part of the statement (in green) then prescribes which sign should close the triangle to respect the social balance. Such triangles are called strongly balanced. Definition 3.1.1 (strongly balanced triangle). A triangle is strongly balanced if it has zero or two negative signs. Otherwise it is unbalanced.

We use triangles as the building block of our definition of strongly balanced graphs.

Definition 3.1.2 (strongly balanced complete graphs). A complete signed graph is strongly balanced if all its triangles are strongly balanced. It is interesting to note that this local property of triangles gives raise to a global structure. Namely, a strongly balanced complete graph can be split in two (possibly empty) clusters L and R such that all edges within L and R are positive, while all edges between L and R are negative. Indeed, let us pick an arbitrary node u and let L = N + (u) \ {u} and R = V \ L (where N + (u) denotes the set of all positive neighbors of u). Now let v, w ∈ L \ {u} and x, y ∈ R. From Figure 3.1 and the fact that every triangle is balanced, we can see that (v, w) and (x, y) are positive edges, while (w, y) is a negative edge. We say that L and R are 2-consistent clusters and more generally, we define Definition 3.1.3 (K-consistent clustering). Given a graph G = (V, E), a clustering C = {C 1 , . . . , C K } of V is consistent with the signs of E if for every edge (u, v) in E:

(i) (u, v) is positive if C(u) = C(v), and (ii) (u, v) is negative if C(u) = C(v).
We just saw that if a complete graph is strongly balanced, then it admits a 2-consistent clustering (Theorem 4 proves that the converse is true). It is natural to ask under which conditions a complete signed graph admits a K > 2 consistent clustering. This is where the notion of weak balance comes handy. It was noted by sociologists that among triangles with odd number of negative edges, one is more stable and common than that the other. Therefore, Davis [START_REF] Davis | Clustering and structural balance in graphs[END_REF] relax the strong balance into the weak balance by considering triangles with three negative edges to be balanced, as illustrated on Figure 3.2. Formally, we can modify our two previous definitions of balance to arrive at weak balance: Definition 3.1.4 (Weak balance). A triangle is weakly balanced if it has zero, two or three negative signs. Otherwise it is unbalanced. A complete signed graph is weakly balanced if all its triangles are weakly balanced.

In that case as well, the local property of triangles has structural implication for the whole graph. Namely, we can partition a complete weakly balanced graph into K consistent clusters. Consider the same construction as in Figure 3.1, putting a node u and its positive neighborhood in one cluster L and the rest of the nodes in R. For v, w ∈ L \ {u} and x, y ∈ R, we can still conclude that (v, w) is positive and (v, y) is negative. On the other hand, this time weak balance allows (x, y) to be either positive or negative. We can split R further into the positive neighborhood of x and the rest, and keep doing so until we form the k clusters. In the following we will make this argument more formal.

We gave the definitions of balance in the context of complete graphs in order to build intuition about the partition consequences and because it easier to reason on triangles. Furthermore, if we are given a set of objects and a similarity function, we can indeed build a complete graph. However, they are situations were it is not realistic to make the completeness assumption, especially for social networks that are typically very sparse. We therefore extend the concept of balance to general graphs in the following way: Definition 3.1.5 (balanced general graphs). A general graph is strongly (respectively weakly) balanced if there is a sign assignment of all its missing edges such that the resulting complete signed graph is strongly (respectively weakly) balanced.

To recover the characterization of signed graphs by consistent clustering, we need to consider longer cycles than triangles. Definition 3.1.6 (balanced cycles). A cycle is strongly (respectively weakly) unbalanced if it has an odd number of negative edges (respectively exactly one negative edge). Otherwise it is strongly (respectively weakly) balanced.

Being strongly balanced is equivalent to having a 2-consistent clustering, as proved in the landmark theorem of Harary [Har53, Theorem 3]:

Theorem 4 (Structural Theorem). A graph G = (V, E) is strongly balanced if and only if V admits a 2-consistent clustering.

According to Hüffner et al. [START_REF] Hüffner | Separator-based data reduction for signed graph balancing[END_REF], a similar theorem was proved earlier by König [Kön36, Theorem X.11], although Zaslavsky [START_REF] Zaslavsky | A mathematical bibliography of signed and gain graphs and allied areas[END_REF] notes that it was stated "without the terminology of signs, while [START_REF] Harary | On the notion of balance of a signed graph[END_REF] has the first recognition of the crucial fact that labelling edges by elements of a group-specifically, the sign group-can lead to a general theory."

A useful characterization, whose proof can be found in [EK10, page 122], is the following: Theorem 5. A graph G = (V, E) is strongly balanced if and only if all its cycles are strongly balanced.

Similar results also hold for weak balance. Theorem 6. A graph G = (V, E) is weakly balanced if and only if V admits a K-consistent clustering.

Proof. Let first assume that G is weakly balanced. According to the definition, we can choose sign for the missing edges such that it becomes a complete graph with all its triangles weakly balanced. As mentioned earlier, we can then pick a node u 1 and let C 1 = N + (u) ∪ {u}. All nodes in C 1 are connected positively with each other and negatively with nodes in V \ C 1 . Note also that |V \ C 1 | < |V | and therefore we can repeat this procedure until V is exhausted, at which point we have obtained our K consistent clusters.

Conversely, assume we have K consistent clusters. We can complete the graph G by letting the edge (u, v) be positive if u and v are in the same cluster and negative otherwise. Let us the pick three arbitrary nodes u, v, w ∈ V . There are three cases: they are either all in the same cluster, all in different clusters or two of them are in a first cluster and the third node is in a second cluster. In every case, one can check they form a weakly balanced triangle and therefore G is weakly balanced.

Theorem 7 (Theorem 1 of [START_REF] Davis | Clustering and structural balance in graphs[END_REF]). A graph G = (V, E) is weakly balanced if and only if all its cycles are weakly balanced.

In real data though, we do not expect either strong or weak balance to hold, for they are fairly demanding model. Indeed, three of the real networks considered in Chapter 2 have been repeatedly shown to be unbalanced, although the extent of this unbalance depends on the importance given to longer cycles compared with triads [FIA11; EB14; SA17]. Easley et al. [EK10, Section 5.5] nonetheless prove that even when only a fraction of the triangle are strongly balanced in a complete graph, the two clusters structure is still present, although it does not cover the whole graph any more.

Theorem 8. Let < 1 /8 and δ = 3

√ . If at least 1 -of all triangles in a signed complete graph are strongly balanced, then either (a) there is a set consisting of at least 1δ of the nodes in which at least 1δ of all edges are positive, or else (b) the nodes can be divided into two groups L and R such that (i) at least 1δ of the edges in L are positive, (ii) at least 1δ of the edges in R are positive, and (iii) at least 1δ of the edges between L and R are negative.

Using these results on weak balance, we can now make our new bias explicit. Drawing parallel with the previous generative model, we assume that each node u is endowed with an integer C(u) ∈ {1, . . . , K} that specifies its cluster and we let the undirected edge u, v be positive if C(u) = C(v) and negative otherwise. In other words, this corresponds to picking an arbitrary partition of V and setting the signs of E in such a way that the partition becomes a consistent clustering. This can still be seen as a generative process because of the initial integer drawn uniformly at random, but one that avoids the two pitfalls discussed before. Indeed, there is no decision to be made by nodes, since the previous probability has been replaced by a binary function (either both nodes are in the same cluster or not). Furthermore, since the clustering function is symmetric, the model is inherently suited to undirected graphs.

CORRELATION CLUSTERING

As mentioned earlier, most real signed graphs are not fully balanced and thus do not have a perfectly consistent clustering. The problem of quantifying how much a given graph departs from this ideal situation is called CORRELATION CLUSTERING. After defining it formally in Section 3.2.1, we show that CORRELATION CLUSTERING is a learning problem on its own, and discuss how it relates to the binary classification problem of predicting edge signs, among many others applications. We then present numerous methods to solve CORRELATION CLUSTERING in Section 3.2.2, ranging from exact methods (that implicitly leverage our bias) to heuristics (whose greedy and energy minimization frameworks are classic in unsupervised graph cut problems). We also devote a large section to approximation methods, in order to get a sense of why the problem is NP-COMPLETE and even hard to approximate. This allows us to finish in Section 3.2.3 by describing situations related to our bias where is possible to solve the problem efficiently and optimally. In this section, we thus update existing surveys on the same topic [BGL14; IIK16; Wir17] and highlight the importance of CORRELATION CLUSTERING to solve our binary learning problem.

Problem setting and connection to EDGE SIGN PREDICTION

Like other clustering frameworks, in CORRELATION CLUSTERING, we are given a set of objects and we want to gather them into groups (called clusters) so that objects belonging to one cluster are similar to each other while being dissimilar to objects from all the other clusters.

In CORRELATION CLUSTERING, we formalize this problem by considering objects as the nodes of a graph G, whose edge weights encode similarity. Namely, in the most general case, for nodes u and v, the edge between u and v is associated with two positive numbers: w + u,v denotes the strength of the similarity between u and v, while w - u,v denotes the strength of the dissimilarity between u and v. Note, however, that in many applications, only one of these two numbers is nonzero, in which case we more conveniently set

w u,v = w + u,v if w + u,v > 0 and w - u,v = 0 -w - u,v if w + u,v = 0 and w - u,v > 0 Now consider a clustering C of V , that is a function from V to N |V |
>0 that assigns to each node a cluster index. For instance, C(u) = 3 means that u belongs to the third cluster. We will also abuse the notation and let C be a set of clusters {C 1 , C 2 , . . . , C K } that form a partition of V . 2 We can evaluate how C fits our clustering paradigm in two ways, either by the number of agreements, that is the weighted number of positive edges inside clusters plus the weighted number of negative edges across clusters; or by the number of disagreements, that is the weighted number of negative edges inside clusters plus the weighted number of positive edges across clusters. Given a cost function c, which is usually the identity, CORRELATION CLUSTERING can then be seen as a graph optimization problem, either of maximizing agreements (MAXAGREE):

max C C(u)=C(v) c(w + uv ) + C(u) =C(v) c(w - uv ) (3.1)
or minimizing disagreements (MINDISAGREE)3 :

min C C(u)=C(v) c(w - uv ) + C(u) =C(v) c(w + uv ) (3.2)
Although an optimal clustering C achieves the same value on both (3.1) and (3.2), we will see in Section 3.2.2 that the latter objective is in some sense "easier". Another interesting feature of the CORRELATION CLUSTERING problem is that contrary to other clustering formulations, it does not require us to set the number of clusters K beforehand. Instead, K emerges as a natural property of the solution. Since clustering is an unsupervised problem, this is generally handy. However, in some situations, we may have prior knowledge on how many clusters are the data, or external constraints. This can be handled with parametrized version of CORRELATION CLUSTERING, namely MAXAGREE[K] and MINDISAGREE[K] where the optimization is over clustering with exactly K clusters.

In Figure 3.3, we show a simple instance of CORRELATION CLUSTERING and one of its optimal solution. C incurs two disagreements: the negative edge between nodes 1 and 2 within the blue cluster, and the positive edge (6, 7) between the orange and green clusters. Those disagreements are created by two cycles with one negative edge and thus cannot be avoided, meaning that C is optimal. However, it is not the unique solution: for instance, merging the orange and green clusters would also yield two disagreements.

Connection with EDGE SIGN PREDICTION and other applications EDGE SIGN

PREDICTION can clearly be casted as a supervised classification problem where, given a labelled training set of m 0 edges X = {(e 1 , y 1 ), . . . , (e m 0 , y m 0 )}, we can extract for every edge a feature vector in R d , pick an hypothesis class like linear models, find among this class the hypothesis minimizing the empirical risk, and use it to predict the sign of edges in the testing set. In contrast, CORRELATION CLUSTERING in an agnostic problem. More precisely, the hypothesis class is fixed, as it consists of all possible partitions of the nodeset. We are trying to find the partition that best approximates the observed signs, but each weakly unbalanced cycle will cost us at least one unavoidable mistake. Despite this fundamental difference in their nature, CORRELATION CLUSTERING is connected with EDGE SIGN PREDICTION at two levels, theoretical and practical. On the theoretical side, computing the minimum number of disagreements gives a measure of the EDGE SIGN PREDICTION problem complexity. Intuitively, given a labelled subgraph H of G (that is the training set), if there are few disagreements, the "quasi" connected components of H + should give a reasonable estimation of the underlying consistent clustering. Predicting the signs based on this clustering should results in few errors. More formally, one can show that by finding a clustering C of H that has less disagreements than κ times the smallest number of disagreements ∆, the number of errors on the testing set will be at most of order κ∆ + √ mn log n [Ces+12b, Theorem 6]. These theoretical considerations naturally lead to a practical batch algorithm, which first finds a clustering minimizing as much as possible the number of disagreements on the training set and then predicts signs according to this clustering. Our assumption in doing so is that, according to our bias, our graph has few disagreements. Therefore, provided we can find a clustering close to the optimal, we will not make too many prediction mistakes.

Besides the special case of predicting edge sign, Demaine et al. [Dem+06, Section 5] point out that CORRELATION CLUSTERING is well suited to several more general situations:

• when the items to be clustered do not belong to a natural metric space (preventing approaches such as k-means) but we still know for some pairs whether they are similar or not.

• when we do not know the number of clusters beforehand but we have a similarity measure. In that case, we can select a problem-specific similarity threshold and set all edges with a similarity larger than the threshold to be positive while the others are set to negative.

• when we have a classic clustering problem (that is a set of objects, a distance between them and an objective function to minimize) with additional pairwise constraints of the form must-link/cannot-link. Instead of restraining a clustering algorithm to the space of feasible solution, we convert the distances between objects and the constraints into signed edges and solve the resulting CORRELATION CLUSTERING problem.

Furthermore, as mentioned in the introduction of this thesis on page 4, CORRE-LATION CLUSTERING have been used in many domains. For instance in computer vision, it is used to segment images in 2D [Kim+11; BG11; ZYH14] or 3D [And+12; BHK15], to simplify 3D shapes [START_REF] Gori | Flowrep: descriptive curve networks for free-form design shapes[END_REF] and to track targets across sequential video frames [START_REF] Solera | Learning to divide and conquer for online multi-target tracking[END_REF]. When the nodes are words, CORRELATION CLUSTERING is employed to identify coreference [MW05; ES09] or cluster synonym words [START_REF]Predicting Emotional Word Ratings using Distributional Representations and Signed Clustering[END_REF]. It plays a large role in biology, to clusters genes [START_REF] Ben-Dor | Clustering gene expression patterns[END_REF], identify mutation regions in chromosomes [START_REF] Das | Sdhap: haplotype assembly for diploids and polyploids via semi-definite programming[END_REF] or finding stable subsystems [START_REF] Dasgupta | Algorithmic and complexity results for decompositions of biological networks into monotone subsystems[END_REF]. Applied

State of the art

As we saw, solving CORRELATION CLUSTERING on a training set would give us a principled way of predicting edge signs. In this section, we thus look at existing methods for CORRELATION CLUSTERING. We start with exact methods and fixed parameter algorithms on complete graphs, noting they implicitly assume our learning bias. However, in practice, these methods do not scale well, especially if their assumptions are not met. Indeed, we next present hardness results about CORRELATION CLUSTERING, namely that it is NP-HARD to approximate with some constant factor greater than one, and then describe some proposed approximations. Although this appears rather negative at first, we show that heuristics can solve large instances with satisfactory results. We conclude by further highlighting the learning aspect of CORRELATION CLUSTERING through approaches in the active and online settings.

Exact methods

Because of the complexity of the CORRELATION CLUSTERING, one has to rely on approximations to solve large instances of the problem. However, we can imagine offline signed social networks with only few nodes, in which case it is reasonable to expect to find the optimal clustering. Furthermore, this can also be useful to evaluate in practice the quality of heuristic methods, albeit in non asymptotic settings. The methods used to solve CORRELATION CLUSTERING exactly depend on the type of graph considered. More precisely, the fixed parameter algorithms we describe now only handle complete graphs, while the next set of methods also work on general graphs.

Complete graphs

Let us first describe the cluster editing problem, where one is given a general unsigned input graph H = (V, E) and wants to find the smallest number of edges that have to be added or deleted to turn H into node-disjoint union of cliques. It is possible to alternate between an instance H of cluster editing and an instance G of CORRELATION CLUSTERING on complete graphs in polynomial time. To see why, let G = (V, (E + , E -)) be a complete signed graph where V is the nodeset of h, E + is the set E of edges of H and E -is the set of all edges that are not in H. The optimal clusters of G are the cliques of H, the negative disagreements within clusters are edges added to H and the positive disagreements between clusters are deleted from H. To the best of our knowledge, the problem was first introduced under this name by Shamir et al. [START_REF] Shamir | Cluster graph modification problems[END_REF]. 4 They show the problem is NP-COMPLETE, even if the number of clusters K ≥ 2 is set beforehand5 and provide a 0.878-approximation in the weighted K = 2 case using the celebrated Goemans-Williamson SDP relaxation [START_REF] Goemans | Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming[END_REF].

If we parameterize the problem by the number d of edges that need to be added or deleted (that is, the number of disagreements), then it can be solved in polynomial time in the size of the input graph (but not in d). The best known approximation so far is O(1.62 d + m + n) [START_REF] Böcker | A golden ratio parameterized algorithm for cluster editing[END_REF], which search for conflict triples (i.e. unbalanced triangles in the sign language) and branch by either deleting or merging one positive edge. If we additionally look for exactly K clusters (i.e. MINDISAGREE[K]), there is a fixed parameters algorithm running in O(2 O( √ kd) + n + m) [START_REF] Fomin | Tight bounds for parameterized complexity of cluster editing with a small number of clusters[END_REF]. If at most a edges can be added and at most b edges can be deleted at each node, and if the minimum size of a cluster is at least 2(a + b), then the problem can be solved in polynomial time [START_REF] Abu-Khzam | On the complexity of multi-parameterized cluster editing[END_REF] by using various reduction rules. Finally, for planar graph, there is a PTAS6 running in O(n2 -1 log( -1 ) ) obtained by dividing the graph into independent components of bounded treewidth [START_REF] Berger | A ptas for the cluster editing problem on planar graphs[END_REF]. In other words, if we are given a complete signed graph whose positive subgraph is planar, then we can minimize the number of disagreements arbitrarily close to the optimal, provided we have enough time to do so.

General graphs

On general graphs, a natural idea is to formulate the clustering problem as an integer problem and solve it optimally in polynomial time. Assign a binary variable x uv to each pair of nodes (so that x uv = x vu ). For a given clustering C, let x uv = 0 if u and v are in the same cluster and x uv = 1 is u and v are in different clusters. Noting that 1x uv is 1 if the edge (u, v) is within a cluster and 0 otherwise, the weighted number of disagreements is then w(C) = (u,v)∈E -w uv (1x uv ) + (u,v)∈E + w uv x uv . By construction, if edges (u, v) and (v, w) are within the same cluster, then (v, w) is also within that cluster. In terms of x variable, we have that x uv = 0 ∧ x vw = 0 =⇒ x uw = 0. For x to be a valid cluster assignment, we thus require that all variables are either 0 or 1 and respect the triangle inequality. minimize

(u,v)∈E - w uv (1 -x uv ) + (u,v)∈E + w uv x uv (3.3) subject to x uw ≤ x uv + x vw ≤ x uv ∈ {0, 1} (3.4)
x uv = x vu Figueiredo et al. [START_REF] Figueiredo | Mixed integer programming formulations for clustering problems related to structural balance[END_REF] solve this binary integer program on random instances and show that depending on the negative edge density, the FICO Xpress solver starts to be unable to finish within a one-hour time limit when n ≥ 40. Aref et al. [START_REF] Aref | An exact method for computing the frustration index in signed networks using binary programming[END_REF] describe four linear and quadratic binary integer problems to model CORRELATION CLUSTERING with two clusters along with some preprocessing optimizations. With the Gurobi software, they solve instances with up to 3200 edges in less than a second. They also sketch extensions to weighted graphs and more than two clusters. In a different direction, Berg et al. [START_REF] Berg | Cost-optimal constrained correlation clustering via weighted partial maximum satisfiability[END_REF], encode the linear and quadratic integer formulations into weighted MaxSAT instances and use the state of the art solver MaxHS7 [START_REF] Davies | Exploiting the power of mip solvers in maxsat[END_REF] to get the exact solution on instances with at most 1000 nodes in less than a few hours.

There is also a fixed parameter algorithm to solve MINDISAGREE for general unweighted graphs. Indeed, we shall see later than the problem is equivalent to k-MINIMUM MULTICUT, which in this case asks for the minimal number d of edges to remove in order to separate every pair of nodes in S = {(s i , t i )} k i=1 ∈ V 2 . It turns out that k-MINIMUM MULTICUT can be solved exactly in time 2 O(d 3 ) • n O(1) [START_REF] Marx | Fixed-parameter tractability of multicut parameterized by the size of the cutset[END_REF].

After defining the matrix A by A uv = w + uvw - uv , Veldt et al. [START_REF] Veldt | Correlation clustering with low-rank matrices[END_REF] show that the MAXAGREE problem can be written up to constant factor by defining one vector x u per node as:

maximize u<v A uv x T u x v (3.5) subject to x uw ∈ {e 1 , . . . , e n }
where e i are the canonical vectors of R n . They show that when the matrix A is positive semidefinite of rank k, the CORRELATION CLUSTERING problem can be solved exactly in O(n k 2 ) time. More practically, they also give an algorithm that closely approximate the objective of (3.5) in O(nk) time.

Reflecting on these methods, we can conclude that CORRELATION CLUSTER-ING is solvable in (near) polynomial time when the problem instance follow our learning bias. Indeed, fixed parameter algorithms are efficient when the number of disagreements is low, which corresponds to a quasi consistent clustering. Likewise, the small rank assumption of the adjacency matrix is equivalent to assuming a consistent clustering [Chi+14, Theorem 13]. Even when solving integer programs, one can use the bias to add inequalities and thus tighten the polytope of admissible solutions. On the other hand, we will now see that in the worst case, for example if an instance does not obey our bias, then there is little hope to obtain an optimal solution.

Hardness and approximations

Although the same problem was considered earlier [DM96; BSY99], Bansal et al.

[BBC02] coin the term CORRELATION CLUSTERING and are the first to study this problem complexity. Namely, for complete, unweighted signed graphs, they show that both MINDISAGREE and MAXAGREE are NP-COMPLETE. Along the way, they give a 17429-approximation of MINDISAGREE and a PTAS that, for any ∈ [0, 1], runs in O(n 2 e O( -10 log -1 ) ) and returns with probability 1 -/3 a solution with at most n 2 fewer agreements than the optimal solution of MAXAGREE.

The next year, several authors independently strengthened these results and extended them to weighted and general graphs, as summarized in Table 3.4. In the most complete paper, Charikar et al. [START_REF] Charikar | Clustering with qualitative information[END_REF] show that on complete graphs, in sharp contrast with MAXAGREE that admits a PTAS, MINDISAGREE is APX-HARD, "that is, is NP-hard to approximate within some constant factor greater than one". Unfortunately, it is not easy to give intuition why, for this follows from a "somewhat intricate reduction from max 2-colorable subgraph problem on bounded degree 3-uniform hypergraphs". On general graphs though, both versions of the optimization problem are APX-HARD. It is showed for MAXAGREE by a reduction from MAX 3SAT [CGW03, Theorem 9] and for MINDISAGREE by using a reduction from the k-MINIMUM MULTICUT problem [CGW03, Theorem 8]. In the case of MINDISAGREE, Emanuel et al. [START_REF] Emanuel | Correlation clustering: minimizing disagreements on arbitrary weighted graphs[END_REF] show a reduction in the other direction. Because k-MINIMUM MULTICUT is an interesting graph cut problem, and its complexity has been well studied, we here give more details on its equivalence with CORRELATION CLUSTERING.

For that we follow [START_REF] Demaine | Correlation clustering in general weighted graphs[END_REF] but omit the full proofs to avoid redundancy. In k-MINIMUM MULTICUT, given a weighted graph H = (V H , E H , w H ) and a collection of k sources and targets S = {(s i , t i )} k i=1 ∈ V 2 H , we want to find the lightest set of edges T ∈ E H whose removal disconnects every pair s i from the corresponding t i . Let us first describe a polynomial transformation from k-MINIMUM MULTICUT to CORRELATION CLUSTERING. Namely, given an instance (H, S) of k-MINIMUM MULTICUT, we let W H = e∈E H w H (e) be the total weight of H and G H be the same graph as H with all its edges labeled positively. We then connect every pair (s i , t i ) by a negative edge of weight W H + 1. One can check ([Dem+06, Theorem 4.7]) that a multicut T in H with weight W induces a clustering in G H of weighted disagreement W by the connected components of G + \ T . Likewise, a clustering of G H with weight W induces a multicut on H with weight as most W . In the unweighted case, the reduction is similar but the heavy negative edges are simulated in the following way: include every source and target in a clique of n nodes and connect that clique to the corresponding source or target by n negative edges.

We next present the polynomial transformation from CORRELATION CLUSTER-ING to k-MINIMUM MULTICUT. Given G = (V, (E + , E -), w) we let H G be the graph induced by E + with the same weight. Then for every negative edge (u, v) ∈ E - of weight w - uv , we create a new vertex v uv , connect u to v uv with weight w - uv and let (v uv , v) be a source-target pair added to S G . This construction is depicted in Figure 3. If a certain conjecture in computational complexity is true 8 , then for every constant c > 0, it is NP-HARD to approximate k-MINIMUM MULTICUT within a factor c [START_REF] Chawla | On the hardness of approximating multicut and sparsest-cut[END_REF]. On general graphs, the best known approximation factor in O(log k) [START_REF] Garg | Approximate max-flow min-(multi)cut theorems and their applications[END_REF], which according to the presented reduction translates to O(log

|E -|) = O(log n 2 ) = O(log n).
This approximation ratio is achieved by first solving the relaxation of the integer problem (3.3), where the constraint (3.4) is replaced by x uv ∈ [0, 1]. From now on, we refer to that relaxation as the canonical MINDISAGREE LP. Once this canonical MINDISAGREE LP is solved, we interpret x uv as a distance: the larger it is and the more we want u and v to be in different clusters. We can then use the REGIONGROWING method [START_REF] Garg | Approximate max-flow min-(multi)cut theorems and their applications[END_REF]. The idea is to pick a random center u and to add to u's cluster all the nodes at distance less than r from u before removing that cluster and repeating the process. r < 1 /2 is chosen adaptively such that the weight of positive edges leaving the cluster is less than c log(n + 1) times the fractional weights of the positive edges inside the cluster.

However, Charikar et al. [CGW03, Theorem 2] note that the LP formulation has a poor integrality gap when it comes to MAXAGREE, thus they turn to a Semi Definite Program. Say that each cluster is associated with a basis vector, then for each node u in a cluster, we set a u to be the corresponding basis vector. If u and v are in the same cluster, we then have a u • a v = 1 while if they belong to different clusters, a u • a v = 0. The weighted number of agreements can then be represented by maximize In complete graphs, Charikar et al. [CGW03, Section 3] also give an improved 4-approximation to MINDISAGREE, by rounding the same LP and using a simpler version of REGIONGROWING with fixed radius. Namely, we pick a ball center u uniformly at random with radius 1 /2: if the average distance of the nodes in the ball to u is less than 1 /4, the ball forms a cluster, otherwise {u} forms a singleton cluster. We then remove the corresponding nodes from the graph and repeat until all nodes are clustered.

(u,v)∈E + w uv (a u • a v ) + (u,v)∈E - w uv (1 -a u • a v ) (3.6) subject to a u • a u = 1 a u • a v ≥
Not long after, Ailon et al.

[ACN05] came up with a better approximation. To explain it, we first describe their randomized combinatorial algorithm KWIKCLUS-TER, which gives a 3-approximation of MINDISAGREE on complete unweighted graphs (and was later derandomized while preserving its approximation guarantee [START_REF] Van Zuylen | Deterministic algorithms for rank aggregation and other ranking and clustering problems[END_REF]). At each iteration, we pick a node u uniformly at random (called the pivot) and we create a cluster containing u and all its neighbors linked by a positive edges. On weighted complete graphs, they tweak this algorithm by using the solution of the canonical LP to obtain different approximation factor depending on the constraints imposed on the weight. Recall that in the general formulation of the problem, each edge carries two positive numbers: w + u,v and w - u,v . If the weights respect the probability constraints stating that for all edge (u, v) in E, w + u,v +w - u,v = 1, this tweaking provide a 2.5-approximation. Note that unweighted graphs naturally fit into that case, as each edge is either labeled + or -. If the weights additionally respect the triangular inequality constraints stating that w - u,v ≤ w - u,w + w - w,v , this Table 3.5 -Best current results on CORRELATION CLUSTERING problems. The "easiest" setting is MAXAGREE on complete graphs, for it admits PTASs. All others cases are APX-HARD. However, we see that on the diagonal (that is MINDISAGREE on complete graphs and MAXAGREE on general graphs), there exists constant factor approximations. This is not the case for the most "difficult" problem, MINDISAGREE on general graphs.

G MINDISAGREE MAXAGREE

Complete 2.06 (and 1.5 if G is weighted and weights obey the triangular inequality) [START_REF]Near optimal lp rounding algorithm for correlationclustering on complete and complete k-partite graphs[END_REF] PTAS from Bansal et al.

[BBC02] and by setting

k = Ω(1/ ) in [GG06] General O(log n) [CGW03] 0.7666 [Swa04]
become a 2-approximation. After solving the canonical LP with additional probability constraints, when picking a node u, each of its neighbors v is added to the cluster of u with probability x uv . Chawla et al. [START_REF]Near optimal lp rounding algorithm for correlationclustering on complete and complete k-partite graphs[END_REF] improve these two factors to respectively 2.06 and 1.5 by exploiting the same idea but setting the probability to include each neighbor v of u in the cluster of u to be 1 - This concludes the presentation of the known approximation results on COR-RELATION CLUSTERING, that we gather in Table 3.5. To summarize, except for maximizing the number of agreements on a complete graphs, we cannot hope for exact solutions for the other problems in the worst case, since they are APX-HARD. Moreover, even constant factor approximation are ruled out for MINDISAGREE on general graphs, as showed by the equivalence with k-MINIMUM MULTICUT. Currently there are three main approaches to approximate MINDISAGREE. First, on a general graph, we can solve the canonical LP, and use the REGIONGROWING method to round the fractional solution, resulting in an O(log n) approximation. Second, on complete graphs, the simplest solution is the KWIKCLUSTER algorithm, that grows clusters out of randomly selected pivots, in a way reminiscent of the proof of Theorem 6. Third, the approximation factor of this general idea can be improved by interpreting the fractional solution of the linear program as probability to be included in the pivot's cluster. The cost of solving linear programs with so many variables and constraints being prohibitive, real world general graphs requires to move from methods with guarantees to more heuristic approaches we describe next. Before that, we make two remarks. The first one regards a line of work improving scalability of these approximations by presenting distributed solutions. The second one is concerned with the case where the number of clusters is set beforehand, and proves that the problem becomes easier.

f + (x uv ) if (u, v) ∈ E + and 1 -f -(x uv ) if (u, v) ∈ E -,
To handle the massive size of some large real world dataset, parallelizing existing algorithms is a natural approach. Due to its simplicity and approximation guarantee, KWIKCLUSTER has been adapted three times for that purpose. First, Chierichetti et al.

[CDK14] describe how to uniformly sample several pivots in the same round, remove pivots that are adjacent through positive edges and then grow the corresponding clusters with potential conflicts solved according to the node order in a global permutation drawn at the beginning of the algorithm. On general graphs, this requires O(log n diam(G)) rounds, which on complete graphs reduces to O(log n). Furthermore, this almost preserves the approximation factor, which is 3 + 14 1-7 , where is a parameter smaller than 1. Finally, this allows to experimentally cluster graphs with millions of nodes and billions of edges. Second, Pan et al. [START_REF] Pan | Parallel correlation clustering on big graphs[END_REF] describe an equivalent version of KWIKCLUSTER where a permutation π of the nodes is drawn at the start of the algorithm and the pivots are chosen sequentially in the order of π. The exact same partition can be obtained when several pivots are chosen at the same time by different threads as long as they respect two concurrency rules: (i) two nodes u and v can become pivot at the same time if they are not connected with a positive edge, otherwise only the one with the smallest index in π becomes pivot; (ii) if w is a positive neighbor of two pivots u and v, it is affected to the cluster of the pivot with the smallest index in π. Enforcing these rules preserves the factor 3 approximation and the algorithms terminate after O(log n diam(G)) rounds. The authors also present a version where each round is faster as it does not enforce rule (i). However, this weaken the approximation guarantee to (3 + )OP T + O( n log 2 n). In practice, the solution is very close the one of KWIKCLUSTER, although it degrades as the number of threads increases.

The number of rounds preserving the 3-approximation is lowered to O(log log n) by Ahn et al. [START_REF] Ahn | Correlation clustering in data streams[END_REF]. They also present results obtained in a single pass, which corresponds to the streaming model: the algorithm receives the edges of G one by one and upon seeing the last one outputs its result. The additional constraint is that this algorithm can only use O(n polylog n) space. In this setting, the authors show a polynomial time (1 -)-approximation of MAXAGREE if the weights are bounded (and 0.766(1 -) if the weights are arbitrary); and an O(log |E -|) approximation of MINDISAGREE in polynomial time with arbitrary weights. This is done by combining graph sketching and a method to solve convex programs in a space efficient manner.

Bonchi et al. [START_REF] Bonchi | Local correlation clustering[END_REF] suggest a different paradigm to solve CORRELATION CLUSTER-ING, that can be applied in a distributed setting to obtain a scalable approach. Namely, given a node u, they want to output a globally consistent cluster index C(u) while making at most t queries to a sign oracle. Here t is a parameter that depends on the quality of clustering produced but not on the size of the graph. And because this procedure is local to each node, it can be run in parallel. Finally, one can get a full clustering by computing C(u) for all the nodes in the graph. Despite the problem being apparently more challenging, they obtain approximation factors that are close to the best known (which in this model would make Ω(n 2 ) total queries). More precisely, they use two techniques. The first one is inspired by KWIKCLUSTER. It starts by finding a good set of pivots, seeing the problem as finding a maximal independent set on a sampled part of the positive graph. Then, for a given node, it finds the closest such pivot or creates a singleton. Given a quality parameter ∈ (0, 1), it yields a 4 • OP T + n 2 approximation of MINDISAGREE requiring O( n ) time and queries [BGK13, Theorem 3.3]. Roughly stated, the second technique relies on an existing low-rank approximation of the adjacency matrix, that partition the graph into similar sized classes such that edges between those classes behave as in a random graph. This initial partition is "coarsened" into a good clustering by considering all possible ways of assigning classes to clusters. It gives an OP T + n 2 additive approximation for MINDISAGREE that runs in time n • poly( 1 / ) + 2 poly( 1 / ) [BGK13, Corollary 3.7].

As mentioned earlier, not having to set the number of clusters is an attractive feature of the CORRELATION CLUSTERING problem, but in some cases we may want to use prior knowledge. The problem was studied by Giotis et al. [START_REF] Giotis | Correlation clustering with a fixed number of clusters[END_REF] on general graphs and we compile their results in Table 3.6. On complete unweighted graphs and with K being the number of clusters, they provide PTASs for MAXAGREE[K] running in nk O( -3 log( K )) time and for MINDISAGREE[K] running in n O( -2 9 K ) log(n) time. The latter was improved by Karpinski et al. [START_REF] Karpinski | Linear time approximation schemes for the gale-berlekamp game and related minimization problems[END_REF], with a PTAS running in n 2 2 O( -3 K 6 log d) and that can handle weighted graphs. For K = 2, MINDISAGREE[2] admits a faster local search method with a factor 2 approximation [START_REF] Coleman | A local-search 2-approximation for 2-correlation-clustering[END_REF]. For complete weighted graph, Bonizzoni et al. [START_REF] Bonizzoni | On the approximation of correlation clustering and consensus clustering[END_REF] provide a PTAS for MAXAGREE[K] under the condition that the ratio between the largest and smallest weights is bounded by a constant.

Heuristics

While all the methods we discuss so far are either exact or come with some approximation guarantees, practitioners have also develop approaches that are designed to efficiently reach a solution that is satisfactory enough for the application at hand.

Greedy methods

Several of such methods fall into the greedy framework. For instance, while studying small scale signed social networks, Doreian et al. [START_REF] Doreian | A partitioning approach to structural balance[END_REF] describe the following procedure. Start with a random initial k clustering of the graphs and for T steps, sample randomly P neighboring partitions, compute their number of disagreements and move the one with the least disagreements. Neighboring partitions are obtained either by moving one node from cluster to another or by exchanging a pair of nodes between two clusters. The overall complexity is O(nT P ). This is quite similar to the Best One Element Move described in [START_REF] Gionis | Clustering aggregation[END_REF], except that there the exchange operation is replaced by moving one node to its own singleton cluster. The Cluster Affinity Search Technique algorithm [BSY99] instead grows clusters one by one by maintaining for the current cluster the affinity of all nodes, which is the sum of the weights between that node and the nodes in the cluster. Nodes above a certain threshold are added to the current cluster and nodes below the threshold are removed, with the affinity to the current cluster being recomputed after each addition or deletion.

After defining the net weight of an edge to be w ± uv = w + uvw - uv , Elsner et al. [START_REF] Elsner | Bounding and comparing methods for correlation clustering beyond ilp[END_REF] describe three folklore heuristics that start with empty clusters and add node one by one: "The BEST algorithm adds each node u to the cluster with the strongest w ± connecting to u, or to a new singleton if none of the w ± are positive. The FIRST algorithm adds each node u to the cluster containing the most recently considered node v with w ± uv > 0. The VOTE algorithm adds each node to the cluster that minimizes the CORRELATION CLUSTERING objective, i.e. to the cluster maximizing the total net weight or to a singleton if no total is positive." Empirically, VOTE turns out to be the best. Among other related heuristics, Lingas et al. [START_REF] Lingas | Iterative merging heuristics for correlation clustering[END_REF] describe the random maximum merging algorithm, that starts with singleton clusters and keep merging two clusters chosen at random among those whose merge would result in the maximum improvement of the score function. This runs in O(n 2 log n), and empirically results in fewer disagreements than KWIKCLUSTER. Building upon their previous GRASP work [START_REF] Drummond | Efficient solution of the correlation clustering problem: an application to structural balance[END_REF], an iterated local search (ILS) heuristic is presented in [START_REF] Levorato | An ils algorithm to evaluate structural balance in signed social networks[END_REF][START_REF] Levorato | Evaluating balancing on social networks through the efficient solution of correlation clustering problems[END_REF]. Each iteration of this algorithm starts by greedily building a clustering in a fashion similar to VOTE, albeit with more randomness in the node ordering. This clustering is locally improved by moving blocks of r ∈ {1, 2} nodes from one cluster to another as long as the number of disagreements decreases, a phase called neighborhood descent. The algorithm then enters an inner loop where the current clustering is perturbed by t random one-node-moves and updated if a subsequent neighborhood descent can improve it compared with before the perturbation. The authors note that both the outer loop and the neighborhood descent can be run in parallel, which allow them to process a 10 000 nodes graph on 10 cores in around 700 seconds. Bastos et al. [START_REF] Bastos | Efficient algorithms for cluster editing[END_REF] show how to use ILS to generate initial solutions of the Cluster Editing problem that are then fed to an integer program. Instead of moving nodes, Wang et al. [START_REF] Wang | Restoring: a greedy heuristic approach based on neighborhood for correlation clustering[END_REF] starts from the observation that in a balanced graph, G + is a disjoint union of cliques in which all the nodes of a given cluster share the shame neighbors. Their algorithm is initialized with the set E + s of edges where w + uv > w - uv , and repeatedly samples an edge (u, v) from E + s before trying to make the neighborhoods of u and v coincide by adding or removing edges in E + s . Yet another idea is to modify the canonical LP to replace the binary variable x uv by a k clusters indicator matrix L ∈ R k×n where L iu is 1 is u ∈ C i and -1 otherwise [START_REF] Wang | A scalable approach for general correlation clustering[END_REF]. Indeed, x u• ≡ L C(u)• , where C is the cluster assignment. By relaxing the integer constraints on L, it is possible to do alternate optimization on L and C.

Physics inspired energy methods

Here we describe a physics particle model that can readily be adapted to model CORRELATION CLUSTERING and provide additional heuristic methods. The Potts model [START_REF] Wu | The potts model[END_REF] describes a general model of spins organized in a lattice and being in one of k possible states. A spin u interacts with each of its neighbors v through a coupling J uv . The energy of this system, called the Hamiltonian, is defined by H(σ) =u,v J uv δ(σ u , σ v ) where σ is the spin configuration (that is, σ u ∈ {1, . . . , k} ∀u) and δ(σ u , σ v ) is equal to one if u and v are in the same state and zero otherwise. It is a general principle of physics that isolated systems tend to minimize their energy, which in this case amounts to finding a spin configuration minimizing the Hamiltonian. Viewing the spins as nodes of a graph, the couplings as the graph weighted edges and the k possible states as clusters, it is quite natural to formulate the clustering problem as a Hamiltonian minimization problem [START_REF] Reichardt | Statistical mechanics of community detection[END_REF].

Letting A be the matrix such that A uv = w + uvw - uv on a general directed graph, Traag et al. [START_REF] Traag | Community detection in networks with positive and negative links[END_REF] reward positive and absent negative links within cluster and penalize negative and absent positive links across clusters to come up with the following Hamiltonian:

H(σ) = -u,v (A uv -(γ + p + uv -γ -p - uv )) δ(σ u , σ v )
where γ + and γ -are user parameters and p ± uv are null model probabilities, which are equal to

p ± uv = |E ± | |V |(|V |-1) , or p ± uv = d ± out (u)d ± in (v) |E ± |
in the degree corrected model. They note that setting γ + = 0 = γ -make minimizing the Hamiltonian equivalent to the MINDISAGREE problem, which they do by simulated annealing [START_REF] Kirkpatrick | Optimization by simulated annealing[END_REF]. The idea is to start from a random partition, and jump to another partition by moving a single node from one cluster to another. Such moves are made with a probability proportional to how each move reduces the Hamiltonian and as the procedure goes on, large jumps are made less and less probable by reducing a parameter called the temperature. One advantage of this energy formulation is that it requires only one variable per node instead of one variable per edge as in the case of the linear program.

In the case of 2-CORRELATION CLUSTERING, Facchetti et al. [START_REF] Facchetti | Computing global structural balance in large-scale signed social networks[END_REF] rewrite the Hamiltonian as

H(σ) = -1 2 σ T Aσ = -1 2 1 T T σ AT σ 1.
There T σ = diag(σ) (where σ ∈ {0, 1} n ) is the outcome of a local search algorithm such that A σ = T σ AT σ has the same number of disagreements as A but the smallest number of negative edges. This is called a gauge transformation in the spin glass literature and the benefit of that heuristic is that it scales gracefully to large graphs. Bagon et al. [START_REF] Bagon | Large scale correlation clustering optimization[END_REF] also write the MAXAGREE objective as a Potts model, and show that it can be interpreted as the log posterior of a partition matrix under a simple generative model and as a pair-wise conditional random field energy without unary terms. This allows them to adapt existing discrete energy optimization algorithms in order to cope with the following three challenges of the CORRELATION CLUSTERING energy: "(i) the energy is non sub-modular, (ii) the number of clusters is not known in advance, and (iii) there is no unary term". Doing so, they are able to handle large problems with more than 100K nodes. Also adopting an energy minimization approach, Kappes et al. [START_REF] Kappes | Multicuts and perturb & map for probabilistic graph clustering[END_REF] assign a probability to each cut of a signed graph proportional to the exponential of the number of disagreements of that cut. They also develop efficient cut sampling methods. Several of these methods have recently been evaluated empirically by Levinkov et al. [START_REF] Levinkov | A comparative study of local search algorithms for correlation clustering[END_REF].

Active and online settings

Except for the local algorithms of Bonchi et al. [START_REF] Bonchi | Local correlation clustering[END_REF], all works presented so far considered the batch case of CORRELATION CLUSTERING, where the whole graph and all the signs are available at all time and with no cost. This might not always be the case, for instance if the graph is too large to fit in memory or if the edge labels are given by an expensive external procedure.

Case in point, in the context of entity resolution, Kanani et al. [START_REF] Kanani | Improving author coreference by resource-bounded information gathering from the web[END_REF] consider the CORRELATION CLUSTERING problem where, by querying an oracle, one can either reduce the uncertainty about one edge weight or add an extra node with edges connecting it to existing nodes. However, this requires web queries that are resource bounded and thus yields an active setting learning problem, where one has to choose the most informative queries given a budget. A formal definition is given in [START_REF] Kanani | Resource-bounded information gathering for correlation clustering[END_REF] and in practice, after finding an initially good partition, they select in each cluster a node to be its centroid, and query the edges connecting the centroid as ordered by an entropy-based criterion. In the case the oracle answering the queries is consistent, there exists an information theoretic bound on the number of queries needed to recover the clusters, and an algorithm matching that bound up to a O(log n) factor [START_REF] Mazumdar | Query complexity of clustering with side information[END_REF]. Ailon et al. [ABE14, Section 5] present another active algorithm that solves MINDISAGREE[k] with a linear number of queries but an exponential running time in n. In a noisy setting, Mitzenmacher et al. [START_REF] Mitzenmacher | Predicting signed edges with o(nlog n) queries[END_REF] assume there is a planted k-partition and that we have access to an oracle that for u, v ∈ V 2 returns C(u)-C(v) mod k with probability 1-p and a noisy C(u)-C(v)±1 mod k otherwise. For k = 2, this is exactly 2-CORRELATION CLUSTERING, whereas for k > 2, the oracle gives more information than simply the sign of the path from u to v. Whenever p < 1 /2, they show that O(n 1+ 1 log log n log n) queries are enough to recover the planted partition in polynomial time and with high probability. Those queries are actually random (i.e. non-adaptive), and the clusters are found by looking at almost edge-disjoint path between all pairs of nodes.

In the online setting, Mathieu et al. [START_REF] Mathieu | Online correlation clustering[END_REF] give a greedy algorithm that upon node arrival creates a singleton cluster and then merge all pairs of clusters for which it increases the total number of agreements. For MINDISAGREE, this is O(n)-competitive algorithm and they show such ratio is optimal by exhibiting an instance9 on which any strategy ends up with nk disagreements whereas optimal cost is k. On the MAXAGREE side, this greedy strategy result in a 1 /2-competitive algorithm. If it is randomly mixed with a DENSE variation, it increases to 1 /2 + η, still far from the demonstrated 0.834 upper bound. This state of the art depicts a nuanced landscape for the CORRELATION CLUS-TERING problem. On one hand, minimizing the disagreements is difficult, although it is the most natural way given our bias. On the other hand, it can be solved with approximation guarantees, at least in theory. Indeed, the best methods require solving linear programs with m variables and up to O(n 3 ) constraints, which results in a high complexity of O(m 4.5 ) [MG07, Section 7.2]. Fortunately, we can obtain reasonable solutions in practice, thanks to efficient heuristics (using the output of KWIKCLUSTER as initialization for instance). Furthermore, instances obeying our bias can be solved more easily and almost exactly. We next present settings where instances are indeed close to our bias. Before that, we remark that while KWIKCLUSTER is often used on general graphs in practice (by assuming the missing edges are negative), doing so does not preserve its approximation guarantee. Consider the graph G showed in Figure 3.5. Assume there are n nodes connected positively to the center of the star, and k < n -1 negative edges between the peripheral nodes. We define G as the complete graph obtained from G by setting all missing edges to be negative. On G , the optimal solution is to have a cluster with the center of the star and one peripheral node, while others peripheral nodes are put in singleton clusters. This creates n -1 positive disagreements. When running KWIKCLUSTER on G , if the first pivot is a peripheral node, this is the solution we obtain. Otherwise, with probability 1 /n, the center of the star is chosen as pivot, creating a single big cluster that incurs ( 1 /2)(n -1)(n -2) negative disagreements. In expectation, KWIKCLUSTER thus achieves

1 n (n -1)(n -2) 2 + (n -1) 2 = 3n -4 2n (n -1) ∼ 3 2 n
disagreements, which is roughly a 3 /2-approximation as n grows larger. Now let us get back to the original G. Here the optimal solution is to have a single big cluster, which incurs k negative disagreements. With the same reasoning as before, the expected number of disagreements of KWIKCLUSTER is now

1 n k + (n -1) 2 . If for instance k =
√ n (as in Figure 3.5), then the previous expression is equivalent to n as n tends to infinity. KWIKCLUSTER is thus √ n worst in expectation than the optimal when run on this general graph.

Beyond worst case instances

We now look at two settings where our bias is likely to be present. In the first one, we assume the input signs are randomly perturbed of an initial assignment derived from a consistent clustering. In the second one, we do not assume anything about such a consistent clustering but expect instead that the optimal solution is "clear" enough that it does not change when weights are multiplicatively perturbed.

CORRELATION CLUSTERING under noise

As we have seen, assuming the Unique Games Conjecture, the k-MINIMUM MULTICUT problem, and therefore the COR-RELATION CLUSTERING problem on a general graph, cannot be approximated to within a constant factor in the worst case. But maybe we can do better in the average case, which motivates the study of semi-random model, where real graphs are seen as being obtained from the controlled perturbation of a perfectly clusterable graph. In the simplest case, each edge sign is independently flipped with probability p ∈ [0, 1 /2). This situation on complete graphs was considered in [BBC02, Section 6], showing a simple algorithm that with high probability makes Õ(n

2 ) mistakes, and in [BSY99, Theorem 2.6], with an algorithm recovering with high probability the planted partition of an unweighted graph in O(n 2 (log n) c ), where c depends on the size of the smallest cluster and on the noise probability.

Joachims et al. [START_REF] Joachims | Error bounds for correlation clustering[END_REF] analyze a more refined weighted model, where weights are generated by a probability distribution whose mean on true positive edges is larger than µ + > 0 and whose mean on true negative edges is smaller than µ -< 0. They give a finite-sample bound on the number of nodes misclustered w.r.t the planted partition as a function on the probability distribution parameters. Indeed, as pointed out by Ailon et al. [START_REF] Ailon | Correlation clustering revisited: the "true" cost of error minimization problems[END_REF], uniform and independent noise is not a good model of real situations, where the input of CORRELATION CLUSTERING, i.e. the similarity between nodes, is often the result of a preprocessing, which may present strong correlations. Therefore, instead of measuring the quality of a solution against the input (i.e. the similarity information), they argue it is more sensible to measure it against the (unknown) true optimal clustering that gave rise to the input, and show that KWIKCLUSTER allows this, thanks to a new analysis. Mathieu et al. [START_REF] Mathieu | Correlation clustering with noisy input[END_REF] also consider an adversarial model, where all edges are flipped with probability p but the adversary then decides whether to reveal the true sign or the flipped sign. On complete unweighted graphs, they find a solution of MINDISAGREE at most 1 + O(n -1 /6 ) times the optimal whenever p ≤ 1 /2n -1 /3 by rounding the usual SDP solution. If, in addition, there is no adversary, p ≤ 1 /3 and each planted cluster has at least Θ( √ n) nodes, then the planted partition can be recovered exactly. Makarychev et al. [START_REF] Makarychev | Correlation clustering with noisy partial information[END_REF] study the same adversarial model but on general weighted graphs, giving a PTAS for MINDISAGREE when p ≤ 1 /4. Under additional assumptions on the density of edges, they present another algorithm that finds the ground truth clustering with an arbitrarily small classification error.

CORRELATION CLUSTERING under stability assumption Whereas clustering objective functions are NP-HARD to optimize, we expect meaningful instances in which we are interested to have additional structure which allows for guaranteed polynomial time algorithms. We refer the reader to a critical overview [START_REF] Ben-David | Computational feasibility of clustering under clusterability assumptions[END_REF] of some such notions of structure proposed recently. Informally, a general idea is that the clustering should not change (or at least very little) if the data are slightly perturbed. For instance, a weighted graph is α-stable (with α > 1) for some partition objective if its optimal partition remains the same whenever every weight w i is multiplied by a factor c i between 1 and α. Another notion is the (c, )-approximation stability. Formally, a dataset X is (c, )-stable if, for an objective function Φ, any clustering whose cost is within a factor c of the optimal cost OP T Φ is -close to the optimal clustering (as measured by the fraction of points on which the two clusterings disagree). When the objective is MINDISAGREE on complete graphs, Balcan et al. [START_REF] Balcan | Finding low error clusterings[END_REF] show that for (1 + α, )-instances, the approximation algorithms we describe in Section 3.2.2.2 find a solution that is ( 49 /α + 1) close to the optimal clustering.

It is also possible to study the stability of a CORRELATION CLUSTERING instance over a general graph w.r.t edge weights via its canonical MINDISAGREE LP. Nowozin et al. [START_REF] Nowozin | Solution stability in linear programming relaxations: graph partitioning and unsupervised learning[END_REF] present a method to determine to which extent the weights of a CORRELATION CLUSTERING instance can be perturbed before the optimal solution changes.

Variants and extensions

So far we focused on CORRELATION CLUSTERING in its rawest form, that is solving the MINDISAGREE and MAXAGREE objectives in the case where the general binarylabeled graph is known in advance. We will now see first some special cases, namely when the weights obey the triangle inequality (to solve CONSENSUS CLUSTERING) or when the graph is bipartite and then move to variants. We will consider more general objectives, when the edge labels are categorical instead of binary, when nodes can belong simultaneously to several clusters or when we optimize local objectives per nodes instead of global ones. Finally we will also look at clustering in signed graphs in general, using spectral methods or heuristics from the community detection literature.

CONSENSUS CLUSTERING

In CONSENSUS CLUSTERING, the goal is to output a clustering which best summarizes (or agrees with) the several given input clusterings of the same set of objects. Motivations include robustness -by using an ensemble of clusterings from diverse methods-and privacy -if the clusterings were computed by different parties each considering only a subset of the objects attributes. We can build the complete graph of these objects, with weights set to the fraction of clusterings that place two objects in different clusters, thus representing a kind of distance in the space of clusterings. As first show by Filkov et al. [START_REF] Filkov | Integrating microarray data by consensus clustering[END_REF] , finding the optimal clustering is therefore an instance of CORRELATION CLUSTERING where the weights obey the triangular inequality. Gionis et al. [START_REF] Gionis | Clustering aggregation[END_REF] give a deterministic 3-approximation using the REGION-GROWING method. Later Bonizzoni et al. [START_REF]On the approximation of correlation clustering and consensus clustering[END_REF] show that the minimization version is APX-HARD, even when the input is made of only three clusterings and give a com-binatorial 4 /5-approximation for the maximization problem. Experimental evaluations are conducted by Bertolacci et al. [START_REF] Bertolacci | Are approximation algorithms for consensus clustering worthwhile?[END_REF] and Filkov et al. [START_REF] Filkov | Consensus clustering algorithms: comparison and refinement[END_REF]. The former describe a scalable approach that first samples a small portion of the data, runs a (potentially computationally expensive) approximation algorithm and finally augment the resulting partition by adding to it the unsampled nodes one by one. Experiments confirm that the running time is greatly improved compared with the linear program methods while the resulting objective value is essentially the same. Note, however, that LP methods can be applied in practice thanks to some tricks [START_REF] Downing | Improved consensus clustering via linear programming[END_REF]. If we have k input clusterings C 1 , . . . , C k and we parameterized the problem by t, which is the sum over the input clusterings of the number of pairs of objects that are clustered differently by a solution C and C i , then there is a polynomial algorithm running in O(4.24 t /k • t /k 3 + kn 2 ) [START_REF] Dörnfelder | On the parameterized complexity of consensus clustering[END_REF].

Bipartite CORRELATION CLUSTERING Bipartite graphs are an interesting special case for CORRELATION CLUSTERING, as they often appear in the context of recommendation systems, where users rate products positively or negatively, although in this setting we cannot expect to have complete bipartite graphs in practice. The first results was given by Amit [START_REF] Amit | The bicluster graph editing problem[END_REF], who obtains an 11-approximation for MINDISAGREE by adapting the REGIONGROWING method. The KWIKCLUSTER is adapted to the bipartite case by Ailon et al. [START_REF] Ailon | Improved approximation algorithms for bipartite correlation clustering[END_REF], who prove it results in a randomized 4-approximation (and provide a matching deterministic approximation by rounding a LP). By using their idea of rounding the results of the LP differently for positive, negative and in that case same-side absent edges, Chawla et al. [START_REF]Near optimal lp rounding algorithm for correlationclustering on complete and complete k-partite graphs[END_REF] bring down this approximation factor to 3, even for K ≥ 2-partite graphs. Through formulating the MAXAGREE[k] problem as a bilinear maximization problem and computing a low-rank approximation of the graph biadjacency matrix, Asteris et al. [START_REF] Asteris | Bipartite correlation clustering -maximizing agreements[END_REF] obtain a efficient PTAS, that is a (1δ) approximation running in time exponential in k and δ -1 but linear in n. By an appropriate choice of k, it is possible to use this PTAS to solve the general MAXAGREE problem. Beyond those results on CORRELATION CLUSTERING, we further discuss bipartite signed graphs in Section 5.2.1.

Categorical edge labelling

In the so called CHROMATIC-CORRELATION CLUSTER- ING setting, "positive" edges are now associated with one of L possible colors and the goal is to form clusters mostly made up of edges with the one same color. Namely, a disagreement is now a negative edge between clusters or a within-cluster edge whose color differs from the majority color of that cluster. This is motivated by edge-labeled graphs in social networks, biology and citation networks and we will further discuss such applications in Chapter 4. As a generalisation of CORRELATION CLUSTERING, it is NP-COMPLETE but Bonchi et al. [START_REF] Bonchi | Chromatic correlation clustering[END_REF] present a modification of the KWIKCLUS-TER algorithm that pick edges instead of nodes as pivots, and grow clusters by adding monochromatic triangles. This gives an approximation factor of six times the maximum degree of the graph. They also present a method when the number k of clusters is fixed beforehand, starting with an initial partition and then alternating between finding the majority color of clusters and finding better clusters. An improved heuristic algorithm is given in [START_REF] Gothania | A fast chromatic correlation clustering algorithm[END_REF]. Unfortunately, the maximum degree of a graph can be as large as n. However, Anava et al. [START_REF] Anava | Improved theoretical and practical guarantees for chromatic correlation clustering[END_REF] present constant factor approximations. Namely, they show that the problem can be reduced to classical CORRELATION CLUSTERING by setting all edges incident to a node u to negative if they are not of the majority color of u. They then apply the regular KWIKCLUSTER and show this gives an 11-approximation to the original problem. Furthermore, they also write a linear program and round it using the REGIONGROWING method to obtain an approximation factor of 4. Bonchi et al. [START_REF] Bonchi | Chromatic correlation clustering[END_REF] extend this line of work to the case were a single edge can carry a set of labels and adapt their randomized algorithm so that the approximation factor is multiplied by the size of the input label set.

Overlapping CORRELATION CLUSTERING While in CORRELATION CLUSTERING, each node is assigned to a single cluster, in other settings we may want to relax this constraint. Given a complete weighted graph, Bonchi et al. [START_REF] Bonchi | Overlapping correlation clustering[END_REF] want to output a clustering C that minimizes the following cost:

(u,v)∈E |H(C(u), C(v)) -w uv |
where H is a similarity function between two sets of labels, chosen in their paper to be the Jaccard similarity or a 0/1 indicator of nonempty intersection. These problems are showed to be NP-COMPLETE, and approximated by a local search algorithm, iteratively optimizing the assignment of one node while all others are fixed. As one of the demonstration of their theoretical work, Johansson et al. [START_REF] Johansson | Weighted theta functions and embeddings with applications to max-cut, clustering and summarization[END_REF] show a faster solution based on a weighted extension of the Lovász's theta function, the corresponding geometric embedding of graphs and a solver derived from one-class SVM, while Andrade et al. [And+14] propose a genetic algorithm to solve this problem. Finally, Rebagliati et al.

[RRP13] also deals with overlapping clustering by relaxing the problem to a stochastic setting and using "the Baum-Eagon inequality, which provides an effective iterative scheme for maximizing polynomial functions in probability domains". In classical CORRELATION CLUSTERING, all nodes have an identical role, in the sense that they contribute equally to the final objective in terms of (dis)agreements. Here, we instead look at approaches where we either add a local penalty to each in order to better control their behavior, or where we altogether modify the objective to focus on (dis)agreements at specific nodes.

Local CORRELATION CLUSTERING

Puleo et al. [START_REF] Puleo | Correlation clustering with constrained cluster sizes and extended weights bounds[END_REF] adapt the linear program of [START_REF] Charikar | Clustering with qualitative information[END_REF] and its REGIONGROWING method to the case where all clusters have to contain less than K nodes, by assigning to each node u a penalty µ u . If u is placed in a cluster C i , the original MINDISAGREE objective is penalized by an extra µ u (|C i | -(K + 1)). By varying µ v between 0 and 1 and because the positive weights are assumed to smaller than 1, this cluster size constraint can be made hard or soft. They also handle more general weights, since they allow w - uv to be as large as τ for τ ∈ [1, ∞) while still guaranteeing a 5 -1 /τ-approximation on complete graphs, and adapt KWIKCLUSTER to unweighted graphs with the hard cluster size constraint, obtaining a randomized 7-approximation. These soft constraints are for instance used in a biological application where nodes are genes and where singleton and giant clusters are uninformative [START_REF] Hou | A new correlation clustering method for cancer mutation analysis[END_REF] Puleo et al. [START_REF]Correlation clustering and biclustering with locally bounded errors[END_REF] also modify the MINDISAGREE objective to make it more general. Based on the classic CORRELATION CLUSTERING linear program, they define a "fractional clustering of G as a vector x indexed by V such that x uv ∈ [0, 1] for all uv ∈ V 2 and such that x vz ≤ x vw + x wz for all distinct v, w, z ∈ V ". They also define "the error vector err(x) of x, as a real vector indexed by V whose coordinates are"

err(x) u = v∈N + (u) x uv + v∈N -(u) (1 -x uv )
Given a function f : R n ≥0 → R verifying two elementary conditions, the problem is then to find a fractional clustering x minimizing f (err(x)). The classical CORRELA-TION CLUSTERING corresponds to setting f (x) = 1 2 1 (x) whereas the authors here are interested in Minimax CORRELATION CLUSTERING that arises by setting f (x) = ∞ (x). Minimizing the maximum number of disagreements incurred by a single node is motivated by the example of recommendation systems: if errors correspond to unsatisfying recommendations, we do not want a single user to suffer many of them. Minimax CORRELATION CLUSTERING is NP-COMPLETE on both complete graphs and complete bipartite graphs but by modifying the REGIONGROWING method, the authors respectively a 48 and 10 approximation, the latter for the one-sided error (that only counts disagreements for the nodes in one of the two clusters). The idea is to chose pivots not randomly but by maximizing a given criteria and to grow balls with a radius α computed numerically to optimize the approximation factor. Interestingly, and in contrast with the classic CORRELATION CLUSTERING situation, minimax MAXAGREE is not easier than minimax MINDISAGREE and seems not to have a constant factor approximation, even on complete graphs. Furthermore, these algorithms are deterministic, as opposed to many CORRELATION CLUSTERING approximations, since bounds on expected disagreements of an edge does not translate easily on their maximum. Charikar et al. [START_REF] Charikar | Local guarantees in graph cuts and clustering[END_REF] improve these two factors to 7, using a simpler version of the algorithm of Puleo et al. [START_REF]Correlation clustering and biclustering with locally bounded errors[END_REF]. Namely, find the ball of radius 1 /7 with the largest number node and create a cluster from its center with a radius of 3 /7. They also show that on general weighted graphs, the LP has a large integrality gap of n /2. Yet they combine it with a combinatorial approach to reach a O( √ n) approximation. Finally, they consider the complementary problem of maximizing the minimum number of agreements at a single node, and provide a 1 2+ approximation.

Spectral Clustering A classic method for clustering graphs is to leverage their spectral properties. Namely, if A is the adjacency matrix of G and D its degree diagonal matrix (that is D u,u = deg(u)), the Laplacian of G, defined by L G = D -A, is a symmetric positive semidefinite matrix. As such, it has n real non-negative eigenvalues, and its spectrum provides additional information on the connectivity of G. For instance, 0 is always the smallest eigenvalue and its multiplicity is equal to number of connected components of G, while -if G is connected-the second eigenvalue is the algebraic connectivity of G, whose magnitude is an indication of how well connected is the graph. This matrix is typically used for clustering by computing its first k eigenvectors, which embed the n nodes of G in R k , where there are then clustered with the k-means algorithm. This can be seen as a relaxation of the discrete RATIOCUT objective, which asks for the partition {C 1 , . . . ,

C k } minimizing 1 2 k i=1 cut(Ci, Ci) |Ci|
, where Ci is the complement of C i in V and cut(B, C) = u∈B,v∈C w uv is the total weight of the edges between B and C [START_REF] Luxburg | A tutorial on spectral clustering[END_REF]. By considering the symmetric normalized Laplacian L sym = D 1 /2 LD 1 /2 , it is also possible to approximate the normalized cut objective (NCUT), where |C i | is replaced by vol(C i ) = u∈Ci deg(u). We will now see how these kinds of approaches can be extended to signed graphs, noting first that they require to fix the number of clusters beforehand and are looking for clusters balanced in size, which makes the problem related but not equivalent to CORRELATION CLUSTERING.

The first line of research consider only MINDISAGREE[2]. For instance, Coleman et al. [START_REF]Spectral clustering with inconsistent advice[END_REF] show that both normalized cut and MINDISAGREE[2] objectives can be written as a SDP (or equivalently as eigenvalue problems) and thus combined, the intuition being that we look for NCUT solutions whose number of disagreements is not too much more than the approximate optimal.

Letting NCUT(C,

C) = cut(C, C) bal(C) with bal(C) = 2 vol(C)vol( C) vol(V )
, Rangapuram et al. [START_REF] Rangapuram | Constrained 1-spectral clustering[END_REF] define a new objective:

Fγ (C) = cut(C, C) + γ M (C) + N (C) bal(C)
where γ ∈ R + is a parameter, while M (C) and N (C) are respectively the number of positive and negative disagreements of the (C, C) clustering. They show how to optimize a tight continuous relaxation of Fγ as the non-negative ratio of a difference of convex function and a convex function.

On the other hand, one can also adapt these two cut objectives to directly include negative edges. Kunegis et al. [START_REF] Kunegis | Spectral analysis of signed graphs for clustering, prediction and visualization[END_REF] define the signed Laplacian as L = D -A, where D is the signed degree matrix such that Duu = v∈N (u) |A uv |, as well as a signed variant of the symmetric normalized Laplacian Lsym = D1 /2 L D1 /2 . They show that the signed Laplacian is positive semidefinite, and even positive-definite as soon as the graph is unbalanced (that is contains a cycle with an odd number of negative edges). From positive and negative cuts defined as cut (C,C) which can then be used to defined signed RATIOCUT and NCUT. Arguing that those definitions force negatively linked nodes to be symmetric around the origin, do not take into account the balance of negative edges in each cluster and are difficult to extend to more than two clusters, Zheng et al. [START_REF] Zheng | Spectral embedding of signed networks[END_REF] instead propose two new normalized cuts:

+ (B, C) = u∈B,v∈C w + uv and cut -(B, C) = u∈B,v∈C w - uv , a natural signed cut is scut(B, C) = 2cut + (B, C) + cut -(B, B) + cut -
SN Scut(C 1 , . . . , C k ) = k i=1 cut + (C i , Ci ) -cut -(C i , Ci ) vol(C i ) BN Scut(C 1 , . . . , C k ) = k i=1 cut + (C i , Ci ) -cut -(C i , Ci ) + vol -(C i ) vol(C i )
Noting that if x i ∈ R n is the vector indicator of cluster C i (that is the u th entry of x i is 1 is u belongs to C i and 0 otherwise),

x T i Lx i = 2cut -(C i , C i )+cut -(C i , Ci )+cut + (C i , Ci ),
Sedoc et al. [START_REF] Sedoc | Semantic word clusters using signed spectral clustering[END_REF] introduce the following cut objective:

sN cut(C 1 , . . . , C k ) = k i=1 2cut -(C i , C i ) + cut -(C i , Ci ) + cut + (C i , Ci ) vol(C i )
Additional cut formulations for k-clusters are also presented in [START_REF] Chiang | Scalable clustering of signed networks using balance normalized cut[END_REF], although Knyazev [Kny17] argue that using the non signed Laplacian and considering negative eigenvalues might be just as effective, citing for instance numerical instability of signed Laplacian. Finally, Mercado et al. [START_REF] Mercado | Clustering signed networks with the geometric mean of laplacians[END_REF] show that the Laplacians defined so far can be seen as arithmetic means of the Laplacian L + of the positive subgraph G + = (V, E + ) and the signless Laplacian Q -of the negative subgraph G -= (V, E -), where Q -= D -+ A -. They suggest instead to use a geometric mean, defined for two positive matrices A and

B as A#B = A 1 /2 A -1 /2 BA -1 /2 1 /2 A 1 /2
. This suggestion is based on the fact that if u is a common eigenvector of both A and B with eigenvalue λ and µ respectively, then u is an eigenvector of A + B with eigenvalue λ + µ and an eigenvector of A#B with eigenvalue √ λµ. Therefore, the k smallest eigenvalues of the geometric mean Laplacian will be influenced by both smallest eigenvalues of L + (corresponding to assortative clusters in G + ) and of Q -(corresponding to disassortative clusters in G -), while this is not the case for the arithmetic mean of Laplacians.

Community detection

The clustering problem is often named community detection in the context of social networks, and several methods developed by practitioners have been extended to signed graphs. While they do not necessarily considered the COR-RELATION CLUSTERING objective, and especially not its optimum, we still give a brief overview of them, as they tend to have been more tested experimentally. For instance, to find the cluster of node u, Yang et al. [START_REF] Yang | Community mining from signed social networks[END_REF] use a random walk approach on the positive subgraph G + to compute the probability of each node to reach u in T steps, sort the nodes accordingly and then find a threshold based on the number of disagreements. The one node move local heuristic that we described in the Physics-inspired paragraph on page 62 can also be formulated as genetic algorithms that simultaneously try to minimize the number of disagreements and maximize a signed variant of the modularity [LLL13; AP13]. Anchuri et al. [START_REF] Anchuri | Communities and balance in signed networks: a spectral approach[END_REF] also consider these two objectives by seeing them as eigenvalue problems and devise an iterative splitting procedure. The overlapping community detection variant is considered by Chen et al. [START_REF] Chen | Overlapping community detection in networks with positive and negative links[END_REF], who used a signed probabilistic mixture model. Namely, an edge selects a pair of cluster r, s with probability ω rs (where r = s if the edge is positive and r = s otherwise) and chooses its endpoints u and v with probability θ ru and θ sv . θ ru is therefore the soft membership of node u to cluster r, and those parameters are estimated using the expectation-maximization algorithm. The same model is extended to directed graphs by Jiang [START_REF] Jiang | Stochastic block model and exploratory analysis in signed networks[END_REF], who strangely enough names it stochastic blockmodel, although the focus is still on edge and not nodes. In a similar spirit to MAXAGREE[k], Chu et al. [START_REF] Chu | Finding gangs in war from signed networks[END_REF] focus on finding k subgraphs dense in positive edges and densely connected by negative edges to each other. They dub such subgraph Oppositive Cohesive Groups, or more vividly Gangs in War, and after formulating the problem as a constrained quadratic optimization, they propose a faster iterative local search heuristic. When k = 2, these subgraphs are called antagonistic communities and a specific data mining approach was proposed by Gao et al. [START_REF] Gao | On detecting maximal quasi antagonistic communities in signed graphs[END_REF].

Let us review this material about CORRELATION CLUSTERING in the light of our thesis objective: efficiently and accurately characterize edges in complex networks, or rather signed graphs in this chapter. The results on the hardness of CORRELATION CLUSTERING, and the fact that the best approximations rely on a linear program with a large number of constraints seems to run counter to such efficiency and accuracy requirements. To avoid this pitfall, we presented many existing heuristics. More importantly, recall that our learning bias is that nodes are assigned to K groups, and that signs are consistent with those groups. Informally, the closer an input signed graph is to this ideal situation, the more regular it is with respect to this bias, and thus the easier the EDGE SIGN PREDICTION problem is. Accordingly, the works described in Section 3.2.3 point out that such non worst case instances are indeed where we expect clustering algorithms to be able to identify those K groups with little to no error. In the next section, we present an algorithm relying on a similar intuition, in an active setting and where K = 2.

Low stretch trees and spanners

We now show how to apply the learning bias of Section 3.1 on undirected graphs. Furthermore, we assume that the strong balance holds, meaning that there are only two groups according to Theorem 4. In other words, the labelling of E is consistent with a two-clustering of V . Namely, V can be partitioned in two clusters such that edges within each cluster are positive and edges across clusters are negative. In that case, the following multiplicative rule holds: for any nodes u, v in V , and any path p between u and v in G, the sign y u,v is equal to the product of the signs along p. Hereafter, we call this product the parity of p, and denote it by π(p). While it is a simple and convenient hypothesis, this is too strong of a requirement to be satisfied in practice. Therefore, we relax it by assuming that, starting from a consistent labeling Y , we can only observe a randomly perturbed version Y of Y . Specifically, given a constant q ∈ [0, 1 /2), every sign of Y is flipped with a probability smaller than q. We denote by E flip ⊂ E the set of edges whose sign has been flipped.

In this section, we are interested in active learning algorithms that first query a subset E train of the edges, observe the signs in E train and use them to predict the remaining signs. More precisely, we focus on an algorithm that queries a spanning tree T of G and predicts the sign of an edge (u, v) ∈ E test = E \ E T as the parity of path T (u, v). Intuitively, since each sign has been potentially flipped, the longer the path in T , the more likely its parity will be not be equal to the true sign y u,v . Therefore we would like each such path to be as short as possible. Formally, the number of mistakes of such an algorithm is upper bounded by [Ces+12b, Equation (3)]

|E flip | + (u,v)∈Etest e∈E I e ∈ path T (u, v) I {e ∈ E flip }
which in expectation is equal to:

q   |E| + (u,v)∈Etest | path T (u, v)|   (3.7)
In the following, we describe a way to build spanning trees tailored for this situation. More precisely, we implement and analyze a suggestion made to us by Vitale [START_REF] Vitale | [END_REF].

GALAXY TREE: a spanning tree designed for sign prediction

To achieve our objective of building a spanning tree that minimizes the distances between all connected pair of nodes in the original graph, we rely on a particular subgraph structure, namely the star. We therefore introduce two algorithmic primitives. EXTRACT-STARS, which partition a graph G into a set of disjoint stars. And COLLAPSE-STARS, which selects edges from E to assemble these stars into a new, smaller graph. Given a graph topology G 0 = (V 0 , E 0 ) and assuming for simplicity that G 0 consists of a single connected component,10 the GALAXY TREE algorithm repeatedly applies these two primitives to produce a sequence of graphs {G t } K t=0 of decreasing size, until G K is made of a single node. All the edges selected while reaching this stage then form the spanning tree we were looking for. This can be seen as a simple instantiation of a general procedure described in [Alo+95, Section 5.2].

In the following, we provide a more precise description of our two primitives and analyze their complexity. Then we state formally the complete GALAXY TREE algorithm, prove its termination and correctness, and show a detailed example of its execution. Finally, we provide a conjecture on the number of iterations needed to finish.

c i p 1 p 2 p 3 p 4 h 1 h 2 h 3 S t i Figure 3
.6 -A sample star created during the t th collapse level. The black node is the center c i of the star S t i , which is also made of the four light gray peripheral nodes as well as the solid edges. The 2-hops neighbors of c i are the white nodes h 1 to h 3 , whose degree will decrease once S t i is removed from G t .

EXTRACT-STARS EXTRACT-STARS takes as input a graph

G t = (V t , E t ). While the nodeset V t is not exhausted, it repeatedly samples a node c i , creates a star S t i with c i at its center and the neighbors of c i on the periphery, removes all the nodes of S t i from V t and all the edges incident to S t i from E t , and finally decrements accordingly the degree of the 2-hop neighbors of c i (see Figure 3.6 for a visual representation of this notation). Upon completion, it returns a list of stars, the set of all the edges within a star, and a map (or associative array) that associates each node of V t to the index of the unique star it belongs to. According to the definition of Mehlhorn et al. [START_REF] Mehlhorn | Hash tables and associative arrays[END_REF], an associative array is an abstract data type composed of a collection of (key, value) pairs, such that each possible key appears at most once in the collection. It efficiently supports the addition, removal and modification of a pair, as well as the lookup of a value associated with a particular key.

As showed in the following pseudo code11 , we sample centers by choosing the node with the current highest degree, with ties broken arbitrarily.12 This is achieved efficiently by maintaining a max-priority queue Q, initially containing all the nodes of V t . The priority of a node is its current degree and we equip Q with two standard operations described by Cormen et al. [Cor+09, section 6.5]: EXTRACT-MAX(Q) removes and returns the node of Q with the largest degree and DECREASE-KEY(Q, u, ∆) decrements the degree of the node u by an amount ∆. We also assume that G is the adjacency list of the graph, so that G[u] is the set of neighbors of u, i.e. G[u] ≡ N (u). Finally membership is the map storing for each node the index of the star it belongs to. This map is updated by the STAR function, which creates a star given a center c, a list periphery of peripheral nodes, and a star index i. After creating the i th star, for every node u belonging to that star, the STAR function sets membership[u] = i.

1: function EXTRACT-STARS(G t = (V t , E t )) 2:
Let Q be the max-priority queue described above 3:

Let remaining be a set of nodes, initially containing all the nodes in V t

4:

Let membership be an empty map 5:

stars ← [] 6: inner_edges ← ∅ 7: while Q is not empty do 8: c ← EXTRACT-MAX(Q) 9: if c not

COLLAPSE-STARS

The second primitive, COLLAPSE-STARS takes as input the edges E t of the current graph, along with the membership result of EXTRACT-STARS and an eccentricity array. It builds a new graph G t+1 where each star s u from EXTRACT-STARS becomes a node and where there is a link between two nodes s u and s v if the nodes in V t making up s u and s v are connected in E t . The eccentricity array is needed because when connecting two stars, we would prefer to join their center rather than two of their peripheral points. Therefore, we maintain an eccentricity count for all of the nodes of the original G 0 , which is incremented by 1 each time a node is chosen to be on the periphery of a star. In other words, the eccentricity of an original node quantifies to which extent it has been pushed to the border of the galaxy. COLLAPSE-STARS not only returns the new graph G t+1 but also a map cross_edges. This map is used to keep track of which edge in E t connects any pair of node in V t+1 (as illustrated in Figure 3.7). It associates to any edge in E t+1 a set of edges from E t . This set can have an arbitrary size during the execution of COLLAPSE-STARS, yet it is guaranteed to contain a single edge upon termination. Although this does not appear in the following pseudo code, in practice we ensure that (s u , s v ) and (s v , s u ) refer to the same edge in cross_edges.

1: function COLLAPSE-STARS(E t , membership, eccentricity) 2:
Let G t+1 be an empty graph 3:

Let cross_edges be the map described above 4:

for every edge (u, v) in E t do 5:

s u ← membership[u] 6: s v ← membership[v] 7:
if u and v are not in the same star (i.e. s u = s v ) then

8: cross_edges[(s u , s v )] ← cross_edges[(s u , s v )] {(u, v)} 9:
for every pair of will-be connected stars (s u , s v ) in cross_edges do 10: Figure 3.7 -In this graph, there are four possible edges between the two stars s 1 and s 2 , and all are part of cross_edges[(s 1 , s 2 )], here represented in light red. Those edges are labeled with the sum of eccentricity of their underlying endpoints. White node have an eccentricity of 0, and gray node an eccentricity of 1. The edge with the minimal eccentricity is linking a peripheral node of s 1 directly to the center of s 2 .

possible_underlying_edges ← cross_edges[(s u , s v )] 11: (u 0 , v 0 ) ← arg min (u,v)∈possible_underlying_edges eccentricity[u] + eccentricity[v]
12:

cross_edges[(s u , s v )] ← {(u 0 , v 0 )} 13: E t+1 ← E t+1 {(s u , s v )} 14: return G t+1 , cross_edges Proposition 2. For any graph G = (V, E), COLLAPSE-STARS(E, membership, eccentricity) terminates in O(|E|) time.
Proof. The analysis of COLLAPSE-STARS is rather straightforward because the function only executes two loops over E. During the for loop of line 4, it only performs constant time operations on maps. Likewise, in the for loop of line 9, the most expensive operation is finding the minimum in line 11. Computing the eccentricity of an edge is done in constant time and only once for each edge of E. As a consequence, the total time of COLLAPSE-STARS is indeed O(|E|).

Putting the pieces together EXTRACT-STARS and COLLAPSE-STARS are truly the core the of GALAXY TREE algorithm but to obtain our final spanning tree, we need additional work, namely updating the eccentricity of the nodes of V 0 and keeping track of each edge within and between stars along every collapse steps. Despite our earlier promise, this entails showing some bookkeeping code, because it has an influence on the runtime of GALAXY TREE. Namely in the listing of Algorithm 1 on the next page, we use the three following maps: name keys set at t values set at t comment star_membership nodes in V t nodes in V t+1 this is the map returned by EXTRACT-STARS(G t ) f ull_membership nodes in V 0 nodes in V t+1 at t = 0, this is equals to star_membership and then it gets updated at every iteration to maintain its original keys set. original_node nodes in V t sets of V 0 nodes This can be seen as the reverse of f ull_membership and it is the one needed to update the eccentricity of the original nodes.

As described in Algorithm 1, at every collapse level, we first extract stars from the current graph (line 5), then update the eccentricity and node mappings (lines 7-8) and finally collapse the graph (line 9). We perform these operations until there Algorithm 1 GALAXY TREE(G 0 = (V 0 , E 0 )) are no edge connecting stars anymore. At this point, we revisit every outer edge to build the spanning tree (line 13).

Proposition 3. For any connected graph G 0 , GALAXY TREE(G 0 ) terminates after K ≤ |V 0 | iterations. Furthermore, it runs in O(K|E 0 |) time and returns a spanning tree of G 0 .

We will need the following lemma.

Lemma 4. If G 0 is connected, then any subsequent graph G t , t ≤ K is also connected.

Proof. Suppose not and assume, to the contrary, that there exists at least one (or more) disconnected graph in {G t } K t=1 and let t 0 be the smallest index such that G t 0 is disconnected. Then there exist two nodes s u and s v in V t 0 with no path between them in E t 0 . Letting U, V ⊂ V t 0 -1 be respectively the nodes forming stars u and v, this implies there is no path between nodes in U and nodes in V. However, G t 0 -1 is connected by hypothesis, which leads to a contradiction.

Proof of Proposition 3. Let us first show that GALAXY TREE terminates in less than |V 0 | iterations. This follows from the fact that every time we collapse the graph G t , we strictly reduce the number of nodes. Indeed, according to Lemma 4, G t is connected so at least two nodes of V t are joined by an edge and will form a star, i.e. a single node. We can thus claim that |V t+1 | < |V t |. Note also that for all t, |V t | > 0 and that when |V t | = 1, EXTRACT-STARS creates a singleton and COLLAPSE-STARS does not return any outer edges. At this point, GALAXY TREE finishes, proving that the number of iterations K satisfies K ≤ |V 0 |.

Then we analyze the time complexity. We already know that during the t th iteration, EXTRACT-STARS and COLLAPSE-STARS take time O(|E t |). We only provide the actual python code instead of pseudo code for the remaining functions since they do not present any relevant algorithmic aspect. However, we can say that UPDATE-ECCENTRICITY and 13 Finally, we prove that GALAXY TREE indeed returns a spanning tree of G 0 . Specifically, we show that by starting from G K and going in reverse order, we build a spanning tree T t of every G t , including eventually G 0 . This process is illustrated in Figure 3.8 and works as follow: when moving from G t+1 to G t , we expand each node of G t+1 into a star of nodes of V t , and update all the outer edges we met so far to have their endpoints in V t . These outer edges are exactly T t+1 and by translating them into E t and adding the inner star edges of G t , we build T t . G K consists of a single node with no edges so we have trivially that T K = G K . Then, assume we have a spanning tree T t+1 of G t+1 , so that all nodes in V t+1 are connected without cycle. Expanding each node of V t+1 into a star of V t nodes ensures that the spanning property is preserved since each node of V t is covered by construction. Those stars are also without cycle by construction. The second step, translating the existing edges of T t+1 , only connects nodes belonging to different stars and because those edges were cycle free in G t+1 , this remains the case in G t . Therefore we have build a spanning tree T t of G t . By repeating this procedure, the GALAXY TREE algorithm eventually builds a spanning tree of G 0 .

The bound of Proposition 3 amounts to O(|V 0 ||E 0 |), which we believe is overly pessimistic. Indeed, we expect the number of nodes to decrease by more than one at each step, potentially leading to only O(log |V 0 |) iterations.

Example of GALAXY TREE

We illustrate the operation of the GALAXY TREE algorithm on a small example. Let us start with the initial graph G 0 depicted in Figure 3.9 on page 77 and initialize the eccentricity of all nodes to 0. When running EXTRACT-STARS, we see that the maximum degree is 4, achieved at nodes {1, 6, 11, 16, 21, 26, 31, 36, 41}. For the sake of simplicity, assume nodes are picked according to their index. First, node 1 forms the star S 1 1 with peripheral nodes 2, 3, 4 and 5. This increments the eccentricity of those peripheral nodes by 1. Then node 6 forms its star S 1 2 with 7, 8, 9 and 10. The process continues until node 41 is chosen to be the center of star S 1 9 , at which point the max-priority queue has been exhausted and EXTRACT-STARS finishes.

We then call COLLAPSE-STARS. This will connect all possible pairs of stars. For instance, the edge between nodes 19 and 29 leads to the edge between S 1 4 and S 1 6 . This is actually the only possible edge between S 1 4 and S 1 6 . Consider on the other hand the case of edges (2, 6) and (2, 9). They both connect S 1 1 and S 1 2 . Yet at this point of the algorithm, the eccentricity of node 2 is 1, the eccentricity of node 6 is 0 and the eccentricity of node 9 is 1. The edge (2, 6) has therefore the smallest total eccentricity and is chosen to connect S 1 1 and S 1 2 . The full result of the COLLAPSE-STARS procedure is G 1 , which can be seen on Figure 3.10a.

We now run EXTRACT-STARS on G 1 . Because all nodes have degree 2, they could all be chosen to be the center of a star. We again assume they are picked according to their index and therefore we choose S 1 1 to be the center of the star S 2 1 with peripheral nodes S 1 2 and S 1 3 . The original nodes belonging to those peripheral stars (nodes 4 to 15) have their eccentricity incremented by 1. The next node with highest degree in G 1 is now S 1 4 , which forms a star with S 1 5 and S 1 6 . This choice means that nodes 21 through 30 have their eccentricity incremented by 1. Finally, S 1 7 forms the last star with S 1 8 and S 1 9 . Then COLLAPSE-STARS connects the resulting three stars, and this time there is only a single choice between each pair of stars, leading to the graph G 2 showed in Figure 3.10b

The action of EXTRACT-STARS on G 2 is quite simple, because there is only one star that can be created, so let say we choose S 2 1 as its center, with S 2 2 and S 2 3 as peripheral nodes. This increases the eccentricity of their underlying G 0 nodes by 1 (namely nodes 16 to 45). Because there is only one star S 3 1 left, COLLAPSE-STARS returns G 3 showed in Figure 3.10c and an empty list of outer edges, meaning that the inner loop of GALAXY TREE is finished and we can go through every edges we chose between stars at every level to recover the final spanning tree, a process illustrated in Figure 3.8 on the current page. For completeness, we can also look at the edges which are not part of the spanning tree, that is the test edges, and compute how long are the paths joining their endpoints. Indeed, this is the second quantity appearing in the mistakes bound of equation (3.7). As shown in Table 3.7, this average length is 7. Edges forming the final spanning tree are solid while the others are dashed. Their colors indicate at which iteration they were chosen to be inside a star. The four shades of gray, from white to dark gray denote increasing node eccentricity (as computed at the end of the algorithm). The i th star created during the j th iteration of the algorithm is denoted S j i . Refer to the main text for the complete description of the execution. Discussion on the number of iterations needed A crucial quantity of the GALAXY TREE algorithm, both in terms of complexity and resulting stretch, is the number of collapses K needed before termination. While this is still elusive to express in the general case, let us first look at some simple cases. For instance, a very sparse example of graph is the line graph. While it is already a tree, let us look how the GALAXY TREE algorithm operates on it. Say we have n nodes and m edges in that line. Here, a star is made at most of three consecutive nodes. In the worst case, centers will be chosen such that we have a succession of three and two nodes stars, as in Figure 3.11. This will results in 2n /5 stars organized in a line, which is less than half of n. Thus we can conclude that in this case, GALAXY TREE will finish after O(log n) iterations. Note that a barbell graph (two cliques connected by a line) would requires a number of iterations logarithmic in the length of that central line, despite having many more nodes and edges. In fact, it is helpful to look at this question from the other direction. Namely, we can wonder what is the minimum number n(K) of nodes necessary for a graph to requires at least K collapses before being reduced to a single node. Checking by hand, we have that n(1) = 2, n(2) = 4 and n(3) = 8. Furthermore, this number is achieved by a line or circle consisting of exactly 2 K nodes. By induction, we can even show that on any line or circle graphs with 2 n nodes, the GALAXY TREE algorithm will terminate after n collapses. Assuming we could prove that n(K) = 2 K , given a graph G with n = 2 log 2 n nodes, by definition of n(K) = 2 K , it would require less than log 2 n = O(log n) collapses before being reduced to a single node. However, we were not able to rule out the possibility that there is a graph more parsimonious than a line still able to survive the same number of collapses. Therefore, we cannot conclude that n(K) = 2 K and we leave this statement as a conjecture.

Variants of EXTRACT-STARS

The execution of EXTRACT-STARS is mainly deterministic, except for the fact the ties between nodes with the same highest degree are broken arbitrarily. This allows for an efficient implementation, and simplifies the analysis of the resulting sequence of stars and therefore the induced spanning tree. However, it can be detrimental in an adversarial context, where we could end up with a tree forcing a lot of mistakes. We add an element of randomization to EXTRACT-STARS by letting it use two optional arguments, a threshold function τ or a degree function d. Such functions modify the center sampling process in the following way:

• if n t,i is the number of node remaining in V t before choosing the i th center, choose a node uniformly at random among those with a degree larger than τ (n t,i ). The idea is to choose among a small set of high degree nodes, for instance by letting τ (n) = √ n. Note however we cannot guarantee there will always be nodes with degree above the threshold, in which case we default on the highest degree node

• if deg i (u)
is the degree of node u before choosing the i th center, choose node proportionally to d(deg i (u)). Again, the degree function is designed so that it favors the selection of high degree nodes. For instance, one could use

d(deg i (u)) = deg i (u) 2
. These two variants are more time consuming because they require additional bookkeeping. Therefore, we don't provide a full complexity analysis and only briefly sketch their implementations here. 14 For the threshold function, we maintain two queues, high and low, containing nodes whose degree is respectively above and below the current threshold. We select a node uniformly at random in high, remove the corresponding star from G t , recompute the new threshold and if necessary, move nodes which fell under the threshold from high to low and those who climb above the threshold from low to high. For the degree function, we can draw any node as center proportionally to its weight (where the weight of node u is defined as

d f (deg(u))
). Yet we cannot use the standard method of computing the cumulative sum of weights since some of them change at each iteration. Therefore, we construct a binary tree whose leaves are the nodes of V t and where each tree nodes maintain the sum of weights in its left and right subtrees. To sample, we draw a random number r between 0 and the total weight of the tree and go down from the root to the leaf spanning the weight interval containing r. When degrees are updated (or graph node removed), we update the weights along a path from the corresponding leaves to the root of the tree.

A variant of the GALAXY TREE algorithm as a whole we did not explore so much in practice is its ability to produce spanners by stopping early. Basically if we stop at iteration t, we output the graph G t (which in general is not a tree) with its edges unfolded to lie in E 0 . This corresponds to a trade off between having more edges than |V 0 | -1 but potentially making shorter connection. This would also be interesting for the EDGE SIGN PREDICTION problem. Assuming the treewidth of G t is decreasing with t, we could try to show there is "reasonable" number of paths between u and v, compute their parity and take a majority vote on the results.

Related work

Looking for a subgraph H of G that best preserves the distance in G while being sparse is an old problem, driven originally by network design in fields such as transportation [START_REF] Quandt | Models of transportation and optimal network construction[END_REF] and electrical circuits [START_REF] Knight | Logical design of electrical networks[END_REF]. The way we define "preserving the distance", and the exact form of H give rise to several problems, which we summarize later in Table 3.9. We first give some definitions, then cover the most relevant problems in details, and finally give some pointers for the others problems.

Let the distance between u and v in G be

d G (u, v) = e∈path G (u,v) (e) ,
where (e) is the length of the edge e and path G (u, v) is the shortest path between u and v in G. In the following, we consider only the uniform case, in which the length of an edge is equal to its weight. The stretch of an edge (u, v) in H is defined as

stretch(u, v) = d H (u, v) d G (u, v) .
We may then want to minimize the stretch of: 1) some pairs of nodes. That is, given L and R in V , minimize u∈L,v∈R stretch(u, v) 2) all pairs of nodes corresponding to edges of G, i.e. minimize (u,v)∈E stretch(u, v) 3) all pairs of nodes, i.e. minimize (u,v)∈V 2 stretch(u, v) Note that for unweighted graphs, the second problem reduces to minimizing

stretch(H) = (u,v)∈E | path H (u, v)|. (3.8)
If furthermore H is tree, this is equivalent to minimize the second term of equation (3.7). Therefore we focus mainly of that definition of stretch, and consider the other two only briefly.

The second point affecting the problem is the structure of H. The only requirements are that it must be spanning all the nodes involved in the computation of the chosen stretch, and that

∀(u, v) ∈ E, d H (u, v) ≥ d G (u, v).
Beside that, H can be a tree of G, a general subgraph of G or even a subset of V 2 (i.e. containing edges not in E). We focus mainly on the first two cases, since they are covered by the GALAXY TREE algorithm.

Trees One early mention of seeking a low-stretch spanning tree is given by Hu [START_REF] Hu | Optimum communication spanning trees[END_REF], albeit in more general form: Problem 1 (Optimal Communication Spanning Tree). Given a set of nodes V = {v 1 , . . . , v n }, a set of distances d ij and a set of requirements r ij between v i and v j , find a spanning tree connecting these n nodes such that the total cost of communication of the spanning tree is a minimum among all spanning trees. The cost of communication for a pair of nodes is r i,j multiplied by the sum of the distances of arcs which form the unique path connecting v i and v j in the spanning tree. The cost of a spanning tree is the sum of costs over all pairs of nodes.

For a weighted graph G = (V, E, w), by letting d ij = w ij and r i,j = I {(i, j) ∈ E}, finding an Optimal Communication Spanning Tree thus amounts to finding a lowstretch spanning tree. Table 3.8 present a list of works where the stretch was improved.

We start with the seminal paper of Alon et al. [START_REF] Alon | A graph-theoretic game and its application to the k-server problem[END_REF]. It touches on many topics, and frame the problem in a game theoretic way but here we only focus on two of their results: a lower bound of Ω(log n) for the average stretch of any tree and their construction of a tree with exp O( √ log n log log n) average stretch in time O(m 2 ). The lower bound follows from an existing result in extremal graph theory [Bol04, pages 107-109]: there is a positive constant a such that for all n ∈ N, one can construct a graph G with n vertices and 2n edges such that every cycle G has a length of at least a log n. Now consider any spanning tree T of G. While all the n -1 edges of T have a stretch of 1, the n + 1 remaining ones form a cycle in T hence in G as well and thus incur a stretch of at least a log n. This shows that the average stretch is at least 1 2 a log n. They construct a low stretch spanning tree in a bottom up manner like the GALAXY TREE algorithm. First, they extend the definition of stretch to multigraph [Alo+95, Section 4] and then describe a procedure to transform in linear time any multigraph G with n nodes to a multigraph G on the same nodeset with at most n(n + 1) edges such the average stretch of G is at most twice that of G [Alo+95, Lemma 5.2]. The next ingredient is an algorithm to build a low diameter decomposition of a multigraph G, parametrized by a number x(n) depending of n. It works by repeatedly selecting an arbitrary node and growing a ball around it until the number of edges leaving the ball is at most a fraction 1 /x(n) of the number of edges with both endpoints in the ball. The key property of this decomposition is that it yields a partition of G in clusters such that the radius of each cluster is small (namely at most O(x(n) log n)) and there is most a fraction 1 /x(n) of edges between clusters. Finally, the iterative procedure is a follows: once a partition has been built, we compute a shortest path spanning tree in each cluster that are then collapsed into super nodes to form the next graph G and the process repeats. Another difference from GALAXY TREE, besides the partition procedure, is that G is a multigraph, taking into account the number of edges joining cluster, while COLLAPSE-STARS picks only the most direct one.

Another interesting idea from this paper is to consider a distribution over trees instead of a single instance, especially when one is concerned about the maximum stretch instead of the average one. For instance, on a cycle with n nodes, a tree is obtained by removing one edge, and that edge incurs a stretch of n -1. The uniform distribution over such trees has a maximum stretch of 2 1 -1 n [START_REF] Karp | A 2k-competitive algorithm for the circle[END_REF].

Table 3.8 -Reproduction of Table 1 from [START_REF] Abraham | Using petal-decompositions to build a low stretch spanning tree[END_REF], showing the evolution of the best asymptotic average stretch over time.

work average stretch time

[Alo+95] exp(O( √ log n log log n)) O(m 2 ) [Elk+05] O((log n) 2 log log n) O(m log 2 n) [ABN08] O(log n(log log n) 3 ) O(m log 2 n) [KMP11] O(log n(log log n) 3 ) O(m log n log log n) [AN12] O(log n log log n) O(m log n log log n)
The idea of recursively partitioning the graph and construction a low-stretch spanning tree in each part is common to all the papers of Table 3.8. Elkin et al.

[Elk+05] devise a (δ, )-star decomposition such that all the stars have comparably low radius. It was modified in [START_REF] Abraham | Nearly tight low stretch spanning trees[END_REF] to improve the stretch. Then Koutis et al. [START_REF] Koutis | A nearly-m o limits@logn time solver for sdd linear systems[END_REF] improve the runtime by rounding the edge weights to the closest power of 2 and using a modified implementation of the Dijkstra's algorithm in the case of at most k distinct edge weights [START_REF] Orlin | A faster algorithm for the single source shortest path problem with few distinct positive lengths[END_REF]. Finally, Abraham et al. [START_REF] Abraham | Using petal-decompositions to build a low stretch spanning tree[END_REF] describe an even more complex but tighter petal decomposition.

Spanners

As we mentioned, by stopping the GALAXY TREE algorithm before it finishes, we obtain a set of edges spanning the graphs that is not a tree. Such structure are called spanner. More precisely, the subgraph H is said to be an tspanner of G if, for a parameter t ≥ 1, and for every pair u, v ∈ V of vertices, it holds that d H (u, v) ≤ t • d G (u, v). The problem was introduced by Peleg et al.

[PS89] and Peleg et al. [START_REF] Peleg | An optimal synchronizer for the hypercube[END_REF] and has been extensively studied since then, for it has many applications in network design. It was also showed to be NP-HARD to approximate [START_REF] Elkin | The hardness of approximating spanner problems[END_REF]. The most simple construction is a greedy algorithm [START_REF] Althöfer | On sparse spanners of weighted graphs[END_REF] that works similarly to the minimum spanning tree construction. Starting from an empty subgraph H, it goes through every edge (u, v) of G sorted by weight and check if there is a path between u and v in H of length at most t. If this is the case, the edge (u, v) is dropped, otherwise it is inserted in H. This results in a (2t -1)-spanner with O(n 1+1/t ) edges, which is an optimal trade-off between those two quantifies. Furthermore on weighted graphs, the greedy spanner total weight is essentially optimal [START_REF] Filtser | The greedy spanner is existentially optimal[END_REF]. However, the best implementation of it, using a dynamic data structure [START_REF] Roditty | On dynamic shortest paths problems[END_REF] is not scalable for it runs in O(tn 2+ 1 /t ) and cannot easily be parallelized. Parallelization therefore requires other kind of approaches [START_REF] Pettie | Distributed algorithms for ultrasparse spanners and linear size skeletons[END_REF][START_REF] Miller | Improved parallel algorithms for spanners and hopsets[END_REF]. Recently, Elkin et al. [START_REF] Elkin | Efficient algorithms for constructing very sparse spanners and emulators[END_REF] showed how to obtain, for any > 0, a (2t -1)-spanner with O(n 1+1/k / ) edges in t rounds, with probability at least 1 -.

Other problems Finding low stretch trees and spanners with respect to the existing edges is the most relevant problem when addressing the EDGE SIGN PREDIC-TION problem. For the sake of completeness, we nonetheless give an overview of some related problems.

For instance, Johnson et al. [START_REF] Johnson | The complexity of the network design problem[END_REF] define the following problem, where the stretch is defined over all possible pairs of nodes15 : Problem 2 (Network Design Problem). Given an undirected integer-weighted graph G = (V, E, w), a budget B ∈ N and a criterion threshold C ∈ N, does there exist a spanning subgraph G = (V, E ) of G with weight w(E ) ≤ B and criterion value F (G ) ≤ C, where the criterion function F (G ) denotes the sum of the weights of the shortest paths in G between all vertex pairs? They prove that finding such a subgraph is NP-COMPLETE, by exhibiting a reduction from the KNAPSACK problem. They also prove that the less general problem of finding a spanning tree on an unweighted graph, that is Problem 3 (Simple Network Design Problem). Problem 2 with w being the equal to 1 for all edges in E and

B = |V | -1.
is also NP-COMPLETE by reduction from EXACT 3-COVER. However, it has recently been show that this Simple Network Design problem can be approximated to a constant factor 6 [START_REF] Chepoi | Constant approximation algorithms for embedding graph metrics into trees and outerplanar graphs[END_REF]. Moreover, even when the graph is weighted, Abraham et al. [START_REF] Abraham | Embedding metrics into ultrametrics and graphs into spanning trees with constant average distortion[END_REF] achieve a universal constant bound for any weighted graph.

Another problem appear when the low-stretch structure H can include edges not in G (as long as the distances in H remain larger than the distances in G). This is captured by the following problem [START_REF] Scott | The optimal network problem: some computational procedures[END_REF]:

Problem 4 (Optimal Network Problem). Given a set V of n vertices, find a set of spanning edges E ⊂ V 2 that minimizes the sum of the length of the shortest paths between all vertex pairs while the total length of the resulting network does not exceed some upper bound B ∈ N.

This can be seen as a special case of Problem 2 with G being the unweighted n-complete graph. Scott [START_REF] Scott | The optimal network problem: some computational procedures[END_REF] proposes a backtracking solution and two local search approximate algorithms. Some early branch and bound heuristic solutions to Problem 4 are surveyed in [Min89, Section 2.3.2] although they do not come with asymptotic guarantee on the stretch. Furthermore, Wong [START_REF] Wong | Worst-case analysis of network design problem heuristics[END_REF] proves that for any ∈ (0, 1), finding a |V | 1-approximation is NP-COMPLETE. However, if we consider the average stretches over a distribution of trees, then this approximation factor can be reduced to Θ(log n) [START_REF] Fakcharoenphol | A tight bound on approximating arbitrary metrics by tree metrics[END_REF].

Finally, the stretch can also be computed for a subset of the edges. This is useful in cases where we have prior information on the importance of individual nodes or edges. For instance, Abraham et al. [START_REF] Abraham | Ramsey spanning trees and their applications[END_REF] show that for every t, any n-nodes graph G = (V, E) has a subset S of size at least n 1-1 /k , and a spanning tree that has stretch O(k log log n) between any node in S and any node in V . Likewise, Gupta et al. [START_REF] Gupta | Last but not least: online spanners for buy-at-bulk[END_REF] describe how to maintain a light subgraph H that minimizes the distance between pairs of source and sink that are given in an online fashion.

As shown by Table 3.9, those problems defined in the seventies are still being discussed nowadays in top tier conferences, proving their relevance and impact beyond the EDGE SIGN PREDICTION problem.

Empirical evaluation

In this section, we provide empirical evidences of the properties of GALAXY TREE over several classes of graph, and compare it with a BREADTH FIRST TREE baseline. Namely, we consider three kinds of graph topology (with both synthetic and real world instances that carry signs on their edges) and evaluate (i) what average stretch is reached by various trees and (ii) how accurate is the sign prediction.

O (log n(log log n)) [AN12] (2t -1)-spanner, O(n 1+1/t ) edges [Alt+93]
Θ(log n) in expectation [START_REF] Fakcharoenphol | A tight bound on approximating arbitrary metrics by tree metrics[END_REF] all possible pairs 6 for unweighted graphs [START_REF] Chepoi | Constant approximation algorithms for embedding graph metrics into trees and outerplanar graphs[END_REF] and O(1) in general [START_REF] Abraham | Embedding metrics into ultrametrics and graphs into spanning trees with constant average distortion[END_REF] no need for extra edges in that case

Graph topology

The three kinds of topology we consider are:

GRID which are 2D lattices, where each node has four neighbors except on the boundary. The synthetic ones are square, while the "real world" ones represents the four neighbors pixel connectivity of the pictures showed in Figure 3.12 on page 87.

PREFERENTIAL ATTACHMENT which are built synthetically according to the model of Barabási et al. [START_REF] Barabási | Emergence of scaling in random networks[END_REF]. While this does not follow the more rigorous specification of Bollobás et al. [START_REF] Bollobás | The diameter of a scale-free random graph[END_REF], informally, we start with a line graph of m nodes and add node one by one until the graph consists of n nodes. Each time a new node is added, it is connected to m of the existing nodes with a probability proportional to their degree. Here we choose m = 3.13, that is when adding a new node, we pick 3 or 4 existing neighbors such the initial expected number of neighbors for each new nodes is 3.13. Such graphs are quite sparse and have short diameter, thus providing a crude but reasonable approximation of online social networks. Therefore, the real world instances of the PREFERENTIAL ATTACHMENT model are WIKIPEDIA, SLASHDOT and EPINION networks from Chapter 2 along with GOOGLE+. The last one is constructed from ego networks of GOOGLE+16 by keeping the largest connected component of users whose gender is known. Basic statistics of those real PREFERENTIAL ATTACHMENT graphs are presented in (Table 3.10).

TRIANGLE which consists of a Delaunay triangulation of a set of points randomly located in a 2D space.17

Stretch

The first property of Galaxy trees we wish to evaluate is their stretch, which depends only of graph topology. Recall that following equation Equation 3.8, we define the average test edge stretch as

1 |Etest| (u,v)∈Etest |path T u,v |, where |path T u,v
| is the unique path between u and v in T .

As we consider unweighted graphs, we compare GALAXY TREE with a natural baseline, namely a spanning tree rooted at the highest degree node and obtained through a breadth first visit of the graph. This involves randomness in the order in which nodes are visited. Likewise in GALAXY TREE, the choice of the edge linking two stars is not always unique, meaning that we have to break ties at random. Therefore, for each graph, we repeat the tree construction 12 times and present the average result, noting that the variance (showed as error bar in Figure 3.13) is small.

On PREFERENTIAL ATTACHMENT and TRIANGLE, we see in Figure 3.13 on page 88 that both trees exhibits logarithmic stretch, although with a larger constant for GALAXY TREE. Note that this is also the case for others low stretch tree methods [PKK14, Section 5.3.1]. On GRID however, GALAXY TREE preserves this logarithmic stretch growth while this is visually no longer the case for BREADTH FIRST TREE. In that case, we cannot expect a better stretch than log n 2048 according to [Alo+95, Theorem 6.6].

Sign prediction

The second design goal of Galaxy trees is to accurately predict the sign of edges in E test . Except for the three real datasets that already include signs18 , all the other are constructed, meaning we have to set sign on their edges in the first place. This is done by partitioning the nodes into two clusters. For GOOGLE+ we use node gender, for pictures we use node color (black or white), and for all others, we propagate labels 0 and 1 from randomly selected high degree nodes. Once each node belongs to one of the two clusters, we set the sign of an edge between two nodes to be + if they are in the same cluster andotherwise. Predicting using path parity will thus gives perfect result. To test performance in real situations, we then add noise, that is we select a fraction of edges uniformly at random and flip their sign.

Like in Section 2.5, we evaluate the performance of our prediction using the Matthews Correlation Coefficient (MCC), defined in equation Equation 2.9 on page 26. As showed in Figure 3.14 on page 89, when the noise level is low, GALAXY TREE performs better than BREADTH FIRST TREE. As the noise level gets higher, they have similar performance. Note also than in Figure 3.14c, GALAXY TREE is less sensible to the size of the graph.

To further assess the quality of our trees, we plug them in them into an existing heuristic method to predict edge sign: A sym exp [START_REF] Kunegis | The slashdot zoo: mining a social network with negative edges[END_REF]. It computes the exponential of the adjacency matrix after it has been reduce to z dimension. This allows to count the sign of all paths between two pairs of nodes with decreasing weight depending of their length. To simulate an active learning setting, we reveal only a subset of edge in A. This subset can be: i) the edges forming a BREADTH FIRST TREE, ii) the edges forming a GALAXY TREE iii) |V | -1 edges chosen uniformly at random. We set the parameter z equal to 15 because i) it is one of the best in [KLB09, Fig. 11] and ii) it performs well on real datasets in [Ces+12a, Fig. 3].

As the A sym exp has a O(n 3 ) complexity and uses quite some memory at prediction time, the larger graphs used previously are not all included. The conclusion of Figure 3.15 on page 89 is that except on social networks, it is better to use spanning trees than random edges. Specifically, GALAXY TREE on GRID and BREADTH FIRST TREE elsewhere.

Conclusions

Summary

In this chapter, we expanded our efforts on characterizing edges in complex networks by addressing the EDGE SIGN PREDICTION problem in undirected signed graphs. We first demonstrated, conceptually and practically, that the methods we designed in Chapter 2 are not suitable to this new setting. Drawing on the well established theory of social balance, we therefore came up with a different learning bias. More precisely, we posit that nodes latently belong to K groups. Then that the edge signs are initially set to be positive if both endpoints belong to the same group and negative otherwise. And that finally we observe such assignment after it had been perturbed uniformly at random.

We connected twice this bias for the EDGE SIGN PREDICTION problem with the CORRELATION CLUSTERING problem. First, solving CORRELATION CLUSTERING on a training set would provide us with the most reasonable partition of the nodes into K groups, allowing us to predict the test signs accordingly. Second, the optimal value of the CORRELATION CLUSTERING objective is a measure of the difficulty of the EDGE SIGN PREDICTION problem [START_REF]A correlation clustering approach to link classification in signed networks[END_REF]. We thus presented a detailed overview of the CORRELATION CLUSTERING problem and its proposed solution. Our goal was to point out that despite its hardness in the worst case, there exist efficient heuristics in practice, as well as methods with formal guarantees in more favorable cases. We furthermore noted that such cases are those when the sign assignment is close to follow our bias.

Such general CORRELATION CLUSTERING approaches would typically be used in a batch setting. In the last part of this chapter, we instead shifted our attention to the active setting, in the special case where K = 2. This allowed us to exploit our bias in a more specific way. Namely, we described an implementation of the GALAXY TREE spanning data structure [START_REF] Vitale | [END_REF]. The idea is to query the sign of edges that connect the endpoints of every other edge by the shortest possible path. Our experiments showed that this GALAXY TREE construction used in active EDGE SIGN PREDICTION is competitive for some classes of graphs.

Future work

Beyond those preliminary experiments, we could push further our methods along several directions:

1) The basic analysis of Proposition 3 shows that on any graph with n nodes, the GALAXY TREE will perform at most n collapses before terminating. However, as we mentioned on page 76 at the end of Section 3.3.1, we believe that this maximum number of collapses might be as low as O(log n). It turns out we were not able to prove this statement. This is unfortunate, for besides giving us guarantees on the algorithm runtime, this would also inform us about the stretch of the resulting tree. Indeed, let us consider the reverse direction in which the GALAXY TREE algorithm operates. That is, instead of a sequence of collapses transforming the original graph into a single node, we look at a sequence of expansions turning this single node into the original graph.

Informally, the length of the longest path can only increase by a factor of five as each pair of nodes in G t+1 expands into two connected stars in G t . 19 Whereas the behavior of the average stretch is less straightforward, it is reasonable to expect that the smaller the number of expansions, the lower the stretch.

2) As mentioned in Section 2.4, EDGE SIGN PREDICTION could be extended to weighted signed graphs. This is also relevant in undirected signed graphs. For instance, the signs might have been generated by thresholding an expensive symmetric similarity function. In that case, we might want to avoid extra evaluation of that function by predicting the signed weight of some new edges.

A natural way to modify the GALAXY TREE algorithm is the following. When choosing the center of stars, pick the node with the current highest weighted degree. When choosing an edge to connect two centers, pick among the edges with the lowest endpoints eccentricity the one with the lowest weight. What is less clear is to which extent such changes would affect the analysis.

3) We wish we had more time to further explore CORRELATION CLUSTERING under stability assumptions, especially as a way to handle the case where K > 2. Recall that a clustering instance, made of a weighted graph and a partition objective, is α-stable (with α > 1) if its optimal partition remains the same whenever every weight w i is multiplied by a factor c i between 1 and α. Furthermore, we say that an algorithm is robust if, given a clustering instance I, it yields one of the two following outcomes in polynomial time: (i) if I is α-stable, returns the optimal solution of I;

(ii) if I is not α-stable, either returns the optimal solution of I or reports that I is not α-stable. This is a handy property because, in general, we cannot practically check the stability of an instance. Indeed, since we do not know the optimal solution, we cannot tell whether it changes or not under perturbations.

As an example, let us consider the k-MINIMUM MULTIWAY CUT problem. Given a weighted undirected graph G and a set of k terminal nodes, the goal is to partition G into k clusters, each containing exactly one terminal, such that the total weight of edges across clusters is minimized. This bears some superficial similarity with k-MINIMUM MULTICUT. Recall from Section 3.2.2.2 that k-MINIMUM MULTICUT requires us to find the lightest set of edges to cut in order to separate k pairs of nodes. Recall further that k-MINIMUM MULTICUT is equivalent to CORRELATION CLUSTERING. As for k-MINIMUM MULTI-WAY CUT, we can cast it as a CORRELATION CLUSTERING problem [BBC04, Theorem 24]. Interestingly, there exists a robust algorithm for 2 -2 /k-stable instances of k-MINIMUM MULTIWAY CUT [START_REF] Angelidakis | Algorithms for stable and perturbation-resilient problems[END_REF]. Under the assumption that graphs obeying our learning bias are α-stable, for some α depending of the graph irregularity, having such a stable algorithm for k-MINIMUM MULTICUT would be very valuable. However, this might prove challenging. Indeed, k-MINIMUM MULTICUT is harder than k-MINIMUM MULTIWAY CUT, as illustrated by their respective approximation factor, O(log n) and less than 1.3 [START_REF] Buchbinder | Simplex transformations and the multiway cut problem[END_REF].

Edge clustering in node attributed graphs

In this chapter, we extend the problem of predicting edge types beyond the binary case. Furthermore, we assume that we are provided with node attributes in addition to the mere graph topology we used in the previous chapters. Our motivation is twofold: intrinsic and extrinsic. First, we want an interpretable model that, given two connected nodes, can explain why they are linked. This leads us to use a linear model, for it is both easily amenable to interpretation and computationally efficient. Second, edge-labeled graphs are susceptible to further studies by existing networks analysis methods, as we will describe shortly. Thus, our method is a useful way of enriching data to extract new insights from it. As demonstrated by our prior extensive coverage, predicting edge sign is an interesting and challenging problem in its own right. Yet it is natural to extend and generalize it. First, there are other edge binary classification problems that arise in graphs. For instance, predicting whether a link in a social network is strong or weak [De +14; RTG17]; predicting whether two authors in a citation network are advisor-advisee or regular coauthors [START_REF] Wang | Mining advisor-advisee relationships from research publication networks[END_REF]; predicting whether two persons in a company network have a relationship of manager-subordinate or not [START_REF] Diehl | Relationship identification for social network discovery[END_REF]; predicting whether two proteins interact physically or genetically; or whether two neurons have a chemical or ionic link between them [START_REF] Nicosia | Measuring and modeling correlations in multiplex networks[END_REF].

Second, and more generally, we are also interested in multiclass problems, within the context of multilayer graphs [START_REF] Kivela | Multilayer networks[END_REF][START_REF] Boccaletti | The structure and dynamics of multilayer networks[END_REF]. While Kivela et al. [START_REF] Kivela | Multilayer networks[END_REF] define an elaborate and highly expressive notion of multilayer graph, in this chapter we shall restrict ourselves to the following1 : Definition 4.0.1 (k-multilayer graph). A k-multilayer graph G is a pair (V, {E 1 , . . . , E k }), where V is a set of n nodes and each E i ⊂ V × V is a set of edges, for i ∈ k . We call the subgraph G i = (V, E i ) the i th layer of G and it will be convenient to think of G as the superposition of its k layers, so that by a slight abuse of notation we can write G = k i=1 G i . We naturally identify each layer with a type of relation, so that our classification problem is to decide to which layer a given edge belongs. Furthermore, we make the simplifying assumption that two nodes can be connected in at most one layer (i.e. ∀u, v ∈ V, (u, v) ∈ E i ⇒ ∀j = i, (u, v) / ∈ E j ), meaning that the resulting superposition of all layers does not turn into a multigraph. As mentioned in the introduction, such multigraphs can be used to model different relation types in networks related to social interactions [ST10; KMK11], bibliographical citation [Cai+05; DKK11], economic exchange [Kal+10; BFG10; CMM15] and biology [START_REF] Li | Integrative analysis of many weighted co-expression networks using tensor computation[END_REF][START_REF] Kéfi | Network structure beyond food webs: mapping non-trophic and trophic interactions on chilean rocky shores[END_REF][START_REF] Muldoon | Network and multilayer network approaches to understanding human brain dynamics[END_REF]. This additional information on the nature of the relationships allow traditional graph tasks to be performed more precisely and with higher granularity. Consider for instance node ranking [START_REF] Coscia | you know because i know": a multidimensional network approach to human resources problem[END_REF]; node clustering [Bot+15, Section 2; Kiv+14, Section 4.5.1] and the study of information spreading [START_REF] Salehi | Spreading processes in multilayer networks[END_REF]. However, to produce such finer results, the methods we just referenced require labeled edges that in many cases are not available, thus justifying the usefulness of our work. Indeed, in Table 4.1, we give some examples of networks whose edges are currently unlabeled and what type of relations they could possibly denotes. In other words, given a small list of possible reasons in terms of attributes and two nodes, why are they connected? Besides predicting potentially more than two classes, the other difference with the setting of the previous chapters is that we now assume that nodes are associated with attributes, which we call profiles. One could argue this makes our framework less general, for attributes are not always available, or may come with missing values. Another recent finding is that, despite a natural assumption, the nodes profiles in attributed graphs are not necessarily correlated with known ground truth communities [START_REF] Hric | Community detection in networks: structural communities versus ground truth[END_REF], casting further doubt on the usefulness of that extra information. This raises the question of why we cannot simply extend the troll/trust method of counting the type of edges incident to every node and learn a model from such features. A similar tension between relying solely on always available albeit sparse topological information or leveraging possibly richer attributes information also exists in link prediction [START_REF] Martínez | A survey of link prediction in complex networks[END_REF]. However, the presence of profiles is motivated by the last fundamental difference with the previous problems. In this chapter, we do not have access to any direct supervision in the form of labels. Rather than a strict classification problem, we are thus faced with unsupervised clustering, guided by the information contained in node profiles. Note that contrary to the relation between CORRELATION CLUSTERING and EDGE SIGN PREDICTION, this time the objects we seek to cluster are edges and not nodes. Furthermore, we stress that the profiles crucially increase the interpretability of our models.

It would be tempting to assume that nodes are connected because they share the same attributes. Indeed, a common learning bias in graphs is that nodes are connected together because they are similar, or as often said proverbially: "birds of a feather flock together". Formally this is called the homophily principle [START_REF] Mcpherson | Birds of a feather: homophily in social networks[END_REF] and has been consistently verified, both in offline [HL78; Mar88] and online social networks [The09b; BAX12]. While our every day life experience makes homophily not surprising in social networks, the same assortative patterns [START_REF] Newman | Assortative mixing in networks[END_REF] are present in other kinds of networks [START_REF] Murai | Assortative mixing in bittorrent-like networks[END_REF]. Here though, we assume that links cannot only be explained by a global homophily along all attributes but rather by partial homophily, partial heterophily or even both. An example of this combination is the balanced news diet problem [START_REF] Garimella | Balancing information exposure in social networks[END_REF], where we try to connect users of a social network to prevent them from being locked into topical echo chambers. In that situation, a user A might be more likely to listen to the opposite point of view of user B about gun control if they both share common demographics and interests. This suggests, as we make it formal in the next section, that node attributes can include polarized opinions, i.e. both positive and negative numerical values.

After having motivated our problem of predicting multiple edge types in attributed graphs, we carry on by stating more formally the EDGE ATTRIBUTED CLUSTERING problem in Section 4.1. Namely, we define a similarity between nodes profiles and seek a small number of explanations (corresponding to edge types) achieving good scores as prescribed by a linear model. After explaining how this problem relates to the EDGE SIGN PREDICTION problem on 2-clusters signed graphs on page 97, we introduce in Section 4.2 several methods to solve it in practice. This ranges from tailoring the k-MEANS algorithm to our goodness measure (Section 4.2.1), to a convex relaxation of our objective function (Section 4.2.2) all the way through a richer and higher capacity matrix formulation allowing overlapping clustering of edges (Section 4.2.3). Because finding publicly available real data with both node attributes and edge types proved difficult, we describe a synthetic model in Section 4.3, on which we perform extensive experiments. We then survey in Section 4.4 works that are related to the EDGE ATTRIBUTED CLUSTERING, while noting that very few address it directly. Finally, we suggest in Section 4.5 several directions in which our model could be extended to handle more general classes of graphs.

Attributed graphs and problem definition

We start by introducing some notations and terminology about edge types in nodeattributed graphs. Then we present two options to model the interactions between nodes and the role of attributes in "explaining" edges (Section 4.1.1). Choosing one of these two options leads us to formulate the EDGE ATTRIBUTED CLUSTERING problem in Section 4.1.2. We then describe additional constraints allowing us to take into account the topology of the graph. Finally, we elaborate on the relation between EDGE ATTRIBUTED CLUSTERING and EDGE SIGN PREDICTION in signed graphs.

Setting and modelling

Like we wrote in the introduction, to classify edges, we make use of both the topology of the graph and the attributes of its nodes. Namely, we are given an undirected, unweighted graph G = (V, E). We also have side information in addition to the graph topology. It takes the form, for every node u, of a d-dimensional feature vector x u ∈ R d that we call the profile of u. Because we want to model heterophily, we more explicitly have that x u ∈ [-1, 1] d . To justify this statement, let us take the example of encoding the gender of a user in a social network, assuming there is only two values possible: male and female. Traditionally, this attribute would be set to 1 if the user is female and 0 otherwise. Instead, we use +1 and -1, to clearly emphasize the difference between the two values. The same is valid for continuous variables. For instance, an attribute could range from -1 to +1 to represent the ideological stance of a user on the conservative/liberal scale [START_REF] Bakshy | Exposure to ideologically diverse news and opinion on facebook[END_REF]. 2 This applies even to non diverging positive quantities such as age, which can be projected to [-1, 1] using pre-processing like standardization or z-score.

Stacking all these node profiles in a |V | × d matrix X, our input so far is (G, X). As it is standard in classification problem, we are also given the number of edge types k. 3 Moreover, recall that we do not merely want to classify edges into type, but also provide an explanation of why two nodes are connected. This requires the introduction of three further concepts: 1) how to represent a connection (i.e. an edge), 2) how to represent an edge's explanation, and 3) how to evaluate which explanation is the best. We now present our modelling choice: 1) We naturally describe an edge (u, v) through the profiles of its endpoints. More precisely, we combine x u and x v by an operator s defined as:

s(u, v) = x u • x v = s uv ∈ R d ,
where • stands for the Hadamard (or component wise) product. As showed in Table 4.2, for each dimension i ∈ d , the vector s uv holds a value indicating whether the profiles of u and v agree, disagree or are indifferent along that dimension.

Table 4.2 -We show the effect of the Hadamard product in combining information from two users u and v. Instead of actual numbers, in this table we use + to denote a large positive value, -to denote a large negative value, 0 to denote a small absolute value and to denote an arbitrary value.

x u;i x v;i s uv;i

+ + + - - + + - - 0 0
2) The k explanations take the form of bounded norm vectors D k = {w 1 , w 2 , . . . , w k } ⊂ B d , where B d is the 2 -unit ball of R d . We call such a vector w , for ∈ k , a direction. As we shall explain more formally later on, those directions are exactly the parameters learned while solving our problem. 3) Given an edge (u, v), all the k directions {w 1 , w 2 , . . . , w k } provide a possible explanation of why u and v are connected. We score these explanations by assigning them a goodness of explanation4 defined as:

g(s uv , w ) = s uv T w .

As the w notation hints, a direction can be interpreted as a set of weights. These weights indicates to which extent each attribute contributes to the connection, according to that direction. For a given set of directions D k , the classification of edges is naturally performed by the following operator E, returning the index of the direction with the maximal goodness:

E : E × R d × . . . × R d -→ k (u, v), D k -→ arg max ∈ k g(s uv , w )
Because it will be clear from context, we lighten the notation and simply write E(u, v) instead of E ((u, v), D k ). This choice of s and g enjoy two interesting properties. First, it encompasses all cases of homophily, indifference and heterophily. For instance, given

x u = 1 0 -1 T and x v = -1 1 -1 T , the direction w = -1 / √ 2 0 1 / √ 2 T ∈ B d
achieves the maximal goodness. In doing so, it highlights that u and v disagree on the first component (i.e. heterophily), agree on the last component (i.e. homophily) and that the middle component is irrelevant (i.e. indifference). This also shows that, second, given the linear nature of the goodness g once s uv has been computed, it provides interpretable explanation of why two nodes are connected.

Those two properties of s and g should not be taken for granted. Indeed, a natural choice for the operator s is the difference between the profiles, that is s uv = x u -x v . In that spirit, if u and v are close along a direction w , we expect x u T w to be close to x v T w , as measured by |x u T wx v T w |. And because it is easier to deal with smooth functions, we define the goodness to be g(s uv , w ) =s uv T w 2 , where the minus preserve the semantic than the "better" the direction w , the larger the goodness. However, with this formulation, there is no way to set w ;i in order to "highlight" the fact that x u;i and x v;i are widely different. Furthermore, the square in the goodness expression loses the linearity.

We conclude with a last comment on the norm of the directions. While being exactly of norm one is not primordial, the role of bounding the norm of the directions (i.e. having d i=1 w 2 ;i ≤ B 2 ) is to tie the d dimensions together. Without such a constraint, and with our choice of s and g, we could simply study each dimension separately, whereas this bound adds a coupling across different attributes, since increasing the weight of an attribute mechanically affects the weights of the other attributes. For consistency, we also consider node profiles to be normalized and having unit norm. A further practical justification is that if nodes are users of a social network and the profiles measure their activity across several domains, this allows to compare users with various level of total activity.

Learning problem and additional constraints

Problem definition With working definitions of s and g at hand, we now formally write down the problem of finding the directions maximizing the goodness of the graph.

Problem 5 (EDGE ATTRIBUTED CLUSTERING). Given a graph G = (V, E), node profiles X ∈ [-1, 1] n×d and an integer k ∈ N, find a set of k norm-bounded directions D k = {w 1 , w 2 , . . . , w k } ∈ B d and associate to every edge of E the direction with the maximal goodness. Formally, solve arg max

D k ={w 1 ,...,w k }⊂B d u,v∈E g(s uv , w E(u,v) ) (4.1)
where s uv = x u • x v , g(s uv , w ) = s uv T w and E(u, v) = arg max ∈ k g(s uv , w ).

Example In Figure 4.1, we present a small instance of the EDGE ATTRIBUTED CLUSTERING problem, where we are given the graph shown in Figure 4.1a, whose node attributes are listed in Table 4.1b. The first two attributes are categorical, denoting the company employing each user, either G or F. Instead of a classic binary one-hot encoding, we additionally use -1 to denote that a user had been employed by a given company. Gender is encoded as presented earlier, and we also apply a pre processing to transform age between -1 and 1. Finally, opinion represents the view of former F employees about a societal issue, with 0 denoting a neutral or unknown position. In Table 4.1b, we also show two directions that explain the connections in the graph G. The first covers being of the same age and same companies, especially at G, while the second directions covers former employees of F having different age and opinion. Not only are the results interpretable as claimed before, but we can also assess the strength of each explanation, using their goodness. For instance, we see that the edge between x and y has a low score, because although those two users have the same age and used to work at F, not both of them works at G. .126

(a) The graph G. Larger nodes represent old users while smaller ones are younger. Their gender is inscribed in the node and their shape represent their opinion: rectangle for -1, circle for 0 and diamond for +1. Finally, orange stands for being employed at company G and faded blue for having been employed at company F. As for the edges, the red ones are labeled by the first direction and the green ones by the second direction, and we label them with the associated (normalized) goodness. to increase as well, at the cost of interpretability. There could be a principled approach to finding the "best" k, based on a information theoretical measure of the complexity of D k and the minimum description length principle [START_REF] Grünwald | A tutorial introduction to the minimum description length principle[END_REF]. However, in the interest of simplicity, in the following we focus on the formulation stated in Problem 5, where k is given. In practice, we further constrain k to be small (say less than 10). Within the multilayer framework, k could also be seen as the number of underlying layers. To retain a fair level of interpretability, we deem appropriate to have k not too large.

G F gender age opinion

u +1 0 -1 -0.8 0 v +1 -1 -1 -0.7 +1 w +1 0 +1 0.6 +1 x +1 -1 +1 0.9 -1 y 0 -1 -1 0.7 -1 z 0 -1 +1 -0.6 +1 w 1 +3 +1 0 +1 0 w 2 0 +4 0 -2 -4 (

Topological constraints

In addition to assuming that the input graph is the superposition of a small number of layers, another way to leverage the graph topology is to define local constraints at the node level. Indeed, one of our motivation is that it is not enough to look at edges in isolation, but rather we expect the connections of one node to influence the connections of its neighbors. We now present two of these local, node-level constraints.

First, if a graph was to perfectly follow our model, every edge (u, v) would be maximally explained by one of the k directions. This would happen if the profiles of u and v were collinear with this direction, up to sign reversal. When considering several edges, we could imagine that the nodes profiles result from the following process. Two nodes u and v establish a connection by first picking a direction w ∈ D k and add to their base vector b u and b v ∈ R d a fraction a uv > 0 of w . We therefore subtract the following term

L node = u∈V x u -b u - v∈N (u) a uv w E(u,v) 2 2 (4.2)
from (4.1). The cost we pay is introducing the extra parameters {a uv } u,v∈E and {b u } u∈V to be learned, but we will see in the next section how to handle that in practice. On the other hand, it has two advantages. First, this is another way, besides goodness, in which the (given) profiles provide prior information about the (unknown) directions, and thus restrain the search space. Second, this makes the directions dependent of the topology of the graph, by coupling them with the neighborhood of each node.

Second, we also make the assumption that each node u is only involved in k local < k different directions. The intuition is that when d is large enough, users have to focus their energy and can only express their interest in a few number of dimensions. In other words, all the edges incident to u can only be associated with one of k local directions. For some values of k local , this is a NP-COMPLETE graph colouring problem and thus cannot necessarily be enforced exactly. Still, it can be seen as a learning bias and will be used to generate synthetic data. Furthermore, we can also express it as an optimization constraint. Specifically, suppose that for an edge (u, v) associated with the direction w i ∈ D k , we have a k-dimensional vector y uv equal to e i (that is all zero except for the i th component set to one). Let us stack all such vectors into a |E| × k matrix Y , and let C ∈ R |V |×|E| be the incidence matrix of G5 , with c u being its u th row. Then c T u Y is a k-dimensional vector that counts how many times each direction is incident to node u. We thus want the 0 norm of c T u Y (i.e. the number of its non-zero component) to be upper bounded by k local . This is not a convex constraint but we can relax it, for such sparsity inducing problems have been well studied [START_REF] Bach | Optimization with sparsity-inducing penalties[END_REF]. Namely, we let y uv be a softmax membership k-dimensional vector, that is we set, ∀i ∈ k , y uv;i = exp(βg(s uv , w i )) k j=1 exp(βg(s uv , w j ))

, with β > 0 a large positive constant. And we replace the 0 norm by a 1 norm (or even a k-support norm [START_REF] Argyriou | Sparse prediction with the k-support norm[END_REF]), yielding the following L local term to be minimized:

L local = u∈V v∈N (u) y uv 1 = u∈V   k i=1 v∈N (u)
exp(βg(s uv , w i ))

k j=1 exp(βg(s uv , w j ))   (4.3)
It is reasonable to further expect that two neighboring nodes share one common direction among their k local ones. One way to ensure this condition is to imagine that the set of allowed k local directions changes smoothly over the graph. This property depends of the topology, as we can see by considering two extreme cases: a line and a complete graph. On the line, once a node has picked two allowed directions from its two neighbors, the remaining ones can be chosen arbitrarily. On the complete graph on the other hand, this choice is much more constrained.

Connection with signed graphs

The learning bias of Section 3.1.3 implicitly defined an idealized EDGE SIGN PREDICTION problem as:

Problem 6 (idealized EDGE SIGN PREDICTION). Given n nodes belonging latently to K groups, predict whether two nodes u and v belong to the same group (that is (u, v) is positive) or not (that is (u, v) is negative).

In Section 3.3 we studied in particular the important special case where K = 2. Now consider an instance of Problem 5 where k = 2, and where the profiles and edges are based on group membership. We now check that the optimal solution of such an instance is consistent with the solution of this idealized EDGE SIGN PREDICTION problem on the same graph.

Let us first define this instance formally. Given a graph G = (V, E), call the K = 2 groups of Problem 6 L and R. We then define the nodes profiles X as follow. We first draw uniformly at random a vector p from {-1, 1} d , then let the profiles in L be p Then D 2 provides the following solution to Problem 6: each every edge explained by w 1 is predicted positive, and every edge explained by w 2 is predicted negative.

Proof. Take any w 1 , w 2 ∈ S d × S d such that w 1 = w 2 . Assume without loss of generality that w 1 = arg max w∈{w 1 ,w 2 } s inner T w. This implies that:

s inner T w 1 = 1 d d i=1 w 1;i > 1 d d i=1 w 2;i = s inner T w 2 .
In turn, we have that:

s outer T w 2 = - 1 d d i=1 w 2;i > - 1 d d i=1
w 1;i = s outer T w 1 .

We can thus rewrite the maximization objective (4.1) of Problem 5 as:

arg
s outer T w 2 = arg max w 1 ,w 2 ∈S d m i 1 d T w 1 + m o -1 d T w 2
One can check using Lagrange multipliers that for any c ∈ R n , max w∈S d c T w = c c . The solution of (4.4) is therefore:

w 1 = 1 √ d and w 2 = -1 √ d
As claimed, the edges associated with w 1 (respectively w 2 ) are the edges that are to be predicted positive (respectively negative) in Problem 6. This is nothing more than a sanity check, as the profiles explicitly encode the membership of each node to one of the two groups. Indeed, the proof makes no use of the dimension d of the profiles, so that d = 1 would be enough. Slightly more interesting is the case were those profiles are set up as before, but perturbed uniformly at random. Namely, the sign of each coordinate is changed with probability p < 1 /2. One can then check that

E [s inner ] = (1 -2p) 2 d 1 = -E [s inner ]
and that furthermore, the variance of each component is 8(p-p 2 )-16(p-p 2 ) 2 d 2

. While the previous {w 1 , w 2 } solution is now only optimal in expectation, we see that the variance decrease quadratically with the dimension of the profiles. In practice we thus expect this solution to be very close to the optimal. Intuitively this can be seen as taking the majority of d votes on whether each node belongs to one group or the other.

It is natural to ask how this result can be extended to more than K = 2 groups. The most natural idea is to use one-hot encoding of the group membership as profiles, with additional negative values as in the example of Figure 4.1. Namely, set the profiles of the i th group as follow: all component are equal to -b except for the i th one, which is equal to a, where a and b are real constants such that a 2 + (K -1)b 2 = 1. However, in that case, we check numerically that w + = 1 √ d and w -= -1 √ d is the not the optimal solution, and that a solution achieving a larger objective function value does not cluster the edges between inner and outer groups.

Proposed approaches

We now introduce the five methods solving Problem 5 that we will experimentally compare in the next section. Recall that given a graph topology (V, E) and node profiles X, we first build the edges representation S = {s uv } (u,v)∈E and then look for k bounded directions that maximize the total goodness of the graph, possibly under topological constraints. We start with a straightforward baseline, which simply performs a 1) k-MEANS clustering of S, and propose a post-processing to improve it, plugging our custom goodness metric into the existing 2) LLOYD algorithm for clustering. Because these two methods consider each edge in isolation with no regard for the topology, we then describe a convex relaxation of the objective in equation (4.1), which is further 3) COMBINED with the node constraint of equation (4.2). Finally, we depart slightly from finding k directions by considering a low-rank matrix formulation, where each edge is assigned a linear combination of a small number of base directions, paving the way to overlapping edge clustering. More precisely, we give a convex formulation of that objective that can be solved by a 4) FRANK-WOLFE algorithm, as well as by an alternating optimization method, where we make the low-rank factorization 5) EXPLICIT.

k-MEANS baseline and improvement

A simple and natural baseline is to cluster the set of all similarity edge vectors {s uv : (u, v) ∈ E} using the k-means algorithm, where k is the number of directions set in Problem 5. Formally, given the graph G = (V, E) and the node profiles X as input, we order the edges from e 1 to e m . Then we build the matrix S ∈ R |E|×d , whose i th row is the similarity s uv = x u • x v between the profiles of the endpoints of e i = (u, v). Running k-means on the rows of S will thus partition E into k sets of edges, call them E 1 , . . . , E k . A drawback of that simplicity is that it does not solve Problem 5. Indeed, the partition of E does not provide the set of directions D k = {w 1 , w 2 , . . . , w k } ∈ S d we are looking for. A natural way of obtaining directions from that partition is to set w to be the normalized cluster center of E . We refer to this method as k-MEANS, and assuming it requires T iterations to converge, it has a linear complexity of O(T d|E|). However, this does not guarantee that we have assigned the best direction to every edge. We therefore propose the following heuristic, inspired by the Lloyd algorithm for k-means, but where the directions in D k play the role of centroids. Namely, we alternate between two steps: (i) create a new partition {E 1 , . . . , E k } by assigning every edge (u, v) to the direction w maximizing its goodness s uv T w, that is

E = (u, v) ∈ E : w = arg max w i ∈D k s uv T w i
(ii) update the directions to make them optimal with respect to the edges currently assigned to them. Specifically, set

D k = w = s s k =1
, where s = u,v∈E s uv is the sum of all the similarity vectors of the edge in E , until the edge assignment is stable or we reach a maximum number of iterations. By analogy with the k-means algorithm, obtaining convergence guarantees in the general case would be surprising, but in practice we expect that only a small number T of iterations would be needed. We call this method LLOYD and note it also has a linear complexity of O(T d|E|).

In both cases, those methods do not take the topology of the graph into account, motivating the following approaches.

Convex relaxation

By unrolling the operator E in (4.1), we can rewrite our optimization objective as arg max

D k ={w 1 ,...,w k }⊂B d u,v∈E max ∈ k g(s uv , w ) (4.5)
The difficulty in optimizing directly (4.5) stems from the inner max operator, which we replace here by a convex relaxation. Indeed, one can check that given a set of numbers S = {a 1 , a 2 , . . . , a |S| } and a real number β > 0, we have

max a i ∈S a i = lim β→+∞ 1 β log   |S| i=1 exp (βa i )  
Using this fact, and dividing the goodness term and the L node term of (4.2) respectively by the number of edges and nodes so that they have comparable magnitude, we now have the following objective:

arg max w 1 ,...,w k ∈S d , auv∈R, bu∈R d 1 |E|β u,v∈E log k =1 exp βs uv T w - µ |V | u∈V x u -b u - v∈N (u) a uv w E(u,v) 2 , ( 4
.6) where µ > 0 is a trade-off parameter.

Note that in the second term, w E(u,v) = arg max w ∈D k s uv T w . However, the arg max function is again not convex and we thus use the same relaxation as before. Namely, this relaxation A takes as input a k-tuple of directions and return their sum weighted by the softmax function as follow:

A : R d × . . . × R d -→ R d (w 1 , . . . , w k ) -→ k i=1 exp(βsuv T w i ) k j=1 exp(βsuv T w j ) w i
This function has an explicit derivative with respect to any w i , but it is rather costly to compute and in practice we found automatic differentiation [START_REF] Maclaurin | Autograd: effortless gradients in numpy[END_REF] to be more efficient.

We make two further simplifications. First, notice that the set S d × . . . × S d is not convex. To ease the optimization procedure, we thus only impose the directions w to lie within the unit ball and expect them to have norm close to one in order to maximize the dot product in the s uv T w term6 . Second, while (4.6) requires to optimize kd + |E| + |V |d parameters, we fix a uv = deg(u) and b u = 0 for all edges and nodes in order to be independent of the size of the graph. The final problem solved by our COMBINED method using (full) projected gradient descent is therefore:

arg max w 1 ,...,w k ∈B d 1 |E|β u,v∈E log k =1 exp βs uv T w - µ |V | u∈V x u - 1 deg(u) v∈N (u) A(w 1 , . . . , w k ) 2 (4.7)

Matrix optimization

Besides relaxing the max operator from (4.1), another solution to remove it would be to increase k to be equal to |E|. As we already mentioned, this makes the problem trivial. To avoid this, we add the constraint that all those |E| directions are linear combinations of a small number k of basis directions. In order to formulate this more clearly, we rewrite the previous objectives with matrices. We first show this can be expressed in a form that is amenable to the Frank-Wolfe algorithm. Then we take the alternative road of explicit low-rank matrix factorization, and we conclude by commenting on the respective merits and disadvantage of all these approaches.

FRANK-WOLFE method

Seeking one direction for every edge, our goal is thus to maximize u,v∈E s uv T w uv . This objective can be written in matricial form as tr(SW ) = S T , W F , where S ∈ R |E|×d is the s uv vectors for all edges stacked vertically and W ∈ R d×|E| is the matrix of all directions stacked horizontally and A, B F = tr A T B is the Frobenius inner product of two real matrices. More specifically, because of the unit norm constraint on directions, we look for W in the set M d×|E| of d × |E| matrices with unit 2 norm columns. To avoid the trivial solution of letting each direction w uv merely be suv suv , we further constraint W to be of low rank. That is, what we want to optimize is min

W ∈M d×|E| -S T , W F + rank(W ) .
However, while the Frobenius inner product is linear in W , the rank operator is highly irregular and optimizing it directly is NP-COMPLETE. Therefore, we relax the rank term by the nuclear norm of W , defined as W * = min(d,|E|) i=1 σ i (W ), where σ i (W ) is the i th largest singular value of W . The intuition is that the rank of W is equal to the number of non-zero singular value, while the nuclear norm is essentially an 1 norm on the spectrum of W . Therefore, as minimizing the 1 enforces sparsity, minimizing the nuclear norm tends to make the rank smaller by ensuring the presence of zeros in the singular values of W .7 This results in the following optimization problem:

min W ∈M d×|E| -S T , W F + λ W * ,
where λ is a regularization parameter. Since • * is a norm, this objective function is convex (although the domain over which we optimize is not), but it does not have a gradient, forcing us to rely on potentially costly proximal gradient descent methods [START_REF] Parikh | Proximal algorithms[END_REF]. Instead, we consider the equivalent problem obtained by replacing the regularization parameter with an upper bound δ on the nuclear norm of W , yielding: min

W ∈M d×|E| W * ≤δ -S T , W F .
The remaining issue is that W ∈ M d×|E| is not a convex constraint. As in the previous formulation, we thus relax it by simply imposing that the 2 norm of the columns of W are not too large. In fact, we already have a bound on these column norms, because of the nuclear norm constraint.

Proposition 5. For any W ∈ R d×|E| such that W * ≤ δ, the i th column W :,i of W satisfies W :,i 2 ≤ δ √ d.

Proof. Let W be a matrix in R d×m with d < m such that W * ≤ δ. Furthermore, let its singular value decomposition be such that we can write W = d l=1 σ l u l v T l , where u l and v l are d unit-norm vectors, of dimension d and m respectively. We can thus express the general term W k,i as

d l=1 σ l v l,i u l,k . Because v l is unit norm, v l,i ≤ 1 and W 2 k,i ≤ d l=1 σ l u l,k 2 ≤ d l=1 σ 2 l d l=1 u 2 l,k
using the Cauchy-Schwartz inequality. The norm of the i th column of W then satisfies:

W :,i 2 = m k=1 W 2 k,i 1 2 ≤ m k=1 d l=1 σ 2 l d l=1 u 2 l,k 1 2 = d l=1 σ 2 l d l=1 m k=1 u 2 l,k 1 2 From d l=1 |σ l | ≤ δ, we have that d l=1 σ 2 l ≤ δ 2 . Combined with u l being unit norm, we conclude that W :,i 2 ≤ δ √ d.
However, setting δ = 1 / √ d is too stringent in high dimension, and because this inequality is rather loose, it will likely results in very small column norms. Instead of relying solely on the nuclear norm constraint, we therefore add an additional regularization term equal to µ |E| i=1 W :,i 2 2 = µ W, W F . Our final convex formulation is then:

min W ∈R d×|E| W * ≤δ µW -S T , W F . (4.8)
Because (4.8) is of the form min x∈X f (x) where f and X are convex, we can solve it using the Frank-Wolfe algorithm [FW56; Jag13], which enjoys properties like a O( 1 t ) convergence rate, a low computational cost by avoiding projection step and the sparsity of its solution. It is an iterative procedure that maintains an estimate x (t) of the solution and works succinctly as follow: it linearizes the objective f at the current position x (t) by computing ∇f (x (t) ), it finds a minimum s (t) of that linearization within the domain X by solving arg min s∈X s, ∇f (x (t) ) and it moves toward that minimizer by a step γ, setting x (t+1) = (1γ)x (t) + γs (t) .8 In the case of (4.8), the linear minimization step amounts to arg min W * ≤δ 2µW -S T = arg max W * ≤δ S T -2µW . Moreover, the δ-ball of the nuclear norm is the convex hull of the matrix of the form δuv T with u 2 = 1 = v 2 [START_REF] Jaggi | Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization[END_REF]. By letting u (t) 1 and v (t) 1 be respectively the left and right vectors associated with the largest singular value of S T -2µW (t) , the minimizer s (t) we are looking for is therefore δu

(t) 1 v (t) 1
T , and we have W

(t+1) = (1 -γ)W (t) + t 2+t δu (t) 1 v (t) 1
T . We perform a small number T of such iterations, and these two eigenvectors are computed in a time linear in the number of non-zero entries of S T -2µW (t) [START_REF] Kuczy | Estimating the largest eigenvalue by the power and lanczos algorithms with a random start[END_REF]. Thus the overall complexity of this method, which we call FRANK-WOLFE, is O(T d|E|).

As said at the beginning of this section, the matrix formulation is not exactly solving Problem 5. Therefore, for evaluation purposes, here we describe two ways to extract k directions and an edge assignment from the matrix W obtained through this FRANK-WOLFE method. The first is rather straightforward. It performs a kmeans clustering of the columns of W and use it as assignment, while the directions are the normalized cluster centers. The second involves an extra optimization step. Specifically, letting r be the rank of W , we first compute a reduced SVD such that W = U ΣV T , and then let P = U ∈ M d×r and Q T = ΣV T ∈ R r×|E| . We then note that for any invertible matrix R ∈ R r×r , we have W = P R -1 RQ T . Thus we look for a matrix R that makes the columns of RQ T as close as possible of having unit 2 -norm, that is R = min R∈R r×r E i=1 RQ T :,i 2 2 -1 . Finally, as before, we cluster the columns of RQ T using k-means, and use the columns of P R -1 as basis directions, noting they are not necessarily unit norm anymore.

EXPLICIT low rank factorization

We can avoid the rank regularization term altogether by making the low-rank decomposition of W explicit, that by writing W = P Q T and optimizing over both P and Q. We thus call this method EXPLICIT. However, this comes at cost of convexity. Indeed, given a rank k ≤ min(d, |E|), the problem becomes, assuming A and B are fixed:

min P ∈M d×k ,Q∈M |E|×k h(P, Q) = -S T , P Q T F + µ X -B -C A • QP T 2 F (4.9)
The second term corresponds to the node loss term of equation (4.2), expressed in matricial form as X -B -C A • W T 2 F , where B ∈ R |V |×d is the node bias vectors b u stacked vertically, C ∈ R |V |×|E| is the incidence matrix of G, A ∈ R |E|×d is the matrix whose each row is the corresponding a uv 1 vector and • F denotes the Frobenius norm. As previously, we set B = 0 and the rows of A to be one over the degree of the node at the origin of the corresponding edge.

While (4.9) is not jointly convex in both P and Q, it is convex in one of them when the other is fixed. Therefore it can be solved via alternating optimization over P and Q [BH02] using standard (projected) gradient descent algorithms, with the following partial derivatives:

∂ h ∂P = S T Q + 2µA T • (((X -B) T -A T • (P Q T )C T )C)Q ∂ h ∂Q = SP + 2µA • (C T (X -B -C(A • (QP T ))))P.
Because the problem is not jointly convex, we are less concerned with the domains of the variables not being convex, as we anyway have no guarantee to find a global minimum. We also do not add a regularization term of the norm of the columns of P Q T . Indeed, since P ∈ M d×k and Q ∈ M |E|×k , each column of the resulting P Q T is a linear combination of k unit norm vectors of P with weights whose square sum to 1 and thus their norm is bounded by √ k. As opposed to all the previous methods, computing the full gradient is not linear in the number of edges. Indeed, computing the product S T Q requires O(|E| ω ) arithmetic operations, where ω ≥ 2 is the optimal exponent of matrix multiplication algorithm. This could be mitigated by using stochastic gradient descent. Finally, to obtain a solution of the EDGE ATTRIBUTED CLUSTERING Problem 5, we take the directions to be the columns of the resulting P , and the assignment to be a k-means clustering of the rows of Q.

indeed the one achieving maximum goodness. We will explain later why this is not always the case. Our methods essentially proceeds as follow:

1) We create an appropriate number of small Erdős-Rényi subgraphs and assign to each node a set of k local directions such that most pairs of adjacent nodes have at least one directions in common (as we motivated after having introduced equation (4.3) on page 97). 2) We connect the blocks by several pairs of nodes having one direction in common, and pick for each edge (u, v) a direction index y uv ∈ k among the ones shared by its endpoint. 3) We draw the k directions at random, ensuring they respect the parameter n o . 4) Finally, we optimize the profiles so as to simultaneously maximize the edges goodness, minimize the term L node of equation (4.2) and enforces as much as possible that for every edge (u, v) ∈ E, E(u, v) = y uv . Steps 1 and 2 generate a graph topology (V, E), while steps 3 and 4 generates directions and profiles (D , X). Note however that step 3 is independent of the steps 1 and 2, while step 4 is dependent on all the previous steps. We now describe each of these steps in more details. 1) We first create small Erdős-Rényi subgraphs [ER59; Gil59] that we call blocks.

Then we assign to each node a k local -tuple of directions such that two adjacent nodes have at least one direction in common. For convenience, let us first identify directions with colors and thus call such a k local -tuple of directions a palette. For a given palette p, we call adjacent palettes, denoted adj(p), all the palettes different from p9 but sharing one color with p. Finally, we say that an edge u, v is colorless if the palette of u and v have no color in common. Because in the general case, it is not always possible to assign a palette to every node such that there is no colorless edge,10 we now describe a simple heuristic. It performs a breath first visit of a subgraph, starting from a random node. Upon visiting an uncolored node u, it builds a set P of palette respecting the colouring constraint. Namely, for every colored neighbor v of u, it retrieves the adjacen palettes of v's palette and builds their intersection:

P = v∈N (u) adj(p v ) .
Then it selects uniformly at random a palette from P , or an arbitrary palette if P is empty: see Figure 4.2 for a small illustration. Once all nodes have been colored, we count the number of edges that are colorless. We repeat this procedure, keeping track of the palettes assignment minimizing the number of colorless edges.

2) Once we colored the nodes of every block, we look at pair of blocks, build a list of all edges between blocks that are not colorless and sample from it to connect blocks. The last step is to a assign a color y uv to each edge (u, v) from the shared color of its endpoint (or an arbitrary color for colorless edges). We note that this is reminiscent of the stochastic block model, although the probabilities of edges between blocks are not uniform, for they depend of the palette assigned to every node. .2 -A small example of the node palette assignment, with k = 4 colors (blue, green, red and orange) and palettes of size k local = 3. We assume nodes 1 and 2 have already been visited, and got assigned the palette (blue, green, red) and (blue, green, orange) respectively. Moreover, we are currently visiting node 3 while node 4 is yet uncolored. In that case, there are two possible palettes for node 3. If we select (green, red, orange), a possible color assignment for the edges (1, 2), (1, 3) and (2, 3) is respectively blue, red, and green.

3) We then generate k sparse, unit-norm directions D k = {w 1 , . . . , w k }, independently of the graph. An underlying assumption of our model is that each direction should provide an explanation of why two nodes are connected that is markedly different from the explanation of another direction. A way to achieve that is to create directions with only a few non-zeros components, and ensure that the non-zero components of one direction do not appear at the same position as the non-zero components of another direction. Therefore, we control through n o the number of dimensions where more than one non-zero component exists across the k directions. That is, when n o = 0, each direction has exactly k /d nonzero components that are disjoint. 11 On the other hand, if for instance n o = 5, then 5 of the d dimensions will have two directions with non-zero components on it. Those non-zero components are drawn uniformly at random from [-1, 1] and each direction is then normalized. For instance, with d = 6, k = 3 and n o = 2, we could generate the following directions, where the blue non-zero components are "overlapping" with the black ones:

w 1 w 2 w 3         0.729 0 0 0 0.450 -0.516                 0 0 -0.483 -0.533 0.694 0                 0 -0.639 -0.769 0 0 0        
4) Returning to directions instead of colors, we have a direction w yuv ∈ D k assigned to every edge (u, v). For easy of notation, in that paragraph, we will let w uv = w yuv . The final step in generating our node-attributed graph is to find a set of user profiles X = {x u } u∈V that maximizes the edge goodness and minimizes the node loss term, defined respectively as: where for simplicity, we fix as before b u = 0 and a uv = 1 deg(u) . Furthermore, to ensure that the direction assigned to any edge (u, v) achieves a higher goodness than the other directions (that is ∀w ∈ D k \ {w uv } , g(s uv , w uv ) ≥ g(s uv , w )), we also minimize a cross-entropy loss commonly used in non-binary classification problem [Bis06, Section 4.3.4]. Specifically, denoting by p uv, = exp(g(suv,wuv)) k =1 exp(g(suv,w )) the softmax "probability" of s uv being explained by direction , the cross-entropy loss is defined by:

L edge = u,v∈E
L cross-entropy = - u,v∈E k l=1 I {w uv = w } p uv, . (4

Results

In this section, we start by describing how we generate test instances of Problem 5 for the five methods we want to compare. Then we present two evaluation measures: how well those methods classify edges, and how well they recover known directions. We use the first measure to comment on the quality of the generated instances. Finally we present the results of the five methods in Table 4.4 and Table 4.5.

Our test instances are generated as follow. We first choose n = 500, and generate once a graph topology with directions assigned to all edges according to steps 1 and 2. This results in a graph G with roughly 1 350 edges. We then generate the directions D k according to step 3 and the following choice of parameters: d = 35, k = 7, k local = 3 and n o = 0. Finally, we create the profiles X as described in step 4. To account for the randomness of this generation process, we repeat our measurement over 200 such generations of D k and X. On the other hand, we verify experimentally that generating various graphs while keeping D k fixed gives the same results, hence we will not report them in the main text.

In the rest of this section, we will refer to the set of parameters d = 35, k = 7, k local = 3 and n o = 0 as the default configuration. We also try other configurations to study the parameter sensibility of our methods. Specifically, we experiment with less directions (k = 5) or more directions (k = 9, d = 36)13 , with two higher levels of overlap between the directions (n o = 6 and n o = 12), with more directions per node (k local = 4) and in larger dimension (d = 77).

Let us now describe how we evaluate the results of our methods on such instances. As said at the beginning of this section, we extract from the output of each method a set D k = { w 1 , . . . , w k } of k directions, and assign a direction to an edge (u, v) as E(u, v) = arg max ∈ k g(s uv , w ). Recall that during the second of the step of the instance generation, we assigned a direction index y uv to every edge. However, we cannot simply evaluate our predictions by simply comparing E(u, v) and y uv . Indeed, here is no guarantee that the D k directions are recovered in the same order as they were generated. In case of strong disagreement between the predictions and ground truth labels, it might additionally be difficult to find a permutation to conciliate them. Therefore, we see the problem as a clustering one with known ground truth, and turn to the Adjusted Mutual Information (AMI) [START_REF] Vinh | Information theoretic measures for clusterings comparison: is a correction for chance necessary?[END_REF]. It is an information theoretic measure that enjoys several properties: it is invariant to permutations of the labels, invariant to the shape of clusters and it is bounded between 0 and 1, where an AMI of 1 indicates perfect correlation with the ground truth labels and 0 indicates perfect independence with the ground truth labels.

Besides the AMI score, we also measure how far the recovered directions D k are from the actual ones, generated initially. More precisely, we associate each vector w ∈ D k with the closest one in D k and report the average 2 distance between the two elements of these k pairs, that is

d(D k , D k ) = 1 k ∈ k min w i ∈ D k w -w i 2
Before moving to the results, let us first note that, in the default configuration, we do not always generate user profiles leading to a perfect assignment of edge according to their goodness. This can be seen in Table 4.3. In general, this is not always possible. Consider a k-regular subgraph, where the k edges incident to every node are each assigned a different directions. By symmetry, it is impossible to find node profiles that would achieve maximum goodness for all the edges. This is partly what motivated the k local local constraint. Even so, and more pragmatically, whereas those AMI scores could be made higher by increasing the coefficient λ in (4.11), this would imply a lower edge goodness. With our choice of λ, only a few percent of the edges have a mismatch between the directions we assigned them and the one with the largest goodness. Note also that depending of the parameters of the generation, the AMI is not the same. Therefore, in the following, we divide all the AMI scores by this natural score, in order to obtain a "standardized" measure. As showed in Table 4.4, COMBINED is always the best method and, excluding EXPLICIT for now, the LLOYD heuristic is second except in one case, while k-MEANS is third. Note that although the differences are not large in absolute value, they are generally statistically significant. Coming back to EXPLICIT, not only does it have the same performance as COMBINED, but we verified that it actually returns the same predictions, up to a few edges that are classified differently. This can be explained because they optimize the same objective and EXPLICIT is initialized with the solution from COMBINED. However, we also verified that both edge score and node loss are respectively higher and lower at the end of the optimization. Finally, the results of FRANK-WOLFE are much worse than all the others, despite the optimization leading to edge scores comparable to the other methods. This suggests that the subsequent clustering is not adequate. Regarding the generative parameters, and compared with the default configuration, the problem is easier when the k directions are spread in a larger number of dimensions, and more difficult when each nodes is involved in four directions instead of three. On the other hand, whereas we expect the performance to decrease with increase in the overall number of directions or their overlap, there is not consistent evidence of that.

Table 4.4 -Standardized AMI of 5 methods, when generating directions with 7 configurations. For each configuration, we generate directions 200 times, and report the mean standardized AMI along with the standard deviation. Among the first three methods (that do not use matrix formulation), we highlight the best one in bold brown and the second best one in italic red. When the difference between a score and the next best one is statistically significant (i.e. when we can confidently reject the hypothesis that the two distributions have the same mean), we furthermore show in parenthesis the supporting p-value of a paired Student's t-test. Similar conclusions carry out when we evaluate methods according to their ability of recovering the original directions. Keeping in mind that in the case of perfect recovery, d(D k , D k ) would be equal to zero, while the distance between two d-dimensional unit vectors is √ 2 in expectation14 , we see in Table 4.5 that no method gets very close to the original directions. COMBINED is again the closer overall, but now k-MEANS and LLOYD are alternating at the second place. The fact D k cannot be completely recover is not surprising, for in absence of prior information the problem is under-constrained. Indeed, while COMBINED and EXPLICIT deliver almost the exact same prediction, their directions are clearly different.

Related work

Whereas we believe we are the first to study Problem 5, the prevalence of multilayer graphs in real world applications, as well as the wealth of information provided by nodes attributes has attracted a lot of attention. We start by presenting methods to classify edges, noting they either require supervision or do not consider node attributes. We then discuss works predicting relations in knowledge graphs, and more generally embedding edges in low dimensional spaces. Another approach we highlight includes combining topology and subspace clustering. We also note how ideas such as heterophily, learning k ways to weight edge's endpoint similarity and distinguishing a superposition of layers have been used in different contexts. Finally, we conclude by the more distant but very rich topic of generative model for attributed graphs and how it enables overlapping node clustering. The first one is based on a graphical model where each node is an edge. The label of an edge is influenced by the attributes of its endpoints, the correlation between the type of neighboring edges and some global constraints. These three influences are modeled by pairwise factors. The parameters of the model are learned by maximizing the likelihood of the observed labels through gradient descent. Those parameters are then use to predict the remaining labels that further maximize the total likelihood [START_REF] Tang | Learning to infer social ties in large networks[END_REF]. An extension of this graphical model is to actively query the most informative edges, in the case where it is costly to ask social network users to label their relationships [START_REF] Zhuang | Actively learning to infer social ties[END_REF]. Another way to leverage additional supervision is to use transfer learning [START_REF] Tang | Transfer learning to infer social ties across heterogeneous networks[END_REF]. Namely, given two distinct and partially labeled graphs, where the fraction of labelled edges is much higher for the first graph than the second one, the goal is to fully label the edges of the second graph, exploiting the rich information of the first one. This requires the attributes to be comparable between the two graphs, and such attributes are therefore computed based on topological features. Finally, a last direct approach to labelling edges is presented by Aggarwal et al. [START_REF] Aggarwal | Edge classification in networks[END_REF]. They consider that the input graph has three types of edges labeled by -1, labeled by +1 or unlabeled. In each of the three induced subgraphs, they define a similarity between two nodes u and v as the Jaccard coefficient of the two sets of edges incident to u and v. A weighted combination of these three coefficients is the final similarity between u and v. Then, S k (u) are the top-k most similar nodes to u, and the label of the edge (u, v) is chosen as the majority label in (S k (u) × S k (v)) E. All these works assign a label to the edges of the input graph, but they either require supervision or they do not use attributes, and in both cases they provide little interpretability over their results.

Edge classification

Prediction in knowledge graphs Besides social networks, the task of predicting the semantic of an edge has also been addressed in the context of knowledge graphs. In such graphs, nodes are abstract concepts and concrete entities from the real world, and edges are directed predicates representing facts connecting two entities [START_REF] Nickel | A review of relational machine learning for knowledge graphs[END_REF]. A typical example is an edge (h, , t) = (Donald J. Trump, President of, USA) 15 where the (h, , t) notation stands respectively for head, type of relation and tail. This illustrates some differences with our setting. First, there might be a very large number of types of edge in knowledge graphs, since they cover a domain as large as possible. Second, this coverage makes it difficult to describe directly entities by a consistent set of d attributes, and one usually has to rely on low dimensional embeddings. [START_REF] Bordes | Translating embeddings for modeling multi-relational data[END_REF] is primarily concerned with learning such embeddings in R k for both nodes and relations, that are denoted by the boldface letters h, , t. Similarly to word embedding, the intuition is that relations can be modeled by translations. Thus, the existence of an (h, l, t) edge offers evidence that h + ≈ t. Given a training set of existing triplets S and a dissimilarity measure D, the authors learn an embedding in a fashion reminiscent of metric learning [START_REF] Bellet | Metric learning[END_REF]. Namely, they create negative examples by corrupting at random training examples and then minimize with a margin the positive part of the difference between dissimilarity of positive and negative examples. More generally, we refer the interested reader to a recent survey on embedding knowledge graphs [START_REF] Wang | Knowledge graph embedding: a survey of approaches and applications[END_REF], which among other ideas present the interesting notion of composing relations along paths between two entities. We note that, in contrast with us, those methods require some form of supervision when it comes to predict edge types. One way to avoid supervision is to rely on a large amount of unstructured text, extract entities from this corpus, map entities and possible relations to a knowledge base and finally embed them in a low dimensional space using text features such as POS tags [START_REF] Ren | Cotype: joint extraction of typed entities and relations with knowledge bases[END_REF].

Edge embedding A related idea is embedding edges in general graphs, not necessarily in the context of knowledge graphs. This is the counterpart of node embedding techniques we mentioned in Section 2.4. The most straightforward way is to keep track of edges instead of nodes when performing random walks, learn an embedding from this corpus using word2vec-like methods and then cluster these representations using k-means [START_REF] Li | Edge representation learning for community detection in large scale information networks[END_REF]. We cannot directly compare this with our work since there is no attribute and thus the clustering is not interpretable (indeed the goal of [START_REF] Li | Edge representation learning for community detection in large scale information networks[END_REF] is eventually to find an overlapping clustering of the nodes, not of the edges). More closely related to our work, Ahmed et al. [START_REF] Ahmed | Edge role discovery via higher-order structures[END_REF] seek to assign a role to the edges of a graph. First they learn edge features as follow: starting from topological features, they use combination operators (like max, min, mean, sum, product and so on) to iteratively learn higher order features while pruning those that are correlated in order to avoid a combinatorial explosion. Once they obtain such a d-dimensional feature vector for each edge, they stack them in a matrix S ∈ R |E|×d and look for U ∈ R |E|×r and V ∈ R r×d such that the distance between S and U V T is minimized. V is a description of r roles in terms of features while U is the mixed membership of each edge. Finally the authors choose the number of roles r using the Minimum Description Length principle. Their approach bears some similarities with our EXPLICIT method, but have the additional properties of scaling linearly with the number of edges and being partially parallelizable. However, the process in which edge feature are learned does not lend itself easily to interpretation.

Subspace clustering Another direction is to see the problem as clustering the similarity vectors s uv , like our approaches in Section 4.2.1. Because of our bias that nodes are connected through partial homophily and heterophily, this can more precisely be casted as a subspace clustering problem [START_REF] Vidal | Subspace clustering[END_REF]. Namely, we are given a set of m points in R d , with the assumption they belong to the union of affine subspaces of unknown dimension. The goal is to recover the number K of such subspaces, their dimension, their parameters and the assignment of the m point to those subspaces. If K is known, one can use iterative, k-mean like methods (reminiscent of our LLOYD method). Otherwise, it also possible to factorize the matrix of all the data points and interpret it as a similarity matrix, to use iterative statistical approaches such as random sample consensus or to build a similarity graph and apply spectral clustering algorithms to it. Finally, it is also possible to leverage sparsity assumptions and express each point as a sparse linear combination of other points, and use these coefficients as weights of a similarity graph, which is then clustered into subspaces by a spectral method [START_REF] Elhamifar | Sparse subspace clustering: algorithm, theory, and applications[END_REF]. While subspace clustering comes with information theoretic guarantees of its ability to retrieve optimal subspaces, when formulated directly over the {s uv } (u,v)∈E , it makes no use of graph topology, as we consider each edge independently of all others. As an example of applying related ideas to graph data, Huang et al. [START_REF] Huang | Dense community detection in multi-valued attributed networks[END_REF] build a d-dimensional grid of the attributes space and use subspace clustering to find cells that have low entropy and high connectivity, before merging such cells into clusters of nodes.

Further similar ideas Taskar et al. [START_REF] Taskar | Link prediction in relational data[END_REF] represent a node-attributed graph with partially labelled edges as a probabilistic graphical model (called Relational Markov Model) and learns its parameters from the data using gradient descent. It is a flexible modeling approach that let the designer express complex graph patterns. Using a slightly less principled approach, Stattner et al. [START_REF] Stattner | Social-based conceptual links: conceptual analysis applied to social networks[END_REF] first cluster nodes based solely on their attributes and then count the links between such groups to keep only the maximally frequent ones, arguing this reveals the sets of attributes that support the connection between two nodes. In the same data mining vein, and related to our departure from global homophily, Liang et al. [START_REF] Liang | Mining social ties beyond homophily[END_REF] look for link patterns whose support and confidence are not only above some given thresholds, but that also diverge from homophily. Indeed, the authors explicitly exclude attributes taking the same value within such link patterns. The objective of Zhang et al. [START_REF] Zhang | Community detection in networks with node features[END_REF] is eventually to hard cluster the nodes into communities but their work shares similarities with ours. Namely, they tweak the traditional modularity objective (maximizing the density of intra community edges) by weighting the edges with the similarity of their endpoints, and simultaneously learn a weight vector w for a community C . Formally they defined the R criterion as :

R(C, β; α, µ) = k =1 1 |C | α u,v∈C
A uv µe -g(s(u,v),w )

where A is the adjacency matrix, α penalizes unbalanced communities, µ is a trade off parameter between information from edges and node features, and g corresponds to our goodness function while s is an arbitrary similarity function between nodes. They optimize it by alternately optimizing over the labels with fixed parameters and over the parameters with fixed labels, using block coordinate descent. Finally, we mention a more quantitative work. Abraham et al. [START_REF] Abraham | Low-distortion inference of latent similarities from a multiplex social network[END_REF] assume that there are K social categories modelled by K Euclidean spaces D i . Nodes of the graph have an associated point in each of these spaces. The key assumption is that space have small local correlation: informally, the intersection of any two small balls from two distinct spaces is small. These K spaces give rise to small world networks G i , where the edge probability is proportional to D i (u, v) -d and we observe the real network G = i G i . From G, the proposed algorithm recovers in O(n polylog n) time a bounded approximation D i of all D i , that is there exists positive constants σ, δ and ∆ such that σD i (u, v) ≤ D i (u, v) ≤ δD i (u, v) + ∆.

Generative models and node clustering Although there has not been so many works on clustering edges in node attributed networks, there is a wealth of papers on clustering nodes in such networks. Since this is not exactly our topic, we only mention a selection of the most relevant to our objective (especially the overlapping ones), and refer the interested reader to the survey of Bothorel et al. [START_REF] Bothorel | Clustering attributed graphs: models, measures and methods[END_REF] 16 . We also reiterate the warning given in the introduction of this chapter. Namely, in large graphs, node attributes are not necessarily aligned with annotated communities. Such attributes thus need to be used in addition to topological information. This is further covered in [START_REF] Fortunato | Community detection in networks: a user guide[END_REF]Section 3.4].

An early attempt is the SA-Cluster algorithm of Zhou et al. [START_REF] Zhou | Graph clustering based on structural/attribute similarities[END_REF] that partitions the nodes of a graph based on a distance combining nodes structural and attributes similarities, along with its faster incremental version [START_REF]Clustering large attributed graphs: an efficient incremental approach[END_REF]. Nodes in the same cluster are well connected and have a set of similar attributes. The balance between these two objectives was further studied in a setting where it is tunable by the user of the algorithm [START_REF] Baroni | Efficiently clustering very large attributed graphs[END_REF]. In a more "data mining fashion", the same problem can also be worded as finding subgraphs induced by a set of attributes that are more dense than what one would expect in a null model [START_REF] Silva | Mining attribute-structure correlated patterns in large attributed graphs[END_REF].

Instead of detecting all such subgraphs or communities, one can adopt a querybased approach. Namely, given a query node u and a set S of attributes, find all the subgraphs containing u that are both tightly connected and share enough common attributes [START_REF] Fang | Effective community search for large attributed graphs[END_REF]. In that spirit of focusing on a given node, Leskovec et al.

[LM12] introduce a generative model for ego networks in social networks where the neighbors of a node u can belong to k categories (such as family, colleagues, school friends). Using our notations, those k categories (or circle) {C i } k i=1 of a node u are defined by a vector w i and a weight α i . The probability p(e) of an edge e = (u, v) favors the presence of edges with high goodness within circles, as expressed by where s is an arbitrary similarity function between two node profiles. The unsupervised problem of maximizing the likelihood of observing the input graph according to this probability distribution is solved by alternating two steps: assigning nodes to circles and optimizing the parameters {w i , α i } k i=1 given a circle assignment. Similar to us, the vector w i explain why some nodes belong to the i th circle. However, it is not obvious how to transfer this knowledge to the edges themselves.

In fact, this generative model approach has proved very fruitful when it comes to community detection in node attributed graphs [Xu+14a; ZDB17; YML13; Kat+16; WF16; NC16]. The general idea is the following. First, design a model to generate some of the following aspects: the nodes attributes, the topology of the graph and the community membership of the nodes. Then, infer the parameters that maximize the likelihood of observing the current graph. Finally, extract from these parameters community membership for every nodes. We give a very succinct description of the generative models of some representative recent works in Table 4.6. Seen at the light of our Problem 5, given the overlapping membership obtained by inference of a generative model, and an edge (u, v), one could look at the highest shared community coefficient between u and v and use that to explain the edge (among the K(K+1) 2 possible edge types induced by the K communities and their K(K-1) 2 pairs). However, this is clearly an ad-hod post processing, as indeed these works are concerned with nodes and not edges clustering.

We also point to the work of De Bacco et al. [De +17], which despite not considering node attributes, present a model with interesting applications. Specifically, the nodes have mixed membership to K overlapping groups and each of the L layers is generated by a specific K × K block matrix, taking into account the group membership, like in a degree corrected SBM. Unlike us, this allows for multigraphs, although the authors assume for simplicity that this does not happen. Once its parameters are found, this model can then be used to predict the existence of extra edges in each layer, which can be seen as edge type prediction. Furthermore, measuring the extent to which one layer helps predict links in another layer provides a way to measure the relationships between the layers, from redundancy to complete independence, allowing information compression.

Open directions

We could extend our approach to more general types of graphs and tasks:

• First, as we mentioned earlier, networks are dynamic and evolve over time, with nodes and edges constantly appearing and disappearing [START_REF] Aggarwal | Evolutionary network analysis: a survey[END_REF]. Slicing time into successive periods is one way to create edge types, one per period. communities. Beyond predicting the relative order of links, it would also be interesting to include time into node similarity itself. For instance, while the photo similarity between two Flickr users rises before they connect, it later decreases if they have the same level of popularity, as both try to differentiate themselves [START_REF] Zeng | Social ties and user content generation: evidence from flickr[END_REF]. More generally, by defining our problem on a static snapshot of a graph, our model is agnostic to the question of whether edges were created because of the similarity/dissimilarity of the linked nodes' attributes or because attributes started to change after the edge creation. This is traditionally framed as dichotomy between homophily and contagion, and is an active topic of research in social networks analysis [AKM08; ST11] as noted by Golder et al. [START_REF] Golder | Digital footprints: opportunities and challenges for online social research[END_REF]: "Homophily refers to a variety of selection mechanisms by which a social tie is more likely between individuals with similar attributes and environmental exposures [START_REF] Mcpherson | Birds of a feather: homophily in social networks[END_REF]. Contagion refers to influence mechanisms (e.g. imitation or peer pressure) by which traits diffuse along network edges. Homophily and contagion offer competing explanations for network autocorrelation, which refers to the greater similarity in the attributes of closely connected nodes."

• Another rich class of graphs where predicting edge types would be a challenging and rewarding task is multigraph, which allow several edges between two nodes [START_REF] Berlingerio | Foundations of multidimensional network analysis[END_REF]. It can also be seen as the flattening of several graphs sharing the same nodeset, but where the provenance of each edge would have been lost and need to be recovered. With our original formulation of goodness as s uv T w uv , only one direction can achieve the higher score (discarding the rare case of tie). One alternative we proposed was to let each w uv be a linear combination of a small number of base directions. Symmetrically, another idea could be to have more than one way to compute the similarity s uv between two nodes.

• Generalizing the balance theory from signed graphs, we could imagine that in triangles or short cycles, only certain combinations of directions are valid (or at least desirable). Interestingly, this would likely require different optimization algorithms, probably non convex. However, while this is a general concept, its exact implementation might be application-dependant. Furthermore, in our current setting, we do not know in advance the semantic of each direction, nor do we have labeled edges. Therefore, it is not clear how those constraints would be specified, or whether they can be learned, if applicable. A limited solution exists in the context of heterogeneous information networks, where in addition to having several types of edges, there are also several types of nodes. Since not all types of node can be connected with any types of edge, the possible connections are themselves represented as a meta graph called the network schema [Shi+17, Definition 3]. This approach would need to be extended to deal with higher order patterns, for instance in a manner reminiscent of composition of paths in multilayer graphs [Sun+11; Wan+17b, Section 4.2].

• Among tasks that could benefit from labeling the edge of complex networks, we mention two. The first is graph summarization where the idea is to transmit a fraction of a large graph plus additional information that can be used to reconstruct the original graph. Namely, our goal here is to send only a few node profiles along with the edge directions, enabling the missing profiles to be reconstructed from the known goodness values. Note that we spare information regarding the profiles but still transmit the full topology. Therefore, this differs from lossless schemes [Chi+09; Bol+11] that leverage typical edge locality patterns to encode edges using as few bits as possible, or from lossy graph sketching methods [START_REF] Ahn | Graph sketches: sparsification, spanners, and subgraphs[END_REF] discussed in Section 3.3.2, which preserve spectral and topological properties of the original graph. The second is node embedding, where every node u is represented by a vector of size k containing the relative proportion of each direction with the edges incident to u.

Chapter 5

Conclusion

Contributions

The objective of this thesis was to provide "efficient and accurate methods to predict edge type in complex networks". By complex networks, we mean graphs where edges can have one among k semantics (or types) and where nodes are connected by both partial homophily and heterophily.

• In Chapter 2 [Le +17], we first focused on directed signed graphs, as they model social interactions in several domains, such as e-commerce and collaborations. In such graphs, edges have two possible semantics, positive or negative. Our goal was to solve a supervised binary classification problem by predicting the sign of test edges. For that, we first introduced a simple sign generative model. It endows nodes with two parameters -trollness and unpleasantness-, governing their incoming and outgoing behavior. This might seem like a simplistic way of expressing the notion of partial homophily and heterophily. However, we presented two methods to approximate this Bayes predictor, suggesting a trade off between theoretical guarantees and input requirement. We showed experimentally that those methods are competitive with the state of the art, while being more scalable. Motivated by the regularity measure offered by trollness and unpleasantness, and the inability of existing methods to cope with new nodes joining the graph, we furthermore developed the first online algorithm for the EDGE SIGN PREDICTION problem.

• We started Chapter 3 by showing that the methods of the previous chapter are not straightforward to extend to undirected signed graphs. Therefore, we changed our learning bias from a generative model to a clustering assumption inspired by social balance theories. Namely, we posit that the nodes are assigned to one of K groups, and that the sign of the edge (u, v) is positive if the nodes u and v belong to the same group and negative otherwise. We then presented two approaches to leverage this bias. First, we drew a connection between the EDGE SIGN PREDICTION problem and the CORRELATION CLUS-TERING problem. While CORRELATION CLUSTERING is hard to approximate in general, we suggested, based on existing works, that the more an instance complies with our bias, the better CORRELATION CLUSTERING methods could recover the K underlying groups and thus provide accurate prediction. Second, we addressed the problem in an active setting, where K = 2. For that, we implemented a promising spanning tree algorithm [START_REF] Vitale | [END_REF], that queries some edges in order to connect the remaining edges by paths as short as possible. Experimentally, we showed that on both synthetic and real graphs, our method is competitive in terms of sign prediction with a strong natural baseline.

• Finally, in Chapter 4, we considered the case where there are more than two types of edge. This means that instead of signed graphs, our input shifted to multilayer graphs, which are also ubiquitous in describing real world phenomena. Moreover, we modified the learning setting, so that no edge label are available, but instead nodes carry a profile vector. However, our goal remained to predict edge type, in what could then be described as a clustering problem with side information and interpretability requirements. In order to do so, we first defined a similarity between nodes, which accommodates both heterophily and homophily. We then formulated the problem as simultaneously finding k special vectors called directions, and assigning one of them to every edge in order to maximize a scalar goodness function. After introducing baselines oblivious of the graph topology, we added constraints resulting in one vectorial and one matricial optimization objectives. Finally, we showed on synthetic data how those methods can recover the planted ground truth.

Future work

In this section, we revisit the problems summarized above by showing how they could be applied in a new context or enriched by existing node embedding methods.

Reciprocal recommendation

We first present a task tightly related to our edge characterization problem, namely the reciprocal recommendation problem [START_REF] Pizzato | Recommending people to people: the nature of reciprocal recommenders with a case study in online dating[END_REF]. As we shall see, it indeed incorporates aspects from all the problems we discussed previously, along with new constraints. Broadly speaking, given two sets of users L and R, we want to recommend users in R to users in L, and vice versa. These recommendations are based on mutual interest, but in most cases there is an asymmetry between the two groups.

To better illustrate this general definition, we now give some concrete examples.

• One recent application that quickly rose to prominence is online dating [Krz+15; Xia+16; AB16]. In the most common case, users are divided between men and women, and we want to recommend users of the opposite gender in order to create couples. A well known example of such a system is Tinder [START_REF] Tyson | A first look at user activity on tinder[END_REF].

• In another domain, we can also match job seekers and employers [Sit+12; Hon+13; KA15].

• As a specific case in point, Zhang et al. [START_REF] Zhang | A research analytics framework-supported recommendation approach for supervisor selection[END_REF] propose a method to match prospective PhD students and their future supervisors.

• In large organizations or associations, having a mentor is usually beneficial for both employees and employers [START_REF] Allen | Career benefits associated with mentoring for proteges: a meta-analysis[END_REF] and finding mentee/mentor pairs can be cast as a reciprocal recommendation problem [START_REF] Steinmacher | Recommending mentors to software project newcomers[END_REF].

• After moving to their new jobs, people also need a place to live. Again, we can match tenants and landlords, or people to share a flat.

• Finally, in a more critical setting, finding relevant matches between patients and organ donors can improve the success of transplants [START_REF] Yoon | Personalized donor-recipient matching for organ transplantation[END_REF].

Reciprocal recommendations present specific challenges [START_REF] Andrews | Recommender systems for online dating[END_REF]. Among them are asymmetry and volume. Asymmetry refers to the imbalance between the two groups, in terms of size and bargaining power. For instance, depending of the economic situation and the field, there might be a lot of job seekers for a few open positions, in which case employers can be very selective. Likewise, it is generally observed that dating websites have a higher proportion of men, meaning that women can afford to choose among numerous suitors [START_REF] Tyson | A first look at user activity on tinder[END_REF]. By volume, we mean that users on both side can only handle a few recommendations at the same time. This is sharp contrast with traditional user-item recommendation systems, where recommending simultaneously the same item (say a movie) to thousands of users is not an issue. On the other hand, we have to keep users engaged and prevent them from being idle. This is done by ensuring they are regularly the subject or object of some recommendations.

As the name implies, reciprocal recommendation has often be seen as a special case of recommender system [START_REF] Andrews | Recommender systems for online dating[END_REF]. Besides this paradigm, we could also cast it as an adaptive, active, cost sensitive EDGE SIGN PREDICTION problem on bipartite directed signed graphs with side information. Let us decompose this last sentence, starting with the input graph. It is naturally bipartite, because of the presence of two groups in which there is no internal recommendation. Furthermore, every time one user expresses interest or disinterest in another user, this creates a directed signed edge. Such a graph is thus a special case of the Directed Signed Social Networks studied in Chapter 2. Each nodes carries two kind of (side) information. The first is explicitly provided by the user, in order to arise interest in users from the other group. The second is implicitly inferred from the user activity. In particular, the interactions with other nodes allow us to build an implicit preference model for every user. Combining these two sources of information yield a profile of each user, like in Chapter 4. Making recommendations in such a graph therefore boils down to finding non existing positive edges. However, to cope with the two challenges presented in the previous paragraph, we follow the setting of Chapter 3 and use an active approach. More precisely, respecting the volume constraints would require an adaptive algorithm, which acquires the most useful information without overor under-loading any nodes. As for the group asymmetry, this could be addressed by using a cost sensitive classification error, so that the error rate at a given node is inversely proportional to its importance in the network.

As in any worthy machine learning endeavor, reliably evaluating and comparing approaches require large amount of real data. This is particularly true in such an applied setting, where the most realistic evaluation is to perform A/B testing on a live system with enough traffic to deliver statistical significance. This explains why most previous studies rely on the cooperation of industrial partners [And15; KA15; Krz+15; Xia+15; AB16]. Thus, this is also something we would like to do in the future.

Representation learning

A fundamental question underlying many machine learning approaches is the representation of data. When extracting information from the real world, we are usually given some freedom in how to present it to learning algorithms. Ideally, we would like to use the representation providing the best performance in the learning task. One solution is to ask expert domains to craft relevant features based on their prior knowledge. While this has achieved successes, it is not scalable to the present day situation, where the deluge of available data has prompted the use of machine learning in every corner of the society. An alluring alternative to expert knowledge is to learn the data representation itself, guided by the task.

As an illustration in the context of this thesis, we assumed that data was organized as a graph, where nodes were possibly associated with a feature vector. However, there might an different representation of those nodes that would better suit our needs. Indeed, we mentioned already in Section 2.4 and Section 4.4 that a recent and promising trend regarding learning in graph is node embedding. In addition to the references cited earlier, this also witnessed by recent works on embedding attributed nodes, either in signed graphs [START_REF] Cheng | Unsupervised feature selection in signed social networks[END_REF][START_REF] Wang | Attributed signed network embedding[END_REF][START_REF] Wang | Shine: signed heterogeneous information network embedding for sentiment link prediction[END_REF] or in general graphs [Lia+17; HLH17; Li+17a]. At first sight, those methods seem radically different from traditional graph methods. However, just like word embedding methods have been linked to matrix factorization [LG14; LC14; Aro+16], so have been popular and successful node embedding methods [START_REF] Qiu | GNetwork Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec[END_REF]. These methods are the simple ones that do not consider node attributes and multiple types of edges, such as DeepWalk [START_REF] Perozzi | Deepwalk: online learning of social representations[END_REF], LINE [START_REF] Tang | Line: large-scale information network embedding[END_REF] and node2vec [START_REF] Grover | Node2vec: scalable feature learning for networks[END_REF]. For instance, Qiu et al. [START_REF] Qiu | GNetwork Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec[END_REF] show that the node embedding of DeepWalk on the weighted graph G are approximately the left singular vectors of the following matrix:

log vol(G) 1 T T r=1 (D -1 A) r D -1 -log b (5.1)
where vol(G) is the sum of all weighted degrees, T is the context window size, D is the diagonal matrix of degrees, A is the adjacency matrix of the graph and r is the number of negative samples in skip-gram. There are several ways such findings could be used in our context:

• If the edges of the input graph are fully labeled into k types, then we have k matrices D k and A k , by considering the subgraphs induced by each edge type.

If we simply decompose the k matrices constructed according to equation (5.1), we would have k embedding for each node. Instead, it seems natural to stack those matrices together to form two third order tensors D ∈ R n×n×k and A ∈ R n×n×k . One could then use one tensor decomposition methods [KB09; Sid+17] to extract node embedding taking into account the multiplicity of edge types.

• If the edges of the input graph are partially labeled, then predicting edge type can be seen as a tensor completion problem. Like in the matrix case, we seek a low rank tensor that coincides with the observed entries of the input data. Furthermore, when we have access to node profiles, we can use methods tailored to leverage such side information [Son+17, Section 4].

• If the input graph has no edge labels at all but we are given node profiles, we can use existing attributed embedding approaches [Lia+17; HLH17; Li+17a] to generate extra information about each node, before applying the methods presented in Chapter 4 on these extended profiles. We note however that doing so might reduce the interpretability of the final result.

2. 1

 1 Part of a DSSN centered on node u. . . . . . . . . . . . . . . . . . . . 2.2 Illustration of the graph transformations turning EDGE SIGN PRE-DICTION into node classification . . . . . . . . . . . . . . . . . . . . . 2.3 The node-labeling of G illustrating the relation between the edge label complexity and the cutsize. Here the four edges part of the cut are in heavy black. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 The sign y u,v of the edge u → v is positive with probability 1 2 (p u + q v ). . . 2.5 Synthetic distributions on WIKIPEDIA. . . . . . . . . . . . . . . . . . . 2.6 Results on WIKIPEDIA with BLC(tr, un) on a training set of size 40% . 3.1 A two-clustering of a complete strongly balanced graph . . . . . . . . . . 3.2 The four possible undirected triads, as classified by the two structural balance theories introduced in the main text . . . . . . . . . . . . . . . 3.3 Small example of CORRELATION CLUSTERING . . . . . . . . . . . . . 3.4 The transformation from CORRELATION CLUSTERING on G to k-MINIMUM MULTICUT on H (reproduced from [Dem+06]) . . . . . . 3.5 A positive star with few negative edges . . . . . . . . . . . . . . . . . . . 3.6 A sample star . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Cross edges between two stars . . . . . . . . . . . . . . . . . . . . . . 3.8 Unfolding stars to recover spanning trees . . . . . . . . . . . . . . . . 3.9 The hierarchical structure of stars created by GALAXY TREE . . . . . 3.10 The other iterations of GALAXY TREE . . . . . . . . . . . . . . . . . . 3.11 A line graph with stars in blue. . . . . . . . . . . . . . . . . . . . . . . 3.12 Real world pictures and their binarized version . . . . . . . . . . . . . 3.13 Stretch over graphs of increasing size . . . . . . . . . . . . . . . . . . 3.14 MCC over various graphs . . . . . . . . . . . . . . . . . . . . . . . . . 3.15 A sym exp over various graphs . . . . . . . . . . . . . . . . . . . . . . . 4.1 A small instance of EDGE ATTRIBUTED CLUSTERING and an handcrafted solution, albeit non-optimal. . . . . . . . . . . . . . . . . . . . 4.2 Finding the k local directions of node when generating synthetic graph vii List of Tables

  3.10 Dataset description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Real world attributed graphs . . . . . . . . . . . . . . . . . . . . . . . 4.2 Combining node profiles through the Hadamard product . . . . . . . 4.3 Qualite of our synthetic instances according to generation parameters 4.4 Performance of our proposed methods on synthetic instances . . . . 4.5 Same as Table 4.4, but reporting d(D k , D k ), which should be as close as possible to 0. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Generative model of node attributed graphs with community structure114 ix x

Figure 2 . 2 -

 22 Figure 2.2 -(a) A directed edge-labeled graph G. (b) Its corresponding graph G resulting from the first reduction we describe. The square nodes in G correspond to the edges in G, and carry the same labels as their corresponding edges. On the other hand, the 2|V | circle nodes in G are unlabeled. Observe that some nodes in G are isolated and thus unimportant. These are exactly the nodes in G corresponding to the nodes having in G no outgoing or no incoming edges: for instance nodes 3 and 4 in G. (c) The weighted graph resulting from the second reduction we describe.

  Figure 2.3 -The nodelabeling of G illustrating the relation between the edge label complexity and the cutsize. Here the four edges part of the cut are in heavy black.

  η foredges density of η for + edges (d) Density of η for both signs.

Figure 2 . 5 -

 25 Figure 2.5 -Synthetic distributions on WIKIPEDIA.

  Figure 3.1 -A twoclustering of a complete strongly balanced graph

Figure 3 . 2 -

 32 Figure 3.2 -The four possible undirected triads, as classified by the two structural balance theories introduced in the main text

Figure 3

 3 Figure3.3 -A small graph with eight nodes and ten edges. Solid edges represent positive edges and dashed edges represent negative edges. A clustering C is showed with 3 clusters: {1, 2, 4}, {3, 5, 6} and {7, 8}. C incurs two disagreements: the negative edge between nodes 1 and 2 within the blue cluster, and the positive edge (6, 7) between the orange and green clusters. Those disagreements are created by two cycles with one negative edge and thus cannot be avoided, meaning that C is optimal. However, it is not the unique solution: for instance, merging the orange and green clusters would also yield two disagreements.

  4 and one can show ([Dem+06, Theorem 4.4]) that it takes a linear time to construct a multicut of weight W in H G from a clustering of weight W in G, and vice versa.

Figure 3

 3 Figure 3.4 -The transformation from CORRELATION CLUSTERING on G to k-MINIMUM MULTICUT on H (reproduced from [Dem+06])

  with a careful choice of f + and f -. They also give a derandomized version of their algorithm in the full version of the paper [Cha+14, Theorem 23].

Figure

  Figure 3.5 -A positive star with few negative edges

  s 1 , s 2 )]

  UPDATE-NODES-MAPPING take O(|V 0 |) time, since they go through every node of the original graph. Because G 0 is connected, |V 0 | ≤ |E 0 |, and since for all t, |E t | ≤ |E 0 |, we have that the GALAXY TREE inner loop takes O(K|E 0 |) time. Finally, since ASSEMBLE-SPANNING-TREE goes again through every edge visited at every collapse step, it also takes O(K|E 0 |) time, which is thus the overall complexity of the GALAXY TREE algorithm.

Figure 3 . 8 -

 38 Figure 3.8 -Unfolding stars to recover spanning trees

Figure 3 . 9 -

 39 Figure3.9 -The execution of the GALAXY TREE algorithm. The original graph is made of the solid and dashed edges connecting the nodes labeled by their index. Edges forming the final spanning tree are solid while the others are dashed. Their colors indicate at which iteration they were chosen to be inside a star. The four shades of gray, from white to dark gray denote increasing node eccentricity (as computed at the end of the algorithm). The i th star created during the j th iteration of the algorithm is denoted S j i . Refer to the main text for the complete description of the execution.

  Figure 3.10 -The other iterations of GALAXY TREE

Figure 3 .

 3 Figure 3.11 -A line graph with stars in blue.

Figure 3

 3 Figure 3.12 -Real world pictures and their binarized version

  Figure 3.15 -A sym exp over various graphs

Figure 4

 4 Figure 4.1 -A small instance of EDGE ATTRIBUTED CLUSTERING and an handcrafted solution, albeit non-optimal.

√d

  and the profiles in R bep √ d . We call this instance I = (G, X, k = 2). With this choice, we have s uv = 1 d = s inner if u and v are in the same group, -1 d = s outer if u and v are in different groups. Let us denote by m i the number of edges that are internal to L and R, and by m o = |E|m i the number of edges between L and R. Proposition 4. Let D 2 = {w 1 , w 2 } be an optimal solution of Problem 5 on the instance I.

Figure 4

 4 Figure4.2 -A small example of the node palette assignment, with k = 4 colors (blue, green, red and orange) and palettes of size k local = 3. We assume nodes 1 and 2 have already been visited, and got assigned the palette (blue, green, red) and (blue, green, orange) respectively. Moreover, we are currently visiting node 3 while node 4 is yet uncolored. In that case, there are two possible palettes for node 3. If we select (green, red, orange), a possible color assignment for the edges (1, 2), (1, 3) and (2, 3) is respectively blue, red, and green.

,11

  s uv T w uv andL node = u∈V x ub u -v∈N (u)a uv w uv 2 We choose d to be a multiple of k.

  p((u, v) = e ∈ E) ∝ exp   C i ⊇u,v g(s(u, v), w i ) -C i u,v α i g(s(u, v), w i )   , (4.12)

  For instance, Mucha et al. [Muc+10] study the 110 congresses of the US senate between the years 1789 and 2008, where 1 884 individual Senators are linked by their voting similarity. They show how it improves the detection of relevant

1

  List of notations used in this thesis . . . . . . . . . . . . . . . . . . . . x 2.1 The label regularity values for the nodes of the example G graph of Figure 2.2a. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Directed Signed Social Networks dataset properties . . . . . . . . . . 2.3 Comparison of BLC(tr, un) with the Bayes optimal predictor . . . . . 2.4 MCC with increasing training set size . . . . . . . . . . . . . . . . . . 2.5 Training time on a 15% training set . . . . . . . . . . . . . . . . . . . . 2.6 MCC difference when using reciprocal edges . . . . . . . . . . . . . . 2.7 Same as Table 2.4, but when selecting the training set by the timestamp of the edges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 The effect of not sampling edges at random . . . . . . . . . . . . . . . 2.9 Values of s j (i) for i ≤ 7 and j ≤ 3. . . . . . . . . . . . . . . . . . . . . 2.10 Values of m r,c for r ≤ 7, c ≤ 4 and |E| → ∞. . . . . . . . . . . . . . . . This table is the same as in Table 2.4 on page 31, but this time on three smaller, directed biological networks. . . . . . . . . . . . . . . . 3.3 MCC on the six previous datasets with direction removed . . . . . . 3.4 Hardness results of CORRELATION CLUSTERING . . . . . . . . . . . . 3.5 Best current results on CORRELATION CLUSTERING problems . . . . 3.6 Approximation results for CORRELATION CLUSTERING on general graphs with K clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Length of the paths not in the resulting spanning tree. . . . . . . . . . 3.8 Reproduction of Table

2.11 Normalized logistic regression coefficients averaged over 12 runs (with one standard deviation) . . . . . . . . . . . . . . . . . . . . . . . 3.1 Biological dataset properties. The columns have the same meaning as in Table 2.2 on page 26 . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2

  .1, following this construction would result in a labeled version of G shown in Figure2.3. We see that by labeling circle nodes by the majority label of the square nodes there are connected to, we have that the cutsize of G (Y ) equals Ψ in (Y ) + Ψ out (Y ). 6 A similar reasoning apply in the batch case with the Ψ 2 G (Y ) measure, which can be seen as a soft quadratic version of the cutsize.

	Table 2.1 -The label regularity values for the nodes of the example G graph of
	Figure 2.2a.
	node u	1	2	3	4 total

Table 2 .

 2 2 -Dataset properties. The 5 th column gives the fraction of positive labels. The next two columns provide two different measures of label regularity, while the last two columns give the proportion of reciprocal edges, and among them the fraction with different signs.

	Dataset	|V |	|E|	|E| |V |	|E + | |E|	Ψ 2 G (Y ) |E|	Ψ G (Y ) |E|	reciprocal edges	reciprocal disagreement
	CITATIONS ADVOGATO WIKIPEDIA SLASHDOT EPINION WIK. EDITS 138 587 740 106 4 831 39 452 5 417 51 312 7 114 103 108 14.5 78.8% 8.2 72.3% 9.5 75.1% 82 140 549 202 6.7 77.4% 131 580 840 799 6.4 85.3% 5.3 87.9%	.076 .061 .063 .059 .031 .034	.191 .132 .142 .143 .074 .086	5.1% 33.6% 5.6% 17.7% 30.8% 6.5%	27.1% 28.6% 10.0% 4.0% 2.1% 14.6%

Table 2 .

 2 3 -Comparing BLC(tr, un) with the Bayes optimal on three metrics: Accuracy, MCC and ability to estimate η. The results are averaged over 25 random sampling of E 0 for each dataset and each training size, and we report one standard deviation after every number.

	Dataset	|E 0 | |E|	BLC(tr, un) Accuracy	Bayes Accuracy	BLC(tr, un) MCC	Bayes MCC	MAE(η, η) for the whole testing set	MAE(η, η) for highly sampled edges
	CITATIONS ADVOGATO WIKIPEDIA SLASHDOT EPINION	20% 66.1 ± 0.6 75.1 ± 0.1 16.2 ± 0.5 29.7 ± 0.2 40% 68.2 ± 0.4 74.7 ± 0.2 19.1 ± 0.6 29.6 ± 0.4 80% 71.4 ± 0.4 75.1 ± 0.5 23.4 ± 0.9 29.7 ± 1.1 20% 66.5 ± 0.6 75.1 ± 0.1 17.1 ± 0.6 30.3 ± 0.2 40% 69.6 ± 0.4 75.4 ± 0.1 21.0 ± 0.5 30.8 ± 0.3 80% 71.6 ± 0.5 75.4 ± 0.4 23.5 ± 0.9 30.5 ± 1.0 20% 70.5 ± 0.4 75.3 ± 0.1 21.8 ± 0.4 30.4 ± 0.2 40% 72.3 ± 0.2 75.3 ± 0.1 25.3 ± 0.5 30.7 ± 0.3 80% 73.3 ± 0.3 75.3 ± 0.3 28.0 ± 0.6 30.9 ± 0.6 20% 66.9 ± 0.3 75.2 ± 0.0 18.3 ± 0.3 30.2 ± 0.1 40% 69.2 ± 0.2 75.2 ± 0.0 20.6 ± 0.2 30.1 ± 0.1 80% 70.9 ± 0.2 75.1 ± 0.1 22.8 ± 0.3 30.1 ± 0.2 20% 67.9 ± 0.3 75.2 ± 0.0 19.5 ± 0.2 30.4 ± 0.1 40% 69.7 ± 0.2 75.1 ± 0.0 21.2 ± 0.3 30.0 ± 0.1 80% 71.2 ± 0.1 75.2 ± 0.1 23.5 ± 0.2 30.3 ± 0.2	.256 ± .00 .205 ± .00 .152 ± .00 .242 ± .00 .192 ± .00 .142 ± .00 .169 ± .00 .122 ± .00 .090 ± .00 .224 ± .00 .182 ± .00 .144 ± .00 .194 ± .00 .159 ± .00 .128 ± .00	.119 ± .01 .085 ± .00 .059 ± .00 .116 ± .00 .083 ± .00 .060 ± .00 .108 ± .00 .075 ± .00 .054 ± .00 .098 ± .00 .068 ± .00 .048 ± .00 .089 ± .00 .063 ± .00 .044 ± .00

WIK. EDITS 20% 67.8 ± 0.3 75.3 ± 0.0 19.2 ± 0.2 30.5 ± 0.1 .221 ± .00 .097 ± .00 40% 69.6 ± 0.2 75.2 ± 0.0 21.4 ± 0.2 30.3 ± 0.1 .180 ± .00 .067 ± .00 80% 71.1 ± 0.2 75.3 ± 0.1 23.7 ± 0.2 30.4 ± 0.2 .144 ± .00 .047 ± .00

  While we show those patterns on WIKIPEDIA with a 40% training size, they are consistent across datasets and training size.

		0.95	.127	.126	.131	.126	.133		.125	.127	.120	.118	.111		0.16				Mean Absolute Error
		0.85	.124	.123	.133	.126	.132		.127	.126	.122	.119	.112		0.15
		0.75	.128	.143	.136	.133	.143		.133	.133	.129	.126	.115
		0.65	.130	.127	.135	.130	.126		.132	.123	.118	.116	.109		0.14
	q	0.55 0.45	.130 .132	.136 .133	.143 .134	.141 .130	.146 .131		.133 .136	.130 .131	.129 .117	.127 .123	.118 .114	MAE	0.13
		0.35	.133	.133	.134	.122	.128		.131	.124	.119	.113	.106		0.12
		0.25	.131	.147	.135	.138	.137		.138	.133	.136	.123	.122
		0.15	.133	.139	.157	.138	.153		.144	.139	.138	.130	.134		0.11
		0.05	0.05 .145	0.15 .145	0.25 .154	0.35 .144	0.45 .157	p	0.55 .146	0.65 .142	0.75 .141	0.85 .143	0.95 .128		0.10	8	9	10 log number of edges	11	12
		0.95													4
		0.85
															2
		0.75
		0.65 0.55 0.45 0.35												100 times the accuracy	-4 -2 0
		0.25													-6
		0.15
		0.05													-8
			0.05	0.15	0.25	0.35	0.45	p	0.55	0.65	0.75	0.85	0.95		-0.4	-0.2		0.0 2 -p+q 1 2	0.2	0.4

p+q 2 = 1 2 , for in that region, both predictors can only rely on random predictions. (a) Mean absolute error (MAE) of the η(u, v) estimation. (b) MAE in each cell as a function of the number of sampled edges. q -.297 -3.79 -5.53 -5.60 -4.77 -4.72 -4.33 -3.07 -1.96 -1.21 -4.00 4.36 -1.91 -4.81 -5.58 -5.09 -3.53 -2.71 -2.09 -1.33 -4.02 -6.82 -2.08 -2.82 -7.22 -5.83 -4.65 -3.77 -3.08 -1.65 -2.45 -6.74 -6.98 -6.34 -4.45 -5.53 -3.69 -3.23 -3.32 -1.69 -3.66 -.617 -3.24 -1.47 -5.94 -.915 -2.05 -4.51 -3.68 -3.22 -4.02 -4.15 -3.13 -4.94 -4.13 -1.18 -2.60 -5.72 -5.16 -3.63 -2.25 -3.10 -5.14 -5.03 -3.79 -4.36 -1.57 -2.65 -2.29 -3.77 -2.00 -4.65 -4.35 -2.68 -5.85 -4.63 .982 -6.10 -1.66 -4.21 -.965 -3.38 -4.89 -3.25 -4.19 -7.61 -5.50 -5.88 -.955 -2.66 -3.10 -3.58 -1.89 -3.67 -8.35 -4.05 -7.37 -3.86 -8.00 -2.99 (c) 100 time the difference of accuracy between BLC(tr, un) and Bayes. (d) Difference of accuracy between BLC(tr, un) and Bayes as p+q 2 moves away from 1 2 . Figure 2.6 -Results on WIKIPEDIA with BLC(tr, un) on a training set of size 40%

Table 2

 2

.4 -MCC with increasing training set size, with one standard deviation over 12 random sampling of E 0 . The last five columns refer to the methods we took from the literature. For the sake of readability, we multiplied all MCC values by 100. The best number in each row is highlighted in bold brown and the second one in italic red. If the difference is statistically significant (p-value of a paired Student's t-test less than 0.005), the best score is underlined. |E 0 | |E| L. PROP. BLC(tr, un) LOGREG UNREG. LOWRANK MAXNORM 16 TRIADS RANKNODES BAYESIAN

  .4.

	Dataset	L. PROP.	BLC(tr, un)	LOGREG	UNREG.	LOWRANK	MAXNORM	16 TRIADS	RANKNODES	BAYESIAN
	CITATIONS ADVOGATO WIKIPEDIA SLASHDOT EPINION WIK. EDITS	19,2 21,4 42,2 645 1 447 897	0,6 0,8 1,7 8,2 10,6 9,6	3,6 5,7 8,0 42,9 72,5 60,2	2 827 5 538 10 613 78 658 142 507 208 676	3 222 4 094 8 277 67 454 127 641 125 884	23 229 37 440 59 992 350 742 692 560 632 663	7,5 10,9 17,7 139 234 188	157 224 255 2 446 3 381 4 233	4 787 6 597 12 583 68 008 111 435 95 374

Table 2

 2

	.9) for examples)
	a i	a a j	0	1	2	3
	1	1	1	1	1
	2	1	2	3	4
	3	1	3	6	10
	4	1	4	10 20
	5	1	5	15 35
	6	1	6	21 56
	7	1	7	28 84

  Table 2.10 -Values of m r,c for r ≤ 7, c ≤ 4 and |E| → ∞.

	/2 5 6/2 5 4/2 5
	6 1/2 6 5/2 6 10/2 6 10/2 6
	7 1/2 7 6/2 7 15/2 7 20/2 7

Table 2 .

 2 11 -Normalized logistic regression coefficients averaged over 12 runs (with one standard deviation)In the previous chapter, we presented a learning bias for the EDGE SIGN PRE-DICTION problem, namely our sign generative model. By leveraging its rigorous theoretical guarantees, we have demonstrated empirically its good performance on real Directed Signed Social Networks. However, in this chapter, we consider other kinds of signed networks and show they require another bias and different,

	|E 0 | |E|	w 2	τ

CITATIONS

5% 0.929 ± 0.05 0.637 ± 0.04 10% 0.980 ± 0.02 0.700 ± 0.02 15% 0.993 ± 0.04 0.728 ± 0.03 20% 1.016 ± 0.02 0.747 ± 0.02 more combinatorial algorithms. More precisely, we start by pointing in Section 3.1 that our previous bias may not apply to directed graphs from domains other than social science, or to undirected graphs. Motivated by balance theories, we then introduce a new bias, assuming that ideally, nodes are grouped in K clusters such that all edges within clusters are positive and all edges between cluster are negative.

Table 3 .

 3 1 -Biological dataset properties. The columns have the same meaning as in Table2.2 on page 26

	Dataset	|V |	|E|	|E| |V |	|E + | |E|	Ψ 2 G (Y ) |E|	Ψ G (Y ) |E|	reciprocal edges	reciprocal disagreement
	HIPPOCAMPAL CANCER REGULONDB	501 1 046 2.1 69.5% 1 240 3 065 2.5 78.4% 1 700 2 570 1.5 50.3%	.056 .047 .060	.124 .108 .148	0.2% 5.5% 0.0%	0.0% 27.1% 0.0%
	one can imagine contexts where this model is not applicable. Another model, e.g.
	for proteins, could be that proteins belongs to functional groups and that two
	proteins interact positively if they belong to the same group, negatively otherwise.

Table 3 .

 3 2 -This table is the same as in Table 2.4 on page 31, but this time on three smaller, directed biological networks. ± 4.3 21.1 ± 4.0 19.0 ± 3.1 17.7 ± 4.0 17.2 ± 2.4

		|E 0 | |E|	L. PROP. BLC(tr, un) LOGREG	UNREG.	LOWRANK MAXNORM 16 TRIADS RANKNODES BAYESIAN
		5% 14.1 ± 5.7 11 .5 ± 6.3 9.6 ± 6.7	8.5 ± 4.7 10.9 ± 6.3	2.3 ± 3.4	3.7 ± 4.0	11.4 ± 7.4	5.4 ± 5.2
	HIPPOCAMPAL	9% 18 .1 ± 6.1 16.9 ± 6.0 15.4 ± 4.2 12.2 ± 4.2 15.3 ± 4.9 15% 20 .3 ± 5.5 18.6 ± 4.9 17.6 ± 4.4 14.3 ± 4.2 18.0 ± 4.5 20% 22.0	1.8 ± 4.6 2.7 ± 2.6	7.0 ± 3.3 12.1 ± 4.9 24.4 ± 3.7 21.2 ± 4.8	15.0 ± 5.7 19.5 ± 4.5

  ± 2.8 48.7 ± 2.3 44.6 ± 3.2 47.4 ± 2.8 55 .8 ± 1.8 26.0 ± 2.6 50.3 ± 4.6 59.7 ± 1.7

			.0 ± 2.2	10.3 ± 4.3 21.7 ± 2.9	15.6 ± 3.4
		15% 24 .6 ± 2.3 21.3 ± 3.5 20.1 ± 2.2 15.5 ± 2.0 19.6 ± 2.7	0.5 ± 2.0	14.8 ± 2.6 24.8 ± 3.2	20.3 ± 3.0
		20% 26 .5 ± 2.0 24.0 ± 2.3 24.2 ± 2.2 19.7 ± 2.4 22.7 ± 3.4	3.5 ± 2.5	18.8 ± 3.2 27.4 ± 2.7	22.4 ± 2.5
		25% 29 .3 ± 2.1 25.9 ± 1.6 26.3 ± 2.1 21.4 ± 1.2 24.8 ± 2.9	7.2 ± 2.6	21.2 ± 3.0 29.6 ± 2.0	24.8 ± 3.0
		50% 34.6 ± 1.7 34.7 ± 1.9 34 .8 ± 1.7 32.0 ± 2.9 31.3 ± 3.2	22.4 ± 2.6 31.7 ± 1.8 35.3 ± 3.3	33.7 ± 2.0
		90% 39.8 ± 4.4 42.4 ± 4.5 42.8 ± 4.7 39.3 ± 5.3 34.4 ± 5.3	34.3 ± 4.3 40.6 ± 6.5 43.9 ± 3.9 42 .8 ± 4.9
		5% 26.5 ± 8.9 31.5 ± 4.7 20.4 ± 13.6 29.0 ± 2.9 32.6 ± 4.1 -0.5 ± 2.2 7.7 ± 7.6	31 .7 ± 7.3	5.0 ± 7.7
		10% 39.3 ± 4.9 43.6 ± 2.9 38.5 ± 5.2 37.2 ± 2.3 40.1 ± 2.3 -2.3 ± 2.2 19.6 ± 7.2 43 .3 ± 3.7	8.2 ± 5.6
	REGULONDB	15% 44.9 ± 2.1 45 .6 ± 2.4 40.9 ± 6.0 40.2 ± 2.1 44.9 ± 2.9 20% 46 .8 ± 2.5 46.1 ± 2.3 42.0 ± 3.3 42.0 ± 2.2 46.1 ± 2.9 25% 49 .1 ± 2.4 48.4 ± 3.1 44.4 ± 2.6 43.8 ± 2.7 48.2 ± 2.0 50% 52.4	0.5 ± 2.3 3.0 ± 2.7 5.6 ± 2.1	29.9 ± 4.9 49.2 ± 3.0 35.3 ± 5.1 51.4 ± 3.5 20.3 ± 11.5 17.4 ± 9.6 38.6 ± 5.6 54.5 ± 2.1 32.7 ± 8.5

Table 3 .

 3 3 -MCC results on the six datasets from Chapter 2, after removing the directions as described in the main text. These results are presented like in Table2.4, except we have transposes the rows and the columns, and we show only three relevant methods. BLC(tr, un) 32 .8 ± 0.5 38 .9 ± 0.7 41 .8 ± 0.8 44 .3 ± 0.8 46 .3 ± 0.7 49 .8 ± 0.5 52 .6 ± 0.8 ± 0.342.6 ± 0.4 45.8 ± 0.2 47.8 ± 0.4 49.2 ± 0.2 52.5 ± 0.3 54 .1 ± 0.5 BLC(tr, un) 35 .0 ± 0.6 41 .4 ± 0.4 44 .9 ± 0.3 46 .9 ± 0.5 48 .2 ± 0.LOWRANK 39 .0 ± 0.3 41 .8 ± 0.3 44 .8 ± 0.5 48 .0 ± 0.6 50 .5 ± 0.4 57.2 ± 0.3 59.1 ± 0.6 EPINION L. PROP. 45.1 ± 0.4 50 .0 ± 0.4 52 .6 ± 0.3 54 .4 ± 0.2 55 .8 ± 0.3 59 .3 ± 0.1 62 .0 ± 0.2 LOWRANK 44 .1 ± 0.1 50.8 ± 0.7 55.9 ± 0.5 59.3 ± 0.4 61.7 ± 0.2 66.8 ± 0.2 68.4 ± 0.8 WIK. EDITS BLC(tr, un) 28 .5 ± 0.3 32 .9 ± 0.2 34 .8 ± 0.1

	|E0| |E|	5%	10%	15%	20%	25%	50%	90%
					CITATIONS
	L. PROP. BLC(tr, un) 18 .1 ± 0.7 22.2 ± 0.7 28.6 ± 0.5 32.0 ± 0.5 34 .4 ± 0.3 36 .9 ± 0.5 25.3 ± 0.8 29.7 ± 0.4 32.6 ± 0.4 35.8 ± 0.4 42 .2 ± 0.4 45 .0 ± 1.0 41.7 ± 0.4 44.9 ± 0.9 LOWRANK 17.3 ± 0.5 25 .3 ± 0.3 31 .4 ± 0.5 36.3 ± 0.8 41.2 ± 1.2 55.4 ± 0.8 60.6 ± 3.9 ADVOGATO
	L. PROP.	36.5 ± 0.4 41.4 ± 0.5 43.9 ± 0.4 45.7 ± 0.5 47.0 ± 0.5 50.4 ± 0.4 52.7 ± 0.7
	LOWRANK	28.8 ± 0.6	32.2 ± 0.4	33.9 ± 0.6	35.0 ± 0.5 WIKIPEDIA	36.3 ± 0.7	41.1 ± 1.0	46.2 ± 3.4
	L. PROP. LOWRANK	36.3 4 30.6 ± 0.7 37.2 ± 0.4 40.7 ± 0.5 43.4 ± 0.6 45.8 ± 0.6 51 .8 ± 0.7 54.6 ± 4.0 51.3 ± 0.4 52.7 ± 0.6 SLASHDOT
	L. PROP. BLC(tr, un)	39.4 ± 0.1 44.9 ± 0.2 47.8 ± 0.1 49.7 ± 0.1 51.0 ± 0.1 54 .4 ± 0.1 56 .4 ± 0.2 35.5 ± 0.3 40.7 ± 0.1 43.8 ± 0.2 46.0 ± 0.2 47.3 ± 0.2 51.5 ± 0.1 54.0 ± 0.4
	BLC(tr, un)	40.7 ± 0.6	47.2 ± 0.5	50.2 ± 0.6	53.1 ± 0.2	54.6 ± 0.2	58.6 ± 0.1	60.9 ± 0.2
	LOWRANK	26.7 ± 0.4	30.9 ± 0.4	34.	35.5 ± 0.2	36.1 ± 0.1	37.2 ± 0.1	37.6 ± 0.3

L. PROP.

34.4 ± 0.3 37.1 ± 0.4 37.9 ± 0.3 38.4 ± 0.6 38 .5 ± 0.4 38 .9 ± 0.3 39 .4 ± 0.3 2 ± 0.6 36 .6 ± 0.8 38.7 ± 0.7 44.0 ± 0.4 46.3 ± 1.8

  to social networks, it helps analyze political assemblies [Men+15; Jia15; LF17] and international relations [TB09; MMP12]. Finally, given large databases with duplicated records, it naturally models entity resolution [GZ07; Has+09; ARS09; SW14].

Table 3 .

 3 4 -Hardness results of CORRELATION CLUSTERING

			MINDISAGREE		MAXAGREE
	graph	weighted	unweighted	weighted	unweighted
	Complete General	APX-HARD [CGW03] APX-HARD [CGW03; DI03]	NP-COMPLETE [BBC02], APX-HARD [CGW03] APX-HARD [CGW03; EF03]	NP-COMPLETE [BBC02] APX-HARD [CGW03, Thm. 9]

Table 3

 3

	.6 -Approximation results for CORRELATION CLUSTERING on general graphs with K clusters

  in remaining then For any connected graph G = (V, E), EXTRACT-STARS(G) terminates in O(|E|) time.Proof. EXTRACT-STARS terminates because at each iteration of the while loop line 7, we remove one node from Q and never add any. Let us now analyze its complexity. We first build a priority queue of all the nodes according to their degree (line 2), which requires |V | insertions into Q. Then we execute |V | iterations of the while loop. However, the main idea here is that we process each node and each edge exactly once. We first find the center c of the next star by extracting the maximum of the queue (line 8) and testing if c is still part of the graph, which happens |V | times in total. Then we build the corresponding star (line 11-14). In total, we test the membership of |V | nodes in line 11, STAR updates the membership map |V | times in line 12, inner_edges consists of |E| edges at most in line 13 and remaining is only updated |V | times in line 14. Finally, we decrease the priority (i.e. the degree) of all nodes adjacent to the new star (line 15-17). Each decrement is supported by a single edge, thus DECREASE-KEYS is called at most |E| times. Since all queue operations require constant time when using a Strict Fibonacci Heap[START_REF] Brodal | Strict fibonacci heaps[END_REF], the complexity is O(|E| + |V |), which is also O(|E|).

	17:	DECREASE-KEY(Q, h, 1)
	18:	return stars, inner_edges, membership
	Proposition 1.
	10:	continue	c is part of an existing star so there is nothing to do
	11: 12: 13:	periphery ← G t [c] remaining stars ← stars {STAR(c, periphery, |stars|)}

inner_edges ← inner_edges {(c, p) : p ∈ periphery} 14: remaining ← remaining \ {{c i } ∪ periphery} 15: for p in periphery do 16: for h in G t [p] remaining do

  Let eccentricity be an array of size |V 0 | initially all set to 0 2: G t ← G 0 3: inner_edges_seq, outer_edges_seq ← [], []

	4: repeat
	5: 6:	stars, inner_edges, star_membership ← EXTRACT-STARS(G t ) Add inner_edges to the list inner_edges_seq
	7:	UPDATE-ECCENTRICITY(stars, eccentricity, original_node)
	8:	f ull_membership, MAPPING(f ull_membership, star_membership) original_node	←	UPDATE-NODES-
	9:	G t+1 ,outer_edges ← COLLAPSE-STARS(E t \inner_edges,star_membership, eccentricity)
	10:	Add outer_edges to the list outer_edges_seq
	11: 12: until |outer_edges| > 0 G t ← G t+1 13: return ASSEMBLE-SPANNING-TREE(inner_edges_seq, outer_edges_seq)
		def update_eccentricity(stars, eccentricity, original_node): for star in stars: for top_p in star.points: for real in original_node[top_p]: eccentricity[real] += 1 def update_nodes_mapping(star_membership, new_sm, first_iter): """After a collapse step, update the mapping between node indices in the new graph and original ones.""" if first_iter: star_membership = new_sm else: new_mm = {} for orig_nodes, previous_star in star_membership.items(): new_mm[orig_nodes] = new_sm[previous_star] star_membership = new_mm original_basis = defaultdict(set) for orig_id, star_id in star_membership.items(): original_basis[star_id].add(orig_id) return star_membership, original_basis def assemble_spanning_tree(stars_edges, interstellar_edges): final_edges = []

1: data = zip(reversed(stars_edges), reversed(interstellar_edges[:-1])) for current_level_edges, translation_to_lower_level in data: final_edges.extend((e for one_star_edges in current_level_edges for e in one_star_edges)) for i, e in enumerate(final_edges): final_edges[i] = translation_to_lower_level[e] return final_edges+[e for star in stars_edges[0] for e in star]

Table 3 .

 3 7 -Length of the paths not in the resulting spanning tree.

	test edge path in the tree

Table 3 .

 3 9 -A summary of the lowest stretches achievable for various problems.

	kind of	only existing edges		extra edges allowed
	stretch
		tree	not tree
	some	O(k log log n) [Abr+17]	[Gup+17, Section 4]	-
	pairs
	all
	existing
	pairs

Table 3

 3

			.10 -Dataset description
		|V |	|E| fraction of + edges	2|E| |V |•(|V |-1)
	WIKIPEDIA GOOGLE+ SLASHDOT EPINION	7 065 74 917 10 130 461 99 936 82 052 498 527 119 070 701 569	78.5% 67.6% 76.4% 83.2%	4.00 • 10 3 3.61 • 10 3 1.48 • 10 4 9.90 • 10 5

Table 4 .

 4 1 -Real attributed graphs that we have collected or are easily accessible. Unfortunately, none of them comes with known edge types, which implies the last column is merely speculation.

	Source	Node	Edge	Node profiles	Possible edge meanings
	DBLP Github Amazon Wikipedia article author devel-oper video game products Flickr user Foursquare venue IMDB movie BlaBlaCar driver or passen-ger	citation following "frequently bought together" internal link category, length, edit history, keyword count of their papers popularity, number of lines written in various programming languages price, genre, time of release, popularity and evaluation bag of words following activity statistics, groups membership, count of photo tags "frequently visited together" location, popularity, category, the time distribution at which they are visited sharing actors genre, release date and popularity shared ride experience, location, preferences, rating	scientific topic common hobby, involvement in the same open source project, same workplace, famous role model part of a series, cheap bundle, same genre and level of satisfaction generalization, specialization, organisation geography, same tags distribution, famous role model, closeness, part of a "series" (e.g. movie theater → restaurant → bar → club) part of a series, common theme, same time period and genre geography, professional, leisure

  b) The numerical attributes of the six nodes. All are categorical encoded with a ternary scheme, except for age, which has been transformed to fit in [-1, 1]. They are followed by the (non normalized) two directions w 1 and w 2 .

  max

	w 1 ,w 2 ∈S d	u,v∈E	max w∈{w 1 ,w 2 }	s uv	T w =	(4.4)
	arg max		s inner	T w 1 +
	w 1 ,w 2 ∈S d	u,v∈E suv=s inner			u,v∈E suv=souter

  [START_REF] Maclaurin | Autograd: effortless gradients in numpy[END_REF] 12 , projecting the current iterate back to the set of matrix with unit L 2 norm columns M n×d at each step. Finally, to minimize L node , we iterate over the nodes and take some gradient

				.10)
	In practice, we first minimize
	λL cross-entropy -L edge	(4.11)
	with respect to X using the Adam algorithm [KB15] and automatic differentia-
	tion [steps to minimize x u -1 deg(u)	v∈N u w uv	2 , but only to the extent that w

uv remains the direction with the highest goodness on the edge (u, v) ∈ E.

Table 4 .

 4 3 -The degree to which the edge assignment of step 2 agree with the directions and profiles of steps 3 and 4. This is measured by the AMI between the assignment {y uv } uv∈E and what maximal goodness would assign, i.e. {E(u, v)} uv∈E . We report the average AMI and standard deviation over 200 generations of D k , for different configurations.

	D k parameters AMI	default .922 ± .03 .925 ± .03 k = 5	k = 9, d = 36 .905 ± .02	n o = 6 .880 ± .04 .874 ± .05 .893 ± .03 .973 ± .01 n o = 12 k local = 4 d = 77

Table 4 .

 4 5 -Same as Table4.4, but reporting d(D k , D k ), which should be as close as possible to 0.

	D k parameters	k-MEANS	LLOYD	COMBINED	FRANK-WOLFE	EXPLICIT
	default k = 5 k = 9, d = 36 .541 ± .07 .533 ± .08 .581 ± .08 (4.78•10 -30 ) .606 ± .09 .528 ± .09 (9.61•10 -4 ) .521 ± .07 (5.08•10 -23 ) .520 ± .05 .516 ± .06 (4.95•10 -5 ) .560 ± .05 (8.07•10 -24 ) 1.007 ± .05 .877 ± .05 .811 ± .06 n o = 6 .554 ± .08 (2.74•10 -52 ) .589 ± .08 .538 ± .05 (2.38•10 -12 ) .908 ± .05 n o = 12 .554 ± .09 (1.50•10 -58 ) .595 ± .08 .545 ± .06 (4.08•10 -5 ) .935 ± .05 k local = 4 .565 ± .08 .564 ± .09 .546 ± .05 (6.30•10 -8 ) .928 ± .06 d = 77 .571 ± .07 (2.02•10 -89 ) .602 ± .07 .567 ± .05 1.033 ± .06	.564 ± .06 .633 ± .06 .551 ± .06 .596 ± .07 .604 ± .08 .588 ± .06 .641 ± .05

  To the best of our knowledge, there are not many works that directly attempt to classify edges in attributed networks. A first natural idea is actually to rely on the numerous existing node clustering algorithms, by considering the line graph L(G) of G. L(G) is a graph whose each of the |E| nodes corresponds to an edge of G, and where two nodes of L(G) are connected whenever their corresponding edges share a common node in G. Evans et al.[START_REF] Evans | Line graphs, link partitions, and overlapping communities[END_REF] apply a modularity algorithm to L(G) is order to partition the edges of G but, contrary to us, their final goal is to discover an overlapping clustering of the nodes of G, and they rely solely on the graph topology. In their review of mining social networks, Tang et al. [TL15, Section 2.3] present several methods to solve what they call inferring social ties.

Table 4 .

 4 6 -We summarize how each model generates: 1) the membership of a node u to a community c ∈ k , 2) the attributes of u knowing its community membership and 3) the edges between nodes. P uv = 1exp(c F uc F vc )

	ref	community	attributes	links
	[Xu+14a]single community drawn from a multinomial	drawn from a distribution parametrized by the node community	drawn from a distribution parametrized by the pair of endpoints communities
	[YML13] intensity of membership F uc ∈ [0, ∞) are given	logistic function of F u1 , . . . , F uC	sharing more communities makes link more likely:
	[Kat+16] multinomial of dimension k	drawn from one normal distribution per community	Stochastic Block Model (SBM), where blocks are identified with communities
	[ZDB17] gamma distribution, whose parameters depends of the nodes attribute	given binary attributes	SBM, with the block matrix drawn from a hierarchical relational gamma process
	[LM12; ML14]	overlapping circle defined by θ k , but not generated	given	higher probability of appearing within common circle, as given in (4.12)
	[WF16] logistic function of the attributes given	SBM
	[NC16] one multinomial for each attribute discrete value of the single	given	degree corrected SBM

In the rest of this thesis, we use the terms graph and network interchangeably.

Note that by "type of relation", we do not refer to some graph-theoretical characteristic of an edge, such as being directed or weighted. Indeed, all the graphs we consider have homogeneous edges with that respect. Instead, we mean domain specific semantic, as we will make clear in examples.

https://en.wikipedia.org/wiki/Seven_Bridges_of_Königsberg

http://app.mappr.io/play/chile-marine-intertidal-network

This chapter is closely based on a paper written with several co-authors [Le +17].

This semantic detail allows us to treat citation networks as social networks.

In fact, for the sake of this specific argument, nothing prevents from retaining of G either only the edges (uout, eu,v) or only the edges (eu,v, vin), resulting in a cutsize of Ψout(Y ) and Ψin(Y ), respectively.

The hat symbol denote quantities computed solely on the training set.

https://investor.fb.com/investor-news/press-release-details/2017/ Facebook-Reports-Second-Quarter-2017-Results/

As we will see, in real data, around 80% if the edges are positive, meaning this assumption of uniformity over [0, 1] 2 is unlikely to fully holds, for otherwise the signs would more balanced.

as we do in Section 2.9.1.3.

While we note here that such linear transformation of the variables does not change the problem, we provide more details in Section 2.9.1.3 of the supplementary material.

Recall that the ego network of a node u is the subgraph induced by the neighbors N (u) of u.

We download the 7 th of July, 2014 version from http://www.trustlet.org/datasets/ advogato/.

This is the KONECT version of the "Wikisigned" dataset, from which we removed self-loops.

We again removed self-loops and merged multi-edges which are all of the same sign.

More precisely, these figures are normalized histograms, meaning the counts are scaled down so that the bars total area sums up to one.

We reproduce this equation here for convenience: sign 1 -tr(u) + 1 -un(v) -τ -1 2 .

Here and in the following, we multiply all MCC value by 100 to improve readability.

We have also tried to minimize (2.8) by removing the [-1, +1] constraints, but got similar MCC results as the ones we report for UNREG.

Comparison of training time performances is fair since all algorithms have been carefully implemented using the same stack of Python libraries, and run on the same machine (16 Xeon cores and 192GB Ram).

corresponding to 16 × 16 feature per edge

As done in the proof of Theorem 1, on the facing page.

http://regulondb.ccg.unam.mx/menu/download/datasets/index.jsp

my friend's enemy is my enemy + --

That is, ∀i Ci ⊂ V , K i=1 Ci = V and ∀i = j, Ci ∩ Cj = ∅.

Note that in the data mining literature, CORRELATION CLUSTERING may refer to another problem. Namely, it is a special case of high-dimensional data clustering, where features are locally correlated in various ways across different clusters that reside in arbitrarily oriented subspaces[START_REF] Kriegel | Clustering high-dimensional data: a survey on subspace clustering, patternbased clustering, and correlation clustering[END_REF].

However, the same problem was studied before and we refer the reader to the comprehensive survey of Böcker et al.[START_REF] Böcker | Cluster editing[END_REF] for additional details, whereas we shall only give important and more recent pointers here.

The reduction is from 3-exact 3 cover, see [SST02, Theorems 1, 2 and Corollary 1].

An algorithm A is a polynomial-time approximation scheme (PTAS) for a minimization (respectively maximization) problem P in NP if given any > 0 and any instance x of P of size n, A produces, in time polynomial in n, a solution that is within a factor 1 + (respectively 1 -) of being optimal with respect to x. Note that the time is not necessarily polynomial in , so that a running time of O(n 1 ) would qualify [Aus+99, Definition 3.10].

http://maxhs.org

Namely, the Unique Game Conjecture of Khot[START_REF] Khot | On the power of unique 2-prover 1-round games[END_REF].

Namely, two positive cliques A and B joined by positive edges except between a ∈ A and {b1, . . . , b k } ∈ B. Those nodes are given first and thus form a cluster, which yields at least one disagreement every time one the n -(k + 1) remaining nodes is added.

For we can otherwise run our algorithm in parallel on each connected components of G0.

Note that for clarity, we removed some bookkeeping code in all listings, mainly the part related to maintaining mapping between nodes at different collapse level. However, the full python implementation is available at https://github.com/daureg/magnet/blob/master/veverica/new_ galaxy.py#L27.

 12 We also consider more involved heuristics but, in the interest of simplicity, they are presented later on page 78.

We can get rid of this O(|V0|) at each iteration by ignoring eccentricity and taking a random edge between star, although it is not clear at the moment how it would affect the quality of the resulting tree.

Although they are available online at https://github.com/daureg/magnet/blob/ master/veverica/{ThresholdSampler.py, NodeSampler.py}.

We adapt their notations to match ours

Available at http://snap.stanford.edu/data/egonets-Gplus.html

As implemented by the graph-tool library (https://graph-tool.skewed.de)

We nonetheless perform some preprocessing in order to make them undirected and to remove the small proportion of conflicting edges (e.g. positive from u to v but negative from v to u).

More precisely, consider the edge (u, v) in Gt+1. The two nodes are expanded into the two stars su and sv in Gt, respectively with peripheral nodes {u1, . . . , u k } and {v1, . . . , v } and center cu and cv. In the worst case, there is now a path of length 5 in Gt, e.g. (u1, cu, u k , v , cv, c1).

Which roughly corresponds to the single aspect, node-aligned multilayer graph of[START_REF] Kivela | Multilayer networks[END_REF].

Note of course that in both cases, negative values do not represent a judgement but are merely a reversible mathematical convenience.

Because our problem is essentially unsupervised, assuming that we are given k is a simplification, on which we elaborate on page 96.

Whereas the name is inspired by the concept of goodness of fit, there is no formal relation.

The incidence matrix of a graph G, usually denoted by B, is the |V |×|E| matrix such that Bi,j = 1 if the node vi and edge ej are incident and 0 otherwise.

Indeed, since the first term is convex, its maximum can only be reach at the boundary of the feasible domain [Roc70, Theorem 32.1]. However, we are also subtracting another convex term and the resulting function is therefore more complicated.

As mentioned in Section 2.4 about[START_REF] Wang | Online matrix completion for signed link prediction[END_REF], a tighter convex relaxation of the low-rank constraint is the max norm, but we do not pursue that venue further.

γ is either found by a line search or, in our case, set to t /2+t.

We ruled out p being adjacent to itself to avoid the trivial solution of all nodes being assigned the same palette.

For instance, consider a 4-clique with k = 4 and k local = 2. Without loss of generality, say we assign the palette (1, 2) to node 1. The adjacent palettes are then (1, 3), (1, 4), (2, 3) and (2, 4). If we assign those starting by 1 (respectively 2) to the second and third node, then the fourth palette has to contain a 1 (respectively 2), which is not possible (because two connected nodes cannot have the same palette). On the other hand, if we assign (1, 3) to the second, the third node can only have (2, 3), meaning the fourth node has again no palette available. The same situation arises with (1, 4) and (2, 4) respectively.

Both being implemented by the Pytorch package: http://pytorch.org/.

We add one dimension to ensure that d remains a multiple of k.

One can derive this from the fact after normalization, vectors whose coordinates are drawn from a standard normal distribution are uniformly distributed on the unit sphere[START_REF] Muller | A note on a method for generating points uniformly on n-dimensional spheres[END_REF].

At least at the time of writing

Especially the section 2 dealing with edge labeled graphs.

Acknowledgements

Discussion While the EXPLICIT formulation is not convex, in practice we initialize it with the solution from COMBINED and it thus converge after few iterations. Namely, we set P 0 ∈ M d×k to be the final directions found by COMBINED and the rows of Q 0 ∈ M |E|×k to be equal to the corresponding e E(u,v) . Furthermore, in the projection step of the Q variable, we actually ensure that all the entries of Q are positive, which makes them easier to interpret as weights for the basis directions of P . This is something that we cannot guarantee on the decomposition of the matrix W obtained by the FRANK-WOLFE method. On the other hand, if we run the FRANK-WOLFE algorithm for more than k iterations, the rank of W can be higher than k, which might allow a finer clustering of the edges.

Compared with the COMBINED approach of Section 4.2.2, these two matrix formulations require learning more parameters. Indeed, it increases from dk to d|E|, as it each edge has a different direction, albeit made up of a small number of basis directions. In the case of EXPLICIT, we could furthermore impose a sparsity constraint on the columns of Q to reduces the number of coefficient to learn. Yet this additional complexity has the advantage of allowing overlapping clustering of the edges. This is useful in cases where two nodes have a relationship that is best described by a combination of the k base directions. Moreover, in the case of FRANK-WOLFE, this additional flexibility comes with no increase in computational cost.

Synthetic experiments

In the previous section, we introduced five methods to address Problem 5, which we now compare in our experiments. Specifically, this includes:

1. the k-MEANS baseline, 2. our LLOYD-like iterative refinement, 3. the COMBINED optimization of the two terms in (4.7), 4. the FRANK-WOLFE convex optimization of (4.8) and 5. the EXPLICIT explicit low-rank factorization, optimizing (4.9). They all take as input a node-attributed graph, that is a pair G = (V, E) and X. For evaluation purposes, we extract from their output a set of k directions and an assignment of every edge to one of these directions. We try those five methods on data generated synthetically to fit our model described previously in Section 4.1.1, and study how they perform under various conditions.

Data generation

While there exists several methods to generate attributed graph with community structure [YML13; Xu+14b; Kat+16], here we present the one we devised for our initial experiments, as it is specifically tailored to the model introduced in Section 4.1.1. It takes the following parameters as input:

• the number of nodes n,

• the dimension of their profiles d,

• the total number of directions k,

• the maximum number k local of directions incident to each node, and • an integer n o controlling to which extent two distinct directions explain common dimensions. It then returns:

• a graph topology G = (V, E), • a profile matrix X ∈ R n×d , • a set of k directions D k ⊂ B d , and • an assignment from every edge (u, v) to a direction index y uv ∈ k such that in most cases, E(u, v) = y uv . In other words, the direction assigned to (u, v) is