Robust Kernel Density Estimation with Median-of-Means principle Contents

Après avoir soutenu ma thèse il y a quelques mois, je me lance dans l'écriture des remerciements. Étonnamment, c'est peut-être la seule partie de mon manuscrit de thèse que j'arrivais déjà à imaginer dès le tout début de ces quatre années. Je n'avais rien trouvé, pas même mon sujet de thèse, que je me voyais rédiger des remerciements bien écrits et originaux : la seule partie à la lecture appréciable pour ma famille et mes amis. Il s'agit nalement du dernier élément que j'appose à mon manuscrit. Et maintenant que tout est ni, la pression redescendue, je rédige des remerciements qui ressembleront en dé nitive à tous les autres (désolé !).

Je tiens tout d'abord à remercier mon directeur de thèse, Nicolas Vayatis, ainsi que mon co-encadrant, Argyris Kalogeratos, pour leur soutien et leur con ance au cour de ces quatre dernières années. Je remercie également Sigfox et en particulier Olivier Isson qui a su me donner toutes les libertés nécessaires à l'élaboration d'une thèse Cifre. Merci aussi pour ces très bons moments passés autour des meilleures tables de Toulouse.

Je remercie ensuite tous les membres du jury qui ont accepté de participer à ma soutenance de thèse. Merci à Fabrice Rossi et George Michailidis d'avoir tout de suite accepté leur rôle de rapporteurs. Merci au président, Charles Bouveyron et merci également aux deux examinateurs.trices, Tabea Rebafka et Gilles Blanchard.

Mes prochains remerciements vont à l'armée de doctorants du CMLA/Centre Borelli. C'est aussi grâce à chacun de vous que ces années de thèse, parfois di ciles, se sont si bien passées. Merci pour l'entraide, les rires, les ragots. Merci surtout d'avoir supporté ma grosse voix et écouté mes plaintes. Mention spécial à la team des Saints-Pères bien sûr, avec qui j'ai partagé tout cela "puissance dix".

Mon plus gros des "shout out" va bien évidemment à Pierre Humbert, mon frère dans ce labo, avec qui j'ai eu la chance de contribuer et de discuter longuement de tout et de rien. Merci notamment pour nos discussions sur des concepts mathématiques des plus basiques, pendant lesquelles nous n'avons aucune honte à nous poser mutuellement des questions auxquels un étudiant de L1 maths pourrait répondre. Je ne vais pas m'éterniser car on en a déjà beaucoup parlé mais merci pour tout ! Je remercie également mon binôme du côté de Sigfox, Kevin Elgui. On se suit depuis le MVA et j'espère qu'on sera amené à retravailler ensemble un jour ! Merci pour les discussions de maths, les cafés à rallonge et surtout merci pour toutes ces bières et bons repas partagés à Toulouse.

En n, merci à mes parents, mon beau-père, mon frère et ma soeur. Merci aussi à tous mes copains et copines dont la liste serait un peu longue à énumérer. Ce n'était pas toujours évident de discuter de ma thèse avec vous, mais nalement tant mieux ! C'est aussi cela qui m'a permis de ne plus y penser, de décompresser et de terminer cette thèse sereinement.

Notations

Sets, matrix and vectors

[n]

Set of integers {1, . . . , n}

S S

Set S minus the subset S ⊂ S

x T , M T Transpose of vector x, matrix M

x j Vector x deprived of its j-th coordinate

x I Over the past decades, the signi cant increase of the amount of multivariate data available in all sectors, from medicine to industry or social networks, has created urgent needs for data analysis and modeling in order to accomplish various tasks. Among them, the tasks of anomaly, or change-point, detection (called event detection when referring to both of them) in massive amounts of data are of major importance. In a nutshell, anomaly detection refers to the task of nding, in a data set, a small amount of vector data that deviates from the normal behavior of the vast majority. This task can be applied to any data set and in many real-world applications. For example, it can be used to discover dysfunctional elements in a company's production line. On the other hand, the theoretical basis of change-point detection relies on the vast area of time-series analysis, and it seeks to nd time-instances at which there is a change in the regime underlying the data (Figure 1.1). In life science and biology applications, it can for example correspond to the moments when there is a change in the state of the of the monitored system (e.g. an individual sleeping or not via electroencephalogram analysis, the beginning of puberty via the hormone secretion study etc.). New challenges to these problems have been posed by data that appear naturally over a network structure, which interconnects the observations or the variables representing them. This is particularly the case for data coming from social, communication, transport or sensor networks, where the data collection takes place at the level of nodes in a graph. In the social network example, the nodes may refer to users and an edge indicates the social link between two of them; in a sensor network, an edge may simply correspond to the spatial distance between two sensors. Usually, the graph brings knowledge on the process that generates the data (e.g. two linked nodes are highly correlated or has very close values), and being able to build models or learning algorithms -including anomaly or change-point detection methods -from these data, while considering their network structure, is of major concerns to improve learning performances.

This type of collected data are referred as graph data, graph vectors or graph signals. As stated earlier, they simply refer to vector data for which each component is associated to a node in a graph. While in some cases the graph is naturally given and therefore known a priori (e.g. the social network or the sensor network example given above), there are many cases where the data admit an underlying graph structure that is not available and needs to be learned from them. This is notably the case in biology, where we are interested in knowing which genes (or proteins) are expressed with each other [62,[START_REF] Kitano | Computational systems biology[END_REF]. More generally, this need for graph inference can appear in any type of data for which one wishes to know which variables are linked with which others, in the sense that they behave in a similar way statistically. This task can have strong impact on the visualization and the understanding of the data being processed, but also, as said before, on the ability to build more e cient learning algorithms.

This thesis work is deeply rooted in the various topics mentioned above. In particular, it focuses on event detection in graph data, whether the graph is known or not. It also focus on the inference of the graph itself, and on the detection of changes in the graph structure underlying the data. Among the notable achievements of this thesis are the methods developed, but also the fact that for the purpose of their development, several mathematical tools and techniques were employed.

1. CONTEXT OF THE THESIS 17

Industrial context

This doctoral thesis was carried out thanks to CIFRE (Convention Individuelle de Formation par la Recherche) and commissioned by the ANRT (Agence Nationale de la Recherche et de la Technologie). It was sponsored by Sigfox, a world-wide telecommunication network operator dedicated to the Internet-Of-Things (IoT). Speci cally, Sigfox owns antennas, referred as Base Stations (BSs) that are set up on towers (like a cell phone company), and receives data transmissions from devices like parking sensors, re detector, water meters, etc. The devices are held by customers, and the role of Sigfox is essentially to collect the (encrypted) data transmitted by the devices, via the BSs, and send them to a cloud to which the customer has access. The particularity of Sigfox is its simple protocol, which is suitable for the devices to send small amounts of data (12 bytes per sent "messages") but received at a long-range and in a very low-power consuming way. Without going into much more detail, let us brie y describe the process of sending a message in the Sigfox protocol.

Let a device that needs to transmit a small amount of data (e.g. temperature, binary information, etc.) to its owner. The information is encoded in a signal that is sent three times, at three di erent frequency levels, for robustness. The signals are sent without any selection protocol on the Base Stations, they are simply sent "in the air" (broadcast), hoping that they will be received by at least one nearby BS (thanks to a vast coverage, the signals are usually received by many of them). Once at least one BS receives one of the three repeated signals, it is decoded and sent to the cloud using standard internet protocol such as 3G. A scheme of Sigfox network's architecture is provided by gure 1.3. For a more complete description on Sigfox technology and Low-Power Wide-Area Network (LPWAN) in general, we invite the reader to look at [START_REF] Chaudhari | LPWAN Technologies for IoT and M2M Applications[END_REF].

Initially, the principal objective of the collaboration with Sigfox was to propose and develop data-based methods to detect anomalies at the level of a Base Station (e.g. caused by a breakdown). This was a new subject, poorly dealt by the company's researchers and engineers, but the need for it became more and more important due to the signi cant expansion of the network. Until then, very little use was made of the collected data for this purpose and only simple methods, based on threshold exceedances, has been employed. The thresholds were set a priori by eld experts and led to a very high false positive rate.

Various data are collected at the level of each BS, from hardware (e.g. temperature of the engine) to software information (e.g. the OS version used). According to the experts, the important features to detect anomalies are those linked with the spectrum of the signals captured by the BSs. Unfortunately, the whole spectrum is not collected but only few statistics summarizing it are computed instead every second. These includes essentially some quantile information on the intensities recorded all over the spectrum. After few weeks of data analysis made at the beginning of this research, it has been nally decided that only the "reception" information about each BS would be kept. In other words, this corresponds the information on the activity of the network: for each message broadcasted, which BS has received it or not. This decision was essentially motivated by the fact that this kind of data are raw (contrarily to the spectrum data that are already processed) and with a priori few errors in it. Moreover, it is intuitively expected that a failure at the level of a BS will directly impact its level of activity, and most probably with a decrease in its total number of received signals.

An interesting property of the reception data is that they appear naturally over a graph structure induced by the spatial distribution of the BSs. Taking for example the vector that indicates, for a xed sent signal, which BS has received it or not, it is empirically observed that nearby BSs will have more chance to be 'activated' (in the sense of reception) together, and conversely. Another example includes the vector that speci es the number of signals received by each BS over a certain period of time, where close BSs will have highly correlated values (see Figure 1.4).

Under these observations, we made the conclusion that the studied vector data can be considered as graph vectors, making the thesis move towards the task of event detection in a set of graph signals. Nevertheless, quickly, and as explained in detail in the following section, it was realized that the induced spatial graph was not all the time adapted to the data, leading to the second axis of the thesis related to the inference of the network structure itself.

OBJECTIVES AND MOTIVATIONS

Objectives and Motivations

In the following, we describe the di erent objectives that will be at the core of this document.

Detecting behavioral changes or anomalies at node-level in communication networks. This rst objective is applied and essentially motivated by the industrial problem raised by the collaboration with Sigfox. However, this detection task can arise in many communication network-related settings (e.g. in social or computer networks), and in order for this work to be more generally applicable, we will seek to propose methods for event detection in a large spectrum of communications networks, including Sigfox. If, in the context of the industrial collaboration, such a task could boil down to the detection of a BS failure, in a sensor network for example [START_REF] Xie | Anomaly detection in wireless sensor networks: A survey[END_REF] it can characterize the detection of an issue at the level of a sensor or the value it quanti es. In a computer or a social network, such detection problem can arise, for example, in network security where the anomaly can come from an attack (e.g. hacking, identity fraud, etc.) [START_REF] Ahmed | A survey of network anomaly detection techniques[END_REF]. Previous examples illustrate well the importance of such a task: non-detected anomalies can have signi cant impacts in the performances of the considered network. To sum up, our objectives will be to, rst, give a simple de nition of a communication network which corresponds to a wide range of real-life deployed networks. Similarly, we will consider simple notions of anomalies that can be observed in various types of networks. Then, we will propose a way to detect anomalies. And in order to remain close to our industrial application, we will restrict ourselves to node-level detection, i.e. the event detection at the level of a single entity (e.g. a base station, a computer, etc.).

Detecting change-points or anomalies in a set of graph signals. This more general task was initially motivated by the conclusion we made in the previous section, namely the fact that the data of interest are graph vectors. But of course, as explained in the previous section, these types of signals are frequently observed in real-world applications, Each dimension of the time-series corresponds to a node in the corresponding graph [START_REF] Hallac | Network inference via the time-varying graphical lasso[END_REF]. and building new algorithms for detecting anomalies or change-points which are adapted to them is important. When the graph is known, it constitutes a prior knowledge about the model that generates the data. It is therefore conceivable that the performance of the algorithms can be improved by using this additional information [START_REF] Egilmez | Spectral anomaly detection using graph-based ltering for wireless sensor networks[END_REF]. It allows the construction of new features or embeddings based on the information brought by the graph, and thus, allows to discover more complex types of events [START_REF] Chen | Change-point detection of gaussian graph signals with partial information[END_REF]. Typically, an anomaly detected at the level of one node can come from a value which is abnormal, with respect to istself and with respect to its neighbors. When the graph is unknown, no much things can be done. However, knowing that there exists an underlying graph structure tells an important thing: if one wants to detect an anomaly or a change-point at the level of a speci c dimension (i.e a speci c node), the values observed at the other nodes should be considered as well since they are linked through graph edges [START_REF] Bars | A probabilistic framework to node-level anomaly detection in communication networks[END_REF]. Despite this, an unknown graph suggests a rst step that would learn the graph itself, in order to understand better the data and apply more adapted event detection algorithms. This learning problem constitutes our third objective.

Learning the graph structure underlying vector data. This objective is met in many elds and can be applied to any kind of vector data. In an extension of the notion of graph signals previously mentioned, here the objective is to infer the underlying structure of the data. In other words, the goal is to learn notions of relationships between variables, i.e. with which other variables, a variable of interest is more similar or related with (e.g. in term of correlation, conditional independence, scale, values etc.). Such learning procedure is done using a set of vectors assumed to admit the same underlying graph. During the learning process, structural penalties can be imposed, such as sparsity of the graph [START_REF] Dong | Learning laplacian matrix in smooth graph signal representations[END_REF][START_REF] Friedman | Sparse inverse covariance estimation with the graphical lasso[END_REF][START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF], and in the present work, we will study some of them. Inferring such a graph structure has several applications. First of all, it helps to understand the considered vector data with a simple visualization brought by the graph. Moreover, such a learning task is often linked with a model. This is particularly the case with Markov Random Fields [START_REF] Koller | Probabilistic graphical models: principles and techniques[END_REF], which in many situations assume a linear relationship between the variables, a relationship determined by the graph itself [START_REF] Hö Ing | Estimation of sparse binary pairwise markov networks using pseudo-likelihoods[END_REF][START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF]. With such a modeling, we can imagine that one could try to predict the value of a node as a function of the other node values and the graph weights. This illustrates well the applicability of the learned graph. It can also be used in many machine learning algorithms that require a graph, typically the spectral clustering algorithm [START_REF] Ng | On spectral clustering: Analysis and an algorithm[END_REF], semi-supervised algorithms [START_REF] Belkin | Semi-supervised learning on riemannian manifolds[END_REF], in the framework of the Graph Signal Processing [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF], etc. One of the most famous real-world application of graph learning is in biology, with genes interaction network emphasizing genes that are most of the time expressed together. At Sigfox, we could think that the graph is directly given by the spatial position of the BSs (Figure 1.4), assuming that nearby BS will receive a lot of messages in common. In practice this is not always true, two BSs located at a small distance from each other can be separated by a wall or be at di erent altitudes, making them quite di erent in their ability to receive a same signal. This observation made us conclude that the graph learning task could also be interesting with Sigfox reception data.

Detecting changes in the underlying structure of vector data. This objective can be seen as a combination of graph learning and change-point detection. In fact, contrarily to the vast majority of change-point detection techniques that look for a signi cant change in the mean of a time-series, the task here is to detect a change in the underlying graph structure itself. Thus the objective is twofold, nding time-instances between which all the observed vector data has the same underlying graph structure as well as learning these graphs. An illustration of such a task is provided by Figure 1.5 for a real-valued time-series with 3 dimensions. In addition to determining instances of time at which the system has undergone some changes, methods that meet this objective also take advantage of the bene ts that can be brought by the graph inference (see the previous paragraph), i.e. modeling, applicability of machine learning algorithms, etc. In particular, the visualization aspect mentioned above allows a strong understanding and interpretability of the found change-points [START_REF] Bars | Learning the piece-wise constant graph structure of a varying ising model[END_REF].

Dealing with binary graph signals. This last objective is essentially motivated by the fact that the considered vector data are binary. Indeed let us recall that in our application setting, a raw data is a vector corresponding to a message broadcasted by a device in Sigfox network. This vector encodes which BS have received the signal of the message or not. Nevertheless, this problem remains important in many other contexts [5], particularly due to the fact that it is often less studied than real-valued vector data or time-series.

Background

In this section, we propose to brie y recall some fundamentals on event detection, namely anomaly and change-point detection, on graph theory, and on vectors observed over graphs. The objective is to provide some basic de nitions, notions, and algorithms that will be useful in the rest of the manuscript.

Anomaly detection

In its most classical version, anomaly detection refers to the task of nding in a data set a small amount of vector data that has been generated by a di erent distribution model than the majority of them. This simple formulation has motivated many statistical anomaly detection methods which assume that anomalies stand in regions of low density. Among them, the one of [START_REF] Eskin | Anomaly detection over noisy data using learned probability distributions[END_REF] assumes that the normal data are generated by a known in advance probability distribution and considers abnormal the data that stands in a region of low probability.

While being the most generic, the previous formulation actually corresponds to a particular framework of anomaly detection. In fact, depending on the labels available, anomaly detection tasks can be divided in three types. The supervised anomaly detection consists in training the algorithm based on a labeled (normal/abnormal) data set including both normal and abnormal observations. This type of detection is therefore highly related to the problem of classifying imbalanced data [START_REF] Sun | Classi cation of imbalanced data: A review[END_REF]. The second scenario is the novelty detection framework, also referred as one-class classi cation or semi-supervised anomaly detection. In this setting, only normal data are available for the learning phase. This is the case in applications where normal behaviors are known but e.g. intrusion or attacks induce an unknown behavior and must be detected. This scenario is the one considered in Chapter 2. Finally, the unsupervised setup refers to the one presented in the previous paragraph: the learning data set contains both normal and abnormal data but no labels are available.

Usually, most anomaly detection algorithms do not simply map the input vector to a binary value (indicating an anomaly or not). Instead, they return a real-valued function, referred as scoring function, that outputs, for a given input vector, a real value as score of abnormality. The advantage of using such a scoring function is that it allows to rank the samples from the less to the most abnormal. This is very important when one has a lot of data to deal with and wants to prioritize some anomalies. Moreover, if one wants a binary output, it su ces to x a threshold above which the score will be considered abnormal.

There exists a wide variety of anomaly detection algorithms, from those based on density learning [START_REF] Breunig | Lof: identifying density-based local outliers[END_REF][START_REF] Schölkopf | Estimating the support of a high-dimensional distribution[END_REF][START_REF] Scott | Learning minimum volume sets[END_REF][START_REF] Vert | Consistency and convergence rates of one-class svms and related algorithms[END_REF], like previously explained, to those based on, e.g. decision trees [START_REF] Liu | Isolation forest[END_REF]. This great variety of algorithms is accentuated by the kind of available labels, as explained earlier, but also by the type of the analyzed data. Here we focus on classical vector data, but these can be temporal or even text data. It would therefore be impossible to make an exhaustive list of these methods here, and for complete surveys one may refer to [START_REF] Chandola | Anomaly detection: A survey[END_REF][START_REF] Patcha | An overview of anomaly detection techniques: Existing solutions and latest technological trends[END_REF]. In the following, we present an e cient anomaly detection algorithm, for standard vector data, that performs well on both the unsupervised and the semi-supervised labeling scenarios.

One-class SVM

The One-class Support Vector Machine (SVM), rst introduced in [START_REF] Schölkopf | Estimating the support of a high-dimensional distribution[END_REF], extends the standard SVM for classi cation with two classes to the problem of novelty detection. Indeed, rather than having access to a labeled data set with both positive and negative labels, it assumes that all the input data belong to class 1 (the normal class). Then, instead of constructing a hyperplane separating two classes, it constructs a hyperplane separating the mapped input points from the origin of the mapped space, treated here as the only point of the second class.

Formally, in its soft-margin version, the One-class SVM works as follows.

Let x (1) , . . . , x (n) ∈ R d be n observations and Φ : R d → H be a feature map in a Reproducing Kernel Hilbert Space [START_REF] Aronszajn | Theory of reproducing kernels[END_REF] H with kernel k (usually the Gaussian kernel). To separate the data from the origin, the one-class SVM solves the following quadratic program:

min ω∈R d ,ξ∈R n ,ρ∈R 1 2 ω 2 + 1 n n i=1 ξ i -νρ s.t w, Φ(x (i) ) ≥ ρ -ξ i , i = 1, . . . , n ξ i ≥ 0,
where ν ∈ (0, 1) is a hyperparameter that prevents over tting and allows the training data set to contain outliers. In fact, it can be showed that ν corresponds to an upper bound on the fraction of anomalies that are allowed in the learning set. The decision function used to detect anomalies is given by: f (x) = sign( w, Φ(x (i) )ρ), which will, as required, be positive for most of the learning vectors.

From the previous de nition, the One-class SVM can be seen as the estimation of a space having minimum volume while containing almost all the input data. In fact, it is linked with the estimation of a Minimum Volume set [START_REF] Scott | Learning minimum volume sets[END_REF] with mass 1ν, i.e. the set of minimum volume with respect to the Lebesgue measure but having a mass of 1ν with respect to the probability measure of the normal data.

Quality measures

Like any other learning method, being able to evaluate the quality of anomaly detection algorithms is very important. When labels are available, every quality measure used for binary classi er and scoring functions can also be employed in this case. These include, for example, the analysis of the Receiver Operating Characteristic (ROC) curve and the associated Area Under the Curve (AUC). One of the drawbacks of using such measures is that they are not particularly adapted for imbalanced classes. For this reason, measures related to the normal class, such as the false positive rate, may be preferred.

When few or no labels are available, the question of evaluating the quality of the algorithms remains an open question. Some tracks, seeking to extend the notion of ROC curves to the no label scenario, have notably been studied in [START_REF] Goix | How to evaluate the quality of unsupervised anomaly detection algorithms[END_REF] and are related to Excess-Mass and Mass-Volume curves. However, these methods will not be considered in the present manuscript and, when necessary, labels will be available.

Change-point detection

Change-point detection is a particular task of time-series analysis. Its objective is to nd time-instances at which signi cant changes occur in the underlying model of a given time-series. A stated in Section 1.1, this task has many applications, whether it is in speech processing [76], climatology [START_REF] Reeves | A review and comparison of changepoint detection techniques for climate data[END_REF], network tra c data analysis [START_REF] Lévy-Leduc | Detection and localization of change-points in highdimensional network tra c data[END_REF], etc. An illustration of this segmentation problem is provided in Figure 1.1.

The change-point detection task can be divided in two main categories: o ine and online detection. In the o ine scenario, the segmentation is performed after the signal has been observed entirely, it is also referred as a posteriori detection. On the contrary, in the online scenario we aim at nding the change-points in real time, while the vector data are being observed. This task is also referred as sequential change-point detection.

In the present section we focus on the o ine scenario, which is considered in Chapter 4. This o ine task can again be split in two subcategories: the case where the number of change-points to discover is known and the situation when it is unknown. Most of the time, the di erence between the resolution of the two problems lies in the addition of a term penalizing the number of estimated change-points in the optimization program. Before going further, let us describe the problem more formally.

We consider the statistical framework described in the review of [START_REF] Truong | Selective review of o ine change point detection methods[END_REF]. Let {x (i) } n i=1 be a time series in R d assumed to be piecewise stationary, meaning that there exist instances T = {t 1 , . . . , t K } ⊂ {1 . . . , n} at which the model underlying the time-series changes. The objective of change-point detection is to recover the indexes T , and therefore the number of change-points when it is unknown. To do this, most of the methods found in the literature seek a set of indexes T = {t k } K k=1 ⊂ {1 . . . , n}, estimating T , that minimizes a function of the form

K k=0 c({x (i) } t k+1 i=t k +1 ), (1.1)
with t 0 = 0, t K+1 = n and c(•) is a cost function that evaluates the quality of each learned segment. When the number of change-points is known, K = K , otherwise, a term penalizing the size of T is added. All change-point detection methods then di er in two main aspects: either on the cost function that is used, usually linked to the model underlying the data (e.g. parametric or not), or on the method used to solve the minimization problem stated above. The problem being combinatorial, many solutions have been proposed (greedy resolution, dynamic programming, heuristics, etc.). The wide variety of cost functions and minimization methods does not allow us to be exhaustive and we invite the reader to see the review of [START_REF] Truong | Selective review of o ine change point detection methods[END_REF] for examples and in-depth discussion. Nevertheless, we give below an example of a model and an associated cost function that we think are important to know.

Example 1.1. The maximum likelihood approach. In this example, the samples of the time-series are assumed independent and identically distributed (iid) piece-wise constantly. In other words, for a given family of parametric distribution densities {f (•|θ)}, we have ∀i = 1 . . . , n:

x (i) ∼ K k=1 f (•|θ k )1{t k ≤ k < t k+1 }.
One way to learn the parameters of such model, and thus nd the change-point, is to perform maximum likelihood. This is equivalent to taking the following cost function:

c({x (i) } t k+1 i=t k +1 ) = -sup θ t k+1 i=t k +1 log f (x (i) |θ).
The previous example is probably one of the most considered frameworks for changepoint detection [START_REF] Frick | Multiscale change point inference[END_REF][START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF][START_REF] Sen | On tests for detecting change in mean[END_REF]. The piece-wise i.i.d. framework is actually considered in Chapter 4, but with a slightly di erent cost function.

Quality measures

Here again, assuming the access to the true change-points, many metrics has been proposed to evaluate the quality of segmentation algorithms. Among them, metrics based on those of binary prediction (change-point or not) such as the F 1 -score. However, these do not take into account the temporal aspect of the problem and one should therefore prefer Hausdor 's metric. The latter de nes the error h(T , T ) of the set of estimated change-points from the real ones as the greatest temporal distance between a change-point and its prediction:

h(T , T ) max max t ∈T min t∈T |t -t |, max t∈T min t ∈T |t -t | .
Such metric has the advantage to penalize both over-segmentation and under-segmentation.

While the previous metric evaluates the quality of an algorithm empirically, it is also important to evaluate the quality from a theoretical point of view. This is done with the notion of consistency [START_REF] Truong | Selective review of o ine change point detection methods[END_REF] which states that as soon as the number of samples in each segment tends to in nity, we must have P(K = K ) → 0 and n -1 h(T , T ) → 0 in probability.

Graph theory and models for graph vectors

Basic de nitions

Graphs are mathematical objects describing potentially complex systems via a set of interconnected entities, referred as nodes. They appear in many elds and applications, particularly those involving the notion of networks, such as biological networks, neural networks, sensor networks, computer networks, telecommunication networks, social networks, transportation networks... Graphs are then the most widely used tool to describe and model these networks. In the following, we give basic de nitions and concepts of graph theory. De nition 1.1. (Directed graph.) A directed graph G = (V, E) is de ned via a nite set of nodes (or vertices) V = {v 1 , . . . , v p } and a set of edges E ⊂ V × V, i.e. pairs of nodes that are considered neighbors. In particular, if (u, v) ∈ E, we say that u is a parent of v and v is a child of u.

In the following we do not consider self-edges, meaning that for any node u in V, (u, u) / ∈ E. Moreover, we may refer to a node v i ∈ V simply by its index i. Remark 1.1. Although largely assumed, the positive assumptions of the weights is not mandatory. When this is not supposed, it will be speci ed.

De nition 1.2. (Undirected graph.) An undirected graph G = (V, E) is a directed graph whose edge set E is symmetric i.e. ∀(u, v) ∈ E, (v, u) ∈ E. In this context,
In the following, unless speci ed, the graphs are assumed undirected.

De nition 1.4. (Adjacency matrix.) Let G = (V, E) be a graph of size p. Its adjacency matrix A ∈ {0, 1} p×p is a binary matrix whose entries indicate the presence or absence of edge. ∀i, j ∈ {1, . . . , p}:

A ij = 1 if (v i , v j ) ∈ E 0 otherwise.
The adjacency matrix entirely describes its corresponding graph, allowing mathematical manipulations and the formulation of some graph characteristics. For example, the total number of edges of a graph simply corresponds to ||A|| 1 and the degree of a node i is j A ij . A is always symmetric for undirected graphs and its generalization to weighted graphs is the weight matrix W whose entries corresponds to the edges weights. In the following, the graphs are assumed weighted.

De nition 1.5. (Degree and degree matrix.) Let G = (V, E) be a weighted graph of size p with weight matrix W . ∀i ∈ {1, . . . , p}, the degree of the node v i is d i = j W ij . The degree matrix D of the graph is the diagonal matrix that contains all the nodes degrees. In the same way as the adjacency matrix, a Laplacian matrix describes entirely its associated graph. In particular, it is known to carry important topological characteristics of the graph and to be linked with spectral graph theory. For example, the number of eigenvalues of L that are equal to zero corresponds to the number of connected components of the graph. A connected component being a subset of nodes for which there always exist a path between them, and there exist no path with any other node. An illustration of a graph with two connected components is given in Figure 1.8e.

De nition 1.7. (Graph vector.) A graph vector, also referred as graph signal or graph function, is a function x : V → R that assigns a real value to all nodes of a graph G = (V, E). This function can be represented by a simple vector x ∈ R p with x i the value of x at node v i .

This last object allows to de ne vector data that are observed over a network structure, which is a fundamental aspect of the data that we consider throughout the thesis. Nevertheless, a question remains: what the graph bring to the modeling of this type of vectors. Indeed, taken like that, a graph vector remains a simple vector. In the next sections, we present two di erent points of view that answer this question. These two frameworks are both considered in the rest of the thesis.

The Graph Signal Processing framework

Graph Signal Processing (GSP) [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF][START_REF] Shuman | The emerging eld of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF] is a relatively recent eld. Its aim is to extend most of the tools developed in the eld of signal and image processing to graph signals. Thus, notions such as smoothness, sampling or spectral representation of a signal has been extended to cover this type of data. In fact, temporal signal and images are seen as special cases of graph signals where the associated graph corresponds to either a line for a temporal signal or a grid for an image (see Figure 1.8). Within the context of GSP, the graph can now be arbitrary.

In this framework, and similarly to temporal signals or images, the value recorded at a speci c node is seen as a shifted version of the values recorded at its neighbors. In the following, we recall some basic notions of GSP and properties assumed to be shared by most of the graph vectors.

De nition 1.8. (Smoothness.) Let G = (V, E) be a graph of size p with weight matrix W , L be its Laplacian matrix, and y ∈ R p be a graph signal seen over it. We say that y is s-smooth with respect to the graph G if

y T Ly = 1 2 i,j∈[p] w ij (y i -y j ) 2 ≤ s . (1.2) CHAPTER 1. INTRODUCTION
The previous de nition provides a notion of smoothness for graph signals. Intuitively, a graph signal y is s-smooth with respect to G if adjacent nodes of the graph carry su ciently similar signal values. The smaller s is, the smoother the graph signal is. In the GSP framework, graph signals are most of the time assumed smooth with respect to their associated graphs, i.e. with small s values.

Remark 1.2. The particular case where s = 0 implies that all neighboring nodes have the same value.

De nition 1.9. (Graph Fourier Transform.) Let G = (V, E) be an undirected graph with no self-loops, and L = XΛX T be the eigenvalue decomposition of its Laplacian matrix. Then, the Graph Fourier Transform (GFT) of a graph signal y ∈ R p is given by h = X T y, where the components of h are interpreted as Fourier coe cients, the eigenvalues Λ as distinct frequencies, and the eigenvectors X as a decomposition basis. This de nition was initially motivated by the fact that, applied to temporal signals or images, it was recovering the classical Fourier transform. Moreover, it is empirically observed that eigenvectors of X associated to the smallest eigenvalues in Λ are showing less variability across neighboring nodes values than those associated with the biggest, motivating as well the comparison with Fourier analysis.

De nition 1.10. (Spectral sparsity.) Let k ∈ N + , we say that a graph signal y admits a k-sparse spectral representation (equivalently that y is k-bandlimited) with respect to a graph G, if for h = X T y we have

h 0 ≤ k , (1.3)
where h 0 stands for the number of non-zero elements of h.

Regarding this de nition, y admits k-sparse spectral representation if the number of non-zero elements in its Fourier coe cients vector is less or equal to k. k-bandlimitedness is the second property assumed to be shared by most graph vectors of the GSP framework. When k is small, this implies that the entire signal can be recovered from a small number of nodes values. Finally, note that if the smoothness property is also assumed, zero-coe cient will have more chance to be associated with large eigenvalues (i.e. high frequencies). This is in accordance with the spectral domain analysis of standard signals where high variability across neighboring nodes values is mostly explained by noise.

A probabilistic framework

In the previous part, graph functions and their corresponding graph was linked together via properties coming from classical signal processing, namely smoothness and bandlimitedness. In the framework presented here, graph vectors are seen as random vectors drawn from a probability distribution known as Markov Random Fields (MRF). For this type of probability distribution, the graph encodes a particular dependency structure that is explained in the following.

BACKGROUND 29

De nition 1.11. (Conditional independence.) Let X, Y and Z be three real-valued random variables and denote by F X|Z=z (x) (respectively F Y |Z=z (y)) the cumulative distribution function (cdf) of X (respectively Y ) knowing Z = z. We say that X and Y are conditionally independent on Z, denoted X ⊥ ⊥ Y |Z if and only if, ∀x, y, z we have:

F X,Y |Z=z (x, y) = F X|Z=z (x)F Y |Z=z (y).
To some extent, the fact that X and Y are conditionally independent with respect to Z tells us that given Z, then knowing X does not bring any information on the likelihood of Y and conversely.

Remark 1.3. Two conditionally independent variables can be dependent and conversely.

De nition 1.12. (Markov Random Field.) Let G = (V, E) be an undirected graph and X = (X i ) i∈V be a random vector whose entries are indexed by the vertices V. We say that X is drawn from a MRF associated to G if the following properties hold:

(a) X u ⊥ ⊥ X v | X V {u,v} , for any edge (u, v) / ∈ E. (b) X u ⊥ ⊥ X V N (u) | X N (u) , ∀u ∈ V and where N (u) = {v ∈ V : (u, v) ∈ E} is the neighborhood of u. (c) X A ⊥ ⊥ X B | X S ,
for any disjoint subset A, B, S ⊂ V such that S separates A and B i.e. any path from A to B (and conversely) passes through S.

Remark 1.4. It can be showed [START_REF] Lauritzen | Graphical models[END_REF] that (c) ⇒ (b) ⇒ (a). For certain probability distribution, the reverse is true as well, its notably the case of variables admitting a positive density function.

Given the previous de nition, we understand that building MRFs is not straightforward. For this reason, MRFs are often reduced to the class of probability distribution that factorizes, a class of probability distribution that has been shown to validate the three properties required to be an MRF.

De nition 1.13. (Factorization.) Let G = (V, E) be an undirected graph and X be a random vector indexed by V with probability distribution P X (•). Recall that a clique is a subset of nodes that are all connected together, we say that P X (•) factorizes in G if it is of the form:

P X (x) = 1 Z C∈C ψ C (x C ), (1.4)
where C is the set of all cliques of G, ψ C (•) are non-negative potential functions and Z is the normalizing constant.

The potential functions can be arbitrary and, for a given probability distribution that factorizes, are not necessarily de ned uniquely. Famous examples of distributions that factorize, and are therefore MRFs, are the Gaussian Graphical Models and Ising models. The rst one is easy to characterize: for any graph G with weight matrix W , the vector X ∼ N (•, W -1 ) factorizes in G. Ising models are considered in Chapter 4 and reminders are made in it.

CHAPTER 1. INTRODUCTION 4 Organization of the thesis

The thesis is organized as follows. Note that each chapter can be read independently.

• Chapter 2: Anomaly detection in communication networks: applications to Sigfox. This chapter is essentially dedicated to the resolution of the rst objective described in Section 2. The anomaly detection task is taken from the angle of activity monitoring in a communication network. In other words, anomalies are spotted based on e.g. an abnormal number of interactions, amount of exchanged information, number of received signals, etc. This very general framework allows it to be applicable to a large class of communication networks, including Sigfox.

In the chapter, we rst brie y overview the literature on anomaly detection in networks, emphasizing on the graph representation aspects behind it. We also present a simple novelty detection algorithm that aims to detect abnormal levels of communication activity at the level of a node. This algorithm relies on the intuition that the level of activity of a node can be determined or predicted by looking at the level of activity recorded at its neighboring nodes. Thanks to having access to a 'normal' data set and conventional supervised learning methods, the relationship between nodes activity can be learned. Afterwards, an anomaly is detected when the predicted level of activity is far from the real one. This method is showed to perform well on both synthetic data and data coming from Sigfox network, which allows us to conclude to the resolution of the rst objective. In addition we will see that the presented approach is linked with some other objectives, particularly with the task of graph-based event detection and graph inference that are core-subjects of the thesis and are investigated in the remaining chapters.

• Chapter 3: Structure inference from smooth and bandlimited graph signals.

In this chapter we consider the problem of learning the underlying structure of a set of graph vectors, i.e. the graph on which they are observed . This chapter is thus linked with the third objective described in Section 2. The graph vectors are assumed to enjoy a sparse representation in the graph spectral domain, a feature which is known to carry information related to the cluster structure of a graph. The signals are also assumed to behave smoothly with respect to the underlying graph structure. For the graph learning problem, we propose a new optimization program to learn the Laplacian of this graph and provide two algorithms to solve it, called IGL-3SR and FGL-3SR. Based on a 3-steps alternating procedure, both algorithms rely on standard minimization methods -such as manifold gradient descent or linear programmingand have lower complexity compared to state-of-the-art algorithms. While IGL-3SR ensures convergence, FGL-3SR acts as a relaxation and is signi cantly faster since its alternating process relies on multiple closed-form solutions. Both algorithms are evaluated on synthetic and real data.

• Chapter 4: Detecting changes in the graph structure of a varying Ising model. This last chapter addresses the demands of the last two objectives de ned above. It adopts the probabilistic framework for the modeling of graph vectors and assumes them to be drawn from an Ising model. In particular, the chapter focuses on the 5. PUBLICATIONS estimation of multiple change-points in a time-varying Ising model that evolves piecewise constantly. The aim is to identify both the moments at which signi cant changes occur in the Ising model, as well as the underlying graph structure of each segment of the signal i.e. the part between two change-points. For this purpose, we propose to estimate the neighborhood of each node by maximizing a penalized version of its conditional log-likelihood. The objective of the penalization is twofold: it imposes sparsity in the learned graphs and, thanks to a fused-type penalty, it also enforces them to evolve piece-wise constantly. Using few assumptions, we provide two changepoints consistency theorems. Those are the rst in the context of unknown number of change-points detection in time-varying Ising model. Finally, experimental results on several synthetic data sets and real-world examples demonstrate the performance of our method.

Publications

The work presented in this manuscript has resulted in publications and submissions in international conferences and journals:

1 Detecting anomalies in networks: a graph perspective

Introduction

Over the past years, thanks to an increasing availability of data and tools to analyze them, detecting anomalies in communication networks has become an important area of research. This task, which looks for events that deviates from the normal network behavior, arises in many network related problems such as monitoring and security [START_REF] Ahmed | A survey of network anomaly detection techniques[END_REF][START_REF] Thottan | Anomaly detection approaches for communication networks[END_REF].

In the present chapter, we refer to communication network, or simply network, as a set of entities that can interact or exchange information between each other. This simple formulation allows to cover a large spectrum of networks such as computer networks [START_REF] Huang | In-network PCA and anomaly detection[END_REF][START_REF] Xie | Anomaly detection in wireless sensor networks: A survey[END_REF], e-mail networks [START_REF] Wan | Link-based event detection in email communication networks[END_REF], telecommunication network [START_REF] Bars | A probabilistic framework to node-level anomaly detection in communication networks[END_REF] or social networks [START_REF] Heard | Bayesian anomaly detection methods for social networks[END_REF][START_REF] Ji | Incremental local evolutionary outlier detection for dynamic social networks[END_REF]. However, with such a wide range of networks types, notions of interaction and anomaly may di er a lot. For example, in computer or wireless networks, an interaction refer to bits of information that pass from one node to another (possibly via other nodes) and an anomaly can refer to a network attack. In an e-mail or a social network, the interaction are directly node-to-node and an anomaly can refer to account hacks for example.

The standard way to detect anomalies in networks [START_REF] Ahmed | A survey of network anomaly detection techniques[END_REF] is to, rst, preprocess the available data (data engineering step) and then apply standard anomaly detection techniques, such as Local Outlier Factor [START_REF] Breunig | Lof: identifying density-based local outliers[END_REF] or One-Class SVM (see the background in Chapter 1), over those preprocessed data. The way those two steps are performed may depend on the labeling of the data (e.g. with or without labeled anomalies), the kind of available data (e.g. the content of the exchanged information: images, text...), and for example, the ability or not to build a statistical model from them. We can see from this simple procedure that the most important part is the preprocessing step, which obviously depends heavily on the type of considered network, the content of the exchanged information and the type of anomalies we aim to discover. In the next section, we consider a framework to describe and preprocess the data which is adapted to many di erent situation and which requires data usually available at low cost.

Representing the network activity via dynamic graphs

An intuitive way to represent a communication event involving a set of entities in a network is via a graph. In such a graph, the nodes correspond to all the entities of the network and the edges indicate the structure of the event i.e which node directly interacted with which other node. Taking Sigfox network as an example, a message sent from a device to a set of surrounding Base Station, can be represented by a graph with edges connecting the involved BSs to the device, or simply connecting the BSs together if we omit the device (see gure 2.1). With this formulation, the stream of communication events generates a graph evolving over time, with edges appearing and disappearing every time a communication event begins or nishes [START_REF] Latapy | Stream graphs and link streams for the modeling of interactions over time[END_REF].

In practice, analyzing such a stream of interactions can be di cult and a standard approach is to analyze the network activity via a time-discretized series of graphs [START_REF] Corneli | Multiple change points detection and clustering in dynamic networks[END_REF]. In other words, the time is discretized (over a daily basis for example) and at each timestamp, a graph aggregating all the interactions that occurred at the corresponding period of time is built. Usually, the weights of the graph edges counts the number of interactions that occurred (or the amount of data shared) between two nodes over the speci c timestamp (see Figure 2.2). In addition to being adapted to many communication networks, the use of this graphrepresentation is motivated by the fact that, in reality, content-speci c features of the exchanged information are usually kept undisclosed so as to preserve privacy (e.g the content of the e-mail). Furthermore, it captures the structure of the network such as clusters or isolated nodes. Consequently, most studies on network anomaly detection only deal with this time-series of graphs representation of their data to build the features [START_REF] Corneli | Multiple change points detection and clustering in dynamic networks[END_REF][START_REF] Heard | Bayesian anomaly detection methods for social networks[END_REF][START_REF] Priebe | Scan statistics on Enron graphs[END_REF][START_REF] Wan | Link-based event detection in email communication networks[END_REF] and can thus rely to the vast literature on dynamic graph anomaly detection, brie y recalled is the next section. Although adapted to many kinds of networks, this simple data-representation has nevertheless some limitations. First, it does not allow to recover every types of anomalies, but mostly those related to the communication activity and the network structure. Furthermore, the aggregated representation loses information about interactions involving more than two nodes (e.g. multiple receivers) since edges keep in track only node-to-node interactions.

Recall on the anomaly detection for dynamic graphs

The task of anomaly detection in a time-series of graphs, referred as dynamic graph [START_REF] Ranshous | Anomaly detection in dynamic networks: a survey[END_REF] is closely related to the one of classical anomaly detection for temporal data. Similarly, the goal is to nd a subset of graphs that deviates from the majority. The standard approach consists in computing several graph features (e.g. degree of a node, edges weights, centrality measures, shortest path etc.) over each graph, and then apply classical anomaly detection techniques on the derived set of vector data [START_REF] Chandola | Anomaly detection: A survey[END_REF][START_REF] Cheng | Detection and characterization of anomalies in multivariate time series[END_REF][START_REF] Gupta | Outlier detection for temporal data: A survey[END_REF]. The methods we can nd in the literature di er in the same way as in the classical case. They vary notably according to the availability of the labels i.e supervised, semi-supervised (access to a dataset of normal data) [START_REF] Di | Complexity-penalized estimation of minimum volume sets for dependent data[END_REF][START_REF] Schölkopf | Estimating the support of a high-dimensional distribution[END_REF][START_REF] Scott | Learning minimum volume sets[END_REF] or unsupervised (no label available) [START_REF] Breunig | Lof: identifying density-based local outliers[END_REF][START_REF] Huang | In-network PCA and anomaly detection[END_REF], the type of used method i.e probabilistic model-based [4,[START_REF] Corneli | Multiple change points detection and clustering in dynamic networks[END_REF][START_REF] Heard | Bayesian anomaly detection methods for social networks[END_REF][START_REF] Neil | Scan statistics for the online detection of locally anomalous subgraphs[END_REF][START_REF] Peel | Detecting change points in the large-scale structure of evolving networks[END_REF][START_REF] Priebe | Scan statistics on Enron graphs[END_REF][START_REF] Wang | Locality statistics for anomaly detection in time series of graphs[END_REF], distance based [START_REF] Breunig | Lof: identifying density-based local outliers[END_REF] etc.

Apart from the graph-based feature engineering, the dynamic graph's anomaly detection also di er from the classical case in the notion of scale of abnormality. In fact, the anomaly can be spotted at the level of a node [START_REF] Heard | Bayesian anomaly detection methods for social networks[END_REF][START_REF] Ji | Incremental local evolutionary outlier detection for dynamic social networks[END_REF], a subgraph [START_REF] Neil | Scan statistics for the online detection of locally anomalous subgraphs[END_REF], or the whole graph-level [START_REF] Corneli | Multiple change points detection and clustering in dynamic networks[END_REF]. For a complete survey on dynamic graph's anomaly detection, the reader should refer to the survey of [START_REF] Ranshous | Anomaly detection in dynamic networks: a survey[END_REF].

A regression-based novelty detector

In this section, we describe an intuitive way to detect abnormal level of communication activity recorded at the level of a node. The proposed approach rely on the intuition that networks often has an underlying structure with clusters and neighborhood, meaning that a node usually communicates with a certain subset of other nodes. Based on this intuition, the idea of the presented approach is to consider the communication activity recorded at a node as abnormal by looking at its neighbors activity. More precisely, we assume that it is possible to predict the activity of a node, based on the level of activity of the remaining nodes. Before going into further details, let de ne our model and the objectives of the detection task.

Model description

We consider a communication network with N entities, referred as nodes, that can interact between each other. In this network, we assume that a communication event can occur at any moment, and can be shared by many nodes at the same time.

De nition 2.1. (Communication event): A communication event is entirely described by an N -dimensional binary vector X = (X 1 , . . . , X N ) ∈ {0, 1} N , referred as ngerprint, indicating which node took part in the event. X j = 1 if node j is involved in the event, and 0 otherwise.

With such a de nition of communication event, all the involved nodes are assumed to communicate equally between each others, with no notion of roles (e.g. sender or receiver). From a graph point of view, a communication event thus creates a clique containing all the 2. A REGRESSION BASED NOVELTY DETECTOR 37 involved nodes.

Since no notion of roles nor protocol are assumed, many examples of communication networks would t this simple setting. At Sigfox for example, each Base Station corresponds to a node, a message corresponds to a communication event, where the ngerprint X indicates which BS has received the message (X j = 1 means that the j-th BS has received it). We can also think of an e-mail network, where each node corresponds to an e-mail address, an e-mail corresponds to an event, where the corresponding ngerprint indicates which email addresses received it. Last but not least, the network can correspond to a company where each node is an employee and a communication event corresponds to a meeting.

Assumption 2.1. An event X corresponds to a random vector of size N with probability distribution denoted P X . Given a set of communication events {X (1) , . . . , X (n) }, we assume that the communication events are all independent (but not necessarily identically distributed).

De nition 2.2. (Conditional probability function): Let X j be the ngerprint of the event X that indicates the participation of all nodes except node j. Let x j be a realization of X j and denote η * j (x j ) the probability that node j participates in the event X, provided the ngerprint x j :

η * j (x j ) P(X j = 1|X j = x j ) (2.1) = E [X j |X j = x j ] .
Knowing the ngerprint over all the other nodes allows us to express the behavior of node j as a Bernoulli random variable:

X j ∼ B η * j (x j ) . (2.2)
Assumption 2.2. All the considered communication events have the same conditional probability functions. In other word, let {X (1) , . . . , X (n) } be any set of communication event, then, for any node j and any xed ngerprint

x j ∈ {0, 1} N -1 , we have ∀i = [n], X (i) j iid ∼ B η * j (x j ) .
The previous assumption states that even if the events are non identically distributed, the conditional probability function, and thus the dependency structure between the nodes, is the same for each event. The aforementioned assumption can be understood easily for Sigfox. Since the network is constantly evolving, with the appearance of new devices, at di erent location, the joint probability distribution of an event can change easily. However, the BS are spatially distributed on earth and the fact that a BS receives or not a message, highly depends on the fact that its neighboring BS has received it or not. Since their positions are xed, one can easily imagine that the dependency structure this spatial proximity implies will not change.

CHAPTER 2. ANOMALY DETECTION IN COMMUNICATION NETWORKS:

APPLICATIONS TO SIGFOX

Objective and ideal scoring function

Let D n = {X (i) } n i=1 be a set of n communication events. Let M n j = n i=1 X

(i) j be the random variable of the number of events recorded at a node j ∈ [N ]. Our objective is, for a xed node j ∈ [N ], to determine if the observed volume of events in which that node participates, denoted m n j = n i=1 x (i) j , is abnormal. Assuming the access to all the probability distributions, a way to perform this task is to provide con dence levels for M n j and look whenever m n j stands in a region of low probability.

In the present approach, we propose to use the knowledge brought by the ngerprints of the other nodes and build these con dence levels knowing them. Thus, an anomaly will be detected if, regarding the communication activity recorded at every node but j, the number of event the node j has participated in is too low are high. Below we de ne our ideal scoring function, assuming the access to the underlying probabilistic model. De nition 2.3. The conditional cumulative distribution function of M n j , knowing the ngerprints {x

(i) j } n
i=1 is denoted by:

F M n j (m) = P M n j ≤ m ∀i = 1, . . . , n, X (i) j = x (i) j (2.3)
Knowing the ngerprints of the other nodes, M n j corresponds to a sum of n Bernoulli random variables, with parameter p

(i) j η * j (x (i) j ) for i ∈ [n]
. This distribution is called Poisson Binomial [START_REF] Hong | On computing the distribution function for the poisson binomial distribution[END_REF][START_REF] Tang | The poisson binomial distribution-old & new[END_REF] and its cumulative distribution function (cdf) has a closed-form solution given by:

F M n j (m) = m k=1 A⊂[n],|A|=k i∈A p (i) j l∈A c (1 -p (l) j ) (2.4)
This function can be computed e ciently via the method presented in [START_REF] Hong | On computing the distribution function for the poisson binomial distribution[END_REF], it is based on the computation of the Discrete Fourier Transform of the characteristic function of the Poisson Binomial distribution. We refer by PoiBin(m; p

s(m n j ) = max F M n j (m n j ), 1 -F M n j (m n j ) (2.5)
Such scoring function is in [0, 1] and the closer to 1 is s(m n j ), the more m n j is considered abnormal. Indeed, when the rst term in the max of equation 2.5 is close to one, it means that m n j stands in the right-hand tail of the distribution of M n j , which encode an abnormally high number of recorded events at node j. On the opposite, a high value of the second term will encode an abnormally low number of observed communication events.

A REGRESSION BASED NOVELTY DETECTOR 39

A supervised learning framework

In practice, we do not have access to the true conditional probability function η * j (•) and we propose to estimate it in order to build a proper scoring function. To this end, we suppose we have access to a training set of communication events D train , assumed to have been recorded at times of normal communication behavior. With our de nition of η * j (•) (De nition 2.2), the estimation problem refers to the task of estimating a conditional probability function. Under our speci c modeling, η corresponds to the conditional expectation of X j , knowing the value of X j and thus can be seen as the function regressing X j using X j . This regression function can be learned using the healthy dataset D train and any regression algorithm outputting values between 0 and 1 (e.g logistic regression, binary trees, random forest etc.). Such regression algorithm is referred as Regressor in algorithm 2.1, it takes as input a set of observed variables {x (i) , y (i) } where x stands for the explanatory variable and y for the target one and outputs the corresponding regression function.

Let η j (•) be our estimated regression function of X j given X j . To build our anomaly scoring function, we propose to simply replace p (i) j by p (i) j η j (x (i) j ) in the conditional cumulative distribution function (cdf) of Eq. 2.4 and replace this new version of conditional cdf in Eq. 2.5. In practice, outputting a score is not enough to spot an anomaly, and one has to x a threshold s above which the score will be considered abnormal. While this threshold is not always easy to x in practice, the fact that our scoring function belongs to [0, 1] and has a probabilistic interpretation allows us to x it intuitively. Indeed, if the conditional cdf is well estimated, our scoring function allows to control the false positive rate: xing the threshold to s ∈ [0, 1] will result in a false positive rate of 1s. Thus, relying to the standard threshold of statistical test theory, it can xed to e.g 0.95 or 0.99.

In practice, the η j function may not be perfectly estimated. Another way to x the threshold and control the false positive rate is via cross-validation on the training set of communication events. To do so, one has to x an acceptable false positive rate (e.g 0.05 or 0.01), then via cross-validation nd the value of threshold s that generates a false positive rate close to that xed value.

Summary

To sum up, we observe an healthy set of communication events D train . For any xed node j, we learn the probability that this node will take part in an event, given the ngerprint of the other nodes. This is done using D train and any regression algorithm. We observe a new set of communication events, D n , for which we want to know if the number of events recorded at node j is abnormally low or high. We evaluate for each event in D n the probability that j will take part in the corresponding event and evaluate the anomaly score as described upper. If the score is above a certain threshold, the anomaly is spotted. The overall procedure is summarized in Algorithm 2.1. APPLICATIONS TO SIGFOX Algorithm 2.1 Node-wise network anomaly detection Input: D train , D n , node j, threshold s Regression algorithm:

Regressor(•) Output: 1 if anomaly, 0, otherwise η ←-Regressor D train = {x (i) j , x(i) j } for i = 1 . . . , n do pi ←-η(x (i) j ) end for F ←-PoiBin x (i) j ; p (

Applications

In the present section we propose to evaluate the performance of the regression-based approach presented in the previous section. The evaluation is performed over two datasets. The rst one is simulated in order to imitate the communication activity recorded in network like Sigfox and the second one is made of real-word communication events recorded at Sigfox. These experiments aim to show the importance of considering the activity recorded at other nodes while performing anomaly detection at a speci c node. In addition, the objective is to show the superiority of the regression-based approach compared to the methods presented in the following paragraph.

Comparative methods We compare our regression-based approach with three other methods related to the graph-based algorithms presented in Section 1. All three methods split the training dataset in order to build a set of graphs on which features are engineered. Each graph encodes the communication activity of the network over the di erent subdataset and are build in the same manner: the edges count the number of shared events between two nodes and a node's value captures the total number of events it has participated in. Similarly, a unique graph is build over the dataset for which we want to known if it is abnormal or not. In practice, the datasets used to build the graphs can be of di erent sizes and the di erent values in the graphs are normalized by the number of events contained in each dataset.

The comparative methods works the same way. A feature vector aiming to characterize the behavior of the node of interest is build over each graph. Then, a One-class SVM [START_REF] Schölkopf | Estimating the support of a high-dimensional distribution[END_REF] (see chapter 1) builds an hyperplane separating normal feature vectors and abnormal ones using the set of normal vectors build from the training set of graphs. The anomaly score of the graph of interest simply corresponds to the distance of the corresponding feature vector to the separating hyperplane.

The three methods only di er in the feature vector build from each graph. The rst one only looks at the value of the node of interest i.e the number of events it has participated in. The second one calculates the weighted degree of the node of interest. The last one evaluates the di erence between the value of the node of interest and the value of its nearest neighboring nodes. The latter is motivated by the intuition that some nodes will have highly correlated variations of activity. It is actually the only multivariate feature vector. The nearest neighbors are selected by looking at the nodes with whom the node of interest shares more events in the training dataset.

Computational remarks All the following experiments has been implemented in Python and performed on a personal laptop computer. The conditional probability functions dened in 2.1 are learned via a Random Forest Regressor [START_REF] Breiman | Random forests[END_REF], using the version implemented in scikit-learn [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]. Same for the One-class SVM algorithm needed in the comparative methods, also implemented in scikit-learn. For both algorithms, the scikit-learn's by default set of hyperparameters are used.

To build the scoring function of equation 2.5 we have to be able to compute the cumulative distribution function of a Poisson Binomial distribution (equation 2.4). This can be done e ciently via the method presented in [START_REF] Hong | On computing the distribution function for the poisson binomial distribution[END_REF], implemented in the Python package PoiBin1 .

Simulated experiment

As stated upper, the goal of the following experiments is to apply the di erent approaches on datasets that simulate the communication activity of a network like Sigfox. We keep the nodes two-dimensional spatial arrangement of Sigfox network and propose the following simulation process.

Simulation process

Sampling the spatial network structure. Draw N node's two-dimensional locations (i.e. analogous to BSs) according to a mixture model M of K bivariate Gaussian distributions. The mixture model allows to simulate clusters of nodes, which is observed in particular at Sigfox (e.g one city corresponds to a cluster of BS).

Sampling a communication event. First generate an event location (analogous to a device's location) drawn from M, as in the previous step. Then for each node v ∈ [N ], let its location x v and draw a Bernoulli with a parameter inversely proportional to the distance d(x v , ). Here, the Bernoulli indicates if the node has participated or not in the event. In our experiments, we set the Bernoulli parameters to be equal to exp(-

1 σv d(x v , ))
, where σ v is a node-dependent visibility parameter that controls how much the node v participates in the di erent events. With such modeling we can see that the closer the location of the event is to a node, the higher is the chance for the node to participate in the event. Furthermore, nearby nodes will have more chance to participates in the same events, which is often observed at Sigfox. Simulating a normal dataset. Draw n independent ngerprints following the process described in the previous paragraph. APPLICATIONS TO SIGFOX Simulating an abnormal dataset. One way to simulate an anomaly at the level of a node v is to make this node participating in less (or more) events than usual. One way to do it is by changing its visibility parameter σ v . The more di erent this parameter is from the normal network behavior, the easier the anomaly will be spotted. Then, the dataset is drawn the same way as a normal one.

Three levels of abnormality

In order to demonstrate the performance of our method, we apply the above generative process in three di erent situations with di erent anomaly 'complexity', whereas sharing the following properties:

• N = 100 communication nodes are drawn from a mixture of K = 10 bivariate Gaussian variables with equal mixture's weights. Each center are drawn uniformly over the square [-5, 5] × [-5, 5] and the covariance matrices always correspond to the identity matrix of size two.

• The training set contains 20000 communication events. It is equally split in 100 blocks of 200 events for the comparatives methods. The visibility parameters used for each node is the same and is equal to 2.6.

• 200 testing dataset are sampled. Half of them are normal and sampled according to the same process as the training dataset, the other half is abnormal. Each testing dataset have di erent size where the number of events is drawn from a uniform distribution between 100 and 300.

• A single arbitrary node is chosen to be anomalous and it is the same for each abnormal testing dataset. The anomaly is simulated by a decreasing in the visibility parameter. This is in accordance with the idea that an anomaly at the level of a Base Station implies a decreasing in the number of observed signals.

The three experiments thus only di er in the decreasing of the visibility parameter. The rst experiment, referred as easy imposes a important decreasing and replace the visibility parameter of the corresponding node by 1.3, the second one, referred as normal, replace it by 1.62 and the last one, referred as hard replace it by 1.95.

Results

The empirical performances of the di erent approaches is visualized on the ROC curves of gure 2.3. The regression-based approach is evaluated via two scoring functions: a Bilateral and a Unilateral. The Bilateral corresponds to the one de ned in the previous section (equation 2.5). However, the Unilateral score only evaluates the right-hand element in the maximum of equation 2.5. Thus, it only looks for abnormally low level of the node's events participation: the bigger the score is, the more the number of event the node has participated in stands in the left-hand tail of the Poisson Binomial distribution. The use of the Unilateral score is motivated by the fact that the anomalies we simulate imply a decreasing in the number of event's participation. Using it should improve the rate of false positive. The number of neighbors necessary to compute the third SVM-based algorithm has been selected to be the one with best performances and was xed at 7. Several conclusions can be made from the ROC curves. First of all, no matter the complexity of the experiment, the regression-based method always outperforms the Oneclass SVM-based approaches. Indeed, the ROC curves associated to the rsts are always above the latter, leading to better Area Under the Curve (AUC). Moreover, we must note that as expected, the Unilateral score is slightly better than the Bilateral.

Among the three SVM-based approaches, several observation must also be made. The method looking at the weighted degree appears to be the worst, being even inferior to the simplest approach which is based on the node's value. This illustrates an important point: building naive features, even graph-based, won't solve an anomaly detection problem like this one if they are not adapted to it. On the contrary, the one with the most sophisticate features, which looks at the di erence between the node and its nearest neighbors, outperforms the other two. This illustrates the fact that considering the other nodes values is important while performing the detection task. Moreover, it shows that being able to select only a subset of important nodes (here 7), referred as neighbors, improves again the performance.

Finally, it appears that our method is slightly more robust to the increasing complexity of the di erent experiments. Indeed, we observe a decreasing of 0.04 (respectively 0.14) between the AUC of the easy and the normal (respectively hard) experiments for the Unilateral score. This decreasing is more important for the "di erence with neighbors"'s One-class SVM, with a decreasing of 0.12 (respectively 0.25) between the AUC of the easy and the normal (respectively hard) experiments.

Sigfox application

We now propose to verify that the conclusions we made from the previous experiments remain valid in a real-world application. In particular, we apply the di erent methods to a set of communication events recorded at Sigfox.

Dataset description

In the present experiment, we have access to a set of approximately 232000 Sigfox's messages recorded at the level of N = 34 Base Stations over a period of 5 months (from January to June of 2017). Relying to the framework described in Section 2, each BS characterizes a node and each message corresponds to a unique communication event, where the ngerprint indicates which BS has received the message. All the BSs are located near Toulouse, France and their positions can be found in Figure 2.4. Among those BSs, one has been spotted as abnormal by Sigfox's experts (in red on Figure 2.4), approximately from march, while the others has been considered working well.

The objective of the experiment is to learn the normal behavior of the red BS over the month of January (approx. 35000 messages) and then tell if its behavior is abnormal over the remaining period. The same way it would be done in a realistic situation at Sigfox, the prediction is made daily over the remaining 4 months, resulting in 120 test dataset and therefore predictions. Over each day (i.e dataset) of the testing period, approximately 1600 messages are observed in average, but this number can highly vary from one day to another.

As stated upper, the red BS is considered abnormal by the experts approximately from March. This results in 90 abnormal testing dataset and 30 normal ones. To increase a bit the size of the experiment and make the classes more balanced, we also apply the exact same learning and testing process for a normal BS (in green on Figure 2.4). In the end, 240 test dataset are evaluated as abnormal or not, including 90 days considered abnormal and 150 normal ones.

Finally, to be able to apply the comparative methods, the learning dataset is split according to the daily basis of the testing period. Hence, 31 graphs and consequently 31 feature vectors are build to learn the normal behavior of a node over a day. For each built graph, the nodes carry the number of messages the associated BS has received during the corresponding day and the edges encode the number of messages received in common between two BSs.

Results

First results and comparison between the di erent methods are evaluated via the ROC curves in Figure 2.5. Several conclusions we made from the experiments on simulated data can be made as well. First, the regression-based method with unilateral score clearly outperforms the others, illustrating its superiority for this type of tasks. Moreover, among the SVM-based approaches, the one that looks at the di erence between the node and its nearest neighbors (all the other BS were taken this time), clearly outperforms the other two. We can even observe that this method is slightly better than the Bilateral score with an AUC of 0.89 for itself and an AUC of 0.84 for the latter. According to the ROC curves, this seems to come from the fact that the Bilateral method gives an important anomaly score to high value of received messages.

To visualize a bit better the performance of the regression-based algorithm, we propose 2 0 1 7 -0 3 -0 1 2 0 1 7 -0 3 -1 5 2 0 1 7 -0 4 -0 1 2 0 1 7 -0 4 -1 5 2 0 1 7 -0 5 -0 1 2 0 1 7 -0 5 -1 to x a threshold above which the score is considered abnormal, plot the "con dence intervals" associated to it and look whenever the number of messages received by the BS of interest lays out of them (Figure 2.6). In other word, for each day of the testing period, we look for the interval of values for which the score is not exceeded. If the number of messages received by the antenna does not belong to these values, the anomaly is detected. The threshold is x via cross validation on the training dataset, in order to obtain a false positive rate of 0.01 over it. While the number of messages received by the green BS should never go out the intervals, the red one should have an exceeding score for the abnormal days, indicated between the two red lines of Figure 2.6a and 2.6b. Those gures tells us several things. The high amount of false positive appearing in the Bilateral method were essentially coming from high values of received messages by the normal BS and are corrected by using the Unilateral approach (Figure 2.6d). We also observe that the total number of messages received by a BS, and therefore the total number of messages sent in the network, can highly variate: this seems to be a reason of the high false positive rate of the Bilateral method and may indicate that the learning phase was not including enough examples. Finally, looking at how regularly the number of messages received by the abnormal BS goes out the intervals, we could ask ourselves if the anomaly began a bit later, around March the 15th.

2 0 1 7 -0 2 -0 1 2 0 1 7 -0 2 -1 5 2 0 1 7 -0 3 -0 1 2 0 1 7 -0 3 -1 5 2 0 1 7 -0 4 -0 1 2 0 1 7 -0 4 -1 5 2 0 1 7 -0 5 -0 1 2 0 1 7 -0 5 -
2 7 -0 2 -0 1 2 0 1 7 -0 2 -1 5 2 0 1 7 -0 3 -0 1 2 0 1 7 -0 3 -1 5 2 0 1 7 -0 4 -0 1 2 0 1 7 -0 4 -1 5 2 0 1 7 -0 5 -0 1 2 0 1 7 -0 5 -1 5 2 0 1 7 -0 6 -
2 0 1 7 -0 2 -0 1 2 0 1 7 -0 2 -1 5 2 0 1 7 -0 3 -0 1 2 0 1 7 -0 3 -1 5 2 0 1 7 -0 4 -0 1 2 0 1 7 -0 4 -1 5 2 0 1 7 -0 5 -0 1 2 0 1 7 -0 5 -1 5 2 0 1 7 -0 6 -
Overall, we can conclude that the proposed regression-based method is adapted the this kind of problem.

Conclusion and discussion

In conclusion, this chapter has enabled us to propose a simple and e ective method for detecting anomalies in a communication network. The rst objective we had set ourselves has thus been ful lled, and in particular the objective of the industrial collaboration with Sigfox. Some links can also be made with the other objectives, i.e. the problem of detecting events in a set of graph signals and the graph inference task. Indeed, for the rst, characterizing the anomaly of a node based on the values at the other nodes implies a notion of structure underlying the data, implicitly used by the considered approach. Therefore, one could imagine this type of anomaly detection for other types of graph vector data, other than that of a communication network. Also, we have seen that the graph can be used to determine the neighborhood of a node and thus improve the performance of certain methods. This is notably the case of the last approach based on the one-class SVM, but also ours. Indeed, the graph can be used for dimensionality reduction. Therefore, instead of using all the other nodes during the learning phase, one could only use the neighbors of the considered node.

Finally, a link can also be made with the task of graph inference. With an appropriate regression algorithm, the learning step of the presented approach can be seen as the inference of the underlying neighborhood of node j. Indeed, in some sense, we are looking for the nodes that interacts often with j. In particular, a 1 -regularized logistic regression would perfectly match the Ising model structure inference presented by [START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF]. With such approach, the probability function η j is modeled via a sigmoid of a linear combination of the other nodes ngerprints. Thus, each weights is related to a node and the bigger a weight is, the stronger the interaction between j and the corresponding node is (see chapter 4 for more details). The task of graph inference is properly investigated in the next chapter.

CHAPTER 3. STRUCTURE INFERENCE FROM SMOOTH AND BANDLIMITED GRAPH SIGNALS

Abstract

In this chapter, we consider the problem of learning a graph structure from multivariate signals, known as graph signals. Such signals are multivariate observations carrying measurements corresponding to the nodes of an unknown graph, which we desire to infer. They are assumed to enjoy a sparse representation in the graph spectral domain, a feature which is known to carry information related to the cluster structure of a graph. The signals are also assumed to behave smoothly with respect to the underlying graph structure. For the graph learning problem, we propose a new optimization program to learn the Laplacian of this graph and provide two algorithms to solve it, called IGL-3SR and FGL-3SR. Based on a 3-steps alternating procedure, both algorithms rely on standard minimization methods -such as manifold gradient descent or linear programming-and have lower complexity compared to state-of-the-art algorithms. While IGL-3SR ensures convergence, FGL-3SR acts as a relaxation and is signi cantly faster since its alternating process relies on multiple closed-form solutions. Both algorithms are evaluated on synthetic and real data. They are shown to perform as good or better than their competitors in terms of both numerical performance and scalability. Finally, we present a probabilistic interpretation of the optimization program as a Factor Analysis Model.

Associated publications:

Learning Laplacian Matrix from Bandlimited Graph Signals [START_REF] Bars | Learning laplacian matrix from bandlimited graph signals[END_REF]

Introduction

Hidden structures in multivariate or multimodal signals can be captured through the notion of graph. The availability of such a graph is a core assumption in many computational tasks such as spectral clustering, semi-supervised learning, graph signal processing, etc. However, in most situations, no natural graph can be derived or de ned and the underlying graph must be inferred from available data. This task, often referred to as graph learning, has received signi cant attention in elds such as machine learning, signal processing, biology , meteorology, and others [START_REF] Friedman | Sparse inverse covariance estimation with the graphical lasso[END_REF][START_REF] Hecker | Gene regulatory network inference: data integration in dynamic models -A review[END_REF][START_REF] William | Representation learning on graphs: Methods and applications[END_REF]. Learning a graph is an ill-posed problem as several graphs can explain the same set of observations. Previous works have been devoted to introducing underlying models or constraints that would narrow down the range of possible solutions. For instance, physical constraints can be imposed to suggest epidemic models or other information propagation and interaction models [START_REF] Du | Learning networks of heterogeneous in uence[END_REF][START_REF] Gomez-Rodriguez | Estimating di usion networks: Recovery conditions, sample complexity & soft-thresholding algorithm[END_REF][START_REF] Rodriguez | Uncovering the temporal dynamics of di usion networks[END_REF]. From a statistical perspective, the graph learning task is seen as the estimation of the parameters of a certain probability distribution parametrized by the graph itself. Generally, the assumed class of distributions is either a Bayesian Network in the case of directed graph, or a Markov Random Field for undirected graphs 1. INTRODUCTION 51 [START_REF] Koller | Probabilistic graphical models: principles and techniques[END_REF][START_REF] Tarzanagh | Estimation of graphical models through structured norm minimization[END_REF][START_REF] Wang | Inference for high-dimensional exponential family graphical models[END_REF][START_REF] Yang | Graphical models via univariate exponential family distributions[END_REF]. Hence, the graph structure encompasses the conditional dependencies between variables. Two variables will be connected in the graph if they are dependent conditionally on all the other variables. In the particular case of Gaussian Random Fields, the graph estimation consists in the estimation of the inverse covariance matrix, known as the precision matrix [START_REF] Banerjee | Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data[END_REF][START_REF] Friedman | Sparse inverse covariance estimation with the graphical lasso[END_REF]. In the latter reference, the proposed estimation method corresponds to the well-known Graph-Lasso algorithm, which relies on the assumption that the precision matrix is subject to a sparsity constraint.

More recently, Graph Signal Processing (GSP) [START_REF]Cooperative and Graph Signal Processing -Principles and Applications[END_REF][START_REF] Shuman | The emerging eld of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF], has generalized the standard concepts and tools of signal processing to multivariate signals recorded over graph structures. Notions such as smoothness, sampling, ltering, etc., have been adapted to this framework, opening a new eld that paves the way to further developments in graph learning [START_REF] Pasdeloup | Characterization and inference of graph di usion processes from observations of stationary signals[END_REF][START_REF] Thanou | Learning heat di usion graphs[END_REF]. In this framework, the smoothness of observations with respect to the true underlying graph is a common assumption [START_REF] Chepuri | Learning sparse graphs under smoothness prior[END_REF][START_REF] Daitch | Fitting a graph to vector data[END_REF][START_REF] Dong | Learning graphs from data: A signal representation perspective[END_REF][START_REF] Egilmez | Graph learning from data under structural and Laplacian constraints[END_REF][START_REF] Kalofolias | How to learn a graph from smooth signals[END_REF], which asks for graphs on which signals have small local variations among adjacent nodes. Another naturally arising property of real-world problems is the sparsity of the observations in the graph spectral basis [START_REF] Sardellitti | Graph topology inference based on sparsifying transform learning[END_REF][START_REF] Valsesia | Sampling of graph signals via randomized local aggregations[END_REF]. In data clustering, for instance, the vector of labels seen as a signal over the vertices of a graph, exhibits a sparse spectral representation. It is smooth within each cluster and varies across di erent clusters (Figure 3.1). Hence, building such graph is relevant for graph-based clustering approaches, such as spectral clustering. Furthermore, such sparsity assumption is also relevant for the sampling task. Indeed, by making use of this property, it is possible under mild conditions to reconstruct the observations for nodes that have not been sampled [START_REF] Chen | Sampling theory for graph signals[END_REF]. These properties, all borrowed from the GSP eld, can be seen as constraints or regularizations for the graph learning task, and o er a new perspective on the topic.

Aim and main contributions. In the present chapter, we introduce an optimization problem to learn a graph from signals that are assumed to be smooth and admitting a sparse representation in the spectral domain of the graph. The main contributions can be summarized as follows:

• The graph learning task problem is cast as the optimization of a smooth nonconvex objective function over a nonconvex set (Section 2). This challenging problem is e ciently solved by introducing a framework that combines barrier methods, alternating minimization, and manifold optimization (Section 3). A relaxed algorithm is also proposed, which allows to scale in time with the graph dimensions (Section 4).

• A factor analysis model for smooth graph signals with sparse spectral representation is introduced (Section 5). This model provides a probabilistic interpretation of our optimization program and links its objective function to a maximum a posteriori estimation.

• The proposed algorithms are tested on several synthetic and real databases, and compared to state-of-the-art approaches (Section 7). Experimental results show that our approach allows to obtain similar or better performance than standard existing methods while signi cantly lowering the necessary computing resources.

Background and notations. Throughout all the chapter, we consider an undirected and weighted graph G with no self-loops. It is de ned as a pair G = (V, E) with vertices (or SIGNALS nodes) V = {1, . . . , N }, and set edges

E = {(i, j, w ij ), i, j ∈ V } with weights w ij ∈ R + arranged in a weight matrix W ∈ R N ×N + .
More particularly, we focus on its combinatorial graph Laplacian matrix which entirely describes it and is given by L = D -W , where D is the diagonal degree matrix and W the weight matrix. As G is undirected, L is a symmetric positive semi-de nite matrix. Its eigenvalue decomposition can be written as L = XΛX T , with Λ = diag(λ 1 , . . . , λ N ) a diagonal matrix containing the eigenvalues and X = (x 1 , . . . , x N ) a matrix with the eigenvectors as columns. We also consider graph signals (or graph functions) on this graph. A graph signal is de ned as a function y : V -→ R N that assigns a scalar value to each vertex. This function can be represented as a vector y ∈ R N , with y j the function value at the j-th vertex. Also, with 1 N we denote the constant unitary vector of size N , and with 0 N the vector containing only zeros. All remaining notations are given throughout the chapter.

Problem Statement

This section describes the graph learning problem for smooth and sparse graph signals.

Setup and working assumptions

The general task of graph learning aims at building a graph G that best explains the structure of n observed graph signals {y

(i) } n i=1 of size N , composing a matrix Y = [y (1) , • • • , y (n) ] ∈ R N ×n .
The proposed graph learning framework takes as input the matrix Y and outputs the Laplacian matrix L associated to G (note that both notions are equivalent). Our learning process is based on the following assumptions: On the smoothness assumption. According to the De nition 1.8 of chapter 1, low s values tend to favor smooth signals for which adjacent nodes carry similar signal values. This property has consequently been widely considered for the graph learning task [START_REF] Daitch | Fitting a graph to vector data[END_REF][START_REF] Dong | Learning laplacian matrix in smooth graph signal representations[END_REF].

On the spectral sparsity assumption. This property is known as bandlimitedness in the GSP eld. In general, it assumes that the null components of h are those associated to the largest eigenvalues (frequencies). Essentially, this additional hypothesis expresses a fundamental principle of signal processing which suggests ltering-out the high-frequency band of a signal, as it carries mainly noise and little or no information.

The bandlimitedness property is very common for graph signals, especially in GSP where it is the main hypothesis of several graph sampling methods [START_REF] Anis | Towards a sampling theorem for signals on arbitrary graphs[END_REF][START_REF] Chen | Sampling theory for graph signals[END_REF][START_REF] Marques | Sampling of graph signals with successive local aggregations[END_REF][START_REF] Narang | Signal processing techniques for interpolation in graph structured data[END_REF]. In a word, graph sampling refers to task of recovering a whole graph signal from a subset of nodes values. The main property behind the bandlimitedness assumption is that a k-sparse spectral representation implies that only k nodes values are necessary to recover the whole graph signal.

Furthermore, such property is known to carry cluster information on the graph signal: a k-sparse spectral representation implies k clusters. We can visualize this with the following trivial example. Let y = (1, 1, -1, -1) be a graph signal encoding labels of two clusters. If we take the graph with two connected components, the rst two nodes being connected together, so as the last two, then the graph exactly matches the cluster structure and the two columns of the spectral basis X associated to the smallest frequencies are

x 1 = 2 -1 2 (1, 1, 0, 0) and x 2 = 2 -1 2 (0, 0, 1, 1
). Thus, y can be written as a linear combination of x 1 and x 2 , making it orthogonal with the remaining columns of X and therefore 2-bandlimited..

Graph Learning for Smooth and Sparse Spectral Representation

A general graph learning scheme consists in learning the adjacency or the Laplacian matrix. However, since the constraint of Assumption 3.2 (sparsity of the graph signals over the eigen-basis of the Laplacian matrix) is easier to be expressed in the spectral domain, in this chapter we focus on learning the eigendecomposition of the Laplacian matrix L = XΛX T . The optimization problem incorporates a linear least square regression term depending of Y , X, and H, which controls the distance of the new representation XH to the observations Y . In addition, due to Assumption 3.2, we add two penalization terms: One to control the smoothness of the new representation, depending on Λ and H; the other one to control the sparsity on the spectral domain, which only depends on H. Finally, as we want to learn a Laplacian matrix satisfying Assumption 3.1, equality and inequality constraints relative to

CHAPTER 3. STRUCTURE INFERENCE FROM SMOOTH AND BANDLIMITED GRAPH SIGNALS

X and Λ are necessary. To that end, we introduce the following optimization problem:

min H,X,Λ Y -XH 2 F + α Λ 1/2 H 2 F + β H S , (3.1)
s.t.        X T X = I N , x 1 = 1 √ N 1 N , (a) (XΛX T ) k, ≤ 0 k = , (b) Λ = diag(0, λ 2 , . . . , λ N ) 0 , (c) tr(Λ) = N ∈ R + * , (d)
where I N is the identity matrix of size N , tr(•) denotes the trace, and Λ 0 indicates that the matrix is semi-de nite positive.

This problem aims at conjointly learning the Laplacian L (i.e. (X, Λ)) and a smooth bandlimited approximation XH of the observed signals Y . Here, H is the same size as Y and corresponds to the spectral representation of the graph signals through the GFT.

Interpretation of the terms. In the objective function (3.1), the rst term corresponds to the quadratic approximation error of Y by XH, where • F is the Frobenius norm. The second term is a smoothness regularization equally imposed to each column of XH. Indeed, from Equation (1.8), we have i y

(i)T Ly (i) = tr(Y T LY ) = L 1/2 Y 2
F . Rewriting this for the set of graph signals in XH, we obtain:

L 1/2 XH 2 F = XΛ 1/2 X T XH 2 F = Λ 1/2 H 2 F = N i=1 λ i H i,: 2 2 ,
where H i,: is the i-th row of the matrix H. This kind of regularization is very common in graph learning [START_REF] Chepuri | Learning sparse graphs under smoothness prior[END_REF][START_REF] Kalofolias | How to learn a graph from smooth signals[END_REF]. From its de nition, we can see that it tends to be low when high values of {λ i } N i=1 are associated to rows of H with low 2 -norm. This corroborates the idea that the {λ i } N i=1 can be interpreted as frequencies and the elements of H as Fourier coe cients.

The last term, β H S , is a sparsity regularization. In this work, we propose to either use the 2,1 (sum of the 2 -norm of each row of H) or 2,0 (number of rows with 2 -norm di erent than 0) that induces a row-sparse solution H.

Remark on the choice of • S -In the context of GSP, it is natural to assume that the graph signals are bandlimited at the same dimensions. This property is enforced by • S and has two main advantages: it is a key assumption for sampling over a graph and this particular structure is better for inferring graphs with clusters [START_REF] Sardellitti | Graph topology inference based on sparsifying transform learning[END_REF]. Therefore, in this chapter, the use of the classical 0 -norm and the 1 -norm have not been investigated since they would impose sparsity at every dimension of the matrix H 'independently', which would consequently break the bandlimitedness assumption.

The hyperparameters, α, β > 0 are controlling respectively the smoothness of the approximated signals and the sparsity of H. A discussion on the in uence of these hyperparameters and an e cient way to x them is provided in Section 7.3.1. Finally, the rst three constraints (3.1a), (3.1b), (3.1c) enforce XΛX T to be a Laplacian matrix of a graph with a single connected component (Assumption 3.1). More speci cally, by de nition,

L = D -W with W ∈ R N ×N + , thus we necessary have ∀k = , L k, = (XΛX T ) k, ≤ 0 (constraint (3.1b)).

PROBLEM STATEMENT 55

Furthermore, as XΛX T is the eigendecomposition of the Laplacian matrix of an undirected graph with a single connected component (Assumption 3.1), (3.1a) and (3.1c)). The last constraint (3.1d) was proposed in Dong et al. [START_REF] Dong | Learning laplacian matrix in smooth graph signal representations[END_REF] to impose structure in the learned graph so that the trivial solution Λ = 0 is avoided. A discussion about values other than N is made in Kalofolias [START_REF] Kalofolias | How to learn a graph from smooth signals[END_REF].

X T X = I N , x 1 = 1 √ N 1 N and λ 1 = 0 < λ 2 ≤ . . . ≤ λ N (constraints
The objective function (3.1) is not jointly convex but when • S is taken to be the 2,1 norm, it is convex with respect to each of the block-variables H, X, or Λ, taken independently. A natural approach to solve this problem is therefore to alternate between the three variables, minimizing over one while keeping the others xed. However, due to the equality constraint (3.1a) and inequalities (3.1b), the feasible set is not convex with respect to X. Hence, this approach raises several di culties that will be discussed and handled in the following section.

Reformulation of the problem

As stated in Section 2.2, problem (3.1) is not jointly convex and cannot be solved easily with constraints (3.1a) and (3.1b). In this section, we propose to rewrite constraints (3.1a) and (3.1b), in order to de ne a new equivalent optimization problem that can be solved with well-known techniques.

Reformulation of the constraint (3.1a)

In this section, we show that the constraints (3.1a) can be reformulated as a constraint over the space of orthogonal matrices in R (N -1)×(N -1) . Although such transformation does not change the convexity of the feasible set, we will see in Section 3.3 that there exist e cient algorithms that perform optimization over such manifold.

De nition 3.1. (Orthogonal group) -The space of orthogonal matrices in R N ×N , called orthogonal group, is the space:

Orth(N ) = {X ∈ R N ×N | X T X = I N } . Lemma 3.1. -Given X, X 0 ∈ R N ×N two orthogonal matrices, both having their rst column equal to 1 √ N 1 N (constraint (3.1a
)), we have the following equality

X = X 0 1 0 T N -1 0 N -1 [X T 0 X] 2:,2:
, with [X T 0 X] 2:,2: denoting the submatrix of X T 0 X containing everything but the rst row and column of itself. Furthermore, [X T 0 X] 2:,2: is in Orth(N -1). The above lemma allows us to build an equivalent formulation of Problem (3.1) given by the following proposition.

H,U,Λ Y -X 0 1 0 T N -1 0 N -1 U H 2 F + α Λ 1/2 H 2 F + β H S f (H, U, Λ) , (3.2)

CHAPTER 3. STRUCTURE INFERENCE FROM SMOOTH AND BANDLIMITED GRAPH SIGNALS

s.t.              U T U = I N -1 , (a') X 0 1 0 T N -1 0 N -1 U Λ 1 0 T N -1 0 N -1 U T X T 0 k, ≤ 0 k = , (b') Λ = diag(0, λ 2 , . . . , λ N ) 0 , (c) tr(Λ) = N ∈ R + * . (d)
The latter proposition says that since the rst column of X is xed and known, it is su cient to look for an optimal rotation of a valid matrix X 0 that preserves the rst column. Such a rotation matrix is given above and is parametrized by a U in Orth(N -1). Note that in practice, to nd a matrix X 0 satisfying (3.1a), we build the Laplacian of any graph with a single connected component and take its eigenvectors.

Log-barrier method for constraint (3.2b')

In order to deal with constraint (3.2b ), we propose to use a log-barrier method. This barrier function allows us to consider an approximation of problem (3.2) where the inequality constraint (3.2b') is made implicit in the objective function. Denoting by f (•) the objective function of (3.2), we want to solve min

H,U,Λ f (H, U, Λ) + 1 t φ(U, Λ) s.t. (3.2a'), (3.2c), (3.2d) , (3.3)
where t is a xed positive constant and φ(•) is the log-barrier function associated to the constraint (3.2b ).

De nition 3.2. (Log-barrier function) -Let the following matrix in R N ×N :

h(U, Λ) = X 0 1 0 T N -1 0 N -1 U Λ 1 0 T N -1 0 N -1 U T X T 0 ,
involved in the constraint (3.2b ). The associated log-barrier function φ : R (N -1)×(N -1) × R N ×N --→ R is de ned by:

φ(U, Λ) = - N -1 k=1 N >k log -h(U, Λ) k, , (3.4)
with

dom(φ) = (U, Λ) ∈ R (N -1)×(N -1) × R N ×N | ∀1 ≤ k < ≤ N, h(U, Λ) k, < 0 , i.
e. its domain is the set of points that strictly satisfy the inequality constraints (3.2b').

This barrier function allows us to perform block-coordinate descent on three easier to solve subproblems, as we discuss in the next section.

Resolution of the problem: IGL-3SR

In this section, we describe our method, the Iterative Graph Learning for Smooth and Sparse Spectral Representation (IGL-3SR), and its di erent steps to solve Problem (3.3). Given a xed t > 0, we propose to use a block-coordinate descent on H, U , and Λ, which permits to split the problem in three partial minimizations that we discuss in this section. One of 3. RESOLUTION OF THE PROBLEM: IGL 3SR 57 the main advantages of IGL-3SR is that each subproblem can be solved e ciently and as the objective function is lower-bounded by 0, this procedure ensures convergence. The summary of the method is presented in Algorithm 3.1.

Optimization with respect to H

For xed U and Λ, the minimization Problem (3.3) with respect to H is:

min H Y -XH 2 F + α Λ 1/2 H 2 F +β H S , where X = X 0 1 0 T N -1 0 N -1 U . (3.5) When • S is set to • 2,0 (resp. • 2,1
), this problem is a particular case of what is known as Sparsify Transform Learning [START_REF] Ravishankar | Learning sparsifying transforms[END_REF] (resp. is a particular case of the Group Lasso [START_REF] Yuan | Model selection and estimation in regression with grouped variables[END_REF] known as Multi-Task Feature Learning [START_REF] Argyriou | Multi-task feature learning[END_REF]). Moreover, as X is orthogonal, we are able to nd closed-form solutions (Proposition 3.2).

Proposition 3.2. (Closed-form solution for the 2,0 and 2,1 -norms) -The solutions of Problem (3.5) when • S is set to • 2,0 or • 2,1 , are given in the following.

• Using the 2,0 -norm, the optimal solution of (3.5) is given by the matrix H ∈ R N ×n where for

1 ≤ i ≤ N, H i,: = 0 if 1 1+αλ i (X T Y ) i,: 2 2 ≤ β , 1 (1+αλ i ) (X T Y ) i,: else . (3.6)
• Using the 2,1 -norm, the optimal solution of (3.5) is given by the matrix H ∈ R N ×n , where for

1 ≤ i ≤ N, H i,: = 1 1 + αλ i 1 - β 2 1 (X T Y ) i,: 2 + (X T Y ) i,: , (3.7)
where (t) + max{0, t} is the positive part function.

Optimization with respect to Λ

For xed H and U , the optimization Problem (3.3) with respect to Λ is:

min Λ α tr(HH T Λ) Λ 1/2 H 2 F + 1 t φ(U, Λ) s.t. Λ = diag(0, λ 2 , . . . , λ N ) 0 , (c) tr(Λ) = N ∈ R + * . (d) (3.8)
This objective function is di erentiable and convex with respect to Λ, and the constraints de ne a Simplex. Thus, several convex optimization solvers can be employed, such as those implemented in CVXPY [START_REF] Diamond | Cvxpy: A python-embedded modeling language for convex optimization[END_REF]. Popular algorithms are interior-point methods or projected gradient descent methods [START_REF] Maingé | Strong convergence of projected subgradient methods for nonsmooth and nonstrictly convex minimization[END_REF]. Using one algorithm of the latter type, we compute the gradient of 3.8 and project each iteration onto the Simplex [START_REF] Duchi | E cient projections onto the l 1-ball for learning in high dimensions[END_REF]. SIGNALS Figure 3.2: The principle of the manifold gradient descend given schematically. T X Orth(N ) is the tangent space of Orth(N ) at X. The red line corresponds to a curve in Orth(N ) passing through the point X in the direction of the arrow. At each iteration, considering that X is the point of the current solution, a search direction belonging to T X Orth(N ) is rst de ned, and then a descent along a curve of the manifold is performed (at the direction of the black arrow along the red line).

Optimization with respect to U

For xed H and Λ, the optimization Problem (3.3) with respect to U is:

min U Y -X 0 1 0 T N -1 0 N -1 U H 2 F + 1 t φ(U, Λ) s.t. U T U = I (N -1) . (a') (3.9)
The objective function is not convex but twice di erentiable and the constraint (a') involves the set of orthogonal matrices Orth(N -1) which is not convex. Orthogonality constraint is central to many machine learning optimization problems including Principal Component Analysis (PCA), Sparse PCA, and Independent Component Analysis (ICA) [START_REF] Hyvärinen | Independent component analysis: algorithms and applications[END_REF][START_REF] Shalit | Coordinate-descent for learning orthogonal matrices through givens rotations[END_REF][START_REF] Zou | Sparse principal component analysis[END_REF].

Unfortunately, optimizing over this constraint is a major challenge since simple updates such as matrix addition usually break orthonormality. One class of algorithms tackles this issue by taking into account that the orthogonal group Orth(N ) is a Riemannian submanifold embedded in R N ×N . In this chapter, we focus on manifold adaptation of descent algorithms to solve Problem (3.9). The generalization of gradient descent methods to a manifold consists in selecting, at each iteration, a search direction belonging to the tangent space of the manifold de ned at the current point X, and then performing a descent along a curve of the manifold. Figure 3.2 provides pictures this principle.

De nition 3.3. (Tangent space at a point of Orth(N )) -Let X ∈ Orth(N ). The tangent space of Orth(N ) at point X, denoted by T X Orth(N ) is a 1 2 N (N -1) dimensional vector space de ned by:

T X Orth(N ) = XΩ | Ω ∈ R N ×N is skew-symmetric .
When we endow each tangent space with the standard inner product, we are able to de ne a notion of Riemannian gradient that allows us to nd the best direction for the descent. For an objective function f : R N ×N → R, the Riemannian gradient de ned over Orth(N ) is given by: grad f (X) = P X (∇ X f (X)) , (3.10)

RESOLUTION OF THE PROBLEM: IGL 3SR 59

where P X is the projection onto the tangent space at X, which is equal to P X (ξ) = 1 2 X(X T ξξ T X), and ∇ X is the standard Euclidean gradient. At each iteration, the manifold gradient descent computes the Riemannian gradient (3.10) that gives a direction in the tangent space. Then the update is given by applying a retraction onto this direction, up to a step-size. A retraction consists in an update mapping from the tangent space to the manifold, and there are many possible ways to perform that [3,[START_REF] Arora | On learning rotations[END_REF][START_REF] Edelman | The geometry of algorithms with orthogonality constraints[END_REF][START_REF] Meyer | Geometric optimization algorithms for linear regression on xed-rank matrices[END_REF]. From the last equation, we see that in order to solve problem (3.9) with this method, we need the Euclidean gradient of the objective function, namely those of f (•) and φ(•). These are given in the following proposition.

Proposition 3.3. (Euclidean gradient with respect to U ) -The Euclidean gradient of f (•) and φ(•) with respect to U are:

∇ U f (H, U, Λ) = -2 (HY T X 0 ) 2:,2: T + 2U (HH T ) 2:,2: , ∇ U φ(U, Λ) = - N -1 k=1 N >k B k, + B T k, U Λ 2:,2: h(U, Λ) k, , with ∀1 ≤ k, ≤ N, B k, = X T 0 e k e T X 0 2:,2:
, and h(•) from De nition 3.2.

Log-barrier method and initialization

Choice of the t parameter.

The quality of the approximation of Problem (3.2) by Problem (3.3) improves as t > 0 grows. However, taking a too large t at the beginning may lead to numerical issues. As a solution, we use the path-following method, which computes the solution for a sequence of increasing values of t until the desired accuracy. This method requires an initial value for t, denoted t (0) , and a parameter µ such that t ( +1) = µt ( ) . For an in-depth discussion we refer to Boyd and Vandenberghe [START_REF] Boyd | Convex optimization[END_REF].

Initialization. At the beginning, our IGL-3SR method requires a feasible solution to initialize the algorithm. One possible choice is to take U as the identity matrix I N -1 and to replace (X 0 , Λ) by the eigenvalue decomposition of the complete graph with trace equals to N . Indeed, its eigenvalue decomposition will always satisfy the constraints and belong to the domain of the barrier function. The initialization of H is not needed as we start directly with the H-step.

IGL-3SR is summarized in Algorithm 3.1.

Computational complexity of IGL-3SR

Considering a graph with N nodes and n > N graph signals:

• H-step (non-iterative) -The closed-form solution requires to compute the matrix product X T Y , which is of complexity O(nN 2 ).

• Λ-step (iterative) -When using a projected gradient descent method, the complexity of each iteration is O(nN 2 ) to compute the gradient and O(N log(N )) for the projection [START_REF] Duchi | E cient projections onto the l 1-ball for learning in high dimensions[END_REF]. Hence, denoting by τ Λ the number of iterations in each Λ-step, the complexity is

O(τ Λ • nN 2 ) SIGNALS Algorithm 3.1 The IGL-3SR algorithm with 2,1 -norm Input: Y ∈ R N ×n , α, β
Input of the barrier method: t (0) , t max , µ -see Section 3.4 Output: H, X, Λ Initialization: L 0 (e.g. with a complete graph) -see Section 3.4

t ←-t (0) (X 0 , Λ) ←-SVD(L 0 ) U ←-I N -
for i = 1, . . . , N do H i,: ←- 1 1 + αλ i 1 - β 2 1 (X T Y ) i,: 2 + (X T Y ) i,:
end for

Λ-step: Solve Problem (3.8) Λ ←-arg min Λ α tr(HH T Λ) + 1 t φ(U, Λ) s.t. Λ = diag(0, λ 2 , . . . , λ N ) 0 , tr(Λ) = N ∈ R + * U -step: Solve Problem (3.9) while not convergence do U ←-retraction(U ( (HY T X 0 ) 2:,2: U -U T (HY T X 0 ) 2:,2: T ))
end while end while t ←µt end while

• X-step (iterative) -The complexity of each iteration is O(nN 2 ) to compute the Riemannian gradient and O(N 3 ) when we use the QR factorization as retraction [START_REF] Boyd | Introduction to applied linear algebra: vectors, matrices, and least squares[END_REF]. Hence, denoting by τ X the number of iterations in each X-step, the complexity is

O(τ X • nN 2 ).
Overall -The complexity to go through the big loop of IGL-3SR once (i.e. once through each of the H, Λ, and X steps) is of order O(max(τ Λ , τ X ) • nN 2 ). However, recall that τ Λ and τ X can be large in practice for reaching a good solution. In the following, we propose a relaxation for a faster resolution that relies on closed-form solutions.

A relaxation for a faster resolution: FGL-3SR

In this section, we propose another algorithm called Fast Graph Learning for Smooth and Sparse Spectral Representation (FGL-3SR) to approximately solve the initial Problem (3.1). FGL-3SR has a signi cantly reduced computational complexity due to a well-chosen relaxation. As in the previous section, we use a block-coordinate descent on H, X, and Λ, which permits to decompose the problem in three partial minimizations. FGL-3SR relies on a simpli cation of the minimization step in X by removing the constraint (3.1b). This simpli cation allows us to compute a closed-form on this step which greatly accelerates the minimization. However, the constraints (3.1a) and (3.1b) are equally important to obtain

A RELAXATION FOR A FASTER RESOLUTION: FGL 3SR

a valid Laplacian matrix at the end, and reducing the problem does not ensure that the constraint (3.1b) will be satis ed. The following proposition explains why we can get rid of constraint (3.1b) at the X-step, while still being able to ensure that the matrix will be a proper Laplacian at the end of the algorithm.

Proposition 3.4. (Feasible eigenvalues) -Given any X ∈ R N ×N being an orthogonal matrix with rst column being equal to 1

√ N 1 N (constraint (3.1a))
, there always exists a matrix Λ ∈ R N ×N such that the following constraints are satis ed:

   (XΛX T ) i,j ≤ 0 i = j , (3.1b) Λ = diag(0, λ 2 , . . . , λ N ) 0 , (3.1c) tr(Λ) = c ∈ R + * . (3.1d)
In Proposition 3.5 of the next section, we will see that, by ignoring constraint (3.1b) at the X-step, we can compute a closed-form solution to the optimization problem. For this reason, we propose to use the closed-form solution that we derive to learn X, and right after always optimize with respect to Λ. Hence, we are sure that we will obtain a proper Laplacian at the end of the process (Proposition 3.4). The initialization and the optimization with respect to H are not concerned by this relaxation and can therefore be performed as in IGL-3SR (see Sections 3.1 and 3.4).

Optimization with respect to X

As already explained, during the X-step, we solve the program

min X Y -XH 2 F s.t. X T X = I N , x 1 = 1 √ N 1 N , (3.1a) (3.11)
where the constraint (3.1b) is missing. The closed-form solution is given next.

Proposition 3.5. (Closed-form solution of Problem (3.11)) -Let X 0 be any matrix that belongs to the constraints set (3.1a), and M = (X T 0 Y H T ) 2:,2: the submatrix containing everything but the input's rst row and rst column. Finally, let P DQ T be the SVD of M . Then, the problem admits the following closed form solution:

X = X 0 1 0 T N -1 0 N -1 P Q T . (3.12)
In practice, X 0 can be xed to the current value of X.

Optimization with respect to Λ

With respect to Λ, the optimization Problem (3.1) becomes:

min Λ α tr(HH T Λ) Λ 1/2 H 2 F s.t.    (XΛX T ) i,j ≤ 0 i = j , (b) Λ = diag(0, λ 2 , . . . , λ N ) 0 , (c) tr(Λ) = N ∈ R + * , (d)
(3.13)
which is a linear program that can be solved e ciently using linear cone programs. Note that this will involve an optimization over N parameters with

for i = 1, . . . , N do H i,: ←- 1 1 + αλ i 1 - β 2 1 (X T Y ) i,: 2 + (X T Y ) i,:
end for

X-step: Compute the closed-form solution of Proposition (3.5) M ←-(X T Y H T ) 2:,2: (P, D, Q T ) ←-SVD(M ) X ←-X 1 0 T N -1 0 N -1 P Q T Λ-step: Solve the linear Program (3.13) Λ ←-arg min Λ α tr(HH T Λ) s.t.    (XΛX T ) i,j ≤ 0 i = j Λ = diag(0, λ 2 , . . . , λ N ) 0 tr(Λ) = N ∈ R + * end for

Computational complexity of FGL-3SR

Considering a graph with N nodes and n graph signals:

• H-step -The closed-form solution requires to compute the matrix product X T Y , which is of complexity O(nN 2 ).

• X-step -The closed-form solution requires to compute the SVD of (X T 0 Y H T ) 2:,2: ∈ R (N -1)×(N -1) , which is of complexity O(N 3 ) [START_REF] Cline | Computation of the singular value decomposition[END_REF].

• Λ-step -Solving the LP can be done with interior-point methods or with the ellipsoid method [START_REF] Vandenberghe | The CVXOPT linear and quadratic cone program solvers[END_REF]. For accuracy ε, the ellipsoid method yields a complexity of

O(max(m, N ) • N 3 log (1/ε)), where m = 1 2 N (N -1) + N + 1
is the number of constraints [START_REF] Bubeck | Convex optimization: Algorithms and complexity[END_REF].

Overall -As m > N , the complexity for FGL-3SR is of order O(N 5 ) when using the ellipsoid method. In contrast, the most competitive related algorithm of the literature (ESA-GL [START_REF] Sardellitti | Graph topology inference based on sparsifying transform learning[END_REF]) relies on a semi-de nite program and is of order at least O(N 8 ) (see Section 6). As will be clearly demonstrated in Section 7, in practice the empirical execution time of FGL-3SR is lower than IGL-3SR and ESA-GL.

Di erences between IGL-3SR and FGL-3SR

The two proposed algorithms are based on a modi cation of the initial optimization Problem (3.1). Indeed, both of them relax the constraint (3.1b), ∀k = , (XΛX T ) k, ≤ 0, but using two di erent approaches. IGL-3SR approximates the initial optimization problem through 5. A PROBABILISTIC INTERPRETATION 63 the use of a log-barrier function. The advantage of the barrier is twofold: rst, it allows to overcome the technical constraint (3.1b) and solve the program using a block-coordinate descent algorithm; second, the use of the barrier makes the block-variables separable over the constraint set, allowing the convergence of the objective function of IGL-3SR. In addition, IGL-3SR always keeps the set of variables in the initial set of constraints, essential for the matrix XΛX T to be a proper Laplacian.

On the other hand, FGL-3SR, instead of using a log-barrier function to relax the constraint (3.1b), it removes it from the X-step. Recall that we are perfectly able to do that as we know from Proposition 3.4 that for any X returned by the X-step, there exist a Λ making XΛX T a Laplacian. This relaxation speeds-up drastically the X-step while loosing the convergence property and the decreasing over the initial constraints set.

A probabilistic interpretation

In this section, we introduce a representation model adapted to smooth graph signals with sparse spectral representation. The goal of this model is to provide a probabilistic interpretation of Problem (3.1) and link its objective function to a maximum a posteriori estimation (Proposition 3.6).

Given a Laplacian matrix L = XΛX T , we propose the following Factor Analysis Framework to model a graph signal y:

y = Xh + m y + ε , (3.14)
where m y ∈ R N is the mean of the graph signal y and ε is a Gaussian noise with zero mean and covariance σ 2 I N . Here, the latent variable h = (h 1 , . . . , h N ) controls y through the eigenvector matrix X of L. The choice of the representation matrix X is particularly adapted since it re ects the topology of the graph and provides a spectral embedding of its vertices. Moreover, as seen in Section 2, X can be interpreted as a graph Fourier basis, which makes it an intuitive choice for the representation matrix. In a noiseless scenario with m y = 0, h actually corresponds to the GFT of y.

To comply with the spectral sparsity assumption (Assumption 3.2), we now propose a distribution that allows h to admit zero-valued components. To this end, we introduce independent latent Bernoulli variables γ i with success probability p i ∈ [0, 1]. Knowing γ 1 , . . . , γ N , the conditional distribution for h is:

h|γ ∼ N (0, Λ † ) , (3.15)
where Λ † is the Moore-Penrose pseudo-inverse of the diagonal matrix containing the values

{λ i 1{γ i = 1}} N
i=1 . In this model, γ i controls the sparsity of the i-th element of h. Indeed, if γ i = 0, then h i = 0 almost surely. In the other hand, if γ i = 1 then h i follows a Gaussian distribution with zero-mean and variance equal to 1/λ i . This is adapted to the smoothness hypothesis as for high value of λ i (high frequency), the distribution of h i concentrates more around 0, leading to small value of λ i h 2 i . The associated probability of success p i can be chosen a priori. One way to chose it is to take p i inversely proportional to λ i . Indeed, this would increase the probability to be sparse at dimensions where the associated eigenvalue is high. Note that, since λ 1 = 0, h 1 follows a centered degenerate Gaussian, i.e h 1 is equal to 0 almost surely. Furthermore, if p i = 1 for all i, our model reduces to the one proposed by Dong et al. [START_REF] Dong | Learning laplacian matrix in smooth graph signal representations[END_REF], which was only focused on the smoothness assumption. SIGNALS De nition 3.4. (Prior and conditional distributions) -The following equations summarize the prior and important conditional distributions of our model:

p(h i |γ i , λ i ) ∝ exp(-λ i h 2 i )1{γ i = 1} + 1{h i = 0, γ i = 0} , (3.16) p(y|h, X) ∝ exp(- 1 σ 2 y -Xh -m y 2
2 ) , (3.17)

p(γ i ) ∝ p γ i i (1 -p i ) 1-γ i . (3.18)
For simplicity, in the following we consider that m y = 0 and p 1 = 0.

Lemma 3.2. -Assume the proposed Model (3.14). If p 1 = 0 and p i ∈ (0, 1), ∀i ≥ 2, then:

-log(p(h|y, X, Λ)) ∝ 1 σ 2 y -Xh 2 2 + 1 2 h T Λh + N i=1 1{h i = 0} p i log( λ i √ 2π ) -log(p i ) -log( λ i √ 2π ) . De nition 3.5. (Lambert W-Function) -The Lambert W -Function, denoted by W (•), is the inverse function of f : W -→ W e W .
In particular, we consider W to be the principal branch of the Lambert function, de ned over [-1/e, ∞).

Proposition 3.6. (A posteriori distribution of h) -Let C > 0, and assume for all i ≥ 2 that

p i = e -C if λ i = √ 2π, whereas p i = -W -e -C log(λ i / √ 2π) λ i / √ 2π 1 log(λ i / √ 2π) otherwise.
Then, p i ∈ (0, 1) and there exist constants α, β > 0 such that:

-log(p(h|y, X, Λ)) ∝ y -Xh 2 2 + αh T Λh + β h 0 .
This proposition tells us that: for a given Laplacian matrix, the maximum a posteriori estimate of h would corresponds to the minimum of Problem (3.1).

Related work on GSP-based graph learning methods

Here we detail the two state-of-the-art methods for graph learning in the GSP context that are closer to our work and that will be used for our experimental comparison in Section 7. [START_REF] Dong | Learning laplacian matrix in smooth graph signal representations[END_REF]. This method supposes that the observed graph signals are smooth with respect to the underlying graph, but do not consider the spectral sparsity assumption. To learn the graph, they propose to solve the optimization problem:

GL-SigRep

min L, Ỹ Y -Ỹ 2 F + α L 1/2 Ỹ 2 F + β L 2 F s.t.    L k, = L ,k ≤ 0 k = , L1 = 0 , tr(L) = N ∈ R + * . ( 3
min H,X Y -XH 2 F s.t. X T X = I N , x 1 = 1 √ N 1 N , H 2,0 ≤ K ∈ N , (3.20)
which is solved using an alternating minimization. Once estimates for H and X have been computed, they solve a second optimization problem in order to learn the Laplacian L associated to the learned basis X. This is done by minimizing:

min L∈R N ×N , C K ∈R K×K tr( H T K C K H K ) + µ L 2 F s.t.        L k, = L ,k ≤ 0 k = , L1 N = 0 N , L X K = X K C K , C K 0 , tr(L) = N ∈ R + * , (3.21
) where C K ∈ R K×K and X K corresponds to the columns of X associated to the non-zero rows of H denoted H K . Thus, the second step aims at estimating a Laplacian that enforces the smoothness of the learned signal representation X H. This semi-de nite program requires the computation of over 12 N (N -1) + 1

CHAPTER 3. STRUCTURE INFERENCE FROM SMOOTH AND BANDLIMITED GRAPH SIGNALS

Evaluation metrics

We provide visual and quantitative comparisons of the learned Laplacian L and its weight matrix W using the performance measures: Recall, Precision, and F 1 -measure, which are standard for this type of evaluation [START_REF] Pasdeloup | Characterization and inference of graph di usion processes from observations of stationary signals[END_REF]. The F 1 -measure evaluates the quality of the estimated support -the non-zero entries -of the graph and is given by:

F 1 = 2 × precision × recall precision + recall .
As in [START_REF] Pasdeloup | Characterization and inference of graph di usion processes from observations of stationary signals[END_REF], the F 1 -measure is computed on a thresholded version of the estimated weight matrix W . This threshold is equal to the average value of the o -diagonal entries of W (same process as in [START_REF] Sardellitti | Graph topology inference based on sparsifying transform learning[END_REF]).

In addition, we compute the correlation coe cient ρ(L, L) between the true Laplacian entries L i,j and their estimates L i,j

ρ(L, L) = ij (L ij -L m )( L ij -L m ) ij (L ij -L m ) 2 ij ( L ij -L m ) 2 , ( 3.22)
where L m and L m are the average values of the entries of the true and estimated Laplacian matrices, respectively. This ρ coe cient evaluates the quality of the weights distribution over the edges.

Experiments on synthetic data

We now evaluate and compare all algorithms on several types of synthetic data. Details about graphs, associated graph signals, and evaluation protocol used for the experiments, are detailed in the sequel.

Graphs and signals. We carried out experiments on graphs with 20, 50, and 100 vertices, following: i) a Random Geometric (RG) graph model with a 2-D uniform distribution for the coordinates of the nodes and a truncated Gaussian kernel of width size 0.5 for the edges, where weights smaller than 0.75 were set to 0; ii) an Erdős-Rényi (ER) model with edge probability 0.2. Given a graph, the sampling process was made according to Model (3.16) that we presented in Section 5. The mean value of each signal was set to 0, the variance of the noise was set to 0.5, and the sparsity was chosen to obtain observations with k-sparse spectral representation, where k is equal to half the number of nodes (i.e 10, 20, 50).

For each type of graph, we ran 10 experiments with 1000 graph signals generated as explained above. For all the methods, the hyperparameters α and β are set by maximizing the F 1 -measure on the thresholded W , as explained in Section 7.1.

Choice of • S . In the following we make all experiments for IGL-3SR and FGL-3SR with the 2,1 -norm. This is motivated by an important fact brought by the closed-form solutions given in Proposition 3.2. Indeed, for 2,1 -norm, the sparsity of H is only controlled by β (Equation (3.7)). On the contrary, when using the 2,0 -norm, the value of α also in uences the sparsity (Equation (3.6)). This is an important behavior, as the tuning of β and α becomes independent -at least with respect to the H-step -and therefore, as we will see in Section 7. 3 Quantitative results. Average evaluation metrics and their standard deviation are collected in Table 3.1. The results show that the use of the sparsity constraint improves the quality of the learned graphs. Indeed, the two proposed methods IGL-3SR and FGL-3SR, as well as ESA-GL, have better overall performance in all the metrics than GL-SigRep that only considers the smoothness aspect. This had to be expected as our methods match perfectly to the sparse (bandlimited) condition.

Comparing the results across the di erent types of synthetic graphs, our methods are robust while being more e cient on RG graphs.

In general, IGL-3SR, and FGL-3SR present similar performance to ESA-GL. However IGL-3SR seems preferable in the case of RG graphs. For 100 nodes, the computational resources necessary for GL-SigRep was already too demanding, therefore only the results for the rest three methods are reported. We can see that, while IGL-3SR has better results than FGL-3SR, the time necessary to estimate the graph is much longer. In addition, examples of learned graphs are displayed in Figure 3.3 with the ground-truth on the left and the learned weighted adjacency matrices (after thresholding). The evolution of the F 1 -measure regarding the value of the threshold is also displayed and shows that a large range of threshold could have been used to obtain similar performance. All these results, combined with those of Table 3.1, indicate that in this sampling process the proposed FGL-3SR method managed to infer accurate graphs despite the relaxation. Speed performance. Figure 3.4 displays the evolution of the empirical computation time as the number of nodes increases. FGL-3SR appears to be much faster than the other methods. Furthermore, we observe that our methods are scalable over a wider range of graph sizes than the competitors. Indeed, even quite small graphs of 100 and 150 nodes, respectively, were already too 'large' for the two competitors to be able to produce results, and they even led to memory allocation errors.

In uence of the hyperparameters

We now study how hyperparameters of IGL-3SR and FGL-3SR in uence their overall performance, with respect to the F 1 -measure. This study is made on a RG graph with

In uence of α and β

We rst highlight the in uence of α and β on FGL-3SR. We run and collect the F 1 -measure for 20 values of α (resp. β) in [10 -5 , 100] (resp. [10 -5 , 60]). The resulting heatmaps are displayed in Figure 3.5. The most important observation is that the value of α does not seem to impact the quality of the resulted graphs. Indeed, for a xed value of β, the F 1 -measure is stable when α varies. However, it is interesting that the convergence curve of FGL-3SR (Figure 3.6) is directly impacted by α: large values for α tend to produce oscillations on the convergence curves. Thus, setting to a small value α > 0 is suggested. Contrary to α, tuning the parameter β is critical since high β values cause a drastic decrease in F 1 -measure. This sharp decrease appears when the chosen β imposes too much sparsity for the learned H. One may note that the best β corresponds to the value just before the sharp decrease, and this is the value that should be chosen. Although the previous analysis has been done on FGL-3SR, during our experimental studies, α and β in uenced the F 1 -measure similarly when using IGL-3SR.

In uence of t

We now highlight the in uence of t on IGL-3SR. Figure 3.7 shows the learned graphs for several values of t ∈ [10, 10 4 ]. This experiment brings two main messages: rst, when t is too low, the learned graph is very close to the complete graph, whereas when t increases the learned graph becomes more structured and tends to be sparse. This result was expected since a larger t brings the barrier closer to the true constraint, i.e. we allow elements of the resulting Laplacian matrix to be closer to 0. Second, it appears that α also in uences the nal results in a similar way to t. Again, this was expected as the minimization of the objective function during the Λ-step of Problem (3.3) is equivalent to the minimization of tr(HH T Λ) + 1 α t φ(U, Λ). For a discussion on the initial value of t, t (0) , and the step size µ such that t ( +1) = µt ( ) , both relative to the barrier method, we refer the reader to [START_REF] Boyd | Convex optimization[END_REF]. However, recall that t is not a hyperparameter to tune in practice, and should be taken as large as possible. The mere goal is to prevent numerical issues. Fortunately, a wide range of values for t (0) and µ achieves that goal [START_REF] Boyd | Convex optimization[END_REF]. Tuning the hyperparameters. The hyperparameter α does not seems to have a substantial impact on the F 1 -measure. However, a low value of it may be preferred in FGL-3SR for convergence purpose (Figure 3.6). The parameter t always needs to be maximal provided that it does not cause numerical issues. Classical heuristics and methods, like the one presented in Section 3.4, can be used to tune t [START_REF] Boyd | Convex optimization[END_REF]. Hence, according to our experiments, it remains only β as a critical hyperparameter to tune for both these methods. Based on Figure 3.5, one way to x it is to nd the largest β value that leads to satisfying results in terms of signal reconstruction. Alternatively, if we have an idea about the number of clusters k that resides on the graph, we could select a β value that produces a k-sparse spectral representation. Bearing in mind that other related works require the tuning of two hyperparameters, our approach turns out to be of higher value for practical application on real data where these parameters are unknown and must be tuned.

Temperature data

We used hourly temperature (C • ) measurements on 32 weather stations in Brittany, France, during a period of 31 days [START_REF] Chepuri | Learning sparse graphs under smoothness prior[END_REF]. The dataset contains 24 × 31 = 744 multivariate observations, i.e. Y ∈ R 32×744 , that are assumed to correspond to an unknown graph, which is our objective to infer. For our two algorithms, we set α = 10 -4 , and β is chosen so that we obtain a 2-sparse spectral representation, which this last assumes that there are two clusters of weather stations. The graphs obtained with each of the method are displayed in Figure 3.8 (a-b). They are in accordance with the one found in [START_REF] Chepuri | Learning sparse graphs under smoothness prior[END_REF] on the same dataset. Both the proposed methods provide similar results, which shows that the relaxation used in FGL-3SR has a moderate in uence in practice in this real-world problem. Although ground-truth is not available for this use-case, the quality of the learned graph can be assessed when using it as input in standard tasks such as graph clustering or sampling. For instance, when applying spectral clustering [START_REF] Ng | On spectral clustering: Analysis and an algorithm[END_REF] with two clusters on the resulting Laplacian matrices, it can be seen that both methods split the learned graph in two parts corresponding to the north and the south The learned graphs can be also employed in the graph sampling task. Indeed, due to the constraints used in the optimization problem, the graph signals are bandlimited with respect to learned graphs. For instance, in this example the graph signals are 2-bandlimited. This property means that it is possible to select only 2 nodes and to reconstruct the graph signal values of the 30 remaining nodes using linear interpolation. Figure 3.9 displays an example of such reconstruction: thanks to the learned graph structure, the use of only 2 nodes allows to reconstruct su ciently well the whole data matrix with a mean absolute error of 0.614. Again, this is a very interesting result that indirectly shows the quality of the learned graph.

Results on the ADHD dataset

In this third experiment, we consider the Attention De cit Hyperactivity Disorder (ADHD) dataset [START_REF] Bellec | The neuro bureau ADHD-200 preprocessed repository[END_REF] composed of functional Magnetic Resonance Imaging (fMRI) data. ADHD is a mental pathophysiology characterized by an excessive activity [START_REF] Boyle | Trends in the prevalence of developmental disabilities in US children, 1997-2008[END_REF]. We study the restingstate fMRI of 20 subjects with ADHD and 20 healthy subjects available from Nilearn [START_REF] Abraham | Machine learning for neuroimaging with scikitlearn[END_REF]. Each fMRI consists in a series of images measuring the brain activity. These images are processed as follows. First we split the brain into 39 Regions Of Interest (ROIs) with the Multi-Subject Dictionary Learning atlas [START_REF] Varoquaux | Multi-subject dictionary learning to segment an atlas of brain spontaneous activity[END_REF] (see Figure 3.10a). Each ROI de nes a node of our graph and the signal value at a certain node is an aggregation of the fMRI values over the associated ROI. For each subject, we therefore obtain a matrix in R n×39 , where n stands for the number of images in the fMRI for the subject.

We then estimate the graph of each subject using such preprocessed data. Examples of learned graphs with FGL-3SR for an ADHD subject and a healthy subject are displayed in Figure 3.10. Visually, they reveal strong symmetric links between the right and left hemisphere of the brain. This phenomenon is common in resting-state fMRI where one hemisphere tends to correlate highly with the homologous anatomical location in the opposite hemisphere [START_REF] Damoiseaux | Consistent resting-state networks across healthy subjects[END_REF][START_REF] Smith | Correspondence of the brain's functional architecture during activation and rest[END_REF]. Pointing out di erences, though, the graph from the ADHD subject seems less structured and contains several spurious links (diagonal and north-south connections).

Aiming to better highlight the potential value of quality learned graphs for such studies, we proceed and use the Laplacian matrices of the brain graphs to classify the subjects, as proposed in several resting-state fMRI studies [2,[START_REF] Dadi | Benchmarking functional connectome-based predictive models for restingstate fMRI[END_REF]. First, we subtract the average graph for all subjects, which in fact removes the symmetrical connections common to all subjects), and then we use a 3-Nearest Neighbors classi cation algorithm. We use the correlation coe cient of Equation (3.22) as distance metric between Laplacian matrices, and a leave-one-out cross-validation strategy. The classi cation accuracy of the described approach reaches 65%. This level shall be compared with the performance obtained using simple correlation graphs [2] that, on these 40 subjects, leads to an accuracy of 52.5%. It appears that in this context the use of a more sophisticated graph learning process allows a subject characterization that goes beyond considering basic statistical correlation e ects. Interestingly, this score is also comparable with state-of-the-art results reported in Sen et al. [START_REF] Sen | A general prediction model for the detection of ADHD and autism using structural and functional MRI[END_REF] for the same task, but on a larger database (67.3% of accuracy), using more sophisticated and specially-tailored processing steps, as well as carefully chosen classi ers.

Sigfox application

For the sake of completeness, we nally propose to apply our graph-learning method to the Sigfox training dataset used in Section 3.2 of the previous chapter. Recall that it consists in a set of approximately 35000 Sigfox messages recorded over one month at the level of 34 BSs and in this framework, one observation consists in a binary vector indicating which BS has received the message. To improve the speed of convergence, only 5000 messages are sampled randomly from this dataset. Before presenting the results, it should be taken into consideration that the proposed method has not been designed for this type of binary data. In particular, there is nothing here to suggest that smoothness can be characteristic of the underlying graph. In fact, many BS may have the same value 0 (did not receive the message), without being necessarily linked to each other. The learned graph is presented in gure 3.11. We can observe that the spatial structure is globally preserved by the inferred graph, a potential indicator of a well-learned graph. Only the group of BSs located in the lower right corner (about 10 BSs at the same position) do not seem to respect this structure by establishing connections with BSs far away from them. Although it is di cult to explain the reason of such an observation, it should be noted that this set of BSs is located at Sigfox headquarters, a place where a very large number of messages are sent, and therefore received, in order to perform tests. It is therefore conceivable that such bias could have led to the observed results. Nevertheless, it remains very likely that the method presented here is not adapted to this type of data, reason why the work presented in the following chapter has been investigated.

Conclusions

This chapter presented a data-driven graph learning approach by employing a combination of two assumptions that are well known in the Graph Signal Processing framework. The rst is the most standard in the literature of GSP and concerns the smoothness of the graph signals with respect to the underlying graph structure. The second is the spectral sparsity assumption, also referred as bandlimited property, which is notably known to carry the presence of clusters in real-world graphs. We proposed two algorithms to solve the corresponding optimization problem. The rst one, IGL-3SR, e ectively minimizes the objective function and has the advantage to decrease at each iteration. To address its low speed of convergence, we propose FGL-3SR that is a fast and scalable alternative. The ndings of our empirical evaluation on synthetic data showed that the proposed approaches are as good or better performing than the reference state-of-the-art algorithms in term of reconstructing the unknown underlying graph and of computational cost (running time). Experiments on real-world benchmark use-cases suggest that our algorithms learn graphs that are useful and promising for any graph-based machine learning methodology, such as graph clustering, subsampling, etc.

The objective that was set in the introduction regarding the structural inference from graph signals has thus been well addressed here. Nevertheless, several objectives remain. We have seen that the proposed method was not necessarily well adapted to Sigfox data and therefore learning a graph that is suited to binary data seems important. Moreover, the task of event detection has been omitted in this chapter. For these reasons, the next chapter will try to combine the two tasks by proposing a method to detect change-points Proof. Let consider X, X 0 ∈ R N ×N two orthogonal matrix with rst column equals to 1 √ N 1 N . We have the following equalities:

X 0 1 0 T N -1 0 N -1 [X T 0 X] 2:,2: =     . . . . . . X 0(:,1) X 0(:,2:) [X T 0 X] 2:,2: . . . . . .     =     . . . . . . 1 √ N 1 N X :,2: . . . . . .     = X .
Furthermore, thanks to the orthogonality of X and X 0 , we have

[X T 0 X] 2:,2: [X T 0 X] 2:,2:
T = X T 0,(2:,:) X :,2: X T 0,(2:,:) X :,2:

T = X T 0,(2:,:) X :,2: X T :,2: [X T 0,(2:,:) ] T = I N -1 .
By symmetry we conclude that [X T 0 X] 2:,2: ∈ Orth(N -1).

Proposition 3.1 -Given X 0 ∈ R N ×N an orthogonal matrix with rst column equals to 1 √ N 1 N , an equivalent formulation of optimization problem (3.1) is given by: min

H,U,Λ Y -X 0 1 0 T N -1 0 N -1 U H 2 F + α Λ 1/2 H 2 F + β H S f (H, U, Λ) , s.t.              U T U = I N -1 , (a') X 0 1 0 T N -1 0 N -1 U Λ 1 0 T N -1 0 N -1 U T X T 0 k, ≤ 0 k = , (b') Λ = diag(0, λ 2 , . . . , λ N ) 0 , (c) tr(Λ) = N ∈ R + * . (d)
Proof. From the previous lemma, we know that X can be decompose into two orthogonal matrices X 0 and U = [X T 0 X] 2:,2: . Hence, we can optimize with respect to U instead of X and the second part of the constraint (3.1a) is automatically satis ed. To make the equivalence, we just replace X from the main optimization problem to X 0 1 0 T N -1 0 N -1 U where U is now imposed to be orthogonal. • Using the 2,0 -norm, the optimal solution of (3.5) is given by the matrix H ∈ R N ×n where for

1 ≤ i ≤ N, H i,: = 0 if (X T Y ) i,: 2
2 /(1 + αλ i ) ≤ β , (X T Y ) i,: /(1 + αλ i ) else .
• Using the 2,1 -norm, the optimal solution of (3.5) is given by the matrix H ∈ R N ×n where for

1 ≤ i ≤ N, H i,: = 1 1 + αλ i 1 - β 2 1 (X T Y ) i,: 2 + (X T Y ) i,:
, where (t) + max{0, t} is the positive part function. SIGNALS Proof. In the following, we suppose that Y = 0 since in this trivial case, the solution is simply given by H = 0.

Closed-form solution for the 2,0 . Recall that H 2,0 = i=1 1 { H i,: 2 =0} , the objective function can be written as:

f (X, Λ, H) = X T Y -H 2 F + α Λ 1/2 H 2 F + β H 2,0 = Y 2 F + N i=1 n j=1 H 2 i,j -2(X T Y ) i,j H i,j + αλ i H 2 i,j + β1 { H i,: 2 =0} = Y 2 F + N i=1 H i,: 2
2 -2 (X T Y ) i,: , H i,: + αλ i H i,: 2
2 + β1 { H i,: 2 =0} = Y 2 F + N i=1 (1 + αλ i ) H i,: 2
2 -2 (X T Y ) i,: , H i,: + β1 { H i,: 2 =0} = Y 2 F + N i=1
fi (X, Λ, H i,: ) .

Our objective function is written as a sum of independent objective functions, each associated with a di erent H i,: . Hence, we can optimize the problem for each i. Our problem for a given i is:

min H i,: ∈R n (1 + αλ i ) H i,: 2 2 -2 (X T Y ) i,: , H i,: + β1 { H i,: 2 =0} .
When we restrict the minimization to H i,: 2 = 0, the unique solution is H i,: = 0 n and fi (X, Λ, H i,: ) = 0.

When H i,: 2 = 0, the objective function is convex and di erentiable, thus it su ce to take the following derivative equal to 0:

∂ ∂H i,:
fi (H i,:

) = 2(1 + αλ i )H i,: -2(X T Y ) i,: = 0 , H i,: = (X T Y ) i,: /(1 + αλ i ) .
With this solution, the objective function fi is equal to:

f (X, Λ, H i,: ) = (1 + αλ i ) (X T Y ) i,: /(1 + αλ i ) 2 2 -2 (X T Y ) i,: , (X T Y ) i,: /(1 + αλ i ) + β = 1 1 + αλ i (X T Y ) i,: 2
2 - 2 1 + αλ i (X T Y ) i,: 2
2 + β = β - 1 1 + αλ i (X T Y ) i,: 2 2 .
Hence, whenever

1 1 + αλ i (X T Y ) i,: 2
2 ≤ β, the objective function is positive, making H i,: = 0 a better choice for the minimization and conversely. In conclusion, for all 9. TECHNICAL PROOFS 79 1 ≤ i ≤ N, the solution is:

H i,: = 0 if (X T Y ) i,: 2
2 /(1 + αλ i ) ≤ β , (X T Y ) i,: /(1 + αλ i ) else .
Closed-form solution for the 2,1 . Similarly to the 2,0 case, the objective function can be decomposed by a sum of independent objectives functions.

f (X, Λ, H) = X T Y -H 2 F + α Λ 1/2 H 2 F + β H 2,1 = Y 2 F + N i=1 n j=1 H 2 i,j -2(X T Y ) i,j H i,j + αλ i H 2 i,j + β j=1 H 2 i,j = Y 2 F + N i=1 H i,: 2
2 -2 (X T Y ) i,: , H i,: + αλ i H i,: 2 2 + β H i,: 2 = Y 2 F + N i=1 (1 + αλ i ) H i,: 2
2 -2 (X T Y ) i,: , H i,: + β H i,: 2 = Y 2 F + N i=1 fi (X, Λ, H i,:
) .

Again, we can optimize the problem for each row i of H independently. Our problem for a given i is:

min H i,: ∈R n (1 + αλ i ) H i,: 2 2 -2 (X T Y ) i,: , H i,: + β H i,: 2 .
(3.23)

Although non-di erentiable at H i,: = 0 n , this function is convex and we need to nd H i,: such that the vector 0 n belongs to the subdi erential of fi denoted by ∂ fi (H i,: ) and is equal to:

∂ fi (H i,: ) =    B 2 -2(X T Y ) i,: , β if H i,: = 0 n , 2 1 + αλ i + β 2 1 H i,: 2 H i,: -2(X T Y ) i,: otherwise .
Where B 2 stand for the 2 -norm bowl.

Remark that when (X T Y ) i,:

2 ≤ β 2 , 0 n ∈ B 2 -2(X T Y ) i,:
, β and thus in this case H i,: = 0 n .

On the contrary, when (X T Y ) i,: 2 > β 2 , we must nd H i,: such that:

1 + αλ i + β 2 1 H i,: 2 H i,: = (X T Y ) i,:
. SIGNALS By tacking the norm of the previous equation, we obtain

1 + αλ i + β 2 1 H i,: 2 H i,: 2 = (X T Y ) i,: 2 ⇐⇒(1 + αλ i ) H i,: 2 + β 2 = (X T Y ) i,: 2 ⇐⇒ H i,: 2 = (X T Y ) i,: 2 - β 2 /(1 + αλ i ) > 0 .
We can now replace H i,: 2 in the initial equation and get H i,: .

1 + αλ i + β(1 + αλ i ) 2 (X T Y ) i,: 2 -β H i,: = (1 + αλ i ) (X T Y ) i,: 2 (X T Y ) i,: 2 -β/2 H i,: = (X T Y ) i,: ⇐⇒H i,: = (X T Y ) i,: 2 -β/2 (1 + αλ i ) (X T Y ) i,: 2 (X T Y ) i,: = 1 1 + αλ i 1 - β 2 1 (X T Y ) i,: 2 (X T Y ) i,:
, which concludes the proof.

Proposition 3.3 (Euclidean gradient with respect to U ) -The Euclidean gradient of f and φ with respect to U are

∇ U f (H, U, Λ) = -2 (HY T X 0 ) 2:,2: T + 2U (HH T ) 2:,2: , ∇ U φ(U, Λ) = - N -1 k=1 N >k B k, + B T k, U Λ 2:,2: h(U, Λ) k, .
with ∀1 ≤ k, ≤ N, B k, = X T 0 e k e T X 0 2:,2: , and h(•) from De nition 3.2.

Proof. We begin by computing the gradient of the main objective, with respect to U . Recall the objective function with respect to U :

f (H, U, Λ) = -2tr(Y T X 0 1 0 T N -1 0 N -1 U H) + tr(H T 1 0 T N -1 0 N -1 U T U H) .
The corresponding gradient is the following.

∇ U f (H, U, Λ) = -2∇ U tr(Y T X 0 1 0 T N -1 0 N -1 U H) + ∇ U tr(H T 1 0 T N -1 0 N -1 U T U H) = -2∇ U tr HY T X 0 1 0 T N -1 0 N -1 U + ∇ U tr HH T 1 0 T N -1 0 N -1 U T U = -2∇ U (HY T X 0 ) 1,1 • 1 + tr (HY T X 0 ) 2:,2: U + ∇ U (HH T ) 1,1 • 1 + tr (HH T ) 2:,2: U T U = -2 (HY T X 0 ) 2:,2:
T + 2U (HH T ) 2:,2: .

TECHNICAL PROOFS 81

We now derive the gradient of the barrier function φ(U, Λ) with respect to U :

∇ U φ(U, Λ) = - N -1 k=1 N >k ∇ U log -h(U, Λ) k, = - N -1 k=1 N >k 1 h(U, Λ) k, ∇ U h(U, Λ) k, .
We can write the h function as:

h(U, Λ) k, = e k e T , h(U, Λ) = X T 0 e k e T X 0 , 1 0 T N -1 0 N -1 U Λ 1 0 T N -1 0 N -1 U T = X T 0 e k e T X 0 , λ 1 0 T N -1 0 N -1 U Λ 2:,2: U T = tr X T 0 e e T k X 0 0 0 T N -1 0 N -1 U Λ 2:,2: U T = X T 0 e e T k X 0 1,1 • 0 + tr X T 0 e e T k X 0 2:,2: U Λ 2:,2: U T = tr B T k, U Λ 2:,2: U T .
In conclusion we have

∇ U h(U, Λ) k, = B k, +B T k, U Λ 2:,2:
, which nishes the proof.

Proposition 3.4 (Feasible eigenvalues) -Given any X ∈ R N ×N being an orthogonal matrix with rst column equals to 1/ √ N (constraint (3.1a)), there always exist a matrix Λ ∈ R N ×N such that the following constraints are satis ed:

   (XΛX T ) i,j ≤ 0 i = j , (3b) Λ = diag(0, λ 2 , . . . , λ N ) 0 , (3c) tr(Λ) = c ∈ R + * . (3d)
Proof. Let us consider a positive real value c > 0. Taking Λ = diag(0, c, . . . , c)/(N -1) leads to tr(Λ) = c and ∀i = j, (XΛX T ) i,j = -c/N < 0. However, this solution with constant eigenvalues actually corresponds to the complete graph. For our purpose, it is the worst case scenario as it contains no structural information between the nodes. . Let X 0 be any matrix that belongs to the constraints set (a), and M = (X T 0 Y H T ) 2:,2: the submatrix containing everything but the input's rst row and rst column. Finally, let P DQ T be the SVD of M . Then, the problem admits the following closed form solution

X = X 0 1 0 T N -1 0 N -1 P Q T .
Proof. One can observe that the relaxed optimization problem is equivalent to nding:

G = argmin G Y -X 0 1 0 T N -1 0 N -1 G G H 2 F , (3.24)

CHAPTER 3. STRUCTURE INFERENCE FROM SMOOTH AND BANDLIMITED GRAPH SIGNALS

s.t. G T G = I N -1 . This is obtained by replacing X with X 0 G.

Solving the above Equation (3.24) is equivalent to nding:

G = arg max G tr HY T X 0 G = arg max G tr M T G , s.t. G T G = I N -1 .
Then, as proved in [START_REF] Zou | Sparse principal component analysis[END_REF], we nally have G * = P Q T , which completes the proof.

Lemma 3.2 -Assume the proposed Model (3.14). If p 1 = 0 and p i ∈ (0, 1), ∀i ≥ 2, then,

-log(p(h|y, X, Λ)) ∝ 1 σ 2 y -Xh 2 2 + 1 2 h T Λh + N i=1 1 {h i =0} p i log( λ i √ 2π ) -log(p i ) -log( λ i √ 2π
) .

Proof. Based on the Factor Analysis model and the independence of h i 's, log(p(h|y, X, Λ)) ∝ log (p(y|h, X, Λ)) + log (p(h|X, Λ))

∝ - 1 2σ 2 y -Xh 2 2 + N i=1 log (p(h i |λ i )) . (3.25)
Let us now focus on log (p(h i |λ i )), for which we have:

log (p(h i |λ i )) = log   γ i ={0,1} p(h i , γ i |λ i )   = log   γ i ={0,1} p(h i , γ i |λ i ) p(γ i |h i , λ i ) p(γ i |h i , λ i )   ≥ (=) γ i ={0,1} p(γ i |h i , λ i ) log p(h i , γ i |λ i ) p(γ i |h i , λ i ) .
The last equality is obtain using the concavity of the logarithm and Jensen inequality. For this particular case, it correspond to an equality. Then we have:

log (p(h i |λ i )) = γ i ={0,1} p(γ i |h i , λ i ) log (p(h i , γ i |λ i )) ( ) - γ i ={0,1} p(γ i |h i , λ i ) log (p(γ i |h i , λ i )) . ( )
Before computing the previous two sums, we need to observe that:

p(γ i = 1|h i ) = 1 if h i = 0 , p i if h i = 0 .

TECHNICAL PROOFS 83

We can now compute ( ) and ( ) as follows:

( ) = γ i ={0,1} p(γ i |h i , λ i ) [log (p(h i |γ i , λ i )) + log (p(γ i |λ i ))] = 1 {h i =0} + p i 1 {h i =0} log λ i √ 2π - 1 2 λ i h 2 i + log (p i ) + (1 -p i )1 {h i =0} log 1 {h i =0} + log (1 -p i ) ( ) = [p i log(p i ) + (1 -p i ) log(1 -p i )] 1 {h i =0} .

Finally we can compute log (p(h

i |λ i )): log (p(h i |λ i )) = ( ) -( ) = 1 {h i =0} log λ i √ 2π - 1 2 λ i h 2 i + log (p i ) + p i log λ i √ 2π
1 {h i =0} = 1 {h i =0} log λ i √ 2π + log (p i ) -p i log λ i √ 2π + p i log λ i √ 2π + - 1 2 λ i h 2 i ∝ 1 {h i =0} log λ i √ 2π + log (p i ) -p i log λ i √ 2π - 1 2 λ i h 2 i .
Note that with our parametrization, the particular case i = 1 leads to log (p(h 1 |λ 1 )) = 0. Now plugging our result in equation (3.25) and multiplying on both side by -1, we get our nal result.

Proposition 3.6 (A posteriori distribution of h) -Let C > 0, and assume for all i ≥ 2 that

p i = e -C if λ i = √ 2π and p i = -W -e -C log(λ i / √ 2π) λ i / √ 2π
/ log(λ i / √ 2π) if not. Then, p i ∈ (0, 1) and there exist constants α, β > 0 such that:

-log(p(h|y, X, Λ)) ∝ y -Xh 2 2 + αh T Λh + β h 0 .
Proof. To show that the p i 's are well-de ned and belongs to (0, 1), it su ces to apply Lemma 3.3 with x = λ i / √ 2π.

We now proof the main result of the proposition. If λ i = √ 2π, then p i = e -C < 1 and

p i log( λ i √ 2π ) -log(p i ) -log( λ i √ 2π ) = -log(p i ) = C . 2π, then -p i log(λ i / √ 2π) = W -e -C log(λ i / √ 2π) λ i / √ 2π
. Since W corresponds to the inverse function of f (W ) = W e W , we have:

-p i log(λ i / √ 2π)e -p i log(λ i / √ 2π) = - e -C log(λ i / √ 2π) λ i / √ 2π ⇐⇒ -p i log(λ i / √ 2π)e -p i log(λ i / √ 2π) = - e -C log(λ i / √ 2π) λ i / √ 2π ⇐⇒ log p i log(λ i / √ 2π) e -p i log(λ i / √ 2π) = log e -C log(λ i / √ 2π) λ i / √ 2π ⇐⇒ log(p i ) + log log(λ i / √ 2π) -p i log(λ i / √ 2π) = -C + log log(λ i / √ 2π) -log(λ i / √ 2π) .
Same as the case where λ i = √ 2π, the nal equality gives us:

p i log( λ i √ 2π ) -log(p i ) -log( λ i √ 2π ) = C . (3.26)
Plugging the equation (3.26) into the nal result of proposition 1, we obtain:

-log(p(h|y, X, Λ)) ∝ 1 2σ 2 y -Xh 2 2 + 1 2 h T Λh + C h 0 ∝ y -Xh 2 2 + αh T Λh + β h 0 ,
taking α = σ 2 and β = 2Cσ 2 . This concludes the proof.

Lemma 3.3. Let C > 0. For any x > 0, 0 ≤ -W - e -C log(x) x / log(x) ≤ 1 . (3.27)
Proof. First, we show that this function is decreasing for x > 0. The derivative of the function is given by

∂ ∂x -W - e -C log(x) x / log(x) = W -e -C log(x) x W -e -C log(x) x + log(x)
x log 2 (x) W -e -C log(x)

x + 1

.

(3.28)

For x > 0 and C > 0,

-1/e < -e -(C+1) = min x>0 - e -C log(x) x ≤ - e -C log(x) x . (3.29)
As W (•) is strictly increasing for x > -1/e, we have W -e -C log(x)

x > W (-1/e) = -1. Hence, the bottom part of the previous equation is always positive.

TECHNICAL PROOFS 85

For 0

< x ≤ 1, W -e -C log(x) x is positive. Furthermore, -e -C log(x) x < - log(x) x ⇐⇒ W -e -C log(x) x < W -log(x) x = -log(x) (3.30) ⇐⇒ W -e -C log(x) x + log(x) < 0 . (3.31)
Hence, when 0 < x ≤ 1, the upper part of the previous equation is negative.

For 1 < x ≤ e, W -e -C log(x)

x is negative. Furthermore,

- 1 e ≤ - log(x) x < -e -C log(x) x ⇐⇒ W - log(x) x = -log(x) < W -e -C log(x) x (3.32) ⇐⇒ W -e -C log(x) x + log(x) > 0 . (3.33)
Hence, when 1 < x ≤ e, the upper part of the previous equation is negative again.

For x > e, W -e -C log(x)

x is negative. Furthermore, W -e -C log(x)

x > -1 and log(x) > 1. Hence, the addition is positive and the upper part of the previous equation is negative again.

We just have shown that the derivative is negative for x > 0. Hence, the initial function is decreasing on this interval. We now go back to the initial inequality (3.27). The left part of the inequality is straightforward as for x large enough, the function corresponds to the product of two positive functions. The function being decreasing, the lower bound follows. For the upper bound, let us remind that for y > e, we have the inequality W (y) < log(y) [START_REF] Hoorfar | Approximation of the lambert w function and hyperpower function[END_REF]. Let f (x) = -e -C log(x)

x , for x small enough we have:

W (f (x)) < log(f (x)) ⇔ -W (f (x)) > -log(f (x)) ⇔ -W (f (x)) / log(x) < -log(f (x))/ log(x) .
Tacking the limit when x -→ 0 + conclude the proof,

lim x→0 + -log(f (x))/ log(x) = lim x→0 + -log(- e -C log(x) x )/ log(x) = lim x→0 + -log(e -C ) + log(-log(x)) -log(x) / log(x) = lim x→0 + C log(x) + log(log(1/x)) log(1/x) + 1 = 1 .
Detecting changes in the graph structure of a varying Ising model

Abstract

This chapter is dedicated to the estimation of changes in the underlying structure of a series of binary graph signals, assumed to be drawn from the probability distribution known as Ising model. In particular, the present work focuses on the estimation of multiple change-points in a time-varying Ising model that evolves piece-wise constantly. The aim is to identify both the moments at which signi cant changes occur in the Ising model, as well as the underlying graph structures. For this purpose, we propose to estimate the neighborhood of each node by maximizing a penalized version of its conditional log-likelihood. The objective of the penalization is twofold: it imposes sparsity in the learned graphs and, thanks to a fused-type penalty, it also enforces them to evolve piece-wise constantly. Using few assumptions, we provide two changepoints consistency theorems. Those are the rst in the context of unknown number of change-points detection in time-varying Ising model. Finally, experimental results on several synthetic datasets and a real-world dataset demonstrate the performance of our method.

Associated publication:

Learning the piece-wise constant graph structure of a varying Ising model [START_REF] Bars | Learning the piece-wise constant graph structure of a varying ising model[END_REF], Le Bars, Batiste, Humbert Pierre, Kalogeratos, Argyris and Vayatis, Nicolas Accepted in Proceedings of the 37 th International Conference on Machine Learning (ICML).

Introduction

Graphs are fundamental tools to model and study static or varying relationships between variables of potentially high-dimensional vector data. They have many applications in physics, computer vision and statistics [START_REF] Choi | Exploiting hierarchical context on a large database of object categories[END_REF][START_REF] Marbach | Wisdom of crowds for robust gene network inference[END_REF]. In the static scenario, learning relationships between variables is referred to as graph inference and emerges in many elds such as in graph signal processing [START_REF] Dong | Learning laplacian matrix in smooth graph signal representations[END_REF][START_REF] Bars | Learning laplacian matrix from bandlimited graph signals[END_REF], in probabilistic modeling, or in physics and biology [START_REF] Du | Learning networks of heterogeneous in uence[END_REF][START_REF] Rodriguez | Uncovering the temporal dynamics of di usion networks[END_REF]. In this work, we consider a probabilistic framework where the observed data are drawn from an Ising model, a discrete Markov Random Field (MRF) with {-1, 1}-outputs. MRF are undirected probabilistic graphical models [START_REF] Koller | Probabilistic graphical models: principles and techniques[END_REF] where a set of random variables is represented as di erent nodes of a graph. An edge between two nodes in this graph indicates the conditional dependency between the two corresponding random variables, given the other variables.

Learning the structure of an MRF using a set of observations has been widely investigated [START_REF] Banerjee | Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data[END_REF][START_REF] Meinshausen | High-dimensional graphs and variable selection with the lasso[END_REF]. In particular for Gaussian graphical models [START_REF] Ren | Asymptotic normality and optimalities in estimation of large gaussian graphical models[END_REF][START_REF] Yuan | Model selection and estimation in the gaussian graphical model[END_REF] with the well-known graphical lasso [START_REF] Friedman | Sparse inverse covariance estimation with the graphical lasso[END_REF]. The Ising model inference task has also been addressed in the past [START_REF] Goel | Learning ising models with independent failures[END_REF][START_REF] Hö Ing | Estimation of sparse binary pairwise markov networks using pseudo-likelihoods[END_REF][START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF][START_REF] Vu Ray | Interaction screening: E cient and sample-optimal learning of ising models[END_REF][START_REF] Xue | Nonconcave penalized composite conditional likelihood estimation of sparse ising models[END_REF]. However, previous methods do not consider the case where the underlying structure is evolving through time.

Over the past years, there has been a burst of interest in learning the structure of timevarying MRF [START_REF] Hallac | Network inference via the time-varying graphical lasso[END_REF][START_REF] Yang | Estimating time-varying graphical models[END_REF]. This task combined with the change-point detection, which is the detection of the moments in time at which signi cant changes in the graph structure occur, is of particular interest. Those have been widely investigated for piece-wise constant Gaussian graphical models [START_REF] Gibberd | Regularized estimation of piecewise constant gaussian graphical models: The group-fused graphical lasso[END_REF][START_REF] Londschien | Change point detection for graphical models in presence of missing values[END_REF][START_REF] Wang | Fast and scalable learning of sparse changes in high-dimensional gaussian graphical model structure[END_REF], in all types of the change-point detection objectives: single change-point [START_REF] Bybee | Change-point computation for large graphical models: a scalable algorithm for gaussian graphical models with change-points[END_REF], multiple change-points [START_REF] Gibberd | Multiple changepoint estimation in high-dimensional gaussian graphical models[END_REF], o ine detection [START_REF] Gibberd | Regularized estimation of piecewise constant gaussian graphical models: The group-fused graphical lasso[END_REF][START_REF] Kolar | Estimating networks with jumps[END_REF], online detection [START_REF] Keshavarz | Sequential change-point detection in high-dimensional gaussian graphical models[END_REF], etc.

The advancements related to the time-varying Ising model are though limited. Especially, the combination of multiple change-points detection and structure inference has not been studied properly in the past. In [5,[START_REF] Kolar | Estimating time-varying networks[END_REF], the authors learn the parameters of a time-varying Ising model without looking for change-points since the network is allowed to change at each timestamp. In [59], the authors assume that the change-point location is known and only focus on the inference of the structural changes between Ising models. More recently, the problem of detecting a single change-point has been studied in [START_REF] Roy | Change point estimation in high dimensional markov random-eld models[END_REF].

Contribution. This work focuses on the estimation of multiple change-points in a timevarying Ising model that evolves piece-wise constantly. The aim is to identify both the moments at which signi cant changes occur in the Ising model, as well as the underlying graph structure of the model among consecutive change-points. Our work extends the

THE TIME VARYING ISING MODEL

work in [START_REF] Gibberd | Regularized estimation of piecewise constant gaussian graphical models: The group-fused graphical lasso[END_REF][START_REF] Kolar | Estimating networks with jumps[END_REF] on Gaussian graphical models, to the case of an Ising model. We also derive two change-points consistency theorems that, to our knowledge, we are the rst to demonstrate. More speci cally, our method follows a "node-wise regression" approach [START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF] and estimates the neighborhood of each node by maximizing a penalized version of its conditional log-likelihood. The penalization allows us to e ciently recover sparse graphs and, thanks to the use of a group-fused penalty [START_REF] Bleakley | The group fused lasso for multiple change-point detection[END_REF][START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF][START_REF] Kolar | Estimating networks with jumps[END_REF], as well to recover the change-points. The proposed method is referred as TVI-FL, which stands for Time-Varying Ising model identi ed with Fused and Lasso penalties.

Organization. The chapter is organized as follows. First, we brie y recall important properties of the static Ising model and describe its piece-wise constant version. Second, we present our methodology to infer the piece-wise constant graph structure over time and the moments in time at which signi cant changes occur. Next, we present our main theoretical results which consist in two change-point consistency theorems. Finally, we empirically demonstrate, on multiple synthetic datasets and a real-world problem, that our method is the best suited to recover both structure and change-points.

The time-varying Ising model

The static Ising model is a discrete MRF with {-1, 1}-outputs. This model is de ned by a graph G = (V, E) where an edge between two nodes indicates that the two corresponding random variables are dependent given the other ones. We associate this graph to a symmetric weight matrix Ω ∈ R p×p whose non-zero elements correspond to the set of edges E. Formally, we have ω ab = 0 i (a, b) ∈ E where ω ab stands for the (a, b)-th element of Ω. An Ising model is thus entirely described by its associated weight matrix Ω. Let X ∼ I(Ω) be a random vector following an Ising model with weight matrix Ω. Let x ∈ {-1, 1} p be a realization and x a , x b respectively its a-th and b-th coordinates. Then, its probability function is given by:

P Ω (X = x) = 1 Z(Ω) exp a<b x a x b ω ab , (4.1)
where Z(Ω) = x∈{-1,1} p exp a<b x a x b ω ab is the normalizing constant. For clarity in the following we denote P Ω (X = x) = P Ω (x).

A time-varying Ising model is de ned by a set of n graphs G (i) = (V, E (i) ), i ∈ {1, . . . , n} over a xed set of nodes V through a time-varying set of edges {E (i) } n i=1 . Similarly to the static case, each G (i) is associated to a symmetric weight matrix Ω (i) ∈ R p×p and a distribution P Ω (i) given by Eq. (4.1). A random variable associated to this model is a set of n independent random vectors X (i) ∼ I(Ω (i) ). A single realization is therefore a set of n vectors, each denoted by x (i) ∈ {-1, 1} p .

In the sequel, we assume in addition that the model is piece-wise constant, i.e. there exist a collection of D timestamps D {T 1 , . . . , T D } ⊂ {2, . . . , n}, sorted in ascending order, and a set of symmetric matrices {Θ (j) } D+1 j=1 such that ∀i ∈ {1, . . . , n}:

Ω (i) = D j=0 Θ (j+1) 1{T j ≤ i < T j+1 } , (4.2)

CHAPTER 4. DETECTING CHANGES IN THE GRAPH STRUCTURE OF A VARYING ISING MODEL

where T 0 = 1, T D+1 = n + 1. D thus corresponds to the set of change-points. According to Eq. (4.2), for a xed j ∈ {0, . . . , D}, the set x (i) : T j ≤ i < T j+1 contains i.i.d. vectors drown from P Θ (j+1) .

Learning Methodology

Assuming the observation of a single realization {x (i) } n i=1 of the described time-varying model at each timestamp, our objective is twofold. We want to recover the set of changepoints D, as well as the graph structure underlying the observed data vectors, i.e. which edges are activated at each timestamp. In practice, we may observe multiple data vectors at each timestamp. However, since this does not change our analysis, we leave the related discussion for the experimental section. Next, we now describe our methodology to perform the aforementioned tasks.

Neighborhood selection strategy. Due to the intractability of the normalizing constant Z(•), classical maximum likelihood approaches are di cult to apply in practice. Hence, an intuitive approach is to extend the neighborhood selection strategy introduced for the static setting in [START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF] to our time-varying setting. Instead of maximizing the global likelihood of Eq. (4.1), this approach maximizes, for each node a ∈ V , the conditional likelihood of the node knowing the other nodes in V a. The conditional probability of observing a node's value, knowing the others, when X ∼ I(Ω), is:

P ωa (x a |x a ) = exp 2x a b∈V a x b ω ab exp 2x a b∈V a x b ω ab + 1 , (4.3)
where ω a denotes the a-th column (or row) of Ω that is used to parametrize the probability function of Eq. (4.3). Here, x a denotes the vector x without the coordinate a.

For each node, we thus propose to maximize a penalized version of the conditional likelihood of Eq. 4.3. The detailed procedure is explained below.

Optimization program

The neighborhood selection strategy works as follows. For each node a = 1, . . . , p, we solve the regularized optimization program:

ω a = arg min ω∈R p-1×n L a (ω) + pen λ 1 ,λ 2 (ω). (4.4)
In this equation, L a (ω) stands for the node-wise negative conditional log-likelihood of node a, knowing x

(i) a : L a (ω) - n i=1 log P ω (i) (x (i) a |x (i) a ) (4.5) = n i=1 log exp ω (i) x (i) a + exp -ω (i) x (i) a - n i=1 x (i) a ω (i) x (i) a , (4.6)
where ω (i) is the i-th column of ω. The last line is obtained by plugging Eq. (4.3) in Eq. (4.5) with ω (i) instead of ω a . With such objective function, we learn at each timestamp i the neighborhood ω (i) of node a via a penalized logistic regression method. Penalty term. Provided two hyperparameters, λ 1 , λ 2 > 0, we propose the following penalty term for Eq. 4.4:

pen λ 1 ,λ 2 (ω) = λ 1 n i=2 ω (i) -ω (i-1) 2 + λ 2 n i=1 ω (i) 1 .
The overall two-term penalty is necessary for recovering e ciently the piece-wise constant graph structure. The second term is quite standard: it allows the estimated parameter vectors to be sparse and thus imposes structure in the learned graphs. On the other hand, without the rst term, we would t for each timestamp i ∈ {1, . . . , n} a parameter vector ω (i) that perfectly matches the observation x (i) (in terms of likelihood). In such a situation, we would obtain as many di erent parameter vectors ω as there is di erent samples, making the piece-wise constant assumption of Eq. 4.2 impossible to recover. This is why we propose a group-fused penalty, consisting in the 2 -norm of the di erence between two consecutive parameter vectors. The sum of the 2 -norms acts as a group-lasso penalty on temporal di erence between consecutive parameter vectors, which encourages the two vectors to be equal. This allows us to learn e ciently an evolving piece-wise constant structure and also to detect the change-points. In the past, the authors of [5] proposed to use a sum of 1 -norm in order to impose structure in the variation of the weight matrices. However, such penalization does not allow to recover a piece-wise constantly evolving graph structure. This problem is illustrated in gure 4.1: by penalizing each dimension independently, the 1 -norm easily leads to consecutive parameter vectors having di erent values, making the piece-wise constant assumption more di cult to recover.

In conclusion, the hyperparameter λ 1 controls the number of estimated change-points -the larger λ 1 is, the fewer the estimated number of change-points will be. Similarly, when λ 2 increases, the sparsity of each parameter vector increases as well. A priori, choosing the hyperparameters is not straightforward. However, since our objective function corresponds to a penalized logistic regression problem, we can use existing model selection criteria. We discuss further about this aspect in the experimental section.

CHAPTER 4. DETECTING CHANGES IN THE GRAPH STRUCTURE OF A VARYING ISING MODEL

Change-point detection and structure estimation

Assume that the optimization program (4.4) is solved. The set of estimated change-points D is:

D = T j ∈ {2, . . . , n} : ω ( T j ) a -ω ( T j -1) a 2 = 0 .
Namely, this corresponds to the set of timestamps at which the estimated parameter vectors have changed. For each submodel j = 1, . . . , | D| + 1, the a-th column of Θ (j) is estimated by θ

(j) a ω ( T j-1 ) a = . . . = ω ( T j -1) a
. The non-zero elements of θ

(j)
a indicate the neighborhood of a.

One should notice that this estimation leads to a non-symmetric weight matrix. To overcome this problem, is was proposed in [START_REF] Kolar | Estimating networks with jumps[END_REF][START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF] to either use the min or max operator. In the present work, to estimate the structure of the j-th graph, we take:

E (j) = {(a, b) : max(| θ (j) ab |, | θ (j) ba |) > 0}, where θ (j) ab is the b-th element of θ (j)
a , and conversely for θ (j) ba . In this case, there is an edge between two nodes if at least one of them contains the other node in its neighborhood.

Theoretical analysis

In this section, we present two change-point consistency theorems for TVI-FL. The theorems state that, as the number of samples n tends to in nity, the change-points are estimated more and more precisely. |T j -T j-1 |.

The second quantity is the minimal variation in the model parameters between two changepoints, which is given by:

ξ min min a∈V,j∈[D] θ (j+1) a -θ (j) a 2 .
We now introduce three standard assumptions on the Ising model inference and changepoints detection. They are assumed to be true for each node a ∈ V .

(A1) There exist two constants φ min > 0 and φ max < ∞ such that ∀j ∈

[D + 1], φ min ≤ Λ min E Θ (j) [X a X a ] and φ max ≥ Λ max E Θ (j) [X a X a ] .
Here Λ min (•) and Λ max (•) denote, respectively, the smallest and largest eigenvalues of the input matrix. (A1) is standard in such problems: it ensures that the covariates are not too dependent, making the model identi able [START_REF] Kolar | Estimating networks with jumps[END_REF][START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF]. In fact, this assumption is always veried if the support of the model is su ciently large. Indeed, if at least p linearly independent vectors have a non-zero probability to be observed, then the matrix E Θ (j) [X a X a ] will have full rank.

(A2) There exists a constant M such that max j∈[D+1] θ (j) a 2 ≤ M .

(A3) The sequence {T j } D j=1 satis es, for each j, T j = nτ j , where x is the largest integer smaller than or equal to x and {τ j } D j=1 is a xed, unknown sequence of the change-point fractions belonging to [0, 1].

This last assumption says that as n grows, the new observations are sampled uniformly across all the D + 1 sub-models.

Main results

Next, we present our theoretical results on change-point consistency. The proofs are made for one node, a, but generalize to all the other nodes. We rst provide the optimality conditions necessary to demonstrate the main results. Lemma 4.1. (Optimality Conditions) A matrix ω is optimal for our problem i there exists a collection of subgradient vectors {ẑ (i) } n i=2 and {ŷ

(i) } n i=1 , with ẑ(i) ∈ ∂ ω (i) -ω (i-1)
2 and ŷ(i) ∈ ∂ ω (i) 1 , such that ∀k = 1, . . . , n we have:

n i=k x (i) a tanh ω (i) x (i) a ) -tanh ω (i) a x (i) a ) - n i=k x (i) a x (i) a -E Ω (i) X a |X a = x (i) a + λ 1 ẑ(k) + λ 2 n i=k ŷ(i) = 0 p-1 , (4.7)
where tanh is the hyperbolic tangent function, 0 p-1 is the zero vector of size p-1, ẑ(1) = 0 p-1 , and

ẑ(i) = ω (i) -ω (i-1) ω (i) -ω (i-1) 2 if ω (i) -ω (i-1) = 0, ∈ B 2 (0, 1) otherwise, ŷ(i) = sign( ω (i) ) if x = 0, ∈ B 1 (0, 1)
otherwise.

Proof. The proof is given in the Appendix. It consists in writing the sub-di erential of the objective function and say, thanks to the convexity, that 0 belongs to it.

Theorem 4.1. (Change-point consistency) Let {x i } n i=1 be a sequence of observations drawn from the model presented in Sec. 2. Suppose (A1-A3) hold, and assume that λ 1 λ 2 = O( log(n)/n). Let {δ n } n≥1 be a non-increasing sequence that converges to 0, and such that ∀n > 0, ∆ min ≥ nδ n , with nδ n → +∞. Assume further that (i) λ 1 nδnξ min → 0, (ii)

P(d( D D) ≤ nδ n ) -→ n→∞ 1.
Proof. A detailed proof is provided in Appendix. The proof applies multiple times the di erent tricks used to prove Theorem 4.1 and the Lemmas also given in the Appendix.

Proposition 1 is of fundamental importance as it tells us that, even though the number of change-points has been overestimated, asymptotically, all the true change-points belong to the set of estimated change-points.

Experimental study

This section provides numerical arguments showing the empirical performance of TVI-FL. All the experiments were implemented using Python and conducted on a personal laptop. The code of TVI-FL is available online1 , so as a Jupyter Notebook reproducing results and gures of the real-world example.

Optimization procedure

Despite being non-di erentiable, the convexity of the objective function allows the use of existing convex optimization algorithms of the literature. In this work, we use the python package CVXPY [START_REF] Diamond | CVXPY: A Python-embedded modeling language for convex optimization[END_REF] that allows us to solve our problem e ciently. Note also that the optimization for each node is independent to the other nodes, and hence the approach allows e cient parallel implementations.

In the situation where more than one data vector is observed at each timestamp, one has simply to replace the node-wise negative log-likelihood in Eq. 4.5 with:

- n i=1 n (i) l=1 log P ω (i) (x (il) a |x (il) a ) , (4.10)
where n (i) stands for the number of data vectors observed at timestamp i, and x (il) for the l-th observed vector at time i.

Tuning the hyperparameters λ 1 and λ 2 . As stated in Sec. 3.1, it is possible to employ any model selection technique suited for logistic regression. In the experiments, we use and compare two techniques. The rst is the Akaike Information Criterion (AIC) that computes the average of the following quantity for all nodes:

AIC( ω a ) 2L a ( ω a ) + 2 Dim( ω a ) , (4.11)
where

Dim( ω a ) = n i=1 1{ ω (i) a = ω (i-1) a } b∈V a 1{sign( ω (i) ab ) = 0}

CHAPTER 4. DETECTING CHANGES IN THE GRAPH STRUCTURE OF A VARYING ISING MODEL

counts the number of parameters that are estimated. By convention, ω

(0) a = ω (1)
a . In this case, set of hyperparameters that minimize the AIC are nally selected.

The second technique, based on cross-validation (CV), assumes that more than one sample is observed at each moment in time i = 1, . . . , n. Thus, the time-series can be split in a part for the learning phase and another part for the testing phase. In our experiments, we selected the hyperparameters maximizing the AUC i.e. the area under the ROC-curve associated to the classi cation score (the probability to be equal to either 1 or -1).

For both model selection techniques, AIC and CV, the hyperparameters are found using either the standard random-search or grid-search strategies.

Experimental setup

Baseline method. As mentioned in Sec. 1, no existing work in the literature deals properly with the considered multiple change-point detection task. Several methods deal with varying Gaussian graphical models [START_REF] Kolar | Estimating networks with jumps[END_REF][START_REF] Yang | Estimating time-varying graphical models[END_REF], varying Ising models with smooth structural changes over time [START_REF] Kolar | Estimating time-varying networks[END_REF], or the detection of a single change-point in the varying Ising model [START_REF] Roy | Change point estimation in high dimensional markov random-eld models[END_REF]. The closest work we can compare with is the Tesla method [5,[START_REF] Kolar | Estimating time-varying networks[END_REF]. Its major di erence to our approach is the use of the 1 -norm instead of the 2 -norm as a fused-penalty. This di erence is very signi cant, theoretically and practically.

Indeed, using an 1 -norm fused-penalty does not encourage the recovery of a graphical model that evolves piece-wise constantly as a whole, which makes it less adaptable to recover change-points. More speci cally, such a term does not encourage two consecutive parameter vectors to be equal at every dimensions: the regularization only a ects each dimension independently. Thus, despite the edge weights may evolve independently in a piece-wise constant fashion, those changes occur at arbitrary timestamps and does not aggregate to a globally piece-wise constant behavior. An illustration of this phenomena and a comparison with the 2 -norm can be found in the Appendix. Nonetheless, the same way the standard linear regression can be used to recover sparse parameters, Tesla can still be used to recover change-points in practice. Hence, we choose this method as our baseline because, despite the lack of any theoretical guarantee, it can still be applicable, provided a su ciently large sample size and appropriately tuned regularization.

Performance metrics. We use two suitable metrics to evaluate the quality of TVI-FL on the learned graphs and change-points. The rst one, very standard in change-point detection tasks [START_REF] Truong | Selective review of o ine change point detection methods[END_REF] and known as the Hausdor metric, measures the longest temporal distance between a change-point in D and its prediction in D:

h(D, D) 1 n max max t∈D min t∈ D |t -t|, max t∈ D min t∈D |t -t| .
The lower this metric is, the better is the estimation. The second one, the F 1 -score, measures the goodness of the learned graphs structures (high value is better) by the quantity:

F 1 = 2 × precision × recall precision + recall ,
which combines the two following classic measures:

precision = 1 n n i=1 a<b 1{(a, b) ∈ E i ∧ (a, b) ∈ E i } 1{(a, b) ∈ E i } , recall = 1 n n i=1 a<b 1{(a, b) ∈ E i ∧ (a, b) ∈ E i } 1{(a, b) ∈ E i } .

Application to synthetic data

Simulation design. We compare the performance of our method TVI-FL against Tesla using several independent synthetic datasets. We rst x certain characteristics for all generated datasets: each of them has n = 100 timestamps, |D| = 2 change-points at the 51-st and 81-st timestamps, hence resulting in 3 submodels being valid respectively for 50, 30, and 20 timestamps. We consider the graph structure of each submodel to be an independent random d-regular graph of p = 20 nodes, where at each time the degree of the all nodes can be d ∈ {2, 3, 4}.

To generate a piece-wise constant Ising model: • We rst pick a degree value d ∈ {2, 3, 4} and draw independently 3 random d-regular graphs, one for each submodel. Same as in [5], the edge weights are drawn from a uniform distribution taken over [-1, -0.5] ∪ [0.5, 1]. • For each submodel, we draw observations using Gibbs sampling with a burn-in period of 1000 samples. Moreover, we collect one observation every 20 samples (lag) to avoid dependencies between them. In fact, instead of a single observation, for each timestamp i ∈ {1, . . . , n} we generate multiple observations n (i) in {4, 6, 8}, which requires to use the likelihood of Eq. (4.10). Besides, to be able to perform CV, we also sample 5 more observations per timestamp and use them only in the testing phase. With the above procedure we generate 10 di erent piece-wise constant Ising models for each degree d, which makes 30 models to learn in total. In addition, for each model, we generate 3 di erent sets of observations, one for each n (i) ∈ {4, 6, 8}, that constitute the individual learning problems of our evaluation. This results in 90 experiments in total.

For each experiment, we use a random-search strategy to nd the best pair of hyperparameters (λ 1 , λ 2 ) in [4,[START_REF] Backurs | Space and time e cient kernel density estimation in high dimensions[END_REF] × [START_REF] Chen | Sampling theory for graph signals[END_REF][START_REF] Daitch | Fitting a graph to vector data[END_REF]. This is done individually for the TVI-FL and Tesla methods. The selected hyperparameters are those minimizing the AIC or maximizing the AUC (see Sec. 5.1).

Results. The average value and standard deviation of the corresponding h-score and F 1 -score over each group of 10 experiments are reported in Tab. 4.1. The results clearly show that TVI-FL outperforms Tesla, regardless which model selection criterion we consult. This was expected as Telsa is not designed to recover Ising models that are evolving piecewise constantly (see Sec. 5.2). Furthermore, while in some cases Tesla nds a number of change-points closer to the true number, the associated h-scores are still higher than those of TVI-FL. Yet, Tesla is still not irrelevant to the task and in fact there are cases in which it reaches competitive performance scores to those of TVI-FL. Another nding is that AIC seems to favor a low number of estimated changes-points. It achieves better h-scores for this simulated process, while the AUC criterion seems to give priority to the recovery of the graph structure, illustrated by higher F 1 -scores. We show that empirically it is possible to obtain both low h-score and high F 1 -score via better hyperparameters tuning. Speci cally, for each experiment and for each degree d, we select the model with the highest F 1 -score when the associated h-score ≤ h min , with h min ∈ {0, .01, .02, .03}. This allows, respectively at most 0 to 3 timestamps of o set error between an estimated and a real change-point. In the results of Figure 4.2 we observe that even with very low h-score, high F 1 -score are reachable. Furthermore, the TVI-FL method always provides better F 1 -score than Tesla, con rming once again its superior performance.

Observations AIC AUC

Degree per timestamps Method

h-score ↓ F1-score ↑ D h-score ↓ F1-score ↑ D d = 2 n (i) =

Finding change-points in the real world: a voting dataset

Dataset and setup. In this section we evaluate the empirical performance of the TVI-FL method in a real-world use case. In particular, we analyze the di erent votes of the Illinois House of Representatives during the period of the 114-th and 115-th US Congresses (2015-2019), which are available at voteview.com [START_REF] Lewis | Voteview: Congressional roll-call votes database[END_REF]. The Illinois House of Representatives has 18 seats (one per district), each one corresponds to a US Representative belonging to the Democratic or the Republican parties. A Representative may or may not get reelected at the end of a Congress, which a ects if he/she will retain his/her seat in the new Congress. The speci c dataset we used contains 1264 votes, each of them represented by a vector of size p = 18, where a dimension is equal to 1 if the respective Representative of that seat has voted Yes, and -1 if it has voted No. When no information is provided about the vote of a seat (e.g. due to an absence), we impute the majority vote of its party.

It is always di cult to interpret a large number of change-points. For this reason, we choose to use the AIC criterion, which was found in Sec. 5 to favor smaller number of change-points. As for model tuning, we use a grid-search strategy to nd the best values for the hyperparameters. Results. Figure 4.3 (bottom) shows the cumulative function of the votes of each of the 18 seats, in temporal order, and the three change-points (dashed vertical line) detected by TVI-FL. The rst two change-points are di cult to interpret; it seems though that the second one corresponds to the pre-election period when a Congress comes to its end and votes get usually less polarized. Nevertheless, it must be noted that the structural changes of the rst two change-points are signi cantly lower compared to the third one. In fact, this last estimated change-point corresponds exactly to the time at which the Congress has changed. This signi cant change-point seems due to the non-reelection of some Representatives. More speci cally, the Representative of 10-th seat was the only one who was not reelected at the end of the 114-th Congress: the Republican Robert Dold, who was replaced by the Democrat Brad Schneider. This switch apparently lead to a signi cant variation in the structure of the underlying graph. Figure 4.3 (top) shows the graphs of positive weights, before and after this signi cant change-point. As expected, two clusters appear, one with the seats of Democrats and the other with those of the Republicans. Moreover, the 10-th seat becomes more connected with the cluster of Democrats after the time of change: the node loses 3 connections to the Republican cluster and gains 5 connections to Democrats and gets connected with all of them. More generally, all its weights with the Republican cluster decreases, contrarily to its weights with the Democratic cluster that do increase. This observation explains the origins of the structural change. Finally, it is interesting to observe that before and after the change-point, the 10-th seat is the only one wellconnected to both political groups. This makes us to conclude that this seat is represented by a super-collaborator, a role that some Representatives get by acting more independently and position themselves in the middle of the parties [START_REF] Andris | The rise of partisanship and super-cooperators in the US House of Representatives[END_REF]. Similarly, it is not surprising for Dan Lipinski, who had the 3-rd seat, to present in the learned graphs 2 connections with Republicans, as he is known to be a conservative Democrat.

Overall, this experiment shows that TVI-FL is suited to nd change-points in a realworld binary dataset, while also to recover the underlying evolving graph structure. This way, it increases the interpretability of the detected change-points. After applying the Telsa method on the same problem, we observed similar results and for this reason we omit them from the presentation.

Application to Sigfox dataset

We nally apply TVI-FL to the Sigfox dataset used in the previous chapters. Recall that in consists in a set of Sigfox messages recorded over a period of ve months at the level of 34 Base Stations. In this experiment, we propose to directly apply our algorithm to the 4 last months, referred as the testing dataset in the experiment of chapter 2. Over this dataset, the Sigfox messages are recorded daily, resulting in n = 120 days/timestamps over which we select randomly n i = 200 messages per day. Around the 30th day, one BS has been declared as working poorly. The goal of this experiment is therefore to see if we detect such timestamp as a change-point and if the learned graphs are well-estimated.

Here again, we selected the two hyperparameters using the AIC selection strategy. It resulted in the detection of 12 change-points for which most of them were di cult to interpret. However, only one change-point was detected in the neighborhood of the failing BS, actually detected at the 30th day, beginning time of failure. Moreover, the change in the neighborhood of this BS is characterized by a global decreasing in its associated weights. Such decreasing seems to be characteristic of an anomaly since it somehow means that the BS is sharing fewer messages with its neighbors than before. We illustrate such change in the network structure by representing a graph learned before and a graph learned after the 30th day ( gure 4.4).

In the gure, only the positive weights are represented and we clearly see a loss of edges in the neighborhood of the red node. Moreover, we also see that the learned graphs are in accordance with the spatial distribution of the di erent BSs, an indicator of the goodness of t of the estimated graphs.

Conclusions

The aim of this chapter was to answer the last two objectives raised in the introduction of this thesis. In other words, the detection of changes in the underlying structure of binary vectors. We proposed TVI-FL, an e cient way to learn a time-varying Ising model with piece-wise constantly evolving structure. Our method is able to both detect the changepoints at which the structure of the model changes and the structure themselves. Our work is the rst to provide change-point consistency theorems in this context. Those theoretical guarantees are reinforced by an empirical study. Using two di erent model selection criteria, ISING MODEL the proposed method is showed to outperform the closest baseline algorithm. Moreover, although not speci cally designed to work on Sigfox network data, we illustrated the applicability of TVI-FL to them in a promising last experiment.

Nevertheless, there are still few works to be addressed. These include for example the proof of consistent graph structure recovery (sparsistency) or the use of the recent Interaction Screening Objective [START_REF] Vu Ray | Interaction screening: E cient and sample-optimal learning of ising models[END_REF] in place of the standard conditional likelihood.

Technical proofs 7.1 Main results

In the following, we recall and prove the main results given in the chapter. The proofs uses in many situations the di erent lemmas given next. Lemma 4.1. (Optimality Conditions) A matrix ω is optimal for our problem i there exists a collection of subgradient vectors {ẑ (i) } n i=2 and {ŷ

(i) } n i=1 , with ẑ(i) ∈ ∂ ω (i) -ω (i-1) 2 and ŷ(i) ∈ ∂ ω (i)
1 , such that ∀k = 1, . . . , n we have:

n i=k x (i) a tanh ω (i) x (i) a ) -tanh ω (i) a x (i) a ) - n i=k x (i) a x (i) a -E Ω (i) X a |X a = x (i) a + λ 1 ẑ(k) + λ 2 n i=k ŷ(i) = 0 p-1 ,
where tanh is the hyperbolic tangent function, 0 p-1 is the zero vector of size p-1, ẑ(1) = 0 p-1 , and

ẑ(i) = ω (i) -ω (i-1) ω (i) -ω (i-1) 2 if ω (i) -ω (i-1) = 0, ∈ B 2 (0, 1) otherwise, ŷ(i) = sign( ω (i) ) if x = 0, ∈ B 1 (0, 1)
otherwise.

Proof. The proof follows those of [START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF][START_REF] Kolar | Estimating networks with jumps[END_REF] and [START_REF] Gibberd | Multiple changepoint estimation in high-dimensional gaussian graphical models[END_REF]. We rst introduce the following change of variables:

γ (i) = ω (i) if i = 1 ω (i) -ω (i-1) otherwise.
Thus ω (i) = i l=1 γ (l) , which leads to a change in the objective function 4.4 presented in Section 3.1.

{γ (i) } n i=1 = arg min γ∈R p-1×n n i=1 log exp i l=1 γ (l) x (i) a + exp - i l=1 γ (l) x (i) a - n i=1 x (i) a i l=1 γ (l) x (i) a + λ 1 n i=2 γ (i) 2 + λ 2 n i=1 i l=1 γ (l) 1 . (4.12)
This problem is convex, thus a necessary and su cient condition for {γ (i) } n i=1 to be a solution is that for all k = 1, . . . , n, the (p -1)-dimensional zero-vector 0, belongs to the 7. TECHNICAL PROOFS 103 subdi erential of (4.12), taken with respect to γ (k) :

0 ∈ n i=k x (i) a tanh i l=1 γ(l) x (i) a -x (i) a + λ 1 ∂ γ(k) 2 + λ 2 n i=k ∂ i l=1 γ(l) 1 .

Recall that

∂ x 2 = x x 2 if x = 0 B 2 (0, 1) otherwise ∂ x 1 = {sign(x)} if x = 0 B 1 (0, 1) otherwise .
Reapplying the change of variable, we obtain:

0 = n i=k x (i) a tanh ω(i) x (i) a -x (i) a + λ 1 ẑ(k) + λ 2 n i=k ŷ(i) Noting that E Ω (i) X a |X a = x (i) a = tanh ω (i) a x (i)
a ) , we obtain the nal result.

P( max j=1,...,D | Tj -T j | ≤ nδ n ) -→ n→∞ 1.
Proof. The proof follows the steps given in [START_REF] Gibberd | Multiple changepoint estimation in high-dimensional gaussian graphical models[END_REF][START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF][START_REF] Kolar | Estimating networks with jumps[END_REF]. First of all,Thanks to the union bound,

P( max j=1,...,D | Tj -T j | > nδ n ) ≤ D j=1 P(| Tj -T j | > nδ n ),
thus it su ces to show for each j = 1, . . . , D, that P(| Tj -

T j | > nδ n ) → 0. We denote by A n,j the event | Tj -T j | > nδ n .
Similarly to [START_REF] Kolar | Estimating networks with jumps[END_REF], we rst consider the good case where we assume that the event

C n = | Tj -T j | < ∆ min 2 occurs.

Bounding the good case

For each j = 1, . . . , D, we are going to show that P(A n,j ∩ C n ) -→ 0. In particular, we suppose that Tj ≤ T j as the proof for Tj ≥ T j will be the same by symmetry. ISING MODEL Applying Lemma 4.1 with k = Tj and k = T j , subtracting one with the other and applying the 2 -norm, we obtain

0 = T j -1 i= Tj x (i) a tanh ω (i) x (i) a ) -tanh ω (i) a x (i) a ) - T j -1 i= Tj x (i) a x (i) a -E Ω (i) X a |X a = x (i) a + λ 1 (ẑ ( T j ) -ẑ(T j ) ) + λ 2 T j -1 i= Tj ŷ(i) 2 ≥ T j -1 i= Tj x (i) a tanh ω (i) x (i) a ) -tanh ω (i) a x (i) a ) - T j -1 i= Tj x (i) a x (i) a -E Ω (i) X a |X a = x (i) a 2 -λ 2 T j -1 i= Tj ŷ(i) 2 -λ 1 (ẑ ( T j ) -ẑ(T j ) ) 2 We have λ 1 (ẑ ( T j ) -ẑ(T j ) ) 2 ≤ 2λ 1 and λ 2 T j -1 i= Tj ŷ(i) 2 ≤ (T j -Tj ) √ p -1λ 2 .
Furthermore, one may notice that for all i ∈ Tj , . . . , T j -1 , ω (i) = θj+1 a and ω

(i) a = θ j a .
Adding and subtracting tanh (θ j+1 a ) x (i) a ) , then applying again the triangle inequality leads to the following result:

2λ 1 + (T j -Tj ) p -1λ 2 ≥ R 1 2 -R 2 2 -R 3 2 (4.13)
with

R 1 = T j -1 i= T j x (i) a tanh (θ j a ) x (i) a ) -tanh (θ j+1 a ) x (i) a ) (4.14) R 2 = T j -1 i= T j x (i) a tanh ( θj+1 a ) x (i) a ) -tanh (θ j+1 a ) x (i) a ) (4.15) R 3 = T j -1 i= T j x (i) a x (i) a -E Θ (j) X a |X a = x (i) a (4.16)
The event (4.13) occurs with probability one and it can be showed that it is included in the event:

{2λ 1 + (T j -Tj ) p -1λ 2 ≥ 1 3 R 1 2 } ∪ { R 2 2 ≥ 1 3 R 1 2 } ∪ { R 3 2 ≥ 1 3 R 1 2 }
Thus, we have:

P(A n,j ∩ C n ) ≤P(A n,j ∩ C n ∩ {2λ 1 + (T j -Tj ) p -1λ 2 ≥ 1 3 R 1 2 }) + P(A n,j ∩ C n ∩ { R 2 2 ≥ 1 3 R 1 2 }) 7. TECHNICAL PROOFS 105 + P(A n,j ∩ C n ∩ { R 3 2 ≥ 1 3 R 1 2 })
P(A n,j,1 ) + P(A n,j,3 ) + P(A n,j,3 )

Now, We are going to show that each one of the three events has a probability that converges to 0 as n grows. Let's focus on A n,j,1 . Applying the mean-value theorem, we have for all i = T j , . . . , T j -1:

tanh (θ j a ) x (i) a ) -tanh (θ j+1 a ) x (i) a ) = (1 -tanh 2 ( θiT x (i) a ))x (i) a (θ j a -θ j+1 a ) (4.17)
with θi = α i θ j a +(1-α i )θ j+1 a , for a certain α i ∈ [0, 1]. Combining (4.17) with the de nition of R 1 , we obtain:

R 1 2 = T j -1 i= T j x (i) a tanh (θ j a ) x (i) a ) -tanh (θ j+1 a ) x (i) a ) 2 (4.18)
= (T j -T j ) 1

T j -T j T j -1 i= T j (1 -tanh 2 ( θiT x (i) a )) × x (i) a x (i) a (θ j a -θ j+1 a ) 2 (4.19)
≥ (T j -T j ) × Λ min   1 T j -T j T j -1 i= T j (1 -tanh 2 ( θiT x (i) a ))x (i) a x (i) a 
 × θ j a -θ j+1 a 2 (4.20)
Since, ∀j, θ j a 2

≤ M (A2), we have θi 2 ≤ M and | θiT x

(i) a | ≤ M • √ p -1. Thus, there exist a constant M > 0 such that 1 -tanh 2 ( θiT x (i)
a ) ≥ M . Combining this with the fact that each matrix x (i) a x (i) a are positive semide nite, we have:

R 1 2 ≥ (T j -T j ) M Λ min   1 T j -T j T j -1 i= T j x (i) a x (i) a   ξ min (4.21)
Thus, the event

{2λ 1 + (T j -Tj ) √ p -1λ 2 ≥ 1 3 R 1 2 } is included in the event 2λ 1 + (T j -Tj ) p -1λ 2 ≥ (T j -T j ) M Λ min   1 T j -T j T j -1 i= T j x (i) a x (i) a   ξ min (4.22)
Denoting by {4.22} the event of equation (4.22), we have:

P(A n,j,1 ) ≤ P(A n,j ∩ C n ∩ {4.22}) ≤ P A n,j ∩ C n ∩ {4.22} ∩ {Λ min   1 T j -T j T j -1 i= T j x (i) a x (i) a   > φ min 2 } + P A n,j ∩ C n ∩ {Λ min   1 T j -T j T j -1 i= T j x (i) a x (i) a   ≤ φ min 2 } ISING MODEL
Using Lemma 4.3 with v n = nδ n and = φ min 2 , we can bound the right-hand side of the upper equation. We also re-write the rst term so that we obtain:

P(A n,j,1 ) ≤ P(A n,j ∩ C n ∩ { 2λ 1 T j -T j + p -1λ 2 > M φ min 2 ξ min }) + c 1 exp - 2 nδ n 2 + 2 log(n) ≤ P(ξ -1 min 2λ 1 nδ n + ξ -1 min p -1λ 2 > M φ min 2 ) + c 1 exp - 2 nδ n 2 + 2 log(n) (4.23)
Thanks to (iii), we have nδ n that goes to in nity faster than log(n), thus the second term of the sum goes to 0 as n grows. Furthermore, using (i) and (ii) we have:

P(ξ -1 min 2λ 1 nδ n +ξ -1 min p -1λ 2 > M φ min 2 ) -→ n→0 P(0 + 0 > M φ min 2 ) = 0
Which concludes that P(A n,j,1 ) → 0.

We now focus on the event A n,j,2 . Let Tj 2 -1 (T j + T j+1 ) and remark that between T j and Tj , ω (i) = θ j+1 . Now, using Lemma 4.1 with k = Tj and k = T j and similar operation used to show equation (4.13), we have:

2λ 1 + ( Tj -T j ) p -1λ 2 ≥ Tj -1 i=T j x (i) a tanh ( θj+1 a ) x (i) a ) -tanh (θ j+1 a ) x (i) a ) 2 (4.24)
-

Tj -1 i=T j x (i) a x (i) a -E Θ (j+1) X a |X a = x (i) a 2 (4.25)
In the following we note

ε i j+1 x (i) a -E Θ (j+1) X a |X a = x (i)
a . Using the fact that θj+1 a 2 is necessarily bounded, Lemma 4.3 with = φ min /2 and similar arguments that we used for A n,j,1 , we can write that the rst term in the right-hand side of the previous equation is lower-bounded by:

(T j -Tj ) M φ min 2 θj+1 a -θ j+1 a 2
with probability tending to one. Here, M corresponds to a positive constant derived the same way as M in the previous part of the proof. In consequence, we can write

θj+1 a -θ j+1 a 2 ≤ 8λ 1 + 4( Tj -T j ) √ p -1λ 2 + 4 Tj -1 i=T j x (i) a ε i j+1 2 M φ min (T j+1 -T j ) (4.26)
7. TECHNICAL PROOFS 107 which holds with probability tending to one.

Furthermore, with probability also tending to one it can be shown using the same arguments used to prove equation (4.21

) that R 1 2 ≥ (T j -T j ) M φ min ξ min /2 and R 2 2 ≤ θj+1 a -θ j+1 a 2
φ max (T j -T j )/2. Combining that with equation (4.31), we can write:

P(A n,j,2 ) ≤ c 1 exp (-c 2 nδ n + 2 log(n)) + P A n,j ∩ C n ∩ 1 3 M M φ 2 min φ -1 max ξ min (T j+1 -T j ) ≤ 8λ 1 + 4( Tj -T j ) p -1λ 2 + 4 Tj -1 i=T j x (i)
a ε i j+1 2 ≤ P(c 3 φ 2 min φ -1 max ξ min ∆ min ≤ λ 1 ) + P(c 4 φ 2 min φ -1 max ξ min ≤ p -1λ 2 ) + P   c 5 φ 2 min φ -1 max ξ min ≤ ( Tj -T j ) -1 Tj -1 i=T j x (i)
a ε i j+1 2   + c 1 exp (-c 2 nδ n + 2 log(n))
With c 1 , . . . , c 5 positive constants.

The rst two terms tends to 0 as n goes to in nity thanks to the hypothesis (i) and (ii) of the theorem. Indeed, since ∆ min > nδ n and (nδ n ξ min ) -1 λ 1 → 0 (i), the rst term tends to P(c 3 φ 2 min φ -1 max ≤ 0) = 0 and the second term tends to 0 since ξ -1 min √ p -1λ 2 → 0 (ii). The fourth term directly tends to 0. Applying Lemma 4.4, we can upper bound the third term by:

P c 5 φ 2 min φ -1 max ξ min ≤ ( Tj -T j ) -1/2 2 p log(n) + c 6 exp(-2p log(n)) ≤ P c 5 φ 2 min φ -1 max ξ min ≤ (nδ n ) -1/2 2 p log(n) + c 6 exp(-2p log(n))
with c 6 an other positive constant.

Since (ξ min √ nδ n ) -1 p log(n) → 0 (iii), the previous equation tends to 0, which make P(A n,j,2 ) tends to 0 as well.

Finally, we upper bound the probability on the event A n,j,3 . As before, we know that R 1 2 ≥ (T j -T j ) M φ min ξ min /2 with probability at least 1c 1 exp(-c 2 nδ n + 2 log(n)), thus we have:

P(A n,j,3 ) ≤ P M φ min ξ min 6 ≤ R 3 2 T j -T j + c 1 exp(-c 2 nδ n + 2 log(n))
Using Lemma 4.5, we can upper bound the rst term by:

P M φ min ξ min 6 ≤ 2 p log(n) T j -T j + c 2 exp(-c 3 log(n)) ≤ P   M φ min ξ min 6 ≤ 2 p log(n) nδ n   + c 2 exp(-c 3 log(n))

CHAPTER 4. DETECTING CHANGES IN THE GRAPH STRUCTURE OF A VARYING ISING MODEL

which tends to 0 thanks to (iii). Since the symmetric case follows exactly the same arguments, we have shown that P(A n,j ∩ C n ) → 0. We now need to prove that P(A n,j ∩ C c n ) → 0.

Bounding the bad case

Again, the proof follows the one of [START_REF] Gibberd | Multiple changepoint estimation in high-dimensional gaussian graphical models[END_REF][START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF][START_REF] Kolar | Estimating networks with jumps[END_REF]. Let de ne the following complementary events:

D (l) n ∃j ∈ [D], T j ≤ T j-1 ∩ C c n (4.27) D (m) n ∀j ∈ [D], T j-1 < T j < T j+1 ∩ C c n (4.28) D (r) n ∃j ∈ [D], T j ≥ T j+1 ∩ C c n . (4.29)
We can write

P(A n,j ∩ C c n ) = P(A n,j ∩ D (l) n ) + P(A n,j ∩ D (m) n ) + P(A n,j ∩ D (r) n ).
Again, the goal is to prove that the three terms tends to 0. We will assume that T j ≤ T j as the other case can be done by symmetry. Let's rst focus on the middle term, it has been shown in [START_REF] Gibberd | Multiple changepoint estimation in high-dimensional gaussian graphical models[END_REF][START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF][START_REF] Kolar | Estimating networks with jumps[END_REF] that it can be upper bounded in the following way:

P(A n,j ∩ D (m) n ) ≤ P(A n,j ∩ {( T j+1 -T j ) ≥ ∆ min 2 } ∩ D (m) n ) + P({(T j+1 -T j+1 ) ≥ ∆ min 2 } ∩ D (m) n ) ≤ P(A n,j ∩ {( T j+1 -T j ) ≥ ∆ min 2 } ∩ D (m) n ) + D k=j+1 P({( T k+1 -T k ) ≥ ∆ min 2 } ∩ {(T k -T k ) ≥ ∆ min 2 } ∩ D (m) n ) (4.30)
Let's bound the rst term. Assuming the event

A n,j ∩ {( T j+1 -T j ) ≥ ∆ min 2 } ∩ D (m) n
and applying Lemma 4.1 with k = T j and k = T j , we can prove similarly as equation (4.31) that:

θj+1 a -θ j a 2 ≤ 4λ 1 + 2(T j -T j ) √ p -1λ 2 + 2 T j -1 i= T j x (i) a ε i j 2 M φ min (T j -T j ) ≤ c 1 φ -1 min (nδ n ) -1 λ 1 + c 2 φ -1 min p -1λ 2 + c 3 φ -1 min (T j -T j ) -1 T j -1 i= T j x (i) a ε i j 2
with probability tending to one. Using Lemma 4.5 we can bound the third term and obtain:

θj+1 a -θ j a 2 ≤ c 1 φ -1 min (nδ n ) -1 λ 1 + c 2 φ -1 min p -1λ 2 + c 3 φ -1 min ( nδ n ) -1 p log(n)
with probability tending to one. Similarly, applying the same lemmas with k = T j and either k = T j+1 , if T j+1 ≤ T j+1 or k = T j+1 otherwise, we have:

7. TECHNICAL PROOFS 109 θj+1 a -θ j+1 a 2 ≤ c 4 φ -1 min (nδ n ) -1 λ 1 + c 5 φ -1 min p -1λ 2 + c 6 φ -1 min ( nδ n ) -1 p log(n)
with probability tending to one.

Since

ξ min ≤ θ j a -θ j+1 a 2 ≤ θj+1 a -θ j a 2 + θj+1 a -θ j+1 a 2
, we nally upper bound the considered probability by:

P(A n,j ∩ {( T j+1 -T j ) ≥ ∆ min 2 } ∩ D (m) n ) ≤ P(ξ min ≤ c 7 φ -1 min (nδ n ) -1 λ 1 + c 8 φ -1 min p -1λ 2 + c 9 φ -1 min ( nδ n ) -1 p log(n))
Which tends to 0 thanks to the hypothesis (i), (ii) and (iii). The other probabilities in the upper bound on

P(A n,j ∩ D (m)
n ) also tends to 0. The proof follows exactly the previous one. We proved that P(A n,j ∩ D (m) n ) → 0, we will now show the same for

P(A n,j ∩ D (l) n ).
The proof exactly follows the one of [START_REF] Gibberd | Multiple changepoint estimation in high-dimensional gaussian graphical models[END_REF] where it has been showed that:

P(D (l) n ) ≤ D j=1 2 j-1 P(max{l ∈ [D] : T l ≤ T l-1 }) ≤ 2 D-1 D j=1 l>j P({T l -T l ≥ ∆ min 2 } ∩ { T l+1 -T l ≥ ∆ min 2 })
Now, as shown in [START_REF] Gibberd | Multiple changepoint estimation in high-dimensional gaussian graphical models[END_REF] and with the same arguments used to bound the elements of (4.30), we have P(D (l) n ) → 0. Similarly we can show P(D (r) n ) → 0 as n → 0. Finally we have P(A n,j ∩ C c n ) → 0, which concludes the proof.

Proposition 4.1. Let {x i } n i=1 be a sequence of observation drawn from the model presented in Sec. 2. Assume the condition of Theorem 1 are respected. Then, if for a x D max we have D ≤ D ≤ D max then:

P(d( D D) ≤ nδ n ) -→ n→∞ 1.
Proof. As stated upper, the proof, here again, follows the one of [START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF]. We are going to show that:

P({d( D D) ≥ nδ n } ∩ {D ≤ D ≤ D max }) ≤ Dmax K=D P({d( D D) ≥ nδ n } ∩ { D = K}) -→ n→∞ 0.
First, we note that for K = D, we have

P({d( D D) ≥ nδ n } ∩ { D = K}) -→
P(∀1 ≤ l ≤ K, | T l -T k | ≥ nδ n ) -→ n→∞ 0.
Like in [START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF], we rewrite the event {∀1 ≤ l ≤ K, | T l -T k | ≥ nδ n } as the disjoint union of the events:

E n,k,1 = {∀1 ≤ l ≤ K, | T l -T k | ≥ nδ n and T l < T k } E n,k,2 = {∀1 ≤ l ≤ K, | T l -T k | ≥ nδ n and T l > T k } E n,k,3 = {∃1 ≤ l ≤ K -1, | T l -T k | ≥ nδ n , | T l+1 -T k | ≥ nδ n and T l < T k < T l+1 }
and propose to show that the probability of each events tends to 0 as n grows. Let's begin with P(E n,k,1 ) and note that it is equal to:

P(E n,k,1 ∩ { T K > T k-1 }) + P(E n,k,1 ∩ { T K ≤ T k-1 })
First, we are going to upper bound the left-hand element of the previous equation. Applying Lemma 4.1 with t = T K and t = T k , we can prove similarly to the equation (4.13) in the good case scenario of the previous theorem that:

2λ 1 + (T k -T K ) p -1λ 2 ≥ R 1 2 -R 2 2 -R 3 2 with R 1 = T k -1 i= T K x (i) a tanh (θ k a ) T x (i) a ) -tanh (θ k+1 a ) T x (i) a ) R 2 = T k -1 i= T K x (i) a tanh ( θK+1 a ) T x (i) a ) -tanh (θ k+1 a ) T x (i) a ) R 3 = T k -1 i= T K x (i) a x (i) a -E Θ (k) X a |X a = x (i) a .
Like in the previous theorem, we can upperbound P(E n,k,1 ∩ { T k > T k-1 }) by:

P(E (1)
n,k,1 ) + P(E

n,k,1 ) + P(E

where

E (1) n,k,1 = {2λ 1 + (T k -T K ) p -1λ 2 ≥ 1 3 R 1 2 } 7. TECHNICAL PROOFS 111 E (2) n,k,1 = { R 2 2 ≥ 1 3 R 1 2 } E (3) n,k,1 = { R 3 2 ≥ 1 3 R 1 2 }
To show that P(E

n,k,1 ) tends to 0 it su ces to follow the proof used to show that P(A n,j,1 ) tends to 0 in the good scenario of the previous theorem.

Similarly, to show that P(E

n,k,1 ) tends to 0 it su ces to follow the proof used for P(A n,j,2 ). Applying lemma 4.1 with t = T k ans t = T k+1 we can show that with probability tending to one:

θ K+1 a -θ k+1 a 2 ≤ 4λ 1 + 2(T k+1 -T k ) √ p -1λ 2 + 2 T k+1 i=T k x (i)
a ε i j+1 2 M φ min (T k+1 -T k ) (4.31)
The rest follows exactly the arguments used to show the limit of P(A n,j,2 ).

Finally, P(E

n,k,1 ) tends to 0 the same way P(A n,j,3 ) was tending to 0 in the previous proof.

The proof to show that

P(E n,k,1 ∩ { T K ≤ T k-1 }
) tends to 0 is the same. It su ces to apply lemma 4.1 with t = T k-1 and t = T k to split the event in 3 sub-events and follow the proof. By symmetry, we also have P(E n,k,2 ) → 0.

Let's now focus on E n,k,3 . Like in [START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF], the event is split is four independent events:

E n,k,3 = E (1) n,k,3 ∪ E (2) n,k,3 ∪ E (3) n,k,3 ∪ E (4) n,k,3 with E (1)
n,k,3 = E n,k,3 ∩ {T k-1 < T l < T l+1 < T k+1 } E (2) n,k,3 = E n,k,3 ∩ {T k-1 < T l < T k+1 , T l+1 > T k+1 } E (3) n,k,3 = E n,k,3 ∩ { T l < T k-1 , T k-1 < T l+1 < T k+1 } E (4) n,k,3 = E n,k,3 ∩ { T l < T k-1 , T l+1 > T k+1 }
To prove that each one of the previous events have a probability that tends to 0 as n grows, we invite the reader to read the proof of [START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF]. It consist in multiple applications of the di erent Lemmas, the same way we used them in the previous part. Only the time at which lemma 4.1 is used changes and are given by [START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF]. This concludes the proof. ISING MODEL that l < r and rl > v n with v n a positive serie, we have ∀ > 0:

P Λ min 1 r -l + 1 r i=l x (i) a x (i) a ≤ φ min - ≤ 2(p -1) 2 exp - 2 v n 2 (4.32)
and

P Λ max 1 r -l + 1 r i=l x (i) a x (i) a ≥ φ max + ≤ 2(p -1) 2 exp - 2 v n 2 (4.33) Proof. Let Σ = 1 r-l+1 r i=l x (i) a x (i) a and Σ = E X a X a .
We rst proove the inequality (4.32). Recall that for a symmetric matrix M , we have Λ max (M ) ≤ M F , the Frobenius norm of M . We have

Λ min ( Σ) = min v 2 =1 v Σv (4.34) ≥ min v 2 =1 v Σv -max v 2 =1 v ( Σ -Σ)v (4.35) ≥ Λ min (Σ) -Λ max ( Σ -Σ) (4.36)
≥ φ min -Σ -Σ F (4.37) Let s (i)
mq be the (m, q)-th coordinate of x

mq the one of Σ-Σ. Note that E s

P Σ -Σ F > = P ( m,q s 2 mq ) 1/2 > (4.38) = P m,q s 2 mq > 2 (4.39) ≤ m,q P s 2 mq > 2 (4.40) ≤ m,q P (|s mq | > ) (4.41)
Thanks to Hoe ding's inequality, we have

P (|s mq | > ) ≤ 2 exp - 2 (r-l+1) 2
. Since rl > v n , we also have

P (|s mq | > ) ≤ 2 exp - 2 vn 2
. It follows from (4.41) that

P Σ -Σ F > ≤ 2(p -1) 2 exp - 2 vn 2
. We deduce that:

P Λ min ( Σ) ≥ φ min - ≥ 1 -2(p -1) 2 exp - 2 v n 2 , (4.42)
which concludes the proof for (4.32).

To prove (4.33) it su ces to note that Λ max ( Σ) ≤ φ max + Σ -Σ F and use the same arguments.

TECHNICAL PROOFS 113

Lemma 4.3. Let {x (i) } n i=1 be a set of i.i.d observation sampled from an Ising model with parameter Θ ∈ R p×p and assume that assumption (A1) is satis ed. Let R and L be two random variable such that R, L ∈ [n], L < R and R -L > v n almost surely, with v n a positive serie. For a xed node a and any > 0, there exist a constant c 1 > 0 such that:

P Λ min 1 R -L + 1 R i=L x (i) a x (i) a ≤ φ min - ≤ c 1 exp - 2 v n 2 + 2 log(n) (4.43)
and

P Λ max 1 R -L + 1 R i=L x (i) a x (i) a ≥ φ max + ≤ c 1 exp - 2 v n 2 + 2 log(n) (4.44) Proof. We note Σ(L, R) = 1 R-L+1 R i=L x (i) a x (i) a and I (l, r) ∈ [n] 2 : r -l > v n
We rst prove the inequality (4.43):

P Λ max Σ(L, R) ≥ φ max + = (l,r)∈I P Λ max Σ(L, R) , L = l, R = r (4.45) ≤ (l,r)∈I P Λ max Σ(L, R) L = l, R = r (4.46)
Using Lemma 4.2 we can bound (4.46):

(4.46) ≤ (l,r)∈I 2(p -1) 2 exp - 2 v n 2 (4.47) ≤ |I|c 1 exp - 2 v n 2 (4.48) ≤ n 2 c 1 exp - 2 v n 2 (4.49) ≤ c 1 exp - 2 v n 2 + 2 log(n) (4.50)
with c 1 = 2(p -1). This concludes the proof for (4.43). Same arguments are used to prove (4.44).

Lemma 4.4. Let {x (i) } n i=1 be a set of independent observation sampled from the time-varying Ising model (Section 2). Then, ∀j ∈ [D] and ∀r, l ∈ {T j , . . . , T j+1 -1} such that l < r, we have:

P 1 r -l + 1 R 3 (l, r) 2 ≤ 2 p log(n) r -l + 1 ≥ 1 -2(p -1) exp (-2p log(n)) (4.51) with R 3 (l, r) = r i=l x (i) a x (i) a -E Θ j X a |X a = x (i) a ISING MODEL
Proof. Let Z ij be the the j-th element of the vector

1 r-l+1 x (i) a x (i) a -E Θ X a |X a = x (i) a . Note that |Z ij | ≤ 2 r-l+1 and E [Z ij ] = 0.
Let > 0, we have:

P 1 r -l + 1 R 3 (l, r) 2 ≥ = P   j =a ( r i=l Z ij ) 2 ≥   = P   j =a ( r i=l Z ij ) 2 ≥ 2   ≤ j =a P | r i=l Z ij | ≥ ≤ 2(p -1) exp - 2 (r -l + 1) 2
Now, if we x = 2 p log(n) r-l+1 , we obtain:

P 1 r -l + 1 R 3 (l, r) 2 ≤ 2 p log(n) r -l + 1 ≥ 1 -2(p -1) exp (-2p log(n))
Lemma 4.5. Let {x (i) } n i=1 be a set of independent observation sampled from the time-varying Ising model (Section 2). We have:

P   j∈[D] l,r∈I j 1 r -l + 1 R j 3 (l, r) 2 ≤ 2 p log(n) r -l + 1   ≥ 1 -c 2 exp (-c 3 log(n)) (4.52) with R j 3 (l, r) = r i=l x (i) a x (i) a -E Θ j X a |X a = x (i) a
, c 2 , c 3 some positive constants and I j (l, r) ∈ {T j , . . . , T j+1 -1} 2 : r > l .

Proof. The proof is a simple application of Lemma 4.4:

P   j∈[D] l,r∈I j 1 r -l + 1 R j 3 (l, r) 2 ≥ 2 p log(n) r -l + 1   ≤ j∈[D] l,r∈I j P 1 r -l + 1 R j 3 (l, r) 2 ≥ 2 p log(n) r -l + 1 ≤ 2Dn 2 (p -1) exp (-2p log(n)) ≤ c 2 exp (-2p log(n) + 2 log(n)) ≤ c 2 exp (-c 3 log(n))
since p > 1. This concludes the proof.

Conclusion and perspectives

In this thesis, several contributions have been made in the context of vector data that are observed on network structures. Such data, known as graph vectors or graph signals, are recorded in many real-world scenarios and there is an increasing need to design learning algorithms adapted to them. In this work, two a priori distinct tasks has been considered. On one hand, the problem of event detection, which can be split in two di erent problems i.e. anomaly detection and change-point detection. On the other hand, the graph learning problem that is useful in the most common scenario where the underlying network structure over which the vectors are observed is unknown. Although a priori distinct, these two tasks were successfully linked in the nal chapter, where the graph was allowed to change in time. This thesis work also illustrated the idea that a concrete problem, in our case the detection of a Sigfox Base Station failure, can lead to more general and theoretical questions. Finally, the implementation of the di erent algorithms that has been proposed were made available online to allow and encourage the reproducibility of the results, which was a commitment from the beginning of the thesis project.

Regarding the various contributions presented in the manuscript and the research perspectives they suggest: In Chapter 2, we presented a simple novelty detection algorithm aiming to detect abnormal level of communication activity at the level of a node in a communication network. The algorithm relied on the intuition that the level of activity of a node can be predicted by looking at the level of activity recorded at its neighboring nodes. Thanks to an access to a normal data set and conventional supervised learning methods, the relationship between nodes activity has been learned. Afterwards, an anomaly was detected whenever the predicted level of activity was far from the real one. The method was shown to perform well on both synthetic and real-world data. In particular, it was shown to solve the problem raised by the industrial collaboration, i.e. the objective of detecting BSs failures in Sigfox network.

There are still some research perspectives to investigate. This includes direct use, when it is known, of the network structure on which the data are observed. For example at Sigfox, the knowledge of the positions of the BSs allows us to reduce the dimension of the learning problem by selecting only neighboring BSs. Can this knowledge have another use? Moreover, one can think about inferring the graph structure during the learning phase. This could be done with, for example, a logistic regression, in the manner of the Ising models of Chapter 4. Finally, a proper theoretical investigation on prediction-error based anomaly detection algorithms should be done in future works. In particular, this could be made in the framework of [START_REF] Clémençon | Scoring anomalies: a m-estimation formulation[END_REF] or [START_REF] Goix | On anomaly ranking and excess-mass curves[END_REF] which characterize the quality of a scoring function using notions related to the estimation of minimum volume sets.

In Chapter 3, we elaborated a new graph learning algorithm in the framework of GSP. The graph vectors were assumed to enjoy a sparse representation in the graph spectral domain, a feature which is known to carry information related to the cluster structure of a graph and which is also a key hypothesis of sampling algorithms. The signals were ISING MODEL as well assumed to behave smoothly with respect to the underlying graph structure. To tackle the problem, we proposed a new optimization program that learns the Laplacian of the graph and we provided two algorithms to solve it, called IGL-3SR and FGL-3SR. Based on a 3-steps alternating procedure, both algorithms relied on standard minimization methods such as manifold gradient descent or linear programming. While IGL-3SR ensures convergence, FGL-3SR acts as a relaxation and is signi cantly faster since its alternating process relies on multiple closed-form solutions.

Some further investigation remains to be done. In particular, one could think of studying the convergence properties of the FGL-3SR algorithm. Also, a statistical inference method could be considered to learn the graph. The factor analysis model presented in Section 5 of Chapter 3 seems to be a good basic model for a statistical estimation of the parameters. The latter could be learned using the EM algorithm or variational methods.

Finally, in Chapter 4, we combined the problems of event detection, in particular change-point detection, with the task of graph inference. This time, the problem was tackled by developing a probabilistic framework where the graph vectors were assumed to be drawn from Ising models. We assumed that the graph structures, i.e the parameter of the Ising models, were allowed to change over time, in a piece-wise constant fashion. Thus, the objective was to identify both the moments at which signi cant changes occurred in the Ising model, as well as the underlying graph structure governing the signal behavior segment-wise. For this purpose, we proposed to estimate the neighborhood of each node by maximizing a penalized version of its conditional log-likelihood. The objective of the penalization was twofold: it imposed sparsity in the learned graphs and, thanks to a fusedtype penalty, it also enforced them to evolve piece-wise constantly. In the end, we provided two change-points consistency theorems and demonstrated the performance of our method on several synthetic data sets and real-world examples.

Here again, some research perspectives remain. Among them, the investigation of the sparsistency which corresponds to the consistency of the estimated graph structures: when the number of sample grows in each segment, we must tend to recover the edges of the underlying graph. Another track for further study is the use of a di erent objective function, e.g. the Interaction Screening Objective [START_REF] Vu Ray | Interaction screening: E cient and sample-optimal learning of ising models[END_REF], which has been shown to be of good quality from both a computational and a statistical point of view in the static scenario.

Overall, the objectives set at the beginning of the manuscript have been met and, as stated above, there are still many directions to explore. In addition to these, a last track was investigated at the end of the thesis, the problem of robustness in machine learning and in particular for the task of non-parametric density estimation (see Chapter A in appendix). This recent interest suggests that studying the robustness of the methods proposed throughout the manuscript is also an interesting track of research that will be investigated. PRINCIPLE To be Submitted.

Authors with equal contribution to this work

Introduction

Over the past years, the task of learning in the presence of outliers has become an increasingly important objective in both statistics and machine learning. Indeed, in many situations, training data can be contaminated by undesired samples, which may badly a ect the resulting learning task, especially in adversarial settings. Building robust estimators and algorithms that are resilient to outliers is therefore becoming crucial in many learning procedures. In particular, the inference of a probability density function from a contaminated random sample is of major concerns. Density estimation methods are mostly divided into parametric and nonparametric techniques. Among the nonparametric family, the Kernel Density Estimator (KDE) is probably the most known and used for both univariate and multivariate densities [START_REF] Parzen | On estimation of a probability density function and mode[END_REF][START_REF] Scott | Multivariate density estimation: theory, practice, and visualization[END_REF][START_REF] Silverman | Density estimation for statistics and data analysis[END_REF], but it also known to be sensitive to dataset contaminated by outliers [START_REF] Kim | On the robustness of kernel density m-estimators[END_REF][START_REF] Kim | Robust kernel density estimation[END_REF][START_REF] Vandermeulen | Robust kernel density estimation by scaling and projection in hilbert space[END_REF]. The construction of robust KDE is therefore an important area of research, that can have useful applications such as anomaly detection and resilience to adversarial data corruption. Yet, only few works have proposed such robust estimators.

Kim and Scott [START_REF] Kim | Robust kernel density estimation[END_REF] proposed to combine KDE with ideas from M-estimation to construct the so-called Robust Kernel Density Estimator (RKDE). However, no consistency results were provided and robustness was rather shown experimentally. Later, RKDE was proven to converge to the true density, however at the condition that the dataset remains uncorrupted [START_REF] Vandermeulen | Consistency of robust kernel density estimators[END_REF]. More recently, Vandermeulen and Scott [START_REF] Vandermeulen | Robust kernel density estimation by scaling and projection in hilbert space[END_REF] proposed another robust estimator, called Scaled and Projected KDE (SPKDE). Authors proved the L 1 -consistency of SPKDE under a variant of the Huber's ε-contamination model where two strong assumptions are made [START_REF] Huber | Robust estimation of a location parameter[END_REF]. First, the contamination parameter ε is known, and second, the outliers are drawn from an uniform distribution when outside the support of the true density. Unfortunately, as they did not provided rates of convergence, it still remains unclear at which speed SPKDE converges to the true density. Finally, both RKDE and SPKDE require iterative algorithms to compute their estimators, increasing the overall complexity of their construction.

In statistical analysis, another idea to construct robust estimators is to use the Median-of-Means principle (MoM). Introduced by Nemirovsky and Yudin [START_REF] Nemirovsky | Problem complexity and method e ciency in optimization[END_REF], Jerrum et al. [START_REF] Jerrum | Random generation of combinatorial structures from a uniform distribution[END_REF], and Alon et al. [START_REF] Alon | The space complexity of approximating the frequency moments[END_REF], the MoM was rst designed to estimate the mean of a real random variable. It relies on the simple idea that rather than taking the average of all the observations, the sample is split in several non-overlapping blocks over which the mean is computed. The MoM estimator is then de ned as the median of these means. Easy to compute, the MoM properties have been studied by Minsker [START_REF] Minsker | Geometric median and robust estimation in banach spaces[END_REF] and Devroye et al. [START_REF] Devroye | Sub-gaussian mean estimators[END_REF] to estimate the means of heavy-tailed distributions. Furthermore, due to its robustness to outliers, MoM-based estimators have recently gained a renewed of interest in the machine learning community [START_REF] Lecué | Learning from mom's principles: Le cam's approach[END_REF][START_REF] Lecué | Robust classi cation via mom minimization[END_REF].

Contributions. In this work, we propose a new robust nonparametric density estimator based on the combination of the Kernel Density Estimation method and the Median-of-Means principle (MoM-KDE). We place ourselves in a more general framework than the classical Huber contamination model, called O ∪ I, which gets rid of any assumption on the outliers. We demonstrate the statistical performance of the estimator through nite-sample high-con dence error bounds in the L ∞ -norm and show that MoM-KDE's convergence rate is the same as KDE without outliers. Additionally, we prove the consistency in the L 1 -norm, which is known to re ect the global performance of the estimate. To the best of our knowledge, this is the rst work that presents such results in the context of robust kernel density estimation, especially under the O ∪ I framework. Finally, we demonstrate the empirical performance of MoM-KDE on both synthetic and real data and show the practical interest of such estimator as it has a lower complexity than the baseline RKDE and SPKDE.

Median-of-Means Kernel Density Estimation

We rst recall the classical kernel density estimator. Let X 1 , • • • , X n be independent and identically distributed (i.i.d.) random variables that have a probability density function (pdf) f (•) with respect to the Lebesgue measure on R d . The Kernel Density Estimate of f (KDE), also called the Parzen-Rosenblatt estimator, is a nonparametric estimator given by

fn (x) = 1 nh d n i=1 K X i -x h , (A.1)
where h > 0 and K : R d -→ R + is an integrable function satisfying K(u)du = 1 [START_REF] Tsybakov | Introduction to nonparametric estimation[END_REF].

Such a function K(•) is called a kernel and the parameter h is called the bandwidth of the estimator. The bandwidth is a smoothing parameter that controls the bias-variance tradeo of fn (•) with respect to the input data.

While this estimator is central in statistic, a major drawback is its weakness against outliers [START_REF] Kim | Robust kernel density estimation[END_REF][START_REF] Kim | On the robustness of kernel density m-estimators[END_REF][START_REF] Kim | Robust kernel density estimation[END_REF][START_REF] Vandermeulen | Robust kernel density estimation by scaling and projection in hilbert space[END_REF]. Indeed, as it assigns uniform weights 1/n to every points regardless of whether X i is an outlier or not, inliers and outliers contribute equally in the construction of the KDE, which results in undesired "bumps" over outlier locations in the nal estimated density (see Figure A.1). In the following, we propose a KDE-based density estimator robust to the presence of outliers in the sample set. These outliers are considered in a general framework described in the next section.

Outlier setup

Throughout the chapter, we consider the O ∪ I framework introduced by Lecué and Lerasle [START_REF] Lecué | Learning from mom's principles: Le cam's approach[END_REF]. This very general framework allows the presence of outliers in the dataset and relax the standard i.i.d. assumption on each observation. We therefore assume that the n random variables are partitioned into two (unknown) groups: a subset {X i | i ∈ I} made of inliers, and another subset {X i | i ∈ O} made of outliers such that O ∩ I = ∅ and O ∪ I = {1, . . . , n}. While we suppose the X i∈I are i.i.d. from a distribution that admits a density f with respect to the Lebesgue measure, no assumption is made on the outliers X i∈O . Hence, these outlying points can be dependent, adversarial, or not even drawn from a proper probability distribution.

KDE [125] - O(n) no RKDE [93] O(niter • n 2 ) O(n) yes SPKDE [163] O(niter • n 2 ) O(n) yes MoM-KDE - O(n) no
thus requiring an additional non-negligible computing capacity. Indeed, each one of them rely on an iterative method -the iteratively reweighted least squares algorithm and the projected gradient descent algorithm, that both have a complexity of O(n iter • n 2 ), where n iter is the number of needed iterations to reach a reasonable accuracy. MoM-KDE on the other hand does not require any learning procedure. Note that the evaluation step can be accelerated through several ways, hence potentially reducing computational time of all these competing methods [7,[START_REF] Backurs | Space and time e cient kernel density estimation in high dimensions[END_REF][START_REF] Gray | Nonparametric density estimation: Toward computational tractability[END_REF][START_REF] Wang | Nonparametric density estimation for high-dimensional data-algorithms and applications[END_REF]. Theoretical time complexities are gathered in Table A.1.

Theoretical analysis

In this section, we give a nite-sample high-probability error bound in the L ∞ -norm for MoM-KDE under the O ∪ I framework. To our knowledge, we are the rst to provide such error bounds in robust kernel density estimation under this framework. In particular, our objective is to prove that even with a contaminated dataset, MoM-KDE achieves a similar convergence rate than KDE without outliers [START_REF] Jiang | Uniform convergence rates for kernel density estimation[END_REF][START_REF] Sriperumbudur | Consistency and rates for clustering with dbscan[END_REF][START_REF] Wang | Dbscan: Optimal rates for density-based cluster estimation[END_REF]. In order to build this high-probability error bound, it is assumed, among other standard hypotheses, that the true density is Holder-continuous, a smoothness property usually considered in KDE analysis [START_REF] Jiang | Uniform convergence rates for kernel density estimation[END_REF][START_REF] Tsybakov | Introduction to nonparametric estimation[END_REF][START_REF] Wang | Dbscan: Optimal rates for density-based cluster estimation[END_REF]. In addition, we show the consistency in the L 1 -norm. In this last result, we will see that the aforementioned assumptions are not necessary to obtain the consistency. In the following, we give the necessary de nitions and assumptions to perform our non-asymptotic analysis.

Setup and assumptions

Let us rst list the usual assumptions, notably on the considered kernel function, that will allow us to derive our results. They are standard in KDE analysis, and are chosen for their simplicity of comprehension [START_REF] Jiang | Uniform convergence rates for kernel density estimation[END_REF][START_REF] Tsybakov | Introduction to nonparametric estimation[END_REF]. More general hypotheses could be made in order to obtain the same results, notably assuming kernel of order (see for example the works of Tsybakov [START_REF] Tsybakov | Introduction to nonparametric estimation[END_REF] and Wang et al. [START_REF] Wang | Dbscan: Optimal rates for density-based cluster estimation[END_REF]).

Assumption 1. (Bounded density) f ∞ < ∞.
We make the following assumptions on the kernel K. All the above assumptions are respected by most of the popular kernels, in particular the Gaussian, Exponential, Uniform, Triangular, Cosine kernel, etc. Furthermore, the last assumption implies that for any m > 0, we have u m K(u)du < ∞ ( nite norm moment) [START_REF] Jiang | Uniform convergence rates for kernel density estimation[END_REF]. Finally, when taken together, these assumptions imply that the kernel satis es the VC property [START_REF] Wang | Dbscan: Optimal rates for density-based cluster estimation[END_REF]. Theses are key properties to provide the bounds presented in the next section.

Before stating our main results, we recall the de nition of the Holder class of functions.

De nition A.2. (Holder class) Let T be an interval of R d , and 0 < α ≤ 1 and L > 0 be two constants. We say that a function f : T → R belongs to the Holder class Σ(L, α) if it satis es

∀x, x ∈ T, |f (x) -f (x )| ≤ L x -x α .
(A.3)

THEORETICAL ANALYSIS 123

This de nition implies a smoothness regularization on the function f , and is a convenient property to bound the bias of KDE-based estimators.

L ∞ and L 1 consistencies of MoM-KDE

This section states our central nding, a L ∞ nite-sample error bound for MoM-KDE that proves its consistency and yields the same convergence rate as KDE with uncontaminated data. The latter is given by the following Lemma partly proven by Sriperumbudur and Steinwart [START_REF] Sriperumbudur | Consistency and rates for clustering with dbscan[END_REF] and veri ed several times in the literature [START_REF] Giné | Rates of strong uniform consistency for multivariate kernel density estimators[END_REF][START_REF] Jiang | Uniform convergence rates for kernel density estimation[END_REF][START_REF] Wang | Dbscan: Optimal rates for density-based cluster estimation[END_REF].

Lemma A.1. (L ∞ error-bound of the KDE without anomalies) Suppose that f belongs to the class of densities P(α, L) de ned as

P(α, L) f | f ≥ 0, f (
fn -f ∞ ≤ C 1 S| log(h)| nh d + C 2 h α , (A.5)
where C 2 = L u α K(u)du < ∞ and C 1 is a constant that only depends on f ∞ , the dimension d, and the kernel properties.

This Lemma comes from the well-known bias-variance decomposition, where we separately bound the variance (see Theorem 3.1 of Sriperumbudur and Steinwart [START_REF] Sriperumbudur | Consistency and rates for clustering with dbscan[END_REF]) and the bias (see e.g. [START_REF] Tsybakov | Introduction to nonparametric estimation[END_REF] or [START_REF] Rigollet | Optimal rates for plug-in estimators of density level sets[END_REF]). It shows the consistency of KDE without anomalies, as soon as h → 0 and nh d → ∞.

We now present our main result. Its objective is to show that even under the O ∪ I framework, we do not need any additional hypothesis -besides the ones of the previous lemma -to show that MoM-KDE achieves the same convergence rate as KDE when used with uncontaminated data.

Proposition A.1. ( ∞ -error-bound of the MoM-KDE under the O ∪I) Suppose that f belongs to the class of densities P(α, L) and grant assumptions 1 to 4. Let S be the number of blocks, δ > 0 such that S > (2 + δ)|O|, and ∆ = (1/(2 + δ) -|O|/S). Then, for any h ∈ (0, 1), δ su ciently small, and n ≥ 1 such that nh d ≥ S log(2(2 + δ)/δ), and nh d ≥ S| log(h)|, we have with probability at least 1exp(-2∆ 2 S),

fMoM -f ∞ ≤ C 1 S log 2(2+δ) δ | log(h)| nh d + C 2 h α , (A.6)
where C 2 = L u α K(u)du < ∞ and C 1 is a constant that only depends on f ∞ , the dimension d, and the kernel properties. PRINCIPLE

The proof is given in the supplementary material. From equation (A.6), the optimal choice of the bandwidth is h S log(n)

n 1/(2α+d)
leading to the nal rate of

S log(n) n α/(2α+d)
. This convergence rate is the same (up to a constant) to the one of KDE without anomalies, with the same exponential control (Lemma A.1). Note that when there is no outlier, i.e. |O| = 0, the bound holds for S = 1, and we recover the classical KDE minimax optimal rate [START_REF] Wang | Dbscan: Optimal rates for density-based cluster estimation[END_REF]. In addition, the previous proposition states that the convergence of the MoM-KDE only depends on the number of outliers in the dataset, and not on their "type". This estimator is therefore robust in a wide range of scenarios, including the adversarial one.

We now give a 1 -consistency result under mild hypotheses, which is known to re ect the global performance of the estimate. Indeed, small 1 error leads to accurate probability estimation [START_REF] Devroye | Nonparametric Density Estimation: The L1 View[END_REF].

Proposition A.2. ( 1 -consistency in probability) If n/S → ∞, h → 0, nh d → ∞, and S > 2|O|, then fMoM -f 1 P -→ n→∞ 0 . (A.7)
This result is obtained by bounding the left-hand part by the errors in the healthy blocks only, i.e. those without anomalies. Under the hypothesis of the proposition, these errors are known to converge to 0 in probability [START_REF] Wang | Dbscan: Optimal rates for density-based cluster estimation[END_REF]. The complete proof is given in supplementary material. Contrary to SPKDE [START_REF] Vandermeulen | Robust kernel density estimation by scaling and projection in hilbert space[END_REF], no assumption on the outliers generation process is necessary to obtain this consistency result. Moreover, while we need to assume that the proportion of outliers is perfectly known to prove the convergence of SPKDE, the MoM-KDE converges whenever the number of outliers is overestimated.

In uence function in the O ∪ I framework

As a measure of robustness, we now introduce an In uence Function (IF) adapted to the O ∪ I framework. It is inspired from the classical IF, rst proposed by [START_REF] Frank | The in uence curve and its role in robust estimation[END_REF], which measures how an estimator changes when the initial distribution is modi ed by adding a small amount of contamination at a point x . Therefore, it provides a notion of stability in the Huber model framework [START_REF] Donald | Stability comparison of estimators[END_REF][START_REF] Michiel Debruyne | Robustness and stability of reweighted kernel based regression[END_REF]. We now de ne a similar concept under the O ∪I.

De nition A.3. (IF O∪I ) Let T n (x 0 ; I n ) be a density estimator evaluated at x 0 and learned with an healthy data set

I n = {X i } n i=1 . Let m ∈ N and x ∈ R d .
The IF O∪I is de ned as:

IF O∪I (x 0 , x , m; I n , T n ) T n (x 0 ; I n ) -T n (x 0 ; I n ∪ {x } m i=1 ) ,
where by healthy points we mean inliers i.e. samples that are independently drawn from the true density function.

Given this de nition, IF O∪I quanti es how much the value at x 0 of an estimated density function changes whenever the healthy dataset is increased by m points located at x . Therefore, the link with the notion of stability is made obvious: the smaller IF O∪I is, the more stable and thus robust the estimator is.

NUMERICAL EXPERIMENTS 125

In the next proposition, we provide a lower bound on the number of added samples m over which the IF O∪I of the MoM-KDE is lower that the one of KDE with high probability.

Proposition A.3. Let x , x 0 ∈ R d and I n be an healthy data set. Grant assumptions 1 to 4 and denote

a i∈In K X i -x 0 h , b K x -x 0 h .
Let S > 2m with m ∈ 0, n 2 the number of added samples and δ > 0 such that |ba/n| > C ρ 2δS/n.

If m ≥ Cρ √ 2nδS |b-a/n|-Cρ √ 2δS/n
, then with probability higher than 1 -4 exp(-δ) we have:

IF O∪I (x 0 , x , m; I n , fMoM ) ≤ IF O∪I (x 0 , x , m; I n , fKDE ) .
Given the previous proposition, the lower bound on m over which the MoM-KDE is better than KDE is not necessarily easy to interpret. When everything is xed except x 0 and x , we see that the bound is low whenever |ba/n| = |K(

x 0 -x h ) -1 n K( x 0 -X i h )| is large. A su
cient condition for this is to take x far from the sampling set I n , i.e take x as an outlier. Under this condition, the bound will get even lower whenever x 0 gets closer to x .

Numerical experiments

In this section, we display numerical results supporting the relevance of MoM-KDE. All experiences were run over a personal laptop computer using Python. The code of MoM-KDE is made available online1 .

Comparative methods. In the following experiments, we propose to compare MoM-KDE to the classical KDE and two robust versions of KDE, called RKDE [START_REF] Kim | Robust kernel density estimation[END_REF] and SPKDE [START_REF] Vandermeulen | Robust kernel density estimation by scaling and projection in hilbert space[END_REF].

As previously explained, RKDE takes the ideas of robust M-estimation and translate it to kernel density estimation. Authors point out that classical KDE estimator can be seen as the minimizer of a squared error loss in the Reproducing Kernel Hilbert Space H corresponding to the chosen kernel. Instead of minimizing this loss, they propose to minimize a robust version of it, i ρ( φ(X i )g H ), with respect to g ∈ H. Here φ is the canonical feature map and ρ(•) is either the robust Huber or Hampel function. The solution of the newly expressed problem is then found using the iteratively reweighted least squares algorithm. SPKDE proposes to scale the standard KDE in a way that it decontaminates the dataset. This is done by minimizing the function β fng 2 with respect to g, belonging to the convex hull of {k h (•, X i )} n i=1 . Here, β is an hyperparameter that controls the robustness and fn is the KDE estimator. The minimization is shown to be equivalent to a quadratic program over the simplex, solved via projected gradient descent.

Conclusion

This additional chapter introduced MoM-KDE, a new e cient way to perform robust kernel density estimation. The method has been shown to be consistent in both L ∞ and L 1 error-norm in presence of very generic outliers, enjoying a similar rate of convergence than the KDE without outliers. MoM-KDE achieved good empirical results in various situations while having a lower computational complexity than its competitors. This work proposed to use the coordinate-wise median to construct its robust estimator. Future works will investigate the use of other generalization of the median in high dimension, e.g. the geometric median. In addition, further investigation will include a deeper statistical analysis under the hurdle contamination model in order to analyse the minimax optimality [START_REF] Liu | Density estimation with contamination: minimax rates and theory of adaptation[END_REF] of MoM-KDE.

fn -f ∞ ≤ C 1 S| log(h)| nh d + C 2 h α ,
where

C 2 = L u α K(u)du < ∞
fMoM -f ∞ ≤ C 1 S log 2(2+δ) δ | log(h)| nh d + C 2 h α ,
where C 2 = L u α K(u)du < ∞ and C 1 is a constant that only depends on f ∞ , the dimension d, and the kernel properties.

Proof. From the de nition of the MoM-KDE, we have the following implication [START_REF] Lecué | Robust classi cation via mom minimization[END_REF] sup

x fMoM (x) -f (x) ≥ ε =⇒ sup x S k=1 I fns (x) -f (x) > ε ≥ S/2 .
Thus to upper-bound the probability of the left-hand event, it su ces to upper-bound the probability of the right-hand event. Moreover, we have

fns (x) -f (x) ≤ sup x fns (x) -f (x) =⇒ I fns (x) -f (x) > ε ≤ I sup x fns (x) -f (x) > ε =⇒ S k=1 I fns (x) -f (x) > ε ≤
I sup x fns (x) -f (x) > ε = S s=1 Z s = s∈S Z s + s∈S C Z s ≤ s∈S Z s + |O| = s∈S [Z k -E (Z s ) + E (Z s )] + |O| = s∈S [Z s -E (Z s )] + s∈S E (Z s ) + |O| ≤ S s=1 [Z s -E (Z s )] + S • E (Z 1 ) + |O| ≤ S s=1 [Z s -E (Z s )] + S • P sup x fn 1 (x) -f (x) > ε + |O| , (A.8)
where Z 1 is assumed, without loss of generality, to be associated to a block not containing outliers. This block always exists thanks to the hypothesis S > (2 + δ)|O|.

Let ε = C 1 S log( 2(2+δ) δ )| log(h)| nh d + C 2 h α , then using Lemma 1 with S = log( 2(2+δ) δ ), we have P sup x fn 1 (x) -f (x) > ε ≤ δ 2(2 + δ) .
Combining this last inequality with equation (A.8) leads to (A.9) PRINCIPLE From the L 1 -consistency of the KDE in probability, if the number of anomalies grows at a small enough speed [START_REF] Devroye | Nonparametric Density Estimation: The L1 View[END_REF], the left part is bounded, i.e. We now upper-bound the right part of equation (A.9). Let consider a particular block A s where s ∈ S C . In this block, the estimator f ns is selected and is calculated with samples containing anomalies. As ∀x ∈ A s , f ns (x) is the median (by de nition), if S > 2|O|, we can always nd a s ∈ S such that f ns (x) ≤ f n s (x) or f ns (x) ≥ f n s (x).

P S s=1 I sup x fns (x) -f (x) > ε ≥ S/2 ≤ P S s=1 [Z s -E (Z s )] + S • δ 2(2 + δ + |O| ≥ S/2 6. TECHNICAL PROOFS 133 ≤ P S s=1 [Z s -E (Z s )] ≥ S 1 2 - δ 2(2 + δ) - |O| S ≤ P S s=1 [Z s -E (Z s )] ≥ S 1 2 + δ - |O| S Tacking ∆ = 1 2+δ -|O| S > 0
Now let denote A + s = x ∈ A s | fns (x) ≥ f (x) and A - s = x ∈ A s | fns (x) < f (x) . We have A + s ∪ A - s =
A s and each one of these blocks can be decomposed respectively into |S| sub-blocks (not necessarily disjoint) {A s ,+ s } s ∈S and {A s ,- s } s ∈S such that ∀s ∈ S,

A s ,+ s = x ∈ A s | fn s (x) ≥ fns (x) ≥ f (x) and A s ,- s = x ∈ A s | fn s (x) ≤ fns (x) < f (x) .
Finally, the right-hand term of equation (A.9) can be upper-bounded by

s∈S C As fns (x) -f (x) dx ≤ s∈S C A + s fns (x) -f (x) dx + A - s fns (x) -f (x) dx ≤ s∈S C s ∈S A s ,+ s fns (x) -f (x) dx + A s ,- s fns (x) -f (x) dx ≤ s∈S C s ∈S A s ,+ s fn s (x) -f (x) dx + A s ,- s fn s (x) -f (x) dx
a i∈I K X i -x 0 h , b K x -x 0 h .
Let S > 2m with m ∈ 0, n 2 the number of added samples and δ > 0 such that |ba/n| > C ρ 2δS/n.

If m ≥ Cρ √ 2nδS |b-a/n|-Cρ √ 2δS/n
, then with probability higher than 1 -4 exp(-δ) we have: IF O∪I (x 0 , x , m; I n , fMoM ) ≤ IF O∪I (x 0 , x , m; I n , fKDE ) .

TECHNICAL PROOFS 135

Proof.

-The IF O∪I for the KDE is

IF O∪I (x 0 , x , m, I n ; fKDE ) = 1 (n + m)h d n i=1 K X i -x 0 h + m i=1 K x -x 0 h - 1 nh d n i=1 K X i -x 0 h = 1 h d 1 n + m n i=1 K X i -x 0 h - 1 n n i=1 K X i -x 0 h + m n + m K x -x 0 h = 1 h d 1 n + m - 1 n n i=1 K X i -x 0 h + m n + m K x -x 0 h = 1 h d 1 n + m - 1 n a + m n + m b = 1 h d nb -a n 2 /m + n , with a n i=1 K X i -x 0 h , and b K x -x 0 h .
-IF O∪I for the MoM-KDE Let S > 2m be the number of blocks in the MoM-KDE, {B s } S s=1 and { Bs } S s =1 be respectively the blocks of the contaminated data set I n ∪ {x } m and the healthy data set. We have:

IF O∪I (x 0 , x , m, I n ; fMoM ) = 1 h d Median S n + m   i∈In∩Bs K X i -x 0 h + i∈{x }∩Bs K x -x 0 h   S s=1 -Median    S n i∈ Bs K X i -x 0 h    S s =1 ≤ 1 h d S n + m i∈Bs K X i -x 0 h - S n i∈ Bs K X i -x 0 h ,
where Bs is the block selected by the median for the healthy MoM-KDE. The inequality is obtained by noticing that, with S > 2m, there always exists an healthy block B s that makes it true. Finally, denoting

i∈Bs S n + m K X i -x 0 h - i∈ Bs S n K X i -x 0 h = Z (s) -Z (s ) ,
and using both the triangular and Hoe ding inequalities, and the fact that E(Z (s) ) = E(Z (s ) ), we have

h d • IF O∪I (x 0 , x , m, I n ; fMoM ) ≤ Z (s) -Z (s ) -E(Z (s) ) + E(Z (s ) ) PRINCIPLE ≤ Z (s) -E(Z (s) ) + Z (s ) -E(Z (s ) ) ,
and for t > 0,

P Z (s) -E(Z (s) ) ≥ t ≤ 2 exp - 2t 2 (n + m)) SC 2 ρ P Z (s ) -E(Z (s ) ) ≥ t ≤ 2 exp - 2t 2 n SC 2 ρ = 2 exp - 2n s t 2 C 2 ρ ,
where n s = n/S. Furthermore, given two real-valued random variables X, Y , we know that for t > 0,

P (|X| + |Y | ≥ 2t) ≤ P (|X| ≥ t) + P (|Y | ≥ t) .
Therefore, we have

P Z (s) -E(Z (s) ) + Z (s ) -E(Z (s ) ) ≥ t ≤ 4 exp - n s t 2 2C 2 ρ . Setting t = C ρ √ 2δ √ n s
with δ > 0, nally gives P Z (s) -Z (s ) < t ≥ 1 -4 exp(-δ). We now know that with probability 1 -4 exp(-δ), IF O∪I (x 0 , x , m,

I n ; fMoM ) < 1 h d C ρ √ 2δ √ n s
and we seek for which value of m, this value is smaller than the IF O∪I of the KDE i.e.

C ρ √ 2δ √ n s ≤ nb -a n 2 /m + n ⇐⇒ n 2 /m + n ≤ √ n s |nb -a| C ρ √ 2δ ⇐⇒ 1/m ≤ √ n s |nb -a| C ρ √ 2δ -n /n 2 ⇐⇒ m ≥ n 2 √ n s |nb -a| C ρ √ 2δ -n ⇐⇒ m ≥ n 2 √ n|nb -a| C ρ √ 2δS -n ⇐⇒ m ≥ n √ n|b -a/n| C ρ √ 2δS - 1
. Parmi cette grande quantité de données, un certain nombre d'entre elles apparaissent naturellement sur une structure en réseau, interconnectant les observations ou les variables qui les caractérisent. C'est notamment le cas des données provenant de réseaux sociaux, réseaux de communication, réseaux de transport ou de réseaux de capteurs, où elles sont collectées au niveau des noeuds d'un graphe. Par exemple dans un réseau social, les noeuds du graphe associé peuvent correspondre aux utilisateurs et une arête peut indiquer le lien social qui réside entre deux d'entre eux. Dans un réseau de capteurs, une arête peut simplement caractériser la distance spatiale entre deux capteurs. Généralement, le graphe apporte des connaissances sur le processus qui génère les données (par exemple, les valeurs observées à deux noeuds liés dans le graphe peuvent être fortement corrélés ou être très proches), et construire des modèles ou des algorithmes d'apprentissage -y compris des méthodes de détection d'anomalies ou de détection de ruptures -à partir de ce type de données, tout en prenant en compte la structure de réseau associée, est important pour améliorer les performances d'apprentissage.

B

Introduction en français

Ce type de données est appelé vecteurs sur graphe ou signaux sur graphe. Comme indiqué plus haut, elles se réfèrent simplement à des données vectorielles dont chaque composante est associée à un unique noeud d'un même graphe. Alors que dans certains cas, le graphe est naturellement donné et donc connu a priori (par exemple, dans le cas du réseau social ou du réseau de capteurs donné ci-dessus), il existe de nombreux cas où les données admettent une structure de graphe sous-jacente qui n'est pas disponible et que l'on cherche à apprendre à partir de ces données. C'est par exemple le cas en biologie, où l'on s'intéresse à savoir quels gènes (ou protéines) sont exprimés les uns avec les autres [62,[START_REF] Kitano | Computational systems biology[END_REF]. Plus généralement, ce besoin d'inférence de graphe peut apparaître pour tout type de données pour lesquelles on souhaite savoir quelles variables sont liées à quelles autres. Cette tâche peut avoir un fort impact sur la visualisation et la compréhension des données traitées, mais aussi, comme dit précédemment, sur la capacité à construire des algorithmes d'apprentissage plus e caces.

Ce travail de thèse est profondément ancré dans les di érents sujets mentionnés cidessus. Il se concentre en particulier sur la détection d'événements dans un ensemble de 1. CONTEXTE DE LA THÈSE 139 vecteurs sur graphe, que le graphe soit connu ou non. Il se concentre également sur la tâche d'inférence du graphe lui-même, et sur la détection de changement dans la structure du graphe sous-jacente aux données.

Contexte industriel

Cette thèse de doctorat a été réalisée grâce au programme CIFRE (Convention Individuelle de Formation par la Recherche) et l'ANRT (Agence Nationale de la Recherche et de la Technologie). Elle a été sponsorisée par Sigfox, un opérateur mondial de télécommunication dédié à l'Internet des objets (IoT). Plus précisément, Sigfox possède des antennes, appelées stations de base (BS), qui sont installées sur des tours (de la même manière qu'un opérateur de téléphonie mobile) et reçoivent des transmissions de données provenant d'objets tels que des capteurs de stationnement, des alarmes à incendie, des compteurs d'eau ou d'électricité, etc. Ces objets sont détenus par des clients, et le rôle de Sigfox est essentiellement de collecter les données transmises par les objets, via les BS, et de les renvoyer sur un cloud auquel le client a accès. La particularité de Sigfox est son protocole simple, qui permet aux objets d'envoyer des petites quantités de données (12 bytes maximum par "message"), de manière peu coûteuse en énergie et qui sont reçues e cacement par les BS sur de très longue distance. Sans entrer dans les détails, décrivons brièvement le processus d'envoi d'un message dans le protocole Sigfox.

Soit un objet cherchant à transmettre une petite quantité de données (température, préssion, informations binaires, etc.) à son propriétaire. L'information est encodée dans un signal qui est envoyé trois fois, à trois niveaux de fréquence di érents. Les signaux sont envoyés sans aucun protocole de sélection de stations de base alentours, ils sont simplement envoyés "dans l'air", en espérant qu'ils seront reçus par au moins une station de base environnante (grâce à une vaste couverture, les signaux sont souvent reçus par un grand nombre d'entre elles). Une fois que certaines des stations de base alentours ont reçu au moins un des trois signaux, celui-ci est décodé et envoyé dans un cloud en utilisant un Les données collectées au niveau de chaque BS sont diverses, allant de données matérielles (par exemple la température du moteur) aux informations logicielles (par exemple la version du système d'exploitation utilisée). Toutefois selon les experts, les plus importantes pour détecter les anomalies sont celles liées au spectre des signaux "observés" par la station de base. Malheureusement, l'entiereté du spectre n'est pas collecté et seules quelques statistiques qui le résument le sont. Ces dernières étant essentiellement quelques quantiles des intensités enregistrées sur l'ensemble du spectre. Après plusieurs semaines d'analyse de données, il a nalement été décidé que seules les informations de "réception" de chaque station de base seraient conservées. En d'autres termes, les informations sur l'activité du réseau, c'est-à-dire pour chaque message envoyé dans le réseau, quelle BS l'a reçu ou non. Cette décision est essentiellement motivée par le fait que ces données sont brutes (contrairement aux données du spectre qui sont déjà traitées) et qu'elles comportent a priori peu d'erreurs. En outre, on s'attend intuitivement à ce qu'une défaillance au niveau d'une station de base ait un impact direct sur son niveau d'activité (par exemple, avec une diminution du nombre total de signaux reçus).

Une propriété intéressante de ces données de réception est qu'elles apparaissent naturellement sur une structure de graphe, induite par la distribution spatiale des stations de base. En prenant par exemple le vecteur qui indique, pour un certain message envoyé, quelle station de base l'a reçu ou non, on observe empiriquement que des stations de base proches ont plus de chances de recevoir un même signal, et inversement. Un autre exemple est le vecteur qui spéci e le nombre de signaux reçus par chaque station de base sur une période donnée. Dans ce cas, les stations de base proches auront des valeurs fortement corrélées (voir gure B.4). Au vu de ces observations et de la structure en réseau inhérente aux données, nous pouvons conclure qu'elles peuvent être considérées comme des vecteurs sur graphes, faisant ainsi le lien vers le problème de détection d'événements dans un ensemble de signaux sur graphes. De plus, rapidement, et comme expliqué en détail dans la section suivante, il a été réalisé que le graphe spatial induit n'était pas toujours adapté aux données, ce qui a conduit au deuxième axe de la thèse, à savoir l'inférence de la structure de graphe elle-même.

Objectifs et motivations

Dans ce qui suit, nous décrivons les di érents objectifs qui seront au coeur de ce document.

Détecter des changements de comportement ou des anomalies au niveau des noeuds d'un réseau de communication. Ce premier objectif est appliqué et essentiellement motivé par le problème industriel soulevé par la collaboration avec Sigfox. Cependant, ce problème peut se poser pour de nombreux autres type de réseaux de communication (e.g. certain réseaux sociaux ou réseaux d'ordinateurs), et a n que ce travail de thèse soit le plus générale possible, nous chercherons à proposer une ou plusieurs méthodes de détection d'événements applicable pour un large spectre de réseaux de communication, incluant Sigfox. Si dans le cadre de notre collaboration industrielle, une telle tâche pourrait se résumer à la détection d'une panne de station de base, dans un réseau de capteurs [START_REF] Xie | Anomaly detection in wireless sensor networks: A survey[END_REF] elle peut caractériser la détection d'un problème au niveau d'un capteur ou de la valeur qu'il quanti e. Dans un réseau d'ordinateurs ou un réseau social, un tel problème de détection peut se poser, par exemple, pour des questions de sécurité du réseau où l'anomalie peut provenir d'une attaque (piratage, fraude à l'identité, etc.) [START_REF] Ahmed | A survey of network anomaly detection techniques[END_REF]. Les exemples précédents illustrent bien l'importance d'une telle tâche : les anomalies non détectées peuvent avoir des répercussions importantes sur les performances du réseau considéré. En résumé, nos objectifs seront, tout d'abord, de donner une dé nition simple d'un réseau de communication, qui puisse correspondre à un large éventail de réseaux. De même, nous examinerons des notions simples d'anomalies qui peuvent être observées pour di érents types de réseaux. En n, nous proposerons un moyen de détecter les anomalies. A n de rester proche de notre application industrielle, nous nous limiterons à la détection au niveau des noeuds, c'est-à-dire à la détection d'événements au niveau d'une seule entité (par exemple une station de base, un ordinateur, etc.).

Détecter des ruptures ou des anomalies dans un ensemble de signaux sur graphe. Cette tâche plus générale a été initialement motivée par la conclusion que nous avons faite dans la section précédente, à savoir le fait que nos données sont des vecteurs sur graphes. De plus, comme expliqué dans la section précédente, ces types de signaux sont fréquemment observés dans le monde réel, et il est donc important de construire des algorithmes de détection d'anomalies ou de détection de ruptures qui leur soient adaptés. Lorsque le graphe est connu, il constitue une connaissance a priori du modèle qui génère les données. Il est donc concevable que les performances des algorithmes puissent être améliorées en utilisant ces informations supplémentaires [START_REF] Egilmez | Spectral anomaly detection using graph-based ltering for wireless sensor networks[END_REF]. La connaissance du graphe permet notamment la construction de nouvelles variables ou représentations basées sur celui-ci, et ainsi, permet de découvrir des types d'événements plus complexes [START_REF] Chen | Change-point detection of gaussian graph signals with partial information[END_REF]. Typiquement, une anomalie détectée au niveau d'un noeud peut provenir d'une valeur anormale, par rapport à ellemême et par rapport à ses voisins. Lorsque le graphe est inconnu, peu de conclusions supplémentaire peuvent être faite. Cependant, savoir qu'il existe une structure de graphe sous-jacente indique une chose importante : si l'on veut détecter une anomalie ou un point de rupture au niveau d'une dimension spéci que (c'est-à-dire un noeud spéci que), les valeurs observées aux autres noeuds doivent être prises en compte également puisqu'elles sont liées par les arêtes du graphe [START_REF] Bars | A probabilistic framework to node-level anomaly detection in communication networks[END_REF]. Un graphe inconnu suggère toutefois une première étape d'apprentissage du graphe lui-même, a n de mieux comprendre les données et d'appliquer des algorithmes de détection d'événements plus adaptés. Cette procédure d'apprentissage constitue notre troisième objectif.

Apprendre la structure du graphe sous-jacente à des données vectorielles. Cet objectif se pose dans de nombreux domaines et peut être appliqué à tout type de données vectorielles. La tâche est ici d'inférer la structure de graphe sous-jacente, et bien sûr inconnue, à des données. En d'autres termes, l'objectif est d'apprendre des relations entre des variables, c'est-à-dire avec quelles autres variables, une variable d'intérêt est plus similaire ou liée (en terme de corrélation, d'indépendance conditionnelle, d'échelle, de valeurs, etc.). Cette procédure d'apprentissage se fait en utilisant un ensemble de vecteurs, tous supposés admettre le même graphe sous-jacent. Au cours du processus d'apprentissage, des pénalités structurelles peuvent être imposées, notamment sur la parcimonie du graphe [START_REF] Dong | Learning laplacian matrix in smooth graph signal representations[END_REF][START_REF] Friedman | Sparse inverse covariance estimation with the graphical lasso[END_REF][START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF], et plusieurs d'entre elles seront étudiées dans cette thèse.

L'apprentissage d'une telle structure de graphe a de nombreuses applications di érentes. Tout d'abord, elle permet de comprendre et d'interpêter les données vectorielles considérées grâce à la visualisation simple apportée par le graphe. De plus, une telle tâche d'apprentissage relie souvent les données à un modèle. C'est par exemple le cas des champs aléatoires de Markov [START_REF] Koller | Probabilistic graphical models: principles and techniques[END_REF], qui dans de nombreuses situations suppose une relation linéaire entre les variables, relation déterminée par le graphe lui-même [START_REF] Hö Ing | Estimation of sparse binary pairwise markov networks using pseudo-likelihoods[END_REF][START_REF] Ravikumar | High-dimensional ising model selection using l1-regularized logistic regression[END_REF]. Dans de tels cas, on peut prédire la valeur d'un noeud en fonction des valeurs des autres noeud. Cela illustre bien l'applicabilité du graphe appris. En n, le graphe peut également être utilisé dans de nombreux algorithmes d'apprentissage qui nécessitent un graphe, typiquement l'algorithme du partitionnement spectral [START_REF] Ng | On spectral clustering: Analysis and an algorithm[END_REF], certains algorithmes d'apprentissage semi-supervisés [START_REF] Belkin | Semi-supervised learning on riemannian manifolds[END_REF], dans le cadre du traitement du signal sur graphe [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF] etc.

L'une des applications les plus célèbres de l'apprentissage des graphes dans le monde réel se trouve en biologie, avec les réseaux d'interaction des gènes. Ceux-ci mettent en évidence les gènes qui sont la plupart du temps exprimés ensemble. À Sigfox, on pourrait penser que le graphe est directement donné par la position spatiale des stations de base (Figure B.4), vu que que des stations de base voisines recevront probablement beaucoup de messages en commun. En pratique, ce n'est pas toujours vrai, deux stations de base situées à une faible distance (à vol d'oiseau) l'une de l'autre peuvent être séparées par un mur ou être à des altitudes di érentes, ce qui les rend di érentes dans leur capacité à recevoir des signaux. Cette observation nous a permis de conclure que la tâche d'apprentissage de graphe pouvait également être intéressante avec les données de réception Sigfox.

Détecter des changements dans la structure de graphe sous-jacente à des données vectorielles. Cet objectif peut être considéré comme une combinaison de l'apprentissage de graphe et de la détection de ruptures. En fait, contrairement à la grande majorité des techniques de détection de ruptures qui recherchent un changement signi catif dans la moyenne d'une série temporelle, la tâche ici est de détecter un changement dans le graphe sous-jacent aux données. L'objectif est donc double : trouver des moments entre lesquels toutes les données vectorielles observées ont la même structure de graphe sous-jacente et apprendre ces graphes. Une illustration de cette tâche est fournie par la Figure B.5 pour une série temporelle à valeur réelle et à 3 dimensions. En plus de déterminer les moments où le système a changé, les méthodes qui répondent à cet objectif tirent également parti des avantages liés à l'inférence des graphes (voir le paragraphe précédent), c'est-à-dire la modélisation, l'applicabilité de certains algorithmes d'apprentissage automatique, etc. En particulier, la visualisation que cet apprentissage permet entraîne une forte compréhension et une grande capacité d'interprétation des points de rupture trouvés [START_REF] Bars | Learning the piece-wise constant graph structure of a varying ising model[END_REF].

Travailler avec des signaux sur graphe binaires. Ce dernier objectif est essentiellement motivé par le fait que les données vectorielles considérées sont binaires. En e et, rappelons que dans notre cadre d'application, une donnée brute est un vecteur, caractérisant un message Sigfox, qui encode quelle BS a reçu le signal ou non. Néanmoins, ce problème reste important dans de nombreux autres contextes [5], notamment parce qu'il est souvent moins étudié que pour les données vectorielles ou les séries temporelles à valeur réelle.

Préliminaires

Dans cette section, nous proposons de rappeler brièvement quelques fondamentaux sur la détection d'événements, à savoir la détection d'anomalies et la détection de ruptures, sur la théorie des graphes et sur les vecteurs observés sur graphes. L'objectif est de fournir quelques dé nitions, propriétés et algorithmes de base qui seront utiles dans le reste du manuscrit.

Détection d'anomalies

Dans sa version la plus classique, le problème de détection d'anomalies cherche à trouver dans un ensemble de données, une petite quantité de vecteurs qui ont été générés par une distribution de probabilité di érente de celle générant la majorité des points. Cette formulation simple a motivé de nombreuses méthodes statistiques de détection d'anomalies qui supposent essentiellement que les anomalies se trouvent dans des régions de faible densité. Parmi ces méthodes, celle de [START_REF] Eskin | Anomaly detection over noisy data using learned probability distributions[END_REF] qui suppose que les données normales sont générées par une distribution de probabilité connue à l'avance et qui considère les points se trouvant dans une région de faible probabilité comme anormaux.

Tout en étant la plus générale, la formulation précédente correspond en fait à un cadre particulier de détection d'anomalies. En e et, sur la base des étiquettes disponibles, les tâches de détection d'anomalies peuvent être divisées en trois types. La détection d'anomalie supervisée, qui consiste à entraîner l'algorithme sur la base d'un ensemble de données étiquetées comprenant des observations normales et anormales. Ce cadre est donc fortement lié à un problème de classi cation supervisée [START_REF] Sun | Classi cation of imbalanced data: A review[END_REF]. Le second scénario est le Il existe une grande variété d'algorithmes de détection d'anomalies, de ceux basés sur l'inférence de la densité des données [START_REF] Breunig | Lof: identifying density-based local outliers[END_REF][START_REF] Schölkopf | Estimating the support of a high-dimensional distribution[END_REF][START_REF] Scott | Learning minimum volume sets[END_REF][START_REF] Vert | Consistency and convergence rates of one-class svms and related algorithms[END_REF], à ceux basés par exemple sur les arbres de décision [START_REF] Liu | Isolation forest[END_REF]. Cette grande variété d'algorithme est accentuée par les di érents étiquetages expliqués plus haut, mais aussi par le type de données analysées. Nous nous sommes ici concentrés sur les données vectorielles classiques, mais il pourrait s'agir de données temporelles ou même textuelles. Il serait donc impossible de dresser ici une liste exhaustive de ces méthodes, et pour des études complètes, on peut se référer à [START_REF] Chandola | Anomaly detection: A survey[END_REF][START_REF] Patcha | An overview of anomaly detection techniques: Existing solutions and latest technological trends[END_REF]. Dans ce qui suit, nous présentons un algorithme e cace de détection d'anomalies, pour des données vectorielles standard, qui donne de bons résultats dans les scénarios d'étiquetage non supervisé et semi-supervisé.

SVM à une classe

La machine à vecteurs de support (SVM) à une classe, introduite pour la première fois dans [START_REF] Schölkopf | Estimating the support of a high-dimensional distribution[END_REF], étend le SVM standard pour la classi cation à deux classes au problème de la détection de nouveauté. En e et, plutôt que d'avoir accès à un ensemble de données étiquetées avec des étiquettes positives et négatives, il suppose que les données d'entrée appartiennent toutes à la classe 1 (la classe normale). Ensuite, au lieu de construire un hyperplan séparant deux classes, il construit un hyperplan séparant les points d'entrée (mappé dans un espace de redescription), de l'origine de l'espace, traité ici comme le seul point de la seconde classe. Formellement, dans sa version à marge souple, le SVM à une classe fonctionne comme suit.

Soient x (1) , . . . , x (n) ∈ R d , n observations et Φ : R d → H une fonction a valeures dans H, un espace de Hilbert à noyau reproduisant [START_REF] Aronszajn | Theory of reproducing kernels[END_REF] associé au noyau k. Pour séparer les données de l'origine, le SVM à une classe résout le programme quadratique suivant :

min ω∈R d ,ξ∈R n ,ρ∈R 1 2 ω 2 + 1 n n i=1 ξ i -νρ s.t w, Φ(x (i) ) ≥ ρ -ξ i , i = 1, . . . , n ξ i ≥ 0,
où ν ∈ (0, 1) est un hyperparamètre qui empêche le sur-apprentissage et permet à l'ensemble des données d'entraînement de contenir des valeurs anormales. En fait, on peut montrer que ν correspond à une borne supérieure sur la fraction d'anomalies qui peuvent être présete dans l'ensemble d'apprentissage. La fonction de score utilisée pour détecter les anomalies est donnée par

f (x) = sign( w, Φ(x (i) ) -ρ),
qui, comme souhaité, sera positive pour la plupart des vecteurs d'apprentissage. D'après la dé nition précédente, le SVM à une classe peut être vu comme un algorithme d'estimation d'un espace de volume minimum contenant presque toutes les données d'entrée. Il est lié à l'estimation d'un ensemble de volume minimal [START_REF] Scott | Learning minimum volume sets[END_REF] de masse 1ν, c'est-à-dire l'ensemble de volume minimal, par rapport à la mesure de Lebesgue, ayant une masse de 1ν, par rapport à la mesure de probabilité des données normales.

Mesures de qualité

Pouvoir évaluer la qualité des algorithmes de détection d'anomalies est, comme pour toute autre méthode d'apprentissage, très important. Lorsque des étiquettes sont disponibles, toutes les mesures de qualité utilisées pour évaluer la qualité d'une fonction de score ou d'un algorithme de classi cation binaires peuvent également être utilisées. On pourra par exemple analyser la courbe ROC et l'aire sous cette courbe (AUC). L'un des inconvénients de l'utilisation de ces mesures est qu'elles ne sont pas particulièrement adaptées à des classes déséquilibrées. Pour cette raison, des mesures liées à la classe normale, telles que le taux de faux positifs, peuvent être préférées.

Lorsque peu ou pas de labels sont disponibles, la question de l'évaluation de la qualité des algorithmes reste ouverte. Certaines pistes, cherchant à étendre la notion de courbes ROC au scénario sans étiquette, ont notamment été étudiées dans [START_REF] Goix | How to evaluate the quality of unsupervised anomaly detection algorithms[END_REF] et sont liées aux courbes d'excès de masse et Masse-Volume. Toutefois, ces méthodes ne seront pas prises en compte dans le présent manuscrit et, le cas échéant, des étiquettes seront disponibles.

Détection de ruptures

La détection de ruptures est une tâche particulière de l'analyse des séries temporelles. Son objectif est de trouver des moments où des changements signi catifs se sont produits dans le modèle sous-jacent à une série temporelle. Comme indiqué dans la Section 1.1, cette tâche a de nombreuses applications, que ce soit dans le traitement de la parole [76], en climatologie [START_REF] Reeves | A review and comparison of changepoint detection techniques for climate data[END_REF], dans l'analyse des données de tra c d'un réseau [START_REF] Lévy-Leduc | Detection and localization of change-points in highdimensional network tra c data[END_REF] etc. Une illustration de ce problème de segmentation est fournie dans la gure B.1.

La tâche de détection de ruptures peut être divisée en deux grandes catégories : la détection hors ligne dite a posteriori, et celle en ligne dite séquentielle. Dans le cadre hors ligne, la segmentation est e ectuée après que le signal ait été entièrement observé. Au contraire, dans le scénario en ligne, on cherche à trouver les moments de rupture en temps réel, au fur et à mesure que les vecteurs sont observées. Dans cette section, nous nous concentrons sur le scénario hors ligne, lui-même considéré au chapitre 4. Ce problème de détection de rupture hors ligne peut à nouveau être divisé en deux catégories, le cas où le nombre de rupture à retrouver est connu et lorsqu'il est inconnu. La plupart du temps, la résolution des deux problèmes di ère avec l'ajout, dans le programme d'optimisation, d'un terme pénalisant le nombre de rupture estimé. Avant d'aller plus loin, décrivons maintenant notre problème de manière plus formelle.

Nous considérons que le cadre statistique décrit dans [START_REF] Truong | Selective review of o ine change point detection methods[END_REF]. Soit {x (i) } n i=1 une série temporelle à valeurs dans R d et supposée stationnaire par morceaux, ce qui signi e qu'il existe des instances de temps T = {t 1 , . . . , t K }} ⊂ {1 . . . , n} auxquelles le modèle sous-jacent à la série temporelle à changé. L'objectif de la détection de rupture est de trouver les temps T , et donc le nombre de ruptures lorsque celui-ci est inconnu. Pour cela, la plupart des méthodes que l'on trouve dans la littérature recherchent un ensemble d'indices de temps T = {t k } K k=1 ⊂ {1 . . . , n}, estimant T et tel qu'ils minimisent une fonction de la forme Quand le nombre de ruptures est connu K = K , sinon, un terme pénalisant la taille de T est ajouté à la fonction objective. Les méthodes de détection de ruptures di èrent ensuite sur deux aspects principaux. Soit sur la fonction de coût utilisée, généralement liée au modèle sous-jacent aux données (e.g. paramétrique ou non), soit sur la méthode utilisée pour résoudre le problème de minimisation mentionné ci-dessus. Le problème étant combinatoire, de nombreuses solutions ont été proposées (méthode gourmande, programmation dynamique etc.). La grande variété des fonctions de coût et des méthodes de minimisation ne nous permet pas d'être exhaustifs et nous invitons le lecteur à consulter la revue de [START_REF] Truong | Selective review of o ine change point detection methods[END_REF] pour des exemples et des discussions approfondies. Néanmoins, nous donnons ci-dessous un exemple de modèle et de fonction de coût associée qu'il nous semble important de connaître.

Example B.1. L'approche du maximum de vraisemblance. Dans cet exemple, les échantillions de la séries temporelle sont supposés indépendants et identiquement distribués (iid) par morceaux. En d'autres termes, pour une famille de densités paramétriques {f (•|θ)} donnée, nous avons ∀i = 1 . . . , n :

x (i) ∼ K k=1 f (•|θ k )1{t k ≤ k < t k+1 }.
Une façon d'apprendre les paramètres d'un tel modèle, et donc d'estimer les ruptures, est de maximiser la vraisemblance. La fonction de coût correspondante est la suivante :

c({x (i) } t k+1 i=t k +1 ) = -sup θ t k+1 i=t k +1 log f (x (i) |θ).
Le cadre précédent est probablement l'un des plus considérés pour la détection de rupture [START_REF] Frick | Multiscale change point inference[END_REF][START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF][START_REF] Sen | On tests for detecting change in mean[END_REF]. Le modèle i.i.d. par morceaux est par ailleurs celui considéré au chapitre 4, mais avec une fonction de coût légèrement di érente.

Mesures de qualité

Là encore, en supposant l'accès aux véritables points de rupture, de nombreuses mesures ont été proposées pour évaluer la qualité des algorithmes de segmentation. Parmi elles, des métriques basées sur celles de la classi cation binaire (rupture ou non) telles que le score F 1 . Cependant, ces dernières ne prennent pas en compte l'aspect temporel du problème et on préférera par exemple utiliser la métrique de Hausdor . Celle-ci dé nit l'erreur h(T , T ) de l'ensemble des points de rupture estimés par rapport aux points réels comme étant la plus grande distance temporelle entre un point de changement réel et un estimé : Une telle mesure a l'avantage de pénaliser à la fois une sur-segmentation et une soussegmentation.

Alors que les mesures précédente évaluent la qualité d'un algorithme de segmentation de manière empirique, il est également important d'évaluer la qualité d'un algorithme d'un point de vue théorique. Cela se concrétise avec démontrant la consistence des estimateurs [START_REF] Truong | Selective review of o ine change point detection methods[END_REF], qui précise que lorsque le nombre d'échantillons dans chaque segment tend vers l'in ni, on a P(K = K ) → 0 et n -1 h(T , T ) → 0 en probabilité.

Théorie des graphes et modèles pour les vecteurs sur graphes 3.3.1 Dé nitions basiques

Les graphes sont des objets mathématiques décrivant des systèmes potentiellement complexes via un ensemble d'entités interconnectées, appelées noeuds. Ils apparaissent dans de nombreux domaines et applications, notamment ceux qui font intervenir la notion de réseaux, tels que les réseaux biologiques, les réseaux de neurones, les réseaux de capteurs, les réseaux informatiques, les réseaux de télécommunication, les réseaux sociaux, les réseaux de transport etc. Les graphes sont alors les outils les plus utilisés pour décrire et modéliser ces réseaux. Dans ce qui suit, nous donnons quelques dé nitions et concepts de base de la théorie des graphes. Tout comme la matrice d'adjacence, la matrice laplacienne décrit entièrement le graphe qui lui est associé. En particulier, elle est connue pour contenir d'importantes caractéristiques topologiques du graphe et pour être liée à la théorie spectrale des graphes. Par exemple, le nombre de valeurs propres nulles de L correspond au nombre de composante connexe du graphe. Une composante connexe étant un sous-ensemble de noeuds pour lesquels il existe toujours un chemin entre eux, et pour lesquels il n'existe aucun chemin avec un autre noeud. Une illustration d'un graphe avec deux composantes connexes est donnée sur la gure B.8e.

De nition B.7. (Vecteur sur graphe.) Un vecteur sur graphe, également appelé signal sur graphe ou fonction sur graphe, est une fonction x : V → R qui assigne une valeur réelle à tous les noeuds d'un graphe G = (V, E). Cette fonction peut être représentée par un simple vecteur x ∈ R p avec x i la valeur de x au noeud i.

Ce dernier objet permet de dé nir des données vectorielles qui sont observées sur une structure en réseau, un aspect fondamental des données qui sont considérées tout au long de ce travail de thèse. Néanmoins, une question demeure : qu'apporte le graphe à la modélisation de ce type de vecteurs ? En e et, pris comme cela un vecteur sur graphe reste un simple vecteur. Dans les sections suivantes, nous présentons deux points de vue di érents qui répondent à cette question. Ces deux cadres sont tous deux considérés dans des chapitres indépendant de la thèse.

Le cadre du traitement du signal sur graphe

Le traitement du signal sur graphe (Graph Signal Processing -GSP) [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF][START_REF] Shuman | The emerging eld of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF] est un domaine relativement récent dont l'objectif est d'étendre la plupart des outils développés dans le domaine du traitement du signal et des images au traitement de signaux sur graphes. Ainsi, des notions telles que la régulariré des signaux, l'échantillonnage ou la représentation spectrale d'un signal ont été développées pour ce type de données. En fait, les signaux temporels et les images sont considérés comme des cas particuliers de signaux sur graphe où le graphe associé correspond soit à une ligne pour un signal temporel, soit à une grille pour une image (voir gure B.8). Dans le contexte du GSP le graphe peut maintenant être quelconque. En n, dans ce cadre, et comme pour les signaux temporels ou les images, la valeur enregistrée à un noeud est vue comme une version décalée des valeurs enregistrées aux noeuds voisins.

Dans la suite nous rappelons quelques notions de base du GSP et les propriétés supposées être partagées par la plupart des vecteurs sur graphes. w ij (y iy j ) 2 ≤ s .

La dé nition précédente donne une notion de régularité pour les signaux sur graphes. Intuitivement, un signal sur graphe y a une régularité de niveau s par rapport à G si les noeuds adjacents du graphe ont des valeurs de signal su samment proche. Plus s est petit, plus le signal sur graphe est lisse. Dans le cadre du GSP, les signaux sur graphes sont la plupart du temps supposés lisses par rapport à leurs graphes associés (s petit).

Remark B.2. Le cas particulier où s = 0 implique que tous les noeuds voisins ont la même valeur.

De nition B.9. (Transformée de Fourier graphique.) Soit G = (V, E) un graphe non orienté et L = XΛX T la décomposition en éléments propres de sa matrice laplacienne. La transformée de Fourier graphique (GFT) d'un signal sur graphe y ∈ R p est donnée par h = X T y, où les composantes de h sont interprétées comme des coe cients de Fourier, les valeurs propres Λ comme des fréquences et les vecteurs propres X comme une base de Fourier.

Cette dé nition est initialement motivée par le fait qu'appliquée aux signaux temporels ou aux images, on retrouve la transformée de Fourier classique. De plus, il est empiriquement observé que les vecteurs propres de X associés aux plus petites valeurs propres de Λ présentent moins de variabilité entre les valeurs des noeuds que ceux associés aux plus grandes, ce qui motive également la comparaison avec l'analyse de Fourier. Avec cette dé nition, y admet une représentation spectrale k-parcimonieuse si le nombre d'éléments non nuls dans son vecteur de coe cients de Fourier est inférieur ou égal à k. Dans le cadre du GSP, être k-bandlimité est la seconde propriété supposée être partagée par la plupart des vecteurs sur graphes. Lorsque k est petit, cela indique notamment que le signal peut être reconstruit à partir d'un petit nombre de valeurs de noeuds. De plus, combiné à la notion de régularité, les coe cients nuls auront plus de chances d'être associés à des grandes valeurs propres (autrement dit des fréquences haute). Cela est en général observé lors de l'analyse spectrale de signaux temporels standards où la grande variabilité des valeurs observées à des noeuds voisins s'explique principalement par la présence de bruit.

Un cadre probabiliste

Dans la partie précédente, les vecteurs et leur graphe associé étaient liés entre eux par des propriétés issues du traitement du signal, à savoir la régularité ou lissité et la parcimonie de la représentation spectrale. Dans le cadre présenté ici, les signaux sur graphes sont vu comme des vecteurs aléatoires tirés selon une distribution de probabilité particulière : un champ aléatoire de Markov (Markov Random Fields -MRF). Pour ce type de distribution, le graphe encode une structure de dépendance particulière qui est expliquée dans ce qui suit. Compte tenu de la dé nition précédente, nous comprenons que la construction de MRF n'est pas évidente. Pour cette raison, les MRF sont souvent réduits à la classe des distributions de probabilité qui se factorise, une classe de distribution qui a la particularité de valider les trois propriétés requises pour être un MRF.

De nition B.13. (Factorisation.) Soit G = (V, E) un graphe non orienté et X un vecteur aléatoire indexé par V admettant une mesure de probabilité P X (•). Rappelons qu'une clique est un sous-ensemble de noeuds qui sont tous reliés entre eux. On dit que P X (•) se factorise en G si elle est de la forme :

P X (x) = 1 Z C∈C ψ C (x C ), (B.3)
où C est l'ensemble de toutes les cliques de G, ψ C (•) sont des fonctions de potentiel postitives et Z est une constante de normalisation.

Les fonctions de potentiel peuvent être arbitraires et, pour une distribution de probabilité donnée qui se factorise, ne sont pas nécessairement dé nies de manière unique. Les modèles graphiques gaussiens et les modèles d'Ising sont des exemples célèbres de distributions qui factorisent, et qui sont donc des MRF. Le premier est facile à caractériser : pour tout graphe G admettant une matrice de poids W , le vecteur X ∼ N (•, W -1 ) se factorise en G. Quant aux modèles d'Ising, ils sont considérés au chapitre 4 dans lequel des rappels les concernant y sont faits. Cette méthode se montre performante aussi bien sur des données synthétiques que sur des données provenant du réseau Sigfox, ce qui nous permet de conclure à la résolution du premier objectif. De plus, l'approche présentée est liée à d'autres objectifs présentés en section 2, en particulier à la tâche de détection d'événements pour des vecteurs sur graphes et d'inférence de graphes, sujets au coeur de la thèse dans les chapitres suivants.

• Chapitre 3: Inférence de structure à partir de signaux sur graphes réguliés et bandlimités.

Dans ce chapitre, on étudie le problème de l'apprentissage de la structure sous-jacente à un ensemble de vecteurs, c'est-à-dire le graphe sur lequel ils sont observés. Ce chapitre est donc lié au troisième objectif décrit dans la section 2 et se place dans le cadre du traitement du signal sur graphe. On suppose que les vecteurs sur graphe admettent une représentation parcimonieuse dans le domaine spectral du graphe, une propriété qui notamment caractérise des clusters dans un graphe. De plus, les signaux sont supposés régulié par rapport au graphe sous-jacent. A n d'inférer le graphe, nous proposons un programme d'optimisation qui apprend la matrice Laplacienne associée. Nous présentons deux algorithmes pour le résoudre, appelés IGL-3SR et FGL-3SR. Basés sur une procédure de descente par block, les deux algorithmes reposent sur des méthodes de minimisation standard -telles que de la descente de gradient sur une variété ou de la programmation linéaire -et sont de compléxité moins grande que les algorithmes de l'état de l'art. Alors que IGL-3SR est assuré de converger, FGL-3SR procède à une relaxation qui lui permet d'être signi cativement Enn, nous proposons de combiner la tâche d'apprentissage des graphes avec le problème de détection de ruptures. Cette fois, un cadre probabiliste est considéré pour modéliser les vecteurs, supposés ainsi être distribués selon un certain champ aléatoire de Markov. Dans notre modélisation, le graphe sous-jacent aux données peut changer dans le temps et un point de rupture est détecté chaque fois qu'il change de manière signicative.

Title:

Event detection and structure inference for graph vectors Keywords: Graphs, Networks, Anomaly and change-point detection, Markov Random Fields Abstract: This thesis addresses dierent problems around the analysis and the modeling of graph signals i.e. vector data that are observed over graphs. In particular, we are interested in two tasks. The rst one is the problem of event detection, i.e. anomaly or changepoint detection, in a set of graph vectors. The second task concerns the inference of the graph structure underlying the observed graph vectors contained in a data set.

At rst, our work takes an applicationoriented aspect in which we propose a method for detecting antenna failures or breakdowns in a telecommunication network. The proposed approach is designed to be eective for communication networks in a broad sense and it im-plicitly takes into account the underlying graph structure of the data.

In a second time, a new method for graph structure inference within the framework of Graph Signal Processing is investigated. In this problem, notions of both local and global smoothness, with respect to the underlying graph, are imposed to the vectors.

Finally, we propose to combine the graph learning task with the change-point detection problem. This time, a probabilistic framework is considered to model the vectors, assumed to be distributed from a specic Markov Random Field. In the considered modeling, the graph underlying the data is allowed to evolve in time and a change-point is actually detected whenever this graph changes signicantly.

Université Paris-Saclay Espace Technologique / Immeuble Discovery

Figure 1 . 1 :

 11 Figure 1.1: Simple illustration of a change-point detection task [157]. In this example, the signal is univariate, 4 change-points are detected, resulting in 5 regimes of di erent behavior.

Figure 1 . 2 :

 12 Figure 1.2: Example of signals recorded over a graph structure. (Left) Sensors measuring brain activity over time via electroencephalogram (EEG) are placed on a human head. Their positions over the head induce a graph structure that captures their proximity. A graph signal corresponds to a vector of size equal to the number of sensors (i.e nodes) containing the EEG values at a speci c timestamp. In this scenario, it is generally expected that nearby nodes in the graph will have similar values. (Right) Time-series of observations recorded at a single node, pointed by the dotted line.

Figure 1 . 3 :

 13 Figure 1.3: Summary of Sigfox's architecture.

Figure 1 . 4 :

 14 Figure 1.4: (Left) A subset of Base Stations (BSs) located near Toulouse, France. (Right) A naive graph representation that connects two nodes if the BSs are located less than 2km away from each other. The subplots indicates the number of messages received daily over one month by 3 nodes. Clearly, the two linked nodes (purple curves) are also closer to each other in terms of received network tra c, in both scale and correlation, compared to the isolated one (red curve).

Figure 1 . 5 :

 15 Figure 1.5: Illustration of the change-point detection task combined with graph inference.Each dimension of the time-series corresponds to a node in the corresponding graph[START_REF] Hallac | Network inference via the time-varying graphical lasso[END_REF].

Figure 1 . 6 :

 16 Figure 1.6: An illustrative example of anomalies in a two-dimensional data set.

Figure 1 . 7 :

 17 Figure 1.7: Exemples of directed, undirected and weighted graphs with four nodes.

  there is no notion of parent nor children and connected nodes are simply referred as neighbors. De nition 1.3. (Weighted graph.) A weighted graph G = (V, E) is a graph whose edge set E = {(u, v, w uv ), u, v ∈ V} associates to each edge (u, v) ∈ E a weight w uv ∈ R + . If the graph is undirected, we have ∀(u, v) ∈ E, w uv = w vu .

De nition 1 . 6 .

 16 (Combinatorial graph Laplacian.) The combinatorial graph Laplacian of a graph G with weight matrix W and degree matrix D is the matrix L = D -W .

Figure 1 . 8 :

 18 Figure 1.8: Some examples of important graph topologies. The last one (e) admits two connected components.

Figure 2 . 1 :

 21 Figure 2.1: A communication event at Sigfox and its graph representation. Only the Base Stations (BSs) are considered in the graph.

Figure 2 . 2 :

 22 Figure 2.2: Aggregated representation of a stream of communication events over two time intervals.

j

  ) the computation of such function using their algorithm. Assuming the knowledge of the parameters p (i) j , we can propose the following anomaly scoring function. De nition 2.4. (Anomaly scoring function): Under the previous model, we de ne the anomaly scoring function of the volume of events recorded at node j by:

  AUC = 0.99) Score Unilateral (AUC = 0.99) SVM -Node (AUC = 0.89) SVM -Weighted degree (AUC = 0.86) SVM -Difference w. neighbors (AUC = 0.97) AUC = 0.93) Score Unilateral (AUC = 0.95) SVM -Node (AUC = 0.80) SVM -Weighted degree (AUC = 0.69) SVM -Difference w. neighbors (AUC = 0.85) AUC = 0.81) Score Unilateral (AUC = 0.85) SVM -Node (AUC = 0.66) SVM -Weighted degree (AUC = 0.63) SVM -Difference w. neighbors (AUC = 0.72) (c) Hard experiment

Figure 2 . 3 :

 23 Figure 2.3: ROC curves and their respective AUC obtained over the simulated experiments. Each gure corresponds to a di erent level of complexity, from the simplest (a) to the hardest (c).

Figure 2 . 4 :

 24 Figure 2.4: Positions of the 34 Base Stations. In red, the failing BS, in green, the one taken as reference and in blue the others. Note that some BS have very close locations, making them appear as a single one over the gure.

  AUC = 0.83) Score Unilateral (AUC = 0.96) SVM -Node (AUC = 0.60) SVM -Weighted degree (AUC = 0.71) SVM -Difference w. neighbors (AUC = 0.89)

Figure 2 . 5 :

 25 Figure 2.5: ROC curves and their respective AUC obtained over the real-world experiment.

Figure 2 . 6 :

 26 Figure 2.6: Number of messages received by the corresponding BS, the con dence intervals for a xed threshold and the timestamps detected to be anomalous.

Assumption 3 . 1 . 1 √N 1 N

 3111 (Assumption on the graph G) -G is undirected, with no self-loops and has a single connected component. With Assumption 3.1, L is a symmetric positive semi-de nite matrix with eigenvalue decomposition L = XΛX T , where λ 1 = 0 and x 1 = [35]. Assumption 3.2. (Assumption on the signals Y ) -Graph signals Y de ned over the true underlying graph G are assumed s-smooth (Def. 1.8) and admit a k-sparse spectral representation (Def. 1.10), with unknown values for s and k.

Figure 3 . 1 :

 31 Figure 3.1: Two graph signals observed on the same graph of 200 nodes. (a) The rst signal admits smoothness at the level of adjacent nodes, and a 100-sparse spectral representation. (b)The second signal admits also smoothness, but in this case it extends to larger clusters of connected nodes. As a consequence, this graph signal enjoys a 3-sparse spectral representation.

Figure 3 .

 3 Figure 3.1 shows visually an example of two graph signals that illustrate the intuition behind our two core assumptions.

Proposition 3 . 1 . 1 √N 1 N

 3111 -Given X 0 ∈ R N ×N an orthogonal matrix with rst column being equal to , an equivalent formulation of optimization problem (3.1) is given by min

  Graph learning on ER synthetic graphs.

Figure 3 . 3 :

 33 Figure 3.3: Graph learning results on random synthetic graphs of 20 nodes: (a) for a RG graph, and (b) for an ER graph. Each of the two sub gures presents: (top row) the evolution of the F 1 -measure with respect to di erent threshold values and the dashed line indicates the chosen threshold value; (bottom row) shows as leftmost the ground truth adjacency matrix, followed by the respective learned adjacency matrices (thresholded) by the compared methods.

Figure 3 . 4 :

 34 Figure 3.4: Average and standard deviation of the computation time over 10 trials for IGL-3SR, FGL-3SR, ESA-GL, and GL-SigRep, as the number of nodes increases. GL-SigRep and ESA-GL failed to produce a result for graphs with more than 100 and 150 nodes, respectively. (a) The total computation times, and (b) the time needed for a single iteration of each algorithm. For IGL-3SR and FLG-3SR, a single iteration means the computation of the 3 steps one time.

  Average of the F 1 -measure. (b) Standard deviation of the F 1 -measure.

  Average of the F 1 -measure (focus).

  Standard deviation of the F 1 -measure (focus).

Figure 3 . 5 :

 35 Figure 3.5: Evolution of the average (a)(c) and standard deviation (b)(d) of the F 1 -measure over 10 runs of FGL-3SR on RG graphs with 20 nodes. At the top gure row β ∈ [0, 100], and at the bottom row β ∈ [20, 70].

Figure 3 . 6 :

 36 Figure 3.6: Convergence curves of the objective function as the number of iterations increases, using FGL-3SR with (a) α = 10 -5 , (b) α = 10 -1 , (c) α = 1.

Figure 3 . 7 :

 37 Figure 3.7: Learned graphs with increasing t values: (top row) α = 10 -4 , (bottom row) α = 10 -3 .

Figure 3 . 8 :

 38 Figure 3.8: (Top row) Learned graph with (a) IGL-3SR and (b) FGL-3SR. The node color corresponds to the average temperature in C • during all the period of observation. (Bottom row) Graph segmentation in two parts (red vs. green nodes) with spectral clustering using the Laplacian matrix learned by (c) IGL-3SR and (d) FGL-3SR.

Figure 3 . 9 :

 39 Figure 3.9: (a) The 2 nodes kept for the signal interpolation are shown in black. (b) The true signal at the target node (in red) shown on the left and its reconstruction using only the 2 selected nodes shown on the left (in black).

CHAPTER 3 .

 3 STRUCTURE INFERENCE FROM SMOOTH AND BANDLIMITED GRAPH SIGNALS Indicative Regions of Interest (ROIs) from Varoquaux et al. [164].

Figure 3 .

 3 Figure 3.10: (a) Indicative ROIs from the Multi-Subject Dictionary Learning atlas extracted in Varoquaux et al. [164] with sparse dictionary learning. Results: Graphs returned by FGL-3SR, separately for (b) an ADHD patient and (c) a healthy subject, where darker edges indicate larger weights of connection.

Figure 3 . 11 :

 311 Figure 3.11: Graph obtained with FGL-3SR over the Sigfox training dataset.

Proposition 3 . 2 (

 32 Closed-form solution for the 2,0 and 2,1 -norms) -The solutions of problem (3.5) when • S is set to • 2,0 or • 2,1 are given in the following.

Proposition 3 . 5 (

 35 Closed-form solution of problem (3.11)) -Consider the optimization problem(3.11)

3. LEARNING METHODOLOGY 91 Figure 4 . 1 :

 9141 Figure 4.1: Comparison of the learned parameter vectors when using either 2 or 1 -norm in the fused penalty. White squares indicates dimensions at which the two consecutive parameter vectors are di erent. Black squares where they are equal. The presence of at least one white square indicates a change-point.

4. 1

 1 Technical assumptions We denote by D = | D| (the set's cardinality) the total number of detected change-points, respectively for the real changes D = |D|, and by [D] the set of indices {1, . . . , D}. Let us now de ne two important quantities. The rst is the minimal time di erence between two change-points: ∆ min min j∈[D]

√ p-1λ 2 ξ min → 0 , 1 . ( 4 . 8 ) 4 . 1 .

 014841 and (iii) √ p log(n) ξ min √ nδn → 0. Then, if the correct number of change-points are estimated, we have D = D and:P( max j=1,...,D | Tj -T j | ≤ nδ n ) -→ n→∞ Proposition Let {x i } ni=1 be a sequence of observations drawn from the model presented in Sec. 2. Assume the conditions of Theorem 4.1 are respected. Then, if for a x D max < ∞, we have D ≤ D ≤ D max then:

degree = 2

 2 degree = 3 degree = 4

Figure 4 . 2 :

 42 Figure 4.2: Average value of the best F 1 -score obtained when the h-score is below a certain threshold. These thresholds are respectively (from top left to bottom right), 0.0, 0.01, 0.02, 0.03, i.e. at most 0, 1, 2, or 3 timestamps of o set error between an estimated and a real change-point. Each pairs of bars corresponds to di erent d-regular graphs, with d ∈ {2, 3, 4}. The error bars correspond to ± (std).

100 CHAPTER 4 .

 1004 DETECTING CHANGES IN THE GRAPH STRUCTURE OF A VARYING ISING MODEL

Figure 4 . 3 :

 43 Figure 4.3: (Top) The two graphs before and after the strongest estimated change-point: the third one that indeed corresponds to the end of the 114-th Congress. (Bottom) The cumulative functions of votes of the 18 seats over the two Congresses.

Figure 4 . 4 :

 44 Figure 4.4: Learned graphs at two di erent timestamps. Only the edges with positive weights are represented and the abnormal BS is in red. (Left) A graph learned before the BS failure, recorded on the 30th day. (Right) A graph learned after this day.

Theorem 4 . 1 .

 41 (Change-point consistency) Let {x i } n i=1 be a sequence of observations drawn from the piece-wise constant Ising model presented in Sec. 2. Suppose (A1-A3) hold, and assume that λ 1 λ 2 = O( log(n)/n). Let {δ n } n≥1 be a non-increasing sequence that converges to 0, and such that ∀n > 0, ∆ min ≥ nδ n , with nδ n → +∞. Assume further that(i) λ 1 nδnξ min → 0,(ii)√ p-1λ 2 ξ min → 0, and (iii) √ p log(n) ξ min √ nδn → 0. Then, if the correct number of change-points are estimated, we have D = D and:

n→∞ 0 thanks to Theorem 1 . 110 CHAPTER 4 .

 11104 Thus it su ces to show that: DETECTING CHANGES IN THE GRAPH STRUCTURE OF A VARYING ISING MODEL Dmax K=D+1 P({d( D D) ≥ nδ n } ∩ { D = K})

2 .

 2 Let's analyze the quantity P Σ -Σ F > with > 0:

Figure A. 1 :

 1 Figure A.1: True density, outliers, KDE, and MoM-KDE. (a) Estimates from a 1-D true density and outliers from a normal density centered in µ O = 10 with variance σ 2 O = 0.1. (b) Estimates from a 2-D true density and outliers from a normal density centered in µ O = (3, 3) with variance σ 2 O = 0.5I 2 .

Assumption 2 .

 2 (Density kernel) ∀u ∈ R d , K(u) ≥ 0, and K(u)du = 1.Assumption 3. (Spherically symmetric and non-increasing) There exists a non-increasing functionk : R + -→ R + such that K(u) = k( u ) for all u ∈ R d , where • is any norm of R d .Assumption 4. (Exponentially decaying tail) There exists positive constants ρ, C ρ , t 0 > 0 such that for all t > t 0 k(t) ≤ C ρ • exp(-t ρ ) .

  x)dx = 1, and f ∈ Σ(α, L) , (A.4) where Σ(α, L) is the Holder class of function on R d (De nition A.2). Grant assumptions 1 to 4 and let n > 1, h ∈ (0, 1) and S ≥ 1 such that nh d ≥ S and nh d ≥ |log(h)|. Then with probability at least 1exp(-S), we have

Figure A. 3 :

 3 Figure A.3: Anomaly detection with real datasets, measured with AUC over varying outlier proportion. A higher score means a better detection of the outliers. For Digits, we specify which classes are chosen to be inliers (I) and outliers (O).

  1. (L ∞ error-bound of the KDE without anomalies) Suppose that f belongs to the class of densities P(α, L) de ned asP(α, L) f | f ≥ 0, f(x)dx = 1, and f ∈ Σ(α, L) , where Σ(α, L) is the Hölder class of function on R d . Grant assumptions 1 to 4 and let n > 1, h ∈ (0, 1) and S ≥ 1 such that nh d ≥ S and nh d ≥ |log(h)|. Then with probability at least 1exp(-S), we have

  and C 1 is a constant that only depends on f ∞ , the dimension d, and the kernel properties.Proposition A.1. (L ∞ error-bound of the MoM-KDE under the O ∪ I) Suppose that f belongs to the class of densities P(α, L) and grant assumptions 1 to 4. Let S be the number of blocks, δ > 0 such that S > (2 + δ)|O|, and ∆ = (1/(2 + δ) -|O|/S). Then, for any h ∈ (0, 1), δ su ciently small, and n ≥ 1 such that nh d ≥ S log(2(2 + δ)/δ), and nh d ≥ S| log(h)|, we have with probability at least 1exp(-2∆ 2 S),

I

  )f (x) > ε =⇒ sup x S s=1 I fns (x)f (x) > ε ≤ S s=1 I sup x fns (x)f (x) > ε , fns (x)f (x) > ε ≥ S/2 )f (x) > ε ≥ S/2 . Let Z s = I sup x fns (x)f (x) > ε and let S = s ∈ {1, • • • , S} | B s ∩ O = ∅ i.e.the set of indices s such that the block B s does not contain any outliers. Since s∈S C I(•) is bounded by |O|, almost surely, the following holds.S s=1

AI

  and applying Hoe ding's inequality to the right-hand side of the previous equation givesP S s=1 I sup x fns (x)f (x) > ε ≥ S/2 ≤ e -2S∆ 2 ,which concludes the proof.Proposition A.2. (L 1 -consistency in probability) If n/S → ∞, h → 0, nh d → ∞,and S > 2|O|, then fMoM -We rst rewrite the MoM-KDE as fMoM (x) = S s=1 fns (x)I As (x) , where A s = x | fMoM (x) = fns (x) . Without loss of generality, we assume that s = R d , and S s=1 As (x) = 1 . fMoM (x)f (x) dx = S s=1 fns (x)I As (x)f (x) dx = S s=1 fns (x)f (x) I As (x) dx ≤ S s=1 fns (x)f (x) I As (x)dx = S s=1 As fns (x)f (x) dx = s∈S As fns (x)f (x) dx + s∈S C As fns (x)f (x) dx .

  )f (x) dx ≤ s∈S fns (x)f (x) dx P

≤Proposition A. 3 .

 3 s∈S C s ∈S fn s (x)f (x) dx + fn s (x)f (x) dx Since ∀s ∈ S we have | fn s (x)f (x)|dx P -→ n→∞ 0, we can conclude using similar arguments as those used for (A.10) that s∈S C As | fns (x)-f (x)|dx P Let x , x 0 ∈ R d and I n be an healthy data set. Grant assumptions 1 to 4 and denote

Figure B. 1 :

 1 Figure B.1: Illustration simple d'un problème de détection de rupture [157]. Dans cet exemple, le signal est univarié, 4 points de rupture sont détectés, résultant en 5 sousmodèles.

Figure B. 2 :

 2 Figure B.2: Exemple de signaux enregistrés sur une structure de graphe. À gauche, des capteurs mesurant l'activité cérébrale dans le temps, représenté par un électroencéphalogramme (EEG), sont placés sur une tête humaine. Leur position sur la tête induit une structure graphique qui encode leur proximité. Dans cet éxemple, un signal sur graphe correspond à un vecteur de taille égale au nombre de capteurs (c'est-à-dire de noeuds) contenant les valeurs d'EEG à un moment précis. Dans ce scénario, on peut s'attendre à ce que les noeuds proches dans le graphe aient des valeurs similaires. À droite, la série temporelle d'observations enregistrées à un seul noeud.

Figure B. 3 :

 3 Figure B.3: Résumé de l'architecture de Sigfox.

Figure B. 4 :

 4 Figure B.4: À gauche, un sous-ensemble de stations de base situées près de Toulouse, en France. À droite, une représentation naïve du réseau sous forme de graphe, qui relie deux noeuds si les stations de base sont situées à moins de 2km l'une de l'autre. Les di érentes courbes indiquent pour 3 des noeuds, le nombre de messages reçus quotidiennement sur un mois. Il est clair que les deux noeuds reliés (avec des courbes violettes) sont plus similaire, tant en termes d'échelle que de corrélation, que le noeud isolé (avec une courbe rouge).

Figure B. 5 :

 5 Figure B.5: Illustration de la tâche de détection de ruptures combinée à l'inférence de graphes. Chaque dimension de la série temporelle correspond à un noeud du graphe [73].

Figure B. 6 :

 6 Figure B.6: Un exemple d'anomalies pour un ensemble de données bidimensionnelles.

  (i) } t k+1 i=t k +1 ), (B.1)où t 0 = 0, t K+1 = n et c(•) est une fonction de coût qui évalue la qualité de la ségmentation.

De nition B. 1 .

 1 (Graphe dirigé.) Un graphe dirigé G = (V, E) est dé ni via un ensemble ni de noeuds V = {v 1 , . . . , v p } et un ensemble d'arêtes E ⊂ V × V, qui relient des paires de noeuds entre eux. Si (u, v) ∈ E, on dit que u est un parent de v et que v est un enfant de u. Dans ce qui suit, on considère que pour tout noeud u dans V, (u, u) / ∈ E. De plus, on peut se référer à un noeud v i ∈ V simplement par son indice i. De nition B.2. (Graph non dirigé.) Un graphe non dirigé G = (V, E) est un graphe dirigé dont l'ensemble des arêtes E est symétrique. Autrement dit, ∀(u, v) ∈ E, (v, u) ∈ E. Dans ce contexte il n'y a donc pas de notion de parent ou d'enfant et deux noeuds connectés sont simplement appelés voisins. De nition B.3. (Graphe pondéré.) Un graphe pondéré G = (V, E) est un graphe dont l'ensemble des arêtes E = {(u, v, w uv ), u, v ∈ V} associe à chaque arête (u, v) ∈ E un poids w uv ∈ R + . Si le graphe est en plus non orienté, on a ∀(u, v) ∈ E, w uv = w vu .

Figure B. 7 :

 7 Figure B.7: Exemples de graphes dirigés, non dirigés et pondérés avec quatre noeuds.

Figure B. 8 :

 8 Figure B.8: Quelques exemples de graphes importants. Le dernier (e) admet deux composantes connexes.

De nition B. 8 . 2 i

 82 (Régularité.) Soit G = (V, E) un graphe de taille p avec une matrice de poids W , L sa matrice laplacienne, et y ∈ R p un signal sur ce graphe. Nous disons que y a une régularité de niveau s par rapport au graphe G siy T Ly = 1 ,j∈[p]

  De nition B.10. (Parcimonie spectrale.) Soit k ∈ N + , on dit qu'un signal sur graphe y admet une représentation spectrale k-parcimonieuse (ou bien que y est k-bandlimité) par rapport à un graphe G, si pour h = X T y nous avonsh 0 ≤ k , (B.2)où h 0 correspond au nombre d'éléments non nuls de h.

  De nition B.11. (Indépendance conditionnelle.) Soit X, Y et Z trois variables aléatoires à valeur réelle. On désigne par F X|Z=z (x) (respectivement F Y |Z=z (y)) la fonction de répartition (cdf) de X (respectivement Y ) sachant Z = z. On dit alors que X et Y sont conditionnellement indépendantes sachant Z, dénoté X ⊥ ⊥ Y |Z si et seulement si ∀x, y, z nous avons :F X,Y |Z=z (x, y) = F X|Z=z (x)F Y |Z=z (y).Dans une certaine mesure, le fait que X et Y soient conditionnellement indépendants par rapport à Z nous indique que, étant donné Z, connaître X n'apporte aucune information sur Y et inversement. Remark B.3. Deux variables conditionnellement indépendantes peuvent être dépendantes et inversement.De nition B.12. (Champ aléatoire de Markov.) Soit G = (V, E) un graphe non orienté et X = (X i ) i∈V un vecteur aléatoire dont les entrées sont indexées par les noeuds V. Nous disons que X est tiré d'un MRF associé à G si les propriétés suivantes sont respectées :(a) X u ⊥ ⊥ X v | X V {u,v} , pour toute arête (u, v) / ∈ E. (b) X u ⊥ ⊥ X V N (u) | X N (u) , ∀u ∈ V où N (u) = {v ∈ V : (u, v) ∈ E} est le voisinage de u. (c) X A ⊥ ⊥ X B | X S ,pour tout sous-ensembles disjoint A, B, S ⊂ V tels que S sépare A et B i.e. tout chemin de A à B (et inversement) passe par S. Remark B.4. On peut montrer [100] que (c) ⇒ (b) ⇒ (a). Pour certaines distributions de probabilité, la réciproque est également vraie, c'est notamment le cas des variables admettant une fonction de densité positive.

  , Le Bars, Batiste , Humbert, Pierre , Oudre, Laurent and Kalogeratos, Argyris Accepted in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

Authors with equal contribution to this work Learning Laplacian Matrix from Graph Signals with Sparse Spectral Representation [84], Humbert, Pierre , Le Bars, Batiste , Oudre, Laurent, Kalogeratos, Argyris, and Vayatis Nicolas Submitted to The Journal of Machine Learning Research (JMLR).

Authors with equal contribution to this work.

  1 while t ≤ t max do while not convergence do H-step: Compute the closed-form solution of Proposition (3.2)

  This is a two-step algorithm where the signals are supposed to admit a sparse representation with respect to the learned graph. The di erence to our method is two-fold. First, ESA-GL does not include the smoothness assumption while learning the Fourier basis X. This brings a di erent two-step optimization program. Second, the complexity of the ESA-GL algorithm (at least O(N 8 )) is much higher than ours (O(N 5 ) for FGL-3SR -see Section 4.3), and hence is prohibitive for large graphs. The rst step consists in tting an orthonormal basis such that the observed graph signals Y admit a sparse representation with respect to this basis. They consider the problem:

.19)

Remark that since no constraints are imposed on the spectral representation of the signals, the Laplacian matrix is directly learned. The optimization procedure to solve (3.19) consists in an alternating minimization over L and Ỹ . With respect to Ỹ the problem has a closed-form solution whereas for L, the authors propose to use a Quadratic Program solver involving

1

2 N (N -1) parameters and 1 2 N (N -1) + N + 1 constraints.

7. EXPERIMENTAL EVALUATION 65

ESA-GL

[START_REF] Sardellitti | Graph topology inference based on sparsifying transform learning[END_REF]

.

Table 3 .

 3 .1, easier to tune.

				RG graph model			ER graph model
	N	Metrics	IGL-3SR	FGL-3SR	ESA-GL	GL-SigRep	IGL-3SR	FGL-3SR	ESA-GL	GL-SigRep
		Precision	0.973 (±0.042) 0.952 (±0.042)	0.899 (±0.054) 0.929 (±0.068) 0.952 (±0.045) 0.819 (±0.080)	0.931 (±0.045)	0.704 (±0.125)
		Recall	0.974 (±0.018) 0.985 (±0.023) 0.968 (±0.052) 0.967 (±0.028) 0.927 (±0.046)	0.824 (±0.105) 0.951 (±0.041) 0.899 (±0.075)
	20	F1-measure 0.974 (±0.028) 0.968 (±0.027)	0.929 (±0.032) 0.947 (±0.040) 0.938 (±0.028)	0.816 (±0.068) 0.941 (±0.038) 0.779 (±0.071)
		ρ(L, L)	0.938 (±0.052) 0.903 (±0.029)	0.925 (±0.050) 0.786 (±0.037) 0.917 (±0.035) 0.730 (±0.063)	0.897 (±0.045)	0.199 (±0.074)
		Time	< 1min	< 10s	< 5s	< 5s	< 1min	< 10s	< 5s	< 5s
		Precision	0.901 (±0.022) 0.817 (±0.041)	0.845 (±0.088) 0.791 (±0.055) 0.820 (±0.027)	0.791 (±0.047) 0.854 (±0.038) 0.476 (±0.037)
		Recall	0.902 (±0.018)	0.807 (±0.036) 0.910 (±0.040) 0.720 (±0.059) 0.812 (±0.042)	0.740 (±0.049)	0.830 (±0.051) 0.856 (±0.023)
	50	F1-measure 0.901 (±0.014) 0.812 (±0.017)	0.868 (±0.036) 0.750 (±0.001) 0.815 (±0.021)	0.761 (±0.031) 0.841 (±0.021) 0.610 (±0.026)
		ρ(L, L)	0.863 (±0.020) 0.743 (±0.031)	0.832 (±0.033) 0.549 (±0.022) 0.783 (±0.026)	0.728 (±0.020) 0.816 (±0.058) 0.058 (±0.002)
		Time	< 17mins	< 40s	< 60s	< 40s	< 17mins	< 40s	< 60s	< 40s
		Precision	0.713 (±0.012) 0.711 (±0.029)	0.667 (±0.022)	-	0.677 (±0.044) 0.640 (±0.033)	0.654 (±0.038)	-
		Recall	0.751 (±0.067) 0.584 (±0.011)	0.743 (±0.017)	-	0.580 (±0.021)	0.543 (±0.027) 0.637 (±0.023)	-
	100	F1-measure 0.732 (±0.034) 0.641 (±0.010)	0.703 (±0.012)	-	0.623 (±0.009) 0.586 (±0.016)	0.589 (±0.019)	-
		ρ(L, L)	0.612 (±0.045) 0.483 (±0.015)	0.596 (±0.033)	-	0.551 (±0.016) 0.512 (±0.0223) 0.644 (±0.023)	-
		Time	< 50mins	< 2mins	< 4mins	-	< 50mins	< 2mins	< 4mins	-

1: Comparison of the four methods on ve quality metrics (avg ± std) for graphs of N = {20, 50, 100} nodes, and for xed number of n = 1000 graph signals.

Table 4 .

 4

1: Results for the model with the lowest AIC, and that with the highest AUC. The average ± (std) of the metrics is reported.

Table A

 A

		.1: Computational complexity
	Method	Learning	Evaluation Iterative method

Contents 1

 1 Contexte de la thèse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 1.1 Contexte general . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 1.2 Contexte industriel . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Au cours des dernières décennies, la quantité croissante de données multivariées disponibles dans tous les secteurs, de la médecine à l'industrie en passant par les réseaux sociaux, a entraîné un besoin important d'analyse et de modélisation de ces données dans le but d'accomplir diverses tâches. Parmi celles-ci, les tâches de détection d'anomalies et de détection de ruptures (appelées détection d'événements lorsque l'on se réfère aux deux) au sein de cette quantité massive de données sont d'une importance majeure. En quelques mots, la détection d'anomalies cherche à trouver, dans un ensemble de données, une petite quantité d'entre elles qui s'écarte du comportement normal de la grande majorité. Cette tâche peut être appliquée à n'importe quel ensemble de données et, dans une application réelle, elle peut par exemple caractériser la découverte d'un élément dysfonctionnel dans

2 Objectifs et motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 3 Préliminaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 3.1 Détection d'anomalies . . . . . . . . . . . . . . . . . . . . . . . . . 144 3.2 Détection de ruptures . . . . . . . . . . . . . . . . . . . . . . . . . 146 3.3 Théorie des graphes et modèles pour les vecteurs sur graphes . . . 148 4 Organisation du manuscrit . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 5 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 1 Contexte de la thèse 1.1 Contexte general la chaîne de production d'une entreprise. D'autre part, la détection de rupture est liée au vaste domaine de l'analyse de séries temporelles. Ici, on cherche à déterminer des moments où le modèle sous-jacent aux données a changé (Figure B.1). En sciences de la vie ou en biologie, cette tâche peut par exemple correspondre à la détection d'un changement d'état (e.g. un individu se réveillant, le début de la puberté etc.)

  3. PRÉLIMINAIRES 145 cadre dit de détection de nouveauté, également appelé classi cation à une classe ou détection d'anomalie semi-supervisée. Dans ce cadre, seules des données normales sont disponibles pour la phase d'apprentissage. C'est le cas dans les applications où les comportements normaux sont connus mais où, par exemple, les intrusions ou les attaques sont inconnues et doivent être détecté. Ce scénario est celui considéré au chapitre 2. En n, la détection d'anomalie non supervisé, qui fait référence à celle présentée dans le paragraphe précédent : aucune étiquette n'est disponible et l'ensemble des données d'apprentissage contient des données normales et anormales. La plupart des algorithmes de détection d'anomalies ne se contentent pas d'associé le vecteur d'entrée à une valeur binaire, indiquant si celui-ci est normal ou non. Ils renvoient plutôt une fonction à valeur réelle, appelée fonction de score, qui produit, pour un vecteur d'entrée donné, un score d'anomalie à valeur réelle. La force de l'utilisation d'une telle fonction de score est qu'elle permet de classer les échantillons du moins anormal au plus anormal. Cette fonction est très puissante lorsque l'on a beaucoup de données à analyser et que l'on veut classer certaines anomalies par ordre de priorité. De plus, si l'on veut une sortie binaire, il su t de xer un seuil au-dessus duquel le score sera considéré comme anormal.

  La thèse est organisée comme suit. Chaque chapitre peut être lu indépendamment. • Chapitre 2: Détection d'anomalies dans des réseaux de communication : applications à Sigfox. Ce chapitre est essentiellement consacré à la résolution du premier objectif décrit dans la section 2. La tâche de détection d'anomalies est prise sous l'angle de l'analyse de l'activité d'un réseau de communication. En d'autres termes, les anomalies sont détectées sur la base, par exemple, d'un nombre anormal d'interactions ou de quantité d'informations échangées entre deux noeuds. Ce cadre très général nous permet de considérer une large classe de réseaux de communication, qui inclue Sigfox. Dans ce chapitre, nous commençons par un bref aperçu de la littérature sur la détection d'anomalies dans les réseaux, en mettant l'accent sur les méthodes faisant appel à des représentation graphique du reseau et de son activité. Nous présentons également un algorithme simple de détection de nouveauté qui vise à détecter un niveau anormal d'activité au niveau d'un noeud. Cet algorithme repose sur l'intuition que le niveau d'activité d'un noeud peut être déterminé ou prédit en examinant le niveau d'activité enregistré au niveau des noeuds voisins. En utilisant un ensemble de données normales et des méthodes d'apprentissage supervisé conventionnelles, on cherche alors à apprendre cette relation entre l'activité des noeuds. Ainsi, une anomalie est détectée lorsqu'un niveau d'activité prédit est loin du niveau d'activité réellement observé.

	4. ORGANISATION DU MANUSCRIT	153
	4 Organisation du manuscrit

  Détection d'événements et inférence de structure pour des vecteurs sur graphes Mots clés: Graphes, Réseaux, Détection d'anomalies et de ruptures, Champs aléatoire de MarkovRésumé: Cette thèse aborde diérents problèmes autour de l'analyse et la modélisation de signaux sur graphes, autrement dit des données vectorielles observées sur des graphes. Nous nous intéressons en particulier à deux tâches spécique. La première est le problème de détection d'événements, c'est-à-dire la détection d'anomalies ou de ruptures, dans un ensemble de vecteurs sur graphes. La seconde tâche consiste en l'inférence de la structure de graphe sousjacente aux vecteurs contenus dans un ensemble de données.Dans un premier temps notre travail est orienté vers l'application. Nous proposons une méthode pour détecter des pannes ou des défaillances d'antenne dans un réseau de télécommunication. La méthodologie proposée est conçue pour être ecace pour des réseaux de commu-nication au sens large et tient implicitement compte de la structure sous-jacente des données.Dans un deuxième temps, une nouvelle méthode d'inférence de graphes dans le cadre du Graph Signal Processing est étudiée. Dans ce problème, des notions de régularité local et global, par rapport au graphe sous-jacent, sont imposées aux vecteurs.

	Titre:

Authors with equal contribution to the work.

https://github.com/tsakim/poibin

7 -0 2 -0 1 2 0 1 7 -0 2 -1 5

(a) (b)

K(K -1) parameters that, as we show empirically in the next section, can be di cult to compute for graphs with large number of nodes. For more details on the optimization program and the additional matrix C K , the readers shall refer to the aforementioned paper.

https://github.com/pierreHmbt/GL-3SR

https://github.com/BatisteLB/TVI-FL

https://github.com/lminvielle/mom-kde. For the sake of comparison, we also implemented RKDE and SPKDE.

http://www.raetschlab.org/Members/raetsch/benchmark/

Route de l'Orme aux Merisiers RD 128 / 91190 Saint-Aubin, France

Remerciements

Experimental evaluation

The two proposed algorithms, IGL-3SR and FGL-3SR, are now evaluated and compared with the two state-of-the-art methods presented earlier, GL-SigRep and ESA-GL. The results of our empirical evaluation are organized in three subsections: Section 7.2 and 7.3 use synthetic data for rst comparing the di erent methods and then study the in uence of the hyperparameters; Section 7. 4 displays several examples on real-world data.

All experiments were conducted on a single personal computer: a personal laptop with with 4-core 2.5GHz Intel CPUs and Linux/Ubuntu OS. For the Λ-step of both algorithms, we use the Python's CVXPY package [START_REF] Diamond | Cvxpy: A python-embedded modeling language for convex optimization[END_REF]. For the X-step of IGL-3SR, we use the conjugate gradient descent solver combined with an adaptive line search, both provided by Pymanopt [START_REF] Townsend | Pymanopt: A python toolbox for optimization on manifolds using automatic di erentiation[END_REF], a Python toolbox for optimization on manifolds. Note that this package only requires the gradients given in Proposition 3.3. The source code of our implementations is available online 1 .

CHAPTER 3. STRUCTURE INFERENCE FROM SMOOTH AND BANDLIMITED GRAPH SIGNALS

in the underlying structure of binary graph vectors.

Technical proofs

This section provides the technical proofs of the di erent propositions exposed in the chapter.

Recall that lower case variables refer to vectors/scalars while upper case variables denote matrices. The table below provides the main notations used in the technical discussion that follows.

x T , M T Transpose of vector x, matrix M . tr(M )

Trace of matrix M . diag(x) Diagonal matrix containing the vector x. M k,l (k, l)-th element of the matrix

Submatrix containing the elements of M from the k-th row to the last row, and from the l-th column to the last column. M 0 M is a positive semi-de nite matrix.

M †

The Moore-Penrose pseudoinverse of M . e k

Vector containing zeros except a 1 at position k.

I n

Identity matrix of size n.

n

Vector of size n containing only zeros.

n

Vector of size n containing only ones.

A (•)

The indicator function over the set A. x 0

The number of non-zero elements of a vector x.

• F

The Frobenius norm.

The 2,1 -norm, with M 2,1 = i=1 M i,: 2 . ∇f Gradient of the function f .

•, •

Inner product function. Orth(N ) The set of all orthogonal matrices of size N × N .

O( • )

Order of magnitude (e.g. of computational complexity). τ Number of iterations needed for an optimization procedure. Lemma 3.1 -Given X, X 0 ∈ R N ×N two orthogonal matrices with rst column equals to

)), we have the following equality:

, with [X T 0 X] 2:,2: denoting the submatrix of X T 0 X containing everything but the rst row and column of itself. Furthermore, remark that [X T 0 X] 2:,2: is in Orth(N -1).

CHAPTER 4. DETECTING CHANGES IN THE GRAPH STRUCTURE OF A VARYING ISING MODEL

Proof. We extend the proof given in [START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF][START_REF] Kolar | Estimating networks with jumps[END_REF] to the particular case of the Ising model. While the major steps are essentially the same, the Lemmas needed have been adapted to our case. We give here the proof's main steps. Thanks to the union bound, the probability of the complementary in Eq. (4.8) can be upper bounded by:

To prove Eq. (4.8), it is now su cient to show ∀j = 1, . . . , D that P(| Tj -

2 } and its complementary C c n . The rest of the proof is divided in two parts: bounding the good scenario, i.e. show that P({| Tj -T j | > nδ n }∩C n ) → 0, and doing the same for the bad scenario, i.e P({|

To bound the good scenario, the proof applies Lemma 4.1 to bound the considered probability by three others probabilities. These latter are then asymptotically bounded by 0, thanks to a combination of Assumptions (A1-A3), assumptions of the theorem and concentration inequalities related to the considered time-varying Ising model (given by the Lemmas of the Appendix).

To bound the bad case scenario, the three following complementary events are de ned:

Thus, it su ces to prove that P({| Tj -

To prove this, similar arguments to those used for the good case are employed.

Note that with δ n = log(n) γ /n, for any γ > 1 and ξ min = Ω( log(n)/ log(n) γ ), the conditions of the theorem are met. With this parameterization, we obtain a convergence rate of order O(log(n) γ /n) for the estimation of the change-points. More precisely, for any δ > 0 and su ciently large n, we have with probability at least 1δ that

In conclusion, we obtain the same rate of convergence to that of the single change-point detection method given in [START_REF] Roy | Change point estimation in high dimensional markov random-eld models[END_REF]. It is almost optimal up to a logarithmic factor. The main drawback of the previous theorem is that it assumes that the number of change-points have been correctly estimated. In practice this is complicated to verify, while proving that the right number of change-points are consistently estimated is also di cult to get for this type of methods [START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF]. Nevertheless, in practice we may have an idea about an upper bound on the true number of change-points.

The next proposition provides a consistency result when the number of change-points is overestimated. Let us rst introduce the metric d(A B) de ned as:

Supplementary Lemmas

Below, the di erent lemmas necessary to prove the main results are given.

Lemma 4.2. Let {x (i) } n i=1 be a set of i.i.d observation sampled from an Ising model with parameter Θ ∈ R p×p and assume that assumption (A1) is satis ed. Then, ∀r, l ∈ [n] such

Abstract

The work presented here is signi cantly di erent from that previously studied. For the sake of consistency with the rest of the thesis, it is therefore placed here rather than in the body of the document. In this additional chapter, we introduce a robust nonparametric density estimator combining the popular Kernel Density Estimation method and the Median-of-Means principle (MoM-KDE). This estimator is shown to achieve robustness to any kind of anomalous data, even in the case of adversarial contamination. In particular, while previous works only prove consistency results under known contamination model, this work provides nite-sample high-probability error-bounds without a priori knowledge on the outliers. Finally, when compared with other robust kernel estimators, we show that MoM-KDE achieves competitive results while having signi cant lower computational complexity.

Associated publication:

Robust Kernel Density Estimation with Median-of-Means principle, Humbert, Pierre , Le Bars, Batiste , Minvielle, Ludovic , and Vayatis, Nicolas PRINCIPLE

The O ∪ I framework is related to the well-known Huber's ε-contamination model [START_REF] Huber | Robust estimation of a location parameter[END_REF] where it is assumed that data are i.i.d. with distribution g = εf I + (1ε)f O , and ε ∈ [0, 1); the distribution f I being related to the inliers and f O to the outliers. However, there are several important di erences. First, in the O ∪ I the proportion of outliers is xed and equals to |O|/n, whereas it is random in the Huber's ε-contamination model [START_REF] Lerasle | Lecture notes: Selected topics on robust statistical learning theory[END_REF]. Second, the O ∪ I is less restrictive. Indeed, contrary to Huber's model which considers that inliers and outliers are respectively i.i.d from the same distributions, O ∪ I does not make a single assumption on the outliers.

MoM-KDE

We now present our main contribution, a robust kernel density estimator based on the MoM. This estimator is essentially motivated by the fact that the classical kernel density estimation at one point corresponds to an empirical average (see Equation (A.1)). Therefore, the MoM principle appears to be an intuitive solution to build a robust version of the KDE. A formal de nition of MoM-KDE is given below.

De nition

where fns (x 0 ) is the value of the standard kernel density estimator at x 0 obtained via the samples of the s-th block B s . Note that fMoM (•) is not necessarily a density as its integral may not be equal to 1. When needed, we thus normalize it by its integral the same way it is proposed by Devroye and Lugosi [START_REF] Devroye | Combinatorial methods in density estimation[END_REF].

Broadly speaking, MoM estimators appear to be a good tradeo between the unbiased but non robust empirical mean and the robust but biased median [START_REF] Lecué | Robust classi cation via mom minimization[END_REF]. A visual example of the robustness of MoM-KDE is displayed in Figure A.1. We now give a simple example highlighting the robustness of MoM-KDE.

Example A.1. (MoM-KDE v.s. Uniform KDE) Let the inliers be i.i.d. samples from a uniform distribution on the interval [-1, 1] and the outliers be i.i.d. samples from another uniform distribution on [-3, 3]. Let the kernel function be the uniform kernel, x 0 = 2 and h ∈ (0, 1). Then if S > 2|O|, we obtain

This result shows that the MoM-KDE makes (almost surely) no error at the point x 0 . On the contrary, the KDE here has a non-negligible probability to make an error.

Time complexity

The complexity of MoM-KDE to evaluate one point is the same as the standard KDE, O(n); O(S • n S ) for the block-wise evaluation and O(n) to compute the median with the medianof-medians algorithm [START_REF] Blum | Time bounds for selection[END_REF]. Since RKDE and SPKDE are KDEs with modi ed weights, they also perform the evaluation step in O(n) time. However, these weights need to be learnt, PRINCIPLE Metrics. The performance of the MoM-KDE is measured through three metrics, two are used to measure the similarity between the estimated and the true density, and one describes performances of an anomaly detector based on the estimated density. The rst one is the Kullback-Leibler divergence [START_REF] Kullback | On information and su ciency[END_REF] which is the most used in robust KDE [START_REF] Kim | Robust kernel density estimation[END_REF][START_REF] Kim | On the robustness of kernel density m-estimators[END_REF][START_REF] Kim | Robust kernel density estimation[END_REF][START_REF] Vandermeulen | Robust kernel density estimation by scaling and projection in hilbert space[END_REF]. Used to measure the similarity between distributions, it is de ned as

As the Kullback-Leibler divergence is non-symmetric and may have in nite values when distributions do not share the same support, we also consider the Jensen-Shannon divergence [START_REF] Endres | A new metric for probability distributions[END_REF][START_REF] Liese | On divergences and informations in statistics and information theory[END_REF]. It is a symmetrized version of D KL , with positive values, bounded by 1 (when the logarithm is used in base 2), and has found applications in many elds, such as deep learning [START_REF] Goodfellow | Generative adversarial nets[END_REF] or transfer learning [START_REF] Segev | Learn on source, re ne on target: A model transfer learning framework with random forests[END_REF]. It is de ned as

Motivated by real-world application, the third metric is not related to the true density, which is usually not available in practical cases. Instead, we quantify the capacity of the learnt density to detect anomalies using the well-known Area Under the ROC Curve criterion (AUC). An input point x 0 is considered abnormal whenever f (x 0 ) is below a given threshold.

Hyperparameters. All estimators are built using the Gaussian kernel. The number of blocks in MoM-KDE is selected on a regular grid of 20 values between 1 and 2|O| + 1 in order to obtain the lowest D JS . The bandwidth h is chosen for KDE via the pseudolikelihood k-cross-validation method [START_REF] Berwin | Bandwidth selection in kernel density estimation: A review[END_REF], and used for all estimators. The construction of RKDE follows exactly the indications of its authors [START_REF] Kim | Robust kernel density estimation[END_REF] and ρ(•) is taken to be the Hampel function as they empirically showed that it is the most robust. For SPKDE, the true ratio of anomalies is given as an input parameter.

Results on synthetic data.

To evaluate the e ciency of the MoM-KDE against KDE and its robust competitors, we set up several outlier situations. In all theses situations, we draw N = 1000 inliers from an equally distributed mixture of two normal distribution N (µ 1 , σ 1 ) and N (µ 2 , σ 2 ) with µ 1 = 0, µ 2 = 6, and σ 1 = σ 2 = 0.5. The outliers however are sampled through various schemes: (d) Adversarial Thin Gaussian. A low variance normal distribution N (0, 0.01) located on one of the inliers' Gaussian mode. This scenario can be seen as adversarial as an ill-intentioned agent may hide wrong points in region of high density. It is the most challenging setting for standard robust estimators as they are in general robust to outliers located outside the support of the density we wish to estimate.

For all situations, we consider several ratios of contamination and set the number of outliers |O| in order to obtain a ratio |O|/n ranging from 0.05 to 0.5 with 0.05-wide steps. Finally, to evaluate the pertinence of our results, for each set of parameters, data are generated 10 times. We display in Figure A.2 the results over synthetic data using the D JS score. The average scores and standard deviations over the 10 experiments are represented for each outlier scheme and ratio. Overall, the results show the good performance of MoM-KDE in all the considered situations. Furthermore, they highlight the dependency of the two competitors to the type of outliers. Indeed, as SPKDE is designed to handle uniformly distributed outliers, the algorithm struggles when confronted with di erently distributed outliers (see Figure A.2 b, c, d). RKDE performs generally better, but fails against adversarial contamination, which may be explained by its tendency to down-weight points located in low-density PRINCIPLE regions, which for this particular case correspond to the inliers. Results over D KL and AUC showing very similar results, they are not reported here.

Results on real data.

Experiments are also conducted over six classi cation datasets: Banana, German, Titanic, Breast-cancer, Iris, and Digits. They contain respectively n = 5300, 1000, 2201, 569, 150 and 1797 data points having d = 2, 20, 3, 30, 4 and 64 input dimensions. They are all publicly available either from open repositories 2 (for the rst three) or directly from Scikitlearn package (for the last three) [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]. We follow the approach of Kim and Scott [START_REF] Kim | Robust kernel density estimation[END_REF] that consists in setting the class labeled 0 as outliers and the rest as inliers. To arti cially control the outlier proportion, we randomly downsample the abnormal class to reach a ratio |O|/n ranging from 0.05 to 0.5 with 0.05-wide steps. When a dataset does not contain enough outliers to reach a given ratio, we similarly downsample the inliers. For each dataset and each ratio, the experiments are performed 50 times, the random downsampling resulting in di erent learning datasets. The empirical performance is evaluated through the capacity of each estimator to detect anomalies, which we measure with the AUC.

Results are displayed in Figure A.3. With the Digits dataset, we also explore additional scenarios with changing inlier and outlier classes (speci ed in gure titles). Overall, results are in line with performances observed over synthetic experiments, achieving good results in comparison to its competitors. Note that even in the highest dimensional scenarios, i.e.