Laure Tougne

Full Professor, Marc Van Droogenbroeck

Lucia Maddalena

Jenny Benois-Pineau

François Bremond

Antonino Staiano

Associate Professor, Laura Toni

Thierry Bouwmans

Fragkiskos D Malliaros

Dorina Thanou

Naoufel Werghi

Ananda S Chowdhury

Maryam Sultana

Vincenzo Scarrica

Anindya Mondal

Wieke Prummel

Marwa Chendeb

El Rai

Shashant R It

Belmar Garcia-Garcia

Ségolène Martin

Kavya Gupta

Claire Rossignol

Thomas Guilmeau

Sagar Verma

Karelia Peña-Peña

Jhon Carlos Mendoza

Alejandro Castro

Christian Escobar

Carlos Restrepo

Angela Cuadros

Mariano Burich

Claudio César Claros

Gloria Zuluaga

Heriberto Giraldo

Diana Giraldo

Leonor Suarez

Omaira Zuluaga

Nidia Botello

Jhony H Giraldo

Belmar Arif Mahmood

Sajid Javed

Soon Ki Jung

Vincenzo Giraldo

Antonino Scarrica

Francesco Staiano

Thierry Camastra

In this thesis, entitled "Graph-based Algorithms in Computer Vision, Machine Learning, and Signal Processing", we propose novel approaches in video and image processing, graph neural networks, and recovery of time-varying graph signals. As a result, this document is divided into three main parts: 1) computer vision, 2) machine learning, and 3) signal processing. Our main motivation is to use the geometrical information that we can capture from the data to avoid data hungry methods, i.e., learning with minimal supervision. All our contributions rely heavily on the developments of Graph Signal Processing (GSP) and spectral graph theory. In particular, the sampling and reconstruction theory of graph signals play a central role in this thesis. The main contributions of this thesis are as follows:

First and foremost, I would like to thank Thierry Bouwmans for giving me the opportunity to pursue a Ph.D

1. We propose Graph Moving Object Segmentation (GraphMOS) [START_REF] Giraldo | Graph moving object segmentation[END_REF][2][3][4] in Chapter 3. GraphMOS is composed of segmentation, background initialization, graph construction, unseen sampling, and a semi-supervised learning method inspired by the theory of recovery of graph signals for Moving Object Segmentation (MOS). GraphMOS has the advantage of requiring less labeled data than deep learning methods while having competitive results on both static and moving camera videos. GraphMOS is evaluated on six publicly available datasets outperforming several state-of-the-art methods in challenging conditions at the time of publication.

2. We pose MOS as a node classification problem using Graph Convolutional Networks (GCNs) [START_REF] Giraldo | Graph CNN for moving object detection in complex environments from unseen videos[END_REF] in Chapter 4. Our algorithm, dubbed as GraphMOS-Net, presents several advantages regarding GraphMOS while keeping its good properties. In particular, GraphMOS-Net requires little labeled information to periii form well as GraphMOS [START_REF] Giraldo | Graph moving object segmentation[END_REF]. However, we train a neural network model in [START_REF] Giraldo | Graph CNN for moving object detection in complex environments from unseen videos[END_REF], which makes testing faster than in GraphMOS. Similarly, adding new videos does not require solving again the optimization problem as in [START_REF] Giraldo | Graph moving object segmentation[END_REF].

3. We propose HyperGraph Convolutional Networks for Weakly-supervised Semantic Segmentation (HyperGCN-WSS) [6] in Chapter 5. HyperGCN-WSS follows the same line of research as in the previous algorithms [START_REF] Giraldo | Graph moving object segmentation[END_REF][START_REF] Giraldo | Graph CNN for moving object detection in complex environments from unseen videos[END_REF] about learning with minimal supervision. However, HyperGCN-WSS addresses the problem of Weakly-supervised Semantic Segmentation (WSS), using a similar structure as in GraphMOS and GraphMOS-Net. HyperGCN-WSS constructs hypergraphs from the images in the dataset. Then, we train a specialized Hy-perGraph Convolutional Network (HyperGCN) architecture using some weak signals. HyperGCN-WSS is evaluated on the PASCAL VOC 2012 dataset for semantic segmentation. HyperGCN-WSS shows competitive performance against some state-of-the-art methods at the time of publication. 4. We propose a new scalable convolutional operator for Graph Neural Networks (GNNs) based on the Sobolev norm of graph signals [START_REF] Giraldo | Sparse Sobolev graph neural networks[END_REF] in Chapter 6. To this end, we use Hadamard products between matrices to keep the same sparsity level in graph representations, and therefore we dub our proposed algorithm as Sparse Sobolev GCN (S-SobGNN). Our architecture computes a cascade of filters on each layer with increasing Hadamard powers to get a more diverse set of functions, and then an attention layer selects the best operations. We provide several mathematical notions of our filtering operation based on the spectral graph theory and the Schur product theorem. S-SobGNN is evaluated in several tasks including, semi-supervised learning, and graph node classification. S-SobGNN shows competitive performance in all applications as compared to several state-of-the-art methods at the time of publication.

5. We introduce a topological relationship between over-smoothing and over-squashing in Chapter 7. We use concepts of spectral graph theory and differential geometry to show that both problems are intrinsically related to the spectral gap of the Laplacian representation of the graph. As a result, there is a trade-off between iv these two problems from a topological perspective, i.e., we cannot simultaneously alleviate both over-smoothing and over-squashing. Similarly, we use one bound of the Ollivier's Ricci curvature to propose a new rewiring algorithm for GNNs. Our proposed algorithm is less computationally expensive than previous curvature-based rewiring methods in the literature, while keeping fundamental theoretical properties. We also perform a thorough comparison of our algorithm with previous methods to alleviate over-smoothing and over-squashing, seeking to gain a better understanding of both problems and their practical solutions. 6. We introduce a novel algorithm based on the extension of a Sobolev smoothness function for the reconstruction of time-varying graph signals from discrete samples [START_REF] Giraldo | On the minimization of Sobolev norms of time-varying graph signals: Estimation of new Coronavirus disease 2019 cases[END_REF][START_REF] Giraldo | Reconstruction of time-varying graph signals via Sobolev smoothness[END_REF] in Chapter 8. We explore some theoretical aspects of the convergence rate of our Time-varying Graph signal Reconstruction via Sobolev Smoothness (GraphTRSS) algorithm by studying the condition number of the Hessian associated with our optimization problem. Our algorithm has the advantage of converging faster than other methods that are based on Laplacian operators without requiring expensive eigenvalue decomposition or matrix inversions. GraphTRSS is evaluated on several datasets including two datasets and it has outperformed many existing state-of-the-art methods for time-varying graph signal reconstruction at the time of publication. Graph-TRSS has also shown excellent performance on two environmental datasets for the recovery of particulate matter and sea surface temperature signals. We pose the problem of Moving Object Segmentation (MOS) as a graph learning problem. Therefore, we solve MOS using concepts of Graph Signal Processing (GSP) and Graph Neural Networks (GNNs).

We pose the problem of Weakly-supervised Semantic Segmentation (WSS) as a hypergraph learning problem. Consequently, we solve WSS using concepts of hypergraph neural networks.

We propose a new graph convolutional operator for GNNs based on the Sobolev norm in GSP. Similarly, we introduce a new sparse graph norm to maintain scalability.

We introduce a topological relationship between the over-smoothing and over-squashing problems.

In addition, we present a new rewiring algorithm to alleviate both issues.

We with its corresponding eigenvector. The lowest frequency is 𝜆 1 = 0, corresponding to a constant graph signal, i.e., the Laplacian quadratic form of eigenvector u 1 is given such that u T 1 Lu 1 = 𝜆 1 = 0. . . . . . .

3-1 Comparisons of the visual results of the proposed Graph Moving Ob-

ject Segmentation (GraphMOS) algorithm with existing state-of-theart methods on five MOS challenging video sequences taken from CD-Net2014 [10] and UCSD [START_REF] Mahadevan | Spatiotemporal saliency in dynamic scenes[END_REF] datasets. The compared methods are:

PAWCS [START_REF] St-Charles | A self-adjusting approach to change detection based on background word consensus[END_REF], IUTIS-5 [START_REF] Bianco | Combination of video change detection algorithms by genetic programming[END_REF], BSUV-Net [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF], ROSL [START_REF] Shu | Robust orthonormal subspace learning: Efficient recovery of corrupted low-rank matrices[END_REF], and DECOLOR [START_REF] Zhou | Moving object detection by detecting contiguous outliers in the low-rank representation[END_REF]. Our proposed algorithm performs significantly better than the compared methods in these challenging sequences. . . . . . . . . . . .

3-2

The pipeline of the MOS algorithm with the reconstruction of graph signals. The algorithm uses background initialization and superpixel segmentation [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF][START_REF] Yang | Superpixel segmentation with fully convolutional networks[END_REF]. Each superpixel represents a node in a graph, and the representation of each node is obtained with motion, intensity, texture, and deep features. The ground-truth is used to decide if a node is a moving (green nodes) or a static object (blue nodes). Black nodes correspond to the non-labeled images in the dataset. Finally, some nodes are sampled and the semi-supervised algorithm reconstructs all the labels in the graph. . . . . . . . . . . . . . . . . . . . . . . . . . . xix 3-3 Results of the semantic, instance, and superpixel segmentation using DeepLab [START_REF] Chen | DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[END_REF], Mask R-CNN [20], and SLIC [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF]

6-1

The pipeline of our S-SobGNN algorithm with 𝑘-NN or smooth-learned graphs. Our GNN can be used in a broad range of data such as images, and text, among others. However, the step of mapping the original dataset to the data matrix X ∈ R 𝑁 ×𝑀 could be different in each case.

Our framework is composed of (a) inference of the graph topology and Neighbor Interpolation (NNI) [START_REF] Sibson | A brief description of natural neighbor interpolation[END_REF], Graph Regularization (GR) [START_REF] Narang | Localized iterative methods for interpolation in graph structured data[END_REF],

Tikhonov regularization [START_REF] Perraudin | Towards stationary time-vertex signal processing[END_REF][START_REF] Loukas | Stationary time-vertex signal processing[END_REF], Time-varying Graph Signal Reconstruction (TGSR) [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF], and Random Sampling and Decoder (RSD) [START_REF] Puy | Random sampling of bandlimited signals on graphs[END_REF].

Graphs

A graph is a mathematical entity that can represent relationships between discrete objects. These objects are mathematical abstractions called nodes or vertices, where the relationships between them are captured with the so-called edges or links. A graph is thus given by a set of nodes and a set of edges. Graphs can be undirected like in friendship: a person A is friend of a person B, and vice versa. Graphs can also be directed like in a virtual social network of followers: a person A can follow a person B, but it could not be the case in the other sense. Similarly, we can have unweighted graphs where we have a connection or not, like in a social network (we do not say a person A is 50% friend of a person B ). Graphs can also be weighted, where we represent how strong is the connection between nodes, like in Google Maps where we know how far is some place A to another place B. Unless explicitly stated otherwise, we consider undirected, connected, and weighted graphs in this thesis.

The fundamentals and applications of graphs have gained significant attention in recent years. Notably, Graph Neural Networks (GNNs) and Graph Signal Processing (GSP) have been extensively studied [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF][START_REF] Shuman | The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF][START_REF] Bruna | Spectral networks and locally connected networks on graphs[END_REF][START_REF] Defferrard | Convolutional neural networks on graphs with fast localized spectral filtering[END_REF][START_REF] Hamilton | Inductive representation learning on large graphs[END_REF][START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF][START_REF] Veličković | Graph attention networks[END_REF][START_REF] Wu | A comprehensive survey on graph neural networks[END_REF]. GNNs extend the concepts of Convolutional Neural Networks (CNNs) [START_REF] Lecun | Deep learning[END_REF] to non-Euclidean data modeled as graphs.

Similarly, GSP extends the concepts of classical digital signal processing to signals supported on graphs [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF]. GNNs and GSP have numerous applications such as semi-supervised learning [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF][START_REF] Veličković | Graph attention networks[END_REF], point cloud semantic segmentation [START_REF] Wang | Dynamic graph CNN for learning on point clouds[END_REF][START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF], prediction of individual relations in social networks [START_REF] Uwents | Neural networks for relational learning: An experimental comparison[END_REF], and modeling of proteins for drug discovery [START_REF] Zitnik | Predicting multicellular function through multilayer tissue networks[END_REF][START_REF] Gainza | Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning[END_REF]. Similarly, other graph techniques have been recently applied to image, video, and medical image processing applications [START_REF] Giraldo | Graph moving object segmentation[END_REF][START_REF] Egilmez | Graph-based transforms for video coding[END_REF][START_REF] Javed | Robust structural low-rank tracking[END_REF][44].

Even though graph-based algorithms have had a lot of success in recent years, there are still several open questions. These topics range from fundamental aspects of the algorithms, to specific issues related to the application domains. In this thesis, we use graphs to model the complex relationships that exist between instances in some data. Thus, we exploit this intrinsic information to solve problems in: 1) learning with minimal supervision in computer vision and machine learning, and 2) reconstruction of time-varying signals that live on graphs.

Scope and Research Questions

This thesis is divided into three parts. Part 1 introduces novel concepts of graphs in some problems of computer vision like moving objects and semantic segmentation.

Part 2 addresses problems of machine learning on graphs. In particular, we introduce several contributions in GNNs using concepts of GSP, where we study concepts like homophily [START_REF] Zhu | Beyond homophily in graph neural networks: Current limitations and effective designs[END_REF], graph convolutions [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF], over-smoothing [START_REF] Li | Deeper insights into graph convolutional networks for semi-supervised learning[END_REF], and over-squashing [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF]. Therefore, we have to solve an optimization problem each time we want to evaluate our algorithm. GraphMOS is evaluated on several datasets for MOS.

2. How can we overcome the limitations of GraphMOS using concepts of GNNs?

GraphMOS requires to solve an optimization problem if new instances are added to the dataset, which limits its practical applications. To address this problem, we propose to solve MOS with GCNs (GraphMOS-Net) in Chapter 4.

GraphMOS-Net performs better than GraphMOS in several challenging conditions. Similarly, GraphMOS-Net is faster than GraphMOS in evaluation. Our algorithm uses a sparse Sobolev norm using concepts of GSP. Similarly, we add a linear combination mechanism to make graph convolutions more expressive, while keeping scalability. S-SobGNN is evaluated in several tasks including, semi-supervised learning, graph node classification, and MOS.

What is the relationship between over-smoothing and over-squashing in GNNs?

We introduce a topological relationship between over-smoothing and over-squashing in Chapter 7. This underlying relationship is linked using the spectral gap of the Laplacian representation of the graph. We also introduce a new Stochastic Jost and Liu curvature Rewiring (SJLR) algorithm to alleviate over-smoothing and over-squashing. SJLR is evaluated in several benchmarking datasets for node classification.

Chapter 2 Background

In this chapter, we provide an introduction to the mathematical notation and some background topics that will be used in this thesis. More specifically, Section 2.1 presents the mathematical notation in this thesis. Section 2.2 introduces the basic concepts and definitions of graphs and graph signals. Section 2.3 explains the fundamental concepts of sampling and reconstruction of graph signals, and Section 2.4 explains the concept of smooth graph signals. Finally, Section 2.5 presents some concluding remarks of this chapter. Later chapters in this thesis introduce additional background concepts whenever required.

Mathematical Notation

In this thesis, calligraphic letters such as 𝒱 denote sets and |𝒱| represents its cardinality. Uppercase boldface letters such as A represent matrices, and lowercase boldface letters such as x denote vectors. I is the identity matrix and 1 is a vector of ones with appropriate dimensions. (•) T and (•) H represent transposition and Hermitian transpose. The vectorization of matrix A is denoted as vec (A), and diag(x) is the diagonal matrix with entries {𝑥 1 , 𝑥 2 , . . . , 𝑥 𝑁 } as its diagonal elements. ‖ • ‖ 2 is the ℓ 2 -norm of a vector, ‖ • ‖ 𝐹 is the Frobenius norm, and ‖ • ‖ 1,1 is the entry-wise norm-1 of a matrix. 𝜆 max (A) and 𝜆 min (A) represent the maximum and minimum eigenvalues of matrix A, respectively. 𝜅(•) is the condition number of a matrix, and ≜ is the symbol for "equal by definition". ∇ x 𝑓 (x) is the gradient of certain vector-valued function 𝑓 (x), and ∇ 2

x 𝑓 (x) is its Hessian. Finally, ∘ and ⊗ represent the Hadamard and Kronecker product between matrices, respectively.

Graph Signals

Let 𝐺 = (𝒱, ℰ) be an undirected and weighted graph. 𝒱 = {1, . . . , 𝑁 } is the set of 𝑁 nodes and ℰ = {(𝑖, 𝑗)} is the set of edges, where (𝑖, 𝑗) is an edge between the vertices 𝑖 and 𝑗. A ∈ R 𝑁 ×𝑁 is the adjacency matrix of 𝐺 such that A(𝑖, 𝑗) = 𝑎 𝑖𝑗 ∈ R + is the weight connecting vertices 𝑖 and 𝑗. As a consequence, A is symmetric for undirected graphs. A graph signal is a function 𝑦 : 𝒱 → R defined on the nodes of 𝐺, and it can be represented as y ∈ R 𝑁 where y(𝑖) is the function evaluated on the 𝑖th node. Moreover, D ∈ R 𝑁 ×𝑁 is the diagonal degree matrix of 𝐺 such that D(𝑖, 𝑖) = ∑ 𝑁 𝑗=1 A(𝑖, 𝑗) ∀ 𝑖 = 1, . . . , 𝑁 , and 𝑑 𝑖 = D(𝑖, 𝑖). There are several definitions in the literature for the Laplacian matrix. For example, L = D -A is the combinatorial Laplacian matrix, and Δ = I -D -1 2 AD -1 2 is the symmetric normalized Laplacian [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF]. The Laplacian matrix is a positive semi-definite matrix for undirected graphs with eigenvalues 1 0 =

𝜆 1 ≤ 𝜆 2 ≤ • • • ≤ 𝜆 𝑁 and corresponding eigenvectors {u 1 , u 2 , . . . , u 𝑁 }.
The graph Fourier basis of 𝐺 is defined by the spectral decomposition of L = UΛU T [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF], where U = [u 1 , u 2 , . . . , u 𝑁 ] and Λ = diag(𝜆 1 , 𝜆 2 , . . . , 𝜆 𝑁 ), where 𝜆 𝑖 is the frequency associated to the 𝑖th eigenvalue [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF]. Therefore, the Graph Fourier Transform (GFT) ŷ of the signal y is defined as ŷ = U T y, and the inverse GFT is

given by y = Uŷ [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF].

Definition 2.2.1. A graph signal y is called bandlimited if ∃ 𝜌 ∈ {1, 2, . . . , 𝑁 -1} such that its GFT satisfies ŷ(𝑖) = 0 ∀ 𝑖 > 𝜌.
The smallest 𝜌 that holds Definition 2.2.1 is called the bandwidth of y. Using these notions of frequency, Pesenson [START_REF] Pesenson | Sampling in Paley-Wiener spaces on combinatorial graphs[END_REF] defined the space of all 𝜔-bandlimited signals as

Sampling and Reconstruction of Graph Signals

Given the definition of graph signals and the notions of bandlimitedness in terms of 𝑃 𝑊 𝜔 (𝐺), the next logical step is to find a bound for the minimum number of samples required to reach perfect recovery of y ∈ 𝑃 𝑊 𝜔 (𝐺). The answer is that one needs at least 𝜌 (bandwidth) labeled nodes to achieve perfect reconstruction. Intuitively, a graph signal y is smooth in 𝐺 when y ∈ 𝑃 𝑊 𝜔 (𝐺). For example, suppose a sensor network of temperatures in a specific region. One would expect that the temperature of two or more nearby localities should be similar, i.e., the value of the graph signal evaluated in two or more strongly connected nodes should not be very different. As a consequence, probably one just needs the temperature of some of these nodes to reconstruct the whole graph signal in the other vertices.

To add mathematical precision to the notion of perfect reconstruction, the sampling of a graph signal is defined in terms of a subset of nodes 𝒮 ⊂ 𝒱 with 𝒮 = Theorem 2.3.1 (Chen's theorem [START_REF] Chen | Discrete signal processing on graphs: Sampling theory[END_REF]). Let M satisfy rank(MU 𝜌 ) = 𝜌. For all y ∈ 𝑃 𝑊 𝜔 (𝐺), perfect recovery, i.e., y = ΦMy, is achieved by choosing:

Φ = U 𝜌 V, (2.1)
with VMU 𝜌 a 𝜌 × 𝜌 identity matrix.

Proof: see [START_REF] Chen | Discrete signal processing on graphs: Sampling theory[END_REF].

Theorem 2.3.1 states that perfect reconstruction of graph signal from its samples is possible when y lies in 𝑃 𝑊 𝜔 (𝐺), and the number of samples is at least 𝜌. Then, perfect reconstruction is achieved by choosing the interpolation operator as in (2.1).

A common approach to obtain a reconstructed version of y is given by:

arg min z∈span(U𝜌) ||Mz -y(𝒮)|| 2 2 = U 𝜌 (MU 𝜌 ) † y(𝒮), (2.2)
where

U 𝜌 = [u 1 , u 2 , . . . ,

Smooth Graph Signals

The notion of bandlimitedness in Definition 2.2.1 is related to the smoothness of y.

When y ∈ 𝑃 𝑊 𝜔 (𝐺), the variation of y is smooth in the vertex domain. From Definition 2.2.1, one knows that the GFT is required to check if y ∈ 𝑃 𝑊 𝜔 (𝐺). However, the computation of the GFT requires the calculation of the eigenbasis, which is computationally prohibitive for large graphs. In this thesis, the computation of U is avoided by leveraging notions of global smoothness in 𝐺.

Formally, notions of smoothness in y were introduced through concepts of local variation and the discrete 𝑝-Dirichlet form [START_REF] Shuman | The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF]. The local variation of y at vertex 𝑖 is defined as:

‖∇ 𝑖 y‖ 2 ≜ [ ∑ 𝑗∈𝒩 𝑖 A(𝑖, 𝑗)[y(𝑗) -y(𝑖)] 2 ] 1 2 , (2.3)
where 𝒩 𝑖 is the set of vertices connected to the node 𝑖 by one edge. Thus, the discrete 𝑝-Dirichlet form is defined as 𝑆 𝑝 (y) ≜ 1

𝑝

∑

𝑖∈𝒱 ‖∇ 𝑖 y‖ 𝑝 2 , then:

𝑆 𝑝 (y) = 1 𝑝 ∑ 𝑖∈𝒱 [ ∑ 𝑗∈𝒩 𝑖 A(𝑖, 𝑗)[y(𝑗) -y(𝑖)] 2 ] 𝑝 2 .
(2.4)

For example, when 𝑝 = 2:

𝑆 2 (y) = ∑ (𝑖,𝑗)∈ℰ A(𝑖, 𝑗)[y(𝑗) -y(𝑖)] 2 = y T Ly. (2.5)
𝑆 2 (y) is known as the graph Laplacian quadratic form [START_REF] Shuman | The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF]. Notice that 𝑆 2 (y) = 0 ⇐⇒ y = 𝜏 1, where 𝜏 is a constant; and more generally, 𝑆 2 (y) is small when the graph signal y has similar values at neighboring nodes connected by an edge, i.e., when the signal is smooth.

The Laplacian quadratic form in (2.5) has been used as regularizer in reconstruction of graph signals, and semi-supervised learning problems [START_REF] Belkin | Manifold regularization: A geometric framework for learning from labeled and unlabeled examples[END_REF], where this regularizer looks for smooth graph signals. Intuitively, there is a relationship between the smoothness of a graph signal and its bandwidth. For example, the variational problem [START_REF] Pesenson | Variational splines and Paley-Wiener spaces on combinatorial graphs[END_REF] leads to the same solution of Eqn. (2.2) when y holds Definition 2.2.1 [52] (for further details see Section 3.3.6). Therefore, the minimization of the 𝑝-Dirichlet form is aligned with the prior assumption of bandlimitedness, without the explicit computation of the eigenbasis U and the bandwidth 𝜌 of y. Formally, since L is positive semi-definite for undirected graphs, all the eigenvalues are non-negative and real, and a full set of orthogonal eigenvectors can be obtained as explained in Section 2.2. The matrix of eigenvectors U is known as the GFT matrix of the graph. The eigenvalue-eigenvector pairs can be viewed as successive optimizers of the Rayleigh quotient, where the 𝑖th pair 𝜆 𝑖 , u 𝑖 solves:

u 𝑖 = arg min y T u 𝑖 ′ =0,𝑖 ′ =0,...,𝑖-1 y T Ly y T y , (2.6)
with

𝜆 𝑖 = u T 𝑖 Lu 𝑖 if u T 𝑖 u 𝑖 = 1.
The term y T Ly is precisely the Laplacian quadratic form of the graph signal y [START_REF] Shuman | The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF], i.e., the GFT provides an orthogonal basis with in-Figure 2-1: Example of elementary frequencies obtained from the Laplacian matrix on a sensor network of 𝑁 = 500. Each graph shows a frequency 𝜆 𝑖 with its corresponding eigenvector. The lowest frequency is 𝜆 1 = 0, corresponding to a constant graph signal, i.e., the Laplacian quadratic form of eigenvector u 1 is given such that u T 1 Lu 1 = 𝜆 1 = 0.

creased variation [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF]. Figure 2-1 shows an example of the eigenvectors of a weighted undirected sensor network with 500 nodes. One can notice that the eigenvector u 𝑖 has more variations as the value of 𝜆 𝑖 increases, where the Laplacian quadratic form of u 1 , given as u T 1 Lu 1 = 𝜆 1 = 0, corresponds to a constant-valued eigenvector. Since the GFT of a graph signal is given such that ŷ = U T y, one can precisely represent a bandlimited graph signal in the Fourier domain as ŷ = [u 1 , . . . , u 𝜌 , 0, . . . , 0] T y according to Definition 2.2.1, i.e., a mapping of the first 𝜌 eigenvectors. As a consequence, a bandlimited graph signal is smooth on undirected graphs.

In semi-supervised learning, the number of samples required to get perfect classification increases when the bandwidth 𝜌 increases. In practice, graph signals are only approximately bandlimited [START_REF] Anis | Efficient sampling set selection for bandlimited graph signals using graph spectral proxies[END_REF]. As a consequence, the classification error is bounded by a value 𝜑 [52].

Conclusions

Graph representation learning and its applications have gained significant attention in recent years. GNNs and GSP have been extensively studied [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF][START_REF] Bruna | Spectral networks and locally connected networks on graphs[END_REF][START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF]. GNNs and GSP have numerous applications in machine learning [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] and computer vision [START_REF] Giraldo | Graph moving object segmentation[END_REF][START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF].

Despite the success of GNNs and GSP in many applications, there are still several open problems. For instance, GNNs suffer from the over-smoothing (and maybe the over-squashing) issue when several graph-convolutional layers are stacked to create deep models [START_REF] Li | Deeper insights into graph convolutional networks for semi-supervised learning[END_REF]. This problem limits the performance of deep GNN architectures.

The following chapters of this thesis deal with some fundamental problems in graph representation learning and its applications in several fields of computer science.

13

Part I

Computer Vision

Chapter 3

Graph Moving Object Segmentation

Introduction

MOS is a relevant topic in computer vision and video analysis. MOS is a preprocessing task in many applications such as robotics system, intelligent transportation, and intelligent visual surveillance systems of human activities in public spaces, among others [START_REF] Garcia-Garcia | Background subtraction in real applications: Challenges, current models and future directions[END_REF]. The main objective of MOS is to separate the moving objects called foreground, from the static component called background [START_REF] Lee | Effective Gaussian mixture learning for video background subtraction[END_REF][START_REF] Sheikh | Bayesian modeling of dynamic scenes for object detection[END_REF][START_REF] Haines | Background subtraction with Dirichlet process mixture models[END_REF][START_REF] Bouwmans | Traditional and recent approaches in background modeling for foreground detection: An overview[END_REF]. In the literature, MOS has been considered as a classification problem where each pixel is classified for either background or moving object in a sequence taken from a static or moving camera, and therefore this problem is also known as background subtraction and moving object detection [START_REF] Bouwmans | Decomposition into low-rank plus additive matrices for background/foreground separation: A review for a comparative evaluation with a large-scale dataset[END_REF]. Many efforts have been reported in the literature to improve the classical methods progressively in applications where challenges are becoming more complex. However, none of the methods can simultaneously address all the key challenges that are present in videos during long sequences as in the real cases [10]. In fact, several studies are focused on designing methods for specific challenges in MOS such as turbulence [START_REF] Oreifej | Simultaneous video stabilization and moving object detection in turbulence[END_REF], dynamic backgrounds [START_REF] Pham | Detection of dynamic background due to swaying movements from motion features[END_REF], and camouflaged moving objects [START_REF] Li | A fusion framework for camouflaged moving foreground detection in the wavelet domain[END_REF]. Furthermore, many studies are limited to deal with shadows and the sequences taken from Pan-Tilt-Zoom (PTZ) cameras because of their challenging nature [START_REF] Garcia-Garcia | Background subtraction in real applications: Challenges, current models and future directions[END_REF][START_REF] Bouwmans | Robust PCA via principal component pursuit: A review for a comparative evaluation in video surveillance[END_REF][START_REF] Bouwmans | Deep neural network concepts for background subtraction: A systematic review and comparative evaluation[END_REF][START_REF] Maddalena | Towards benchmarking scene background initialization[END_REF].

MOS methods can broadly be categorized into unsupervised and supervised learn-ing schemes. Many unsupervised learning methods have been proposed in the literature, and they rely on background models to predict the foreground objects [START_REF] Bouwmans | Traditional and recent approaches in background modeling for foreground detection: An overview[END_REF][START_REF] Bouwmans | Robust PCA via principal component pursuit: A review for a comparative evaluation in video surveillance[END_REF][START_REF] Bouwmans | Background modeling using mixture of Gaussians for foreground detection-a survey[END_REF].

However, these methods show performance degradation in the presence of dynamic background scenes. Supervised learning methods are based on CNNs [START_REF] Bouwmans | Deep neural network concepts for background subtraction: A systematic review and comparative evaluation[END_REF]. CNN-based methods have demonstrated better performance than the unsupervised methods, however, most of these models fail to get optimal performance when employed on unseen videos (poor generalization). For example, the FgSegNet method uses 200 images from the test video for training and the remaining frames from the same video for evaluation [START_REF] Lim | Learning multi-scale features for foreground segmentation[END_REF]. The performance of FgSegNet dramatically decreases when applied to unseen videos [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF]. Fig. 3-1 shows some visual results of the state-of-the-art methods for MOS under challenging background scenarios. Despite significant efforts and competitive performance in particular challenges, there are still several open issues for the MOS task [START_REF] Garcia-Garcia | Background subtraction in real applications: Challenges, current models and future directions[END_REF][START_REF] Bouwmans | Deep neural network concepts for background subtraction: A systematic review and comparative evaluation[END_REF]. 1) None of the methods can effectively handle all MOS challenges in the presence of static and moving camera sequences. 2) Some CNNs-based models do not have a good performance on unseen videos, or their generalizations to other videos are hardly predictable [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF]. 3) Deep learning methods lack theoretical guarantees and explanations about the sample complexity required during the endto-end learning process. The classical fundamental theorem of statistical learning, involving the Vapnik-Chervonenkis dimension, bounds the sample complexity in machine learning [START_REF] Shalev-Shwartz | Understanding machine learning: From theory to algorithms[END_REF]. However, this bound does not guarantee the performance in deep learning regimen because of the huge amount of parameters in common deep neural networks.

In recent years, a growing number of graph-based methods have been proposed for many computer vision and machine learning applications such as object tracking [START_REF] Javed | Robust structural low-rank tracking[END_REF][START_REF] Li | Visual tracking via dynamic graph learning[END_REF], MOS [START_REF] Javed | Spatiotemporal lowrank modeling for complex scene background initialization[END_REF][START_REF] Javed | Background-foreground modeling based on spatiotemporal sparse subspace clustering[END_REF][START_REF] Javed | Moving object detection in complex scene using spatiotemporal structured-sparse rpca[END_REF], and tissue community detection [44,[START_REF] Javed | Cellular community detection for tissue phenotyping in colorectal cancer histology images[END_REF]. Graphs can model complex relationships on data [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF]. These structures capture the intrinsic geometric structure in data and can model data points and complex interactions among them.

Each node or vertex on the graph represents one data point to which a label can be associated, and a graph can be formed by connecting vertices with edge weights that are assigned based on distance values among the data points in the feature space. The compared methods are: PAWCS [START_REF] St-Charles | A self-adjusting approach to change detection based on background word consensus[END_REF], IUTIS-5 [START_REF] Bianco | Combination of video change detection algorithms by genetic programming[END_REF], BSUV-Net [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF], ROSL [START_REF] Shu | Robust orthonormal subspace learning: Efficient recovery of corrupted low-rank matrices[END_REF], and DECOLOR [START_REF] Zhou | Moving object detection by detecting contiguous outliers in the low-rank representation[END_REF]. Our proposed algorithm performs significantly better than the compared methods in these challenging sequences.

Social, financial, ecological networks, and the human brain are few examples of such data structure that can be modeled on graphs [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF][START_REF] Shuman | The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF]. GSP enables different types of learning and filtering operations on values associated with graph nodes [START_REF] Chen | Discrete signal processing on graphs: Sampling theory[END_REF][START_REF] Anis | Efficient sampling set selection for bandlimited graph signals using graph spectral proxies[END_REF][START_REF] Romero | Kernel-based reconstruction of graph signals[END_REF][START_REF] Parada-Mayorga | Blue-noise sampling on graphs[END_REF].

For inference, these graph models are used to classify graph signals. Therefore, semisupervised learning can be modeled as the reconstruction of a graph signal from its samples [START_REF] Anis | Efficient sampling set selection for bandlimited graph signals using graph spectral proxies[END_REF]. When data labels are presented as signals on a graph, graph signal regularization techniques can be used in the process of estimating the unknown labels for graph nodes classification [START_REF] Anis | Efficient sampling set selection for bandlimited graph signals using graph spectral proxies[END_REF][START_REF] Romero | Kernel-based reconstruction of graph signals[END_REF].

In this chapter, we pose the problem of MOS as a semi-supervised learning prob- [10], I2R [START_REF] Li | Foreground object detection from videos containing complex background[END_REF], Scene Background Initialization (SBI2015) [START_REF] Maddalena | Towards benchmarking scene background initialization[END_REF], and UCSD background subtraction [START_REF] Mahadevan | Spatiotemporal saliency in dynamic scenes[END_REF] datasets.

lem
The advantages of GraphMOS are: 1) its good performance even when the background scene rapidly changes, which is difficult to handle using existing MOS methods (Fig. 3-1), and 2) its theoretical foundation, unlike other state-of-the-art methods.

The main contributions of the current work are summarized as follows:

• The MOS problem is posed as a graph learning problem by using the concepts of GSP. To the best of our knowledge, this is the first work that exploits the sampling and reconstruction of graph signals for MOS.

• One theoretical development is introduced, showing the upper-bound for the sample complexity required in semi-supervised learning under some prior assumptions in Corollary 3.3.1.

• Extensive evaluations are performed on four publicly available MOS benchmark datasets, and we compared our algorithms with 20 existing state-of-theart methods with rigorous analysis. Unlike previous methods in the literature, our proposed algorithm can be applied to MOS with static and moving camera sequences.

Related Works

This section presents brief reviews for 1) GSP and its application to computer vision, and 2) unsupervised and supervised MOS algorithms.

Graph Signal Processing

Even though the study of graphs is an ancient field, Sandryhaila and Moura were the first authors to introduce the term of discrete signal processing on graphs and later coined with the name of GSP [START_REF] Sandryhaila | Discrete signal processing on graphs[END_REF]. Graph signal processing emerged with the idea of developing tools to analyze data living in irregular and complex structures [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF].

From one point of view, the first developments of GSP come from the studies of lowdimensional representations for high-dimensional data through spectral graph theory, and the graph Laplacian [START_REF] Chung | Spectral graph theory[END_REF]. From another perspective, several authors developed compression schemes, wavelet decomposition, filter banks on graphs, regression al-gorithms, and denoising using the graph Laplacian motivated by the data collected from sensor networks [START_REF] Zhu | Approximating signals supported on graphs[END_REF][START_REF] Hammond | Wavelets on graphs via spectral graph theory[END_REF][START_REF] Narang | Local two-channel critically sampled filter-banks on graphs[END_REF][START_REF] Guestrin | Distributed regression: an efficient framework for modeling sensor network data[END_REF][START_REF] Wagner | Distributed wavelet de-noising for sensor networks[END_REF].

GSP has also been widely used in image processing and computer graphics. For example, Shi and Malik represented images as graphs to treat segmentation as a graph partitioning problem [START_REF] Shi | Normalized cuts and image segmentation[END_REF]. In the same way, image filtering techniques can be interpreted from a graph point of view [START_REF] Cheung | Graph spectral image processing[END_REF]. Similarly in computer graphics, models like meshes can be naturally modeled with graphs to apply graph-based filtering and multi-resolution operations [START_REF] Guskov | Multiresolution signal processing for meshes[END_REF][START_REF] Zhou | 3D surface filtering using spherical harmonics[END_REF].

In video processing, GSP is useful to model the spatiotemporal relationships among frames. For instance, the graph Cartesian product could be useful to process videos taking into account the spatiotemporal relationships of the pixels [START_REF] Hammack | Handbook of product graphs[END_REF].

Interested readers can explore more details about GSP and its machine learning applications in a recent survey [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF]. In our proposed algorithm, graphs are used to model the relationship among the nodes on videos where the graph signal represents the class foreground or background of the set of nodes in a dataset. Finally, the reconstruction of graph signals is applied to classify if a certain node is a moving or static object for MOS.

Moving Object Segmentation

There are many unsupervised methods in the literature to address the problem of MOS. These methods can be grouped as statistical [START_REF] Stauffer | Adaptive background mixture models for real-time tracking[END_REF], fuzzy [START_REF] Baf | Fuzzy integral for moving object detection[END_REF], subspace learning [START_REF] Dong | Adaptive learning of multi-subspace for foreground detection under illumination changes[END_REF], robust principal component analysis [START_REF] Shang | Bilinear factor matrix norm minimization for robust PCA: Algorithms and applications[END_REF][START_REF] Vaswani | Robust subspace learning: Robust PCA, robust subspace tracking, and robust subspace recovery[END_REF], neural networks [START_REF] Ramirez-Quintana | Self-adaptive SOM-CNN neural system for dynamic object detection in normal and complex scenarios[END_REF], and filteringbased [START_REF] Cinar | Adaptive background estimation using an information theoretic cost for hidden state estimation[END_REF] models. Interested readers may explore a complete review of unsupervised methods in the survey papers [START_REF] Bouwmans | Traditional and recent approaches in background modeling for foreground detection: An overview[END_REF], [START_REF] Bouwmans | Robust PCA via principal component pursuit: A review for a comparative evaluation in video surveillance[END_REF], and [START_REF] Bouwmans | Background modeling using mixture of Gaussians for foreground detection-a survey[END_REF]. With the success of deep CNNs on a wide variety of computer vision applications [START_REF] Lecun | Deep learning[END_REF], several studies have also been proposed for MOS [START_REF] Bouwmans | Deep neural network concepts for background subtraction: A systematic review and comparative evaluation[END_REF][START_REF] Braham | Deep background subtraction with scene-specific convolutional neural networks[END_REF][START_REF] Mandal | 3DFR: A swift 3D feature reductionist framework for scene independent change detection[END_REF] applications.

The MOS supervised methods can be classified as basic CNN [START_REF] Braham | Deep background subtraction with scene-specific convolutional neural networks[END_REF], multiscale CNN [START_REF] Lim | Foreground segmentation using convolutional neural networks for multiscale feature encoding[END_REF][START_REF] Wang | Interactive deep learning method for segmenting moving objects[END_REF], fully CNN [START_REF] Garcia-Gonzalez | Foreground detection by probabilistic modeling of the features discovered by stacked denoising autoencoders in noisy video sequences[END_REF], 3D CNN [START_REF] Mandal | 3DFR: A swift 3D feature reductionist framework for scene independent change detection[END_REF], and Generative Adversarial Networks (GANs) [START_REF] Sultana | Unsupervised deep context prediction for background estimation and foreground segmentation[END_REF]. Some studies are also contributed to improving the loss functions during the training and analysis of the CNNs for the problem of MOS [START_REF] Jung | Cosine focal loss-based change detection for video surveillance systems[END_REF]. A complete review of deep learning-based MOS methods can be explored in a recent survey [START_REF] Bouwmans | Deep neural network concepts for background subtraction: A systematic review and comparative evaluation[END_REF].

Although the results of many fully supervised deep learning-based methods show impressive performance for MOS task, the performance of the best FgSegNet method proposed by Lim et al. [START_REF] Lim | Learning multi-scale features for foreground segmentation[END_REF] shows that CNNs methods are not efficient for unseen videos because of the lack of generalization capabilities as proved by Tezcan et al. [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF].

This lack could be due to the limited amount of data available to train these deep learning methods for MOS, the lack of theoretical developments about the sample complexity required in deep learning, and the high model complexities of common deep neural networks. In this chapter, we fill this lack by proposing a semi-supervised graph learning algorithm for MOS in unseen videos. The question of sample complexity is also solved by using the fundamental concepts of GSP under the assumption of bandlimitedness of the underlying graph signals in MOS. It is also evident that a single method, either supervised or unsupervised, cannot effectively handle all the MOS challenges of unseen videos [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF]. Our proposed algorithm also addresses these challenges for sequences taken from both static and moving cameras.

Moving Object Segmentation and Graph Signal Processing

This section presents the basic concepts of our proposed algorithm. The novices in GSP are referred to the review papers [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF][START_REF] Shuman | The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF]. GraphMOS can be viewed as three-step algorithm as follows: 1) a superpixel segmentation method [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF][START_REF] Vargas-Muñoz | An iterative spanning forest framework for superpixel segmentation[END_REF] is used to segment homogeneous regions on each frame; 2) deep and handcrafted The algorithm uses background initialization and superpixel segmentation [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF][START_REF] Yang | Superpixel segmentation with fully convolutional networks[END_REF]. Each superpixel represents a node in a graph, and the representation of each node is obtained with motion, intensity, texture, and deep features. The groundtruth is used to decide if a node is a moving (green nodes) or a static object (blue nodes). Black nodes correspond to the non-labeled images in the dataset. Finally, some nodes are sampled and the semi-supervised algorithm reconstructs all the labels in the graph.

features representation from each superpixel, including optical flow and background initialization, are used to represent the spatiotemporal information of each superpixel node; and 3) a graph reasoning algorithm, including graph construction and semisupervised learning, is used to classify between the static and moving objects with few labeled samples.

Graph Nodes Representation

In our proposed algorithm, there is a need for some mechanism to represent the graph nodes. The pixels of the video frames can represent the nodes of the graph.

However, computational complexity issues arise within the proposed algorithm when using pixel-level nodes. Therefore, we use a group of pixels to define each node in a graph. To that end, one common way is to decompose the input sequence into a regular block structure or superpixels. However, other instance and semantic segmentation methods can also be employed for node representation.

In this chapter, we employ several segmentation methods including, superpixel segmentation [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF], semantic segmentation [START_REF] Chen | DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[END_REF], instance segmentation [20], block-based decomposition, and background subtraction [START_REF] St-Charles | SuBSENSE: A universal change detection method with local adaptive sensitivity[END_REF] to represent the nodes in a graph, as well as to compare the performance of each of the methods. However, the aforementioned segmentation methods present a fundamental limitation in the proposed algorithm. For instance, if the segmentation methods [20], [START_REF] Chen | DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[END_REF], and [START_REF] St-Charles | SuBSENSE: A universal change detection method with local adaptive sensitivity[END_REF] do not segment moving objects, the proposed algorithm will not be able to classify background or foreground objects effectively. On the other hand, superpixel segmentation and block-based decomposition methods can process all the regions in the frames, however, these approaches may contain more graph nodes than the segmentation methods. In the later subsections, we summarize each of these methods in more detail.

Superpixel Segmentation

In our proposed algorithm, we use a Simple Linear Iterative Clustering (SLIC) method for superpixel segmentation [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF]. SLIC adapts a k-means approach to generate a set of superpixels. The desired number of approximately equally-sized superpixels per image 𝜁 is an important parameter in SLIC for GraphMOS, because large values of 𝜁 allow our algorithm to process more detailed regions, but may induce computational burdens on GraphMOS. Several ablation studies are performed in Section 3.5.3 to show the performance of the proposed algorithms by varying the number of superpixels.

Instance Segmentation

We also test several configurations of instance segmentation methods in GraphMOS.

We employ Mask Region Convolutional Neural Network (Mask R-CNN) [20] and Cascade Mask R-CNN [START_REF] Cai | Cascade R-CNN: High quality object detection and instance segmentation[END_REF] with Residual Networks (ResNet) [START_REF] He | Deep residual learning for image recognition[END_REF] and ResNeSt [START_REF] Zhang | ResNeSt: Split-attention networks[END_REF] for instance segmentation. Mask R-CNN builds upon Faster R-CNN by adding a branch for predicting an object mask in parallel with the already existing Faster R-CNN network for bounding box recognition [START_REF] Ren | Faster R-CNN: Towards real-time object detection with region proposal networks[END_REF]. Cascade Mask R-CNN builds upon

Semantic Segmentation

We use the DeepLab method [START_REF] Chen | DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[END_REF] for semantic segmentation to represent the nodes in the graph. However, our algorithm explicitly needs to know the exact location of each segmented instance in a frame to represent each node in a graph. Fig. 3-3 shows the segmentation results of the DeepLab, Mask R-CNN, and SLIC methods on a video frame selected from the dynamic backgrounds category of the CDNet2014 dataset.

GraphMOS relies mainly on superpixel and instance segmentation because semantic segmentation methods do not give information about the location of each specific instance (for example, cars as shown in Fig. 3-3). As a consequence, GraphMOS is unable to differentiate between the parked cars in the background just behind the moving cars in the foreground when relying on semantic segmentation.

Other Segmentation Methods

We also decompose each video frame into non-overlapping blocks in GraphMOS. We use small blocks of size 8 × 8 to represent each node. We have also employed the background subtraction method SUBSENSE [START_REF] St-Charles | SuBSENSE: A universal change detection method with local adaptive sensitivity[END_REF] to extract graph nodes. Several ablation studies are presented in Section 3.5.3 for comparing different segmentation methods.

Background Initialization and Feature Extraction

The MOS on unseen videos in static camera sequences can use the background of the scene as additional information. For the sake of simplicity, the temporal median filter is used as background initialization. The videos are processed in gray-scales in the current chapter.

The representation of the nodes is achieved with optical flow, intensity, texture, and deep features. The feature extraction module processes the ROI of the segmented regions1 in the current frame for the current, previous, and background frames, as well as the absolute value of the difference between the current and background frames.

Let I 𝑡 𝑣 and I 𝑡-1 𝑣 be the gray-scale crops corresponding to the node 𝑣 ∈ 𝒱 in the current (𝑡) and previous frames (𝑡 -1), respectively. Let B 𝑣 be the crop of the background image corresponding to the node 𝑣. Let 𝒫 𝑣 be the set of indices corresponding to the 𝑣th segmented region. Finally, let v 𝑡 𝑥 (𝒫 𝑣 ) and v 𝑡 𝑦 (𝒫 𝑣 ) be the optical flow vectors of the current frame with support in the set of indices 𝒫 𝑣 for the horizontal and vertical direction, respectively. We compute the optical flow by employing the Lucas-Kanade method [START_REF] Lucas | An iterative image registration technique with an application to stereo vision[END_REF]. Fig. 34shows the procedure to extract the features of each segmented region when the segmentation algorithm is a Mask R-CNN. The texture representation is obtained by estimating the local binary patterns [START_REF] Ojala | Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[END_REF] from the optical flow vectors v 𝑡 𝑥 (𝒫 𝑣 ) and v 𝑡 𝑦 (𝒫 𝑣 ) are used to compute histograms and some descriptive statistics (the minimum, maximum, mean, standard deviation, mean absolute deviation, and range). Finally, the deep features of each segmented region are extracted. Inspired by the visual object tracking community [START_REF] Danelljan | ECO: Efficient convolution operators for tracking[END_REF][START_REF] Ma | Robust visual tracking via hierarchical convolutional features[END_REF], we use a pre-trained VGG-m model [START_REF] Chatfield | Return of the devil in the details: Delving deep into convolutional nets[END_REF] to extract the features from the 5th convolutional layer (Conv-5) and then a principal component analysis is applied to compress a high-dimensional feature vector into a low-dimensional vector. The representation of node 𝑣 is obtained concatenating all the previous features, i.e., optical flow, intensity, texture, and deep features. Each instance is represented by a 𝑀 -dimensional vector x 𝑣 . The readers are referred to Appendix B to see more details about x 𝑣 .

Graph Construction

The construction of the graph aims to get geometrical information from the datasets, leading to a reduction in the number of labels required in the learning process. In this chapter, the construction of the graph is achieved with a 𝑘-Nearest Neighborhood (𝑘-NN) with a Gaussian kernel. Let X ∈ R 𝑁 ×𝑀 be the matrix of 𝑁 nodes, in which each node is an 𝑀 -dimensional vector and X = [x 1 , x 2 , . . . , x 𝑁 ] T . Firstly, the 𝑘-NN algorithm is used to connect the nodes in the graph. Afterward, vertices are connected to get an undirected and weighted graph. The weight between two connected vertices 𝑖, 𝑗 is given such that 𝑎 𝑖𝑗 = exp -

‖x 𝑖 -x 𝑗 ‖ 2 2 𝜎 2
, where 𝜎 is the standard deviation of the Gaussian function given as 𝜎 = 1

|ℰ|

∑

(𝑖,𝑗)∈ℰ ‖x 𝑖 -x 𝑗 ‖ 2 . As a result, the adjacency matrix A(𝑖, 𝑗) = 𝑎 𝑖𝑗 ∀ (𝑖, 𝑗) ∈ ℰ is obtained from this process.

A reasonable thought is to use a complete graph to avoid the optimization of parameter 𝑘. However, a complete graph requires a prohibitive amount of memory to store matrices A and L when dealing with huge graphs. For example, a complete graph with 𝑁 = 200000 would require 320 gigabytes of memory to store A, where each edge in A is represented with a variable of type double of 8 bytes. Other strategies for the representation of the graph can be used in GraphMOS. For example, Gangapure et al. [START_REF] Gangapure | Superpixel-based causal multisensor video fusion[END_REF] proposed a superpixel based causal multisensor video fusion method, where the key idea is to leverage temporal information for video and then construct spatiotemporal graph models.

Graph Signal

The graph signal is a matrix Y ∈ R 𝑁 ×𝑄 , where 𝑄 is the number of classes of the problem. y 𝑞 = Y :,𝑞 is the graph signal associated with the 𝑞th class, where Y :,𝑞 is the 𝑞th column vector of matrix Y. Each row of Y represents if certain segmented region belongs to the 𝑞th class. In this chapter, the graph signal is given by the membership function Y 𝑐 of each class 𝑐, which takes a value of 1 on a node which belongs to the class and is 0 otherwise. For example, in MOS 𝑄 is equal to 2 that corresponds to the classes static object [1, 0] and moving object [0, 1]. Given a segmented region, using some of the methods in Section 3.3.1, we can establish that node is a moving object if the statistical mode in the ground-truth is the class moving object and vice versa.

Notice this can be easily extended to multiple classes, e.g., in semantic segmentation with multiple classes. 3, the answer is that one needs at least 𝜌 (bandwidth) labeled nodes to achieve perfect classification. This also holds for regression of graph signals, i.e., given y ∈ 𝑃 𝑊 𝜔 (𝐺), the number of sampled nodes required to get perfect reconstruction is 𝜌.

Sampling of Graph signals and Sample Complexity for

Semi-supervised Learning

In Section 2.3, we provided Theorem 2.3.1 that states when perfect reconstruction is possible. One can relate Theorem 2.3.1 to the sample complexity in semi-supervised learning as follows:

Corollary 3.3.1. Let Y ∈ R 𝑁 ×𝑄
be a graph signal associated with a semi-supervised learning problem, where 𝑄 is the number of classes; and let 𝑁 𝑠 be the sample complexity of the semi-supervised learning problem. Y 𝑖,: = 𝛿 T 𝑞 , where 𝛿 𝑞 is a 𝑄-dimensional Kronecker column vector centered at 𝑞, and Y 𝑖,: is the 𝑖th row of Y. Y has a set of cutoff frequencies {𝜔 1 , . . . , 𝜔 𝑞 }, with corresponding bandwidths {𝜌 1 , . . . , 𝜌 𝑞 } for each graph signal Y :,𝑞 ∀ 1 ≤ 𝑞 ≤ 𝑄. Then, 𝑁 𝑠 is bounded such that:

𝑁 𝑠 ≤ max{𝜌 1 , . . . , 𝜌 𝑞 }. (3.1)
Proof. From theorem 2.3.1, one can get a perfect reconstruction of each graph signal

Y :,𝑖 ∈ 𝑃 𝑊 𝜔 𝑖 (𝐺) ∀ 1 ≤ 𝑖 ≤ 𝑞 using Eqn. (2.
2) with at least 𝜌 𝑖 samples. As a consequence, the worst-case scenario in the sample complexity 𝑁 𝑠 for perfect reconstruction of Y is max{𝜌 1 , . . . , 𝜌 𝑞 }.

Minimization of the Sobolev Norm

One of the semi-supervised learning methods in this chapter is based on the variational splines of Pesenson [START_REF] Pesenson | Variational splines and Paley-Wiener spaces on combinatorial graphs[END_REF].

Definition 3.3.1. For a fixed 𝜖 ≥ 0 the Sobolev norm is introduced by the following formula:

‖f ‖ 𝛼,𝜖 = ‖(L + 𝜖I) 𝛼/2 f ‖ 2 , 𝛼 ∈ R. (3.2)
Given a graph signal of sampled labels y 𝑞 (𝒮) = My 𝑞 , a positive 𝛼 > 0, and a non-negative 𝜖 ≥ 0, the variational problem for semi-supervised learning is stated as follows: find a vector z 𝑞 with the following properties: z 𝑞 (𝒮) = Mz 𝑞 = y 𝑞 (𝒮), and z 𝑞

minimizes functional z 𝑞 → ‖(L + 𝜖I) 𝛼/2 z 𝑞 ‖ 2 .
In other words, the variational problem solves the following optimization problem:

arg min z𝑞 ‖z 𝑞 ‖ 2 𝛼,𝜖 s.t. Mz 𝑞 = y 𝑞 (𝒮) → arg min z𝑞 z T 𝑞 (L + 𝜖I) 𝛼 z 𝑞 s.t. Mz 𝑞 -y 𝑞 (𝒮) = 0. (3.3) Equation (3.
3) is a convex optimization problem since the term z T 𝑞 (L + 𝜖I) 𝛼 z 𝑞 is a quadratic convex function in z 𝑞 ; and the term Mz 𝑞 -y 𝑞 (𝒮) is affine in z 𝑞 . The semi-supervised learning problem is solved by determining the solution of (3.3) for 𝑞 = 1, . . . , 𝑄.

The minimization of the Sobolev norm in (3.3) is closely related to the Laplacian quadratic form. The Laplacian quadratic form of y is given by y T Ly as shown in (2.5).

In the same way, y T L 𝛼 y is known as the empirical iterated Laplacian regularizer or higher-order regularization [START_REF] Zhou | Semi-supervised learning by higher order regularization[END_REF] and has been used for regression and classification tasks in semi-supervised learning [START_REF] Belkin | Manifold regularization: A geometric framework for learning from labeled and unlabeled examples[END_REF]. On the other hand, the Sobolev norm is such that y T (L + 𝜖I) 𝛼 y. The Sobolev norm of y is the higher-order regularization with an addition of the semi-definite perturbation matrix 𝜖I. For 𝜖 > 0, (L + 𝜖I) is always invertible even though det(L) = 0 for undirected and connected graphs. Intuitively, the value 𝜖 is related to the stability of the inverse of (L + 𝜖I).

For the Laplacian matrix L, we know that:

𝜅(L) = |𝜆 max (L)| |𝜆 min (L)| ≈ 𝜆 max (L) 0 → ∞, (3.4)
where 𝜅(L) is the condition number of L. Since 𝜅(L) → ∞, we have a bad-conditioned problem when relying on the Laplacian matrix alone. On the other hand, for the Sobolev term, we have that:

L + 𝜖I = UΛU T + 𝜖I = U(Λ + 𝜖I)U T . (3.5)
Therefore, 𝜆 min (L + 𝜖I) = 𝜖, i.e., L + 𝜖I is positive definite (L + 𝜖I ≻ 0), and:

𝜅(L + 𝜖I) = |𝜆 max (L + 𝜖I)| |𝜆 min (L + 𝜖I)| = 𝜆 max (L) + 𝜖 𝜖 < 𝜅(L) ∀ 𝜖 > 0. (3.6)
Namely, L + 𝜖I has a better condition number than L, where 𝜖 is fundamentally related to how well conditioned is the variational problem.

Since (L + 𝜖I) 𝛼 is always invertible for 𝛼 > 0 and 𝜖 > 0, one can show that the semi-supervised learning problem in (3.3) has a closed-form solution given by:

((L + 𝜖I) -1 ) 𝛼 M T (M((L + 𝜖I) -1 ) 𝛼 M T ) -1 Y(𝒮), (3.7)
where Y(𝒮) is the sub-matrix of Y with rows indexed by 𝒮. The proof is shown in Appendix C. For small values of 𝑁 , the minimization of the Sobolev norm can be solved with the closed-form solution in (3.7). For larger values of 𝑁 , this minimization can be achieved with iterative methods such as the interior-point method.

Minimization of the Total Variation

Another reconstruction algorithm for semi-supervised learning in this chapter is based on the Total Variation (TV) of graph signals. TV of graph signals is defined as [START_REF] Mahmood | Adaptive graphbased total variation for tomographic reconstructions[END_REF]:

‖y‖ TV = ∑ 𝑖∈𝒱 ∑ 𝑗∈𝒩 𝑖 √ A(𝑖, 𝑗)‖y(𝑗) -y(𝑖)‖ 1 . (3.8)
The minimization of the TV of y is related to the cluster assumption [START_REF] Jung | Semi-supervised learning in network-structured data via total variation minimization[END_REF], and end ups in piecewise constant signals. The semi-supervised algorithm solves the following optimization problem:

arg min z𝑞 ‖z 𝑞 ‖ TV s.t. Mz 𝑞 = y 𝑞 (𝒮), (3.9)
for 𝑞 = 1, . . . , 𝑄. Basically, Eqn. (3.9) is minimizing the TV of the reconstructed graph signal such that Mz 𝑞 = y 𝑞 (𝒮). The minimization of the TV in (3.9) involves a non-differentiable objective function, which discards any gradient descent method.

In the current chapter, the minimization of the TV is solved with a primal dual approach [START_REF] Chambolle | A first-order primal-dual algorithm for convex problems with applications to imaging[END_REF]. Let P ∈ R |ℰ|×𝑁 be the incidence matrix of 𝐺 defined as:

P(𝑒, 𝑖) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ √ A(𝑒) if 𝑖 = min{𝑖, 𝑗} - √ A(𝑒) if 𝑖 = max{𝑖, 𝑗} 0 otherwise, (3.10)
where 𝑖 ∈ 𝒱 and 𝑒 ∈ ℰ (in this case the edges are represented by the numbers {1, . . . , |ℰ|}). The matrix P allows to represent the TV of the graph signal as ‖y‖ TV = ‖Py‖ 1 [START_REF] Jung | Semi-supervised learning in network-structured data via total variation minimization[END_REF]. As a consequence, Eqn. (3.9) can be formulated as the primal equivalent unconstrained convex optimization problem:

arg min z𝑞 𝑔(Pz 𝑞 ) + ℎ(z 𝑞 ), (3.11)
𝑔(x) ≜ ‖x‖ 1 , and ℎ(z 𝑞 ) ≜ ⎧ ⎪ ⎨ ⎪ ⎩ ∞ if z 𝑞 / ∈ 𝒬 0 if z 𝑞 ∈ 𝒬, (3.12)
where ℎ(z 𝑞 ) is the indicator function and [START_REF] Boyd | Convex optimization[END_REF].

𝒬 = {z 𝑞 ∈ R 𝑁 : Mz 𝑞 = y 𝑞 (𝒮)}. Definition 3.3.2. Let 𝑓 : R 𝑛 → R. The function 𝑓 ′ : R 𝑛 → R is the convex conjugate such that 𝑓 ′ (x) ≜ sup x x T x -𝑓 (x)
The dual problem associated with the minimization of the TV is given by arg max x -ℎ ′ (-P T x) -𝑔 ′ (x), where ℎ ′ is the convex conjugate of the function ℎ. The mini-mization of the TV is solved with the primal-dual approach of first order [START_REF] Chambolle | A first-order primal-dual algorithm for convex problems with applications to imaging[END_REF] using the Unlocbox toolbox [START_REF] Perraudin | UNLocBoX a Matlab convex optimization toolbox using proximal splitting methods[END_REF]. For further details, the readers are referred to the references [START_REF] Jung | Semi-supervised learning in network-structured data via total variation minimization[END_REF][START_REF] Perraudin | UNLocBoX a Matlab convex optimization toolbox using proximal splitting methods[END_REF][START_REF] Komodakis | Playing with duality: An overview of recent primal-dual approaches for solving large-scale optimization problems[END_REF].

GraphMOS in a Nutshell

GraphMOS is a transductive algorithm because they require that all nodes in the graph are present to solve the semi-supervised learning problem. Our algorithm is introduced with a batch method for superpixel segmentation with parameters batch size 𝜂, and ground-truth size 𝜒, where 𝜂 is the number of frames that will be processed on each iteration, and 𝜒 is the number of frames with ground-truth that will be randomly selected from other sequences (unseen scheme) to solve the semi-supervised learning problem. Algorithm 1 shows the pseudo-code for GraphMOS with batch processing. GraphMOS can also be solved using the whole dataset to construct the graph, but in this case, our algorithm will select a subset of nodes 𝒮 ∈ 𝒱 with an unseen scheme to construct the graph signal Y. Notice that in the case of processing the whole dataset the parameter 𝜂 is not required, but one has to use instance or semantic segmentation methods to reduce the number of nodes. Also, notice that the parameters 𝜖 and 𝛼 are not required when solving the semi-supervised learning problem with TV minimization.

Experimental Framework

This section introduces the datasets used in the current chapter, the evaluation metrics, the experiments, and the implementation details of GraphMOS.

Datasets

Our proposed algorithm is evaluated on a variety of datasets for MOS. More specifically, we evaluated the performance of GraphMOS for the sequences taken from static and moving cameras on several background modeling challenges.

Algorithm 1 Graph Moving Object Segmentation

Input: Video sequences in the dataset. Initialization: Parameters 𝜁, 𝑘, 𝜖, 𝛼, 𝜂, 𝜒.

1: Select 𝑧th video to be segmented 2: while there are still frames to process in video 𝑧 do 3:

Select 𝜂 frames, in order, from 𝑧 for batch processing

4:

Randomly select 𝜒 frames from other videos 5:

Compute 𝜁 superpixels for the 𝜂 + 𝜒 frames 6:

Compute X from the 𝜂 + 𝜒 frames 7:

Compute Y from the 𝜒 ground-truth frames 8:

Find the set of 𝑘-NN of x 𝑖 ∀ 𝑖 ∈ 𝒩 9:

Compute 𝜎 = 1 |ℰ| ∑ (𝑖,𝑗)∈ℰ ‖x 𝑖 -x 𝑗 ‖ 2 10:
for (𝑖, 𝑗) in the set of 𝑘-NN do 11:

A(𝑖, 𝑗) = exp (-‖x 𝑖 -x 𝑗 ‖ 2 2 /𝜎 2 ) 12:
end for 13:

D = diag(A1), L = D -A 14:
Solve either (3.3) or (3.9) for 𝑞 = 1, . . . , 𝑄

15:

Get Z as the solution of the semi-supervised problem

16:

Use Z to compute the moving objects of the 𝜂 frames 17: end while

MOS from Static Camera

For sequences taken from static camera, GraphMOS is evaluated on three datasets including CDNet2014 [10], I2R [START_REF] Li | Foreground object detection from videos containing complex background[END_REF], and SBI2015 [START_REF] Maddalena | Towards benchmarking scene background initialization[END_REF]. CDNet2014 is the reference dataset in the MOS community. This dataset is categorized into 11 main challenges including, bad weather, low frame rate, night videos, turbulence, baseline, dynamic backgrounds, PTZ, camera jitter, intermittent object motion, shadow, and thermal.

PTZ category presents sequences taken from PTZ cameras, while the camera jitter category contains sequences of jittering effects. Each challenge contains from four up to six videos. Every video contains a certain amount of ground-truth frames, in which the ground truth shows the foreground and background. The average resolution of the sequences is 320 × 240.

I2R dataset contains nine challenging sequences with an average resolution of 240 × 192. Each sequence contains only 20 images of ground-truth. SBI2015 dataset contains 14 sequences with an average resolution of 328 × 246. The dataset presents the challenges of cluttered moving objects and slow object motion. This dataset was originally designed to evaluate background initialization methods. However, Wang et al. provided the ground-truth for MOS for SBI2015 [START_REF] Wang | Interactive deep learning method for segmenting moving objects[END_REF].

MOS for Moving Camera

GraphMOS is also evaluated on the UCSD dataset [START_REF] Mahadevan | Spatiotemporal saliency in dynamic scenes[END_REF] containing 18 challenging moving camera sequences. This dataset also presents severe dynamic background variations. Each video of UCSD is partially or fully annotated with pixel-level groundtruth images of foreground and background. The average resolution of the dataset is 230 × 320.

Evaluation Metrics

The F-measure metric is used for the MOS task to compare the performance of our algorithm with state-of-the-art methods [10]. The F-measure metric is defined as follows:

F-measure = 2 Precision × Recall Precision + Recall , where (3.13)
Recall = TP TP + FN , Precision = TP TP + FP , (3.14)
where TP, FP, and FN are the number of true positives, false positives, and false negatives pixels, respectively. [START_REF] Anis | Efficient sampling set selection for bandlimited graph signals using graph spectral proxies[END_REF], i.e., the GFT of y has an exponentially decaying shape. In this experiment, we compute the bandwidth as the 𝜌 where we have at least 90% of the spectral energy of y. The spectral energy is defined as

Experiments

∑ 𝑖∈𝒱 ŷ2 (𝑖).
As a consequence of this approximation, the classification error can be bounded by some value 𝜑 [52]. The last experiment computes the classification error in the nonsampled nodes for the sample sizes in the set 𝑚 = {10, 20, 30, . . . , 400}, using random sampling and the Sobolev minimization with 𝜖 = 0.2 and 𝛼 = 1. This experiment is performed with a Monte Carlo cross-validation with 200 repetitions.

Parameters Settings

Our algorithms contain several parameters such as the number of superpixels 𝜁, the number of k neighbors for k -NN in the graph construction, the parameters 𝜖 and 𝛼 for the Sobolev norm, the batch size 𝜂, and the number of selected ground-truth frames 𝜒. Table 3.1 shows a summary of the best parameters for GraphMOS within each dataset in the current work. Table 3.9 shows an ablation study for the parameter 𝜁.

Implementation Details

The instance and semantic segmentation algorithms were implemented using Pytorch and Detectron2 [START_REF] Wu | Detectron2[END_REF]. The algorithm for the reconstruction of graph signals was implemented using the graph signal processing toolbox [START_REF] Perraudin | GSPBOX: A toolbox for signal processing on graphs[END_REF] and the Matlab convex optimization toolbox [START_REF] Perraudin | UNLocBoX a Matlab convex optimization toolbox using proximal splitting methods[END_REF]. The code has been made available2 . For the comparison with our algorithm, most of the MOS methods were implemented with the BGSLibrary [START_REF] Sobral | BGS library: A library framework for algorithm's evaluation in foreground/background segmentation[END_REF] and the LRSLibrary [START_REF] Sobral | LRSLibrary: Low-rank and sparse tools for background modeling and subtraction in videos[END_REF] with default parameters by reference.

The

Results and Discussion

GraphMOS is compared with 20 state-of-the-art methods including SWCD [START_REF] Isik | SWCD: a sliding window and self-regulated learning-based background updating method for change detection in videos[END_REF],

FTSG [START_REF] Wang | Static and moving object detection using flux tensor with split gaussian models[END_REF], SuBSENSE [START_REF] St-Charles | SuBSENSE: A universal change detection method with local adaptive sensitivity[END_REF], WeSamBE [START_REF] Jiang | WeSamBE: A weight-sample-based method for background subtraction[END_REF], PAWCS [START_REF] St-Charles | A self-adjusting approach to change detection based on background word consensus[END_REF], WisenetMD [START_REF] Lee | WisenetMD: motion detection using dynamic background region analysis[END_REF],

IUTIS-5 [START_REF] Bianco | Combination of video change detection algorithms by genetic programming[END_REF], SemanticBGS [START_REF] Braham | Semantic background subtraction[END_REF], BSUV-net [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF], MoG [START_REF] Stauffer | Adaptive background mixture models for real-time tracking[END_REF], DECOLOR [START_REF] Zhou | Moving object detection by detecting contiguous outliers in the low-rank representation[END_REF], ViBe [START_REF] Barnich | ViBe: A universal background subtraction algorithm for video sequences[END_REF], 3WD [START_REF] Oreifej | Simultaneous video stabilization and moving object detection in turbulence[END_REF], GRASTA [START_REF] He | Incremental gradient on the grassmannian for online foreground and background separation in subsampled video[END_REF], FgSegNet v2 [START_REF] Lim | Learning multi-scale features for foreground segmentation[END_REF], ROSL [START_REF] Shu | Robust orthonormal subspace learning: Efficient recovery of corrupted low-rank matrices[END_REF], ADMM [START_REF] Parikh | Proximal algorithms[END_REF], non-cvxRPCA [START_REF] Kang | Robust PCA via nonconvex rank approximation[END_REF], and OR1MP [START_REF] Wang | Orthogonal rankone matrix pursuit for low rank matrix completion[END_REF] for the MOS task. Mask R-CNN can directly solve the MOS problem. However, Mask R-CNN does not include temporal information, leading to an ill-conditioned problem. In our experiments, we have also trained

Mask R-CNN directly on CDNet2014 sequences with an unseen scheme. Each Mask R-CNN is trained with 1000 epochs using learning rate of 0.00025 and a batch size of 2.

Qualitative Evaluations

Quantitative Results

Tables 3.2-3.5 show the comparisons of the qualitative results of the GraphMOS algorithm on CDNet2014, I2R, SBI2015, and UCSD datasets for MOS. Within these GraphMOS achieved significantly high performance for these challenges.

In the I2R dataset (Table 3.3), GraphMOS has obtained the best performance of 91.96% overall, which is almost 10.0% better than the second-best performing show clutter background scenes and slowly moving foreground objects. Therefore, the majority of the compared methods are not able to handle the overwhelming outliers of the moving objects in these sequences efficiently as compared to GraphMOS.

Similarly, in the case of the UCSD dataset (Table 3.5), GraphMOS also shows the best performance of 75.95%. The sequences in this dataset are taken from a moving camera. Therefore, it can be noticed in the performance comparison that none of the compared methods can perform favorably better since state-of-the-art methods such as MoG, DECOLOR, ViBe, GRASTA, SuBSENSE, and ADMM are designed to handle the static camera sequences. Our proposed GraphMOS algorithm shows better performance because of its generalization capabilities to tackle unseen videos on both static and moving cameras.

Ablation Studies

Several ablation studies are performed to analyze the performance of our proposed algorithm. These ablations include the analysis of different segmentation methods used in graph construction, feature extraction, the number of superpixels used in the SLIC method, the construction of the graph, and the semi-supervised learning method.

Segmentation Methods for Graph Nodes Table 3.6 shows the performance comparison of GraphMOS for I2R, CDNet2014, SBI2015, and UCSD datasets using different segmentation methods for node representation during graph construction. Overall, the superpixel segmentation-based node representation for graph construction achieves the best performance as compared to other heavyweight semantic and instance segmentation methods. This is because the superpixel method segments all the homogeneous regions in the video frames, and then the graph is constructed for the semi-supervised learning task. In contrast, the instances of the videos such as moving objects, static objects, or other undesirable objects may or may not be segmented accurately by the heavyweight deep learning DeepLab and Mask R-CNN methods. Besides, the degradation in the performance of DeepLab with respect to the instance segmentation methods also suggests the unsuitability of semantic segmentation to solve the MOS problem.

Features Analysis

To analyze the effectiveness of the proposed algorithms, we also compare the MOS results with different features extracted from graph nodes on five different datasets.

Table 3.8 shows the performance comparisons of the proposed algorithms using different features representation. The deep features are extracted from the 4th and 5th convolutional layers of the VGG-m model [START_REF] Chatfield | Return of the devil in the details: Delving deep into convolutional nets[END_REF]. Overall, the deep features show competitive performance as compared to handcrafted features, while the incorporation of both deep features (Conv-5) and handcrafted features further improves the average F-measure score in all datasets.

Number of Superpixels

Table 3.9 shows the performance comparison of the proposed algorithms with varying number of superpixels. The best performance for each dataset is given by 200 or 300 superpixels per image.

Construction of the Graph

Table 3. of these sequences since there is not a smooth change between frames, and perhaps the optical flow features do not work well in this challenge. As a consequence, the algorithm may take advantage of the local information of similar sequences in the same challenge. This is opposite to the idea to leverage global information to reduce the sample complexity in GraphMOS.

Semi-supervised Learning Table 3.11 shows the performance of GraphMOS with several configurations of the semi-supervised learning algorithm. The results suggest that the minimization of the TV is better in the challenges: camera jitter, low frame rate, PTZ, and shadow; while the minimization of the Sobolev norm is better for the other challenges. Both Sobolev norm and TV minimization show complementary results. In a specific application one

Sample Complexity

Conclusions

In this chapter, we proposed new branch of algorithms for the tasks of MOS based on graph signal processing concepts. The pipeline of the algorithm involves segmenta- The second direction is to explore the graph signal processing concepts applied in bounding boxes for applications such as multi-object tracking. Another important direction is to study an inductive learning framework, which aims to address the problems of real-time processing [START_REF] Cioppa | Real-time semantic background subtraction[END_REF] for MOS. Further questions in these directions are: how can one use the structure of certain datasets to improve the generalization of state-of-the-art deep learning methods? How can one design an algorithm to train a neural network capable of learning from the labels and the structure of a dataset? What is the relationship between the sampling of graph signals and the problems in video analysis? Perhaps, the concepts of graph signal processing, such as active semi-supervised learning and graph convolutional networks, could lead to new developments in the field of computer vision and end-to-end architectures for video analysis with semi-supervised learning. For example, GraphMOS requires less labeled information than supervised models, and it performs better than unsupervised methods. However, the addition of new videos or nodes to the problem requires solving again the optimization problem, which makes GraphMOS unsuitable for practical applications.

In this chapter, we propose a novel semi-supervised algorithm dubbed GraphMOS Network (GraphMOS-Net) based on Graph Convolutional Networks (GCNs) [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] for the MOS problem. Similar to GraphMOS in Chapter 3, GraphMOS-Net models the instances in videos as nodes embedded in a graph. In this chapter, the instances are obtained with a Cascade Mask R-CNN [START_REF] Cai | Cascade R-CNN: High quality object detection and instance segmentation[END_REF]. The representation of the nodes is obtained with background initialization, optical flow, intensity, and texture features [START_REF] Lucas | An iterative image registration technique with an application to stereo vision[END_REF][START_REF] Ojala | Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[END_REF]. Moreover, the nodes are associated either with the class moving or static object using ground-truth information. Finally, a GCN is trained with a small percentage of labeled nodes to perform a semi-supervised learning classification [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] with an unseen scheme [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF]. After training, GraphMOS-Net can simply evaluate the trained GCN architecture if new videos are added to the problem. Therefore, GraphMOS-Net overcomes the main limitation of GraphMOS that requires solving the optimization problem (Sobolev or TV minimization in (3.3) and (3.9)) again if new nodes are added to the dataset. GraphMOS-Net outperforms some state-ofthe-art methods in several challenges of the CDNet2014 [10] and UCSD background subtraction [START_REF] Mahadevan | Spatiotemporal saliency in dynamic scenes[END_REF] datasets. The main contributions of this chapter are summarized as follows:

• We pose the problem of MOS as a binary classification problem on the graph where each node is classified into two distinct components including background and foreground using GCNs.

• We perform rigorous experiments using the CDNet2014 dataset [10] for MOS.

Our results demonstrate that GraphMOS-Net uses a limited amount of labeled data and outperforms some state-of-the-art classical and deep learning methods.

Moving Object Segmentation and Graph Convolutional Networks

GraphMOS-Net consists of several components including instance segmentation, background model initialization, features extraction, graph construction, and GCN training, as shown in Fig. 4-1. GraphMOS-Net is summarized as follows: a) we compute the instance segmentation mask of the input sequence using the Mask R-CNN method [20]; b) the instances and background model are used to derive the features to be used in the graph construction, we use texture features, motion features using optical flow estimation [START_REF] Lucas | An iterative image registration technique with an application to stereo vision[END_REF], and intensity features; c) we construct the graph where a node is assigned to each object instance represented by the group of the aforementioned features; and d) we train a GCN to classify the nodes into either background or foreground. The different components of GraphMOS-Net will be described next.

Graph Semi-supervised Learning Algorithm

GraphMOS algorithm in Chapter 3 relies on the sampling and reconstruction of graph signals [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF][START_REF] Chen | Discrete signal processing on graphs: Sampling theory[END_REF]52,[START_REF] Parada-Mayorga | Blue-noise sampling on graphs[END_REF]. As a consequence, we defined the sampled nodes (or training set in our case) in Chapter 3 as a subset of nodes 𝒮 ⊂ 𝒱 with 𝒮 = {𝑠 1 , 𝑠 2 , . . . , 𝑠 𝑚 }, where 𝑚 = |𝒮| ≤ 𝑁 is the number of sampled nodes. GraphMOS applies a variational or TV method in graphs [START_REF] Pesenson | Variational splines and Paley-Wiener spaces on combinatorial graphs[END_REF] to solve the semi-supervised learning problem. The application of these variational methods requires that all inferred nodes are present at the time of solving the optimization problem. GraphMOS-Net avoids this issue by training a GCN, and we refer to the set 𝒮 as the training set in this chapter.

The layer-wise propagation rule of GraphMOS-Net, inspired from Kipf and Welling [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] method, is given as follows:

H (𝑙+1) = 𝜎( D-1 2 Ã D-1 2 H (𝑙) W (𝑙) ) (4.1)
where à = A + I is the adjacency matrix of 𝐺 with added self-connections1 , I is the identity matrix, D is the degree matrix of Ã, W (𝑙) is the matrix of trainable weights in layer 𝑙, 𝜎(•) denotes an activation function, and H (𝑙) is the matrix of activations in layer 𝑙 such that H (0) = X is the input matrix denoting the representation of the nodes. The propagation rule in (4.1) is motivated by the first-order approximation of localized spectral filters on graphs [START_REF] Defferrard | Convolutional neural networks on graphs with fast localized spectral filtering[END_REF][START_REF] Hammond | Wavelets on graphs via spectral graph theory[END_REF]. For further details, the readers are referred to [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF].

GraphMOS-Net uses a GCN with one hidden layer to solve the semi-supervised learning problem. The forward of our model is computed using the propagation rule in (4.1) as follows:

Ȳ = 𝑓 (X, A) = softmax ( D-1 2 Ã D-1 2 ReLU ( D-1 2 Ã D-1 2 XW (0) ) W (1) ) , (4.2)
where Ȳ is the output of the GCN; W (0) ∈ R 𝐶×𝐻 is the weights of the first hidden layer such that 𝐶 is the dimension of vector x 𝑣 , i.e., 𝐶 = 853, and 𝐻 is the number of feature maps; W (1) ∈ R 𝐻×𝐹 is the weights of the output layer such that 𝐹 is the number of classes (i.e., 𝐹 = 2: background and foreground);

ReLU(•) = max(0, •)
is the rectified linear unit; and finally the softmax activation function is defined as

softmax(𝑥 𝑖 ) = 1 𝑄 exp (𝑥 𝑖 ) where 𝑄 = ∑ 𝑖 exp (𝑥 𝑖 ).
Here the softmax function is applied row-wise. We set the number of feature maps 𝐻 in the hidden layer to 16 in our model.

The loss function of this GCN is defined as the cross-entropy error over all labeled nodes:

ℒ = - ∑ 𝑖∈𝒮 𝐹 ∑ 𝑗=1 Y(𝑖, 𝑗) ln Ȳ(𝑖, 𝑗). (4.3)

Experimental Framework

Databases

In this chapter, we use the CDNet2014 [10] and the UCSD background subtraction [START_REF] Mahadevan | Spatiotemporal saliency in dynamic scenes[END_REF] datasets previously introduced in Section 3.4.1. CDNet2014 contains 11 challenges including, bad weather, low frame rate, night videos, PTZ, turbulence, baseline, dynamic background, camera jitter, intermittent object motion, shadow, and thermal.

On the other hand, UCSD contains 18 videos mainly composed of moving camera sequences, with 30 up to 246 frames each sequence.

Training, Validation, and Test Nodes

For CDNet2014, we evaluate the performance of GraphMOS-Net with a 4-fold crossvalidation strategy using a video agnostic scheme (unseen videos), i.e., the testing nodes should come from different videos regarding the training nodes. We grouped the videos in four folds as evenly as possible, using the same partition as in [START_REF] Tezcan | BSUV-Net 2.0: Spatio-temporal data augmentations for video-agnostic supervised background subtraction[END_REF].

We use the 4-fold cross-validation strategy in this chapter to have a more direct and fair comparison against [START_REF] Tezcan | BSUV-Net 2.0: Spatio-temporal data augmentations for video-agnostic supervised background subtraction[END_REF]. However, other strategies such as Monte Carlo crossvalidation could be used. For each fold, we randomly take a small percentage of the nodes in the set ℳ = {0.1%, 0.5%, 5%, 10%} for training, and 1% for validation, i.e., we use a small portion of the data to train our model. For UCSD, we train our GraphMOS-Net in a subset of nodes of CDNet2014 and we test in UCSD dataset.

For the training and validation sets we randomly choose nodes from the challenges baseline, dynamic background, shadow, and PTZ of CDNet2014. We use the Fmeasure metric as defined in Section 3.4.2 to evaluate the performance of GraphMOS-Net.

Experiments

Implementation Details

The Cascade Mask R-CNN and the GCN were implemented using Pytorch and De-tectron2 [START_REF] Zhang | ResNeSt: Split-attention networks[END_REF][START_REF] Wu | Detectron2[END_REF]. We train the GCN with: 1) Adam optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF], 2) a learning rate of 0.01, 3) weight decay of 5e-4, 4) a dropout rate of 0.5 in the first graphconvolutional layer, and 5) a maximum of 600 epochs. Similarly, we train the GCN using a learning scheduler that reduces the learning rate when the loss function has stopped improving in the validation set. Our scheduler has a reducing factor of 0.5, patience of 5 epochs, and a minimum learning rate of 1e-6 (the GCN stops the learning procedure either if we reach the maximum number of epochs or if we reach the minimum learning rate). We set the training batch to the full training set in each iteration since the whole database fits in memory.

For the comparison with GraphMOS-Net, the background subtraction algorithms MoG [START_REF] Stauffer | Adaptive background mixture models for real-time tracking[END_REF], SuBSENSE [START_REF] St-Charles | SuBSENSE: A universal change detection method with local adaptive sensitivity[END_REF], and ViBe [START_REF] Barnich | ViBe: A universal background subtraction algorithm for video sequences[END_REF] for USCD were implemented with the BGSLibrary [START_REF] Sobral | BGS library: A library framework for algorithm's evaluation in foreground/background segmentation[END_REF]; while GRASTA [START_REF] He | Incremental gradient on the grassmannian for online foreground and background separation in subsampled video[END_REF] and DECOLOR [START_REF] Zhou | Moving object detection by detecting contiguous outliers in the low-rank representation[END_REF] were implemented using

Results and Discussion

GraphMOS-Net is compared, either in CDNet2014 or UCSD, with the following state-of-the-art algorithms for MOS: SWCD [START_REF] Isik | SWCD: a sliding window and self-regulated learning-based background updating method for change detection in videos[END_REF], FTSG [START_REF] Wang | Static and moving object detection using flux tensor with split gaussian models[END_REF], SuBSENSE [START_REF] St-Charles | SuBSENSE: A universal change detection method with local adaptive sensitivity[END_REF],

PAWCS [START_REF] St-Charles | A self-adjusting approach to change detection based on background word consensus[END_REF], WisenetMD [START_REF] Lee | WisenetMD: motion detection using dynamic background region analysis[END_REF], IUTIS-5 [START_REF] Bianco | Combination of video change detection algorithms by genetic programming[END_REF], SemanticBGS [START_REF] Braham | Semantic background subtraction[END_REF], FgSegNet v2 [START_REF] Lim | Learning multi-scale features for foreground segmentation[END_REF],

BSUV-Net [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF], GraphBGS-TV2 [8], GraphBGS3 [3], MoG [START_REF] Stauffer | Adaptive background mixture models for real-time tracking[END_REF], DECOLOR [START_REF] Zhou | Moving object detection by detecting contiguous outliers in the low-rank representation[END_REF],

ViBe [START_REF] Barnich | ViBe: A universal background subtraction algorithm for video sequences[END_REF], 3WD [START_REF] Oreifej | Simultaneous video stabilization and moving object detection in turbulence[END_REF], GRASTA [START_REF] He | Incremental gradient on the grassmannian for online foreground and background separation in subsampled video[END_REF], ROSL [START_REF] Shu | Robust orthonormal subspace learning: Efficient recovery of corrupted low-rank matrices[END_REF], ADMM [START_REF] Parikh | Proximal algorithms[END_REF], noncvxRPCA [START_REF] Kang | Robust PCA via nonconvex rank approximation[END_REF],

and OR1MP [START_REF] Wang | Orthogonal rankone matrix pursuit for low rank matrix completion[END_REF]. The results of FgSegNet v2 [START_REF] Lim | Learning multi-scale features for foreground segmentation[END_REF] for CDNet2014 are reported using unseen videos for the evaluation of the network, the performance comes from Tezcan et al. paper [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF]. across almost all the videos of UCSD. In the overall performance, GraphMOS-Net comes second, lagging slightly behind GraphBGS. We believe it is a consequence of increasing the representational power when solving the semi-supervised learning problem with GCNs leading to a slightly worse generalization. Similarly, GraphMOS-Net outperforms subspace learning methods such as DECOLOR and GRASTA. Subspace learning methods generally have problems when dealing with moving camera sequences.

We can also notice the degradation of the deep learning method FgSegNet v2 compared to the original results [START_REF] Lim | Learning multi-scale features for foreground segmentation[END_REF] when tested on unseen videos in Table 4.1.

Similarly, the performance of BSUV-Net drops when applied on UCSD, these results could suggest a weakness of deep learning methods in generalization for MOS (keep in mind that BSUV-Net uses the output of a fully CNN trained in ADE20K dataset [START_REF] Zhou | Scene parsing through ADE20K dataset[END_REF] as input of the network). In Table 4.2, we evaluated the BSUV-Net model provided by the authors of [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF] that was trained in the CDNet2014. In our case, we trained GraphMOS-Net in a small subset of nodes of CDNet2014 including the challenges baseline, dynamic background, shadow, and PTZ. This suggests that GraphMOS-Net generalizes better than BSUV-Net when applied on a different dataset from which it was trained. For the sake of clarification, we did not use the ground truth of UCSD in the training procedure. GraphMOS-Net benefits from the higher modeling capacity of GCNs by improving upon the GraphBGS method as shown in Tables 4.1 and 4.2. This chapter opens some avenues for future research. One interesting topic is the study of an inductive framework [START_REF] Hamilton | Inductive representation learning on large graphs[END_REF] for GraphMOS-Net in the context of MOS.

Similarly, other GNN architectures could be explored like Graph Attention Networks (GAT) [START_REF] Veličković | Graph attention networks[END_REF] and GraphSAGE [START_REF] Hamilton | Inductive representation learning on large graphs[END_REF], among others. Chapter 5 further studies the topic of GNNs in computer vision, but in this case, we will explore the problem of weakly supervised semantic segmentation using HyperGraph Convolutional Networks (Hy-perGCNs).

Chapter 5

Hypergraph Convolutional Networks for Semantic Segmentation Semantic segmentation is an important task in computer vision with multiple applications in image, 3D, and video processing [START_REF] Giraldo | Graph moving object segmentation[END_REF][START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF][START_REF] Ronneberger | U-Net: Convolutional networks for biomedical image segmentation[END_REF]. The main objective of semantic segmentation is to classify all the pixels in the images into some predefined classes. Deep learning models have dominated the study of semantic segmentation in recent years [START_REF] Chen | DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[END_REF][START_REF] Long | Fully convolutional networks for semantic segmentation[END_REF][START_REF] Badrinarayanan | SegNet: A deep convolutional encoder-decoder architecture for image segmentation[END_REF]. However, these deep learning methods are usually very complex models containing millions of learnable parameters, and thus they require a lot of densely annotated images to perform well [START_REF] Giraldo | Graph moving object segmentation[END_REF]2].

Currently, there is an increasing interest in weakly supervised learning [START_REF] Zhou | A brief introduction to weakly supervised learning[END_REF], where the predictions are obtained with a limited amount of labels. As a result, several studies have proposed WSS methods [START_REF] Xing | Weakly supervised semantic segmentation with superpixel embedding[END_REF][START_REF] Lin | ScribbleSup: Scribble-supervised convolutional networks for semantic segmentation[END_REF][START_REF] Pu | GraphNet: Learning image pseudo annotations for weakly-supervised semantic segmentation[END_REF][START_REF] Zhang | Affinity attention graph neural network for weakly supervised semantic segmentation[END_REF], where graphical models have played a central role. In particular, GCNs have been widely explored in WSS, reaching state-of-the-art performances [START_REF] Pu | GraphNet: Learning image pseudo annotations for weakly-supervised semantic segmentation[END_REF][START_REF] Zhang | Affinity attention graph neural network for weakly supervised semantic segmentation[END_REF]. However, these methods have focused on constructing graphs from individual images using spatial information. Therefore, these models waste crucial information that can be obtained from other images in the dataset.

In the current work, we propose a new algorithm named HyperGraph Convolutional Networks for Weakly-supervised Semantic Segmentation (HyperGCN-WSS), using scribbles and clicks as weak signals.

The key idea of our algorithm is to rely on spatial information as in [START_REF] Pu | GraphNet: Learning image pseudo annotations for weakly-supervised semantic segmentation[END_REF][START_REF] Zhang | Affinity attention graph neural network for weakly supervised semantic segmentation[END_REF], as well as on structural information that can be captured from other instances in the dataset. Our algorithm uses HyperGCNs [START_REF] Bai | Hypergraph convolution and hypergraph attention[END_REF] to capture such spatial-structural information. HyperGCN-WSS is composed of 1) SLIC superpixel segmentation [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF] for node representation, 2) VGG-16 for feature extraction [START_REF] Simonyan | Very deep convolutional networks for largescale image recognition[END_REF], 3) spatial and 𝑘-NN graph construction, 4) HyperGCN [START_REF] Bai | Hypergraph convolution and hypergraph attention[END_REF] to generate pseudo-labels, and 5) DeepLabV3+ [START_REF] Chen | Encoder-decoder with atrous separable convolution for semantic image segmentation[END_REF] for semantic segmentation using the pseudo-labels as the target image. Fig. 5-1 shows the motivation of HyperGCN-WSS, where one labeled superpixel (with a scribble) is connected to another non-labeled superpixel in the dataset, allowing the propagation of information between different instances in the dataset. For clicks, we have some pixels with the ground-truth information. HyperGCN-WSS is evaluated in the PASCAL VOC 2012 dataset [START_REF] Everingham | The PASCAL visual object classes challenge: A retrospective[END_REF] for semantic segmentation using scribbles and clicks as weak signals. Our algorithm shows competitive performance against previous methods.

The main contributions of this chapter are presented as follows: 1) we propose a new algorithm for WSS, 2) we show that using HyperGCNs is better than using

Proposed Method

Preliminaries

Nodes Representation and Graph Construction

We use the SLIC superpixel segmentation [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF] method to represent the nodes in the graph 𝐺 (or 𝐺 ℎ ). Furthermore, the input feature description of each node is obtained with some pre-trained CNN. In the current chapter, we use the outputs of the 10th ReLU layer of the VGG16 [START_REF] Simonyan | Very deep convolutional networks for largescale image recognition[END_REF] (the 23th layer of the network), which contains 512 features maps. Additionally, an average pooling is performed on the superpixel regions of each feature map to obtain the feature representation, i.e., each node is represented with a 512-dimensional vector.

In the current chapter, we construct two types of graphs: 1) spatial graphs in the superpixels of each image, and 2) 𝑘-NN graphs with k = 10 on some embedding space.

Let 𝛼 be the number of images in the dataset, let 𝜉 be the number of superpixels for SLIC, and let 𝜇 be the maximum number of nodes we allow for each graph (𝜇 = 40000 in the experiments). Therefore, we construct 𝜏 = ⌊ 𝛼×𝜉 𝜇 ⌉ graphs, where we have 𝛾 = ⌊ 𝛼 𝜏 ⌉ images per graph. For the spatial graphs, we connect all the nodes that are in the neighborhood of each superpixel as shown in Fig. 5-2. Therefore, we create a block diagonal matrix with the 𝛾 adjacency matrices of each image, i.e., we have 𝛾 unconnected subgraphs for each spatial graph. For the 𝑘-NN graph, we use an embedding representation to compute the Euclidean distances. For example, these embeddings can be intermediate outputs of a GCN or a HyperGCN. The weights of the edges for the spatial and 𝑘-NN graphs are given by the Gaussian function

exp (-‖x 𝑖 -x 𝑗 ‖ 2 2 /𝜎 2 )
, where x 𝑖 is the embedding (or feature representation) of the

Graph and Hypergraph Convolutional Networks

In this chapter, we use graph convolutions and hypergraph convolutions in our Hy-perGCN architecture. For the graph convolutions, we use the model of Kipf and

Welling [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF]. For the hypergraph convolutions, we use the model of Bai et al. [START_REF] Bai | Hypergraph convolution and hypergraph attention[END_REF].

The graph convolution in [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] was previously introduced in (4.1). Furthermore, the hypergraph convolution in [START_REF] Bai | Hypergraph convolution and hypergraph attention[END_REF] is given as follows:

X (𝑙+1) = 𝜎(D -1 2 ℎ H 𝑖 W 𝑒 B -1 H T 𝑖 D -1 2 ℎ H (𝑙) W (𝑙) ), (5.1)
where H (𝑙) is the matrix of activations in layer 𝑙 (matrix of features or embeddings), W (𝑙) is the matrix of trainable weights in layer 𝑙, and 𝜎(•) is an activation function.

HyperGCN Architecture

Experiments and Results

Dataset and Evaluation Metrics

HyperGCN-WSS is evaluated on PASCAL VOC 2012 [START_REF] Everingham | The PASCAL visual object classes challenge: A retrospective[END_REF] dataset for semantic segmentation. We also use the dataset of scribbles [START_REF] Lin | ScribbleSup: Scribble-supervised convolutional networks for semantic segmentation[END_REF] and random clicks as weak signals. PASCAL VOC 2012 has 20 semantic classes and one background category.

We use the augmented version of the PASCAL dataset provided by [START_REF] Hariharan | Semantic contours from inverse detectors[END_REF], resulting in 10582 images in the training set, and 1449 images in the validation set. In this chapter, we use the training set in [START_REF] Hariharan | Semantic contours from inverse detectors[END_REF] for training and validation, and we leave the validation set as the test set. We use the mean Intersection over Union (mIoU) metric [START_REF] Everingham | The PASCAL visual object classes challenge: A retrospective[END_REF] for evaluation.

Implementation Details

HyperGCN-WSS is implemented using PyTorch with a learning rate of 0.01 and weight decay of 5e-4. Each GCL or HCL has 256 hidden units and a dropout rate of 50%. Each stage of HyperGCN-WSS is trained for a maximum of 1000 epochs using Adam [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF]. For scribbles, we use 5% of the scribbles for validation. For clicks, we use 1% of the clicks for validations. We train HyperGCN-WSS using a learning scheduler that reduces the learning rate when the loss function has stopped improving in the validation set. Our scheduler has a reducing factor of 0.5, patience of 25 epochs, and a minimum learning rate of 1e-6 (HyperGCN-WSS stops the learning procedure either if we reach the maximum number of epochs or if we reach the minimum learning rate).

The final activation of the Multi-layer Perceptrons are logarithmic softmax, and we use the negative log-likelihood as loss functions.

Experiments

In this chapter, we perform experiments in 1) the dataset of scribbles [START_REF] Lin | ScribbleSup: Scribble-supervised convolutional networks for semantic segmentation[END_REF] and 2) some random clicks that are given by a percentage of 𝑁 . The percentage of clicks is given by the set ℳ = { 1 32 , 1 16 , 1 8 , 1 4 }. For example, for 1 32 ∈ ℳ and 𝜉 = 100 number of superpixels, we have around 3.125 random clicks per image for training.

Similarly, we analyze the impact of the number of superpixels 𝜉 in the set 𝒮 = {50, 100, 200, 400, 800}, for both scribbles and clicks. We report the mIoU of the pseudo-labels in the training set for each loss function ℒ 𝑖 ∀ 𝑖 ∈ {1, 2, 3} to assess the propagation of information of our algorithm after each stage. We also report the mIoU after the DeepLab training in the validation set. We do not perform an extensive search of semantic segmentation models like in [START_REF] Pu | GraphNet: Learning image pseudo annotations for weakly-supervised semantic segmentation[END_REF][START_REF] Tang | Normalized cut loss for weakly-supervised CNN segmentation[END_REF] because that is not the scope of this chapter.

Results and Discussions

Fig. 5-4 shows some visual results of HyperGCN-WSS before and after applying Conditional Random Field (CRF) [START_REF] Krähenbühl | Efficient inference in fully connected CRFs with Gaussian edge potentials[END_REF]. CRFs are commonly applied as a postprocessing step to improve the performance of image segmentation methods, as in [START_REF] Chen | DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[END_REF][START_REF] Lin | ScribbleSup: Scribble-supervised convolutional networks for semantic segmentation[END_REF][START_REF] Pu | GraphNet: Learning image pseudo annotations for weakly-supervised semantic segmentation[END_REF]. We employ CRFs in this chapter for visualization purposes, aiming to have a visual reference regarding previous methods. Similarly, Table 5.1 shows the mIoU of the pseudo-labels after each loss function ℒ 𝑖 ∀ 𝑖 ∈ {1, 2, 3} in the training Table 5.1: Accuracy in mIoU (%) in the train set of PASCAL VOC after each loss function ℒ 𝑖 ∀ 𝑖 ∈ {1, 2, 3} in our algorithm. set of PASCAL VOC with scribbles and clicks. Notice that we do not use the full labeled annotation of the training set of PASCAL VOC, so Table 5.1 shows how well the information is propagated to the other nodes in the graph. We notice that there is a gap in performance between using the GCN alone and the HyperGCN. For example, in scribbles and clicks, there is a gap of around 4% between ℒ 1 and ℒ 2 . The best results for each weak signal in

Weak Signals

𝜉 = 50 𝜉 = 100 𝜉 = 200 𝜉 = 400 𝜉 = 800 ℒ 1 ℒ 2 ℒ 3 ℒ 1 ℒ 2 ℒ 3 ℒ 1 ℒ 2 ℒ 3 ℒ 1 ℒ 2 ℒ 3 ℒ 1 ℒ 2 ℒ

Conclusions

In this chapter, we introduced a new WSS algorithm called HyperGCN-WSS. Our algorithm is composed of SLIC superpixel segmentation, CNN feature extraction, hypergraph construction, a specialized HyperGCN architecture, and the DeepLabV3+ model. This new HyperGCN architecture combines graph and hypergraph convolutions. HyperGCN-WSS used spatial graphs constructed in the neighborhood of the superpixels, and 𝑘-NN graphs constructed in some embedding representation. We showed that using hypergraph convolutions is better than using graph convolutions alone. Similarly, our algorithm showed competitive performance against previous methods.

Chapters 6 and 7 delve into the machine learning part of this thesis. We focus in two problems of graph-based algorithms in machine learning: 1) Chapter 6 explores the learning of graphs for GNNs, and how to make graph convolutions more expressive without compromising scalability, and 2) Chapter 7 studies the well-known problem of over-smoothing and over-squashing in GNNs under a spectral graph theory and differential geometry approach.

67

Part II

Machine Learning

Chapter 6

Sparse Sobolev Graph Neural Networks

Introduction

In previous chapters we explored applications of graph-based algorithms in computer vision. In this chapter and in Chapter 7, we explore machine learning aspects of GNNs. In particular, this chapter explores topics on learning graphs from data, and making GNNs more expressive without compromising scalability.

Graph representation learning and its applications have gained significant attention in recent years. Notably, GNNs have been extensively studied [START_REF] Bruna | Spectral networks and locally connected networks on graphs[END_REF][START_REF] Defferrard | Convolutional neural networks on graphs with fast localized spectral filtering[END_REF][START_REF] Hamilton | Inductive representation learning on large graphs[END_REF][START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF][START_REF] Veličković | Graph attention networks[END_REF][START_REF] Wu | A comprehensive survey on graph neural networks[END_REF]. GNNs have numerous applications such as semi-supervised learning [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF][START_REF] Veličković | Graph attention networks[END_REF], point cloud semantic segmentation [START_REF] Wang | Dynamic graph CNN for learning on point clouds[END_REF][START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF], prediction of individual relations in social networks [START_REF] Uwents | Neural networks for relational learning: An experimental comparison[END_REF],

and modeling of proteins for drug discovery [START_REF] Zitnik | Predicting multicellular function through multilayer tissue networks[END_REF][START_REF] Gainza | Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning[END_REF]. Similarly, other graph techniques have been recently applied to image, video, and medical image processing applications [START_REF] Giraldo | Graph moving object segmentation[END_REF][START_REF] Egilmez | Graph-based transforms for video coding[END_REF][START_REF] Javed | Robust structural low-rank tracking[END_REF][44].

GNNs perform well in problems of strong homophily [START_REF] Zhu | Beyond homophily in graph neural networks: Current limitations and effective designs[END_REF]. The homophily of a graph is higher when neighbor nodes are in the same category or have similar features [START_REF] Zhu | Beyond homophily in graph neural networks: Current limitations and effective designs[END_REF]. However, most of the state-of-the-art techniques on GNNs focus on problems where the graph topologies are already given by the problem [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF][START_REF] Veličković | Graph attention networks[END_REF]. Therefore, we cannot control the homophily level of the datasets. Examples of applications where the graphs are naturally given by the problem include citation, social, and quantum chemistry networks [START_REF] Hu | OGB-LSC: A large-scale challenge for machine learning on graphs[END_REF]. As a consequence, interesting tasks where the graph topologies are not readily available, remain mostly unexplored [START_REF] Jiang | Semi-supervised learning with graph learning-convolutional networks[END_REF].

In this chapter, we study the inference of smooth graph topologies to promote homophily. We observe that smooth graphs increase the performance of baseline GNN architectures compared to the empirical 𝑘-NN method to construct graphs. Similarly, we propose a new GNN model that computes a cascade of higher-order filtering operations, allowing a more diverse set of functions on each layer. Our model is inspired by the Sobolev norm of GSP [START_REF] Giraldo | Reconstruction of time-varying graph signals via Sobolev smoothness[END_REF][START_REF] Pesenson | Variational splines and Paley-Wiener spaces on combinatorial graphs[END_REF]. This norm takes powers of the sparse adjacency matrix of a graph, converting it into a dense matrix quickly. The densification of the adjacency matrix results in memory and scalability problems in GNNs. Therefore, we modify the Sobolev norm using concepts of the Hadamard product between matrices (also known as the element-wise product) to maintain the sparsity of the adjacency matrix, and we dub our proposed architecture as Sparse Sobolev GNN (S-SobGNN).

We rely on the spectral graph theory [START_REF] Chung | Spectral graph theory[END_REF] and the Schur product theorem [START_REF] Horn | Matrix analysis[END_REF] to explain some properties of our filtering operation and show that this new sparse Sobolev norm satisfies the properties of vector norms. In our proposed architecture, we employ a linear combination layer at the end of each cascade of filters to select the best power functions. S-SobGNN shows good performance improvements in several applications in the presence of wide instead of deep networks. Therefore, we improve expressiveness by 1) learning smooth graphs that promote homophily and 2) computing a more diverse set of sparse graph-convolutional functions.

We evaluate the generalization capabilities of S-SobGNN in many applications including instance and node classification. First, we test S-SobGNN in semi-supervised learning tasks in several domains including, tissue phenotyping in colon cancer histology images [START_REF] Kather | Multi-class texture analysis in colorectal cancer histology[END_REF], text classification of news [START_REF] Lang | NewsWeeder: Learning to filter netnews[END_REF], activity recognition with sensors [START_REF] Anguita | A public domain dataset for human activity recognition using smartphones[END_REF], and recognition of spoken letters [START_REF] Fanty | Spoken letter recognition[END_REF]. Finally, we test S-SobGNN in the benchmark Pattern and Cluster datasets for node classification [START_REF] Dwivedi | Benchmarking graph neural networks[END_REF]. Our algorithm outperforms many state-of-the-art methods in semi-supervised learning, node classification, and MOS in graphs.

The main contributions of this chapter are summarized as follows:

1. We demonstrate that the smooth graphs show significant performance improvement compared to 𝑘-NN graphs for GNNs by promoting homophily.

2. We propose a new GNN architecture that computes a cascade of higher-order filters inspired by the Sobolev norm of GSP.

3. Several mathematical insights of S-SobGNN are introduced using the spectral graph theory [START_REF] Chung | Spectral graph theory[END_REF] and the Schur product theorem [START_REF] Horn | Matrix analysis[END_REF].

4. We perform extensive experimental evaluations on six publicly available benchmark datasets and compared S-SobGNN with 14 state-of-the-art GNN architectures with rigorous analysis. Our algorithm shows competitive results against several methods.

The rest of this chapter is organized as follows. Section 6.2 presents the related work, Section 6.3 introduces the basic concepts and the background on learning graphs from data. Section 6.4 explains the proposed GNN model. Section 6.5 presents the experimental framework and results. Finally, Section 6.6 shows the conclusions.

Related Work

Inference of Graph Topology

Many studies in GNNs assume that the graph topology is naturally given by the problem. For instance, nodes and edges in citation networks correspond to the documents and citation links [START_REF] Sen | Collective classification in network data[END_REF]. Nevertheless, learning a graph topology from data is essential when a natural choice is not readily available. Furthermore, many interesting problems, where the graph topology is not available, have been little studied in GNNs [START_REF] Jiang | Semi-supervised learning with graph learning-convolutional networks[END_REF].

Learning (or inferring) graphs from data is an emerging field of study in GSP [START_REF] Dong | Learning graphs from data: A signal representation perspective[END_REF].

The inference of a meaningful graph is crucial for the further processing steps. In the literature, the 𝑘-NN plus Gaussian kernel method is a typical procedure to construct a graph [START_REF] Shuman | The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF]. However, some authors have proposed more meaningful methods. For example, Kalofolias showed in 2016 that the typical 𝑘-NN procedure is a particular case of a more general method of learning smooth graphs from data, where the smoothness is measured with a function of the Laplacian [START_REF] Kalofolias | How to learn a graph from smooth signals[END_REF]. Later, other studies, including [START_REF] Dong | Learning Laplacian matrix in smooth graph signal representations[END_REF] and [START_REF] Kalofolias | Large scale graph learning from smooth signals[END_REF], have also used the intuition of learning smooth graphs using the Laplacian concepts with different optimization procedures. Interested readers can explore more details about learning graphs from data in a recent survey [START_REF] Dong | Learning graphs from data: A signal representation perspective[END_REF]. In our proposed algorithm, we explore the inference of smooth graphs in GNNs [START_REF] Kalofolias | How to learn a graph from smooth signals[END_REF]. We show that the learned smooth graphs improve the performance of S-SobGNN and other state-of-the-art methods by promoting homophily.

Graph Neural Networks

The study of graphs is a well-established field in deep learning and mathematics [START_REF] Chung | Spectral graph theory[END_REF][START_REF] Bronstein | Geometric deep learning: going beyond euclidean data[END_REF]. With its emergence in 2014 and motivated by the success of CNNs in regular-structured data like images [START_REF] Lecun | Deep learning[END_REF], Bruna et al. proposed the first modern GNN by extending the convolutional operator of CNNs to graphs [START_REF] Bruna | Spectral networks and locally connected networks on graphs[END_REF]. Afterward, Defferrard et al. used the concepts of GSP to propose localized spectral filtering [START_REF] Defferrard | Convolutional neural networks on graphs with fast localized spectral filtering[END_REF] in 2016. Subsequently, Kipf and Welling approximated the filtering operation of spectral filtering to perform efficient convolution operations on graphs [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] in 2017.

Other major GNNs works include the study of inductive representation learning on graphs [START_REF] Hamilton | Inductive representation learning on large graphs[END_REF] in 2017 and the development of GATs [START_REF] Veličković | Graph attention networks[END_REF] in 2018. A complete review of GNNs can also be explored in the survey [START_REF] Wu | A comprehensive survey on graph neural networks[END_REF].

In this chapter, we introduce a new GNN architecture inspired by the Sobolev norm of GSP as introduced in Definition 3.3.1. This norm adds a matrix 𝜖I to the Laplacian, where 𝜖 ∈ R + . Then, the Sobolev norm takes powers of the modified Laplacian matrix. The notion of taking powers of the matrix that represents the graph has been previously explored in GNNs [START_REF] Abu-El-Haija | MixHop: Higher-order graph convolutional architectures via sparsified neighborhood mixing[END_REF][START_REF] Rossi | SIGN: Scalable inception graph neural networks[END_REF]; however, the powers of the adjacency matrix (or Laplacian) quickly become a dense matrix, resulting in scalability and memory problems. As a result, we modify the Sobolev norm using the Hadamard product between matrices to keep the sparsity level of the adjacency matrix.

Learning Graphs from Data

Preliminaries

Given a set of classes for the vertices 𝒱, the homophily captures the property of a node to have the same class as its neighbors. Pei et al. [START_REF] Pei | Geom-GCN: Geometric graph convolutional networks[END_REF] proposed a metric to measure the level of homophily in a graph as follows:

𝐻(𝐺) = 1 |𝒱| ∑ 𝑖∈𝒱 |𝒩 𝑠 𝑖 | |𝒩 𝑖 | , (6.1)
where 𝒩 𝑖 is the set of vertices connected to the 𝑖th node by only one edge, and 𝒩 𝑠 𝑖 is the set of nodes that share the same labels as the 𝑖th node. 𝐻(𝐺) → 1 when we have graphs with strong homophily, while 𝐻(𝐺) → 0 for graphs with strong heterophily.

As a consequence, low homophily means high heterophily and vice versa.

Inferring Smooth Graphs

We use the smoothness as a prior assumption to infer the graphs in this chapter.

However, we need to extend the definition of the Laplacian quadratic form in (2.5) to multiple-dimensional signals to learn graphs from data. Let X ∈ R 𝑁 ×𝑀 be the data matrix of 𝑁 instances, where each instance is an 𝑀 -dimensional vector and

X = [x 1 , x 2 , . . . , x 𝑁 ] T .
x 𝑣 ∈ R 𝑀 could be any meaningful 𝑀 -dimensional feature representation of the 𝑣th instance in the problem we are approaching. For example, in image classification, this representation could be the features from some CNN.

We can also think of X as a set of 𝑀 one-dimensional graph signals such that X =

[x ′ 1 , x ′ 2 , . . . , x ′ 𝑀 ]
, where x ′ 𝑖 ∈ R 𝑁 is the 𝑖th column vector of X. The extension of the graph Laplacian quadratic form in Eq. (2.5) to multiple dimensions for X in 𝐺 is given as follows [START_REF] Kalofolias | How to learn a graph from smooth signals[END_REF]:

1 2 ∑ 𝑖,𝑗 A(𝑖, 𝑗)‖x 𝑖 -x 𝑗 ‖ 2 = tr(X T LX). (6.2)
When we infer graphs based on the minimization of Eq. ( 6.2), we end up with the smoothest representation in the columns of X. As a consequence, we refer to Eq. (6.2) as the smoothness of X. Eq. (6.2) has been used in several studies to infer the topology of 𝐺 [START_REF] Dong | Learning graphs from data: A signal representation perspective[END_REF][START_REF] Kalofolias | How to learn a graph from smooth signals[END_REF][START_REF] Dong | Learning Laplacian matrix in smooth graph signal representations[END_REF]. Notice that tr(X T LX) = ∑ 𝑀 𝑖=1 x ′T 𝑖 Lx ′ 𝑖 in Eq. (6.2), i.e., the smoothness of the data matrix X is the sum of all graph Laplacian quadratic forms of each graph signal x ′ 𝑖 ∀ 𝑖 ∈ {1, . . . , 𝑀 }. Furthermore, let Z ∈ R 𝑁 ×𝑁 + be the pairwise distances matrix of X such that Z(𝑖, 𝑗) = ‖x 𝑖 -x 𝑗 ‖ 2 2 ∀ 1 ≤ 𝑖, 𝑗 ≤ 𝑁 . We thus can rewrite (6.2) as follows [START_REF] Kalofolias | How to learn a graph from smooth signals[END_REF]:

tr(X T LX) = 1 2 tr(AZ) = 1 2 ‖A ∘ Z‖ 1,1 . (6.3)
Kalofolias [START_REF] Kalofolias | How to learn a graph from smooth signals[END_REF] proposed to infer a smooth topology of graphs using the following optimization function:

arg min A∈𝒜 ‖A ∘ Z‖ 1,1 -𝛾1 T log(A1) + 𝛽 2 ‖A‖ 2 𝐹 , (6.4)
where 𝒜 is the set of valid adjacency matrices (symmetric, with zero diagonal, and non-negative), and the parameters 𝛾 > 0 and 𝛽 ≥ 0 control the connectivity of the graph. The term ‖A ∘ Z‖ 1,1 in Eq. (6.4) is associated with the smoothness of the data matrix, and therefore we are looking for the smoothest representation out of X. Moreover, the second and third terms of Eq. (6.4) prevent the trivial solution and control sparsity. The optimization problem in Eq. (6.4) can be solved using primal-dual techniques [START_REF] Komodakis | Playing with duality: An overview of recent primal-dual approaches for solving large-scale optimization problems[END_REF].

Reducing Hyperparameters

One can relate the parameters 𝛾 and 𝛽 of Eq. (6.4) with the number of 𝑘 neighbors of each node as follows:

Proposition 1 (Kalofolias and Perraudin [START_REF] Kalofolias | Large scale graph learning from smooth signals[END_REF]). Let A * (Z, 𝛾, 𝛽) be the solution of Eq. (6.4) for the matrix of distances Z and parameters 𝛾, 𝛽 > 0. Therefore, the same solution can be achieved with parameters 𝛾 = 1 and 𝛽 = 1, by multiplying Z by 𝜃 = 1/ √ 𝛾𝛽 and the resulting A * by 𝛿 = √ 𝛾/𝛽.

Proof: see [START_REF] Kalofolias | Large scale graph learning from smooth signals[END_REF].

In some applications, the multiplication of A * by a constant 𝛿 does not change the functionality of the adjacency matrix, leading to a reduction in the number of hyperparameters. Similarly, we can find two bounds for the parameter 𝜃 to get approximately 𝑘 edges per node as follows [START_REF] Kalofolias | Large scale graph learning from smooth signals[END_REF]:

𝑁 ∑ 𝑗=1 1 𝑁 √ 𝑘 Z2 (𝑘 + 1, 𝑗) -C(𝑘, 𝑗) Z(𝑘 + 1, 𝑖) < 𝜃 𝑘 , 𝜃 𝑘 ≤ 𝑁 ∑ 𝑗=1 1 𝑁 √ 𝑘 Z2 (𝑘, 𝑗) -C(𝑘, 𝑗) Z(𝑘, 𝑗) , (6.5)
where Z is obtained by sorting each column of Z in increasing order, C(𝑘, 𝑗) = ∑ 𝑘 𝑖=1 Z(𝑖, 𝑗) ∀ 1 ≤ 𝑘, 𝑗 ≤ 𝑁 , and 𝜃 𝑘 is the parameter to scale the matrix Z and get approximately 𝑘 edges per node in A.

In this chapter, we learn the smooth graph as follows:

arg min A∈𝒜 ‖A ∘ 𝜃 𝑘 Z‖ 1,1 -1 T log(A1) + 1 2 ‖A‖ 2 𝐹 . (6.6)
The solution of Eq. (6.6) is a smooth adjacency matrix A with approximately 𝑘 edges per node, promoting homophily [START_REF] Zhu | Beyond homophily in graph neural networks: Current limitations and effective designs[END_REF]. For example, the smooth graph in Fig.

6-1 has a clear cluster structure compared to the 𝑘-NN graph. In this chapter, we feed S-SobGNN and other state-of-the-art methods with smooth graphs learned from Eq. (6.6) and with 𝑘-NN graphs to assess their performance on graph representation learning tasks.

Sparse Sobolev Graph Neural Networks

Sobolev Norm

The Sobolev norm in GSP, introduced in Definition 3.3.1, has been used as a regularization term to solve problems in 1) video processing [START_REF] Giraldo | Graph moving object segmentation[END_REF]3], 2) modeling of infectious diseases [START_REF] Giraldo | On the minimization of Sobolev norms of time-varying graph signals: Estimation of new Coronavirus disease 2019 cases[END_REF], and 3) interpolation of graph signals [START_REF] Giraldo | Reconstruction of time-varying graph signals via Sobolev smoothness[END_REF][START_REF] Pesenson | Variational splines and Paley-Wiener spaces on combinatorial graphs[END_REF].

When L is symmetric in the Sobolev norm in Definition 3.3.1, we have that ‖x‖ 2 𝜌,𝜖 can be rewritten as follows:

‖x‖ 2 𝜌,𝜖 = x T (L + 𝜖I) 𝜌 x. (6.7)
The analysis of Eq. (6.7) is divided into two steps: 1) when 𝜖 = 0, and 2) when

𝜌 = 1.
Let U be an orthonormal matrix U T U = I, and for 𝜖 = 0 in Eq. (6.7) we have:

x T L 𝜌 x = x T UΛ 𝜌 U T x = xT Λ 𝜌 x = 𝑁 ∑ 𝑖=1 x2 (𝑖)𝜆 𝜌 𝑖 . (6.8)
Notice that the spectral components x(𝑖) are penalized with powers of the eigenvalues 𝜆 𝜌 𝑖 of L. Since the eigenvalues are ordered in increasing order, the higher frequencies of x are penalized more than the lower frequencies when 𝜌 = 1, leading to a smooth function in 𝐺. For 𝜌 > 1, the GFT x is penalized with a more diverse set of eigenval-ues. We can have a similar analysis for the adjacency matrix A using the eigenvalue decomposition A 𝜌 = (VΣV H ) 𝜌 = VΣ 𝜌 V H , where V is the matrix of eigenvectors, and Σ is the matrix of eigenvalues of A. In the case of A, the GFT x = V H x.

For the second analysis of Eq. (6.7), for 𝜌 = 1, we have:

‖x‖ 2 𝜌,𝜖 = x T (L + 𝜖I)x. (6.9)
The term (L + 𝜖I) in Eq. ( 6.9) is associated with a better condition number1 than using L alone. Better condition numbers can be associated with faster convergence rates in gradient descent methods, as shown in [START_REF] Giraldo | On the minimization of Sobolev norms of time-varying graph signals: Estimation of new Coronavirus disease 2019 cases[END_REF]. For the Laplacian matrix L, we know that 𝜅(L) → ∞ from 3.4, i.e., we have a bad-conditioned problem when relying on the Laplacian matrix alone. On the other hand, for the Sobolev term, we have that: Namely, L + 𝜖I has a better condition number than L. It might not be evident why a better condition number could help in GNNs, where the inverses of the Laplacian or adjacency matrices are not required to perform the propagation rules. However, some authors have seen the adverse effects of bad-behaved matrices. For example, Kipf and Welling [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] used a renormalization trick such that A + I is used in their filtering operation to avoid exploding/vanishing gradients. Similarly, Wu et al. [START_REF] Wu | Simplifying graph convolutional networks[END_REF] showed that adding the identity matrix to A shrinks the graph spectral domain, resulting in a low-pass-type filter.

L + 𝜖I = UΛU T + 𝜖I = U(Λ + 𝜖I)U T . ( 6
The previous theoretical analysis shows the benefits of the Sobolev norm regarding 1) the more diverse frequency computation in Eq. (6.8), and 2) the better condition number in Eq. ( 6.11).

Sparse Sobolev Norm

The use of L or A in GNNs is computationally efficient because these matrices are usually sparse. Therefore, we can perform a small number of sparse matrix operations.

For the Sobolev norm, the term (L + 𝜖I) 𝜌 can quickly become a dense matrix for large values of 𝜌, leading to scalability and memory problems. To mitigate this problem, we use the sparse Sobolev norm to keep the same sparsity level. Let us first define the theoretical aspects of the sparse Sobolev norm, and then we will show its application to GNNs. Definition 6.4.1. Let L ∈ R 𝑁 ×𝑁 be the Laplacian matrix of a graph 𝐺. For fixed parameters 𝜖 ≥ 0 and 𝜌 ∈ N, the sparse Sobolev term for GNNs is introduced as the 𝜌 Hadamard multiplications of (L + 𝜖I) (also known as the Hadamard powers) such that:

(L + 𝜖I) (𝜌) = (L + 𝜖I) ∘ (L + 𝜖I) ∘ • • • ∘ (L + 𝜖I). (6.12)
For example, (L + 𝜖I) (2) = (L + 𝜖I) ∘ (L + 𝜖I). As a result, the sparse Sobolev norm is given by: Proof: See Appendix D.

‖x‖ (𝜌),𝜖 ≜ ‖(L + 𝜖I) (𝜌/2) x‖. ( 6
The sparse Sobolev term in (6.12) has the property of keeping the same sparsity level of L + 𝜖I for any value of 𝜌. Notice that (L + 𝜖I) 𝜌 is equal to the sparse Sobolev term if 1) we restrict 𝜌 to be in N, and 2) we replace the common matrix multiplication with the Hadamard product. The theoretical properties of the Sobolev norm in (6.8) and (6.11) do not extend trivially to its sparse counterpart. However, we can develop some theoretical insights using concepts of Kronecker products and the Schur product theorem [START_REF] Horn | Matrix analysis[END_REF].

Lemma 6.4.2. Let L be any Laplacian matrix of a graph with eigenvalue decomposition L = UΛU T , we have that:

L ∘ L = L (2) = P T 𝑁 (U ⊗ U)(Λ ⊗ Λ)(U T ⊗ U T )P 𝑁 , (6.14)
where P 𝑁 ∈ {0, 1} 𝑁 2 ×𝑁 is a partial permutation matrix.

Proof: See Appendix E.

Eq. (6.14) is a closed-form solution regarding the spectrum of the Hadamard power for 𝜌 = 2. Thus, the spectrum of the Hadamard multiplication is a compressed form of the Kronecker product of the spectral components. The sparse Sobolev term we use in our S-SobGNN is given by (L + 𝜖I) (𝜌) so that the spectral components of the graph are changing for each value of 𝜌.

For the condition number of the Hadamard powers, we can use the Schur product theorem (in Appendix D, Theorem D.0.1). We know that (L + 𝜖I) (𝜌) ≻ 0 ∀ 𝜖 > 0 since (L + 𝜖I) ≻ 0 ∀ 𝜖 > 0, and therefore 𝜅((L + 𝜖I) (𝜌) ) < ∞. For the case of the adjacency matrix, the eigenvalues of A lie in the interval [-𝑑, 𝑑], where 𝑑 is the maximal degree of the vertices in 𝐺 (see Theorem 8.5 in [START_REF] Nica | A brief introduction to spectral graph theory[END_REF]). Therefore, we can bound the eigenvalues of A into the interval [-1, 1] by normalizing the adjacency matrix such that

A 𝑁 = D -1 2 AD -1 2 .
As a result, we know that

A 𝑁 + 𝜖I ≻ 0 ∀ 𝜖 > 1,
and

(A 𝑁 + 𝜖I) (𝜌) ≻ 0 ∀ 𝜖 > 1.
Finally, we can say that the theoretical developments of the sparse Sobolev norm hold to some extent the same theoretical developments of Section 6.4.1, i.e., a more diverse set of frequencies and a better condition number.

Graph Neural Network Architecture

Kipf and Welling [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] proposed one of the most successful yet simple GNN, called GCN. The propagation rule of GCN and the architectural details were introduced in Section 4.2.2.

In this chapter, we introduce a new filtering operation based on the sparse Sobolev term such that our propagation rule is given as follows:

B (𝑙+1) 𝜌 = 𝜎( D-1 2 𝜌 Ā𝜌 D-1 2 𝜌 H(𝑙) W (𝑙) 𝜌 ), (6.15)
where Ā𝜌 = (A+𝜖I) (𝜌) is the 𝜌th sparse Sobolev term of A, D𝜌 is the degree matrix of

Experiments and Results

The real performance of new convolutional operations in GNNs becomes illuded because authors can exhaustively fine-tune hyper-parameters and architectures in specific applications to improve accuracy. The aim to outperform other state-of-the-art methods may lead to very complex and computationally expensive models with millions of parameters [START_REF] Patterson | Carbon emissions and large neural network training[END_REF]. However, our objective is to demonstrate a new useful filter by performance comparisons in similar data regimens while keeping a comparable amount of parameters. This section presents several experiments and results for semi-supervised learning, and node classification (the benchmarking datasets).

S-SobGNN is compared with 14 GNN architectures. S-SobGNN is compared with: Chebyshev filters (Cheby) [START_REF] Defferrard | Convolutional neural networks on graphs with fast localized spectral filtering[END_REF], Topology Adaptive GCNs (TAGConv) [START_REF] Du | Topology adaptive graph convolutional networks[END_REF],

GCN [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF], GraphSage [START_REF] Hamilton | Inductive representation learning on large graphs[END_REF], MoNet [START_REF] Monti | Geometric deep learning on graphs and manifolds using mixture model CNNs[END_REF], GatedGCN [START_REF] Bresson | Residual gated graph ConvNets[END_REF], GAT [START_REF] Veličković | Graph attention networks[END_REF], k-Graph Neural Network (k-GNN) [START_REF] Morris | Weisfeiler and Leman go neural: Higher-order graph neural networks[END_REF], GIN [START_REF] Xu | How powerful are graph neural networks?[END_REF], RingGNN [START_REF] Chen | On the equivalence between graph isomorphism testing and function approximation with GNNs[END_REF], GCN via Initial residual and Identity mapping (GCNII) [START_REF] Chen | Simple and deep graph convolutional networks[END_REF], GCN plus DropEdge (GCN+DE) [START_REF] Rong | DropEdge: Towards deep graph convolutional networks on node classification[END_REF], Graph Sampling based Inductive learning (GraphSAINT) [START_REF] Zeng | GraphSAINT: Graph sampling based inductive learning method[END_REF], and SIGN [START_REF] Rossi | SIGN: Scalable inception graph neural networks[END_REF].

Implementation Details

S-SobGNN is implemented using PyTorch. The 𝑘-NN and the learned graph methods are implemented using the GSP toolbox [START_REF] Perraudin | GSPBOX: A toolbox for signal processing on graphs[END_REF] and UNLocBox [START_REF] Perraudin | UNLocBoX a Matlab convex optimization toolbox using proximal splitting methods[END_REF]. The values of 𝑘 are searched in the set {10, 20, 30, 40}, and then the value 𝜃 𝑘 in Eq. (6.6) is computed as the middle value of the interval in Eq. (6.5). The state-of-the-art GNNs are implemented with PyTorch Geometric [START_REF] Fey | Fast graph representation learning with PyTorch Geometric[END_REF]. For the semi-supervised learning experiments, the standard implementation of S-SobGNN has 1) two layers, 2) 16

hidden units in the first convolutional layer (and the output layer contains 𝐹 output units), 3) a learning rate of 0.01, 4) weight decay of 5e-4, and 5) a dropout rate of 0.5 in the first graph-convolutional layer. S-SobGNN and the other methods are trained with 500 epochs using Adam optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF]. For the benchmarking, we add some additional features following the same experimental setting as in [START_REF] Dwivedi | Benchmarking graph neural networks[END_REF]: 1) S-SobGNN has four layers, 2) we restrict 𝛼 = 2 to keep a low parameter budget, 3) we add residual connections and batch normalization to each graph convolutional layer, and 4) we add a Multi-Layer Perceptron (MLP) at the end of our architecture.

Semi-supervised Learning

Datasets: We test S-SobGNN in several semi-supervised learning tasks including, tissue phenotyping in colon cancer histology images [START_REF] Kather | Multi-class texture analysis in colorectal cancer histology[END_REF], text classification of news [START_REF] Lang | NewsWeeder: Learning to filter netnews[END_REF], activity recognition using sensors [START_REF] Anguita | A public domain dataset for human activity recognition using smartphones[END_REF], and recognition of spoken letters [START_REF] Fanty | Spoken letter recognition[END_REF]. These are semi-supervised problems because S-SobGNN uses both labeled and unlabeled data in the training procedure. We also test the smooth learned and the 𝑘-NN graph representations as inputs of the GNNs in these tasks. news from 20 classes [START_REF] Lang | NewsWeeder: Learning to filter netnews[END_REF]. In this chapter, we use a subset of 10 categories: computer graphics, windows operative system, IBM hardware, MAC hardware, autos, motorcycles, cryptography, electronics, medicine, and space. Each news is represented with the Term-Frequency Inverse-Document-Frequency (TFIDF) as in [START_REF] Gadde | Active semi-supervised learning using sampling theory for graph signals[END_REF].

Multi-class in

Human Activity Recognition (HAR): This dataset contains recordings of 30 people while they were doing activities of daily living [START_REF] Anguita | A public domain dataset for human activity recognition using smartphones[END_REF]. These people were carrying waist-mounted smartphones with embedded sensors. HAR contains six classes: standing, sitting, laying down, walking, walking downstairs, and walking upstairs. Each recording is represented as in [START_REF] Anguita | A public domain dataset for human activity recognition using smartphones[END_REF].

Isolated Letter Recognition:

The Isolet dataset has recordings from 30 speakers of the 26 letters of the English alphabet. Isolet has 6238 samples represented as 617dimensional feature vectors. The feature representation method is explained in [START_REF] Fanty | Spoken letter recognition[END_REF].

Experiments

Several graphs are inferred for Cancer-B, Cancer-M, 20News, HAR, and Isolet, using a 𝑘-NN method and the algorithm presented in Section 6.

Results

Table 6.1 shows the summary of the comparison of state-of-the-art methods and S-SobGNN. The results are computed as the best mean in the set of training percentages for all datasets without restriction in the number of parameters. S-SobGNN presents competitive results in all datasets. Furthermore, we can see that smooth learned graphs present better results than 𝑘-NN graphs. Indeed, k-GNN and GCNII improve by a large margin their results with learned graphs compared to 𝑘-NN graphs. Therefore, learned graphs present advantages when we do not have a natural choice for the construction of the graph. Notice also that learned graphs outperform the 𝑘-NN graphs in almost all cases.

Similarly, S-SobGNN shows stable results for the 𝑘 values that we tested. Table 6.5 shows the sparsity percentages of the sparse and non-sparse Sobolev norm for different values of 𝜌. We can notice how the non-sparse filter loses sparsity quickly for higher values of 𝜌.

Benchmarking GNNs in Node Classification

Datasets: S-SobGNN is evaluated in the datasets Pattern and Cluster. These are the two artificial datasets for node classification of the benchmarking for graph neural net- works [START_REF] Dwivedi | Benchmarking graph neural networks[END_REF]. Pattern and Cluster are generated with stochastic block models for node classification, and they are used to model communities in social networks. For example, Pattern dataset tests the accuracy of some method to recognize predetermined subgraphs, while Cluster evaluates the identification of clusters in a semi-supervised scheme.

Experiments

For the benchmarking, we follow the same experimental setting as in [START_REF] Dwivedi | Benchmarking graph neural networks[END_REF], i.e., we run our model four times with four different seeds.

Results

Table 6.6 shows the comparison of S-SobGNN with other methods in the benchmarking of graph neural networks for node classification [START_REF] Dwivedi | Benchmarking graph neural networks[END_REF]. S-SobGNN shows good results in Pattern and Cluster. Most importantly, S-SobGNN shows very competitive results in time per epoch (T-Epoch), where we trained our architecture in an Nvidia GTX 1070 GPU. The results of the other state-of-the-art methods are directly extracted from [START_REF] Dwivedi | Benchmarking graph neural networks[END_REF], where they trained these models with four Nvidia 2080Ti

GPUs, which are more powerful than ours. The computational-time benefits of our architecture are due to the sparsity concepts developed in this chapter. Table 6.4: Accuracy (in %) of S-SobGNN with variations of the parameters 𝜖 and 𝛼, as well as variations of the parameter 𝑘 for the 𝑘-NN and the algorithm to learn graphs (Lrnd.) in all datasets for semi-supervised learning. Cancer-B 92.83|93. [START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF] Table 6.6: Benchmarking results for several state-of-the-art methods and S-SobGNN, where L denotes the number of hidden layers. The results are averaged over four runs with four different seeds.

Conclusion

In this chapter, we inferred smooth graphs and introduced a new GNN architecture with applications in semi-supervised learning, and node classification. We observed that smooth inferred graphs improve the performance of baseline GNN architectures by promoting homophily. Furthermore, our S-SobGNN model used the Sobolev norm of GSP to compute a more diverse set of functions for GNNs. Then, we extended the concept of Sobolev norms using the Hadamard product between matrices to keep the sparsity level of the graph representations. Several theoretical notions of our filtering operation were provided in Sections 6.4.1 and 6.4.2. Our architecture was designed with a cascade of filtering operations on each layer with an attention function to select the best operations. S-SobGNN was evaluated on publicly available data including, 1)

four datasets for semi-supervised learning, and 2) two datasets for node classification.

S-SobGNN outperformed several state-of-the-art methods in all applications.

This chapter opens several research directions. The first direction could be the application of S-SobGNN in the modeling of proteins for drug discovery [START_REF] Gainza | Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning[END_REF]. Another important question is how we can further propose novel functions in the adjacency matrix to improve the expressiveness of GNNs maintaining sparsity?

Next Chapter 7 studies the relationship between the over-smoothing and oversquashing problems when stacking multiple graph convolutional layers in GNNs.

91

Chapter 7

On the Trade-off between Over-smoothing and Over-squashing in GNNs

Introduction

In the previous Chapter 6, we studied ideas to increase the expressive power of GNNs without compromising scalability. In this chapter, we study the relationship between over-smoothing and over-squashing in GNNs from a topological perspective.

Despite the successful developments and applications of GNNs, they can suffer from the typical limitations of neural networks like over-fitting, and vanishing gradients [START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF]. Similarly, GNNs have two known inherent limitations that are not fully understood yet: over-smoothing [START_REF] Li | Deeper insights into graph convolutional networks for semi-supervised learning[END_REF] and over-squashing [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF]. Both problems, oversmoothing and over-squashing, occur when we stack several graph convolutional layers trying to capture long-range dependencies among samples in the data. The accepted explanation for over-smoothing is that node representations become indistinguishable when the number of convolutional layers increases [START_REF] Li | Deeper insights into graph convolutional networks for semi-supervised learning[END_REF][START_REF] Chen | Measuring and relieving the over-smoothing problem for graph neural networks from the topological view[END_REF]. Similarly, we know that over-squashing consists of the distortion of an exponentially amount of information from distant nodes trying to pass through some edges, called "bottlenecks", in the graph [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF][START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF]. The solution to these two problems is useful in situations where we have graphs with large diameters and long-range relationships between nodes, otherwise, shallow GNNs could be enough [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF]. Several methods have been proposed to address over-smoothing [START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF][START_REF] Rong | DropEdge: Towards deep graph convolutional networks on node classification[END_REF][START_REF] Klicpera | Predict then propagate: Graph neural networks meet personalized PageRank[END_REF][START_REF] Oono | Graph neural networks exponentially lose expressive power for node classification[END_REF][START_REF] Zhao | PairNorm: Tackling oversmoothing in GNNs[END_REF][START_REF] Zhou | Towards deeper graph neural networks with differentiable group normalization[END_REF][START_REF] Chen | Iterative deep graph learning for graph neural networks: Better and robust node embeddings[END_REF][START_REF] Chien | Adaptive universal generalized PageRank graph neural network[END_REF] and over-squashing [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF][START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF]. However, both problems have been studied independently without a precise explanation concerning the connection between these issues.

In this chapter, we introduce a topological relationship between over-smoothing and over-squashing. From the side of over-smoothing, we use the random walk matrix to show how node representations exponentially converge to a stationary distribution [START_REF] Chung | Spectral graph theory[END_REF]. We observe that the convergence rate of this exponential-decaying function depends on the first non-zero eigenvalue of the Laplacian matrix of the graph. From the side of over-squashing, we rely on the work of Topping et al. [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] to show how bottlenecks (and thus over-squashing) are also closely related to the first non-zero eigenvalue of the graph. Therefore, we can use the Cheeger inequality1 [START_REF] Cheeger | A lower bound for the smallest eigenvalue of the Laplacian[END_REF] to show the underlying trade-off between over-smoothing and over-squashing, i.e., we cannot arbitrarily improve one of them without worsening the other. Likewise, we propose a heuristic algorithm to find a good approximation for the trade-off between oversmoothing and over-squashing. Our algorithm uses a bound of the Ollivier's Ricci curvature [START_REF] Ollivier | Ricci curvature of markov chains on metric spaces[END_REF] presented in [START_REF] Jost | Ollivier's Ricci curvature, local clustering and curvaturedimension inequalities on graphs[END_REF]. We name this bound Jost and Liu Curvature (JLC) and our algorithm Stochastic Jost and Liu curvature Rewiring (SJLR). JLC is used to add edges in the graph as a pre-processing step, and then some edges are dropped in the training step based on the curvature metric and the node embeddings. We rewire the graph in two different stages to alleviate over-smoothing and over-squashing, so we try to address both problems simultaneously. The JLC metric is less computationally complex than the Balanced Forman Curvature (BFC) presented in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] while keeping crucial theoretical properties. SJLR algorithm shows competitive performance in benchmarking homophilous and heterophyllous graph datasets for node classification.

The main contributions of this chapter are summarized as follows: 1) we present a topological relationship between over-smoothing and over-squashing, 2) we propose a new rewiring algorithm based on the JLC metric presented in [START_REF] Jost | Ollivier's Ricci curvature, local clustering and curvaturedimension inequalities on graphs[END_REF] to address both over-smoothing and over-squashing, 3) we perform extensive experiments showing the properties of our SJLR algorithm, and 4) we release a codebase2 based on PyG [START_REF] Fey | Fast graph representation learning with PyTorch Geometric[END_REF] for training, hyper-parameters tuning, and evaluation of methods aiming to alleviate over-smoothing or over-squashing (further details of the codebase are discussed in Appendix F). The rest of the chapter is organized as follows. Section 7.2 presents the related work. Section 7.3 introduces some preliminary concepts. Section 7.4 presents the relationship between over-smoothing and over-squashing. Section 7.5 explains the proposed SJLR algorithm. Finally, Sections 7.6 and 7.7 show the experiments and conclusions, respectively.

Related Work

Over-smoothing was first discussed in [START_REF] Li | Deeper insights into graph convolutional networks for semi-supervised learning[END_REF] to explain why node embeddings of distinct classes become indistinguishable when stacking multiple layers in GNNs. Later, several methods were proposed to ease over-smoothing [START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF][START_REF] Rong | DropEdge: Towards deep graph convolutional networks on node classification[END_REF][START_REF] Chen | Measuring and relieving the over-smoothing problem for graph neural networks from the topological view[END_REF][START_REF] Klicpera | Predict then propagate: Graph neural networks meet personalized PageRank[END_REF][START_REF] Oono | Graph neural networks exponentially lose expressive power for node classification[END_REF][START_REF] Zhao | PairNorm: Tackling oversmoothing in GNNs[END_REF][START_REF] Zhou | Towards deeper graph neural networks with differentiable group normalization[END_REF][START_REF] Chen | Iterative deep graph learning for graph neural networks: Better and robust node embeddings[END_REF][START_REF] Chien | Adaptive universal generalized PageRank graph neural network[END_REF][START_REF] Zeng | Decoupling the depth and scope of graph neural networks[END_REF][START_REF] Huang | Towards deepening graph neural networks: A GNTK-based optimization perspective[END_REF].

We can briefly classify these approaches as:

1. Graph rewiring methods: Klicpera et al. [START_REF] Klicpera | Predict then propagate: Graph neural networks meet personalized PageRank[END_REF][START_REF] Klicpera | Diffusion improves graph learning[END_REF] proposed an improved propagation scheme based on personalized PageRank for rewiring the graph as a preprocessing step. Rong et al. [START_REF] Rong | DropEdge: Towards deep graph convolutional networks on node classification[END_REF] and Huang et al. [START_REF] Huang | Towards deepening graph neural networks: A GNTK-based optimization perspective[END_REF] introduced methods that drop edges from the graph when training the GNN. Finally, Chen et al. [START_REF] Chen | Measuring and relieving the over-smoothing problem for graph neural networks from the topological view[END_REF] presented an approach that adaptively changes the graph topology.

2. Normalization techniques: Zhao and Akoglu [START_REF] Zhao | PairNorm: Tackling oversmoothing in GNNs[END_REF] and Zhou et al. [START_REF] Zhou | Towards deeper graph neural networks with differentiable group normalization[END_REF] proposed node-embeddings normalization techniques to address over-smoothing directly, i.e., they tried to avoid nodes becoming indistinguishable. Similarly, Oono

and Suzuki [START_REF] Oono | Graph neural networks exponentially lose expressive power for node classification[END_REF] presented a procedure that normalizes the weights of the GNN architecture.

3. Architectural changes: Li et al. [START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF] proposed dilated convolutions and resid-ual/dense connections in GNNs to create deep architectures. Chen et al. [START_REF] Chen | Iterative deep graph learning for graph neural networks: Better and robust node embeddings[END_REF] introduced an iteratively learning graph structure and graph embedding such that their method learns a better graph structure based on better node embeddings, and vice versa. Chien et al. [START_REF] Chien | Adaptive universal generalized PageRank graph neural network[END_REF] presented a new graph convolutional filter inspired by the graph signal processing literature [START_REF] Sandryhaila | Discrete signal processing on graphs: Frequency analysis[END_REF] to avoid over-smoothing.

4. Subgraphs approaches: Zeng et al. [START_REF] Zeng | Decoupling the depth and scope of graph neural networks[END_REF] recently proposed a new idea to avoid over-smoothing, where localized subgraphs are extracted to train GNNs of arbitrary depths.

Alon and Yahav [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF] introduced over-squashing to explain why GNNs struggle to propagate information between distant nodes in the graph. They also presented a rewiring method where a fully-adjacent matrix is added in the last GNN layer.

Later, Topping et al. [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] proposed a theory explaining where the over-squashing comes from and how to alleviate it from a topological point of view. They presented a rewiring method using concepts of Ricci flow curvature from differential geometry [START_REF] Hamilton | The Ricci flow on surfaces[END_REF]. These two studies [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF][START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] focused only on over-squashing, while in this chapter we introduce for the first time the trade-off between both issues, over-smoothing and over-squashing. Similarly, we try to address both problems simultaneously, using JLC and node-embedding metrics to rewire the graph.

Preliminaries

In this chapter we only consider unweighted graphs, i.e., A ∈ {0, 1} 𝑁 ×𝑁 .

Cheeger Inequality and Cheeger Constant

Definition 7.3.1. Let 𝐺 = (𝒱, ℰ) be a graph, and let 𝒮 ⊂ 𝒱 be a subset of nodes.

Let 𝜕𝒮 be the set of edges going from a node in 𝒮 to a node in 𝒱 ∖ 𝒮, i.e., 𝜕𝒮 ≜ {{𝑢, 𝑣} ∈ ℰ : 𝑢 ∈ 𝒮, 𝑣 ∈ 𝒱 ∖ 𝒮}. Therefore, we can define the Cheeger constant ℎ 𝐺 of 𝐺 as:

ℎ 𝐺 ≜ min 𝒮 ℎ 𝐺 (𝒮), (7.1)
where ℎ 𝐺 (𝒮) = |𝜕𝒮|/ min(vol(𝒮), vol(𝒱 ∖ 𝒮)), vol(𝒮) = ∑ 𝑖∈𝒮 𝑑 𝑖 .

Intuitively, the Cheeger constant in Definition 7.3.1 is small when there exists a bottleneck in 𝐺, i.e., when there are two sets of nodes with few edges between them.

Similarly, we know that ℎ 𝐺 > 0 if and only if 𝐺 is a connected graph [START_REF] Chung | Spectral graph theory[END_REF]. We can relate the Cheeger constant ℎ 𝐺 with the first non-zero eigenvalue of ℒ through the Cheeger inequality:

2ℎ 𝐺 ≥ 𝜆 2 ≥ ℎ 2 𝐺 2 . (7.2)
We notice from (7.2) that for having less "bottleneckness" in the graph, we need to promote big values of ℎ 𝐺 , i.e., having large values of 𝜆 2 will increase the Cheeger constant since ℎ 𝐺 ≥ 𝜆 2 /2.

Over-squashing

The over-squashing problem is a more recent and less understood problem than oversmoothing. A graph learning problem has long-range dependencies when the outputs of GNNs depend on features of interacting distant nodes. In that scenario, information from non-adjacent nodes should be propagated through the network without distortion. Let ℬ 𝑟 (𝑖) ≜ {𝑗 ∈ 𝒱 : 𝑑 𝐺 (𝑖, 𝑗) ≤ 𝑟} be the receptive field of an 𝑟-layer GNN for the node 𝑖, where 𝑑 𝐺 (𝑖, 𝑗) is the shortest-path distance between the nodes 𝑖 and 𝑗, and 𝑟 ∈ N. In many graphs, |ℬ 𝑟 (𝑖)| grows exponentially with 𝑟 and then representations of an exponential amount of neighboring nodes should be compressed into fixed-size vectors. This phenomenon is referred to as over-squashing3 of information [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF][START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF].

7.4 Understanding the Over-smoothing vs. Oversquashing Trade-off

The Stationary Distribution on Graphs

Let P = D -1 A be the random walk transition matrix. For any initial distribution 𝑓 : 𝒱 → R with ∑ 𝑣∈𝒱 𝑓 (𝑣) = 1, the distribution after 𝑘 steps is given by f T P 𝑘 , where f ∈ R 𝑁 ×1 is the vector of initial distributions such that f (𝑖) is the function evaluated on the 𝑖th node. The random walk is ergodic when there is a unique stationary distribution 𝜋 satisfying that lim 𝑠→∞ f T P 𝑠 = 𝜋 [START_REF] Chung | Spectral graph theory[END_REF].

Lemma 7.4.1 (Chung [START_REF] Chung | Spectral graph theory[END_REF]). Let P be the random walk transition matrix, let 𝜋 be the stationary distribution of the corresponding walk, and let f be any initial distribution.

For 𝑠 ∈ N + , we know:

‖f T P 𝑠 -𝜋‖ 2 ≤ 𝑒 -𝑠𝜆 ′ max 𝑖 √ 𝑑 𝑖 min 𝑗 √ 𝑑 𝑗 , where 𝜆 ′ = ⎧ ⎪ ⎨ ⎪ ⎩ 𝜆 2 if 1 -𝜆 2 ≥ 𝜆 𝑁 -1, 2 -𝜆 𝑁 otherwise. (7.3)
Therefore, we can compute the value of 𝑠 such that ‖f T P 𝑠 -𝜋‖ 2 ≤ 𝜖 as follows:

𝑠 ≥ 1 𝜆 ′ log ( max 𝑖 √ 𝑑 𝑖 𝜖 min 𝑗 √ 𝑑 𝑗 ) . (7.4)
Proof: See [START_REF] Chung | Spectral graph theory[END_REF].

Notice that 𝜆 ′ is either 𝜆 2 or 2 -𝜆 𝑁 in Lemma 7.4.1. However, we can show that only 𝜆 2 is crucial. Suppose that 𝜆 𝑁 -1 > 1 -𝜆 2 , so that 𝜆 ′ = 2 -𝜆 𝑁 . We can consider the lazy walk on the graph 𝐺 ′ formed by A + I. Therefore, the new graph Laplacian has eigenvalues λ𝑖 = 𝜆 𝑖 /2 ≤ 1 [START_REF] Chung | Spectral graph theory[END_REF], and then 1 -λ2 ≥ 1 -λ𝑁 ≥ 0.

The key message of Lemma 7.4.1 is a simplified version of the same observations of previous works [START_REF] Rong | DropEdge: Towards deep graph convolutional networks on node classification[END_REF] and [START_REF] Oono | Graph neural networks exponentially lose expressive power for node classification[END_REF], i.e., GNNs converge exponentially to the stationary distribution when stacking several layers. However, we show that the convergence of this exponential function depends on the first non-zero eigenvalue of the graph developments in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF]. In other words, we can use an SGC with a random walk kernel to show how the node embeddings converge to the stationary distribution when we stack several layers according to Lemma 7.4.3. Similarly, Topping et al. [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] explained how reducing bottlenecks in the graphs can alleviate over-squashing, i.e., the receptive field of each node in a deep GNN will be polynomial in the hop-distance rather than exponential (see Corollary 3 in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF]). We leave for future work the analysis of the relationship between over-smoothing and over-squashing for more complex GNNs with non-linear activation functions.

Let 𝑓 (𝜆 2 , 𝜖) = 1 𝜆 2 log ( max 𝑖 √ 𝑑 𝑖 𝜖 min 𝑗 √ 𝑑 𝑗 )
be the mixing steps of our graph, i.e., the lower bound in the maximum number of layers of an SGC such that the difference between the initial and stationary distribution is at most 𝜖. Figure 7-1 shows, for 𝜖 = 5 × 10 -4 , how 𝑓 (𝜆 2 , 𝜖) and 𝜆 2 change when we add or remove edges in two artificial stochastic block model graphs and one Erdős-Rényi graph. We notice that we can increase the mixing steps by removing edges, i.e., we can alleviate over-smoothing by making the graph more "bottleneckness". This partially explains why DropEdge [START_REF] Rong | DropEdge: Towards deep graph convolutional networks on node classification[END_REF] can alleviate over-smoothing. On the other hand, we increase 𝜆 2 by adding edges as shown in Fig. 7-1, so we promote higher values of the Cheeger constant ℎ 𝐺 , i.e., we can alleviate over-squashing by making the graph less "bottleneckness". This can partially explain why the methodology by Topping et al. [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] can alleviate over-squashing. However, we notice that there is a trade-off between 𝑓 (𝜆 2 , 𝜖) and 𝜆 2 from a topological point of view, i.e., we can increase 𝑓 (𝜆 2 , 𝜖) by removing key edges but 𝜆 2 will decrease, and vice versa. The algorithm to add and remove edges is explained in Section 7.5.

Algorithm 2 Stochastic Jost and Liu Curvature Rewiring

Input: Graph 𝐺, GNN architecture Initialization: Hyper-parameters 𝑝 𝐴 , 𝑝 𝐷 , 𝜏 , 𝛼. (𝑖 ′ , 𝑗 ′ ) = arg min (𝑖,𝑗) JLC(𝑖, 𝑗)

4:

Compute the set of edges

𝒜 = {(𝒩 𝑖 ′ ∖ 𝑗 ′ ) × (𝒩 𝑗 ′ ∖ 𝑖 ′ ) : (𝒩 𝑖 ′ ∖ 𝑗 ′ ) × (𝒩 𝑗 ′ ∖ 𝑖 ′ ) / ∈ ℰ 𝑚 } 5:
Compute for all 0 < 𝑝 ≤ |𝒜|, 𝜎(𝑝) = JLC ′ (𝑟, 𝑠)-JLC(𝑖 ′ , 𝑗 ′ ), where (𝑟, 𝑠) = 𝒜(𝑝)

6:

Add randomly an edge (𝑟 ′ , 𝑠 ′ ) ∈ 𝒜 to ℰ 𝑚 according to the distribution softmax(𝜏 𝜎)

7:
Update JLC(𝑖, 𝑗) ∀ (𝑖, 𝑗) ∈ ℰ 𝑚+1 , where ℰ 𝑚+1 = ℰ 𝑚 ∪ (𝑟 ′ , 𝑠 ′ ) 8: end for 9: Normalize JLC(𝑖, 𝑗) to be in [0, 1] for all (𝑖, 𝑗) ∈ ℰ 𝐴 . 10: for each layer 𝑙 in GNN while training do

11: Compute d (𝑙) (𝑝) = ‖h (𝑙) 𝑖 -h (𝑙) 𝑗 ‖ 2 ∀ (𝑖, 𝑗) ∈ ℰ 𝐴 12:
Drop 𝐷 edges according to the probability distribution softmax(𝛼𝜑+(1-𝛼)d (𝑙) 𝑛 ) 13: end for nodes. Topping et al. [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] proposed a method where edges are added and removed as a pre-processing step. However, the reason for removing edges was not properly justified in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF]. We can argue that removing edges is also important to alleviate oversmoothing according to the developments in Section 7.4. Our SJLR algorithm also adds and removes edges but in a fundamentally different way. We keep both processes separate so that we can alleviate over-squashing and over-smoothing according to the dataset as shown in Fig. 7-2. We define a set of hyper-parameters for SJLR: 1) let 𝑝 𝐴 be the percentage of added edges, 2) let 𝑝 𝐷 be the percentage of dropped edges, 3) let 𝜏 be a variable controlling how stochastic SJLR is when adding edges, and 4) let 𝛼 ∈ [0, 1] be a variable controlling how important is the JLC metric against the embedding information while dropping edges. We optimize the hyper-parameters in the validation set so that SJLR can choose either if the specific dataset requires more adding or removal of edges, i.e., SJLR tries to find the "sweet point" in the trade-off between over-squashing and over-smoothing. Algorithm 2 presents our SJLR approach in detail. The SJLR algorithm has as input an initial graph 𝐺 and a given GNN architecture. 𝒩 𝑙 is the set of neighbors of node 𝑙, and 𝜎 ∈ R |𝒜| is the vector of JLC improvements if adding an edge from 𝒜. Similarly, JLC ′ (𝑟, 𝑠) is the JLC metric computed in 𝐺 ′ = (𝒱, ℰ 𝑚 ∪ (𝑟, 𝑠)).

H (𝑙) ∈ R 𝑁 ×𝐹 = [h (𝑙) 1 , . . . h (𝑙)
𝑁 ] T is the 𝐹 -dimensional embeddings of all nodes after 𝑙 layers such that each h 𝑛 ∈ [0, 1] |ℰ 𝐴 | is the vector of normalized distances between node embeddings. SJLR algorithm is divided into two parts: 1) the addition of edges using the JLC metric, and 2) the removal of edges while training the GNN.

Notice that SJLR is general to any GNN architecture. However, we only test our algorithm using GCNs [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] in Section 7.6 due to limited computational resources.

Experimental Framework and Results

Experiments

We perform a set of experiments to compare SJLR with several approaches in the literature, striving to understand the underlying relationship between over-smoothing and over-squashing. SJLR is compared with seven state-of-the-art methods to alleviate over-smoothing or over-squashing, including: Residual/Dense Connections (RDC) [START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF], Graph Diffusion Convolution (GDC) with personalized PageRank kernel [START_REF] Klicpera | Diffusion improves graph learning[END_REF], DropEdge (DE) [START_REF] Rong | DropEdge: Towards deep graph convolutional networks on node classification[END_REF], PairNorm (PN) [START_REF] Zhao | PairNorm: Tackling oversmoothing in GNNs[END_REF], Differentiable Group Normalization (DGN) [START_REF] Zhou | Towards deeper graph neural networks with differentiable group normalization[END_REF], Fully-Adjacent (FA) layers [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF], and Stochastic Discrete Ricci Flow (SDRF) [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF]. Our GNN base model is GCN [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] for all experiments. We evaluate all methods in nine datasets: Cornell, Texas, and Wisconsin from the We-bKB project 4 , Chameleon [START_REF] Rozemberczki | Multi-scale attributed node embedding[END_REF], Squirrel [START_REF] Rozemberczki | Multi-scale attributed node embedding[END_REF], Actor [START_REF] Tang | Social influence analysis in large-scale networks[END_REF], Cora [START_REF] Mccallum | Automating the construction of internet portals with machine learning[END_REF], Citeseer [START_REF] Sen | Collective classification in network data[END_REF],

and Pubmed [START_REF] Namata | Query-driven active surveying for collective classification[END_REF]. These databases are related to data mining problems, where the objectives are to classify web pages or scientific papers into predefined classes, for example. We optimize the hyperparameters with a random search procedure by maximizing the average accuracy in the validation sets. The search space and the best hyperparameters for each experiment are provided in Appendix H. We consider the largest connected component of the graph for each dataset as in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF][START_REF] Klicpera | Diffusion improves graph learning[END_REF]. We split the data into train/validation/test sets, where we first divide the data into a development set and a test set. This is done once to avoid using the test set in the hyperparameter optimization procedure. We follow the same experimental framework as in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF][START_REF] Klicpera | Diffusion improves graph learning[END_REF], i.e., we optimize the hyperparameters for all dataset-preprocessing combinations separately by random search over 100 data splits. Furthermore, we report average accuracies on the test set with 95% confidence intervals calculated by bootstrapping with 1000 samples. The development set contains 1500 nodes and the rest of the nodes are used for testing in Cora, Citeseer, and Pubmed. The train set contains 20 nodes of each class while the rest of the nodes are used for validation.

For the other datasets, we use a 60/20/20 split of the nodes, i.e., 60% for training, 20% for validation, and 20% for testing. We use the same method and random seeds as in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF][START_REF] Klicpera | Diffusion improves graph learning[END_REF], so we expect to have the same partitions and comparable results.

The search space for the number of layers of the GNN in the first experiment is {1, 2, 3} as in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] (for further details please see Appendix H). However, this search space does not provide insights about the problems we could face when stacking several layers. As a result, we perform a second experiment where we stack several layers in the set ℱ = {2, 8, 32}. We follow the same methodology as in the first experiment for each method and for each number of layers in ℱ, regarding data partitions and hyperparameters optimization. Finally, we calculate the average running time over ten repetitions to compute the metrics BFC and JLC for all edges in three artificial graphs with different amounts of nodes.

Implementation Details

All methods are implemented using PyTorch and PyG [START_REF] Fey | Fast graph representation learning with PyTorch Geometric[END_REF]. We use the same architectural components in all methods for a fair comparison. We use GCN [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] as graph convolutional layers, notice however that other GNNs like GAT [START_REF] Veličković | Graph attention networks[END_REF] can be used as well. We use Rectified Linear Unit (ReLU) and log-softmax as activation functions in our GNN architectures. The number of GNN layers, hidden units, learning rate, weight decay, and dropout rate for each method are optimized in the validation set on each experiment (for further details please see Appendix H). For GDC, we apply weight decay regularization only in the first graph convolutional layer, otherwise we do not get comparable results as in [START_REF] Klicpera | Diffusion improves graph learning[END_REF]. All methods are trained for 1000 epochs using Adam optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF]. We do not use early stopping or learning schedulers for any method. We make all graphs undirected, and we also remove all the self loops from the input graph. We evaluate all methods under the same configuration unless otherwise stated. We implemented SDRF [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] at our best understanding because there was not an official or available implementation of the method at the time of executing the experiments. However, we used JLC instead of BFC for our implementation of SDRF [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] because we do not have the computational resources to run the hyperparameter optimization using BFC. The hyperparameter optimizations were executed on several GPUs including: two Nvidia GeForce RTX 2080, two Nvidia GeForce GTX 1070, and two Nvidia Tesla V100.

Results

Table 7.1 shows the results of the first experiment related to the random search with few convolutional layers. We can notice two general trends: 1) rewiring methods like DE and SJLR dominate in heterophyllous datasets like Cornell, Texas, Wisconsin, Chameleon, Squirrel, and Actor; and 2) GDC leads in homophilous datasets like Cora, Citeseer, and Pubmed, while SJLR and FA offer competitive performances in some cases. Regarding the comparison between the curvature-based methods SJLR and SDRF [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF], we notice that SJLR outperforms SDRF in six out of nine datasets. Both SJLR and SDRF are executed using the same JLC metric in Table 7.1, and therefore we are assessing their performance based on the way the edges are added or removed.

We can argue that SJLR is thus a critical improvement over SDRF regarding the practical adoption of curvature-based methods in GNNs. We can also notice that no single method can address all kinds of datasets, and perhaps an ensemble of different algorithms can be more suitable to outperform all previous approaches, which is not the scope of this chapter and we leave for future work. Finally, we remark that some methods like GDC [START_REF] Klicpera | Diffusion improves graph learning[END_REF] and FA [START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF] require specific architectural changes to work properly. For example, we achieve the specific results of GDC only when applying weight decay in the first graph convolutional layer as stated in Section 7.6.2.

Figure 7-3 shows the average running time over ten repetitions to compute the BFC and JLC for three artificial graphs with different amounts of nodes. We notice the large gap between the computation time of the JLC and BFC, which makes JLC more suitable for practical applications.

Limitations

Several studies in GNNs, like the ones reviewed in Section 7.2, have focused on solving the problems of over-smoothing or over-squashing without a clear idea of their practical importance. For example, the definitions of homophily in the literature usually consider only first-neighboring relationships. Similarly, the benchmarking graphs in Table 7.1 do not have large diameters. Therefore, we do not have mathematical or practical tools to assess how many "long-range dependencies" exist between samples in a specific dataset. Perhaps, the benchmarking datasets we have been testing so far do not have many long-range dependencies, and carefully designed GNNs can be enough in these scenarios [START_REF] Ma | Is homophily a necessity for graph neural networks?[END_REF]. As a consequence, the results in Table 7.1 and also the results in previous works [START_REF] Li | DeepGCNs: Can GCNs go as deep as CNNs?[END_REF][START_REF] Alon | On the bottleneck of graph neural networks and its practical implications[END_REF][START_REF] Rong | DropEdge: Towards deep graph convolutional networks on node classification[END_REF][START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF][START_REF] Zhao | PairNorm: Tackling oversmoothing in GNNs[END_REF][START_REF] Zhou | Towards deeper graph neural networks with differentiable group normalization[END_REF][START_REF] Klicpera | Diffusion improves graph learning[END_REF] might be only partially meaningful.

Conclusions

In this chapter, we presented a connection between the problems of over-smoothing and over-squashing in GNNs. We showed how both issues are intrinsically related to the eigenvalue gap of the normalized Laplacian matrix. As a result, we used the Cheeger inequality to show that we cannot arbitrarily improve over-smoothing and over-squashing simultaneously from a topological perspective, i.e., there exists a trade-off between both problems. We also introduced a new Stochastic Jost and Liu curvature Rewiring (SJLR) algorithm based on a bound of the Ollivier's Ricci curvature to alleviate over-smoothing and over-squashing. Our algorithm is less computationally complex than previous methods while guaranteeing important theoretical properties. SJLR outperforms some previous methods in homophilous and heterophyllous graph datasets for node classification.

The simple theoretical results presented in this chapter strive to improve our understanding of the problems we face when training deep GNNs. However, there

are still many open problems in this domain. For example, two interesting questions for future work could be: 1) do we really need deep GNNs? and 2) when do we need deep GNNs? The answer to these two questions is practically meaningful and will prove the practical significance of this and previous works. The previous Chapters 6 and 7 focused on the machine learning part of this thesis. We introduced new concepts on graphs trying to overcome some limitations of GNNs such as expresiveness, over-smoothing, and over-squashing. In this chapter, we explore the signal processing part of this thesis. Here we focus in the reconstruction of timevarying graph signals.

The sampling and reconstruction of graph signals are fundamental tasks in GSP that have recently received considerable attention. Naturally, the mathematics of sampling theory and spectral graph theory have been combined, leading to generalized Nyquist sampling principles for graphs [START_REF] Chen | Discrete signal processing on graphs: Sampling theory[END_REF][START_REF] Anis | Efficient sampling set selection for bandlimited graph signals using graph spectral proxies[END_REF][START_REF] Anis | Towards a sampling theorem for signals on arbitrary graphs[END_REF][START_REF] Shomorony | Sampling large data on graphs[END_REF], where most of these previous works have focused on static graph signals. The reconstruction of time-varying graph signals is an important problem that has not been well explored 1 . The reconstruction of time-varying graph signals from discrete samples has several real world applications, such as the estimation of new cases of infectious diseases [START_REF] Giraldo | On the minimization of Sobolev norms of time-varying graph signals: Estimation of new Coronavirus disease 2019 cases[END_REF], or the recovery of the sea surface temperature for the study of the earth's climate dynamics [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF].

Some recent methods for the reconstruction of time-varying graph signals have also used the assumption that temporal differences of graph signals are smooth.

For example, Qiu et al. [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] extended the definition of smooth signals from static to time-varying graph signals, and solved an optimization problem for reconstruction. However, Qiu's method may have slow convergence rate because the eigenvalue spread of the Hessian associated with their optimization problem may be large. The techniques presented in [START_REF] Gavili | On the shift operator, graph frequency, and optimal filtering in graph signal processing[END_REF][START_REF] Hua | Online distributed learning over graphs with multitask graph-filter models[END_REF][START_REF] Xia | Distributed adaptive multi-task learning based on partially observed graph signals[END_REF] may be used to circumvent the problem of slow convergence. However, these techniques require eigenvalue decomposition or matrix inversion, which are computationally expensive for large graphs.

In the current chapter, we propose a new algorithm to reconstruct time-varying graph signals from samples. Our algorithm is based on the extension of the Sobolev norm defined in GSP for time-varying graph signals [3,[START_REF] Pesenson | Variational splines and Paley-Wiener spaces on combinatorial graphs[END_REF]. Therefore, we dub our algorithm as Time-varying Graph signal Reconstruction via Sobolev Smoothness (GraphTRSS). Our algorithm uses a Sobolev smoothness function to formulate an optimization problem for the reconstruction of a time-varying graphs signal from its samples, where the samples are obtained with a random sampling strategy. The graph is constructed with a 𝑘-NN method. The optimization problem of GraphTRSS is solved with the conjugate gradient method. We analyze the convergence rate of our algorithm by studying the condition number of the Hessian associated with our optimization function, and we conclude that our algorithm converges faster than its closest competitor [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] under certain conditions. Moreover, GraphTRSS does not require expensive eigenvalue decomposition or matrix inversion. Finally, we evaluate our algorithm on synthetic data, two COVID-19, and two environmental datasets, where our algorithm outperforms several state-of-the-art methods.

The main contributions of this chapter are summarized as follows:

• We use the concept of Sobolev norms from static graph signals to define a smoothness function for time-varying graph signals, and then we use this new conception to introduce an algorithm for reconstruction.

• We provide several mathematical insights of GraphTRSS. Specifically, we analyze the convergence rate of our algorithm by studying the condition number of the Hessian associated with our problem.

• GraphTRSS improves convergence speed without relying on expensive matrix inversion or eigenvalue decomposition.

Related Work

The problems of sampling and reconstruction of graph signals have been widely explored in GSP [START_REF] Chen | Discrete signal processing on graphs: Sampling theory[END_REF][START_REF] Anis | Efficient sampling set selection for bandlimited graph signals using graph spectral proxies[END_REF][START_REF] Romero | Kernel-based reconstruction of graph signals[END_REF][START_REF] Parada-Mayorga | Blue-noise sampling on graphs[END_REF][START_REF] Lorenzo | Adaptive least mean squares estimation of graph signals[END_REF][START_REF] Chepuri | Graph sampling for covariance estimation[END_REF][START_REF] Valsesia | Sampling of graph signals via randomized local aggregations[END_REF][START_REF] Venkitaraman | Predicting graph signals using kernel regression where the input signal is agnostic to a graph[END_REF]. Pesenson [START_REF] Pesenson | Sampling in Paley-Wiener spaces on combinatorial graphs[END_REF] introduced concepts of Paley-Wiener spaces in graphs, where a graph signal can be determined by its samples in a set of nodes called uniqueness set. One can say that a set of nodes is a uniqueness set of a certain graph if the fact that two graph signals in the Paley-Wiener space of the graph coincide in the uniqueness set implies that they coincide in the whole set of nodes. As a result, a bandlimited graph signal can exactly be reconstructed from its samples if the graph signal is sampled according to its uniqueness set. However, the bandlimitedness assumption is not realistic, i.e., the graph signals in real-world datasets tend to be approximately bandlimited instead of strictly bandlimited. Therefore, several researchers have proposed reconstruction algorithms based on the smoothness assumption of graph signals [START_REF] Narang | Localized iterative methods for interpolation in graph structured data[END_REF][START_REF] Belkin | Regularization and semi-supervised learning on large graphs[END_REF][START_REF] Chen | Signal recovery on graphs: Fundamental limits of sampling strategies[END_REF], where the smoothness is measured with a Laplacian function. Similarly, other studies have used the Total Variation of graph signals [START_REF] Chen | Signal recovery on graphs: Variation minimization[END_REF], or extensions of the concept of stationarity on graph signals [START_REF] Loukas | Stationary time-vertex signal processing[END_REF][START_REF] Perraudin | Stationary signal processing on graphs[END_REF] for reconstruction.

For time-varying graph signals, scientists have developed notions of joint harmonic analysis linking together the time-domain signal processing techniques with GSP [START_REF] Grassi | A time-vertex signal processing framework: Scalable processing and meaningful representations for time-series on graphs[END_REF], while other researchers have proposed reconstruction algorithms assuming bandlimitedness of the signals at each instant [START_REF] Chen | Signal recovery on graphs: Variation minimization[END_REF][START_REF] Wang | A distributed tracking algorithm for reconstruction of graph signals[END_REF]. Most of these methods do not fully exploit the underlying temporal correlations of the time-varying graph signals. Qiu et al. [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] proposed an approach where the temporal correlations are captured with a temporal difference operator in the time-varying graph signal. However, Qiu's method may have slow convergence because the optimization problem depends on the Laplacian matrix. In particular, the eigenvalue spread of the Hessian associated with their problem may be large, leading to poor condition numbers. Other studies have reported the undesirable effects in convergence due to the large eigenvalue spread when dealing with shift operators derived from the Laplacian or the adjacency matrix. For example, Hua et al. [START_REF] Hua | Online distributed learning over graphs with multitask graph-filter models[END_REF] used a method of second-order moments to solve the problem of slow convergence speed, and this approach showed improvement in performance at the expense of additional computational cost by expensive matrix inversion operations. We can also circumvent the problem of slow convergence by using energy-preserving shift operators as in [START_REF] Gavili | On the shift operator, graph frequency, and optimal filtering in graph signal processing[END_REF]. For example, Xia et al. [START_REF] Xia | Distributed adaptive multi-task learning based on partially observed graph signals[END_REF] exploited an energy-preserving shift operator for distributed learning problems over graphs. However, the energy-preserving shift operators require the eigendecomposition of the adjacency (or Laplacian) matrix, which is computational prohibitive for large graphs.

In the current chapter, we assume smoothness in the temporal differences of graph signals as in [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]. However, our algorithm improves the condition number of the Hessian associated with our problem without requiring expensive eigenvalue decompositions or matrix inversions. We show that GraphTRSS converges faster than Qiu's method [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] under some conditions.

Reconstruction of Time-Varying Graph Signals

Several studies have used the smoothness assumption in graphs to solve problems of reconstruction and sampling of graph signals. Formally, notions of smoothness in x have been introduced with concepts of local variation [START_REF] Shuman | The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF] as in Section 2.4. Some researchers have used the graph Laplacian quadratic form as a regularization term to solve problems of reconstruction of graph signals [START_REF] Puy | Random sampling of bandlimited signals on graphs[END_REF]. For time-varying graph signals, an extension of the graph Laplacian quadratic form has also been used for reconstruction [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF].

Let X = [x 1 ,
𝑆 2 (X) = 𝑀 ∑ 𝑖=1 x T 𝑖 Lx 𝑖 = tr(X T LX). (8.1)
However, notice that (8.1) does not have temporal relationships between graph signals at different times 𝑖. To include time information, Qiu et al. [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] introduced the temporal difference operator D ℎ ∈ R 𝑀 ×(𝑀 -1) defined as follows:

D ℎ = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ -1 1 -1 1 . . . . . . -1 1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ∈ R 𝑀 ×(𝑀 -1) . (8.2)
As a consequence, the temporal difference graph signal is such that:

XD ℎ = [x 2 -x 1 , x 3 -x 2 , . . . , x 𝑀 -x 𝑀 -1 ]. ( 8
ℬ 𝛼 (𝐺) = { X : tr ( (XD ℎ ) T LXD ℎ ) ≤ (𝑀 -1)𝛼 } , (8.4)
where 𝛼 indicates the smoothness level of the time-varying graph signal.

min X 1 2 ‖J ∘ X -Y‖ 2 𝐹 + 𝜐 2 tr ( ( XD ℎ ) T L XD ℎ ) , (8.5)
where J ∈ {0, 1} 𝑁 ×𝑀 is the sampling matrix of X, 𝜐 is a regularization parameter, and Y ∈ R 𝑁 ×𝑀 is the matrix of signals that we know (the observed values). The sampling matrix J is defined as follows:

J(𝑖, 𝑗) = ⎧ ⎪ ⎨ ⎪ ⎩ 1 if 𝑖 ∈ 𝒮 𝑗 , 0 if 𝑖 / ∈ 𝒮 𝑗 , (8.6)
where 𝒮 𝑗 is the set of sampled nodes at column 𝑗.

Theorem 8.3.1 (Qiu et al. [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]). The solution of (8.5) is unique when the following conditions are satisfied by the sampling matrix J:

1. For any 𝑛 ∈ {1, . . . , 𝑁 }, ∃ 𝑚 ∈ {1, . . . , 𝑀 } such that J(𝑛, 𝑚) = 1.

2. There is a fiducial time 𝑚 0 ∈ {1, . . . , 𝑀 }, such that for any 𝑚 ∈ {1, . . . , 𝑀 }, with 𝑚 ̸ = 𝑚 0 , there exist a node 𝑛 𝑚 ∈ {1, . . . , 𝑁 } satisfying that J(𝑛 𝑚 , 𝑚 0 ) = J(𝑛 𝑚 , 𝑚) = 1.

Proof: see [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF].

Theorem 8.3.1 provides some properties that J should satisfy to obtain a unique solution for (8.5). Notice that deterministic sampling methods [START_REF] Chen | Discrete signal processing on graphs: Sampling theory[END_REF][START_REF] Anis | Efficient sampling set selection for bandlimited graph signals using graph spectral proxies[END_REF][START_REF] Tsitsvero | Signals on graphs: Uncertainty principle and sampling[END_REF][START_REF] Lorenzo | Sampling and recovery of graph signals[END_REF] or sampling on product graphs [START_REF] Ortiz-Jiménez | Sampling and reconstruction of signals on product graphs[END_REF] do not satisfy the first condition of Theorem 8.3.1, so it is not suitable for these problems. In contrast, in the current chapter we propose a random sampling strategy as in [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF].

Eqn. (8.5) reconstructs a time-varying graph signal X with 1) a small error ‖J∘ X-Y‖ 2 𝐹 , and 2) a small value of the temporal difference graph signal smoothness tr(( XD ℎ ) T L XD ℎ ). Finally, the parameter 𝜐 in (8.5) weights the importance between the error and smoothness terms. This parameter 𝜐 is usually tuned experimentally.

In the current chapter, we explore two additional temporal difference operators, a two-time steps operator:

D ℎ2 = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ -1 2 -1 -1 2 . . . -1 . . . -1 . . . 2 -1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ∈ R 𝑀 ×(𝑀 -2) , (8.7)
and a three-time steps operator:

D ℎ3 = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ -1 -1 -1 4 -1 . . . -1 4 . . . -1 -1 -1 . . . -1 -1 . . . 4 . . . -1 -1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ∈ R 𝑀 ×(𝑀 -4) . (8.8)
The objective of introducing (8.7) and (8.8) is to capture different kinds of temporal relationships besides XD ℎ . For example,

XD ℎ2 = [2x 2 -x 1 -x 3 , 2x 3 -x 2 - x 4 , . . . , 2x 𝑀 -1 -x 𝑀 -2 -x 𝑀 ].

Sobolev Smoothness of Time-Varying Graph Signals

In the current chapter, we extend the definition of Sobolev norms in GSP [3,[START_REF] Pesenson | Variational splines and Paley-Wiener spaces on combinatorial graphs[END_REF]

‖x‖ 2 𝛽,𝜖 = x T (L + 𝜖I) 𝛽 x, (8.9)
where 𝜖 ≥ 0, and 𝛼 ∈ R. Notice the Sobolev norm in (8. 𝑆

𝛽,𝜖 (X) ≜ 𝑀 ∑ 𝑖=2 (x 𝑖 -x 𝑖-1 ) T (L + 𝜖I) 𝛽 (x 𝑖 -x 𝑖-1 ) = tr ( (XD ℎ ) T (L + 𝜖I) 𝛽 (XD ℎ ) ) . ( 8
.10)

Sobolev Reconstruction

We use the Sobolev smoothness in Definition 8.4.1 to formulate two new reconstruction algorithms. The first algorithm solves the problem for the noiseless case, while the second algorithm solves the noisy case.

In the noiseless case, we assume that the sampling mechanism does not add noise to the problem. As a consequence, the observed graph signal is given by Y = J ∘ X.

One can formulate an optimization problem to get an approximate reconstruction of the time-varying graph signal as follows:

min X 1 2 tr ( ( XD ℎ ) T (L + 𝜖I) 𝛽 XD ℎ ) s.t. J ∘ X = Y. (8.11)
The optimization function in (8.11) reconstructs a smooth spatiotemporal graph signal X given the constraint Y = J ∘ X. The noiseless case can be solved by a gradient projection algorithm. The iterative update is such that:

X𝑡+1 = ( X𝑡 -𝜉∇ X𝑓 𝑠 ( X𝑡 ) ) + , (8.12)
where

𝑓 𝑠 ( X𝑡 ) = 1 2 tr ( ( X𝑡 D ℎ ) T (L + 𝜖I) 𝛽 X𝑡 D ℎ )
, 𝜉 is the step size, ∇ X𝑓 𝑠 ( X𝑡 ) is the gradient of function 𝑓 𝑠 ( X𝑡 ) given as:

∇ X𝑓 𝑠 ( X𝑡 ) = (L + 𝜖I) 𝛽 X𝑡 D ℎ D T ℎ , (8.13)
and (V) + is the projection of signal V to the signal space Y = J ∘ X given as follows [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]:

(V) + = Y + V -J ∘ V. (8.14)
The above formulation can be extended to the noisy case as follows. We take into account the noise by relaxing the constraint in (8.11):

min X 1 2 ‖J ∘ X -Y‖ 2 𝐹 + 𝜐 2 tr ( ( XD ℎ ) T (L + 𝜖I) 𝛽 XD ℎ ) . ( 8

.15)

The optimization problem in (8.15) reconstructs a time-varying graph signal with a small error (given by the first term) and a small value for the Sobolev smoothness of its temporal difference signal. Here, we assume that the difference of graph signals XD ℎ presents better smoothness properties than X alone.

In the current chapter, we solve (8.5) and (8.15) using the conjugate gradient The framework of our algorithm (GraphTRSS) using a matrix of coordinates M ∈ R 𝑁 ×2 to construct a graph with 𝑁 regions in the world with confirmed cases of COVID-19 by November 18, 2020. The graph is constructed with a 𝑘-NN method. GraphTRSS uses the operator D ℎ ∈ R 𝑀 ×(𝑀 -1) to capture temporal information in the time-varying signal X ∈ R 𝑁 ×𝑀 with 𝑀 temporal snapshots, and it also uses different sampling strategies J ∈ {0, 1} 𝑁 ×𝑀 according to the desired output (reconstruction or forecasting). Finally, the optimization function, which includes the error and Sobolev terms, reconstructs or predicts the missing values, i.e., the indexes of X where J has values zero. method as in [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]. Therefore, we update the search direction ∆ X𝑡 and the step size 𝜇 on each iteration 𝑡 as follows:

𝜇 = - ⟨∆ X𝑡 , ∇ X𝑓 𝑢 ( X𝑡 )⟩ ⟨∆ X𝑡 , ∇ X𝑓 𝑢 (∆ X𝑡 ) + Y⟩ , (8.16)
where

∇ X𝑓 𝑢 ( X) = J ∘ X -Y + 𝜐L XD ℎ D T ℎ to solve (8.5), or ∇ X𝑓 𝑢 ( X) = J ∘ X - Y + 𝜐(L + 𝜖I) 𝛽 XD ℎ D T ℎ to solve (8.15). Using 𝜇, we have that X𝑡+1 = X𝑡 + 𝜇∆ X𝑡 , where ∆ X𝑡 = -∇ X𝑓 𝑢 ( X𝑡 ) + 𝛾∆ X𝑡-1 and 𝛾 = ‖∇ X𝑓𝑢( X𝑡 )‖ 2 𝐹 ‖∇ X𝑓𝑢( X𝑡-1 )‖ 2 𝐹
. The stopping condition is given either by achieving a maximum number of iterations or when:

‖∆ X𝑡 ‖ 𝐹 ≤ 𝛿, (8.17)
where 𝛿 = 10 -6 in the experiments for both (8.5) and (8.15). of the Sobolev minimization in (8.15) is its fast convergence with respect to the Laplacian minimization in (8.5). In the following section, we analyze some properties of the convergence rate of the optimization problems described in (8.5) and (8.15).

Convergence Rate

Intuitively, the convergence of a gradient descent method is faster when we have wellconditioned optimization problems. Fig. 8-2 shows a toy example of the convergence between well and ill-conditioned problems in the contour of a convex error surface.

One can notice that the well-conditioned problem goes smoothly and faster to the global minimum, while the ill-conditioned problem has an erratic convergence. Similarly, the ill-conditioned problem can take longer to satisfy the stopping condition of certain optimization algorithms because of the erratic convergence.

Formally, the rate of convergence of a gradient descent method is at best linear.

We can accelerate this rate if we reduce the condition number of the Hessian associated with the objective function of the problem [START_REF] Arora | More on numerical methods for unconstrained optimum design[END_REF].

Theorem 8.4.1. Let ∇ 2 z 𝑓 𝑆 (z) = Q + [𝜐(D ℎ D T ℎ ) ⊗ (L + 𝜖I) 𝛽 ] and ∇ 2 z 𝑓 𝐿 (z) = Q + [𝜐(D ℎ D T ℎ ) ⊗ L]
be the Hessian associated with the Sobolev and Laplacian time-varying graph signal reconstruction problems in (8.15) and (8.5), respectively, where Q = diag(vec (J)) ∈ R 𝑀 𝑁 ×𝑀 𝑁 and J ̸ = 0. We have that:

1. 0 ≤ 𝜆 𝑚𝑖𝑛 (∇ 2 z 𝑓 𝑆 (z)) ≤ (𝜆 𝑁 + 𝜖) 𝛽 𝜆 (𝐷)𝑁 and 0 ≤ 𝜆 𝑚𝑖𝑛 (∇ 2 z 𝑓 𝐿 (z)) ≤ 𝜆 𝑁 𝜆 (𝐷)𝑁 if 𝜆 𝑁 , 𝜆 (𝐷)𝑁 ≥ 1, where 𝜆 (𝐷)𝑁 = 𝜆 𝑚𝑎𝑥 (D ℎ D T ℎ ). 2. When 𝜐 → ∞, 𝜅(∇ 2 z 𝑓 𝑆 (z)) → ∞ and 𝜅(∇ 2 z 𝑓 𝐿 (z)) → ∞. 3. When 𝜖 → ∞ and 𝛽 > 0, or when 𝛽 → ∞ and 𝜆 𝑁 + 𝜖 > 1, 𝜆 𝑚𝑎𝑥 (∇ 2 z 𝑓 𝑆 (z)) → ∞ and so 𝜅(∇ 2 z 𝑓 𝑆 (z)) → ∞. Proof: See Appendix I.
The main result of Theorem 8.4.1 is that the upper bound of the minimum eigenvalue for the Hessian associated with GraphTRSS is looser than the corresponding upper bound of TGSR in [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] for 𝜖 > 0, 𝛽 > 1, and 𝜆 𝑁 + 𝜖 ≥ 1, which favors better condition numbers for the Sobolev problem. In practice, there is a range of values of 𝜖 and 𝛽 where 𝜅(∇ 2 z 𝑓 𝑆 (z)) < 𝜅(∇ 2 z 𝑓 𝐿 (z)), and then the GraphTRSS converges faster than the problem described in (8.5). We compute the condition numbers of the Hessian of both GraphTRSS and TGSR in four real-world datasets in Section 8.6, Fig. 8-12 to show that indeed 𝜅(∇ 2 z 𝑓 𝑆 (z)) < 𝜅(∇ 2 z 𝑓 𝐿 (z)) in a large range of 𝜖 values. Theorem 8.4.1 also shows some intuitions on how to choose the parameters for GraphTRSS and TGSR regarding convergence rate in results 2) and 3). For example, we should keep small values of 𝜐, 𝜖, and 𝛽 to avoid harming the convergence rate of these algorithms. Similarly, we should maintain small values of 𝜖 and 𝛽 to get benefits in the convergence rate of GraphTRSS. The final question is about the influence of parameter 𝛽 in the Sobolev reconstruction algorithm in (8.15). Intuitively, the parameter 𝛽 is changing the shape of the frequencies of L. For simplicity, let us assume 𝜖 = 0, and let us consider only the time-varying graph signal X without the temporal difference operator D ℎ in (8.15).

Since the matrix of eigenvectors of L is an orthogonal matrix (U T U = I), we have:

tr(X T L 𝛽 X) = tr( XT Λ 𝛽 X) = 𝑀 ∑ 𝑡=1 𝑁 ∑ 𝑖=1 x2 𝑡 (𝑖)𝜆 𝛽 𝑖 , (8.18)
where X = [x 1 , x2 , . . . , x𝑀 ] = U T X. Eqn. (8.18) is penalizing each Fourier term of each x𝑡 with powers of the eigenvalues of L. For example, when 𝛽 = 1, we penalize the higher frequencies stronger than the lower frequencies of each x𝑡 , i.e., a smooth function in 𝐺. For 𝛽 > 1, we also obtain a smooth function, but the penalization in the higher frequencies is more prominent, while for 𝛽 < 1, we get more penalization in the lower frequencies. Fig. 8-3 shows the eigenvalues of L 𝛽 for several values of 𝛽, where we normalized the eigenvalues such that (𝜆 𝑖 /𝜆 𝑁 ) 𝛽 ∀ 𝑖 ∈ 𝒱 for visualization purposes.

There are two important aspects for choosing the parameters 𝜖 and 𝛽 when using (8.15). From a graph topological point of view, we notice that adding 𝜖I to the Laplacian adds self-loops at the vertices in the graph with weights 𝜖, and from Theorem 8.4.1, we notice that the value of 𝜖 should be small to avoid harming 𝜅(∇ 2 z 𝑓 𝑆 (z)). Therefore, it is reasonable to maintain a small value of 𝜖 so that the graph topology is not heavily modified and also to avoid harming 𝜅(∇ 2 z 𝑓 𝑆 (z)). Secondly, we also know from Theorem 8.4.1 that choosing a large value of 𝛽 harms 𝜅(∇ 2 z 𝑓 𝑆 (z)). In practice, different datasets can benefit from specific penalization shapes like the ones in Fig. 8-3. As a result, we set 𝛽 = 1 in the experiments to maintain a good 𝜅(∇ 2 z 𝑓 𝑆 (z)) and be as fair as possible on the comparison with Qiu's method [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]. However, we do additional experiments where we see that different values of 𝛽 can improve the performance of (8.15) in different datasets, i.e., different datasets require different assumptions. Finally, we can conclude that TGSR by Qiu et al. [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] is a specific case of GraphTRSS when 𝜖 = 0 and 𝛽 = 1. 123

Experimental Framework

This section presents the datasets used in this chapter and the experimental framework details. We divide our experiments into three parts: 1) synthetic dataset, 2) COVID-19 datasets, and 3) environmental datasets. The graph 𝐺 can be constructed based on the coordinate locations of the nodes in each dataset with 𝑘-NN algorithm. Let M ∈ R 𝑁 ×2 be the matrix of coordinates of all nodes such that M = [m 1 , . . . , m 𝑁 ] T , where m 𝑖 ∈ R 2 is the vector with the latitude and the longitude of node 𝑖. The weights of each edge (𝑖, 𝑗) are given by the Gaussian kernel:

W(𝑖, 𝑗) = exp ( - ‖m 𝑖 -m 𝑗 ‖ 2 2 𝜎 2 ) , (8.19)
where ‖m 𝑖 -m 𝑗 ‖ 2 2 is the Euclidean distance between the vertices 𝑖 and 𝑗, and 𝜎 is the standard deviation given by 𝜎 = 1

|ℰ|

∑

(𝑖,𝑗)∈ℰ ‖m 𝑖 -m 𝑗 ‖ 2 . The Gaussian kernel in (8.19) assigns higher weights to geographically-close locations and vice versa. The construction of the graph depends on the specific application, and GraphTRSS is not sensitive to the graph construction methods.

Datasets

GraphTRSS is evaluated on one synthetic dataset and four real-world datasets including, 1) a synthetic graph, 2) global COVID-19 cases [START_REF] Dong | An interactive web-based dashboard to track COVID-19 in real time[END_REF], 3) USA COVID-19 cases [START_REF] Dong | An interactive web-based dashboard to track COVID-19 in real time[END_REF], 4) mean concentration of Particulate Matter (PM) 2.5 [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF], and 5) sea surface temperature [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF].

Synthetic Graph and Signals

We use the synthetic dataset created by Qiu et al. [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF], where 100 nodes are generated randomly from the uniform distribution in a 100 × 100 square area. Therefore, Qiu et al. [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] used a 𝑘-NN to construct the graph. The time-varying graph signal is generated with the recursive function x 𝑡 = x 𝑡-1 + L -1 2 f 𝑡 , where: 1) x 1 is a low-frequency graph signal with energy 10 4 , 2) L -1 2 = UΛ -1 2 U T , where Λ -

Global COVID-19

We use the global COVID-19 dataset provided by the Johns Hopkins University [START_REF] Dong | An interactive web-based dashboard to track COVID-19 in real time[END_REF]. This dataset contains the cumulative number of daily COVID-19 cases for 265 locations in the world (Fig. 8-1 shows the locations in the dataset), and we use the data between January 22, 2020, and November 18, 2020; i.e., 302 days. The graph is constructed with the 𝑘-NN method with 𝑘 = 10 using the coordinates of each place [START_REF] Ortega | Graph signal processing: Overview, challenges, and applications[END_REF].

USA COVID-19

We also use the USA COVID-19 dataset provided by the Johns Hopkins University [START_REF] Dong | An interactive web-based dashboard to track COVID-19 in real time[END_REF]. Fig. 8-4 shows the map with the points of the graph in the USA COVID-19 dataset. This dataset is more fine-grained than the global set since the data contain 3232 localities in the US. We use the same temporal window and 𝑘-NN method as in the global dataset. The experiments related to the US can give us a different view of our algorithm since several travel restrictions were applied between countries in the pandemic, while inside countries, there was slightly more freedom to move around.

Sea Surface Temperature

We use the sea surface temperature captured monthly and provided by the NOAA Physical Sciences Laboratory (PSL) from their website 3 . In this chapter, we use the same experimental framework as in [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] for comparison purposes, i.e., we use a subset of 100 points on the Pacific Ocean within a time frame of 600 months. Fig. 8-5 shows the locations of the spots in the sea for this dataset.

Evaluation Metrics

Experiments

For the synthetic dataset, we perform experiments analyzing several sampling densities, Signal-to-Noise Ratio (SNR) levels, smoothness 𝛼 of the synthetic time-varying graph signal, and the condition number of ∇ 2 z 𝑓 𝑆 (z) and ∇ 2 z 𝑓 𝐿 (z). For the real dataset, we perform several experiments for reconstruction and forecasting. In the same way, we study the convergence, the 𝜖 and 𝛽 parameters, and the temporal difference operators for GraphTRSS. We compare GraphTRSS with Natural Neighbor Interpolation (NNI) by Sibson [START_REF] Sibson | A brief description of natural neighbor interpolation[END_REF], Graph Regularization (GR) by Narang et al. [START_REF] Narang | Localized iterative methods for interpolation in graph structured data[END_REF], Tikhonov regularization by Perraudin et al. [START_REF] Perraudin | Towards stationary time-vertex signal processing[END_REF][START_REF] Loukas | Stationary time-vertex signal processing[END_REF], Time-varying Graph Signal Reconstruction (TGSR) by Qiu et al. [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF], and Random Sampling and Decoder (RSD) by Puy et al. [START_REF] Puy | Random sampling of bandlimited signals on graphs[END_REF]. NNI [START_REF] Sibson | A brief description of natural neighbor interpolation[END_REF] is based on Voronoi tessellation, where the interpolation is given by a linear combination of the neighbor points of the element to be interpolated in the Voronoi partition. GR [START_REF] Narang | Localized iterative methods for interpolation in graph structured data[END_REF] can be viewed as the solution of (8.5) without the time component. Tikhonov regularization [START_REF] Perraudin | Towards stationary time-vertex signal processing[END_REF] is based on joint stationarity, where the solution is constrained to be smooth in the graph and in time. RSD method [START_REF] Puy | Random sampling of bandlimited signals on graphs[END_REF] solves an optimization problem where we have smoothness in the graph and a probability distribution function for the sampling. The role of RSD and GR in the experiments is to see how these static graph signal methods compare to time-varying graph signal algorithms. Similarly, since RSD and GR cannot reconstruct graph signals where we do not have any sample at a specific time 𝑡, they are not included in some of the experiments as in forecasting. GR, Tikhonov, TGSR, and GraphTRSS have a maximum number of 20000 iterations in the optimization algorithm for a fair comparison.

We optimize the parameters of each method in all experiments for a fair comparison.

In the reconstruction experiments, we adopt two sampling strategies. The first reconstruction experiment performs a random sampling on each temporal snapshots for the number of nodes 𝑁 , i.e., we randomly select a percentage of nodes on each time step as in [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]. For RSD by Puy et al. [START_REF] Puy | Random sampling of bandlimited signals on graphs[END_REF], we use their optimal sampling procedure because their reconstruction algorithm depends on their specific sampling distribution. Puy et al. [START_REF] Puy | Random sampling of bandlimited signals on graphs[END_REF] proposed to estimate the reconstructed static graph signal x rec by solving the following problem:

x rec = arg min z ‖P -1 2 (Mz -x(𝒮))‖ 2 + 𝜂z T 𝑔(L)z, (8.20)
where 𝜂 ∈ R + , 𝑔 : R → R is a non-negative and non-decreasing polynomial function, M is the sampling matrix, 𝒮 is the set of sampled nodes, and P ∈ R 𝑁 ×𝑁 is a random matrix that is designed jointly with M. RSD approach samples nodes using the diagonal of P as the probability distribution where the nodes are drawn, while the sampling method we use for the other methods uses a uniform distribution. In the current chapter, we use the optimal computation of P that requires the spectral decomposition of L. However, for large graphs, this optimal approach may be computationally prohibitive. good reconstruction errors after this sampling density on average. However, notice that real-world datasets are not strictly bandlimited (as shown in Fig. 8-6), and we have noise, so perfect reconstruction is not possible.

We evaluate each method with a Monte Carlo cross-validation with 100 repetitions for each sampling density (50 repetitions for the COVID-19 USA dataset). Deterministic sampling methods do not work well with these reconstruction methods as stated in Section 8.3.

In the forecasting experiment, we compute the error metrics for several temporal snapshots 𝑡 in the set {1, 2, . . . , 10}. For example, since COVID-19 datasets are sampled daily, we will predict the new COVID-19 cases in the last day, in the two last days, and so on until the last ten days. However, for the sea surface temperature, we will predict up to ten months since this dataset is sampled monthly, i.e., we are interpolating multiple time steps ahead in all datasets.

We perform some studies to analyze the parameters of GraphTRSS. The first of these studies computes the average RMSE (in ℳ) and the average number of iterations required to satisfy the stopping condition with variations in 𝜖. The second study performs reconstruction with the three temporal difference operators (one, two, and three steps). The last study computes reconstruction with several values of 𝛽.

These studies are evaluated with a Monte Carlo cross-validation with ten repetitions.

We also compare the loss function vs the iteration number for TGSR and Graph-TRSS in reconstruction, where the loss is the evaluation of (8.5) and (8.15) at each iteration. In this case, 50 repetitions for each sampling densities in ℳ are performed, where each repetition is computed with the best parameters of each method. For a fair comparison, GraphTRSS and Qiu's method use the same sampling matrix J. We also compute the running times for several sampling densities. Finally, we compute 𝜅(∇ 2 z 𝑓 𝑆 (z)) and 𝜅(∇ 2 z 𝑓 𝐿 (z)) for 𝛽 = 1 and several values of 𝜖 in all real datasets. All experiments were executed in MATLAB R2017b on a 2.3GHz MacBook Pro with 16GB memory. The code of GraphTRSS has been made available4 .

Results and Discussion

This section presents the results of the experiments on the selected datasets. GR and RSD methods cannot reconstruct entire temporal snapshots because these algorithms only use spatial information. Therefore, GR and RSD are not included in the experiments of entire snapshots sampling and forecasting. A detailed analysis for each dataset is provided as follows. GraphTRSS is better than the other methods for the reconstruction experiments when using different sampling strategies. We can notice that the results in the COVID- the two-time steps temporal operator D ℎ2 shows a big performance improvement compared to the one-step temporal operator for the sea surface temperature dataset.

Synthetic Graph and Signals

Environmental Datasets

Perhaps, we could propose other temporal operators and more elaborated optimization functions based on several 𝛽 parameters, which we leave for future work.

Additional Analysis

Table 8.2 shows the number of iterations that TGSR method and GraphTRSS require to satisfy the stopping condition in (8.17). Table 8. 𝛽 𝜆 (𝐷)𝑁 for 𝜆 𝑚𝑎𝑥 (L), 𝜆 𝑚𝑎𝑥 (Δ), 𝜆 (𝐷)𝑁 ≥ 1. However, notice that the eigenvalue spreading is smaller if 𝜆 𝑚𝑎𝑥 (Δ) < 𝜆 𝑚𝑎𝑥 (L), i.e., we can have better convergences rates in both GraphTRSS and TGSR method [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]. Fig. 8-13 shows a small experiment where we compared GraphTRSS and TGSR method [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] when using normalized (Δ) and unnormalized (L) Laplacian matrices. For the comparison with Δ and L, we notice that in both datasets and for both GraphTRSS and TGSR, the convergence rate is faster when using the normalized Laplacian as shown in Fig. 8 Notice that the improvement in the condition number depends on the specific duce a new optimization problem for reconstruction from samples. We showed the good convergence properties of GraphTRSS by relying on the condition number of the Hessian associated with our problem, as shown in Theorem 8.4.1. Our algorithm improves convergence rate without relying on expensive matrix inversions or eigenvalue decompositions. GraphTRSS shows promising results for the estimation of new COVID-19 cases in both global and USA datasets, as well as in the reconstruction of environmental variables (PM 2.5 concentration and sea surface temperature). Graph-TRSS could be useful in several scenarios, for example, 1) one could try to estimate the number of new COVID-19 cases where we have missing or unreliable information, 2) we can use our algorithm as a pre-processing step to get more reliable data for further forecasting of new cases with other well-established models of infectious diseases [START_REF] Hethcote | The mathematics of infectious diseases[END_REF], and 3) we can also use GraphTRSS as a pre-processing step for further forecasting of new COVID-19 cases with other graph-based techniques such as [START_REF] Isufi | Forecasting time series with VARMA recursions on graphs[END_REF].

GraphTRSS is also evaluated directly on forecasting, showing promising results.

This chapter shows several insights for possible future directions. For example, our algorithm and other GSP techniques could potentially increase the performance of various models in infectious disease (not only for COVID-19) for forecasting and imputation of data, among others. In the same way, this chapter could be the foundation for applying other GSP-based methods in virology. These methods can be, for example, graph neural networks [START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF][START_REF] Bronstein | Geometric deep learning: going beyond euclidean data[END_REF] and learning graphs from data [START_REF] Dong | Learning graphs from data: A signal representation perspective[END_REF].

Chapter 9 Conclusions

We have studied several problems in image and video processing, graph neural networks, and time-varying graph signals. This thesis has been divided into three main parts: 1) computer vision in Chapters 3-5, 2) machine learning in Chapters 6 and 7, and 3) signal processing in Chapter 8. Specifically, addressed some fundamental questions like: 1) the sample complexity of semi-supervised algorithms based on graph in Chapter 3, 2) the definition of sparse norms for GNNs in Chapter 6,3) the relationship between over-smoothing and over-squashing in Chapter 7, and 4) the convergence rate of graph reconstruction algorithms in Chapter 8. Similarly, we have addressed the applications of: 1) moving object segmentation, 2) semantic segmentation, 3) cancer detection in images, 4) text classification, 5) action recognition, 6) node classification in graphs, and 7) reconstruction of time-varying graph signals for COVID-19 and environmental data. We proposed several algorithms that either: 1) reached state-of-the-art performances at the time of publication, or 2) explained certain important phenomena in data science.

Even though there is a big variety in the topics addressed in this thesis, most of them can be motivated by the desire of solving problems in computer vision, machine learning, or signal processing using limited supervision. To that end, we have used concepts from spectral graph theory, graph signal processing, graph neural networks, and also differential geometry specifically in Chapter 7. We summarize several ideas for future work that can be derived from this thesis as follows: bins is computed between -𝜋 and 𝜋 for 𝜃, and between 0 and 30 for m. Let 𝑓 𝑂𝐹 : a → R 1×49 be the function representing the histogram extraction process for the optical flow, then the whole representation of the motion mov 𝑣 ∈ R 1×109 of a node 𝑣 is obtained as follows [𝑓 𝑂𝐹 (𝜃), 𝑓 𝑂𝐹 (m), max(m), m, max(m) -min(m), std(m), mad(m), min(𝜃), max(𝜃), θ, max(𝜃)-min(𝜃), std(𝜃), mad(𝜃)], where ā, std(a), and mad(a) are the mean, the standard deviation, and the mean absolute deviation of vector a, respectively. of a node 𝑣 is obtained as dl 𝑣 = 𝑓 𝐷𝐿 (I 𝑡 𝑣 ). 5. Finally, the feature vector x 𝑣 ∈ R 1×11669 is obtained as follows [lbp 𝑣 , mov 𝑣 , in 𝑣 , dl 𝑣 ].

• PairNorm: Flag to activate the PairNorm normalization.

• DiffGroupNorm: Flag to activate the differentiable group normalization.

• DropEdge: Flag to activate the DropEdge method.

• JostLiuCurvature: Flag to activate our SJLR algorithm.

• FALayer: Flag to activate the fully adjacent layer.

• GraphDifussion: Flag to activate the graph diffusion convolution with personalized PageRank kernel.

• RicciCurvature: Flag to activate the SJLR method.

• GNN: String to determine the type of GNN (only 'GCN' is included in the first version of the code).

• dataset: String to determine the dataset for the experiment with the options 'Cora', 'Citeseer', 'Pubmed', 'Cornell', 'Texas', 'Wisconsin', 'chameleon', 'squirrel', 'Actor', 'Computers', 'Photo', and 'CoauthorCS'.

The main file has a broad set of arguments for hyperparameter search or for executing methods with specific hyperparameters-further details can be found in the main Python file in the repository. For example, we can reproduce the results of the best hyperparameters found in [START_REF] Klicpera | Diffusion improves graph learning[END_REF] for GDC with personalized PageRank kernel using the following command: (1 -𝜆 𝑖 ) 𝑠 𝑎 𝑖 𝜑 𝑖 D

Figure 0 -

 0 Figure 0-1 shows a graphical summary of the main contributions of this thesis.

  propose a new reconstruction algorithm for graph signals based on the extension of a Sobolev smoothness function from static to time-varying graph signals.

Figure 0 - 1 : 1

 011 Figure 0-1: Main contributions presented in this thesis.

3 - 4 3 - 6 4 - 2 4 - 3 5 - 1 5 - 2 5 - 4

 34364243515254 methods on the sequence fall taken from the CDNet2014 dataset. The green-colored cars in (b), instances in different colors in (c), and homogeneous regions in (d) represent the nodes of the graph. . . . . . . . . . . . . . . . . . . Procedure to represent the nodes of the graph with a Mask R-CNN as backbone. Each mask of the segmented image represents a node in the graph, and the representation of the node is achieved with intensity, optical flow, texture, and deep features. . . . . . . . . . . . . . . . . . 3-5 Comparison of the qualitative results of GraphMOS on CDNet2014 and UCSD datasets with existing state-of-the-art methods. Our algorithm performs better than the state-of-the-art methods in these challenging scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results of the experiment related to the sample complexity. Left: power spectrum of the graph signal ŷ2 related the moving objects, right: classification error vs sample size of the semi-supervised learning algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1 GraphMOS-Net uses background initialization and instance segmentation. Each instance represents a node in a graph using motion, intensity, and texture features. Finally, a GCN classifies if each node is a moving or static object with an unseen scheme. . . . . . . . . . . . . Visual results in CDNet2014 . . . . . . . . . . . . . . . . . . . . . . . Visual results in UCSD . . . . . . . . . . . . . . . . . . . . . . . . . . The idea of HyperGCN-WSS is to rely both on the spatial and structural information in the datasets. . . . . . . . . . . . . . . . . . . . . The pipeline of HyperGCN-WSS. Our algorithm uses SLIC superpixel segmentation, VGG16 feature extraction, average pooling, spatial and 𝑘-NN graph construction, a specialized HyperGCN architecture, and a DeepLabV3+ model. . . . . . . . . . . . . . . . . . . . . . . . . . . xx 5-3 HyperGCN-WSS architecture with skip connections, as well as several graph and hypergraph convolutional layers. X is the matrix of features from VGG16. HyperGCN-WSS is trained in three stages, where we have three loss functions ℒ 𝑖 . . . . . . . . . . . . . . . . . . . . . . . . Some visual results on PASCAL VOC 2012 with our HyperGCN-WSS, using scribbles or clicks as weak signals. . . . . . . . . . . . . . . . . .

6 - 2 7 - 1 7 - 2 107 xxi 8 - 1 8 - 3 8 - 4 8 - 7

 62717210781838487 (b) the S-SobGNN architecture. Finally, a loss function (such as the cross-entropy) is computed for the training procedure. . . . . . . . . . Basic configuration of our S-SobGNN architecture with 𝑛 layers and 𝛼 filters per layer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-3 Average training time per epoch (T-Epoch) for several GNNs on 20News with variations in the number of nodes 𝑁 . . . . . . . . . . . . . . . . Mixing steps 𝑓 (𝜆 2 , 𝜖) for 𝜖 = 5 × 10 -4 vs. number of removed or added edges for two stochastic block model graphs with a) two clusters, b) five clusters, and c) one Erdős-Rényi graph. . . . . . . . . . . . . . . The pipeline of the proposed Stochastic Jost and Liu curvature Rewiring (SJLR) algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3 Average running time for balanced Forman curvature and Jost and Liu curvature for variations in the number of nodes in three artificial graphs.The framework of our algorithm (GraphTRSS) using a matrix of coordinates M ∈ R 𝑁 ×2 to construct a graph with 𝑁 regions in the world with confirmed cases of COVID-19 by November 18, 2020. The graph is constructed with a 𝑘-NN method. GraphTRSS uses the operator D ℎ ∈ R 𝑀 ×(𝑀 -1) to capture temporal information in the time-varying signal X ∈ R 𝑁 ×𝑀 with 𝑀 temporal snapshots, and it also uses different sampling strategies J ∈ {0, 1} 𝑁 ×𝑀 according to the desired output (reconstruction or forecasting). Finally, the optimization function, which includes the error and Sobolev terms, reconstructs or predicts the missing values, i.e., the indexes of X where J has values zero. . . 8-2 Contour plots of two error surfaces of well and ill-conditioned problems showing the evolution of a gradient descent method. . . . . . . . . . . Eigenvalue penalization of the Laplacian matrix for different values of 𝛽 from the dataset of COVID-19. . . . . . . . . . . . . . . . . . . . . Graph with the places in the United States in the Johns Hopkins University dataset [21]. The graph was constructed with a 𝑘-NN with 𝑘 = 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-5 Graph with the spots in the sea for the dataset of temperature. The graph was constructed with a 𝑘-NN with 𝑘 = 10. . . . . . . . . . . . 8-6 Graph Fourier transform of some elements of XD ℎ for COVID-19 global dataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparison of GraphTRSS with several methods in the literature on synthetic data for four experiments on: a) reconstruction with several sampling densities, b) variation of the SNR, c) variation of the smoothness level in (8.4), and d) 𝜅(∇ 2 z 𝑓 𝑆 (z)) and 𝜅(∇ 2 z 𝑓 𝐿 (z)) with several values of 𝜖 and 𝛽. Our algorithm is compared with Natural

xxii 8 - 8

 88 Comparison of GraphTRSS with several methods in the literature on COVID-19 global dataset for several experiments in terms of: a) random sampling, b) entire snapshots sampling, c) forecasting, d) variation of parameter 𝜖, e) variation of parameter 𝛽, f) several temporal difference operators, g) convergence comparison, and h) running time. 8-9 Comparison of GraphTRSS with several methods in the literature on COVID-19 USA dataset for several experiments in terms of: a) random sampling, b) entire snapshots sampling, c) forecasting, d) variation of parameter 𝜖, e) variation of parameter 𝛽, f) several temporal difference operators, g) convergence comparison, and h) running time. . . . . . 8-10 Comparison of GraphTRSS with several methods in the literature in the PM 2.5 dataset for several experiments in terms: a) random sampling, b) entire snapshots sampling, c) forecasting, d) variation of parameter 𝜖, e) variation of parameter 𝛽, f) several temporal difference operators, g) convergence comparison, and h) running time. . . . . . 8-11 Comparison of GraphTRSS with several methods in the literature in the sea surface temperature dataset for several experiments in terms of: a) random sampling, b) entire snapshots sampling, c) forecasting, d) variation of parameter 𝜖, e) variation of parameter 𝛽, f) several temporal difference operators, g) convergence comparison, and h) running time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-12 Condition number of the Hessian associated with the optimization problems for GraphTRSS (𝜅(∇ 2 z 𝑓 𝑆 (z))) and TGSR [26] (𝜅(∇ 2 z 𝑓 𝐿 (z))) for all datasets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-13 Normalized Δ and unnormalized L Laplacian experiments in COVID-19 global and sea surface temperature, a) reconstruction error using Δ, b) loss vs number of iterations using Δ, and c) loss vs number of iterations using L. . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 Architecture of Mask R-CNN [20] for GraphMOS. . . . . . . . . . . . xxiii xxiv List of Tables 3.1 Summary of the parameters used in our experiments for each dataset.

3. 2

 2 Comparisons of average F-measure on CDNet2014 dataset. . . . . . . 3.3 Comparison of F-measure results over the sequences of I2R dataset. . 3.4 Comparison of F-measure results over the sequences of SBI2015 dataset. 3.5 Comparison of F-measure results over the videos of UCSD background subtraction dataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Performance comparisons in terms of average F-measure score for different segmentation methods used for graph construction. Only handcrafted features are used to report the performance. These ablations studies involves: graph construction using DeepLab with ResNet 101 (DeepLab), Mask R-CNN with ResNet 50 (Mask R-50), Cascade Mask R-CNN with ResNeSt 200 (Cascade RS-200), SuBSENSE, and Superpixel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Performance comparison in terms of average F-measure score of superpixel and block-based segmentation for graph construction methods. The performance is reported by using both handcrafted and deep features representation of graph nodes. . . . . . . . . . . . . . . . . . . . 3.8 Performance comparisons in terms of average F-measure score on five datasets using distinct node features representations. Handcrafted, deep features, and the concatenation of handcrafted and deep features (Hand + Deep (Conv-5)) are used to represent graph nodes. . . . . . 3.9 Performance comparisons in terms of average F-measure score for the number of superpixels in the SLIC method. . . . . . . . . . . . . . . . xxv 3.10 Average F-measure with variations in the construction of the graph. This ablation involves: 𝑘-NN with 𝑘 = 40, 𝑘 = 30, 𝑘 = 20, and 𝑘 = 10. 3.11 Average F-measure with variations in the semi-supervised learning algorithm. This ablation involves: Sobolev minimization (Sob.) with 𝜖 = 50, 𝜖 = 0.5, 𝜖 = 0.2, and Total Variation minimization (TV). . . . 4.1 Comparisons of average F-measure over nine challenges of CDNet2014. GraphMOS-Net is compared with unsupervised and supervised algorithms in MOS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Comparison of F-measure results over the videos of UCSD background subtraction dataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Accuracy in mIoU (%) in the train set of PASCAL VOC after each loss function ℒ 𝑖 ∀ 𝑖 ∈ {1, 2, 3} in our algorithm. . . . . . . . . . . . . . . . 5.2 Accuracy of HyperGCN-WSS and previous methods in the validation set of PASCAL VOC. S: Scribbles. C: Clicks. . . . . . . . . . . . . . 6.1 Accuracy (in %) for several state-of-the-art methods and our proposed S-SobGNN architecture in several datasets for semi-supervised learning, inferring the graphs with 𝑘-NN and the protocol to learn the graph as in Section 6.3.3. #Param is the number of learnable parameters, Acc. means accuracy, s.d. denotes standard deviation, T-Epoch is the average time per epoch. The best and second-best performing methods on each category are shown in red and blue, respectively. . . . . . . 6.2 Accuracy (in %) for several state-of-the-art methods and our proposed S-SobGNN architecture in several datasets for semi-supervised learning, inferring the graphs with 𝑘-NN and the protocol to learn the graph as in Section 6.3.3. All methods have approximately the same number of learnable parameters. . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Homophily index for the datasets for semi-supervised learning with different values of 𝑘. . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi 6.4 Accuracy (in %) of S-SobGNN with variations of the parameters 𝜖 and 𝛼, as well as variations of the parameter 𝑘 for the 𝑘-NN and the algorithm to learn graphs (Lrnd.) in all datasets for semi-supervised learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Sparsity percentage for the sparse and non-sparse Sobolev filtering matrices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Benchmarking results for several state-of-the-art methods and S-SobGNN, where L denotes the number of hidden layers. The results are averaged over four runs with four different seeds. . . . . . . . . . . . . . . . . . 7.1 Results of the first experiment regarding the comparison of our SJLR algorithm with several state-of-the-art methods to alleviate over-smoothing and over-squashing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Summary of the average error metrics in the real datasets for several sampling schemes. The best and second-best performing methods on each category are shown in red and blue, respectively. . . . . . . . . 8.2 The average number of iterations to satisfy the stopping condition for GraphTRSS and TGSR [26]. . . . . . . . . . . . . . . . . . . . . . . . H.1 Best hyperparameters for Cornell dataset. . . . . . . . . . . . . . . . H.2 Best hyperparameters for Texas dataset. . . . . . . . . . . . . . . . . H.3 Best hyperparameters for Wisconsin dataset. . . . . . . . . . . . . . . H.4 Best hyperparameters for Chameleon dataset. . . . . . . . . . . . . . H.5 Best hyperparameters for Squirrel dataset. . . . . . . . . . . . . . . . H.6 Best hyperparameters for Actor dataset. . . . . . . . . . . . . . . . . H.7 Best hyperparameters for Cora dataset. . . . . . . . . . . . . . . . . . H.8 Best hyperparameters for Citeseer dataset. . . . . . . . . . . . . . . . H.9 Best hyperparameters for Pubmed dataset. . . . . . . . . . . . . . . . xxvii xxviii Dedicated to my beloved family: Camila, Gloria, Heriberto, and Diana.

Finally, part 3 1 .

 31 presents a new algorithm for the reconstruction of time-varying graph signals. This algorithm converges faster than other methods in the literature without requiring expensive eigenvalue decompositions or matrix inversions. This signal processing approach is evaluated for the reconstruction of COVID-19 and environmental data. The contributions of this thesis have been driven by the following research questions: Can concepts of GSP be used in the problem of Moving Object Segmentation (MOS) under the paradigm of learning with minimal supervision? We introduce Graph Moving Object Segmentation (GraphMOS) in Chapter 3. Graph-MOS uses concepts of sampling and reconstruction in GSP to address MOS. GraphMOS takes a minimal supervision approach leading to a new family of semi-supervised learning algorithms in MOS. GraphMOS is inspired in the reconstruction of graph signals to solve the semi-supervised learning problem.

3 . 4 .

 34 Can we address semantic segmentation using the concepts developed in previous chapters? We introduce HyperGraph Convolutional Networks for Weaklysupervised Semantic Segmentation (HyperGCN-WSS) in Chapter 5. Our algorithm uses hypergraphs to capture spatial information in the images, as well as structural information in the dataset. HyperGCN-WSS solves the problem of semantic segmentation with minimal supervision using weak signals like scribbles and clicks. HyperGCN-WSS is evaluated on the PASCAL VOC 2012 dataset. How can we improve the expressiveness of GNNs while keeping scalability? We introduce Sparse Sobolev Graph Neural Networks (S-SobGNNs) in Chapter 6.

6 .

 6 How can we increase the convergence rate of GSP algorithms for reconstruction? We introduce Time-varying Graph signals Reconstruction via Sobolev Smoothness (GraphTRSS) in Chapter 8. GraphTRSS converges faster than previous methods for reconstruction of time-varying graph signals, without requiring expensive eigenvalue decompositions or matrix inversions. GraphTRSS is evaluated on several datasets including two COVID-19 datasets and two environmental datasets, outperforming many existing methods for time-varying graph signal reconstruction.In addition to the contributions listed above, we include in Chapter 2 the mathematical notation and background of this thesis. After presenting the main body of this thesis in Parts 1, 2, and 3, we conclude and show potential directions for future work in Chapter 9.

{𝑠 1 ,

 1 𝑠 2 , . . . , 𝑠 𝑚 }, where 𝑚 = |𝒮| ≤ 𝑁 is the number of sampled nodes. The sampled graph signal is defined as y(𝒮) = My, where M is a binary decimation matrix whose entries are given by M = [𝛿 𝑠 1 , . . . , 𝛿 𝑠𝑚 ] T and 𝛿 𝑣 is the 𝑁 -dimensional Kronecker column-vector centered at 𝑣. The recovery of a graph signal from its samples y(𝒮) can be represented as ỹ = ΦMy, where Φ ∈ R 𝑁 ×𝑚 is an interpolation matrix. Perfect recovery is achievable if ΦM = I, i.e., ỹ = Iy = y. Since rank(ΦM) ≤ 𝑚 ≤ 𝑁 , perfect reconstruction is not possible in general. However, perfect reconstruction from a sampled graph signal y(𝒮) is possible when the sampling size |𝒮| ≥ 𝜌[START_REF] Chen | Discrete signal processing on graphs: Sampling theory[END_REF].

Figure 3 - 1 :

 31 Figure 3-1: Comparisons of the visual results of the proposed Graph Moving Object Segmentation (GraphMOS) algorithm with existing state-of-the-art methods on five MOS challenging video sequences taken from CDNet2014[10] and UCSD[START_REF] Mahadevan | Spatiotemporal saliency in dynamic scenes[END_REF] datasets. The compared methods are: PAWCS[START_REF] St-Charles | A self-adjusting approach to change detection based on background word consensus[END_REF], IUTIS-5[START_REF] Bianco | Combination of video change detection algorithms by genetic programming[END_REF], BSUV-Net[START_REF] Tezcan | BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos[END_REF], ROSL[START_REF] Shu | Robust orthonormal subspace learning: Efficient recovery of corrupted low-rank matrices[END_REF], and DECOLOR[START_REF] Zhou | Moving object detection by detecting contiguous outliers in the low-rank representation[END_REF]. Our proposed algorithm performs significantly better than the compared methods in these challenging sequences.

Figure 3 - 2

 32 shows an overview of GraphMOS. GraphMOS consists of several components including (a) superpixel segmentation, (b) background model initialization, (c) features extraction, (d) graph construction, (e) graph signal representation, (f) sampling of graph signals with an unseen scheme, and (g) a semi-supervised algorithm inspired from the theory of GSP.

Figure 3 - 2 :

 32 Figure 3-2: The pipeline of the MOS algorithm with the reconstruction of graph signals. The algorithm uses background initialization and superpixel segmentation[START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF][START_REF] Yang | Superpixel segmentation with fully convolutional networks[END_REF]. Each superpixel represents a node in a graph, and the representation of each node is obtained with motion, intensity, texture, and deep features. The groundtruth is used to decide if a node is a moving (green nodes) or a static object (blue nodes). Black nodes correspond to the non-labeled images in the dataset. Finally, some nodes are sampled and the semi-supervised algorithm reconstructs all the labels in the graph.

Figure 3 - 3 :

 33 Figure 3-3: Results of the semantic, instance, and superpixel segmentation using DeepLab [19], Mask R-CNN [20], and SLIC [17] methods on the sequence fall taken from the CDNet2014 dataset. The green-colored cars in (b), instances in different colors in (c), and homogeneous regions in (d) represent the nodes of the graph.

Figure 3 - 4 :

 34 Figure 3-4: Procedure to represent the nodes of the graph with a Mask R-CNN as backbone. Each mask of the segmented image represents a node in the graph, and the representation of the node is achieved with intensity, optical flow, texture, and deep features.

  Given the graph signals related to the problem of MOS and the notions of bandlimitedness in terms of 𝑃 𝑊 𝜔 (𝐺) from Section 2.2, we want to know what is the minimum amount of labeled nodes required to have a perfect classification in semi-supervised learning, given the prior assumption that the labels of the nodes are in the Paley-Wiener space of the graph? Similar to the sampling and reconstruction of graph signals in Section 2.

A

  thorough comparison with state-of-the-art methods, along with a set of comprehensive ablation studies of GraphMOS are performed. The ablation studies analyze some specific elements of the pipeline in Fig.3-2, like the segmentation method, the number of superpixels in the case of SLIC, the feature extraction procedure, the construction of the graph and the semi-supervised learning algorithm. The instance segmentation methods are performed with Mask R-CNN using ResNet-50 as the backbone, while Cascade Mask R-CNN is performed using

Fig. 3 -Figure 3 - 5 :

 335 Fig. 3-5 shows some of the visual results of the proposed algorithms compared with several state-of-the-art methods on CDNet2014 and UCSD datasets. The CDNet2014

  10 summarizes the performance of GraphMOS with several configurations of the construction of the graph in the CDNet2014 dataset. The results suggest that: the challenges dynamic background, intermittent object motion, low frame rate, and thermal have the better results with 𝑘 = 10; the challenges camera jitter and shadow have the better results with 𝑘 = 20; and finally the results with 𝑘 = 30 and 𝑘 = 40 are very similar. These differences between the results can be partially explained because for 𝑘 = 10 and 𝑘 = 20 some sub-graphs are generated, i.e., for the semisupervised learning algorithm is executed independently for each sub-graph. On the other hand, 𝐺 is connected for 𝑘 = 30 and 𝑘 = 40. As a consequence, for low values of 𝑘, GraphMOS is losing global information of the database. The good results in the challenge low frame rate motion can be explained in part because of the nature

Figure 3 - 6 :

 36 Figure 3-6: Results of the experiment related to the sample complexity. Left: power spectrum of the graph signal ŷ2 related the moving objects, right: classification error vs sample size of the semi-supervised learning algorithm.

Figure 3 -

 3 Figure 3-6 shows the results of the experiment related to the sample complexity. The first plot in Fig. 3-6 shows the power spectrum of the graph signal associated with the moving objects, where the power spectrum is defined as the GFT square ŷ2 𝑖 . The second plot shows the classification error vs the sample size, and the bound of the sample complexity in the problem computed as in (3.1). The sample complexity computed in this experiment is 𝑁 𝑠 ≤ 59 with corresponding cutoff frequency 𝜔 = 0.0089. The average errors are 0.0576 and 0.0393 for the sampling sizes 𝑚 = 60 and 𝑚 = 400, respectively, i.e., the change in the classification error is bounded by 𝜑 = 0.0576 after the sample complexity. One can notice from Fig. 3-6 that the classification error just has a small change after a sample size of 𝑁 𝑠 .

  tion, background initialization, features extraction for the representation of the nodes in a graph, construction of the graph, and finally semi-supervised learning algorithms inspired by the theory of the graph signals reconstruction. In the same way, several theoretical insights about the sample complexity and the graph signals reconstruction are explored in this chapter. More specifically, Corollary 3.3.1 is introduced, showing a bound for the sample complexity given a smoothness prior assumption. The proposed algorithms are evaluated on four publicly available MOS datasets. Through an extensive series of experiments, the proposed algorithms have consistently outperformed existing state-of-the-art methods by a significant margin. This chapter opens several future research directions in computer vision and machine learning. The first important direction is to further explore a generalized theory of graph signal processing in the field of MOS. The graph signals can be extended to fuzzy concepts leading to a richer representation of moving and static objects.

Chapter 4

 4 addresses the problem of training a neural network capable of learning from the labels and the structure of a dataset for MOS. Similarly, Chapter 5 focuses on the problem of using GNNs in the problem of semantic segmentation.

4. 1

 1 IntroductionWe introduced GraphMOS in Chapter 3 for the MOS problem. GraphMOS presents several advantages regarding previous supervised and unsupervised methods for MOS.

Figure 4 - 1 :

 41 Figure 4-1: GraphMOS-Net uses background initialization and instance segmentation. Each instance represents a node in a graph using motion, intensity, and texture features. Finally, a GCN classifies if each node is a moving or static object with an unseen scheme.

  4.2.1 Segmentation, Feature Extraction, and Graph ConstructionThe first stages of GraphMOS-Net are similar to those of GraphMOS in Chapter 3. For segmentation, we use a Cascade Mask R-CNN[START_REF] Cai | Cascade R-CNN: High quality object detection and instance segmentation[END_REF] with ResNeSt of 200 layers[START_REF] Zhang | ResNeSt: Split-attention networks[END_REF]. The feature extraction uses the handcrafted features of GraphMOS as explained in Section 3.3.2 and Appendix B, i.e., no deep features are considered in this chapter. Finally, the construction of the graph is done with the 𝑘-NN method as in Section 3.3.3 with 𝑘 = 30.

For

  CDNet2014, GraphMOS-Net constructs a graph for the whole dataset, resulting in an undirected graph of 310289 nodes. For each partition of the dataset a percentage of the total amount of nodes in the set ℳ is used for training. For example, 𝒮 contains 310 nodes for 0.1% ∈ ℳ (keep in mind that usually each image contains several nodes as shown in Fig. 4-1). The training of our GCN is validated with a Monte Carlo cross-validation with 3 repetitions for each training density and split of the database. As a consequence, we train 4 × 3 × |ℳ| = 48 GCNs for the validation of GraphMOS-Net. For UCSD, GraphMOS-Net constructs a graph with the whole UCSD dataset plus the challenges baseline, dynamic background, shadow, and PTZ of CDNet2014; resulting in a graph of 104261 nodes. In this case, the test set is the whole UCSD dataset, and a small number of nodes in the set ℳ is used for training. A Monte Carlo cross-validation with 5 repetitions is performed for each training density, i.e., we train 5 × |ℳ| = 20 GCNs for UCSD dataset. For the sake of clarification, the metrics in the experiments are computed on the unseen set of nodes in each experiment. The CDNet2014 and UCSD experiments follow the guidelines for the training, validation, and testing sets of Section 4.3.2.

Figure 4 - 2 :

 42 Figure 4-2: Visual results in CDNet2014

Fig. 4 -Figure 4 - 3 :

 443 Fig. 4-2 shows some visual results of SubSENSE, PAWCS, IUTIS-5, BSUV-Net, and GraphMOS-Net in the CDNet2014 dataset. The sequences present challenging scenarios such as bad weather conditions, dynamic backgrounds, and PTZ cameras, among others. GraphMOS-Net presents high-quality visual results in all cases. In

4. 5 Conclusions

 5 This chapter introduces a new algorithm for MOS based on GCN. The pipeline of the method involves a Cascade Mask R-CNN to get the object instances from the videos; a graph construction with k-nearest neighbors, where nodes are associated to object instances, and represented with feature vector encompassing optical flow, intensity and texture descriptors. We use a compact GCN model trained in a semi-supervised setting. Our algorithm outperforms previous state-of-the-art unsupervised, semisupervised, and supervised learning algorithms for MOS in several challenges of the CDNet2014 and UCSD datasets.

5. 1 Introduction

 1 We introduced GraphMOS-Net in Chapter 4. GraphMOS-Net deals with the limitation of the former GraphMOS from Chapter 3, where the semi-supervised learning problem should be solved again when new nodes are added to the optimization function. In this chapter, we propose a new graph-based learning algorithm for the task of Weakly-supervised Semantic Segmentation (WSS).

Figure 5 - 1 :

 51 Figure 5-1: The idea of HyperGCN-WSS is to rely both on the spatial and structural information in the datasets.

Figure 5 - 2 :

 52 Figure 5-2: The pipeline of HyperGCN-WSS. Our algorithm uses SLIC superpixel segmentation, VGG16 feature extraction, average pooling, spatial and 𝑘-NN graph construction, a specialized HyperGCN architecture, and a DeepLabV3+ model.

Fig. 5 -

 5 Fig. 5-2 shows the pipeline of HyperGCN-WSS, where we have superpixel segmentation, feature extraction, hypergraph construction, HyperGCN, and a DeepLab model for semantic segmentation.

A

  simple graph 𝐺 = (𝒱, ℰ) is a mathematical entity where we have a set of nodes 𝒱 ∈ {1, . . . , 𝑁 } and a set of edges ℰ = {(𝑖, 𝑗)} as explained in Section 2. Notice that A (the adjacency matrix) can only represent edges that connect two nodes. A hypergraph 𝐺 ℎ = (𝒱, ℰ) is a generalization of simple graphs 𝐺, where the edges can connect multiple nodes. Let W 𝑒 ∈ R 𝑀 ×𝑀 be the diagonal matrix of hyperedge weights, where W 𝑒 (𝑒, 𝑒) is the weight of the 𝑒th hyperedge and 𝑀 = |ℰ|. Let H 𝑖 ∈ {0, 1} 𝑁 ×𝑀 be the incidence matrix of 𝐺 ℎ such that H 𝑖 (𝑖, 𝑒) = 1 if the 𝑖th node is incident to the edge 𝑒, and 0 otherwise, i.e., H 𝑖 (𝑖, 𝑒) = 1 if the node 𝑖 is connected by the edge 𝑒. Let D ℎ ∈ R 𝑁 ×𝑁 be the diagonal matrix of node degree, where D ℎ (𝑖, 𝑖) = ∑ 𝑀 𝑒=1 W 𝑒 (𝑒, 𝑒)H 𝑖 (𝑖, 𝑒). Finally, let B ∈ R 𝑀 ×𝑀 be the diagonal matrix of hyperedge degree, where B(𝑒, 𝑒) = ∑ 𝑁 𝑖=1 H 𝑖 (𝑖, 𝑒). In the current chapter, we use the hypergraphs to represent two kinds of relationships in the nodes: 1) the spatial relationships of the nodes on each image and 2) the relationship of nodes from different images in the dataset.

Figure 5 - 3 :

 53 Figure 5-3: HyperGCN-WSS architecture with skip connections, as well as several graph and hypergraph convolutional layers. X is the matrix of features from VGG16. HyperGCN-WSS is trained in three stages, where we have three loss functions ℒ 𝑖 .

Fig. 5 -

 5 Fig.5-3shows the architecture of our HyperGCN-WSS. The input X ∈ R 𝑁 ×512 is the matrix of features from the VGG16 network. Each Graph Convolutional Layer (GCL) contains batch normalization[START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF], Exponential Linear Unit (ELU)[START_REF] Clevert | Fast and accurate deep network learning by exponential linear units (ELUs)[END_REF] as activation function, and it could have a residual connection[START_REF] He | Deep residual learning for image recognition[END_REF] as shown in Fig.5-3. The GCLs implement the propagation rule in (4.1). The Hypergraph Convolutional Layers (HCLs) are similar to the GCLs, but instead of using (4.1), they implement (5.1).Our architecture also contains Multi-layer Perceptrons that classify the embedding of

Figure 5 - 4 :

 54 Figure 5-4: Some visual results on PASCAL VOC 2012 with our HyperGCN-WSS, using scribbles or clicks as weak signals.

Figure 6 -

 6 Figure 6-1 shows an overview of our framework. Our algorithm consists of two steps 1) the inference or learning of a graph topology and 2) the proposed S-SobGNN architecture applied to semi-supervised learning or node classification.

Figure 6 - 1 :

 61 Figure 6-1: The pipeline of our S-SobGNN algorithm with 𝑘-NN or smooth-learned graphs. Our GNN can be used in a broad range of data such as images, and text, among others. However, the step of mapping the original dataset to the data matrix X ∈ R 𝑁 ×𝑀 could be different in each case. Our framework is composed of (a) inference of the graph topology and (b) the S-SobGNN architecture. Finally, a loss function (such as the cross-entropy) is computed for the training procedure.

. 10 )

 10 Therefore, 𝜆 min (L + 𝜖I) = 𝜖, i.e., L + 𝜖I is positive definite (L + 𝜖I ≻ 0), and:𝜅(L + 𝜖I) = |𝜆 max (L + 𝜖I)| |𝜆 min (L + 𝜖I)| = 𝜆 max (L) + 𝜖 𝜖 < 𝜅(L) ∀ 𝜖 > 0. (6.11)

. 13 )

 13 Let ⟨x, y⟩ (𝜌),𝜖 = x T (L + 𝜖I) (𝜌) y be the inner product between two graph signals x and y that induces the associated sparse Sobolev norm. Theorem 6.4.1. The sparse Sobolev norm ‖x‖ (𝜌),𝜖 ≜ ‖(L + 𝜖I) (𝜌/2) x‖ satisfies the basic properties of vector norms for 𝜖 > 0 (for 𝜖 = 0 we have a semi-norm).

Ā𝜌 2 𝜌∀-1 2 𝜌Figure 6 - 2 :

 2262 Figure 6-2: Basic configuration of our S-SobGNN architecture with 𝑛 layers and 𝛼 filters per layer.

  Colorectal Cancer Histology: This cancer dataset comes from 10 anonymized tissue slides from the University Medical Center Mannheim[START_REF] Kather | Multi-class texture analysis in colorectal cancer histology[END_REF]. The dataset has eight classes: tumor epithelium, simple stroma, complex stroma, immune cells, debris, normal mucosal glands, adipose tissue, and background (no tissue). This chapter does not explore the feature representation, and only uses Local Binary Patterns (LBP)[START_REF] Ojala | Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[END_REF] to represent each image. We explore two partitions of this dataset: 1) Cancer Binary (Cancer-B) considers only tumor epithelium and simple stroma, and 2) Cancer Multi-class (Cancer-M) tests all classes. The 20 Newsgroups: The 20 Newsgroups dataset (20News) contains roughly 20000

3 . 3 (

 33 we refer to this algorithm as learned graphs in the results). For Cancer-B and Cancer-M, we compute two graphs with 𝑁 = 1250 and 𝑁 = 5000, respectively. For 20News, HAR, and Isolet, we infer ten graphs with 𝑁 = 3000. The samples for each dataset are divided evenly across different classes. For each dataset, we vary the training percentage in the set {1%, 2%, . . . , 19%, 20%}, and the rest of the samples are for testing. No validation set is used (we do not fine-tune hyperparameters in this experiment). We compare S-SobGNN against state-of-the-art GNNs using two strategies. In the first strategy, we train the state-of-the-art models with default parameters without any restriction. For the second strategy, 1) we train the state-of-the-art models with approximately the same amount of learnable parameters, 2) we do not consider any specialized layer, and 3) we use the same learning rate, number of epochs, dropout, weight decay, activation layers, and number of layers for each model. Moreover, several ablation studies analyze the values of 𝜖, 𝛼, and 𝑘 of S-SobGNN. The semi-supervised learning experiments are tested with a Monte Carlo cross-validation with ten repetitions for each training percentage. We also compute the sparsity percentages of our filter for 𝜌 ∈ {1, 2, . . . , 5} for the sparse and non-sparse Sobolev norms. Therefore, we compute the average training time from ten repetitions on 20News with variations of 𝑁 in {100, 200, . . . , 5500}, where the training percentage is set to 20%, and we use the default implementation of the state-of-the-art GNNs. Finally, we compute the homophily metric 𝐻(𝐺) for the 𝑘-NN and learned graphs in each dataset.

Figure 6 - 3 :

 63 Figure 6-3: Average training time per epoch (T-Epoch) for several GNNs on 20News with variations in the number of nodes 𝑁 .

Finally, Fig. 6 -

 6 3 shows the average training time per epoch (T-Epoch) on 20News for several GNNs with a GPU implementation using an Nvidia GTX 1070. The GNNs in Fig. 6-3 shows a linear complexity in the number of nodes for the given range of 𝑁 . S-SobGNN has a low training time compared with several state-of-the-art GNNs due to the sparsity concepts developed in this chapter.

  , 𝑁 = 1250 Cancer-M, 𝑁 = 5000 20News, 𝑁 = 3000 HAR, 𝑁 = 3000 Isolet, 𝑁 = 3000 𝜌 = 1 𝜌 = 2 𝜌 = 3 𝜌 = 4 𝜌 = 5 𝜌 = 1 𝜌 = 2 𝜌 = 3 𝜌 = 4 𝜌 = 5 𝜌 = 1 𝜌 = 2 𝜌 = 3 𝜌 = 4 𝜌 = 5 𝜌 = 1 𝜌 = 2 𝜌 = 3 𝜌 = 4 𝜌 = 5 𝜌 = 1 𝜌 = 2 𝜌 = 3 𝜌 = 4 𝜌 = 5 Sparse 96.63

Figure 7 - 1 :

 71 Figure 7-1: Mixing steps 𝑓 (𝜆 2 , 𝜖) for 𝜖 = 5 × 10 -4 vs. number of removed or added edges for two stochastic block model graphs with a) two clusters, b) five clusters, and c) one Erdős-Rényi graph.

1 :

 1 Compute JLC(𝑖, 𝑗) ∀ (𝑖, 𝑗) ∈ ℰ 1 , 𝐴 = 𝑝 𝐴 |ℰ 1 |, and 𝐷 = 𝑝 𝐷 |ℰ 1 |, where ℰ 1 = ℰ 2: for 𝑚 = 1 until 𝑚 = 𝐴 do 3:

  𝑖 ∈ R 𝐹 . Finally, 𝜑 ∈ R |ℰ 𝐴 | is the vector with the normalized values of JLC(𝑖, 𝑗) ∀ (𝑖, 𝑗) ∈ ℰ 𝐴 , and d (𝑙)

Figure 7 - 3 :

 73 Figure 7-3: Average running time for balanced Forman curvature and Jost and Liu curvature for variations in the number of nodes in three artificial graphs.

The following Chapter 8

 8 explores the signal processing part of this thesis, where we are concerned with the problem of reconstruction of missing values in time-varying graph signals.

9 )Definition 8 . 4 . 1 (

 9841 is equal to the Laplacian quadratic form in (2.5) when 𝜖 = 0 and 𝛽 = 1. We use the Sobolev norm to define a new smoothness function for time-varying graph signals as follows: Sobolev smoothness of time-varying graph signals). Let X = [x 1 , x 2 , . . . , x 𝑀 ] be a time-varying graph signal, let D ℎ be the temporal difference operator, and let L be the combinatorial Laplacian matrix of a graph 𝐺. For fixed parameters 𝜖 ≥ 0 and 𝛽 ∈ R + , the Sobolev smoothness of X is given as follows:

Figure 8 - 1 :

 81 Figure 8-1:The framework of our algorithm (GraphTRSS) using a matrix of coordinates M ∈ R 𝑁 ×2 to construct a graph with 𝑁 regions in the world with confirmed cases of COVID-19 by November 18, 2020. The graph is constructed with a 𝑘-NN method. GraphTRSS uses the operator D ℎ ∈ R 𝑀 ×(𝑀 -1) to capture temporal information in the time-varying signal X ∈ R 𝑁 ×𝑀 with 𝑀 temporal snapshots, and it also uses different sampling strategies J ∈ {0, 1} 𝑁 ×𝑀 according to the desired output (reconstruction or forecasting). Finally, the optimization function, which includes the error and Sobolev terms, reconstructs or predicts the missing values, i.e., the indexes of X where J has values zero.

Fig. 8 -Figure 8 - 2 :

 882 Fig. 8-1 shows the pipeline of our algorithm applied to a graph with the regions in the world with confirmed cases of COVID-19 by November 18, 2020. The advantage

Figure 8 - 3 :

 83 Figure 8-3: Eigenvalue penalization of the Laplacian matrix for different values of 𝛽 from the dataset of COVID-19.

1 2 =Figure 8 - 4 : 1 22 1 2

 28411 Figure 8-4: Graph with the places in the United States in the Johns Hopkins University dataset [21]. The graph was constructed with a 𝑘-NN with 𝑘 = 10.

Figure 8 - 5 :

 85 Figure 8-5: Graph with the spots in the sea for the dataset of temperature. The graph was constructed with a 𝑘-NN with 𝑘 = 10.

Figure 8 - 6 :

 86 Figure 8-6: Graph Fourier transform of some elements of XD ℎ for COVID-19 global dataset.

Fig. 8 -

 8 Fig.8-7shows the results in the synthetic dataset. GraphTRSS performs better than the existing methods for all values in ℳ for the random sampling scheme, as shown in Fig. 8-7(a). Our algorithm is also more robust against noise than the compared methods, as shown in Fig. 8-7(b). Similarly, Fig. 8-7(c) shows the benefits of GraphTRSS regarding different levels of smoothness 𝛼 as given in Definition 8.3.1. Our algorithm shows better performance than other time-varying graph signal methods as TGSR and

Fig. 8 -

 8 Fig. 8-8 and 8-9 show the results of the experiments in the COVID-19 datasets.

Figure 8 - 9 :

 89 Figure 8-9: Comparison of GraphTRSS with several methods in the literature on COVID-19 USA dataset for several experiments in terms of: a) random sampling, b) entire snapshots sampling, c) forecasting, d) variation of parameter 𝜖, e) variation of parameter 𝛽, f) several temporal difference operators, g) convergence comparison, and h) running time. 134

Fig. 8 -

 8 Fig. 8-8(g) shows that GraphTRSS converges faster than TGSR method for COVID-19 global, where X* is the solution of the optimization problem, and X𝑡 is the solution at iteration 𝑡. Fig. 8-8(h) and 8-9(h) show the running time for each method. These experiments were computed with the best parameters of each method from the random sampling experiment. Arguably, GraphTRSS shows the best compromise between accuracy and running time among several methods for COVID-19 datasets, as shown in Fig. 8-8 and 8-9.

Fig. 8 -Figure 8 - 10 :

 8810 Fig. 8-10 and 8-11 show the results for the experiments in the environmental datasets.The analysis of the comparison between our algorithm and the other methods is mostly similar to the analysis for Fig.8-8 and 8-9, i.e., GraphTRSS shows the best compromise between accuracy and running time. However, we should notice two interesting results in Fig.8-10 and 8-11. Firstly, we notice that the parameters 𝛽 = 1.5 or 𝛽 = 2 show better results than 𝛽 = 1 in the environmental datasets. Secondly,

Fig. 8 -Figure 8 - 12 :

 8812 Fig.8-12 shows the condition number of the Hessian functions for the COVID and environmental datasets with random sampling with density 0.5. The condition number of the Hessian of (8.5) is not displayed in Fig.8-12 for COVID-19 USA because, in that case, 𝜅(∇ 2 z 𝑓 𝐿 (z)) → ∞. Furthermore, as predicted by Theorem 8.4.1: 1) there is a range of values of 𝜖 where 𝜅(∇ 2 z 𝑓 𝑆 (z)) < 𝜅(∇ 2 z 𝑓 𝐿 (z)) and then we get benefits in convergence rate, and 2) 𝜅(∇ 2 z 𝑓 𝑆 (z)) grows quickly for large values of 𝜖. Notice that the condition numbers of ∇ 2 z 𝑓 𝑆 (z) and ∇ 2 z 𝑓 𝐿 (z) depend on the specific eigenvalues of the graphs we are addressing. As a consequence, it is reasonable to use the normalized Laplacian matrix Δ = D -1 2 LD -1 2 since 𝜆 𝑚𝑎𝑥 (Δ) ≤ 2. The conclusions of Theorem 8.4.1 still hold with the normalized Laplacian matrix because now

  -13(b) for both COVID-19 and sea surface temperature datasets. However, GraphTRSS is still faster than TGSR in the COVID-19 dataset when using the normalized Laplacian since the conclusions of Theorem 8.4.1 still hold.

3 .

 3 The intensity features are computed with histograms of 127 bins between 0 and 255 for I 𝑡 𝑣 (𝒫 𝑣 ), I 𝑡-1 𝑣 (𝒫 𝑣 ), B 𝑣 (𝒫 𝑣 ) and |I 𝑡 𝑣 (𝒫 𝑣 ) -B 𝑣 (𝒫 𝑣 )|. Let 𝑓 𝐼𝑁 : a → R 1×127 be the function representing the histogram extraction process for the intensity vectors, then the whole representation of the intensity in 𝑣 ∈ R 1×508 of a node 𝑣 is obtained as follows [𝑓 𝐼𝑁 (I 𝑡 𝑣 (𝒫 𝑣 )), 𝑓 𝐼𝑁 (I 𝑡-1 𝑣 (𝒫 𝑣 )), 𝑓 𝐼𝑁 (B 𝑣 (𝒫 𝑣 )), 𝑓 𝐼𝑁 (|I 𝑡 𝑣 (𝒫 𝑣 ) -B 𝑣 (𝒫 𝑣 )|)].

4 .

 4 The deep features are computed using the 5th layer (Conv-5) of the VGG model[START_REF] Chatfield | Return of the devil in the details: Delving deep into convolutional nets[END_REF]. The image I 𝑡 𝑣 is resized to be compatible with the VGG network, and then a principal component analysis is applied to compress 512 dimensional features into 64 features from Conv-5. Let 𝑓 𝐷𝐿 : A → R 1×10816 be the function representing the deep feature extraction, then the deep features dl 𝑣 ∈ R 1×10816

$ 1 2. 1 2- 1 2 ‖ 2 = 1 2

 11121 python main . py --v e r b o s e --l r 0 . 0 1 --weight_decay 0 . 0 9 --hidden_units 64 --n_layers_set 2 --dropout 0 . 5 --G r a p h D i f u s s i o n --alphaGDC 0 . 0 5 --k 128 --d a t a s e t Cora --GNN GCN Similarly, we can reproduce the results of SJLR in Table 7.1 using the Cornell dataset with the following command: $ python main . py --v e r b o s e --l r 0 . 0 1 2 2 --weight_decay 4e-4 --hidden_units 32 --n_layers_set 2 --dropout 0 . 3 1 0 9 --J o s t L i u C u r v a t u r e --pA 0 . 0 4 9 5 --pD 0 . 8 9 1 8 --tau 4 2 9 . 2 1 --alpha 0 . 7 7 4 2 --d a t a s e t C o r n e l l --GNN GCN This codebase is an open source project and is being released under the MIT license. since P = D -1 2 (I -Δ)D Suppose we write f T D -1 2 = ∑ 𝑖 𝑎 𝑖 𝜑 𝑖 , then:‖f T P 𝑠 -𝜋‖ 2 = ‖ ∑ 𝑖 𝑎 𝑖 𝜑 𝑖 (I -Δ) 𝑠 D 𝑎 1 𝜑 1 (1 -𝜆 1 ) 𝑠 D ‖ ∑ 𝑖̸ =1 𝑎 𝑖 𝜑 𝑖 (1 -𝜆 𝑖 ) 𝑠 D ‖ 2 = ‖ ∑ 𝑖̸ =1

  𝑣 (𝒫 𝑣 ) and |I 𝑡 𝑣 (𝒫 𝑣 ) -B 𝑣 (𝒫 𝑣 )|. The vectors of orientations and magnitudes obtained

	in I 𝑡 𝑣 , 𝑣 (𝒫 𝑣 ), I 𝑡-1 𝑣 -B 𝑣 |. The intensity histograms are computed in I 𝑡 𝑣 , B 𝑣 and |I 𝑡 I 𝑡-1 𝑣 (𝒫 𝑣 ), Background Original Frames B Instances LBP Features Histograms Descriptive Statistics Deep Features

Table 3 .

 3 1: Summary of the parameters used in our experiments for each dataset.Finally, the last experiment is devoted to testing Corollary 3.3.1. Since the calculation of U is computationally prohibitive for the configurations involved in the ablation studies, in this case, a graph with 𝑁 = 7443 nodes is constructed with the sequences backdoor, bungalows, and busStation of the challenge shadow of CD-

	Dataset	𝜁	𝑘	𝜖 𝛼 𝜂	𝜒
	CDNet2014 200 30 0.2 1 100 350
	I2R	200 30 0.2 1 100 150
	SBI2015	300 30 0.2 1 100 60
	UCSD	200 30 0.2 1 100 40

Net2014, using a 𝑘-NN with 𝑘 = 30, and a Mask R-CNN. In practice, graph signals are only approximately bandlimited

  experiments are executed on a laptop with a processor Intel Core i7 with 16 gigabytes of memory RAM. Using handcrafted features, GraphMOS algorithm with Mask R-CNN takes 3.73 FPS, with Cascaded Mask R-CNN it takes 1.76 FPS, with Deeplab method it takes 13.14 FPS, and with SuBSENSE method GraphMOS takes 15.11 FPS to segment the moving objects. In case of using deep features with block

and superpixel-based segmentation methods, the experiments of GraphMOS are conducted on a powerful Nvidia DGX-2 server machine. GraphMOS takes 1.42 and 2.31 FPS for MOS using superpixel and block-based graph construction.

Table 3 .

 3 2: Comparisons of average F-measure on CDNet2014 dataset.

	Challenge	SWCD	FTSG	SuBSENSE	WeSamBE	PAWCS	WisenetMD	IUTIS-5	SemanticBGS	FgSegNet v2	Mask R-CNN	BSUV-Net	GraphMOS
	Bad Weather	0.8233 0.8228 0.8619 0.8608 0.8152 0.8616 0.8248	0.8260	0.3277	0.7952 0.8713 0.9411
	Baseline	0.9214 0.9330 0.9503 0.9413 0.9397 0.9487 0.9567	0.9604	0.6926	0.7608 0.9693 0.9710
	Camera Jitter 0.7411 0.7513 0.8152 0.7976 0.8137 0.8228 0.8332 0.8388	0.4266	0.7021	0.7743 0.9233
	Dynamic-B	0.8645 0.8792 0.8177 0.7440 0.8938 0.8376 0.8902 0.9489	0.3634	0.5880	0.7967 0.8922
	I-O Motion	0.7092 0.7891 0.6569 0.7392 0.7764 0.7264 0.7296 0.7878	0.2002	0.4724	0.7499 0.6455
	Low-F rate	0.7374 0.6259 0.6445 0.6602 0.6588 0.6404 0.7743 0.7888	0.2482	0.5776	0.6797 0.6910
	Night Videos	0.5807 0.5130 0.5599 0.5929 0.4152 0.5701 0.5290	0.5014	0.2800	0.3898 0.6987 0.8211
	PTZ	0.4545 0.3241 0.3476 0.3844 0.4615 0.3367 0.4282	0.5673	0.3503	0.6214 0.6282 0.8511
	Shadow	0.8779 0.8832 0.8986 0.8999 0.8913 0.8984 0.9084 0.9478	0.5295	0.8781	0.9233 0.9901
	Thermal	0.8581 0.7768 0.8171 0.7962 0.8324 0.8152 0.8303	0.8219	0.6038	0.4986 0.8581 0.9010
	Turbulence	0.7735 0.7127 0.7792 0.7737 0.6450 0.8304 0.7836	0.6921	0.0643	0.1707	0.7051 0.8233
	Overall	0.7583 0.7283 0.7408 0.7446 0.7403 0.7535 0.7717 0.7892	0.3715	0.5868	0.7868 0.8592

Table 3 .

 3 3: Comparison of F-measure results over the sequences of I2R dataset. Escalator 0.5204 0.7183 0.5988 0.4706 0.4265 0.5756 0.3967 0.3716 0.4205 0.3673 0.6994 0.7557 0.9081 Fountain 0.7428 0.8618 0.5635 0.3500 0.6620 0.7937 0.2835 0.2185 0.2664 0.2646 0.8644 0.8306 0.8516 Hall 0.6152 0.5597 0.6377 0.5036 0.5355 0.7708 0.6751 0.3898 0.4819 0.5450 0.8118 0.7654 0.8670 Lobby 0.5760 0.5654 0.1488 0.5958 0.4059 0.2412 0.2329 0.1453 0.5157 0.2207 0.7659 0.7213 0.9555 Shopping. 0.6820 0.6800 0.5418 0.6832 0.6724 0.7594 0.6927 0.3764 0.6749 0.6198 0.8070 0.7619 0.7712 WaterSur. 0.6500 0.8873 0.8544 0.3008 0.7725 0.9365 0.6706 0.5479 0.4651 0.3607 0.9358 0.9494 0.9899

	Sequence	MoG	DECOLOR	ViBe	3WD	GRASTA	SuBSENSE	ROSL	ADMM	noncvxRPCA	OR1MP	BSUV-Net	BSUV-Net+	GraphMOS
	Bootstrap 0.6328 0.6248 0.6134 0.5393 0.6017 0.6704 0.6449 0.5085 0.6142 0.6459 0.7133 0.7636 0.9722
	Campus	0.2818 0.7652 0.4604 0.2258 0.2152 0.7498 0.1746 0.1913 0.1742 0.1714 0.8468 0.8543 0.9711
	Curtain	0.6276 0.8342 0.6781 0.5127 0.7816 0.9436 0.7009 0.5211 0.4747 0.4624 0.9055 0.9373 0.9899

Overall 0.5920 0.7219 0.5663 0.4646 0.5635 0.7157 0.4969 0.3634 0.4542 0.4064 0.8166 0.8155 0.9196 tables, the best and second best performing methods are shown in red and blue, respectively. Our proposed algorithms show competitive performance as compared to state-of-the-art methods on all datasets.

For the CDNet2014 dataset (Table

3

.2), our proposed algorithm GraphMOS obtained the best performance in terms of average F-measure score of 85.92%, which is 7.0% better than the second-best performing method SemanticBGS. Moreover, GraphMOS achieved the best results in seven out of eleven challenges including Bad Weather (94.11%), Baseline (97.10%), Camera Jitter (92.33%), Night Videos (82.11%), PTZ (85.11%), Shadow (99.01%), and Thermal (90.10%) while it obtained favorable performance for the remaining attributes including Dynamic Background, Turbulence, Low Framerate, and Intermittent Object Motion. In all these challenges, the sequences in NightVideos and PTZ are very challenging since the majority of the compared methods are not able to achieve more than 70.0% F-measure score, while

Table 3 .

 3 4: Comparison of F-measure results over the sequences of SBI2015 dataset.

	Sequence	MoG	DECOLOR	ViBe	3WD	GRASTA	SuBSENSE	ROSL	ADMM	noncvxRPCA	OR1MP	BSUV-Net	BSUV-Net+	GraphMOS
	Board	0.7656 0.5033 0.7377 0.3959 0.5592 0.6588 0.6440 0.1420 0.5304 0.5259 0.9560 0.9886 0.9931
	CAVIAR1	0.5974 0.9183 0.8051 0.4282 0.2104 0.8783 0.8533 0.5065 0.4204 0.4653 0.9260 0.9358 0.9744
	CAVIAR2	0.4597 0.4324 0.7347 0.4470 0.0463 0.8740 0.2966 0.2452 0.1933 0.1813 0.8572 0.8649 0.9210
	CaVignal	0.5057 0.4827 0.3497 0.3767 0.3812 0.4080 0.6226 0.4305 0.4720 0.3727 0.4628 0.4773 0.7322
	Candela	0.4927 0.5025 0.5020 0.4702 0.1929 0.6959 0.4913 0.4410 0.4730 0.6941 0.8997 0.8597 0.7551
	Hall&Mon.	0.5952 0.7826 0.6017 0.4479 0.1491 0.7559 0.6081 0.1521 0.4525 0.4827 0.8931 0.9346 0.9122
	HighwayI	0.6272 0.6976 0.4150 0.4123 0.5522 0.5073 0.6836 0.5170 0.5733 0.5660 0.8440 0.8337 0.9880
	HighwayII	0.8144 0.8925 0.5554 0.7426 0.4833 0.8779 0.7808 0.7429 0.7335 0.7287 0.9690 0.9592 0.9547
	HumanBod. 0.7553 0.8265 0.4268 0.5074 0.5352 0.8560 0.7606 0.3115 0.5765 0.5954 0.9314 0.9503 0.9522
	IBMtest2	0.7108 0.8823 0.7001 0.8162 0.3825 0.9281 0.8579 0.5076 0.6714 0.6460 0.9722 0.9643 0.9856
	People&Fol. 0.5920 0.2601 0.6111 0.2911 0.3460 0.4251 0.4108 0.0569 0.3924 0.3754 0.8837 0.6930 0.9059
	Toscana	0.6093 0.3669 0.7307 0.3132 0.4188 0.8256 0.7083 0.2533 0.6779 0.4320 0.9110 0.9193 0.9411
	Foliage	0.5786 0.3178 0.5539 0.3376 0.4148 0.1962 0.4341 0.2105 0.4617 0.4481 0.4371 0.3450 0.7792
	Snellen	0.5498 0.4023 0.3083 0.3213 0.4104 0.2467 0.4052 0.1099 0.4345 0.4083 0.3674 0.3786 0.7380
	Overall	0.6181 0.5906 0.5737 0.4506 0.3630 0.6524 0.6112 0.3305 0.5045 0.4944 0.8079 0.7932 0.9328

Table 3 .

 3 5: Comparison of F-measure results over the videos of UCSD background subtraction dataset.

	Sequence	MoG	DECOLOR	ViBe	3WD	GRASTA	SuBSENSE	ROSL	ADMM	noncvxRPCA	OR1MP	BSUV-Net	BSUV-Net+	GraphMOS
	Birds	0.1427 0.1457 0.3354 0.1308 0.1320 0.4832 0.1478 0.0227	0.1432 0.1394 0.3314 0.2625 0.7897
	Boats	0.0881 0.2179 0.1854 0.1576 0.0678 0.4550 0.1637 0.1212	0.1380 0.1100 0.4586 0.6621 0.8311
	Bottle	0.1856 0.4765 0.4512 0.1364 0.1159 0.6570 0.2069 0.6589	0.1974 0.1795 0.8528 0.5039 0.9011
	Chopper 0.3237 0.6214 0.4930 0.3171 0.0842 0.6723 0.2920 0.1250	0.3103 0.2653 0.2805 0.3020 0.8230
	Cyclists	0.0915 0.2224 0.1211 0.1003 0.1243 0.1445 0.1366 0.1093	0.1317 0.1242 0.0051 0.4138 0.7911
	Flock	0.2706 0.2943 0.2306 0.2007 0.1612 0.2492 0.3409 0.1088	0.3220 0.2605 0.1160 0.0025 0.6722
	Freeway 0.2622 0.5229 0.4002 0.5028 0.0814 0.5518 0.3875 0.0816	0.3126 0.1549 0.4780 0.1185 0.4831
	Hockey	0.3867 0.3449 0.4195 0.2789 0.3149 0.3611 0.4106 0.2981	0.3411 0.4296 0.6460 0.6908 0.8211
	Jump	0.2679 0.3135 0.2636 0.2481 0.4175 0.2295 0.4198 0.0609	0.3180 0.3073 0.5491 0.8697 0.8233
	Landing 0.0335 0.0640 0.0433 0.0457 0.0414 0.0026 0.0506 0.0826 0.0480 0.0442 0.0021 0.0012 0.4122
	Ocean	0.1113 0.1315 0.1648 0.2055 0.1144 0.2533 0.1422 0.1809	0.1274 0.1252 0.4117 0.5335 0.9422
	Peds	0.3731 0.7942 0.5257 0.7536 0.4653 0.5154 0.7418 0.6667	0.4333 0.4297 0.6958 0.6738 0.9411
	Skiing	0.2038 0.3473 0.1441 0.1981 0.0927 0.2482 0.1942 0.0519	0.1812 0.1791 0.0841 0.0602 0.7622
	Surf	0.0489 0.0647 0.0462 0.0579 0.0523 0.0467 0.0453 0.0162	0.0325 0.0317 0.0884	0	0.7322
	Surfers	0.0542 0.1959 0.1189 0.0962 0.0742 0.1393 0.1184 0.1950	0.1083 0.1044 0.2612 0.4776 0.7591
	Trafic	0.2188 0.2732 0.1445 0.2032 0.0368 0.1165 0.1042 0.1044	0.0949 0.0882	0	0	0.6670
	Overall	0.1914 0.3144 0.2555 0.2271 0.1485 0.3203 0.2439 0.1803	0.2025 0.1858 0.3288 0.3483 0.7595
	In the SBI2015 dataset (Table 3.4), the GraphMOS has also achieved the best per-
	formance in 11 out of 14 videos. Overall, the proposed algorithm obtained a 93.28%
	F-measure score, which is approximately 12.50% larger than BSUV-Net+ (80.79%)
	and 13.96% larger than BSUV-Net (79.32%) methods. The sequences in this dataset

BSUV-Net method. BSUV-Net+, SuBSENSE, and DECOLOR methods have obtained favorable performance, while the remaining compared methods are not able to achieve more than 60.0% F-measure score, which further shows the challenging nature of the sequences present in this dataset.

Table 3 .

 3 6: Performance comparisons in terms of average F-measure score for different segmentation methods used for graph construction. Only handcrafted features are used to report the performance. These ablations studies involves: graph construction using DeepLab with ResNet 101 (DeepLab), Mask R-CNN with ResNet 50 (Mask R-50), Cascade Mask R-CNN with ResNeSt 200 (Cascade RS-200), SuBSENSE, and Superpixel.

	Dataset	SuBSENSE	DeepLab	Mask R-50	Cascade RS-200	Superpixel
	I2R	0.7527 0.4145 0.8397	0.8196	0.8510
	CDNet2014 0.7396 0.6119 0.6466	0.7283	0.8138
	SBI2015	0.6488 0.7534 0.7538	0.7832	0.8611
	UCSD	0.3388 0.4088 0.5553	0.6751	0.7113

Table 3 .

 3 7: Performance comparison in terms of average F-measure score of superpixel and block-based segmentation for graph construction methods. The performance is reported by using both handcrafted and deep features representation of graph nodes.

	Segmentation	I2R	CDNet2014	SBI2015	UCSD
	Block-based	0.8408 0.7494 0.8080 0.6607
	Superpixel	0.9196 0.8592 0.9328 0.7595

Table 3 .

 3

[START_REF] Giraldo | Sparse Sobolev graph neural networks[END_REF]

shows the performance comparisons of superpixel and block-based node representation in graph construction. The average F-measure is reported by using handcrafted plus deep features representation of the node. Superpixel-based method clearly outperforms the block-based node representation method in all datasets.

Table 3 .

 3 8: Performance comparisons in terms of average F-measure score on five datasets using distinct node features representations. Handcrafted, deep features, and the concatenation of handcrafted and deep features (Hand + Deep (Conv-5)) are used to represent graph nodes.

	Features	I2R	CDNet2014	SBI2015	UCSD	DAVIS2016
	Handcrafted features	0.8510 0.8210 0.8611 0.7113 0.8711
	Deep Features (Conv-5) 0.8701 0.8491 0.8722 0.7320 0.8993
	Deep Features (Conv-4) 0.8665 0.8410 0.8711 0.7222 0.8801
	Hand + Deep (Conv-5) 0.9196 0.8673 0.8952 0.7595 0.9121

Table 3 .

 3 9: Performance comparisons in terms of average F-measure score for the number of superpixels in the SLIC method.

	Superpixels (𝜁)	I2R	CDNet2014	SBI2015	UCSD	DAVIS2016
	100	0.9054 0.8397 0.8632 0.7265 0.8810
	200	0.9196 0.8592 0.8790 0.7595 0.9020
	300	0.9191 0.8555 0.8952 0.7496 0.9120
	400	0.9094 0.8473 0.8810 0.7406 0.9030
	500	0.8983 0.8442 0.8734 0.7363 0.8821

Table 3 .

 3 10: Average F-measure with variations in the construction of the graph. This ablation involves: 𝑘-NN with 𝑘 = 40, 𝑘 = 30, 𝑘 = 20, and 𝑘 = 10.

	Challenge	𝑘 = 40 𝑘 = 30 𝑘 = 20 𝑘 = 10
	Bad Weather	0.8294 0.8372 0.8307 0.8294
	Baseline	0.9398 0.9424 0.9394 0.9307
	Camera Jitter 0.7005 0.7001 0.7016 0.6848
	Dynamic-B	0.7334 0.7431 0.7799 0.7996
	I-O Motion	0.3607 0.4051 0.4632 0.5539
	Low-F rate	0.5538 0.5581 0.5584 0.5647
	PTZ	0.7599 0.7486 0.7499 0.7595
	Shadow	0.9653 0.9658 0.9659 0.9647
	Thermal	0.7292 0.7294 0.7306 0.7351
	Overall	0.7302 0.7366 0.7466 0.7580

Table 3 .

 3 11: Average F-measure with variations in the semi-supervised learning algorithm. This ablation involves: Sobolev minimization (Sob.) with 𝜖 = 50, 𝜖 = 0.5, 𝜖 = 0.2, and Total Variation minimization (TV).

	Challenge	Sob. 𝜖 = 50	Sob. 𝜖 = 0.5	Sob. 𝜖 = 0.2	TV
	Bad Weather	0.8096 0.8337 0.8372 0.8291
	Baseline	0.8958	0.9301 0.9424 0.9394
	Camera Jitter 0.6443	0.6952 0.7001 0.7024
	Dynamic-B	0.7706 0.7760 0.7431 0.7486
	I-O Motion	0.5306 0.4447 0.4051 0.3500
	Low-F rate	0.5254	0.5567 0.5581 0.5928
	PTZ	0.7420 0.7495 0.7486 0.7899
	Shadow	0.9463 0.9650 0.9658 0.9658
	Thermal	0.7158 0.7286 0.7294 0.7229
	Overall	0.7312 0.7422 0.7366 0.7379

Table 4 .

 4 1: Comparisons of average F-measure over nine challenges of CDNet2014. GraphMOS-Net is compared with unsupervised and supervised algorithms in MOS.

	Challenge	SWCD	FTSG	SuBSENSE	PAWCS	WisenetMD	IUTIS-5	SemanticBGS	FgSegNet v2	BSUV-Net	GraphBGS-TV	GraphBGS	GraphMOS-Net
	Bad Weather 0.8233 0.8228 0.8619 0.8152 0.8616 0.8248	0.8260	0.7952	0.8713	0.8072	0.9085	0.9142
	Baseline	0.9214 0.9330 0.9503 0.9397 0.9487 0.9567	0.9604	0.6926 0.9693	0.9432	0.9535	0.9627
	Camera Jitter 0.7411 0.7513 0.8152 0.8137 0.8228 0.8332	0.8388	0.4266	0.7743	0.7191	0.8826	0.8980
	Dynamic-B	0.8645 0.8792 0.8177 0.8938 0.8376 0.8902 0.9489	0.3634	0.7967	0.7365	0.8353	0.8849
	I-O Motion	0.7092 0.7891 0.6569 0.7764 0.7264 0.7296 0.7878	0.2002	0.7499	0.4199	0.5036	0.7028
	Low-F rate	0.7374 0.6259 0.6445 0.6588 0.6404 0.7743 0.7888	0.2482	0.6797	0.5103	0.6022	0.6945
	PTZ	0.4545 0.3241 0.3476 0.4615 0.3367 0.4282	0.5673	0.3503	0.6282	0.8014	0.7993	0.8329
	Shadow	0.8779 0.8832 0.8986 0.8913 0.8984 0.9084	0.9478	0.5295	0.9233	0.9658	0.9712	0.9753
	Thermal	0.8581 0.7768 0.8171 0.8324 0.8152 0.8303	0.8219	0.6038	0.8581	0.7305	0.8594	0.8647
	Overall	0.7764 0.7539 0.7566 0.7870 0.7653 0.7923 0.8320	0.4158	0.8056	0.7371	0.8128	0.8589

Table 4 .

 4 2: Comparison of F-measure results over the videos of UCSD background subtraction dataset.

	Sequence	MoG	DECOLOR	ViBe	3WD	GRASTA	SuBSENSE	ROSL	ADMM	noncvxRPCA	OR1MP	BSUV-Net	BSUV-Net+	GraphBGS	GraphMOS-Net
	Birds	0.1427 0.1457 0.3354 0.1308 0.1320 0.4832 0.1478 0.0227	0.1432 0.1394 0.3314 0.2625 0.7495	0.7391
	Boats	0.0881 0.2179 0.1854 0.1576 0.0678 0.4550 0.1637 0.1212	0.1380 0.1100 0.4586 0.6621 0.7765	0.8007
	Bottle	0.1856 0.4765 0.4512 0.1364 0.1159 0.6570 0.2069 0.6589	0.1974 0.1795 0.8528 0.5039 0.8741	0.8741
	Chopper 0.3237 0.6214 0.4930 0.3171 0.0842 0.6723 0.2920 0.1250	0.3103 0.2653 0.2805 0.3020 0.7766	0.7844
	Cyclists	0.0915 0.2224 0.1211 0.1003 0.1243 0.1445 0.1366 0.1093	0.1317 0.1242 0.0051 0.4138 0.7417	0.7479
	Flock	0.2706 0.2943 0.2306 0.2007 0.1612 0.2492 0.3409 0.1088	0.3220 0.2605 0.1160 0.0025 0.5903	0.5871
	Freeway 0.2622 0.5229 0.4002 0.5028 0.0814 0.5518 0.3875 0.0816	0.3126 0.1549 0.4780 0.1185 0.3719	0.3516
	Hockey	0.3867 0.3449 0.4195 0.2789 0.3149 0.3611 0.4106 0.2981	0.3411 0.4296 0.6460 0.6908 0.7664	0.7804
	Jump	0.2679 0.3135 0.2636 0.2481 0.4175 0.2295 0.4198 0.0609	0.3180 0.3073 0.5491 0.8697 0.7734	0.7602
	Landing 0.0335 0.0640 0.0433 0.0457 0.0414 0.0026 0.0506 0.0826 0.0480 0.0442 0.0021 0.0012 0.2452	0.1840
	Ocean	0.1113 0.1315 0.1648 0.2055 0.1144 0.2533 0.1422 0.1809	0.1274 0.1252 0.4117 0.5335 0.8593	0.8593
	Peds	0.3731 0.7942 0.5257 0.7536 0.4653 0.5154 0.7418 0.6667	0.4333 0.4297 0.6958 0.6738 0.8518	0.8512
	Skiing	0.2038 0.3473 0.1441 0.1981 0.0927 0.2482 0.1942 0.0519	0.1812 0.1791 0.0841 0.0602 0.5963	0.5953
	Surf	0.0489 0.0647 0.0462 0.0579 0.0523 0.0467 0.0453 0.0162	0.0325 0.0317 0.0884	0	0.5851	0.6139
	Surfers	0.0542 0.1959 0.1189 0.0962 0.0742 0.1393 0.1184 0.1950	0.1083 0.1044 0.2612 0.4776 0.6719	0.6611
	Trafic	0.2188 0.2732 0.1445 0.2032 0.0368 0.1165 0.1042 0.1044	0.0949 0.0882	0	0	0.5722	0.5722
	Overall	0.1914 0.3144 0.2555 0.2271 0.1485 0.3203 0.2439 0.1803	0.2025 0.1858 0.3288 0.3483 0.6751	0.6727

3

  Scribbles 51.85 54.49 54.23 50.04 54.31 54.51 46.03 49.72 50.72 39.86 44.65 45.51 32.42 37.99 39.51 Clicks 1 32 41.35 42.13 42.05 41.43 42.61 42.30 40.46 43.06 42.97 39.36 43.76 44.28 32.56 37.91 39.87 Clicks 1 16 45.59 46.78 46.70 46.06 47.98 47.72 44.81 49.17 49.19 39.84 45.84 46.98 33.60 38.77 41.48 Clicks 1 8 50.04 51.71 51.48 51.08 54.21 54.15 47.94 52.98 53.18 41.33 46.48 47.98 33.16 38.76 41.24 Clicks 1 4 53.44 55.76 55.52 53.63 57.60 57.51 49.74 54.51 55.26 41.42 46.92 48.82 34.19 37.49 40.07

Table 5 .

 5 2: Accuracy of HyperGCN-WSS and previous methods in the validation set of PASCAL VOC. S: Scribbles. C: Clicks.

	Method	Weak Signal	CRF mIoU (%)
	ScribbleSup [145]	S	✓	63.1
	RAWKS [157]	S	✓	61.4
	NormalizedCutLoss [155]	S	-	60.5
	GraphNet [146]	S	-	63.3
	HyperGCN-WSS (ours)	S	-	65.3
	HyperGCN-WSS (ours)	C	-	65.4

Table 5

 5

.1 (in bold) correspond to low values of superpixels 𝜉. The reason is that the dimensions of the features maps of the VGG16 in the 23 layer are around 8 times smaller than the original image. Therefore, having big values of 𝜉 means smaller superpixels, which are hard to adequately represent with deep layers of CNNs. Finally, Table 5.2 shows the comparison of HyperGCN-WSS with previous methods. Our algorithm shows competitive performance against previous methods.

Table 6 .

 6

	and 2) the same hyperparameters such as learning rate, weigh decays, dropout, and
	others. S-SobGNN contains only one filtering operation	D-1 2 𝜌 Ā𝜌	D-1 2 𝜌	per layer in

2 presents the comparison of S-SobGNN against state-of-the-art methods when all GNNs have: 1) approximately the same amount of learnable parameters,

Table 6 . 2

 62

, and therefore no linear combination layer is required (in this case, we test S-SobGNN with values of 𝜌 in the set {1, 2, . . . , 6}). S-SobGNN shows competitive results on all datasets, and in some cases, our architecture presents an improvement of +2% accuracy regarding the second best method.

Table 6 .

 6 3 shows the homophily metric for the datasets in the semi-supervised learning experiments. The learned graphs readily show stronger homophily in all cases. This explains the fact that all GNNs in Tables 6.1 and 6.2 in almost all cases performed better in learned graphs than in 𝑘-NN graphs.

Table 6 .

 6 1: Accuracy (in %) for several state-of-the-art methods and our proposed S-SobGNN architecture in several datasets for semi-supervised learning, inferring the graphs with 𝑘-NN and the protocol to learn the graph as in Section 6.3.3. #Param is the number of learnable parameters, Acc. means accuracy, s.d. denotes standard deviation, T-Epoch is the average time per epoch. The best and second-best performing methods on each category are shown in red and blue, respectively. 𝑁 = 3000 #Param Test Acc.±s.d. 𝑘-NN Test Acc.±s.d. Learned T-Epoch #Param Test Acc.±s.d. 𝑘-NN Test Acc.±s.d. Learned T-Epoch

	Model Isolet, Cheby [31] HAR, 𝑁 = 3000 36481 94.193 ± 1.370 94.835 ± 1.222 18.27ms 40121 88.856 ± 0.462	88.015 ± 0.899	12.52ms
	TAGConv [178]	36310	90.898 ± 0.882	92.583 ± 1.450	42.66ms	41194	86.216 ± 1.887	86.054 ± 1.912	24.67ms
	GCN [33]	9094	91.992 ± 0.589	93.679 ± 0.778	3.34ms	10330	86.490 ± 0.679	87.122 ± 0.711	3.32ms
	GAT [34]	36498	90.028 ± 1.088	91.407 ± 0.804	14.15ms	41422	85.495 ± 1.291	86.533 ± 1.122	11.34ms
	k-GNN [181]	72646	22.659 ± 17.88	42.138 ± 19.46	17.75ms	82394	43.199 ± 14.34	75.993 ± 9.593	10.76ms
	GCNII [184]	52742	22.992 ± 6.596	85.305 ± 8.714	13.06ms	57626	16.746 ± 4.172	80.802 ± 2.624	7.61ms
	GCN+DE [185]	9094	92.621 ± 0.651	94.325 ± 0.821	3.78ms	9094	87.087 ± 0.660	87.814 ± 0.724	3.56ms
	GraphSAINT [186]	554758	88.433 ± 1.769	90.138 ± 1.025	29.87ms	598810	83.668 ± 0.670	84.840 ± 1.040	17.13ms
	S-SobGNN	45470	95.104 ± 0.475	95.912 ± 0.682	8.64ms	41320	89.279 ± 0.513	89.157 ± 0.555	7.78ms

Table 6 . 2

 62

	Model	HAR, 𝑁 = 3000 #Param Test Acc.±s.d. 𝑘-NN Test Acc.±s.d. Learned #Param Test Acc.±s.d. 𝑘-NN Test Acc.±s.d. Learned Isolet, 𝑁 = 3000
	Cheby [31]	9545	89.238 ± 2.684	89.863 ± 4.649	10497	67.447 ± 4.341	68.450 ± 3.614
	GCN [33]	9094	91.610 ± 1.036	93.325 ± 1.065	10330	87.113 ± 1.154	86.930 ± 1.370
	GAT [34]	9138	83.563 ± 13.73	89.333 ± 2.664	10414	84.190 ± 1.100	84.851 ± 1.071
	k-GNN [181]	9086	28.075 ± 16.51	59.382 ± 18.08	10322	5.386 ± 3.783	29.988 ± 15.44
	SIGN [173]	8526	90.016 ± 3.680	90.275 ± 3.556	9686	70.749 ± 4.350	69.828 ± 10.93
	S-SobGNN	9094	93.976 ± 0.637	95.225 ± 0.931	10330	88.127 ± 0.800	88.570 ± 0.807

: Accuracy (in %) for several state-of-the-art methods and our proposed S-SobGNN architecture in several datasets for semi-supervised learning, inferring the graphs with 𝑘-NN and the protocol to learn the graph as in Section 6.3.3. All methods have approximately the same number of learnable parameters.

Table 6 .

 6 3: Homophily index for the datasets for semi-supervised learning with different values of 𝑘.

	Dataset	𝑘 = 10 𝑘-NN|Lrnd.	𝑘 = 20 𝑘-NN|Lrnd.	𝑘 = 30 𝑘-NN|Lrnd.	𝑘 = 40 𝑘-NN|Lrnd.
	Cancer-B 0.8625|0.8838 0.8447|0.8670 0.8334|0.8568 0.8230|0.8463
	Cancer-M 0.6267|0.6534 0.5954|0.6242 0.5765|0.6054 0.5629|0.5921
	20News 0.5557|0.6103 0.4968|0.5348 0.4615|0.4919 0.4354|0.4628
	HAR 0.8849|0.9087 0.8478|0.8851 0.8211|0.8660 0.8007|0.8500
	Isolet 0.7346|0.7619 0.6851|0.7255 0.6454|0.6943 0.6105|0.6664

  45|89.27 88.83|90.71 89.75|91.86 90.00|92.18 90.95|92.92 57.41|59.46 59.66|62.49 63.20|63.99 63.36|64.63 64.48|65.22 73.17|73.95 75.69|76.17 75.89|76.64 75.38|77.33 75.69|76.71 𝜖 = 1 87.83|89.56 89.35|91.44 90.72|92.35 91.26|92.39 92.00|92.95 56.77|57.97 61.35|62.26 62.27|63.63 63.98|64.78 64.69|64.59 72.73|75.95 75.52|75.24 76.94|76.30 76.49|75.76 76.67|75.63 𝜖 = 1.5 87.82|89.35 90.21|91.42 91.30|92.07 91.89|92.51 92.43|92.94 55.99|57.26 61.00|62.90 61.98|62.99 63.54|64.48 64.02|65.15 73.16|73.23 75.85|75.33 76.57|75.77 76.24|74.85 75.97|74.51 𝜖 = 2 88.34|89.83 90.21|90.91 91.03|91.85 92.12|92.72 92.36|92.73 56.66|56.47 60.56|61.36 62.89|63.62 63.25|63.59 64.95|65.49 73.46|72.84 76.19|75.31 76.53|75.12 76.29|74.55 76.35|75.73

	HAR, 𝑁 = 3000	Isolet, 𝑁 = 3000	Dataset	𝑘 = 10 𝑘-NN|Lrnd.

Table 6 .

 6 92.82|92.84 92.20|92.91 92.07|92.62 𝜖 = 0.5 91.80|94.68 92.53|95.35 92.97|95.82 93.10|95.91 93.02|95.59 84.92|87.66 87.01|88.41 87.03|88.49 86.96|88.06 87.02|88.23 Cancer-M 64.38|64.75 64.08|64.70 64.30|65.11 64.69|65.28 𝜖 = 1 92.64|95.49 93.15|95.57 93.86|95.62 94.17|95.47 94.74|95.40 87.35|89.16 88.42|88.77 88.86|88.16 88.74|88.50 88.47|87.34 20News 76.11|76.16 75.92|76.50 76.40|76.63 76.31|76.36 𝜖 = 1.5 93.13|95.33 94.08|95.43 94.10|95.07 94.96|95.08 95.10|95.03 88.06|88.19 88.87|87.79 89.28|88.26 88.68|87.97 88.85|87.98 HAR 95.52|96.08 95.65|96.30 95.53|95.99 94.92|96.00 𝜖 = 2 93.61|95.29 94.30|95.66 94.41|95.48 95.05|95.18 94.89|95.26 87.83|88.51 89.13|88.67 88.98|87.49 88.41|87.09 88.41|87.25 Isolet 88.13|88.34 88.93|88.53 88.87|88.67 89.01|89.33

5: Sparsity percentage for the sparse and non-sparse Sobolev filtering matrices.

  96.63 96.63 96.63 96.63 99.17 99.17 99.17 99.17 99.17 98.58 98.58 98.58 98.58 98.58 98.03 98.03 98.03 98.03 98.03 99.01 99.01 99.01 99.01 99.01 Non-Sparse 96.63 80.33 53.82 29.32 12.46 99.17 94.05 82.48 67.67 52.55 98.58 65.65 0.615 0 0 98.03 82.29 59.74 46.05 35.07 99.01 92.73 75.92 46.19 20.41

Table 7 . 1 :

 71 Results of the first experiment regarding the comparison of our SJLR algorithm with several state-of-the-art methods to alleviate over-smoothing and oversquashing. Certain results were copied directly from[START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] because of limited computational resources.The best and second-best performing methods on each dataset are shown in red and blue, respectively.

	Method	Cornell	Texas	Wisconsin	Chameleon	Squirrel	Actor	Cora	Citeseer	Pubmed
	Baseline 66.18 ± 1.64	59.44 ± 1.22	52.11 ± 1.08 40.82 ± 0.47 41.77 ± 0.33	29.16 ± 0.30 82.02 ± 0.28 69.23 ± 0.30	78.44 ± 0.37
	RDC [38] 66.77 ± 1.68	58.52 ± 1.11	51.85 ± 1.03 41.50 ± 0.57 41.88 ± 0.35 29.26 ± 0.33	81.79 ± 0.27	69.10 ± 0.34	78.49 ± 0.36
	GDC [202] 67.01 ± 1.43	56.97 ± 1.07	59.97 ± 1.03	38.27 ± 0.50	36.64 ± 0.32 32.68 ± 0.17 82.52 ± 0.26 69.28 ± 0.28 78.84 ± 0.38
	DE [185] 80.41 ± 1.22 80.12 ± 1.07 84.17 ± 1.08 40.48 ± 0.51	41.21 ± 0.31 33.20 ± 0.23 81.59 ± 0.25	69.04 ± 0.39	78.27 ± 0.38
	PN [193] 65.26 ± 1.57	61.40 ± 1.00	52.77 ± 0.89	40.56 ± 0.43	40.92 ± 0.32	28.74 ± 0.22	79.62 ± 0.31	67.27 ± 0.40	77.29 ± 0.34
	DGN [194] 67.49 ± 1.64	60.19 ± 0.93	52.33 ± 0.97	39.80 ± 0.57 41.81 ± 0.32 28.26 ± 0.31	81.55 ± 0.26	69.13 ± 0.36	78.39 ± 0.36
	FA [47] 53.57 ± 0.00	59.26 ± 0.00	42.98 ± 0.49	27.28 ± 0.40	31.51 ± 0.00	23.84 ± 0.43 *	29.85 ± 0.00	23.23 ± 0.00	79.48 ± 0.12 *
	SDRF [190] 58.26 ± 1.43	54.88 ± 1.45	55.62 ± 0.92	40.11 ± 0.47	41.78 ± 0.30	28.83 ± 0.36	81.05 ± 0.28	69.21 ± 0.33	78.40 ± 0.33
	SJLR (ours) 71.90 ± 1.97 66.18 ± 1.44 66.12 ± 1.27 39.11 ± 0.51	40.64 ± 0.27	31.29 ± 0.24	81.27 ± 0.24 70.51 ± 0.32 78.03 ± 0.35

*

  x 2 , . . . , x 𝑀 ] be a time-varying graph signal, where x 𝑖 ∈ R 𝑁 is a graph signal in 𝐺 at time 𝑖. One can extend the concept of graph Laplacian quadratic form to time-varying graph signals by summing the Laplacian quadratic form of each graph signal x 𝑖 . Then, we have:

  .3) Qiu et al.[START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] noted that 𝑆 2 (XD ℎ ) exhibits better smoothness properties compared to 𝑆 2 (X) in real-world datasets for time-varying graph signals, i.e., the difference signalx 𝑖 -x 𝑖-1 exhibits smoothness in the graph even if the signal x 𝑖 is not smooth across the graph. They defined the structure of time-varying graph signals as follows[START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]:

	Definition 8.3.1. The 𝛼-structured set ℬ 𝛼 (𝐺) composed of smoothly evolving graph
	signals is defined as:

  from static graph signals to a smoothness function for time-varying graph signals, and then we formulate a new reconstruction algorithm. The Sobolev norm was defined by Pesenson[START_REF] Pesenson | Variational splines and Paley-Wiener spaces on combinatorial graphs[END_REF], and introduced in Chapter 3, Eqn. (3.2). When L is symmetric (as is the case in this chapter), we have that:

  We use the Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and the Mean Absolute Percentage Error (MAPE) to compare GraphTRSS with the meth-ods of the literature. RMSE = where x is the recovered signal, x * is the ground truth signal, and 𝑁 𝑥 is the length of the signal.

				√	∑ 𝑁𝑥 𝑖=1 (x 𝑖 -x * 𝑖 ) 2 𝑁𝑥	, MAE =	∑ 𝑁𝑥 𝑖=1 |x 𝑖 -x * 𝑖 | 𝑁𝑥	, and MAPE =
	1 𝑁𝑥	∑ 𝑁𝑥 𝑖=1 |	x * 𝑖 -x 𝑖 𝑖 x *	|,

Table 8 .

 8 1: Summary of the average error metrics in the real datasets for several sampling schemes. The best and second-best performing methods on each category are shown in red and blue, respectively.

						Random Sampling
	Method	COVID-19 Global	COVID-19 USA	Sea Surface Temperature PM 2.5 Concentration
		RMSE	MAE	MAPE RMSE MAE MAPE RMSE MAE MAPE RMSE MAE MAPE
	NNI (Sibson [22])	1555.14	189.15	4.23	54.92	10.90	2.17	0.77	0.56	0.07	4.95	2.96	0.59
	GR (Narang et al. [23])	4744.25 1119.07	90.95	65.49	13.94	3.47	2.43	1.76	0.40	5.37	3.32	0.67
	Tikhonov (Perraudin et al. [24])	1253.71	183.60	6.07	37.00	6.03	1.03	0.95	0.70	0.12	4.32	2.66	0.55
	TGSR (Qiu et al. [26])	1136.39 213.96	11.30	33.51	6.14	1.19	0.36	0.26	0.03	3.90	2.28	0.39
	RSD (Puy et al. [27])	2045.37	506.30	36.17	58.59	12.05	2.50	5.56	4.62	0.97	5.43	3.46	0.72
	GraphTRSS (ours)	1134.15 152.76	2.41	33.47 5.96	1.10	0.36	0.26	0.03	3.83	2.21	0.38
						Entire Snapshots Sampling
	Method	COVID-19 Global	COVID-19 USA	Sea Surface Temperature PM 2.5 Concentration
		RMSE	MAE	MAPE RMSE MAE MAPE RMSE MAE MAPE RMSE MAE MAPE
	NNI (Sibson [22])	1342.08	170.64	3.40	56.69	11.71	2.60	0.98	0.68	0.08	10.87	4.44	0.85
	Tikhonov (Perraudin et al. [24])	1255.05 182.97	5.94	36.96	6.12	1.04	1.17	0.86	0.13	4.54	2.81	0.56
	TGSR (Qiu et al. [26])	1459.45	788.10	100.06 36.38 13.75	4.74	19.19 19.15	2.13	10.94	9.31	1.49
	GraphTRSS (ours)	1114.16 143.69	1.49	32.96 5.45	0.88	0.94	0.67	0.07	4.42	2.70	0.48
							Forecasting
	Method	COVID-19 Global	COVID-19 USA	Sea Surface Temperature PM 2.5 Concentration
		RMSE	MAE	MAPE RMSE MAE MAPE RMSE MAE MAPE RMSE MAE MAPE
	NNI (Sibson [22])	16722.30 2371.30	6.43	904.23 283.23 19.54	3.04	2.23	0.23	30.71 17.15	2.76
	Tikhonov (Perraudin et al. [24])	3612.06 1190.20 30.48	99.88	31.2	2.03	1.94	1.53	0.30	4.80	3.23	0.67
	TGSR (Qiu et al. [26])	3313.50 2472.08 238.65	91.5	54.36	6.88	18.72 18.69	2.39	10.09	9.11	1.27
	GraphTRSS (ours)	2416.30 583.36	4.80	75.33 20.65	0.86	1.20	0.98	0.13	4.36	2.83	0.54

of all results over the set ℳ. GraphTRSS shows the best performance in almost all cases against the other methods.

8.6.3 COVID-19 Datasets

  19 USA dataset have lower RMSE than the global dataset, i.e., we can get better estimations if we have more fine-grained data as in the case of the COVID-19 USA dataset. From the experiment with entire temporal snapshots, we can notice that TGSR method[START_REF] Qiu | Time-varying graph signal reconstruction[END_REF] performs poorly compared to the other methods for what we observe in random sampling. This behavior is expected because entire snapshots sampling does not satisfy the second condition of Theorem 8.3.1 for TGSR by Qiu et al.[START_REF] Qiu | Time-varying graph signal reconstruction[END_REF].All tested methods, including ours, are designed to reconstruct time-varying graph signals, and therefore these methods do not have prior assumptions about the behavior in the time domain of the underlying processes. Correspondingly, in forecasting, we COVID-19 USA New Cases

Table 8 .

 8 2:The average number of iterations to satisfy the stopping condition for GraphTRSS and TGSR[START_REF] Qiu | Time-varying graph signal reconstruction[END_REF].

	Method	COVID-19 Global COVID-19 USA PM 2.5 Conc. Sea Surface Temp.
	TGSR [26]	1735.9	48659	4690.1	418.8
	GraphTRSS	510.0	427	4266.6	416.0

  𝜆 2 ≥ 𝜆 𝑁 -1, 2 -𝜆 𝑁 otherwise. (G.4) From Section 7.4.1 we know that only 𝜆 2 is important in (G.4) since we can apply lazy walk. As a result, we can compute the value of 𝑠 such that ‖f T P 𝑠 -𝜋‖ 2 ≤ 𝜖 as 𝜅(∇ 2 z 𝑓 𝐿 (z)). 4. when 𝛽 → ∞ and 𝜆 𝑁 + 𝜖 > 1, 𝜆 𝑚𝑎𝑥 (∇ 2 z 𝑓 𝑆 (z)) → ∞ according to (I.11) and then 𝜅(∇ 2 z 𝑓 𝑆 (z)) → ∞.

														1 2 ‖ 2
	≤ (1 -𝜆 ′ ) 𝑠 max 𝑖 min 𝑗	√ 𝑑 𝑖 √ 𝑑 𝑗
	≤ 𝑒 -𝑠𝜆 ′ max 𝑖 min 𝑗 if 1 -follows: ⎧ √ 𝑑 𝑖 √ 𝑑 𝑗 , where 𝜆 ′ = ⎪ ⎨ ⎪ ⎩ 𝜆 2 𝑠 ≥ 1 𝜆 2 log ( max 𝑖 √ 𝑑 𝑖 𝜖 min 𝑗 √ 𝑑 𝑗 ) → 𝜆 2 ≥ 1 𝑠 log ( max 𝑖 √ 𝑑 𝑖 𝜖 min 𝑗 √ 𝑑 𝑗	)	.	(G.5)
	Finally, using the Cheeger inequality in (7.2) we have that:
	2ℎ 𝐺 ≥ 𝜆 2 ≥	1 𝑠	log	(	max 𝑖 𝜖 min 𝑗	√ 𝑑 𝑖 √ 𝑑 𝑗	)	→ 2ℎ 𝐺 ≥	1 𝑠	log	(	max 𝑖 𝜖 min 𝑗	√ √ 𝑑 𝑖 𝑑 𝑗	)	.	(G.6)

The segmented regions are: each superpixel for SLIC, each block of pixels for block-based method, each mask for instance segmentation, distinct regions of one category for semantic segmentation, and distinct foreground regions for SuBSENSE.

https://github.com/jhonygiraldo/GraphMOS

The self-connections given by à = A + I were introduced by Kipf and Welling[START_REF] Kipf | Semi-supervised classification with graph convolutional networks[END_REF] to avoid numerical instabilities and exploding/vanishing gradients.

GraphBGS-TV is a particular instance of GraphMOS where the semi-supervised learning problem is solved with TV minimization as in (3.9).

 3 GraphBGS is a particular instance of GraphMOS where the segmentation algorithm is a Cascade Mask R-CNN.

The condition number 𝜅(L) associated with the square matrix L is a measure of how well or ill conditioned is the inversion of L.

The Cheeger inequality bounds the first non-zero eigenvalue of the Laplacian with the Cheeger constant, which is at the same time related to the bottlenecks of the graph.

https://github.com/jhonygiraldo/Stochastic_Jost_Liu_Curvature_Rewiring

Further details about over-squashing can be found in Section 2.2 in[START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF].

http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-11/www/wwkb/

One can think of the reconstruction of time-varying graph signals as a matrix completion problem where each row (or column) is associated with a node, and each column (or row) is associated with time.

https://www.epa.gov/outdoor-air-quality-data

https://psl.noaa.gov/

https://github.com/jhonygiraldo/GraphTRSS

* Certain hyperparameters were copied directly from[START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF].

Acknowledgments

The research in Chapter 7 was supported by DATAIA convergence institute as part of the "Programme d'Investissement d'Avenir", (ANR-17-CONV-0003) operated by CentraleSupélec.

𝜆 2 . We will use this result later to show the underlying relationship between oversmoothing and over-squashing. Similarly, we can have a simplified explanation of why sparsification methods in GNNs, like DropEdge [START_REF] Rong | DropEdge: Towards deep graph convolutional networks on node classification[END_REF], can alleviate over-smoothing. Lemma 7.4.2 (Chung [START_REF] Chung | Spectral graph theory[END_REF]). Let 𝐺 be a graph with diameter 𝐷𝑖 ≥ 4, then:

Proof: See [START_REF] Nilli | On the second eigenvalue of a graph[END_REF].

Lemma 7.4.2 shows an upper bound of 𝜆 2 so that reducing the maximum degree of 𝐺 promotes small values of 𝜆 2 , i.e., having a sparser graph, like the DropEdge method does, promotes low values of 𝜆 2 and thus high values of 𝑠 in (7.4).

Over-smoothing and Over-squashing

We can establish a link between the Cheeger constant ℎ 𝐺 and the parameter 𝑠 in (7.4) as follows:

Lemma 7.4.3. Let ℎ 𝐺 be the Cheeger constant of a given graph 𝐺, and let 𝑠 be the number of required steps such that the ℓ 2 distance between f T P s and 𝜋 is at most 𝜖.

Therefore, we have that:

Proof: see Appendix G.

From Lemma 7.4.3, we notice that if 𝑠 → ∞ then ℎ 𝐺 → 0, i.e., if we want to avoid converging to the stationary distribution, we need to promote a bottleneckkind structure in the graph. Similarly, if ℎ 𝐺 → ∞ then 𝑠 → 0, i.e., if we want to promote less "bottleneckness" in the graph, we need to accelerate the convergence to the stationary distribution.

We can make the connection between over-smoothing and over-squashing using Lemma 7.4.3, the Simple Graph Convolution (SGC) model [START_REF] Wu | Simplifying graph convolutional networks[END_REF], and the theoretical We keep the addition and removal of edges in two different stages attempting to find a good solution in the over-smoothing vs. over-squashing trade-off.

Topping et al. [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] proposed a method to alleviate over-squashing using concepts of Ricci flow curvature. We can understand curvature-based methods using the following analysis: Theorem 7.5.1 (Theorem 4.2 in [START_REF] Lin | Ricci curvature of graphs[END_REF]). Let 𝐺 be a finite graph, let 𝜆 2 be its first nonzero Laplacian eigenvalue, and let 𝜅(𝑖, 𝑗) be the Ricci curvature as defined in [START_REF] Lin | Ricci curvature of graphs[END_REF].

If for any edge (𝑖, 𝑗), 𝜅(𝑖, 𝑗) ≥ 𝜅 > 0, then 𝜆 2 ≥ 𝜅.

Proof: See [START_REF] Lin | Ricci curvature of graphs[END_REF]. From Theorem 7.5.1 and Corollary 7.5.2, we can conclude that if we have positive Ricci curvature everywhere, then 2ℎ 𝐺 ≥ 𝜅. Therefore, increasing curvature will make the graph less "bottleneckness". Toppings et al. [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] defined a new metric, called Balanced Forman Curvature (BFC), to solve the problem of over-squashing. BFC requires counting triangles, 4cycles, and the maximal number of 4-cycles based at edge (𝑖, 𝑗) traversing a common node for each edge we are processing. This process makes BFC very computationally complex and unsuitable for practical applications. In this chapter, we use a bound of the Ollivier's Ricci curvature [START_REF] Ollivier | Ricci curvature of markov chains on metric spaces[END_REF] presented in [START_REF] Jost | Ollivier's Ricci curvature, local clustering and curvaturedimension inequalities on graphs[END_REF], namely JLC. Definition 7.5.1 (Jost and Liu Curvature (JLC) [START_REF] Jost | Ollivier's Ricci curvature, local clustering and curvaturedimension inequalities on graphs[END_REF]). For any edge (𝑖, 𝑗) in a finite graph:

where #(𝑖, 𝑗) is the number of triangles which include (𝑖, 𝑗) as nodes, 𝑐 + ≜ max(𝑐, 0), 𝑐 ∨ 𝑡 ≜ max(𝑐, 𝑡), and 𝑐 ∧ 𝑡 ≜ min(𝑐, 𝑡).

Theorem 7.5.3 (Jost and Liu [START_REF] Jost | Ollivier's Ricci curvature, local clustering and curvaturedimension inequalities on graphs[END_REF]). On a locally finite graph we have that 𝜅(𝑖, 𝑗) ≥

JLC(𝑖, 𝑗).

Proof: See [START_REF] Jost | Ollivier's Ricci curvature, local clustering and curvaturedimension inequalities on graphs[END_REF]. From Corollary 7.5.4 we can conclude that if we have positive JLC everywhere in the graph, 𝜅(𝑖, 𝑗) will also be positive everywhere. As a result, having positive JLC in the graph ensures that the receptive field of each node in a deep GNN will be polynomial in the hop-distance rather than exponential (see Corollary 3 in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF]). As a result, JLC is less computationally complex than the BFC in [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF] while keeping the same theoretical properties about the polynomial receptive field growth.

Curvature Rewiring Algorithm

Our proposed algorithm uses JLC and features information to perform rewiring, i.e., we remove and add edges based on their JLC metric and feature information of the Our algorithm is compared with Natural Neighbor Interpolation (NNI) [START_REF] Sibson | A brief description of natural neighbor interpolation[END_REF], Graph Regularization (GR) [START_REF] Narang | Localized iterative methods for interpolation in graph structured data[END_REF], Tikhonov regularization [START_REF] Perraudin | Towards stationary time-vertex signal processing[END_REF][START_REF] Loukas | Stationary time-vertex signal processing[END_REF], Time-varying Graph Signal Reconstruction (TGSR) [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF], and Random Sampling and Decoder (RSD) [START_REF] Puy | Random sampling of bandlimited signals on graphs[END_REF]. other static graph signal reconstruction schemes as GR. Finally, Fig. 8-7 structure of each graph. Therefore, using the normalized Laplacian

Synthetic Graph and Signals

The interested readers are referred to Section VI in the supplementary material for a small experiment with normalized Laplacian matrices.

The parameters 𝜖 and 𝜐 are optimized for each dataset to get good performance.

However, we can notice in Fig. 8-12 that an 𝜖 value close to 10 -1 is a good starting point since it improves the condition number, and it does not modify heavily the Laplacian matrix. Similar analysis about the parameter 𝜐 can be found in [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]. In practice, the parameters of GraphTRSS can be optimized in a small part of the dataset.

Conclusions

In this chapter, a new algorithm called GraphTRSS is introduced for the reconstruc- Appendix A

Instance Segmentation

The basic building blocks of the instance segmentation methods tested in Chapters 3 and 4 are: Mask R-CNN [20], Cascade Mask R-CNN [START_REF] Cai | Cascade R-CNN: High quality object detection and instance segmentation[END_REF], ResNet [START_REF] He | Deep residual learning for image recognition[END_REF],

ResNeSt [START_REF] Zhang | ResNeSt: Split-attention networks[END_REF], and Feature Pyramid Network (FPN) [START_REF] Lin | Feature pyramid networks for object detection[END_REF]. Mask R-CNN builds upon Faster R-CNN [START_REF] Ren | Faster R-CNN: Towards real-time object detection with region proposal networks[END_REF] by adding a branch for predicting an object mask in parallel with the already existing Faster R-CNN network for bounding box recognition.

Appendix B Vector of Features

We introduced in Section 3.3.2 the matrices representing the intensity and background images I 𝑡 𝑣 , I 𝑡-1 𝑣 , and B 𝑣 as well as the set of indices corresponding to the 𝑣-th segmented region 𝒫 𝑣 and the optical flow vectors of the current frame with support in the set of indices 𝒫 𝑣 for the horizontal and vertical direction v 𝑡 𝑥 (𝒫 𝑣 ) and v 𝑡 𝑦 (𝒫 𝑣 ). The vector x 𝑣 is obtained with the following steps:

1. The Local Binary Patterns (LBP) [START_REF] Ojala | Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[END_REF] are computed on the matrices: I 𝑡 𝑣 , I 𝑡-1 𝑣 , B 𝑣 , and |I 𝑡 𝑣 -B 𝑣 |. The LBP for each matrix is computed using 8 neighbors and a radius of 1 for each pixel in the input image, a linear interpolation to compute pixel neighbors, and finally a ℓ 2 normalization for each LBP cell histogram. For each matrix, a vector R 1×59 is obtained for LBP representation.

Let 𝑓 𝐿𝐵𝑃 : A → R 1×59 be the function that represents the process of LBP feature extraction, the final LBP features lbp 𝑣 ∈ R 1×236 for a node 𝑣 is given as

The vector of orientations 𝜃 ∈ R |𝒫𝑣| and the vector of magnitudes m ∈ R |𝒫𝑣| of the optical flow are computed as follows:

where 𝜇 ∈ R |𝒮| is the vector of Lagrange multipliers, and ℒ(z 𝑞 , 𝜇) = z T 𝑞 (L + 𝜖I) 𝛼 z 𝑞 + 𝜇 T (Mz 𝑞 -y 𝑞 (𝒮)) is the Lagrange function. For an unconstrained problem, the well known necessary and sufficient condition for z 𝑞 to be optimal [START_REF] Boyd | Convex optimization[END_REF]:

Now, using Eqn. (C.3):

and also:

Appendix D

Proof of Theorem 6.4.1

Proof. 1) For the triangle inequality we have that: 2) Let 𝑠 be a scalar, then for the absolute homogeneity property we have:

3) For the positive definiteness property, we need the Schur product theorem:

Theorem D.0.1. (Schur product theorem) Let S, T ∈ R 𝑛×𝑛 be two positive definite matrices of dimension 𝑛 × 𝑛, then S ∘ T is also positive definite.

Proof: see [START_REF] Horn | Matrix analysis[END_REF].

For 𝜖 = 0 and 𝜌 = 1 the sparse Sobolev norm becomes the graph Laplacian quadratic form [START_REF] Shuman | The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains[END_REF], so that ‖x‖ 2 (1),0 = x T Lx = 0 ⇐⇒ x = 𝜏 1 (with 𝜏 some 151 constant value), and then ‖x‖ (𝜌),𝜖 becomes a semi-norm because it satisfies only the first two conditions of vector norms, i.e., triangle inequality and absolute homogeneity.

However, for 𝜖 > 0, the term L+𝜖I ≻ 0 according to Eq. (3.6), and then (L+𝜖I) (𝜌) ≻ 0 using the Schur product theorem. As a result:

As a consequence, the sparse Sobolev norm satisfies the properties of vector norms for all 𝜖 > 0 according to Eq. (D.2), (D.3), and (D.4). Similarly, the sparse Sobolev definition satisfies the properties of semi-norms for 𝜖 = 0 according to Eq. (D.2) and (D.3).

Appendix E

Proof of Lemma 6.4.2

Proof. We can compute the spectral decomposition of the Kronecker product between matrices using concepts of product graphs [START_REF] Varma | Sampling theory for graph signals on product graphs[END_REF]. The spectrum of the Kronecker product of several graphs is given by the Kronecker products of its spectral components.

For example, the spectral decomposition of L ⊗ L is given by [START_REF] Varma | Sampling theory for graph signals on product graphs[END_REF]:

Similarly, there is a relationship between the Kronecker and the Hadamard products as follows:

Theorem E.0.1. (Visick [START_REF] Visick | A quantitative version of the observation that the Hadamard product is a principal submatrix of the Kronecker product[END_REF]) For S, T ∈ R 𝑛×𝑚 , we have:

where P 𝑛 ∈ {0, 1} 𝑛 2 ×𝑛 and P 𝑚 ∈ {0, 1} 𝑚 2 ×𝑚 are partial permutation matrices. If S, T ∈ R 𝑛×𝑛 are square matrices, we have that S ∘ T = P T 𝑛 (S ⊗ T)P 𝑛 . Proof: see [START_REF] Visick | A quantitative version of the observation that the Hadamard product is a principal submatrix of the Kronecker product[END_REF].

We can get a general form of the spectrum of the Hadamard product (for the Hadamard power of order 2) using Eq. (E.1) and Theorem E.0.1 as follows:

Appendix F

Codebase

We have created a codebase written in PyTorch and PyG [START_REF] Fey | Fast graph representation learning with PyTorch Geometric[END_REF] for benchmarking methods to alleviate over-smoothing and over-squashing. The code contains: 1) a GNN model, 2) functions to compute the JLC and BFC metrics, 3) a data loader (a modified version of the code released in [START_REF] Klicpera | Diffusion improves graph learning[END_REF]) with 12 datasets, 4) a random hyperparameter search tool, and 5) a main file with the training procedure. The GNN is fully parameterized and we can do experiments combining several kinds of methods.

For example, we can easily run a random hyperparameter search using PairNorm (PN)

and Residual Dense Connections (RDC) using the following command: The full set of arguments that can be modified in a random hyperparameter search are:

• no_cuda: Flag to run the experiment in the CPU.

• gpu_number: Integer specifying the identifier of the GPU where the experiment is being executed.

• ResidualDenseConnection: Flag to activate the residual dense connections.

Appendix G

Proof of Lemma 7.4.3

Proof. Let P = D -1 A be the random walk transition matrix, and let 𝑓 : 𝒱 → R be any initial distribution with vector representation f ∈ R 𝑁 ×1 . If we want to measure the distance between f T P 𝑠 and the stationary distribution in the ℓ 2 norm we need to compute:

Let 𝜑 𝑖 be the orthonormal eigenfunction associated with 𝜆 𝑖 . We know that 𝜑 1 =

.Therefore, we have:

, where

As a consequence we have that:

Appendix H Hyperparameters Search

We define a range of values or a set of number for the hyperparameter search that we called the search space: 1) learning rate 𝑙𝑟 ∈ [0.005, 0.02], 2) weight decay 𝑤𝑑 ∈ [START_REF] Zhou | Towards deeper graph neural networks with differentiable group normalization[END_REF] * Certain hyperparameters were copied directly from [START_REF] Topping | Understanding over-squashing and bottlenecks on graphs via curvature[END_REF]. Proof. The problem associated with the Laplacian matrix in (8.5) can be rewritten as follows [START_REF] Qiu | Time-varying graph signal reconstruction[END_REF]:

where z = vec ( X). Moreover, the gradient of 𝑓 𝐿 (z) in (I.1) is such that:

and the Hessian matrix of 𝑓 𝐿 (z) is given as follows:

Similarly, the Hessian matrix associated with the Sobolev formulation is given by:

ℎ is a positive semi-definite matrix, so it has a matrix of eigenvalues Λ 𝐷 = diag(𝜆 (𝐷)1 , 𝜆 (𝐷)2 , . . . , 𝜆 (𝐷)𝑁 ), with 0 ≤ 𝜆 (𝐷)1 ≤ 𝜆 (𝐷)2 ≤ • • • ≤ 𝜆 (𝐷)𝑁 , and the corre-sponding matrix of eigenvectors U 𝐷 . As a consequence, from (I.3), we have:

where we used the property of Kronecker products (A⊗B)(C⊗D) = AC⊗BD [START_REF] Horn | Matrix analysis[END_REF].

Q is positive semi-definite because it is a diagonal matrix with diagonal elements either 0 or 1. As a result, from (I.5), we know that

For the Sobolev problem, we have that:

and then:

where 𝜆 𝑚𝑖𝑛 ((D ℎ D T ℎ ) ⊗ (L + 𝜖I) 𝛽 ) = 0 and 𝜆 𝑚𝑎𝑥 ((D ℎ D T ℎ ) ⊗ (L + 𝜖I) 𝛽 ) = (𝜆 𝑁 + 𝜖) 𝛽 𝜆 (𝐷)𝑁 if (𝜆 𝑁 + 𝜖) 𝛽 , 𝜆 (𝐷)𝑁 ≥ 1. We have a summation of two Hermitian matrices in (I.5) and (I.7) (we can trivially prove that (I.5) and (I.7) are Hermitian matrices since (A ⊗ B) T = A T ⊗ B T ). Therefore, we cannot get an exact value for the eigenvalues of these Hessian functions. The problem of describing the possible eigenvalues of the sum of two Hermitian matrices in terms of their spectra has attracted the interest of mathematicians for decades. The most complete description was conjectured by Horn et al. [START_REF] Horn | Eigenvalues of sums of hermitian matrices[END_REF]. The interested readers are referred to [START_REF] Knutson | Honeycombs and sums of hermitian matrices[END_REF] for an interesting discussion in the topic. However, we can get some bounds using the developments of Weyl [START_REF] Horn | Matrix analysis[END_REF]: